Harness engineering beginner tutorial, from 0 to 1
TypeScript agentagenticagentic-aiaiai-agentai-agentsdshdsh-pluginharnessharness-engineeringharness-frameworkllm
Fix awkward translated terms in both Chinese and English lectures
sanbuphyb030de783e6f
diff --git a/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md b/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.mdindex 8f2ed0c..ebc17bb 100644--- a/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md+++ b/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md@@ -19,17 +19,13 @@ OpenAI's 2025 harness engineering article puts it even more bluntly. They said t## Where Agents Actually Get Stuck-The specific failure modes really come down to just a handful.+The specific failure modes really come down to just a handful:-The first type: the task was never clearly defined in the first place. "Add a search feature" — that sentence means almost nothing. Search what? Full-text or structured queries? Should results be paginated? Highlighted? You didn't spell it out, so the agent has to guess. A correct guess is luck; a wrong one means rework that costs several times more than being specific would have in the first place.--Architectural conventions are another hidden pitfall. Your whole team uses the new SQLAlchemy 2.0 syntax, but the agent writes 1.x code by default. All API endpoints must go through OAuth 2.0 authentication, but that rule only exists in your head and a Slack message from three months ago. The agent has no idea — it's not that it doesn't want to comply, it literally has never seen the rule.--The environment is a frequent source of trouble too. Incomplete dev setup, missing dependencies, wrong tool versions — the agent burns precious context window on `pip install` errors and Node version conflicts instead of doing the actual work you gave it.--Even more common is the complete absence of verification. No tests, no lint, or verification commands that were never communicated to the agent. The agent writes code, looks it over, decides it seems fine, and declares completion. Anthropic also observed an interesting phenomenon: when agents sense their context is running low, they rush to finish, skip verification steps, and choose a simple solution over the optimal one. They call this "context anxiety."--Tasks that span multiple sessions fare even worse. All discoveries from the previous session are lost. Every new session has to re-explore the project structure and re-understand the code organization. Agents without persistent state see failure rates spike sharply on tasks exceeding 30 minutes.+- **Vague requirements — the agent can only guess.** "Add a search feature" — that sentence means almost nothing. Search what? Full-text or structured queries? Should results be paginated? Highlighted? You didn't spell it out, so the agent has to guess. A correct guess is luck; a wrong one means rework that costs several times more than being specific would have in the first place.+- **Implicit conventions not written down — the agent has no way to comply.** Your whole team uses the new SQLAlchemy 2.0 syntax, but the agent writes 1.x code by default. All API endpoints must go through OAuth 2.0 authentication, but that rule only exists in your head and a Slack message from three months ago. The agent has no idea — it's not that it doesn't want to comply, it literally has never seen the rule.+- **Incomplete environment setup — the agent spends energy fixing the environment.** Incomplete dev setup, missing dependencies, wrong tool versions — the agent burns precious context window on `pip install` errors and Node version conflicts instead of doing the actual work you gave it.+- **No verification methods — the agent calls it done when it feels done.** No tests, no lint, or verification commands that were never communicated to the agent. The agent writes code, looks it over, decides it seems fine, and declares completion. Anthropic also observed an interesting phenomenon: when agents sense their context is running low, they rush to finish, skip verification steps, and choose a simple solution over the optimal one. They call this "context anxiety."+- **Cross-session state loss — every new session starts from scratch.** All discoveries from the previous session are lost. Every new session has to re-explore the project structure and re-understand the code organization. Agents without persistent state see failure rates spike sharply on tasks exceeding 30 minutes.## Key Terminologydiff --git a/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md b/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.mdindex 60feb9a..fda307f 100644--- a/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md+++ b/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md@@ -33,7 +33,7 @@ Look at a few tools you already know:- **The repo is the single source of truth**: Anything the agent cannot see, for all practical purposes, does not exist. OpenAI treats the repo as the "system of record" — all necessary context must live there, delivered through structured files and clear directory organization.- **Give a map, not a manual**: OpenAI's experience is that `AGENTS.md` should be a directory page, not an encyclopedia. Around 100 lines is enough. If it does not fit, split it into a `docs/` directory and let the agent read on demand.- **Constrain, don't micromanage**: A good harness uses executable rules to constrain the agent, rather than enumerating instructions one by one. OpenAI says "enforce invariants, don't micromanage implementation"; Anthropic found that agents confidently praise their own work, and the solution is to separate "the person who does the work" from "the person who checks the work."-- **Remove components one at a time**: To quantify each harness component's marginal contribution, remove them one at a time and see which removal causes the biggest performance drop. This tells you which components are most valuable right now, and it also reveals which ones are not yet contributing meaningfully. Anthropic used this method and discovered that as models get stronger, some components stop being critical — but new critical components always emerge.+- **Remove one at a time and observe**: To quantify each harness component's marginal contribution, remove them one at a time and see which removal causes the biggest performance drop. This tells you which components are most valuable right now, and it also reveals which ones are not yet contributing meaningfully. Anthropic used this method and discovered that as models get stronger, some components stop being critical — but new critical components always emerge.## The Five-Subsystem Harness Model@@ -68,7 +68,7 @@ Verification commands:Missing any one of the five subsystems means an incomplete harness, and the agent will always feel awkward to use.-**Quantifying harness component value**: Use an "ablation experiment under the same model." Keep the model fixed, remove the five subsystems one at a time, and see which subsystem's removal causes the biggest performance drop. The component with the largest drop has the highest marginal contribution for the current task and is worth prioritizing. Whether to strengthen it depends on failure attribution, not just the size of the drop. Components with near-zero impact should not be dismissed outright: they may be redundant, poorly designed, or simply not exercised by the current task. This experiment answers "which component is most valuable right now" — it cannot, by itself, prove "where the bottleneck is." To truly locate a bottleneck, you must first examine failure records and attributions: was the task unclear, was context insufficient, was the environment unreproducible, was verification feedback missing, or was state management broken? Component ablation results can only serve as supporting evidence.+**Quantifying harness component value**: Use a "controlled variable exclusion test." Keep the model fixed, remove the five subsystems one at a time, and see which subsystem's removal causes the biggest performance drop. The component with the largest drop has the highest marginal contribution for the current task and is worth prioritizing. Whether to strengthen it depends on failure attribution, not just the size of the drop. Components with near-zero impact should not be dismissed outright: they may be redundant, poorly designed, or simply not exercised by the current task. This experiment answers "which component is most valuable right now" — it cannot, by itself, prove "where the bottleneck is." To truly locate a bottleneck, you must first examine failure records and attributions: was the task unclear, was context insufficient, was the environment unreproducible, was verification feedback missing, or was state management broken? Component ablation results can only serve as supporting evidence.## A Team's Real Story@@ -89,7 +89,7 @@ Four iterations, the model did not change at all, and success rate went from 20%- Harness = Instructions + Tools + Environment + State + Feedback. All five subsystems are essential.- If it is not model weights, it is harness. Your harness determines how much of the model's capability gets realized.- Among the five subsystems, the feedback subsystem usually has the lowest investment and highest return. Get your verification commands right first.-- Use "ablation experiments under the same model" to quantify each subsystem's marginal contribution; to locate the real bottleneck, rely on failure records and attribution, not ablation alone.+- Use "controlled variable exclusion tests" to quantify each subsystem's marginal contribution; to locate the real bottleneck, rely on failure records and attribution, not ablation alone.- Harness rots like code does. Audit regularly, and pay down harness debt just like you pay down technical debt.## Further Reading@@ -104,6 +104,6 @@ Four iterations, the model did not change at all, and success rate went from 20%1. **Five-tuple harness audit**: Take a project where you currently use an AI agent and do a complete audit using the five-tuple framework. Score each subsystem 1-5. Find the lowest-scoring subsystem, spend 30 minutes improving it, and then observe the change in agent performance.-2. **Component-value ablation under the same model**: Pick one model and one challenging task. Sequentially remove instructions (delete AGENTS.md), remove feedback (do not provide verification commands), remove state (no progress files) — removing only one at a time, and measure the performance drop. Use the results to rank each subsystem's marginal value for the current task. If you want to find the bottleneck, you must also record failure logs and do root-cause attribution alongside the ablation.+2. **Controlled variable exclusion test**: Pick one model and one challenging task. Sequentially remove instructions (delete AGENTS.md), remove feedback (do not provide verification commands), remove state (no progress files) — removing only one at a time, and measure the performance drop. Use the results to rank each subsystem's marginal value for the current task. If you want to find the bottleneck, you must also record failure logs and do root-cause attribution alongside the ablation.3. **Affordance analysis**: Find a scenario in your project where the agent "wants to do something but can't" (e.g., knows it should use parameterized queries but does not know your project's ORM patterns). Analyze whether this is a Gulf of Execution (does not know how to operate) or a Gulf of Evaluation (does not know whether it did things right), then design a harness improvement to bridge the gap.diff --git a/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md b/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.mdindex a3fecdc..4954f11 100644--- a/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md+++ b/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md@@ -34,7 +34,7 @@ flowchart LRWarning["If a rule isn't in the repo,<br/>the agent can't see it"] --> Agent```-How do you test whether your map is good enough? Run a "cold-start test": open a brand-new agent session, give it only the repository contents, and see if it can answer five basic questions.+How do you test whether your map is good enough? Run a "fresh session test": open a brand-new agent session, give it only the repository contents, and see if it can answer five basic questions.```mermaidflowchart TB@@ -57,7 +57,7 @@ If it can't answer, the map has blank spots. Where the map is blank, the agent h- **Knowledge Visibility Gap**: The proportion of total project knowledge that's NOT in the repository. The bigger the gap, the higher the agent's failure rate. You can estimate it like this: count all the implicit knowledge about this project that lives in people's heads, then see how much of it actually made it into the repo. The difference is your visibility gap.- **System of Record**: The code repository as the authoritative source for project decisions, architecture constraints, execution state, and verification standards. The repo has the final say — nowhere else counts. If the information "this road is closed" only lives in Old Zhang's head, then every single time you have to ask Old Zhang. Write it in the repo, and nobody has to ask.-- **Cold-Start Test**: The five questions from the previous section. How many the agent can answer is how complete your map is.+- **Fresh Session Test**: The five questions from the previous section. How many the agent can answer is how complete your map is.- **Discovery Cost**: How much context budget the agent burns to find a single key piece of information in the repo. The more hidden the information, the higher the discovery cost, and the less budget remains for the actual task. Critical information should be placed where the agent sees it first — not buried ten directory levels deep.- **Knowledge Decay Rate**: The proportion of knowledge entries in the repo that become stale per unit of time. Documentation drifting out of sync with code is the biggest enemy — worse than no documentation at all is documentation that's out of date.- **ACID Analogy**: Applying database transaction principles (Atomicity, Consistency, Isolation, Durability) to agent state management. We'll expand on this below.@@ -68,7 +68,7 @@ If it can't answer, the map has blank spots. Where the map is blank, the agent h**Principle 2: Use a standardized entry file.** `AGENTS.md` (or `CLAUDE.md`) is the agent's "landing page." It doesn't need to contain all information, but it must let the agent quickly answer three questions: "What is this project," "How do I run it," and "How do I verify it." 50–100 lines is enough.-**Principle 3: Minimal but complete.** Every piece of knowledge should have a clear use case. If removing a rule doesn't affect the agent's decision quality, that rule shouldn't exist. But every question from the cold-start test must have an answer. This is an ongoing balance to maintain — not too much, not too little, just enough.+**Principle 3: Minimal but complete.** Every piece of knowledge should have a clear use case. If removing a rule doesn't affect the agent's decision quality, that rule shouldn't exist. But every question from the fresh session test must have an answer. This is an ongoing balance to maintain — not too much, not too little, just enough.**Principle 4: Update with code.** Bind knowledge updates to code changes. The simplest approach: put architecture docs in the corresponding module directory. When you modify code, you naturally notice the doc. After code changes, CI can remind you to check whether the docs need updating.@@ -110,12 +110,12 @@ The team executed a transformation:3. Created a centralized `CONSTRAINTS.md` using explicit "MUST / MUST NOT" language for hard constraints4. Added `PROGRESS.md` in each service directory tracking current work status-After transformation: the same agent could answer all key project questions on cold start, and task completion quality improved significantly.+After transformation: the same agent could answer all key project questions on a fresh session, and task completion quality improved significantly.## Key Takeaways- Knowledge not in the repo doesn't exist for the agent. Putting critical decision information into the repository is the most fundamental harness investment — draw a good map so you don't get lost.-- Use the "cold-start test" to evaluate repo quality: can a brand-new session answer five basic questions using only repo contents?+- Use the "fresh session test" to evaluate repo quality: can a brand-new session answer five basic questions using only repo contents?- Knowledge should be near code, minimal but complete, and updated together with code. It's not about writing more docs — it's about putting information in the right place.- Use ACID principles for agent state: atomic commits, consistency verification, concurrency isolation, and durable critical knowledge.- Knowledge decay is the biggest enemy. Out-of-date documentation is more dangerous than no documentation at all — it sends the agent in the wrong direction while the agent thinks it's on the right track.@@ -130,7 +130,7 @@ After transformation: the same agent could answer all key project questions on c## Exercises-1. **Cold-start test**: Open a completely fresh agent session in your project (provide no verbal context whatsoever), let it see only the repository contents, then ask it five questions: What is this system? How is it organized? How do I run it? How do I verify it? What's the current progress? Record which ones it can't answer, then improve the repo until it can answer all of them.+1. **Fresh session test**: Open a completely fresh agent session in your project (provide no verbal context whatsoever), let it see only the repository contents, then ask it five questions: What is this system? How is it organized? How do I run it? How do I verify it? What's the current progress? Record which ones it can't answer, then improve the repo until it can answer all of them.2. **Knowledge externalization quantification**: List all decisions and constraints important to development work in your project. Mark each item as inside or outside the repo. Calculate your knowledge visibility gap (the proportion of items outside the repo). Make a plan to bring the gap below 10%.diff --git a/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md b/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.mdindex bfe7bf4..7c9cddd 100644--- a/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md+++ b/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md@@ -28,11 +28,11 @@ This is a perfectly natural reaction. "Add a rule" every time something goes wro## Core Concepts- **Instruction Bloat**: When an instruction file occupies 10-15% of the context window, it starts crowding out budget for code reading and task reasoning. A 600-line `AGENTS.md` might consume 10,000-20,000 tokens — that's 8-15% of a 128K window.-- **Lost in the Middle Effect**: Liu et al.'s 2023 research showed that LLMs utilize information in the middle of long texts significantly less effectively than information at the beginning or end. A critical constraint buried at line 300 of a 600-line file has a very high probability of being ignored.+- **Lost in the Middle**: Information in the middle of long texts is easily overlooked. Liu et al.'s 2023 research showed that LLMs utilize information in the middle of long texts significantly less effectively than information at the beginning or end. A critical constraint buried at line 300 of a 600-line file has a very high probability of being ignored.- **Instruction Signal-to-Noise Ratio (SNR)**: The proportion of instructions in a file that are relevant to the current task. Being forced to read 50 lines of deployment instructions during a bug fix — that's low SNR.-- **Routing File**: A short entry file whose purpose is to guide the agent toward more detailed documentation, rather than containing everything itself. 50-200 lines is sufficient.-- **Progressive Disclosure**: Give overview information first, detailed information when needed. Good harness design is like good UI design — don't dump all options on the user at once.-- **Priority Ambiguity**: When all instructions appear in the same format and location, the agent cannot distinguish non-negotiable hard constraints from suggestive soft guidelines.+- **Entry File**: A short entry file whose purpose is to guide the agent toward more detailed documentation, rather than containing everything itself. 50-200 lines is sufficient.+- **Reveal on Demand**: Give overview information first, detailed information when needed. Good harness design is like good UI design — don't dump all options on the user at once.+- **Can't Tell What Matters**: When all instructions appear in the same format and location, the agent cannot distinguish non-negotiable hard constraints from suggestive soft guidelines.## Instruction Architecture@@ -102,10 +102,10 @@ Agent performance started declining noticeably: during simple bug fixes the agenThe team executed a split refactoring:1. `AGENTS.md` trimmed to 80 lines: only project overview, run commands, and 15 global hard constraints2. Created topic documents: `docs/api-patterns.md` (120 lines), `docs/database-rules.md` (60 lines), `docs/testing-standards.md` (80 lines)-3. Added topic document links in the routing file+3. Added topic document links in the entry file4. Historical notes either converted to test cases or deleted outright-After refactoring: success rate on the same task set improved from 45% to 72%. Security constraint compliance rose from 60% to 95%, because the rule moved from the middle of the file to the top of the routing file — no longer "lost in the middle."+After refactoring: success rate on the same task set improved from 45% to 72%. Security constraint compliance rose from 60% to 95%, because the rule moved from the middle of the file to the top of the entry file — no longer "lost in the middle."## Key Takeaways@@ -127,6 +127,6 @@ After refactoring: success rate on the same task set improved from 45% to 72%. S1. **SNR Audit**: Take your current entry instruction file and list all instruction entries. Pick 5 different common task types and mark whether each instruction is relevant to that task. Calculate the SNR for each task type. Instructions that are noise for most tasks should be moved to topic documents.-2. **Progressive Disclosure Refactor**: If you have an instruction file over 300 lines, split it into: (a) a routing file under 100 lines, (b) 3-5 topic documents. Run the same set of tasks (at least 5) before and after refactoring, and compare success rates.+2. **Reveal on Demand Refactor**: If you have an instruction file over 300 lines, split it into: (a) an entry file under 100 lines, (b) 3-5 topic documents. Run the same set of tasks (at least 5) before and after refactoring, and compare success rates.3. **Lost in the Middle Verification**: In a long instruction file, place a critical constraint at the top, middle, and bottom respectively, running the same task set each time (at least 5 runs per position). See if there's a difference in compliance rate. You might be surprised by how strong the position effect is.diff --git a/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md b/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.mdindex 336369d..389f227 100644--- a/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md+++ b/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md@@ -15,11 +15,11 @@ Context windows are finite. This isn't a problem that model upgrades can solveA deeper problem: the information agents produce isn't uniformly important. Intermediate reasoning steps contain the "why" of decisions — why option A was chosen over B, why this library instead of that one, why a particular optimization was skipped. The final output only contains the "what" — the code itself. Compaction strategies usually preserve the latter but lose the former. The next session sees the code but doesn't know why it's written that way, and might "optimize" away a deliberate design decision.-Anthropic observed something interesting in their long-running agent research: when agents sense context is running low, they exhibit "premature convergence" behavior — rushing to finish current work, skipping verification steps, or choosing a simple solution over the optimal one. Anthropic calls this "context anxiety."+Anthropic observed something interesting in their long-running agent research: when agents sense context is running low, they exhibit a "rushed finish" behavior — rushing to finish current work, skipping verification steps, or choosing a simple solution over the optimal one. Anthropic calls this "context anxiety."## Session Continuity Flow-Without continuity artifacts, every new session has to start from scratch:+Without state persistence files, every new session has to start from scratch:```mermaidflowchart LR@@ -29,7 +29,7 @@ flowchart LRGuess --> Drift["Work gets repeated<br/>and recovery is slow"]```-With continuity artifacts, new sessions can pick up quickly:+With state persistence files, new sessions can pick up quickly:```mermaidflowchart LR@@ -49,10 +49,10 @@ flowchart LR## Core Concepts- **Context windows are finite**: No matter what window size is claimed (128K, 200K, 1M), long tasks will eventually exhaust them. After exhaustion, either compaction (losing information) or reset (starting a new session) is required — both lose something.-- **Continuity artifacts**: Persisted state files that let a new session unambiguously resume where the last one left off. The most basic form includes progress logs, verification records, and next actions.+- **State persistence files**: Persisted state files that let a new session unambiguously resume where the last one left off. The most basic form includes progress logs, verification records, and next actions.- **Rebuild cost**: The time a new session needs to reach an executable state. A good harness can compress rebuild cost from 15 minutes to 3 minutes.- **Drift**: The gap between the agent's understanding and the actual state of the code repository. Every session boundary introduces drift; without control, it compounds session after session.-- **Context anxiety**: A phenomenon observed by Anthropic — agents exhibit premature convergence behavior when approaching context limits, ending tasks early to avoid information loss. At its core, it's an irrational resource anxiety.+- **Context anxiety**: A phenomenon observed by Anthropic — agents exhibit rushed finish behavior when approaching context limits, ending tasks early to avoid information loss. At its core, it's an irrational resource anxiety.- **Compaction vs. reset**: Compaction summarizes context within the same session (keeps "what," may lose "why"); reset opens a new session rebuilding from persisted state (clean but depends on artifact completeness).## What Happens When Continuity Breaks@@ -71,7 +71,7 @@ Both OpenAI and Anthropic emphasize structured state persistence in their documeCore approach: **Treat the agent like an engineer whose short-term memory gets wiped at every session.** Before it "clocks out," it must write down critical information so the next "shift" agent can pick up quickly.-**Tool 1: Progress file (PROGRESS.md).** The most basic continuity artifact:+**Tool 1: Progress file (PROGRESS.md).** The most basic state persistence file:```markdown# Project Progress@@ -131,11 +131,11 @@ Core approach: **Treat the agent like an engineer whose short-term memory gets w### A Deeper Look at Context Anxiety-Anthropic's March 2026 research further revealed the specific manifestations of context anxiety: on Sonnet 4.5, when context approaches the window limit, the agent shows strong "premature convergence" behavior.+Anthropic's March 2026 research further revealed the specific manifestations of context anxiety: on Sonnet 4.5, when context approaches the window limit, the agent shows strong "rushed finish" behavior.Two strategies address this:-**Compaction**: Summarizing early conversation within the same session. Advantage: maintains continuity, the agent can see "what." Disadvantage: "why" is often lost in summaries — why option B was chosen over A, why a particular optimization was skipped. More critically, compaction doesn't eliminate context anxiety — the agent knows context was once large, and psychologically still tends to rush to closure.+**Compaction**: Summarizing early conversation within the same session. Advantage: maintains continuity, the agent can see "what." Disadvantage: "why" is often lost in summaries — why option B was chosen over A, why a particular optimization was skipped. More critically, compaction doesn't eliminate context anxiety — the agent knows context was once large, and psychologically still tends to rush to finish.**Context reset**: Completely clearing context, opening a new session, rebuilding from persisted artifacts. Advantage: clean mental state — the new session has no "I'm running out of time" anxiety. Disadvantage: depends on the completeness of handoff artifacts. If the progress file is missing critical information, the new session may waste time going in the wrong direction.@@ -147,9 +147,9 @@ Anthropic's actual data: for Sonnet 4.5, context anxiety is severe enough that cAn agent was tasked with implementing a blog system with user authentication — 12 feature points, estimated 5 sessions needed.-**Baseline without continuity artifacts**: Session 1 implemented the user model and basic routes. Session 2 started without the agent remembering the auth middleware's interface contract, spending ~15 minutes inferring the previous design intent. By session 3, accumulated drift caused the agent to start reimplementing already-completed features. By session 5, the repo contained lots of redundant code but the core auth feature still hadn't passed end-to-end tests. Only 7 of 12 feature points completed, 3 with hidden correctness issues.+**Baseline without state persistence files**: Session 1 implemented the user model and basic routes. Session 2 started without the agent remembering the auth middleware's interface contract, spending ~15 minutes inferring the previous design intent. By session 3, accumulated drift caused the agent to start reimplementing already-completed features. By session 5, the repo contained lots of redundant code but the core auth feature still hadn't passed end-to-end tests. Only 7 of 12 feature points completed, 3 with hidden correctness issues.-**With continuity artifacts**: Using progress files, decision logs, verification records, and git checkpoints. State report updated automatically at each session end. Session 2's rebuild cost dropped to ~3 minutes. By session 5, all 12 feature points completed and verified.+**With state persistence files**: Using progress files, decision logs, verification records, and git checkpoints. State report updated automatically at each session end. Session 2's rebuild cost dropped to ~3 minutes. By session 5, all 12 feature points completed and verified.Quantitative comparison: rebuild time reduced ~78%, feature completion rate from 58% to 100%, hidden defect rate from 43% down to 8%.@@ -171,7 +171,7 @@ Quantitative comparison: rebuild time reduced ~78%, feature completion rate from## Exercises-1. **Continuity loss measurement**: Pick a development task needing at least 3 sessions. Without providing any continuity artifacts, record at each session start how much context the agent spends "figuring out what happened last time." After each session, create a progress file and let the next session start from it. Compare rebuild costs with and without progress files.+1. **State persistence measurement**: Pick a development task needing at least 3 sessions. Without providing any state persistence files, record at each session start how much context the agent spends "figuring out what happened last time." After each session, create a progress file and let the next session start from it. Compare rebuild costs with and without progress files.2. **Handoff template design**: Design a minimal handoff template with four fields: repo state (commit hash), runtime state (test pass rate), blockers, next actions. Let a completely fresh agent session restore project state using only this template. Record ambiguities encountered during restoration, iterate to improve the template.diff --git a/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md b/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.mdindex 18ecbb2..48e1834 100644--- a/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md+++ b/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md@@ -29,7 +29,7 @@ flowchart TBsubgraph Right["Dedicated initialization phase (right)"]R1["Session 1: environment runnable"] --> R2["Example test passing"]- R2 --> R3["Bootstrap contract + task list written"]+ R2 --> R3["Startup readiness checklist + task list written"]R3 --> R4["Clean checkpoint committed"]R4 --> R5["Later sessions start directly on verified tasks"]end@@ -52,11 +52,11 @@ OpenAI's Codex harness engineering guide also emphasizes the "repository as oper## Core Concepts- **Initialization Phase**: The first phase in the agent's lifecycle — it only establishes the prerequisites for subsequent implementation, with no feature development. Its output is infrastructure, not business code.-- **Bootstrap Contract**: The conditions under which a project can be unambiguously operated by a fresh agent session: can start, can test, can see progress, can pick up next steps. Four conditions, all required.-- **Cold Start vs Warm Start**: Cold start is from an empty directory where the agent must infer project structure on its own; warm start is from a template or existing project where infrastructure is already in place. Warm start far outperforms cold start.-- **Handoff Readiness**: The project is in a state at any given moment where a fresh agent can take over. No verbal explanation needed — just looking at the repo contents is enough to continue working.-- **Time to First Verification**: The time from project start until the first feature point passes verification. This is the core metric for measuring initialization efficiency.-- **Downstream Usability**: The proportion of subsequent sessions that can successfully execute tasks without relying on implicit knowledge. This is the best measure of initialization quality.+- **Startup Readiness Checklist**: The conditions under which a project can be unambiguously operated by a fresh agent session: can start, can test, can see progress, can pick up next steps. Four conditions, all required.+- **From Scratch vs From Template**: Starting from scratch means the agent must infer project structure on its own from an empty directory; starting from a template means infrastructure is already in place. Starting from a template far outperforms starting from scratch.+- **Always Ready to Hand Off**: The project is in a state at any given moment where a fresh agent can take over. No verbal explanation needed — just looking at the repo contents is enough to continue working.+- **Time from Start to First Passing Test**: The time from project start until the first feature point passes verification. This is the core metric for measuring initialization efficiency.+- **Success Rate of Subsequent Sessions**: The proportion of subsequent sessions that can successfully execute tasks without relying on implicit knowledge. This is the best measure of initialization quality.## How to Do Initialization Right@@ -66,9 +66,9 @@ OpenAI's Codex harness engineering guide also emphasizes the "repository as oper**2. Verifiable test framework.** At least one example test passes, proving the test framework itself is properly configured.-**3. Bootstrap contract document.** A clear document telling subsequent sessions:+**3. Startup readiness checklist document.** A clear document telling subsequent sessions:```markdown-# Initialization Contract+# Startup Readiness Checklist## Start Commands- Install dependencies: `make setup`@@ -109,9 +109,9 @@ OpenAI's Codex harness engineering guide also emphasizes the "repository as oper**5. Git commit as checkpoint.** After initialization completes, commit a clean checkpoint. All subsequent work starts from this checkpoint.-**Warm start strategy**: Don't start from an empty directory. Use a project template (create-react-app, fastapi-template, etc.) to preset standard directory structure, dependency configuration, and test framework. Bake common initialization steps into the template, leaving only project-specific initialization work.+**Starting from a template**: Don't start from an empty directory. Use a project template (create-react-app, fastapi-template, etc.) to preset standard directory structure, dependency configuration, and test framework. Bake common initialization steps into the template, leaving only project-specific initialization work.-**Initialization completion criteria**: Not "how much code was written," but whether the bootstrap contract's four conditions are all met: can start, can test, can see progress, can pick up next steps. Use this checklist to validate initialization:+**Initialization completion criteria**: Not "how much code was written," but whether the startup readiness checklist's four conditions are all met: can start, can test, can see progress, can pick up next steps. Use this checklist to validate initialization:```markdown## Initialization Acceptance Checklist@@ -128,16 +128,16 @@ Two initialization approaches for a React frontend project, compared:**Mixed approach**: The agent simultaneously created project scaffolding and implemented the first feature in session 1. At session end, the repo had runnable code but no explicit start/test command documentation, no progress tracking file, no task breakdown. Session 2 spent about 20 minutes inferring project structure, test framework, and build process.-**Dedicated initialization**: Session 1 did only initialization — created directory structure from a template, configured the test framework (Vitest + React Testing Library), wrote and verified one example test, created the bootstrap contract document and task breakdown file, committed the initial checkpoint. Session 2's rebuild time was under 3 minutes, and it started working directly from the task list.+**Dedicated initialization**: Session 1 did only initialization — created directory structure from a template, configured the test framework (Vitest + React Testing Library), wrote and verified one example test, created the startup readiness checklist and task breakdown file, committed the initial checkpoint. Session 2's rebuild time was under 3 minutes, and it started working directly from the task list.Full project cycle comparison: the mixed approach's total rebuild time (across all sessions) was about 60% more than the dedicated initialization approach. The extra 20 minutes spent on initialization was recovered many times over in subsequent sessions. Invest a bit more time up front to do initialization properly, and subsequent efficiency is actually higher.## Key Takeaways- Initialization and implementation have different optimization targets — mixing them only drags both down.-- Initialization's output isn't business code, it's infrastructure: runnable environment, verifiable tests, bootstrap contract, task breakdown.-- Validate initialization with the bootstrap contract's four conditions: can start, can test, can see progress, can pick up next steps.-- Warm start beats cold start. Use project templates to preset standardized infrastructure.+- Initialization's output isn't business code, it's infrastructure: runnable environment, verifiable tests, startup readiness checklist, task breakdown.+- Validate initialization with the startup readiness checklist's four conditions: can start, can test, can see progress, can pick up next steps.+- Starting from a template beats starting from scratch. Use project templates to preset standardized infrastructure.- Time invested in initialization is fully recovered in the next 3-4 sessions. This isn't extra cost — it's upfront investment.## Further Reading@@ -150,8 +150,8 @@ Full project cycle comparison: the mixed approach's total rebuild time (across a## Exercises-1. **Bootstrap contract design**: Write a complete bootstrap contract for a project you're developing. Then open a completely fresh agent session, show it only repo contents (no verbal context at all), and have it try to start the project, run tests, and understand current progress. Record every problem it encounters — each one corresponds to a missing clause in your bootstrap contract.+1. **Startup readiness checklist design**: Write a complete startup readiness checklist for a project you're developing. Then open a completely fresh agent session, show it only repo contents (no verbal context at all), and have it try to start the project, run tests, and understand current progress. Record every problem it encounters — each one corresponds to a missing clause in your startup readiness checklist.-2. **Comparison experiment**: Pick a moderately complex new project. Approach A: let the agent initialize and do first implementation simultaneously. Approach B: spend one session on dedicated initialization, start implementation in session 2. After 4 sessions, compare time to first verification, rebuild cost, and feature completion rate.+2. **Comparison experiment**: Pick a moderately complex new project. Approach A: let the agent initialize and do first implementation simultaneously. Approach B: spend one session on dedicated initialization, start implementation in session 2. After 4 sessions, compare time from start to first passing test, rebuild cost, and feature completion rate.3. **Initialization acceptance checklist**: Design an initialization acceptance checklist for your project. Have a fresh agent session execute each checklist item and record which pass and which fail. The failing items are where your harness needs strengthening.diff --git a/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.md b/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.mdindex c1a13ec..70a6079 100644--- a/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.md+++ b/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.md@@ -21,11 +21,11 @@ OpenAI 在 2025 年的 harness engineering 文章里把这件事说得更直白具体的失败模式其实就那么几种:-- **任务没交代清楚。** "加个搜索功能",这话说了等于没说。搜索的对象是什么?全文本还是结构化查询?结果要不要分页、要不要高亮?你没说明白,agent 就只好自己猜。猜对了算运气好,猜错了你再改,来回一折腾,比一开始说清楚多花好几倍的时间。-- **架构约定是暗坑。** 你们全组都用 SQLAlchemy 2.0 的新语法,但 agent 默认写了 1.x 的代码;所有 API 端点必须走 OAuth 2.0 认证,可这条规矩只存在于你脑子里和三个月前一条 Slack 消息里。Agent 压根不知道有这么回事,不是不想遵守,是真没见过。-- **环境常常出问题。** 开发环境配置不完整、依赖缺了、工具版本不对,agent 把宝贵的上下文窗口花在了 `pip install` 报错、Node 版本冲突这些事上,真正该干的活反而没精力做。-- **没有验证手段。** 没有测试、没有 lint、或者验证命令根本没告诉 agent。Agent 写完代码,自己看了看觉得没问题,就说完成了。Anthropic 还观察到一个有意思的现象:当 agent 感觉上下文快满了,它们会匆忙结束当前工作,跳过验证步骤,选一个简单的方案而不是最优方案。他们把这叫"上下文焦虑"。-- **跨会话的状态丢失。** 上次会话的发现全丢了,每个新会话都得重新探索项目结构、理解代码组织。缺乏持久化状态的 agent 在超过 30 分钟的任务中失败率急剧上升。+- **需求描述模糊,agent 只能猜。** "加个搜索功能",这话说了等于没说。搜索的对象是什么?全文本还是结构化查询?结果要不要分页、要不要高亮?你没说明白,agent 就只好自己猜。猜对了算运气好,猜错了你再改,来回一折腾,比一开始说清楚多花好几倍的时间。+- **隐性约定没写下来,agent 无从遵守。** 你们全组都用 SQLAlchemy 2.0 的新语法,但 agent 默认写了 1.x 的代码;所有 API 端点必须走 OAuth 2.0 认证,可这条规矩只存在于你脑子里和三个月前一条 Slack 消息里。Agent 压根不知道有这么回事,不是不想遵守,是真没见过。+- **环境配置有缺口,agent 把精力花在修环境上。** 开发环境配置不完整、依赖缺了、工具版本不对,agent 把宝贵的上下文窗口花在了 `pip install` 报错、Node 版本冲突这些事上,真正该干的活反而没精力做。+- **缺少验证手段,agent 自己觉得做完了就算完成。** 没有测试、没有 lint、或者验证命令根本没告诉 agent。Agent 写完代码,自己看了看觉得没问题,就说完成了。Anthropic 还观察到一个有意思的现象:当 agent 感觉上下文快满了,它们会匆忙结束当前工作,跳过验证步骤,选一个简单的方案而不是最优方案。他们把这叫"上下文焦虑"。+- **跨会话状态丢失,每个新会话都要重新探索。** 上次会话的发现全丢了,每个新会话都得重新探索项目结构、理解代码组织。缺乏持久化状态的 agent 在超过 30 分钟的任务中失败率急剧上升。## 关键名词解释diff --git a/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.md b/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.mdindex b7f52d5..f69e099 100644--- a/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.md+++ b/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.md@@ -33,7 +33,7 @@ OpenAI 在他们的 harness engineering 文章里把 harness 的核心原则表- **仓库是唯一事实来源**:agent 看不到的东西,对它来说就不存在。OpenAI 把仓库当作"记录系统",所有必要的上下文都必须在仓库里,通过结构化的文件和清晰的目录组织来呈现。- **给地图,不给说明书**:OpenAI 的经验是 `AGENTS.md` 应该是目录页,不是百科全书。100 行左右就够了,放不下就拆分到 `docs/` 目录里,让 agent 按需去读。- **约束而非微操**:好的 harness 用可执行的规则来约束 agent,而不是在指令里逐条叮嘱。OpenAI 说"执行不变量,不要微管实现";Anthropic 发现 agent 会自信地夸赞自己的工作,解决方案是把"干活的人"和"检查的人"分开。-- **逐个组件拆除法**:想量化 harness 各组件的边际贡献,就逐个移除,看哪个移除后性能下降最多。它能告诉你哪些组件当前最有价值,也能暴露哪些组件暂时贡献不明显。Anthropic 用这个方法发现:随着模型变强,某些组件不再关键,但总会有新的关键组件出现。+- **逐个移除看效果**:想量化 harness 各组件的边际贡献,就逐个移除,看哪个移除后性能下降最多。它能告诉你哪些组件当前最有价值,也能暴露哪些组件暂时贡献不明显。Anthropic 用这个方法发现:随着模型变强,某些组件不再关键,但总会有新的关键组件出现。## Harness 五子系统模型@@ -68,7 +68,7 @@ flowchart LR五个子系统缺一个,harness 就不完整,agent 用起来总会别扭。-**量化 harness 组件价值的方法**:用"等模型对照下的组件拆除实验"。保持模型不变,逐个移除五个子系统,看哪个子系统缺失时性能下降最多。下降最多的组件,说明它在当前任务里边际贡献最大,值得优先保留;是否要加强它,取决于失败归因,而不是只看下降幅度。几乎没有影响的组件,也不能直接判定为无用:它可能是冗余、设计失效,或还没有被这个任务充分触发。这个实验回答的是"当前哪个组件最有价值",不能单独证明"瓶颈在哪里"。真正定位瓶颈,要先看失败记录和归因:任务没说清楚、上下文不足、环境不可复现、验证反馈缺失,还是状态管理断裂;组件拆除结果只能作为辅助证据。+**量化 harness 组件价值的方法**:用"控制变量排除法"。保持模型不变,逐个移除五个子系统,看哪个子系统缺失时性能下降最多。下降最多的组件,说明它在当前任务里边际贡献最大,值得优先保留;是否要加强它,取决于失败归因,而不是只看下降幅度。几乎没有影响的组件,也不能直接判定为无用:它可能是冗余、设计失效,或还没有被这个任务充分触发。这个实验回答的是"当前哪个组件最有价值",不能单独证明"瓶颈在哪里"。真正定位瓶颈,要先看失败记录和归因:任务没说清楚、上下文不足、环境不可复现、验证反馈缺失,还是状态管理断裂;组件拆除结果只能作为辅助证据。## 一个团队的真实经历@@ -89,7 +89,7 @@ flowchart LR- Harness = 指令 + 工具 + 环境 + 状态 + 反馈,五个子系统缺一不可。- 不是模型权重的部分全是 harness,你的 harness 决定了模型能力能被发挥多少。- 五个子系统中,反馈子系统通常是投入最少、回报最高的。先把验证命令写清楚。-- 用"等模型对照下的组件拆除实验"量化各子系统的边际贡献;定位真正瓶颈要靠失败记录和归因,不能只靠拆除实验。+- 用"控制变量排除法"量化各子系统的边际贡献;定位真正瓶颈要靠失败记录和归因,不能只靠拆除实验。- Harness 和代码一样会腐化,定期审计,像还技术债一样还 harness 债。## 延伸阅读diff --git a/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md b/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.mdindex 4e9a9b8..bb26751 100644--- a/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md+++ b/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md@@ -34,7 +34,7 @@ flowchart LRWarning["规则不写进仓库<br/>agent 就看不见"] --> Agent```-怎么检验你的地图画得够不够好?做一个"冷启动测试":开一个全新的 agent 会话,只让它看仓库内容,看它能不能回答五个基本问题。+怎么检验你的地图画得够不够好?做一个"全新会话测试":开一个全新的 agent 会话,只让它看仓库内容,看它能不能回答五个基本问题。```mermaidflowchart TB@@ -57,7 +57,7 @@ flowchart TB- **知识可见性缺口**:项目总知识中不在仓库里的比例。缺口越大,agent 失败的概率越高。你可以这样估算:把脑子里关于这个项目的隐性知识全算上,再看有多少写进了仓库,两者的差距就是可见性缺口。- **系统记录(System of Record)**:代码仓库作为项目决策、架构约束、执行状态和验证标准的权威信息源。仓库说了算,别的地方说了不算。如果"此路不通"这个信息只在老张的脑子里,那每次都得问老张。写进仓库,谁都不用问。-- **冷启动测试**:上一节说的五个问题。能回答几个,你的地图就画了几分。+- **全新会话测试**:上一节说的五个问题。能回答几个,你的地图就画了几分。- **发现成本**:agent 为了在仓库里找到一条关键信息需要消耗多少上下文。信息放得越隐蔽,发现成本越高,留给实际任务的预算越少。关键信息应该放在 agent 最先看到的位置,而不是藏在十层目录深处。- **知识衰减率**:仓库中单位时间内变得过时的知识条目比例。文档和代码脱节是最大的敌人,比没有文档更危险的是过时的文档。- **ACID 类比**:把数据库的事务管理原则(原子性、一致性、隔离性、持久性)用到 agent 的状态管理上。后面会展开讲。@@ -68,7 +68,7 @@ flowchart TB**原则 2:用标准化的入口文件。** `AGENTS.md`(或 `CLAUDE.md`)是 agent 的"着陆页"。它不需要包含所有信息,但必须能让 agent 快速回答"这是什么项目"、"怎么跑"、"怎么验证"这三个问题。50-100 行就够了。-**原则 3:最小但完备。** 每条知识都应该有明确的使用场景。如果你删掉某条规则不影响 agent 的决策质量,那这条规则就不应该存在。但冷启动测试中的每个问题都必须有答案。这是一个需要持续调整的平衡,不多不少,刚好够用。+**原则 3:最小但完备。** 每条知识都应该有明确的使用场景。如果你删掉某条规则不影响 agent 的决策质量,那这条规则就不应该存在。但全新会话测试中的每个问题都必须有答案。这是一个需要持续调整的平衡,不多不少,刚好够用。**原则 4:和代码一起更新。** 把知识更新跟代码变更绑定在一起。最简单的方法:把架构文档放在对应的模块目录里。改代码的时候自然会看到文档,改代码之后 CI 提醒你检查文档是否需要更新。@@ -115,7 +115,7 @@ project/## 核心要点- 不在仓库里的知识对 agent 来说等于不存在。把关键决策信息放进仓库是最基本的 harness 投资,画好地图才不会迷路。-- 用"冷启动测试"检验仓库质量:全新会话能不能只看仓库回答五个基本问题。+- 用"全新会话测试"检验仓库质量:全新会话能不能只看仓库回答五个基本问题。- 知识要靠近代码、最小但完备、跟代码一起更新。不是写更多文档,是把信息放到正确的位置。- 用 ACID 原则管理 agent 状态:原子提交、一致性验证、隔离并发、持久化关键知识。- 知识衰减是最大敌人。过时的文档比没有文档更危险,它会让 agent 走错方向还以为自己是对的。@@ -130,7 +130,7 @@ project/## 练习-1. **冷启动测试**:在你的项目里开一个全新的 agent 会话(不提供任何口头上下文),只让它看仓库内容,然后问它五个问题:这是什么系统?怎么组织的?怎么运行?怎么验证?现在进度如何?记录它答不上来的问题,然后改进仓库让它能答上来。+1. **全新会话测试**:在你的项目里开一个全新的 agent 会话(不提供任何口头上下文),只让它看仓库内容,然后问它五个问题:这是什么系统?怎么组织的?怎么运行?怎么验证?现在进度如何?记录它答不上来的问题,然后改进仓库让它能答上来。2. **知识外置化量化**:列出你的项目中所有对开发工作重要的决策和约束。标注每个条目是在仓库内还是仓库外。算一下你的知识可见性缺口有多大(不在仓库里的占总数的比例)。制定计划把缺口降到 10% 以下。diff --git a/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md b/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.mdindex d9a796f..f397180 100644--- a/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md+++ b/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md@@ -28,11 +28,11 @@## 核心概念- **指令膨胀**:指令文件一旦占到上下文窗口的 10-15%,就开始挤占代码阅读和任务推理的预算。一个 600 行的 `AGENTS.md` 可能占用 10,000-20,000 tokens,对 128K 的窗口来说就是 8-15%。-- **中间迷失效应**:Liu 等人 2023 年的研究表明,LLM 对长文本中间部分的信息利用效率明显低于两端。埋在 600 行文件第 300 行的关键约束,被忽略的概率非常高。+- **长文本中间信息容易被忽略**:Liu 等人 2023 年的研究表明,LLM 对长文本中间部分的信息利用效率明显低于两端。埋在 600 行文件第 300 行的关键约束,被忽略的概率非常高。- **指令信噪比(SNR)**:文件中与当前任务相关的指令占总指令的比例。做 bug 修复时被要求读 50 行部署指令,SNR 就很低。-- **路由文件**:短小的入口文件,作用是引导 agent 去找更详细的文档,而不是自己包含所有内容。50-200 行就够了。-- **渐进式披露**:先给概要信息,需要的时候再给详细信息。好的 harness 设计和好的 UI 设计一样,不把所有选项一次性砸到用户脸上。-- **优先级模糊度**:当所有指令以相同格式和位置呈现时,agent 分不清哪些是不可违反的硬约束,哪些只是建议性的软约束。+- **入口文件**:短小的入口文件,作用是引导 agent 去找更详细的文档,而不是自己包含所有内容。50-200 行就够了。+- **按需展开**:先给概要信息,需要的时候再给详细信息。好的 harness 设计和好的 UI 设计一样,不把所有选项一次性砸到用户脸上。+- **分不清轻重**:当所有指令以相同格式和位置呈现时,agent 分不清哪些是不可违反的硬约束,哪些只是建议性的软约束。## 指令文件架构@@ -102,10 +102,10 @@ Agent 表现开始明显下降:简单 bug 修复任务中 agent 花大量上团队做了一次拆分重构:1. `AGENTS.md` 裁剪到 80 行:只保留项目概览、运行命令、15 条全局硬约束2. 创建专题文档:`docs/api-patterns.md`(120 行)、`docs/database-rules.md`(60 行)、`docs/testing-standards.md`(80 行)-3. 路由文件添加指向专题文档的链接+3. 入口文件添加指向专题文档的链接4. 历史笔记要么转成测试用例,要么删除-重构后:同一任务集的成功率从 45% 提升到 72%。安全约束遵循率从 60% 提升到 95%,因为规则从文件中间移到了路由文件顶部,不再被"中间迷失"了。+重构后:同一任务集的成功率从 45% 提升到 72%。安全约束遵循率从 60% 提升到 95%,因为规则从文件中间移到了入口文件顶部,不再被"中间迷失"了。## 核心要点@@ -127,6 +127,6 @@ Agent 表现开始明显下降:简单 bug 修复任务中 agent 花大量上1. **信噪比审计**:拿你当前的入口指令文件,列出所有指令条目。选 5 个不同的常见任务类型,标注每条指令是否跟该任务相关。计算每个任务类型的 SNR。那些对大多数任务都是噪声的指令,移到专题文档里。-2. **渐进式披露重构**:如果你有一个超过 300 行的指令文件,把它拆成:(a) 不超过 100 行的路由文件,(b) 3-5 个专题文档。重构前后各跑同一组任务(至少 5 个),对比成功率。+2. **按需展开重构**:如果你有一个超过 300 行的指令文件,把它拆成:(a) 不超过 100 行的入口文件,(b) 3-5 个专题文档。重构前后各跑同一组任务(至少 5 个),对比成功率。3. **中间迷失验证**:在一个长指令文件里,把一条关键约束分别放在顶部、中间、底部各跑一组任务(每组至少 5 次),看遵循率有没有差别。你可能会惊讶于位置效应有多大。diff --git a/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md b/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.mdindex 2f00597..ccc98af 100644--- a/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md+++ b/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md@@ -15,11 +15,11 @@更深层的问题在于,agent 产生的信息不是均匀重要的。中间推理步骤包含决策的"为什么",比如为什么选了方案 A 而不是方案 B,为什么用了这个库而不是那个库,为什么跳过了某个优化。最终输出只包含"是什么",即代码本身。压缩策略通常保留后者但丢了前者。下一个会话看到代码却不知道为什么这么写,可能会"优化"掉一个有意为之的设计决策。-Anthropic 在长运行 agent 研究中观察到一个有意思的现象:当 agent 感觉上下文快满了,它们会表现出"过早收敛"的行为,匆忙结束当前工作,跳过验证步骤,或者选一个简单的方案而不是最优方案。Anthropic 把这叫"上下文焦虑"。+Anthropic 在长运行 agent 研究中观察到一个有意思的现象:当 agent 感觉上下文快满了,它们会表现出"赶工收尾"的行为,匆忙结束当前工作,跳过验证步骤,或者选一个简单的方案而不是最优方案。Anthropic 把这叫"上下文焦虑"。## 会话连续性流程-没有连续性工件的时候,每个新会话都要重新摸索:+没有状态持久化文件的时候,每个新会话都要重新摸索:```mermaidflowchart LR@@ -29,7 +29,7 @@ flowchart LRGuess --> Drift["代码会重复改<br/>恢复速度也很慢"]```-有连续性工件的时候,新会话能快速接上:+有状态持久化文件的时候,新会话能快速接上:```mermaidflowchart LR@@ -49,7 +49,7 @@ flowchart LR## 核心概念- **上下文窗口是有限的**:不管模型宣称多大的窗口(128K、200K、1M),长任务总会用完。用完之后要么压缩(丢信息),要么重置(开新会话),两种方式都会丢东西。-- **连续性工件**:持久化的状态文件,让新会话能无歧义地恢复到上次离开的地方。最基本的形式包括进度日志、验证记录和下一步行动。+- **状态持久化文件**:持久化的状态文件,让新会话能无歧义地恢复到上次离开的地方。最基本的形式包括进度日志、验证记录和下一步行动。- **重建成本**:新会话恢复到可执行状态所需的时间。好的 harness 能把重建成本从 15 分钟压到 3 分钟。- **漂移(Drift)**:agent 的理解跟代码仓库实际状态之间的偏差。每次会话边界都会引入漂移,不加控制会越漂越远。- **上下文焦虑**:Anthropic 观察到的现象,agent 在接近上下文限制时表现异常,过早结束任务以避免信息丢失,本质上是一种非理性的资源焦虑。@@ -71,7 +71,7 @@ OpenAI 和 Anthropic 都在他们的文档里强调了结构化状态持久化核心思路:**把 agent 当成一个每次会话都会清空短期记忆的工程师来管理。** 每次它要"下班"之前,必须把关键信息写下来,让下一个"接班"的 agent 能快速上手。-**工具 1:进度文件(PROGRESS.md)**。这是最基本的连续性工件:+**工具 1:进度文件(PROGRESS.md)**。这是最基本的状态持久化文件:```markdown# 项目进度@@ -131,7 +131,7 @@ OpenAI 和 Anthropic 都在他们的文档里强调了结构化状态持久化### 上下文焦虑的深层分析-Anthropic 在 2026 年 3 月发布的研究进一步揭示了上下文焦虑的具体表现:在 Sonnet 4.5 上,当上下文接近窗口限制时,agent 会表现出强烈的"过早收敛"行为。+Anthropic 在 2026 年 3 月发布的研究进一步揭示了上下文焦虑的具体表现:在 Sonnet 4.5 上,当上下文接近窗口限制时,agent 会表现出强烈的"赶工收尾"行为。针对这个现象,有两种策略:@@ -147,9 +147,9 @@ Anthropic 的实际数据:对于 Sonnet 4.5,上下文焦虑足够严重,一个 agent 被要求实现一个带用户认证的博客系统,12 个功能点,预计需要 5 个会话。-**没有连续性工件的基线**:会话 1 实现了用户模型和基础路由。会话 2 开始时,agent 不记得认证中间件的接口约定,花了约 15 分钟推断上次的设计意图。到会话 3,累积漂移导致 agent 开始重复已实现的功能。到会话 5,仓库有大量冗余代码,但核心认证功能仍未通过端到端测试。12 个功能点只完成了 7 个,其中 3 个有隐含的正确性问题。+**没有状态持久化文件的基线**:会话 1 实现了用户模型和基础路由。会话 2 开始时,agent 不记得认证中间件的接口约定,花了约 15 分钟推断上次的设计意图。到会话 3,累积漂移导致 agent 开始重复已实现的功能。到会话 5,仓库有大量冗余代码,但核心认证功能仍未通过端到端测试。12 个功能点只完成了 7 个,其中 3 个有隐含的正确性问题。-**有连续性工件的对照**:使用进度文件、决策日志、验证记录和 git 检查点。每个会话结束时自动更新状态报告。会话 2 的重建成本降到约 3 分钟。到会话 5,所有 12 个功能点完成且通过验证。+**有状态持久化文件的对照**:使用进度文件、决策日志、验证记录和 git 检查点。每个会话结束时自动更新状态报告。会话 2 的重建成本降到约 3 分钟。到会话 5,所有 12 个功能点完成且通过验证。定量对比:重建时间减少约 78%,功能完成率从 58% 提升到 100%,隐含缺陷率从 43% 降到 8%。@@ -171,7 +171,7 @@ Anthropic 的实际数据:对于 Sonnet 4.5,上下文焦虑足够严重,## 练习-1. **连续性损耗度量**:选一个需要至少 3 个会话的开发任务。不提供任何连续性工件,在每个会话开始时记录 agent 花了多少上下文来"搞清楚上次做了什么"。会话结束后,创建进度文件,让下一个会话从进度文件开始。对比有进度文件和没有时的重建成本。+1. **连续性损耗度量**:选一个需要至少 3 个会话的开发任务。不提供任何状态持久化文件,在每个会话开始时记录 agent 花了多少上下文来"搞清楚上次做了什么"。会话结束后,创建进度文件,让下一个会话从进度文件开始。对比有进度文件和没有时的重建成本。2. **交接模板设计**:设计一个最小化的交接模板,包含四个字段:仓库状态(commit hash)、运行时状态(测试通过率)、阻塞项、下一步行动。让一个全新的 agent 会话只凭这个模板恢复项目状态,记录恢复过程中出现的歧义点,迭代改进模板。diff --git a/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md b/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.mdindex 1bcab9a..6245079 100644--- a/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md+++ b/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md@@ -52,11 +52,11 @@ OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"## 核心概念- **初始化阶段**:agent 生命周期中的第一个阶段,只建立后续实现所需的执行前提,不做功能开发。它的产出是基础设施,而不是业务代码。-- **自举契约**:一个项目能被全新 agent 会话无歧义操作的条件:能启动、能测试、能看进度、能接手下一步。四个条件缺一不可。-- **冷启动 vs 热启动**:冷启动是从空目录开始,agent 需要自行推断项目结构;热启动是从模板或已有项目开始,基础设施已经就位。热启动的效果远好于冷启动。-- **交接就绪性**:项目在任何时刻都处于"可以被全新 agent 接手"的状态。不需要口头解释,只看仓库内容就能接着干。-- **首次验证时间**:从项目开始到第一个功能点通过验证的时间,是衡量初始化效率的核心指标。-- **下游可用性**:后续会话不需要依赖隐式知识就能成功执行任务的比例,这是初始化质量的最佳衡量标准。+- **启动就绪清单**:一个项目能被全新 agent 会话无歧义操作的条件:能启动、能测试、能看进度、能接手下一步。四个条件缺一不可。+- **从零开始 vs 从模板开始**:从零开始意味着 agent 需要自行推断项目结构,效果差;从模板开始意味着基础设施已经就位,效果好得多。能用模板就用模板。+- **随时可接手**:项目在任何时刻都处于"可以被全新 agent 接手"的状态。不需要口头解释,只看仓库内容就能接着干。+- **从开始到第一次测试通过**:衡量初始化效率的核心指标。时间越短,初始化越高效。+- **后续会话的成功率**:后续会话不需要依赖隐式知识就能成功执行任务的比例,这是初始化质量的最佳衡量标准。## 怎么做好初始化@@ -66,7 +66,7 @@ OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"**2. 可验证的测试框架。** 至少有一个示例测试能通过,证明测试框架本身是配好的。-**3. 自举契约文档。** 一个明确的文档告诉后续会话:+**3. 启动就绪清单文档。** 一个明确的文档告诉后续会话:```markdown# 初始化契约@@ -111,7 +111,7 @@ OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"**热启动策略**:不要从空目录开始。用一个项目模板(create-react-app、fastapi-template 等)预置好标准的目录结构、依赖配置和测试框架。把通用的初始化步骤预置到模板里,只留下项目特有的初始化工作。-**初始化的完成条件**:不是"写了多少代码",而是自举契约的四个条件都满足了:能启动、能测试、能看进度、能接手下一步。用这个检查清单验收初始化:+**初始化的完成条件**:不是"写了多少代码",而是启动就绪清单的四个条件都满足了:能启动、能测试、能看进度、能接手下一步。用这个检查清单验收初始化:```markdown## 初始化验收清单@@ -128,15 +128,15 @@ OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"**混合方式**:agent 在第一个会话中同时做了项目脚手架创建和首个功能实现。会话结束时,仓库有可运行的代码,但没有显式的启动和测试命令文档、没有进度跟踪文件、没有任务分解。第二个会话花了约 20 分钟推断项目结构、测试框架和构建流程。-**独立初始化**:第一个会话只做初始化,用项目模板创建目录结构、配置测试框架(Vitest + React Testing Library)、写一个示例测试并验证通过、创建自举契约文档和任务分解文件、提交初始检查点。第二个会话的重建时间不到 3 分钟,直接从任务列表开始工作。+**独立初始化**:第一个会话只做初始化,用项目模板创建目录结构、配置测试框架(Vitest + React Testing Library)、写一个示例测试并验证通过、创建启动就绪清单文档和任务分解文件、提交初始检查点。第二个会话的重建时间不到 3 分钟,直接从任务列表开始工作。整个项目周期对比:混合方式的总重建时间(跨所有会话)比独立初始化多约 60%。独立初始化多花的那 20 分钟在后续会话中被成倍收回。前期多投入一点时间把初始化做扎实,后续的效率反而更高。## 核心要点- 初始化和实现的优化目标不同,混在一起只会互相拖后腿。-- 初始化的产出不是业务代码,而是基础设施:可运行的环境、可验证的测试、自举契约、任务分解。-- 用"自举契约"的四个条件验收初始化:能启动、能测试、能看进度、能接手下一步。+- 初始化的产出不是业务代码,而是基础设施:可运行的环境、可验证的测试、启动就绪清单、任务分解。+- 用"启动就绪清单"的四个条件验收初始化:能启动、能测试、能看进度、能接手下一步。- 热启动优于冷启动,用项目模板预置标准化的基础设施。- 初始化投入的时间会在后续 3-4 个会话中完全收回,这不是额外的成本,而是前期投资。@@ -150,7 +150,7 @@ OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"## 练习-1. **自举契约设计**:为你正在开发的项目写一个完整的自举契约。然后开一个全新的 agent 会话,只给它看仓库内容(不给任何口头上下文),让它尝试启动项目、跑测试、了解当前进度。记录它遇到的问题,每个问题都对应自举契约中缺失的一个条款。+1. **启动就绪清单设计**:为你正在开发的项目写一个完整的启动就绪清单。然后开一个全新的 agent 会话,只给它看仓库内容(不给任何口头上下文),让它尝试启动项目、跑测试、了解当前进度。记录它遇到的问题,每个问题都对应启动就绪清单中缺失的一个条款。2. **对比实验**:选一个中等复杂度的新项目。方式 A 让 agent 初始化和首次实现同时做,方式 B 先花一个会话做独立初始化,第二个会话再开始实现。在 4 个会话后对比首次验证时间、重建成本和功能完成率。