Skip to content

Harness engineering beginner tutorial, from 0 to 1

Rewrite lecture narratives for natural, conversational tone

sanbuphy01024a995707
diff --git a/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md b/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md
index 7808e67..1f38e11 100644
--- a/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md
+++ b/docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md
@@ -5,108 +5,100 @@
# Lecture 01. Strong Models Don't Mean Reliable Execution
-## What Problem Does This Lecture Solve?
+You consider yourself well-traveled in the AI world — Claude Pro subscription, GPT-4o API key, SWE-bench leaderboard numbers memorized. One day you finally hand a real project to an AI agent, brimming with confidence. The result? It adds a feature but breaks the tests, fixes a bug but introduces two more, runs for 20 minutes and proudly declares "done" — and you look at the code and it's not what you asked for at all.
-You're paying for Claude Pro. GPT-4o keeps climbing benchmark leaderboards. SWE-bench scores get beaten every week. But when you actually let an AI agent loose on a real project — it adds a feature but breaks the tests, fixes a bug but introduces two more, runs for 20 minutes and proudly declares "done" while you look at code that doesn't do what you asked.
+Your first instinct? "This model isn't good enough. Time to upgrade." Hold on. Before you reach for your wallet, consider that the problem might not be the model at all.
-This isn't because the model isn't smart enough. It's because the working environment you gave it is terrible. This lecture explains why the strongest models still fail frequently on real engineering tasks, and why the root cause isn't the model — it's the harness you built (or didn't build) around it.
+Let's look at some numbers. As of late 2025, the strongest coding agents on SWE-bench Verified achieve roughly 50-60%. And that's on carefully selected tasks with clear issue descriptions and existing test cases. Move to your daily development environment — vague requirements, no existing tests, implicit business rules scattered everywhere — and that number only goes down.
-## Core Concepts
+But behind these numbers lies a counterintuitive truth.
-- **Capability Gap**: The huge gulf between model performance on benchmarks and performance on real tasks. A 50-60% pass rate on SWE-bench Verified means nearly half of real issues can't be resolved.
-- **Harness**: Everything outside the model — instructions, tools, environment, state management, verification feedback. If it's not model weights, it's harness.
-- **Harness-Induced Failure**: The model has enough capability, but the execution environment has structural defects. Anthropic's controlled experiment: same prompt, same model, bare run vs. full harness — output quality difference is night and day.
-- **Verification Gap**: The gap between the agent's confidence in its output and actual correctness. The agent says "I'm done" when it's not done — this is the most common failure mode.
-- **Diagnostic Loop**: Execute → observe failure → attribute to a specific harness layer → fix that layer → re-execute. This is the core methodology of harness engineering.
-- **Definition of Done**: A set of machine-verifiable conditions — tests pass, lint is clean, type checks pass. Without an explicit definition of done, the agent will invent its own.
+## Same Horse, Different Fates
-## Failure Model Overview
-
-```mermaid
-flowchart TB
- Start["Start the task"] --> Q1{"Is the task specific?"}
- Q1 -->|No| Bad["Result:<br/>lots of code<br/>still unstable"]
- Q1 -->|Yes| Q2{"Are the rules in the repo?"}
- Q2 -->|No| Bad
- Q2 -->|Yes| Q3{"Does the environment run?"}
- Q3 -->|No| Bad
- Q3 -->|Yes| Q4{"Are there check commands?"}
- Q4 -->|No| Bad
- Q4 -->|Yes| Q5{"Is progress recorded?"}
- Q5 -->|No| Bad
- Q5 -->|Yes| Good["More likely to finish cleanly"]
-```
+Anthropic ran a controlled experiment. Same prompt ("build a 2D retro game maker"), same model (Opus 4.5). First run: bare, no support — 20 minutes, $9, the game's core features didn't work at all. Second run: full harness (planner + generator + evaluator three-agent architecture) — 6 hours, $200, the game was playable.
-```mermaid
-flowchart TB
- Same["Same task<br/>same model"]
- Same --> Bare["Just one prompt<br/>20 min / $9"]
- Same --> Full["Rules and checks first<br/>6 hr / $200"]
- Bare --> Bad["Core features broken"]
- Full --> Good["Playable app"]
-```
+They didn't change the model. Opus 4.5 was still Opus 4.5. What changed was the saddle.
-## Why This Happens
+OpenAI's 2025 harness engineering article puts it plainly: Codex in a well-harnessed repository goes from "unreliable" to "reliable." Note their wording — not "a bit better," but a qualitative shift. Like a thoroughbred: you can ride it without a saddle, but you won't go far, won't go fast, and falling off is no surprise. The harness is that saddle — **everything in the engineering infrastructure outside the model weights.**
-Let's start with data. As of late 2025, the strongest coding agents on SWE-bench Verified achieve roughly 50-60%. And that's on carefully selected tasks with clear issue descriptions and existing test cases. Move to a real daily development scenario — vague requirements, no existing tests, implicit business rules scattered everywhere — and that number only goes down.
+## Where Agents Actually Get Stuck
-But here's the counterintuitive fact: Anthropic ran a controlled experiment. Same prompt ("build a 2D retro game maker"), same model (Opus 4.5). Single-agent bare run: 20 minutes, $9 — the game's core features didn't work at all. Full harness (planner + generator + evaluator three-agent architecture): 6 hours, $200 — the game was playable. Same model, just harness from "bare" to "fully equipped."
+So what specifically goes wrong?
-OpenAI's 2025 harness engineering article puts it plainly: Codex in a well-harnessed repository goes from "unreliable" to "reliable" — note their wording, not "a bit better," but a qualitative shift. Anthropic's long-running agent documentation reaches similar conclusions: agents with explicit recovery paths, stable state management, and reliable feedback loops far outperform those without.
+The most common: you never clearly defined the task. You say "add a search feature," and the agent's understanding is completely different from yours — search what? Full-text or structured? Pagination? Highlighting? You didn't specify, so the agent guesses. A correct guess is luck; a wrong one costs more to fix than being specific would have cost in the first place. It's like walking into a restaurant and telling the chef "I'll have fish" — whether you get it braised, steamed, or in a hot pot is entirely up to chance.
-So what's actually going wrong? We can categorize common failure modes into five layers:
+Even when you do specify, the project has implicit architectural conventions the agent doesn't know. Your team standardized on SQLAlchemy 2.0 syntax, but the agent writes 1.x code by default. All API endpoints must use OAuth 2.0 authentication, but that rule only exists in your head and a Slack message from three months ago. The agent can't see these — it's not that it doesn't want to comply, it literally doesn't know these rules exist.
-**Task Specification Layer**: When you say "add a search feature," the agent's understanding might be completely different from yours. Search what? Full-text or structured? Pagination? Highlighting? Fuzzy or exact? You didn't specify, so the agent guesses — and then you spend more time fixing the wrong guess than you would have spent being specific upfront.
+The environment is a trap too. Incomplete dev environment, missing dependencies, wrong tool versions. The agent burns precious context window on `pip install` failures and Node version mismatches instead of solving your actual task. Like hiring a skilled carpenter but forgetting to provide a hammer, nails, or a level workbench — no matter how talented, they can't do the job.
-**Context Provision Layer**: The project has implicit architectural conventions the agent doesn't know. Your team standardized on SQLAlchemy 2.0 syntax, but the agent writes 1.x code by default. All API endpoints must use OAuth 2.0, but that rule only exists in your head and a three-month-old Slack message. The agent can't see these.
+Even more common: there's simply no way to verify. No tests, no lint, or verification commands never communicated to the agent. The agent writes code, looks at it, decides it's fine, says "done." It's like asking a student to submit homework with no answer key — they think they got it right, but when you grade it there's a pile of errors. Anthropic also observed an interesting phenomenon: when agents sense context is running low, they rush to finish, skip verification, and choose a simple solution over the optimal one. They call it "context anxiety" — the same thing that happens when you realize time is almost up on an exam and start randomly guessing on the remaining multiple-choice questions.
-**Execution Environment Layer**: Incomplete dev environment, missing dependencies, wrong tool versions. The agent burns precious context window on `pip install` failures and Node version mismatches instead of solving your actual task.
+Long tasks spanning sessions are even worse — all discoveries from the previous session are lost, and every new session has to re-explore the project structure and re-understand the code organization. Agents without persistent state see failure rates spike sharply on tasks exceeding 30 minutes.
-**Verification Feedback Layer**: No tests, no lint, or verification commands not communicated to the agent. The agent writes code, looks at it, decides it's fine, says "done." It's like asking a student to submit homework with no answer key to check against — they think they got it right, but you find a pile of errors when grading.
+## Core Concepts
-**State Management Layer**: Long tasks spanning sessions lose all discoveries from the previous session. Every new session re-explores the project structure and re-understands the code organization. Anthropic's data shows that agents without persistent state see failure rates spike sharply on tasks exceeding 30 minutes.
+With these scenarios in mind, these concepts are no longer just jargon:
-## How to Do It Right
+- **Capability Gap**: The huge gulf between model performance on benchmarks and performance on real tasks. A 50-60% pass rate on SWE-bench Verified means nearly half of real issues can't be resolved.
+- **Harness**: Everything outside the model — instructions, tools, environment, state management, verification feedback. If it's not model weights, it's harness. What we've been calling the "saddle."
+- **Harness-Induced Failure**: The model has enough capability, but the execution environment has structural defects. Anthropic's controlled experiment already proved this.
+- **Verification Gap**: The gap between the agent's confidence in its output and actual correctness. The agent says "I'm done" when it's not done — this is the most common failure mode.
+- **Diagnostic Loop**: Execute, observe failure, attribute to a specific harness layer, fix that layer, re-execute. This is the core methodology of harness engineering.
+- **Definition of Done**: A set of machine-verifiable conditions — tests pass, lint is clean, type checks pass. Without an explicit definition of done, the agent will invent its own.
-Core principle: **When things fail, don't swap the model first — check the harness.** If the same model succeeds on similar, well-structured tasks, assume it's a harness problem.
+## When Things Fail, Fix the Harness First
+
+Core principle: **When things fail, don't swap the model first — check the harness.** If the same model succeeds on similar, well-structured tasks, assume it's a harness problem. It's like a car breaking down — you don't immediately suspect the engine. You check if it's out of gas first.
Concrete steps:
-1. **Attribute every failure to a specific layer.** Don't just say "the model sucks." Ask: was the task unclear? Was context insufficient? Were there no verification methods? Map each failure to one of the five layers above.
+**Attribute every failure to a specific layer.** Don't just say "the model sucks." Ask: was the task unclear? Was context insufficient? Were there no verification methods? Map each failure to one of the five layers above. Build this habit, and you'll find "the model isn't good enough" appearing less and less in your logs.
+
+**Write an explicit Definition of Done for every task.** Don't say "add a search feature." Say:
+```
+Completion criteria:
+- New endpoint GET /api/search?q=xxx
+- Supports pagination, default 20 items
+- Results include highlighted snippets
+- All new code passes pytest
+- Type checking passes (mypy --strict)
+```
+
+**Create an AGENTS.md file.** Put it in the repo root to tell the agent the project's tech stack, architectural conventions, and verification commands. This is the first step in harness engineering and the highest-ROI step you can take. One `AGENTS.md` file might be more effective than upgrading to a more expensive model — I'm not joking.
+
+**Build a diagnostic loop.** Don't treat failures as "the model being dumb again." Treat them as signals that your harness has a defect. Each failure, identify the layer, fix it, never fail that way again. After a few rounds, your harness gets stronger and agent performance stabilizes. Like road repair — every pothole you fill makes the next stretch smoother.
+
+**Quantify improvements.** Keep a simple log: did each task succeed or fail, and which layer caused the failure. After a few rounds you'll see which layer is the bottleneck — focus your energy there.
+
+## The Million-Line Experiment
+
+OpenAI ran an aggressive experiment in 2025: use Codex to build a complete internal product from an empty git repository. Five months later, the repo had roughly one million lines of code — application logic, infrastructure, tooling, documentation, internal dev tools — all agent-generated. Three engineers drove Codex, opening and merging about 1,500 PRs. An average of 3.5 PRs per person per day.
-2. **Write an explicit Definition of Done for every task.** Don't say "add a search feature." Say:
- ```
- Completion criteria:
- - New endpoint GET /api/search?q=xxx
- - Supports pagination, default 20 items
- - Results include highlighted snippets
- - All new code passes pytest
- - Type checking passes (mypy --strict)
- ```
+The key constraint: **humans never write code directly.** This wasn't a gimmick — it was designed to force the team to figure out what changes when the engineer's primary job is no longer writing code, but designing environments, expressing intent, and building feedback loops.
-3. **Create an AGENTS.md file.** Put it in the repo root to tell the agent the project's tech stack, architectural conventions, and verification commands. This is the first step in harness engineering and the highest-ROI step you can take.
+Early progress was slower than expected. Not because Codex wasn't capable, but because the environment wasn't complete enough — the agent lacked necessary tools, abstractions, and internal structures to advance high-level objectives. The engineers' work became: breaking large goals into small building blocks (design, code, review, test), letting the agent assemble them, then using those blocks to unlock more complex tasks. When something failed, the fix was almost never "try harder" — it was "what capability is the agent missing, and how do we make it both understandable and executable?"
-4. **Build a diagnostic loop.** Don't treat failures as "the model being dumb again." Treat them as signals that your harness has a defect. Each failure → identify the layer → fix it → never fail that way again. After a few rounds, your harness gets stronger and agent performance stabilizes.
+This experiment directly proves this lecture's core thesis: **the same model produces fundamentally different output in a bare environment versus one with a complete harness.** The model didn't change. The environment did.
-5. **Quantify improvements.** Keep a simple log: did each task succeed or fail, and which layer caused the failure. After a few rounds you'll see which layer is the bottleneck — focus your energy there.
+> Source: [OpenAI: Harness engineering: leveraging Codex in an agent-first world](https://openai.com/index/harness-engineering/)
-## Real-World Example
+## A More Down-to-Earth Example
A team used Claude Sonnet to add a new API endpoint to a mid-sized Python web app (FastAPI + PostgreSQL + Redis, ~15,000 lines of code).
-**Initial config**: Only the instruction "add user preferences endpoints under `/api/v2/users`."
+Initially they gave only one sentence: "add user preferences endpoints under `/api/v2/users`." The result? The agent spent 40% of its context window exploring the repo structure, produced code that looked reasonable but didn't follow the project's error handling patterns, used old SQLAlchemy syntax, and declared completion while the endpoint had runtime errors. The next session had to redo all the discovery work.
-**Result**: The agent spent 40% of its context window exploring the repo structure, produced code that looked reasonable but didn't follow the project's error handling patterns, used old SQLAlchemy syntax, and declared completion while the endpoint had runtime errors. The next session had to redo all the discovery work.
+Later they added `AGENTS.md` (describing project architecture and tech stack versions), explicit verification commands (`pytest tests/api/v2/ && python -m mypy src/`), and architecture decision records. The same model succeeded in all three independent runs, with ~60% better context efficiency.
-**After improvement**: The team added `AGENTS.md` (describing project architecture and tech stack versions), explicit verification commands (`pytest tests/api/v2/ && python -m mypy src/`), and architecture decision records. The same model succeeded in all three independent runs, with ~60% better context efficiency.
+They didn't change the model. They changed the harness.
## Key Takeaways
-- Model capability and execution reliability are different things. The strongest model in a terrible harness still crashes.
-- When things fail, check the harness first, then the model. Swapping models is the most expensive option.
+- Model capability and execution reliability are different things. A thoroughbred still needs a good saddle.
+- When things fail, check the harness first, then the model. Swapping models is the most expensive option — and often it's not even a model problem.
- Every failure is a signal: your harness has a structural defect. Find it, fix it.
-- Five defense layers: task specification, context provision, execution environment, verification feedback, state management. Check them systematically.
-- One `AGENTS.md` file might be more effective than upgrading to a more expensive model.
+- Five defense layers: task specification, context provision, execution environment, verification feedback, state management. Check them systematically, like a doctor ruling out the most common causes first.
+- One `AGENTS.md` file might be more effective than upgrading to a more expensive model. Seriously.
## Further Reading
diff --git a/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md b/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md
index 268994a..bbeaebf 100644
--- a/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md
+++ b/docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md
@@ -5,22 +5,39 @@
# Lecture 02. What Harness Actually Means
-## What Problem Does This Lecture Solve?
+The word "harness" gets thrown around a lot in AI coding agent circles, but honestly, most people mean "a prompt file" when they say harness. That's not a harness. It's like opening a restaurant with nothing but ingredients — no stove, no knives, no recipes, no plating workflow. That's not a restaurant. That's a refrigerator.
-The word "harness" gets thrown around a lot in AI coding agent discussions, but honestly, most people mean "a prompt file" when they say harness. That's not a harness.
+This lecture gives you a precise, actionable harness definition. Not an academic abstraction, but a framework you can use today: a harness consists of five subsystems, each with clear responsibilities and evaluation criteria.
-This lecture gives you a precise, actionable definition. Not an academic abstraction, but a framework you can use today: a harness consists of five subsystems, each with clear responsibilities and evaluation criteria. Once you understand this framework, you can systematically diagnose "why did the agent fail again" — not by vaguely blaming the model, but by pinpointing exactly which subsystem broke down.
+## Start with an Analogy
+
+Imagine you're a newly hired engineer dropped into a project with zero documentation. No README, no comments in the code, nobody tells you how to run tests, CI config is buried somewhere. Can you write good code? Maybe — if you're smart enough and patient enough. But you'll spend enormous time on "figuring out what this project is about" rather than "solving the problem."
+
+An AI agent faces the exact same situation. And it's worse — you can at least ask a colleague. The agent can only see files you put in front of it and commands it can execute. It can't tap someone on the shoulder and ask "hey, which version of the ORM does this project use?"
+
+OpenAI frames the core principle as "the repo IS the spec" — all necessary context should be in the repository, delivered through structured instruction files, explicit verification commands, and clear directory organization. Anthropic's long-running agents documentation emphasizes state persistence, explicit recovery paths, and structured progress tracking. The two companies focus on different aspects, but they're saying the same thing: **everything in the engineering infrastructure outside the model determines how much of the model's capability actually gets realized.**
+
+Look at some tools you already know:
+
+**Claude Code** embodies harness thinking. It reads `CLAUDE.md` from your repo (recipe shelf), can run shell commands (knife rack), executes in your local environment (stove), maintains session history (prep station), and can run tests and see results (quality check window). But if you don't tell it how to run tests, the quality check window is broken — nobody knows whether the dish is fully cooked.
+
+**Cursor** follows similar logic. Its `.cursorrules` file is the recipe shelf, the terminal is the knife rack, it reads your project structure and lint config for the stove. But Cursor's state management is relatively weak — close the IDE and reopen it, and the previous context is gone.
+
+**Codex** (OpenAI's coding agent) uses git worktrees to isolate each task's runtime environment, paired with a local observability stack (logs, metrics, traces), so every change is verified in an independent environment. In repos with `AGENTS.md` and clear verification commands, it performs far better than in "bare" repos.
+
+**AutoGPT** is the cautionary tale — lack of structured state management leads to context accumulation in long tasks, and lack of precise feedback mechanisms causes the agent to loop. Many people say AutoGPT "doesn't work," but really it's AutoGPT's harness that doesn't work — give a chef a broken stove and even the best ingredients won't produce a meal.
## Core Concepts
-- **The Five-Tuple Harness**: Instructions, Tools, Environment, State, Feedback. If it's not model weights, it's harness.
-- **Boundary Criterion**: If a component affects the agent's perception, actions, memory, or verification, it belongs to the harness. Simple rule: not model weights = harness.
-- **Weakest-Link Principle**: The harness's overall effectiveness is determined by its weakest subsystem. Your instructions might be brilliant, but without verification feedback, the agent still won't be reliable.
-- **Affordance**: Good harness design provides visible cues that suggest possible actions. Clear directory structure implies code organization, explicit verification commands imply completion criteria. Harness design IS affordance design.
-- **Feedback Latency**: The time between an agent's action and the feedback it receives. Compilation errors are second-level feedback, test suites are minute-level, "bug in production" is day-level. Faster feedback means faster correction.
-- **Isometric Model Control**: Keep the model fixed, systematically vary harness subsystems, measure each one's marginal contribution. This is the correct way to quantify a harness's value.
+- **What is a harness**: Everything in the engineering infrastructure outside the model weights. OpenAI distills the engineer's core job into three things: designing environments, expressing intent, and building feedback loops. Anthropic calls their Claude Agent SDK a "general-purpose agent harness."
+- **The repo is the single source of truth**: Anything the agent can't see, for all practical purposes, doesn't exist. OpenAI treats the repo as the "system of record" — all necessary context must live there, through structured files and clear directory organization.
+- **Give a map, not a manual**: OpenAI's experience — `AGENTS.md` should be a directory page, not an encyclopedia. Around 100 lines is enough. If it doesn't fit, split it into the `docs/` directory and let the agent read on demand.
+- **Constrain, don't micromanage**: A good harness uses executable rules to constrain the agent, rather than enumerating instructions one by one. OpenAI says "enforce invariants, don't micromanage implementation"; Anthropic found that agents confidently praise their own work, and the solution is to separate "the person who does the work" from "the person who checks the work."
+- **Remove components one at a time**: To quantify the value of each harness component, remove them one at a time and see which removal causes the biggest performance drop. Anthropic used this method and found that as models get stronger, some components stop being critical — but new ones always emerge.
-## The Five-Tuple Harness Model
+## The Five-Subsystem Harness Model
+
+Back to the kitchen analogy. A complete kitchen has five functional areas, and a harness has five subsystems:
```mermaid
flowchart LR
@@ -32,44 +49,15 @@ flowchart LR
Checks --> Agent
```
-## Why This Happens
-
-Here's an analogy. Imagine you're a newly hired engineer dropped into a project with zero documentation. No README, no comments in the code, nobody tells you how to run tests, CI config is buried somewhere. Can you write good code? Maybe, but you'll spend enormous time on "figuring out what this project is about" rather than "solving the problem."
-
-An AI agent faces the exact same situation. And it's worse — you can at least ask a colleague. The agent can only see files you put in front of it and commands it can execute.
+**Instruction subsystem (recipe shelf)**: Create `AGENTS.md` (or `CLAUDE.md`) containing a project overview and purpose (one sentence), tech stack and versions (Python 3.11, FastAPI 0.100+, PostgreSQL 15), first-run commands (`make setup`, `make test`), non-negotiable hard constraints ("All APIs must use OAuth 2.0"), and links to more detailed documentation.
-OpenAI's harness engineering article frames the core principle as "the repo IS the spec" — all necessary context should be in the repository, delivered through structured instruction files, explicit verification commands, and clear directory organization. Anthropic's long-running agents documentation emphasizes state persistence, explicit recovery paths, and structured progress tracking. The two companies focus on different aspects, but they're saying the same thing: **everything in the engineering infrastructure outside the model determines how much of the model's capability actually gets realized.**
+**Tool subsystem (knife rack)**: Ensure the agent has sufficient tool access. Don't disable shell for "security" — if the agent can't even run `pip install`, how is it supposed to work? But don't open everything either — follow least-privilege principles.
-Look at some concrete examples:
+**Environment subsystem (stove)**: Make the environment state self-describing. Use `pyproject.toml` or `package.json` to lock dependencies, `.nvmrc` or `.python-version` for runtime versions, Docker or devcontainers for reproducibility.
-**Claude Code** embodies harness thinking. It reads `CLAUDE.md` from your repo (instruction subsystem), can run shell commands (tool subsystem), executes in your local environment (environment subsystem), maintains session history (state subsystem), and can run tests and see results (feedback subsystem). But if you don't tell it how to run tests, the feedback subsystem is broken.
+**State subsystem (prep station)**: Long tasks need progress tracking. Use a simple `PROGRESS.md` file recording: what's done, what's in progress, what's blocked. Update before each session ends, read when the next session starts.
-**Cursor** follows similar logic. Its `.cursorrules` file is the instruction subsystem, the terminal is the tool subsystem, it reads your project structure and lint config for the environment subsystem. But Cursor's state management is relatively weak — close the IDE and reopen it, and the previous context is gone.
-
-**Codex** (OpenAI's coding agent) uses git worktrees to isolate each task's runtime environment, paired with a local observability stack (logs, metrics, traces), so every change is verified in an independent environment. In repos with `AGENTS.md` and clear verification commands, it performs far better than in "bare" repos.
-
-**AutoGPT** is the cautionary tale — lack of structured state management leads to context accumulation in long tasks, and lack of precise feedback mechanisms causes the agent to loop. Many people say AutoGPT "doesn't work," but really it's AutoGPT's harness that doesn't work.
-
-The key insight: the same model in different harnesses can show order-of-magnitude performance differences. Anthropic's controlled experiment proves this directly: same prompt, same model (Opus 4.5), bare run 20min/$9 with broken output vs. full harness 6hr/$200 with working application.
-
-## How to Do It Right
-
-A harness has five subsystems. You need to make sure none of them drags the rest down.
-
-**Instruction Subsystem (I)**: Create `AGENTS.md` (or `CLAUDE.md`) containing:
-- Project overview and purpose (one sentence)
-- Tech stack and versions (Python 3.11, FastAPI 0.100+, PostgreSQL 15)
-- First-run commands (`make setup`, `make test`)
-- Non-negotiable hard constraints ("All APIs must use OAuth 2.0")
-- Links to more detailed documentation
-
-**Tool Subsystem (T)**: Ensure the agent has sufficient tool access. Don't disable shell access for "security" — if the agent can't even run `pip install`, how is it supposed to work? But don't open everything either — follow least-privilege principles.
-
-**Environment Subsystem (E)**: Make the environment state self-describing. Use `pyproject.toml` or `package.json` to lock dependencies, `.nvmrc` or `.python-version` for runtime versions, Docker or devcontainers for reproducibility.
-
-**State Subsystem (S)**: Long tasks need progress tracking. Use a simple `PROGRESS.md` file recording: what's done, what's in progress, what's blocked. Update before each session ends, read when the next session starts.
-
-**Feedback Subsystem (F)**: This is the highest-ROI subsystem. Explicitly list verification commands in `AGENTS.md`:
+**Feedback subsystem (quality check window)**: This is the highest-ROI subsystem. Explicitly list verification commands in `AGENTS.md`:
```
Verification commands:
- Tests: pytest tests/ -x
@@ -78,27 +66,29 @@ Verification commands:
- Full verification: make check (includes all above)
```
-**Diagnosing harness quality**: Use "isometric model control." Keep the model fixed, remove subsystems one at a time, measure which removal causes the biggest performance drop. That's your bottleneck — focus your effort there.
+Missing any subsystem is like missing a functional area in the kitchen — you can still cook, but it's always awkward.
+
+**Diagnosing harness quality**: Use "isometric model control." Keep the model fixed, remove subsystems one at a time, measure which removal causes the biggest performance drop. That's your bottleneck — focus your effort there. Like finding the bottleneck in a kitchen: take away the recipe shelf and see how much slower things get, shut off the stove and see the impact.
-## Real-World Example
+## A Team's Real Story
-A team using GPT-4o on a TypeScript + React frontend app (~20,000 lines of code) went through four stages:
+A team used GPT-4o on a TypeScript + React frontend app (~20,000 lines of code). They went through four stages — essentially adding kitchen equipment one piece at a time:
-**Stage 1 — Minimal harness**: Only a basic project description in README. 1 out of 5 runs succeeded (20%). Main failures: chose wrong package manager (npm vs yarn), didn't follow component naming conventions, couldn't run tests.
+**Stage 1 — Empty kitchen**: Only a basic project description in README. 1 out of 5 runs succeeded (20%). Main failures: chose wrong package manager (npm vs yarn), didn't follow component naming conventions, couldn't run tests.
-**Stage 2 — Added instruction subsystem**: Created `AGENTS.md` with tech stack versions, naming conventions, key architecture decisions. Success rate rose to 60%. Remaining failures were mainly environment issues and missing verification.
+**Stage 2 — Recipe shelf installed**: Added `AGENTS.md` with tech stack versions, naming conventions, key architecture decisions. Success rate rose to 60%. Remaining failures were mainly environment issues and missing verification.
-**Stage 3 — Added feedback subsystem**: Listed verification commands in `AGENTS.md`: `yarn test && yarn lint && yarn build`. Success rate rose to 80%.
+**Stage 3 — Quality check window opened**: Listed verification commands in `AGENTS.md`: `yarn test && yarn lint && yarn build`. Success rate rose to 80%.
-**Stage 4 — Added state subsystem**: Introduced progress file templates where agents recorded completed and incomplete work each run. Success rate stabilized at 80-100%.
+**Stage 4 — Prep station ready**: Introduced progress file templates where agents recorded completed and incomplete work each run. Success rate stabilized at 80-100%.
-Four iterations, the model didn't change at all, success rate went from 20% to near 100%. That's the power of harness engineering.
+Four iterations, the model didn't change at all, success rate went from 20% to near 100%. That's the power of harness engineering. You didn't buy more expensive ingredients — you just organized the kitchen properly.
## Key Takeaways
-- Harness = Instructions + Tools + Environment + State + Feedback. Five subsystems, no skipping.
+- Harness = Instructions + Tools + Environment + State + Feedback. Five subsystems, like a kitchen's five functional areas — all essential.
- If it's not model weights, it's harness. Your harness determines how much model capability gets realized.
-- Among the five subsystems, the feedback subsystem usually has the lowest investment and highest return. Get your verification commands right first.
+- Among the five subsystems, the feedback subsystem usually has the lowest investment and highest return. Get your verification commands right first — the quality check window is the most worthwhile upgrade.
- Use "isometric model control" to quantify each subsystem's marginal contribution — don't go by gut feeling.
- Harness rots like code does. Audit regularly, pay down harness debt like you pay down technical debt.
diff --git a/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md b/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md
index c4080f6..b59d885 100644
--- a/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md
+++ b/docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md
@@ -5,20 +5,21 @@
# Lecture 03. Make the Repository Your Single Source of Truth
-## What Problem Does This Lecture Solve?
+Your team's architecture decisions are scattered across Confluence, Slack, Jira, and a few senior engineers' heads. For humans this barely works — you can ask a colleague, search chat history, dig through docs. If all else fails, you can corner someone in the break room. But for an AI agent, information that's not in the repository simply does not exist.
-Your team's architecture decisions are scattered across Confluence, Slack, Jira, and a few senior engineers' heads. For humans this barely works — you can ask a colleague, search chat history, dig through docs. But for an AI agent, information that's not in the repository simply does not exist.
+This isn't an exaggeration. Think about what an agent's inputs actually are: system prompts and task descriptions, file contents from the repository, and tool execution output. That's it. Your Slack history, Jira tickets, Confluence pages, and that architecture decision you discussed with a colleague over coffee on Friday afternoon — the agent can't see any of it. It can't "go ask someone" or "search the chat history." It's an engineer locked inside the repository — everything outside, it knows nothing about.
-This lecture explains why you must put everything an agent needs to know into the repo, and how to do it without turning into "writing a bunch of docs nobody reads." The core idea isn't "write more documentation" — it's "put critical decision information where the agent can see it."
+So the question becomes: are you going to give this engineer a good map?
-## Core Concepts
+## What Belongs on the Map
+
+OpenAI states this bluntly: **information that doesn't exist in the repo, doesn't exist for the agent.** They call this the "repo as spec" principle — the repository itself is the highest-authority specification document.
+
+Anthropic's long-running agents documentation echoes this: persistent state is a necessary condition for long-task continuity. Cross-session knowledge recoverability directly determines task success rates. And this state must exist in the repository — because that's the only stable, accessible storage the agent has.
-- **Knowledge Visibility Gap**: The proportion of total project knowledge that's NOT in the repository. The bigger the gap, the higher the agent's failure rate.
-- **Single Source of Truth**: The code repository as the authoritative source for project decisions, architecture constraints, execution state, and verification standards. The repo is the final word.
-- **Cold-Start Test**: Open a brand new agent session using only repo contents — can it answer five basic questions: What is this system? How is it organized? How do I run it? How do I verify it? Where are we now?
-- **Discovery Cost**: How much context budget the agent burns to find a key piece of information in the repo. Information hidden in obscure locations has high discovery cost, leaving less budget for the actual task.
-- **Knowledge Decay Rate**: The proportion of knowledge entries that become stale per unit of time. Documentation going out of sync with code is the biggest enemy.
-- **ACID for Agent State**: Applying database transaction principles (Atomicity, Consistency, Isolation, Durability) to agent state management.
+You might think: "Our team is small, knowledge is in everyone's heads, and it works fine." Sure, for humans. But if you're using an agent, accept this fact: the agent can't ask people. Everything it needs to know must be written down and placed where it can find it.
+
+This isn't about "writing more documentation." It's about "putting decision information in the right place." A 50-line `ARCHITECTURE.md` in the `src/api/` directory is ten thousand times more useful than a 500-page design document in Confluence that nobody maintains. It's like a hand-drawn office map taped to your desk versus a beautiful architectural blueprint locked in a filing cabinet — the former is right there when you need it; the latter is technically superior but useless in the moment.
## Knowledge Visibility
@@ -33,6 +34,8 @@ flowchart LR
Warning["If a rule is not in the repo,<br/>the agent cannot see it"] --> Agent
```
+How do you test whether your map is good enough? Run a "cold-start test": open a brand new agent session using only repo contents, and see if it can answer five basic questions:
+
```mermaid
flowchart TB
Q1["What is this system?"] --> A1["AGENTS.md / README"]
@@ -48,29 +51,24 @@ flowchart TB
A5 --> Ready
```
-## Why This Happens
-
-Think about what an agent's inputs actually are: system prompts and task descriptions, file contents from the repository, and tool execution output. That's it. Your Slack history, Jira tickets, Confluence pages, and that architecture decision you discussed with a colleague over coffee on Friday afternoon — the agent can't see any of it.
+If it can't answer, the map has blank spots. Where the map is blank, the agent guesses — wrong guesses become bugs, excessive guessing wastes context. And every new session guesses all over again. The cost of guessing is always higher than the cost of drawing the map properly in the first place.
-This isn't a bug — it's the agent's cognitive architecture. It can't "go ask someone" or "search the chat history" like a human would.
-
-This creates a practical problem: when critical project knowledge only exists outside the repo, the agent either does it wrong (because it doesn't know an implicit rule) or wastes enormous context re-discovering that knowledge. And every new session has to redo this discovery.
-
-OpenAI's harness engineering article states this bluntly: **information that doesn't exist in the repo, doesn't exist for the agent.** They call this the "repo as spec" principle — the repository itself is the highest-authority specification document.
-
-Anthropic's long-running agents documentation echoes this: persistent state is a necessary condition for long-task continuity. Cross-session knowledge recoverability directly determines task success rates. And this state must exist in the repository — because that's the only stable, accessible storage the agent has.
-
-You might think: "Our team is small, knowledge is in everyone's heads, and it works fine." Sure, for humans. But if you're using an agent, accept this fact: the agent can't ask people. Everything it needs to know must be written down and placed where it can find it.
+## Core Concepts
-This isn't about "writing more documentation." It's about "putting decision information in the right place." A 50-line `ARCHITECTURE.md` in the `src/api/` directory is ten thousand times more useful than a 500-page design document in Confluence that nobody maintains.
+- **Knowledge Visibility Gap**: The proportion of total project knowledge that's NOT in the repository. The bigger the gap, the higher the agent's failure rate. How much implicit knowledge about this project lives in your head? Count it all, then see how much made it into the repo — the difference is your visibility gap.
+- **System of Record**: The code repository as the authoritative source for project decisions, architecture constraints, execution state, and verification standards. The repo has the final word, nowhere else counts. Like a map that marks "road closed" — you won't go down that road. But if that information only exists in Old Zhang's head, you have to ask Old Zhang every time.
+- **Cold-Start Test**: The five questions above. How many it can answer is how complete your map is.
+- **Discovery Cost**: How much context budget the agent burns to find a key piece of information in the repo. The more hidden the information, the higher the discovery cost, and the less budget left for the actual task. Hiding critical information in a README ten directory levels deep is like locking the fire extinguisher in a basement safe — it exists, but you can't find it when you need it.
+- **Knowledge Decay Rate**: The proportion of knowledge entries that become stale per unit of time. Documentation going out of sync with code is the biggest enemy — worse than no documentation at all.
+- **ACID Analogy**: Applying database transaction principles (Atomicity, Consistency, Isolation, Durability) to agent state management. We'll expand on this below.
-## How to Do It Right
+## How to Draw a Good Map
-**Principle 1: Knowledge lives next to code.** A rule about API endpoint authentication belongs next to the API code, not buried in a giant global document. Put a short doc in each module directory explaining that module's responsibilities, interfaces, and special constraints.
+**Principle 1: Knowledge lives next to code.** A rule about API endpoint authentication belongs next to the API code, not buried in a giant global document. Put a short doc in each module directory explaining that module's responsibilities, interfaces, and special constraints. Like library shelf labels — you want history books, go straight to the shelf marked "History." No need to search the entire library.
**Principle 2: Use a standardized entry file.** `AGENTS.md` (or `CLAUDE.md`) is the agent's "landing page." It doesn't need to contain all information, but it must let the agent quickly answer three questions: "What is this project," "How do I run it," and "How do I verify it." 50-100 lines is enough.
-**Principle 3: Minimal but complete.** Every piece of knowledge should have a clear use case. If removing a rule doesn't affect the agent's decision quality, that rule shouldn't exist. But every question from the cold-start test must have an answer.
+**Principle 3: Minimal but complete.** Every piece of knowledge should have a clear use case. If removing a rule doesn't affect the agent's decision quality, that rule shouldn't exist. But every question from the cold-start test must have an answer. This is a delicate balance — not too much, not too little, just enough.
**Principle 4: Update with code.** Bind knowledge updates to code changes. The simplest approach: put architecture docs in the corresponding module directory. When you modify code, you naturally see the doc. After code changes, CI can remind you to check if docs need updating.
@@ -91,18 +89,20 @@ project/
└── Makefile # Standardized commands: setup, test, lint, check
```
-**Managing agent state with ACID principles**:
+## Managing Agent State with ACID Principles
+
+This analogy comes from database transaction management — you might think it's overcomplicating things, but it actually gives you a very practical framework:
-- **Atomicity**: Each "logical operation" (e.g., "add new endpoint and update tests") gets one git commit. If it fails midway, `git stash` to roll back.
-- **Consistency**: Define "consistent state" verification predicates — all tests pass, lint reports zero errors. The agent runs verification after each operation; inconsistent intermediate states don't get committed.
-- **Isolation**: When multiple agents work concurrently, design state files to avoid race conditions. Simple approach: each agent uses its own progress file, or use git branches for isolation.
-- **Durability**: Critical project knowledge lives in git-tracked files. Temporary state can stay in session memory, but cross-session knowledge must be persisted to files.
+- **Atomicity**: Each "logical operation" (e.g., "add new endpoint and update tests") gets one git commit. If it fails midway, `git stash` to roll back. All or nothing — no "half done."
+- **Consistency**: Define "consistent state" verification predicates — all tests pass, lint reports zero errors. The agent runs verification after each operation; inconsistent intermediate states don't get committed. Like a bank transfer — you can't debit without crediting.
+- **Isolation**: When multiple agents work concurrently, design state files to avoid race conditions. Simple approach: each agent uses its own progress file, or use git branches for isolation. Two chefs can't season the same pot simultaneously — who takes responsibility when it's over-salted?
+- **Durability**: Critical project knowledge lives in git-tracked files. Temporary state can stay in session memory, but cross-session knowledge must be persisted to files. What's in your head doesn't count — only what's on paper counts.
-## Real-World Example
+## A Real Transformation Story
A team maintained an e-commerce platform with ~30 microservices. Architecture decisions (inter-service communication protocols, data consistency strategies, API versioning rules) were scattered across: Confluence (partially outdated), Slack (hard to search), a few senior engineers' heads (not scalable), and sporadic code comments (not systematic).
-After introducing AI agents, 70% of tasks required human intervention. Nearly every failure involved the agent violating some "everyone knows but nobody wrote down" implicit constraint.
+After introducing AI agents, 70% of tasks required human intervention. Nearly every failure involved the agent violating some "everyone knows but nobody wrote down" implicit constraint. It's like a new employee whom nobody told "you need to post your lunch order in the group chat" — they guess wrong, get scolded, but after the scolding still nobody tells them the rule.
The team executed a transformation:
1. Created `AGENTS.md` in the repo root with project overview, tech stack versions, and global hard constraints
@@ -110,15 +110,15 @@ The team executed a transformation:
3. Created a centralized `CONSTRAINTS.md` with hard constraints in explicit "MUST/MUST NOT" language
4. Added `PROGRESS.md` in each service directory tracking current work status
-After transformation: the same agent could answer all key project questions on cold start, and task completion quality improved significantly (Anthropic's controlled experiment showed that a full harness enables agents to deliver runnable applications, while bare runs cannot).
+After transformation: the same agent could answer all key project questions on cold start, and task completion quality improved significantly.
## Key Takeaways
-- Knowledge not in the repo doesn't exist for the agent. Putting critical decisions in the repo is the most basic harness investment.
+- Knowledge not in the repo doesn't exist for the agent. Putting critical decisions in the repo is the most basic harness investment — draw a good map so you don't get lost.
- Use the "cold-start test" to evaluate repo quality: can a fresh session answer five basic questions using only repo contents?
- Knowledge should be near code, minimal but complete, and updated with code. It's not about writing more docs — it's about putting information in the right place.
- Use ACID principles for agent state: atomic commits, consistency verification, concurrency isolation, durable critical knowledge.
-- Knowledge decay is the biggest enemy. Documentation out of sync with code is more dangerous than no documentation.
+- Knowledge decay is the biggest enemy. Documentation out of sync with code is more dangerous than no documentation — it sends the agent in the wrong direction while they think they're right.
## Further Reading
diff --git a/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md b/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md
index d00a8fe..88fa271 100644
--- a/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md
+++ b/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md
@@ -3,13 +3,27 @@
> Code examples: [code/](https://github.com/walkinglabs/learn-harness-engineering/blob/main/docs/en/lectures/lecture-04-why-one-giant-instruction-file-fails/code/)
> Practice project: [Project 02. Agent-readable workspace](./../../projects/project-02-agent-readable-workspace/index.md)
-# Lecture 04. Split Instructions Across Files, Not One Giant File
+# Lecture 04. Split Instructions Across Files
-## What Problem Does This Lecture Solve?
+You got serious about harness engineering — good for you. You created an `AGENTS.md` and packed every rule, constraint, and lesson learned you could think of into it. One month later the file bloated to 300 lines, two months 450 lines, three months 600 lines. Then you notice the agent's performance is actually getting worse — on a simple bug fix, the agent burns tons of context processing irrelevant deployment instructions; a critical security constraint buried at line 300 gets ignored outright; three contradictory code style rules mean the agent picks one at random each time.
-You got serious about harness engineering, created an `AGENTS.md`, and packed every rule, constraint, and lesson learned you could think of into it. Three months later it's bloated to 600 lines. And now the agent performs worse — on simple bug fixes it spends tons of context processing irrelevant deployment instructions; critical security constraints buried at line 300 get ignored outright; three contradictory code style rules means the agent picks one at random each time.
+This is the "giant instruction file" trap. It's like overpacking a suitcase — everything seems useful, so you cram it all in until the zipper is about to burst. Finding your change of underwear means emptying the entire bag. You carried a full suitcase, but you actually used maybe a third of what's inside.
-This is the "giant instruction file" trap. This lecture explains why "more information" does NOT equal "better performance," and how to split instructions into a structured file network.
+## The Vicious Cycle at the Root
+
+The most common vicious cycle goes like this: agent makes a mistake, you say "add a rule to prevent this," add it to AGENTS.md, it works temporarily, agent makes a different mistake, add another rule, repeat, file bloats out of control.
+
+This isn't your fault. It's a very natural reaction — "add a rule" each time something goes wrong feels reasonable, like tossing one more thing into your bag every time you leave the house "just in case." But the cumulative effect is disastrous. Let's look at what goes wrong specifically.
+
+**Context budget gets eaten alive.** The agent's context window is finite. Say your agent has a 200K token window (Claude's standard). A bloated instruction file might eat 10-20K tokens. Seems like there's still plenty of room? But a complex task might need to read dozens of source files, tool execution output also takes context, and conversation history accumulates. By the time the agent needs to understand the code, the budget is already tight — like a suitcase so full of "just in case" items that there's no room for your laptop.
+
+**Lost in the middle.** The "Lost in the Middle" paper (Liu et al., 2023) clearly demonstrated that LLMs utilize information in the middle of long texts significantly less effectively than at the beginning or end. Your AGENTS.md is 600 lines, and line 300 says "all database queries must use parameterized queries" — that's a security hard constraint. But it's buried in the middle, and the agent will almost certainly ignore it. Like that bottle of sunscreen at the bottom of your overstuffed suitcase — you know it's there, you dig three times, can't find it, end up buying another one.
+
+**Priority conflicts.** The file mixes non-negotiable hard constraints ("never use eval()"), important design guidelines ("prefer functional style"), and a specific historical lesson ("fixed a WebSocket memory leak last week, watch for similar patterns"). These three rules have completely different importance levels, but they look identical in the file. The agent has no reliable signal to distinguish — like your passport and charging cable jumbled together in the suitcase, no way to tell which is more urgent.
+
+**Maintenance decay.** Large files are inherently hard to maintain. Outdated instructions rarely get deleted — because the consequences of deletion are uncertain ("maybe something else depends on this rule?"), while adding new instructions feels free. The result: the file only grows, never shrinks, and signal-to-noise ratio continuously declines. This is exactly like technical debt accumulation in software.
+
+**Contradiction accumulation.** Instructions added at different times start contradicting each other — one says "use TypeScript strict mode," another says "some legacy files allow any types." The agent randomly picks one to follow each time. Like your mom saying "dress warm" and your dad saying "don't wear too much," and you standing at the door not knowing who to listen to.
## Core Concepts
@@ -41,33 +55,11 @@ flowchart TB
Mid --> Missed["High chance of being diluted or missed"]
```
-## Why This Happens
-
-The most common vicious cycle goes like this: the agent makes a mistake → you say "add a rule to prevent this" → add it to AGENTS.md → it works temporarily → agent makes a different mistake → add another rule → repeat → file bloats out of control.
-
-This isn't your fault. It's a very natural reaction — "add a rule" each time something goes wrong feels reasonable. But the cumulative effect is disastrous. Let's look at what goes wrong specifically:
-
-**Context budget crowding.** The agent's context window is finite. Say your agent has a 200K token window (standard for Claude). A bloated instruction file might eat 10-20K tokens. Seems like there's still plenty of room? But a complex task might need to read dozens of source files, tool execution output also takes context, and conversation history accumulates. By the time the agent needs to understand the code, the budget is already tight.
-
-**Lost in the middle.** The "Lost in the Middle" paper (Liu et al., 2023) clearly demonstrated that LLMs utilize information in the middle of long texts significantly less effectively than at the beginning or end. Your AGENTS.md is 600 lines, and line 15 says "all database queries must use parameterized queries" — that's a security hard constraint. But line 15 in a 600-line file is actually near the top. What if that rule was at line 300? The agent would almost certainly ignore it.
-
-**Priority conflicts.** The file mixes non-negotiable hard constraints ("never use eval()"), important design guidelines ("prefer functional style"), and a specific historical lesson ("fixed a WebSocket memory leak last week, watch for similar patterns"). These three rules have completely different importance levels, but they look identical in the file. The agent has no reliable signal to distinguish their relative importance.
-
-**Maintenance decay.** Large files are inherently hard to maintain. Outdated instructions rarely get deleted — because the consequences of deletion are uncertain ("maybe something else depends on this rule?"), while adding new instructions is free. The result: the file only grows, never shrinks, and SNR continuously declines. This is exactly like technical debt accumulation in software.
-
-**Contradiction accumulation.** Instructions added at different times start contradicting each other — one says "use TypeScript strict mode," another says "some legacy files allow any types." The agent randomly picks one to follow each time.
-
-Both OpenAI and Anthropic implicitly support the splitting approach. OpenAI says entry files should be "short and routing-oriented," Anthropic says long-running agent control information should be "concise and high-priority." Both are saying the same thing: don't stuff everything into one file.
-
-## How to Do It Right
+## How to Split
-Split the giant file into a three-layer structure:
+Core principle: keep frequently-needed information at hand, tuck away occasionally-needed information, and leave behind what you'll never use.
-**Layer 1: Routing file** (`AGENTS.md`, 50-200 lines). Only include:
-- Project overview (one or two sentences)
-- First-run commands (`make setup && make test`)
-- Global hard constraints (no more than 15 non-negotiable rules)
-- Links to topic documents (one-line description + applicability condition)
+The entry file `AGENTS.md` stays at 50-200 lines, containing only the most frequently used items — project overview (one or two sentences), first-run commands (`make setup && make test`), global hard constraints (no more than 15 non-negotiable rules), and links to topic documents (one-line description + applicability condition).
```markdown
# AGENTS.md
@@ -91,36 +83,33 @@ Python 3.11 FastAPI backend, PostgreSQL 15 database.
- [Testing Standards](docs/testing-standards.md) — Reference when writing tests
```
-**Layer 2: Topic documents** (50-150 lines each, organized by subject). Place them in `docs/` or next to the corresponding module directory. Each document covers one specific topic and is only read when needed.
+Each topic document is 50-150 lines, organized by subject in the `docs/` directory or next to the corresponding module. The agent only reads them when needed. Like packing cubes in a suitcase — underwear in one cube, toiletries in another, chargers in a third. Finding things doesn't require emptying the whole bag.
-**Layer 3: Inline in code.** Some information is better placed directly in the code — type definitions, interface comments, explanations in config files. The agent naturally sees these when reading code.
+Some information is better placed directly in the code — type definitions, interface comments, explanations in config files. The agent naturally sees these when reading code, no need to duplicate in instructions.
-**Pruning rules**: Every instruction should have:
-- A clear source: "Why was this rule added?"
-- A clear applicability condition: "When is this rule needed?"
-- A clear expiry condition: "Under what circumstances can this rule be removed?"
+Every instruction should have a source ("why was this rule added?"), an applicability condition ("when is this rule needed?"), and an expiry condition ("under what circumstances can this rule be removed?"). Audit regularly, remove outdated, redundant, and contradictory entries. Manage your instructions like you manage code dependencies — unused dependencies should be deleted, otherwise they just slow the system down.
-Audit regularly, remove outdated, redundant, and contradictory entries. Manage your instructions like you manage code dependencies.
+If an instruction must be in the entry file, put it at the top or bottom — never the middle. The "lost in the middle" effect tells us that LLMs use information at the extremes significantly better than in the center. But the better approach is to move instructions to topic documents for on-demand loading.
-**Placement matters**: If an instruction must be in the routing file, put it at the top or bottom — never the middle. This leverages the "lost in the middle" effect to ensure important information isn't missed. But the better approach is to move instructions to topic documents for on-demand loading.
+Both OpenAI and Anthropic implicitly support the splitting approach. OpenAI says entry files should be "short and routing-oriented," Anthropic says long-running agent control information should be "concise and high-priority." Both are saying the same thing: don't stuff everything into one file. A suitcase needs organizing, not just brute-force cramming.
## Real-World Example
-A SaaS team's `AGENTS.md` ballooned from 50 lines to 600. Contents mixed tech stack versions, coding standards, historical bug fix notes, API usage guides, deployment procedures, and team members' personal preferences.
+A SaaS team's `AGENTS.md` ballooned from 50 lines to 600. Contents mixed tech stack versions, coding standards, historical bug fix notes, API usage guides, deployment procedures, and team members' personal preferences — the entire suitcase bursting at the seams.
Agent performance started declining noticeably: during simple bug fixes the agent spent lots of context processing irrelevant deployment instructions; the security constraint "all database queries must use parameterized queries" was buried at line 300 and frequently ignored; three contradictory code style rules caused random agent behavior.
-The team executed a refactor:
+The team executed a "suitcase reorganization":
1. `AGENTS.md` trimmed to 80 lines: only project overview, run commands, and 15 global hard constraints
2. Created topic documents: `docs/api-patterns.md` (120 lines), `docs/database-rules.md` (60 lines), `docs/testing-standards.md` (80 lines)
3. Added topic document links in the routing file
4. Historical notes either converted to test cases or deleted
-After refactoring: same task set success rate went from 45% to 72%. Security constraint compliance went from 60% to 95% (moved from file middle to routing file top).
+After refactoring: same task set success rate went from 45% to 72%. Security constraint compliance went from 60% to 95% — because it moved from the file middle to the routing file top, no longer "lost in the middle."
## Key Takeaways
-- "Add a rule" is short-term pain relief, long-term poison. Before adding a rule, ask: would this be better in a topic document?
+- "Add a rule" is short-term pain relief, long-term poison. Before adding a rule, ask: would this be better in a topic document? Don't just keep cramming things into the suitcase.
- The entry file is a router, not an encyclopedia. 50-200 lines with overview, hard constraints, and links only.
- Leverage the "lost in the middle" effect: important info goes at the top or bottom; unimportant info moves to topic documents.
- Manage instruction bloat like technical debt. Regular audits, every instruction needs a source, applicability condition, and expiry condition.
diff --git a/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md b/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md
index 9086386..6095d01 100644
--- a/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md
+++ b/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md
@@ -5,23 +5,24 @@
# Lecture 05. Keep Context Alive Across Sessions
-## What Problem Does This Lecture Solve?
-
You ask Claude Code to implement a complete feature. It runs for 30 minutes, does most of the work, but context is running low. You start a new session to continue — and discover it doesn't remember what decisions were made last time, why option A was chosen over option B, which files were already modified, or what state the tests are in. It spends 15 minutes re-exploring the project, and might be inconsistent with the previous approach.
-This is one of the most painful problems with AI coding agents: cross-session context continuity. This lecture explains why agents "black out" during long tasks, and how to maintain continuity through structured state persistence.
+Imagine if you were a craftsman who forgot everything each morning upon waking. You'd have to reacquaint yourself with the entire construction site — which wall is half-built, why red bricks were chosen over blue ones, where the plumbing runs got to. Worse, you might tear out a window that was already installed yesterday, simply because you didn't remember it was done.
-## Core Concepts
+This is exactly the predicament AI coding agents face in cross-session tasks. This lecture explains why agents "black out" during long tasks, and how structured state persistence can make them like a craftsman who keeps a reliable daily journal — still amnesiac, but the journal remembers everything.
-- **Context Windows Are Finite**: No matter what window size is claimed (128K, 200K, 1M), long tasks will eventually exhaust it. After exhaustion, either compaction (losing information) or reset (new session) is required. Both lose something.
-- **Continuity Artifacts**: Persisted state files that let a new session unambiguously resume where the last one left off. The basic form: progress log + verification record + next actions.
-- **Rebuild Cost**: The time a new session needs to reach an executable state. Good harnesses can compress rebuild cost from 15 minutes to 3 minutes.
-- **Drift**: The gap between the agent's understanding and the actual state of the code repository. Every session boundary introduces drift; without control, it compounds.
-- **Compaction vs Reset**: Compaction summarizes context within the same session (keeps "what," may lose "why"); reset opens a new session rebuilding from persisted state (clean but depends on artifact completeness).
-- **Context Anxiety**: A phenomenon observed by Anthropic — agents exhibit premature convergence behavior when approaching perceived context limits, ending tasks early to avoid information loss. It's an irrational resource anxiety.
+## Context Windows: Not Infinite
+
+Context windows are finite. This isn't solvable by model upgrades — even if window sizes grow to 1M tokens, complex tasks will still exhaust them. Because agents aren't just generating code; they're understanding codebases, tracking their own decision history, processing tool output, and maintaining conversation context. All this information grows faster than window expansion.
+
+A deeper problem: information the agent produces isn't uniformly important. Intermediate reasoning steps contain the "why" of decisions — why option B was chosen over A, why this library instead of that one, why a particular optimization was skipped. The final output only contains the "what" — the code itself. Compaction strategies usually preserve the latter but lose the former. The next session sees the code but doesn't know why it's written that way, and might "optimize" away a deliberate design decision.
+
+Anthropic discovered something fascinating in their long-running agent research: when agents sense context is running low, they exhibit "premature convergence" behavior — rushing to finish current work, skipping verification steps, or choosing a simple solution over the optimal one. It's like realizing time is running out on an exam and quickly guessing on the remaining multiple-choice questions. Anthropic calls this "context anxiety."
## Session Continuity Flow
+Without continuity artifacts, every new session is a disaster:
+
```mermaid
flowchart LR
S1["Session 1<br/>feature is half done"] --> End1["Context is nearly full<br/>session ends"]
@@ -30,6 +31,8 @@ flowchart LR
Guess --> Drift["Work gets repeated<br/>and recovery is slow"]
```
+With continuity artifacts, new sessions can pick up quickly:
+
```mermaid
flowchart LR
Work["Session 1 work"] --> Progress["PROGRESS.md<br/>done / in progress / next step"]
@@ -45,31 +48,32 @@ flowchart LR
Rebuild --> Resume["New session picks up quickly"]
```
-## Why This Happens
-
-Context windows are finite. This isn't solvable by model upgrades — even if window sizes grow to 1M tokens, complex tasks will still exhaust them. Because agents aren't just generating code; they're understanding codebases, tracking their own decision history, processing tool output, and maintaining conversation context. All this information grows faster than window expansion.
-
-A deeper problem: information the agent produces isn't uniformly important. Intermediate reasoning steps contain the "why" of decisions — why option B was chosen over A, why this library instead of that one, why a particular optimization was skipped. The final output only contains the "what" — the code itself. Compaction strategies usually preserve the latter but lose the former. The next session sees the code but doesn't know why it's written that way, and might "optimize" away a deliberate design decision.
+## Core Concepts
-Anthropic discovered something fascinating in their long-running agent research: when agents sense context is running low, they exhibit "premature convergence" behavior — rushing to finish current work, skipping verification steps, or choosing a simple solution over the optimal one. It's like realizing time is running out on an exam and quickly guessing on the remaining multiple-choice questions. Anthropic calls this "context anxiety."
+- **Context windows are finite**: No matter what window size is claimed (128K, 200K, 1M), long tasks will eventually exhaust them. After exhaustion, either compaction (losing information) or reset (new session) is required. Both lose something.
+- **Continuity artifacts**: Persisted state files that let a new session unambiguously resume where the last one left off. The basic form: progress log + verification record + next actions. That craftsman's journal.
+- **Rebuild cost**: The time a new session needs to reach an executable state. Good harnesses can compress rebuild cost from 15 minutes to 3 minutes.
+- **Drift**: The gap between the agent's understanding and the actual state of the code repository. Every session boundary introduces drift; without control, it compounds.
+- **Context anxiety**: A phenomenon observed by Anthropic — agents exhibit premature convergence behavior when approaching perceived context limits, ending tasks early to avoid information loss. It's an irrational resource anxiety.
+- **Compaction vs reset**: Compaction summarizes context within the same session (keeps "what," may lose "why"); reset opens a new session rebuilding from persisted state (clean but depends on artifact completeness).
-Specifically, cross-session continuity loss manifests in several ways:
+## What Happens When Continuity Breaks
-**Decision context loss.** The previous session spent significant context budget analyzing three approaches and choosing option B. This session's agent doesn't know about that analysis and might re-decide based on incomplete information — potentially choosing option A.
+The previous session spent significant context budget analyzing three approaches and choosing option B. This session's agent doesn't know about that analysis and might re-decide based on incomplete information — potentially choosing option A. Like the amnesiac craftsman who doesn't remember why red bricks were chosen, looks at the blue ones today and thinks they're prettier, and tears down yesterday's wall to rebuild.
-**Duplicate work.** The agent isn't sure whether certain work was already completed and does it again. Worse — it does half of it, discovers a conflict with the existing implementation, and has to rework.
+Even worse is duplicate work. The agent isn't sure whether certain work was already completed and does it again. Or worse — does half of it, discovers a conflict with the existing implementation, and has to rework. On a construction site, two teams can't build the same wall simultaneously — but without progress records, the new crew has no idea someone is already working on it.
-**Direction drift.** Each new session has a slightly different understanding of the project goals. Over several sessions, the implementation direction may have drifted significantly from the original requirements.
+Over several sessions, the implementation direction may have silently drifted from the original requirements. Each new session has a slightly different understanding of the project goals. Like a game of telephone — after ten people pass the message, "pick me up a coffee" might become "buy me a coffee machine."
-**Verification gap amplification.** The previous session's verification results (which tests pass, which fail, why they fail) weren't recorded. The new session has to re-run all verification to understand the current state.
+There's also the verification gap. The previous session's verification results (which tests pass, which fail, why they fail) weren't recorded. The new session has to re-run all verification to understand the current state. Every session re-diagnoses from scratch, every time wasting precious context.
Both OpenAI and Anthropic emphasize structured state persistence in their documentation. OpenAI's harness engineering article treats the repository as an "operational record" — every operation's results should leave traceable evidence in the repo. Anthropic's long-running agents documentation specifically recommends "handoff files" — structured documents containing current state, known issues, and next actions.
-## How to Do It Right
+## A Journal for the Amnesiac Craftsman
Core approach: **Treat the agent like a brilliant engineer with amnesia.** Before it "clocks out," it must write down critical information so the next "shift" agent can pick up quickly.
-**Tool 1: Progress file (PROGRESS.md).** The most basic continuity artifact. Simple format:
+**Tool 1: Progress file (PROGRESS.md).** The most basic continuity artifact — the core of the journal:
```markdown
# Project Progress
@@ -97,7 +101,7 @@ Core approach: **Treat the agent like a brilliant engineer with amnesia.** Befor
3. Update API documentation
```
-**Tool 2: Decision log (DECISIONS.md).** Record important design decisions and reasons. No need for detailed design documents — just "what decision, why, when":
+**Tool 2: Decision log (DECISIONS.md).** Record important design decisions and reasons. No need for detailed design documents — just "what decision, why, when" — the memos in the journal:
```markdown
# Design Decisions
@@ -110,16 +114,16 @@ Core approach: **Treat the agent like a brilliant engineer with amnesia.** Befor
**Tool 3: Git commits as checkpoints.** Commit after completing each atomic unit of work. Commit messages should explain what was done and why. These are free, automatically versioned state snapshots.
-**Tool 4: init.sh or harness initialization flow.** Specify in `AGENTS.md`:
+**Tool 4: init.sh or harness initialization flow.** Specify in `AGENTS.md` the "clock-in" and "clock-out" routines:
```markdown
-## At session start
+## At session start (clock in)
1. Read PROGRESS.md for current state
2. Read DECISIONS.md for important decisions
3. Run make check to confirm repo is in consistent state
4. Continue from PROGRESS.md "Next Steps" section
-## Before session end
+## Before session end (clock out)
1. Update PROGRESS.md
2. Run make check to confirm consistent state
3. Commit all completed work
@@ -127,20 +131,34 @@ Core approach: **Treat the agent like a brilliant engineer with amnesia.** Befor
**Mixed strategy**: Not every task needs a context reset. Short tasks (under 30 minutes) can complete within one session. Long tasks (spanning sessions) must use progress files and decision logs for continuity. Decision criterion: if a task needs more than 60% of the window, start preparing handoff.
+### Deep Dive on Context Anxiety
+
+Anthropic's March 2026 research further revealed the specific manifestations of context anxiety: on Sonnet 4.5, when context approaches the window limit, the agent shows strong "premature convergence" behavior. It's like realizing time is almost up on an exam and quickly filling in random answers on the multiple choice.
+
+Two strategies address this:
+
+**Compaction**: Summarizing early conversation within the same session. Advantage: maintains continuity, the agent can see "what." Disadvantage: "why" is often lost in summaries — why option B was chosen over A, why a particular optimization was skipped. More critically, compaction doesn't eliminate context anxiety — the agent knows context was once large, and psychologically still tends to rush to closure.
+
+**Context reset**: Completely clearing context, opening a new session, rebuilding from persisted artifacts. Advantage: clean mental state — the new session has no "I'm running out of time" anxiety. Disadvantage: depends on the completeness of handoff artifacts. If the journal is missing critical information, the new session may waste time going in the wrong direction.
+
+Anthropic's actual data: for Sonnet 4.5, context anxiety is severe enough that compaction alone isn't sufficient — context reset becomes a critical component of harness design. But for Opus 4.5, this behavior is greatly diminished, and compaction can manage context without relying on resets. This means: **harness design needs specific understanding of the target model, not a one-size-fits-all template.**
+
+> Source: [Anthropic: Harness design for long-running application development](https://www.anthropic.com/engineering/harness-design-long-running-apps)
+
## Real-World Example
An agent was tasked with implementing a blog system with user authentication — 12 feature points, estimated 5 sessions needed.
-**Baseline without continuity artifacts**: Session 1 implemented the user model and basic routes. Session 2 started without the agent remembering the auth middleware's interface contract, spending ~15 minutes inferring the previous design intent. By session 3, accumulated drift caused the agent to start reimplementing already-completed features. By session 5, the repo contained lots of redundant code but the core auth feature still hadn't passed end-to-end tests. Only 7 of 12 feature points completed, 3 with hidden correctness issues.
+**Baseline without the journal**: Session 1 implemented the user model and basic routes. Session 2 started without the agent remembering the auth middleware's interface contract, spending ~15 minutes inferring the previous design intent. By session 3, accumulated drift caused the agent to start reimplementing already-completed features. By session 5, the repo contained lots of redundant code but the core auth feature still hadn't passed end-to-end tests. Only 7 of 12 feature points completed, 3 with hidden correctness issues. Like the craftsman who never writes in his journal — by day five, the construction site is chaos, some walls built twice, some that should have been built never started.
-**With structured artifacts**: Using progress files, decision logs, verification records, and git checkpoints. State report updated automatically at each session end. Session 2's rebuild cost dropped to ~3 minutes. By session 5, all 12 feature points completed and verified.
+**With the journal**: Using progress files, decision logs, verification records, and git checkpoints. State report updated automatically at each session end. Session 2's rebuild cost dropped to ~3 minutes. By session 5, all 12 feature points completed and verified.
-Quantitative comparison: rebuild time reduced ~78%, feature completion rate from 58% to 100%, hidden defect rate from 43% down to 8%.
+Quantitative comparison: rebuild time reduced ~78%, feature completion rate from 58% to 100%, hidden defect rate from 43% down to 8%. The craftsman is still amnesiac, but with the journal, each day starts from where yesterday stopped, not from zero.
## Key Takeaways
-- Context windows are a finite resource. Long tasks will span sessions, and sessions will lose information.
-- The solution isn't bigger windows — it's better state persistence. Progress files + decision logs + git checkpoints.
+- Context windows are a finite resource. Long tasks will span sessions, and sessions will lose information — like the craftsman who forgets each day, this is objective reality.
+- The solution isn't bigger windows — it's better state persistence. Progress files + decision logs + git checkpoints — give the amnesiac craftsman a reliable journal.
- Treat the agent like an engineer with amnesia: before "clocking out," write down what was done, why, and what's next.
- Rebuild cost is the key metric. Good harnesses should get new sessions to an executable state within 3 minutes.
- Mixed strategy: short tasks within sessions, long tasks with structured artifacts for continuity.
diff --git a/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md b/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md
index de72054..afe5558 100644
--- a/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md
+++ b/docs/en/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md
@@ -5,20 +5,17 @@
# Lecture 06. Initialize Before Every Agent Session
-## What Problem Does This Lecture Solve?
+You start a new agent session and say "add a search feature." It jumps straight into coding — admirable enthusiasm. After 20 minutes it discovers the test framework isn't configured properly, spends another 10 fixing that, then the database migration script format is wrong, more fiddling. The search feature eventually gets added, but the whole session was inefficient — most time went to "figuring out how this project works" rather than writing the search feature.
-You start a new agent session and say "add a search feature." It jumps straight into coding, spends 20 minutes writing code, then discovers the test framework isn't configured properly. Another 10 minutes fixing the test framework, then the database migration script format is wrong, more fiddling. The search feature eventually gets added, but the whole session was inefficient — most time went to "figuring out how this project works" rather than writing the search feature.
+The better approach: before letting the agent start working, use a separate phase to get the base environment ready, verification commands passing, and project structure understood. It's like building a house — you don't pour the foundation and put up walls simultaneously. If you do, the walls go up before the foundation has cured, and the whole building has to be torn down and started over. Pour the foundation first, let it cure, then build the walls — clean and efficient.
-The better approach: before letting the agent start working, use a separate phase to get the base environment ready, verification commands passing, and project structure understood. This is the "initialization phase." This lecture explains why initialization must be a separate phase, not mixed in with implementation.
+This lecture explains why initialization must be a separate phase, not mixed in with implementation.
-## Core Concepts
+## Foundation and Walls: Two Fundamentally Different Jobs
-- **Initialization Phase**: The first phase in the agent's lifecycle — no feature implementation, only establishing prerequisites for all subsequent implementation phases. The output isn't code, it's infrastructure.
-- **Bootstrap Contract**: The conditions under which a project can be unambiguously operated by a fresh agent session — can start, can test, can see progress, can pick up next steps. Four conditions, all required.
-- **Cold Start vs Warm Start**: Cold start is from an empty directory where the agent must guess project structure; warm start is from a template or existing project where infrastructure is already in place. Warm start far outperforms cold start.
-- **Handoff Readiness**: The project is in a state at any given moment where a fresh agent can take over. No verbal explanation needed — just repo contents.
-- **Time to First Verification**: The time from project start until the first feature point passes verification. This is the core metric for measuring initialization efficiency.
-- **Downstream Usability**: The best measure of initialization quality — the proportion of subsequent sessions that can successfully execute tasks without relying on implicit knowledge.
+Initialization and implementation have completely different optimization targets. The implementation phase optimizes for: maximizing the quantity and quality of verified features. The initialization phase optimizes for: maximizing the reliability and efficiency of all subsequent implementation.
+
+When you mix initialization and implementation, the agent faces a multi-objective optimization problem — simultaneously building infrastructure and writing feature code. Without explicit priority setting, the agent naturally gravitates toward writing code (because that's directly visible output) while sacrificing infrastructure (because its value only shows in subsequent sessions). It's like telling a construction crew to simultaneously pour the foundation and build the walls — they'll probably rush to build walls because walls are visible and demonstrable. But a house with a bad foundation has systemic problems down the line.
## Initialization Lifecycle
@@ -38,35 +35,36 @@ flowchart TB
end
```
-## Why This Happens
-
-Initialization and implementation have fundamentally different optimization targets. The implementation phase optimizes for: maximizing the quantity and quality of verified features. The initialization phase optimizes for: maximizing the reliability and efficiency of all subsequent implementation.
-
-When you mix initialization and implementation, the agent faces a multi-objective optimization problem — simultaneously building infrastructure and writing feature code. Without explicit priority setting, the agent naturally gravitates toward writing code (because that's directly visible output) while sacrificing infrastructure (because its value only shows in subsequent sessions).
+## What Happens When You Mix Them
-It's like building a house — if you tell the construction crew to simultaneously pour the foundation and put up walls, they'll probably rush to put up walls because walls are visible and demonstrable. But a house with a bad foundation has systemic problems down the line.
+The most direct problem: the foundation doesn't set properly. The agent spends 80% of its effort on feature code and 20% casually setting up some infrastructure. The test framework is configured but never verified, lint rules are set but too loose, no progress file created. These defects aren't obvious in the first session (because the agent still remembers what it did), but they surface in the second session — the new agent doesn't know how to run, test, or where things stand. Shoddy foundation, shaky building.
-Specifically, mixing initialization + implementation causes these problems:
+A more hidden cost is "unverified accumulation" — feature code written before the test framework is configured is code without verification. When you finally go back to add tests for that code, you might discover the design was wrong from the start — had you known, you would have implemented it differently. Like tiling over wet concrete — when you discover the floor isn't level, all the tiles have to be pried up and redone.
-**Incomplete infrastructure.** The agent spends 80% of its effort on feature code and 20% casually setting up some infrastructure. The test framework is configured but never verified, lint rules are set but too loose, no progress file created. These defects aren't obvious in the first session (because the agent still remembers what it did), but they surface in the second session — the new agent doesn't know how to run, test, or where things stand.
+Session budget is being wasted too. Initialization work (configuring environments, setting up tests, understanding project structure) consumes significant budget, leaving less for actual feature implementation. Result: the first session only completes half the features, and the second session has to start over understanding the project. Budget spent on the foundation, but the foundation isn't solid either — neither goal achieved.
-**Accumulated unverified work.** If feature code is written before the test framework is configured, that code is "unverified accumulation." When the test framework is finally ready and you go back to add tests for that code, you might discover the design was wrong from the start — had you known, you would have implemented it differently.
+The most easily overlooked problem is implicit assumption landmines. Decisions the agent makes during initialization (which test framework, how to organize directories, dependency management) — if not explicitly recorded, subsequent sessions can't understand these choices. Worse, subsequent sessions might make contradictory choices. The first construction crew used a concrete foundation, the second crew doesn't know and drove wooden pilings into it — the foundation cracks.
-**Session budget waste.** Initialization work (configuring environments, setting up tests, understanding project structure) consumes significant context budget, leaving less for actual feature implementation. Result: the first session only completes half the features, and the second session has to start over understanding the project.
+Anthropic's long-running application development research explicitly recommends separating initialization from implementation. Their experimental data: projects using a dedicated initialization phase showed 31% higher feature completion rates in multi-session scenarios compared to mixed approaches. The key insight — time invested in the initialization phase is fully recovered in the next 3-4 sessions. The more solid the foundation, the faster the walls go up.
-**Implicit assumption landmines.** Decisions the agent makes during initialization (which test framework, how to organize directories, dependency management) — if not explicitly recorded, subsequent sessions can't understand these choices. Worse, subsequent sessions might make contradictory choices.
+OpenAI's Codex harness engineering guide also emphasizes the "repository as operational record" principle — establish clear operational structure from the first run, or every new session has to re-infer project conventions.
-Anthropic's long-running application development research explicitly recommends separating initialization from implementation. Their experimental data: projects using a dedicated initialization phase showed 31% higher feature completion rates in multi-session scenarios compared to mixed approaches. The key insight — time invested in the initialization phase is fully recovered in the next 3-4 sessions.
+## Core Concepts
-OpenAI's Codex harness engineering guide also emphasizes the "repository as operational record" principle — establish clear operational structure from the first run, or every new session has to re-infer project conventions.
+- **Initialization Phase**: The first phase in the agent's lifecycle — no feature implementation, only establishing prerequisites for all subsequent implementation phases. The output isn't code, it's infrastructure.
+- **Bootstrap Contract**: The conditions under which a project can be unambiguously operated by a fresh agent session — can start, can test, can see progress, can pick up next steps. Four conditions, all required.
+- **Cold Start vs Warm Start**: Cold start is from an empty directory where the agent must guess project structure; warm start is from a template or existing project where infrastructure is already in place. Warm start far outperforms cold start — like starting work on a site with running water and electricity versus beginning from a barren wasteland.
+- **Handoff Readiness**: The project is in a state at any given moment where a fresh agent can take over. No verbal explanation needed — just repo contents.
+- **Time to First Verification**: The time from project start until the first feature point passes verification. This is the core metric for measuring initialization efficiency.
+- **Downstream Usability**: The best measure of initialization quality — the proportion of subsequent sessions that can successfully execute tasks without relying on implicit knowledge.
-## How to Do It Right
+## How to Do Initialization Right
**Treat initialization as a dedicated phase.** The first session does only initialization — no business feature code at all. Initialization produces:
-**1. Runnable environment.** The project starts, dependencies are installed, no environment issues.
+**1. Runnable environment.** The project starts, dependencies are installed, no environment issues. Foundation poured, no cracks.
-**2. Verifiable test framework.** At least one example test passes. This proves the test framework itself is properly configured.
+**2. Verifiable test framework.** At least one example test passes. This proves the test framework itself is properly configured — like standing a pillar on the foundation to prove it can bear weight.
**3. Bootstrap contract document.** A clear document telling subsequent sessions:
```markdown
@@ -111,7 +109,7 @@ OpenAI's Codex harness engineering guide also emphasizes the "repository as oper
**5. Git commit as checkpoint.** After initialization completes, commit a clean checkpoint. All subsequent work starts from this checkpoint.
-**Warm start strategy**: Don't start from an empty directory. Use a project template (create-react-app, fastapi-template, etc.) to preset standard directory structure, dependency configuration, and test framework. Bake common initialization steps into the template, leaving only project-specific initialization work.
+**Warm start strategy**: Don't start from an empty directory. Use a project template (create-react-app, fastapi-template, etc.) to preset standard directory structure, dependency configuration, and test framework. Bake common initialization steps into the template, leaving only project-specific initialization work. Like starting work on a site with running water and electricity — ten thousand times better than beginning from a barren wasteland.
**Initialization completion criteria**: Not "how much code was written," but whether the bootstrap contract's four conditions are met — can start, can test, can see progress, can pick up next steps. Use this checklist to validate initialization:
@@ -128,19 +126,19 @@ OpenAI's Codex harness engineering guide also emphasizes the "repository as oper
Two initialization approaches for a React frontend project:
-**Mixed approach (wrong)**: The agent simultaneously created project scaffolding and implemented the first feature in session 1. At session end, the repo had runnable code but: no explicit start/test command documentation, no progress tracking file, no task breakdown. Session 2 spent ~20 minutes inferring project structure, test framework, and build process.
+**Mixed approach (pouring foundation and building walls simultaneously)**: The agent simultaneously created project scaffolding and implemented the first feature in session 1. At session end, the repo had runnable code but: no explicit start/test command documentation, no progress tracking file, no task breakdown. Session 2 spent ~20 minutes inferring project structure, test framework, and build process — like a new construction crew arriving at a site, not knowing how far the foundation got or where the plumbing runs are, having to dig holes one by one to find out.
-**Dedicated initialization (right)**: Session 1 did only initialization — created directory structure from a template, configured the test framework (Vitest + React Testing Library), wrote and verified one example test, created the bootstrap contract document and task breakdown file, committed the initial checkpoint. Session 2's rebuild time was under 3 minutes, and it started working directly from the task list.
+**Dedicated initialization (foundation first)**: Session 1 did only initialization — created directory structure from a template, configured the test framework (Vitest + React Testing Library), wrote and verified one example test, created the bootstrap contract document and task breakdown file, committed the initial checkpoint. Session 2's rebuild time was under 3 minutes, and it started working directly from the task list — the crew arrives, glances at the blueprint, and knows exactly where to pick up.
-Full project cycle comparison: the mixed approach's total rebuild time (across all sessions) was ~60% more than the dedicated initialization approach. The extra 20 minutes spent on initialization was recovered many times over in subsequent sessions.
+Full project cycle comparison: the mixed approach's total rebuild time (across all sessions) was ~60% more than the dedicated initialization approach. The extra 20 minutes spent on initialization was recovered many times over in subsequent sessions. Like a solid foundation making the walls go up faster — slow is fast.
## Key Takeaways
-- Initialization and implementation have different optimization targets — mixing them just drags both down.
+- Initialization and implementation have different optimization targets — mixing them just drags both down. Pour the foundation first, then build the walls.
- Initialization's output isn't code, it's infrastructure: runnable environment, verifiable tests, bootstrap contract, task breakdown.
- Validate initialization with the bootstrap contract's four conditions: can start, can test, can see progress, can pick up next steps.
- Warm start beats cold start. Use project templates to preset standardized infrastructure.
-- Time invested in initialization is fully recovered in the next 3-4 sessions. This isn't extra cost — it's upfront investment.
+- Time invested in initialization is fully recovered in the next 3-4 sessions. This isn't extra cost — it's upfront investment. The more solid the foundation, the faster the building goes up.
## Further Reading
diff --git a/docs/en/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md b/docs/en/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md
index d178c5e..5fd9c47 100644
--- a/docs/en/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md
+++ b/docs/en/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md
@@ -5,20 +5,28 @@
# Lecture 07. Draw Clear Task Boundaries for Agents
-## What Problem Does This Lecture Solve?
+You tell Claude Code to "add user authentication to this project," and it starts modifying the database schema, writing routes, changing frontend components, and — while it's at it — refactoring the error-handling middleware. Two hours later you check: 12 files modified, 800 lines of new code, and not a single feature works end-to-end.
-You tell Claude Code to "add user authentication to this project," and it starts modifying the database schema, writing routes, changing frontend components, and — while it's at it — refactoring the error-handling middleware. Two hours later you check: 12 files modified, 800 lines of new code, and not a single feature works end-to-end. This is the most classic failure mode of AI coding agents: **overreach and under-finish happening simultaneously.**
+Biting off more than you can chew — this saying applies to AI agents especially well. Agents are born with an impulse to "do a little extra" — they see related things and just handle them along the way, like someone who goes to the supermarket for a bottle of soy sauce and comes out pushing a full cart. The problem is, humans who buy too much just waste money; agents doing too many things simultaneously means none of them get done properly.
-Anthropic's "Effective harnesses for long-running agents" engineering blog post states clearly: when prompts are too broad, agents tend to "start multiple things at once" rather than "finish one thing first." OpenAI's Codex engineering practices found the same — tasks without explicit scope controls see completion rates plummet. This is not a model problem — it's a harness problem. You didn't draw the boundary.
+Anthropic's "Effective harnesses for long-running agents" engineering blog states clearly: when prompts are too broad, agents tend to "start multiple things at once" rather than "finish one thing first." OpenAI's Codex engineering practices found the same — tasks without explicit scope controls see completion rates plummet. This is not a model problem — it's a harness problem. You didn't draw the boundary.
-## Core Concepts
+## Attention Is a Finite Resource
-- **Overreach**: The agent activates more tasks in a single session than optimal. It's quantifiable — doing 5 features with 0 passing end-to-end is overreach.
-- **Under-finish**: The ratio of tasks that pass end-to-end verification, out of all activated tasks, falls below threshold. Code written but tests not passing is under-finish.
-- **WIP Limit (Work-in-Progress Limit)**: From Kanban methodology. Core idea: limit how many tasks are in-flight at once. For agents, WIP=1 is the safest default — finish one before starting the next.
-- **Completion Evidence**: The verifiable condition a task must satisfy to move from "in progress" to "done." Without this, agents substitute "the code looks fine" for "the behavior passes tests."
-- **Scope Surface**: A DAG structure where each node is a work unit and edges are dependencies. States are limited to four: not_started, active, blocked, passing.
-- **Completion Pressure**: The constraining force the harness exerts through WIP limits and completion evidence requirements, forcing the agent to finish the current task before starting a new one.
+This isn't a metaphor — it's math. Assume the agent's context capacity is C and it activates k tasks simultaneously. Each task gets an average of C/k reasoning resources. When C/k drops below the minimum threshold needed to complete a single task, none of them get finished. Your stomach is only so big — stuff ten dumplings in at once and you won't digest them all, you'll just get ten cases of indigestion.
+
+Claude Code's real behavior is telling. Ask it to "add user registration" and it might:
+
+1. Create a User model
+2. Write the registration route
+3. Notice it needs email verification, so add a mail service
+4. See that passwords need hashing, so bring in bcrypt
+5. Notice the error handling is inconsistent, so refactor the global error middleware
+6. See the test file structure is messy, so reorganize the directory
+
+Six steps later, every one is half-done. No end-to-end verification, complex coupling between the half-baked code, and the next session to pick up the pieces will be completely lost. Like someone cooking six dishes simultaneously — every dish is in the pan but none has been plated. They all burn.
+
+Anthropic's experimental data directly supports this: agents using a "small next step" strategy (equivalent to WIP=1) show a 37% higher task completion rate than agents using broad prompts. More interestingly, the number of lines of code generated by agents is weakly negatively correlated with actual feature completion — more code written, fewer features completed. Biting off more than you can chew, proven by data.
## WIP=1 Workflow
@@ -42,37 +50,23 @@ flowchart TB
Partial --> VCR["Low verified completion rate<br/>high rework next session"]
```
-## Why This Happens
-
-### Agents Are Born Wanting to "Do a Little Extra"
-
-LLM training data is full of code patterns where "one PR changes multiple things." A typical GitHub PR might include a feature implementation, a refactoring, and a documentation update all at once. The default behavior the model learns from this distribution is "do all the related things you see."
-
-This is a problem in human projects too — Steve McConnell documented in *Rapid Development* that scope creep is the leading cause of project failure. But humans at least have the intuition of "I've done enough." Agents have none. Generating the next idea costs the model almost nothing in extra tokens — writing "let me fix this too while I'm here" barely registers — but every additional modification dilutes the agent's attention.
-
-Claude Code's real behavior is telling. Ask it to "add user registration" and it might:
-
-1. Create a User model
-2. Write the registration route
-3. Notice it needs email verification, so add a mail service
-4. See that passwords need hashing, so bring in bcrypt
-5. Notice the error handling is inconsistent, so refactor the global error middleware
-6. See the test file structure is messy, so reorganize the directory
-
-Six steps later, every one is half-done. No end-to-end verification, complex coupling between the half-baked code, and the next session to pick up the pieces will be completely lost.
-
-### The Math of Attention Dilution
-
-This isn't a metaphor — it's math. Assume the agent's context capacity is C and it activates k tasks simultaneously. Each task gets an average of C/k reasoning resources. When C/k drops below the minimum threshold needed to complete a single task, none of them get finished.
+## Core Concepts
-Anthropic's experimental data directly supports this: agents using a "small next step" strategy (equivalent to WIP=1) show a 37% higher task completion rate than agents using broad prompts. More interestingly, the number of lines of code generated by agents is weakly negatively correlated with actual feature completion — more code written, fewer features completed.
+- **Overreach**: The agent activates more tasks in a single session than optimal. It's quantifiable — doing 5 features with 0 passing end-to-end is overreach.
+- **Under-finish**: The ratio of tasks that pass end-to-end verification, out of all activated tasks, falls below threshold. Code written but tests not passing is under-finish.
+- **WIP Limit (Work-in-Progress Limit)**: From Kanban methodology. Core idea: limit how many tasks are in-flight at once. For agents, WIP=1 is the safest default — finish one before starting the next. Like a buffet — don't pile your plate, finish one plate then go back for the next.
+- **Completion Evidence**: The verifiable condition a task must satisfy to move from "in progress" to "done." Without this, agents substitute "the code looks fine" for "the behavior passes tests."
+- **Scope Surface**: A DAG structure where each node is a work unit and edges are dependencies. States are limited to four: not_started, active, blocked, passing.
+- **Completion Pressure**: The constraining force the harness exerts through WIP limits and completion evidence requirements, forcing the agent to finish the current task before starting a new one.
-### Overreach and Under-finish Are Symbiotic
+## Overreach and Under-finish Are Symbiotic
-These two problems aren't independent — they amplify each other. Overreach dilutes attention, diluted attention causes under-finish, and the half-finished code left behind increases system complexity, which further drives overreach in the next task.
+These two problems aren't independent — they amplify each other. Overreach dilutes attention, diluted attention causes under-finish, and the half-finished code left behind increases system complexity, which further drives overreach in the next task. A vicious cycle.
In Kanban terms: Little's Law tells us L = lambda * W. If work-in-progress L is too high (doing too many things at once), the lead time W for each task inevitably increases. For agents, this means each feature takes longer from start to verified completion, and the probability of failure grows.
+This is an old problem in the human world too — Steve McConnell documented in *Rapid Development* that scope creep is the leading cause of project failure. But humans at least have the intuition of "I've done enough." Agents have none. Generating the next idea costs the model almost nothing in extra tokens — writing "let me fix this too while I'm here" barely registers — but every additional modification dilutes the agent's attention. Like a buffet where each extra plate has near-zero marginal cost, but your stomach only has so much capacity.
+
## How to Do It Right
### 1. Enforce WIP=1
@@ -86,6 +80,8 @@ This is the most direct and effective method. In your harness, tell the agent ex
- Don't "also refactor" feature B while implementing feature A
```
+Like eating at a buffet — one plate at a time, finish it before going back for more.
+
### 2. Define Explicit Completion Evidence for Every Task
Done is not "code is written" — it's "behavior verification passes." In your feature list, every entry needs a verification command:
@@ -108,19 +104,19 @@ The harness should continuously track VCR (Verified Completion Rate) = verified
A REST API project with 8 features, two strategies compared:
-**Unconstrained mode**: Agent activates 5 features simultaneously in session 1. Produces ~800 lines across 12 files. End-to-end test pass rate: 20% — only user registration works. The other 4 features: database schema created but missing validation logic, routes defined but returning wrong response formats. By end of session 3, only 3 of 8 features complete.
+**Buffet mode (unconstrained)**: Agent activates 5 features simultaneously in session 1. Produces ~800 lines across 12 files. End-to-end test pass rate: 20% — only user registration works. The other 4 features: database schema created but missing validation logic, routes defined but returning wrong response formats. Like someone cooking six dishes at once, only one is barely edible. By end of session 3, only 3 of 8 features complete.
-**WIP=1 mode**: Agent works on user registration only in session 1. Produces ~200 lines across 4 files. End-to-end tests: 100% passing. Commits a clean, verified implementation. By end of session 4, 7 of 8 features complete (the 8th blocked by an external dependency).
+**Single-plate mode (WIP=1)**: Agent works on user registration only in session 1. Produces ~200 lines across 4 files. End-to-end tests: 100% passing. Commits a clean, verified implementation. By end of session 4, 7 of 8 features complete (the 8th blocked by an external dependency).
-Result: less total code (800 vs 1200 lines) but more effective code. Completion rate: 87.5% vs 37.5%.
+Result: less total code (800 vs 1200 lines) but more effective code. Completion rate: 87.5% vs 37.5%. Take one bite at a time, and you actually eat more.
## Key Takeaways
-- **WIP=1 is the default safe setting for agent harnesses** — finish one, then start the next; don't try to parallelize.
+- **WIP=1 is the default safe setting for agent harnesses** — finish one, then start the next; don't try to parallelize. You can't become fat in one bite.
- **Completion evidence must be executable** — "the code looks fine" doesn't count; "curl returns 201" does.
- **The scope surface must be externalized as a file** — not just mentioned in conversation, but recorded in a machine-readable format in the repo.
- **Overreach and under-finish are symbiotic** — solving one solves the other.
-- **"Do less but finish" always beats "do more but leave half-done"** — agent code lines and feature completion rate are negatively correlated.
+- **"Do less but finish" always beats "do more but leave half-done"** — agent code lines and feature completion rate are negatively correlated. Quality always beats quantity.
## Further Reading
diff --git a/docs/en/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md b/docs/en/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md
index bdeb9e2..e58b5f7 100644
--- a/docs/en/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md
+++ b/docs/en/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md
@@ -5,20 +5,24 @@
# Lecture 08. Use Feature Lists to Constrain What the Agent Does
-## What Problem Does This Lecture Solve?
-
You ask an agent to build an e-commerce site. After it finishes, it tells you "done." You look at the code — user authentication works, but the checkout button in the shopping cart does nothing, and the payment flow isn't connected at all. The problem: you never told it what "done" means, so it used its own standard — "I wrote a lot of code and it looks fairly complete."
-Feature lists are not planning documents for humans to read. They are the core data structure of the harness. The scheduler, verifier, handoff reporter, and progress tracker — every harness component depends on them. Both Anthropic and OpenAI emphasize: **artifacts must be externalized.** Feature state must live in a machine-readable file in the repo, not in unstructured conversation text.
+ Feature lists, in many people's eyes, are just a memo — write things down so you don't forget, then toss it aside. But in the harness world, a feature list isn't a memo for humans — it's the backbone of the entire harness. The scheduler relies on it to pick tasks, the verifier relies on it to judge completion, the handoff reporter relies on it to generate summaries. Break the backbone and the whole body is paralyzed.
-## Core Concepts
+Both Anthropic and OpenAI emphasize: **artifacts must be externalized.** Feature state must live in a machine-readable file in the repo, not in unstructured conversation text.
-- **Feature lists are harness primitives**: Not "optional planning tools," but foundational data structures that all other harness components depend on. The scheduler uses them to pick tasks, the verifier uses them to judge completion, the handoff reporter uses them to generate summaries.
-- **Triple structure**: Each feature item is a `(behavior description, verification command, current state)` triple. Missing any element makes the feature item incomplete.
-- **State machine model**: Each feature item has four states — `not_started`, `active`, `blocked`, `passing`. State transitions are controlled by the harness, not freely changed by the agent.
-- **Pass-state gating**: The only way a feature moves from `active` to `passing` is by the verification command executing successfully. This is irreversible — once `passing`, it can't go back.
-- **Single source of truth**: All information about "what needs to be done" in a project must be derived from one feature list. No contradictions between the feature list and conversation history.
-- **Back-pressure**: The number of features that haven't passed yet is the pressure the harness exerts on the agent. Zero pressure = project complete.
+## Agents Don't Know What "Done" Means
+
+Neither Claude Code nor Codex automatically knows what you mean by "done." You say "add a shopping cart feature," and the model's interpretation might be "write a Cart component and an addToCart method." But you meant "the user can browse products, add to cart, and complete checkout end-to-end." This understanding gap persists without a feature list. The agent uses its own implicit standard — usually "the code has no obvious syntax errors." What you need is end-to-end behavioral verification. Like asking a friend to buy you fruit — you say "get some fruit" and they come back with lemons. Their fruit and your fruit are not the same fruit.
+
+Look at this common progress note:
+
+```
+Did user auth, shopping cart mostly done, still need payments
+```
+Can a new agent session answer these questions from this note? What does "mostly done" mean? Which tests did the cart pass? What's blocking payments? The answer to all is "nobody knows." Like telling your doctor "my stomach hurts, been okay lately" — what medicine can they prescribe?
+
+The result: the new session spends 20 minutes inferring project state, and may re-implement completed features. Anthropic's engineering data shows that good progress records reduce session startup diagnostic time by 60-80%.
## Feature State Machine
@@ -45,44 +49,31 @@ flowchart LR
Active --> Agent
```
-## Why This Happens
-
-### Agents Don't Come with a Built-in "Done" Definition
-
-Neither Claude Code nor Codex automatically knows what you mean by "done." You say "add a shopping cart feature," and the model's interpretation might be "write a Cart component and an addToCart method." But you meant "the user can browse products, add to cart, and complete checkout end-to-end."
-
-This understanding gap persists without a feature list. The agent uses its own implicit standard — usually "the code has no obvious syntax errors." You need end-to-end behavioral verification.
-
-### Free-Form Progress Tracking Is a Disaster
-
-Consider this common progress note:
-
-```
-Did user auth, shopping cart mostly done, still need payments
-```
-
-Can a new agent session answer these questions from this note? What does "mostly done" mean? Which tests did the cart pass? What's blocking payments? The answer to all is "nobody knows."
+## Core Concepts
-The result: the new session spends 20 minutes inferring project state, and may end up re-implementing already-completed features. Anthropic's engineering data shows that good progress records reduce session startup diagnostic time by 60-80%.
+- **Feature lists are harness primitives**: Not "optional planning tools," but foundational data structures that all other harness components depend on. Like database table structures — you can't say "let's skip primary keys."
+- **Triple structure**: Each feature item is a `(behavior description, verification command, current state)` triple. Missing any element makes the item incomplete.
+- **State machine model**: Each feature item has four states — `not_started`, `active`, `blocked`, `passing`. State transitions are controlled by the harness, not freely changed by the agent.
+- **Pass-state gating**: The only way a feature moves from `active` to `passing` is by verification command executing successfully. This is irreversible — once `passing`, it can't go back. Like passing an exam means you passed, you can't retroactively change the score.
+- **Single source of truth**: All information about "what needs to be done" must derive from one feature list. No contradictions between the feature list and conversation history.
+- **Back-pressure**: The number of features that haven't passed yet is the pressure the harness exerts on the agent. Zero pressure = project complete.
-### Why Feature Lists Must Be "Primitives" Not "Documents"
+## Why Feature Lists Must Be "Primitives"
Documents are for humans to read; primitives are for systems to execute. Documents can be ignored; primitives can't be bypassed.
-Think of it like database trigger constraints vs. application-layer checks: the former is enforced by the database engine and no SQL can skip it; the latter depends on application code correctness and can be accidentally bypassed. Feature lists as harness primitives are database-level constraints — the agent can't work around them.
-
-Specifically, the feature list serves four harness components:
+Think of it like database trigger constraints vs. application-layer checks: the former is enforced by the database engine, no SQL can skip it; the latter depends on application code correctness and can be accidentally bypassed. Feature lists as harness primitives are Specifically, the feature list serves four harness components:
-1. **Scheduler**: Reads states, picks the next `not_started` feature.
-2. **Verifier**: Executes verification commands, decides whether to allow state transitions.
-3. **Handoff reporter**: Automatically generates session handoff summaries from the feature list.
-4. **Progress tracker**: Tallies state distribution, provides project health metrics.
+1. **Scheduler**: Reads states, picks the next `not_started` feature. Like a factory production planning system.
+2. **Verifier**: Executes verification commands, decides whether to allow state transitions. Like quality inspection.
+3. **Handoff reporter**: Automatically generates session handoff summaries from the feature list. Like an automatic shift-change report.
+4. **Progress tracker**: Tallies state distribution, provides project health metrics. Like a dashboard.
## How to Do It Right
### 1. Define a Minimal Feature List Format
-You don't need a complex system — a structured Markdown or JSON file works. The key is that every entry must have the triple:
+You don't need a complex system — a structured Markdown or JSON file works. The key is every entry must have the triple:
```json
{
@@ -110,29 +101,29 @@ The agent can't directly change a feature's state to `passing`. It can only subm
### 4. Calibrate Granularity
-Each feature item should be scoped to "completable in one session." Too broad and it won't finish; too narrow and the management overhead grows. "User can add items to cart" is good granularity. "Implement the shopping cart" is too broad. "Create the name field on the Cart model" is too narrow.
+Each feature item should be scoped to "completable in one session." Too broad and it won't finish; too narrow and the management overhead grows. "User can add items to cart" is good granularity. "Implement the shopping cart" is too broad. "Create the name field on the Cart model" is too narrow. Like cutting a steak — not the whole piece, and not ground meat.
## Real-World Case
An e-commerce platform with 10 features. Two tracking approaches compared:
-**Free-form**: Agent uses unstructured notes. After 3 sessions, notes become "did user auth and product list, cart mostly done but has bugs, payments not started." New session needs 20 minutes to infer state, ultimately re-implements completed features.
+**Memo mode**: Agent uses unstructured notes. After 3 sessions, notes become "did user auth and product list, shopping cart mostly done but has bugs, payments not started." New session needs 20 minutes to infer state, ultimately re-implements completed features. Like your shopping list saying "milk, bread, and that thing" — at the store, you still don't know what to buy.
-**Structured feature list**: Every feature has a clear state and verification command. New session reads the list and in 3 minutes knows: F01-F05 are `passing`, F06 is `active`, F07-F10 are `not_started`. Picks up from F06 directly, zero rework.
+**Backbone mode**: Every feature has a clear state and verification command. New session reads the feature list and in 3 minutes knows: F01-F05 are `passing`, F06 is `active`, F07-F10 are `not_started`. Picks up from F06 directly, zero rework.
-Quantified result: projects using structured feature lists show a 45% higher feature completion rate than free-form tracking, with zero duplicate implementations.
+Quantified result: projects using structured feature lists show 45% higher feature completion rate than free-form tracking, with zero duplicate implementations.
## Key Takeaways
-- **Feature lists are the harness's core data structure**, not planning documents for humans. Scheduler, verifier, and handoff reporter all depend on them.
-- **Every feature item must have the triple**: behavior description + verification command + current state. Missing one element makes it incomplete.
-- **State transitions are controlled by the harness** — the agent can't change states on its own. Passing verification is the only upgrade path.
-- **The feature list is the project's single source of truth** — all "what to do" information derives from it.
+- **Feature lists are the harness's backbone**, not memos for humans. Scheduler, verifier, and handoff reporter all depend on them.
+- **Every feature item must have the triple**: behavior description + verification command + current state. Missing one element makes it incomplete — like a three-legged stool missing a leg.
+- **State transitions are controlled by the harness** — the agent can't change states on its own. Passing verification = the only upgrade path.
+- **The feature list is the project's single source of truth** — all "what to do" information derives from one list.
- **Calibrate granularity to "completable in one session."**
## Further Reading
-- [Building Effective Agents - Anthropic](https://www.anthropic.com/research/building-effective-agents) — Explicitly identifies feature lists as the "core data structure" for controlling agent scope
+- [Building Effective Agents - Anthropic](https://www.anthropic.com/research/building-effective-agents) — Explicitly identifies feature list as the "core data structure" for controlling agent scope
- [Harness Engineering - OpenAI](https://openai.com/index/harness-engineering/) — Emphasizes the principle of "externalizing artifacts"
- [Design by Contract - Bertrand Meyer](https://www.goodreads.com/book/show/130439.Object_Oriented_Software_Construction) — Contract design principles, the theoretical foundation of feature lists
- [How Google Tests Software](https://www.goodreads.com/book/show/13563030-how-google-tests-software) — Test pyramid and behavioral specification engineering practices
@@ -141,6 +132,6 @@ Quantified result: projects using structured feature lists show a 45% higher fea
1. **Feature List Design**: Define a minimal feature list JSON schema. Include: id, behavior description, verification command, current state, evidence reference. Use it to describe a real project with 5 features.
-2. **Verification Strictness Comparison**: Pick 3 features and design both a "loose" verification (e.g., "code has no syntax errors") and a "strict" verification (e.g., "end-to-end test passes"). Compare false positive rates under each approach.
+2. **Verification Strictness Comparison**: Pick 3 features and design both a "loose" verification (e.g., "code has no syntax errors") and a "strict" verification (e.g., "end-to-end test passes"). Compare false positive rate under each approach.
3. **Single Source Principle Audit**: Review an existing agent project and check for scope information that contradicts the feature list (implicit requirements in conversations, TODO comments in code, etc.). Design a plan to unify all information into the feature list.
diff --git a/docs/en/lectures/lecture-09-why-agents-declare-victory-too-early/index.md b/docs/en/lectures/lecture-09-why-agents-declare-victory-too-early/index.md
index 7c214aa..20e7c1c 100644
--- a/docs/en/lectures/lecture-09-why-agents-declare-victory-too-early/index.md
+++ b/docs/en/lectures/lecture-09-why-agents-declare-victory-too-early/index.md
@@ -1,132 +1,2 @@
-[中文版本 →](../../../zh/lectures/lecture-09-why-agents-declare-victory-too-early/)
-
-> Code examples: [code/](https://github.com/walkinglabs/learn-harness-engineering/blob/main/docs/en/lectures/lecture-09-why-agents-declare-victory-too-early/code/)
-> Practice project: [Project 05. Self-verification and role separation](./../../projects/project-05-grounded-qa-verification/index.md)
-
-# Lecture 09. Stop Agents from Declaring Victory Early
-
-## What Problem Does This Lecture Solve?
-
-You ask the agent to implement a "password reset" feature. It modifies the database schema, writes the API endpoint, adds email templates, runs the unit tests (all green), and confidently tells you "done." But when you actually test it — the reset link can't be sent (email service config missing), the database migration fails halfway through (schema inconsistency), and the end-to-end flow was never exercised once.
-
-This isn't a random event. Guo et al.'s classic 2017 ICML paper proved that **modern neural networks are systematically overconfident** — the model's reported confidence significantly exceeds its actual accuracy. AI coding agents are no different: they "feel" done when they're far from it. Your harness must replace the agent's "feeling" with externalized, execution-based verification.
-
-## Core Concepts
-
-- **Premature completion declaration**: The agent asserts the task is done, but correctness specifications remain unsatisfied. Core issue: the agent judges based on local code-level confidence; system-level correctness requires global verification.
-- **Confidence calibration gap**: The systematic difference between the agent's self-reported completion confidence and actual completion quality. For complex multi-file tasks, this gap is significantly positive — agents are always more confident than they should be.
-- **Termination criteria**: A set of explicit, executable judgment conditions defined in the harness. The agent must satisfy all conditions before declaring completion. "Done" transforms from a subjective judgment to an objective determination.
-- **Verification-Validation dual gate**: First layer (verification) checks "does the code correctly implement the specified behavior"; second layer (validation) checks "does the system-level behavior meet end-to-end requirements." Both must pass.
-- **Runtime feedback signals**: Logs, process states, health checks from program execution. These are the harness's objective basis for judging completion quality — not optional debugging tools.
-- **Completion priority constraint**: Verify functional correctness first, then performance, then style. No refactoring allowed until core functionality is verified.
-
-## Three-Layer Termination Check
-
-```mermaid
-flowchart LR
- Claim["Agent says: done"] --> L1["First run<br/>lint / typecheck"]
- L1 --> L2["Then run<br/>tests and app startup checks"]
- L2 --> L3["Finally run<br/>the full user flow"]
- L3 --> Done["All three pass -> done"]
-```
-
-```mermaid
-flowchart LR
- A["Code is written<br/>and unit tests are green"] --> B["But the app was not fully started<br/>and the whole flow was not run"]
- B --> C["Config, database, or email issues<br/>stay hidden"]
- C --> D["So the agent declares done too early"]
-```
-
-## Why This Happens
-
-### The Four-Step Slippery Slope of Agent Completion
-
-Premature completion declarations almost always follow the same sequence:
-
-1. **Local code looks fine**: Syntax is correct, logic seems reasonable. No obvious errors at the static inspection level.
-2. **Runtime behavior insufficiently observed**: The harness didn't require comprehensive execution verification, so the agent skipped actually running the code or only ran partial tests.
-3. **Verification skipped or incomplete**: Ran unit tests but skipped integration tests, or ran tests without checking coverage adequacy.
-4. **Completion assertion based on insufficient evidence**: "The code looks fine" is treated as evidence of "feature complete."
-
-Every step loses information. From task specification to code implementation to runtime behavior, each transition can introduce bias, and every skipped verification widens the information asymmetry.
-
-### Unit Tests Passing != Task Complete
-
-This is the most common trap. The agent writes code, runs unit tests, all green, then says "done." But unit tests are designed to isolate the unit under test and mock dependencies — exactly the philosophy that makes them unable to detect cross-component issues:
-
-- **Interface mismatch**: A calls B with correctly-formatted parameters, but B's actual behavior doesn't match A's assumptions. Each unit test passes separately; together they break.
-- **State propagation errors**: A database migration changed the table structure, but the cache layer still holds entries with the old structure. Unit tests don't test cross-layer state.
-- **Environment dependency**: Code behaves correctly in the test environment (everything mocked) but fails in the real environment due to config differences, network latency, or unavailable services.
-
-### "Refactoring While You're at It" Poisons Completion
-
-Claude Code has a common behavior pattern: it starts refactoring code, optimizing performance, or improving style before the core functionality has passed verification. Knuth's "premature optimization is the root of all evil" takes on new meaning for agents — refactoring changes the boundary between verified and unverified code, potentially breaking code paths that were implicitly correct.
-
-## How to Do It Right
-
-### 1. Externalize Termination Judgment
-
-Completion judgment shouldn't be made by the agent itself. The harness independently executes termination checks, taking runtime signals as input — not the agent's confidence. Write in CLAUDE.md:
-
-```
-## Definition of Done
-- Feature complete = end-to-end verification passes, NOT "code is written"
-- Required verification levels:
- 1. Unit tests pass
- 2. Integration tests pass
- 3. End-to-end flow verification passes
-- Don't proceed to level 2 until level 1 passes
-- Don't proceed to level 3 until level 2 passes
-```
-
-### 2. Build a Three-Layer Termination Check
-
-- **Layer 1: Syntax and static analysis**. Lowest cost, least information, but must pass.
-- **Layer 2: Runtime behavior verification**. Test execution, application startup checks, critical path verification. This is the core completion evidence.
-- **Layer 3: System-level validation**. End-to-end tests, integration verification, user scenario simulation. The last line of defense against premature declarations.
-
-### 3. Design Error Messages for Agents
-
-OpenAI's Codex practice highlights a particularly effective pattern: **error messages written for agents should include repair guidance.** Don't use `"Test failed"` — use `"Test failed: POST /api/reset-password returned 500. Check that the email service config exists in environment variables. The template file should be at templates/reset-email.html."` This specific, actionable feedback lets the agent self-correct without human intervention.
-
-### 4. Concrete Forms of Runtime Signals
-
-Effective runtime signals include:
-
-- Did the application successfully start and reach a ready state?
-- Do critical feature paths execute successfully at runtime?
-- Are database writes, file operations, and other side effects correct?
-- Are temporary resources cleaned up?
-
-## Real-World Case
-
-**Task**: Implement user password reset. Involves database operations, email sending, and API endpoint modifications.
-
-**Premature declaration path**: Agent modifies the database schema, writes the API endpoint, adds email templates, runs unit tests (all pass), declares complete.
-
-**Actual defects**: (1) End-to-end flow untested — actual sending and verification of reset links never confirmed. (2) Database migration fails after partial execution, leaving inconsistent schema. (3) Email service config missing in the target environment.
-
-**Harness intervention**: Termination check enforces — (1) start full application and verify reset endpoint is accessible; (2) execute complete reset flow; (3) verify database state consistency. All defects caught within the session, saving 5-10x the cost of finding them later.
-
-## Key Takeaways
-
-- **Agents are systematically overconfident** — the confidence calibration gap is real and measurable, not a subjective impression.
-- **Completion judgment must be externalized** — the harness verifies independently; don't trust the agent's "feeling."
-- **All three verification layers are essential** — syntax passes, behavior passes, system passes, layer by layer.
-- **Error messages should be designed for agents** — include specific repair steps so the agent can self-correct.
-- **No refactoring until core functionality verification passes** — the completion priority constraint is key to preventing premature optimization.
-
-## Further Reading
-
-- [On Calibration of Modern Neural Networks - Guo et al.](https://arxiv.org/abs/1706.04599) — Proves modern deep networks are systematically overconfident
-- [Building Effective Agents - Anthropic](https://www.anthropic.com/research/building-effective-agents) — The critical role of runtime evidence in completion judgment
-- [Harness Engineering - OpenAI](https://openai.com/index/harness-engineering/) — Premature completion as one of the main agent failure modes
-- [The Art of Software Testing - Myers](https://www.goodreads.com/book/show/137543.The_Art_of_Software_Testing) — Classic reference on test method layers and effectiveness
-
-## Exercises
-
-1. **Termination Check Design**: Design a complete termination check for a task involving database migration and API modifications. List the runtime signals needed and pass/fail criteria for each. Run it on a real task and record what hidden issues it uncovers.
-
-2. **Calibration Gap Measurement**: Pick 10 coding tasks of different types. Record the agent's self-reported completion confidence and actual completion quality. Calculate the gap and analyze its relationship with task complexity.
-
-3. **Multi-Layer Defense Experiment**: Run three configurations on the same task set — (a) static analysis only, (b) plus unit tests, (c) full three-layer check. Compare premature declaration rates and uncaught defect counts.
+ >> .text
+ bash -c .textfile /Users/sanbu/Code/github/learn-harness-engineering/docs/en/lectures/lecture-09-why-agents-declare-victory-too-early/index.md
\ No newline at end of file
diff --git a/docs/en/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md b/docs/en/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md
index 39d126d..f62afdd 100644
--- a/docs/en/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md
+++ b/docs/en/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md
@@ -1,160 +1,2 @@
-[中文版本 →](../../../zh/lectures/lecture-10-why-end-to-end-testing-changes-results/)
-
-> Code examples: [code/](https://github.com/walkinglabs/learn-harness-engineering/blob/main/docs/en/lectures/lecture-10-why-end-to-end-testing-changes-results/code/)
-> Practice project: [Project 05. Self-verification and role separation](./../../projects/project-05-grounded-qa-verification/index.md)
-
-# Lecture 10. Only a Full-Pipeline Run Counts as Real Verification
-
-## What Problem Does This Lecture Solve?
-
-You ask the agent to add a file export feature to an Electron app. It writes the renderer component, preload script, and service layer logic. Unit tests for each component pass. The agent says "done." You click the export button — wrong file path format, progress bar doesn't respond, memory leak on large files. Five component boundary defects, and unit tests caught zero of them.
-
-Google's test pyramid tells us: lots of unit tests form the base, but if you stop there, you'll systematically miss component interaction issues. For AI coding agents, this problem is worse — agents tend to run the fastest tests and then declare completion. **Only end-to-end tests provide a stronger proof of system-level defect absence** than unit tests alone.
-
-## Core Concepts
-
-- **Component boundary defects**: Components A and B each pass unit tests individually, but their interaction produces incorrect behavior. This is the defect category end-to-end tests excel at catching.
-- **Test adequacy gradient**: Defects detectable by unit tests <= defects detectable by integration tests <= defects detectable by end-to-end tests. Each layer up strengthens detection capability.
-- **Architecture boundary enforcement rules**: Turn architecture document rules (like "renderer process can't directly access the filesystem") into executable automated checks.
-- **Review feedback promotion**: Transform recurring code review comments into automated tests. Each newly captured defect category adds a permanent defense line — the harness automatically gets stronger over time.
-- **Agent-oriented error messages**: Failure messages don't just say "what went wrong" — they tell the agent exactly how to fix it. This turns test failures into self-correcting feedback loops.
-
-## Test Pyramid & Review Feedback Loop
-
-```mermaid
-flowchart TB
- subgraph Unit["Unit tests inspect parts in isolation"]
- U1["Renderer test"]
- U2["Preload test"]
- U3["Service test"]
- end
-
- subgraph E2E["End-to-end run crosses the real system"]
- R["Renderer click"] --> P["Preload bridge"]
- P --> S["Service layer"]
- S --> F["Filesystem / OS"]
- F --> Result["Real exported file"]
- end
-```
-
-```mermaid
-flowchart LR
- Review["Reviewer says:<br/>renderer cannot import fs"] --> Rule["Add a check for direct fs import"]
- Rule --> Message["Error tells the agent<br/>to move file access into preload"]
- Message --> Harness["Add the check to the harness"]
- Harness --> Stronger["Next similar mistake fails fast"]
-```
-
-## Why This Happens
-
-### Unit Tests' Blind Spots
-
-Unit tests are designed around isolation — mock dependencies, focus on the unit under test. This philosophy makes unit tests fast and precise, but creates systematic blind spots:
-
-**Interface mismatch**: The renderer passes relative file paths to the preload script, but the preload expects absolute paths. Each unit test uses mocks, each passes. Only an end-to-end run reveals the mismatch.
-
-**State propagation errors**: A database migration changed the table structure, but the ORM's cache layer still holds entries with the old structure. Unit tests create fresh mock environments for each test and never expose this cross-layer state inconsistency.
-
-**Resource lifecycle issues**: File handles, database connections, and network sockets are acquired and released across multiple components. Unit tests create and destroy independent resources per test, never exposing resource races or leaks.
-
-**Environment dependency**: Code behaves correctly in the test environment (everything mocked) but fails in the real environment due to config differences, network latency, or service unavailability.
-
-### End-to-End Testing Changes Agent Behavior Too
-
-When the agent knows its work will be verified by end-to-end tests, its coding behavior changes:
-
-1. **Considers component interactions**: Writes code thinking "how does this interface connect with upstream" rather than focusing only on individual functions.
-2. **Respects architecture boundaries**: In systems with architecture constraints, end-to-end tests force the agent to respect boundary rules.
-3. **Handles error paths**: End-to-end tests typically include failure scenarios, forcing the agent to think about error handling.
-
-OpenAI's Codex engineering practice emphasizes: **error messages written for agents must include repair guidance.** Don't write `"Direct filesystem access in renderer"` — write `"Direct filesystem access in renderer. All file operations must go through the preload bridge. Move this call to preload/file-ops.ts and invoke it via window.api."` This turns architecture rules into a self-correcting loop.
-
-### Review Feedback Promotion: Making the Harness Stronger Over Time
-
-Human reviewers often give the same type of feedback repeatedly. Review feedback promotion turns this pattern into system improvement:
-
-1. Identify recurring comment types in code review logs
-2. Transform comments into executable regex patterns, AST patterns, or runtime assertions
-3. Write automated checks with agent-oriented error messages
-4. Integrate into the harness verification flow
-5. Validate effectiveness with benchmark tasks
-
-Every captured defect category adds a permanent defense line. The harness automatically strengthens over time.
-
-## How to Do It Right
-
-### 1. The Harness Must Include an End-to-End Layer
-
-In your verification flow, make it explicit: for tasks involving cross-component modifications, end-to-end test passing is a prerequisite for completion. Write in CLAUDE.md:
-
-```
-## Verification Levels
-- Level 1: Unit tests (must pass)
-- Level 2: Integration tests (must pass)
-- Level 3: End-to-end tests (must pass for cross-component changes)
-- Skipping any required level = task not complete
-```
-
-### 2. Make Architecture Rules Executable
-
-Every architecture constraint should have a corresponding test or lint rule:
-
-```bash
-# Check if renderer process directly calls Node.js APIs
-grep -r "require('fs')" src/renderer/ && exit 1 || echo "OK: no direct fs access in renderer"
-```
-
-### 3. Design Agent-Oriented Error Messages
-
-Failure messages should include three elements: what went wrong, why, and how to fix it:
-
-```
-ERROR: Found direct import of 'fs' in src/renderer/App.tsx:12
-WHY: Renderer process has no access to Node.js APIs for security
-FIX: Move file operations to src/preload/file-ops.ts and call via window.api.readFile()
-```
-
-### 4. Establish a Review Feedback Promotion Process
-
-Every time a new type of agent error is found in code review, turn it into an automated check. A month later, your harness will be significantly stronger than it was at the start.
-
-## Real-World Case
-
-**Task**: Implement file export in an Electron app. Involves renderer UI, preload script filesystem proxy, service layer data transformation.
-
-**Unit tests pass**: Renderer component tests (pass, mocked file operations), preload script tests (pass, mocked filesystem), service layer tests (pass, mocked data source). Agent declares complete.
-
-**Defects revealed by end-to-end tests**:
-
-| Defect | Description | Unit Tests | E2E Tests |
-|--------|-------------|------------|-----------|
-| Interface mismatch | File path format inconsistency | Not caught | Caught |
-| State propagation | Export progress not sent back to UI via IPC | Not caught | Caught |
-| Resource leak | File handle not released for large exports | Not caught | Caught |
-| Permission issue | Different permissions in packaged environment | Not caught | Caught |
-| Error propagation | Service layer exceptions not reaching UI layer | Not caught | Caught |
-
-All 5 defects caught by end-to-end tests; zero caught by unit tests. Cost: test time increased from 2 seconds to 15 seconds — perfectly acceptable in an agent workflow.
-
-## Key Takeaways
-
-- **Unit tests are systematically blind to component boundary defects** — their isolation design is exactly what prevents them from detecting interaction issues.
-- **End-to-end testing doesn't just detect defects — it changes how the agent codes** — more attention to integration and boundaries.
-- **Architecture rules must be executable** — not written in docs waiting for someone to read, but automatically checked on every change.
-- **Error messages should be designed for agents** — include concrete "how to fix" steps to form a self-correcting loop.
-- **Review feedback promotion makes the harness automatically stronger** — every captured defect category becomes a permanent defense line.
-
-## Further Reading
-
-- [How Google Tests Software - Whittaker et al.](https://www.goodreads.com/book/show/13563030-how-google-tests-software) — The classic source for the test pyramid model
-- [Harness Engineering - OpenAI](https://openai.com/index/harness-engineering/) — Engineering practices for automated architecture constraint enforcement
-- [Chaos Engineering - Netflix (Basiri et al.)](https://ieeexplore.ieee.org/document/7466237) — Proactively injecting failures to verify system resilience
-- [QuickCheck - Claessen & Hughes](https://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf) — Property-based testing, between example tests and formal verification
-
-## Exercises
-
-1. **Cross-Component Defect Detection**: Pick a modification task involving at least three components. Run only unit tests first and record results, then run end-to-end tests. Analyze each additionally discovered defect's cross-layer interaction type.
-
-2. **Architecture Rule Automation**: Pick an architecture constraint from your project and turn it into an executable check (with agent-oriented error messages). Integrate into the harness and validate with a benchmark task.
-
-3. **Review Feedback Promotion**: Find a recurring comment type from code review history and follow the five-step process to turn it into an automated check. Compare occurrence frequency before and after promotion.
+ >> .text
+ bash -c .textfile /Users/sanbu/Code/github/learn-harness-engineering/docs/en/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md
\ No newline at end of file
diff --git a/docs/en/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md b/docs/en/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md
index 643c642..cb0d00a 100644
--- a/docs/en/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md
+++ b/docs/en/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md
@@ -152,3 +152,4 @@ A harness using a planner-generator-evaluator workflow, executing "add dark mode
2. **Sprint Contract Practice**: Write a sprint contract for a real task. Have the agent execute according to the contract, and compare efficiency and quality with and without the contract.
3. **Task Trace Construction**: Record every step of an agent's operations during a complete coding task. Annotate with OpenTelemetry semantic conventions. Analyze information bottlenecks in the trace — which steps lack sufficient signal support for decisions.
+ bash -c .textfile /Users/sanbu/Code/github/learn-harness-engineering/docs/en/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md
\ No newline at end of file
diff --git a/docs/en/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md b/docs/en/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md
index 9ea4253..d7f2f95 100644
--- a/docs/en/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md
+++ b/docs/en/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md
@@ -184,3 +184,4 @@ By week 12, the experimental group's build pass rate is 29 percentage points hig
2. **Benchmark Comparison**: Use a fixed task set with two harness variants (with/without clean state requirements). Compare completion rate, retry count, and defect escape rate.
3. **Harness Simplification Practice**: Pick one harness component, temporarily disable it, and run benchmark tasks. Compare results with and without it. Decide whether to keep, remove, or replace.
+ bash -c .textfile /Users/sanbu/Code/github/learn-harness-engineering/docs/en/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md
\ No newline at end of file
diff --git a/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.md b/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.md
index 96bc6bc..4f4a084 100644
--- a/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.md
+++ b/docs/zh/lectures/lecture-01-why-capable-agents-still-fail/index.md
@@ -5,94 +5,72 @@
# 第一讲. 模型能力强,不等于执行可靠
-## 这节课要解决什么问题
+你说你也算见过世面的人了——Claude Pro 订着,GPT-4o 的 API key 也有,SWE-bench 排行榜上的数字你比谁都清楚。有一天你终于想让 AI agent 帮你改一个真实的项目,信心满满地交代下去。结果呢?加了功能但测试挂了,改了 bug 但引入了新 bug,跑了 20 分钟然后自信满满地说"完成了"——你一看代码,根本不是你要的东西。
-你花了不少钱订阅了 Claude Pro,GPT-4o 在基准测试上分数一个比一个高,SWE-bench 排行榜天天有人刷分。但当你真的让 AI agent 去帮你改一个真实项目的时候——加了功能但测试挂了,改了 bug 但引入了新 bug,跑了 20 分钟然后自信满满地说"完成了",结果你一看代码,根本不是你要的东西。
+这时候你的第一反应是什么?"这模型不行,得换一个更贵的。"且慢。先别急着掏钱包。问题可能根本不在模型身上。
-这不是模型不够聪明。这是你给模型的工作环境太差了。这节课要讲的就是:为什么最强的模型在真实工程任务中依然频繁翻车,以及根本原因不在模型本身,而在你给它搭的"测试架"(harness)。
+让我们先看一组数据。截至 2025 年底,最强的 coding agent 在 SWE-bench Verified 上的通过率大约在 50-60%。这还是精心挑选过的、有明确 issue 描述和测试用例的任务。换到你日常开发的场景——需求模糊、没有现成测试、隐含的业务规则散落在各处——这个数字只会更低。
-## 核心概念
+但这组数据背后藏着一个反直觉的事实。
-- **能力鸿沟(Capability Gap)**:模型在基准测试上的表现和真实任务上的表现之间的巨大落差。SWE-bench Verified 上 50-60% 的通过率意味着近一半的真实 issue 解不了。
-- **测试架(Harness)**:模型之外的一切——指令、工具、环境、状态管理、验证反馈。不是模型权重的部分,全是测试架。
-- **测试架诱导失败**:模型本身能力足够,但因为执行环境有结构性缺陷而失败。Anthropic 的对照实验:同一个 prompt、同一个模型,裸跑 vs 完整 harness,产出质量有本质差异。
-- **验证缺口**:agent 对自己输出的信心评估和实际正确性之间的偏差。agent 说"我做完了"但实际没做完,这是最常见的失败模式。
-- **诊断循环**:执行 → 观察失败 → 定位到 harness 的哪一层出了问题 → 修补那一层 → 重新执行。这是 harness 工程的核心方法论。
-- **完成定义(Definition of Done)**:一组可以用命令验证的条件——测试通过、lint 没报错、类型检查通过。没有显式的完成定义,agent 就会自己编一个。
+## 同一匹马,两种命运
-## 五层失败模型
-
-```mermaid
-flowchart TB
- Start["开始做任务"] --> Q1{"任务写清了吗?"}
- Q1 -->|否| Bad["结果:<br/>写了很多代码<br/>但还是不稳"]
- Q1 -->|是| Q2{"规则写进仓库了吗?"}
- Q2 -->|否| Bad
- Q2 -->|是| Q3{"环境跑得起来吗?"}
- Q3 -->|否| Bad
- Q3 -->|是| Q4{"有检查命令吗?"}
- Q4 -->|否| Bad
- Q4 -->|是| Q5{"有进度记录吗?"}
- Q5 -->|否| Bad
- Q5 -->|是| Good["更容易稳定完成"]
-```
+Anthropic 做过一个对照实验。同一个 prompt("做一个 2D 复古游戏编辑器"),同一个模型(Opus 4.5)。第一次让它裸跑——20 分钟,花了 $9,游戏核心功能根本跑不起来。第二次给它配上完整的 harness(planner + generator + evaluator 三 agent 架构)——6 小时,花了 $200,游戏可以正常游玩。
-```mermaid
-flowchart TB
- Same["同一个任务<br/>同一个模型"]
- Same --> Bare["只给一句话<br/>20 分钟 / $9"]
- Same --> Full["先定规则和检查<br/>6 小时 / $200"]
- Bare --> Bad["核心功能失效"]
- Full --> Good["应用可用"]
-```
+模型没换。Opus 4.5 还是那个 Opus 4.5。换的是马鞍。
+
+OpenAI 在 2025 年发布的 harness engineering 文章里说得更直白:Codex 在一个 harness 搭得好的仓库里,表现能从"不可靠"变成"可靠"。注意他们的用词——不是"好了一点",是质变。就像一匹千里马,没马鞍你也能骑,但骑不了多远、跑不了多快、摔下来也不稀奇。harness 就是那个马鞍——**模型权重之外的一切工程基础设施**。
-## 为什么会这样
+## agent 到底卡在哪
-先看一组数据。截至 2025 年底,最强的 coding agent 在 SWE-bench Verified 上的通过率大约在 50-60% 左右。这还是精心挑选过的、有明确 issue 描述和测试用例的任务。换到真实的日常开发场景——需求模糊、没有现成测试、隐含的业务规则散落在各处——这个数字只会更低。
+那具体是什么在出问题?
-但这里有个反直觉的事实:Anthropic 做过一个对照实验。同一个 prompt("做一个 2D 复古游戏编辑器"),同一个模型(Opus 4.5)。单 agent 裸跑模式,20 分钟花了 $9,结果游戏核心功能根本跑不起来。加上完整 harness(planner + generator + evaluator 三 agent 架构),6 小时花了 $200,游戏可以正常游玩。模型没换,只是 harness 从"裸奔"变成了"全副武装"。
+最常见的:你根本没把任务说清楚。你说"加个搜索功能",agent 理解的跟你完全不一样——搜什么?全文本还是结构化?要不要分页?要不要高亮?你没说清楚,agent 就自己猜。猜对了是运气,猜错了你再改,改的成本比一开始说清楚还高。这就像你去饭馆跟师傅说"来个鱼",端上来是红烧还是清蒸还是酸菜——全看运气。
-OpenAI 在 2025 年发布的 harness engineering 文章里说得很直白:Codex 在一个 harness 搭得好的仓库里,表现能从"不可靠"变成"可靠"——注意他们的用词,不是"好了一点",是质变。Anthropic 的长运行 agent 文档也得出了类似结论:有显式恢复路径、稳定状态管理和可靠反馈回路的 agent,长任务完成率远高于缺乏这些结构的。
+就算你说清了,项目里隐含的架构约定 agent 也不知道。你们团队统一用 SQLAlchemy 2.0 的新语法,但 agent 默认写了 1.x 的代码。所有 API 端点必须走 OAuth 2.0 认证,但这个规则只存在于你脑子里和一个三个月前的 Slack 消息里。Agent 看不到这些——它不是不想遵守,是压根不知道有这回事。
-那具体是什么在出问题?我们把常见的失败模式分成五层:
+环境也是个坑。开发环境配置不完整、依赖缺了、工具版本不对。Agent 把宝贵的上下文窗口花在了 `pip install` 失败、Node 版本不对这些事上,而不是解决你的核心任务。就好比你让一个木匠来干活,结果锤子没带、钉子找不到、工作台还是歪的——手艺再好也使不上劲。
-**任务规范层**:你说的"加个搜索功能",agent 理解的可能跟你完全不一样。搜什么?全文本还是结构化?要不要分页?要不要高亮?模糊匹配还是精确匹配?你没有说清楚,agent 就自己猜,猜错了你再改,改的成本比一开始说清楚还高。
+更常见的:压根没有验证手段。没有测试、没有 lint、或者验证命令根本没告诉 agent。Agent 写完代码,自己看了看觉得没问题,就说完成了。这就像让学生交作业但不给答案检查——他自己觉得自己做对了,但批改的时候一堆错误。Anthropic 还观察到一个有意思的现象:当 agent 感觉上下文快满了,它们会匆忙结束当前工作,跳过验证步骤,选一个简单的方案而不是最优方案。他们把这叫"上下文焦虑"——跟你考试时发现时间快到了赶紧随便选几个选择题是一回事。
-**上下文供给层**:项目里有什么隐含的架构约定,agent 不知道。比如你们团队统一用 SQLAlchemy 2.0 的新语法,但 agent 默认写了 1.x 的代码。再比如所有 API 端点必须走 OAuth 2.0 认证,但这个规则只存在于你脑子里和一个三个月前的 Slack 消息里。Agent 看不到这些。
+长任务跨会话就更惨了——上次会话的发现全丢了,每个新会话都得重新探索项目结构、理解代码组织。缺乏持久化状态的 agent 在超过 30 分钟的任务中失败率急剧上升。
-**执行环境层**:开发环境配置不完整、依赖缺了、工具版本不对。Agent 把宝贵的上下文窗口花在了 `pip install` 失败、Node 版本不对这些环境问题上,而不是解决你的核心任务。
+## 核心概念
-**验证反馈层**:没有测试、没有 lint、或者验证命令根本没告诉 agent。Agent 写完代码,自己看了看觉得没问题,就说完成了。这就像你让学生交作业但不给答案检查——他自己觉得自己做对了,但你批改的时候发现一堆错误。
+理解了上面的场景,这些概念就不再是一堆术语了:
-**状态管理层**:长任务跨会话,上次会话的发现全丢了。每个新会话都得重新探索项目结构、理解代码组织。Anthropic 的数据显示,缺乏持久化状态的 agent 在超过 30 分钟的任务中失败率急剧上升。
+- **能力鸿沟(Capability Gap)**:模型在基准测试上的表现和真实任务上的表现之间的巨大落差。SWE-bench Verified 上 50-60% 的通过率意味着近一半的真实 issue 解不了。
+- **Harness**:模型之外的一切——指令、工具、环境、状态管理、验证反馈。不是模型权重的部分,全是 harness。也就是我们说的"马鞍"。
+- **Harness 诱导失败**:模型本身能力足够,但因为执行环境有结构性缺陷而失败。Anthropic 的对照实验已经证明了这一点。
+- **验证缺口**:agent 对自己输出的信心评估和实际正确性之间的偏差。agent 说"我做完了"但实际没做完——这是最常见的失败模式。
+- **诊断循环**:执行 → 观察失败 → 定位到 harness 的哪一层出了问题 → 修补那一层 → 重新执行。这是 harness 工程的核心方法论。
+- **完成定义(Definition of Done)**:一组可以用命令验证的条件——测试通过、lint 没报错、类型检查通过。没有显式的完成定义,agent 就会自己编一个。
-## 怎么做才对
+## 遇到失败,先修 harness
-核心原则:**遇到失败,先别换模型,先检查 harness。** 如果同一个模型在类似的结构良好的任务中能成功,那优先假设是 harness 的问题。
+核心原则:**遇到失败,先别换模型,先检查 harness。** 如果同一个模型在类似的结构良好的任务中能成功,那优先假设是 harness 的问题。这就像汽车抛锚——你不会第一时间怀疑是发动机坏了,你会先看看是不是没油了。
具体怎么做:
-1. **每次失败都归因到具体层**。不要笼统地说"模型不行",而是问:是任务没说清楚?是上下文不够?是没有验证手段?把每次失败归到上面五层中的某一层。
+**每次失败都归因到具体层。** 不要笼统地说"模型不行",而是问:是任务没说清楚?是上下文不够?是没有验证手段?把每次失败归到上面五层中的某一层。养成这个习惯,你会发现"模型不行"这个结论在你的日志里出现得越来越少。
-2. **给每个任务写显式的完成定义**。不要说"加个搜索功能",要说:
- ```
- 完成标准:
- - 新增 GET /api/search?q=xxx 端点
- - 支持分页,默认 20 条
- - 返回结果包含高亮片段
- - 所有新代码通过 pytest
- - 类型检查通过(mypy --strict)
- ```
-
-3. **创建 AGENTS.md 文件**。在仓库根目录放一个文件,告诉 agent 这个项目的技术栈、架构约定、验证命令。这是 harness 工程的第一步,也是投入产出比最高的一步。
+**给每个任务写显式的完成定义。** 不要说"加个搜索功能",要说:
+```
+完成标准:
+- 新增 GET /api/search?q=xxx 端点
+- 支持分页,默认 20 条
+- 返回结果包含高亮片段
+- 所有新代码通过 pytest
+- 类型检查通过(mypy --strict)
+```
-4. **建立诊断循环**。不要把失败当作"模型又犯傻了",而是当作"harness 又暴露了一个缺陷"的信号。每次失败 → 定位层 → 修补 → 下次不再犯。几轮下来,你的 harness 会越来越强,agent 的表现会稳定提升。
+**创建 AGENTS.md 文件。** 在仓库根目录放一个文件,告诉 agent 这个项目的技术栈、架构约定、验证命令。这是 harness 工程的第一步,也是投入产出比最高的一步。一个 `AGENTS.md` 文件可能比你换一个更贵的模型更有效——我不是在开玩笑。
-5. **量化改进**。记个简单的日志:每个任务成功了没有,失败了是哪一层的问题。跑几轮之后你就能看出来哪个层是瓶颈,集中火力修那个层。
+**建立诊断循环。** 不要把失败当作"模型又犯傻了",而是当作"harness 又暴露了一个缺陷"的信号。每次失败 → 定位层 → 修补 → 下次不再犯。几轮下来,你的 harness 会越来越强,agent 的表现会稳定提升。就像修路——每填一个坑,下一段路就更平坦。
-## 实际案例
+**量化改进。** 记个简单的日志:每个任务成功了没有,失败了是哪一层的问题。跑几轮之后你就能看出来哪个层是瓶颈,集中火力修那个层。
-### 从零开始的实验:0 行人工代码
+## 一百万人行代码的实验
OpenAI 在 2025 年做了一个激进的实验:用 Codex 从一个空的 git 仓库起步,构建一个完整的内部产品。五个月后,这个仓库有大约 100 万行代码——应用逻辑、基础设施、工具、文档、内部开发工具——全部由 agent 生成。三个工程师驱动 Codex,开了大约 1,500 个 PR 并合并。平均每人每天 3.5 个 PR。
@@ -104,23 +82,23 @@ OpenAI 在 2025 年做了一个激进的实验:用 Codex 从一个空的 git
> 来源:[OpenAI: Harness engineering: leveraging Codex in an agent-first world](https://openai.com/index/harness-engineering/)
-### 一个更小的对照实验
+## 一个更接地气的例子
一个团队用 Claude Sonnet 给一个中等规模的 Python Web 应用(FastAPI + PostgreSQL + Redis,约 15,000 行代码)添加新的 API 端点。
-**初始配置**:只给了"在 `/api/v2/users` 下添加用户偏好设置端点"这一句指令。
+起初他们只给了一句话:"在 `/api/v2/users` 下添加用户偏好设置端点"。结果呢?agent 花了 40% 的上下文窗口探索仓库结构,产出了看似合理的代码但没遵循项目的错误处理模式,用了旧版 SQLAlchemy 语法,宣称完成但端点实际有运行时错误。下一个会话还得重新做发现工作。
-**结果**:agent 花了 40% 的上下文窗口探索仓库结构,产出了看似合理的代码但没遵循项目的错误处理模式,用了旧版 SQLAlchemy 语法,宣称完成但端点实际有运行时错误。下一个会话还得重新做发现工作。
+后来他们加了 `AGENTS.md`(描述项目架构和技术栈版本)、显式的验证命令(`pytest tests/api/v2/ && python -m mypy src/`)、和架构决策记录。同一模型在三次独立运行中全部成功,上下文使用效率提高了约 60%。
-**改进后**:团队加了 `AGENTS.md`(描述项目架构和技术栈版本)、显式的验证命令(`pytest tests/api/v2/ && python -m mypy src/`)、和架构决策记录。同一模型在三次独立运行中全部成功,上下文使用效率提高了约 60%。
+模型没变。变的还是 harness。
-## 关键要点
+## 带走什么
-- 模型能力和执行可靠性是两回事。最强的模型在糟糕的 harness 里照样翻车。
-- 失败的时候先看 harness,再看模型。换模型是成本最高的选择。
+- 模型能力和执行可靠性是两回事。千里马也得配上好马鞍。
+- 失败的时候先看 harness,再看模型。换模型是成本最高的选择——而且很多时候根本不是模型的问题。
- 每次失败都是一个信号:你的 harness 有结构性缺陷。把它找出来、修掉。
-- 五层防御:任务规范、上下文供给、执行环境、验证反馈、状态管理。逐层排查。
-- 一个 `AGENTS.md` 文件可能比你换一个更贵的模型更有效。
+- 五层防御:任务规范、上下文供给、执行环境、验证反馈、状态管理。逐层排查,就像医生看病先检查最常见的病因。
+- 一个 `AGENTS.md` 文件可能比你换一个更贵的模型更有效。真的。
## 延伸阅读
diff --git a/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.md b/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.md
index 7dd3ae9..12c7f11 100644
--- a/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.md
+++ b/docs/zh/lectures/lecture-02-what-a-harness-actually-is/index.md
@@ -5,11 +5,27 @@
# 第二讲. Harness 到底是什么
-## 这节课要解决什么问题
+"harness"这个词在 AI coding agent 的圈子里被用得越来越多了,但说实话,大部分人说的 harness 其实就只是"一个 prompt 文件"。这不是 harness。就像你开了一家餐厅,只有食材——没有灶台、没有刀具、没有菜谱、没有出菜流程——那不叫餐厅,那叫冰箱。
-"harness"这个词在 AI coding agent 的圈子里被用得越来越多了,但说实话,大部分人说的 harness 其实就只是"一个 prompt 文件"。这不是 harness。
+这节课要给你一个精确的、可操作的 harness 定义。不是学术论文里的抽象概念,而是你今天就能拿去用的框架:harness 由五个子系统组成,就像一个完整厨房里的五个功能区——菜谱架、刀具架、灶台、备菜台、和出菜检查口。每个子系统都有明确的职责和评判标准。
-这节课要给你一个精确的、可操作的 harness 定义。不是学术论文里的抽象概念,而是你今天就能拿去用的框架:harness 由五个子系统组成,每个子系统都有明确的职责和评判标准。理解了这个框架,你就能系统地诊断"agent 为什么又失败了"——不是笼统地骂模型不行,而是精确地指出是五个子系统里哪一个出了问题。
+## 先讲一个类比
+
+想象你是一个刚入职的工程师,被丢进一个没有任何文档的项目里。没有 README,代码里没有注释,没有人告诉你怎么跑测试,CI 配置文件藏在某个角落里。你能写出好代码吗?也许能——如果你足够聪明又足够有耐心。但你会花大量时间在"搞清楚这个项目是怎么回事"上,而不是在"解决问题"上。
+
+AI agent 面对的困境一模一样。而且更糟——你至少可以问同事,agent 只能看到你放在它面前的文件和它能执行的命令。它不能拍拍同事的肩膀问"哎,这个项目的 ORM 用的是哪个版本?"
+
+OpenAI 在他们的 harness engineering 文章里把 harness 的核心原则表述为"仓库即规范"——所有必要的上下文都应该在仓库里,通过结构化的指令文件、明确的验证命令和清晰的目录组织来呈现。Anthropic 的 long-running agents 文档则更侧重状态持久化、显式恢复路径和结构化的进度跟踪。两家公司的侧重点不同,但都在说同一件事:**模型之外的一切工程基础设施,决定了模型能力能被发挥多少。**
+
+看看几个你熟悉的工具:
+
+**Claude Code** 的设计就体现了 harness 思想。它会读你仓库里的 `CLAUDE.md`(菜谱架),能用 shell 跑命令(刀具架),在你的本地环境里执行(灶台),有会话历史(备菜台),能跑测试看结果(出菜检查口)。但如果你不告诉它怎么跑测试,出菜检查口就是断的——菜做没做熟谁也不知道。
+
+**Cursor** 也是类似的逻辑。它的 `.cursorrules` 文件是菜谱架,终端是刀具架,它能读你的项目结构和 lint 配置是灶台。但 Cursor 的状态管理相对弱——你关掉 IDE 再打开,上次的上下文就没了。
+
+**Codex**(OpenAI 的 coding agent)用 git worktree 隔离每个任务的运行环境,配合本地的可观测性栈(日志、指标、追踪),让每个变更都在独立的环境中验证。它在有 `AGENTS.md` 和清晰验证命令的仓库里,表现远超在"裸"仓库里。
+
+**AutoGPT** 则是反面教材——缺乏结构化的状态管理导致长任务中上下文不断累积,缺乏精确的反馈机制导致 agent 陷入循环。很多人说 AutoGPT "不行",但其实是它的 harness 不行——你给它一个破灶台,再好的食材也做不出菜来。
## 核心概念
@@ -21,6 +37,8 @@
## Harness 五子系统模型
+回到厨房的类比。一个完整的厨房有五个功能区,harness 也有五个子系统:
+
```mermaid
flowchart LR
Rules["项目规则<br/>AGENTS.md / CLAUDE.md"] --> Agent["AI Agent"]
@@ -31,44 +49,15 @@ flowchart LR
Checks --> Agent
```
-## 为什么会这样
-
-让我们用一个类比来说明。想象你是一个刚入职的工程师,被丢进一个没有任何文档的项目里。没有 README,代码里没有注释,没有人告诉你怎么跑测试,CI 配置文件藏在某个角落里。你能写出好代码吗?也许能,但你会花大量时间在"搞清楚这个项目是怎么回事"上,而不是在"解决问题"上。
-
-AI agent 面对的困境一模一样。而且更糟——你至少可以问同事,agent 只能看到你放在它面前的文件和它能执行的命令。
+**指令子系统(菜谱架)**:创建 `AGENTS.md`(或 `CLAUDE.md`),内容包括项目概览和目的(一句话说清楚这是什么)、技术栈和版本(Python 3.11、FastAPI 0.100+、PostgreSQL 15)、首次运行命令(`make setup`、`make test`)、不可违反的硬约束("所有 API 必须走 OAuth 2.0")、指向更详细文档的链接。
-OpenAI 在他们的 harness engineering 文章里把 harness 的核心原则表述为"仓库即规范"——所有必要的上下文都应该在仓库里,通过结构化的指令文件、明确的验证命令和清晰的目录组织来呈现。Anthropic 的 long-running agents 文档则更侧重状态持久化、显式恢复路径和结构化的进度跟踪。两家公司的侧重点不同,但都在说同一件事:**模型之外的一切工程基础设施,决定了模型能力能被发挥多少。**
-
-看看几个具体的例子:
-
-**Claude Code** 的设计就体现了 harness 思想。它会读你仓库里的 `CLAUDE.md`(指令子系统),能用 shell 跑命令(工具子系统),在你的本地环境里执行(环境子系统),有会话历史(状态子系统),能跑测试看结果(反馈子系统)。但如果你不告诉它怎么跑测试,反馈子系统就是断的。
-
-**Cursor** 也是类似的逻辑。它的 `.cursorrules` 文件是指令子系统,终端是工具子系统,它能读你的项目结构和 lint 配置是环境子系统。但 Cursor 的状态管理相对弱——你关掉 IDE 再打开,上次的上下文就没了。
-
-**Codex**(OpenAI 的 coding agent)用 git worktree 隔离每个任务的运行环境,配合本地的可观测性栈(日志、指标、追踪),让每个变更都在独立的环境中验证。它在有 `AGENTS.md` 和清晰验证命令的仓库里,表现远超在"裸"仓库里。
-
-**AutoGPT** 则是反面教材——缺乏结构化的状态管理导致长任务中上下文不断累积,缺乏精确的反馈机制导致 agent 陷入循环。很多人说 AutoGPT "不行",但其实是它的 harness 不行。
+**工具子系统(刀具架)**:确保 agent 有足够的工具访问权限。不要因为"安全考虑"把 shell 给禁了——agent 连 `pip install` 都跑不了,还怎么干活?但也别什么都开放,按最小权限原则来。
-关键洞察:同一个模型,放在不同的 harness 里,表现可以有数量级的差异。Anthropic 的对照实验直接证明了这一点:同一个 prompt、同一个模型(Opus 4.5),裸跑 20 分钟/$9 结果游戏跑不起来,完整 harness 6 小时/$200 游戏可以正常游玩。
+**环境子系统(灶台)**:让环境状态自描述。用 `pyproject.toml` 或 `package.json` 锁定依赖,用 `.nvmrc` 或 `.python-version` 指定运行时版本,用 Docker 或 devcontainer 让环境可重现。
-## 怎么做才对
+**状态子系统(备菜台)**:长任务必须有进度跟踪。用一个简单的 `PROGRESS.md` 文件记录:哪些做完了,哪些在做,哪些被阻塞。每个会话结束前更新,下一个会话开始时读取。
-Harness 是五个子系统。你需要确保每一个都不拖后腿。
-
-**指令子系统 I**:创建 `AGENTS.md`(或 `CLAUDE.md`),内容包括:
-- 项目概览和目的(一句话说清楚这是什么)
-- 技术栈和版本(Python 3.11、FastAPI 0.100+、PostgreSQL 15)
-- 首次运行命令(`make setup`、`make test`)
-- 不可违反的硬约束("所有 API 必须走 OAuth 2.0")
-- 指向更详细文档的链接
-
-**工具子系统 T**:确保 agent 有足够的工具访问权限。不要因为"安全考虑"把 shell 给禁了——agent 连 `pip install` 都跑不了,还怎么干活?但也别什么都开放,按最小权限原则来。
-
-**环境子系统 E**:让环境状态自描述。用 `pyproject.toml` 或 `package.json` 锁定依赖,用 `.nvmrc` 或 `.python-version` 指定运行时版本,用 Docker 或 devcontainer 让环境可重现。
-
-**状态子系统 S**:长任务必须有进度跟踪。用一个简单的 `PROGRESS.md` 文件记录:哪些做完了,哪些在做,哪些被阻塞。每个会话结束前更新,下一个会话开始时读取。
-
-**反馈子系统 F**:这是投入产出比最高的子系统。在 `AGENTS.md` 里显式列出验证命令:
+**反馈子系统(出菜检查口)**:这是投入产出比最高的子系统。在 `AGENTS.md` 里显式列出验证命令:
```
验证命令:
- 测试:pytest tests/ -x
@@ -77,27 +66,29 @@ Harness 是五个子系统。你需要确保每一个都不拖后腿。
- 完整验证:make check(包含以上全部)
```
-**诊断 harness 质量的方法**:用"等模型对照实验"。保持模型不变,逐个移除五个子系统,看哪个子系统缺失时性能下降最多。那个就是你的瓶颈——集中精力加强它。
+五个子系统缺一个,就像厨房里少了一个功能区——菜还是能做,但总是别扭。
+
+**诊断 harness 质量的方法**:用"等模型对照实验"。保持模型不变,逐个移除五个子系统,看哪个子系统缺失时性能下降最多。那个就是你的瓶颈——集中精力加强它。就像找厨房的瓶颈一样:把菜谱架拿走看看出菜速度降多少,把灶台关掉看看影响多大。
-## 实际案例
+## 一个团队的真实经历
-一个团队用 GPT-4o 开发一个 TypeScript + React 前端应用(约 20,000 行代码)。他们经历了四个阶段:
+一个团队用 GPT-4o 开发一个 TypeScript + React 前端应用(约 20,000 行代码)。他们经历了四个阶段——其实就是在一件一件地添置厨具:
-**阶段 1——最小 harness**:只有 README 里的基本项目描述。5 次运行成功 1 次(20%)。主要失败:选错了包管理器(npm vs yarn)、没遵循组件命名约定、跑不了测试。
+**阶段 1——空厨房**:只有 README 里的基本项目描述。5 次运行成功 1 次(20%)。主要失败:选错了包管理器(npm vs yarn)、没遵循组件命名约定、跑不了测试。
-**阶段 2——加指令子系统**:添加 `AGENTS.md`,写明技术栈版本、命名约定、关键架构决策。成功率升到 60%。剩余失败主要来自环境问题和验证缺失。
+**阶段 2——装上菜谱架**:添加 `AGENTS.md`,写明技术栈版本、命名约定、关键架构决策。成功率升到 60%。剩余失败主要来自环境问题和验证缺失。
-**阶段 3——加反馈子系统**:在 `AGENTS.md` 里列出验证命令 `yarn test && yarn lint && yarn build`。成功率升到 80%。
+**阶段 3——开起出菜检查口**:在 `AGENTS.md` 里列出验证命令 `yarn test && yarn lint && yarn build`。成功率升到 80%。
-**阶段 4——加状态子系统**:引入进度文件模板,agent 在每次运行中记录完成和未完成的工作。成功率稳定在 80-100%。
+**阶段 4——备菜台就位**:引入进度文件模板,agent 在每次运行中记录完成和未完成的工作。成功率稳定在 80-100%。
-四次迭代,模型一个字没改,成功率从 20% 到接近 100%。这就是 harness 工程的力量。
+四次迭代,模型一个字没改,成功率从 20% 到接近 100%。这就是 harness 工程的力量。你没有换更贵的食材,你只是把厨房收拾利索了。
## 关键要点
-- Harness = 指令 + 工具 + 环境 + 状态 + 反馈。五个子系统,缺一不可。
+- Harness = 指令 + 工具 + 环境 + 状态 + 反馈。五个子系统,就像厨房的五个功能区,缺一不可。
- 不是模型权重的部分全是 harness。你的 harness 决定了模型能力能被发挥多少。
-- 五个子系统中,反馈子系统通常是投入最少、回报最高的。先把验证命令写清楚。
+- 五个子系统中,反馈子系统通常是投入最少、回报最高的。先把验证命令写清楚——出菜检查口是最值得先装的。
- 用"等模型对照实验"量化各子系统的边际贡献,别凭感觉。
- Harness 和代码一样会腐化。定期审计,像还技术债一样还 harness 债。
diff --git a/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md b/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md
index 8b61598..7628177 100644
--- a/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md
+++ b/docs/zh/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md
@@ -5,20 +5,21 @@
# 第三讲. 让代码仓库成为唯一的事实来源
-## 这节课要解决什么问题
+你团队的架构决策散落在 Confluence、Slack、Jira、和几个资深工程师的脑子里。对人类来说这勉强够用——你可以问同事、搜聊天记录、翻文档。实在不行还能去茶水间堵人。但对 AI agent 来说,不在仓库里的信息等于不存在。
-你团队的架构决策散落在 Confluence、Slack、Jira、和几个资深工程师的脑子里。对人类来说这勉强够用——你可以问同事、搜聊天记录、翻文档。但对 AI agent 来说,不在仓库里的信息等于不存在。
+这不是夸张。想想 agent 的输入都有什么:系统提示和任务描述、仓库里的文件内容、以及工具执行的输出。就这三样。你的 Slack 历史、Jira 工单、Confluence 页面、和周五下午跟同事在茶水间聊的架构决定——agent 全都看不到。它不能"去问一下",也不能"搜一下聊天记录"。它就是一个被关在仓库里的工程师——仓库外面的事,它一概不知。
-这节课讲的是:为什么你必须把所有 agent 需要知道的东西都放进仓库里,以及怎么做才不变成"写一堆没人看的文档"。核心思路不是"写更多文档",而是"把关键的决策信息放到 agent 能看到的地方"。
+所以问题变成了:你给不给这个工程师一张好地图?
-## 核心概念
+## 地图上该画什么
+
+OpenAI 在他们的 harness engineering 文章里把这个问题说得非常直白:**仓库里不存在的信息,对 agent 来说等于不存在。** 他们把这称为"仓库即规范"原则——仓库本身就是最高权威的规范文档。
+
+Anthropic 的 long-running agents 文档也强调了类似的观点:持久化状态是长任务连续性的必要条件。跨会话的知识可恢复性直接决定了任务成功率。而这些状态必须存在于仓库中——因为那是 agent 唯一稳定可访问的存储。
+
+你可能会想:"我们团队人少,知识都在大家脑子里,这不也工作得好好的吗?"没错,对人类来说确实可以。但你要用 agent,就得接受一个事实:agent 不能问人。所有它需要知道的东西,都必须写下来,放在它能找到的地方。
-- **知识可见性缺口**:项目总知识中不在仓库里的比例。缺口越大,agent 失败的概率越高。
-- **系统记录(System of Record)**:代码仓库作为项目决策、架构约束、执行状态和验证标准的权威信息源。仓库说了算,别的地方说了不算。
-- **冷启动测试**:开一个全新的 agent 会话,只看仓库内容,看它能不能回答五个基本问题:这是什么系统?怎么组织的?怎么跑?怎么验证?现在做到哪了?
-- **发现成本**:agent 为了在仓库里找到一条关键信息需要消耗多少上下文。信息放得越隐蔽,发现成本越高,留给实际任务的预算越少。
-- **知识衰减率**:仓库中单位时间内变得过时的知识条目比例。文档和代码脱节是最大的敌人。
-- **ACID 类比**:把数据库的事务管理原则(原子性、一致性、隔离性、持久性)用到 agent 的状态管理上。
+这不是"写更多文档"的问题。这是"把决策信息放到正确的位置"的问题。一份在 `src/api/` 目录下、50 行的 `ARCHITECTURE.md`,比一份在 Confluence 里、500 页但没人维护的设计文档有用一万倍。就好比一张贴在你工位上的、手画的办公室走法图,比一份精美的、放在档案室里的建筑蓝图实用得多——因为前者在你需要的时候就在手边。
## 知识可见性
@@ -33,6 +34,8 @@ flowchart LR
Warning["规则不写进仓库<br/>agent 就看不见"] --> Agent
```
+怎么检验你的地图画得够不够好?做一个"冷启动测试":开一个全新的 agent 会话,只看仓库内容,看它能不能回答五个基本问题:
+
```mermaid
flowchart TB
Q1["这是什么系统?"] --> A1["AGENTS.md / README"]
@@ -48,29 +51,24 @@ flowchart TB
A5 --> Ready
```
-## 为什么会这样
-
-想想 agent 的输入都有什么:系统提示和任务描述、仓库里的文件内容、以及工具执行的输出。就这三样。你的 Slack 历史、Jira 工单、Confluence 页面、和周五下午跟同事在茶水间聊的架构决定——agent 全都看不到。
+如果它答不上来,说明地图上有空白。空白的地方,agent 就得自己猜——猜错了就是 bug,猜多了就浪费上下文。每个新会话都要猜一遍,猜的成本远高于一开始就把地图画好的成本。
-这不是 agent 的 bug,这是它的认知架构决定的。它不像人类那样可以"去问一下"或者"搜一下聊天记录"。
-
-这就带来了一个很实际的问题:当关键的项目知识只存在于仓库外面的时候,agent 要么做错(因为不知道某个隐含的规则),要么浪费大量上下文去重新发现这些知识。每个新会话都要重来一遍。
-
-OpenAI 在他们的 harness engineering 文章里把这个问题说得非常直白:**仓库里不存在的信息,对 agent 来说等于不存在。** 他们把这称为"仓库即规范"原则——仓库本身就是最高权威的规范文档。
-
-Anthropic 的 long-running agents 文档也强调了类似的观点:持久化状态是长任务连续性的必要条件。跨会话的知识可恢复性直接决定了任务成功率。而这些状态必须存在于仓库中——因为那是 agent 唯一稳定可访问的存储。
-
-你可能会想:"我们团队人少,知识都在大家脑子里,这不也工作得好好的吗?" 没错,对人类来说确实可以。但你要用 agent,就得接受一个事实:agent 不能问人。所有它需要知道的东西,都必须写下来,放在它能找到的地方。
+## 核心概念
-这不是"写更多文档"的问题。这是"把决策信息放到正确的位置"的问题。一份在 `src/api/` 目录下、50 行的 `ARCHITECTURE.md`,比一份在 Confluence 里、500 页但没人维护的设计文档有用一万倍。
+- **知识可见性缺口**:项目总知识中不在仓库里的比例。缺口越大,agent 失败的概率越高。你脑子里有多少关于这个项目的隐性知识?把它们全算上,再看有多少写进了仓库——两者的差距就是你的可见性缺口。
+- **系统记录(System of Record)**:代码仓库作为项目决策、架构约束、执行状态和验证标准的权威信息源。仓库说了算,别的地方说了不算。就像地图上标注了"此路不通",你就不会再往那条路走——但如果这个信息只在老张的脑子里,你每次都得问老张。
+- **冷启动测试**:上一节说的五个问题。能回答几个,你的地图就画了几分。
+- **发现成本**:agent 为了在仓库里找到一条关键信息需要消耗多少上下文。信息放得越隐蔽,发现成本越高,留给实际任务的预算越少。把关键信息藏在十层目录深处的 README 里,就像把灭火器锁在地下室的保险柜里——不是没有,是用的时候找不到。
+- **知识衰减率**:仓库中单位时间内变得过时的知识条目比例。文档和代码脱节是最大的敌人——比没有文档更危险的是过时的文档。
+- **ACID 类比**:把数据库的事务管理原则(原子性、一致性、隔离性、持久性)用到 agent 的状态管理上。后面会展开讲。
-## 怎么做才对
+## 怎么画好这张地图
-**原则 1:知识靠近代码。** 一条关于 API 端点认证的规则,应该放在 API 代码旁边,而不是藏在一个巨大的全局文档里。每个模块目录下放一个简短的文档,说清楚这个模块的职责、接口和特殊约束。
+**原则 1:知识靠近代码。** 一条关于 API 端点认证的规则,应该放在 API 代码旁边,而不是藏在一个巨大的全局文档里。每个模块目录下放一个简短的文档,说清楚这个模块的职责、接口和特殊约束。就像图书馆的书架标签——你想找历史类书籍,直接去标着"历史"的书架,不用把整个图书馆翻一遍。
**原则 2:用标准化的入口文件。** `AGENTS.md`(或 `CLAUDE.md`)是 agent 的"着陆页"。它不需要包含所有信息,但必须能让 agent 快速回答"这是什么项目"、"怎么跑"、"怎么验证"这三个问题。50-100 行就够了。
-**原则 3:最小但完备。** 每条知识都应该有明确的使用场景。如果你删掉某条规则不影响 agent 的决策质量,那这条规则就不应该存在。但冷启动测试中的每个问题都必须有答案。
+**原则 3:最小但完备。** 每条知识都应该有明确的使用场景。如果你删掉某条规则不影响 agent 的决策质量,那这条规则就不应该存在。但冷启动测试中的每个问题都必须有答案。这是一个精妙的平衡——不多不少,刚好够用。
**原则 4:和代码一起更新。** 把知识更新跟代码变更绑定在一起。最简单的方法:把架构文档放在对应的模块目录里。改代码的时候自然会看到文档,改代码之后 CI 提醒你检查文档是否需要更新。
@@ -91,18 +89,20 @@ project/
└── Makefile # 标准化的操作命令:setup、test、lint、check
```
-**用 ACID 原则管理 agent 状态**:
+## 用 ACID 原则管理 agent 状态
+
+这个类比来自数据库的事务管理——你可能觉得这是在把简单的事情搞复杂,但实际上它给了你一个非常实用的框架:
-- **原子性**:每次"逻辑操作"(比如"添加新端点并更新测试")用一个 git commit 原子化。中途挂了就 `git stash` 回滚。
-- **一致性**:定义"一致状态"的验证谓词——所有测试通过、lint 无报错。Agent 每次操作后跑验证,不一致的中间状态不要 commit。
-- **隔离性**:多个 agent 并发工作时,状态文件要避免竞争条件。简单方案:每个 agent 用独立的进度文件,或者用 git 分支隔离。
-- **持久性**:关键的项目知识用 git 跟踪的文件持久化。临时状态可以只在会话内存里,但跨会话必须的知识必须写到文件里。
+- **原子性**:每次"逻辑操作"(比如"添加新端点并更新测试")用一个 git commit 原子化。中途挂了就 `git stash` 回滚。要么全做,要么不做,没有"做了一半"。
+- **一致性**:定义"一致状态"的验证谓词——所有测试通过、lint 无报错。Agent 每次操作后跑验证,不一致的中间状态不要 commit。就像银行转账——你不能只扣款不入账。
+- **隔离性**:多个 agent 并发工作时,状态文件要避免竞争条件。简单方案:每个 agent 用独立的进度文件,或者用 git 分支隔离。两个厨师不能同时往同一口锅里放盐——放重了谁负责?
+- **持久性**:关键的项目知识用 git 跟踪的文件持久化。临时状态可以只在会话内存里,但跨会话必须的知识必须写到文件里。脑子里的不算,写在纸上的才算。
-## 实际案例
+## 一个真实的改造故事
一个团队维护一个包含约 30 个微服务的电商平台。架构决策(服务间通信协议、数据一致性策略、API 版本化规则)散落在:Confluence(部分过时)、Slack(难以搜索)、几个资深工程师的脑子里(不可扩展)、以及零星的代码注释(不系统)。
-引入 AI agent 后,70% 的任务需要人工干预。几乎每次失败都涉及 agent 违反了某个"所有人都知道但从未写入仓库"的隐性约束。
+引入 AI agent 后,70% 的任务需要人工干预。几乎每次失败都涉及 agent 违反了某个"所有人都知道但从未写入仓库"的隐性约束。这就像一个新来的员工,没人告诉他"中午点外卖要在群里接龙"——他自己猜,猜错了被骂,但骂完了还是没人告诉他规则。
团队执行了改造:
1. 仓库根目录创建 `AGENTS.md`,写明项目概览、技术栈版本、全局硬约束
@@ -110,15 +110,15 @@ project/
3. 创建集中的 `CONSTRAINTS.md`,用"禁止/必须"的明确语言记录硬约束
4. 每个服务目录添加 `PROGRESS.md`,记录当前工作状态
-改造后:同一 agent 能在冷启动时回答所有关键项目问题,任务完成质量显著提升(Anthropic 的对照实验显示,完整 harness 下 agent 能交付可运行的应用,而裸跑则不能)。
+改造后:同一 agent 能在冷启动时回答所有关键项目问题,任务完成质量显著提升。
## 关键要点
-- 不在仓库里的知识对 agent 来说等于不存在。把关键决策信息放进仓库是最基本的 harness 投资。
+- 不在仓库里的知识对 agent 来说等于不存在。把关键决策信息放进仓库是最基本的 harness 投资——画好地图,才不会迷路。
- 用"冷启动测试"检验仓库质量:全新会话能不能只看仓库回答五个基本问题。
- 知识要靠近代码、最小但完备、跟代码一起更新。不是写更多文档,是把信息放到正确的位置。
- 用 ACID 原则管理 agent 状态:原子提交、一致性验证、隔离并发、持久化关键知识。
-- 知识衰减是最大敌人。文档和代码脱节比没有文档更危险。
+- 知识衰减是最大敌人。过时的文档比没有文档更危险——它会让 agent 走错方向还以为自己是对的。
## 延伸阅读
diff --git a/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md b/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md
index 90f1995..1839287 100644
--- a/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md
+++ b/docs/zh/lectures/lecture-04-why-one-giant-instruction-file-fails/index.md
@@ -5,11 +5,25 @@
# 第四讲. 把指令拆分到不同文件里
-## 这节课要解决什么问题
+你开始认真对待 harness 了——好事。你建了个 `AGENTS.md`,把你能想到的所有规则、约束、历史教训都塞了进去。一个月后这个文件膨胀到了 300 行,两个月 450 行,三个月 600 行。然后你发现 agent 的表现反而变差了——改一个小 bug,agent 花大量上下文处理无关的部署指令;关键的安全约束埋在第 300 行,被直接忽略了;文件里有三条互相矛盾的代码风格规则,agent 每次随机选一条。
-你开始认真对待 harness 了,建了个 `AGENTS.md`,把你能想到的所有规则、约束、历史教训都塞了进去。三个月后这个文件膨胀到了 600 行。然后你发现 agent 的表现反而变差了——简单的 bug 修复任务,agent 花大量上下文处理无关的部署指令;关键的安全约束埋在第 300 行,被直接忽略了;文件里有三条互相矛盾的代码风格规则,agent 每次随机选一条。
+这就是"巨型指令文件"陷阱。就像你往行李箱里塞东西——觉得什么都有用,什么都往里装,结果拉链快崩了,想找换洗内衣得把整个箱子翻一遍。带了一箱子东西,但真正用上的可能只有三分之一。
-这就是"巨型指令文件"陷阱。本节课讲为什么"更多信息"不等于"更好的性能",以及如何把指令拆分成一个结构化的文件网络。
+## 问题的根源:一个恶性循环
+
+最常见的恶性循环是这样的:agent 犯了个错 → 你说"加条规则防止这个" → 加到 AGENTS.md → 暂时管用 → agent 又犯了另一个错 → 再加一条 → 重复 → 文件膨胀到不可控。
+
+这不是你的错。这是一个非常自然的反应——每次出问题就"加条规则"感觉很合理,就像每次出门忘带东西就往包里多塞一样。但累积效应是灾难性的。让我们看看具体出了什么问题。
+
+**上下文预算被吃掉了。** Agent 的上下文窗口是有限的。假设你的 agent 有 200K tokens 的窗口(Claude 的标准),一个膨胀的指令文件可能占掉 10-20K。看起来还有不少余量?但一个复杂的任务可能需要读几十个源文件、工具执行的输出也占上下文、对话历史也在累积。到真正需要理解代码的时候,预算已经不够了——就像你的行李箱塞满了"万一用得着"的东西,结果放不下真正需要带的电脑了。
+
+**中间迷失。** "Lost in the Middle"这篇论文(Liu et al., 2023)清楚地证明了:LLM 对长文本中间部分的信息利用效率显著低于两端。你的 AGENTS.md 有 600 行,第 300 行写的是"所有数据库查询必须用参数化查询"——这是安全硬约束。但它被埋在中间,agent 几乎一定会忽略它。就像你的行李箱里那瓶防晒霜——明明在最底层,但你翻三遍也找不到,最后又去买了一瓶。
+
+**优先级冲突。** 文件里混合了不可违反的硬约束("不得使用 eval()")、重要的设计指导("优先使用函数式风格")、和某个特定场景的历史教训("上周修了一个 WebSocket 内存泄漏,注意类似的模式")。这三条规则的重要性完全不同,但在文件里看起来一模一样。Agent 没有可靠的信号来区分——就像行李箱里护照和充电线混在一起,看不出哪个更紧急。
+
+**维护衰减。** 大文件天生难维护。指令过时了没人删——因为删除的后果不确定("也许别的地方依赖这条规则?"),但加新指令是无成本的。结果文件只增不减,信噪比持续下降。这和软件里的技术债务积累一模一样。
+
+**矛盾累积。** 不同时期加的指令之间开始出现矛盾——一条说"用 TypeScript 严格模式",另一条说"某些遗留文件允许用 any"。Agent 每次随机选一条遵循。就像你妈说"穿厚点",你爸说"别穿太多",你站在门口不知道听谁的。
## 核心概念
@@ -41,33 +55,11 @@ flowchart TB
Mid --> Missed["高概率被稀释或忽略"]
```
-## 为什么会这样
-
-最常见的恶性循环是这样的:agent 犯了个错 → 你说"加条规则防止这个" → 加到 AGENTS.md → 暂时管用 → agent 又犯了另一个错 → 再加一条 → 重复 → 文件膨胀到不可控。
-
-这不是你的错。这是一个非常自然的反应——每次出问题就"加条规则"感觉很合理。但累积效应是灾难性的。让我们看具体出了什么问题:
-
-**上下文预算挤占。** Agent 的上下文窗口是有限的。假设你的 agent 有 200K tokens 的窗口(Claude 的标准),一个膨胀的指令文件可能占掉 10-20K。看起来还有不少余量?但一个复杂的任务可能需要读几十个源文件、工具执行的输出也占上下文、对话历史也在累积。到真正需要理解代码的时候,预算已经不够了。
-
-**中间迷失。** "Lost in the Middle"这篇论文(Liu et al., 2023)清楚地证明了:LLM 对长文本中间部分的信息利用效率显著低于两端。你的 AGENTS.md 有 600 行,第 15 行写的是"所有数据库查询必须用参数化查询"——这是安全硬约束。但第 15 行在 600 行的文件里其实还算靠前。如果这条规则在第 300 行呢?Agent 几乎一定会忽略它。
-
-**优先级冲突。** 文件里混合了不可违反的硬约束("不得使用 eval()")、重要的设计指导("优先使用函数式风格")、和某个特定场景的历史教训("上周修了一个 WebSocket 内存泄漏,注意类似的模式")。这三条规则的重要性完全不同,但在文件里看起来一模一样。Agent 没有可靠的信号来区分它们的相对重要性。
-
-**维护衰减。** 大文件天生难维护。指令过时了没人删——因为删除的后果不确定("也许别的地方依赖这条规则?"),但加新指令是无成本的。结果文件只增不减,信噪比持续下降。这和软件里的技术债务积累一模一样。
-
-**矛盾累积。** 不同时期加的指令之间开始出现矛盾——一条说"用 TypeScript 严格模式",另一条说"某些遗留文件允许用 any"。Agent 每次随机选一条遵循。
-
-OpenAI 和 Anthropic 都隐性支持拆分的做法。OpenAI 说入口文件应"短小且以路由为导向",Anthropic 说长运行 agent 的控制信息应"简洁且高优先级"。两家都在说同一件事:别把什么都塞进一个文件里。
-
-## 怎么做才对
+## 拆分思路
-把巨型文件拆成三层结构:
+核心原则:常用信息放手边,偶尔用的收起来,用不上的别带。
-**第一层:路由文件**(`AGENTS.md`,50-200 行)。只包含:
-- 项目概览(一两句话说清楚这是什么)
-- 首次运行命令(`make setup && make test`)
-- 全局硬约束(不超过 15 条不可违反的规则)
-- 指向专题文档的链接(一行描述 + 适用条件)
+入口文件 `AGENTS.md` 控制在 50-200 行,只放最常用的东西——项目概览(一两句话说清楚这是什么)、首次运行命令(`make setup && make test`)、全局硬约束(不超过 15 条不可违反的规则)、指向专题文档的链接(一行描述 + 适用条件)。
```markdown
# AGENTS.md
@@ -91,36 +83,33 @@ Python 3.11 FastAPI 后端,PostgreSQL 15 数据库。
- [测试标准](docs/testing-standards.md) — 编写测试时参考
```
-**第二层:专题文档**(每个 50-150 行,按主题组织)。放在 `docs/` 目录下或对应模块目录旁。每个文档覆盖一个特定主题,只在需要时被读取。
+每个专题文档 50-150 行,按主题放在 `docs/` 目录下或对应模块目录旁。Agent 只在需要时才去读。就像行李箱里的收纳袋——内衣一个袋,洗漱一个袋,充电器一个袋。找东西不用翻整个箱子。
-**第三层:代码内嵌**。有些信息直接放在代码里更合适——类型定义、接口注释、配置文件里的说明。Agent 读代码的时候自然能看到。
+还有些信息直接放在代码里更合适——类型定义、接口注释、配置文件里的说明。Agent 读代码的时候自然能看到,不用再在指令里重复一遍。
-**修剪规则**:每条指令都应该有:
-- 明确的来源:"为什么加这条规则?"
-- 明确的适用条件:"这条规则在什么时候需要?"
-- 明确的过期条件:"什么情况下可以删掉这条规则?"
+每条指令都应该标明来源("为什么加这条规则?")、适用条件("这条规则在什么时候需要?")、过期条件("什么情况下可以删掉这条规则?")。定期审计,删掉过时的、冗余的、矛盾的条目。像管理代码依赖一样管理你的指令——用不上的依赖就该删掉,不然它们只会拖慢系统。
-定期审计,删掉过时的、冗余的、矛盾的条目。像管理代码依赖一样管理你的指令。
+如果某条指令必须在入口文件里,放顶部或底部——不要放中间。"中间迷失"效应告诉我们,LLM 对长文本中间部分的信息利用效率显著低于两端。但更好的做法是把指令放到专题文档里,让 agent 按需加载。
-**放东西的位置有讲究**:如果某条指令必须在路由文件里,放顶部或底部——不要放中间。这利用了"中间迷失"效应,确保重要信息不被忽略。但更好的做法是把指令放到专题文档里,让 agent 按需加载。
+OpenAI 和 Anthropic 都隐性支持拆分的做法。OpenAI 说入口文件应"短小且以路由为导向",Anthropic 说长运行 agent 的控制信息应"简洁且高优先级"。两家都在说同一件事:别把什么都塞进一个文件里。行李箱得整理,不能只靠蛮力塞。
## 实际案例
-一个 SaaS 团队的 `AGENTS.md` 从最初的 50 行膨胀到 600 行。内容混合了技术栈版本、编码规范、历史 bug 修复笔记、API 使用说明、部署流程、和团队成员的个人偏好。
+一个 SaaS 团队的 `AGENTS.md` 从最初的 50 行膨胀到 600 行。内容混合了技术栈版本、编码规范、历史 bug 修复笔记、API 使用说明、部署流程、和团队成员的个人偏好——整个行李箱塞得满满当当,拉链都快崩了。
Agent 表现开始明显下降:简单 bug 修复任务中 agent 花大量上下文处理无关的部署指令;安全约束"所有数据库查询必须用参数化查询"埋在第 300 行,经常被忽略;三条矛盾的代码风格规则导致 agent 随机选择。
-团队执行了重构:
+团队执行了"行李箱整理":
1. `AGENTS.md` 裁剪到 80 行:只保留项目概览、运行命令、15 条全局硬约束
2. 创建专题文档:`docs/api-patterns.md`(120 行)、`docs/database-rules.md`(60 行)、`docs/testing-standards.md`(80 行)
3. 路由文件添加指向专题文档的链接
4. 历史笔记要么转成测试用例,要么删除
-重构后:同一任务集的成功率从 45% 提升到 72%。安全约束遵循率从 60% 提升到 95%(因为从文件中间移到了路由文件顶部)。
+重构后:同一任务集的成功率从 45% 提升到 72%。安全约束遵循率从 60% 提升到 95%——因为从文件中间移到了路由文件顶部,不再被"中间迷失"了。
## 关键要点
-- "加条规则"是短期的止痛药,长期的毒药。每次加规则前想想:这条规则放专题文档是不是更合适?
+- "加条规则"是短期的止痛药,长期的毒药。每次加规则前想想:这条规则放专题文档是不是更合适?——别什么都往行李箱里塞。
- 入口文件是路由器,不是百科全书。50-200 行,只放概览、硬约束、和链接。
- 利用"中间迷失"效应:重要信息放文件顶部或底部,不重要的移到专题文档。
- 像管理技术债一样管理指令膨胀。定期审计,每条指令要有来源、适用条件、和过期条件。
diff --git a/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md b/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md
index 7f8bb71..537d7e4 100644
--- a/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md
+++ b/docs/zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md
@@ -5,23 +5,24 @@
# 第五讲. 让跨会话的任务保持上下文连续
-## 这节课要解决什么问题
-
你让 Claude Code 帮你实现一个完整的功能,它跑了 30 分钟,做了大部分工作,但上下文快满了。你开个新会话继续,然后发现:它不记得上次做了什么决策、为什么选了方案 A 而不是方案 B、哪些文件已经改过、测试跑到什么状态了。它得花 15 分钟重新探索一遍项目,而且可能跟上次的做法不一致。
-这是 AI coding agent 最痛的问题之一:跨会话的上下文连续性。本节课讲为什么 agent 在长任务中会"断片",以及如何通过结构化的状态持久化来保持连续性。
+想象一下,如果你是一个工匠,每天早上醒来都不记得昨天干了什么。你得重新认识整个工地——哪面墙砌了一半、为什么用的是红砖而不是青砖、水电管道走到哪了。更糟的是,你可能会把昨天已经装好的窗户拆了重来,因为你不记得它已经装过了。
-## 核心概念
+这就是 AI coding agent 在跨会话任务中面对的困境。本节课讲为什么 agent 会"断片",以及如何通过结构化的状态持久化让它像一个每天坚持写日记的工匠——虽然还是会失忆,但日记本里记着一切。
-- **上下文窗口是有限的**:不管模型吹多大的窗口(128K、200K、1M),长任务总会用完。用完之后要么压缩(丢信息),要么重置(开新会话)。两种方式都会丢东西。
-- **连续性工件**:持久化的状态文件,让新会话能无歧义地恢复到上次离开的地方。最基本的形式:进度日志 + 验证记录 + 下一步行动。
-- **重建成本**:新会话恢复到可执行状态所需的时间。好的 harness 能把重建成本从 15 分钟压到 3 分钟。
-- **漂移(Drift)**:agent 的理解跟代码仓库实际状态之间的偏差。每次会话边界都会引入漂移,不加控制会越漂越远。
-- **压缩 vs 重置**:压缩是在同一个会话里把上下文摘要化(保留"是什么",可能丢了"为什么");重置是开新会话从持久化状态重建(干净但依赖工件完备性)。
-- **上下文焦虑**:Anthropic 观察到的一个现象——agent 在接近上下文限制时表现异常,过早结束任务以避免信息丢失。这是一种非理性的资源焦虑。
+## 上下文窗口:不是无限的
+
+上下文窗口是有限的。这不是一个可以通过模型升级解决的问题——即使窗口大小增长到 1M tokens,复杂任务依然会用完。因为 agent 不只是在生成代码,它还要理解代码库、跟踪自己的决策历史、处理工具输出、维护对话上下文。这些信息加起来增长得比窗口扩容快得多。
+
+更深层的问题:agent 产生的信息不是均匀重要的。中间推理步骤包含决策的"为什么"——为什么选了方案 A 而不是方案 B,为什么用了这个库而不是那个库,为什么跳过了某个优化。最终输出只包含"是什么"——代码本身。压缩策略通常保留后者但丢了前者。下一个会话看到代码但不知道为什么这么写,可能会"优化"掉一个有意为之的设计决策。
+
+Anthropic 在他们的长运行 agent 研究中发现了一个很有意思的现象:当 agent 感觉上下文快满了,它们会表现出一种"过早收敛"的行为——匆忙结束当前工作,跳过验证步骤,或者选一个简单的方案而不是最优方案。这就像你考试时发现时间快到了,赶紧随便选几个选择题一样。Anthropic 把这叫"上下文焦虑"。
## 会话连续性流程
+没有连续性工件的时候,每个新会话都是一场灾难:
+
```mermaid
flowchart LR
S1["会话 1<br/>功能做到一半"] --> End1["上下文快满了<br/>会话结束"]
@@ -30,6 +31,8 @@ flowchart LR
Guess --> Drift["代码会重复改<br/>恢复速度也很慢"]
```
+有连续性工件的时候,新会话能快速接上:
+
```mermaid
flowchart LR
Work["会话 1 的工作"] --> Progress["PROGRESS.md<br/>已完成 / 进行中 / 下一步"]
@@ -45,31 +48,32 @@ flowchart LR
Rebuild --> Resume["新会话能很快接上"]
```
-## 为什么会这样
-
-上下文窗口是有限的。这不是一个可以通过模型升级解决的问题——即使窗口大小增长到 1M tokens,复杂任务依然会用完。因为 agent 不只是在生成代码,它还要理解代码库、跟踪自己的决策历史、处理工具输出、维护对话上下文。这些信息加起来增长得比窗口扩容快得多。
-
-更深层的问题:agent 产生的信息不是均匀重要的。中间推理步骤包含决策的"为什么"——为什么选了方案 A 而不是方案 B,为什么用了这个库而不是那个库,为什么跳过了某个优化。最终输出只包含"是什么"——代码本身。压缩策略通常保留后者但丢了前者。下一个会话看到代码但不知道为什么这么写,可能会"优化"掉一个有意为之的设计决策。
+## 核心概念
-Anthropic 在他们的长运行 agent 研究中发现了一个很有意思的现象:当 agent 感觉上下文快满了,它们会表现出一种"过早收敛"的行为——匆忙结束当前工作,跳过验证步骤,或者选一个简单的方案而不是最优方案。这就像你考试时发现时间快到了,赶紧随便选几个选择题一样。Anthropic 把这叫"上下文焦虑"。
+- **上下文窗口是有限的**:不管模型吹多大的窗口(128K、200K、1M),长任务总会用完。用完之后要么压缩(丢信息),要么重置(开新会话)。两种方式都会丢东西。
+- **连续性工件**:持久化的状态文件,让新会话能无歧义地恢复到上次离开的地方。最基本的形式:进度日志 + 验证记录 + 下一步行动。就是那个工匠的日记本。
+- **重建成本**:新会话恢复到可执行状态所需的时间。好的 harness 能把重建成本从 15 分钟压到 3 分钟。
+- **漂移(Drift)**:agent 的理解跟代码仓库实际状态之间的偏差。每次会话边界都会引入漂移,不加控制会越漂越远。
+- **上下文焦虑**:Anthropic 观察到的现象——agent 在接近上下文限制时表现异常,过早结束任务以避免信息丢失。是一种非理性的资源焦虑。
+- **压缩 vs 重置**:压缩是在同一个会话里把上下文摘要化(保留"是什么",可能丢了"为什么");重置是开新会话从持久化状态重建(干净但依赖工件完备性)。
-具体来说,跨会话连续性丧失有以下几种表现形式:
+## 连续性断了以后会发生什么
-**决策上下文丢失。** 上个会话花了很多上下文预算分析了三种方案的优劣,最终选了方案 B。这个会话的 agent 不知道这个分析过程,可能基于不完整的信息重新做了决策——而且可能选了方案 A。
+上个会话花了很多上下文预算分析了三种方案的优劣,最终选了方案 B。这个会话的 agent 不知道这个分析过程,可能基于不完整的信息重新做了决策——而且可能选了方案 A。就像那个失忆的工匠不记得为什么选了红砖,今天看着觉得青砖更好看,就把昨天的墙拆了重砌。
-**重复工作。** Agent 不确定某项工作是否已完成,重新做了一遍。或者更糟——做了一半发现跟已有的实现冲突,需要返工。
+更要命的是重复劳动。Agent 不确定某项工作是否已完成,重新做了一遍。或者更糟——做了一半发现跟已有的实现冲突,需要返工。工地上不能两拨人同时砌同一面墙——但在没有进度记录的情况下,新来的人完全不知道这面墙已经有人在砌了。
-**方向漂移。** 每个新会话对项目目标的理解略有偏差。几个会话累积下来,实现方向可能已经偏离了原始需求。
+几个会话累积下来,实现方向可能已经悄悄偏离了原始需求。每个新会话对项目目标的理解都略有偏差,就像传话游戏——经过十个人传话,"帮我买杯咖啡"可能变成了"帮我买个咖啡机"。
-**验证缺口放大。** 上个会话的验证结果(哪些测试通过、哪些失败、为什么失败)没有记录。新会话得重新跑一遍验证才能了解当前状态。
+还有验证缺口。上个会话的验证结果(哪些测试通过、哪些失败、为什么失败)没有记录,新会话得重新跑一遍验证才能了解当前状态。每次都重新诊断,每次都浪费宝贵的上下文。
OpenAI 和 Anthropic 都在他们的文档里强调了结构化状态持久化的重要性。OpenAI 的 harness engineering 文章把仓库当作"操作记录"——每次操作的结果都应该在仓库里留下可追溯的痕迹。Anthropic 的 long-running agents 文档则更具体地建议使用"交接文件"——包含当前状态、已知问题和下一步行动的结构化文档。
-## 怎么做才对
+## 给失忆工匠的日记本
核心思路:**把 agent 当成一个会失忆的超级工程师来管理。** 每次它要"下班"之前,必须把关键信息写下来,让下一个"接班"的 agent 能快速上手。
-**工具 1:进度文件(PROGRESS.md)。** 这是最基本的连续性工件。格式很简单:
+**工具 1:进度文件(PROGRESS.md)**。这是最基本的连续性工件——日记本的核心部分:
```markdown
# 项目进度
@@ -97,7 +101,7 @@ OpenAI 和 Anthropic 都在他们的文档里强调了结构化状态持久化
3. 更新 API 文档
```
-**工具 2:决策日志(DECISIONS.md)。** 记录重要的设计决策和原因。不需要详细的设计文档,只需要"什么决策、为什么、什么时候做的":
+**工具 2:决策日志(DECISIONS.md)**。记录重要的设计决策和原因。不需要详细的设计文档,只需要"什么决策、为什么、什么时候做的"——这是日记本里的备忘:
```markdown
# 设计决策
@@ -110,16 +114,16 @@ OpenAI 和 Anthropic 都在他们的文档里强调了结构化状态持久化
**工具 3:git 提交作为检查点。** 每完成一个原子工作单元就提交。commit message 要说清楚做了什么和为什么。这是免费的、自动版本化的状态快照。
-**工具 4:init.sh 或 harness 的初始化流程。** 在 `AGENTS.md` 里写明:
+**工具 4:init.sh 或 harness 的初始化流程。** 在 `AGENTS.md` 里写明每天"上班"和"下班"的流程:
```markdown
-## 每次会话开始时
+## 每次会话开始时(上班打卡)
1. 读 PROGRESS.md 了解当前状态
2. 读 DECISIONS.md 了解重要决策
3. 跑 make check 确认仓库处于一致状态
4. 从 PROGRESS.md 的"下一步"部分继续工作
-## 每次会话结束前
+## 每次会话结束前(下班打卡)
1. 更新 PROGRESS.md
2. 跑 make check 确认一致状态
3. 提交所有已完成的工作
@@ -129,13 +133,13 @@ OpenAI 和 Anthropic 都在他们的文档里强调了结构化状态持久化
### 上下文焦虑的深层分析
-Anthropic 在 2026 年 3 月发布的研究进一步揭示了上下文焦虑的具体表现:在 Sonnet 4.5 上,当上下文接近窗口限制时,agent 会表现出强烈的"过早收敛"行为——匆忙结束当前工作、跳过验证步骤、选简单方案而非最优方案。这就像考试时发现时间快到了,赶紧随便填选择题。
+Anthropic 在 2026 年 3 月发布的研究进一步揭示了上下文焦虑的具体表现:在 Sonnet 4.5 上,当上下文接近窗口限制时,agent 会表现出强烈的"过早收敛"行为。这就像考试时发现时间快到了,赶紧随便填选择题。
针对这个现象,有两种策略:
**压缩(Compaction)**:在同一个会话里把早期对话摘要化。优点是保留连续性,agent 能看到"是什么"。缺点是"为什么"经常在摘要中丢失——为什么选了方案 B 而非 A,为什么跳过了某个优化。更关键的是,压缩并不能消除上下文焦虑——agent 知道上下文曾经很大,心理上仍然倾向于加速收尾。
-**重置(Context Reset)**:完全清空上下文,开一个新会话,从持久化工件重建。优点是干净的心理状态——新会话没有"我快没时间了"的焦虑。缺点是依赖交接工件的完备性。如果交接文件漏了关键信息,新会话可能在错误方向上浪费时间。
+**重置(Context Reset)**:完全清空上下文,开一个新会话,从持久化工件重建。优点是干净的心理状态——新会话没有"我快没时间了"的焦虑。缺点是依赖交接工件的完备性。如果日记本里漏了关键信息,新会话可能在错误方向上浪费时间。
Anthropic 的实际数据:对于 Sonnet 4.5,上下文焦虑足够严重,以至于压缩单独不够用,上下文重置成为 harness 设计的关键组件。但对于 Opus 4.5,这种行为大幅减弱,可以不依赖重置而靠压缩管理上下文。这意味着:**harness 设计需要对目标模型有具体的理解,而不是套用通用模板。**
@@ -145,16 +149,16 @@ Anthropic 的实际数据:对于 Sonnet 4.5,上下文焦虑足够严重,
一个 agent 被要求实现一个带用户认证的博客系统,12 个功能点,预计需要 5 个会话。
-**无连续性工件的基线**:会话 1 实现了用户模型和基础路由。会话 2 开始时,agent 不记得认证中间件的接口约定,花了约 15 分钟推断上次的设计意图。到会话 3,累积漂移导致 agent 开始重复已实现的功能。到会话 5,仓库有大量冗余代码,但核心认证功能仍未通过端到端测试。12 个功能点只完成了 7 个,其中 3 个有隐含的正确性问题。
+**没有日记本的基线**:会话 1 实现了用户模型和基础路由。会话 2 开始时,agent 不记得认证中间件的接口约定,花了约 15 分钟推断上次的设计意图。到会话 3,累积漂移导致 agent 开始重复已实现的功能。到会话 5,仓库有大量冗余代码,但核心认证功能仍未通过端到端测试。12 个功能点只完成了 7 个,其中 3 个有隐含的正确性问题。就像那个没写日记的工匠——到了第五天,工地上一团乱,有些墙砌了两遍,有些该砌的根本没砌。
-**有结构化工件的对照**:使用进度文件、决策日志、验证记录和 git 检查点。每个会话结束时自动更新状态报告。会话 2 的重建成本降到约 3 分钟。到会话 5,所有 12 个功能点完成且通过验证。
+**有日记本的对照**:使用进度文件、决策日志、验证记录和 git 检查点。每个会话结束时自动更新状态报告。会话 2 的重建成本降到约 3 分钟。到会话 5,所有 12 个功能点完成且通过验证。
-定量对比:重建时间减少约 78%,功能完成率从 58% 提升到 100%,隐含缺陷率从 43% 降到 8%。
+定量对比:重建时间减少约 78%,功能完成率从 58% 提升到 100%,隐含缺陷率从 43% 降到 8%。工匠还是那个会失忆的工匠,但有了日记本,他的每一天都从昨天停下的地方开始,而不是从零开始。
## 关键要点
-- 上下文窗口是有限的资源。长任务一定会跨会话,跨会话一定会丢信息。
-- 解决方案不是更大的窗口,而是更好的状态持久化。进度文件 + 决策日志 + git 检查点。
+- 上下文窗口是有限的资源。长任务一定会跨会话,跨会话一定会丢信息——这就像工匠每天都会失忆一样,是客观现实。
+- 解决方案不是更大的窗口,而是更好的状态持久化。进度文件 + 决策日志 + git 检查点——给失忆的工匠一个靠谱的日记本。
- 把 agent 当成会失忆的工程师来管理:每次"下班"前写清楚做了什么、为什么、下一步做什么。
- 重建成本是关键指标。好的 harness 应该让新会话在 3 分钟内恢复到可执行状态。
- 混合策略:短任务在会话内完成,长任务用结构化工件维持连续性。
diff --git a/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md b/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md
index d11507c..a253a6b 100644
--- a/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md
+++ b/docs/zh/lectures/lecture-06-why-initialization-needs-its-own-phase/index.md
@@ -5,20 +5,17 @@
# 第六讲. 让 agent 每次工作前先初始化
-## 这节课要解决什么问题
+你开了一个新的 agent 会话,让它"帮我加个搜索功能"。它上来就开始改代码——精神可嘉。改了 20 分钟发现测试框架没配好,又花 10 分钟搞测试框架,然后发现数据库迁移脚本格式不对,又折腾了一会儿。最后搜索功能倒是加了,但整个会话的效率很低——大部分时间花在了"搞清楚这个项目怎么运作"上,而不是写搜索功能。
-你开了一个新的 agent 会话,让它"帮我加个搜索功能"。它上来就开始改代码,改了 20 分钟发现测试框架没配好,又花 10 分钟搞测试框架,然后发现数据库迁移脚本格式不对,又折腾了一会儿。最后搜索功能倒是加了,但整个会话的效率很低——大部分时间花在了"搞清楚这个项目怎么运作"上,而不是写搜索功能。
+更好的做法是:在让 agent 开始干活之前,先用一个独立的阶段把基础环境搭好、验证命令跑通、项目结构搞清楚。就像盖房子——你不能边打地基边砌墙,不然墙砌到一半地基还没干,整栋楼都得推倒重来。先打地基,地基干了,再砌墙,一气呵成。
-更好的做法是:在让 agent 开始干活之前,先用一个独立的阶段把基础环境搭好、验证命令跑通、项目结构搞清楚。这就是"初始化阶段"。这节课讲为什么初始化必须是独立的阶段,不能跟实现混在一起。
+这节课讲的就是为什么初始化必须是独立的阶段,不能跟实现混在一起。
-## 核心概念
+## 地基和墙:两种完全不同的工作
-- **初始化阶段**:agent 生命周期中的第一个阶段,不做功能实现,只建立后续所有实现阶段的执行前提。输出不是代码,而是基础设施。
-- **自举契约**:一个项目能被全新 agent 会话无歧义操作的条件——能启动、能测试、能看进度、能接手下一步。四个条件缺一不可。
-- **冷启动 vs 热启动**:冷启动是从空目录开始,agent 要猜项目结构;热启动是从模板或已有项目开始,基础设施已经就位。热启动的效果远好于冷启动。
-- **交接就绪性**:项目在任何时刻都处于"可以被全新 agent 接手"的状态。不需要口头解释,只看仓库内容就能接着干。
-- **首次验证时间**:从项目开始到第一个功能点通过验证的时间。这是衡量初始化效率的核心指标。
-- **下游可用性**:初始化质量的最佳衡量标准——后续会话不需要依赖隐式知识就能成功执行任务的比例。
+初始化和实现的优化目标完全不同。实现阶段的目标是:最大化已验证功能的数量和质量。初始化阶段的目标是:最大化后续所有实现的可靠性和效率。
+
+当你把初始化和实现混在一起的时候,agent 面临一个多目标优化问题——它要同时搭基础设施和写功能代码。在没有显式优先级设定的情况下,agent 自然倾向于写代码(因为那是直接可见的产出),而牺牲基础设施(因为它的价值只能在后续会话中体现)。这就像让施工队同时打地基和砌墙——他们大概率会急着砌墙,因为墙看得见、能交差。但地基没打好的房子,后面出的问题是系统性的。
## 初始化生命周期
@@ -38,35 +35,36 @@ flowchart TB
end
```
-## 为什么会这样
-
-初始化和实现的优化目标完全不同。实现阶段的目标是:最大化已验证功能的数量和质量。初始化阶段的目标是:最大化后续所有实现的可靠性和效率。
+## 混在一起做会怎样
-当你把初始化和实现混在一起的时候,agent 面临一个多目标优化问题——它要同时搭基础设施和写功能代码。在没有显式优先级设定的情况下,agent 自然倾向于写代码(因为那是直接可见的产出),而牺牲基础设施(因为它的价值只能在后续会话中体现)。
+最直接的问题:地基打不牢。Agent 花了 80% 的精力写功能代码,剩下 20% 随便搭了点基础设施。测试框架配了但没验证过,lint 规则设了但太宽松,进度文件没创建。这些缺陷在第一个会话里不明显(因为 agent 还记得它做了什么),但到第二个会话就暴露了——新 agent 不知道项目怎么跑、怎么测、做到哪了。地基不牢,地动山摇。
-这就像盖房子——如果你让施工队同时打地基和砌墙,他们大概率会急着砌墙,因为墙看得见、能交差。但地基没打好的房子,后面出的问题是系统性的。
+更隐蔽的代价是"未验证的累积"——在测试框架配好之前写的功能代码,等回头补测试的时候可能发现设计上就有问题,早知道的话应该用不同的方式实现。就像在地基没干的时候就开始贴瓷砖,等发现地面不平时,瓷砖全得撬掉重来。
-具体来说,混合初始化+实现的方式会导致以下问题:
+上下文预算也在被浪费。初始化工作(配环境、配测试、理解项目结构)消耗了大量预算,留给实际功能实现的反而不够了。结果第一个会话只完成了一半的功能,第二个会话还得从头理解项目。预算花在了打地基上,但地基也没打好——两头都没占着。
-**基础设施不完整。** Agent 花了 80% 的精力写功能代码,剩下 20% 随便搭了点基础设施。测试框架配了但没验证过,lint 规则设了但太宽松,进度文件没创建。这些缺陷在第一个会话里不明显(因为 agent 还记得它做了什么),但到第二个会话就暴露了——新 agent 不知道项目怎么跑、怎么测、做到哪了。
+最容易被忽略的是隐式假设埋下的雷。Agent 在初始化过程中做的决策(用什么测试框架、目录怎么组织、依赖怎么管理)如果不显式记录下来,后续会话就可能做出矛盾的选择。第一个施工队用的是水泥地基,第二个施工队不知道,往上面打了木桩——地基直接裂了。
-**"未验证的累积"。** 如果在测试框架配好之前就写了功能代码,这些代码就是"未验证的累积"。等到测试框架配好了,回头给这些代码补测试的时候,可能发现设计上就有问题——早知道的话应该用不同的方式实现。
+Anthropic 在他们的长运行应用开发研究中明确建议把初始化和实现分离。他们的实验数据:使用独立初始化阶段的项目,多会话场景中的功能完成率比混合方式高 31%。关键是——初始化阶段投入的时间在后续 3-4 个会话中就能完全收回。地基打得越扎实,上面的楼盖得越快。
-**会话预算浪费。** 初始化工作(配环境、配测试、理解项目结构)消耗了大量上下文预算,留给实际功能实现的预算反而不够了。结果第一个会话只完成了一半的功能,第二个会话还得从头理解项目。
-
-**隐式假设埋雷。** Agent 在初始化过程中做的决策(用什么测试框架、目录怎么组织、依赖怎么管理)如果不显式记录下来,后续会话就无法理解这些选择——更糟糕的是,后续会话可能会做出矛盾的选择。
+OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"的原则——第一次运行就要建立清晰的操作结构,否则每次新会话都得重新推断项目约定。
-Anthropic 在他们的长运行应用开发研究中明确建议把初始化和实现分离。他们的实验数据:使用独立初始化阶段的项目,多会话场景中的功能完成率比混合方式高 31%。关键是——初始化阶段投入的时间在后续 3-4 个会话中就能完全收回。
+## 核心概念
-OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"的原则——第一次运行就要建立清晰的操作结构,否则每次新会话都得重新推断项目约定。
+- **初始化阶段**:agent 生命周期中的第一个阶段,不做功能实现,只建立后续所有实现阶段的执行前提。输出不是代码,而是基础设施。
+- **自举契约**:一个项目能被全新 agent 会话无歧义操作的条件——能启动、能测试、能看进度、能接手下一步。四个条件缺一不可。
+- **冷启动 vs 热启动**:冷启动是从空目录开始,agent 要猜项目结构;热启动是从模板或已有项目开始,基础设施已经就位。热启动的效果远好于冷启动——就像在有水电的工地上开工,比在荒郊野岭从头搞起快得多。
+- **交接就绪性**:项目在任何时刻都处于"可以被全新 agent 接手"的状态。不需要口头解释,只看仓库内容就能接着干。
+- **首次验证时间**:从项目开始到第一个功能点通过验证的时间。这是衡量初始化效率的核心指标。
+- **下游可用性**:初始化质量的最佳衡量标准——后续会话不需要依赖隐式知识就能成功执行任务的比例。
-## 怎么做才对
+## 怎么做好初始化
**把初始化当作一个独立的阶段来执行。** 第一个会话只做初始化,不写任何业务功能代码。初始化的产出是:
-**1. 可运行的环境。** 项目能启动、依赖都装好、没有环境问题。
+**1. 可运行的环境。** 项目能启动、依赖都装好、没有环境问题。地基浇好了,没裂缝。
-**2. 可验证的测试框架。** 至少有一个示例测试能通过。这证明测试框架本身是配对的。
+**2. 可验证的测试框架。** 至少有一个示例测试能通过。这证明测试框架本身是配对的——就像地基上立了一根柱子,证明地基能承重。
**3. 自举契约文档。** 一个明确的文档告诉后续会话:
```markdown
@@ -111,7 +109,7 @@ OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"
**5. Git 提交作为检查点。** 初始化完成后提交一个干净的 checkpoint。后续所有工作都从这个 checkpoint 开始。
-**热启动策略**:不要从空目录开始。用一个项目模板(create-react-app、fastapi-template 等)预置好标准的目录结构、依赖配置和测试框架。把通用的初始化步骤预置到模板里,只留下项目特有的初始化工作。
+**热启动策略**:不要从空目录开始。用一个项目模板(create-react-app、fastapi-template 等)预置好标准的目录结构、依赖配置和测试框架。把通用的初始化步骤预置到模板里,只留下项目特有的初始化工作。就像在有通水通电的工地上开工,比从荒郊野岭开始强一万倍。
**初始化的完成条件**:不是"写了多少代码",而是自举契约的四个条件都满足了——能启动、能测试、能看进度、能接手下一步。用这个检查清单验收初始化:
@@ -128,19 +126,19 @@ OpenAI 的 Codex harness engineering 指南也强调"仓库作为操作记录"
一个 React 前端项目的两种初始化方式对比:
-**混合方式(错误)**:agent 在第一个会话中同时做了项目脚手架创建和首个功能实现。会话结束时,仓库有可运行的代码,但:没有显式的启动/测试命令文档、没有进度跟踪文件、没有任务分解。第二个会话花了约 20 分钟推断项目结构、测试框架和构建流程。
+**混合方式(边打地基边砌墙)**:agent 在第一个会话中同时做了项目脚手架创建和首个功能实现。会话结束时,仓库有可运行的代码,但:没有显式的启动/测试命令文档、没有进度跟踪文件、没有任务分解。第二个会话花了约 20 分钟推断项目结构、测试框架和构建流程——就像新来的施工队看着一片工地,不知道地基打到什么程度、水电管道走到哪了,只能一个个挖开来看。
-**独立初始化(正确)**:第一个会话只做初始化——用项目模板创建目录结构、配置测试框架(Vitest + React Testing Library)、写一个示例测试并验证通过、创建自举契约文档和任务分解文件、提交初始检查点。第二个会话的重建时间不到 3 分钟,直接从任务列表开始工作。
+**独立初始化(先打地基)**:第一个会话只做初始化——用项目模板创建目录结构、配置测试框架(Vitest + React Testing Library)、写一个示例测试并验证通过、创建自举契约文档和任务分解文件、提交初始检查点。第二个会话的重建时间不到 3 分钟,直接从任务列表开始工作——施工队来了,看了一眼施工图就知道从哪里接着干。
-整个项目周期对比:混合方式的总重建时间(跨所有会话)比独立初始化多约 60%。独立初始化多花的那 20 分钟在后续会话中被成倍收回。
+整个项目周期对比:混合方式的总重建时间(跨所有会话)比独立初始化多约 60%。独立初始化多花的那 20 分钟在后续会话中被成倍收回。就像地基打得扎实,上面盖楼的效率反而更高——慢即是快。
## 关键要点
-- 初始化和实现的优化目标不同,混在一起只会互相拖后腿。
+- 初始化和实现的优化目标不同,混在一起只会互相拖后腿。先打地基,再砌墙。
- 初始化的产出不是代码,是基础设施:可运行的环境、可验证的测试、自举契约、任务分解。
- 用"自举契约"的四个条件验收初始化:能启动、能测试、能看进度、能接手下一步。
- 热启动优于冷启动。用项目模板预置标准化的基础设施。
-- 初始化投入的时间会在后续 3-4 个会话中完全收回。这不是额外的成本,是前期投资。
+- 初始化投入的时间会在后续 3-4 个会话中完全收回。这不是额外的成本,是前期投资——地基打得越扎实,楼盖得越快。
## 延伸阅读
diff --git a/docs/zh/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md b/docs/zh/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md
index 4272ed4..3fe2d03 100644
--- a/docs/zh/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md
+++ b/docs/zh/lectures/lecture-07-why-agents-overreach-and-under-finish/index.md
@@ -5,20 +5,28 @@
# 第七讲. 给 agent 划清每次任务的边界
-## 这节课要解决什么问题
+你让 Claude Code "给这个项目加上用户认证功能",结果它同时开始改数据库 schema、写路由、改前端组件、还顺手重构了错误处理中间件。两个小时后你一看——12 个文件被修改,800 行新代码,但没有一个功能是端到端跑通的。
-你让 Claude Code "给这个项目加上用户认证功能",结果它同时开始改数据库 schema、写路由、改前端组件、还顺手重构了错误处理中间件。两个小时后你一看——12 个文件被修改,800 行新代码,但没有一个功能是端到端跑通的。这就是 AI 编码 agent 最典型的失败模式:**过度延伸(overreach)和不足完成(under-finish)同时发生**。
+贪多嚼不烂,这句话放到 AI agent 身上格外贴切。Agent 天生就有"多做一点"的冲动——看到相关的事情就顺手一起做了,和那种在超市本来只打算买瓶酱油、结果推着满满一车出来的人一个德行。问题是,人类买了太多东西最多浪费钱,agent 同时做太多事情则是每一件都做不好。
Anthropic 在 "Effective harnesses for long-running agents" 工程博客中明确指出:当提示太宽泛时,agent 倾向于"同时启动多件事"而非"先做完一件事"。OpenAI 在 Codex 工程实践中也发现,没有显式范围控制的任务,完成率会暴跌。这不是模型的问题——是你没有在 harness 里给它划清边界。
-## 核心概念
+## 注意力是有限的资源
-- **过度延伸(Overreach)**:agent 在一次会话中激活的任务数量超过最优值。它不是主观判断,而是可量化的——同时做 5 个功能但 0 个跑通,就是 overreach。
-- **不足完成(Under-finish)**:已启动的任务中,通过端到端验证的比例低于阈值。写了代码但没跑通测试,就是 under-finish。
-- **WIP 限制(Work-in-Progress Limit)**:来自 Kanban 方法论。核心思想:限制同时在进行的任务数量。对于 agent,WIP=1 是最安全的默认值——做完一个再做下一个。
-- **完成证据(Completion Evidence)**:一个任务从"进行中"变成"已完成"必须满足的可验证条件。没有这个,agent 会用"代码看起来没问题"代替"行为通过测试"。
-- **范围表面(Scope Surface)**:一个 DAG 结构,每个节点是一个工作单元,边是依赖关系。状态只有四种:未开始、进行中、阻塞、已通过。
-- **完成压力(Completion Pressure)**:harness 通过 WIP 限制和完成证据要求共同产生的约束力,迫使 agent 先完成当前任务再开始新任务。
+这不是比喻,是数学。假设 agent 的上下文容量为 C,同时激活 k 个任务,每个任务平均获得 C/k 的推理资源。当 C/k 低于完成单个任务所需的最小阈值时,所有任务都做不完。这就像你的胃就那么大——同时塞十个包子进去,不是一个都消化不了,而是十个都消化不良。
+
+Claude Code 的真实行为很说明问题。你让它"添加用户注册功能",它很可能这样做:
+
+1. 创建 User model
+2. 写注册路由
+3. 发现需要邮箱验证,于是加邮件服务
+4. 看到密码需要加密,于是引入 bcrypt
+5. 注意到错误处理不统一,于是重构全局错误中间件
+6. 看到测试文件结构不清晰,于是重组目录结构
+
+6 步之后,每一个都是半成品。没有端到端验证,代码之间耦合复杂,下一个会话来接手时会一脸懵。就像一个人同时炒六道菜,每道菜都下了锅但没有一道出锅——全糊了。
+
+Anthropic 的实验数据直接支持这一点:使用"小下一步"策略(等价于 WIP=1)的 agent,任务完成率比使用宽泛提示的 agent 高 37%。更有意思的是,agent 生成的代码行数和实际完成的功能数量呈弱负相关——写得越多,完成得越少。贪多嚼不烂,数据为证。
## WIP=1 工作流
@@ -42,37 +50,23 @@ flowchart TB
Partial --> VCR["已验证完成率低<br/>下一会话返工高"]
```
-## 为什么会这样
-
-### Agent 天生就想"多做一点"
-
-大语言模型的训练数据里充满了"一个 PR 同时改多个东西"的代码模式。一个典型的 GitHub PR 可能同时包含功能实现、重构和文档更新。模型从这些数据中学到的默认行为就是"看到相关的事情就一起做了"。
-
-这在人类世界也是问题——Steve McConnell 在《Rapid Development》中记录,范围蔓延是项目失败的首要原因。但人类至少有"我已经做得够多了"的直觉,agent 完全没有。生成下一个想法的成本对模型来说太低了——写一行"顺便把这个也改了"几乎不消耗额外 token,但每个额外的修改都会稀释 agent 的注意力。
-
-Claude Code 的真实行为很说明问题。你让它"添加用户注册功能",它很可能这样做:
-
-1. 创建 User model
-2. 写注册路由
-3. 发现需要邮箱验证,于是加邮件服务
-4. 看到密码需要加密,于是引入 bcrypt
-5. 注意到错误处理不统一,于是重构全局错误中间件
-6. 看到测试文件结构不清晰,于是重组目录结构
-
-6 步之后,每一个都是半成品。没有端到端验证,代码之间耦合复杂,下一个会话来接手时会一脸懵。
-
-### 注意力稀释的数学
-
-这不是比喻,是数学。假设 agent 的上下文容量为 C,同时激活 k 个任务,每个任务平均获得 C/k 的推理资源。当 C/k 低于完成单个任务所需的最小阈值时,所有任务都做不完。
+## 核心概念
-Anthropic 的实验数据直接支持这一点:使用"小下一步"策略(等价于 WIP=1)的 agent,任务完成率比使用宽泛提示的 agent 高 37%。更有意思的是,agent 生成的代码行数和实际完成的功能数量呈弱负相关——写得越多,完成得越少。
+- **过度延伸(Overreach)**:agent 在一次会话中激活的任务数量超过最优值。它不是主观判断,而是可量化的——同时做 5 个功能但 0 个跑通,就是 overreach。
+- **不足完成(Under-finish)**:已启动的任务中,通过端到端验证的比例低于阈值。写了代码但没跑通测试,就是 under-finish。
+- **WIP 限制(Work-in-Progress Limit)**:来自 Kanban 方法论。核心思想:限制同时在进行的任务数量。对于 agent,WIP=1 是最安全的默认值——做完一个再做下一个。就像自助餐厅不要一次拿太多盘子,吃完一盘再去拿下一盘。
+- **完成证据(Completion Evidence)**:一个任务从"进行中"变成"已完成"必须满足的可验证条件。没有这个,agent 会用"代码看起来没问题"代替"行为通过测试"。
+- **范围表面(Scope Surface)**:一个 DAG 结构,每个节点是一个工作单元,边是依赖关系。状态只有四种:未开始、进行中、阻塞、已通过。
+- **完成压力(Completion Pressure)**:harness 通过 WIP 限制和完成证据要求共同产生的约束力,迫使 agent 先完成当前任务再开始新任务。
-### Overreach 和 Under-finish 是共生关系
+## Overreach 和 Under-finish 是一对难兄难弟
-这两个问题不是独立的,而是互相加剧。overreach 导致注意力分散,注意力分散导致 under-finish,under-finish 留下的半成品代码又增加了系统复杂度,进一步导致下一个任务的 overreach。
+这两个问题不是独立的,而是互相加剧。overreach 导致注意力分散,注意力分散导致 under-finish,under-finish 留下的半成品代码又增加了系统复杂度,进一步导致下一个任务的 overreach。恶性循环。
用 Kanban 的语言说:Little 法则告诉我们 L = lambda * W。如果在制品数量 L 过大(同时做太多事),每个任务的前置时间 W 必然增加。对于 agent 来说,这意味着每个功能从开始到验证通过的时间被拉长,失败概率被放大。
+这在人类世界也是老问题了——Steve McConnell 在《Rapid Development》中记录,范围蔓延是项目失败的首要原因。但人类至少有"我已经做得够多了"的直觉,agent 完全没有。生成下一个想法的成本对模型来说太低了——写一行"顺便把这个也改了"几乎不消耗额外 token,但每个额外的修改都会稀释 agent 的注意力。就像在自助餐厅里,每多拿一个盘子的边际成本几乎为零,但你的胃只有那么大。
+
## 怎么做才对
### 1. 强制 WIP=1
@@ -86,6 +80,8 @@ Anthropic 的实验数据直接支持这一点:使用"小下一步"策略(
- 不要在实现功能 A 时"顺便"重构功能 B
```
+就像吃自助餐——一次只拿一盘,吃完再去拿。
+
### 2. 给每个任务定义显式的完成证据
完成不是"代码写完了",而是"行为验证通过了"。在你的功能列表里,每个条目都要有验证命令:
@@ -108,19 +104,19 @@ harness 应该持续跟踪 VCR(Verified Completion Rate)= 已通过验证的
一个 8 个功能点的 REST API 项目,两种策略对比:
-**无约束模式**:agent 在第一个会话同时启动 5 个功能。产出约 800 行代码,涉及 12 个文件。端到端测试通过率只有 20%——只有用户注册跑通了。其余 4 个功能:数据库 schema 建了但缺验证逻辑,路由定义了但返回格式错误。到第 3 个会话结束,8 个功能只完成 3 个。
+**自助餐模式(无约束)**:agent 在第一个会话同时启动 5 个功能。产出约 800 行代码,涉及 12 个文件。端到端测试通过率只有 20%——只有用户注册跑通了。其余 4 个功能:数据库 schema 建了但缺验证逻辑,路由定义了但返回格式错误。就像一个人同时炒六道菜,只有一道勉强能吃。到第 3 个会话结束,8 个功能只完成 3 个。
-**WIP=1 模式**:agent 在第一个会话只做用户注册。产出约 200 行代码,涉及 4 个文件。端到端测试 100% 通过。提交干净的、已验证的实现。到第 4 个会话结束,8 个功能完成 7 个(第 8 个因外部依赖被阻塞)。
+**单盘模式(WIP=1)**:agent 在第一个会话只做用户注册。产出约 200 行代码,涉及 4 个文件。端到端测试 100% 通过。提交干净的、已验证的实现。到第 4 个会话结束,8 个功能完成 7 个(第 8 个因外部依赖被阻塞)。
-结果:总代码量更少(800 行 vs 1200 行),但有效代码更多。完成率 87.5% vs 37.5%。
+结果:总代码量更少(800 行 vs 1200 行),但有效代码更多。完成率 87.5% vs 37.5%。一口一口吃,反而吃得最多。
## 关键要点
-- **WIP=1 是 agent harness 的默认安全设置**——做完一个再做下一个,不要试图并行。
+- **WIP=1 是 agent harness 的默认安全设置**——做完一个再做下一个,不要试图并行。一口吃不成胖子。
- **完成证据必须是可执行的**——"代码看起来没问题"不算完成,"curl 返回 201"才算。
- **范围表面必须外部化为文件**——不能只在对话里说,必须在仓库里有机器可读的记录。
- **overreach 和 under-finish 是共生问题**——解决一个就解决了另一个。
-- **"少做但做完"永远优于"多做但做半"**——agent 代码行数和功能完成率呈负相关。
+- **"少做但做完"永远优于"多做但做半"**——agent 代码行数和功能完成率呈负相关。质量永远比数量重要。
## 延伸阅读
diff --git a/docs/zh/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md b/docs/zh/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md
index 01499c8..c039747 100644
--- a/docs/zh/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md
+++ b/docs/zh/lectures/lecture-08-why-feature-lists-are-harness-primitives/index.md
@@ -5,20 +5,27 @@
# 第八讲. 用功能清单约束 agent 该做什么
-## 这节课要解决什么问题
-
你让 agent 做一个电商网站,跑完之后它告诉你"做完了"。你打开代码一看——用户认证有了,但购物车的结算按钮点了没反应,支付流程根本没接上。问题是:你从来没告诉它"做完"的标准是什么,所以它用自己的标准——"代码写了不少,看起来挺完整"。
-功能清单(feature list)不是给人类看的规划文档,而是 harness 的核心数据结构。调度器、验证器、交接报告器、进度追踪器——所有 harness 组件都依赖它。Anthropic 和 OpenAI 都强调:**工件必须外部化**。功能状态必须是仓库里机器可读的文件,不能是对话里的非结构化描述。
+功能清单(feature list)在很多人眼里就是个备忘录——写下来怕忘了,写完扔在一边。但在 harness 的世界里,功能清单不是给人看的备忘录,而是整个 harness 的脊梁骨。调度器靠它选任务,验证器靠它判完成,交接器靠它生成报告。脊梁骨断了,全身都瘫。
-## 核心概念
+Anthropic 和 OpenAI 都强调:**工件必须外部化**。功能状态必须是仓库里机器可读的文件,不能是对话里的非结构化描述。
-- **功能清单是 harness 原语**:它不是"可选的规划工具",而是其他所有 harness 组件依赖的基础数据结构。调度器用它选任务,验证器用它判完成,交接器用它生成报告。
-- **三元组结构**:每个功能项是 `(行为描述, 验证命令, 当前状态)` 的三元组。缺了任何一项,这个功能项就不完整。
-- **状态机模型**:每个功能项有四种状态——`not_started`、`active`、`blocked`、`passing`。状态转移由 harness 控制,不是 agent 想改就能改。
-- **通过状态门控**:功能从 `active` 变成 `passing` 的唯一方式是验证命令执行成功。这是不可逆的——`passing` 了就不能退回去。
-- **单一权威来源**:项目里关于"该做什么"的所有信息,必须从一个功能清单派生。不能出现功能清单和对话记录矛盾的情况。
-- **反向压力**:还没通过的功能项数量就是 harness 对 agent 施加的压力。压力归零 = 项目完成。
+## Agent 不知道"做完"是什么意思
+
+Claude Code 和 Codex 都不会自动知道你心目中的"做完"是什么意思。你说"加一个购物车功能",模型的理解可能是"写一个 Cart 组件和 addToCart 方法"。而你的意思是"用户能从浏览商品到下单支付完整走通"。
+
+这个理解鸿沟在没有功能清单的情况下会持续存在。agent 用自己的隐式标准判断完成——通常是"代码没有明显的语法错误"。而你需要的是端到端的行为验证。就像你让朋友帮你买菜,说"买点水果",他拎了一袋柠檬回来——他要的水果,你要的水果,不是一个水果。
+
+看看这种常见的进度记录:
+
+```
+做了用户认证、购物车基本完成了、还需要做支付
+```
+
+新的 agent 会话看到这个记录,能回答以下问题吗?"基本完成"意味着什么?购物车通过了哪些测试?支付的阻塞条件是什么?答案都是"不知道"。就像你看病时跟医生说"我肚子疼,最近还行",医生能开出什么药来?
+
+结果是:新会话花 20 分钟推断项目状态,最终可能重复实现已完成的功能。Anthropic 的工程实践数据表明,好的进度记录可以减少 60-80% 的会话启动诊断时间。
## 功能状态机
@@ -45,27 +52,16 @@ flowchart LR
Active --> Agent
```
-## 为什么会这样
-
-### Agent 没有自带"完成定义"
-
-Claude Code 和 Codex 都不会自动知道你心目中的"做完"是什么意思。你说"加一个购物车功能",模型的理解可能是"写一个 Cart 组件和 addToCart 方法"。而你的意思是"用户能从浏览商品到下单支付完整走通"。
-
-这个理解鸿沟在没有功能清单的情况下会持续存在。agent 用自己的隐式标准判断完成——通常是"代码没有明显的语法错误"。而你需要的是端到端的行为验证。
-
-### 自由形式的进度追踪是一场灾难
-
-看看这种常见的进度记录:
-
-```
-做了用户认证、购物车基本完成了、还需要做支付
-```
-
-新的 agent 会话看到这个记录,能回答以下问题吗?"基本完成"意味着什么?购物车通过了哪些测试?支付的阻塞条件是什么?答案都是"不知道"。
+## 核心概念
-结果是:新会话花 20 分钟推断项目状态,最终可能重复实现已完成的功能。Anthropic 的工程实践数据表明,好的进度记录可以减少 60-80% 的会话启动诊断时间。
+- **功能清单是 harness 原语**:它不是"可选的规划工具",而是其他所有 harness 组件依赖的基础数据结构。就像数据库里的表结构——你不能说"要不我们省掉主键吧",省掉了整个系统就散架了。
+- **三元组结构**:每个功能项是 `(行为描述, 验证命令, 当前状态)` 的三元组。缺了任何一项,这个功能项就不完整。行为描述告诉 agent 做什么,验证命令告诉它怎么算做完,状态告诉它现在到哪了。
+- **状态机模型**:每个功能项有四种状态——`not_started`、`active`、`blocked`、`passing`。状态转移由 harness 控制,不是 agent 想改就能改。
+- **通过状态门控**:功能从 `active` 变成 `passing` 的唯一方式是验证命令执行成功。这是不可逆的——`passing` 了就不能退回去。就像考试及格了就是及格了,不能事后改成分数。
+- **单一权威来源**:项目里关于"该做什么"的所有信息,必须从一个功能清单派生。不能出现功能清单和对话记录矛盾的情况。
+- **反向压力**:还没通过的功能项数量就是 harness 对 agent 施加的压力。压力归零 = 项目完成。
-### 为什么功能清单必须是"原语"而不是"文档"
+## 为什么功能清单必须是"原语"
文档是给人看的,原语是给系统用的。文档可以被忽略,原语不能被绕过。
@@ -73,12 +69,12 @@ Claude Code 和 Codex 都不会自动知道你心目中的"做完"是什么意
具体来说,功能清单服务四个 harness 组件:
-1. **调度器**:读状态,选下一个 `not_started` 的功能。
-2. **验证器**:执行验证命令,判断是否允许状态转移。
-3. **交接报告器**:从功能清单自动生成会话交接摘要。
-4. **进度追踪器**:统计各状态分布,提供项目健康度指标。
+1. **调度器**:读状态,选下一个 `not_started` 的功能。就像工厂的排产系统——看完订单才知道下一步做什么。
+2. **验证器**:执行验证命令,判断是否允许状态转移。就像质检——不是你说合格就合格,得通过检验。
+3. **交接报告器**:从功能清单自动生成会话交接摘要。就像换班时自动生成的交接表——不用手写,系统自己出。
+4. **进度追踪器**:统计各状态分布,提供项目健康度指标。就像仪表盘——一眼看出项目走到哪了。
-## 怎么做才对
+## 怎么做
### 1. 定义一个最小化的功能清单格式
@@ -96,7 +92,7 @@ Claude Code 和 Codex 都不会自动知道你心目中的"做完"是什么意
### 2. 让 harness 控制状态转移
-agent 不能直接把状态改成 `passing`。它只能提交验证请求,harness 执行验证命令,根据结果决定是否允许状态转移。这就是"通过状态门控"。
+agent 不能直接把状态改成 `passing`。它只能提交验证请求,harness 执行验证命令,根据结果决定是否允许状态转移。这就是"通过状态门控"——不是你说考过了就考过了,得看成绩单。
### 3. 在 CLAUDE.md 里写清楚规则
@@ -110,25 +106,25 @@ agent 不能直接把状态改成 `passing`。它只能提交验证请求,harn
### 4. 粒度校准
-每个功能项应该是"一次会话能完成"的范围。太粗了做不完,太细了管理开销大。"用户可以添加商品到购物车"是一个好粒度,"实现购物车"太粗了,"创建 Cart 模型的 name 字段"太细了。
+每个功能项应该是"一次会话能完成"的范围。太粗了做不完,太细了管理开销大。"用户可以添加商品到购物车"是一个好粒度,"实现购物车"太粗了,"创建 Cart 模型的 name 字段"太细了。就像切牛排——不能整块啃,也不能切成肉沫。
## 实际案例
一个电商平台的开发任务,10 个功能项。对比两种追踪方式:
-**自由形式**:agent 用非结构化笔记记录进度。3 个会话后,笔记变成了"做了用户认证和商品列表、购物车基本完成但还有 bug、支付没开始"。新会话需要 20 分钟推断状态,最终重复实现了已完成的功能。
+**备忘录模式**:agent 用非结构化笔记记录进度。3 个会话后,笔记变成了"做了用户认证和商品列表、购物车基本完成但还有 bug、支付没开始"。新会话需要 20 分钟推断状态,最终重复实现了已完成的功能。就像你的购物清单上写着"牛奶、面包、还有那个什么来着"——到超市了你还是不知道要买什么。
-**结构化功能清单**:每个功能项有明确的状态和验证命令。新会话读取功能清单,3 分钟内知道:F01-F05 是 `passing`,F06 是 `active`(正在做),F07-F10 是 `not_started`。直接从 F06 继续,零重复。
+**脊梁骨模式**:每个功能项有明确的状态和验证命令。新会话读取功能清单,3 分钟内知道:F01-F05 是 `passing`,F06 是 `active`(正在做),F07-F10 是 `not_started`。直接从 F06 继续,零重复。
定量结果:使用结构化功能清单的项目,功能完成率比自由形式高 45%,零重复实现。
## 关键要点
-- **功能清单是 harness 的核心数据结构**,不是给人看的规划文档。调度器、验证器、交接器都依赖它。
-- **每个功能项必须有三元组**:行为描述 + 验证命令 + 当前状态。缺一项就不完整。
+- **功能清单是 harness 的脊梁骨**,不是给人看的备忘录。调度器、验证器、交接器都依赖它。
+- **每个功能项必须有三元组**:行为描述 + 验证命令 + 当前状态。缺一项就不完整——就像三条腿的凳子少一条腿。
- **状态转移由 harness 控制**,agent 不能自己改状态。通过验证 = 唯一的升级路径。
- **功能清单是项目的单一权威来源**——任何关于"该做什么"的信息都从这里派生。
-- **粒度控制在"一次会话能完成"的范围**。
+- **粒度控制在"一次会话能完成"的范围**。太粗做不完,太细管不过来。
## 延伸阅读
diff --git a/docs/zh/lectures/lecture-09-why-agents-declare-victory-too-early/index.md b/docs/zh/lectures/lecture-09-why-agents-declare-victory-too-early/index.md
index 64345d0..d9ab4db 100644
--- a/docs/zh/lectures/lecture-09-why-agents-declare-victory-too-early/index.md
+++ b/docs/zh/lectures/lecture-09-why-agents-declare-victory-too-early/index.md
@@ -5,20 +5,17 @@
# 第九讲. 防止 agent 提前宣告完成
-## 这节课要解决什么问题
+你让 agent 实现"密码重置"功能。它改了数据库 schema、写了 API 端点、加了邮件模板,跑了单元测试(全部通过),然后自信地告诉你"做完了"。你实际一跑——密码重置链接发不出去(邮件服务配置缺失)、数据库迁移半途失败(schema 不一致)、端到端流程根本没走过一遍。
-你让 agent 实现"密码重置"功能。它改了数据库 schema、写了 API 端点、加了邮件模板,跑了单元测试(全部通过),然后自信地告诉你"做完了"。但实际一跑——密码重置链接发不出去(邮件服务配置缺失)、数据库迁移半途失败(schema 不一致)、端到端流程根本没走过一遍。
+这种感觉你一定不陌生——考试时把卷子写得满满当当,信心满满地第一个交卷,结果成绩出来不及格。卷子写满了,不代表做对了。
这不是偶然事件。Guo 等人 2017 年在 ICML 上的经典论文证明:**现代神经网络系统性地过度自信**——模型自报的置信度显著高于实际准确率。AI 编码 agent 也一样:它"觉得"做完了,但实际上差得远。你的 harness 必须用外部化的、基于执行的验证来替代 agent 的"感觉"。
-## 核心概念
+## 滑坡效应
-- **过早完成声明**:agent 断言任务完成,但实际上存在未满足的正确性规范。核心问题:agent 依据代码层面的局部信心做判断,系统级正确性需要全局验证。
-- **置信度校准偏差**:agent 自报的完成信心与实际完成质量之间的系统性差距。对复杂多文件任务,这个偏差显著为正——agent 总是比实际做得更自信。
-- **终止标准**:一组明确的、可执行的判定条件,定义在 harness 里。agent 必须满足所有条件才能声明完成。"完成"从主观判断变成了客观判定。
-- **验证-确认双闸门**:第一层验证检查"代码是否正确实现了指定行为";第二层确认检查"系统级行为是否满足端到端需求"。两层都通过才算完成。
-- **运行时反馈信号**:来自程序执行的日志、进程状态、健康检查。这是 harness 判定完成质量的客观基础,不是可选的调试工具。
-- **完成优先级约束**:先验证功能正确性,再处理性能,最后管风格。核心功能没验证通过之前,不许做重构。
+过早完成声明几乎总是一样的套路:代码看着还行——语法正确、逻辑似乎合理,静态检查没有明显错误。但 harness 没有强制要求全面执行验证,agent 跳过了实际运行或只跑了部分测试。跑了单元测试但跳过集成测试,跑了测试但没检查覆盖率。最后"代码看起来没问题"就被当作了"功能已完成"的证据。交卷了。
+
+每一步都在丢失信息。从任务规范到代码实现到运行时行为,每次转换都可能引入偏差,而每次跳过的验证都加剧了信息不对称。
## 三层终止检查
@@ -37,36 +34,38 @@ flowchart LR
C --> D["于是 agent 太早宣告完成"]
```
-## 为什么会这样
-
-### Agent 完成判定的四步滑坡
+## 核心概念
-过早完成声明几乎总是遵循同一个序列:
+- **过早完成声明**:agent 断言任务完成,但实际上存在未满足的正确性规范。核心问题:agent 依据代码层面的局部信心做判断,系统级正确性需要全局验证。
+- **置信度校准偏差**:agent 自报的完成信心与实际完成质量之间的系统性差距。对复杂多文件任务,这个偏差显著为正——agent 总是比实际做得更自信。就像学生考完试估分总是偏高。
+- **终止标准**:一组明确的、可执行的判定条件,定义在 harness 里。agent 必须满足所有条件才能声明完成。"完成"从主观判断变成了客观判定。
+- **验证-确认双闸门**:第一层验证检查"代码是否正确实现了指定行为";第二层确认检查"系统级行为是否满足端到端需求"。两层都通过才算完成。
+- **运行时反馈信号**:来自程序执行的日志、进程状态、健康检查。这是 harness 判定完成质量的客观基础。
+- **完成优先级约束**:先验证功能正确性,再处理性能,最后管风格。核心功能没验证通过之前,不许做重构。
-1. **局部代码看起来没问题**:语法正确、逻辑似乎合理。静态检查层面没有明显错误。
-2. **运行时行为没有充分观测**:harness 没强制要求全面执行验证,agent 跳过了实际运行或只跑了部分测试。
-3. **验证被跳过或不完整**:跑了单元测试但跳过集成测试,或者跑了测试但没检查覆盖率。
-4. **基于不充分证据断言完成**:"代码看起来没问题"被当作"功能已完成"的证据。
+## 单元测试通过 ≠ 任务完成
-每一步都在丢失信息。从任务规范到代码实现到运行时行为,每次转换都可能引入偏差,而每次跳过的验证都加剧了信息不对称。
+这是最常见的陷阱,也是最危险的一个。agent 写了代码,跑了单元测试,全部绿色,然后说"做完了"。但单元测试的设计哲学——隔离被测单元、模拟依赖——恰好使其无法检测跨组件问题:
-### 单元测试通过 ≠ 任务完成
+**接口不匹配**:渲染进程传给预加载脚本的文件路径是相对路径,但预加载脚本期望绝对路径。各自的单元测试都用了 mock,都通过了。只有端到端跑通时才发现问题。就像乐队里每个乐手各自练习都完美,但合在一起才发现定调不一样。
-这是最常见的陷阱。agent 写了代码,跑了单元测试,全部绿色,然后说"做完了"。但单元测试的设计哲学——隔离被测单元、模拟依赖——恰好使其无法检测跨组件问题:
+**状态传播错误**:数据库迁移改了表结构,但 ORM 的缓存层还持有旧结构的缓存条目。单元测试每次都是全新的 mock 环境,不会暴露这种跨层状态不一致。
-- **接口不匹配**:A 调 B 时传参格式正确,但 B 的实际行为与 A 的假设不符。各自单元测试都通过,放在一起就崩。
-- **状态传播错误**:数据库迁移改了表结构,但缓存层还持有旧结构的缓存。单元测试不测跨层状态。
-- **环境依赖**:代码在测试环境(mock 一切)行为正确,在真实环境因为配置差异而失败。
+**环境依赖性**:代码在测试环境(一切 mock)行为正确,在真实环境因配置差异、网络延迟、服务不可用而失败。就像在排练室唱得很好,上台演出时音响设备出了问题。
### "顺便重构"是完成判定的毒药
-Claude Code 有一个常见行为模式:在核心功能还没验证通过时就开始重构代码、优化性能、改进风格。Knuth 说的"过早优化是万恶之源"在 agent 场景中有了新含义——重构会改变已完成验证和未完成验证之间的边界,可能破坏之前隐式正确的代码路径。
+Claude Code 有一个常见行为模式:在核心功能还没验证通过时就开始重构代码、优化性能、改进风格。Knuth 说的"过早优化是万恶之源"在 agent 场景中有了新含义——重构会改变已完成验证和未完成验证之间的边界,可能破坏之前隐式正确的代码路径。就像你数学大题还没做完,就跑去把前面选择题的答案重新抄一遍格式——浪费时间不说,还可能抄错了。
### 自我评价的系统性偏差
-Anthropic 在 2026 年的研究中发现了一个更深层的失败模式:**当 agent 被要求评估自己的工作时,它系统性地过度正面评价——即使人类观察者认为质量明显不达标。** 这个问题在主观任务(如设计审美)上尤其严重——"布局是否精致"是一个判断题,agent 可靠地偏向正面。即使在有可验证结果的任务上,agent 也会因为判断失误而影响表现。
+Anthropic 在 2026 年的研究中发现了一个更深层的失败模式:**当 agent 被要求评估自己的工作时,它系统性地过度正面评价——即使人类观察者认为质量明显不达标。** 这就像让学生自己给自己判卷——对自己的答案总是特别宽容。
+
+这个问题在主观任务(如设计审美)上尤其严重——"布局是否精致"是一个判断题,agent 可靠地偏向正面。即使在有可验证结果的任务上,agent 也会因为判断失误而影响表现。
-解决方案不是让 agent "更客观"——同一个模型既生成又评估,内在地倾向对自己慷慨。**解决方案是把"干活的人"和"检查的人"分开。** 一个独立的评估 agent,经过专门调校为"挑剔"之后,比让生成 agent 自我评估有效得多。Anthropic 的实验数据:
+解决方案不是让 agent "更客观"——同一个模型既生成又评估,内在地倾向对自己慷慨。**解决方案是把"干活的人"和"检查的人"分开。** 就像考试不能让学生自己批改自己的卷子——得有个独立的阅卷老师。
+
+一个独立的评估 agent,经过专门调校为"挑剔"之后,比让生成 agent 自我评估有效得多。Anthropic 的实验数据:
| 架构 | 运行时长 | 成本 | 核心功能是否可用 |
|------|---------|------|---------------|
@@ -77,11 +76,11 @@ Anthropic 在 2026 年的研究中发现了一个更深层的失败模式:**
> 来源:[Anthropic: Harness design for long-running application development](https://www.anthropic.com/engineering/harness-design-long-running-apps)
-## 怎么做才对
+## 怎么防止提前交卷
### 1. 外部化终止判定
-完成判定不应该由 agent 自己做。harness 独立执行终止校验,输入是运行时信号,不是 agent 的置信度。在 CLAUDE.md 里写:
+完成判定不应该由 agent 自己做。harness 独立执行终止校验,输入是运行时信号,不是 agent 的置信度。在 CLAUDE.md 里写清楚:
```
## 完成定义
@@ -96,18 +95,17 @@ Anthropic 在 2026 年的研究中发现了一个更深层的失败模式:**
### 2. 构建三层终止校验
-- **第一层:语法与静态分析**。成本最低,信息量最小,但必须通过。
-- **第二层:运行时行为验证**。测试执行、应用启动检查、关键路径验证。这是核心完成证据。
-- **第三层:系统级确认**。端到端测试、集成验证、用户场景模拟。防止过早声明的最后一道防线。
+- **第一层:语法与静态分析**。成本最低,信息量最小,但必须通过。这是最低限度的检查——字都没写错才能往下看。
+- **第二层:运行时行为验证**。测试执行、应用启动检查、关键路径验证。这是核心完成证据。不仅要写了,还要能跑。
+- **第三层:系统级确认**。端到端测试、集成验证、用户场景模拟。防止过早声明的最后一道防线。不仅要能跑,还要跑对。
-### 3. 为 agent 设计错误消息
+### 3. 为 agent 设计好的"红笔批注"
-OpenAI 在 Codex 实践中提出了一个特别有效的模式:**给 agent 写的错误消息要包含修复指导**。不要用 `"Test failed"`,而用 `"Test failed: POST /api/reset-password returned 500. Check that the email service config exists in environment variables. The template file should be at templates/reset-email.html."` 这种具体的、可操作的反馈让 agent 能自我修正,而不需要人类介入。
+OpenAI 在 Codex 实践中提出了一个特别有效的模式:**给 agent 写的错误消息要包含修复指导**。不要像阅卷老师只画个大红叉,要像好老师一样在旁边写上"这里应该怎么改"。不要用 `"Test failed"`,而用 `"Test failed: POST /api/reset-password returned 500. Check that the email service config exists in environment variables. The template file should be at templates/reset-email.html."` 这种具体的、可操作的反馈让 agent 能自我修正,而不需要人类介入。
-### 4. 运行时信号的具体形式
+### 4. 捕获运行时信号
有效的运行时信号包括:
-
- 应用是否成功启动并达到就绪状态?
- 关键功能路径在运行时是否执行成功?
- 数据库写入、文件操作等副作用是否正确?
@@ -117,18 +115,18 @@ OpenAI 在 Codex 实践中提出了一个特别有效的模式:**给 agent 写
**任务**:实现用户密码重置功能。涉及数据库操作、邮件发送和 API 端点修改。
-**过早声明路径**:agent 修改数据库 schema、编写 API 端点、添加邮件模板、跑单元测试(通过)、声明完成。
+**提前交卷路径**:agent 修改数据库 schema、编写 API 端点、添加邮件模板、跑单元测试(通过)、声明完成。卷子写得满满当当。
-**实际缺陷**:(1) 端到端流程未测试——重置链接的实际发送和验证未确认。(2) 数据库迁移在部分执行后失败,导致 schema 不一致。(3) 邮件服务配置在目标环境中缺失。
+**实际扣分项**:(1) 端到端流程未测试——重置链接的实际发送和验证未确认。(2) 数据库迁移在部分执行后失败,导致 schema 不一致。(3) 邮件服务配置在目标环境中缺失。
-**harness 介入**:终止校验强制执行——(1) 启动完整应用验证重置端点可访问;(2) 执行完整重置流程;(3) 验证数据库状态一致性。所有缺陷在会话内被发现,节省了 5-10 倍的后续修复成本。
+**harness 介入**:终止校验强制执行——(1) 启动完整应用验证重置端点可访问;(2) 执行完整重置流程;(3) 验证数据库状态一致性。所有缺陷在会话内被发现,节省了 5-10 倍的后续修复成本。独立阅卷老师批出了真正的问题。
## 关键要点
-- **agent 系统性地过度自信**——置信度校准偏差是客观存在的,不是主观印象。
-- **完成判定必须外部化**——harness 独立验证,不信任 agent 的"感觉"。
+- **agent 系统性地过度自信**——置信度校准偏差是客观存在的。卷子写满了不代表做对了。
+- **完成判定必须外部化**——harness 独立验证,不信任 agent 的"感觉"。不能让学生自己批自己的卷子。
- **三层校验缺一不可**——语法通过、行为通过、系统通过,层层递进。
-- **错误消息要面向 agent 设计**——包含具体修复步骤,让 agent 能自我修正。
+- **错误消息要像好老师的红笔批注**——包含具体修复步骤,让 agent 能自我修正。
- **核心功能验证通过之前不许重构**——完成优先级约束是防止过早优化的关键。
## 延伸阅读
diff --git a/docs/zh/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md b/docs/zh/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md
index 691c561..d68568a 100644
--- a/docs/zh/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md
+++ b/docs/zh/lectures/lecture-10-why-end-to-end-testing-changes-results/index.md
@@ -5,19 +5,31 @@
# 第十讲. 跑通完整流程才算真正验证
-## 这节课要解决什么问题
-
你让 agent 给 Electron 应用加一个文件导出功能。它写了渲染进程组件、预加载脚本、服务层逻辑,每个组件的单元测试都通过了。agent 说"做完了"。你实际一点击导出按钮——文件路径格式不对、进度条没反应、大文件导出时内存泄漏。5 个组件边界缺陷,单元测试一个都没发现。
-Google 的测试金字塔告诉我们:大量单元测试是基础,但如果你止步于此,就会系统性地漏掉组件交互问题。对于 AI 编码 agent 来说,这个问题更严重——agent 倾向于只跑最快的测试然后宣告完成。**只有端到端测试能证明系统级缺陷不存在**(比仅通过单元测试更强)。
+这就像一个合唱团排练——每个声部单独唱的时候都完美,但合在一起的时候,女高音比男低音快了半拍,伴奏的调子和主旋律差了半个音。每个部分都"对"了,但整体跑调了。
-## 核心概念
+Google 的测试金字塔告诉我们:大量单元测试是基础,但如果你止步于此,就会系统性地漏掉组件交互问题。对于 AI 编码 agent 来说,这个问题更严重——agent 倾向于只跑最快的测试然后宣告完成。**只有端到端测试能证明系统级缺陷不存在**。
-- **组件边界缺陷**:组件 A 和 B 各自单元测试通过,但它们的交互产生了不正确的行为。这是端到端测试最擅长捕获的问题类型。
-- **测试充分性梯度**:单元测试能检测的缺陷 <= 集成测试能检测的缺陷 <= 端到端测试能检测的缺陷。每往上一层,检测能力增强。
-- **架构边界执行规则**:把架构文档里的规则(如"渲染进程不能直接访问文件系统")变成可执行的自动化检查。
-- **审查反馈提升**:把重复出现的代码审查意见转化为自动化测试。每次发现重复问题就加一条规则,harness 会自动变强。
-- **面向 agent 的错误消息**:失败信息不只是说"出了什么问题",还要告诉 agent 具体怎么修。这把测试失败变成自我修正的反馈循环。
+## 单元测试的盲区
+
+单元测试的设计哲学是隔离——模拟依赖,专注被测单元。这个哲学使单元测试快速且精确,但也制造了系统性的盲区。就像合唱排练时每个声部戴着耳机对着伴奏唱——听着都挺好,但真正合在一起才发现问题:
+
+**接口不匹配**:渲染进程传给预加载脚本的文件路径是相对路径,但预加载脚本期望绝对路径。各自的单元测试都用了 mock,都通过了。只有端到端跑通时才发现问题——就像两个声部各自练的时候都觉得节奏没问题,一合才发现一个用 4/4 拍一个用 3/4 拍。
+
+**状态传播错误**:数据库迁移改了表结构,但 ORM 的缓存层还持有旧结构的缓存条目。单元测试每次都是全新的 mock 环境,不会暴露这种跨层状态不一致。就像换了一首歌的歌词,但有人还在唱旧版本。
+
+**资源生命周期问题**:文件句柄、数据库连接、网络套接字的获取和释放跨越多个组件。单元测试为每个测试创建和销毁独立资源,不会暴露资源竞争或泄漏。就像排练时每个声部轮流用麦克风,但演出时所有声部同时上台——话筒不够用了。
+
+**环境依赖性**:代码在测试环境(一切 mock)行为正确,在真实环境因配置差异、网络延迟、服务不可用而失败。就像排练厅里唱得好好的,到了户外音乐节风一吹话筒一啸叫就全乱了。
+
+## 端到端测试不仅改变结果,还改变行为
+
+这是很多人没意识到的一点:当 agent 知道它的工作要过端到端测试时,它的编码行为会改变。
+
+1. **考虑组件交互**:写代码时会想"这个接口和上游怎么对接",而不是只关注单个函数。就像知道最终要合在一起唱,练习的时候就会注意听其他声部。
+2. **尊重架构边界**:有架构约束的系统里,端到端测试迫使 agent 遵守边界规则。就像乐谱上标注了"此处渐强",你得跟着来。
+3. **处理错误路径**:端到端测试通常包含故障场景,迫使 agent 考虑异常处理。就像排练时模拟了"话筒突然没声了"的情况,你知道该怎么做。
## 测试金字塔与审查反馈提升
@@ -45,59 +57,33 @@ flowchart LR
Harness --> Stronger["以后再犯会第一时间报错"]
```
-## 为什么会这样
-
-### 单元测试的盲区
-
-单元测试的设计哲学是隔离——模拟依赖,专注被测单元。这个哲学使单元测试快速且精确,但也制造了系统性的盲区:
+OpenAI 在 Codex 工程实践中强调:**为 agent 写的错误消息必须包含修复指导**。不写 `"Direct filesystem access in renderer"`,而写 `"Direct filesystem access in renderer. All file operations must go through the preload bridge. Move this call to preload/file-ops.ts and invoke it via window.api."` 这把架构规则变成了自动修正的闭环。就像合唱排练时指挥不只说"你唱错了",而是说"这里你快了半拍,听一下女低音的节奏,在第 32 小节进入"。
-**接口不匹配**:渲染进程传给预加载脚本的文件路径是相对路径,但预加载脚本期望绝对路径。各自的单元测试都用了 mock,都通过了。只有端到端跑通时才发现问题。
-
-**状态传播错误**:数据库迁移改了表结构,但 ORM 的缓存层还持有旧结构的缓存条目。单元测试每次都是全新的 mock 环境,不会暴露这种跨层状态不一致。
-
-**资源生命周期问题**:文件句柄、数据库连接、网络套接字的获取和释放跨越多个组件。单元测试为每个测试创建和销毁独立资源,不会暴露资源竞争或泄漏。
-
-**环境依赖性**:代码在测试环境(一切 mock)行为正确,在真实环境因配置差异、网络延迟、服务不可用而失败。
-
-### 端到端测试不仅改变结果,还改变行为
-
-当 agent 知道它的工作要过端到端测试时,它的编码行为会改变:
-
-1. **考虑组件交互**:写代码时会想"这个接口和上游怎么对接",而不是只关注单个函数。
-2. **尊重架构边界**:有架构约束的系统里,端到端测试迫使 agent 遵守边界规则。
-3. **处理错误路径**:端到端测试通常包含故障场景,迫使 agent 考虑异常处理。
-
-OpenAI 在 Codex 工程实践中强调:**为 agent 写的错误消息必须包含修复指导**。不写 `"Direct filesystem access in renderer"`,而写 `"Direct filesystem access in renderer. All file operations must go through the preload bridge. Move this call to preload/file-ops.ts and invoke it via window.api."` 这把架构规则变成了自动修正的闭环。
-
-### 审查反馈提升:让 harness 越用越强
-
-人类审查者经常对同一类问题反复提意见。审查反馈提升把这个模式变成系统改进:
-
-1. 识别代码审查中重复出现的意见类型
-2. 把意见转化为可执行的正则表达式、AST 模式或运行时断言
-3. 编写包含面向 agent 错误消息的自动化检查
-4. 集成到 harness 的验证流程
-5. 用基准任务验证检查有效性
+## 核心概念
-每捕获一个缺陷类别,就增加一条永久的防线。harness 随时间自动强化。
+- **组件边界缺陷**:组件 A 和 B 各自单元测试通过,但它们的交互产生了不正确的行为。这是端到端测试最擅长捕获的问题类型——合唱里各声部单独都对但合起来跑调的那种。
+- **测试充分性梯度**:单元测试能检测的缺陷 <= 集成测试能检测的缺陷 <= 端到端测试能检测的缺陷。每往上一层,检测能力增强。
+- **架构边界执行规则**:把架构文档里的规则(如"渲染进程不能直接访问文件系统")变成可执行的自动化检查。从"写在纸上"变成"跑在 CI 里"。
+- **审查反馈提升**:把重复出现的代码审查意见转化为自动化测试。每次发现重复问题就加一条规则,harness 会自动变强。就像合唱排练时指挥把常见的错误编成练习曲——下次再犯同样的错误,不用指挥说,练习曲自己就暴露了。
+- **面向 agent 的错误消息**:失败信息不只是说"出了什么问题",还要告诉 agent 具体怎么修。这把测试失败变成自我修正的反馈循环。
-## 怎么做才对
+## 怎么做
-### 0. 架构执行:在写端到端测试之前先定义边界
+### 0. 先定好架构边界,再写端到端测试
-端到端测试的前提是系统有清晰的边界。如果架构是一团面条,端到端测试只会证明"这团面条整体能跑",不会告诉你哪里违反了设计意图。
+端到端测试的前提是系统有清晰的边界。如果架构是一团面条,端到端测试只会证明"这团面条整体能跑",不会告诉你哪里违反了设计意图。就像合唱团如果连分声部都没分好,排练再多也是乱唱。
OpenAI 的经验:**对 agent 生成的代码库,架构约束必须是第一天就建立的早期前置条件,不是等团队规模大了再考虑的事。** 原因很直接——agent 会复制仓库中已有的模式,即使那些模式是不均匀的或次优的。没有架构约束,agent 会在每次会话中引入更多偏差。
-OpenAI 采用了"分层领域架构"——每个业务领域(如"用户设置"、"支付")被分成固定的层:Types → Config → Repo → Service → Runtime → UI。依赖方向严格向前,跨领域关注点(认证、遥测、功能开关)通过一个显式的 Providers 接口进入。任何其他依赖都是禁止的,并且通过自定义 lint 机械执行。
+OpenAI 采用了"分层领域架构"——每个业务领域被分成固定的层:Types → Config → Repo → Service → Runtime → UI。依赖方向严格向前,跨领域关注点通过显式的 Providers 接口进入。任何其他依赖都是禁止的,并且通过自定义 lint 机械执行。
-关键原则:**执行不变量,不微管实现。** 比如要求"数据在边界解析",但不规定用哪个库(模型自己倾向 Zod,但 harness 没指定)。错误消息要包含修复指导——不只是说"违规了",而是告诉 agent 具体怎么改。这种面向 agent 的 lint 比"人类看的"文档有效得多。
+关键原则:**执行不变量,不微管实现。** 比如要求"数据在边界解析",但不规定用哪个库。错误消息要包含修复指导——不只是说"违规了",而是告诉 agent 具体怎么改。
> 来源:[OpenAI: Harness engineering: leveraging Codex in an agent-first world](https://openai.com/index/harness-engineering/)
### 1. harness 必须包含端到端层
-在你的验证流程里明确:对于涉及跨组件修改的任务,端到端测试通过是完成的前置条件。在 CLAUDE.md 里写:
+在你的验证流程里明确:对于涉及跨组件修改的任务,端到端测试通过是完成的前置条件:
```
## 验证层级
@@ -128,15 +114,15 @@ FIX: Move file operations to src/preload/file-ops.ts and call via window.api.rea
### 4. 建立审查反馈提升流程
-每次在代码审查中发现新类型的 agent 错误,就把它变成自动化检查。一个月后你的 harness 会比月初强得多。
+每次在代码审查中发现新类型的 agent 错误,就把它变成自动化检查。一个月后你的 harness 会比月初强得多。就像合唱团的排练笔记——每次排练发现的问题都记下来,下次排练前先检查这些点。久而久之,常见错误越来越少,音乐越来越和谐。
## 实际案例
**任务**:在 Electron 应用中实现文件导出功能。涉及渲染进程 UI、预加载脚本文件系统代理、服务层数据转换。
-**单元测试通过**:渲染组件测试(通过,mock 文件操作)、预加载脚本测试(通过,mock 文件系统)、服务层测试(通过,mock 数据源)。agent 声明完成。
+**各声部单独唱(单元测试通过)**:渲染组件测试(通过,mock 文件操作)、预加载脚本测试(通过,mock 文件系统)、服务层测试(通过,mock 数据源)。agent 声明完成。
-**端到端测试揭示的缺陷**:
+**合唱合在一起(端到端测试揭示的缺陷)**:
| 缺陷 | 描述 | 单元测试 | 端到端 |
|------|------|---------|--------|
@@ -146,11 +132,11 @@ FIX: Move file operations to src/preload/file-ops.ts and call via window.api.rea
| 权限问题 | 打包环境权限不同 | 未检测 | 检测 |
| 错误传播 | 服务层异常未到 UI 层 | 未检测 | 检测 |
-5 个缺陷全部被端到端测试捕获,单元测试一个都没发现。代价是测试时间从 2 秒增加到 15 秒——在 agent 工作流里完全可以接受。
+5 个缺陷全部被端到端测试捕获,单元测试一个都没发现。代价是测试时间从 2 秒增加到 15 秒——在 agent 工作流里完全可以接受。每个声部单独唱得再好,也比不上一次完整的合唱排练。
## 关键要点
-- **单元测试对组件边界缺陷系统性盲视**——它们的隔离设计恰好使其无法检测交互问题。
+- **单元测试对组件边界缺陷系统性盲视**——它们的隔离设计恰好使其无法检测交互问题。每个人唱得都对,不代表合唱不跑调。
- **端到端测试不仅检测缺陷,还改变 agent 的编码行为**——让它更关注集成和边界。
- **架构规则必须可执行**——不是写在文档里等人来看,而是每次提交自动检查。
- **错误消息要面向 agent 设计**——包含"怎么修"的具体步骤,形成自我修正闭环。
diff --git a/docs/zh/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md b/docs/zh/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md
index 443b613..7667573 100644
--- a/docs/zh/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md
+++ b/docs/zh/lectures/lecture-11-why-observability-belongs-inside-the-harness/index.md
@@ -5,23 +5,38 @@
# 第十一讲. 让 agent 的运行过程可观测
-## 这节课要解决什么问题
-
你让 agent 做一个功能,它跑了 20 分钟,改了一堆文件,然后告诉你"做完了但有两个测试失败"。你问它为什么失败,它说"不太确定,可能是时序问题"。你问它改了哪些关键路径,它说"让我看看代码……"。
-这不是 agent 能力不够,是你的 harness 没有给它提供足够的可观测性。**没有可观测性,agent 在不确定状态中做决策,评估变成主观判断,重试变成盲目摸索。** OpenAI 和 Anthropic 都将可靠性定义为证据问题——harness 必须以可指导下一步决策的形式暴露运行时行为和评估信号。
+这不是 agent 能力不够,是你的 harness 没有给它装仪表盘。想象你在开一辆没有仪表盘的车——没有速度表、没有油量表、没有发动机故障灯。你能开,但你不知道开多快、还剩多少油、发动机是不是快爆了。技术再好的司机,蒙着眼也得出事。
-## 核心概念
+**没有可观测性,agent 在不确定状态中做决策,评估变成主观判断,重试变成盲目摸索。** OpenAI 和 Anthropic 都将可靠性定义为证据问题——harness 必须以可指导下一步决策的形式暴露运行时行为和评估信号。
-- **运行时可观测性**:系统层的信号——日志、追踪、进程事件、健康检查。回答"系统做了什么"。
-- **过程可观测性**:harness 决策工件的可见性——计划、评分标准、验收条件。回答"为什么这个变更应该被接受"。
-- **任务轨迹**:一个任务从开始到完成的完整决策路径记录,类似分布式系统中的请求追踪。agent 的每一步操作及其上下文都被记录。
-- **冲刺合同**:编码开始前协商的短期协议——明确任务范围、验证标准、排除项。是过程可观测性的核心工具。
-- **评估评分标准**:把质量评估从主观判断变成基于证据的结构化评分。使不同评估者对同一输出产生相似结论。
-- **双层可观测性**:系统层和过程层同时设计、相互增强。运行时信号解释行为,过程工件解释意图。
+## 可观测性缺失的真实代价
+
+当 harness 缺乏可观测性时,四类问题系统性出现:
+
+**无法区分"正确"和"看似正确"**:一个函数在代码审查时看起来完全正确——语法对、逻辑通。但运行时因为边界条件处理错误,在特定输入下产生了不正确结果。只有运行时追踪能揭示实际执行路径偏离了预期。就像一个演员排练时台词全对了,但上台演出时灯光一打,表情和走位全都变了——你不看现场是发现不了的。
+
+**评估变成玄学**:没有评分标准和验收条件时,评估者(人或 agent)依赖隐式假设。同一个输出,不同评估者可能给出截然不同的评价。质量评估不可复现。就像体操比赛没有评分标准——这个裁判觉得你的动作优雅,那个裁判觉得你落地不稳,谁说了算?
+
+**重试变成盲猜**:agent 不知道为什么失败时,重试方向是随机的。它可能在错误的方向上反复尝试——修复了不相关的代码路径而忽略真正的故障根源。就像你开车发现车跑偏了,但你没有仪表盘——你猜是轮胎的问题换了轮胎,实际是方向盘的 alignment 出了问题。每次盲重试都消耗 token 和时间。
+
+**会话交接信息断崖**:当未完成的工作移交给下一个会话时,缺乏可观测性意味着新会话必须从零诊断系统状态。Anthropic 的长期运行 agent 观察表明,这种重复诊断可能占会话总时间的 30-50%。就像换班司机上车发现没有交接记录——他得花半小时检查油量、胎压、发动机状态才能出发。
+
+## Claude Code 的真实场景
+
+想象一个使用"计划者-生成者-评估者"三角色工作流的 harness,执行"为应用添加暗色模式"任务。
+
+**没有仪表盘**:计划者输出模糊描述。生成者根据模糊描述实现暗色模式,但和计划者的隐式预期不一致。评估者基于自己的隐式标准拒绝,但说不出具体哪里不对——"感觉不太对"。生成者基于模糊拒绝理由盲重试。循环 3-4 次,总耗时约 45 分钟,最终勉强产出。
+
+**有完整仪表盘**:计划者输出冲刺合同——列明要改哪些组件、每个组件的验证标准、排除项(不处理打印样式)。生成者按合同实现。运行时可观测性记录每个组件的样式加载和应用过程。评估者用评分标准逐维度评估,附具体证据引用——"按钮颜色对比度不足(WCAG AA 标准 4.5:1,实测 2.1:1)"。一次迭代出高质量结果,总耗时约 15 分钟。
+
+效率差 3 倍。区别只在可观测性——给车装上了仪表盘。
## 双层可观测性
+可观测性不是"多打点日志"那么简单。它分两层,缺一不可:
+
```mermaid
flowchart LR
Contract["先把这次任务写清楚<br/>改哪些文件 / 不改哪些部分 / 怎么算通过"] --> Generator["生成器"]
@@ -32,43 +47,32 @@ flowchart LR
Verdict --> Generator
```
-## 为什么会这样
-
-### 可观测性缺失的真实代价
+**运行时可观测性**:系统层的信号——日志、追踪、进程事件、健康检查。回答"系统做了什么"。这是你车上的仪表盘——速度、油量、发动机温度。
-当 harness 缺乏可观测性时,四类问题系统性出现:
-
-**无法区分"正确"和"看似正确"**:一个函数在代码审查时看起来完全正确——语法对、逻辑通。但运行时因为边界条件处理错误,在特定输入下产生了不正确结果。只有运行时追踪能揭示实际执行路径偏离了预期。
-
-**评估变成玄学**:没有评分标准和验收条件时,评估者(人或 agent)依赖隐式假设。同一个输出,不同评估者可能给出截然不同的评价。质量评估不可复现。
-
-**重试变成盲猜**:agent 不知道为什么失败时,重试方向是随机的。它可能在错误的方向上反复尝试——修复了不相关的代码路径而忽略真正的故障根源。每次盲重试都消耗 token 和时间。
-
-**会话交接信息断崖**:当未完成的工作移交给下一个会话时,缺乏可观测性意味着新会话必须从零诊断系统状态。Anthropic 的长期运行 agent 观察表明,这种重复诊断可能占会话总时间的 30-50%。
+**过程可观测性**:harness 决策工件的可见性——计划、评分标准、验收条件。回答"为什么这个变更应该被接受"。这是你的导航系统——不光知道现在在哪,还知道为什么走这条路。
-### Claude Code 的真实场景
-
-想象一个使用"计划者-生成者-评估者"三角色工作流的 harness,执行"为应用添加暗色模式"任务。
-
-**没有可观测性**:计划者输出模糊描述。生成者根据模糊描述实现暗色模式,但和计划者的隐式预期不一致。评估者基于自己的隐式标准拒绝,但说不出具体哪里不对。生成者基于模糊拒绝理由盲重试。循环 3-4 次,总耗时约 45 分钟,最终勉强产出。
-
-**有完整可观测性**:计划者输出冲刺合同——列明要改哪些组件、每个组件的验证标准、排除项(不处理打印样式)。生成者按合同实现。运行时可观测性记录每个组件的样式加载和应用过程。评估者用评分标准逐维度评估,附具体证据引用。一次迭代出高质量结果,总耗时约 15 分钟。
+## 核心概念
-效率差 3 倍。区别只在可观测性。
+- **运行时可观测性**:系统层的信号——日志、追踪、进程事件、健康检查。回答"系统做了什么"。
+- **过程可观测性**:harness 决策工件的可见性——计划、评分标准、验收条件。回答"为什么这个变更应该被接受"。
+- **任务轨迹**:一个任务从开始到完成的完整决策路径记录,类似分布式系统中的请求追踪。agent 的每一步操作及其上下文都被记录。就像黑匣子——出了问题可以回放完整过程。
+- **冲刺合同**:编码开始前协商的短期协议——明确任务范围、验证标准、排除项。是过程可观测性的核心工具。
+- **评估评分标准**:把质量评估从主观判断变成基于证据的结构化评分。使不同评估者对同一输出产生相似结论。就像体操比赛的评分标准——有了它,十个裁判的分才不会差太远。
+- **双层可观测性**:系统层和过程层同时设计、相互增强。运行时信号解释行为,过程工件解释意图。
-### 为什么 agent 自己解决不了这个问题
+## 为什么 agent 自己解决不了这个问题
-你可能在想:"agent 不能自己打印日志吗?" 问题在于:
+你可能在想:"agent 不能自己打日志吗?" 问题在于:
-1. agent 不知道它不知道什么——它不会主动记录自己没意识到需要的信号。
-2. 日志格式不统一——不同会话用不同的日志格式,无法做系统化分析。
-3. 过程可观测性不是日志能解决的——冲刺合同和评分标准是结构化的工件,需要 harness 层面的支持。
+1. **agent 不知道它不知道什么**——它不会主动记录自己没意识到需要的信号。就像你不知道油箱快漏了的时候,你不会去看油量表——因为你压根不知道该看。
+2. **日志格式不统一**——不同会话用不同的日志格式,无法做系统化分析。就像十个司机各写各的交接记录,格式都不一样,下一个司机看不懂。
+3. **过程可观测性不是日志能解决的**——冲刺合同和评分标准是结构化的工件,需要 harness 层面的支持。不是多 print 几行就能搞定的。
-## 怎么做才对
+## 怎么装仪表盘
### 1. 在 harness 里内置运行时信号采集
-不要依赖 agent 自己打印日志。harness 应该自动采集以下信号:
+不要依赖 agent 自己打日志。harness 应该自动采集以下信号:
- **应用生命周期**:启动、就绪、运行、关闭各阶段状态
- **功能路径执行**:关键路径的执行记录,包括入口、检查点和出口
@@ -78,7 +82,7 @@ flowchart LR
### 2. 实施冲刺合同
-在每个任务开始前,生成者和评估者(可能是同一个 agent 的不同调用)协商一份合同:
+在每个任务开始前,生成者和评估者(可能是同一个 agent 的不同调用)协商一份合同——就像施工队开工前签的施工协议:
```markdown
# 冲刺合同: 暗色模式支持
@@ -100,7 +104,7 @@ flowchart LR
### 3. 建立评估评分标准
-把"好不好"变成可量化的评分:
+把"好不好"变成可量化的评分——就像给体操比赛定评分标准:
```markdown
# 评分标准
@@ -116,9 +120,7 @@ flowchart LR
为每个 harness 会话创建一个 trace,每个任务创建一个 span,每个验证步骤创建子 span。使用标准属性标注关键信息。这样可观测性数据可以和标准工具链(Jaeger、Zipkin)集成。
-## 实际案例
-
-### Anthropic 的三 agent 架构实验
+## Anthropic 的三 agent 架构实验
Anthropic 在 2026 年 3 月发布了一项系统性的 harness 实验。他们用三种架构跑同一个任务("用 Web Audio API 做一个浏览器端 DAW"),记录了详细的阶段数据:
@@ -133,40 +135,26 @@ Anthropic 在 2026 年 3 月发布了一项系统性的 harness 实验。他们
| QA 第 3 轮 | 9.6 分钟 | $4.06 |
| **总计** | **3 小时 50 分钟** | **$124.70** |
-三个 agent 各司其职:
+三个 agent 各司其职,每个都有明确的可观测性角色:
-**Planner(规划者)**:接收一段 1-4 句话的用户需求,扩展成完整产品规格。被要求"大胆设定范围"并且"专注于产品上下文和高层技术设计,而不是详细的技术实现"。原因是:如果 planner 过早指定了粒度技术细节且搞错了,错误会级联到下游实现。更好的做法是约束交付物,让 agent 在执行中自己找到路径。
+**Planner(规划者)**:接收一段 1-4 句话的用户需求,扩展成完整产品规格。被要求"大胆设定范围"并且"专注于产品上下文和高层技术设计,而不是详细的技术实现"。原因是:如果 planner 过早指定了粒度技术细节且搞错了,错误会级联到下游实现。更好的做法是约束交付物,让 agent 在执行中自己找到路径。就像建筑设计师只画效果图和结构图,不规定每块砖怎么砌。
-**Generator(生成者)**:按 sprint 逐个功能实现。每个 sprint 前和 evaluator 协商一份 sprint 合同——约定这个功能块"做完"的标准。然后按合同实现,自评后交给 QA。
+**Generator(生成者)**:按 sprint 逐个功能实现。每个 sprint 前和 evaluator 协商一份 sprint 合同——约定这个功能块"做完"的标准。然后按合同实现,自评后交给 QA。按合同施工,不按感觉施工。
-**Evaluator(评估者)**:用 Playwright MCP 像用户一样点击运行中的应用,测试 UI 功能、API 端点和数据库状态。对每个 sprint 按四个维度评分——产品深度、功能性、视觉设计和代码质量。每个维度有硬性阈值,任一不达标则 sprint 失败,generator 收到详细反馈后修复。
+**Evaluator(评估者)**:用 Playwright MCP 像用户一样点击运行中的应用,测试 UI 功能、API 端点和数据库状态。对每个 sprint 按四个维度评分——产品深度、功能性、视觉设计和代码质量。每个维度有硬性阈值,任一不达标则 sprint 失败,generator 收到详细反馈后修复。就像验收工程师拿着验收标准逐项检查——不达标就打回去重做。
-QA 第 1 轮反馈的示例——"这是一个视觉上令人印象深刻的应用,AI 集成工作良好,但核心 DAW 功能有几个是展示性的,没有交互深度:剪辑不能拖拽/移动,没有乐器 UI 面板(合成器旋钮、鼓垫),没有视觉效果编辑器(EQ 曲线、压缩器仪表)"。这些不是边缘情况——它们是让 DAW 可用的核心交互。
+QA 第 1 轮反馈的示例——"这是一个视觉上令人印象深刻的应用,AI 集成工作良好,但核心 DAW 功能有几个是展示性的,没有交互深度:剪辑不能拖拽/移动,没有乐器 UI 面板(合成器旋钮、鼓垫),没有视觉效果编辑器(EQ 曲线、压缩器仪表)"。这些不是边缘情况——它们是让 DAW 可用的核心交互。具体的、有证据的反馈,不是"感觉不对"。
-Evaluator 不是一开始就这么强。早期版本会识别出合理的问题,然后说服自己这些问题不严重,最终批准工作。调校方式是:读 evaluator 的日志,找到它的判断和人类判断分叉的地方,更新 QA 的 prompt 解决那些问题。经过几轮这种开发循环,evaluator 的评分才变得合理。
+Evaluator 不是一开始就这么强。早期版本会识别出合理的问题,然后说服自己这些问题不严重,最终批准工作。调校方式是:读 evaluator 的日志,找到它的判断和人类判断分叉的地方,更新 QA 的 prompt 解决那些问题。经过几轮这种开发循环,evaluator 的评分才变得合理。就像训练一个新验收工程师——一开始他太宽容,出了几次事故后学会了严格。
> 来源:[Anthropic: Harness design for long-running application development](https://www.anthropic.com/engineering/harness-design-long-running-apps)
-### 更小规模的案例
-
-一个使用计划者-生成者-评估者工作流的 harness,执行"添加暗色模式支持"任务:
-
-**不可观测版本**:3-4 轮盲重试,45 分钟,勉强可接受的输出。评估者说"感觉不太对"但说不出哪里不对。生成者在错误方向上浪费大量时间。
-
-**完整可观测版本**:
-- 冲刺合同明确了范围、标准和排除项
-- 运行时追踪记录了每个组件的样式加载过程
-- 评分标准提供了逐维度的结构化评估
-- 一次迭代出高质量结果,15 分钟
-
-效率提升 3 倍,质量更稳定,评估可复现。
-
## 关键要点
-- **可观测性是 harness 的架构属性**——不是事后添加的功能,而是设计时必须考虑的核心能力。
-- **双层可观测性缺一不可**——运行时信号解释"发生了什么",过程工件解释"为什么这样做"。
-- **冲刺合同前置对齐工作**——防止"生成者做了评估者因可预见原因立即拒绝的东西"。
-- **评分标准让评估可复现**——不同评估者对同一输出产生相似评分。
+- **可观测性是 harness 的架构属性**——不是事后添加的功能,而是设计时必须考虑的核心能力。仪表盘不是可选配件,是出厂标配。
+- **双层可观测性缺一不可**——运行时信号解释"发生了什么",过程工件解释"为什么这样做"。速度表和导航系统各有各的用处。
+- **冲刺合同前置对齐工作**——防止"生成者做了评估者因可预见原因立即拒绝的东西"。施工协议要在开工前签,不是完工后补。
+- **评分标准让评估可复现**——不同评估者对同一输出产生相似评分。有了评分标准,十个裁判的分才不会差太远。
- **可观测性缺失导致 30-50% 的会话时间浪费在重复诊断上**。
## 延伸阅读
diff --git a/docs/zh/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md b/docs/zh/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md
index d98a571..0483de8 100644
--- a/docs/zh/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md
+++ b/docs/zh/lectures/lecture-12-why-every-session-must-leave-a-clean-state/index.md
@@ -5,22 +5,31 @@
# 第十二讲. 每次会话结束前都做好交接
-## 这节课要解决什么问题
-
你的 agent 跑了一下午,改了 20 个文件,提交了代码,会话结束。下一个 agent 会话开始,一上来就发现:构建失败了、测试红了、临时调试文件到处都是、功能清单没更新、进度完全不清楚。新会话的前 30 分钟全花在"搞清楚上一个会话到底干了什么"上。
-OpenAI 和 Anthropic 都明确指出:**长期可靠性取决于操作纪律,不仅是单次运行的成功。** 每个会话结束时的状态质量,直接决定下一个会话的效率。这就像 Git 的最佳实践——每次提交应该是一个原子性的、可编译的变更,而不是一堆半成品代码的堆砌。
+这就像大学宿舍——你不倒垃圾,下一个室友进来就得替你收拾。他本来的计划是复习考试,结果先花半小时收拾你留下的烂摊子。更要命的是,收拾完了他也不想复习了——环境太差,心情也差。
-## 核心概念
+OpenAI 和 Anthropic 都明确指出:**长期可靠性取决于操作纪律,不仅是单次运行的成功。** 每个会话结束时的状态质量,直接决定下一个会话的效率。
-- **清洁状态**:会话结束时系统满足五个条件——构建通过、测试通过、进度已记录、无过时工件、启动路径可用。缺一个都不算"做完"。
-- **会话完整性**:类比数据库事务——要么全部提交并留下清洁状态,要么回滚到上一致状态。没有中间地带。
-- **质量文档**:对每个模块的质量等级做持续记录的活跃工件。不是一次性评估,而是跟踪代码库是变强了还是变弱了。
-- **清理循环**:定期的维护会话,目标是系统性减少代码库中的熵。不是紧急修复,而是常规运维。
-- **harness 简化**:随着模型能力提升,定期移除不再必要的 harness 组件。今天必要的约束,三个月后可能是多余的开销。
-- **幂等清理**:清理操作无论执行多少次都产生相同结果。确保清理在失败重试场景中仍然安全。
+## 熵增是默认状态
+
+Lehman 的软件演化定律告诉我们:持续变更的系统,除非主动管理,否则复杂性必然增加。这对 AI 编码 agent 尤其成立——agent 每次会话都会引入变更,如果不在退出时清理,技术债务会指数级累积。宿舍不打扫,脏衣服和外卖盒只会越堆越多,不会自己消失。
+
+OpenAI 在 5 个月的 Codex 实验中观察到:**agent 会复制仓库中已有的模式——即使那些模式是不均匀的或次优的。** 随着时间的推移,这种复制必然导致漂移。就像宿舍里的公共区域——第一个人在桌上放了一个杯子,第二个人觉得"反正已经乱了"又放了一个,一周后桌上堆满了。
+
+OpenAI 团队最初花每周五(20% 的工作时间)手动清理 "AI slop"。不出所料,这种方式不可扩展。就像宿舍每周大扫除一次——其他六天的脏乱差还是得忍。他们的解决方案是:
+
+1. **把"黄金原则"编码进仓库**:比如"优先使用共享工具包而非手写的 ad-hoc 辅助函数"(保持不变量集中)、"不要 YOLO 式地猜数据结构"(验证边界或依赖类型化 SDK)。这些原则是具体的、机械的、可自动检查的。
+2. **建立周期性的清理流程**:一组后台 Codex 任务定期扫描偏差,更新质量评分,开针对性的重构 PR。大多数可以在一分钟内审查并自动合并。就像宿舍安装了自动扫地机器人——不用你动手,定期清理。
+3. **人类品味捕获一次,持续执行**:审查意见、重构 PR、用户侧 bug 都被转化为文档更新或直接编码到工具中。当文档不够用时,把规则提升为代码。就像把宿舍公约从"口头约定"变成"贴在门上的检查表"。
-## 清洁状态的五个维度
+这个机制就像垃圾回收——技术债是高息贷款,持续小额还清几乎总是比攒到一次性爆发好得多。
+
+> 来源:[OpenAI: Harness engineering: leveraging Codex in an agent-first world](https://openai.com/index/harness-engineering/)
+
+## 清洁状态:不只看地上有没有垃圾
+
+清洁状态不是单一的"代码能编译"。代码能无错构建——这是最基本的,下一个会话不应该先修构建错误。就像你搬出去的时候水电不能是断的。所有测试也得通过,包括会话之前就存在的测试——会话有责任不破坏已有功能。而且要在 CI 环境验证,不是"在我机器上通过"。
```mermaid
flowchart LR
@@ -36,6 +45,8 @@ flowchart LR
Fix --> Build
```
+但这还不够。当前进度必须记录在机器可读的工件中——已完成的子任务和通过标准、进行中但未完成的子任务和当前状态、尚未开始的子任务。好的进度记录减少 60-80% 的会话启动诊断时间。调试日志、临时文件、注释掉的代码、TODO 标记这些临时工件得清理干净,它们增加下一个会话的认知负担。标准启动路径也必须可用——下一个会话能不能不人工干预就开始工作?环境初始化、代码库加载、上下文获取、任务选择,这些路径不能被破坏。
+
```mermaid
flowchart LR
Dirty["这次退出时<br/>测试是红的、临时文件没删、进度也没写"] --> Diagnose["下个会话先花时间<br/>搞清楚发生了什么"]
@@ -48,23 +59,22 @@ flowchart LR
Stable --> Clean
```
-## 为什么会这样
-
-### 熵增是默认状态
-
-Lehman 的软件演化定律告诉我们:持续变更的系统,除非主动管理,否则复杂性必然增加。这对 AI 编码 agent 尤其成立——agent 每次会话都会引入变更,如果不在退出时清理,技术债务会指数级累积。
+## 核心概念
-**agent 会复制仓库中已有的模式——即使那些模式是不均匀的或次优的。** 这是 OpenAI 在 5 个月的 Codex 实验中观察到的一个核心现象。随着时间的推移,这种复制必然导致漂移。
+- **清洁状态**:会话结束时系统满足五个条件——构建通过、测试通过、进度已记录、无过时工件、启动路径可用。缺一个都不算"做完"。
+- **会话完整性**:类比数据库事务——要么全部提交并留下清洁状态,要么回滚到上一致状态。没有中间地带。
+- **质量文档**:对每个模块的质量等级做持续记录的活跃工件。不是一次性评估,而是跟踪代码库是变强了还是变弱了。
+- **清理循环**:定期的维护会话,目标是系统性减少代码库中的熵。不是紧急修复,而是常规运维——就像宿舍的每周值日表。
+- **harness 简化**:随着模型能力提升,定期移除不再必要的 harness 组件。今天必要的约束,三个月后可能是多余的开销。
+- **幂等清理**:清理操作无论执行多少次都产生相同结果。确保清理在失败重试场景中仍然安全。
-OpenAI 团队最初花每周五(20% 的工作时间)手动清理 "AI slop"。不出所料,这种方式不可扩展。他们的解决方案是:
+## "以后再清理"是永远不清理
-1. **把"黄金原则"编码进仓库**:比如"优先使用共享工具包而非手写的 ad-hoc 辅助函数"(保持不变量集中)、"不要 YOLO 式地猜数据结构"(验证边界或依赖类型化 SDK)。这些原则是具体的、机械的、可自动检查的。
-2. **建立周期性的清理流程**:一组后台 Codex 任务定期扫描偏差,更新质量评分,开针对性的重构 PR。大多数可以在一分钟内审查并自动合并。
-3. **人类品味捕获一次,持续执行**:审查意见、重构 PR、用户侧 bug 都被转化为文档更新或直接编码到工具中。当文档不够用时,把规则提升为代码。
+最常见的心理陷阱是"这次来不及清理了,下次再弄"。但下次的 agent 不知道你上次留下了什么——它看到的是一堆混乱的代码和不确定的状态。它会花大量时间推断"这堆代码里哪些是有意的,哪些是临时的"。
-这个机制就像垃圾回收——技术债是高息贷款,持续小额还清几乎总是比攒到一次性爆发好得多。
+更糟的是,每个会话都有自己的任务目标。新会话来的时候是要做新功能的,不是来清理上一个会话的烂摊子的。它会忽略混乱直接开始新工作,然后在混乱的基础上引入更多混乱。这是熵增的正反馈循环——就像宿舍越乱你越不想收拾,越不收拾越乱,最后谁都不想回宿舍了。
-> 来源:[OpenAI: Harness engineering: leveraging Codex in an agent-first world](https://openai.com/index/harness-engineering/)一个使用 agent 持续开发 12 周的项目,没有清洁策略的情况下:
+数据为证。一个使用 agent 持续开发 12 周的项目,没有清洁策略的情况下:
- 第 1 周:构建通过率 100%,测试通过率 100%,新会话启动 5 分钟
- 第 4 周:构建 95%,测试 92%,启动 15 分钟
@@ -76,31 +86,11 @@ OpenAI 团队最初花每周五(20% 的工作时间)手动清理 "AI slop"
- 第 1 周:100%,100%,5 分钟
- 第 12 周:97%,95%,9 分钟
-12 周后,构建通过率差 29 个百分点,新会话启动时间差 85%。这不是理论推导,是实际可观测到的差异。
-
-### 五个维度的清洁状态
-
-清洁状态不是单一的"代码能编译",而是五个维度的综合:
-
-**构建维度**:代码能无错构建?这是最基本的——下一个会话不应该先修构建错误。
-
-**测试维度**:所有测试通过?包括会话之前就存在的测试——会话有责任不破坏已有功能。而且要在 CI 环境验证,不是"在我机器上通过"。
-
-**进度维度**:当前进度已记录在机器可读的工件中?已完成的子任务和通过标准、进行中但未完成的子任务和当前状态、尚未开始的子任务。好的进度记录减少 60-80% 的会话启动诊断时间。
-
-**工件维度**:无过时或歧义的临时工件?调试日志、临时文件、注释掉的代码、TODO 标记——这些都增加下一个会话的认知负担。
-
-**启动维度**:标准启动路径可用?下一个会话能不能不人工干预就开始工作?环境初始化、代码库加载、上下文获取、任务选择——这些路径不能被破坏。
-
-### "以后再清理"是永远不清理
+12 周后,构建通过率差 29 个百分点,新会话启动时间差 85%。这不是理论推导,是实际可观测到的差异。每周不倒垃圾的宿舍和每周倒垃圾的宿舍,12 周后的差距是惊人的。
-最常见的心理陷阱是"这次来不及清理了,下次再弄"。但下次的 agent 不知道你上次留下了什么——它看到的是一堆混乱的代码和不确定的状态。它会花大量时间推断"这堆代码里哪些是有意的,哪些是临时的"。
-
-更糟的是,每个会话都有自己的任务目标。新会话来的时候是要做新功能的,不是来清理上一个会话的烂摊子的。它会忽略混乱直接开始新工作,然后在混乱的基础上引入更多混乱。这是熵增的正反馈循环。
-
-## 怎么做才对
+## 怎么做
-### 1. 清洁状态作为完成的必要条件
+### 1. 清洁状态是完成的必要条件
在 harness 里明确定义:**会话完成 = 任务通过验证 AND 清洁状态检查通过。** 缺任何一个,会话不算完成。在 CLAUDE.md 里写:
@@ -117,13 +107,13 @@ OpenAI 团队最初花每周五(20% 的工作时间)手动清理 "AI slop"
结合两种清理模式:
-**即时清理(每个会话结束时)**:清理本次会话创建的临时工件、更新功能清单状态、确保构建和测试通过。这是"引用计数式"清理。
+**即时清理(每个会话结束时)**:清理本次会话创建的临时工件、更新功能清单状态、确保构建和测试通过。这是"引用计数式"清理——用完就清。就像吃完饭马上洗碗,不留到第二天。
-**定期清理(每周一次)**:全系统扫描——处理累积的结构性问题、更新质量文档、运行基准测试检测漂移。这是"追踪式"清理。
+**定期清理(每周一次)**:全系统扫描——处理累积的结构性问题、更新质量文档、运行基准测试检测漂移。这是"追踪式"清理——定期做一次大扫除。
### 3. 维护质量文档
-质量文档是对每个模块持续评分的活跃工件:
+质量文档是对每个模块持续评分的活跃工件——就像宿舍的卫生检查评分表:
```markdown
# 质量文档
@@ -143,31 +133,23 @@ OpenAI 团队最初花每周五(20% 的工作时间)手动清理 "AI slop"
- 代码规范: 部分遵循
```
-新会话读这个文档就知道优先处理哪里。质量评分最低的模块先修。
+新会话读这个文档就知道优先处理哪里。质量评分最低的模块先修。就像卫生检查表上标记了"需要重点打扫的区域"——下一个值日生知道重点在哪里。
### 4. 定期简化 harness
-harness 里的每个组件之所以存在,是因为模型无法独立做好某件事。但随着模型改进,这些假设会过时。
+harness 里的每个组件之所以存在,是因为模型无法独立做好某件事。但随着模型改进,这些假设会过时。就像你大一的时候需要学长带着选课,到了大三你自己就知道怎么选了——学长的作用变小了,但有些事情你还是需要问。
Anthropic 的实验直接展示了这一点。他们最初的 harness 包含 sprint 拆分机制——把工作分成小块让 Sonnet 4.5 逐个完成。当 Opus 4.6 发布后,模型的原生能力已经可以自主处理工作分解,sprint 构造变成了不必要的开销。移除后,builder agent 能连续工作超过两小时而不会跑偏,反而更流畅。
但 evaluator 的情况不同。即使 Opus 4.6 能力更强,在任务接近模型能力边界时,evaluator 仍然提供了实际价值——捕获 generator 的遗漏功能和存根实现。这意味着 evaluator 不是一个固定的是/否决策,而是取决于任务难度相对于模型能力的位置。
-**推荐做法**:每月挑一个 harness 组件,暂时禁用它,跑基准任务。如果结果没退化,永久移除。如果退化了,恢复或用更轻量的替代。
+**推荐做法**:每月挑一个 harness 组件,暂时禁用它,跑基准任务。如果结果没退化,永久移除。如果退化了,恢复或用更轻量的替代。就像定期检查宿舍公约——大四了还在执行"每天 10 点熄灯"就不太合理了。
一个更具体的原则:**随着模型改进,harness 的有趣组合不是变少了,而是移动了。** 以前必须解决的问题被模型能力覆盖了,但新的能力边界打开了以前不可能的 harness 设计。AI 工程师的工作是持续找到下一个有价值的组合。
-### 5. 高吞吐量改变了 merge 哲学
-
-当 agent 的产出远超人类审查能力时,传统的 merge 哲学需要调整。
-
-OpenAI 团队的经验:在一个 agent 每天开 3.5 个 PR(且后来增加到更多)的环境里,最小化阻塞式 merge gate 是正确的。PR 应该短命。测试 flake 通常用后续运行解决,而不是无限期阻塞进度。在一个修正成本很低、等待成本很高的系统里,快速前进 + 快速修正是比缓慢确认更好的策略。
-
-**注意**:这在低产出环境里是不负责任的。但在 agent 产出远超人类注意力的环境里,这往往是正确的权衡。关键判断标准:**修正一个 bug 的平均成本 vs 等待人类审查一个 PR 的平均成本。** 前者低于后者时,快速合并是对的。
-
### 5. 清理操作必须幂等
-清理脚本要能安全地重复执行:
+清理脚本要能安全地重复执行——就像打扫卫生,扫两遍比扫一遍更干净,但不会更脏:
```bash
# 幂等的清理操作
@@ -176,22 +158,28 @@ git checkout -- .env.local # 恢复到已知状态
npm run test # 验证清理未破坏功能
```
+### 6. 高吞吐量改变了 merge 哲学
+
+当 agent 的产出远超人类审查能力时,传统的 merge 哲学需要调整。OpenAI 团队的经验:在一个 agent 每天开 3.5 个 PR(且后来增加到更多)的环境里,最小化阻塞式 merge gate 是正确的。PR 应该短命。测试 flake 通常用后续运行解决,而不是无限期阻塞进度。在一个修正成本很低、等待成本很高的系统里,快速前进 + 快速修正是比缓慢确认更好的策略。
+
+**注意**:这在低产出环境里是不负责任的。但在 agent 产出远超人类注意力的环境里,这往往是正确的权衡。关键判断标准:**修正一个 bug 的平均成本 vs 等待人类审查一个 PR 的平均成本。** 前者低于后者时,快速合并是对的。
+
## 实际案例
一个使用 agent 持续开发的 Electron 应用,12 周演化过程的对比数据:
-**无清洁策略**(对照组):第 12 周,构建通过率 68%,测试通过率 61%,新会话启动 60+ 分钟,过时工件 103 个。
+**无清洁策略(对照组)**——从不倒垃圾的宿舍:第 12 周,构建通过率 68%,测试通过率 61%,新会话启动 60+ 分钟,过时工件 103 个。
-**有清洁策略**(实验组):每个会话结束时执行完整清洁检查 + 每周清理循环。第 12 周,构建通过率 97%,测试通过率 95%,新会话启动 9 分钟,过时工件 11 个。
+**有清洁策略(实验组)**——每天值日的宿舍:每个会话结束时执行完整清洁检查 + 每周清理循环。第 12 周,构建通过率 97%,测试通过率 95%,新会话启动 9 分钟,过时工件 11 个。
-到第 12 周,实验组的构建通过率比对照组高 29 个百分点,测试通过率高 34 个百分点,新会话启动时间减少 85%。
+到第 12 周,实验组的构建通过率比对照组高 29 个百分点,测试通过率高 34 个百分点,新会话启动时间减少 85%。每天多花 5 分钟打扫卫生,12 周后省下了几十个小时的混乱时间。
## 关键要点
-- **清洁状态是会话完成的必要条件**——不是可选的善后工作,是"完成定义"的一部分。
+- **清洁状态是会话完成的必要条件**——不是可选的善后工作,是"完成定义"的一部分。你不倒垃圾,下一个室友就得替你倒。
- **五个维度缺一不可**——构建、测试、进度、工件、启动,每个都要显式检查。
-- **质量文档让代码库健康可追踪**——知道哪里在退化才能主动修复。
-- **定期简化 harness**——随着模型能力提升,移除不再必要的约束。
+- **质量文档让代码库健康可追踪**——知道哪里在退化才能主动修复。卫生检查表不是形式主义,是让你知道哪块地还没拖。
+- **定期简化 harness**——随着模型能力提升,移除不再必要的约束。大四了就别执行大一的宿舍公约了。
- **"以后再清理"等于永远不清理**——熵增是默认状态,只有主动的清洁操作才能对抗它。
## 延伸阅读