Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

prompt: a spec task ends at its own tests

justrach390f26b2d8ae
diff --git a/docs/adr/0249-a-spec-task-ends-at-its-own-tests.md b/docs/adr/0249-a-spec-task-ends-at-its-own-tests.md
new file mode 100644
index 00000000..fbed80df
--- /dev/null
+++ b/docs/adr/0249-a-spec-task-ends-at-its-own-tests.md
@@ -0,0 +1,34 @@
+# 0249. A spec task ends at its own tests
+
+Status: accepted 2026-10-03
+
+## Context
+
+ADR 0024 added a sentence to the work note: a named SPEC.md is the whole
+contract, and a green public test is not the whole spec. It was written for
+one spec miss, a streaming-JSON task whose whitespace-only input counts as
+empty.
+
+Paired runs on the spec-driven SWE tasks showed the sentence no longer does
+that job — with it, without it, and in a comparison harness that never had
+it, that edge is still missed — but it changes the shape of every spec task.
+Once the task's own tests pass, the model writes a second, ad-hoc
+verification script against the spec's clauses: one long extra call per
+task that found nothing the hidden checks wanted. The comparison harness,
+which stops at the task's tests, passed the same tasks. A rewording that
+kept the sentence but asked for a reading check instead of a script
+recovered only part of the cost.
+
+## Decision
+
+Remove the sentence from the work note. The rest of the note already says
+to make the requested thing work, prove it, and add no unrequested tests,
+so the task's own tests are the proof. The lean work note keeps its copy;
+it was not measured.
+
+## Consequences
+
+Spec tasks end once their tests pass: fewer output tokens and shorter turns,
+with the same pass rate in paired runs. If a model starts missing spec
+clauses no public test covers, the answer belongs in that task's tests, not
+in a standing instruction every spec task pays for.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 949f1a91..d530eedf 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -275,6 +275,7 @@ record only when you need the evidence or the edge cases.
| [0246](0246-jev-picks-each-turns-effort-alongside-the-first-request.md) | On an eligible model at the default effort, graff asks Jev for the turn's effort alongside the first request; the pick applies from the next request, holds for that turn, is never saved, and never overrides a user's effort. `chatgpt-new` GPT-6 models are eligible. |
| [0247](0247-parked-work-is-named-above-the-prompt.md) | A parked turn names the work it waits on: the standing block draws `↻ waiting on <work> — graff continues when it finishes` while a job or agent that will wake the session runs, the yield prints only for a server, and a wake prints what finished before continuing. |
| [0248](0248-commentary-prose-does-not-tighten-the-stall-budget.md) | Commentary prose (the heads-up before a tool-call batch, which the server holds open until it releases the calls) counts as visible text but does not tighten the stall budget; final-answer prose and whitelisted tool-argument prose still do. Refines 0241. |
+| [0249](0249-a-spec-task-ends-at-its-own-tests.md) | The work note no longer says a green public test is not the whole spec: it made the model write an extra verification script on every spec task without changing the pass rate. Spec tasks end at their own tests; the lean note is unchanged. Refines 0024. |
## When to write one
diff --git a/src/prompt_golden.zig b/src/prompt_golden.zig
index 7c94957f..788f69ba 100644
--- a/src/prompt_golden.zig
+++ b/src/prompt_golden.zig
@@ -163,8 +163,6 @@ pub const full_prompt =
\\will not change; reread only on stale source, ambiguity, or failure.
\\When a Project layout segment is present, it is the tree — read the
\\files you need straight from it instead of ls/find exploration turns.
- \\When a named SPEC.md (or equivalent contract) is in the task, satisfy
- \\every clause — a green public test is not the whole spec.
\\
\\Long commands and background subagents keep running while you work, and
\\each one reports back when it finishes: do not poll or sleep to wait for
diff --git a/src/prompt_text.zig b/src/prompt_text.zig
index c2cab7f7..ae0d1e83 100644
--- a/src/prompt_text.zig
+++ b/src/prompt_text.zig
@@ -228,8 +228,6 @@ pub const work_note =
\\will not change; reread only on stale source, ambiguity, or failure.
\\When a Project layout segment is present, it is the tree — read the
\\files you need straight from it instead of ls/find exploration turns.
- \\When a named SPEC.md (or equivalent contract) is in the task, satisfy
- \\every clause — a green public test is not the whole spec.
;
/// Gate: `caps.local_tools`. Run until idle (run_idle.zig): a reply that ends