graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
Python
prompt: a spec task ends at its own tests
justrach390f26b2d8ae
diff --git a/docs/adr/0249-a-spec-task-ends-at-its-own-tests.md b/docs/adr/0249-a-spec-task-ends-at-its-own-tests.mdnew file mode 100644index 00000000..fbed80df--- /dev/null+++ b/docs/adr/0249-a-spec-task-ends-at-its-own-tests.md@@ -0,0 +1,34 @@+# 0249. A spec task ends at its own tests++Status: accepted 2026-10-03++## Context++ADR 0024 added a sentence to the work note: a named SPEC.md is the whole+contract, and a green public test is not the whole spec. It was written for+one spec miss, a streaming-JSON task whose whitespace-only input counts as+empty.++Paired runs on the spec-driven SWE tasks showed the sentence no longer does+that job — with it, without it, and in a comparison harness that never had+it, that edge is still missed — but it changes the shape of every spec task.+Once the task's own tests pass, the model writes a second, ad-hoc+verification script against the spec's clauses: one long extra call per+task that found nothing the hidden checks wanted. The comparison harness,+which stops at the task's tests, passed the same tasks. A rewording that+kept the sentence but asked for a reading check instead of a script+recovered only part of the cost.++## Decision++Remove the sentence from the work note. The rest of the note already says+to make the requested thing work, prove it, and add no unrequested tests,+so the task's own tests are the proof. The lean work note keeps its copy;+it was not measured.++## Consequences++Spec tasks end once their tests pass: fewer output tokens and shorter turns,+with the same pass rate in paired runs. If a model starts missing spec+clauses no public test covers, the answer belongs in that task's tests, not+in a standing instruction every spec task pays for.diff --git a/docs/adr/README.md b/docs/adr/README.mdindex 949f1a91..d530eedf 100644--- a/docs/adr/README.md+++ b/docs/adr/README.md@@ -275,6 +275,7 @@ record only when you need the evidence or the edge cases.| [0246](0246-jev-picks-each-turns-effort-alongside-the-first-request.md) | On an eligible model at the default effort, graff asks Jev for the turn's effort alongside the first request; the pick applies from the next request, holds for that turn, is never saved, and never overrides a user's effort. `chatgpt-new` GPT-6 models are eligible. || [0247](0247-parked-work-is-named-above-the-prompt.md) | A parked turn names the work it waits on: the standing block draws `↻ waiting on <work> — graff continues when it finishes` while a job or agent that will wake the session runs, the yield prints only for a server, and a wake prints what finished before continuing. || [0248](0248-commentary-prose-does-not-tighten-the-stall-budget.md) | Commentary prose (the heads-up before a tool-call batch, which the server holds open until it releases the calls) counts as visible text but does not tighten the stall budget; final-answer prose and whitelisted tool-argument prose still do. Refines 0241. |+| [0249](0249-a-spec-task-ends-at-its-own-tests.md) | The work note no longer says a green public test is not the whole spec: it made the model write an extra verification script on every spec task without changing the pass rate. Spec tasks end at their own tests; the lean note is unchanged. Refines 0024. |## When to write onediff --git a/src/prompt_golden.zig b/src/prompt_golden.zigindex 7c94957f..788f69ba 100644--- a/src/prompt_golden.zig+++ b/src/prompt_golden.zig@@ -163,8 +163,6 @@ pub const full_prompt =\\will not change; reread only on stale source, ambiguity, or failure.\\When a Project layout segment is present, it is the tree — read the\\files you need straight from it instead of ls/find exploration turns.- \\When a named SPEC.md (or equivalent contract) is in the task, satisfy- \\every clause — a green public test is not the whole spec.\\\\Long commands and background subagents keep running while you work, and\\each one reports back when it finishes: do not poll or sleep to wait fordiff --git a/src/prompt_text.zig b/src/prompt_text.zigindex c2cab7f7..ae0d1e83 100644--- a/src/prompt_text.zig+++ b/src/prompt_text.zig@@ -228,8 +228,6 @@ pub const work_note =\\will not change; reread only on stale source, ambiguity, or failure.\\When a Project layout segment is present, it is the tree — read the\\files you need straight from it instead of ls/find exploration turns.- \\When a named SPEC.md (or equivalent contract) is in the task, satisfy- \\every clause — a green public test is not the whole spec.;/// Gate: `caps.local_tools`. Run until idle (run_idle.zig): a reply that ends