graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
Codegraff v0.0.302.20
v0.0.302.19 was tagged but never published, so this release carries all of it:
tool results inline up to 64 KiB, no false stall before a batch of tool calls,
spec tasks that end at their own tests, Jev picking each turn's effort, parked
work named above the prompt, no PR readiness gate, and the build on stable Zig
0.17.0. On top of that it keeps a stdio MCP server usable after one oversized
message, fixes the home directory on Windows when HOME is set but empty,
and adds an opt-in two-pass compactor.
One oversized MCP message no longer breaks the server
graff reads a stdio MCP server one line at a time into a 1 MiB buffer. A
message larger than that, such as a progress notification carrying a long
payload, failed the read and stayed in the buffer, so every later call to
that server failed with McpResponseTooLarge until the session restarted.
The rest of the oversized line is now discarded. An oversized notification is
skipped and the reply being awaited is read after it; an oversized reply fails
only its own call (#1504).
Windows: an empty HOME is not a home
Key storage, sign-ins, sessions and the saved model all live under the home
directory: $HOME, or %USERPROFILE% on Windows. An empty HOME counted as a
home, so it hid %USERPROFILE% and those paths resolved against the root of
the drive. Empty values now fall through to the next source.
Experimental: two-pass compaction
GRAFF_COMPACT_MIX=1 turns on a compactor built for comparison runs. A first,
low-effort pass extracts a short state ledger from the conversation:
decisions, approaches already ruled out and why, identifiers and values in
flight copied verbatim, the next step, and open requests. The handoff summary
then carries that ledger as a checklist and runs at the session's effort. If
the first pass fails, the summary request is exactly the single-pass one. It is
off by default; evals/compact_ab has a mix arm to price it.
From v0.0.302.19
v0.0.302.19 cut the time graff spends on real repository work. Tool results come back inline up to 64 KiB, a heads-up before a batch of tool calls no longer trips a false stall, and a spec task ends at its own tests. It also named parked background work above the prompt, removed the PR readiness gate, and built with the stable Zig 0.17.0.
Evals (measured on v0.0.302.19)
Measured with the v0.0.302.19 binary against v0.0.302.18 and Pi 1.0 with its codemode, on a ChatGPT plan, with graff logged in to Codegraff as it is for its users. Arms are interleaved. Cost is the list price of each run's tokens; Jev's own charge is added at its real price.
Real fixes in the graff repository
Two merged graff pull requests (#195 and #726) replayed from their parent
commits on gpt-6.1-sol, one run at a time: three runs of #195 and two of #726
per arm. Every run ends with the repository's full zig build test, so most
of the time is the build.
| Pass | Wall per run | Median | List cost per run | Cache hit | |
|---|---|---|---|---|---|
| v0.0.302.19 | 5/5 | 180.8s | 171.5s | $0.064 | 85% |
| Pi 1.0 + codemode | 5/5 | 197.5s | 160.2s | $0.046 | 56% |
One graff run was graded a second time because its first grading hung in an unrelated test of the repository; its fix matched the other runs and passed. graff was faster on #726 (201.6s against 262.6s) and Pi on #195 (154.1s against 167.0s). graff reads whole source files into its context, so it costs more here. In #1502's own comparison on these tasks, the 64 KiB threshold took graff from 191.8s to 172.3s per run.
Standard tasks
11 tasks (five MCP tasks, a git digest and five solo coding tasks), 3 runs each, 8 at a time. Every arm passed every run.
gpt-6.1-sol
| Pass | Wall per task | Median | List cost per task | Cache hit | Output tokens per task | |
|---|---|---|---|---|---|---|
| v0.0.302.18 | 33/33 | 36.1s | 29.2s | $0.0204 | 88.1% | 692 |
| v0.0.302.19 | 33/33 | 35.8s | 27.8s | $0.0191 | 89.4% | 710 |
| Pi 1.0 + codemode | 33/33 | 39.4s | 33.2s | $0.0292 | 45.6% | 624 |
gpt-6-astra
| Pass | Wall per task | Median | List cost per task | Cache hit | Output tokens per task | |
|---|---|---|---|---|---|---|
| v0.0.302.18 | 33/33 | 43.0s | 36.2s | $0.109 | 92.5% | 778 |
| v0.0.302.19 | 33/33 | 41.8s | 33.3s | $0.117 | 90.1% | 750 |
| Pi 1.0 + codemode | 33/33 | 42.4s | 34.7s | $0.138 | 57.1% | 659 |
Spec tasks
Seven tasks graded by hidden spec checks, 3 runs each, 8 at a time. No arm
passes json-stream, so 18/21 is every arm's score.
gpt-6.1-sol
| Pass | Wall per task | Median | List cost per task | Cache hit | Output tokens per task | |
|---|---|---|---|---|---|---|
| v0.0.302.18 | 18/21 | 38.8s | 38.6s | $0.0220 | 90.8% | 751 |
| v0.0.302.19 | 18/21 | 40.0s | 35.8s | $0.0247 | 88.8% | 777 |
| Pi 1.0 + codemode | 18/21 | 41.4s | 40.6s | $0.0245 | 41.7% | 777 |
gpt-6-astra
| Pass | Wall per task | Median | List cost per task | Cache hit | Output tokens per task | |
|---|---|---|---|---|---|---|
| v0.0.302.18 | 18/21 | 48.0s | 46.5s | $0.125 | 93.5% | 877 |
| v0.0.302.19 | 18/21 | 43.2s | 39.1s | $0.128 | 91.7% | 761 |
| Pi 1.0 + codemode | 18/21 | 48.3s | 42.1s | $0.111 | 60.5% | 921 |
v0.0.302.19 has the lowest median in every suite. Jev chose the default
effort in every standard-task run. On the spec tasks it chose high effort for
config-parse in five of six runs and for map-conflict in three. The
map-conflict runs were no slower, but on gpt-6.1-sol the config-parse runs
took 57s against 41s at the default effort with the same result, which accounts
for v0.0.302.19's gap to v0.0.302.18 on that suite.
Large files arrive in one read
A tool result over the inline threshold comes back as a handle with a preview,
and the model reads the rest in pages. The threshold was 16 KiB, so a
whole-file read of an ordinary 600-line module (20 to 35 KB) came back as a
preview, and the model spent extra round trips paging through it. The default
is now 64 KiB. It is still clamped to half the context window, so small
windows keep their bound, and GRAFF_TOOL_HANDLE_BYTES still overrides it
(ADR 0251).
No false stalls before a batch of tool calls
v0.0.302.18 gave the full stall budget back once a heads-up's output item closed (ADR 0241). The server holds a commentary heads-up open while the model composes a batch of tool calls and then releases the close and all the calls together, so the item closed only after the compose. A batch that took longer than the tightened budget was cut off as a stall and generated again. Commentary prose now counts as visible text but leaves the full budget in place; final-answer prose tightens it as before, on the WebSocket and SSE readers alike (ADR 0248).
A spec task ends at its own tests
The work note told the model that a green public test is not the whole spec. On spec tasks the model then wrote a second, ad-hoc check script after the task's own tests passed. Paired runs passed the same tasks without the sentence, so it is gone (ADR 0249).
Jev picks each turn's effort
Logged in to Codegraff, graff offered GPT-6 and MiMo v2.6 models the
jev_effort tool, and in eval runs they never called it. graff now asks Jev
itself when a turn starts at the session's default effort, alongside the
turn's first request, so it adds no wait; the pick applies from the turn's
next request. GPT-6 on the ChatGPT-plan sign-in is now eligible. The pick
holds for that turn only, is never saved, and never overrides an effort you
chose. Jev sees a summary built locally from the request with code, paths,
file names, URLs, emails and identifiers removed. GRAFF_JEV_AUTO=0 turns it
off
(ADR 0246).
Parked work is named above the prompt
A shell command that outlives the interactive wait parks and hands the prompt
back; its exit wakes the session, which continues on its own. The prompt used
to come back under a generic line, and a parked turn read as graff stopping
mid-task. While such work runs, the prompt now shows
↻ waiting on <work> — graff continues when it finishes, and a wake prints
↻ <command> exited 0 — continuing before the turn resumes
(ADR 0247).
PR publication is not gated
gh pr create and gh pr ready used to go through a readiness preflight:
successful local checks, passing head CI with a wait on a pending run, a model
review of the PR body's claims, and an obligation that held back completion
until the head's checks passed. They now run like any other command, and a
draft completes like any other task. Whether a PR may merge with red or
pending CI is the repository's call, made by its branch protection and review.
A mutation of a branch, issue or PR that another live session has claimed is
still refused
(ADR 0250).
Build and tests
- graff builds with the stable Zig 0.17.0 release;
build.zig.zon, CI,install.shand the cloud-agent installer all pin it.graff learn initno longer hangs on Windows (#1493). graff learn initpins evaluator programs up to 256 MiB, which leaves room for the Linux debug build (#1501).- Two integration tests no longer race the removal of their temporary directories (#1499).