Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python
3.2 KB
Raw Blame

Codegraff v0.0.302.17

graff's code mode, rlm, now gives a script the whole result of an MCP call. Measured against the previous release and against Pi 1.0 with its codemode, on two models:

Evals

gpt-6-astra: 27% less time on tasks that join MCP data

v0.0.302.16, this release and Pi 1.0 with codemode in its default tools, on gpt-6-astra through a ChatGPT plan, interleaved over 11 tasks, 3 runs each:

Pass Wall per task MCP join tasks Model calls per task Input tokens per task
v0.0.302.16 33/33 54.1s 51.4s 4.5 49.4k
v0.0.302.17 33/33 48.3s 37.7s 3.8 44.0k
Pi 1.0 + codemode 33/33 41.7s 34.8s 4.8 20.9k

The join tasks need fields the slim view drops, or a join with local files or git history: count issue keys in src/, roll issues up by priority, and find high-priority issues no recent commit mentions. No run of this release paged a stored result back in; 6 of 33 runs of v0.0.302.16 did. Pi with codemode is still faster overall and reads less than half graff's input, mostly because graff's prompt is larger.

MiMo v2.6 Pro: the five MCP tasks

The same model, endpoint and key for every arm, interleaved, 3 runs each:

Pass Wall per task MCP join tasks MCP report tasks
v0.0.302.16 13/15 37.1s 39.9s 32.9s
v0.0.302.17 14/15 31.0s 34.1s 26.2s
v0.0.302.16, rlm off 15/15 38.3s 51.7s 18.3s
Pi 1.0 + codemode 15/15 39.3s 40.6s 37.4s

The misses on both graff builds were one report run each that listed the issues' display keys instead of their ids. On MiMo rlm still costs time on simple fan-out: with rlm off graff answered the report tasks with eight parallel calls. Across all 11 tasks v0.0.302.16 took 55.1s per task with rlm off against 81.5s with it on, so --no-rlm (or GRAFF_RLM=0) is worth trying on MiMo. Full results.

rlm scripts keep whole MCP results

graff slims large MCP list results before the model reads them: rows keep their id, identifier, title and name, and a comment list becomes a count and its latest author. Until this release that applied inside rlm scripts too, so a script could never reach a field the slim view dropped, such as an issue's priority or estimate. The model had to leave the script, call the tool directly and page the stored result back in, and the pages' byte-range markers broke the JSON when it joined them.

From this release (ADR 0238):

  • A bind keeps the tool's whole result. It reaches the model only if the script prints it.
  • print() still shows the slim view, and now says it is one.
  • project(x, field) reads any field. Over an each() bind of comment lists, n and latest_author still read the fold, so existing scripts keep working.
  • write_file("f.json", x) saves the whole result for a shell script.
  • Inside a script, read_tool_result("tr_N") binds a stored result whole.

A direct MCP call is unchanged: slimmed, with the full result behind a handle.