Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python
9.7 KB
Raw Blame

v0.0.276 (2026-08-25)

RLM + mid-stream spec-ptc is now the default harness loop. graff writes a code REPL, speculates host calls as that script streams, and keeps binds across turns. --old / --no-rlm restore the previous structured-only catalog.

This is the Zig port of alexzhang13/spec-ptc and alexzhang13/rlm — not Prime Agent's IPython/ZeroMQ kernel, and not a Python shadow REPL. Prime's shape we kept, still in Zig: persistent binds and subagent("task") through graff's existing subagent tool.

What RLM actually does

rlm is a folded native tool (same stack as workflow / subagent). The name shows up on load_tool_schemas; the full description does not sit on every prefix. --schema and the 64-cell tool-catalog kernel do not grow a new always-on tool.

Inside the REPL, these work:

  • Host tools as functions. read_file, edit_file, write_file, codedb, bash, webfetch, plus sleep_ms, llm_query, print, and subagent.
  • Mid-stream speculation. Closed statements with literal args launch while code is still streaming (semicolons count). runScript claims the futures. This includes -p, where there is no REPL printer.
  • Persistent binds. s = read_file("x") then print(s) on a later rlm call reuses s. /new and /clear drop the store.
  • print(read_file(path)) returns the file, not the literal read_file("path"). Nested host calls inside print(...) are speculated and claimed.
  • subagent("task") is real graff subagent. Independent literal calls fan out as the script streams. run_in_background=true returns an id; v1 print(h) is the child's report. Spawn guidance is Codex's sidecar rule: keep the critical-path next step local.
  • llm_query is a bounded tools-off sub-call on the session provider.

Flags: --old / --no-rlm / GRAFF_OLD=1 / GRAFF_RLM=0 restore structured-only. --rlm / GRAFF_RLM=1 force it on. Last CLI flag wins; env does not clobber --old or --rlm.

See ADR 0022.

Performance

All numbers are live grok-4.6 SuperGrok OAuth, one rep, 2026-08-25. SuperGrok is flat-rate ($0.0000 both ways). Token/call cuts are the spend win on a metered XAI_API_KEY.

Folded catalog — no prefix tax

The first cut appended the full rlm spec every turn (+7,411 input tokens). Folding it behind load_tool_schemas dropped that to +304 tokens. Core 12-task suite: both 12/12, 86.6s vs 86.4s.

Persist / bind-reuse

The model does s = read_file(...) then print(s). Without persist it discovers rlm is off and rewrites the long way.

wall input calls
--old 99.6s 135k 11
default rlm 39.2s 25.3k 5
−61% −81%

Confirms #619 (97.5s / 145k → 41.4s / 30.9k).

Default RLM vs --old (rlm suite)

graff-evals/run.py --suite rlm --harness graff-dev-old,graff-dev

harness pass wall input output calls
--old 5/5 161s 203k 4186 26
default rlm 5/5 136s (−16%) 105k (−48%) 1669 (−60%) 22

Scatter wall is one-rep fragile. An earlier opt-in run was 86.8s → 23.7s (needle-files 62.8s → 5.6s). The default-RLM rerun did not reproduce that hang; --old needle-files was already 6.8s. Bind-reuse is the stable win.

One-shot lean (graff -p) and SWE vs grok-build

Cursor's follow-up on this branch (PR #621) cut the one-shot prefix and the DeepSWE-shaped suite until graff matched grok-build on pass rate and beat it on wall, tokens, calls, and RSS.

What landed:

  • Unfold rlm on -p. Drop MCP, skills, parallel essays, git-safety essays, and load_tool_schemas even when home MCP is deferred. Need MCP? --no-lean.
  • Keep file/shell tools on the lean catalog. An rlm-only keep-list made grok-4.6 retry scripts instead of editing (4/6, 122 calls). Restored bash/read/edit/write/codedb/attempt_completion/rlm.
  • print(read_file(path)) is the file. The leftover turn tax was a Zig miss: print(...) skipped nested host calls, so the model re-read.
  • -p skips the shared-tree checkpoint. Parallel eval sandboxes live inside the harness git checkout; six -j sessions were locking each other's edit_file. Interactive multi-session still checkpoints.

First-request "say OK" (empty cwd): 3880 → 2204 input tokens. Per-call prefix is 5.0k vs grok-build 5.2k.

DeepSWE-shaped suite after those cuts (run-20260825-091123.jsonl, -j 6):

harness pass wall (sum) input calls RSS
graff default rlm 5/6 198s 106k 24 8.5M
grok-build 5/6 506s 194k 39 177M

Every graff task is four API calls (rlm → edit(s) → test → done). label-sort still fails both harnesses. Do not copy grok's heap for the remaining call gap — the transferable prefix cuts are exhausted (ADR 0024, ADR 0025).

Process Io stays Zig Threaded. io_uring was probed at 19–20 µs/round serial pread vs 69–75 µs/round submit-and-wait on 64×4 KiB batches. ublk is a block-device protocol, not a faster read_file.

Prompt-cache max is on

load_tool_schemas("rlm") is now a guaranteed first-turn load. Rewriting tools JSON there was a 0% cache-read on the next call, so GRAFF_STABLE_CATALOG is the default. The loaded schema still rides the load result; the catalog head stays byte-identical. Opt out with GRAFF_STABLE_CATALOG=0.

Sibling subagent() calls share four role-lane x-grok-conv-id / prompt_cache_key values (official xAI sticky routing). They do not reuse the root id — the child prefix is a different messages array. /btw still shares the parent key.

See ADR 0011.

What we did not take

Compared against Codex, grok-build, kimi-cli, and deepseek-harness, then kept Zig:

  • No V8 Code Mode / IPython kernel. rlm is already the analog.
  • No extra spawn_agent catalog tools. subagent + agent_output stay.
  • No parent-history fork into children. Cold spawn, same as Codex default.
  • No sharing the root cache key with children. Role-lanes among siblings.
  • No rlm-only catalog. File/shell tools stay; hiding them tanks pass rate.
  • No io_uring process Io, no ublk.

We did take Codex's sidecar-vs-critical-path rule and dropped the routing essay from the always-on subagent description (ADR 0023).

Also in this cut

  • Foreground bash auto-backgrounds after 120s (or timeout ms) instead of waiting forever. The process is promoted onto the job registry, not killed. Subagent bash still kills at 120s. ADR 0026 / #620.
  • Kimi Coding keeps User-Agent graff/<version> and matches kimi-code device-header shapes. ADR 0027 / #617.
  • Codex session_id is the prompt_cache_key, not a per-process UUID. ADR 0028.
  • High-effort Codex thinking no longer trips the stall watchdog; the stall notice is honest.
  • imagegen: Codex 0.144+ rejects C0 / invalid UTF-8 in the engine prompt; those bytes are stripped (#576).
  • Tournament-barrier tests record attempts before the rendezvous (#467).
  • REPL/TUI prints a dim line when a newer graff release is available.

Downloads

All six CLI archives, the curl installer, and checksums:

File Platform
graff-aarch64-linux.tar.gz Linux arm64
graff-aarch64-macos.tar.gz macOS Apple Silicon — Developer ID signed, Apple notarized
graff-aarch64-windows.tar.gz Windows arm64
graff-x86_64-linux.tar.gz Linux x86_64
graff-x86_64-macos.tar.gz macOS Intel — Developer ID signed, Apple notarized
graff-x86_64-windows.tar.gz Windows x86_64
install.sh curl installer
SHA256SUMS checksums for the six archives
curl -fsSL https://github.com/justrach/codegraff/releases/download/v0.0.276/install.sh | bash

Both macOS binaries are Developer ID Application signed (hardened runtime, timestamped) and Apple notarized (notary-local, Accepted). Stapling is skipped on the bare CLI (stapler error 73); Gatekeeper checks the ticket online at first launch. GitHub also attaches the usual source zip/tarball for the tag.