graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
The self-correcting release: the harness measured itself, then fixed what it found
v0.0.237 is one closed loop, run end to end in a day: eval the harness, mine the trajectories, redesign what lost, verify the redesign with the same evals — and ship the machinery that makes the next loop automatic.
Ultracode: smallest rung that fits
The old ultracode was measured against a single agent on five deterministic evals and lost: 80 mean score at 3.7× the calls, three budget deaths, and a bugfix run that spent 30 calls orchestrating without ever editing the file. It was redesigned around an escalation ladder informed by how Codex, OpenCode, and grok-build gate delegation:
- The codeword now means "escalate to the smallest rung that fits": solo for 1–2 known files, one scout when exploration would flood context, a fleet only for 3+ independent workstreams or after a failed attempt, judges only for audit-class asks.
- An admission gate and a budget-reservation ledger enforce it: duplicate briefs collapse at spawn, implement phases carry an edit contract (no diff = error + retry), and a run that can't afford its later phases truncates gracefully instead of dying mid-flight.
- Re-measured on the same five evals: 100 mean at 1.22× baseline calls, zero budget deaths — and 72% fewer uncached input tokens and 78% less output than the old ultracode. A genuinely large audit task still fans out (4 workers) and lands its answer from partial evidence.
- Every rung decision writes an
orchtrajectory row keyed by task class, budget band, and model stratum — the hand ladder is the bootstrap; the fleet learns the policy from its own outcomes.
The trajectory failures, closed
From two production post-mortems (a design-fleet session and its failed sequel):
- Constraints survive now (#381):
/never <text>— or the model calling the append-onlynote_constrainttool when you reject something — lands in.graff/playbook.jsonl; a HARD CONSTRAINTS block derived from the file at brief-assembly time rides every worker brief and the root prompt, immune to compaction and session boundaries. Measured: 0/4 obeyed before, 4/4 after. - Reskin fleets get caught (#382): 3+ near-identical variant briefs
draw a calibrated similarity warning on both fan-out paths; the new
concept-fleetshape demands a design thesis and one signature system per variant. - Image tasks route to eyes (#380): a task naming an image re-seats
automatic workers onto a vision-capable model (
source=vision-ask), and a worker that admits it couldn't view the image gets flagged so inference can never masquerade as observation. - The harness learns from its runs (#383): target-met evals distill up to three provenance-stamped insight bullets, curated by deterministic code, injected as an advisory block below user constraints.
The five-flake-a-day CI bug, root-caused
The ubuntu pty "shutdown hang" was never a shutdown hang: a Ctrl-D landing
in a canonical-mode tty window is delivered by Linux as NUL, and readline
dropped it — the REPL then waited at the prompt, correctly, forever.
Type-ahead bytes are now normalized across the mode switch (15/15
container hangs → 25/25 passes), and teardown stamps 12 phases under
GRAFF_SHUTDOWN_DEBUG so any future hang names its stalling phase (#364).
Filed for what's next
- #385: stage the image bytes into vision-routed workers (routing landed; sight is next). #390: reserve the final narration call on tight budgets.
- #384: weakness mining — automate the trajectory post-mortem that produced this release. #387–#389: usage-footer and gateway-error robustness found by this release's own eval studies.
Full Changelog: https://github.com/justrach/codegraff/compare/v0.0.236...v0.0.237