graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
MiMo v2.6: v0.0.302.13 against v0.0.302.14 and OpenCode 2
Measured 2026-10-02. Every arm sent the same model id to the same endpoint with the same key, on one machine, interleaved task by task, 2 runs of each of the 21 tasks.
v0.0.302.14 changes what graff's default effort means on MiMo (ADR 0235): thinking is Off below high, so MiMo acts without first reasoning at length. /effort high turns thinking back on.
MiMo v2.6 Pro
| Arm | What ran |
|---|---|
| v0.0.302.13 | The published graff-aarch64-macos release binary as a scripted graff repl, at the default effort (MiMo thinking on). |
| v0.0.302.14 | This release's build as a scripted graff repl, at the default effort (MiMo thinking off). |
| OpenCode 2 | @opencode/cli 2.0.21: opencode run --standalone --auto --format json, through graff-evals/opencode2_run.py, at its own defaults. |
| Pass | Wall per task | Input read from cache | Output tokens per task | Model calls per task | |
|---|---|---|---|---|---|
| v0.0.302.13 | 42/42 | 53.1s | 86% | 2,353 | 4.3 |
| v0.0.302.14 | 39/42 | 18.1s | 93% | 544 | 4.6 |
| OpenCode 2 | 42/42 | 30.9s | 87% | 1,004 | 4.6 |
By suite:
| Suite | v0.0.302.13 | v0.0.302.14 | OpenCode 2 |
|---|---|---|---|
| core (11 coding tasks), wall | 15.8s | 11.5s | 17.4s |
| core (11 coding tasks), pass | 22/22 | 21/22 | 22/22 |
| mcp (10 MCP tasks), wall | 94.2s | 25.5s | 45.7s |
| mcp (10 MCP tasks), pass | 20/20 | 18/20 | 20/20 |
v0.0.302.14 finishes the 21 tasks 2.9x as fast as v0.0.302.13 (18.1s against 53.1s per task) and in 41% less time than OpenCode 2 (30.9s). The MCP tasks show it most: 25.5s against 94.2s for v0.0.302.13 and 45.7s for OpenCode 2. v0.0.302.13 wrote 2,353 output tokens per task, nearly all of it thinking; v0.0.302.14 writes 544.
Without thinking, MiMo missed 3 of 42 runs: one reply on file-ops was a JSON fragment instead of a tool call, and on linear-quiet-warm it twice reported the issues' display numbers instead of their ids. v0.0.302.13 and OpenCode 2 passed every run. For work that needs careful reading, /effort high turns MiMo's thinking back on.
MiMo v2.6 Flash
Measured the same way an hour earlier. The thinking-on arm is the v0.0.302.14 code without #1453 and #1454, so it thinks at the default effort.
| Pass | Wall per task | Output tokens per task | Model calls per task | |
|---|---|---|---|---|
| graff, thinking on | 42/42 | 35.1s | 1,473 | 5.5 |
| v0.0.302.14 (thinking off) | 40/42 | 16.7s | 551 | 5.4 |
| OpenCode 2 | 42/42 | 26.6s | 1,025 | 5.2 |
With thinking off, Flash finishes the tasks 2.1x as fast (16.7s against 35.1s) and in 37% less time than OpenCode 2 (26.6s). Its two misses were the same task, linear-nohint-warm, where it reported the issues' display numbers instead of their ids.
Per task (Pro)
Wall time is the mean of 2 runs; a task that failed in a run is marked with its pass count.
| Task | Suite | v0.0.302.13 | v0.0.302.14 | OpenCode 2 |
|---|---|---|---|---|
csv-sum |
core | 15.2s | 13.6s | 22.6s |
dead-code |
core | 6.5s | 5.9s | 10.2s |
exact-reply |
core | 5.9s | 5.5s | 6.2s |
file-ops |
core | 11.3s | 3.7s (1/2) | 18.4s |
fix-fib |
core | 17.7s | 14.0s | 22.6s |
git-ops |
core | 53.6s | 22.3s | 26.4s |
json-transform |
core | 10.7s | 10.2s | 13.2s |
recall-noise |
core | 7.2s | 11.8s | 13.5s |
refactor-rename |
core | 16.3s | 16.9s | 15.6s |
regex-count |
core | 16.3s | 12.0s | 21.2s |
write-tests |
core | 13.7s | 10.4s | 21.5s |
linear-issues |
mcp | 122.3s | 16.5s | 37.8s |
linear-nohint |
mcp | 55.2s | 17.1s | 40.3s |
linear-nohint-warm |
mcp | 46.9s | 14.2s | 39.7s |
linear-quiet |
mcp | 88.7s | 19.2s | 35.5s |
linear-quiet-warm |
mcp | 79.1s | 25.7s (0/2) | 52.0s |
linear-reduce |
mcp | 100.5s | 21.4s | 43.1s |
linear-reduce-warm |
mcp | 149.8s | 22.9s | 44.4s |
linear-sidecar |
mcp | 154.6s | 45.7s | 54.2s |
linear-split |
mcp | 87.9s | 53.4s | 69.8s |
linear-warm |
mcp | 56.7s | 18.6s | 39.8s |
Reproduce
graff-evals/run.py with --model mimo-v2.6-pro (or mimo-v2.6-flash), one graff arm per binary plus opencode2, --suite core,mcp --reps 2 --interleave. A graff arm for a previous release is a copy of a scripted-repl arm whose command points at that release's binary.