Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python
2.7 KB
Raw Blame
{"kind": "iterate", "candidate_id": "x-search-off", "results": "/workspace/graff-evals/results/run-20260830-034621.jsonl", "blocked": [], "keep": true, "why": "keep: wall_s: 63.96 → 54.54 (win, -14.7%); first_out_s: 2.706666666666667 → 1.67 (win, -38.3%); tok_calls: 9 → 9 (wash, +0.0%); list_tokens: 36374 → 22482 (win, -38.2%); list_usd: 0.04728 → 0.024272 (win, -48.7%)", "axes": {"wall_s": {"champ": 63.96, "cand": 54.54, "rel": -0.1473, "verdict": "win"}, "first_out_s": {"champ": 2.706666666666667, "cand": 1.67, "rel": -0.383, "verdict": "win"}, "tok_calls": {"champ": 9, "cand": 9, "rel": 0.0, "verdict": "wash"}, "list_tokens": {"champ": 36374, "cand": 22482, "rel": -0.3819, "verdict": "win"}, "list_usd": {"champ": 0.04728, "cand": 0.024272, "rel": -0.4866, "verdict": "win"}}, "wins": 4, "champ": {"pass": "3/3", "wall_s": 63.96, "first_out_s": 2.706666666666667, "tok_calls": 9, "list_tokens": 36374, "list_usd": 0.04728, "rss_peak_kb": 93452}, "cand": {"pass": "3/3", "wall_s": 54.54, "first_out_s": 1.67, "tok_calls": 9, "list_tokens": 22482, "list_usd": 0.024272, "rss_peak_kb": 93460}, "theirs": {"pass": "3/3", "wall_s": 41.64, "first_out_s": 3.5700000000000003, "tok_calls": 8, "list_tokens": 132858, "list_usd": 0.120476, "rss_peak_kb": 157500}, "ts": "2026-08-30T03:47:30Z"}
{"kind": "iterate", "candidate_id": "oneshot-no-learn-pin", "results": "/workspace/graff-evals/results/run-20260830-035954.jsonl", "keep": true, "why": "keep vs pre-fix graff-dev: wall 64.0s\u219221.8s, first 2.7s\u21921.9s, calls 9\u21928, tokens 36374\u219232583, list$ 0.0473\u21920.0479 wash; vs grok-build we win wall 21.8s<27.3s and list$ 0.0479<0.1424. x-search-off retracted (grok has web_search on).", "retracted": ["x-search-off"], "ts": "2026-08-30T04:00:43Z"}
{"kind": "iterate", "candidate_id": "hardlink-learn-pin", "results": "/workspace/graff-evals/results/run-20260830-042417.jsonl", "keep": true, "why": "shared hardlink pin. live 3-task: graff-dev 3/3 22.2s 8 calls ~32k $0.0374 90.7M; grok 3/3 37.0s 7 calls ~116k $0.1474 155.4M; graff-dev-repl 3/3 25.2s 9 calls ~64k $0.1185 91.8M. not Pareto (they win calls). oneshot-no-learn-pin retracted (path split).", "retracted": ["oneshot-no-learn-pin"], "ts": "2026-08-30T04:24:17Z"}
{"kind": "iterate", "candidate_id": "background-learn-init", "results": "/workspace/graff-evals/results/run-20260830-042417.jsonl", "keep": true, "why": "session-end learn init was 37.7s of suite gen after a hardlink pin; detach with the same process-group spawn as a trial so -p/REPL/TUI teardown does not wait. same 3-task set still not Pareto (calls 8/9 vs grok 7). TUI PTY fix-fib: OK at 5.9s, turn 7.4s, .graff 52K, no 127M pin (4 calls, learn-auto did not fire).", "ts": "2026-08-30T04:37:00Z"}