graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
Python
Compaction never summarizes tool results the model has not read
justrachdd864e24418b
diff --git a/docs/adr/0272-compaction-never-summarizes-unread-tool-results.md b/docs/adr/0272-compaction-never-summarizes-unread-tool-results.mdnew file mode 100644index 00000000..7687d96f--- /dev/null+++ b/docs/adr/0272-compaction-never-summarizes-unread-tool-results.md@@ -0,0 +1,49 @@+# 0272. Compaction never summarizes tool results the model has not read++Status: accepted 2026-10-09++## Context++The pre-send gate in the turn loop compacts as soon as the context crosses the+compaction point. A batch of large tool results is what usually pushes it over,+so the gate ran with that batch still unread. The cut keeps an ~8k-token+verbatim suffix ending at a clean user turn, and a batch of several large+results never fits in it. The summarizer therefore saw the batch, the model+never did, and the model answered from a summary that had dropped the specific+values. In one case it invented plausible ones instead of reading the files+again. In another it re-read them, the new batch crossed the line again, and+the cycle repeated.++On the Interactions wire no step carries a `role`, so `cleanUserTurn` found no+boundary and every compaction summarized the whole history.++Gemini's default compaction point was 80% of a 1,048,576-token window, about+839k tokens. Every request near that line carried the whole history at full+input price.++## Decision++- `compact_cut.unreadToolBatch` finds the model turn whose tool results end the+ history. `recentContextStart` never cuts after it, so that model turn and its+ results survive a compaction verbatim on every wire.+- The pre-send gate skips compaction when an unread batch ends the history and+ the summarizable prefix in front of it is under a quarter of the compaction+ point. Instead the request is sent, and compaction runs at the next check,+ once the model has read the batch. At 95% of the window the ordinary recovery+ runs regardless.+- The summary instruction asks for exact values to be copied character for+ character. It also asks the summary to name any result that was not read+ closely enough to copy from, so the model reads it again instead of recalling+ it.+- On the Interactions wire, a `user_input` step is a user turn boundary.+- Gemini's default compaction point is 80% of the window, capped at 300k tokens+ (`Provider.gemini_compact_tokens`). A percentage set through `--compact-at`,+ `GRAFF_COMPACT_PCT` or `/compact-at` applies as before.++## Consequences++A batch larger than the compaction point is sent once, over the line, instead+of being summarized unseen. The window still bounds it, and so do the+per-result caps. A compaction that keeps a large batch can leave the context+above the line until the next model turn. Gemini sessions compact earlier: more+summaries per long session, and smaller and cheaper requests between them.diff --git a/docs/adr/README.md b/docs/adr/README.mdindex 843abb92..f36b0ae3 100644--- a/docs/adr/README.md+++ b/docs/adr/README.md@@ -290,6 +290,7 @@ record only when you need the evidence or the edge cases.| [0261](0261-client-summary-stays-the-compactor.md) | The client summary stays the compactor: `GRAFF_CLEF_COMPACT` defaults off (Clef rarely pruned and mostly fell back to the summary in measurement); first-party Responses routes use only the provider's own compaction. Supersedes 0260's default. || [0270](0270-a-closed-stdio-mcp-server-restarts-on-its-next-call.md) | A stdio MCP server whose connection closes goes back to dormant with its launch config and restarts on its next call; the failed call is never replayed, and after two restarts in a session its tools are withdrawn as before. || [0271](0271-gemini-interactions-continue-the-stored-interaction.md) | Direct Gemini (Interactions) requests are stored and continue the last Interaction with `previous_interaction_id` plus only the new steps while a content hash proves graff's history extends it; any rewrite, model switch or rejected id falls back to a full replay; usage meters the whole prompt from the per-invocation counts. Tool choice is `validated`, `minimal` thinking maps to `low`, thought summaries stream, and Gemini 3.1 Pro with tools uses its custom-tools variant. `GRAFF_INTERACTIONS_STORE=0` keeps it stateless. |+| [0272](0272-compaction-never-summarizes-unread-tool-results.md) | A compaction never summarizes tool results the model has not read: the cut keeps the newest model turn and its results verbatim, and the pre-send gate sends a batch that dominates the context and compacts after the model reads it. Summaries copy exact values. Interactions `user_input` steps are turn boundaries. Gemini compacts by default at 80% capped at 300k tokens. |## When to write onediff --git a/src/agent_compact_test.zig b/src/agent_compact_test.zigindex 84ce8449..bcd02a88 100644--- a/src/agent_compact_test.zig+++ b/src/agent_compact_test.zig@@ -540,11 +540,12 @@ test "recentContextStart keeps NOTHING verbatim for a subagent-shaped history (tvar arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);defer arena_state.deinit();const a = arena_state.allocator();+ // Only the newest model turn and its unread result stay (unreadToolBatch).const short = try th.paddedSubHistory(a, 1, "ok");- try std.testing.expectEqual(short.items.len, recentContextStart(short.items, 8000));+ try std.testing.expectEqual(short.items.len - 2, recentContextStart(short.items, 8000));const pad = util.repeatBytes("x", 4000);const long = try th.paddedSubHistory(a, 10, &pad);- try std.testing.expectEqual(long.items.len, recentContextStart(long.items, 8000));+ try std.testing.expectEqual(long.items.len - 2, recentContextStart(long.items, 8000));}test "a compacting subagent's handoff restates its task prompt verbatim" {var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);diff --git a/src/agent_turn.zig b/src/agent_turn.zigindex 81e09337..35be120d 100644--- a/src/agent_turn.zig+++ b/src/agent_turn.zig@@ -43,7 +43,7 @@ pub fn run(self: *Agent, root_turn_prepared: *std.atomic.Value(bool)) anyerror![// meter (last_context_tokens, server-reported) catches up. Estimate the// full input locally and compact BEFORE sending so we never ship an// over-cap request (codex run_pre_sampling_compact / opencode isOverflow).- if (self.inputOverCompactThreshold()) {+ if (self.inputOverCompactThreshold() and !readBatchFirst(self)) {// Bracket the compaction with codex-WS resets, mirroring how every// other compactOrRecover call site is bookended by runTurn's// closeCodexWs (299 + defer 300). Mid-turn the WS can be live with a@@ -91,3 +91,21 @@ pub fn run(self: *Agent, root_turn_prepared: *std.atomic.Value(bool)) anyerror![self.empty_completion_retries = 0;}}++/// Over the compaction line, but the history ends on a tool batch the model+/// has not read and almost everything else is that batch. A compaction keeps+/// the batch verbatim (compact_cut.unreadToolBatch), so it could only+/// summarize the small prefix in front of it: a summary round trip that buys+/// little. Send the request instead and compact once the model has read the+/// batch. Near the window (95%) the ordinary recovery runs regardless.+fn readBatchFirst(self: *Agent) bool {+ const compact_cut = @import("compact_cut.zig");+ const items = self.messages.items;+ if (compact_cut.unreadToolBatch(items) == null) return false;+ if (self.provider.nearContextLimit(self.effectiveContextTokens())) return false;+ const start = compact_cut.recentContextStart(items, @import("agent_compact.zig").recent_context_tokens);+ const prefix = compact_cut.suffixTokens(items[0..start], 0);+ if (prefix >= self.provider.compactAt() / 4) return false;+ if (self.tracer) |tr| tr.note("compact", "deferred: the model has not read the newest tool results");+ return true;+}diff --git a/src/compact_at.zig b/src/compact_at.zigindex 33c51da9..b53162a6 100644--- a/src/compact_at.zig+++ b/src/compact_at.zig@@ -1,5 +1,6 @@//! Where automatic compaction starts, as a percent of the model's context-//! window (default 80; `Provider.compactAt`). The first one set wins:+//! window (default 80, capped at 300k tokens on Gemini; `Provider.compactAt`).+//! A set percentage applies as is. The first one set wins://!//! 1. `--compact-at N` on the command line (the SDK's `compactAt` option)//! 2. `GRAFF_COMPACT_PCT=N` (how Harness passes its setting)@@ -84,6 +85,13 @@ pub fn current() u8 {return if (provider_mod.g_compact_pct_override) |p| @min(p, 100) else default_pct;}+/// The percent the threshold actually sits at: the default is capped in+/// tokens on some models (Provider.gemini_compact_tokens).+fn shownPct(p: provider_mod.Provider) u64 {+ if (p.context == 0) return current();+ return (p.compactAt() * 100 + p.context / 2) / p.context;+}+/// `/compact-at` shows the point, `/compact-at N` sets it (and persists it/// for this workspace), `/compact-at default` goes back to 80%.pub fn handle(root: *Agent, line: []const u8, out: *Io.Writer) !bool {@@ -93,7 +101,7 @@ pub fn handle(root: *Agent, line: []const u8, out: *Io.Writer) !bool {const arg = std.mem.trim(u8, line[name.len..], " \t");if (arg.len == 0) {try out.print("compaction starts at {d}% of the context window ({d}k of {d}k tokens){s}\n", .{- current(),+ shownPct(root.provider),root.provider.compactAt() / 1000,root.provider.context / 1000,if (provider_mod.g_compact_pct_override == null) " · default" else "",@@ -101,7 +109,12 @@ pub fn handle(root: *Agent, line: []const u8, out: *Io.Writer) !bool {} else if (std.ascii.eqlIgnoreCase(arg, "default") or std.ascii.eqlIgnoreCase(arg, "reset")) {provider_mod.g_compact_pct_override = null;const saved = save(root.io, root.gpa, null);- try out.print("compaction starts at the default {d}% of the context window{s}\n", .{ default_pct, if (saved) "" else " (not persisted)" });+ try out.print("compaction starts at the default {d}% of the context window ({d}k of {d}k tokens){s}\n", .{+ shownPct(root.provider),+ root.provider.compactAt() / 1000,+ root.provider.context / 1000,+ if (saved) "" else " (not persisted)",+ });} else if (parse(arg)) |pct| {provider_mod.g_compact_pct_override = pct;const saved = save(root.io, root.gpa, pct);@@ -153,3 +166,21 @@ test "the threshold follows the setting in both directions" {provider_mod.g_compact_pct_override = parse("90");try std.testing.expectEqual(@as(u64, 180_000), p.compactAt());}++test "Gemini's default stops at 300k tokens; a set percentage still applies" {+ const saved_override = provider_mod.g_compact_pct_override;+ defer provider_mod.g_compact_pct_override = saved_override;+ provider_mod.g_compact_pct_override = null;+ for ([_][]const u8{ "gemini-3.8-flash", "google/gemini-3.8-flash" }) |model| {+ const p: provider_mod.Provider = .{ .id = "t", .kind = .interactions, .auth = .goog_api_key, .url = "", .api_key = "", .model = model, .context = 1_048_576 };+ try std.testing.expectEqual(@as(u64, 300_000), p.compactAt());+ try std.testing.expectEqual(@as(u64, 29), shownPct(p));+ }+ const small: provider_mod.Provider = .{ .id = "t", .kind = .openai, .auth = .bearer, .url = "", .api_key = "", .model = "gemini-small", .context = 200_000 };+ try std.testing.expectEqual(@as(u64, 160_000), small.compactAt()); // 80% is already under the cap+ const other: provider_mod.Provider = .{ .id = "t", .kind = .openai, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-6", .context = 1_000_000 };+ try std.testing.expectEqual(@as(u64, 800_000), other.compactAt());+ provider_mod.g_compact_pct_override = 80;+ const g: provider_mod.Provider = .{ .id = "t", .kind = .interactions, .auth = .goog_api_key, .url = "", .api_key = "", .model = "gemini-3.8-flash", .context = 1_000_000 };+ try std.testing.expectEqual(@as(u64, 800_000), g.compactAt());+}diff --git a/src/compact_cut.zig b/src/compact_cut.zigindex 31aadd18..ab39f562 100644--- a/src/compact_cut.zig+++ b/src/compact_cut.zig@@ -171,9 +171,41 @@ pub fn recentContextStart(items: []const Value, token_budget: u64) usize {if (pinOpening(items)) |t| {if (start > t) start = t;}+ if (unreadToolBatch(items)) |u| {+ if (start > u) start = u;+ }return start;}+/// A tool result on any wire, including the Interactions `function_result` step.+fn isToolResult(m: Value) bool {+ if (@import("tool_spill.zig").isToolOutputMsg(m)) return true;+ if (m != .object) return false;+ const t = m.object.get("type") orelse return false;+ return t == .string and std.mem.eql(u8, t.string, "function_result");+}++fn isUserMessage(m: Value) bool {+ if (m != .object) return false;+ if (m.object.get("role")) |r| return r == .string and std.mem.eql(u8, r.string, "user");+ const t = m.object.get("type") orelse return false;+ return t == .string and std.mem.eql(u8, t.string, "user_input");+}++/// When history ends on tool results the model has not read yet, the index of+/// the model turn that asked for them (its calls, plus any text or reasoning+/// beside them). A compaction keeps everything from there verbatim: summarizing+/// a batch before the model's first look hands it a summary in place of the+/// results, and it answers from the summary — or guesses — instead of from+/// what the tools returned.+pub fn unreadToolBatch(items: []const Value) ?usize {+ var i = items.len;+ while (i > 0 and isToolResult(items[i - 1])) i -= 1;+ if (i == items.len) return null;+ while (i > 0 and !isToolResult(items[i - 1]) and !isUserMessage(items[i - 1])) i -= 1;+ return i;+}+/// Index to cut history at for an emergency trim: the first clean user turn/// at or after the midpoint, so messages[cut..] is always a valid/// conversation start (never an orphaned tool_result). null when there is no@@ -204,7 +236,7 @@ pub fn noteCut(tracer: ?*trace.Tracer, items: []const Value, start: usize) voidpub fn cleanUserTurn(m: Value) bool {if (m != .object) return false;- const role = m.object.get("role") orelse return false;+ const role = m.object.get("role") orelse return interactionsUserTurn(m.object);if (role != .string or !std.mem.eql(u8, role.string, "user")) return false;const content = m.object.get("content") orelse return true;switch (content) {@@ -224,6 +256,14 @@ pub fn cleanUserTurn(m: Value) bool {}}+/// The Interactions wire has no roles: a user turn is a `user_input` step.+fn interactionsUserTurn(obj: std.json.ObjectMap) bool {+ const t = obj.get("type") orelse return false;+ if (t != .string or !std.mem.eql(u8, t.string, "user_input")) return false;+ const content = obj.get("content") orelse return true;+ return content != .string or !peer_context.isPeerInjectContent(content.string);+}+fn imageUser(arena: std.mem.Allocator, kind: []const u8, payload: []const u8) !Value {var image: std.json.ObjectMap = .empty;try image.put(arena, "type", .{ .string = kind });@@ -437,3 +477,59 @@ test "noteStall: same boundary with no shrink escalates; a real shrink resets" {try std.testing.expect(!noteStall(&s, 10, 39_000));try std.testing.expect(!noteStall(&s, 20, 80_000));}++fn parseTest(arena: std.mem.Allocator, text: []const u8) !Value {+ return std.json.parseFromSliceLeaky(Value, arena, text, .{});+}++test "an unread tool batch stays verbatim through a compaction cut" {+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);+ defer arena_state.deinit();+ const arena = arena_state.allocator();+ const util = @import("util.zig");+ const big = &util.repeatBytes("r", 40_000);++ // First turn of a session: the prompt, one model turn of calls, and its+ // results. Before, the whole run was summarized (start == items.len).+ var msgs = std.json.Array.init(arena);+ try msgs.append(try parseTest(arena, "{\"role\":\"user\",\"content\":\"read every part\"}"));+ try msgs.append(try parseTest(arena, "{\"role\":\"assistant\",\"content\":\"reading\",\"tool_calls\":[{\"id\":\"a\"},{\"id\":\"b\"}]}"));+ for ([_][]const u8{ "a", "b" }) |id| {+ var o: std.json.ObjectMap = .empty;+ try o.put(arena, "role", .{ .string = "tool" });+ try o.put(arena, "tool_call_id", .{ .string = id });+ try o.put(arena, "content", .{ .string = big });+ try msgs.append(.{ .object = o });+ }+ try std.testing.expectEqual(@as(?usize, 1), unreadToolBatch(msgs.items));+ try std.testing.expectEqual(@as(usize, 1), recentContextStart(msgs.items, 8_000));++ // Once the model has answered, the batch is ordinary history again.+ try msgs.append(try parseTest(arena, "{\"role\":\"assistant\",\"content\":\"done\"}"));+ try std.testing.expectEqual(@as(?usize, null), unreadToolBatch(msgs.items));++ // A read batch, then a new one: only the newest model turn is held back.+ try msgs.append(try parseTest(arena, "{\"role\":\"user\",\"content\":\"again\"}"));+ try msgs.append(try parseTest(arena, "{\"role\":\"assistant\",\"content\":null,\"tool_calls\":[{\"id\":\"c\"}]}"));+ try msgs.append(try parseTest(arena, "{\"role\":\"tool\",\"tool_call_id\":\"c\",\"content\":\"x\"}"));+ try msgs.append(try parseTest(arena, "{\"role\":\"assistant\",\"content\":null,\"tool_calls\":[{\"id\":\"d\"}]}"));+ try msgs.append(try parseTest(arena, "{\"role\":\"tool\",\"tool_call_id\":\"d\",\"content\":\"y\"}"));+ try std.testing.expectEqual(@as(?usize, 8), unreadToolBatch(msgs.items));+}++test "Interactions steps: user_input is a turn boundary and function_result an unread batch" {+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);+ defer arena_state.deinit();+ const arena = arena_state.allocator();+ var msgs = std.json.Array.init(arena);+ try msgs.append(try parseTest(arena, "{\"type\":\"user_input\",\"content\":\"first\"}"));+ try msgs.append(try parseTest(arena, "{\"type\":\"model_output\",\"content\":[{\"type\":\"text\",\"text\":\"ok\"}]}"));+ try msgs.append(try parseTest(arena, "{\"type\":\"user_input\",\"content\":\"second\"}"));+ try msgs.append(try parseTest(arena, "{\"type\":\"thought\",\"signature\":\"s\"}"));+ try msgs.append(try parseTest(arena, "{\"type\":\"function_call\",\"id\":\"c1\",\"name\":\"read_file\",\"arguments\":{}}"));+ try msgs.append(try parseTest(arena, "{\"type\":\"function_result\",\"call_id\":\"c1\",\"name\":\"read_file\",\"result\":[{\"type\":\"text\",\"text\":\"x\"}]}"));+ try std.testing.expect(cleanUserTurn(msgs.items[2]));+ try std.testing.expect(!cleanUserTurn(msgs.items[5]));+ try std.testing.expectEqual(@as(?usize, 3), unreadToolBatch(msgs.items));+ try std.testing.expectEqual(@as(?usize, 2), turnOpeningUserIndex(msgs.items));+}diff --git a/src/compact_handoff_note.zig b/src/compact_handoff_note.zigindex dcb0ae09..06606e7f 100644--- a/src/compact_handoff_note.zig+++ b/src/compact_handoff_note.zig@@ -45,7 +45,25 @@ const Value = std.json.Value;const agent_mod = @import("agent.zig");const Agent = agent_mod.Agent;-const compact_instruction = @import("prompts.zig").compact_instruction;+/// What the summarizer is asked for (re-exported as prompts.compact_instruction).+pub const compact_instruction =+ \\Summarize this entire conversation for a context handoff. Capture: the+ \\user's goals, all important facts and decisions, file paths and code+ \\that was created or modified, command results that matter, the current+ \\task checklist and each item's status, and any pending or unfinished+ \\work. Be thorough but compact. Reply with only the summary.+ \\Copy exact values the remaining work depends on (names, ids, numbers,+ \\error text, anything the user asked to have reported) character for+ \\character. Where a tool result was not read closely enough to copy from,+ \\say so and name the file or command, so it is read again rather than+ \\recalled.+ \\Report only what this conversation actually contains. Do not write that a+ \\rule, constraint or preference was recorded — recorded constraints live in+ \\a durable ledger injected separately (with caps), so a summary+ \\that asserts one invents it, and the agent reading you cannot tell the+ \\difference (#738). A standing instruction the user gave in words belongs+ \\here as what they said, attributed to them, not as machine state.+;const session_transcript = @import("session_transcript.zig");const tool_spill = @import("tool_spill.zig");const peer_context = @import("peer_context.zig");diff --git a/src/prompts.zig b/src/prompts.zigindex e2c4c4d5..96f292fd 100644--- a/src/prompts.zig+++ b/src/prompts.zig@@ -472,19 +472,7 @@ pub const sub_system_prompt =\\Files changed, Verified, Skipped, and Open questions (omit empty).++ parallel_tools_note ++ text.public_write_note ++ text.constraint_authority_note ++ text.claim_ownership_note;-pub const compact_instruction =- \\Summarize this entire conversation for a context handoff. Capture: the- \\user's goals, all important facts and decisions, file paths and code- \\that was created or modified, command results that matter, the current- \\task checklist and each item's status, and any pending or unfinished- \\work. Be thorough but compact. Reply with only the summary.- \\Report only what this conversation actually contains. Do not write that a- \\rule, constraint or preference was recorded — recorded constraints live in- \\a durable ledger injected separately (with caps), so a summary- \\that asserts one invents it, and the agent reading you cannot tell the- \\difference (#738). A standing instruction the user gave in words belongs- \\here as what they said, attributed to them, not as machine state.-;+pub const compact_instruction = @import("compact_handoff_note.zig").compact_instruction;test { // #421/#410: prompt snapshots and goal-prefix behavior must stay reachable._ = @import("prompt_snapshot_tests.zig");diff --git a/src/provider.zig b/src/provider.zigindex 7ae665e5..3d000e34 100644--- a/src/provider.zig+++ b/src/provider.zig@@ -210,8 +210,18 @@ pub const Provider = struct {/// GRAFF_COMPACT_PCT overrides the percentage, clamped to 1..100 (#204). Unlike/// codex's one-directional clamp, the override may lower OR raise the threshold.pub fn compactAt(p: Provider) u64 {- const pct: u64 = if (g_compact_pct_override) |o| @min(o, 100) else 80;- return p.context / 100 * pct;+ if (g_compact_pct_override) |o| return p.context / 100 * @min(o, 100);+ const at = p.context / 100 * 80;+ return if (isGeminiModel(p.model)) @min(at, gemini_compact_tokens) else at;+ }++ /// Gemini's default compaction point unless a percentage is set: 80% of its+ /// 1M window put ~800k billed tokens on every request near the line (ADR 0272).+ pub const gemini_compact_tokens: u64 = 300_000;++ pub fn isGeminiModel(model: []const u8) bool {+ const bare = if (std.mem.lastIndexOfScalar(u8, model, '/')) |i| model[i + 1 ..] else model;+ return std.mem.startsWith(u8, bare, "gemini-");}/// The provider's explicit server-side compaction endpoint, if we use it.diff --git a/src/provider_tests.zig b/src/provider_tests.zigindex 5b0c9fc3..299e1817 100644--- a/src/provider_tests.zig+++ b/src/provider_tests.zig@@ -56,6 +56,7 @@ test "perOutputCap (#201): window-proportional with an absolute ceiling, keep_retest "compactAt (#204): GRAFF_COMPACT_PCT overrides the 80% default, both directions" {var p: Provider = undefined;p.context = 100_000;+ p.model = "m";try std.testing.expectEqual(@as(u64, 80_000), p.compactAt());provider_mod.g_compact_pct_override = 70;defer provider_mod.g_compact_pct_override = null;@@ -68,11 +69,12 @@ test "gemini routes to google, not the gateway, and carries the real 1M window"const all = Keys{ .values = @splat("k") };// Before the google row existed, gemini-* was uncatalogued: providerFor fell// through to the codegraff gateway and contextFor returned default_context,- // so a 1,048,576-token model auto-compacted at 160k instead of 838k.+ // so a 1,048,576-token model auto-compacted at 160k. The window is real;+ // the default compaction point is Gemini's own token cap.const gemini = try all.providerFor("gemini-3.8-flash");try std.testing.expectEqualStrings("google", gemini.id);try std.testing.expectEqual(@as(u64, 1_048_576), gemini.context);- try std.testing.expect(gemini.compactAt() > 800_000);+ try std.testing.expectEqual(Provider.gemini_compact_tokens, gemini.compactAt());// Gemini talks Google's first-party Interactions wire, not the OpenAI// compatibility shim, and authenticates with x-goog-api-key.try std.testing.expectEqual(Provider.Kind.interactions, gemini.kind);