graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
Python
Resumed sessions keep their compaction-state count (#1686)
Rach Pradhan147dc8e3362f
diff --git a/docs/adr/0307-resumed-sessions-keep-their-compaction-state-count.md b/docs/adr/0307-resumed-sessions-keep-their-compaction-state-count.mdnew file mode 100644index 00000000..aad6397a--- /dev/null+++ b/docs/adr/0307-resumed-sessions-keep-their-compaction-state-count.md@@ -0,0 +1,42 @@+# 0307. Resumed sessions keep their compaction-state count++Status: accepted 2026-10-29++## Context++A Responses compaction item carries opaque encrypted state. Its byte size is a+poor proxy for its token cost, so graff estimates it at bytes/16 until the+provider counts a full request carrying it; that count then replaces the+estimate (#1686, `opaque_state_usage.zig`). The count lived only in+process-global slots in `context_tokens.zig`.++A new process restoring the same session therefore went back to bytes/16. In a+controlled reproduction, an 8,000,000-byte state counted at about 80k tokens+was restored as about 508k in a 270k window. The ACP meter showed 188%, and the+next ordinary prompt paid for a forced compaction that the warm process did not+need. Saving the paired meter counters was not enough, because restore+recomputes the local component without the count.++## Decision++The session file records the count as `opaque_state_count`:++- `provider` and `model` the count was made under,+- `state_bytes`, the history's total compaction-state size (the calibration key),+- `state_digest`, an order-independent digest of every state blob, in hex,+- `tokens`, the provider's count for that state.++It is written only when the process has a count for the saved history's state.+`loadSession` reapplies it before rebasing the context meter, and only when the+restored history has the same size and digest and the session's provider and+model match. A changed state, a different model or provider, or a missing or+malformed record leaves the bytes/16 estimate in place.++## Consequences++A resumed session reports the occupancy the provider last counted, and does not+compact solely because the process restarted. Genuine overflow protection is+unchanged: the compaction threshold and the near-limit gate still apply to the+restored estimate, and history added after the count is still counted locally.+Sessions saved before this change, or whose state was never counted, restore as+before and recalibrate on their next full request.diff --git a/docs/adr/README.md b/docs/adr/README.mdindex c9802646..79e1e387 100644--- a/docs/adr/README.md+++ b/docs/adr/README.md@@ -331,6 +331,7 @@ record only when you need the evidence or the edge cases.| [0304](0304-file-tools-allow-in-root-symlinks.md) | File tools allow canonical in-root symlinks, reject escaping or unresolved links, and retain lexical confinement and missing write targets; the check is not an atomic sandbox. || [0305](0305-cancelled-checklist-items-carry-a-reason.md) | A `cancelled` todo needs a short reason or the write is rejected whole; it stays listed with that reason, stops blocking completion and never counts as verified. A verification item can be cancelled after a newer user message or an answered `ask_user`. || [0306](0306-model-catalogs-never-hold-startup.md) | Boot and `graff/models` never wait on a catalog GET past a budget: cached catalogs activate now and refresh behind on detached workers whose only output is the disk cache; only uncached ones wait, together, up to 800ms. |+| [0307](0307-resumed-sessions-keep-their-compaction-state-count.md) | A session saves the provider's count of its compaction state, scoped to provider, model, size and digest; loading reapplies it before rebasing the meter, so a restart neither inflates the meter nor forces a compaction. || [0302](0302-sse-idle-watchdogs-follow-body-byte-progress.md) | SSE idle watchdogs observe partial body-byte progress, while separate unfinished-event time and size bounds preserve recovery. |diff --git a/src/context_tokens.zig b/src/context_tokens.zigindex a1be6008..4d1e0be8 100644--- a/src/context_tokens.zig+++ b/src/context_tokens.zig@@ -52,10 +52,52 @@ pub fn stateTokens(bytes: usize) u64 {return bytes / compaction_state_bytes_per_token;}+/// The provider's count for `bytes` of compaction state, or null while this+/// process has only the bytes/16 estimate.+pub fn calibratedState(bytes: usize) ?u64 {+ if (bytes == 0) return null;+ for (&g_state_counts) |*slot| {+ const v = slot.load(.monotonic);+ if (v >> token_bits == bytes) return v & token_mask;+ }+ return null;+}+pub fn compactionStateBytes(value: Value) usize {return mediaStats(value).compaction_state_bytes;}+/// Which compaction state a history carries: its total size (the calibration+/// key) and an order-independent digest of every state blob, so a persisted+/// count is reapplied only to the same state, never to a different blob that+/// happens to have the same length.+pub const StateIdentity = struct { bytes: usize = 0, digest: u64 = 0 };++pub fn compactionStateIdentity(value: Value) StateIdentity {+ var id: StateIdentity = .{};+ switch (value) {+ .array => |items| for (items.items) |item| {+ const child = compactionStateIdentity(item);+ id.bytes +|= child.bytes;+ id.digest +%= child.digest;+ },+ .object => |obj| {+ if (compactionStateLen(obj)) |len| {+ id.bytes +|= len;+ id.digest +%= std.hash.Wyhash.hash(0, stringValue(obj.get("encrypted_content")).?);+ }+ var it = obj.iterator();+ while (it.next()) |entry| {+ const child = compactionStateIdentity(entry.value_ptr.*);+ id.bytes +|= child.bytes;+ id.digest +%= child.digest;+ }+ },+ else => {},+ }+ return id;+}+/// Learn the state's share from a counted request: the reported input less/// the estimate for everything else in it (`request_estimate` includes the/// state at its current estimate). Never below one token per KiB of state.diff --git a/src/opaque_state_persist.zig b/src/opaque_state_persist.zignew file mode 100644index 00000000..c334829f--- /dev/null+++ b/src/opaque_state_persist.zig@@ -0,0 +1,78 @@+//! #1686: a resumed session keeps the provider's count of its compaction state.+//!+//! The count a full request teaches (opaque_state_usage.zig) lives in+//! process-global slots, so a new process restoring the same history fell back+//! to bytes/16. An 8 MB state counted at ~80k then read as ~508k: the restored+//! meter showed 188% and the next prompt paid for a forced compaction the warm+//! process never needed. The count is saved with the session, scoped to the+//! provider, the model, and the exact state (size plus a digest of every blob),+//! and reapplied on load only when all of them still match. A changed state,+//! a different model, or a missing or malformed record leaves bytes/16 in place.++const std = @import("std");+const Value = std.json.Value;+const context_tokens = @import("context_tokens.zig");+const Provider = @import("provider.zig").Provider;++pub const field = "opaque_state_count";++/// Write the record when this process has counted the history's state.+pub fn write(s: *std.json.Stringify, provider: Provider, history: Value) !void {+ const id = context_tokens.compactionStateIdentity(history);+ const tokens = context_tokens.calibratedState(id.bytes) orelse return;+ var digest_buf: [16]u8 = undefined;+ try s.objectField(field);+ try s.beginObject();+ try s.objectField("provider");+ try s.write(provider.id);+ try s.objectField("model");+ try s.write(provider.model);+ try s.objectField("state_bytes");+ try s.write(id.bytes);+ // Hex: a u64 digest does not round-trip through JSON's signed integers.+ try s.objectField("state_digest");+ try s.write(std.fmt.bufPrint(&digest_buf, "{x:0>16}", .{id.digest}) catch unreachable);+ try s.objectField("tokens");+ try s.write(tokens);+ try s.endObject();+}++/// Reapply a saved count to the restored history. Call after the history is+/// in its final restored form and the session's provider is active.+pub fn restore(provider: Provider, history: Value, session: std.json.ObjectMap) void {+ const saved = recordFrom(session.get(field) orelse return) orelse return;+ if (!std.mem.eql(u8, saved.provider, provider.id) or !std.mem.eql(u8, saved.model, provider.model)) return;+ const id = context_tokens.compactionStateIdentity(history);+ if (id.bytes != saved.state_bytes or id.digest != saved.state_digest) return;+ context_tokens.calibrateState(saved.state_bytes, saved.tokens);+}++const Record = struct {+ provider: []const u8,+ model: []const u8,+ state_bytes: usize,+ state_digest: u64,+ tokens: u64,+};++fn recordFrom(value: Value) ?Record {+ if (value != .object) return null;+ const obj = value.object;+ return .{+ .provider = str(obj.get("provider")) orelse return null,+ .model = str(obj.get("model")) orelse return null,+ .state_bytes = @intCast(positive(obj.get("state_bytes")) orelse return null),+ .state_digest = std.fmt.parseInt(u64, str(obj.get("state_digest")) orelse return null, 16) catch return null,+ .tokens = positive(obj.get("tokens")) orelse return null,+ };+}++fn str(value: ?Value) ?[]const u8 {+ const v = value orelse return null;+ return if (v == .string) v.string else null;+}++fn positive(value: ?Value) ?u64 {+ const v = value orelse return null;+ return if (v == .integer and v.integer > 0) @intCast(v.integer) else null;+}diff --git a/src/opaque_state_persist_tests.zig b/src/opaque_state_persist_tests.zignew file mode 100644index 00000000..3f450a51--- /dev/null+++ b/src/opaque_state_persist_tests.zig@@ -0,0 +1,199 @@+//! #1686: a resumed session keeps the provider's count of its compaction state.++const std = @import("std");+const Value = std.json.Value;+const Agent = @import("agent.zig").Agent;+const context_tokens = @import("context_tokens.zig");+const persist = @import("opaque_state_persist.zig");+const session = @import("session.zig");++const window: u64 = 270_000;+const state_len: usize = 8_000_000;++fn history(a: std.mem.Allocator, fill: u8) !std.json.Array {+ const head = "{\"type\":\"compaction\",\"encrypted_content\":\"";+ const json = try a.alloc(u8, head.len + state_len + 2);+ @memcpy(json[0..head.len], head);+ @memset(json[head.len .. head.len + state_len], fill);+ @memcpy(json[head.len + state_len ..], "\"}");+ var msgs = std.json.Array.init(a);+ try msgs.append(try std.json.parseFromSliceLeaky(Value, a, json, .{}));+ try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"role\":\"user\",\"content\":\"hi\"}", .{}));+ return msgs;+}++fn agent(msgs: std.json.Array) Agent {+ var root: Agent = undefined;+ root.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = window };+ root.messages = msgs;+ root.sub = false;+ root.strict = false;+ root.sys_normal = "";+ root.sys_strict = "";+ root.tools_responses = "";+ root.last_cache_read = 0;+ return root;+}++/// What a warm process saves after the provider counted the state at 80k:+/// the meter pair and the persisted count, as one session object.+const Saved = struct { context: u64, local: u64, obj: std.json.ObjectMap };++fn countAndSave(a: std.mem.Allocator, root: *Agent) !Saved {+ context_tokens.learnState(state_len, 80_000, root.fullRequestEstimateTokens());+ const local = root.fullRequestEstimateTokens();+ var aw: std.Io.Writer.Allocating = .init(a);+ var s: std.json.Stringify = .{ .writer = &aw.writer };+ try s.beginObject();+ try persist.write(&s, root.provider, .{ .array = root.messages });+ try s.endObject();+ const parsed = try std.json.parseFromSliceLeaky(Value, a, aw.writer.buffered(), .{});+ return .{ .context = 80_001, .local = local, .obj = parsed.object };+}++/// A new process: empty calibration, then loadSession's restore order.+fn reload(root: *Agent, saved: Saved) void {+ context_tokens.resetStateCountsForTest();+ persist.restore(root.provider, .{ .array = root.messages }, saved.obj);+ session.restoreContextMeter(root, saved.context, saved.local);+}++test "a reloaded session keeps the provider's count of its compaction state (#1686)" {+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);+ defer arena_state.deinit();+ const a = arena_state.allocator();+ context_tokens.resetStateCountsForTest();+ defer context_tokens.resetStateCountsForTest();++ var root = agent(try history(a, 'A'));+ const saved = try countAndSave(a, &root);+ try std.testing.expect(saved.obj.get(persist.field) != null);++ // Without the record a new process reads the state at bytes/16 (the bug).+ var cold = saved;+ cold.obj = (try std.json.parseFromSliceLeaky(Value, a, "{}", .{})).object;+ reload(&root, cold);+ try std.testing.expect(root.effectiveContextTokens() > window);+ try std.testing.expect(root.inputOverCompactThreshold());++ // With it, the reload keeps the counted occupancy and needs no compaction.+ reload(&root, saved);+ try std.testing.expect(root.effectiveContextTokens() < 100_000);+ try std.testing.expect(!root.inputOverCompactThreshold());+}++test "a saved state count applies only to the same state, provider, and model (#1686)" {+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);+ defer arena_state.deinit();+ const a = arena_state.allocator();+ context_tokens.resetStateCountsForTest();+ defer context_tokens.resetStateCountsForTest();++ var counted = agent(try history(a, 'A'));+ const saved = try countAndSave(a, &counted);++ // Same length, different blob: the digest refuses it.+ var other = agent(try history(a, 'B'));+ reload(&other, saved);+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);+ try std.testing.expect(other.inputOverCompactThreshold());++ var moved = agent(counted.messages);+ moved.provider.model = "gpt-6";+ reload(&moved, saved);+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);+ moved.provider.model = "gpt-5";+ moved.provider.id = "openai";+ reload(&moved, saved);+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);++ const malformed = [_][]const u8{+ "{\"opaque_state_count\":{\"provider\":\"codex\",\"model\":\"gpt-5\",\"state_bytes\":8000000,\"state_digest\":\"zz\",\"tokens\":72000}}",+ "{\"opaque_state_count\":{\"provider\":\"codex\",\"model\":\"gpt-5\",\"state_bytes\":8000000,\"state_digest\":\"0\",\"tokens\":-1}}",+ "{\"opaque_state_count\":\"codex\"}",+ "{}",+ };+ for (malformed) |text| {+ const parsed = try std.json.parseFromSliceLeaky(Value, a, text, .{});+ reload(&counted, .{ .context = saved.context, .local = saved.local, .obj = parsed.object });+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);+ }+}++test "an uncounted compaction state writes no record (#1686)" {+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);+ defer arena_state.deinit();+ const a = arena_state.allocator();+ context_tokens.resetStateCountsForTest();+ defer context_tokens.resetStateCountsForTest();++ const root = agent(try history(a, 'A'));+ var aw: std.Io.Writer.Allocating = .init(a);+ var s: std.json.Stringify = .{ .writer = &aw.writer };+ try s.beginObject();+ try persist.write(&s, root.provider, .{ .array = root.messages });+ try s.endObject();+ try std.testing.expectEqualStrings("{}", aw.writer.buffered());+}++test "session save and load both carry the compaction-state count (#1686)" {+ const src = @embedFile("session.zig");+ try std.testing.expect(std.mem.indexOf(u8, src, "opaque_state_persist.zig\").write(&s, root.provider, Value{ .array = saved_msgs })") != null);+ // Restore runs before the meter is rebased, or the rebase reads bytes/16.+ const restore_at = std.mem.indexOf(u8, src, "opaque_state_persist.zig\").restore(root.provider") orelse return error.TestUnexpectedResult;+ const rebase_at = std.mem.indexOf(u8, src, " restoreContextMeter(root, saved_context_tokens, saved_local_tokens);") orelse return error.TestUnexpectedResult;+ try std.testing.expect(restore_at < rebase_at);+}++test "a session saved after a counted request loads without a forced compaction (#1686)" {+ const writer = @import("session_writer.zig");+ const transcript = @import("session_transcript.zig");+ const gpa = std.testing.allocator;+ const io = std.testing.io;+ var arena_state = std.heap.ArenaAllocator.init(gpa);+ defer arena_state.deinit();+ const a = arena_state.allocator();+ writer.resetForTest();+ defer writer.resetForTest();+ transcript.resetForTest();+ defer transcript.resetForTest();+ context_tokens.resetStateCountsForTest();+ defer context_tokens.resetStateCountsForTest();+ var tmp = std.testing.tmpDir(.{});+ defer tmp.cleanup();+ var client: std.http.Client = .{ .allocator = gpa, .io = io };+ defer client.deinit();+ var keys: @import("provider.zig").Keys = .{ .values = @splat("test-key") };+ var provider = try keys.providerById("openai", "gpt-6-sol");+ provider.context = window;+ var root: Agent = .{+ .gpa = gpa,+ .arena = a,+ .io = io,+ .client = &client,+ .provider = provider,+ .messages = try history(a, 'A'),+ .sub = false,+ .label = "root",+ .out = null,+ .home = try std.fmt.allocPrint(a, ".zig-cache/tmp/{s}", .{tmp.sub_path}),+ };+ try root.ensureRootTools(root.provider.kind);++ // Warm process: the provider counted a full request carrying the state at 80k.+ context_tokens.learnState(state_len, 80_000, root.fullRequestEstimateTokens());+ root.last_context_tokens = 80_001;+ root.context_local_tokens = root.fullRequestEstimateTokens();+ try std.testing.expect(!root.inputOverCompactThreshold());+ try session.saveSessionTo(&root, a, tmp.dir, "reload-1686");+ session.flushSaves();++ // New process: no calibration in memory, then the real load path.+ context_tokens.resetStateCountsForTest();+ root.last_context_tokens = 0;+ root.context_local_tokens = 0;+ try session.loadSession(&root, &keys, a, "reload-1686");+ root.provider.context = window;+ try std.testing.expect(root.effectiveContextTokens() < 100_000);+ try std.testing.expect(!root.inputOverCompactThreshold());+}diff --git a/src/session.zig b/src/session.zigindex 699cb1a2..14d4479d 100644--- a/src/session.zig+++ b/src/session.zig@@ -355,6 +355,9 @@ fn queueSave(root: *Agent, arena: Allocator, dir: Io.Dir, name: []const u8) !u64// this component with today's prompt/tool-schema estimate.try s.objectField("context_local_tokens");try s.write(@min(context_estimate.local, @as(u64, std.math.maxInt(i64))));+ // #1686: the local component above uses this process's count of the+ // compaction state; a resume needs that count, not bytes/16.+ try @import("opaque_state_persist.zig").write(&s, root.provider, Value{ .array = saved_msgs });try root.compaction_window.write(&s);// #330: the --json protocol sequence high-water mark. A REPLACEMENT graff// resuming this session continues the numbering from here instead of@@ -542,6 +545,9 @@ pub fn loadSession(root: *Agent, keys: *Keys, arena: Allocator, name: []const u8root.goal_note_age = 0;root.session_parent = parent;root.session_title = title;+ // #1686: reapply the saved count of this exact compaction state first, so+ // the rebased meter does not fall back to bytes/16 in a new process.+ @import("opaque_state_persist.zig").restore(root.provider, .{ .array = root.messages }, obj);// Rebase the saved server-only delta onto today's prompt/tool-schema input.restoreContextMeter(root, saved_context_tokens, saved_local_tokens);root.cap_new = false; // per-provider (like sox/effort below); relearn on rejectiondiff --git a/src/test_hooks.zig b/src/test_hooks.zigindex 5c48814e..1611424b 100644--- a/src/test_hooks.zig+++ b/src/test_hooks.zig@@ -340,6 +340,7 @@ test {_ = @import("mcp_connect_notice.zig"); // ADR 0230_ = @import("image_sniff.zig"); // #1762: image media type from the bytes_ = @import("catalog_bounded.zig"); // model discovery: catalog GETs never hold boot past a budget+ _ = @import("opaque_state_persist_tests.zig"); // #1686: a resumed session keeps its compaction-state count_ = @import("oauth_callback.zig");_ = @import("cache_key_tests.zig"); // ADR 0223: moved off http_headers.zig (600-line cap)_ = @import("router_headers_tests.zig"); // moved off http_headers.zig (600-line cap)