Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

Resumed sessions keep their compaction-state count (#1686)

Rach Pradhan147dc8e3362f
diff --git a/docs/adr/0307-resumed-sessions-keep-their-compaction-state-count.md b/docs/adr/0307-resumed-sessions-keep-their-compaction-state-count.md
new file mode 100644
index 00000000..aad6397a
--- /dev/null
+++ b/docs/adr/0307-resumed-sessions-keep-their-compaction-state-count.md
@@ -0,0 +1,42 @@
+# 0307. Resumed sessions keep their compaction-state count
+
+Status: accepted 2026-10-29
+
+## Context
+
+A Responses compaction item carries opaque encrypted state. Its byte size is a
+poor proxy for its token cost, so graff estimates it at bytes/16 until the
+provider counts a full request carrying it; that count then replaces the
+estimate (#1686, `opaque_state_usage.zig`). The count lived only in
+process-global slots in `context_tokens.zig`.
+
+A new process restoring the same session therefore went back to bytes/16. In a
+controlled reproduction, an 8,000,000-byte state counted at about 80k tokens
+was restored as about 508k in a 270k window. The ACP meter showed 188%, and the
+next ordinary prompt paid for a forced compaction that the warm process did not
+need. Saving the paired meter counters was not enough, because restore
+recomputes the local component without the count.
+
+## Decision
+
+The session file records the count as `opaque_state_count`:
+
+- `provider` and `model` the count was made under,
+- `state_bytes`, the history's total compaction-state size (the calibration key),
+- `state_digest`, an order-independent digest of every state blob, in hex,
+- `tokens`, the provider's count for that state.
+
+It is written only when the process has a count for the saved history's state.
+`loadSession` reapplies it before rebasing the context meter, and only when the
+restored history has the same size and digest and the session's provider and
+model match. A changed state, a different model or provider, or a missing or
+malformed record leaves the bytes/16 estimate in place.
+
+## Consequences
+
+A resumed session reports the occupancy the provider last counted, and does not
+compact solely because the process restarted. Genuine overflow protection is
+unchanged: the compaction threshold and the near-limit gate still apply to the
+restored estimate, and history added after the count is still counted locally.
+Sessions saved before this change, or whose state was never counted, restore as
+before and recalibrate on their next full request.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index c9802646..79e1e387 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -331,6 +331,7 @@ record only when you need the evidence or the edge cases.
| [0304](0304-file-tools-allow-in-root-symlinks.md) | File tools allow canonical in-root symlinks, reject escaping or unresolved links, and retain lexical confinement and missing write targets; the check is not an atomic sandbox. |
| [0305](0305-cancelled-checklist-items-carry-a-reason.md) | A `cancelled` todo needs a short reason or the write is rejected whole; it stays listed with that reason, stops blocking completion and never counts as verified. A verification item can be cancelled after a newer user message or an answered `ask_user`. |
| [0306](0306-model-catalogs-never-hold-startup.md) | Boot and `graff/models` never wait on a catalog GET past a budget: cached catalogs activate now and refresh behind on detached workers whose only output is the disk cache; only uncached ones wait, together, up to 800ms. |
+| [0307](0307-resumed-sessions-keep-their-compaction-state-count.md) | A session saves the provider's count of its compaction state, scoped to provider, model, size and digest; loading reapplies it before rebasing the meter, so a restart neither inflates the meter nor forces a compaction. |
| [0302](0302-sse-idle-watchdogs-follow-body-byte-progress.md) | SSE idle watchdogs observe partial body-byte progress, while separate unfinished-event time and size bounds preserve recovery. |
diff --git a/src/context_tokens.zig b/src/context_tokens.zig
index a1be6008..4d1e0be8 100644
--- a/src/context_tokens.zig
+++ b/src/context_tokens.zig
@@ -52,10 +52,52 @@ pub fn stateTokens(bytes: usize) u64 {
return bytes / compaction_state_bytes_per_token;
}
+/// The provider's count for `bytes` of compaction state, or null while this
+/// process has only the bytes/16 estimate.
+pub fn calibratedState(bytes: usize) ?u64 {
+ if (bytes == 0) return null;
+ for (&g_state_counts) |*slot| {
+ const v = slot.load(.monotonic);
+ if (v >> token_bits == bytes) return v & token_mask;
+ }
+ return null;
+}
+
pub fn compactionStateBytes(value: Value) usize {
return mediaStats(value).compaction_state_bytes;
}
+/// Which compaction state a history carries: its total size (the calibration
+/// key) and an order-independent digest of every state blob, so a persisted
+/// count is reapplied only to the same state, never to a different blob that
+/// happens to have the same length.
+pub const StateIdentity = struct { bytes: usize = 0, digest: u64 = 0 };
+
+pub fn compactionStateIdentity(value: Value) StateIdentity {
+ var id: StateIdentity = .{};
+ switch (value) {
+ .array => |items| for (items.items) |item| {
+ const child = compactionStateIdentity(item);
+ id.bytes +|= child.bytes;
+ id.digest +%= child.digest;
+ },
+ .object => |obj| {
+ if (compactionStateLen(obj)) |len| {
+ id.bytes +|= len;
+ id.digest +%= std.hash.Wyhash.hash(0, stringValue(obj.get("encrypted_content")).?);
+ }
+ var it = obj.iterator();
+ while (it.next()) |entry| {
+ const child = compactionStateIdentity(entry.value_ptr.*);
+ id.bytes +|= child.bytes;
+ id.digest +%= child.digest;
+ }
+ },
+ else => {},
+ }
+ return id;
+}
+
/// Learn the state's share from a counted request: the reported input less
/// the estimate for everything else in it (`request_estimate` includes the
/// state at its current estimate). Never below one token per KiB of state.
diff --git a/src/opaque_state_persist.zig b/src/opaque_state_persist.zig
new file mode 100644
index 00000000..c334829f
--- /dev/null
+++ b/src/opaque_state_persist.zig
@@ -0,0 +1,78 @@
+//! #1686: a resumed session keeps the provider's count of its compaction state.
+//!
+//! The count a full request teaches (opaque_state_usage.zig) lives in
+//! process-global slots, so a new process restoring the same history fell back
+//! to bytes/16. An 8 MB state counted at ~80k then read as ~508k: the restored
+//! meter showed 188% and the next prompt paid for a forced compaction the warm
+//! process never needed. The count is saved with the session, scoped to the
+//! provider, the model, and the exact state (size plus a digest of every blob),
+//! and reapplied on load only when all of them still match. A changed state,
+//! a different model, or a missing or malformed record leaves bytes/16 in place.
+
+const std = @import("std");
+const Value = std.json.Value;
+const context_tokens = @import("context_tokens.zig");
+const Provider = @import("provider.zig").Provider;
+
+pub const field = "opaque_state_count";
+
+/// Write the record when this process has counted the history's state.
+pub fn write(s: *std.json.Stringify, provider: Provider, history: Value) !void {
+ const id = context_tokens.compactionStateIdentity(history);
+ const tokens = context_tokens.calibratedState(id.bytes) orelse return;
+ var digest_buf: [16]u8 = undefined;
+ try s.objectField(field);
+ try s.beginObject();
+ try s.objectField("provider");
+ try s.write(provider.id);
+ try s.objectField("model");
+ try s.write(provider.model);
+ try s.objectField("state_bytes");
+ try s.write(id.bytes);
+ // Hex: a u64 digest does not round-trip through JSON's signed integers.
+ try s.objectField("state_digest");
+ try s.write(std.fmt.bufPrint(&digest_buf, "{x:0>16}", .{id.digest}) catch unreachable);
+ try s.objectField("tokens");
+ try s.write(tokens);
+ try s.endObject();
+}
+
+/// Reapply a saved count to the restored history. Call after the history is
+/// in its final restored form and the session's provider is active.
+pub fn restore(provider: Provider, history: Value, session: std.json.ObjectMap) void {
+ const saved = recordFrom(session.get(field) orelse return) orelse return;
+ if (!std.mem.eql(u8, saved.provider, provider.id) or !std.mem.eql(u8, saved.model, provider.model)) return;
+ const id = context_tokens.compactionStateIdentity(history);
+ if (id.bytes != saved.state_bytes or id.digest != saved.state_digest) return;
+ context_tokens.calibrateState(saved.state_bytes, saved.tokens);
+}
+
+const Record = struct {
+ provider: []const u8,
+ model: []const u8,
+ state_bytes: usize,
+ state_digest: u64,
+ tokens: u64,
+};
+
+fn recordFrom(value: Value) ?Record {
+ if (value != .object) return null;
+ const obj = value.object;
+ return .{
+ .provider = str(obj.get("provider")) orelse return null,
+ .model = str(obj.get("model")) orelse return null,
+ .state_bytes = @intCast(positive(obj.get("state_bytes")) orelse return null),
+ .state_digest = std.fmt.parseInt(u64, str(obj.get("state_digest")) orelse return null, 16) catch return null,
+ .tokens = positive(obj.get("tokens")) orelse return null,
+ };
+}
+
+fn str(value: ?Value) ?[]const u8 {
+ const v = value orelse return null;
+ return if (v == .string) v.string else null;
+}
+
+fn positive(value: ?Value) ?u64 {
+ const v = value orelse return null;
+ return if (v == .integer and v.integer > 0) @intCast(v.integer) else null;
+}
diff --git a/src/opaque_state_persist_tests.zig b/src/opaque_state_persist_tests.zig
new file mode 100644
index 00000000..3f450a51
--- /dev/null
+++ b/src/opaque_state_persist_tests.zig
@@ -0,0 +1,199 @@
+//! #1686: a resumed session keeps the provider's count of its compaction state.
+
+const std = @import("std");
+const Value = std.json.Value;
+const Agent = @import("agent.zig").Agent;
+const context_tokens = @import("context_tokens.zig");
+const persist = @import("opaque_state_persist.zig");
+const session = @import("session.zig");
+
+const window: u64 = 270_000;
+const state_len: usize = 8_000_000;
+
+fn history(a: std.mem.Allocator, fill: u8) !std.json.Array {
+ const head = "{\"type\":\"compaction\",\"encrypted_content\":\"";
+ const json = try a.alloc(u8, head.len + state_len + 2);
+ @memcpy(json[0..head.len], head);
+ @memset(json[head.len .. head.len + state_len], fill);
+ @memcpy(json[head.len + state_len ..], "\"}");
+ var msgs = std.json.Array.init(a);
+ try msgs.append(try std.json.parseFromSliceLeaky(Value, a, json, .{}));
+ try msgs.append(try std.json.parseFromSliceLeaky(Value, a, "{\"role\":\"user\",\"content\":\"hi\"}", .{}));
+ return msgs;
+}
+
+fn agent(msgs: std.json.Array) Agent {
+ var root: Agent = undefined;
+ root.provider = .{ .id = "codex", .kind = .responses, .auth = .bearer, .url = "", .api_key = "", .model = "gpt-5", .context = window };
+ root.messages = msgs;
+ root.sub = false;
+ root.strict = false;
+ root.sys_normal = "";
+ root.sys_strict = "";
+ root.tools_responses = "";
+ root.last_cache_read = 0;
+ return root;
+}
+
+/// What a warm process saves after the provider counted the state at 80k:
+/// the meter pair and the persisted count, as one session object.
+const Saved = struct { context: u64, local: u64, obj: std.json.ObjectMap };
+
+fn countAndSave(a: std.mem.Allocator, root: *Agent) !Saved {
+ context_tokens.learnState(state_len, 80_000, root.fullRequestEstimateTokens());
+ const local = root.fullRequestEstimateTokens();
+ var aw: std.Io.Writer.Allocating = .init(a);
+ var s: std.json.Stringify = .{ .writer = &aw.writer };
+ try s.beginObject();
+ try persist.write(&s, root.provider, .{ .array = root.messages });
+ try s.endObject();
+ const parsed = try std.json.parseFromSliceLeaky(Value, a, aw.writer.buffered(), .{});
+ return .{ .context = 80_001, .local = local, .obj = parsed.object };
+}
+
+/// A new process: empty calibration, then loadSession's restore order.
+fn reload(root: *Agent, saved: Saved) void {
+ context_tokens.resetStateCountsForTest();
+ persist.restore(root.provider, .{ .array = root.messages }, saved.obj);
+ session.restoreContextMeter(root, saved.context, saved.local);
+}
+
+test "a reloaded session keeps the provider's count of its compaction state (#1686)" {
+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
+ defer arena_state.deinit();
+ const a = arena_state.allocator();
+ context_tokens.resetStateCountsForTest();
+ defer context_tokens.resetStateCountsForTest();
+
+ var root = agent(try history(a, 'A'));
+ const saved = try countAndSave(a, &root);
+ try std.testing.expect(saved.obj.get(persist.field) != null);
+
+ // Without the record a new process reads the state at bytes/16 (the bug).
+ var cold = saved;
+ cold.obj = (try std.json.parseFromSliceLeaky(Value, a, "{}", .{})).object;
+ reload(&root, cold);
+ try std.testing.expect(root.effectiveContextTokens() > window);
+ try std.testing.expect(root.inputOverCompactThreshold());
+
+ // With it, the reload keeps the counted occupancy and needs no compaction.
+ reload(&root, saved);
+ try std.testing.expect(root.effectiveContextTokens() < 100_000);
+ try std.testing.expect(!root.inputOverCompactThreshold());
+}
+
+test "a saved state count applies only to the same state, provider, and model (#1686)" {
+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
+ defer arena_state.deinit();
+ const a = arena_state.allocator();
+ context_tokens.resetStateCountsForTest();
+ defer context_tokens.resetStateCountsForTest();
+
+ var counted = agent(try history(a, 'A'));
+ const saved = try countAndSave(a, &counted);
+
+ // Same length, different blob: the digest refuses it.
+ var other = agent(try history(a, 'B'));
+ reload(&other, saved);
+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);
+ try std.testing.expect(other.inputOverCompactThreshold());
+
+ var moved = agent(counted.messages);
+ moved.provider.model = "gpt-6";
+ reload(&moved, saved);
+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);
+ moved.provider.model = "gpt-5";
+ moved.provider.id = "openai";
+ reload(&moved, saved);
+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);
+
+ const malformed = [_][]const u8{
+ "{\"opaque_state_count\":{\"provider\":\"codex\",\"model\":\"gpt-5\",\"state_bytes\":8000000,\"state_digest\":\"zz\",\"tokens\":72000}}",
+ "{\"opaque_state_count\":{\"provider\":\"codex\",\"model\":\"gpt-5\",\"state_bytes\":8000000,\"state_digest\":\"0\",\"tokens\":-1}}",
+ "{\"opaque_state_count\":\"codex\"}",
+ "{}",
+ };
+ for (malformed) |text| {
+ const parsed = try std.json.parseFromSliceLeaky(Value, a, text, .{});
+ reload(&counted, .{ .context = saved.context, .local = saved.local, .obj = parsed.object });
+ try std.testing.expect(context_tokens.calibratedState(state_len) == null);
+ }
+}
+
+test "an uncounted compaction state writes no record (#1686)" {
+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
+ defer arena_state.deinit();
+ const a = arena_state.allocator();
+ context_tokens.resetStateCountsForTest();
+ defer context_tokens.resetStateCountsForTest();
+
+ const root = agent(try history(a, 'A'));
+ var aw: std.Io.Writer.Allocating = .init(a);
+ var s: std.json.Stringify = .{ .writer = &aw.writer };
+ try s.beginObject();
+ try persist.write(&s, root.provider, .{ .array = root.messages });
+ try s.endObject();
+ try std.testing.expectEqualStrings("{}", aw.writer.buffered());
+}
+
+test "session save and load both carry the compaction-state count (#1686)" {
+ const src = @embedFile("session.zig");
+ try std.testing.expect(std.mem.indexOf(u8, src, "opaque_state_persist.zig\").write(&s, root.provider, Value{ .array = saved_msgs })") != null);
+ // Restore runs before the meter is rebased, or the rebase reads bytes/16.
+ const restore_at = std.mem.indexOf(u8, src, "opaque_state_persist.zig\").restore(root.provider") orelse return error.TestUnexpectedResult;
+ const rebase_at = std.mem.indexOf(u8, src, " restoreContextMeter(root, saved_context_tokens, saved_local_tokens);") orelse return error.TestUnexpectedResult;
+ try std.testing.expect(restore_at < rebase_at);
+}
+
+test "a session saved after a counted request loads without a forced compaction (#1686)" {
+ const writer = @import("session_writer.zig");
+ const transcript = @import("session_transcript.zig");
+ const gpa = std.testing.allocator;
+ const io = std.testing.io;
+ var arena_state = std.heap.ArenaAllocator.init(gpa);
+ defer arena_state.deinit();
+ const a = arena_state.allocator();
+ writer.resetForTest();
+ defer writer.resetForTest();
+ transcript.resetForTest();
+ defer transcript.resetForTest();
+ context_tokens.resetStateCountsForTest();
+ defer context_tokens.resetStateCountsForTest();
+ var tmp = std.testing.tmpDir(.{});
+ defer tmp.cleanup();
+ var client: std.http.Client = .{ .allocator = gpa, .io = io };
+ defer client.deinit();
+ var keys: @import("provider.zig").Keys = .{ .values = @splat("test-key") };
+ var provider = try keys.providerById("openai", "gpt-6-sol");
+ provider.context = window;
+ var root: Agent = .{
+ .gpa = gpa,
+ .arena = a,
+ .io = io,
+ .client = &client,
+ .provider = provider,
+ .messages = try history(a, 'A'),
+ .sub = false,
+ .label = "root",
+ .out = null,
+ .home = try std.fmt.allocPrint(a, ".zig-cache/tmp/{s}", .{tmp.sub_path}),
+ };
+ try root.ensureRootTools(root.provider.kind);
+
+ // Warm process: the provider counted a full request carrying the state at 80k.
+ context_tokens.learnState(state_len, 80_000, root.fullRequestEstimateTokens());
+ root.last_context_tokens = 80_001;
+ root.context_local_tokens = root.fullRequestEstimateTokens();
+ try std.testing.expect(!root.inputOverCompactThreshold());
+ try session.saveSessionTo(&root, a, tmp.dir, "reload-1686");
+ session.flushSaves();
+
+ // New process: no calibration in memory, then the real load path.
+ context_tokens.resetStateCountsForTest();
+ root.last_context_tokens = 0;
+ root.context_local_tokens = 0;
+ try session.loadSession(&root, &keys, a, "reload-1686");
+ root.provider.context = window;
+ try std.testing.expect(root.effectiveContextTokens() < 100_000);
+ try std.testing.expect(!root.inputOverCompactThreshold());
+}
diff --git a/src/session.zig b/src/session.zig
index 699cb1a2..14d4479d 100644
--- a/src/session.zig
+++ b/src/session.zig
@@ -355,6 +355,9 @@ fn queueSave(root: *Agent, arena: Allocator, dir: Io.Dir, name: []const u8) !u64
// this component with today's prompt/tool-schema estimate.
try s.objectField("context_local_tokens");
try s.write(@min(context_estimate.local, @as(u64, std.math.maxInt(i64))));
+ // #1686: the local component above uses this process's count of the
+ // compaction state; a resume needs that count, not bytes/16.
+ try @import("opaque_state_persist.zig").write(&s, root.provider, Value{ .array = saved_msgs });
try root.compaction_window.write(&s);
// #330: the --json protocol sequence high-water mark. A REPLACEMENT graff
// resuming this session continues the numbering from here instead of
@@ -542,6 +545,9 @@ pub fn loadSession(root: *Agent, keys: *Keys, arena: Allocator, name: []const u8
root.goal_note_age = 0;
root.session_parent = parent;
root.session_title = title;
+ // #1686: reapply the saved count of this exact compaction state first, so
+ // the rebased meter does not fall back to bytes/16 in a new process.
+ @import("opaque_state_persist.zig").restore(root.provider, .{ .array = root.messages }, obj);
// Rebase the saved server-only delta onto today's prompt/tool-schema input.
restoreContextMeter(root, saved_context_tokens, saved_local_tokens);
root.cap_new = false; // per-provider (like sox/effort below); relearn on rejection
diff --git a/src/test_hooks.zig b/src/test_hooks.zig
index 5c48814e..1611424b 100644
--- a/src/test_hooks.zig
+++ b/src/test_hooks.zig
@@ -340,6 +340,7 @@ test {
_ = @import("mcp_connect_notice.zig"); // ADR 0230
_ = @import("image_sniff.zig"); // #1762: image media type from the bytes
_ = @import("catalog_bounded.zig"); // model discovery: catalog GETs never hold boot past a budget
+ _ = @import("opaque_state_persist_tests.zig"); // #1686: a resumed session keeps its compaction-state count
_ = @import("oauth_callback.zig");
_ = @import("cache_key_tests.zig"); // ADR 0223: moved off http_headers.zig (600-line cap)
_ = @import("router_headers_tests.zig"); // moved off http_headers.zig (600-line cap)