Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

Claude: the system breakpoint closes before the per-repo context (ADR 0282)

justrachdc96e2122b81
diff --git a/docs/adr/0282-claude-system-breakpoint-closes-before-the-repo-context.md b/docs/adr/0282-claude-system-breakpoint-closes-before-the-repo-context.md
new file mode 100644
index 00000000..d211f48a
--- /dev/null
+++ b/docs/adr/0282-claude-system-breakpoint-closes-before-the-repo-context.md
@@ -0,0 +1,36 @@
+# 0282. The Claude system breakpoint closes before the per-repo context
+
+Status: accepted 2026-10-13
+
+## Context
+
+On the Anthropic API graff sent the system prompt as one text block with the
+cache breakpoint on it (ADR 0220), so the cached prefix was tools + the whole
+system prompt. The project layout sits near the end of that prompt (and the
+project-instructions block before it), so two repos shared none of it: the
+first request in any new repo wrote the full ~10k-token prefix at 1.25x.
+
+Measured on Claude Sonnet 5.5 against Claude Code (graff-evals, 13 Oct):
+request dumps of two tasks differed only in the layout lines, and each new
+task's first call wrote ~10k tokens. That was most of graff's
+`refactor-rename` cost ($0.031 vs Claude Code's $0.024), whose static prefix
+is shared across every session.
+
+## Decision
+
+On Claude, the system prompt goes as two text blocks with the same bytes. The
+first ends where the per-repo blocks begin (the blocks ADR 0243 moves on
+Responses: the project-instructions section and the layout) and carries the
+breakpoint. The second holds the rest (layout, skills, constraint ledger) with
+no breakpoint; the last-message breakpoint still covers it. With no per-repo
+block, the prompt stays one block. Other anthropic-format providers (Kimi,
+MiniMax) are unchanged.
+
+## Consequences
+
+A new repo's first request reads tools + instructions from the cache and
+writes only its own tail: 1.2k written tokens instead of 10.1k on a task the
+cache had not seen ($0.017 vs $0.033 for the session). The breakpoint count is
+unchanged. Instructions that vary per session before the layout (a goal, a
+mode) still split the cache there; moving those after the breakpoint is the
+next step if they show up in request dumps.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 7e41ff9d..2f96607c 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -292,6 +292,7 @@ record only when you need the evidence or the edge cases.
| [0278](0278-t3-code-acp-profile.md) | An ACP client named `t3-code…` gets a profile: slash commands are the first prompt block, the client's `cwd` is never swapped for an auto-isolated tree, a `model` config option lists reachable models, and `/model` and `/yolo` are not advertised. |
| [0280](0280-mcp-servers-report-state-and-reconnect.md) | Every configured MCP server has one status row (`graff mcp status --json`, `/mcp`); `graff mcp reconnect` / `/mcp reconnect` run a real handshake and refresh its catalog; `graff mcp enable|disable` live in `~/.codegraff/mcp-state.json`; the catalog cache keys on cwd + env, and server `env` overlays graff's own. |
| [0281](0281-anthropic-console-sign-in-reads-the-sdk-profile.md) | Anthropic Console sign-in: graff reads the `user_oauth` profile Anthropic's CLI writes (`ant auth login`), in the SDK's precedence (API key wins), sends it as Bearer, and refreshes through `ant auth print-credentials`; `graff login anthropic` runs `ant auth login`. No claude.ai subscription login. |
+| [0282](0282-claude-system-breakpoint-closes-before-the-repo-context.md) | On Claude, the system prompt is two text blocks with the same bytes: the cache breakpoint closes the static instructions, before the project instructions and layout, so every repo shares the tools + instructions prefix. |
## When to write one
diff --git a/src/agent_request_body.zig b/src/agent_request_body.zig
index 1dbbcee8..7f0735f1 100644
--- a/src/agent_request_body.zig
+++ b/src/agent_request_body.zig
@@ -84,17 +84,16 @@ pub fn buildBody(self: *Agent, tools_in: ?[]const u8, force_tool: bool, stream:
const sys = try @import("agent_request_body_responses.zig").schemaAwarePrompt(self);
const cc = @import("cache_ttl.zig").control(self); // #1320: 1h when calls outlive 5m
if (is_claude or is_kimi) {
+ // ADR 0282: on Claude the breakpoint closes the static block and
+ // the per-repo tail (layout, project instructions, skills,
+ // constraints) follows as a second block, so a new repo reads
+ // tools + instructions from the cache. The bytes are unchanged;
+ // the last-message breakpoint still covers the tail.
+ const at = if (is_claude) @import("repo_context.zig").staticEnd(self, sys) else sys.len;
+ const head = if (at == 0) sys else sys[0..at];
try s.beginArray();
- try s.beginObject();
- try s.objectField("type");
- try s.write("text");
- try s.objectField("text");
- try s.write(sys);
- if (cc) |c| {
- try s.objectField("cache_control");
- try s.print("{s}", .{c});
- }
- try s.endObject();
+ try writeSystemBlock(&s, head, cc);
+ if (head.len < sys.len) try writeSystemBlock(&s, sys[head.len..], null);
try s.endArray();
} else {
try s.write(sys);
@@ -507,6 +506,20 @@ test "openai-wire bodies send a sticky prompt_cache_key; children isolate" {
try std.testing.expect(std.mem.indexOf(u8, ob, needle) != null);
}
+/// One Anthropic system text block, with a breakpoint when `cc` is set.
+fn writeSystemBlock(s: *std.json.Stringify, text: []const u8, cc: ?[]const u8) !void {
+ try s.beginObject();
+ try s.objectField("type");
+ try s.write("text");
+ try s.objectField("text");
+ try s.write(text);
+ if (cc) |c| {
+ try s.objectField("cache_control");
+ try s.print("{s}", .{c});
+ }
+ try s.endObject();
+}
+
test { // Claude request-shape tests (ADR 0219) live beside this file
_ = @import("agent_request_body_claude_tests.zig");
}
diff --git a/src/agent_request_body_claude_tests.zig b/src/agent_request_body_claude_tests.zig
index d6ca4fbf..bcbb1ef3 100644
--- a/src/agent_request_body_claude_tests.zig
+++ b/src/agent_request_body_claude_tests.zig
@@ -245,3 +245,50 @@ test "Claude's own error text takes the retry-without ladder (ADR 0219)" {
// Anything else is a real failure.
try std.testing.expect(!retryWithout(&agent, "messages: at least one message is required", &force, &stream_usage));
}
+
+test "ADR 0282: the Claude system breakpoint closes before the per-repo blocks" {
+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
+ defer arena_state.deinit();
+ const a = arena_state.allocator();
+ const layouts = [_][]const u8{
+ "\n\n# Project layout (working tree at session start, depth 3; may omit later changes)\nlogs/\nlogs/auth.log\n",
+ "\n\n# Project layout (working tree at session start, depth 3; may omit later changes)\napi/\napi/routes.py\n",
+ };
+ var heads: [2][]const u8 = undefined;
+ for (layouts, 0..) |layout, i| {
+ var agent = try claudeAgent(a, "claude-sonnet-5-5");
+ const sys = try std.mem.concat(a, u8, &.{ "BASE", layout, "\n\n# Skills\nstatic tail" });
+ agent.sys_base = sys;
+ agent.sys_normal = sys;
+ agent.repo_map_snapshot = layout;
+ const body = try agent.buildBody(bash_tools, false, true, true);
+ defer std.testing.allocator.free(body);
+ const parsed = try std.json.parseFromSliceLeaky(Value, a, try a.dupe(u8, body), .{});
+ const blocks = parsed.object.get("system").?.array.items;
+ try std.testing.expectEqual(@as(usize, 2), blocks.len);
+ // The breakpoint sits on the static block; the tail carries none.
+ try std.testing.expect(blocks[0].object.get("cache_control") != null);
+ try std.testing.expect(blocks[1].object.get("cache_control") == null);
+ const head = blocks[0].object.get("text").?.string;
+ const tail = blocks[1].object.get("text").?.string;
+ // Same bytes as one block: only the cache boundary moved.
+ try std.testing.expectEqualStrings(sys, try std.mem.concat(a, u8, &.{ head, tail }));
+ heads[i] = head;
+ }
+ // Two repos share the cached block byte for byte.
+ try std.testing.expectEqualStrings("BASE", heads[0]);
+ try std.testing.expectEqualStrings(heads[0], heads[1]);
+}
+
+test "ADR 0282: a Claude system prompt without per-repo blocks stays one block" {
+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
+ defer arena_state.deinit();
+ const a = arena_state.allocator();
+ var agent = try claudeAgent(a, "claude-sonnet-5-5");
+ const body = try agent.buildBody(bash_tools, false, true, true);
+ defer std.testing.allocator.free(body);
+ const parsed = try std.json.parseFromSliceLeaky(Value, a, try a.dupe(u8, body), .{});
+ const blocks = parsed.object.get("system").?.array.items;
+ try std.testing.expectEqual(@as(usize, 1), blocks.len);
+ try std.testing.expect(blocks[0].object.get("cache_control") != null);
+}
diff --git a/src/repo_context.zig b/src/repo_context.zig
index 4117802d..d9dc2a1a 100644
--- a/src/repo_context.zig
+++ b/src/repo_context.zig
@@ -61,6 +61,24 @@ pub fn split(self: *Agent, instructions: []const u8) !Split {
return .{ .instructions = rest, .context = ctx.items };
}
+/// ADR 0282: the offset where the per-repo blocks begin in `instructions`
+/// (the same blocks `split` moves), or `instructions.len` when neither is
+/// present. The Anthropic wire ends its cached system block here, so every
+/// repo shares the tools + instructions prefix and only the rest is written.
+pub fn staticEnd(self: *Agent, instructions: []const u8) usize {
+ var end = instructions.len;
+ const baked = @import("hot_context.zig").bakedSection();
+ const blocks = [_]?[]const u8{ if (baked.len != 0) baked else null, @import("session_prompt.zig").snapshot(self) };
+ for (blocks) |maybe| {
+ const block = maybe orelse continue;
+ if (block.len == 0) continue;
+ var at = std.mem.indexOf(u8, instructions, block) orelse continue;
+ if (block[0] != '\n' and at >= 2 and std.mem.eql(u8, instructions[at - 2 .. at], "\n\n")) at -= 2;
+ end = @min(end, at);
+ }
+ return end;
+}
+
/// The context as a developer message, the role ADR 0208 gives mid-session
/// context on Responses. Written with the conversation's first item only:
/// chained requests already hold it server-side.