Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

Gemini: validated tools, thought summaries, whole-prompt usage, step history

justrach2195a8e7c03d
diff --git a/docs/adr/0271-gemini-interactions-continue-the-stored-interaction.md b/docs/adr/0271-gemini-interactions-continue-the-stored-interaction.md
index f15e2816..e456db1c 100644
--- a/docs/adr/0271-gemini-interactions-continue-the-stored-interaction.md
+++ b/docs/adr/0271-gemini-interactions-continue-the-stored-interaction.md
@@ -5,13 +5,17 @@ Status: accepted 2026-10-08
## Context
The direct Google provider speaks the Interactions API. graff sent
-`store:false` and the whole step list on every request, so a long session paid
-for its full history as input on every call; implicit caching only discounted
-part of it. The endpoint can hold the conversation itself: a request that names
-a stored Interaction in `previous_interaction_id` sends only the steps added
-since, and the endpoint reports only those as `total_input_tokens` (the whole
-prompt is reported separately as `raw_prompt_token`). In a probe, a 38k-token
-history continued with a short question reported 134 input tokens.
+`store:false` and the whole step list on every request, re-uploading the
+conversation each call. The endpoint can hold the conversation itself: a
+request that names a stored Interaction in `previous_interaction_id` sends only
+the steps added since, and Google recommends that path for implicit caching of
+the history.
+
+On such a request the flat usage totals (`total_input_tokens`,
+`total_cached_tokens`) cover only the new steps, while
+`model_invocation_token_counts` and `raw_prompt_token` report the whole prompt
+the model read and the part served from cache. Google documents the continued
+history as input, so graff meters the whole prompt.
The earlier writer skipped this because the id had to stay consistent with
graff's own history through compaction, trims and edits.
@@ -33,16 +37,36 @@ server holds. `interactions_chain.zig` owns the rule:
with the full history.
- `system_instruction`, `tools` and `generation_config` are sent on every
request, as the endpoint does not carry them over.
-- The context meter reads `raw_prompt_token`, so compaction still sees the real
- window size; cost reads what was billed; the held prefix shows as a cache read.
+- Usage reads the per-invocation counts (`promptUsage`): the context meter and
+ the cost both use the whole prompt, with its cached part at the cache rate.
`GRAFF_INTERACTIONS_STORE=0` (or off/false/no) restores the stateless shape:
`store:false` and a full replay on every request.
## Consequences
-Long Gemini sessions stop re-sending their history, which cuts input billed per
-call to roughly the new steps. Interactions are now retained by Google under its
+Long Gemini sessions stop re-uploading their history, and the server-held prefix
+stays stable for implicit caching (a 107k-token stable prompt read 102,400
+tokens from cache). Interactions are now retained by Google under its
storage policy; users who must not have that set `GRAFF_INTERACTIONS_STORE=0`.
Hashing the history costs one serialization pass per request, which the full
replay already paid.
+
+The same change brings the wire in line with what Gemini 3 supports:
+
+- `tool_choice` is `validated` (auto with constrained decoding of calls), and
+ `any` when a tool is forced.
+- `thinking_level` is never `minimal`, which Gemini 3.1 Pro and 3.7/3.8 Flash
+ reject; the lightest effort sends `low`.
+- `thinking_summaries: "auto"` is sent explicitly (without it no summaries
+ stream, despite the documented default), and `thought_summary` deltas feed
+ the reasoning panel.
+- Requests with tools on `gemini-3.1-pro-preview` go to
+ `gemini-3.1-pro-preview-customtools`, Google's variant that prefers the
+ harness's own tools over shell, at the same price. The chain keys on the wire
+ model, so a request without tools on the other variant replays in full.
+- Chat-shaped user/assistant text turns that shared paths append (the
+ REPL/--json/ACP user turn, interrupt markers, the compaction request) are
+ rewritten as `user_input` / `model_output` steps before each send
+ (`history_wire.prepare`); the endpoint rejected the whole request on one, so
+ only `-p` worked on this provider.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 3aed0ab3..843abb92 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -289,7 +289,7 @@ record only when you need the evidence or the edge cases.
| [0260](0260-clef-compaction-replaces-client-summary.md) | `GRAFF_CLEF_COMPACT=1` routes client-summary compactions through the gateway's server-side `POST /v1/compact` (Clef decision-model pruning, text verbatim); OpenAI-family server paths are untouched and any failure falls back to the summary. |
| [0261](0261-client-summary-stays-the-compactor.md) | The client summary stays the compactor: `GRAFF_CLEF_COMPACT` defaults off (Clef rarely pruned and mostly fell back to the summary in measurement); first-party Responses routes use only the provider's own compaction. Supersedes 0260's default. |
| [0270](0270-a-closed-stdio-mcp-server-restarts-on-its-next-call.md) | A stdio MCP server whose connection closes goes back to dormant with its launch config and restarts on its next call; the failed call is never replayed, and after two restarts in a session its tools are withdrawn as before. |
-| [0271](0271-gemini-interactions-continue-the-stored-interaction.md) | Direct Gemini (Interactions) requests are stored and continue the last Interaction with `previous_interaction_id` plus only the new steps while a content hash proves graff's history extends it; any rewrite, model switch or rejected id falls back to a full replay. `GRAFF_INTERACTIONS_STORE=0` keeps it stateless. |
+| [0271](0271-gemini-interactions-continue-the-stored-interaction.md) | Direct Gemini (Interactions) requests are stored and continue the last Interaction with `previous_interaction_id` plus only the new steps while a content hash proves graff's history extends it; any rewrite, model switch or rejected id falls back to a full replay; usage meters the whole prompt from the per-invocation counts. Tool choice is `validated`, `minimal` thinking maps to `low`, thought summaries stream, and Gemini 3.1 Pro with tools uses its custom-tools variant. `GRAFF_INTERACTIONS_STORE=0` keeps it stateless. |
## When to write one
diff --git a/src/agent_overflow.zig b/src/agent_overflow.zig
index 6e771d49..ec55815b 100644
--- a/src/agent_overflow.zig
+++ b/src/agent_overflow.zig
@@ -221,8 +221,8 @@ pub fn readCompletion(kind: Provider.Kind, root: std.json.ObjectMap) Completion
.interactions => {
if (usageOf(root)) |u| {
// A continued Interaction reports only its new steps as input;
- // raw_prompt_token is the whole prompt the window holds.
- c.input_tokens = nonNegative(@max(usageInt(u, "total_input_tokens"), usageInt(u, "raw_prompt_token")));
+ // the window holds the whole prompt (interactions_steps.promptUsage).
+ c.input_tokens = nonNegative(@import("interactions_steps.zig").promptUsage(u).prompt);
c.output_tokens = nonNegative(usageInt(u, "total_output_tokens") +| usageInt(u, "total_thought_tokens"));
}
if (str(root, "status")) |s| c.stop = stopFromString(kind, s);
diff --git a/src/agent_request_body.zig b/src/agent_request_body.zig
index 1dbbcee8..9d198135 100644
--- a/src/agent_request_body.zig
+++ b/src/agent_request_body.zig
@@ -30,7 +30,7 @@ pub fn buildBody(self: *Agent, tools_in: ?[]const u8, force_tool: bool, stream:
var s: std.json.Stringify = .{ .writer = &aw.writer };
try s.beginObject();
try s.objectField("model");
- try s.write(self.provider.model);
+ try s.write(if (self.provider.kind == .interactions) @import("interactions_wire.zig").wireModel(self.provider.model, tools != null) else self.provider.model);
switch (self.provider.kind) {
.anthropic => {
const is_kimi = std.mem.eql(u8, self.provider.id, "kimi");
diff --git a/src/history_wire.zig b/src/history_wire.zig
index 29ab1983..825e8814 100644
--- a/src/history_wire.zig
+++ b/src/history_wire.zig
@@ -27,7 +27,7 @@ pub fn prepare(a: std.mem.Allocator, kind: Kind, history: *std.json.Array) void
repair(a, &block.object, "content", true);
}
},
- .interactions => {},
+ .interactions => @import("interactions_steps.zig").normalizeHistory(a, history),
}
}
diff --git a/src/interactions_chain.zig b/src/interactions_chain.zig
index 354d7c86..e8f6dbce 100644
--- a/src/interactions_chain.zig
+++ b/src/interactions_chain.zig
@@ -2,10 +2,11 @@
//!
//! A stored Interaction keeps every step it was sent and every step it
//! produced. A follow-up that names it in `previous_interaction_id` sends only
-//! the steps added since, and the endpoint reports — and bills — only those as
-//! input: a 38k-token history continued with a short question comes back as
-//! ~130 `total_input_tokens`, with the whole prompt in `raw_prompt_token`.
-//! Before this, graff sent `store:false` and replayed every step every request.
+//! the steps added since; the server reads the history it holds, which keeps
+//! the prompt prefix stable for implicit caching and stops graff re-uploading
+//! the conversation. (The flat usage totals then cover only the new steps; the
+//! meters read the whole prompt, see interactions_steps.promptUsage.) Before
+//! this, graff sent `store:false` and replayed every step every request.
//!
//! The chain is used only while graff's history is a clean extension of what
//! the server holds, and the guard is content rather than bookkeeping:
@@ -81,7 +82,7 @@ fn digest(h: Wyhash) u64 {
/// Decide what the next request sends, and remember what it carried so the
/// answer can extend the chain. Called once per body build.
-pub fn plan(self: *Agent) !Plan {
+pub fn plan(self: *Agent, wire_model: []const u8) !Plan {
const c = &self.ix_chain;
c.sent_state = null;
if (!g_store) {
@@ -89,7 +90,7 @@ pub fn plan(self: *Agent) !Plan {
return .{};
}
const items = self.messages.items;
- const model = modelFp(self.provider.model);
+ const model = modelFp(wire_model);
var h = Wyhash.init(0);
var out: Plan = .{};
var hashed: usize = 0;
@@ -165,7 +166,7 @@ fn answer(arena: std.mem.Allocator, id: []const u8) !std.json.ObjectMap {
/// One model turn: plan the request, then record and append the answer.
fn turn(agent: *Agent, arena: std.mem.Allocator, id: []const u8, step_json: []const u8) !Plan {
- var p = try plan(agent);
+ var p = try plan(agent, agent.provider.model);
// prev_id points into the chain, which record() is about to overwrite.
if (p.prev_id) |held| p.prev_id = try arena.dupe(u8, held);
const st = try parse(arena, step_json);
@@ -193,7 +194,7 @@ test "a turn that extends history chains and sends only the new steps" {
// A tool result rides on the newest Interaction.
try agent.messages.append(try steps.functionResult(a, "c1", "bash", "out"));
- const p3 = try plan(&agent);
+ const p3 = try plan(&agent, agent.provider.model);
try std.testing.expectEqualStrings("v1_two", p3.prev_id.?);
try std.testing.expectEqual(@as(usize, 4), p3.from);
}
@@ -211,7 +212,7 @@ test "any change to what the server holds falls back to a full replay" {
_ = try turn(&edited, a, "v1_one", reply);
edited.messages.items[0] = try steps.userInput(a, "summary of earlier work");
try edited.messages.append(try steps.userInput(a, "next"));
- try std.testing.expect((try plan(&edited)).prev_id == null);
+ try std.testing.expect((try plan(&edited, edited.provider.model)).prev_id == null);
// /clear then a history that grows back past the watermark.
var cleared = testAgent(a);
@@ -219,7 +220,7 @@ test "any change to what the server holds falls back to a full replay" {
_ = try turn(&cleared, a, "v1_one", reply);
cleared.messages.clearRetainingCapacity();
for ([_][]const u8{ "a", "b", "c" }) |t| try cleared.messages.append(try steps.userInput(a, t));
- try std.testing.expect((try plan(&cleared)).prev_id == null);
+ try std.testing.expect((try plan(&cleared, cleared.provider.model)).prev_id == null);
// A model switch.
var switched = testAgent(a);
@@ -227,13 +228,13 @@ test "any change to what the server holds falls back to a full replay" {
_ = try turn(&switched, a, "v1_one", reply);
switched.provider.model = "gemini-3.8-pro";
try switched.messages.append(try steps.userInput(a, "next"));
- try std.testing.expect((try plan(&switched)).prev_id == null);
+ try std.testing.expect((try plan(&switched, switched.provider.model)).prev_id == null);
// Nothing new to send: an empty delta is never chained.
var idle = testAgent(a);
try idle.messages.append(try steps.userInput(a, "first"));
_ = try turn(&idle, a, "v1_one", reply);
- try std.testing.expect((try plan(&idle)).prev_id == null);
+ try std.testing.expect((try plan(&idle, idle.provider.model)).prev_id == null);
}
test "a message appended while the request was in flight leaves no chain" {
@@ -243,7 +244,7 @@ test "a message appended while the request was in flight leaves no chain" {
const steps = @import("interactions_steps.zig");
var agent = testAgent(a);
try agent.messages.append(try steps.userInput(a, "first"));
- _ = try plan(&agent);
+ _ = try plan(&agent, agent.provider.model);
try agent.messages.append(try steps.userInput(a, "steer")); // the server never saw this
try record(&agent, try answer(a, "v1_one"), &.{});
try std.testing.expect(agent.ix_chain.id() == null);
@@ -261,7 +262,7 @@ test "the opt-out keeps every request stateless" {
try agent.messages.append(try steps.userInput(a, "first"));
_ = try turn(&agent, a, "v1_one", "{\"type\":\"model_output\",\"content\":[]}");
try agent.messages.append(try steps.userInput(a, "next"));
- try std.testing.expect((try plan(&agent)).prev_id == null);
+ try std.testing.expect((try plan(&agent, agent.provider.model)).prev_id == null);
try std.testing.expect(agent.ix_chain.id() == null);
}
diff --git a/src/interactions_steps.zig b/src/interactions_steps.zig
index 4ff38890..4a8e4377 100644
--- a/src/interactions_steps.zig
+++ b/src/interactions_steps.zig
@@ -67,21 +67,51 @@ pub fn assistantText(arena: Allocator, root: std.json.ObjectMap) ![]const u8 {
/// other wires report a cache read.
///
/// A request that continued a stored Interaction reports only its new steps as
-/// `total_input_tokens`; the whole prompt is `raw_prompt_token`. The context
-/// meter reads the whole prompt (compaction depends on it), the cost only what
-/// was billed, and the held prefix shows as a cache read: it was not resent.
+/// `total_input_tokens`. The whole prompt the model read is in
+/// `model_invocation_token_counts` (and `raw_prompt_token`), with the part read
+/// from cache beside it; Google documents the continued history as input, so
+/// both the context meter and the cost read the whole prompt (see promptUsage).
pub fn recordUsage(self: *@import("agent.zig").Agent, u: std.json.ObjectMap, fallback: u64) void {
const ctx = @import("agent_context.zig");
- const in_tokens = ctx.usageInt(u, "total_input_tokens");
- const prompt = @max(in_tokens, ctx.usageInt(u, "raw_prompt_token"));
+ const p = promptUsage(u);
const out_tokens = ctx.usageInt(u, "total_output_tokens") +| ctx.usageInt(u, "total_thought_tokens");
- const total = @max(ctx.usageInt(u, "total_tokens"), prompt +| out_tokens);
+ const total = @max(ctx.usageInt(u, "total_tokens"), p.prompt +| out_tokens);
if (total > 0) ctx.replaceContextTokens(self, @intCast(total)) else ctx.floorContextTokens(self, fallback);
self.last_usage_includes_output = total > 0;
- const cached = ctx.usageInt(u, "total_cached_tokens");
- const held = prompt - in_tokens;
- if (cached +| held > 0) self.last_cache_read = @intCast(cached +| held);
- self.recordCost(@max(in_tokens - cached, 0), cached, 0, out_tokens);
+ if (p.cached > 0) self.last_cache_read = @intCast(p.cached);
+ self.recordCost(@max(p.prompt - p.cached, 0), p.cached, 0, out_tokens);
+}
+
+pub const PromptUsage = struct { prompt: i64, cached: i64 };
+
+/// The prompt the model read and how much of it came from cache. Per-invocation
+/// counts are the full prompt even on a continued Interaction, where the flat
+/// `total_input_tokens` / `total_cached_tokens` cover only the new steps.
+pub fn promptUsage(u: std.json.ObjectMap) PromptUsage {
+ const ctx = @import("agent_context.zig");
+ var inv_prompt: i64 = 0;
+ var inv_cached: i64 = 0;
+ if (u.get("model_invocation_token_counts")) |mi| if (mi == .array) for (mi.array.items) |inv| {
+ if (inv != .object) continue;
+ inv_prompt +|= modalitySum(inv.object.get("prompt_tokens_details"));
+ inv_cached +|= modalitySum(inv.object.get("cache_tokens_details"));
+ };
+ const prompt = @max(@max(ctx.usageInt(u, "total_input_tokens"), ctx.usageInt(u, "raw_prompt_token")), inv_prompt);
+ const cached = @min(@max(ctx.usageInt(u, "total_cached_tokens"), inv_cached), prompt);
+ return .{ .prompt = prompt, .cached = cached };
+}
+
+fn modalitySum(v: ?Value) i64 {
+ const list = v orelse return 0;
+ if (list != .array) return 0;
+ var n: i64 = 0;
+ for (list.array.items) |d| {
+ if (d != .object) continue;
+ if (d.object.get("tokens")) |t| if (t == .integer and t.integer > 0) {
+ n +|= t.integer;
+ };
+ }
+ return n;
}
/// A function_result rebuilt out of band (vision/repl replay), where only the
@@ -107,6 +137,86 @@ pub fn imagePart(arena: Allocator, ib: *std.json.ObjectMap, img: anytype) !void
}
}
+/// `{"type":"model_output","content":[{"type":"text","text":"…"}]}` — an
+/// assistant text turn graff writes itself (an interrupt marker, a review
+/// reply), in the shape the endpoint echoes back.
+pub fn modelOutput(arena: Allocator, text: []const u8) !Value {
+ var part: std.json.ObjectMap = .empty;
+ try part.put(arena, "type", .{ .string = "text" });
+ try part.put(arena, "text", .{ .string = try arena.dupe(u8, text) });
+ var content = std.json.Array.init(arena);
+ try content.append(.{ .object = part });
+ var obj: std.json.ObjectMap = .empty;
+ try obj.put(arena, "type", .{ .string = "model_output" });
+ try obj.put(arena, "content", .{ .array = content });
+ return .{ .object = obj };
+}
+
+/// Rewrite chat-shaped text turns (`{"role":"user"|"assistant","content":…}`)
+/// as steps. Several shared paths append those (the REPL/--json/ACP user turn,
+/// interrupt markers, the compaction request), and the endpoint rejects the
+/// whole request on one ("use step_list input format instead of turn_list").
+/// Runs before every send (history_wire.prepare); a converted turn stays
+/// converted, so the history a stored Interaction was built from is stable.
+/// Empty turns are dropped: an empty text part is rejected too.
+pub fn normalizeHistory(arena: Allocator, history: *std.json.Array) void {
+ var changed = false;
+ for (history.items) |item| if (chatTurn(item) != null) {
+ changed = true;
+ break;
+ };
+ if (!changed) return;
+ var out = std.json.Array.init(arena);
+ out.ensureTotalCapacity(history.items.len) catch return;
+ for (history.items) |item| {
+ const role = chatTurn(item) orelse {
+ out.appendAssumeCapacity(item);
+ continue;
+ };
+ const text = turnText(arena, item.object.get("content")) catch return;
+ if (std.mem.trim(u8, text, " \t\r\n").len == 0) continue;
+ var step_v = (if (std.mem.eql(u8, role, "assistant")) modelOutput(arena, text) else userInput(arena, text)) catch return;
+ const origin = @import("session_wake.zig").origin_key;
+ if (item.object.get(origin)) |o| step_v.object.put(arena, origin, o) catch return;
+ out.appendAssumeCapacity(step_v);
+ }
+ history.* = out;
+}
+
+/// The role of a chat-shaped user/assistant text turn, or null for anything
+/// else (steps, tool messages, assistant turns carrying tool calls).
+fn chatTurn(item: Value) ?[]const u8 {
+ if (item != .object) return null;
+ if (item.object.get("type") != null) return null;
+ if (item.object.get("tool_calls") != null) return null;
+ const role = item.object.get("role") orelse return null;
+ if (role != .string) return null;
+ if (!std.mem.eql(u8, role.string, "user") and !std.mem.eql(u8, role.string, "assistant")) return null;
+ const content = item.object.get("content") orelse return null;
+ if (content == .string) return role.string;
+ if (content != .array) return null;
+ for (content.array.items) |part| {
+ if (part != .object) return null;
+ const t = part.object.get("type") orelse return null;
+ if (t != .string) return null;
+ if (!std.mem.eql(u8, t.string, "text") and !std.mem.eql(u8, t.string, "input_text") and !std.mem.eql(u8, t.string, "output_text")) return null;
+ }
+ return role.string;
+}
+
+fn turnText(arena: Allocator, content: ?Value) ![]const u8 {
+ const c = content orelse return "";
+ if (c == .string) return c.string;
+ var out: std.ArrayList(u8) = .empty;
+ for (c.array.items) |part| {
+ const t = part.object.get("text") orelse continue;
+ if (t != .string) continue;
+ if (out.items.len > 0) try out.append(arena, '\n');
+ try out.appendSlice(arena, t.string);
+ }
+ return out.items;
+}
+
/// One turn on the Interactions wire: echo the model's steps into history,
/// dispatch any function_call steps, and answer with the visible text once the
/// model stops calling tools. Mirrors stepOpenAI / stepResponses.
@@ -193,3 +303,50 @@ test "assistantText concatenates model_output text and ignores thoughts" {
defer parsed.deinit();
try std.testing.expectEqualStrings("one two", try assistantText(arena, parsed.value.object));
}
+
+test "a continued Interaction is metered on the whole prompt it read" {
+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
+ defer arena_state.deinit();
+ const arena = arena_state.allocator();
+ // Shape returned for a follow-up on a stored Interaction: the flat totals
+ // cover only the new step, the invocation counts the whole prompt.
+ const chained = try std.json.parseFromSliceLeaky(Value, arena,
+ \\{"total_input_tokens":134,"total_cached_tokens":114,"raw_prompt_token":38383,
+ \\ "model_invocation_token_counts":[{"prompt_tokens_details":[{"modality":"text","tokens":38383}],
+ \\ "cache_tokens_details":[{"modality":"text","tokens":32768}]}]}
+ , .{});
+ const p = promptUsage(chained.object);
+ try std.testing.expectEqual(@as(i64, 38383), p.prompt);
+ try std.testing.expectEqual(@as(i64, 32768), p.cached);
+ // A stateless request without invocation detail falls back to the totals.
+ const flat = try std.json.parseFromSliceLeaky(Value, arena, "{\"total_input_tokens\":500,\"total_cached_tokens\":200}", .{});
+ const q = promptUsage(flat.object);
+ try std.testing.expectEqual(@as(i64, 500), q.prompt);
+ try std.testing.expectEqual(@as(i64, 200), q.cached);
+}
+
+test "chat-shaped text turns become steps before they reach the wire" {
+ var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
+ defer arena_state.deinit();
+ const a = arena_state.allocator();
+ const messages = @import("messages.zig");
+ var history = std.json.Array.init(a);
+ try history.append(try messages.textMessage(a, "user", "Is 391 prime?"));
+ try history.append(try std.json.parseFromSliceLeaky(Value, a, "{\"type\":\"thought\",\"signature\":\"S\"}", .{}));
+ try history.append(try messages.textMessage(a, "assistant", "No: 17 x 23."));
+ try history.append(try messages.textMessage(a, "assistant", " "));
+ try history.append(try @import("session_wake.zig").message(a, "[a note]"));
+ normalizeHistory(a, &history);
+ try std.testing.expectEqual(@as(usize, 4), history.items.len); // the blank turn is dropped
+ try std.testing.expectEqualStrings("user_input", history.items[0].object.get("type").?.string);
+ try std.testing.expectEqualStrings("Is 391 prime?", history.items[0].object.get("content").?.string);
+ try std.testing.expectEqualStrings("thought", history.items[1].object.get("type").?.string);
+ try std.testing.expectEqualStrings("model_output", history.items[2].object.get("type").?.string);
+ try std.testing.expectEqualStrings("No: 17 x 23.", history.items[2].object.get("content").?.array.items[0].object.get("text").?.string);
+ // A wake note keeps its origin tag (it is stripped on the wire, not here).
+ try std.testing.expect(history.items[3].object.get(@import("session_wake.zig").origin_key) != null);
+ // Idempotent: a second pass leaves the converted history alone.
+ const before = history.items.ptr;
+ normalizeHistory(a, &history);
+ try std.testing.expectEqual(before, history.items.ptr);
+}
diff --git a/src/interactions_wire.zig b/src/interactions_wire.zig
index 2139c465..8bf5d96f 100644
--- a/src/interactions_wire.zig
+++ b/src/interactions_wire.zig
@@ -25,20 +25,28 @@ const std = @import("std");
const Agent = @import("agent.zig").Agent;
const serde = @import("serde.zig");
-/// graff's effort tag → Interactions `thinking_level`. The endpoint names its
-/// own accepted set in the rejection: high, low, medium, minimal, none.
+/// graff's effort tag → Interactions `thinking_level`. Only low, medium and
+/// high are accepted by every current model: Gemini 3.1 Pro and 3.7/3.8 Flash
+/// reject `minimal` with a 400, so the lightest effort maps to `low`.
pub fn thinkingLevel(effort: []const u8) []const u8 {
if (std.mem.eql(u8, effort, "max") or std.mem.eql(u8, effort, "high")) return "high";
if (std.mem.eql(u8, effort, "medium")) return "medium";
- if (std.mem.eql(u8, effort, "minimal") or std.mem.eql(u8, effort, "none")) return "minimal";
return "low";
}
+/// The model id sent on the wire. Gemini 3.1 Pro has a variant tuned for
+/// harnesses that bring their own tools (it prefers them over shell), at the
+/// same price; requests that carry tools use it.
+pub fn wireModel(model: []const u8, has_tools: bool) []const u8 {
+ if (has_tools and std.mem.eql(u8, model, "gemini-3.1-pro-preview")) return "gemini-3.1-pro-preview-customtools";
+ return model;
+}
+
pub fn write(self: *Agent, s: *std.json.Stringify, tools: ?[]const u8, force_tool: bool, stream: bool) !void {
try s.objectField("system_instruction");
try s.write(try @import("agent_request_body_responses.zig").schemaAwarePrompt(self));
const chain = @import("interactions_chain.zig");
- const plan = try chain.plan(self);
+ const plan = try chain.plan(self, wireModel(self.provider.model, tools != null));
// Stored by default (the field's default is true) so the next request can
// continue this one; GRAFF_INTERACTIONS_STORE=0 keeps nothing server-side.
if (!chain.g_store) {
@@ -53,11 +61,17 @@ pub fn write(self: *Agent, s: *std.json.Stringify, tools: ?[]const u8, force_too
try s.beginObject();
try s.objectField("thinking_level");
try s.write(thinkingLevel(@tagName(self.reasoning)));
- // tool_choice lives HERE, not at the top level, and its enum is lowercase:
- // auto | any | none. "required"/"ANY" are both rejected outright.
+ // Documented as the default, but summaries only stream when asked for;
+ // they feed the reasoning panel (title.reasoningDelta).
+ try s.objectField("thinking_summaries");
+ try s.write("auto");
+ // tool_choice lives HERE, not at the top level, and is lowercase:
+ // auto | any | none | validated. "required"/"ANY" are rejected outright.
+ // `validated` is auto with constrained decoding of the call, so arguments
+ // always match the declared schema.
if (tools != null) {
try s.objectField("tool_choice");
- try s.write(if (force_tool) "any" else "auto");
+ try s.write(if (force_tool) "any" else "validated");
}
try s.endObject();
if (tools) |t| {
@@ -75,13 +89,19 @@ pub fn write(self: *Agent, s: *std.json.Stringify, tools: ?[]const u8, force_too
}
test "thinking_level maps graff's efforts onto the accepted set" {
- // The set is the endpoint's own: "Valid values are: high, low, medium,
- // minimal, none" — anything else 400s the request.
+ // Only levels every current model accepts are ever sent.
try std.testing.expectEqualStrings("high", thinkingLevel("max"));
try std.testing.expectEqualStrings("high", thinkingLevel("high"));
try std.testing.expectEqualStrings("medium", thinkingLevel("medium"));
try std.testing.expectEqualStrings("low", thinkingLevel("low"));
- try std.testing.expectEqualStrings("minimal", thinkingLevel("minimal"));
+ // `minimal` 400s on Gemini 3.1 Pro and 3.7/3.8 Flash.
+ try std.testing.expectEqualStrings("low", thinkingLevel("minimal"));
// An unknown tag degrades to a level the endpoint accepts, never to a 400.
try std.testing.expectEqualStrings("low", thinkingLevel("nonsense"));
}
+
+test "Gemini 3.1 Pro uses its custom-tools variant only when tools ride the request" {
+ try std.testing.expectEqualStrings("gemini-3.1-pro-preview-customtools", wireModel("gemini-3.1-pro-preview", true));
+ try std.testing.expectEqualStrings("gemini-3.1-pro-preview", wireModel("gemini-3.1-pro-preview", false));
+ try std.testing.expectEqualStrings("gemini-3.8-flash", wireModel("gemini-3.8-flash", true));
+}
diff --git a/src/title.zig b/src/title.zig
index 4d47c311..ac381b6a 100644
--- a/src/title.zig
+++ b/src/title.zig
@@ -87,9 +87,18 @@ pub fn printSessionHeader(w: *Io.Writer, title: []const u8, folder: []const u8)
/// codex/responses a `response.reasoning_summary_text.delta`.
pub fn reasoningDelta(kind: Provider.Kind, obj: std.json.ObjectMap) []const u8 {
return switch (kind) {
- // Gemini never streams reasoning PROSE — a thought step carries only an
- // opaque signature, so there is nothing to show in the thinking panel.
- .interactions => "",
+ // A thought step streams its summary as `thought_summary` deltas
+ // (`thinking_summaries` defaults to auto); its signature is opaque.
+ .interactions => blk: {
+ const d = obj.get("delta") orelse break :blk "";
+ if (d != .object) break :blk "";
+ const dt = d.object.get("type") orelse break :blk "";
+ if (dt != .string or !std.mem.eql(u8, dt.string, "thought_summary")) break :blk "";
+ const c = d.object.get("content") orelse break :blk "";
+ if (c != .object) break :blk "";
+ const x = c.object.get("text") orelse break :blk "";
+ break :blk if (x == .string) x.string else "";
+ },
.anthropic => blk: {
const d = obj.get("delta") orelse break :blk "";
if (d != .object) break :blk "";
@@ -247,6 +256,11 @@ test "reasoningDelta extracts thinking/reasoning per provider wire format (#75)"
// codex/responses stream a reasoning summary delta
const re = mk(a, "{\"type\":\"response.reasoning_summary_text.delta\",\"delta\":\"plan\"}");
try std.testing.expectEqualStrings("plan", reasoningDelta(.responses, re));
+ // gemini (Interactions) streams a thought summary; the signature is opaque
+ const gs = mk(a, "{\"index\":0,\"delta\":{\"type\":\"thought_summary\",\"content\":{\"text\":\"**Assessing**\"}}}");
+ try std.testing.expectEqualStrings("**Assessing**", reasoningDelta(.interactions, gs));
+ const sig = mk(a, "{\"index\":0,\"delta\":{\"type\":\"thought_signature\",\"signature\":\"OPAQUE\"}}");
+ try std.testing.expectEqualStrings("", reasoningDelta(.interactions, sig));
// a plain answer-text delta carries no reasoning
const txt = mk(a, "{\"choices\":[{\"delta\":{\"content\":\"answer\"}}]}");
try std.testing.expectEqualStrings("", reasoningDelta(.openai, txt));