graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
Python
stall: commentary prose does not tighten the read budget
justrachd6fb867740df
diff --git a/docs/adr/0248-commentary-prose-does-not-tighten-the-stall-budget.md b/docs/adr/0248-commentary-prose-does-not-tighten-the-stall-budget.mdnew file mode 100644index 00000000..fda1ed77--- /dev/null+++ b/docs/adr/0248-commentary-prose-does-not-tighten-the-stall-budget.md@@ -0,0 +1,34 @@+# 0248. Commentary prose does not tighten the stall budget++Status: accepted 2026-10-03++## Context++ADR 0241 tightens the between-frames read budget while prose streams and gives+the full budget back when the prose item closes. On the Codex Responses+backend, GPT-6 models write a commentary heads-up (`phase: "commentary"`)+before a batch of tool calls. The server streams that heads-up, then holds+the message open — no `output_text.done`, no `output_item.done` — while the+model composes the calls, and releases the message close and every call+together once they are composed. A batch that takes longer than the tightened+budget (a quarter of the configured one) was killed as a stall and re-sent,+paying the whole generation again. Every stall seen in evaluation runs since+ADR 0241 had this shape: a few seconds of heads-up text, then the tightened+budget expiring on a healthy compose.++## Decision++- Track whether the open output item is a commentary message (`Phase` in+ `agent_ws_signal.zig`), fed WebSocket frames and SSE `data:` lines alike.+- Commentary prose still counts as visible text (first-token time, traces),+ but it does not tighten the budget: the full configured budget stands until+ the item closes, as it does for a silent reasoning phase.+- Final-answer messages, messages without a phase, and whitelisted+ tool-argument prose (`attempt_completion`, `ask_user`) tighten as before, so+ a stream that dies mid-answer is still caught at the tightened budget.++## Consequences++A stream that dies during a commentary heads-up waits the full budget instead+of a quarter of it, the same wait a silent reasoning phase already gets. The+trace's first-text note says which regime the budget is in.diff --git a/docs/adr/README.md b/docs/adr/README.mdindex 4f9d59e8..949f1a91 100644--- a/docs/adr/README.md+++ b/docs/adr/README.md@@ -274,6 +274,7 @@ record only when you need the evidence or the edge cases.| [0245](0245-fullscreen-tui-is-an-external-client.md) | The fullscreen UI is maintained separately; `graff tui` launches installed `graff-tui` before engine startup, sibling first then PATH, without downloading. TTY `graff repl` uses the line REPL. Supersedes 0041, 0042 and the fullscreen portion of 0142. || [0246](0246-jev-picks-each-turns-effort-alongside-the-first-request.md) | On an eligible model at the default effort, graff asks Jev for the turn's effort alongside the first request; the pick applies from the next request, holds for that turn, is never saved, and never overrides a user's effort. `chatgpt-new` GPT-6 models are eligible. || [0247](0247-parked-work-is-named-above-the-prompt.md) | A parked turn names the work it waits on: the standing block draws `↻ waiting on <work> — graff continues when it finishes` while a job or agent that will wake the session runs, the yield prints only for a server, and a wake prints what finished before continuing. |+| [0248](0248-commentary-prose-does-not-tighten-the-stall-budget.md) | Commentary prose (the heads-up before a tool-call batch, which the server holds open until it releases the calls) counts as visible text but does not tighten the stall budget; final-answer prose and whitelisted tool-argument prose still do. Refines 0241. |## When to write onediff --git a/src/agent_stream.zig b/src/agent_stream.zigindex 670eaee4..ff5a351f 100644--- a/src/agent_stream.zig+++ b/src/agent_stream.zig@@ -233,6 +233,7 @@ pub fn postStreamWithClient(self: *Agent, client: *std.http.Client, body: []consvar got_body = false; // #134: true once response bytes have been received (gates the post-completion read-error handling)var saw_done = false; // #133: true only once the provider's terminal event landed — a close before this is a drop, not a clean endvar prose_open = false; // #56/ADR 0241: prose is streaming in an item that has not closed+ var phase: @import("agent_ws_signal.zig").Phase = .{}; // ADR 0248: commentary does not tightendefer line.deinit();stream: while (true) {@@ -316,7 +317,8 @@ pub fn postStreamWithClient(self: *Agent, client: *std.http.Client, body: []constry full.writer.writeByte('\n');got_body = true; // #134: response bytes are in `full`; a later read error is a clean closeself.printDelta(line.writer.buffered());- if (self.partial_text.items.len > prose_len) prose_open = true else if (@import("agent_ws_signal.zig").closesItem(gpa, line.writer.buffered())) prose_open = false;+ phase.feed(gpa, line.writer.buffered());+ if (self.partial_text.items.len > prose_len) prose_open = !phase.commentary else if (@import("agent_ws_signal.zig").closesItem(gpa, line.writer.buffered())) prose_open = false;if (main_mod.g_thinking_fold_request) {main_mod.g_thinking_fold_request = false;sink.emit(self.io, .thinking_fold_toggle);diff --git a/src/agent_stream_stall_test.zig b/src/agent_stream_stall_test.zigindex 45d6490b..2458bd2a 100644--- a/src/agent_stream_stall_test.zig+++ b/src/agent_stream_stall_test.zig@@ -306,12 +306,16 @@ test "Chat terminal marker keeps delayed usage and completes silent trailer; non}/// ADR 0241 on SSE: a Responses stream whose prose item closes, then thinks in-/// silence past the tightened budget before it completes.+/// silence past the tightened budget before it completes. ADR 0248 reuses it+/// with a commentary heads-up the server holds open through the think.const ResponsesSrv = struct {const delta = "data: {\"type\":\"response.output_text.delta\",\"delta\":\"hi\"}\n\n";const item_done = "data: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\"}}\n\n";+ const commentary_added = "data: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}\n\n";const completed = "data: {\"type\":\"response.completed\",\"response\":{\"id\":\"r1\",\"output\":[]}}\n\n";const think_ms = 1000;+ var head: []const u8 = delta ++ item_done;+ var tail: []const u8 = completed;fn run(io: Io, server: *std.Io.net.Server, done: *std.atomic.Value(bool)) void {const c = while (true) {const conn = server.accept(io) catch return;@@ -321,17 +325,20 @@ const ResponsesSrv = struct {defer c.close(io);var wbuf: [1024]u8 = undefined;var sw = std.Io.net.Stream.Writer.init(c, io, &wbuf);- sw.interface.writeAll("HTTP/1.1 200 OK\r\ncontent-type: text/event-stream\r\nconnection: close\r\n\r\n" ++ delta ++ item_done) catch return;+ sw.interface.writeAll("HTTP/1.1 200 OK\r\ncontent-type: text/event-stream\r\nconnection: close\r\n\r\n") catch return;+ sw.interface.writeAll(head) catch return;sw.interface.flush() catch return;io.sleep(.fromMilliseconds(think_ms), .awake) catch return;- sw.interface.writeAll(completed) catch return;+ sw.interface.writeAll(tail) catch return;sw.interface.flush() catch return;while (!done.load(.acquire)) io.sleep(.fromMilliseconds(10), .awake) catch return;}};-test "ADR 0241: on the Responses SSE wire a closed prose item gets the full budget back" {+fn thinkPastTightenedBudget(head: []const u8, tail: []const u8) !void {if (builtin.os.tag == .windows) return error.SkipZigTest;+ ResponsesSrv.head = head;+ ResponsesSrv.tail = tail;const gpa = std.testing.allocator;const io = std.testing.io;@@ -380,5 +387,15 @@ test "ADR 0241: on the Responses SSE wire a closed prose item gets the full budgdefer gpa.free(body);try std.testing.expect(nowMs(io) - t0 >= ResponsesSrv.think_ms - 100); // the think really happenedtry std.testing.expect(std.mem.indexOf(u8, body, "response.completed") != null);- try std.testing.expect(agent.partial_text.items.len != 0); // prose did flow, so the budget had tightened+ try std.testing.expect(agent.partial_text.items.len != 0); // prose did flow+}++test "ADR 0241: on the Responses SSE wire a closed prose item gets the full budget back" {+ try thinkPastTightenedBudget(ResponsesSrv.delta ++ ResponsesSrv.item_done, ResponsesSrv.completed);+}++// ADR 0248 on SSE: the server holds a commentary heads-up open through the+// compose; commentary prose must not tighten, so the think finishes.+test "ADR 0248: on the Responses SSE wire a commentary heads-up held open keeps the full budget" {+ try thinkPastTightenedBudget(ResponsesSrv.commentary_added ++ ResponsesSrv.delta, ResponsesSrv.item_done ++ ResponsesSrv.completed);}diff --git a/src/agent_ws.zig b/src/agent_ws.zigindex d7972c85..fca9eb36 100644--- a/src/agent_ws.zig+++ b/src/agent_ws.zig@@ -563,7 +563,7 @@ pub fn postResponsesWs(self: *Agent, body: []const u8) ![]u8 {if (sig.step(gpa, fbuf.items) and !text_seen) {text_seen = true;self.traceFirstToken();- if (self.tracer) |tr| tr.note("ws", "first output text — tightening stall budget");+ if (self.tracer) |tr| tr.note("ws", if (sig.open) "first output text — tightening stall budget" else "first output text — commentary, budget unchanged");}// Wrap the ws frame as an SSE data: line so parseResponses/isStreamEnddiff --git a/src/agent_ws_mock.zig b/src/agent_ws_mock.zigindex 1cbbcfc3..1c440471 100644--- a/src/agent_ws_mock.zig+++ b/src/agent_ws_mock.zig@@ -54,6 +54,11 @@ pub var slow_first_frame_ms: i64 = 700;pub const item_done_event = "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\"}}";pub const think_after_prose_ms = 1000;+/// ADR 0248: a commentary heads-up the server holds open while it composes the+/// tool calls it then releases together (`commentary_then_think`).+pub const commentary_added_event = "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}";+pub const commentary_done_event = "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}";+/// A loopback WebSocket peer with a scripted failure mode.pub const Mock = struct {pub const Mode = enum {@@ -90,6 +95,10 @@ pub const Mock = struct {/// ADR 0241: a heads-up delta, its item closes, a silent think past the/// tightened budget (a tool call being composed), then completion.prose_close_then_think,+ /// ADR 0248: a commentary heads-up whose message stays OPEN through a+ /// silent compose past the tightened budget; then it closes and the+ /// turn completes.+ commentary_then_think,};pub fn run(io: Io, server: *std.Io.net.Server, mode: Mode, done: *std.atomic.Value(bool)) void {@@ -140,6 +149,14 @@ pub const Mock = struct {io.sleep(.fromMilliseconds(think_after_prose_ms), .awake) catch return idle(io, done);writeTextFrame(&sw.interface, completed_event) catch return idle(io, done);},+ .commentary_then_think => {+ readRealFrame(io, &sr.interface, &sw.interface) catch return idle(io, done);+ writeTextFrame(&sw.interface, commentary_added_event) catch return idle(io, done);+ writeTextFrame(&sw.interface, delta_event) catch return idle(io, done);+ io.sleep(.fromMilliseconds(think_after_prose_ms), .awake) catch return idle(io, done);+ writeTextFrame(&sw.interface, commentary_done_event) catch return idle(io, done);+ writeTextFrame(&sw.interface, completed_event) catch return idle(io, done);+ },.generic_error_then_close => {readRealFrame(io, &sr.interface, &sw.interface) catch return;writeTextFrame(&sw.interface, generic_error_event) catch return;diff --git a/src/agent_ws_signal.zig b/src/agent_ws_signal.zigindex 97a29eb8..ccea09c7 100644--- a/src/agent_ws_signal.zig+++ b/src/agent_ws_signal.zig@@ -81,6 +81,42 @@ pub fn closesItem(gpa: std.mem.Allocator, frame: []const u8) bool {return ty == .string and std.mem.eql(u8, ty.string, ev);}+/// ADR 0248: is the open output item a COMMENTARY message — the heads-up the+/// model writes before its tool calls? The server holds that message open+/// (no output_text.done, no output_item.done) until it releases the calls,+/// which it sends together once all of them are composed. Silence after+/// commentary is that composition, not a dead stream, so commentary prose+/// must not tighten the budget. A final answer (or a message with no phase)+/// still does. Fed WS frames and SSE `data:` lines alike.+pub const Phase = struct {+ commentary: bool = false,++ pub fn feed(self: *Phase, gpa: std.mem.Allocator, frame: []const u8) void {+ if (std.mem.indexOf(u8, frame, "response.output_item.") == null) return;+ if (closesItem(gpa, frame)) {+ self.commentary = false;+ return;+ }+ var payload = std.mem.trim(u8, frame, " \t\r\n");+ if (std.mem.startsWith(u8, payload, "data:")) payload = std.mem.trim(u8, payload["data:".len..], " \t");+ var scratch = std.heap.ArenaAllocator.init(gpa);+ defer scratch.deinit();+ const v = std.json.parseFromSliceLeaky(std.json.Value, scratch.allocator(), payload, .{ .allocate = .alloc_always }) catch return;+ if (v != .object) return;+ const ty = v.object.get("type") orelse return;+ if (ty != .string or !std.mem.eql(u8, ty.string, "response.output_item.added")) return;+ const item = v.object.get("item") orelse return;+ if (item != .object) return;+ const it = item.object.get("type") orelse return;+ const phase = item.object.get("phase") orelse {+ self.commentary = false;+ return;+ };+ self.commentary = it == .string and std.mem.eql(u8, it.string, "message") and+ phase == .string and std.mem.eql(u8, phase.string, "commentary");+ }+};+/// The turn's tokens-flowing signal: both producers, and the little bit of/// state producer 2 needs (which output item is the open whitelisted call).///@@ -98,6 +134,8 @@ pub const TokenSignal = struct {/// ADR 0241: prose is streaming in an item that has not closed yet. The WS/// read budget is tightened only while this holds.open: bool = false,+ /// ADR 0248: commentary prose flows without tightening (see Phase).+ phase: Phase = .{},/// True when a frame proves visible prose is flowing.pub fn flowing(self: *TokenSignal, gpa: std.mem.Allocator, frame: []const u8) bool {@@ -105,11 +143,13 @@ pub const TokenSignal = struct {return self.argProse(gpa, frame);}- /// Fed every frame. True when this frame is prose, which opens the gate;- /// a frame that closes an output item shuts it again (ADR 0241).+ /// Fed every frame. True when this frame is prose, which opens the gate+ /// unless it is commentary (ADR 0248); a frame that closes an output item+ /// shuts it again (ADR 0241).pub fn step(self: *TokenSignal, gpa: std.mem.Allocator, frame: []const u8) bool {+ self.phase.feed(gpa, frame);if (self.flowing(gpa, frame)) {- self.open = true;+ if (!self.phase.commentary) self.open = true;return true;}if (closesItem(gpa, frame)) self.open = false;@@ -253,3 +293,44 @@ test "ADR 0241: closesItem reads WS frames and SSE lines, not prose that quotestry std.testing.expect(!closesItem(gpa, "event: response.output_text.delta"));try std.testing.expect(!closesItem(gpa, "event: content_block_stop"));}++test "ADR 0248: commentary prose flows without tightening; a final answer still tightens" {+ const gpa = std.testing.allocator;+ var sig: TokenSignal = .{};+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}"));+ try std.testing.expect(sig.phase.commentary);+ // Visible prose (TTFT and the trace still see it) but the gate stays shut.+ try std.testing.expect(sig.step(gpa, "{\"type\":\"response.output_text.delta\",\"output_index\":1,\"delta\":\"Found the bugs; fixing them.\"}"));+ try std.testing.expect(!sig.open);+ // The server closes the message only when it releases the composed calls.+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}"));+ try std.testing.expect(!sig.phase.commentary);+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":2,\"item\":{\"type\":\"message\",\"phase\":\"final_answer\"}}"));+ try std.testing.expect(sig.step(gpa, "{\"type\":\"response.output_text.delta\",\"output_index\":2,\"delta\":\"Done.\"}"));+ try std.testing.expect(sig.open);+}++test "ADR 0248: a message without a phase, and attempt_completion prose, tighten as before" {+ const gpa = std.testing.allocator;+ var sig: TokenSignal = .{};+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\"}}"));+ try std.testing.expect(sig.step(gpa, "{\"type\":\"response.output_text.delta\",\"output_index\":0,\"delta\":\"hi\"}"));+ try std.testing.expect(sig.open);+ var arg: TokenSignal = .{};+ try std.testing.expect(!arg.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"name\":\"attempt_completion\"}}"));+ try std.testing.expect(arg.step(gpa, "{\"type\":\"response.function_call_arguments.delta\",\"output_index\":0,\"delta\":\"{\\\"result\\\":\\\"ok\"}"));+ try std.testing.expect(arg.open);+}++test "ADR 0248: Phase reads SSE data lines and ignores prose that quotes the phase" {+ const gpa = std.testing.allocator;+ var ph: Phase = .{};+ ph.feed(gpa, "data: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}");+ try std.testing.expect(ph.commentary);+ ph.feed(gpa, "data: {\"type\":\"response.output_text.delta\",\"delta\":\"response.output_item.added commentary\"}");+ try std.testing.expect(ph.commentary);+ ph.feed(gpa, "data: {\"type\":\"response.output_item.done\",\"output_index\":0}");+ try std.testing.expect(!ph.commentary);+ ph.feed(gpa, "data: {\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"name\":\"edit_file\"}}");+ try std.testing.expect(!ph.commentary);+}diff --git a/src/agent_ws_stall_test.zig b/src/agent_ws_stall_test.zigindex e23f94cc..4af059c1 100644--- a/src/agent_ws_stall_test.zig+++ b/src/agent_ws_stall_test.zig@@ -171,6 +171,61 @@ test "ADR 0241: a closed prose item gets the full budget back for a silent thinktry std.testing.expect(!traced(&tw, "\"detail\":\"stall\""));}+// ADR 0248: the server holds a commentary heads-up open while it composes the+// tool calls it releases together, so ADR 0241's item-close never comes in+// time. Commentary must not tighten: a compose past the tightened budget with+// the message still open has to finish, not stall.+test "ADR 0248: a commentary heads-up held open keeps the full budget" {+ if (builtin.os.tag == .windows) return error.SkipZigTest;+ const gpa = std.testing.allocator;+ const io = std.testing.io;++ var addr = try std.Io.net.IpAddress.parseLiteral("127.0.0.1:0");+ var server = try std.Io.net.IpAddress.listen(&addr, io, .{});+ defer server.deinit(io);+ var done: std.atomic.Value(bool) = .init(false);+ var fut = io.async(Mock.run, .{ io, &server, Mock.Mode.commentary_then_think, &done });+ defer fut.await(io);+ defer done.store(true, .release);++ const saved_stream = http.stream_stall_ms;+ const saved_floor = http_stall.idle_floor_ms;+ http.stream_stall_ms = 2000;+ http_stall.idle_floor_ms = 100;+ defer http.stream_stall_ms = saved_stream;+ defer http_stall.idle_floor_ms = saved_floor;+ try std.testing.expect(mock.think_after_prose_ms > http_stall.budgetMs(http.stream_stall_ms, true));++ var arena_state = std.heap.ArenaAllocator.init(gpa);+ defer arena_state.deinit();+ const arena = arena_state.allocator();++ var tw: Io.Writer.Allocating = .init(gpa);+ defer tw.deinit();+ var tracer: trace.Tracer = .{ .io = io, .gpa = gpa, .out = &tw.writer, .start = Io.Timestamp.now(io, .awake) };++ var url_buf: [64]u8 = undefined;+ const url = try std.fmt.bufPrint(&url_buf, "http://127.0.0.1:{d}/x", .{server.socket.address.getPort()});+ var agent = mockAgent(gpa, arena, io, url);+ agent.tracer = &tracer;+ defer if (agent.codex_ws) |c| {+ c.dead = true;+ c.deinit(gpa);+ agent.codex_ws = null;+ };++ const body = "{\"model\":\"gpt-5\",\"input\":[]}";+ const t0 = nowMs(io);+ const out = try agent_ws.postResponsesWs(&agent, body);+ defer gpa.free(out);+ try std.testing.expect(nowMs(io) - t0 >= mock.think_after_prose_ms - 100); // the compose really happened+ try std.testing.expect(std.mem.indexOf(u8, out, mock.commentary_done_event) != null);+ try std.testing.expect(std.mem.indexOf(u8, out, completed_event) != null);+ try std.testing.expect(traced(&tw, "commentary, budget unchanged")); // prose did flow+ try std.testing.expect(traced(&tw, "\"detail\":\"completed\""));+ try std.testing.expect(!traced(&tw, "\"detail\":\"stall\""));+}+// The control: same harness, same tightened budgets, a server that finishes.// Proves the stall is the SERVER's silence, not the mock — and pins the notes.test "#401 control: a mock that completes the response still finishes the turn cleanly" {