Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

stall: commentary prose does not tighten the read budget

justrachd6fb867740df
diff --git a/docs/adr/0248-commentary-prose-does-not-tighten-the-stall-budget.md b/docs/adr/0248-commentary-prose-does-not-tighten-the-stall-budget.md
new file mode 100644
index 00000000..fda1ed77
--- /dev/null
+++ b/docs/adr/0248-commentary-prose-does-not-tighten-the-stall-budget.md
@@ -0,0 +1,34 @@
+# 0248. Commentary prose does not tighten the stall budget
+
+Status: accepted 2026-10-03
+
+## Context
+
+ADR 0241 tightens the between-frames read budget while prose streams and gives
+the full budget back when the prose item closes. On the Codex Responses
+backend, GPT-6 models write a commentary heads-up (`phase: "commentary"`)
+before a batch of tool calls. The server streams that heads-up, then holds
+the message open — no `output_text.done`, no `output_item.done` — while the
+model composes the calls, and releases the message close and every call
+together once they are composed. A batch that takes longer than the tightened
+budget (a quarter of the configured one) was killed as a stall and re-sent,
+paying the whole generation again. Every stall seen in evaluation runs since
+ADR 0241 had this shape: a few seconds of heads-up text, then the tightened
+budget expiring on a healthy compose.
+
+## Decision
+
+- Track whether the open output item is a commentary message (`Phase` in
+ `agent_ws_signal.zig`), fed WebSocket frames and SSE `data:` lines alike.
+- Commentary prose still counts as visible text (first-token time, traces),
+ but it does not tighten the budget: the full configured budget stands until
+ the item closes, as it does for a silent reasoning phase.
+- Final-answer messages, messages without a phase, and whitelisted
+ tool-argument prose (`attempt_completion`, `ask_user`) tighten as before, so
+ a stream that dies mid-answer is still caught at the tightened budget.
+
+## Consequences
+
+A stream that dies during a commentary heads-up waits the full budget instead
+of a quarter of it, the same wait a silent reasoning phase already gets. The
+trace's first-text note says which regime the budget is in.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 4f9d59e8..949f1a91 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -274,6 +274,7 @@ record only when you need the evidence or the edge cases.
| [0245](0245-fullscreen-tui-is-an-external-client.md) | The fullscreen UI is maintained separately; `graff tui` launches installed `graff-tui` before engine startup, sibling first then PATH, without downloading. TTY `graff repl` uses the line REPL. Supersedes 0041, 0042 and the fullscreen portion of 0142. |
| [0246](0246-jev-picks-each-turns-effort-alongside-the-first-request.md) | On an eligible model at the default effort, graff asks Jev for the turn's effort alongside the first request; the pick applies from the next request, holds for that turn, is never saved, and never overrides a user's effort. `chatgpt-new` GPT-6 models are eligible. |
| [0247](0247-parked-work-is-named-above-the-prompt.md) | A parked turn names the work it waits on: the standing block draws `↻ waiting on <work> — graff continues when it finishes` while a job or agent that will wake the session runs, the yield prints only for a server, and a wake prints what finished before continuing. |
+| [0248](0248-commentary-prose-does-not-tighten-the-stall-budget.md) | Commentary prose (the heads-up before a tool-call batch, which the server holds open until it releases the calls) counts as visible text but does not tighten the stall budget; final-answer prose and whitelisted tool-argument prose still do. Refines 0241. |
## When to write one
diff --git a/src/agent_stream.zig b/src/agent_stream.zig
index 670eaee4..ff5a351f 100644
--- a/src/agent_stream.zig
+++ b/src/agent_stream.zig
@@ -233,6 +233,7 @@ pub fn postStreamWithClient(self: *Agent, client: *std.http.Client, body: []cons
var got_body = false; // #134: true once response bytes have been received (gates the post-completion read-error handling)
var saw_done = false; // #133: true only once the provider's terminal event landed — a close before this is a drop, not a clean end
var prose_open = false; // #56/ADR 0241: prose is streaming in an item that has not closed
+ var phase: @import("agent_ws_signal.zig").Phase = .{}; // ADR 0248: commentary does not tighten
defer line.deinit();
stream: while (true) {
@@ -316,7 +317,8 @@ pub fn postStreamWithClient(self: *Agent, client: *std.http.Client, body: []cons
try full.writer.writeByte('\n');
got_body = true; // #134: response bytes are in `full`; a later read error is a clean close
self.printDelta(line.writer.buffered());
- if (self.partial_text.items.len > prose_len) prose_open = true else if (@import("agent_ws_signal.zig").closesItem(gpa, line.writer.buffered())) prose_open = false;
+ phase.feed(gpa, line.writer.buffered());
+ if (self.partial_text.items.len > prose_len) prose_open = !phase.commentary else if (@import("agent_ws_signal.zig").closesItem(gpa, line.writer.buffered())) prose_open = false;
if (main_mod.g_thinking_fold_request) {
main_mod.g_thinking_fold_request = false;
sink.emit(self.io, .thinking_fold_toggle);
diff --git a/src/agent_stream_stall_test.zig b/src/agent_stream_stall_test.zig
index 45d6490b..2458bd2a 100644
--- a/src/agent_stream_stall_test.zig
+++ b/src/agent_stream_stall_test.zig
@@ -306,12 +306,16 @@ test "Chat terminal marker keeps delayed usage and completes silent trailer; non
}
/// ADR 0241 on SSE: a Responses stream whose prose item closes, then thinks in
-/// silence past the tightened budget before it completes.
+/// silence past the tightened budget before it completes. ADR 0248 reuses it
+/// with a commentary heads-up the server holds open through the think.
const ResponsesSrv = struct {
const delta = "data: {\"type\":\"response.output_text.delta\",\"delta\":\"hi\"}\n\n";
const item_done = "data: {\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\"}}\n\n";
+ const commentary_added = "data: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}\n\n";
const completed = "data: {\"type\":\"response.completed\",\"response\":{\"id\":\"r1\",\"output\":[]}}\n\n";
const think_ms = 1000;
+ var head: []const u8 = delta ++ item_done;
+ var tail: []const u8 = completed;
fn run(io: Io, server: *std.Io.net.Server, done: *std.atomic.Value(bool)) void {
const c = while (true) {
const conn = server.accept(io) catch return;
@@ -321,17 +325,20 @@ const ResponsesSrv = struct {
defer c.close(io);
var wbuf: [1024]u8 = undefined;
var sw = std.Io.net.Stream.Writer.init(c, io, &wbuf);
- sw.interface.writeAll("HTTP/1.1 200 OK\r\ncontent-type: text/event-stream\r\nconnection: close\r\n\r\n" ++ delta ++ item_done) catch return;
+ sw.interface.writeAll("HTTP/1.1 200 OK\r\ncontent-type: text/event-stream\r\nconnection: close\r\n\r\n") catch return;
+ sw.interface.writeAll(head) catch return;
sw.interface.flush() catch return;
io.sleep(.fromMilliseconds(think_ms), .awake) catch return;
- sw.interface.writeAll(completed) catch return;
+ sw.interface.writeAll(tail) catch return;
sw.interface.flush() catch return;
while (!done.load(.acquire)) io.sleep(.fromMilliseconds(10), .awake) catch return;
}
};
-test "ADR 0241: on the Responses SSE wire a closed prose item gets the full budget back" {
+fn thinkPastTightenedBudget(head: []const u8, tail: []const u8) !void {
if (builtin.os.tag == .windows) return error.SkipZigTest;
+ ResponsesSrv.head = head;
+ ResponsesSrv.tail = tail;
const gpa = std.testing.allocator;
const io = std.testing.io;
@@ -380,5 +387,15 @@ test "ADR 0241: on the Responses SSE wire a closed prose item gets the full budg
defer gpa.free(body);
try std.testing.expect(nowMs(io) - t0 >= ResponsesSrv.think_ms - 100); // the think really happened
try std.testing.expect(std.mem.indexOf(u8, body, "response.completed") != null);
- try std.testing.expect(agent.partial_text.items.len != 0); // prose did flow, so the budget had tightened
+ try std.testing.expect(agent.partial_text.items.len != 0); // prose did flow
+}
+
+test "ADR 0241: on the Responses SSE wire a closed prose item gets the full budget back" {
+ try thinkPastTightenedBudget(ResponsesSrv.delta ++ ResponsesSrv.item_done, ResponsesSrv.completed);
+}
+
+// ADR 0248 on SSE: the server holds a commentary heads-up open through the
+// compose; commentary prose must not tighten, so the think finishes.
+test "ADR 0248: on the Responses SSE wire a commentary heads-up held open keeps the full budget" {
+ try thinkPastTightenedBudget(ResponsesSrv.commentary_added ++ ResponsesSrv.delta, ResponsesSrv.item_done ++ ResponsesSrv.completed);
}
diff --git a/src/agent_ws.zig b/src/agent_ws.zig
index d7972c85..fca9eb36 100644
--- a/src/agent_ws.zig
+++ b/src/agent_ws.zig
@@ -563,7 +563,7 @@ pub fn postResponsesWs(self: *Agent, body: []const u8) ![]u8 {
if (sig.step(gpa, fbuf.items) and !text_seen) {
text_seen = true;
self.traceFirstToken();
- if (self.tracer) |tr| tr.note("ws", "first output text — tightening stall budget");
+ if (self.tracer) |tr| tr.note("ws", if (sig.open) "first output text — tightening stall budget" else "first output text — commentary, budget unchanged");
}
// Wrap the ws frame as an SSE data: line so parseResponses/isStreamEnd
diff --git a/src/agent_ws_mock.zig b/src/agent_ws_mock.zig
index 1cbbcfc3..1c440471 100644
--- a/src/agent_ws_mock.zig
+++ b/src/agent_ws_mock.zig
@@ -54,6 +54,11 @@ pub var slow_first_frame_ms: i64 = 700;
pub const item_done_event = "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\"}}";
pub const think_after_prose_ms = 1000;
+/// ADR 0248: a commentary heads-up the server holds open while it composes the
+/// tool calls it then releases together (`commentary_then_think`).
+pub const commentary_added_event = "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}";
+pub const commentary_done_event = "{\"type\":\"response.output_item.done\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}";
+
/// A loopback WebSocket peer with a scripted failure mode.
pub const Mock = struct {
pub const Mode = enum {
@@ -90,6 +95,10 @@ pub const Mock = struct {
/// ADR 0241: a heads-up delta, its item closes, a silent think past the
/// tightened budget (a tool call being composed), then completion.
prose_close_then_think,
+ /// ADR 0248: a commentary heads-up whose message stays OPEN through a
+ /// silent compose past the tightened budget; then it closes and the
+ /// turn completes.
+ commentary_then_think,
};
pub fn run(io: Io, server: *std.Io.net.Server, mode: Mode, done: *std.atomic.Value(bool)) void {
@@ -140,6 +149,14 @@ pub const Mock = struct {
io.sleep(.fromMilliseconds(think_after_prose_ms), .awake) catch return idle(io, done);
writeTextFrame(&sw.interface, completed_event) catch return idle(io, done);
},
+ .commentary_then_think => {
+ readRealFrame(io, &sr.interface, &sw.interface) catch return idle(io, done);
+ writeTextFrame(&sw.interface, commentary_added_event) catch return idle(io, done);
+ writeTextFrame(&sw.interface, delta_event) catch return idle(io, done);
+ io.sleep(.fromMilliseconds(think_after_prose_ms), .awake) catch return idle(io, done);
+ writeTextFrame(&sw.interface, commentary_done_event) catch return idle(io, done);
+ writeTextFrame(&sw.interface, completed_event) catch return idle(io, done);
+ },
.generic_error_then_close => {
readRealFrame(io, &sr.interface, &sw.interface) catch return;
writeTextFrame(&sw.interface, generic_error_event) catch return;
diff --git a/src/agent_ws_signal.zig b/src/agent_ws_signal.zig
index 97a29eb8..ccea09c7 100644
--- a/src/agent_ws_signal.zig
+++ b/src/agent_ws_signal.zig
@@ -81,6 +81,42 @@ pub fn closesItem(gpa: std.mem.Allocator, frame: []const u8) bool {
return ty == .string and std.mem.eql(u8, ty.string, ev);
}
+/// ADR 0248: is the open output item a COMMENTARY message — the heads-up the
+/// model writes before its tool calls? The server holds that message open
+/// (no output_text.done, no output_item.done) until it releases the calls,
+/// which it sends together once all of them are composed. Silence after
+/// commentary is that composition, not a dead stream, so commentary prose
+/// must not tighten the budget. A final answer (or a message with no phase)
+/// still does. Fed WS frames and SSE `data:` lines alike.
+pub const Phase = struct {
+ commentary: bool = false,
+
+ pub fn feed(self: *Phase, gpa: std.mem.Allocator, frame: []const u8) void {
+ if (std.mem.indexOf(u8, frame, "response.output_item.") == null) return;
+ if (closesItem(gpa, frame)) {
+ self.commentary = false;
+ return;
+ }
+ var payload = std.mem.trim(u8, frame, " \t\r\n");
+ if (std.mem.startsWith(u8, payload, "data:")) payload = std.mem.trim(u8, payload["data:".len..], " \t");
+ var scratch = std.heap.ArenaAllocator.init(gpa);
+ defer scratch.deinit();
+ const v = std.json.parseFromSliceLeaky(std.json.Value, scratch.allocator(), payload, .{ .allocate = .alloc_always }) catch return;
+ if (v != .object) return;
+ const ty = v.object.get("type") orelse return;
+ if (ty != .string or !std.mem.eql(u8, ty.string, "response.output_item.added")) return;
+ const item = v.object.get("item") orelse return;
+ if (item != .object) return;
+ const it = item.object.get("type") orelse return;
+ const phase = item.object.get("phase") orelse {
+ self.commentary = false;
+ return;
+ };
+ self.commentary = it == .string and std.mem.eql(u8, it.string, "message") and
+ phase == .string and std.mem.eql(u8, phase.string, "commentary");
+ }
+};
+
/// The turn's tokens-flowing signal: both producers, and the little bit of
/// state producer 2 needs (which output item is the open whitelisted call).
///
@@ -98,6 +134,8 @@ pub const TokenSignal = struct {
/// ADR 0241: prose is streaming in an item that has not closed yet. The WS
/// read budget is tightened only while this holds.
open: bool = false,
+ /// ADR 0248: commentary prose flows without tightening (see Phase).
+ phase: Phase = .{},
/// True when a frame proves visible prose is flowing.
pub fn flowing(self: *TokenSignal, gpa: std.mem.Allocator, frame: []const u8) bool {
@@ -105,11 +143,13 @@ pub const TokenSignal = struct {
return self.argProse(gpa, frame);
}
- /// Fed every frame. True when this frame is prose, which opens the gate;
- /// a frame that closes an output item shuts it again (ADR 0241).
+ /// Fed every frame. True when this frame is prose, which opens the gate
+ /// unless it is commentary (ADR 0248); a frame that closes an output item
+ /// shuts it again (ADR 0241).
pub fn step(self: *TokenSignal, gpa: std.mem.Allocator, frame: []const u8) bool {
+ self.phase.feed(gpa, frame);
if (self.flowing(gpa, frame)) {
- self.open = true;
+ if (!self.phase.commentary) self.open = true;
return true;
}
if (closesItem(gpa, frame)) self.open = false;
@@ -253,3 +293,44 @@ test "ADR 0241: closesItem reads WS frames and SSE lines, not prose that quotes
try std.testing.expect(!closesItem(gpa, "event: response.output_text.delta"));
try std.testing.expect(!closesItem(gpa, "event: content_block_stop"));
}
+
+test "ADR 0248: commentary prose flows without tightening; a final answer still tightens" {
+ const gpa = std.testing.allocator;
+ var sig: TokenSignal = .{};
+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}"));
+ try std.testing.expect(sig.phase.commentary);
+ // Visible prose (TTFT and the trace still see it) but the gate stays shut.
+ try std.testing.expect(sig.step(gpa, "{\"type\":\"response.output_text.delta\",\"output_index\":1,\"delta\":\"Found the bugs; fixing them.\"}"));
+ try std.testing.expect(!sig.open);
+ // The server closes the message only when it releases the composed calls.
+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.done\",\"output_index\":1,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}"));
+ try std.testing.expect(!sig.phase.commentary);
+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":2,\"item\":{\"type\":\"message\",\"phase\":\"final_answer\"}}"));
+ try std.testing.expect(sig.step(gpa, "{\"type\":\"response.output_text.delta\",\"output_index\":2,\"delta\":\"Done.\"}"));
+ try std.testing.expect(sig.open);
+}
+
+test "ADR 0248: a message without a phase, and attempt_completion prose, tighten as before" {
+ const gpa = std.testing.allocator;
+ var sig: TokenSignal = .{};
+ try std.testing.expect(!sig.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\"}}"));
+ try std.testing.expect(sig.step(gpa, "{\"type\":\"response.output_text.delta\",\"output_index\":0,\"delta\":\"hi\"}"));
+ try std.testing.expect(sig.open);
+ var arg: TokenSignal = .{};
+ try std.testing.expect(!arg.step(gpa, "{\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"function_call\",\"name\":\"attempt_completion\"}}"));
+ try std.testing.expect(arg.step(gpa, "{\"type\":\"response.function_call_arguments.delta\",\"output_index\":0,\"delta\":\"{\\\"result\\\":\\\"ok\"}"));
+ try std.testing.expect(arg.open);
+}
+
+test "ADR 0248: Phase reads SSE data lines and ignores prose that quotes the phase" {
+ const gpa = std.testing.allocator;
+ var ph: Phase = .{};
+ ph.feed(gpa, "data: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"message\",\"phase\":\"commentary\"}}");
+ try std.testing.expect(ph.commentary);
+ ph.feed(gpa, "data: {\"type\":\"response.output_text.delta\",\"delta\":\"response.output_item.added commentary\"}");
+ try std.testing.expect(ph.commentary);
+ ph.feed(gpa, "data: {\"type\":\"response.output_item.done\",\"output_index\":0}");
+ try std.testing.expect(!ph.commentary);
+ ph.feed(gpa, "data: {\"type\":\"response.output_item.added\",\"output_index\":1,\"item\":{\"type\":\"function_call\",\"name\":\"edit_file\"}}");
+ try std.testing.expect(!ph.commentary);
+}
diff --git a/src/agent_ws_stall_test.zig b/src/agent_ws_stall_test.zig
index e23f94cc..4af059c1 100644
--- a/src/agent_ws_stall_test.zig
+++ b/src/agent_ws_stall_test.zig
@@ -171,6 +171,61 @@ test "ADR 0241: a closed prose item gets the full budget back for a silent think
try std.testing.expect(!traced(&tw, "\"detail\":\"stall\""));
}
+// ADR 0248: the server holds a commentary heads-up open while it composes the
+// tool calls it releases together, so ADR 0241's item-close never comes in
+// time. Commentary must not tighten: a compose past the tightened budget with
+// the message still open has to finish, not stall.
+test "ADR 0248: a commentary heads-up held open keeps the full budget" {
+ if (builtin.os.tag == .windows) return error.SkipZigTest;
+ const gpa = std.testing.allocator;
+ const io = std.testing.io;
+
+ var addr = try std.Io.net.IpAddress.parseLiteral("127.0.0.1:0");
+ var server = try std.Io.net.IpAddress.listen(&addr, io, .{});
+ defer server.deinit(io);
+ var done: std.atomic.Value(bool) = .init(false);
+ var fut = io.async(Mock.run, .{ io, &server, Mock.Mode.commentary_then_think, &done });
+ defer fut.await(io);
+ defer done.store(true, .release);
+
+ const saved_stream = http.stream_stall_ms;
+ const saved_floor = http_stall.idle_floor_ms;
+ http.stream_stall_ms = 2000;
+ http_stall.idle_floor_ms = 100;
+ defer http.stream_stall_ms = saved_stream;
+ defer http_stall.idle_floor_ms = saved_floor;
+ try std.testing.expect(mock.think_after_prose_ms > http_stall.budgetMs(http.stream_stall_ms, true));
+
+ var arena_state = std.heap.ArenaAllocator.init(gpa);
+ defer arena_state.deinit();
+ const arena = arena_state.allocator();
+
+ var tw: Io.Writer.Allocating = .init(gpa);
+ defer tw.deinit();
+ var tracer: trace.Tracer = .{ .io = io, .gpa = gpa, .out = &tw.writer, .start = Io.Timestamp.now(io, .awake) };
+
+ var url_buf: [64]u8 = undefined;
+ const url = try std.fmt.bufPrint(&url_buf, "http://127.0.0.1:{d}/x", .{server.socket.address.getPort()});
+ var agent = mockAgent(gpa, arena, io, url);
+ agent.tracer = &tracer;
+ defer if (agent.codex_ws) |c| {
+ c.dead = true;
+ c.deinit(gpa);
+ agent.codex_ws = null;
+ };
+
+ const body = "{\"model\":\"gpt-5\",\"input\":[]}";
+ const t0 = nowMs(io);
+ const out = try agent_ws.postResponsesWs(&agent, body);
+ defer gpa.free(out);
+ try std.testing.expect(nowMs(io) - t0 >= mock.think_after_prose_ms - 100); // the compose really happened
+ try std.testing.expect(std.mem.indexOf(u8, out, mock.commentary_done_event) != null);
+ try std.testing.expect(std.mem.indexOf(u8, out, completed_event) != null);
+ try std.testing.expect(traced(&tw, "commentary, budget unchanged")); // prose did flow
+ try std.testing.expect(traced(&tw, "\"detail\":\"completed\""));
+ try std.testing.expect(!traced(&tw, "\"detail\":\"stall\""));
+}
+
// The control: same harness, same tightened budgets, a server that finishes.
// Proves the stall is the SERVER's silence, not the mock — and pins the notes.
test "#401 control: a mock that completes the response still finishes the turn cleanly" {