Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

Preserve prompt retries after connection failures

Rach Pradhan613117eadfd3
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 12bfed27..65668c71 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,13 @@ The release workflow uses a tag's section here as its release notes (a
hand-written `docs/releases/<tag>.md` wins if present), so keeping this file
current is part of cutting a release.
+## v0.302.51
+
+- Retrying the same prompt after a connection failure works again, including after loading the failed conversation. Earlier work stays in history; reconnecting does not require a new chat.
+- Includes independent conversation forks for subagents and workflow fan-out (`fork_from`), with optional retained workers for later continuation.
+- Includes bounded workflow synthesis that returns earlier findings when the final report cannot complete.
+- Retains all changes from v0.302.50.
+
## v0.302.48
- oMLX is a built-in local provider (`omlx`, `127.0.0.1:8000`). Set its key with `graff key set omlx <key>` or `OMLX_API_KEY`. `graff models` lists the models it serves, `--model omlx` starts on the first one, `/effort` is sent as `reasoning_effort`, and it is billed as local (#1695).
diff --git a/build.zig b/build.zig
index d2a44ac2..9ec7da12 100644
--- a/build.zig
+++ b/build.zig
@@ -230,6 +230,9 @@ pub fn build(b: *std.Build) void {
const retry_cancellation_test = b.addSystemCommand(&.{ "python3", "scripts/test-retry-cancellation.py" });
retry_cancellation_test.addArtifactArg(exe);
test_step.dependOn(&retry_cancellation_test.step);
+ const offline_recovery_test = b.addSystemCommand(&.{ "python3", "scripts/test-offline-recovery.py" });
+ offline_recovery_test.addArtifactArg(exe);
+ test_step.dependOn(&offline_recovery_test.step);
const run_tool_budget_test = b.addSystemCommand(&.{ "python3", "scripts/test-run-tool-budget.py" });
run_tool_budget_test.addArtifactArg(exe);
test_step.dependOn(&run_tool_budget_test.step);
diff --git a/docs/adr/0301-failed-requests-settle-the-pending-turn.md b/docs/adr/0301-failed-requests-settle-the-pending-turn.md
new file mode 100644
index 00000000..9602a1cd
--- /dev/null
+++ b/docs/adr/0301-failed-requests-settle-the-pending-turn.md
@@ -0,0 +1,36 @@
+# 0301. Exhausted request retries settle the pending turn in history
+
+## Context
+
+A request that exhausts its transport or server retries returns an error while
+leaving its user prompt as the last message. The duplicate-turn guard correctly
+rejects consecutive host replays, but cannot distinguish that unanswered prompt
+from a new, intentional retry. After reconnecting, the same prompt can therefore
+return without making any model request. Saving and loading preserves the trap.
+
+## Decision
+
+At the shared request loop's terminal retry boundary, append a truthful assistant
+marker saying the model request did not complete and earlier work remains in the
+conversation. It is a historical failure boundary, not a fabricated answer or a
+claim that tools did not run. The next identical prompt is a new user turn, both
+in memory and after saving/loading. REPL, JSON and ACP use the same engine path.
+
+Keep bounded retries, authentication handling, cancellation, duplicate-host
+protection and provider selection unchanged. Do not stream the historical marker
+as an additional answer or restart the failed request automatically. Ordinary
+cloud inference still requires connectivity; existing local-model routes remain
+the way to perform inference without it.
+
+## Consequences
+
+A saved conversation retains the failed attempt and its preceding tool results,
+so reconnecting does not require restarting or replacing the conversation. The
+marker also covers exhausted throttling/server retries, which otherwise leave
+the same ambiguous trailing prompt. It is not a promise to resume a failed turn
+without a new prompt, and does not add connectivity probes or automatic fallback.
+
+`scripts/test-offline-recovery.py` drives actual ACP v1/v2 and JSON processes
+against a failing loopback service, restores that service, and retries the exact
+same prompt. ACP also reloads the failed checkpoint before retrying. The fixture
+uses fake credentials and does not change the host's network configuration.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index f18431c4..967e4d16 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -324,6 +324,8 @@ record only when you need the evidence or the edge cases.
| [0299](0299-tool-commands-launch-apps-in-the-background.md) | On macOS tool commands' `open` launches apps without activating them (a `-g` shim first on the tool PATH); `GRAFF_GUI_FOREGROUND=1` on a command is the explicit foreground opt-in. |
| [0300](0300-a-session-shuts-down-the-simulators-it-booted.md) | Shell commands that look like simulator work are bracketed by booted-device snapshots; devices a session booted are leased under `~/.codegraff/simulators/` and shut down when the last live leaseholder ends, never devices booted before. `GRAFF_KEEP_SIMULATORS=1` opts out. |
+| [0301](0301-failed-requests-settle-the-pending-turn.md) | Exhausted request retries leave a truthful failure boundary in history so an identical prompt can run after reconnecting, including after save/load; duplicate-host protection and bounded retries stay unchanged. |
+
## When to write one
Write an ADR when a decision is load-bearing and non-obvious: it was reached
diff --git a/scripts/test-offline-recovery.py b/scripts/test-offline-recovery.py
new file mode 100644
index 00000000..de559cfe
--- /dev/null
+++ b/scripts/test-offline-recovery.py
@@ -0,0 +1,77 @@
+#!/usr/bin/env python3
+"""A failed connection must not swallow the same prompt after reconnecting.
+
+Uses the retry fixture's loopback server, fake credentials and isolated HOME.
+No real network outage, credentials, or model service is needed.
+"""
+import importlib.util
+import json
+from pathlib import Path
+import sys
+import tempfile
+
+REPO = Path(__file__).resolve().parents[1]
+spec = importlib.util.spec_from_file_location('retry_fixture', REPO / 'scripts/test-retry-cancellation.py')
+fixture = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(fixture)
+
+
+class Client(fixture.Client):
+ def until(self, predicate, timeout=30):
+ return super().until(predicate, timeout=timeout)
+
+
+def exercise(binary, mode, reload_saved=False):
+ with tempfile.TemporaryDirectory(prefix='graff-offline-recovery-') as temp:
+ model = fixture.Model('transport')
+ client = None
+ try:
+ root = Path(temp)
+ client = Client(binary, root, model, mode)
+ identifier = client.begin()
+ if mode == 'json':
+ failed = client.until(lambda event: event.get('type') == 'error', timeout=30)
+ elif mode == 'acp2':
+ failed = client.until(lambda event: event.get('params', {}).get('update', {}).get('stopReason') == '_error', timeout=30)
+ else:
+ failed = client.until(lambda event: event.get('id') == identifier, timeout=30)
+ assert 'error' in failed, failed
+ assert 'network error:' in json.dumps(failed), failed
+ assert client.proc.poll() is None, 'connection failure killed the session'
+ before = len(model.requests())
+ assert before > 1, 'transient connection retries were bypassed'
+ if reload_saved:
+ saved = list((root / '.graff/sessions').glob('*.session.json'))
+ assert len(saved) == 1, saved
+ name = saved[0].name.removesuffix('.session.json')
+ loaded = client.call('session/load', {'sessionId': name, 'cwd': str(root), 'mcpServers': []})
+ assert 'result' in loaded, loaded
+ client.sid = name
+ model.recover()
+ followup = client.begin() # Deliberately identical to the failed prompt.
+ client.finish(followup)
+ assert len(model.requests()) == before + 1, 'reconnected prompt was lost or replayed'
+ assert client.proc.poll() is None, 'session exited after reconnect'
+ print(f'PASS {mode}/' + ('reload' if reload_saved else 'live') + ': same prompt recovers after exhausted connection retries', flush=True)
+ except BaseException:
+ stderr = Path(temp) / 'stderr'
+ if stderr.exists():
+ print(stderr.read_text()[-2000:], file=sys.stderr)
+ raise
+ finally:
+ if client:
+ client.close()
+ model.close()
+
+
+def main():
+ binary = Path(sys.argv[1] if len(sys.argv) > 1 else REPO / 'zig-out/bin/graff').resolve()
+ modes = sys.argv[2:] or ('acp1', 'acp2', 'json')
+ for mode in modes:
+ exercise(binary, mode)
+ if mode == 'acp1': # The v2 draft does not advertise session/load.
+ exercise(binary, mode, reload_saved=True)
+
+
+if __name__ == '__main__':
+ main()
diff --git a/src/agent_request.zig b/src/agent_request.zig
index 5a41a098..ede0d7cf 100644
--- a/src/agent_request.zig
+++ b/src/agent_request.zig
@@ -391,6 +391,7 @@ pub fn request(self: *Agent, tools_in: ?[]const u8) !std.json.ObjectMap {
const failure_kind = if (err == error.RateLimited) "rate limited (429)" else if (err == error.ServerError) "server error (5xx)" else "network error";
self.last_api_error = std.fmt.allocPrint(self.arena, "{s}: {s}{s}{s} (gave up after {d} attempts)", .{ failure_kind, @errorName(err), if (down.len > 0) " at " else "", down, max_attempts }) catch null;
self.last_request_write_failed = std.mem.eql(u8, @errorName(err), "WriteFailed");
+ try scratch.preserveFailedRequest(self);
if (telemetry.g_telem) |t| t.errorEvent("net", @errorName(err));
if (self.tracer) |tr| tr.api(self.label, self.sub, self.provider.model, 0, body.len, 0, 0, 0, true);
return error.ApiError;
diff --git a/src/agent_request_scratch.zig b/src/agent_request_scratch.zig
index 0f0ceb98..68afe2f3 100644
--- a/src/agent_request_scratch.zig
+++ b/src/agent_request_scratch.zig
@@ -23,6 +23,13 @@ pub fn reset(scratch: *std.heap.ArenaAllocator) void {
}
}
+/// A terminal request failure is a completed attempt, not an unanswered host
+/// replay. Keep a truthful boundary in history so an identical retry works,
+/// including after saving and loading the conversation.
+pub fn preserveFailedRequest(self: *Agent) !void {
+ try self.messages.append(try @import("messages.zig").textMessage(self.arena, "assistant", "[The model request failed before a complete response was received. Earlier work remains in this conversation. Retry when the connection or service is available.]"));
+}
+
/// Detached recaps are cosmetic: keep recovered transport details in the trace,
/// but do not surface them as raw worker chatter in the normal REPL.
pub fn showRecoveredTransportRetry(kind: run_budget_mod.CallKind) bool {