Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

tool results: inline up to 64 KiB

justrach8e40807ebd36
diff --git a/docs/adr/0251-tool-results-inline-up-to-64-kib.md b/docs/adr/0251-tool-results-inline-up-to-64-kib.md
new file mode 100644
index 00000000..4deb8895
--- /dev/null
+++ b/docs/adr/0251-tool-results-inline-up-to-64-kib.md
@@ -0,0 +1,33 @@
+# 0251. Tool results inline up to 64 KiB
+
+Status: accepted 2026-10-03. Raises the #440 handle threshold.
+
+## Context
+
+A tool result over the handle threshold reaches the model as a bounded
+preview plus a handle (#440). The threshold was 16 KiB. graff's own modules
+stop at 600 lines, which is 20-35 KB, so a whole-file read of an ordinary
+module came back as a handle showing only its first half. On real-repository
+fixes the model then paged the handle with `read_tool_result` or re-read line
+ranges: three to five extra round trips per task, each costing seconds of
+model latency.
+
+Paired runs on real-repository fixes with the threshold at 64 KiB showed no
+handle previews and no paging, fewer model calls and shorter tasks, and total
+input tokens did not grow, because fewer calls re-send less context.
+
+## Decision
+
+- The default threshold is 64 KiB. `GRAFF_TOOL_HANDLE_BYTES` still overrides
+ it.
+- Small windows are unchanged in practice: `effectiveThreshold` already clamps
+ the threshold to `Provider.perOutputCap()`, half the context window, so a
+ 32k-token model still gets about 16 KB inline.
+- The codedb guard's small-read exemption follows the same number: a read that
+ comes back whole is not a search and needs no codedb detour.
+
+## Consequences
+
+On large-window models one result can cost up to 64 KiB of context instead of
+16 KiB; repeat sends of it are cache reads. Results larger than that still
+become handles.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 51f8e4cc..a0a0ea29 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -277,6 +277,7 @@ record only when you need the evidence or the edge cases.
| [0248](0248-commentary-prose-does-not-tighten-the-stall-budget.md) | Commentary prose (the heads-up before a tool-call batch, which the server holds open until it releases the calls) counts as visible text but does not tighten the stall budget; final-answer prose and whitelisted tool-argument prose still do. Refines 0241. |
| [0249](0249-a-spec-task-ends-at-its-own-tests.md) | The work note no longer says a green public test is not the whole spec: it made the model write an extra verification script on every spec task without changing the pass rate. Spec tasks end at their own tests; the lean note is unchanged. Refines 0024. |
| [0250](0250-pr-publication-is-not-gated.md) | PR publication is not gated: no local-check, head-CI or claim-review preflight and no completion obligation; drafts complete like any task. Artifact claims and disclosure policy are unchanged. Supersedes 0120, 0123, 0126, 0133 and parts of 0100/0104. |
+| [0251](0251-tool-results-inline-up-to-64-kib.md) | Tool results inline up to 64 KiB (was 16 KiB); `effectiveThreshold` still clamps to half the context window, so small windows are unchanged. The codedb small-read exemption follows the same number. |
## When to write one
diff --git a/src/tool_handle.zig b/src/tool_handle.zig
index 2ec0f1a7..8994891d 100644
--- a/src/tool_handle.zig
+++ b/src/tool_handle.zig
@@ -42,17 +42,17 @@ pub const handles_dir = ".graff/tool-results";
/// by this same number, so one result costs at most one threshold of context
/// whatever its size.
///
-/// 16 KiB is ~4k tokens — still a constant, bounded price, but big enough that
-/// a typical search result or file read comes back INLINE. That matters more
-/// than the context it spends: every handle forces the model into a follow-up
-/// read, and a round-trip costs ~5-8s of model latency while the tokens cost
-/// cache-read prices on big-window models. Measured on K3 (1M window, 2026-08
-/// graff×kimi-cli benchmark): raising 4 KiB → 32 KiB cut runEval/providers
-/// wall time ~16% by removing 2-3 handle-chase round-trips per task; 16 KiB
-/// keeps half that win while staying safe on 32k-class windows. The caps this
+/// 64 KiB (ADR 0251) is big enough that a whole source file comes back INLINE:
+/// a 600-line module runs 20-35 KB, and at 16 KiB its second half became a
+/// handle the model then paged through, three to five extra round trips on a
+/// real-repository task. Every handle forces a follow-up read, and a round
+/// trip costs seconds of model latency while the tokens cost cache-read
+/// prices; fewer calls also re-send less context. Small windows are safe by
+/// construction: effectiveThreshold clamps this to Provider.perOutputCap(),
+/// half the window, so a 32k-token model still gets ~16 KB. The caps this
/// contract replaced were not constant: 128 KiB of bash stdout, 64 KiB of
/// codedb output, and up to 136 KiB of a single result at send time.
-pub const default_threshold_bytes: usize = 16384;
+pub const default_threshold_bytes: usize = 64 * 1024;
pub var threshold_bytes: usize = default_threshold_bytes;
/// Ceiling on what one process may leave in `handles_dir`. Every handle is a