graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs
Python
tool results: inline up to 64 KiB
justrach8e40807ebd36
diff --git a/docs/adr/0251-tool-results-inline-up-to-64-kib.md b/docs/adr/0251-tool-results-inline-up-to-64-kib.mdnew file mode 100644index 00000000..4deb8895--- /dev/null+++ b/docs/adr/0251-tool-results-inline-up-to-64-kib.md@@ -0,0 +1,33 @@+# 0251. Tool results inline up to 64 KiB++Status: accepted 2026-10-03. Raises the #440 handle threshold.++## Context++A tool result over the handle threshold reaches the model as a bounded+preview plus a handle (#440). The threshold was 16 KiB. graff's own modules+stop at 600 lines, which is 20-35 KB, so a whole-file read of an ordinary+module came back as a handle showing only its first half. On real-repository+fixes the model then paged the handle with `read_tool_result` or re-read line+ranges: three to five extra round trips per task, each costing seconds of+model latency.++Paired runs on real-repository fixes with the threshold at 64 KiB showed no+handle previews and no paging, fewer model calls and shorter tasks, and total+input tokens did not grow, because fewer calls re-send less context.++## Decision++- The default threshold is 64 KiB. `GRAFF_TOOL_HANDLE_BYTES` still overrides+ it.+- Small windows are unchanged in practice: `effectiveThreshold` already clamps+ the threshold to `Provider.perOutputCap()`, half the context window, so a+ 32k-token model still gets about 16 KB inline.+- The codedb guard's small-read exemption follows the same number: a read that+ comes back whole is not a search and needs no codedb detour.++## Consequences++On large-window models one result can cost up to 64 KiB of context instead of+16 KiB; repeat sends of it are cache reads. Results larger than that still+become handles.diff --git a/docs/adr/README.md b/docs/adr/README.mdindex 51f8e4cc..a0a0ea29 100644--- a/docs/adr/README.md+++ b/docs/adr/README.md@@ -277,6 +277,7 @@ record only when you need the evidence or the edge cases.| [0248](0248-commentary-prose-does-not-tighten-the-stall-budget.md) | Commentary prose (the heads-up before a tool-call batch, which the server holds open until it releases the calls) counts as visible text but does not tighten the stall budget; final-answer prose and whitelisted tool-argument prose still do. Refines 0241. || [0249](0249-a-spec-task-ends-at-its-own-tests.md) | The work note no longer says a green public test is not the whole spec: it made the model write an extra verification script on every spec task without changing the pass rate. Spec tasks end at their own tests; the lean note is unchanged. Refines 0024. || [0250](0250-pr-publication-is-not-gated.md) | PR publication is not gated: no local-check, head-CI or claim-review preflight and no completion obligation; drafts complete like any task. Artifact claims and disclosure policy are unchanged. Supersedes 0120, 0123, 0126, 0133 and parts of 0100/0104. |+| [0251](0251-tool-results-inline-up-to-64-kib.md) | Tool results inline up to 64 KiB (was 16 KiB); `effectiveThreshold` still clamps to half the context window, so small windows are unchanged. The codedb small-read exemption follows the same number. |## When to write onediff --git a/src/tool_handle.zig b/src/tool_handle.zigindex 2ec0f1a7..8994891d 100644--- a/src/tool_handle.zig+++ b/src/tool_handle.zig@@ -42,17 +42,17 @@ pub const handles_dir = ".graff/tool-results";/// by this same number, so one result costs at most one threshold of context/// whatever its size.///-/// 16 KiB is ~4k tokens — still a constant, bounded price, but big enough that-/// a typical search result or file read comes back INLINE. That matters more-/// than the context it spends: every handle forces the model into a follow-up-/// read, and a round-trip costs ~5-8s of model latency while the tokens cost-/// cache-read prices on big-window models. Measured on K3 (1M window, 2026-08-/// graff×kimi-cli benchmark): raising 4 KiB → 32 KiB cut runEval/providers-/// wall time ~16% by removing 2-3 handle-chase round-trips per task; 16 KiB-/// keeps half that win while staying safe on 32k-class windows. The caps this+/// 64 KiB (ADR 0251) is big enough that a whole source file comes back INLINE:+/// a 600-line module runs 20-35 KB, and at 16 KiB its second half became a+/// handle the model then paged through, three to five extra round trips on a+/// real-repository task. Every handle forces a follow-up read, and a round+/// trip costs seconds of model latency while the tokens cost cache-read+/// prices; fewer calls also re-send less context. Small windows are safe by+/// construction: effectiveThreshold clamps this to Provider.perOutputCap(),+/// half the window, so a 32k-token model still gets ~16 KB. The caps this/// contract replaced were not constant: 128 KiB of bash stdout, 64 KiB of/// codedb output, and up to 136 KiB of a single result at send time.-pub const default_threshold_bytes: usize = 16384;+pub const default_threshold_bytes: usize = 64 * 1024;pub var threshold_bytes: usize = default_threshold_bytes;/// Ceiling on what one process may leave in `handles_dir`. Every handle is a