Skip to content

graff — a fast agentic coding harness in Zig: multi-provider, MCP, workflows, DGM evolution loop, TS/Python SDKs

Python

ADR 0207: Jev file retrieval is not economical as a default

justrach12e342d2fbd9
diff --git a/docs/adr/0207-jev-file-retrieval-is-not-economical.md b/docs/adr/0207-jev-file-retrieval-is-not-economical.md
new file mode 100644
index 00000000..4d298ee9
--- /dev/null
+++ b/docs/adr/0207-jev-file-retrieval-is-not-economical.md
@@ -0,0 +1,68 @@
+# 0207. Jev file retrieval is not economical as a default
+
+Status: rejected 2026-09-27
+
+## Context
+
+Jev is a decision model (TypeSafe System One) that returns a probability for
+each yes/no question about a supplied state. It is billed at a published
+$0.042 per million input tokens and nothing for output, and graff already
+reaches it for `jev_effort`. jevgrep (`jg`) uses it to find the files that
+answer a plain-language repository question: it walks the tree and asks Jev
+about each directory, file and source chunk. The question was whether graff
+should retrieve files the same way, either as a `locate` tool or as a ranking
+stage after `codedb context`.
+
+We measured retrieval only, with no coding model in the loop. The cases are
+15 merged graff PRs that close an issue. The question is the issue title and
+body; the answer is the non-test `src/*.zig` files the PR changed. Each case
+ran on a checkout of the PR's base, so the fix itself could not leak into the
+index. Three arms ranked files:
+
+| Arm | Jev requests per question | Any answer file in top 5 | Answer files in top 5 | In top 8 |
+|---|---|---|---|---|
+| `codedb context` | 0 | 53% | 38% | 42% |
+| `codedb` candidates (≤40) reranked by Jev | ~20–40 | 73% | 54% | 56% |
+| Jev over every `src/` file | ~550 | 93% | 80% | 80% |
+
+Each Jev request carried the issue text and the file's header comments and
+declarations, about 900 input tokens. The full scan was the most accurate arm,
+but it is one request per file: about 0.5M input tokens (≈ $0.02) and roughly
+30 seconds with 16 concurrent requests for graff's ~550 source files. Cost and
+latency grow linearly with the file count, so a 5,000-file repository would
+need ≈ $0.20 and several minutes for each question. `jg` as shipped made about
+1,000 requests per question here and printed 150–244 "relevant" files
+(20–48k tokens of output), too much to hand a model unfiltered.
+
+The rerank arm is cheap, but its gain is bounded by `codedb`'s candidate set:
+in several cases no answer file was among the candidates, so reranking could
+not recover it.
+
+A cheap per-question retrieval call does not save money when the main model
+runs on a flat-rate plan, which is how graff's coding models are commonly
+used. There, Jev adds metered spend and up-front latency, and the benefit
+would have to come from fewer model calls or less exploration, which this
+measurement does not show.
+
+## Decision
+
+graff does not run Jev for file retrieval. There is no `locate` tool, no Jev
+stage in `codedb context`, and no jevgrep integration, on by default or behind
+a flag. `codedb` remains the retrieval path. `jev_effort` is unaffected.
+
+## Consequences
+
+Sessions pay nothing and wait for nothing beyond `codedb` to find files, and
+no source content is sent to a third-party classifier for retrieval.
+
+The measurement exposed weaknesses in `codedb context` for plain-language
+questions, which are worth fixing there instead: a task over 1,024 bytes is
+rejected outright, keyword selection on issue text often picks generic words,
+and the semantic lane was unavailable in every run.
+
+Revisit when any of these holds: a pruned traversal (directory decisions
+first, skipping rejected branches) reaches full-scan recall within about 100
+requests per question; Jev's price or latency drops by an order of magnitude;
+or a whole-task comparison shows fewer model calls or a higher solve rate per
+task with retrieval than without. A revisit must score whole tasks, including
+failed ones, against a single fixed baseline per task.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 33243a3b..85e4a36f 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -233,6 +233,7 @@ record only when you need the evidence or the edge cases.
| [0204](0204-kimi-platform-is-graff.md) | Kimi requests send `X-Msh-Platform: graff` and `graff/<version>` everywhere (OAuth and inference), never `kimi_code_cli`; a Kimi Code console key bills as plan quota. Supersedes 0027. |
| [0205](0205-subagent-progress-is-standard-tool-content.md) | By default a subagent streams its work onto the parent tool call as content-only `tool_call_update` rows (rolling log + `_meta["graff/subagent"]`); the draft child-session stream only when both sides opt in; `GRAFF_ACP_SUBAGENT_PROGRESS=0` turns it off. |
| [0206](0206-acp-v2-preview-behind-a-gate.md) | ACP v2 draft shapes behind `GRAFF_ACP_V2=1` plus a client `protocolVersion` >= 2; v1 stays the default and unchanged. Prompt is acknowledged with `messageId` on insertion, the outcome travels as `state_update`. |
+| [0207](0207-jev-file-retrieval-is-not-economical.md) | Rejected: no Jev file retrieval (`locate` tool, Jev rerank after `codedb context`, or jevgrep). A full Jev scan found answer files best (80% in top 5 vs 38% for `codedb context`) but costs one request per file and grows with repo size; `codedb` stays the retrieval path. |
## When to write one