Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.
Retrieval research lessons
This document records historical experiments and their limits. The old spike's seven-of-ten cost-win target is retired; it is not a product acceptance rule. The current architecture defines version-to-version improvement, with official task completion primary and full Sol cost reported. The final research report summarizes the adopted changes and measured limits. The local research ledger retains raw attempt provenance. The earlier ledger preserves the original experiments; its acceptance decisions describe the rules used then.
Measure the completed coding task
A useful retrieval packet is an intermediate result. Smaller output, fewer searches, or higher classifier scores can be outweighed by additional reasoning, patch revisions, and tests. Count the full Sol Gateway bill per attempt, including failed attempts; exclude Jev charges and Jev tokens under the user's policy. Returned source still contributes to the coding agent's bill. Keep experimental spending separate from candidate task cost; the spend audit states its scope.
Reuse each task/model/harness baseline permanently. Freeze a candidate within a confirmation cohort, retain every planned outcome, and keep pilots separate. A lower-cost failed patch is not a successful cost win. Do not assemble a claimed winning strategy by choosing the cheapest result from different variants per task.
Selection and presentation are different decisions
A file can be worth checking without deserving its complete contents in initial context. Likewise, a declaration location can be useful even when its source does not pass the excerpt threshold. Optional ranked leads let the coding agent fill gaps without making every returned path a reading obligation. The unit-location cohort supports this design, but individual savings remain noisy.
Do not assume that more source improves research. The source-inclusion diagnostic distinguished several primary implementations from generic nearby declarations, yet its full packet was still broad. Small operator-selected groups are diagnostics, not a substitute for evaluating the entire returned packet and the agent's subsequent actions.
Bind code to the thing being classified
Jev requests use native objects and arrays passed to the AI SDK; code is a string
inside those objects. The SDK serializes the HTTP body. Byte-size checks and trace
files use JSON serialization, but the application does not pass a stringified
state to evaluate.
The three-shape comparison tested shared source with declaration pointers, shared source plus per-declaration source, and declaration source alone. Removing the shared blob improved several primary-block scores. It did not solve related-helper discovery, and the native confirmation lost a Django solve. Preserve the representation finding without claiming that it establishes a better end-to-end strategy.
Line numbers belong in local source-location metadata; IDs can bind classification questions to source objects. The ID comparison found similar scores overall, with some threshold-sensitive differences. Removing metadata is not proof of unchanged model behavior.
For exact questions and state fields, use the frozen request contract and verbatim diagnostic requests. Those saved diagnostic requests are not captures of native benchmark HTTP calls. Do not explain the implementation with invented implementation-versus-test question routing: the frozen candidate applies the same predicate to every declaration within a pass.
Keep selected source separate from surrounding context
Comments and local context preserve meaning, but automatic expansion into nearby methods can include code the classifier never selected. The stage audit found that rendering context and feeding expanded ranges into another expansion pass substantially increased output. Store selected ranges separately from rendered ranges. This is a design lesson; removing the expansion in a trial did not by itself establish an end-to-end win.
Respect repository eligibility rules before classifying content. The Git-ignore audit identified generated files in retrieval output that ordinary ripgrep discovery skipped. The corrected filter diagnostic removed those files while retaining tracked source. Its native pilot still cost more than baseline, so input cleanup is not a demonstrated cost optimization. Searching nested repositories under a non-Git computer root remains unvalidated.
The focused source-inclusion comparison also cautions against treating less output as progress on its own: removing implicit neighbor expansion reduced the native Django packet from 4,441 to 3,543 lines, but lost the official solve. Both packets were truncated in model-facing tool output. Fresh classifications and agent sampling differ between these single attempts, so neither the size difference nor the regression establishes a causal effect of expansion alone.
Seeing a file does not guarantee a correct patch
Django failures repeatedly read the relevant SQLite code but tested a combined type-and-collation change, missing a collation-only change. Retrieval recall alone cannot explain that failure. Conversely, the Matplotlib missing-helper case exposes a discovery limitation: exact symbol-reference questions can omit sibling APIs with the same failure mechanism. A conditional ownership diagnostic found that helper through a factual code predicate. The subsequent native ownership trial returned the helper in source and reading leads, yet Sol still changed only the query-named annotation constructor and failed the official check. This exposes a presentation question as well as discovery: a returned declaration may need a concise explanation of why its behavior matters. The evidence-label follow-up also failed: the helper and its label reached Sol, but Sol left it unchanged. Neither recall nor a brief rationale alone established a solve improvement.
The instrumented retrieval also recorded 137 failed declaration-call attempts. An incomplete packet with no excerpt is not evidence that Jev confidently rejected every declaration. Keep service failures separate from healthy negative decisions when interpreting retrieval experiments.
Official grading is the primary benchmark measure, not proof of all behavior. An additional compatibility check found changed behavior in an officially passing patch. Keep this evidence beside the official score rather than silently rewriting it. Pylint has a separate test-collection caveat; its raw non-passes must not be presented as demonstrated behavioral failures or reclassified as solves.
Implementation evidence and test examples serve different purposes
A primary implementation body can save a direct read. The primary-file candidate repeated its scikit-learn official cost win, but lost the Django baseline solve. Making implementation evidence more prominent is not enough to preserve quality. The failed Django packet was not truncated and included the relevant condition; using the full issue as the retrieval query did not materially improve supporting coverage.
An existing test need not already reproduce the complete defect to be useful. Its setup and assertions can help isolate the changed behavior. The test-example pilot surfaced the collation test read first in a previously successful trace; Sol then built a focused regression and passed officially at a small cost saving. That is provisional evidence: query and classification varied, and the frozen confirmation ended with five verified cost wins, three definite non-wins, and one interrupted attempt with unknown full cost; one task was unrun under its stopping rule. That experiment did not establish a replacement under the criteria used at the time.
Classifier questions must distinguish storing input from merely reading it. Paired source-shape checks did not fix false ownership labels on getters. An explicit state-write predicate reduced those false positives but made the desired helper threshold-sensitive. A precise question can trade recall for precision; do not equate clearer wording with an end-to-end improvement.
Limits on conclusions
All ten tasks have now been inspected and used in development. They span Python repositories, but are a small purposive sample with single fixed baselines. Repeated tuning on them is not untouched holdout evidence, and neither language generality nor whole-computer scaling has been established. A new product version must be compared with the preceding released version under the current architecture’s evaluation policy. Historical spike thresholds and supplemental compatibility checks do not decide promotion. Aggregate savings alone do not establish better task completion.