Skip to content

Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.

Retain interrupted retrieval failure and define prospective work-time budget

David Zhang5864e94db966
diff --git a/specs/jevgrep/README.md b/specs/jevgrep/README.md
index 2d1ee3e..6df794b 100644
--- a/specs/jevgrep/README.md
+++ b/specs/jevgrep/README.md
@@ -7,10 +7,10 @@ Jev classification through Vercel AI Gateway. This is the build plan; production
## Next Agent Prompt
You are implementing this spec. Status: **implementing**, updated **2026-09-26**.
-Next: collect the running Django attempt in the isolated
-[source placement experiment](assets/presentation-study.md), then grade/account
-and inspect model-visible evidence before following its registered continuation
-rule. Do not launch a replacement. Keep the production renderer and skill unchanged.
+Next: verify and review the prospective [agent-work timing protocol](assets/timing-protocol.md),
+then freeze the faithful production package under that protocol for Django first.
+The [source placement experiment](assets/presentation-study.md) is terminal and
+failed; its remaining nine cells stay unrun. Keep production renderer and skill unchanged.
The no-model counterfactual preserves all evidence while moving first-ranked
source inside the observed tool prefix. Real SQLite replays match the frozen oracle, and an 81-request
three-file replay proves both pipelines exceed the follow-up threshold and skip
@@ -20,8 +20,7 @@ failed the same Django regression at Sol $1.2353512; its remaining nine tasks st
unrun and the production skill is unchanged. Packet-visibility comparison is
complete: initial truncation hid selected source, but a later read and explicit
recognition preceded replacing the failing regression with a type-changing test.
-Do not infer another query-breadth fix. The new presentation candidate's Django
-attempt is the only active paid task.
+Do not infer another query-breadth fix. No paid task is active.
The frozen `installed-jg-cpython-parity-v1` cohort is complete and not accepted:
7/10 solves versus 8/10 baseline, six cost wins, and 35.27% lower full Sol cost.
[Confirmation evidence](assets/cpython-confirmation.md) owns all outcomes, trace
diff --git a/specs/jevgrep/assets/presentation-study.md b/specs/jevgrep/assets/presentation-study.md
index 7334864..88bb275 100644
--- a/specs/jevgrep/assets/presentation-study.md
+++ b/specs/jevgrep/assets/presentation-study.md
@@ -85,7 +85,41 @@ Version remains the unpublishable development checkpoint `0.0.0`; release
validation correctly rejects it. No release or promotion occurs. Preparation and
all ten no-call validations passed. All 179 installed files match the failed query
parent except the renderer bundle; this also verifies installed dependency bytes.
-The one Django attempt is running under
-`evals/runs/swebench/installed-jg-adjacent-source-v1/plan.json`. Collect its terminal
-receipt, official grade, complete bill and model-visible output before applying
-the continuation rule. Do not start a replacement attempt.
+The one Django attempt under
+`evals/runs/swebench/installed-jg-adjacent-source-v1/plan.json` is terminal and
+officially unresolved, with no evaluator/infrastructure error. The remaining
+nine cells stay unrun. No replacement attempt or production promotion occurs.
+
+## Retained failed result
+
+Full Sol cost is **$0.398754 across 21 generations**, versus the fixed $1.5055472
+baseline. It is not a cost win because the patch fails the required alteration
+behavior. Jev has 3,550 client calls and 3,541 priced responses, with known
+observed subtotal **$1.131532920**; the full total is unknown and excluded.
+
+The exact query was:
+
+> How are database column attributes propagated when relationship fields derive their database type from referenced fields, and where is this behavior tested?
+
+Retrieval ran for 702.456 seconds before Sol sent Ctrl-C, explicitly because of
+the 900-second task budget. The harness did not time out. The CLI returned exit
+130 and an explicitly incomplete packet: 213 files, 159 source blocks and 242,840
+bytes. SQLite ranked 35 with no selected excerpt or lead; neither its path nor
+its guard survived tool truncation. First-ranked ForeignKey source did survive.
+This different selection does not test the offline counterfactual's old packet.
+
+The additional pass made 1,352 recorded requests, each carrying the same 26
+evidence items across 13 paths: 44,730 source bytes, 49,017 compact JSON bytes.
+That is below the unchanged 64,000-byte feedback threshold. The agent then
+changed only ForeignKey metadata propagation and passed 13 field tests; it did
+not test actual database alteration or inspect SQLite afterward. It finished at
+770.921 seconds, leaving 129 seconds unused. Clock pressure is a demonstrated
+reason for retrieval cancellation, not proof it alone caused the narrow patch.
+
+The inner tool clipped 60,710 estimated tokens to 30,000, and the outer wrapper
+then clipped its serialized result to 10,000. Query, ranking, selection, feedback
+execution, cancellation and effective tool allowance all differ from the parent.
+No causal formatting benefit is established. Detailed trace/visibility evidence:
+`/tmp/jg-adjacent-source-django-comparison/`. The candidate remains experimental.
+The [future timing protocol](timing-protocol.md) addresses the unintended
+wall-clock criterion prospectively; this failed result is never rescored.
diff --git a/specs/jevgrep/assets/timing-protocol.md b/specs/jevgrep/assets/timing-protocol.md
new file mode 100644
index 0000000..edff7f3
--- /dev/null
+++ b/specs/jevgrep/assets/timing-protocol.md
@@ -0,0 +1,56 @@
+# Agent-work budget for cost and solve evaluation
+
+The user's target is official solve preservation and full Sol cost; elapsed time
+is diagnostic. A 900-second wall-clock harness guard became a performance
+criterion when Sol explicitly cancelled `jg` after 702 seconds to leave time for
+implementation. [The retained presentation trial](presentation-study.md) remains
+a failed result under its registered protocol. No old plan, attempt, grade or
+baseline is changed or rerun.
+
+## Prospective correction
+
+New plans use schema 3 and freeze an explicit timing policy: 900 seconds outside
+waiting for a standalone `jg` search, with a 24-hour wall-clock runaway guard.
+Only observed direct search-command intervals qualify. Auth, doctor, cache,
+help/version, unrelated commands and compound shell commands receive no credit.
+Concurrent unrelated commands count as work; multiple search waits cannot earn
+double credit. Completion, cancellation and terminal turns end credit. Receipts
+retain wall time, credited retrieval intervals and charged work time.
+
+The treatment prompt explicitly describes that implemented clock. Its remaining
+task, model, effort, skill, official evaluator, full-bill accounting and acceptance
+gate stay fixed. Baseline prompt bytes and all baseline evidence remain pinned.
+Old schema-2 studies use their archived runners; there is no compatibility shim
+that silently reinterprets them under the new rule.
+
+All ten saved baseline agents completed without timeout, in 74.527–596.433 seconds.
+They have no `jg` wait to exclude, so their recorded 900-second allowance remains
+900 seconds of work. Baselines are reused, never regenerated. Changing the
+treatment timing instruction remains a protocol difference and cannot be hidden
+when comparing studies or attributing any downstream outcome. It does not alter
+the requirement to preserve all eight baseline solves and achieve seven fully
+billed successful strict cost wins in one prospective cohort.
+
+## Next candidate and gate
+
+First verify the clock with real subprocess journeys in Docker: excluded search
+wait, counted ordinary work, cancellation and overlapping work, and both work
+and hard-wall termination. Retain existing baseline integrity, grading and
+billing checks. No paid attempt starts before those checks and review finish.
+
+Return to the faithful production package for the next frozen quality study:
+SHA-256 `cff8b8607308531c26ec8736c07102449d1b785c95421d826a95a9efd299439e`,
+with the original accepted skill apart from executable rename. Neither rejected
+query wording nor experimental presentation has earned promotion. Django runs
+first once, using the new protocol. A solved result below its immutable baseline
+permits the remaining nine cells in that same new plan; otherwise retain and
+diagnose. No outcome from prior studies contributes to its aggregate.
+
+This tests the restored architecture under the cost/quality objective. It is not
+evidence that the timing change repairs the previously observed test-substitution
+mistake, and no claim of superiority follows from parser or clock conformance.
+
+## Status
+
+Harness correction is isolated and under verification. No new paid study is
+prepared or running. Production CLI and canonical skill remain unchanged.