Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.
TypeScript ai-sdkclaude-codeclicode-searchcodexcoding-agentscontext-retrievaldeveloper-toolsjevsemantic-searchtypescriptvercel-ai-gateway
docs: consolidate replacement cohort evidence and trace limits
David Zhangbeb3038d27e7
diff --git a/specs/jevgrep/README.md b/specs/jevgrep/README.mdindex 51e4693..e062eef 100644--- a/specs/jevgrep/README.md+++ b/specs/jevgrep/README.md@@ -7,15 +7,13 @@ Jev classification through Vercel AI Gateway. This is the build plan; production## Next Agent PromptYou are implementing this spec. Status: **implementing**, updated **2026-09-26**.-The work-clock v1 confirmation stopped after Docker's disk-space failure during-Astropy. That attempt is failed and recovered separately; two completed wins and-seven unstarted cells remain retained. Do not resume or replace its failed cell.-Docker is recovered and disposable build cache was cleared. The new full `installed-jg-cpython-work-clock-v2` study is prepared and all ten-no-call validations pass with identical package and protocol. Scikit-learn and Django have-official cost wins; Astropy is running. Continue the same sequential runner,-execute each cell once and grade/account all outcomes.-Never pool studies or rerun a baseline. [Timing protocol and evidence](assets/timing-protocol.md)-own the outage, recovered bills and prospective replacement rationale.+The active full study is `installed-jg-cpython-work-clock-v2`. Continue collecting+terminal grades and complete bills from its existing sequential runner, then+aggregate all ten outcomes. [Confirmation evidence](assets/work-clock-confirmation.md)+owns results and trace findings. Its identical package and protocol passed every+no-call validation. No candidate changes, replacement attempts or baseline reruns.+The earlier v1 study remains stopped after a documented Docker outage; its+recovered costs and two wins cannot be pooled into v2.Keep production retrieval and the canonical skill unchanged. The installedpackage matches the restored bundled-CPython candidate; only the prospectivediff --git a/specs/jevgrep/assets/timing-protocol.md b/specs/jevgrep/assets/timing-protocol.mdindex f355dfc..bb9aeb1 100644--- a/specs/jevgrep/assets/timing-protocol.md+++ b/specs/jevgrep/assets/timing-protocol.md@@ -174,17 +174,5 @@ and timing policy match v1 byte-for-byte where applicable. Proof:all 1,198 manifest hashes and the unchanged failed receipt before paid continuation.-V2's first cell, scikit-learn, officially resolved with zero infrastructure/errors.-Its complete 10-generation Sol cost is **$0.2414888** versus **$0.2944360**-baseline. Jev observed cost is complete at **$0.070983906** (277 client calls),-separately excluded from scoring. The same unconditional random-state validation-caveat applies to its patch. Django is running next; full acceptance remains open.---V2 Django officially resolved with zero infrastructure/errors. Its complete-35-generation Sol cost is **$1.2852270**, below **$1.5055472** baseline. Jev's-known subtotal is **$0.152654124**; full observed cost remains unknown and excluded-from scoring. The patch includes SQLite collation-only rebuilding, old/new-collation comparison for related columns, and MySQL nullability handling. Its-added regression still combines a type change and collation change; the official-evaluator supplies the same-type regression. Astropy is running next.+[Replacement confirmation](work-clock-confirmation.md) owns v2 results and paired+trace findings. This document retains the protocol and outage rationale.diff --git a/specs/jevgrep/assets/work-clock-confirmation.md b/specs/jevgrep/assets/work-clock-confirmation.mdnew file mode 100644index 0000000..50f2cd9--- /dev/null+++ b/specs/jevgrep/assets/work-clock-confirmation.md@@ -0,0 +1,63 @@+# Faithful package: replacement work-clock confirmation++Status: running; full-cohort acceptance remains unproven. The authoritative plan+and receipts are in `evals/runs/swebench/installed-jg-cpython-work-clock-v2/`.+[Timing protocol](timing-protocol.md) owns the prospective clock rule, the v1+Docker outage, retained interruption costs and replacement rationale. No v1+outcome contributes to this cohort. Saved baselines are never rerun.++The production archive, skill, archived harness, task inputs and all 179 installed+files match v1. Artifact proof: `/tmp/jg-work-clock-v2-freeze-proof.json`.+All ten no-call validations passed before execution. The package stays frozen;+this run changes no retrieval policy.++## Retained results++Sol is full task cost; Jev is separately reported and excluded. Pending results+are not wins, and observed Jev subtotals are not complete totals. This checkpoint+will be replaced by the final aggregate after all cells finish.++| Task | Official result | Sol | Baseline Sol | Jev observed API cost |+| --- | --- | ---: | ---: | --- |+| scikit-learn__scikit-learn-13124 | solved | $0.2414888 | $0.2944360 | $0.070983906 |+| django__django-15629 | solved | $1.2852270 | $1.5055472 | known $0.152654124; total unknown |+| astropy__astropy-13579 | solved | $0.3548522 | $0.4286310 | $0.105473382 |+| pydata__xarray-3305 | solved | $0.3180666 | $0.4937066 | known $0.174302688; total unknown |+| psf__requests-1142 | solved | $0.2466230 | $0.2685004 | known $0.030577680; total unknown |+| sympy__sympy-16792 | pending | pending | $0.5479750 | pending |+| pytest-dev__pytest-6197 | pending | pending | $2.3444580 | pending |+| sphinx-doc__sphinx-8638 | pending | pending | $1.0594724 | pending |+| matplotlib__matplotlib-26466 | pending | pending | $0.3957160 | pending |+| pylint-dev__pylint-4604 | pending | pending | $0.2836264 | pending |++## Paired trace findings++Django received 43 files without source excerpts and explicitly read SQLite before+patching. Its patch includes the same-type rebuild trigger, old/new collation+comparison and MySQL nullability handling. Own regression covers BigAutoField to+CharField, FK/O2O nullability and constraints; official tests supply same-type,+reversal and many-to-many coverage. Final broad tests: 283 passes, 29 skips.+MySQL execution was not tested. More shell reads than v1 accompany higher Sol+cost, but there is no isolated causal cost attribution. Evidence:+`/tmp/jg-work-clock-v2-django-comparison/`.++Astropy received 45 files and no excerpts. Its shorter source-read, reproduction,+patch and test sequence resembles the accepted spike; no concrete functional+defect was established. Own regression is one scalar low-level case. Focused tests+passed 41; broader tests passed 107, skipped seven and failed one because of the+same expired leap-second data seen in baseline/spike. The official evaluator+separately resolved the task. Evidence: `/tmp/jg-work-clock-v2-astropy-comparison/`.++Xarray received 17 files and five excerpts, retaining the decisive Variable source.+One patch and four focused passing tests avoided the original faithful run's shell+status error. New regression checks default attr removal and explicit retention;+coverage remains narrower than the spike's full-module run. Evidence:+`/tmp/jg-work-clock-v2-xarray-comparison/`.++Scikit-learn repeats the established compatibility caveat despite its official+solve: unconditional random-state validation rejects invalid seeds previously+ignored when shuffling is disabled. Official benchmark success does not establish+all behavior equivalent. No benchmark patch is silently repaired after grading.++These tasks fit the old wall-clock allowance too. Their improvements do not+establish a clock effect or determinism across independent live model trajectories.