Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.
TypeScript ai-sdkclaude-codeclicode-searchcodexcoding-agentscontext-retrievaldeveloper-toolsjevsemantic-searchtypescriptvercel-ai-gateway
Preregister isolated mechanism-first skill trial and correct release evidence
David Zhang91d24c19f5ad
diff --git a/specs/jevgrep/README.md b/specs/jevgrep/README.mdindex 246e968..2c9c065 100644--- a/specs/jevgrep/README.md+++ b/specs/jevgrep/README.md@@ -7,7 +7,8 @@ Jev classification through Vercel AI Gateway. This is the build plan; production## Next Agent PromptYou are implementing this spec. Status: **implementing**, updated **2026-09-26**.-Next: diagnose the Django regression with one isolated query-framing experiment.+Next: prepare and run Django first in the isolated+[mechanism-first skill experiment](assets/query-framing-study.md).The frozen `installed-jg-cpython-parity-v1` cohort is complete and not accepted:7/10 solves versus 8/10 baseline, six cost wins, and 35.27% lower full Sol cost.[Confirmation evidence](assets/cpython-confirmation.md) owns all outcomes, trace@@ -25,7 +26,8 @@ exercised corpus; task-quality acceptance failed and remains required.scikit-learn officially solved but exceeded baseline cost; Django was interruptedwith its patch and traces retained; seven cells never started. These supersededstudies remain stopped. The bundled-CPython cohort is terminal; no Sol run is-active. The isolated diagnostic makes only bounded Jev classification calls.+active. The completed query diagnostics made only bounded Jev calls. A separately+identified skill candidate is prepared for an actual Django task test.The changed-policy cohort `installed-jg-final-cohort-v1` was stopped and markedsuperseded, retaining completed outcomes and interrupted traces. It is not adiff --git a/specs/jevgrep/assets/python-runtime.md b/specs/jevgrep/assets/python-runtime.mdindex f5b3085..674f596 100644--- a/specs/jevgrep/assets/python-runtime.md+++ b/specs/jevgrep/assets/python-runtime.md@@ -73,8 +73,11 @@ whole runtime as Apache-only. Packaging owns those notices and their verificatio`evals/runs/swebench/installed-jg-cpython-parity-v1/plan.json` freezes all ten taskson the archive above and the existing Sol model/harness/baselines. Its exactarchive passed four offline installed checks on each of Linux amd64 and arm64,-and the native macOS smoke identified the same SHA-256. Release content validation-and all ten dry-run validations passed before paid execution. Current official+and the native macOS smoke identified the same SHA-256. All ten dry-run+validations passed before paid execution. Release-validator unit tests passed,+but the direct archive validation intentionally rejected development version+`0.0.0` before content checks. An earlier statement that this archive passed+release validation was incorrect; the recorded output confirms that rejection. Current officialresults and paired trace findings belong in the[cohort confirmation](cpython-confirmation.md). Superseded studies remainhistorical and contribute no outcomes to this cohort. Platform transcripts arediff --git a/specs/jevgrep/assets/query-framing-study.md b/specs/jevgrep/assets/query-framing-study.mdnew file mode 100644index 0000000..dad6ac5--- /dev/null+++ b/specs/jevgrep/assets/query-framing-study.md@@ -0,0 +1,66 @@+# Mechanism-first query experiment++The faithful bundled-CPython port remains unchanged. Its complete cohort missed+acceptance; [confirmation evidence](cpython-confirmation.md) owns those results.+This separately identified experiment tests the caller's initial query scope,+not a replacement parser, traversal strategy, threshold or source representation.++## Contract registered before the coding-agent run++Candidate branch commit: `99b6afefd0dfe33311a2c1139c67551a2d7e4478`.+Candidate package SHA-256:+`2b2b4712fcff4780d747063390bb31a460b7f368c260aefcce5dc5af9e254dcb`.+Only the packaged skill differs from the parent archive. Every other member's+payload and metadata is identical; proof lives in `/tmp/jg-mechanism-query-v1/`.+The production worktree's canonical skill is unchanged. Both archives remain+unpublishable development version `0.0.0`; release validation intentionally rejects+that version. No validation-only version shadow is substituted for the candidate.++Replace only the skill's instruction to describe symptoms, expected behavior and+reproduction clues with:++> Start by asking how the existing mechanism works and where it is tested. Leave the symptom, desired fix, and platform-specific restriction out of this first query; apply those constraints when interpreting the result and implementing.++This is a coupled change to query scope, not just tone. It contains no benchmark+names, implementation paths, missing conditions, or evaluator answer hints.+The original issue remains unchanged in the agent prompt, and every later skill+instruction remains byte-identical.++Prepare one new ten-task frozen plan, `installed-jg-mechanism-query-v1`, using the+same package installation process, registry, task snapshots, baseline prompts,+Sol model/effort, 900-second agent limit and official grading/accounting. Baselines+run zero times. Execute Django first, once. All internal repairs, tool calls and+tests performed by that attempt remain in its full bill. No interactive hints,+replacement attempts or mid-run candidate edits are allowed.++The first check is whether the actual initial query follows the new instruction+and provides useful implementation/test evidence. This diagnoses compliance; it+does not replace outcome measurement. Local outcome: official Django solve with a+fully accounted Sol bill, compared with its fixed $1.5055472 baseline and the+parent's failed $1.0015696 attempt. Jev and experimental research spend stay+separate from scored Sol cost, including incomplete provider observations.++If Django solves below baseline cost, continue the same frozen plan's remaining+cells without rerunning Django. If it still fails, retain the result and diagnose+before spending on the remainder. A solved but more expensive outcome is a+partial win to investigate, not grounds to discard the strategy automatically.+Promotion still requires the original full-cohort gate: preserve all eight baseline+solves and reach at least seven successful lower-cost solves. Never pool outcomes+from different candidates. A focused result or classifier score cannot complete+the product goal.++## Evidence and parameter-effect map++| Hypothesis | Controlled observation | Verdict / next discriminator |+| --- | --- | --- |+| Query content affects declaration selection | Same sklearn source/questions: current query 0.26, spike query 0.50, neutral mechanism query 0.69 | Supported as query sensitivity in one observation each; no task-quality claim |+| Neutral wording alone fixes Django selection | Same SQLite method/source/questions: current 0.09, neutral same concepts 0.06 | Not supported |+| Broader mechanism scope admits relevant missing context | Removing collation and MySQL focus together raises that Django method to 0.52 | Coupled scope effect in one observation; individual contributions and downstream value unknown |+| Initial mechanism-first skill improves the failed task | Candidate defined above; coding-agent result pending | Inspect actual query, packet, reads, patch, tests, official grade and complete bill |++Each diagnostic used three successful Jev calls, no SDK retries and a 60-second+diagnostic timeout, without changing production's timeout. Observed research Jev+cost: $0.000414750 for sklearn and $0.000318360 for Django. Exact preregistration,+requests, responses and unchanged-source proofs are in `/tmp/jg-query-sensitivity/`+and `/tmp/jg-django-query-sensitivity/`. These are development diagnostics on+known cases; they establish no untouched generalization.