Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.
TypeScript ai-sdkclaude-codeclicode-searchcodexcoding-agentscontext-retrievaldeveloper-toolsjevsemantic-searchtypescriptvercel-ai-gateway
Record platform and full verification with unresolved spike parity
David Zhangf9854bfddec0
diff --git a/package.json b/package.jsonindex 4974785..eb7388d 100644--- a/package.json+++ b/package.json@@ -22,12 +22,13 @@"format": "oxfmt --write .","format:check": "oxfmt --check .","test:reference": "bash scripts/test-reference.sh",- "test:parser": "bash scripts/test-reference.sh node --experimental-strip-types --test test/parser/source.test.ts && bash scripts/test-installed.sh --test-name-pattern \"runtime has|parses Python\"",+ "test:parser": "bash scripts/test-reference.sh node --experimental-strip-types --test test/parser/source.test.ts && bash scripts/test-installed.sh --test-name-pattern \"runtime has|local commands|parses Python\"","test:filesystem": "bash scripts/test-reference.sh bun test packages/core/test/filesystem.test.ts test/retrieval.test.ts && bash scripts/test-installed.sh --test-name-pattern \"filesystem policy\"","test:installed": "bash scripts/test-installed.sh","test:e2e": "bun run test:installed","verify": "bun run check-types && bun run lint && bun run test && bun run test:installed",- "eval:swebench": "python3 evals/implementation/swebench/installed.py"+ "eval:swebench": "python3 evals/implementation/swebench/installed.py",+ "test:native": "node scripts/test-native.mjs"},"devDependencies": {"oxfmt": "0.57.0",diff --git a/scripts/test-installed.sh b/scripts/test-installed.shindex 026911b..7eada81 100755--- a/scripts/test-installed.sh+++ b/scripts/test-installed.sh@@ -48,4 +48,6 @@ node_args=(node --test --test-concurrency=1)if [[ -n "$pattern" ]]; then node_args+=(--test-name-pattern "$pattern"); finode_args+=(/test/installed.test.mjs)docker build "${platform[@]}" "${build_args[@]}" -f "$context_dir/test/runtime.Dockerfile" -t "$image" "$context_dir"-docker run --rm --network none --read-only --tmpfs /tmp:rw,nosuid,nodev,size=256m "${platform[@]}" "$image" "${node_args[@]}"+docker run --rm --network none --read-only --cpus 2 --memory 2g --pids-limit 256 \+ --cap-drop ALL --security-opt no-new-privileges \+ --tmpfs /tmp:rw,nosuid,nodev,size=256m "${platform[@]}" "$image" "${node_args[@]}"diff --git a/specs/jevgrep/README.md b/specs/jevgrep/README.mdindex 87fa50e..ae0b6db 100644--- a/specs/jevgrep/README.md+++ b/specs/jevgrep/README.md@@ -15,9 +15,13 @@ Requests trial officially solved and reduced Sol cost against uncapped, but stilcost more than baseline. See [source-budget decision](assets/source-budget-decision.md)and [the uncapped trace report](assets/installed-requests-checkpoint.md).-Slices 01–07 have their named evidence. Native macOS installed verification is-being completed independently. Remaining work: frozen cohort acceptance, supported-Linux architecture checks, release closeout, whole-spec review and close-spec.+Slices 01–07 have their named evidence. Native macOS and both Linux architectures have+focused [installed-runtime evidence](assets/runtime-verification.md). The full deterministic `verify` gate passes. Remaining work: diagnose the+spike-versus-production expansion difference, finish frozen cohort acceptance,+release closeout, whole-spec review and close-spec.+The spike expands already expanded context on its second pass; production does+not. Current helper/template tests do not establish full-pipeline parity. Diagnose+this with identical-source/answer replay while the frozen cohort stays unchanged.Never rerun saved baselines or pool candidates. Report Jev API cost separately;only full Sol task cost is scored. See [integrated verification](assets/integration-verification.md),[contracts](contracts.md), and [decision map](map.md).diff --git a/specs/jevgrep/assets/integration-verification.md b/specs/jevgrep/assets/integration-verification.mdindex f566494..bd54806 100644--- a/specs/jevgrep/assets/integration-verification.md+++ b/specs/jevgrep/assets/integration-verification.md@@ -98,3 +98,17 @@ tarball additionally passes installed runtime, source-budget and head checks.Maintained cohort accounting passes 14 Docker tests, including retained responsecosts when transport logs are missing. Missing coverage leaves total Jev costunknown while preserving its known subtotal; Sol scoring is unchanged.++## Full deterministic closeout gate++`bun run verify` passed on the integrated source-budget/native-harness branch:+62 Bun tests (402 assertions), 15 Node parser tests, three release-validator tests,+14 maintained runner tests, and all 20 installed journeys. Typecheck and lint also+passed. Installed containers now have CPU/memory/process ceilings and dropped+capabilities, matching the spec's bounded test-isolation requirement. The native+smoke uses the same portable journeys; the parser selector retains the split+local-command case so its earlier coverage is not lost.++This is deterministic correctness evidence. It does not resolve the known+spike-versus-production context-expansion difference or substitute for the live+cohort quality gate. See [runtime evidence](runtime-verification.md).diff --git a/specs/jevgrep/assets/runtime-verification.md b/specs/jevgrep/assets/runtime-verification.mdnew file mode 100644index 0000000..971f72e--- /dev/null+++ b/specs/jevgrep/assets/runtime-verification.md@@ -0,0 +1,31 @@+# Installed runtime verification++The frozen cohort archive has SHA-256+`98bcf47e7de30ad82c8b452a86b958505c3befe6512d00f98486101c6a9fc802`.+It contains the 1,500-byte source policy and canonical skill used by every+prospective treatment. Its production source and skill were byte-compared with+the committed implementation before removing the candidate worktree.++Native macOS Apple Silicon: Darwin arm64 25.6.0, Node 24.14.0. The exact archive+passed the two reused installed journeys: help/version/skill without credentials,+and a real Python search over three hierarchy branches through a loopback HTTP+Gateway fixture. npm installed outside the checkout with temporary HOME/XDG and+only Node available on the runtime PATH. Removing the packaged grammar made search+fail with exit 1 while local commands still passed. Independent Codex review found+no actionable issue. This proves the printed Node/macOS combination, not every+Node version on macOS.++Linux amd64: the exact same archive passed runtime isolation/keyless commands,+Python search and bounded-source preservation in the Node 22 Docker runtime.+The installed image has no checkout or Python/Bun/compiler prerequisites. The+corresponding Linux arm64 candidate journeys also passed during slice 07.++Reproduce native evidence with `node scripts/test-native.mjs --prebuilt PACKAGE`.+For Linux, set `JEVGREP_TEST_PLATFORM=linux/amd64` or `linux/arm64` and run+`scripts/test-installed.sh --prebuilt PACKAGE`. The default installed suite is+broader than this focused architecture check; its full closeout remains separate.++The npm-facing README was added after the cohort archive was frozen. It documents+setup and explicit skill installation; it does not change executable, grammar,+dependencies or skill. Final packaging validation must include this documentation+and exercise its exact archive before a later user-requested publication.diff --git a/specs/jevgrep/assets/source-budget-decision.md b/specs/jevgrep/assets/source-budget-decision.mdindex 2bbc797..765834d 100644--- a/specs/jevgrep/assets/source-budget-decision.md+++ b/specs/jevgrep/assets/source-budget-decision.md@@ -66,3 +66,26 @@ and $0.006980 respectively. Fewer turns and less generated text explain much ofthe saving against uncapped, but their causal connection to the byte cap remainsuncertain. Better test evidence is a possible future hypothesis, not an untestedchange folded into this frozen policy.++## Accepted spike comparison prompted by user review++The accepted spike's Requests treatment cost $0.2597596 versus $0.2685004+baseline (3.26% lower). Its stdout was 7,059 UTF-8 bytes, compared with 2,852+uncapped production and 2,183 capped production. It returned five relevant files,+reported incomplete discovery and a scoped `AGENTS.md` lookup, and supplied more+request construction, header preparation and surrounding method context. The+production packets supplied six paths, less source and no scoped guidance lookup.+These are actual packet differences, not proof of their causal contribution.++The production skill was also rewritten from the accepted skill: detailed+symbol-first reads, lookup reuse and explicit polling instructions were shortened.+Production traces include repeated guidance searches and broader follow-up reads.+All runs retain the faulty content-length method. The source-cap comparison+therefore does not establish parity with the winning spike; one can reduce cost+against the uncapped production run while still regress against the spike.++The causal split between changed representation, changed agent instructions,+live classification/query variation and Sol behavior is unresolved. This needs+an exact spike-versus-production trace/request comparison before attributing the+regression to normal variance or calling the production policy a reproduced win.+The active frozen cohort remains unchanged and must be reported in full.diff --git a/specs/jevgrep/choices.md b/specs/jevgrep/choices.mdindex c91ef79..bea8e9f 100644--- a/specs/jevgrep/choices.md+++ b/specs/jevgrep/choices.md@@ -165,3 +165,15 @@ cannot satisfy the cohort gate, and the runner has no baseline execution path.Gap: artifact layout and execution mechanics were delegated. Reach: results fromdifferent candidates cannot be pooled into acceptance; failed attempts and unknownbills remain visible. Verdict: sound. Confidence: high.++### Reuse installed command journeys for native macOS verification++When: supported-runtime verification. Install a tarball into a temporary npm+prefix, give its process a PATH containing only Node, and drive the same keyless+commands and Python HTTP search used in Docker. Filesystem and failure suites+remain Docker-only. The native result records its actual Node/macOS version.++Gap: the native smoke mechanism was delegated. Reach: portable assertions stay+shared while Linux tool-absence assertions retain their own Docker scope; a Mac+with Python installed cannot accidentally satisfy a runtime dependency through+PATH. Verdict: sound. Confidence: high.