Skip to content

Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.

docs: close verified provider support specification

David Zhang05229ca7c6d0
diff --git a/docs/architecture.md b/docs/architecture.md
index e17811a..c5e9521 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -102,7 +102,9 @@ SDK handles HTTP serialization. Keep provider authentication and model access at
the core boundary so future providers do not reshape traversal or stdout.
The CLI auth module owns one saved provider/key. The core preset owner resolves
its endpoint/model; the evaluator owns retries and answer-cache identity. Provider
-selection never changes traversal or source rendering.
+selection never changes traversal or source rendering. See the
+[provider support record](../specs/done/provider-support/README.md) for the setup
+constraints and transport preservation evidence.
Acceptance measures the entire downstream Sol task: an official solve at a lower
full task cost than its fixed baseline, while preserving baseline solves. Jev
diff --git a/specs/done/provider-support/README.md b/specs/done/provider-support/README.md
new file mode 100644
index 0000000..7e68bbd
--- /dev/null
+++ b/specs/done/provider-support/README.md
@@ -0,0 +1,84 @@
+# Saved-provider support
+
+Jevgrep supports Vercel AI Gateway, native TypeSafe and OpenRouter through one
+TypeSafe-compatible AI SDK adapter. The implementation landed in `9496540` after
+the protocol checkpoint `06daef9`; it has not been published to npm.
+
+## Why selection belongs in auth
+
+Coding agents should research code without managing provider routing. A human
+chooses a provider and key during `jg auth`, and ordinary search and doctor use
+that saved record until auth replaces it. This avoids ambiguous environment-key
+precedence and accidental provider fallback. Automation has one explicit path:
+`jg auth --provider NAME --stdin`.
+
+The only compatibility exception is deliberate: a saved key without a provider
+still means Vercel. Reading it does not migrate or rewrite the file. Environment
+credentials are no longer a product input; users who relied on them need auth.
+This is a simpler setup contract, not a multi-account credential registry.
+
+## What must stay true
+
+- Provider changes select transport, not retrieval policy. Structured state,
+ question wording/order, hierarchy, thresholds, excerpts and search stdout keep
+ their frozen-reference contracts.
+- The application owns actual HTTP attempts, retries, shared rate-limit cooldown
+ and cancellation. SDK retries remain disabled. The 50,000-attempt guard protects
+ against runaway work; it is not an optimization budget.
+- Source freshness is checked before attempts and cached evidence use. An invalid
+ provider answer is failure or incomplete discovery, never negative evidence.
+- Exact answer-cache identity separates provider, endpoint, model and protocol,
+ as well as existing parser/prompt/policy identity. Old-wire answers cannot
+ certify the new route.
+- Auth saves one private record atomically and never verifies a key over the
+ network. Doctor checks separately with synthetic source. Keys never belong in
+ chat or CLI output.
+- Tests route requests outside the product. There is no public endpoint/model
+ override or provider flag on search/doctor, and no automatic fallback.
+
+The [preset owner](../../../packages/core/src/providers.ts),
+[evaluator](../../../packages/core/src/evaluator.ts), and
+[auth module](../../../apps/cli/src/auth.ts) hold the current mechanics.
+[Provider replay](../../../test/reference/provider-parity.test.ts) pins semantic
+requests and complete stdout; [installed journeys](../../../test/installed.test.mjs)
+exercise the packed product. The [preservation contract](parity.md) maps the other
+invariants to their checks. [Research](research.md) records the documented routes,
+model choices, pinned SDK and immutable inputs rather than relying on memory.
+
+## What the implementation taught us
+
+Changing the adapter and exposing saved-provider setup are one product cutover:
+shipping the picker first would allow a setup the old evaluator could not honor.
+The protocol was proved separately before that cutover.
+
+A test-only Node preload preserves the SDK's original JSON body while redirecting
+its fixed destination to a local fixture. Rebuilding the body as a Request stream
+obscured HTTP 401 in Node; retaining the original body fixed the test transport.
+The preload is not part of the npm package. Historical replay continues to use its
+original protocol; only production requests are strictly normalized for comparison.
+
+New benchmark plans freeze that preload and use schema 4. Archived studies retain
+their original runners; neither baselines nor historical results are rewritten.
+The [maintained runner](../../../evals/implementation/swebench/installed.py) and
+[broker](../../../evals/implementation/swebench/gateway_broker.py) remain Vercel-only
+for evaluation. They retain observed Jev billing separately from scored coding-agent
+cost, and unavailable totals remain unknown.
+
+## Evidence and limits
+
+[Final verification](assets/final-verification.md) records the exact candidate,
+passed gates and review. The same tarball passed 32 installed Docker tests and
+10 native macOS journeys. The three-provider replay preserves the frozen packet;
+this is transport evidence, not a new solve-rate or savings benchmark.
+
+Vercel and TypeSafe passed live doctor checks using isolated credentials from
+the user-provided local environment file. OpenRouter live access remains
+unverified because no key was available. No claim of identical live model probabilities, equal
+solve quality, fresh cost savings or faster retrieval follows from fixture tests.
+No npm release or paid agent benchmark was performed.
+
+The [auth record and complete packet](assets/auth-verification.md),
+[evaluator verification](assets/evaluator-verification.md), and
+[protocol reproduction](assets/protocol-verification.md) retain the review evidence.
+The [choices ledger](choices.md) explains the additional benchmark-format decision.
+There was no visual design or image requirement for this change.
diff --git a/specs/done/provider-support/assets/auth-verification.md b/specs/done/provider-support/assets/auth-verification.md
new file mode 100644
index 0000000..1d78e89
--- /dev/null
+++ b/specs/done/provider-support/assets/auth-verification.md
@@ -0,0 +1,38 @@
+# Saved-provider verification
+
+2026-09-26. Installed Docker journeys cover Vercel, TypeSafe and OpenRouter:
+auth from stdin, provider-specific doctor, search, cache reuse, replacement and
+no fallback. They check private storage, ignored environment keys/URLs, invalid
+providers/keys, legacy bytes and file identity remaining unchanged. The Docker
+PTY test selects OpenRouter, checks hidden entry, and cancels at both prompts;
+prior credentials survive and no temporary credential file remains.
+
+All 32 installed Docker tests and all 10 native macOS smoke journeys passed on
+the same candidate archive. The native run used Node v24.14.0, darwin/arm64 25.6.0.
+
+## Live check
+
+The actual built CLI ran this sequence with isolated config/cache directories:
+
+```text
+jg auth --provider vercel --stdin [key supplied privately; exit 0]
+jg doctor
+Jev connection verified through Vercel AI Gateway.
+
+jg auth --provider typesafe --stdin [key supplied privately; exit 0]
+jg doctor
+Jev connection verified through TypeSafe.
+```
+
+Both doctor commands exited 0 with empty stderr. The live probe used an authorized local key, never printed it, and removed its
+temporary saved credentials.
+The user supplied the two keys in `.env.local`. No OpenRouter key was available;
+its live service access remains unverified. No repository source was used in the live check.
+
+## Exact verified reference packet
+
+This is the complete frozen packet matched byte-for-byte by each provider's
+production replay; it is not a sample invented for documentation.
+
+[Read the exact stdout bytes](provider-stdout.txt). The text artifact preserves
+trailing spaces on numbered blank source lines that Markdown formatters remove.
diff --git a/specs/provider-support/assets/evaluator-verification.md b/specs/done/provider-support/assets/evaluator-verification.md
similarity index 100%
rename from specs/provider-support/assets/evaluator-verification.md
rename to specs/done/provider-support/assets/evaluator-verification.md
diff --git a/specs/done/provider-support/assets/final-verification.md b/specs/done/provider-support/assets/final-verification.md
new file mode 100644
index 0000000..542dd18
--- /dev/null
+++ b/specs/done/provider-support/assets/final-verification.md
@@ -0,0 +1,53 @@
+# Final provider-support verification
+
+2026-09-26. Implementation gates passed; no npm release or paid task benchmark
+was run. Existing baselines, historical study artifacts, the reference manifest,
+covered source files, corpus and frozen research skill remain unchanged.
+
+| Gate | Result |
+| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
+| `bun run verify` | Exit 0: types, lint, 125 core/CLI/reference, 18 Node protocol, 23 parser, 4 release, 27 maintained harness and 32 installed Docker tests |
+| `bun run format:check` | Exit 0 |
+| Latest schema-4 harness, local Python and Docker | 27/27 in each, exit 0; rerun after the final plan-version adjustment |
+| Exact candidate, installed Docker `--prebuilt` | 32/32, exit 0 |
+| Same candidate, native `--prebuilt` | 10/10, exit 0; Node v24.14.0, darwin/arm64 25.6.0 |
+| Independent whole-diff Codex review | No actionable defects; pending suite completion was not assumed by the reviewer |
+| Live built CLI `jg doctor`, isolated saved Vercel setup | Exit 0, expected success text, empty stderr |
+| Live built CLI `jg doctor`, isolated saved TypeSafe setup | Exit 0, expected success text, empty stderr |
+| Live OpenRouter | Unverified: credentials unavailable in the authorized local environment |
+
+Candidate: `@dzhng/jevgrep` source version 0.2.0, prepared with the existing package
+validator but **not published**. SHA-256:
+
+```text
+d8d7d0012da3163f2f615fddc01e07795251af28296950b5f248d990322bd6b2
+```
+
+The packaged file allowlist excludes the routing preload and fixtures; runtime
+requirements remain Node only. Docker runtime tests use network isolation and
+synthetic local HTTP; dependency downloads used Docker Desktop's existing proxy
+through a temporary Docker client config, without changing the user's settings.
+
+The broker's real local-upstream test verifies auth replacement, exact native
+request/response capture, and rejection before forwarding for wrong targets,
+credentials or bodies. Synthetic accounting retains 22 input tokens when cost
+is absent while the full Jev total remains unknown. Jev observations remain
+excluded from scored coding-agent cost. This proves maintained tooling behavior,
+not a newly measured solve rate, cost saving or latency.
+
+Review and cleanup preserved the one evaluator/one preset owner, removed the old
+module and environment setup path, retained exact-version license validation,
+and avoided a public transport override. The sole new data-format choice is
+recorded in [the choices ledger](../choices.md). No new benchmark cohort was
+prepared or executed.
+
+The user subsequently supplied TypeSafe and Gateway keys in `.env.local`. Both
+passed isolated live auth/doctor checks; neither key was printed or committed,
+and the user’s saved setup was not changed. OpenRouter availability was checked
+without printing values and no key was present.
+
+The fresh closeout audit corrected an evidence pointer, moved the exact packet to
+a raw text artifact to preserve trailing spaces, and added content-type validation
+before reference normalization. The strengthened transport/replay/provider-parity
+gate passed 15/15 in Docker. Removing the assertion in a temporary copy caused the
+new regression to fail at the expected point. The production tarball was unchanged.
diff --git a/specs/provider-support/assets/protocol-verification.md b/specs/done/provider-support/assets/protocol-verification.md
similarity index 96%
rename from specs/provider-support/assets/protocol-verification.md
rename to specs/done/provider-support/assets/protocol-verification.md
index 15d8236..3726b1c 100644
--- a/specs/provider-support/assets/protocol-verification.md
+++ b/specs/done/provider-support/assets/protocol-verification.md
@@ -35,8 +35,7 @@ that prototype was removed. No product transport workaround is needed.
The adapter archive includes its license. Its provider-utils 5.0.49 dependency
omits a license, like 5.0.45. The upstream repository LICENSE at tag revision
`ee3169b3c4880e2abe4d0d7c781243bb81822ec4` is byte-identical to the existing retained
-5.0.45 license. Slice 2 must extend the exact-version notice mapping before the
-adapter becomes part of the product bundle.
+5.0.45 license. The product notice owner now maps both verified versions to that retained license.
Reproduce with `node --test packages/core/test-node/provider-protocol.mjs`;
the same command runs inside `scripts/test-reference.sh`, and the normal suite
diff --git a/specs/done/provider-support/assets/provider-stdout.txt b/specs/done/provider-support/assets/provider-stdout.txt
new file mode 100644
index 0000000..28eaa4b
--- /dev/null
+++ b/specs/done/provider-support/assets/provider-stdout.txt
@@ -0,0 +1,40 @@
+Jevgrep: 3 relevant files.
+AGENTS.md lookup (root and returned-file ancestors): none found.
+- "src/backend/events.ts" — implementation, caller, test, fixture, helper; selected source and structural context below
+ Reading lead source: lines 1-1
+ Reading lead BackendTelemetry.recordEvent: lines 4-6
+- "src/telemetry.ts" — implementation, caller, test, fixture, helper; selected source and structural context below
+ Reading lead Telemetry.recordEvent: lines 3-5
+- "tests/telemetry.test.ts" — implementation, caller, test, fixture, helper; selected source and structural context below
+ Reading lead source: lines 1-1
+ Reading lead testEventName: lines 3-5
+End file list.
+
+Source block "src/backend/events.ts" lines 1-8:
+1: import { Telemetry } from '../telemetry';
+2: // Backends preserve the same event contract.
+3: export class BackendTelemetry extends Telemetry {
+4: recordEvent(name: string) {
+5: return super.recordEvent(name);
+6: }
+7: }
+8:
+
+Source block "src/telemetry.ts" lines 1-7:
+1: // Events preserve the caller's name.
+2: export class Telemetry {
+3: recordEvent(name: string) {
+4: return { name, recorded: true };
+5: }
+6: }
+7:
+
+Source block "tests/telemetry.test.ts" lines 1-6:
+1: import { Telemetry } from '../src/telemetry';
+2: // Regression example: do not rename the caller's event.
+3: export function testEventName() {
+4: return new Telemetry().recordEvent('opened').name === 'opened';
+5: }
+6:
+
+End context.
diff --git a/specs/provider-support/choices.md b/specs/done/provider-support/choices.md
similarity index 100%
rename from specs/provider-support/choices.md
rename to specs/done/provider-support/choices.md
diff --git a/specs/provider-support/parity.md b/specs/done/provider-support/parity.md
similarity index 63%
rename from specs/provider-support/parity.md
rename to specs/done/provider-support/parity.md
index 606b35e..88ec45f 100644
--- a/specs/provider-support/parity.md
+++ b/specs/done/provider-support/parity.md
@@ -5,18 +5,18 @@ The immutable input tree and historical hashes are identified in
manifest-covered files and prior studies byte-for-byte unchanged. The production
harness may change its setup/routing; it must not loosen the expected result.
-| Preserved behavior | Existing evidence | Owner and acceptance gate |
-| -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
-| Structured state, exact instructions and question IDs/order | `test/reference/requests.test.ts`, `corpus.json` | Slice 2: compare new-wire semantic requests to frozen corpus through built production CLI |
-| Hierarchy, thresholds, role/evidence/reference decisions | `test/reference/discovery-parity.test.ts`, `selection.test.ts`, `repository-context-parity.test.ts` | Slice 2: existing decision regressions remain green |
-| Full stdout, source selection/order and excerpts | `replay.test.ts`, `evidence-order.test.ts`, Python/comment parity tests | Slice 2: exact complete stdout plus existing ordering/parser tests |
-| Retries, cooldown, auth abort, timeout and attempt guard | `test/gateway.test.ts` | Slice 2: real native-wire HTTP failure matrix; rename test consistently if module renamed |
-| Source freshness on cache reads and every retry | `test/retrieval-freshness.test.ts` | Slice 2: mutation/deletion during wait cannot upload stale source |
-| Validated answer-only cache | `packages/core/test/cache.test.ts` and evaluator validation | Slice 2: same namespace hits; provider/endpoint/model/protocol differences miss; invalid/missing answers never cache as negative evidence |
-| Private atomic auth, cancellation and key validation | `apps/cli/test/auth.test.ts` | Slice 3: new/legacy records, replacement and failure paths; assert old bytes survive cancellation |
-| Installed runtime, ignores, signals, pipe closure, stdout-only | `test/installed.test.mjs`, `scripts/test-native.mjs` | Slices 2–3: actual packed CLI with controlled HTTP; all new provider cases selected by native gate |
-| Frozen research skill policy and installation | `test/reference/skill.test.ts`, release/skill tests | Slice 3: only setup/auth guidance changes, research body preserved |
-| Raw eval observations and honest missing billing | `evals/implementation/swebench/test_installed.py` | Slice 4: offline broker/receipt fixtures, frozen baselines/studies unchanged |
+| Preserved behavior | Existing evidence | Owner and acceptance gate |
+| -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
+| Structured state, exact instructions and question IDs/order | `test/reference/requests.test.ts`, `corpus.json` | Evaluator: compare new-wire semantic requests to frozen corpus through built production CLI |
+| Hierarchy, thresholds, role/evidence/reference decisions | `test/reference/discovery-parity.test.ts`, `selection.test.ts`, `repository-context-parity.test.ts` | Evaluator: existing decision regressions remain green |
+| Full stdout, source selection/order and excerpts | `replay.test.ts`, `evidence-order.test.ts`, Python/comment parity tests | Evaluator: exact complete stdout plus existing ordering/parser tests |
+| Retries, cooldown, auth abort, timeout and attempt guard | `test/evaluator.test.ts` | Evaluator: real native-wire HTTP failure matrix |
+| Source freshness on cache reads and every retry | `test/retrieval-freshness.test.ts` | Evaluator: mutation/deletion during wait cannot upload stale source |
+| Validated answer-only cache | `packages/core/test/cache.test.ts` and evaluator validation | Evaluator: same namespace hits; provider/endpoint/model/protocol differences miss; invalid/missing answers never cache as negative evidence |
+| Private atomic auth, cancellation and key validation | `apps/cli/test/auth.test.ts`, `apps/cli/test/process.test.ts` | Auth: new/legacy records, replacement and failure paths; assert old bytes survive cancellation |
+| Installed runtime, ignores, signals, pipe closure, stdout-only | `test/installed.test.mjs`, `scripts/test-native.mjs` | Installed CLI: actual packed CLI with controlled HTTP; all new provider cases selected by native gate |
+| Frozen research skill policy and installation | `test/reference/skill.test.ts`, release/skill tests | Auth: only setup/auth guidance changes, research body preserved |
+| Raw eval observations and honest missing billing | `evals/implementation/swebench/test_installed.py` | Harness: offline broker/receipt fixtures, frozen baselines/studies unchanged |
## Permitted differences
diff --git a/specs/provider-support/research.md b/specs/done/provider-support/research.md
similarity index 82%
rename from specs/provider-support/research.md
rename to specs/done/provider-support/research.md
index c8578ef..c925449 100644
--- a/specs/provider-support/research.md
+++ b/specs/done/provider-support/research.md
@@ -1,7 +1,7 @@
# Provider contract evidence
Inspected 2026-09-26. These are documentation/source observations, not live service
-results. The implementation starts with a small adapter reproduction, not another
+results. Verification uses a small adapter reproduction, not another
retrieval experiment.
## Fixed presets
@@ -19,13 +19,13 @@ OpenRouter chat completions or its separate `/api/alpha/decisions` API.
Versioned native/OpenRouter names do not prove identical backends or answers to
Vercel's alias. Preserve existing thresholds; do not claim cross-provider solve
quality from transport fixtures. Record actual returned model identity in external
-probe evidence when available. A conflicting live API result goes into this plan
+probe evidence when available. A conflicting live API result goes into this record
before changing a preset; never silently try other providers/models.
## Adapter identity and behavior
-Use `@ai-sdk/[email protected]` with the existing `[email protected]`, subject to the
-published-package reproduction gate in slice 1. Registry metadata inspected:
+Use `@ai-sdk/[email protected]` with the existing `[email protected]`, verified by the
+[published-package reproduction](assets/protocol-verification.md). Registry metadata inspected:
```text
dist.integrity = sha512-nP5NJGCwZ7jDE2dImurTU9igbj1OUeFAXLvZKcM05KlBpLxhOddBe3sMftAhQq3Zq+B8NqLx1dcW4vQVD2l32w==
@@ -50,18 +50,18 @@ the maintained broker's raw-response decoder, not reasons to add CLI telemetry.
Native TypeSafe billing may be absent. Unknown totals remain unknown; observed
Jev billing stays separate from scored coding-agent task cost.
-The adapter's error classes differ from Gateway's. Slice 1 reproduces HTTP status,
-timeout, disconnect, and malformed-response behavior; slice 2 owns the mapping to
-existing retry/split semantics. Primary [TypeSafe API documentation](https://docs.typesafe.ai/api)
+The adapter's error classes differ from Gateway's. The [protocol checks](assets/protocol-verification.md) reproduce HTTP status,
+timeout, disconnect, and malformed-response behavior; the evaluator maps those
+to the existing retry/split semantics. Primary [TypeSafe API documentation](https://docs.typesafe.ai/api)
and the Vercel guide describe native errors; fixtures must use their shapes.
## Immutable preservation baseline
Repository input: `ca7054ed4c50283030102d15cd73d816e85bc569`. This Git tree pins the
-current production implementation, lockfile, policy, parser versions, source tree,
+pre-change production implementation, lockfile, policy, parser versions, source tree,
skill, test harness, and previously recorded evidence. The additional historical
-source hashes are owned by [the reference manifest](../../test/reference/manifest.json).
-Keep that manifest, its covered files, [the corpus](../../test/reference/corpus.json),
+source hashes are owned by [the reference manifest](../../../test/reference/manifest.json).
+Keep that manifest, its covered files, [the corpus](../../../test/reference/corpus.json),
and all historical benchmark results unchanged.
The corpus compares a request multiset: independent network arrival order is not
@@ -70,7 +70,7 @@ selection/evidence ordering, and complete rendered stdout are invariants. Keep
existing decision/order regressions alongside semantic wire comparison; never
serialize production concurrency to make a replay easier.
-The [corrected repeat](../done/jevgrep/assets/variance-repeat.md) is historical
+The [corrected repeat](../jevgrep/assets/variance-repeat.md) is historical
evidence, not a new-provider acceptance result. Baselines are immutable and are
never rerun for this feature. Neither equal success rate nor fresh savings is
-established by this plan.
+established by transport checks.
diff --git a/specs/provider-support/README.md b/specs/provider-support/README.md
deleted file mode 100644
index ba0beb0..0000000
--- a/specs/provider-support/README.md
+++ /dev/null
@@ -1,145 +0,0 @@
-# Saved-provider support
-
-Add native TypeSafe and OpenRouter to `jg`, using one TypeSafe-compatible AI SDK
-adapter for all three services. Choose the provider once during authentication;
-ordinary searches keep the same interface and retrieval behavior.
-
-## Next Agent Prompt
-
-Implementation is complete, last updated 2026-09-26. All four slices and required
-checks pass; see [final verification](assets/final-verification.md). Close and
-archive this spec after the final claim audit. TypeSafe/OpenRouter live access
-remains unverified because credentials were unavailable; offline support is
-verified and Vercel passed a live doctor check. No npm publication is selected.
-
-You have no unresolved product question. Published-package behavior and live
-provider access need verification, not invented compatibility code. Missing
-credentials must not block offline implementation; report the missing live check
-honestly. Do not collect secrets in chat or overwrite the user's own saved setup
-for tests. Use isolated HOME/config/cache locations.
-
-- [x] [1. Protocol reproduction and controlled routing](slices/01-protocol.md)
-- [x] [2. Shared evaluator and production parity](slices/02-evaluator.md)
-- [x] [3. Saved-provider auth, installed journeys, and onboarding](slices/03-auth.md)
-- [x] [4. Maintained eval routing and final verification](slices/04-tooling.md)
-
-Update this prompt and the owning slice with evidence, remaining work and any
-deviations before ending each pass. Review each implementation slice before
-committing. Keep the checkout clean on main; do not create orphan worktrees or
-archive branches. There is no new benchmark run or npm publication in this plan.
-
-## Roadmap and review surfaces
-
-```mermaid
-flowchart LR
- A[1: Prove SDK wire contract] --> B[2: Replace evaluator and prove parity]
- B --> C[3: Choose and save provider in auth]
- C --> D[4: Repair maintained eval routing and close checks]
-```
-
-| Slice | Question answered | Reviewable result |
-| ----- | ---------------------------------------------------------- | ----------------------------------------------------------------------------- |
-| 1 | Does the pinned adapter implement the documented protocol? | Synthetic HTTP transcript and pass/fail probe |
-| 2 | Can the new transport preserve existing retrieval? | Built CLI reproduces the frozen corpus; failure and cache tests |
-| 3 | Does one setup reliably select the service? | Installed Docker auth → doctor → search for every provider, plus native smoke |
-| 4 | Does maintained tooling still observe the actual product? | Offline broker/receipt fixtures and final verification record |
-
-These are CLI/library checkpoints, with text transcripts as review artifacts.
-There is no visual design or screenshot deliverable. Human feedback is welcome
-but does not block reversible implementation; decide from the specified gates
-and record the reasoning if no feedback arrives.
-
-## Binding product contract
-
-`jg auth` asks **provider first, then hidden key**. It saves one
-`{provider, apiKey}` record. Re-running auth replaces that setup; searches and
-doctor use it until replaced. No automatic provider fallback, account registry,
-logout, or provider-switch command.
-
-For automation, use `jg auth --provider <vercel|typesafe|openrouter> --stdin`.
-The provider is mandatory with stdin. As a simplicity decision for this plan,
-`--provider` is accepted only together with `--stdin`; interactive auth always
-shows the picker. No provider/model/endpoint options on search or doctor.
-
-Credentials come only from the saved record. Provider API-key, endpoint and model
-environment variables cannot override it; XDG storage locations remain supported.
-Auth saves without a network call; doctor separately tests a synthetic question
-and identifies the saved provider. No secrets in stdout, exceptions or fixtures.
-
-**The sole required backward compatibility:** absence of the `provider` property
-means Vercel. Do not rewrite credentials during reads or require reauthentication.
-A present invalid value, including null, is an error. No migration command,
-format-version field, or preservation of environment-based authentication.
-
-The selected provider receives the same eligible repository content as before.
-Keep the existing head-friendly source packet, excerpts, stdout-only behavior,
-exit semantics, filesystem exclusions, optional parsers and skill research policy.
-No CLI report/trace files; only existing intentional auth/cache/skill persistence.
-
-## Single owners and scope boundaries
-
-| Concept | Owner |
-| ------------------------------------------------------------ | ------------------------------------------------------------------------- |
-| Fixed provider IDs, labels, endpoint/model/protocol identity | One small core preset module |
-| Saved key/provider, config paths, validation, atomic writes | CLI auth module |
-| HTTP attempts, retries, cooldown, answer validation | One evaluator using `@ai-sdk/typesafe-ai` |
-| Exact answer-cache storage | Existing cache module; evaluator supplies namespace |
-| Hierarchy, prompts, thresholds, batching, source freshness | Existing retrieval/request/source owners, unchanged |
-| Rendering and exit handling | Existing CLI owners |
-| Local HTTP redirection for verification | External test-owned Node preload, never packaged as product configuration |
-| Paid-call forwarding and raw accounting | Maintained benchmark broker/runner |
-
-No second evaluator, provider plugin registry, hand-written native client, or
-runtime protocol normalizer. Normalize transport only in tests. The intermediate
-Vercel-only CLI wiring in slice 2 disappears when slice 3 passes saved provider
-identity; it is not a compatibility layer to retain. Rename `gateway.ts` to
-`evaluator.ts` with its imports when replacing the implementation, leaving no
-forwarding module behind.
-
-Preserve the 50,000 **actual HTTP attempt** guard solely as runaway protection,
-SDK retries disabled, existing application retry/split policy, shared cooldown,
-cancellation and source freshness on every attempt. No performance tuning,
-prompt changes, new discovery strategy, parser changes, or benchmark optimization.
-
-## Acceptance and limits
-
-Each slice carries its focused gate. Final acceptance requires `bun run verify`,
-`bun run format:check`, and the native packaged smoke, with all-provider installed
-Docker journeys included in the normal suite. Freeze one candidate tarball and
-run it through both installed Docker and native checks, recording its SHA-256.
-Fixtures use actual SDK HTTP and installed CLI, not a mocked evaluator.
-
-Perform bounded synthetic live doctor checks for providers whose credentials are
-available through an authorized local source, using isolated saved records. Keep
-those results separate from offline checks; unavailable credentials mean live
-access is **unverified**, not failed or passed. A documented route that actually
-rejects its preset is a correctness issue to resolve before claiming support.
-
-Historical cost/quality claims retain their original scope. Jev cost is excluded
-from scored coding-agent task cost, but available observed billing is preserved
-separately; missing totals are unknown. No baseline reruns, personal-repo evals,
-fresh SWE-bench cohort, speed claim, release version selection, or npm tag push.
-
-## Plan audit
-
-Three blind read-only drafts informed this plan: a fewest-slices draft proposed
-one atomic change, a seam-focused draft proposed five stages, and Claude Opus's
-risk-first draft proposed eight. This plan takes their shared protocol-first
-gate, strict reference comparison and external routing seam. Four slices keep
-auth/retrieval/tooling independently reviewable without making docs or packaging
-separate workstreams. A single large slice would delay useful evidence; eight
-would leave too many small migration boundaries to maintain.
-
-Rejected alternatives: an internal baseURL override is unnecessary when fetch
-redirection is test-owned; a multi-provider benchmark runner expands this feature
-without helping product verification; interactive `--provider` adds a second way
-to skip the required picker. Keep only the stdin automation form. Package notices
-are part of the adapter/build gate, not deferred release cleanup.
-
-The interview was completed in the map, including the user's explicit legacy
-credential exception; no repeat permission or compatibility question is needed.
-The roadmap front-loads protocol evidence, then preserves the existing product
-before exposing additional auth choices. [Parity](parity.md) names each preserved
-behavior, its evidence and owning gate. [Research](research.md) distinguishes
-documented presets from unverified live access. Every slice names its remaining
-implementation discretion; a choice outside that list requires updating this plan.
diff --git a/specs/provider-support/assets/auth-verification.md b/specs/provider-support/assets/auth-verification.md
deleted file mode 100644
index e0a2cf5..0000000
--- a/specs/provider-support/assets/auth-verification.md
+++ /dev/null
@@ -1,80 +0,0 @@
-# Saved-provider verification
-
-2026-09-26. Installed Docker journeys cover Vercel, TypeSafe and OpenRouter:
-auth from stdin, provider-specific doctor, search, cache reuse, replacement and
-no fallback. They check private storage, ignored environment keys/URLs, invalid
-providers/keys, legacy bytes and file identity remaining unchanged. The Docker
-PTY test selects OpenRouter, checks hidden entry, and cancels at both prompts;
-prior credentials survive and no temporary credential file remains.
-
-All 32 installed Docker tests and all 10 native macOS smoke journeys passed on
-the same candidate archive. The native run used Node v24.14.0, darwin/arm64 25.6.0.
-
-## Live check
-
-The actual built CLI ran this sequence with isolated config/cache directories:
-
-```text
-jg auth --provider vercel --stdin [key supplied privately; exit 0]
-jg doctor
-Jev connection verified through Vercel AI Gateway.
-```
-
-Doctor exited 0 with empty stderr. The key came from the authorized local Duet
-environment, was never printed, and the temporary credentials were removed.
-TypeSafe/OpenRouter keys were unavailable there; live service access for those
-providers remains unverified. No repository source was used in the live check.
-
-A fresh smaller-model skill check correctly directed a missing-credential user
-to terminal `jg auth`, ignored an OpenRouter environment key, avoided asking for
-secrets, and continued ordinary discovery. The initial consultation was invalid:
-its instruction prohibited shell commands needed to read the skill. That was
-corrected before the fresh check; no product behavior was changed to fit it.
-
-## Exact verified reference packet
-
-This is the complete frozen packet matched byte-for-byte by each provider's
-production replay; it is not a sample invented for documentation.
-
-```text
-Jevgrep: 3 relevant files.
-AGENTS.md lookup (root and returned-file ancestors): none found.
-- "src/backend/events.ts" — implementation, caller, test, fixture, helper; selected source and structural context below
- Reading lead source: lines 1-1
- Reading lead BackendTelemetry.recordEvent: lines 4-6
-- "src/telemetry.ts" — implementation, caller, test, fixture, helper; selected source and structural context below
- Reading lead Telemetry.recordEvent: lines 3-5
-- "tests/telemetry.test.ts" — implementation, caller, test, fixture, helper; selected source and structural context below
- Reading lead source: lines 1-1
- Reading lead testEventName: lines 3-5
-End file list.
-
-Source block "src/backend/events.ts" lines 1-8:
-1: import { Telemetry } from '../telemetry';
-2: // Backends preserve the same event contract.
-3: export class BackendTelemetry extends Telemetry {
-4: recordEvent(name: string) {
-5: return super.recordEvent(name);
-6: }
-7: }
-8:
-
-Source block "src/telemetry.ts" lines 1-7:
-1: // Events preserve the caller's name.
-2: export class Telemetry {
-3: recordEvent(name: string) {
-4: return { name, recorded: true };
-5: }
-6: }
-7:
-
-Source block "tests/telemetry.test.ts" lines 1-6:
-1: import { Telemetry } from '../src/telemetry';
-2: // Regression example: do not rename the caller's event.
-3: export function testEventName() {
-4: return new Telemetry().recordEvent('opened').name === 'opened';
-5: }
-6:
-
-End context.
-```
diff --git a/specs/provider-support/assets/final-verification.md b/specs/provider-support/assets/final-verification.md
deleted file mode 100644
index 62afb05..0000000
--- a/specs/provider-support/assets/final-verification.md
+++ /dev/null
@@ -1,41 +0,0 @@
-# Final provider-support verification
-
-2026-09-26. Implementation gates passed; no npm release or paid task benchmark
-was run. Existing baselines, historical study artifacts, the reference manifest,
-covered source files, corpus and frozen research skill remain unchanged.
-
-| Gate | Result |
-| ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
-| `bun run verify` | Exit 0: types, lint, 125 core/CLI/reference, 18 Node protocol, 23 parser, 4 release, 27 maintained harness and 32 installed Docker tests |
-| `bun run format:check` | Exit 0 |
-| Latest schema-4 harness, local Python and Docker | 27/27 in each, exit 0; rerun after the final plan-version adjustment |
-| Exact candidate, installed Docker `--prebuilt` | 32/32, exit 0 |
-| Same candidate, native `--prebuilt` | 10/10, exit 0; Node v24.14.0, darwin/arm64 25.6.0 |
-| Independent whole-diff Codex review | No actionable defects; pending suite completion was not assumed by the reviewer |
-| Live built CLI `jg doctor`, isolated saved Vercel setup | Exit 0, expected success text, empty stderr |
-| Live TypeSafe/OpenRouter | Unverified: credentials unavailable in the authorized local environment |
-
-Candidate: `@dzhng/jevgrep` source version 0.2.0, prepared with the existing package
-validator but **not published**. SHA-256:
-
-```text
-d8d7d0012da3163f2f615fddc01e07795251af28296950b5f248d990322bd6b2
-```
-
-The packaged file allowlist excludes the routing preload and fixtures; runtime
-requirements remain Node only. Docker runtime tests use network isolation and
-synthetic local HTTP; dependency downloads used Docker Desktop's existing proxy
-through a temporary Docker client config, without changing the user's settings.
-
-The broker's real local-upstream test verifies auth replacement, exact native
-request/response capture, and rejection before forwarding for wrong targets,
-credentials or bodies. Synthetic accounting retains 22 input tokens when cost
-is absent while the full Jev total remains unknown. Jev observations remain
-excluded from scored coding-agent cost. This proves maintained tooling behavior,
-not a newly measured solve rate, cost saving or latency.
-
-Review and cleanup preserved the one evaluator/one preset owner, removed the old
-module and environment setup path, retained exact-version license validation,
-and avoided a public transport override. The sole new data-format choice is
-recorded in [the choices ledger](../choices.md). No new benchmark cohort was
-prepared or executed.
diff --git a/specs/provider-support/map.md b/specs/provider-support/map.md
deleted file mode 100644
index 7267892..0000000
--- a/specs/provider-support/map.md
+++ /dev/null
@@ -1,119 +0,0 @@
-# TypeSafe and OpenRouter support: decision map
-
-Completed explore-unknowns walk, 2026-09-26. Territory inspected at `3cd6dc8`.
-This preserves the exploration decisions. The [implementation spec](README.md)
-now owns sequencing and verification; no provider implementation or live testing
-is claimed here.
-
-## Known knowns
-
-Add native TypeSafe and OpenRouter while retaining Vercel AI Gateway. Use one
-TypeSafe-compatible AI SDK adapter for all three, selected through fixed provider
-presets. Keep structured state, question wording, thresholds, hierarchy, source
-selection and stdout behavior unchanged.
-
-The current [evaluator](../../packages/core/src/gateway.ts) owns requests, retries,
-timeouts, cooldown and validation; [retrieval](../../packages/core/src/retrieve.ts)
-owns source-freshness hooks. [Cache identity](../../packages/core/src/cache.ts)
-already has provider/model fields. [Authentication](../../apps/cli/src/auth.ts)
-currently saves one `{apiKey}` record, and [CLI wiring](../../apps/cli/src/index.ts)
-passes its key into both search and doctor.
-
-The official [AI SDK TypeSafe adapter](https://github.com/vercel/ai/blob/main/packages/typesafe-ai/src/typesafe-ai-evaluation-model.ts)
-maps boolean questions to native `noul` questions and maps answers back to
-probabilities. It appends `/systemone`. Documented compatible base URLs for that
-adapter are:
-
-| Provider | Base URL |
-| ---------- | ------------------------------------------ |
-| Vercel | `https://ai-gateway.vercel.sh/typesafe/v1` |
-| TypeSafe | `https://api.typesafe.ai/v1` |
-| OpenRouter | `https://openrouter.ai/api/v1` |
-
-Sources: [Vercel compatibility](https://vercel.com/changelog/ai-gateway-now-supports-typesafe-clients-and-http-api-for-jev),
-[TypeSafe API](https://docs.typesafe.ai/api), and
-[OpenRouter compatibility](https://openrouter.ai/blog/insights/what-is-jev/).
-These are documented contracts, not results from live calls in this walk.
-
-## Known unknowns resolved
-
-| Question | Decision and why | Closed by |
-| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- |
-| One or two adapters? | One TypeSafe-compatible adapter for all providers; configuration selects the service. Verify Vercel parity before replacement. | User |
-| Where is the provider selected? | `jg auth` asks for provider first, then key. Searches and doctor always use the saved provider. | User |
-| Store several accounts? | One record only. Re-running auth replaces provider and key; no logout or provider-switch command. | User |
-| Invocation overrides? | No provider, endpoint or model options on search/doctor. Use fixed presets. | User's simplicity constraint |
-| Environment keys? | Saved credentials only; do not implicitly read provider key or endpoint environment variables. XDG storage paths remain supported. | User |
-| Automation? | `jg auth --provider <vercel\|typesafe\|openrouter> --stdin` explicitly selects the provider and reads the key without prompting. Require provider for stdin setup. | Shown workflow |
-| Legacy credentials? | If the provider field is absent, interpret the saved key as Vercel without requiring re-authentication or rewriting the file during a search. Invalid present values are errors. | User correction |
-| Validate during auth? | Retain save-only auth and separate synthetic `jg doctor`; preserve the previous setup on cancellation or invalid input. | Existing contract |
-| Automatic fallback? | Never change providers after failure; retry the selected service under the existing bounded policy, then return honest failure/partial results. | User's fixed-provider constraint |
-| Credential storage? | Retain the same config location, atomic replacement and owner-only permissions; never echo keys. | Existing contract |
-
-New saved data has this shape (the key below is a placeholder):
-
-```json
-{ "provider": "openrouter", "apiKey": "<secret>" }
-```
-
-Legacy `{ "apiKey": "<secret>" }` means Vercel. No format-version field, stored
-model/URL, multi-account registry or credential migration command is needed.
-
-## Unknown knowns extracted
-
-The intended consumer is a coding agent after a human performs one simple setup
-on macOS or Linux. Provider flexibility belongs in setup, not every invocation.
-Existing authenticated users should keep working. The user explicitly narrowed
-the earlier no-backward-compatibility preference for providerless credentials.
-
-The reviewed interaction is: choose provider, enter a hidden key, save, optionally
-run doctor, then use ordinary `jg "question"` commands. Doctor identifies the
-saved provider in its result. Missing credentials should direct agents to human
-authentication or explicit stdin setup; the skill must stop recommending implicit
-`AI_GATEWAY_API_KEY` configuration. Agents continue ordinary discovery when setup
-is unavailable rather than collecting secrets in chat.
-
-## Unknown unknowns: findings and verification gates
-
-The sweep covered auth/parser/CLI wiring, evaluator/cache/retrieval, auth and HTTP
-tests, installed and reference fixtures, native smoke setup, benchmark runner and
-broker, package metadata, and README/skill onboarding.
-
-- **Protocol parity — sharp edge.** [Reference tests](../../test/reference/replay.test.ts)
- pin the old Gateway envelope and exact output. The new wire format moves the
- model into the body and uses `noul`. Normalize only these intentional transport
- differences; preserve state, instructions, question IDs/order, decisions and
- complete stdout. Keep frozen reference files and past results immutable.
-- **Retry/freshness coupling — sharp edge.** [Gateway orchestration](../../packages/core/src/gateway.ts)
- classifies some failures by Gateway-specific exception names. Verify equivalent
- handling for the new adapter's errors. SDK retries stay disabled, actual HTTP
- attempts count toward the 50,000 guard, and every retry still revalidates source.
- Preserve shared rate-limit cooldown, cancellation and sibling auth-failure abort.
-- **Cache crossover — decided.** Cache identity must include selected provider,
- effective endpoint, model and protocol/policy identity. Do not reuse old-wire
- Vercel answers as proof of new-wire behavior or mix providers' answers.
-- **Test and eval routing — sharp edge.** [Installed tests](../../test/installed.test.mjs),
- [native smoke](../../scripts/test-native.mjs), and the maintained
- [benchmark runner](../../evals/implementation/swebench/installed.py)/
- [broker](../../evals/implementation/swebench/gateway_broker.py) rely on Gateway
- environment credentials and its old HTTP path. Adapt their credential setup and
- controlled transport routing without adding public search overrides. Exercise
- all three presets and legacy credentials through the installed CLI. Never rerun
- baselines or edit old frozen study artifacts to make the new harness pass.
-- **Observed Jev billing — sharp edge.** The benchmark decoder currently reads
- `providerMetadata.gateway.cost`. A native response may have different or missing
- billing metadata. Preserve available raw observations and report unavailable
- totals as unknown, never zero. Jev remains excluded from scored agent task cost.
-- **Model identity and live behavior — OPEN verification.** Provider model names
- and aliases differ, and aliases can change their backing model. The subsequent [research record](research.md) resolves documented model IDs;
- slice 1 still verifies the published adapter response/error contract. Before claiming live support, run bounded synthetic doctor checks
- with each provider's credentials when available. No keys were read and no live
- model calls were made during this walk. Missing access must be reported, not
- replaced with a claim that fixtures prove service access or equal solve quality.
-
-## Superseded implementation handoff
-
-The earlier request to implement directly from this map is superseded by the
-[spec README and Next Agent Prompt](README.md#next-agent-prompt). Follow its slice
-order and preservation gates. Keep this map as user-decision rationale, not a
-competing build plan. No paid benchmark repeat or release version was selected.
diff --git a/specs/provider-support/slices/01-protocol.md b/specs/provider-support/slices/01-protocol.md
deleted file mode 100644
index 29c6eab..0000000
--- a/specs/provider-support/slices/01-protocol.md
+++ /dev/null
@@ -1,65 +0,0 @@
-# 1. Reproduce the provider protocol
-
-**Unlock:** prove the chosen SDK and controlled routing before touching retrieval.
-No dependency on another slice. Status: complete; see [verification](../assets/protocol-verification.md).
-
-## Seam and artifact
-
-Pin the adapter identified in [research](../research.md), inspect its published
-artifact, and use `experimental_evaluate` with structured synthetic state and two
-boolean questions. Send through the actual adapter to local HTTP fixtures for
-each documented preset. Keep the probe test-owned and add it to the ordinary test
-entry point; do not add a user-facing probe command or another native client.
-
-Keep `[email protected]` and the historical oracle's resolved dependency closure fixed.
-Adding the adapter may add its own provider-utils version; it must not silently
-upgrade the oracle through lockfile deduplication. Inspect the new dependencies'
-license files too: `scripts/package-notices.mjs` currently has an exact-version
-exception for `@ai-sdk/[email protected]`. Record whether the new artifact
-supplies its own license before changing that exception.
-
-The test transport seam is a Node `--import` preload outside the npm package.
-It observes the original fetch URL, allows only the three exact preset endpoints,
-then redirects to a local fixture preserving method, headers, body and abort
-signal. Unknown destinations fail closed. No production environment override,
-TLS bypass, or user config extension. For core tests the same narrow fetch
-redirection can be passed as a dependency. Test expectations remain independent
-of the production preset table so a wrong URL/model cannot validate itself.
-
-Save a concise `assets/protocol-verification.md` with package integrity, tested
-request/answer/error shapes and reproduction command. It contains synthetic data
-only. This is the first human review surface; no live credential is needed.
-
-## Verification and verdict
-
-- Assert exact endpoint suffix, model, bearer key, structured nested state,
- question IDs/order/instructions, boolean-to-noul conversion, and finite answer
- probability conversion. Inspect the pinned SDK's real response schema; provide
- required response fields rather than bypassing its validator.
-- Reproduce 401/403, 408, 409, 429 with both Retry-After forms, 5xx, timeout, socket
- disconnect, malformed JSON, missing and out-of-range answers. Record actual
- error status/name/cause available to the application. No assumption that old
- Gateway exception names survive.
-- Run through the current Node Docker reference harness. Prove cancellation
- reaches real HTTP and no hidden SDK retries occur when `maxRetries: 0`.
-- Prove the preload rejects a wrong original destination and that fixture
- redirects are not baked into the packed product. Keep original reference tests
- green. No live calls or benchmark execution is required for this verdict.
-
-Acceptance: a reproducible adapter contract for all presets and observed error
-classification inputs. If the pinned package disagrees with its docs, resolve
-that concrete mismatch and update the research record before slice 2. Do not
-redesign transport around an assumption or change prompts/thresholds.
-
-## Decision budget
-
-Delegated: probe/fixture filenames, internal helper names, concise transcript
-format, and the smallest compatible package pin correction if the inspected
-artifact fails the stated API contract (record exact reason/integrity). Presets
-are the documented choices, not an invitation to pick newer aliases. Human
-feedback can change presentation of the transcript; an API contradiction changes
-the plan. Proceed on passing evidence without waiting for sign-off.
-
-An adapter requiring an `ai` upgrade, clamping invalid probabilities, or changing
-request meaning fails this slice. Reslice that concrete conflict before proceeding;
-do not hide it with a permissive fixture or silently alter the frozen oracle.
diff --git a/specs/provider-support/slices/02-evaluator.md b/specs/provider-support/slices/02-evaluator.md
deleted file mode 100644
index 3020460..0000000
--- a/specs/provider-support/slices/02-evaluator.md
+++ /dev/null
@@ -1,76 +0,0 @@
-# 2. Replace the evaluator and prove production parity
-
-**Unlock:** one provider-neutral evaluator preserves the existing search contract.
-Depends on slice 1. Status: complete; see [verification](../assets/evaluator-verification.md).
-
-## API seam and ownership
-
-Add one core preset owner exposing a `ProviderId` union (`vercel`, `typesafe`,
-`openrouter`) and immutable label/baseURL/model/protocol records from research.
-Rename `gateway.ts` to `evaluator.ts` and update consumers directly. Keep
-`EvaluationRequest`, the retrieval `Evaluator` interface and failure kinds stable.
-
-`createEvaluator` takes explicit `provider: ProviderId`, validated `apiKey`, and
-the existing cache/policy/signal/attempt settings. Resolve URL/model internally;
-remove the old general baseURL option. A fetch dependency may exist for core
-testing, but CLI users get no transport override. Always pass the key to
-`createTypeSafeAi`; never depend on SDK environment discovery.
-
-At this checkpoint CLI search/doctor pass Vercel explicitly: public setup remains
-Vercel-only until slice 3. Remove `AI_GATEWAY_BASE_URL` wiring now, and move
-production test routing to slice 1's preload. Slice 3 removes the explicit Vercel
-selection when saved credentials become the shared input. Do not add a temporary
-provider flag, dual evaluator, wrapper module, or dispatcher registry.
-
-Keep one owner of actual attempts and retries. Cache identity includes provider
-ID, effective endpoint, requested model, a native protocol identity and existing
-policy/parser/prompt identities. Cache storage/schema/TTL stay unchanged. Old-wire
-entries naturally miss; no deletion or migration is necessary.
-
-## Gates and review surface
-
-Implement the failure mapping from the published SDK evidence, preserving the
-existing application's behavior rather than copying only old error names:
-
-- SDK `maxRetries: 0`; one fetch increments one request. Guard remains 50,000.
-- Existing normal/navigation retry limits, navigation transient split eligibility,
- special 429 handling and shared cooldown. Do not broaden retry policy incidentally.
-- Preserve the Node disconnect recovery checked by
- `test/reference/disconnect.test.ts`. Distinguish transport failures from invalid
- JSON/answers; do not classify every status-less exception as transient. Do not
- inherit SDK `isRetryable` wholesale (for example, navigation 409 must not gain
- a new retry/split merely because the adapter marks it retryable).
-- 401/403 abort in-flight siblings, cooldown waits and future attempts. User
- cancellation/pipe closure stops work. Retry-time source validation still runs
- after cooldown and before HTTP; invalid source never uploads again.
-- Missing/malformed/nonfinite/out-of-range answers remain failure/incomplete,
- never negative evidence. Only validated answers enter cache.
-- Prove same-identity cache reuse and misses across provider, endpoint, model and
- protocol identities without making those fields public CLI options.
-
-Apply [the parity contract](../parity.md) through the built production CLI as soon
-as the adapter is wired. Keep historical replay on its existing Gateway protocol;
-adapt only production setup and strict transport normalization. Keep the manifest,
-corpus, historical source and existing `replay.test.ts` assertions unchanged.
-Add provider-parameterized production parity cases without replacing the original
-Vercel comparison. Every provider must reproduce the same packet under identical
-controlled probabilities. Retain selection, source-order, comment and parser gates.
-
-Keep the package build green with the new adapter's actual bundled dependency
-notices. If a new upstream package omits a license, use a verified matching
-upstream license and record its provenance under the existing notices mechanism;
-never disable validation or apply the old version's exception indiscriminately.
-
-Run `bun run test:reference` plus the ordinary evaluator/cache/freshness tests and
-`bun run check-types`. Adapt installed Vercel fixture routing in this slice so
-`bun run test:installed` remains green; native fixtures share the same transport.
-Review the complete stdout packet and a request diff with only allowed transport
-changes. Record results in `assets/evaluator-verification.md`.
-
-## Decision budget
-
-Delegated: preset/type naming, error-inspection helper placement and cache protocol
-token spelling. Error classification must be justified by slice 1's observations;
-the retry policy itself is not delegated. No concurrency serialization, new
-retrieval logic or response truncation to pass parity. User feedback could change
-the chosen provider architecture; absent that, proceed on the frozen gates.
diff --git a/specs/provider-support/slices/03-auth.md b/specs/provider-support/slices/03-auth.md
deleted file mode 100644
index 1529721..0000000
--- a/specs/provider-support/slices/03-auth.md
+++ /dev/null
@@ -1,84 +0,0 @@
-# 3. Save a provider and complete the installed journey
-
-**Unlock:** users can choose any supported service once and use ordinary `jg`.
-Depends on slice 2. Status: complete; see [verification](../assets/auth-verification.md).
-
-## API and interaction
-
-The auth module owns `Credentials = {provider: ProviderId; apiKey: string}` and a
-single saved-credentials loader replacing `loadApiKey`. Core owns provider IDs and
-labels; auth owns config path/key validation and filesystem effects. Search and
-doctor pass the loaded record to the same evaluator. No separate resolution tree.
-
-Interactive `jg auth` uses the existing prompt library: select Vercel AI Gateway,
-TypeSafe or OpenRouter, then enter the provider's hidden key. Noninteractive auth
-requires `--provider` and `--stdin` together; reject unknown IDs, bare `--stdin`,
-bare `--provider`, and provider flags on every other command. Errors direct users
-to the right setup without printing a key. Both prompt and result use stdout.
-
-Retain the current config location and private atomic replacement. Save only
-`{provider, apiKey}` after both inputs validate; cancel/invalid input leaves the
-previous record untouched. Preserve the 8 KiB key/input bound, whitespace rules,
-0700 directory and 0600 file permissions. Check cancellation before replacement.
-No network during auth; successful text names the provider and points to doctor.
-
-Absent provider property normalizes in memory to Vercel. A present invalid value
-errors. Reads never rewrite bytes. Remove production API-key/endpoint/model env
-overrides and missing-key guidance recommending them. XDG paths continue working.
-
-Doctor retains its synthetic question, identifies the saved provider on success
-and actionable failure, and never uploads the user's repository. Search stdout
-and research policy remain unchanged. Reauth is the only supported way to replace
-the selected service; no per-query switching or failure fallback.
-
-## Installed acceptance
-
-Use real installed npm tarball binaries in the existing Node-only, network-isolated
-Docker harness. The preload redirects approved destinations to actual local HTTP;
-it does not fake the evaluator. All cases live in normal test entry points:
-
-- For each provider: stdin auth → inspect exact private record → doctor → search
- → repeated cache hit; replace auth with another provider and prove a new call
- using that provider/key. Auth itself makes zero calls.
-- Providerless legacy record: doctor/search use Vercel and file bytes stay intact.
- Invalid present provider fails before HTTP, including null and empty string.
-- Conflicting environment keys/URLs/models cannot override a valid record;
- environment-only setup cannot authenticate. Missing-credential fixtures use an
- empty isolated config directory, not an empty environment key. No saved key leaks in diagnostics.
-- Invalid key/provider, stdin overflow, prompt cancel and SIGINT preserve previous
- bytes and clean temporary credential files. Use a PTY for interactive provider
- ordering, hidden entry and cancellation; it is test tooling, not a runtime
- dependency. Exercise the noninteractive cases through the installed CLI too.
-- Retain existing filesystem, source freshness, partial failure, signal/EPIPE,
- stdout/stderr and skill-installer coverage; no alternate provider traffic after
- authentication/network failure.
-
-Expand the native smoke selector and asserted pass count so all new provider
-journeys actually execute. Copy the preload in normal/prebuilt Docker paths and
-native setup; keep it outside packaged files. The existing package must still run
-with Node alone, no host Python/Bun/rg/compiler requirement.
-
-## Onboarding and review
-
-Update root/packaged README, `docs/architecture.md`, help, `.env.example` if obsolete, and the shipped
-`skills/jevgrep/SKILL.md` setup section together. Explain provider-first auth,
-replacement, legacy saved-key behavior and removal of environment-only setup.
-Keep `jg skill` / `npx skills add dzhng/jevgrep --skill jevgrep` installation easy
-to find and CLI installation guidance intact. Never ask an agent to request a key
-in chat. Preserve the frozen research body, not merely its meaning.
-
-Human review surface: short auth/doctor transcript with fixture keys redacted and
-the complete search packet. Save verification in `assets/auth-verification.md`.
-Run auth/parser tests, `bun run verify`, format check and native packaged smoke.
-For live doctor use isolated saved records and authorized local credentials only;
-record provider/date/model when present and success/error without secrets. Clearly
-separate unavailable credentials from offline pass results. Missing live access
-does not block remaining implementation or authorize a provider fallback.
-
-## Decision budget
-
-Delegated: prompt wording, test PTY tooling and helper layout. Picker order is
-Vercel, TypeSafe, OpenRouter; no additional account UX. The documented automation
-grammar and saved-data compatibility are fixed. Human feedback can adjust wording
-or report a service incompatibility; passing offline evidence permits proceeding
-without waiting for approval.
diff --git a/specs/provider-support/slices/04-tooling.md b/specs/provider-support/slices/04-tooling.md
deleted file mode 100644
index 9853fe3..0000000
--- a/specs/provider-support/slices/04-tooling.md
+++ /dev/null
@@ -1,67 +0,0 @@
-# 4. Preserve maintained eval routing and close verification
-
-**Unlock:** the maintained harness can observe the actual installed product after
-environment-based auth is removed. Depends on slice 3. Status: complete; see [verification](../assets/final-verification.md).
-
-## Seam and scope
-
-Update only the maintained `evals/implementation/swebench/installed.py`, broker
-and their tests/supporting fixtures. Historical frozen runners, traces, result
-files, baseline records and studies are immutable. Do not run an agent benchmark.
-The coding-agent inference path remains unchanged; this slice concerns Jev calls.
-
-The existing harness remains Vercel-backed; adding a benchmark provider matrix is
-out of scope. Prepare an ordinary saved `{provider: "vercel", apiKey: brokerToken}`
-record in the isolated treatment container instead of API-key environment auth.
-Reuse the externally injected Node routing seam to send the exact Vercel native
-endpoint to model-egress. The broker keeps the real key, validates the permitted
-path/model and forwards to Vercel's documented TypeSafe-compatible route. It must
-not allow an agent-selected origin or leak real credentials into the container.
-
-Keep the preload/routing configuration harness-owned and include its bytes in
-the maintained candidate's freeze/identity inputs. It must not affect unrelated
-Codex inference requests. Runtime source bytes, skill and packed artifact remain
-the actual candidate; no patched CLI or hidden public env override. Frozen old
-studies retain their original broker code rather than adding runtime legacy paths
-to the new broker for hypothetical compatibility.
-
-Raw request/response observations remain external to the CLI and exclude auth
-headers. Decode available native billing/usage from retained bodies with explicit
-schema fixtures. Preserve support in the analysis decoder for historical Gateway
-observations, because reading old results is a real consumer. Do not rewrite them.
-Count missing/unmatched responses honestly: known subtotal can coexist with an
-unknown full total. Zero is valid only when explicitly observed, not a default.
-Jev cost/tokens never enter scored coding-agent cost; keep reporting available Jev
-observations separately. There is no token-count or timing optimization here.
-
-## Offline verification and final artifact
-
-- Extend maintained Python tests for saved credentials, preload freezing, accepted
- route/model, rejection of wrong targets/tokens, raw capture and billing decode.
- Use local fake upstreams; no paid calls or baseline reruns.
-- Check Vercel native `provider_metadata.gateway.cost`/generation ID, historical
- `providerMetadata.gateway.cost`, snake-case usage, missing/invalid billing and
- incomplete request logs. OpenRouter billing fixtures may cover `usage.cost`
- without expanding the benchmark runner to additional providers.
-- Run the Python suite through `bun run test`; all normal regression suites must
- still pass. Run `bun run verify` and `bun run format:check` after final changes.
-- Pack one final candidate using existing release tooling and record SHA-256.
- Run `bash scripts/test-installed.sh --prebuilt <tarball>` and
- `node scripts/test-native.mjs --prebuilt <tarball>` against those same bytes.
- These are verification operations, not a release/tag/npm publish.
-- Review the full change with the repo review workflow and an independent Codex
- review. Record actual gate results and per-provider live-check status in
- `assets/final-verification.md`, then update the README handoff/checklist.
-
-Human review surface: small synthetic receipt showing agent cost and separate
-known/unknown Jev observations, plus the final verification record. No newly
-measured savings or success-rate claim. Report any live-provider access still
-unverified in final delivery.
-
-## Decision budget
-
-Delegated: fixture organization, exact preload mount location and decoder helper
-names. Credential replacement and network routing remain external harness
-operations; the product API cannot grow to simplify tests. A route requiring a
-new product setting is a plan violation to resolve, not an implicit permission.
-User feedback could authorize a future benchmark/release, but silence does not.
diff --git a/test/reference/transport.test.ts b/test/reference/transport.test.ts
index d0ea332..793ad91 100644
--- a/test/reference/transport.test.ts
+++ b/test/reference/transport.test.ts
@@ -12,6 +12,7 @@ test("wire normalization retains every corpus field and rejects unknown envelope
method: "POST",
headers: {
authorization: "Bearer fixture",
+ "content-type": "application/json",
"x-jevgrep-original-url": preset.url,
},
});
@@ -19,7 +20,11 @@ test("wire normalization retains every corpus field and rejects unknown envelope
const original = JSON.parse(serialized);
const historical = new Request("http://localhost/v4/ai/evaluation-model", {
method: "POST",
- headers: { authorization: "Bearer fixture", "ai-model-id": "typesafe-ai/jev" },
+ headers: {
+ authorization: "Bearer fixture",
+ "ai-model-id": "typesafe-ai/jev",
+ "content-type": "application/json",
+ },
});
expect(JSON.stringify(transport.decode(historical, original, false))).toBe(serialized);
const wire = {
@@ -33,6 +38,9 @@ test("wire normalization retains every corpus field and rejects unknown envelope
),
};
expect(JSON.stringify(transport.decode(request, wire, true))).toBe(serialized);
+ const invalidContentType = new Request(request);
+ invalidContentType.headers.set("content-type", "text/plain");
+ expect(() => transport.decode(invalidContentType, wire, true)).toThrow();
expect(() => transport.decode(request, { ...wire, extra: true }, true)).toThrow();
const id = Object.keys(wire.questions)[0]!;
expect(() =>
diff --git a/test/reference/transport.ts b/test/reference/transport.ts
index 96a58b4..44596a6 100644
--- a/test/reference/transport.ts
+++ b/test/reference/transport.ts
@@ -39,6 +39,7 @@ export async function referenceTransport(provider: Provider = "vercel", key = "f
production: boolean,
) {
expect(request.method).toBe("POST");
+ expect(request.headers.get("content-type")).toMatch(/^application\/json(?:;|$)/);
expect(request.headers.get("authorization")).toBe(`Bearer ${key}`);
const body = raw as {
model?: string;