Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.
TypeScript ai-sdkclaude-codeclicode-searchcodexcoding-agentscontext-retrievaldeveloper-toolsjevsemantic-searchtypescriptvercel-ai-gateway
docs: close verified provider support specification
David Zhang05229ca7c6d0
diff --git a/docs/architecture.md b/docs/architecture.mdindex e17811a..c5e9521 100644--- a/docs/architecture.md+++ b/docs/architecture.md@@ -102,7 +102,9 @@ SDK handles HTTP serialization. Keep provider authentication and model access atthe core boundary so future providers do not reshape traversal or stdout.The CLI auth module owns one saved provider/key. The core preset owner resolvesits endpoint/model; the evaluator owns retries and answer-cache identity. Provider-selection never changes traversal or source rendering.+selection never changes traversal or source rendering. See the+[provider support record](../specs/done/provider-support/README.md) for the setup+constraints and transport preservation evidence.Acceptance measures the entire downstream Sol task: an official solve at a lowerfull task cost than its fixed baseline, while preserving baseline solves. Jevdiff --git a/specs/done/provider-support/README.md b/specs/done/provider-support/README.mdnew file mode 100644index 0000000..7e68bbd--- /dev/null+++ b/specs/done/provider-support/README.md@@ -0,0 +1,84 @@+# Saved-provider support++Jevgrep supports Vercel AI Gateway, native TypeSafe and OpenRouter through one+TypeSafe-compatible AI SDK adapter. The implementation landed in `9496540` after+the protocol checkpoint `06daef9`; it has not been published to npm.++## Why selection belongs in auth++Coding agents should research code without managing provider routing. A human+chooses a provider and key during `jg auth`, and ordinary search and doctor use+that saved record until auth replaces it. This avoids ambiguous environment-key+precedence and accidental provider fallback. Automation has one explicit path:+`jg auth --provider NAME --stdin`.++The only compatibility exception is deliberate: a saved key without a provider+still means Vercel. Reading it does not migrate or rewrite the file. Environment+credentials are no longer a product input; users who relied on them need auth.+This is a simpler setup contract, not a multi-account credential registry.++## What must stay true++- Provider changes select transport, not retrieval policy. Structured state,+ question wording/order, hierarchy, thresholds, excerpts and search stdout keep+ their frozen-reference contracts.+- The application owns actual HTTP attempts, retries, shared rate-limit cooldown+ and cancellation. SDK retries remain disabled. The 50,000-attempt guard protects+ against runaway work; it is not an optimization budget.+- Source freshness is checked before attempts and cached evidence use. An invalid+ provider answer is failure or incomplete discovery, never negative evidence.+- Exact answer-cache identity separates provider, endpoint, model and protocol,+ as well as existing parser/prompt/policy identity. Old-wire answers cannot+ certify the new route.+- Auth saves one private record atomically and never verifies a key over the+ network. Doctor checks separately with synthetic source. Keys never belong in+ chat or CLI output.+- Tests route requests outside the product. There is no public endpoint/model+ override or provider flag on search/doctor, and no automatic fallback.++The [preset owner](../../../packages/core/src/providers.ts),+[evaluator](../../../packages/core/src/evaluator.ts), and+[auth module](../../../apps/cli/src/auth.ts) hold the current mechanics.+[Provider replay](../../../test/reference/provider-parity.test.ts) pins semantic+requests and complete stdout; [installed journeys](../../../test/installed.test.mjs)+exercise the packed product. The [preservation contract](parity.md) maps the other+invariants to their checks. [Research](research.md) records the documented routes,+model choices, pinned SDK and immutable inputs rather than relying on memory.++## What the implementation taught us++Changing the adapter and exposing saved-provider setup are one product cutover:+shipping the picker first would allow a setup the old evaluator could not honor.+The protocol was proved separately before that cutover.++A test-only Node preload preserves the SDK's original JSON body while redirecting+its fixed destination to a local fixture. Rebuilding the body as a Request stream+obscured HTTP 401 in Node; retaining the original body fixed the test transport.+The preload is not part of the npm package. Historical replay continues to use its+original protocol; only production requests are strictly normalized for comparison.++New benchmark plans freeze that preload and use schema 4. Archived studies retain+their original runners; neither baselines nor historical results are rewritten.+The [maintained runner](../../../evals/implementation/swebench/installed.py) and+[broker](../../../evals/implementation/swebench/gateway_broker.py) remain Vercel-only+for evaluation. They retain observed Jev billing separately from scored coding-agent+cost, and unavailable totals remain unknown.++## Evidence and limits++[Final verification](assets/final-verification.md) records the exact candidate,+passed gates and review. The same tarball passed 32 installed Docker tests and+10 native macOS journeys. The three-provider replay preserves the frozen packet;+this is transport evidence, not a new solve-rate or savings benchmark.++Vercel and TypeSafe passed live doctor checks using isolated credentials from+the user-provided local environment file. OpenRouter live access remains+unverified because no key was available. No claim of identical live model probabilities, equal+solve quality, fresh cost savings or faster retrieval follows from fixture tests.+No npm release or paid agent benchmark was performed.++The [auth record and complete packet](assets/auth-verification.md),+[evaluator verification](assets/evaluator-verification.md), and+[protocol reproduction](assets/protocol-verification.md) retain the review evidence.+The [choices ledger](choices.md) explains the additional benchmark-format decision.+There was no visual design or image requirement for this change.diff --git a/specs/done/provider-support/assets/auth-verification.md b/specs/done/provider-support/assets/auth-verification.mdnew file mode 100644index 0000000..1d78e89--- /dev/null+++ b/specs/done/provider-support/assets/auth-verification.md@@ -0,0 +1,38 @@+# Saved-provider verification++2026-09-26. Installed Docker journeys cover Vercel, TypeSafe and OpenRouter:+auth from stdin, provider-specific doctor, search, cache reuse, replacement and+no fallback. They check private storage, ignored environment keys/URLs, invalid+providers/keys, legacy bytes and file identity remaining unchanged. The Docker+PTY test selects OpenRouter, checks hidden entry, and cancels at both prompts;+prior credentials survive and no temporary credential file remains.++All 32 installed Docker tests and all 10 native macOS smoke journeys passed on+the same candidate archive. The native run used Node v24.14.0, darwin/arm64 25.6.0.++## Live check++The actual built CLI ran this sequence with isolated config/cache directories:++```text+jg auth --provider vercel --stdin [key supplied privately; exit 0]+jg doctor+Jev connection verified through Vercel AI Gateway.++jg auth --provider typesafe --stdin [key supplied privately; exit 0]+jg doctor+Jev connection verified through TypeSafe.+```++Both doctor commands exited 0 with empty stderr. The live probe used an authorized local key, never printed it, and removed its+temporary saved credentials.+The user supplied the two keys in `.env.local`. No OpenRouter key was available;+its live service access remains unverified. No repository source was used in the live check.++## Exact verified reference packet++This is the complete frozen packet matched byte-for-byte by each provider's+production replay; it is not a sample invented for documentation.++[Read the exact stdout bytes](provider-stdout.txt). The text artifact preserves+trailing spaces on numbered blank source lines that Markdown formatters remove.diff --git a/specs/provider-support/assets/evaluator-verification.md b/specs/done/provider-support/assets/evaluator-verification.mdsimilarity index 100%rename from specs/provider-support/assets/evaluator-verification.mdrename to specs/done/provider-support/assets/evaluator-verification.mddiff --git a/specs/done/provider-support/assets/final-verification.md b/specs/done/provider-support/assets/final-verification.mdnew file mode 100644index 0000000..542dd18--- /dev/null+++ b/specs/done/provider-support/assets/final-verification.md@@ -0,0 +1,53 @@+# Final provider-support verification++2026-09-26. Implementation gates passed; no npm release or paid task benchmark+was run. Existing baselines, historical study artifacts, the reference manifest,+covered source files, corpus and frozen research skill remain unchanged.++| Gate | Result |+| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |+| `bun run verify` | Exit 0: types, lint, 125 core/CLI/reference, 18 Node protocol, 23 parser, 4 release, 27 maintained harness and 32 installed Docker tests |+| `bun run format:check` | Exit 0 |+| Latest schema-4 harness, local Python and Docker | 27/27 in each, exit 0; rerun after the final plan-version adjustment |+| Exact candidate, installed Docker `--prebuilt` | 32/32, exit 0 |+| Same candidate, native `--prebuilt` | 10/10, exit 0; Node v24.14.0, darwin/arm64 25.6.0 |+| Independent whole-diff Codex review | No actionable defects; pending suite completion was not assumed by the reviewer |+| Live built CLI `jg doctor`, isolated saved Vercel setup | Exit 0, expected success text, empty stderr |+| Live built CLI `jg doctor`, isolated saved TypeSafe setup | Exit 0, expected success text, empty stderr |+| Live OpenRouter | Unverified: credentials unavailable in the authorized local environment |++Candidate: `@dzhng/jevgrep` source version 0.2.0, prepared with the existing package+validator but **not published**. SHA-256:++```text+d8d7d0012da3163f2f615fddc01e07795251af28296950b5f248d990322bd6b2+```++The packaged file allowlist excludes the routing preload and fixtures; runtime+requirements remain Node only. Docker runtime tests use network isolation and+synthetic local HTTP; dependency downloads used Docker Desktop's existing proxy+through a temporary Docker client config, without changing the user's settings.++The broker's real local-upstream test verifies auth replacement, exact native+request/response capture, and rejection before forwarding for wrong targets,+credentials or bodies. Synthetic accounting retains 22 input tokens when cost+is absent while the full Jev total remains unknown. Jev observations remain+excluded from scored coding-agent cost. This proves maintained tooling behavior,+not a newly measured solve rate, cost saving or latency.++Review and cleanup preserved the one evaluator/one preset owner, removed the old+module and environment setup path, retained exact-version license validation,+and avoided a public transport override. The sole new data-format choice is+recorded in [the choices ledger](../choices.md). No new benchmark cohort was+prepared or executed.++The user subsequently supplied TypeSafe and Gateway keys in `.env.local`. Both+passed isolated live auth/doctor checks; neither key was printed or committed,+and the user’s saved setup was not changed. OpenRouter availability was checked+without printing values and no key was present.++The fresh closeout audit corrected an evidence pointer, moved the exact packet to+a raw text artifact to preserve trailing spaces, and added content-type validation+before reference normalization. The strengthened transport/replay/provider-parity+gate passed 15/15 in Docker. Removing the assertion in a temporary copy caused the+new regression to fail at the expected point. The production tarball was unchanged.diff --git a/specs/provider-support/assets/protocol-verification.md b/specs/done/provider-support/assets/protocol-verification.mdsimilarity index 96%rename from specs/provider-support/assets/protocol-verification.mdrename to specs/done/provider-support/assets/protocol-verification.mdindex 15d8236..3726b1c 100644--- a/specs/provider-support/assets/protocol-verification.md+++ b/specs/done/provider-support/assets/protocol-verification.md@@ -35,8 +35,7 @@ that prototype was removed. No product transport workaround is needed.The adapter archive includes its license. Its provider-utils 5.0.49 dependencyomits a license, like 5.0.45. The upstream repository LICENSE at tag revision`ee3169b3c4880e2abe4d0d7c781243bb81822ec4` is byte-identical to the existing retained-5.0.45 license. Slice 2 must extend the exact-version notice mapping before the-adapter becomes part of the product bundle.+5.0.45 license. The product notice owner now maps both verified versions to that retained license.Reproduce with `node --test packages/core/test-node/provider-protocol.mjs`;the same command runs inside `scripts/test-reference.sh`, and the normal suitediff --git a/specs/done/provider-support/assets/provider-stdout.txt b/specs/done/provider-support/assets/provider-stdout.txtnew file mode 100644index 0000000..28eaa4b--- /dev/null+++ b/specs/done/provider-support/assets/provider-stdout.txt@@ -0,0 +1,40 @@+Jevgrep: 3 relevant files.+AGENTS.md lookup (root and returned-file ancestors): none found.+- "src/backend/events.ts" — implementation, caller, test, fixture, helper; selected source and structural context below+ Reading lead source: lines 1-1+ Reading lead BackendTelemetry.recordEvent: lines 4-6+- "src/telemetry.ts" — implementation, caller, test, fixture, helper; selected source and structural context below+ Reading lead Telemetry.recordEvent: lines 3-5+- "tests/telemetry.test.ts" — implementation, caller, test, fixture, helper; selected source and structural context below+ Reading lead source: lines 1-1+ Reading lead testEventName: lines 3-5+End file list.++Source block "src/backend/events.ts" lines 1-8:+1: import { Telemetry } from '../telemetry';+2: // Backends preserve the same event contract.+3: export class BackendTelemetry extends Telemetry {+4: recordEvent(name: string) {+5: return super.recordEvent(name);+6: }+7: }+8:++Source block "src/telemetry.ts" lines 1-7:+1: // Events preserve the caller's name.+2: export class Telemetry {+3: recordEvent(name: string) {+4: return { name, recorded: true };+5: }+6: }+7:++Source block "tests/telemetry.test.ts" lines 1-6:+1: import { Telemetry } from '../src/telemetry';+2: // Regression example: do not rename the caller's event.+3: export function testEventName() {+4: return new Telemetry().recordEvent('opened').name === 'opened';+5: }+6:++End context.diff --git a/specs/provider-support/choices.md b/specs/done/provider-support/choices.mdsimilarity index 100%rename from specs/provider-support/choices.mdrename to specs/done/provider-support/choices.mddiff --git a/specs/provider-support/parity.md b/specs/done/provider-support/parity.mdsimilarity index 63%rename from specs/provider-support/parity.mdrename to specs/done/provider-support/parity.mdindex 606b35e..88ec45f 100644--- a/specs/provider-support/parity.md+++ b/specs/done/provider-support/parity.md@@ -5,18 +5,18 @@ The immutable input tree and historical hashes are identified inmanifest-covered files and prior studies byte-for-byte unchanged. The productionharness may change its setup/routing; it must not loosen the expected result.-| Preserved behavior | Existing evidence | Owner and acceptance gate |-| -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |-| Structured state, exact instructions and question IDs/order | `test/reference/requests.test.ts`, `corpus.json` | Slice 2: compare new-wire semantic requests to frozen corpus through built production CLI |-| Hierarchy, thresholds, role/evidence/reference decisions | `test/reference/discovery-parity.test.ts`, `selection.test.ts`, `repository-context-parity.test.ts` | Slice 2: existing decision regressions remain green |-| Full stdout, source selection/order and excerpts | `replay.test.ts`, `evidence-order.test.ts`, Python/comment parity tests | Slice 2: exact complete stdout plus existing ordering/parser tests |-| Retries, cooldown, auth abort, timeout and attempt guard | `test/gateway.test.ts` | Slice 2: real native-wire HTTP failure matrix; rename test consistently if module renamed |-| Source freshness on cache reads and every retry | `test/retrieval-freshness.test.ts` | Slice 2: mutation/deletion during wait cannot upload stale source |-| Validated answer-only cache | `packages/core/test/cache.test.ts` and evaluator validation | Slice 2: same namespace hits; provider/endpoint/model/protocol differences miss; invalid/missing answers never cache as negative evidence |-| Private atomic auth, cancellation and key validation | `apps/cli/test/auth.test.ts` | Slice 3: new/legacy records, replacement and failure paths; assert old bytes survive cancellation |-| Installed runtime, ignores, signals, pipe closure, stdout-only | `test/installed.test.mjs`, `scripts/test-native.mjs` | Slices 2–3: actual packed CLI with controlled HTTP; all new provider cases selected by native gate |-| Frozen research skill policy and installation | `test/reference/skill.test.ts`, release/skill tests | Slice 3: only setup/auth guidance changes, research body preserved |-| Raw eval observations and honest missing billing | `evals/implementation/swebench/test_installed.py` | Slice 4: offline broker/receipt fixtures, frozen baselines/studies unchanged |+| Preserved behavior | Existing evidence | Owner and acceptance gate |+| -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |+| Structured state, exact instructions and question IDs/order | `test/reference/requests.test.ts`, `corpus.json` | Evaluator: compare new-wire semantic requests to frozen corpus through built production CLI |+| Hierarchy, thresholds, role/evidence/reference decisions | `test/reference/discovery-parity.test.ts`, `selection.test.ts`, `repository-context-parity.test.ts` | Evaluator: existing decision regressions remain green |+| Full stdout, source selection/order and excerpts | `replay.test.ts`, `evidence-order.test.ts`, Python/comment parity tests | Evaluator: exact complete stdout plus existing ordering/parser tests |+| Retries, cooldown, auth abort, timeout and attempt guard | `test/evaluator.test.ts` | Evaluator: real native-wire HTTP failure matrix |+| Source freshness on cache reads and every retry | `test/retrieval-freshness.test.ts` | Evaluator: mutation/deletion during wait cannot upload stale source |+| Validated answer-only cache | `packages/core/test/cache.test.ts` and evaluator validation | Evaluator: same namespace hits; provider/endpoint/model/protocol differences miss; invalid/missing answers never cache as negative evidence |+| Private atomic auth, cancellation and key validation | `apps/cli/test/auth.test.ts`, `apps/cli/test/process.test.ts` | Auth: new/legacy records, replacement and failure paths; assert old bytes survive cancellation |+| Installed runtime, ignores, signals, pipe closure, stdout-only | `test/installed.test.mjs`, `scripts/test-native.mjs` | Installed CLI: actual packed CLI with controlled HTTP; all new provider cases selected by native gate |+| Frozen research skill policy and installation | `test/reference/skill.test.ts`, release/skill tests | Auth: only setup/auth guidance changes, research body preserved |+| Raw eval observations and honest missing billing | `evals/implementation/swebench/test_installed.py` | Harness: offline broker/receipt fixtures, frozen baselines/studies unchanged |## Permitted differencesdiff --git a/specs/provider-support/research.md b/specs/done/provider-support/research.mdsimilarity index 82%rename from specs/provider-support/research.mdrename to specs/done/provider-support/research.mdindex c8578ef..c925449 100644--- a/specs/provider-support/research.md+++ b/specs/done/provider-support/research.md@@ -1,7 +1,7 @@# Provider contract evidenceInspected 2026-09-26. These are documentation/source observations, not live service-results. The implementation starts with a small adapter reproduction, not another+results. Verification uses a small adapter reproduction, not anotherretrieval experiment.## Fixed presets@@ -19,13 +19,13 @@ OpenRouter chat completions or its separate `/api/alpha/decisions` API.Versioned native/OpenRouter names do not prove identical backends or answers toVercel's alias. Preserve existing thresholds; do not claim cross-provider solvequality from transport fixtures. Record actual returned model identity in external-probe evidence when available. A conflicting live API result goes into this plan+probe evidence when available. A conflicting live API result goes into this recordbefore changing a preset; never silently try other providers/models.## Adapter identity and behavior-Use `@ai-sdk/[email protected]` with the existing `[email protected]`, subject to the-published-package reproduction gate in slice 1. Registry metadata inspected:+Use `@ai-sdk/[email protected]` with the existing `[email protected]`, verified by the+[published-package reproduction](assets/protocol-verification.md). Registry metadata inspected:```textdist.integrity = sha512-nP5NJGCwZ7jDE2dImurTU9igbj1OUeFAXLvZKcM05KlBpLxhOddBe3sMftAhQq3Zq+B8NqLx1dcW4vQVD2l32w==@@ -50,18 +50,18 @@ the maintained broker's raw-response decoder, not reasons to add CLI telemetry.Native TypeSafe billing may be absent. Unknown totals remain unknown; observedJev billing stays separate from scored coding-agent task cost.-The adapter's error classes differ from Gateway's. Slice 1 reproduces HTTP status,-timeout, disconnect, and malformed-response behavior; slice 2 owns the mapping to-existing retry/split semantics. Primary [TypeSafe API documentation](https://docs.typesafe.ai/api)+The adapter's error classes differ from Gateway's. The [protocol checks](assets/protocol-verification.md) reproduce HTTP status,+timeout, disconnect, and malformed-response behavior; the evaluator maps those+to the existing retry/split semantics. Primary [TypeSafe API documentation](https://docs.typesafe.ai/api)and the Vercel guide describe native errors; fixtures must use their shapes.## Immutable preservation baselineRepository input: `ca7054ed4c50283030102d15cd73d816e85bc569`. This Git tree pins the-current production implementation, lockfile, policy, parser versions, source tree,+pre-change production implementation, lockfile, policy, parser versions, source tree,skill, test harness, and previously recorded evidence. The additional historical-source hashes are owned by [the reference manifest](../../test/reference/manifest.json).-Keep that manifest, its covered files, [the corpus](../../test/reference/corpus.json),+source hashes are owned by [the reference manifest](../../../test/reference/manifest.json).+Keep that manifest, its covered files, [the corpus](../../../test/reference/corpus.json),and all historical benchmark results unchanged.The corpus compares a request multiset: independent network arrival order is not@@ -70,7 +70,7 @@ selection/evidence ordering, and complete rendered stdout are invariants. Keepexisting decision/order regressions alongside semantic wire comparison; neverserialize production concurrency to make a replay easier.-The [corrected repeat](../done/jevgrep/assets/variance-repeat.md) is historical+The [corrected repeat](../jevgrep/assets/variance-repeat.md) is historicalevidence, not a new-provider acceptance result. Baselines are immutable and arenever rerun for this feature. Neither equal success rate nor fresh savings is-established by this plan.+established by transport checks.diff --git a/specs/provider-support/README.md b/specs/provider-support/README.mddeleted file mode 100644index ba0beb0..0000000--- a/specs/provider-support/README.md+++ /dev/null@@ -1,145 +0,0 @@-# Saved-provider support--Add native TypeSafe and OpenRouter to `jg`, using one TypeSafe-compatible AI SDK-adapter for all three services. Choose the provider once during authentication;-ordinary searches keep the same interface and retrieval behavior.--## Next Agent Prompt--Implementation is complete, last updated 2026-09-26. All four slices and required-checks pass; see [final verification](assets/final-verification.md). Close and-archive this spec after the final claim audit. TypeSafe/OpenRouter live access-remains unverified because credentials were unavailable; offline support is-verified and Vercel passed a live doctor check. No npm publication is selected.--You have no unresolved product question. Published-package behavior and live-provider access need verification, not invented compatibility code. Missing-credentials must not block offline implementation; report the missing live check-honestly. Do not collect secrets in chat or overwrite the user's own saved setup-for tests. Use isolated HOME/config/cache locations.--- [x] [1. Protocol reproduction and controlled routing](slices/01-protocol.md)-- [x] [2. Shared evaluator and production parity](slices/02-evaluator.md)-- [x] [3. Saved-provider auth, installed journeys, and onboarding](slices/03-auth.md)-- [x] [4. Maintained eval routing and final verification](slices/04-tooling.md)--Update this prompt and the owning slice with evidence, remaining work and any-deviations before ending each pass. Review each implementation slice before-committing. Keep the checkout clean on main; do not create orphan worktrees or-archive branches. There is no new benchmark run or npm publication in this plan.--## Roadmap and review surfaces--```mermaid-flowchart LR- A[1: Prove SDK wire contract] --> B[2: Replace evaluator and prove parity]- B --> C[3: Choose and save provider in auth]- C --> D[4: Repair maintained eval routing and close checks]-```--| Slice | Question answered | Reviewable result |-| ----- | ---------------------------------------------------------- | ----------------------------------------------------------------------------- |-| 1 | Does the pinned adapter implement the documented protocol? | Synthetic HTTP transcript and pass/fail probe |-| 2 | Can the new transport preserve existing retrieval? | Built CLI reproduces the frozen corpus; failure and cache tests |-| 3 | Does one setup reliably select the service? | Installed Docker auth → doctor → search for every provider, plus native smoke |-| 4 | Does maintained tooling still observe the actual product? | Offline broker/receipt fixtures and final verification record |--These are CLI/library checkpoints, with text transcripts as review artifacts.-There is no visual design or screenshot deliverable. Human feedback is welcome-but does not block reversible implementation; decide from the specified gates-and record the reasoning if no feedback arrives.--## Binding product contract--`jg auth` asks **provider first, then hidden key**. It saves one-`{provider, apiKey}` record. Re-running auth replaces that setup; searches and-doctor use it until replaced. No automatic provider fallback, account registry,-logout, or provider-switch command.--For automation, use `jg auth --provider <vercel|typesafe|openrouter> --stdin`.-The provider is mandatory with stdin. As a simplicity decision for this plan,-`--provider` is accepted only together with `--stdin`; interactive auth always-shows the picker. No provider/model/endpoint options on search or doctor.--Credentials come only from the saved record. Provider API-key, endpoint and model-environment variables cannot override it; XDG storage locations remain supported.-Auth saves without a network call; doctor separately tests a synthetic question-and identifies the saved provider. No secrets in stdout, exceptions or fixtures.--**The sole required backward compatibility:** absence of the `provider` property-means Vercel. Do not rewrite credentials during reads or require reauthentication.-A present invalid value, including null, is an error. No migration command,-format-version field, or preservation of environment-based authentication.--The selected provider receives the same eligible repository content as before.-Keep the existing head-friendly source packet, excerpts, stdout-only behavior,-exit semantics, filesystem exclusions, optional parsers and skill research policy.-No CLI report/trace files; only existing intentional auth/cache/skill persistence.--## Single owners and scope boundaries--| Concept | Owner |-| ------------------------------------------------------------ | ------------------------------------------------------------------------- |-| Fixed provider IDs, labels, endpoint/model/protocol identity | One small core preset module |-| Saved key/provider, config paths, validation, atomic writes | CLI auth module |-| HTTP attempts, retries, cooldown, answer validation | One evaluator using `@ai-sdk/typesafe-ai` |-| Exact answer-cache storage | Existing cache module; evaluator supplies namespace |-| Hierarchy, prompts, thresholds, batching, source freshness | Existing retrieval/request/source owners, unchanged |-| Rendering and exit handling | Existing CLI owners |-| Local HTTP redirection for verification | External test-owned Node preload, never packaged as product configuration |-| Paid-call forwarding and raw accounting | Maintained benchmark broker/runner |--No second evaluator, provider plugin registry, hand-written native client, or-runtime protocol normalizer. Normalize transport only in tests. The intermediate-Vercel-only CLI wiring in slice 2 disappears when slice 3 passes saved provider-identity; it is not a compatibility layer to retain. Rename `gateway.ts` to-`evaluator.ts` with its imports when replacing the implementation, leaving no-forwarding module behind.--Preserve the 50,000 **actual HTTP attempt** guard solely as runaway protection,-SDK retries disabled, existing application retry/split policy, shared cooldown,-cancellation and source freshness on every attempt. No performance tuning,-prompt changes, new discovery strategy, parser changes, or benchmark optimization.--## Acceptance and limits--Each slice carries its focused gate. Final acceptance requires `bun run verify`,-`bun run format:check`, and the native packaged smoke, with all-provider installed-Docker journeys included in the normal suite. Freeze one candidate tarball and-run it through both installed Docker and native checks, recording its SHA-256.-Fixtures use actual SDK HTTP and installed CLI, not a mocked evaluator.--Perform bounded synthetic live doctor checks for providers whose credentials are-available through an authorized local source, using isolated saved records. Keep-those results separate from offline checks; unavailable credentials mean live-access is **unverified**, not failed or passed. A documented route that actually-rejects its preset is a correctness issue to resolve before claiming support.--Historical cost/quality claims retain their original scope. Jev cost is excluded-from scored coding-agent task cost, but available observed billing is preserved-separately; missing totals are unknown. No baseline reruns, personal-repo evals,-fresh SWE-bench cohort, speed claim, release version selection, or npm tag push.--## Plan audit--Three blind read-only drafts informed this plan: a fewest-slices draft proposed-one atomic change, a seam-focused draft proposed five stages, and Claude Opus's-risk-first draft proposed eight. This plan takes their shared protocol-first-gate, strict reference comparison and external routing seam. Four slices keep-auth/retrieval/tooling independently reviewable without making docs or packaging-separate workstreams. A single large slice would delay useful evidence; eight-would leave too many small migration boundaries to maintain.--Rejected alternatives: an internal baseURL override is unnecessary when fetch-redirection is test-owned; a multi-provider benchmark runner expands this feature-without helping product verification; interactive `--provider` adds a second way-to skip the required picker. Keep only the stdin automation form. Package notices-are part of the adapter/build gate, not deferred release cleanup.--The interview was completed in the map, including the user's explicit legacy-credential exception; no repeat permission or compatibility question is needed.-The roadmap front-loads protocol evidence, then preserves the existing product-before exposing additional auth choices. [Parity](parity.md) names each preserved-behavior, its evidence and owning gate. [Research](research.md) distinguishes-documented presets from unverified live access. Every slice names its remaining-implementation discretion; a choice outside that list requires updating this plan.diff --git a/specs/provider-support/assets/auth-verification.md b/specs/provider-support/assets/auth-verification.mddeleted file mode 100644index e0a2cf5..0000000--- a/specs/provider-support/assets/auth-verification.md+++ /dev/null@@ -1,80 +0,0 @@-# Saved-provider verification--2026-09-26. Installed Docker journeys cover Vercel, TypeSafe and OpenRouter:-auth from stdin, provider-specific doctor, search, cache reuse, replacement and-no fallback. They check private storage, ignored environment keys/URLs, invalid-providers/keys, legacy bytes and file identity remaining unchanged. The Docker-PTY test selects OpenRouter, checks hidden entry, and cancels at both prompts;-prior credentials survive and no temporary credential file remains.--All 32 installed Docker tests and all 10 native macOS smoke journeys passed on-the same candidate archive. The native run used Node v24.14.0, darwin/arm64 25.6.0.--## Live check--The actual built CLI ran this sequence with isolated config/cache directories:--```text-jg auth --provider vercel --stdin [key supplied privately; exit 0]-jg doctor-Jev connection verified through Vercel AI Gateway.-```--Doctor exited 0 with empty stderr. The key came from the authorized local Duet-environment, was never printed, and the temporary credentials were removed.-TypeSafe/OpenRouter keys were unavailable there; live service access for those-providers remains unverified. No repository source was used in the live check.--A fresh smaller-model skill check correctly directed a missing-credential user-to terminal `jg auth`, ignored an OpenRouter environment key, avoided asking for-secrets, and continued ordinary discovery. The initial consultation was invalid:-its instruction prohibited shell commands needed to read the skill. That was-corrected before the fresh check; no product behavior was changed to fit it.--## Exact verified reference packet--This is the complete frozen packet matched byte-for-byte by each provider's-production replay; it is not a sample invented for documentation.--```text-Jevgrep: 3 relevant files.-AGENTS.md lookup (root and returned-file ancestors): none found.-- "src/backend/events.ts" — implementation, caller, test, fixture, helper; selected source and structural context below- Reading lead source: lines 1-1- Reading lead BackendTelemetry.recordEvent: lines 4-6-- "src/telemetry.ts" — implementation, caller, test, fixture, helper; selected source and structural context below- Reading lead Telemetry.recordEvent: lines 3-5-- "tests/telemetry.test.ts" — implementation, caller, test, fixture, helper; selected source and structural context below- Reading lead source: lines 1-1- Reading lead testEventName: lines 3-5-End file list.--Source block "src/backend/events.ts" lines 1-8:-1: import { Telemetry } from '../telemetry';-2: // Backends preserve the same event contract.-3: export class BackendTelemetry extends Telemetry {-4: recordEvent(name: string) {-5: return super.recordEvent(name);-6: }-7: }-8:--Source block "src/telemetry.ts" lines 1-7:-1: // Events preserve the caller's name.-2: export class Telemetry {-3: recordEvent(name: string) {-4: return { name, recorded: true };-5: }-6: }-7:--Source block "tests/telemetry.test.ts" lines 1-6:-1: import { Telemetry } from '../src/telemetry';-2: // Regression example: do not rename the caller's event.-3: export function testEventName() {-4: return new Telemetry().recordEvent('opened').name === 'opened';-5: }-6:--End context.-```diff --git a/specs/provider-support/assets/final-verification.md b/specs/provider-support/assets/final-verification.mddeleted file mode 100644index 62afb05..0000000--- a/specs/provider-support/assets/final-verification.md+++ /dev/null@@ -1,41 +0,0 @@-# Final provider-support verification--2026-09-26. Implementation gates passed; no npm release or paid task benchmark-was run. Existing baselines, historical study artifacts, the reference manifest,-covered source files, corpus and frozen research skill remain unchanged.--| Gate | Result |-| ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |-| `bun run verify` | Exit 0: types, lint, 125 core/CLI/reference, 18 Node protocol, 23 parser, 4 release, 27 maintained harness and 32 installed Docker tests |-| `bun run format:check` | Exit 0 |-| Latest schema-4 harness, local Python and Docker | 27/27 in each, exit 0; rerun after the final plan-version adjustment |-| Exact candidate, installed Docker `--prebuilt` | 32/32, exit 0 |-| Same candidate, native `--prebuilt` | 10/10, exit 0; Node v24.14.0, darwin/arm64 25.6.0 |-| Independent whole-diff Codex review | No actionable defects; pending suite completion was not assumed by the reviewer |-| Live built CLI `jg doctor`, isolated saved Vercel setup | Exit 0, expected success text, empty stderr |-| Live TypeSafe/OpenRouter | Unverified: credentials unavailable in the authorized local environment |--Candidate: `@dzhng/jevgrep` source version 0.2.0, prepared with the existing package-validator but **not published**. SHA-256:--```text-d8d7d0012da3163f2f615fddc01e07795251af28296950b5f248d990322bd6b2-```--The packaged file allowlist excludes the routing preload and fixtures; runtime-requirements remain Node only. Docker runtime tests use network isolation and-synthetic local HTTP; dependency downloads used Docker Desktop's existing proxy-through a temporary Docker client config, without changing the user's settings.--The broker's real local-upstream test verifies auth replacement, exact native-request/response capture, and rejection before forwarding for wrong targets,-credentials or bodies. Synthetic accounting retains 22 input tokens when cost-is absent while the full Jev total remains unknown. Jev observations remain-excluded from scored coding-agent cost. This proves maintained tooling behavior,-not a newly measured solve rate, cost saving or latency.--Review and cleanup preserved the one evaluator/one preset owner, removed the old-module and environment setup path, retained exact-version license validation,-and avoided a public transport override. The sole new data-format choice is-recorded in [the choices ledger](../choices.md). No new benchmark cohort was-prepared or executed.diff --git a/specs/provider-support/map.md b/specs/provider-support/map.mddeleted file mode 100644index 7267892..0000000--- a/specs/provider-support/map.md+++ /dev/null@@ -1,119 +0,0 @@-# TypeSafe and OpenRouter support: decision map--Completed explore-unknowns walk, 2026-09-26. Territory inspected at `3cd6dc8`.-This preserves the exploration decisions. The [implementation spec](README.md)-now owns sequencing and verification; no provider implementation or live testing-is claimed here.--## Known knowns--Add native TypeSafe and OpenRouter while retaining Vercel AI Gateway. Use one-TypeSafe-compatible AI SDK adapter for all three, selected through fixed provider-presets. Keep structured state, question wording, thresholds, hierarchy, source-selection and stdout behavior unchanged.--The current [evaluator](../../packages/core/src/gateway.ts) owns requests, retries,-timeouts, cooldown and validation; [retrieval](../../packages/core/src/retrieve.ts)-owns source-freshness hooks. [Cache identity](../../packages/core/src/cache.ts)-already has provider/model fields. [Authentication](../../apps/cli/src/auth.ts)-currently saves one `{apiKey}` record, and [CLI wiring](../../apps/cli/src/index.ts)-passes its key into both search and doctor.--The official [AI SDK TypeSafe adapter](https://github.com/vercel/ai/blob/main/packages/typesafe-ai/src/typesafe-ai-evaluation-model.ts)-maps boolean questions to native `noul` questions and maps answers back to-probabilities. It appends `/systemone`. Documented compatible base URLs for that-adapter are:--| Provider | Base URL |-| ---------- | ------------------------------------------ |-| Vercel | `https://ai-gateway.vercel.sh/typesafe/v1` |-| TypeSafe | `https://api.typesafe.ai/v1` |-| OpenRouter | `https://openrouter.ai/api/v1` |--Sources: [Vercel compatibility](https://vercel.com/changelog/ai-gateway-now-supports-typesafe-clients-and-http-api-for-jev),-[TypeSafe API](https://docs.typesafe.ai/api), and-[OpenRouter compatibility](https://openrouter.ai/blog/insights/what-is-jev/).-These are documented contracts, not results from live calls in this walk.--## Known unknowns resolved--| Question | Decision and why | Closed by |-| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- |-| One or two adapters? | One TypeSafe-compatible adapter for all providers; configuration selects the service. Verify Vercel parity before replacement. | User |-| Where is the provider selected? | `jg auth` asks for provider first, then key. Searches and doctor always use the saved provider. | User |-| Store several accounts? | One record only. Re-running auth replaces provider and key; no logout or provider-switch command. | User |-| Invocation overrides? | No provider, endpoint or model options on search/doctor. Use fixed presets. | User's simplicity constraint |-| Environment keys? | Saved credentials only; do not implicitly read provider key or endpoint environment variables. XDG storage paths remain supported. | User |-| Automation? | `jg auth --provider <vercel\|typesafe\|openrouter> --stdin` explicitly selects the provider and reads the key without prompting. Require provider for stdin setup. | Shown workflow |-| Legacy credentials? | If the provider field is absent, interpret the saved key as Vercel without requiring re-authentication or rewriting the file during a search. Invalid present values are errors. | User correction |-| Validate during auth? | Retain save-only auth and separate synthetic `jg doctor`; preserve the previous setup on cancellation or invalid input. | Existing contract |-| Automatic fallback? | Never change providers after failure; retry the selected service under the existing bounded policy, then return honest failure/partial results. | User's fixed-provider constraint |-| Credential storage? | Retain the same config location, atomic replacement and owner-only permissions; never echo keys. | Existing contract |--New saved data has this shape (the key below is a placeholder):--```json-{ "provider": "openrouter", "apiKey": "<secret>" }-```--Legacy `{ "apiKey": "<secret>" }` means Vercel. No format-version field, stored-model/URL, multi-account registry or credential migration command is needed.--## Unknown knowns extracted--The intended consumer is a coding agent after a human performs one simple setup-on macOS or Linux. Provider flexibility belongs in setup, not every invocation.-Existing authenticated users should keep working. The user explicitly narrowed-the earlier no-backward-compatibility preference for providerless credentials.--The reviewed interaction is: choose provider, enter a hidden key, save, optionally-run doctor, then use ordinary `jg "question"` commands. Doctor identifies the-saved provider in its result. Missing credentials should direct agents to human-authentication or explicit stdin setup; the skill must stop recommending implicit-`AI_GATEWAY_API_KEY` configuration. Agents continue ordinary discovery when setup-is unavailable rather than collecting secrets in chat.--## Unknown unknowns: findings and verification gates--The sweep covered auth/parser/CLI wiring, evaluator/cache/retrieval, auth and HTTP-tests, installed and reference fixtures, native smoke setup, benchmark runner and-broker, package metadata, and README/skill onboarding.--- **Protocol parity — sharp edge.** [Reference tests](../../test/reference/replay.test.ts)- pin the old Gateway envelope and exact output. The new wire format moves the- model into the body and uses `noul`. Normalize only these intentional transport- differences; preserve state, instructions, question IDs/order, decisions and- complete stdout. Keep frozen reference files and past results immutable.-- **Retry/freshness coupling — sharp edge.** [Gateway orchestration](../../packages/core/src/gateway.ts)- classifies some failures by Gateway-specific exception names. Verify equivalent- handling for the new adapter's errors. SDK retries stay disabled, actual HTTP- attempts count toward the 50,000 guard, and every retry still revalidates source.- Preserve shared rate-limit cooldown, cancellation and sibling auth-failure abort.-- **Cache crossover — decided.** Cache identity must include selected provider,- effective endpoint, model and protocol/policy identity. Do not reuse old-wire- Vercel answers as proof of new-wire behavior or mix providers' answers.-- **Test and eval routing — sharp edge.** [Installed tests](../../test/installed.test.mjs),- [native smoke](../../scripts/test-native.mjs), and the maintained- [benchmark runner](../../evals/implementation/swebench/installed.py)/- [broker](../../evals/implementation/swebench/gateway_broker.py) rely on Gateway- environment credentials and its old HTTP path. Adapt their credential setup and- controlled transport routing without adding public search overrides. Exercise- all three presets and legacy credentials through the installed CLI. Never rerun- baselines or edit old frozen study artifacts to make the new harness pass.-- **Observed Jev billing — sharp edge.** The benchmark decoder currently reads- `providerMetadata.gateway.cost`. A native response may have different or missing- billing metadata. Preserve available raw observations and report unavailable- totals as unknown, never zero. Jev remains excluded from scored agent task cost.-- **Model identity and live behavior — OPEN verification.** Provider model names- and aliases differ, and aliases can change their backing model. The subsequent [research record](research.md) resolves documented model IDs;- slice 1 still verifies the published adapter response/error contract. Before claiming live support, run bounded synthetic doctor checks- with each provider's credentials when available. No keys were read and no live- model calls were made during this walk. Missing access must be reported, not- replaced with a claim that fixtures prove service access or equal solve quality.--## Superseded implementation handoff--The earlier request to implement directly from this map is superseded by the-[spec README and Next Agent Prompt](README.md#next-agent-prompt). Follow its slice-order and preservation gates. Keep this map as user-decision rationale, not a-competing build plan. No paid benchmark repeat or release version was selected.diff --git a/specs/provider-support/slices/01-protocol.md b/specs/provider-support/slices/01-protocol.mddeleted file mode 100644index 29c6eab..0000000--- a/specs/provider-support/slices/01-protocol.md+++ /dev/null@@ -1,65 +0,0 @@-# 1. Reproduce the provider protocol--**Unlock:** prove the chosen SDK and controlled routing before touching retrieval.-No dependency on another slice. Status: complete; see [verification](../assets/protocol-verification.md).--## Seam and artifact--Pin the adapter identified in [research](../research.md), inspect its published-artifact, and use `experimental_evaluate` with structured synthetic state and two-boolean questions. Send through the actual adapter to local HTTP fixtures for-each documented preset. Keep the probe test-owned and add it to the ordinary test-entry point; do not add a user-facing probe command or another native client.--Keep `[email protected]` and the historical oracle's resolved dependency closure fixed.-Adding the adapter may add its own provider-utils version; it must not silently-upgrade the oracle through lockfile deduplication. Inspect the new dependencies'-license files too: `scripts/package-notices.mjs` currently has an exact-version-exception for `@ai-sdk/[email protected]`. Record whether the new artifact-supplies its own license before changing that exception.--The test transport seam is a Node `--import` preload outside the npm package.-It observes the original fetch URL, allows only the three exact preset endpoints,-then redirects to a local fixture preserving method, headers, body and abort-signal. Unknown destinations fail closed. No production environment override,-TLS bypass, or user config extension. For core tests the same narrow fetch-redirection can be passed as a dependency. Test expectations remain independent-of the production preset table so a wrong URL/model cannot validate itself.--Save a concise `assets/protocol-verification.md` with package integrity, tested-request/answer/error shapes and reproduction command. It contains synthetic data-only. This is the first human review surface; no live credential is needed.--## Verification and verdict--- Assert exact endpoint suffix, model, bearer key, structured nested state,- question IDs/order/instructions, boolean-to-noul conversion, and finite answer- probability conversion. Inspect the pinned SDK's real response schema; provide- required response fields rather than bypassing its validator.-- Reproduce 401/403, 408, 409, 429 with both Retry-After forms, 5xx, timeout, socket- disconnect, malformed JSON, missing and out-of-range answers. Record actual- error status/name/cause available to the application. No assumption that old- Gateway exception names survive.-- Run through the current Node Docker reference harness. Prove cancellation- reaches real HTTP and no hidden SDK retries occur when `maxRetries: 0`.-- Prove the preload rejects a wrong original destination and that fixture- redirects are not baked into the packed product. Keep original reference tests- green. No live calls or benchmark execution is required for this verdict.--Acceptance: a reproducible adapter contract for all presets and observed error-classification inputs. If the pinned package disagrees with its docs, resolve-that concrete mismatch and update the research record before slice 2. Do not-redesign transport around an assumption or change prompts/thresholds.--## Decision budget--Delegated: probe/fixture filenames, internal helper names, concise transcript-format, and the smallest compatible package pin correction if the inspected-artifact fails the stated API contract (record exact reason/integrity). Presets-are the documented choices, not an invitation to pick newer aliases. Human-feedback can change presentation of the transcript; an API contradiction changes-the plan. Proceed on passing evidence without waiting for sign-off.--An adapter requiring an `ai` upgrade, clamping invalid probabilities, or changing-request meaning fails this slice. Reslice that concrete conflict before proceeding;-do not hide it with a permissive fixture or silently alter the frozen oracle.diff --git a/specs/provider-support/slices/02-evaluator.md b/specs/provider-support/slices/02-evaluator.mddeleted file mode 100644index 3020460..0000000--- a/specs/provider-support/slices/02-evaluator.md+++ /dev/null@@ -1,76 +0,0 @@-# 2. Replace the evaluator and prove production parity--**Unlock:** one provider-neutral evaluator preserves the existing search contract.-Depends on slice 1. Status: complete; see [verification](../assets/evaluator-verification.md).--## API seam and ownership--Add one core preset owner exposing a `ProviderId` union (`vercel`, `typesafe`,-`openrouter`) and immutable label/baseURL/model/protocol records from research.-Rename `gateway.ts` to `evaluator.ts` and update consumers directly. Keep-`EvaluationRequest`, the retrieval `Evaluator` interface and failure kinds stable.--`createEvaluator` takes explicit `provider: ProviderId`, validated `apiKey`, and-the existing cache/policy/signal/attempt settings. Resolve URL/model internally;-remove the old general baseURL option. A fetch dependency may exist for core-testing, but CLI users get no transport override. Always pass the key to-`createTypeSafeAi`; never depend on SDK environment discovery.--At this checkpoint CLI search/doctor pass Vercel explicitly: public setup remains-Vercel-only until slice 3. Remove `AI_GATEWAY_BASE_URL` wiring now, and move-production test routing to slice 1's preload. Slice 3 removes the explicit Vercel-selection when saved credentials become the shared input. Do not add a temporary-provider flag, dual evaluator, wrapper module, or dispatcher registry.--Keep one owner of actual attempts and retries. Cache identity includes provider-ID, effective endpoint, requested model, a native protocol identity and existing-policy/parser/prompt identities. Cache storage/schema/TTL stay unchanged. Old-wire-entries naturally miss; no deletion or migration is necessary.--## Gates and review surface--Implement the failure mapping from the published SDK evidence, preserving the-existing application's behavior rather than copying only old error names:--- SDK `maxRetries: 0`; one fetch increments one request. Guard remains 50,000.-- Existing normal/navigation retry limits, navigation transient split eligibility,- special 429 handling and shared cooldown. Do not broaden retry policy incidentally.-- Preserve the Node disconnect recovery checked by- `test/reference/disconnect.test.ts`. Distinguish transport failures from invalid- JSON/answers; do not classify every status-less exception as transient. Do not- inherit SDK `isRetryable` wholesale (for example, navigation 409 must not gain- a new retry/split merely because the adapter marks it retryable).-- 401/403 abort in-flight siblings, cooldown waits and future attempts. User- cancellation/pipe closure stops work. Retry-time source validation still runs- after cooldown and before HTTP; invalid source never uploads again.-- Missing/malformed/nonfinite/out-of-range answers remain failure/incomplete,- never negative evidence. Only validated answers enter cache.-- Prove same-identity cache reuse and misses across provider, endpoint, model and- protocol identities without making those fields public CLI options.--Apply [the parity contract](../parity.md) through the built production CLI as soon-as the adapter is wired. Keep historical replay on its existing Gateway protocol;-adapt only production setup and strict transport normalization. Keep the manifest,-corpus, historical source and existing `replay.test.ts` assertions unchanged.-Add provider-parameterized production parity cases without replacing the original-Vercel comparison. Every provider must reproduce the same packet under identical-controlled probabilities. Retain selection, source-order, comment and parser gates.--Keep the package build green with the new adapter's actual bundled dependency-notices. If a new upstream package omits a license, use a verified matching-upstream license and record its provenance under the existing notices mechanism;-never disable validation or apply the old version's exception indiscriminately.--Run `bun run test:reference` plus the ordinary evaluator/cache/freshness tests and-`bun run check-types`. Adapt installed Vercel fixture routing in this slice so-`bun run test:installed` remains green; native fixtures share the same transport.-Review the complete stdout packet and a request diff with only allowed transport-changes. Record results in `assets/evaluator-verification.md`.--## Decision budget--Delegated: preset/type naming, error-inspection helper placement and cache protocol-token spelling. Error classification must be justified by slice 1's observations;-the retry policy itself is not delegated. No concurrency serialization, new-retrieval logic or response truncation to pass parity. User feedback could change-the chosen provider architecture; absent that, proceed on the frozen gates.diff --git a/specs/provider-support/slices/03-auth.md b/specs/provider-support/slices/03-auth.mddeleted file mode 100644index 1529721..0000000--- a/specs/provider-support/slices/03-auth.md+++ /dev/null@@ -1,84 +0,0 @@-# 3. Save a provider and complete the installed journey--**Unlock:** users can choose any supported service once and use ordinary `jg`.-Depends on slice 2. Status: complete; see [verification](../assets/auth-verification.md).--## API and interaction--The auth module owns `Credentials = {provider: ProviderId; apiKey: string}` and a-single saved-credentials loader replacing `loadApiKey`. Core owns provider IDs and-labels; auth owns config path/key validation and filesystem effects. Search and-doctor pass the loaded record to the same evaluator. No separate resolution tree.--Interactive `jg auth` uses the existing prompt library: select Vercel AI Gateway,-TypeSafe or OpenRouter, then enter the provider's hidden key. Noninteractive auth-requires `--provider` and `--stdin` together; reject unknown IDs, bare `--stdin`,-bare `--provider`, and provider flags on every other command. Errors direct users-to the right setup without printing a key. Both prompt and result use stdout.--Retain the current config location and private atomic replacement. Save only-`{provider, apiKey}` after both inputs validate; cancel/invalid input leaves the-previous record untouched. Preserve the 8 KiB key/input bound, whitespace rules,-0700 directory and 0600 file permissions. Check cancellation before replacement.-No network during auth; successful text names the provider and points to doctor.--Absent provider property normalizes in memory to Vercel. A present invalid value-errors. Reads never rewrite bytes. Remove production API-key/endpoint/model env-overrides and missing-key guidance recommending them. XDG paths continue working.--Doctor retains its synthetic question, identifies the saved provider on success-and actionable failure, and never uploads the user's repository. Search stdout-and research policy remain unchanged. Reauth is the only supported way to replace-the selected service; no per-query switching or failure fallback.--## Installed acceptance--Use real installed npm tarball binaries in the existing Node-only, network-isolated-Docker harness. The preload redirects approved destinations to actual local HTTP;-it does not fake the evaluator. All cases live in normal test entry points:--- For each provider: stdin auth → inspect exact private record → doctor → search- → repeated cache hit; replace auth with another provider and prove a new call- using that provider/key. Auth itself makes zero calls.-- Providerless legacy record: doctor/search use Vercel and file bytes stay intact.- Invalid present provider fails before HTTP, including null and empty string.-- Conflicting environment keys/URLs/models cannot override a valid record;- environment-only setup cannot authenticate. Missing-credential fixtures use an- empty isolated config directory, not an empty environment key. No saved key leaks in diagnostics.-- Invalid key/provider, stdin overflow, prompt cancel and SIGINT preserve previous- bytes and clean temporary credential files. Use a PTY for interactive provider- ordering, hidden entry and cancellation; it is test tooling, not a runtime- dependency. Exercise the noninteractive cases through the installed CLI too.-- Retain existing filesystem, source freshness, partial failure, signal/EPIPE,- stdout/stderr and skill-installer coverage; no alternate provider traffic after- authentication/network failure.--Expand the native smoke selector and asserted pass count so all new provider-journeys actually execute. Copy the preload in normal/prebuilt Docker paths and-native setup; keep it outside packaged files. The existing package must still run-with Node alone, no host Python/Bun/rg/compiler requirement.--## Onboarding and review--Update root/packaged README, `docs/architecture.md`, help, `.env.example` if obsolete, and the shipped-`skills/jevgrep/SKILL.md` setup section together. Explain provider-first auth,-replacement, legacy saved-key behavior and removal of environment-only setup.-Keep `jg skill` / `npx skills add dzhng/jevgrep --skill jevgrep` installation easy-to find and CLI installation guidance intact. Never ask an agent to request a key-in chat. Preserve the frozen research body, not merely its meaning.--Human review surface: short auth/doctor transcript with fixture keys redacted and-the complete search packet. Save verification in `assets/auth-verification.md`.-Run auth/parser tests, `bun run verify`, format check and native packaged smoke.-For live doctor use isolated saved records and authorized local credentials only;-record provider/date/model when present and success/error without secrets. Clearly-separate unavailable credentials from offline pass results. Missing live access-does not block remaining implementation or authorize a provider fallback.--## Decision budget--Delegated: prompt wording, test PTY tooling and helper layout. Picker order is-Vercel, TypeSafe, OpenRouter; no additional account UX. The documented automation-grammar and saved-data compatibility are fixed. Human feedback can adjust wording-or report a service incompatibility; passing offline evidence permits proceeding-without waiting for approval.diff --git a/specs/provider-support/slices/04-tooling.md b/specs/provider-support/slices/04-tooling.mddeleted file mode 100644index 9853fe3..0000000--- a/specs/provider-support/slices/04-tooling.md+++ /dev/null@@ -1,67 +0,0 @@-# 4. Preserve maintained eval routing and close verification--**Unlock:** the maintained harness can observe the actual installed product after-environment-based auth is removed. Depends on slice 3. Status: complete; see [verification](../assets/final-verification.md).--## Seam and scope--Update only the maintained `evals/implementation/swebench/installed.py`, broker-and their tests/supporting fixtures. Historical frozen runners, traces, result-files, baseline records and studies are immutable. Do not run an agent benchmark.-The coding-agent inference path remains unchanged; this slice concerns Jev calls.--The existing harness remains Vercel-backed; adding a benchmark provider matrix is-out of scope. Prepare an ordinary saved `{provider: "vercel", apiKey: brokerToken}`-record in the isolated treatment container instead of API-key environment auth.-Reuse the externally injected Node routing seam to send the exact Vercel native-endpoint to model-egress. The broker keeps the real key, validates the permitted-path/model and forwards to Vercel's documented TypeSafe-compatible route. It must-not allow an agent-selected origin or leak real credentials into the container.--Keep the preload/routing configuration harness-owned and include its bytes in-the maintained candidate's freeze/identity inputs. It must not affect unrelated-Codex inference requests. Runtime source bytes, skill and packed artifact remain-the actual candidate; no patched CLI or hidden public env override. Frozen old-studies retain their original broker code rather than adding runtime legacy paths-to the new broker for hypothetical compatibility.--Raw request/response observations remain external to the CLI and exclude auth-headers. Decode available native billing/usage from retained bodies with explicit-schema fixtures. Preserve support in the analysis decoder for historical Gateway-observations, because reading old results is a real consumer. Do not rewrite them.-Count missing/unmatched responses honestly: known subtotal can coexist with an-unknown full total. Zero is valid only when explicitly observed, not a default.-Jev cost/tokens never enter scored coding-agent cost; keep reporting available Jev-observations separately. There is no token-count or timing optimization here.--## Offline verification and final artifact--- Extend maintained Python tests for saved credentials, preload freezing, accepted- route/model, rejection of wrong targets/tokens, raw capture and billing decode.- Use local fake upstreams; no paid calls or baseline reruns.-- Check Vercel native `provider_metadata.gateway.cost`/generation ID, historical- `providerMetadata.gateway.cost`, snake-case usage, missing/invalid billing and- incomplete request logs. OpenRouter billing fixtures may cover `usage.cost`- without expanding the benchmark runner to additional providers.-- Run the Python suite through `bun run test`; all normal regression suites must- still pass. Run `bun run verify` and `bun run format:check` after final changes.-- Pack one final candidate using existing release tooling and record SHA-256.- Run `bash scripts/test-installed.sh --prebuilt <tarball>` and- `node scripts/test-native.mjs --prebuilt <tarball>` against those same bytes.- These are verification operations, not a release/tag/npm publish.-- Review the full change with the repo review workflow and an independent Codex- review. Record actual gate results and per-provider live-check status in- `assets/final-verification.md`, then update the README handoff/checklist.--Human review surface: small synthetic receipt showing agent cost and separate-known/unknown Jev observations, plus the final verification record. No newly-measured savings or success-rate claim. Report any live-provider access still-unverified in final delivery.--## Decision budget--Delegated: fixture organization, exact preload mount location and decoder helper-names. Credential replacement and network routing remain external harness-operations; the product API cannot grow to simplify tests. A route requiring a-new product setting is a plan violation to resolve, not an implicit permission.-User feedback could authorize a future benchmark/release, but silence does not.diff --git a/test/reference/transport.test.ts b/test/reference/transport.test.tsindex d0ea332..793ad91 100644--- a/test/reference/transport.test.ts+++ b/test/reference/transport.test.ts@@ -12,6 +12,7 @@ test("wire normalization retains every corpus field and rejects unknown envelopemethod: "POST",headers: {authorization: "Bearer fixture",+ "content-type": "application/json","x-jevgrep-original-url": preset.url,},});@@ -19,7 +20,11 @@ test("wire normalization retains every corpus field and rejects unknown envelopeconst original = JSON.parse(serialized);const historical = new Request("http://localhost/v4/ai/evaluation-model", {method: "POST",- headers: { authorization: "Bearer fixture", "ai-model-id": "typesafe-ai/jev" },+ headers: {+ authorization: "Bearer fixture",+ "ai-model-id": "typesafe-ai/jev",+ "content-type": "application/json",+ },});expect(JSON.stringify(transport.decode(historical, original, false))).toBe(serialized);const wire = {@@ -33,6 +38,9 @@ test("wire normalization retains every corpus field and rejects unknown envelope),};expect(JSON.stringify(transport.decode(request, wire, true))).toBe(serialized);+ const invalidContentType = new Request(request);+ invalidContentType.headers.set("content-type", "text/plain");+ expect(() => transport.decode(invalidContentType, wire, true)).toThrow();expect(() => transport.decode(request, { ...wire, extra: true }, true)).toThrow();const id = Object.keys(wire.questions)[0]!;expect(() =>diff --git a/test/reference/transport.ts b/test/reference/transport.tsindex 96a58b4..44596a6 100644--- a/test/reference/transport.ts+++ b/test/reference/transport.ts@@ -39,6 +39,7 @@ export async function referenceTransport(provider: Provider = "vercel", key = "fproduction: boolean,) {expect(request.method).toBe("POST");+ expect(request.headers.get("content-type")).toMatch(/^application\/json(?:;|$)/);expect(request.headers.get("authorization")).toBe(`Bearer ${key}`);const body = raw as {model?: string;