Skip to content

Claude Code. Any Model. The most powerful AI coding agent now speaks every language.

release: v9.4.0 — OpenCode Zen User-Agent, ReasoningControl none, hermetic catalog (#258)

Jack Rudenko13c858e68389
diff --git a/ai-docs/architecture/adapters.md b/ai-docs/architecture/adapters.md
index 263b9484..1cc50cc6 100644
--- a/ai-docs/architecture/adapters.md
+++ b/ai-docs/architecture/adapters.md
@@ -55,6 +55,25 @@ upstream's routing identity, so under `serve` every sessionless conversation sha
one `x-opencode-session`. OpenCode's error text calls a missing header a routing
inefficiency, so a shared value is not expected to fail, but it is less precise.
+### Zen also needs a User-Agent, and its two routes proved it separately
+
+`OpenCodeZenTransport` sends `User-Agent: claudish/<version>` beside the session
+id. OpenCode's docs ask a client to identify itself "rather than a generic SDK or
+HTTP-library name", and this relay enforces it: `providers/model-discovery.ts`
+records that a UA-less roster request to Zen Go answers `403 error code: 1010`,
+Cloudflare's browser-integrity block, while the identical request carrying one
+returns 200 with 26 models (measured 2026-08-18). The chat path sits behind the
+same edge and went without a UA until 2026-09-15.
+
+Read the two together, because the failure modes are unalike and each is mistaken
+for a credential problem. A missing UA is a Cloudflare 403 at the edge, before the
+relay sees the request. A missing session id is a 400 from the relay itself. Both
+send a user to check a key that is fine.
+
+Precedence is the same on both routes: the generated headers are written FIRST and
+the provider definition's own `headers` merge over them, so an endpoint that pins
+either value keeps it. Auth is applied last and nothing can displace it.
+
## Composition in ComposedHandler
```
ComposedHandler = FormatConverter (explicit adapter) + ModelTranslator (auto-selected) + ProviderTransport
diff --git a/ai-docs/architecture/testing.md b/ai-docs/architecture/testing.md
index 3644c026..0c61830e 100644
--- a/ai-docs/architecture/testing.md
+++ b/ai-docs/architecture/testing.md
@@ -13,6 +13,57 @@
**Adding regression tests**: After extracting fixtures from a debug log, add a `describe("Regression: <model>")` block. Template is at the bottom of the test file.
+## Three shared resources a test must never reach, and why prevention beats restore
+
+`scripts/guard-real-config.ts` — the thing `bun run test:safe` runs the suite
+under — does two separate jobs, and conflating them is how each keeps decaying.
+
+**It SNAPSHOTS files.** `GUARDED` lists `~/.claudish/config.json` and
+`~/.claudish/all-models.json`. Each is read before the run and compared after.
+Present and changed → restored from the snapshot. Absent before and present
+after → DELETED, because on a clean machine a test fixture that survives becomes
+the machine's first real catalog, and a cold cache is indistinguishable from a
+poisoned one to every caller. Present but unreadable → it refuses and says so,
+which is why `snapshot()` establishes presence with `existsSync` separately from
+reading the bytes: folding "unreadable" into "absent" would make the delete
+branch destroy the file.
+
+**It PREVENTS three things outright,** via env vars set on the child:
+
+| Variable | Stops | Why prevention, not restore |
+|---|---|---|
+| `CLAUDISH_DISABLE_KEYCHAIN=1` | `security` against the login keychain | A keychain mutation cannot be snapshotted the way JSON bytes can |
+| `CLAUDISH_DISABLE_OP=1` | 1Password handshakes | Denials suppress authorization machine-wide for 15s, for every process |
+| `CLAUDISH_DISABLE_CATALOG_WARM=1` | the live hosted-catalog fetch | The restore returns the bytes; it cannot return the hermeticity |
+
+The third was added 2026-09-15 and is gated inside `refreshCatalog()` rather than
+`warmCatalog()`, because `ensureCatalogReady()` refreshes through the same
+function — a gate on one of two entry points is how a gate stops being true. It
+returns `reason: "disabled"`, never `"network"`: a caller that logs "the catalog
+could not be reached" when nothing tried to reach it sends the reader to debug a
+working connection.
+
+**What it caught, and how the leak hid.** `createProxyServer` fires
+`warmCatalog().catch(() => {})` unconditionally (`proxy-server.ts:1193`), so
+`handlers/explicit-spec-no-credential.test.ts` live-fetched the hosted catalog
+and rewrote the real `all-models.json` with a fresh `lastUpdated` — even though
+that file's own header says "no network call happens", which was true of the
+upstream model call and wrong about the proxy.
+
+It hid itself by RACING. A sibling file cleaned up with
+`_setCatalogEntriesForTest(null)`, and `null` is not a reset: the read path
+short-circuits on `!== undefined`, so it is a sticky process-wide "catalog is
+empty" override. Every later lookup returned instantly, the process exited
+before the ~2s fetch resolved, and the write never landed. Whether the leak was
+visible depended on which files ran alongside it. `_resetCatalogClient()` is the
+real teardown; `_setCatalogEntriesForTest(null)` belongs only inside a test body
+that wants the cold-catalog state.
+
+Evidence: that file tripped the guard 3/3 times alone at ~2.0s, ran clean at 55ms
+against a dead catalog host, and runs at 66ms with the kill switch. This is the
+same non-hermeticity that turned two DeepSeek tests red mid-release when the
+hosted catalog changed its answer.
+
## A gate must not also gate its own diagnostic
`e2e-channel.test.ts` Group 2 spawns a real `claude -p`, so it needs a working credential.
@@ -96,3 +147,22 @@ that compares against a runtime-provided oracle is only as stable as the runtime
Distinct from the credential-gated live tests, which skip in CI and are documented as
non-blocking; this one is hermetic and still environmental.
+
+**Since 2026-09-15 the two budgeted sweeps enforce this themselves.** They read
+`bun-version` out of `test.yml` at test time and `test.skipIf` when `Bun.version`
+differs, naming both versions in the skip. Three details are load-bearing:
+
+- The pin is PARSED, never copied. A second hardcoded `"1.3.10"` in the test file
+ would drift from the workflow, which is the failure being fixed.
+- A parse failure FAILS OPEN and runs the tests. A gate that silently disables
+ coverage when it cannot find its input is worse than no gate.
+- Only the two oracle-calibrated sweeps are gated. The first test in the block
+ asserts five literal widths, is not oracle-dependent, and runs everywhere.
+
+The 2026-09-04 measurement above named `U+2630–U+2637` and `U+268A–U+268F`. The
+full sweep on bun 1.4.0 is wider: `total 3017`, of which the `other` bucket —
+budgeted at 17 — is 576, almost all `U+1160..U+11FF`, conjoining Hangul Jamo that
+1.4.0 measures as zero-width. Same cause, larger blast radius than first recorded.
+
+When the pin is bumped, RE-BASELINE the budgets against the new oracle. Never
+widen them to clear a red run: the budget is the entire assertion.
diff --git a/ai-docs/release.md b/ai-docs/release.md
index 97592bdf..778a05af 100644
--- a/ai-docs/release.md
+++ b/ai-docs/release.md
@@ -127,9 +127,20 @@ Two constraints hold regardless of who authorises:
TODO — not settled by detection, and worth answering on the next release:
- whether a failed magmux download from `MadAppGang/magmux` should block a release
- or pin to a known-good magmux tag instead of `latest`;
-- whether the two known-red `displayWidth` oracle tests in
- `packages/cli/src/tui/viz/color.test.ts` should be quarantined, since they fail
- on `main` and every release ships over them.
-
-verified: 2026-09-14 @ 22cef36
+ or pin to a known-good magmux tag instead of `latest`.
+
+SETTLED 2026-09-15 — the two `displayWidth` tests in
+`packages/cli/src/tui/viz/color.test.ts`. They were NOT flaky and must NOT be
+quarantined. They compare `fallbackClusterWidth` against `Bun.stringWidth`, which
+is a LIVE ORACLE whose Unicode tables move between Bun releases, and their budgets
+(1081 total, 17 in `other`) are calibrated to one build. `test.yml` pins CI to Bun
+`1.3.10`; a developer on Bun `1.4.0` measures `total 3017` with `other` at 576,
+almost all `U+1160..U+11FF` — conjoining Hangul Jamo, which 1.4.0 measures as
+zero-width. No claudish code changed. The tests now read the pin out of
+`test.yml` and skip with a message naming both versions when the running Bun is
+not the pinned one, so the local red is gone and the gate's coverage is not.
+
+When the Bun pin is bumped, RE-BASELINE those budgets against the new oracle.
+Never widen them to clear a red run: the budget is the whole assertion.
+
+verified: 2026-09-15 @ 22cef36
diff --git a/ai-docs/reports/probe-reports-missing-key-that-works-20260915.md b/ai-docs/reports/probe-reports-missing-key-that-works-20260915.md
new file mode 100644
index 00000000..b2dd5321
--- /dev/null
+++ b/ai-docs/reports/probe-reports-missing-key-that-works-20260915.md
@@ -0,0 +1,62 @@
+# `--probe zengo@` reports a missing key while that key authenticates
+
+Measured 2026-09-15 against the build at `v9.4.0`'s candidate commit, on a
+machine where `zengo@` works.
+
+## What was observed
+
+```
+claudish --probe zengo@minimax-m3
+→ opencode-zen-go · 0/1 live
+→ ○ missing
+→ Key —
+```
+
+Minutes later, on the same machine and the same build, a real run through the
+same provider succeeded:
+
+```
+[Proxy] Created OpenCode Zen Go (composed): minimax-m3
+[Zen Go] Calling API: https://opencode.ai/zen/go/v1/chat/completions
+[Zen Go] Response status: 200
+```
+
+Zero `[Fallback]` lines. The credential is present in the macOS Keychain under
+the item `claudish` / account `OPENCODE_GO_API_KEY`.
+
+## Why this is NOT the failure it resembles
+
+This looks like the credential-name drift that produced the v7.11.0 Test All
+regression, and it is not. The name matches exactly:
+`providers/provider-definitions.ts:849` declares
+`apiKeyEnvVar: "OPENCODE_GO_API_KEY"`, and `OPENCODE_GO_API_KEY` is the account
+name stored in the Keychain. The same line's `siblingKeyEnvVars: ["OPENCODE_API_KEY"]`
+is also unrelated — that is the metered Zen key, a different credential.
+
+So the probe is not looking for the wrong name. It is not consulting the
+Keychain backend for this provider at all, or it is consulting it in a way that
+answers "absent" for a key that reads fine.
+
+## Why it matters more than a cosmetic wrong label
+
+`○ missing` and `Key —` are a claim about the machine, and the machine was never
+asked — the same shape as the gate-that-gates-its-own-diagnostic failure recorded
+in `ai-docs/architecture/testing.md`. It is indistinguishable from the output on
+a machine that genuinely has no key, so it sends a user to fix a working
+credential. `ai-docs/architecture/keychain.md` records the distinction this
+probably turns on: enumerate-for-presence and read-for-value are different
+operations against `security(1)` and do not always agree.
+
+## Not yet established
+
+- Whether both probe UIs are affected. `ai-docs/reports/`-adjacent notes record
+ that claudish has TWO separate probe paths (the config TUI and `--probe`);
+ only `--probe` was observed here.
+- Whether other Keychain-backed providers show the same thing, or only
+ `opencode-zen-go`.
+- Whether `CLAUDISH_DISABLE_KEYCHAIN` was somehow in the environment. It was not
+ set by the caller, but this was not asserted directly.
+
+Reproduce before fixing: run `--probe` against a Keychain-only provider, then run
+the same provider for real, and compare. Do not test against the real Keychain by
+killing `security` processes — see `ai-docs/architecture/keychain.md`.
diff --git a/package.json b/package.json
index 0f301261..440e3022 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "claudish-monorepo",
- "version": "9.3.2",
+ "version": "9.4.0",
"private": true,
"description": "Monorepo for Claudish - Run Claude Code with any model",
"type": "module",
@@ -20,7 +20,7 @@
"typecheck": "bun run --cwd packages/cli typecheck && bun run --cwd packages/macos-bridge typecheck",
"lint": "bun run --cwd packages/cli lint && bun run --cwd packages/macos-bridge lint",
"format": "bun run --cwd packages/cli format && bun run --cwd packages/macos-bridge format",
- "test": "bun run --cwd packages/cli test && bun run --cwd packages/macos-bridge test",
+ "test": "bun run --cwd packages/cli test && bun run --cwd packages/macos-bridge test && bun test ./scripts/*.test.ts",
"test:safe": "bun run scripts/guard-real-config.ts -- bun run test",
"test:mcp": "bun run packages/cli/src/mcp-e2e/runner.ts",
"test:mcp:op": "bun run packages/cli/src/mcp-e2e/runner.ts --group op",
diff --git a/packages/cli/package.json b/packages/cli/package.json
index 058b1589..a37495b6 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -1,6 +1,6 @@
{
"name": "claudish",
- "version": "9.3.2",
+ "version": "9.4.0",
"description": "Run Claude Code with any model - OpenRouter, Ollama, LM Studio & local models",
"type": "module",
"main": "./dist/index.js",
diff --git a/packages/cli/src/adapters/effort-mapping.test.ts b/packages/cli/src/adapters/effort-mapping.test.ts
index 095b657c..825e7759 100644
--- a/packages/cli/src/adapters/effort-mapping.test.ts
+++ b/packages/cli/src/adapters/effort-mapping.test.ts
@@ -740,14 +740,14 @@ describe("DeepSeek V4 reasoning_effort + thinking", () => {
});
test("catalog reasoning opinion overrides the V4 alias name rule", () => {
- // Measured 2026-09-14: deepseek-chat advertises supported:false, control:"none";
- // control is omitted here only because ReasoningControl does not model "none".
+ // Mirrors the live catalog shape measured 2026-09-14: deepseek-chat
+ // advertises supported:false, control:"none".
const cleanupOpinionCatalog = seedDefaultCatalog([
{
modelId: "deepseek-chat",
aliases: [],
sources: {},
- reasoning: { supported: false },
+ reasoning: { supported: false, control: "none" },
},
]);
diff --git a/packages/cli/src/handlers/advisor-routing-and-state.test.ts b/packages/cli/src/handlers/advisor-routing-and-state.test.ts
index cc82d000..fdbea47c 100644
--- a/packages/cli/src/handlers/advisor-routing-and-state.test.ts
+++ b/packages/cli/src/handlers/advisor-routing-and-state.test.ts
@@ -1,5 +1,5 @@
import { afterEach, beforeEach, describe, expect, it } from "bun:test";
-import { _setCatalogEntriesForTest } from "../providers/catalog-client.js";
+import { _resetCatalogClient, _setCatalogEntriesForTest } from "../providers/catalog-client.js";
import {
ADVISOR_STUB_PATHS,
_debug_resetTrackedAdvisorIds,
@@ -56,11 +56,15 @@ afterEach(() => {
});
beforeEach(() => {
+ // Reset process-global state, then pin an empty catalog so these tests never
+ // fall through to the developer's real disk cache.
+ _resetCatalogClient();
_setCatalogEntriesForTest(null);
});
afterEach(() => {
- _setCatalogEntriesForTest(null);
+ // `null` is a sticky empty-catalog override, not a reset between tests.
+ _resetCatalogClient();
});
describe("advisorRouteFor", () => {
diff --git a/packages/cli/src/launcher/catalog-warm.test.ts b/packages/cli/src/launcher/catalog-warm.test.ts
index cbbf9f95..37d0a507 100644
--- a/packages/cli/src/launcher/catalog-warm.test.ts
+++ b/packages/cli/src/launcher/catalog-warm.test.ts
@@ -293,7 +293,21 @@ describe("warmCatalogIfNeeded", () => {
expect(refreshSpy).toHaveBeenCalledTimes(1);
});
- test("missing cache + fetch_failed → 'hard_fail' with verbatim error on stderr", async () => {
+ test("missing cache + disabled refresh → 'skipped' without hard-fail copy", async () => {
+ mockReadResult = null;
+ mockRefreshOutcome = { kind: "fetch_failed", reason: "disabled" };
+ const config = makeConfig({ model: "gpt-4o", quiet: false });
+
+ const result = await warmCatalogIfNeeded(config, { now: NOW });
+
+ expect(result).toBe("skipped");
+ expect(refreshSpy).toHaveBeenCalledTimes(1);
+ const stderrText = stderrChunks.join("");
+ expect(stderrText).toContain("Catalog refresh disabled");
+ expect(stderrText).not.toContain("Error: cannot reach model catalog and no cached copy found.");
+ });
+
+ test("missing cache + genuine network failure → 'hard_fail' with verbatim error", async () => {
mockReadResult = null;
mockRefreshOutcome = { kind: "fetch_failed", reason: "network" };
const config = makeConfig({ model: "gpt-4o" });
diff --git a/packages/cli/src/launcher/catalog-warm.ts b/packages/cli/src/launcher/catalog-warm.ts
index 31768f79..3144eb13 100644
--- a/packages/cli/src/launcher/catalog-warm.ts
+++ b/packages/cli/src/launcher/catalog-warm.ts
@@ -273,6 +273,29 @@ export async function warmCatalogIfNeeded(
return "ok";
}
+ // `disabled` is not a failure and must never reach the branches below. Nobody
+ // attempted a fetch, so the cache state is irrelevant: with no cache at all
+ // the `missing` branch would print HARD_FAIL_MESSAGE — "cannot reach model
+ // catalog… Check network connection" — and `index.ts` turns `hard_fail` into
+ // `process.exit(1)`. That sends a reader to debug a working connection over a
+ // condition their own harness created, which is the entire reason this reason
+ // exists as a distinct value rather than reusing `network`.
+ //
+ // It reaches here through `CLAUDISH_DISABLE_CATALOG_WARM=1`, which
+ // `scripts/guard-real-config.ts` sets on the whole test run — and the e2e
+ // suites spawn `src/index.ts` as a child, which inherits it. So on a machine
+ // with no `~/.claudish/all-models.json`, the untreated path fails those tests
+ // with a network diagnosis.
+ //
+ // "skipped" is the accurate answer, and it is the same one
+ // `--models-skip-update` already produces: a refresh nobody attempted.
+ if (outcome.reason === "disabled") {
+ if (!config.quiet) {
+ process.stderr.write(" Catalog refresh disabled (CLAUDISH_DISABLE_CATALOG_WARM=1).\n");
+ }
+ return "skipped";
+ }
+
// Fetch failed. Decide based on prior cache state.
if (state === "stale") {
const ageMs = now.getTime() - Date.parse(cache!.lastUpdated);
diff --git a/packages/cli/src/providers/all-models-cache.ts b/packages/cli/src/providers/all-models-cache.ts
index 09f81213..e6157fda 100644
--- a/packages/cli/src/providers/all-models-cache.ts
+++ b/packages/cli/src/providers/all-models-cache.ts
@@ -37,12 +37,24 @@ import type { AggregatorEntry } from "../model-loader.js";
* - `effort` — a discrete level from the model's own `efforts` list.
* - `adaptive` — the model chooses; a level may optionally be suggested.
* - `budget` — a token budget (`budget_tokens`).
+ * - `none` — the model exposes no knob. The catalog pairs it with
+ * `supported: false`, and `query-handler.ts` also uses it as the default for
+ * an entry that carries no `reasoning` block at all, so it is the single
+ * commonest value on the wire rather than an edge case.
*
* Kept as a string union of what Firebase currently emits, but consumers must
* treat an unrecognized value as "no information" rather than an error —
* the catalog is external data and may grow new kinds.
+ *
+ * `none` was missing here until 2026-09-15 while the backend had always sent
+ * it (`models-index/functions/src/schema.ts:105` declares the same five, and
+ * `schema-runtime.ts:610` validates against them). Nothing failed at runtime,
+ * because every consumer tests for the value it wants — `control === "effort"`,
+ * `control === "budget"` — so an unlisted member falls through to the right
+ * answer. It failed at the type level instead: a test fixture could not write
+ * the value the live catalog returns, which is how the gap surfaced.
*/
-export type ReasoningControl = "toggle" | "effort" | "adaptive" | "budget";
+export type ReasoningControl = "toggle" | "effort" | "adaptive" | "budget" | "none";
/**
* Per-model reasoning capability from the Firebase slim catalog.
diff --git a/packages/cli/src/providers/catalog-client.test.ts b/packages/cli/src/providers/catalog-client.test.ts
index 98ae3a6f..daab3081 100644
--- a/packages/cli/src/providers/catalog-client.test.ts
+++ b/packages/cli/src/providers/catalog-client.test.ts
@@ -1,7 +1,79 @@
-import { describe, expect, test } from "bun:test";
-import { resolveTargetForCatalog } from "./catalog-client.js";
+import { afterEach, beforeEach, describe, expect, mock, test } from "bun:test";
+import {
+ _resetCatalogClient,
+ catalogWarmDisabledFor,
+ refreshCatalog,
+ resolveTargetForCatalog,
+} from "./catalog-client.js";
import { parseModelSpec } from "./model-parser.js";
+const realFetch = globalThis.fetch;
+let previousDisableCatalogWarm: string | undefined;
+
+beforeEach(() => {
+ previousDisableCatalogWarm = process.env.CLAUDISH_DISABLE_CATALOG_WARM;
+ _resetCatalogClient();
+});
+
+afterEach(() => {
+ if (previousDisableCatalogWarm === undefined) {
+ delete process.env.CLAUDISH_DISABLE_CATALOG_WARM;
+ } else {
+ process.env.CLAUDISH_DISABLE_CATALOG_WARM = previousDisableCatalogWarm;
+ }
+ globalThis.fetch = realFetch;
+ _resetCatalogClient();
+});
+
+describe("refreshCatalog catalog-warm kill switch", () => {
+ // This regression is a race: a sibling test's sticky empty-catalog override
+ // could let the process exit before a live fetch reached the developer's
+ // cache. The fetch assertion proves the kill switch returns before that path.
+ test("returns disabled before network or disk side effects", async () => {
+ process.env.CLAUDISH_DISABLE_CATALOG_WARM = "1";
+ const fetchStub = mock(
+ async () =>
+ new Response(
+ JSON.stringify({
+ models: [
+ {
+ modelId: "offline-test-model",
+ aliases: [],
+ sources: { test: { externalId: "test/offline-test-model" } },
+ },
+ ],
+ plans: [],
+ }),
+ { status: 200, headers: { "Content-Type": "application/json" } }
+ )
+ );
+ globalThis.fetch = fetchStub as unknown as typeof fetch;
+
+ const outcome = await refreshCatalog(100);
+
+ expect(outcome).toEqual({ kind: "fetch_failed", reason: "disabled" });
+ expect(fetchStub).not.toHaveBeenCalled();
+ });
+
+ test('recognizes only "1" as disabled', () => {
+ expect(catalogWarmDisabledFor("1")).toBe(true);
+ });
+
+ // Pass negative values as arguments: this avoids writing a process-global
+ // another file's detached warmCatalog() may read, and a required parameter is
+ // the only way to express "explicitly unset" because a default fires on undefined.
+ for (const [label, switchValue] of [
+ ['"true"', "true"],
+ ['"0"', "0"],
+ ['""', ""],
+ ["undefined", undefined],
+ ] as const) {
+ test(`does not disable catalog warm for ${label}`, () => {
+ expect(catalogWarmDisabledFor(switchValue)).toBe(false);
+ });
+ }
+});
+
describe("resolveTargetForCatalog", () => {
test("rewrites a changed explicit MiniMax spec and returns its resolution", () => {
const resolution = {
diff --git a/packages/cli/src/providers/catalog-client.ts b/packages/cli/src/providers/catalog-client.ts
index 68ccabcb..4031e8de 100644
--- a/packages/cli/src/providers/catalog-client.ts
+++ b/packages/cli/src/providers/catalog-client.ts
@@ -79,7 +79,7 @@ export type DiskCache = DiskCacheV2;
*/
export type RefreshOutcome =
| { kind: "refreshed"; modelCount: number }
- | { kind: "fetch_failed"; reason: "timeout" | "network" | "http_error" | "empty" };
+ | { kind: "fetch_failed"; reason: "timeout" | "network" | "http_error" | "empty" | "disabled" };
/** Result of resolving a user-typed model name for a provider. */
export interface ModelResolutionResult {
@@ -385,7 +385,64 @@ export function logResolution(
* disk cache, and marks the warm as settled. On any failure leaves both caches
* untouched and returns the reason. Never throws.
*/
+/**
+ * The RULE for `CLAUDISH_DISABLE_CATALOG_WARM`: exactly `"1"` disables the warm.
+ * Every other value — `"true"`, `"0"`, `""`, and unset — leaves it enabled.
+ *
+ * Separated from the environment read, and taking a REQUIRED parameter, because
+ * both halves of that shape matter:
+ *
+ * - Taking the value as an argument is what lets the rule be tested WITHOUT
+ * writing the real `process.env`. That is not a convenience. `proxy-server.ts`
+ * fires an un-awaited `warmCatalog()` on every `createProxyServer`, so a
+ * detached refresh can reach this check at any instant during a suite. A test
+ * that proves "only 1 disables" by assigning `"true"` to the shared variable
+ * opens a window in which such a refresh passes the gate and reads the live
+ * catalog — the leak the switch exists to stop, reintroduced by the test for
+ * it, and intermittent because it depends on file ordering. Measured
+ * 2026-09-15: that is how a full `test:safe` run rewrote the real
+ * `~/.claudish/all-models.json` while every assertion passed.
+ *
+ * - The parameter is required rather than defaulted to the env var, because a
+ * default fires on `undefined` and so cannot express "explicitly unset". The
+ * defaulted version made `catalogWarmDisabled(undefined)` read the real
+ * environment — which the guard sets to `"1"` — so the case asserting that an
+ * unset variable leaves the warm ENABLED could never pass under `test:safe`.
+ */
+export function catalogWarmDisabledFor(value: string | undefined): boolean {
+ return value === "1";
+}
+
+/** The rule above, applied to the live environment. */
+function catalogWarmDisabled(): boolean {
+ return catalogWarmDisabledFor(process.env.CLAUDISH_DISABLE_CATALOG_WARM);
+}
+
export async function refreshCatalog(timeoutMs: number): Promise<RefreshOutcome> {
+ // `CLAUDISH_DISABLE_CATALOG_WARM=1` turns every refresh into a no-op, which is
+ // the same contract `CLAUDISH_DISABLE_KEYCHAIN` and `CLAUDISH_DISABLE_OP` give
+ // the other two shared resources a test must not reach.
+ //
+ // This is the gate for ALL callers rather than one inside `warmCatalog`,
+ // because `ensureCatalogReady` refreshes too and a second entry point is how
+ // the first gate stops being true.
+ //
+ // Measured 2026-09-15: `handlers/explicit-spec-no-credential.test.ts` calls
+ // `createProxyServer`, `proxy-server.ts:1193` fires `warmCatalog()` on every
+ // create, and the fetch below then rewrote the developer's real
+ // `~/.claudish/all-models.json` with a fresh `lastUpdated` — a live network
+ // read inside a suite that is supposed to be hermetic. It hid itself by
+ // RACING: a sibling file that leaves a sticky empty-catalog override lets the
+ // process exit before the ~2s fetch resolves, so whether the leak appeared
+ // depended on which files ran alongside it.
+ //
+ // Returning `disabled` rather than `network` matters. A caller that logs
+ // "the catalog could not be reached" when nobody tried to reach it sends the
+ // reader to debug their connection.
+ if (catalogWarmDisabled()) {
+ return { kind: "fetch_failed", reason: "disabled" };
+ }
+
const plansPromise = fetchSubscriptionPlans(timeoutMs);
let response: Response;
try {
diff --git a/packages/cli/src/providers/transport/opencode-zen.test.ts b/packages/cli/src/providers/transport/opencode-zen.test.ts
index 5f2d1c29..d65d0569 100644
--- a/packages/cli/src/providers/transport/opencode-zen.test.ts
+++ b/packages/cli/src/providers/transport/opencode-zen.test.ts
@@ -70,7 +70,7 @@ describe("conversationKey", () => {
});
describe("OpenCodeZenTransport", () => {
- test("adds the conversation header without replacing base authorization", async () => {
+ test("adds client identity headers without replacing base authorization", async () => {
const request = requestFor(SESSION_ID_A);
const transport = new OpenCodeZenTransport(
remoteProvider("opencode-zen-go"),
@@ -80,9 +80,28 @@ describe("OpenCodeZenTransport", () => {
const headers = await transport.getHeaders(request);
expect(headers.Authorization).toBe("Bearer zen-key");
+ expect(headers["User-Agent"]).toMatch(/^claudish\/\d+\.\d+\.\d+$/);
expect(headers["x-opencode-session"]).toBe(conversationKey(request));
});
+ test("lets provider-defined identity headers override generated values", async () => {
+ const provider = {
+ ...remoteProvider("opencode-zen-go"),
+ headers: {
+ "User-Agent": "pinned",
+ "x-opencode-session": "pinned-session",
+ },
+ };
+ const transport = new OpenCodeZenTransport(provider, "minimax-m3", "zen-key");
+
+ const headers = await transport.getHeaders(requestFor(SESSION_ID_A));
+ expect(headers).toMatchObject({
+ Authorization: "Bearer zen-key",
+ "User-Agent": "pinned",
+ "x-opencode-session": "pinned-session",
+ });
+ });
+
test("derives each header from its request rather than transport instance state", async () => {
const transport = new OpenCodeZenTransport(
remoteProvider("opencode-zen-go"),
diff --git a/packages/cli/src/providers/transport/opencode-zen.ts b/packages/cli/src/providers/transport/opencode-zen.ts
index f71f73d5..ce5af9e7 100644
--- a/packages/cli/src/providers/transport/opencode-zen.ts
+++ b/packages/cli/src/providers/transport/opencode-zen.ts
@@ -14,17 +14,46 @@
*
* The metered `opencode-zen` tier sends it too, on the same docs sentence, but that
* is UNVERIFIED live: there is no OPENCODE_API_KEY on the machine that measured Go.
+ *
+ * The same docs paragraph asks a client to "identify itself with its own user agent
+ * … rather than a generic SDK or HTTP-library name", and this relay enforces that
+ * on its other route: `model-discovery.ts:465-481` records that a UA-less roster
+ * request to Zen Go answers `403 error code: 1010` — Cloudflare's browser-integrity
+ * block — while the identical request carrying one returns 200. The chat path sits
+ * behind the same edge, so it sends the same `claudish/<version>` string. Credit to
+ * @oudivad, who reported both headers together in PR #249; this file took the
+ * session id first and the UA second.
*/
+import { VERSION } from "../../version.js";
import { conversationKey } from "./conversation-key.js";
import { OpenAIProviderTransport } from "./openai.js";
export class OpenCodeZenTransport extends OpenAIProviderTransport {
- /** Derived from `claudeRequest` on every call — never cached on the instance. */
+ /**
+ * Both headers are written BEFORE the base transport's, so auth and any
+ * provider-declared `headers` still win — the precedence `model-discovery.ts`
+ * uses for the roster request on this same relay. No builtin declares either
+ * key today, so nothing changes in practice; it means a custom endpoint that
+ * pins its own value keeps it.
+ *
+ * The two are not equally inert, though. For the User-Agent, losing to a
+ * pinned value is harmless — it is an identity string either way. For
+ * `x-opencode-session` it is a REVERSAL: this header used to be spread after
+ * the base headers and always won. A custom endpoint that pins a static
+ * session id now collapses every conversation onto one upstream routing
+ * identity, silently. That is the right default (an explicit config should
+ * beat a derived value) but it is a real trade, not a no-op.
+ *
+ * The session id is derived from `claudeRequest` on every call, never cached
+ * on the instance: one transport is shared by every request for a model, so
+ * an instance field would leak one conversation's id into the next.
+ */
override async getHeaders(claudeRequest?: unknown): Promise<Record<string, string>> {
return {
- ...(await super.getHeaders()),
+ "User-Agent": `claudish/${VERSION}`,
"x-opencode-session": conversationKey(claudeRequest),
+ ...(await super.getHeaders()),
};
}
}
diff --git a/packages/cli/src/tui/viz/color.test.ts b/packages/cli/src/tui/viz/color.test.ts
index 4bdf6cd7..68591796 100644
--- a/packages/cli/src/tui/viz/color.test.ts
+++ b/packages/cli/src/tui/viz/color.test.ts
@@ -1,6 +1,7 @@
/** Every assertion fails on a SPECIFIC known regression, not on "it runs". All
* `color.ts` output is lowercase `#rrggbb`, so tokens compare through `lo()`. */
import { describe, expect, test } from "bun:test";
+import { readFileSync } from "node:fs";
import { blend1D, blendStops, darken, heatRamp, lighten, pickInk } from "./color";
import {
displayWidth,
@@ -15,6 +16,31 @@ import { ramps, tokens } from "./tokens";
const lo = (s: string) => s.toLowerCase();
const chan = (hex: string) => [1, 3, 5].map((i) => Number.parseInt(hex.slice(i, i + 2), 16));
+const readPinnedBunVersion = (): string | undefined => {
+ try {
+ const workflow = readFileSync(
+ new URL("../../../../../.github/workflows/test.yml", import.meta.url),
+ "utf8"
+ );
+ const lines = workflow.split(/\r?\n/);
+ const setupBunLine = lines.findIndex((line) =>
+ /^\s*-\s+uses:\s*oven-sh\/setup-bun@/.test(line)
+ );
+ if (setupBunLine === -1) return undefined;
+
+ const stepIndent = lines[setupBunLine]!.search(/\S/);
+ for (const line of lines.slice(setupBunLine + 1)) {
+ if (line.trim() === "") continue;
+ if (line.search(/\S/) === stepIndent && /^\s*-\s+/.test(line)) break;
+ const pin = line.match(/^\s*bun-version:\s*"(\d+\.\d+\.\d+)"\s*$/);
+ if (pin) return pin[1];
+ }
+ } catch {
+ // Fail open: an unavailable workflow must not silently disable coverage.
+ }
+ return undefined;
+};
+
/** WCAG 2.x relative luminance and contrast ratio, reimplemented from the spec.
* INDEPENDENT of `color.ts` on purpose — a test that measures contrast with the
* implementation's own luminance function agrees with itself no matter how wrong
@@ -368,7 +394,29 @@ describe("displayWidth fallback — measured against the oracle, with a budget",
expect(fbWidth("\u{1F5A5}")).toBe(1);
});
- test("zero disagreement on every class that reaches a dashboard", () => {
+ // These budgets are calibrated against one Bun build; `Bun.stringWidth` is a live
+ // oracle whose Unicode tables move. Bun 1.4.0 measured total=3017 and
+ // { marks: 2146, unassigned: 246, format: 23, regionalIndicator: 26, other: 576 },
+ // with other sample U+980, U+C80, U+D3A, and U+1160..U+1168, plus band extras
+ // U+2630..U+2637 and U+268A..U+268F. When the pin is bumped, re-baseline the
+ // budgets; do not widen them because of a red local run.
+ const pinnedBunVersion = readPinnedBunVersion();
+ const skipOracleBudgets = pinnedBunVersion !== undefined && Bun.version !== pinnedBunVersion;
+ const oracleBudgetSkipReason = skipOracleBudgets
+ ? `running Bun ${Bun.version}; budgets are pinned to Bun ${pinnedBunVersion}`
+ : "";
+ const oracleBudgetTestNames = [
+ "zero disagreement on every class that reaches a dashboard",
+ "the whole-Unicode disagreement budget holds, by category",
+ ] as const;
+ const oracleBudgetName = (name: string) =>
+ oracleBudgetSkipReason === "" ? name : `${name} — SKIPPED: ${oracleBudgetSkipReason}`;
+ if (skipOracleBudgets) {
+ for (const name of oracleBudgetTestNames)
+ console.warn(`SKIPPED ${name} — ${oracleBudgetSkipReason}`);
+ }
+
+ test.skipIf(skipOracleBudgets)(oracleBudgetName(oracleBudgetTestNames[0]), () => {
const REAL = [
"CPU 78%",
"日本語テスト",
@@ -402,32 +450,37 @@ describe("displayWidth fallback — measured against the oracle, with a budget",
// The full oracle sweep, and the numbers the source comment quotes. These are a
// RATCHET: 11,205 was the old table's score and 1,081 is this one's. Widening a run
// by guess moves this number and fails here.
- test("the whole-Unicode disagreement budget holds, by category", () => {
- const isMark = /^\p{M}/u;
- const isUnassigned = /^\p{Cn}/u;
- const isFormat = /^\p{Cf}/u;
- let total = 0;
- const cat = { marks: 0, unassigned: 0, format: 0, regionalIndicator: 0, other: 0 };
- for (let cp = 0; cp <= 0x10ffff; cp++) {
- if (cp >= 0xd800 && cp <= 0xdfff) continue;
- const c = String.fromCodePoint(cp);
- if (fallbackClusterWidth(c) === Bun.stringWidth(c)) continue;
- total++;
- if (isMark.test(c)) cat.marks++;
- else if (isUnassigned.test(c)) cat.unassigned++;
- else if (isFormat.test(c)) cat.format++;
- else if (cp >= 0x1f1e6 && cp <= 0x1f1ff) cat.regionalIndicator++;
- else cat.other++;
- }
- expect(total).toBeLessThanOrEqual(1081); // the previous hand table scored 11,205
- // None of the residual is reachable from real single-line text: a lone combining
- // mark, a codepoint this engine's Unicode tables predate, a bidi format control,
- // or one half of a flag pair (the PAIR is one cluster and measures 2 correctly).
- expect(cat.marks + cat.unassigned + cat.format + cat.regionalIndicator).toBeGreaterThanOrEqual(
- total - 17
- );
- expect(cat.other).toBeLessThanOrEqual(17);
- }, 30_000);
+ test.skipIf(skipOracleBudgets)(
+ oracleBudgetName(oracleBudgetTestNames[1]),
+ // biome-ignore lint/complexity/noExcessiveCognitiveComplexity: exhaustive oracle categorization
+ () => {
+ const isMark = /^\p{M}/u;
+ const isUnassigned = /^\p{Cn}/u;
+ const isFormat = /^\p{Cf}/u;
+ let total = 0;
+ const cat = { marks: 0, unassigned: 0, format: 0, regionalIndicator: 0, other: 0 };
+ for (let cp = 0; cp <= 0x10ffff; cp++) {
+ if (cp >= 0xd800 && cp <= 0xdfff) continue;
+ const c = String.fromCodePoint(cp);
+ if (fallbackClusterWidth(c) === Bun.stringWidth(c)) continue;
+ total++;
+ if (isMark.test(c)) cat.marks++;
+ else if (isUnassigned.test(c)) cat.unassigned++;
+ else if (isFormat.test(c)) cat.format++;
+ else if (cp >= 0x1f1e6 && cp <= 0x1f1ff) cat.regionalIndicator++;
+ else cat.other++;
+ }
+ expect(total).toBeLessThanOrEqual(1081); // the previous hand table scored 11,205
+ // None of the residual is reachable from real single-line text: a lone combining
+ // mark, a codepoint this engine's Unicode tables predate, a bidi format control,
+ // or one half of a flag pair (the PAIR is one cluster and measures 2 correctly).
+ expect(
+ cat.marks + cat.unassigned + cat.format + cat.regionalIndicator
+ ).toBeGreaterThanOrEqual(total - 17);
+ expect(cat.other).toBeLessThanOrEqual(17);
+ },
+ 30_000
+ );
});
// ---------------------------------------------------------------------------
diff --git a/packages/cli/src/version.ts b/packages/cli/src/version.ts
index 9eb91e11..560e3686 100644
--- a/packages/cli/src/version.ts
+++ b/packages/cli/src/version.ts
@@ -1,2 +1,2 @@
// Auto-generated by scripts/generate-version.ts — do not edit
-export const VERSION = "9.3.2";
+export const VERSION = "9.4.0";
diff --git a/scripts/guard-real-config.test.ts b/scripts/guard-real-config.test.ts
new file mode 100644
index 00000000..5fb44460
--- /dev/null
+++ b/scripts/guard-real-config.test.ts
@@ -0,0 +1,108 @@
+import { describe, expect, test } from "bun:test";
+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+const GUARD_SCRIPT = join(import.meta.dir, "guard-real-config.ts");
+const decoder = new TextDecoder();
+
+interface ScratchHome {
+ home: string;
+ claudishDir: string;
+ configPath: string;
+}
+
+function withScratchHome(run: (fixture: ScratchHome) => void): void {
+ const home = mkdtempSync(join(tmpdir(), "claudish-guard-test-"));
+ const claudishDir = join(home, ".claudish");
+ mkdirSync(claudishDir);
+ try {
+ run({ home, claudishDir, configPath: join(claudishDir, "config.json") });
+ } finally {
+ rmSync(home, { recursive: true, force: true });
+ }
+}
+
+function runGuard(fixture: ScratchHome, commandSource: string): ReturnType<typeof Bun.spawnSync> {
+ return Bun.spawnSync(
+ [process.execPath, "run", GUARD_SCRIPT, "--", process.execPath, "-e", commandSource],
+ {
+ env: {
+ ...process.env,
+ HOME: fixture.home,
+ GUARD_TEST_CLAUDISH_DIR: fixture.claudishDir,
+ GUARD_TEST_CONFIG_PATH: fixture.configPath,
+ },
+ stdout: "pipe",
+ stderr: "pipe",
+ }
+ );
+}
+
+function stderrOf(result: ReturnType<typeof Bun.spawnSync>): string {
+ return decoder.decode(result.stderr);
+}
+
+describe("guard-real-config", () => {
+ test("moves a file created during the command to .guard-created and exits 1", () => {
+ withScratchHome((fixture) => {
+ const createdContent = '{"createdBy":"guard-test"}\n';
+ const result = runGuard(
+ fixture,
+ `import { writeFileSync } from "node:fs"; writeFileSync(process.env.GUARD_TEST_CONFIG_PATH!, ${JSON.stringify(createdContent)}, "utf8");`
+ );
+
+ const movedPath = `${fixture.configPath}.guard-created`;
+ expect(result.exitCode).toBe(1);
+ expect(existsSync(fixture.configPath)).toBe(false);
+ expect(readFileSync(movedPath, "utf8")).toBe(createdContent);
+ expect(stderrOf(result)).toContain("moved aside");
+ });
+ });
+
+ test("restores a modified existing file byte-exactly and exits 1", () => {
+ withScratchHome((fixture) => {
+ const originalContent = '{\n "token": "original",\n "spacing": true\n}\n';
+ writeFileSync(fixture.configPath, originalContent, "utf8");
+
+ const result = runGuard(
+ fixture,
+ 'import { writeFileSync } from "node:fs"; writeFileSync(process.env.GUARD_TEST_CONFIG_PATH!, "changed", "utf8");'
+ );
+
+ expect(result.exitCode).toBe(1);
+ expect(readFileSync(fixture.configPath, "utf8")).toBe(originalContent);
+ expect(stderrOf(result)).toContain("Restored it from the pre-run snapshot.");
+ });
+ });
+
+ test("returns the child exit code 0 when no guarded file changes", () => {
+ withScratchHome((fixture) => {
+ const originalContent = '{"unchanged":true}\n';
+ writeFileSync(fixture.configPath, originalContent, "utf8");
+
+ const result = runGuard(fixture, "process.exit(0);");
+
+ expect(result.exitCode).toBe(0);
+ expect(readFileSync(fixture.configPath, "utf8")).toBe(originalContent);
+ expect(stderrOf(result)).not.toContain("MUTATED");
+ });
+ });
+
+ test("treats a non-ENOENT stat failure as unreadable, never absent", () => {
+ withScratchHome((fixture) => {
+ rmSync(fixture.claudishDir, { recursive: true });
+ writeFileSync(fixture.claudishDir, "parent-is-not-a-directory", "utf8");
+ const result = runGuard(
+ fixture,
+ 'import { mkdirSync, rmSync, writeFileSync } from "node:fs"; rmSync(process.env.GUARD_TEST_CLAUDISH_DIR!); mkdirSync(process.env.GUARD_TEST_CLAUDISH_DIR!); writeFileSync(process.env.GUARD_TEST_CONFIG_PATH!, "created-after-stat-error", "utf8");'
+ );
+
+ expect(result.exitCode).toBe(1);
+ expect(existsSync(fixture.configPath)).toBe(true);
+ expect(existsSync(`${fixture.configPath}.guard-created`)).toBe(false);
+ expect(readFileSync(fixture.configPath, "utf8")).toBe("created-after-stat-error");
+ expect(stderrOf(result)).toContain("CANNOT RESTORE");
+ });
+ });
+});
diff --git a/scripts/guard-real-config.ts b/scripts/guard-real-config.ts
index 1a091c7f..71ded287 100644
--- a/scripts/guard-real-config.ts
+++ b/scripts/guard-real-config.ts
@@ -22,25 +22,85 @@
* reads the failure their config is already replaced, and — as happened here —
* the `.bak` alongside it may be a clobbered copy from an earlier run. Putting
* the bytes back is the part that actually saves them.
+ *
+ * GUARDED is a list rather than one path because the convention decayed again,
+ * in exactly the shape the paragraph above describes. `all-models.json` is the
+ * hosted model catalog's disk cache, and `effort-mapping.test.ts` seeds a
+ * fixture into it and restores in a `finally`. That is the same save/restore
+ * pattern the config files used, with the same hole: a killed or timed-out run
+ * skips the restore and the two-entry fixture becomes the machine's permanent
+ * catalog. It fails the way the config damage did — silently, as wrong routing
+ * and wrong reasoning decisions rather than an error — because a stale catalog
+ * is indistinguishable from a cold one to every caller.
+ *
+ * The seam that would remove the need is missing, which is why the test writes
+ * the real path at all: `findCacheEntry(modelId, cachePath?)` accepts an
+ * override, but `BaseApiFormat.lookupReasoningCapability` calls
+ * `lookupModelReasoning(this.modelId)` with no way to pass one, so a dialect
+ * test cannot redirect the read. `catalog-client.ts` solved the same problem
+ * with an in-memory seam (`_setCatalogEntriesForTest`); `model-catalog.ts` has
+ * no equivalent. Until it does, this guard is the backstop.
*/
import { spawn } from "node:child_process";
-import { copyFileSync, existsSync, readFileSync, writeFileSync } from "node:fs";
+import { copyFileSync, readFileSync, renameSync, statSync, writeFileSync } from "node:fs";
import { homedir } from "node:os";
import { join } from "node:path";
-const CONFIG = join(homedir(), ".claudish", "config.json");
+/**
+ * Every real file a test run must give back unchanged. `label` names it in the
+ * failure, and `remedy` states the isolation the author should have used — the
+ * two files have different answers, so the message cannot be generic.
+ */
+const GUARDED = [
+ {
+ path: join(homedir(), ".claudish", "config.json"),
+ label: "REAL CONFIG",
+ remedy: "isolate with setConfigFileOverride(<temp path>) instead of writing the real file",
+ },
+ {
+ path: join(homedir(), ".claudish", "all-models.json"),
+ label: "REAL MODEL CATALOG CACHE",
+ remedy:
+ "pass a temp path to readAllModelsCache/writeAllModelsCache, or seed in memory the way " +
+ "catalog-client.ts's _setCatalogEntriesForTest does, instead of writing the real cache",
+ },
+] as const;
interface Snapshot {
existed: boolean;
content: string | null;
+ /** `existed` is true but the bytes could not be read, so no restore is possible. */
+ unreadable: boolean;
}
-function snapshot(): Snapshot {
+/**
+ * Presence and content are established SEPARATELY, and ONLY `ENOENT` counts as
+ * absent. Folding any other failure into "absent" is a data-loss bug, because
+ * absence authorises a delete below: a file the guard merely could not STAT
+ * would be removed after the run as though the command had created it, and the
+ * backup loop skips `existed: false` files, so that is the single branch with no
+ * `.guard-backup` to recover from.
+ *
+ * `existsSync` is wrong here for exactly that reason — it answers `false` on
+ * every stat error, not just a missing file. A transient `EACCES` on
+ * `~/.claudish` (a macOS TCC denial, a sandbox, a stalled network mount) is
+ * enough. `statSync` with an explicit `ENOENT` test is the same check without
+ * the conflation; anything else routes to `unreadable`, which refuses to act.
+ */
+function snapshot(path: string): Snapshot {
try {
- if (!existsSync(CONFIG)) return { existed: false, content: null };
- return { existed: true, content: readFileSync(CONFIG, "utf-8") };
+ statSync(path);
+ } catch (err) {
+ if ((err as NodeJS.ErrnoException | null)?.code === "ENOENT") {
+ return { existed: false, content: null, unreadable: false };
+ }
+ // Present-or-unknown. Never absent, so the delete branch cannot fire.
+ return { existed: true, content: null, unreadable: true };
+ }
+ try {
+ return { existed: true, content: readFileSync(path, "utf-8"), unreadable: false };
} catch {
- return { existed: false, content: null };
+ return { existed: true, content: null, unreadable: true };
}
}
@@ -63,11 +123,23 @@ function describeDelta(before: string | null, after: string | null): string[] {
const inB = k in b;
if (inA && !inB) notes.push(` - REMOVED ${k}`);
else if (!inA && inB) notes.push(` - added ${k}`);
- else if (JSON.stringify(a[k]) !== JSON.stringify(b[k])) notes.push(` - changed ${k}`);
+ else if (JSON.stringify(a[k]) !== JSON.stringify(b[k]))
+ notes.push(` - changed ${k}${describeSize(a[k], b[k])}`);
}
return notes;
}
+/**
+ * Array lengths either side of a change, because the catalog's damage IS its
+ * length: "changed entries" and "changed entries (947 -> 2)" are the same
+ * finding, but only the second one reads as a clobbered file at a glance.
+ */
+function describeSize(before: unknown, after: unknown): string {
+ if (!Array.isArray(before) || !Array.isArray(after)) return "";
+ if (before.length === after.length) return "";
+ return ` (${before.length} -> ${after.length} entries)`;
+}
+
const args = process.argv.slice(2);
// Tolerate the `--` separator npm/bun users reflexively add.
const cmd = args[0] === "--" ? args.slice(1) : args;
@@ -76,11 +148,12 @@ if (cmd.length === 0) {
process.exit(2);
}
-const before = snapshot();
-if (before.existed) {
- // A dated sidecar, so the guard's own copy can never overwrite a real backup.
+const before = new Map(GUARDED.map((g) => [g.path, snapshot(g.path)]));
+for (const g of GUARDED) {
+ if (!before.get(g.path)?.existed) continue;
+ // A sidecar, so the guard's own copy can never overwrite a real backup.
try {
- copyFileSync(CONFIG, `${CONFIG}.guard-backup`);
+ copyFileSync(g.path, `${g.path}.guard-backup`);
} catch {
// Best effort: the in-memory snapshot is the real restore path.
}
@@ -100,47 +173,96 @@ if (before.existed) {
// `CLAUDISH_DISABLE_OP=1` is set for the same reason: 1Password arbitrates its
// handshake machine-wide, and a burst of denials from a test run suppresses
// authorization for every process on the machine for 15 seconds.
+//
+// `CLAUDISH_DISABLE_CATALOG_WARM=1` is the third. `createProxyServer` fires
+// `warmCatalog()` on every create, which live-fetches the hosted catalog and
+// writes `all-models.json` — so a suite that builds a proxy reads the network
+// and rewrites the machine. Prevention beats the snapshot here for the same
+// reason it does for the keychain: the restore below puts the bytes back, but
+// it cannot put back the hermeticity. A test whose result depends on what the
+// hosted catalog said this morning is the failure that turned two DeepSeek
+// tests red mid-release.
const child = spawn(cmd[0], cmd.slice(1), {
stdio: "inherit",
- env: { ...process.env, CLAUDISH_DISABLE_KEYCHAIN: "1", CLAUDISH_DISABLE_OP: "1" },
+ env: {
+ ...process.env,
+ CLAUDISH_DISABLE_KEYCHAIN: "1",
+ CLAUDISH_DISABLE_OP: "1",
+ CLAUDISH_DISABLE_CATALOG_WARM: "1",
+ },
});
-/** Report the mutation, put the original bytes back, and explain the fix. */
-function reportAndRestore(after: Snapshot): void {
+/** Report one file's mutation, put the original bytes back, and explain the fix. */
+function reportAndRestore(guarded: (typeof GUARDED)[number], after: Snapshot): void {
+ const prior = before.get(guarded.path) ?? { existed: false, content: null, unreadable: false };
+ const verb = prior.existed ? "wrote" : "CREATED";
process.stderr.write(
- `\n\x1b[31m✗ REAL CONFIG MUTATED\x1b[0m — something in that command wrote ${CONFIG}\n`
+ `\n\x1b[31m✗ ${guarded.label} MUTATED\x1b[0m — something in that command ${verb} ${guarded.path}\n`
);
- for (const line of describeDelta(before.content, after.content)) {
+ for (const line of describeDelta(prior.content, after.content)) {
process.stderr.write(`${line}\n`);
}
try {
- if (before.existed && before.content !== null) {
- writeFileSync(CONFIG, before.content, "utf-8");
- process.stderr.write("\n Restored your config from the pre-run snapshot.\n");
+ if (prior.existed && prior.content !== null) {
+ writeFileSync(guarded.path, prior.content, "utf-8");
+ process.stderr.write("\n Restored it from the pre-run snapshot.\n");
+ } else if (prior.existed && prior.unreadable) {
+ // Presence was certain, the bytes were not. Writing anything here would
+ // invent content; say so instead and leave the file for the developer.
+ process.stderr.write(
+ "\n \x1b[31mCANNOT RESTORE\x1b[0m — the file existed before the run but could not be read,\n" +
+ " so there is no snapshot to put back. Its current contents are whatever the run left.\n"
+ );
+ } else {
+ // Absent before, present after: the run CREATED a real file. Taking it out
+ // of the way is the restore. Leaving it is worse than it looks — on a clean
+ // machine a two-entry fixture becomes the catalog, and a cold cache and a
+ // poisoned one are indistinguishable to every caller.
+ //
+ // RENAMED, never deleted, even though the diagnosis says the run made it.
+ // The guard exists to prevent irreversible loss, so its own most
+ // destructive branch must not be the one place it cannot be wrong: a
+ // concurrent writer — a live `claudish serve`, an MCP server, a sibling
+ // agent session — can legitimately create this file during a multi-minute
+ // run, and that is indistinguishable from a test creating it. Renaming
+ // costs a stray file; deleting costs someone's API keys.
+ renameSync(guarded.path, `${guarded.path}.guard-created`);
+ process.stderr.write(
+ "\n No such file existed before the run, so it was moved aside to\n" +
+ ` ${guarded.path}.guard-created rather than deleted.\n` +
+ " If a concurrent process created it legitimately, move it back.\n"
+ );
}
} catch (err) {
process.stderr.write(
`\n \x1b[31mRESTORE FAILED\x1b[0m (${err instanceof Error ? err.message : String(err)}).\n` +
- ` A copy is at ${CONFIG}.guard-backup\n`
+ ` A copy is at ${guarded.path}.guard-backup\n`
);
}
process.stderr.write(
- "\n Tests must isolate with setConfigFileOverride(<temp path>) instead of\n" +
- " writing the real file. See scripts/guard-real-config.ts for the history.\n\n"
+ `\n Tests must ${guarded.remedy}.\n` +
+ " See scripts/guard-real-config.ts for the history.\n\n"
);
}
child.on("exit", (code, signal) => {
- const after = snapshot();
- const changed =
- before.existed !== after.existed || (before.content ?? null) !== (after.content ?? null);
-
- if (changed) {
- reportAndRestore(after);
- process.exit(1);
+ // Every guarded file is checked and restored before the first exit, so one
+ // clobbered file cannot mask a second. Reporting only the first would send
+ // the author to fix one test while another keeps rewriting the machine.
+ let mutated = false;
+ for (const g of GUARDED) {
+ const prior = before.get(g.path) ?? { existed: false, content: null, unreadable: false };
+ const after = snapshot(g.path);
+ const changed =
+ prior.existed !== after.existed || (prior.content ?? null) !== (after.content ?? null);
+ if (!changed) continue;
+ mutated = true;
+ reportAndRestore(g, after);
}
+
+ if (mutated) process.exit(1);
if (signal) {
- process.stderr.write(`\n[guard] command terminated by ${signal}; config intact.\n`);
+ process.stderr.write(`\n[guard] command terminated by ${signal}; guarded files intact.\n`);
process.exit(1);
}
process.exit(code ?? 0);