Claude Code. Any Model. The most powerful AI coding agent now speaks every language.
#!/usr/bin/env bun
/**
* Run a command and refuse to let it damage the developer's real claudish config.
*
* bun run scripts/guard-real-config.ts -- bun test
* bun run test:safe
*
* Why this exists: three e2e test files used to sandbox by overwriting the real
* `~/.claudish/config.json` and restoring it in `afterEach`. A killed or
* timed-out run skipped the restore, so the fixture became permanent. That
* destroyed a real user's `onepasswordAccount` and `onepasswordEnvironments`,
* which silently disabled 1Password for every claudish run on that machine —
* and because the failure mode is "1Password is never consulted", nothing
* errored. It just quietly stopped working.
*
* Those files now use `setConfigFileOverride`, but the fix is a convention and
* conventions decay. This guard is the enforcement: it does not care WHICH test
* misbehaves, only that the file came out the way it went in. Snapshot, run,
* compare, restore, fail loudly.
*
* Restoring rather than only reporting is deliberate. By the time a developer
* reads the failure their config is already replaced, and — as happened here —
* the `.bak` alongside it may be a clobbered copy from an earlier run. Putting
* the bytes back is the part that actually saves them.
*
* GUARDED is a list rather than one path because the convention decayed again,
* in exactly the shape the paragraph above describes. `all-models.json` was the
* hosted model catalog's disk cache (since contract 3 it is
* `cloud-models-catalog-v3.json`), and `effort-mapping.test.ts` seeded a
* fixture into it and restores in a `finally`. That is the same save/restore
* pattern the config files used, with the same hole: a killed or timed-out run
* skips the restore and the two-entry fixture becomes the machine's permanent
* catalog. It fails the way the config damage did — silently, as wrong routing
* and wrong reasoning decisions rather than an error — because a stale catalog
* is indistinguishable from a cold one to every caller.
*
* The seam that would remove the need is missing, which is why the test writes
* the real path at all: `findCacheEntry(modelId, cachePath?)` accepts an
* override, but `BaseApiFormat.lookupReasoningCapability` calls
* `lookupModelReasoning(this.modelId)` with no way to pass one, so a dialect
* test cannot redirect the read. `catalog-client.ts` solved the same problem
* with an in-memory seam (`_setCatalogEntriesForTest`); `model-catalog.ts` has
* no equivalent. Until it does, this guard is the backstop.
*/
import { spawn } from "node:child_process";
import {
chmodSync,
copyFileSync,
mkdtempSync,
readFileSync,
renameSync,
rmSync,
statSync,
writeFileSync,
} from "node:fs";
import { homedir, tmpdir } from "node:os";
import { join } from "node:path";
import { SESSION_ENV_VARS } from "./session-env-vars.ts";
/**
* Every real file a test run must give back unchanged. `label` names it in the
* failure, and `remedy` states the isolation the author should have used — the
* files have different answers, so the message cannot be generic.
*/
const GUARDED = [
{
path: join(homedir(), ".claudish", "config.json"),
label: "REAL CONFIG",
remedy: "isolate with setConfigFileOverride(<temp path>) instead of writing the real file",
},
{
path: join(homedir(), ".claudish", "cloud-models-catalog-v3.json"),
label: "REAL CLOUD MODELS CATALOG CACHE",
remedy:
"pass a temp path to readAllModelsCache/writeAllModelsCache, or seed in memory the way " +
"catalog-client.ts's _setCatalogEntriesForTest does, instead of writing the real cache",
},
{
// The route registry sidecar (`routeRegistryPathFor`): a test that writes it
// hands the real request path another generation's base URLs.
path: join(homedir(), ".claudish", "cloud-models-catalog-v3.routes.json"),
label: "REAL ROUTE REGISTRY SIDECAR",
remedy:
"pass a temp cache path to writeAllModelsCache/writeRouteRegistry, or run the refresh in a " +
"child process with its own HOME, instead of writing the real sidecar",
},
{
// The efforts sidecar (`effortRecommendationsPathFor`): a test that writes it
// hands the real request path recommended reasoning efforts nobody published.
path: join(homedir(), ".claudish", "cloud-models-catalog-v3.efforts.json"),
label: "REAL EFFORTS SIDECAR",
remedy:
"pass a temp `effortsPath` to writeEffortRecommendations / getRecommendedModels, or run " +
"the refresh in a child process with its own HOME",
},
{
// The pre-v3 cache file. This build no longer writes it, but older builds on the
// same machine still read it, so a test that touches it breaks them.
path: join(homedir(), ".claudish", "all-models.json"),
label: "REAL PRE-V3 MODEL CATALOG CACHE",
remedy: "nothing in this build writes all-models.json; find the test that still names it",
},
{
path: join(homedir(), ".claudish", "openrouter-models.json"),
label: "REAL OPENROUTER MODEL LIST",
remedy: "run the MCP search_models path against a temp HOME instead of the real cache dir",
},
{
// The Muse Code login store. Bun 1.4.0's `os.homedir()` ignores a HOME
// assigned in-process, so a test that "sets a temp HOME" and then calls
// `defaultMuseStoreDir()` writes HERE. Measured: a full-suite run left this
// file and the borrowed copy below in a real home dir, holding TESTTOKEN
// placeholders, and later files read them as the user's login.
path: join(homedir(), ".claudish", "meta-muse-oauth.json"),
label: "REAL MUSE CODE LOGIN",
secret: true,
remedy:
"pass a temp `dir` to the Muse store functions and to setMuseCoreTestDeps, or run in a " +
"child process spawned with its own HOME; assigning process.env.HOME does not move homedir()",
},
{
// The borrowed 0600 copy of Muse's own Keychain login that team/MCP slots sign with.
path: join(homedir(), ".claudish", "meta-muse-borrowed.json"),
label: "REAL MUSE CODE BORROWED LOGIN",
secret: true,
remedy:
"pass a temp `dir` to the Muse store functions and to setMuseCoreTestDeps, or run in a " +
"child process spawned with its own HOME; assigning process.env.HOME does not move homedir()",
},
{
path: join(homedir(), ".claudish", "catalog-incompatible.json"),
label: "REAL CATALOG CONTRACT SENTINEL",
remedy:
"pass a temp path to markCatalogIncompatible/readCatalogIncompatibility, or run the " +
"refresh in a child process with its own HOME, instead of writing the real sentinel",
},
] as const;
interface Snapshot {
existed: boolean;
content: string | null;
/** `existed` is true but the bytes could not be read, so no restore is possible. */
unreadable: boolean;
}
/**
* Presence and content are established SEPARATELY, and ONLY `ENOENT` counts as
* absent. Folding any other failure into "absent" is a data-loss bug, because
* absence authorises a delete below: a file the guard merely could not STAT
* would be removed after the run as though the command had created it, and the
* backup loop skips `existed: false` files, so that is the single branch with no
* `.guard-backup` to recover from.
*
* `existsSync` is wrong here for exactly that reason — it answers `false` on
* every stat error, not just a missing file. A transient `EACCES` on
* `~/.claudish` (a macOS TCC denial, a sandbox, a stalled network mount) is
* enough. `statSync` with an explicit `ENOENT` test is the same check without
* the conflation; anything else routes to `unreadable`, which refuses to act.
*/
function snapshot(path: string): Snapshot {
try {
statSync(path);
} catch (err) {
if ((err as NodeJS.ErrnoException | null)?.code === "ENOENT") {
return { existed: false, content: null, unreadable: false };
}
// Present-or-unknown. Never absent, so the delete branch cannot fire.
return { existed: true, content: null, unreadable: true };
}
try {
return { existed: true, content: readFileSync(path, "utf-8"), unreadable: false };
} catch {
return { existed: true, content: null, unreadable: true };
}
}
/** Which top-level keys changed — enough to see what a rogue test clobbered. */
function describeDelta(before: string | null, after: string | null): string[] {
const parse = (s: string | null): Record<string, unknown> => {
if (!s) return {};
try {
return JSON.parse(s) as Record<string, unknown>;
} catch {
return {};
}
};
const a = parse(before);
const b = parse(after);
const keys = new Set([...Object.keys(a), ...Object.keys(b)]);
const notes: string[] = [];
for (const k of [...keys].sort()) {
const inA = k in a;
const inB = k in b;
if (inA && !inB) notes.push(` - REMOVED ${k}`);
else if (!inA && inB) notes.push(` - added ${k}`);
else if (JSON.stringify(a[k]) !== JSON.stringify(b[k]))
notes.push(` - changed ${k}${describeSize(a[k], b[k])}`);
}
return notes;
}
/**
* Array lengths either side of a change, because the catalog's damage IS its
* length: "changed entries" and "changed entries (947 -> 2)" are the same
* finding, but only the second one reads as a clobbered file at a glance.
*/
function describeSize(before: unknown, after: unknown): string {
if (!Array.isArray(before) || !Array.isArray(after)) return "";
if (before.length === after.length) return "";
return ` (${before.length} -> ${after.length} entries)`;
}
const args = process.argv.slice(2);
// Tolerate the `--` separator npm/bun users reflexively add.
const cmd = args[0] === "--" ? args.slice(1) : args;
if (cmd.length === 0) {
process.stderr.write("usage: guard-real-config.ts [--] <command> [args...]\n");
process.exit(2);
}
const before = new Map(GUARDED.map((g) => [g.path, snapshot(g.path)]));
/**
* A credential store gets no `.guard-backup` sidecar and is restored 0600: a
* sidecar would be a second plaintext copy of a real login that `claudish
* logout` never removes. The in-memory snapshot is its only restore path.
*/
function isSecret(g: (typeof GUARDED)[number]): boolean {
return "secret" in g && g.secret === true;
}
for (const g of GUARDED) {
if (!before.get(g.path)?.existed) continue;
if (isSecret(g)) continue;
// A sidecar, so the guard's own copy can never overwrite a real backup.
try {
copyFileSync(g.path, `${g.path}.guard-backup`);
} catch {
// Best effort: the in-memory snapshot is the real restore path.
}
}
// The developer's real CREDENTIAL STORES are guarded the same way as their
// config file, and for the same reason: a test that reaches one is not
// hermetic, and the damage lands on the machine rather than on the test.
//
// `CLAUDISH_DISABLE_KEYCHAIN=1` makes `hasKeychainSource()` return false, so no
// suite can spawn `security` against the login keychain. This is not
// hypothetical: `keychain.enabled` in a real config was enough to make an
// unrelated test run enumerate it. The keychain is worse than the config file
// in one respect — a mutation there cannot be snapshotted and restored the way
// the bytes of a JSON file can — so it is prevented rather than repaired.
//
// `CLAUDISH_DISABLE_OP=1` is set for the same reason: 1Password arbitrates its
// handshake machine-wide, and a burst of denials from a test run suppresses
// authorization for every process on the machine for 15 seconds.
//
// `CLAUDISH_DISABLE_CATALOG_WARM=1` is the third. `createProxyServer` fires
// `warmCatalog()` on every create, which live-fetches the hosted catalog and
// writes `all-models.json` — so a suite that builds a proxy reads the network
// and rewrites the machine. Prevention beats the snapshot here for the same
// reason it does for the keychain: the restore below puts the bytes back, but
// it cannot put back the hermeticity. A test whose result depends on what the
// hosted catalog said this morning is the failure that turned two DeepSeek
// tests red mid-release.
//
// `CLAUDISH_CATALOG_INCOMPATIBLE_PATH` is the fourth, and it REDIRECTS rather
// than disables. The contract sentinel is state a real build writes on every
// launch against a newer catalog, so on a developer's machine it usually
// EXISTS — and every default-path read in the suite then sees "this catalog is
// unreadable". Measured 2026-09-18: 25 tests failed on that alone, and CI,
// whose home directory never holds the file, stayed green. A fresh directory
// per run means no run can read a sentinel an earlier run wrote either.
const sentinelDir = mkdtempSync(join(tmpdir(), "claudish-guard-sentinel-"));
const childEnv: Record<string, string | undefined> = {
...process.env,
CLAUDISH_DISABLE_KEYCHAIN: "1",
CLAUDISH_DISABLE_OP: "1",
CLAUDISH_DISABLE_CATALOG_WARM: "1",
CLAUDISH_CATALOG_INCOMPATIBLE_PATH: join(sentinelDir, "catalog-incompatible.json"),
};
// The session-identity variables reach everything a claudish-started session runs,
// a test run included (a runner is often such a session). Inherited,
// `CLAUDISH_UPSTREAM_ERROR_LOG` makes the suite append its test doubles to that
// session's upstream-errors.jsonl (D5), the hook and token-file names point the
// suite at the session's other records, and `CLAUDISH_PANE_CHILD` puts every
// claudish child and MCP server the suite spawns into nested mode (four tools,
// model launches refused with exit 64). The bun test preload deletes them from
// `process.env` (first layer), but a child spawned with no `env` option gets the
// ORIGINAL environment back. Deleting them here, from the OS environment of the
// whole run, is the second layer: no descendant of a guarded run can inherit them.
for (const name of SESSION_ENV_VARS) delete childEnv[name];
const child = spawn(cmd[0], cmd.slice(1), {
stdio: "inherit",
env: childEnv,
});
/** Put the pre-run bytes back; a credential store is written 0600 whatever the run left. */
function restoreBytes(guarded: (typeof GUARDED)[number], content: string): void {
if (!isSecret(guarded)) {
writeFileSync(guarded.path, content, "utf-8");
return;
}
writeFileSync(guarded.path, content, { mode: 0o600 });
chmodSync(guarded.path, 0o600);
}
/** Where a failed restore's copy is, or that a credential store has none. */
function backupNote(guarded: (typeof GUARDED)[number]): string {
return isSecret(guarded)
? " No sidecar copy exists for a credential store; log in again if it is lost.\n"
: ` A copy is at ${guarded.path}.guard-backup\n`;
}
/** Report one file's mutation, put the original bytes back, and explain the fix. */
function reportAndRestore(guarded: (typeof GUARDED)[number], after: Snapshot): void {
const prior = before.get(guarded.path) ?? { existed: false, content: null, unreadable: false };
const verb = prior.existed ? "wrote" : "CREATED";
process.stderr.write(
`\n\x1b[31m✗ ${guarded.label} MUTATED\x1b[0m — something in that command ${verb} ${guarded.path}\n`
);
for (const line of describeDelta(prior.content, after.content)) {
process.stderr.write(`${line}\n`);
}
try {
if (prior.existed && prior.content !== null) {
restoreBytes(guarded, prior.content);
process.stderr.write("\n Restored it from the pre-run snapshot.\n");
} else if (prior.existed && prior.unreadable) {
// Presence was certain, the bytes were not. Writing anything here would
// invent content; say so instead and leave the file for the developer.
process.stderr.write(
"\n \x1b[31mCANNOT RESTORE\x1b[0m — the file existed before the run but could not be read,\n" +
" so there is no snapshot to put back. Its current contents are whatever the run left.\n"
);
} else {
// Absent before, present after: the run CREATED a real file. Taking it out
// of the way is the restore. Leaving it is worse than it looks — on a clean
// machine a two-entry fixture becomes the catalog, and a cold cache and a
// poisoned one are indistinguishable to every caller.
//
// RENAMED, never deleted, even though the diagnosis says the run made it.
// The guard exists to prevent irreversible loss, so its own most
// destructive branch must not be the one place it cannot be wrong: a
// concurrent writer — a live `claudish serve`, an MCP server, a sibling
// agent session — can legitimately create this file during a multi-minute
// run, and that is indistinguishable from a test creating it. Renaming
// costs a stray file; deleting costs someone's API keys.
renameSync(guarded.path, `${guarded.path}.guard-created`);
process.stderr.write(
"\n No such file existed before the run, so it was moved aside to\n" +
` ${guarded.path}.guard-created rather than deleted.\n` +
" If a concurrent process created it legitimately, move it back.\n"
);
}
} catch (err) {
process.stderr.write(
`\n \x1b[31mRESTORE FAILED\x1b[0m (${err instanceof Error ? err.message : String(err)}).\n` +
backupNote(guarded)
);
}
process.stderr.write(
`\n Tests must ${guarded.remedy}.\n` +
" See scripts/guard-real-config.ts for the history.\n\n"
);
}
child.on("exit", (code, signal) => {
rmSync(sentinelDir, { recursive: true, force: true });
// Every guarded file is checked and restored before the first exit, so one
// clobbered file cannot mask a second. Reporting only the first would send
// the author to fix one test while another keeps rewriting the machine.
let mutated = false;
for (const g of GUARDED) {
const prior = before.get(g.path) ?? { existed: false, content: null, unreadable: false };
const after = snapshot(g.path);
const changed =
prior.existed !== after.existed || (prior.content ?? null) !== (after.content ?? null);
if (!changed) continue;
mutated = true;
reportAndRestore(g, after);
}
if (mutated) process.exit(1);
if (signal) {
process.stderr.write(`\n[guard] command terminated by ${signal}; guarded files intact.\n`);
process.exit(1);
}
process.exit(code ?? 0);
});
child.on("error", (err) => {
process.stderr.write(`[guard] failed to run command: ${err.message}\n`);
process.exit(1);
});