Claude Code. Any Model. The most powerful AI coding agent now speaks every language.
claudish serve — Claude Desktop custom-model gateway
Status: implemented, unreleased (on branch worktree-models-list-serve).
Audience: the next developer picking this up — the claude-desktop-profiles
macOS app author, and anyone touching provider routing.
This is the developer handoff for the serve feature: a standalone inference
gateway that lets Claude Desktop's "third-party inference" mode display and
route to non-Anthropic models through claudish. It also covers the supporting
providers --json command and the provider-slug rename that the feature depends
on, plus the live-catalog routing-alignment validation that was run to
confirm the slugs line up.
1. What problem this solves
Claude Desktop can point its model picker at a third-party OpenAI-compatible endpoint. But it is opinionated about what it will show:
- It builds its picker only from a live
GET /v1/modelscall, and silently drops any model id it does not recognize as a Claude slot. You cannot make it displaygrok-4orgemini-3-pro; it will only show ids likeclaude-sonnet-4-6,claude-opus-4-1, etc. - It then sends
POST /v1/messageswithbody.model = <that slot id>.
So to surface an arbitrary external model in Claude Desktop, you have to:
- Advertise a Claude-recognized slot id on
/v1/models. - When a request comes in for that slot, rewrite it to the real model the user assigned to that slot and route it through claudish's normal pipeline.
claudish serve is that gateway. The claude-desktop-profiles app writes a
models.json mapping slots → real models, spawns claudish serve, and points
Claude Desktop at it.
2. The pieces (what was implemented)
Five files. Two new, three touched. (A separate, independent change — the provider-slug rename — rides in the same branch; see §6.)
| File | Change |
|---|---|
src/serve-command.ts |
new — claudish serve entry point: parse flags, load + validate models.json, start the proxy parked. |
src/providers-command.ts |
new — claudish providers --json: credential-presence report (no key material) so the profiles app knows which providers are configured here. |
src/proxy-server.ts |
SlotRoute type + slotMap/servedSlotIds options; exact-slot routing in getHandlerForRequest; GET /v1/models route. |
src/index.ts |
subcommand dispatch for serve and providers. |
2.1 claudish serve --port <n> --models <path>
claudish serve --port 8787 --models ~/.../models.json
- Parses only
--port/-pand--models/-m. It deliberately does not reuse the full CLI parser —servemust not inherit Claude-Code-launch semantics (no child spawn, no interactive prompts). - Loads and strictly validates
models.json(see §3). Malformed input exits non-zero with a specific message — the operator finds out at startup, not via an empty picker at runtime. - Reads
OPENROUTER_API_KEY/ANTHROPIC_API_KEYfrom the environment.index.ts'sloadStoredApiKeys()has already merged~/.claudish/config.jsonkeys intoprocess.envat module load, so both env-set and config-stored keys are visible.servebypassesparseArgs, so these must be read explicitly here or null-provider→OpenRouter andclaude-*passthrough slots would have no credentials. - Calls
createProxyServer(...)with{ slotMap, servedSlotIds }and then parks (await new Promise(() => {})). Nofinally/shutdown, no Claude spawn: the profiles app owns this process's lifecycle andSIGTERMs it on stop. (stats-buffer.tsalready installsSIGINT/SIGTERM→ flush →process.exit(0)handlers on import, so termination is already clean;serveadds none of its own.)
2.2 claudish providers --json
{ "providers": [
{ "slug": "x-ai", "ready": true, "authSource": "env" },
{ "slug": "google", "ready": false, "authSource": null }
] }
- One entry per non-virtual provider.
slugis the canonicalBUILTIN_PROVIDERSname (tui/providers.tssetscatalogName: def.name) — the exact stringserveaccepts as a pinnedprovider, and the stringparseModelSpec/resolveRemoteProvidermatch on. Not the picker label (slugis"google", never"gemini"). - Security: presence + source only. It must never touch the
--probeprovenance path (which prints unmasked keys) and must never emit any value, masked fragment, or key material.
2.3 GET /v1/models (in proxy-server.ts)
Returns the OpenAI list shape, data[] populated from options.servedSlotIds
— the slot ids, not the real model ids. Defaults to [] for non-serve
callers (the picker is irrelevant to them, so this route is inert in normal
claudish runs).
2.4 Exact-slot routing (the core)
In getHandlerForRequest, before the substring tier match:
const slot = options.slotMap?.get(requestedModel); // exact id, not substring
if (slot) {
target = slot.provider ? `${slot.provider}@${slot.model}` : slot.model;
slotMatched = true; // skip tier match + --model fallback entirely
}
Exact-id lookup is deliberate: two slots can share a tier substring
(claude-opus-4-1 and claude-opus-4-20250514 both contain opus). The old
substring modelMap would collide them; the slot map routes them distinctly.
After rewriting target, control falls through to the existing pipeline —
so pinned providers take the explicit-provider@model path, null-provider
slots take auto-route + catalog vendor-prefix resolution, and claude-* reals
take native Anthropic passthrough. No new routing engine; just a new front door.
3. models.json contract (what the profiles app writes)
[
{ "slot": "claude-sonnet-4-6", "model": "grok-4", "provider": "x-ai" },
{ "slot": "claude-opus-4-1", "model": "gemini-3-pro-preview", "provider": null },
{ "slot": "claude-haiku-4-5", "model": "kimi-k2", "provider": "kimi" }
]
| field | meaning | rules |
|---|---|---|
slot |
the Claude-recognized id Claude Desktop sends as body.model |
required, non-empty, unique (duplicates throw at load) |
model |
the real model id to route to | required, non-empty |
provider |
pinned routing slug, or null/omitted = autoroute |
optional; if present must be a string |
The single most important contract rule (see §5 for why):
providermust be a slug fromclaudish providers --json, ornull. It is a routing slug (aBUILTIN_PROVIDERSname), not the model's origin-vendor slug from the catalog. Pinnullfor anything you're unsure of — autoroute is the safe default and covers every catalog model.
4. Routing-alignment validation (live catalog)
The rename in §6 exists so claudish and the cloud catalog speak one slug. This was verified against the live deployed catalog, not just the repo:
GET https://us-central1-claudish-6da10.cloudfunctions.net/queryModels?limit=200&offset=…
→ 898 models, paged
Method: collect every distinct provider value the live catalog serves,
then check each against claudish's routable slug set (BUILTIN_PROVIDERS names
- shortcuts + the vendor → native provider bridge: at the time a hand-written
table,
FIREBASE_SLUG_TO_PROVIDER_NAME, since derived bynativeProviderForVendor).
Headline results:
- The rename succeeded.
x-ai(7 models) andz-ai(22 models) are now identity-aligned: the catalog's slug is claudish's canonicalBUILTIN_PROVIDERSname. A profiles app reading the catalogproviderfield for an xAI or Z.AI model and pinning it verbatim now routes correctly. - Providers claudish routes directly all align by identity:
openai,google,deepseek,minimax,qwen,x-ai,z-ai. - One real divergence among direct providers: catalog
moonshotai↔ claudishkimi.nativeProviderForVendor(providers/route-candidates.ts) bridgesmoonshotai → kimifor the recommended-catalog renderer, but the serve slot path does not apply that map — it builds${slot.provider}@${slot.model}verbatim. Soprovider: "moonshotai"pinned intomodels.jsonis not a routable claudish slug. Pin"kimi"(ornull) instead.
What is not a gap (don't be alarmed by it): the catalog also carries
origin-vendor slugs like mistralai, meta-llama, nvidia, cohere,
microsoft, etc. (≈220 models). These are who made the model, not claudish
routing providers — they were never direct-provider candidates and all route
fine via OpenRouter/autoroute when you pin provider: null. They are a
non-issue as long as the profiles app pins from the providers --json set or
null (§5).
Reproduce the validation:
# distinct provider slugs in the live catalog
python3 - <<'PY'
import json, urllib.request
base='https://us-central1-claudish-6da10.cloudfunctions.net/queryModels'
prov=set(); off=0
while True:
d=json.load(urllib.request.urlopen(f'{base}?limit=200&offset={off}', timeout=25))
for m in d['models']: prov.add(m['provider'])
if not d.get('hasMore') or not d['models']: break
off+=len(d['models'])
print(sorted(prov))
PY
# claudish's routable slugs
claudish providers --json | python3 -c "import json,sys;print([p['slug'] for p in json.load(sys.stdin)['providers']])"
Note on
aggregators[]: CLAUDE.md describes anaggregators[]routing index ({provider, externalId, confidence}) on catalog documents. In the currently deployed catalog that field is empty (0/898). Until it's populated, the model's top-levelproviderfield is the only provider signal the profiles app can read — and that's the field this validation checked.
5. ⚠️ The one sharp edge — pinned-provider failure mode
If models.json pins a provider that is not a claudish routable slug
(e.g. the catalog origin-vendor moonshotai, mistralai, …), the request does
not clean-error with "unknown provider." It falls through to OpenRouter, and
the observable outcome depends on whether OPENROUTER_API_KEY is set —
either way the user never learns their provider pin was invalid. The flow:
getHandlerForRequestbuildstarget = "moonshotai@kimi-k2".- The explicit-provider path can't resolve
moonshotaito a direct profile, socreateHandlerForProviderreturnsnull(provider-profiles.ts:397), the resolution path returnsnull(proxy-server.ts:305)… - …and the request falls through to the OpenRouter handler.
Verified empirically (serve --debug, POST the slot, watch
[Serve] slot … → moonshotai@kimi-k2):
OPENROUTER_API_KEY |
Result |
|---|---|
| set | HTTP 200 — silently served by OpenRouter (here it even resolved kimi-k2 on OpenRouter, so it looks like it worked). Pure silent mis-route — the worst case, because nothing signals the pin was wrong. |
| unset | HTTP 401 authentication_error from the OpenRouter handler — an opaque auth failure, not a "you pinned a bad provider" message. |
This is the known "missing-profile → silent OpenRouter fallback" behavior. It makes the §3 contract rule load-bearing:
The profiles app MUST pin a
providers --jsonslug ornull. Never pin a raw catalog origin-vendor slug.
Recommended fix on the profiles-app side (pick one):
- Populate the provider picker from
claudish providers --json— then a raw catalog slug likemoonshotaiis never offered; the user pickskimi, or leaves itnull. (Lowest-risk;providers-command.tsexists for exactly this.) - Or always write
provider: nulland let claudish autoroute. Simplest; you lose the ability to pin a specific provider but gain "can't get it wrong."
Optional hardening on the claudish side (not yet done — a decision for the
next dev): make serve reject (or warn + downgrade to null) a slot.provider
that isn't in the providers --json set, instead of letting it reach the silent
OpenRouter fallback. This would turn the sharp edge into a startup error. See
memory: project_silent_provider_fallback.md.
Open verification: the
claude-desktop-profilesapp source is not in this repo or~/mag, so its actualprovider-pinning logic was not read. The §3/§5 contract is whatserverequires; confirm the app honors it (ideally option 1 above) before shipping.
6. The provider-slug rename (independent change, same branch)
xai → x-ai, zai → z-ai across BUILTIN_PROVIDERS (18 files). This is the
prerequisite that makes §4 identity-aligned. Done as a full canonical
rename; old forms kept as input aliases so existing commands still route:
// provider-definitions.ts
name: "x-ai", shortcuts: ["x-ai", "xai", "grok"], shortestPrefix: "x-ai",
name: "z-ai", shortcuts: ["z-ai", "zai"], shortestPrefix: "z-ai",
Credentials do not break. Verified empirically:
| Credential path | Keyed by | Breaks? |
|---|---|---|
Env XAI_API_KEY / ZAI_API_KEY |
env-var name (unchanged) | No |
config.json stored key |
apiKeyEnvVar = XAI_API_KEY (unchanged) |
No |
| OAuth token | oauthSlug (xai/zai have none) |
N/A |
Typed xai@ / zai@ |
kept as shortcuts | No |
Only the canonical name string changed — surfacing correctly as x-ai/z-ai
in providers --json and matching the catalog.
Commit guidance: the serve feature (serve-command.ts,
providers-command.ts, proxy-server.ts, index.ts) and the rename (18 files)
are logically independent — split into two commits so the rename can be
reverted without unwinding the feature. Exclude .gitignore — its only diff
(.claudemem/ removal) is unrelated session noise.
7. Verification status
- Build: CLI package bundles clean (418 modules). (The
macos-bridgepackage fails to build — missingnode_modulesthere — but it is a separate package, untouched by this work; purely environmental.) - Tests: provider-name suites pass (
default-routing-rules,routing-rules,model-selector,model-catalog→ 158/158). Prior session reported 379/379 across all 13 provider-name suites. - Live alignment: validated against the deployed catalog (898 models) — §4.
- Serve path, locally: verified end-to-end against the worktree build —
GET /v1/modelsreturns the slot ids;POST /v1/messagesroutes through the slot rewrite;providers --jsonemitsx-ai/z-aiwith no oldxai/zai. The §5 pinned-unrouteable-provider failure mode was reproduced (silent OpenRouter mis-route with a key; 401 without). - Profiles-app contract: not yet run against the real
claude-desktop-profilesapp (not on this machine). Confirming the app pins from theproviders --jsonset (not raw catalog slugs) is the remaining gate before release — see §5 open verification.
8. Running the dev build (for the cloud-profiles developer)
Officially, claudish is the global install (bun install -g claudish, or
Homebrew). serve and providers will reach users through that channel. But
this branch is not yet released, so to test the profiles app against it you
must point at this worktree. The two facts that make this fiddly:
- The global
claudish(v7.2.0) does NOT haveserve/providers. They are branch-only. Running the released binary just prints the help banner — it silently ignores the unknown subcommand. You will get nothing useful until this branch ships. - The profiles app resolves
claudishfromPATH(bare name — there is noCLAUDISH_PATH/binary-override env var in the codebase; the channelsession-managerspawns"claudish"the same way). So the dev hook is to put aclaudishthat runs this worktree earlier onPATH.
Recommended: a PATH wrapper that runs this worktree's source
Faithful to "it relies on the global claudish" — the profiles app spawns bare
claudish exactly as in production; the wrapper just intercepts it for dev.
# 1. Create a wrapper somewhere early on the profiles app's PATH (e.g. ~/bin):
mkdir -p ~/bin
cat > ~/bin/claudish <<'EOF'
#!/bin/sh
exec bun --no-env-file run <claudish-checkout>/packages/cli/src/index.ts "$@"
EOF
chmod +x ~/bin/claudish
# 2. Put ~/bin ahead of the npm global bin on PATH (in your shell rc, or the
# environment the profiles app inherits):
export PATH="$HOME/bin:$PATH"
# 3. Verify it's this build, not the global one:
claudish providers --json # → {"providers":[ … "slug":"x-ai" … "z-ai" … ]}
claudish serve --port 8787 --models ~/models.json
After this branch merges and you bun install -g claudish (or brew upgrade),
delete ~/bin/claudish so you're back on the real global binary.
Quickest: call the source directly (no PATH change)
If the profiles app can be configured with an explicit command for dev instead
of a bare claudish:
bun --no-env-file run <claudish-checkout>/packages/cli/src/index.ts serve --port 8787 --models ~/models.json
bun --no-env-file run <claudish-checkout>/packages/cli/src/index.ts providers --json
<claudish-checkout> is the absolute path of your clone. Keep --no-env-file in both
recipes: it stops Bun loading the env files of whatever directory the command runs in,
which is what the installed claudish does. Without it claudish cannot tell those
values from yours, so it ignores CLAUDISH_CONFIG and the other control variables, and
every name the files set, with one stderr line.
Two traps — do NOT use these in this worktree
bun link/ installing this package globally → produces aclaudishthat crashes withCannot find module '@opentui/core'. This worktree does not have full deps installed (@opentuiis absent from itsnode_modules), and the build marks@opentui/*/reactas external, so the bundleddist/index.jscan't resolve them at runtime when run outside a proper install.bun dist/index.js …directly → same@opentui/corefailure, for the same reason.
Why the source run (bun run src/index.ts) sidesteps this: @opentui is
lazy-imported on only the config TUI path (index.ts →
import("./tui/index.js")). serve and providers never touch the TUI, so a
source run of those subcommands never needs @opentui — Bun resolves only what
those codepaths import. (If you do want the build/link route to work, run a
full bun install in the worktree first — unnecessary for testing serve.)