Skip to content

Claude Code. Any Model. The most powerful AI coding agent now speaks every language.

Merge origin/main (v10.1.0) into the v10.1.1 release branch

Jack Rudenko0fc4cdc1da28
diff --cc CLAUDE.md
index d139cfb8,6c28a187..9b638ee7
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@@ -52,7 -55,7 +55,7 @@@ negative case: if you are about to typ
| route variant | effort model, model preset | a preset entry of a base model (`routeVariant`, e.g. `kimi-k3-256k`, context=256k) |
| connection | aggregator row, provider row | one mapped way to call a model: an `aggregators[]` entry with `routeStatus: "mapped"` |
| input modality, output modality | capability (for what goes in or out) | what a model accepts and produces: text, image, audio, video, file |
--| chat model | text model, LLM | takes text in and gives only text out; video input alone never excludes one |
++| chat model | text model, LLM | text among its input modalities AND among its output modalities; other modalities on either side never exclude one (`in: [file,image,text]` is a chat model, `in: [audio]` is not) |
| type | shape, kind, variant (for a discriminator) | the field that says how to read the rest of an object: `pricing.type` is `flat`, `tiered`, `free` or `unavailable`. The catalog spelled it `shape` until the cutover on generation `g-20260921062451697-f490edba`; claudish reads `type` only, with no alias |
| route candidate | option, hop (before filtering), chain entry | one `(provider, wire id)` pair gathered for a model, BEFORE the credential and availability filters. A candidate is a proposal; a hop is what survived |
| tier | class, category, rank, level | a provider's routing class: `subscription`, `dynamic-subscription`, `native`, `gateway`, `fallback`. Ordering is by tier first, and only then by vendor and price |
diff --cc packages/cli/src/model-selector-discovery.test.ts
index 00000000,13476639..65f91f52
mode 000000,100644..100644
--- a/packages/cli/src/model-selector-discovery.test.ts
+++ b/packages/cli/src/model-selector-discovery.test.ts
@@@ -1,0 -1,538 +1,625 @@@
+ /**
+ * The picker's discovery outcome — `buildDiscoveredModelOutcome`, the notice
+ * formatter behind it, and the chat-capability chokepoint both doors return
+ * through.
+ *
+ * The defect these pin: `buildDiscoveredModelRows` returned `[]` for five
+ * distinguishable states — a rejected key, an unreachable endpoint, a genuinely
+ * empty dynamic models catalog, a dynamic models catalog where nothing was chat-capable, and a dynamic models catalog that
+ * collapsed to nothing — and the caller fell through to the cloud catalog for
+ * all of them in silence. The user saw "fewer model names, like the provider
+ * does not have any models". So every test here is a state that used to be
+ * indistinguishable from the other four.
+ *
+ * Offline by construction: `globalThis.fetch`, `credentials.getRequestAuth` and
- * the `CatalogClient` are all substituted directly. No `mock.module()` — it
- * bleeds into sibling Bun test files.
++ * the `CatalogClient` are all substituted directly. The capability projection
++ * gets a file-local module fixture that is restored in `afterAll`, so no row
++ * depends on the developer's real cloud models catalog cache.
+ *
- * Nothing here asserts a price, a context window or a capability flag: those
- * come from the on-disk model catalog, which is warm on a dev machine and cold
- * on a fresh clone. Ids, counts and outcome kinds are the assertions.
++ * Nothing here asserts a price or context window. Ids, counts, evidence-based
++ * capability decisions and outcome kinds are the assertions.
+ *
+ * Run: bun test packages/cli/src/model-selector-discovery.test.ts
+ */
+
-import { afterEach, beforeEach, describe, expect, mock, spyOn, test } from "bun:test";
++import { afterAll, afterEach, beforeEach, describe, expect, mock, spyOn, test } from "bun:test";
++import type { DiskCacheV3, SlimModelEntry } from "./providers/all-models-cache.js";
++import * as __realAllModelsCache from "./providers/all-models-cache.js";
++
++const __realAllModelsCacheExports = { ...__realAllModelsCache };
++
++function catalogCapability(
++ modelId: string,
++ inputModalities: string[] | null | undefined,
++ outputModalities: string[]
++): SlimModelEntry {
++ return { modelId, aliases: [], inputModalities, outputModalities };
++}
++
++const catalogCapabilityFixture: DiskCacheV3 = {
++ version: 3,
++ lastUpdated: "2026-09-23T00:00:00.000Z",
++ catalogGenerationId: "model-selector-discovery-test",
++ entries: [
++ ...[
++ "catalog-chat-1",
++ "catalog-chat-2",
++ "claude-opus-5",
++ "gpt-5.2-codex",
++ "gemini-3-pro",
++ "kimi-k3",
++ "deepseek-v3",
++ "llama3.2",
++ "MiniMax-M2",
++ ].map((id) => catalogCapability(id, ["text"], ["text"])),
++ catalogCapability("gpt-image-2.5-flare", ["text"], ["image"]),
++ catalogCapability("text-embedding-3-large", ["text"], ["embedding"]),
++ catalogCapability("whisper-1", ["audio"], ["text"]),
++ catalogCapability("dall-e-3", ["text"], ["image"]),
++ catalogCapability("tts-1-hd", ["text"], ["audio"]),
++ catalogCapability("nomic-embed-text", ["text"], ["embedding"]),
++ catalogCapability("bge-m3", ["text"], ["embedding"]),
++ ],
++ models: [],
++ plans: [],
++};
++
++mock.module("./providers/all-models-cache.js", () => ({
++ ...__realAllModelsCacheExports,
++ readAllModelsCache: (path?: string) =>
++ path ? __realAllModelsCacheExports.readAllModelsCache(path) : catalogCapabilityFixture,
++}));
+
+ import { credentials } from "./auth/credentials/authority.js";
+ import {
+ _modelsCatalogCollapse,
+ buildDiscoveredModelOutcome,
+ buildDiscoveredModelRows,
+ formatDiscoveryFailureNotice,
+ toPickerRows,
+ warnDiscoveryFailure,
+ } from "./model-selector.js";
+ import type { ModelInfo } from "./model-selector.js";
+ import type { CatalogClient, CatalogModel } from "./providers/model-catalog.js";
+ import {
++ type DiscoveredModel,
+ type DiscoveryFailure,
+ type DiscoveryFailureKind,
++ type FetcherResult,
+ describeDiscoveryFailure,
+ invalidateModelDiscovery,
++ registerModelDiscoveryFetcher,
+ } from "./providers/model-discovery.js";
+ import type { ProviderDefinition } from "./providers/provider-definitions.js";
+ import { clearRuntimeRegistry, registerRuntimeProvider } from "./providers/runtime-providers.js";
++import { _clearChatCapabilityIndex } from "./providers/transport/probe-discovery.js";
+
+ const PROVIDER = "picker-outcome-test";
+ const DISPLAY_NAME = "Picker Outcome Test";
+ const ENV_VAR = "PICKER_OUTCOME_TEST_API_KEY";
+ const KEY_URL = "https://picker-outcome.invalid/key";
++const FORMAT = "picker-outcome-test-format";
++const ENDPOINT = "https://picker-outcome.invalid/v1/models";
+
+ const realFetch = globalThis.fetch;
+ const realGetRequestAuth = credentials.getRequestAuth;
+ const realCollapse = _modelsCatalogCollapse.collapse;
+
+ function defineProvider(overrides: Partial<ProviderDefinition> = {}): ProviderDefinition {
+ return {
+ name: PROVIDER,
+ displayName: DISPLAY_NAME,
+ transport: "openai",
+ baseUrl: "https://picker-outcome.invalid",
+ apiPath: "/v1/chat/completions",
+ apiKeyEnvVar: ENV_VAR,
+ apiKeyDescription: "Offline test key",
+ apiKeyUrl: KEY_URL,
+ shortcuts: [],
+ legacyPrefixes: [],
- modelDiscovery: { path: "/v1/models", format: "openai-models-list" },
++ modelDiscovery: { path: "", format: FORMAT as "ollama-tags" },
+ createHandler: { kind: "none", reason: "virtual", note: "Offline test fixture" },
+ isDirectApi: true,
+ ...overrides,
+ };
+ }
+
-/** The provider's live endpoint answers with exactly these ids. */
-function serves(ids: string[]): void {
- globalThis.fetch = mock(
- async () =>
- new Response(JSON.stringify({ data: ids.map((id) => ({ id })) }), {
- status: 200,
- headers: { "Content-Type": "application/json" },
- })
- ) as unknown as typeof fetch;
++let fetcherResult: FetcherResult;
++
++function reported(
++ capability: NonNullable<DiscoveredModel["reported"]>,
++ ...ids: string[]
++): DiscoveredModel[] {
++ return ids.map((id) => ({ id, reported: capability }));
++}
++
++/** The provider's dynamic models catalog answers with exactly these evidence-bearing rows. */
++function serves(models: DiscoveredModel[]): void {
++ fetcherResult = { kind: "models", models, endpoint: ENDPOINT };
+ }
+
+ function rejectsWith(status: number): void {
- globalThis.fetch = mock(
- async () => new Response(JSON.stringify({ error: "rejected" }), { status })
- ) as unknown as typeof fetch;
++ fetcherResult = {
++ kind: "failed",
++ failure: {
++ kind: status === 401 ? "unauthorized" : "http-error",
++ endpoint: ENDPOINT,
++ status,
++ detail: '{"error":"rejected"}',
++ },
++ };
+ }
+
+ /** A CatalogClient that answers from memory. `null` rows means "catalog fails". */
+ function stubCatalog(rows: CatalogModel[] | null): CatalogClient {
+ return {
+ async modelsByVendor() {
+ if (rows === null) throw new Error("catalog unavailable");
+ return rows;
+ },
+ async vendorsForModel() {
+ return null;
+ },
+ async searchModels() {
+ return [];
+ },
+ servedByVendor() {
+ return rows === null ? [] : rows;
+ },
+ };
+ }
+
+ const CATALOG_ROWS: CatalogModel[] = [
+ { modelId: "catalog-chat-1", displayName: "Catalog Chat 1", provider: "moonshotai" },
+ { modelId: "catalog-chat-2", displayName: "Catalog Chat 2", provider: "moonshotai" },
+ ];
+
+ beforeEach(() => {
+ invalidateModelDiscovery();
+ clearRuntimeRegistry();
++ _clearChatCapabilityIndex();
++ fetcherResult = {
++ kind: "failed",
++ failure: { kind: "unreachable", endpoint: ENDPOINT, detail: "fixture not configured" },
++ };
+ registerRuntimeProvider(defineProvider());
++ registerModelDiscoveryFetcher(FORMAT, async () => fetcherResult);
+ credentials.getRequestAuth = mock(async () => ({
+ headers: { Authorization: "Bearer offline-test-token" },
+ }));
+ globalThis.fetch = mock(async () => {
+ throw new Error("Unexpected fetch call");
+ }) as unknown as typeof fetch;
+ });
+
+ afterEach(() => {
+ invalidateModelDiscovery();
+ clearRuntimeRegistry();
++ _clearChatCapabilityIndex();
+ globalThis.fetch = realFetch;
+ credentials.getRequestAuth = realGetRequestAuth;
+ _modelsCatalogCollapse.collapse = realCollapse;
+ });
+
++afterAll(() => {
++ _clearChatCapabilityIndex();
++ mock.module("./providers/all-models-cache.js", () => __realAllModelsCacheExports);
++});
++
+ describe("buildDiscoveredModelOutcome — all five states, told apart", () => {
+ test("STATE A · a healthy dynamic models catalog is `rows`, non-empty, with servedCount === chatCount", async () => {
- serves(["alpha-chat", "beta-chat", "gamma-chat"]);
++ serves(reported("chat", "alpha-chat", "beta-chat", "gamma-chat"));
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+
+ expect(outcome.kind).toBe("rows");
+ if (outcome.kind !== "rows") throw new Error("unreachable");
+ expect(outcome.rows.map((r) => r.id).sort()).toEqual(["alpha-chat", "beta-chat", "gamma-chat"]);
+ expect(outcome.rows.length).toBeGreaterThan(0);
+ expect(outcome.servedCount).toBe(3);
+ expect(outcome.chatCount).toBe(3);
+ });
+
+ test("a PARTIALLY filtered dynamic models catalog keeps the two counts apart — the fact discarded today", async () => {
- serves(["alpha-chat", "text-embedding-3-large", "whisper-1"]);
++ serves([
++ ...reported("chat", "alpha-chat"),
++ ...reported("not-chat", "text-embedding-3-large", "whisper-1"),
++ ]);
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+
+ expect(outcome.kind).toBe("rows");
+ if (outcome.kind !== "rows") throw new Error("unreachable");
+ expect(outcome.rows.map((r) => r.id)).toEqual(["alpha-chat"]);
+ expect(outcome.servedCount).toBe(3);
+ expect(outcome.chatCount).toBe(1);
+ expect(outcome.chatCount).toBeLessThan(outcome.servedCount);
+ });
+
+ test("STATE B · served N, none chat-capable → all-filtered with the count and ≤3 sample ids", async () => {
+ // Invisible today: the provider is healthy, the key is fine, and the picker
+ // silently shows the cloud catalog instead. The sample ids are what make the
+ // notice self-explaining — they are usually embeddings or route wildcards.
- serves(["text-embedding-3-large", "whisper-1", "dall-e-3", "nomic-embed-text"]);
++ serves(
++ reported("not-chat", "text-embedding-3-large", "whisper-1", "dall-e-3", "nomic-embed-text")
++ );
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+
+ expect(outcome.kind).toBe("all-filtered");
+ if (outcome.kind !== "all-filtered") throw new Error("unreachable");
+ expect(outcome.servedCount).toBe(4);
+ expect(outcome.sampleIds.length).toBe(3);
+ for (const id of outcome.sampleIds) {
+ expect(["text-embedding-3-large", "whisper-1", "dall-e-3", "nomic-embed-text"]).toContain(id);
+ }
+ expect(outcome.fallbackRows.map((r) => r.id).sort()).toEqual([
+ "catalog-chat-1",
+ "catalog-chat-2",
+ ]);
+ });
+
+ test("STATE C · a genuinely empty dynamic models catalog is its own variant, carrying the failure", async () => {
+ serves([]);
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+
+ expect(outcome.kind).toBe("empty-models-catalog");
+ if (outcome.kind !== "empty-models-catalog") throw new Error("unreachable");
+ expect(outcome.failure.kind).toBe("empty-models-catalog");
+ expect(outcome.failure.endpoint).toContain("/v1/models");
+ expect(outcome.fallbackRows.length).toBe(2);
+ });
+
+ test("STATE D · a rejected key is `failed` — a DIFFERENT variant from empty-models-catalog", async () => {
+ rejectsWith(401);
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+
+ expect(outcome.kind).toBe("failed");
+ if (outcome.kind !== "failed") throw new Error("unreachable");
+ expect(outcome.failure.kind).toBe("unauthorized");
+ expect(outcome.failure.status).toBe(401);
+ // The whole point of the split: a 401 and an empty list are not the same
+ // thing, and the type no longer lets a renderer treat them as one.
+ expect(outcome.kind).not.toBe("empty-models-catalog");
+ expect(outcome.notice.join("")).toContain(DISPLAY_NAME);
+ expect(outcome.notice.join("")).toContain(ENV_VAR);
+ expect(outcome.fallbackRows.map((r) => r.id).sort()).toEqual([
+ "catalog-chat-1",
+ "catalog-chat-2",
+ ]);
+ });
+
+ test("STATE E · a dynamic models catalog that collapses to nothing is collapsed-empty, never rows:[]", async () => {
+ // Unreachable through the one shipped resolver (every entry lands in a group
+ // and every group yields a choice), and modelled anyway: `{kind:"rows",
+ // rows: []}` would be an empty panel with no explanation — the exact defect
+ // class this type exists to remove, reintroduced inside it.
+ _modelsCatalogCollapse.collapse = () => [];
- serves(["alpha-chat", "beta-chat"]);
++ serves(reported("chat", "alpha-chat", "beta-chat"));
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+
+ expect(outcome.kind).toBe("collapsed-empty");
+ if (outcome.kind !== "collapsed-empty") throw new Error("unreachable");
+ expect(outcome.servedCount).toBe(2);
+ expect(outcome.chatCount).toBe(2);
+ expect(outcome.fallbackRows.length).toBe(2);
+ });
+
+ test("a provider with no modelDiscovery is `unsupported`, which renders nothing", async () => {
+ clearRuntimeRegistry();
+ registerRuntimeProvider(defineProvider({ modelDiscovery: undefined }));
+
+ expect(
+ await buildDiscoveredModelOutcome(PROVIDER, DISPLAY_NAME, stubCatalog(CATALOG_ROWS))
+ ).toEqual({ kind: "unsupported", reason: "no-descriptor" });
+ });
+
+ test("`rows` is non-empty on every path that produces it", async () => {
+ // The invariant `rows: [ModelInfo, ...ModelInfo[]]` states at the type level;
+ // this asserts it at runtime across every shape that reaches the branch.
+ for (const ids of [["only-one"], ["a-chat", "b-chat"], ["a-chat", "whisper-1"]]) {
+ invalidateModelDiscovery();
- serves(ids);
++ serves(
++ ids.map((id) => ({
++ id,
++ reported: id === "whisper-1" ? "not-chat" : "chat",
++ }))
++ );
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+ expect(outcome.kind).toBe("rows");
+ if (outcome.kind !== "rows") throw new Error("unreachable");
+ expect(outcome.rows.length).toBeGreaterThan(0);
+ }
+ });
+ });
+
+ describe("buildDiscoveredModelOutcome — the fallback leg", () => {
+ test("a failing catalog leaves fallbackRows empty instead of rejecting", async () => {
+ rejectsWith(500);
+
+ const outcome = await buildDiscoveredModelOutcome(PROVIDER, DISPLAY_NAME, stubCatalog(null));
+
+ expect(outcome.kind).toBe("failed");
+ if (outcome.kind !== "failed") throw new Error("unreachable");
+ expect(outcome.fallbackRows).toEqual([]);
+ // With nothing to show, "manual model entry" is finally the true sentence.
+ expect(outcome.notice.join("")).toContain("Falling back to manual model entry.");
+ });
+
+ test("with catalog rows in hand the notice names the catalog, not manual entry", async () => {
+ rejectsWith(401);
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog(CATALOG_ROWS)
+ );
+
+ if (outcome.kind !== "failed") throw new Error("unreachable");
+ const notice = outcome.notice.join("");
+ expect(notice).toContain("cloud-catalog entries below — not its live model list.");
+ expect(notice).not.toContain("Falling back to manual model entry.");
+ });
+
+ test("the catalog fallback is chat-filtered too, via the same chokepoint", async () => {
+ rejectsWith(401);
+
+ const outcome = await buildDiscoveredModelOutcome(
+ PROVIDER,
+ DISPLAY_NAME,
+ stubCatalog([
+ ...CATALOG_ROWS,
+ { modelId: "gpt-image-2.5-flare", displayName: "Image", provider: "openai" },
+ ])
+ );
+
+ if (outcome.kind !== "failed") throw new Error("unreachable");
+ expect(outcome.fallbackRows.map((r) => r.id)).not.toContain("gpt-image-2.5-flare");
+ });
+ });
+
+ describe("buildDiscoveredModelRows stays the wrapper the classic path uses", () => {
+ test("rows on success, [] for every other outcome", async () => {
- serves(["alpha-chat"]);
++ serves(reported("chat", "alpha-chat"));
+ expect(
+ (await buildDiscoveredModelRows(PROVIDER, DISPLAY_NAME, stubCatalog(CATALOG_ROWS))).map(
+ (r) => r.id
+ )
+ ).toEqual(["alpha-chat"]);
+
+ invalidateModelDiscovery();
+ rejectsWith(401);
+ expect(
+ await buildDiscoveredModelRows(PROVIDER, DISPLAY_NAME, stubCatalog(CATALOG_ROWS))
+ ).toEqual([]);
+ });
+
+ test("the failure is still RECORDED, so warnDiscoveryFailure can find it afterwards", async () => {
+ // The classic path throws the outcome away and asks the module-global map
+ // why the list was empty. Moving the classification must not break that.
+ rejectsWith(401);
+ await buildDiscoveredModelRows(PROVIDER, DISPLAY_NAME, stubCatalog(CATALOG_ROWS));
+
+ const stderr = spyOn(process.stderr, "write").mockImplementation(() => true);
+ try {
+ warnDiscoveryFailure(PROVIDER, DISPLAY_NAME, {
+ apiKeyEnvVar: ENV_VAR,
+ apiKeyUrl: KEY_URL,
+ });
+ expect(stderr.mock.calls.length).toBeGreaterThan(0);
+ expect(stderr.mock.calls.map(([c]) => String(c)).join("")).toContain("HTTP 401");
+ } finally {
+ stderr.mockRestore();
+ }
+ });
+ });
+
+ describe("formatDiscoveryFailureNotice", () => {
+ const def = { apiKeyEnvVar: ENV_VAR, apiKeyUrl: KEY_URL };
+
+ function failure(kind: DiscoveryFailureKind): DiscoveryFailure {
+ return {
+ kind,
+ provider: PROVIDER,
+ endpoint: "https://picker-outcome.invalid/v1/models",
+ status: kind === "unauthorized" || kind === "http-error" ? 401 : undefined,
+ detail: "upstream said no",
+ };
+ }
+
+ test.each(["unauthorized", "no-credentials"] as const)(
+ "%s earns the env var and the key URL — the two actionable kinds",
+ (kind) => {
+ const text = formatDiscoveryFailureNotice(DISPLAY_NAME, failure(kind), def).join("");
+
+ expect(text).toContain(DISPLAY_NAME);
+ expect(text).toContain(ENV_VAR);
+ expect(text).toContain(KEY_URL);
+ expect(text).toContain("Falling back to manual model entry.");
+ }
+ );
+
+ test.each(["http-error", "unreachable", "malformed"] as const)(
+ "%s omits credential guidance, because there is no credential step to take",
+ (kind) => {
+ const text = formatDiscoveryFailureNotice(DISPLAY_NAME, failure(kind), def).join("");
+
+ expect(text).toContain(DISPLAY_NAME);
+ expect(text).not.toContain(ENV_VAR);
+ expect(text).not.toContain(KEY_URL);
+ expect(text).toContain("Falling back to manual model entry.");
+ }
+ );
+
+ test("the three fallbacks produce three different final lines", () => {
+ const lines = (f: Parameters<typeof formatDiscoveryFailureNotice>[3]) =>
+ formatDiscoveryFailureNotice(DISPLAY_NAME, failure("unauthorized"), def, f).join("");
+
+ expect(lines("manual-entry")).toContain("Falling back to manual model entry.");
+ expect(lines("catalog")).toContain(
+ `Showing ${DISPLAY_NAME}'s cloud-catalog entries below — not its live model list.`
+ );
+ // "unknown" means nothing has decided yet, so it claims nothing.
+ expect(lines("unknown")).not.toContain("Falling back");
+ expect(lines("unknown")).not.toContain("cloud-catalog");
+ });
+
+ test("the default is byte-identical to what warnDiscoveryFailure writes to stderr", async () => {
+ // The coupling between the formatter and the sink, asserted rather than
+ // assumed: the classic path's wording must not drift when the TUI's does.
+ rejectsWith(401);
+ await buildDiscoveredModelRows(PROVIDER, DISPLAY_NAME, stubCatalog(null));
+
+ const stderr = spyOn(process.stderr, "write").mockImplementation(() => true);
+ let written: string;
+ try {
+ warnDiscoveryFailure(PROVIDER, DISPLAY_NAME, def);
+ written = stderr.mock.calls.map(([chunk]) => String(chunk)).join("");
+ } finally {
+ stderr.mockRestore();
+ }
+
+ const recorded: DiscoveryFailure = {
+ kind: "unauthorized",
+ provider: PROVIDER,
+ endpoint: "https://picker-outcome.invalid/v1/models",
+ status: 401,
+ detail: '{"error":"rejected"}',
+ };
+ expect(written).toBe(formatDiscoveryFailureNotice(DISPLAY_NAME, recorded, def).join(""));
+ });
+ });
+
+ describe("no rendered notice interpolates `undefined`", () => {
+ // A Record, not an array, so the COMPILER enforces "every kind": adding a
+ // kind to `DiscoveryFailureKind` without listing it here fails typecheck. An
+ // array did not — `incomplete` arrived in the union and this list, which
+ // calls itself exhaustive, kept passing without it.
+ const EVERY_KIND: Record<DiscoveryFailureKind, true> = {
+ "no-credentials": true,
+ unauthorized: true,
+ "http-error": true,
+ unreachable: true,
+ malformed: true,
+ incomplete: true,
+ "empty-models-catalog": true,
+ };
+ const KINDS = Object.keys(EVERY_KIND) as DiscoveryFailureKind[];
+
+ test("every kind × endpoint × status × detail × fallback", () => {
+ // The `empty-models-catalog` records on the fetcher half carried no endpoint for
+ // years, so any copy that names one renders the literal string `undefined`
+ // to the user. Exhaustive rather than representative, because the optional
+ // fields are independent and a single missed guard is invisible in review.
+ let checked = 0;
+ for (const kind of KINDS) {
+ for (const endpoint of [undefined, "https://picker-outcome.invalid/v1/models"]) {
+ for (const status of [undefined, 401, 500]) {
+ for (const detail of [undefined, "", "upstream said no"]) {
+ const f: DiscoveryFailure = { kind, provider: PROVIDER, endpoint, status, detail };
+
+ expect(describeDiscoveryFailure(f)).not.toContain("undefined");
+
+ for (const fallback of ["manual-entry", "catalog", "unknown"] as const) {
+ for (const def of [
+ { apiKeyEnvVar: ENV_VAR, apiKeyUrl: KEY_URL },
+ { apiKeyEnvVar: "", apiKeyUrl: "" },
+ ]) {
+ const text = formatDiscoveryFailureNotice(DISPLAY_NAME, f, def, fallback).join("");
+ expect(text).not.toContain("undefined");
+ checked++;
+ }
+ }
+ }
+ }
+ }
+ }
+ expect(checked).toBe(KINDS.length * 2 * 3 * 3 * 3 * 2);
+ });
+ });
+
+ describe("toPickerRows — one chat-capability chokepoint for both doors", () => {
+ const row = (id: string): ModelInfo => ({
+ id,
+ name: id,
+ description: id,
+ provider: "test",
+ });
+
- test("drops real non-chat ids", () => {
++ test("drops ids the fixture describes as non-chat", () => {
+ // `gpt-image-2.5-flare` is the model F1 observed being selected by pressing
+ // Enter twice at launch: the catalog paths applied no capability filter at
+ // all, only the live-discovery path did.
+ const dropped = [
+ "gpt-image-2.5-flare",
+ "text-embedding-3-large",
+ "whisper-1",
+ "dall-e-3",
+ "tts-1-hd",
+ "nomic-embed-text",
+ "bge-m3",
+ "gemini/*",
+ ];
+
+ expect(toPickerRows(dropped.map(row))).toEqual([]);
+ });
+
- test("keeps real chat ids", () => {
++ test("keeps ids the fixture describes as chat", () => {
+ const kept = [
+ "claude-opus-5",
+ "gpt-5.2-codex",
+ "gemini-3-pro",
+ "kimi-k3",
+ "deepseek-v3",
+ "llama3.2",
+ "MiniMax-M2",
+ ];
+
+ expect(
+ toPickerRows(kept.map(row))
+ .map((r) => r.id)
+ .sort()
+ ).toEqual([...kept].sort());
+ });
+
++ test("drops a catalog-unknown id because no source supplies chat evidence", () => {
++ expect(toPickerRows([row("fixture-has-no-evidence-for-this-id")])).toEqual([]);
++ });
++
+ test("dedupes by id while filtering", () => {
+ expect(
+ toPickerRows([row("claude-opus-5"), row("claude-opus-5"), row("whisper-1")]).map((r) => r.id)
+ ).toEqual(["claude-opus-5"]);
+ });
+ });
diff --cc packages/cli/src/model-selector.test.ts
index 1ce45257,9b24851e..d7df467d
--- a/packages/cli/src/model-selector.test.ts
+++ b/packages/cli/src/model-selector.test.ts
@@@ -174,148 -165,6 +174,148 @@@ describe("warnDiscoveryFailure", () =>
);
});
+describe("provider-reported chat capability in discovered model callers", () => {
+ const provider = "reported-capability-test";
+ const format = "reported-capability-test-format";
+ let discovered: DiscoveredModel[] = [];
+
+ const catalog = createCatalogClient({
+ getModelsByProvider: async () => {
+ throw new Error("local discovered rows must not query the cloud catalog");
+ },
+ readSlimCache: () => null,
+ });
+
+ beforeEach(() => {
+ invalidateModelDiscovery();
+ clearRuntimeRegistry();
+ _clearChatCapabilityIndex();
+ discovered = [];
+
+ const localTemplate = getProviderByName("ollama")!;
+ registerRuntimeProvider({
+ ...localTemplate,
+ name: provider,
+ displayName: "Reported Capability Test",
+ shortcuts: [],
+ legacyPrefixes: [],
+ modelDiscovery: {
+ path: "",
+ format: format as "ollama-tags",
+ },
+ });
- registerModelDiscoveryFetcher(format, async () => discovered);
++ registerModelDiscoveryFetcher(format, async () => ({ kind: "models", models: discovered }));
+ });
+
+ afterEach(() => {
+ invalidateModelDiscovery();
+ clearRuntimeRegistry();
+ _clearChatCapabilityIndex();
+ });
+
+ test("buildDiscoveredModelRows keeps catalog-unknown rows reported as chat", async () => {
+ discovered = [
+ { id: "swe-1-7-medium", reported: "chat" },
+ { id: "gemini-3.6-flash-high", reported: "chat" },
+ { id: "llama3.2:3b-q4_K_M", reported: "chat" },
+ ];
+
+ const rows = await buildDiscoveredModelRows(provider, "Reported Capability Test", catalog);
+
+ expect(rows.map((row) => row.id).sort()).toEqual(
+ ["swe-1-7-medium", "gemini-3.6-flash-high", "llama3.2:3b-q4_K_M"].sort()
+ );
+ });
+
+ test("buildDiscoveredModelRows drops a row reported as not-chat", async () => {
+ discovered = [{ id: "nomic-embed-text:latest", reported: "not-chat" }];
+
+ expect(await buildDiscoveredModelRows(provider, "Reported Capability Test", catalog)).toEqual(
+ []
+ );
+ });
+
+ test("buildDiscoveredModelRows still drops a catalog-unknown row with no report", async () => {
+ discovered = [{ id: "granite3.2-vision:2b-instruct-q4_K_M" }];
+
+ expect(await buildDiscoveredModelRows(provider, "Reported Capability Test", catalog)).toEqual(
+ []
+ );
+ });
+
+ test("discoverProviderProbeModel picks from catalog-unknown rows reported as chat", async () => {
+ discovered = [
+ { id: "swe-1-7-medium", reported: "chat" },
+ { id: "gemini-3.6-flash-high", reported: "chat" },
+ { id: "llama3.2:3b-q4_K_M", reported: "chat" },
+ ];
+
+ const outcome = await discoverProviderProbeModel(provider, "Reported Capability Test");
+
+ expect(outcome.model).not.toBeNull();
+ expect(discovered.map((row) => row.id)).toContain(outcome.model!);
+ expect(outcome.reason).toBeUndefined();
+ });
+
+ test("discoverProviderProbeModel still rejects a catalog-unknown row with no report", async () => {
+ discovered = [{ id: "granite3.2-vision:2b-instruct-q4_K_M" }];
+
+ expect(await discoverProviderProbeModel(provider, "Reported Capability Test")).toEqual({
+ model: null,
+ reason: "no chat-capable model among the 1 listed by Reported Capability Test",
+ });
+ });
+});
+
+describe("Ollama discovery capability reports", () => {
+ const realFetch = globalThis.fetch;
+
+ beforeEach(() => {
+ invalidateModelDiscovery("ollama");
+ });
+
+ afterEach(() => {
+ globalThis.fetch = realFetch;
+ invalidateModelDiscovery("ollama");
+ });
+
+ test("the Ollama fetcher carries completion reports and preserves an absent capability", async () => {
+ globalThis.fetch = mock(
+ async () =>
+ new Response(
+ JSON.stringify({
+ models: [
+ { name: "llama3.2:3b-q4_K_M", capabilities: ["completion", "tools"] },
+ { name: "granite3.2-vision:2b-instruct-q4_K_M" },
+ ],
+ }),
+ { status: 200 }
+ )
+ ) as unknown as typeof fetch;
+
+ expect(await discoverProviderModels("ollama")).toEqual([
+ {
+ id: "llama3.2:3b-q4_K_M",
+ displayName: "llama3.2:3b-q4_K_M",
+ supportsTools: true,
+ reported: "chat",
+ },
+ {
+ id: "granite3.2-vision:2b-instruct-q4_K_M",
+ displayName: "granite3.2-vision:2b-instruct-q4_K_M",
+ supportsTools: false,
+ reported: undefined,
+ },
+ ]);
+ });
+
+ test("ollamaReported maps completion, embedding, and silence", () => {
+ expect(ollamaReported({ capabilities: ["completion", "tools"] })).toBe("chat");
+ expect(ollamaReported({ capabilities: ["embedding"] })).toBe("not-chat");
+ expect(ollamaReported({})).toBeUndefined();
+ });
+});
+
describe("buildExplicitModelSpec", () => {
test.each([
["zen", "claude-opus-4-7", "zen@claude-opus-4-7"],
diff --cc packages/cli/src/model-selector.ts
index 193cf76b,564c6630..0f33a086
--- a/packages/cli/src/model-selector.ts
+++ b/packages/cli/src/model-selector.ts
@@@ -49,7 -52,14 +52,14 @@@ import
getProviderByName,
} from "./providers/provider-definitions.js";
import { getRuntimeProviders } from "./providers/runtime-providers.js";
- import { isReportedChatCapable } from "./providers/transport/probe-discovery.js";
-import { isChatCapable } from "./providers/transport/probe-discovery.js";
++import { isChatCapable, isReportedChatCapable } from "./providers/transport/probe-discovery.js";
+ // STATIC, and safe: both are true leaves with ZERO imports of their own
+ // (`picker/import-direction.test.ts` asserts exactly that, by name). A static
+ // import of the picker ITSELF would pull a renderer into this module's graph and
+ // from there into the MCP stdio path — which is why `selectModelInteractive`
+ // reaches it by `await import()` instead.
+ import { canDrawTui } from "./tui/runtime/can-draw-tui.js";
+ import { NoTtyError, PickerCancelled } from "./tui/runtime/picker-cancelled.js";
/**
* Model data structure
@@@ -439,6 -449,69 +449,67 @@@ function sortModelsNewestFirst(models:
return [...models].sort(compareByReleaseDateDesc);
}
+ /**
+ * Catalog models → picker rows: drop what cannot serve a chat completion, then
+ * dedupe, then order newest-first.
+ *
+ * **A chokepoint, not a site fix.** Pressing Enter twice at launch selected
+ * `gpt-image-2.5-flare`, an image-generation model: `isChatCapable` was applied
+ * inside `buildDiscoveredModelOutcome` and nowhere else, so every catalog path —
+ * `searchModels`, `modelsByVendor`, the top-100 default list — applied no
+ * capability filter at all. The same class of leak has been patched
+ * twice before at the site where it was noticed (`probe-discovery.ts:498`, then
+ * `pplx-embed`, then `bge`), and recurred each time, because a list is made in
+ * more than one place. Every list either UI can render now returns through here.
+ *
- * HONEST LIMIT, AND IT IS LOGGED RATHER THAN LEFT AS A COMMENT. `isChatCapable`
- * reads a model ID, and an ID is not a modality contract. The transcribe / voice
- * / video families now have patterns (`probe-discovery.ts`, which records why),
- * but a generator whose name says nothing — `seedance-2.5` (video),
- * `flux-2-pro` (image) — cannot be excluded without a vendor-family blacklist,
- * which is the workaround that has already recurred three times.
++ * The rule is evidence, never a name: a row is kept only when the cloud models
++ * catalog publishes it as a chat model — text among its inputs AND among its
++ * outputs, with any other modalities alongside. A row the catalog does not
++ * describe is `unknown` and dropped. `gemini-3.5-transcribe` (`in: ["audio"]`)
++ * is out by its input list. `gpt-realtime-2` (`in: [audio,image,text]`,
++ * `out: [audio,text]`) is IN, by decision: the name patterns that used to hide
++ * the realtime family went with the name-based fallback, and a model that
++ * accepts text and writes text is a chat model.
+ *
- * The complete fix is a modality field on the slim catalog payload; `ModelDoc`
- * carries `capabilities.imageGeneration` / `audioInput` / `audioOutput` /
- * `embedding` and the slim entry carries none of them, so every aggregator-served
- * list is name-classified only. That is a models-index gap, filed there. What is
- * owed HERE is visibility, so the residue is named ONCE per process in the debug
- * log — a picker screenshot is otherwise the only place it shows up.
++ * What is left to watch is the catalog being WRONG, so a row that survives but
++ * whose name reads like a generator — `seedance-2.5`, `flux-2-pro` — is named
++ * ONCE per process in the debug log. That decides nothing; it points at a
++ * catalog row to correct in models-index.
+ */
+ export function toPickerRows(models: ModelInfo[]): ModelInfo[] {
+ const kept = sortModelsNewestFirst(dedupeModels(models.filter((m) => isChatCapable(m.id))));
+ logModalityGapOnce(kept);
+ return kept;
+ }
+
-/** Names that are known non-chat but carry no modality signal in the ID itself. */
++/** Generator families whose rows, if the catalog calls them chat, are worth a second look. */
+ const UNINFERABLE_NON_CHAT = [/\bseedance\b/i, /\bflux\b/i, /\bveo\b/i, /\bimagen\b/i];
+
+ let _modalityGapLogged = false;
+
+ /**
- * Say in the log which rows survived the name filter but are known-suspect, and
- * say why nothing more can be done in this repo.
++ * Say in the log which rows the catalog kept as chat although their family
++ * generates images or video — a catalog row to check, not a filter to add here.
+ *
+ * ONCE per process: `toPickerRows` is the chokepoint for every list either UI
+ * renders, so a per-call line would be dozens of identical lines per picker open.
+ * It never writes to a terminal — `log()` goes to the debug file — because the
+ * renderer owns the terminal while the picker is up.
+ */
+ function logModalityGapOnce(kept: ModelInfo[]): void {
+ if (_modalityGapLogged) return;
+ const suspect = kept.filter((m) => UNINFERABLE_NON_CHAT.some((re) => re.test(m.id)));
+ if (suspect.length === 0) return;
+ _modalityGapLogged = true;
+ log(
- `[Models] ${suspect.length} row(s) may not be chat models and cannot be classified from the id: ` +
++ `[Models] ${suspect.length} row(s) the catalog publishes as chat, from a generator family: ` +
+ `${suspect
+ .slice(0, 8)
+ .map((m) => m.id)
- .join(
- ", "
- )}. The slim catalog carries no modality field — models-index gap, not a CLI regex.`
++ .join(", ")}. If one is not a chat model, its models-index row is wrong.`
+ );
+ }
+
/**
* Get free models. Free model discovery used to come from OpenCode Zen
* (via models.dev), which has been removed. Free models now live in the
@@@ -1334,16 -1658,117 +1656,121 @@@ export async function buildDiscoveredMo
provider: string,
displayName: string,
catalog: CatalogClient
- ): Promise<ModelInfo[]> {
+ ): Promise<PickerDiscoveryOutcome> {
+ const [modelsCatalogResult, fallbackResult] = await Promise.allSettled([
+ discoverProviderModelsCatalog(provider),
+ loadModelsForPickerProvider(provider, catalog),
+ ]);
+
+ const fallbackRows = fallbackResult.status === "fulfilled" ? fallbackResult.value : [];
+ // `discoverProviderModelsCatalog` never rejects by contract, but a contract is not a
+ // type: honour it rather than asserting it, so a future regression there shows
+ // up as a named failure instead of an unhandled rejection behind a renderer.
+ const modelsCatalog: ModelsCatalogOutcome =
+ modelsCatalogResult.status === "fulfilled"
+ ? modelsCatalogResult.value
+ : {
+ kind: "failed",
+ failure: {
+ kind: "unreachable",
+ provider,
+ detail: String(
+ (modelsCatalogResult.reason as Error)?.message ?? modelsCatalogResult.reason
+ ),
+ },
+ };
+
+ if (modelsCatalog.kind === "unsupported")
+ return { kind: "unsupported", reason: modelsCatalog.reason };
+
+ if (modelsCatalog.kind === "failed") {
+ const { failure } = modelsCatalog;
+ // `empty-models-catalog` is its own variant rather than a `failed` with a kind to
+ // switch on, because it is not an error: the endpoint answered. It renders
+ // in the NOTICE tier, visibly distinct from a rejected key.
+ if (failure.kind === "empty-models-catalog") {
+ return { kind: "empty-models-catalog", failure, fallbackRows };
+ }
+ const def = getProviderByName(provider);
+ const notice = formatDiscoveryFailureNotice(
+ displayName,
+ failure,
+ // A runtime custom endpoint may not be in the definition table at all;
+ // empty strings suppress the two guidance lines, which is right — there is
+ // no env var or key URL to name.
+ def ?? { apiKeyEnvVar: "", apiKeyUrl: "" },
+ fallbackRows.length > 0 ? "catalog" : "manual-entry"
+ );
+ return { kind: "failed", failure, notice, fallbackRows };
+ }
+
+ const served = rankDiscoveredModels(modelsCatalog.models);
+ const servedCount = served.length;
- const discovered = served.filter((m) => isChatCapable(m.id));
+ // `m.reported` carries the provider's own chat/not-chat statement. Judging the
+ // bare `m.id` alone discarded it, and once `unknown` stopped counting as chat
+ // that emptied the picker for every provider whose ids the cloud catalog does
+ // not list verbatim: Ollama 19 → 1, Devin 247 → 3, Antigravity 21 → 7.
- const discovered = rankDiscoveredModels(await discoverProviderModels(provider)).filter((m) =>
- isReportedChatCapable(m.id, m.reported)
- );
- if (discovered.length === 0) return [];
++ const discovered = served.filter((m) => isReportedChatCapable(m.id, m.reported));
+ const chatCount = discovered.length;
+ if (chatCount === 0) {
+ // Served N, none of them chat-capable. Invisible today, and self-explaining
+ // only if the ids come with it — they are usually embeddings or wildcard
+ // routes, which a reader recognises at a glance.
+ return {
+ kind: "all-filtered",
+ servedCount,
+ sampleIds: served
- .filter((m) => !isChatCapable(m.id))
++ .filter((m) => !isReportedChatCapable(m.id, m.reported))
+ .slice(0, 3)
+ .map((m) => m.id),
+ fallbackRows,
+ };
+ }
+
+ const [head, ...tail] = buildRowsFromDiscovered(provider, displayName, discovered, fallbackRows);
+ if (head === undefined) {
+ // Unreachable through the one shipped resolver, and deliberately still
+ // modelled: `{kind:"rows", rows: []}` would be an empty panel with no
+ // explanation, which is the exact defect class this type removes. Destructured
+ // rather than length-checked so the non-empty tuple needs no cast to assert.
+ return { kind: "collapsed-empty", servedCount, chatCount, fallbackRows };
+ }
+ return { kind: "rows", rows: [head, ...tail], servedCount, chatCount };
+ }
+
+ /**
+ * The picker rows for a provider's dynamic models catalog, or `[]` for any of the six
+ * reasons there isn't one.
+ *
+ * A wrapper over {@link buildDiscoveredModelOutcome} since the outcome type
+ * landed. The classic inquirer path keeps using it and keeps asking
+ * `warnDiscoveryFailure` why the list was empty afterwards — which still works,
+ * because every failure is still recorded in `model-discovery.ts`'s map.
+ */
+ export async function buildDiscoveredModelRows(
+ provider: string,
+ displayName: string,
+ catalog: CatalogClient
+ ): Promise<ModelInfo[]> {
+ const outcome = await buildDiscoveredModelOutcome(provider, displayName, catalog);
+ return outcome.kind === "rows" ? outcome.rows : [];
+ }
+ /**
+ * Chat-capable discovered models → picker rows. Pure; no I/O.
+ *
+ * `catalogRows` is the vendor catalog's list for this provider, already fetched
+ * by the caller. A per-token discovery provider should show the real rate, and
+ * neither the live endpoint nor the slim catalog reports one — so the price
+ * comes from there. Flat-rate providers (every discovery provider today) ignore
+ * it entirely.
+ */
+ function buildRowsFromDiscovered(
+ provider: string,
+ displayName: string,
+ discovered: DiscoveredModel[],
+ catalogRows: ModelInfo[]
+ ): ModelInfo[] {
const subscription = isSubscriptionProvider(provider);
const local = getProviderByName(provider)?.isLocal === true;
// Neither a flat-rate plan nor a local daemon charges per token, and the
diff --cc packages/cli/src/providers/model-availability.test.ts
index 019ea630,7374cbe5..9344bc04
--- a/packages/cli/src/providers/model-availability.test.ts
+++ b/packages/cli/src/providers/model-availability.test.ts
@@@ -96,10 -69,6 +96,13 @@@ function stubModelsCatalog(...ids: stri
) as unknown as typeof fetch;
}
+function stubDevinModelsCatalog(): void {
- registerModelDiscoveryFetcher("devin-connect", async () => devinModelsCatalog);
++ registerModelDiscoveryFetcher("devin-connect", async () => ({
++ kind: "models",
++ models: devinModelsCatalog,
++ }));
+}
+
beforeEach(() => {
_resetCatalogClient();
// Keep every test off the real on-disk catalog, including discovery tests.
diff --cc packages/cli/src/providers/model-discovery-builtins.ts
index f024cf90,91e0bc8c..6dff1e2c
--- a/packages/cli/src/providers/model-discovery-builtins.ts
+++ b/packages/cli/src/providers/model-discovery-builtins.ts
@@@ -27,22 -39,56 +39,61 @@@ function oneLine(text: string, max = 20
* Carries the full variant metadata, not just id/name/window: the picker folds
* ~170 uids into ~42 rows and needs the group label, the cost multiplier, the
* promo and the vendor's own default flag to do it.
+ *
+ * **Honest limit on the classification.** `getServedDevinModels` catches its own
+ * rpc errors and returns `modelsCatalogCache ?? []` (devin-models.ts), and
+ * `postUnary` answers `null` rather than throwing, so a failed rpc still arrives
+ * here as an empty list and is reported as `empty-models-catalog`. What this DOES
+ * distinguish is the two states that are knowable without touching that module:
+ * no credential at all, and a throw from anywhere in the chain.
*/
- async function fetchDevinModelsCatalog(): Promise<DiscoveredModel[]> {
- const { getServedDevinModels } = await import("./devin/devin-models.js");
- const served = await getServedDevinModels();
- if (served.length === 0) return [];
+ async function fetchDevinModelsCatalog(): Promise<FetcherResult> {
+ let endpoint: string | undefined;
+ try {
+ const { readDevinApiKey, readDevinServerUrl } = await import("./devin/devin-credentials.js");
+ endpoint = readDevinServerUrl();
+ // A logged-out account is a CREDENTIAL problem, and it is the kind that
+ // earns a "how to fix it" line. Reporting it as an empty dynamic models catalog says the
+ // subscription serves no models, which is a claim about Devin rather than
+ // about this machine.
+ if (!readDevinApiKey()) {
+ return {
+ kind: "failed",
+ failure: {
+ kind: "no-credentials",
+ endpoint,
+ // The definition's own `apiKeyDescription`, verbatim: there is no
+ // `claudish login devin` to point at — the token is the Devin CLI's,
+ // and claudish only reads it.
+ detail: "no Devin CLI session token (~/.local/share/devin/credentials.toml)",
+ },
+ };
+ }
+
+ const { getServedDevinModels } = await import("./devin/devin-models.js");
+ const served = await getServedDevinModels();
+ if (served.length === 0) return { kind: "models", models: [], endpoint };
- const { devinModelsCatalogEntry } = await import("./model-resolvers/devin.js");
- return served.map((model) => {
- const { wireId, ...rest } = devinModelsCatalogEntry(model);
- // `chat` is Devin's statement, not ours. This list is the set of models
- // Devin's own agent can drive — capability ∩ entitlement, already filtered
- // to configs with a context window — so every uid in it answers chat turns.
- // Its uids are knob-encoded (`swe-1-7-medium`) and the cloud catalog never
- // lists them verbatim, so judged by id alone 244 of 247 read as unknown.
- return { id: wireId, ...rest, reported: "chat" as const };
- });
+ const { devinModelsCatalogEntry } = await import("./model-resolvers/devin.js");
+ return {
+ kind: "models",
+ endpoint,
+ models: served.map((model) => {
+ const { wireId, ...rest } = devinModelsCatalogEntry(model);
- return { id: wireId, ...rest };
++ // `chat` is Devin's statement, not ours. This list is the set of models
++ // Devin's own agent can drive — capability ∩ entitlement, already filtered
++ // to configs with a context window — so every uid in it answers chat turns.
++ // Its uids are knob-encoded (`swe-1-7-medium`) and the cloud catalog never
++ // lists them verbatim, so judged by id alone 244 of 247 read as unknown.
++ return { id: wireId, ...rest, reported: "chat" as const };
+ }),
+ };
+ } catch (err: unknown) {
+ return {
+ kind: "failed",
+ failure: { kind: "unreachable", endpoint, detail: oneLine(String(err)) },
+ };
+ }
}
/**
@@@ -72,62 -118,94 +123,111 @@@ function isUndeclaredEditorInternal(id
* in the catalog. `resolveDiscoveredContextLength` already prefers a discovered
* window over the catalog; this is what gives it one to prefer.
*/
- async function fetchAntigravityModelsCatalog(): Promise<DiscoveredModel[]> {
- const { getValidAntigravityAccessToken } = await import("../auth/antigravity-token.js");
- const { setupAntigravityUser, getServedAntigravityModels } = await import(
- "../auth/antigravity-user.js"
- );
- const token = await getValidAntigravityAccessToken();
- if (!token) return [];
+ async function fetchAntigravityModelsCatalog(): Promise<FetcherResult> {
+ let endpoint: string | undefined;
+ try {
+ const { getValidAntigravityAccessToken } = await import("../auth/antigravity-token.js");
+ const { setupAntigravityUser, getServedAntigravityModels, antigravityHost } = await import(
+ "../auth/antigravity-user.js"
+ );
+ endpoint = `${antigravityHost()}/v1internal:fetchAvailableModels`;
+ const token = await getValidAntigravityAccessToken();
+ // No token is exactly "logged out", which is one of the two kinds that earns
+ // the env-var / key-URL guidance. It used to read as an empty dynamic models catalog, i.e.
+ // "your subscription serves nothing".
+ if (!token) {
+ return {
+ kind: "failed",
+ failure: {
+ kind: "no-credentials",
+ endpoint,
+ detail: "not signed in — run `claudish login antigravity`",
+ },
+ };
+ }
- const { projectId } = await setupAntigravityUser(token);
- const { servedIds, meta, excludedIds } = await getServedAntigravityModels(token, projectId);
- // Declared first, guess second. `excludedIds` is the backend's own verdict —
- // internal flags, per-feature role bindings, and retired ids (which look
- // entirely normal but answer 400).
- const declaredExcluded = excludedIds ?? new Set<string>();
- const selectable = servedIds.filter(
- (id) => !declaredExcluded.has(id) && !isUndeclaredEditorInternal(id)
- );
- return selectable.map((id) => {
- const m = meta[id];
- // contextWindow is left UNSET when the backend reported none
- // (gemini-3.1-flash-image does), so the catalog still gets its turn rather
- // than the row rendering a fabricated 0 as "N/A".
- // Every id here is a tuned variant (`-high`, `-tiered`) the catalog does not
- // carry; see `ignoreCatalogReleaseDate` for the ordering this protects.
- // `chat` is the backend's statement: everything it declares non-selectable —
- // internal flags, per-feature role bindings, retired ids — was removed above,
- // and so were the undeclared editor-completion models. What remains is the
- // set it offers for chat. Judged by id alone, 14 of 21 read as unknown,
- // because these tuned variants are ids the cloud catalog does not carry.
- return m?.contextWindow
- ? {
- id,
- contextWindow: m.contextWindow,
- ignoreCatalogReleaseDate: true,
- reported: "chat" as const,
- }
- : { id, ignoreCatalogReleaseDate: true, reported: "chat" as const };
- });
+ const { projectId } = await setupAntigravityUser(token);
+ const { servedIds, meta, excludedIds } = await getServedAntigravityModels(token, projectId);
+ // Declared first, guess second. `excludedIds` is the backend's own verdict —
+ // internal flags, per-feature role bindings, and retired ids (which look
+ // entirely normal but answer 400).
+ const declaredExcluded = excludedIds ?? new Set<string>();
+ const selectable = servedIds.filter(
+ (id) => !declaredExcluded.has(id) && !isUndeclaredEditorInternal(id)
+ );
+ return {
+ kind: "models",
+ endpoint,
+ models: selectable.map((id) => {
+ const m = meta[id];
+ // contextWindow is left UNSET when the backend reported none
+ // (gemini-3.1-flash-image does), so the catalog still gets its turn rather
+ // than the row rendering a fabricated 0 as "N/A".
+ // Every id here is a tuned variant (`-high`, `-tiered`) the catalog does not
+ // carry; see `ignoreCatalogReleaseDate` for the ordering this protects.
++ // `chat` is the backend's statement: everything it declares non-selectable —
++ // internal flags, per-feature role bindings, retired ids — was removed above,
++ // and so were the undeclared editor-completion models. What remains is the
++ // set it offers for chat. Judged by id alone, 14 of 21 read as unknown,
++ // because these tuned variants are ids the cloud catalog does not carry.
+ return m?.contextWindow
- ? { id, contextWindow: m.contextWindow, ignoreCatalogReleaseDate: true }
- : { id, ignoreCatalogReleaseDate: true };
++ ? {
++ id,
++ contextWindow: m.contextWindow,
++ ignoreCatalogReleaseDate: true,
++ reported: "chat" as const,
++ }
++ : { id, ignoreCatalogReleaseDate: true, reported: "chat" as const };
+ }),
+ };
+ } catch (err: unknown) {
+ return {
+ kind: "failed",
+ failure: { kind: "unreachable", endpoint, detail: oneLine(String(err)) },
+ };
+ }
}
- /** Ollama's daemon speaks its own listing shape and carries capability data no OpenAI list has. */
- async function fetchOllamaModelsCatalog(): Promise<DiscoveredModel[]> {
- const { fetchOllamaModels } = await import("./ollama-discovery.js");
+ /**
+ * Ollama's daemon speaks its own listing shape and carries capability data no OpenAI list has.
+ *
+ * `throwOnError` is what makes "the daemon is not running" distinguishable from
+ * "the daemon is running and nothing is pulled". `fetchOllamaModels` is
+ * documented as never throwing, and every other caller still gets that; only
+ * this one opts in, because only this one has somewhere to report the
+ * difference. Both states used to be `empty-models-catalog`, so a stopped daemon
+ * read as "this provider has no models" — the complaint, restated.
+ */
+ async function fetchOllamaModelsCatalog(): Promise<FetcherResult> {
+ const { fetchOllamaModels, ollamaBaseUrl } = await import("./ollama-discovery.js");
+ const { ollamaReported } = await import("./transport/probe-discovery.js");
- const installed = await fetchOllamaModels({ enrichCapabilities: false });
- return installed.map((model) => ({
- id: model.name,
- displayName: model.name,
- supportsTools: model.supportsTools,
- // The daemon's own `capabilities`, read from `/api/tags`: `completion` means
- // chat, `embedding` means not. This row used to carry only name, display name
- // and a tools flag, so the capability data was fetched and then dropped here,
- // and 18 of 19 local builds — none of which the cloud catalog lists — read as
- // unknown. A daemon too old to report the field leaves it unset.
- reported: ollamaReported(model),
- }));
+ const endpoint = `${ollamaBaseUrl()}/api/tags`;
+ try {
+ const installed = await fetchOllamaModels({
+ enrichCapabilities: false,
+ throwOnError: true,
+ });
+ return {
+ kind: "models",
+ endpoint,
+ models: installed.map((model) => ({
+ id: model.name,
+ displayName: model.name,
+ supportsTools: model.supportsTools,
++ // The daemon's own `capabilities`, read from `/api/tags`: `completion` means
++ // chat, `embedding` means not. This row used to carry only name, display name
++ // and a tools flag, so the capability data was fetched and then dropped here,
++ // and 18 of 19 local builds — none of which the cloud catalog lists — read as
++ // unknown. A daemon too old to report the field leaves it unset.
++ reported: ollamaReported(model),
+ })),
+ };
+ } catch (err: unknown) {
+ return {
+ kind: "failed",
+ failure: { kind: "unreachable", endpoint, detail: oneLine(String(err)) },
+ };
+ }
}
registerModelDiscoveryFetcher("devin-connect", fetchDevinModelsCatalog);
diff --cc packages/cli/src/providers/ollama-discovery.ts
index b3840acd,a22ea2cf..6c8da46d
--- a/packages/cli/src/providers/ollama-discovery.ts
+++ b/packages/cli/src/providers/ollama-discovery.ts
@@@ -34,16 -36,22 +36,26 @@@ export function ollamaBaseUrl(): strin
interface FetchOllamaOptions {
/**
- * When true (default), enrich each model with capabilities via `/api/show`
- * (adds the tools indicator but one extra request per model). Pass false to
- * skip enrichment for a snappier interactive picker — embedding models are
- * still filtered out by name in that case.
+ * When true (default), fall back to `POST /api/show` for any model whose
+ * `/api/tags` row carried no `capabilities` array — one extra request per
+ * such model. Pass false to skip that and accept the silence.
+ *
+ * Ollama publishes `capabilities` inline in `/api/tags` (verified against a
+ * live daemon, 2026-09-23: every one of 20 installed models carried it), so
+ * on a current daemon this fallback never fires and the fan-out costs
+ * nothing. It stays for older daemons, which omit the field.
*/
enrichCapabilities?: boolean;
+ /**
+ * Re-throw a daemon failure instead of absorbing it into `[]`.
+ *
+ * Default `false`, so every existing caller is byte-identical. Opt in only
+ * when you can report the difference: "the daemon is not running" and "the
+ * daemon is running and nothing is pulled" are the same `[]` otherwise, and
+ * model discovery used to record both as `empty-models-catalog` — telling the user
+ * their local provider serves no models when in fact nothing answered.
+ */
+ throwOnError?: boolean;
}
/**
diff --cc packages/cli/src/providers/transport/probe-discovery.test.ts
index 3f780f71,ee183424..62b4bb53
--- a/packages/cli/src/providers/transport/probe-discovery.test.ts
+++ b/packages/cli/src/providers/transport/probe-discovery.test.ts
@@@ -6,51 -6,16 +6,115 @@@
* Run: bun test src/providers/transport/probe-discovery.test.ts
*/
-import { afterEach, beforeEach, describe, expect, mock, test } from "bun:test";
+import { afterAll, afterEach, beforeEach, describe, expect, mock, test } from "bun:test";
++import { mkdtempSync, rmSync } from "node:fs";
++import { tmpdir } from "node:os";
++import { join } from "node:path";
+import type { DiskCacheV3, SlimModelEntry } from "../all-models-cache.js";
+import * as __realAllModelsCache from "../all-models-cache.js";
+
+const __realAllModelsCacheExports = { ...__realAllModelsCache };
+
+function chatCatalogEntry(modelId: string): SlimModelEntry {
+ return { modelId, aliases: [], outputModalities: ["text"] };
+}
+
+const catalogFixture: DiskCacheV3 = {
+ version: 3,
+ lastUpdated: "2026-09-23T00:00:00.000Z",
+ catalogGenerationId: "probe-discovery-test",
+ entries: [
+ chatCatalogEntry("gpt-4o"),
+ chatCatalogEntry("gpt-4o-mini"),
+ chatCatalogEntry("claude-haiku-4"),
+ chatCatalogEntry("lite-x"),
+ chatCatalogEntry("model-a"),
+ ],
+ models: [],
+ plans: [],
+};
+
++const modalityFixtureDir = mkdtempSync(join(tmpdir(), "claudish-probe-discovery-"));
++const modalityFixturePath = join(modalityFixtureDir, "cloud-models-catalog-v3.json");
++__realAllModelsCacheExports.writeAllModelsCache(
++ {
++ version: 3,
++ lastUpdated: "2026-09-23T00:00:00.000Z",
++ catalogGenerationId: "probe-discovery-modality-test",
++ entries: [
++ {
++ modelId: "claude-opus-5",
++ aliases: [],
++ inputModalities: ["file", "image", "text"],
++ outputModalities: ["text"],
++ },
++ {
++ modelId: "gpt-realtime-2",
++ aliases: [],
++ inputModalities: ["audio", "image", "text"],
++ outputModalities: ["audio", "text"],
++ },
++ {
++ modelId: "gemini-3.5-transcribe",
++ aliases: [],
++ inputModalities: ["audio"],
++ outputModalities: ["text"],
++ },
++ {
++ modelId: "gpt-realtime-translate",
++ aliases: [],
++ inputModalities: ["audio"],
++ outputModalities: ["audio", "text"],
++ },
++ { modelId: "inkling", aliases: [], outputModalities: ["text"] },
++ {
++ modelId: "mistral-medium-2604",
++ aliases: [],
++ inputModalities: null,
++ outputModalities: ["text"],
++ },
++ {
++ modelId: "o3-mini-high",
++ aliases: [],
++ inputModalities: [],
++ outputModalities: ["text"],
++ },
++ {
++ modelId: "gpt-image-2.5-flare",
++ aliases: [],
++ inputModalities: ["text"],
++ outputModalities: ["image"],
++ },
++ ],
++ models: [],
++ plans: [],
++ },
++ modalityFixturePath
++);
++
+// OpenAI-compatible listings carry no capability field, so these tests supply
+// the catalog evidence that admission uses instead of reading the user's cache.
+mock.module("../all-models-cache.js", () => ({
+ ...__realAllModelsCacheExports,
+ readAllModelsCache: (path?: string) =>
+ path ? __realAllModelsCacheExports.readAllModelsCache(path) : catalogFixture,
+}));
+
+afterAll(() => {
+ mock.module("../all-models-cache.js", () => __realAllModelsCacheExports);
++ rmSync(modalityFixtureDir, { recursive: true, force: true });
+});
import {
+ _clearChatCapabilityIndex,
_clearProbeDiscoveryCache,
+ classifyChatCapability,
discoverViaLMStudio,
discoverViaOllama,
discoverViaOpenAIModels,
invalidateProbeDiscovery,
+ isChatCapable,
+ isReportedChatCapable,
rankProbeCandidates,
} from "./probe-discovery.js";
@@@ -86,19 -59,78 +150,26 @@@ describe("rankProbeCandidates", () =>
expect(ranked[2]).toBe("model-70b"); // doesn't match the small pattern
});
- test("filters out non-chat-capable model names", () => {
- const ranked = rankProbeCandidates([
- "gpt-4o-mini",
- "dall-e-3",
- "text-embedding-3-small",
- "whisper-1",
- "tts-1",
- "gemini-2.0-flash-exp-image-generation",
- "voxtral-mini-latest",
- "claude-haiku-4",
- ]);
- // Only chat models survive. Order: small-name first, then alphabetical.
- expect(ranked).toEqual(["claude-haiku-4", "gpt-4o-mini"]);
- });
-
- test("drops the REALTIME/VOICE family — observed leaking into the Codex dynamic models catalog", () => {
- // All five were on screen in a live provider-scoped list, offered as launchable
- // coding models. The selected row's own catalog sentence disqualified it: "a
- // distilled reasoning model for faster, lower-cost realtime VOICE interactions...
- // audio and text inputs over WebRTC, WebSocket, or SIP".
- //
- // This pins the NAME FALLBACK, so the catalog is pinned EMPTY. Without a path
- // the classifier reads the developer's real cloud models catalog cache, where
- // these ids publish output `["audio","text"]` — and published output modality
- // outranks every name rule — so the test passed on a clean CI machine and
- // failed on any machine that had fetched the v3 catalog.
- for (const id of [
- "gpt-live-1",
- "gpt-realtime-2",
- "gpt-realtime-2.1",
- "gpt-realtime-2.1-mini",
- "gpt-realtime-translate",
- ]) {
- expect({ id, chat: classifyChatCapability(id, NO_CLOUD_CATALOG) }).toEqual({
- id,
- chat: "not-chat",
- });
- }
- });
-
- test("`live` IS A BOUNDED TOKEN — a substring would eat four plausible ids", () => {
- // `-` is a non-word character, so `\blive\b` matches `gpt-live-1` and does NOT
- // match any of these. A bare `/live/` would drop all four, and each of them is a
- // shape a real dynamic models catalog produces: a benchmark-tuned fine-tune, a deployment slug,
- // a vendor codename, a probe endpoint.
- for (const id of [
- "livecodebench-qwen-32b",
- "mistral-delivery-agent",
- "olive-7b",
- "liveness-probe-1",
- ]) {
- expect({ id, chat: isChatCapable(id) }).toEqual({ id, chat: true });
- }
- });
-
- test("`realtime` and `translate` are bounded too, and keep their near-misses", () => {
- expect(isChatCapable("gemini-live-2.5-flash")).toBe(false);
- expect(isChatCapable("seed-translate-1")).toBe(false);
- // Pins that the patterns are not `/real/` and `/trans/`, which would have been
- // the obvious sloppy spelling and would have dropped both of these.
- expect(isChatCapable("unreal-coder-7b")).toBe(true);
- expect(isChatCapable("trans-coder-7b")).toBe(true);
- // And the ids the whole list exists to keep.
- expect(isChatCapable("gpt-5-codex")).toBe(true);
- expect(isChatCapable("kimi-k2.7-code-highspeed")).toBe(true);
- });
-
- test("returns empty when all candidates are non-chat", () => {
- const ranked = rankProbeCandidates(["text-embedding-3-large", "whisper-1", "tts-1-hd"]);
- expect(ranked).toEqual([]);
+ test("orders candidates without re-judging capability and drops wildcards", () => {
+ expect(rankProbeCandidates(["b-model", "a-model", "gemini/*"])).toEqual(["a-model", "b-model"]);
+ });
+
+ test("provider admission excludes an embedding model before ranking", () => {
+ const admitted = [
+ isReportedChatCapable("vector-model", "not-chat") ? "vector-model" : null,
+ isReportedChatCapable("chat-model", "chat") ? "chat-model" : null,
+ ].filter((name): name is string => name !== null);
+
+ expect(rankProbeCandidates(admitted)).toEqual(["chat-model"]);
+ });
+
++ test("isChatCapable requires positive catalog evidence", () => {
++ expect({ known: isChatCapable("gpt-4o"), unknown: isChatCapable("catalog-unknown") }).toEqual({
++ known: true,
++ unknown: false,
++ });
+ });
+
test("drops wildcard route patterns", () => {
const ranked = rankProbeCandidates(["gemini/*", "gem-mad/*", "gpt-4o-mini"]);
expect(ranked).toEqual(["gpt-4o-mini"]);
@@@ -126,13 -158,8 +197,43 @@@
});
});
++describe("catalog modality evidence", () => {
++ test("classifies Claude file/image/text input plus text output as chat", () => {
++ expect(classifyChatCapability("claude-opus-5", modalityFixturePath)).toBe("chat");
++ });
++
++ test("classifies realtime audio/image/text input plus audio/text output as chat", () => {
++ expect(classifyChatCapability("gpt-realtime-2", modalityFixturePath)).toBe("chat");
++ });
++
++ test("classifies transcribe audio-only input plus text output as not-chat", () => {
++ expect(classifyChatCapability("gemini-3.5-transcribe", modalityFixturePath)).toBe("not-chat");
++ });
++
++ test("classifies realtime translation audio-only input plus audio/text output as not-chat", () => {
++ expect(classifyChatCapability("gpt-realtime-translate", modalityFixturePath)).toBe("not-chat");
++ });
++
++ test("treats absent, null, and empty input lists as silence when text output is published", () => {
++ expect(
++ ["inkling", "mistral-medium-2604", "o3-mini-high"].map((id) =>
++ classifyChatCapability(id, modalityFixturePath)
++ )
++ ).toEqual(["chat", "chat", "chat"]);
++ });
++
++ test("classifies text input plus image-only output as not-chat", () => {
++ expect(classifyChatCapability("gpt-image-2.5-flare", modalityFixturePath)).toBe("not-chat");
++ });
++});
++
describe("discoverViaOpenAIModels", () => {
const realFetch = globalThis.fetch;
+ beforeEach(() => {
+ _clearProbeDiscoveryCache();
+ // The classifier memoizes its catalog projection across files; drop it so this fixture is read.
+ _clearChatCapabilityIndex();
+ });
afterEach(() => {
globalThis.fetch = realFetch;
_clearProbeDiscoveryCache();
diff --cc packages/cli/src/providers/transport/probe-discovery.ts
index e2126385,aa3681d8..f7b0926a
--- a/packages/cli/src/providers/transport/probe-discovery.ts
+++ b/packages/cli/src/providers/transport/probe-discovery.ts
@@@ -107,6 -151,6 +107,8 @@@ interface CatalogCapabilityIndex
textOutput: Set<string>;
/** Ids whose published output modalities EXCLUDE `"text"`: they produce something else only. */
nonTextOutput: Set<string>;
++ /** Ids whose published input modalities EXCLUDE `"text"`: nothing typed reaches them. */
++ nonTextInput: Set<string>;
/** Ids the catalog declares video GENERATORS (`videoOutput: true`). */
videoOutput: Set<string>;
/** Ids with ANY published `videoOutput`, `false` included: a statement, not a silence. */
@@@ -138,11 -174,11 +140,8 @@@ function catalogKey(name: string): stri
*
* Unknown arrives as an absent field; `null` and `[]` read as unknown too, because
* no model produces nothing — such a row joins neither set and is left to the
-- * `videoOutput` boolean and the name rules. The test is "includes text", never
++ * `videoOutput` boolean; with neither, it stays unknown. The test is "includes text", never
* "equals text": `["audio", "text"]` speaks AND writes, so it can answer a chat turn.
-- *
-- * An INPUT modality is never filed here, in either direction: a model that accepts
-- * video, audio or images is still a chat model.
*/
function indexOutputModality(
index: CatalogCapabilityIndex,
@@@ -154,6 -190,6 +153,31 @@@
for (const k of keys) target.add(k);
}
++/**
++ * File a catalog row's keys when its published INPUT modalities leave out text.
++ *
++ * A chat model takes text in and gives text out; other modalities on either side
++ * never exclude one, so `["file","image","text"]` (every Claude model) is a chat
++ * model. What this catches is the other case: `gemini-3.5-transcribe` publishes
++ * `in: ["audio"]`, `out: ["text"]` — it writes text, but nothing typed reaches it.
++ * Measured on the live catalog: 13 rows publish text output with no text input,
++ * all ASR, captioning or live-translation models.
++ *
++ * Only a PUBLISHED list denies. An absent, `null` or `[]` input list is a silence
++ * (models-index contract: null is unknown, never "not a chat model"), and it
++ * leaves the output evidence standing — `inkling`, `mistral-medium-2604` and
++ * `o3-mini-high` are the three rows it keeps.
++ */
++function indexInputModality(
++ index: CatalogCapabilityIndex,
++ keys: string[],
++ modalities: string[] | null | undefined
++): void {
++ if (!Array.isArray(modalities) || modalities.length === 0) return;
++ if (modalities.includes("text")) return;
++ for (const k of keys) index.nonTextInput.add(k);
++}
++
function catalogCapabilityIndex(cachePath?: string): CatalogCapabilityIndex {
const key = cachePath ?? "";
const hit = _catalogChatIndex.get(key);
@@@ -163,9 -199,8 +187,10 @@@
chat: new Set(),
textOutput: new Set(),
nonTextOutput: new Set(),
++ nonTextInput: new Set(),
videoOutput: new Set(),
videoOutputKnown: new Set(),
+ known: new Set(),
};
for (const entry of readAllModelsCache(cachePath)?.entries ?? []) {
const keys = [catalogKey(entry.modelId), ...(entry.aliases ?? []).map(catalogKey)];
@@@ -174,6 -208,6 +199,7 @@@
// still contributes its published output modality — and where the two disagree
// the modality list wins, being the more specific statement.
indexOutputModality(index, keys, entry.outputModalities);
++ indexInputModality(index, keys, entry.inputModalities);
// `videoOutput` is read in both directions; `videoInput` is never read as a
// denial, because a model that reads video is still a chat model.
if (entry.videoOutput !== undefined) {
@@@ -209,35 -231,20 +235,35 @@@
/**
* Classify whether a model can answer a chat turn — see {@link ChatCapability}.
*
- * The cloud models catalog's published OUTPUT MODALITY outranks every name rule,
- * in both directions: a model named like an image generator whose published output
- * is `["text"]` IS a chat model, and a model with a blameless name whose output is
- * `["image"]` is NOT. Only where the catalog publishes no output modality — local
- * servers, custom endpoints, LiteLLM deployments, catalog rows without the field —
- * do the older `videoOutput` boolean and the name rules decide.
+ * NO NAME RULES. There used to be two lists of regexes — `/\bembed/`, `/\btts\b/`,
+ * `/\bwhisper\b/`, `/\bvideo\b/`, `/\bsora\b/` and so on — consulted whenever the
+ * catalog published no output modality. They are deleted. A regex over a model id
+ * is a guess, it silently miscategorises every name that does not follow the
+ * convention it encodes, and it produced a filter nobody could reason about: a
+ * model was hidden because of how it was spelled.
*
- * Order: wildcard route, non-text output, text output, `videoOutput: true`,
- * {@link NON_CHAT_PATTERNS}, the video name guess (only while `videoOutput` is
- * unpublished), the chat-shaped capability flags, then `"unknown"`.
+ * What replaced them is evidence the providers were already publishing and
+ * claudish was not reading. Ollama returns `capabilities` per model in
+ * `GET /api/tags` (`["embedding"]` for `nomic-embed-text`), LM Studio returns a
+ * `type`. Callers that hold such a listing pass it as `reported`.
+ *
+ * `reported` outranks the catalog because it is the more specific statement: the
+ * catalog describes a canonical model, the provider describes the build it will
+ * actually serve.
+ *
- * Order: wildcard route, the provider's own statement, non-text output, text
- * output, `videoOutput: true`, the chat-shaped capability flags, then
- * `"unknown"` — which is now a refusal, not a pass.
++ * Order: wildcard route, the provider's own statement, non-text output, an input
++ * list without text, text output, `videoOutput: true`, the chat-shaped capability
++ * flags, then `"unknown"` — which is now a refusal, not a pass.
*
* @param cachePath Override the catalog cache path. Tests only.
+ * @param reported The provider's own capability for this model, when its
+ * listing published one.
*/
-export function classifyChatCapability(name: string, cachePath?: string): ChatCapability {
+export function classifyChatCapability(
+ name: string,
+ cachePath?: string,
+ reported?: ReportedCapability
+): ChatCapability {
// Wildcard entries ("gemini/*") are LiteLLM route patterns, not models.
if (name.includes("*")) return "not-chat";
@@@ -246,9 -251,15 +272,10 @@@
const index = catalogCapabilityIndex(cachePath);
const key = catalogKey(name);
- // Catalog evidence, ahead of every name rule.
if (index.nonTextOutput.has(key)) return "not-chat";
++ if (index.nonTextInput.has(key)) return "not-chat";
if (index.textOutput.has(key)) return "chat";
-
if (index.videoOutput.has(key)) return "not-chat";
- if (NON_CHAT_PATTERNS.some((re) => re.test(name))) return "not-chat";
- if (!index.videoOutputKnown.has(key) && VIDEO_OUTPUT_NAME_PATTERNS.some((re) => re.test(name))) {
- return "not-chat";
- }
if (index.chat.has(key)) return "chat";
return "unknown";
}