Claude Code. Any Model. The most powerful AI coding agent now speaks every language.
fix: a metered price of zero is missing data, not free
Jack Rudenko9bedcf7ba84b
diff --git a/ai-docs/reports/BACKEND_zero_prices_models_index-20260921.md b/ai-docs/reports/BACKEND_zero_prices_models_index-20260921.mdnew file mode 100644index 00000000..1ebaae28--- /dev/null+++ b/ai-docs/reports/BACKEND_zero_prices_models_index-20260921.md@@ -0,0 +1,62 @@+# To models-index: the rename and the Poe pick landed, and one new price defect++Ready to send. Everything below was read from the live service today, generation+`g-20260921062451697-f490edba`.++---++**Re: `pricing.type` and the Poe pick are live — thank you; one data defect to fix**++Both items we were waiting on are delivered and correct on our side. We read them live and then+through our own reader.++- **The discriminator is `type`.** The pricing object's keys are now+ `batchDiscountPct, cachedRead, cachedWrite, input, output, tiers, type`. The old spelling is+ gone entirely, which is exactly what we asked for — no alias to carry on either side. Values:+ `flat` 975, `unavailable` 467, `free` 30, `tiered` 28. claudish reads `type` and only `type`.+- **The Poe pick is published.** `probeModels` now returns `poe/gateway` →+ `gemini-3.8-flash`, where it previously reported `no_verified_probe_model`. It survives our+ route-to-provider mapping and reaches our probe path unchanged. 25 picks, 5 unavailable+ routes, and all five of those are `client_model_selection_required`+ (`antigravity`, `grok-subscription`, `vertex`, `sakana-subscription`, `devin`) — the four+ dynamic subscriptions plus Vertex, which is the agreed design rather than a gap.+- **Our reader.** A forced refresh reads 1,123 entries across 2 pages on that generation. Of+ 1,734 mapped connections, 923 now carry a price we can order by and 811 do not.++## The defect: 167 metered connections quote a price of zero++167 connections publish `type: "flat"` with `input: 0` and `output: 0`. By route:++| Route | Zero-priced connections |+|---|---|+| `together-ai/gateway` | 163 |+| `openrouter/gateway` | 3 |+| `opencode/zen` | 1 |++Examples on `together-ai/gateway`: `glm-5.3-fp8`, `glm-5.3-fp8-lora`, `minimax-h3`, `flux-3`,+`seedance-2.5`, `flash-image-3.1-lite`. Together AI charges for all of those, so zero is not+their price — it is a price nobody measured, published as if it were measured.++Why it matters more than an unpriced row: **zero is the strongest possible price.** A consumer+ordering connections by cost puts a zero first, so the effect of a missing measurement is to+PROMOTE that connection above every genuinely cheap one. An `unavailable` row merely sorts last.+This is the one direction in which a data gap costs a user money rather than a better option.++What we ask: publish `type: "unavailable"` where the rate is unknown, and reserve zero for a rate+that is genuinely zero. The contract already separates those two cases cleanly, and 30+connections use `type: "free"` correctly today.++**What claudish does until then.** We treat a `flat` or `tiered` price summing to exactly zero as+unknown, so those 167 connections sort last instead of first. The rule is asymmetric on purpose:+a zero on ONE side is kept, because free input with paid output is a real tariff. Only zero on+both sides is rejected. We would rather delete that rule than keep it — it is a guess about your+data, which is exactly what both sides agreed to avoid — so it goes as soon as the rows say+`unavailable`.++## Nothing else outstanding from our side++The two design answers we sent still stand: unknown modalities as an absent field are right for+us, and we order tiered prices by the first tier, never averaging tiers and never choosing a tier+from a request's size. Modalities are in use now — the catalog's `outputModalities` decides+whether claudish offers a model as a chat model, in both directions, and 16 models whose names+look non-chat but whose published output includes text stopped being hidden by our old name rule.diff --git a/packages/cli/src/providers/connection-price.test.ts b/packages/cli/src/providers/connection-price.test.tsnew file mode 100644index 00000000..c1801f4e--- /dev/null+++ b/packages/cli/src/providers/connection-price.test.ts@@ -0,0 +1,339 @@+import { describe, expect, test } from "bun:test";+import type { AggregatorEntry } from "../model-loader.js";+import {+ type ConnectionPrice,+ compareByConnectionPrice,+ connectionPrice,+} from "./connection-price.js";++type Pricing = AggregatorEntry["pricing"];++// Trimmed verbatim from the local live cloud models catalog generation+// g-20260920154425586-418ad3dd. That generation predates the discriminator+// rename, so these source rows intentionally retain its published `shape` field.+const liveRows = {+ deepseekFlat: {+ modelId: "deepseek-v4.1-flash",+ route: { routeId: "ollama", routeProfileId: "cloud" },+ externalModelId: "deepseek-v4.1-flash:cloud",+ pricing: { shape: "flat", input: 0.3, output: 1.2, cachedRead: 0.006 },+ },+ qwenFirstTier: {+ modelId: "qwen3.7-flash",+ route: { routeId: "qwen", routeProfileId: "dashscope-direct" },+ externalModelId: "qwen3.7-flash",+ pricing: {+ shape: "tiered",+ input: 0.03,+ output: 0.13,+ tiers: [+ {+ maxInputTokens: 32000,+ input: 0.03,+ output: 0.13,+ cachedRead: 0.006,+ cachedWrite: 0.038,+ },+ {+ maxInputTokens: 256000,+ input: 0.1,+ output: 0.4,+ cachedRead: 0.02,+ cachedWrite: 0.125,+ },+ {+ maxInputTokens: 1000000,+ input: 0.2,+ output: 0.8,+ cachedRead: 0.04,+ cachedWrite: 0.25,+ },+ ],+ cachedRead: 0.006,+ cachedWrite: 0.038,+ },+ },+ perplexityEmbedding: {+ modelId: "pplx-embed-v1-0.6b",+ route: { routeId: "openrouter", routeProfileId: "gateway" },+ externalModelId: "perplexity/pplx-embed-v1-0.6b",+ pricing: { shape: "flat", input: 0.004, output: 0 },+ },+ nexFree: {+ modelId: "nex-n2.5-mini",+ route: { routeId: "openrouter", routeProfileId: "gateway" },+ externalModelId: "nex-agi/nex-n2.5-mini:free",+ pricing: { shape: "flat", input: 0, output: 0 },+ },+ qwenUnavailable: {+ modelId: "qwen3.8-omni-flash",+ route: { routeId: "qwen", routeProfileId: "dashscope-direct" },+ externalModelId: "qwen3.8-omni-flash",+ pricing: { shape: "unavailable" },+ },+} as const;++// Trimmed verbatim from the `glm-5.3-fp8` Together AI connection in live+// generation g-20260921062451697-f490edba. Its published flat 0/0 price is the+// measured catalog defect this regression protects against.+const liveTogetherUnmeasured = {+ modelId: "glm-5.3-fp8",+ route: { routeId: "together-ai", routeProfileId: "gateway" },+ externalModelId: "zai-org/GLM-5.3-FP8",+ pricing: { type: "flat", input: 0, output: 0 },+} as const;++// The implementation notes record g-20260921062451697-f490edba as the live+// cutover to `type`, including 30 `free` connections. These inputs project that+// new discriminator onto the real identities and rates captured above.+const livePrices = {+ deepseekFlat: {+ type: "flat",+ input: liveRows.deepseekFlat.pricing.input,+ output: liveRows.deepseekFlat.pricing.output,+ cachedRead: liveRows.deepseekFlat.pricing.cachedRead,+ },+ qwenFirstTier: {+ type: "tiered",+ input: liveRows.qwenFirstTier.pricing.input,+ output: liveRows.qwenFirstTier.pricing.output,+ tiers: liveRows.qwenFirstTier.pricing.tiers.map((tier) => ({ ...tier })),+ cachedRead: liveRows.qwenFirstTier.pricing.cachedRead,+ cachedWrite: liveRows.qwenFirstTier.pricing.cachedWrite,+ },+ perplexityEmbedding: {+ type: "flat",+ input: liveRows.perplexityEmbedding.pricing.input,+ output: liveRows.perplexityEmbedding.pricing.output,+ },+ nexFree: { type: "free" },+ unavailable: { type: "unavailable" },+} satisfies Record<string, NonNullable<Pricing>>;++// Verbatim pre-cutover pricing from deepseek-v4.1-flash on+// the (ollama, cloud) route binding in g-20260920154425586-418ad3dd. It+// deliberately has usable numbers but no `type`; connectionPrice must not+// treat `shape` as an alias.+const livePriceWithoutType = {+ ...liveRows.deepseekFlat.pricing,+} as unknown as Pricing;++const unknownPrice: ConnectionPrice = { known: false, label: "unknown" };++describe("connectionPrice", () => {+ test("flat pricing sums the live input and output rates", () => {+ expect(connectionPrice(livePrices.deepseekFlat)).toEqual({+ known: true,+ perMillionTokens: 1.5,+ label: "$1.50/M",+ });+ });++ test("a live Together AI connection with an unmeasured price is unknown", () => {+ expect(connectionPrice(liveTogetherUnmeasured.pricing)).toEqual(unknownPrice);+ });++ test.each([+ ["free input and paid output", { type: "flat", input: 0, output: 5 }],+ ["paid input and free output", { type: "flat", input: 5, output: 0 }],+ ] as const)("a flat price with %s remains known", (_case, pricing) => {+ expect(connectionPrice(pricing)).toEqual({+ known: true,+ perMillionTokens: 5,+ label: "$5.00/M",+ });+ });++ test("a live free connection is a known zero price", () => {+ expect(connectionPrice(livePrices.nexFree)).toEqual({+ known: true,+ perMillionTokens: 0,+ label: "free",+ });+ });++ test("tiered pricing uses the first live qwen3.7-flash tier, never the mean", () => {+ const result = connectionPrice(livePrices.qwenFirstTier);++ expect(result).toEqual({+ known: true,+ perMillionTokens: 0.16,+ label: "$0.16/M (first tier)",+ });+ expect(result.known && result.perMillionTokens).not.toBeCloseTo((0.16 + 0.5 + 1) / 3, 10);+ expect(result.known && result.perMillionTokens).not.toBe(0.5);+ expect(result.known && result.perMillionTokens).not.toBe(1);+ });++ // Contract-only case: generation g-20260921062451697-f490edba has no tiered+ // connection whose first tier sums to zero.+ test("a tiered price with an unmeasured first-tier price is unknown", () => {+ expect(+ connectionPrice({+ type: "tiered",+ input: 0,+ output: 0,+ tiers: [{ maxInputTokens: 32000, input: 0, output: 0 }],+ })+ ).toEqual(unknownPrice);+ });++ // Contract-only fallback cases from connection-price.md. The live tiered rows+ // in g-20260921062451697-f490edba all publish a usable first tier.+ test.each([+ ["missing tiers", undefined],+ ["empty tiers", []],+ ["malformed first tier", [{ maxInputTokens: 32000, input: 0.03 }]],+ ])("tiered pricing with %s falls back to the top-level first-tier rates", (_case, tiers) => {+ expect(connectionPrice({ type: "tiered", input: 0.03, output: 0.13, tiers })).toEqual({+ known: true,+ perMillionTokens: 0.16,+ label: "$0.16/M (first tier)",+ });+ });++ test("a live sub-cent price never renders as zero", () => {+ expect(connectionPrice(livePrices.perplexityEmbedding)).toEqual({+ known: true,+ perMillionTokens: 0.004,+ label: "$0.004/M",+ });+ });++ test("unavailable pricing is unknown", () => {+ expect(connectionPrice(livePrices.unavailable)).toEqual(unknownPrice);+ });++ test("usable numbers without type remain unknown", () => {+ expect(connectionPrice(livePriceWithoutType)).toEqual(unknownPrice);+ });++ test("a live connection with no pricing object is unknown", () => {+ expect(connectionPrice(undefined)).toEqual(unknownPrice);+ });++ // Contract-only forward-compatibility case from connection-price.md; the two+ // cited live generations publish only flat, tiered, free and unavailable.+ test("an unrecognised type is unknown rather than guessed or thrown", () => {+ const futurePricing = { type: "future", input: 0.3, output: 1.2 } as unknown as Pricing;+ expect(connectionPrice(futurePricing)).toEqual(unknownPrice);+ });++ // Contract-only malformed-flat cases from connection-price.md; no live row in+ // g-20260921062451697-f490edba omits one side of a declared flat price.+ test.each([+ ["input", { type: "flat", output: 1.2 }],+ ["output", { type: "flat", input: 0.3 }],+ ["both rates", { type: "flat" }],+ ] as const)("a flat price missing %s is unknown", (_case, pricing) => {+ expect(connectionPrice(pricing)).toEqual(unknownPrice);+ });++ // Contract-only invalid-number cases from the agreed no-sentinel rule in+ // connection-price.md. The cited live generations publish no such rates.+ test.each([+ ["negative", -1],+ ["NaN", Number.NaN],+ ["Infinity", Number.POSITIVE_INFINITY],+ ])("a flat price with a %s rate is unknown", (_case, input) => {+ expect(connectionPrice({ type: "flat", input, output: 0 })).toEqual(unknownPrice);+ });++ // Contract-only malformed-tier case from connection-price.md; no cited live+ // tiered row lacks both a usable first tier and a usable top-level pair.+ test("a tiered price with no usable first-tier or top-level pair is unknown", () => {+ expect(connectionPrice({ type: "tiered", input: 0.03, tiers: [{ input: 0.03 }] })).toEqual(+ unknownPrice+ );+ });++ test("unknown results are fresh objects", () => {+ const first = connectionPrice(undefined);+ const second = connectionPrice(undefined);+ expect(first).not.toBe(second);+ expect(first).toEqual(second);+ });+});++describe("compareByConnectionPrice", () => {+ test("known prices sort ascending", () => {+ const cheap = connectionPrice(livePrices.perplexityEmbedding);+ const expensive = connectionPrice(livePrices.deepseekFlat);+ expect(compareByConnectionPrice(cheap, expensive)).toBeLessThan(0);+ expect(compareByConnectionPrice(expensive, cheap)).toBeGreaterThan(0);+ });++ test("unknown never outranks a priced connection", () => {+ const priced = connectionPrice(livePrices.qwenFirstTier);+ const unavailable = connectionPrice(livePrices.unavailable);+ expect(compareByConnectionPrice(priced, unavailable)).toBeLessThan(0);+ expect(compareByConnectionPrice(unavailable, priced)).toBeGreaterThan(0);+ });++ test("a genuinely free connection sorts before every paid connection", () => {+ const free = connectionPrice(livePrices.nexFree);+ const paid = connectionPrice(livePrices.perplexityEmbedding);+ expect(compareByConnectionPrice(free, paid)).toBeLessThan(0);+ });++ test("an unmeasured Together AI price cannot sort ahead of a genuinely free connection", () => {+ const unmeasured = connectionPrice(liveTogetherUnmeasured.pricing);+ const free = connectionPrice(livePrices.nexFree);+ expect(compareByConnectionPrice(unmeasured, free)).toBeGreaterThan(0);+ });++ test("a negative sentinel is unknown and cannot sort ahead of free", () => {+ const sentinel = connectionPrice({ type: "flat", input: -1, output: 0 });+ const free = connectionPrice(livePrices.nexFree);+ expect(sentinel).toEqual(unknownPrice);+ expect(compareByConnectionPrice(sentinel, free)).toBeGreaterThan(0);+ });++ test("equal known prices tie exactly", () => {+ const first = connectionPrice({ type: "flat", input: 0.03, output: 0.13 });+ const second = connectionPrice(livePrices.qwenFirstTier);+ expect(compareByConnectionPrice(first, second)).toBe(0);+ });++ test("two unknown prices tie exactly", () => {+ expect(compareByConnectionPrice(connectionPrice(undefined), unknownPrice)).toBe(0);+ });++ test("a realistic mixed list sorts free, then ascending, then stable unknowns", () => {+ const connections = [+ { id: "deepseek-v4.1-flash/deepseek-direct", pricing: livePrices.unavailable },+ { id: "qwen3.7-flash/qwen-direct", pricing: livePrices.qwenFirstTier },+ { id: "nex-n2.5-mini/openrouter", pricing: livePrices.nexFree },+ { id: "qwen3.8-omni-flash/qwen-direct", pricing: livePrices.unavailable },+ { id: "pplx-embed-v1-0.6b/openrouter", pricing: livePrices.perplexityEmbedding },+ ];++ connections.sort((a, b) =>+ compareByConnectionPrice(connectionPrice(a.pricing), connectionPrice(b.pricing))+ );++ expect(connections.map(({ id }) => id)).toEqual([+ "nex-n2.5-mini/openrouter",+ "pplx-embed-v1-0.6b/openrouter",+ "qwen3.7-flash/qwen-direct",+ "deepseek-v4.1-flash/deepseek-direct",+ "qwen3.8-omni-flash/qwen-direct",+ ]);+ });++ test("an all-unknown live list retains caller order", () => {+ const connections = [+ "deepseek-v4.1-flash/deepseek-direct",+ "deepseek-v4.1-flash/fireworks",+ "qwen3.8-omni-flash/qwen-direct",+ ].map((id) => ({ id, price: connectionPrice(livePrices.unavailable) }));++ connections.sort((a, b) => compareByConnectionPrice(a.price, b.price));++ expect(connections.map(({ id }) => id)).toEqual([+ "deepseek-v4.1-flash/deepseek-direct",+ "deepseek-v4.1-flash/fireworks",+ "qwen3.8-omni-flash/qwen-direct",+ ]);+ });+});diff --git a/packages/cli/src/providers/connection-price.ts b/packages/cli/src/providers/connection-price.tsindex ea8999fa..f8d78c8e 100644--- a/packages/cli/src/providers/connection-price.ts+++ b/packages/cli/src/providers/connection-price.ts@@ -21,11 +21,13 @@** **What an unknown price means.** claudish reads the discriminator* `pricing.type` and only `pricing.type` — there is no alias and no fallback- * to any other field name. Unknown therefore means one of two concrete things:+ * to any other field name. Unknown therefore means one of three concrete things:* the catalog published `type: "unavailable"` for that connection (467 of them- * on generation `g-20260921062451697-f490edba`), or the connection carries no- * pricing object at all. Unknown is a sizeable minority of any real list, so- * every caller must render and sort it, never assume it away.+ * on generation `g-20260921062451697-f490edba`), the connection carries no+ * pricing object at all, or it quotes a metered price of exactly zero, which is+ * missing data rather than a price ({@link isUnmeasuredZero}). Measured on that+ * generation, 644 of 1,734 mapped connections read as unknown, so every caller+ * must render and sort it — it is a sizeable minority, never a rare case.** Absent-means-unknown is a rule, never an inference: without the* discriminator this module does not know how to read the rest of the object,@@ -73,6 +75,28 @@ function sumRates(input: number | undefined, output: number | undefined): numberreturn inputRate + outputRate;}+/**+ * A summed rate of exactly zero under `flat` or `tiered` is MISSING DATA, not free.+ *+ * The contract has a way to say free, and 30 connections use it: `type: "free"`.+ * A metered gateway that quotes `flat` with `input: 0, output: 0` is quoting a+ * price nobody measured. Measured on generation `g-20260921062451697-f490edba`:+ * 167 connections do exactly that, and 163 of them are `together-ai/gateway`, on+ * models Together AI plainly charges for (`glm-5.3-fp8`, `minimax-h3`, `flux-3`).+ *+ * Reading those as free would put them FIRST in every price comparison, so+ * routing would prefer a gateway on a number that does not exist — the one+ * ordering error that costs a user money rather than merely a better option. An+ * unknown price sorts last instead, which is recoverable: the connection is still+ * reachable, just not preferred on a fabricated zero.+ *+ * Asymmetric on purpose. A zero on ONE side is kept: free input with paid output+ * is a real tariff several gateways publish. Only zero on BOTH sides is rejected.+ */+function isUnmeasuredZero(perMillionTokens: number): boolean {+ return perMillionTokens === 0;+}+/** Strip the trailing zeros a fixed-precision render leaves behind: `"0.0060"` → `"0.006"`. */function trimTrailingZeros(text: string): string {if (!text.includes(".") || text.includes("e")) return text;@@ -105,6 +129,7 @@ function perMillionLabel(perMillionTokens: number): string {* | `"tiered"` | `tiers[0].input + tiers[0].output` | `"$0.16/M (first tier)"` |* | `"free"` | `0` | `"free"` |* | `"unavailable"` | — | `"unknown"` |+ * | `"flat"`/`"tiered"` summing to exactly 0 | — | `"unknown"` (see {@link isUnmeasuredZero}) |* | absent / unrecognised / no pricing object | — | `"unknown"` |** No branch here is speculative: on generation `g-20260921062451697-f490edba`@@ -123,7 +148,7 @@ export function connectionPrice(pricing: AggregatorEntry["pricing"]): Connectionswitch (pricing.type) {case "flat": {const perMillionTokens = sumRates(pricing.input, pricing.output);- if (perMillionTokens === null) return unknownPrice();+ if (perMillionTokens === null || isUnmeasuredZero(perMillionTokens)) return unknownPrice();return { known: true, perMillionTokens, label: perMillionLabel(perMillionTokens) };}@@ -132,7 +157,7 @@ export function connectionPrice(pricing: AggregatorEntry["pricing"]): Connectionconst perMillionTokens =(firstTier ? sumRates(firstTier.input, firstTier.output) : null) ??sumRates(pricing.input, pricing.output);- if (perMillionTokens === null) return unknownPrice();+ if (perMillionTokens === null || isUnmeasuredZero(perMillionTokens)) return unknownPrice();return {known: true,perMillionTokens,