πͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
chore(pricing): add Claude Haiku 5.5, remove retired Gemini pickers (#18424)
langfuse-bot734cc86329ae
diff --git a/.agents/skills/add-model-price/references/provider-sources-and-price-keys.md b/.agents/skills/add-model-price/references/provider-sources-and-price-keys.mdindex 7b8050020b..a8f5ed8275 100644--- a/.agents/skills/add-model-price/references/provider-sources-and-price-keys.md+++ b/.agents/skills/add-model-price/references/provider-sources-and-price-keys.md@@ -891,6 +891,76 @@ file and `openAIModels`in July 27 2026 audit. Official sources:`https://developethe AWS Bedrock Public Extended Access SKU, and the legacy Claude3.x/Gemini 1.x catalog tail were not re-checked this run β no new evidence,standing exclusions.+- **Claude Haiku 5.5 (added October 7 2026)** β Anthropic released+ `claude-haiku-5-5`, confirmed via+ `https://platform.claude.com/docs/en/about-claude/pricing` and+ `https://platform.claude.com/docs/en/models/overview`. API ID / alias /+ Bedrock ID / Google Cloud ID / Microsoft Foundry ID / Claude Platform on AWS+ ID are all the dateless `claude-haiku-5-5` / `anthropic.claude-haiku-5-5`+ pattern. **Unlike every other Claude 4.6-or-later model, Haiku 5.5 is NOT on+ the flat 1M-context list** β the pricing page states explicitly: "Claude 4.6+ and later models (except Claude Haiku 5.5) ... include the full 1M token+ context window at standard pricing" and "Claude Haiku 5.5 is priced by+ prompt length: a prompt of over 100,000 tokens pays higher prices." It still+ has a 1M-token context window (per the models-overview comparison table),+ but the *price* steps up once the prompt exceeds 100,000 tokens β a materially+ lower threshold than the 200K/272K thresholds used elsewhere in the catalog.+ Standard tier (prompt <= 100,000 tokens): $0.10/MTok input, $0.125/MTok 5m+ cache write, $0.20/MTok 1h cache write, $0.01/MTok cache read (standard+ 0.1x), $0.50/MTok output. Large Context tier (prompt > 100,000 tokens): all+ five rates exactly 5x the Standard tier ($0.50/$0.625/$1.00/$0.05/$2.50).+ Modeled as a `usageDetailPattern: "(input|prompt|cached)"`, `gt`, `100000`+ tier, mirroring the Gemini large-context tier mechanism rather than the+ service_tier/speed `model_parameters` mechanism. **No Fast mode**: the+ Fast-mode table lists only Opus 5.5/5/4.8. Batch is $0.05/$0.25 (<=100K) and+ $0.25/$1.25 (>100K) MTok input/output (50% of standard) but, per existing+ precedent, no Batch tier was added (no Anthropic model in this file has+ one). Added to the pricing file and to `anthropicModels` in `types.ts`+ immediately after `claude-haiku-4-5-20251001`'s sibling block (after+ `claude-opus-5-5`, not as the first entry). matchPattern:+ `(?i)^((anthropic\/)?claude-haiku-5-5|(eu\.|us\.|apac\.|au\.|jp\.|global\.)?anthropic\.claude-haiku-5-5(-v1(:0)?)?)$`+ β verified via the bundled match-pattern tester that it does not collide+ with `claude-haiku-4-5-20251001` or the Sonnet/Opus 5.5 siblings (all fully+ anchored with `^...$`).+- **October 7 2026 audit: Gemini selectable-model lifecycle cleanup; no price+ drift found** β Re-fetched the full Anthropic pricing page plus the+ models-overview comparison table (found Claude Haiku 5.5, see above; every+ other Anthropic price β including `claude-opus-4-1-20250805` and+ `claude-opus-4-20250514`, both still shown on the main pricing table despite+ being "retired, except on Bedrock/Google Cloud" β matched verbatim), the+ OpenAI aggregate Standard/Long-Context/Fast-mode/Flex pricing tables (every+ price already in the file, from `gpt-6-astra` down through `babbage-002`,+ matched verbatim; no new flagship model found), the Gemini 3.x pricing page+ (`gemini-3.8/3.7/3.6-flash`, `gemini-3.5-flash`, `gemini-3.5-flash-lite`,+ `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`+ all matched verbatim), and the Gemini models catalog plus deprecations page.+ The catalog and deprecations page together **confirm three already-suspected+ selectable-model removals and surface one more**: `gemini-3.1-flash-lite-preview`+ and `gemini-3-pro-preview` are explicitly labeled "(Shut down)" on+ `ai.google.dev/gemini-api/docs/models`; `gemini-2.0-flash` is labeled "(Shut+ down)" there and its deprecations-page shutdown date (June 1, 2026) has+ passed; `gemini-2.5-flash-lite-preview-09-2025` is not on the models page at+ all and its deprecations-page shutdown date (March 31, 2026) has also+ passed. All four were removed from `googleAIStudioModels` (already absent+ from `vertexAIModels` since the October manual cleanup referenced above).+ Their pricing entries stay in the file for historical cost lookups.+ `gemini-2.0-flash-thinking-exp-01-21`, `gemini-1.5-pro`, `gemini-1.5-flash`,+ and `gemini-1.5-flash-8b` are still not mentioned on either the models+ catalog or the deprecations page at all (neither "available" nor "shut+ down") β per the removal criteria this is still ambiguous, not a confirmed+ shutdown, so they were left in `googleAIStudioModels` unchanged; this+ remains an open, standing gap, not a new finding. A targeted re-fetch of+ `gemini-3.5-flash-lite`'s cache-read price on `ai.google.dev/gemini-api/docs/pricing`+ returned a bare "Not available" without the page's usual explicit Free/Paid+ column separation β the same column-collapse failure mode documented in the+ July/August 2026 entries above. Per that standing lesson, this single+ ambiguous result was not treated as evidence of a change; the file's+ existing $0.03/MTok cache-read price (confirmed via an explicit+ Free/Paid-separated fetch in a prior audit) was left unchanged.+ `gpt-5.3-codex`, `gpt-5-chat-latest`, and the full Gemini 2.5-family pricing+ page (`ai.google.dev/pricing`) were not independently re-fetched this run;+ AWS Bedrock and TypeSafe Jev were not re-checked either β no drift signal+ for any of them, prices carry forward from their last confirmed audit.Capture:diff --git a/packages/shared/src/server/llm/types.ts b/packages/shared/src/server/llm/types.tsindex ebe57d9e48..ff1c6d18ad 100644--- a/packages/shared/src/server/llm/types.ts+++ b/packages/shared/src/server/llm/types.ts@@ -424,6 +424,7 @@ export const anthropicModels = ["claude-mythos-5-1","claude-opus-5-5","claude-opus-5",+ "claude-haiku-5-5","claude-haiku-4-5-20251001","claude-opus-4-8","claude-opus-4-7",@@ -474,12 +475,8 @@ export const googleAIStudioModels = ["gemini-3.5-flash-lite","gemini-3.1-pro-preview","gemini-3.1-flash-lite",- "gemini-3.1-flash-lite-preview",- "gemini-3-pro-preview","gemini-3-flash-preview","gemini-2.5-flash-lite",- "gemini-2.5-flash-lite-preview-09-2025",- "gemini-2.0-flash","gemini-2.0-flash-thinking-exp-01-21","gemini-1.5-pro","gemini-1.5-flash",diff --git a/worker/src/constants/default-model-prices.json b/worker/src/constants/default-model-prices.jsonindex 500431b846..cecb2bbc5d 100644--- a/worker/src/constants/default-model-prices.json+++ b/worker/src/constants/default-model-prices.json@@ -4827,6 +4827,64 @@}]},+ {+ "id": "dbd69017-f8ff-4ac2-8cc3-044381a0e744",+ "modelName": "claude-haiku-5-5",+ "matchPattern": "(?i)^((anthropic\\/)?claude-haiku-5-5|(eu\\.|us\\.|apac\\.|au\\.|jp\\.|global\\.)?anthropic\\.claude-haiku-5-5(-v1(:0)?)?)$",+ "createdAt": "2026-10-07T00:00:00.000Z",+ "updatedAt": "2026-10-07T00:00:00.000Z",+ "tokenizerConfig": null,+ "tokenizerId": "claude",+ "pricingTiers": [+ {+ "id": "dbd69017-f8ff-4ac2-8cc3-044381a0e744_tier_default",+ "name": "Standard",+ "isDefault": true,+ "priority": 0,+ "conditions": [],+ "prices": {+ "input": 0.1e-6,+ "input_tokens": 0.1e-6,+ "output": 0.5e-6,+ "output_tokens": 0.5e-6,+ "cache_creation_input_tokens": 0.125e-6,+ "input_cache_creation": 0.125e-6,+ "input_cache_creation_5m": 0.125e-6,+ "input_cache_creation_1h": 0.2e-6,+ "cache_read_input_tokens": 0.01e-6,+ "input_cached_tokens": 0.01e-6,+ "input_cache_read": 0.01e-6+ }+ },+ {+ "id": "a55aed67-7bf3-43cd-b56d-e82975db3bc4",+ "name": "Large Context (>100K)",+ "isDefault": false,+ "priority": 1,+ "conditions": [+ {+ "usageDetailPattern": "(input|prompt|cached)",+ "operator": "gt",+ "value": 100000,+ "caseSensitive": false+ }+ ],+ "prices": {+ "input": 0.5e-6,+ "input_tokens": 0.5e-6,+ "output": 2.5e-6,+ "output_tokens": 2.5e-6,+ "cache_creation_input_tokens": 0.625e-6,+ "input_cache_creation": 0.625e-6,+ "input_cache_creation_5m": 0.625e-6,+ "input_cache_creation_1h": 1e-6,+ "cache_read_input_tokens": 0.05e-6,+ "input_cached_tokens": 0.05e-6,+ "input_cache_read": 0.05e-6+ }+ }+ ]+ },{"id": "cmig1hb7i000104l72qrzgc6h","modelName": "gemini-2.5-pro",