Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat(costs): add OpenRouter model prices (#1083)
Elyas Mehtabuddine1c1b03172a7
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex f5dd62e40..0a91a88b4 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -108,8 +108,14 @@ jobs:runs-on: ubuntu-lateststeps:- uses: actions/checkout@v4+ - uses: actions/setup-python@v5+ id: python+ with:+ python-version: "3.12"- name: Install Rust componentsrun: rustup component add rustfmt clippy+ - name: Select Python for PyO3+ run: echo "PYO3_PYTHON=${{ steps.python.outputs.python-path }}" >> "$GITHUB_ENV"- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1with:cache-bin: falsediff --git a/Cargo.lock b/Cargo.lockindex 93d7f5e3f..14ebd4442 100644--- a/Cargo.lock+++ b/Cargo.lock@@ -3648,6 +3648,7 @@ dependencies = ["serde","serde_json","switchyard-protocol",+ "tempfile","toml 0.9.12+spec-1.1.0",]diff --git a/crates/switchyard-costs/Cargo.toml b/crates/switchyard-costs/Cargo.tomlindex 7176256d8..aa3189afb 100644--- a/crates/switchyard-costs/Cargo.toml+++ b/crates/switchyard-costs/Cargo.toml@@ -17,3 +17,6 @@ switchyard-protocol.workspace = trueserde.workspace = trueserde_json.workspace = truetoml = "0.9"++[dev-dependencies]+tempfile = "3"diff --git a/crates/switchyard-costs/prices.toml b/crates/switchyard-costs/prices.tomlindex ee8ec09d1..48463dfd0 100644--- a/crates/switchyard-costs/prices.toml+++ b/crates/switchyard-costs/prices.toml@@ -1,298 +1,218 @@# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.# SPDX-License-Identifier: Apache-2.0-# Rates are USD per million text tokens. Quote IDs so dots remain part of the ID.+# Rates are USD per million text tokens. Each group lists the exact IDs that share its rates.+# Groups expand into individual model lookups when this file is loaded.# These defaults stay fixed until Switchyard is updated; an override replaces this table.# Estimates exclude service tiers, long-context premiums, tools, and time-based cache storage.# OpenAI cache-write rates cover 30-minute retention, not 12-hour retention.+# OpenRouter IDs in this file use the advertised token rates from 2026-10-21.+# https://openrouter.ai/api/v1/models+# OpenRouter rates can differ from direct-provider rates and can change with the selected provider.+# This table uses the input rate for OpenRouter cache reads without a published discount.+# This table uses the input rate for OpenRouter cache writes without a separate fee.+# OpenRouter's Google cache-write fields describe time-based storage.+# This table uses the input rate for those Google cache writes.+# This table excludes long-context, service-tier, and time-of-day adjustments.+# OpenAI's standard short-context rates were checked on 2026-10-21.# https://developers.openai.com/api/docs/pricing# GPT-5.6 and newer use the default 30-minute cache-write rate.+# codex-auto-review uses the gpt-6-sol rates for cost and baseline estimates.+# Gateway estimates use list rates where shown; a reported charge takes precedence.# Older models have no extra write fee, so separately counted writes use the input rate.-["gpt-6-astra"]+[[prices]]+models = ["gpt-6-astra", "openai/gpt-6-astra", "openai/gpt-6-astra-pro"]input = 10output = 50cache_read = 1cache_write = 12.5-["gpt-6.1-sol"]+[[prices]]+models = ["gpt-6.1-sol", "openai/gpt-6.1-sol", "openai/gpt-6.1-sol-pro"]input = 2output = 10cache_read = 0.1cache_write = 2.5-["gpt-6-sol"]+[[prices]]+models = ["gpt-6-sol", "codex-auto-review", "openai/gpt-6-sol", "openai/gpt-6-sol-pro"]input = 2output = 10cache_read = 0.2cache_write = 2.5-["gpt-6-luna"]+[[prices]]+models = ["gpt-6-luna", "openai/gpt-6-luna", "openai/gpt-6-luna-pro"]input = 0.1output = 0.5cache_read = 0.01cache_write = 0.125-["gpt-5.6-sol"]+[[prices]]+models = ["gpt-5.6-sol", "openai/gpt-5.6-sol", "openrouter/openai/gpt-5.6-sol"]input = 4output = 20cache_read = 0.4cache_write = 5-["gpt-5.6-luna"]+[[prices]]+models = ["gpt-5.6-luna", "openai/gpt-5.6-luna"]input = 0.2output = 1.2cache_read = 0.02cache_write = 0.25-["gpt-5.6-terra"]+[[prices]]+models = ["gpt-5.6-terra", "openai/gpt-5.6-terra", "openrouter/openai/gpt-5.6-terra"]input = 2output = 12cache_read = 0.2cache_write = 2.5-["gpt-5.5"]+[[prices]]+models = ["gpt-5.5", "openai/gpt-5.5", "azure/openai/gpt-5.5"]input = 5output = 30cache_read = 0.5cache_write = 5-["gpt-5.4-mini"]+[[prices]]+models = ["gpt-5.4-mini"]input = 0.75output = 4.5cache_read = 0.075cache_write = 0.75-["gpt-5.4-nano"]+[[prices]]+models = ["gpt-5.4-nano"]input = 0.2output = 1.25cache_read = 0.02cache_write = 0.2-["gpt-5.2"]+[[prices]]+models = ["gpt-5.2", "openai/gpt-5.2"]input = 1.75output = 14cache_read = 0.175cache_write = 1.75-["gpt-4.1"]+[[prices]]+models = ["gpt-4.1", "o3"]input = 2output = 8cache_read = 0.5cache_write = 2-["gpt-4.1-mini"]+[[prices]]+models = ["gpt-4.1-mini"]input = 0.4output = 1.6cache_read = 0.1cache_write = 0.4-["o3"]-input = 2-output = 8-cache_read = 0.5-cache_write = 2--["o4-mini"]+[[prices]]+models = ["o4-mini"]input = 1.1output = 4.4cache_read = 0.275cache_write = 1.1-# codex-auto-review uses the gpt-6-sol rates for cost and baseline estimates.--["codex-auto-review"]-input = 2-output = 10-cache_read = 0.2-cache_write = 2.5--# These exact IDs occur in the shipped configs. They use OpenAI list-rate estimates;-# marketplace and cloud charges may differ. A reported charge takes precedence.--["openai/gpt-5.5"]-input = 5-output = 30-cache_read = 0.5-cache_write = 5--["openai/gpt-5.2"]-input = 1.75-output = 14-cache_read = 0.175-cache_write = 1.75--["openai/gpt-5.6-sol"]-input = 4-output = 20-cache_read = 0.4-cache_write = 5--["openai/gpt-5.6-luna"]-input = 0.2-output = 1.2-cache_read = 0.02-cache_write = 0.25--["openai/gpt-5.6-terra"]-input = 2-output = 12-cache_read = 0.2-cache_write = 2.5--["azure/openai/gpt-5.5"]-input = 5-output = 30-cache_read = 0.5-cache_write = 5--["openrouter/openai/gpt-5.6-sol"]-input = 4-output = 20-cache_read = 0.4-cache_write = 5--["openrouter/openai/gpt-5.6-terra"]-input = 2-output = 12-cache_read = 0.2-cache_write = 2.5-# Anthropic's standard token rates were checked on 2026-10-21.# https://platform.claude.com/docs/en/about-claude/pricing# The cache_write field uses the five-minute rate.# The cache_write_1h field uses the one-hour rate.-["claude-sonnet-5"]+[[prices]]+models = ["claude-sonnet-5", "anthropic/claude-sonnet-5"]input = 2output = 10cache_read = 0.2cache_write = 2.5cache_write_1h = 4-["claude-sonnet-5-5"]+[[prices]]+models = ["claude-sonnet-5-5", "anthropic/claude-sonnet-5.5"]input = 2output = 10cache_read = 0.1cache_write = 2.5cache_write_1h = 4-["claude-opus-5-5"]+[[prices]]+models = ["claude-opus-5-5", "anthropic/claude-opus-5.5"]input = 4output = 20cache_read = 0.2cache_write = 5cache_write_1h = 8-["claude-opus-4-7"]-input = 5-output = 25-cache_read = 0.5-cache_write = 6.25-cache_write_1h = 10--["claude-opus-4-8"]-input = 5-output = 25-cache_read = 0.5-cache_write = 6.25-cache_write_1h = 10--["claude-haiku-4-5"]-input = 1-output = 5-cache_read = 0.1-cache_write = 1.25-cache_write_1h = 2--["claude-haiku-4-5-20251001"]-input = 1-output = 5-cache_read = 0.1-cache_write = 1.25-cache_write_1h = 2--# These exact IDs and token rates were checked against OpenRouter on 2026-10-21.-# https://openrouter.ai/api/v1/models--["anthropic/claude-sonnet-5"]-input = 2-output = 10-cache_read = 0.2-cache_write = 2.5-cache_write_1h = 4--["anthropic/claude-opus-4.7"]-input = 5-output = 25-cache_read = 0.5-cache_write = 6.25-cache_write_1h = 10--["anthropic/claude-opus-4.8"]+[[prices]]+models = [+ "claude-opus-4-7",+ "anthropic/claude-opus-4.7",+ "claude-opus-4-8",+ "anthropic/claude-opus-4.8",+ "claude-opus-5",+ "anthropic/claude-opus-5",+ "claude-opus-4-6",+ "anthropic/claude-opus-4.6",+]input = 5output = 25cache_read = 0.5cache_write = 6.25cache_write_1h = 10-# This gateway ID uses Anthropic list rates as an estimate. Gateway charges may differ.-["azure/anthropic/claude-haiku-4-5"]+[[prices]]+models = [+ "claude-haiku-4-5",+ "claude-haiku-4-5-20251001",+ "anthropic/claude-haiku-4.5",+ "azure/anthropic/claude-haiku-4-5",+]input = 1output = 5cache_read = 0.1cache_write = 1.25cache_write_1h = 2-# These direct Claude IDs use Anthropic standard rates, checked on 2026-10-21.-# https://platform.claude.com/docs/en/about-claude/pricing-# Haiku 5.5 rates apply up to 100,000 prompt tokens. Longer prompts cost five times more.--["claude-opus-5"]-input = 5-output = 25-cache_read = 0.5-cache_write = 6.25-cache_write_1h = 10--["claude-fable-5"]+[[prices]]+models = ["claude-fable-5", "anthropic/claude-fable-5"]input = 10output = 50cache_read = 1cache_write = 12.5cache_write_1h = 20-["claude-fable-5-1"]+[[prices]]+models = ["claude-fable-5-1", "anthropic/claude-fable-5.1"]input = 10output = 50cache_read = 0.25cache_write = 12.5cache_write_1h = 20-["claude-haiku-5-5"]+# Haiku 5.5 rates apply up to 100,000 prompt tokens. Longer prompts cost five times more.++[[prices]]+models = ["claude-haiku-5-5", "anthropic/claude-haiku-5.5"]input = 0.1output = 0.5cache_read = 0.01cache_write = 0.125cache_write_1h = 0.2-["claude-opus-4-6"]-input = 5-output = 25-cache_read = 0.5-cache_write = 6.25-cache_write_1h = 10--["claude-sonnet-4-6"]-input = 3-output = 15-cache_read = 0.3-cache_write = 3.75-cache_write_1h = 6--["claude-sonnet-4-5"]+[[prices]]+models = [+ "claude-sonnet-4-6",+ "anthropic/claude-sonnet-4.6",+ "claude-sonnet-4-5",+ "anthropic/claude-sonnet-4.5",+]input = 3output = 15cache_read = 0.3@@ -306,35 +226,24 @@ cache_write_1h = 6# Google bills cache storage by time, and this table excludes it.# Separately counted cache writes, including one-hour writes, use the input rate.-["gemini-4-argon"]+[[prices]]+models = ["gemini-4-argon"]input = 2output = 10cache_read = 0.1cache_write = 2cache_write_1h = 2-["gemini-3.8-flash"]-input = 0.75-output = 3.75-cache_read = 0.075-cache_write = 0.75-cache_write_1h = 0.75--["gemini-3.7-flash"]+[[prices]]+models = ["gemini-3.8-flash", "gemini-3.7-flash", "gemini-3.6-flash"]input = 0.75output = 3.75cache_read = 0.075cache_write = 0.75cache_write_1h = 0.75-["gemini-3.6-flash"]-input = 0.75-output = 3.75-cache_read = 0.075-cache_write = 0.75-cache_write_1h = 0.75--["gemini-3.5-flash-lite"]+[[prices]]+models = ["gemini-3.5-flash-lite"]input = 0.3output = 2.5cache_read = 0.03@@ -346,31 +255,372 @@ cache_write_1h = 0.3# DeepSeek bills the older Flash IDs as deepseek-flash. This table uses one rate at all times.# Separately counted cache writes, including one-hour writes, use the input rate.-["deepseek-flash"]+[[prices]]+models = ["deepseek-flash", "deepseek-v4-flash", "deepseek-v4-flash-vision-exp"]input = 0.3output = 1.2cache_read = 0.006cache_write = 0.3cache_write_1h = 0.3-["deepseek-v4-flash"]+[[prices]]+models = ["deepseek-v4-pro"]+input = 1.32+output = 3.96+cache_read = 0.044+cache_write = 1.32+cache_write_1h = 1.32++[[prices]]+models = ["openai/gpt-oss-120b"]+input = 0.15+output = 0.6+cache_read = 0.075+cache_write = 0.15++[[prices]]+models = ["openai/gpt-oss-20b"]+input = 0.018+output = 0.09+cache_read = 0.009+cache_write = 0.018++[[prices]]+models = ["google/gemini-4-argon"]+input = 2+output = 10+cache_read = 0.1+cache_write = 2++[[prices]]+models = ["google/gemini-3.8-flash"]+input = 0.825+output = 4.125+cache_read = 0.0825+cache_write = 0.825++[[prices]]+models = ["google/gemini-3.7-flash", "google/gemini-3.6-flash"]+input = 0.75+output = 3.75+cache_read = 0.075+cache_write = 0.75++[[prices]]+models = ["google/gemini-3.5-flash"]+input = 1.5+output = 9+cache_read = 0.15+cache_write = 1.5++[[prices]]+models = ["google/gemini-3.5-flash-lite", "google/gemini-2.5-flash"]input = 0.3-output = 1.2-cache_read = 0.006+output = 2.5+cache_read = 0.03cache_write = 0.3-cache_write_1h = 0.3-["deepseek-v4-flash-vision-exp"]+[[prices]]+models = ["google/gemini-3.1-flash-lite"]+input = 0.25+output = 1.5+cache_read = 0.025+cache_write = 0.25++[[prices]]+models = ["google/gemini-3.1-pro-preview"]+input = 2+output = 12+cache_read = 0.2+cache_write = 2++[[prices]]+models = ["google/gemini-2.5-pro"]+input = 1.25+output = 10+cache_read = 0.125+cache_write = 1.25++[[prices]]+models = ["google/gemini-2.5-flash-lite"]+input = 0.1+output = 0.4+cache_read = 0.01+cache_write = 0.1++[[prices]]+models = ["x-ai/grok-4.7", "x-ai/grok-4.6"]+input = 2+output = 6+cache_read = 0.5+cache_write = 2++[[prices]]+models = ["x-ai/grok-4.3"]+input = 1.25+output = 2.5+cache_read = 0.2+cache_write = 1.25++[[prices]]+models = ["moonshotai/kimi-k2.6"]+input = 0.65+output = 3.41+cache_read = 0.15+cache_write = 0.65++[[prices]]+models = ["moonshotai/kimi-k2.7-code"]+input = 0.6712+output = 3.35+cache_read = 0.18+cache_write = 0.6712++[[prices]]+models = ["z-ai/glm-5.1"]+input = 1.4+output = 4.4+cache_read = 0.26+cache_write = 1.4++[[prices]]+models = ["z-ai/glm-5.3-prime"]+input = 2.8+output = 8.8+cache_read = 0.56+cache_write = 2.8++[[prices]]+models = ["z-ai/glm-5.3-flashx"]+input = 0.37+output = 1.25+cache_read = 0.09+cache_write = 0.37++[[prices]]+models = ["qwen/qwen3.8-max-0902"]+input = 2+output = 6+cache_read = 0.25+cache_write = 2++[[prices]]+models = ["qwen/qwen3.8-flash"]+input = 0.15+output = 0.47+cache_read = 0.016+cache_write = 0.15++[[prices]]+models = ["qwen/qwen3.8-max-prime"]+input = 4+output = 12+cache_read = 0.5+cache_write = 4++[[prices]]+models = ["qwen/qwen3.7-plus"]+input = 0.32+output = 1.28+cache_read = 0.064+cache_write = 0.4++[[prices]]+models = ["qwen/qwen3.7-max"]+input = 1.475+output = 4.425+cache_read = 0.295+cache_write = 1.84375++[[prices]]+models = ["qwen/qwen3.7-flash"]+input = 0.03+output = 0.13+cache_read = 0.006+cache_write = 0.038++[[prices]]+models = ["minimax/minimax-m3", "minimax/minimax-m2.7"]input = 0.3output = 1.2-cache_read = 0.006+cache_read = 0.06cache_write = 0.3-cache_write_1h = 0.3-["deepseek-v4-pro"]-input = 1.32-output = 3.96-cache_read = 0.044-cache_write = 1.32-cache_write_1h = 1.32+[[prices]]+models = ["nvidia/nemotron-3-ultra-550b-a55b"]+input = 0.6+output = 2.4+cache_read = 0.12+cache_write = 0.6+[[prices]]+models = ["nvidia/nemotron-3.5-super-vl"]+input = 0.15+output = 0.5+cache_read = 0.02+cache_write = 0.15++[[prices]]+models = ["nvidia/nemotron-3.5-lightning"]+input = 0.07+output = 0.2+cache_read = 0.04+cache_write = 0.07++[[prices]]+models = ["nvidia/nemotron-3-super-120b-a12b"]+input = 0.3+output = 2.4+cache_read = 0.15+cache_write = 0.3++[[prices]]+models = ["mistralai/mistral-large-4-0"]+input = 1.36+output = 4.18+cache_read = 0.14+cache_write = 1.36++[[prices]]+models = ["mistralai/mistral-medium-3-5"]+input = 1.5+output = 7.5+cache_read = 1.5+cache_write = 1.5++[[prices]]+models = ["mistralai/mistral-small-2603"]+input = 0.15+output = 0.6+cache_read = 0.015+cache_write = 0.15++[[prices]]+models = ["mistralai/devstral-2512"]+input = 0.4+output = 2+cache_read = 0.04+cache_write = 0.4++[[prices]]+models = [+ "meta/muse-spark-open",+ "meta/muse-spark-1.3",+ "meta/muse-spark-1.1",+ "meta/muse-spark-1.2",+]+input = 1.25+output = 4.25+cache_read = 0.15+cache_write = 1.25++[[prices]]+models = ["meta/muse-wonder"]+input = 4+output = 20+cache_read = 0.2+cache_write = 4++[[prices]]+models = ["stepfun/step-5-preview"]+input = 1+output = 2.7+cache_read = 0.1+cache_write = 1++[[prices]]+models = ["xiaomi/mimo-v2.6-pro"]+input = 0.435+output = 0.87+cache_read = 0.0036+cache_write = 0.435++[[prices]]+models = ["xiaomi/mimo-v2.6-flash"]+input = 0.14+output = 0.28+cache_read = 0.0028+cache_write = 0.14++# Only these explicit OpenRouter :free IDs use zero rates, checked on 2026-10-21.+# The same models without :free keep their paid rates.++[[prices]]+models = [+ "minimax/minimax-m3.1-flash:free",+ "nvidia/nemotron-3-ultra-550b-a55b:free",+ "nvidia/nemotron-3.5-super-vl:free",+ "nvidia/nemotron-3.5-lightning:free",+]+input = 0+output = 0+cache_read = 0+cache_write = 0++# These OpenRouter rates include provider discounts and can change with provider routing.+# OpenRouter's endpoint API lists provider offers for these GLM, Kimi, and DeepSeek rates.+# https://openrouter.ai/api/v1/models/{author}/{model}/endpoints+# DeepSeek 1017 and Hy4 use the advertised off-peak rates. Peak rates are higher.++[[prices]]+models = ["deepseek/deepseek-v4.1-flash"]+input = 0.02394+output = 0.375+cache_read = 0.000342+cache_write = 0.02394++[[prices]]+models = ["deepseek/deepseek-v4.1-flash-1017"]+input = 0.15+output = 0.6+cache_read = 0.003+cache_write = 0.15++[[prices]]+models = ["deepseek/deepseek-v4-flash-0731"]+input = 0.04444+output = 0.13332+cache_read = 0.001414+cache_write = 0.04444++[[prices]]+models = ["deepseek/deepseek-v4-pro-0813"]+input = 0.66+output = 1.98+cache_read = 0.022+cache_write = 0.66++[[prices]]+models = ["z-ai/glm-5.3"]+input = 0.079+output = 1.4+cache_read = 0.078+cache_write = 0.079++[[prices]]+models = ["z-ai/glm-5.3-flash"]+input = 0.02282+output = 0.05385+cache_read = 0.004885+cache_write = 0.02282++[[prices]]+models = ["z-ai/glm-5.2"]+input = 0.0341+output = 7+cache_read = 0.033+cache_write = 0.0341++[[prices]]+models = ["moonshotai/kimi-k3"]+input = 0.239+output = 11.936+cache_read = 0.21+cache_write = 0.239++[[prices]]+models = ["tencent/hy4-preview"]+input = 0.7506+output = 2.2509+cache_read = 0.0378+cache_write = 0.7506diff --git a/crates/switchyard-costs/src/lib.rs b/crates/switchyard-costs/src/lib.rsindex 6c9de8be5..751d872ad 100644--- a/crates/switchyard-costs/src/lib.rs+++ b/crates/switchyard-costs/src/lib.rs@@ -53,6 +53,56 @@ struct Rates {cache_write_1h: Option<f64>,}+#[derive(Deserialize)]+#[serde(untagged, deny_unknown_fields)]+enum PricingFile {+ Grouped { prices: Vec<PriceGroup> },+ Keyed(Pricing),+}++// These explicit rate fields let Serde reject unknown and repeated fields.+#[derive(Deserialize)]+#[serde(deny_unknown_fields)]+struct PriceGroup {+ models: Vec<String>,+ input: f64,+ output: f64,+ cache_read: f64,+ cache_write: f64,+ cache_write_1h: Option<f64>,+}++impl PricingFile {+ /// This method expands groups once so cost calculations keep using exact model lookups.+ fn into_pricing(self) -> Result<Pricing, String> {+ let pricing = match self {+ Self::Keyed(pricing) => pricing,+ Self::Grouped { prices } => {+ let mut rates = HashMap::new();+ for group in prices {+ if group.models.is_empty() {+ return Err("a pricing group must contain at least one model ID".into());+ }+ let rate = Rates {+ input: group.input,+ output: group.output,+ cache_read: group.cache_read,+ cache_write: group.cache_write,+ cache_write_1h: group.cache_write_1h,+ };+ for model in group.models {+ if rates.insert(model.clone(), rate.clone()).is_some() {+ return Err(format!("duplicate pricing model {model:?}"));+ }+ }+ }+ Pricing(rates)+ }+ };+ pricing.validate()+ }+}+/// This value identifies how a saved monetary amount was obtained.#[derive(Clone, Copy, Debug, PartialEq, Deserialize, Serialize)]#[serde(rename_all = "snake_case")]@@ -101,16 +151,17 @@ impl Pricing {pricing.map_err(|error| format!("invalid pricing file {}: {error}", path.display()))}- /// This function reads a pricing file's JSON and checks every model ID and rate.+ /// This function reads JSON with model-keyed rates or a `prices` array of model groups.+ /// This function checks every model ID and rate before returning the table.pub fn parse(json: &[u8]) -> Result<Self, String> {- let pricing: Self = serde_json::from_slice(json).map_err(|error| error.to_string())?;- pricing.validate()+ let file: PricingFile = serde_json::from_slice(json).map_err(|error| error.to_string())?;+ file.into_pricing()}/// TOML rates use the same typed fields and validation as JSON rates.fn parse_toml(text: &str) -> Result<Self, String> {- let pricing: Self = toml::from_str(text).map_err(|error| error.to_string())?;- pricing.validate()+ let file: PricingFile = toml::from_str(text).map_err(|error| error.to_string())?;+ file.into_pricing()}/// Both parsers use this check so model IDs and rate limits agree across formats.diff --git a/crates/switchyard-costs/tests/pricing.rs b/crates/switchyard-costs/tests/pricing.rsnew file mode 100644index 000000000..d096348f4--- /dev/null+++ b/crates/switchyard-costs/tests/pricing.rs@@ -0,0 +1,232 @@+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+// SPDX-License-Identifier: Apache-2.0++use switchyard_costs::{CostSource, Pricing};+use switchyard_protocol::{InputCacheUsage, Usage};++type TestResult = Result<(), Box<dyn std::error::Error>>;++const GROUPED_TOML: &str = r#"+[[prices]]+models = ["direct", "provider/model"]+input = 1+output = 2+cache_read = 0.1+cache_write = 1.25+cache_write_1h = 2+"#;++#[test]+fn grouped_and_keyed_files_price_the_same_usage() -> TestResult {+ let dir = tempfile::tempdir()?;+ let usage = Usage {+ input_tokens: Some(1_000_000),+ output_tokens: Some(2_000_000),+ cache: Some(Box::new(InputCacheUsage {+ cached_input_tokens: Some(3_000_000),+ cache_creation_input_tokens: Some(5_000_000),+ cache_creation_1h_input_tokens: Some(2_000_000),+ })),+ ..Usage::default()+ };+ for (name, text, models) in [+ (+ "grouped.ToMl",+ GROUPED_TOML,+ vec!["direct", "provider/model"],+ ),+ (+ "grouped.json",+ r#"{"prices":[{"models":["direct","provider/model"],"input":1,"output":2,"cache_read":0.1,"cache_write":1.25,"cache_write_1h":2}]}"#,+ vec!["direct", "provider/model"],+ ),+ (+ "keyed.toml",+ "[\"provider/model\"]\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1.25\ncache_write_1h=2",+ vec!["provider/model"],+ ),+ (+ "keyed.json",+ r#"{"provider/model":{"input":1,"output":2,"cache_read":0.1,"cache_write":1.25,"cache_write_1h":2}}"#,+ vec!["provider/model"],+ ),+ (+ "keyed-reserved.toml",+ "[prices]\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1.25\ncache_write_1h=2",+ vec!["prices"],+ ),+ (+ "keyed-reserved.json",+ r#"{"prices":{"input":1,"output":2,"cache_read":0.1,"cache_write":1.25,"cache_write_1h":2}}"#,+ vec!["prices"],+ ),+ ("empty-grouped.toml", "prices=[]", vec![]),+ ("empty-grouped.json", r#"{"prices":[]}"#, vec![]),+ ("empty-keyed.toml", "", vec![]),+ ("empty-keyed.json", "{}", vec![]),+ ] {+ let path = dir.path().join(name);+ std::fs::write(&path, text)?;+ let prices = Pricing::load(&path)?;+ for model in models {+ let cost = prices.estimate(model, &usage).ok_or("missing estimate")?;+ assert!((cost.usd - 13.05).abs() < 1e-9, "{name}: {model}");+ assert_eq!(cost.source, CostSource::Estimated);+ }+ assert!(!prices.contains("provider/typo"));+ assert_eq!(prices.estimate("provider/typo", &usage), None);+ }+ Ok(())+}++// These fixtures keep repeated fields in JSON text so decoding cannot erase them first.+#[test]+fn rejects_invalid_groups_and_preserved_raw_json_fields() {+ for text in [+ r#"{"prices":[{"models":[],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,+ r#"{"prices":[{"models":["a","a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1},{"models":["a"],"input":3,"output":4,"cache_read":0.2,"cache_write":3}]}"#,+ r#"{"prices":[{"models":[""],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,+ r#"{"prices":[{"models":[" a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1}]}"#,+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1,"cache_wirte":1}]}"#,+ r#"{"prices":[{"models":["a"],"models":["b"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,+ r#"{"prices":[{"models":["a"],"input":1,"input":3,"output":2,"cache_read":0.1,"cache_write":1}]}"#,+ r#"{"prices":[],"prices":[]}"#,+ r#"{"prices":[],"unexpected":true}"#,+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}],"b":{"input":1,"output":2,"cache_read":0.1,"cache_write":1}}"#,+ r#"{"a":{"input":1,"output":2,"cache_read":0.1,"cache_write":1},"a":{"input":3,"output":4,"cache_read":0.2,"cache_write":3}}"#,+ r#"{"a":{"input":1,"input":3,"output":2,"cache_read":0.1,"cache_write":1}}"#,+ r#"{"a":{"input":1,"output":2,"cache_read":0.1,"cache_write":1,"cache_wirte":1}}"#,+ ] {+ assert!(Pricing::parse(text.as_bytes()).is_err(), "accepted {text}");+ }+}++#[test]+fn rejects_invalid_toml_before_returning_a_price_table() -> TestResult {+ let dir = tempfile::tempdir()?;+ let path = dir.path().join("prices.toml");+ for text in [+ "[[prices]]\nmodels=[]\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",+ "[[prices]]\nmodels=['a','a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",+ "[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1\n[[prices]]\nmodels=['a']\ninput=3\noutput=4\ncache_read=0.2\ncache_write=3",+ "[[prices]]\nmodels=['a ']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",+ "[[prices]]\nmodels=['a']\nmodels=['b']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",+ "[[prices]]\nmodels=['a']\ninput=1\ninput=3\noutput=2\ncache_read=0.1\ncache_write=1",+ "unexpected=true\n[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",+ "['b']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1\n[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",+ "[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1\ncache_wirte=1",+ ] {+ std::fs::write(&path, text)?;+ let error = Pricing::load(&path).err().ok_or("invalid pricing loaded")?;+ assert!(error.starts_with("invalid pricing file "), "{error}");+ assert!(error.contains(&path.display().to_string()), "{error}");+ }+ Ok(())+}++#[test]+fn every_grouped_rate_must_be_finite_and_in_range() -> TestResult {+ let dir = tempfile::tempdir()?;+ let path = dir.path().join("prices.toml");+ for field in [+ "input",+ "output",+ "cache_read",+ "cache_write",+ "cache_write_1h",+ ] {+ for value in ["-1", "10001", "nan", "inf"] {+ let mut text = String::from("[[prices]]\nmodels=['a']\n");+ for name in [+ "input",+ "output",+ "cache_read",+ "cache_write",+ "cache_write_1h",+ ] {+ text.push_str(&format!(+ "{name}={}\n",+ if name == field { value } else { "1" }+ ));+ }+ std::fs::write(&path, &text)?;+ assert!(Pricing::load(&path).is_err(), "accepted {field}={value}");+ }+ }+ Ok(())+}++#[test]+fn bundled_aliases_keep_all_cache_rates() -> TestResult {+ let prices = Pricing::bundled()?;+ let usage = Usage {+ input_tokens: Some(1_000_000),+ output_tokens: Some(1_000_000),+ cache: Some(Box::new(InputCacheUsage {+ cached_input_tokens: Some(1_000_000),+ cache_creation_input_tokens: Some(2_000_000),+ cache_creation_1h_input_tokens: Some(1_000_000),+ })),+ ..Usage::default()+ };+ for model in [+ "claude-haiku-4-5",+ "claude-haiku-4-5-20251001",+ "anthropic/claude-haiku-4.5",+ "azure/anthropic/claude-haiku-4-5",+ ] {+ let cost = prices.estimate(model, &usage).ok_or("missing alias")?;+ assert!((cost.usd - 9.35).abs() < 1e-9, "{model}");+ }+ Ok(())+}++#[test]+fn bundled_provider_prices_and_optional_rates_stay_distinct() -> TestResult {+ let prices = Pricing::bundled()?;+ let mut usage = Usage {+ input_tokens: Some(1_000_000),+ output_tokens: Some(0),+ ..Usage::default()+ };+ for (model, expected) in [+ ("gemini-3.8-flash", 0.75),+ ("google/gemini-3.8-flash", 0.825),+ ] {+ assert_eq!(+ prices.estimate(model, &usage).ok_or("missing model")?.usd,+ expected+ );+ }+ // A missing one-hour rate must keep the estimate unknown, even if other rates match.+ usage.cache = Some(Box::new(InputCacheUsage {+ cached_input_tokens: Some(0),+ cache_creation_input_tokens: Some(1_000_000),+ cache_creation_1h_input_tokens: Some(1_000_000),+ }));+ assert_eq!(+ prices+ .estimate("gemini-3.7-flash", &usage)+ .ok_or("missing hourly rate")?+ .usd,+ 1.5+ );+ assert_eq!(prices.estimate("google/gemini-3.7-flash", &usage), None);+ usage.cache = None;+ for (model, expected) in [+ ("nvidia/nemotron-3-ultra-550b-a55b", 0.6),+ ("nvidia/nemotron-3-ultra-550b-a55b:free", 0.0),+ ] {+ assert_eq!(+ prices+ .estimate(model, &usage)+ .ok_or("missing free or paid rate")?+ .usd,+ expected+ );+ }+ assert!(!prices.contains("google/gemini-3.8-flash:free"));+ Ok(())+}diff --git a/crates/switchyard-menubar/src/pricing.rs b/crates/switchyard-menubar/src/pricing.rsindex e2cf1c98d..5cff7ac49 100644--- a/crates/switchyard-menubar/src/pricing.rs+++ b/crates/switchyard-menubar/src/pricing.rs@@ -105,6 +105,31 @@ mod tests {assert!((savings.percent().expect("percent") - 80.0).abs() < 1e-9);}+ #[test]+ fn settings_load_bundled_groups_for_baseline_estimates() -> Result<(), String> {+ let dir = tempfile::tempdir().map_err(|error| error.to_string())?;+ let config = crate::config::Config::load(&dir.path().join("menubar.toml"))?;+ let mut totals = Totals::default();+ totals.routed.insert(+ "answer-model".to_string(),+ ModelTokens {+ requests: 1,+ input: 1_000_000,+ output: 1_000_000,+ cached_input: 1_000_000,+ cache_write: 2_000_000,+ cache_write_1h: 1_000_000,+ },+ );+ record_cost(&mut totals, 0.5);+ for model in ["claude-haiku-4-5", "anthropic/claude-haiku-4.5"] {+ let savings = estimate(&totals, &config.prices, model).ok_or("missing baseline")?;+ assert!((savings.baseline - 9.35).abs() < 1e-9, "{model}");+ assert!((savings.saved() - 8.85).abs() < 1e-9, "{model}");+ }+ Ok(())+ }+#[test]fn classifier_overhead_counts_against_savings() {let mut totals = Totals::default();diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.mdindex 6b2d795cc..ce468ef08 100644--- a/crates/switchyard-server/README.md+++ b/crates/switchyard-server/README.md@@ -148,7 +148,9 @@ cache_write = 2```The `.toml` extension check ignores case. Other filenames are read as JSON, so existing-JSON overrides still work.+JSON overrides still work. TOML and JSON also accept groups of exact IDs that share+one set of rates, as shown in the [pi guide](../../docs/integrations/pi.md#estimate-the-cost).+The bundled table uses this grouped format and can be supplied to `--pricing-file`.Session stats include cost totals. Missing usage or rates leave cost unknown; `known_usd`keeps the priced subtotal and `total_usd` is null when any cost is unknown. Rates load oncediff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rsindex bb0dbf32e..c9d668766 100644--- a/crates/switchyard-server/tests/server.rs+++ b/crates/switchyard-server/tests/server.rs@@ -4550,8 +4550,11 @@ async fn routing_log_prefers_canonical_and_preserves_legacy_fallback() -> TestRelet upstream = MockUpstream::start().await?;let temp_dir = tempfile::tempdir()?;let log_path = temp_dir.path().join("routing.jsonl");- let state = random_state(&upstream.base_url, &[(ROUTE_MODEL, &["model/a"])])?- .with_routing_log(&log_path)?;+ let state = random_state(+ &upstream.base_url,+ &[(ROUTE_MODEL, &["anthropic/claude-haiku-4.5"])],+ )?+ .with_routing_log(&log_path)?;let app = build_switchyard_router(state);let request = HttpRequest::builder()@@ -4581,7 +4584,10 @@ async fn routing_log_prefers_canonical_and_preserves_legacy_fallback() -> TestReassert_eq!(stats["total_calls"], 1);assert_eq!(stats["total_prompt_tokens"], 10);assert_eq!(stats["total_cached_tokens"], 7);- assert_eq!(stats["models"]["model/a"]["completion_tokens"], 2);+ assert_eq!(+ stats["models"]["anthropic/claude-haiku-4.5"]["completion_tokens"],+ 2+ );let legacy = send(&app,@@ -4619,6 +4625,9 @@ async fn routing_log_prefers_canonical_and_preserves_legacy_fallback() -> TestRelet first: Value =serde_json::from_str(records.lines().next().ok_or("routing log was empty")?)?;assert_eq!(first["session_id"], "canonical-session");+ assert_eq!(first["model"], "anthropic/claude-haiku-4.5");+ assert_eq!(first["cost"]["source"], "estimated");+ assert!((first["cost"]["usd"].as_f64().ok_or("missing cost")? - 0.0000137).abs() < 1e-12);assert_eq!(first["turn_id"], "canonical-turn");assert_eq!(first["origin"], r#"custom-agent/"quoted"\path"#);let second: Value = serde_json::from_str(records.lines().nth(1).ok_or("missing record")?)?;@@ -4713,8 +4722,11 @@ async fn routing_log_keeps_the_canonical_session_id_until_a_stream_drains() -> Tlet upstream = MockUpstream::start().await?;let temp_dir = tempfile::tempdir()?;let log_path = temp_dir.path().join("routing.jsonl");- let state = random_state(&upstream.base_url, &[(ROUTE_MODEL, &["model/a"])])?- .with_routing_log(&log_path)?;+ let state = random_state(+ &upstream.base_url,+ &[(ROUTE_MODEL, &["anthropic/claude-haiku-4.5"])],+ )?+ .with_routing_log(&log_path)?;let app = build_switchyard_router(state);// `send_with_headers` collects the response body, so the stream wrapper reaches@@ -4755,6 +4767,9 @@ async fn routing_log_keeps_the_canonical_session_id_until_a_stream_drains() -> Tlet record: Value = serde_json::from_str(&std::fs::read_to_string(log_path)?)?;assert_eq!(record["route_id"], ROUTE_MODEL);+ assert_eq!(record["model"], "anthropic/claude-haiku-4.5");+ assert_eq!(record["cost"]["source"], "estimated");+ assert!((record["cost"]["usd"].as_f64().ok_or("missing cost")? - 0.0000312).abs() < 1e-12);assert_eq!(record["turn_id"], "streaming-turn");assert_eq!(record["algorithm"], "random");assert_eq!(record["origin"], "codex-cli");diff --git a/docs/integrations/pi.md b/docs/integrations/pi.mdindex a456358d1..ba9700c59 100644--- a/docs/integrations/pi.md+++ b/docs/integrations/pi.md@@ -187,9 +187,30 @@ Put each quoted model ID in a top-level table, as shown above. Quotes keep dotswithin the ID. The `.toml` extension check ignores case; other filenames are readas JSON, so existing JSON overrides still work.-For your own models, copy the entry for each model and replace the key with its exact-`model` value from the routing log. Use the served model ID rather than the routing alias,-and replace the example's numbers with your provider's rates. List recorded model IDs with:+To maintain one set of rates for several exact IDs, list them in a group:++```toml+[[prices]]+models = ["claude-haiku-4-5", "anthropic/claude-haiku-4.5"]+input = 1+output = 5+cache_read = 0.1+cache_write = 1.25+cache_write_1h = 2+```++Use one `[[prices]]` record per group, with at least one ID in `models`. JSON accepts+the same records in a top-level `prices` array. Each file uses either grouped records+or model-keyed tables. The bundled table uses groups and can also be loaded with+`--pricing-file`.++All IDs in a group share every rate, including `cache_write_1h` when present. Keep+separate groups when provider prices differ. The loader rejects duplicate IDs within+or across groups, invalid IDs or rates, and unknown fields before using the table.++For your own models, use each exact `model` value from the routing log as a table key+or an ID in a group. Use the served model ID rather than the routing alias, and replace+the example's numbers with your provider's rates. List recorded model IDs with:```bashjq -r '.model' PATH | sort -u@@ -199,8 +220,9 @@ Every entry requires `input`, `output`, `cache_read`, and `cache_write`. Set a ronly when that usage is free. Each rate must be a finite number from 0 to 10000 USD permillion tokens.-The example's `cache_write` price is for a 5-minute cache. A 1-hour cache costs 2 times-the input price instead of 1.25 times; set `cache_write_1h` to `4.00` for that example.+The Claude Sonnet 5 example uses the 5-minute `cache_write` price. A 1-hour cache+costs 2 times the input price instead of 1.25 times; set `cache_write_1h` to `4.00`+for that example.Include that rate to estimate reported one-hour cache writes; without it, the estimateremains unknown.