Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

feat(costs): add OpenRouter model prices (#1083)

Elyas Mehtabuddine1c1b03172a7
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index f5dd62e40..0a91a88b4 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -108,8 +108,14 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ id: python
+ with:
+ python-version: "3.12"
- name: Install Rust components
run: rustup component add rustfmt clippy
+ - name: Select Python for PyO3
+ run: echo "PYO3_PYTHON=${{ steps.python.outputs.python-path }}" >> "$GITHUB_ENV"
- uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
with:
cache-bin: false
diff --git a/Cargo.lock b/Cargo.lock
index 93d7f5e3f..14ebd4442 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -3648,6 +3648,7 @@ dependencies = [
"serde",
"serde_json",
"switchyard-protocol",
+ "tempfile",
"toml 0.9.12+spec-1.1.0",
]
diff --git a/crates/switchyard-costs/Cargo.toml b/crates/switchyard-costs/Cargo.toml
index 7176256d8..aa3189afb 100644
--- a/crates/switchyard-costs/Cargo.toml
+++ b/crates/switchyard-costs/Cargo.toml
@@ -17,3 +17,6 @@ switchyard-protocol.workspace = true
serde.workspace = true
serde_json.workspace = true
toml = "0.9"
+
+[dev-dependencies]
+tempfile = "3"
diff --git a/crates/switchyard-costs/prices.toml b/crates/switchyard-costs/prices.toml
index ee8ec09d1..48463dfd0 100644
--- a/crates/switchyard-costs/prices.toml
+++ b/crates/switchyard-costs/prices.toml
@@ -1,298 +1,218 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
-# Rates are USD per million text tokens. Quote IDs so dots remain part of the ID.
+# Rates are USD per million text tokens. Each group lists the exact IDs that share its rates.
+# Groups expand into individual model lookups when this file is loaded.
# These defaults stay fixed until Switchyard is updated; an override replaces this table.
# Estimates exclude service tiers, long-context premiums, tools, and time-based cache storage.
# OpenAI cache-write rates cover 30-minute retention, not 12-hour retention.
+# OpenRouter IDs in this file use the advertised token rates from 2026-10-21.
+# https://openrouter.ai/api/v1/models
+# OpenRouter rates can differ from direct-provider rates and can change with the selected provider.
+# This table uses the input rate for OpenRouter cache reads without a published discount.
+# This table uses the input rate for OpenRouter cache writes without a separate fee.
+# OpenRouter's Google cache-write fields describe time-based storage.
+# This table uses the input rate for those Google cache writes.
+# This table excludes long-context, service-tier, and time-of-day adjustments.
+
# OpenAI's standard short-context rates were checked on 2026-10-21.
# https://developers.openai.com/api/docs/pricing
# GPT-5.6 and newer use the default 30-minute cache-write rate.
+# codex-auto-review uses the gpt-6-sol rates for cost and baseline estimates.
+# Gateway estimates use list rates where shown; a reported charge takes precedence.
# Older models have no extra write fee, so separately counted writes use the input rate.
-["gpt-6-astra"]
+[[prices]]
+models = ["gpt-6-astra", "openai/gpt-6-astra", "openai/gpt-6-astra-pro"]
input = 10
output = 50
cache_read = 1
cache_write = 12.5
-["gpt-6.1-sol"]
+[[prices]]
+models = ["gpt-6.1-sol", "openai/gpt-6.1-sol", "openai/gpt-6.1-sol-pro"]
input = 2
output = 10
cache_read = 0.1
cache_write = 2.5
-["gpt-6-sol"]
+[[prices]]
+models = ["gpt-6-sol", "codex-auto-review", "openai/gpt-6-sol", "openai/gpt-6-sol-pro"]
input = 2
output = 10
cache_read = 0.2
cache_write = 2.5
-["gpt-6-luna"]
+[[prices]]
+models = ["gpt-6-luna", "openai/gpt-6-luna", "openai/gpt-6-luna-pro"]
input = 0.1
output = 0.5
cache_read = 0.01
cache_write = 0.125
-["gpt-5.6-sol"]
+[[prices]]
+models = ["gpt-5.6-sol", "openai/gpt-5.6-sol", "openrouter/openai/gpt-5.6-sol"]
input = 4
output = 20
cache_read = 0.4
cache_write = 5
-["gpt-5.6-luna"]
+[[prices]]
+models = ["gpt-5.6-luna", "openai/gpt-5.6-luna"]
input = 0.2
output = 1.2
cache_read = 0.02
cache_write = 0.25
-["gpt-5.6-terra"]
+[[prices]]
+models = ["gpt-5.6-terra", "openai/gpt-5.6-terra", "openrouter/openai/gpt-5.6-terra"]
input = 2
output = 12
cache_read = 0.2
cache_write = 2.5
-["gpt-5.5"]
+[[prices]]
+models = ["gpt-5.5", "openai/gpt-5.5", "azure/openai/gpt-5.5"]
input = 5
output = 30
cache_read = 0.5
cache_write = 5
-["gpt-5.4-mini"]
+[[prices]]
+models = ["gpt-5.4-mini"]
input = 0.75
output = 4.5
cache_read = 0.075
cache_write = 0.75
-["gpt-5.4-nano"]
+[[prices]]
+models = ["gpt-5.4-nano"]
input = 0.2
output = 1.25
cache_read = 0.02
cache_write = 0.2
-["gpt-5.2"]
+[[prices]]
+models = ["gpt-5.2", "openai/gpt-5.2"]
input = 1.75
output = 14
cache_read = 0.175
cache_write = 1.75
-["gpt-4.1"]
+[[prices]]
+models = ["gpt-4.1", "o3"]
input = 2
output = 8
cache_read = 0.5
cache_write = 2
-["gpt-4.1-mini"]
+[[prices]]
+models = ["gpt-4.1-mini"]
input = 0.4
output = 1.6
cache_read = 0.1
cache_write = 0.4
-["o3"]
-input = 2
-output = 8
-cache_read = 0.5
-cache_write = 2
-
-["o4-mini"]
+[[prices]]
+models = ["o4-mini"]
input = 1.1
output = 4.4
cache_read = 0.275
cache_write = 1.1
-# codex-auto-review uses the gpt-6-sol rates for cost and baseline estimates.
-
-["codex-auto-review"]
-input = 2
-output = 10
-cache_read = 0.2
-cache_write = 2.5
-
-# These exact IDs occur in the shipped configs. They use OpenAI list-rate estimates;
-# marketplace and cloud charges may differ. A reported charge takes precedence.
-
-["openai/gpt-5.5"]
-input = 5
-output = 30
-cache_read = 0.5
-cache_write = 5
-
-["openai/gpt-5.2"]
-input = 1.75
-output = 14
-cache_read = 0.175
-cache_write = 1.75
-
-["openai/gpt-5.6-sol"]
-input = 4
-output = 20
-cache_read = 0.4
-cache_write = 5
-
-["openai/gpt-5.6-luna"]
-input = 0.2
-output = 1.2
-cache_read = 0.02
-cache_write = 0.25
-
-["openai/gpt-5.6-terra"]
-input = 2
-output = 12
-cache_read = 0.2
-cache_write = 2.5
-
-["azure/openai/gpt-5.5"]
-input = 5
-output = 30
-cache_read = 0.5
-cache_write = 5
-
-["openrouter/openai/gpt-5.6-sol"]
-input = 4
-output = 20
-cache_read = 0.4
-cache_write = 5
-
-["openrouter/openai/gpt-5.6-terra"]
-input = 2
-output = 12
-cache_read = 0.2
-cache_write = 2.5
-
# Anthropic's standard token rates were checked on 2026-10-21.
# https://platform.claude.com/docs/en/about-claude/pricing
# The cache_write field uses the five-minute rate.
# The cache_write_1h field uses the one-hour rate.
-["claude-sonnet-5"]
+[[prices]]
+models = ["claude-sonnet-5", "anthropic/claude-sonnet-5"]
input = 2
output = 10
cache_read = 0.2
cache_write = 2.5
cache_write_1h = 4
-["claude-sonnet-5-5"]
+[[prices]]
+models = ["claude-sonnet-5-5", "anthropic/claude-sonnet-5.5"]
input = 2
output = 10
cache_read = 0.1
cache_write = 2.5
cache_write_1h = 4
-["claude-opus-5-5"]
+[[prices]]
+models = ["claude-opus-5-5", "anthropic/claude-opus-5.5"]
input = 4
output = 20
cache_read = 0.2
cache_write = 5
cache_write_1h = 8
-["claude-opus-4-7"]
-input = 5
-output = 25
-cache_read = 0.5
-cache_write = 6.25
-cache_write_1h = 10
-
-["claude-opus-4-8"]
-input = 5
-output = 25
-cache_read = 0.5
-cache_write = 6.25
-cache_write_1h = 10
-
-["claude-haiku-4-5"]
-input = 1
-output = 5
-cache_read = 0.1
-cache_write = 1.25
-cache_write_1h = 2
-
-["claude-haiku-4-5-20251001"]
-input = 1
-output = 5
-cache_read = 0.1
-cache_write = 1.25
-cache_write_1h = 2
-
-# These exact IDs and token rates were checked against OpenRouter on 2026-10-21.
-# https://openrouter.ai/api/v1/models
-
-["anthropic/claude-sonnet-5"]
-input = 2
-output = 10
-cache_read = 0.2
-cache_write = 2.5
-cache_write_1h = 4
-
-["anthropic/claude-opus-4.7"]
-input = 5
-output = 25
-cache_read = 0.5
-cache_write = 6.25
-cache_write_1h = 10
-
-["anthropic/claude-opus-4.8"]
+[[prices]]
+models = [
+ "claude-opus-4-7",
+ "anthropic/claude-opus-4.7",
+ "claude-opus-4-8",
+ "anthropic/claude-opus-4.8",
+ "claude-opus-5",
+ "anthropic/claude-opus-5",
+ "claude-opus-4-6",
+ "anthropic/claude-opus-4.6",
+]
input = 5
output = 25
cache_read = 0.5
cache_write = 6.25
cache_write_1h = 10
-# This gateway ID uses Anthropic list rates as an estimate. Gateway charges may differ.
-["azure/anthropic/claude-haiku-4-5"]
+[[prices]]
+models = [
+ "claude-haiku-4-5",
+ "claude-haiku-4-5-20251001",
+ "anthropic/claude-haiku-4.5",
+ "azure/anthropic/claude-haiku-4-5",
+]
input = 1
output = 5
cache_read = 0.1
cache_write = 1.25
cache_write_1h = 2
-# These direct Claude IDs use Anthropic standard rates, checked on 2026-10-21.
-# https://platform.claude.com/docs/en/about-claude/pricing
-# Haiku 5.5 rates apply up to 100,000 prompt tokens. Longer prompts cost five times more.
-
-["claude-opus-5"]
-input = 5
-output = 25
-cache_read = 0.5
-cache_write = 6.25
-cache_write_1h = 10
-
-["claude-fable-5"]
+[[prices]]
+models = ["claude-fable-5", "anthropic/claude-fable-5"]
input = 10
output = 50
cache_read = 1
cache_write = 12.5
cache_write_1h = 20
-["claude-fable-5-1"]
+[[prices]]
+models = ["claude-fable-5-1", "anthropic/claude-fable-5.1"]
input = 10
output = 50
cache_read = 0.25
cache_write = 12.5
cache_write_1h = 20
-["claude-haiku-5-5"]
+# Haiku 5.5 rates apply up to 100,000 prompt tokens. Longer prompts cost five times more.
+
+[[prices]]
+models = ["claude-haiku-5-5", "anthropic/claude-haiku-5.5"]
input = 0.1
output = 0.5
cache_read = 0.01
cache_write = 0.125
cache_write_1h = 0.2
-["claude-opus-4-6"]
-input = 5
-output = 25
-cache_read = 0.5
-cache_write = 6.25
-cache_write_1h = 10
-
-["claude-sonnet-4-6"]
-input = 3
-output = 15
-cache_read = 0.3
-cache_write = 3.75
-cache_write_1h = 6
-
-["claude-sonnet-4-5"]
+[[prices]]
+models = [
+ "claude-sonnet-4-6",
+ "anthropic/claude-sonnet-4.6",
+ "claude-sonnet-4-5",
+ "anthropic/claude-sonnet-4.5",
+]
input = 3
output = 15
cache_read = 0.3
@@ -306,35 +226,24 @@ cache_write_1h = 6
# Google bills cache storage by time, and this table excludes it.
# Separately counted cache writes, including one-hour writes, use the input rate.
-["gemini-4-argon"]
+[[prices]]
+models = ["gemini-4-argon"]
input = 2
output = 10
cache_read = 0.1
cache_write = 2
cache_write_1h = 2
-["gemini-3.8-flash"]
-input = 0.75
-output = 3.75
-cache_read = 0.075
-cache_write = 0.75
-cache_write_1h = 0.75
-
-["gemini-3.7-flash"]
+[[prices]]
+models = ["gemini-3.8-flash", "gemini-3.7-flash", "gemini-3.6-flash"]
input = 0.75
output = 3.75
cache_read = 0.075
cache_write = 0.75
cache_write_1h = 0.75
-["gemini-3.6-flash"]
-input = 0.75
-output = 3.75
-cache_read = 0.075
-cache_write = 0.75
-cache_write_1h = 0.75
-
-["gemini-3.5-flash-lite"]
+[[prices]]
+models = ["gemini-3.5-flash-lite"]
input = 0.3
output = 2.5
cache_read = 0.03
@@ -346,31 +255,372 @@ cache_write_1h = 0.3
# DeepSeek bills the older Flash IDs as deepseek-flash. This table uses one rate at all times.
# Separately counted cache writes, including one-hour writes, use the input rate.
-["deepseek-flash"]
+[[prices]]
+models = ["deepseek-flash", "deepseek-v4-flash", "deepseek-v4-flash-vision-exp"]
input = 0.3
output = 1.2
cache_read = 0.006
cache_write = 0.3
cache_write_1h = 0.3
-["deepseek-v4-flash"]
+[[prices]]
+models = ["deepseek-v4-pro"]
+input = 1.32
+output = 3.96
+cache_read = 0.044
+cache_write = 1.32
+cache_write_1h = 1.32
+
+[[prices]]
+models = ["openai/gpt-oss-120b"]
+input = 0.15
+output = 0.6
+cache_read = 0.075
+cache_write = 0.15
+
+[[prices]]
+models = ["openai/gpt-oss-20b"]
+input = 0.018
+output = 0.09
+cache_read = 0.009
+cache_write = 0.018
+
+[[prices]]
+models = ["google/gemini-4-argon"]
+input = 2
+output = 10
+cache_read = 0.1
+cache_write = 2
+
+[[prices]]
+models = ["google/gemini-3.8-flash"]
+input = 0.825
+output = 4.125
+cache_read = 0.0825
+cache_write = 0.825
+
+[[prices]]
+models = ["google/gemini-3.7-flash", "google/gemini-3.6-flash"]
+input = 0.75
+output = 3.75
+cache_read = 0.075
+cache_write = 0.75
+
+[[prices]]
+models = ["google/gemini-3.5-flash"]
+input = 1.5
+output = 9
+cache_read = 0.15
+cache_write = 1.5
+
+[[prices]]
+models = ["google/gemini-3.5-flash-lite", "google/gemini-2.5-flash"]
input = 0.3
-output = 1.2
-cache_read = 0.006
+output = 2.5
+cache_read = 0.03
cache_write = 0.3
-cache_write_1h = 0.3
-["deepseek-v4-flash-vision-exp"]
+[[prices]]
+models = ["google/gemini-3.1-flash-lite"]
+input = 0.25
+output = 1.5
+cache_read = 0.025
+cache_write = 0.25
+
+[[prices]]
+models = ["google/gemini-3.1-pro-preview"]
+input = 2
+output = 12
+cache_read = 0.2
+cache_write = 2
+
+[[prices]]
+models = ["google/gemini-2.5-pro"]
+input = 1.25
+output = 10
+cache_read = 0.125
+cache_write = 1.25
+
+[[prices]]
+models = ["google/gemini-2.5-flash-lite"]
+input = 0.1
+output = 0.4
+cache_read = 0.01
+cache_write = 0.1
+
+[[prices]]
+models = ["x-ai/grok-4.7", "x-ai/grok-4.6"]
+input = 2
+output = 6
+cache_read = 0.5
+cache_write = 2
+
+[[prices]]
+models = ["x-ai/grok-4.3"]
+input = 1.25
+output = 2.5
+cache_read = 0.2
+cache_write = 1.25
+
+[[prices]]
+models = ["moonshotai/kimi-k2.6"]
+input = 0.65
+output = 3.41
+cache_read = 0.15
+cache_write = 0.65
+
+[[prices]]
+models = ["moonshotai/kimi-k2.7-code"]
+input = 0.6712
+output = 3.35
+cache_read = 0.18
+cache_write = 0.6712
+
+[[prices]]
+models = ["z-ai/glm-5.1"]
+input = 1.4
+output = 4.4
+cache_read = 0.26
+cache_write = 1.4
+
+[[prices]]
+models = ["z-ai/glm-5.3-prime"]
+input = 2.8
+output = 8.8
+cache_read = 0.56
+cache_write = 2.8
+
+[[prices]]
+models = ["z-ai/glm-5.3-flashx"]
+input = 0.37
+output = 1.25
+cache_read = 0.09
+cache_write = 0.37
+
+[[prices]]
+models = ["qwen/qwen3.8-max-0902"]
+input = 2
+output = 6
+cache_read = 0.25
+cache_write = 2
+
+[[prices]]
+models = ["qwen/qwen3.8-flash"]
+input = 0.15
+output = 0.47
+cache_read = 0.016
+cache_write = 0.15
+
+[[prices]]
+models = ["qwen/qwen3.8-max-prime"]
+input = 4
+output = 12
+cache_read = 0.5
+cache_write = 4
+
+[[prices]]
+models = ["qwen/qwen3.7-plus"]
+input = 0.32
+output = 1.28
+cache_read = 0.064
+cache_write = 0.4
+
+[[prices]]
+models = ["qwen/qwen3.7-max"]
+input = 1.475
+output = 4.425
+cache_read = 0.295
+cache_write = 1.84375
+
+[[prices]]
+models = ["qwen/qwen3.7-flash"]
+input = 0.03
+output = 0.13
+cache_read = 0.006
+cache_write = 0.038
+
+[[prices]]
+models = ["minimax/minimax-m3", "minimax/minimax-m2.7"]
input = 0.3
output = 1.2
-cache_read = 0.006
+cache_read = 0.06
cache_write = 0.3
-cache_write_1h = 0.3
-["deepseek-v4-pro"]
-input = 1.32
-output = 3.96
-cache_read = 0.044
-cache_write = 1.32
-cache_write_1h = 1.32
+[[prices]]
+models = ["nvidia/nemotron-3-ultra-550b-a55b"]
+input = 0.6
+output = 2.4
+cache_read = 0.12
+cache_write = 0.6
+[[prices]]
+models = ["nvidia/nemotron-3.5-super-vl"]
+input = 0.15
+output = 0.5
+cache_read = 0.02
+cache_write = 0.15
+
+[[prices]]
+models = ["nvidia/nemotron-3.5-lightning"]
+input = 0.07
+output = 0.2
+cache_read = 0.04
+cache_write = 0.07
+
+[[prices]]
+models = ["nvidia/nemotron-3-super-120b-a12b"]
+input = 0.3
+output = 2.4
+cache_read = 0.15
+cache_write = 0.3
+
+[[prices]]
+models = ["mistralai/mistral-large-4-0"]
+input = 1.36
+output = 4.18
+cache_read = 0.14
+cache_write = 1.36
+
+[[prices]]
+models = ["mistralai/mistral-medium-3-5"]
+input = 1.5
+output = 7.5
+cache_read = 1.5
+cache_write = 1.5
+
+[[prices]]
+models = ["mistralai/mistral-small-2603"]
+input = 0.15
+output = 0.6
+cache_read = 0.015
+cache_write = 0.15
+
+[[prices]]
+models = ["mistralai/devstral-2512"]
+input = 0.4
+output = 2
+cache_read = 0.04
+cache_write = 0.4
+
+[[prices]]
+models = [
+ "meta/muse-spark-open",
+ "meta/muse-spark-1.3",
+ "meta/muse-spark-1.1",
+ "meta/muse-spark-1.2",
+]
+input = 1.25
+output = 4.25
+cache_read = 0.15
+cache_write = 1.25
+
+[[prices]]
+models = ["meta/muse-wonder"]
+input = 4
+output = 20
+cache_read = 0.2
+cache_write = 4
+
+[[prices]]
+models = ["stepfun/step-5-preview"]
+input = 1
+output = 2.7
+cache_read = 0.1
+cache_write = 1
+
+[[prices]]
+models = ["xiaomi/mimo-v2.6-pro"]
+input = 0.435
+output = 0.87
+cache_read = 0.0036
+cache_write = 0.435
+
+[[prices]]
+models = ["xiaomi/mimo-v2.6-flash"]
+input = 0.14
+output = 0.28
+cache_read = 0.0028
+cache_write = 0.14
+
+# Only these explicit OpenRouter :free IDs use zero rates, checked on 2026-10-21.
+# The same models without :free keep their paid rates.
+
+[[prices]]
+models = [
+ "minimax/minimax-m3.1-flash:free",
+ "nvidia/nemotron-3-ultra-550b-a55b:free",
+ "nvidia/nemotron-3.5-super-vl:free",
+ "nvidia/nemotron-3.5-lightning:free",
+]
+input = 0
+output = 0
+cache_read = 0
+cache_write = 0
+
+# These OpenRouter rates include provider discounts and can change with provider routing.
+# OpenRouter's endpoint API lists provider offers for these GLM, Kimi, and DeepSeek rates.
+# https://openrouter.ai/api/v1/models/{author}/{model}/endpoints
+# DeepSeek 1017 and Hy4 use the advertised off-peak rates. Peak rates are higher.
+
+[[prices]]
+models = ["deepseek/deepseek-v4.1-flash"]
+input = 0.02394
+output = 0.375
+cache_read = 0.000342
+cache_write = 0.02394
+
+[[prices]]
+models = ["deepseek/deepseek-v4.1-flash-1017"]
+input = 0.15
+output = 0.6
+cache_read = 0.003
+cache_write = 0.15
+
+[[prices]]
+models = ["deepseek/deepseek-v4-flash-0731"]
+input = 0.04444
+output = 0.13332
+cache_read = 0.001414
+cache_write = 0.04444
+
+[[prices]]
+models = ["deepseek/deepseek-v4-pro-0813"]
+input = 0.66
+output = 1.98
+cache_read = 0.022
+cache_write = 0.66
+
+[[prices]]
+models = ["z-ai/glm-5.3"]
+input = 0.079
+output = 1.4
+cache_read = 0.078
+cache_write = 0.079
+
+[[prices]]
+models = ["z-ai/glm-5.3-flash"]
+input = 0.02282
+output = 0.05385
+cache_read = 0.004885
+cache_write = 0.02282
+
+[[prices]]
+models = ["z-ai/glm-5.2"]
+input = 0.0341
+output = 7
+cache_read = 0.033
+cache_write = 0.0341
+
+[[prices]]
+models = ["moonshotai/kimi-k3"]
+input = 0.239
+output = 11.936
+cache_read = 0.21
+cache_write = 0.239
+
+[[prices]]
+models = ["tencent/hy4-preview"]
+input = 0.7506
+output = 2.2509
+cache_read = 0.0378
+cache_write = 0.7506
diff --git a/crates/switchyard-costs/src/lib.rs b/crates/switchyard-costs/src/lib.rs
index 6c9de8be5..751d872ad 100644
--- a/crates/switchyard-costs/src/lib.rs
+++ b/crates/switchyard-costs/src/lib.rs
@@ -53,6 +53,56 @@ struct Rates {
cache_write_1h: Option<f64>,
}
+#[derive(Deserialize)]
+#[serde(untagged, deny_unknown_fields)]
+enum PricingFile {
+ Grouped { prices: Vec<PriceGroup> },
+ Keyed(Pricing),
+}
+
+// These explicit rate fields let Serde reject unknown and repeated fields.
+#[derive(Deserialize)]
+#[serde(deny_unknown_fields)]
+struct PriceGroup {
+ models: Vec<String>,
+ input: f64,
+ output: f64,
+ cache_read: f64,
+ cache_write: f64,
+ cache_write_1h: Option<f64>,
+}
+
+impl PricingFile {
+ /// This method expands groups once so cost calculations keep using exact model lookups.
+ fn into_pricing(self) -> Result<Pricing, String> {
+ let pricing = match self {
+ Self::Keyed(pricing) => pricing,
+ Self::Grouped { prices } => {
+ let mut rates = HashMap::new();
+ for group in prices {
+ if group.models.is_empty() {
+ return Err("a pricing group must contain at least one model ID".into());
+ }
+ let rate = Rates {
+ input: group.input,
+ output: group.output,
+ cache_read: group.cache_read,
+ cache_write: group.cache_write,
+ cache_write_1h: group.cache_write_1h,
+ };
+ for model in group.models {
+ if rates.insert(model.clone(), rate.clone()).is_some() {
+ return Err(format!("duplicate pricing model {model:?}"));
+ }
+ }
+ }
+ Pricing(rates)
+ }
+ };
+ pricing.validate()
+ }
+}
+
/// This value identifies how a saved monetary amount was obtained.
#[derive(Clone, Copy, Debug, PartialEq, Deserialize, Serialize)]
#[serde(rename_all = "snake_case")]
@@ -101,16 +151,17 @@ impl Pricing {
pricing.map_err(|error| format!("invalid pricing file {}: {error}", path.display()))
}
- /// This function reads a pricing file's JSON and checks every model ID and rate.
+ /// This function reads JSON with model-keyed rates or a `prices` array of model groups.
+ /// This function checks every model ID and rate before returning the table.
pub fn parse(json: &[u8]) -> Result<Self, String> {
- let pricing: Self = serde_json::from_slice(json).map_err(|error| error.to_string())?;
- pricing.validate()
+ let file: PricingFile = serde_json::from_slice(json).map_err(|error| error.to_string())?;
+ file.into_pricing()
}
/// TOML rates use the same typed fields and validation as JSON rates.
fn parse_toml(text: &str) -> Result<Self, String> {
- let pricing: Self = toml::from_str(text).map_err(|error| error.to_string())?;
- pricing.validate()
+ let file: PricingFile = toml::from_str(text).map_err(|error| error.to_string())?;
+ file.into_pricing()
}
/// Both parsers use this check so model IDs and rate limits agree across formats.
diff --git a/crates/switchyard-costs/tests/pricing.rs b/crates/switchyard-costs/tests/pricing.rs
new file mode 100644
index 000000000..d096348f4
--- /dev/null
+++ b/crates/switchyard-costs/tests/pricing.rs
@@ -0,0 +1,232 @@
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+use switchyard_costs::{CostSource, Pricing};
+use switchyard_protocol::{InputCacheUsage, Usage};
+
+type TestResult = Result<(), Box<dyn std::error::Error>>;
+
+const GROUPED_TOML: &str = r#"
+[[prices]]
+models = ["direct", "provider/model"]
+input = 1
+output = 2
+cache_read = 0.1
+cache_write = 1.25
+cache_write_1h = 2
+"#;
+
+#[test]
+fn grouped_and_keyed_files_price_the_same_usage() -> TestResult {
+ let dir = tempfile::tempdir()?;
+ let usage = Usage {
+ input_tokens: Some(1_000_000),
+ output_tokens: Some(2_000_000),
+ cache: Some(Box::new(InputCacheUsage {
+ cached_input_tokens: Some(3_000_000),
+ cache_creation_input_tokens: Some(5_000_000),
+ cache_creation_1h_input_tokens: Some(2_000_000),
+ })),
+ ..Usage::default()
+ };
+ for (name, text, models) in [
+ (
+ "grouped.ToMl",
+ GROUPED_TOML,
+ vec!["direct", "provider/model"],
+ ),
+ (
+ "grouped.json",
+ r#"{"prices":[{"models":["direct","provider/model"],"input":1,"output":2,"cache_read":0.1,"cache_write":1.25,"cache_write_1h":2}]}"#,
+ vec!["direct", "provider/model"],
+ ),
+ (
+ "keyed.toml",
+ "[\"provider/model\"]\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1.25\ncache_write_1h=2",
+ vec!["provider/model"],
+ ),
+ (
+ "keyed.json",
+ r#"{"provider/model":{"input":1,"output":2,"cache_read":0.1,"cache_write":1.25,"cache_write_1h":2}}"#,
+ vec!["provider/model"],
+ ),
+ (
+ "keyed-reserved.toml",
+ "[prices]\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1.25\ncache_write_1h=2",
+ vec!["prices"],
+ ),
+ (
+ "keyed-reserved.json",
+ r#"{"prices":{"input":1,"output":2,"cache_read":0.1,"cache_write":1.25,"cache_write_1h":2}}"#,
+ vec!["prices"],
+ ),
+ ("empty-grouped.toml", "prices=[]", vec![]),
+ ("empty-grouped.json", r#"{"prices":[]}"#, vec![]),
+ ("empty-keyed.toml", "", vec![]),
+ ("empty-keyed.json", "{}", vec![]),
+ ] {
+ let path = dir.path().join(name);
+ std::fs::write(&path, text)?;
+ let prices = Pricing::load(&path)?;
+ for model in models {
+ let cost = prices.estimate(model, &usage).ok_or("missing estimate")?;
+ assert!((cost.usd - 13.05).abs() < 1e-9, "{name}: {model}");
+ assert_eq!(cost.source, CostSource::Estimated);
+ }
+ assert!(!prices.contains("provider/typo"));
+ assert_eq!(prices.estimate("provider/typo", &usage), None);
+ }
+ Ok(())
+}
+
+// These fixtures keep repeated fields in JSON text so decoding cannot erase them first.
+#[test]
+fn rejects_invalid_groups_and_preserved_raw_json_fields() {
+ for text in [
+ r#"{"prices":[{"models":[],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,
+ r#"{"prices":[{"models":["a","a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,
+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1},{"models":["a"],"input":3,"output":4,"cache_read":0.2,"cache_write":3}]}"#,
+ r#"{"prices":[{"models":[""],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,
+ r#"{"prices":[{"models":[" a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,
+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1}]}"#,
+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1,"cache_wirte":1}]}"#,
+ r#"{"prices":[{"models":["a"],"models":["b"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}]}"#,
+ r#"{"prices":[{"models":["a"],"input":1,"input":3,"output":2,"cache_read":0.1,"cache_write":1}]}"#,
+ r#"{"prices":[],"prices":[]}"#,
+ r#"{"prices":[],"unexpected":true}"#,
+ r#"{"prices":[{"models":["a"],"input":1,"output":2,"cache_read":0.1,"cache_write":1}],"b":{"input":1,"output":2,"cache_read":0.1,"cache_write":1}}"#,
+ r#"{"a":{"input":1,"output":2,"cache_read":0.1,"cache_write":1},"a":{"input":3,"output":4,"cache_read":0.2,"cache_write":3}}"#,
+ r#"{"a":{"input":1,"input":3,"output":2,"cache_read":0.1,"cache_write":1}}"#,
+ r#"{"a":{"input":1,"output":2,"cache_read":0.1,"cache_write":1,"cache_wirte":1}}"#,
+ ] {
+ assert!(Pricing::parse(text.as_bytes()).is_err(), "accepted {text}");
+ }
+}
+
+#[test]
+fn rejects_invalid_toml_before_returning_a_price_table() -> TestResult {
+ let dir = tempfile::tempdir()?;
+ let path = dir.path().join("prices.toml");
+ for text in [
+ "[[prices]]\nmodels=[]\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",
+ "[[prices]]\nmodels=['a','a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",
+ "[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1\n[[prices]]\nmodels=['a']\ninput=3\noutput=4\ncache_read=0.2\ncache_write=3",
+ "[[prices]]\nmodels=['a ']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",
+ "[[prices]]\nmodels=['a']\nmodels=['b']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",
+ "[[prices]]\nmodels=['a']\ninput=1\ninput=3\noutput=2\ncache_read=0.1\ncache_write=1",
+ "unexpected=true\n[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",
+ "['b']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1\n[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1",
+ "[[prices]]\nmodels=['a']\ninput=1\noutput=2\ncache_read=0.1\ncache_write=1\ncache_wirte=1",
+ ] {
+ std::fs::write(&path, text)?;
+ let error = Pricing::load(&path).err().ok_or("invalid pricing loaded")?;
+ assert!(error.starts_with("invalid pricing file "), "{error}");
+ assert!(error.contains(&path.display().to_string()), "{error}");
+ }
+ Ok(())
+}
+
+#[test]
+fn every_grouped_rate_must_be_finite_and_in_range() -> TestResult {
+ let dir = tempfile::tempdir()?;
+ let path = dir.path().join("prices.toml");
+ for field in [
+ "input",
+ "output",
+ "cache_read",
+ "cache_write",
+ "cache_write_1h",
+ ] {
+ for value in ["-1", "10001", "nan", "inf"] {
+ let mut text = String::from("[[prices]]\nmodels=['a']\n");
+ for name in [
+ "input",
+ "output",
+ "cache_read",
+ "cache_write",
+ "cache_write_1h",
+ ] {
+ text.push_str(&format!(
+ "{name}={}\n",
+ if name == field { value } else { "1" }
+ ));
+ }
+ std::fs::write(&path, &text)?;
+ assert!(Pricing::load(&path).is_err(), "accepted {field}={value}");
+ }
+ }
+ Ok(())
+}
+
+#[test]
+fn bundled_aliases_keep_all_cache_rates() -> TestResult {
+ let prices = Pricing::bundled()?;
+ let usage = Usage {
+ input_tokens: Some(1_000_000),
+ output_tokens: Some(1_000_000),
+ cache: Some(Box::new(InputCacheUsage {
+ cached_input_tokens: Some(1_000_000),
+ cache_creation_input_tokens: Some(2_000_000),
+ cache_creation_1h_input_tokens: Some(1_000_000),
+ })),
+ ..Usage::default()
+ };
+ for model in [
+ "claude-haiku-4-5",
+ "claude-haiku-4-5-20251001",
+ "anthropic/claude-haiku-4.5",
+ "azure/anthropic/claude-haiku-4-5",
+ ] {
+ let cost = prices.estimate(model, &usage).ok_or("missing alias")?;
+ assert!((cost.usd - 9.35).abs() < 1e-9, "{model}");
+ }
+ Ok(())
+}
+
+#[test]
+fn bundled_provider_prices_and_optional_rates_stay_distinct() -> TestResult {
+ let prices = Pricing::bundled()?;
+ let mut usage = Usage {
+ input_tokens: Some(1_000_000),
+ output_tokens: Some(0),
+ ..Usage::default()
+ };
+ for (model, expected) in [
+ ("gemini-3.8-flash", 0.75),
+ ("google/gemini-3.8-flash", 0.825),
+ ] {
+ assert_eq!(
+ prices.estimate(model, &usage).ok_or("missing model")?.usd,
+ expected
+ );
+ }
+ // A missing one-hour rate must keep the estimate unknown, even if other rates match.
+ usage.cache = Some(Box::new(InputCacheUsage {
+ cached_input_tokens: Some(0),
+ cache_creation_input_tokens: Some(1_000_000),
+ cache_creation_1h_input_tokens: Some(1_000_000),
+ }));
+ assert_eq!(
+ prices
+ .estimate("gemini-3.7-flash", &usage)
+ .ok_or("missing hourly rate")?
+ .usd,
+ 1.5
+ );
+ assert_eq!(prices.estimate("google/gemini-3.7-flash", &usage), None);
+ usage.cache = None;
+ for (model, expected) in [
+ ("nvidia/nemotron-3-ultra-550b-a55b", 0.6),
+ ("nvidia/nemotron-3-ultra-550b-a55b:free", 0.0),
+ ] {
+ assert_eq!(
+ prices
+ .estimate(model, &usage)
+ .ok_or("missing free or paid rate")?
+ .usd,
+ expected
+ );
+ }
+ assert!(!prices.contains("google/gemini-3.8-flash:free"));
+ Ok(())
+}
diff --git a/crates/switchyard-menubar/src/pricing.rs b/crates/switchyard-menubar/src/pricing.rs
index e2cf1c98d..5cff7ac49 100644
--- a/crates/switchyard-menubar/src/pricing.rs
+++ b/crates/switchyard-menubar/src/pricing.rs
@@ -105,6 +105,31 @@ mod tests {
assert!((savings.percent().expect("percent") - 80.0).abs() < 1e-9);
}
+ #[test]
+ fn settings_load_bundled_groups_for_baseline_estimates() -> Result<(), String> {
+ let dir = tempfile::tempdir().map_err(|error| error.to_string())?;
+ let config = crate::config::Config::load(&dir.path().join("menubar.toml"))?;
+ let mut totals = Totals::default();
+ totals.routed.insert(
+ "answer-model".to_string(),
+ ModelTokens {
+ requests: 1,
+ input: 1_000_000,
+ output: 1_000_000,
+ cached_input: 1_000_000,
+ cache_write: 2_000_000,
+ cache_write_1h: 1_000_000,
+ },
+ );
+ record_cost(&mut totals, 0.5);
+ for model in ["claude-haiku-4-5", "anthropic/claude-haiku-4.5"] {
+ let savings = estimate(&totals, &config.prices, model).ok_or("missing baseline")?;
+ assert!((savings.baseline - 9.35).abs() < 1e-9, "{model}");
+ assert!((savings.saved() - 8.85).abs() < 1e-9, "{model}");
+ }
+ Ok(())
+ }
+
#[test]
fn classifier_overhead_counts_against_savings() {
let mut totals = Totals::default();
diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.md
index 6b2d795cc..ce468ef08 100644
--- a/crates/switchyard-server/README.md
+++ b/crates/switchyard-server/README.md
@@ -148,7 +148,9 @@ cache_write = 2
```
The `.toml` extension check ignores case. Other filenames are read as JSON, so existing
-JSON overrides still work.
+JSON overrides still work. TOML and JSON also accept groups of exact IDs that share
+one set of rates, as shown in the [pi guide](../../docs/integrations/pi.md#estimate-the-cost).
+The bundled table uses this grouped format and can be supplied to `--pricing-file`.
Session stats include cost totals. Missing usage or rates leave cost unknown; `known_usd`
keeps the priced subtotal and `total_usd` is null when any cost is unknown. Rates load once
diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rs
index bb0dbf32e..c9d668766 100644
--- a/crates/switchyard-server/tests/server.rs
+++ b/crates/switchyard-server/tests/server.rs
@@ -4550,8 +4550,11 @@ async fn routing_log_prefers_canonical_and_preserves_legacy_fallback() -> TestRe
let upstream = MockUpstream::start().await?;
let temp_dir = tempfile::tempdir()?;
let log_path = temp_dir.path().join("routing.jsonl");
- let state = random_state(&upstream.base_url, &[(ROUTE_MODEL, &["model/a"])])?
- .with_routing_log(&log_path)?;
+ let state = random_state(
+ &upstream.base_url,
+ &[(ROUTE_MODEL, &["anthropic/claude-haiku-4.5"])],
+ )?
+ .with_routing_log(&log_path)?;
let app = build_switchyard_router(state);
let request = HttpRequest::builder()
@@ -4581,7 +4584,10 @@ async fn routing_log_prefers_canonical_and_preserves_legacy_fallback() -> TestRe
assert_eq!(stats["total_calls"], 1);
assert_eq!(stats["total_prompt_tokens"], 10);
assert_eq!(stats["total_cached_tokens"], 7);
- assert_eq!(stats["models"]["model/a"]["completion_tokens"], 2);
+ assert_eq!(
+ stats["models"]["anthropic/claude-haiku-4.5"]["completion_tokens"],
+ 2
+ );
let legacy = send(
&app,
@@ -4619,6 +4625,9 @@ async fn routing_log_prefers_canonical_and_preserves_legacy_fallback() -> TestRe
let first: Value =
serde_json::from_str(records.lines().next().ok_or("routing log was empty")?)?;
assert_eq!(first["session_id"], "canonical-session");
+ assert_eq!(first["model"], "anthropic/claude-haiku-4.5");
+ assert_eq!(first["cost"]["source"], "estimated");
+ assert!((first["cost"]["usd"].as_f64().ok_or("missing cost")? - 0.0000137).abs() < 1e-12);
assert_eq!(first["turn_id"], "canonical-turn");
assert_eq!(first["origin"], r#"custom-agent/"quoted"\path"#);
let second: Value = serde_json::from_str(records.lines().nth(1).ok_or("missing record")?)?;
@@ -4713,8 +4722,11 @@ async fn routing_log_keeps_the_canonical_session_id_until_a_stream_drains() -> T
let upstream = MockUpstream::start().await?;
let temp_dir = tempfile::tempdir()?;
let log_path = temp_dir.path().join("routing.jsonl");
- let state = random_state(&upstream.base_url, &[(ROUTE_MODEL, &["model/a"])])?
- .with_routing_log(&log_path)?;
+ let state = random_state(
+ &upstream.base_url,
+ &[(ROUTE_MODEL, &["anthropic/claude-haiku-4.5"])],
+ )?
+ .with_routing_log(&log_path)?;
let app = build_switchyard_router(state);
// `send_with_headers` collects the response body, so the stream wrapper reaches
@@ -4755,6 +4767,9 @@ async fn routing_log_keeps_the_canonical_session_id_until_a_stream_drains() -> T
let record: Value = serde_json::from_str(&std::fs::read_to_string(log_path)?)?;
assert_eq!(record["route_id"], ROUTE_MODEL);
+ assert_eq!(record["model"], "anthropic/claude-haiku-4.5");
+ assert_eq!(record["cost"]["source"], "estimated");
+ assert!((record["cost"]["usd"].as_f64().ok_or("missing cost")? - 0.0000312).abs() < 1e-12);
assert_eq!(record["turn_id"], "streaming-turn");
assert_eq!(record["algorithm"], "random");
assert_eq!(record["origin"], "codex-cli");
diff --git a/docs/integrations/pi.md b/docs/integrations/pi.md
index a456358d1..ba9700c59 100644
--- a/docs/integrations/pi.md
+++ b/docs/integrations/pi.md
@@ -187,9 +187,30 @@ Put each quoted model ID in a top-level table, as shown above. Quotes keep dots
within the ID. The `.toml` extension check ignores case; other filenames are read
as JSON, so existing JSON overrides still work.
-For your own models, copy the entry for each model and replace the key with its exact
-`model` value from the routing log. Use the served model ID rather than the routing alias,
-and replace the example's numbers with your provider's rates. List recorded model IDs with:
+To maintain one set of rates for several exact IDs, list them in a group:
+
+```toml
+[[prices]]
+models = ["claude-haiku-4-5", "anthropic/claude-haiku-4.5"]
+input = 1
+output = 5
+cache_read = 0.1
+cache_write = 1.25
+cache_write_1h = 2
+```
+
+Use one `[[prices]]` record per group, with at least one ID in `models`. JSON accepts
+the same records in a top-level `prices` array. Each file uses either grouped records
+or model-keyed tables. The bundled table uses groups and can also be loaded with
+`--pricing-file`.
+
+All IDs in a group share every rate, including `cache_write_1h` when present. Keep
+separate groups when provider prices differ. The loader rejects duplicate IDs within
+or across groups, invalid IDs or rates, and unknown fields before using the table.
+
+For your own models, use each exact `model` value from the routing log as a table key
+or an ID in a group. Use the served model ID rather than the routing alias, and replace
+the example's numbers with your provider's rates. List recorded model IDs with:
```bash
jq -r '.model' PATH | sort -u
@@ -199,8 +220,9 @@ Every entry requires `input`, `output`, `cache_read`, and `cache_write`. Set a r
only when that usage is free. Each rate must be a finite number from 0 to 10000 USD per
million tokens.
-The example's `cache_write` price is for a 5-minute cache. A 1-hour cache costs 2 times
-the input price instead of 1.25 times; set `cache_write_1h` to `4.00` for that example.
+The Claude Sonnet 5 example uses the 5-minute `cache_write` price. A 1-hour cache
+costs 2 times the input price instead of 1.25 times; set `cache_write_1h` to `4.00`
+for that example.
Include that rate to estimate reported one-hour cache writes; without it, the estimate
remains unknown.