Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

docs(stage-router): mark capable_first experimental, and warn at startup (#363)

sabhatinas22b78d9fe4df
diff --git a/crates/switchyard-server/src/config.rs b/crates/switchyard-server/src/config.rs
index b644d4a04..f7be8c646 100644
--- a/crates/switchyard-server/src/config.rs
+++ b/crates/switchyard-server/src/config.rs
@@ -928,6 +928,11 @@ fn build_algorithm(
classifier,
..
} => {
+ if matches!(picker, PickerMode::CapableFirst) {
+ tracing::warn!(
+ "stage_router route {route_name} uses picker \"capable_first\", which is experimental: published thresholds and routing results all come from \"efficient_first\", so there is no calibrated confidence_threshold for it and no measured accuracy or cost. Use \"efficient_first\" unless you are running your own calibration."
+ );
+ }
let capable = resolve_target(route_name, capable_target, targets)?;
let efficient = resolve_target(route_name, efficient_target, targets)?;
let mut config = StageRouterConfig::new(*picker, *confidence_threshold);
diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.md
index c42d0560f..6caa71fd5 100644
--- a/docs/reference/toml_schema.md
+++ b/docs/reference/toml_schema.md
@@ -174,7 +174,7 @@ optional `handoff_notes` and `classifier` tables and for tuning.
|---|:---:|---|---|
| `capable_target` | Yes | — | Capable tier. |
| `efficient_target` | Yes | — | Efficient tier. |
-| `picker` | Yes | — | `capable_first` or `efficient_first`. Tier used when the signals are not confident. |
+| `picker` | Yes | — | `efficient_first`, or `capable_first` (experimental, unbenchmarked). Tier used when the signals are not confident. |
| `confidence_threshold` | Yes | — | Corroboration a decisive pick needs. In `[0, 1]`. |
| `recent_turn_window` | No | `3` | Trailing tool results the signals are computed over. |
| `capable_system_prompt` | No | unset | System prompt handed to the capable tier. |
diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.md
index 26bee1189..c471a9882 100644
--- a/docs/routing_algorithms/stage_router_routing.md
+++ b/docs/routing_algorithms/stage_router_routing.md
@@ -67,13 +67,22 @@ the classifier). Raising the threshold shrinks that path; lowering it widens it.
The picker name says which tier is the **default**: the tier used when the
signals are ambiguous and no classifier verdict is available.
-- **`capable_first`**: capable is the default; drop to efficient only when the
- signals (or the classifier) clearly say so. Quality-first.
- **`efficient_first`**: efficient is the default; escalate to capable only when
the signals (or the classifier) clearly say so. Cost-first.
+- **`capable_first`** *(experimental)*: capable is the default; drop to efficient
+ only when the signals (or the classifier) clearly say so. Quality-first.
Both pickers read the same signals; only the default tier differs.
+!!! warning "`capable_first` is experimental"
+
+ Every published threshold and routing result comes from `efficient_first`
+ runs. `capable_first` works and the server accepts it, but it has not been
+ benchmarked, so there are no calibrated thresholds for it and no measured
+ accuracy or cost figures to set expectations against. The server logs a
+ warning at startup when a route selects it. Use `efficient_first` unless you
+ are running your own calibration.
+
## Tuning `confidence_threshold`
The scorer rates each turn from `0` (signals are neutral) to `1` (signals point