Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs(stage-router): mark capable_first experimental, and warn at startup (#363)
sabhatinas22b78d9fe4df
diff --git a/crates/switchyard-server/src/config.rs b/crates/switchyard-server/src/config.rsindex b644d4a04..f7be8c646 100644--- a/crates/switchyard-server/src/config.rs+++ b/crates/switchyard-server/src/config.rs@@ -928,6 +928,11 @@ fn build_algorithm(classifier,..} => {+ if matches!(picker, PickerMode::CapableFirst) {+ tracing::warn!(+ "stage_router route {route_name} uses picker \"capable_first\", which is experimental: published thresholds and routing results all come from \"efficient_first\", so there is no calibrated confidence_threshold for it and no measured accuracy or cost. Use \"efficient_first\" unless you are running your own calibration."+ );+ }let capable = resolve_target(route_name, capable_target, targets)?;let efficient = resolve_target(route_name, efficient_target, targets)?;let mut config = StageRouterConfig::new(*picker, *confidence_threshold);diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.mdindex c42d0560f..6caa71fd5 100644--- a/docs/reference/toml_schema.md+++ b/docs/reference/toml_schema.md@@ -174,7 +174,7 @@ optional `handoff_notes` and `classifier` tables and for tuning.|---|:---:|---|---|| `capable_target` | Yes | — | Capable tier. || `efficient_target` | Yes | — | Efficient tier. |-| `picker` | Yes | — | `capable_first` or `efficient_first`. Tier used when the signals are not confident. |+| `picker` | Yes | — | `efficient_first`, or `capable_first` (experimental, unbenchmarked). Tier used when the signals are not confident. || `confidence_threshold` | Yes | — | Corroboration a decisive pick needs. In `[0, 1]`. || `recent_turn_window` | No | `3` | Trailing tool results the signals are computed over. || `capable_system_prompt` | No | unset | System prompt handed to the capable tier. |diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.mdindex 26bee1189..c471a9882 100644--- a/docs/routing_algorithms/stage_router_routing.md+++ b/docs/routing_algorithms/stage_router_routing.md@@ -67,13 +67,22 @@ the classifier). Raising the threshold shrinks that path; lowering it widens it.The picker name says which tier is the **default**: the tier used when thesignals are ambiguous and no classifier verdict is available.-- **`capable_first`**: capable is the default; drop to efficient only when the- signals (or the classifier) clearly say so. Quality-first.- **`efficient_first`**: efficient is the default; escalate to capable only whenthe signals (or the classifier) clearly say so. Cost-first.+- **`capable_first`** *(experimental)*: capable is the default; drop to efficient+ only when the signals (or the classifier) clearly say so. Quality-first.Both pickers read the same signals; only the default tier differs.+!!! warning "`capable_first` is experimental"++ Every published threshold and routing result comes from `efficient_first`+ runs. `capable_first` works and the server accepts it, but it has not been+ benchmarked, so there are no calibrated thresholds for it and no measured+ accuracy or cost figures to set expectations against. The server logs a+ warning at startup when a route selects it. Use `efficient_first` unless you+ are running your own calibration.+## Tuning `confidence_threshold`The scorer rates each turn from `0` (signals are neutral) to `1` (signals point