Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

feat(routing): add auto algorithm type (#654)

ryan-lempka8b3c62bc107a
diff --git a/crates/switchyard-runner/src/algorithm.rs b/crates/switchyard-runner/src/algorithm.rs
index 0e1ede289..e5db6d477 100644
--- a/crates/switchyard-runner/src/algorithm.rs
+++ b/crates/switchyard-runner/src/algorithm.rs
@@ -258,6 +258,16 @@ pub enum AlgorithmSpec {
#[serde(default)]
subagents: Option<SubagentRouteConfig>,
},
+ /// Picks a routing strategy automatically, preset with recommended knobs.
+ /// Currently a `stage_router` with `picker = "efficient_first"` and
+ /// `confidence_threshold = 0.5`; change `build_algorithm`'s `Auto` arm to
+ /// repoint it at a different algorithm or preset.
+ Auto {
+ /// The capable tier.
+ capable_target: String,
+ /// The efficient tier.
+ efficient_target: String,
+ },
/// A judge picks the tier at each user turn; a stage router runs the turns within it.
Composite {
/// Judge that picks the tier. Called through its own target.
@@ -464,6 +474,10 @@ impl AlgorithmSpec {
}
names
}
+ Self::Auto {
+ capable_target,
+ efficient_target,
+ } => vec![capable_target.as_str(), efficient_target.as_str()],
Self::Composite {
stage, subagents, ..
} => {
@@ -553,6 +567,7 @@ impl AlgorithmSpec {
| Self::Passthrough { .. }
| Self::LlmClassifier { .. }
| Self::StageRouter { .. }
+ | Self::Auto { .. }
| Self::Composite { .. }
| Self::PrefillRouter { .. } => None,
}
@@ -1023,6 +1038,21 @@ fn build_algorithm(
let parent: Arc<dyn Algorithm> = Arc::new(algorithm);
attach_subagent_router(route_name, parent, subagents.as_ref(), targets)
}
+ AlgorithmSpec::Auto {
+ capable_target,
+ efficient_target,
+ } => {
+ let capable = resolve_target_model_id(route_name, capable_target, targets)?;
+ let efficient = resolve_target_model_id(route_name, efficient_target, targets)?;
+ let config = StageRouterConfig::new(PickerMode::EfficientFirst, 0.5);
+ let algorithm = StageRouter::new(capable, efficient, config).map_err(|error| {
+ AlgorithmConfigError::with_source(
+ format!("auto route {route_name}: {error}"),
+ error,
+ )
+ })?;
+ Ok(Arc::new(algorithm))
+ }
AlgorithmSpec::Composite {
classifier,
stage,
diff --git a/crates/switchyard-runner/src/config.rs b/crates/switchyard-runner/src/config.rs
index a4bb309aa..5317df264 100644
--- a/crates/switchyard-runner/src/config.rs
+++ b/crates/switchyard-runner/src/config.rs
@@ -982,6 +982,22 @@ confidence_threshold = 0.5
assert!(error_message(&config).contains("unknown field"));
}
+ #[test]
+ fn auto_route_builds_a_stage_router_with_no_extra_fields() -> RunnerResult<()> {
+ let config = format!(
+ r#"{VALID_CONFIG}
+[routes.auto]
+id = "switchyard/auto"
+type = "auto"
+capable_target = "strong"
+efficient_target = "weak"
+"#
+ );
+ let runner = runner_from_toml(&config)?;
+ assert!(runner.route("switchyard/auto").is_some());
+ Ok(())
+ }
+
#[test]
fn composite_stage_block_rejects_an_unknown_field() {
let config = composite_config().replace(
diff --git a/docs/getting_started.md b/docs/getting_started.md
index 9e7d011b4..7c1cb362b 100644
--- a/docs/getting_started.md
+++ b/docs/getting_started.md
@@ -64,7 +64,7 @@ Cargo builds the release binary and installs it into `~/.cargo/bin` by default.
The Rust server reads an explicit TOML file.
-Create `routes.toml` with an LLM-classifier route:
+Create `routes.toml` with an auto route:
```toml
schema_version = 1
@@ -84,12 +84,9 @@ llm_client = "openrouter"
[routes.smart]
id = "switchyard"
-type = "llm_classifier"
-mode = "capability"
-classifier_target = "weak"
-strong_target = "strong"
-weak_target = "weak"
-base_threshold = 0.5
+type = "auto"
+capable_target = "strong"
+efficient_target = "weak"
```
`format` selects the upstream protocol and must be `openai_chat`,
@@ -131,11 +128,12 @@ curl http://localhost:4000/v1/chat/completions \
#### Choose a route type
-This guide uses `llm_classifier`, which asks a classifier target whether each
-request should use the weak or strong target. The Rust server also supports:
+This guide uses `auto`, which routes with Switchyard's recommended default
+settings. The Rust server also supports:
| Algorithm | Use it when | Config |
|---|---|---|
+| Auto | You want a recommended default instead of picking a strategy yourself. | `auto` |
| [Random](routing_algorithms/random_routing.md) | You need a weighted split for A/B tests or baselines. | `random` |
| [LLM classifier](routing_algorithms/llm_classifier_routing.md) | Request content should decide whether to use the weak or strong target. | `llm_classifier` |
| [Stage router](routing_algorithms/stage_router_routing.md) | Tool-result and progress signals should select an efficient or capable target. | `stage_router` |
diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.md
index 2708b68db..a819a6178 100644
--- a/docs/reference/toml_schema.md
+++ b/docs/reference/toml_schema.md
@@ -252,6 +252,21 @@ optional `handoff_notes` and `classifier` tables and for tuning.
| `classifier.response_format_type` | No | `json_schema` | Structured-output mode for the optional classifier judge. Use `json_object` when the classifier provider does not support JSON Schema; Switchyard adds the schema to the prompt and validates the verdict locally. |
| `subagents` | No | unset | Nested `passthrough` or custom `llm_classifier` policy used only for delegated sub-agent work. See [Sub-Agent-Aware Routing](../routing_algorithms/subagent_routing.md). |
+### `auto`
+
+Uses Switchyard's recommended default routing strategy instead of one you pick
+yourself: a `stage_router` preset with `picker = "efficient_first"` and
+`confidence_threshold = 0.5`, no classifier. See
+[Stage-Router Routing](../routing_algorithms/stage_router_routing.md) for a
+deeper dive on the current default, or the
+[strategy table](../routing_algorithms/overview.md#choose-a-strategy) to pick
+one manually.
+
+| Key | Required | Default | Meaning |
+|---|:---:|---|---|
+| `capable_target` | Yes | — | Capable tier. |
+| `efficient_target` | Yes | — | Efficient tier. |
+
### `composite`
Composes other algorithms, letting one set another's
diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.md
index 648e6866b..6350bc24e 100644
--- a/docs/routing_algorithms/overview.md
+++ b/docs/routing_algorithms/overview.md
@@ -16,6 +16,7 @@ configuration and tuning. For the vocabulary these pages use, see
| [Random Routing](random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |
| [LLM Classifier Routing](llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` |
| [Stage-Router Routing](stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` |
+| Auto Routing | You want a recommended default instead of picking a strategy yourself. For a deeper dive on the current default, see [Stage-Router Routing](stage_router_routing.md); for full control, pick one of the strategies above instead. | `auto` |
| [Composite Routing](composite_routing.md) | Routing algorithms are composed, one setting the configuration of another before handing off. Today an LLM classifier sets a stage router's default tier. | `composite` |
| [Escalation-Router Routing](escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` |
| [Advisor-Gate Routing](advisor_gate_routing.md) | One model should serve every turn, with a stronger reviewer approving its "done" claims or sending back a redo plan. | `advisor` |