Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

docs: simplify routing choices (#763)

ryan-lempka4645dad0f61e
diff --git a/README.md b/README.md
index f94d3a077..a30261864 100644
--- a/README.md
+++ b/README.md
@@ -249,24 +249,24 @@ export OPENAI_BASE_URL="http://localhost:4000/v1"
## Routing Algorithms
-Most use an LLM as a judge. All of them pick between an **efficient** model and a
-**capable** one; what differs is when the decision is made and how.
+Start with **Auto**. Choose Task or Execution when you want more control over
+how requests move between an efficient model and a capable one.
-| Algorithm | How it decides | Route `type` | Benchmark |
-|---|---|---|---|
-| **[Capability](docs/routing_algorithms/llm_classifier_routing.md)** | The first request is judged by an LLM. | `llm_classifier` | 71.2% at $79.32 |
-| **[Stage](docs/routing_algorithms/stage_router_routing.md)** | Tool responses are judged by pattern matching or an LLM. | `stage_router` | 72.7% at $68.19 |
-| **[Capability + Stage](docs/routing_algorithms/composite_routing.md)** | Combines the two above. | `composite` | not yet benchmarked |
-| **[Escalation](docs/routing_algorithms/escalation_router_routing.md)** | Starts efficient. Responses are judged by an LLM for issues, then escalated. | `llm_classifier` + `mode = "escalation"` | 75.7% at $85.00 |
-| **[Advisor Gate](docs/routing_algorithms/advisor_gate_routing.md)** | One model serves every turn; a stronger advisor approves its plans and "done" claims, or sends it back. | `advisor` | lifts a weak executor 43.8% → 54.7% |
-| **[Sub-Agent-Aware](docs/routing_algorithms/subagent_routing.md)** | Delegated sub-agent traffic routes separately from the parent agent. | `subagents` on `passthrough` or `stage_router` | not yet benchmarked |
-| **[Custom](docs/routing_algorithms/llm_classifier_routing.md#custom-multi-target-routing)** | The first request is judged by an LLM against criteria you define, routing among 2+ of your own models. | `llm_classifier` + `target_selector` policy | not yet benchmarked |
-| **[Random](docs/routing_algorithms/random_routing.md)** | Each request is routed at random, uniform or weighted. | `random` | baseline mechanism |
-
-Benchmarks are Terminal-Bench 2.1 against a $98.06 Opus 4.8 baseline at 76.0%.
-A `passthrough` route registers one target under one model ID with no routing
-decision. See the [Routing Overview](docs/routing_algorithms/overview.md) for
-the common route shape and self-hosted targets.
+| Choice | Use it when | Route `type` |
+|---|---|---|
+| **[Auto](docs/routing_algorithms/overview.md#auto)** | You want Switchyard's recommended preset. | `auto` |
+| **[Task](docs/routing_algorithms/llm_classifier_routing.md)** | You want an LLM to judge which model can handle the task. | `llm_classifier` |
+| **[Execution](docs/routing_algorithms/stage_router_routing.md)** | You want tool results and agent progress to guide each request. | `stage_router` |
+
+Auto currently uses Execution with fixed defaults and no LLM judge. Task uses
+the LLM classifier's capability mode. Execution uses the stage router.
+These names do not change the TOML configuration keys.
+
+> Auto requires a [source build](docs/getting_started.md#build-from-source) until v0.3.0 is published. The quickstart above uses `stage_router` directly.
+
+See the [full routing catalog](docs/routing_algorithms/overview.md#more-options)
+for composite routing, escalation, custom policies, and other options.
+Performance results are listed under [Benchmark Provenance](#benchmark-provenance).
## Documentation
@@ -285,9 +285,9 @@ the common route shape and self-hosted targets.
| Configuration | Accuracy | Total cost | vs. Opus 4.8 baseline |
|---|---:|---:|---|
| Opus 4.8 baseline | 76.0% | $98.06 | — |
-| **[Escalation](#routing-algorithms)** | 75.7% | $85.00 | 99.6% of accuracy, 13.3% cheaper |
-| **[Stage](#routing-algorithms)** | 72.7% | $68.19 | 95.7% of accuracy, 30.5% cheaper |
-| **[Capability](#routing-algorithms)** | 71.2% | $79.32 | 93.7% of accuracy, 19.1% cheaper |
+| **[Escalation](docs/routing_algorithms/escalation_router_routing.md)** | 75.7% | $85.00 | 99.6% of accuracy, 13.3% cheaper |
+| **[Execution (Stage)](docs/routing_algorithms/stage_router_routing.md)** | 72.7% | $68.19 | 95.7% of accuracy, 30.5% cheaper |
+| **[Task (Capability)](docs/routing_algorithms/llm_classifier_routing.md)** | 71.2% | $79.32 | 93.7% of accuracy, 19.1% cheaper |
| Kimi K2.6 alone | 55.8% | $76.28 | |
| GLM 5.2 alone | 52.4% | $16.47 | |
| DeepSeek V4 Pro alone | 48.7% | $96.92 | |
diff --git a/docs/getting_started.md b/docs/getting_started.md
index 965312914..66e1106ba 100644
--- a/docs/getting_started.md
+++ b/docs/getting_started.md
@@ -145,15 +145,14 @@ curl http://localhost:4000/v1/chat/completions \
#### Choose a route type
-This guide uses `auto`, which routes with Switchyard's recommended default
-settings. The Rust server also supports:
+Start with **Auto**, as shown above. Choose Task or Execution when you want
+more control over how requests are routed.
-| Algorithm | Use it when | Config |
+| Choice | Use it when | Route `type` |
|---|---|---|
-| Auto | You want a recommended default instead of picking a strategy yourself. | `auto` |
-| [Random](routing_algorithms/random_routing.md) | You need a weighted split for A/B tests or baselines. | `random` |
-| [LLM classifier](routing_algorithms/llm_classifier_routing.md) | Request content should decide whether to use the weak or strong target. | `llm_classifier` |
-| [Stage router](routing_algorithms/stage_router_routing.md) | Built-in or configured tool-activity signals should select an efficient or capable target. | `stage_router` |
+| **[Auto](routing_algorithms/overview.md#auto)** | You want Switchyard's recommended preset. | `auto` |
+| **[Task](routing_algorithms/llm_classifier_routing.md)** | You want an LLM to judge which model can handle the task. | `llm_classifier` |
+| **[Execution](routing_algorithms/stage_router_routing.md)** | You want tool results and agent progress to guide each request. | `stage_router` |
A single TOML file can declare multiple routes. The table key, such as
`routes.smart`, is a local configuration name; each route's `id` is exposed as a
diff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.md
index f7839fa0f..c1b1af8af 100644
--- a/docs/routing_algorithms/llm_classifier_routing.md
+++ b/docs/routing_algorithms/llm_classifier_routing.md
@@ -1,7 +1,11 @@
# LLM Classifier Routing
-LLM classifier routing supports capability classification, trajectory escalation,
-and custom schema-driven routing across two or more targets.
+**Task** routing uses the LLM classifier's `capability` mode to judge whether
+an efficient model can handle the task or a capable model is needed. Configure
+it with `type = "llm_classifier"` and `mode = "capability"`.
+
+The same classifier also supports [escalation](escalation_router_routing.md)
+and [custom routing](#custom-multi-target-routing) across two or more targets.
## Configure a classifier route
diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.md
index 074b3c151..fa26620c5 100644
--- a/docs/routing_algorithms/overview.md
+++ b/docs/routing_algorithms/overview.md
@@ -10,16 +10,51 @@ configuration and tuning. For the vocabulary these pages use, see
## Choose a strategy
+Start with **Auto**. Choose Task or Execution when you want more control over
+how requests move between an efficient model and a capable one.
+
+| Choice | Use it when | Route `type` |
+|---|---|---|
+| **[Auto](#auto)** | You want Switchyard's recommended preset. | `auto` |
+| **[Task](llm_classifier_routing.md)** | You want an LLM to judge which model can handle the task. | `llm_classifier` |
+| **[Execution](stage_router_routing.md)** | You want tool results and agent progress to guide each request. | `stage_router` |
+
+Task uses the LLM classifier's `capability` mode. Its `classify_trigger` setting
+controls whether the judge runs for each request, each user turn, or once per
+session. Execution uses the stage router, which reads tool-result history and
+can optionally call an LLM judge. The TOML configuration keys are unchanged.
+
+### Auto
+
+Auto is a preset, not a separate routing algorithm. It currently uses Execution
+with `picker = "efficient_first"`, `confidence_threshold = 0.5`, and no LLM
+judge. It does not compare strategies at runtime. Use `stage_router` directly
+to tune these settings.
+
+> Auto requires a [source build](../getting_started.md#build-from-source) until v0.3.0 is published.
+
+See the [Auto configuration reference](../reference/toml_schema.md#auto) for
+the required targets.
+
+## More options
+
+These options remain available when you need a different routing policy.
+
| Strategy | Use it when | Route `type` |
|---|---|---|
+| [Composite](composite_routing.md) | Combine Task and Execution. A classifier sets the stage router's default tier. | `composite` |
+| [Escalation](escalation_router_routing.md) | Start on the efficient model and escalate when an LLM judge detects trouble. | `llm_classifier` with `mode = "escalation"` |
+| [Custom](llm_classifier_routing.md#custom-multi-target-routing) | Route among two or more models using your own classification schema and rules. | `llm_classifier` with `mode = "custom"` |
+| [Advisor Gate](advisor_gate_routing.md) | Keep one executor model and have a stronger advisor review its plans and completion claims. | `advisor` |
| [Sub-Agent-Aware Routing](subagent_routing.md) | Delegated sub-agents should use a separate routing policy from the parent agent. | `passthrough` or `stage_router` with `subagents` |
| [Random Routing](random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |
-| [LLM Classifier Routing](llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` |
-| [Stage-Router Routing](stage_router_routing.md) | Built-in or configured tool-activity signals should route most turns without an extra classifier call. | `stage_router` |
-| Auto Routing | You want a recommended default instead of picking a strategy yourself. For a deeper dive on the current default, see [Stage-Router Routing](stage_router_routing.md); for full control, pick one of the strategies above instead. | `auto` |
-| [Composite Routing](composite_routing.md) | Routing algorithms are composed, one setting the configuration of another before handing off. Today an LLM classifier sets a stage router's default tier. | `composite` |
-| [Escalation-Router Routing](escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` |
-| [Advisor-Gate Routing](advisor_gate_routing.md) | One model should serve every turn, with a stronger reviewer approving its "done" claims or sending back a redo plan. | `advisor` |
+| [Fixed Model](#direct-model-routes) | Send every request to one target without a routing decision. | `passthrough` |
+
+### Experimental
+
+[Prefill Router](../reference/toml_schema.md#prefill_router) uses a
+checkpoint-backed classifier. It is experimental in v0.3.0. Switchyard does not
+provide or support a checkpoint, exporter, or compatible encoder assets.
## Common route shape
diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.md
index 59489c166..7882b1258 100644
--- a/docs/routing_algorithms/stage_router_routing.md
+++ b/docs/routing_algorithms/stage_router_routing.md
@@ -1,5 +1,8 @@
# Stage-Router Routing
+**Execution** routing uses the stage router to choose a model from tool results
+and agent progress. Configure it with `type = "stage_router"`.
+
Stage-router routing sends each request to either a **capable** model or a
cheaper **efficient** one, depending on where the agent is in its run. The goal
is to spend the capable model on the turns that need it (exploration, error
diff --git a/mkdocs.yml b/mkdocs.yml
index 3a9565fb5..bcd196402 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -27,10 +27,10 @@ nav:
- NeMo Relay: integrations/nemo_relay.md
- Routing:
- Overview: routing_algorithms/overview.md
+ - Task (LLM Classifier): routing_algorithms/llm_classifier_routing.md
+ - Execution (Stage Router): routing_algorithms/stage_router_routing.md
- Sub-Agent-Aware Routing: routing_algorithms/subagent_routing.md
- Random Routing: routing_algorithms/random_routing.md
- - LLM Classifier Routing: routing_algorithms/llm_classifier_routing.md
- - Stage-Router Routing: routing_algorithms/stage_router_routing.md
- Composite Routing: routing_algorithms/composite_routing.md
- Escalation-Router Routing: routing_algorithms/escalation_router_routing.md
- Advisor-Gate Routing: routing_algorithms/advisor_gate_routing.md