Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs: simplify routing choices (#763)
ryan-lempka4645dad0f61e
diff --git a/README.md b/README.mdindex f94d3a077..a30261864 100644--- a/README.md+++ b/README.md@@ -249,24 +249,24 @@ export OPENAI_BASE_URL="http://localhost:4000/v1"## Routing Algorithms-Most use an LLM as a judge. All of them pick between an **efficient** model and a-**capable** one; what differs is when the decision is made and how.+Start with **Auto**. Choose Task or Execution when you want more control over+how requests move between an efficient model and a capable one.-| Algorithm | How it decides | Route `type` | Benchmark |-|---|---|---|---|-| **[Capability](docs/routing_algorithms/llm_classifier_routing.md)** | The first request is judged by an LLM. | `llm_classifier` | 71.2% at $79.32 |-| **[Stage](docs/routing_algorithms/stage_router_routing.md)** | Tool responses are judged by pattern matching or an LLM. | `stage_router` | 72.7% at $68.19 |-| **[Capability + Stage](docs/routing_algorithms/composite_routing.md)** | Combines the two above. | `composite` | not yet benchmarked |-| **[Escalation](docs/routing_algorithms/escalation_router_routing.md)** | Starts efficient. Responses are judged by an LLM for issues, then escalated. | `llm_classifier` + `mode = "escalation"` | 75.7% at $85.00 |-| **[Advisor Gate](docs/routing_algorithms/advisor_gate_routing.md)** | One model serves every turn; a stronger advisor approves its plans and "done" claims, or sends it back. | `advisor` | lifts a weak executor 43.8% → 54.7% |-| **[Sub-Agent-Aware](docs/routing_algorithms/subagent_routing.md)** | Delegated sub-agent traffic routes separately from the parent agent. | `subagents` on `passthrough` or `stage_router` | not yet benchmarked |-| **[Custom](docs/routing_algorithms/llm_classifier_routing.md#custom-multi-target-routing)** | The first request is judged by an LLM against criteria you define, routing among 2+ of your own models. | `llm_classifier` + `target_selector` policy | not yet benchmarked |-| **[Random](docs/routing_algorithms/random_routing.md)** | Each request is routed at random, uniform or weighted. | `random` | baseline mechanism |--Benchmarks are Terminal-Bench 2.1 against a $98.06 Opus 4.8 baseline at 76.0%.-A `passthrough` route registers one target under one model ID with no routing-decision. See the [Routing Overview](docs/routing_algorithms/overview.md) for-the common route shape and self-hosted targets.+| Choice | Use it when | Route `type` |+|---|---|---|+| **[Auto](docs/routing_algorithms/overview.md#auto)** | You want Switchyard's recommended preset. | `auto` |+| **[Task](docs/routing_algorithms/llm_classifier_routing.md)** | You want an LLM to judge which model can handle the task. | `llm_classifier` |+| **[Execution](docs/routing_algorithms/stage_router_routing.md)** | You want tool results and agent progress to guide each request. | `stage_router` |++Auto currently uses Execution with fixed defaults and no LLM judge. Task uses+the LLM classifier's capability mode. Execution uses the stage router.+These names do not change the TOML configuration keys.++> Auto requires a [source build](docs/getting_started.md#build-from-source) until v0.3.0 is published. The quickstart above uses `stage_router` directly.++See the [full routing catalog](docs/routing_algorithms/overview.md#more-options)+for composite routing, escalation, custom policies, and other options.+Performance results are listed under [Benchmark Provenance](#benchmark-provenance).## Documentation@@ -285,9 +285,9 @@ the common route shape and self-hosted targets.| Configuration | Accuracy | Total cost | vs. Opus 4.8 baseline ||---|---:|---:|---|| Opus 4.8 baseline | 76.0% | $98.06 | — |-| **[Escalation](#routing-algorithms)** | 75.7% | $85.00 | 99.6% of accuracy, 13.3% cheaper |-| **[Stage](#routing-algorithms)** | 72.7% | $68.19 | 95.7% of accuracy, 30.5% cheaper |-| **[Capability](#routing-algorithms)** | 71.2% | $79.32 | 93.7% of accuracy, 19.1% cheaper |+| **[Escalation](docs/routing_algorithms/escalation_router_routing.md)** | 75.7% | $85.00 | 99.6% of accuracy, 13.3% cheaper |+| **[Execution (Stage)](docs/routing_algorithms/stage_router_routing.md)** | 72.7% | $68.19 | 95.7% of accuracy, 30.5% cheaper |+| **[Task (Capability)](docs/routing_algorithms/llm_classifier_routing.md)** | 71.2% | $79.32 | 93.7% of accuracy, 19.1% cheaper || Kimi K2.6 alone | 55.8% | $76.28 | || GLM 5.2 alone | 52.4% | $16.47 | || DeepSeek V4 Pro alone | 48.7% | $96.92 | |diff --git a/docs/getting_started.md b/docs/getting_started.mdindex 965312914..66e1106ba 100644--- a/docs/getting_started.md+++ b/docs/getting_started.md@@ -145,15 +145,14 @@ curl http://localhost:4000/v1/chat/completions \#### Choose a route type-This guide uses `auto`, which routes with Switchyard's recommended default-settings. The Rust server also supports:+Start with **Auto**, as shown above. Choose Task or Execution when you want+more control over how requests are routed.-| Algorithm | Use it when | Config |+| Choice | Use it when | Route `type` ||---|---|---|-| Auto | You want a recommended default instead of picking a strategy yourself. | `auto` |-| [Random](routing_algorithms/random_routing.md) | You need a weighted split for A/B tests or baselines. | `random` |-| [LLM classifier](routing_algorithms/llm_classifier_routing.md) | Request content should decide whether to use the weak or strong target. | `llm_classifier` |-| [Stage router](routing_algorithms/stage_router_routing.md) | Built-in or configured tool-activity signals should select an efficient or capable target. | `stage_router` |+| **[Auto](routing_algorithms/overview.md#auto)** | You want Switchyard's recommended preset. | `auto` |+| **[Task](routing_algorithms/llm_classifier_routing.md)** | You want an LLM to judge which model can handle the task. | `llm_classifier` |+| **[Execution](routing_algorithms/stage_router_routing.md)** | You want tool results and agent progress to guide each request. | `stage_router` |A single TOML file can declare multiple routes. The table key, such as`routes.smart`, is a local configuration name; each route's `id` is exposed as adiff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.mdindex f7839fa0f..c1b1af8af 100644--- a/docs/routing_algorithms/llm_classifier_routing.md+++ b/docs/routing_algorithms/llm_classifier_routing.md@@ -1,7 +1,11 @@# LLM Classifier Routing-LLM classifier routing supports capability classification, trajectory escalation,-and custom schema-driven routing across two or more targets.+**Task** routing uses the LLM classifier's `capability` mode to judge whether+an efficient model can handle the task or a capable model is needed. Configure+it with `type = "llm_classifier"` and `mode = "capability"`.++The same classifier also supports [escalation](escalation_router_routing.md)+and [custom routing](#custom-multi-target-routing) across two or more targets.## Configure a classifier routediff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.mdindex 074b3c151..fa26620c5 100644--- a/docs/routing_algorithms/overview.md+++ b/docs/routing_algorithms/overview.md@@ -10,16 +10,51 @@ configuration and tuning. For the vocabulary these pages use, see## Choose a strategy+Start with **Auto**. Choose Task or Execution when you want more control over+how requests move between an efficient model and a capable one.++| Choice | Use it when | Route `type` |+|---|---|---|+| **[Auto](#auto)** | You want Switchyard's recommended preset. | `auto` |+| **[Task](llm_classifier_routing.md)** | You want an LLM to judge which model can handle the task. | `llm_classifier` |+| **[Execution](stage_router_routing.md)** | You want tool results and agent progress to guide each request. | `stage_router` |++Task uses the LLM classifier's `capability` mode. Its `classify_trigger` setting+controls whether the judge runs for each request, each user turn, or once per+session. Execution uses the stage router, which reads tool-result history and+can optionally call an LLM judge. The TOML configuration keys are unchanged.++### Auto++Auto is a preset, not a separate routing algorithm. It currently uses Execution+with `picker = "efficient_first"`, `confidence_threshold = 0.5`, and no LLM+judge. It does not compare strategies at runtime. Use `stage_router` directly+to tune these settings.++> Auto requires a [source build](../getting_started.md#build-from-source) until v0.3.0 is published.++See the [Auto configuration reference](../reference/toml_schema.md#auto) for+the required targets.++## More options++These options remain available when you need a different routing policy.+| Strategy | Use it when | Route `type` ||---|---|---|+| [Composite](composite_routing.md) | Combine Task and Execution. A classifier sets the stage router's default tier. | `composite` |+| [Escalation](escalation_router_routing.md) | Start on the efficient model and escalate when an LLM judge detects trouble. | `llm_classifier` with `mode = "escalation"` |+| [Custom](llm_classifier_routing.md#custom-multi-target-routing) | Route among two or more models using your own classification schema and rules. | `llm_classifier` with `mode = "custom"` |+| [Advisor Gate](advisor_gate_routing.md) | Keep one executor model and have a stronger advisor review its plans and completion claims. | `advisor` || [Sub-Agent-Aware Routing](subagent_routing.md) | Delegated sub-agents should use a separate routing policy from the parent agent. | `passthrough` or `stage_router` with `subagents` || [Random Routing](random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |-| [LLM Classifier Routing](llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` |-| [Stage-Router Routing](stage_router_routing.md) | Built-in or configured tool-activity signals should route most turns without an extra classifier call. | `stage_router` |-| Auto Routing | You want a recommended default instead of picking a strategy yourself. For a deeper dive on the current default, see [Stage-Router Routing](stage_router_routing.md); for full control, pick one of the strategies above instead. | `auto` |-| [Composite Routing](composite_routing.md) | Routing algorithms are composed, one setting the configuration of another before handing off. Today an LLM classifier sets a stage router's default tier. | `composite` |-| [Escalation-Router Routing](escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` |-| [Advisor-Gate Routing](advisor_gate_routing.md) | One model should serve every turn, with a stronger reviewer approving its "done" claims or sending back a redo plan. | `advisor` |+| [Fixed Model](#direct-model-routes) | Send every request to one target without a routing decision. | `passthrough` |++### Experimental++[Prefill Router](../reference/toml_schema.md#prefill_router) uses a+checkpoint-backed classifier. It is experimental in v0.3.0. Switchyard does not+provide or support a checkpoint, exporter, or compatible encoder assets.## Common route shapediff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.mdindex 59489c166..7882b1258 100644--- a/docs/routing_algorithms/stage_router_routing.md+++ b/docs/routing_algorithms/stage_router_routing.md@@ -1,5 +1,8 @@# Stage-Router Routing+**Execution** routing uses the stage router to choose a model from tool results+and agent progress. Configure it with `type = "stage_router"`.+Stage-router routing sends each request to either a **capable** model or acheaper **efficient** one, depending on where the agent is in its run. The goalis to spend the capable model on the turns that need it (exploration, errordiff --git a/mkdocs.yml b/mkdocs.ymlindex 3a9565fb5..bcd196402 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -27,10 +27,10 @@ nav:- NeMo Relay: integrations/nemo_relay.md- Routing:- Overview: routing_algorithms/overview.md+ - Task (LLM Classifier): routing_algorithms/llm_classifier_routing.md+ - Execution (Stage Router): routing_algorithms/stage_router_routing.md- Sub-Agent-Aware Routing: routing_algorithms/subagent_routing.md- Random Routing: routing_algorithms/random_routing.md- - LLM Classifier Routing: routing_algorithms/llm_classifier_routing.md- - Stage-Router Routing: routing_algorithms/stage_router_routing.md- Composite Routing: routing_algorithms/composite_routing.md- Escalation-Router Routing: routing_algorithms/escalation_router_routing.md- Advisor-Gate Routing: routing_algorithms/advisor_gate_routing.md