Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

docs(routing): document model selection and routing checks (#998)

eric-liu-nvidia6f4e7749083a
diff --git a/README.md b/README.md
index 67291e800..f1f64d5db 100644
--- a/README.md
+++ b/README.md
@@ -35,7 +35,7 @@ LiteLLM and NeMo Relay run in your own deployment. Follow each gateway's guidanc
### Try routing locally
-[Try routing locally](docs/getting_started.md#server-path) with the standalone proxy for demos and evaluation.
+[Try routing locally](docs/getting_started.md#server-path) with the standalone proxy for demos and evaluation, then [verify routing is active](docs/getting_started.md#verify-routing-is-active).
Agent-specific guides are available for [pi](docs/integrations/pi.md) and
[Oh My Pi](docs/integrations/oh_my_pi.md).
diff --git a/docs/getting_started.md b/docs/getting_started.md
index 41c1a0d23..7d5d91cff 100644
--- a/docs/getting_started.md
+++ b/docs/getting_started.md
@@ -102,6 +102,9 @@ capable_target = "strong"
efficient_target = "weak"
```
+The two targets are placeholders. See [Model Selection](routing_algorithms/model_selection.md)
+to choose models for the capable and efficient tiers.
+
`format` selects the upstream protocol and must be `openai_chat`,
`openai_responses`, or `anthropic_messages`. `api_key_env` names the environment
variable the server reads; the secret does not belong in the TOML file.
@@ -146,6 +149,100 @@ curl http://localhost:4000/v1/chat/completions \
-d '{"model":"switchyard","messages":[{"role":"user","content":"hello"}]}'
```
+### Verify routing is active
+
+The first request usually lands on the efficient model. That does not mean
+routing is off. Auto decides from tool results, and a request without tool
+history has none, so the efficient tier serves it by design. Use these checks
+to see routing work. Your model IDs and token counts will differ from the
+examples.
+
+1. **Confirm the configuration loads.** The dry run prints the route IDs the
+ server will serve:
+
+ ```bash
+ switchyard-server --config routes.toml --dry-run
+ ```
+
+ ```text
+ server OK: switchyard
+ ```
+
+2. **Read the served model from the response header.** Every response carries
+ `x-model-router-selected-model`. A chat-only request is served by the
+ efficient model:
+
+ ```bash
+ curl -si http://localhost:4000/v1/chat/completions \
+ -H "Content-Type: application/json" \
+ -d '{"model":"switchyard","messages":[{"role":"user","content":"hello"}]}' \
+ | grep -i x-model-router-selected-model
+ ```
+
+ ```text
+ x-model-router-selected-model: openai/gpt-4o-mini
+ ```
+
+3. **Send a request that escalates.** A tool result that reports a critical
+ error forces the capable tier. Save this request as `escalate.json`:
+
+ ```json
+ {
+ "model": "switchyard",
+ "messages": [
+ {"role": "user", "content": "Run the test suite."},
+ {"role": "assistant", "content": null, "tool_calls": [
+ {"id": "call_1", "type": "function",
+ "function": {"name": "bash", "arguments": "{\"command\":\"pytest -q\"}"}}
+ ]},
+ {"role": "tool", "tool_call_id": "call_1",
+ "content": "MemoryError: cannot allocate memory\nKilled"},
+ {"role": "user", "content": "Fix it."}
+ ]
+ }
+ ```
+
+ ```bash
+ curl -si http://localhost:4000/v1/chat/completions \
+ -H "Content-Type: application/json" -d @escalate.json \
+ | grep -i x-model-router-selected-model
+ ```
+
+ ```text
+ x-model-router-selected-model: openai/gpt-4o
+ ```
+
+ The same hard failure appearing twice in recent tool results, or a
+ context-compaction marker, also forces the capable tier. See
+ [decision sources](routing_algorithms/stage_router_routing.md#decision-sources).
+
+4. **Count decisions by source.** `GET /v1/stats` reports calls per model and
+ routing decisions by source:
+
+ ```bash
+ curl -s http://localhost:4000/v1/stats | jq -c '.models | map_values(.calls)'
+ curl -s http://localhost:4000/v1/stats | jq -c '.algorithm_stats.stage_router.routing_decisions'
+ ```
+
+ ```json
+ {"openai/gpt-4o":1,"openai/gpt-4o-mini":1}
+ {"fall_open":{"total":1,"targets":{"openai/gpt-4o-mini":1}},"override":{"total":1,"targets":{"openai/gpt-4o":1}}}
+ ```
+
+ `fall_open` counts turns with no usable signals that landed on the picker's
+ default tier. `override` counts turns a hard error forced to the capable
+ tier. Counts reset when the server restarts or on `POST /v1/stats/reset`.
+
+5. **Keep a record of every request.** Start the server with
+ `--routing-log-file routing.jsonl` to append one JSON line per served request:
+
+ ```json
+ {"ts":"2026-10-15T06:16:22.643Z","route_id":"switchyard","algorithm":"stage_router","origin":null,"task":null,"trial_id":null,"session_id":null,"model":"openai/gpt-4o","tier":"","prompt_tokens":42,"cached_tokens":0,"cache_creation_tokens":0,"completion_tokens":5,"reasoning_tokens":0,"total_tokens":47}
+ ```
+
+ The record names the served model and its token counts. It does not record
+ why that model was chosen; use `GET /v1/stats` for the decision counts.
+
### Routing algorithms
#### Choose a route type
diff --git a/docs/integrations/pi.md b/docs/integrations/pi.md
index 8560fa2b5..75a2853b3 100644
--- a/docs/integrations/pi.md
+++ b/docs/integrations/pi.md
@@ -68,7 +68,9 @@ pi -p --provider switchyard --model switchyard "List the files in this directory
## Check the routing
The command `curl -s localhost:4000/v1/stats | jq '.models | map_values({calls, prompt_tokens})'`
-lists each target model with its call count. Every response carries the header
+lists each target model with its call count. For a step-by-step check with an
+escalating request, see
+[Verify routing is active](../getting_started.md#verify-routing-is-active). Every response carries the header
`x-model-router-selected-model`. With `--routing-log-file PATH`, the server writes one
record per call. In the record below, `session_id` is the value of pi's `x-session-id`
header:
diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.md
index b3160a4c7..b70ba9531 100644
--- a/docs/reference/toml_schema.md
+++ b/docs/reference/toml_schema.md
@@ -376,19 +376,44 @@ optional `handoff_notes` and `classifier` tables and for tuning.
### `auto`
-Uses Switchyard's recommended default routing strategy instead of one you pick
-yourself: a `stage_router` preset with `picker = "efficient_first"` and
-`confidence_threshold = 0.5`, no classifier. See
-[Stage-Router Routing](../routing_algorithms/stage_router_routing.md) for a
-deeper dive on the current default, or the
-[strategy table](../routing_algorithms/overview.md#choose-a-strategy) to pick
-one manually.
+`auto` is a preset. It builds a `stage_router` route with fixed settings and no
+classifier. Use it when you want the recommended starting point without tuning.
+See [Auto](../routing_algorithms/overview.md#auto) for what the preset does and
+when it does not route.
| Key | Required | Default | Meaning |
|---|:---:|---|---|
| `capable_target` | Yes | — | Capable tier. |
| `efficient_target` | Yes | — | Efficient tier. |
+`auto` accepts no other keys. Adding one is a configuration error. For example,
+adding `confidence_threshold = 0.5` to an `auto` route fails `--dry-run` with:
+
+```text
+unknown field `confidence_threshold`, expected `capable_target` or `efficient_target`
+```
+
+An `auto` route behaves the same as this `stage_router` route. It has no
+classifier, no handoff notes, and no per-tier system prompts:
+
+```toml
+[routes.smart]
+id = "switchyard"
+type = "stage_router"
+capable_target = "strong"
+efficient_target = "weak"
+picker = "efficient_first"
+confidence_threshold = 0.5
+recent_turn_window = 3
+capable_hold_turns = 2
+```
+
+To tune the preset, replace the `auto` route with this block and edit the keys.
+The [stage_router](#stage_router) table lists every key. See
+[Stage-Router Routing](../routing_algorithms/stage_router_routing.md) for
+tuning guidance and the [model selection guide](../routing_algorithms/model_selection.md)
+for choosing the two targets.
+
### `composite`
Composes other algorithms, letting one set another's
diff --git a/docs/routing_algorithms/escalation_router_routing.md b/docs/routing_algorithms/escalation_router_routing.md
index 994359a08..5a1209780 100644
--- a/docs/routing_algorithms/escalation_router_routing.md
+++ b/docs/routing_algorithms/escalation_router_routing.md
@@ -252,6 +252,8 @@ retained nor logged.
## Related
- [Routing Overview](overview.md): compare all supported routing strategies.
+- [Model Selection](model_selection.md): choose the weak, strong, and judge
+ models, with measured pairs.
- [LLM Classifier Routing](llm_classifier_routing.md): pick a tier up front
instead of judging the run.
- [Architecture](../architecture.md): the end-to-end request lifecycle and system
diff --git a/docs/routing_algorithms/model_selection.md b/docs/routing_algorithms/model_selection.md
new file mode 100644
index 000000000..bb4c83d5c
--- /dev/null
+++ b/docs/routing_algorithms/model_selection.md
@@ -0,0 +1,306 @@
+# Model Selection
+
+Switchyard routes each request to one of the models you configure. The models
+you put in each role change results more than any threshold does. This page says
+what each role needs, how to check a model before you route to it, which pairs
+have measured results, and the mistakes we have seen in real runs.
+
+## Roles
+
+Each strategy names its models by role. The TOML keys differ by route type:
+
+| Route `type` | Serves most turns | Takes the hard turns | Decides |
+|---|---|---|---|
+| `auto`, `stage_router`, `composite` | `efficient_target` | `capable_target` | Tool-result signals. `composite` adds a classifier. |
+| `plan_execute` | `efficient_target`, after the first edit | `capable_target`, while planning | The first file edit. |
+| `llm_classifier` (capability, escalation) | `weak_target` | `strong_target` | The judge at `classifier_target`. |
+| `llm_classifier` (custom) | Your target names | Your target names | The judge in `models.judge`. |
+| `advisor` | `executor_target` | `advisor_target` reviews. It does not serve turns. | The gate rules. |
+
+A role is not a property of a model. The same model can be the capable tier in
+one route and the efficient tier in another. See
+[Core Concepts](../core_concepts.md#routing-algorithms).
+
+What each role needs:
+
+- **Capable, strong, or advisor.** Your best model on this workload. Every
+ escalation lands here. Routing cannot solve a task this model cannot solve.
+- **Efficient, weak, or executor.** Serves most turns. It must call the agent's
+ tools correctly, follow the output format, hold the agent's context, and still
+ complete tasks. Judge it by the tasks it completes on your workload, not by its
+ price.
+- **Judge or classifier.** Returns schema-valid JSON on every call, and does so
+ quickly. See
+ [Judge model compatibility](llm_classifier_routing.md#judge-model-compatibility).
+
+## Check the efficient model before you route
+
+Run these checks before you measure routing. Add a `passthrough` route for the
+efficient model to the file from
+[Getting Started](../getting_started.md#server-path), start the server with
+`--routing-log-file routing.jsonl`, and send the checks to that route:
+
+```toml
+[routes.efficient_only]
+id = "efficient-only"
+type = "passthrough"
+target = "weak"
+```
+
+### 1. Tool calling
+
+Send a request with one tool and read the call back:
+
+```bash
+curl -s http://localhost:4000/v1/chat/completions \
+ -H "Content-Type: application/json" \
+ -d '{
+ "model": "efficient-only",
+ "messages": [{"role": "user", "content": "What is the weather in Paris? Use the tool."}],
+ "tools": [{"type": "function", "function": {
+ "name": "get_weather",
+ "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
+ }' | jq '.choices[0].message.tool_calls[0].function.arguments | fromjson'
+```
+
+Pass: the output is a JSON object with a `city` key. Fail: `null`, or a
+`fromjson` error. Then run your agent against this route on one short task.
+Models that pass with one tool can produce malformed calls once the agent sends
+its full tool list. The agent reports those as invalid tool calls, or it repeats
+the same step.
+
+### 2. Structured output
+
+Judges, and some agents, need JSON that matches a schema:
+
+```bash
+curl -s http://localhost:4000/v1/chat/completions \
+ -H "Content-Type: application/json" \
+ -d '{
+ "model": "efficient-only",
+ "messages": [{"role": "user", "content": "Is 7 a prime number?"}],
+ "response_format": {"type": "json_schema", "json_schema": {"name": "verdict", "strict": true,
+ "schema": {"type": "object", "properties": {"prime": {"type": "boolean"}},
+ "required": ["prime"], "additionalProperties": false}}}
+ }' | jq '.choices[0].message.content | fromjson'
+```
+
+Pass: `{"prime": true}`. An HTTP 400 means the provider does not offer JSON
+Schema mode for this model. For a judge, set
+`response_format_type = "json_object"` on the `llm_classifier` route. See
+[Judge model compatibility](llm_classifier_routing.md#judge-model-compatibility).
+
+### 3. Context window
+
+The efficient model must hold the agent's largest prompt. Find that size after
+a run through the passthrough route:
+
+```bash
+jq -s 'map(.prompt_tokens) | max' routing.jsonl
+```
+
+Compare it with the context length your provider lists for the model.
+OpenRouter reports it as `context_length` in `GET https://openrouter.ai/api/v1/models`.
+Set the route's `context_window` to the smaller of the two tiers' limits so
+clients that read it compact in time. The server does not enforce it. See the
+[route keys](../reference/toml_schema.md#routesname).
+
+### 4. Reasoning
+
+A model that reasons at length on every turn is slow and costly in the
+efficient role. Read the reasoning share from the log:
+
+```bash
+jq -c '{model, completion_tokens, reasoning_tokens}' routing.jsonl
+```
+
+Providers that do not report reasoning tokens leave the field at `0`. When
+reasoning tokens dominate on simple turns, set `reasoning_effort` on the target
+for `openai_chat` and `openai_responses` clients, or use the provider's own
+control through `extra_body`. See [Reasoning effort](#reasoning-effort).
+
+### 5. Wire format
+
+`format` on the LLM client must match the endpoint. `--dry-run` checks the file
+only. It does not contact the provider. Send one request to the passthrough
+route and confirm an HTTP 200 with a `choices` array. The same model can also
+behave differently on different endpoints. See
+[Pin the provider and endpoint](#pin-the-provider-and-endpoint).
+
+## Measured pairs
+
+These pairs have results from checked-in profiles or a public report. Each row
+describes one pair on one workload. The thresholds and results do not transfer
+to another pair or another workload. The profile headers give the serving stack,
+agent version, and scoring rule for each run, and absolute results vary by
+serving stack.
+
+| Efficient tier | Capable tier | Strategy | Workload | Result | Single-model anchors |
+|---|---|---|---|---|---|
+| GPT-5.6 Luna | GPT-5.6 Sol | [`stage_router`](../../benchmark/routing-profiles/deepswe-v11-stage-router-luna-sol.toml): `efficient_first`, threshold 0.5, window 5, hold 2, handoff note | DeepSWE v1.1, 113 tasks, Codex CLI, k=1 | 76/113 solved (67.3% strict) | None in the same run |
+| GPT-5.6 Luna, executes | GPT-5.6 Sol, plans | [`plan_execute`](../../benchmark/routing-profiles/deepswe-v11-plan-execute-luna-sol.toml), both at effort `max` | DeepSWE v1.1, 113 tasks, Codex CLI, k=3 | 60.8% ± 5.3 at $180.51 ± 9.43 per run | Sol alone 62.8% ± 3.1 at $720.95. Luna alone 51.0% ± 3.7 at $50.02. |
+| GPT-5.6 Luna, executes | GPT-5.6 Sol, reviews | [`advisor`](../../benchmark/routing-profiles/deepswe-v11-advisor-gate-luna-sol.toml), both at effort `max` | DeepSWE v1.1, 113 tasks, Codex CLI, k=3 | 57.5% ± 3.2 at $208 ± 22 per run | Sol alone 62.8% ± 3.1 at $743. Luna alone 46.9% and 52.2% at about $49. |
+| Qwen3.6-35B-A3B, also the judge | Claude Opus 4.7 | `llm_classifier` custom, [balanced](../../benchmark/routing-profiles/tau2-telecom-custom-opus-qwen-balanced.toml) and [aggressive](../../benchmark/routing-profiles/tau2-telecom-custom-opus-qwen-aggressive.toml) | tau2-bench telecom, 114 tasks | 0.903 ± 0.071 solve rate with 45% of turns on the weak tier. 0.891 ± 0.029 with about 85%. | Within noise of the strong tier, per the profile headers |
+| Nemotron 3.5 Lightning | Claude Opus 4.8, with Gemini 3.1 Flash-Lite as judge | `llm_classifier` escalation, `confirmations = 2` | LangChain Deep Agents suite, 145 tasks, 5 runs ([report](https://www.langchain.com/blog/switchyard-agent-routing-benchmark)) | 80.0% at $3.00 per run, with 7% of calls on Opus | Opus alone 86.0% at $11.45. Nemotron alone 77.7% at $0.72. |
+
+What the rows show about model choice:
+
+- The gap between the two single models sets the room for routing. On DeepSWE,
+ Sol alone beat Luna alone by about 11 to 16 points. The plan/execute and
+ advisor arms landed between them at about a quarter of Sol's cost.
+- When the efficient model is close to the capable model, escalation keeps most
+ of the accuracy at a fraction of the cost. On the LangChain suite the two
+ single models were 8 points apart. Routing kept 80 of the 86 points at about
+ a quarter of the cost.
+- The judge's cost counts. In the LangChain runs the judge took 21.2% of spend.
+ Choose a judge that is small and fast, and include it in the cost per run.
+
+Other checked-in configurations:
+
+- [`tb-lite-llm-classifier-opus-kimi-gemini.toml`](../../benchmark/server-configs/tb-lite-llm-classifier-opus-kimi-gemini.toml)
+ (Claude Opus 4.7 strong, Kimi K2.7 code weak, Gemini 3.5 Flash judge) is a
+ smoke test. It has no published result.
+- [`tb21-escalation-opus-glm-deepseek.toml`](../../benchmark/routing-profiles/tb21-escalation-opus-glm-deepseek.toml)
+ (Claude Opus 4.8 strong, GLM 5.2 weak, DeepSeek V4 Flash judge) is the
+ escalation configuration used for the v0.2.0 Terminal-Bench 2.1 runs. Its
+ header links the post about those runs.
+
+## Counter-examples
+
+These cases came from users' runs. Each describes one pair on one workload.
+
+- **The efficient model was too weak.** A first stage-router run scored below
+ the single capable model. The efficient tier failed tasks the capable model
+ solves, and the signals escalated too late to recover them. Check: run the
+ [fixed arms](#check-for-an-inverted-pair) first. When the efficient-only arm
+ is far below the capable-only arm, routing has to escalate a large share of
+ turns to close the gap, and the savings shrink.
+- **The pair was inverted.** On one coding benchmark the model in the efficient
+ tier solved more tasks than the model in the capable tier. Every escalation
+ then cost more and solved less. Check:
+ [the inversion check](#check-for-an-inverted-pair).
+- **The tool list broke the efficient model.** A 4-billion-parameter local model
+ called tools correctly in a small test and produced malformed calls once the
+ agent sent its full list of 27 tools. Check: [tool calling](#1-tool-calling)
+ with the agent itself, not only with a one-tool request.
+- **Switching cost more than it saved.** A pair matched the single capable
+ model's accuracy but cost more. Each switch between models lost the prompt
+ cache, so the next turn paid full price for the whole prompt, and the
+ efficient model kept too little of the traffic to make up for it. Check:
+ `cached_tokens` in the routing log on the turn after a model change, and cost
+ per run for the routed arm against the fixed arms, not per-token prices.
+
+## Check for an inverted pair
+
+Symptom: the routed run scores below the efficient-only run, or `GET /v1/stats`
+shows most turns on the capable tier while results do not improve.
+
+Detection: run both models alone on the same tasks. Two `passthrough` routes in
+one file do this:
+
+```toml
+[routes.capable_only]
+id = "capable-only"
+type = "passthrough"
+target = "strong"
+
+[routes.efficient_only]
+id = "efficient-only"
+type = "passthrough"
+target = "weak"
+```
+
+Run your agent against each route ID on the same set of tasks, at least 20.
+Count solved tasks per arm. Read token use per arm from the routing log:
+
+```bash
+jq -s 'group_by(.route_id) | map({route: .[0].route_id, calls: length,
+ prompt_tokens: (map(.prompt_tokens) | add),
+ completion_tokens: (map(.completion_tokens) | add)})' routing.jsonl
+```
+
+Multiply the totals by each model's prices to get cost per arm. The pi
+integration page shows [one way to do that](../integrations/pi.md#estimate-the-cost).
+
+Fix: if the efficient-only arm solves at least as many tasks as the capable-only
+arm, swap the two roles, or serve the efficient model alone with `passthrough`.
+If both arms solve the same tasks, routing has little to gain on this workload.
+The room for routing is the set of tasks only the capable model solves.
+
+## Pin the provider and endpoint
+
+A model ID does not fix how the model is served. Three things move results
+between runs.
+
+**Provider.** Through an aggregator, one model ID can be served by several
+providers with different quantization, context length, and tool-call support.
+Symptom: the same configuration gives different results on different days, and
+the routing log's `model` field does not change. Detection: the aggregator's
+response metadata or dashboard shows which provider served each call. Fix: pin
+the provider in the target's `extra_body`. On OpenRouter, `order` names the
+providers to try and `allow_fallbacks = false` stops it from trying others:
+
+```toml
+[targets.weak]
+id = "qwen/qwen3-35b-a3b"
+llm_client = "openrouter"
+extra_body = { provider = { order = ["your-provider"], allow_fallbacks = false } }
+```
+
+Use the provider slug from the model's page on OpenRouter.
+
+**Endpoint and wire format.** The same model can behave differently on
+`openai_chat`, `openai_responses`, and `anthropic_messages`. One measured case
+is Claude behind an LLM gateway. On `anthropic_messages` the thinking level
+worked and the prompt cache was read. On the two OpenAI formats a thinking level
+returned HTTP 400, and `openai_responses` never read the cache. See
+[Claude through an LLM gateway](../integrations/pi.md#claude-through-an-llm-gateway).
+Record `format` and `base_url` with every result.
+
+**Streamed token counts on `/v1/messages`.** When a client uses Anthropic
+Messages and the selected target is on an OpenAI format, the streamed
+`message_start` event reports `input_tokens: 0`. The counts the upstream
+reports arrive in the final `message_delta` event. Symptom: the client shows zero prompt tokens, or a
+context meter that never moves, for turns served by those targets. Detection:
+compare the client's numbers with `prompt_tokens` in the routing log, which
+records the upstream's counts. Fix: read usage from `message_delta`, or use the
+routing log and `GET /v1/stats` for accounting.
+
+### Reasoning effort
+
+Effort changes a model's results and cost as much as the model choice does.
+Record it with every result. Three rules apply:
+
+- On `openai_responses` the effort field is `reasoning.effort`. A top-level
+ `reasoning_effort` key in `extra_body` does not set it. Use the target's
+ `reasoning_effort` key instead. It writes `reasoning.effort` on
+ `openai_responses` and `reasoning_effort` on `openai_chat`, replacing the value
+ the client sent. The DeepSWE profiles above use it to run both tiers at `max`.
+- `reasoning_effort` is rejected on `anthropic_messages` clients. The server does
+ not start:
+
+ ```text
+ target weak reasoning_effort is only supported on openai_chat and openai_responses clients
+ ```
+
+ For those targets the effort the client sends is passed through in
+ Anthropic's form.
+
+- One route cannot hold the same model at two efforts. Two targets with the same
+ model ID on one client but different `reasoning_effort` or `extra_body` fail
+ to load:
+
+ ```text
+ targets strong and strong_max both name model openai/gpt-4o on llm client openrouter but with different reasoning_effort, extra_body, or omit_body_fields; one target per model id is kept, so give each its own model id or llm client
+ ```
+
+ Separate routes can use separate `llm_clients` entries for the same
+ endpoint. See the [target keys](../reference/toml_schema.md#targetsname).
+
+## Related
+
+- [Routing Overview](overview.md): choose a strategy and read what Auto resolves to.
+- [Verify routing is active](../getting_started.md#verify-routing-is-active): see the served model and the decision counts.
+- [Stage Router](stage_router_routing.md#tuning-confidence_threshold): tune the threshold once the pair is right.
+- [Escalation-Router Routing](escalation_router_routing.md#judge-model-compatibility): judge requirements for escalation routes.
+- [Benchmarks](../../benchmark/README.md): how the measured runs were made.
diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.md
index 3878d7d3c..372de7db4 100644
--- a/docs/routing_algorithms/overview.md
+++ b/docs/routing_algorithms/overview.md
@@ -24,15 +24,57 @@ controls whether the judge runs for each request, each user turn, or once per
session. Execution uses the stage router, which reads tool-result history and
can optionally call an LLM judge. The TOML configuration keys are unchanged.
+Whichever strategy you choose, the two models you put behind it matter more than
+any threshold. See [Model Selection](model_selection.md).
+
### Auto
-Auto is a preset, not a separate routing algorithm. It currently uses Execution
-with `picker = "efficient_first"`, `confidence_threshold = 0.5`, and no LLM
-judge. It does not compare strategies at runtime. Use `stage_router` directly
-to tune these settings.
+Auto is a preset, not a separate routing algorithm. An `auto` route builds a
+stage router with these settings. It does not compare strategies at runtime.
-See the [Auto configuration reference](../reference/toml_schema.md#auto) for
-the required targets.
+| Setting | Value | What it does |
+|---|---|---|
+| `picker` | `efficient_first` | The efficient tier serves a turn when the tool signals are not confident. |
+| `confidence_threshold` | `0.5` | How strongly the tool signals must agree before they pick a tier on their own. |
+| `recent_turn_window` | `3` | How many recent tool results the signals are computed over. |
+| `capable_hold_turns` | `2` | How many turns stay on the capable tier after an escalation. A clean test pass ends the hold early. |
+| Classifier | none | No LLM judge is called. |
+
+Auto accepts no tuning keys. To change a setting, write the equivalent
+`stage_router` route shown in the [TOML schema](../reference/toml_schema.md#auto)
+and edit it there. Choose the two models with the
+[model selection guide](model_selection.md).
+
+Only the `efficient_first` picker has published results. The `capable_first`
+picker is [experimental](stage_router_routing.md#pickers).
+
+#### When Auto does not route
+
+Auto decides from tool results in the request. In these cases every request goes
+to one tier:
+
+- **No tool results in the request.** Chat-only requests, and the first request
+ of an agent session, carry no tool history. The signals abstain and the
+ efficient tier serves the turn. This is by design. `GET /v1/stats` counts these
+ turns under `algorithm_stats.stage_router.routing_decisions.fall_open`. See
+ [Verify routing is active](../getting_started.md#verify-routing-is-active).
+- **Both tiers name the same target.** The server accepts a route whose
+ `capable_target` and `efficient_target` are the same target. Every request then
+ goes to that one model, and `GET /v1/stats` lists a single entry under `.models`.
+- **No session ID on the request.** The capable hold is kept per session. A
+ request without a session ID gets fresh state, so a hold set by one request is
+ gone on the next. Requests with a session ID but no agent ID share one hold
+ within that session.
+
+A critical error, a repeated identical failure, or a context-compaction marker in
+the tool history still sends the turn to the capable tier. See
+[decision sources](stage_router_routing.md#decision-sources).
+
+Auto uses the settings of the published stage-router results, with two
+differences: those runs set `recent_turn_window = 5` and added a handoff note.
+Every published result comes from a coding agent whose requests carry tool
+results. The [model selection guide](model_selection.md#measured-pairs) lists the
+measured model pairs.
## More options
diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.md
index 5f71e322c..366ee9932 100644
--- a/docs/routing_algorithms/stage_router_routing.md
+++ b/docs/routing_algorithms/stage_router_routing.md
@@ -336,6 +336,8 @@ paths through its cascade:
## Related
+- [Model Selection](model_selection.md): choose and check the two models before
+ tuning the threshold.
- [Architecture](../architecture.md): the end-to-end request lifecycle and
system boundaries.
- [TOML Schema](../reference/toml_schema.md#stage_router): the complete
diff --git a/mkdocs.yml b/mkdocs.yml
index a291889d3..6c5309ffc 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -31,6 +31,7 @@ nav:
- Single-provider coding agents: recipes/single_provider_coding_agents.md
- Routing:
- Overview: routing_algorithms/overview.md
+ - Model Selection: routing_algorithms/model_selection.md
- Task (LLM Classifier): routing_algorithms/llm_classifier_routing.md
- Execution (Stage Router): routing_algorithms/stage_router_routing.md
- Plan/Execute Routing: routing_algorithms/plan_execute_routing.md
diff --git a/tests/getting_started/test_getting_started.py b/tests/getting_started/test_getting_started.py
index 1b8fe89f5..fb428324d 100644
--- a/tests/getting_started/test_getting_started.py
+++ b/tests/getting_started/test_getting_started.py
@@ -16,6 +16,14 @@ def test_getting_started_documents_current_paths() -> None:
assert "nemo-switchyard[cli]" not in guide
assert "cargo install --locked switchyard-server" in guide
assert "switchyard-server --config routes.toml --dry-run" in guide
+ assert (
+ guide.index("### Run the server")
+ < guide.index("### Verify routing is active")
+ < guide.index("### Routing algorithms")
+ )
+ assert "x-model-router-selected-model" in guide
+ assert "/v1/stats" in guide
+ assert "--routing-log-file" in guide
for legacy_command in (
"switchyard configure",