Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs: make the README and quickstart library-first (#1058)
nachiketb-nvidia5b986f818055
diff --git a/.github/workflows/getting-started.yml b/.github/workflows/getting-started.ymlindex 6b7511b02..114956935 100644--- a/.github/workflows/getting-started.yml+++ b/.github/workflows/getting-started.yml@@ -1,16 +1,15 @@name: Getting Started-# Executable coverage for docs/getting_started.md.-#-# Fails when the documented Rust server flow drifts from the product, including-# CLI changes, TOML schema changes, and operational endpoint regressions. It lives in-# tests/getting_started/ so it reproduces locally with+# Structural drift checks for the README, libsy quickstart, and local evaluation guide.+# These checks do not execute model calls or validate provider access. Reproduce with# `uv run pytest tests/getting_started`.on:pull_request:paths:+ - "README.md"- "docs/getting_started.md"+ - "docs/operations/local_evaluation.md"- "crates/**"- "tests/getting_started/**"- "Cargo.toml"@@ -22,7 +21,9 @@ on:push:branches: [main]paths:+ - "README.md"- "docs/getting_started.md"+ - "docs/operations/local_evaluation.md"- "crates/**"- "tests/getting_started/**"- "Cargo.toml"diff --git a/INSTALLATION.md b/INSTALLATION.mdindex bf07cf47a..fed288a95 100644--- a/INSTALLATION.md+++ b/INSTALLATION.md@@ -32,8 +32,8 @@ switchyard-server --config routes.toml --dry-runswitchyard-server --config routes.toml --port 4000```-See [Getting Started](docs/getting_started.md#server-path) for a complete TOML-deployment and [`switchyard-server`](crates/switchyard-server/README.md) for the+See [Local Evaluation](docs/operations/local_evaluation.md) for a complete TOML+example and [`switchyard-server`](crates/switchyard-server/README.md) for theconfiguration reference.## Desktop installerdiff --git a/README.md b/README.mdindex 053e12812..60ac10607 100644--- a/README.md+++ b/README.md@@ -15,7 +15,7 @@Switchyard is an open-source library that helps an AI agent choose which model handles each request. It combines efficient models with more capable models so you can balance task success, cost, and latency on your workload.-Use Switchyard through a gateway integration, try it with a local proxy, or embed it in your own harness. You choose the model pool. Switchyard supplies the routing decision. Your gateway or application owns the surrounding service.+Adopt Switchyard by embedding its Rust core, **libsy**, in your gateway or agent harness. You own the model clients, credentials, and service. Switchyard supplies the routing decision.<p align="center"><a href="docs/getting_started.md">Quickstart</a>  · @@ -29,14 +29,18 @@ Use Switchyard through a gateway integration, try it with a local proxy, or embe## How it works-1. Configure the models your application can use and choose a routing algorithm.+1. Configure the models an agent can use and choose a routing algorithm.2. The algorithm examines the request or the agent's recent tool activity. Some algorithms call a model to judge the task.-3. Your gateway or application sends the request to the selected model. Routing can change as the agent continues its work.+3. A gateway or agent sends the request to the selected model. Routing can change as the agent continues its work.Evaluate the complete agent, model pool, and routing configuration against your single-model baseline. A cheaper model call does not guarantee a cheaper successfully completed task. The [tuning loop](docs/tuning/index.md) gives the steps for that evaluation, and [Troubleshooting](docs/operations/troubleshooting.md) lists the settings that disable routing without an error.## How to use it+- **[Through an existing gateway](#through-an-existing-gateway):** Use an available gateway integration.+- **[Embed libsy in your harness](#embed-libsy-in-your-harness):** Add libsy to a service you operate.+- **[Evaluate Switchyard locally](#evaluate-switchyard-locally):** Compare routing configurations with your own model pool using the demo server.+### Through an existing gateway| Gateway | Start here | Deployment and limits |@@ -52,36 +56,52 @@ before enabling the plugin.LiteLLM and NeMo Relay run in your own deployment. Follow each gateway's guidance for credentials and service operation.-### Try routing locally--[Try routing locally](docs/getting_started.md#server-path) with the standalone proxy for demos and evaluation, then [verify routing is active](docs/getting_started.md#verify-routing-is-active).+<a id="start-with-libsy"></a>++### Embed libsy in your harness++`Algorithm::run_stream` hands model calls and the final routing outcome back to your application. In this development-API outline, comments mark the work your code supplies:++```rust+let mut steps = algorithm.run_stream(request, models, state);+while let Some(step) = steps.next().await {+ match step? {+ Step::CallModel(call) => {+ // Send call.request through your LLM client.+ // Give the call future to call.respond(...).await.+ }+ Step::CallDecision(call) => {+ // Send call.request through your decision-model client.+ // Give its typed response future to call.respond(...).await.+ }+ Step::Done(outcome) => {+ match outcome.response {+ Some(response) => {+ // Return this answer; no final LLM call is needed.+ }+ None => {+ // Send outcome.request through your client for the final answer,+ // or pass it to your gateway to make that call later.+ }+ }+ }+ }+}+```++- **Inputs:** supply an algorithm, model pool, request, and state.+- **Calls:** your clients handle model calls. Stage returns a selection; other algorithms may return an answer.+- **Start here:** [Rust quickstart](docs/getting_started.md) · [Python bindings](docs/reference/host_integration.md#python-bindings).++### Evaluate Switchyard locally++The standalone server is provided only for demos and local evaluation. It is not a supported production deployment path.++[Evaluate routing locally](docs/operations/local_evaluation.md) with the standalone server and your own model pool, then [verify routing is active](docs/operations/local_evaluation.md#verify-routing-is-active).Agent-specific guides are available for [pi](docs/integrations/pi.md) and[Oh My Pi](docs/integrations/oh_my_pi.md).-### Embed the library in your harness--[Embed the library in your harness](docs/getting_started.md#library-path) to run routing inside your Rust application. For Python, see the [embedding example](examples/libsy.py).--## What’s new--Features and integrations recently merged into `main`.--- **2026-10-16 · Route between Sonnet and Opus in Claude Code**\- Use your Claude login for automatic routing on Linux x86_64 and see the selected model in the status line. [Setup guide →](examples/claude/README.md)--- **2026-10-15 · Use decision models in composite routing**\- Use Mercury Decide to judge tasks and Stage routing to adjust model selection as tools run. [Example configuration →](dev-server/config.toml)--- **2026-10-13 · Use Jev to choose between models**\- Configure Jev as a task judge with candidate descriptions and a probability cutoff. [Configuration guide →](docs/routing_algorithms/composite_routing.md#decision-model-classifier)--- **2026-10-12 · Bring Switchyard to TypeScript and Pi**\- Select models in Node.js or try automatic routing through Pi’s virtual models. [TypeScript bindings →](bindings/typescript/README.md) · [Pi example →](examples/pi/README.md)--- **2026-10-08 · Run Codex with Switchyard on macOS**\- Install a background router and Codex profile, then track usage and estimated savings from the menu bar. [Setup guide →](crates/switchyard-menubar/README.md)-## Routing algorithmsStart with Auto. Choose Task, Execution, or Composite when you need more control. These names describe routing choices. The configuration keys are unchanged.@@ -93,7 +113,7 @@ Start with Auto. Choose Task, Execution, or Composite when you need more control| **[Execution](docs/routing_algorithms/stage_router_routing.md)** | Uses recent tool activity and outcome signals to choose a model as the agent runs. | `type = "stage_router"` || **[Composite](docs/routing_algorithms/composite_routing.md)** | Combines Task and Execution: a classifier sets the default model tier when execution signals are uncertain. | `type = "composite"` |-Auto is a fixed preset in v0.3.0. It does not compare strategies at runtime. The TOML runner supports `type = "auto"`. For direct Python embedding, use `stage_router(picker="efficient_first", confidence_threshold=0.5)` for the same preset.+Auto is a fixed preset in v0.3.0. It does not compare strategies at runtime. The TOML runner supports `type = "auto"`. In Rust, use `StageRouterConfig::new(PickerMode::EfficientFirst, 0.5)` for the same preset.The [routing overview](docs/routing_algorithms/overview.md) retains the full catalogue, including [Plan/Execute](docs/routing_algorithms/plan_execute_routing.md), [escalation](docs/routing_algorithms/escalation_router_routing.md), [advisor](docs/routing_algorithms/advisor_gate_routing.md), [sub-agent](docs/routing_algorithms/subagent_routing.md), and [random](docs/routing_algorithms/random_routing.md) strategies. Integration support varies: check the gateway guide before choosing an algorithm.@@ -109,7 +129,7 @@ For latency and routing overhead testing, see [Soak Testing](docs/operations/soa- [Benchmark setup and profiles](benchmark/README.md): run your own baseline and routing comparisons.- [Core concepts](docs/core_concepts.md): clients, targets, routes, and model IDs.- [TOML schema](docs/reference/toml_schema.md): configuration fields and defaults.-- [Architecture](docs/architecture.md): library and runtime components.+- [Architecture](docs/architecture.md): libsy and runtime components.- [Installation](INSTALLATION.md): package and platform requirements.## Components@@ -122,10 +142,29 @@ releases. Pin the version you integrate.| `switchyard-libsy` | **Beta** | Routing embedded in your own gateway or harness. You own model calls, credentials, and retries. | Trial integrations. API will change before v1.0. || `switchyard-llm-client` | **Alpha** | HTTP model calls and protocol translation alongside libsy. | Experiments and pilots. || `switchyard-runner` | **Alpha** | Running configured routes inside another runtime, such as NeMo Relay. | Integration work and supervised pilots. |-| `switchyard-server` | **Demo** | A standalone OpenAI- and Anthropic-compatible proxy, including a local Codex service. | Demos, evaluation, and personal use on single-user machines. Not for production. |+| `switchyard-server` | **Demo** | A standalone OpenAI- and Anthropic-compatible proxy. | Demos and local evaluation only; not a supported production path. |+## What’s new++Features and integrations recently merged into `main`.++- **2026-10-16 · Route between Sonnet and Opus in Claude Code**\+ Use your Claude login for automatic routing on Linux x86_64 and see the selected model in the status line. [Setup guide →](examples/claude/README.md)++- **2026-10-15 · Use decision models in composite routing**\+ Use Mercury Decide to judge tasks and Stage routing to adjust model selection as tools run. [Example configuration →](dev-server/config.toml)++- **2026-10-13 · Use Jev to choose between models**\+ Configure Jev as a task judge with candidate descriptions and a probability cutoff. [Configuration guide →](docs/routing_algorithms/composite_routing.md#decision-model-classifier)++- **2026-10-12 · Bring Switchyard to TypeScript and Pi**\+ Select models in Node.js or try automatic routing through Pi’s virtual models. [TypeScript bindings →](bindings/typescript/README.md) · [Pi example →](examples/pi/README.md)++- **2026-10-08 · Run Codex with Switchyard on macOS**\+ Install a background router and Codex profile, then track usage and estimated savings from the menu bar. [Setup guide →](crates/switchyard-menubar/README.md)+## Community and license[Report an issue](https://github.com/NVIDIA-NeMo/Switchyard/issues) · [Contribute](CONTRIBUTING.md) · [Code of conduct](CODE_OF_CONDUCT.md)diff --git a/docs/assets/quickstart/openrouter.rs b/docs/assets/quickstart/openrouter.rsnew file mode 100644index 000000000..878b45ae9--- /dev/null+++ b/docs/assets/quickstart/openrouter.rs@@ -0,0 +1,89 @@+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+// SPDX-License-Identifier: Apache-2.0++use std::{collections::HashMap, sync::Arc};+use futures::StreamExt;+use switchyard_libsy::{+ Algorithm, LibsyError, PickerMode, RuntimeModels, StageRouter, StageRouterConfig, State, Step,+};+use switchyard_protocol::{Category, LlmResponse, Request, Response, completion_text, text_request};+use switchyard_translation::{WireFormat, decode_aggregated_response, encode_request};++#[tokio::main(flavor = "current_thread")]+async fn main() -> Result<(), Box<dyn std::error::Error>> {+ // Start with Efficient when tool history gives no reason to change models.+ let router = Arc::new(StageRouter::new(StageRouterConfig::new(+ PickerMode::EfficientFirst, 0.5,+ ))?);+ let efficient = "z-ai/glm-5.2";+ let capable = "anthropic/claude-opus-4.8";+ let client = OpenRouterClient {+ http: reqwest::Client::builder()+ .timeout(std::time::Duration::from_secs(60)).build()?,+ api_key: std::env::var("OPENROUTER_API_KEY")?,+ };+ // Assign model roles; Any lists the full pool.+ let models = Arc::new(RuntimeModels::new(HashMap::from([+ (Category::Efficient, vec![efficient.into()]),+ (Category::Capable, vec![capable.into()]),+ (Category::Any, vec![efficient.into(), capable.into()]),+ ])));+ let request = Request {+ llm_request: text_request(None, "Run the tests."),+ ..Default::default()+ };++ let mut steps = router.run_stream(request, models, State::default());+ while let Some(step) = steps.next().await {+ match step? {+ Step::CallModel(call) => {+ // Give libsy the future so pending routing calls can be canceled.+ let request = call.request.clone();+ call.respond(client.call(request)).await?;+ }+ Step::CallDecision(_) => {+ // This Stage preset makes no decision calls; other presets may need a client.+ return Err("A decision client is required for this algorithm.".into());+ }+ Step::Done(outcome) => {+ println!("Selected: {}", outcome.selected_model_id()?);+ // Stage has selected the model. Your application now makes the final call.+ let response = client.call(outcome.request).await?;+ if let LlmResponse::Agg(answer) = response.llm_response {+ println!("Answer: {}", completion_text(&answer));+ }+ }+ }+ }+ Ok(())+}++struct OpenRouterClient {+ http: reqwest::Client,+ api_key: String,+}++impl OpenRouterClient {+ async fn call(&self, request: Request) -> switchyard_libsy::Result<Response> {+ // Convert libsy's request to the provider's wire format.+ let body = encode_request(&request.llm_request, WireFormat::OpenAiChat)+ .map_err(|error| LibsyError::external("encoding OpenRouter request", error))?;+ let response = self.http+ .post("https://openrouter.ai/api/v1/chat/completions")+ .bearer_auth(&self.api_key)+ .json(&body)+ .send().await+ .and_then(reqwest::Response::error_for_status)+ .map_err(|error| LibsyError::external("OpenRouter request", error))?;+ let upstream_headers = response.headers().clone();+ let body = response.json::<serde_json::Value>().await+ .map_err(|error| LibsyError::external("reading OpenRouter response", error))?;+ let answer = decode_aggregated_response(&body, WireFormat::OpenAiChat)+ .map_err(|error| LibsyError::external("decoding OpenRouter response", error))?;+ Ok(Response {+ llm_response: LlmResponse::Agg(answer),+ metadata: request.metadata,+ upstream_headers,+ })+ }+}diff --git a/docs/getting_started.md b/docs/getting_started.mdindex 1c47af2a1..f9ed945a0 100644--- a/docs/getting_started.md+++ b/docs/getting_started.md@@ -1,396 +1,268 @@# Getting Started with Switchyard-Switchyard has two native Rust execution paths:+Add routing to your existing Rust gateway or agent harness with **libsy**, Switchyard's core library. Your application keeps its model clients, credentials, and service.-- **Server path:** build and run the standalone Rust server for API clients and- custom deployments.-- **Library path:** embed the routing algorithms directly in your own Rust- application with `switchyard-libsy`.+Start with a local routing decision. Then add tool history or send the selected request through your own model client. Python bindings are covered [separately](reference/host_integration.md#python-bindings).-## Server Path+## Install-Use this path when you want a standalone proxy for API clients or need to-operate the Rust server directly.--### Prerequisites--- Git, a native build toolchain, and Rust with Cargo-- An API key for OpenRouter, OpenAI, Anthropic, or another OpenAI-compatible endpoint.- To use OpenRouter, create an account at [openrouter.ai](https://openrouter.ai/)- and generate a key from the [OpenRouter keys page](https://openrouter.ai/keys).--On Ubuntu or WSL, install the build prerequisites and Rust with `rustup`:+You need Rust 1.96.1+ with Cargo and a native build toolchain. See [installation requirements](../INSTALLATION.md#requirements). These examples use the development API, pinned to one Git revision. The released `0.3.0` crates do not include `Step::CallDecision` and use a different `respond` API.```bash-sudo apt-get update-sudo apt-get install -y build-essential curl git-curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh-source "$HOME/.cargo/env"+cargo new switchyard-demo+cd switchyard-demo+# Keep libsy and its protocol types on the same API revision.+cargo add switchyard-libsy switchyard-protocol --git https://github.com/NVIDIA-NeMo/Switchyard.git --rev 5415db205c1f4db7e079731e7bf33a6609c67c8e+# futures reads the step stream; Tokio runs the asynchronous routing tasks.+cargo add [email protected]+cargo add tokio@1 --features macros,rt```-For v0.3.0, the standalone server is release-validated on Ubuntu 24.04,-Linux x86_64. Other platforms are outside the release-validation scope.+## Route a request-If you try a source build on macOS or native Windows, follow the-[official Rust installation instructions](https://rust-lang.org/tools/install/).+Replace `src/main.rs` with this complete example. It selects a model locally, without an API key:-Install `uv` for the repository's Python-based tooling and CI checks. It is not-required to build or run the Rust server:--```bash-curl -LsSf https://astral.sh/uv/install.sh | sh+```rust+use std::{collections::HashMap, sync::Arc};+use futures::StreamExt;+use switchyard_libsy::{+ Algorithm, PickerMode, RuntimeModels, StageRouter, StageRouterConfig, State, Step,+};+use switchyard_protocol::{Category, Request, text_request};++#[tokio::main(flavor = "current_thread")]+async fn main() -> Result<(), Box<dyn std::error::Error>> {+ // Start with Efficient when tool history gives no reason to change models.+ let router = Arc::new(StageRouter::new(StageRouterConfig::new(+ PickerMode::EfficientFirst, 0.5,+ ))?);+ let efficient = "Efficient";+ let capable = "Capable";+ // Assign model roles; Any lists the full pool. These are local demo IDs.+ let models = Arc::new(RuntimeModels::new(HashMap::from([+ (Category::Efficient, vec![efficient.into()]),+ (Category::Capable, vec![capable.into()]),+ (Category::Any, vec![efficient.into(), capable.into()]),+ ])));+ // Leave the model unset so Stage can select it.+ let request = Request {+ llm_request: text_request(None, "Run the tests."),+ ..Default::default()+ };++ // State holds routing state; next() reads each result from the step stream.+ let mut steps = router.run_stream(request, models, State::default());+ while let Some(step) = steps.next().await {+ match step? {+ Step::CallModel(_) => {+ // Other presets may ask your LLM client to serve a routing-time call.+ return Err("An LLM client is required for this algorithm.".into());+ }+ Step::CallDecision(_) => {+ // Typed decision questions need a decision-model client.+ return Err("A decision client is required for this algorithm.".into());+ }+ Step::Done(outcome) => {+ println!("Selected: {}", outcome.selected_model_id()?);+ // Stage selects the model; your client or gateway can send+ // outcome.request for the final answer, now or later.+ }+ }+ }+ Ok(())+}```-If either installer updates your shell configuration, restart the shell before-continuing. Verify the tools:-```bash-git --version-rustc --version-cargo --version-uv --version+# Model selection is local; this makes no provider requests.+cargo run```-### Install the server+Expected output, after Cargo's build messages:-Install the Rust server from crates.io:--```bash-cargo install --locked switchyard-server-switchyard-server --help+```text+Selected: Efficient```-Cargo builds the release binary and installs it into `~/.cargo/bin` by default.+This Stage preset yields only `Done`, with `outcome.response` set to `None`. Your application owns the final model call. The loop also shows where other algorithms can ask for intermediate model calls.-#### Build from source+`0.5` is a signal-confidence threshold, not a measured probability of task success. This preset is **Auto** in TOML. See the [Stage router guide](routing_algorithms/stage_router_routing.md) for signal rules, and evaluate your workload before relying on savings.-Install the server from `main` to use unreleased features:--```bash-cargo install --locked \- --git https://github.com/NVIDIA-NeMo/Switchyard.git \- --branch main \- switchyard-server-```+<details markdown="1">+<summary>Try the same prompt after a tool failure</summary>-### Configure+Keep the same program. Insert this block immediately before `let mut steps = ...`, then run `cargo run` again:-The Rust server reads an explicit TOML file.+```rust+use switchyard_protocol::{ContentBlock, Message, Role, ToolCall, ToolResult};++let mut request = request;+// Describe the agent's past activity; this example does not execute bash.+request.llm_request.messages.extend([+ Message {+ role: Role::Assistant,+ content: vec![ContentBlock::ToolCall(ToolCall {+ id: "c1".into(),+ name: "bash".into(),+ arguments: r#"{"command":"cargo test"}"#.parse()?,+ })],+ },+ Message {+ role: Role::User,+ content: vec![ContentBlock::ToolResult(ToolResult {+ // The result refers to the tool call above by its ID.+ tool_call_id: "c1".into(),+ is_error: Some(true),+ content: vec![ContentBlock::Text {+ text: "MemoryError: cannot allocate memory\nKilled".into(),+ }],+ })],+ },+]);+```-Create `routes.toml` with an auto route:+The critical failure makes Stage select **Capable** for the same user prompt:-```toml-schema_version = 1+```text+Selected: Capable+```-[llm_clients.openrouter]-format = "openai_chat"-base_url = "https://openrouter.ai/api/v1"-api_key_env = "OPENROUTER_API_KEY"+Both runs are local and need no API key. Remove the inserted block to return to the fresh request.-[targets.weak]-id = "openai/gpt-4o-mini"-llm_client = "openrouter"+</details>-[targets.strong]-id = "openai/gpt-4o"-llm_client = "openrouter"+## Use libsy with your client-[routes.smart]-id = "switchyard"-type = "auto"-capable_target = "strong"-efficient_target = "weak"-```+**You do not have to use Switchyard's HTTP client.** Keep your existing SDK, or create your own client. The same program can send Stage's selected request through [OpenRouter](https://openrouter.ai/docs/quickstart).-The two targets are placeholders. See [Model Selection](routing_algorithms/model_selection.md)-to choose models for the capable and efficient tiers.--`format` selects the upstream protocol and must be `openai_chat`,-`openai_responses`, or `anthropic_messages`. `api_key_env` names the environment-variable the server reads; the secret does not belong in the TOML file.-A client can set `forward_auth = true` instead of `api_key_env` to send each-caller's credential to that upstream. OpenAI clients forward `authorization`,-`chatgpt-account-id`, and `x-openai-fedramp`. Anthropic clients forward-`authorization` or `x-api-key`. Enable this only for an upstream that should-receive the caller's login. Other application headers are preserved, so they-may contain provider-specific credentials. The forwarding clients in a route,-including efficient and capable targets, must use one credential family: all-OpenAI formats or all `anthropic_messages`. The exception is one host that-serves both formats, such as an LLM gateway that accepts each caller's gateway-key on every endpoint: a route may mix the two families when all of its-forwarding clients use the same scheme, host, and port. Such a route serves-Chat Completions and Responses callers and forwards the caller's bearer token-to every forwarding client. The server returns 400 to a caller whose API the-route does not serve. Clients that use `api_key_env` send the server's own key,-so these limits do not apply to them.--### Run the server--Export the provider credential, validate the configuration without binding a-socket, then start the release binary:+Add the dependencies:```bash-export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret-switchyard-server --config routes.toml --dry-run-switchyard-server --config routes.toml \- --host 127.0.0.1 --port 4000+# Translation converts neutral protocol types to and from provider JSON.+cargo add switchyard-translation --git https://github.com/NVIDIA-NeMo/Switchyard.git --rev 5415db205c1f4db7e079731e7bf33a6609c67c8e+# reqwest handles HTTPS and JSON; serde_json represents the provider's reply.+cargo add [email protected] --no-default-features --features json,rustls+cargo add serde_json@1```-Any client that speaks OpenAI Chat Completions, Anthropic Messages, or OpenAI-Responses API can connect. The route `id` is the model name clients use.+For the complete program, replace `src/main.rs` with the [OpenRouter example](assets/quickstart/openrouter.rs). It includes the router above and the adapter below. To extend your existing file instead, apply the edits shown here.-In another terminal:+Create an [OpenRouter key](https://openrouter.ai/settings/keys) with access to your chosen models:```bash-curl http://localhost:4000/health-curl http://localhost:4000/v1/models-curl http://localhost:4000/v1/chat/completions \- -H "Content-Type: application/json" \- -d '{"model":"switchyard","messages":[{"role":"user","content":"hello"}]}'+export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret```-### Verify routing is active--The first request usually lands on the efficient model. That does not mean-routing is off. Auto decides from tool results, and a request without tool-history has none, so the efficient tier serves it by design. Use these checks-to see routing work. Your model IDs and token counts will differ from the-examples.--1. **Confirm the configuration loads.** The dry run prints the route IDs the- server will serve:-- ```bash- switchyard-server --config routes.toml --dry-run- ```-- ```text- server OK: switchyard- ```--2. **Read the served model from the response header.** Every response carries- `x-model-router-selected-model`. A chat-only request is served by the- efficient model:-- ```bash- curl -si http://localhost:4000/v1/chat/completions \- -H "Content-Type: application/json" \- -d '{"model":"switchyard","messages":[{"role":"user","content":"hello"}]}' \- | grep -i x-model-router-selected-model- ```-- ```text- x-model-router-selected-model: openai/gpt-4o-mini- ```--3. **Send a request that escalates.** A tool result that reports a critical- error forces the capable tier. Save this request as `escalate.json`:-- ```json- {- "model": "switchyard",- "messages": [- {"role": "user", "content": "Run the test suite."},- {"role": "assistant", "content": null, "tool_calls": [- {"id": "call_1", "type": "function",- "function": {"name": "bash", "arguments": "{\"command\":\"pytest -q\"}"}}- ]},- {"role": "tool", "tool_call_id": "call_1",- "content": "MemoryError: cannot allocate memory\nKilled"},- {"role": "user", "content": "Fix it."}- ]- }- ```-- ```bash- curl -si http://localhost:4000/v1/chat/completions \- -H "Content-Type: application/json" -d @escalate.json \- | grep -i x-model-router-selected-model- ```-- ```text- x-model-router-selected-model: openai/gpt-4o- ```-- The same hard failure appearing twice in recent tool results, or a- context-compaction marker, also forces the capable tier. See- [decision sources](routing_algorithms/stage_router_routing.md#decision-sources).--4. **Count decisions by source.** `GET /v1/stats` reports calls per model and- routing decisions by source:-- ```bash- curl -s http://localhost:4000/v1/stats | jq -c '.models | map_values(.calls)'- curl -s http://localhost:4000/v1/stats | jq -c '.algorithm_stats.stage_router.routing_decisions'- ```-- ```json- {"openai/gpt-4o":1,"openai/gpt-4o-mini":1}- {"fall_open":{"total":1,"targets":{"openai/gpt-4o-mini":1}},"override":{"total":1,"targets":{"openai/gpt-4o":1}}}- ```-- `fall_open` counts turns with no usable signals that landed on the picker's- default tier. `override` counts turns a hard error forced to the capable- tier. Counts reset when the server restarts or on `POST /v1/stats/reset`.--5. **Keep a record of every request.** Start the server with- `--routing-log-file routing.jsonl` to append one JSON line per served request:-- ```json- {"ts":"2026-10-15T06:16:22.643Z","route_id":"switchyard","algorithm":"stage_router","origin":null,"task":null,"trial_id":null,"session_id":null,"model":"openai/gpt-4o","tier":"","prompt_tokens":42,"cached_tokens":0,"cache_creation_tokens":0,"completion_tokens":5,"reasoning_tokens":0,"total_tokens":47}- ```-- The record names the served model and its token counts. It does not record- why that model was chosen; use `GET /v1/stats` for the decision counts.--### Routing algorithms--#### Choose a route type--Start with **Auto**, as shown above. Choose Task or Execution when you want-more control over how requests are routed.+In `main`, replace the two demo model-ID assignments with:-| Choice | Use it when | Route `type` |-|---|---|---|-| **[Auto](routing_algorithms/overview.md#auto)** | You want Switchyard's recommended preset. | `auto` |-| **[Task](routing_algorithms/llm_classifier_routing.md)** | You want an LLM to judge which model can handle the task. | `llm_classifier` |-| **[Execution](routing_algorithms/stage_router_routing.md)** | You want tool results and agent progress to guide each request. | `stage_router` |--A single TOML file can declare multiple routes. The table key, such as-`routes.smart`, is a local configuration name; each route's `id` is exposed as a-model on `GET /v1/models`.--See [Routing Overview](routing_algorithms/overview.md) to compare strategies,-and the [`switchyard-server` guide](../crates/switchyard-server/README.md) for-the complete TOML schema, route options, TLS, and metrics.--### Troubleshooting--**No API key / auth error**--```bash-test -n "$OPENROUTER_API_KEY" && echo "key is set" || echo "key is missing"-switchyard-server --config routes.toml --dry-run+```rust+// Use provider model IDs for the same Efficient and Capable roles.+let efficient = "z-ai/glm-5.2";+let capable = "anthropic/claude-opus-4.8";+// Reuse one HTTP client, with a 60-second timeout for each call.+let client = OpenRouterClient {+ http: reqwest::Client::builder()+ .timeout(std::time::Duration::from_secs(60)).build()?,+ api_key: std::env::var("OPENROUTER_API_KEY")?,+};```-Confirm that `api_key_env` in `routes.toml` names the environment variable you-exported. The dry run validates the schema, environment lookup, target-references, and route construction without starting the server.--**Connection refused**--Check health: `curl http://localhost:4000/health`+Inside `match step?`, replace the `CallModel` and `Done` arms with these. Keep `CallDecision` unchanged; it needs a separate [decision client](reference/host_integration.md#serve-routing-time-calls).-**Telemetry header**--Switchyard documents an `X-Switchyard-Version` header for release attribution-on outbound LLM calls. No request or response content is included. The 0.2.0-native server does not currently send this header upstream (see-[Known Issues](known_issues.md)), so no opt-out is required at the moment.-------## Library Path--Use this path when you want routing inside your own Rust application rather than-behind a proxy. `switchyard-libsy` never calls a model itself: an algorithm-picks a target and hands the model call back to you.--For plan/execute, supply conversation history, including tool calls and results,-before routing. Responses API continuation IDs alone are not enough. See the-[Responses API history requirement](routing_algorithms/plan_execute_routing.md#responses-api-history-requirement).--### Add the dependencies--```toml-[dependencies]-async-trait = "0.1"-futures = "0.3"-switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git", tag = "v0.3.0" }-switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git", tag = "v0.3.0" }-tokio = { version = "1", features = ["macros", "rt"] }+```rust+Step::CallModel(call) => {+ // Pass the future to respond so libsy can cancel pending routing calls.+ let request = call.request.clone();+ call.respond(client.call(request)).await?;+}+Step::Done(outcome) => {+ println!("Selected: {}", outcome.selected_model_id()?);+ // Stage has selected the model. Your application now makes the final call.+ let response = client.call(outcome.request).await?;+ if let LlmResponse::Agg(answer) = response.llm_response {+ println!("Answer: {}", completion_text(&answer));+ }+}```-### Choose an algorithm+Append the adapter to the same file:-| Type | Purpose |-|---|---|-| `LlmTaskClassifier` | Ask a judge model to choose an efficient or capable target. |-| `StageRouter` | Route from signals already in the conversation, such as tool results and errors, with an optional judge fallback. |-| `LlmTaskClassifier` with escalation | Every turn runs on the efficient target first, and a judge reads that answer to decide whether to send the same request to the capable target. |-| `Random` | Select among any number of targets, uniform or weighted. |+<details markdown="1">+<summary>OpenRouter adapter: append to src/main.rs</summary>-These are the same strategies the server exposes as route types, so a deployment-can move between the server and library paths without changing routing-behaviour.--### Drive the algorithm--An algorithm yields a stream of steps. Each `Step::CallModel` is a routing-time classifier or-judge call your host performs over its own transport. The run ends with `Step::Done` carrying a-`RoutingOutcome`: the selected model, ordered fallbacks, rewritten request, and an optional-response when routing already produced the answer. Otherwise the host makes the terminal answer-call from that outcome. Serving these calls yourself is what lets libsy embed in a host that-already owns its HTTP stack, retries, and credentials.--Pass each call's work to `respond` before awaiting it. In Rust:+This reuses `Request` from the imports above. It uses `reqwest` for HTTP and `switchyard-translation` for provider JSON. It returns a buffered response and requires no client trait implementation.```rust-use switchyard_libsy::{CallModel, LibsyError, Result};-use switchyard_protocol::RoutedLlmClient;--async fn serve(call: CallModel, client: &dyn RoutedLlmClient) -> Result<()> {- let request = call.request.clone();- let model = call.models[0].clone();- call.respond(async move {- client.call(request).await.map_err(|error| LibsyError::client_call(model, error))- }).await+use switchyard_libsy::LibsyError;+use switchyard_protocol::{LlmResponse, Response, completion_text};+use switchyard_translation::{+ WireFormat, decode_aggregated_response, encode_request,+};++struct OpenRouterClient {+ // Reuse one HTTP client for connection pooling across model calls.+ http: reqwest::Client,+ // The application loads this from the environment instead of source code.+ api_key: String,}-```-In Python, the client coroutine returns `LlmResponse.Agg` or `LlmResponse.Stream`:--```python-await call.respond(client.call(call.request, call.models[0]))+impl OpenRouterClient {+ async fn call(&self, request: Request) -> switchyard_libsy::Result<Response> {+ // Encode the neutral request as OpenRouter Chat Completions JSON.+ // libsy has already set its model to the selected target.+ let body = encode_request(&request.llm_request, WireFormat::OpenAiChat)+ .map_err(|error| LibsyError::external("encoding OpenRouter request", error))?;+ // Your client performs the provider call and supplies the API key.+ let response = self.http+ .post("https://openrouter.ai/api/v1/chat/completions")+ .bearer_auth(&self.api_key)+ .json(&body)+ .send().await+ // Return HTTP failures before trying to decode an answer.+ .and_then(reqwest::Response::error_for_status)+ .map_err(|error| LibsyError::external("OpenRouter request", error))?;+ // Save headers before consuming the body, then read the full JSON reply.+ let upstream_headers = response.headers().clone();+ let body = response.json::<serde_json::Value>().await+ .map_err(|error| LibsyError::external("reading OpenRouter response", error))?;+ // Convert the provider's reply back to libsy's neutral response type.+ let answer = decode_aggregated_response(&body, WireFormat::OpenAiChat)+ .map_err(|error| LibsyError::external("decoding OpenRouter response", error))?;+ Ok(Response {+ // Agg holds a complete answer; a streaming client would return Stream.+ llm_response: LlmResponse::Agg(answer),+ // Keep any correlation metadata attached to the routed request.+ metadata: request.metadata,+ upstream_headers,+ })+ }+}```-`respond` now accepts a future or awaitable instead of a completed response. Move-the provider await into this method. Python `call.fail(error)` is also awaitable.-Decision calls use the same pattern with their existing decision response type.+</details>-When an algorithm stops waiting for a call, `respond` cancels that work and-returns normally. Python waits for task cleanup before returning. For concurrent-algorithms, schedule each Python handler with `asyncio.create_task` and cancel and-await remaining handlers when the run ends; see the complete-[Python example](../examples/libsy.py). Awaiting handlers one at a time inside the-step loop serializes their execution. Rust hosts can use `libsy::drive` to manage-concurrent handlers and their lifetime.+Run the program again. **This makes one billable model call**:-After a response stream is delivered, its consumer controls its lifetime.-Cancellation cannot guarantee that an upstream provider stops generating or billing.+```bash+cargo run+```-Successful runs also include `OutcomeMetadata`: a unique `outcome_id`, the algorithm-name, and optional JSON evidence. In Python, read `outcome.metadata.outcome_id`,-`outcome.metadata.algorithm`, and `outcome.metadata.evidence` after checking that-`outcome.metadata` is present. Evidence is a normal Python value, usually a dictionary;-algorithms without evidence return `None`.+For the fresh request, expect this shape; the answer text depends on the model:-With a host-installed OpenTelemetry subscriber, the existing `libsy.run` span records-the same identity, selected models, and supported evidence fields. See-the [OpenTelemetry reference](reference/opentelemetry.md) for field names, metrics,-and export setup. libsy does not install an exporter or send telemetry itself.+```text+Selected: z-ai/glm-5.2+Answer: <model reply>+```-For the request, response, and streaming types the steps carry, see-[`switchyard-protocol`](../crates/protocol/README.md).+With the optional tool-failure history, the selected model is `anthropic/claude-opus-4.8`. For routing only, keep the original `Done` arm. Other algorithms may supply an answer in `outcome.response`; the [general integration pattern](../README.md#start-with-libsy) shows how to reuse it.----+Your client returns `switchyard_protocol::Response` and adapts errors to `libsy::Result`. The [optional Switchyard HTTP client](../crates/libsy-llm-client/README.md) provides translation, streaming, retries, and fallback. Use [`libsy::drive`](reference/host_integration.md#concurrent-calls-and-cancellation) for concurrent algorithms; this loop serves calls sequentially.## Next steps-- [Core Concepts](core_concepts.md): LLM clients, targets, and routes-- [`switchyard-server`](../crates/switchyard-server/README.md): server configuration,- routing algorithms, TLS, and metrics-- [Rust API reference](reference/rust_api.md): generated libsy, protocol, and translation- documentation, crate setup, and API boundaries-- [`switchyard-translation`](../crates/switchyard-translation/README.md):- request, response, and stream translation+- **Explore the Rust API:** [libsy crate](../crates/libsy/README.md) and [generated API reference](reference/rust_api.md).+- **Use a gateway integration:** [LiteLLM](../examples/litellm/README.md) or [NeMo Relay](integrations/nemo_relay.md). Check each guide's supported algorithms and versions.+- **Choose another algorithm:** [routing overview](routing_algorithms/overview.md).+- **Try the standalone proxy:** [local evaluation guide](operations/local_evaluation.md). The server is for demos, evaluation, and personal use, not a supported production path.+- **Use Python:** [Python bindings](reference/host_integration.md#python-bindings).+- **Use an agent directly:** [Claude Code example](../examples/claude/README.md), [pi](integrations/pi.md), or [Oh My Pi](integrations/oh_my_pi.md).diff --git a/docs/index.md b/docs/index.mdindex b023ffdc1..577dd1d5a 100644--- a/docs/index.md+++ b/docs/index.md@@ -1,27 +1,29 @@# Switchyard-Switchyard routes and translates LLM traffic for coding agents and API clients.-It supports OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages.+Switchyard helps an AI agent choose which model handles each request. Embed its Rust core, **libsy**, in your gateway or agent harness. Your application keeps its model clients, credentials, and service.++**[Start with the Rust quickstart](getting_started.md)**: add the crates, run a routing example, and send the selected request through your model client.## Choose a Path| Goal | Path | Start here ||---|---|---|-| Run Switchyard as a standalone proxy for API clients | Server Path | [Build and run the Rust server](getting_started.md#server-path) |+| Add routing to your Rust gateway or application | libsy | [Rust quickstart](getting_started.md) |+| Use libsy from Python | Python Bindings | [Python embedding](reference/host_integration.md#python-bindings) |+| Evaluate routing locally with your own models | Local Evaluation | [Run the demo server](operations/local_evaluation.md) || Route Codex or Claude Code using your existing login | Server Path | [Single-provider coding agents](recipes/single_provider_coding_agents.md) |-| Add Switchyard routing to a Rust application | Library Path | [`switchyard-libsy`](../crates/libsy/README.md) || Add Switchyard routing to NeMo Relay | Native Plugin Path | [Use Switchyard with NeMo Relay](integrations/nemo_relay.md) || Point the pi coding agent at the standalone proxy | Server Path | [Use Switchyard with pi](integrations/pi.md) || Point Oh My Pi at the standalone proxy | Server Path | [Use Switchyard with Oh My Pi](integrations/oh_my_pi.md) |-The Server Path builds and runs the standalone `switchyard-server` binary.+The standalone server is for demos, evaluation, and personal use. It is not a supported production deployment path. Its HTTP API supports OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages.## Explore- [Benchmarks](benchmarks.md): compare Terminal-Bench 2.1 and DeepSWE results and open their routing configs- [Core Concepts](core_concepts.md): learn the LLM client, target, and route layers- [Routing Algorithms](routing_algorithms/overview.md): choose how requests select a model-- [Architecture](architecture.md): understand the proxy and library components+- [Architecture](architecture.md): understand libsy and the proxy components- [Server CLI Reference](cli_reference.md): inspect `switchyard-server` commands and options- [Rust API](reference/rust_api.md): browse libsy, protocol, and translation crate documentation- [Context-Window Handling](operations/context_window.md): configure eviction and retry behaviordiff --git a/docs/integrations/nemo_relay.md b/docs/integrations/nemo_relay.mdindex 83bba4022..3af0ed16c 100644--- a/docs/integrations/nemo_relay.md+++ b/docs/integrations/nemo_relay.md@@ -49,7 +49,7 @@ See the [upstream error compatibility note](#upstream-error-compatibility) beforusing older Relay versions.Routing does not require Relay. You can instead run the-[standalone server](../getting_started.md#server-path) or embed+[standalone server](../operations/local_evaluation.md) or embed[`switchyard-libsy`](../../crates/libsy/README.md) directly.### Measure the Cost of Routingdiff --git a/docs/integrations/oh_my_pi.md b/docs/integrations/oh_my_pi.mdindex 2fadf0fa4..ff349625b 100644--- a/docs/integrations/oh_my_pi.md+++ b/docs/integrations/oh_my_pi.md@@ -5,7 +5,7 @@ It reads its model providers from `~/.omp/agent/models.yml`. Add Switchyard ther`omp` sends every model call to `switchyard-server`. `omp` has no `--base-url` flag, andsetting `OPENAI_BASE_URL` does not change the address of its built-in `openai` provider.This page was tested with Oh My Pi 18.1.21 against the-[Getting Started](../getting_started.md#server-path) server on `http://localhost:4000`+[local evaluation](../operations/local_evaluation.md) server on `http://localhost:4000`with route id `switchyard`.## Configurediff --git a/docs/integrations/pi.md b/docs/integrations/pi.mdindex e0f9cc832..a456358d1 100644--- a/docs/integrations/pi.md+++ b/docs/integrations/pi.md@@ -5,7 +5,7 @@`switchyard-server`, which picks the target model for each call. pi does not read`OPENAI_BASE_URL`, so a provider entry is how you point pi at a proxy. Thispage was tested with pi 0.84.3 against the-[Getting Started](../getting_started.md#server-path) server on `http://localhost:4000`+[local evaluation](../operations/local_evaluation.md) server on `http://localhost:4000`with route id `switchyard`.## Configure@@ -70,7 +70,7 @@ pi -p --provider switchyard --model switchyard "List the files in this directoryThe command `curl -s localhost:4000/v1/stats | jq '.models | map_values({calls, prompt_tokens})'`lists each target model with its call count. For a step-by-step check with anescalating request, see-[Verify routing is active](../getting_started.md#verify-routing-is-active). Every response carries the header+[Verify routing is active](../operations/local_evaluation.md#verify-routing-is-active). Every response carries the header`x-model-router-selected-model`. With `--routing-log-file PATH`, the server writes onerecord per call. In the record below, `session_id` is the value of pi's `x-session-id`header:diff --git a/docs/operations/local_evaluation.md b/docs/operations/local_evaluation.mdnew file mode 100644index 000000000..6e14d5233--- /dev/null+++ b/docs/operations/local_evaluation.md@@ -0,0 +1,165 @@+# Evaluate Routing Locally++Use the standalone server to try routing configurations with your own model pool. It is for demos, evaluation, and personal use on single-user machines. **It is not a supported production deployment path.** To add routing to your gateway or application, use the [libsy quickstart](../getting_started.md).++## Install the server++You need Rust 1.96.1+, a native build toolchain, `curl`, and an OpenRouter API key. Install Rust using the [official instructions](https://rust-lang.org/tools/install/) and check [platform requirements](../../INSTALLATION.md#requirements). The v0.3.0 server is release-validated on Ubuntu 24.04, Linux x86_64.++Install the released server:++```bash+cargo install --locked switchyard-server --version 0.3.0+switchyard-server --help+```++For unreleased features, [build from main](../../crates/switchyard-server/README.md). Keep your configuration and binary on compatible versions.++## Configure your model pool++Create `routes.toml`:++```toml+schema_version = 1++[llm_clients.openrouter]+format = "openai_chat"+base_url = "https://openrouter.ai/api/v1"+api_key_env = "OPENROUTER_API_KEY"++[targets.efficient]+id = "z-ai/glm-5.2"+llm_client = "openrouter"++[targets.capable]+id = "anthropic/claude-opus-4.8"+llm_client = "openrouter"++[routes.smart]+id = "switchyard"+type = "auto"+capable_target = "capable"+efficient_target = "efficient"+```++This uses the same Auto preset as the first libsy example: Stage routing with `efficient_first`, a confidence threshold of `0.5`, and no classifier call. The model pool also appears in our [Terminal-Bench 2.1 results](../benchmarks.md#terminal-bench-21). This small Auto example is not the benchmark configuration; use the linked benchmark profiles to reproduce their settings.++To use your own pool:++| Change | Where |+| --- | --- |+| Efficient and capable models | Replace `targets.efficient.id` and `targets.capable.id`. |+| Provider endpoint | Change `llm_clients.openrouter.base_url` and `api_key_env`. Use a matching `format`: `openai_chat`, `openai_responses`, or `anthropic_messages`. |+| Different providers for each model | Add another `llm_clients` entry and set each target's `llm_client`. |+| Public model name used by callers | Change `routes.smart.id` and the request's `model` together. |++Keep credentials in environment variables. For provider options, credential forwarding, and route types, see the [TOML reference](../reference/toml_schema.md) and [model selection guide](../routing_algorithms/model_selection.md).++## Run the server++Set `OPENROUTER_API_KEY` in your environment with access to the configured models. Then validate and run the configuration:++```bash+export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret+switchyard-server --config routes.toml --dry-run+switchyard-server --config routes.toml --host 127.0.0.1 --port 4000+```++The dry run prints `server OK: switchyard`. It checks configuration and credential lookup, not whether the provider accepts the key.++In a second terminal, check local readiness:++```bash+curl -fsS http://127.0.0.1:4000/health+curl -fsS http://127.0.0.1:4000/v1/models+```++The model listing includes `switchyard`. These local checks make no provider requests.++## Verify routing is active++The next two requests call your providers and may incur charges. Start with a fresh server, or clear its counters before this sequence:++```bash+curl -fsS -X POST http://127.0.0.1:4000/v1/stats/reset+```++First, send a request without tool history:++```bash+curl -i http://127.0.0.1:4000/v1/chat/completions \+ -H 'Content-Type: application/json' \+ -d '{"model":"switchyard","messages":[{"role":"user","content":"Say hello in one sentence."}]}'+```++The response header identifies the efficient model:++```text+x-model-router-selected-model: z-ai/glm-5.2+```++Next, save as `escalate.json`:++```json+{+ "model": "switchyard",+ "messages": [+ {"role": "user", "content": "Run the test suite."},+ {"role": "assistant", "content": null, "tool_calls": [+ {"id": "call_1", "type": "function",+ "function": {"name": "bash", "arguments": "{\"command\":\"pytest -q\"}"}}+ ]},+ {"role": "tool", "tool_call_id": "call_1",+ "content": "MemoryError: cannot allocate memory\nKilled"},+ {"role": "user", "content": "Fix it."}+ ]+}+```++Send it:++```bash+curl -i http://127.0.0.1:4000/v1/chat/completions \+ -H 'Content-Type: application/json' -d @escalate.json+```++The critical error forces the capable model:++```text+x-model-router-selected-model: anthropic/claude-opus-4.8+```++Read the counters with `jq` installed:++```bash+curl -fsS http://127.0.0.1:4000/v1/stats | jq '.models | map_values(.calls)'+curl -fsS http://127.0.0.1:4000/v1/stats | jq '.algorithm_stats.stage_router.routing_decisions'+```++After exactly these two successful completions, expect one call per model:++```json+{"anthropic/claude-opus-4.8":1,"z-ai/glm-5.2":1}+```++The decision counters show one `fall_open` choice and one `override`. Here, `fall_open` means the request had no usable signals and used the efficient-first default. `override` means a hard error forced the capable tier. Additional requests change the totals. Counters reset on server restart or `POST /v1/stats/reset`.++For per-request records, restart with `--routing-log-file routing.jsonl`. Each record includes the selected model and token usage. Decision reasons are reported by the stats endpoint. These checks confirm routing behavior, not task quality or savings.++## Compare configurations++- [Benchmark results](../benchmarks.md) link to model pools, results, and runnable profiles. For Terminal-Bench 2.1, the [escalation profile](../../benchmark/routing-profiles/tb21-escalation-opus-glm-deepseek.toml) includes its judge and reasoning settings. Its comments explain the endpoint differences from the measured runs.+- [Benchmark setup](../../benchmark/README.md) explains how to compare routing against single-model baselines on the same tasks.+- [Routing overview](../routing_algorithms/overview.md) explains Auto, Task, Execution, and Composite. A TOML file can expose multiple routes through different route IDs.+- [Server reference](../../crates/switchyard-server/README.md) covers configuration, TLS, logging, and metrics.++## Troubleshooting++| Symptom | Check |+| --- | --- |+| Missing credential | Set the environment variable named by `api_key_env` in the server's terminal. Run `--dry-run` again. |+| Provider authentication or model error | Confirm the key grants access to the exact configured model IDs and endpoint. A successful dry run does not check this. |+| Connection refused | Check the server terminal and `curl -fsS http://127.0.0.1:4000/health`. |+| Every fresh chat selects the efficient model | Expected for this preset without tool signals. Use the escalation request above. |++Stop the server with Ctrl+C when finished.diff --git a/docs/operations/troubleshooting.md b/docs/operations/troubleshooting.mdindex 3d54ae117..4a912d469 100644--- a/docs/operations/troubleshooting.md+++ b/docs/operations/troubleshooting.md@@ -9,7 +9,7 @@ log field, a log line, or a `--dry-run` message.| Symptom | Check | Likely causes, most common first | Fix ||---|---|---|---|-| The first request lands on the efficient model | `x-model-router-selected-model` on the response | Auto and Execution decide from tool results, and a request without tool history has none. This is by design. | Send a request with a critical tool error, as in [Verify routing is active](../getting_started.md#verify-routing-is-active). |+| The first request lands on the efficient model | `x-model-router-selected-model` on the response | Auto and Execution decide from tool results, and a request without tool history has none. This is by design. | Send a request with a critical tool error, as in [Verify routing is active](local_evaluation.md#verify-routing-is-active). || Every request lands on one model | `curl -s localhost:4000/v1/stats | jq -c '.models | map_values(.calls)'` and `.algorithm_stats.stage_router.routing_decisions` | Chat-only traffic (`fall_open` only); both tiers name the same target or the same model ID; `capable_first` with a threshold the signals cannot reach; one tier is down and the other serves its fallbacks. | See the matching rows in [Silent misconfigurations](#silent-misconfigurations) and [When Auto does not route](../routing_algorithms/overview.md#when-auto-does-not-route). || The judge never changes the outcome | `switchyard_classifier_fail_open_total` on `/metrics`, and the log line `judge verdict unavailable; routing without one` with its `reason` | The provider rejects JSON Schema mode (`json_object` needed); the verdict is truncated by `max_output_tokens`; the judge model returns prose instead of JSON. | [Judge model compatibility](../routing_algorithms/llm_classifier_routing.md#judge-model-compatibility). || Escalation never latches | The log line `stateful escalation has no session ID; routing state will not persist` | The client sends no session ID. No escalation state survives a request without one, so the route never latches at any `confirmations`. The warning fires only when `confirmations` is 2 or more. | Send `x-switchyard-session-id`. See [Tuning options](../routing_algorithms/escalation_router_routing.md#tuning-options). |@@ -22,7 +22,7 @@ log field, a log line, or a `--dry-run` message.| The client shows zero input tokens on `/v1/messages` | `prompt_tokens` in the routing log | The streamed `message_start` event reports zero usage when the target is on an OpenAI format. The real counts arrive in `message_delta`. | [Streamed token counts](../routing_algorithms/model_selection.md#pin-the-provider-and-endpoint). || Cost per run is higher than the single capable model | `cached_tokens` in the routing log on the turn after a model change; cost per arm | Each switch loses the prompt cache; the judge adds a call per turn; the efficient model is too weak and most turns escalate. | [Counter-examples](../routing_algorithms/model_selection.md#counter-examples) and the [inverted-pair check](../routing_algorithms/model_selection.md#check-for-an-inverted-pair). || The prompt cache is never read | `cached_tokens` stays `0` on repeated prompts | The gateway does not cache on that endpoint, or the prompt is too short to cache. | [Prompt caching](../integrations/pi.md#prompt-caching). |-| `/v1/stats` has no `tiers` key | The `/v1/stats` body | The Rust server reports decisions under `algorithm_stats.<algorithm>`. | Read `algorithm_stats.stage_router.routing_decisions`, as in [Verify routing is active](../getting_started.md#verify-routing-is-active). |+| `/v1/stats` has no `tiers` key | The `/v1/stats` body | The Rust server reports decisions under `algorithm_stats.<algorithm>`. | Read `algorithm_stats.stage_router.routing_decisions`, as in [Verify routing is active](local_evaluation.md#verify-routing-is-active). || The routing log does not say why a model was chosen | A routing-log record | The log records the served model and token counts only. | Use the decision counts in `/v1/stats`. For overrides, the `stage router override` log line also names the reason. See [Decision sources](../routing_algorithms/stage_router_routing.md#decision-sources). || Sub-agent requests ignore the `subagents` policy | The log line `this route has sub-agent routing but the calling harness does not send sub-agent identity`. It fires only for Claude Code versions known to lack child identity. For other harnesses, check whether delegated requests carry a session ID and an agent ID. | The harness sends no sub-agent identity, so delegated requests follow the parent route. | [Sub-Agent-Aware Routing](../routing_algorithms/subagent_routing.md). |@@ -81,7 +81,7 @@ confidence_threshold = 0.9 # the signals reach 0.8741 at most toward efficien## Related-- [Verify routing is active](../getting_started.md#verify-routing-is-active)+- [Verify routing is active](local_evaluation.md#verify-routing-is-active)- [Tuning](../tuning/index.md)- [Model Selection](../routing_algorithms/model_selection.md)- [TOML Schema](../reference/toml_schema.md)diff --git a/docs/reference/host_integration.md b/docs/reference/host_integration.mdnew file mode 100644index 000000000..60db01d5b--- /dev/null+++ b/docs/reference/host_integration.md@@ -0,0 +1,155 @@+# Host Integration++Use this guide when embedding libsy algorithms that ask the host to serve model or decision calls. For a first routing result and model-client handoff, start with the [libsy quickstart](../getting_started.md).++## Development API++This page uses the development API, including its asynchronous `respond` method. The Rust examples pin the same revision as the [quickstart](../getting_started.md). The released `v0.3.0` callback API is synchronous. Use matching source-built [Python bindings](../../INSTALLATION.md#development) for this page's Python callback example.++## Route in Rust++With Rust 1.96.1+ and a native build toolchain, create a project:++```bash+cargo new switchyard-rust-demo+cd switchyard-rust-demo+```++Add these dependencies to `Cargo.toml`. Both Switchyard crates use the same pinned revision:++```toml+[dependencies]+async-trait = "0.1"+futures = "0.3"+switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git", rev = "5415db205c1f4db7e079731e7bf33a6609c67c8e" }+switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git", rev = "5415db205c1f4db7e079731e7bf33a6609c67c8e" }+tokio = { version = "1", features = ["macros", "rt"] }+```++Replace `src/main.rs` with this complete routing-only example:++```rust+use std::{collections::HashMap, sync::Arc};++use futures::StreamExt;+use switchyard_libsy::{+ Algorithm, PickerMode, RuntimeModels, StageRouter, StageRouterConfig, State, Step,+};+use switchyard_protocol::{Category, LlmRequest, Message, Request, Role};++#[tokio::main(flavor = "current_thread")]+async fn main() -> Result<(), Box<dyn std::error::Error>> {+ let router = Arc::new(StageRouter::new(StageRouterConfig::new(+ PickerMode::EfficientFirst, 0.5,+ ))?);+ let models = Arc::new(RuntimeModels::new(HashMap::from([+ (Category::Efficient, vec!["Efficient".into()]),+ (Category::Capable, vec!["Capable".into()]),+ (Category::Any, vec!["Efficient".into(), "Capable".into()]),+ ])));+ let request = Request {+ llm_request: LlmRequest {+ messages: vec![Message::text(Role::User, "Hello")],+ ..Default::default()+ },+ ..Default::default()+ };+ let mut steps = router.run_stream(request, models, State::default());+ while let Some(step) = steps.next().await {+ match step? {+ Step::Done(outcome) => println!("Selected: {}", outcome.selected_model_ids[0]),+ _ => unreachable!("This Stage configuration makes no routing-time calls"),+ }+ }+ Ok(())+}+```++Run `cargo run`. It prints `Selected: Efficient` without making a network call. This example uses the pinned development API, including its `State` argument. For a complete routing and client example, follow the [Rust quickstart](../getting_started.md#use-libsy-with-your-client). Use [switchyard-translation](../../crates/switchyard-translation/README.md) to convert provider requests and libsy's protocol types, or [switchyard-llm-client](../../crates/libsy-llm-client/README.md) to drive an algorithm over HTTP.++## Serve routing-time calls++An algorithm yields a stream of steps. Each `Step::CallModel` is a routing-time classifier or judge call your host performs over its own transport. The run ends with `Step::Done` carrying a `RoutingOutcome`: the selected model, ordered fallbacks, rewritten request, and an optional response when routing already produced the answer. Otherwise the host makes the terminal answer call from that outcome.++Pass each call's work to `respond` before awaiting it. In Rust:++```rust+use switchyard_libsy::{CallDecision, CallModel, LibsyError, Result};+use switchyard_protocol::{RoutedDecisionClient, RoutedLlmClient};++async fn serve(call: CallModel, client: &dyn RoutedLlmClient) -> Result<()> {+ let request = call.request.clone();+ let model = call.models[0].clone();+ call.respond(async move {+ client.call(request).await.map_err(|error| LibsyError::client_call(model, error))+ }).await+}++async fn serve_decision(call: CallDecision, client: &dyn RoutedDecisionClient) -> Result<()> {+ let request = call.request.clone();+ let model = call.model.clone();+ call.respond(async move {+ client.call(request).await.map_err(|error| LibsyError::client_call(model, error))+ }).await+}+```++These traits belong to `switchyard-protocol`. Implement them over your own transport or SDK. The quickstart's custom [OpenRouter client](../getting_started.md#use-libsy-with-your-client) calls HTTP directly; `run_stream` also works without implementing these traits. Switchyard's HTTP client is optional.++In Python, the client coroutine returns `LlmResponse.Agg` or `LlmResponse.Stream`:++```python+await call.respond(client.call(call.request, call.models[0]))+```++`respond` accepts a future or awaitable instead of a completed response. Move the provider await into this method. Python `call.fail(error)` is also awaitable. Decision calls use the same pattern with their existing decision response type.++## Concurrent calls and cancellation++In Rust, `respond` drops the supplied work and returns `Ok(())` when the algorithm stops waiting for that call. Dropping a pending `respond` or `libsy::drive` future also drops its host work. Use `libsy::drive` to manage concurrent handlers and their lifetime.++Python `respond` cancels and awaits the supplied provider work before returning. For concurrent algorithms, the host creates handler tasks with `asyncio.create_task` and must cancel and await remaining handlers when the run ends; see the complete [Python example](../../examples/libsy.py). Awaiting handlers one at a time inside the step loop serializes their execution.++After a response stream is delivered, its consumer controls its lifetime. Cancellation cannot guarantee that an upstream provider stops generating or billing.++## Metadata and telemetry++Successful runs also include `OutcomeMetadata`: a unique `outcome_id`, the algorithm name, and optional JSON evidence. In Python, read `outcome.metadata.outcome_id`, `outcome.metadata.algorithm`, and `outcome.metadata.evidence` after checking that `outcome.metadata` is present. Evidence is a normal Python value, usually a dictionary; algorithms without evidence return `None`.++With a host-installed OpenTelemetry subscriber, the existing `libsy.run` span records the same identity, selected models, and supported evidence fields. See the [OpenTelemetry reference](opentelemetry.md) for field names, metrics, and export setup. libsy does not install an exporter or send telemetry itself.++For the request, response, and streaming types the steps carry, see [`switchyard-protocol`](../../crates/protocol/README.md).++For plan/execute, supply conversation history, including tool calls and results, before routing. Responses API continuation IDs alone are not enough. See the [Responses API history requirement](../routing_algorithms/plan_execute_routing.md#responses-api-history-requirement).++## Python bindings++Python bindings run the same Rust algorithms. With Python 3.10+ and [uv](https://docs.astral.sh/uv/getting-started/installation/), create a project and install the released package:++```bash+uv init --bare switchyard-python-demo+cd switchyard-python-demo+uv add "nemo-switchyard==0.3.0"+```++Save as `route.py`:++```python+import asyncio+from switchyard.libsy import Step, algorithms++async def main():+ router = algorithms.stage_router(+ picker="efficient_first", confidence_threshold=0.5)+ models = {"efficient": ["Efficient"], "capable": ["Capable"]}+ models["any"] = ["Efficient", "Capable"]+ text = {"type": "text", "text": "Hello"}+ request = {"messages": [{"role": "user", "content": [text]}]}+ async for step in router.run_stream(request, models):+ if isinstance(step, Step.Done):+ print("Selected:", step.outcome.selected_model_ids[0])++asyncio.run(main())+```++`uv run route.py` prints `Selected: Efficient` without making a network call. This routing-only example works with the released package. For the development API's host callbacks and cancellation, build the matching bindings from source as described at the top of this page, then see [the complete host example](../../examples/libsy.py).diff --git a/docs/routing_algorithms/escalation_router_routing.md b/docs/routing_algorithms/escalation_router_routing.mdindex 5a1209780..baae15401 100644--- a/docs/routing_algorithms/escalation_router_routing.md+++ b/docs/routing_algorithms/escalation_router_routing.md@@ -191,7 +191,7 @@ verdict without new evidence resets the streak to zero.## Run the routeAfter installing the Rust server, as described in-[Getting Started](../getting_started.md#install-the-server), export the provider+[Local Evaluation](../operations/local_evaluation.md#install-the-server), export the providercredential, validate the configuration, and start the binary:```bashdiff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.mdindex ff4c3cd4f..45a1a6a0e 100644--- a/docs/routing_algorithms/llm_classifier_routing.md+++ b/docs/routing_algorithms/llm_classifier_routing.md@@ -313,7 +313,7 @@ state and to keep dynamic routing across turns.## Run the route-After [installing the Rust server](../getting_started.md#install-the-server), export+After [installing the Rust server](../operations/local_evaluation.md#install-the-server), exportthe provider credential, validate the configuration, and start the releasebinary:diff --git a/docs/routing_algorithms/model_selection.md b/docs/routing_algorithms/model_selection.mdindex 519942617..396c7068f 100644--- a/docs/routing_algorithms/model_selection.md+++ b/docs/routing_algorithms/model_selection.md@@ -37,14 +37,14 @@ What each role needs:Run these checks before you measure routing. Add a `passthrough` route for theefficient model to the file from-[Getting Started](../getting_started.md#server-path), start the server with+[Local Evaluation](../operations/local_evaluation.md), start the server with`--routing-log-file routing.jsonl`, and send the checks to that route:```toml[routes.efficient_only]id = "efficient-only"type = "passthrough"-target = "weak"+target = "efficient"```### 1. Tool calling@@ -295,7 +295,7 @@ Record it with every result. Three rules apply:## Related- [Routing Overview](overview.md): choose a strategy and read what Auto resolves to.-- [Verify routing is active](../getting_started.md#verify-routing-is-active): see the served model and the decision counts.+- [Verify routing is active](../operations/local_evaluation.md#verify-routing-is-active): see the served model and the decision counts.- [Stage Router](stage_router_routing.md#tuning-confidence_threshold): tune the threshold once the pair is right.- [Escalation-Router Routing](escalation_router_routing.md#judge-model-compatibility): judge requirements for escalation routes.- [Benchmarks](../../benchmark/README.md): how the measured runs were made.diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.mdindex f0fd2bb3c..2767579e5 100644--- a/docs/routing_algorithms/overview.md+++ b/docs/routing_algorithms/overview.md@@ -50,7 +50,7 @@ quality, follow the [tuning loop](../tuning/index.md).| Route `type` | Decides from | Extra model calls | Switches | Session ID | When it cannot decide | Measured | Start from ||---|---|---|---|---|---|---|---|-| `auto` | Tool results in the request | None | Every request | Optional. Keeps the capable hold. | Efficient tier | The preset of the measured `stage_router` settings, with a shorter window. See [Auto](#auto). | The [Getting Started](../getting_started.md#server-path) route file |+| `auto` | Tool results in the request | None | Every request | Optional. Keeps the capable hold. | Efficient tier | The preset of the measured `stage_router` settings, with a shorter window. See [Auto](#auto). | The [local evaluation](../operations/local_evaluation.md#configure-your-model-pool) route file || `stage_router` | Tool results in the request | Optional judge on ambiguous turns | Every request | Optional. Keeps the capable hold. | Picker default | [Terminal-Bench 2.1 and DeepSWE results](../benchmarks.md) with the [DeepSWE v1.1 profile](../../benchmark/routing-profiles/deepswe-v11-stage-router-luna-sol.toml) | That profile || `llm_classifier`, capability | A judge's view of the task | One judge call per request, user turn, or session | Per trigger | Optional. Needed to retain a verdict. | Strong target | [Terminal-Bench 2.1 results](../benchmarks.md#terminal-bench-21) for an LLM judge and for Jev as a decision judge. The LLM-judge config is not published; a [smoke-test config](../../benchmark/server-configs/tb-lite-llm-classifier-opus-kimi-gemini.toml) is checked in. | That config || `llm_classifier`, escalation | A judge's view of the finished weak turn | One judge call per unlatched turn | Latches once. Reversible with de-escalation. | Required to latch. Without it, `confirmations = 1` escalates single turns only. | An unreadable verdict keeps the current tier. A failed judge call stops the request in the server. | [Terminal-Bench 2.1 results](../benchmarks.md#terminal-bench-21) with the [checked-in profile](../../benchmark/routing-profiles/tb21-escalation-opus-glm-deepseek.toml), and a [public report](https://www.langchain.com/blog/switchyard-agent-routing-benchmark) | That profile |@@ -173,7 +173,7 @@ to one tier:of an agent session, carry no tool history. The signals abstain and theefficient tier serves the turn. This is by design. `GET /v1/stats` counts theseturns under `algorithm_stats.stage_router.routing_decisions.fall_open`. See- [Verify routing is active](../getting_started.md#verify-routing-is-active).+ [Verify routing is active](../operations/local_evaluation.md#verify-routing-is-active).- **Both tiers name the same target.** The server accepts a route whose`capable_target` and `efficient_target` are the same target. Every request thengoes to that one model, and `GET /v1/stats` lists a single entry under `.models`.@@ -287,7 +287,7 @@ the configured endpoint.## Run a deploymentAfter installing the Rust server, as described in-[Getting Started](../getting_started.md#install-the-server), export the provider+[Local Evaluation](../operations/local_evaluation.md#install-the-server), export the providercredential, validate the configuration, and start the binary:```bashdiff --git a/docs/routing_algorithms/random_routing.md b/docs/routing_algorithms/random_routing.mdindex a90422fc5..a353c9b12 100644--- a/docs/routing_algorithms/random_routing.md+++ b/docs/routing_algorithms/random_routing.md@@ -77,7 +77,7 @@ therefore use different targets.## Run the route-After [installing the Rust server](../getting_started.md#install-the-server), export+After [installing the Rust server](../operations/local_evaluation.md#install-the-server), exportthe provider credential, validate the configuration, and start the releasebinary:diff --git a/docs/tuning/index.md b/docs/tuning/index.mdindex f4d1d740e..e3c0480a0 100644--- a/docs/tuning/index.md+++ b/docs/tuning/index.md@@ -15,7 +15,7 @@ Start where your goal is:## The loop1. **Verify routing is active.** Follow- [Verify routing is active](../getting_started.md#verify-routing-is-active).+ [Verify routing is active](../operations/local_evaluation.md#verify-routing-is-active).Produces: the selected-model header from two requests and a `/v1/stats`capture. Costs: two requests, no agent run.diff --git a/mkdocs.yml b/mkdocs.ymlindex 0de16cb87..376fd9333 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -1,5 +1,5 @@site_name: Switchyard-site_description: Typed, composable LLM routing and format translation for Python+site_description: Embed libsy, Switchyard's Rust LLM routing library, in your gateway or agent harnessrepo_url: https://github.com/NVIDIA-NeMo/Switchyardrepo_name: NVIDIA-NeMo/Switchyardedit_uri: edit/main/docs/@@ -19,6 +19,7 @@ nav:- Home: index.md- Get Started:- Getting Started: getting_started.md+ - Evaluate Locally: operations/local_evaluation.md- Known Issues: known_issues.md- Benchmarks: benchmarks.md- Concepts:@@ -49,6 +50,7 @@ nav:- Context-Window Handling: operations/context_window.md- Soak Testing: operations/soak_test.md- Reference:+ - Host Integration: reference/host_integration.md- Server CLI Reference: cli_reference.md- TOML Schema: reference/toml_schema.md- OpenTelemetry: reference/opentelemetry.mddiff --git a/tests/getting_started/test_getting_started.py b/tests/getting_started/test_getting_started.pyindex fb428324d..3095a48df 100644--- a/tests/getting_started/test_getting_started.py+++ b/tests/getting_started/test_getting_started.py@@ -7,23 +7,51 @@ from pathlib import Pathdef test_getting_started_documents_current_paths() -> None:- guide = (- Path(__file__).resolve().parents[2] / "docs" / "getting_started.md"- ).read_text()+ root = Path(__file__).resolve().parents[2]+ guide = (root / "docs" / "getting_started.md").read_text()+ readme = (root / "README.md").read_text()+ server = (root / "docs" / "operations" / "local_evaluation.md").read_text()- assert guide.index("## Server Path") < guide.index("## Library Path")+ assert (+ guide.index("## Route a request")+ < guide.index("## Use libsy with your client")+ < guide.index("## Next steps")+ )+ example = guide.split("```rust\n", 1)[1].split("```", 1)[0]+ for step in ("CallModel", "CallDecision", "Done"):+ assert f"Step::{step}(" in readme+ assert f"Step::{step}(" in example+ readme_example = readme.split("```rust\n", 1)[1].split("```", 1)[0]+ assert not any(line.startswith("use ") for line in readme_example.splitlines())+ for placeholder in ("call_model(", "call_decision(", "handle_outcome("):+ assert placeholder not in readme_example+ assert "cargo add switchyard-libsy switchyard-protocol --git" in guide+ assert "--rev 5415db205c1f4db7e079731e7bf33a6609c67c8e" in guide+ assert "call.respond(client.call(request)).await?" in guide+ assert "struct OpenRouterClient" in guide+ assert '.post("https://openrouter.ai/api/v1/chat/completions")' in guide+ assert "You do not have to use Switchyard's HTTP client." in guide+ assert "cargo add switchyard-llm-client" not in guide+ assert "client.call(outcome.request).await?" in guide+ assert "```python" not in guide + readme+ assert "Selected: Efficient" in guide+ assert "Selected: Capable" in guide+ assert "## See routing change" not in guide+ assert "operations/local_evaluation.md" in guideassert "switchyard launch" not in guideassert "nemo-switchyard[cli]" not in guide- assert "cargo install --locked switchyard-server" in guide- assert "switchyard-server --config routes.toml --dry-run" in guide+ assert "cargo install --locked switchyard-server" not in guide+ assert "cargo install --locked switchyard-server --version 0.3.0" in server+ assert "switchyard-server --config routes.toml --dry-run" in server+ assert "not a supported production deployment path" in serverassert (- guide.index("### Run the server")- < guide.index("### Verify routing is active")- < guide.index("### Routing algorithms")+ server.index("## Run the server")+ < server.index("## Verify routing is active")+ < server.index("## Compare configurations"))- assert "x-model-router-selected-model" in guide- assert "/v1/stats" in guide- assert "--routing-log-file" in guide+ assert "x-model-router-selected-model" in server+ assert "/v1/stats" in server+ assert "--routing-log-file" in serverfor legacy_command in ("switchyard configure",@@ -33,8 +61,7 @@ def test_getting_started_documents_current_paths() -> None:"--routing-profiles","routes.yaml",):- assert legacy_command not in guide+ assert legacy_command not in guide + server-# TODO: Add Python snippet coverage when the supported Rust-backed API is finalized.-# TODO: Add TOML schema coverage if the guide grows beyond one maintained example.+# TODO: Execute the documented Rust examples in isolated CI environments.