Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs: correct AUTO probe order, extras, saved-bundle note, and stale API references (#110)
Elyas Mehtabuddinb28bcf6fd240
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 07187269d..895befa69 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -34,14 +34,16 @@ traffic that sits between client applications and LLM backends.- **Observability** — Prometheus `/metrics`, a JSON `/v1/stats`(`/v1/routing/stats` alias), and per-request cost/token/latency stats. See[Metrics Reference](docs/METRICS_REFERENCE.md).-- **Python library** — `SwitchyardRecipes` (`passthrough_recipe`,- `random_routing_recipe`, `stage_router_recipe`, `deterministic_routing_recipe`,- …) and typed `ChatRequest` / `ChatResponse` containers for in-process use.+- **Python library** — `ProfileSwitchyard` driven by typed profile configs+ (`PassthroughProfileConfig`, `RandomRoutingProfileConfig`,+ `StageRouterProfileConfig`, …) and typed `ChatRequest` / `ChatResponse`+ containers for in-process use.- **Rust core** (PyO3) — chain execution, the latency-aware router, and thetool-result signal collector are implemented in Rust and re-exported toPython.- **Packaging** — `pip install nemo-switchyard` with optional extras `[server]`,- `[cli]`, `[gpu]`, `[all]`. See [Installation](INSTALLATION.md).+ `[cli]`, `[tracing]`, `[intake]`, `[affinity-redis]`, `[all]`. See+ [Installation](INSTALLATION.md).### Deprecateddiff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 8213bed5a..bc7a44ab3 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -171,7 +171,7 @@ See [Agents](AGENTS.md) for the full architecture guide. Key points:- **Typed requests/responses** — use `ChatRequest` and `ChatResponse` subtypes- **Composable chain** — `RequestProcessor` → `LLMBackend` → `ResponseProcessor` → `TranslationEngine`-- **Recipes** — pre-built chains in `switchyard/lib/recipes.py`+- **Profiles** — pre-built chains in `switchyard/lib/profiles/`When adding a new component:diff --git a/DEVELOPMENT.md b/DEVELOPMENT.mdindex 12b1245f5..7c92bb70f 100644--- a/DEVELOPMENT.md+++ b/DEVELOPMENT.md@@ -46,7 +46,7 @@ switchyard/│ ├── lib/ # Core library│ │ ├── roles.py # RequestProcessor, LLMBackend, ResponseProcessor ABCs│ │ ├── switchyard.py # Switchyard chain executor-│ │ ├── recipes.py # SwitchyardRecipes (passthrough, random_routing, …)+│ │ ├── profiles/ # Profile configs/runtimes (passthrough, random_routing, …)│ │ ├── proxy_context.py # ProxyContext — per-request state│ │ ├── chat_request/ # Typed request hierarchy│ │ ├── chat_response/ # Typed response hierarchydiff --git a/INSTALLATION.md b/INSTALLATION.mdindex 270c450a0..a6f03e423 100644--- a/INSTALLATION.md+++ b/INSTALLATION.md@@ -67,7 +67,7 @@ All optional dependencies for complete feature set:pip install nemo-switchyard[all]```-Equivalent to: `switchyard[server,cli]`+Equivalent to: `nemo-switchyard[server,cli,tracing,intake,affinity-redis]`### Common Combinations@@ -114,13 +114,13 @@ pip install nemo-switchyard[all]Embed Switchyard with minimal overhead:```python-from switchyard import SwitchyardRecipes+from switchyard import PassthroughProfileConfig, ProfileSwitchyard# Core library only — no server/CLI dependencies-switchyard = SwitchyardRecipes.passthrough_recipe(+switchyard = ProfileSwitchyard(PassthroughProfileConfig(api_key="sk-...",base_url="https://api.openai.com/v1",-)+).build())```## Troubleshootingdiff --git a/README.md b/README.mdindex 5e1b696d4..46ae31bbf 100644--- a/README.md+++ b/README.md@@ -200,7 +200,7 @@ Optional extras:```bashpip install "nemo-switchyard[server]" # FastAPI / Uvicorn HTTP endpointspip install "nemo-switchyard[cli]" # Interactive CLI launchers (Claude / Codex)-pip install "nemo-switchyard[all]" # Server, CLI, and tracing extras+pip install "nemo-switchyard[all]" # Server, CLI, tracing, intake, and affinity-redis extras```See [Installation](INSTALLATION.md) for a full breakdown of what each extra adds.diff --git a/docs/cli_reference.md b/docs/cli_reference.mdindex b08ebc714..618942b25 100644--- a/docs/cli_reference.md+++ b/docs/cli_reference.md@@ -50,9 +50,10 @@ upstream requests. Configuration files use these lowercase values:`auto` resolves formats in this order:-1. Probe `/v1/messages`; use `anthropic` when supported.-2. Probe `/v1/responses`; use `responses` when supported.-3. Fall back to `openai` and `/v1/chat/completions`.+1. Probe `/v1/chat/completions`; use `openai` when supported.+2. Probe `/v1/messages`; use `anthropic` when supported.+3. Probe `/v1/responses`; use `responses` when supported.+4. Fall back to `openai` (`/v1/chat/completions`).Single-model Claude Code and Codex launches use `auto`. OpenClaw is pinned to`openai`. Prefer an explicit format when the upstream contract is known so@@ -437,6 +438,13 @@ switchyard [--routing-profiles PATH] configure [--show [--check] [--json] | --re| `--no-tui` | Use plain text prompts instead of the TUI selector. || `--check` | With `--show`, call `GET /models` against the resolved provider and report pass/fail in the output. |+> **Saved bundles keep `${VAR}` references literal.** A saved routing-profile+> bundle stores `${OPENROUTER_API_KEY}` (and any other `${VAR}`) verbatim. The+> referenced environment variables must therefore+> be present in the environment at `serve` / `launch` time; on another machine or+> shell, export them again or Switchyard aborts with+> `missing environment variable(s): NAME`.+**Skill distillation config**```jsondiff --git a/examples/minimal.py b/examples/minimal.pyindex a78abe053..26555b07b 100644--- a/examples/minimal.py+++ b/examples/minimal.py@@ -25,7 +25,7 @@ from pathlib import Path# Add package to path for development (not needed when installed via pip)sys.path.insert(0, str(Path(__file__).parent.parent))-from switchyard import ChatRequest, SwitchyardRecipes+from switchyard import ChatRequest, PassthroughProfileConfig, ProfileSwitchyardasync def main():@@ -39,10 +39,10 @@ async def main():return# Create a passthrough proxy that forwards to OpenAI- switchyard = SwitchyardRecipes.passthrough_recipe(+ switchyard = ProfileSwitchyard(PassthroughProfileConfig(api_key=api_key,base_url="https://api.openai.com/v1",- )+ ).build())print("=" * 60)print("Switchyard Minimal Example")@@ -64,8 +64,8 @@ async def main():response = await switchyard.call(request)print("\nResponse:")- print(f" Content: {response.body['choices'][0]['message']['content']}")- print(f" Tokens: {response.body['usage']['total_tokens']}")+ print(f" Content: {response['choices'][0]['message']['content']}")+ print(f" Tokens: {response['usage']['total_tokens']}")print("\n" + "=" * 60)print("Example completed!")diff --git a/examples/route.yaml b/examples/route.yamlindex 952f4190c..9575ab5e1 100644--- a/examples/route.yaml+++ b/examples/route.yaml@@ -24,6 +24,8 @@ routes:llm-classifier: # `type: deterministic` — LLM-as-classifiertype: deterministicprofile: coding_agent # general | coding_agent | openclaw+ fallback_target_on_evict: strong # tier to retry on context-window evict+classifier:model: google/gemini-3.5-flashapi_key: ${OPENROUTER_API_KEY}