Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

docs: correct AUTO probe order, extras, saved-bundle note, and stale API references (#110)

Elyas Mehtabuddinb28bcf6fd240
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 07187269d..895befa69 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -34,14 +34,16 @@ traffic that sits between client applications and LLM backends.
- **Observability** — Prometheus `/metrics`, a JSON `/v1/stats`
(`/v1/routing/stats` alias), and per-request cost/token/latency stats. See
[Metrics Reference](docs/METRICS_REFERENCE.md).
-- **Python library** — `SwitchyardRecipes` (`passthrough_recipe`,
- `random_routing_recipe`, `stage_router_recipe`, `deterministic_routing_recipe`,
- …) and typed `ChatRequest` / `ChatResponse` containers for in-process use.
+- **Python library** — `ProfileSwitchyard` driven by typed profile configs
+ (`PassthroughProfileConfig`, `RandomRoutingProfileConfig`,
+ `StageRouterProfileConfig`, …) and typed `ChatRequest` / `ChatResponse`
+ containers for in-process use.
- **Rust core** (PyO3) — chain execution, the latency-aware router, and the
tool-result signal collector are implemented in Rust and re-exported to
Python.
- **Packaging** — `pip install nemo-switchyard` with optional extras `[server]`,
- `[cli]`, `[gpu]`, `[all]`. See [Installation](INSTALLATION.md).
+ `[cli]`, `[tracing]`, `[intake]`, `[affinity-redis]`, `[all]`. See
+ [Installation](INSTALLATION.md).
### Deprecated
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 8213bed5a..bc7a44ab3 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -171,7 +171,7 @@ See [Agents](AGENTS.md) for the full architecture guide. Key points:
- **Typed requests/responses** — use `ChatRequest` and `ChatResponse` subtypes
- **Composable chain** — `RequestProcessor` → `LLMBackend` → `ResponseProcessor` → `TranslationEngine`
-- **Recipes** — pre-built chains in `switchyard/lib/recipes.py`
+- **Profiles** — pre-built chains in `switchyard/lib/profiles/`
When adding a new component:
diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md
index 12b1245f5..7c92bb70f 100644
--- a/DEVELOPMENT.md
+++ b/DEVELOPMENT.md
@@ -46,7 +46,7 @@ switchyard/
│ ├── lib/ # Core library
│ │ ├── roles.py # RequestProcessor, LLMBackend, ResponseProcessor ABCs
│ │ ├── switchyard.py # Switchyard chain executor
-│ │ ├── recipes.py # SwitchyardRecipes (passthrough, random_routing, …)
+│ │ ├── profiles/ # Profile configs/runtimes (passthrough, random_routing, …)
│ │ ├── proxy_context.py # ProxyContext — per-request state
│ │ ├── chat_request/ # Typed request hierarchy
│ │ ├── chat_response/ # Typed response hierarchy
diff --git a/INSTALLATION.md b/INSTALLATION.md
index 270c450a0..a6f03e423 100644
--- a/INSTALLATION.md
+++ b/INSTALLATION.md
@@ -67,7 +67,7 @@ All optional dependencies for complete feature set:
pip install nemo-switchyard[all]
```
-Equivalent to: `switchyard[server,cli]`
+Equivalent to: `nemo-switchyard[server,cli,tracing,intake,affinity-redis]`
### Common Combinations
@@ -114,13 +114,13 @@ pip install nemo-switchyard[all]
Embed Switchyard with minimal overhead:
```python
-from switchyard import SwitchyardRecipes
+from switchyard import PassthroughProfileConfig, ProfileSwitchyard
# Core library only — no server/CLI dependencies
-switchyard = SwitchyardRecipes.passthrough_recipe(
+switchyard = ProfileSwitchyard(PassthroughProfileConfig(
api_key="sk-...",
base_url="https://api.openai.com/v1",
-)
+).build())
```
## Troubleshooting
diff --git a/README.md b/README.md
index 5e1b696d4..46ae31bbf 100644
--- a/README.md
+++ b/README.md
@@ -200,7 +200,7 @@ Optional extras:
```bash
pip install "nemo-switchyard[server]" # FastAPI / Uvicorn HTTP endpoints
pip install "nemo-switchyard[cli]" # Interactive CLI launchers (Claude / Codex)
-pip install "nemo-switchyard[all]" # Server, CLI, and tracing extras
+pip install "nemo-switchyard[all]" # Server, CLI, tracing, intake, and affinity-redis extras
```
See [Installation](INSTALLATION.md) for a full breakdown of what each extra adds.
diff --git a/docs/cli_reference.md b/docs/cli_reference.md
index b08ebc714..618942b25 100644
--- a/docs/cli_reference.md
+++ b/docs/cli_reference.md
@@ -50,9 +50,10 @@ upstream requests. Configuration files use these lowercase values:
`auto` resolves formats in this order:
-1. Probe `/v1/messages`; use `anthropic` when supported.
-2. Probe `/v1/responses`; use `responses` when supported.
-3. Fall back to `openai` and `/v1/chat/completions`.
+1. Probe `/v1/chat/completions`; use `openai` when supported.
+2. Probe `/v1/messages`; use `anthropic` when supported.
+3. Probe `/v1/responses`; use `responses` when supported.
+4. Fall back to `openai` (`/v1/chat/completions`).
Single-model Claude Code and Codex launches use `auto`. OpenClaw is pinned to
`openai`. Prefer an explicit format when the upstream contract is known so
@@ -437,6 +438,13 @@ switchyard [--routing-profiles PATH] configure [--show [--check] [--json] | --re
| `--no-tui` | Use plain text prompts instead of the TUI selector. |
| `--check` | With `--show`, call `GET /models` against the resolved provider and report pass/fail in the output. |
+> **Saved bundles keep `${VAR}` references literal.** A saved routing-profile
+> bundle stores `${OPENROUTER_API_KEY}` (and any other `${VAR}`) verbatim. The
+> referenced environment variables must therefore
+> be present in the environment at `serve` / `launch` time; on another machine or
+> shell, export them again or Switchyard aborts with
+> `missing environment variable(s): NAME`.
+
**Skill distillation config**
```json
diff --git a/examples/minimal.py b/examples/minimal.py
index a78abe053..26555b07b 100644
--- a/examples/minimal.py
+++ b/examples/minimal.py
@@ -25,7 +25,7 @@ from pathlib import Path
# Add package to path for development (not needed when installed via pip)
sys.path.insert(0, str(Path(__file__).parent.parent))
-from switchyard import ChatRequest, SwitchyardRecipes
+from switchyard import ChatRequest, PassthroughProfileConfig, ProfileSwitchyard
async def main():
@@ -39,10 +39,10 @@ async def main():
return
# Create a passthrough proxy that forwards to OpenAI
- switchyard = SwitchyardRecipes.passthrough_recipe(
+ switchyard = ProfileSwitchyard(PassthroughProfileConfig(
api_key=api_key,
base_url="https://api.openai.com/v1",
- )
+ ).build())
print("=" * 60)
print("Switchyard Minimal Example")
@@ -64,8 +64,8 @@ async def main():
response = await switchyard.call(request)
print("\nResponse:")
- print(f" Content: {response.body['choices'][0]['message']['content']}")
- print(f" Tokens: {response.body['usage']['total_tokens']}")
+ print(f" Content: {response['choices'][0]['message']['content']}")
+ print(f" Tokens: {response['usage']['total_tokens']}")
print("\n" + "=" * 60)
print("Example completed!")
diff --git a/examples/route.yaml b/examples/route.yaml
index 952f4190c..9575ab5e1 100644
--- a/examples/route.yaml
+++ b/examples/route.yaml
@@ -24,6 +24,8 @@ routes:
llm-classifier: # `type: deterministic` — LLM-as-classifier
type: deterministic
profile: coding_agent # general | coding_agent | openclaw
+ fallback_target_on_evict: strong # tier to retry on context-window evict
+
classifier:
model: google/gemini-3.5-flash
api_key: ${OPENROUTER_API_KEY}