Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
site_name: Switchyard
site_description: Embed libsy, Switchyard's Rust LLM routing library, in your gateway or agent harness
repo_url: https://github.com/NVIDIA-NeMo/Switchyard
repo_name: NVIDIA-NeMo/Switchyard
edit_uri: edit/main/docs/
extra:
source_ref: !ENV [MKDOCS_SOURCE_REF, main]
copyright: "Copyright © 2025-2026 NVIDIA Corporation"
docs_dir: docs
site_dir: site
use_directory_urls: true
strict: true
hooks:
- mkdocs_hooks.py
nav:
- Home: index.md
- Get Started:
- Getting Started: getting_started.md
- Integrate Switchyard:
- Mechanics of Libsy: mechanics_of_libsy.md
- Switchyard Protocols:
- protocols/index.md
- LLM-Related Protocols: protocols/llm.md
- Decision Model-Related Protocols: protocols/decisions.md
- Embed in Your Gateway or Harness: reference/host_integration.md
- Gateways:
- NeMo Relay: integrations/nemo_relay.md
- Routing:
- Algorithms:
- Overview: routing_algorithms/overview.md
- Task (LLM Classifier): routing_algorithms/llm_classifier_routing.md
- Execution (Stage Router): routing_algorithms/stage_router_routing.md
- Sub-Agent-Aware Routing: routing_algorithms/subagent_routing.md
- Random Routing: routing_algorithms/random_routing.md
- Composite Routing: routing_algorithms/composite_routing.md
- Escalation-Router Routing: routing_algorithms/escalation_router_routing.md
- Advisor-Gate Routing: routing_algorithms/advisor_gate_routing.md
- Verification-Gated Routing: routing_algorithms/vgr_routing.md
- Choose Models: routing_algorithms/model_selection.md
- Evaluate and Tune:
- Benchmarks: benchmarks.md
- Tuning:
- tuning/index.md
- Foundations: tuning/foundations.md
- Examples: tuning/examples.md
- Local Demo:
- Run the Reference Server: operations/local_evaluation.md
- Local Agents:
- Claude Code and Codex: recipes/single_provider_coding_agents.md
- pi: integrations/pi.md
- Oh My Pi: integrations/oh_my_pi.md
- Reference:
- API and Model Compatibility: reference/compatibility.md
- Protocols:
- LLM:
- Requests: reference/protocols/llm_requests.md
- Responses and Streams: reference/protocols/llm_responses.md
- Translation: reference/protocols/translation.md
- Decision Models:
- Questions: reference/protocols/decision_questions.md
- Answers: reference/protocols/decision_answers.md
- Runner Configuration (TOML): reference/toml_schema.md
- OpenTelemetry: reference/opentelemetry.md
- Rust API: reference/rust_api.md
- Context-Window Handling: operations/context_window.md
- Troubleshooting: operations/troubleshooting.md
- Known Issues: known_issues.md
- Demo Server:
- CLI: cli_reference.md
- Soak Testing: operations/soak_test.md
exclude_docs: |
internal/**
research/**
vllm-serve-hidden-state.md
theme:
name: material
palette:
- media: "(prefers-color-scheme: light)"
scheme: default
primary: custom
accent: custom
toggle:
icon: material/weather-sunny
name: Switch to dark mode
- media: "(prefers-color-scheme: dark)"
scheme: slate
primary: custom
accent: custom
toggle:
icon: material/weather-night
name: Switch to light mode
font:
text: Inter
code: JetBrains Mono
features:
- navigation.top
- navigation.sections
- navigation.indexes
- search.suggest
- search.highlight
- content.code.copy
- toc.follow
plugins:
- search:
lang: en
markdown_extensions:
- admonition
- pymdownx.details
- pymdownx.superfences:
custom_fences:
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
- pymdownx.highlight:
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.inlinehilite
- pymdownx.snippets:
base_path: [docs]
check_paths: true
- pymdownx.emoji:
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:material.extensions.emoji.to_svg
- attr_list
- md_in_html
- tables
- toc:
permalink: true
title: On this page
- pymdownx.tasklist:
custom_checkbox: true
extra_css:
- stylesheets/nvidia.css