Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust
2.3 KB
Raw Blame

Installation Guide

Switchyard has separate packages for Python integrations and standalone Rust serving.

Requirements

  • Python 3.10 or newer for nemo-switchyard
  • Rust 1.96.1 or newer for switchyard-server and the Rust libraries
  • Linux x86_64 wheels require an x86-64-v3 / AVX2-class CPU
  • Linux aarch64 wheels require a Neoverse N1-class CPU

Python Bindings

Install the Python package to embed libsy algorithms or host the native server through PyO3:

pip install nemo-switchyard

The base package has no Python runtime dependencies. Its native extension owns the libsy and server implementations.

Standalone Server

Install the native Rust proxy from crates.io:

cargo install --locked switchyard-server
switchyard-server --config routes.toml --dry-run
switchyard-server --config routes.toml --port 4000

See Local Evaluation for a complete TOML example and switchyard-server for the configuration reference.

Experimental desktop app

The desktop app, terminal app, and their installer live in experimental/switchyard-desktop. They are outside Switchyard release QA and have no support or compatibility guarantee. The installer also sets up the server as a user service and the Linux Codex service. It does not save API keys: the installed server starts only after you save them in the app's Settings → Server keys. See the desktop installation guide.

Rust Libraries

Add the crates needed by an embedded application:

[dependencies]
switchyard-libsy = "0.2.0"
switchyard-protocol = "0.2.0"
switchyard-llm-client = "0.2.0"
switchyard-translation = "0.2.0"

switchyard-libsy owns algorithms, switchyard-protocol owns provider-neutral request and response types, switchyard-translation owns wire conversion, and switchyard-llm-client performs translated HTTP calls.

Development

From a checkout:

uv sync
uv run maturin develop
cargo test --workspace
uv run pytest tests/ -v

The dev dependency group contains testing and linting tools and is not exposed in the published wheel metadata.

See the experimental desktop and terminal quickstart for first-run setup, reviewed settings changes, usage, recovery, and safe uninstall.