Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
chore: refactor random routing doc for libsy (#223)
Ayush Agarwal6ff491acd7eb
diff --git a/docs/routing_algorithms/random_routing.md b/docs/routing_algorithms/random_routing.mdindex f81e89b0e..17da19eb0 100644--- a/docs/routing_algorithms/random_routing.md+++ b/docs/routing_algorithms/random_routing.md@@ -1,63 +1,99 @@# Random Routing-Random routing sends a fixed percentage of traffic to a `strong` model and the-rest to a `weak` model. It does not inspect the prompt.--Use it for A/B tests, benchmark baselines, and gradual traffic ramps. If routing-should depend on task difficulty, use-[LLM Classifier Routing](llm_classifier_routing.md) or-[Stage-Router Routing](stage_router_routing.md).--## Algorithm--For each request, Switchyard flips a weighted coin:--- `strong_probability: 0.0` sends all traffic to `weak`.-- `strong_probability: 0.3` sends about 30% to `strong`.-- `strong_probability: 0.5` is an even split.-- `strong_probability: 1.0` sends all traffic to `strong`.--Each request is independent, so short runs may not match the configured-percentage exactly. `rng_seed` is optional and only useful when you need a-repeatable sequence for tests or benchmarks.--## Behavior--Random routing is per request. A multi-turn conversation can move between-models from turn to turn. If you need conversation-level stickiness, use-[Sticky Routing](sticky_routing.md) where it is supported, or select a direct-tier model instead of the routing model.--## Enable It--Use `type: random_routing` under `routes:`:--```yaml-defaults:- api_key: ${OPENROUTER_API_KEY}- base_url: https://openrouter.ai/api/v1- format: openai--routes:- ab-test:- type: random_routing- strong:- model: openai/gpt-4o- weak:- model: openai/gpt-4o-mini- strong_probability: 0.3- rng_seed: 42- fallback_target_on_evict: strong+Random routing selects one configured target for each request without inspecting+the prompt. It supports two or more targets.++Use it for A/B tests, benchmark baselines, and controlled traffic splits. Use a+content-aware routing algorithm instead when the request itself should determine+the target.++## Configure a random route++This example sends approximately 30% of requests to a strong model and 70% to a+weak model:++```toml+schema_version = 1++[llm_clients.openrouter]+format = "openai_chat"+base_url = "https://openrouter.ai/api/v1"+api_key_env = "OPENROUTER_API_KEY"++[targets.strong]+id = "openai/gpt-4o"+llm_client = "openrouter"++[targets.weak]+id = "openai/gpt-4o-mini"+llm_client = "openrouter"++[routes.ab_test]+id = "ab-test"+type = "random"+targets = ["strong", "weak"]+weights = [3, 7]+seed = 42+```++The configuration has two layers:++- `[targets.strong]` and `[targets.weak]` define models that Switchyard can call.+ Their `id` values are the model identifiers sent to the upstream provider.+- `[routes.ab_test]` defines the random route. The table name `ab_test` is local+ to the TOML file, while its `id`, `ab-test`, is the model name clients send to+ Switchyard.++## Control the traffic split++The route's `targets` list establishes the order:++```toml+targets = ["strong", "weak"]+```++The corresponding `weights` list assigns a relative weight to each target in+that same order:++```toml+weights = [3, 7]```-Run it with:+The weights do not need to add up to `1`; Switchyard treats `3:7` as a 30%/70%+split. More generally:++- Every target name must be unique and refer to a target defined in the same+ TOML file.+- `weights` must contain one non-negative, finite number per target.+- Omitting `weights` gives every target equal weight.+- A weight of `0` disables that target. At least one weight must be greater than+ `0`.+- `seed` is optional. Set it to reproduce the same selection sequence for the+ same request order in tests or benchmarks. Omit it for normal traffic.++Selection is independent for each request, so short runs may not match the+configured proportions exactly. Separate turns in the same conversation can+therefore use different targets.++## Run the route++After [building the Rust server](../getting_started.md#install), export the+provider credential, validate the configuration, and start the release binary:```bash-switchyard serve --routing-profiles routes.yaml --port 4000+export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret+./target/release/switchyard-server --config routes.toml --dry-run+./target/release/switchyard-server --config routes.toml \+ --host 127.0.0.1 --port 4000```-The route id (`ab-test`) is the model id clients select when they want the-weighted split.+Send a request using the route ID:++```bash+curl http://localhost:4000/v1/chat/completions \+ -H "Content-Type: application/json" \+ -d '{"model":"ab-test","messages":[{"role":"user","content":"hello"}]}'+```## Experimental LiteLLM integration