Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

chore: refactor random routing doc for libsy (#223)

Ayush Agarwal6ff491acd7eb
diff --git a/docs/routing_algorithms/random_routing.md b/docs/routing_algorithms/random_routing.md
index f81e89b0e..17da19eb0 100644
--- a/docs/routing_algorithms/random_routing.md
+++ b/docs/routing_algorithms/random_routing.md
@@ -1,63 +1,99 @@
# Random Routing
-Random routing sends a fixed percentage of traffic to a `strong` model and the
-rest to a `weak` model. It does not inspect the prompt.
-
-Use it for A/B tests, benchmark baselines, and gradual traffic ramps. If routing
-should depend on task difficulty, use
-[LLM Classifier Routing](llm_classifier_routing.md) or
-[Stage-Router Routing](stage_router_routing.md).
-
-## Algorithm
-
-For each request, Switchyard flips a weighted coin:
-
-- `strong_probability: 0.0` sends all traffic to `weak`.
-- `strong_probability: 0.3` sends about 30% to `strong`.
-- `strong_probability: 0.5` is an even split.
-- `strong_probability: 1.0` sends all traffic to `strong`.
-
-Each request is independent, so short runs may not match the configured
-percentage exactly. `rng_seed` is optional and only useful when you need a
-repeatable sequence for tests or benchmarks.
-
-## Behavior
-
-Random routing is per request. A multi-turn conversation can move between
-models from turn to turn. If you need conversation-level stickiness, use
-[Sticky Routing](sticky_routing.md) where it is supported, or select a direct
-tier model instead of the routing model.
-
-## Enable It
-
-Use `type: random_routing` under `routes:`:
-
-```yaml
-defaults:
- api_key: ${OPENROUTER_API_KEY}
- base_url: https://openrouter.ai/api/v1
- format: openai
-
-routes:
- ab-test:
- type: random_routing
- strong:
- model: openai/gpt-4o
- weak:
- model: openai/gpt-4o-mini
- strong_probability: 0.3
- rng_seed: 42
- fallback_target_on_evict: strong
+Random routing selects one configured target for each request without inspecting
+the prompt. It supports two or more targets.
+
+Use it for A/B tests, benchmark baselines, and controlled traffic splits. Use a
+content-aware routing algorithm instead when the request itself should determine
+the target.
+
+## Configure a random route
+
+This example sends approximately 30% of requests to a strong model and 70% to a
+weak model:
+
+```toml
+schema_version = 1
+
+[llm_clients.openrouter]
+format = "openai_chat"
+base_url = "https://openrouter.ai/api/v1"
+api_key_env = "OPENROUTER_API_KEY"
+
+[targets.strong]
+id = "openai/gpt-4o"
+llm_client = "openrouter"
+
+[targets.weak]
+id = "openai/gpt-4o-mini"
+llm_client = "openrouter"
+
+[routes.ab_test]
+id = "ab-test"
+type = "random"
+targets = ["strong", "weak"]
+weights = [3, 7]
+seed = 42
+```
+
+The configuration has two layers:
+
+- `[targets.strong]` and `[targets.weak]` define models that Switchyard can call.
+ Their `id` values are the model identifiers sent to the upstream provider.
+- `[routes.ab_test]` defines the random route. The table name `ab_test` is local
+ to the TOML file, while its `id`, `ab-test`, is the model name clients send to
+ Switchyard.
+
+## Control the traffic split
+
+The route's `targets` list establishes the order:
+
+```toml
+targets = ["strong", "weak"]
+```
+
+The corresponding `weights` list assigns a relative weight to each target in
+that same order:
+
+```toml
+weights = [3, 7]
```
-Run it with:
+The weights do not need to add up to `1`; Switchyard treats `3:7` as a 30%/70%
+split. More generally:
+
+- Every target name must be unique and refer to a target defined in the same
+ TOML file.
+- `weights` must contain one non-negative, finite number per target.
+- Omitting `weights` gives every target equal weight.
+- A weight of `0` disables that target. At least one weight must be greater than
+ `0`.
+- `seed` is optional. Set it to reproduce the same selection sequence for the
+ same request order in tests or benchmarks. Omit it for normal traffic.
+
+Selection is independent for each request, so short runs may not match the
+configured proportions exactly. Separate turns in the same conversation can
+therefore use different targets.
+
+## Run the route
+
+After [building the Rust server](../getting_started.md#install), export the
+provider credential, validate the configuration, and start the release binary:
```bash
-switchyard serve --routing-profiles routes.yaml --port 4000
+export OPENROUTER_API_KEY="your-openrouter-key" # pragma: allowlist secret
+./target/release/switchyard-server --config routes.toml --dry-run
+./target/release/switchyard-server --config routes.toml \
+ --host 127.0.0.1 --port 4000
```
-The route id (`ab-test`) is the model id clients select when they want the
-weighted split.
+Send a request using the route ID:
+
+```bash
+curl http://localhost:4000/v1/chat/completions \
+ -H "Content-Type: application/json" \
+ -d '{"model":"ab-test","messages":[{"role":"user","content":"hello"}]}'
+```
## Experimental LiteLLM integration