Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
docs: update note on default behavior for failed llm judge calls (#1281)
Bhuvan Agrawal6a30670ba158
diff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.mdindex a27582c84..8b5918d1f 100644--- a/docs/routing_algorithms/llm_classifier_routing.md+++ b/docs/routing_algorithms/llm_classifier_routing.md@@ -82,9 +82,11 @@ An invalid, inconsistent, or unparseable verdict routes toTo stop waiting for a judge that accepts the request but never finishes itsresponse, set `timeout_ms` on the judge's `[llm_clients]` entry(see the [TOML schema](../reference/toml_schema.md)); it covers the judge's-retries and the complete verdict body. When the deadline expires, the Rust server returns-`504` without calling `strong_target` or `weak_target`. Other HTTP client failures-also stop routing after retries. The deadline applies to every call+retries and the complete verdict body. With `fail_open = true` (the default),+judge timeouts and other HTTP client failures after retries fall back to+`strong_target`. With `fail_open = false`, these failures stop routing without+calling `strong_target` or `weak_target`: the Rust server returns `504` for a+timeout and propagates other HTTP client errors. The deadline applies to every callthrough that client. Give the judge its own entry if the answering models need adifferent deadline, even when they use the same provider. Without a deadline,the request can wait indefinitely for the judge.