Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
fix(libsy): allow message_hash_fallback with classify_trigger = user_turn (#815)
Deepak Jain (DJ)9d0ae83a14ae
diff --git a/crates/libsy/src/algorithms/llm_class.rs b/crates/libsy/src/algorithms/llm_class.rsindex 12d9241d0..488bc6585 100644--- a/crates/libsy/src/algorithms/llm_class.rs+++ b/crates/libsy/src/algorithms/llm_class.rs@@ -418,10 +418,12 @@ impl TaskClassifierConfig {message: "max_output_tokens must be at least 1".to_string(),});}- if self.message_hash_fallback && self.classify_trigger != ClassifyTrigger::NewSession {+ // Only `every_request` is rejected: it retains no target, so a fallback identity has nothing to key on. Both retaining triggers can key the retained target on a message hash when the caller sends no session id.+ if self.message_hash_fallback && self.classify_trigger == ClassifyTrigger::EveryRequest {return Err(LibsyError::AlgorithmError {- message: "message_hash_fallback requires classify_trigger = new_session"- .to_string(),+ message:+ "message_hash_fallback requires classify_trigger = new_session or user_turn"+ .to_string(),});}Ok(())@@ -490,10 +492,12 @@ impl CustomClassifierConfig {message: "max_output_tokens must be at least 1".to_string(),});}- if self.message_hash_fallback && self.classify_trigger != ClassifyTrigger::NewSession {+ // Only `every_request` is rejected: it retains no target, so a fallback identity has nothing to key on. Both retaining triggers can key the retained target on a message hash when the caller sends no session id.+ if self.message_hash_fallback && self.classify_trigger == ClassifyTrigger::EveryRequest {return Err(LibsyError::AlgorithmError {- message: "message_hash_fallback requires classify_trigger = new_session"- .to_string(),+ message:+ "message_hash_fallback requires classify_trigger = new_session or user_turn"+ .to_string(),});}Ok(())@@ -713,10 +717,13 @@ impl LlmTaskClassifier {inner: Arc<dyn Classifier<State>>,config: ClassifierRouteConfig,) -> Result<Self> {- if config.message_hash_fallback && config.classify_trigger != ClassifyTrigger::NewSession {+ // Only `every_request` is rejected: it retains no target, so a fallback identity has nothing to key on. Both retaining triggers can key the retained target on a message hash when the caller sends no session id.+ if config.message_hash_fallback && config.classify_trigger == ClassifyTrigger::EveryRequest+ {return Err(LibsyError::AlgorithmError {- message: "message_hash_fallback requires classify_trigger = new_session"- .to_string(),+ message:+ "message_hash_fallback requires classify_trigger = new_session or user_turn"+ .to_string(),});}// Affinity comes first so a retained assignment short-circuits the judge call.@@ -1277,6 +1284,40 @@ mod tests {Ok(())}+ #[test]+ fn message_hash_fallback_accepts_retaining_triggers() -> Result<()> {+ // Only `every_request` is rejected: it retains no target, so a fallback+ // identity has nothing to key on. Both retaining triggers may key the+ // retained target on a message hash when the caller sends no session id.+ for trigger in [ClassifyTrigger::NewSession, ClassifyTrigger::UserTurn] {+ let config = TaskClassifierConfig {+ base_threshold: 0.5,+ classify_trigger: trigger,+ message_hash_fallback: true,+ ..TaskClassifierConfig::default()+ };+ LlmTaskClassifier::new(LlmClassifierConfig::Capability { config }).map_err(+ |error| LibsyError::AlgorithmError {+ message: format!("{trigger:?} with message_hash_fallback rejected: {error}"),+ },+ )?;+ }+ let every_request = TaskClassifierConfig {+ base_threshold: 0.5,+ classify_trigger: ClassifyTrigger::EveryRequest,+ message_hash_fallback: true,+ ..TaskClassifierConfig::default()+ };+ assert!(+ LlmTaskClassifier::new(LlmClassifierConfig::Capability {+ config: every_request+ })+ .is_err(),+ "every_request with message_hash_fallback should stay rejected"+ );+ Ok(())+ }+#[test]fn an_unusable_verdict_is_ambiguous() -> Result<()> {let policy = policy();diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.mdindex 45176afb6..7cf6930e8 100644--- a/crates/switchyard-server/README.md+++ b/crates/switchyard-server/README.md@@ -131,7 +131,7 @@ routes to `weak_target` or `strong_target`. Beyond the three targets it accepts| `base_threshold` | *required* | Lowest solve probability that routes a task to `weak_target`. Raise it to send less traffic to the weak model. || `threshold_step` | `0.0` | Finite, non-negative amount added once for uncertain or unmatched verdicts and twice for unsupported verdicts. `base_threshold + 2 * threshold_step` must be at most `1`. || `classify_trigger` | `every_request` | When the judge runs. `every_request` judges every request including tool continuations, `user_turn` judges each new user message and holds that target across the tool calls between, `new_session` judges once and reuses that target for the session. |-| `message_hash_fallback` | `false` | Extends affinity to clients that send no session header, keying on the first user message. Requires `classify_trigger = "new_session"`. |+| `message_hash_fallback` | `false` | Extends affinity to clients that send no session header, keying on the first user message. Requires `classify_trigger = "new_session"` or `"user_turn"`. |Session affinity retains a decision for the process lifetime, including a `strong_target`fallback produced while the judge was unreachable. `message_hash_fallback` keys on requestdiff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.mdindex f644cae30..f0c7d5cc5 100644--- a/docs/reference/toml_schema.md+++ b/docs/reference/toml_schema.md@@ -241,7 +241,7 @@ Capability mode classifies before serving. See| `base_threshold` | Yes | — | Lowest solve probability that routes to the weak target. In `[0, 1]`. || `threshold_step` | No | `0.0` | Finite, non-negative amount added once for uncertain or unmatched verdicts and twice for unsupported verdicts. `base_threshold + 2 * threshold_step` must be at most `1`. || `classify_trigger` | No | `every_request` | When the judge runs. `every_request` judges every request, tool continuations included. `user_turn` judges each new user message and retains that target across intervening tool calls only when requests carry a session ID; without a session ID, it behaves like `every_request`. `new_session` judges once and reuses that target for the session. |-| `message_hash_fallback` | No | `false` | Keys affinity on the first user message. Requires `classify_trigger = "new_session"`. |+| `message_hash_fallback` | No | `false` | Retains the target against a hash of the first user message when a request carries no session ID. Requires `classify_trigger = "new_session"` or `"user_turn"`. || `recent_turn_window` | No | unset | When unset, the judge sees the opening task and latest user follow-up, when present. When set, it also sees trailing turns. || `prompt` | No | packaged prompt | Replaces the capability prompt. The packaged schema is sent separately as structured-output configuration. |@@ -282,7 +282,7 @@ how one route chooses between more than two models.| `response_schema` | Yes | — | Inner JSON Schema encoded as a TOML string. Switchyard adds the provider wrapper. || `policy` | Yes | — | Policy table. `target_selector` accepts a JSON Pointer such as `/decision/target`. || `classify_trigger` | No | `every_request` | When the judge runs. `every_request` judges every request, tool continuations included. `user_turn` judges each new user message and retains that target across intervening tool calls only when requests carry a session ID; without a session ID, it behaves like `every_request`. `new_session` judges once and reuses that target for the session. |-| `message_hash_fallback` | No | `false` | Keys affinity on the first user message. Requires `classify_trigger = "new_session"`. |+| `message_hash_fallback` | No | `false` | Retains the target against a hash of the first user message when a request carries no session ID. Requires `classify_trigger = "new_session"` or `"user_turn"`. || `recent_turn_window` | No | unset | When unset, the judge sees the opening task and latest user follow-up, when present. When set, it also sees trailing turns. |The selected JSON label must name a configured group. A label naming a targetdiff --git a/docs/routing_algorithms/llm_classifier_routing.md b/docs/routing_algorithms/llm_classifier_routing.mdindex d57d70dde..c1d63bfb4 100644--- a/docs/routing_algorithms/llm_classifier_routing.md+++ b/docs/routing_algorithms/llm_classifier_routing.md@@ -123,7 +123,7 @@ for the server merge behavior.| `threshold_step` | `0.0` | Amount added for each boundary step. Must be finite and non-negative, and `base_threshold + 2 * threshold_step` must not exceed `1`. || `recent_turn_window` | unset | When unset, the judge sees the opening user task and the latest user message when they differ. When set to `N`, it sees the opening user task and the last `N` conversation messages after that task. `0` keeps only the opening task. Client system and developer instructions are not shown to the judge. || `classify_trigger` | `every_request` | When the judge runs. `every_request` judges every request, tool continuations included. `user_turn` judges each new user message and holds that target across the tool calls between. `new_session` judges once and reuses that target for the session. |-| `message_hash_fallback` | `false` | When session metadata is absent, keys affinity from the first user-message text. Requires `classify_trigger = "new_session"`. |+| `message_hash_fallback` | `false` | When session metadata is absent, keys affinity from the first user-message text. Requires `classify_trigger = "new_session"` or `"user_turn"`. || `prompt` | packaged capability prompt | Replaces the classifier's system prompt. The packaged verdict schema and routing policy remain active. || `response_format_type` | `json_schema` | Structured-output mode for capability and escalation judges. Use `json_object` for providers without JSON Schema support. || `max_output_tokens` | `4096` | Maximum completion tokens available to the classifier verdict. Must be at least `1`. |