Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat: task affinity based llm classifier with message hash fallback (#168)
Ayush Agarwalf3e29af0650e
diff --git a/crates/libsy/README.md b/crates/libsy/README.mdindex 2d1cc32bf..4f622649f 100644--- a/crates/libsy/README.md+++ b/crates/libsy/README.md@@ -8,11 +8,11 @@ so it drops into a proxy, gateway, or agent runtime.## Example-Build a target set, pick an algorithm, run a request:+Build the targets, pick an algorithm, run a request:```rust-use switchyard_libsy::{Algorithm, Context, RoutedLlmClient, LlmTarget, LlmTargetSet, Request, State};-use switchyard_libsy::algorithms::{FallThrough, LlmTaskClassifier};+use switchyard_libsy::{Algorithm, Context, RoutedLlmClient, LlmTarget, Request};+use switchyard_libsy::algorithms::{LlmTaskClassifier, TaskClassifierConfig};use switchyard_protocol::{completion_text, text_request};use std::sync::Arc;@@ -20,15 +20,13 @@ use std::sync::Arc;let client = Arc::new(MyClient { /* .. */ }) as Arc<dyn RoutedLlmClient>;let target = |name: &str| LlmTarget { semantic_name: name.into(), llm_client: Some(client.clone()) };-let classifier = target("classifier");-let strong = target("strong");-let weak = target("weak");-let targets = LlmTargetSet::new(vec![strong.clone(), weak.clone()]);-let algo: Arc<dyn Algorithm> = Arc::new(- FallThrough::<State>::new_with_state(targets).with_classifier(Arc::new(LlmTaskClassifier::new(- classifier, weak, strong, 0.5,- )?)),-);+let algo: Arc<dyn Algorithm> = Arc::new(LlmTaskClassifier::new(+ target("classifier"), // the judge+ target("weak"), // efficient tier+ target("strong"), // capable tier+ // Every field but `base_threshold` has a default; see "Tuning the classifier" below.+ TaskClassifierConfig { base_threshold: 0.5, ..Default::default() },+)?);let req = Request {// `text_request` is the single-turn shortcut; build an `LlmRequest` directly for@@ -222,15 +220,39 @@ pub trait Decision: Send + Sync {}```-Compose a classifier into `FallThrough`; the fall-through algorithm calls the judge,-applies its policy, and then invokes the selected target:+`LlmTaskClassifier` owns its fall-through cascade: it calls the judge, applies its policy, and+then invokes the selected target:```rust-let classifier = LlmTaskClassifier::new(judge_target, weak_target, strong_target, 0.5)?;-let router =- FallThrough::<State>::new_with_state(targets).with_classifier(Arc::new(classifier));+let router = LlmTaskClassifier::new(+ judge_target,+ weak_target,+ strong_target,+ TaskClassifierConfig { base_threshold: 0.5, ..Default::default() },+)?;```+### Tuning the classifier++The judge returns a `p_solve` — its estimate that the efficient tier completes the task+correctly — plus a confidence and a capability boundary. `TaskClassifierConfig` decides what+to do with that verdict. Anything invalid, abstained, or below a threshold routes to the+capable target, so every knob below trades cost against quality in the same direction.++| Field | Default | Meaning |+|---|---|---|+| `base_threshold` | *required* | Lowest `p_solve` that routes a supported task to the efficient target. Raise it to send less traffic to the cheap model. |+| `min_confidence` | `0.0` | Lowest judge confidence that permits efficient routing. `0.0` disables the gate. |+| `capability_elevated_floor` | `None` | A higher `p_solve` floor applied only when the judge marks the task `uncertain`, `unsupported`, or `unmatched`. `None` reuses `base_threshold`. |+| `session_affinity` | `false` | Retains the first decision per session and reuses it on later turns, so the judge is called once per session instead of once per turn. |+| `message_hash_fallback` | `false` | Extends affinity to clients that send no session header, keying on a hash of the first user message. Requires `session_affinity`. |++Two things to understand before enabling affinity. The retained decision is **sticky for the+process lifetime** — including a fail-closed `capable` verdict produced when the judge was+unreachable — so a transient judge failure pins that identity until restart. And+`message_hash_fallback` keys on request *content*, not a correlation id, so unrelated callers+sending identical text share one assignment.+## ErrorsAll libsy-owned APIs return [`switchyard_libsy::Result<T>`], whose error is@@ -301,8 +323,8 @@ agents live in [`examples`](examples/) folder.- [`Random`](src/algorithms/rand.rs) — uniform or weighted random over the set(one call).-- [`LlmTaskClassifier`](src/algorithms/llm_class.rs) — capability judge for a- [`FallThrough`](src/algorithms/fall_through.rs) classifier cascade.+- [`LlmTaskClassifier`](src/algorithms/llm_class.rs) — task-level capability routing with an+ internal [`FallThrough`](src/algorithms/fall_through.rs) cascade.- [`EnsembleOrchAlgo`](examples/ensemble.rs) — stateful: fan out tocandidates, judge the best, commit to the winner after N exploration turns.diff --git a/crates/libsy/examples/research_agent.rs b/crates/libsy/examples/research_agent.rsindex 0056ac02b..885a3882f 100644--- a/crates/libsy/examples/research_agent.rs+++ b/crates/libsy/examples/research_agent.rs@@ -6,7 +6,7 @@//! Every target owns an `RoutedLlmClient`, so the agent runs each request to completion with//! [`Algorithm::run`]: it serves each offloaded call with the routed//! target's `default_client` and returns the final response — no stream to drive. The-//! classifier cascade runs inside `FallThrough`; the agent never sees it. To drive the step+//! classifier cascade runs inside `LlmTaskClassifier`; the agent never sees it. To drive the step//! stream yourself instead, use//! `Algorithm::run_stream`. Run with://! cargo run -p libsy --example research_agent@@ -14,10 +14,10 @@use std::sync::Arc;use async_trait::async_trait;-use switchyard_libsy::algorithms::{FallThrough, LlmTaskClassifier};+use switchyard_libsy::algorithms::{LlmTaskClassifier, TaskClassifierConfig};use switchyard_libsy::{Algorithm, Context, Decision, LibsyError, LlmResponse, LlmTarget, LlmTargetSet, Request,- Response, Result, RoutedLlmClient, State,+ Response, Result, RoutedLlmClient,};use switchyard_protocol::{completion_text, text_request, text_response};@@ -25,7 +25,7 @@ const CLASSIFIER: &str = "classifier/model";const STRONG: &str = "strong/model";const WEAK: &str = "weak/model";/// Lowest judge-estimated solve probability that still routes to the weak model.-const THRESHOLD: f64 = 0.5;+const BASE_THRESHOLD: f64 = 0.5;/// Stub transport. Real integrators implement `RoutedLlmClient` over their own HTTP.struct StubClient;@@ -102,11 +102,18 @@ async fn main() -> Result<()> {let classifier = target_set.get_target(CLASSIFIER)?;let weak = target_set.get_target(WEAK)?;let strong = target_set.get_target(STRONG)?;- let algo: Arc<dyn Algorithm> = Arc::new(- FallThrough::<State>::new_with_state(target_set).with_classifier(Arc::new(- LlmTaskClassifier::new(classifier, weak, strong, THRESHOLD)?,- )),- );+ let algo: Arc<dyn Algorithm> = Arc::new(LlmTaskClassifier::new(+ classifier,+ weak,+ strong,+ TaskClassifierConfig {+ base_threshold: BASE_THRESHOLD,+ min_confidence: 0.0,+ capability_elevated_floor: None,+ session_affinity: false,+ message_hash_fallback: false,+ },+ )?);let agent = ResearchAgent { algo };println!("{}", agent.run("what is switchyard?").await?);diff --git a/crates/libsy/examples/research_agent_core.rs b/crates/libsy/examples/research_agent_core.rsindex 475cf468f..3c0571b13 100644--- a/crates/libsy/examples/research_agent_core.rs+++ b/crates/libsy/examples/research_agent_core.rs@@ -11,10 +11,10 @@use std::sync::Arc;-use switchyard_libsy::algorithms::{FallThrough, LlmTaskClassifier};+use switchyard_libsy::algorithms::{LlmTaskClassifier, TaskClassifierConfig};use switchyard_libsy::{Algorithm, Context, Decision, LibsyError, LlmResponse, LlmTarget, LlmTargetSet, Request,- Response, Result, State, Step,+ Response, Result, Step,};use switchyard_protocol::{completion_text, text_request, text_response};use tokio_stream::StreamExt;@@ -23,7 +23,7 @@ const CLASSIFIER: &str = "classifier/model";const STRONG: &str = "strong/model";const WEAK: &str = "weak/model";/// Lowest judge-estimated solve probability that still routes to the weak model.-const THRESHOLD: f64 = 0.5;+const BASE_THRESHOLD: f64 = 0.5;/// The "real" model call the agent makes to fulfill a promise. The core never/// makes the call itself — it hands back a request and waits for the response.@@ -110,11 +110,18 @@ async fn main() -> Result<()> {let classifier = target_set.get_target(CLASSIFIER)?;let weak = target_set.get_target(WEAK)?;let strong = target_set.get_target(STRONG)?;- let algo: Arc<dyn Algorithm> = Arc::new(- FallThrough::<State>::new_with_state(target_set).with_classifier(Arc::new(- LlmTaskClassifier::new(classifier, weak, strong, THRESHOLD)?,- )),- );+ let algo: Arc<dyn Algorithm> = Arc::new(LlmTaskClassifier::new(+ classifier,+ weak,+ strong,+ TaskClassifierConfig {+ base_threshold: BASE_THRESHOLD,+ min_confidence: 0.0,+ capability_elevated_floor: None,+ session_affinity: false,+ message_hash_fallback: false,+ },+ )?);let mut agent = ResearchAgent { algo };println!("{}", agent.run("what is switchyard?").await?);diff --git a/crates/libsy/src/algorithms.rs b/crates/libsy/src/algorithms.rsindex 1d64e936f..b36b542f8 100644--- a/crates/libsy/src/algorithms.rs+++ b/crates/libsy/src/algorithms.rs@@ -13,7 +13,7 @@ pub mod passthrough;pub mod rand;pub use fall_through::{FallThrough, FallThroughDecision};-pub use llm_class::LlmTaskClassifier;+pub use llm_class::{LlmTaskClassifier, TaskClassifierConfig};pub use noop::{Noop, NoopDecision};pub use passthrough::{Passthrough, PassthroughDecision};pub use rand::{Random, RandomClassifier, RandomDecision};diff --git a/crates/libsy/src/algorithms/fall_through.rs b/crates/libsy/src/algorithms/fall_through.rsindex 0152af477..8a6e70fdb 100644--- a/crates/libsy/src/algorithms/fall_through.rs+++ b/crates/libsy/src/algorithms/fall_through.rs@@ -263,7 +263,6 @@ whereself.execute(ctx, driver, request).await}}-#[cfg(test)]mod tests {use super::*;diff --git a/crates/libsy/src/algorithms/llm_class.rs b/crates/libsy/src/algorithms/llm_class.rsindex a3dc63c57..255a0c43b 100644--- a/crates/libsy/src/algorithms/llm_class.rs+++ b/crates/libsy/src/algorithms/llm_class.rs@@ -1,39 +1,44 @@// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.// SPDX-License-Identifier: Apache-2.0-//! Capability routing with a judge-backed classifier.+//! Task-level capability routing with a judge-backed classifier.//!-//! The classifier judges the full inbound request and emits one decisive target for a-//! [`FallThrough`](super::FallThrough) cascade. Invalid, abstained, or unavailable judge output-//! always selects the capable target.+//! The algorithm owns a [`FallThrough`](super::FallThrough) cascade. Its classifier judges the+//! full inbound request and selects one decisive target. Invalid, abstained, or unavailable judge+//! output always selects the capable target.++use std::sync::Arc;use async_trait::async_trait;use serde::Deserialize;use serde_json::Value;use switchyard_protocol::{LlmRequest, Message, OutputParams, Role};-use super::util::{Judge, JudgeClassifier, JudgeConfig, JudgePolicy};+use super::util::{AffinityRouter, Judge, JudgeClassifier, JudgeConfig, JudgePolicy};+use super::FallThrough;use crate::{- Classification, Classifier, Driver, LibsyError, LlmTarget, Request, Result, Score, State,+ Algorithm, Classification, Classifier, Context, Driver, LibsyError, LlmTarget, LlmTargetSet,+ Request, Response, Result, RoutedLlmClient, Score, State,};// TODO: As a first implementation, keeping the prompt and schema paths hardcoded. Add a way to dynamically load and parse user passed prompt and schema.const PROMPT_TEMPLATE: &str = include_str!("../prompts/capability-classifier/prompt.md");const SCHEMA_TEMPLATE: &str = include_str!("../prompts/capability-classifier/schema.json");-// TODO: There can be more knobs to tune the classifier after its verdict is parsed. Add more later.-const RECENT_MESSAGE_WINDOW: usize = 5;+/// Telemetry label for this algorithm's spans, metrics, and logs.+const ALGORITHM_NAME: &str = "llm_task_classifier";#[derive(Deserialize)]#[serde(deny_unknown_fields)]/// Parsed judge output. Only confidence, abstention, and solve probability affect v0 routing.+/// These fields are parsed from the judge output and used to route the request.+/// For supporting a new schema, we need to add a new Verdict struct and parse the newstruct TaskClassifierVerdict {#[serde(rename = "recommended_route")]_recommended_route: String,p_solve: f64,confidence: f64,abstain: bool,- #[serde(rename = "capability_boundary")]- _capability_boundary: String,+ capability_boundary: String,#[serde(rename = "primary_rule")]_primary_rule: String,#[serde(rename = "crux")]@@ -44,52 +49,28 @@ impl TaskClassifierVerdict {/// Reject out-of-range probabilities before the policy can route efficiently. Range/// containment also rejects NaN and the infinities, which compare false against both bounds.fn is_valid(&self) -> bool {- (0.0..=1.0).contains(&self.p_solve) && (0.0..=1.0).contains(&self.confidence)+ (0.0..=1.0).contains(&self.p_solve)+ && (0.0..=1.0).contains(&self.confidence)+ && matches!(+ self.capability_boundary.as_str(),+ "supported" | "uncertain" | "unsupported" | "unmatched"+ )}-}-/// Keeps client instructions, the initial task, and bounded recent dialogue for the judge.-fn trim_messages(messages: &[Message], recent_turn_window: usize) -> Vec<Message> {- let mut system = Vec::new();- let mut first_user = None;- let mut first_user_idx = None;- for (idx, message) in messages.iter().enumerate() {- match message.role {- Role::System | Role::Developer => system.push(message.clone()),- Role::User if first_user.is_none() => {- first_user = Some(message.clone());- first_user_idx = Some(idx);- }- _ => {}- }- }- let Some(first_user) = first_user else {- return system;- };- let tail = messages- .iter()- .enumerate()- .filter(|(idx, message)| {- *idx > first_user_idx.unwrap_or(0)- && !matches!(message.role, Role::System | Role::Developer)- })- .map(|(_, message)| message.clone())- .collect::<Vec<_>>();- if recent_turn_window == 0 {- let mut out = system;- out.push(first_user);- if let Some(last_user) = tail.iter().rev().find(|message| message.role == Role::User) {- out.push(last_user.clone());- }- return out;+ /// Whether this verdict needs the elevated capability threshold.+ /// When capability boundary is "uncertain", "unsupported", or "unmatched", we need to use the elevated capability threshold to route the request to weak model+ /// This is to ensure that we are not routing too many requests to the weak model.+ fn is_capability_elevated(&self) -> bool {+ matches!(+ self.capability_boundary.as_str(),+ "uncertain" | "unsupported" | "unmatched"+ )}- let mut out = system;- out.push(first_user);- let start = tail.len().saturating_sub(recent_turn_window);- out.extend_from_slice(&tail[start..]);- out}+/// The judge is responsible for any kind of llm judge based calls+/// Example: A judge can be a capability classifier, a escalation classifier etc+/// Builds capability-specific judge requests from shared classifier configuration.struct CapabilityJudge {config: JudgeConfig,}@@ -97,9 +78,20 @@ struct CapabilityJudge {impl Judge for CapabilityJudge {type Verdict = TaskClassifierVerdict;+ /// For different judges, the request building logic can be different+ /// To have a single interface for all judges, we may make a common request building logic here.fn build_request(&self, _state: &State, request: &Request) -> Request {- // The judge owns the leading system prompt; client instructions and task context follow.- let mut messages = trim_messages(&request.llm_request.messages, RECENT_MESSAGE_WINDOW);+ // For task-based routing, classify only the newest user message with the judge prompt.+ // For any turn window size setting, use TaskClassifierConfig to define the window size.+ let mut messages = request+ .llm_request+ .messages+ .iter()+ .rev()+ .find(|message| message.role == Role::User)+ .cloned()+ .into_iter()+ .collect::<Vec<_>>();messages.insert(0,Message::text(Role::System, self.config.system_prompt.clone()),@@ -123,20 +115,30 @@ impl Judge for CapabilityJudge {struct TaskClassifierPolicy {efficient_target: String,capable_target: String,- /// Lowest `p_solve` that still routes to the efficient target.- threshold: f64,+ config: TaskClassifierConfig,}impl TaskClassifierPolicy {fn new(efficient_target: impl Into<String>,capable_target: impl Into<String>,- threshold: f64,+ config: TaskClassifierConfig,) -> Self {Self {efficient_target: efficient_target.into(),capable_target: capable_target.into(),- threshold,+ config,+ }+ }++ /// Returns the required solve probability for one validated verdict.+ fn threshold(&self, verdict: &TaskClassifierVerdict) -> f64 {+ if verdict.is_capability_elevated() {+ self.config+ .capability_elevated_floor+ .unwrap_or(self.config.base_threshold)+ } else {+ self.config.base_threshold}}}@@ -145,11 +147,14 @@ impl JudgePolicy for TaskClassifierPolicy {type Verdict = TaskClassifierVerdict;fn to_classification(&self, verdict: Option<&Self::Verdict>) -> Classification {- // Judge output is untrusted. Anything incomplete, invalid, abstained, or below the- // threshold routes to the capable target rather than risking an underpowered route.+ // Judge output is untrusted. Anything incomplete, invalid, abstained, or below its+ // configured confidence or capability threshold routes to the capable target.let target = match verdict {Some(verdict)- if verdict.is_valid() && !verdict.abstain && verdict.p_solve >= self.threshold =>+ if verdict.is_valid()+ && !verdict.abstain+ && verdict.confidence >= self.config.min_confidence+ && verdict.p_solve >= self.threshold(verdict) =>{&self.efficient_target}@@ -162,44 +167,128 @@ impl JudgePolicy for TaskClassifierPolicy {}}-/// A full-request capability classifier configured with the packaged prompt and schema.-pub struct LlmTaskClassifier {+#[derive(Clone, Debug, Default, Deserialize)]+/// Thresholds that control capability classifier routing.+pub struct TaskClassifierConfig {+ /// Lowest solve probability that routes a supported task to the efficient target.+ pub base_threshold: f64,+ /// Lowest judge confidence that permits efficient routing.+ #[serde(default)]+ pub min_confidence: f64,+ /// Higher solve-probability floor for uncertain, unmatched, and unsupported tasks.+ #[serde(default)]+ pub capability_elevated_floor: Option<f64>,+ /// Enables session affinity before the judge-backed classifier.+ #[serde(default)]+ pub session_affinity: bool,+ /// Uses the first user message as the SessionKey for sticky routing when session metadata is unavailable.+ #[serde(default)]+ pub message_hash_fallback: bool,+}++impl TaskClassifierConfig {+ /// Validates routing thresholds before the classifier is constructed.+ fn validate(&self) -> Result<()> {+ if !(0.0..=1.0).contains(&self.base_threshold) {+ return Err(LibsyError::AlgorithmError {+ message: format!(+ "base_threshold must be between 0 and 1, got {}",+ self.base_threshold+ ),+ });+ }+ if !(0.0..=1.0).contains(&self.min_confidence) {+ return Err(LibsyError::AlgorithmError {+ message: format!(+ "min_confidence must be between 0 and 1, got {}",+ self.min_confidence+ ),+ });+ }+ if let Some(floor) = self.capability_elevated_floor {+ if !(0.0..=1.0).contains(&floor) {+ return Err(LibsyError::AlgorithmError {+ message: format!(+ "capability_elevated_floor must be between 0 and 1, got {floor}"+ ),+ });+ }+ if floor <= self.base_threshold {+ return Err(LibsyError::AlgorithmError {+ message: format!(+ "capability_elevated_floor must be greater than base_threshold, got {floor}"+ ),+ });+ }+ }+ if self.message_hash_fallback && !self.session_affinity {+ return Err(LibsyError::AlgorithmError {+ message: "message_hash_fallback requires session_affinity".to_string(),+ });+ }+ Ok(())+ }+}++struct TaskClassifier {classifier: JudgeClassifier<CapabilityJudge, TaskClassifierPolicy>,efficient_target: String,capable_target: String,}+/// A task-level capability routing algorithm with an internal fall-through cascade.+pub struct LlmTaskClassifier {+ route: FallThrough<State>,+ classifier: Arc<TaskClassifier>,+}+impl LlmTaskClassifier {- /// Selects `efficient_target` when the judge's `p_solve` reaches `threshold`, and- /// `capable_target` otherwise. Errors if `threshold` is outside `[0.0, 1.0]`.+ /// Routes according to `config` and returns errors for invalid thresholds.pub fn new(judge_target: LlmTarget,efficient_target: LlmTarget,capable_target: LlmTarget,- threshold: f64,+ config: TaskClassifierConfig,) -> Result<Self> {- if !(0.0..=1.0).contains(&threshold) {- return Err(LibsyError::AlgorithmError {- message: format!("threshold must be between 0 and 1, got {threshold}"),- });- }- let judge = CapabilityJudge {- config: Self::load_judge_config()?,- };- let efficient_target = efficient_target.semantic_name;- let capable_target = capable_target.semantic_name;- Ok(Self {+ config.validate()?;+ let judge_config = Self::load_judge_config()?;+ let targets = LlmTargetSet::new(vec![efficient_target.clone(), capable_target.clone()]);+ let session_affinity = config.session_affinity;+ let message_hash_fallback = config.message_hash_fallback;+ let classifier = Arc::new(TaskClassifier {classifier: JudgeClassifier::new(- judge,+ CapabilityJudge {+ config: judge_config,+ },judge_target,TaskClassifierPolicy::new(- efficient_target.clone(),- capable_target.clone(),- threshold,+ efficient_target.semantic_name.clone(),+ capable_target.semantic_name.clone(),+ config,),),- efficient_target,- capable_target,+ efficient_target: efficient_target.semantic_name,+ capable_target: capable_target.semantic_name,+ });++ // The cascade is an internal detail: callers drive the algorithm, not its parts.+ // Affinity comes first so a retained assignment short-circuits the judge call.+ let mut route = FallThrough::<State>::new_with_state(targets).with_name(ALGORITHM_NAME);+ if session_affinity {+ let affinity = if message_hash_fallback {+ AffinityRouter::new().with_message_hash_fallback()+ } else {+ AffinityRouter::new()+ };+ // Both roles must share one `Arc` so the classifier reads what the processor wrote.+ let affinity = Arc::new(affinity);+ route = route+ .with_processor(affinity.clone())+ .with_classifier(affinity);+ }+ Ok(Self {+ route: route.with_classifier(classifier.clone()),+ classifier,})}@@ -230,7 +319,7 @@ impl LlmTaskClassifier {}#[async_trait]-impl Classifier<State> for LlmTaskClassifier {+impl Classifier<State> for TaskClassifier {fn routing_tier(&self, selected_model: &str) -> Option<&'static str> {if self.efficient_target == self.capable_target {None@@ -253,6 +342,42 @@ impl Classifier<State> for LlmTaskClassifier {}}+#[async_trait]+impl Classifier<State> for LlmTaskClassifier {+ fn routing_tier(&self, selected_model: &str) -> Option<&'static str> {+ self.classifier.routing_tier(selected_model)+ }++ async fn score(+ &self,+ state: &mut State,+ request: &mut Request,+ driver: Option<&Driver>,+ ) -> Result<Classification> {+ self.classifier.score(state, request, driver).await+ }+}++#[async_trait]+impl Algorithm for LlmTaskClassifier {+ fn name(&self) -> &str {+ "llm_task_classifier"+ }++ fn count_tokens_client(&self) -> Option<Arc<dyn RoutedLlmClient>> {+ self.route.count_tokens_client()+ }++ async fn create_run_task(+ self: Arc<Self>,+ ctx: Context,+ driver: Driver,+ request: Request,+ ) -> Result<Response> {+ self.route.execute(ctx, driver, request).await+ }+}+#[cfg(test)]mod tests {use std::sync::Arc;@@ -260,14 +385,24 @@ mod tests {use parking_lot::Mutex;use super::*;- use switchyard_protocol::{completion_text, text_response, LlmClientError};+ use switchyard_protocol::{completion_text, text_response, LlmClientError, Metadata};- use crate::{Algorithm, Context, LlmResponse, LlmTargetSet, Response, RoutedLlmClient};+ use crate::{Algorithm, Context, LlmResponse, Response, RoutedLlmClient};const TEST_THRESHOLD: f64 = 0.5;+ fn test_config(base_threshold: f64) -> TaskClassifierConfig {+ TaskClassifierConfig {+ base_threshold,+ min_confidence: 0.0,+ capability_elevated_floor: None,+ session_affinity: false,+ message_hash_fallback: false,+ }+ }+fn policy() -> TaskClassifierPolicy {- TaskClassifierPolicy::new("efficient", "capable", TEST_THRESHOLD)+ TaskClassifierPolicy::new("efficient", "capable", test_config(TEST_THRESHOLD))}/// A verdict whose non-routing fields are fixed — only the three the policy reads vary.@@ -277,7 +412,7 @@ mod tests {p_solve,confidence,abstain,- _capability_boundary: "supported".to_string(),+ capability_boundary: "supported".to_string(),_primary_rule: "SUP-1".to_string(),_crux: "test crux".to_string(),}@@ -352,19 +487,17 @@ mod tests {}}- fn router(client: Arc<dyn RoutedLlmClient>) -> Result<Arc<super::super::FallThrough<State>>> {+ fn router(client: Arc<dyn RoutedLlmClient>) -> Result<Arc<LlmTaskClassifier>> {let target = |name: &str| LlmTarget {semantic_name: name.to_string(),llm_client: Some(client.clone()),};- let targets = LlmTargetSet::new(vec![target("efficient"), target("capable")]);- let efficient = targets.get_target("efficient")?;- let capable = targets.get_target("capable")?;- Ok(Arc::new(- super::super::FallThrough::<State>::new_with_state(targets).with_classifier(Arc::new(- LlmTaskClassifier::new(target("judge"), efficient, capable, TEST_THRESHOLD)?,- )),- ))+ Ok(Arc::new(LlmTaskClassifier::new(+ target("judge"),+ target("efficient"),+ target("capable"),+ test_config(TEST_THRESHOLD),+ )?))}fn classify_request() -> Request {@@ -378,6 +511,29 @@ mod tests {}}+ fn classify_session_request() -> Request {+ Request {+ metadata: Some(Metadata {+ session_id: Some("session-1".to_string()),+ ..Metadata::default()+ }),+ ..classify_request()+ }+ }++ fn classify_follow_up_request() -> Request {+ let mut request = classify_request();+ request+ .llm_request+ .messages+ .push(Message::text(Role::Assistant, "I will add the test."));+ request.llm_request.messages.push(Message::text(+ Role::User,+ "Now run the test suite and report the result.",+ ));+ request+ }+#[tokio::test]async fn an_unreachable_judge_routes_capable_instead_of_failing_the_request() -> Result<()> {let router = router(Arc::new(UnreachableJudgeClient))?;@@ -408,6 +564,67 @@ mod tests {Ok(())}+ #[tokio::test]+ async fn classifier_config_enables_session_affinity() -> Result<()> {+ let client = Arc::new(PerRequestClient::default());+ let target = |name: &str| LlmTarget {+ semantic_name: name.to_string(),+ llm_client: Some(client.clone()),+ };+ let router = Arc::new(LlmTaskClassifier::new(+ target("judge"),+ target("efficient"),+ target("capable"),+ TaskClassifierConfig {+ session_affinity: true,+ ..test_config(TEST_THRESHOLD)+ },+ )?);++ router+ .clone()+ .run(Context::default(), classify_session_request())+ .await?;+ router+ .clone()+ .run(Context::default(), classify_session_request())+ .await?;++ assert_eq!(client.calls(), vec!["judge", "efficient", "efficient"]);+ Ok(())+ }++ #[tokio::test]+ async fn classifier_config_reuses_message_hash_affinity_for_a_follow_up() -> Result<()> {+ let client = Arc::new(PerRequestClient::default());+ let target = |name: &str| LlmTarget {+ semantic_name: name.to_string(),+ llm_client: Some(client.clone()),+ };+ let router = Arc::new(LlmTaskClassifier::new(+ target("judge"),+ target("efficient"),+ target("capable"),+ TaskClassifierConfig {+ session_affinity: true,+ message_hash_fallback: true,+ ..test_config(TEST_THRESHOLD)+ },+ )?);++ router+ .clone()+ .run(Context::default(), classify_request())+ .await?;+ router+ .clone()+ .run(Context::default(), classify_follow_up_request())+ .await?;++ assert_eq!(client.calls(), vec!["judge", "efficient", "efficient"]);+ Ok(())+ }+#[test]fn the_threshold_boundary_is_inclusive() -> Result<()> {let policy = policy();@@ -421,27 +638,60 @@ mod tests {#[test]fn the_threshold_moves_the_routing_boundary() -> Result<()> {let borderline = verdict(0.5, 1.0, false);- let strict = TaskClassifierPolicy::new("efficient", "capable", 0.9);- let lenient = TaskClassifierPolicy::new("efficient", "capable", 0.1);+ let strict = TaskClassifierPolicy::new("efficient", "capable", test_config(0.9));+ let lenient = TaskClassifierPolicy::new("efficient", "capable", test_config(0.1));assert_eq!(selected(&strict, Some(&borderline))?, "capable");assert_eq!(selected(&lenient, Some(&borderline))?, "efficient");Ok(())}#[test]- fn an_out_of_range_threshold_is_rejected() -> Result<()> {+ fn invalid_classifier_config_is_rejected() -> Result<()> {let target = |name: &str| LlmTarget {semantic_name: name.to_string(),llm_client: None,};for bad in [1.5, -0.1, f64::NAN, f64::INFINITY] {assert!(- LlmTaskClassifier::new(target("judge"), target("e"), target("c"), bad).is_err(),- "threshold {bad} should be rejected"+ LlmTaskClassifier::new(+ target("judge"),+ target("e"),+ target("c"),+ test_config(bad),+ )+ .is_err(),+ "base threshold {bad} should be rejected");}- LlmTaskClassifier::new(target("judge"), target("e"), target("c"), 0.0)?;- LlmTaskClassifier::new(target("judge"), target("e"), target("c"), 1.0)?;+ for config in [+ TaskClassifierConfig {+ base_threshold: 0.5,+ min_confidence: 1.1,+ capability_elevated_floor: None,+ session_affinity: false,+ message_hash_fallback: false,+ },+ TaskClassifierConfig {+ base_threshold: 0.5,+ min_confidence: 0.0,+ capability_elevated_floor: Some(0.5),+ session_affinity: false,+ message_hash_fallback: false,+ },+ TaskClassifierConfig {+ base_threshold: 0.5,+ min_confidence: 0.0,+ capability_elevated_floor: None,+ session_affinity: false,+ message_hash_fallback: true,+ },+ ] {+ assert!(+ LlmTaskClassifier::new(target("judge"), target("e"), target("c"), config).is_err()+ );+ }+ LlmTaskClassifier::new(target("judge"), target("e"), target("c"), test_config(0.0))?;+ LlmTaskClassifier::new(target("judge"), target("e"), target("c"), test_config(1.0))?;Ok(())}@@ -450,12 +700,43 @@ mod tests {let policy = policy();let invalid_probability = verdict(1.1, 1.0, false);let abstained = verdict(1.0, 1.0, true);+ let invalid_boundary = TaskClassifierVerdict {+ capability_boundary: "unknown".to_string(),+ ..verdict(1.0, 1.0, false)+ };assert_eq!(selected(&policy, Some(&invalid_probability))?, "capable");assert_eq!(selected(&policy, Some(&abstained))?, "capable");+ assert_eq!(selected(&policy, Some(&invalid_boundary))?, "capable");assert_eq!(selected(&policy, None)?, "capable");Ok(())}+ #[test]+ fn elevated_capability_floor_is_a_targeted_safety_brake() -> Result<()> {+ let policy = TaskClassifierPolicy::new(+ "efficient",+ "capable",+ TaskClassifierConfig {+ capability_elevated_floor: Some(0.45),+ ..test_config(0.25)+ },+ );+ let supported = verdict(0.30, 1.0, false);+ let elevated = TaskClassifierVerdict {+ capability_boundary: "uncertain".to_string(),+ ..verdict(0.30, 1.0, false)+ };+ let strong_elevated = TaskClassifierVerdict {+ capability_boundary: "unsupported".to_string(),+ ..verdict(0.50, 1.0, false)+ };++ assert_eq!(selected(&policy, Some(&supported))?, "efficient");+ assert_eq!(selected(&policy, Some(&elevated))?, "capable");+ assert_eq!(selected(&policy, Some(&strong_elevated))?, "efficient");+ Ok(())+ }+#[test]fn capability_judge_builds_a_structured_request() -> Result<()> {let judge = CapabilityJudge {@@ -484,23 +765,17 @@ mod tests {let judge_request = judge.build_request(&State::default(), &request);assert_eq!(judge_request.llm_request.model, request.llm_request.model);- assert_eq!(- judge_request.llm_request.messages.len(),- RECENT_MESSAGE_WINDOW + 4- );+ assert_eq!(judge_request.llm_request.messages.len(), 2);let contents = judge_request.llm_request.messages.iter().filter_map(|message| message.text_content("\n")).collect::<Vec<_>>();- assert!(contents.contains(&"initial task".to_string()));- assert!(contents.contains(&"recent 1".to_string()));- assert!(contents.contains(&"recent 5".to_string()));- assert!(contents.contains(&"client instructions".to_string()));- assert!(contents.contains(&"client developer instructions".to_string()));- assert!(!contents.contains(&"old response".to_string()));- assert!(!contents.contains(&"old follow-up".to_string()));+ assert!(contents.contains(&"recent 4".to_string()));+ assert!(!contents.contains(&"initial task".to_string()));+ assert!(!contents.contains(&"recent 5".to_string()));+ assert!(!contents.contains(&"client instructions".to_string()));assert_eq!(judge_request.llm_request.output.response_format,judge.config.response_schemadiff --git a/crates/libsy/src/algorithms/util/affinity.rs b/crates/libsy/src/algorithms/util/affinity.rsindex 8f33db35a..c71b9fb0a 100644--- a/crates/libsy/src/algorithms/util/affinity.rs+++ b/crates/libsy/src/algorithms/util/affinity.rs@@ -17,11 +17,12 @@//! session. [`AffinityRouter::for_subagents`] narrows affinity to explicitly identified//! child agents, leaving root traffic to later classifiers on every turn.-use std::collections::{HashMap, HashSet};+use std::collections::{hash_map::DefaultHasher, HashMap, HashSet};+use std::hash::{Hash, Hasher};use async_trait::async_trait;use parking_lot::Mutex;-use switchyard_protocol::Request;+use switchyard_protocol::{Request, Role};use crate::{Classification, Classifier, Event, Processor, Score};@@ -57,6 +58,8 @@ pub struct AffinityRouter {latch_only: Option<HashSet<String>>,/// Whether root-session requests should abstain instead of being retained.subagents_only: bool,+ /// In absence of headers, use the message hash based fallback key to do task based routing+ message_hash_fallback: bool,/// Retained assignments, shared across this router's processor and classifier roles.////// Held on the instance so the two roles share one process-local map through a@@ -80,6 +83,12 @@ impl AffinityRouter {}}+ /// Uses the first user-message text as a fallback key when metadata has no session.+ pub fn with_message_hash_fallback(mut self) -> Self {+ self.message_hash_fallback = true;+ self+ }+/// Restricts latching to `models`; a decision for any other model routes but is not/// retained.pub fn with_latch_only(mut self, models: impl IntoIterator<Item = impl Into<String>>) -> Self {@@ -96,18 +105,34 @@ impl AffinityRouter {/// Derives the stable identity this router should retain for `request`.fn affinity_key(&self, request: &Request) -> Option<AffinityKey> {- let metadata = request.metadata.as_ref()?;- let session = metadata.session_id.clone()?;- if metadata.is_subagent {- Some(AffinityKey::Subagent {- session,- agent: metadata.agent_id.clone()?,- })- } else if self.subagents_only {- None- } else {- Some(AffinityKey::Session(session))+ if let Some(metadata) = request.metadata.as_ref() {+ if let Some(session) = metadata.session_id.clone() {+ return if metadata.is_subagent {+ metadata+ .agent_id+ .clone()+ .map(|agent| AffinityKey::Subagent { session, agent })+ } else if self.subagents_only {+ None+ } else {+ Some(AffinityKey::Session(session))+ };+ }}++ // If headers are not present and we are not a subagent, use the message hash based fallback key to do task based routing+ let is_subagent = request+ .metadata+ .as_ref()+ .is_some_and(|metadata| metadata.is_subagent);+ (!self.subagents_only && !is_subagent && self.message_hash_fallback)+ .then(|| {+ first_user_message_hash(request).map(|hash| {+ tracing::debug!(affinity_key = %hash, "affinity using message hash fallback");+ AffinityKey::Session(hash)+ })+ })+ .flatten()}}@@ -131,6 +156,20 @@ where}}+/// Hashes the first user message so later turns retain the initial task's affinity.+/// For benchmarking purpose with harnesses, task instructions are added as a user prompt to the request so we hash the initial user message.+/// TODO: Have not considered multi-modal payloads yet. That needs to be handled separately.+fn first_user_message_hash(request: &Request) -> Option<String> {+ let message = request+ .llm_request+ .messages+ .iter()+ .find(|message| message.role == Role::User)?;+ let mut hasher = DefaultHasher::new();+ message.text_content("")?.hash(&mut hasher);+ Some(format!("{:016x}", hasher.finish()))+}+#[async_trait]impl<S> Classifier<S> for AffinityRouterwhere@@ -171,7 +210,9 @@ mod tests {use std::sync::Arc;- use switchyard_protocol::{text_request, Decision, Metadata};+ use switchyard_protocol::{+ text_request, ContentBlock, Decision, LlmRequest, Message, Metadata,+ };/// Boxed, thread-safe error type keeping the test helpers ergonomic.type BoxErr = Box<dyn std::error::Error + Send + Sync>;@@ -201,6 +242,30 @@ mod tests {}}+ fn task_request(+ metadata: Option<Metadata>,+ first_user: &str,+ follow_up: Option<&str>,+ ) -> Request {+ let mut messages = vec![+ Message::text(Role::System, "follow repository instructions"),+ Message::text(Role::User, first_user),+ ];+ if let Some(follow_up) = follow_up {+ messages.push(Message::text(Role::Assistant, "I will inspect the code."));+ messages.push(Message::text(Role::User, follow_up));+ }+ Request {+ llm_request: LlmRequest {+ model: Some("auto".to_string()),+ messages,+ ..LlmRequest::default()+ },+ raw_request: None,+ metadata,+ }+ }+fn session(session_id: &str, agent_id: &str) -> Metadata {Metadata {session_id: Some(session_id.to_string()),@@ -374,6 +439,112 @@ mod tests {Ok(())}+ #[tokio::test]+ async fn message_hash_fallback_uses_the_first_user_message() -> Result<(), BoxErr> {+ let router = AffinityRouter::new().with_message_hash_fallback();+ let mut state = ();++ let mut first = task_request(+ None,+ "Add a unit test for this function.",+ Some("Now run the test suite."),+ );+ retain(&router, &mut state, &mut first, "weak").await?;++ let mut follow_up = task_request(+ None,+ "Add a unit test for this function.",+ Some("Now file a pull request."),+ );+ assert_eq!(+ scores(&router, &mut state, &mut follow_up)+ .await?+ .first()+ .map(|score| score.target.as_str()),+ Some("weak")+ );++ let mut other_task = task_request(+ None,+ "Reimplement this binary from two input/output pairs.",+ Some("Now run the test suite."),+ );+ assert!(scores(&router, &mut state, &mut other_task)+ .await?+ .is_empty());+ Ok(())+ }++ #[test]+ fn user_message_hash_ignores_non_text_provider_payloads() {+ let request = |user_message| Request {+ llm_request: LlmRequest {+ messages: vec![user_message],+ ..LlmRequest::default()+ },+ raw_request: None,+ metadata: None,+ };+ let text_only = request(Message::text(Role::User, "Implement the parser."));+ let text_with_reasoning = request(Message {+ role: Role::User,+ content: vec![+ ContentBlock::Text {+ text: "Implement the parser.".to_string(),+ },+ ContentBlock::Reasoning {+ text: "Internal provider reasoning.".to_string(),+ signature: Some("provider-signature".to_string()),+ },+ ],+ });++ assert_eq!(+ first_user_message_hash(&text_only),+ first_user_message_hash(&text_with_reasoning)+ );+ }++ #[tokio::test]+ async fn metadata_session_takes_precedence_over_message_hash() -> Result<(), BoxErr> {+ let router = AffinityRouter::new().with_message_hash_fallback();+ let mut state = ();++ let mut first = task_request(+ Some(session("session-1", "agent-a")),+ "Implement the parser.",+ None,+ );+ retain(&router, &mut state, &mut first, "strong").await?;++ let mut other_session = task_request(+ Some(session("session-2", "agent-a")),+ "Implement the parser.",+ None,+ );+ assert!(scores(&router, &mut state, &mut other_session)+ .await?+ .is_empty());+ Ok(())+ }++ #[tokio::test]+ async fn message_hash_fallback_abstains_for_subagents() -> Result<(), BoxErr> {+ let router = AffinityRouter::new().with_message_hash_fallback();+ let mut state = ();+ let mut subagent = task_request(+ Some(Metadata {+ is_subagent: true,+ ..Metadata::default()+ }),+ "Implement the parser.",+ None,+ );++ assert!(scores(&router, &mut state, &mut subagent).await?.is_empty());+ Ok(())+ }+#[tokio::test]async fn one_router_serves_both_roles() -> Result<(), BoxErr> {// The same instance is registered under both SDK roles; a decision folded in viadiff --git a/crates/libsy/src/lib.rs b/crates/libsy/src/lib.rsindex e4a6bea86..0c54c3015 100644--- a/crates/libsy/src/lib.rs+++ b/crates/libsy/src/lib.rs@@ -74,8 +74,7 @@//!//! [`algorithms::Random`] provides uniform or weighted random routing.//!-//! [`algorithms::LlmTaskClassifier`] classifies with one model; compose it with-//! [`algorithms::FallThrough`] to route to the selected target.+//! [`algorithms::LlmTaskClassifier`] uses one model to classify and route to its selected target.mod core;pub use core::*;diff --git a/crates/libsy/tests/observability.rs b/crates/libsy/tests/observability.rsindex 70dee0411..e22cef8eb 100644--- a/crates/libsy/tests/observability.rs+++ b/crates/libsy/tests/observability.rs@@ -26,10 +26,10 @@ use tracing_subscriber::layer::{Context as LayerContext, SubscriberExt};use tracing_subscriber::registry::LookupSpan;use tracing_subscriber::Layer;-use switchyard_libsy::algorithms::{FallThrough, LlmTaskClassifier};+use switchyard_libsy::algorithms::{LlmTaskClassifier, TaskClassifierConfig};use switchyard_libsy::{AggLlmResponse, Algorithm, Context, Decision, Driver, LibsyError, LlmResponse, LlmTarget,- LlmTargetSet, Metadata, Request, Response, RoutedLlmClient, State, Step, Usage,+ LlmTargetSet, Metadata, Request, Response, RoutedLlmClient, Step, Usage,};use switchyard_protocol::{text_request, text_response, LlmClientError};@@ -725,11 +725,18 @@ async fn classifier_metrics_count_only_the_final_routed_call() -> switchyard_liblet targets = LlmTargetSet::new(vec![target("weak"), target("strong")]);let weak = targets.get_target("weak")?;let strong = targets.get_target("strong")?;- let router = Arc::new(- FallThrough::<State>::new_with_state(targets).with_classifier(Arc::new(- LlmTaskClassifier::new(target("classifier"), weak, strong, 0.5)?,- )),- );+ let router = Arc::new(LlmTaskClassifier::new(+ target("classifier"),+ weak,+ strong,+ TaskClassifierConfig {+ base_threshold: 0.5,+ min_confidence: 0.0,+ capability_elevated_floor: None,+ session_affinity: false,+ message_hash_fallback: false,+ },+ )?);let (trace, _response) = router.run(diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.mdindex 8cfec5844..f83760b0b 100644--- a/crates/switchyard-server/README.md+++ b/crates/switchyard-server/README.md@@ -34,7 +34,7 @@ type = "llm_classifier"classifier_target = "model_a"strong_target = "model_a"weak_target = "model_b"-threshold = 0.5+base_threshold = 0.5[routes.passthrough]id = "switchyard/passthrough"@@ -64,6 +64,23 @@ never contains the secret itself. If omitted, the client sends no authenticationRandom-route `weights` are relative, follow target order, and do not need to sum to one. Omit themfor equal weighting. The optional `seed` reproduces the selection sequence for the same call order.+An `llm_classifier` route sends each task to `classifier_target` for a capability verdict, then+routes to `weak_target` or `strong_target`. Beyond the three targets it accepts these keys; only+`base_threshold` is required, and anything the judge cannot decide routes to `strong_target`:++| Key | Default | Meaning |+|---|---|---|+| `base_threshold` | *required* | Lowest solve probability that routes a task to `weak_target`. Raise it to send less traffic to the weak model. |+| `min_confidence` | `0.0` | Lowest judge confidence that permits weak routing. `0.0` disables the gate. |+| `capability_elevated_floor` | unset | Higher solve-probability floor applied only to tasks the judge marks uncertain, unsupported, or unmatched. Unset reuses `base_threshold`. |+| `session_affinity` | `false` | Reuses a session's first routing decision on later turns, so the judge is called once per session rather than once per turn. |+| `message_hash_fallback` | `false` | Extends affinity to clients that send no session header, keying on the first user message. Requires `session_affinity = true`. |++Session affinity retains a decision for the process lifetime, including a `strong_target`+fallback produced while the judge was unreachable. `message_hash_fallback` keys on request+content rather than a session id, so unrelated callers sending identical text share one+assignment.+## Metrics`GET /metrics` exposes Prometheus text from the server's process-wide OpenTelemetry provider.diff --git a/crates/switchyard-server/src/config.rs b/crates/switchyard-server/src/config.rsindex 3315a8914..5074b4ff5 100644--- a/crates/switchyard-server/src/config.rs+++ b/crates/switchyard-server/src/config.rs@@ -8,8 +8,8 @@ use std::fs;use std::path::Path;use std::sync::Arc;-use libsy::algorithms::{FallThrough, LlmTaskClassifier, Noop, Passthrough, Random};-use libsy::{Algorithm, LlmTarget, LlmTargetSet, RoutedLlmClient, State};+use libsy::algorithms::{LlmTaskClassifier, Noop, Passthrough, Random, TaskClassifierConfig};+use libsy::{Algorithm, LlmTarget, LlmTargetSet, RoutedLlmClient};use serde::Deserialize;use switchyard_llm_client::{Backend, HttpBackendConfig, ModelConfig, TranslatingLlmClient};@@ -180,7 +180,8 @@ enum RouteConfig {classifier_target: String,strong_target: String,weak_target: String,- threshold: f64,+ #[serde(flatten)]+ classifier_config: TaskClassifierConfig,},}@@ -263,22 +264,19 @@ fn build_algorithm(classifier_target,strong_target,weak_target,- threshold,+ classifier_config,..} => {let classifier = resolve_target(route_name, classifier_target, targets)?;let strong = resolve_target(route_name, strong_target, targets)?;let weak = resolve_target(route_name, weak_target, targets)?;- // The judge is called through its own target, so it is not a routing destination.- let target_set = LlmTargetSet::new(vec![strong.clone(), weak.clone()]);// The weak model is the efficient tier; the strong model is the capable one.- let judge =- LlmTaskClassifier::new(classifier, weak, strong, *threshold).map_err(|error| {- ServerError::new(format!("llm_classifier route {route_name}: {error}"))- })?;- let router =- FallThrough::<State>::new_with_state(target_set).with_classifier(Arc::new(judge));- Ok(Arc::new(router))+ let algorithm =+ LlmTaskClassifier::new(classifier, weak, strong, classifier_config.clone())+ .map_err(|error| {+ ServerError::new(format!("llm_classifier route {route_name}: {error}"))+ })?;+ Ok(Arc::new(algorithm))}}}@@ -362,7 +360,7 @@ type = "llm_classifier"classifier_target = "classifier"strong_target = "strong"weak_target = "weak"-threshold = 0.5+base_threshold = 0.5[routes.passthrough]id = "switchyard/passthrough"@@ -439,8 +437,15 @@ target = "weak""at least one weight must be positive",),(- VALID_CONFIG.replace("threshold = 0.5", "threshold = 1.5"),- "threshold must be between 0 and 1",+ VALID_CONFIG.replace("base_threshold = 0.5", "base_threshold = 1.5"),+ "base_threshold must be between 0 and 1",+ ),+ (+ VALID_CONFIG.replace(+ "base_threshold = 0.5",+ "base_threshold = 0.5\nmessage_hash_fallback = true",+ ),+ "message_hash_fallback requires session_affinity",),(VALID_CONFIG.replace("schema_version = 1", "schema_version = 2"),@@ -470,6 +475,16 @@ target = "weak"Ok(())}+ #[test]+ fn accepts_session_affinity_with_message_hash_fallback() -> ServerResult<()> {+ let configured = VALID_CONFIG.replace(+ "base_threshold = 0.5",+ "base_threshold = 0.25\nmin_confidence = 0.0\ncapability_elevated_floor = 0.45\nsession_affinity = true\nmessage_hash_fallback = true",+ );+ server_state_from_toml(&configured)?;+ Ok(())+ }+#[test]fn api_key_environment_reference_is_validated() {let missing = VALID_CONFIG.replacen(diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rsindex c963b393b..95173c7dd 100644--- a/crates/switchyard-server/tests/server.rs+++ b/crates/switchyard-server/tests/server.rs@@ -321,7 +321,7 @@ type = "llm_classifier"classifier_target = "classifier"strong_target = "strong"weak_target = "weak"-threshold = 0.5+base_threshold = 0.5[routes.passthrough]id = "switchyard/passthrough"