Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat(libsy): add hierarchical routing (#533)
ryan-lempkac6651746bd38
diff --git a/README.md b/README.mdindex 142f80ee7..7f8a23e91 100644--- a/README.md+++ b/README.md@@ -90,6 +90,7 @@ algorithm list, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs.| [LLM Classifier](docs/routing_algorithms/llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` || [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Signals already in the conversation, such as tool results and errors, should route most turns without an extra model call. | `stage_router` || [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Every turn runs on the weak tier first, and a judge reads that answer to decide whether to send the same request to the strong tier. | `llm_classifier` with `mode = "escalation"` |+| [Hierarchical](docs/routing_algorithms/hierarchical_routing.md) | Routing algorithms are composed in a hierarchy, where one sets the configuration of another before handing off. Today an LLM classifier sets the tier a stage router falls open to. | `hierarchical` || [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |A `passthrough` route registers one target under one model ID with no routingdiff --git a/crates/libsy/src/algorithms.rs b/crates/libsy/src/algorithms.rsindex 81bd8312d..ac97b6e15 100644--- a/crates/libsy/src/algorithms.rs+++ b/crates/libsy/src/algorithms.rs@@ -9,6 +9,7 @@pub mod advisor_gate;mod escalation;pub mod fall_through;+pub mod hierarchical;pub mod llm_class;pub mod noop;pub mod passthrough;diff --git a/crates/libsy/src/algorithms/fall_through.rs b/crates/libsy/src/algorithms/fall_through.rsindex 23e336cf1..0a0f8471b 100644--- a/crates/libsy/src/algorithms/fall_through.rs+++ b/crates/libsy/src/algorithms/fall_through.rs@@ -239,8 +239,13 @@ whererequest: &mut Request,) -> Result<(ModelId, Option<Response>)> {// 1. Processor chain accumulates request-side facts into the composition's state.+ // The driver is offered here so a processor may consult a model first.for processor in &self.processors {- processor.process(state, Event::Request(request)).await?;+ let event = Event::Request {+ request,+ driver: Some(driver),+ };+ processor.process(state, event).await?;}// 2. Fall through the cascade: the first classifier to score decides (argmax). The@@ -740,7 +745,7 @@ mod tests {impl Processor for RecordingProcessor {async fn process(&self, _state: &mut (), event: Event<'_>) -> Result<()> {let kind = match event {- Event::Request(_) => "request",+ Event::Request { .. } => "request",Event::Decision { .. } => "decision",_ => "other",};@@ -769,7 +774,7 @@ mod tests {#[async_trait]impl Processor for Appender {async fn process(&self, _state: &mut (), event: Event<'_>) -> Result<()> {- if let Event::Request(request) = event {+ if let Event::Request { request, .. } = event {request.llm_request.messages@@ -846,7 +851,7 @@ mod tests {#[async_trait]impl Processor<TurnState> for CountingProcessor {async fn process(&self, state: &mut TurnState, event: Event<'_>) -> Result<()> {- if let Event::Request(_) = event {+ if let Event::Request { .. } = event {state.count += 1;}Ok(())diff --git a/crates/libsy/src/algorithms/hierarchical.rs b/crates/libsy/src/algorithms/hierarchical.rsnew file mode 100644index 000000000..30aebcb02--- /dev/null+++ b/crates/libsy/src/algorithms/hierarchical.rs@@ -0,0 +1,307 @@+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+// SPDX-License-Identifier: Apache-2.0++//! Routing that stacks a judge over a stage router.+//!+//! The judge runs as a [`Processor`]: it sets configuration the stage router reads,+//! and picks no target itself.++use std::collections::HashMap;+use std::sync::Arc;++use async_trait::async_trait;+use parking_lot::Mutex;++use super::fall_through::FallThrough;+use super::llm_class::{LlmClassifierConfig, LlmTaskClassifier, TaskClassifierConfig};+use super::stage::{StageRouterConfig, build_stage_route};+use super::util::affinity::{ClassifyTrigger, evict_if_full, has_new_user_turn, retention_key};+use super::util::stage::{StageTargets, Tier, set_fall_open};+use crate::core::algorithm::{Algorithm, Driver, RoutingIdentity};+use crate::core::classifier::Classifier;+use crate::core::processor::{Event, Processor};+use crate::core::state::State;+use crate::{LibsyError, Result};+use switchyard_protocol::{ModelId, Request};++const HIERARCHICAL: &str = "hierarchical";++/// Sets the stage router's fall-open tier from a judge verdict.+///+/// Retains the tier per routing identity, so it survives requests that carry no+/// session ID when `message_hash_fallback` is on. The retained tier is replayed+/// into state on every request so the cascade below reads it.+struct TierSetter {+ judge: Arc<dyn Classifier<State>>,+ targets: StageTargets,+ trigger: ClassifyTrigger,+ message_hash_fallback: bool,+ tiers: Mutex<HashMap<RoutingIdentity, Tier>>,+}++impl TierSetter {+ /// Two requests for one identity can both pass this and both judge, since a+ /// judge call sits between here and [`retain`](Self::retain). The later wins.+ fn is_due(&self, identity: Option<&RoutingIdentity>, request: &Request) -> bool {+ match self.trigger {+ ClassifyTrigger::UserTurn => has_new_user_turn(&request.llm_request.messages),+ // Unkeyed requests cannot be told apart, so every one is a new session.+ ClassifyTrigger::NewSession => {+ identity.is_none_or(|identity| !self.tiers.lock().contains_key(identity))+ }+ // Rejected by the constructor, and only in the enum for the standalone route.+ ClassifyTrigger::EveryRequest => true,+ }+ }++ fn retain(&self, identity: RoutingIdentity, tier: Tier) {+ let mut tiers = self.tiers.lock();+ // A user turn re-decides, so it overwrites. A session keeps its first verdict,+ // matching how affinity retains an assignment.+ let writable = self.trigger == ClassifyTrigger::UserTurn || !tiers.contains_key(&identity);+ if writable {+ evict_if_full(&mut tiers);+ tiers.insert(identity, tier);+ }+ }+}++#[async_trait]+impl Processor<State> for TierSetter {+ async fn process(&self, state: &mut State, event: Event<'_>) -> Result<()> {+ let Event::Request { request, driver } = event else {+ return Ok(());+ };+ let identity = retention_key(request, self.message_hash_fallback);+ if self.is_due(identity.as_ref(), request) {+ let (classification, _) = self.judge.score(state, request, driver).await?;+ if let Some(winner) = classification.argmax(false)?+ && let Some(tier) = self.targets.tier_for(&winner.target)+ {+ set_fall_open(state, tier);+ if let Some(identity) = identity {+ self.retain(identity, tier);+ }+ return Ok(());+ }+ }+ // Either not this request's turn or the judge had no verdict, so the last+ // tier stands rather than dropping back to the picker default.+ if let Some(tier) = identity.and_then(|identity| self.tiers.lock().get(&identity).copied())+ {+ set_fall_open(state, tier);+ }+ Ok(())+ }+}++/// A judge stacked over a stage router.+pub struct HierarchicalRouterConfig {+ /// Target the judge is called through. Not a routing destination.+ pub judge_target: ModelId,+ /// Judge settings, including how often `classify_trigger` runs it.+ pub judge: TaskClassifierConfig,+ /// Serves the turns, with the tier the judge picked as its fall-open default.+ pub stage: StageRouterConfig,+}++/// Runs a stage router with a tier the judge picks.+pub struct HierarchicalRouter {+ route: FallThrough<State>,+}++impl HierarchicalRouter {+ /// Stacks the judge over a stage router across the same tier pair.+ ///+ /// Errors on a configuration either algorithm rejects and on `every_request`.+ ///+ /// A stage router carrying its own judge is allowed, but that judge sits ahead+ /// of the fall-open tier and so answers most of the turns this one set a tier for.+ pub fn new(+ capable: ModelId,+ efficient: ModelId,+ config: HierarchicalRouterConfig,+ ) -> Result<Self> {+ if config.judge.classify_trigger == ClassifyTrigger::EveryRequest {+ return Err(LibsyError::AlgorithmError {+ message: "hierarchical: classify_trigger must be user_turn or new_session"+ .to_string(),+ });+ }+ let trigger = config.judge.classify_trigger;+ let message_hash_fallback = config.judge.message_hash_fallback;+ // Only the judge's Classifier face is used, so its own affinity never runs.+ // Leaving these set would apply the standalone route's pairing rules to a+ // trigger this router implements itself.+ let judge_config = TaskClassifierConfig {+ classify_trigger: ClassifyTrigger::EveryRequest,+ message_hash_fallback: false,+ ..config.judge+ };+ let judge = LlmTaskClassifier::new(LlmClassifierConfig::Capability {+ judge_target: config.judge_target,+ efficient_target: efficient.clone(),+ capable_target: capable.clone(),+ config: judge_config,+ })?;+ let setter = TierSetter {+ judge: Arc::new(judge),+ targets: StageTargets::new(capable.clone(), efficient.clone()),+ trigger,+ message_hash_fallback,+ tiers: Mutex::new(HashMap::new()),+ };+ let route = build_stage_route(capable, efficient, config.stage)?+ .with_name(HIERARCHICAL)+ .with_processor(Arc::new(setter));+ Ok(Self { route })+ }+}++#[async_trait]+impl Algorithm for HierarchicalRouter {+ fn name(&self) -> &str {+ HIERARCHICAL+ }++ async fn route(+ self: Arc<Self>,+ driver: Driver,+ request: Request,+ ) -> Result<crate::RoutingOutcome> {+ self.route.execute(driver, request).await+ }+}++#[cfg(test)]+mod tests {+ use std::sync::Arc;++ use switchyard_protocol::{Message, Role};++ use super::*;+ use crate::algorithms::util::stage::PickerMode;+ use crate::algorithms::util::tier_fixtures::{JUDGE, Recorder, turn_request};+ use crate::core::testing::test_drive;++ fn user_turn_request() -> Request {+ let mut request = turn_request(false);+ request+ .llm_request+ .messages+ .push(Message::text(Role::User, "now rewrite the parser"));+ request+ }++ /// The same request shape with no session ID, so only the hash can key it.+ fn unkeyed(mut request: Request) -> Request {+ if let Some(metadata) = request.metadata.as_mut() {+ metadata.session_id = None;+ }+ request+ }++ fn hash_keyed_router() -> Result<Arc<HierarchicalRouter>> {+ Ok(Arc::new(HierarchicalRouter::new(+ ModelId::from("strong"),+ ModelId::from("weak"),+ HierarchicalRouterConfig {+ judge_target: ModelId::from(JUDGE),+ judge: TaskClassifierConfig {+ base_threshold: 0.5,+ classify_trigger: ClassifyTrigger::UserTurn,+ message_hash_fallback: true,+ ..Default::default()+ },+ stage: StageRouterConfig::new(PickerMode::EfficientFirst, 0.5),+ },+ )?))+ }++ fn router() -> Result<Arc<HierarchicalRouter>> {+ Ok(Arc::new(HierarchicalRouter::new(+ ModelId::from("strong"),+ ModelId::from("weak"),+ HierarchicalRouterConfig {+ judge_target: ModelId::from(JUDGE),+ judge: TaskClassifierConfig {+ base_threshold: 0.5,+ classify_trigger: ClassifyTrigger::UserTurn,+ ..Default::default()+ },+ stage: StageRouterConfig::new(PickerMode::EfficientFirst, 0.5),+ },+ )?))+ }++ #[test]+ fn rejects_every_request_as_a_trigger() {+ let config = HierarchicalRouterConfig {+ judge_target: ModelId::from(JUDGE),+ judge: TaskClassifierConfig::default(),+ stage: StageRouterConfig::new(PickerMode::EfficientFirst, 0.5),+ };+ assert!(matches!(+ HierarchicalRouter::new(ModelId::from("strong"), ModelId::from("weak"), config),+ Err(LibsyError::AlgorithmError { .. })+ ));+ }++ #[tokio::test]+ async fn a_session_without_an_id_keys_on_the_message_hash() -> Result<()> {+ let recorder = Arc::new(Recorder::default());+ *recorder.judge_p_solve.lock() = 0.1;+ let router = hash_keyed_router()?;++ test_drive(+ router.clone(),+ unkeyed(user_turn_request()),+ recorder.serve(),+ )+ .await?;+ test_drive(+ router.clone(),+ unkeyed(turn_request(false)),+ recorder.serve(),+ )+ .await?;++ assert_eq!(+ recorder.judge_calls(),+ 1,+ "a tool step is not a user turn, session id or not"+ );+ assert_eq!(+ recorder.routed()[1].target,+ "strong",+ "and the tier survives the tool step"+ );+ Ok(())+ }++ #[tokio::test]+ async fn the_judge_sets_the_tier_once_a_turn_and_the_signals_run_within_it() -> Result<()> {+ let recorder = Arc::new(Recorder::default());+ *recorder.judge_p_solve.lock() = 0.1;+ let router = router()?;++ test_drive(router.clone(), user_turn_request(), recorder.serve()).await?;+ test_drive(router.clone(), turn_request(false), recorder.serve()).await?;++ let routed = recorder.routed();+ assert_eq!(+ routed[0].target, "strong",+ "a quiet turn falls open to the verdict"+ );+ assert_eq!(+ routed[1].target, "strong",+ "which holds across the tool steps after it"+ );+ assert_eq!(+ recorder.judge_calls(),+ 1,+ "a tool step is not a new user turn"+ );+ Ok(())+ }+}diff --git a/crates/libsy/src/algorithms/stage.rs b/crates/libsy/src/algorithms/stage.rsindex fbec4b007..c2dfccd98 100644--- a/crates/libsy/src/algorithms/stage.rs+++ b/crates/libsy/src/algorithms/stage.rs@@ -154,7 +154,7 @@ impl StageRouter {/// Errors if either threshold in `config` is outside `[0.0, 1.0]`.pub fn new(capable: ModelId, efficient: ModelId, config: StageRouterConfig) -> Result<Self> {Ok(Self {- route: build_route(capable, efficient, config)?,+ route: build_stage_route(capable, efficient, config)?,})}}@@ -174,8 +174,9 @@ impl Algorithm for StageRouter {}}-/// Wires the cascade the wrapper drives.-fn build_route(+/// Wires the cascade the wrapper drives. Exposed so a composition above can+/// stack a prelude onto it.+pub(crate) fn build_stage_route(capable: ModelId,efficient: ModelId,config: StageRouterConfig,@@ -240,17 +241,14 @@ mod tests {use async_trait::async_trait;use parking_lot::Mutex;- use serde_json::json;- use switchyard_protocol::{- ContentBlock, LlmRequest, Message, Role, ToolCall, ToolResult, WireFormat,- };use super::*;use crate::algorithms::util::stage::{DECISION_SOURCE_KEY, clear_fall_open, set_fall_open};+ use crate::algorithms::util::tier_fixtures::{JUDGE, Recorder, turn_request};use crate::core::processor::{Event, Processor};use crate::core::state::StateValue;- use crate::core::testing::{Serve, reply, test_drive};- use switchyard_protocol::{Metadata, Response};+ use crate::core::testing::test_drive;+ use switchyard_protocol::Response;/// A classifier that always picks `target`, standing in for a cascade member.struct Fixed(&'static str);@@ -359,62 +357,6 @@ mod tests {// ── routing integration tests ────────────────────────────────────────────const ESCALATION: &str = "the previous model was stalling; pick up the diagnosis";- const JUDGE: &str = "judge";-- #[derive(Clone, Debug)]- struct Call {- target: String,- messages: Vec<String>,- }-- /// Records what each target receives.- #[derive(Default)]- struct Recorder {- calls: Mutex<Vec<Call>>,- judge_p_solve: Mutex<f64>,- }-- impl Recorder {- fn routed(&self) -> Vec<Call> {- self.calls- .lock()- .iter()- .filter(|call| call.target != JUDGE)- .cloned()- .collect()- }-- /// Serves every call, recording it. The judge target gets a structured verdict- /// back so the fallback classifier has an answer without a real model.- fn serve(self: &Arc<Self>) -> impl Serve {- let recorder = Arc::clone(self);- move |target: ModelId, request: Request| {- let recorder = Arc::clone(&recorder);- async move {- let target = target.to_string();- recorder.calls.lock().push(Call {- target: target.clone(),- messages: request- .llm_request- .messages- .iter()- .filter_map(|message| message.text_content("|"))- .collect(),- });- let completion = if target == JUDGE {- let p_solve = *recorder.judge_p_solve.lock();- format!(- r#"{{"crux":"bounded task","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":{p_solve}}}"#- )- } else {- target- };- Ok(reply(completion))- }- }- }- }-fn recording_router(config: StageRouterConfig) -> Result<Arc<StageRouter>> {Ok(Arc::new(StageRouter::new(ModelId::from("strong"),@@ -443,55 +385,6 @@ mod tests {c}- fn turn_request(failed: bool) -> Request {- let content = if failed {- "fatal runtime error: out of memory"- } else {- "ok"- };- Request {- llm_request: LlmRequest {- model: Some("auto".to_string()),- messages: vec![- Message::text(Role::User, "fix the build"),- Message {- role: Role::Assistant,- content: vec![ContentBlock::ToolCall(ToolCall {- id: "call_1".to_string(),- name: "Bash".to_string(),- arguments: json!({"command": "cargo test"}),- })],- },- Message {- role: Role::Tool,- content: vec![ContentBlock::ToolResult(ToolResult {- tool_call_id: "call_1".to_string(),- content: vec![ContentBlock::Text {- text: content.to_string(),- }],- is_error: Some(failed),- })],- },- ],- ..LlmRequest::default()- },- raw_request: Some(json!({- "model": "auto",- "messages": [- {"role": "user", "content": "fix the build"},- {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function",- "function": {"name": "Bash", "arguments": "{\"command\": \"cargo test\"}"}}]},- {"role": "tool", "tool_call_id": "call_1", "content": content},- ],- })),- metadata: Some(Metadata {- wire_format: Some(WireFormat::OpenAiChat),- session_id: Some("session-1".to_string()),- ..Default::default()- }),- }- }-/// Stands in for a decider ahead of stage: sets the override once, clears it/// once, and leaves the turns between alone.#[derive(Default)]@@ -502,7 +395,7 @@ mod tests {#[async_trait]impl Processor<State> for TierDecider {async fn process(&self, state: &mut State, event: Event<'_>) -> Result<()> {- if matches!(event, Event::Request(_)) {+ if matches!(event, Event::Request { .. }) {let mut requests = self.requests.lock();*requests += 1;match *requests {@@ -521,7 +414,7 @@ mod tests {// The picker would fall open to "strong"; the override says "weak".let config = StageRouterConfig::new(PickerMode::CapableFirst, 0.5);let route: Arc<dyn Algorithm> = Arc::new(- build_route(ModelId::from("strong"), ModelId::from("weak"), config)?+ build_stage_route(ModelId::from("strong"), ModelId::from("weak"), config)?.with_processor(Arc::new(TierDecider::default())),);diff --git a/crates/libsy/src/algorithms/util.rs b/crates/libsy/src/algorithms/util.rsindex 86a548049..3f1202e26 100644--- a/crates/libsy/src/algorithms/util.rs+++ b/crates/libsy/src/algorithms/util.rs@@ -10,6 +10,8 @@ pub(crate) mod robustness;pub(crate) mod stage;pub mod subagent;pub(crate) mod target_selector;+#[cfg(test)]+pub(crate) mod tier_fixtures;pub(crate) mod tool_signals;use switchyard_protocol::ModelId;diff --git a/crates/libsy/src/algorithms/util/affinity.rs b/crates/libsy/src/algorithms/util/affinity.rsindex 6710ce3f2..7c696aa4e 100644--- a/crates/libsy/src/algorithms/util/affinity.rs+++ b/crates/libsy/src/algorithms/util/affinity.rs@@ -57,7 +57,7 @@ fn is_user_turn(message: &Message) -> bool {.all(|block| matches!(block, ContentBlock::ToolResult(_)))}-fn has_new_user_turn(messages: &[Message]) -> bool {+pub(crate) fn has_new_user_turn(messages: &[Message]) -> bool {messages.last().is_some_and(is_user_turn)}@@ -143,19 +143,9 @@ impl AffinityRouter {return None;}- let key = match (RoutingIdentity::from_request(request), is_subagent) {- (Some(identity), _) => Some(identity),- // Child requests require their explicit session + agent identity and never fall- // back to task text.- (None, true) => None,- (None, false) if self.message_hash_fallback => {- first_user_message_hash(request).map(|hash| {- tracing::debug!(affinity_key = %hash, "affinity using message hash fallback");- RoutingIdentity::Session(hash)- })- }- (None, false) => None,- };+ // Child requests require their explicit session + agent identity and never fall+ // back to task text.+ let key = retention_key(request, self.message_hash_fallback && !is_subagent);// Affinity that never keys anything is silent otherwise: the route reports itself as// configured while every turn is classified afresh. Say so once rather than per turn.if key.is_none() && self.should_warn_unkeyed() {@@ -201,6 +191,24 @@ where}}+/// The key a request's retained value is stored under.+///+/// Prefers the request's own identity and falls back to hashing the first user+/// message when the caller allows it, which is the only handle on a client that+/// sends no session ID.+pub(crate) fn retention_key(request: &Request, hash_fallback: bool) -> Option<RoutingIdentity> {+ if let Some(identity) = RoutingIdentity::from_request(request) {+ return Some(identity);+ }+ if !hash_fallback {+ return None;+ }+ first_user_message_hash(request).map(|hash| {+ tracing::debug!(retention_key = %hash, "retaining by message hash fallback");+ RoutingIdentity::Session(hash)+ })+}+/// Hashes the first user message so later turns retain the initial task's affinity./// For benchmarking purpose with harnesses, task instructions are added as a user prompt to the request so we hash the initial user message./// TODO: Have not considered multi-modal payloads yet. That needs to be handled separately.@@ -249,11 +257,11 @@ where}/// Evicts one arbitrary assignment when the map has reached [`MAX_ASSIGNMENTS`].-fn evict_if_full(assignments: &mut HashMap<RoutingIdentity, ModelId>) {- if assignments.len() >= MAX_ASSIGNMENTS- && let Some(evicted) = assignments.keys().next().cloned()+pub(crate) fn evict_if_full<V>(retained: &mut HashMap<RoutingIdentity, V>) {+ if retained.len() >= MAX_ASSIGNMENTS+ && let Some(evicted) = retained.keys().next().cloned(){- assignments.remove(&evicted);+ retained.remove(&evicted);}}diff --git a/crates/libsy/src/algorithms/util/prompts.rs b/crates/libsy/src/algorithms/util/prompts.rsindex 37314d43c..d8bc939fd 100644--- a/crates/libsy/src/algorithms/util/prompts.rs+++ b/crates/libsy/src/algorithms/util/prompts.rs@@ -355,7 +355,13 @@ mod tests {let processor = SystemPromptProcessor::new(prompts());let mut request = Request::default();processor- .process(&mut (), Event::Request(&mut request))+ .process(+ &mut (),+ Event::Request {+ request: &mut request,+ driver: None,+ },+ ).await?;assert!(instructions(&request).is_empty());Ok(())diff --git a/crates/libsy/src/algorithms/util/stage.rs b/crates/libsy/src/algorithms/util/stage.rsindex a1c0560bb..f1aeefe65 100644--- a/crates/libsy/src/algorithms/util/stage.rs+++ b/crates/libsy/src/algorithms/util/stage.rs@@ -83,7 +83,7 @@ impl Tier {/// strings the capability route reports too, so a deployment running both/// sees one tier vocabulary. Also the encoding a default-tier override is/// stored under, so changing these strings invalidates one.- fn label(self) -> &'static str {+ pub(crate) fn label(self) -> &'static str {match self {Self::Capable => "strong",Self::Efficient => "weak",@@ -127,16 +127,21 @@ impl StageTargets {}}- /// The tier label for a routed target, or `None` for one outside the pair.- pub fn label_for(&self, target: &ModelId) -> Option<&'static str> {+ /// The tier a routed target belongs to, or `None` for one outside the pair.+ pub fn tier_for(&self, target: &ModelId) -> Option<Tier> {if *target == self.capable {- Some(Tier::Capable.label())+ Some(Tier::Capable)} else if *target == self.efficient {- Some(Tier::Efficient.label())+ Some(Tier::Efficient)} else {None}}++ /// The tier label for a routed target, or `None` for one outside the pair.+ pub fn label_for(&self, target: &ModelId) -> Option<&'static str> {+ self.tier_for(target).map(Tier::label)+ }}/// Which tier to default to when the scorer is not confident.diff --git a/crates/libsy/src/algorithms/util/tier_fixtures.rs b/crates/libsy/src/algorithms/util/tier_fixtures.rsnew file mode 100644index 000000000..1e126e3f1--- /dev/null+++ b/crates/libsy/src/algorithms/util/tier_fixtures.rs@@ -0,0 +1,133 @@+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+// SPDX-License-Identifier: Apache-2.0++//! Shared fixtures for the tier-routing algorithms.+//!+//! [`Recorder`] answers [`JUDGE`] with a settable verdict, so a cascade that+//! consults a capability judge runs without a model.++use std::sync::Arc;++use parking_lot::Mutex;+use serde_json::json;+use switchyard_protocol::{+ ContentBlock, LlmRequest, Message, Metadata, ModelId, Request, Role, ToolCall, ToolResult,+ WireFormat,+};++use crate::core::testing::{Serve, reply};++/// Target the fake judge answers on.+pub(crate) const JUDGE: &str = "judge";++#[derive(Clone, Debug)]+pub(crate) struct Call {+ pub target: String,+ pub messages: Vec<String>,+}++/// Records what each target receives.+#[derive(Default)]+pub(crate) struct Recorder {+ pub calls: Mutex<Vec<Call>>,+ pub judge_p_solve: Mutex<f64>,+}++impl Recorder {+ pub fn routed(&self) -> Vec<Call> {+ self.filter_calls(|target| target != JUDGE)+ }++ pub fn judge_calls(&self) -> usize {+ self.filter_calls(|target| target == JUDGE).len()+ }++ fn filter_calls(&self, keep: impl Fn(&str) -> bool) -> Vec<Call> {+ self.calls+ .lock()+ .iter()+ .filter(|call| keep(&call.target))+ .cloned()+ .collect()+ }++ /// Serves every call, recording it. The judge target gets a structured verdict+ /// back so the fallback classifier has an answer without a real model.+ pub fn serve(self: &Arc<Self>) -> impl Serve {+ let recorder = Arc::clone(self);+ move |target: ModelId, request: Request| {+ let recorder = Arc::clone(&recorder);+ async move {+ let target = target.to_string();+ recorder.calls.lock().push(Call {+ target: target.clone(),+ messages: request+ .llm_request+ .messages+ .iter()+ .filter_map(|message| message.text_content("|"))+ .collect(),+ });+ let completion = if target == JUDGE {+ let p_solve = *recorder.judge_p_solve.lock();+ format!(+ r#"{{"crux":"bounded task","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":{p_solve}}}"#+ )+ } else {+ target+ };+ Ok(reply(completion))+ }+ }+ }+}++/// One tool step: a task, a `Bash` call, and its result.+pub(crate) fn turn_request(failed: bool) -> Request {+ let content = if failed {+ "fatal runtime error: out of memory"+ } else {+ "ok"+ };+ Request {+ llm_request: LlmRequest {+ model: Some("auto".to_string()),+ messages: vec![+ Message::text(Role::User, "fix the build"),+ Message {+ role: Role::Assistant,+ content: vec![ContentBlock::ToolCall(ToolCall {+ id: "call_1".to_string(),+ name: "Bash".to_string(),+ arguments: json!({"command": "cargo test"}),+ })],+ },+ Message {+ role: Role::Tool,+ content: vec![ContentBlock::ToolResult(ToolResult {+ tool_call_id: "call_1".to_string(),+ content: vec![ContentBlock::Text {+ text: content.to_string(),+ }],+ is_error: Some(failed),+ })],+ },+ ],+ ..LlmRequest::default()+ },+ raw_request: Some(json!({+ "model": "auto",+ "messages": [+ {"role": "user", "content": "fix the build"},+ {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function",+ "function": {"name": "Bash", "arguments": "{\"command\": \"cargo test\"}"}}]},+ {"role": "tool", "tool_call_id": "call_1", "content": content},+ ],+ })),+ metadata: Some(Metadata {+ wire_format: Some(WireFormat::OpenAiChat),+ session_id: Some("session-1".to_string()),+ ..Default::default()+ }),+ }+}diff --git a/crates/libsy/src/algorithms/util/tool_signals.rs b/crates/libsy/src/algorithms/util/tool_signals.rsindex 388417c45..a974184f7 100644--- a/crates/libsy/src/algorithms/util/tool_signals.rs+++ b/crates/libsy/src/algorithms/util/tool_signals.rs@@ -291,7 +291,7 @@ impl Default for ToolSignalProcessor {#[async_trait]impl Processor<State> for ToolSignalProcessor {async fn process(&self, state: &mut State, event: Event<'_>) -> Result<()> {- if let Event::Request(req) = event {+ if let Event::Request { request: req, .. } = event {let tool_signal = ToolSignals::from_request(req, Some(self.recent_window));state.tool_signals = Some(tool_signal);}diff --git a/crates/libsy/src/core/processor.rs b/crates/libsy/src/core/processor.rsindex af03bdb89..04d38eda4 100644--- a/crates/libsy/src/core/processor.rs+++ b/crates/libsy/src/core/processor.rs@@ -2,6 +2,7 @@// SPDX-License-Identifier: Apache-2.0use crate::Result;+use crate::core::algorithm::Driver;use async_trait::async_trait;use switchyard_protocol::{AggLlmResponse, ModelId, Request};@@ -12,7 +13,12 @@ use switchyard_protocol::{AggLlmResponse, ModelId, Request};/// the chain and to the model call. The observation-only variants stay immutable.pub enum Event<'a> {/// The inbound request that begins a turn.- Request(&'a mut Request),+ Request {+ /// The request, rewritable in place.+ request: &'a mut Request,+ /// Offered so a processor can consult a model before the cascade runs.+ driver: Option<&'a Driver>,+ },/// A routing decision paired with the request that produced it.////// The request is rewritable: a processor may add instructions or notes here that@@ -46,7 +52,7 @@ mod tests {/// The key each event variant tallies under.fn event_key(event: &Event<'_>) -> &'static str {match event {- Event::Request(_) => "requests",+ Event::Request { .. } => "requests",Event::Decision { .. } => "decisions",Event::ModelResponse(_) => "model_responses",}@@ -85,7 +91,13 @@ mod tests {let selected_model_id = ModelId::from("test/model");// Feed one of every event variant through the processor.processor- .process(&mut state, Event::Request(&mut req))+ .process(+ &mut state,+ Event::Request {+ request: &mut req,+ driver: None,+ },+ ).await?;processor.process(&mut state, Event::ModelResponse(&response))@@ -113,7 +125,13 @@ mod tests {for _ in 0..3 {processor- .process(&mut state, Event::Request(&mut req))+ .process(+ &mut state,+ Event::Request {+ request: &mut req,+ driver: None,+ },+ ).await?;}@@ -128,7 +146,7 @@ mod tests {impl Processor for RewritingProcessor {async fn process(&self, _state: &mut (), event: Event<'_>) -> Result<()> {match event {- Event::Request(request) | Event::Decision { request, .. } => {+ Event::Request { request, .. } | Event::Decision { request, .. } => {request.llm_request.model = Some("rewritten".to_string());}_ => {}@@ -144,7 +162,13 @@ mod tests {assert_eq!(req.model_id(), Some("auto".into()));RewritingProcessor- .process(&mut state, Event::Request(&mut req))+ .process(+ &mut state,+ Event::Request {+ request: &mut req,+ driver: None,+ },+ ).await?;// The edit outlives the call, so the next component sees the rewritten request.diff --git a/crates/libsy/src/lib.rs b/crates/libsy/src/lib.rsindex 47a26c083..80f31459d 100644--- a/crates/libsy/src/lib.rs+++ b/crates/libsy/src/lib.rs@@ -15,6 +15,7 @@ pub use error::{DriverError, LibsyError, Result};mod algorithms;pub use algorithms::advisor_gate::{AdvisorGate, AdvisorGateConfig, GateTrigger};+pub use algorithms::hierarchical::{HierarchicalRouter, HierarchicalRouterConfig};pub use algorithms::llm_class::{CustomClassifierConfig, CustomClassifierPolicy, LlmClassifierConfig, LlmTaskClassifier,TaskClassifierConfig,diff --git a/crates/switchyard-runner/src/algorithm.rs b/crates/switchyard-runner/src/algorithm.rsindex b4d400d74..a71d32731 100644--- a/crates/switchyard-runner/src/algorithm.rs+++ b/crates/switchyard-runner/src/algorithm.rs@@ -11,9 +11,10 @@ use std::sync::Arc;use libsy::{AdvisorGate, AdvisorGateConfig, Algorithm, ClassifierContractConfig, ClassifierResponseFormat,ClassifyTrigger, CustomClassifierConfig, CustomClassifierPolicy, EscalationJudgeConfig,- GateTrigger, HandoffNoteConfig, LlmClassifierConfig, LlmFallback, LlmTaskClassifier, Noop,- Passthrough, PickerMode, Random, StageRouter, StageRouterConfig, SubagentRouter,- SubagentRouterConfig, TargetPrompts, TaskClassifierConfig,+ GateTrigger, HandoffNoteConfig, HierarchicalRouter, HierarchicalRouterConfig,+ LlmClassifierConfig, LlmFallback, LlmTaskClassifier, Noop, Passthrough, PickerMode, Random,+ StageRouter, StageRouterConfig, SubagentRouter, SubagentRouterConfig, TargetPrompts,+ TaskClassifierConfig,};use serde::Deserialize;use switchyard_protocol::ModelId;@@ -246,26 +247,10 @@ pub enum AlgorithmSpec {},/// Picks a tier per turn by scoring signals from recent tool results.StageRouter {- /// The capable tier.- capable_target: String,- /// The efficient tier.- efficient_target: String,+ #[serde(flatten)]+ tiers: StageTierConfig,/// Tier to use when the signals are not confident.picker: PickerMode,- /// How much agreement a decisive pick needs, from 0 to 1.- confidence_threshold: f64,- /// How many trailing tool results the signals are scored over.- #[serde(default)]- recent_turn_window: Option<usize>,- /// Notes handed to a tier when the router switches to it.- #[serde(default)]- handoff_notes: Option<HandoffNoteConfig>,- /// System prompt handed to the capable tier.- #[serde(default)]- capable_system_prompt: Option<String>,- /// System prompt handed to the efficient tier.- #[serde(default)]- efficient_system_prompt: Option<String>,/// Judge consulted for turns the tool signals cannot decide.#[serde(default)]classifier: Option<StageClassifierConfig>,@@ -273,6 +258,16 @@ pub enum AlgorithmSpec {#[serde(default)]subagents: Option<SubagentRouteConfig>,},+ /// A judge picks the tier at each user turn; a stage router runs the turns within it.+ Hierarchical {+ /// Judge that picks the tier. Called through its own target.+ classifier: StageClassifierConfig,+ /// The stage router the judge hands off to.+ stage: StageTierConfig,+ /// Separate policy for delegated sub-agent work.+ #[serde(default)]+ subagents: Option<SubagentRouteConfig>,+ },/// Serves every turn from one target, and has a second model review some of/// those turns before the caller sees them.Advisor {@@ -364,6 +359,33 @@ pub struct StageClassifierConfig {pub max_output_tokens: u64,}+/// The tier pair and scoring settings shared by every stage-router-backed route.+///+/// `deny_unknown_fields` here is what rejects a typo on a flattened `stage_router`+/// route, since the enum's own `deny_unknown_fields` does not apply through a flatten.+#[derive(Clone, Debug, Deserialize)]+#[serde(deny_unknown_fields)]+pub struct StageTierConfig {+ /// The capable tier.+ pub capable_target: String,+ /// The efficient tier.+ pub efficient_target: String,+ /// How much agreement a decisive pick needs, from 0 to 1.+ pub confidence_threshold: f64,+ /// How many trailing tool results the signals are scored over.+ #[serde(default)]+ pub recent_turn_window: Option<usize>,+ /// Notes handed to a tier when the router switches to it.+ #[serde(default)]+ pub handoff_notes: Option<HandoffNoteConfig>,+ /// System prompt handed to the capable tier.+ #[serde(default)]+ pub capable_system_prompt: Option<String>,+ /// System prompt handed to the efficient tier.+ #[serde(default)]+ pub efficient_system_prompt: Option<String>,+}+impl StageClassifierConfig {fn task_classifier_config(&self) -> TaskClassifierConfig {TaskClassifierConfig {@@ -421,12 +443,24 @@ impl AlgorithmSpec {}}Self::StageRouter {- capable_target,- efficient_target,- subagents,- ..+ tiers, subagents, ..} => {- let mut names = vec![capable_target.as_str(), efficient_target.as_str()];+ let mut names = vec![+ tiers.capable_target.as_str(),+ tiers.efficient_target.as_str(),+ ];+ if let Some(subagents) = subagents {+ names.extend(subagents.routing_target_names());+ }+ names+ }+ Self::Hierarchical {+ stage, subagents, ..+ } => {+ let mut names = vec![+ stage.capable_target.as_str(),+ stage.efficient_target.as_str(),+ ];if let Some(subagents) = subagents {names.extend(subagents.routing_target_names());}@@ -464,6 +498,16 @@ impl AlgorithmSpec {names.extend(subagents.classifier_target_name());}}+ Self::Hierarchical {+ classifier,+ subagents,+ ..+ } => {+ names.push(&classifier.target);+ if let Some(subagents) = subagents {+ names.extend(subagents.classifier_target_name());+ }+ }Self::Advisor { advisor_target, .. } => names.push(advisor_target),_ => {}}@@ -889,18 +933,21 @@ fn build_algorithm(Ok(Arc::new(algorithm))}AlgorithmSpec::StageRouter {- capable_target,- efficient_target,+ tiers,picker,- confidence_threshold,- recent_turn_window,- handoff_notes,- capable_system_prompt,- efficient_system_prompt,classifier,subagents,..} => {+ let StageTierConfig {+ capable_target,+ efficient_target,+ confidence_threshold,+ recent_turn_window,+ handoff_notes,+ capable_system_prompt,+ efficient_system_prompt,+ } = tiers;if matches!(picker, PickerMode::CapableFirst) {tracing::warn!("stage_router route {route_name} uses picker \"capable_first\", which is experimental: published thresholds and routing results all come from \"efficient_first\", so there is no calibrated confidence_threshold for it and no measured accuracy or cost. Use \"efficient_first\" unless you are running your own calibration."@@ -939,6 +986,39 @@ fn build_algorithm(let parent: Arc<dyn Algorithm> = Arc::new(algorithm);attach_subagent_router(route_name, parent, subagents.as_ref(), targets)}+ AlgorithmSpec::Hierarchical {+ classifier,+ stage,+ subagents,+ } => {+ let capable = resolve_target_model_id(route_name, &stage.capable_target, targets)?;+ let efficient = resolve_target_model_id(route_name, &stage.efficient_target, targets)?;+ let judge_target = resolve_target_model_id(route_name, &classifier.target, targets)?;+ let mut stage_config =+ StageRouterConfig::new(PickerMode::EfficientFirst, stage.confidence_threshold);+ stage_config.recent_window = stage.recent_turn_window;+ stage_config.handoff_notes = stage.handoff_notes.clone();+ stage_config.tier_prompts = tier_prompts(+ &capable,+ stage.capable_system_prompt.as_deref(),+ &efficient,+ stage.efficient_system_prompt.as_deref(),+ );+ let config = HierarchicalRouterConfig {+ judge_target,+ judge: classifier.task_classifier_config(),+ stage: stage_config,+ };+ let algorithm =+ HierarchicalRouter::new(capable, efficient, config).map_err(|error| {+ AlgorithmConfigError::with_source(+ format!("hierarchical route {route_name}: {error}"),+ error,+ )+ })?;+ let parent: Arc<dyn Algorithm> = Arc::new(algorithm);+ attach_subagent_router(route_name, parent, subagents.as_ref(), targets)+ }AlgorithmSpec::Advisor {executor_target,advisor_target,diff --git a/crates/switchyard-runner/src/config.rs b/crates/switchyard-runner/src/config.rsindex 74428895b..26527d99d 100644--- a/crates/switchyard-runner/src/config.rs+++ b/crates/switchyard-runner/src/config.rs@@ -644,6 +644,41 @@ base_threshold = 0.5)}+ fn hierarchical_config() -> String {+ format!(+ r#"{VALID_CONFIG}+[targets.tier_judge]+id = "tier-judge/model"+llm_client = "primary"++[routes.hier]+id = "switchyard/hier"+type = "hierarchical"++[routes.hier.classifier]+target = "tier_judge"+base_threshold = 0.5+classify_trigger = "user_turn"++[routes.hier.stage]+capable_target = "strong"+efficient_target = "weak"+confidence_threshold = 0.5+"#+ )+ }++ #[test]+ fn hierarchical_route_builds_and_claims_both_tiers_and_its_judge() -> RunnerResult<()> {+ let runner = runner_from_toml(&hierarchical_config())?;+ assert!(+ runner+ .models()+ .any(|model| model.id.as_str() == "switchyard/hier")+ );+ Ok(())+ }+#[test]fn builds_all_supported_algorithm_types() -> RunnerResult<()> {let state = runner_from_toml(VALID_CONFIG)?;@@ -785,6 +820,24 @@ base_threshold = 0.5);}+ #[test]+ fn stage_router_rejects_an_unknown_field() {+ let config = stage_config().replace(+ "picker = \"efficient_first\"",+ "picker = \"efficient_first\"\nmagic = true",+ );+ assert!(error_message(&config).contains("unknown field"));+ }++ #[test]+ fn hierarchical_stage_block_rejects_an_unknown_field() {+ let config = hierarchical_config().replace(+ "confidence_threshold = 0.5",+ "confidence_threshold = 0.5\nmagic = true",+ );+ assert!(error_message(&config).contains("unknown field"));+ }+#[test]fn rejects_unknown_fields_and_algorithm_types() {let unknown_field =diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.mdindex b607644e9..fdf6a9c4b 100644--- a/docs/reference/toml_schema.md+++ b/docs/reference/toml_schema.md@@ -211,6 +211,32 @@ optional `handoff_notes` and `classifier` tables and for tuning.| `classifier.response_format_type` | No | `json_schema` | Structured-output mode for the optional classifier judge. Use `json_object` when the classifier provider does not support JSON Schema; Switchyard adds the schema to the prompt and validates the verdict locally. || `subagents` | No | unset | Nested `passthrough` or custom `llm_classifier` policy used only for delegated sub-agent work. See [Sub-Agent-Aware Routing](../routing_algorithms/subagent_routing.md). |+### `hierarchical`++Stacks other algorithms in a hierarchy, where one can set another's+configuration. Today a classifier sets the tier a stage router falls open to when its own signals are not confident, leaving its scoring and escalation logic untouched. See+[Hierarchical Routing](../routing_algorithms/hierarchical_routing.md).++| Key | Required | Default | Meaning |+|---|:---:|---|---|+| `classifier.target` | Yes | — | Target the tier judge is called through. Not a routing destination. |+| `classifier.base_threshold` | Yes | — | `p_solve` floor that still routes to the efficient tier. In `[0, 1]`. |+| `classifier.classify_trigger` | Yes | — | `user_turn` re-picks the tier whenever the user speaks, `new_session` picks once and holds it. `every_request` is rejected here: a judge call per tool step is the cost this route exists to avoid. |+| `classifier.message_hash_fallback` | No | `false` | Retains the tier by hashing the first user message, for clients that send no session ID. Unlike the `llm_classifier` route, this works on either trigger. Conversations opening with the same text share a tier. |+| `stage.capable_target` | Yes | — | Capable tier. |+| `stage.efficient_target` | Yes | — | Efficient tier. |+| `stage.confidence_threshold` | Yes | — | Corroboration a decisive signal needs. In `[0, 1]`. |+| `stage.recent_turn_window` | No | `3` | Trailing tool results the signals are computed over. |+| `stage.capable_system_prompt` | No | unset | System prompt handed to the capable tier. |+| `stage.efficient_system_prompt` | No | unset | System prompt handed to the efficient tier. |+| `subagents` | No | unset | Nested policy used only for delegated sub-agent work. |++The tier is retained per session. A deployment that sends no session ID needs+`classifier.message_hash_fallback = true`, which keys on the first user message+instead. The stage table takes no `picker`: the classifier supplies that tier per turn. A turn the+classifier cannot reach falls open to the efficient tier. Leaving out+`classifier` is recommended: that judge runs ahead of the fall-open tier.+## Validation Errors`--dry-run` prefixes configuration failures withdiff --git a/docs/routing_algorithms/hierarchical_routing.md b/docs/routing_algorithms/hierarchical_routing.mdnew file mode 100644index 000000000..18784d1a6--- /dev/null+++ b/docs/routing_algorithms/hierarchical_routing.md@@ -0,0 +1,55 @@+# Hierarchical Routing++Routing algorithms composed in a hierarchy. An algorithm higher in the hierarchy+sets the configuration of the one below it before handing off, so each algorithm+runs where its evidence is strongest.++## What exists today++One pairing: an LLM classifier sets the tier a stage router falls open to when+its own signals are not confident. The stage router's `picker` mode is unchanged,+and so is how it scores signals, escalates, and de-escalates. Both algorithms and the+relationship between them are fixed, and the configuration below names them+directly rather than composing arbitrary algorithms.++The general form, where any algorithm can target another and set its+configuration, is the direction rather than the current behaviour. Expect this+route's configuration to change as more pairings are added.++## Configuration++```toml+[routes.switchyard]+id = "switchyard"+type = "hierarchical"++[routes.switchyard.classifier]+target = "judge"+base_threshold = 0.5+classify_trigger = "user_turn"++[routes.switchyard.stage]+capable_target = "strong"+efficient_target = "weak"+confidence_threshold = 0.5+```++`[routes.switchyard.classifier]` takes the `stage_router` classifier fields.+`classify_trigger` sets how often the judge runs: `user_turn` re-picks the tier+whenever the user speaks, `new_session` picks once and holds it.+`[routes.switchyard.stage]` takes the `stage_router` fields except `picker`, whose+job the classifier does per turn. Leave `classifier` out as well: that judge runs+ahead of the fall-open tier, so it answers most of the turns this one decided.++## Sessions++The tier is retained per session, so send `x-switchyard-session-id` and each+conversation keeps its own. A client that sends no session ID can set+`classifier.message_hash_fallback = true`, which keys on the first user message+instead. Conversations that open with the same text then share a tier.++## Example use++An interactive coding agent in auto-mode. The classifier reads the user turn and+picks the tier. The stage router runs the tool execution loop on its own signals+until the next user turn.diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.mdindex 635959a4a..2503af57e 100644--- a/docs/routing_algorithms/overview.md+++ b/docs/routing_algorithms/overview.md@@ -16,6 +16,7 @@ configuration and tuning. For the vocabulary these pages use, see| [Random Routing](random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` || [LLM Classifier Routing](llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` || [Stage-Router Routing](stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` |+| [Hierarchical Routing](hierarchical_routing.md) | Routing algorithms are composed in a hierarchy, where one sets the configuration of another before handing off. Today an LLM classifier sets a stage router's default tier. | `hierarchical` || [Escalation-Router Routing](escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` || [Advisor-Gate Routing](advisor_gate_routing.md) | One model should serve every turn, with a stronger reviewer approving its "done" claims or sending back a redo plan. | `advisor` |diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.mdindex 0955d60fa..0f77bdfb8 100644--- a/docs/routing_algorithms/stage_router_routing.md+++ b/docs/routing_algorithms/stage_router_routing.md@@ -233,8 +233,11 @@ efficient_system_prompt = "follow the settled plan"### Optional: LLM classifier fallback-By default the router uses tool signals only. To break ties on low-confidence-turns with a model call, add a `[routes.stage.classifier]` block and set+The block is optional, and omitting it is the default. With no+`[routes.stage.classifier]` block there is no judge and no fallthrough to one:+a turn the signals leave undecided goes straight to the `picker` tier.++To break ties on low-confidence turns with a model call, add the block and set`confidence_threshold` above `0.0`. The classifier is consulted only for turnsthat fall below the threshold:@@ -254,6 +257,11 @@ request. Set `response_format_type = "json_object"` for providers that only suppJSON Object mode; Switchyard then adds the schema to the judge prompt and validatesthe returned object locally. The verdict schema and routing thresholds remain unchanged.+Leave the block out under+[Hierarchical Routing](hierarchical_routing.md). There the tier is already picked+at the user turn, so a fallthrough judge would sit between the signals and that+decision and overrule it.+Give the classifier its own LLM client or quota bucket where possible. Sharingone provider bucket with the efficient tier adds a request per classified turnand can cause sustained 429s at scale.diff --git a/mkdocs.yml b/mkdocs.ymlindex 05fd415bb..46f642364 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -29,6 +29,7 @@ nav:- Random Routing: routing_algorithms/random_routing.md- LLM Classifier Routing: routing_algorithms/llm_classifier_routing.md- Stage-Router Routing: routing_algorithms/stage_router_routing.md+ - Hierarchical Routing: routing_algorithms/hierarchical_routing.md- Escalation-Router Routing: routing_algorithms/escalation_router_routing.md- Advisor-Gate Routing: routing_algorithms/advisor_gate_routing.md- Operations: