Skip to content

Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.

Rust

feat(libsy): add hierarchical routing (#533)

ryan-lempkac6651746bd38
diff --git a/README.md b/README.md
index 142f80ee7..7f8a23e91 100644
--- a/README.md
+++ b/README.md
@@ -90,6 +90,7 @@ algorithm list, or the [`switchyard-libsy`](crates/libsy/README.md) crate docs.
| [LLM Classifier](docs/routing_algorithms/llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` |
| [Stage Router](docs/routing_algorithms/stage_router_routing.md) | Signals already in the conversation, such as tool results and errors, should route most turns without an extra model call. | `stage_router` |
| [Escalation Router](docs/routing_algorithms/escalation_router_routing.md) | Every turn runs on the weak tier first, and a judge reads that answer to decide whether to send the same request to the strong tier. | `llm_classifier` with `mode = "escalation"` |
+| [Hierarchical](docs/routing_algorithms/hierarchical_routing.md) | Routing algorithms are composed in a hierarchy, where one sets the configuration of another before handing off. Today an LLM classifier sets the tier a stage router falls open to. | `hierarchical` |
| [Random](docs/routing_algorithms/random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |
A `passthrough` route registers one target under one model ID with no routing
diff --git a/crates/libsy/src/algorithms.rs b/crates/libsy/src/algorithms.rs
index 81bd8312d..ac97b6e15 100644
--- a/crates/libsy/src/algorithms.rs
+++ b/crates/libsy/src/algorithms.rs
@@ -9,6 +9,7 @@
pub mod advisor_gate;
mod escalation;
pub mod fall_through;
+pub mod hierarchical;
pub mod llm_class;
pub mod noop;
pub mod passthrough;
diff --git a/crates/libsy/src/algorithms/fall_through.rs b/crates/libsy/src/algorithms/fall_through.rs
index 23e336cf1..0a0f8471b 100644
--- a/crates/libsy/src/algorithms/fall_through.rs
+++ b/crates/libsy/src/algorithms/fall_through.rs
@@ -239,8 +239,13 @@ where
request: &mut Request,
) -> Result<(ModelId, Option<Response>)> {
// 1. Processor chain accumulates request-side facts into the composition's state.
+ // The driver is offered here so a processor may consult a model first.
for processor in &self.processors {
- processor.process(state, Event::Request(request)).await?;
+ let event = Event::Request {
+ request,
+ driver: Some(driver),
+ };
+ processor.process(state, event).await?;
}
// 2. Fall through the cascade: the first classifier to score decides (argmax). The
@@ -740,7 +745,7 @@ mod tests {
impl Processor for RecordingProcessor {
async fn process(&self, _state: &mut (), event: Event<'_>) -> Result<()> {
let kind = match event {
- Event::Request(_) => "request",
+ Event::Request { .. } => "request",
Event::Decision { .. } => "decision",
_ => "other",
};
@@ -769,7 +774,7 @@ mod tests {
#[async_trait]
impl Processor for Appender {
async fn process(&self, _state: &mut (), event: Event<'_>) -> Result<()> {
- if let Event::Request(request) = event {
+ if let Event::Request { request, .. } = event {
request
.llm_request
.messages
@@ -846,7 +851,7 @@ mod tests {
#[async_trait]
impl Processor<TurnState> for CountingProcessor {
async fn process(&self, state: &mut TurnState, event: Event<'_>) -> Result<()> {
- if let Event::Request(_) = event {
+ if let Event::Request { .. } = event {
state.count += 1;
}
Ok(())
diff --git a/crates/libsy/src/algorithms/hierarchical.rs b/crates/libsy/src/algorithms/hierarchical.rs
new file mode 100644
index 000000000..30aebcb02
--- /dev/null
+++ b/crates/libsy/src/algorithms/hierarchical.rs
@@ -0,0 +1,307 @@
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+//! Routing that stacks a judge over a stage router.
+//!
+//! The judge runs as a [`Processor`]: it sets configuration the stage router reads,
+//! and picks no target itself.
+
+use std::collections::HashMap;
+use std::sync::Arc;
+
+use async_trait::async_trait;
+use parking_lot::Mutex;
+
+use super::fall_through::FallThrough;
+use super::llm_class::{LlmClassifierConfig, LlmTaskClassifier, TaskClassifierConfig};
+use super::stage::{StageRouterConfig, build_stage_route};
+use super::util::affinity::{ClassifyTrigger, evict_if_full, has_new_user_turn, retention_key};
+use super::util::stage::{StageTargets, Tier, set_fall_open};
+use crate::core::algorithm::{Algorithm, Driver, RoutingIdentity};
+use crate::core::classifier::Classifier;
+use crate::core::processor::{Event, Processor};
+use crate::core::state::State;
+use crate::{LibsyError, Result};
+use switchyard_protocol::{ModelId, Request};
+
+const HIERARCHICAL: &str = "hierarchical";
+
+/// Sets the stage router's fall-open tier from a judge verdict.
+///
+/// Retains the tier per routing identity, so it survives requests that carry no
+/// session ID when `message_hash_fallback` is on. The retained tier is replayed
+/// into state on every request so the cascade below reads it.
+struct TierSetter {
+ judge: Arc<dyn Classifier<State>>,
+ targets: StageTargets,
+ trigger: ClassifyTrigger,
+ message_hash_fallback: bool,
+ tiers: Mutex<HashMap<RoutingIdentity, Tier>>,
+}
+
+impl TierSetter {
+ /// Two requests for one identity can both pass this and both judge, since a
+ /// judge call sits between here and [`retain`](Self::retain). The later wins.
+ fn is_due(&self, identity: Option<&RoutingIdentity>, request: &Request) -> bool {
+ match self.trigger {
+ ClassifyTrigger::UserTurn => has_new_user_turn(&request.llm_request.messages),
+ // Unkeyed requests cannot be told apart, so every one is a new session.
+ ClassifyTrigger::NewSession => {
+ identity.is_none_or(|identity| !self.tiers.lock().contains_key(identity))
+ }
+ // Rejected by the constructor, and only in the enum for the standalone route.
+ ClassifyTrigger::EveryRequest => true,
+ }
+ }
+
+ fn retain(&self, identity: RoutingIdentity, tier: Tier) {
+ let mut tiers = self.tiers.lock();
+ // A user turn re-decides, so it overwrites. A session keeps its first verdict,
+ // matching how affinity retains an assignment.
+ let writable = self.trigger == ClassifyTrigger::UserTurn || !tiers.contains_key(&identity);
+ if writable {
+ evict_if_full(&mut tiers);
+ tiers.insert(identity, tier);
+ }
+ }
+}
+
+#[async_trait]
+impl Processor<State> for TierSetter {
+ async fn process(&self, state: &mut State, event: Event<'_>) -> Result<()> {
+ let Event::Request { request, driver } = event else {
+ return Ok(());
+ };
+ let identity = retention_key(request, self.message_hash_fallback);
+ if self.is_due(identity.as_ref(), request) {
+ let (classification, _) = self.judge.score(state, request, driver).await?;
+ if let Some(winner) = classification.argmax(false)?
+ && let Some(tier) = self.targets.tier_for(&winner.target)
+ {
+ set_fall_open(state, tier);
+ if let Some(identity) = identity {
+ self.retain(identity, tier);
+ }
+ return Ok(());
+ }
+ }
+ // Either not this request's turn or the judge had no verdict, so the last
+ // tier stands rather than dropping back to the picker default.
+ if let Some(tier) = identity.and_then(|identity| self.tiers.lock().get(&identity).copied())
+ {
+ set_fall_open(state, tier);
+ }
+ Ok(())
+ }
+}
+
+/// A judge stacked over a stage router.
+pub struct HierarchicalRouterConfig {
+ /// Target the judge is called through. Not a routing destination.
+ pub judge_target: ModelId,
+ /// Judge settings, including how often `classify_trigger` runs it.
+ pub judge: TaskClassifierConfig,
+ /// Serves the turns, with the tier the judge picked as its fall-open default.
+ pub stage: StageRouterConfig,
+}
+
+/// Runs a stage router with a tier the judge picks.
+pub struct HierarchicalRouter {
+ route: FallThrough<State>,
+}
+
+impl HierarchicalRouter {
+ /// Stacks the judge over a stage router across the same tier pair.
+ ///
+ /// Errors on a configuration either algorithm rejects and on `every_request`.
+ ///
+ /// A stage router carrying its own judge is allowed, but that judge sits ahead
+ /// of the fall-open tier and so answers most of the turns this one set a tier for.
+ pub fn new(
+ capable: ModelId,
+ efficient: ModelId,
+ config: HierarchicalRouterConfig,
+ ) -> Result<Self> {
+ if config.judge.classify_trigger == ClassifyTrigger::EveryRequest {
+ return Err(LibsyError::AlgorithmError {
+ message: "hierarchical: classify_trigger must be user_turn or new_session"
+ .to_string(),
+ });
+ }
+ let trigger = config.judge.classify_trigger;
+ let message_hash_fallback = config.judge.message_hash_fallback;
+ // Only the judge's Classifier face is used, so its own affinity never runs.
+ // Leaving these set would apply the standalone route's pairing rules to a
+ // trigger this router implements itself.
+ let judge_config = TaskClassifierConfig {
+ classify_trigger: ClassifyTrigger::EveryRequest,
+ message_hash_fallback: false,
+ ..config.judge
+ };
+ let judge = LlmTaskClassifier::new(LlmClassifierConfig::Capability {
+ judge_target: config.judge_target,
+ efficient_target: efficient.clone(),
+ capable_target: capable.clone(),
+ config: judge_config,
+ })?;
+ let setter = TierSetter {
+ judge: Arc::new(judge),
+ targets: StageTargets::new(capable.clone(), efficient.clone()),
+ trigger,
+ message_hash_fallback,
+ tiers: Mutex::new(HashMap::new()),
+ };
+ let route = build_stage_route(capable, efficient, config.stage)?
+ .with_name(HIERARCHICAL)
+ .with_processor(Arc::new(setter));
+ Ok(Self { route })
+ }
+}
+
+#[async_trait]
+impl Algorithm for HierarchicalRouter {
+ fn name(&self) -> &str {
+ HIERARCHICAL
+ }
+
+ async fn route(
+ self: Arc<Self>,
+ driver: Driver,
+ request: Request,
+ ) -> Result<crate::RoutingOutcome> {
+ self.route.execute(driver, request).await
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use switchyard_protocol::{Message, Role};
+
+ use super::*;
+ use crate::algorithms::util::stage::PickerMode;
+ use crate::algorithms::util::tier_fixtures::{JUDGE, Recorder, turn_request};
+ use crate::core::testing::test_drive;
+
+ fn user_turn_request() -> Request {
+ let mut request = turn_request(false);
+ request
+ .llm_request
+ .messages
+ .push(Message::text(Role::User, "now rewrite the parser"));
+ request
+ }
+
+ /// The same request shape with no session ID, so only the hash can key it.
+ fn unkeyed(mut request: Request) -> Request {
+ if let Some(metadata) = request.metadata.as_mut() {
+ metadata.session_id = None;
+ }
+ request
+ }
+
+ fn hash_keyed_router() -> Result<Arc<HierarchicalRouter>> {
+ Ok(Arc::new(HierarchicalRouter::new(
+ ModelId::from("strong"),
+ ModelId::from("weak"),
+ HierarchicalRouterConfig {
+ judge_target: ModelId::from(JUDGE),
+ judge: TaskClassifierConfig {
+ base_threshold: 0.5,
+ classify_trigger: ClassifyTrigger::UserTurn,
+ message_hash_fallback: true,
+ ..Default::default()
+ },
+ stage: StageRouterConfig::new(PickerMode::EfficientFirst, 0.5),
+ },
+ )?))
+ }
+
+ fn router() -> Result<Arc<HierarchicalRouter>> {
+ Ok(Arc::new(HierarchicalRouter::new(
+ ModelId::from("strong"),
+ ModelId::from("weak"),
+ HierarchicalRouterConfig {
+ judge_target: ModelId::from(JUDGE),
+ judge: TaskClassifierConfig {
+ base_threshold: 0.5,
+ classify_trigger: ClassifyTrigger::UserTurn,
+ ..Default::default()
+ },
+ stage: StageRouterConfig::new(PickerMode::EfficientFirst, 0.5),
+ },
+ )?))
+ }
+
+ #[test]
+ fn rejects_every_request_as_a_trigger() {
+ let config = HierarchicalRouterConfig {
+ judge_target: ModelId::from(JUDGE),
+ judge: TaskClassifierConfig::default(),
+ stage: StageRouterConfig::new(PickerMode::EfficientFirst, 0.5),
+ };
+ assert!(matches!(
+ HierarchicalRouter::new(ModelId::from("strong"), ModelId::from("weak"), config),
+ Err(LibsyError::AlgorithmError { .. })
+ ));
+ }
+
+ #[tokio::test]
+ async fn a_session_without_an_id_keys_on_the_message_hash() -> Result<()> {
+ let recorder = Arc::new(Recorder::default());
+ *recorder.judge_p_solve.lock() = 0.1;
+ let router = hash_keyed_router()?;
+
+ test_drive(
+ router.clone(),
+ unkeyed(user_turn_request()),
+ recorder.serve(),
+ )
+ .await?;
+ test_drive(
+ router.clone(),
+ unkeyed(turn_request(false)),
+ recorder.serve(),
+ )
+ .await?;
+
+ assert_eq!(
+ recorder.judge_calls(),
+ 1,
+ "a tool step is not a user turn, session id or not"
+ );
+ assert_eq!(
+ recorder.routed()[1].target,
+ "strong",
+ "and the tier survives the tool step"
+ );
+ Ok(())
+ }
+
+ #[tokio::test]
+ async fn the_judge_sets_the_tier_once_a_turn_and_the_signals_run_within_it() -> Result<()> {
+ let recorder = Arc::new(Recorder::default());
+ *recorder.judge_p_solve.lock() = 0.1;
+ let router = router()?;
+
+ test_drive(router.clone(), user_turn_request(), recorder.serve()).await?;
+ test_drive(router.clone(), turn_request(false), recorder.serve()).await?;
+
+ let routed = recorder.routed();
+ assert_eq!(
+ routed[0].target, "strong",
+ "a quiet turn falls open to the verdict"
+ );
+ assert_eq!(
+ routed[1].target, "strong",
+ "which holds across the tool steps after it"
+ );
+ assert_eq!(
+ recorder.judge_calls(),
+ 1,
+ "a tool step is not a new user turn"
+ );
+ Ok(())
+ }
+}
diff --git a/crates/libsy/src/algorithms/stage.rs b/crates/libsy/src/algorithms/stage.rs
index fbec4b007..c2dfccd98 100644
--- a/crates/libsy/src/algorithms/stage.rs
+++ b/crates/libsy/src/algorithms/stage.rs
@@ -154,7 +154,7 @@ impl StageRouter {
/// Errors if either threshold in `config` is outside `[0.0, 1.0]`.
pub fn new(capable: ModelId, efficient: ModelId, config: StageRouterConfig) -> Result<Self> {
Ok(Self {
- route: build_route(capable, efficient, config)?,
+ route: build_stage_route(capable, efficient, config)?,
})
}
}
@@ -174,8 +174,9 @@ impl Algorithm for StageRouter {
}
}
-/// Wires the cascade the wrapper drives.
-fn build_route(
+/// Wires the cascade the wrapper drives. Exposed so a composition above can
+/// stack a prelude onto it.
+pub(crate) fn build_stage_route(
capable: ModelId,
efficient: ModelId,
config: StageRouterConfig,
@@ -240,17 +241,14 @@ mod tests {
use async_trait::async_trait;
use parking_lot::Mutex;
- use serde_json::json;
- use switchyard_protocol::{
- ContentBlock, LlmRequest, Message, Role, ToolCall, ToolResult, WireFormat,
- };
use super::*;
use crate::algorithms::util::stage::{DECISION_SOURCE_KEY, clear_fall_open, set_fall_open};
+ use crate::algorithms::util::tier_fixtures::{JUDGE, Recorder, turn_request};
use crate::core::processor::{Event, Processor};
use crate::core::state::StateValue;
- use crate::core::testing::{Serve, reply, test_drive};
- use switchyard_protocol::{Metadata, Response};
+ use crate::core::testing::test_drive;
+ use switchyard_protocol::Response;
/// A classifier that always picks `target`, standing in for a cascade member.
struct Fixed(&'static str);
@@ -359,62 +357,6 @@ mod tests {
// ── routing integration tests ────────────────────────────────────────────
const ESCALATION: &str = "the previous model was stalling; pick up the diagnosis";
- const JUDGE: &str = "judge";
-
- #[derive(Clone, Debug)]
- struct Call {
- target: String,
- messages: Vec<String>,
- }
-
- /// Records what each target receives.
- #[derive(Default)]
- struct Recorder {
- calls: Mutex<Vec<Call>>,
- judge_p_solve: Mutex<f64>,
- }
-
- impl Recorder {
- fn routed(&self) -> Vec<Call> {
- self.calls
- .lock()
- .iter()
- .filter(|call| call.target != JUDGE)
- .cloned()
- .collect()
- }
-
- /// Serves every call, recording it. The judge target gets a structured verdict
- /// back so the fallback classifier has an answer without a real model.
- fn serve(self: &Arc<Self>) -> impl Serve {
- let recorder = Arc::clone(self);
- move |target: ModelId, request: Request| {
- let recorder = Arc::clone(&recorder);
- async move {
- let target = target.to_string();
- recorder.calls.lock().push(Call {
- target: target.clone(),
- messages: request
- .llm_request
- .messages
- .iter()
- .filter_map(|message| message.text_content("|"))
- .collect(),
- });
- let completion = if target == JUDGE {
- let p_solve = *recorder.judge_p_solve.lock();
- format!(
- r#"{{"crux":"bounded task","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":{p_solve}}}"#
- )
- } else {
- target
- };
- Ok(reply(completion))
- }
- }
- }
- }
-
fn recording_router(config: StageRouterConfig) -> Result<Arc<StageRouter>> {
Ok(Arc::new(StageRouter::new(
ModelId::from("strong"),
@@ -443,55 +385,6 @@ mod tests {
c
}
- fn turn_request(failed: bool) -> Request {
- let content = if failed {
- "fatal runtime error: out of memory"
- } else {
- "ok"
- };
- Request {
- llm_request: LlmRequest {
- model: Some("auto".to_string()),
- messages: vec![
- Message::text(Role::User, "fix the build"),
- Message {
- role: Role::Assistant,
- content: vec![ContentBlock::ToolCall(ToolCall {
- id: "call_1".to_string(),
- name: "Bash".to_string(),
- arguments: json!({"command": "cargo test"}),
- })],
- },
- Message {
- role: Role::Tool,
- content: vec![ContentBlock::ToolResult(ToolResult {
- tool_call_id: "call_1".to_string(),
- content: vec![ContentBlock::Text {
- text: content.to_string(),
- }],
- is_error: Some(failed),
- })],
- },
- ],
- ..LlmRequest::default()
- },
- raw_request: Some(json!({
- "model": "auto",
- "messages": [
- {"role": "user", "content": "fix the build"},
- {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function",
- "function": {"name": "Bash", "arguments": "{\"command\": \"cargo test\"}"}}]},
- {"role": "tool", "tool_call_id": "call_1", "content": content},
- ],
- })),
- metadata: Some(Metadata {
- wire_format: Some(WireFormat::OpenAiChat),
- session_id: Some("session-1".to_string()),
- ..Default::default()
- }),
- }
- }
-
/// Stands in for a decider ahead of stage: sets the override once, clears it
/// once, and leaves the turns between alone.
#[derive(Default)]
@@ -502,7 +395,7 @@ mod tests {
#[async_trait]
impl Processor<State> for TierDecider {
async fn process(&self, state: &mut State, event: Event<'_>) -> Result<()> {
- if matches!(event, Event::Request(_)) {
+ if matches!(event, Event::Request { .. }) {
let mut requests = self.requests.lock();
*requests += 1;
match *requests {
@@ -521,7 +414,7 @@ mod tests {
// The picker would fall open to "strong"; the override says "weak".
let config = StageRouterConfig::new(PickerMode::CapableFirst, 0.5);
let route: Arc<dyn Algorithm> = Arc::new(
- build_route(ModelId::from("strong"), ModelId::from("weak"), config)?
+ build_stage_route(ModelId::from("strong"), ModelId::from("weak"), config)?
.with_processor(Arc::new(TierDecider::default())),
);
diff --git a/crates/libsy/src/algorithms/util.rs b/crates/libsy/src/algorithms/util.rs
index 86a548049..3f1202e26 100644
--- a/crates/libsy/src/algorithms/util.rs
+++ b/crates/libsy/src/algorithms/util.rs
@@ -10,6 +10,8 @@ pub(crate) mod robustness;
pub(crate) mod stage;
pub mod subagent;
pub(crate) mod target_selector;
+#[cfg(test)]
+pub(crate) mod tier_fixtures;
pub(crate) mod tool_signals;
use switchyard_protocol::ModelId;
diff --git a/crates/libsy/src/algorithms/util/affinity.rs b/crates/libsy/src/algorithms/util/affinity.rs
index 6710ce3f2..7c696aa4e 100644
--- a/crates/libsy/src/algorithms/util/affinity.rs
+++ b/crates/libsy/src/algorithms/util/affinity.rs
@@ -57,7 +57,7 @@ fn is_user_turn(message: &Message) -> bool {
.all(|block| matches!(block, ContentBlock::ToolResult(_)))
}
-fn has_new_user_turn(messages: &[Message]) -> bool {
+pub(crate) fn has_new_user_turn(messages: &[Message]) -> bool {
messages.last().is_some_and(is_user_turn)
}
@@ -143,19 +143,9 @@ impl AffinityRouter {
return None;
}
- let key = match (RoutingIdentity::from_request(request), is_subagent) {
- (Some(identity), _) => Some(identity),
- // Child requests require their explicit session + agent identity and never fall
- // back to task text.
- (None, true) => None,
- (None, false) if self.message_hash_fallback => {
- first_user_message_hash(request).map(|hash| {
- tracing::debug!(affinity_key = %hash, "affinity using message hash fallback");
- RoutingIdentity::Session(hash)
- })
- }
- (None, false) => None,
- };
+ // Child requests require their explicit session + agent identity and never fall
+ // back to task text.
+ let key = retention_key(request, self.message_hash_fallback && !is_subagent);
// Affinity that never keys anything is silent otherwise: the route reports itself as
// configured while every turn is classified afresh. Say so once rather than per turn.
if key.is_none() && self.should_warn_unkeyed() {
@@ -201,6 +191,24 @@ where
}
}
+/// The key a request's retained value is stored under.
+///
+/// Prefers the request's own identity and falls back to hashing the first user
+/// message when the caller allows it, which is the only handle on a client that
+/// sends no session ID.
+pub(crate) fn retention_key(request: &Request, hash_fallback: bool) -> Option<RoutingIdentity> {
+ if let Some(identity) = RoutingIdentity::from_request(request) {
+ return Some(identity);
+ }
+ if !hash_fallback {
+ return None;
+ }
+ first_user_message_hash(request).map(|hash| {
+ tracing::debug!(retention_key = %hash, "retaining by message hash fallback");
+ RoutingIdentity::Session(hash)
+ })
+}
+
/// Hashes the first user message so later turns retain the initial task's affinity.
/// For benchmarking purpose with harnesses, task instructions are added as a user prompt to the request so we hash the initial user message.
/// TODO: Have not considered multi-modal payloads yet. That needs to be handled separately.
@@ -249,11 +257,11 @@ where
}
/// Evicts one arbitrary assignment when the map has reached [`MAX_ASSIGNMENTS`].
-fn evict_if_full(assignments: &mut HashMap<RoutingIdentity, ModelId>) {
- if assignments.len() >= MAX_ASSIGNMENTS
- && let Some(evicted) = assignments.keys().next().cloned()
+pub(crate) fn evict_if_full<V>(retained: &mut HashMap<RoutingIdentity, V>) {
+ if retained.len() >= MAX_ASSIGNMENTS
+ && let Some(evicted) = retained.keys().next().cloned()
{
- assignments.remove(&evicted);
+ retained.remove(&evicted);
}
}
diff --git a/crates/libsy/src/algorithms/util/prompts.rs b/crates/libsy/src/algorithms/util/prompts.rs
index 37314d43c..d8bc939fd 100644
--- a/crates/libsy/src/algorithms/util/prompts.rs
+++ b/crates/libsy/src/algorithms/util/prompts.rs
@@ -355,7 +355,13 @@ mod tests {
let processor = SystemPromptProcessor::new(prompts());
let mut request = Request::default();
processor
- .process(&mut (), Event::Request(&mut request))
+ .process(
+ &mut (),
+ Event::Request {
+ request: &mut request,
+ driver: None,
+ },
+ )
.await?;
assert!(instructions(&request).is_empty());
Ok(())
diff --git a/crates/libsy/src/algorithms/util/stage.rs b/crates/libsy/src/algorithms/util/stage.rs
index a1c0560bb..f1aeefe65 100644
--- a/crates/libsy/src/algorithms/util/stage.rs
+++ b/crates/libsy/src/algorithms/util/stage.rs
@@ -83,7 +83,7 @@ impl Tier {
/// strings the capability route reports too, so a deployment running both
/// sees one tier vocabulary. Also the encoding a default-tier override is
/// stored under, so changing these strings invalidates one.
- fn label(self) -> &'static str {
+ pub(crate) fn label(self) -> &'static str {
match self {
Self::Capable => "strong",
Self::Efficient => "weak",
@@ -127,16 +127,21 @@ impl StageTargets {
}
}
- /// The tier label for a routed target, or `None` for one outside the pair.
- pub fn label_for(&self, target: &ModelId) -> Option<&'static str> {
+ /// The tier a routed target belongs to, or `None` for one outside the pair.
+ pub fn tier_for(&self, target: &ModelId) -> Option<Tier> {
if *target == self.capable {
- Some(Tier::Capable.label())
+ Some(Tier::Capable)
} else if *target == self.efficient {
- Some(Tier::Efficient.label())
+ Some(Tier::Efficient)
} else {
None
}
}
+
+ /// The tier label for a routed target, or `None` for one outside the pair.
+ pub fn label_for(&self, target: &ModelId) -> Option<&'static str> {
+ self.tier_for(target).map(Tier::label)
+ }
}
/// Which tier to default to when the scorer is not confident.
diff --git a/crates/libsy/src/algorithms/util/tier_fixtures.rs b/crates/libsy/src/algorithms/util/tier_fixtures.rs
new file mode 100644
index 000000000..1e126e3f1
--- /dev/null
+++ b/crates/libsy/src/algorithms/util/tier_fixtures.rs
@@ -0,0 +1,133 @@
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+//! Shared fixtures for the tier-routing algorithms.
+//!
+//! [`Recorder`] answers [`JUDGE`] with a settable verdict, so a cascade that
+//! consults a capability judge runs without a model.
+
+use std::sync::Arc;
+
+use parking_lot::Mutex;
+use serde_json::json;
+use switchyard_protocol::{
+ ContentBlock, LlmRequest, Message, Metadata, ModelId, Request, Role, ToolCall, ToolResult,
+ WireFormat,
+};
+
+use crate::core::testing::{Serve, reply};
+
+/// Target the fake judge answers on.
+pub(crate) const JUDGE: &str = "judge";
+
+#[derive(Clone, Debug)]
+pub(crate) struct Call {
+ pub target: String,
+ pub messages: Vec<String>,
+}
+
+/// Records what each target receives.
+#[derive(Default)]
+pub(crate) struct Recorder {
+ pub calls: Mutex<Vec<Call>>,
+ pub judge_p_solve: Mutex<f64>,
+}
+
+impl Recorder {
+ pub fn routed(&self) -> Vec<Call> {
+ self.filter_calls(|target| target != JUDGE)
+ }
+
+ pub fn judge_calls(&self) -> usize {
+ self.filter_calls(|target| target == JUDGE).len()
+ }
+
+ fn filter_calls(&self, keep: impl Fn(&str) -> bool) -> Vec<Call> {
+ self.calls
+ .lock()
+ .iter()
+ .filter(|call| keep(&call.target))
+ .cloned()
+ .collect()
+ }
+
+ /// Serves every call, recording it. The judge target gets a structured verdict
+ /// back so the fallback classifier has an answer without a real model.
+ pub fn serve(self: &Arc<Self>) -> impl Serve {
+ let recorder = Arc::clone(self);
+ move |target: ModelId, request: Request| {
+ let recorder = Arc::clone(&recorder);
+ async move {
+ let target = target.to_string();
+ recorder.calls.lock().push(Call {
+ target: target.clone(),
+ messages: request
+ .llm_request
+ .messages
+ .iter()
+ .filter_map(|message| message.text_content("|"))
+ .collect(),
+ });
+ let completion = if target == JUDGE {
+ let p_solve = *recorder.judge_p_solve.lock();
+ format!(
+ r#"{{"crux":"bounded task","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":{p_solve}}}"#
+ )
+ } else {
+ target
+ };
+ Ok(reply(completion))
+ }
+ }
+ }
+}
+
+/// One tool step: a task, a `Bash` call, and its result.
+pub(crate) fn turn_request(failed: bool) -> Request {
+ let content = if failed {
+ "fatal runtime error: out of memory"
+ } else {
+ "ok"
+ };
+ Request {
+ llm_request: LlmRequest {
+ model: Some("auto".to_string()),
+ messages: vec![
+ Message::text(Role::User, "fix the build"),
+ Message {
+ role: Role::Assistant,
+ content: vec![ContentBlock::ToolCall(ToolCall {
+ id: "call_1".to_string(),
+ name: "Bash".to_string(),
+ arguments: json!({"command": "cargo test"}),
+ })],
+ },
+ Message {
+ role: Role::Tool,
+ content: vec![ContentBlock::ToolResult(ToolResult {
+ tool_call_id: "call_1".to_string(),
+ content: vec![ContentBlock::Text {
+ text: content.to_string(),
+ }],
+ is_error: Some(failed),
+ })],
+ },
+ ],
+ ..LlmRequest::default()
+ },
+ raw_request: Some(json!({
+ "model": "auto",
+ "messages": [
+ {"role": "user", "content": "fix the build"},
+ {"role": "assistant", "tool_calls": [{"id": "call_1", "type": "function",
+ "function": {"name": "Bash", "arguments": "{\"command\": \"cargo test\"}"}}]},
+ {"role": "tool", "tool_call_id": "call_1", "content": content},
+ ],
+ })),
+ metadata: Some(Metadata {
+ wire_format: Some(WireFormat::OpenAiChat),
+ session_id: Some("session-1".to_string()),
+ ..Default::default()
+ }),
+ }
+}
diff --git a/crates/libsy/src/algorithms/util/tool_signals.rs b/crates/libsy/src/algorithms/util/tool_signals.rs
index 388417c45..a974184f7 100644
--- a/crates/libsy/src/algorithms/util/tool_signals.rs
+++ b/crates/libsy/src/algorithms/util/tool_signals.rs
@@ -291,7 +291,7 @@ impl Default for ToolSignalProcessor {
#[async_trait]
impl Processor<State> for ToolSignalProcessor {
async fn process(&self, state: &mut State, event: Event<'_>) -> Result<()> {
- if let Event::Request(req) = event {
+ if let Event::Request { request: req, .. } = event {
let tool_signal = ToolSignals::from_request(req, Some(self.recent_window));
state.tool_signals = Some(tool_signal);
}
diff --git a/crates/libsy/src/core/processor.rs b/crates/libsy/src/core/processor.rs
index af03bdb89..04d38eda4 100644
--- a/crates/libsy/src/core/processor.rs
+++ b/crates/libsy/src/core/processor.rs
@@ -2,6 +2,7 @@
// SPDX-License-Identifier: Apache-2.0
use crate::Result;
+use crate::core::algorithm::Driver;
use async_trait::async_trait;
use switchyard_protocol::{AggLlmResponse, ModelId, Request};
@@ -12,7 +13,12 @@ use switchyard_protocol::{AggLlmResponse, ModelId, Request};
/// the chain and to the model call. The observation-only variants stay immutable.
pub enum Event<'a> {
/// The inbound request that begins a turn.
- Request(&'a mut Request),
+ Request {
+ /// The request, rewritable in place.
+ request: &'a mut Request,
+ /// Offered so a processor can consult a model before the cascade runs.
+ driver: Option<&'a Driver>,
+ },
/// A routing decision paired with the request that produced it.
///
/// The request is rewritable: a processor may add instructions or notes here that
@@ -46,7 +52,7 @@ mod tests {
/// The key each event variant tallies under.
fn event_key(event: &Event<'_>) -> &'static str {
match event {
- Event::Request(_) => "requests",
+ Event::Request { .. } => "requests",
Event::Decision { .. } => "decisions",
Event::ModelResponse(_) => "model_responses",
}
@@ -85,7 +91,13 @@ mod tests {
let selected_model_id = ModelId::from("test/model");
// Feed one of every event variant through the processor.
processor
- .process(&mut state, Event::Request(&mut req))
+ .process(
+ &mut state,
+ Event::Request {
+ request: &mut req,
+ driver: None,
+ },
+ )
.await?;
processor
.process(&mut state, Event::ModelResponse(&response))
@@ -113,7 +125,13 @@ mod tests {
for _ in 0..3 {
processor
- .process(&mut state, Event::Request(&mut req))
+ .process(
+ &mut state,
+ Event::Request {
+ request: &mut req,
+ driver: None,
+ },
+ )
.await?;
}
@@ -128,7 +146,7 @@ mod tests {
impl Processor for RewritingProcessor {
async fn process(&self, _state: &mut (), event: Event<'_>) -> Result<()> {
match event {
- Event::Request(request) | Event::Decision { request, .. } => {
+ Event::Request { request, .. } | Event::Decision { request, .. } => {
request.llm_request.model = Some("rewritten".to_string());
}
_ => {}
@@ -144,7 +162,13 @@ mod tests {
assert_eq!(req.model_id(), Some("auto".into()));
RewritingProcessor
- .process(&mut state, Event::Request(&mut req))
+ .process(
+ &mut state,
+ Event::Request {
+ request: &mut req,
+ driver: None,
+ },
+ )
.await?;
// The edit outlives the call, so the next component sees the rewritten request.
diff --git a/crates/libsy/src/lib.rs b/crates/libsy/src/lib.rs
index 47a26c083..80f31459d 100644
--- a/crates/libsy/src/lib.rs
+++ b/crates/libsy/src/lib.rs
@@ -15,6 +15,7 @@ pub use error::{DriverError, LibsyError, Result};
mod algorithms;
pub use algorithms::advisor_gate::{AdvisorGate, AdvisorGateConfig, GateTrigger};
+pub use algorithms::hierarchical::{HierarchicalRouter, HierarchicalRouterConfig};
pub use algorithms::llm_class::{
CustomClassifierConfig, CustomClassifierPolicy, LlmClassifierConfig, LlmTaskClassifier,
TaskClassifierConfig,
diff --git a/crates/switchyard-runner/src/algorithm.rs b/crates/switchyard-runner/src/algorithm.rs
index b4d400d74..a71d32731 100644
--- a/crates/switchyard-runner/src/algorithm.rs
+++ b/crates/switchyard-runner/src/algorithm.rs
@@ -11,9 +11,10 @@ use std::sync::Arc;
use libsy::{
AdvisorGate, AdvisorGateConfig, Algorithm, ClassifierContractConfig, ClassifierResponseFormat,
ClassifyTrigger, CustomClassifierConfig, CustomClassifierPolicy, EscalationJudgeConfig,
- GateTrigger, HandoffNoteConfig, LlmClassifierConfig, LlmFallback, LlmTaskClassifier, Noop,
- Passthrough, PickerMode, Random, StageRouter, StageRouterConfig, SubagentRouter,
- SubagentRouterConfig, TargetPrompts, TaskClassifierConfig,
+ GateTrigger, HandoffNoteConfig, HierarchicalRouter, HierarchicalRouterConfig,
+ LlmClassifierConfig, LlmFallback, LlmTaskClassifier, Noop, Passthrough, PickerMode, Random,
+ StageRouter, StageRouterConfig, SubagentRouter, SubagentRouterConfig, TargetPrompts,
+ TaskClassifierConfig,
};
use serde::Deserialize;
use switchyard_protocol::ModelId;
@@ -246,26 +247,10 @@ pub enum AlgorithmSpec {
},
/// Picks a tier per turn by scoring signals from recent tool results.
StageRouter {
- /// The capable tier.
- capable_target: String,
- /// The efficient tier.
- efficient_target: String,
+ #[serde(flatten)]
+ tiers: StageTierConfig,
/// Tier to use when the signals are not confident.
picker: PickerMode,
- /// How much agreement a decisive pick needs, from 0 to 1.
- confidence_threshold: f64,
- /// How many trailing tool results the signals are scored over.
- #[serde(default)]
- recent_turn_window: Option<usize>,
- /// Notes handed to a tier when the router switches to it.
- #[serde(default)]
- handoff_notes: Option<HandoffNoteConfig>,
- /// System prompt handed to the capable tier.
- #[serde(default)]
- capable_system_prompt: Option<String>,
- /// System prompt handed to the efficient tier.
- #[serde(default)]
- efficient_system_prompt: Option<String>,
/// Judge consulted for turns the tool signals cannot decide.
#[serde(default)]
classifier: Option<StageClassifierConfig>,
@@ -273,6 +258,16 @@ pub enum AlgorithmSpec {
#[serde(default)]
subagents: Option<SubagentRouteConfig>,
},
+ /// A judge picks the tier at each user turn; a stage router runs the turns within it.
+ Hierarchical {
+ /// Judge that picks the tier. Called through its own target.
+ classifier: StageClassifierConfig,
+ /// The stage router the judge hands off to.
+ stage: StageTierConfig,
+ /// Separate policy for delegated sub-agent work.
+ #[serde(default)]
+ subagents: Option<SubagentRouteConfig>,
+ },
/// Serves every turn from one target, and has a second model review some of
/// those turns before the caller sees them.
Advisor {
@@ -364,6 +359,33 @@ pub struct StageClassifierConfig {
pub max_output_tokens: u64,
}
+/// The tier pair and scoring settings shared by every stage-router-backed route.
+///
+/// `deny_unknown_fields` here is what rejects a typo on a flattened `stage_router`
+/// route, since the enum's own `deny_unknown_fields` does not apply through a flatten.
+#[derive(Clone, Debug, Deserialize)]
+#[serde(deny_unknown_fields)]
+pub struct StageTierConfig {
+ /// The capable tier.
+ pub capable_target: String,
+ /// The efficient tier.
+ pub efficient_target: String,
+ /// How much agreement a decisive pick needs, from 0 to 1.
+ pub confidence_threshold: f64,
+ /// How many trailing tool results the signals are scored over.
+ #[serde(default)]
+ pub recent_turn_window: Option<usize>,
+ /// Notes handed to a tier when the router switches to it.
+ #[serde(default)]
+ pub handoff_notes: Option<HandoffNoteConfig>,
+ /// System prompt handed to the capable tier.
+ #[serde(default)]
+ pub capable_system_prompt: Option<String>,
+ /// System prompt handed to the efficient tier.
+ #[serde(default)]
+ pub efficient_system_prompt: Option<String>,
+}
+
impl StageClassifierConfig {
fn task_classifier_config(&self) -> TaskClassifierConfig {
TaskClassifierConfig {
@@ -421,12 +443,24 @@ impl AlgorithmSpec {
}
}
Self::StageRouter {
- capable_target,
- efficient_target,
- subagents,
- ..
+ tiers, subagents, ..
} => {
- let mut names = vec![capable_target.as_str(), efficient_target.as_str()];
+ let mut names = vec![
+ tiers.capable_target.as_str(),
+ tiers.efficient_target.as_str(),
+ ];
+ if let Some(subagents) = subagents {
+ names.extend(subagents.routing_target_names());
+ }
+ names
+ }
+ Self::Hierarchical {
+ stage, subagents, ..
+ } => {
+ let mut names = vec![
+ stage.capable_target.as_str(),
+ stage.efficient_target.as_str(),
+ ];
if let Some(subagents) = subagents {
names.extend(subagents.routing_target_names());
}
@@ -464,6 +498,16 @@ impl AlgorithmSpec {
names.extend(subagents.classifier_target_name());
}
}
+ Self::Hierarchical {
+ classifier,
+ subagents,
+ ..
+ } => {
+ names.push(&classifier.target);
+ if let Some(subagents) = subagents {
+ names.extend(subagents.classifier_target_name());
+ }
+ }
Self::Advisor { advisor_target, .. } => names.push(advisor_target),
_ => {}
}
@@ -889,18 +933,21 @@ fn build_algorithm(
Ok(Arc::new(algorithm))
}
AlgorithmSpec::StageRouter {
- capable_target,
- efficient_target,
+ tiers,
picker,
- confidence_threshold,
- recent_turn_window,
- handoff_notes,
- capable_system_prompt,
- efficient_system_prompt,
classifier,
subagents,
..
} => {
+ let StageTierConfig {
+ capable_target,
+ efficient_target,
+ confidence_threshold,
+ recent_turn_window,
+ handoff_notes,
+ capable_system_prompt,
+ efficient_system_prompt,
+ } = tiers;
if matches!(picker, PickerMode::CapableFirst) {
tracing::warn!(
"stage_router route {route_name} uses picker \"capable_first\", which is experimental: published thresholds and routing results all come from \"efficient_first\", so there is no calibrated confidence_threshold for it and no measured accuracy or cost. Use \"efficient_first\" unless you are running your own calibration."
@@ -939,6 +986,39 @@ fn build_algorithm(
let parent: Arc<dyn Algorithm> = Arc::new(algorithm);
attach_subagent_router(route_name, parent, subagents.as_ref(), targets)
}
+ AlgorithmSpec::Hierarchical {
+ classifier,
+ stage,
+ subagents,
+ } => {
+ let capable = resolve_target_model_id(route_name, &stage.capable_target, targets)?;
+ let efficient = resolve_target_model_id(route_name, &stage.efficient_target, targets)?;
+ let judge_target = resolve_target_model_id(route_name, &classifier.target, targets)?;
+ let mut stage_config =
+ StageRouterConfig::new(PickerMode::EfficientFirst, stage.confidence_threshold);
+ stage_config.recent_window = stage.recent_turn_window;
+ stage_config.handoff_notes = stage.handoff_notes.clone();
+ stage_config.tier_prompts = tier_prompts(
+ &capable,
+ stage.capable_system_prompt.as_deref(),
+ &efficient,
+ stage.efficient_system_prompt.as_deref(),
+ );
+ let config = HierarchicalRouterConfig {
+ judge_target,
+ judge: classifier.task_classifier_config(),
+ stage: stage_config,
+ };
+ let algorithm =
+ HierarchicalRouter::new(capable, efficient, config).map_err(|error| {
+ AlgorithmConfigError::with_source(
+ format!("hierarchical route {route_name}: {error}"),
+ error,
+ )
+ })?;
+ let parent: Arc<dyn Algorithm> = Arc::new(algorithm);
+ attach_subagent_router(route_name, parent, subagents.as_ref(), targets)
+ }
AlgorithmSpec::Advisor {
executor_target,
advisor_target,
diff --git a/crates/switchyard-runner/src/config.rs b/crates/switchyard-runner/src/config.rs
index 74428895b..26527d99d 100644
--- a/crates/switchyard-runner/src/config.rs
+++ b/crates/switchyard-runner/src/config.rs
@@ -644,6 +644,41 @@ base_threshold = 0.5
)
}
+ fn hierarchical_config() -> String {
+ format!(
+ r#"{VALID_CONFIG}
+[targets.tier_judge]
+id = "tier-judge/model"
+llm_client = "primary"
+
+[routes.hier]
+id = "switchyard/hier"
+type = "hierarchical"
+
+[routes.hier.classifier]
+target = "tier_judge"
+base_threshold = 0.5
+classify_trigger = "user_turn"
+
+[routes.hier.stage]
+capable_target = "strong"
+efficient_target = "weak"
+confidence_threshold = 0.5
+"#
+ )
+ }
+
+ #[test]
+ fn hierarchical_route_builds_and_claims_both_tiers_and_its_judge() -> RunnerResult<()> {
+ let runner = runner_from_toml(&hierarchical_config())?;
+ assert!(
+ runner
+ .models()
+ .any(|model| model.id.as_str() == "switchyard/hier")
+ );
+ Ok(())
+ }
+
#[test]
fn builds_all_supported_algorithm_types() -> RunnerResult<()> {
let state = runner_from_toml(VALID_CONFIG)?;
@@ -785,6 +820,24 @@ base_threshold = 0.5
);
}
+ #[test]
+ fn stage_router_rejects_an_unknown_field() {
+ let config = stage_config().replace(
+ "picker = \"efficient_first\"",
+ "picker = \"efficient_first\"\nmagic = true",
+ );
+ assert!(error_message(&config).contains("unknown field"));
+ }
+
+ #[test]
+ fn hierarchical_stage_block_rejects_an_unknown_field() {
+ let config = hierarchical_config().replace(
+ "confidence_threshold = 0.5",
+ "confidence_threshold = 0.5\nmagic = true",
+ );
+ assert!(error_message(&config).contains("unknown field"));
+ }
+
#[test]
fn rejects_unknown_fields_and_algorithm_types() {
let unknown_field =
diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.md
index b607644e9..fdf6a9c4b 100644
--- a/docs/reference/toml_schema.md
+++ b/docs/reference/toml_schema.md
@@ -211,6 +211,32 @@ optional `handoff_notes` and `classifier` tables and for tuning.
| `classifier.response_format_type` | No | `json_schema` | Structured-output mode for the optional classifier judge. Use `json_object` when the classifier provider does not support JSON Schema; Switchyard adds the schema to the prompt and validates the verdict locally. |
| `subagents` | No | unset | Nested `passthrough` or custom `llm_classifier` policy used only for delegated sub-agent work. See [Sub-Agent-Aware Routing](../routing_algorithms/subagent_routing.md). |
+### `hierarchical`
+
+Stacks other algorithms in a hierarchy, where one can set another's
+configuration. Today a classifier sets the tier a stage router falls open to when its own signals are not confident, leaving its scoring and escalation logic untouched. See
+[Hierarchical Routing](../routing_algorithms/hierarchical_routing.md).
+
+| Key | Required | Default | Meaning |
+|---|:---:|---|---|
+| `classifier.target` | Yes | — | Target the tier judge is called through. Not a routing destination. |
+| `classifier.base_threshold` | Yes | — | `p_solve` floor that still routes to the efficient tier. In `[0, 1]`. |
+| `classifier.classify_trigger` | Yes | — | `user_turn` re-picks the tier whenever the user speaks, `new_session` picks once and holds it. `every_request` is rejected here: a judge call per tool step is the cost this route exists to avoid. |
+| `classifier.message_hash_fallback` | No | `false` | Retains the tier by hashing the first user message, for clients that send no session ID. Unlike the `llm_classifier` route, this works on either trigger. Conversations opening with the same text share a tier. |
+| `stage.capable_target` | Yes | — | Capable tier. |
+| `stage.efficient_target` | Yes | — | Efficient tier. |
+| `stage.confidence_threshold` | Yes | — | Corroboration a decisive signal needs. In `[0, 1]`. |
+| `stage.recent_turn_window` | No | `3` | Trailing tool results the signals are computed over. |
+| `stage.capable_system_prompt` | No | unset | System prompt handed to the capable tier. |
+| `stage.efficient_system_prompt` | No | unset | System prompt handed to the efficient tier. |
+| `subagents` | No | unset | Nested policy used only for delegated sub-agent work. |
+
+The tier is retained per session. A deployment that sends no session ID needs
+`classifier.message_hash_fallback = true`, which keys on the first user message
+instead. The stage table takes no `picker`: the classifier supplies that tier per turn. A turn the
+classifier cannot reach falls open to the efficient tier. Leaving out
+`classifier` is recommended: that judge runs ahead of the fall-open tier.
+
## Validation Errors
`--dry-run` prefixes configuration failures with
diff --git a/docs/routing_algorithms/hierarchical_routing.md b/docs/routing_algorithms/hierarchical_routing.md
new file mode 100644
index 000000000..18784d1a6
--- /dev/null
+++ b/docs/routing_algorithms/hierarchical_routing.md
@@ -0,0 +1,55 @@
+# Hierarchical Routing
+
+Routing algorithms composed in a hierarchy. An algorithm higher in the hierarchy
+sets the configuration of the one below it before handing off, so each algorithm
+runs where its evidence is strongest.
+
+## What exists today
+
+One pairing: an LLM classifier sets the tier a stage router falls open to when
+its own signals are not confident. The stage router's `picker` mode is unchanged,
+and so is how it scores signals, escalates, and de-escalates. Both algorithms and the
+relationship between them are fixed, and the configuration below names them
+directly rather than composing arbitrary algorithms.
+
+The general form, where any algorithm can target another and set its
+configuration, is the direction rather than the current behaviour. Expect this
+route's configuration to change as more pairings are added.
+
+## Configuration
+
+```toml
+[routes.switchyard]
+id = "switchyard"
+type = "hierarchical"
+
+[routes.switchyard.classifier]
+target = "judge"
+base_threshold = 0.5
+classify_trigger = "user_turn"
+
+[routes.switchyard.stage]
+capable_target = "strong"
+efficient_target = "weak"
+confidence_threshold = 0.5
+```
+
+`[routes.switchyard.classifier]` takes the `stage_router` classifier fields.
+`classify_trigger` sets how often the judge runs: `user_turn` re-picks the tier
+whenever the user speaks, `new_session` picks once and holds it.
+`[routes.switchyard.stage]` takes the `stage_router` fields except `picker`, whose
+job the classifier does per turn. Leave `classifier` out as well: that judge runs
+ahead of the fall-open tier, so it answers most of the turns this one decided.
+
+## Sessions
+
+The tier is retained per session, so send `x-switchyard-session-id` and each
+conversation keeps its own. A client that sends no session ID can set
+`classifier.message_hash_fallback = true`, which keys on the first user message
+instead. Conversations that open with the same text then share a tier.
+
+## Example use
+
+An interactive coding agent in auto-mode. The classifier reads the user turn and
+picks the tier. The stage router runs the tool execution loop on its own signals
+until the next user turn.
diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.md
index 635959a4a..2503af57e 100644
--- a/docs/routing_algorithms/overview.md
+++ b/docs/routing_algorithms/overview.md
@@ -16,6 +16,7 @@ configuration and tuning. For the vocabulary these pages use, see
| [Random Routing](random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |
| [LLM Classifier Routing](llm_classifier_routing.md) | Request content should decide whether a turn needs the weak or strong tier. | `llm_classifier` |
| [Stage-Router Routing](stage_router_routing.md) | Tool-result and agent-progress signals should route most turns without an extra classifier call. | `stage_router` |
+| [Hierarchical Routing](hierarchical_routing.md) | Routing algorithms are composed in a hierarchy, where one sets the configuration of another before handing off. Today an LLM classifier sets a stage router's default tier. | `hierarchical` |
| [Escalation-Router Routing](escalation_router_routing.md) | Start every task on the weak tier and escalate to strong when an LLM judge detects trouble. | `llm_classifier` with `escalation` |
| [Advisor-Gate Routing](advisor_gate_routing.md) | One model should serve every turn, with a stronger reviewer approving its "done" claims or sending back a redo plan. | `advisor` |
diff --git a/docs/routing_algorithms/stage_router_routing.md b/docs/routing_algorithms/stage_router_routing.md
index 0955d60fa..0f77bdfb8 100644
--- a/docs/routing_algorithms/stage_router_routing.md
+++ b/docs/routing_algorithms/stage_router_routing.md
@@ -233,8 +233,11 @@ efficient_system_prompt = "follow the settled plan"
### Optional: LLM classifier fallback
-By default the router uses tool signals only. To break ties on low-confidence
-turns with a model call, add a `[routes.stage.classifier]` block and set
+The block is optional, and omitting it is the default. With no
+`[routes.stage.classifier]` block there is no judge and no fallthrough to one:
+a turn the signals leave undecided goes straight to the `picker` tier.
+
+To break ties on low-confidence turns with a model call, add the block and set
`confidence_threshold` above `0.0`. The classifier is consulted only for turns
that fall below the threshold:
@@ -254,6 +257,11 @@ request. Set `response_format_type = "json_object"` for providers that only supp
JSON Object mode; Switchyard then adds the schema to the judge prompt and validates
the returned object locally. The verdict schema and routing thresholds remain unchanged.
+Leave the block out under
+[Hierarchical Routing](hierarchical_routing.md). There the tier is already picked
+at the user turn, so a fallthrough judge would sit between the signals and that
+decision and overrule it.
+
Give the classifier its own LLM client or quota bucket where possible. Sharing
one provider bucket with the efficient tier adds a request per classified turn
and can cause sustained 429s at scale.
diff --git a/mkdocs.yml b/mkdocs.yml
index 05fd415bb..46f642364 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -29,6 +29,7 @@ nav:
- Random Routing: routing_algorithms/random_routing.md
- LLM Classifier Routing: routing_algorithms/llm_classifier_routing.md
- Stage-Router Routing: routing_algorithms/stage_router_routing.md
+ - Hierarchical Routing: routing_algorithms/hierarchical_routing.md
- Escalation-Router Routing: routing_algorithms/escalation_router_routing.md
- Advisor-Gate Routing: routing_algorithms/advisor_gate_routing.md
- Operations: