Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat(runner): add label enforcement to the algorithm factory (#1238)
Alex Fournier29cfc8eb642f
diff --git a/crates/switchyard-runner/src/algorithm_factory.rs b/crates/switchyard-runner/src/algorithm_factory.rsindex 0d606aae1..9a0a7ac2f 100644--- a/crates/switchyard-runner/src/algorithm_factory.rs+++ b/crates/switchyard-runner/src/algorithm_factory.rs@@ -6,27 +6,32 @@use std::collections::BTreeMap;use std::sync::Arc;-use libsy::{Algorithm, RuntimeModels};+use libsy::{Algorithm, RuntimeModels, State, StepStream};+use parking_lot::Mutex;+use switchyard_protocol::{LlmClientError, ModelId, Request};-use crate::RunnerError;-use crate::config::DeploymentConfig;+use crate::config::{DeploymentConfig, MAX_ELIGIBLE_LANES, TargetBindings};+use crate::eligibility::{EligibilityPolicy, has_target_requirements};+use crate::{RunnerError, TargetModel};pub(crate) type AlgorithmAndModels = (Arc<dyn Algorithm>, RuntimeModels);/// Parsed deployment configuration for building algorithms with host-owned execution.pub struct AlgorithmFactory {- pub(crate) config: DeploymentConfig,+ pub(crate) config: Arc<DeploymentConfig>,}impl AlgorithmFactory {/// Parses deployment TOML without resolving credentials or constructing provider clients.- /// Invalid TOML returns a configuration error; routing settings are checked by [`Self::build`].+ /// Invalid TOML returns a configuration error; routing settings are checked when building.pub fn from_toml(source: &str) -> Result<Self, RunnerError> {let mut config: DeploymentConfig = toml::from_str(source).map_err(|error| {RunnerError::configuration_source(format!("failed to parse TOML: {error}"), error)})?;config.infer_decision_judges();- Ok(Self { config })+ Ok(Self {+ config: Arc::new(config),+ })}/// Builds algorithms and their model groups, keyed by configured route `id`.@@ -44,9 +49,125 @@ impl AlgorithmFactory {self.config.build_algorithms().map(|entry| {- let (_, config, algorithm) = entry?;- Ok((config.id.to_string(), algorithm))+ let setup = entry?;+ Ok((setup.config.id.to_string(), (setup.algorithm, setup.models)))}).collect()}++ /// Builds reusable algorithms keyed by route ID that check target requirements.+ /// Each Decision model ID must identify one client across a route's alternatives.+ /// The host handles model/Decision calls, approved answer/fallback targets, prompts,+ /// provider settings, credentials and transport. No provider clients are built or credentials read.+ pub fn build_configured(&self) -> Result<BTreeMap<String, ConfiguredAlgorithm>, RunnerError> {+ self.config.validate()?;+ self.config.validate_target_approvals()?;+ self.config+ .build_algorithms()+ .map(|entry| {+ let mut setup = entry?;+ self.config.validate_decision_models(setup.route_name)?;+ let policy = self.config.eligibility_policy(setup.route_name);+ if policy.is_some() {+ setup.models = setup.models.with_target_restriction();+ }+ Ok((+ setup.config.id.to_string(),+ ConfiguredAlgorithm {+ target_models: self.config.route_target_models(setup.route_name)?,+ standard: AlgorithmSetup {+ algorithm: setup.algorithm,+ models: Arc::new(setup.models),+ },+ eligibility: policy.map(|policy| ConfiguredEligibility {+ policy,+ deployment: Arc::clone(&self.config),+ route_name: setup.route_name.to_owned(),+ setups: Mutex::default(),+ }),+ },+ ))+ })+ .collect()+ }+}++#[derive(Clone)]+struct AlgorithmSetup {+ algorithm: Arc<dyn Algorithm>,+ models: Arc<RuntimeModels>,+}++struct ConfiguredEligibility {+ policy: EligibilityPolicy,+ deployment: Arc<DeploymentConfig>,+ route_name: String,+ setups: Mutex<BTreeMap<TargetBindings, AlgorithmSetup>>,+}++impl ConfiguredEligibility {+ fn select(+ &self,+ request: &Request,+ standard: &AlgorithmSetup,+ ) -> Result<AlgorithmSetup, RunnerError> {+ let requirements = self.policy.admit(request)?;+ let Some(bindings) = self+ .deployment+ .select_eligible_targets(&self.route_name, requirements.required_labels())?+ else {+ return Ok(standard.clone());+ };+ // No model I/O under this lock.+ let mut setups = self.setups.lock();+ if let Some(setup) = setups.get(&bindings) {+ return Ok(setup.clone());+ }+ if setups.len() >= MAX_ELIGIBLE_LANES {+ return Err(LlmClientError::TemporarilyUnavailable.into());+ }+ let (algorithm, models) = self+ .deployment+ .build_bound_algorithm(&self.route_name, &bindings)?;+ let setup = AlgorithmSetup {+ algorithm,+ models: Arc::new(models),+ };+ setups.insert(bindings, setup.clone());+ Ok(setup)+ }+}++/// A reusable algorithm with target requirement checks.+pub struct ConfiguredAlgorithm {+ standard: AlgorithmSetup,+ eligibility: Option<ConfiguredEligibility>,+ target_models: BTreeMap<ModelId, TargetModel>,+}++impl ConfiguredAlgorithm {+ /// All possible call targets, including judges, subagents and ordered alternatives.+ /// Keys are LLM target names or Decision provider model IDs, matching emitted calls.+ /// This is a lookup, not an eligible pool; execute only returned call/outcome candidates.+ pub fn target_models(&self) -> &BTreeMap<ModelId, TargetModel> {+ &self.target_models+ }++ /// Checks requirements before yielding model or Decision calls.+ /// Invalid requirements or unavailable approved targets fail before the algorithm starts.+ /// Requires Tokio. Reuse the handle to retain task requirements and algorithm-owned state;+ /// `state` is per-run routing state. Answer/fallback attempts must stay within returned candidates.+ pub fn run_stream(&self, request: Request, state: State) -> Result<StepStream, RunnerError> {+ let setup = match &self.eligibility {+ Some(eligibility) => eligibility.select(&request, &self.standard)?,+ None if has_target_requirements(&request) => {+ return Err(LlmClientError::InvalidRequest {+ message: "route does not accept target requirements".into(),+ }+ .into());+ }+ None => self.standard.clone(),+ };+ Ok(setup.algorithm.run_stream(request, setup.models, state))+ }}diff --git a/crates/switchyard-runner/src/config.rs b/crates/switchyard-runner/src/config.rsindex 1093ecbfd..9dda3efa5 100644--- a/crates/switchyard-runner/src/config.rs+++ b/crates/switchyard-runner/src/config.rs@@ -9,6 +9,7 @@ use std::path::Path;use std::sync::Arc;use std::time::Duration;+use libsy::{Algorithm, RuntimeModels};use parking_lot::Mutex;use serde::de::DeserializeOwned;use serde::{Deserialize, Deserializer};@@ -137,7 +138,7 @@ pub(crate) fn load_runner(path: impl AsRef<Path>) -> RunnerResult<Runner> {}pub(crate) fn runner_from_toml(source: &str) -> RunnerResult<Runner> {- Arc::new(AlgorithmFactory::from_toml(source)?.config).build()+ AlgorithmFactory::from_toml(source)?.config.build()}#[derive(Debug, Deserialize)]@@ -167,6 +168,14 @@ pub(crate) struct RouteConfig {eligibility: Option<TargetEligibilityConfig>,}+/// A route's algorithm and model groups with their source configuration.+pub(crate) struct RouteSetup<'a> {+ pub(crate) route_name: &'a str,+ pub(crate) config: &'a RouteConfig,+ pub(crate) algorithm: Arc<dyn Algorithm>,+ pub(crate) models: RuntimeModels,+}+#[derive(Debug, Deserialize)]#[serde(deny_unknown_fields)]struct TargetEligibilityConfig {@@ -210,7 +219,52 @@ fn callable_targets(algorithm: &AlgorithmSpec) -> BTreeSet<&str> {algorithm.callable_target_names().into_iter().collect()}-type TargetBindings = Vec<(String, String)>;+fn check_random_targets(+ algorithm: &AlgorithmSpec,+ targets: &LaneTargets<'_>,+ has_overrides: bool,+) -> RunnerResult<()> {+ if matches!(algorithm, AlgorithmSpec::Random { .. }) && has_overrides {+ let mut models = BTreeSet::new();+ for name in algorithm.routing_target_names() {+ let target = targets[name].name;+ if !models.insert(target) {+ return Err(LlmClientError::InvalidRequest {+ message: format!(+ "random route alternatives map multiple pool entries to target {target}; use distinct targets"+ ),+ }+ .into());+ }+ }+ }+ Ok(())+}++// Routing uses target names; telemetry uses provider IDs.+fn build_resolved_algorithm<'a>(+ route_name: &str,+ algorithm: &AlgorithmSpec,+ targets: &LaneTargets<'_>,+ decisions: impl Iterator<Item = (&'a str, &'a ModelId)>,+) -> RunnerResult<AlgorithmAndModels> {+ let model_ids = targets+ .iter()+ .map(|(name, target)| ((*name).to_owned(), ModelId::from(target.name)))+ .chain(decisions.map(|(name, model)| (name.to_owned(), model.clone())))+ .collect();+ let (algorithm, models) = algorithm+ .build_with_runtime_models(route_name, &model_ids)+ .map_err(|error| RunnerError::configuration_source(error.to_string(), error))?;+ let provider_models = targets+ .values()+ .map(|target| (ModelId::from(target.name), target.config.id.clone()))+ .collect();+ Ok((algorithm, models.with_provider_models(provider_models)))+}++pub(crate) type TargetBindings = Vec<(String, String)>;+pub(crate) const MAX_ELIGIBLE_LANES: usize = 128;// Cache by resolved targets, so equivalent requirements reuse algorithm-owned state.pub(crate) struct EligibleLanes {@@ -222,82 +276,39 @@ pub(crate) struct EligibleLanes {}impl EligibleLanes {- const MAX_LANES: usize = 128;-pub(crate) fn select(&self,requirements: &BTreeSet<String>,standard: &Arc<ExecutionLane>,) -> RunnerResult<Arc<ExecutionLane>> {- let route = &self.deployment.routes[&self.route_name];- let config = route- .eligibility- .as_ref()- .ok_or_else(|| RunnerError::configuration("route has no eligibility configuration"))?;- let callable = callable_targets(&route.algorithm);- let mut allowed = BTreeSet::new();- let mut overrides = TargetOverrides::new();- for name in &callable {- let candidates = std::iter::once(*name).chain(- config- .target_alternatives- .get(*name)- .into_iter()- .flatten()- .map(String::as_str),- );- if let Some(target) = candidates.into_iter().find(|name| {- self.deployment- .target_labels(name)- .is_some_and(|labels| requirements.is_subset(labels))- }) {- allowed.insert(*name);- if target != *name {- overrides.insert(*name, target);- }- }- }- if overrides.is_empty() && allowed.len() == callable.len() {+ let Some(key) = self+ .deployment+ .select_eligible_targets(&self.route_name, requirements)?+ else {return Ok(Arc::clone(standard));- }- let key = allowed- .iter()- .map(|name| {- (- (*name).to_owned(),- bound_target(name, &overrides).to_owned(),- )- })- .collect();+ };let mut lanes = self.lanes.lock();if let Some(lane) = lanes.get(&key) {return Ok(Arc::clone(lane));}- if lanes.len() >= Self::MAX_LANES {+ if lanes.len() >= MAX_ELIGIBLE_LANES {return Err(LlmClientError::TemporarilyUnavailable.into());}- let algorithm = route.algorithm.restricted_to(&allowed).map_err(|error| {- LlmClientError::InvalidRequest {- message: error.to_string(),- }- })?;+ let algorithm = self+ .deployment+ .bound_algorithm_spec(&self.route_name, &key)?;+ let overrides = key+ .iter()+ .map(|(name, target)| (name.as_str(), target.as_str()))+ .collect();let targets = self.deployment.resolve_lane_targets(&algorithm, &overrides)?;- if matches!(algorithm, AlgorithmSpec::Random { .. }) && !overrides.is_empty() {- let mut models = BTreeSet::new();- for name in algorithm.routing_target_names() {- let target = targets[name].name;- if !models.insert(target) {- return Err(LlmClientError::InvalidRequest {- message: format!(- "random route alternatives map multiple pool entries to target {target}; use distinct targets"- ),- }- .into());- }- }- }+ check_random_targets(+ &algorithm,+ &targets,+ key.iter().any(|(name, target)| name != target),+ )?;let decisions = self.deployment.resolve_decision_targets(&self.route_name,&algorithm,@@ -319,6 +330,11 @@ impl EligibleLanes {}}+struct DecisionTargetMapping<'a> {+ target_name: &'a str,+ model_id: &'a ModelId,+}+#[derive(Clone, Copy)]struct ResolvedDecisionTarget<'a> {model: &'a ModelId,@@ -409,6 +425,207 @@ impl DeploymentConfig {.any(|route| route.eligibility.is_some())}+ pub(crate) fn eligibility_policy(&self, route_name: &str) -> Option<EligibilityPolicy> {+ let config = self.routes.get(route_name)?.eligibility.as_ref()?;+ Some(EligibilityPolicy::new(+ config.require_explicit_requirements,+ matches!(config.scope, RequirementScope::Task),+ ))+ }++ pub(crate) fn route_target_models(+ &self,+ route_name: &str,+ ) -> RunnerResult<BTreeMap<ModelId, TargetModel>> {+ let route = self+ .routes+ .get(route_name)+ .ok_or_else(|| RunnerError::configuration(format!("unknown route {route_name}")))?;+ route+ .callable_target_names()+ .into_iter()+ .chain(+ route+ .eligibility+ .iter()+ .flat_map(|config| config.target_alternatives.values().flatten())+ .map(String::as_str),+ )+ .map(|name| {+ if let Some(target) = self.decision_targets.get(name) {+ Ok((+ target.id.clone(),+ TargetModel {+ client: target.decision_client.clone(),+ id: target.id.clone(),+ },+ ))+ } else {+ let target = self.targets.get(name).ok_or_else(|| {+ RunnerError::configuration(format!(+ "route {route_name} references unknown target {name}"+ ))+ })?;+ Ok((+ ModelId::from(name),+ TargetModel {+ client: target.llm_client.clone(),+ id: target.id.clone(),+ },+ ))+ }+ })+ .collect()+ }++ /// None keeps the standard setup; Some contains eligible target mappings.+ /// An empty selection never falls back to standard targets.+ pub(crate) fn select_eligible_targets(+ &self,+ route_name: &str,+ requirements: &BTreeSet<String>,+ ) -> RunnerResult<Option<TargetBindings>> {+ let route = self+ .routes+ .get(route_name)+ .ok_or_else(|| RunnerError::configuration(format!("unknown route {route_name}")))?;+ let config = route+ .eligibility+ .as_ref()+ .ok_or_else(|| RunnerError::configuration("route has no eligibility configuration"))?;+ let callable = callable_targets(&route.algorithm);+ let bindings: Vec<_> = callable+ .iter()+ .filter_map(|name| {+ std::iter::once(*name)+ .chain(+ config+ .target_alternatives+ .get(*name)+ .into_iter()+ .flatten()+ .map(String::as_str),+ )+ .find(|target| {+ self.target_labels(target)+ .is_some_and(|labels| requirements.is_subset(labels))+ })+ .map(|target| (*name, target))+ })+ .collect();+ if bindings.len() == callable.len() && bindings.iter().all(|(name, target)| name == target)+ {+ return Ok(None);+ }+ Ok(Some(+ bindings+ .into_iter()+ .map(|(name, target)| (name.to_owned(), target.to_owned()))+ .collect(),+ ))+ }++ fn bound_algorithm_spec(+ &self,+ route_name: &str,+ bindings: &TargetBindings,+ ) -> RunnerResult<AlgorithmSpec> {+ let route = self+ .routes+ .get(route_name)+ .ok_or_else(|| RunnerError::configuration(format!("unknown route {route_name}")))?;+ let allowed = bindings.iter().map(|(name, _)| name.as_str()).collect();+ route.algorithm.restricted_to(&allowed).map_err(|error| {+ LlmClientError::InvalidRequest {+ message: error.to_string(),+ }+ .into()+ })+ }++ pub(crate) fn build_bound_algorithm(+ &self,+ route_name: &str,+ bindings: &TargetBindings,+ ) -> RunnerResult<AlgorithmAndModels> {+ let algorithm = self.bound_algorithm_spec(route_name, bindings)?;+ let overrides = bindings+ .iter()+ .map(|(name, target)| (name.as_str(), target.as_str()))+ .collect();+ let targets = self.resolve_lane_targets(&algorithm, &overrides)?;+ check_random_targets(+ &algorithm,+ &targets,+ bindings.iter().any(|(name, target)| name != target),+ )?;+ let decisions = self+ .resolve_decision_models(route_name, &algorithm, &overrides)?+ .into_iter()+ .map(|mapping| (mapping.target_name, mapping.model_id));+ let (algorithm, models) =+ build_resolved_algorithm(route_name, &algorithm, &targets, decisions)?;+ Ok((algorithm, models.with_target_restriction()))+ }++ fn resolve_decision_models<'a>(+ &'a self,+ route_name: &str,+ algorithm: &'a AlgorithmSpec,+ overrides: &TargetOverrides<'_>,+ ) -> RunnerResult<Vec<DecisionTargetMapping<'a>>> {+ algorithm+ .decision_target_names()+ .into_iter()+ .map(|name| {+ let bound = bound_target(name, overrides);+ let target = self.decision_targets.get(bound).ok_or_else(|| {+ RunnerError::configuration(format!(+ "route {route_name} requires unknown decision target {bound}"+ ))+ })?;+ Ok(DecisionTargetMapping {+ target_name: name,+ model_id: &target.id,+ })+ })+ .collect()+ }++ // Host-owned Decision calls identify the model, not the selected target or client.+ pub(crate) fn validate_decision_models(&self, route_name: &str) -> RunnerResult<()> {+ let route = self+ .routes+ .get(route_name)+ .ok_or_else(|| RunnerError::configuration(format!("unknown route {route_name}")))?;+ let mut clients = HashMap::new();+ for name in route.algorithm.decision_target_names() {+ let alternatives = route+ .eligibility+ .as_ref()+ .and_then(|config| config.target_alternatives.get(name));+ for candidate in+ std::iter::once(name).chain(alternatives.into_iter().flatten().map(String::as_str))+ {+ let target = self.decision_targets.get(candidate).ok_or_else(|| {+ RunnerError::configuration(format!(+ "route {route_name} requires unknown decision target {candidate}"+ ))+ })?;+ if clients+ .insert(&target.id, &target.decision_client)+ .is_some_and(|client| client != &target.decision_client)+ {+ return Err(RunnerError::configuration(format!(+ "route {route_name} maps decision model {} to different clients",+ target.id+ )));+ }+ }+ }+ Ok(())+ }+fn target_labels(&self, name: &str) -> Option<&BTreeSet<String>> {self.targets.get(name)@@ -535,9 +752,7 @@ impl DeploymentConfig {}/// Resolves each route once; native and host-owned execution share this path.- pub(crate) fn build_algorithms(- &self,- ) -> impl Iterator<Item = RunnerResult<(&str, &RouteConfig, AlgorithmAndModels)>> {+ pub(crate) fn build_algorithms(&self) -> impl Iterator<Item = RunnerResult<RouteSetup<'_>>> {let targets = self.build_targets();self.routes.iter().map(move |(route_name, config)| {let decision_targets = config.algorithm.decision_target_names();@@ -593,13 +808,16 @@ impl DeploymentConfig {RunnerError::configuration_source(format!("route {route_name}: {error}"), error)})?;}- Ok((route_name.as_str(), config, (algorithm, models)))+ Ok(RouteSetup {+ route_name: route_name.as_str(),+ config,+ algorithm,+ models,+ })})}- fn build(self: Arc<Self>) -> RunnerResult<Runner> {- self.validate()?;-+ pub(crate) fn validate_target_approvals(&self) -> RunnerResult<()> {let mut approvals = HashMap::new();for (target_name, target) in &self.targets {if let Some((first_name, labels)) = approvals.insert(@@ -612,6 +830,12 @@ impl DeploymentConfig {)));}}+ Ok(())+ }++ fn build(self: Arc<Self>) -> RunnerResult<Runner> {+ self.validate()?;+ self.validate_target_approvals()?;let mut provider_api_keys = Vec::new();let clients = Arc::new(self.build_clients(&mut provider_api_keys)?);@@ -620,7 +844,12 @@ impl DeploymentConfig {let mut routes = Vec::with_capacity(self.routes.len());let mut model_properties_probes = BTreeMap::new();for entry in self.build_algorithms() {- let (route_name, config, (algorithm, models)) = entry?;+ let RouteSetup {+ route_name,+ config,+ algorithm,+ models,+ } = entry?;let capabilities = config.capabilities();if let AlgorithmSpec::Vgr { config: vgr } = &config.algorithm&& let Some(probe) =@@ -817,25 +1046,12 @@ impl DeploymentConfig {let (algorithm, mut models) = if let Some(built) = built {built} else {- let model_ids = targets- .iter()- .map(|(name, target)| ((*name).to_owned(), ModelId::from(target.name)))- .chain(- decisions- .iter()- .map(|(name, target)| ((*name).to_owned(), target.model.clone())),- )- .collect();- let (algorithm, models) = route- .build_with_runtime_models(route_name, &model_ids)- .map_err(|error| RunnerError::configuration_source(error.to_string(), error))?;- let models = models.with_provider_models(- targets- .values()- .map(|target| (ModelId::from(target.name), target.config.id.clone()))- .collect(),- );- (algorithm, models)+ build_resolved_algorithm(+ route_name,+ route,+ targets,+ decisions.iter().map(|(name, target)| (*name, target.model)),+ )?};let (route_clients, caller_auth) =self.build_route_clients(route_name, route, targets, decisions, clients)?;@@ -1928,6 +2144,76 @@ cutoff = 0.4Ok(())}+ #[test]+ fn factory_decision_ids_identify_one_client_across_alternatives() -> RunnerResult<()> {+ let source = format!(+ r#"{}+[decision_clients.other]+format = "system_one"+endpoint = "https://private.example.test/v1/decisions"+api_key_env = "PATH"+timeout_ms = 5000++[decision_targets.alternative]+id = "decision/alternative"+decision_client = "other"+labels = ["private"]++[routes.typed.eligibility.target_alternatives]+standard = ["alternative"]+"#,+ decision_eligibility_config()+ );+ AlgorithmFactory::from_toml(&source)?.build_configured()?;++ let shared_id = source.replace("decision/alternative", "decision/standard");+ let ambiguous = shared_id.replace(+ "decision_client = \"privacy\"\nlabels = [\"private\"]",+ "decision_client = \"privacy\"",+ );+ Runner::from_toml(&ambiguous)?;+ let later_alternative = format!(+ r#"{}+[decision_targets.later]+id = "decision/standard"+decision_client = "other"+labels = ["private", "customer_data"]+"#,+ source.replace("[\"alternative\"]", "[\"alternative\", \"later\"]")+ );+ for source in [&ambiguous, &later_alternative] {+ let error = AlgorithmFactory::from_toml(source)?+ .build_configured()+ .err()+ .expect("ambiguous endpoint must fail during construction");+ assert!(+ error+ .to_string()+ .contains("maps decision model decision/standard to different clients"),+ "{error}"+ );+ }++ let same_client = shared_id.replace(+ "decision_client = \"other\"",+ "decision_client = \"privacy\"",+ );+ AlgorithmFactory::from_toml(&same_client)?.build_configured()?;+ let separate_routes = format!(+ r#"{}+[routes.other]+id = "switchyard/other"+type = "llm_classifier"+classifier_target = "alternative"+strong_target = "strong"+weak_target = "weak"+"#,+ shared_id.replace("standard = [\"alternative\"]", "")+ );+ AlgorithmFactory::from_toml(&separate_routes)?.build_configured()?;+ Ok(())+ }+// Decision IDs remain provider keys, so bindings cannot give one ID two clients.#[tokio::test]async fn alternatives_reject_ambiguous_composite_decision_clients() -> RunnerResult<()> {@@ -1972,6 +2258,10 @@ secondary = ["approved_secondary"]let runner = Runner::from_toml(&source)?;let mut request = Request::default();require_target_labels(&mut request, ["private"])?;+ let factory_error = AlgorithmFactory::from_toml(&source)?+ .build_configured()+ .err()+ .expect("ambiguous factory decisions must fail during construction");let error = runner.route("switchyard/ambiguous").expect("composite route")@@ -1979,13 +2269,35 @@ secondary = ["approved_secondary"].await.err().expect("ambiguous decision clients must fail before model I/O");- assert!(matches!(error, RunnerError::Configuration { .. }));- assert!(- error- .to_string()- .contains("maps decision model decision/shared to different clients"),- "{error}"- );+ for error in [error, factory_error] {+ assert!(matches!(error, RunnerError::Configuration { .. }));+ assert!(+ error+ .to_string()+ .contains("maps decision model decision/shared to different clients"),+ "{error}"+ );+ }+ let standard_alias = source.replace(+ "id = \"decision/secondary\"\ndecision_client = \"privacy\"",+ "id = \"decision/standard\"\ndecision_client = \"other\"",+ );+ let ungoverned = standard_alias+ .split_once("[routes.ambiguous.eligibility.target_alternatives]")+ .expect("alternatives fixture")+ .0;+ for source in [standard_alias.as_str(), ungoverned] {+ let error = AlgorithmFactory::from_toml(source)?+ .build_configured()+ .err()+ .expect("ambiguous standard decisions must fail during construction");+ assert!(+ error+ .to_string()+ .contains("maps decision model decision/standard to different clients"),+ "{error}"+ );+ }Ok(())}diff --git a/crates/switchyard-runner/src/lib.rs b/crates/switchyard-runner/src/lib.rsindex 3e0ac98f6..366a261e9 100644--- a/crates/switchyard-runner/src/lib.rs+++ b/crates/switchyard-runner/src/lib.rs@@ -32,7 +32,7 @@ pub use algorithm::{VgrModeConfig, VgrRouteConfig,};#[cfg(not(target_os = "wasi"))]-pub use algorithm_factory::AlgorithmFactory;+pub use algorithm_factory::{AlgorithmFactory, ConfiguredAlgorithm};#[cfg(not(target_os = "wasi"))]pub use failure::{RouteErrorKind, RouteErrorPhase, RouteErrorSummary, stream_error_summary};// Re-exported because `Route::new` takes it, so a host wiring routes does not need a libsy dep.diff --git a/crates/switchyard-runner/tests/algorithm_factory.rs b/crates/switchyard-runner/tests/algorithm_factory.rsindex 9bce587a3..1a00e581c 100644--- a/crates/switchyard-runner/tests/algorithm_factory.rs+++ b/crates/switchyard-runner/tests/algorithm_factory.rs@@ -82,6 +82,18 @@ fn full_deployment_builds_without_provider_credentials() {);}+ let configured = factory.build_configured().expect("configured handles");+ let parent = configured["public/main"].target_models();+ assert_eq!(+ parent.keys().map(ModelId::as_str).collect::<Vec<_>>(),+ ["capable", "fast"]+ );+ assert_eq!(parent[&ModelId::from("capable")].client, "answers");+ assert_eq!(parent[&ModelId::from("capable")].id, "upstream/capable");+ let judged = configured["public/classify"].target_models();+ assert_eq!(judged[&ModelId::from("upstream/judge")].client, "judge");+ assert!(!judged.contains_key(&ModelId::from("judge")));+let error = Runner::from_toml(DEPLOYMENT).err().expect("the native runner still requires configured credentials");diff --git a/crates/switchyard-runner/tests/factory_target_requirements.rs b/crates/switchyard-runner/tests/factory_target_requirements.rsnew file mode 100644index 000000000..f276a60ed--- /dev/null+++ b/crates/switchyard-runner/tests/factory_target_requirements.rs@@ -0,0 +1,341 @@+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+// SPDX-License-Identifier: Apache-2.0++//! Configured factories enforce target requirements before host-owned calls.++use std::collections::BTreeMap;++use futures_util::StreamExt;+use libsy::{RoutingOutcome, State, Step, StepStream};+use serde_json::json;+use switchyard_protocol::{LlmResponse, Request, Response, text_request, text_response};+use switchyard_runner::{AlgorithmFactory, Runner, require_target_labels, route_targets};+use wiremock::matchers::{body_partial_json, method, path};+use wiremock::{Mock, MockServer, ResponseTemplate};++const MISSING_KEY: &str = "SWITCHYARD_FACTORY_TEST_KEY_THAT_IS_NOT_SET";+const STAGE: &str = "type = 'stage_router'\nefficient_target = 'fast'\ncapable_target = 'capable'\npicker = 'efficient_first'\nconfidence_threshold = 0.6";++fn deployment(scope: &str, algorithm: &str, judge_alternative: &str) -> String {+ let targets = [+ ("fast", "standard", &[] as &[&str]),+ ("capable", "standard", &[]),+ ("private_fast", "approved", &["private"]),+ ("private_capable", "approved", &["private"]),+ ("both_fast", "approved", &["private", "customer_data"]),+ ("both_capable", "approved", &["private", "customer_data"]),+ ("judge", "standard", &[]),+ ("approved_judge", "approved", &["private", "customer_data"]),+ ]+ .into_iter()+ .map(|(name, client, labels)| {+ let id = if matches!(name, "fast" | "private_fast") {+ "shared-model"+ } else {+ name+ };+ format!(+ "[targets.{name}]\nid = {id:?}\nllm_client = {client:?}\nlabels = {}\n",+ json!(labels)+ )+ })+ .collect::<String>();+ format!(+ r#"+schema_version = 1+[llm_clients.standard]+format = "openai_chat"+base_url = "http://127.0.0.1:1/v1"+api_key_env = "{MISSING_KEY}"+[llm_clients.approved]+format = "openai_chat"+base_url = "http://127.0.0.1:2/v1"+api_key_env = "{MISSING_KEY}"+{targets}+[decision_clients.scoring]+format = "openai_decisions"+endpoint = "http://127.0.0.1:1/v1/decisions"+api_key_env = "{MISSING_KEY}"+timeout_ms = 1000+[decision_targets.judge_decision]+id = "decision/standard"+decision_client = "scoring"+[decision_targets.approved_decision]+id = "decision/approved"+decision_client = "scoring"+labels = ["private", "customer_data"]+[routes.coding]+id = "switchyard/coding"+{algorithm}+[routes.coding.eligibility]+require_explicit_requirements = true+scope = "{scope}"+[routes.coding.eligibility.target_alternatives]+fast = ["private_fast", "both_fast"]+capable = ["private_capable", "both_capable"]+{judge_alternative}+"#+ )+}++fn request(task: Option<&str>, labels: Option<&[&str]>) -> Request {+ let mut request = Request {+ llm_request: text_request(None, "Summarize these notes."),+ ..Request::default()+ };+ request.metadata.get_or_insert_default().task_id = task.map(str::to_owned);+ if let Some(labels) = labels {+ require_target_labels(&mut request, labels).expect("trusted labels");+ }+ request+}++async fn outcome(mut stream: StepStream) -> RoutingOutcome {+ let Step::Done(result) = stream+ .next()+ .await+ .expect("routing step")+ .expect("successful routing")+ else {+ panic!("unexpected model call");+ };+ assert!(stream.next().await.is_none());+ *result+}++#[tokio::test]+async fn configured_factory_matches_runner_without_reading_credentials() {+ assert!(std::env::var_os(MISSING_KEY).is_none());+ let source = deployment("request", STAGE, "");+ let factory = AlgorithmFactory::from_toml(&source).expect("deployment");+ let configured = factory.build_configured().expect("no credentials needed");+ let native = Runner::from_toml(&source.replace(MISSING_KEY, "PATH")).expect("native route");+ let route = &configured["switchyard/coding"];+ let state: State = serde_json::from_value(json!({"host_marker": {"bool": true}})).unwrap();+ for (labels, expected) in [+ (&[] as &[&str], ["fast", "capable"]),+ (&["private", "customer_data"], ["both_fast", "both_capable"]),+ ] {+ let request = request(None, Some(labels));+ let native = native+ .route("switchyard/coding")+ .unwrap()+ .decide(request.clone())+ .await+ .unwrap();+ let selected = outcome(route.run_stream(request, state.clone()).unwrap()).await;+ assert_eq!(selected.selected_model_ids, expected);+ assert_eq!(selected.selected_model_ids, native.selected_model_ids);+ assert_eq!(+ selected.state.extra["host_marker"],+ state.extra["host_marker"]+ );+ }+}++#[tokio::test]+async fn host_resolves_alternatives_from_the_configured_handle() {+ let standard = MockServer::start().await;+ let approved = MockServer::start().await;+ for (server, client) in [(&standard, "standard"), (&approved, "approved")] {+ Mock::given(method("POST"))+ .and(path("/v1/chat/completions"))+ .and(body_partial_json(json!({"model": "shared-model"})))+ .respond_with(ResponseTemplate::new(200).set_body_json(json!({"client": client})))+ .expect(1)+ .mount(server)+ .await;+ }+ let clients = BTreeMap::from([("standard", standard.uri()), ("approved", approved.uri())]);+ let source = deployment("request", STAGE, "");+ let configured = AlgorithmFactory::from_toml(&source)+ .unwrap()+ .build_configured()+ .unwrap();+ let route = &configured["switchyard/coding"];+ assert_eq!(route.target_models().len(), 6);+ let roles = route_targets(&source).unwrap();+ assert_eq!(+ roles[0]+ .answer_targets+ .iter()+ .map(|target| target.name.as_str())+ .collect::<Vec<_>>(),+ ["capable", "fast"]+ );+ assert!(roles[0].judge_targets.is_empty());++ let http = reqwest::Client::new();+ for (labels, expected) in [(&[] as &[&str], "standard"), (&["private"], "approved")] {+ let selected = outcome(+ route+ .run_stream(request(None, Some(labels)), State::default())+ .unwrap(),+ )+ .await;+ let target = &route.target_models()[&selected.selected_model_ids[0]];+ let response: serde_json::Value = http+ .post(format!(+ "{}/v1/chat/completions",+ clients[target.client.as_str()]+ ))+ .json(&json!({+ "model": target.id,+ "messages": [{"role": "user", "content": "Summarize these notes."}]+ }))+ .send()+ .await+ .unwrap()+ .error_for_status()+ .unwrap()+ .json()+ .await+ .unwrap();+ assert_eq!(response["client"], expected);+ }+ standard.verify().await;+ approved.verify().await;+}++#[test]+fn invalid_requirements_fail_before_a_stream_is_created() {+ let configured = AlgorithmFactory::from_toml(&deployment("task", STAGE, ""))+ .unwrap()+ .build_configured()+ .unwrap();+ let route = &configured["switchyard/coding"];+ for request in [+ request(Some("task"), None),+ request(None, Some(&[])),+ request(Some("task"), Some(&["unapproved"])),+ ] {+ assert!(route.run_stream(request, State::default()).is_err());+ }+ let source = deployment("request", STAGE, "");+ let ungoverned = source.split("[routes.coding.eligibility]").next().unwrap();+ let configured = AlgorithmFactory::from_toml(ungoverned)+ .unwrap()+ .build_configured()+ .unwrap();+ assert!(+ configured["switchyard/coding"]+ .run_stream(request(None, Some(&[])), State::default())+ .is_err()+ );+}++#[tokio::test]+async fn task_requirements_accumulate_and_survive_unavailable_targets() {+ let source = deployment("task", STAGE, "");+ let configured = AlgorithmFactory::from_toml(&source)+ .unwrap()+ .build_configured()+ .unwrap();+ let route = &configured["switchyard/coding"];+ for (task, labels, expected) in [+ ("shared", &["private"] as &[&str], "private_fast"),+ ("shared", &["customer_data"], "both_fast"),+ ("shared", &[], "both_fast"),+ ("other", &[], "fast"),+ ] {+ let selected = outcome(+ route+ .run_stream(request(Some(task), Some(labels)), State::default())+ .unwrap(),+ )+ .await;+ assert_eq!(selected.selected_model_ids[0], expected);+ }+ for labels in [&["unapproved"] as &[&str], &[]] {+ assert!(+ route+ .run_stream(request(Some("shared"), Some(labels)), State::default())+ .is_err()+ );+ }+}++#[tokio::test]+async fn approved_judges_are_served_by_the_host() {+ for decision in [false, true] {+ let judge = if decision { "judge_decision" } else { "judge" };+ let settings = if decision {+ "[routes.coding.decision]\ncutoff = 0.7"+ } else {+ "base_threshold = 0.5"+ };+ let algorithm = format!(+ "type = 'llm_classifier'\nclassifier_target = '{judge}'\nstrong_target = 'capable'\nweak_target = 'fast'\nfail_open = false\n{settings}"+ );+ let alternative = if decision {+ "judge_decision = ['approved_decision']"+ } else {+ "judge = ['approved_judge']"+ };+ let configured =+ AlgorithmFactory::from_toml(&deployment("request", &algorithm, alternative))+ .unwrap()+ .build_configured()+ .unwrap();+ let route = &configured["switchyard/coding"];+ let mut stream = route+ .run_stream(+ request(None, Some(&["private", "customer_data"])),+ State::default(),+ )+ .unwrap();+ match stream.next().await.unwrap().unwrap() {+ Step::CallModel(call) => {+ assert!(!decision);+ assert_eq!(call.models, ["approved_judge"]);+ let target = &route.target_models()[&call.models[0]];+ assert_eq!(target.client, "approved");+ assert_eq!(target.id, "approved_judge");+ let response = Response {+ llm_response: LlmResponse::Agg(text_response(+ None,+ r#"{"crux":"bounded task","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":0.9}"#,+ )),+ metadata: None,+ upstream_headers: Default::default(),+ };+ call.respond(std::future::ready(Ok(response)))+ .await+ .unwrap();+ }+ Step::CallDecision(call) => {+ assert!(decision);+ assert_eq!(call.model, "decision/approved");+ assert_eq!(call.request.model.as_ref(), Some(&call.model));+ let target = &route.target_models()[&call.model];+ assert_eq!(target.client, "scoring");+ assert_eq!(target.id, call.model);+ let response = serde_json::from_value(json!({+ "answers": {+ "route": {+ "value": {+ "type": "choice",+ "data": {+ "selected": "efficient",+ "probabilities": {+ "efficient": 0.9,+ "capable": 0.1+ }+ }+ }+ }+ }+ }))+ .unwrap();+ call.respond(std::future::ready(Ok(response)))+ .await+ .unwrap();+ }+ Step::Done(_) => panic!("judge must be served by the host"),+ }+ assert_eq!(+ outcome(stream).await.selected_model_ids,+ ["both_fast", "both_capable"]+ );+ }+}diff --git a/docs/internal/target-eligibility.md b/docs/internal/target-eligibility.mdindex fa2b487bc..1fb71a322 100644--- a/docs/internal/target-eligibility.md+++ b/docs/internal/target-eligibility.md@@ -92,6 +92,32 @@ optional policy, both allow normal routing. Withlabels accumulate within one request but are not retained across requests, evenwhen they carry the same task ID.+### Host-owned model calls++Use the configured factory when the host makes every model call, including judges:++```rust+use switchyard_runner::{AlgorithmFactory, require_target_labels};++let routes = AlgorithmFactory::from_toml(config)?.build_configured()?;+let route = routes.get("switchyard/assistant")+ .ok_or_else(|| anyhow::anyhow!("unknown route"))?;+require_target_labels(&mut request, ["private", "customer_data"])?;+let steps = route.run_stream(request, Default::default())?;+```++Requirements are checked before the algorithm starts. The stream uses the existing+model-call, Decision-call and routing-outcome types. The host resolves configured+LLM target names to its deployments and applies their prompts, provider settings,+credentials and transport. Final answer attempts must stay within the returned+approved candidates. Decision calls retain their provider model IDs.++Build once and reuse the handles across turns. Each build has independent task+requirements and algorithm-owned state. The second `run_stream` argument is the+caller's per-run libsy state; it does not replace retained task requirements.+No provider clients are built or credential values read. The original `build()`+method still returns bare algorithms and rejects eligibility configurations.+## Task requirementsSet `scope = "task"` to retain requirements across requests. Every request needs@@ -104,9 +130,9 @@ both from then on. Missing or empty requirements do not remove retained labels.If the combined requirements cannot be served, the request fails and the labelsremain retained.-State is in memory and belongs to one route instance. There is no reset, expiry,-replication, restart persistence or revocation. Overlapping in-flight requests-keep the requirements under which they were admitted.+State is in memory and belongs to one route or configured factory handle. There+is no reset, expiry, replication, restart persistence or revocation. Overlapping+in-flight requests keep the requirements under which they were admitted.## Execution@@ -117,7 +143,8 @@ flowchart TDP -->|"no valid setup"| E["Error: no provider call"]P --> R["Existing routing algorithm"]R --> G["libsy checks allowed model IDs"]- G --> C["LLM client executes approved calls and fallbacks"]+ G -->|"Route execution"| C["LLM client executes approved calls and fallbacks"]+ G -->|"Configured factory"| F["Host executes approved calls and fallbacks"]```Runner owns label matching and reuses setups that resolve to the same configured@@ -132,7 +159,7 @@ cleared and routing state is not evicted.Provider-held Responses continuation state and forwarded caller authenticationare rejected on eligibility routes.-An integration must enter through Runner; attaching metadata to an algorithm-only-binding does not perform admission. This core adds no classifier, model runtime,-new HTTP input or provider format. Existing execution signatures and Relay's C-ABI stay unchanged.+An integration must enter through a configured route or `build_configured()`;+attaching metadata to a bare algorithm does not check requirements. This core+adds no classifier, model runtime, new HTTP input or provider format. Existing+execution signatures and Relay's C ABI stay unchanged.