Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
feat(vgr): expose verified routing surface (#719)
GDRS249484be44df
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5ba34afc1..2d43f0d47 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,10 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).### Added+- **VGR routing (planned for 0.4.0)** — adds `type = "vgr"` deployment routes+ and the `AlgorithmSpec::Vgr` variant. **Rust source-breaking change:** callers+ that match exhaustively on `AlgorithmSpec` must handle the new variant.+- **Speculative workflows** - algorithms can run model and decision calls inparallel, stop one call, and let another continue. This includes decisionmodels used as speculative tool callers. The same controls support generaldiff --git a/crates/libsy-llm-client/src/backend.rs b/crates/libsy-llm-client/src/backend.rsindex 60b962a07..461fca4ed 100644--- a/crates/libsy-llm-client/src/backend.rs+++ b/crates/libsy-llm-client/src/backend.rs@@ -31,6 +31,7 @@ const OPENAI_OVERFLOW_PHRASES: &[&str] = &["please reduce the length of the input","exceeds the maximum allowed input length","exceeds the maximum allowed length",+ "exceeds the available context size","is longer than the model's context length",// llama-server, both of its ERROR_TYPE_EXCEED_CONTEXT_SIZE messages// (tools/server/server-context.cpp). The structured check below catches@@ -207,6 +208,10 @@ impl Backend {}}+ pub(crate) fn base_url(&self) -> &str {+ &self.config().base_url+ }+/// The fully resolved upstream URL for this backend's endpoint.////// Tolerates base URLs that already include the provider path (or a barediff --git a/crates/libsy-llm-client/src/client.rs b/crates/libsy-llm-client/src/client.rsindex 76679b47d..2db3950bf 100644--- a/crates/libsy-llm-client/src/client.rs+++ b/crates/libsy-llm-client/src/client.rs@@ -297,6 +297,9 @@ impl TranslatingLlmClient {// value and any `reasoning` default a target set through `extra_body`.apply_reasoning_effort(&mut body, backend);if matches!(backend, Backend::Anthropic(_)) {+ if nvidia_inference_api_backend(backend) {+ strip_anthropic_tool_strict(&mut body);+ }enable_anthropic_prompt_caching(&mut body);}if matches!(backend, Backend::OpenAiChat(_)) {@@ -1096,6 +1099,26 @@ fn strip_anthropic_incompatible_fields(body: &mut Value) {}}+// NVIDIA Inference API's Anthropic-compatible schema rejects the strict tool-use+// extension that native Anthropic accepts.+fn nvidia_inference_api_backend(backend: &Backend) -> bool {+ reqwest::Url::parse(backend.base_url())+ .ok()+ .is_some_and(|url| {+ url.host_str()+ .is_some_and(|host| host.eq_ignore_ascii_case("inference-api.nvidia.com"))+ })+}++fn strip_anthropic_tool_strict(body: &mut Value) {+ let Some(tools) = body.get_mut("tools").and_then(Value::as_array_mut) else {+ return;+ };+ for tool in tools.iter_mut().filter_map(Value::as_object_mut) {+ tool.remove("strict");+ }+}+// Removes replayed `thinking` blocks that carry no signature.//// Anthropic requires signed thinking blocks on replay. A router can serve earlier@@ -1589,6 +1612,18 @@ mod tests {assert!(!convert_reqwest_error(error).to_string().contains("CANARY"));}+ #[test]+ fn nvidia_anthropic_tools_drop_unsupported_strict_field() {+ let backend = Backend::Anthropic(config("https://inference-api.nvidia.com"));+ assert!(nvidia_inference_api_backend(&backend));+ assert!(!nvidia_inference_api_backend(&Backend::Anthropic(config(+ "https://inference-api.nvidia.com.example.test"+ ))));+ let mut body = json!({"tools": [{"name": "a", "input_schema": {}, "strict": true}]});+ strip_anthropic_tool_strict(&mut body);+ assert!(body["tools"][0].get("strict").is_none());+ }+#[test]fn anthropic_prompt_caching_marks_final_message() {let caller_managed = json!({@@ -2242,6 +2277,66 @@ mod tests {Ok(())}+ #[tokio::test]+ async fn nvidia_anthropic_requests_strip_strict_from_extra_body()+ -> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/messages"))+ .respond_with(ResponseTemplate::new(200).set_body_json(json!({+ "id": "msg_1",+ "type": "message",+ "role": "assistant",+ "model": "claude",+ "content": [{"type": "text", "text": "ok"}],+ "stop_reason": "end_turn",+ "usage": {"input_tokens": 1, "output_tokens": 1}+ })))+ .expect(1)+ .mount(&server)+ .await;++ let mut backend = config(&format!(+ "http://inference-api.nvidia.com:{}",+ server.address().port()+ ));+ backend.extra_body.insert(+ "tools".into(),+ json!([{+ "name": "lookup",+ "input_schema": {"type": "object", "properties": {}},+ "strict": true+ }]),+ );+ let mut client = TranslatingLlmClient::new(&[ModelConfig::new(+ "claude",+ Backend::Anthropic(backend),+ None,+ )])?;+ client.client = reqwest::Client::builder()+ .no_proxy()+ .resolve("inference-api.nvidia.com", *server.address())+ .build()?;+ client+ .call_rewrite_model_raw(+ json!({+ "model": "claude",+ "max_tokens": 7,+ "messages": [{"role": "user", "content": "hi"}]+ }),+ None,+ Some(&ModelId::from("claude")),+ WireFormat::AnthropicMessages,+ )+ .await?;++ let requests = server.received_requests().await.unwrap();+ let body: Value = serde_json::from_slice(&requests[0].body)?;+ assert_eq!(body["tools"][0]["name"], "lookup");+ assert!(body["tools"][0].get("strict").is_none());+ Ok(())+ }+// A router can serve earlier turns from an OpenAI target and later turns from// an Anthropic one, so the Anthropic leg must drop OpenAI-only fields the// caller keeps sending or the upstream rejects the whole request.@@ -2725,12 +2820,14 @@ mod tests {}#[tokio::test]- async fn context_overflow_400_is_mapped()+ async fn llama_cpp_context_overflow_400_is_mapped()-> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {let server = MockServer::start().await;Mock::given(method("POST")).respond_with(ResponseTemplate::new(400).set_body_json(json!({- "error": {"code": "context_length_exceeded", "message": "too big"}+ "error": {+ "message": "request (6016 tokens) exceeds the available context size (4096 tokens), try increasing it"+ }}))).mount(&server).await;diff --git a/crates/libsy/src/algorithms/vgr/config.rs b/crates/libsy/src/algorithms/vgr/config.rsindex d171a3913..1b6471a52 100644--- a/crates/libsy/src/algorithms/vgr/config.rs+++ b/crates/libsy/src/algorithms/vgr/config.rs@@ -84,6 +84,11 @@ impl VgrConfig {}pub(super) fn validate(&self) -> Result<()> {+ if self.targets.local == self.targets.cloud {+ return Err(LibsyError::AlgorithmError {+ message: "vgr requires distinct local and cloud model IDs".into(),+ });+ }if self.deadline.is_zero() || self.breaker.threshold == 0 {return Err(LibsyError::AlgorithmError {message: "vgr deadline and breaker threshold must be non-zero".into(),@@ -110,3 +115,14 @@ impl VgrConfig {.map(|tail| i32::try_from(tail.max(1)).unwrap_or(i32::MAX))}}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn direct_callers_must_use_distinct_local_and_cloud_ids() {+ assert!(super::super::Vgr::new(VgrConfig::new("same".into(), "same".into())).is_err());+ assert!(super::super::Vgr::new(VgrConfig::new("local".into(), "cloud".into())).is_ok());+ }+}diff --git a/crates/switchyard-runner/src/algorithm.rs b/crates/switchyard-runner/src/algorithm.rsindex c83e9ee76..a994ffc9c 100644--- a/crates/switchyard-runner/src/algorithm.rs+++ b/crates/switchyard-runner/src/algorithm.rs@@ -10,13 +10,13 @@ use std::path::PathBuf;use std::sync::Arc;use libsy::{- AdvisorGate, AdvisorGateConfig, Algorithm, CapabilityJudgeConfig, ClassifierContractConfig,- ClassifierResponseFormat, ClassifyTrigger, CompositeRouter, CompositeRouterConfig,- CustomClassifierConfig, CustomClassifierPolicy, DecisionJudgeConfig, EscalationJudgeConfig,- GateTrigger, HandoffNoteConfig, LlmCapabilityConfig, LlmClassifierConfig, LlmFallback,- LlmTaskClassifier, Noop, Passthrough, PickerMode, PlanExecute, PlanExecuteConfig, Random,- RuntimeModels, StageRouter, StageRouterConfig, SubagentRouter, SubagentRouterConfig,- TaskClassifierConfig, ToolSemantics,+ AdvisorGate, AdvisorGateConfig, Algorithm, BreakerConfig, CapabilityJudgeConfig,+ ClassifierContractConfig, ClassifierResponseFormat, ClassifyTrigger, CompositeRouter,+ CompositeRouterConfig, CustomClassifierConfig, CustomClassifierPolicy, DecisionJudgeConfig,+ EscalationJudgeConfig, GateTrigger, HandoffNoteConfig, LlmCapabilityConfig,+ LlmClassifierConfig, LlmFallback, LlmTaskClassifier, Noop, Passthrough, PickerMode,+ PlanExecute, PlanExecuteConfig, Random, RuntimeModels, StageRouter, StageRouterConfig,+ SubagentRouter, SubagentRouterConfig, TaskClassifierConfig, ToolSemantics, Vgr, VgrConfig,};use serde::Deserialize;use serde_json::Value;@@ -498,6 +498,10 @@ pub enum AlgorithmSpec {#[serde(default = "default_fail_open")]fail_open: bool,},+ Vgr {+ #[serde(flatten)]+ config: VgrRouteConfig,+ },/// Routes using a checkpoint-backed prefill classifier.PrefillRouter {/// Target names in checkpoint output order.@@ -516,6 +520,43 @@ pub enum AlgorithmSpec {},}+#[derive(Clone, Debug, Deserialize)]+#[serde(deny_unknown_fields)]+pub struct VgrRouteConfig {+ pub local_target: String,+ pub cloud_target: String,+ /// Defaults to `local_target`.+ #[serde(default)]+ pub judge_target: Option<String>,+ #[serde(default)]+ pub cloud_judge_target: Option<String>,+ #[serde(default)]+ pub mode: VgrModeConfig,+ #[serde(default)]+ pub active_approval: Option<String>,+ #[serde(default = "default_vgr_deadline")]+ pub deadline_seconds: f64,+ #[serde(default = "default_true")]+ pub task_typing: bool,+ #[serde(default = "default_vgr_breaker_threshold")]+ pub breaker_threshold: u32,+ #[serde(default = "default_vgr_breaker_cooldown")]+ pub breaker_cooldown_seconds: f64,+ /// Clean trailing tool results that let a long run recover on the cloud judge.+ #[serde(default)]+ pub confirmed_recovery_min_clean_tail: Option<u32>,+}++#[derive(Clone, Debug, Default, Deserialize)]+#[serde(rename_all = "snake_case")]+pub enum VgrModeConfig {+ #[default]+ Off,+ Evaluate,+ Shadow,+ Active,+}+/// What fires an advisor route's review.#[derive(Clone, Debug, Default, Deserialize, PartialEq)]#[serde(rename_all = "snake_case")]@@ -734,6 +775,9 @@ impl AlgorithmSpec {Self::Advisor {executor_target, ..} => vec![executor_target],+ Self::Vgr { config } => {+ vec![config.local_target.as_str(), config.cloud_target.as_str()]+ }Self::PrefillRouter { targets, .. } => targets.iter().map(String::as_str).collect(),}}@@ -770,6 +814,10 @@ impl AlgorithmSpec {names.push(&classifier.target);}Self::Advisor { advisor_target, .. } => names.push(advisor_target),+ Self::Vgr { config } => {+ names.extend(config.judge_target.as_deref());+ names.extend(config.cloud_judge_target.as_deref());+ }_ => {}}// A sub-agent classifier calls its own judge, which is never a completion target.@@ -863,6 +911,22 @@ impl AlgorithmSpec {(Category::Any, vec![executor_target.clone()]),(Category::Judge, vec![advisor_target.clone()]),]),+ Self::Vgr { config } => {+ let judge = config.judge_target.as_ref().unwrap_or(&config.local_target);+ let judges = std::iter::once(judge)+ .chain(&config.cloud_judge_target)+ .cloned()+ .collect();+ category_models([+ (Category::Efficient, vec![config.local_target.clone()]),+ (Category::Capable, vec![config.cloud_target.clone()]),+ (+ Category::Any,+ vec![config.local_target.clone(), config.cloud_target.clone()],+ ),+ (Category::Judge, judges),+ ])+ }};let subagents = match self {@@ -896,6 +960,13 @@ impl AlgorithmSpec {advisor_target,..} => Some((executor_target, advisor_target)),+ Self::Vgr { config } => Some((+ &config.local_target,+ config+ .judge_target+ .as_deref()+ .unwrap_or(&config.local_target),+ )),Self::Noop { .. }| Self::Random { .. }| Self::Passthrough { .. }@@ -1592,6 +1663,7 @@ fn build_algorithm(})?;Ok(Arc::new(algorithm))}+ AlgorithmSpec::Vgr { config } => build_vgr(route_name, config, targets),AlgorithmSpec::PrefillRouter {targets: names,checkpoint,@@ -1637,6 +1709,56 @@ fn build_algorithm(}}+fn build_vgr(+ route_name: &str,+ config: &VgrRouteConfig,+ targets: &BTreeMap<String, ModelId>,+) -> AlgorithmResult<Arc<dyn Algorithm>> {+ let local = resolve_target_model_id(route_name, &config.local_target, targets)?;+ let cloud = resolve_target_model_id(route_name, &config.cloud_target, targets)?;+ let optional = |name: &Option<String>| {+ name.as_deref()+ .map(|name| resolve_target_model_id(route_name, name, targets))+ .transpose()+ };+ let mut runtime = VgrConfig::new(local, cloud);+ runtime.targets.judge = optional(&config.judge_target)?;+ runtime.targets.cloud_judge = optional(&config.cloud_judge_target)?;+ runtime.mode = match config.mode {+ VgrModeConfig::Off => libsy::ServingMode::Off,+ VgrModeConfig::Evaluate => libsy::ServingMode::Evaluate,+ VgrModeConfig::Shadow => libsy::ServingMode::Shadow,+ VgrModeConfig::Active => libsy::ServingMode::Active {+ approval: config.active_approval.clone().unwrap_or_default(),+ },+ };+ runtime.deadline = duration(route_name, "deadline_seconds", config.deadline_seconds)?;+ runtime.task_typing = config.task_typing;+ runtime.confirmed_recovery_min_clean_tail = config.confirmed_recovery_min_clean_tail;+ runtime.breaker = BreakerConfig {+ threshold: config.breaker_threshold,+ cooldown: duration(+ route_name,+ "breaker_cooldown_seconds",+ config.breaker_cooldown_seconds,+ )?,+ };+ Vgr::new(runtime)+ .map(|algorithm| Arc::new(algorithm) as Arc<dyn Algorithm>)+ .map_err(|error| {+ AlgorithmConfigError::with_source(format!("vgr route {route_name}: {error}"), error)+ })+}++fn duration(route: &str, field: &str, seconds: f64) -> AlgorithmResult<std::time::Duration> {+ std::time::Duration::try_from_secs_f64(seconds).map_err(|error| {+ AlgorithmConfigError::with_source(+ format!("vgr route {route}: {field} must be a non-negative number"),+ error,+ )+ })+}+const fn default_max_reviews() -> u32 {1}@@ -1672,6 +1794,22 @@ fn warn_single_target_classifier(route_name: &str, models: &CategoryModelConfig)}}+const fn default_vgr_deadline() -> f64 {+ 30.0+}++const fn default_true() -> bool {+ true+}++const fn default_vgr_breaker_threshold() -> u32 {+ 5+}++const fn default_vgr_breaker_cooldown() -> f64 {+ 30.0+}+fn resolve_target_model_id(route_name: &str,name: &str,diff --git a/crates/switchyard-runner/src/config.rs b/crates/switchyard-runner/src/config.rsindex 70f472ec9..b13b1cfa8 100644--- a/crates/switchyard-runner/src/config.rs+++ b/crates/switchyard-runner/src/config.rs@@ -1153,6 +1153,42 @@ target = "strong"}}+ fn vgr_config() -> String {+ format!(+ r#"{VALID_CONFIG}++[routes.vgr]+id = "switchyard/vgr"+type = "vgr"+local_target = "weak"+cloud_target = "strong"+judge_target = "classifier"+mode = "active"+active_approval = "prospective-validation-and-canary-approved"+task_typing = false+"#+ )+ }++ #[test]+ fn vgr_route_builds_and_rejects_unsafe_configuration() -> RunnerResult<()> {+ let runner = runner_from_toml(&vgr_config())?;+ assert!(runner.route("switchyard/vgr").is_some());++ let identical_ids =+ vgr_config().replace("local_target = \"weak\"", "local_target = \"strong\"");+ assert!(error_message(&identical_ids).contains("distinct local and cloud model IDs"));++ let bad_approval =+ vgr_config().replace("prospective-validation-and-canary-approved", "approved");+ assert!(error_message(&bad_approval).contains("approval attestation"));++ let missing_target =+ vgr_config().replace("local_target = \"weak\"", "local_target = \"missing\"");+ assert!(error_message(&missing_target).contains("unknown target missing"));+ Ok(())+ }+fn with_subagent_llm_classifier(config: &str, route: &str, extra: &str) -> String {let mut configured = config.to_string();configured.push_str(&format!("\n[routes.{route}.subagents]\n"));diff --git a/crates/switchyard-runner/src/lib.rs b/crates/switchyard-runner/src/lib.rsindex 2908f1047..36ff10841 100644--- a/crates/switchyard-runner/src/lib.rs+++ b/crates/switchyard-runner/src/lib.rs@@ -15,7 +15,7 @@ pub use config::api_key_variables;pub use algorithm::{AdvisorTriggerConfig, AlgorithmConfigError, AlgorithmSpec, CategoryModelConfig, ClassifierMode,ClassifierPolicyConfig, DecisionJudgeRouteConfig, LlmClassifierRouteConfig,- StageClassifierConfig, SubagentRouteConfig,+ StageClassifierConfig, SubagentRouteConfig, VgrModeConfig, VgrRouteConfig,};pub use failure::{RouteErrorKind, RouteErrorPhase, RouteErrorSummary, stream_error_summary};// Re-exported because `Route::new` takes it, so a host wiring routes does not need a libsy dep.diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rsindex ed207fc73..6e6ad34b3 100644--- a/crates/switchyard-server/tests/server.rs+++ b/crates/switchyard-server/tests/server.rs@@ -14,7 +14,7 @@ use axum::extract::{DefaultBodyLimit, Path, State};use axum::http::{HeaderMap, HeaderValue, Request as HttpRequest, StatusCode, Uri};use axum::response::sse::{Event, Sse};use axum::response::{IntoResponse, Response as HttpResponse};-use axum::routing::post;+use axum::routing::{get, post};use axum::{Json, Router};use http_body_util::BodyExt;use libsy::{Algorithm, Random};@@ -50,6 +50,7 @@ impl MockUpstream {async fn start() -> TestResult<Self> {let calls = Arc::new(Mutex::new(Vec::new()));let app = Router::new()+ .route("/props", get(upstream_props)).route("/v1/chat/completions", post(upstream_chat)).route("/buffered/responses", post(upstream_buffered_responses)).route(@@ -118,6 +119,11 @@ fn has_system_prompt(call: &Value, expected: &str) -> bool {})}+async fn upstream_props(State(calls): State<Arc<Mutex<Vec<Value>>>>) -> Json<Value> {+ calls.lock().await.push(json!({"path": "/props"}));+ Json(json!({"default_generation_settings": {"n_ctx": 98_304}}))+}+async fn upstream_chat(State(calls): State<Arc<Mutex<Vec<Value>>>>,Json(body): Json<Value>,@@ -443,6 +449,10 @@ async fn upstream_chat(r#"{"crux":"bounded task","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":0.1,"unexpected":true}"#.to_string()} else if model == "model/classifier" {r#"{"crux":"bounded task","primary_rule":"SUP-1","capability_boundary":"supported","p_solve":0.9}"#.to_string()+ } else if model == "model/vgr-judge" {+ "yes".to_string()+ } else if model == "model/vgr-reject" {+ "no".to_string()} else {"ok".to_string()};@@ -1673,6 +1683,75 @@ impl Response {}}+#[tokio::test]+async fn vgr_serves_verified_attempts_locally_and_escalates_rejected_ones() -> TestResult {+ let upstream = MockUpstream::start().await?;+ for (judge, served) in [+ ("model/vgr-judge", "model/vgr-local"),+ ("model/vgr-reject", "model/vgr-cloud"),+ ] {+ let app = build_switchyard_router(load_test_config(&format!(+ r#"+schema_version = 1++[llm_clients.primary]+format = "openai_chat"+base_url = "{}"++[targets.local]+id = "model/vgr-local"+llm_client = "primary"++[targets.cloud]+id = "model/vgr-cloud"+llm_client = "primary"++[targets.judge]+id = "{judge}"+llm_client = "primary"++[routes.vgr]+id = "switchyard/vgr"+type = "vgr"+context_window = 32768+local_target = "local"+cloud_target = "cloud"+judge_target = "judge"+mode = "active"+active_approval = "prospective-validation-and-canary-approved"+task_typing = false+"#,+ upstream.base_url+ ))?);+ let response = send(+ &app,+ "POST",+ "/v1/chat/completions",+ Some(json!({+ "model": "switchyard/vgr",+ "messages": [{"role": "user", "content": "answer the question"}]+ })),+ )+ .await?;+ assert_eq!(response.status, StatusCode::OK);+ assert_eq!(+ response.headers.get("x-model-router-selected-model"),+ Some(&HeaderValue::from_str(served)?)+ );+ let models = send(&app, "GET", "/v1/models", None).await?.json()?;+ assert_eq!(models["data"][0]["context_length"], 32_768);+ assert!(+ upstream+ .calls+ .lock()+ .await+ .iter()+ .all(|call| call["path"] != "/props")+ );+ }+ Ok(())+}+fn metric_line<'a>(metrics: &'a str, name: &str, labels: &[(&str, &str)]) -> Option<&'a str> {metrics.lines().find(|line| {line.starts_with(name)diff --git a/docs/routing_algorithms/overview.md b/docs/routing_algorithms/overview.mdindex 1a1eac7f0..3878d7d3c 100644--- a/docs/routing_algorithms/overview.md+++ b/docs/routing_algorithms/overview.md@@ -47,6 +47,7 @@ These options remain available when you need a different routing policy.| [Advisor Gate](advisor_gate_routing.md) | Keep one executor model and have a stronger advisor review its plans and completion claims. | `advisor` || [Sub-Agent-Aware Routing](subagent_routing.md) | Delegated sub-agents should use a separate routing policy from the parent agent. | `passthrough`, `stage_router`, or `composite` with `subagents` || [Random Routing](random_routing.md) | You need a fixed traffic split for A/B tests, baselines, or cost experiments. | `random` |+| [Verification-Gated Routing](vgr_routing.md) | Serve a local attempt only after bounded verification, and send the request to the capable model otherwise. | `vgr` || [Fixed Model](#direct-model-routes) | Send every request to one target without a routing decision. | `passthrough` |### Experimentaldiff --git a/docs/routing_algorithms/vgr_routing.md b/docs/routing_algorithms/vgr_routing.mdnew file mode 100644index 000000000..ba0e1dde8--- /dev/null+++ b/docs/routing_algorithms/vgr_routing.md@@ -0,0 +1,53 @@+# Verification-Gated Routing++Verification-Gated Routing (VGR) generates a candidate on a local tier. For a+final answer, it gathers bounded evidence about that exact candidate and serves+it only when the policy allows it. All other final-answer outcomes use the cloud+tier. Missing context, unavailable verifiers, malformed verdicts, and expired+decision deadlines fail closed to cloud for final answers.++Tool-calling turns continue on the local tier unless the turn judge provides+evidence to escalate. An unavailable or inconclusive turn judge leaves the tool+turn on the local tier. A local endpoint failure triggers cloud fallback.++## Configure a route++Declare local, cloud, and optional verifier targets using the normal deployment+schema, then reference their target names:++```toml+[routes.assistant]+id = "assistant"+type = "vgr"+local_target = "local"+cloud_target = "cloud"+judge_target = "local"+cloud_judge_target = "cloud"+mode = "shadow"+deadline_seconds = 30+task_typing = true+breaker_threshold = 5+breaker_cooldown_seconds = 30+confirmed_recovery_min_clean_tail = 1+```++`judge_target` defaults to `local_target`. `cloud_judge_target` is optional.+The local and cloud targets must resolve to distinct model IDs. When the+route sets `context_window`, `/v1/models` advertises that configured limit.+`confirmed_recovery_min_clean_tail`, when set, lets an agentic+run that recovered from tool errors commit locally once that many trailing tool+results are clean and the cloud judge confirms the evidence.++## Serving modes++- `off` skips candidate generation and serves cloud. This is the default.+- `shadow` computes a decision but serves cloud.+- `evaluate` serves the policy decision for isolated evaluation.+- `active` serves the policy decision in production and requires+ `active_approval = "prospective-validation-and-canary-approved"`.++VGR buffers a local candidate before releasing it. A streamed candidate that is+licensed is replayed as a stream; a rejected candidate is not sent to the+client.++`x-model-router-selected-model` on each response names the tier that served it.diff --git a/mkdocs.yml b/mkdocs.ymlindex 57cf32d62..a291889d3 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -39,6 +39,7 @@ nav:- Composite Routing: routing_algorithms/composite_routing.md- Escalation-Router Routing: routing_algorithms/escalation_router_routing.md- Advisor-Gate Routing: routing_algorithms/advisor_gate_routing.md+ - Verification-Gated Routing: routing_algorithms/vgr_routing.md- Operations:- Context-Window Handling: operations/context_window.md- Soak Testing: operations/soak_test.md