Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
fix(server): reject inputs disabled by route capabilities (#750)
Elyas Mehtabuddincfe38952c390
diff --git a/crates/switchyard-runner/src/route.rs b/crates/switchyard-runner/src/route.rsindex e3640c5f4..9d177c768 100644--- a/crates/switchyard-runner/src/route.rs+++ b/crates/switchyard-runner/src/route.rs@@ -17,7 +17,9 @@ use crate::DecisionTarget;/// Capabilities declared for one route.////// `GET /v1/models` includes `context_window`, `tool_calling`, and `vision` in each-/// standard `data` entry, using `null` for unset values. `reasoning` remains route metadata.+/// standard `data` entry, using `null` for unset values. It omits `reasoning`.+/// The server rejects tool inputs, reasoning controls, or images when the+/// corresponding declaration is explicitly `false` for the selected route.#[derive(Clone, Copy, Default)]pub struct ModelCapabilities {pub context_window: Option<u32>,diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.mdindex 7e3dbb404..bdf2f22fd 100644--- a/crates/switchyard-server/README.md+++ b/crates/switchyard-server/README.md@@ -152,6 +152,17 @@ with `codex --model route-id`; route aliases do not appear automatically in Codepicker. Unknown aliases use Codex's generic defaults and do not receive Switchyard'sroute-specific context limits or tool settings.+For registered routes, the server returns HTTP 400 before dispatch when a request+contains inputs disabled by `vision = false`, `reasoning = false`, or+`tool_calling = false`. OpenAI errors use the `unsupported_capability` code;+Anthropic errors use `invalid_request_error`. Remove the unsupported input or select+a compatible route. Explicit `true` and unset declarations do not restrict requests.++Codex keeps its own model settings, so it may still send inputs that the server rejects.+The server preserves caller instructions and does not remove images, reasoning controls,+or tools to make a request fit. These checks apply to registered server routes;+transparent forwarding through `fallback_client` and direct library calls remain unchanged.+To add instructions for a target, set `system_prompt` on its `[targets.<name>]` entry.Switchyard prepends that text when the selected target serves a completion and retainsthe caller's instructions. Omit the setting to add no target instructions.diff --git a/crates/switchyard-server/src/capabilities.rs b/crates/switchyard-server/src/capabilities.rsnew file mode 100644index 000000000..6ac5379d5--- /dev/null+++ b/crates/switchyard-server/src/capabilities.rs@@ -0,0 +1,275 @@+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.+// SPDX-License-Identifier: Apache-2.0++//! Reject inputs that the selected route explicitly disables.++use serde_json::Value;+use switchyard_protocol::{ContentBlock, LlmRequest};+use switchyard_runner::ModelCapabilities;++/// Return the first explicitly disabled capability used by decoded or preserved input.+/// Forwarding can retain provider JSON fields that decoding omits.+pub(crate) fn unsupported_capability(+ capabilities: ModelCapabilities,+ request: &LlmRequest,+ body: &Value,+) -> Option<&'static str> {+ if capabilities.reasoning == Some(false)+ && (request.reasoning.effort.is_some()+ || request+ .reasoning+ .raw+ .as_ref()+ .is_some_and(|value| !value.is_null())+ || ["reasoning", "reasoning_effort", "thinking"]+ .iter()+ .any(|key| body.get(key).is_some_and(|value| !value.is_null()))+ || body+ .pointer("/output_config/effort")+ .is_some_and(|value| !value.is_null()))+ {+ return Some("reasoning");+ }+ if capabilities.tool_calling == Some(false)+ && (!request.tools.is_empty()+ || request.tool_choice.is_some()+ || ["tools", "functions"].iter().any(|key| {+ body.get(key)+ .and_then(Value::as_array)+ .is_some_and(|tools| !tools.is_empty())+ })+ || ["tool_choice", "parallel_tool_calls", "function_call"]+ .iter()+ .any(|key| body.get(key).is_some_and(|value| !value.is_null())))+ {+ return Some("tool_calling");+ }+ if capabilities.tool_calling != Some(false) && capabilities.vision != Some(false) {+ return None;+ }+ request+ .instructions+ .iter()+ .map(|instruction| instruction.content.as_slice())+ .chain(+ request+ .messages+ .iter()+ .map(|message| message.content.as_slice()),+ )+ .find_map(|content| unsupported_content(capabilities, content))+ .or_else(|| unsupported_preserved_content(capabilities, body))+}++// Some provider fields survive forwarding without becoming decoded content blocks.+// Inspect protocol content and tool outputs, leaving tool arguments and text untouched.+fn unsupported_preserved_content(+ capabilities: ModelCapabilities,+ body: &Value,+) -> Option<&'static str> {+ let mut pending: Vec<_> = ["messages", "input", "system"]+ .iter()+ .filter_map(|key| body.get(key))+ .collect();+ while let Some(value) = pending.pop() {+ if let Some(items) = value.as_array() {+ pending.extend(items);+ continue;+ }+ let kind = value+ .get("type")+ .and_then(Value::as_str)+ .unwrap_or_default();+ if capabilities.vision == Some(false)+ && matches!(+ kind,+ "image" | "image_url" | "input_image" | "computer_screenshot"+ )+ {+ return Some("vision");+ }+ if capabilities.tool_calling == Some(false)+ && (matches!(+ kind,+ "additional_tools"+ | "tool_use"+ | "tool_result"+ | "mcp_list_tools"+ | "mcp_approval_request"+ | "mcp_approval_response"+ ) || kind.ends_with("_call")+ || kind.ends_with("_call_output")+ || kind.ends_with("_tool_use")+ || kind.ends_with("_tool_result"))+ {+ return Some("tool_calling");+ }+ pending.extend(value.get("content"));+ if kind.ends_with("_call_output") {+ pending.extend(value.get("output"));+ }+ }+ None+}++/// Check decoded blocks, including images nested inside tool-result content.+fn unsupported_content(+ capabilities: ModelCapabilities,+ content: &[ContentBlock],+) -> Option<&'static str> {+ content.iter().find_map(|block| match block {+ ContentBlock::Image { .. } if capabilities.vision == Some(false) => Some("vision"),+ ContentBlock::ToolCall(_) | ContentBlock::ToolResult(_)+ if capabilities.tool_calling == Some(false) =>+ {+ Some("tool_calling")+ }+ ContentBlock::ToolResult(result) => unsupported_content(capabilities, &result.content),+ _ => None,+ })+}++#[cfg(test)]+mod tests {+ use super::*;+ use serde_json::json;+ use switchyard_translation::WireFormat::{AnthropicMessages, OpenAiChat, OpenAiResponses};+ use switchyard_translation::{WireFormat, decode_request, encode_request};++ type TestResult<T = ()> = Result<T, Box<dyn std::error::Error>>;++ // Decode each fixture so disabled, enabled, and unset cases use the same request.+ fn assert_requires_capability(+ format: WireFormat,+ body: &Value,+ disabled: ModelCapabilities,+ capability: &str,+ ) -> TestResult<LlmRequest> {+ let request = decode_request(format, body)?;+ for (capabilities, expected) in [+ (disabled, Some(capability)),+ (ModelCapabilities::default(), None),+ (+ ModelCapabilities {+ tool_calling: Some(true),+ reasoning: Some(true),+ vision: Some(true),+ ..Default::default()+ },+ None,+ ),+ ] {+ assert_eq!(+ unsupported_capability(capabilities, &request, body),+ expected,+ "{body}"+ );+ }+ Ok(request)+ }++ // Tool controls and history, including MCP approvals, can survive forwarding+ // without appearing in decoded tools.+ #[test]+ fn rejects_tool_controls_and_history_preserved_outside_normalized_tools() -> TestResult {+ let capabilities = ModelCapabilities {+ tool_calling: Some(false),+ ..Default::default()+ };+ for body in [+ json!({"input": "hello", "tools": [{"type": "web_search"}]}),+ json!({"input": "hello", "functions": [{"name": "lookup", "parameters": {"type": "object"}}]}),+ json!({"input": "hello", "tool_choice": "none"}),+ json!({"input": "hello", "tool_choice": {"type": "web_search"}}),+ json!({"input": "hello", "parallel_tool_calls": false}),+ json!({"input": [{"type": "additional_tools", "tools": []}]}),+ json!({"input": [{"type": "shell_call", "id": "shell_1", "call_id": "call_1", "action": {"commands": ["pwd"]}}]}),+ json!({"input": [{"type": "function_call_output", "call_id": "call_1", "output": "done"}]}),+ json!({"input": [{"type": "mcp_list_tools", "server_label": "server", "tools": []}]}),+ json!({"input": [{"type": "mcp_approval_request", "id": "approval_1", "name": "lookup", "arguments": "{}", "server_label": "server"}]}),+ json!({"input": [{"type": "mcp_approval_response", "approval_request_id": "approval_1", "approve": true}]}),+ json!({"input": [{"type": "mcp_approval_response", "approval_request_id": "approval_1", "approve": false}]}),+ ] {+ assert_requires_capability(OpenAiResponses, &body, capabilities, "tool_calling")?;+ }+ let body = json!({"messages": [{"role": "user", "content": "hello"}], "tools": [], "functions": []});+ let request = decode_request(OpenAiChat, &body)?;+ assert_eq!(unsupported_capability(capabilities, &request, &body), None);+ Ok(())+ }++ // Preserved reasoning controls still reach the provider when decoding omits them.+ #[test]+ fn rejects_preserved_reasoning_controls() -> TestResult {+ let capabilities = ModelCapabilities {+ reasoning: Some(false),+ ..Default::default()+ };+ for (format, body, pointer) in [+ (+ OpenAiChat,+ json!({"messages": [{"role": "user", "content": "hello"}], "reasoning": {"enabled": true}}),+ "/reasoning",+ ),+ (+ OpenAiChat,+ json!({"messages": [{"role": "user", "content": "hello"}], "reasoning_effort": 123}),+ "/reasoning_effort",+ ),+ (+ AnthropicMessages,+ json!({"messages": [{"role": "user", "content": "hello"}], "output_config": {"effort": 123}}),+ "/output_config/effort",+ ),+ ] {+ let request = assert_requires_capability(format, &body, capabilities, "reasoning")?;+ assert_eq!(+ encode_request(&request, format)?.pointer(pointer),+ body.pointer(pointer)+ );+ }+ Ok(())+ }++ // Decoded file-ID images and preserved computer screenshots both require vision.+ #[test]+ fn rejects_file_id_images_and_computer_screenshots() -> TestResult {+ let capabilities = ModelCapabilities {+ vision: Some(false),+ ..Default::default()+ };+ for body in [+ json!({"input": [{"role": "user", "content": [{"type": "input_image", "file_id": "file_1"}]}]}),+ json!({"input": [{"type": "computer_call_output", "call_id": "call_1", "output": {"type": "computer_screenshot", "image_url": "https://example.test/image.png"}}]}),+ ] {+ let request =+ assert_requires_capability(OpenAiResponses, &body, capabilities, "vision")?;+ assert_eq!(+ encode_request(&request, OpenAiResponses)?["input"],+ body["input"]+ );+ }+ Ok(())+ }++ // Allowing tool results must not hide unsupported images nested in their content.+ #[test]+ fn rejects_images_inside_tool_results_when_tools_are_allowed() -> TestResult {+ let body = json!({"messages": [{"role": "user", "content": [{+ "type": "tool_result", "tool_use_id": "call_1", "content": [{+ "type": "image", "source": {"type": "url", "url": "https://example.test/image.png"}+ }]+ }]}]});+ assert_requires_capability(+ AnthropicMessages,+ &body,+ ModelCapabilities {+ tool_calling: Some(true),+ vision: Some(false),+ ..Default::default()+ },+ "vision",+ )?;+ Ok(())+ }+}diff --git a/crates/switchyard-server/src/lib.rs b/crates/switchyard-server/src/lib.rsindex 799f2aa09..58ebc0d97 100644--- a/crates/switchyard-server/src/lib.rs+++ b/crates/switchyard-server/src/lib.rs@@ -3,6 +3,7 @@//! Rust HTTP server for libsy algorithms.+mod capabilities;pub mod config;mod metrics;mod observability;@@ -1009,6 +1010,18 @@ fn resolve_route("invalid_request_error",));}+ if let Some(capability) =+ capabilities::unsupported_capability(route.capabilities(), &llm_request, &body)+ {+ return Err(error_response(+ StatusCode::BAD_REQUEST,+ format!(+ "route {requested_model} declares {capability} = false; remove the unsupported input or select another route"+ ),+ "invalid_request_error",+ "unsupported_capability",+ ));+ }let request = Request {llm_request,raw_request: Some(body),diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rsindex f05094f7a..29bb561e7 100644--- a/crates/switchyard-server/tests/server.rs+++ b/crates/switchyard-server/tests/server.rs@@ -3452,6 +3452,165 @@ target = "shared"Ok(())}+// Build routes through the TOML loader to test disabled, enabled, and unset capabilities.+fn capability_app(base_url: &str, format: &str) -> TestResult<Router> {+ Ok(build_switchyard_router(load_test_config(&format!(+ r#"+schema_version = 1+[llm_clients.upstream]+format = "{format}"+base_url = "{base_url}"+[targets]+shared = {{ id = "model/efficient", llm_client = "upstream" }}+[routes]+restricted = {{ id = "restricted", type = "passthrough", target = "shared", vision = false, reasoning = false, tool_calling = false }}+enabled = {{ id = "enabled", type = "passthrough", target = "shared", vision = true, reasoning = true, tool_calling = true }}+undeclared = {{ id = "undeclared", type = "passthrough", target = "shared" }}+"#,+ ))?))+}++// Chat Completions, Responses, and Messages reject disabled inputs before dispatch.+// Allowed Responses input retains instructions and options after translation+// to Chat Completions.+#[tokio::test]+async fn disabled_route_capabilities_reject_requests_before_calling_upstream() -> TestResult {+ let upstream = MockUpstream::start().await?;+ let app = capability_app(&upstream.base_url, "openai_chat")?;+ let cases = [+ (+ "/v1/chat/completions",+ "vision",+ json!({"messages": [{"role": "user", "content": [{"type": "image_url", "image_url": {"url": "https://example.test/image.png"}}]}]}),+ ),+ (+ "/v1/messages",+ "vision",+ json!({"messages": [{"role": "user", "content": [{"type": "image", "source": {"type": "url", "url": "https://example.test/image.png"}}]}]}),+ ),+ (+ "/v1/responses",+ "vision",+ json!({"input": [{"role": "user", "content": [{"type": "input_image", "image_url": "https://example.test/image.png"}]}]}),+ ),+ (+ "/v1/chat/completions",+ "reasoning",+ json!({"messages": [{"role": "user", "content": "hello"}], "reasoning_effort": "high"}),+ ),+ (+ "/v1/messages",+ "reasoning",+ json!({"messages": [{"role": "user", "content": "hello"}], "thinking": {"type": "enabled", "budget_tokens": 1024}}),+ ),+ (+ "/v1/responses",+ "reasoning",+ json!({"input": "hello", "reasoning": {"summary": "auto"}}),+ ),+ (+ "/v1/chat/completions",+ "tool_calling",+ json!({"messages": [{"role": "user", "content": "hello"}], "tools": [{"type": "function", "function": {"name": "exec_command", "parameters": {"type": "object"}}}]}),+ ),+ (+ "/v1/messages",+ "tool_calling",+ json!({"messages": [{"role": "user", "content": "hello"}], "tools": [{"name": "exec_command", "input_schema": {"type": "object"}}]}),+ ),+ (+ "/v1/responses",+ "tool_calling",+ json!({"input": [{"type": "additional_tools", "tools": [{"type": "function", "name": "exec_command", "parameters": {"type": "object"}}]}, {"role": "user", "content": "hello"}]}),+ ),+ ];+ for (endpoint, capability, mut body) in cases {+ body["model"] = json!("restricted");+ let response = send(&app, "POST", endpoint, Some(body)).await?;+ assert_eq!(+ response.status,+ StatusCode::BAD_REQUEST,+ "{endpoint}: {capability}"+ );+ let error = response.json()?;+ assert_eq!(error["error"]["type"], "invalid_request_error");+ if endpoint == "/v1/messages" {+ assert_eq!(error["type"], "error");+ } else {+ assert_eq!(error["error"]["code"], "unsupported_capability");+ }+ assert!(+ error["error"]["message"]+ .as_str()+ .unwrap_or_default()+ .contains(&format!("{capability} = false"))+ );+ }+ assert!(upstream.calls.lock().await.is_empty());++ let body = json!({+ "model": "restricted", "instructions": "Keep the caller's instructions.", "input": "hello"+ });+ let response = send(&app, "POST", "/v1/responses", Some(body)).await?;+ assert_eq!(response.status, StatusCode::OK);+ for model in ["enabled", "undeclared"] {+ let response = send(&app, "POST", "/v1/responses", Some(json!({+ "model": model,+ "instructions": "Keep the caller's instructions.",+ "input": [{"role": "user", "content": [{"type": "input_image", "image_url": "https://example.test/image.png"}]}],+ "reasoning": {"effort": "high"},+ "tools": [{"type": "function", "name": "exec_command", "parameters": {"type": "object"}}]+ }))).await?;+ assert_eq!(response.status, StatusCode::OK);+ }+ let calls = upstream.calls.lock().await;+ assert_eq!(calls.len(), 3);+ assert!(+ calls+ .iter()+ .all(|call| has_system_prompt(call, "Keep the caller's instructions."))+ );+ for call in &calls[1..] {+ assert_eq!(call["reasoning_effort"], "high");+ assert_eq!(call["tools"][0]["function"]["name"], "exec_command");+ assert!(call["messages"].as_array().is_some_and(|messages| {+ messages+ .iter()+ .any(|message| message["content"][0]["type"] == "image_url")+ }));+ }+ Ok(())+}++// Approval replies can resume tool use without a tools field or a decoded tool call.+#[tokio::test]+async fn tool_approval_replies_follow_route_capabilities() -> TestResult {+ let upstream = MockUpstream::start().await?;+ let base_url = format!("{}/buffered", upstream.base_url.trim_end_matches("/v1"));+ let app = capability_app(&base_url, "openai_responses")?;+ let input = json!([+ {"type": "mcp_approval_response", "approval_request_id": "approval_1", "approve": true}+ ]);+ for model in ["restricted", "enabled", "undeclared"] {+ let body = json!({"model": model, "input": input});+ let response = send(&app, "POST", "/v1/responses", Some(body)).await?;+ if model == "restricted" {+ assert_eq!(response.status, StatusCode::BAD_REQUEST);+ let error = response.json()?;+ assert_eq!(error["error"]["code"], "unsupported_capability");+ assert!(upstream.calls.lock().await.is_empty());+ } else {+ assert_eq!(response.status, StatusCode::OK);+ }+ }+ let calls = upstream.calls.lock().await;+ assert_eq!(calls.len(), 2);+ for call in calls.iter() {+ assert_eq!(call["input"], input);+ }+ Ok(())+}+// Tool results and blank user messages must not replace the classifier's task text.#[tokio::test]async fn subagent_tool_continuations_are_classified_on_every_request_across_apis() -> TestResult {diff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.mdindex 9b1d97780..d168a64df 100644--- a/docs/reference/toml_schema.md+++ b/docs/reference/toml_schema.md@@ -132,9 +132,9 @@ Every route takes the common keys below, plus the keys for its type.| `id` | Yes | — | Public model ID that callers send in requests. || `type` | Yes | — | Routing algorithm for this route. || `context_window` | No | unset | Positive token count advertised for this route by `GET /v1/models`. Unset values appear as `null`. This does not enforce a request limit. |-| `tool_calling` | No | unset | Whether `GET /v1/models` advertises tool-calling support for this route. Unset values appear as `null`. |-| `reasoning` | No | unset | Declared reasoning support, stored in route metadata. The server does not include it in `GET /v1/models`. |-| `vision` | No | unset | Image-input support advertised in `GET /v1/models` under `data[].capabilities.vision`. Unset values appear as `null`. Declare `true` only when every target the route can select accepts images. |+| `tool_calling` | No | unset | Tool-calling support advertised by `GET /v1/models`. When `false`, the server rejects tool definitions, tool controls, and tool history with HTTP 400 before dispatch. Unset values appear as `null`. Explicit `true` and unset values do not restrict requests. |+| `reasoning` | No | unset | When `false`, the server rejects reasoning controls with HTTP 400 before dispatch. Explicit `true` and unset values do not restrict requests. The server does not include this declaration in `GET /v1/models`. |+| `vision` | No | unset | Image-input support advertised in `GET /v1/models` under `data[].capabilities.vision`. When `false`, the server rejects images with HTTP 400 before dispatch, including images in tool results. Unset values appear as `null`. Explicit `true` and unset values do not restrict requests. Declare `true` only when every target the route can select accepts images. |### `noop`