Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Rust
fix: Don't forward HTTP headers in forward_auth=false (#690)
Graham Kingce1482e90843
diff --git a/crates/libsy-llm-client/README.md b/crates/libsy-llm-client/README.mdindex b62da05e1..bad3f86d0 100644--- a/crates/libsy-llm-client/README.md+++ b/crates/libsy-llm-client/README.md@@ -228,15 +228,16 @@ fn build_multi_format_client(- The `Backend` variant sets auth: OpenAI formats send `Authorization: Bearer <key>`;Anthropic sends `x-api-key: <key>` plus `anthropic-version`.-- `request.metadata.http_headers` are forwarded upstream, **except** reserved ones:- `host`, `content-length`, `connection`, and the backend-owned- login, API-key, version, and content headers. So a caller's placeholder- credential never overrides the backend's real key.+- By default, only `x-request-id` from `request.metadata.http_headers` is forwarded.+ With `forward_auth`, application headers are forwarded, while connection, destination,+ body, and the selected provider's owned headers are removed or rebuilt.- `HttpBackendConfig::forward_auth` uses the caller's credential instead of the- backend's configured key. OpenAI backends forward `authorization`,- `chatgpt-account-id`, and `x-openai-fedramp`. Anthropic backends forward- `authorization` or `x-api-key`; they also keep `oauth-*` values from- `anthropic-beta` and remove other caller-supplied beta values.+ backend's configured key and forwards other application headers. OpenAI backends+ normalize `authorization`, `chatgpt-account-id`, and `x-openai-fedramp`.+ Anthropic backends forward `authorization` or `x-api-key`; they also keep+ `oauth-*` values from `anthropic-beta` and remove other caller-supplied beta values.+ All backends reachable through a forwarding route must use the same provider.+ Headers owned by other providers are preserved as application headers.- Per-backend custom headers go in `HttpBackendConfig::extra_headers`. Set credentials with`api_key`. OpenAI backends reject `Authorization`; Anthropic backends reject `x-api-key`and `anthropic-version`. Header names are case-insensitive.diff --git a/crates/libsy-llm-client/src/backend.rs b/crates/libsy-llm-client/src/backend.rsindex 71f5d281d..da43e2c63 100644--- a/crates/libsy-llm-client/src/backend.rs+++ b/crates/libsy-llm-client/src/backend.rs@@ -45,7 +45,9 @@ pub struct HttpBackendConfig {pub base_url: String,/// API key for the provider, loaded by the caller. `None` sends no configured auth.pub api_key: Option<String>,- /// Whether this backend forwards the caller's provider credential instead.+ /// Whether this backend forwards the caller's provider credential and application headers.+ ///+ /// All backends reachable through a forwarding route must use the same provider.pub forward_auth: bool,/// Custom headers added to every outbound call to this backend.///@@ -180,6 +182,24 @@ impl Backend {self.config().forward_auth}+ pub(crate) fn is_provider_owned_header(&self, name: &str) -> bool {+ match self {+ Backend::OpenAiChat(_) | Backend::OpenAiResponses(_) => {+ ["authorization", "chatgpt-account-id", "x-openai-fedramp"]+ .iter()+ .any(|owned| name.eq_ignore_ascii_case(owned))+ }+ Backend::Anthropic(_) => [+ "authorization",+ "x-api-key",+ "anthropic-beta",+ "anthropic-version",+ ]+ .iter()+ .any(|owned| name.eq_ignore_ascii_case(owned)),+ }+ }+/// Applies only the caller credential accepted by this provider.pub(crate) fn apply_forwarded_auth(&self,@@ -216,35 +236,6 @@ impl Backend {builder}- /// Removes an echoed caller credential before an upstream error is returned or logged.- pub(crate) fn redact_forwarded_auth(- &self,- mut body: String,- metadata: Option<&Metadata>,- ) -> String {- if !self.is_forwarding_auth() {- return body;- }- let Some(headers) = metadata.and_then(|metadata| metadata.http_headers.as_ref()) else {- return body;- };- let secret_headers: &[&str] = match self {- Backend::OpenAiChat(_) | Backend::OpenAiResponses(_) => {- &["authorization", "chatgpt-account-id"]- }- Backend::Anthropic(_) => &["authorization", "x-api-key"],- };- for name in secret_headers {- let Some(value) = headers.get(*name).and_then(|value| value.to_str().ok()) else {- continue;- };- if !value.is_empty() {- body = body.replace(value, "[REDACTED]");- }- }- body- }-/// Custom per-backend headers to forward on every call.pub fn extra_headers(&self) -> &BTreeMap<String, String> {&self.config().extra_headersdiff --git a/crates/libsy-llm-client/src/client.rs b/crates/libsy-llm-client/src/client.rsindex 2dacc6383..272e37ada 100644--- a/crates/libsy-llm-client/src/client.rs+++ b/crates/libsy-llm-client/src/client.rs@@ -29,28 +29,27 @@ use crate::error::{LlmClientError, Result};use crate::metrics;use crate::raw::RawResponse;-// Headers this client owns or that are hop-by-hop. Backends apply an explicitly-// enabled caller credential after generic metadata forwarding skips these.-// Azure/OpenAI credentials and tenant selectors are also backend-owned.-const RESERVED_HEADERS: &[&str] = &[+// Caller headers safe to send when caller auth forwarding is disabled.+const ALLOWED_METADATA_HEADERS: &[&str] = &["x-request-id"];++// Headers tied to the inbound connection, destination, or body. The HTTP client+// must rebuild these for the upstream request, even when forwarding auth.+const NON_FORWARDABLE_HEADERS: &[&str] = &["host","content-length","connection",- "authorization",- "proxy-authorization",+ "proxy-connection",+ "keep-alive","proxy-authenticate",- "cookie",- "set-cookie",- "x-api-key",- "chatgpt-account-id",- "x-openai-fedramp",- "api-key",- "openai-organization",- "openai-project",- "anthropic-beta",- "anthropic-version",+ "proxy-authorization",+ "te",+ "trailer",+ "transfer-encoding",+ "upgrade","content-type",+ "content-encoding","accept-encoding",+ "expect",];const INITIAL_RETRY_DELAY: Duration = Duration::from_millis(250);@@ -341,7 +340,7 @@ impl TranslatingLlmClient {&self.client};let builder = client.post(url).json(body);- let builder = forward_metadata_headers(builder, metadata);+ let builder = forward_metadata_headers(builder, metadata, backend);let builder = backend.apply_forwarded_auth(builder, metadata);let builder = apply_extra_headers(builder, backend);let builder = backend.apply_auth(builder);@@ -394,7 +393,7 @@ impl TranslatingLlmClient {});}};- let body = backend.redact_forwarded_auth(body, metadata);+ let body = redact_forwarded_headers(body, metadata, backend.is_forwarding_auth());metrics::record_upstream_attempt(Some(status.as_u16()));let error =if status == reqwest::StatusCode::BAD_REQUEST && backend.is_context_overflow(&body) {@@ -520,8 +519,10 @@ impl TranslatingLlmClient {/// model that answered rather than the route the caller addressed.////// `http_headers` are carried through as the request's- /// [`Metadata::http_headers`] and forwarded to the upstream (minus the reserved- /// set); pass `None` to forward nothing.+ /// [`Metadata::http_headers`]. Backends with `forward_auth` disabled forward only+ /// allowed metadata headers; `forward_auth` backends forward all application+ /// headers. All backends reachable through a forwarding route must use the same+ /// provider. Transport headers are always rebuilt. Pass `None` to forward nothing.pub async fn call_rewrite_model_raw(&self,raw_http_request: Value,@@ -721,21 +722,29 @@ fn convert_reqwest_error(error: reqwest::Error) -> LlmClientError {}}-// Forwards caller-supplied metadata headers except credentials and client-owned headers.+// Forwards safe metadata headers, or all application headers when forwarding auth.fn forward_metadata_headers(- mut builder: RequestBuilder,+ builder: RequestBuilder,metadata: Option<&Metadata>,+ backend: &Backend,) -> RequestBuilder {let Some(headers) = metadata.and_then(|metadata| metadata.http_headers.as_ref()) else {return builder;};+ let mut forwarded = HeaderMap::with_capacity(headers.len());for (name, value) in headers {- if is_reserved_header(name.as_str()) {+ let is_allowed = if backend.is_forwarding_auth() {+ !is_non_forwardable_header(name.as_str(), headers)+ && !backend.is_provider_owned_header(name.as_str())+ } else {+ is_allowed_metadata_header(name.as_str())+ };+ if !is_allowed {continue;}- builder = builder.header(name, value);+ forwarded.append(name, value.clone());}- builder+ builder.headers(forwarded)}// Adds the backend's custom per-call headers.@@ -1061,11 +1070,48 @@ fn ensure_openai_stream_usage(body: &mut Value) {}}-// Case-insensitive membership test against RESERVED_HEADERS.-fn is_reserved_header(name: &str) -> bool {- RESERVED_HEADERS+fn is_allowed_metadata_header(name: &str) -> bool {+ ALLOWED_METADATA_HEADERS.iter()- .any(|reserved| name.eq_ignore_ascii_case(reserved))+ .any(|allowed| name.eq_ignore_ascii_case(allowed))+}++fn is_non_forwardable_header(name: &str, headers: &HeaderMap) -> bool {+ NON_FORWARDABLE_HEADERS+ .iter()+ .any(|blocked| name.eq_ignore_ascii_case(blocked))+ || headers.get_all("connection").iter().any(|value| {+ value+ .as_bytes()+ .split(|byte| *byte == b',')+ .any(|option| option.trim_ascii().eq_ignore_ascii_case(name.as_bytes()))+ })+}++// Unknown application headers can carry credentials when auth forwarding is enabled.+fn redact_forwarded_headers(+ mut body: String,+ metadata: Option<&Metadata>,+ is_forwarding_auth: bool,+) -> String {+ if !is_forwarding_auth {+ return body;+ }+ let Some(headers) = metadata.and_then(|metadata| metadata.http_headers.as_ref()) else {+ return body;+ };+ for (name, value) in headers {+ if is_non_forwardable_header(name.as_str(), headers) {+ continue;+ }+ let Ok(value) = value.to_str() else {+ continue;+ };+ if !value.is_empty() {+ body = body.replace(value, "[REDACTED]");+ }+ }+ body}#[cfg(test)]@@ -1078,7 +1124,7 @@ mod tests {use std::thread::JoinHandle;use serde_json::json;- use switchyard_protocol::{LlmRequest, completion_text, text_request};+ use switchyard_protocol::{completion_text, text_request};use wiremock::matchers::{method, path};use wiremock::{Mock, MockServer, ResponseTemplate};@@ -1104,6 +1150,14 @@ mod tests {}}+ fn forwarding_config(base_url: &str) -> HttpBackendConfig {+ HttpBackendConfig {+ api_key: None,+ forward_auth: true,+ ..config(base_url)+ }+ }+// A one-model config list: "gpt" served over OpenAI Chat at base_url.fn chat_map(base_url: &str) -> Vec<ModelConfig> {vec![ModelConfig::new(@@ -1227,6 +1281,15 @@ mod tests {}}+ fn request_with_headers(model: &str, headers: HeaderMap) -> Request {+ let mut request = request_for(Some(model), false);+ request.metadata = Some(Metadata {+ http_headers: Some(headers),+ ..Default::default()+ });+ request+ }+#[tokio::test]async fn transport_errors_drop_the_upstream_url() {let error = reqwest::Client::new()@@ -2223,7 +2286,7 @@ mod tests {}#[tokio::test]- async fn forwards_metadata_headers_except_reserved()+ async fn forwards_only_allowlisted_metadata_headers()-> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {let server = MockServer::start().await;Mock::given(method("POST"))@@ -2245,53 +2308,154 @@ mod tests {http::HeaderValue::from_static("Bearer client-key"),);headers.insert(- "accept-encoding",- http::HeaderValue::from_static("gzip, br"),+ "x-goog-api-key",+ http::HeaderValue::from_static("client-google-key"),);headers.insert(- "api-key",- http::HeaderValue::from_static("client-azure-key"),+ "x-custom-internal-secret",+ http::HeaderValue::from_static("client-custom-key"),);+ let request = request_with_headers("gpt", headers);++ let client = TranslatingLlmClient::new(&chat_map(&format!("{}/v1", server.uri())))?;++ // The allowed request id survives and caller authorization cannot replace+ // the backend's credential.+ client.call_rewrite_model(request, None).await?;+ let received = server+ .received_requests()+ .await+ .ok_or("request recording should be enabled")?;+ let received = received.first().ok_or("expected one upstream request")?;+ assert!(!received.headers.contains_key("x-goog-api-key"));+ assert!(!received.headers.contains_key("x-custom-internal-secret"));+ Ok(())+ }++ #[tokio::test]+ async fn forwards_application_headers_with_forward_auth()+ -> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(wiremock::matchers::header(+ "authorization",+ "Bearer client-key",+ ))+ .and(wiremock::matchers::header(+ "x-goog-api-key",+ "client-google-key",+ ))+ .respond_with(ResponseTemplate::new(200).set_body_json(json!({+ "id": "1", "model": "gpt",+ "choices": [{"index": 0, "message": {"role": "assistant", "content": "ok"}, "finish_reason": "stop"}],+ "usage": {}+ })))+ .mount(&server)+ .await;++ let mut headers = http::HeaderMap::new();headers.insert(- "openai-organization",- http::HeaderValue::from_static("org-client"),+ "authorization",+ http::HeaderValue::from_static("Bearer client-key"),);headers.insert(- "openai-project",- http::HeaderValue::from_static("proj-client"),+ "x-goog-api-key",+ http::HeaderValue::from_static("client-google-key"),);- let request = Request {- llm_request: LlmRequest {- model: Some("gpt".to_string()),- ..LlmRequest::default()- },- raw_request: None,- metadata: Some(Metadata {- session_id: None,- agent_id: None,- task_id: None,- correlation_id: None,- extra_metadata: None,- http_headers: Some(headers),- wire_format: None,- ..Default::default()- }),- };-- let client = TranslatingLlmClient::new(&chat_map(&format!("{}/v1", server.uri())))?;+ headers.insert("host", http::HeaderValue::from_static("client.example"));+ headers.insert("content-type", http::HeaderValue::from_static("text/plain"));+ headers.insert("connection", http::HeaderValue::from_static("x-hop-by-hop"));+ headers.insert(+ "x-hop-by-hop",+ http::HeaderValue::from_static("client-only"),+ );+ let request = request_with_headers("gpt", headers);+ let backend_config = forwarding_config(&format!("{}/v1", server.uri()));+ let client = TranslatingLlmClient::new(&[ModelConfig::new(+ "gpt",+ Backend::OpenAiChat(backend_config),+ None,+ )])?;- // Matchers assert forwarded x-request-id survives and reserved- // authorization is the backend's, not the client's.client.call_rewrite_model(request, None).await?;let received = server.received_requests().await.ok_or("request recording should be enabled")?;let received = received.first().ok_or("expected one upstream request")?;- assert!(!received.headers.contains_key("accept-encoding"));- assert!(!received.headers.contains_key("api-key"));- assert!(!received.headers.contains_key("openai-organization"));- assert!(!received.headers.contains_key("openai-project"));+ assert_ne!(received.headers["host"], "client.example");+ assert_eq!(received.headers["content-type"], "application/json");+ assert!(!received.headers.contains_key("connection"));+ assert!(!received.headers.contains_key("x-hop-by-hop"));+ Ok(())+ }++ #[tokio::test]+ async fn forward_auth_preserves_anthropic_headers_and_redacts_errors()+ -> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {+ let server = MockServer::start().await;+ Mock::given(method("POST"))+ .and(path("/v1/messages"))+ .and(|request: &wiremock::Request| {+ request+ .headers+ .get("anthropic-beta")+ .and_then(|value| value.to_str().ok())+ == Some("oauth-2025-04-20")+ && request.headers.get_all("anthropic-version").iter().count() == 1+ && request+ .headers+ .get("anthropic-version")+ .and_then(|value| value.to_str().ok())+ == Some("2023-06-01")+ })+ .respond_with(ResponseTemplate::new(401).set_body_json(json!({+ "error": {"message": "rejected client-google-key"}+ })))+ .mount(&server)+ .await;++ let mut headers = http::HeaderMap::new();+ headers.insert(+ "authorization",+ http::HeaderValue::from_static("Bearer client-key"),+ );+ headers.insert(+ "x-goog-api-key",+ http::HeaderValue::from_static("client-google-key"),+ );+ headers.insert(+ "anthropic-beta",+ http::HeaderValue::from_static("oauth-2025-04-20,prompt-caching-2024-07-31"),+ );+ headers.insert(+ "anthropic-version",+ http::HeaderValue::from_static("caller-version"),+ );+ let backend_config = forwarding_config(&server.uri());+ let client = TranslatingLlmClient::new(&[ModelConfig::new(+ "claude",+ Backend::Anthropic(backend_config),+ None,+ )])?;+ let raw = json!({+ "model": "claude",+ "max_tokens": 8,+ "messages": [{"role": "user", "content": "hello"}]+ });++ let Err(LlmClientError::UpstreamHttp { body, .. }) = client+ .call_rewrite_model_raw(+ raw,+ Some(headers),+ Some(&ModelId::from("claude")),+ WireFormat::AnthropicMessages,+ )+ .await+ else {+ panic!("expected an upstream HTTP error");+ };+ assert_eq!(body, r#"{"error":{"message":"rejected [REDACTED]"}}"#);Ok(())}@@ -2605,7 +2769,7 @@ mod tests {Ok(())}- // Raw path forwards caller headers (minus the reserved set) to the upstream.+ // Raw path forwards allowed caller headers to the upstream.#[tokio::test]async fn call_rewrite_model_raw_forwards_headers()-> std::result::Result<(), Box<dyn Error + Sync + Send + 'static>> {@@ -2631,8 +2795,8 @@ mod tests {let client = TranslatingLlmClient::new(&chat_map(&format!("{}/v1", server.uri())))?;let raw = json!({"model": "gpt", "messages": [{"role": "user", "content": "hi"}]});- // Matchers assert the forwarded x-request-id survives and reserved- // authorization is the backend's, not the client's.+ // The allowed request id survives and caller authorization cannot replace+ // the backend's credential.client.call_rewrite_model_raw(raw,diff --git a/crates/switchyard-server/README.md b/crates/switchyard-server/README.mdindex 9c53520c2..885852a6c 100644--- a/crates/switchyard-server/README.md+++ b/crates/switchyard-server/README.md@@ -86,8 +86,10 @@ A client can set `forward_auth = true` instead of `api_key_env` to send thecaller's credential to the configured upstream. OpenAI clients forward`authorization`, `chatgpt-account-id`, and `x-openai-fedramp`. Anthropic clientsforward `authorization` or `x-api-key`. Enable this only when every forwarding-client's `base_url` should receive the caller's login. A forwarding route must-be called through the matching provider API.+client's `base_url` should receive the caller's login. All backends reachable+through the route must use the same provider. Other application headers are+preserved and may contain provider-specific credentials. A forwarding route+must be called through the matching provider API.Target-level `extra_body` values are shallow-merged into the upstream request whenthe request does not already contain that key.Target-level `system_prompt` values are prepended when that target serves a completion.diff --git a/crates/switchyard-server/tests/server.rs b/crates/switchyard-server/tests/server.rsindex 568bcd453..03a605a23 100644--- a/crates/switchyard-server/tests/server.rs+++ b/crates/switchyard-server/tests/server.rs@@ -368,8 +368,14 @@ async fn upstream_messages_requires_forwarded_oauth(.get("anthropic-version").and_then(|value| value.to_str().ok())== Some("2023-06-01")- && !headers.contains_key("chatgpt-account-id")- && !headers.contains_key("x-openai-fedramp");+ && headers+ .get("chatgpt-account-id")+ .and_then(|value| value.to_str().ok())+ == Some("account-123")+ && headers+ .get("x-openai-fedramp")+ .and_then(|value| value.to_str().ok())+ == Some("true");if !has_expected_headers {return (StatusCode::UNAUTHORIZED,@@ -422,8 +428,14 @@ async fn upstream_responses_requires_forwarded_auth(.get("x-openai-fedramp").and_then(|value| value.to_str().ok())== Some("true")- && !headers.contains_key("x-api-key")- && !headers.contains_key("anthropic-beta");+ && headers+ .get("x-api-key")+ .and_then(|value| value.to_str().ok())+ == Some("provider-api-key")+ && headers+ .get("anthropic-beta")+ .and_then(|value| value.to_str().ok())+ == Some("provider-beta");if !has_expected_headers {return (StatusCode::UNAUTHORIZED,@@ -2316,8 +2328,8 @@ target = "claude"&[("authorization", "Bearer claude-oauth-token"),("anthropic-beta", "oauth-2025-04-20,unsupported-beta"),- ("chatgpt-account-id", "must-not-cross-providers"),- ("x-openai-fedramp", "must-not-cross-providers"),+ ("chatgpt-account-id", "account-123"),+ ("x-openai-fedramp", "true"),],).await?;@@ -2372,8 +2384,8 @@ target = "openai"("authorization", "Bearer codex-login-token"),("chatgpt-account-id", "account-123"),("x-openai-fedramp", "true"),- ("x-api-key", "must-not-cross-providers"),- ("anthropic-beta", "oauth-must-not-cross-providers"),+ ("x-api-key", "provider-api-key"),+ ("anthropic-beta", "provider-beta"),],).await?;diff --git a/docs/getting_started.md b/docs/getting_started.mdindex 5f61e8034..3b5842577 100644--- a/docs/getting_started.md+++ b/docs/getting_started.md@@ -96,8 +96,10 @@ A client can set `forward_auth = true` instead of `api_key_env` to send eachcaller's credential to that upstream. OpenAI clients forward `authorization`,`chatgpt-account-id`, and `x-openai-fedramp`. Anthropic clients forward`authorization` or `x-api-key`. Enable this only for an upstream that should-receive the caller's login. The server rejects a forwarding route called-through the other provider's API.+receive the caller's login. All backends reachable through the route, including+efficient and capable targets, must use the same provider. Other application+headers are preserved, so they may contain provider-specific credentials. The+server rejects a forwarding route called through the other provider's API.### Run the serverdiff --git a/docs/reference/toml_schema.md b/docs/reference/toml_schema.mdindex 7d4b8b914..b482ecb6e 100644--- a/docs/reference/toml_schema.md+++ b/docs/reference/toml_schema.md@@ -51,7 +51,7 @@ route reaches no upstream. A file without a `[targets]` table is rejected with| `format` | Yes | — | `openai_chat`, `openai_responses`, or `anthropic_messages`. || `base_url` | Yes | — | Upstream base URL. || `api_key_env` | No | unset | Name of the environment variable holding the key. Omit to send no authentication. |-| `forward_auth` | No | `false` | Forward the caller's provider credential to this upstream. |+| `forward_auth` | No | `false` | Forward the caller's provider credential and application headers. All backends reachable through the route must use the same provider. || `extra_headers` | No | `{}` | Custom HTTP headers sent to the model server. Set credentials with `api_key_env` or `forward_auth`; the server rejects headers owned by the selected auth mode. Header names are case-insensitive. || `max_retries` | No | `2` | Retry budget, `0`–`10`. |@@ -76,11 +76,13 @@ values.This setting gives `base_url` the caller's login. Enable it only when thatupstream should receive the credential, and use HTTPS unless the upstream runs-on loopback. Forwarding clients do not follow HTTP redirects. Check every-forwarding client used by a route, including classifier and judge targets. The-server rejects an Anthropic forwarding route called through an OpenAI endpoint,-or an OpenAI forwarding route called through an Anthropic endpoint, before it-calls an upstream.+on loopback. All backends reachable through the route must use the same+provider because other application headers are preserved and may contain+provider-specific credentials. Forwarding clients do not follow HTTP redirects.+Check every forwarding client used by a route, including classifier and judge+targets. The server rejects an Anthropic forwarding route called through an+OpenAI endpoint, or an OpenAI forwarding route called through an Anthropic+endpoint, before it calls an upstream.## `[targets.<name>]`