🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
feat(ai-gateway): expose resolution context in response headers, spans and logs (#18247)
Hassieb Pakzadd4574d17206c
diff --git a/ai-gateway/README.md b/ai-gateway/README.mdindex cb7f04150b..09029b52fc 100644--- a/ai-gateway/README.md+++ b/ai-gateway/README.md@@ -102,7 +102,9 @@ host-only development, set `LANGFUSE_AI_GATEWAY_LISTEN_ADDRESS=127.0.0.1:8080`.Logs and operational OpenTelemetry exports describe gateway health independentlyof the inference generations sent to Langfuse. They do not include request/response-content, credentials, key metadata, or URL query strings. Health probes are excluded+content, credentials, key metadata, or URL query strings. They do carry the+[resolution context](#resolution-context), including the provider connection name+and the requested model. Health probes are excludedfrom request logs and spans.Use `LANGFUSE_LOG_FORMAT=text` locally and `json` for log collectors. Built-in `tracing-subscriber` formatters handle both modes. JSON event fields@@ -167,6 +169,33 @@ as authentication failures, and model or token-count routes omit them. Upload isbest-effort, so a dropped generation can leave these IDs unresolved. The gatewaydoes not set CORS headers, so no `Access-Control-Expose-Headers` is involved.+### Resolution context++Once Web resolution succeeds, the request also carries its tenant, and once it is+sent upstream, the provider connection it was sent to. Requests rejected+before resolution carry none of them. A resolved request that fails before reaching+the provider, such as a 413 body or 503 execution capacity rejection, carries only+the tenant. Provider failures before response headers (502/504) still name the+connection that was tried.++| Response header | Server span field | JSON log key | Value |+| --- | --- | --- | --- |+| `langfuse-organization-id` | `langfuse.organization.id` | `organization_id` | Organization of the gateway key |+| `langfuse-project-id` | `langfuse.project.id` | `project_id` | Ingestion project |+| `langfuse-provider` | `gateway.provider` | `provider` | Stable lowercase provider type, `openai` or `anthropic` |+| `langfuse-provider-connection-id` | `gateway.provider.connection.id` | `provider_connection_id` | Opaque provider connection ID |+| — | `gateway.provider.connection.name` | `provider_connection_name` | The connection's user-chosen name; never sent to callers |+| — | `gateway.upstream.model` | `upstream_model` | The request's `model`, which the relay forwards unchanged; never sent to callers |++The provider fields describe the latest upstream attempt, so with fallbacks they+name the connection that served the response, or the last one tried. The model+comes from the request capture's existing parse, so it is recorded only on the+generation routes listed above, when the body is uncompressed JSON of at most 5 MiB.+The span fields are declared empty on the `http.server` span and recorded after+resolution; child spans do not repeat them. JSON log lines emitted inside the+request after they are recorded repeat them as top-level keys, like `request_id`.+They are never metric attributes.+Each inference request phase has a child span of the server span so a waterfallhas no unattributed wall time:@@ -822,6 +851,9 @@ invalid response, even if Web selected it.| `openai` | `openai.responses`, `openai.chat-completions` | `https://api.openai.com/v1` | `{"type":"Bearer","token":…}` → `Authorization: Bearer` || `anthropic` | `anthropic.messages` | `https://api.anthropic.com/v1` | `{"type":"x-api-key","header":"x-api-key","value":…}` → `x-api-key` |+The connection's `name` is optional in the contract, so the gateway also resolves+against a Web deployment that predates it. Web always sends it.+`ProviderConnection::credential()` exposes the secret in its header position(`ProviderCredential::Bearer` or `ProviderCredential::XApiKey`) and has no `Debug`output. Ingestion tokens remain opaque. Resolution caching is a separate slice.diff --git a/ai-gateway/src/capture/anthropic_messages.rs b/ai-gateway/src/capture/anthropic_messages.rsindex 7a7f44f450..3d158a1210 100644--- a/ai-gateway/src/capture/anthropic_messages.rs+++ b/ai-gateway/src/capture/anthropic_messages.rs@@ -280,6 +280,10 @@ impl AnthropicMessagesCapture {self.facts.capture_complete = self.request_complete && self.facts.output_complete;}+ pub fn requested_model(&self) -> Option<&str> {+ self.facts.requested_model.as_deref()+ }+pub fn into_facts(mut self) -> ProviderFacts {if matches!(self.body, ResponseBody::Sse(_)) {self.end_body();diff --git a/ai-gateway/src/capture/mod.rs b/ai-gateway/src/capture/mod.rsindex dfedf15781..70337a9813 100644--- a/ai-gateway/src/capture/mod.rs+++ b/ai-gateway/src/capture/mod.rs@@ -82,6 +82,14 @@ impl ProtocolCapture {}}+ fn requested_model(&self) -> Option<&str> {+ match self {+ Self::OpenAiResponses(capture) => capture.requested_model(),+ Self::OpenAiChatCompletions(capture) => capture.requested_model(),+ Self::AnthropicMessages(capture) => capture.requested_model(),+ }+ }+fn client_metadata(&self) -> Option<&Map<String, Value>> {match self {Self::OpenAiResponses(capture) => capture.client_metadata(),@@ -197,6 +205,13 @@ impl ExecutionCapture {self.delivery = Some((telemetry, delivery));}+ /// The request's `model`, if the capture could parse it.+ pub fn requested_model(&self) -> Option<&str> {+ self.protocol+ .as_ref()+ .and_then(ProtocolCapture::requested_model)+ }+pub fn record_response(&mut self, status: u16, headers: &HeaderMap) {if let Some(protocol) = &mut self.protocol {self.http_status = Some(status);diff --git a/ai-gateway/src/capture/openai_chat_completions.rs b/ai-gateway/src/capture/openai_chat_completions.rsindex bb29946883..a7ce3fdef1 100644--- a/ai-gateway/src/capture/openai_chat_completions.rs+++ b/ai-gateway/src/capture/openai_chat_completions.rs@@ -341,6 +341,10 @@ impl OpenAiChatCompletionsCapture {self.facts.capture_complete = self.request_complete && self.facts.output_complete;}+ pub fn requested_model(&self) -> Option<&str> {+ self.facts.requested_model.as_deref()+ }+pub fn into_facts(mut self) -> ProviderFacts {if matches!(self.body, ResponseBody::Sse(_)) {self.end_body();diff --git a/ai-gateway/src/capture/openai_responses.rs b/ai-gateway/src/capture/openai_responses.rsindex 5270cf489e..34cb05ba62 100644--- a/ai-gateway/src/capture/openai_responses.rs+++ b/ai-gateway/src/capture/openai_responses.rs@@ -100,6 +100,10 @@ impl OpenAiResponsesCapture {self.facts.capture_complete = self.request_complete && self.facts.output_complete;}+ pub fn requested_model(&self) -> Option<&str> {+ self.facts.requested_model.as_deref()+ }+pub fn into_facts(mut self) -> ProviderFacts {if matches!(self.body, ResponseBody::Sse(_)) {self.end_body();diff --git a/ai-gateway/src/correlation.rs b/ai-gateway/src/correlation.rsindex 62542dbbd0..6416e7583f 100644--- a/ai-gateway/src/correlation.rs+++ b/ai-gateway/src/correlation.rs@@ -1,5 +1,7 @@//! One gateway request ID per inference request, shared by the caller's response,//! logs, operational spans, the Web resolution call and the Langfuse generation.+//! Once resolution succeeds, the request also carries its tenant and the provider+//! connection it was sent to.use axum::{extract::Request,http::{HeaderMap, HeaderName, HeaderValue},@@ -7,9 +9,16 @@ use axum::{response::Response,};+use crate::resolution::{ProviderConnection, ResolvedRequestContext};+pub(crate) const REQUEST_ID_HEADER: HeaderName = HeaderName::from_static("langfuse-request-id");const TRACE_ID_HEADER: HeaderName = HeaderName::from_static("langfuse-trace-id");const OBSERVATION_ID_HEADER: HeaderName = HeaderName::from_static("langfuse-observation-id");+const ORGANIZATION_ID_HEADER: HeaderName = HeaderName::from_static("langfuse-organization-id");+const PROJECT_ID_HEADER: HeaderName = HeaderName::from_static("langfuse-project-id");+const PROVIDER_HEADER: HeaderName = HeaderName::from_static("langfuse-provider");+const PROVIDER_CONNECTION_ID_HEADER: HeaderName =+ HeaderName::from_static("langfuse-provider-connection-id");const CLIENT_REQUEST_ID_HEADER: &str = "x-request-id";const MAX_CLIENT_REQUEST_ID_BYTES: usize = 256;@@ -21,11 +30,43 @@ pub(crate) struct GenerationIds {pub observation_id: String,}+#[derive(Clone, Debug)]+struct Tenant {+ organization_id: String,+ project_id: String,+}++/// The provider connection and model of an upstream attempt. With several+/// attempts the latest one wins, so it names the connection that served the+/// response, or the last one tried.+#[derive(Clone, Debug)]+pub(crate) struct UpstreamTarget {+ provider: &'static str,+ connection_id: String,+ connection_name: Option<String>,+ model: Option<String>,+}++impl UpstreamTarget {+ pub fn for_connection(connection: &ProviderConnection, model: Option<String>) -> Self {+ Self {+ provider: connection.provider().as_str(),+ connection_id: connection.id().to_owned(),+ connection_name: connection.name().map(str::to_owned),+ model,+ }+ }+}+#[derive(Clone, Debug)]pub(crate) struct RequestCorrelation {id: String,client_id: Option<String>,generation: Option<GenerationIds>,+ tenant: Option<Tenant>,+ upstream: Option<UpstreamTarget>,+ /// The request's `http.server` span, which outlives the phase spans.+ server_span: tracing::Span,}impl Default for RequestCorrelation {@@ -34,6 +75,9 @@ impl Default for RequestCorrelation {id: uuid::Uuid::now_v7().hyphenated().to_string(),client_id: None,generation: None,+ tenant: None,+ upstream: None,+ server_span: tracing::Span::none(),}}}@@ -75,6 +119,55 @@ impl RequestCorrelation {self.generation = Some(ids);}+ pub fn record_resolution(&mut self, context: &ResolvedRequestContext) {+ let attribution = context.attribution();+ self.server_span+ .record("langfuse.organization.id", attribution.organization_id());+ self.server_span+ .record("langfuse.project.id", attribution.project_id());+ self.tenant = Some(Tenant {+ organization_id: attribution.organization_id().to_owned(),+ project_id: attribution.project_id().to_owned(),+ });+ }++ pub fn record_upstream(&mut self, target: UpstreamTarget) {+ let span = &self.server_span;+ span.record("gateway.provider", target.provider);+ span.record(+ "gateway.provider.connection.id",+ target.connection_id.as_str(),+ );+ if let Some(name) = &target.connection_name {+ span.record("gateway.provider.connection.name", name.as_str());+ }+ if let Some(model) = &target.model {+ span.record("gateway.upstream.model", model.as_str());+ }+ self.upstream = Some(target);+ }++ /// Only a resolved request advertises its tenant, and only a request sent+ /// upstream its provider. The connection name and model stay out of responses.+ pub fn apply_resolution_headers(&self, headers: &mut HeaderMap) {+ let Some(tenant) = &self.tenant else {+ return;+ };+ let mut values = vec![+ (ORGANIZATION_ID_HEADER, tenant.organization_id.as_str()),+ (PROJECT_ID_HEADER, tenant.project_id.as_str()),+ ];+ if let Some(upstream) = &self.upstream {+ values.push((PROVIDER_HEADER, upstream.provider));+ values.push((PROVIDER_CONNECTION_ID_HEADER, &upstream.connection_id));+ }+ for (name, value) in values {+ if let Ok(value) = HeaderValue::from_str(value) {+ headers.insert(name, value);+ }+ }+ }+/// Only a request that will produce a Langfuse generation advertises its IDs.pub fn apply_generation_headers(&self, headers: &mut HeaderMap) {let Some(generation) = &self.generation else {@@ -94,12 +187,13 @@ impl RequestCorrelation {/// Runs inside the server span before any rejection, so every response, log line/// and phase span of the request can carry the ID.pub(crate) async fn assign_request_id(mut request: Request, next: Next) -> Response {- let correlation = RequestCorrelation::from_headers(request.headers());+ let mut correlation = RequestCorrelation::from_headers(request.headers());let span = tracing::Span::current();span.record("gateway.request.id", correlation.id());if let Some(client_id) = correlation.client_id() {span.record("gateway.client.request.id", client_id);}+ correlation.server_span = span;let value = HeaderValue::from_str(correlation.id()).expect("a UUID is a valid header value");request.extensions_mut().insert(correlation);let mut response = next.run(request).await;diff --git a/ai-gateway/src/http.rs b/ai-gateway/src/http.rsindex f1b803a58b..261b929a60 100644--- a/ai-gateway/src/http.rs+++ b/ai-gateway/src/http.rs@@ -105,6 +105,7 @@ async fn handle(state: InferenceRouteState, mut request: Request, route: Route)Err(error) => error.into_native_response(route.api_format(), correlation.id()),};correlation.apply_generation_headers(response.headers_mut());+ correlation.apply_resolution_headers(response.headers_mut());response}@@ -121,7 +122,7 @@ async fn relay(.ok_or(InferenceHttpError::Unavailable)?;let gateway_key = gateway_key(request.headers(), route.api_format())?.to_owned();let (permit, context) = inference- .resolve_and_admit(&gateway_key, route.api_format(), correlation.id())+ .resolve_and_admit(&gateway_key, route.api_format(), correlation).await.map_err(|error| match error {RequestPreparationError::Resolution(error) => InferenceHttpError::Resolution(error),diff --git a/ai-gateway/src/http/tests.rs b/ai-gateway/src/http/tests.rsindex ed83c74f62..267b4f2358 100644--- a/ai-gateway/src/http/tests.rs+++ b/ai-gateway/src/http/tests.rs@@ -216,6 +216,23 @@ fn request_id(response: &reqwest::Response) -> String {id}+/// A request that never reaches the provider has no generation, and names only+/// its tenant, and only if it was resolved.+fn assert_pre_upstream_headers(headers: &reqwest::header::HeaderMap, resolved: bool) {+ assert!(!headers.contains_key("langfuse-trace-id"));+ assert!(!headers.contains_key("langfuse-observation-id"));+ for name in ["langfuse-organization-id", "langfuse-project-id"] {+ assert_eq!(headers.contains_key(name), resolved, "{name}");+ }+ for name in [+ "langfuse-provider",+ "langfuse-provider-connection-id",+ "langfuse-model",+ ] {+ assert!(!headers.contains_key(name), "{name}");+ }+}+#[tokio::test]async fn every_error_carries_a_fresh_request_id_in_its_header_and_native_body() {let web = FakeServer::start(|request| async move {@@ -240,23 +257,31 @@ async fn every_error_carries_a_fresh_request_id_in_its_header_and_native_body()let unconfigured = Gateway::start(None).await;let configured = Gateway::start(Some(inference(&web, &provider))).await;let mut seen = std::collections::HashSet::new();- for (request, status, anthropic) in [- (unconfigured.post(), StatusCode::SERVICE_UNAVAILABLE, false),+ for (request, status, anthropic, resolved) in [+ (+ unconfigured.post(),+ StatusCode::SERVICE_UNAVAILABLE,+ false,+ false,+ ),(unconfigured.messages(),StatusCode::SERVICE_UNAVAILABLE,true,+ false,),- (configured.post(), StatusCode::UNAUTHORIZED, false),+ (configured.post(), StatusCode::UNAUTHORIZED, false, false),(unconfigured.chat_completions(),StatusCode::SERVICE_UNAVAILABLE,false,+ false,),(configured.chat_completions().bearer_auth("rejected"),StatusCode::UNAUTHORIZED,false,+ false,),(configured@@ -265,16 +290,19 @@ async fn every_error_carries_a_fresh_request_id_in_its_header_and_native_body().body(vec![b'x'; MAX_REQUEST_BYTES + 1]),StatusCode::PAYLOAD_TOO_LARGE,false,+ true,),(configured.post().bearer_auth("rejected"),StatusCode::UNAUTHORIZED,false,+ false,),(configured.messages().header("x-api-key", "rejected"),StatusCode::UNAUTHORIZED,true,+ false,),(configured@@ -283,6 +311,7 @@ async fn every_error_carries_a_fresh_request_id_in_its_header_and_native_body().body(vec![b'x'; MAX_REQUEST_BYTES + 1]),StatusCode::PAYLOAD_TOO_LARGE,false,+ true,),] {let response = request@@ -292,8 +321,7 @@ async fn every_error_carries_a_fresh_request_id_in_its_header_and_native_body().unwrap();assert_eq!(response.status(), status);let id = request_id(&response);- assert!(!response.headers().contains_key("langfuse-trace-id"));- assert!(!response.headers().contains_key("langfuse-observation-id"));+ assert_pre_upstream_headers(response.headers(), resolved);let body: serde_json::Value =serde_json::from_slice(&response.bytes().await.unwrap()).unwrap();assert_eq!(body["request_id"], id.as_str());@@ -308,6 +336,65 @@ async fn every_error_carries_a_fresh_request_id_in_its_header_and_native_body()assert_eq!(provider.calls(), 0);}+#[tokio::test]+async fn resolved_requests_name_their_tenant_and_the_provider_connection_sent_to() {+ let web = FakeServer::start(|request| async move {+ let body = to_bytes(request.into_body(), 1024).await.unwrap();+ let api_format =+ serde_json::from_slice::<serde_json::Value>(&body).unwrap()["apiFormat"].clone();+ resolution_response_for(+ serde_json::from_value(api_format).unwrap(),+ "provider-secret",+ )+ })+ .await;+ let provider = FakeServer::start(|_| async {+ Response::builder()+ .header("content-type", "application/json")+ .body(Body::from("{}"))+ .unwrap()+ })+ .await;+ let gateway = Gateway::start(Some(inference(&web, &provider))).await;+ let client = reqwest::Client::new();+ let anthropic = |path: &str, body: &'static str| {+ client+ .post(format!("{}/anthropic/v1/{path}", gateway.url))+ .header("x-api-key", "gateway-anthropic")+ .header("content-type", "application/json")+ .body(body)+ };+ for (request, provider_type) in [+ (+ anthropic(+ "messages/count_tokens",+ r#"{"messages":[],"model":"claude-opus-4-1"}"#,+ ),+ "anthropic",+ ),+ (gateway.models().bearer_auth("gateway-openai"), "openai"),+ (+ anthropic("messages", r#"{"model":"claude-opus-4-1","messages":[]}"#),+ "anthropic",+ ),+ ] {+ let response = request.send().await.unwrap();+ assert_eq!(response.status(), StatusCode::OK);+ let headers = response.headers();+ let header = |name: &str| headers.get(name).map(|value| value.to_str().unwrap());+ assert_eq!(header("langfuse-organization-id"), Some("org-1"));+ assert_eq!(header("langfuse-project-id"), Some("project-1"));+ assert_eq!(header("langfuse-provider"), Some(provider_type));+ assert_eq!(+ header("langfuse-provider-connection-id"),+ Some("connection-1")+ );+ // The model is an operator-side span and log field only.+ assert_eq!(header("langfuse-model"), None);+ }+ assert_eq!(provider.calls(), 3);+}+impl Drop for Gateway {fn drop(&mut self) {self.task.abort();diff --git a/ai-gateway/src/inference.rs b/ai-gateway/src/inference.rsindex 0a963c9054..4c13ca773f 100644--- a/ai-gateway/src/inference.rs+++ b/ai-gateway/src/inference.rs@@ -57,11 +57,12 @@ impl InferenceService {/// Authenticate with a separate bounded budget before reserving execution capacity./// The API format comes from the public route, so Web selects a compatible connection.+ /// A successful resolution is recorded even if execution admission then fails.pub(crate) async fn resolve_and_admit(&self,gateway_key: &str,api_format: ApiFormat,- request_id: &str,+ correlation: &mut RequestCorrelation,) -> Result<(RequestPermit, ResolvedRequestContext), RequestPreparationError> {let span = tracing::info_span!("resolution",@@ -69,7 +70,7 @@ impl InferenceService {gateway.outcome = tracing::field::Empty);async {- let prepared = self.prepare(gateway_key, api_format, request_id).await;+ let prepared = self.prepare(gateway_key, api_format, correlation).await;tracing::Span::current().record("gateway.outcome",match &prepared {@@ -88,7 +89,7 @@ impl InferenceService {&self,gateway_key: &str,api_format: ApiFormat,- request_id: &str,+ correlation: &mut RequestCorrelation,) -> Result<(RequestPermit, ResolvedRequestContext), RequestPreparationError> {let context = {let _permit = self.resolution_capacity.try_acquire().map_err(|_| {@@ -97,10 +98,11 @@ impl InferenceService {})?;let _active = crate::observability::Active::new("resolution");self.control_plane- .resolve(gateway_key, api_format, request_id)+ .resolve(gateway_key, api_format, correlation.id()).await.map_err(RequestPreparationError::Resolution)?};+ correlation.record_resolution(&context);let permit = self.provider.try_admit()diff --git a/ai-gateway/src/observability/http.rs b/ai-gateway/src/observability/http.rsindex e379e6e09e..67c116281a 100644--- a/ai-gateway/src/observability/http.rs+++ b/ai-gateway/src/observability/http.rs@@ -21,7 +21,7 @@ pub fn instrument(router: Router) -> Router {TraceLayer::new_for_http().make_span_with(|request: &Request<Body>| {let (method, route) = request_labels(request);- let span = tracing::info_span!(parent: None, "http.server", otel.name = %format_args!("{method} {route}"), otel.kind = "server", http.request.method = method, http.route = route, http.request.body.size = tracing::field::Empty, http.response.status_code = tracing::field::Empty, otel.status_code = tracing::field::Empty, gateway.request.id = tracing::field::Empty, gateway.client.request.id = tracing::field::Empty, provider_request_id = tracing::field::Empty, gateway.outcome = tracing::field::Empty, gateway.first_byte_ms = tracing::field::Empty);+ let span = tracing::info_span!(parent: None, "http.server", otel.name = %format_args!("{method} {route}"), otel.kind = "server", http.request.method = method, http.route = route, http.request.body.size = tracing::field::Empty, http.response.status_code = tracing::field::Empty, otel.status_code = tracing::field::Empty, gateway.request.id = tracing::field::Empty, gateway.client.request.id = tracing::field::Empty, langfuse.organization.id = tracing::field::Empty, langfuse.project.id = tracing::field::Empty, gateway.provider = tracing::field::Empty, gateway.provider.connection.id = tracing::field::Empty, gateway.provider.connection.name = tracing::field::Empty, gateway.upstream.model = tracing::field::Empty, provider_request_id = tracing::field::Empty, gateway.outcome = tracing::field::Empty, gateway.first_byte_ms = tracing::field::Empty);// Operational tracing has no caller or ambient trace context.let _ = span.set_parent(Context::new());spandiff --git a/ai-gateway/src/observability/http/tests.rs b/ai-gateway/src/observability/http/tests.rsindex fb90de8281..25fd158371 100644--- a/ai-gateway/src/observability/http/tests.rs+++ b/ai-gateway/src/observability/http/tests.rs@@ -57,11 +57,11 @@ impl Recording {tracing_subscriber::fmt::layer().json().flatten_event(true)- .map_event_format(crate::observability::logs::WithRequestId)+ .map_event_format(crate::observability::logs::WithCorrelationFields).with_writer(move || buffer.clone()),).with(tracing_opentelemetry::layer().with_tracer(provider.tracer("test")))- .with(crate::observability::logs::RequestIdSpans);+ .with(crate::observability::logs::CorrelationSpans);Self {_guard: tracing::subscriber::set_default(subscriber),_provider: provider,@@ -510,6 +510,39 @@ fn assert_request_correlation(assert_attribute(server, "gateway.request.id", request_id.to_owned());assert_attribute(server, "gateway.client.request.id", "client-request-canary");assert_attribute(server, "provider_request_id", "req_upstream");+ let resolution = [+ ("langfuse.organization.id", "organization_id", "org-1"),+ ("langfuse.project.id", "project_id", "project-1"),+ ("gateway.provider", "provider", "openai"),+ (+ "gateway.provider.connection.id",+ "provider_connection_id",+ "connection-1",+ ),+ (+ "gateway.provider.connection.name",+ "provider_connection_name",+ "Production key",+ ),+ ("gateway.upstream.model", "upstream_model", "test-model"),+ ];+ for (attribute, _, expected) in resolution {+ assert_attribute(server, attribute, expected);+ }+ for (name, expected) in [+ ("langfuse-organization-id", "org-1"),+ ("langfuse-project-id", "project-1"),+ ("langfuse-provider", "openai"),+ ("langfuse-provider-connection-id", "connection-1"),+ ] {+ assert_eq!(header(name), expected, "{name}");+ }+ assert!(response.values().all(|value| {+ value+ .to_str()+ .is_ok_and(|value| !value.contains("Production key"))+ }));+ assert!(!response.contains_key("langfuse-model"));let events: Vec<_> = recording.events().into_iter()@@ -525,6 +558,11 @@ fn assert_request_correlation(}for event in &events {assert_eq!(event["request_id"], request_id, "{event}");+ if event["message"] == "gateway response started" {+ for (_, key, expected) in resolution {+ assert_eq!(event[key], expected, "{event}");+ }+ }}for (name, headers, payload) in observed {assert_eq!(diff --git a/ai-gateway/src/observability/logs.rs b/ai-gateway/src/observability/logs.rsindex 195da09237..8097079ba8 100644--- a/ai-gateway/src/observability/logs.rs+++ b/ai-gateway/src/observability/logs.rs@@ -1,5 +1,6 @@-//! JSON span fields are nested, so the request ID is repeated as a top-level-//! `request_id` on every event emitted inside a request, for log search.+//! JSON span fields are nested, so the request's correlation fields are repeated+//! as top-level keys, such as `request_id`, on every event emitted inside a+//! request, for log search.use std::fmt;use tracing::{@@ -13,55 +14,85 @@ use tracing_subscriber::{registry::LookupSpan,};-const REQUEST_ID_FIELD: &str = "gateway.request.id";+/// Span field and its top-level log key. All are recorded on the server span.+const CORRELATION_FIELDS: [(&str, &str); 7] = [+ ("gateway.request.id", "request_id"),+ ("langfuse.organization.id", "organization_id"),+ ("langfuse.project.id", "project_id"),+ ("gateway.provider", "provider"),+ ("gateway.provider.connection.id", "provider_connection_id"),+ (+ "gateway.provider.connection.name",+ "provider_connection_name",+ ),+ ("gateway.upstream.model", "upstream_model"),+];-struct LoggedRequestId(String);+#[derive(Default)]+struct LoggedFields([Option<String>; CORRELATION_FIELDS.len()]);-/// Keeps each span's `gateway.request.id` where [`WithRequestId`] can find it.-pub(crate) struct RequestIdSpans;+/// Keeps each span's correlation fields where [`WithCorrelationFields`] can find them.+pub(crate) struct CorrelationSpans;-impl<S> Layer<S> for RequestIdSpans+impl<S> Layer<S> for CorrelationSpanswhereS: Subscriber + for<'a> LookupSpan<'a>,{fn on_new_span(&self, attributes: &span::Attributes<'_>, id: &span::Id, ctx: Context<'_, S>) {- let mut visitor = RequestIdVisitor(None);+ let mut visitor = CorrelationVisitor::default();attributes.record(&mut visitor);- store(id, &ctx, visitor.0);+ store(id, &ctx, visitor);}fn on_record(&self, id: &span::Id, values: &span::Record<'_>, ctx: Context<'_, S>) {- let mut visitor = RequestIdVisitor(None);+ let mut visitor = CorrelationVisitor::default();values.record(&mut visitor);- store(id, &ctx, visitor.0);+ store(id, &ctx, visitor);}}-fn store<S>(id: &span::Id, ctx: &Context<'_, S>, request_id: Option<String>)+fn store<S>(id: &span::Id, ctx: &Context<'_, S>, visitor: CorrelationVisitor)whereS: Subscriber + for<'a> LookupSpan<'a>,{- if let (Some(request_id), Some(span)) = (request_id, ctx.span(id)) {- span.extensions_mut().replace(LoggedRequestId(request_id));+ if visitor.0.0.iter().all(Option::is_none) {+ return;}+ let Some(span) = ctx.span(id) else {+ return;+ };+ let mut extensions = span.extensions_mut();+ if let Some(logged) = extensions.get_mut::<LoggedFields>() {+ for (slot, value) in logged.0.iter_mut().zip(visitor.0.0) {+ if value.is_some() {+ *slot = value;+ }+ }+ return;+ }+ extensions.insert(visitor.0);}-struct RequestIdVisitor(Option<String>);+#[derive(Default)]+struct CorrelationVisitor(LoggedFields);-impl Visit for RequestIdVisitor {+impl Visit for CorrelationVisitor {fn record_str(&mut self, field: &Field, value: &str) {- if field.name() == REQUEST_ID_FIELD {- self.0 = Some(value.to_owned());+ if let Some(index) = CORRELATION_FIELDS+ .iter()+ .position(|(name, _)| *name == field.name())+ {+ self.0.0[index] = Some(value.to_owned());}}fn record_debug(&mut self, _: &Field, _: &dyn fmt::Debug) {}}-/// Wraps a JSON event formatter and prepends the innermost span's request ID.-pub(crate) struct WithRequestId<F>(pub F);+/// Wraps a JSON event formatter and prepends the innermost span's correlation fields.+pub(crate) struct WithCorrelationFields<F>(pub F);-impl<S, N, F> FormatEvent<S, N> for WithRequestId<F>+impl<S, N, F> FormatEvent<S, N> for WithCorrelationFields<F>whereS: Subscriber + for<'a> LookupSpan<'a>,N: for<'a> FormatFields<'a> + 'static,@@ -73,14 +104,14 @@ wheremut writer: Writer<'_>,event: &Event<'_>,) -> fmt::Result {- let request_id = ctx.event_scope().and_then(|scope| {+ let fields = ctx.event_scope().and_then(|scope| {scope.into_iter().find_map(|span| {span.extensions()- .get::<LoggedRequestId>()- .map(|id| id.0.clone())+ .get::<LoggedFields>()+ .map(|fields| fields.0.clone())})});- let Some(request_id) = request_id else {+ let Some(fields) = fields else {return self.0.format_event(ctx, writer, event);};let mut line = String::new();@@ -88,9 +119,20 @@ wherelet Some(rest) = line.strip_prefix('{') else {return writer.write_str(&line);};- let request_id = serde_json::to_string(&request_id).map_err(|_| fmt::Error)?;- write!(writer, "{{\"request_id\":{request_id}")?;- if !rest.starts_with('}') {+ writer.write_char('{')?;+ let mut first = true;+ for ((_, key), value) in CORRELATION_FIELDS.iter().zip(fields) {+ let Some(value) = value else {+ continue;+ };+ if !first {+ writer.write_char(',')?;+ }+ first = false;+ let value = serde_json::to_string(&value).map_err(|_| fmt::Error)?;+ write!(writer, "\"{key}\":{value}")?;+ }+ if !first && !rest.starts_with('}') {writer.write_char(',')?;}writer.write_str(rest)diff --git a/ai-gateway/src/observability/mod.rs b/ai-gateway/src/observability/mod.rsindex 1372de5436..45012fa99e 100644--- a/ai-gateway/src/observability/mod.rs+++ b/ai-gateway/src/observability/mod.rs@@ -103,7 +103,7 @@ pub fn init(config: &GatewayConfig) -> Result<Observability, Box<dyn Error>> {LogFormat::Json => tracing_subscriber::fmt::layer().json().flatten_event(true)- .map_event_format(logs::WithRequestId)+ .map_event_format(logs::WithCorrelationFields).with_filter(log_filter).boxed(),LogFormat::Text => tracing_subscriber::fmt::layer()@@ -124,7 +124,7 @@ pub fn init(config: &GatewayConfig) -> Result<Observability, Box<dyn Error>> {tracing_subscriber::registry().with(logger).with(spans)- .with(logs::RequestIdSpans)+ .with(logs::CorrelationSpans).try_init().map_err(|_| "failed to initialize gateway logging")?;Ok(Observability { traces, metrics })diff --git a/ai-gateway/src/providers/mod.rs b/ai-gateway/src/providers/mod.rsindex 5e5871627b..3c0202f94b 100644--- a/ai-gateway/src/providers/mod.rs+++ b/ai-gateway/src/providers/mod.rs@@ -17,7 +17,7 @@ use tokio::{pub use crate::transport::ProviderError;use crate::{capture::{ExecutionCapture, RelayOutcome},- correlation::RequestCorrelation,+ correlation::{RequestCorrelation, UpstreamTarget},resolution::{ApiFormat, Provider, ProviderCredential, ResolvedRequestContext},transport,};@@ -233,6 +233,9 @@ impl ProviderTransport {}_ => body,};+ // The relay forwards the caller's `model` unchanged; only a captured route has parsed it.+ let model = capture.requested_model().map(str::to_owned);+ correlation.record_upstream(UpstreamTarget::for_connection(context.connection(), model));let mut upstream = self.client.request(route.method(), self.request_url(route, query))diff --git a/ai-gateway/src/resolution/contracts.rs b/ai-gateway/src/resolution/contracts.rsindex c296f2f00c..5e3ec18975 100644--- a/ai-gateway/src/resolution/contracts.rs+++ b/ai-gateway/src/resolution/contracts.rs@@ -24,6 +24,14 @@ pub enum Provider {}impl Provider {+ /// The stable lowercase provider type, as Web sends it.+ pub fn as_str(self) -> &'static str {+ match self {+ Self::OpenAi => "openai",+ Self::Anthropic => "anthropic",+ }+ }+pub fn official_origin(self) -> &'static str {match self {Self::OpenAi => "https://api.openai.com/v1",@@ -130,6 +138,9 @@ impl Auth {#[serde(deny_unknown_fields)]pub struct ProviderConnection {id: String,+ // Optional so the gateway keeps resolving against a Web that predates it.+ #[serde(default)]+ name: Option<String>,#[serde(deserialize_with = "string_enum")]provider: Provider,#[serde(deserialize_with = "string_enum")]@@ -142,6 +153,10 @@ impl ProviderConnection {pub fn id(&self) -> &str {&self.id}+ /// The user-chosen display name. Never send it to callers; it is for operators.+ pub fn name(&self) -> Option<&str> {+ self.name.as_deref().filter(|name| !name.trim().is_empty())+ }pub fn provider(&self) -> Provider {self.provider}diff --git a/ai-gateway/src/resolution/tests.rs b/ai-gateway/src/resolution/tests.rsindex 59aa61e038..009cc19513 100644--- a/ai-gateway/src/resolution/tests.rs+++ b/ai-gateway/src/resolution/tests.rs@@ -95,6 +95,7 @@ fn success(project: &str, provider_token: &str) -> Value {"version": 1,"connection": {"id": "connection-1",+ "name": "Primary OpenAI","provider": "openai","api_format": "openai.responses","base_url": "https://api.openai.com/v1",@@ -138,7 +139,9 @@ async fn signs_the_exact_key_and_sends_only_the_api_format() {.await.unwrap();assert_eq!(context.connection().id(), "connection-1");+ assert_eq!(context.connection().name(), Some("Primary OpenAI"));assert_eq!(context.connection().provider(), Provider::OpenAi);+ assert_eq!(context.connection().provider().as_str(), "openai");assert!(matches!(context.connection().credential(),ProviderCredential::Bearer("private-provider-token")@@ -407,7 +410,10 @@ async fn resolves_anthropic_messages_with_the_native_credential_scheme() {.await.unwrap();assert_eq!(context.connection().id(), "connection-anthropic");+ // Web deployments that predate connection names omit the field.+ assert_eq!(context.connection().name(), None);assert_eq!(context.connection().provider(), Provider::Anthropic);+ assert_eq!(context.connection().provider().as_str(), "anthropic");assert_eq!(context.connection().api_format(),ApiFormat::AnthropicMessagesdiff --git a/ai-gateway/src/test_support.rs b/ai-gateway/src/test_support.rsindex 23606f7f9b..37b59075bb 100644--- a/ai-gateway/src/test_support.rs+++ b/ai-gateway/src/test_support.rs@@ -99,17 +99,17 @@ pub(crate) fn resolution_response_for(+ 300;let connection = match api_format {ApiFormat::OpenAiResponses => json!({- "id": "connection-1", "provider": "openai", "api_format": "openai.responses",+ "id": "connection-1", "name": "Production key", "provider": "openai", "api_format": "openai.responses","base_url": "https://api.openai.com/v1","auth": {"type": "Bearer", "token": provider_secret}}),ApiFormat::OpenAiChatCompletions => json!({- "id": "connection-1", "provider": "openai", "api_format": "openai.chat-completions",+ "id": "connection-1", "name": "Production key", "provider": "openai", "api_format": "openai.chat-completions","base_url": "https://api.openai.com/v1","auth": {"type": "Bearer", "token": provider_secret}}),ApiFormat::AnthropicMessages => json!({- "id": "connection-1", "provider": "anthropic", "api_format": "anthropic.messages",+ "id": "connection-1", "name": "Production key", "provider": "anthropic", "api_format": "anthropic.messages","base_url": "https://api.anthropic.com/v1","auth": {"type": "x-api-key", "header": "x-api-key", "value": provider_secret}}),diff --git a/web/src/__tests__/server/ai-gateway-control-plane.servertest.ts b/web/src/__tests__/server/ai-gateway-control-plane.servertest.tsindex 50d5b68cfe..b4409e8b89 100644--- a/web/src/__tests__/server/ai-gateway-control-plane.servertest.ts+++ b/web/src/__tests__/server/ai-gateway-control-plane.servertest.ts@@ -554,6 +554,7 @@ describe("AI gateway control plane", () => {expect(openAiPrimary.routingPriority).toBe(0);expect(result.connection).toEqual({id: openAiPrimary.id,+ name: "OpenAI first",provider: "openai",api_format: apiFormat,base_url: "https://api.openai.com/v1",diff --git a/web/src/features/ai-gateway/server/auth/gatewayApiKeyAuthenticator.ts b/web/src/features/ai-gateway/server/auth/gatewayApiKeyAuthenticator.tsindex 9819044ce1..44e22d1a30 100644--- a/web/src/features/ai-gateway/server/auth/gatewayApiKeyAuthenticator.ts+++ b/web/src/features/ai-gateway/server/auth/gatewayApiKeyAuthenticator.ts@@ -94,6 +94,7 @@ export class GatewayApiKeyAuthenticator {connection: connection? {id: connection.id,+ name: connection.name,provider: connection.provider,encryptedCredential: connection.encryptedCredential,}diff --git a/web/src/features/ai-gateway/server/provider/registry.ts b/web/src/features/ai-gateway/server/provider/registry.tsindex e0e6f51c2d..c9bb3b20df 100644--- a/web/src/features/ai-gateway/server/provider/registry.ts+++ b/web/src/features/ai-gateway/server/provider/registry.ts@@ -25,6 +25,7 @@ export const GatewayResolveResponseSchema = zconnection: z.object({id: z.string(),+ name: z.string(),provider: z.enum(gatewayProviderIds),api_format: GatewayApiFormatSchema,base_url: z.url(),diff --git a/web/src/features/ai-gateway/server/resolve/gatewayResolveCache.ts b/web/src/features/ai-gateway/server/resolve/gatewayResolveCache.tsindex e8c4722cd0..5186f1ce98 100644--- a/web/src/features/ai-gateway/server/resolve/gatewayResolveCache.ts+++ b/web/src/features/ai-gateway/server/resolve/gatewayResolveCache.ts@@ -32,6 +32,7 @@ export const CachedResolveContext = z.object({connection: z.object({id: z.string(),+ name: z.string(),provider: z.enum(["OPENAI", "ANTHROPIC"]),encryptedCredential: z.string(),})diff --git a/web/src/features/ai-gateway/server/resolve/gatewayResolveRepository.ts b/web/src/features/ai-gateway/server/resolve/gatewayResolveRepository.tsindex c9c20502c2..0351dde840 100644--- a/web/src/features/ai-gateway/server/resolve/gatewayResolveRepository.ts+++ b/web/src/features/ai-gateway/server/resolve/gatewayResolveRepository.ts@@ -44,6 +44,7 @@ export class GatewayResolveRepository {take: 1,select: {id: true,+ name: true,provider: true,encryptedCredential: true,},diff --git a/web/src/features/ai-gateway/server/resolve/resolveService.servertest.ts b/web/src/features/ai-gateway/server/resolve/resolveService.servertest.tsindex 001f5255d6..faf755d079 100644--- a/web/src/features/ai-gateway/server/resolve/resolveService.servertest.ts+++ b/web/src/features/ai-gateway/server/resolve/resolveService.servertest.ts@@ -70,6 +70,7 @@ const row = (overrides: {gatewayAiConnections: [{id: "connection-1",+ name: "Production OpenAI",provider: "OPENAI",encryptedCredential: "encrypted-credential",},@@ -95,6 +96,7 @@ describe("GatewayResolveService", () => {version: 1,connection: {id: "connection-1",+ name: "Production OpenAI",provider: "openai",api_format: "openai.responses",base_url: "https://api.openai.com/v1",diff --git a/web/src/features/ai-gateway/server/resolve/resolveService.ts b/web/src/features/ai-gateway/server/resolve/resolveService.tsindex 668c3a594e..6c774d0d3a 100644--- a/web/src/features/ai-gateway/server/resolve/resolveService.ts+++ b/web/src/features/ai-gateway/server/resolve/resolveService.ts@@ -95,6 +95,7 @@ export class GatewayResolveService {version: 1 as const,connection: {id: context.connection.id,+ name: context.connection.name,provider:context.connection.provider.toLowerCase() as GatewayProviderId,api_format: params.apiFormat,