🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
feat(ai-gateway): isolate caller tracing context for generations (#17549)
Hassieb Pakzad81a268c59bb2
diff --git a/ai-gateway/Cargo.lock b/ai-gateway/Cargo.lockindex b20a6bdff7..54ac5b80a9 100644--- a/ai-gateway/Cargo.lock+++ b/ai-gateway/Cargo.lock@@ -39,7 +39,6 @@ dependencies = ["tracing","tracing-opentelemetry","tracing-subscriber",- "uuid",][[package]]@@ -1882,17 +1881,6 @@ version = "1.0.4"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"-[[package]]-name = "uuid"-version = "1.26.1"-source = "registry+https://github.com/rust-lang/crates.io-index"-checksum = "2ef6dac1e96601b4fb3acccccff2139741fcb757cb9a36089bf5be91cfb285ce"-dependencies = [- "getrandom 0.4.3",- "js-sys",- "wasm-bindgen",-]-[[package]]name = "valuable"version = "0.1.1"diff --git a/ai-gateway/Cargo.toml b/ai-gateway/Cargo.tomlindex 975d5aaedc..e6f95b2ea5 100644--- a/ai-gateway/Cargo.toml+++ b/ai-gateway/Cargo.toml@@ -26,7 +26,6 @@ tower-http = { version = "0.7", features = ["trace"] }tracing = "0.1.44"tracing-opentelemetry = { version = "0.33", default-features = false }tracing-subscriber = { version = "0.3.23", default-features = false, features = ["env-filter", "fmt", "json"] }-uuid = { version = "1", features = ["v4"] }[dev-dependencies]http-body = "1"diff --git a/ai-gateway/README.md b/ai-gateway/README.mdindex ddc70f6675..0431135f33 100644--- a/ai-gateway/README.md+++ b/ai-gateway/README.md@@ -108,7 +108,7 @@ trace sampling. `trace` is not an accepted gateway log level.| `OTEL_SERVICE_NAME` | `ai-gateway` | Operational service name || `DD_ENV` | `development` | Deployment environment resource attribute || `BUILD_ID` | Cargo package version | Service version; set to the deployed image's commit SHA |-| `OTEL_TRACES_SAMPLER_ARG` | `1` | Sampling ratio from 0 to 1 for root traces; incoming W3C sampling decisions are respected |+| `OTEL_TRACES_SAMPLER_ARG` | `1` | Sampling ratio from 0 to 1 for independent operational root traces |Tower's `TraceLayer` keeps the server span alive through the response body.The HTTP response log and `http.server.request.duration` measure time to response@@ -118,9 +118,11 @@ timeout, or transport error. HTTP status and stream outcome are separate: a 200response can still fail while streaming.`reqwest-tracing` instruments resolver, provider-header, and ingestion requests.-Only trusted Web calls receive W3C trace context; baggage and operational trace-headers are not sent to the external provider. Inference generations retain their-separate identity. Client spans measure the HTTP send through response headers;+Each request starts a fresh operational trace, independent of incoming trace IDs,+tracestate, baggage, Langfuse headers and sampling decisions. Only trusted Web calls+receive this internal W3C trace context; the external provider receives no tracing+or Langfuse headers. Inference generations use the caller context described below.+Client spans measure the HTTP send through response headers;streaming execution and inference-telemetry delivery are tracked separately.A small Axum middleware records `http.server.request.duration` through the@@ -227,7 +229,8 @@ and never sends the provider credential or the client's gateway key to ingestionIngestion JWTs stay outside serializable capture facts and debug logs. Expired grantsare dropped without re-resolving the execution. Requests explicitly select v4 ingestion.-Each execution becomes one root generation with generated trace/observation IDs,+Each execution becomes one generation with a new observation ID and either the+caller's trace/parent IDs or a generated root trace ID,actual/requested model, model parameters, native usage, and trusted key/connectionattribution. Full mode includes the captured input/output; usage mode omits content.Completion-start time is emitted only for upstream `text/event-stream` responses,@@ -271,6 +274,62 @@ a span collection; future project batching can replace immediate scheduling whilretaining each execution's original attribution/content policy and selecting a validcompatible grant. Capture and provider byte forwarding do not need to change.+## Caller tracing context++Incoming `traceparent` and `tracestate` apply only to the Langfuse generation.+A valid `traceparent` supplies its trace ID and parent observation ID; the generation+always gets a new observation ID. Missing or malformed context starts a new root.+Valid `tracestate` is included in the generation's OTLP `traceState` field; it is not+copied into generation metadata. An unsampled caller still produces a generation.+Operational trace sampling and best-effort ingestion remain independent.++The following optional headers enrich the generation, including in usage mode:++| Header | Format | Python SDK baggage key |+| --- | --- | --- |+| `langfuse-trace-name` | String | `langfuse_trace_name` |+| `langfuse-session-id` | String | `langfuse_session_id` |+| `langfuse-user-id` | String | `langfuse_user_id` |+| `langfuse-tags` | Comma-separated strings | `langfuse_tags` |+| `langfuse-metadata` | Comma-separated `key:value` entries | `langfuse_metadata_<key>` |++For example:++```text+langfuse-trace-name: support-workflow+langfuse-session-id: conversation-123+langfuse-user-id: user-456+langfuse-tags: support,production+langfuse-metadata: team:search,variant:B,note:hello%2C%20world+```++Percent-encode literal commas and other escaped characters in individual custom+header values, and colons inside metadata keys. Metadata splits at the first colon;+values remain strings. A literal `+` stays `+` in custom headers. Tags are trimmed+and deduplicated. Valid explicit headers override baggage for the same field;+metadata merges by key with explicit entries winning. Invalid entries are ignored+independently, and invalid overrides leave valid baggage intact. Caller metadata+cannot replace protected gateway facts or trusted API-key attribution.++Extraction is bounded to 8 KiB across the eight context header values above+(`traceparent`, `tracestate`, `baggage` and the five custom headers). Above that+limit, context is ignored and a fresh generation trace is created. Decoded fields+are limited to 1 KiB; each baggage/tag/metadata list is limited to 64 entries.+Repeated list header lines are combined in order within that same entry limit.+Empty values, control characters, invalid encoding and duplicate scalar headers+are ignored without rejecting inference.++Python callers can use `propagate_attributes(..., as_baggage=True)` with HTTP+instrumentation or explicit OTel header injection. Baggage decoding handles Python's+`+` space encoding and quoted-list tags, as well as JSON-array tags. Only the keys+listed above are mapped; other baggage, including `langfuse_trace_id`, does not+override trace identity or project selection.++Caller context is kept outside ambient operational context and operational logs.+Resolver and ingestion HTTP requests propagate only the gateway's internal trace;+the generation's caller context travels in the ingestion payload. Provider requests+receive neither tracing context nor these custom headers.+## Response captureSet `LANGFUSE_LOG_LEVEL=debug` to print one `gateway response captured` event atdiff --git a/ai-gateway/src/capture/mod.rs b/ai-gateway/src/capture/mod.rsindex f1b86d30ba..b37d850b0e 100644--- a/ai-gateway/src/capture/mod.rs+++ b/ai-gateway/src/capture/mod.rs@@ -119,10 +119,11 @@ impl ExecutionCapture {&mut self,telemetry: telemetry::Telemetry,context: &ResolvedRequestContext,+ headers: &HeaderMap,) {self.delivery = Some((telemetry,- telemetry::DeliveryContext::from_resolved(context),+ telemetry::DeliveryContext::from_resolved(context, headers),));}diff --git a/ai-gateway/src/observability/http.rs b/ai-gateway/src/observability/http.rsindex bfa2800049..38302a2161 100644--- a/ai-gateway/src/observability/http.rs+++ b/ai-gateway/src/observability/http.rs@@ -8,9 +8,7 @@ use axum::{middleware::{self, Next},response::Response,};-use opentelemetry::{propagation::TextMapPropagator, trace::TraceContextExt};-use opentelemetry_http::HeaderExtractor;-use opentelemetry_sdk::propagation::TraceContextPropagator;+use opentelemetry::{Context, trace::TraceContextExt};use tower_http::trace::TraceLayer;use tracing::Span;use tracing_opentelemetry::OpenTelemetrySpanExt;@@ -23,9 +21,9 @@ pub fn instrument(router: Router) -> Router {TraceLayer::new_for_http().make_span_with(|request: &Request<Body>| {let (method, route) = request_labels(request);- let span = tracing::info_span!("http.server", otel.name = %format_args!("{method} {route}"), otel.kind = "server", http.request.method = method, http.route = route, http.response.status_code = tracing::field::Empty, otel.status_code = tracing::field::Empty, provider_request_id = tracing::field::Empty);- let parent = TraceContextPropagator::new().extract(&HeaderExtractor(request.headers()));- let _ = span.set_parent(parent);+ let span = tracing::info_span!(parent: None, "http.server", otel.name = %format_args!("{method} {route}"), otel.kind = "server", http.request.method = method, http.route = route, http.response.status_code = tracing::field::Empty, otel.status_code = tracing::field::Empty, provider_request_id = tracing::field::Empty);+ // Operational tracing has no caller or ambient trace context.+ let _ = span.set_parent(Context::new());span}).on_request(())diff --git a/ai-gateway/src/observability/http/tests.rs b/ai-gateway/src/observability/http/tests.rsindex 1c92a28973..1a84aa9c6f 100644--- a/ai-gateway/src/observability/http/tests.rs+++ b/ai-gateway/src/observability/http/tests.rs@@ -1,4 +1,12 @@use super::*;+use crate::{+ inference::InferenceService,+ providers::openai::{OpenAiProvider, ProviderLimits},+ resolution::ControlPlaneConfig,+ server::GatewayLifecycleState,+ telemetry::Telemetry,+ test_support::{FakeServer, resolution_response},+};use axum::body::Bytes;use axum::{Router,@@ -15,6 +23,7 @@ use std::{sync::{Arc, Mutex},};use tower::ServiceExt;+use tracing::Instrument;use tracing_subscriber::prelude::*;#[derive(Clone, Default)]@@ -76,13 +85,14 @@ fn incoming() -> HttpRequest<Body> {"traceparent","00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01",)+ .header("tracestate", "caller=state-canary").header("baggage", "private=content-canary").body(Body::empty()).unwrap()}#[tokio::test]-async fn inherits_trace_and_preserves_stream_lifetime_bytes_and_trailers() {+async fn starts_independent_trace_and_preserves_stream_lifetime_bytes_and_trailers() {let recording = Recording::start();let app = instrument(Router::new().fallback(|| async {let mut trailers = HeaderMap::new();@@ -93,7 +103,12 @@ async fn inherits_trace_and_preserves_stream_lifetime_bytes_and_trailers() {];Body::new(StreamBody::new(futures_util::stream::iter(frames)))}));- let response = app.oneshot(incoming()).await.unwrap();+ let ambient = tracing::info_span!("ambient");+ let response = app+ .oneshot(incoming())+ .instrument(ambient.clone())+ .await+ .unwrap();assert!(recording.exporter.get_finished_spans().unwrap().is_empty());let summaries = recording.summaries();assert_eq!(summaries.len(), 1);@@ -117,11 +132,16 @@ async fn inherits_trace_and_preserves_stream_lifetime_bytes_and_trailers() {assert_eq!(spans.len(), 1);let server = &spans[0];assert_eq!(server.span_kind, SpanKind::Server);- assert_eq!(+ assert_ne!(server.span_context.trace_id().to_string(),"4bf92f3577b34da6a3ce929d0e0e4736");- assert_eq!(server.parent_span_id.to_string(), "00f067aa0ba902b7");+ assert_ne!(+ server.span_context.trace_id(),+ ambient.context().span().span_context().trace_id()+ );+ assert_eq!(server.parent_span_id, opentelemetry::trace::SpanId::INVALID);+ assert!(server.span_context.trace_state().header().is_empty());assert_eq!(summaries[0]["trace_id"],server.span_context.trace_id().to_string()@@ -209,3 +229,186 @@ async fn provider_http_errors_are_traced_without_changing_the_response() {assert_eq!(upstream.calls(), 1);}}++#[tokio::test]+async fn generation_context_is_isolated_from_operational_spans_and_outbound_headers() {+ opentelemetry::global::set_text_map_propagator(+ opentelemetry_sdk::propagation::TraceContextPropagator::new(),+ );+ let observed = Arc::new(Mutex::new(Vec::new()));+ let web_calls = observed.clone();+ let web = FakeServer::start(move |request| {+ let web_calls = web_calls.clone();+ async move {+ let (parts, body) = request.into_parts();+ let bytes = axum::body::to_bytes(body, 64 * 1024).await.unwrap();+ let ingestion = parts.uri.path() == "/api/public/otel/v1/traces";+ web_calls.lock().unwrap().push((+ if ingestion { "ingestion" } else { "resolver" },+ parts.headers,+ serde_json::from_slice::<Value>(&bytes).unwrap(),+ ));+ if ingestion {+ Response::new(Body::from("{}"))+ } else {+ resolution_response("provider-secret")+ }+ }+ })+ .await;+ let provider_calls = observed.clone();+ let provider = FakeServer::start(move |request| {+ provider_calls.lock().unwrap().push((+ "provider.headers",+ request.headers().clone(),+ Value::Null,+ ));+ async {+ Response::builder()+ .header("content-type", "application/json")+ .body(Body::from(r#"{"id":"resp_1","output":[]}"#))+ .unwrap()+ }+ })+ .await;+ let telemetry =+ Telemetry::new(&ControlPlaneConfig::new(&web.url, "test-service-key").unwrap()).unwrap();+ let service = InferenceService::for_test(+ web.control_plane(),+ OpenAiProvider::for_test(provider.url.clone(), ProviderLimits::default())+ .with_telemetry(telemetry.clone()),+ 1,+ );+ let recording = Recording::start();+ let lifecycle = GatewayLifecycleState::default();+ let app = instrument(crate::http::router(Some(service), lifecycle.clone()));+ let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();+ let mut serving = Box::pin(crate::server::serve(+ listener,+ app.clone(),+ lifecycle,+ std::future::pending(),+ Duration::from_secs(1),+ ));+ assert!(futures_util::poll!(serving.as_mut()).is_pending());+ let response = app.oneshot(generation_request()).await.unwrap();+ assert_eq!(response.status(), 200);+ axum::body::to_bytes(response.into_body(), 4096)+ .await+ .unwrap();+ telemetry+ .shutdown(tokio::time::Instant::now() + Duration::from_secs(2))+ .await;+ let spans = recording.exporter.get_finished_spans().unwrap();+ let server = spans+ .iter()+ .find(|span| span.span_kind == SpanKind::Server)+ .unwrap();+ assert_eq!(spans.len(), 4);+ assert_eq!(server.parent_span_id, opentelemetry::trace::SpanId::INVALID);+ assert_ne!(+ server.span_context.trace_id().to_string(),+ "4bf92f3577b34da6a3ce929d0e0e4736"+ );+ let observed = observed.lock().unwrap();+ assert_eq!(observed.len(), 3);+ for (name, headers, payload) in &*observed {+ let child = spans.iter().find(|span| span.name == *name).unwrap();+ assert_eq!(+ child.span_context.trace_id(),+ server.span_context.trace_id()+ );+ assert!(child.span_context.trace_state().header().is_empty());+ assert_eq!(child.parent_span_id, server.span_context.span_id());+ assert_outbound_context(headers, (*name != "provider.headers").then_some(child));+ if *name == "ingestion" {+ assert_generation_context(payload);+ }+ }+}++fn generation_request() -> HttpRequest<Body> {+ HttpRequest::post("/openai/v1/responses")+ .header("authorization", "Bearer gateway-secret")+ .header(+ "traceparent",+ "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-00",+ )+ .header("tracestate", "caller=state-canary")+ .header(+ "baggage",+ "langfuse_user_id=baggage-user,langfuse_metadata_source=python",+ )+ .header("langfuse-trace-name", "caller trace")+ .header("langfuse-user-id", "caller-user")+ .header("langfuse-session-id", "caller-session")+ .header("langfuse-tags", "one,two")+ .header("langfuse-metadata", "team:search")+ .body(Body::from(r#"{"model":"test-model"}"#))+ .unwrap()+}++fn assert_outbound_context(headers: &HeaderMap, span: Option<&opentelemetry_sdk::trace::SpanData>) {+ if let Some(span) = span {+ assert!(+ headers+ .get("tracestate")+ .is_none_or(axum::http::HeaderValue::is_empty)+ );+ assert_eq!(+ headers["traceparent"],+ format!(+ "00-{}-{}-01",+ span.span_context.trace_id(),+ span.span_context.span_id()+ )+ );+ } else {+ assert!(!headers.contains_key("traceparent"));+ assert!(!headers.contains_key("tracestate"));+ }+ for name in [+ "baggage",+ "langfuse-trace-name",+ "langfuse-user-id",+ "langfuse-session-id",+ "langfuse-tags",+ "langfuse-metadata",+ ] {+ assert!(+ !headers.contains_key(name),+ "unexpected outbound header {name}"+ );+ }+}++fn assert_generation_context(payload: &Value) {+ let span = &payload["resourceSpans"][0]["scopeSpans"][0]["spans"][0];+ assert_eq!(span["traceId"], "4bf92f3577b34da6a3ce929d0e0e4736");+ assert_eq!(span["parentSpanId"], "00f067aa0ba902b7");+ assert_eq!(span["traceState"], "caller=state-canary");+ assert_eq!(span["flags"], 1);+ let attributes: std::collections::HashMap<_, _> = span["attributes"]+ .as_array()+ .unwrap()+ .iter()+ .map(|attribute| {+ (+ attribute["key"].as_str().unwrap(),+ attribute["value"]["stringValue"].as_str().unwrap(),+ )+ })+ .collect();+ for (key, expected) in [+ ("langfuse.trace.name", "caller trace"),+ ("user.id", "caller-user"),+ ("session.id", "caller-session"),+ ("langfuse.trace.tags", r#"["one","two"]"#),+ ] {+ assert_eq!(attributes[key], expected);+ }+ let metadata: Value =+ serde_json::from_str(attributes["langfuse.observation.metadata"]).unwrap();+ assert_eq!(metadata["team"], "search");+ assert_eq!(metadata["source"], "python");+}diff --git a/ai-gateway/src/providers/openai.rs b/ai-gateway/src/providers/openai.rsindex c036702b43..9806f3211d 100644--- a/ai-gateway/src/providers/openai.rs+++ b/ai-gateway/src/providers/openai.rs@@ -140,7 +140,7 @@ impl OpenAiProvider {authorization.set_sensitive(true);let mut capture = ExecutionCapture::openai_responses(&context, headers, &body);if let Some(telemetry) = &self.telemetry {- capture.deliver_to(telemetry.clone(), &context);+ capture.deliver_to(telemetry.clone(), &context, headers);}let response = tokio::time::timeout_at(permitdiff --git a/ai-gateway/src/providers/openai/tests.rs b/ai-gateway/src/providers/openai/tests.rsindex 4bf74c65fc..f49026d650 100644--- a/ai-gateway/src/providers/openai/tests.rs+++ b/ai-gateway/src/providers/openai/tests.rs@@ -35,6 +35,14 @@ async fn preserves_opaque_bytes_and_isolates_request_and_response_headers() {"openai-project","x-api-key","x-forwarded-host",+ "traceparent",+ "tracestate",+ "baggage",+ "langfuse-trace-name",+ "langfuse-session-id",+ "langfuse-user-id",+ "langfuse-tags",+ "langfuse-metadata",] {assert!(!request.headers().contains_key(name), "forwarded {name}");}@@ -61,6 +69,17 @@ async fn preserves_opaque_bytes_and_isolates_request_and_response_headers() {("openai-project", "wrong-project"),("x-api-key", "wrong-key"),("x-forwarded-host", "attacker.example"),+ (+ "traceparent",+ "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01",+ ),+ ("tracestate", "caller=private"),+ ("baggage", "langfuse_user_id=private"),+ ("langfuse-trace-name", "private-trace"),+ ("langfuse-session-id", "private-session"),+ ("langfuse-user-id", "private-user"),+ ("langfuse-tags", "private-tag"),+ ("langfuse-metadata", "private:value"),("content-type", "application/json"),("accept-encoding", "gzip"),("connection", "accept-encoding"),@@ -382,11 +401,25 @@ async fn completed_json_and_sse_upload_once_without_waiting_for_ingestion() {.await;let relay = provider(&upstream, 1).with_telemetry(telemetry.clone());let context = resolved_request_context_with_mode("provider-secret", "full").await;+ let headers = HeaderMap::from_iter([+ (+ axum::http::HeaderName::from_static("traceparent"),+ HeaderValue::from_static("00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-00"),+ ),+ (+ axum::http::HeaderName::from_static("tracestate"),+ HeaderValue::from_static("caller=state"),+ ),+ (+ axum::http::HeaderName::from_static("langfuse-user-id"),+ HeaderValue::from_static("caller-user"),+ ),+ ]);let response = relay.forward(relay.try_admit().unwrap(),context,- &HeaderMap::new(),+ &headers,Bytes::from_static(br#"{"model":"requested","input":"hello"}"#),).await@@ -505,6 +538,20 @@ async fn cancelled_and_timed_out_executions_upload_after_provider_context_is_relfn assert_completed_upload(payload: &serde_json::Value, streaming: bool) {use serde_json::{Value, json};++ let generation = &payload["resourceSpans"][0]["scopeSpans"][0]["spans"][0];+ assert_eq!(generation["traceId"], "4bf92f3577b34da6a3ce929d0e0e4736");+ assert_eq!(generation["parentSpanId"], "00f067aa0ba902b7");+ assert_eq!(generation["traceState"], "caller=state");+ assert!(+ generation["attributes"]+ .as_array()+ .unwrap()+ .iter()+ .any(|attribute| {+ attribute["key"] == "user.id" && attribute["value"]["stringValue"] == "caller-user"+ })+ );let span = &payload["resourceSpans"][0]["scopeSpans"][0]["spans"][0];let attrs = span["attributes"].as_array().unwrap();assert!(diff --git a/ai-gateway/src/telemetry/context.rs b/ai-gateway/src/telemetry/context.rsnew file mode 100644index 0000000000..fdce39073a--- /dev/null+++ b/ai-gateway/src/telemetry/context.rs@@ -0,0 +1,277 @@+//! Caller tracing context belongs only to the recorded generation.+use std::collections::HashSet;++use axum::http::HeaderMap;+use opentelemetry::{Context, propagation::TextMapPropagator, trace::TraceContextExt};+use opentelemetry_http::HeaderExtractor;+use opentelemetry_sdk::{+ propagation::TraceContextPropagator,+ trace::{IdGenerator, RandomIdGenerator},+};+use serde::Serialize;+use serde_json::{Map, Value};++const MAX_HEADER_BYTES: usize = 8 * 1024;+const MAX_FIELD_BYTES: usize = 1024;+const MAX_ENTRIES: usize = 64;+const HEADERS: [&str; 8] = [+ "traceparent",+ "tracestate",+ "baggage",+ "langfuse-user-id",+ "langfuse-session-id",+ "langfuse-trace-name",+ "langfuse-tags",+ "langfuse-metadata",+];+const SCALARS: [(&str, &str, &str); 3] = [+ ("langfuse-user-id", "langfuse_user_id", "user.id"),+ ("langfuse-session-id", "langfuse_session_id", "session.id"),+ (+ "langfuse-trace-name",+ "langfuse_trace_name",+ "langfuse.trace.name",+ ),+];++#[derive(Serialize)]+pub(super) struct GenerationContext {+ pub trace_id: String,+ pub observation_id: String,+ pub parent_span_id: Option<String>,+ pub trace_state: String,+ pub attributes: Map<String, Value>,+ pub metadata: Map<String, Value>,+}++impl GenerationContext {+ pub fn from_headers(headers: &HeaderMap) -> Self {+ let ids = RandomIdGenerator::default();+ let mut result = Self {+ trace_id: ids.new_trace_id().to_string(),+ observation_id: ids.new_span_id().to_string(),+ parent_span_id: None,+ trace_state: String::new(),+ attributes: Map::new(),+ metadata: Map::new(),+ };+ let bytes = HEADERS+ .iter()+ .flat_map(|name| headers.get_all(*name))+ .map(|value| value.as_bytes().len())+ .sum::<usize>();+ if bytes > MAX_HEADER_BYTES {+ return result;+ }+ let mut trace_headers = HeaderMap::new();+ if let Some(value) = single_header(headers, "traceparent") {+ trace_headers.insert("traceparent", value.parse().expect("existing header value"));+ }+ // W3C tracestate is a list and may span multiple HTTP header fields.+ let trace_state = headers+ .get_all("tracestate")+ .iter()+ .filter_map(|value| value.to_str().ok())+ .collect::<Vec<_>>()+ .join(",");+ trace_headers.insert(+ "tracestate",+ trace_state.parse().expect("existing header values"),+ );+ let context = TraceContextPropagator::new()+ .extract_with_context(&Context::new(), &HeaderExtractor(&trace_headers));+ let parent = context.span();+ let parent = parent.span_context();+ if parent.is_valid() && parent.is_remote() {+ result.trace_id = parent.trace_id().to_string();+ result.parent_span_id = Some(parent.span_id().to_string());+ result.trace_state = parent.trace_state().header();+ }+ result.baggage(headers);+ for (header, _, attribute) in SCALARS {+ if let Some(value) = single_header(headers, header).and_then(|v| decode(v, false)) {+ result.attributes.insert(attribute.into(), value.into());+ }+ }+ result.tags(+ list_entries(headers, "langfuse-tags")+ .filter_map(|tag| decode(tag.trim(), false))+ .collect(),+ );+ for entry in list_entries(headers, "langfuse-metadata") {+ if let Some((key, value)) = entry.split_once(':')+ && let (Some(key), Some(value)) =+ (decode(key.trim(), false), decode(value.trim(), false))+ {+ result.metadata.insert(key, value.into());+ }+ }+ result+ }++ fn baggage(&mut self, headers: &HeaderMap) {+ // Do not use the generic baggage propagator: malformed input must never+ // be logged, and Python's exporter encodes spaces with quote_plus.+ for entry in list_entries(headers, "baggage") {+ let entry = entry.split(';').next().unwrap_or_default().trim();+ let Some((key, value)) = entry.split_once('=') else {+ continue;+ };+ let Some(key) = decode(key.trim(), true) else {+ continue;+ };+ let Some(value) = decode(value.trim(), true) else {+ continue;+ };+ if let Some((_, _, attribute)) = SCALARS.iter().find(|(_, name, _)| *name == key) {+ self.attributes.insert((*attribute).into(), value.into());+ } else if key == "langfuse_tags" {+ if let Some(tags) = parse_tags(&value) {+ self.tags(tags);+ }+ } else if let Some(key) = key.strip_prefix("langfuse_metadata_")+ && valid(key)+ {+ self.metadata.insert(key.into(), value.into());+ }+ }+ }++ fn tags(&mut self, tags: Vec<String>) {+ let mut seen = HashSet::new();+ let tags: Vec<Value> = tags+ .into_iter()+ .take(MAX_ENTRIES)+ .filter(|tag| valid(tag) && seen.insert(tag.clone()))+ .map(Value::String)+ .collect();+ if !tags.is_empty() {+ self.attributes+ .insert("langfuse.trace.tags".into(), Value::Array(tags));+ }+ }+}++fn list_entries<'a>(headers: &'a HeaderMap, name: &str) -> impl Iterator<Item = &'a str> {+ headers+ .get_all(name)+ .into_iter()+ .filter_map(|value| value.to_str().ok())+ .flat_map(|value| value.split(','))+ .take(MAX_ENTRIES)+}++fn single_header<'a>(headers: &'a HeaderMap, name: &str) -> Option<&'a str> {+ let mut values = headers.get_all(name).iter();+ let value = values.next()?;+ if values.next().is_some() {+ return None;+ }+ value.to_str().ok()+}++fn valid(value: &str) -> bool {+ !value.trim().is_empty()+ && value.len() <= MAX_FIELD_BYTES+ && !value.chars().any(char::is_control)+}++fn decode(value: &str, plus_as_space: bool) -> Option<String> {+ let mut decoded = Vec::with_capacity(value.len().min(MAX_FIELD_BYTES));+ let mut bytes = value.bytes();+ while let Some(byte) = bytes.next() {+ let byte = match byte {+ b'%' => {+ let high = char::from(bytes.next()?).to_digit(16)?;+ let low = char::from(bytes.next()?).to_digit(16)?;+ u8::try_from(high * 16 + low).ok()?+ }+ b'+' if plus_as_space => b' ',+ _ => byte,+ };+ decoded.push(byte);+ if decoded.len() > MAX_FIELD_BYTES {+ return None;+ }+ }+ let value = String::from_utf8(decoded).ok()?;+ valid(&value).then_some(value)+}++/// Python's `OTel` exporter calls `str(list)`, producing quoted Python strings.+/// Accept only a list of string literals; never evaluate expressions.+fn parse_tags(value: &str) -> Option<Vec<String>> {+ if let Ok(tags) = serde_json::from_str::<Vec<String>>(value) {+ return (tags.len() <= MAX_ENTRIES).then_some(tags);+ }+ let mut chars = value.trim().chars().peekable();+ if chars.next()? != '[' {+ return None;+ }+ let mut tags = Vec::new();+ loop {+ while chars.peek().is_some_and(|c| c.is_whitespace()) {+ chars.next();+ }+ if chars.peek() == Some(&']') {+ chars.next();+ return chars.next().is_none().then_some(tags);+ }+ if tags.len() == MAX_ENTRIES {+ return None;+ }+ let quote = chars.next()?;+ if quote != '\'' && quote != '"' {+ return None;+ }+ let mut tag = String::new();+ loop {+ let character = chars.next()?;+ if character == quote {+ break;+ }+ if character != '\\' {+ tag.push(character);+ continue;+ }+ let escape = chars.next()?;+ tag.push(match escape {+ '\\' | '\'' | '"' => escape,+ 'a' => '\u{7}',+ 'b' => '\u{8}',+ 'f' => '\u{c}',+ 'n' => '\n',+ 'r' => '\r',+ 't' => '\t',+ 'v' => '\u{b}',+ 'x' | 'u' | 'U' => {+ let digits = match escape {+ 'x' => 2,+ 'u' => 4,+ _ => 8,+ };+ let mut code = 0;+ for _ in 0..digits {+ code = code * 16 + chars.next()?.to_digit(16)?;+ }+ char::from_u32(code)?+ }+ _ => return None,+ });+ }+ tags.push(tag);+ while chars.peek().is_some_and(|c| c.is_whitespace()) {+ chars.next();+ }+ match chars.peek()? {+ ',' => {+ chars.next();+ }+ ']' => {}+ _ => return None,+ }+ }+}++#[cfg(test)]+mod tests;diff --git a/ai-gateway/src/telemetry/context/tests.rs b/ai-gateway/src/telemetry/context/tests.rsnew file mode 100644index 0000000000..354c1239b5--- /dev/null+++ b/ai-gateway/src/telemetry/context/tests.rs@@ -0,0 +1,210 @@+use axum::http::HeaderValue;+use opentelemetry::trace::{SpanContext, SpanId, TraceFlags, TraceId, TraceState};+use serde_json::json;++use super::*;++const TRACEPARENT: &str = "00-0123456789abcdef0123456789abcdef-0123456789abcdef-00";++#[test]+fn remote_context_is_generation_parent_even_when_unsampled() {+ let mut headers = HeaderMap::new();+ headers.insert("traceparent", TRACEPARENT.parse().unwrap());+ headers.insert("tracestate", "vendor=opaque-state".parse().unwrap());+ headers.append("tracestate", "second=other-state".parse().unwrap());+ let context = GenerationContext::from_headers(&headers);+ assert_eq!(context.trace_id, "0123456789abcdef0123456789abcdef");+ assert_eq!(context.parent_span_id.as_deref(), Some("0123456789abcdef"));+ assert_ne!(context.observation_id, "0123456789abcdef");+ assert_eq!(+ context.trace_state,+ "vendor=opaque-state,second=other-state"+ );+}++#[test]+fn missing_invalid_and_duplicate_parent_ignore_ambient_context() {+ let ambient = Context::new().with_remote_span_context(SpanContext::new(+ TraceId::from_hex("abcdef0123456789abcdef0123456789").unwrap(),+ SpanId::from_hex("abcdef0123456789").unwrap(),+ TraceFlags::SAMPLED,+ true,+ TraceState::default(),+ ));+ let _guard = ambient.attach();+ for parent in [+ None,+ Some("invalid"),+ Some("00-00000000000000000000000000000000-0123456789abcdef-01"),+ ] {+ let mut headers = HeaderMap::new();+ if let Some(parent) = parent {+ headers.insert("traceparent", parent.parse().unwrap());+ }+ headers.insert(+ "baggage",+ "langfuse_trace_id=abcdef0123456789abcdef0123456789"+ .parse()+ .unwrap(),+ );+ assert_fresh(&GenerationContext::from_headers(&headers));+ }+ let mut headers = HeaderMap::new();+ headers.append("traceparent", TRACEPARENT.parse().unwrap());+ headers.append("traceparent", TRACEPARENT.parse().unwrap());+ assert_fresh(&GenerationContext::from_headers(&headers));+}++fn assert_fresh(context: &GenerationContext) {+ assert!(context.parent_span_id.is_none());+ assert!(context.trace_state.is_empty());+ assert_ne!(context.trace_id, "abcdef0123456789abcdef0123456789");+ assert!(TraceId::from_hex(&context.trace_id).unwrap() != TraceId::INVALID);+ assert!(SpanId::from_hex(&context.observation_id).unwrap() != SpanId::INVALID);+}++#[test]+fn python_sdk_carrier_preserves_quotes_commas_unicode_and_metadata_suffixes() {+ let fixture: Value =+ serde_json::from_str(include_str!("../../../tests/fixtures/python-baggage.json")).unwrap();+ let mut headers = HeaderMap::new();+ for (name, value) in fixture["carrier"].as_object().unwrap() {+ headers.insert(+ name.parse::<axum::http::HeaderName>().unwrap(),+ value.as_str().unwrap().parse().unwrap(),+ );+ }+ let context = GenerationContext::from_headers(&headers);+ assert_eq!(context.attributes["user.id"], fixture["inputs"]["user_id"]);+ assert_eq!(+ context.attributes["session.id"],+ fixture["inputs"]["session_id"]+ );+ assert_eq!(+ context.attributes["langfuse.trace.name"],+ fixture["inputs"]["trace_name"]+ );+ assert_eq!(+ context.attributes["langfuse.trace.tags"],+ fixture["inputs"]["tags"]+ );+ assert_eq!(context.metadata["team_name"], "search+ranking");+ assert_eq!(+ context.metadata["request_note_with_underscores"],+ "snow雪 / 雪 + value"+ );+ assert_eq!(context.metadata["numeric_value"], "42");+ assert_eq!(context.metadata["boolean_value"], "True");+}++#[test]+fn explicit_headers_override_baggage_after_delimiter_splitting() {+ let mut headers = HeaderMap::new();+ headers.insert("baggage", "langfuse_user_id=baggage,langfuse_session_id=session,langfuse_trace_name=workflow,langfuse_tags=%5B%22baggage%22%5D,langfuse_metadata_team_name=old,langfuse_metadata_keep=value,other=ignored".parse().unwrap());+ headers.insert("langfuse-user-id", "user+name%20value".parse().unwrap());+ headers.insert("langfuse-session-id", "custom-session".parse().unwrap());+ headers.insert("langfuse-trace-name", "custom-name".parse().unwrap());+ headers.insert(+ "langfuse-tags",+ " a ,comma%2Ctag,a,plus+tag".parse().unwrap(),+ );+ headers.insert(+ "langfuse-metadata",+ "team_name:new,key%3Aname:value%2Cwith%3Acolon,broken,empty:,bad:%XX"+ .parse()+ .unwrap(),+ );+ headers.append("langfuse-tags", "second,a".parse().unwrap());+ headers.append(+ "langfuse-metadata",+ "team_name:latest,second:value".parse().unwrap(),+ );+ let context = GenerationContext::from_headers(&headers);+ assert_eq!(context.attributes["user.id"], "user+name value");+ assert_eq!(context.attributes["session.id"], "custom-session");+ assert_eq!(context.attributes["langfuse.trace.name"], "custom-name");+ assert_eq!(+ context.attributes["langfuse.trace.tags"],+ json!(["a", "comma,tag", "plus+tag", "second"])+ );+ assert_eq!(+ context.metadata,+ json!({"team_name": "latest", "keep": "value", "key:name": "value,with:colon", "second": "value"})+ .as_object()+ .unwrap()+ .clone()+ );+}++#[test]+fn malformed_fields_and_duplicate_custom_headers_keep_valid_baggage() {+ let mut headers = HeaderMap::new();+ headers.append("baggage", "bad,langfuse_user_id=caller,langfuse_metadata_good=yes;property=value,langfuse_metadata_bad=%0A,langfuse_session_id=%FF".parse().unwrap());+ headers.append(+ "baggage",+ "langfuse_trace_name=workflow,langfuse_metadata_percent=%xy"+ .parse()+ .unwrap(),+ );+ headers.append("langfuse-user-id", "first".parse().unwrap());+ headers.append("langfuse-user-id", "second".parse().unwrap());+ headers.insert("langfuse-trace-name", "bad%0Avalue".parse().unwrap());+ let context = GenerationContext::from_headers(&headers);+ assert_eq!(context.attributes["user.id"], "caller");+ assert_eq!(context.attributes["langfuse.trace.name"], "workflow");+ assert!(!context.attributes.contains_key("session.id"));+ assert_eq!(+ context.metadata,+ json!({"good": "yes"}).as_object().unwrap().clone()+ );+}++#[test]+fn oversized_headers_fields_and_entry_counts_are_bounded() {+ let mut headers = HeaderMap::new();+ headers.insert("traceparent", TRACEPARENT.parse().unwrap());+ headers.insert(+ "baggage",+ HeaderValue::from_str(&"a".repeat(MAX_HEADER_BYTES + 1)).unwrap(),+ );+ assert_fresh(&GenerationContext::from_headers(&headers));+ headers.remove("baggage");+ headers.insert(+ "langfuse-user-id",+ HeaderValue::from_str(&"a".repeat(MAX_FIELD_BYTES + 1)).unwrap(),+ );+ headers.insert(+ "langfuse-metadata",+ (0..MAX_ENTRIES + 3)+ .map(|i| format!("key{i}:value"))+ .collect::<Vec<_>>()+ .join(",")+ .parse()+ .unwrap(),+ );+ headers.append("langfuse-metadata", "overflow:value".parse().unwrap());+ let context = GenerationContext::from_headers(&headers);+ assert!(!context.attributes.contains_key("user.id"));+ assert_eq!(context.metadata.len(), MAX_ENTRIES);+ assert!(!context.metadata.contains_key("overflow"));+ assert_eq!(context.parent_span_id.as_deref(), Some("0123456789abcdef"));+}++#[test]+fn python_string_parser_accepts_escapes_but_no_expressions() {+ assert_eq!(+ parse_tags(r"['\x41', '\u00e9', '\U0001f600']"),+ Some(vec!["A".into(), "é".into(), "😀".into()])+ );+ for invalid in [+ "[1]",+ "['a' + 'b']",+ "[__import__('os')]",+ "['unterminated]",+ r"['\uD800']",+ r"['\Uffffffff']",+ "['a'] trailing",+ ] {+ assert!(parse_tags(invalid).is_none(), "{invalid}");+ }+}diff --git a/ai-gateway/src/telemetry/mapping.rs b/ai-gateway/src/telemetry/mapping.rsindex 7966dcec9b..a7036d34c7 100644--- a/ai-gateway/src/telemetry/mapping.rs+++ b/ai-gateway/src/telemetry/mapping.rs@@ -2,14 +2,30 @@use chrono::{DateTime, SecondsFormat};use serde_json::{Map, Value, json};+use super::context::GenerationContext;use crate::capture::{InferenceFacts, RelayOutcome};-pub(super) fn span(facts: InferenceFacts, trace_id: &str, observation_id: &str) -> Value {+pub(super) fn span(facts: InferenceFacts, context: &GenerationContext) -> Value {let full = facts.metadata.get("ingestion_mode").and_then(Value::as_str) == Some("full");let (level, message) = observation_status(&facts);- let metadata = generation_metadata(&facts);+ let mut metadata: Map<String, Value> = context+ .metadata+ .iter()+ .filter(|(key, _)| !reserved_metadata(key))+ .map(|(key, value)| (key.clone(), value.clone()))+ .collect();+ // Trusted key attribution and gateway facts take precedence over caller metadata.+ metadata.extend(generation_metadata(&facts));let inference = facts.inference;let mut attributes = vec![attribute("langfuse.observation.type", "generation")];+ attributes.extend(context.attributes.iter().map(|(key, value)| {+ attribute(+ key,+ value+ .as_str()+ .map_or_else(|| value.to_string(), str::to_owned),+ )+ }));if let Some(model) = inference.model.or(inference.requested_model) {attributes.push(attribute("langfuse.observation.model.name", model));}@@ -54,16 +70,37 @@ pub(super) fn span(facts: InferenceFacts, trace_id: &str, observation_id: &str)"langfuse.observation.metadata",Value::Object(metadata).to_string(),));- json!({- "traceId": trace_id,- "spanId": observation_id,+ let mut span = json!({+ "traceId": context.trace_id,+ "spanId": context.observation_id,+ // Generation capture is independent of the caller's sampling decision.+ "flags": 1,"name": facts.api_format,"kind": 3,"startTimeUnixNano": facts.start_time_unix_ms.saturating_mul(1_000_000).to_string(),"endTimeUnixNano": facts.start_time_unix_ms.saturating_add(facts.duration_ms).saturating_mul(1_000_000).to_string(),"attributes": attributes,"status": { "code": if level == "ERROR" { 2 } else { 0 } },- })+ });+ if let Some(parent) = &context.parent_span_id {+ span["parentSpanId"] = json!(parent);+ }+ if !context.trace_state.is_empty() {+ span["traceState"] = json!(context.trace_state);+ }+ span+}++fn reserved_metadata(key: &str) -> bool {+ [+ "langfuse.gateway",+ "http_status",+ "scope",+ "resourceAttributes",+ "attributes",+ ]+ .iter()+ .any(|reserved| key == *reserved || key.starts_with(&format!("{reserved}.")))}fn generation_metadata(facts: &InferenceFacts) -> Map<String, Value> {@@ -75,16 +112,7 @@ fn generation_metadata(facts: &InferenceFacts) -> Map<String, Value> {{for (key, value) in attribution {// Attribution remains searchable, but cannot impersonate gateway or OTEL fields.- if ![- "langfuse.gateway",- "http_status",- "scope",- "resourceAttributes",- "attributes",- ]- .iter()- .any(|reserved| key == reserved || key.starts_with(&format!("{reserved}.")))- {+ if !reserved_metadata(key) {metadata.insert(key.clone(), value.clone());}metadata.insert(@@ -196,6 +224,14 @@ fn openai_usage(usage: &Value) -> Option<Value> {mod tests {use super::*;+ fn context() -> GenerationContext {+ GenerationContext {+ trace_id: "0123456789abcdef0123456789abcdef".into(),+ observation_id: "0123456789abcdef".into(),+ ..GenerationContext::from_headers(&axum::http::HeaderMap::new())+ }+ }+fn facts() -> InferenceFacts {let mut facts = InferenceFacts {api_format: "openai.responses",@@ -246,12 +282,51 @@ mod tests {}#[test]- fn full_generation_preserves_native_content_attribution_and_completion_time() {- let span = span(- facts(),- "0123456789abcdef0123456789abcdef",- "0123456789abcdef",+ fn caller_attributes_and_metadata_cannot_replace_trusted_gateway_fields() {+ let mut context = context();+ context.attributes = serde_json::from_value(json!({+ "user.id": "user", "session.id": "session",+ "langfuse.trace.name": "caller workflow", "langfuse.trace.tags": ["tag,one", "tag-two"]+ }))+ .unwrap();+ context.metadata = serde_json::from_value(json!({+ "custom": "value", "team": "untrusted", "http_status": "fake",+ "langfuse.gateway.project_id": "wrong-project",+ "langfuse.gateway.api-key.id": "wrong-key",+ "langfuse.gateway.api-key.metadata.team": "wrong-team",+ "langfuse.gateway.provider.request.fake": "fake",+ "scope": "fake", "resourceAttributes.secret": "fake", "attributes": "fake"+ }))+ .unwrap();+ let attrs = attributes(&span(facts(), &context));+ assert_eq!(attrs["user.id"], "user");+ assert_eq!(attrs["session.id"], "session");+ assert_eq!(attrs["langfuse.trace.name"], "caller workflow");+ assert_eq!(+ serde_json::from_str::<Value>(attrs["langfuse.trace.tags"].as_str().unwrap()).unwrap(),+ json!(["tag,one", "tag-two"]));+ let metadata = metadata(&attrs);+ assert_eq!(metadata["custom"], "value");+ assert_eq!(metadata["team"], "search");+ assert_eq!(metadata["http_status"], 200);+ assert_eq!(metadata["langfuse.gateway.project_id"], "project");+ assert_eq!(metadata["langfuse.gateway.api-key.id"], "key");+ assert_eq!(metadata["langfuse.gateway.api-key.metadata.team"], "search");+ for key in [+ "scope",+ "resourceAttributes.secret",+ "attributes",+ "langfuse.gateway.provider.request.fake",+ ] {+ assert!(metadata.get(key).is_none(), "caller forged {key}");+ }+ assert!(!attrs.contains_key("langfuse.trace.metadata"));+ }++ #[test]+ fn full_generation_preserves_native_content_attribution_and_completion_time() {+ let span = span(facts(), &context());assert_eq!(span["traceId"], "0123456789abcdef0123456789abcdef");assert_eq!(span["spanId"], "0123456789abcdef");assert!(span.get("parentSpanId").is_none());@@ -302,7 +377,7 @@ mod tests {"attributes": "spoofed-attributes"});facts.inference.provider_response_id = Some("response".into());- let attrs = attributes(&span(facts, "trace", "span"));+ let attrs = attributes(&span(facts, &context()));let metadata = metadata(&attrs);assert_eq!(metadata["team"], "search");assert_eq!(metadata["enabled"], true);@@ -348,7 +423,7 @@ mod tests {facts.metadata["ingestion_mode"] = json!("usage");facts.inference.model = None;facts.completion_start_ms = None;- let attrs = attributes(&span(facts, "trace", "span"));+ let attrs = attributes(&span(facts, &context()));for key in ["input", "output", "usage_details", "completion_start_time"] {assert!(!attrs.contains_key(&format!("langfuse.observation.{key}")));}@@ -366,7 +441,7 @@ mod tests {"future_usage": {"cost": 123},});facts.inference.usage_details = Some(usage.clone());- let attrs = attributes(&span(facts, "trace", "span"));+ let attrs = attributes(&span(facts, &context()));assert!(metadata(&attrs).get("native_usage").is_none());let projected: Value = serde_json::from_str(attrs["langfuse.observation.usage_details"]@@ -388,7 +463,7 @@ mod tests {fn unsupported_usage_omits_usage_without_invented_totals() {let mut facts = facts();facts.inference.usage_details = Some(json!({"input_tokens": 9}));- let attrs = attributes(&span(facts, "trace", "span"));+ let attrs = attributes(&span(facts, &context()));assert!(!attrs.contains_key("langfuse.observation.usage_details"));assert!(metadata(&attrs).get("native_usage").is_none());}@@ -403,7 +478,7 @@ mod tests {"user": "legacy-user"})).unwrap();- let attrs = attributes(&span(facts, "trace", "span"));+ let attrs = attributes(&span(facts, &context()));let metadata = metadata(&attrs);assert_eq!(metadata["langfuse.gateway.provider.request.metadata"],@@ -447,7 +522,7 @@ mod tests {facts.http_status = Some(status);facts.inference.provider_status = provider_status.map(str::to_owned);facts.inference.error_message = error.map(str::to_owned);- let attrs = attributes(&span(facts, "trace", "span"));+ let attrs = attributes(&span(facts, &context()));assert_eq!(attrs["langfuse.observation.status_message"], expected);assert_eq!(attrs["langfuse.observation.level"], "ERROR");}@@ -473,7 +548,7 @@ mod tests {facts.http_status = http_status;facts.inference.provider_status = provider_status.map(str::to_owned);facts.outcome = outcome;- let span = span(facts, "trace", "span");+ let span = span(facts, &context());let attrs = attributes(&span);assert_eq!(attrs["langfuse.observation.level"], level);assert_eq!(span["status"]["code"], code);diff --git a/ai-gateway/src/telemetry/mod.rs b/ai-gateway/src/telemetry/mod.rsindex 6e624c47ec..9b6c04be1c 100644--- a/ai-gateway/src/telemetry/mod.rs+++ b/ai-gateway/src/telemetry/mod.rs@@ -1,4 +1,5 @@//! Immediate, best-effort delivery of finalized inference facts.+mod context;mod mapping;mod otlp;@@ -10,15 +11,16 @@ use std::{},};+use axum::http::HeaderMap;use opentelemetry::trace::{FutureExt, TraceContextExt};use tokio::{sync::Semaphore, task::JoinSet, time::Instant};use tracing_opentelemetry::OpenTelemetrySpanExt;-use uuid::Uuid;use crate::{capture::InferenceFacts,resolution::{ControlPlaneConfig, ResolutionError, ResolvedRequestContext},};+use context::GenerationContext;use otlp::Uploader;const MAX_UPLOADS: usize = 32;@@ -31,18 +33,16 @@ pub(crate) struct DeliveryContext {pub project_id: String,pub access_token: String,pub expires_at: u64,- trace_id: String,- observation_id: String,+ generation: GenerationContext,}impl DeliveryContext {- pub fn from_resolved(context: &ResolvedRequestContext) -> Self {+ pub fn from_resolved(context: &ResolvedRequestContext, headers: &HeaderMap) -> Self {Self {project_id: context.attribution().project_id().to_owned(),access_token: context.ingestion().access_token().to_owned(),expires_at: context.ingestion().expires_at(),- trace_id: Uuid::new_v4().simple().to_string(),- observation_id: Uuid::new_v4().simple().to_string()[..16].to_owned(),+ generation: GenerationContext::from_headers(headers),}}}@@ -95,7 +95,9 @@ impl Telemetry {bytes: context.access_token.len() + context.project_id.len(),limit: MAX_FACT_BYTES,};- if serde_json::to_writer(&mut size, &facts).is_err() {+ if serde_json::to_writer(&mut size, &context.generation).is_err()+ || serde_json::to_writer(&mut size, &facts).is_err()+ {self.drop_record("size");return;}@@ -122,7 +124,7 @@ impl Telemetry {async move {let _capacity = capacity;let _bytes = bytes;- let span = mapping::span(facts, &context.trace_id, &context.observation_id);+ let span = mapping::span(facts, &context.generation);match uploader.export(&context, &[span]).await {Ok(()) => {stats.accepted.fetch_add(1, Ordering::Relaxed);diff --git a/ai-gateway/src/telemetry/tests.rs b/ai-gateway/src/telemetry/tests.rsindex 116c2788e5..826399af99 100644--- a/ai-gateway/src/telemetry/tests.rs+++ b/ai-gateway/src/telemetry/tests.rs@@ -29,7 +29,10 @@ fn facts(project: &str) -> InferenceFacts {}async fn grant() -> DeliveryContext {- DeliveryContext::from_resolved(&resolved_request_context("provider-secret").await)+ DeliveryContext::from_resolved(+ &resolved_request_context("provider-secret").await,+ &HeaderMap::new(),+ )}fn uploader(url: &str) -> Uploader {@@ -287,14 +290,18 @@ async fn retained_byte_budget_is_released_after_successful_drain() {let grant = grant().await;let budget = serde_json::to_vec(&facts("project-1")).unwrap().len()+ grant.access_token.len()- + grant.project_id.len();+ + grant.project_id.len()+ + serde_json::to_vec(&grant.generation).unwrap().len();let telemetry = Telemetry::with_uploader(uploader(&web.url), 2, budget);telemetry.record(grant, facts("project-1"));tokio::time::timeout(Duration::from_secs(2), started.notified()).await.unwrap();assert_eq!(telemetry.0.bytes.available_permits(), 0);- let second = DeliveryContext::from_resolved(&resolved_request_context("provider-secret").await);+ let second = DeliveryContext::from_resolved(+ &resolved_request_context("provider-secret").await,+ &HeaderMap::new(),+ );telemetry.record(second, facts("project-1"));assert_eq!(telemetry.0.stats.dropped.load(Ordering::Relaxed), 1);release.notify_one();diff --git a/ai-gateway/tests/fixtures/python-baggage.json b/ai-gateway/tests/fixtures/python-baggage.jsonnew file mode 100644index 0000000000..316e63afc7--- /dev/null+++ b/ai-gateway/tests/fixtures/python-baggage.json@@ -0,0 +1,36 @@+{+ "python": "3.14.3",+ "versions": {+ "langfuse": "4.15.0",+ "opentelemetry-api": "1.40.0",+ "opentelemetry-sdk": "1.40.0"+ },+ "inputs": {+ "user_id": "user+name / snow雪",+ "session_id": "session_with_underscore + value",+ "trace_name": "Gateway \"quoted\" workflow / 雪",+ "tags": [+ "plain",+ "comma,tag",+ "double\"quote",+ "single'quote",+ "both'\"quotes",+ "space tag",+ "plus+tag",+ "snow雪",+ "雪",+ "back\\slash"+ ],+ "metadata": {+ "team_name": "search+ranking",+ "request_note_with_underscores": "snow雪 / 雪 + value",+ "numeric_value": 42,+ "boolean_value": true+ }+ },+ "carrier": {+ "traceparent": "00-0123456789abcdef0123456789abcdef-0123456789abcdef-01",+ "tracestate": "vendor=opaque-state",+ "baggage": "langfuse_user_id=user%2Bname+%2F+snow%E9%9B%AA,langfuse_session_id=session_with_underscore+%2B+value,langfuse_tags=%5B%27plain%27%2C+%27comma%2Ctag%27%2C+%27double%22quote%27%2C+%22single%27quote%22%2C+%27both%5C%27%22quotes%27%2C+%27space+tag%27%2C+%27plus%2Btag%27%2C+%27snow%E9%9B%AA%27%2C+%27%E9%9B%AA%27%2C+%27back%5C%5Cslash%27%5D,langfuse_trace_name=Gateway+%22quoted%22+workflow+%2F+%E9%9B%AA,langfuse_metadata_team_name=search%2Branking,langfuse_metadata_request_note_with_underscores=snow%E9%9B%AA+%2F+%E9%9B%AA+%2B+value,langfuse_metadata_numeric_value=42,langfuse_metadata_boolean_value=True"+ }+}diff --git a/ai-gateway/tests/fixtures/python-baggage.py b/ai-gateway/tests/fixtures/python-baggage.pynew file mode 100644index 0000000000..31d89a81ff--- /dev/null+++ b/ai-gateway/tests/fixtures/python-baggage.py@@ -0,0 +1,41 @@+"""Regenerate with Langfuse 4.15.0 and opentelemetry-api/sdk 1.40.0 installed."""++import importlib.metadata+import json+import platform+from pathlib import Path++from langfuse import propagate_attributes+from opentelemetry.baggage.propagation import W3CBaggagePropagator+from opentelemetry.propagators.composite import CompositePropagator+from opentelemetry.trace import NonRecordingSpan, SpanContext, TraceFlags, TraceState, use_span+from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator++inputs = {+ 'user_id': 'user+name / snow雪',+ 'session_id': 'session_with_underscore + value',+ 'trace_name': 'Gateway "quoted" workflow / 雪',+ 'tags': ['plain', 'comma,tag', 'double"quote', "single'quote", 'both\'"quotes', 'space tag', 'plus+tag', 'snow雪', '雪', 'back\\slash'],+ 'metadata': {'team_name': 'search+ranking', 'request_note_with_underscores': 'snow雪 / 雪 + value', 'numeric_value': 42, 'boolean_value': True},+}+propagator = CompositePropagator([TraceContextTextMapPropagator(), W3CBaggagePropagator()])+span_context = SpanContext(+ trace_id=int('0123456789abcdef0123456789abcdef', 16),+ span_id=int('0123456789abcdef', 16),+ is_remote=False,+ trace_flags=TraceFlags(TraceFlags.SAMPLED),+ trace_state=TraceState([('vendor', 'opaque-state')]),+)+carrier = {}+with use_span(NonRecordingSpan(span_context)):+ with propagate_attributes(**inputs, as_baggage=True):+ propagator.inject(carrier)+result = {+ 'python': platform.python_version(),+ 'versions': {p: importlib.metadata.version(p) for p in ['langfuse', 'opentelemetry-api', 'opentelemetry-sdk']},+ 'inputs': inputs,+ 'carrier': carrier,+}+output = json.dumps(result, ensure_ascii=False, indent=2)+Path(__file__).with_suffix('.json').write_text(output + '\n')+print(output)diff --git a/ai-gateway/tests/observability.rs b/ai-gateway/tests/observability.rsindex 22d1acc8e9..1dff2eff4e 100644--- a/ai-gateway/tests/observability.rs+++ b/ai-gateway/tests/observability.rs@@ -7,6 +7,7 @@ use opentelemetry_proto::tonic::collector::{use opentelemetry_proto::tonic::{common::v1::any_value::Value as AttributeValue,metrics::v1::{ResourceMetrics, metric::Data},+ trace::v1::ResourceSpans,};use prost::Message;use serde_json::Value;@@ -83,6 +84,9 @@ impl Gateway {u8::from(sampled)),)+ .header("tracestate", "caller=state-canary")+ .header("baggage", "langfuse_user_id=baggage-canary")+ .header("langfuse-session-id", "session-canary").header("authorization", "Bearer secret-auth-token").body(r#"{"input":"secret-prompt-content"}"#).send()@@ -176,38 +180,9 @@ async fn exports_otlp_traces_metrics_and_correlated_content_free_logs() {.collect();assert!(!traces.is_empty(), "traces must flush on shutdown");assert!(!metrics.is_empty(), "metrics must flush on shutdown");- let spans: Vec<_> = traces- .iter()- .flat_map(|resource| &resource.scope_spans)- .flat_map(|scope| &scope.spans)- .collect();- assert_eq!(spans.len(), 1);- assert_eq!(spans[0].name, "POST /openai/v1/responses");- assert_eq!(- spans[0].parent_span_id,- [0x00, 0xf0, 0x67, 0xaa, 0x0b, 0xa9, 0x02, 0xb7]- );- assert_eq!(spans[0].kind, 2);- assert!(- spans[0].events.is_empty(),- "log events must not be exported as span events"- );let logs = gateway.logs.lock().unwrap();- let summaries: Vec<_> = logs- .iter()- .filter(|value| value["message"] == "gateway response started")- .collect();- assert_eq!(summaries.len(), 2);- assert_eq!(summaries[0]["trace_id"], "4bf92f3577b34da6a3ce929d0e0e4736");- assert_eq!(- summaries[0]["span_id"].as_str().unwrap(),- opentelemetry::trace::SpanId::from_bytes(spans[0].span_id.as_slice().try_into().unwrap())- .to_string()- );- assert_eq!(summaries[0]["status"], 503);+ assert_operational_traces(&traces, &logs);let serialized = serde_json::to_string(&*logs).unwrap();- assert!(!serialized.contains("secret-auth-token"));- assert!(!serialized.contains("secret-prompt-content"));let exported = format!("{traces:?}{metrics:?}");for secret in ["secret-auth-token",@@ -215,6 +190,9 @@ async fn exports_otlp_traces_metrics_and_correlated_content_free_logs() {"query-canary","host-canary","scheme-canary",+ "state-canary",+ "baggage-canary",+ "session-canary",] {assert!(!exported.contains(secret));assert!(!serialized.contains(secret));@@ -222,6 +200,49 @@ async fn exports_otlp_traces_metrics_and_correlated_content_free_logs() {assert_http_metrics(&metrics);}+fn assert_operational_traces(traces: &[ResourceSpans], logs: &[Value]) {+ let spans: Vec<_> = traces+ .iter()+ .flat_map(|resource| &resource.scope_spans)+ .flat_map(|scope| &scope.spans)+ .collect();+ assert_eq!(+ spans.len(),+ 2,+ "incoming sampling must not control operational tracing"+ );+ assert_ne!(spans[0].trace_id, spans[1].trace_id);+ for span in &spans {+ assert_eq!(span.name, "POST /openai/v1/responses");+ assert!(span.parent_span_id.is_empty());+ assert!(span.trace_state.is_empty());+ assert_eq!(span.kind, 2);+ assert!(+ span.events.is_empty(),+ "log events must not be exported as span events"+ );+ }+ let summaries: Vec<_> = logs+ .iter()+ .filter(|value| value["message"] == "gateway response started")+ .collect();+ assert_eq!(summaries.len(), 2);+ for (summary, span) in summaries.iter().zip(&spans) {+ assert_ne!(summary["trace_id"], "4bf92f3577b34da6a3ce929d0e0e4736");+ assert_eq!(+ summary["trace_id"],+ opentelemetry::trace::TraceId::from_bytes(span.trace_id.as_slice().try_into().unwrap())+ .to_string()+ );+ assert_eq!(+ summary["span_id"],+ opentelemetry::trace::SpanId::from_bytes(span.span_id.as_slice().try_into().unwrap())+ .to_string()+ );+ assert_eq!(summary["status"], 503);+ }+}+fn assert_http_metrics(metrics: &[ResourceMetrics]) {let http_duration = metrics.iter()