The only tool you need to know what is happening and how to fix it.
Go alertingexception-trackerissue-trackinglogsmetricsmonitoringobservability-platformopen-telemetryotelsession-replaytelemetrytraces
docs updated
ddux9eff3534d2de
diff --git a/docs/pages/client/otel/traces.mdx b/docs/pages/client/otel/traces.mdxindex 43f604e8..c363b171 100644--- a/docs/pages/client/otel/traces.mdx+++ b/docs/pages/client/otel/traces.mdx@@ -4,16 +4,20 @@ Traceway ingests OpenTelemetry spans via the OTLP/HTTP protocol and maps them to## Span Mapping-How OTel spans are classified in Traceway depends on whether the span is a root span and its kind:+Spans are classified by **kind and attributes**, not by whether they're a root span. Both root and child spans can be promoted to a top-level concept when the signal is there — a queued job processed as a child of the dispatcher's trace still becomes a Task, an LLM call made inside a request handler still becomes an AI Trace.-| OTel Span | Condition | Traceway Concept |-|-----------|-----------|-----------------|-| Root span | `SpanKind = SERVER` with HTTP attributes | **Endpoint** |-| Root span | All other kinds | **Task** |-| Non-root span | Has a parent span ID | **Span** |-| Exception event | Event named `"exception"` on any span | **Issue** |+| Condition | Traceway Concept |+|-----------|-----------------|+| `SpanKind = SERVER` / `INTERNAL` with HTTP attributes, and the span is a root (or its parent is in another process — a cross-service inbound hop) | **Endpoint** |+| `SpanKind = CONSUMER` (queue / messenger workers) | **Task** |+| Root `INTERNAL` span with a `console.command` attribute (artisan commands, console runners) | **Task** |+| Any span with `gen_ai.*` attributes (root or child of a request/task) | **AI Trace** |+| Any other span with a parent | **Span** (waterfall row under its nearest enclosing entity) |+| Event named `"exception"` on any span | **Issue** (attributed to the owning entity) |-A root span is one with no `parentSpanId`.+Each Endpoint / Task / AI Trace row carries an `is_root` flag. Non-root rows surface a **Non-root** chip in the dashboard list so you can tell at a glance which runs were triggered by another trace, and a **View distributed trace** link on the detail page jumps you to the full waterfall (endpoint → producer → consumer, request → child LLM call, etc.). The `distributed_trace_id` linking the rows is auto-derived from the OTel `trace_id`, so cross-trace lookup works without any vendor attribute.++> **Sub-handler spans are not duplicated.** An in-process child SERVER/INTERNAL+HTTP span (e.g. a framework's "handler /path" sub-span) is kept in the Spans waterfall — Endpoints are only promoted for entry points (root or cross-service hop).## Resource Attributes@@ -87,7 +91,7 @@ const sdk = new NodeSDK({sdk.start();```-This automatically instruments HTTP servers, database clients, and other libraries. Root SERVER spans become Endpoints in Traceway, child spans become Spans, and any exceptions recorded as span events become Issues.+This automatically instruments HTTP servers, database clients, and other libraries. SERVER spans become Endpoints, CONSUMER spans become Tasks, any span with `gen_ai.*` attributes becomes an AI Trace (root or child), and exception events become Issues.## Example: Python with OTel SDKdiff --git a/docs/pages/client/symfony/tasks.mdx b/docs/pages/client/symfony/tasks.mdxindex 8eea0fa0..ed1d0551 100644--- a/docs/pages/client/symfony/tasks.mdx+++ b/docs/pages/client/symfony/tasks.mdx@@ -4,14 +4,18 @@ The bundle automatically traces Symfony Messenger messages. Each consumed messag## Setup-Add `messenger_root_spans: true` to your bundle config so consumed messages appear as independent tasks (not children of the HTTP request that dispatched them):+Nothing extra to configure — every consumed message is traced automatically as soon as the bundle is installed.++Symfony Messenger's default behavior is to parent the consumer span to the dispatcher's trace (so the request that dispatched the message and the worker that processed it appear as one continuous distributed trace). Traceway captures these as Tasks regardless of whether they're root spans, so the consumed message shows up in the Tasks page either way — flagged with a **Non-root** chip when it was triggered from another trace.++If you'd rather have each consumed message start its own independent trace, opt in:```yaml filename="config/packages/open_telemetry.yaml"open_telemetry:messenger_root_spans: true```-That's it — every consumed message is now traced automatically.+This is mostly a presentation choice. With `messenger_root_spans: true`, the consumer is a root Task (no chip) and the cross-trace link to the dispatcher is broken. Without it, the consumer is a non-root Task and the dispatcher → consumer relationship is preserved in the distributed-trace view.## What Gets Captured@@ -77,4 +81,4 @@ class PaymentProcessorHandler## Dispatch Tracing-The bundle also traces the dispatch side — when your controller or service dispatches a message, a PRODUCER span is created. W3C trace context is injected into the message envelope so the consumer span links back to the dispatching trace (unless `messenger_root_spans` is `true`, in which case the consumer creates an independent root span).+The bundle also traces the dispatch side — when your controller or service dispatches a message, a PRODUCER span is created. W3C trace context is injected into the message envelope so the consumer span links back to the dispatching trace (unless `messenger_root_spans` is `true`, in which case the consumer creates an independent root span). Either way, Traceway captures the consumer as a Task and surfaces the cross-trace link when one exists.diff --git a/docs/pages/learn/ai-tracing.mdx b/docs/pages/learn/ai-tracing.mdxindex f6376849..ecc46c4f 100644--- a/docs/pages/learn/ai-tracing.mdx+++ b/docs/pages/learn/ai-tracing.mdx@@ -35,14 +35,15 @@ Every AI trace records:## How It Works-AI traces are ingested via the [OpenTelemetry](/client/otel) OTLP/HTTP endpoint. AI providers that support OTLP trace export (like [OpenRouter](/client/openrouter)) send spans with `gen_ai.*` semantic convention attributes. Traceway detects these attributes and stores them as AI traces.+AI traces are ingested via the [OpenTelemetry](/client/otel) OTLP/HTTP endpoint. Any span carrying `gen_ai.*` semantic convention attributes is promoted to an AI trace — **regardless of whether it's a root span or a child** of an HTTP request, queue job, or scheduled task. This matters in practice: most LLM calls happen inside a request handler, so the gen_ai span is a child of a SERVER span. Traceway captures both.The data flow:-1. **AI provider sends OTLP span** — The span contains `gen_ai.request.model`, `gen_ai.usage.input_tokens`, `gen_ai.usage.total_cost`, and other standardized attributes-2. **Traceway parses gen_ai attributes** — The OTEL trace converter detects spans with `gen_ai.*` attributes and extracts structured fields-3. **Conversation content stored separately** — The full prompt and completion are stored in object storage (S3 or local filesystem) to keep the database lean-4. **Dashboard shows aggregated view** — AI traces are grouped by trace name (your agent or workflow identifier), with drill-down to individual calls+1. **OTLP span arrives** — From an AI provider broadcast (e.g. [OpenRouter](/client/openrouter)) or your own SDK instrumentation. The span contains `gen_ai.request.model`, `gen_ai.usage.input_tokens`, `gen_ai.usage.total_cost`, and other standardized attributes.+2. **Traceway promotes the span by attributes, not by position** — If a span has `gen_ai.*` attributes, it lands in the AI Traces table. Root spans use the OTel trace_id as their id; non-root child spans use their own span_id, so a request handler and the LLM call it made each get their own row without collision.+3. **Conversation content stored separately** — The full prompt and completion are stored in object storage (S3 or local filesystem) to keep the database lean.+4. **Linked across the trace** — `distributed_trace_id` is auto-derived from the OTel trace_id, so a child AI trace and its parent endpoint show up together in the distributed-trace view with no extra configuration.+5. **Dashboard shows aggregated view** — AI traces are grouped by trace name (your agent or workflow identifier), with drill-down to individual calls. Groups containing non-root calls are flagged with a **Non-root** chip.## Three-Level Drill-Down