The Open Framework for Building Agents
fix(eve): preserve agent span names in Datadog exports (#3141)
Chad Hietala9cb899df3fc5
diff --git a/.changeset/clear-exported-agent-traces.md b/.changeset/clear-exported-agent-traces.mdnew file mode 100644index 000000000..60e87326a--- /dev/null+++ b/.changeset/clear-exported-agent-traces.md@@ -0,0 +1,5 @@+---+"eve": patch+---++Add explicit Datadog operation and resource names to framework-owned agent spans while preserving their OpenTelemetry names and trace hierarchy. Agent invocation spans now include `agent.turn.outcome` so completion, failure, and cancellation remain queryable in backends that discard span events.diff --git a/docs/guides/instrumentation.md b/docs/guides/instrumentation.mdindex b4458d1e7..4b69a3133 100644--- a/docs/guides/instrumentation.md+++ b/docs/guides/instrumentation.md@@ -97,6 +97,45 @@ retry behavior. Best-effort activity delivery keeps its single`[eve:execution.activity-submit] activity sink request failed` warning and doesnot mark the active span as failed.+## Exported span names and outcomes++The built-in OpenTelemetry provider preserves each span's OTel name and adds+`operation.name` and `resource.name` for Datadog's operation/resource mapping:++| OTel span name | `operation.name` | `resource.name` |+| ------------------------------------------------------------------------------------------------------------------ | ----------------- | ---------------------- |+| `invoke_agent weather` | `invoke_agent` | `invoke_agent weather` |+| `execute_tool search` | `execute_tool` | `execute_tool search` |+| `chat <model>` | `chat` | `chat <model>` |+| `agent.session`, `agent.step`, `agent.action`, `agent.approval`, `agent.channel.delivery`, `agent.channel.request` | Same as span name | Same as span name |++These attributes apply to eve-owned spans in local tracing and the+[instrumentation provider layout](./instrumentation-providers). They do not+rename Workflow, AI SDK, or other third-party spans, or change trace IDs,+parenting, sampling, or session grouping. Legacy `instrumentation.ts` setup+still owns its exporter configuration and [authored trace+hierarchy](#authored-trace-hierarchy).++In Datadog APM, `operation_name:invoke_agent resource_name:"invoke_agent weather"`+selects named agent invocations. Operation-based dashboards and monitors may+need to replace inferred names such as `otel.span` with these explicit names.+Keep session IDs, turn IDs, and message content in attributes, not operation or+resource names.++If an eve-owned span still appears as `otel.span`, inspect its exported OTel+name, `operation.name`, and `resource.name` before and after any drain or+Collector transforms. Datadog's [operation-name mapping+guide](https://docs.datadoghq.com/opentelemetry/migrate/migrate_operation_names/)+describes the explicit override. Exported fields must survive the ingestion+path; a local export does not prove that a hosted backend retained them.++Agent invocation spans carry `agent.turn.outcome=completed|failed|cancelled`+when the turn has a terminal outcome. A failed invocation also has OTel error+status; cancellation is not an error. The scalar outcome survives backends+that discard span events, including Sentry's [direct OTLP+intake](https://docs.sentry.io/concepts/otlp/direct/traces/). These attributes+remain available when model and tool content is redacted.+## Runtime context_Runtime context_ is an [AI SDK concept](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text): a user-defined object that flows through a generation lifecycle. eve exposes it through `events["step.started"]`, a callback that runs once eve has assembled the model input for an attempt and returns `{ runtimeContext }`. Because eve registers the AI SDK's OpenTelemetry integration with runtime context enabled, those returned values ride onto the model-call span and its children. The field is named `runtimeContext`, not `metadata`, because AI SDK v7 carries per-call attributes on runtime context rather than a dedicated metadata field.@@ -156,7 +195,7 @@ ai.eve.turn {eve.session.id}eve creates the `ai.eve.turn` parent span per turn and passes enriched telemetry to the AI SDK so model calls and tool executions are traced automatically. The AI SDK's OpenTelemetry integration names these spans after the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/), so backends that understand `gen_ai.operation.name` can classify them without extra configuration. The `invoke_agent` span is named after the model; the agent name is on its `gen_ai.agent.name` attribute.-This hierarchy applies when eve passes telemetry to the AI SDK. When the `otel()` provider layout is declared and eve owns the agent spans, eve names the span `invoke_agent <agent>` and emits no step spans. Session, turn, step, and channel context is injected as the framework half of the runtime context (`eve.version`, `eve.session.id`, `eve.environment`, `eve.turn.id`, `eve.turn.sequence`, `eve.step.index`, `eve.channel.kind`) and rides onto the spans alongside any values your `events["step.started"]` callback returns under `runtimeContext`.+This hierarchy applies when eve passes telemetry to the AI SDK. When the `otel()` provider layout is declared and eve owns the agent spans, eve names its invocation span `invoke_agent <agent>` and its model-attempt spans `agent.step`. Session, turn, step, and channel context is injected as the framework half of the runtime context (`eve.version`, `eve.session.id`, `eve.environment`, `eve.turn.id`, `eve.turn.sequence`, `eve.step.index`, `eve.channel.kind`) and rides onto the spans alongside any values your `events["step.started"]` callback returns under `runtimeContext`.Set `traceChannelRequests: true` on `defineInstrumentation` to also wrap each inbound channel HTTP request in a single OpenTelemetry `SERVER` span named for the registered route, which parents the turn tree above (and any `hook.resume` and outgoing HTTP spans):diff --git a/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.ts b/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.tsindex 2988faaaf..7d053ed31 100644--- a/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.ts+++ b/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.ts@@ -94,6 +94,7 @@ export default cases.map(({ parentActive, steering, description }) =>),);+ const interruptedParent = parentActive ? parent : undefined;parent = await parent.session.start(steering? "Actually, use STEERED instead of ORIGINAL."@@ -102,16 +103,13 @@ export default cases.map(({ parentActive, steering, description }) =>);await t.require(parent.sessionId, equals(sessionId));- if (parentActive) {- // start() observes from the current cursor, including the turn- // cancelled by this steering message before its replacement begins.- const cancelled = await parent.result();+ if (interruptedParent !== undefined) {+ // A send is correlated to its new delivery; the previous handle+ // observes the turn that steering interrupts.+ const cancelled = await interruptedParent.result();cancelled.notEvent("turn.failed");cancelled.event("turn.cancelled", { count: 1 });parentTurns.push(cancelled);- parent = t.target.watchTurn(sessionId, {- startIndex: parent.session.state!.streamIndex,- });}// Observe through the result-bearing task wake, not just the parent's// acknowledgment of the steering message or an AGENT_BUSY failure wake.@@ -133,9 +131,13 @@ export default cases.map(({ parentActive, steering, description }) =>});}- const calls = parentTurns.flatMap((turn) =>- turn.events.filter((event) => event.type === "subagent.called"),- );+ // The initial dispatch may have arrived after the acknowledgment.+ const calls = [+ called,+ ...parentTurns.flatMap((turn) =>+ turn.events.filter((event) => event.type === "subagent.called"),+ ),+ ];await t.require(calls,satisfies(diff --git a/packages/eve/src/internal/nitro/routes/channel-dispatch.test.ts b/packages/eve/src/internal/nitro/routes/channel-dispatch.test.tsindex 8d5017580..bcd717975 100644--- a/packages/eve/src/internal/nitro/routes/channel-dispatch.test.ts+++ b/packages/eve/src/internal/nitro/routes/channel-dispatch.test.ts@@ -795,6 +795,8 @@ describe("dispatchChannelRequest tracing", () => {const [span] = await finishedSpans();expect(span!.name).toBe("agent.channel.request");expect(span!.attributes["http.route"]).toBe("/slack");+ expect(span!.attributes["operation.name"]).toBe("agent.channel.request");+ expect(span!.attributes["resource.name"]).toBe("agent.channel.request");} finally {Reflect.deleteProperty(instrumentationRuntime, "instrumentationProviders");}diff --git a/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.ts b/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.tsindex 91247e872..4f770094d 100644--- a/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.ts+++ b/packages/eve/src/internal/nitro/routes/channel-request-instrumentation.ts@@ -11,6 +11,7 @@ import {import { getInstrumentationRuntime } from "#instrumentation/runtime.js";import { recordErrorOnSpan } from "#internal/logging.js";import { markAgentTraceContext } from "#tracing/agent-trace-context.js";+import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";/*** Stable tracer name for every inbound eve channel HTTP request. Kept@@ -78,13 +79,17 @@ export async function traceChannelRequest<T extends Response>(getInstrumentationRuntime()?.instrumentationProviders === true? "agent.channel.request": routeKey;- const span = trace- .getTracer(TRACER_NAME)- .startSpan(- spanName,- { attributes: baseAttributes(request, routeKey), kind: SpanKind.SERVER },- parentContext,- );+ const span = trace.getTracer(TRACER_NAME).startSpan(+ spanName,+ {+ attributes: {+ ...baseAttributes(request, routeKey),+ ...(spanName === "agent.channel.request" ? agentSpanNamingAttributes(spanName) : undefined),+ },+ kind: SpanKind.SERVER,+ },+ parentContext,+ );const activeContext = markAgentTraceContext(trace.setSpan(parentContext, span));try {diff --git a/packages/eve/src/tracing/agent-action-instrumentation.ts b/packages/eve/src/tracing/agent-action-instrumentation.tsindex 1f2b0e9a6..15cbfacd0 100644--- a/packages/eve/src/tracing/agent-action-instrumentation.ts+++ b/packages/eve/src/tracing/agent-action-instrumentation.ts@@ -17,6 +17,7 @@ import type {import { actionIdempotencyKey, attemptIdempotencyKey } from "#instrumentation/lifecycle.js";import { contentAttribute } from "#tracing/agent-otel-content.js";import { setAgentUsage } from "#tracing/agent-otel-usage.js";+import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";import type { AgentActionTraceState, AgentTraceStateStore } from "#tracing/agent-trace-state.js";import { normalizeChannelAudience } from "#shared/channel-audience.js";@@ -132,6 +133,7 @@ export function createAgentActionInstrumentation(input: {"agent.step.attempt": state.attemptIndex,"agent.step.index": state.stepIndex,"agent.turn.id": state.turnId,+ ...agentSpanNamingAttributes("agent.action"),},startTime: state.startTimeMs,},diff --git a/packages/eve/src/tracing/agent-approval-instrumentation.ts b/packages/eve/src/tracing/agent-approval-instrumentation.tsindex bcedd8475..205825b10 100644--- a/packages/eve/src/tracing/agent-approval-instrumentation.ts+++ b/packages/eve/src/tracing/agent-approval-instrumentation.ts@@ -15,6 +15,7 @@ import type {} from "#instrumentation/lifecycle.js";import type { JsonValue } from "#shared/json.js";import { contentAttribute } from "#tracing/agent-otel-content.js";+import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";import { withChannelAudience } from "#tracing/channel-audience-context.js";import type { AgentActionContext } from "#tracing/agent-action-instrumentation.js";import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";@@ -106,6 +107,7 @@ export function createAgentApprovalInstrumentation(input: {"agent.step.attempt": state.attemptIndex,"agent.step.index": state.stepIndex,"agent.turn.id": state.turnId,+ ...agentSpanNamingAttributes("agent.approval"),},startTime: state.startTimeMs,},diff --git a/packages/eve/src/tracing/agent-channel-delivery-instrumentation.ts b/packages/eve/src/tracing/agent-channel-delivery-instrumentation.tsindex 9bea54fb5..44eb3e280 100644--- a/packages/eve/src/tracing/agent-channel-delivery-instrumentation.ts+++ b/packages/eve/src/tracing/agent-channel-delivery-instrumentation.ts@@ -19,6 +19,7 @@ import { sessionIdempotencyKey } from "#instrumentation/lifecycle.js";import type { JsonValue } from "#shared/json.js";import { normalizeChannelAudience, type ChannelAudience } from "#shared/channel-audience.js";import { contentAttribute } from "#tracing/agent-otel-content.js";+import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";import { withChannelAudience } from "#tracing/channel-audience-context.js";import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";import type { AgentSessionTraceState, AgentTraceStateStore } from "#tracing/agent-trace-state.js";@@ -118,6 +119,7 @@ export function createAgentChannelDeliveryInstrumentation(input: {"agent.session.id": event.sessionId,"agent.turn.id": event.turnId,"agent.turn.sequence": event.sequence,+ ...agentSpanNamingAttributes("agent.channel.delivery"),},kind: SpanKind.CONSUMER,links:diff --git a/packages/eve/src/tracing/agent-otel-provider.test.ts b/packages/eve/src/tracing/agent-otel-provider.test.tsindex 97f3bd99d..3affaad1e 100644--- a/packages/eve/src/tracing/agent-otel-provider.test.ts+++ b/packages/eve/src/tracing/agent-otel-provider.test.ts@@ -8,6 +8,7 @@ import {} from "@opentelemetry/sdk-trace-base";import { describe, expect, it, vi } from "vitest";+import { JsonTraceSerializer } from "#compiled/@opentelemetry/otlp-transformer/index.js";import {ROOT_CONTEXT,context,@@ -40,6 +41,13 @@ import {} from "#instrumentation/lifecycle.js";import type { ChannelAudience } from "#shared/channel-audience.js";import { channelAudienceFromContext } from "#tracing/channel-audience-context.js";+import { contentFilteringProcessor } from "#tracing/content-span-processor.js";+import { parseLocalTraceSegment } from "#tracing/local-trace-reader.js";+import {+ composeSpanExportPolicies,+ redactSpanInputs,+ redactSpanOutputs,+} from "#tracing/span-export-policy.js";import type { TraceCapturePolicy } from "#tracing/otel-declaration.js";import {actionIdempotencyKey,@@ -363,6 +371,105 @@ function nanos(hrTime: readonly [number, number]): bigint {}describe("createAgentOtelInstrumentation", () => {+ it.each(["public", "private"] as const)(+ "exports explicit names for %s spans without changing session grouping",+ async (channelAudience) => {+ const metadata = new InMemorySpanExporter();+ const runtime = createRuntime(undefined, undefined, [+ contentFilteringProcessor(+ new SimpleSpanProcessor(metadata),+ composeSpanExportPolicies(redactSpanInputs(), redactSpanOutputs()),+ ),+ ]);+ for (const sequence of [0, 1]) {+ await emitAttempt({+ channelAudience,+ hooks: runtime.hooks,+ runInContext: runtime.runInContext,+ sessionId: "session-1",+ turnId: `turn_${sequence}`,+ turnSequence: sequence,+ });+ }+ await runtime.provider.forceFlush();+ const exported = runtime.exporter.getFinishedSpans();+ const traceId = exported[0]!.spanContext().traceId;+ expect(new Set(exported.map((span) => span.spanContext().traceId)).size).toBe(1);+ for (const spans of [exported, metadata.getFinishedSpans()]) {+ const parsed = parseLocalTraceSegment(+ new TextDecoder().decode(JsonTraceSerializer.serializeRequest(spans)!),+ traceId,+ );+ expect(parsed).toHaveLength(exported.length);+ for (const span of parsed) {+ expect(span.attributes["resource.name"]).toBe(span.name);+ expect(span.attributes["operation.name"]).toBe(+ span.attributes["gen_ai.operation.name"] ?? span.name,+ );+ }+ const session = parsed.find((span) => span.name === "agent.session")!;+ expect(session.attributes["agent.trace.schema.version"]).toBe(3);+ const turns = parsed.filter((span) => span.name === "invoke_agent weather");+ expect(turns).toHaveLength(2);+ expect(turns.every((span) => span.parentSpanId === session.spanId)).toBe(true);+ expect(turns.map((span) => span.attributes["agent.turn.outcome"])).toEqual([+ "completed",+ "completed",+ ]);+ }+ await runtime.provider.shutdown();+ },+ );++ it.each(["completed", "failed", "cancelled"] as const)(+ "exports the %s turn outcome after content redaction and event removal",+ async (outcome) => {+ const metadata = new InMemorySpanExporter();+ const runtime = createRuntime(undefined, undefined, [+ contentFilteringProcessor(+ new SimpleSpanProcessor(metadata),+ composeSpanExportPolicies(redactSpanInputs(), redactSpanOutputs()),+ ),+ ]);+ await publishTurnStarted({+ hooks: runtime.hooks,+ sessionId: "session-1",+ turnId: "turn_0",+ turnSequence: 0,+ });+ const identity = {+ idempotencyKey: turnIdempotencyKey("session-1", "turn_0"),+ sessionId: "session-1",+ turnId: "turn_0",+ };+ await runtime.hooks.publish(+ outcome === "failed"+ ? { ...identity, type: "turn.failed", error: new Error("private failure") }+ : { ...identity, type: outcome === "completed" ? "turn.completed" : "turn.cancelled" },+ );+ await runtime.hooks.publish({+ idempotencyKey: sessionIdempotencyKey("session-1"),+ sessionId: "session-1",+ turnId: "turn_0",+ type: "session.waiting",+ });+ await runtime.provider.forceFlush();+ const spans = metadata.getFinishedSpans();+ const serialized = new TextDecoder().decode(+ JsonTraceSerializer.serializeRequest(+ spans.map((span) => ({ ...span, events: [], spanContext: () => span.spanContext() })),+ )!,+ );+ const turn = parseLocalTraceSegment(serialized, spans[0]!.spanContext().traceId).find(+ (span) => span.name === "invoke_agent weather",+ )!;+ expect(turn.attributes["agent.turn.outcome"]).toBe(outcome);+ expect(turn.statusCode).toBe(outcome === "failed" ? 2 : 0);+ expect(serialized).not.toContain("private failure");+ await runtime.provider.shutdown();+ },+ );+it("prepares a stable stream trace before lifecycle hooks observe the turn", async () => {const runtime = createRuntime();const sessionEvent = {@@ -427,6 +534,10 @@ describe("createAgentOtelInstrumentation", () => {const session = byName(spans, "agent.session")[0]!;expect(session.spanContext().traceId).toBe(seed.traceId);expect(session.spanContext().spanId).toBe(seed.spanId);+ expect(session.attributes).toMatchObject({+ "operation.name": "agent.session",+ "resource.name": "agent.session",+ });});it("falls back to fresh ids when no trace seed is present", async () => {@@ -637,6 +748,10 @@ describe("createAgentOtelInstrumentation", () => {const spans = runtime.exporter.getFinishedSpans();const session = spans.find((span) => span.name === "agent.session")!;const delivery = spans.find((span) => span.name === "agent.channel.delivery")!;+ expect(delivery.attributes).toMatchObject({+ "operation.name": "agent.channel.delivery",+ "resource.name": "agent.channel.delivery",+ });expect(delivery.kind).toBe(SpanKind.CONSUMER);expect(delivery.parentSpanContext?.spanId).toBe(session.spanContext().spanId);expect(delivery.links).toEqual([@@ -1461,6 +1576,8 @@ describe("createAgentOtelInstrumentation", () => {"agent.approval.request": expect.stringContaining("Approve weather?"),"agent.approval.request_id": "approval-1","agent.approval.response": expect.stringContaining("approve"),+ "operation.name": "agent.approval",+ "resource.name": "agent.approval","agent.session.id": "session-1","agent.step.index": 0,"agent.turn.id": "turn-1",diff --git a/packages/eve/src/tracing/agent-otel-provider.ts b/packages/eve/src/tracing/agent-otel-provider.tsindex 493ecf6ca..e9119dc4a 100644--- a/packages/eve/src/tracing/agent-otel-provider.ts+++ b/packages/eve/src/tracing/agent-otel-provider.ts@@ -14,7 +14,7 @@ import { contextStorage } from "#context/container.js";import { SessionTraceSeedKey } from "#context/keys.js";import { withoutInstrumentationContent } from "#instrumentation/content.js";import { instrumentationEventForTraceDecision } from "#instrumentation/content-policy.js";-import type { AgentTraceStateStore, AgentTurnTraceState } from "#tracing/agent-trace-state.js";+import type { AgentTraceStateStore } from "#tracing/agent-trace-state.js";import {contentAttribute,genAiInputMessagesAttribute,@@ -30,9 +30,14 @@ import { createAgentActionInstrumentation } from "#tracing/agent-action-instrumeimport { createAgentApprovalInstrumentation } from "#tracing/agent-approval-instrumentation.js";import { createAgentChannelDeliveryInstrumentation } from "#tracing/agent-channel-delivery-instrumentation.js";import { createAgentToolInstrumentation } from "#tracing/agent-tool-instrumentation.js";+import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";import { markAgentTraceContext } from "#tracing/agent-trace-context.js";import { runtimeContextAttributes } from "#tracing/agent-otel-runtime-context.js";-import { setAgentUsage } from "#tracing/agent-otel-usage.js";+import {+ readGatewayCost,+ setAgentInvocationUsage,+ setAgentUsage,+} from "#tracing/agent-otel-usage.js";import { createAgentOtelSessionContext } from "#tracing/agent-otel-session-context.js";import type { TraceCapturePolicy } from "#tracing/otel-declaration.js";import { isSampledTrace, resolveTracePolicyDecision } from "#tracing/sampled-trace.js";@@ -224,6 +229,7 @@ export function createAgentOtelInstrumentation("agent.step.index": event.scope.stepIndex,"agent.turn.id": event.scope.turnId,"agent.name": event.scope.functionId,+ ...agentSpanNamingAttributes("agent.step"),...runtimeContextAttributes(event.runtimeContext),},links:@@ -312,6 +318,7 @@ export function createAgentOtelInstrumentation("gen_ai.agent.name": agentName,"gen_ai.conversation.id": event.sessionId,"gen_ai.operation.name": "invoke_agent",+ ...agentSpanNamingAttributes(agentSpanName(agentName), "invoke_agent"),},kind: SpanKind.INTERNAL,startTime: turn.startTimeMs,@@ -330,6 +337,14 @@ export function createAgentOtelInstrumentation(setAgentInvocationUsage(span, turn.modelUsage);span.addEvent("turn.started", undefined, turn.startTimeMs);if (turn.terminal !== undefined) {+ span.setAttribute(+ "agent.turn.outcome",+ turn.terminal.type === "turn.completed"+ ? "completed"+ : turn.terminal.type === "turn.cancelled"+ ? "cancelled"+ : "failed",+ );span.addEvent(turn.terminal.type);if (turn.terminal.type === "turn.failed") {recordError(span, turn.terminal.error);@@ -362,6 +377,7 @@ export function createAgentOtelInstrumentation("gen_ai.operation.name": "chat","gen_ai.provider.name": event.model.provider,"gen_ai.request.model": event.model.modelId,+ ...agentSpanNamingAttributes(modelSpanName(event.model.modelId), "chat"),...runtimeContextAttributes(event.runtimeContext),},},@@ -632,43 +648,6 @@ function contextFromSpanContext(spanContext: SpanContext): Context {return trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext(spanContext));}-/**- * Extracts cost data from a step result's provider metadata. Only Vercel AI- * Gateway reports it (`providerMetadata.gateway`): raw inference cost, the- * gateway's surcharged total, the input/output split, and the generation id- * for dashboard reconciliation. Values arrive as USD strings; anything- * missing or non-numeric is skipped, so non-gateway providers get nothing.- */-function readGatewayCost(- providerMetadata: Readonly<Record<string, unknown>>,-): Record<string, string | number> | undefined {- const gateway = providerMetadata.gateway;- if (!isRecord(gateway)) return undefined;- const attributes: Record<string, string | number> = {};- const cost = readUsd(gateway.cost);- if (cost !== undefined) attributes["gen_ai.usage.cost"] = cost;- const gatewayCost = readUsd(gateway.gatewayCost);- if (gatewayCost !== undefined) attributes["gen_ai.usage.gateway_cost"] = gatewayCost;- const inputCost = readUsd(gateway.inputInferenceCost);- if (inputCost !== undefined) attributes["gen_ai.usage.input_cost"] = inputCost;- const outputCost = readUsd(gateway.outputInferenceCost);- if (outputCost !== undefined) attributes["gen_ai.usage.output_cost"] = outputCost;- if (typeof gateway.generationId === "string" && gateway.generationId.length > 0) {- attributes["gen_ai.generation.id"] = gateway.generationId;- }- return Object.keys(attributes).length === 0 ? undefined : attributes;-}--function readUsd(value: unknown): number | undefined {- if (typeof value !== "string") return undefined;- const parsed = Number(value);- return Number.isFinite(parsed) ? parsed : undefined;-}--function isRecord(value: unknown): value is Record<string, unknown> {- return typeof value === "object" && value !== null && !Array.isArray(value);-}-function modelSpanName(modelId: string): string {return `chat ${modelId}`;}@@ -677,16 +656,6 @@ function agentSpanName(agentName: string | undefined): string {return agentName === undefined ? "invoke_agent" : `invoke_agent ${agentName}`;}-function setAgentInvocationUsage(span: Span, modelUsage: AgentTurnTraceState["modelUsage"]): void {- if (modelUsage === undefined) return;- if (modelUsage.inputTokens !== undefined) {- span.setAttribute("gen_ai.usage.input_tokens", modelUsage.inputTokens);- }- if (modelUsage.outputTokens !== undefined) {- span.setAttribute("gen_ai.usage.output_tokens", modelUsage.outputTokens);- }-}-function errorText(error: unknown): unknown {return error instanceof Error ? error.message : error;}diff --git a/packages/eve/src/tracing/agent-otel-session-context.ts b/packages/eve/src/tracing/agent-otel-session-context.tsindex 4d87a5bab..2218d2e27 100644--- a/packages/eve/src/tracing/agent-otel-session-context.ts+++ b/packages/eve/src/tracing/agent-otel-session-context.ts@@ -18,6 +18,7 @@ import {} from "#tracing/sampled-trace.js";import type { AgentSessionTraceState, AgentTraceStateStore } from "#tracing/agent-trace-state.js";import { readInstrumentationDecision } from "#shared/instrumentation-decision.js";+import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";interface AgentOtelSessionContextInput {readonly frameworkVersion: string;@@ -69,6 +70,7 @@ export function createAgentOtelSessionContext("agent.name": session.agentName,"agent.session.id": session.sessionId,"agent.trace.schema.version": 3,+ ...agentSpanNamingAttributes("agent.session"),},root: true,});@@ -96,6 +98,7 @@ export function createAgentOtelSessionContext("agent.name": session.agentName,"agent.session.id": session.sessionId,"agent.trace.schema.version": 3,+ ...agentSpanNamingAttributes("agent.session"),},root: true,});diff --git a/packages/eve/src/tracing/agent-otel-usage.ts b/packages/eve/src/tracing/agent-otel-usage.tsindex 3cdbbba91..0369c0902 100644--- a/packages/eve/src/tracing/agent-otel-usage.ts+++ b/packages/eve/src/tracing/agent-otel-usage.ts@@ -1,6 +1,7 @@import type { Span } from "#compiled/@opentelemetry/api/index.js";import type { InstrumentationUsage } from "#instrumentation/lifecycle.js";+import type { AgentTurnTraceState } from "#tracing/agent-trace-state.js";/** Applies eve's structural token usage attributes to an agent span. */export function setAgentUsage(@@ -30,3 +31,53 @@ export function setAgentUsage(}}}++export function setAgentInvocationUsage(+ span: Span,+ modelUsage: AgentTurnTraceState["modelUsage"],+): void {+ if (modelUsage === undefined) return;+ if (modelUsage.inputTokens !== undefined) {+ span.setAttribute("gen_ai.usage.input_tokens", modelUsage.inputTokens);+ }+ if (modelUsage.outputTokens !== undefined) {+ span.setAttribute("gen_ai.usage.output_tokens", modelUsage.outputTokens);+ }+}++/**+ * Extracts cost data from a step result's provider metadata. Only Vercel AI+ * Gateway reports it (`providerMetadata.gateway`): raw inference cost, the+ * gateway's surcharged total, the input/output split, and the generation id+ * for dashboard reconciliation. Values arrive as USD strings; anything+ * missing or non-numeric is skipped, so non-gateway providers get nothing.+ */+export function readGatewayCost(+ providerMetadata: Readonly<Record<string, unknown>>,+): Record<string, string | number> | undefined {+ const gateway = providerMetadata.gateway;+ if (!isRecord(gateway)) return undefined;+ const attributes: Record<string, string | number> = {};+ const cost = readUsd(gateway.cost);+ if (cost !== undefined) attributes["gen_ai.usage.cost"] = cost;+ const gatewayCost = readUsd(gateway.gatewayCost);+ if (gatewayCost !== undefined) attributes["gen_ai.usage.gateway_cost"] = gatewayCost;+ const inputCost = readUsd(gateway.inputInferenceCost);+ if (inputCost !== undefined) attributes["gen_ai.usage.input_cost"] = inputCost;+ const outputCost = readUsd(gateway.outputInferenceCost);+ if (outputCost !== undefined) attributes["gen_ai.usage.output_cost"] = outputCost;+ if (typeof gateway.generationId === "string" && gateway.generationId.length > 0) {+ attributes["gen_ai.generation.id"] = gateway.generationId;+ }+ return Object.keys(attributes).length === 0 ? undefined : attributes;+}++function readUsd(value: unknown): number | undefined {+ if (typeof value !== "string") return undefined;+ const parsed = Number(value);+ return Number.isFinite(parsed) ? parsed : undefined;+}++function isRecord(value: unknown): value is Record<string, unknown> {+ return typeof value === "object" && value !== null && !Array.isArray(value);+}diff --git a/packages/eve/src/tracing/agent-span-naming.ts b/packages/eve/src/tracing/agent-span-naming.tsnew file mode 100644index 000000000..b268155ae--- /dev/null+++ b/packages/eve/src/tracing/agent-span-naming.ts@@ -0,0 +1,7 @@+// Datadog maps operation and resource separately from the OTel span name.+export function agentSpanNamingAttributes(+ name: string,+ operation: string = name,+): Record<string, string> {+ return { "operation.name": operation, "resource.name": name };+}diff --git a/packages/eve/src/tracing/agent-tool-instrumentation.ts b/packages/eve/src/tracing/agent-tool-instrumentation.tsindex 8111c7963..73929cb00 100644--- a/packages/eve/src/tracing/agent-tool-instrumentation.ts+++ b/packages/eve/src/tracing/agent-tool-instrumentation.ts@@ -16,6 +16,7 @@ import type {} from "#instrumentation/lifecycle.js";import { actionIdempotencyKey } from "#instrumentation/lifecycle.js";import { contentAttribute } from "#tracing/agent-otel-content.js";+import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";import { withChannelAudience } from "#tracing/channel-audience-context.js";import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";import type { AgentActionContext } from "#tracing/agent-action-instrumentation.js";@@ -218,6 +219,7 @@ function toolAttributes(event: InstrumentationToolCallStartedEvent): Record<stri"gen_ai.tool.call.id": event.callId,"gen_ai.tool.name": event.toolName,"gen_ai.tool.type": "function",+ ...agentSpanNamingAttributes(`execute_tool ${event.toolName}`, "execute_tool"),};}diff --git a/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts b/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.tsindex 27c6fb94c..0457fcdc5 100644--- a/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts+++ b/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts@@ -206,6 +206,25 @@ describe("local instrumentation runtime", () => {" chat model-1"," user.model-work",]);+ for (const exported of spans.filter((span) => !span.name.startsWith("user."))) {+ expect(exported.attributes).toEqual(+ expect.arrayContaining([+ { key: "resource.name", value: { stringValue: exported.name } },+ {+ key: "operation.name",+ value: {+ stringValue: exported.name.startsWith("invoke_agent ")+ ? "invoke_agent"+ : exported.name.startsWith("execute_tool ")+ ? "execute_tool"+ : exported.name.startsWith("chat ")+ ? "chat"+ : exported.name,+ },+ },+ ]),+ );+ }expect(span(spans, "agent.step").links).toEqual([expect.objectContaining({attributes: expect.arrayContaining([@@ -251,6 +270,7 @@ describe("local instrumentation runtime", () => {});interface OtlpSpan {+ readonly attributes: ReadonlyArray<{ readonly key: string; readonly value: unknown }>;readonly links?: ReadonlyArray<{readonly attributes: ReadonlyArray<{ readonly key: string; readonly value: unknown }>;readonly spanId: string;