Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

refactor(eve)!: remove action wrappers and unify tool lifecycle (#4398)

Chad Hietala28136be86b04
diff --git a/.changeset/single-tool-span.md b/.changeset/single-tool-span.md
new file mode 100644
index 000000000..d66971e1b
--- /dev/null
+++ b/.changeset/single-tool-span.md
@@ -0,0 +1,7 @@
+---
+"eve": minor
+---
+
+Tool calls now export one `execute_tool` span instead of an `agent.action` wrapper and a tool child. Durable outcomes, approvals, execution metadata, and subagent trace correlation stay on the tool span; the trace schema remains version 4.
+
+`defineInstrumentation` no longer supports `action.*` handlers or `InstrumentationAction*` types; use `tool.call.*`, `InstrumentationToolCall*`, and `InstrumentationToolOutput` instead. Tool events now describe one durable call lifecycle, and start handlers read `event.toolName` instead of `event.name`.
diff --git a/docs/guides/instrumentation/instrumentation.mdx b/docs/guides/instrumentation/instrumentation.mdx
index 82ed776ee..91571bbd1 100644
--- a/docs/guides/instrumentation/instrumentation.mdx
+++ b/docs/guides/instrumentation/instrumentation.mdx
@@ -23,7 +23,7 @@ agent/instrumentation/
audit.ts lifecycle instrumentation
```
-This instrumentation records action timing and identity without receiving tool
+This instrumentation records tool timing and identity without receiving tool
arguments or results:
```ts title="agent/instrumentation/audit.ts"
@@ -31,15 +31,15 @@ import { defineInstrumentation } from "eve/instrumentation";
export default defineInstrumentation({
events: {
- "action.started": (event, ctx) => {
- ctx.state.set({ name: event.name, startedAt: Date.now() });
+ "tool.call.started": (event, ctx) => {
+ ctx.state.set({ name: event.toolName, startedAt: Date.now() });
},
- "action.completed": (event, ctx) => {
+ "tool.call.completed": (event, ctx) => {
const started = ctx.state.get() as { name: string; startedAt: number } | undefined;
if (started === undefined) return;
console.log({
- action: started.name,
+ tool: started.name,
durationMs: Date.now() - started.startedAt,
outcome: event.outcome,
});
@@ -123,12 +123,12 @@ export default defineInstrumentation({
recordOutputs: false,
}),
events: {
- "action.started": async (event) => {
+ "tool.call.started": async (event) => {
await sendAuditRecord({
id: event.idempotencyKey,
input: redactApiKey(event.input),
kind: event.kind,
- name: event.name,
+ name: event.toolName,
});
},
},
@@ -155,21 +155,22 @@ Prefer `recordInputs: false` or `recordOutputs: false` when the destination does
start and its terminal event share an `idempotencyKey`; use it as a stable
operation key when upserting records across replays.
-| Lifecycle | Events | Meaning |
-| ---------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Session | `session.started`, `session.waiting`, `session.completed`, `session.failed` | Track a session from creation through suspension or termination. `session.waiting` is resumable; `session.completed` and `session.failed` are terminal. |
-| Turn | `turn.started`, `turn.completed`, `turn.cancelled`, `turn.failed` | Track one turn. Cancellation is not a failure and commonly leaves the session waiting for another turn. |
-| Channel delivery | `channel.delivery.started`, `channel.delivery.completed`, `channel.delivery.cancelled`, `channel.delivery.failed` | Track an inbound channel delivery while eve processes its turn. These events are emitted when the channel provides delivery metadata. |
-| Step attempt | `step.attempt.started`, `step.attempt.completed`, `step.attempt.failed`, `step.attempt.metadata` | Track each model-step attempt; retries have separate attempts. `step.attempt.metadata` is emitted after a step only when the AI SDK supplies provider metadata. |
-| Model call | `model.call.started`, `model.call.completed`, `model.call.failed` | Track an individual call to the model provider. Completion includes usage and finish reason; content fields follow the trace policy. |
-| Action | `action.started`, `action.completed`, `action.failed` | Track eve's durable dispatch of a tool, skill load, local subagent, or remote agent. A failed event's `outcome` distinguishes failure, rejection, cancellation, and abandonment. |
-| Tool call | `tool.call.started`, `tool.call.completed`, `tool.call.failed` | Track the AI SDK's in-process tool execution boundary, including its input and result when allowed by the trace policy. |
-| User input | `input.requested`, `input.resolved` | Track a question, tool approval, or session-limit request and its resolution. The resolution includes an outcome such as `answered`, `approved`, `denied`, or `cancelled`. |
-| Memory operation | `memory.operation.started`, `memory.operation.completed`, `memory.operation.failed` | Track a memory-store search or write. Events include the operation name, phase, and memory slot; completion may include a record count and records. |
-
-An ordinary tool emits both `action.*` and `tool.call.*` events. Use `action.*` for eve's durable dispatch lifecycle, including tools, skills, subagents, and remote agents. Use `tool.call.*` only when you need the AI SDK's in-process tool execution boundary.
-
-For local and remote subagents, `action.completed` and `action.failed` include the child turn's `usage` directly. Token counts are always present when the child reports usage; `usage.costUsd` is present only when the model provider reports a price. Without the action-level usage, consumers would have to correlate child model events through trace lineage and sum their provider metadata, which is harder to do reliably across retries and parked turns.
+| Lifecycle | Events | Meaning |
+| ---------------- | ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Session | `session.started`, `session.waiting`, `session.completed`, `session.failed` | Track a session from creation through suspension or termination. `session.waiting` is resumable; `session.completed` and `session.failed` are terminal. |
+| Turn | `turn.started`, `turn.completed`, `turn.cancelled`, `turn.failed` | Track one turn. Cancellation is not a failure and commonly leaves the session waiting for another turn. |
+| Channel delivery | `channel.delivery.started`, `channel.delivery.completed`, `channel.delivery.cancelled`, `channel.delivery.failed` | Track an inbound channel delivery while eve processes its turn. These events are emitted when the channel provides delivery metadata. |
+| Step attempt | `step.attempt.started`, `step.attempt.completed`, `step.attempt.failed`, `step.attempt.metadata` | Track each model-step attempt; retries have separate attempts. `step.attempt.metadata` is emitted after a step only when the AI SDK supplies provider metadata. |
+| Model call | `model.call.started`, `model.call.completed`, `model.call.failed` | Track an individual call to the model provider. Completion includes usage and finish reason; content fields follow the trace policy. |
+| Tool call | `tool.call.started`, `tool.call.completed`, `tool.call.failed` | Track a tool, skill load, local subagent, or remote-agent call through durable settlement. A failed event's `outcome` distinguishes failure, rejection, cancellation, and abandonment. |
+| User input | `input.requested`, `input.resolved` | Track a question, tool approval, or session-limit request and its resolution. The resolution includes an outcome such as `answered`, `approved`, `denied`, or `cancelled`. |
+| Memory operation | `memory.operation.started`, `memory.operation.completed`, `memory.operation.failed` | Track a memory-store search or write. Events include the operation name, phase, and memory slot; completion may include a record count and records. |
+
+Each call emits one `tool.call.*` lifecycle to authored providers. SDK execution callbacks do not emit a second lifecycle. Tool calls can remain open across approvals, parked turns, and worker replacement.
+
+For local and remote subagents, `tool.call.completed` and `tool.call.failed` include the child turn's `usage` directly. Token counts are always present when the child reports usage; `usage.costUsd` is present only when the model provider reports a price.
+
+When upgrading, replace `action.started`, `action.completed`, and `action.failed` handlers with their `tool.call.*` equivalents. Read `event.toolName` instead of `event.name` on start events. The OpenTelemetry trace schema remains version 4.
Handlers from different files run concurrently and are failure-isolated. Do
not depend on execution order. Use `flush` to drain buffered records and
diff --git a/docs/guides/instrumentation/otel.mdx b/docs/guides/instrumentation/otel.mdx
index 90fa5e10b..02de644b7 100644
--- a/docs/guides/instrumentation/otel.mdx
+++ b/docs/guides/instrumentation/otel.mdx
@@ -135,8 +135,7 @@ agent turn:
invoke_agent <agent>
└── agent.step
├── chat <model>
- └── agent.action
- └── execute_tool <tool>
+ └── execute_tool <tool>
```
Each root agent turn starts a new trace. If eve receives a local caller's trace
diff --git a/docs/reference/cli.md b/docs/reference/cli.md
index d68f24fef..fd4755612 100644
--- a/docs/reference/cli.md
+++ b/docs/reference/cli.md
@@ -393,11 +393,11 @@ Span rows carry inline metrics when the span recorded them — `↑input`/`↓ou
With local caller trace context, the first local subagent turn uses the caller's trace. Its span is a child of the dispatch span. A remote child starts a separate root trace and links its first turn to the dispatch with `eve.link.type=agent.dispatch`. Remote requests use W3C `tracestate` to identify that dispatch span if platform HTTP handling changes `traceparent`. Later child turns start new traces. All related sessions have the same `gen_ai.conversation.id`. `agent.subagent.name` identifies the subagent.
-Each workflow tool call has an `agent.action` span and an `execute_tool <tool>` span. The first turn of each local session it opens with `ctx.agent` is a child of the call's `agent.action` span. A remote session links to that action from its own trace. An individual `ctx.agent` call does not create a separate dispatch span. Only agent execution uses `invoke_agent`.
+Each workflow tool call has one `execute_tool <tool>` span. The first turn of each local session it opens with `ctx.agent` is a child of that tool span. A remote session links to the tool span from its own trace. An individual `ctx.agent` call does not create a separate dispatch span. Only agent execution uses `invoke_agent`.
Outbound MCP `tools/call` requests add MCP semantic attributes to the matching `execute_tool` span. When eve has no tool span to enrich, it creates a `CLIENT` `tools/call <tool>` span. Each `tools/list` discovery also has a `CLIENT` span. Configured OpenTelemetry propagation fields are injected into MCP `params._meta` for JSON-RPC bodies up to 1 MiB and propagation metadata up to 8 KiB; larger requests are sent unchanged. eve removes its audience and session-lineage baggage before forwarding, and keeps the input/output content-capture policy local.
-`connection_execute` records the connector call as a child of its outer `agent.action` span, with `agent.action.parent_call_id` identifying the outer call. Both spans retain `agent.action.kind=tool-call`; consumers can distinguish nested calls by the parent call ID. The nested span's `gen_ai.tool.call.arguments` holds the validated connector input, including schema defaults, when input capture is enabled. Its duration covers the reporting of the nested result, not the connector request; use the outer call's `execute_tool` span for execution timing.
+`connection_execute` records the connector call as a child of its outer `execute_tool` span, with `agent.action.parent_call_id` identifying the outer call. Both spans retain `agent.action.kind=tool-call`; consumers can distinguish nested calls by the parent call ID. The nested span's `gen_ai.tool.call.arguments` holds the validated connector input, including schema defaults, when input capture is enabled. Its duration covers the reporting of the nested result, not the connector request; use the outer call's `execute_tool` span for execution timing.
Each root agent turn starts a new trace. With local caller trace context, that trace includes the first turn of each local subagent it starts. Remote child turns and later local child turns start separate traces. If a worker is replaced, the new worker uses the prepared trace context for the same turn. Supply the conversation ID to show all related traces, oldest first.
diff --git a/packages/eve/src/cli/commands/trace-detail.ts b/packages/eve/src/cli/commands/trace-detail.ts
index 32d78eb93..aec21ca4e 100644
--- a/packages/eve/src/cli/commands/trace-detail.ts
+++ b/packages/eve/src/cli/commands/trace-detail.ts
@@ -14,7 +14,7 @@ import { localTraceSpanCostUsd, type LocalTraceSummary } from "#tracing/local-tr
/**
* Compact metrics for one tree row: token chips (`↑1.4K`/`↓213`), cost
* (`$0.0031`). Only chips whose attributes the span actually carries — rows
- * without usage stay clean. Tool names belong to eve's durable `agent.action`
+ * without usage stay clean. Tool names belong to eve's durable `execute_tool`
* label instead of the AI SDK's child span.
* Raw values: callers sanitize for their output surface.
*/
diff --git a/packages/eve/src/cli/dev/tui/traces/trace-conversation.ts b/packages/eve/src/cli/dev/tui/traces/trace-conversation.ts
index 128abe1be..0364d7a56 100644
--- a/packages/eve/src/cli/dev/tui/traces/trace-conversation.ts
+++ b/packages/eve/src/cli/dev/tui/traces/trace-conversation.ts
@@ -1,7 +1,7 @@
/**
* Conversation view for the `/traces` viewer: the same trace, re-told as a
* flow of user messages, assistant replies, and tool calls instead of a
- * latency waterfall. Activation and action spans provide the user and tool
+ * latency waterfall. Activation and tool spans provide the user and tool
* cards directly; model response spans provide assistant cards.
*/
@@ -59,7 +59,7 @@ export interface ConversationSubagent {
/** Max rendered lines for one collapsed card payload (args, result, or text). */
const CARD_PAYLOAD_LINES = 3;
-/** Builds the conversation flow from eve's activation, model, and action spans. */
+/** Builds the conversation flow from eve's activation, model, and tool spans. */
export function buildConversationItems(trace: LocalTrace): ConversationItem[] {
const byId = new Map(trace.spans.map((span) => [span.spanId, span]));
const subagents = new Map<string, ConversationSubagent>();
@@ -165,14 +165,21 @@ export function buildConversationItems(trace: LocalTrace): ConversationItem[] {
}
continue;
}
- if (span.name === "agent.action") {
+ if (
+ span.attributes["gen_ai.operation.name"] === "execute_tool" ||
+ span.name === "agent.action"
+ ) {
entries.push({
item: {
kind: "tool",
args: stringAttribute(span, "gen_ai.tool.call.arguments"),
durationMs: spanDurationMs(span),
error: span.statusCode === 2,
- name: stripTerminalControls(stringAttribute(span, "agent.action.name") ?? "action"),
+ name: stripTerminalControls(
+ stringAttribute(span, "gen_ai.tool.name") ??
+ stringAttribute(span, "agent.action.name") ??
+ "tool",
+ ),
result: unwrapJsonString(stringAttribute(span, "gen_ai.tool.call.result")),
span,
subagent,
@@ -238,7 +245,7 @@ function turnSubagent(
};
}
const parent = turn.parentSpanId === undefined ? undefined : byId.get(turn.parentSpanId);
- if (parent === undefined || parent.name !== "agent.action") return undefined;
+ if (parent === undefined) return undefined;
const kind = stringAttribute(parent, "agent.action.kind");
if (kind !== "subagent-call" && kind !== "remote-agent-call") return undefined;
const parentTurnId = stringAttribute(parent, "agent.turn.id");
diff --git a/packages/eve/src/harness/tool-loop.test.ts b/packages/eve/src/harness/tool-loop.test.ts
index 13e2b1cc7..da48332a6 100644
--- a/packages/eve/src/harness/tool-loop.test.ts
+++ b/packages/eve/src/harness/tool-loop.test.ts
@@ -10345,7 +10345,7 @@ describe("createToolLoopHarness", () => {
});
const started = vi.fn();
const hooks = createInstrumentationHooks([
- { events: { "action.started": started }, name: "actions" },
+ { events: { "tool.call.started": started }, name: "actions" },
]);
const { emit } = createEventCollector();
const runStep = createToolLoopHarness(
@@ -10361,8 +10361,8 @@ describe("createToolLoopHarness", () => {
expect.objectContaining({
callId: "call-delegate",
kind: "tool-call",
- name: "delegate",
- type: "action.started",
+ toolName: "delegate",
+ type: "tool.call.started",
}),
expect.anything(),
);
diff --git a/packages/eve/src/harness/turn-tag-state.ts b/packages/eve/src/harness/turn-tag-state.ts
index 8444c228e..e09fa788a 100644
--- a/packages/eve/src/harness/turn-tag-state.ts
+++ b/packages/eve/src/harness/turn-tag-state.ts
@@ -254,7 +254,7 @@ export function accumulateTurnUsage(input: {
* Adds usage to session totals without touching the in-flight turn totals:
* the delegated spend a parent counts, or a run's tally of what its
* `ctx.agent` sessions spent. Turn tags attribute only the parent's own model
- * calls (child spend is attributed by the caller's durable `agent.action`
+ * calls (child spend is attributed by the caller's durable `execute_tool`
* span); session totals feed the session token limits and the remaining-quota
* budget granted to later delegations.
*/
diff --git a/packages/eve/src/instrumentation/ai-sdk-hook-bridge.test.ts b/packages/eve/src/instrumentation/ai-sdk-hook-bridge.test.ts
index ad458f166..91fbe118f 100644
--- a/packages/eve/src/instrumentation/ai-sdk-hook-bridge.test.ts
+++ b/packages/eve/src/instrumentation/ai-sdk-hook-bridge.test.ts
@@ -11,8 +11,7 @@ import {
type InstrumentationProviderDefinition,
type InstrumentationStepAttemptMetadataEvent,
type InstrumentationStepAttemptStartedEvent,
- type InstrumentationToolCallStartedEvent,
- type InstrumentationToolCallTerminalEvent,
+ type InstrumentationContextRunner,
} from "#instrumentation/lifecycle.js";
import { captureLogRecords } from "#internal/testing/log-records.js";
@@ -449,7 +448,7 @@ describe("createAiSdkHookBridge", () => {
);
});
- it("terminalizes started operations with the abort reason", async () => {
+ it("does not publish a second tool lifecycle on SDK abort", async () => {
const after = vi.fn();
const hooks = createInstrumentationHooks([
{
@@ -465,10 +464,7 @@ describe("createAiSdkHookBridge", () => {
const reason = new Error("model aborted");
await Reflect.apply(bridge.onAbort!, bridge, [{ callId: "call-1", reason, steps: [] }]);
- expect(after).toHaveBeenCalledExactlyOnceWith(
- expect.objectContaining({ error: reason, type: "tool.call.failed" }),
- expect.anything(),
- );
+ expect(after).not.toHaveBeenCalled();
});
it("publishes an immutable operation projection to every provider", async () => {
@@ -622,88 +618,47 @@ describe("createAiSdkHookBridge", () => {
});
it.each([
- {
- expected: { output: "ok", type: "result" },
- toolOutput: { output: "ok", type: "tool-result" },
- },
- {
- expected: { error: "boom", type: "error" },
- toolOutput: { error: "boom", type: "tool-error" },
- },
- ])(
- "collapses tool output $toolOutput.type onto $expected.type",
- async ({ expected, toolOutput }) => {
- const before = vi.fn((event: InstrumentationToolCallStartedEvent) => {
- expect(Object.isFrozen(event)).toBe(true);
- });
- const after = vi.fn((event: InstrumentationToolCallTerminalEvent) => {
- if (event.type !== "tool.call.completed") throw new Error("expected completed tool call");
- expect(Object.isFrozen(event)).toBe(true);
- expect(Object.isFrozen(event.output)).toBe(true);
- });
- const actionStarted = vi.fn();
- const hooks = createInstrumentationHooks([
- {
- events: {
- "action.started": actionStarted,
- "tool.call.completed": after,
- "tool.call.started": before,
- },
- name: "spy",
- tracePolicy: contentTracePolicy,
+ { output: "ok", type: "tool-result" },
+ { error: "boom", type: "tool-error" },
+ ])("does not publish a tool lifecycle for SDK $type callbacks", async (toolOutput) => {
+ const before = vi.fn();
+ const after = vi.fn();
+ const hooks = createInstrumentationHooks([
+ {
+ events: {
+ "tool.call.completed": after,
+ "tool.call.started": before,
},
- ]);
- const bridge = createAiSdkHookBridge(scope, hooks);
- const toolCall = { input: { q: "eve" }, toolCallId: "tool-1", toolName: "search" };
+ name: "spy",
+ tracePolicy: contentTracePolicy,
+ },
+ ]);
+ const bridge = createAiSdkHookBridge(scope, hooks);
+ const toolCall = { input: { q: "eve" }, toolCallId: "tool-1", toolName: "search" };
- await Reflect.apply(bridge.onToolExecutionStart!, bridge, [{ callId: "call-1", toolCall }]);
- await Reflect.apply(bridge.onToolExecutionEnd!, bridge, [
- { callId: "call-1", toolCall, toolExecutionMs: 1, toolOutput },
- ]);
+ await Reflect.apply(bridge.onToolExecutionStart!, bridge, [{ callId: "call-1", toolCall }]);
+ await Reflect.apply(bridge.onToolExecutionEnd!, bridge, [
+ { callId: "call-1", toolCall, toolExecutionMs: 1, toolOutput },
+ ]);
- expect(before).toHaveBeenCalledExactlyOnceWith(
- {
- callId: "tool-1",
- idempotencyKey: `tool:${scope.attemptId}:tool-1:0`,
- frameworkTool: false,
- input: { q: "eve" },
- scope,
- toolName: "search",
- type: "tool.call.started",
- },
- expect.anything(),
- );
- expect(after).toHaveBeenCalledExactlyOnceWith(
- {
- durationMs: 1,
- idempotencyKey: `tool:${scope.attemptId}:tool-1:0`,
- output: expected,
- scope,
- type: "tool.call.completed",
- },
- expect.anything(),
- );
- expect(actionStarted).not.toHaveBeenCalled();
- },
- );
+ expect(before).not.toHaveBeenCalled();
+ expect(after).not.toHaveBeenCalled();
+ });
it("omits content from the projection when no provider asked for it", async () => {
const modelStarted = vi.fn();
const modelCompleted = vi.fn();
- const toolStarted = vi.fn();
- const toolCompleted = vi.fn();
const hooks = createInstrumentationHooks([
{
events: {
"model.call.completed": modelCompleted,
"model.call.started": modelStarted,
- "tool.call.completed": toolCompleted,
- "tool.call.started": toolStarted,
},
name: "metadata-only",
},
]);
- const bridge = createAiSdkHookBridge(scope, hooks, undefined, undefined, () => true);
+ const run = vi.fn(async (_operation, execute) => execute());
+ const bridge = createAiSdkHookBridge(scope, hooks, run, undefined, () => true);
const toolCall = { input: { q: "eve" }, toolCallId: "tool-1", toolName: "search" };
await Reflect.apply(bridge.onLanguageModelCallStart!, bridge, [
@@ -727,6 +682,9 @@ describe("createAiSdkHookBridge", () => {
},
]);
await Reflect.apply(bridge.onToolExecutionStart!, bridge, [{ callId: "call-1", toolCall }]);
+ await Reflect.apply(bridge.executeTool!, bridge, [
+ { toolCallId: "tool-1", execute: () => Promise.resolve("ok") },
+ ]);
await Reflect.apply(bridge.onToolExecutionEnd!, bridge, [
{
callId: "call-1",
@@ -737,19 +695,19 @@ describe("createAiSdkHookBridge", () => {
]);
expect(modelStarted.mock.calls[0]?.[0].input).toBeUndefined();
- expect(toolStarted.mock.calls[0]?.[0]).toMatchObject({
- frameworkTool: true,
- });
expect(modelCompleted.mock.calls[0]?.[0].content).toBeUndefined();
// Structure survives: usage, the finish reason, and the tool's identity are
// not what was said.
expect(modelCompleted.mock.calls[0]?.[0].finishReason).toBe("stop");
- expect(toolStarted.mock.calls[0]?.[0].input).toBeUndefined();
- expect(toolStarted.mock.calls[0]?.[0].toolName).toBe("search");
- expect(toolCompleted.mock.calls[0]?.[0].output).toEqual({ type: "result" });
+ expect(run.mock.calls[0]?.[0]).toMatchObject({
+ type: "tool.call",
+ toolName: "search",
+ frameworkTool: true,
+ input: undefined,
+ });
});
- it("withholds content from a metadata provider sharing a bus with a content one", async () => {
+ it("does not publish an extra tool start from the SDK callback", async () => {
const metadataOnly = vi.fn();
const wantsContent = vi.fn();
const hooks = createInstrumentationHooks([
@@ -766,9 +724,8 @@ describe("createAiSdkHookBridge", () => {
{ callId: "call-1", toolCall: { input: { q: "eve" }, toolCallId: "t", toolName: "search" } },
]);
- expect(wantsContent.mock.calls[0]?.[0].input).toEqual({ q: "eve" });
- expect(metadataOnly.mock.calls[0]?.[0].input).toBeUndefined();
- expect(metadataOnly.mock.calls[0]?.[0].toolName).toBe("search");
+ expect(wantsContent).not.toHaveBeenCalled();
+ expect(metadataOnly).not.toHaveBeenCalled();
});
it("keeps each provider's state to itself", async () => {
const observed = new Map<string, unknown>();
@@ -836,25 +793,21 @@ describe("createAiSdkHookBridge", () => {
const resolvers = new Map<string, () => void>();
const started = new Map<string, string>();
const terminalStates = new Map<string, unknown>();
- const hooks = createInstrumentationHooks([
- {
- events: {
- async "tool.call.started"(event) {
- started.set(
- event.idempotencyKey,
- await new Promise<string>((resolve) => {
- resolvers.set(event.idempotencyKey, () => resolve(`state:${event.idempotencyKey}`));
- }),
- );
- },
- "tool.call.completed"(event) {
- terminalStates.set(event.idempotencyKey, started.get(event.idempotencyKey));
- },
- },
- name: "parallel",
- },
- ]);
- const bridge = createAiSdkHookBridge(scope, hooks);
+ const hooks = createInstrumentationHooks([]);
+ const run: InstrumentationContextRunner = async (operation, execute) => {
+ started.set(
+ operation.idempotencyKey,
+ await new Promise<string>((resolve) => {
+ resolvers.set(operation.idempotencyKey, () =>
+ resolve(`state:${operation.idempotencyKey}`),
+ );
+ }),
+ );
+ const result = await execute();
+ terminalStates.set(operation.idempotencyKey, started.get(operation.idempotencyKey));
+ return result;
+ };
+ const bridge = createAiSdkHookBridge(scope, hooks, run);
const start = (toolCallId: string) =>
Reflect.apply(bridge.onToolExecutionStart!, bridge, [
{
@@ -862,8 +815,13 @@ describe("createAiSdkHookBridge", () => {
toolCall: { input: {}, toolCallId, toolName: "search" },
},
]);
- const first = start("tool-1");
- const second = start("tool-2");
+ await Promise.all([start("tool-1"), start("tool-2")]);
+ const first = Reflect.apply(bridge.executeTool!, bridge, [
+ { toolCallId: "tool-1", execute: () => Promise.resolve() },
+ ]);
+ const second = Reflect.apply(bridge.executeTool!, bridge, [
+ { toolCallId: "tool-2", execute: () => Promise.resolve() },
+ ]);
await vi.waitFor(() => expect(resolvers.size).toBe(2));
const firstId = `tool:${scope.attemptId}:tool-1:0`;
diff --git a/packages/eve/src/instrumentation/ai-sdk-hook-bridge.ts b/packages/eve/src/instrumentation/ai-sdk-hook-bridge.ts
index a0097eecb..9b295e5ba 100644
--- a/packages/eve/src/instrumentation/ai-sdk-hook-bridge.ts
+++ b/packages/eve/src/instrumentation/ai-sdk-hook-bridge.ts
@@ -9,9 +9,7 @@ import type {
InstrumentationModelCallCompletedEvent,
InstrumentationModelCallStartedEvent,
InstrumentationOperationRef,
- InstrumentationToolCallCompletedEvent,
InstrumentationToolCallStartedEvent,
- InstrumentationToolOutput,
InstrumentationUsage,
} from "#instrumentation/lifecycle.js";
import {
@@ -32,6 +30,7 @@ interface AttemptState {
readonly runtimeContext?: Readonly<Record<string, unknown>>;
readonly scope: InstrumentationAttemptScope;
readonly toolKeys: Map<string, string>;
+ readonly toolCalls: Map<string, InstrumentationToolCallStartedEvent>;
readonly isFrameworkTool?: (name: string) => boolean;
operation?: InstrumentationOperationRef;
// Only the number is kept: it disambiguates call identities within an attempt.
@@ -63,6 +62,7 @@ export function createAiSdkHookBridge(
: undefined,
scope,
toolKeys: new Map(),
+ toolCalls: new Map(),
isFrameworkTool,
};
const nextModelCallKey = (stepNumber: number): string => {
@@ -136,7 +136,7 @@ export function createAiSdkHookBridge(
}),
);
},
- async onToolExecutionStart(event) {
+ onToolExecutionStart(event) {
const key = toolCallIdempotencyKey(
state.scope,
event.toolCall.toolCallId,
@@ -144,21 +144,20 @@ export function createAiSdkHookBridge(
);
state.toolKeys.set(event.toolCall.toolCallId, key);
const started = toToolCallStarted(state, key, event);
- await hooks.publish(started);
+ state.toolCalls.set(event.toolCall.toolCallId, started);
},
executeTool({ toolCallId, execute }) {
- const key = state.toolKeys.get(toolCallId);
- return key === undefined
+ const started = state.toolCalls.get(toolCallId);
+ return started === undefined
? execute()
- : runInContext({ idempotencyKey: key, scope, type: "tool.call" }, execute);
+ : runInContext({ ...started, type: "tool.call" }, execute);
},
- async onToolExecutionEnd(event) {
+ onToolExecutionEnd(event) {
const toolCallId = event.toolCall.toolCallId;
const key = state.toolKeys.get(toolCallId);
if (key === undefined) return;
state.toolKeys.delete(toolCallId);
- const completed = toToolCallCompleted(state, key, event);
- await hooks.publish(completed);
+ state.toolCalls.delete(toolCallId);
},
async onAbort(event) {
await failOpenOperations(event.reason);
@@ -175,14 +174,10 @@ export function createAiSdkHookBridge(
hooks.publish(Object.freeze({ error, idempotencyKey, scope, type: "model.call.failed" })),
);
}
- for (const idempotencyKey of state.toolKeys.values()) {
- pending.push(
- hooks.publish(Object.freeze({ error, idempotencyKey, scope, type: "tool.call.failed" })),
- );
- }
state.modelKeys.clear();
state.modelCallStartEvents.clear();
state.toolKeys.clear();
+ state.toolCalls.clear();
await Promise.all(pending);
}
}
@@ -312,34 +307,7 @@ function toToolCallStarted(
input: state.capturesInputs ? source.toolCall.input : undefined,
scope: state.scope,
toolName: source.toolCall.toolName,
+ startedAtMs: Date.now(),
type: "tool.call.started",
});
}
-
-function toToolCallCompleted(
- state: AttemptState,
- idempotencyKey: string,
- source: TelemetryEvent<"onToolExecutionEnd">,
-): InstrumentationToolCallCompletedEvent {
- return Object.freeze({
- durationMs: source.toolExecutionMs,
- idempotencyKey,
- output: toToolOutput(source.toolOutput, state.capturesOutputs),
- scope: state.scope,
- type: "tool.call.completed",
- });
-}
-
-function toToolOutput(
- toolOutput: TelemetryEvent<"onToolExecutionEnd">["toolOutput"],
- capturesContent: boolean,
-): InstrumentationToolOutput {
- if (toolOutput.type === "tool-result") {
- return Object.freeze(
- capturesContent ? { output: toolOutput.output, type: "result" } : { type: "result" },
- );
- }
- return Object.freeze(
- capturesContent ? { error: toolOutput.error, type: "error" } : { type: "error" },
- );
-}
diff --git a/packages/eve/src/instrumentation/content.ts b/packages/eve/src/instrumentation/content.ts
index fba17b34a..ed08f7daa 100644
--- a/packages/eve/src/instrumentation/content.ts
+++ b/packages/eve/src/instrumentation/content.ts
@@ -26,12 +26,6 @@ export function withInstrumentationDecision(
return decision.recordInputs ? event : Object.freeze({ ...event, title: undefined });
case "channel.delivery.started":
return decision.recordInputs ? event : Object.freeze({ ...event, input: undefined });
- case "action.started":
- return decision.recordInputs ? event : Object.freeze({ ...event, input: undefined });
- case "action.completed":
- return decision.recordOutputs
- ? event
- : Object.freeze({ ...event, output: Object.freeze({ type: event.output.type }) });
case "input.requested":
return decision.recordOutputs ? event : Object.freeze({ ...event, request: undefined });
case "input.resolved":
@@ -61,7 +55,6 @@ export function withInstrumentationDecision(
...event,
providerMetadata: structuralProviderMetadata(event.providerMetadata),
});
- case "action.failed":
case "model.call.failed":
case "memory.operation.failed":
case "session.failed":
diff --git a/packages/eve/src/instrumentation/dispatch.ts b/packages/eve/src/instrumentation/dispatch.ts
index 157e06fc3..96e4f25c9 100644
--- a/packages/eve/src/instrumentation/dispatch.ts
+++ b/packages/eve/src/instrumentation/dispatch.ts
@@ -18,7 +18,7 @@ import type { TraceCaptureContext } from "#shared/trace-policy.js";
import type {
CreateInstrumentationHooksOptions,
- InstrumentationActionFailedEvent,
+ InstrumentationToolCallFailedEvent,
InstrumentationDispatchGroups,
InstrumentationEvent,
InstrumentationEventHandler,
@@ -86,21 +86,11 @@ export function createInstrumentationDispatcher(
);
const failure = terminalActionFailure(snapshot);
for (const action of pendingActions) {
- if (action.toolCall !== undefined) {
- await publish(action.toolCall);
- await publish({
- type: "tool.call.failed",
- idempotencyKey: action.toolCall.idempotencyKey,
- scope: action.scope,
- completedAtMs: Date.now(),
- error: failure.error,
- });
- }
await publish({
...failure,
idempotencyKey: action.idempotencyKey,
scope: action.scope,
- type: "action.failed",
+ type: "tool.call.failed",
});
}
}
@@ -164,7 +154,13 @@ export function createInstrumentationDispatcher(
}
};
- return { capturesContent, capturesInputs, capturesOutputs, forTrace, publish };
+ return {
+ capturesContent,
+ capturesInputs,
+ capturesOutputs,
+ forTrace,
+ publish,
+ };
}
let loggedUnboundPublish = false;
@@ -317,7 +313,7 @@ async function withTimeout(
}
}
-/** Model and SDK tool children are scoped to an attempt; durable pairs are not. */
+/** Model calls are attempt-scoped; tool calls and input waits can outlive an attempt. */
function stateOwner(event: InstrumentationEvent): InstrumentationStateOwner {
if (
event.type === "channel.delivery.started" ||
@@ -335,12 +331,10 @@ function stateOwner(event: InstrumentationEvent): InstrumentationStateOwner {
return { sessionId: event.sessionId, turnId: event.turnId };
}
if (!("scope" in event)) return {};
- if (event.type.startsWith("action.") || event.type.startsWith("input.")) {
+ if (event.type.startsWith("tool.call.") || event.type.startsWith("input.")) {
return { sessionId: event.scope.sessionId, turnId: event.scope.turnId };
}
- return event.type.startsWith("model.call.") ||
- event.type.startsWith("tool.call.") ||
- event.type.startsWith("step.attempt.")
+ return event.type.startsWith("model.call.") || event.type.startsWith("step.attempt.")
? { attemptId: event.scope.attemptId }
: {};
}
@@ -364,7 +358,7 @@ function terminalActionFailure(
| InstrumentationSessionSettledEvent
| InstrumentationTurnFailedEvent
| InstrumentationTurnSettledEvent,
-): Pick<InstrumentationActionFailedEvent, "error" | "errorCode" | "outcome"> {
+): Pick<InstrumentationToolCallFailedEvent, "error" | "errorCode" | "outcome"> {
if (event.type === "session.failed" || event.type === "turn.failed") {
return { error: event.error, outcome: "failed" };
}
diff --git a/packages/eve/src/instrumentation/lifecycle.test.ts b/packages/eve/src/instrumentation/lifecycle.test.ts
index cd0eeefd4..e8ef8b2d9 100644
--- a/packages/eve/src/instrumentation/lifecycle.test.ts
+++ b/packages/eve/src/instrumentation/lifecycle.test.ts
@@ -262,7 +262,7 @@ describe("provider state lifecycle", () => {
const actionKey = actionIdempotencyKey(scope.sessionId, scope.turnId, "call-1");
const hooks = createInstrumentationHooks([
{
- events: { "action.started": (_event, ctx) => ctx.state.set("open") },
+ events: { "tool.call.started": (_event, ctx) => ctx.state.set("open") },
name: "sink",
},
]);
@@ -272,9 +272,9 @@ describe("provider state lifecycle", () => {
idempotencyKey: actionKey,
input: {},
kind: "tool-call",
- name: "tool",
+ toolName: "tool",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await hooks.publish({
idempotencyKey: attemptIdempotencyKey(scope),
@@ -291,8 +291,8 @@ describe("provider state lifecycle", () => {
const hooks = createInstrumentationHooks([
{
events: {
- "action.failed": failed,
- "action.started": (_event, ctx) => ctx.state.set("open"),
+ "tool.call.failed": failed,
+ "tool.call.started": (_event, ctx) => ctx.state.set("open"),
},
name: "sink",
},
@@ -304,9 +304,9 @@ describe("provider state lifecycle", () => {
idempotencyKey: actionKey,
input: {},
kind: "tool-call",
- name: "tool",
+ toolName: "tool",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await hooks.publish({
idempotencyKey: turnIdempotencyKey(scope.sessionId, scope.turnId),
@@ -321,7 +321,7 @@ describe("provider state lifecycle", () => {
expect(failed.mock.calls[0]?.[0]).toMatchObject({
errorCode: "ACTION_CANCELLED",
outcome: "cancelled",
- type: "action.failed",
+ type: "tool.call.failed",
});
});
});
@@ -892,7 +892,7 @@ describe("trace policies", () => {
const observed = vi.fn();
const hooks = createInstrumentationHooks([
{
- events: { "action.started": observed },
+ events: { "tool.call.started": observed },
name: "private-audit",
tracePolicy: () => ({ emit: true, recordInputs: true, recordOutputs: false }),
},
@@ -903,9 +903,9 @@ describe("trace policies", () => {
idempotencyKey: actionIdempotencyKey(scope.sessionId, scope.turnId, "call-1"),
input: { secret: "private" },
kind: "tool-call",
- name: "weather",
+ toolName: "weather",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
expect(observed.mock.calls[0]?.[0].input).toEqual({ secret: "private" });
@@ -1137,9 +1137,9 @@ describe("trace policies", () => {
const metadataOnly = vi.fn();
const wantsContent = vi.fn();
const hooks = createInstrumentationHooks([
- { events: { "action.failed": metadataOnly }, name: "metadata" },
+ { events: { "tool.call.failed": metadataOnly }, name: "metadata" },
{
- events: { "action.failed": wantsContent },
+ events: { "tool.call.failed": wantsContent },
name: "content",
tracePolicy: () => ({ emit: true, recordInputs: true, recordOutputs: true }),
},
@@ -1153,14 +1153,14 @@ describe("trace policies", () => {
idempotencyKey: actionIdempotencyKey(scope.sessionId, scope.turnId, "call-1"),
outcome: "failed",
scope: actionScope,
- type: "action.failed",
+ type: "tool.call.failed",
});
expect(metadataOnly.mock.calls[0]?.[0]).toMatchObject({
error: undefined,
errorCode: "SUBAGENT_EXECUTION_FAILED",
outcome: "failed",
- type: "action.failed",
+ type: "tool.call.failed",
});
expect(Object.isFrozen(metadataOnly.mock.calls[0]?.[0])).toBe(true);
expect(wantsContent.mock.calls[0]?.[0].error).toEqual(error);
@@ -1171,7 +1171,7 @@ describe("trace policies", () => {
it("keeps action outcome and usage when output content is withheld", async () => {
const metadataOnly = vi.fn();
const hooks = createInstrumentationHooks([
- { events: { "action.completed": metadataOnly }, name: "metadata" },
+ { events: { "tool.call.completed": metadataOnly }, name: "metadata" },
]);
await hooks.publish({
acceptedAtMs: 1_234,
@@ -1179,7 +1179,7 @@ describe("trace policies", () => {
outcome: "completed",
output: { output: "private result", type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
usage: { inputTokens: 10, outputTokens: 5 },
});
diff --git a/packages/eve/src/instrumentation/lifecycle.ts b/packages/eve/src/instrumentation/lifecycle.ts
index c68db4ea0..4292104dd 100644
--- a/packages/eve/src/instrumentation/lifecycle.ts
+++ b/packages/eve/src/instrumentation/lifecycle.ts
@@ -83,12 +83,12 @@ export type InstrumentationContentPart =
};
/**
- * How one action ended.
+ * How one tool call ended.
*
* `type` survives a provider that declined content, so whether the tool errored
* is answerable without seeing what it returned.
*/
-export type InstrumentationActionOutput =
+export type InstrumentationToolOutput =
| { readonly type: "result"; readonly output?: unknown }
| { readonly type: "error"; readonly error?: unknown };
@@ -450,16 +450,18 @@ export type InstrumentationModelCallTerminalEvent =
| InstrumentationModelCallCompletedEvent
| InstrumentationModelCallFailedEvent;
-export type InstrumentationToolOutput = InstrumentationActionOutput;
-
-/** How eve dispatches the action. */
-export type InstrumentationActionKind =
+/** How eve runs the tool call. */
+export type InstrumentationToolCallKind =
| "load-skill"
| "remote-agent-call"
| "subagent-call"
| "tool-call";
export interface InstrumentationToolCallStartedEvent {
+ /** Durable dispatch metadata, when this call is owned by eve's tool loop. */
+ readonly kind?: InstrumentationToolCallKind;
+ readonly parentCallId?: string;
+ readonly isWorkflowTool?: boolean;
readonly frameworkTool?: boolean;
readonly startedAtMs?: number;
readonly type: "tool.call.started";
@@ -471,6 +473,9 @@ export interface InstrumentationToolCallStartedEvent {
}
export interface InstrumentationToolCallCompletedEvent {
+ readonly outcome?: InstrumentationToolCallOutcome;
+ readonly usage?: InstrumentationUsage;
+ readonly acceptedAtMs?: number;
readonly completedAtMs?: number;
readonly type: "tool.call.completed";
/** How long the tool's `execute` ran, in milliseconds, when the publisher measured it. */
@@ -481,6 +486,10 @@ export interface InstrumentationToolCallCompletedEvent {
}
export interface InstrumentationToolCallFailedEvent {
+ readonly outcome?: Exclude<InstrumentationToolCallOutcome, "completed">;
+ readonly usage?: InstrumentationUsage;
+ readonly acceptedAtMs?: number;
+ readonly errorCode?: string;
readonly completedAtMs?: number;
readonly type: "tool.call.failed";
/** Content. Absent unless this provider's trace policy records this direction. */
@@ -493,61 +502,13 @@ export type InstrumentationToolCallTerminalEvent =
| InstrumentationToolCallCompletedEvent
| InstrumentationToolCallFailedEvent;
-/**
- * One thing the agent did on the model's behalf. `kind` is what separates a
- * subagent or remote-agent call from an ordinary tool; `name` is the name the
- * model called, which is the tool name for every kind.
- */
-export interface InstrumentationActionStartedEvent {
- readonly type: "action.started";
- /** Time the request was observed, before durable event acceptance. */
- readonly startedAtMs?: number;
- /** The calling action, when this is a nested tool call. */
- readonly parentCallId?: string;
- readonly callId: string;
- readonly idempotencyKey: string;
- /** Content. Absent unless this provider's trace policy records this direction. */
- readonly input?: unknown;
- /** Whether this action owns a durable workflow body. */
- readonly isWorkflowTool?: boolean;
- readonly kind: InstrumentationActionKind;
- readonly name: string;
- readonly scope: InstrumentationAttemptScope;
-}
-
-export type InstrumentationActionOutcome =
+export type InstrumentationToolCallOutcome =
| "abandoned"
| "cancelled"
| "completed"
| "failed"
| "rejected";
-export interface InstrumentationActionCompletedEvent {
- readonly type: "action.completed";
- readonly acceptedAtMs?: number;
- readonly idempotencyKey: string;
- readonly outcome: "completed";
- readonly output: InstrumentationActionOutput;
- readonly scope: InstrumentationAttemptScope;
- readonly usage?: InstrumentationUsage;
-}
-
-export interface InstrumentationActionFailedEvent {
- readonly type: "action.failed";
- readonly acceptedAtMs?: number;
- /** Content. Absent unless this provider's trace policy records this direction. */
- readonly error?: unknown;
- readonly errorCode?: string;
- readonly idempotencyKey: string;
- readonly outcome: Exclude<InstrumentationActionOutcome, "completed">;
- readonly scope: InstrumentationAttemptScope;
- readonly usage?: InstrumentationUsage;
-}
-
-export type InstrumentationActionTerminalEvent =
- | InstrumentationActionCompletedEvent
- | InstrumentationActionFailedEvent;
-
/** The second argument to every handler. */
export interface InstrumentationHandlerContext {
/** Durable state scoped to this provider and this operation. */
@@ -597,9 +558,6 @@ export interface InstrumentationProviderDefinition {
readonly "session.failed"?: InstrumentationEventHandler<InstrumentationSessionFailedEvent>;
readonly "session.started"?: InstrumentationEventHandler<InstrumentationSessionStartedEvent>;
readonly "session.waiting"?: InstrumentationEventHandler<InstrumentationSessionSettledEvent>;
- readonly "action.started"?: InstrumentationEventHandler<InstrumentationActionStartedEvent>;
- readonly "action.completed"?: InstrumentationEventHandler<InstrumentationActionCompletedEvent>;
- readonly "action.failed"?: InstrumentationEventHandler<InstrumentationActionFailedEvent>;
readonly "tool.call.started"?: InstrumentationEventHandler<InstrumentationToolCallStartedEvent>;
readonly "tool.call.completed"?: InstrumentationEventHandler<InstrumentationToolCallCompletedEvent>;
readonly "tool.call.failed"?: InstrumentationEventHandler<InstrumentationToolCallFailedEvent>;
@@ -634,8 +592,6 @@ export type InstrumentationCorrelatedEvent =
| InstrumentationChannelDeliveryTerminalEvent
| InstrumentationInputRequestedEvent
| InstrumentationInputResolvedEvent
- | InstrumentationActionStartedEvent
- | InstrumentationActionTerminalEvent
| memory.InstrumentationMemoryOperationStartedEvent
| memory.InstrumentationMemoryOperationTerminalEvent
| InstrumentationModelCallStartedEvent
@@ -666,6 +622,13 @@ export type InstrumentationExecutionOperation =
readonly idempotencyKey: string;
readonly scope: InstrumentationAttemptScope;
readonly type: "tool.call";
+ readonly callId?: string;
+ readonly toolName?: string;
+ readonly input?: unknown;
+ readonly frameworkTool?: boolean;
+ readonly startedAtMs?: number;
+ readonly completedAtMs?: number;
+ readonly failed?: boolean;
}
| {
readonly idempotencyKey: string;
diff --git a/packages/eve/src/instrumentation/native-events.test.ts b/packages/eve/src/instrumentation/native-events.test.ts
index 1538d58af..7ed6ef408 100644
--- a/packages/eve/src/instrumentation/native-events.test.ts
+++ b/packages/eve/src/instrumentation/native-events.test.ts
@@ -391,69 +391,74 @@ describe("createInstrumentationHandleEvent", () => {
expect(events.slice(0, 5)).toEqual([
{
callId: "delegate-1",
+ frameworkTool: false,
startedAtMs: 1_000,
idempotencyKey: actionIdempotencyKey("session-1", "turn-1", "delegate-1"),
input: { task: "research" },
kind: "subagent-call",
- name: "delegate",
+ toolName: "delegate",
isWorkflowTool: true,
scope,
- type: "action.started",
+ type: "tool.call.started",
},
{
callId: "skill-1",
+ frameworkTool: false,
startedAtMs: 1_000,
idempotencyKey: actionIdempotencyKey("session-1", "turn-1", "skill-1"),
input: { name: "research" },
kind: "load-skill",
- name: "load_skill",
+ toolName: "load_skill",
scope,
- type: "action.started",
+ type: "tool.call.started",
},
{
callId: "remote-1",
+ frameworkTool: false,
startedAtMs: 1_000,
idempotencyKey: actionIdempotencyKey("session-1", "turn-1", "remote-1"),
input: { task: "analyze" },
kind: "remote-agent-call",
- name: "remote",
+ toolName: "remote",
isWorkflowTool: true,
scope,
- type: "action.started",
+ type: "tool.call.started",
},
{
callId: "add-1",
+ frameworkTool: false,
startedAtMs: 1_000,
idempotencyKey: actionIdempotencyKey("session-1", "turn-1", "add-1"),
input: { a: 1, b: 2 },
kind: "tool-call",
- name: "add",
+ toolName: "add",
scope,
- type: "action.started",
+ type: "tool.call.started",
},
{
callId: "workflow-1",
+ frameworkTool: false,
startedAtMs: 1_000,
idempotencyKey: actionIdempotencyKey("session-1", "turn-1", "workflow-1"),
input: { report: "weekly" },
isWorkflowTool: true,
kind: "tool-call",
- name: "publish",
+ toolName: "publish",
scope,
- type: "action.started",
+ type: "tool.call.started",
},
]);
const terminals = events.filter(
(event) =>
- (event as { type: string }).type === "action.failed" ||
- (event as { type: string }).type === "action.completed",
+ (event as { type: string }).type === "tool.call.failed" ||
+ (event as { type: string }).type === "tool.call.completed",
);
expect(terminals[0]).toMatchObject({
errorCode: "ACTION_RESULT_FAILED",
idempotencyKey: actionIdempotencyKey("session-1", "turn-1", "delegate-1"),
outcome: "failed",
scope,
- type: "action.failed",
+ type: "tool.call.failed",
usage: {
costUsd: 0.003,
inputTokenDetails: { cacheReadTokens: 1, cacheWriteTokens: 2 },
@@ -467,7 +472,7 @@ describe("createInstrumentationHandleEvent", () => {
outcome: "completed",
output: { output: "done", type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
usage: {
costUsd: 0.012,
inputTokenDetails: { cacheReadTokens: 3, cacheWriteTokens: 4 },
@@ -481,10 +486,10 @@ describe("createInstrumentationHandleEvent", () => {
outcome: "completed",
output: { output: 3, type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
usage: undefined,
});
- expect(events).toHaveLength(12);
+ expect(events).toHaveLength(8);
expect(events.every(Object.isFrozen)).toBe(true);
});
diff --git a/packages/eve/src/instrumentation/native-events.ts b/packages/eve/src/instrumentation/native-events.ts
index f949991b8..b340034da 100644
--- a/packages/eve/src/instrumentation/native-events.ts
+++ b/packages/eve/src/instrumentation/native-events.ts
@@ -2,8 +2,8 @@ import type { UnstampedMessageStreamEvent } from "#protocol/message.js";
import { contextStorage } from "#context/container.js";
import { instrumentChannelDelivery } from "#instrumentation/channel-delivery.js";
import type {
- InstrumentationActionFailedEvent,
- InstrumentationActionStartedEvent,
+ InstrumentationToolCallFailedEvent,
+ InstrumentationToolCallStartedEvent,
InstrumentationAttemptScope,
InstrumentationHooks,
InstrumentationInputRequestedEvent,
@@ -12,7 +12,6 @@ import type {
InstrumentationPointEvent,
InstrumentationTraceContext,
InstrumentationUsage,
- InstrumentationToolCallCompletedEvent,
} from "#instrumentation/lifecycle.js";
import {
actionIdempotencyKey,
@@ -241,10 +240,11 @@ async function publishActionStarts(
input: capturesInputs ? action.input : undefined,
...(deferred ? { isWorkflowTool: true } : undefined),
kind: action.kind === "workflow-tool-call" ? "tool-call" : action.kind,
- name: actionName(action),
+ toolName: actionName(action),
+ frameworkTool: input.isFrameworkTool?.(actionName(action)) === true,
scope,
- type: "action.started",
- } satisfies InstrumentationActionStartedEvent),
+ type: "tool.call.started",
+ } satisfies InstrumentationToolCallStartedEvent),
);
}
}
@@ -261,24 +261,6 @@ async function publishActionTerminal(
if (correlation === undefined) return;
const { idempotencyKey, scope } = correlation;
const capturesOutputs = hooks.capturesOutputs ?? hooks.capturesContent;
- if (correlation.toolCall !== undefined) {
- await hooks.publish(Object.freeze(correlation.toolCall));
- await hooks.publish(
- Object.freeze({
- type: "tool.call.completed",
- idempotencyKey: correlation.toolCall.idempotencyKey,
- scope,
- completedAtMs:
- contextStorage.getStore()?.get(RuntimeActionSettlementTimesKey)?.[
- event.data.result.callId
- ] ?? Date.now(),
- output:
- event.data.status === "completed"
- ? { type: "result", output: capturesOutputs ? event.data.result.output : undefined }
- : { type: "error", error: capturesOutputs ? event.data.result.output : undefined },
- } satisfies InstrumentationToolCallCompletedEvent),
- );
- }
if (event.data.status === "completed") {
await hooks.publish(
@@ -294,7 +276,7 @@ async function publishActionTerminal(
: { type: "result" },
),
scope,
- type: "action.completed",
+ type: "tool.call.completed",
usage: actionUsage(event.data.result),
}),
);
@@ -316,9 +298,9 @@ async function publishActionTerminal(
idempotencyKey,
outcome: event.data.status,
scope,
- type: "action.failed",
+ type: "tool.call.failed",
usage: actionUsage(event.data.result),
- } satisfies InstrumentationActionFailedEvent),
+ } satisfies InstrumentationToolCallFailedEvent),
);
}
diff --git a/packages/eve/src/instrumentation/provider.ts b/packages/eve/src/instrumentation/provider.ts
index cc851a07e..de1c0f492 100644
--- a/packages/eve/src/instrumentation/provider.ts
+++ b/packages/eve/src/instrumentation/provider.ts
@@ -15,12 +15,6 @@ import type { TraceCapturePolicy } from "#shared/trace-policy.js";
export type { JsonValue } from "#shared/json.js";
export type {
- InstrumentationActionCompletedEvent,
- InstrumentationActionFailedEvent,
- InstrumentationActionKind,
- InstrumentationActionOutcome,
- InstrumentationActionOutput,
- InstrumentationActionStartedEvent,
InstrumentationAttemptScope,
InstrumentationChannelDeliveryInput,
InstrumentationChannelDeliveryOutcome,
@@ -54,6 +48,8 @@ export type {
InstrumentationToolCallCompletedEvent,
InstrumentationToolCallFailedEvent,
InstrumentationToolCallStartedEvent,
+ InstrumentationToolCallKind,
+ InstrumentationToolCallOutcome,
InstrumentationToolOutput,
InstrumentationTraceContext,
InstrumentationTurnFailedEvent,
@@ -62,6 +58,7 @@ export type {
InstrumentationTurnTerminalEvent,
InstrumentationUsage,
} from "#instrumentation/lifecycle.js";
+
export type {
InstrumentationMemoryOperation,
InstrumentationMemoryOperationCompletedEvent,
diff --git a/packages/eve/src/instrumentation/providers.test.ts b/packages/eve/src/instrumentation/providers.test.ts
index 8177eef76..ed8feeea1 100644
--- a/packages/eve/src/instrumentation/providers.test.ts
+++ b/packages/eve/src/instrumentation/providers.test.ts
@@ -240,6 +240,64 @@ describe("finalizeInstrumentationProviders", () => {
expect(started.mock.calls[0]?.[0]).toMatchObject({ turnId: "turn-1" });
});
+ it("publishes one durable tool lifecycle without duplicate SDK callbacks", async () => {
+ const started = vi.fn((_event, ctx) => ctx.state.set({ started: true }));
+ const completed = vi.fn((_event, ctx) => expect(ctx.state.get()).toEqual({ started: true }));
+ await register(
+ "audit",
+ defineInstrumentation({
+ events: {
+ "tool.call.started": started,
+ "tool.call.completed": completed,
+ },
+ }),
+ );
+ const runtime = finalizeInstrumentationProviders({ serviceName: "weather-agent" });
+ const hooks = runtime.hooks.forTrace!(traceContext("unknown"));
+ const scope = {
+ attemptId: "attempt",
+ attemptIndex: 0,
+ sessionId: "session-1",
+ turnId: "turn-1",
+ stepIndex: 0,
+ };
+ await hooks.publish({
+ type: "tool.call.started",
+ idempotencyKey: "call",
+ callId: "call-1",
+ toolName: "weather",
+ kind: "tool-call",
+ input: undefined,
+ scope,
+ });
+ await runtime.runInContext(
+ {
+ type: "tool.call",
+ idempotencyKey: "execution",
+ callId: "call-1",
+ toolName: "weather",
+ input: undefined,
+ scope,
+ },
+ () => Promise.resolve(),
+ );
+ await hooks.publish({ type: "step.attempt.completed", idempotencyKey: "attempt", scope });
+ await hooks.publish({
+ type: "tool.call.completed",
+ idempotencyKey: "call",
+ outcome: "completed",
+ usage: { inputTokens: 10 },
+ output: { type: "result" },
+ scope,
+ });
+ expect(started).toHaveBeenCalledOnce();
+ expect(completed).toHaveBeenCalledOnce();
+ expect(completed.mock.calls[0]?.[0]).toMatchObject({
+ outcome: "completed",
+ usage: { inputTokens: 10 },
+ });
+ });
+
it("still runs execution when no destination was declared", async () => {
// A directory with no `otel()` has nothing to hang a span on, so
// `runInContext` degrades to running the work directly rather than
diff --git a/packages/eve/src/instrumentation/runtime.ts b/packages/eve/src/instrumentation/runtime.ts
index 95ecfcf02..02348942c 100644
--- a/packages/eve/src/instrumentation/runtime.ts
+++ b/packages/eve/src/instrumentation/runtime.ts
@@ -517,36 +517,21 @@ export function bindInstrumentationRuntime(
const hooks = bindHooks(readSessionContext());
const scope = correlation.scope;
const idempotencyKey = toolCallIdempotencyKey(scope, input.callId, 0);
- await hooks.publish({
- type: "tool.call.started",
- callId: input.callId,
- idempotencyKey,
- scope,
- toolName: input.toolName,
- startedAtMs: input.startedAtMs,
- frameworkTool: true,
- input: (hooks.capturesInputs ?? hooks.capturesContent) ? input.input : undefined,
- });
- if (input.failed) {
- await hooks.publish({
- type: "tool.call.failed",
- idempotencyKey,
- scope,
- completedAtMs: input.completedAtMs,
- });
- } else {
- await hooks.publish({
- type: "tool.call.completed",
+ await runtime.runInContext(
+ {
+ type: "tool.call",
+ callId: input.callId,
idempotencyKey,
scope,
+ toolName: input.toolName,
+ startedAtMs: input.startedAtMs,
+ frameworkTool: true,
+ input: (hooks.capturesInputs ?? hooks.capturesContent) ? input.input : undefined,
completedAtMs: input.completedAtMs,
- durationMs: input.completedAtMs - input.startedAtMs,
- output: {
- type: "result",
- output: (hooks.capturesOutputs ?? hooks.capturesContent) ? input.output : undefined,
- },
- });
- }
+ failed: input.failed,
+ },
+ () => Promise.resolve(),
+ );
},
createHandleEvent: (input) => {
const sessionContext = readSessionContext();
diff --git a/packages/eve/src/instrumentation/state.ts b/packages/eve/src/instrumentation/state.ts
index cec0f8973..474dd0b6c 100644
--- a/packages/eve/src/instrumentation/state.ts
+++ b/packages/eve/src/instrumentation/state.ts
@@ -37,8 +37,8 @@ type InstrumentationScopeMap = Readonly<Record<string, InstrumentationAttemptSco
/**
* Provider state lives in serialized Workflow context, not in the harness, so a
- * value staged by `action.started` in one process is still there when
- * `action.completed` runs in another.
+ * value staged by `tool.call.started` in one process is still there when
+ * `tool.call.completed` runs in another.
*/
const InstrumentationStateKey = new ContextKey<InstrumentationStateMap>(
SERIALIZED_INSTRUMENTATION_STATE_KEYS.providerState,
diff --git a/packages/eve/src/public/instrumentation/provider.test.ts b/packages/eve/src/public/instrumentation/provider.test.ts
index a2e296b76..16a6ad3bd 100644
--- a/packages/eve/src/public/instrumentation/provider.test.ts
+++ b/packages/eve/src/public/instrumentation/provider.test.ts
@@ -40,8 +40,8 @@ describe("defineInstrumentation", () => {
it("infers terminal handler events from union-typed discriminants", () => {
const provider = defineInstrumentation({
events: {
- "action.completed": (event) => void [event.acceptedAtMs, event.outcome, event.usage],
- "action.failed": (event) => void [event.acceptedAtMs, event.errorCode, event.outcome],
+ "tool.call.completed": (event) => void [event.acceptedAtMs, event.outcome, event.usage],
+ "tool.call.failed": (event) => void [event.acceptedAtMs, event.errorCode, event.outcome],
"session.completed": (event) => void event.sessionId,
"step.attempt.completed": (event) => void event.scope,
"turn.failed": (event) => void event.error,
@@ -49,6 +49,12 @@ describe("defineInstrumentation", () => {
});
expect(isInstrumentationProvider(provider)).toBe(true);
+ defineInstrumentation({
+ events: {
+ // @ts-expect-error Action hooks were replaced by tool.call hooks.
+ "action.started": () => {},
+ },
+ });
});
it("exposes durable input request and resolution events", () => {
diff --git a/packages/eve/src/tracing/agent-action-instrumentation.ts b/packages/eve/src/tracing/agent-action-instrumentation.ts
deleted file mode 100644
index 185bafc9d..000000000
--- a/packages/eve/src/tracing/agent-action-instrumentation.ts
+++ /dev/null
@@ -1,277 +0,0 @@
-import {
- ROOT_CONTEXT,
- SpanKind,
- type Context,
- type Span,
- type SpanContext,
- type Tracer,
- trace,
-} from "#compiled/@opentelemetry/api/index.js";
-
-import type {
- InstrumentationActionKind,
- InstrumentationActionStartedEvent,
- InstrumentationActionTerminalEvent,
- InstrumentationAttemptScope,
- InstrumentationProviderDefinition,
-} from "#instrumentation/lifecycle.js";
-import { actionIdempotencyKey, attemptIdempotencyKey } from "#instrumentation/lifecycle.js";
-import { agentTraceIdentityAttributes, traceSessionIdOf } from "#tracing/agent-otel-attributes.js";
-import { contentAttribute, textContentAttribute } from "#tracing/agent-otel-content.js";
-import { setAgentUsage } from "#tracing/agent-otel-usage.js";
-import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";
-import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";
-import type { AgentActionTraceState, AgentTraceStateStore } from "#tracing/agent-trace-state.js";
-import { normalizeChannelAudience } from "#shared/channel-audience.js";
-import { isSampledTrace } from "#tracing/sampled-trace.js";
-import { withChannelAudience } from "#tracing/channel-audience-context.js";
-import { AGENT_SPAN_NAMES } from "#tracing/agent-span-contract.js";
-import { recordAgentSpanError as recordError } from "#tracing/agent-span-error.js";
-
-interface AgentActionInstrumentation {
- readonly events: Pick<
- NonNullable<InstrumentationProviderDefinition["events"]>,
- "action.completed" | "action.failed" | "action.started"
- >;
- deleteForSession(sessionId: string): void | PromiseLike<void>;
- failForAttempt(scope: InstrumentationAttemptScope, error: unknown): Promise<void>;
- contextFor(
- sessionId: string,
- turnId: string,
- callId: string,
- ): Promise<AgentActionContext | undefined>;
-}
-
-export interface AgentActionContext {
- readonly context: Context;
- readonly spanContext: SpanContext;
-}
-
-/** Builds durable `agent.action` spans around eve's runtime dispatch boundary. */
-export function createAgentActionInstrumentation(input: {
- readonly frameworkVersion: string;
- readonly idGenerator: AgentSpanIdGenerator;
- readonly recordInputs: boolean;
- readonly recordOutputs: boolean;
- readonly resolveTraceContext: (
- event: InstrumentationActionStartedEvent,
- ) => SpanContext | undefined | PromiseLike<SpanContext | undefined>;
- readonly stateStore: AgentTraceStateStore;
- readonly toolStartTimeForAction: (idempotencyKey: string) => number | undefined;
- readonly tracer: Tracer;
-}): AgentActionInstrumentation {
- const byAttempt = new Map<string, Set<string>>();
-
- const onStarted = async (event: InstrumentationActionStartedEvent): Promise<void> => {
- // SDK execution can precede acceptance of the streamed action request.
- const startTimeMs = Math.min(
- event.startedAtMs ?? Date.now(),
- input.toolStartTimeForAction(event.idempotencyKey) ?? Infinity,
- );
- const traceContext = await input.resolveTraceContext(event);
- if (traceContext === undefined || !isSampledTrace(traceContext)) return;
-
- const existing = await input.stateStore.getAction(event.idempotencyKey);
- const state: AgentActionTraceState = existing ?? {
- attemptIndex: event.scope.attemptIndex,
- callId: event.callId,
- channelAudience: normalizeChannelAudience(event.scope.channelAudience),
- inputAttribute: input.recordInputs ? contentAttribute(event.input) : undefined,
- kind: event.kind,
- name: event.name,
- parent: {
- spanId: input.idGenerator.deriveSpanId(
- event.parentCallId === undefined
- ? attemptIdempotencyKey(event.scope)
- : `action:${actionIdempotencyKey(event.scope.sessionId, event.scope.turnId, event.parentCallId)}`,
- ),
- traceFlags: traceContext.traceFlags,
- traceId: traceContext.traceId,
- },
- parentCallId: event.parentCallId,
- rootSessionId: event.scope.rootSessionId ?? event.scope.sessionId,
- traceSessionId: traceSessionIdOf(event.scope),
- sessionId: event.scope.sessionId,
- spanId: input.idGenerator.deriveSpanId(`action:${event.idempotencyKey}`),
- startTimeMs,
- stepIndex: event.scope.stepIndex,
- turnId: event.scope.turnId,
- };
- await input.stateStore.setAction(event.idempotencyKey, state);
- if (event.isWorkflowTool === true) {
- await input.stateStore.setActionAnchor(event.idempotencyKey, state);
- }
- const keys = byAttempt.get(event.scope.attemptId) ?? new Set<string>();
- keys.add(event.idempotencyKey);
- byAttempt.set(event.scope.attemptId, keys);
- };
-
- const onTerminal = async (event: InstrumentationActionTerminalEvent): Promise<void> => {
- const state = await input.stateStore.getAction(event.idempotencyKey);
- if (state === undefined) return;
- try {
- finishActionSpan(state, event);
- } finally {
- await input.stateStore.deleteAction(event.idempotencyKey);
- forget(event.idempotencyKey);
- }
- };
-
- const startSpan = (state: AgentActionTraceState): Span => {
- const invocation = isAgentInvocation(state.kind);
- const span = input.idGenerator.withSpanId(state.spanId, () =>
- input.tracer.startSpan(
- AGENT_SPAN_NAMES.action,
- {
- attributes: {
- "agent.action.call_id": state.callId,
- "agent.action.kind": state.kind,
- ...(state.parentCallId === undefined
- ? undefined
- : { "agent.action.parent_call_id": state.parentCallId }),
- "agent.action.name": state.name,
- "agent.framework.name": "eve",
- "agent.framework.version": input.frameworkVersion,
- "agent.step.attempt": state.attemptIndex,
- "agent.step.index": state.stepIndex,
- "agent.turn.id": state.turnId,
- ...agentSpanNamingAttributes(AGENT_SPAN_NAMES.action),
- ...agentTraceIdentityAttributes({
- rootSessionId: state.rootSessionId,
- traceSessionId: state.traceSessionId,
- sessionId: state.sessionId,
- }),
- ...(invocation
- ? {
- "gen_ai.agent.name": state.name,
- "agent.invocation.role": "caller",
- }
- : undefined),
- },
- kind: state.kind === "remote-agent-call" ? SpanKind.CLIENT : SpanKind.INTERNAL,
- startTime: state.startTimeMs,
- },
- contextFromActionState(state),
- ),
- );
- if (!invocation && state.inputAttribute !== undefined) {
- span.setAttribute("gen_ai.tool.call.arguments", state.inputAttribute);
- }
- return span;
- };
-
- return {
- async contextFor(sessionId, turnId, callId) {
- const directKey = actionIdempotencyKey(sessionId, turnId, callId);
- const direct = await input.stateStore.getAction(directKey);
- if (direct !== undefined) return actionContext(direct);
- const state = await input.stateStore.findAction(sessionId, callId);
- return state === undefined ? undefined : actionContext(state);
- },
- async deleteForSession(sessionId) {
- await input.stateStore.deleteActions(sessionId);
- await input.stateStore.deleteActionAnchors(sessionId);
- },
- async failForAttempt(scope, error) {
- const keys = byAttempt.get(scope.attemptId);
- if (keys === undefined) return;
- byAttempt.delete(scope.attemptId);
- for (const key of keys) {
- const state = await input.stateStore.getAction(key);
- if (state === undefined) continue;
- const span = startSpan(state);
- recordError(span, error);
- span.end();
- await input.stateStore.deleteAction(key);
- }
- },
- events: {
- "action.completed": onTerminal,
- "action.failed": onTerminal,
- "action.started": onStarted,
- },
- };
-
- function forget(idempotencyKey: string): void {
- for (const [attemptId, keys] of byAttempt) {
- keys.delete(idempotencyKey);
- if (keys.size === 0) byAttempt.delete(attemptId);
- }
- }
-
- function finishActionSpan(
- state: AgentActionTraceState,
- event: InstrumentationActionTerminalEvent,
- ): void {
- const span = startSpan(state);
- span.setAttribute("agent.action.outcome", event.outcome);
- if (event.usage !== undefined) {
- setAgentUsage(span, event.usage);
- }
- if (event.type === "action.failed") {
- if (event.errorCode !== undefined) {
- span.setAttribute("agent.action.error.code", event.errorCode);
- }
- recordActionError(span, event.error, event.errorCode);
- } else if (event.output.type === "error") {
- recordActionError(span, event.output.error);
- } else {
- if (input.recordOutputs && !isAgentInvocation(state.kind)) {
- const result = contentAttribute(event.output.output);
- if (result !== undefined) span.setAttribute("gen_ai.tool.call.result", result);
- }
- }
- span.end(event.acceptedAtMs);
- }
-}
-
-function actionContext(state: AgentActionTraceState): AgentActionContext {
- const spanContext = {
- isRemote: false,
- spanId: state.spanId,
- traceFlags: state.parent.traceFlags,
- traceId: state.parent.traceId,
- };
- return {
- context: withChannelAudience(
- trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext(spanContext)),
- state.channelAudience,
- ),
- spanContext,
- };
-}
-
-function contextFromActionState(state: AgentActionTraceState): Context {
- return withChannelAudience(
- trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext({ ...state.parent, isRemote: false })),
- state.channelAudience,
- );
-}
-
-function isAgentInvocation(kind: InstrumentationActionKind): boolean {
- return kind === "subagent-call" || kind === "remote-agent-call";
-}
-
-function recordActionError(span: Span, error: unknown, errorType?: string): void {
- if (error instanceof Error || error === undefined) {
- recordError(span, error, errorType);
- return;
- }
- const detail = serializedErrorDetail(error);
- if (detail === undefined) {
- recordError(span, undefined, errorType);
- return;
- }
- const normalized = new Error(detail);
- if (errorType !== undefined) normalized.name = errorType;
- recordError(span, normalized, errorType);
-}
-
-function serializedErrorDetail(error: unknown): string | undefined {
- if (typeof error === "string") return textContentAttribute(error);
- const serialized = contentAttribute(error);
- if (typeof error !== "object" || error === null || Array.isArray(error)) return serialized;
- const message = Reflect.get(error, "message");
- if (typeof message !== "string") return serialized;
- return textContentAttribute(serialized === undefined ? message : `${message}\n${serialized}`);
-}
diff --git a/packages/eve/src/tracing/agent-approval-instrumentation.ts b/packages/eve/src/tracing/agent-approval-instrumentation.ts
index 5fde7f5d5..d922c9909 100644
--- a/packages/eve/src/tracing/agent-approval-instrumentation.ts
+++ b/packages/eve/src/tracing/agent-approval-instrumentation.ts
@@ -19,7 +19,7 @@ import { decodeTraceSessionId } from "#tracing/agent-trace-context-codec.js";
import { AGENT_SPAN_NAMES } from "#tracing/agent-span-contract.js";
import { recordAgentSpanError as recordError } from "#tracing/agent-span-error.js";
import { withChannelAudience } from "#tracing/channel-audience-context.js";
-import type { AgentActionContext } from "#tracing/agent-action-instrumentation.js";
+import type { AgentToolContext } from "#tracing/agent-tool-instrumentation.js";
import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";
import { normalizeChannelAudience, type ChannelAudience } from "#shared/channel-audience.js";
@@ -45,7 +45,7 @@ export function createAgentApprovalInstrumentation(input: {
sessionId: string,
turnId: string,
callId: string,
- ) => Promise<AgentActionContext | undefined>;
+ ) => Promise<AgentToolContext | undefined>;
readonly frameworkVersion: string;
readonly idGenerator: AgentSpanIdGenerator;
readonly tracer: Tracer;
diff --git a/packages/eve/src/tracing/agent-otel-provider.test.ts b/packages/eve/src/tracing/agent-otel-provider.test.ts
index cb5518f33..efeef6c74 100644
--- a/packages/eve/src/tracing/agent-otel-provider.test.ts
+++ b/packages/eve/src/tracing/agent-otel-provider.test.ts
@@ -32,7 +32,7 @@ import type { AgentTraceStateStore } from "#tracing/agent-trace-state.js";
import { InMemoryAgentTraceStateStore } from "#internal/testing/in-memory-agent-trace-state-store.js";
import {
createInstrumentationHooks,
- type InstrumentationActionKind,
+ type InstrumentationToolCallKind,
type InstrumentationAttemptScope,
type InstrumentationContextRunner,
type InstrumentationEvent,
@@ -140,7 +140,7 @@ async function emitAttempt(input: {
readonly parentTraceContext?: InstrumentationTraceContext;
readonly runInContext: InstrumentationContextRunner;
readonly providerMetadata?: Readonly<Record<string, unknown>>;
- readonly actionKind?: InstrumentationActionKind;
+ readonly actionKind?: InstrumentationToolCallKind;
readonly runtimeContext?: Readonly<Record<string, unknown>>;
readonly sessionId: string;
readonly skipModelTerminal?: boolean;
@@ -234,9 +234,9 @@ async function emitAttempt(input: {
idempotencyKey: actionKey,
input: { secret: "value" },
kind: input.actionKind ?? "tool-call",
- name: "weather",
+ toolName: "weather",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await Reflect.apply(bridge.onToolExecutionStart!, bridge, [
{
@@ -269,7 +269,7 @@ async function emitAttempt(input: {
outcome: "completed",
output: { output: { temperature: 72 }, type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
usage: input.actionUsage,
}
: {
@@ -278,7 +278,7 @@ async function emitAttempt(input: {
idempotencyKey: actionKey,
outcome: "failed",
scope,
- type: "action.failed",
+ type: "tool.call.failed",
},
);
}
@@ -1425,7 +1425,6 @@ describe("createAgentOtelInstrumentation", () => {
const turn = byName(spans, "invoke_agent weather")[0]!;
const step = byName(spans, "agent.step")[0]!;
const model = byName(spans, "chat claude-test")[0]!;
- const action = byName(spans, "agent.action")[0]!;
const tool = byName(spans, "execute_tool weather")[0]!;
expect(byName(spans, "agent.session")).toHaveLength(0);
@@ -1445,13 +1444,14 @@ describe("createAgentOtelInstrumentation", () => {
apiTrace.getSpan(parent as never)?.spanContext().spanId === model.spanContext().spanId,
),
).toBe(true);
- expect(action.parentSpanContext?.spanId).toBe(step.spanContext().spanId);
- expect(tool.parentSpanContext?.spanId).toBe(action.spanContext().spanId);
+ expect(tool.parentSpanContext?.spanId).toBe(step.spanContext().spanId);
+ expect(byName(spans, "agent.action")).toHaveLength(0);
+ expect(byName(spans, "execute_tool weather")).toHaveLength(1);
expect(turn.attributes["agent.channel.audience"]).toBe("public");
- for (const span of [step, model, action, tool]) {
+ for (const span of [step, model, tool]) {
expect(span.attributes).not.toHaveProperty("agent.channel.audience");
}
- for (const span of [turn, step, model, action, tool]) {
+ for (const span of [turn, step, model, tool]) {
expect(span.attributes).toMatchObject({
"gen_ai.conversation.id": "session-1",
});
@@ -1495,7 +1495,7 @@ describe("createAgentOtelInstrumentation", () => {
"gen_ai.usage.input_tokens": 10,
"gen_ai.usage.output_tokens": 5,
});
- expect(action.attributes).toMatchObject({
+ expect(tool.attributes).toMatchObject({
"agent.action.kind": "tool-call",
"agent.action.name": "weather",
"agent.framework.name": "eve",
@@ -1511,7 +1511,7 @@ describe("createAgentOtelInstrumentation", () => {
});
});
- it("projects production workflow tool calls as agent.action spans", async () => {
+ it("projects production workflow tool calls as tool spans", async () => {
const context = new ContextContainer();
let runtime: TestRuntime | undefined;
@@ -1623,7 +1623,9 @@ describe("createAgentOtelInstrumentation", () => {
});
await runtime!.provider.forceFlush();
- const actions = byName(runtime!.exporter.getFinishedSpans(), "agent.action");
+ const actions = runtime!.exporter
+ .getFinishedSpans()
+ .filter((span) => span.attributes["gen_ai.operation.name"] === "execute_tool");
expect(actions.map((span) => span.attributes["agent.action.call_id"]).sort()).toEqual([
"wait",
"workflow",
@@ -1972,12 +1974,7 @@ describe("createAgentOtelInstrumentation", () => {
const firstSpans = first.exporter.getFinishedSpans();
const redeliverySpans = redelivery.exporter.getFinishedSpans();
- for (const name of [
- "invoke_agent weather",
- "agent.step",
- "agent.action",
- "execute_tool weather",
- ]) {
+ for (const name of ["invoke_agent weather", "agent.step", "execute_tool weather"]) {
expect(byName(redeliverySpans, name)[0]!.spanContext().spanId).toBe(
byName(firstSpans, name)[0]!.spanContext().spanId,
);
@@ -1990,7 +1987,7 @@ describe("createAgentOtelInstrumentation", () => {
);
});
- it("parents a tool to its action when SDK telemetry arrives first", async () => {
+ it("merges early SDK telemetry into one durable tool span", async () => {
const runtime = createRuntime();
const scope: InstrumentationAttemptScope = {
attemptId: "session-1:turn-1:0:0",
@@ -2015,21 +2012,20 @@ describe("createAgentOtelInstrumentation", () => {
scope,
type: "step.attempt.started",
});
- await runtime.hooks.publish({
- callId: "tool-1",
- idempotencyKey: toolKey,
- input: { secret: "value" },
- scope,
- toolName: "weather",
- type: "tool.call.started",
- });
- await runtime.hooks.publish({
- durationMs: 1500,
- idempotencyKey: toolKey,
- output: { output: "sunny", type: "result" },
- scope,
- type: "tool.call.completed",
- });
+ const executionStart = Date.now();
+ await runtime.runInContext(
+ {
+ callId: "tool-1",
+ idempotencyKey: toolKey,
+ input: { secret: "value" },
+ scope,
+ toolName: "weather",
+ type: "tool.call",
+ startedAtMs: executionStart,
+ completedAtMs: executionStart + 1500,
+ },
+ () => Promise.resolve("sunny"),
+ );
const later = Date.now() + 100;
const clock = vi.spyOn(Date, "now").mockReturnValue(later);
await runtime.hooks.publish({
@@ -2037,9 +2033,9 @@ describe("createAgentOtelInstrumentation", () => {
idempotencyKey: actionKey,
input: { secret: "value" },
kind: "tool-call",
- name: "weather",
+ toolName: "weather",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
clock.mockRestore();
await runtime.hooks.publish({
@@ -2047,23 +2043,20 @@ describe("createAgentOtelInstrumentation", () => {
outcome: "completed",
output: { output: "sunny", type: "result" },
scope,
- type: "action.completed",
- });
- const uncorrelatedToolKey = `tool:${scope.attemptId}:tool-2:0`;
- await runtime.hooks.publish({
- callId: "tool-2",
- idempotencyKey: uncorrelatedToolKey,
- input: {},
- scope,
- toolName: "final_output",
- type: "tool.call.started",
- });
- await runtime.hooks.publish({
- idempotencyKey: uncorrelatedToolKey,
- output: { output: "done", type: "result" },
- scope,
type: "tool.call.completed",
});
+ const uncorrelatedToolKey = `tool:${scope.attemptId}:tool-2:0`;
+ await runtime.runInContext(
+ {
+ callId: "tool-2",
+ idempotencyKey: uncorrelatedToolKey,
+ input: {},
+ scope,
+ toolName: "final_output",
+ type: "tool.call",
+ },
+ () => Promise.resolve("done"),
+ );
await runtime.hooks.publish({
idempotencyKey: attemptIdempotencyKey(scope),
scope,
@@ -2072,14 +2065,17 @@ describe("createAgentOtelInstrumentation", () => {
await runtime.provider.forceFlush();
const spans = runtime.exporter.getFinishedSpans();
- const action = byName(spans, "agent.action")[0]!;
const tool = byName(spans, "execute_tool weather")[0]!;
const uncorrelatedTool = byName(spans, "execute_tool final_output")[0]!;
- expect(tool.parentSpanContext?.spanId).toBe(action.spanContext().spanId);
- expect(nanos(action.startTime)).toBeLessThanOrEqual(nanos(tool.startTime));
+ expect(tool.parentSpanContext?.spanId).toBe(
+ byName(spans, "agent.step")[0]!.spanContext().spanId,
+ );
+ expect(byName(spans, "execute_tool weather")).toHaveLength(1);
+ expect(byName(spans, "agent.action")).toHaveLength(0);
+ expect(nanos(tool.startTime)).toBeLessThan(BigInt(later) * 1_000_000n);
// Seconds, as `@ai-sdk/otel` records it; a publisher that measured nothing sets none.
expect(tool.attributes["gen_ai.execute_tool.duration"]).toBe(1.5);
- expect(uncorrelatedTool.attributes["gen_ai.execute_tool.duration"]).toBeUndefined();
+ expect(uncorrelatedTool.attributes["gen_ai.execute_tool.duration"]).toEqual(expect.any(Number));
expect(uncorrelatedTool.parentSpanContext?.spanId).toBe(
byName(spans, "agent.step")[0]!.spanContext().spanId,
);
@@ -2116,9 +2112,9 @@ describe("createAgentOtelInstrumentation", () => {
idempotencyKey: actionKey,
input: { secret: "value" },
kind: "tool-call",
- name: "weather",
+ toolName: "weather",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await first.hooks.publish({
idempotencyKey: attemptIdempotencyKey(scope),
@@ -2142,35 +2138,31 @@ describe("createAgentOtelInstrumentation", () => {
await contextStorage.run(restored, async () => {
// Approval-resumed tools can execute before the replacement AI SDK emits
// a new step start. The persisted action context is still their parent.
- await replacement.hooks.publish({
- callId: "tool-1",
- idempotencyKey: toolKey,
- input: {},
- scope: replacementScope,
- toolName: "weather",
- type: "tool.call.started",
- });
- await replacement.hooks.publish({
- idempotencyKey: toolKey,
- output: { output: "ok", type: "result" },
- scope: replacementScope,
- type: "tool.call.completed",
- });
+ await replacement.runInContext(
+ {
+ callId: "tool-1",
+ idempotencyKey: toolKey,
+ input: {},
+ scope: replacementScope,
+ toolName: "weather",
+ type: "tool.call",
+ },
+ () => Promise.resolve("ok"),
+ );
await replacement.hooks.publish({
acceptedAtMs,
idempotencyKey: actionKey,
outcome: "completed",
output: { output: { temperature: 72 }, type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
});
});
await replacement.provider.forceFlush();
const replacementSpans = replacement.exporter.getFinishedSpans();
- const action = byName(replacementSpans, "agent.action")[0]!;
- const tool = byName(replacementSpans, "execute_tool weather")[0]!;
- expect(action.spanContext().spanId).toBe(tool.parentSpanContext?.spanId);
+ const action = byName(replacementSpans, "execute_tool weather")[0]!;
+ expect(byName(replacementSpans, "execute_tool weather")).toHaveLength(1);
expect(action.parentSpanContext?.spanId).toBe(step.spanContext().spanId);
expect(action.attributes).toMatchObject({
"agent.action.kind": "tool-call",
@@ -2207,9 +2199,9 @@ describe("createAgentOtelInstrumentation", () => {
idempotencyKey: actionKey,
input: { city: "SF" },
kind: "tool-call",
- name: "weather",
+ toolName: "weather",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await first.hooks.publish({
action: { callId: "tool-1", name: "weather" },
@@ -2240,14 +2232,14 @@ describe("createAgentOtelInstrumentation", () => {
outcome: "completed",
output: { output: { temperature: 72 }, type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
});
});
await replacement.provider.forceFlush();
const spans = replacement.exporter.getFinishedSpans();
const approval = byName(spans, "agent.approval")[0]!;
- const action = byName(spans, "agent.action")[0]!;
+ const action = byName(spans, "execute_tool weather")[0]!;
expect(approval.parentSpanContext?.spanId).toBe(action.spanContext().spanId);
expect(approval.status.code).toBe(SpanStatusCode.UNSET);
expect(approval.attributes).toMatchObject({
@@ -2323,9 +2315,9 @@ describe("createAgentOtelInstrumentation", () => {
idempotencyKey: actionKey,
input: { city: "SF" },
kind: "tool-call",
- name: "weather",
+ toolName: "weather",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await runtime.hooks.publish({
action: { callId: "tool-1", name: "weather" },
@@ -2349,7 +2341,7 @@ describe("createAgentOtelInstrumentation", () => {
outcome: "completed",
output: { output: { temperature: 72 }, type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
});
await runtime.hooks.publish({
agentName: "weather",
@@ -2394,18 +2386,16 @@ describe("createAgentOtelInstrumentation", () => {
scope,
type: "step.attempt.started",
});
- await runtime.hooks.publish({
- callId: "tool-1",
- idempotencyKey: "tool:session-1:turn-1:tool-1",
- input: { city: "SF" },
- scope,
- toolName: "weather",
- type: "tool.call.started",
- });
-
const withSpy = vi.spyOn(context, "with");
await runtime.runInContext(
- { idempotencyKey: "tool:session-1:turn-1:tool-1", scope, type: "tool.call" },
+ {
+ idempotencyKey: "tool:session-1:turn-1:tool-1",
+ scope,
+ type: "tool.call",
+ callId: "tool-1",
+ toolName: "weather",
+ input: { city: "SF" },
+ },
async () => undefined,
);
@@ -2450,52 +2440,47 @@ describe("createAgentOtelInstrumentation", () => {
idempotencyKey: actionKey,
input: { issue: "ISSUE-1" },
kind: "tool-call",
- name: "linear__get_issue",
- scope,
- type: "action.started",
- });
- await runtime.hooks.publish({
- callId: "tool-1",
- idempotencyKey: toolKey,
- input: { issue: "ISSUE-1" },
- scope,
toolName: "linear__get_issue",
+ scope,
type: "tool.call.started",
});
-
const withSpy = vi.spyOn(context, "with");
- await runtime.runInContext({ idempotencyKey: toolKey, scope, type: "tool.call" }, async () => {
- const active = withSpy.mock.calls[0]?.[0];
- expect(active).toBeDefined();
- expect(agentToolContentPolicy(active!)).toEqual({
- recordInputs: false,
- recordOutputs: false,
- });
- annotateAgentToolSpan(
- {
- "eve.connection.name": "linear",
- "gen_ai.operation.name": "execute_tool",
- "gen_ai.tool.name": "get_issue",
- "jsonrpc.request.id": "7",
- "mcp.method.name": "tools/call",
- },
- active!,
- );
- });
+ await runtime.runInContext(
+ {
+ idempotencyKey: toolKey,
+ scope,
+ type: "tool.call",
+ callId: "tool-1",
+ toolName: "linear__get_issue",
+ input: { issue: "ISSUE-1" },
+ },
+ async () => {
+ const active = withSpy.mock.calls[0]?.[0];
+ expect(active).toBeDefined();
+ expect(agentToolContentPolicy(active!)).toEqual({
+ recordInputs: false,
+ recordOutputs: false,
+ });
+ annotateAgentToolSpan(
+ {
+ "eve.connection.name": "linear",
+ "gen_ai.operation.name": "execute_tool",
+ "gen_ai.tool.name": "get_issue",
+ "jsonrpc.request.id": "7",
+ "mcp.method.name": "tools/call",
+ },
+ active!,
+ );
+ },
+ );
withSpy.mockRestore();
- await runtime.hooks.publish({
- idempotencyKey: toolKey,
- output: { output: { title: "private issue" }, type: "result" },
- scope,
- type: "tool.call.completed",
- });
await runtime.hooks.publish({
idempotencyKey: actionKey,
outcome: "completed",
output: { output: { title: "private issue" }, type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
});
await runtime.hooks.publish({
idempotencyKey: attemptIdempotencyKey(scope),
@@ -2551,7 +2536,7 @@ describe("createAgentOtelInstrumentation", () => {
expect(byName(runtime.exporter.getFinishedSpans(), "agent.approval")).toHaveLength(0);
});
- it("ends SDK spans but leaves durable actions for their own terminal", async () => {
+ it("ends model spans but leaves durable tools for their own terminal", async () => {
const runtime = createRuntime();
await emitAttempt({
hooks: runtime.hooks,
@@ -2567,7 +2552,7 @@ describe("createAgentOtelInstrumentation", () => {
const spans = runtime.exporter.getFinishedSpans();
expect(byName(spans, "chat claude-test")).toHaveLength(1);
expect(byName(spans, "agent.action")).toHaveLength(0);
- expect(byName(spans, "execute_tool weather")).toHaveLength(1);
+ expect(byName(spans, "execute_tool weather")).toHaveLength(0);
expect(byName(spans, "agent.step")).toHaveLength(1);
});
@@ -2596,9 +2581,9 @@ describe("createAgentOtelInstrumentation", () => {
idempotencyKey: actionIdempotencyKey(sessionId, scope.turnId, "tool-1"),
input: {},
kind: "tool-call",
- name: "publish",
+ toolName: "publish",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await runtime.hooks.publish({
error: publishError,
@@ -2606,10 +2591,12 @@ describe("createAgentOtelInstrumentation", () => {
idempotencyKey: actionIdempotencyKey(sessionId, scope.turnId, "tool-1"),
outcome: "failed",
scope,
- type: "action.failed",
+ type: "tool.call.failed",
});
await runtime.provider.forceFlush();
- return byName(runtime.exporter.getFinishedSpans(), "agent.action")[0]!;
+ return runtime.exporter
+ .getFinishedSpans()
+ .find((span) => span.attributes["agent.action.call_id"] !== undefined)!;
};
const recorded = await run(createRuntime(), "recorded");
@@ -2672,7 +2659,7 @@ describe("createAgentOtelInstrumentation", () => {
await runtime.provider.forceFlush();
const spans = runtime.exporter.getFinishedSpans();
- for (const name of ["chat claude-test", "agent.action", "execute_tool weather"]) {
+ for (const name of ["chat claude-test", "execute_tool weather"]) {
const span = byName(spans, name)[0]!;
expect(span.status).toEqual({ code: SpanStatusCode.ERROR, message: error.message });
expect(span.attributes["error.type"]).toBe("Error");
@@ -2704,7 +2691,7 @@ describe("createAgentOtelInstrumentation", () => {
await runtime.provider.forceFlush();
const spans = runtime.exporter.getFinishedSpans();
- const action = byName(spans, "agent.action")[0];
+ const action = byName(spans, "execute_tool weather")[0];
expect(action?.attributes).toMatchObject({
"agent.action.kind": "subagent-call",
"agent.action.name": "weather",
@@ -2718,9 +2705,8 @@ describe("createAgentOtelInstrumentation", () => {
expect(
Object.keys(action?.attributes ?? {}).some((key) => key.startsWith("gen_ai.usage.")),
).toBe(false);
- expect(action?.attributes).not.toHaveProperty("gen_ai.tool.call.arguments");
- expect(action?.attributes).not.toHaveProperty("gen_ai.tool.call.result");
- expect(action?.attributes).not.toHaveProperty("gen_ai.operation.name");
+ expect(action?.attributes["gen_ai.tool.call.arguments"]).toContain("secret");
+ expect(action?.attributes["gen_ai.operation.name"]).toBe("execute_tool");
expect(byName(spans, "invoke_agent weather")).toHaveLength(1);
expect(byName(spans, "execute_tool weather")).toHaveLength(1);
});
@@ -2764,10 +2750,6 @@ describe("createAgentOtelInstrumentation", () => {
);
expect(tool.attributes["gen_ai.tool.call.arguments"]).toBe('{"secret":"value"}');
expect(tool.attributes["gen_ai.tool.call.result"]).toBe('{"temperature":72}');
- // Runtime action spans carry content for dispatches that have no SDK tool boundary.
- const action = byName(spans, "agent.action")[0]!;
- expect(action.attributes["gen_ai.tool.call.arguments"]).toContain("secret");
- expect(action.attributes["gen_ai.tool.call.result"]).toContain("temperature");
});
it("applies trace content policy only to OTel spans", async () => {
@@ -3090,7 +3072,7 @@ describe("createAgentOtelInstrumentation", () => {
const replacementSpans = replacementRuntime.exporter.getFinishedSpans();
const turn = byName(replacementSpans, "invoke_agent weather")[0]!;
const step = byName(replacementSpans, "agent.step")[0]!;
- const action = byName(replacementSpans, "agent.action")[0]!;
+ const action = byName(replacementSpans, "execute_tool weather")[0]!;
expect(step.parentSpanContext?.spanId).toBe(turn.spanContext().spanId);
expect(action.parentSpanContext?.spanId).toBe(step.spanContext().spanId);
@@ -3209,11 +3191,12 @@ describe("createAgentOtelInstrumentation", () => {
};
const key = actionIdempotencyKey("parent", "turn_0", "dispatch");
await runtime.hooks.publish({
- type: "action.started",
+ type: "tool.call.started",
idempotencyKey: key,
scope,
callId: "dispatch",
- name: "child",
+ toolName: "child",
+ input: undefined,
kind: "subagent-call",
});
const action = (await store.getAction(key))!;
@@ -3338,7 +3321,7 @@ describe("createAgentOtelInstrumentation", () => {
await runtime.provider.forceFlush();
const spans = runtime.exporter.getFinishedSpans();
- const action = byName(spans, "agent.action")[0]!;
+ const action = byName(spans, "execute_tool weather")[0]!;
expect(action.status.code).toBe(SpanStatusCode.ERROR);
expect(action.attributes).toMatchObject({
"agent.action.error.code": "TOOL_CALL_FAILED",
diff --git a/packages/eve/src/tracing/agent-otel-provider.ts b/packages/eve/src/tracing/agent-otel-provider.ts
index 558a5d30d..d5d55325b 100644
--- a/packages/eve/src/tracing/agent-otel-provider.ts
+++ b/packages/eve/src/tracing/agent-otel-provider.ts
@@ -24,7 +24,6 @@ import {
toolResultsContentAttribute,
} from "#tracing/agent-otel-content.js";
import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";
-import { createAgentActionInstrumentation } from "#tracing/agent-action-instrumentation.js";
import { createAgentApprovalInstrumentation } from "#tracing/agent-approval-instrumentation.js";
import { createAgentChannelDeliveryInstrumentation } from "#tracing/agent-channel-delivery-instrumentation.js";
import { createAgentToolInstrumentation } from "#tracing/agent-tool-instrumentation.js";
@@ -120,7 +119,7 @@ export function createAgentOtelInstrumentation(
// A lost serverless worker retries the whole turn step from entry.
const steps = new WeakMap<InstrumentationAttemptScope, SpanState>();
const modelSpans = new WeakMap<InstrumentationAttemptScope, Map<string, SpanState>>();
- const actions = createAgentActionInstrumentation({
+ const tools = createAgentToolInstrumentation({
frameworkVersion: input.frameworkVersion,
idGenerator: input.idGenerator,
recordInputs,
@@ -130,20 +129,6 @@ export function createAgentOtelInstrumentation(
return turn?.context;
},
stateStore: input.stateStore,
- toolStartTimeForAction: (idempotencyKey) => tools.startTimeForAction(idempotencyKey),
- tracer: input.tracer,
- });
- const approvals = createAgentApprovalInstrumentation({
- actionContextFor: actions.contextFor,
- frameworkVersion: input.frameworkVersion,
- idGenerator: input.idGenerator,
- tracer: input.tracer,
- });
- const tools = createAgentToolInstrumentation({
- actionContextFor: actions.contextFor,
- idGenerator: input.idGenerator,
- recordInputs,
- recordOutputs,
resolveFallback: (event) => {
const scope = attemptScopes.get(event.scope.attemptId) ?? event.scope;
const step = steps.get(scope);
@@ -153,6 +138,12 @@ export function createAgentOtelInstrumentation(
},
tracer: input.tracer,
});
+ const approvals = createAgentApprovalInstrumentation({
+ actionContextFor: tools.dispatchContextFor,
+ frameworkVersion: input.frameworkVersion,
+ idGenerator: input.idGenerator,
+ tracer: input.tracer,
+ });
const memory = createAgentMemoryInstrumentation({ ...input, environment });
const { prepareSessionTrace, prepareTurnTrace } = createAgentOtelSessionContext({
...input,
@@ -268,7 +259,7 @@ export function createAgentOtelInstrumentation(
event.type === "step.attempt.failed" ? { error: event.error } : undefined,
);
if (event.type === "step.attempt.failed") {
- await actions.failForAttempt(scope, event.error);
+ await tools.failForAttempt(scope, event.error);
}
attemptScopes.delete(event.scope.attemptId);
const attempt = steps.get(scope);
@@ -361,7 +352,7 @@ export function createAgentOtelInstrumentation(
// turn that still needs its metadata — so only release session-scoped
// state on terminal transitions.
if (event.type === "session.completed" || event.type === "session.failed") {
- await actions.deleteForSession(event.sessionId);
+ await tools.deleteForSession(event.sessionId);
await input.stateStore.deleteSession(event.sessionId);
}
};
@@ -533,12 +524,6 @@ export function createAgentOtelInstrumentation(
hook: {
events: {
...channelDeliveries,
- "action.completed": actions.events["action.completed"],
- "action.failed": actions.events["action.failed"],
- async "action.started"(event, ctx) {
- await actions.events["action.started"]!(event, ctx);
- await tools.actionStarted(event);
- },
...approvals,
...memory.events,
"step.attempt.completed": onStepTerminal,
@@ -601,16 +586,26 @@ export function createAgentOtelInstrumentation(
recordInputs: recordInputs && effective?.action === "record" && effective.recordInputs,
recordOutputs: recordOutputs && effective?.action === "record" && effective.recordOutputs,
};
- if (parent === undefined) return execute();
- const withErrorPolicy = withErrorContent(parent, toolContentPolicy.recordOutputs);
- const operationContext =
- operation.type === "tool.call"
- ? withAgentToolContentPolicy(withErrorPolicy, toolContentPolicy)
- : withErrorPolicy;
- return context.with(
- markAgentTraceContext(withOperationConversation(operationContext, operation.scope)),
- execute,
- );
+ const run = (executionParent: Context | undefined) => {
+ const activeParent = executionParent ?? parent;
+ if (activeParent === undefined) return execute();
+ const withErrorPolicy = withErrorContent(activeParent, toolContentPolicy.recordOutputs);
+ const operationContext =
+ operation.type === "tool.call"
+ ? withAgentToolContentPolicy(withErrorPolicy, toolContentPolicy)
+ : withErrorPolicy;
+ return context.with(
+ markAgentTraceContext(withOperationConversation(operationContext, operation.scope)),
+ execute,
+ );
+ };
+ return operation.type === "tool.call"
+ ? tools.runInContext(
+ { ...operation, input: toolContentPolicy.recordInputs ? operation.input : undefined },
+ run,
+ toolContentPolicy.recordOutputs,
+ )
+ : run(undefined);
},
};
diff --git a/packages/eve/src/tracing/agent-span-contract.ts b/packages/eve/src/tracing/agent-span-contract.ts
index d517fae49..917aec7e0 100644
--- a/packages/eve/src/tracing/agent-span-contract.ts
+++ b/packages/eve/src/tracing/agent-span-contract.ts
@@ -1,7 +1,6 @@
export const AGENT_TRACE_SCHEMA_VERSION = 4;
export const AGENT_SPAN_NAMES = {
- action: "agent.action",
approval: "agent.approval",
channelRequest: "agent.channel.request",
step: "agent.step",
diff --git a/packages/eve/src/tracing/agent-telemetry-contract.integration.test.ts b/packages/eve/src/tracing/agent-telemetry-contract.integration.test.ts
index 162ce2e7c..7361e32be 100644
--- a/packages/eve/src/tracing/agent-telemetry-contract.integration.test.ts
+++ b/packages/eve/src/tracing/agent-telemetry-contract.integration.test.ts
@@ -211,28 +211,21 @@ describe("exported agent telemetry contract", () => {
turnId: "turn_0",
}),
);
- await hooks.publish({
- type: "tool.call.started",
- idempotencyKey: toolKey,
- callId: "call-1",
- input,
- toolName: "connection_execute",
- scope,
- });
const output = await runtime.runInContext(
- { idempotencyKey: toolKey, scope, type: "tool.call" },
+ {
+ idempotencyKey: toolKey,
+ scope,
+ type: "tool.call",
+ callId: "call-1",
+ toolName: "connection_execute",
+ input,
+ },
() =>
resolveConnectionTools()!.connection_execute!.execute!(input, {
callId: "call-1",
messages: [],
} as never),
);
- await hooks.publish({
- type: "tool.call.completed",
- idempotencyKey: toolKey,
- output: { type: "result", output },
- scope,
- });
await emitNestedToolActions(
handleEvent,
{ sequence: 0, stepIndex: 0, turnId: "turn_0" },
@@ -266,7 +259,7 @@ describe("exported agent telemetry contract", () => {
expect(accepted.mock.calls).toHaveLength(4);
const actions = runtime.exporter
.getFinishedSpans()
- .filter((span) => span.name === "agent.action");
+ .filter((span) => span.attributes["gen_ai.operation.name"] === "execute_tool");
expect(actions).toHaveLength(2);
const outer = actions.find((span) => span.attributes["agent.action.call_id"] === "call-1")!;
const nested = actions.find(
@@ -282,7 +275,7 @@ describe("exported agent telemetry contract", () => {
const tool = runtime.exporter
.getFinishedSpans()
.find((span) => span.name === "execute_tool connection_execute")!;
- expect(tool.parentSpanContext?.spanId).toBe(outer.spanContext().spanId);
+ expect(tool.spanContext().spanId).toBe(outer.spanContext().spanId);
expect(nested.attributes["gen_ai.tool.call.arguments"]).toBe(
audience === "public" ? '{"id":"ISSUE-1","includeRelations":false}' : undefined,
);
@@ -397,10 +390,10 @@ describe("exported agent telemetry contract", () => {
);
});
const spans = runtime.exporter.getFinishedSpans();
- const action = spans.find((span) => span.name === "agent.action")!;
+ const action = spans.find((span) => span.name === `execute_tool ${tool.name}`)!;
const execution = spans.find((span) => span.name === `execute_tool ${tool.name}`)!;
expect(execution).toBeDefined();
- expect(execution.parentSpanContext?.spanId).toBe(action.spanContext().spanId);
+ expect(spans.filter((span) => span.name === execution.name)).toHaveLength(1);
expect(execution.attributes["agent.tool.is_framework"]).toBe(true);
expect(action.attributes).not.toHaveProperty("agent.action.origin");
expect(action.attributes).not.toHaveProperty("agent.framework.action");
@@ -508,15 +501,13 @@ describe("exported agent telemetry contract", () => {
);
});
const spans = second.exporter.getFinishedSpans();
- const action = spans.find((span) => span.name === "agent.action")!;
+ const action = spans.find((span) => span.name === "execute_tool reviewer")!;
expect(action.attributes).toMatchObject({
"agent.action.kind": kind,
"agent.invocation.role": "caller",
"gen_ai.agent.name": "reviewer",
});
- expect(
- spans.find((span) => span.name === "execute_tool reviewer")?.parentSpanContext?.spanId,
- ).toBe(action.spanContext().spanId);
+ expect(spans.filter((span) => span.name === "execute_tool reviewer")).toHaveLength(1);
expect(
spans.find((span) => span.name === "execute_tool reviewer")?.attributes[
"agent.tool.is_framework"
@@ -577,21 +568,13 @@ describe("exported agent telemetry contract", () => {
const actionKey = actionIdempotencyKey(sessionId, "turn_0", "tool");
const toolKey = toolCallIdempotencyKey(scope, "tool", 0);
const modelKey = modelCallIdempotencyKey(scope, 0, 0);
- await hooks.publish({
- type: "action.started",
- idempotencyKey: actionKey,
- scope,
- callId: "tool",
- name: "inspect",
- kind: "tool-call",
- input: {},
- });
await hooks.publish({
type: "tool.call.started",
- idempotencyKey: toolKey,
+ idempotencyKey: actionKey,
scope,
callId: "tool",
toolName: "inspect",
+ kind: "tool-call",
input: {},
});
await hooks.publish({
@@ -631,14 +614,19 @@ describe("exported agent telemetry contract", () => {
outcome: "approved",
response: {},
});
+ await runtime.runInContext(
+ {
+ type: "tool.call",
+ idempotencyKey: toolKey,
+ scope,
+ callId: "tool",
+ toolName: "inspect",
+ input: {},
+ },
+ () => Promise.resolve({}),
+ );
await hooks.publish({
type: "tool.call.completed",
- idempotencyKey: toolKey,
- scope,
- output: { type: "result", output: {} },
- });
- await hooks.publish({
- type: "action.completed",
idempotencyKey: actionKey,
scope,
outcome: "completed",
@@ -685,7 +673,6 @@ describe("exported agent telemetry contract", () => {
"agent.step",
"chat test",
"execute_tool inspect",
- "agent.action",
"agent.approval",
"search_memory",
].sort(),
@@ -881,14 +868,6 @@ describe("exported agent telemetry contract", () => {
input: { secret: "private input" },
isWorkflowTool: true,
kind: "tool-call",
- name: "coordinate",
- scope,
- type: "action.started",
- });
- await hooks.publish({
- callId: "workflow",
- idempotencyKey: toolCallIdempotencyKey(scope, "workflow", 0),
- input: { secret: "private input" },
toolName: "coordinate",
scope,
type: "tool.call.started",
@@ -993,18 +972,23 @@ describe("exported agent telemetry contract", () => {
});
});
await contextStorage.run(parent, async () => {
- await hooks.publish({
- idempotencyKey: toolCallIdempotencyKey(scope, "workflow", 0),
- output: { type: "result", output: "private output" },
- scope,
- type: "tool.call.completed",
- });
+ await runtime.runInContext(
+ {
+ type: "tool.call",
+ idempotencyKey: toolCallIdempotencyKey(scope, "workflow", 0),
+ scope,
+ callId: "workflow",
+ toolName: "coordinate",
+ input: { secret: "private input" },
+ },
+ () => Promise.resolve("private output"),
+ );
await hooks.publish({
idempotencyKey: actionKey,
outcome: "completed",
output: { type: "result", output: "private output" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
});
await hooks.publish({
idempotencyKey: attemptIdempotencyKey(scope),
@@ -1133,7 +1117,8 @@ describe("exported agent telemetry contract", () => {
expect(new TextDecoder().decode(bytes)).not.toContain("auth-only-secret");
const workflow = parsed.find(
(span) =>
- span.name === "agent.action" && span.attributes["agent.action.call_id"] === "workflow",
+ span.attributes["gen_ai.operation.name"] === "execute_tool" &&
+ span.attributes["agent.action.call_id"] === "workflow",
)!;
const activation = parsed.find(
(span) => span.name === "invoke_agent child" && isAgentTurnSpan(span),
@@ -1163,9 +1148,8 @@ describe("exported agent telemetry contract", () => {
" invoked from external via channel.request",
" invoke_agent parent",
" agent.step",
- " agent.action coordinate",
+ " execute_tool coordinate",
" agent.approval approved",
- " execute_tool coordinate",
" invoke_agent child",
" agent.step",
" chat test",
diff --git a/packages/eve/src/tracing/agent-tool-instrumentation.ts b/packages/eve/src/tracing/agent-tool-instrumentation.ts
index a028cd322..5231727e7 100644
--- a/packages/eve/src/tracing/agent-tool-instrumentation.ts
+++ b/packages/eve/src/tracing/agent-tool-instrumentation.ts
@@ -10,17 +10,21 @@ import {
} from "#compiled/@opentelemetry/api/index.js";
import type {
- InstrumentationActionStartedEvent,
+ InstrumentationAttemptScope,
+ InstrumentationExecutionOperation,
InstrumentationToolCallStartedEvent,
InstrumentationToolCallTerminalEvent,
} from "#instrumentation/lifecycle.js";
-import { actionIdempotencyKey } from "#instrumentation/lifecycle.js";
-import { contentAttribute } from "#tracing/agent-otel-content.js";
+import { actionIdempotencyKey, attemptIdempotencyKey } from "#instrumentation/lifecycle.js";
+import { contentAttribute, textContentAttribute } from "#tracing/agent-otel-content.js";
import { agentSpanNamingAttributes } from "#tracing/agent-span-naming.js";
import { agentTraceIdentityAttributes, traceSessionIdOf } from "#tracing/agent-otel-attributes.js";
import { withChannelAudience } from "#tracing/channel-audience-context.js";
import type { AgentSpanIdGenerator } from "#tracing/agent-span-id-generator.js";
-import type { AgentActionContext } from "#tracing/agent-action-instrumentation.js";
+import type { AgentActionTraceState, AgentTraceStateStore } from "#tracing/agent-trace-state.js";
+import { normalizeChannelAudience } from "#shared/channel-audience.js";
+import { isSampledTrace } from "#tracing/sampled-trace.js";
+import { setAgentUsage } from "#tracing/agent-otel-usage.js";
import { recordAgentSpanError as recordError } from "#tracing/agent-span-error.js";
import { withAgentToolSpanContext } from "#tracing/agent-tool-span-context.js";
@@ -38,11 +42,22 @@ interface ToolSpanState {
span?: Span;
terminal?: InstrumentationToolCallTerminalEvent;
pendingError?: { readonly error: unknown; readonly errorType?: string };
+ correlated?: true;
+}
+
+export interface AgentToolContext {
+ readonly context: Context;
+ readonly spanContext: SpanContext;
}
interface AgentToolInstrumentation {
- actionStarted(event: InstrumentationActionStartedEvent): Promise<void>;
- startTimeForAction(idempotencyKey: string): number | undefined;
+ deleteForSession(sessionId: string): Promise<void>;
+ failForAttempt(scope: InstrumentationAttemptScope, error: unknown): Promise<void>;
+ dispatchContextFor(
+ sessionId: string,
+ turnId: string,
+ callId: string,
+ ): Promise<AgentToolContext | undefined>;
contextFor(attemptId: string, idempotencyKey: string): Context | undefined;
drain(attemptId: string, failure?: { readonly error: unknown }): void;
readonly events: {
@@ -50,15 +65,19 @@ interface AgentToolInstrumentation {
readonly "tool.call.failed": (event: InstrumentationToolCallTerminalEvent) => Promise<void>;
readonly "tool.call.started": (event: InstrumentationToolCallStartedEvent) => Promise<void>;
};
+ runInContext<T>(
+ operation: Extract<InstrumentationExecutionOperation, { type: "tool.call" }>,
+ execute: (parent: Context | undefined) => PromiseLike<T>,
+ recordOutputs: boolean,
+ ): Promise<T>;
}
-/** Keeps SDK tool spans parented to actions even when SDK telemetry wins the event race. */
+/** Enriches durable tool calls, exporting SDK-only calls under their step. */
export function createAgentToolInstrumentation(input: {
- readonly actionContextFor: (
- sessionId: string,
- turnId: string,
- callId: string,
- ) => Promise<AgentActionContext | undefined>;
+ readonly frameworkVersion: string;
+ readonly resolveTraceContext: (
+ event: InstrumentationToolCallStartedEvent,
+ ) => SpanContext | undefined | PromiseLike<SpanContext | undefined>;
readonly idGenerator: AgentSpanIdGenerator;
readonly recordInputs: boolean;
readonly recordOutputs: boolean;
@@ -66,16 +85,171 @@ export function createAgentToolInstrumentation(input: {
event: InstrumentationToolCallStartedEvent,
) => { readonly context: Context; readonly spanContext: SpanContext } | undefined;
readonly tracer: Tracer;
+ readonly stateStore: AgentTraceStateStore;
}): AgentToolInstrumentation {
const byAction = new Map<string, ToolSpanState>();
const byAttempt = new Map<string, Map<string, ToolSpanState>>();
+ const dispatchesByAttempt = new Map<string, Set<string>>();
+
+ async function dispatchContextFor(
+ sessionId: string,
+ turnId: string,
+ callId: string,
+ ): Promise<AgentToolContext | undefined> {
+ const state =
+ (await input.stateStore.getAction(actionIdempotencyKey(sessionId, turnId, callId))) ??
+ (await input.stateStore.findAction(sessionId, callId));
+ if (state === undefined) return undefined;
+ const spanContext = {
+ isRemote: false,
+ spanId: state.spanId,
+ traceFlags: state.parent.traceFlags,
+ traceId: state.parent.traceId,
+ };
+ return {
+ context: withChannelAudience(contextFromSpanContext(spanContext), state.channelAudience),
+ spanContext,
+ };
+ }
+
+ async function onDispatchStarted(event: InstrumentationToolCallStartedEvent): Promise<void> {
+ const traceContext = await input.resolveTraceContext(event);
+ if (traceContext === undefined || !isSampledTrace(traceContext)) return;
+ const existing = await input.stateStore.getAction(event.idempotencyKey);
+ const state: AgentActionTraceState = existing ?? {
+ attemptIndex: event.scope.attemptIndex,
+ callId: event.callId,
+ channelAudience: normalizeChannelAudience(event.scope.channelAudience),
+ inputAttribute: input.recordInputs ? contentAttribute(event.input) : undefined,
+ kind: event.kind ?? "tool-call",
+ name: event.toolName,
+ toolAttributes: { "agent.tool.is_framework": event.frameworkTool === true },
+ parent: {
+ spanId: input.idGenerator.deriveSpanId(
+ event.parentCallId === undefined
+ ? attemptIdempotencyKey(event.scope)
+ : `action:${actionIdempotencyKey(event.scope.sessionId, event.scope.turnId, event.parentCallId)}`,
+ ),
+ traceFlags: traceContext.traceFlags,
+ traceId: traceContext.traceId,
+ },
+ parentCallId: event.parentCallId,
+ rootSessionId: event.scope.rootSessionId ?? event.scope.sessionId,
+ traceSessionId: traceSessionIdOf(event.scope),
+ sessionId: event.scope.sessionId,
+ spanId: input.idGenerator.deriveSpanId(`action:${event.idempotencyKey}`),
+ startTimeMs: Math.min(
+ event.startedAtMs ?? Date.now(),
+ byAction.get(event.idempotencyKey)?.startTimeMs ?? Infinity,
+ ),
+ stepIndex: event.scope.stepIndex,
+ turnId: event.scope.turnId,
+ };
+ await input.stateStore.setAction(event.idempotencyKey, state);
+ if (event.isWorkflowTool === true)
+ await input.stateStore.setActionAnchor(event.idempotencyKey, state);
+ const keys = dispatchesByAttempt.get(event.scope.attemptId) ?? new Set<string>();
+ keys.add(event.idempotencyKey);
+ dispatchesByAttempt.set(event.scope.attemptId, keys);
+ const execution = byAction.get(event.idempotencyKey);
+ if (execution !== undefined && execution.finished !== true) {
+ await correlate(execution);
+ finishIfReady(execution);
+ }
+ }
+
+ async function onDispatchTerminal(event: InstrumentationToolCallTerminalEvent): Promise<void> {
+ const state = await input.stateStore.getAction(event.idempotencyKey);
+ if (state === undefined) return;
+ try {
+ const span = startDispatchSpan(state);
+ const outcome = event.outcome ?? (event.type === "tool.call.failed" ? "failed" : "completed");
+ span.setAttribute(
+ "agent.action.outcome",
+ state.toolFailed === true && outcome === "completed" ? "failed" : outcome,
+ );
+ if (event.usage !== undefined) setAgentUsage(span, event.usage);
+ if (event.type === "tool.call.failed") {
+ if (event.errorCode !== undefined)
+ span.setAttribute("agent.action.error.code", event.errorCode);
+ recordToolError(span, event.error, event.errorCode);
+ } else if (event.output.type === "error") {
+ recordToolError(span, event.output.error);
+ } else if (state.toolFailed === true) {
+ recordToolError(
+ span,
+ state.toolErrorAttribute,
+ typeof state.toolAttributes?.["error.type"] === "string"
+ ? state.toolAttributes["error.type"]
+ : undefined,
+ );
+ } else if (input.recordOutputs) {
+ const result = contentAttribute(event.output.output);
+ if (result !== undefined) span.setAttribute("gen_ai.tool.call.result", result);
+ }
+ span.end(event.acceptedAtMs ?? state.toolEndTimeMs);
+ } finally {
+ await input.stateStore.deleteAction(event.idempotencyKey);
+ for (const [attemptId, keys] of dispatchesByAttempt) {
+ keys.delete(event.idempotencyKey);
+ if (keys.size === 0) dispatchesByAttempt.delete(attemptId);
+ }
+ }
+ }
+
+ function startDispatchSpan(state: AgentActionTraceState): Span {
+ const invocation = state.kind === "subagent-call" || state.kind === "remote-agent-call";
+ const span = input.idGenerator.withSpanId(state.spanId, () =>
+ input.tracer.startSpan(
+ `execute_tool ${state.name}`,
+ {
+ attributes: {
+ "agent.action.call_id": state.callId,
+ "agent.action.kind": state.kind,
+ "agent.action.name": state.name,
+ ...(state.parentCallId === undefined
+ ? undefined
+ : { "agent.action.parent_call_id": state.parentCallId }),
+ "gen_ai.operation.name": "execute_tool",
+ "gen_ai.tool.call.id": state.callId,
+ "gen_ai.tool.name": state.name,
+ "gen_ai.tool.type": "function",
+ "agent.framework.name": "eve",
+ "agent.framework.version": input.frameworkVersion,
+ "agent.step.attempt": state.attemptIndex,
+ "agent.step.index": state.stepIndex,
+ "agent.turn.id": state.turnId,
+ ...agentSpanNamingAttributes(`execute_tool ${state.name}`, "execute_tool"),
+ ...agentTraceIdentityAttributes({
+ rootSessionId: state.rootSessionId,
+ traceSessionId: state.traceSessionId,
+ sessionId: state.sessionId,
+ }),
+ ...(invocation
+ ? { "gen_ai.agent.name": state.name, "agent.invocation.role": "caller" }
+ : undefined),
+ ...state.toolAttributes,
+ },
+ kind: state.kind === "remote-agent-call" ? SpanKind.CLIENT : SpanKind.INTERNAL,
+ startTime: state.startTimeMs,
+ },
+ withChannelAudience(
+ contextFromSpanContext({ ...state.parent, isRemote: false }),
+ state.channelAudience,
+ ),
+ ),
+ );
+ if (state.inputAttribute !== undefined)
+ span.setAttribute("gen_ai.tool.call.arguments", state.inputAttribute);
+ return span;
+ }
const onStarted = async (event: InstrumentationToolCallStartedEvent): Promise<void> => {
if (byAttempt.get(event.scope.attemptId)?.has(event.idempotencyKey)) return;
const actionKey = actionIdempotencyKey(event.scope.sessionId, event.scope.turnId, event.callId);
const fallback = input.resolveFallback(event);
let state = fallback === undefined ? undefined : reserve(event, actionKey, fallback);
- const actionParent = await input.actionContextFor(
+ const actionParent = await dispatchContextFor(
event.scope.sessionId,
event.scope.turnId,
event.callId,
@@ -84,7 +258,7 @@ export function createAgentToolInstrumentation(input: {
if (actionParent === undefined) return;
state = reserve(event, actionKey, actionParent);
}
- if (actionParent !== undefined) startSpan(state, actionParent.context);
+ if (actionParent !== undefined) await correlate(state);
};
const onTerminal = async (event: InstrumentationToolCallTerminalEvent): Promise<void> => {
@@ -93,29 +267,33 @@ export function createAgentToolInstrumentation(input: {
state.terminal = event;
if (state.span === undefined) {
- const actionParent = await input.actionContextFor(
+ const actionParent = await dispatchContextFor(
state.event.scope.sessionId,
state.event.scope.turnId,
state.event.callId,
);
- if (actionParent !== undefined) startSpan(state, actionParent.context);
+ if (actionParent !== undefined) await correlate(state);
}
finishIfReady(state);
};
return {
- startTimeForAction: (idempotencyKey) => byAction.get(idempotencyKey)?.startTimeMs,
- async actionStarted(event) {
- const state = byAction.get(event.idempotencyKey);
- if (state === undefined || state.span !== undefined || state.finished === true) return;
- const actionParent = await input.actionContextFor(
- event.scope.sessionId,
- event.scope.turnId,
- event.callId,
- );
- if (actionParent === undefined) return;
- startSpan(state, actionParent.context);
- finishIfReady(state);
+ dispatchContextFor,
+ async deleteForSession(sessionId) {
+ await input.stateStore.deleteActions(sessionId);
+ await input.stateStore.deleteActionAnchors(sessionId);
+ },
+ async failForAttempt(scope, error) {
+ const keys = dispatchesByAttempt.get(scope.attemptId);
+ dispatchesByAttempt.delete(scope.attemptId);
+ for (const key of keys ?? []) {
+ const state = await input.stateStore.getAction(key);
+ if (state === undefined) continue;
+ const span = startDispatchSpan(state);
+ recordError(span, error);
+ span.end();
+ await input.stateStore.deleteAction(key);
+ }
},
contextFor: (attemptId, idempotencyKey) =>
byAttempt.get(attemptId)?.get(idempotencyKey)?.context,
@@ -124,15 +302,53 @@ export function createAgentToolInstrumentation(input: {
if (states === undefined) return;
for (const state of states.values()) {
if (state.finished === true) continue;
- if (state.span === undefined) startSpan(state, state.fallbackParent);
+ if (state.span === undefined && state.correlated !== true)
+ startSpan(state, state.fallbackParent);
finish(state, failure);
}
byAttempt.delete(attemptId);
},
events: {
- "tool.call.completed": onTerminal,
- "tool.call.failed": onTerminal,
- "tool.call.started": onStarted,
+ "tool.call.completed": onDispatchTerminal,
+ "tool.call.failed": onDispatchTerminal,
+ "tool.call.started": onDispatchStarted,
+ },
+ async runInContext(operation, execute, recordOutputs) {
+ if (operation.callId === undefined || operation.toolName === undefined) {
+ return execute(
+ byAttempt.get(operation.scope.attemptId)?.get(operation.idempotencyKey)?.context,
+ );
+ }
+ await onStarted({
+ ...operation,
+ callId: operation.callId,
+ toolName: operation.toolName,
+ input: operation.input,
+ type: "tool.call.started",
+ });
+ const state = byAttempt.get(operation.scope.attemptId)?.get(operation.idempotencyKey);
+ const startedAtMs = operation.startedAtMs ?? Date.now();
+ try {
+ const result = await execute(state?.context);
+ await onTerminal({
+ type: operation.failed ? "tool.call.failed" : "tool.call.completed",
+ idempotencyKey: operation.idempotencyKey,
+ scope: operation.scope,
+ completedAtMs: operation.completedAtMs ?? Date.now(),
+ durationMs: (operation.completedAtMs ?? Date.now()) - startedAtMs,
+ output: { type: "result", output: recordOutputs ? result : undefined },
+ });
+ return result;
+ } catch (error) {
+ await onTerminal({
+ type: "tool.call.failed",
+ idempotencyKey: operation.idempotencyKey,
+ scope: operation.scope,
+ completedAtMs: Date.now(),
+ error: recordOutputs ? error : undefined,
+ });
+ throw error;
+ }
},
};
@@ -150,7 +366,7 @@ export function createAgentToolInstrumentation(input: {
actionKey: string,
parent: { readonly context: Context; readonly spanContext: SpanContext },
): ToolSpanState {
- const spanId = input.idGenerator.deriveSpanId(`tool:${event.idempotencyKey}`);
+ const spanId = input.idGenerator.deriveSpanId(`action:${actionKey}`);
const state: ToolSpanState = {
actionKey,
attemptId: event.scope.attemptId,
@@ -213,13 +429,79 @@ export function createAgentToolInstrumentation(input: {
}
function finishIfReady(state: ToolSpanState): void {
- if (state.span === undefined || state.terminal === undefined) return;
+ if ((state.span === undefined && state.correlated !== true) || state.terminal === undefined)
+ return;
finish(state);
}
+ async function correlate(state: ToolSpanState): Promise<void> {
+ const action =
+ (await input.stateStore.getAction(state.actionKey)) ??
+ (await input.stateStore.findAction(state.event.scope.sessionId, state.event.callId));
+ if (action === undefined) return;
+ state.correlated = true;
+ state.context = trace.setSpan(
+ state.context,
+ trace.wrapSpanContext({
+ isRemote: false,
+ spanId: action.spanId,
+ traceFlags: action.parent.traceFlags,
+ traceId: action.parent.traceId,
+ }),
+ );
+ const terminal = state.terminal;
+ const toolFailed =
+ state.pendingError !== undefined ||
+ terminal?.type === "tool.call.failed" ||
+ terminal?.output.type === "error";
+ const error =
+ state.pendingError?.error ??
+ (terminal?.type === "tool.call.failed"
+ ? terminal.error
+ : terminal?.output.type === "error"
+ ? terminal.output.error
+ : undefined);
+ const attributes: Attributes = {
+ ...action.toolAttributes,
+ ...(state.event.scope.functionId === undefined ||
+ action.kind === "subagent-call" ||
+ action.kind === "remote-agent-call"
+ ? undefined
+ : { "gen_ai.agent.name": state.event.scope.functionId }),
+ "agent.tool.is_framework": state.event.frameworkTool === true,
+ ...state.additionalAttributes,
+ ...(state.pendingError?.errorType === undefined
+ ? undefined
+ : { "error.type": state.pendingError.errorType }),
+ ...(terminal?.type === "tool.call.completed" && terminal.durationMs !== undefined
+ ? { "gen_ai.execute_tool.duration": terminal.durationMs / 1000 }
+ : undefined),
+ };
+ await input.stateStore.setAction(
+ actionIdempotencyKey(action.sessionId, action.turnId, action.callId),
+ {
+ ...action,
+ startTimeMs: Math.min(action.startTimeMs, state.startTimeMs),
+ toolAttributes: attributes,
+ toolEndTimeMs: terminal?.completedAtMs ?? action.toolEndTimeMs,
+ toolFailed: toolFailed || action.toolFailed,
+ toolErrorAttribute:
+ input.recordOutputs && error !== undefined
+ ? contentAttribute(error instanceof Error ? error.message : error)
+ : action.toolErrorAttribute,
+ },
+ );
+ }
+
function finish(state: ToolSpanState, failure?: { readonly error: unknown }): void {
const span = state.span;
- if (span === undefined || state.finished === true) return;
+ if (state.finished === true) return;
+ if (state.correlated === true) {
+ state.finished = true;
+ forget(state);
+ return;
+ }
+ if (span === undefined) return;
state.finished = true;
const terminal = state.terminal;
// As `@ai-sdk/otel` records it: execute's own run time, in seconds, success or error.
@@ -237,6 +519,10 @@ export function createAgentToolInstrumentation(input: {
if (result !== undefined) span.setAttribute("gen_ai.tool.call.result", result);
}
span.end(terminal?.completedAtMs);
+ forget(state);
+ }
+
+ function forget(state: ToolSpanState): void {
byAction.delete(state.actionKey);
const states = byAttempt.get(state.attemptId);
states?.delete(state.idempotencyKey);
@@ -266,3 +552,23 @@ function toolAttributes(event: InstrumentationToolCallStartedEvent): Attributes
function contextFromSpanContext(spanContext: SpanContext): Context {
return trace.setSpan(ROOT_CONTEXT, trace.wrapSpanContext(spanContext));
}
+
+function recordToolError(span: Span, error: unknown, errorType?: string): void {
+ if (error instanceof Error || error === undefined) {
+ recordError(span, error, errorType);
+ return;
+ }
+ const serialized =
+ typeof error === "string" ? textContentAttribute(error) : contentAttribute(error);
+ const message =
+ typeof error === "object" && error !== null && !Array.isArray(error)
+ ? Reflect.get(error, "message")
+ : undefined;
+ const detail =
+ typeof message === "string"
+ ? textContentAttribute(serialized === undefined ? message : `${message}\n${serialized}`)
+ : serialized;
+ const normalized = detail === undefined ? undefined : new Error(detail);
+ if (normalized !== undefined && errorType !== undefined) normalized.name = errorType;
+ recordError(span, normalized, errorType);
+}
diff --git a/packages/eve/src/tracing/agent-trace-context-codec.ts b/packages/eve/src/tracing/agent-trace-context-codec.ts
index a14af65d4..15ee140ec 100644
--- a/packages/eve/src/tracing/agent-trace-context-codec.ts
+++ b/packages/eve/src/tracing/agent-trace-context-codec.ts
@@ -185,6 +185,30 @@ function deserializeAction(value: unknown): AgentActionTraceState | undefined {
startTimeMs: value.startTimeMs,
stepIndex: value.stepIndex,
turnId: value.turnId,
+ toolEndTimeMs: typeof value.toolEndTimeMs === "number" ? value.toolEndTimeMs : undefined,
+ toolFailed: value.toolFailed === true,
+ toolErrorAttribute:
+ typeof value.toolErrorAttribute === "string" ? value.toolErrorAttribute : undefined,
+ toolAttributes: isRecord(value.toolAttributes)
+ ? Object.fromEntries(
+ Object.entries(value.toolAttributes).filter(
+ (
+ entry,
+ ): entry is [string, NonNullable<AgentActionTraceState["toolAttributes"]>[string]] => {
+ const attribute = entry[1];
+ return (
+ typeof attribute === "string" ||
+ typeof attribute === "number" ||
+ typeof attribute === "boolean" ||
+ (Array.isArray(attribute) &&
+ attribute.every((entry) =>
+ ["string", "number", "boolean"].includes(typeof entry),
+ ))
+ );
+ },
+ ),
+ )
+ : undefined,
};
}
diff --git a/packages/eve/src/tracing/agent-trace-state.ts b/packages/eve/src/tracing/agent-trace-state.ts
index c694bb117..1e151c1dd 100644
--- a/packages/eve/src/tracing/agent-trace-state.ts
+++ b/packages/eve/src/tracing/agent-trace-state.ts
@@ -1,7 +1,7 @@
-import type { SpanContext } from "#compiled/@opentelemetry/api/index.js";
+import type { Attributes, SpanContext } from "#compiled/@opentelemetry/api/index.js";
import type {
- InstrumentationActionKind,
+ InstrumentationToolCallKind,
InstrumentationParentLineage,
InstrumentationPrincipalSummary,
InstrumentationTraceContext,
@@ -58,7 +58,7 @@ export interface AgentActionTraceState {
readonly callId: string;
readonly channelAudience?: ChannelAudience;
readonly inputAttribute?: string;
- readonly kind: InstrumentationActionKind;
+ readonly kind: InstrumentationToolCallKind;
readonly name: string;
readonly parent: InstrumentationTraceContext;
readonly parentCallId?: string;
@@ -68,6 +68,10 @@ export interface AgentActionTraceState {
readonly startTimeMs: number;
readonly stepIndex: number;
readonly turnId: string;
+ readonly toolAttributes?: Attributes;
+ readonly toolEndTimeMs?: number;
+ readonly toolFailed?: boolean;
+ readonly toolErrorAttribute?: string;
}
/** Provider-owned serializable storage for durable agent trace state. */
diff --git a/packages/eve/src/tracing/invoke-tool-span.ts b/packages/eve/src/tracing/invoke-tool-span.ts
index ee805e676..04d19c88f 100644
--- a/packages/eve/src/tracing/invoke-tool-span.ts
+++ b/packages/eve/src/tracing/invoke-tool-span.ts
@@ -296,6 +296,8 @@ function toolCallEvents(
output: Object.freeze(output),
scope,
type: "tool.call.completed",
+ outcome: outcome.type === "result" ? "completed" : "failed",
+ durationMs: outcome.durationMs,
}),
);
},
diff --git a/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts b/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts
index 1c70b1caf..81fb692cf 100644
--- a/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts
+++ b/packages/eve/src/tracing/local-instrumentation-runtime.scenario.test.ts
@@ -133,9 +133,9 @@ describe("local instrumentation runtime", () => {
idempotencyKey: actionKey,
input: {},
kind: "tool-call",
- name: "weather",
+ toolName: "weather",
scope,
- type: "action.started",
+ type: "tool.call.started",
});
await Reflect.apply(bridge.onToolExecutionStart!, bridge, [
{
@@ -167,7 +167,7 @@ describe("local instrumentation runtime", () => {
outcome: "completed",
output: { output: { temperature: 72 }, type: "result" },
scope,
- type: "action.completed",
+ type: "tool.call.completed",
});
await hooks.publish({
idempotencyKey: attemptIdempotencyKey(scope),
@@ -211,11 +211,10 @@ describe("local instrumentation runtime", () => {
expect(formatTraceTree(spans)).toEqual([
"invoke_agent weather",
" agent.step",
- " agent.action",
- " execute_tool weather",
- " user.tool-work",
" chat model-1",
" user.model-work",
+ " execute_tool weather",
+ " user.tool-work",
]);
for (const exported of spans.filter((span) => !span.name.startsWith("user."))) {
expect(exported.attributes).toEqual(
diff --git a/packages/eve/src/tracing/local-trace-analysis.ts b/packages/eve/src/tracing/local-trace-analysis.ts
index 073ac15db..e58f9aec3 100644
--- a/packages/eve/src/tracing/local-trace-analysis.ts
+++ b/packages/eve/src/tracing/local-trace-analysis.ts
@@ -119,7 +119,9 @@ function recordFor(operation: LocalTraceOperation, startTimeNs: bigint): TraceTi
spanId: span.spanId,
startOffsetMs: Number(span.startTimeNs - startTimeNs) / 1_000_000,
};
- const callId = traceStringAttribute(span, "agent.action.call_id");
+ const callId =
+ traceStringAttribute(span, "gen_ai.tool.call.id") ??
+ traceStringAttribute(span, "agent.action.call_id");
const model = traceModelName(span);
const sessionId = traceSessionId(span);
const name = traceToolName(span);