The Open Framework for Building Agents
feat(eve): report session usage on session events and in eval derived facts (#4137)
Philipp Comansff12e851fd26
diff --git a/.changeset/session-usage-on-waiting.md b/.changeset/session-usage-on-waiting.mdnew file mode 100644index 000000000..492f86e05--- /dev/null+++ b/.changeset/session-usage-on-waiting.md@@ -0,0 +1,5 @@+---+"eve": patch+---++`session.waiting`, `turn.waiting`, `session.failed`, and `session.completed` now carry `usage`, the session's running token usage and cost including what the agents it delegated to spent. Eval results expose it as `derived.usage`, per session and summed for the eval, so a delegating eval no longer reports only its own model calls.diff --git a/docs/concepts/sessions-runs-and-streaming.md b/docs/concepts/sessions-runs-and-streaming.mdindex 239b4317c..4ac6892f9 100644--- a/docs/concepts/sessions-runs-and-streaming.md+++ b/docs/concepts/sessions-runs-and-streaming.md@@ -119,7 +119,7 @@ A nested action is a call a tool makes on the model's behalf, such as each conneNote: consider the privacy, confidentiality, and user-experience implications for displaying, storing, or transmitting reasoning events in your application.-`message.completed` can fire more than once in a turn: the agent often emits interim assistant text before a tool call. To tell tool-call narration from a terminal reply, check `message.completed.data.finishReason`. `step.completed.data.finishReason` mirrors the step outcome, and usage lives on `step.completed`.+`message.completed` can fire more than once in a turn: the agent often emits interim assistant text before a tool call. To tell tool-call narration from a terminal reply, check `message.completed.data.finishReason`. `step.completed.data.finishReason` mirrors the step outcome, and each step's usage lives on `step.completed`. `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` carry `data.usage`: the session's running total of its own model calls plus what the agents it delegated to spent. `costUsd` is absent when no model call reported a cost, and compaction calls are not counted.A delegated subagent publishes progress on its own child-session stream. When a subagent tool call or a workflow tool's `ctx.agent` opens a session, the parent emits `agent.started` with the call's `callId` and `turnId`, its `taskId` when the call runs as a task, the child's `sessionId`, and a `streamPath` that a client follows with `session.agent(started).stream()`. A subagent tool call is a [task](#task-events) whose `task.started` has `kind: "agent"`: its `task.started` and `task.settled` carry the call, and `task.settled.data.output` is the child's reply. A call that fails before the child session opens has no `agent.started`. A settled call does not end the child session; later calls with the same `taskId` reach it.diff --git a/docs/evals/reporters.mdx b/docs/evals/reporters.mdxindex 66c20b26c..c7b4f7025 100644--- a/docs/evals/reporters.mdx+++ b/docs/evals/reporters.mdx@@ -99,6 +99,8 @@ An eval can create several sessions, and a long session can produce several trac`result.result.derived` holds the facts the runner derived from the eval's sessions, such as tool calls and `models`: the distinct ids of the models the steps started with, in session order. Each entry in `result.result.sessions` holds the same facts for one session, in first-use order, so a reporter can tell the primary session's models from those of the other sessions the eval created or attached. A delegated subagent runs in its own session, so its models appear only when the eval attaches that session with `t.target.attachSession`. Compaction and `auto` routing calls are not included.+`derived.usage` is the token usage and cost from each session's latest `session.waiting`, `turn.waiting`, `session.failed`, or `session.completed`, including what its delegated agents spent; on a turn, it is the session's total so far. The eval's `usage` counts each captured session once, by its latest usage, and leaves out sessions another captured session opened, because that session already counts their spend. It is absent when a counted session reported no usage. It leaves out compaction calls, the turns an eval sends directly to a child session it attached, and delegated spend counted after a session's last such event.+Reporter callbacks for an eval stay ordered—eval start, traced session starts, then completion—even when several evals run concurrently. Reach for a custom reporter only when a destination isn't covered. The per-run artifacts under `.eve/evals/` retain the trace contexts alongside the result for ad-hoc inspection.## What to read nextdiff --git a/e2e/fixtures/agent-subagents/evals/delegated-usage-limit.shared.ts b/e2e/fixtures/agent-subagents/evals/delegated-usage-limit.shared.tsindex 8c2a3f9bf..f7d1814be 100644--- a/e2e/fixtures/agent-subagents/evals/delegated-usage-limit.shared.ts+++ b/e2e/fixtures/agent-subagents/evals/delegated-usage-limit.shared.ts@@ -9,7 +9,8 @@ import { SURVEY_WORKER_INPUT_TOKENS } from "../constants";* worker's one model call reports more input tokens than the parent's default* session budget, so the parent's next model call stops at its own* session-limit prompt, whose used figure includes the worker's tokens.- * Approving the prompt lets the parent report the survey. Returns the session+ * Approving the prompt lets the parent report the survey, and the parent's+ * `session.waiting` usage then includes the worker's tokens. Returns the session* for checks on how the parent delegated.*/export async function expectSurveyCountedAgainstParent(t: EveEvalContext, message: string) {@@ -33,5 +34,12 @@ export async function expectSurveyCountedAgainstParent(t: EveEvalContext, messagconst resumed = await session.respond([{ optionId: "continue", requestId: request.requestId }]);resumed.expectOk();resumed.messageIncludes("SURVEY-REPLY Alice's tide survey has 12 stations.");+ t.eventsSatisfy("the parent's session.waiting usage counts the worker's tokens", (events) => {+ const waiting = events.filter((event) => event.type === "session.waiting").at(-1);+ return (+ waiting?.type === "session.waiting" &&+ (waiting.data.usage?.inputTokens ?? 0) >= SURVEY_WORKER_INPUT_TOKENS+ );+ });return session;}diff --git a/packages/eve/extension-contracts/compatibility/channel/v43.ts b/packages/eve/extension-contracts/compatibility/channel/v43.tsnew file mode 100644index 000000000..9a260ece2--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/channel/v43.ts@@ -0,0 +1,12 @@+import { defineChannel } from "#public/channels/index.js";++// Epoch 43 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 44 adds it as optional.+// Channels that read only the continuation token see no change.+export default defineChannel({+ routes: [],+ events: {+ "session.waiting"(event) {+ console.info("session waiting", { continuationToken: event.continuationToken });+ },+ },+});diff --git a/packages/eve/extension-contracts/compatibility/connection/v36.ts b/packages/eve/extension-contracts/compatibility/connection/v36.tsnew file mode 100644index 000000000..42312a47c--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/connection/v36.ts@@ -0,0 +1,10 @@+import { defineMcpClientConnection } from "#public/connections/index.js";++// Epoch 36 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 37 adds it as optional.+export default defineMcpClientConnection({+ description: "Search the incident tracker.",+ url: "https://incidents.example.com/mcp",+ approval({ toolName }) {+ return toolName.endsWith("__search") ? "approved" : "user-approval";+ },+});diff --git a/packages/eve/extension-contracts/compatibility/dynamicInstructions/v30.ts b/packages/eve/extension-contracts/compatibility/dynamicInstructions/v30.tsnew file mode 100644index 000000000..1423d1022--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/dynamicInstructions/v30.ts@@ -0,0 +1,11 @@+import { defineDynamic, defineInstructions } from "#public/instructions/index.js";++// Epoch 30 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 31 adds it as optional.+export default defineDynamic({+ events: {+ "turn.started": (_event, ctx) =>+ defineInstructions({+ markdown: `Triage the incident reported in session ${ctx.session.id}.`,+ }),+ },+});diff --git a/packages/eve/extension-contracts/compatibility/dynamicSkill/v29.ts b/packages/eve/extension-contracts/compatibility/dynamicSkill/v29.tsnew file mode 100644index 000000000..6fd3757ad--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/dynamicSkill/v29.ts@@ -0,0 +1,12 @@+import { defineDynamic, defineSkill } from "#public/skills/index.js";++// Epoch 29 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 30 adds it as optional.+export default defineDynamic({+ events: {+ "turn.started": (_event, ctx) =>+ defineSkill({+ description: "Triage the active incident.",+ markdown: `Triage the incident reported in session ${ctx.session.id}.`,+ }),+ },+});diff --git a/packages/eve/extension-contracts/compatibility/dynamicTool/v68.ts b/packages/eve/extension-contracts/compatibility/dynamicTool/v68.tsnew file mode 100644index 000000000..bf345f288--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/dynamicTool/v68.ts@@ -0,0 +1,14 @@+import { defineDynamic, defineTool } from "#public/tools/index.js";++// Epoch 68 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 69 adds it as optional.+export default defineDynamic({+ events: {+ "turn.started": (_event, ctx) => ({+ session: defineTool({+ description: "Return the active session identifier.",+ inputSchema: { type: "object", properties: {} },+ execute: () => ({ sessionId: ctx.session.id }),+ }),+ }),+ },+});diff --git a/packages/eve/extension-contracts/compatibility/hook/v35.ts b/packages/eve/extension-contracts/compatibility/hook/v35.tsnew file mode 100644index 000000000..c834812ac--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/hook/v35.ts@@ -0,0 +1,14 @@+import { defineHook } from "#public/hooks/index.js";++// Epoch 35 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 36 adds it as optional.+// Hooks that read only the continuation token see no change.+export default defineHook({+ events: {+ "session.waiting"(event, ctx) {+ console.info("session waiting", {+ continuationToken: event.data.continuationToken,+ sessionId: ctx.session.id,+ });+ },+ },+});diff --git a/packages/eve/extension-contracts/compatibility/schedule/v24.ts b/packages/eve/extension-contracts/compatibility/schedule/v24.tsnew file mode 100644index 000000000..863ae5b4c--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/schedule/v24.ts@@ -0,0 +1,22 @@+import { defineChannel } from "#public/channels/index.js";+import { defineSchedule } from "#public/schedules/index.js";++// Epoch 24 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 25 adds it as optional.+const reports = defineChannel({+ routes: [],+ events: {+ "session.waiting"(event) {+ console.info("report waiting", { continuationToken: event.continuationToken });+ },+ },+ receive(input, { from }) {+ return from("nightly-report").send(input.message, { auth: input.auth });+ },+});++export default defineSchedule({+ cron: "0 2 * * *",+ async run({ to, appAuth, waitUntil }) {+ waitUntil(to(reports, {}).send("Prepare the nightly report.", { auth: appAuth }));+ },+});diff --git a/packages/eve/extension-contracts/compatibility/subagent/v27.ts b/packages/eve/extension-contracts/compatibility/subagent/v27.tsnew file mode 100644index 000000000..40252659c--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/subagent/v27.ts@@ -0,0 +1,7 @@+import { defineAgent } from "#public/index.js";++// Epoch 27 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 28 adds it as optional.+export default defineAgent({+ description: "Research the incidents behind a metric change.",+ model: "openai/gpt-5.6-sol",+});diff --git a/packages/eve/extension-contracts/compatibility/tool/v71.ts b/packages/eve/extension-contracts/compatibility/tool/v71.tsnew file mode 100644index 000000000..c301b213c--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/tool/v71.ts@@ -0,0 +1,10 @@+import { z } from "zod";+import { defineTool } from "#public/tools/index.js";++// Epoch 71 `session.waiting`, `session.failed`, `session.completed`, and `turn.waiting` events had no `usage`; epoch 72 adds it as optional.+export default defineTool({+ description: "Summarize the open incidents for a service.",+ inputSchema: z.object({ service: z.string() }),+ outputSchema: z.object({ service: z.string(), summary: z.string() }),+ execute: ({ service }) => ({ service, summary: `No open incidents for ${service}.` }),+});diff --git a/packages/eve/extension-contracts/reports/channel/v44.json b/packages/eve/extension-contracts/reports/channel/v44.jsonnew file mode 100644index 000000000..af730e2ea--- /dev/null+++ b/packages/eve/extension-contracts/reports/channel/v44.json@@ -0,0 +1,21 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "channel",+ "epoch": 44,+ "sha256": "8ba8b7ebd8b86414be4aa8504210b4fb82090acbf0d5abe8f79903c7bd6085da",+ "exports": [+ "DELETE",+ "GET",+ "HEAD",+ "OPTIONS",+ "PATCH",+ "POST",+ "PUT",+ "WS",+ "createWebSocketUpgradeServer",+ "defineChannel",+ "disableRoute",+ "isChannel",+ "isDisabledRouteSentinel"+ ]+}diff --git a/packages/eve/extension-contracts/reports/connection/v37.json b/packages/eve/extension-contracts/reports/connection/v37.jsonnew file mode 100644index 000000000..dc86f6cab--- /dev/null+++ b/packages/eve/extension-contracts/reports/connection/v37.json@@ -0,0 +1,16 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "connection",+ "epoch": 37,+ "sha256": "aaf23a1863d0322eb8f9da99da90573147c9b94da0ab8f8d13fda49d59bf935e",+ "exports": [+ "ConnectionAuthorizationFailedError",+ "ConnectionAuthorizationRequiredError",+ "defineDynamic",+ "defineInteractiveAuthorization",+ "defineMcpClientConnection",+ "defineOpenAPIConnection",+ "isConnectionAuthorizationFailedError",+ "isConnectionAuthorizationRequiredError"+ ]+}diff --git a/packages/eve/extension-contracts/reports/dynamicInstructions/v31.json b/packages/eve/extension-contracts/reports/dynamicInstructions/v31.jsonnew file mode 100644index 000000000..a487a6ced--- /dev/null+++ b/packages/eve/extension-contracts/reports/dynamicInstructions/v31.json@@ -0,0 +1,7 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "dynamicInstructions",+ "epoch": 31,+ "sha256": "08058f0b64f4b76a1adcb69c09377ba21da64340fdc24733e4c945fffde7caec",+ "exports": ["defineDynamic"]+}diff --git a/packages/eve/extension-contracts/reports/dynamicSkill/v30.json b/packages/eve/extension-contracts/reports/dynamicSkill/v30.jsonnew file mode 100644index 000000000..036cdf304--- /dev/null+++ b/packages/eve/extension-contracts/reports/dynamicSkill/v30.json@@ -0,0 +1,7 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "dynamicSkill",+ "epoch": 30,+ "sha256": "3e1844f05fd14258de1479ed8bd4dc03043c0e5b87d435a6d720185f3396054b",+ "exports": ["defineDynamic"]+}diff --git a/packages/eve/extension-contracts/reports/dynamicTool/v69.json b/packages/eve/extension-contracts/reports/dynamicTool/v69.jsonnew file mode 100644index 000000000..1c63f1c86--- /dev/null+++ b/packages/eve/extension-contracts/reports/dynamicTool/v69.json@@ -0,0 +1,16 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "dynamicTool",+ "epoch": 69,+ "sha256": "ea9fbe35180bfd756fa4e7285618639cf5bf216b7e9fa488f46f1ca805cac499",+ "exports": [+ "DynamicToolEntry",+ "DynamicToolEvents",+ "DynamicToolResult",+ "DynamicToolSet",+ "auto",+ "defineDurableCallback",+ "defineDurableSchema",+ "defineDynamic"+ ]+}diff --git a/packages/eve/extension-contracts/reports/hook/v36.json b/packages/eve/extension-contracts/reports/hook/v36.jsonnew file mode 100644index 000000000..9b3985202--- /dev/null+++ b/packages/eve/extension-contracts/reports/hook/v36.json@@ -0,0 +1,7 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "hook",+ "epoch": 36,+ "sha256": "c0ff9bd9f4b2ccdd462f9fd274a3871ddf43c124953e7ba698558cef053b57e2",+ "exports": ["defineHook"]+}diff --git a/packages/eve/extension-contracts/reports/schedule/v25.json b/packages/eve/extension-contracts/reports/schedule/v25.jsonnew file mode 100644index 000000000..9bfaf2dd3--- /dev/null+++ b/packages/eve/extension-contracts/reports/schedule/v25.json@@ -0,0 +1,15 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "schedule",+ "epoch": 25,+ "sha256": "697ab0eefacea4e72d1e837aac5e746d1c7992936a1ac7885800cae49ddbac78",+ "exports": [+ "ScheduleDefinition",+ "ScheduleHandlerArgs",+ "ScheduleRunHandler",+ "ScheduleToFn",+ "TypedReceiveTarget",+ "defineSchedule",+ "isScheduleAuth"+ ]+}diff --git a/packages/eve/extension-contracts/reports/subagent/v28.json b/packages/eve/extension-contracts/reports/subagent/v28.jsonnew file mode 100644index 000000000..52bdfdf74--- /dev/null+++ b/packages/eve/extension-contracts/reports/subagent/v28.json@@ -0,0 +1,23 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "subagent",+ "epoch": 28,+ "sha256": "f9c443883beceedc286257a76db8cbeaeeed3563ce13873a9c91482404bbe2ab",+ "exports": [+ "AgentCompactionDefinition",+ "AgentDefinition",+ "AgentModelDefinition",+ "AgentStaticModelDefinition",+ "DefinedAgent",+ "DynamicLocalSubagentDefinition",+ "DynamicSubagentDefinition",+ "RemoteAgentDefinition",+ "RemoteAgentDefinitionInput",+ "WorkspaceAgentDefinition",+ "WorkspaceAgentTransport",+ "defineAgent",+ "defineDynamic",+ "defineRemoteAgent",+ "defineWorkspaceAgent"+ ]+}diff --git a/packages/eve/extension-contracts/reports/tool/v72.json b/packages/eve/extension-contracts/reports/tool/v72.jsonnew file mode 100644index 000000000..451f8071e--- /dev/null+++ b/packages/eve/extension-contracts/reports/tool/v72.json@@ -0,0 +1,29 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "tool",+ "epoch": 72,+ "sha256": "b25340648af6b8a5671bb6b0d41d36320ab6fbc3f69aa7fb62f434d871ea127a",+ "exports": [+ "AgentRouterInput",+ "AgentRouterTool",+ "WorkflowStepToolContext",+ "WorkflowTool",+ "WorkflowToolDefinition",+ "WorkflowToolInput",+ "WorkflowToolOptions",+ "agentRouter",+ "defaultWebSearch",+ "defineTool",+ "defineWorkflowTool",+ "disableTool",+ "evaluate",+ "isDisabledToolSentinel",+ "isWebSearchToolDefinition",+ "serializeModelInputSchema",+ "toolOutput",+ "toolOutputPart",+ "toolResultFrom",+ "webSearch",+ "workflow"+ ]+}diff --git a/packages/eve/src/acp/adapter.ts b/packages/eve/src/acp/adapter.tsindex 2813560f4..17d763aaa 100644--- a/packages/eve/src/acp/adapter.ts+++ b/packages/eve/src/acp/adapter.ts@@ -211,7 +211,8 @@ export class EveAcpAdapter {let unsupportedEvent: RequestError | undefined;for await (const event of response) {- const turnId = "data" in event && "turnId" in event.data ? event.data.turnId : undefined;+ const data = "data" in event ? event.data : undefined;+ const turnId = data !== undefined && "turnId" in data ? data.turnId : undefined;if (typeof turnId === "string") active.turnId = turnId;if (event.type === "input.requested") inputRequests.push(...event.data.requests);diff --git a/packages/eve/src/channel/adapter.test.ts b/packages/eve/src/channel/adapter.test.tsindex 9189251c2..612f248ef 100644--- a/packages/eve/src/channel/adapter.test.ts+++ b/packages/eve/src/channel/adapter.test.ts@@ -1,3 +1,4 @@+import { TEST_USAGE } from "#internal/testing/events.js";import { describe, expect, it } from "vitest";import type { ChannelAdapter, ChannelAdapterContext, FetchFileResult } from "#channel/adapter.js";@@ -141,10 +142,14 @@ describe("ChannelAdapter helpers", () => {},};- const event = await callAdapterEventHandler(adapter, createSessionWaitingEvent(), context);+ const event = await callAdapterEventHandler(+ adapter,+ createSessionWaitingEvent(TEST_USAGE),+ context,+ );expect(event).toEqual({- data: { continuationToken: "C1:T1", wait: "next-user-message" },+ data: { continuationToken: "C1:T1", usage: TEST_USAGE, wait: "next-user-message" },type: "session.waiting",});expect(observedToken).toBe("temporary");@@ -163,12 +168,12 @@ describe("ChannelAdapter helpers", () => {const event = await callAdapterEventHandler({ kind: "http" },- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),context,);expect(event).toEqual({- data: { continuationToken: "session-1", wait: "next-user-message" },+ data: { continuationToken: "session-1", usage: TEST_USAGE, wait: "next-user-message" },type: "session.waiting",});});diff --git a/packages/eve/src/cli/dev/tui/runner.integration.test.ts b/packages/eve/src/cli/dev/tui/runner.integration.test.tsindex 9e9888920..e15a11b3b 100644--- a/packages/eve/src/cli/dev/tui/runner.integration.test.ts+++ b/packages/eve/src/cli/dev/tui/runner.integration.test.ts@@ -1,3 +1,4 @@+import { TEST_USAGE } from "#internal/testing/events.js";import { afterEach, describe, expect, it, vi } from "vitest";import { Client } from "#client/client.js";import {@@ -113,7 +114,7 @@ describe("eve dev conversation", () => {stepIndex: 0,turnId: "wake",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);await vi.waitFor(() =>expect(assistantText(tui.latest())).toEqual([@@ -195,7 +196,7 @@ describe("eve dev conversation", () => {stepIndex: 0,turnId: "wake",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);await vi.waitFor(() => expect(readToolApproval).toHaveBeenCalledOnce());@@ -246,7 +247,7 @@ describe("eve dev conversation", () => {taskId: "task_1",turnId: "turn_1",}),- createTurnWaitingEvent({ sequence: 1, turnId: "turn_1" }),+ createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 1, turnId: "turn_1" }),],"delivery_1",);@@ -286,7 +287,7 @@ describe("eve dev conversation", () => {status: "completed",}),createTurnCompletedEvent({ sequence: 2, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);await vi.waitFor(() => expect(tui.latest().working).toBe(false));await vi.waitFor(() =>@@ -359,14 +360,17 @@ describe("eve dev conversation", () => {}),]),approval("task_1"),- createTurnWaitingEvent({ sequence: 2, turnId: "turn_1" }),+ createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 2, turnId: "turn_1" }),],"delivery_1",);await vi.waitFor(() => expect(readToolApproval).toHaveBeenCalledOnce());// Bob's approval arrives while Alice is still deciding on the first one.server.emit(- [approval("task_2"), createTurnWaitingEvent({ sequence: 2, turnId: "turn_1" })],+ [+ approval("task_2"),+ createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 2, turnId: "turn_1" }),+ ],"delivery_1",);await vi.waitFor(() => expect(tui.latest().conversation.inputs.approval_task_2).toBeDefined());@@ -415,7 +419,10 @@ describe("eve dev conversation", () => {await vi.waitFor(() => expect(server.requestsTo("POST", "/cancel")).toHaveLength(1));expect(server.requestsTo("POST", "/cancel")[0]?.body).toMatchObject({ turnId: "turn_1" });server.emit(- [createTurnCancelledEvent({ sequence: 1, turnId: "turn_1" }), createSessionWaitingEvent()],+ [+ createTurnCancelledEvent({ sequence: 1, turnId: "turn_1" }),+ createSessionWaitingEvent(TEST_USAGE),+ ],"delivery_1",);await vi.waitFor(() => expect(tui.latest().working).toBe(false));@@ -438,7 +445,10 @@ describe("eve dev conversation", () => {);return () =>server.emit(- [createTurnCancelledEvent({ sequence: 2, turnId }), createSessionWaitingEvent()],+ [+ createTurnCancelledEvent({ sequence: 2, turnId }),+ createSessionWaitingEvent(TEST_USAGE),+ ],delivery,);};@@ -476,6 +486,7 @@ describe("eve dev conversation", () => {[createTurnStartedEvent({ sequence: 0, turnId: "turn_1" }),createSessionFailedEvent({+ usage: TEST_USAGE,code: "SANDBOX_LOST",message: "The sandbox stopped.",sessionId: "session_1",diff --git a/packages/eve/src/cli/dev/tui/runner.test.ts b/packages/eve/src/cli/dev/tui/runner.test.tsindex bdb05b5f6..e39632503 100644--- a/packages/eve/src/cli/dev/tui/runner.test.ts+++ b/packages/eve/src/cli/dev/tui/runner.test.ts@@ -1,3 +1,4 @@+import { TEST_USAGE } from "#internal/testing/events.js";import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";import { Client, ClientError, type AgentInfoResult } from "#client/index.js";@@ -832,7 +833,7 @@ describe("EveTUIRunner gateway-auth failure rendering", () => {async function failuresFor(appRoot?: string): Promise<readonly AgentTUIFailure[]> {const server = new FakeEveServer(({ turnId }) => [{ type: "step.failed", data: { ...gatewayFailure, sequence: 0, stepIndex: 0, turnId } },- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);vi.stubGlobal("fetch", server.fetch);const views: AgentTUIConversationView[] = [];@@ -1280,7 +1281,7 @@ describe("EveTUIRunner registry handoffs", () => {createTurnStartedEvent({ sequence: 0, turnId }),...registryResult("slack-add", "channel/slack"),...registryResult("linear-add", "connection/linear"),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);const handle = vi.fn(async (_command: { name: string; argument: string }) => ({message: "done",@@ -1302,7 +1303,7 @@ describe("EveTUIRunner registry handoffs", () => {const order: string[] = [];const server = serve(({ body, turnId }) =>body?.inputResponses !== undefined- ? (order.push("respond"), [createSessionWaitingEvent()])+ ? (order.push("respond"), [createSessionWaitingEvent(TEST_USAGE)]): [createTurnStartedEvent({ sequence: 0, turnId }),...registryResult("registry-add", "channel/slack"),@@ -1324,7 +1325,7 @@ describe("EveTUIRunner registry handoffs", () => {stepIndex: 0,turnId,}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),],);await new EveTUIRunner({@@ -1378,7 +1379,7 @@ describe("EveTUIRunner registry handoffs", () => {[createTurnStartedEvent({ sequence: 0, turnId: "child_turn" }),...registryResult("registry-add", "channel/slack", "registry_add"),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),],undefined,"child_1",@@ -1392,7 +1393,7 @@ describe("EveTUIRunner registry handoffs", () => {taskId: "task_1",turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),],"delivery_1","session_1",@@ -1526,7 +1527,7 @@ describe("EveTUIRunner requests", () => {it("submits the chosen session-limit continuation", async () => {const server = serve(({ body, turnId }) =>body?.inputResponses !== undefined- ? [createSessionWaitingEvent()]+ ? [createSessionWaitingEvent(TEST_USAGE)]: [createTurnStartedEvent({ sequence: 0, turnId }),createInputRequestedEvent({@@ -1551,7 +1552,7 @@ describe("EveTUIRunner requests", () => {stepIndex: 0,turnId,}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),],);const readInputQuestion = vi.fn(async () => ({ optionId: "continue" }));@@ -1585,7 +1586,7 @@ describe("EveTUIRunner requests", () => {stepIndex: 0,turnId,}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);const readInputQuestion = vi.fn(async () => undefined);const readInput = vi.fn().mockResolvedValueOnce({ type: "submit", text: "Summarize." });@@ -1718,6 +1719,7 @@ describe("EveTUIRunner remote session failures", () => {serve(({ turnId }) => [createTurnStartedEvent({ sequence: 0, turnId }),createSessionFailedEvent({+ usage: TEST_USAGE,code: "HookConflictError",message: "HookConflictError: token in use",sessionId: "session_1",diff --git a/packages/eve/src/cli/dev/tui/terminal-renderer.test.ts b/packages/eve/src/cli/dev/tui/terminal-renderer.test.tsindex 86839b70f..110899759 100644--- a/packages/eve/src/cli/dev/tui/terminal-renderer.test.ts+++ b/packages/eve/src/cli/dev/tui/terminal-renderer.test.ts@@ -13,7 +13,7 @@ import {import { initialConversationState, reduceConversation } from "#client/conversation-reducer.js";import type { EveAgentReducerEvent } from "#client/reducer.js";-import { stampTestEvent } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvent } from "#internal/testing/events.js";import {createActionsRequestedEvent,createMessageAppendedEvent,@@ -3836,7 +3836,7 @@ describe("TerminalRenderer conversation", () => {turnId: "turn_1",}),),- stamped(createTurnWaitingEvent({ sequence: 3, turnId: "turn_1" })),+ stamped(createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 3, turnId: "turn_1" })),];renderer.renderConversation(conversationOf(working, { working: true }));const during = screen.snapshot();diff --git a/packages/eve/src/cli/dev/tui/test/fake-eve-server.ts b/packages/eve/src/cli/dev/tui/test/fake-eve-server.tsindex 3edc9e273..40ddcd0c6 100644--- a/packages/eve/src/cli/dev/tui/test/fake-eve-server.ts+++ b/packages/eve/src/cli/dev/tui/test/fake-eve-server.ts@@ -10,6 +10,14 @@ import {type UnstampedMessageStreamEvent,} from "#protocol/message.js";+/** What the fake server reports a session spent, like a real one does on `session.waiting`. */+const FAKE_USAGE = {+ cacheReadTokens: 0,+ cacheWriteTokens: 0,+ inputTokens: 1200,+ outputTokens: 150,+};+export interface FakeEveRequest {readonly method: string;readonly path: string;@@ -186,7 +194,7 @@ export function reply(text: string): FakeEveTurn {turnId,}),createTurnCompletedEvent({ sequence: 3, turnId }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(FAKE_USAGE),];}diff --git a/packages/eve/src/cli/dev/tui/transcript.test.ts b/packages/eve/src/cli/dev/tui/transcript.test.tsindex c5f6b355f..8a2501deb 100644--- a/packages/eve/src/cli/dev/tui/transcript.test.ts+++ b/packages/eve/src/cli/dev/tui/transcript.test.ts@@ -5,7 +5,7 @@ import {type ConversationEvent,} from "#client/conversation-reducer.js";import type { ConversationState } from "#client/conversation-state.js";-import { stampTestEvent } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvent } from "#internal/testing/events.js";import {createActionInputAppendedEvent,createActionResultEvent,@@ -305,7 +305,7 @@ describe("ConversationTranscript", () => {taskStarted("call_1", "summarize"),toolResult("call_1", "summarize"),toolCall("wait_1", "task_wait"),- event(createTurnWaitingEvent({ sequence: 3, turnId: "turn_1" })),+ event(createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 3, turnId: "turn_1" })),]);const transcript = new ConversationTranscript();const working = view(state, true);diff --git a/packages/eve/src/client/conversation-reducer.test.ts b/packages/eve/src/client/conversation-reducer.test.tsindex 8208f0177..626232050 100644--- a/packages/eve/src/client/conversation-reducer.test.ts+++ b/packages/eve/src/client/conversation-reducer.test.ts@@ -5,7 +5,7 @@ import {type ConversationEvent,} from "#client/conversation-reducer.js";import { openConversationInputs, type ConversationState } from "#client/conversation-state.js";-import { stampTestEvent } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvent } from "#internal/testing/events.js";import {createAgentStartedEvent,createInputRequestedEvent,@@ -172,7 +172,7 @@ describe("conversation reducer tasks and agent sessions", () => {it("marks a parked turn waiting until it resumes or ends", () => {const parked = reduce([createTurnStartedEvent({ sequence: 0, turnId: "t1" }),- createTurnWaitingEvent({ sequence: 0, turnId: "t1" }),+ createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 0, turnId: "t1" }),]);expect(parked.turns.t1).toEqual({ turnId: "t1", status: "active", waiting: true });expect(parked.activeTurnId).toBe("t1");@@ -183,7 +183,7 @@ describe("conversation reducer tasks and agent sessions", () => {expect(resumed.turns.t1).toEqual({ turnId: "t1", status: "active" });const ended = reduce([- createTurnWaitingEvent({ sequence: 0, turnId: "t1" }),+ createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 0, turnId: "t1" }),createTurnCompletedEvent({ sequence: 0, turnId: "t1" }),],resumed,diff --git a/packages/eve/src/client/conversation-state.test.ts b/packages/eve/src/client/conversation-state.test.tsindex e5d3e94a1..fb516a967 100644--- a/packages/eve/src/client/conversation-state.test.ts+++ b/packages/eve/src/client/conversation-state.test.ts@@ -9,7 +9,7 @@ import {isAgentSessionCaughtUp,type ConversationState,} from "#client/conversation-state.js";-import { stampTestEvent } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvent } from "#internal/testing/events.js";import {createAgentStartedEvent,createAuthorizationCompletedEvent,@@ -170,7 +170,7 @@ describe("agent tool sessions", () => {description: "Connect Alice's notes",webhookUrl: "https://example.com/callback",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);expect(caughtUp()).toBe(false);state = observe(state, [diff --git a/packages/eve/src/client/eve-agent-store.test.ts b/packages/eve/src/client/eve-agent-store.test.tsindex 73fecbeec..a63e5594a 100644--- a/packages/eve/src/client/eve-agent-store.test.ts+++ b/packages/eve/src/client/eve-agent-store.test.ts@@ -9,7 +9,7 @@ import { Client } from "#client/client.js";import { defaultMessageReducer } from "#client/message-reducer.js";import { conversationReducer } from "#client/conversation-reducer.js";import type { ConversationState } from "#client/conversation-state.js";-import { stampTestEvent, stampTestEvents } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvent, stampTestEvents } from "#internal/testing/events.js";import {createApprovalSettledEvent,createAuthorizationCompletedEvent,@@ -48,7 +48,7 @@ function turnEvents(): MessageStreamEvent[] {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);}@@ -75,7 +75,7 @@ function streamingTurnEvents(): MessageStreamEvent[] {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);}@@ -390,7 +390,7 @@ describe("EveAgentStore lifecycle", () => {vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) =>init?.method === "POST"? await start.promise- : streamResponse(stampTestEvents([createSessionWaitingEvent()])),+ : streamResponse(stampTestEvents([createSessionWaitingEvent(TEST_USAGE)])),);const store = createStore<readonly string[]>({initialSession: { sessionId: "session_1", streamIndex: 0 },@@ -456,7 +456,7 @@ describe("EveAgentStore held turns", () => {taskId: "task_1",turnId: "turn_1",}),- createTurnWaitingEvent({ sequence: 0, turnId: "turn_1" }),+ createTurnWaitingEvent({ usage: TEST_USAGE, sequence: 0, turnId: "turn_1" }),]).map((event) => ({ ...event, meta: { ...event.meta, deliveryIds: ["delivery_1"] } }));for (const event of parked) live.emit(event);await sending;@@ -482,7 +482,7 @@ describe("EveAgentStore held turns", () => {turnId: "turn_1",}),createTurnCompletedEvent({ sequence: 1, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]).map((event) => ({ ...event, meta: { ...event.meta, id: `resumed-${event.meta.id}` } }));live.emit(resumed[0]!);await vi.waitFor(() => expect(store.snapshot.data.inputs.req_1?.status).toBe("settled"));@@ -523,7 +523,10 @@ describe("EveAgentStore agent-session following", () => {]: []),...(ended- ? [createTurnCompletedEvent({ sequence: 1, turnId: "turn_1" }), createSessionWaitingEvent()]+ ? [+ createTurnCompletedEvent({ sequence: 1, turnId: "turn_1" }),+ createSessionWaitingEvent(TEST_USAGE),+ ]: []),]);@@ -647,7 +650,7 @@ describe("EveAgentStore agent-session following", () => {const sending = store.send({ message: "New question" });const confirmation = stampTestEvents([createMessageReceivedEvent({ message: "New question", sequence: 1, turnId: "turn_2" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]).map((event, index) => ({...event,meta: { ...event.meta, id: `confirmation_${index}`, deliveryIds: ["delivery_1"] },@@ -722,7 +725,9 @@ describe("EveAgentStore prewarming", () => {.spyOn(globalThis, "fetch").mockRejectedValueOnce(error).mockResolvedValueOnce(startedResponse())- .mockResolvedValueOnce(streamResponse(stampTestEvents([createSessionWaitingEvent()])));+ .mockResolvedValueOnce(+ streamResponse(stampTestEvents([createSessionWaitingEvent(TEST_USAGE)])),+ );const onError = vi.fn();const store = createStore({ reducer: defaultMessageReducer() });store.setCallbacks({ onError });@@ -829,7 +834,9 @@ describe("EveAgentStore prewarming", () => {const fetchMock = vi.spyOn(globalThis, "fetch").mockReturnValueOnce(accepted.promise)- .mockResolvedValueOnce(streamResponse(stampTestEvents([createSessionWaitingEvent()])));+ .mockResolvedValueOnce(+ streamResponse(stampTestEvents([createSessionWaitingEvent(TEST_USAGE)])),+ );const onFinish = vi.fn();const onSessionChange = vi.fn();const prepareSend = vi.fn();@@ -926,7 +933,7 @@ describe("EveAgentStore prewarming", () => {stepIndex: 0,turnId: "turn_scheduled",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]).map((event) => ({...event,meta: { ...event.meta, deliveryIds: ["scheduled-delivery"] },@@ -1190,7 +1197,7 @@ describe("EveAgentStore stream overlap", () => {it("applies a pre-v20 event whose envelope has no id", async () => {const legacy = preV20MessageCompletedEvent();- const boundary = stampTestEvents([createSessionWaitingEvent()])[0]!;+ const boundary = stampTestEvents([createSessionWaitingEvent(TEST_USAGE)])[0]!;vi.spyOn(globalThis, "fetch").mockResolvedValueOnce(startedResponse()).mockResolvedValueOnce(streamResponse([legacy, boundary]));@@ -1338,7 +1345,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_0",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]).map((event, index) =>index < 2 ? event : { ...event, meta: { ...event.meta, deliveryIds: ["delivery_1"] } },);@@ -1521,7 +1528,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const initialEvents = [received!, started!];const live = controlledStreamResponse();@@ -1555,6 +1562,7 @@ describe("EveAgentStore session resume", () => {it("publishes a hydrated terminal failure with error status", async () => {const failed = stampTestEvents([createSessionFailedEvent({+ usage: TEST_USAGE,code: "SESSION_FAILED",message: "Session failed.",sessionId: "session_1",@@ -1621,7 +1629,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createMessageReceivedEvent({ message: "Again", sequence: 0, turnId: "turn_2" }),createTurnStartedEvent({ sequence: 1, turnId: "turn_2" }),createMessageCompletedEvent({@@ -1631,7 +1639,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_2",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const settled = events.slice(0, 3);const live = controlledStreamResponse();@@ -1678,7 +1686,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_3",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);const fetchMock = vi.spyOn(globalThis, "fetch")@@ -1707,7 +1715,7 @@ describe("EveAgentStore session resume", () => {turnId: "turn_1",webhookUrl: "https://agent.example.com/callback",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createAuthorizationCompletedEvent({attemptId: "attempt_1",name: "linear",@@ -1716,7 +1724,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const live = controlledStreamResponse();live.response.headers.set("x-eve-stream-tail-index", "1");@@ -1754,7 +1762,7 @@ describe("EveAgentStore session resume", () => {webhookUrl: `https://agent.example.com/callback/${attemptId}`,}),),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createAuthorizationCompletedEvent({attemptId: "first",name: "linear",@@ -1763,7 +1771,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createAuthorizationCompletedEvent({attemptId: "second",name: "linear",@@ -1772,7 +1780,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const live = controlledStreamResponse();live.response.headers.set("x-eve-stream-tail-index", "2");@@ -1808,7 +1816,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createAuthorizationRequiredEvent({authorization: { url: "https://idp.example.com/authorize" },description: "Linear",@@ -1818,7 +1826,7 @@ describe("EveAgentStore session resume", () => {turnId: "turn_2",webhookUrl: "https://agent.example.com/eve/v1/connections/linear/callback/hook",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createAuthorizationCompletedEvent({name: "linear",outcome: "authorized",@@ -1826,7 +1834,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_2",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const settled = events.slice(0, 3);const fetchMock = vi@@ -1855,7 +1863,7 @@ describe("EveAgentStore session resume", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const live = controlledStreamResponse();const requests: string[] = [];@@ -1939,7 +1947,7 @@ describe("EveAgentStore steering", () => {1,),);- live.emit(stampTestEvent(createSessionWaitingEvent(), 2));+ live.emit(stampTestEvent(createSessionWaitingEvent(TEST_USAGE), 2));await Promise.all([first, answering]);expect(store.snapshot.status).toBe("ready");expect(store.snapshot.conversation.inputs["color-a"]?.status).toBe("settled");@@ -2073,7 +2081,7 @@ describe("EveAgentStore steering", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);for (const event of events.slice(0, 3)) active.emit(event);await vi.waitFor(() =>@@ -2110,7 +2118,7 @@ describe("EveAgentStore steering", () => {stepIndex: 1,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]).map((event) => ({...event,meta: { ...event.meta, deliveryIds: ["delivery_1"] },@@ -2151,7 +2159,7 @@ describe("EveAgentStore steering", () => {turnId: "turn_1",}),createMessageReceivedEvent({ message: "Second", sequence: 3, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]).map((event, index) => ({...event,meta: { ...event.meta, deliveryIds: [index === 3 ? "second" : "first"] },@@ -2206,7 +2214,7 @@ describe("EveAgentStore steering", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]).map((event, index) => ({...event,meta: {@@ -2246,7 +2254,7 @@ describe("EveAgentStore steering", () => {const events = stampTestEvents([createMessageReceivedEvent({ message: "First", sequence: 0, turnId: "turn_1" }),createTurnStartedEvent({ sequence: 1, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createMessageReceivedEvent({ message: "Use Linear", sequence: 0, turnId: "turn_2" }),createTurnStartedEvent({ sequence: 1, turnId: "turn_2" }),createAuthorizationRequiredEvent({@@ -2258,7 +2266,7 @@ describe("EveAgentStore steering", () => {turnId: "turn_2",webhookUrl: "https://agent.example.com/callback",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createAuthorizationCompletedEvent({attemptId: "attempt_1",name: "linear",@@ -2267,7 +2275,7 @@ describe("EveAgentStore steering", () => {stepIndex: 0,turnId: "turn_2",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]).map((event, index) => ({...event,meta: { ...event.meta, deliveryIds: [index < 3 ? "first-delivery" : "delivery_1"] },@@ -2319,7 +2327,7 @@ describe("EveAgentStore steering", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),createMessageReceivedEvent({ message: "Instead", sequence: 0, turnId: "turn_2" }),createTurnStartedEvent({ sequence: 1, turnId: "turn_2" }),createMessageCompletedEvent({@@ -2329,7 +2337,7 @@ describe("EveAgentStore steering", () => {stepIndex: 0,turnId: "turn_2",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]).map((event, index) => ({...event,meta: { ...event.meta, deliveryIds: [index < 4 ? "first-delivery" : "delivery_1"] },@@ -2394,6 +2402,7 @@ describe("EveAgentStore terminal failure", () => {it("publishes a live terminal failure with error status", async () => {const failed = stampTestEvents([createSessionFailedEvent({+ usage: TEST_USAGE,code: "SESSION_FAILED",message: "Session failed.",sessionId: "session_1",@@ -2426,7 +2435,7 @@ describe("EveAgentStore cancellation", () => {const [turnStarted, turnCancelled, boundary] = stampTestEvents([createTurnStartedEvent({ sequence: 0, turnId: "turn_1" }),createTurnCancelledEvent({ sequence: 1, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const fetchMock = vi.spyOn(globalThis, "fetch")@@ -2553,7 +2562,7 @@ describe("EveAgentStore session controls", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);vi.spyOn(globalThis, "fetch").mockImplementation(async (_input, init) =>init?.method === "POST"diff --git a/packages/eve/src/client/eve-agent-store.ts b/packages/eve/src/client/eve-agent-store.tsindex 01230be7a..d50614f76 100644--- a/packages/eve/src/client/eve-agent-store.ts+++ b/packages/eve/src/client/eve-agent-store.ts@@ -627,7 +627,9 @@ export class EveAgentStore<TData> {const settled =isSettledSessionTail(this.#events, conversation) && conversation.activeTurnId === undefined;if (this.#status !== "resuming" && this.#error === undefined) {- if ("data" in event && "turnId" in event.data) this.#status = "streaming";+ if ("data" in event && event.data !== undefined && "turnId" in event.data) {+ this.#status = "streaming";+ }if (this.#activeTurn === undefined && settled) this.#status = "ready";}this.#callbacks.onSessionChange?.(this.#session?.state);diff --git a/packages/eve/src/client/message-response.test.ts b/packages/eve/src/client/message-response.test.tsindex 46007cd27..541f0c25d 100644--- a/packages/eve/src/client/message-response.test.ts+++ b/packages/eve/src/client/message-response.test.ts@@ -1,7 +1,7 @@import { describe, expect, it, vi } from "vitest";import { MessageResponse } from "#client/message-response.js";-import { stampTestEvents } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvents } from "#internal/testing/events.js";import {createSessionWaitingEvent,createTurnStartedEvent,@@ -29,7 +29,7 @@ describe("MessageResponse cancellation", () => {const settle = createDeferred<void>();const [turnStarted, boundary] = stampTestEvents([createTurnStartedEvent({ sequence: 0, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const cancelTurn = vi.fn(async () => acceptedCancellation());const response = new MessageResponse({@@ -62,7 +62,7 @@ describe("MessageResponse cancellation", () => {const settle = createDeferred<void>();const [turnStarted, boundary] = stampTestEvents([createTurnStartedEvent({ sequence: 0, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const cancelTurn = vi.fn<(turnId: string) => Promise<ReturnType<typeof acceptedCancellation>>>()@@ -91,7 +91,7 @@ describe("MessageResponse cancellation", () => {it("drops a queued cancellation when the response settles without starting a turn", async () => {const [boundary] = stampTestEvents([- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),] as UnstampedMessageStreamEvent[]);const cancelTurn = vi.fn(async () => acceptedCancellation());const response = new MessageResponse({diff --git a/packages/eve/src/compiler/extension-compatibility.ts b/packages/eve/src/compiler/extension-compatibility.tsindex efb5022d1..fe9efe275 100644--- a/packages/eve/src/compiler/extension-compatibility.ts+++ b/packages/eve/src/compiler/extension-compatibility.ts@@ -22,10 +22,10 @@ interface ExtensionCapabilityContract {const EXTENSION_CAPABILITY_CONTRACTS = {extension: { current: 1, supported: [1], dropped: {} },tool: {- current: 71,+ current: 72,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 29, 30, 31, 32, 34, 35, 54, 55, 63, 64, 65, 66, 67,- 68, 69, 70, 71,+ 68, 69, 70, 71, 72,],dropped: {14: "TaskExec.delegated was removed; migrate to workflow-backed background tools",@@ -72,10 +72,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},dynamicTool: {- current: 68,+ current: 69,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 31, 32, 33, 52, 54,- 60, 61, 62, 63, 64, 65, 66, 67, 68,+ 60, 61, 62, 63, 64, 65, 66, 67, 68, 69,],dropped: {21: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",@@ -114,10 +114,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},channel: {- current: 43,+ current: 44,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 29, 31, 36, 37, 38, 39, 40, 41, 42,- 43,+ 43, 44,],dropped: {12: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",@@ -139,8 +139,8 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},schedule: {- current: 24,- supported: [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24],+ current: 25,+ supported: [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25],dropped: {5: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",16: 'Background task delivery was removed: sends no longer accept taskDeliveryPolicy and message.completed always carries text. input.resolved reports outcome "cancelled" when a workflow tool withdraws a ctx.ask() request, and stream events gained agent.started, published when a workflow run opens a ctx.agent session, and turn.waiting, which parks the open turn instead of emitting turn.completed and session.waiting.',@@ -148,8 +148,8 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},subagent: {- current: 27,- supported: [3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 23, 24, 25, 26, 27],+ current: 28,+ supported: [3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 23, 24, 25, 26, 27, 28],dropped: {1: "Persistent subagent sessions are now the default and the experimental opt-in was removed",2: "Persistent subagent sessions are now the default and the experimental opt-in was removed",@@ -162,10 +162,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},connection: {- current: 36,+ current: 37,supported: [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 26, 32, 33, 34,- 35, 36,+ 35, 36, 37,],dropped: {9: "Dynamic connection resolvers no longer receive conversation or channel continuation data",@@ -180,8 +180,8 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},hook: {- current: 35,- supported: [20, 21, 22, 23, 25, 27, 31, 32, 33, 34, 35],+ current: 36,+ supported: [20, 21, 22, 23, 25, 27, 31, 32, 33, 34, 35, 36],dropped: {1: "Model identity moved from session.started runtime metadata to step.started call attribution.",2: "Model identity moved from session.started runtime metadata to step.started call attribution.",@@ -211,9 +211,9 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},skill: { current: 2, supported: [1, 2], dropped: {} },dynamicSkill: {- current: 29,+ current: 30,supported: [- 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 25, 26, 27, 28, 29,+ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 25, 26, 27, 28, 29, 30,],dropped: {13: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",@@ -224,9 +224,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},instructions: { current: 2, supported: [1, 2], dropped: {} },dynamicInstructions: {- current: 30,+ current: 31,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 26, 27, 28, 29, 30,+ 31,],dropped: {14: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",diff --git a/packages/eve/src/context/hook-lifecycle.integration.test.ts b/packages/eve/src/context/hook-lifecycle.integration.test.tsindex 61c7d4128..4a4631207 100644--- a/packages/eve/src/context/hook-lifecycle.integration.test.ts+++ b/packages/eve/src/context/hook-lifecycle.integration.test.ts@@ -9,7 +9,7 @@ import {createTurnStartedEvent,type UnstampedMessageStreamEvent,} from "#protocol/message.js";-import { stampTestEvent } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvent } from "#internal/testing/events.js";import { mockSandbox } from "#internal/testing/mocks/mock-sandbox.js";import { ContextContainer, contextStorage } from "./container.js";import { dispatchStreamEventHooks } from "./hook-lifecycle.js";@@ -113,6 +113,7 @@ describe("dispatchStreamEventHooks", () => {createStepStartedEvent({ sequence: 0, turnId: "turn_0", stepIndex: 0, modelId: "test" }),{ type: "session.completed" } as const,createSessionFailedEvent({+ usage: TEST_USAGE,code: "TEST_FAILURE",message: "Runtime failure",sessionId: "session_test",diff --git a/packages/eve/src/evals/runner/derive-run-facts.test.ts b/packages/eve/src/evals/runner/derive-run-facts.test.tsindex 96c36f354..626858168 100644--- a/packages/eve/src/evals/runner/derive-run-facts.test.ts+++ b/packages/eve/src/evals/runner/derive-run-facts.test.ts@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";import type { UnstampedMessageStreamEvent } from "#protocol/message.js";import type { JsonObject } from "#shared/json.js";+import type { TokenUsage } from "#shared/token-usage.js";import { stampTestEvents } from "#internal/testing/events.js";import { deriveRunFacts, type DeriveRunFactsOptions } from "#evals/runner/derive-run-facts.js";import type { EveEvalDerivedFacts } from "#evals/types.js";@@ -66,6 +67,22 @@ function stepStarted(modelId: string, stepIndex: number): UnstampedMessageStreamreturn { type: "step.started", data: { modelId, sequence: 1, stepIndex, turnId: "t1" } };}+const USAGE = { cacheReadTokens: 0, cacheWriteTokens: 0, inputTokens: 40, outputTokens: 8 };++function sessionFailed(usage: TokenUsage): UnstampedMessageStreamEvent {+ return {+ type: "session.failed",+ data: { code: "MODEL_CALL_FAILED", message: "failed", sessionId: "s1", usage },+ };+}++function sessionWaiting(usage: TokenUsage | undefined): UnstampedMessageStreamEvent {+ return {+ type: "session.waiting",+ data: { continuationToken: "", wait: "next-user-message", ...(usage && { usage }) },+ };+}+function taskStarted(callId: string,name: string,@@ -443,6 +460,50 @@ describe("deriveRunFacts", () => {expect(facts.models).toEqual(["anthropic/claude-sonnet-4.6", "openai/gpt-5.1"]);});+ it("reports the usage of the session's latest session.waiting", () => {+ const facts = derive([+ sessionWaiting({ cacheReadTokens: 0, cacheWriteTokens: 0, inputTokens: 10, outputTokens: 2 }),+ sessionWaiting({+ cacheReadTokens: 1,+ cacheWriteTokens: 0,+ costUsd: 0.5,+ inputTokens: 40,+ outputTokens: 8,+ }),+ ]);+ expect(facts.usage).toEqual({+ cacheReadTokens: 1,+ cacheWriteTokens: 0,+ costUsd: 0.5,+ inputTokens: 40,+ outputTokens: 8,+ });+ });++ it("reports the usage of a session.failed or session.completed that came after it", () => {+ const failed = { cacheReadTokens: 0, cacheWriteTokens: 0, inputTokens: 90, outputTokens: 9 };+ const completed = { cacheReadTokens: 0, cacheWriteTokens: 0, inputTokens: 70, outputTokens: 7 };+ expect([+ derive([sessionWaiting(USAGE), sessionFailed(failed)]).usage,+ derive([sessionWaiting(USAGE), { type: "session.completed", data: { usage: completed } }])+ .usage,+ ]).toEqual([failed, completed]);+ });++ it("reports the usage of a turn.waiting that parked the turn after it", () => {+ const parked = { cacheReadTokens: 0, cacheWriteTokens: 0, inputTokens: 55, outputTokens: 5 };+ expect(+ derive([+ sessionWaiting(USAGE),+ { type: "turn.waiting", data: { sequence: 2, turnId: "t2", usage: parked } },+ ]).usage,+ ).toEqual(parked);+ });++ it("reports no usage when the latest session event carried none", () => {+ expect(derive([sessionWaiting(USAGE), sessionWaiting(undefined)]).usage).toBeUndefined();+ });+it("captures failure code from session.failed event", () => {const events: UnstampedMessageStreamEvent[] = [{diff --git a/packages/eve/src/evals/runner/derive-run-facts.ts b/packages/eve/src/evals/runner/derive-run-facts.tsindex 7d9425d9f..38bee80d0 100644--- a/packages/eve/src/evals/runner/derive-run-facts.ts+++ b/packages/eve/src/evals/runner/derive-run-facts.ts@@ -6,6 +6,7 @@ import type {import { LOAD_SKILL_TOOL_NAME } from "#runtime/skills/fragment-context.js";import type { InputRequest } from "#shared/input.js";import type { JsonObject, JsonValue } from "#shared/json.js";+import type { TokenUsage } from "#shared/token-usage.js";import type { EveEvalDerivedFacts, EveEvalSubagentCall, EveEvalToolCall } from "#evals/types.js";interface MutableToolCall {@@ -68,6 +69,7 @@ export function deriveRunFacts(let messageCount = 0;let reasoningBlockCount = 0;const models = new Set<string>();+ let usage: TokenUsage | undefined;let failureCode: string | undefined;const ensureToolCall = (callId: string, name: string, input: JsonObject): MutableToolCall => {@@ -157,7 +159,19 @@ export function deriveRunFacts(break;}+ case "session.waiting":+ case "turn.waiting": {+ usage = event.data.usage;+ break;+ }++ case "session.completed": {+ usage = event.data?.usage;+ break;+ }+case "session.failed": {+ usage = event.data.usage;failureCode = event.data.code;break;}@@ -180,6 +194,7 @@ export function deriveRunFacts(messageCount,reasoningBlockCount,models: [...models],+ usage,failureCode,};}diff --git a/packages/eve/src/evals/runner/execute-task.test.ts b/packages/eve/src/evals/runner/execute-task.test.tsindex 9ea5697f1..4e1593f29 100644--- a/packages/eve/src/evals/runner/execute-task.test.ts+++ b/packages/eve/src/evals/runner/execute-task.test.ts@@ -10,6 +10,7 @@ import {import { stampTestEvents } from "#internal/testing/events.js";import { executeTask } from "#evals/runner/execute-task.js";import type { EveEval, EveEvalContext } from "#evals/types.js";+import type { TokenUsage } from "#shared/token-usage.js";import { createEvalTargetHandle } from "#evals/target.js";import { satisfies } from "#evals/expect/index.js";import { z } from "zod";@@ -522,6 +523,157 @@ describe("executeTask", () => {});});+ it("sums the usage of every session except those another captured session opened", async () => {+ const server = createScriptedServer(+ [+ {+ sessionId: "parent",+ events: [+ turnStarted("turn_1"),+ agentStarted("child", "researcher"),+ messageCompleted("parent done", "turn_1"),+ sessionWaiting(usage(100)),+ ],+ },+ {+ sessionId: "independent",+ events: [+ turnStarted("turn_3"),+ messageCompleted("done", "turn_3"),+ sessionWaiting(usage(7)),+ ],+ },+ ],+ {+ streams: [+ {+ sessionId: "child",+ events: [+ turnStarted("turn_2"),+ messageCompleted("done", "turn_2"),+ sessionWaiting(usage(60)),+ ],+ },+ ],+ },+ );+ vi.spyOn(globalThis, "fetch").mockImplementation(server.fetch);++ const { result } = await executeTask({+ client: new Client({ host: target.url }),+ target,+ evaluation: createTestEval(async (t) => {+ await t.send("parent");+ await t.target.attachSession("child");+ await t.send("independent");+ }, "delegated-usage"),+ });++ expect(result.derived.usage).toEqual(usage(107, 2));+ });++ it("counts a session captured twice once, by its latest usage", async () => {+ const server = createScriptedServer([], {+ streams: [+ {+ sessionId: "scheduled",+ events: [+ turnStarted("turn_1"),+ messageCompleted("first", "turn_1"),+ sessionWaiting(usage(10)),+ ],+ },+ {+ sessionId: "scheduled",+ events: [+ turnStarted("turn_2"),+ messageCompleted("second", "turn_2"),+ sessionWaiting(usage(30)),+ ],+ },+ ],+ });+ vi.spyOn(globalThis, "fetch").mockImplementation(server.fetch);++ const { result } = await executeTask({+ client: new Client({ host: target.url }),+ target,+ evaluation: createTestEval(async (t) => {+ await t.target.attachSession("scheduled");+ await t.target.attachSession("scheduled");+ }, "recaptured-usage"),+ });++ expect(result.derived.usage).toEqual(usage(30, 1));+ });++ it("sums cost over the counted sessions that reported one", async () => {+ const server = createScriptedServer([+ {+ sessionId: "priced",+ events: [+ turnStarted("turn_1"),+ messageCompleted("done", "turn_1"),+ sessionWaiting(usage(5, 1, 0.5)),+ ],+ },+ {+ sessionId: "unpriced",+ events: [+ turnStarted("turn_2"),+ messageCompleted("done", "turn_2"),+ sessionWaiting(usage(7)),+ ],+ },+ ]);+ vi.spyOn(globalThis, "fetch").mockImplementation(server.fetch);++ const { result } = await executeTask({+ client: new Client({ host: target.url }),+ target,+ evaluation: createTestEval(async (t) => {+ await t.send("priced");+ await t.send("unpriced");+ }, "partial-cost"),+ });++ expect(result.derived.usage?.costUsd).toBe(0.5);+ });++ it("reports no usage when a counted session reported none", async () => {+ const server = createScriptedServer([+ {+ sessionId: "reported",+ events: [+ turnStarted("turn_1"),+ messageCompleted("done", "turn_1"),+ sessionWaiting({+ cacheReadTokens: 0,+ cacheWriteTokens: 0,+ inputTokens: 5,+ outputTokens: 1,+ }),+ ],+ },+ {+ sessionId: "unreported",+ events: [turnStarted("turn_2"), messageCompleted("done", "turn_2"), sessionCompleted()],+ },+ ]);+ vi.spyOn(globalThis, "fetch").mockImplementation(server.fetch);++ const { result } = await executeTask({+ client: new Client({ host: target.url }),+ target,+ evaluation: createTestEval(async (t) => {+ await t.send("reported");+ await t.send("unreported");+ }, "unreported-usage"),+ });++ expect(result.derived.usage).toBeUndefined();+ });+it("records assertions against individual turns without leaking other turns", async () => {const server = createScriptedServer([{@@ -1067,9 +1219,13 @@ function turnCompleted(turnId: string): UnstampedMessageStreamEvent {return { data: { sequence: 3, turnId }, type: "turn.completed" };}-function sessionWaiting(): UnstampedMessageStreamEvent {+function usage(inputTokens: number, outputTokens = 1, costUsd?: number): TokenUsage {+ return { cacheReadTokens: 0, cacheWriteTokens: 0, costUsd, inputTokens, outputTokens };+}++function sessionWaiting(usage?: TokenUsage): UnstampedMessageStreamEvent {return {- data: { continuationToken: "session-id", wait: "next-user-message" },+ data: { continuationToken: "session-id", wait: "next-user-message", ...(usage && { usage }) },type: "session.waiting",};}diff --git a/packages/eve/src/evals/runner/execute-task.ts b/packages/eve/src/evals/runner/execute-task.tsindex be1061327..c81d16b11 100644--- a/packages/eve/src/evals/runner/execute-task.ts+++ b/packages/eve/src/evals/runner/execute-task.ts@@ -2,6 +2,7 @@ import { runUntilAborted } from "#evals/abort.js";import type { Client } from "#client/client.js";import type { MessageStreamEvent, RuntimeIdentity } from "#protocol/message.js";import { toErrorMessage } from "#shared/errors.js";+import { addTokenUsage, type TokenUsage } from "#shared/token-usage.js";import type {AssertionResult,EveEval,@@ -172,10 +173,32 @@ function combineDerivedFacts(sessions: readonly EveEvalSessionResult[]): EveEvalmessageCount: sum(sessions, (session) => session.derived.messageCount),reasoningBlockCount: sum(sessions, (session) => session.derived.reasoningBlockCount),models: [...new Set(sessions.flatMap((session) => session.derived.models))],+ usage: evalUsage(sessions),failureCode,};}+/**+ * The eval's usage: each captured session's latest usage, counted once however often the eval+ * captured it, less the sessions another captured session opened, whose spend that session already+ * counts. No usage when a counted session reported none.+ */+function evalUsage(sessions: readonly EveEvalSessionResult[]): TokenUsage | undefined {+ const opened = new Set(+ sessions.flatMap((session) =>+ session.events.flatMap((event) =>+ event.type === "agent.started" ? [event.data.sessionId] : [],+ ),+ ),+ );+ const latestById = new Map(sessions.map((session) => [session.sessionId, session.derived.usage]));+ const counted = [...latestById].flatMap(([id, usage]) =>+ id !== undefined && opened.has(id) ? [] : [usage],+ );+ if (!counted.every((usage): usage is TokenUsage => usage !== undefined)) return undefined;+ return counted.reduce(addTokenUsage);+}+function selectPrimarySessionId(sessions: readonly EveEvalSessionResult[]): string | undefined {return sessions.find((session) => session.primary)?.sessionId ?? sessions[0]?.sessionId;}diff --git a/packages/eve/src/evals/types.ts b/packages/eve/src/evals/types.tsindex 7a610dc04..80a0a4a5c 100644--- a/packages/eve/src/evals/types.ts+++ b/packages/eve/src/evals/types.ts@@ -1,3 +1,4 @@+import type { TokenUsage } from "#shared/token-usage.js";import type { Experimental_EvaluationModel as EvaluationModel } from "ai";import type { StandardSchemaV1 } from "#compiled/@standard-schema/spec/index.js";@@ -94,6 +95,14 @@ export interface EveEvalDerivedFacts {* attaches that session. Compaction and `auto` routing calls are not included.*/readonly models: readonly string[];+ /**+ * Token usage from the latest `session.waiting`, `turn.waiting`, `session.failed`, or+ * `session.completed`: the+ * session's own model calls plus what the agents it delegated to spent, so on a turn it is the+ * session's total so far. For an eval, each captured session counts once, by its latest usage,+ * except sessions another captured session opened. Absent when a counted session reported none.+ */+ readonly usage?: TokenUsage;readonly failureCode?: string;}diff --git a/packages/eve/src/execution/session-event-stream.test.ts b/packages/eve/src/execution/session-event-stream.test.tsindex 7491d4073..3daaefed0 100644--- a/packages/eve/src/execution/session-event-stream.test.ts+++ b/packages/eve/src/execution/session-event-stream.test.ts@@ -1,6 +1,6 @@import { afterEach, describe, expect, it, vi } from "vitest";-import { stampTestEvent } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvent } from "#internal/testing/events.js";import { createSessionWaitingEvent, type MessageStreamEvent } from "#protocol/message.js";import { sessions } from "#public/server/index.js";@@ -45,7 +45,9 @@ function mockSessionStream(}const storedEvents = (count: number) =>- Array.from({ length: count }, (_, index) => stampTestEvent(createSessionWaitingEvent(), index));+ Array.from({ length: count }, (_, index) =>+ stampTestEvent(createSessionWaitingEvent(TEST_USAGE), index),+ );async function collect(iterable: AsyncIterable<MessageStreamEvent>) {const events: MessageStreamEvent[] = [];diff --git a/packages/eve/src/execution/session/finalization.test.ts b/packages/eve/src/execution/session/finalization.test.tsindex c9987844d..36c98bf1c 100644--- a/packages/eve/src/execution/session/finalization.test.ts+++ b/packages/eve/src/execution/session/finalization.test.ts@@ -3,6 +3,8 @@ import { finalizeSession } from "#execution/session/finalization.js";import { createDurableSessionState } from "#execution/durable-session-store.js";import { setTurnUsageState, takeSessionUsageDelta } from "#harness/turn-tag-state.js";import type { HarnessSession } from "#harness/types.js";+import { emitTerminalSessionCompletionStep } from "#execution/terminal-session-completion-step.js";+import { emitTerminalSessionFailureStep } from "#execution/terminal-session-failure-step.js";import { notifyTurnCallerStep } from "#subagents/parent-notification.js";vi.mock("#execution/terminate-child-sessions-step.js", () => ({@@ -80,6 +82,29 @@ describe("session finalization with an unsettled caller", () => {},);+ it.each([+ { kind: "expired" as const, step: emitTerminalSessionCompletionStep },+ {+ kind: "failed" as const,+ error: new Error("Owner failed"),+ step: emitTerminalSessionFailureStep,+ },+ ])("ends a $kind session with its usage on the terminal event", async ({ step, ...outcome }) => {+ await finalizeSession(outcome, {+ caller: undefined,+ cursor: { serializedContext: {}, sessionState: sessionWithUnreportedUsage() },+ sessionWritable: new WritableStream(),+ });++ expect(vi.mocked(step).mock.calls[0]?.[0].usage).toEqual({+ cacheReadTokens: 0,+ cacheWriteTokens: 0,+ costUsd: undefined,+ inputTokens: 250,+ outputTokens: 0,+ });+ });+it("does not notify again when an already settled conversation expires", async () => {const result = await finalizeSession({ kind: "expired" },diff --git a/packages/eve/src/execution/session/finalization.ts b/packages/eve/src/execution/session/finalization.tsindex 5371d3964..80c66f18c 100644--- a/packages/eve/src/execution/session/finalization.ts+++ b/packages/eve/src/execution/session/finalization.ts@@ -7,7 +7,7 @@ import type { TurnOutcome } from "#execution/session/turn-step-types.js";import { normalizeSerializableError } from "#execution/workflow-errors.js";import type { WorkflowEntryResult } from "#execution/session/entry-input.js";import type { TokenUsage } from "#shared/token-usage.js";-import { getSessionTokenUsage, takeSessionUsageDelta, toUsage } from "#harness/turn-tag-state.js";+import { getSessionUsage, takeSessionUsageDelta } from "#harness/turn-tag-state.js";import { notifyTurnCallerStep } from "#subagents/parent-notification.js";/** The three ways a session ends. `done` already emitted its terminal event inside the turn. */@@ -37,10 +37,13 @@ export async function finalizeSession(if (sessionState !== undefined) {await terminateChildSessionsStep({ sessionState });}+ const session = sessionState?.snapshot.session;+ const usage = session === undefined ? undefined : getSessionUsage(session);if (outcome.kind === "expired") {await emitTerminalSessionCompletionStep({sessionWritable: context.sessionWritable,serializedContext,+ usage,});} else if (outcome.kind === "failed") {await emitTerminalSessionFailureStep({@@ -48,6 +51,7 @@ export async function finalizeSession(sessionWritable: context.sessionWritable,serializedContext,turnId: outcome.turnId,+ usage,});}@@ -94,7 +98,7 @@ function settledResult(session === undefined || context.caller === undefined? {}: {- sessionUsage: toUsage(getSessionTokenUsage(session)),+ sessionUsage: getSessionUsage(session),turnUsage: takeSessionUsageDelta(session).delta,};switch (outcome.kind) {diff --git a/packages/eve/src/execution/session/turn-step.test.ts b/packages/eve/src/execution/session/turn-step.test.tsindex a8e2156ca..a7d99f3f2 100644--- a/packages/eve/src/execution/session/turn-step.test.ts+++ b/packages/eve/src/execution/session/turn-step.test.ts@@ -1,3 +1,4 @@+import { TEST_USAGE } from "#internal/testing/events.js";import { createTestSessionState } from "#internal/testing/session-state.js";import type { ModelMessage } from "ai";import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";@@ -2810,6 +2811,7 @@ describe("emitTerminalSessionFailureStep", () => {const errorLog = vi.spyOn(console, "error").mockImplementation(() => {});await emitTerminalSessionFailureStep({+ usage: TEST_USAGE,error,sessionWritable: createTestWritable(),serializedContext: serialized,@@ -2859,6 +2861,7 @@ describe("emitTerminalSessionFailureStep", () => {});await emitTerminalSessionFailureStep({+ usage: TEST_USAGE,error,sessionWritable: createTestWritable(),serializedContext: serialized,@@ -2907,6 +2910,7 @@ describe("emitTerminalSessionFailureStep", () => {// a secondary failure during notification.await expect(emitTerminalSessionFailureStep({+ usage: TEST_USAGE,error: new Error("inner"),sessionWritable: createTestWritable(),serializedContext: serialized,diff --git a/packages/eve/src/execution/session/turn-waiting-step.ts b/packages/eve/src/execution/session/turn-waiting-step.tsindex 7913c976d..fd866402d 100644--- a/packages/eve/src/execution/session/turn-waiting-step.ts+++ b/packages/eve/src/execution/session/turn-waiting-step.ts@@ -3,6 +3,8 @@ import {withSessionStateDelta,type SessionStateTransition,} from "#execution/session/state-delta.js";+import { readDurableSession } from "#execution/durable-session-store.js";+import { getSessionUsage } from "#harness/turn-tag-state.js";import { createTurnWaitingEvent } from "#protocol/message.js";/** Publishes `turn.waiting` for the open turn the session workflow just parked. */@@ -13,6 +15,7 @@ export async function publishTurnWaitingStep(return await withSessionStateDelta(target, async (input) => {const { sequence, turnId } = input.sessionState.emissionState;- return await publishSessionEvents(input, [createTurnWaitingEvent({ sequence, turnId })]);+ const usage = getSessionUsage(readDurableSession(input.sessionState));+ return await publishSessionEvents(input, [createTurnWaitingEvent({ sequence, turnId, usage })]);});}diff --git a/packages/eve/src/execution/settle-cancelled-turn-step.ts b/packages/eve/src/execution/settle-cancelled-turn-step.tsindex c5a42c02c..dcb3a6ec2 100644--- a/packages/eve/src/execution/settle-cancelled-turn-step.ts+++ b/packages/eve/src/execution/settle-cancelled-turn-step.ts@@ -18,7 +18,11 @@ import type { HarnessModelMessage } from "#harness/messages.js";import { clearPendingSessionLimitPrompt } from "#harness/input-requests.js";import { getHarnessEmissionState, setHarnessEmissionState } from "#harness/emission.js";import { removeBlockingWorkflowToolRuns } from "#harness/workflow-tool-runs.js";-import { getTurnUsageState, takeSessionUsageDelta } from "#harness/turn-tag-state.js";+import {+ getSessionUsage,+ getTurnUsageState,+ takeSessionUsageDelta,+} from "#harness/turn-tag-state.js";import type { TokenUsage } from "#shared/token-usage.js";export interface CancelledTurnSettleResult {@@ -65,7 +69,12 @@ export async function settleCancelledTurn(const durableState = step.durableSession.state;const { published, result: usage } = await publishFromSessionStep(step, {origin: "own",- publish: (emit) => emitCancelledTurn(emit, getHarnessEmissionState(durableState)),+ publish: (emit) =>+ emitCancelledTurn(+ emit,+ getHarnessEmissionState(durableState),+ getSessionUsage(step.durableSession),+ ),updateSession(session, emissionState) {// `clearPendingSessionLimitPrompt`: cancellation settles with the step's// input snapshot, which can resurrect an already-answered session-limitdiff --git a/packages/eve/src/execution/terminal-session-completion-step.ts b/packages/eve/src/execution/terminal-session-completion-step.tsindex 5820d10b8..76000c52a 100644--- a/packages/eve/src/execution/terminal-session-completion-step.ts+++ b/packages/eve/src/execution/terminal-session-completion-step.ts@@ -1,12 +1,18 @@import { publishTerminalSessionEvent } from "#execution/publish-session-events.js";import { createSessionCompletedEvent } from "#protocol/message.js";+import type { TokenUsage } from "#shared/token-usage.js";/** Emits a terminal `session.completed` outside a turn. */export async function emitTerminalSessionCompletionStep(input: {readonly sessionWritable: WritableStream<Uint8Array>;readonly serializedContext: Record<string, unknown>;+ readonly usage: TokenUsage | undefined;}): Promise<void> {"use step";- await publishTerminalSessionEvent({ ...input, event: createSessionCompletedEvent() });+ await publishTerminalSessionEvent({+ sessionWritable: input.sessionWritable,+ serializedContext: input.serializedContext,+ event: createSessionCompletedEvent(input.usage),+ });}diff --git a/packages/eve/src/execution/terminal-session-failure-step.ts b/packages/eve/src/execution/terminal-session-failure-step.tsindex ec0c568b3..ff956af11 100644--- a/packages/eve/src/execution/terminal-session-failure-step.ts+++ b/packages/eve/src/execution/terminal-session-failure-step.ts@@ -2,6 +2,7 @@ import { publishTerminalSessionEvent } from "#execution/publish-session-events.jimport { summarizeKnownError } from "#harness/semantic-errors/index.js";import { createLogger, formatError } from "#internal/logging.js";import { createSessionFailedEvent } from "#protocol/message.js";+import type { TokenUsage } from "#shared/token-usage.js";const log = createLogger("execution.workflow-entry");@@ -11,6 +12,7 @@ export async function emitTerminalSessionFailureStep(input: {readonly sessionWritable: WritableStream<Uint8Array>;readonly serializedContext: Record<string, unknown>;readonly turnId?: string;+ readonly usage: TokenUsage | undefined;}): Promise<void> {"use step";@@ -41,7 +43,7 @@ export async function emitTerminalSessionFailureStep(input: {await publishTerminalSessionEvent({errorId: typeof details.errorId === "string" ? details.errorId : undefined,- event: createSessionFailedEvent({ code, details, message, sessionId }),+ event: createSessionFailedEvent({ code, details, message, sessionId, usage: input.usage }),sessionWritable: input.sessionWritable,serializedContext: input.serializedContext,turnId: input.turnId,diff --git a/packages/eve/src/execution/tools/workflow/workflow-tool-run.integration.test.ts b/packages/eve/src/execution/tools/workflow/workflow-tool-run.integration.test.tsindex 6c97ee831..b738f9f49 100644--- a/packages/eve/src/execution/tools/workflow/workflow-tool-run.integration.test.ts+++ b/packages/eve/src/execution/tools/workflow/workflow-tool-run.integration.test.ts@@ -234,7 +234,9 @@ describe("workflow tools", () => {// The answer resumes the same turn, which completes once.const turnIds = new Set(answered.flatMap((event) =>- "data" in event && "turnId" in event.data ? [event.data.turnId] : [],+ "data" in event && event.data !== undefined && "turnId" in event.data+ ? [event.data.turnId]+ : [],),);expect([...turnIds]).toEqual([parked!.data.turnId]);diff --git a/packages/eve/src/harness/cancelled-turn-emission.test.ts b/packages/eve/src/harness/cancelled-turn-emission.test.tsindex 9efb92bb9..936cd0ec5 100644--- a/packages/eve/src/harness/cancelled-turn-emission.test.ts+++ b/packages/eve/src/harness/cancelled-turn-emission.test.ts@@ -3,14 +3,23 @@ import { describe, expect, it } from "vitest";import { emitCancelledTurn } from "#harness/cancelled-turn-emission.js";import type { UnstampedMessageStreamEvent } from "#protocol/message.js";+const USAGE = {+ cacheReadTokens: 0,+ cacheWriteTokens: 0,+ costUsd: 0.5,+ inputTokens: 120,+ outputTokens: 30,+};+describe("emitCancelledTurn", () => {- it("emits turn.cancelled → session.waiting and no failure events", async () => {+ it("emits turn.cancelled → session.waiting with the session's usage and no failure events", async () => {const events: UnstampedMessageStreamEvent[] = [];const next = await emitCancelledTurn(async (event) => {events.push(event);},{ sessionStarted: true, sequence: 3, stepIndex: 2, turnId: "turn_3" },+ USAGE,);expect(events.map((event) => event.type)).toEqual(["turn.cancelled", "session.waiting"]);@@ -19,7 +28,7 @@ describe("emitCancelledTurn", () => {type: "turn.cancelled",});expect(events[1]).toEqual({- data: { continuationToken: "", wait: "next-user-message" },+ data: { continuationToken: "", usage: USAGE, wait: "next-user-message" },type: "session.waiting",});expect(next).toEqual({ sessionStarted: true, sequence: 4, stepIndex: 0, turnId: "" });@@ -36,6 +45,7 @@ describe("emitCancelledTurn", () => {events.push(event);},{ sessionStarted: false, sequence: 0, stepIndex: 0, turnId: "" },+ USAGE,);expect(events[0]).toMatchObject({diff --git a/packages/eve/src/harness/cancelled-turn-emission.ts b/packages/eve/src/harness/cancelled-turn-emission.tsindex 948779cb9..26e8f6b58 100644--- a/packages/eve/src/harness/cancelled-turn-emission.ts+++ b/packages/eve/src/harness/cancelled-turn-emission.ts@@ -1,5 +1,6 @@import { createSessionWaitingEvent, createTurnCancelledEvent } from "#protocol/message.js";import type { HarnessEmitFn } from "#harness/types.js";+import type { TokenUsage } from "#shared/token-usage.js";import { activeTurnId } from "#harness/active-turn-id.js";import type { HarnessEmissionState } from "#harness/emission.js";@@ -15,6 +16,7 @@ import type { HarnessEmissionState } from "#harness/emission.js";export async function emitCancelledTurn(emitFn: HarnessEmitFn,state: HarnessEmissionState,+ usage: TokenUsage,): Promise<HarnessEmissionState> {await emitFn(createTurnCancelledEvent({@@ -22,7 +24,7 @@ export async function emitCancelledTurn(turnId: activeTurnId(state),}),);- await emitFn(createSessionWaitingEvent());+ await emitFn(createSessionWaitingEvent(usage));return {sessionStarted: true,diff --git a/packages/eve/src/harness/emission.test.ts b/packages/eve/src/harness/emission.test.tsindex 54f79dfb8..5d1aad5db 100644--- a/packages/eve/src/harness/emission.test.ts+++ b/packages/eve/src/harness/emission.test.ts@@ -54,7 +54,12 @@ describe("emitTurnEpilogue", () => {const emit = createEmitStub();const history = [{ content: "settled reply", role: "assistant" as const }];- await emitTurnEpilogue(emit, EMISSION_STATE, history);+ await emitTurnEpilogue(emit, EMISSION_STATE, history, {+ cacheReadTokens: 0,+ cacheWriteTokens: 0,+ inputTokens: 0,+ outputTokens: 0,+ });expect(emit).toHaveBeenNthCalledWith(1,diff --git a/packages/eve/src/harness/emission.ts b/packages/eve/src/harness/emission.tsindex 860b8a4dd..c63bfff59 100644--- a/packages/eve/src/harness/emission.ts+++ b/packages/eve/src/harness/emission.ts@@ -35,6 +35,7 @@ import {createTurnStartedEvent,} from "#protocol/message.js";import type { JsonObject } from "#shared/json.js";+import type { TokenUsage } from "#shared/token-usage.js";import {createRuntimeToolResultFromStepResult,createRuntimeToolResultFromToolError,@@ -183,7 +184,7 @@ async function emitStepAndTurnFailed(export async function emitFailedStep(emitFn: HarnessEmitFn,state: HarnessEmissionState,- input: FailedStepPayload & { readonly sessionId: string },+ input: FailedStepPayload & { readonly sessionId: string; readonly usage: TokenUsage },): Promise<void> {await emitStepAndTurnFailed(emitFn, state, input);await emitFn(createSessionFailedEvent(input));@@ -196,10 +197,10 @@ export async function emitFailedStep(export async function emitRecoverableFailedTurn(emitFn: HarnessEmitFn,state: HarnessEmissionState,- input: FailedStepPayload & { readonly continuationToken: string },+ input: FailedStepPayload & { readonly continuationToken: string; readonly usage: TokenUsage },): Promise<HarnessEmissionState> {await emitStepAndTurnFailed(emitFn, state, input);- await emitFn(createSessionWaitingEvent());+ await emitFn(createSessionWaitingEvent(input.usage));return {sessionStarted: state.sessionStarted,@@ -220,13 +221,14 @@ export function advanceStep(state: HarnessEmissionState): HarnessEmissionState {}/**- * Emits `turn.completed` and `session.waiting`.+ * Emits `turn.completed` and `session.waiting` with the session's `usage`.* Returns updated emission state with an incremented sequence.*/export async function emitTurnEpilogue(emitFn: HarnessEmitFn,state: HarnessEmissionState,messages: readonly ModelMessage[],+ usage: TokenUsage,): Promise<HarnessEmissionState> {await emitFn(createTurnCompletedEvent({@@ -235,7 +237,7 @@ export async function emitTurnEpilogue(}),messages,);- await emitFn(createSessionWaitingEvent());+ await emitFn(createSessionWaitingEvent(usage));return {sessionStarted: state.sessionStarted,diff --git a/packages/eve/src/harness/issue-3494-diagnosis.integration.test.ts b/packages/eve/src/harness/issue-3494-diagnosis.integration.test.tsindex 43d98ba7f..09ec9e76e 100644--- a/packages/eve/src/harness/issue-3494-diagnosis.integration.test.ts+++ b/packages/eve/src/harness/issue-3494-diagnosis.integration.test.ts@@ -142,7 +142,11 @@ it("continues the second step past an older approval batch", async () => {emittedBySecondStep: events.slice(before).map((event) => event.type),firstTurnEvents: events.filter(- (event) => "data" in event && "turnId" in event.data && event.data.turnId === "turn_1",+ (event) =>+ "data" in event &&+ event.data !== undefined &&+ "turnId" in event.data &&+ event.data.turnId === "turn_1",).map((event) => event.type),};diff --git a/packages/eve/src/harness/session-limit-enforcement.ts b/packages/eve/src/harness/session-limit-enforcement.tsindex 88aac4b96..83a458fc9 100644--- a/packages/eve/src/harness/session-limit-enforcement.ts+++ b/packages/eve/src/harness/session-limit-enforcement.ts@@ -26,6 +26,7 @@ import {bumpSessionRuntimeUsageLimits,getSessionUsageLimitViolation,getSessionTokenUsage,+ getSessionUsage,type SessionUsageLimitViolation,} from "#harness/turn-tag-state.js";import type { HarnessSession, StepResult, ToolLoopHarnessConfig } from "#harness/types.js";@@ -143,7 +144,12 @@ async function parkOnSessionUsageLimit(input: {}),);- emissionState = await emitTurnEpilogue(input.emit, emissionState, parkedSession.history);+ emissionState = await emitTurnEpilogue(+ input.emit,+ emissionState,+ parkedSession.history,+ getSessionUsage(parkedSession),+ );return {next: null,@@ -195,6 +201,7 @@ async function failSessionUsageLimit(input: {details,message,sessionId: input.session.sessionId,+ usage: getSessionUsage(input.session),});}diff --git a/packages/eve/src/harness/tool-loop.test.ts b/packages/eve/src/harness/tool-loop.test.tsindex 88da0f7e0..55b5992ce 100644--- a/packages/eve/src/harness/tool-loop.test.ts+++ b/packages/eve/src/harness/tool-loop.test.ts@@ -75,6 +75,7 @@ import { PendingSkillAnnouncementKey } from "#context/dynamic-skill-lifecycle.jsimport { deserializeContext, serializeContext } from "#context/serialize.js";import { stashToolInterrupt } from "#harness/tool-interrupts.js";import { appendMissingToolResultMessages, createToolLoopHarness } from "#harness/tool-loop.js";+import { countRunUsage } from "#execution/agent-sessions/usage.js";import { createTask, writeTaskTable } from "#execution/tasks/table.js";import { SessionLimitDeclinedError, TurnCancelledError } from "#harness/turn-cancellation.js";import {@@ -420,6 +421,28 @@ function setDelegatedParent(ctx: ContextContainer): void {});}+const DELEGATED_SPEND = {+ cacheReadTokens: 0,+ cacheWriteTokens: 0,+ costUsd: 0.25,+ inputTokens: 100,+ outputTokens: 20,+};++/** A session whose delegated agents already spent {@link DELEGATED_SPEND}. */+function withDelegatedSpend(overrides?: Parameters<typeof createTestSession>[0]): HarnessSession {+ return countRunUsage(createTestSession(overrides), DELEGATED_SPEND);+}++function lastSessionEvent<T extends "session.waiting" | "session.failed">(+ events: readonly UnstampedMessageStreamEvent[],+ type: T,+): Extract<UnstampedMessageStreamEvent, { type: T }> | undefined {+ return events.findLast(+ (event): event is Extract<UnstampedMessageStreamEvent, { type: T }> => event.type === type,+ );+}+function createEventCollector(): {emit: HarnessEmitFn;events: UnstampedMessageStreamEvent[];@@ -1712,6 +1735,81 @@ describe("createToolLoopHarness", () => {});});+ it("reports the session's running usage, delegated spend included, when the turn ends", async () => {+ setupMockAgent({+ finishReason: "stop",+ providerMetadata: { gateway: { cost: "0.125" } },+ response: { messages: [{ content: "Hello!", role: "assistant" }] },+ text: "Hello!",+ toolCalls: [],+ toolResults: [],+ usage: {+ inputTokenDetails: { cacheReadTokens: 2, cacheWriteTokens: 1 },+ inputTokens: 7,+ outputTokens: 3,+ },+ });+ const { emit, events } = createEventCollector();++ await createToolLoopHarness(createTestConfig(emit))(withDelegatedSpend(), { message: "Hi" });++ expect(lastSessionEvent(events, "session.waiting")?.data.usage).toEqual({+ cacheReadTokens: 2,+ cacheWriteTokens: 1,+ costUsd: 0.375,+ inputTokens: 107,+ outputTokens: 23,+ });+ });++ it("reports no session cost on session.waiting when no model call reported one", async () => {+ setupMockAgent({+ finishReason: "stop",+ response: { messages: [{ content: "Hello!", role: "assistant" }] },+ text: "Hello!",+ toolCalls: [],+ toolResults: [],+ usage: { inputTokens: 7, outputTokens: 3 },+ });+ const { emit, events } = createEventCollector();++ await createToolLoopHarness(createTestConfig(emit))(createTestSession(), { message: "Hi" });++ expect(lastSessionEvent(events, "session.waiting")?.data.usage).toEqual({+ cacheReadTokens: 0,+ cacheWriteTokens: 0,+ costUsd: undefined,+ inputTokens: 7,+ outputTokens: 3,+ });+ });++ it("reports the session's usage on session.waiting when a failed model call parks the turn", async () => {+ setupMockAgentError(new Error("Model blew up"));+ const { emit, events } = createEventCollector();++ await createToolLoopHarness(createTestConfig(emit))(withDelegatedSpend(), { message: "Hi" });++ expect(lastSessionEvent(events, "session.waiting")?.data.usage).toEqual(DELEGATED_SPEND);+ });++ it("reports the session's usage on session.failed when model selection fails", async () => {+ const emit: HarnessEmitFn = async (event) => {+ events.push(event);+ if (event.type === "turn.started") {+ throw new DynamicModelSelectionError(new Error("flag service unavailable"));+ }+ };+ const events: UnstampedMessageStreamEvent[] = [];++ await createToolLoopHarness(createTestConfig(emit))(+ withDelegatedSpend({ outputSchema: { type: "object" } }),+ { message: "Hi" },+ );++ expect(lastSessionEvent(events, "session.failed")?.data.usage).toEqual(DELEGATED_SPEND);+ });+it.each([{details: {@@ -4674,7 +4772,7 @@ describe("createToolLoopHarness", () => {);});- it("parks a delegated turn held by its working tasks with turn.waiting", async () => {+ it("parks a delegated turn held by its working tasks with turn.waiting and the session's usage", async () => {setupMockAgent({finishReason: "stop",response: { messages: [{ content: "Checking now.", role: "assistant" }] },@@ -4692,7 +4790,7 @@ describe("createToolLoopHarness", () => {);const result = await contextStorage.run(ctx, () =>- runStep(writeTaskTable(createTestSession(), table), { message: "Delegated turn" }),+ runStep(writeTaskTable(withDelegatedSpend(), table), { message: "Delegated turn" }),);expect(result.held).toBeDefined();@@ -4703,7 +4801,7 @@ describe("createToolLoopHarness", () => {}),);expect(events.at(-1)).toEqual({- data: { sequence: 0, turnId: "turn_0" },+ data: { sequence: 0, turnId: "turn_0", usage: DELEGATED_SPEND },type: "turn.waiting",});expect(events.map((event) => event.type)).not.toContain("session.waiting");diff --git a/packages/eve/src/harness/tool-loop.ts b/packages/eve/src/harness/tool-loop.tsindex 0b64bfb52..98a38d382 100644--- a/packages/eve/src/harness/tool-loop.ts+++ b/packages/eve/src/harness/tool-loop.ts@@ -87,10 +87,12 @@ import { collectDeferredCalls, dispatchApprovedWorkflowCalls } from "#harness/woimport { estimateTokens } from "#harness/token-estimate.js";import {accumulateTurnUsage,+ getSessionUsage,getTurnUsageState,setTurnUsageState,type TokenUsageDelta,} from "#harness/turn-tag-state.js";+import type { TokenUsage } from "#shared/token-usage.js";import {applySessionLimitContinuation,enforceSessionUsageLimit,@@ -554,6 +556,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {details: { errorId },message,sessionId: session.sessionId,+ usage: getSessionUsage(session),});return {@@ -594,7 +597,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {turnId: activeTurnId(emissionState),}),);- await emit?.(createSessionWaitingEvent());+ await emit?.(createSessionWaitingEvent(getSessionUsage(session)));return { next: null, session };}@@ -632,7 +635,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {}}- await emit?.(createSessionWaitingEvent());+ await emit?.(createSessionWaitingEvent(getSessionUsage(session)));return { next: null, session };}@@ -934,7 +937,12 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {]),state: memoryCommit?.state ?? parkedSession.state,};- emissionState = await emitTurnEpilogue(emit, emissionState, parkedSession.history);+ emissionState = await emitTurnEpilogue(+ emit,+ emissionState,+ parkedSession.history,+ getSessionUsage(parkedSession),+ );return {next: null,session: setHarnessEmissionState(parkedSession, emissionState),@@ -943,7 +951,12 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {if (resolvedCoordination.outcome === "resolved") {if (emit) {- emissionState = await emitTurnEpilogue(emit, emissionState, parkedSession.history);+ emissionState = await emitTurnEpilogue(+ emit,+ emissionState,+ parkedSession.history,+ getSessionUsage(parkedSession),+ );parkedSession = setHarnessEmissionState(parkedSession, emissionState);}return { next: null, session: parkedSession };@@ -954,7 +967,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {isHarnessBetweenTurns(pending.session) &&getPendingAuthorization(pending.session.state) === undefined) {- await emit?.(createSessionWaitingEvent());+ await emit?.(createSessionWaitingEvent(getSessionUsage(pending.session)));}return { next: null, session: pending.session };}@@ -1858,6 +1871,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {emissionState = await emitRecoverableFailedTurn(emit, emissionState, {code: "WORKFLOW_STREAM_WRITE_FAILED",continuationToken: session.continuationToken,+ usage: getSessionUsage(session),details: { ...streamWriteDetails, errorId },message: toErrorMessage(finalError),});@@ -1918,6 +1932,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {details,message: errorMessage,sessionId: session.sessionId,+ usage: getSessionUsage(session),});// Delegated runs need a failed terminal result so their caller sees a// failed `subagent-result`; top-level conversation failures already@@ -1937,6 +1952,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {emissionState = await emitRecoverableFailedTurn(emit, emissionState, {code: "MODEL_CALL_FAILED",continuationToken: session.continuationToken,+ usage: getSessionUsage(session),details,message: errorMessage,});@@ -2744,7 +2760,12 @@ async function handleStepResult(input: {}),);- emissionState = await emitTurnEpilogue(emit, emissionState, parkedSession.history);+ emissionState = await emitTurnEpilogue(+ emit,+ emissionState,+ parkedSession.history,+ getSessionUsage(parkedSession),+ );parkedSession = setHarnessEmissionState(parkedSession, emissionState);}@@ -2796,7 +2817,12 @@ async function handleStepResult(input: {// above: the session keeps serving ordinary turns while the challenge// is open, so the stream must close its turn boundary — clients wait// on `session.waiting` and would otherwise hang on the parked turn.- emissionState = await emitTurnEpilogue(emit, emissionState, authorizationHistory);+ emissionState = await emitTurnEpilogue(+ emit,+ emissionState,+ authorizationHistory,+ getSessionUsage(baseSession),+ );}return {@@ -2862,6 +2888,7 @@ async function handleStepResult(input: {createTurnWaitingEvent({sequence: emissionState.sequence,turnId: emissionState.turnId,+ usage: getSessionUsage(nextSession),}),);}@@ -3006,6 +3033,7 @@ async function emitStructuredResult(emissionState: ReturnType<typeof getHarnessEmissionState>,structured: JsonValue,history: readonly HarnessModelMessage[],+ usage: TokenUsage,): Promise<ReturnType<typeof getHarnessEmissionState>> {await emit(createResultCompletedEvent({@@ -3015,7 +3043,7 @@ async function emitStructuredResult(turnId: emissionState.turnId,}),);- return emitTurnEpilogue(emit, emissionState, history);+ return emitTurnEpilogue(emit, emissionState, history, usage);}/**@@ -3038,7 +3066,12 @@ async function finishTurn(input: {if (schema === undefined) {if (emit) {- emissionState = await emitTurnEpilogue(emit, emissionState, session.history);+ emissionState = await emitTurnEpilogue(+ emit,+ emissionState,+ session.history,+ getSessionUsage(session),+ );session = setHarnessEmissionState(session, emissionState);}const settledTurn = { output: stepOutput ?? "" } satisfies SettledTurn;@@ -3054,6 +3087,7 @@ async function finishTurn(input: {emissionState = await emitRecoverableFailedTurn(emit, emissionState, {...OUTPUT_SCHEMA_NOT_FULFILLED,continuationToken: session.continuationToken,+ usage: getSessionUsage(session),});session = setHarnessEmissionState(session, emissionState);}@@ -3066,7 +3100,13 @@ async function finishTurn(input: {session = persistStructuredAssistantTurn(session, history, structured);if (emit) {- emissionState = await emitStructuredResult(emit, emissionState, structured, session.history);+ emissionState = await emitStructuredResult(+ emit,+ emissionState,+ structured,+ session.history,+ getSessionUsage(session),+ );session = setHarnessEmissionState(session, emissionState);}const settledTurn = { output: structured } satisfies SettledTurn;diff --git a/packages/eve/src/harness/turn-tag-state.ts b/packages/eve/src/harness/turn-tag-state.tsindex 0002738f5..8444c228e 100644--- a/packages/eve/src/harness/turn-tag-state.ts+++ b/packages/eve/src/harness/turn-tag-state.ts@@ -78,6 +78,11 @@ export function getSessionTokenUsage(session: Pick<HarnessSession, "state">): Toreturn getTurnUsageState(session.state)?.session ?? ZERO_TOKEN_USAGE;}+/** The session's usage so far, delegated spend included. */+export function getSessionUsage(session: Pick<HarnessSession, "state">): TokenUsage {+ return toUsage(getSessionTokenUsage(session));+}+/** Projects a {@link TokenUsageTotals} down to the cross-cutting {@link TokenUsage} shape. */export function toUsage(totals: TokenUsageTotals): TokenUsage {return {diff --git a/packages/eve/src/instrumentation/native-events.test.ts b/packages/eve/src/instrumentation/native-events.test.tsindex 81b16b7b4..c72373cd4 100644--- a/packages/eve/src/instrumentation/native-events.test.ts+++ b/packages/eve/src/instrumentation/native-events.test.ts@@ -1,3 +1,4 @@+import { TEST_USAGE } from "#internal/testing/events.js";import { describe, expect, it } from "vitest";import { ContextContainer, contextStorage } from "#context/container.js";@@ -119,7 +120,7 @@ describe("createInstrumentationHandleEvent", () => {}),);await handleEvent(createTurnCompletedEvent({ sequence: 0, turnId: "turn-1" }));- await handleEvent(createSessionWaitingEvent());+ await handleEvent(createSessionWaitingEvent(TEST_USAGE));expect(order).toEqual(["durable:session.started",@@ -177,7 +178,7 @@ describe("createInstrumentationHandleEvent", () => {turnId: "turn-1",})!;- await handleEvent(createSessionWaitingEvent());+ await handleEvent(createSessionWaitingEvent(TEST_USAGE));expect(events).toEqual([{diff --git a/packages/eve/src/internal/testing/events.ts b/packages/eve/src/internal/testing/events.tsindex 5314c916d..31ae2b272 100644--- a/packages/eve/src/internal/testing/events.ts+++ b/packages/eve/src/internal/testing/events.ts@@ -216,3 +216,12 @@ export function stampTestEvents(): MessageStreamEvent[] {return events.map((event, index) => stampTestEvent(event, index));}++/** Token usage a session reports in tests that build session events without caring about its value. */+export const TEST_USAGE = {+ cacheReadTokens: 800,+ cacheWriteTokens: 0,+ costUsd: 0.0042,+ inputTokens: 1200,+ outputTokens: 150,+};diff --git a/packages/eve/src/protocol/message.test.ts b/packages/eve/src/protocol/message.test.tsindex 00d01ae79..e8f6a0eba 100644--- a/packages/eve/src/protocol/message.test.ts+++ b/packages/eve/src/protocol/message.test.ts@@ -15,7 +15,6 @@ import {createMessageReceivedEvent,createReasoningAppendedEvent,createResultCompletedEvent,- createSessionWaitingEvent,createStepStartedEvent,createAgentStartedEvent,createTurnCancelledEvent,@@ -301,16 +300,6 @@ describe("message stream protocol", () => {});});- it("publishes the channel-local continuation token on session.waiting", () => {- expect(createSessionWaitingEvent("slack:C1:T1")).toEqual({- data: {- continuationToken: "C1:T1",- wait: "next-user-message",- },- type: "session.waiting",- });- });-it("creates turn.cancelled events", () => {expect(createTurnCancelledEvent({ sequence: 2, turnId: "turn_2" })).toEqual({data: { sequence: 2, turnId: "turn_2" },diff --git a/packages/eve/src/protocol/message.ts b/packages/eve/src/protocol/message.tsindex 2895611f9..53e1f39e0 100644--- a/packages/eve/src/protocol/message.ts+++ b/packages/eve/src/protocol/message.ts@@ -18,8 +18,8 @@ import type {RuntimeToolResultActionResult,} from "#shared/action-types.js";import type { InputRequest, InputResponse } from "#shared/input.js";-import { toChannelLocalContinuationToken } from "#shared/continuation-token.js";import type { JsonObject, JsonValue } from "#shared/json.js";+import type { TokenUsage } from "#shared/token-usage.js";export const EVE_SESSION_ID_HEADER = "x-eve-session-id";export const EVE_STREAM_FORMAT_HEADER = "x-eve-stream-format";@@ -619,6 +619,12 @@ export interface TurnWaitingStreamEvent {data: {sequence: number;turnId: string;+ /**+ * The session's token usage so far: its own model calls plus what the+ * agents it delegated to spent. `costUsd` is absent when no model call+ * reported a cost. Absent on events from eve versions before it was added.+ */+ usage?: TokenUsage;};type: "turn.waiting";}@@ -774,6 +780,12 @@ export interface SessionWaitingStreamEvent {data: {/** Channel-local continuation token, or the immutable session ID for an ID-only session. */continuationToken: string;+ /**+ * The session's token usage so far: its own model calls plus what the+ * agents it delegated to spent. `costUsd` is absent when no model call+ * reported a cost. Absent on events from eve versions before it was added.+ */+ usage?: TokenUsage;wait: "next-user-message";};type: "session.waiting";@@ -788,6 +800,12 @@ export interface SessionFailedStreamEvent {details?: JsonObject;message: string;sessionId: string;+ /**+ * The session's token usage so far: its own model calls plus what the+ * agents it delegated to spent. `costUsd` is absent when no model call+ * reported a cost. Absent on events from eve versions before it was added.+ */+ usage?: TokenUsage;};type: "session.failed";}@@ -796,6 +814,14 @@ export interface SessionFailedStreamEvent {* Stream event emitted when the session completes successfully.*/export interface SessionCompletedStreamEvent {+ /** Absent when the session's usage was unknown, and on events from eve versions before it was added. */+ data?: {+ /**+ * The session's token usage: its own model calls plus what the agents it+ * delegated to spent. `costUsd` is absent when no model call reported a cost.+ */+ usage: TokenUsage;+ };type: "session.completed";}@@ -1683,11 +1709,13 @@ export function createTurnCompletedEvent(input: {export function createTurnWaitingEvent(input: {readonly sequence: number;readonly turnId: string;+ readonly usage: TokenUsage | undefined;}): TurnWaitingStreamEvent {return {data: {sequence: input.sequence,turnId: input.turnId,+ ...(input.usage !== undefined && { usage: input.usage }),},type: "turn.waiting",};@@ -1796,11 +1824,12 @@ export function createCompactionCompletedEvent(input: {* wait.*/export function createSessionWaitingEvent(- namespacedContinuationToken: string = "",+ usage: TokenUsage | undefined,): SessionWaitingStreamEvent {return {data: {- continuationToken: toChannelLocalContinuationToken(namespacedContinuationToken),+ continuationToken: "",+ ...(usage !== undefined && { usage }),wait: "next-user-message",},type: "session.waiting",@@ -1815,6 +1844,7 @@ export function createSessionFailedEvent(input: {readonly details?: JsonObject;readonly message: string;readonly sessionId: string;+ readonly usage: TokenUsage | undefined;}): SessionFailedStreamEvent {return {data: {@@ -1822,6 +1852,7 @@ export function createSessionFailedEvent(input: {details: input.details,message: input.message,sessionId: input.sessionId,+ ...(input.usage !== undefined && { usage: input.usage }),},type: "session.failed",};@@ -1830,8 +1861,12 @@ export function createSessionFailedEvent(input: {/*** Creates the `session.completed` event for one terminal session completion.*/-export function createSessionCompletedEvent(): SessionCompletedStreamEvent {- return { type: "session.completed" };+export function createSessionCompletedEvent(+ usage: TokenUsage | undefined,+): SessionCompletedStreamEvent {+ return usage === undefined+ ? { type: "session.completed" }+ : { data: { usage }, type: "session.completed" };}/**diff --git a/packages/eve/src/react/use-eve-agent.test.ts b/packages/eve/src/react/use-eve-agent.test.tsindex 75d2f7b93..c47e8fa69 100644--- a/packages/eve/src/react/use-eve-agent.test.ts+++ b/packages/eve/src/react/use-eve-agent.test.ts@@ -19,7 +19,7 @@ import {createTurnFailedEvent,type UnstampedMessageStreamEvent,} from "#protocol/message.js";-import { stampTestEvents } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvents } from "#internal/testing/events.js";import type { ClientSessionState } from "#client/types.js";function createStartedMessageResponse(sessionId: string, continuationToken: string): Response {@@ -195,7 +195,7 @@ describe("useEveAgent", () => {const cancel = vi.fn();const body = new ReadableStream<Uint8Array>({start(controller) {- const [waiting] = stampTestEvents([createSessionWaitingEvent()]);+ const [waiting] = stampTestEvents([createSessionWaitingEvent(TEST_USAGE)]);controller.enqueue(new TextEncoder().encode(`${JSON.stringify(waiting)}\n`));},cancel,@@ -242,7 +242,7 @@ describe("useEveAgent", () => {let creates = 0;const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) => {if (init?.method === "POST") return createStartedMessageResponse(`session_${++creates}`, "");- return createEagerStreamResponse([createSessionWaitingEvent()]);+ return createEagerStreamResponse([createSessionWaitingEvent(TEST_USAGE)]);});let agent: UseEveAgentHelpers<EveMessageData> | undefined;function Chat({@@ -293,7 +293,7 @@ describe("useEveAgent", () => {const fetchMock = vi.spyOn(globalThis, "fetch").mockReturnValueOnce(accepted.promise)- .mockResolvedValueOnce(createEagerStreamResponse([createSessionWaitingEvent()]));+ .mockResolvedValueOnce(createEagerStreamResponse([createSessionWaitingEvent(TEST_USAGE)]));let agent: UseEveAgentHelpers<EveMessageData> | undefined;function Chat({ prewarm }: { readonly prewarm: boolean }) {agent = useEveAgent({ prewarm });@@ -356,7 +356,7 @@ describe("useEveAgent", () => {let creates = 0;const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) => {if (init?.method === "POST") return createStartedMessageResponse(`session_${++creates}`, "");- return createEagerStreamResponse([createSessionWaitingEvent()]);+ return createEagerStreamResponse([createSessionWaitingEvent(TEST_USAGE)]);});let agent: UseEveAgentHelpers<EveMessageData> | undefined;let setPrewarm: ((value: boolean) => void) | undefined;@@ -473,7 +473,7 @@ describe("useEveAgent", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const startResponse = createDeferred<Response>();@@ -596,7 +596,7 @@ describe("useEveAgent", () => {.mockImplementation(async (_request, init) =>init?.method === "POST"? await startResponse.promise- : createEagerStreamResponse([createSessionWaitingEvent()]),+ : createEagerStreamResponse([createSessionWaitingEvent(TEST_USAGE)]),);let randomWord = "jazz";@@ -653,7 +653,7 @@ describe("useEveAgent", () => {.mockImplementation(async (_request, init) =>init?.method === "POST"? await startResponse.promise- : createEagerStreamResponse([createSessionWaitingEvent()]),+ : createEagerStreamResponse([createSessionWaitingEvent(TEST_USAGE)]),);let helpers: UseEveAgentHelpers<EveMessageData> | undefined;@@ -732,7 +732,7 @@ describe("useEveAgent", () => {stepIndex: 0,turnId: "turn_2",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];vi.spyOn(globalThis, "fetch")@@ -812,6 +812,7 @@ describe("useEveAgent", () => {turnId: "turn_1",}),createSessionFailedEvent({+ usage: TEST_USAGE,code: "MODEL_CALL_FAILED",message: "Bad Request",sessionId: "session_1",@@ -884,7 +885,7 @@ describe("useEveAgent", () => {sequence: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const startResponse = createDeferred<Response>();@@ -948,7 +949,7 @@ describe("useEveAgent", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);const fetchMock = vi.spyOn(globalThis, "fetch")@@ -994,7 +995,7 @@ describe("useEveAgent", () => {init?.method === "POST"? await startResponse.promise: await startResponse.promise.then(() =>- createEagerStreamResponse([createSessionWaitingEvent()]),+ createEagerStreamResponse([createSessionWaitingEvent(TEST_USAGE)]),),);diff --git a/packages/eve/src/shared/token-usage.ts b/packages/eve/src/shared/token-usage.tsindex 0b5a32e70..28e61d0f1 100644--- a/packages/eve/src/shared/token-usage.ts+++ b/packages/eve/src/shared/token-usage.ts@@ -14,3 +14,17 @@ const tokenUsageSchema = z.object({export const tokenUsageWithCostSchema = tokenUsageSchema.extend({costUsd: z.number().finite().nonnegative().optional(),});++/** Adds two usages; the cost is absent only when neither reported one. */+export function addTokenUsage(a: TokenUsage, b: TokenUsage): TokenUsage {+ return {+ cacheReadTokens: a.cacheReadTokens + b.cacheReadTokens,+ cacheWriteTokens: a.cacheWriteTokens + b.cacheWriteTokens,+ costUsd:+ a.costUsd === undefined && b.costUsd === undefined+ ? undefined+ : (a.costUsd ?? 0) + (b.costUsd ?? 0),+ inputTokens: a.inputTokens + b.inputTokens,+ outputTokens: a.outputTokens + b.outputTokens,+ };+}diff --git a/packages/eve/src/subagents/hitl-proxy.integration.test.ts b/packages/eve/src/subagents/hitl-proxy.integration.test.tsindex a2118de9e..076a83296 100644--- a/packages/eve/src/subagents/hitl-proxy.integration.test.ts+++ b/packages/eve/src/subagents/hitl-proxy.integration.test.ts@@ -267,7 +267,14 @@ describe("subagent HITL proxy → Slack-style text-approve regression (Finding #// `turn.waiting`; the call that asked is still running, so the turn// neither completes nor resets.expect(events.slice(1)).toEqual([- { data: { sequence: 3, turnId: "turn_3" }, type: "turn.waiting" },+ {+ data: {+ sequence: 3,+ turnId: "turn_3",+ usage: { cacheReadTokens: 0, cacheWriteTokens: 0, inputTokens: 0, outputTokens: 0 },+ },+ type: "turn.waiting",+ },]);expect(events[0]?.type).toBe("input.requested");diff --git a/packages/eve/src/subagents/hitl-proxy.ts b/packages/eve/src/subagents/hitl-proxy.tsindex c46895f41..69c5e9036 100644--- a/packages/eve/src/subagents/hitl-proxy.ts+++ b/packages/eve/src/subagents/hitl-proxy.ts@@ -1,4 +1,5 @@import type { SessionInboxAddress } from "#execution/session-inbox/address.js";+import { getSessionUsage } from "#harness/turn-tag-state.js";import type {DeliverPayload,SubagentAuthorizationEventHookPayload,@@ -69,7 +70,13 @@ export async function emitProxiedAuthorizationEvent(input: {async function emitTurnWaiting(emit: HarnessEmitFn, session: HarnessSessionBase): Promise<void> {const turn = getHarnessEmissionState(session.state);- await emit(createTurnWaitingEvent({ sequence: turn.sequence, turnId: turn.turnId }));+ await emit(+ createTurnWaitingEvent({+ sequence: turn.sequence,+ turnId: turn.turnId,+ usage: getSessionUsage(session),+ }),+ );}// ---------------------------------------------------------------------------diff --git a/packages/eve/src/svelte/use-eve-agent.test.ts b/packages/eve/src/svelte/use-eve-agent.test.tsindex b41b0fd6a..1cd8eac52 100644--- a/packages/eve/src/svelte/use-eve-agent.test.ts+++ b/packages/eve/src/svelte/use-eve-agent.test.ts@@ -12,7 +12,7 @@ import {createSessionWaitingEvent,type UnstampedMessageStreamEvent,} from "#protocol/message.js";-import { stampTestEvents } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvents } from "#internal/testing/events.js";function createStartedMessageResponse(sessionId: string, continuationToken: string): Response {return new Response(JSON.stringify({ continuationToken, ok: true, sessionId }), {@@ -109,7 +109,7 @@ describe("useEveAgent (Svelte rune binding)", () => {vi.stubGlobal("window", {});const events = [createMessageReceivedEvent({ message: "Hello", sequence: 0, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const fetchMock = vi.spyOn(globalThis, "fetch")@@ -133,7 +133,7 @@ describe("useEveAgent (Svelte rune binding)", () => {vi.stubGlobal("window", {});const events = [createMessageReceivedEvent({ message: "Hello", sequence: 0, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];vi.spyOn(globalThis, "fetch").mockResolvedValueOnce(createStartedMessageResponse("session_1", "http:session_1"))diff --git a/packages/eve/src/vue/use-eve-agent.test.ts b/packages/eve/src/vue/use-eve-agent.test.tsindex 070357c52..fb1487cc9 100644--- a/packages/eve/src/vue/use-eve-agent.test.ts+++ b/packages/eve/src/vue/use-eve-agent.test.ts@@ -23,7 +23,7 @@ import {createSessionFailedEvent,type UnstampedMessageStreamEvent,} from "#protocol/message.js";-import { stampTestEvents } from "#internal/testing/events.js";+import { TEST_USAGE, stampTestEvents } from "#internal/testing/events.js";function createStartedMessageResponse(sessionId: string, continuationToken: string): Response {return new Response(JSON.stringify({ continuationToken, ok: true, sessionId }), {@@ -175,7 +175,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const startResponse = createDeferred<Response>();@@ -257,6 +257,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {turnId: "turn_1",}),createSessionFailedEvent({+ usage: TEST_USAGE,code: "MODEL_CALL_FAILED",message: "Bad Request",sessionId: "session_1",@@ -328,7 +329,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) =>init?.method === "POST"? await startResponse.promise- : createEagerStreamResponse([createSessionWaitingEvent()]),+ : createEagerStreamResponse([createSessionWaitingEvent(TEST_USAGE)]),);const store = createStore<readonly string[]>({@@ -373,7 +374,7 @@ describe("useEveAgent (Vue composable wiring)", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];vi.spyOn(globalThis, "fetch").mockResolvedValueOnce(createBoundedStreamResponse(events))@@ -411,7 +412,7 @@ describe("useEveAgent (Vue composable wiring)", () => {stepIndex: 0,turnId: "turn_1",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const startResponse = createDeferred<Response>();@@ -452,7 +453,7 @@ describe("useEveAgent (Vue composable wiring)", () => {.mockResolvedValueOnce(createEagerStreamResponse([createMessageReceivedEvent({ message: "After", sequence: 0, turnId: "turn_1" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]),);diff --git a/packages/eve/test/client.test.ts b/packages/eve/test/client.test.tsindex e8e4b00fd..b147b1477 100644--- a/packages/eve/test/client.test.ts+++ b/packages/eve/test/client.test.ts@@ -1,3 +1,4 @@+import { TEST_USAGE } from "#internal/testing/events.js";import { afterEach, describe, expect, expectTypeOf, it, vi } from "vitest";import {@@ -88,7 +89,7 @@ function createEagerStreamResponse(): Response {let turnId = "turn_001";for (const event of events) {- if ("data" in event && "turnId" in event.data) {+ if ("data" in event && event.data !== undefined && "turnId" in event.data) {turnId = event.data.turnId;break;}@@ -134,7 +135,7 @@ function singleTurnEvents(input: {turnId: input.turnId,}),createTurnCompletedEvent({ sequence: input.sequence, turnId: input.turnId }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];}@@ -312,6 +313,7 @@ describe("Session.send (result)", () => {const events: UnstampedMessageStreamEvent[] = [createTurnStartedEvent({ sequence: 1, turnId: "turn_001" }),createSessionFailedEvent({+ usage: TEST_USAGE,code: "internal_error",message: "Something went wrong",sessionId: "session_001",@@ -340,7 +342,7 @@ describe("Session.send (result)", () => {turnId: "turn_001",}),createTurnCompletedEvent({ sequence: 1, turnId: "turn_001" }),- createSessionCompletedEvent(),+ createSessionCompletedEvent(TEST_USAGE),];vi.spyOn(globalThis, "fetch")@@ -380,7 +382,7 @@ describe("Session.send (result)", () => {turnId: "turn_001",}),createTurnCompletedEvent({ sequence: 1, turnId: "turn_001" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const fetchMock = vi@@ -409,7 +411,7 @@ describe("Session.send (result)", () => {stepIndex: 0,turnId: "turn_001",}),- createSessionCompletedEvent(),+ createSessionCompletedEvent(TEST_USAGE),];const secondEvents = singleTurnEvents({message: "New conversation",@@ -476,7 +478,7 @@ describe("Session.send (stream)", () => {turnId: "turn_001",}),);- stream.pushEvent(createSessionWaitingEvent());+ stream.pushEvent(createSessionWaitingEvent(TEST_USAGE));}, 0);await iterationPromise;@@ -500,7 +502,7 @@ describe("Session.send (stream)", () => {expect(res.sessionId).toBe("session_001");setTimeout(() => {- stream.pushEvent(createSessionWaitingEvent());+ stream.pushEvent(createSessionWaitingEvent(TEST_USAGE));}, 0);for await (const _ of res) {@@ -544,7 +546,7 @@ describe("Session.send (reconnection)", () => {turnId: "turn_001",}),createTurnCompletedEvent({ sequence: 1, turnId: "turn_001" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const fetchMock = vi@@ -714,7 +716,7 @@ describe("Session.stream", () => {stepIndex: 0,turnId: "turn_002",}),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),];const fetchMock = vidiff --git a/packages/eve/test/dev-send-message.test.ts b/packages/eve/test/dev-send-message.test.tsindex e9da7d1ef..f13a5640d 100644--- a/packages/eve/test/dev-send-message.test.ts+++ b/packages/eve/test/dev-send-message.test.ts@@ -1,3 +1,4 @@+import { TEST_USAGE } from "#internal/testing/events.js";import { getVercelOidcToken } from "#compiled/@vercel/oidc/index.js";import { afterEach, describe, expect, it, vi } from "vitest";import {@@ -136,7 +137,7 @@ describe("sendDevelopmentMessage", () => {}),);firstStream.pushEvent(createTurnCompletedEvent({ sequence: 1, turnId: "turn_001" }));- firstStream.pushEvent(createSessionWaitingEvent());+ firstStream.pushEvent(createSessionWaitingEvent(TEST_USAGE));}, 0);const first = await firstPromise;@@ -169,7 +170,7 @@ describe("sendDevelopmentMessage", () => {}),);secondStream.pushEvent(createTurnCompletedEvent({ sequence: 2, turnId: "turn_002" }));- secondStream.pushEvent(createSessionWaitingEvent());+ secondStream.pushEvent(createSessionWaitingEvent(TEST_USAGE));}, 0);const second = await secondPromise;@@ -221,7 +222,7 @@ describe("sendDevelopmentMessage", () => {type: "turn.completed",},{- data: { continuationToken: "", wait: "next-user-message" },+ data: { continuationToken: "", usage: TEST_USAGE, wait: "next-user-message" },type: "session.waiting",},]);@@ -281,7 +282,7 @@ describe("sendDevelopmentMessage", () => {turnId: "turn_002",}),createTurnCompletedEvent({ sequence: 2, turnId: "turn_002" }),- createSessionWaitingEvent(),+ createSessionWaitingEvent(TEST_USAGE),]);const fetchMock = vi.spyOn(globalThis, "fetch")@@ -322,7 +323,7 @@ describe("sendDevelopmentMessage", () => {}),);firstStream.pushEvent(createTurnCompletedEvent({ sequence: 1, turnId: "turn_001" }));- firstStream.pushEvent(createSessionWaitingEvent());+ firstStream.pushEvent(createSessionWaitingEvent(TEST_USAGE));}, 0);const first = await firstPromise;@@ -416,7 +417,7 @@ describe("sendDevelopmentMessage", () => {type: "turn.completed",},{- data: { continuationToken: "", wait: "next-user-message" },+ data: { continuationToken: "", usage: TEST_USAGE, wait: "next-user-message" },type: "session.waiting",},]);@@ -474,7 +475,7 @@ describe("sendDevelopmentMessage", () => {}),);stream.pushEvent(createTurnCompletedEvent({ sequence: 1, turnId: "turn_001" }));- stream.pushEvent(createSessionWaitingEvent());+ stream.pushEvent(createSessionWaitingEvent(TEST_USAGE));}, 0);const result = await messagePromise;@@ -533,7 +534,7 @@ describe("sendDevelopmentMessage", () => {}),);stream.pushEvent(createTurnCompletedEvent({ sequence: 1, turnId: "turn_001" }));- stream.pushEvent(createSessionWaitingEvent());+ stream.pushEvent(createSessionWaitingEvent(TEST_USAGE));}, 0);await messagePromise;