Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

feat(eve): sessions - prewarm and stream continuously (#3417)

Andrew Barba6ddfa9bfbe31
diff --git a/.changeset/eval-judge-prompts.md b/.changeset/eval-judge-prompts.md
new file mode 100644
index 000000000..12a7f0178
--- /dev/null
+++ b/.changeset/eval-judge-prompts.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Eval judges now retain the latest prompt across approval responses and stream reads, and include the text sent with file attachments.
diff --git a/.changeset/explicit-eval-sessions.md b/.changeset/explicit-eval-sessions.md
new file mode 100644
index 000000000..1272bde7a
--- /dev/null
+++ b/.changeset/explicit-eval-sessions.md
@@ -0,0 +1,5 @@
+---
+"eve": minor
+---
+
+Eval session ownership is now explicit: `t.session()` creates an empty session, and every `t.send()` creates a fresh session and returns a turn with `.session` for follow-ups. Replace `t.newSession()` with `await t.session()`, move conversation state and operations from `t` to the session handle, and read replies from `turn.message`.
diff --git a/.changeset/resumed-session-ready.md b/.changeset/resumed-session-ready.md
new file mode 100644
index 000000000..66e92941d
--- /dev/null
+++ b/.changeset/resumed-session-ready.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Fix frontend session resume so unused prewarmed sessions accept their first message and steering messages during resumed turns settle without leaving the composer busy.
diff --git a/.changeset/warm-sessions-wait.md b/.changeset/warm-sessions-wait.md
new file mode 100644
index 000000000..5e269876a
--- /dev/null
+++ b/.changeset/warm-sessions-wait.md
@@ -0,0 +1,7 @@
+---
+"eve": patch
+---
+
+Create conversation sessions before their first turn through the eve HTTP channel, TypeScript client, eval drivers, and frontend bindings so applications can move durable session startup off the first-message path.
+
+Frontend bindings support opt-in `prewarm: true`, keep consuming the session stream across turns, and retry a starting inbox without waiting for stream events. Session initialization runs with the first message's identity and context.
diff --git a/apps/docs/app/[lang]/(home)/components/file-tree.tsx b/apps/docs/app/[lang]/(home)/components/file-tree.tsx
index f7d4ecb1c..dc24948c8 100644
--- a/apps/docs/app/[lang]/(home)/components/file-tree.tsx
+++ b/apps/docs/app/[lang]/(home)/components/file-tree.tsx
@@ -195,9 +195,9 @@ import { includes } from "eve/evals/expect";
export default defineEval({
async test(t) {
- await t.send("What is the weather in Brooklyn?");
+ const turn = await t.send("What is the weather in Brooklyn?");
t.succeeded();
- t.check(t.reply, includes("Sunny"));
+ t.check(turn.message, includes("Sunny"));
},
});`,
},
diff --git a/docs/channels/eve.mdx b/docs/channels/eve.mdx
index 42cdb2af0..95bf51698 100644
--- a/docs/channels/eve.mdx
+++ b/docs/channels/eve.mdx
@@ -24,7 +24,7 @@ The default eve channel inspects the agent, creates sessions, accepts callbacks
- `GET /eve/v1/health` (check whether the application is reachable)
- `GET /eve/v1/info` (inspect the agent)
-- `POST /eve/v1/session` (start a session and send its first message)
+- `POST /eve/v1/session` (create a session, optionally sending its first message)
- `POST /eve/v1/session/:sessionId` (send a follow-up)
- `POST /eve/v1/session/:sessionId/cancel` (cancel the in-flight turn)
- `POST /eve/v1/session/:sessionId/clear` (clear the session's model history)
@@ -38,7 +38,21 @@ The session routes use only durable session IDs. Create a session explicitly, th
### Start and continue a session
-Start a session with an initial message, then use the returned `sessionId` for every follow-up and control operation:
+Create a conversation session before its first message with a bodyless request:
+
+```bash
+curl -X POST https://<deployment>/eve/v1/session
+# {"ok":true,"sessionId":"wrun_A","status":"accepted"}
+```
+
+See [Start a session](../concepts/sessions-runs-and-streaming#start-a-session) for the
+initialization boundary, first-turn identity, readiness response, and retry behavior.
+
+The create request still requires route auth. Session audience and the timeout are selected
+at creation; they do not change when the first message arrives. Task mode and turn-scoped
+fields such as `clientContext` and `outputSchema` require a message on the create request.
+
+Include an initial message to create the session and start its first turn in one request:
```bash
curl -X POST https://<deployment>/eve/v1/session \
@@ -47,11 +61,6 @@ curl -X POST https://<deployment>/eve/v1/session \
# {"ok":true,"sessionId":"wrun_A","status":"accepted"}
```
-The `202` response means Workflow accepted the durable run. The route does not wait for command
-inbox or continuation-token ownership. The inbox may still be starting, so an immediate follow-up
-or control request can report that the session is not active. The event-stream client retries the
-brief window before the new run becomes readable.
-
Authenticated callers that may retry a create request can pass their own `operationId` for
create-once semantics. Once the operation owner is active, the same operation under the same
authenticated principal returns that session instead of dispatching the input again. The create
@@ -69,7 +78,7 @@ curl -X POST https://<deployment>/eve/v1/session \
# {"ok":true,"sessionId":"wrun_A","status":"accepted"}
```
-The first request requires `message`. A follow-up request accepts exactly one of
+When the create request includes a message, a follow-up request accepts exactly one of
`message` or `inputResponses`; use the latter to answer a pending HITL request:
```bash
@@ -82,8 +91,9 @@ Follow-up messages use `turnPolicy: "steer"` by default. Before assistant answer
Set `turnPolicy: "queue"` on `eveChannel(...)` when follow-ups should wait for active turns to finish. `inputResponses` answer their addressed requests.
-Sending a message to an unknown, terminal, or not-yet-active session ID returns `409` with
+Sending a message to an unknown or terminal session ID returns `409` with
`{"code":"session_not_active","error":"The session is no longer active.","ok":false}`.
+An existing workflow whose inbox is not yet available returns `409 session_not_ready` instead.
TypeScript clients expose the stable code as `ClientError.code`. The route never
creates or follows a replacement session.
@@ -225,7 +235,7 @@ export default eveChannel({
});
```
-`onMessage` must return an auth result. Return `title` alongside `auth` to set the title when the dispatch starts a run. A successful canonical eve HTTP message always dispatches and therefore always produces or continues a session.
+`onMessage` must return an auth result. Return `title` alongside `auth` to set the title when the dispatch creates a session or sends the first message to a prewarmed session. A successful canonical eve HTTP message always dispatches and therefore always produces or continues a session.
## Clients
diff --git a/docs/concepts/sessions-runs-and-streaming.md b/docs/concepts/sessions-runs-and-streaming.md
index 660d7def3..63d80d5d5 100644
--- a/docs/concepts/sessions-runs-and-streaming.md
+++ b/docs/concepts/sessions-runs-and-streaming.md
@@ -33,16 +33,31 @@ React, Vue, and Svelte apps reach for [`useEveAgent()`](../guides/frontend/overv
## Start a session
+Create and park a conversation session before its first turn by omitting `message`:
+
+```bash
+curl -X POST http://127.0.0.1:2000/eve/v1/session
+```
+
+eve starts the durable workflow, establishes its inbox, and waits for the first message before
+running session-scoped initialization or emitting `session.started`. The first message sent to the returned
+`sessionId` remains `turn_0`. Message-free creation supports conversation mode only and does not
+accept turn-scoped `clientContext`, `outputSchema`, callbacks, or activity observers.
+
+To create the session and start its first turn in one request, include the message:
+
```bash
curl -X POST http://127.0.0.1:2000/eve/v1/session \
-H 'content-type: application/json' \
-d '{"message":"Summarize the latest forecast."}'
```
-eve responds with `202` and the durable `sessionId` in the JSON body and
+In both forms, eve responds with `202` and the durable `sessionId` in the JSON body and
`x-eve-session-id` header as soon as Workflow accepts the run. The command inbox can still be
-starting at that point. An immediate follow-up can return `409 session_not_active`; wait for
-`session.waiting` before sending the next message.
+starting at that point. An immediate follow-up can return `409 session_not_ready`; retry that
+code with bounded backoff. The TypeScript client retries sends for up to 20 seconds and respects
+the caller's abort signal. Do not wait
+for `session.waiting` on a prewarmed session: initialization and its first events require a message.
## Stream a session
diff --git a/docs/evals/assertions.mdx b/docs/evals/assertions.mdx
index 66a1f198b..8d9dbca50 100644
--- a/docs/evals/assertions.mdx
+++ b/docs/evals/assertions.mdx
@@ -50,15 +50,15 @@ attached.messageIncludes("Sunny");
## Value assertions with `t.check`
-`t.check(value, assertion)` grades an explicit value against a builder from `eve/evals/expect`. The value can be `t.reply`, a turn's `.message`, parsed JSON, or any local you computed:
+`t.check(value, assertion)` grades an explicit value against a builder from `eve/evals/expect`. The value can be a turn’s `.message`, parsed JSON, or any local you computed:
```ts
import { includes, equals, matches, satisfies, similarity } from "eve/evals/expect";
-t.check(t.reply, includes(/sunny/i)); // substring or RegExp (gate)
+t.check(turn.message, includes(/sunny/i)); // substring or RegExp (gate)
t.check(parsed, equals({ city: "Brooklyn" })); // deep structural equality (gate)
t.check(parsed, matches(WeatherSchema)); // Standard Schema, e.g. Zod (gate)
-t.check(t.reply, similarity("Sunny, 72F")); // fuzzy 0–1 Levenshtein (soft)
+t.check(turn.message, similarity("Sunny, 72F")); // fuzzy 0–1 Levenshtein (soft)
t.check(
latencyMs,
satisfies((value) => value < 1_000, "latency under one second"),
@@ -114,7 +114,7 @@ const request = session.requireInputRequest({
## Run state and derived facts
-Beyond the raw `t.events` stream, the runner derives typed facts the assertions read: tool calls (name, input, output, lifecycle status), subagent calls, and HITL input requests. A turn that leaves the session open for a next message is the normal end state of a successful turn; parking on unanswered HITL input is tracked separately.
+Beyond the raw `session.events` stream, the runner derives typed facts the assertions read: tool calls (name, input, output, lifecycle status), subagent calls, and HITL input requests. A turn that leaves the session open for a next message is the normal end state of a successful turn; parking on unanswered HITL input is tracked separately.
Typed event matching covers presence, absence, numeric or predicate counts, partial event data, and ordering:
@@ -166,8 +166,8 @@ The defaults are chosen so you rarely set severity. Run-level methods and `inclu
```ts
t.calledTool("get_weather").soft(); // record the tool call as a metric, don't gate
-t.check(t.reply, similarity("Sunny")).label("forecast wording").atLeast(0.8);
-t.check(t.reply, includes("error")).soft(); // track without failing the build
+t.check(turn.message, similarity("Sunny")).label("forecast wording").atLeast(0.8);
+t.check(turn.message, includes("error")).soft(); // track without failing the build
```
## What to read next
diff --git a/docs/evals/cases.mdx b/docs/evals/cases.mdx
index e8e9d9e70..051b8c7e2 100644
--- a/docs/evals/cases.mdx
+++ b/docs/evals/cases.mdx
@@ -15,7 +15,7 @@ export default defineEvalConfig({});
## Single-turn evals
-The common case sends one turn and asserts on the reply. `t.send(input)` resolves once the turn settles, and `t.reply` is the last assistant message:
+The common case sends one turn and asserts on the reply. `t.send(input)` resolves once the turn settles, and the returned turn’s `.message` is the assistant reply:
```ts title="evals/weather/brooklyn-forecast.eval.ts"
import { defineEval } from "eve/evals";
@@ -23,9 +23,9 @@ import { includes } from "eve/evals/expect";
export default defineEval({
async test(t) {
- await t.send("What is the weather in Brooklyn?");
+ const turn = await t.send("What is the weather in Brooklyn?");
t.succeeded();
- t.check(t.reply, includes("Sunny"));
+ t.check(turn.message, includes("Sunny"));
},
});
```
@@ -60,7 +60,7 @@ evals/
## Multi-turn evals
-Drive several turns in sequence for branching, HITL approvals, structured output, attachments, or multiple sessions. Because assertions live in the function, an intermediate value is a local variable. Judge a draft before the next turn overwrites it, then keep going.
+Drive several turns in sequence for branching, HITL approvals, structured output, attachments, or multiple sessions. Because assertions live in the function, an intermediate value is a local variable. Keep each turn in a local variable and continue through its `.session`.
```ts title="evals/draft-then-send.eval.ts"
import { defineEval } from "eve/evals";
@@ -72,7 +72,7 @@ export default defineEval({
t.check(draft.message, includes("Best regards"));
t.judge.autoevals.closedQA("professional tone", { on: draft.message }).atLeast(0.6);
- await t.send("Now send it.");
+ await draft.session.send("Now send it.");
t.calledTool("send_email");
},
});
@@ -88,7 +88,7 @@ export default defineEval({
async test(t) {
const first = await t.send("My favorite word is marigold.");
- const second = await t.send("What is my favorite word?");
+ const second = await first.session.send("What is my favorite word?");
await t.require(second.sessionId, equals(first.sessionId));
t.succeeded();
@@ -96,7 +96,7 @@ export default defineEval({
t.judge.autoevals
.closedQA("The assistant remembers the user's favorite word across turns", {
- on: t.transcript,
+ on: first.session.transcript,
})
.atLeast(0.8);
},
@@ -105,21 +105,56 @@ export default defineEval({
## The drive API
-`t` drives the primary session; `t.newSession()` returns an independent `EveEvalSession` against the same target, whose events feed the same run-level assertions.
+Every `t.session()` or `t.send()` creates a new independent conversation. Continue an existing conversation through its session handle. Events from all sessions feed the same run-level assertions.
-- `t.send(message, options?)` sends a turn and waits for it to settle. It matches `ClientSession.send()` and resolves to a turn carrying `.message` and `.expectOk()`.
-- `t.start(message)` starts a text turn but returns as soon as the server accepts it. The returned live turn exposes `.sessionId`, `.waitForEvent(...)`, `.cancel()`, and `.result()` for coordinating with work that is still running.
-- `t.cancel()` requests cooperative cancellation of the primary session's active turn. Both `accepted` and `no_active_turn` are successful outcomes.
-- `t.sendFile(text, path, mediaType?)` attaches a local file as a data URL.
-- `t.requireInputRequest(filter?)` records a gate, requires exactly one pending request, and returns it. Filters match tool name, action input, prompt, display, and option ids.
-- `t.respond(responses, options?)` answers specific pending input requests and sends them as the next turn.
-- `t.respondAll(optionId)` answers every pending input request with the same option and sends the responses as the next turn.
-- `t.reply` is the last assistant message (or `null`); `t.sessionId` is the current session id; `t.events` is the full typed event stream captured so far.
-- `t.transcript` formats the primary session's observed user and assistant messages in turn order. Pass it as a judge's `on` value to grade the complete conversation. An independent session returned by `t.newSession()` exposes its own `session.transcript`.
+- `t.session(options?)` creates a session without starting a turn or consuming its stream. It resolves on `202 Accepted` with an `EveEvalSession` whose `.sessionId` and `.state` are available. Options accept per-request `headers` and `signal`; creation uses the eval timeout signal by default.
+- `t.send(message, options?)` creates a new session with its first message in one request and waits for the turn to settle. The returned turn carries `.session`, `.message`, and `.expectOk()`.
+- `session.send(message, options?)` sends a follow-up on that session and waits for it to settle.
+- `session.start(message, options?)` starts a text turn and returns as soon as the server accepts it. The live turn exposes `.session`, `.sessionId`, `.waitForEvent(...)`, `.cancel()`, and `.result()`.
+- `session.cancel()` requests cooperative cancellation of that session's active turn. Both `accepted` and `no_active_turn` are successful outcomes.
+- `session.sendFile(text, path, mediaType?)` attaches a local file as a data URL.
+- `session.requireInputRequest(filter?)` records a gate, requires exactly one pending request, and returns it. Filters match tool name, action input, prompt, display, and option ids.
+- `session.respond(responses, options?)` answers specific pending input requests and sends them as the next turn. `session.startRespond(...)` returns a live turn for the same operation.
+- `session.respondAll(optionId)` answers every pending input request with the same option and sends the responses as the next turn.
+- `session.events` contains the events captured for that session. `session.transcript` formats its observed user and assistant messages in turn order.
-The transcript uses `User:` and `Assistant:` labels separated by blank lines. It excludes reasoning, tool calls, tool results, and messages from other sessions. It updates after each turn settles, so read it after `await t.send(...)`, `await t.respond(...)`, or `await live.result()`.
+The transcript uses `User:` and `Assistant:` labels separated by blank lines. It excludes reasoning, tool calls, tool results, and messages from other sessions. It updates after each turn settles, so read it after `await session.send(...)`, `await session.respond(...)`, or `await live.result()`.
-Each `send` (and `respond`/`respondAll`) resolves to an immutable turn with `.message`, `.data`, `.events`, `.inputRequests`, `.toolCalls`, `.sessionId`, `.status`, and `.expectOk()`. Use `.sessionId` to relate turns or attach follow-up work to the session that produced a specific turn. `expectOk()` throws only when the turn ended failed; a session left open for a next message is the normal end state of a successful turn.
+Each `send` (and `respond`/`respondAll`) resolves to a turn with `.message`, `.data`, `.events`, `.inputRequests`, `.toolCalls`, `.session`, `.sessionId`, `.status`, and `.expectOk()`. Turn results retain their captured output and events; `.session` references the live conversation handle. `expectOk()` throws only when the turn ended failed; a session left open for a next message is the normal end state of a successful turn.
+
+## Prewarming a session
+
+Use `t.session()` to create a session before its first message:
+
+```ts title="evals/prewarm.eval.ts"
+import { defineEval } from "eve/evals";
+import { equals } from "eve/evals/expect";
+
+export default defineEval({
+ async test(t) {
+ const session = await t.session();
+ const turn = await session.send("Greet Alice briefly.");
+ await t.require(turn.sessionId, equals(session.sessionId));
+ turn.event("session.started", { count: 1 });
+ turn.event("turn.started", { count: 1, data: { turnId: "turn_0" } });
+ t.succeeded();
+ },
+});
+```
+
+Acceptance is not an initialization barrier. The first message runs session initialization;
+the TypeScript client handles bounded retries if the workflow inbox is still starting.
+Creation failures reject `t.session()` without returning a handle. Each call creates an
+independent session, including concurrent calls. Accepted sessions appear in eval results
+and participate in timeout cleanup even when no message was sent.
+
+## Migrating existing evals
+
+Replace `t.newSession()` and `t.prewarm()` with `const session = await t.session()`.
+Replace consecutive `t.send()` calls with `session.send()` calls, or continue the first
+returned turn through `turn.session.send()`. Read identity, cursor, events, and transcript
+from `session`; read replies from `turn.message`. Session operations such as `respond()`,
+`requireInputRequest()`, and `cancel()` also move from `t` to `session`.
## In-flight turns
@@ -130,7 +165,8 @@ import { defineEval } from "eve/evals";
export default defineEval({
async test(t) {
- const live = await t.start("Run the long operation.");
+ const session = await t.session();
+ const live = await session.start("Run the long operation.");
await live.waitForEvent("actions.requested", {
data: {
@@ -152,7 +188,7 @@ export default defineEval({
Use `expectOk()` only when the next operation depends on that intermediate turn succeeding. A final `t.succeeded()` already records a complete-run gate.
-To intentionally omit an eval for the current target, call `t.skip(reason)` before sending messages or recording assertions. Skipped evals are reported separately and do not affect the exit code.
+To intentionally omit an eval for the current target, call `t.skip(reason)` before creating sessions, sending messages, or recording assertions. Skipped evals are reported separately and do not affect the exit code.
Events from every session are captured in the result and artifacts. `t.log(message)` records debug lines into the eval artifact; `--verbose` also streams them to stdout as evals run. `t.signal` is an `AbortSignal` that fires on timeout.
@@ -174,9 +210,9 @@ export default rows.map((row) =>
defineEval({
description: row.task,
async test(t) {
- await t.send(row.prompt);
+ const turn = await t.send(row.prompt);
t.succeeded();
- t.check(t.reply, equals(row.sql));
+ t.check(turn.message, equals(row.sql));
},
}),
);
diff --git a/docs/evals/judge.mdx b/docs/evals/judge.mdx
index fe9900552..57a4c84b3 100644
--- a/docs/evals/judge.mdx
+++ b/docs/evals/judge.mdx
@@ -19,7 +19,7 @@ export default defineEval({
## The graders
-The judges live under `t.judge.autoevals`. The namespace names the [Braintrust autoevals](https://github.com/braintrustdata/autoevals) grader family, so the factuality and closedQA semantics are autoevals', not eve-invented. Each grader scores `t.reply` by default and is soft by default (tracked, no gate):
+The judges live under `t.judge.autoevals`. The namespace names the [Braintrust autoevals](https://github.com/braintrustdata/autoevals) grader family, so the factuality and closedQA semantics are autoevals', not eve-invented. Each grader scores the most recently settled turn’s assistant message by default and is soft by default (tracked, no gate):
| Grader | Grades |
| ---------------------------------------- | -------------------------------------------------------------------------------------- |
@@ -28,6 +28,8 @@ The judges live under `t.judge.autoevals`. The namespace names the [Braintrust a
| `t.judge.autoevals.closedQA(criteria)` | Whether the reply satisfies a free-form yes/no criterion (no expected answer to match) |
| `t.judge.autoevals.sql(expected)` | Semantic equivalence of two SQL statements |
+Judges receive the latest message text from the session whose turn most recently settled as the input prompt. Approval responses and stream reads preserve that prompt. For multipart messages, including `session.sendFile()`, the prompt contains the text parts.
+
A failed `closedQA` assertion prints the input and judge explanation:
```text
@@ -40,7 +42,7 @@ A failed `closedQA` assertion prints the input and judge explanation:
The reference or criteria is the positional argument. An options object follows:
-- `on` is the value to grade, defaulting to `t.reply`. Pass an intermediate draft or parsed value to grade it instead.
+- `on` is the value to grade, defaulting to the most recently settled turn’s assistant message. Pass an intermediate draft or parsed value to grade it instead.
- `model` and `modelOptions` are a per-call judge override (see below).
```ts
@@ -48,20 +50,20 @@ const draft = await t.send("Draft the welcome email.");
t.judge.autoevals.closedQA("professional tone", { on: draft.message }).atLeast(0.6);
```
-For a multi-turn eval, pass `t.transcript` to grade the primary session's complete observed conversation instead of only its final reply:
+For a multi-turn eval, pass `session.transcript` to grade a session's complete observed conversation instead of only its final reply:
```ts
-await t.send("My favorite word is marigold. Remember it.");
-await t.send("What is my favorite word?");
+const { session } = await t.send("My favorite word is marigold. Remember it.");
+await session.send("What is my favorite word?");
t.judge.autoevals
.closedQA("The assistant remembers the user's favorite word across turns", {
- on: t.transcript,
+ on: session.transcript,
})
.atLeast(0.8);
```
-`t.transcript` contains the session's user and assistant messages in turn order. It excludes reasoning, tool calls, and tool results. See [Multi-turn evals](./cases#multi-turn-evals) for the transcript format and independent sessions.
+`session.transcript` contains the session's user and assistant messages in turn order. It excludes reasoning, tool calls, and tool results. See [Multi-turn evals](./cases#multi-turn-evals) for the transcript format and independent sessions.
## Soft scoring and thresholds
diff --git a/docs/evals/overview.mdx b/docs/evals/overview.mdx
index 0ba269667..3e44f5063 100644
--- a/docs/evals/overview.mdx
+++ b/docs/evals/overview.mdx
@@ -32,10 +32,10 @@ import { includes } from "eve/evals/expect";
export default defineEval({
description: "Basic message and tool-usage coverage for the weather agent.",
async test(t) {
- await t.send("What is the weather in Brooklyn?");
+ const turn = await t.send("What is the weather in Brooklyn?");
t.succeeded();
t.calledTool("get_weather");
- t.check(t.reply, includes("Sunny"));
+ t.check(turn.message, includes("Sunny"));
},
});
```
@@ -100,7 +100,7 @@ model: mockModel({
`t` is both the driver and the assertion surface. There are no separate `input`, `run`, `checks`, or `scores` fields. You write ordinary control flow, sending turns and asserting inline.
-- **Drive** the agent: `t.send(...)`, `t.start(...)`, `t.cancel()`, `t.respond(...)`, `t.respondAll(...)`, `t.sendFile(...)`, `t.requireInputRequest(...)`, `t.newSession()`. Live turns returned by `start()` can wait for typed mid-turn events before cancellation or settlement. Read what came back with `t.reply` (the last assistant message), `t.transcript` (the primary session's user and assistant messages), `t.sessionId`, and `t.events`. See [Cases](./cases).
+- **Drive** the agent: `t.session()` creates an empty session; `t.send(...)` creates a session with its first message and returns a turn. Continue through `turn.session.send(...)`. Session handles also expose `start`, `cancel`, `respond`, `respondAll`, `sendFile`, and `requireInputRequest`. Read output from `turn.message`, and conversation state from `turn.session`. See [Cases](./cases).
- **Assert** with three surfaces, covered next.
## Three assertion surfaces
@@ -108,8 +108,8 @@ model: mockModel({
Each surface matches a genuinely different kind of judgment:
- **Scoped methods** read the final whole run on `t`, snapshot one independent session when invoked there, or inspect one immutable `EveEvalTurn`. See [Assertions](./assertions).
-- **`t.check(value, assertion)`** grades an explicit value with a deterministic builder from `eve/evals/expect`, such as `t.check(t.reply, includes("sunny"))`. Grade `t.reply`, an intermediate draft, parsed JSON, or anything else. See [Assertions](./assertions).
-- **`t.judge.autoevals.*`** is the LLM-as-judge surface, like `t.judge.autoevals.closedQA("cites a source")`. It grades `t.reply` by default; pass `{ on: t.transcript }` to grade a multi-turn conversation. The judge uses the configured judge model, never the agent under test. See [Judge](./judge).
+- **`t.check(value, assertion)`** grades an explicit value with a deterministic builder from `eve/evals/expect`, such as `t.check(turn.message, includes("sunny"))`. Grade `turn.message`, an intermediate draft, parsed JSON, or anything else. See [Assertions](./assertions).
+- **`t.judge.autoevals.*`** is the LLM-as-judge surface, like `t.judge.autoevals.closedQA("cites a source")`. It grades the most recently settled turn’s assistant message by default; pass `{ on: turn.session.transcript }` to grade a multi-turn conversation. The judge uses the configured judge model, never the agent under test. See [Judge](./judge).
## Gate vs soft
diff --git a/docs/evals/targets.mdx b/docs/evals/targets.mdx
index b671e3782..68d8d494c 100644
--- a/docs/evals/targets.mdx
+++ b/docs/evals/targets.mdx
@@ -29,6 +29,11 @@ export default defineEval({
Sessions attached this way are full `EveEvalSession`s: you can keep driving them and assert directly on that session (`session.succeeded()`, `session.calledTool(...)`). Aggregate assertions on `t` continue to read the whole run, including every attached session.
+For session creation and message delivery, use the [drive API](./cases#the-drive-api).
+`t.session()` creates a session without a first message; call `session.send()` or `session.start()`
+on the returned handle to use it. `t.send()` creates a new session with its first message.
+These helpers handle the session protocol, retries, and event collection for the eval.
+
## Authentication
Local targets send no auth: `eve eval` owns the dev server it boots. For a remote `--url`, eve gets the expected Vercel owner and project from `VERCEL_ORG_ID` and `VERCEL_PROJECT_ID` when both are set. Otherwise it reads `.vercel/project.json`. Eve then asks Vercel to resolve the exact HTTPS origin and sends ambient credentials only when the project IDs match. An arbitrary URL remains anonymous.
diff --git a/docs/guides/client/messages.mdx b/docs/guides/client/messages.mdx
index a0799cd6f..4b3c024f3 100644
--- a/docs/guides/client/messages.mdx
+++ b/docs/guides/client/messages.mdx
@@ -36,11 +36,11 @@ console.log(result.status, result.message);
When the stream includes `session.failed`, the turn returns `status: "failed"` rather than throwing. Transport and route errors throw `ClientError`.
-`session.send()` retries `409 session_not_active` three times when a durable run has been accepted
-but its command inbox is still starting. The retries wait 250 ms, 500 ms, and 1 second. Other
-errors, `session.respond()`, control methods, and raw HTTP requests do not use this retry. An unknown
-session still throws `ClientError` immediately, while a terminal session throws after the final
-attempt. The client never creates a replacement session.
+`session.send()` retries `409 session_not_ready` with bounded exponential backoff for up to 20
+seconds when a durable run has been accepted but its command inbox is still starting. The caller's
+abort signal cancels the wait. Other errors, `session.respond()`, control methods, and raw HTTP
+requests do not use this retry. An unknown
+or terminal session throws `ClientError` immediately with `session_not_active`. The client never creates a replacement session.
## Send a full turn payload
diff --git a/docs/guides/client/overview.mdx b/docs/guides/client/overview.mdx
index bb5f1a9bc..5a82a5aa0 100644
--- a/docs/guides/client/overview.mdx
+++ b/docs/guides/client/overview.mdx
@@ -113,7 +113,21 @@ Per-request headers override client-level values with the same name. For example
## Sessions
-For an ID-addressed session, create it explicitly with the first message:
+Create and park an ID-addressed conversation before its first turn when you can start session work
+while the user is still composing:
+
+```ts
+const { session } = await client.sessions.create();
+
+const response = await session.send("Summarize account A.");
+await response.result();
+```
+
+The create promise resolves when the server accepts the durable workflow. It does not open the
+event stream or wait for initialization; an early `session.send()` uses the
+same bounded startup retries as other follow-ups.
+
+To create the session and start its first turn in one request, include the message:
```ts
const { session, response } = await client.sessions.create({ message: "Summarize account A." });
diff --git a/docs/guides/frontend/overview.mdx b/docs/guides/frontend/overview.mdx
index a4c74cbd0..59fa9f754 100644
--- a/docs/guides/frontend/overview.mdx
+++ b/docs/guides/frontend/overview.mdx
@@ -74,9 +74,10 @@ export function Chat() {
| --------- | ----------------------------------------------------------------------------------------- |
| `data` | Projected UI state from the reducer. Defaults to `{ messages }`. |
| `status` | `"ready"`, `"resuming"`, `"submitted"`, `"streaming"`, or `"error"`. Drives the composer. |
-| `error` | The last `Error` thrown, if any. |
+| `error` | The latest session creation, stream, resume, or turn failure. |
| `events` | Raw eve stream events for this session. |
| `session` | Serializable fixed session cursor (`sessionId`, `streamIndex`). |
+| `prewarm` | Create the durable session without starting a turn. |
| `send` | Send text or a multi-part message, with per-turn options. |
| `respond` | Answer pending HITL input requests, with per-turn options. |
| `resume` | Replay an attached session and follow its in-flight turn. |
@@ -85,6 +86,50 @@ export function Chat() {
Most chat UIs only need `data.messages` and `status`. Drop down to `events` when you need the authoritative wire events directly, for example to persist an audit log or build a custom projection.
+Pass `prewarm: true` to start a workflow when the chat mounts and after each reset:
+
+```tsx
+import { useEveAgent } from "eve/react";
+
+export function Chat() {
+ const agent = useEveAgent({ prewarm: true });
+
+ return <button onClick={agent.reset}>New chat</button>;
+}
+```
+
+Prewarming defaults to `false`: the first send creates the session. Enable it when your
+application's auth, headers, and any chat-record binding are ready at mount. Applications
+that prepare those in the submit handler should keep the default.
+
+Prewarming starts the workflow and establishes its inbox, then parks before session hooks,
+dynamic definitions, or sandbox setup run. The first message passes through the channel's
+`onMessage` hook and supplies the identity and context for initialization and `turn_0`.
+
+The explicit `prewarm()` method resolves on `202 Accepted`, as does `client.sessions.create()`.
+It does not wait for an initialization event; concurrent calls share the same create request.
+The composer stays `"ready"`, meaning it can accept input, and prewarming does not run
+`prepareSend`. A concurrent `send()` waits only for acceptance, then posts the message.
+If the inbox is still starting, the client retries `409 session_not_ready` with bounded
+exponential backoff for up to 20 seconds. The caller's abort signal cancels the wait. Stream
+events are not a prerequisite for sending.
+
+A failed create sets `error` and calls `onError` when no send is waiting. A waiting send can
+fall back to creating a session with its message. After creation succeeds, initialization
+failures arrive on the stream during the first turn. The configured session timeout starts
+at creation, so an unused prewarmed session can expire before its first message.
+
+The hook keeps consuming one session stream across turns and idle periods. Replies,
+authorization updates, and turns started elsewhere all update the same projection. A turn settling
+does not close the connection. The transport reconnects from its cursor after a disconnect or the
+server's renewable 60-second lease; lease renewal does not start a turn or clear UI state.
+Unmounting or resetting closes the local stream without cancelling the old durable session.
+
+`reset()` reuses the hook instance and clears the conversation. With `prewarm: true`, it
+immediately prepares the next owned session; otherwise, the next send creates it. An explicitly supplied
+`session` remains externally owned and is never replaced by prewarming or reset. Use
+`initialSession` with `resume: true` to restore a saved chat.
+
`data.messages` are eve-owned `EveMessage[]`. Common text, reasoning, file, and dynamic-tool parts follow the [AI SDK `UIMessage`](https://ai-sdk.dev/docs/reference/ai-sdk-core/ui-message) rendering convention, but the types are not interchangeable. eve also exposes authorization and HITL metadata, and a file part's URL can be absent. Adapt those parts before passing messages to an API typed as `UIMessage[]`.
When the root agent delegates, its stream emits `subagent.called` with the child's `childSessionId`, then `subagent.completed` after admission with a working task receipt. Later task notifications wake the parent with completion, failure, or cancellation. Detailed child progress lives on the child session's stream instead of being flattened into the root `data.messages`. Use the lower-level [TypeScript client](../client/overview#sessions) to attach to that ID when your UI needs live subagent activity. See [What the parent sees](../../subagents#what-the-parent-sees) for the complete contract.
@@ -227,9 +272,9 @@ const agent = useEveAgent({
The hook accepts these lifecycle callbacks:
- `onEvent(event)`: fires for each eve stream event as it arrives.
-- `onError(error)`: fires with the last `Error` when a turn fails.
+- `onError(error)`: fires when session creation, streaming, resume, or a turn fails. A failed prewarm reports here when no send is waiting to retry creation.
- `onFinish(snapshot)`: fires with the final `{ data, status, session, ... }` snapshot once a turn settles.
-- `onSessionChange(session)`: fires when the session cursor advances. Persist it to resume across reloads.
+- `onSessionChange(session)`: fires when a session is created, its cursor advances, or reset clears it. Persist the session state to resume across reloads.
```tsx
const agent = useEveAgent({
@@ -306,9 +351,9 @@ then save a final snapshot in `onFinish`.
`initialEvents` must be an ordered prefix of the same session's stream, but its endpoint does not have to line up exactly with where the stream resumes. When the event count matches `initialSession.streamIndex`, catch-up continues from that cursor. A partial or overlapping saved log falls back to index `0`. Every event carries a stable [`meta.id`](/docs/concepts/sessions-runs-and-streaming#the-event-envelope), and the store drops any event whose id it has already applied, so an overlapping replay renders once and `onEvent` only fires for events your UI has not seen.
-For multiple chat threads, keep one saved event log and session cursor per thread. `agent`, `host`, `reducer`, `session`, `initialEvents`, `initialSession`, `auth`, `headers`, `optimistic`, and `resume` are read when the hook creates its store, so remount the chat component when switching threads, for example with `key={chat.id}`.
+For multiple chat threads, keep one saved event log and session cursor per thread. `agent`, `host`, `reducer`, `session`, `initialEvents`, `initialSession`, `auth`, `headers`, `optimistic`, `prewarm`, and `resume` are read when the hook creates its store, so remount the chat component when switching threads, for example with `key={chat.id}`.
-Pass `resume: true` with `initialSession` to rebuild the projection from the durable stream after mount. While `status` is `"resuming"`, render hydrated `data` but disable message and HITL submission; do not present cancellation or active-turn progress controls. If catch-up finds an in-flight turn, `status` changes to `"streaming"` before the binding follows it to a boundary. A settled tail changes directly to `"ready"` after a bounded catch-up check, without waiting for the live stream idle timeout. If that check finds a newly started turn or pending authorization, eve keeps following it. A terminal session failure changes to `"error"`.
+Pass `resume: true` with `initialSession` to rebuild the projection from the durable stream after mount. While `status` is `"resuming"`, render hydrated `data` but disable message and HITL submission; do not present cancellation or active-turn progress controls. If catch-up finds an in-flight turn, `status` changes to `"streaming"` before the binding follows it to a boundary. The stream reports its initial durable tail so catch-up can finish without closing the connection or waiting for an idle timeout. A settled tail changes directly to `"ready"`; a newer turn or pending authorization remains `"streaming"`. The connection stays open to receive later events. A terminal session failure changes to `"error"`.
```tsx
const agent = useEveAgent({
@@ -332,6 +377,10 @@ const agent = useEveAgent({
});
```
+Headers supplied to `send()` apply to that turn request and become the credentials used by the
+continuous session stream on its next reconnect. A later send replaces them with that turn's
+headers.
+
When a framework integration mounts multiple named agents, pass `agent` instead of `host`:
```tsx
diff --git a/docs/guides/frontend/use-eve-agent-svelte.mdx b/docs/guides/frontend/use-eve-agent-svelte.mdx
index 17b620b67..4c365e41e 100644
--- a/docs/guides/frontend/use-eve-agent-svelte.mdx
+++ b/docs/guides/frontend/use-eve-agent-svelte.mdx
@@ -45,14 +45,14 @@ Call the binding once for a conversation and read its reactive getters directly.
The state fields are reactive getters; the commands are ordinary methods:
-| Property | Svelte shape |
-| ---------------------------------------------- | --------------------------------- |
-| `data` | `TData` |
-| `status` | `UseEveAgentStatus` |
-| `error` | `Error \| undefined` |
-| `events` | `readonly MessageStreamEvent[]` |
-| `session` | `ClientSessionState \| undefined` |
-| `send`, `respond`, `resume`, `cancel`, `reset` | Methods |
+| Property | Svelte shape |
+| --------------------------------------------------------- | --------------------------------- |
+| `data` | `TData` |
+| `status` | `UseEveAgentStatus` |
+| `error` | `Error \| undefined` |
+| `events` | `readonly MessageStreamEvent[]` |
+| `session` | `ClientSessionState \| undefined` |
+| `send`, `respond`, `prewarm`, `resume`, `cancel`, `reset` | Methods |
Read the getters directly in templates, `$derived`, or `$effect`. The [shared returned-state reference](./overview#returned-state) describes what each value and command does.
@@ -107,6 +107,11 @@ See [Human-in-the-loop prompts](./overview#human-in-the-loop-prompts) for reques
## Cancel, reset, and resume
+Pass `prewarm: true` to prepare an owned workflow on mount and after reset. It defaults
+to `false`, so the first send creates the session. The binding waits for create acceptance,
+retries while the inbox starts, and keeps the session stream open across turns. See [Prewarming and returned state](./overview#returned-state) for creation, failure, and
+connection behavior.
+
Call `agent.cancel()` to stop the durable server-side turn while the binding remains attached through settlement. Destroying the component only disconnects its local stream; it does not cancel server execution. Call `agent.reset()` to clear local state and start a new session. Pass `initialSession`, `initialEvents`, and `resume: true` to restore a saved conversation and follow an in-flight turn. During bounded catch-up, `agent.status` is `"resuming"`; keep the hydrated conversation visible, disable submission, and wait for `"ready"`, `"error"`, or `"streaming"`. Use `agent.resume()` directly when restoration is controlled imperatively. See [Resumable sessions](./overview#resumable-sessions) for the persistence contract and [Sending and streaming](./overview#sending-and-streaming) for cancellation behavior.
## Custom host and credentials
diff --git a/docs/guides/frontend/use-eve-agent-vue.mdx b/docs/guides/frontend/use-eve-agent-vue.mdx
index a90989051..ec0a83b47 100644
--- a/docs/guides/frontend/use-eve-agent-vue.mdx
+++ b/docs/guides/frontend/use-eve-agent-vue.mdx
@@ -44,14 +44,14 @@ async function handleSubmit() {
The state fields are computed refs; the commands are ordinary methods:
-| Property | Vue shape |
-| ---------------------------------------------- | ---------------------------------------------- |
-| `data` | `ComputedRef<TData>` |
-| `status` | `ComputedRef<UseEveAgentStatus>` |
-| `error` | `ComputedRef<Error \| undefined>` |
-| `events` | `ComputedRef<readonly MessageStreamEvent[]>` |
-| `session` | `ComputedRef<ClientSessionState \| undefined>` |
-| `send`, `respond`, `resume`, `cancel`, `reset` | Methods |
+| Property | Vue shape |
+| --------------------------------------------------------- | ---------------------------------------------- |
+| `data` | `ComputedRef<TData>` |
+| `status` | `ComputedRef<UseEveAgentStatus>` |
+| `error` | `ComputedRef<Error \| undefined>` |
+| `events` | `ComputedRef<readonly MessageStreamEvent[]>` |
+| `session` | `ComputedRef<ClientSessionState \| undefined>` |
+| `send`, `respond`, `prewarm`, `resume`, `cancel`, `reset` | Methods |
Destructuring preserves reactivity because each state value remains a ref. Read refs with `.value` in `<script>` and without `.value` in a template. The [shared returned-state reference](./overview#returned-state) describes what each value and command does.
@@ -109,6 +109,11 @@ See [Human-in-the-loop prompts](./overview#human-in-the-loop-prompts) for reques
## Cancel, reset, and resume
+Pass `prewarm: true` to prepare an owned workflow on mount and after reset. It defaults
+to `false`, so the first send creates the session. The binding waits for create acceptance,
+retries while the inbox starts, and keeps the session stream open across turns. See [Prewarming and returned state](./overview#returned-state) for creation, failure, and
+connection behavior.
+
Call `cancel()` to stop the durable server-side turn while the composable remains attached through settlement. Disposing the component only disconnects its local stream; it does not cancel server execution. Call `reset()` to clear local state and start a new session. Pass `initialSession`, `initialEvents`, and `resume: true` to restore a saved conversation and follow an in-flight turn. During bounded catch-up, `status.value` is `"resuming"`; keep the hydrated conversation visible, disable submission, and wait for `"ready"`, `"error"`, or `"streaming"`. Use `resume()` directly when restoration is controlled imperatively. See [Resumable sessions](./overview#resumable-sessions) for the persistence contract and [Sending and streaming](./overview#sending-and-streaming) for cancellation behavior.
## Custom host and credentials
diff --git a/e2e/fixtures/agent-background-tools/evals/background-export.update-and-complete.eval.ts b/e2e/fixtures/agent-background-tools/evals/background-export.update-and-complete.eval.ts
index 52ae8d6c4..8d46c57e5 100644
--- a/e2e/fixtures/agent-background-tools/evals/background-export.update-and-complete.eval.ts
+++ b/e2e/fixtures/agent-background-tools/evals/background-export.update-and-complete.eval.ts
@@ -9,6 +9,7 @@ export default defineEval({
"An authored background defineTool yields state and progress, explicitly posts a message, then completes; the parent sees both.",
async test(t) {
const started = await t.send("BACKGROUND-EXPORT-START");
+ const conversation = started.session;
started.expectOk();
started.calledTool("export");
@@ -16,11 +17,11 @@ export default defineEval({
const taskId = readTaskId(receipt.output);
if (taskId === undefined) throw new Error("export receipt is missing taskId.");
- const sessionId = t.sessionId;
+ const sessionId = conversation.sessionId;
if (sessionId === undefined) throw new Error("Eval has no parent session id.");
const updateLive = t.target.watchTurn(sessionId, {
- startIndex: requireStreamIndex(t, "update wait"),
+ startIndex: requireStreamIndex(started.session, "update wait"),
});
const updateTurn = await updateLive.result();
updateTurn.expectOk();
diff --git a/e2e/fixtures/agent-background-tools/evals/empty-delivery-marker.eval.ts b/e2e/fixtures/agent-background-tools/evals/empty-delivery-marker.eval.ts
index fe943ecaf..f2079bae5 100644
--- a/e2e/fixtures/agent-background-tools/evals/empty-delivery-marker.eval.ts
+++ b/e2e/fixtures/agent-background-tools/evals/empty-delivery-marker.eval.ts
@@ -6,12 +6,13 @@ export default defineEval({
"Quoted delivery markers remain visible and in the next turn's context; standalone markers finish silently.",
async test(t) {
+ const session = await t.session();
const examplePrefix =
"Alice is documenting conditional delivery. Return exactly this example:\n";
for (const marker of ["<eve-empty-delivery/>", "&lt;eve-empty-delivery/&gt;"]) {
const message = `The pending-task instruction requires \`${marker}\` and no other text.`;
- const explanation = await t.send(`${examplePrefix}${message}`);
+ const explanation = await session.send(`${examplePrefix}${message}`);
explanation.expectOk();
await t.require(explanation.message, equals(message));
explanation.event("message.completed", {
@@ -22,11 +23,11 @@ export default defineEval({
data: (data) => data.message === null,
});
- const recalled = await t.send("Repeat your previous assistant response verbatim.");
+ const recalled = await session.send("Repeat your previous assistant response verbatim.");
recalled.expectOk();
await t.require(recalled.message, equals(message));
- const silent = await t.send(`${examplePrefix} \n${marker}\t `);
+ const silent = await session.send(`${examplePrefix} \n${marker}\t `);
silent.expectOk();
await t.require(silent.message, equals(undefined));
silent.event("message.completed", {
@@ -38,7 +39,7 @@ export default defineEval({
});
}
- const followUp = await t.send(`${examplePrefix}The documentation is ready.`);
+ const followUp = await session.send(`${examplePrefix}The documentation is ready.`);
followUp.expectOk();
await t.require(followUp.message, equals("The documentation is ready."));
t.noFailedActions();
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/boundary-hook-failure.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/boundary-hook-failure.eval.ts
index 24f256938..cf0d7660a 100644
--- a/e2e/fixtures/agent-basic-runtime/evals/runtime/boundary-hook-failure.eval.ts
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/boundary-hook-failure.eval.ts
@@ -4,7 +4,7 @@ export default defineEval({
description: "Boundary hook failures end one turn while the conversation remains resumable.",
async test(t) {
for (const boundary of ["turn.started", "step.started"]) {
- const session = t.newSession();
+ const session = await t.session();
const failed = await session.send("Deny this turn.", {
headers: { "x-e2e-deny-boundary": boundary },
});
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/client-context.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/client-context.eval.ts
index a1392f5d0..31d63a437 100644
--- a/e2e/fixtures/agent-basic-runtime/evals/runtime/client-context.eval.ts
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/client-context.eval.ts
@@ -19,9 +19,10 @@ export default defineEval({
const first = await t.send('Acknowledge this context with exactly "READY".', {
clientContext: [`The client context token is ${FIRST_CLIENT_CONTEXT_TOKEN}.`],
});
+ const session = first.session;
first.messageIncludes("READY");
- const second = await t.send(
+ const second = await session.send(
"Reply with every token beginning with clientctx- that is visible anywhere in your model input, one per line, and nothing else.",
{
clientContext: [`The client context token is ${SECOND_CLIENT_CONTEXT_TOKEN}.`],
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/frontend-stream-credential-refresh.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/frontend-stream-credential-refresh.eval.ts
new file mode 100644
index 000000000..ad4bd04d6
--- /dev/null
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/frontend-stream-credential-refresh.eval.ts
@@ -0,0 +1,67 @@
+import { EveAgentStore, defaultMessageReducer } from "eve/client";
+import { defineEval } from "eve/evals";
+import { equals } from "eve/evals/expect";
+
+export default defineEval({
+ description: "A frontend stream reconnect uses the latest send's authorization header.",
+ async test(t) {
+ const session = await t.session();
+ const host = `https://${crypto.randomUUID()}.invalid`;
+ const originalFetch = globalThis.fetch;
+ let resolveReconnectAuthorization!: (authorization: string | null) => void;
+ const reconnectAuthorization = new Promise<string | null>((resolve) => {
+ resolveReconnectAuthorization = resolve;
+ });
+ let disconnectStream: (() => void) | undefined;
+ const store = new EveAgentStore({
+ host,
+ initialSession: session.state,
+ reducer: defaultMessageReducer(),
+ });
+
+ globalThis.fetch = async (input, init) => {
+ const url = new URL(input instanceof Request ? input.url : String(input));
+ if (url.origin !== host) return originalFetch(input, init);
+ const response = await t.target.fetch(`${url.pathname}${url.search}`, init);
+ if (!url.pathname.endsWith("/stream") || !response.ok || response.body === null) {
+ return response;
+ }
+ if (disconnectStream !== undefined) {
+ resolveReconnectAuthorization(new Headers(init?.headers).get("authorization"));
+ return response;
+ }
+ const transport = new TransformStream<Uint8Array, Uint8Array>({
+ start(controller) {
+ disconnectStream = () => controller.terminate();
+ },
+ });
+ return new Response(response.body.pipeThrough(transport), response);
+ };
+
+ try {
+ await store.send({
+ message: "Reply with first.",
+ headers: { authorization: "Bearer old" },
+ signal: t.signal,
+ });
+ await t.require(store.snapshot.error, equals(undefined));
+ const firstTurn = await t.target.watchTurn(session.sessionId).result();
+ firstTurn.expectOk();
+
+ await store.send({
+ message: "Reply with second.",
+ headers: { authorization: "Bearer fresh" },
+ signal: t.signal,
+ });
+ await t.require(store.snapshot.error, equals(undefined));
+
+ if (disconnectStream === undefined)
+ throw new Error("The first turn never opened its stream.");
+ disconnectStream();
+ await t.require(await reconnectAuthorization, equals("Bearer fresh"));
+ } finally {
+ store.reset();
+ globalThis.fetch = originalFetch;
+ }
+ },
+});
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/image-attachment.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/image-attachment.eval.ts
index 19f6981ca..6ac68cd52 100644
--- a/e2e/fixtures/agent-basic-runtime/evals/runtime/image-attachment.eval.ts
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/image-attachment.eval.ts
@@ -17,10 +17,11 @@ export default defineEval({
description: "Session runtime smoke: attachments.",
async test(t) {
+ const session = await t.session();
// Eval modules execute from a build cache, so assets resolve against
// the app root (`eve eval` runs with the app as cwd), not import.meta.
const filePath = join(process.cwd(), "evals/assets/cat-image.png");
- const turn = await t.sendFile(
+ const turn = await session.sendFile(
"The attached fixture image depicts a cat. Reply with exactly: cat",
filePath,
"image/png",
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/multi-turn-session.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/multi-turn-session.eval.ts
index d4339ae63..b8354af67 100644
--- a/e2e/fixtures/agent-basic-runtime/evals/runtime/multi-turn-session.eval.ts
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/multi-turn-session.eval.ts
@@ -12,13 +12,19 @@ export default defineEval({
async test(t) {
const first = await t.send("My favorite word is marigold. Remember it.");
+ const session = first.session;
+ const independent = await t.send("Bob opened a separate chat. Greet him briefly.");
+ await t.require(independent.sessionId === first.sessionId, equals(false));
+ independent.event("session.started", { count: 1 });
+ independent.event("turn.started", { count: 1, data: { turnId: "turn_0" } });
- const second = await t.send("What is my favorite word? Reply with just the word.");
+ const second = await session.send("What is my favorite word? Reply with just the word.");
await t.require(second.sessionId, equals(first.sessionId));
+ await t.require(second.session === session, equals(true));
second.messageIncludes(/marigold/i);
- const cancel = await t.cancel();
+ const cancel = await session.cancel();
await t.require(
cancel,
satisfies(
@@ -28,7 +34,7 @@ export default defineEval({
),
);
- const third = await t.send(
+ const third = await session.send(
"Alice is checking the saved conversation. What favorite word did I ask you to remember? Reply with just that word.",
);
await t.require(third.sessionId, equals(first.sessionId));
@@ -39,7 +45,7 @@ export default defineEval({
t.succeeded();
t.messageIncludes(/marigold/i);
- t.check(t.transcript, includes("User:\nMy favorite word is marigold. Remember it."));
- t.check(t.transcript, includes("Assistant:\n"));
+ t.check(session.transcript, includes("User:\nMy favorite word is marigold. Remember it."));
+ t.check(session.transcript, includes("Assistant:\n"));
},
});
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/output-schema-turn.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/output-schema-turn.eval.ts
index 0d1b1d3ec..e9c27dc07 100644
--- a/e2e/fixtures/agent-basic-runtime/evals/runtime/output-schema-turn.eval.ts
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/output-schema-turn.eval.ts
@@ -21,9 +21,10 @@ export default defineEval({
type: "object",
},
});
+ const session = structured.session;
structured.expectOk();
- const plain = await t.send("Reply normally without structured output.");
+ const plain = await session.send("Reply normally without structured output.");
plain.expectOk();
plain.notEvent("result.completed");
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/session-prewarm.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/session-prewarm.eval.ts
new file mode 100644
index 000000000..7d9fe11c7
--- /dev/null
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/session-prewarm.eval.ts
@@ -0,0 +1,25 @@
+import { defineEval } from "eve/evals";
+import { equals } from "eve/evals/expect";
+
+export default defineEval({
+ description: "A prewarmed workflow initializes only when its first message arrives.",
+ async test(t) {
+ const session = await t.session();
+ const sessionId = session.sessionId;
+ const message = `Alice opened this chat while the session warmed up. Greet her briefly and include reference ${crypto.randomUUID()}.`;
+ const live = await session.start(message);
+ const observed = t.target.watchTurn(live.sessionId, { startIndex: 0 });
+ const [result, streamed] = await Promise.all([live.result(), observed.result()]);
+
+ result.expectOk();
+ await t.require(result.sessionId, equals(sessionId));
+ result.event("session.started", { count: 1 });
+ result.event("turn.started", { count: 1, data: { turnId: "turn_0" } });
+ result.event("message.received", { count: 1, data: { message, turnId: "turn_0" } });
+ result.event("step.started", { count: 1, data: { turnId: "turn_0" } });
+ await t.require(
+ streamed.events.map((event) => event.meta.id),
+ equals(result.events.map((event) => event.meta.id)),
+ );
+ },
+});
diff --git a/e2e/fixtures/agent-basic-runtime/evals/runtime/stale-client-cursor.eval.ts b/e2e/fixtures/agent-basic-runtime/evals/runtime/stale-client-cursor.eval.ts
index 8634ec117..4edaa1011 100644
--- a/e2e/fixtures/agent-basic-runtime/evals/runtime/stale-client-cursor.eval.ts
+++ b/e2e/fixtures/agent-basic-runtime/evals/runtime/stale-client-cursor.eval.ts
@@ -3,7 +3,7 @@ import { defineEval } from "eve/evals";
export default defineEval({
description: "A resumed client skips old turns when collecting a newly accepted message.",
async test(t) {
- const original = t.newSession();
+ const original = await t.session();
(await original.send("Reply with first-report-ready.")).expectOk();
(await original.send("Reply with second-report-ready.")).expectOk();
const sessionId = original.state?.sessionId;
diff --git a/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.ts b/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.ts
index 667967e5b..7a405a872 100644
--- a/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.ts
+++ b/e2e/fixtures/agent-cancellation/evals/cancellation/background-steering.eval.ts
@@ -23,11 +23,12 @@ export default cases.map(({ parentActive, steering, description }) =>
tags: ["real-model", "background-steering"],
timeoutMs: 240_000,
async test(t) {
+ const conversation = await t.session();
const memo = crypto.randomUUID();
const original = `WORKER-RESULT:ORIGINAL:${memo}`;
const steered = `WORKER-RESULT:STEERED:${memo}`;
const expected = steering ? steered : original;
- let parent = await t.start(
+ let parent = await conversation.start(
[
"Call steering-worker exactly once in the background with this exact message:",
`ASSIGNMENT ${memo}`,
diff --git a/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-subagent.eval.ts b/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-subagent.eval.ts
index 3b5278607..15ec7c48e 100644
--- a/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-subagent.eval.ts
+++ b/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-subagent.eval.ts
@@ -10,9 +10,10 @@ export default defineEval({
timeoutMs: 240_000,
async test(t) {
+ const session = await t.session();
// Explicit directive phrasing keeps the delegation deterministic so a
// scripted mock responder can drive this eval in the world suites.
- const parent = await t.start(
+ const parent = await session.start(
"Use the workflow tool exactly once to call the sleeper subagent with message 'Call the wait-for-cancellation tool exactly once and wait until this delegated turn is cancelled.' Return the sleeper result.",
);
const called = await parent.waitForEvent("subagent.called", {
@@ -52,7 +53,7 @@ export default defineEval({
parentTurn.notEvent("turn.failed");
parentTurn.notEvent("session.failed");
- const followUp = await t.send("Reply with exactly CANCELLATION-SUBAGENT-FOLLOW-UP-OK.");
+ const followUp = await session.send("Reply with exactly CANCELLATION-SUBAGENT-FOLLOW-UP-OK.");
followUp.expectOk();
followUp.notEvent("turn.cancelled");
followUp.messageIncludes(/CANCELLATION-SUBAGENT-FOLLOW-UP-OK/i);
@@ -61,7 +62,7 @@ export default defineEval({
// [Agents] listing as a parked "(cancelled)" handle. A handle leaked as
// `running` never re-enters the listing, so this catches the abandoned
// cancelled batch regressing to a permanent leak.
- const listing = await t.send(
+ const listing = await session.send(
"Look at the [Agents] listing in your context and reply with the sleeper agent's entry verbatim, including its status.",
);
listing.expectOk();
@@ -69,7 +70,7 @@ export default defineEval({
listing.messageIncludes(/sleeper/i);
listing.messageIncludes(/\(cancelled\)/);
- const resumed = await t.send(
+ const resumed = await session.send(
[
"Use the workflow tool exactly once.",
`In its JavaScript, call ctx.agent for sleeper with agentId ${JSON.stringify(agentId)} and message ${JSON.stringify(RECOVERY_REQUEST)}.`,
diff --git a/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-turn.eval.ts b/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-turn.eval.ts
index 617e146fb..b20a490b8 100644
--- a/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-turn.eval.ts
+++ b/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-turn.eval.ts
@@ -17,7 +17,8 @@ export default defineEval({
timeoutMs: 240_000,
async test(t) {
- const live = await t.start("Please wait for cancellation.");
+ const session = await t.session();
+ const live = await session.start("Please wait for cancellation.");
await live.waitForEvent("actions.requested", {
data: {
actions: (actions) =>
@@ -43,14 +44,14 @@ export default defineEval({
cancelledTurn.notEvent("step.failed");
cancelledTurn.notEvent("session.failed");
- const followUp = await t.send("Reply with exactly CANCELLATION-FOLLOW-UP-OK.");
+ const followUp = await session.send("Reply with exactly CANCELLATION-FOLLOW-UP-OK.");
followUp.expectOk();
followUp.notEvent("turn.cancelled");
followUp.notEvent("turn.failed");
followUp.notEvent("session.failed");
followUp.messageIncludes(/CANCELLATION-FOLLOW-UP-OK/i);
- const late = await t.cancel();
+ const late = await session.cancel();
await t.require(
late,
satisfies(
@@ -59,7 +60,7 @@ export default defineEval({
),
);
- const afterLateCancel = await t.send("Reply with exactly CANCELLATION-LATE-NOOP-OK.");
+ const afterLateCancel = await session.send("Reply with exactly CANCELLATION-LATE-NOOP-OK.");
afterLateCancel.expectOk();
afterLateCancel.notEvent("turn.cancelled");
afterLateCancel.notEvent("turn.failed");
diff --git a/e2e/fixtures/agent-cancellation/evals/cancellation/generated-program-child-hitl.eval.ts b/e2e/fixtures/agent-cancellation/evals/cancellation/generated-program-child-hitl.eval.ts
index 444dd4efb..a31fa4c57 100644
--- a/e2e/fixtures/agent-cancellation/evals/cancellation/generated-program-child-hitl.eval.ts
+++ b/e2e/fixtures/agent-cancellation/evals/cancellation/generated-program-child-hitl.eval.ts
@@ -4,16 +4,16 @@ export default defineEval({
description: "A generated workflow program proxies a child question through the existing owner.",
timeoutMs: 120_000,
async test(t) {
- await t.send(
+ const { session } = await t.send(
"Use the workflow tool exactly once to call the sleeper subagent with the message GENERATED-PROGRAM-CHILD-HITL. Return the child result.",
);
- const request = t.requireInputRequest({
+ const request = session.requireInputRequest({
prompt: "What marker should the child return?",
toolName: "ask_question",
});
if (request.kind !== "question") throw new Error("Child input request is not a question.");
- const resumed = await t.respond([
+ const resumed = await session.respond([
{ requestId: request.requestId, text: "GENERATED-HITL-MARKER" },
]);
resumed.expectOk();
diff --git a/e2e/fixtures/agent-cancellation/evals/cancellation/steer-pending-generation.eval.ts b/e2e/fixtures/agent-cancellation/evals/cancellation/steer-pending-generation.eval.ts
index 38e3eb20b..7ee50f5a7 100644
--- a/e2e/fixtures/agent-cancellation/evals/cancellation/steer-pending-generation.eval.ts
+++ b/e2e/fixtures/agent-cancellation/evals/cancellation/steer-pending-generation.eval.ts
@@ -9,11 +9,17 @@ export default defineEval({
if (process.env.EVE_E2E_MODEL !== "mock") {
t.skip("Requires the deterministic model's pending-generation gate.");
}
- const live = await t.start("Alice is preparing the 2026 report.");
- await live.waitForEvent("step.started");
+ const conversation = await t.session();
+ const live = await conversation.start("Alice is preparing the 2026 report.");
+ const started = await live.waitForEvent("step.started");
+ const observedAt = Date.now();
+ t.log(
+ `Pending generation observed ${observedAt - Date.parse(started.meta.at)} ms after step.started.`,
+ );
const correction = await live.session.start("Alice corrected the report year to 2025.", {
turnPolicy: "steer",
});
+ t.log(`Steering accepted ${Date.now() - observedAt} ms after observing pending generation.`);
const result = await live.result();
await correction.result();
result.event("turn.started", { count: 1 });
diff --git a/e2e/fixtures/agent-compaction-envelope/evals/envelope-growth.eval.ts b/e2e/fixtures/agent-compaction-envelope/evals/envelope-growth.eval.ts
index 7a71b59f0..d7c1d4399 100644
--- a/e2e/fixtures/agent-compaction-envelope/evals/envelope-growth.eval.ts
+++ b/e2e/fixtures/agent-compaction-envelope/evals/envelope-growth.eval.ts
@@ -4,14 +4,15 @@ export default defineEval({
description:
"Growing instructions and tool schemas trigger compaction above prior provider usage.",
async test(t) {
+ const session = await t.session();
for (let index = 0; index < 6; index += 1) {
- const seed = await t.send(
+ const seed = await session.send(
`Seed ${index}: ${"Preserve this repository evidence. ".repeat(40)}`,
);
seed.expectOk();
seed.event("compaction.completed", { count: 0 });
}
- const expanded = await t.send(
+ const expanded = await session.send(
"[expand-envelope] Verify that the expanded request was compacted.",
);
expanded.expectOk();
diff --git a/e2e/fixtures/agent-compaction-measured/evals/measured-tokens.eval.ts b/e2e/fixtures/agent-compaction-measured/evals/measured-tokens.eval.ts
index 001712ba2..565831a4f 100644
--- a/e2e/fixtures/agent-compaction-measured/evals/measured-tokens.eval.ts
+++ b/e2e/fixtures/agent-compaction-measured/evals/measured-tokens.eval.ts
@@ -6,14 +6,15 @@ export default defineEval({
description:
"Provider-measured context pressure produces a summary even when the character estimate is small.",
async test(t) {
+ const session = await t.session();
for (let index = 0; index < 6; index += 1) {
- const seed = await t.send(`${MEASURED_TOKENS_CASE} record evidence ${index}.`);
+ const seed = await session.send(`${MEASURED_TOKENS_CASE} record evidence ${index}.`);
seed.expectOk();
seed.notEvent("compaction.completed");
}
- const measured = await t.send(`${MEASURED_TOKENS_CASE} report high usage.`);
+ const measured = await session.send(`${MEASURED_TOKENS_CASE} report high usage.`);
measured.expectOk();
- const compacted = await t.send(`${MEASURED_TOKENS_CASE} verify the checkpoint.`);
+ const compacted = await session.send(`${MEASURED_TOKENS_CASE} verify the checkpoint.`);
compacted.expectOk();
compacted.event("compaction.completed");
compacted.messageIncludes("MEASURED_COMPACTION_OK");
diff --git a/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/cancelled-first-turn.eval.ts b/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/cancelled-first-turn.eval.ts
index cc35f7a5e..f03eb2432 100644
--- a/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/cancelled-first-turn.eval.ts
+++ b/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/cancelled-first-turn.eval.ts
@@ -14,7 +14,8 @@ export default defineEval({
timeoutMs: 240_000,
async test(t) {
- const live = await t.start(
+ const session = await t.session();
+ const live = await session.start(
"Call the wait-for-cancellation tool and wait until this turn is cancelled.",
);
await live.waitForEvent("actions.requested", {
@@ -40,7 +41,7 @@ export default defineEval({
cancelledTurn.notEvent("turn.failed");
cancelledTurn.notEvent("session.failed");
- const resumed = await t.send(
+ const resumed = await session.send(
'Reply with exactly the text "session model after cancellation" and nothing else.',
);
resumed.expectOk();
diff --git a/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/session-selection.eval.ts b/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/session-selection.eval.ts
index b20c77889..2bc63a438 100644
--- a/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/session-selection.eval.ts
+++ b/e2e/fixtures/agent-dynamic-model-session/evals/dynamic-model/session-selection.eval.ts
@@ -13,13 +13,14 @@ export default defineEval({
const first = await t.send(
'Reply with exactly the text "session model turn one" and nothing else.',
);
+ const session = first.session;
first.expectOk();
first.messageIncludes("session model turn one");
first.eventsSatisfy("the session resolver selects the configured model", (events) =>
events.some((event) => event.type === "step.started" && event.data.modelId === selectedModel),
);
- const second = await t.send(
+ const second = await session.send(
'Reply with exactly the text "session model turn two" and nothing else.',
);
second.expectOk();
diff --git a/e2e/fixtures/agent-evaluate/evals/parallel-children.eval.ts b/e2e/fixtures/agent-evaluate/evals/parallel-children.eval.ts
index f69639703..9e1de0273 100644
--- a/e2e/fixtures/agent-evaluate/evals/parallel-children.eval.ts
+++ b/e2e/fixtures/agent-evaluate/evals/parallel-children.eval.ts
@@ -7,7 +7,7 @@ export default defineEval({
"Alice and Bob need parallel investigations assigned to the worker.",
);
first.expectOk();
- let session: Pick<EveEvalSession, "sessionId" | "state"> = t;
+ let session: Pick<EveEvalSession, "sessionId" | "state"> = first.session;
let combined = first.message ?? "";
for (let attempt = 0; attempt < 6; attempt++) {
if (
diff --git a/e2e/fixtures/agent-evaluate/evals/routing.eval.ts b/e2e/fixtures/agent-evaluate/evals/routing.eval.ts
index 77cba118c..97ccf3577 100644
--- a/e2e/fixtures/agent-evaluate/evals/routing.eval.ts
+++ b/e2e/fixtures/agent-evaluate/evals/routing.eval.ts
@@ -9,7 +9,9 @@ export default defineEval({
first.messageIncludes('"reasoning":"low"');
first.messageIncludes('"requests":1');
first.usedNoTools();
- const second = await t.send("Bob now needs a difficult investigation of that incident.");
+ const second = await first.session.send(
+ "Bob now needs a difficult investigation of that incident.",
+ );
second.expectOk();
second.usedNoTools();
second.messageIncludes('"model":"openai/large"');
diff --git a/e2e/fixtures/agent-memory/evals/memory/custom-provider.eval.ts b/e2e/fixtures/agent-memory/evals/memory/custom-provider.eval.ts
index b6308d3fa..1bf4b6c74 100644
--- a/e2e/fixtures/agent-memory/evals/memory/custom-provider.eval.ts
+++ b/e2e/fixtures/agent-memory/evals/memory/custom-provider.eval.ts
@@ -6,11 +6,12 @@ export default defineEval({
const update = await t.send(
"Update the profile memory to NEW_PROFILE_VALUE, then confirm the update.",
);
+ const session = update.session;
update.expectOk();
update.calledTool("profile__save", { count: 1 });
update.messageIncludes("MEMORY_TOOL_UPDATED");
- const recalled = await t.send(
+ const recalled = await session.send(
"Report the current profile memory exactly as instructed, without calling a tool.",
);
recalled.expectOk();
diff --git a/e2e/fixtures/agent-memory/evals/memory/file-provider.eval.ts b/e2e/fixtures/agent-memory/evals/memory/file-provider.eval.ts
index 42a9a81ec..6ce2553c0 100644
--- a/e2e/fixtures/agent-memory/evals/memory/file-provider.eval.ts
+++ b/e2e/fixtures/agent-memory/evals/memory/file-provider.eval.ts
@@ -18,7 +18,7 @@ export default defineEval({
});
saved.messageIncludes("FILE_MEMORY_SAVED");
- const nextSession = t.newSession();
+ const nextSession = await t.session();
const recalled = await nextSession.send(
"What is the verification phrase in file memory? Reply with the phrase only. Do not call tools.",
);
diff --git a/e2e/fixtures/agent-model/evals/dynamic-model/catalog-cache.eval.ts b/e2e/fixtures/agent-model/evals/dynamic-model/catalog-cache.eval.ts
index afe2955fe..0985ad1bb 100644
--- a/e2e/fixtures/agent-model/evals/dynamic-model/catalog-cache.eval.ts
+++ b/e2e/fixtures/agent-model/evals/dynamic-model/catalog-cache.eval.ts
@@ -9,10 +9,11 @@ export default defineEval({
const first = await t.send(
'[model: catalog] Reply with exactly the text "catalog one" and nothing else.',
);
+ const session = first.session;
first.expectOk();
first.messageIncludes("catalog one");
- const second = await t.send(
+ const second = await session.send(
'[model: catalog] Reply with exactly the text "catalog two" and nothing else.',
);
second.expectOk();
diff --git a/e2e/fixtures/agent-model/evals/dynamic-model/turn-selection.eval.ts b/e2e/fixtures/agent-model/evals/dynamic-model/turn-selection.eval.ts
index dc36f5190..af3f7901f 100644
--- a/e2e/fixtures/agent-model/evals/dynamic-model/turn-selection.eval.ts
+++ b/e2e/fixtures/agent-model/evals/dynamic-model/turn-selection.eval.ts
@@ -10,10 +10,11 @@ export default defineEval({
const selected = await t.send(
'[model: mini] Reply with exactly the text "mini ping" and nothing else.',
);
+ const session = selected.session;
selected.expectOk();
selected.messageIncludes("mini ping");
- const selectedAgain = await t.send(
+ const selectedAgain = await session.send(
'Reply with exactly the text "selected again" and nothing else.',
);
selectedAgain.expectOk();
diff --git a/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger-approval.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger-approval.eval.ts
index 1a1471c50..d331a3bdb 100644
--- a/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger-approval.eval.ts
+++ b/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger-approval.eval.ts
@@ -17,16 +17,17 @@ export default defineEval({
"After the tool runs, reply with the exact words `inventory received` if the tool result contains inventory counts.",
].join("\n"),
);
+ const session = parked.session;
parked.expectOk();
- t.requireInputRequest({
+ session.requireInputRequest({
display: "confirmation",
optionIds: ["approve", "cancel"],
toolName: PETSTORE_APPROVAL_INVENTORY_TOOL,
});
parked.calledTool(PETSTORE_APPROVAL_INVENTORY_TOOL, { status: "pending", count: 1 });
- const approved = await t.respondAll("approve");
+ const approved = await session.respondAll("approve");
approved.expectOk();
approved.event("action.result", {
diff --git a/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.ts b/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.ts
index b4167fdf4..5e2370875 100644
--- a/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.ts
+++ b/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.ts
@@ -15,8 +15,9 @@ export default ["first", "later"].map((launchTurn) =>
tags: ["real-model"],
description: `Real provider cache hits survive five parallel reviews (${launchTurn} turn).`,
async test(t) {
+ const session = await t.session();
if (launchTurn === "later") {
- const planning = await t.send(
+ const planning = await session.send(
"Alice is planning a community centre event with eight workshops and twelve places per workshop. " +
"Please calculate the total number of places. Bob is preparing the purchasing sheets and will send them next for review.",
);
@@ -25,7 +26,7 @@ export default ["first", "later"].map((launchTurn) =>
planning.notEvent("actions.requested");
}
- const started = await t.send(
+ const started = await session.send(
launchTurn === "later"
? `Bob has the purchasing sheets ready for the event we just discussed.\n\n${reviewPacket()}`
: reviewPacket(),
@@ -61,7 +62,7 @@ function expectFiveReviewers(started: EveEvalTurn) {
async function waitForReviews(t: EveEvalContext, started: EveEvalTurn, taskIds: string[]) {
const turns = [started];
- let cursor = t.state?.streamIndex;
+ let cursor = started.session.state.streamIndex;
for (let attempt = 0; attempt < 10 && !allCompleted(turns, taskIds); attempt += 1) {
assert(cursor !== undefined, "parent stream cursor is present");
const live = t.target.watchTurn(started.sessionId, { startIndex: cursor });
diff --git a/e2e/fixtures/agent-self-modification/evals/self-modification/harness.ts b/e2e/fixtures/agent-self-modification/evals/self-modification/harness.ts
index 43bb3b8a7..563ac6f45 100644
--- a/e2e/fixtures/agent-self-modification/evals/self-modification/harness.ts
+++ b/e2e/fixtures/agent-self-modification/evals/self-modification/harness.ts
@@ -128,38 +128,47 @@ export class SelfModificationHarness {
async request(
prompt: string,
- session: Pick<EveEvalSession, "start"> = this.#t,
+ session?: Pick<EveEvalSession, "start">,
): Promise<SelfModificationRun> {
- const liveParent = await session.start(prompt);
+ const liveParent = await (session ?? (await this.#t.session())).start(prompt);
this.#turns.add(liveParent);
+ const parent = await liveParent.result();
+ parent.expectOk();
+ const call = parent.requireToolCall(SELF_MODIFICATION_AGENT);
+ const agentId = typeof call.input.agentId === "string" ? call.input.agentId : undefined;
+ const message = agentId === undefined ? undefined : call.input.message;
+ if (agentId !== undefined && (typeof message !== "string" || message.length === 0))
+ throw new Error("Self-modification continuation omitted its message.");
+
+ let called = [...this.#turns]
+ .filter((turn) => turn.sessionId === parent.sessionId)
+ .flatMap((turn) => turn.events)
+ .find(
+ (event) =>
+ event.type === "subagent.called" &&
+ event.data.name === SELF_MODIFICATION_AGENT &&
+ (agentId === undefined
+ ? liveParent.events.includes(event)
+ : event.data.agentId === agentId),
+ );
let continuation: EveEvalLiveTurn | undefined;
- const called = await liveParent
- .waitForEvent("subagent.called", {
- data: { name: SELF_MODIFICATION_AGENT },
- })
- .catch(async (error) => {
- this.#t.signal.throwIfAborted();
- const parent = await liveParent.result();
- parent.expectOk();
- parent.requireToolCall(SELF_MODIFICATION_AGENT);
- const startIndex = liveParent.session.state?.streamIndex;
- if (startIndex === undefined) throw error;
- continuation = this.#t.target.watchTurn(parent.sessionId, { startIndex });
- this.#turns.add(continuation);
- return continuation.waitForEvent("subagent.called", {
- data: { name: SELF_MODIFICATION_AGENT },
- });
+ if (called?.type !== "subagent.called") {
+ continuation = this.#t.target.watchTurn(parent.sessionId, {
+ startIndex: liveParent.session.state.streamIndex,
});
- const liveChild = this.#t.target.watchTurn(called.data.childSessionId);
- this.#turns.add(liveChild);
- const [parent, child] = await Promise.all([
- liveParent.result(),
- liveChild.result(),
+ this.#turns.add(continuation);
+ const data: { name: string; agentId?: string } = { name: SELF_MODIFICATION_AGENT };
+ if (agentId !== undefined) data.agentId = agentId;
+ called = await continuation.waitForEvent("subagent.called", { data });
+ }
+ const [child] = await Promise.all([
+ this.#readChild(
+ called.data.childSessionId,
+ typeof message === "string" ? message : undefined,
+ ),
continuation?.result().then((turn) => turn.expectOk()),
]);
- parent.expectOk();
this.#t.calledSubagent(SELF_MODIFICATION_AGENT);
- child.expectOk();
return { child, parent, session: liveParent.session };
}
@@ -168,8 +177,8 @@ export class SelfModificationHarness {
}
/** Uses a fresh conversation so the model cannot answer from the authoring exchange alone. */
- verify(prompt: string): Promise<EveEvalTurn> {
- return this.#runTurn(this.#t.newSession(), prompt);
+ async verify(prompt: string): Promise<EveEvalTurn> {
+ return this.#runTurn(await this.#t.session(), prompt);
}
async apply(): Promise<void> {
@@ -255,6 +264,31 @@ export class SelfModificationHarness {
await rm(this.#backupRoot, { recursive: true, force: true });
}
+ async #readChild(sessionId: string, message?: string): Promise<EveEvalTurn> {
+ let startIndex =
+ [...this.#turns].reverse().find((turn) => turn.sessionId === sessionId)?.session.state
+ .streamIndex ?? 0;
+ for (let turns = 0; turns < 16; turns++) {
+ this.#t.signal.throwIfAborted();
+ const live = this.#t.target.watchTurn(sessionId, { startIndex });
+ this.#turns.add(live);
+ const turn = await live.result();
+ turn.expectOk();
+ // A reused child may have unseen diagnostic turns. Wait for the turn
+ // that actually received this delegation, including mid-turn steering.
+ if (
+ message === undefined ||
+ turn.events.some(
+ (event) => event.type === "message.received" && event.data.message?.includes(message),
+ )
+ )
+ return turn;
+ if (turn.status !== "waiting") break;
+ startIndex = live.session.state.streamIndex;
+ }
+ throw new Error("Self-modification child did not receive the delegated message.");
+ }
+
async #runTurn(session: EveEvalSession, prompt: string): Promise<EveEvalTurn> {
const live = await session.start(prompt);
this.#turns.add(live);
diff --git a/e2e/fixtures/agent-self-modification/evals/self-modification/offer-repair.eval.ts b/e2e/fixtures/agent-self-modification/evals/self-modification/offer-repair.eval.ts
index 425799e0b..c92094676 100644
--- a/e2e/fixtures/agent-self-modification/evals/self-modification/offer-repair.eval.ts
+++ b/e2e/fixtures/agent-self-modification/evals/self-modification/offer-repair.eval.ts
@@ -32,6 +32,7 @@ export default defineEval({
const authored = await selfMod.request(
[
`Alice needs a reusable ${TOOL_NAME} action for inventory planning in future conversations.`,
+ `Keep this change scoped to ${TOOL_PATH}; verify it through tool calls without adding other files.`,
"Accept non-negative integer onHand, incoming, and target quantities, each no greater than 1000000.",
"Return reorderUnits as the target minus both on-hand and incoming stock, floored at zero.",
"This action only calculates a recommendation; it must not place an order or contact external services.",
@@ -70,7 +71,10 @@ export default defineEval({
{ on: offered.message },
);
- await selfMod.request("Yes, please do.", authored.session);
+ await selfMod.request(
+ `Yes, please repair ${TOOL_PATH}. Keep this repair scoped to that file and verify it through tool calls without adding other files.`,
+ authored.session,
+ );
await selfMod.assertOnlyChanged([TOOL_PATH]);
await selfMod.apply();
diff --git a/e2e/fixtures/agent-self-modification/test/harness.scenario.test.mjs b/e2e/fixtures/agent-self-modification/test/harness.scenario.test.mjs
index c79fff56a..16e5d84d5 100644
--- a/e2e/fixtures/agent-self-modification/test/harness.scenario.test.mjs
+++ b/e2e/fixtures/agent-self-modification/test/harness.scenario.test.mjs
@@ -40,11 +40,8 @@ function context(target, signal = new AbortController().signal) {
target,
log() {},
calledSubagent() {},
- start: async () => {
- throw new Error("start was not expected");
- },
- newSession: () => {
- throw new Error("newSession was not expected");
+ session: () => {
+ throw new Error("session was not expected");
},
};
}
@@ -55,7 +52,9 @@ function completedTurn(sessionId, events = []) {
events,
expectOk() {},
calledSubagent() {},
- requireToolCall() {},
+ requireToolCall() {
+ return { input: {} };
+ },
};
}
@@ -128,7 +127,7 @@ test("close retires every session before restoring the complete source tree", as
};
parent.waitForEvent = async () => ({ data: parentEvent.data });
target.watchTurn = () => {
- assert.equal(parentFinished, false);
+ assert.equal(parentFinished, true);
return child;
};
@@ -177,8 +176,8 @@ test("close retires every session before restoring the complete source tree", as
{},
(target) => {
const t = context(target);
- t.start = async () => parent;
- t.newSession = () => ({ start: async () => verification });
+ let creations = 0;
+ t.session = async () => ({ start: async () => (++creations === 1 ? parent : verification) });
return t;
},
);
@@ -216,7 +215,7 @@ test("one reset failure still retires other sessions and leaves unsafe source un
},
});
const t = context(target);
- t.start = async () => liveParent;
+ t.session = async () => ({ start: async () => liveParent });
const populated = await SelfModificationHarness.create(t, root);
await populated.request("mutate");
await writeFile(join(root, "keep.txt"), "unsafe mutation");
@@ -359,7 +358,10 @@ test("request falls back to a parent-boundary watch when the initial event is mi
sessionId: "parent",
events: [],
async waitForEvent() {
- return { data: { name: "self-modification", childSessionId: "child" } };
+ return {
+ type: "subagent.called",
+ data: { name: "self-modification", childSessionId: "child" },
+ };
},
async result() {
return completedTurn("parent");
@@ -385,13 +387,66 @@ test("request falls back to a parent-boundary watch when the initial event is mi
},
};
const t = context(target);
- t.start = async () => parent;
+ t.session = async () => ({ start: async () => parent });
const harness = await SelfModificationHarness.create(t, root);
await harness.request("make a change");
await harness.close();
await rm(root, { recursive: true, force: true });
});
+for (const emitsCalled of [false, true]) {
+ test(`request follows a reused agent past stale turns (new called event: ${emitsCalled})`, async () => {
+ const called = {
+ type: "subagent.called",
+ data: { name: "self-modification", agentId: "agent-1", childSessionId: "child" },
+ };
+ const initialParent = liveTurn("parent", [called]);
+ initialParent.waitForEvent = async () => called;
+ const initialChild = liveTurn("child");
+ initialChild.session = { state: { streamIndex: 10 } };
+ const message = "Repair the existing inventory tool.";
+ const repairParent = liveTurn("parent", emitsCalled ? [called] : []);
+ repairParent.waitForEvent = async () => {
+ throw new Error("Session reached session.waiting before the expected event.");
+ };
+ repairParent.result = async () => ({
+ ...completedTurn("parent"),
+ requireToolCall: () => ({ input: { agentId: "agent-1", message } }),
+ });
+ const diagnostic = liveTurn("child", [
+ { type: "message.received", data: { message: "Diagnose only." } },
+ ]);
+ diagnostic.session = { state: { streamIndex: 20 } };
+ diagnostic.result = async () => ({
+ ...completedTurn("child", diagnostic.events),
+ status: "waiting",
+ });
+ const repaired = liveTurn("child", [
+ ...diagnostic.events,
+ { type: "message.received", data: { message } },
+ ]);
+ repaired.session = { state: { streamIndex: 30 } };
+ const observed = [];
+ await withHarness(async ({ harness, target }) => {
+ const children = [initialChild, diagnostic, repaired];
+ target.watchTurn = (sessionId, options) => {
+ observed.push({ sessionId, ...options });
+ return children.shift();
+ };
+ await harness.request("Create the tool.", { start: async () => initialParent });
+ const result = await harness.request("Please repair it.", {
+ start: async () => repairParent,
+ });
+ assert.deepEqual(result.child.events, repaired.events);
+ assert.deepEqual(observed, [
+ { sessionId: "child", startIndex: 0 },
+ { sessionId: "child", startIndex: 10 },
+ { sessionId: "child", startIndex: 20 },
+ ]);
+ });
+ });
+}
+
test("cleanup failure retains a lock that identifies the source backup", async (t) => {
const root = await temporaryCheckout(t, "eve-selfmod-cleanup-failed-");
await writeFile(join(root, "agent", "instructions.md"), "baseline");
diff --git a/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-approve.eval.ts b/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-approve.eval.ts
index 246bf12fd..135a9a598 100644
--- a/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-approve.eval.ts
+++ b/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-approve.eval.ts
@@ -11,7 +11,8 @@ export default defineEval({
"Messages queued during an active turn preserve one limit prompt, and approve resets the budget.",
timeoutMs: 90_000,
async test(t) {
- const active = await t.start(
+ const conversation = await t.session();
+ const active = await conversation.start(
'Call the `hold-open` tool exactly once with marker "limit-race". Wait for its result, then reply with exactly "approved".',
);
await active.waitForEvent("actions.requested");
@@ -35,13 +36,13 @@ export default defineEval({
const prompted = await active.result();
prompted.event("input.requested", { count: 1 });
- const request = t.requireInputRequest({
+ const request = conversation.requireInputRequest({
display: "confirmation",
optionIds: ["continue", "stop"],
toolName: "session_limit_continuation",
});
- const activeState = t.state;
+ const activeState = conversation.state;
if (activeState === undefined) {
throw new Error("The active eval session did not expose client state.");
}
diff --git a/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-continuation.eval.ts b/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-continuation.eval.ts
index 0f00b44e5..d67e07db7 100644
--- a/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-continuation.eval.ts
+++ b/e2e/fixtures/agent-session-limits/evals/limits/session-token-limit-continuation.eval.ts
@@ -16,23 +16,24 @@ export default defineEval({
// The 1-token budget lets this first call finish (limits are checked
// before the next call) but leaves the session over its input limit.
const first = await t.send('Reply with exactly the text "first ping" and nothing else.');
+ const session = first.session;
first.expectOk();
// The next turn must park on the harness-authored prompt before any
// model call happens.
- await t.send('Reply with exactly the text "limit pong" and nothing else.');
- const request = t.requireInputRequest({
+ await session.send('Reply with exactly the text "limit pong" and nothing else.');
+ const request = session.requireInputRequest({
display: "confirmation",
optionIds: ["continue", "stop"],
toolName: "session_limit_continuation",
});
- const resumed = await t.respond([{ optionId: "continue", requestId: request.requestId }]);
+ const resumed = await session.respond([{ optionId: "continue", requestId: request.requestId }]);
resumed.expectOk();
t.succeeded();
t.messageIncludes("limit pong");
- const stopSession = t.newSession();
+ const stopSession = await t.session();
const stopFirst = await stopSession.send(
'Reply with exactly the text "stop ping" and nothing else.',
);
diff --git a/e2e/fixtures/agent-skills/evals/skills/prompt-prefix.eval.ts b/e2e/fixtures/agent-skills/evals/skills/prompt-prefix.eval.ts
index 5473715df..3c27e5415 100644
--- a/e2e/fixtures/agent-skills/evals/skills/prompt-prefix.eval.ts
+++ b/e2e/fixtures/agent-skills/evals/skills/prompt-prefix.eval.ts
@@ -5,8 +5,9 @@ export default defineEval({
description: "Dynamic skill announcements preserve the prompt prefix across durable tool steps.",
async test(t) {
const first = await t.send("Say ready.");
+ const session = first.session;
first.expectOk();
- const checked = await t.send(PREFIX_REQUEST);
+ const checked = await session.send(PREFIX_REQUEST);
checked.expectOk();
checked.calledTool("capture_prompt");
checked.messageIncludes("prompt-prefix-ok");
diff --git a/e2e/fixtures/agent-subagents-hitl/evals/hitl.eval.ts b/e2e/fixtures/agent-subagents-hitl/evals/hitl.eval.ts
index fcd293128..08f301a07 100644
--- a/e2e/fixtures/agent-subagents-hitl/evals/hitl.eval.ts
+++ b/e2e/fixtures/agent-subagents-hitl/evals/hitl.eval.ts
@@ -34,7 +34,7 @@ export default defineEval({
// Background delegation returns its receipt first. The child's approval
// then wakes the parent in a separate server-initiated turn.
- const blocked = await waitForInput(t, t, "get_stock_price");
+ const blocked = await waitForInput(t, started.session, "get_stock_price");
const resumed = await blocked.respondAll("approve");
t.check(resumed.inputRequests, equals([]));
resumed.noFailedActions();
diff --git a/e2e/fixtures/agent-subagents-hitl/evals/mixed-runtime-action-approval.eval.ts b/e2e/fixtures/agent-subagents-hitl/evals/mixed-runtime-action-approval.eval.ts
index 18cf7279e..eaa633a49 100644
--- a/e2e/fixtures/agent-subagents-hitl/evals/mixed-runtime-action-approval.eval.ts
+++ b/e2e/fixtures/agent-subagents-hitl/evals/mixed-runtime-action-approval.eval.ts
@@ -20,6 +20,7 @@ export default defineEval({
`After both results arrive, reply with exactly ${COLLISION_MARKER}.`,
].join("\n"),
);
+ const session = parked.session;
parked.calledTool("collision-gate", { count: 1, status: "pending" });
parked.calledSubagent("collision-child", { count: 1, status: "completed" });
@@ -29,13 +30,13 @@ export default defineEval({
{ type: "input.requested" },
{ type: "session.waiting" },
]);
- t.requireInputRequest({ display: "confirmation", toolName: "collision-gate" });
+ session.requireInputRequest({ display: "confirmation", toolName: "collision-gate" });
- const resumed = await t.respondAll("approve");
+ const resumed = await session.respondAll("approve");
resumed.expectOk();
const completed = resumed.message?.includes(COLLISION_MARKER)
? resumed
- : await waitForMessage(t, t, COLLISION_MARKER);
+ : await waitForMessage(t, parked.session, COLLISION_MARKER);
completed.messageIncludes(COLLISION_MARKER);
t.succeeded();
diff --git a/e2e/fixtures/agent-subagents/evals/agent-messaging-local.eval.ts b/e2e/fixtures/agent-subagents/evals/agent-messaging-local.eval.ts
index f28c0eb71..262c813eb 100644
--- a/e2e/fixtures/agent-subagents/evals/agent-messaging-local.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/agent-messaging-local.eval.ts
@@ -23,7 +23,7 @@ export default defineEval({
started.expectOk();
const firstCompletion = t.target.watchTurn(started.sessionId, {
- startIndex: requireStreamIndex(t),
+ startIndex: requireStreamIndex(started.session),
});
const firstCompletedTurn = await firstCompletion.result();
firstCompletedTurn.expectOk();
diff --git a/e2e/fixtures/agent-subagents/evals/agent-messaging-remote.eval.ts b/e2e/fixtures/agent-subagents/evals/agent-messaging-remote.eval.ts
index 3cfcb880a..fbc620879 100644
--- a/e2e/fixtures/agent-subagents/evals/agent-messaging-remote.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/agent-messaging-remote.eval.ts
@@ -22,7 +22,7 @@ export default defineEval({
);
started.expectOk();
const firstCompletion = t.target.watchTurn(started.sessionId, {
- startIndex: requireStreamIndex(t),
+ startIndex: requireStreamIndex(started.session),
});
const firstCompletedTurn = await firstCompletion.result();
firstCompletedTurn.expectOk();
diff --git a/e2e/fixtures/agent-subagents/evals/descendant-session-limit.eval.ts b/e2e/fixtures/agent-subagents/evals/descendant-session-limit.eval.ts
index 497a1639e..69e87bf6e 100644
--- a/e2e/fixtures/agent-subagents/evals/descendant-session-limit.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/descendant-session-limit.eval.ts
@@ -21,14 +21,14 @@ export default defineEval({
"A descendant session-limit prompt reaches the root; continue resumes the child and stop leaves the root session reusable.",
timeoutMs: 90_000,
async test(t) {
- await t.send(DELEGATE_PROMPT);
- const continueSession = await waitForInput(t, t);
+ const { session } = await t.send(DELEGATE_PROMPT);
+ const continueSession = await waitForInput(t, session);
const continueRequest = continueSession.requireInputRequest({
display: "confirmation",
optionIds: ["continue", "stop"],
toolName: "session_limit_continuation",
});
- const rootSessionId = t.sessionId;
+ const rootSessionId = session.sessionId;
if (rootSessionId === undefined) {
throw new Error("The root session did not expose its session id.");
}
@@ -54,7 +54,7 @@ export default defineEval({
completed.messageIncludes(CHILD_TOKEN);
t.noFailedActions();
- const stopSession = t.newSession();
+ const stopSession = await t.session();
await stopSession.send(DELEGATE_PROMPT);
const blockedStopSession = await waitForInput(t, stopSession);
const stopRequest = blockedStopSession.requireInputRequest({
diff --git a/e2e/fixtures/agent-subagents/evals/dynamic-subagent.eval.ts b/e2e/fixtures/agent-subagents/evals/dynamic-subagent.eval.ts
index b5a18b7fd..9bb516702 100644
--- a/e2e/fixtures/agent-subagents/evals/dynamic-subagent.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/dynamic-subagent.eval.ts
@@ -10,7 +10,7 @@ export default defineEval({
count: 1,
data: { backgroundTask: { status: "working" }, subagentName: "conditional-marker" },
});
- const completed = await waitForMessage(t, t, "DYNAMIC_SUBAGENT_ENABLED");
+ const completed = await waitForMessage(t, selected.session, "DYNAMIC_SUBAGENT_ENABLED");
const omitted = await completed.send("Call omitted-marker exactly once.");
diff --git a/e2e/fixtures/agent-subagents/evals/dynamic-workflow.eval.ts b/e2e/fixtures/agent-subagents/evals/dynamic-workflow.eval.ts
index cbe176827..769f6b5f8 100644
--- a/e2e/fixtures/agent-subagents/evals/dynamic-workflow.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/dynamic-workflow.eval.ts
@@ -21,7 +21,8 @@ export default defineEval({
description:
"Generated workflow-program smoke: model-authored JavaScript fans out two local subagent calls and combines their results.",
async test(t) {
- const parent = await t.start(
+ const session = await t.session();
+ const parent = await session.start(
"Use the workflow tool exactly once to fan out two independent echo-marker subagent calls. In its JavaScript, create the messages 'workflow alpha' and 'workflow beta', map them through ctx.agent calls to echo-marker inside Promise.all, and return the resulting two-element array. Do not call echo-marker outside workflow. Then reply with the returned array verbatim as JSON.",
);
const firstCalled = await parent.waitForEvent("subagent.called", {
diff --git a/e2e/fixtures/agent-subagents/evals/local-delegation.eval.ts b/e2e/fixtures/agent-subagents/evals/local-delegation.eval.ts
index 47d0861d6..b3fa2aa2f 100644
--- a/e2e/fixtures/agent-subagents/evals/local-delegation.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/local-delegation.eval.ts
@@ -16,7 +16,7 @@ export default defineEval({
);
started.expectOk();
const completed = await t.target
- .watchTurn(started.sessionId, { startIndex: requireStreamIndex(t) })
+ .watchTurn(started.sessionId, { startIndex: requireStreamIndex(started.session) })
.result();
completed.expectOk();
completed.messageIncludes(SUBAGENT_TOKEN);
diff --git a/e2e/fixtures/agent-subagents/evals/recursive-root-only.eval.ts b/e2e/fixtures/agent-subagents/evals/recursive-root-only.eval.ts
index 976106af7..a97e07f4d 100644
--- a/e2e/fixtures/agent-subagents/evals/recursive-root-only.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/recursive-root-only.eval.ts
@@ -19,7 +19,7 @@ export default defineEval({
started.expectOk();
const completed = await t.target
- .watchTurn(started.sessionId, { startIndex: requireStreamIndex(t) })
+ .watchTurn(started.sessionId, { startIndex: requireStreamIndex(started.session) })
.result();
completed.expectOk();
completed.messageIncludes(CHILD_TOKEN);
diff --git a/e2e/fixtures/agent-subagents/evals/remote-principal-forwarding.eval.ts b/e2e/fixtures/agent-subagents/evals/remote-principal-forwarding.eval.ts
index f73798cc2..27958c1f8 100644
--- a/e2e/fixtures/agent-subagents/evals/remote-principal-forwarding.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/remote-principal-forwarding.eval.ts
@@ -38,7 +38,7 @@ export default defineEval({
async test(t) {
// Alice creates the child and reads her workspace label.
const aliceTurn = await t.send(CREATE_CHILD_MESSAGE);
- const aliceParent = await waitForRemoteChild(t, t, aliceTurn);
+ const aliceParent = await waitForRemoteChild(t, aliceTurn.session, aliceTurn);
const childSessionId = aliceParent.childSessionId;
const aliceChild = await t.target.watchTurn(childSessionId).result();
await expectWorkspaceReads(t, aliceChild, ALICE_WORKSPACE_LABEL);
diff --git a/e2e/fixtures/agent-subagents/evals/software-factory-workflow.eval.ts b/e2e/fixtures/agent-subagents/evals/software-factory-workflow.eval.ts
index 85db7281b..12a1a6103 100644
--- a/e2e/fixtures/agent-subagents/evals/software-factory-workflow.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/software-factory-workflow.eval.ts
@@ -1,4 +1,5 @@
import { randomUUID } from "node:crypto";
+import { isDeepStrictEqual } from "node:util";
import { defineEval } from "eve/evals";
import { equals } from "eve/evals/expect";
@@ -21,14 +22,17 @@ export default defineEval({
"Alice is preparing a QA summary for a sample software backlog. Use the workflow tool exactly once, and call its subagents only from that program.",
`Create a minimal array of exactly 200 tickets with ids \`T-000\` through \`T-199\`. Each ticket must contain only \`id\`, except \`T-000\` also has \`triageLabel\` set to \`TRIAGE-${token}\` and \`T-199\` also has \`reviewLabel\` set to \`REVIEW-${token}\`.`,
"Use Promise.all to run ticket-triage and ticket-review concurrently, passing the complete tickets array to both calls. Use `outputSchema` to request exactly `{ ticketCount: integer, priorityTicket: string, summaryLabel: string }` from ticket-triage and `{ ticketCount: integer, accepted: boolean, summaryLabel: string }` from ticket-review.",
- "After both complete, call ticket-reproducer with the returned `{ triage, review }` objects and use `outputSchema` to request exactly `{ combinedLabel: string, sampleTicket: string, verificationNote: string }`. Make every listed output field required and allow no additional output fields.",
+ "After both complete, call ticket-reproducer with the returned `{ triage, review }` objects as JSON. The reproducer already has its output rules; pass the objects without adding formatting instructions. Use `outputSchema` to request exactly `{ combinedLabel: string, sampleTicket: string, verificationNote: string }`. Make every listed output field required and allow no additional output fields.",
"Return `{ reproduction, review, triage }` and reply with only that returned JSON object.",
].join(" "),
);
turn.expectOk();
t.succeeded();
- t.calledTool("workflow", { count: 1, output: exactOutput(expected) });
+ t.calledTool("workflow", {
+ count: 1,
+ output: (observed) => isDeepStrictEqual(observed, expected),
+ });
t.calledSubagent(TRIAGE, { count: 1, status: "pending" });
t.calledSubagent(REVIEW, { count: 1, status: "pending" });
t.calledSubagent(REPRODUCE, { count: 1, status: "pending" });
@@ -77,34 +81,6 @@ function expectedFactoryOutput(token: string) {
};
}
-function exactOutput(expected: ReturnType<typeof expectedFactoryOutput>) {
- return (observed: unknown) => structurallyEqual(observed, expected);
-}
-
-function structurallyEqual(left: unknown, right: unknown): boolean {
- if (left === right) return true;
- if (Array.isArray(left) || Array.isArray(right)) {
- return (
- Array.isArray(left) &&
- Array.isArray(right) &&
- left.length === right.length &&
- left.every((value, index) => structurallyEqual(value, right[index]))
- );
- }
- if (left === null || right === null || typeof left !== "object" || typeof right !== "object") {
- return false;
- }
- const leftEntries = Object.entries(left);
- const rightRecord = right as Record<string, unknown>;
- return (
- leftEntries.length === Object.keys(rightRecord).length &&
- leftEntries.every(
- ([key, value]) =>
- Object.hasOwn(rightRecord, key) && structurallyEqual(value, rightRecord[key]),
- )
- );
-}
-
function parseJsonMessage(message: string | undefined): unknown {
if (message === undefined) return undefined;
const trimmed = message.trim();
diff --git a/e2e/fixtures/agent-subagents/evals/workflow-blocking-continuation.eval.ts b/e2e/fixtures/agent-subagents/evals/workflow-blocking-continuation.eval.ts
index 0be4e1521..76f3dbb76 100644
--- a/e2e/fixtures/agent-subagents/evals/workflow-blocking-continuation.eval.ts
+++ b/e2e/fixtures/agent-subagents/evals/workflow-blocking-continuation.eval.ts
@@ -16,13 +16,13 @@ export default defineEval({
const firstTurn = started.message?.includes("SUBAGENT_TOKEN=echo-marker-9F2X")
? undefined
: t.target.watchTurn(started.sessionId, {
- startIndex: requireStreamIndex(t),
+ startIndex: started.session.state.streamIndex,
});
const completed = firstTurn === undefined ? started : await firstTurn.result();
completed.expectOk();
completed.messageIncludes("SUBAGENT_TOKEN=echo-marker-9F2X");
- const second = await (firstTurn?.session ?? t).send(
+ const second = await completed.session.send(
[
"Use the workflow tool exactly once. In its JavaScript, call the same echo-marker child three times sequentially with ctx.agent",
"using the agentId shown in the latest <agents> block, with messages 'blocking second', 'blocking third', and 'blocking fourth'.",
@@ -51,11 +51,3 @@ export default defineEval({
t.messageIncludes("WORKFLOW_PROGRAM_SUBAGENT_LIMIT_REACHED");
},
});
-
-function requireStreamIndex(session: {
- readonly state?: { readonly streamIndex?: number };
-}): number {
- const streamIndex = session.state?.streamIndex;
- if (streamIndex === undefined) throw new Error("Parent session has no stream index.");
- return streamIndex;
-}
diff --git a/e2e/fixtures/agent-task-reporting/evals/reporting.ts b/e2e/fixtures/agent-task-reporting/evals/reporting.ts
index 41bb6e073..407853c9c 100644
--- a/e2e/fixtures/agent-task-reporting/evals/reporting.ts
+++ b/e2e/fixtures/agent-task-reporting/evals/reporting.ts
@@ -36,7 +36,7 @@ export interface ReportingRun {
readonly probeSessions: Map<Check, string>;
readonly parentTurns: EveEvalTurn[];
readonly modelId: string;
- session: EveEvalSession | EveEvalContext;
+ session: EveEvalSession;
}
export async function startWarehouseLookups(t: EveEvalContext): Promise<ReportingRun> {
@@ -84,7 +84,7 @@ Once all three assignments are accepted, let Alice know the checks are underway.
const taskIds = receipts.map((receipt) => receipt.taskId);
const requests = new Map<Check, InputRequest>();
const run: ReportingRun = {
- session: t,
+ session: started.session,
sessionId: started.sessionId,
taskIds,
children,
@@ -307,7 +307,7 @@ export function completedAt(turn: EveEvalTurn): number {
return Date.parse(event.meta.at);
}
-export function requireStreamIndex(session: EveEvalSession | EveEvalContext): number {
+export function requireStreamIndex(session: EveEvalSession): number {
if (session.state === undefined) throw new Error("Task reporting session has no stream index.");
return session.state.streamIndex;
}
diff --git a/e2e/fixtures/agent-tools-hitl-openai/evals/hitl/approve-resume.eval.ts b/e2e/fixtures/agent-tools-hitl-openai/evals/hitl/approve-resume.eval.ts
index f0ec92120..59d685ed0 100644
--- a/e2e/fixtures/agent-tools-hitl-openai/evals/hitl/approve-resume.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl-openai/evals/hitl/approve-resume.eval.ts
@@ -19,16 +19,17 @@ export default defineEval({
description: "HITL regression (#236): text approval executes and replays on OpenAI Responses.",
async test(t) {
const parked = await t.send('Call the guarded-echo tool with note "openai-approve".');
+ const session = parked.session;
parked.calledTool("guarded-echo", { status: "pending", count: 1 });
parked.notEvent("compaction.completed");
- t.requireInputRequest({
+ session.requireInputRequest({
display: "confirmation",
toolName: "guarded-echo",
});
// No compaction and no structured `respond()`: this is the payload shape
// produced when a Telegram reply falls back to an ordinary text message.
- const approved = await t.send("approve");
+ const approved = await session.send("approve");
approved.expectOk();
approved.notEvent("compaction.completed");
approved.event("action.result", {
@@ -43,7 +44,7 @@ export default defineEval({
count: 1,
});
- const followup = await t.send("Reply with exactly OPENAI-REPLAY-OK.");
+ const followup = await session.send("Reply with exactly OPENAI-REPLAY-OK.");
followup.expectOk();
followup.notEvent("compaction.completed");
followup.messageIncludes(/OPENAI-REPLAY-OK/i);
diff --git a/e2e/fixtures/agent-tools-hitl/evals/auth/authorization-nonblocking.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/auth/authorization-nonblocking.eval.ts
index 664e339c4..d3cd7251c 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/auth/authorization-nonblocking.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/auth/authorization-nonblocking.eval.ts
@@ -12,11 +12,14 @@ export default defineEval({
const parked = await t.send(
'Call the auth-probe tool exactly once with marker "nonblocking". Include its result.',
);
+ const session = parked.session;
parked.event("authorization.required", { count: 1 });
parked.notEvent("authorization.completed");
parked.event("session.waiting", { count: 1 });
- const message = await t.send("Do not call any tools. Reply with exactly AUTH-OPEN-MESSAGE-OK.");
+ const message = await session.send(
+ "Do not call any tools. Reply with exactly AUTH-OPEN-MESSAGE-OK.",
+ );
message.expectOk();
if (message.sessionId !== parked.sessionId) {
throw new Error("Message while authorization was open changed session identity.");
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/approve-then-no-regate.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/approve-then-no-regate.eval.ts
index bde31e6be..bd44b5f5e 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/approve-then-no-regate.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/approve-then-no-regate.eval.ts
@@ -12,10 +12,11 @@ export default defineEval({
description: "HITL smoke: an approved once() grant persists for the session.",
async test(t) {
const parked = await t.send('Call the guarded-echo tool with note "first-call".');
- t.requireInputRequest({ toolName: "guarded-echo" });
+ const session = parked.session;
+ session.requireInputRequest({ toolName: "guarded-echo" });
parked.calledTool("guarded-echo", { status: "pending", count: 1 });
- const approved = await t.respondAll("approve");
+ const approved = await session.respondAll("approve");
approved.expectOk();
approved.event("action.result", {
data: {
@@ -31,7 +32,7 @@ export default defineEval({
// A successful turn in an open session ends "waiting"; a re-park
// would surface as pending input requests.
- const second = await t.send('Call the guarded-echo tool again with note "second-call".');
+ const second = await session.send('Call the guarded-echo tool again with note "second-call".');
second.succeeded();
t.succeeded();
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/ask-question-select.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/ask-question-select.eval.ts
index 248659428..3e7af4837 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/ask-question-select.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/ask-question-select.eval.ts
@@ -9,7 +9,7 @@ export default defineEval({
tags: ["real-model"],
description: "HITL smoke: ask-question select parks and resumes with the chosen option.",
async test(t) {
- await t.send(
+ const { session } = await t.send(
[
"Use the `ask_question` tool exactly once to ask me which color I prefer.",
"Set prompt to: 'Pick a color.'",
@@ -19,13 +19,13 @@ export default defineEval({
].join("\n"),
);
- t.requireInputRequest({
+ session.requireInputRequest({
display: (value) => value === undefined || value === "select",
optionIds: ["red", "blue"],
toolName: "ask_question",
});
- await t.respondAll("blue");
+ await session.respondAll("blue");
t.succeeded();
t.messageIncludes(/\bblue\b/i);
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-approve-resume.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-approve-resume.eval.ts
index fc7bbd1f3..cb0b883a5 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-approve-resume.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-approve-resume.eval.ts
@@ -10,15 +10,16 @@ export default defineEval({
"HITL repro (#533): a separate approval response executes an authored always-gated tool.",
async test(t) {
const parked = await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
+ const session = parked.session;
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- const approval = t.requireInputRequest({
+ const approval = session.requireInputRequest({
display: "confirmation",
toolName: TOOL_NAME,
});
// This sends only `inputResponses` in a separate turn. No user message or
// channel context follows the tool approval response in the model input.
- const approved = await t.respond([
+ const approved = await session.respond([
{
optionId: "approve",
requestId: approval.requestId,
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-unrelated-input.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-unrelated-input.eval.ts
index 5d7af4bd5..8a369d9bc 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-unrelated-input.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/authored-always-unrelated-input.eval.ts
@@ -10,13 +10,14 @@ export default defineEval({
"HITL repro (#533): unrelated input does not replay an unresolved authored tool call.",
async test(t) {
const parked = await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
+ const session = parked.session;
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- const approval = t.requireInputRequest({
+ const approval = session.requireInputRequest({
display: "confirmation",
toolName: TOOL_NAME,
});
- const unrelated = await t.send(
+ const unrelated = await session.send(
"Note this unrelated marker and do not call any tools: ORBITAL-PINE-6C3R.",
);
@@ -28,7 +29,7 @@ export default defineEval({
});
unrelated.event("session.waiting", { count: 1 });
- const approved = await t.respond([
+ const approved = await session.respond([
{
optionId: "approve",
requestId: approval.requestId,
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-cancel.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-cancel.eval.ts
index 20d63a415..06b2f4bdf 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-cancel.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-cancel.eval.ts
@@ -7,9 +7,14 @@ export default defineEval({
tags: ["real-model"],
description: "Cancel beats a candidate parked on OAuth and a late callback cannot execute.",
async test(t) {
- await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
- const approval = t.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
- const approvalTurn = await t.startRespond(
+ const { session: conversation } = await t.send(
+ `Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`,
+ );
+ const approval = conversation.requireInputRequest({
+ display: "confirmation",
+ toolName: TOOL_NAME,
+ });
+ const approvalTurn = await conversation.startRespond(
[{ optionId: "approve", requestId: approval.requestId }],
{ headers: { "x-eve-fixture-user": "oauth-cancel-responder" } },
);
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-nonblocking.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-nonblocking.eval.ts
index 26e5963ad..3d31683a4 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-nonblocking.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth-nonblocking.eval.ts
@@ -7,9 +7,14 @@ export default defineEval({
tags: ["real-model"],
description: "A message runs while candidate OAuth stays open, then OAuth settles the approval.",
async test(t) {
- await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
- const approval = t.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
- const approvalTurn = await t.startRespond(
+ const { session: conversation } = await t.send(
+ `Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`,
+ );
+ const approval = conversation.requireInputRequest({
+ display: "confirmation",
+ toolName: TOOL_NAME,
+ });
+ const approvalTurn = await conversation.startRespond(
[{ optionId: "approve", requestId: approval.requestId }],
{ headers: { "x-eve-fixture-user": "oauth-nonblocking-responder" } },
);
@@ -33,7 +38,7 @@ export default defineEval({
}
const callbackUrl = new URL(required.data.authorization.url);
const callbackTurn = t.target.watchTurn(approvalTurn.sessionId, {
- startIndex: t.state?.streamIndex,
+ startIndex: conversation.state?.streamIndex,
});
await new Promise((resolve) => setTimeout(resolve, 250));
const callback = await fetch(callbackUrl);
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth.eval.ts
index 45d1b2294..4b3b7dafa 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response-oauth.eval.ts
@@ -7,9 +7,9 @@ export default defineEval({
tags: ["real-model"],
description: "Approval response policy parks for fake OAuth, resumes, and settles.",
async test(t) {
- await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
- const approval = t.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
- const approvalTurn = await t.startRespond([
+ const { session } = await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
+ const approval = session.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
+ const approvalTurn = await session.startRespond([
{
optionId: "approve",
requestId: approval.requestId,
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response.eval.ts
index a7cc3ae2d..bc41aaf7b 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/authorized-response.eval.ts
@@ -8,10 +8,11 @@ export default defineEval({
description: "Authenticated response policy emits candidate and settlement before execution.",
async test(t) {
const parked = await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
- const approval = t.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
+ const session = parked.session;
+ const approval = session.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- const approved = await t.respond([
+ const approved = await session.respond([
{
optionId: "approve",
requestId: approval.requestId,
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/button-cancel.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/button-cancel.eval.ts
index 71cac2cd7..b683da7b1 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/button-cancel.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/button-cancel.eval.ts
@@ -5,12 +5,15 @@ export default defineEval({
description: "HITL smoke: approvals use structured controls, not text replies.",
async test(t) {
const parked = await t.send('Call the guarded-echo tool with note "text-approve".');
+ const session = parked.session;
parked.calledTool("guarded-echo", { status: "pending", count: 1 });
- const approval = t.requireInputRequest({
+ const approval = session.requireInputRequest({
display: "confirmation",
toolName: "guarded-echo",
});
- const cancelled = await t.respond([{ optionId: "cancel", requestId: approval.requestId }]);
+ const cancelled = await session.respond([
+ { optionId: "cancel", requestId: approval.requestId },
+ ]);
cancelled.expectOk();
t.succeeded();
},
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/deny-then-regate.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/deny-then-regate.eval.ts
index 8980f210e..bec12d3f6 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/deny-then-regate.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/deny-then-regate.eval.ts
@@ -9,10 +9,10 @@ export default defineEval({
tags: ["real-model"],
description: "HITL smoke: a denied once() call does not execute and re-gates the next call.",
async test(t) {
- await t.send('Call the guarded-echo tool with note "denied-call".');
- const request = t.requireInputRequest({ toolName: "guarded-echo" });
+ const { session } = await t.send('Call the guarded-echo tool with note "denied-call".');
+ const request = session.requireInputRequest({ toolName: "guarded-echo" });
- const denied = await t.respondAll("cancel");
+ const denied = await session.respondAll("cancel");
denied.expectOk();
denied.event("action.result", {
data: {
@@ -40,9 +40,9 @@ export default defineEval({
)
.atLeast(0.5);
- await t.send('Call the guarded-echo tool once more with note "retry-call".');
+ await session.send('Call the guarded-echo tool once more with note "retry-call".');
// Denial does not grant: the follow-up call must re-park.
- t.requireInputRequest({ toolName: "guarded-echo" });
+ session.requireInputRequest({ toolName: "guarded-echo" });
t.parked();
t.calledTool("guarded-echo", { status: "rejected", count: 1 });
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-approve-then-followup.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-approve-then-followup.eval.ts
index 70b16faf0..2baaa8a9d 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-approve-then-followup.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-approve-then-followup.eval.ts
@@ -18,13 +18,14 @@ export default defineEval({
description: "HITL regression (#533): a resolved approval park replays on the next turn.",
async test(t) {
const parked = await t.send(`Call the \`${TOOL_NAME}\` tool with note "replay-probe".`);
+ const session = parked.session;
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- t.requireInputRequest({
+ session.requireInputRequest({
display: "confirmation",
toolName: TOOL_NAME,
});
- const approved = await t.respondAll("approve");
+ const approved = await session.respondAll("approve");
approved.expectOk();
approved.event("action.result", {
data: {
@@ -38,7 +39,7 @@ export default defineEval({
count: 1,
});
- const followup = await t.send("Reply with exactly DYNAMIC-REPLAY-OK.");
+ const followup = await session.send("Reply with exactly DYNAMIC-REPLAY-OK.");
followup.expectOk();
followup.messageIncludes(/DYNAMIC-REPLAY-OK/i);
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-guarded-replay.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-guarded-replay.eval.ts
index 46a5c5471..d19f44c32 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-guarded-replay.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-guarded-replay.eval.ts
@@ -13,13 +13,14 @@ export default defineEval({
description: "HITL smoke: replayed dynamic tools preserve approval.",
async test(t) {
const parked = await t.send(`Call the \`${TOOL_NAME}\` tool with note "before-approval".`);
- t.requireInputRequest({
+ const session = parked.session;
+ session.requireInputRequest({
display: "confirmation",
toolName: TOOL_NAME,
});
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- const approved = await t.respondAll("approve");
+ const approved = await session.respondAll("approve");
approved.expectOk();
approved.event("action.result", {
data: {
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-scoped-approval.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-scoped-approval.eval.ts
index 99b2c8b4e..d40875646 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-scoped-approval.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/dynamic-scoped-approval.eval.ts
@@ -7,20 +7,25 @@ export default defineEval({
"Dynamic input-scoped approval grants survive replay without authorizing other scopes.",
async test(t) {
const first = await t.send(`Call the ${TOOL_NAME} tool exactly once with scope "repo-a".`);
+ const session = first.session;
first.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- t.requireInputRequest({ toolName: TOOL_NAME });
- const approved = await t.respondAll("approve");
+ session.requireInputRequest({ toolName: TOOL_NAME });
+ const approved = await session.respondAll("approve");
approved.expectOk();
approved.calledTool(TOOL_NAME, { status: "completed", count: 1 });
- const repeated = await t.send(`Call the ${TOOL_NAME} tool exactly once with scope "repo-a".`);
+ const repeated = await session.send(
+ `Call the ${TOOL_NAME} tool exactly once with scope "repo-a".`,
+ );
repeated.succeeded();
repeated.calledTool(TOOL_NAME, { status: "completed", count: 1 });
- const other = await t.send(`Call the ${TOOL_NAME} tool exactly once with scope "repo-b".`);
+ const other = await session.send(
+ `Call the ${TOOL_NAME} tool exactly once with scope "repo-b".`,
+ );
other.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- t.requireInputRequest({ toolName: TOOL_NAME });
- const approvedOther = await t.respondAll("approve");
+ session.requireInputRequest({ toolName: TOOL_NAME });
+ const approvedOther = await session.respondAll("approve");
approvedOther.expectOk();
t.succeeded();
t.calledTool(TOOL_NAME, { status: "completed", count: 3 });
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/pending-approval-followup-dedup.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/pending-approval-followup-dedup.eval.ts
index 418d74269..1160f8f7c 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/pending-approval-followup-dedup.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/pending-approval-followup-dedup.eval.ts
@@ -16,14 +16,15 @@ export default defineEval({
description: "One pending approval stays singular across many follow-up questions.",
async test(t) {
const parked = await t.send(`Call the ${TOOL_NAME} tool exactly once with marker "${MARKER}".`);
+ const session = parked.session;
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- const approval = t.requireInputRequest({
+ const approval = session.requireInputRequest({
display: "confirmation",
toolName: TOOL_NAME,
});
for (const question of FOLLOW_UP_QUESTIONS) {
- const followup = await t.send(question);
+ const followup = await session.send(question);
followup.expectOk();
followup.usedNoTools();
@@ -31,7 +32,7 @@ export default defineEval({
followup.event("session.waiting", { count: 1 });
}
- const approved = await t.respond([
+ const approved = await session.respond([
{
optionId: "approve",
requestId: approval.requestId,
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/slow-once-approve-replay.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/slow-once-approve-replay.eval.ts
index 993b35530..2623d0406 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/slow-once-approve-replay.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/slow-once-approve-replay.eval.ts
@@ -21,13 +21,14 @@ export default defineEval({
const parked = await t.send(
`Call the ${TOOL_NAME} tool with note "alpha". After its result arrives, call it again with note "beta". Use strictly one call at a time, never in parallel. When both results are in, reply with exactly SLOW-DONE.`,
);
+ const session = parked.session;
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- const request = t.requireInputRequest({
+ const request = session.requireInputRequest({
display: "confirmation",
toolName: TOOL_NAME,
});
- const approved = await t.respond([
+ const approved = await session.respond([
{
requestId: request.requestId,
optionId: "approve",
@@ -41,7 +42,7 @@ export default defineEval({
count: 2,
});
- const followup = await t.send("Reply with exactly SLOW-REPLAY-OK.");
+ const followup = await session.send("Reply with exactly SLOW-REPLAY-OK.");
followup.expectOk();
followup.messageIncludes(/SLOW-REPLAY-OK/i);
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection-idle.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection-idle.eval.ts
index 6e6ce5540..8994eeb06 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection-idle.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection-idle.eval.ts
@@ -11,7 +11,7 @@ export default defineEval({
description:
"HITL smoke: a stale ask-question selection becomes a new user turn when nothing is pending.",
async test(t) {
- await t.send(
+ const { session } = await t.send(
[
"Use the `ask_question` tool exactly once to ask me which context to use.",
"Set prompt to: 'Which context should I use?'",
@@ -23,19 +23,19 @@ export default defineEval({
].join("\n"),
);
- const request = t.requireInputRequest({
+ const request = session.requireInputRequest({
optionIds: ["current", "candidate"],
toolName: "ask_question",
});
- const intervening = await t.send(
+ const intervening = await session.send(
"Use current context instead and reply with exactly INTERVENING-HITL-OK.",
);
intervening.expectOk();
intervening.notEvent("input.requested");
intervening.messageIncludes(/INTERVENING-HITL-OK/i);
- const staleSelection = await t.respond([
+ const staleSelection = await session.respond([
{
requestId: request.requestId,
optionId: "candidate",
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection.eval.ts
index c509aa20c..81f41e9b4 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/stale-ask-question-selection.eval.ts
@@ -11,7 +11,7 @@ export default defineEval({
description:
"HITL smoke: a stale ask-question selection becomes a new user turn while another question is pending.",
async test(t) {
- await t.send(
+ const { session } = await t.send(
[
"Use the `ask_question` tool exactly once to ask me which context to use.",
"Set prompt to: 'Which context should I use?'",
@@ -23,19 +23,19 @@ export default defineEval({
].join("\n"),
);
- const request = t.requireInputRequest({
+ const request = session.requireInputRequest({
optionIds: ["current", "candidate"],
toolName: "ask_question",
});
- const intervening = await t.send(
+ const intervening = await session.send(
"Use current context instead and reply with exactly INTERVENING-HITL-OK.",
);
intervening.expectOk();
intervening.notEvent("input.requested");
intervening.messageIncludes(/INTERVENING-HITL-OK/i);
- const nextQuestionTurn = await t.send(
+ const nextQuestionTurn = await session.send(
[
"Use the `ask_question` tool exactly once to ask which new context to use.",
"Set prompt to: 'Which new context should I use?'",
@@ -48,7 +48,7 @@ export default defineEval({
);
nextQuestionTurn.expectOk();
nextQuestionTurn.event("input.requested", { count: 1 });
- const nextRequest = t.requireInputRequest({
+ const nextRequest = session.requireInputRequest({
optionIds: ["alpha", "beta"],
toolName: "ask_question",
});
@@ -56,7 +56,7 @@ export default defineEval({
throw new Error("The second ask_question call reused the stale request ID.");
}
- const staleSelection = await t.respond([
+ const staleSelection = await session.respond([
{
requestId: request.requestId,
optionId: "candidate",
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.ts
index 00134b285..78367ae8c 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.ts
@@ -8,10 +8,14 @@ export default defineEval({
description: "A rejected responder leaves the approval open for an authorized retry.",
async test(t) {
const parked = await t.send(`Call the \`${TOOL_NAME}\` tool with marker "${MARKER}".`);
- const approval = t.requireInputRequest({ display: "confirmation", toolName: TOOL_NAME });
+ const conversation = parked.session;
+ const approval = conversation.requireInputRequest({
+ display: "confirmation",
+ toolName: TOOL_NAME,
+ });
parked.calledTool(TOOL_NAME, { status: "pending", count: 1 });
- const rejectedTurn = await t.startRespond(
+ const rejectedTurn = await conversation.startRespond(
[{ optionId: "approve", requestId: approval.requestId }],
{ headers: { "x-eve-fixture-user": "unauthorized-responder" } },
);
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/unrelated-message-runs.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/unrelated-message-runs.eval.ts
index 62b7c8637..823b5deab 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/unrelated-message-runs.eval.ts
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/unrelated-message-runs.eval.ts
@@ -13,14 +13,15 @@ export default defineEval({
description: "HITL smoke: unrelated message during approval runs immediately.",
async test(t) {
const parked = await t.send('Call the guarded-echo tool with note "open-approval".');
+ const session = parked.session;
parked.calledTool("guarded-echo", { status: "pending", count: 1 });
- const request = t.requireInputRequest({
+ const request = session.requireInputRequest({
display: "confirmation",
toolName: "guarded-echo",
});
// The message runs now; the approval is untouched.
- const message = await t.send(
+ const message = await session.send(
"Leave the pending approval alone. Do not call any tools. Reply with exactly OPEN-APPROVAL-MSG-OK.",
);
message.expectOk();
@@ -36,7 +37,7 @@ export default defineEval({
message.event("session.waiting", { count: 1 });
// The approval still resolves afterwards and runs the tool once.
- const approved = await t.respond([
+ const approved = await session.respond([
{
requestId: request.requestId,
optionId: "approve",
diff --git a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/hook-stop.eval.ts b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/hook-stop.eval.ts
index 13fd72cac..49601bfae 100644
--- a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/hook-stop.eval.ts
+++ b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/hook-stop.eval.ts
@@ -11,9 +11,10 @@ export default defineEval({
description: "Sandbox: an authored hook can stop compute and the next turn reopens it.",
async test(t) {
const first = await t.send(`Reply with this exact token: ${STOP_SANDBOX_TOKEN}`);
+ const session = first.session;
first.expectOk();
- const second = await t.send(
+ const second = await session.send(
`Run the bash command \`cat ${STOP_SANDBOX_MARKER_PATH}\` and reply with the file contents verbatim.`,
);
diff --git a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/redeploy.eval.ts b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/redeploy.eval.ts
index d18440aed..363b3c845 100644
--- a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/redeploy.eval.ts
+++ b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/redeploy.eval.ts
@@ -88,6 +88,7 @@ export default defineEval({
`Run the bash command \`printf %s ${FILE_TOKEN} > ${FILE_PATH}\`. ` +
"Reply with the single word: done.",
);
+ const session = write.session;
write.expectOk();
write.calledTool("bash");
@@ -100,7 +101,7 @@ export default defineEval({
await waitForAliasToServe(t, INSTRUCTIONS_MARKER);
// t2: the same session reattaches to the same sandbox.
- const persist = await t.send(
+ const persist = await session.send(
`Run the bash command \`cat ${FILE_PATH}\` and reply with the file contents verbatim.`,
);
persist.expectOk();
@@ -115,7 +116,7 @@ export default defineEval({
// t3: the next request is accepted by the new deployment. Its changed
// sandbox resources rotate the versioned key, so the old file is absent.
- const probe = await t.send(
+ const probe = await session.send(
`Run the bash command \`test -f ${FILE_PATH} && echo present || echo absent\` ` +
"and reply with the command output verbatim.",
);
@@ -125,7 +126,7 @@ export default defineEval({
// t4: a fresh session adopts the new deployment — the added skill is
// advertised and usable.
- const adopted = t.newSession();
+ const adopted = await t.session();
const skill = await adopted.send(
`Load the \`${SKILL_NAME}\` skill and follow its instructions exactly.`,
);
diff --git a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/session-persistence.eval.ts b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/session-persistence.eval.ts
index 324f9ce17..29ad9309a 100644
--- a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/session-persistence.eval.ts
+++ b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/session-persistence.eval.ts
@@ -16,9 +16,10 @@ export default defineEval({
`Run the bash command \`printf %s ${PERSIST_TOKEN} > ${PERSIST_PATH}\`. ` +
"Reply with the single word: done.",
);
+ const session = first.session;
first.expectOk();
- const second = await t.send(
+ const second = await session.send(
`Run the bash command \`cat ${PERSIST_PATH}\` and reply with the file contents verbatim.`,
);
diff --git a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/subagent-sharing.eval.ts b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/subagent-sharing.eval.ts
index 100feb530..7eb47fb65 100644
--- a/e2e/fixtures/agent-tools-sandbox/evals/sandbox/subagent-sharing.eval.ts
+++ b/e2e/fixtures/agent-tools-sandbox/evals/sandbox/subagent-sharing.eval.ts
@@ -13,9 +13,10 @@ export default defineEval({
`Run the bash command \`printf %s ${PARENT_TOKEN} > ${PARENT_PATH}\`. ` +
"Reply with the single word: done.",
);
+ const conversation = parentWrite.session;
parentWrite.expectOk();
- const childTurn = await t.send(
+ const childTurn = await conversation.send(
`Ask the \`shared-sandbox\` subagent with message: ` +
`Run the bash command \`cat ${PARENT_PATH} && printf %s ${CHILD_TOKEN} > ${CHILD_PATH}\` ` +
"and reply with the command output verbatim.",
@@ -24,7 +25,7 @@ export default defineEval({
const sessionId = childTurn.sessionId;
if (sessionId === undefined) throw new Error("Shared sandbox turn has no session id.");
const completed = t.target.watchTurn(sessionId, {
- startIndex: requireStreamIndex(t),
+ startIndex: requireStreamIndex(childTurn.session),
});
const childCompletion = await completed.result();
childCompletion.expectOk();
diff --git a/e2e/fixtures/agent-tools/evals/dynamic-tools/conditional.eval.ts b/e2e/fixtures/agent-tools/evals/dynamic-tools/conditional.eval.ts
index 3cd24287e..20c644d38 100644
--- a/e2e/fixtures/agent-tools/evals/dynamic-tools/conditional.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/dynamic-tools/conditional.eval.ts
@@ -10,12 +10,13 @@ export default defineEval({
const first = await t.send(
"Use the `check_stability` tool and tell me the branch and invocations values.",
);
+ const session = first.session;
first.expectOk();
first.calledTool("check_stability", {
output: { branch: "first" },
});
- const second = await t.send(
+ const second = await session.send(
"Use the `check_stability` tool to check stability. Call it now and report the branch and invocations values.",
);
second.calledTool("check_stability", {
diff --git a/e2e/fixtures/agent-tools/evals/dynamic-tools/io-once.eval.ts b/e2e/fixtures/agent-tools/evals/dynamic-tools/io-once.eval.ts
index ab4bf6c34..a4ed3cdea 100644
--- a/e2e/fixtures/agent-tools/evals/dynamic-tools/io-once.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/dynamic-tools/io-once.eval.ts
@@ -9,12 +9,13 @@ export default defineEval({
const first = await t.send(
"Use the `get_io_count` tool and tell me the ioCallCount number from the result.",
);
+ const session = first.session;
first.expectOk();
first.calledTool("get_io_count", {
output: { ioCallCount: 1 },
});
- const second = await t.send(
+ const second = await session.send(
"Use the `get_io_count` tool again right now and tell me the ioCallCount value from the result.",
);
second.calledTool("get_io_count", {
diff --git a/e2e/fixtures/agent-tools/evals/dynamic-tools/messages.eval.ts b/e2e/fixtures/agent-tools/evals/dynamic-tools/messages.eval.ts
index 3433f6d90..3c84556e2 100644
--- a/e2e/fixtures/agent-tools/evals/dynamic-tools/messages.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/dynamic-tools/messages.eval.ts
@@ -12,10 +12,11 @@ export default defineEval({
"Use the check_messages tool with label 'turn1' to record the initial count " +
"from the application's conversation log, then summarize the returned counts.",
);
+ const session = first.session;
first.expectOk();
const firstOutput = first.requireToolCall("check_messages").output;
- const second = await t.send(
+ const second = await session.send(
"Bob has added this follow-up to the support conversation. Use the check_messages " +
"tool with label 'turn2' to record the updated count from the application's " +
"conversation log, then summarize the returned counts so Alice can compare them.",
diff --git a/e2e/fixtures/agent-tools/evals/dynamic-tools/multi-turn.eval.ts b/e2e/fixtures/agent-tools/evals/dynamic-tools/multi-turn.eval.ts
index 4ef697308..58b04bf8f 100644
--- a/e2e/fixtures/agent-tools/evals/dynamic-tools/multi-turn.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/dynamic-tools/multi-turn.eval.ts
@@ -11,12 +11,13 @@ export default defineEval({
const first = await t.send(
`Please call the \`${ECHO_TOOL}\` tool with message 'turn one' and tell me the token it returned.`,
);
+ const session = first.session;
first.expectOk();
first.calledTool(ECHO_TOOL, {
output: { token: DYNAMIC_ECHO_TOKEN },
});
- const second = await t.send(
+ const second = await session.send(
`I need you to call the \`${ECHO_TOOL}\` tool right now with message 'turn two', do not answer from memory. Call it and tell me the token from the result.`,
);
second.calledTool(ECHO_TOOL, {
diff --git a/e2e/fixtures/agent-tools/evals/dynamic-tools/nested.eval.ts b/e2e/fixtures/agent-tools/evals/dynamic-tools/nested.eval.ts
index 9ff33572e..f078b1538 100644
--- a/e2e/fixtures/agent-tools/evals/dynamic-tools/nested.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/dynamic-tools/nested.eval.ts
@@ -10,9 +10,10 @@ export default defineEval({
const first = await t.send(
"Call the `nested_query` tool and tell me exactly what it returned.",
);
+ const session = first.session;
first.expectOk();
- await t.send("Now call the `nested_status` tool and tell me exactly what it returned.");
+ await session.send("Now call the `nested_status` tool and tell me exactly what it returned.");
t.succeeded();
t.calledTool("nested_query", {
diff --git a/e2e/fixtures/agent-tools/evals/dynamic-tools/overwrite.eval.ts b/e2e/fixtures/agent-tools/evals/dynamic-tools/overwrite.eval.ts
index ffd94526c..bdf23977f 100644
--- a/e2e/fixtures/agent-tools/evals/dynamic-tools/overwrite.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/dynamic-tools/overwrite.eval.ts
@@ -8,9 +8,10 @@ export default defineEval({
description: "Dynamic tools smoke: turn-scoped resolver tools win merges over session-scoped.",
async test(t) {
const first = await t.send("Call the `shared` tool and report the source and turn values.");
+ const session = first.session;
first.expectOk();
- await t.send("Call the `session_only` tool and report the source value.");
+ await session.send("Call the `session_only` tool and report the source value.");
t.succeeded();
t.calledTool("shared", {
diff --git a/e2e/fixtures/agent-tools/evals/dynamic-tools/schema-validation.eval.ts b/e2e/fixtures/agent-tools/evals/dynamic-tools/schema-validation.eval.ts
index 22ede0103..b04f68971 100644
--- a/e2e/fixtures/agent-tools/evals/dynamic-tools/schema-validation.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/dynamic-tools/schema-validation.eval.ts
@@ -3,14 +3,15 @@ import { defineEval } from "eve/evals";
export default defineEval({
description: "Dynamic tool replay preserves input transformations and validation across turns.",
async test(t) {
+ const session = await t.session();
for (let turn = 0; turn < 2; turn++) {
- const result = await t.send(
+ const result = await session.send(
'Call `schema_validate` with value " normalized ", including the surrounding spaces, and report its returned value.',
);
result.expectOk();
result.calledTool("schema_validate", { output: { value: "normalized" } });
}
- const invalid = await t.send(
+ const invalid = await session.send(
'Alice is checking a blank form submission. Call `schema_validate` with value " " (one space). If the tool reports a validation error, include "Blank value rejected" in your explanation. Do not retry.',
);
invalid.expectOk();
diff --git a/e2e/fixtures/agent-tools/evals/static-tools/throw-recover.eval.ts b/e2e/fixtures/agent-tools/evals/static-tools/throw-recover.eval.ts
index 26eb7e0b2..f1e63a8b4 100644
--- a/e2e/fixtures/agent-tools/evals/static-tools/throw-recover.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/static-tools/throw-recover.eval.ts
@@ -10,10 +10,11 @@ export default defineEval({
'Call the `always-throws` tool exactly once with reason "smoke". ' +
"After it fails, reply with a one-line acknowledgement that the tool failed.",
);
+ const session = first.session;
first.expectOk();
first.calledTool("always-throws", { status: "failed", count: 1 });
- const second = await t.send(
+ const second = await session.send(
"Are you still responsive? Reply with exactly the single word: yes.",
);
second.messageIncludes(/\byes\b/iu);
diff --git a/e2e/fixtures/agent-tools/evals/static-tools/to-model-output-content-parts.eval.ts b/e2e/fixtures/agent-tools/evals/static-tools/to-model-output-content-parts.eval.ts
index e24185c07..0a506705b 100644
--- a/e2e/fixtures/agent-tools/evals/static-tools/to-model-output-content-parts.eval.ts
+++ b/e2e/fixtures/agent-tools/evals/static-tools/to-model-output-content-parts.eval.ts
@@ -12,7 +12,7 @@ export default defineEval({
tags: ["real-model"],
description: "Static tools smoke: toModelOutput content parts deliver an image to the model.",
async test(t) {
- await t.send(
+ const { session } = await t.send(
`Call \`${TOOL_NAME}\` exactly once, look at the rendered image, and reply with only ` +
"the stripe colors left to right, comma-separated.",
);
@@ -27,7 +27,7 @@ export default defineEval({
// The content part is baked into persisted history, so a follow-up turn
// must answer from replay without re-running the tool.
- await t.send(
+ await session.send(
"Without calling any tool, repeat the stripe colors left to right, comma-separated.",
);
diff --git a/e2e/fixtures/agent-workflow-stress/evals/concurrent-workflow-turns.eval.ts b/e2e/fixtures/agent-workflow-stress/evals/concurrent-workflow-turns.eval.ts
index 89f096b1d..f902c1c45 100644
--- a/e2e/fixtures/agent-workflow-stress/evals/concurrent-workflow-turns.eval.ts
+++ b/e2e/fixtures/agent-workflow-stress/evals/concurrent-workflow-turns.eval.ts
@@ -11,12 +11,11 @@ export default defineEval({
tags: ["stress", "workflow", "concurrent"],
async test(t) {
- const sessions = Array.from({ length: SESSION_COUNT }, () => t.newSession());
const firstBatchStartedAt = performance.now();
const firstTurns = await Promise.all(
- sessions.map(async (session, index) => {
+ Array.from({ length: SESSION_COUNT }, async (_, index) => {
const startedAt = performance.now();
- const result = await session.send(markerFor(index, 1));
+ const result = await t.send(markerFor(index, 1));
return {
durationMs: performance.now() - startedAt,
@@ -28,9 +27,9 @@ export default defineEval({
const firstBatchDurationMs = performance.now() - firstBatchStartedAt;
const secondBatchStartedAt = performance.now();
const secondTurns = await Promise.all(
- sessions.map(async (session, index) => {
+ firstTurns.map(async ({ result: first }, index) => {
const startedAt = performance.now();
- const result = await session.send(markerFor(index, 2));
+ const result = await first.session.send(markerFor(index, 2));
return {
durationMs: performance.now() - startedAt,
diff --git a/e2e/fixtures/agent-workflow-stress/evals/sequential-workflow-turns.eval.ts b/e2e/fixtures/agent-workflow-stress/evals/sequential-workflow-turns.eval.ts
index a0604fcaa..65a9f2c9e 100644
--- a/e2e/fixtures/agent-workflow-stress/evals/sequential-workflow-turns.eval.ts
+++ b/e2e/fixtures/agent-workflow-stress/evals/sequential-workflow-turns.eval.ts
@@ -1,4 +1,4 @@
-import { defineEval } from "eve/evals";
+import { defineEval, type EveEvalSession } from "eve/evals";
import { equals } from "eve/evals/expect";
const TURN_COUNT = 100;
@@ -9,13 +9,13 @@ export default defineEval({
tags: ["stress", "workflow", "sequential"],
async test(t) {
- let sessionId: string | undefined;
+ let session: EveEvalSession | undefined;
const samples: Array<{ durationMs: number; turnNumber: number }> = [];
for (let turnNumber = 1; turnNumber <= TURN_COUNT; turnNumber += 1) {
const marker = `sequential-turn-${String(turnNumber).padStart(3, "0")}`;
const startedAt = performance.now();
- const result = await t.send(marker);
+ const result = await (session === undefined ? t.send(marker) : session.send(marker));
const durationMs = performance.now() - startedAt;
const elapsedSeconds = durationMs / 1_000;
@@ -25,11 +25,11 @@ export default defineEval({
`turn ${String(turnNumber).padStart(3, "0")}/${TURN_COUNT} completed in ${elapsedSeconds.toFixed(3)}s`,
);
- sessionId ??= result.sessionId;
+ session ??= result.session;
const turn = result.expectOk();
- await t.require(turn.sessionId, equals(sessionId));
+ await t.require(turn.sessionId, equals(session.sessionId));
await t.require(turn.message, equals(`stress-ack:${turnNumber}:${marker}`));
}
@@ -38,7 +38,7 @@ export default defineEval({
fixture: "agent-workflow-stress",
scenario: "sequential",
schemaVersion: 1,
- sessionId,
+ sessionId: session?.sessionId,
samples,
unit: "milliseconds",
})}`,
diff --git a/e2e/fixtures/agent-workflow-tools/evals/agent-probe.shared.ts b/e2e/fixtures/agent-workflow-tools/evals/agent-probe.shared.ts
index 78800bb4d..ff7d64aff 100644
--- a/e2e/fixtures/agent-workflow-tools/evals/agent-probe.shared.ts
+++ b/e2e/fixtures/agent-workflow-tools/evals/agent-probe.shared.ts
@@ -15,12 +15,12 @@ export async function runProbe(t: EveEvalContext, probe: ProbeCase): Promise<voi
started.expectOk();
if (probe.kind === "hitl") {
- const blocked = await waitForInput(t, t, "approval-gate");
+ const blocked = await waitForInput(t, started.session, "approval-gate");
const approved = await blocked.respondAll("approve");
approved.expectOk();
await waitForMarker(t, blocked, approved, "WORKFLOW-HITL:approved");
} else {
- const required = await waitForEvent(t, t, started, "authorization.required");
+ const required = await waitForEvent(t, started.session, started, "authorization.required");
const url = required.event.data.authorization?.url;
if (url === undefined) {
throw new Error("Authorization probe produced no callback URL.");
@@ -131,7 +131,7 @@ export async function runStepAuth(
scenario: "EXPLICIT" | "IMPLICIT",
): Promise<void> {
const started = await t.send(`WORKFLOW-STEP-AUTH-${scenario}`);
- const required = await waitForEvent(t, t, started, "authorization.required");
+ const required = await waitForEvent(t, started.session, started, "authorization.required");
const url = fixtureAuthorizationCallback(t.target.url, required.event.data.authorization?.url);
const response = await fetch(url);
@@ -146,7 +146,7 @@ export async function runStepAuth(
export async function runRejectedStepAuth(t: EveEvalContext): Promise<void> {
const started = await t.send("WORKFLOW-STEP-AUTH-REJECTED");
- const required = await waitForEvent(t, t, started, "authorization.required");
+ const required = await waitForEvent(t, started.session, started, "authorization.required");
const url = fixtureAuthorizationCallback(t.target.url, required.event.data.authorization?.url);
const response = await fetch(url);
diff --git a/e2e/fixtures/agent-workflow-tools/evals/confirm-deploy.hitl.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/confirm-deploy.hitl.eval.ts
index 5bc547114..44ad0ca2f 100644
--- a/e2e/fixtures/agent-workflow-tools/evals/confirm-deploy.hitl.eval.ts
+++ b/e2e/fixtures/agent-workflow-tools/evals/confirm-deploy.hitl.eval.ts
@@ -5,7 +5,8 @@ export default defineEval({
"A workflow tool reports progress around a human question, then settles the call in order.",
async test(t) {
const parked = await t.send("WORKFLOW-CONFIRM-START");
- t.requireInputRequest({
+ const session = parked.session;
+ session.requireInputRequest({
display: "confirmation",
optionIds: ["approve", "cancel"],
toolName: "confirm_deploy",
@@ -16,7 +17,7 @@ export default defineEval({
});
parked.calledTool("confirm_deploy", { status: "pending", count: 1 });
- const approved = await t.respondAll("approve");
+ const approved = await session.respondAll("approve");
approved.expectOk();
approved.event("action.result", {
count: 1,
diff --git a/e2e/fixtures/agent-workflow-tools/evals/connection-search.authorization.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/connection-search.authorization.eval.ts
index eea033f19..9602991e2 100644
--- a/e2e/fixtures/agent-workflow-tools/evals/connection-search.authorization.eval.ts
+++ b/e2e/fixtures/agent-workflow-tools/evals/connection-search.authorization.eval.ts
@@ -14,6 +14,7 @@ export default defineEval({
const started = await t.send(
"Alice wants to see which tools are available in private-catalog. Search for its items tools, then report the available tool names.",
);
+ const session = started.session;
started.expectOk();
started.event("authorization.required", { count: 1 });
started.notEvent("authorization.completed");
@@ -24,11 +25,13 @@ export default defineEval({
throw new Error("Connection search did not produce an authorization challenge.");
}
const callback = fixtureAuthorizationCallback(t.target.url, required.data.authorization?.url);
- if (t.sessionId === undefined || t.state === undefined) {
+ if (session.sessionId === undefined || session.state === undefined) {
throw new Error("Connection search did not create a session.");
}
- const resumed = t.target.watchTurn(t.sessionId, { startIndex: t.state.streamIndex });
+ const resumed = t.target.watchTurn(session.sessionId, {
+ startIndex: session.state.streamIndex,
+ });
const response = await fetch(callback);
if (!response.ok) {
throw new Error(`Authorization callback failed (${response.status}).`);
diff --git a/e2e/fixtures/agent-workflow-tools/evals/escalate-deploy.deadline.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/escalate-deploy.deadline.eval.ts
index d6e1e0188..7ca8bc7e8 100644
--- a/e2e/fixtures/agent-workflow-tools/evals/escalate-deploy.deadline.eval.ts
+++ b/e2e/fixtures/agent-workflow-tools/evals/escalate-deploy.deadline.eval.ts
@@ -10,14 +10,15 @@ export default defineEval({
"A workflow tool races ask against a deadline; the answer wins and settles the call.",
async test(t) {
const parked = await t.send("WORKFLOW-ESCALATE-START");
- t.requireInputRequest({
+ const session = parked.session;
+ session.requireInputRequest({
display: "confirmation",
optionIds: ["approve", "cancel"],
toolName: "escalate_deploy",
});
parked.calledTool("escalate_deploy", { status: "pending", count: 1 });
- const answered = await t.respondAll("approve");
+ const answered = await session.respondAll("approve");
answered.expectOk();
answered.calledTool("escalate_deploy", { output: /"decided":"approved"/u });
answered.messageIncludes("WORKFLOW-ESCALATE-RESULT");
diff --git a/e2e/fixtures/agent-workflow-tools/evals/hold-deploy.cancel.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/hold-deploy.cancel.eval.ts
index e8f271b11..e21bc691d 100644
--- a/e2e/fixtures/agent-workflow-tools/evals/hold-deploy.cancel.eval.ts
+++ b/e2e/fixtures/agent-workflow-tools/evals/hold-deploy.cancel.eval.ts
@@ -10,7 +10,8 @@ export default defineEval({
timeoutMs: 60_000,
description: "Cancelling a turn cancels the workflow tool run it is parked on.",
async test(t) {
- const live = await t.start("WORKFLOW-HOLD-START");
+ const session = await t.session();
+ const live = await session.start("WORKFLOW-HOLD-START");
await live.waitForEvent("actions.requested", {
data: {
actions: (actions) =>
@@ -33,7 +34,7 @@ export default defineEval({
turn.notEvent("turn.failed");
turn.notEvent("session.failed");
- const next = await t.send("WORKFLOW-IDLE-PING");
+ const next = await session.send("WORKFLOW-IDLE-PING");
next.expectOk();
next.messageIncludes("WORKFLOW-IDLE");
},
diff --git a/e2e/fixtures/agent-workflow-tools/evals/mixed-agent-resolution.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/mixed-agent-resolution.eval.ts
index 82b5d61b2..ee7ba3ecf 100644
--- a/e2e/fixtures/agent-workflow-tools/evals/mixed-agent-resolution.eval.ts
+++ b/e2e/fixtures/agent-workflow-tools/evals/mixed-agent-resolution.eval.ts
@@ -16,7 +16,7 @@ export default defineEval({
const sessionId = initial.sessionId;
if (sessionId === undefined) throw new Error("Mixed workflow turn has no session id.");
- let session: Pick<EveEvalSession, "state"> = t;
+ let session: Pick<EveEvalSession, "state"> = initial.session;
let completed: EveEvalTurn | undefined;
for (let attempt = 0; attempt < 5; attempt += 1) {
const live = t.target.watchTurn(sessionId, { startIndex: requireStreamIndex(session) });
diff --git a/e2e/fixtures/agent-workflow-tools/evals/report-deploy.background.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/report-deploy.background.eval.ts
index 6c634acab..e036b3aae 100644
--- a/e2e/fixtures/agent-workflow-tools/evals/report-deploy.background.eval.ts
+++ b/e2e/fixtures/agent-workflow-tools/evals/report-deploy.background.eval.ts
@@ -6,6 +6,7 @@ export default defineEval({
"A background workflow tool returns a receipt, reports progress, and wakes the agent with its result.",
async test(t) {
const started = await t.send("WORKFLOW-REPORT-START");
+ const conversation = started.session;
started.expectOk();
started.calledTool("report_deploy");
@@ -13,11 +14,11 @@ export default defineEval({
const taskId = readTaskId(receipt.output);
if (taskId === undefined) throw new Error("report_deploy receipt is missing taskId.");
- const sessionId = t.sessionId;
+ const sessionId = conversation.sessionId;
if (sessionId === undefined) throw new Error("Eval has no parent session id.");
const updateLive = t.target.watchTurn(sessionId, {
- startIndex: requireStreamIndex(t, "update wait"),
+ startIndex: requireStreamIndex(started.session, "update wait"),
});
const updateTurn = await updateLive.result();
updateTurn.expectOk();
diff --git a/e2e/fixtures/fixture-tasks/evals/batching.ts b/e2e/fixtures/fixture-tasks/evals/batching.ts
index 135f94965..8abab6466 100644
--- a/e2e/fixtures/fixture-tasks/evals/batching.ts
+++ b/e2e/fixtures/fixture-tasks/evals/batching.ts
@@ -9,8 +9,8 @@ import {
/** Release children independently of parent wakes, retaining the entire parent stream suffix. */
export async function startBlockedFanout(t: EveEvalContext, count: number) {
- let session: TaskEvalSessionDriver = t;
const started = await t.send("TASK-BATCHING-BENCHMARK");
+ let session: TaskEvalSessionDriver = started.session;
started.expectOk();
started.noFailedActions();
started.messageIncludes("TASK-FANOUT-STARTED");
diff --git a/e2e/fixtures/fixture-tasks/evals/lifecycle.ts b/e2e/fixtures/fixture-tasks/evals/lifecycle.ts
index 8660411b4..a50efc54c 100644
--- a/e2e/fixtures/fixture-tasks/evals/lifecycle.ts
+++ b/e2e/fixtures/fixture-tasks/evals/lifecycle.ts
@@ -59,7 +59,7 @@ export function lifecycleDriver(t: EveEvalContext, key: string) {
async start(message: string) {
const turn = (await t.send(message)).expectOk();
sessionId = turn.sessionId;
- streamIndex = requireSessionStreamIndex(t, "Lifecycle setup");
+ streamIndex = requireSessionStreamIndex(turn.session, "Lifecycle setup");
return record(turn);
},
async gate(marker: "A" | "B" | "parent") {
diff --git a/e2e/fixtures/fixture-tasks/evals/shared.ts b/e2e/fixtures/fixture-tasks/evals/shared.ts
index a4b5c9f45..580025496 100644
--- a/e2e/fixtures/fixture-tasks/evals/shared.ts
+++ b/e2e/fixtures/fixture-tasks/evals/shared.ts
@@ -138,7 +138,7 @@ export function requireTaskView(output: unknown, taskId: string): Record<string,
export async function sendAndFollowQueuedTurn(
t: EveEvalContext,
message: string,
- initialSession: TaskEvalSessionDriver = t,
+ initialSession: TaskEvalSessionDriver,
options: FollowQueuedTurnOptions = {},
): Promise<FollowedQueuedTurn> {
let session = initialSession;
diff --git a/e2e/fixtures/fixture-tasks/evals/task.agent.continue.accepted-terminal-available.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.agent.continue.accepted-terminal-available.eval.ts
index 88461aa20..b1ac9da57 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.agent.continue.accepted-terminal-available.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.agent.continue.accepted-terminal-available.eval.ts
@@ -23,7 +23,7 @@ export default defineTaskEval({
setup.expectOk();
const firstTaskId = requireBackgroundTaskId(setup);
- const first = await waitForTaskInput(t, t, "first_gate");
+ const first = await waitForTaskInput(t, setup.session, "first_gate");
t.log("answering first task gate one");
const firstAnswered = await first.session.respond([
{
diff --git a/e2e/fixtures/fixture-tasks/evals/task.agent.steer.accepted-busy.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.agent.steer.accepted-busy.eval.ts
index 55d3df4c1..1905f4052 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.agent.steer.accepted-busy.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.agent.steer.accepted-busy.eval.ts
@@ -31,7 +31,7 @@ export default defineTaskEval({
const initialTaskId = requireBackgroundTaskId(setup);
const initial = await waitForCompletedTask(
t,
- t,
+ setup.session,
"CHILD-TASK-EXCLUSIVITY-VERIFY",
initialTaskId,
);
@@ -40,7 +40,7 @@ export default defineTaskEval({
initialTaskId,
);
- const race = await sendAndFollowQueuedTurn(t, "CHILD-TASK-EXCLUSIVITY-RACE", t, {
+ const race = await sendAndFollowQueuedTurn(t, "CHILD-TASK-EXCLUSIVITY-RACE", initial.session, {
allowFailedActions: true,
});
const raced = race.turn;
diff --git a/e2e/fixtures/fixture-tasks/evals/task.authorization.callback.accepted-current-attempt.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.authorization.callback.accepted-current-attempt.eval.ts
index f7228262a..7e72b54d1 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.authorization.callback.accepted-current-attempt.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.authorization.callback.accepted-current-attempt.eval.ts
@@ -36,7 +36,12 @@ export default defineTaskEval({
});
const taskId = requireBackgroundTaskId(started);
- const required = await waitForAuthorizationEvent(t, t, started, "authorization.required");
+ const required = await waitForAuthorizationEvent(
+ t,
+ started.session,
+ started,
+ "authorization.required",
+ );
required.turn.event("authorization.required", {
count: 1,
data: { authorization: { userCode: AUTHORIZATION_CODE }, name: AUTHORIZATION_NAME },
diff --git a/e2e/fixtures/fixture-tasks/evals/task.dispatch-batch.start.accepted-partial-failure.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.dispatch-batch.start.accepted-partial-failure.eval.ts
index 2102b19c9..7939d152f 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.dispatch-batch.start.accepted-partial-failure.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.dispatch-batch.start.accepted-partial-failure.eval.ts
@@ -62,21 +62,21 @@ export default defineTaskEval({
const firstCompleted = await waitForCompletedTask(
t,
- t,
+ started.session,
"TASK-D6-PARTIAL-FANOUT-VERIFY",
firstTaskId,
);
assertCompletedBusyWorker(firstCompleted, firstTaskId, "TASK-D6-FIRST-SUCCESS");
const thirdCompleted = await waitForCompletedTask(
t,
- t,
+ started.session,
"TASK-D6-PARTIAL-FANOUT-VERIFY",
thirdTaskId,
);
assertCompletedBusyWorker(thirdCompleted, thirdTaskId, "TASK-D6-THIRD-SUCCESS");
const failed = await waitForTaskStatus(
t,
- t,
+ started.session,
"TASK-D6-PARTIAL-FANOUT-UNKNOWN",
failedTaskId,
"failed",
diff --git a/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.accepted-acknowledged.creator-context.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.accepted-acknowledged.creator-context.eval.ts
index 0d174b415..f0600a5b7 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.accepted-acknowledged.creator-context.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.accepted-acknowledged.creator-context.eval.ts
@@ -27,6 +27,7 @@ export default defineTaskEval({
const firstTurn = await t.send("TASK-AUTH-SNAPSHOT-ROOT", {
headers: { authorization: SESSION_INITIATOR_AUTHORIZATION },
});
+ const session = firstTurn.session;
firstTurn.expectOk();
firstTurn.messageIncludes("TASK-AUTH-SNAPSHOT-ROOT-ACK");
firstTurn.calledTool("snapshot_whoami", { count: 1, status: "completed" });
@@ -38,7 +39,7 @@ export default defineTaskEval({
// A route-authenticated, schedule-shaped turn has no current session principal.
const key = crypto.randomUUID();
- const started = await t.send(`TASK-AUTH-SNAPSHOT ${key}`, {
+ const started = await session.send(`TASK-AUTH-SNAPSHOT ${key}`, {
headers: { [ANONYMOUS_TASK_CREATOR_HEADER]: "1" },
});
started.expectOk();
@@ -55,7 +56,7 @@ export default defineTaskEval({
const gateToken = await waitForGate(t, sessionId, key);
// Another authenticated caller takes the last parent turn before nested dispatch.
- const lastParentTurn = await t.send("TASK-AUTH-SNAPSHOT-LATER", {
+ const lastParentTurn = await session.send("TASK-AUTH-SNAPSHOT-LATER", {
headers: { authorization: LATER_PARENT_CALLER_AUTHORIZATION },
});
lastParentTurn.expectOk();
@@ -68,7 +69,7 @@ export default defineTaskEval({
);
await releaseGate(t, sessionId, key, gateToken);
- const childSessionId = await waitForSubagent(t, t);
+ const childSessionId = await waitForSubagent(t, lastParentTurn.session);
const child = await t.target.watchTurn(childSessionId).result();
child.expectOk();
child.noFailedActions();
diff --git a/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.rejected-unreachable.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.rejected-unreachable.eval.ts
index ea97b5e39..e84581b98 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.rejected-unreachable.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.dispatch.start.rejected-unreachable.eval.ts
@@ -45,7 +45,13 @@ export default defineTaskEval({
);
});
- const failed = await waitForTaskStatus(t, t, "TASK-A3-UNKNOWN-VERIFY", taskId, "failed");
+ const failed = await waitForTaskStatus(
+ t,
+ started.session,
+ "TASK-A3-UNKNOWN-VERIFY",
+ taskId,
+ "failed",
+ );
failed.expectOk();
failed.messageIncludes("TASK-A3-UNKNOWN");
const view = requireTaskView(failed.requireToolCall("task_cancel").output, taskId);
diff --git a/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.local.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.local.eval.ts
index c02a5880c..902b1dd78 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.local.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.local.eval.ts
@@ -21,7 +21,7 @@ export default defineTaskEval({
});
const taskId = requireBackgroundTaskId(started);
- const first = await waitForTaskInput(t, t, "first_gate");
+ const first = await waitForTaskInput(t, started.session, "first_gate");
const answered = await first.session.respond([
{
optionId: "approve",
diff --git a/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.remote.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.remote.eval.ts
index 22d577a6d..997f7d39d 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.remote.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.input.answer.accepted-complete.remote.eval.ts
@@ -44,7 +44,7 @@ export default defineTaskEval({
data: { requests: [{ action: { toolName: "remote_gate" } }] },
}).label("remote input callback reaches the parent");
t.log("Waiting for the remote input callback to reach the parent.");
- const gate = await waitForTaskInput(t, t, "remote_gate");
+ const gate = await waitForTaskInput(t, started.session, "remote_gate");
const answered = await gate.session.respond([
{
optionId: "approve",
diff --git a/e2e/fixtures/fixture-tasks/evals/task.input.route.observed-stale-unrouted.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.input.route.observed-stale-unrouted.eval.ts
index d1982cafc..74481948f 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.input.route.observed-stale-unrouted.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.input.route.observed-stale-unrouted.eval.ts
@@ -22,7 +22,7 @@ export default defineTaskEval({
started.expectOk();
const taskId = requireBackgroundTaskId(started);
- const first = await waitForTaskInput(t, t, "first_gate");
+ const first = await waitForTaskInput(t, started.session, "first_gate");
const firstAnswer = await first.session.respond([
{
optionId: "approve",
diff --git a/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-all-terminal.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-all-terminal.eval.ts
index e46479ee1..1258e8aca 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-all-terminal.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-all-terminal.eval.ts
@@ -42,7 +42,7 @@ export default defineTaskEval({
`${FAN_IN_SIZE} distinct background task receipts`,
),
);
- const blocked = await waitForReleaseRequests(t, t, started);
+ const blocked = await waitForReleaseRequests(t, started.session, started);
const released = await blocked.session.respond(
FAN_IN_CALLS.map(({ marker }) => ({
diff --git a/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts
index 7a23c0109..f322ce728 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts
@@ -30,7 +30,7 @@ export default defineTaskEval({
const started = (await t.send("TASK-FAN-IN")).expectOk();
started.messageIncludes("TASK-FAN-IN-STARTED");
started.calledSubagent("fanout-worker", { count: MARKERS.length });
- let session: TaskEvalSessionDriver = t;
+ let session: TaskEvalSessionDriver = started.session;
const requests = new Map<string, InputRequest>();
const setupEvents: EveEvalTurn["events"][number][] = [];
collectRequests(started);
diff --git a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.accepted-nonterminal.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.accepted-nonterminal.eval.ts
index b7cf53ec7..9fbd4bd51 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.accepted-nonterminal.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.accepted-nonterminal.eval.ts
@@ -27,7 +27,7 @@ export default defineTaskEval({
});
const taskId = requireBackgroundTaskId(started);
- const blocked = await waitForTaskInput(t, t, "release");
+ const blocked = await waitForTaskInput(t, started.session, "release");
const cancelled = await sendAndFollowQueuedTurn(t, "TASK-CANCEL-NOW", blocked.session);
cancelled.turn.expectOk();
cancelled.turn.messageIncludes("TASK-CANCEL-DONE");
diff --git a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.default-preserves-nonterminal.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.default-preserves-nonterminal.eval.ts
index 134cf8c86..ea6db79cc 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.default-preserves-nonterminal.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.default-preserves-nonterminal.eval.ts
@@ -13,7 +13,7 @@ export default defineTaskEval({
const started = await t.send("TASK-CANCEL-SETUP");
started.expectOk();
const taskId = requireBackgroundTaskId(started);
- const blocked = await waitForTaskInput(t, t, "release");
+ const blocked = await waitForTaskInput(t, started.session, "release");
const sessionId = blocked.session.sessionId;
if (sessionId === undefined) throw new Error("Task parent has no session id.");
diff --git a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.noop-already-cancelled.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.noop-already-cancelled.eval.ts
index e0e7db927..b16ac58c1 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.noop-already-cancelled.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.noop-already-cancelled.eval.ts
@@ -31,7 +31,7 @@ export default defineTaskEval({
});
const taskId = requireBackgroundTaskId(started);
- const blocked = await waitForTaskInput(t, t, "release");
+ const blocked = await waitForTaskInput(t, started.session, "release");
const cancelled = await sendAndFollowQueuedTurn(t, "TASK-CANCEL-NOW", blocked.session);
cancelled.turn.expectOk();
cancelled.turn.calledTool("task_cancel", { input: { taskIds: [taskId] } });
diff --git a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.owned-session-accepted-nonterminal.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.owned-session-accepted-nonterminal.eval.ts
index a64585528..e0b046181 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.owned-session-accepted-nonterminal.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.cancel.owned-session-accepted-nonterminal.eval.ts
@@ -20,7 +20,7 @@ export default defineTaskEval({
const started = await t.send("TASK-CANCEL-SETUP");
started.expectOk();
const taskId = requireBackgroundTaskId(started);
- const blocked = await waitForTaskInput(t, t, "release");
+ const blocked = await waitForTaskInput(t, started.session, "release");
const sessionId = blocked.session.sessionId;
if (sessionId === undefined) throw new Error("Task parent has no session id.");
diff --git a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts
index 2c5494f29..11f6c745d 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts
@@ -27,7 +27,9 @@ export default defineTaskEval({
});
const taskId = requireBackgroundTaskId(started);
- const completed = await waitForTaskNotification(t, t, taskId, "completed", [started]);
+ const completed = await waitForTaskNotification(t, started.session, taskId, "completed", [
+ started,
+ ]);
completed.turn.expectOk();
const report = completed.turn.message;
if (report === undefined) throw new Error("Parent did not return the child's tool report.");
diff --git a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.fail.accepted-nonterminal.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.fail.accepted-nonterminal.eval.ts
index 72c21a7cc..9ad9907ba 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.fail.accepted-nonterminal.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.fail.accepted-nonterminal.eval.ts
@@ -28,7 +28,13 @@ export default defineTaskEval({
});
const taskId = requireBackgroundTaskId(started);
- const failed = await waitForTaskStatus(t, t, "TASK-A2-CHILD-FAILURE-VERIFY", taskId, "failed");
+ const failed = await waitForTaskStatus(
+ t,
+ started.session,
+ "TASK-A2-CHILD-FAILURE-VERIFY",
+ taskId,
+ "failed",
+ );
failed.expectOk();
failed.messageIncludes("TASK-A2-FAILED");
const inspected = failed.requireToolCall("task_cancel", { input: { taskIds: [taskId] } });
diff --git a/e2e/fixtures/fixture-tasks/evals/task.parent-interaction.send.accepted-live-children.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.parent-interaction.send.accepted-live-children.eval.ts
index d3f3d511c..250da41e8 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.parent-interaction.send.accepted-live-children.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.parent-interaction.send.accepted-live-children.eval.ts
@@ -33,7 +33,7 @@ export default defineTaskEval({
),
);
- const blockedSession = await waitForReleaseRequests(t, t, started);
+ const blockedSession = await waitForReleaseRequests(t, started.session, started);
const interactive = await sendAndFollowQueuedTurn(
t,
"TASK-FANOUT-INTERACTIVE-CHECK",
diff --git a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts
index d4a569169..24cd57ab8 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts
@@ -36,7 +36,7 @@ export default defineTaskEval({
),
);
- const blocked = await waitForReleaseRequests(t, t, started);
+ const blocked = await waitForReleaseRequests(t, started.session, started);
const released = await blocked.session.respond(
blocked.requests.map((request) => ({
optionId: "approve",
diff --git a/packages/eve/extension-contracts/compatibility/channel/v24.ts b/packages/eve/extension-contracts/compatibility/channel/v24.ts
new file mode 100644
index 000000000..1ae33fff9
--- /dev/null
+++ b/packages/eve/extension-contracts/compatibility/channel/v24.ts
@@ -0,0 +1,16 @@
+import { defineChannel, POST } from "#public/channels/index.js";
+
+export default defineChannel({
+ routes: [
+ POST("/continue/:sessionId", async (_request, { attachSession, params }) => {
+ const result = await attachSession(params.sessionId!).send("Continue.", {
+ auth: null,
+ turnPolicy: "queue",
+ });
+ if (result.status === "session_not_active") {
+ return Response.json({ accepted: false }, { status: 409 });
+ }
+ return Response.json({ sessionId: result.sessionId, deliveryId: result.deliveryId });
+ }),
+ ],
+});
diff --git a/packages/eve/extension-contracts/compatibility/schedule/v12.ts b/packages/eve/extension-contracts/compatibility/schedule/v12.ts
new file mode 100644
index 000000000..8d458bb01
--- /dev/null
+++ b/packages/eve/extension-contracts/compatibility/schedule/v12.ts
@@ -0,0 +1,22 @@
+import { defineChannel } from "#public/channels/index.js";
+import { defineSchedule } from "#public/schedules/index.js";
+
+const updates = defineChannel({
+ routes: [],
+ receive(input, { from }) {
+ return from("daily-updates").send(input.message, { auth: input.auth });
+ },
+});
+
+export default defineSchedule({
+ cron: "0 9 * * *",
+ async run({ to, waitUntil, appAuth }) {
+ const session = await to(updates, {}).send("Prepare the daily summary.", { auth: appAuth });
+ const result = await session.send("Include yesterday's activity.", {
+ auth: appAuth,
+ turnPolicy: "queue",
+ });
+ if (result.status === "session_not_active") return;
+ waitUntil(Promise.resolve(result.sessionId));
+ },
+});
diff --git a/packages/eve/extension-contracts/reports/channel/v25.json b/packages/eve/extension-contracts/reports/channel/v25.json
new file mode 100644
index 000000000..e45395a95
--- /dev/null
+++ b/packages/eve/extension-contracts/reports/channel/v25.json
@@ -0,0 +1,21 @@
+{
+ "kind": "eve-extension-capability-contract",
+ "capability": "channel",
+ "epoch": 25,
+ "sha256": "80bad873b1e2cf5ec8455f19c8ba839cddd5cb3dd917d28be1b5eddeb0b15e4f",
+ "exports": [
+ "DELETE",
+ "GET",
+ "HEAD",
+ "OPTIONS",
+ "PATCH",
+ "POST",
+ "PUT",
+ "WS",
+ "createWebSocketUpgradeServer",
+ "defineChannel",
+ "disableRoute",
+ "isChannel",
+ "isDisabledRouteSentinel"
+ ]
+}
diff --git a/packages/eve/extension-contracts/reports/schedule/v13.json b/packages/eve/extension-contracts/reports/schedule/v13.json
new file mode 100644
index 000000000..640389a06
--- /dev/null
+++ b/packages/eve/extension-contracts/reports/schedule/v13.json
@@ -0,0 +1,14 @@
+{
+ "kind": "eve-extension-capability-contract",
+ "capability": "schedule",
+ "epoch": 13,
+ "sha256": "4957dc66f23e9c3c8fe15e7e67581be85ac6b45d729b42fac9d904e18cb54d56",
+ "exports": [
+ "ScheduleDefinition",
+ "ScheduleHandlerArgs",
+ "ScheduleRunHandler",
+ "ScheduleToFn",
+ "TypedReceiveTarget",
+ "defineSchedule"
+ ]
+}
diff --git a/packages/eve/src/channel/session.ts b/packages/eve/src/channel/session.ts
index c7532ff6c..099ab6dac 100644
--- a/packages/eve/src/channel/session.ts
+++ b/packages/eve/src/channel/session.ts
@@ -76,7 +76,11 @@ interface SessionDeliveryOptions {
}
/** Options for sending a message through a fixed session handle. */
-export type SessionSendOptions = SessionDeliveryOptions & { readonly turnPolicy?: TurnPolicy };
+export type SessionSendOptions = SessionDeliveryOptions & {
+ /** Initial workflow title for a prewarmed session. */
+ readonly title?: string;
+ readonly turnPolicy?: TurnPolicy;
+};
/** Options for answering pending input requests through a fixed session handle. */
export type SessionRespondOptions = SessionDeliveryOptions;
@@ -122,6 +126,7 @@ export function createSession(
payload,
requestId: metadata.requestId,
turnPolicy: options.turnPolicy ?? metadata.turnPolicy ?? DEFAULT_TURN_POLICY,
+ title: options.title,
};
return await runtime.dispatchSession({
command: caller === undefined ? commandWithoutCaller : { ...commandWithoutCaller, caller },
diff --git a/packages/eve/src/channel/types.ts b/packages/eve/src/channel/types.ts
index 763b40241..a33c2fe6b 100644
--- a/packages/eve/src/channel/types.ts
+++ b/packages/eve/src/channel/types.ts
@@ -206,6 +206,8 @@ export type SessionCommand =
| {
readonly auth?: SessionAuthContext | null;
readonly caller?: TurnCaller;
+ /** Initial workflow title when delivering to a prewarmed session. */
+ readonly title?: string;
readonly kind: "send";
readonly payload: DeliverPayload;
readonly delivery?: ChannelDeliveryMetadata;
@@ -230,7 +232,11 @@ export type SessionCommand =
export type SessionSendCommandResult =
| { readonly status: "accepted"; readonly sessionId: string; readonly deliveryId?: string }
- | { readonly status: "session_not_active" };
+ | {
+ readonly status: "session_not_active";
+ /** The workflow exists but its inbox is not yet available; no delivery was accepted. */
+ readonly retryable?: boolean;
+ };
/** Result of terminally resetting a session. */
export type ResetSessionResult =
@@ -269,6 +275,8 @@ export interface DispatchSessionInput<TCommand extends SessionCommand = SessionC
* metadata so both cross the durable hook boundary outside adapter-owned data.
*/
export interface DeliverHookPayload {
+ /** Initial workflow title; ignored once session initialization has run. */
+ readonly title?: string;
readonly auth?: SessionAuthContext | null;
/** Delegated caller waiting for this turn's settled result. */
readonly caller?: TurnCaller;
@@ -529,7 +537,8 @@ export interface RunInput {
*/
readonly initiatorAuth?: SessionAuthContext | null;
readonly input: {
- readonly message: string | UserContent;
+ /** Omitted only when creating a conversation session before its first turn. */
+ readonly message?: string | UserContent;
readonly context?: readonly string[];
readonly outputSchema?: JsonObject;
};
diff --git a/packages/eve/src/cli/dev/tui/runner.test.ts b/packages/eve/src/cli/dev/tui/runner.test.ts
index ba283ff57..26676c5d4 100644
--- a/packages/eve/src/cli/dev/tui/runner.test.ts
+++ b/packages/eve/src/cli/dev/tui/runner.test.ts
@@ -6,7 +6,9 @@ import {
MessageResponse,
type AgentInfoResult,
type ClientSession,
+ type CreateSessionOptions,
type MessageStreamEvent,
+ type SendTurnInput,
} from "#client/index.js";
import { getApplicationInfo } from "#internal/application/paths.js";
import { stampTestEvent } from "#internal/testing/events.js";
@@ -431,10 +433,12 @@ function stubSession(): ClientSession {
}
function mockSessionCreation(client: Client, session: ClientSession) {
- return vi.spyOn(client.sessions, "create").mockImplementation(async (input) => ({
- response: await session.send(input.message, input),
- session,
- }));
+ return vi
+ .spyOn(client.sessions, "create")
+ .mockImplementation(async (input: CreateSessionOptions | SendTurnInput = {}) => {
+ if (!("message" in input)) return { session };
+ return { response: await session.send(input.message, input), session };
+ });
}
/**
@@ -2388,10 +2392,7 @@ describe("EveTUIRunner failure rendering", () => {
{ type: "session.failed", data: { ...failureData, sessionId: "s0" } },
]);
const client = stubClient();
- vi.spyOn(client.sessions, "create").mockImplementation(async (input) => {
- const session = sessionYielding([]);
- return { response: await session.send(input.message, input), session };
- });
+ mockSessionCreation(client, sessionYielding([]));
const renderer: AgentTUIRenderer = {
readPrompt: vi.fn(async () => prompts.shift()),
diff --git a/packages/eve/src/client/eve-agent-projection.ts b/packages/eve/src/client/eve-agent-projection.ts
new file mode 100644
index 000000000..5bab9ebef
--- /dev/null
+++ b/packages/eve/src/client/eve-agent-projection.ts
@@ -0,0 +1,46 @@
+import type { EveAgentReducer, EveAgentReducerEvent } from "#client/reducer.js";
+
+/** Owns reducer replay when optimistic events are replaced by server events. */
+export class EveAgentProjection<TData> {
+ readonly #reducer: EveAgentReducer<TData>;
+ #events: readonly EveAgentReducerEvent[];
+ #data: TData;
+
+ constructor(reducer: EveAgentReducer<TData>, events: readonly EveAgentReducerEvent[]) {
+ this.#reducer = reducer;
+ this.#events = events;
+ this.#data = this.#reduce();
+ }
+
+ get data(): TData {
+ return this.#data;
+ }
+
+ reset(): void {
+ this.#events = [];
+ this.#data = this.#reducer.initial();
+ }
+
+ append(event: EveAgentReducerEvent): void {
+ this.#events = [...this.#events, event];
+ this.#data = this.#reducer.reduce(this.#data, event);
+ }
+
+ replace(
+ predicate: (event: EveAgentReducerEvent) => boolean,
+ replacement: EveAgentReducerEvent,
+ ): void {
+ const index = this.#events.findIndex(predicate);
+ this.#events =
+ index === -1
+ ? [...this.#events, replacement]
+ : this.#events.map((event, i) => (i === index ? replacement : event));
+ this.#data = this.#reduce();
+ }
+
+ #reduce(): TData {
+ let data = this.#reducer.initial();
+ for (const event of this.#events) data = this.#reducer.reduce(data, event);
+ return data;
+ }
+}
diff --git a/packages/eve/src/client/eve-agent-store-helpers.ts b/packages/eve/src/client/eve-agent-store-helpers.ts
index 970cdbfbc..dad193c64 100644
--- a/packages/eve/src/client/eve-agent-store-helpers.ts
+++ b/packages/eve/src/client/eve-agent-store-helpers.ts
@@ -1,4 +1,7 @@
-import type { SendTurnPayload } from "#client/types.js";
+import { updatePendingAuthorizations } from "#client/session-utils.js";
+import type { ActiveTurn } from "#client/eve-agent-store-state.js";
+import type { MessageResponse } from "#client/message-response.js";
+import type { CancelSessionResult, SendTurnPayload } from "#client/types.js";
import { isCurrentTurnBoundaryEvent, type MessageStreamEvent } from "#protocol/message.js";
import type { UserContent } from "ai";
@@ -17,14 +20,6 @@ export function collectPendingAuthorizations(events: readonly MessageStreamEvent
return pending;
}
-export function updatePendingAuthorizations(pending: Set<string>, event: MessageStreamEvent): void {
- if (event.type === "authorization.required" && event.data.webhookUrl !== undefined) {
- pending.add(event.data.name);
- } else if (event.type === "authorization.completed") {
- pending.delete(event.data.name);
- }
-}
-
export function assertExclusiveTurnInput(input: SendTurnPayload): void {
const hasMessage = input.message !== undefined;
const hasResponses = input.inputResponses !== undefined;
@@ -77,3 +72,59 @@ export function toTerminalStreamFailureError(event: MessageStreamEvent): Error |
error.name = event.data.code;
return error;
}
+
+export function createActiveTurn(
+ cancel: (turn: ActiveTurn) => Promise<CancelSessionResult>,
+): ActiveTurn {
+ const response = Promise.withResolvers<MessageResponse | undefined>();
+ const completion = Promise.withResolvers<void>();
+ const turn: ActiveTurn = {
+ abortController: new AbortController(),
+ acceptedFollowUps: 0,
+ cancel: () => cancel(turn),
+ completion: completion.promise,
+ followUpDispatches: new Set(),
+ receivedFollowUps: 0,
+ receivedFollowUpEvents: new Set(),
+ followUpSubmissionIds: new Set(),
+ resolveCompletion: completion.resolve,
+ response: response.promise,
+ resolveResponse: response.resolve,
+ };
+ return turn;
+}
+
+export async function followSteeredTurns(
+ turn: ActiveTurn,
+ events: AsyncIterable<MessageStreamEvent>,
+ isActive: () => boolean,
+): Promise<void> {
+ while (turn.followUpDispatches.size > 0) {
+ await Promise.allSettled(turn.followUpDispatches);
+ }
+ if (turn.receivedFollowUps >= turn.acceptedFollowUps) return;
+ for await (const event of events) {
+ if (!isActive()) return;
+ if (turn.receivedFollowUpEvents.delete(event)) turn.receivedFollowUps += 1;
+ if (isCurrentTurnBoundaryEvent(event)) {
+ while (turn.followUpDispatches.size > 0) {
+ await Promise.allSettled(turn.followUpDispatches);
+ }
+ if (turn.receivedFollowUps >= turn.acceptedFollowUps) return;
+ }
+ }
+}
+
+/** Aborts a caller's wait without cancelling shared work owned by the store. */
+export async function waitWithSignal<T>(promise: Promise<T>, signal?: AbortSignal): Promise<T> {
+ if (signal === undefined) return await promise;
+ const aborted = Promise.withResolvers<never>();
+ const onAbort = () => aborted.reject(signal.reason);
+ signal.addEventListener("abort", onAbort, { once: true });
+ if (signal.aborted) onAbort();
+ try {
+ return await Promise.race([aborted.promise, promise]);
+ } finally {
+ signal.removeEventListener("abort", onAbort);
+ }
+}
diff --git a/packages/eve/src/client/eve-agent-store-state.ts b/packages/eve/src/client/eve-agent-store-state.ts
index 02cd29cc2..08781815a 100644
--- a/packages/eve/src/client/eve-agent-store-state.ts
+++ b/packages/eve/src/client/eve-agent-store-state.ts
@@ -1,5 +1,68 @@
import type { MessageResponse } from "#client/message-response.js";
-import type { CancelSessionResult } from "#client/types.js";
+import type { EveAgentReducer } from "#client/reducer.js";
+import type { ClientSession } from "#client/session.js";
+import type { MessageStreamEvent } from "#protocol/message.js";
+import type {
+ CancelSessionResult,
+ ClientAuth,
+ ClientSessionState,
+ HeadersValue,
+ SendTurnPayload,
+} from "#client/types.js";
+
+/**
+ * Lifecycle state of an eve frontend session store: `ready` (idle),
+ * `resuming` (checking an attached session for continuation), `submitted`
+ * (turn sent, awaiting the first event), `streaming` (events arriving), and
+ * `error` (session creation, streaming, resume, or a turn failed).
+ */
+export type EveAgentStoreStatus = "error" | "ready" | "resuming" | "streaming" | "submitted";
+
+/**
+ * Prepares one outbound turn immediately before the client sends it, for
+ * example to attach fresh one-turn page state through `clientContext`.
+ */
+export type PrepareSend = (input: SendTurnPayload) => SendTurnPayload | Promise<SendTurnPayload>;
+
+/** Immutable projected state of an eve frontend session store. */
+export interface EveAgentStoreSnapshot<TData> {
+ readonly data: TData;
+ /** The latest session creation, stream, resume, or turn failure. */
+ readonly error: Error | undefined;
+ readonly events: readonly MessageStreamEvent[];
+ readonly session: ClientSessionState | undefined;
+ readonly status: EveAgentStoreStatus;
+}
+
+/**
+ * Hooks invoked during session creation and turn processing. The observation callbacks
+ * do not alter execution; `prepareSend` may replace the outbound turn payload.
+ */
+export interface EveAgentStoreCallbacks<TData> {
+ readonly onError?: (error: Error) => void;
+ readonly onEvent?: (event: MessageStreamEvent) => void;
+ readonly onFinish?: (snapshot: EveAgentStoreSnapshot<TData>) => void;
+ readonly onSessionChange?: (session: ClientSessionState | undefined) => void;
+ readonly prepareSend?: PrepareSend;
+}
+
+/**
+ * Configuration for constructing an eve frontend session store. Pass either
+ * connection options for a store-owned session or an existing `ClientSession`.
+ * Saved events must be an ordered prefix of the same session stream.
+ */
+export interface EveAgentStoreInit<TData> {
+ readonly prewarm?: boolean;
+ readonly auth?: ClientAuth;
+ readonly headers?: HeadersValue;
+ readonly host?: string;
+ /** Ordered prefix of the session stream used to rehydrate projected state. */
+ readonly initialEvents?: readonly MessageStreamEvent[];
+ readonly initialSession?: ClientSessionState;
+ readonly optimistic?: boolean;
+ readonly reducer: EveAgentReducer<TData>;
+ readonly session?: ClientSession;
+}
export interface PendingMessageSubmission {
readonly createdAt: number;
@@ -14,6 +77,7 @@ export interface ActiveTurn {
readonly completion: Promise<void>;
readonly followUpDispatches: Set<Promise<void>>;
receivedFollowUps: number;
+ readonly receivedFollowUpEvents: Set<MessageStreamEvent>;
readonly followUpSubmissionIds: Set<string>;
readonly resolveCompletion: () => void;
readonly response: Promise<MessageResponse | undefined>;
diff --git a/packages/eve/src/client/eve-agent-store.test.ts b/packages/eve/src/client/eve-agent-store.test.ts
index f45a2e809..d60a1334c 100644
--- a/packages/eve/src/client/eve-agent-store.test.ts
+++ b/packages/eve/src/client/eve-agent-store.test.ts
@@ -65,13 +65,13 @@ function streamingTurnEvents(): MessageStreamEvent[] {
] as UnstampedMessageStreamEvent[]);
}
-function startedResponse(): Response {
+function startedResponse(deliveryId = "delivery_1"): Response {
return new Response(
JSON.stringify({
ok: true,
sessionId: "session_1",
status: "accepted",
- deliveryId: "delivery_1",
+ deliveryId,
}),
{
headers: { "content-type": "application/json", [EVE_SESSION_ID_HEADER]: "session_1" },
@@ -185,11 +185,331 @@ function preV20MessageCompletedEvent(): MessageStreamEvent {
} as MessageStreamEvent;
}
+const cleanupStores: Array<() => void> = [];
+function createStore<TData>(
+ init: ConstructorParameters<typeof EveAgentStore<TData>>[0],
+): EveAgentStore<TData> {
+ const store = new EveAgentStore<TData>(init);
+ cleanupStores.push(() => detachEveAgentStore(store));
+ return store;
+}
+
afterEach(() => {
+ for (const cleanup of cleanupStores.splice(0)) cleanup();
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
+describe("EveAgentStore prewarming", () => {
+ it("settles a send aborted during preparation without creating a session", async () => {
+ const fetchMock = vi.spyOn(globalThis, "fetch");
+ const store = createStore({ reducer: defaultMessageReducer() });
+ const preparation = Promise.withResolvers<never>();
+ store.setCallbacks({ prepareSend: () => preparation.promise });
+ const controller = new AbortController();
+ const send = store.send({ message: "Hello", signal: controller.signal });
+ controller.abort();
+ await send;
+ expect(fetchMock).not.toHaveBeenCalled();
+ expect(store.snapshot.status).toBe("ready");
+ });
+
+ it("aborts a send waiting for prewarm without cancelling shared session creation", async () => {
+ const accepted = Promise.withResolvers<Response>();
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockReturnValueOnce(accepted.promise)
+ .mockResolvedValueOnce(controlledStreamResponse().response);
+ const store = createStore({ reducer: defaultMessageReducer() });
+ const prewarm = store.prewarm();
+ const controller = new AbortController();
+ const send = store.send({ message: "Hello", signal: controller.signal });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledOnce());
+ controller.abort();
+ await send;
+ expect(store.snapshot.status).toBe("ready");
+ expect(fetchMock.mock.calls[0]![1]!.signal!.aborted).toBe(false);
+ accepted.resolve(startedResponse());
+ await prewarm;
+ expect(store.snapshot.session?.sessionId).toBe("session_1");
+ expect(fetchMock.mock.calls.filter(([, init]) => init?.method === "POST")).toHaveLength(1);
+ });
+
+ it.each(["reset", "detach"] as const)("aborts prewarm transport on %s", async (action) => {
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(
+ (_request, init) =>
+ new Promise((_resolve, reject) => {
+ init?.signal?.addEventListener("abort", () => reject(init.signal?.reason), {
+ once: true,
+ });
+ }),
+ );
+ const store = createStore({ reducer: defaultMessageReducer() });
+ const prewarm = expect(store.prewarm()).rejects.toMatchObject({ name: "AbortError" });
+ const send = store.send({ message: "Hello" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledOnce());
+ if (action === "reset") store.reset();
+ else detachEveAgentStore(store);
+ await Promise.all([prewarm, send]);
+ expect(store.snapshot.status).toBe("ready");
+ expect(store.snapshot.error).toBeUndefined();
+ expect(store.snapshot.session).toBeUndefined();
+ });
+
+ it("reports a standalone creation failure and allows another prewarm", async () => {
+ const error = new Error("create failed");
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockRejectedValueOnce(error)
+ .mockResolvedValueOnce(startedResponse())
+ .mockResolvedValueOnce(streamResponse(stampTestEvents([createSessionWaitingEvent()])));
+ const onError = vi.fn();
+ const store = createStore({ reducer: defaultMessageReducer() });
+ store.setCallbacks({ onError });
+
+ await expect(store.prewarm()).rejects.toBe(error);
+ expect(store.snapshot.status).toBe("error");
+ expect(store.snapshot.error).toBe(error);
+ expect(onError).toHaveBeenCalledExactlyOnceWith(error);
+
+ await store.prewarm();
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(3));
+ expect(store.snapshot.session?.sessionId).toBe("session_1");
+ expect(store.snapshot.status).toBe("ready");
+ expect(store.snapshot.error).toBeUndefined();
+ });
+
+ it("rejects prewarm when the first send it joined fails to create a session", async () => {
+ const accepted = Promise.withResolvers<Response>();
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockReturnValueOnce(accepted.promise);
+ const onError = vi.fn();
+ const store = createStore({ reducer: defaultMessageReducer() });
+ store.setCallbacks({ onError });
+ const error = new Error("create failed");
+
+ const send = store.send({ message: "Hello" });
+ const prewarm = expect(store.prewarm()).rejects.toBe(error);
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledOnce());
+ accepted.reject(error);
+ await Promise.all([send, prewarm]);
+
+ expect(store.snapshot.session).toBeUndefined();
+ expect(store.snapshot.status).toBe("error");
+ expect(onError).toHaveBeenCalledExactlyOnceWith(error);
+ });
+
+ it.each([false, true])(
+ "retries a waiting send after prewarm fails (send fails: %s)",
+ async (sendFails) => {
+ const accepted = Promise.withResolvers<Response>();
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockReturnValueOnce(accepted.promise);
+ const sendError = new Error("send failed");
+ if (sendFails) fetchMock.mockRejectedValueOnce(sendError);
+ else
+ fetchMock
+ .mockResolvedValueOnce(startedResponse())
+ .mockResolvedValueOnce(streamResponse(turnEvents()));
+ const onError = vi.fn();
+ const store = createStore({ reducer: defaultMessageReducer() });
+ store.setCallbacks({ onError });
+ const prewarmError = new Error("prewarm failed");
+
+ const prewarm = expect(store.prewarm()).rejects.toBe(prewarmError);
+ const send = store.send({ message: "Hello" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledOnce());
+ accepted.reject(prewarmError);
+ await Promise.all([prewarm, send]);
+
+ expect(fetchMock.mock.calls[1]![0]).toBe("/eve/v1/session");
+ expect(JSON.parse(String(fetchMock.mock.calls[1]![1]?.body))).toMatchObject({
+ message: "Hello",
+ });
+ if (sendFails) {
+ expect(fetchMock).toHaveBeenCalledTimes(2);
+ expect(onError).toHaveBeenCalledExactlyOnceWith(sendError);
+ expect(store.snapshot.error).toBe(sendError);
+ expect(store.snapshot.status).toBe("error");
+ } else {
+ expect(fetchMock).toHaveBeenCalledTimes(3);
+ expect(onError).not.toHaveBeenCalled();
+ expect(store.snapshot.error).toBeUndefined();
+ expect(store.snapshot.status).toBe("ready");
+ expect(store.snapshot.session?.sessionId).toBe("session_1");
+ }
+ },
+ );
+
+ it.each(["reset", "detach"] as const)(
+ "does not retry a waiting send after %s",
+ async (action) => {
+ const accepted = Promise.withResolvers<Response>();
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockReturnValueOnce(accepted.promise);
+ const onError = vi.fn();
+ const store = createStore({ reducer: defaultMessageReducer() });
+ store.setCallbacks({ onError });
+ const error = new Error("prewarm failed");
+
+ const prewarm = expect(store.prewarm()).rejects.toBe(error);
+ const send = store.send({ message: "Hello" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledOnce());
+ if (action === "reset") store.reset();
+ else detachEveAgentStore(store);
+ accepted.reject(error);
+ await Promise.all([prewarm, send]);
+
+ expect(fetchMock).toHaveBeenCalledOnce();
+ expect(onError).not.toHaveBeenCalled();
+ expect(store.snapshot.session).toBeUndefined();
+ expect(store.snapshot.status).toBe("ready");
+ },
+ );
+
+ it("shares one creation request without starting a turn", async () => {
+ const accepted = Promise.withResolvers<Response>();
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockReturnValueOnce(accepted.promise)
+ .mockResolvedValueOnce(streamResponse(stampTestEvents([createSessionWaitingEvent()])));
+ const onFinish = vi.fn();
+ const onSessionChange = vi.fn();
+ const prepareSend = vi.fn();
+ const store = createStore({ reducer: defaultMessageReducer() });
+ store.setCallbacks({ onFinish, onSessionChange, prepareSend });
+
+ const first = store.prewarm();
+ const second = store.prewarm();
+ expect(first).toBe(second);
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledOnce());
+
+ accepted.resolve(
+ Response.json({ ok: true, sessionId: "session_1", status: "accepted" }, { status: 202 }),
+ );
+ await first;
+
+ expect(fetchMock.mock.calls[0]![1]!.body).toBeUndefined();
+ expect(store.snapshot.session?.sessionId).toBe("session_1");
+ expect(store.snapshot.status).toBe("ready");
+ expect(onSessionChange).toHaveBeenCalledWith({ sessionId: "session_1", streamIndex: 0 });
+ expect(prepareSend).not.toHaveBeenCalled();
+ expect(onFinish).not.toHaveBeenCalled();
+ });
+
+ it("discards a creation result after reset", async () => {
+ const accepted = Promise.withResolvers<Response>();
+ vi.spyOn(globalThis, "fetch").mockReturnValue(accepted.promise);
+ const store = createStore({ reducer: defaultMessageReducer() });
+
+ const prewarm = store.prewarm();
+ store.reset();
+ accepted.resolve(
+ Response.json({ ok: true, sessionId: "stale_session", status: "accepted" }, { status: 202 }),
+ );
+ await prewarm;
+
+ expect(store.snapshot.session).toBeUndefined();
+ });
+
+ it("sends before any stream event and keeps one stream across turns", async () => {
+ const live = controlledStreamResponse();
+ const accepted = Promise.withResolvers<Response>();
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockReturnValueOnce(accepted.promise)
+ .mockResolvedValueOnce(live.response)
+ .mockImplementation(async () => startedResponse());
+ const store = createStore({ reducer: defaultMessageReducer() });
+ const events = turnEvents().map((event) => ({
+ ...event,
+ meta: { ...event.meta, deliveryIds: ["delivery_1"] },
+ }));
+
+ const prewarm = store.prewarm();
+ const firstSend = store.send({ message: "Hello" });
+ accepted.resolve(startedResponse());
+ await prewarm;
+ expect(store.snapshot.events).toEqual([]);
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(3));
+ for (const event of events) live.emit(event);
+ await firstSend;
+ expect(store.snapshot.status).toBe("ready");
+
+ const secondSend = store.send({ message: "Hello again" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(4));
+ for (const event of events)
+ live.emit({ ...event, meta: { ...event.meta, id: `second-${event.meta.id}` } });
+ await secondSend;
+ expect(store.snapshot.status).toBe("ready");
+ expect(fetchMock.mock.calls.filter(([, init]) => init?.method !== "POST")).toHaveLength(1);
+ expect(store.snapshot.session?.streamIndex).toBe(6);
+ });
+
+ it("honors a send's disabled reconnect policy on an existing prewarmed stream", async () => {
+ const live = controlledStreamResponse();
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockResolvedValueOnce(startedResponse())
+ .mockResolvedValueOnce(live.response)
+ .mockResolvedValueOnce(startedResponse());
+ const store = createStore({ reducer: defaultMessageReducer() });
+ await store.prewarm();
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ const send = store.send({ message: "Hello", streamReconnectPolicy: { reconnect: false } });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(3));
+ live.close();
+ await send;
+ expect(store.snapshot.status).toBe("error");
+ expect(fetchMock).toHaveBeenCalledTimes(3);
+ });
+
+ it("uses the latest turn headers when the continuous stream reconnects", async () => {
+ const live = controlledStreamResponse();
+ const reconnected = controlledStreamResponse();
+ const streamAuthorizations: Array<string | null> = [];
+ let delivery = 0;
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) => {
+ if (init?.method === "POST") return startedResponse(`delivery_${(delivery += 1)}`);
+ streamAuthorizations.push(new Headers(init?.headers).get("authorization"));
+ return streamAuthorizations.length === 1 ? live.response : reconnected.response;
+ });
+ const store = createStore({
+ initialSession: { sessionId: "session_1", streamIndex: 0 },
+ reducer: defaultMessageReducer(),
+ });
+ const events = turnEvents();
+ const emitTurn = (deliveryId: string, idPrefix: string) => {
+ for (const event of events) {
+ live.emit({
+ ...event,
+ meta: { ...event.meta, deliveryIds: [deliveryId], id: `${idPrefix}-${event.meta.id}` },
+ });
+ }
+ };
+
+ const first = store.send({
+ headers: { authorization: "Bearer old" },
+ message: "Hello",
+ streamReconnectPolicy: {
+ streamIdleReconnectPolicy: { baseDelayMs: 1, maxAttempts: 5, maxDelayMs: 1 },
+ },
+ });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ emitTurn("delivery_1", "first");
+ await first;
+
+ const second = store.send({
+ headers: { authorization: "Bearer fresh" },
+ message: "Hello again",
+ });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(3));
+ emitTurn("delivery_2", "second");
+ await second;
+
+ live.close();
+ await vi.waitFor(() => expect(streamAuthorizations).toHaveLength(2));
+ expect(streamAuthorizations).toEqual(["Bearer old", "Bearer fresh"]);
+ });
+});
+
describe("EveAgentStore stream overlap", () => {
it("reconstructs a split message across an in-memory stream reconnect", async () => {
const events = streamingTurnEvents();
@@ -198,7 +518,7 @@ describe("EveAgentStore stream overlap", () => {
.mockResolvedValueOnce(startedResponse())
.mockResolvedValueOnce(disconnectingStreamResponse(events.slice(0, 3)))
.mockResolvedValueOnce(streamResponse(events.slice(3)));
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
const streamingText: string[] = [];
store.subscribe(() => {
const part = store.snapshot.data.messages.at(-1)?.parts.at(-1);
@@ -253,7 +573,7 @@ describe("EveAgentStore stream overlap", () => {
.mockResolvedValueOnce(startedResponse())
.mockResolvedValueOnce(versionedDisconnectingStreamResponse(legacyVersion, legacyPrefix))
.mockResolvedValueOnce(versionedStreamResponse("25", current.slice(3)));
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
const streamingText: string[] = [];
store.subscribe(() => {
const part = store.snapshot.data.messages.at(-1)?.parts.at(-1);
@@ -282,7 +602,7 @@ describe("EveAgentStore stream overlap", () => {
it("rejects a prepared turn containing both a message and input responses", async () => {
const fetchMock = vi.spyOn(globalThis, "fetch");
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
store.setCallbacks({
prepareSend: () =>
({
@@ -314,7 +634,7 @@ describe("EveAgentStore stream overlap", () => {
// The server-rendered prefix is replayed ahead of the live tail.
.mockResolvedValueOnce(streamResponse(events));
- const store = new EveAgentStore({
+ const store = createStore({
initialEvents: events.slice(0, 2),
reducer: defaultMessageReducer(),
});
@@ -347,7 +667,7 @@ describe("EveAgentStore stream overlap", () => {
.mockResolvedValueOnce(startedResponse())
.mockResolvedValueOnce(streamResponse([legacy, boundary]));
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
const seen: MessageStreamEvent[] = [];
store.setCallbacks({ onEvent: (event) => seen.push(event) });
@@ -364,7 +684,7 @@ describe("EveAgentStore stream overlap", () => {
it("re-admits events after reset clears the window", async () => {
const events = turnEvents();
- const store = new EveAgentStore({
+ const store = createStore({
initialEvents: events,
reducer: defaultMessageReducer(),
});
@@ -387,6 +707,84 @@ describe("EveAgentStore stream overlap", () => {
});
describe("EveAgentStore session resume", () => {
+ it("resumes an unused prewarmed session and sends its first message on the same stream", async () => {
+ const live = controlledStreamResponse();
+ live.response.headers.set("x-eve-stream-tail-index", "-1");
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockResolvedValueOnce(live.response)
+ .mockResolvedValueOnce(startedResponse());
+ const store = createStore({
+ initialSession: { sessionId: "session_1", streamIndex: 0 },
+ reducer: defaultMessageReducer(),
+ });
+ const resuming = store.resume();
+
+ await vi.waitFor(() => expect(store.snapshot.status).toBe("ready"));
+ await resuming;
+ expect(store.snapshot.events).toEqual([]);
+
+ const sending = store.send({ message: "Hello" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ for (const event of turnEvents()) {
+ live.emit({ ...event, meta: { ...event.meta, deliveryIds: ["delivery_1"] } });
+ }
+ await sending;
+
+ expect(store.snapshot.status).toBe("ready");
+ expect(store.snapshot.data.messages.map((message) => message.role)).toEqual([
+ "user",
+ "assistant",
+ ]);
+ expect(fetchMock.mock.calls[1]![0]).toBe("/eve/v1/session/session_1");
+ expect(fetchMock.mock.calls.filter(([, init]) => init?.method !== "POST")).toHaveLength(1);
+ });
+
+ it.each([true, false])(
+ "settles steering received while following a resumed turn (optimistic=%s)",
+ async (optimistic) => {
+ const events = stampTestEvents([
+ createMessageReceivedEvent({ message: "First", sequence: 0, turnId: "turn_0" }),
+ createTurnStartedEvent({ sequence: 1, turnId: "turn_0" }),
+ createMessageReceivedEvent({ message: "Instead", sequence: 2, turnId: "turn_0" }),
+ createMessageCompletedEvent({
+ finishReason: "stop",
+ message: "Updated reply.",
+ sequence: 3,
+ stepIndex: 0,
+ turnId: "turn_0",
+ }),
+ createSessionWaitingEvent(),
+ ]);
+ const live = controlledStreamResponse();
+ live.response.headers.set("x-eve-stream-tail-index", "1");
+ live.emit(events[0]!);
+ live.emit(events[1]!);
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockResolvedValueOnce(live.response)
+ .mockResolvedValueOnce(startedResponse());
+ const store = createStore({
+ initialSession: { sessionId: "session_1", streamIndex: 0 },
+ optimistic,
+ reducer: defaultMessageReducer(),
+ });
+ const onFinish = vi.fn();
+ store.setCallbacks({ onFinish });
+ const resuming = store.resume();
+ await vi.waitFor(() => expect(store.snapshot.status).toBe("streaming"));
+
+ const steering = store.send({ message: "Instead", turnPolicy: "steer" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ for (const event of events.slice(2)) live.emit(event);
+
+ await vi.waitFor(() => expect(store.snapshot.status).toBe("ready"));
+ await Promise.all([resuming, steering]);
+ expect(store.snapshot.events).toEqual(events);
+ expect(onFinish).toHaveBeenCalledOnce();
+ },
+ );
+
it("finishes a settled replay without waiting for the probe stream to idle", async () => {
const events = turnEvents();
const probe = controlledStreamResponse();
@@ -394,7 +792,7 @@ describe("EveAgentStore session resume", () => {
vi.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(boundedStreamResponse(events))
.mockResolvedValueOnce(probe.response);
- const store = new EveAgentStore({
+ const store = createStore({
initialSession: { sessionId: "session_1", streamIndex: 0 },
reducer: defaultMessageReducer(),
});
@@ -412,7 +810,7 @@ describe("EveAgentStore session resume", () => {
.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(boundedStreamResponse([events[3]!], 3))
.mockResolvedValueOnce(streamResponse(events.slice(4)));
- const store = new EveAgentStore({
+ const store = createStore({
initialEvents: prefix,
initialSession: { sessionId: "session_1", streamIndex: prefix.length },
reducer: defaultMessageReducer(),
@@ -445,7 +843,7 @@ describe("EveAgentStore session resume", () => {
.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(boundedStreamResponse(events.slice(0, 4), 3))
.mockResolvedValueOnce(streamResponse(events.slice(4)));
- const store = new EveAgentStore({
+ const store = createStore({
initialSession: { sessionId: "session_1", streamIndex: 3 },
reducer: defaultMessageReducer(),
});
@@ -477,7 +875,7 @@ describe("EveAgentStore session resume", () => {
.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(boundedStreamResponse([], events.length - 1))
.mockResolvedValueOnce(boundedStreamResponse([], events.length - 1));
- const store = new EveAgentStore({
+ const store = createStore({
initialEvents: events,
initialSession: { sessionId: "session_1", streamIndex: events.length },
reducer: defaultMessageReducer(),
@@ -505,7 +903,7 @@ describe("EveAgentStore session resume", () => {
.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(boundedStreamResponse(events))
.mockResolvedValueOnce(boundedStreamResponse([], events.length - 1));
- const store = new EveAgentStore({
+ const store = createStore({
initialEvents: events.slice(0, 1),
initialSession: { sessionId: "session_1", streamIndex: 2 },
reducer: defaultMessageReducer(),
@@ -541,7 +939,7 @@ describe("EveAgentStore session resume", () => {
.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(boundedStreamResponse([], initialEvents.length - 1))
.mockResolvedValueOnce(live.response);
- const store = new EveAgentStore({
+ const store = createStore({
initialEvents,
initialSession: { sessionId: "session_1", streamIndex: initialEvents.length },
reducer: defaultMessageReducer(),
@@ -573,7 +971,7 @@ describe("EveAgentStore session resume", () => {
}),
])[0]!;
vi.spyOn(globalThis, "fetch").mockResolvedValueOnce(boundedStreamResponse([], 0));
- const store = new EveAgentStore({
+ const store = createStore({
initialEvents: [failed],
initialSession: { sessionId: "session_1", streamIndex: 1 },
reducer: defaultMessageReducer(),
@@ -592,13 +990,12 @@ describe("EveAgentStore session resume", () => {
expect(store.snapshot.error?.message).toBe("Session failed.");
});
- it("probes once beyond a settled replay without following an idle stream", async () => {
+ it("catches up through the initial tail without opening a probe stream", async () => {
const events = turnEvents();
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockResolvedValueOnce(boundedStreamResponse(events))
- .mockResolvedValueOnce(boundedStreamResponse([], events.length - 1));
- const store = new EveAgentStore({
+ .mockResolvedValueOnce(boundedStreamResponse(events));
+ const store = createStore({
initialSession: { sessionId: "session_1", streamIndex: 0 },
reducer: defaultMessageReducer(),
});
@@ -607,19 +1004,18 @@ describe("EveAgentStore session resume", () => {
await store.resume();
- expect(fetchMock).toHaveBeenCalledTimes(2);
+ expect(fetchMock).toHaveBeenCalledTimes(1);
expect(
- new URL(fetchMock.mock.calls[1]![0].toString(), "http://localhost").searchParams.get(
+ new URL(fetchMock.mock.calls[0]![0].toString(), "http://localhost").searchParams.get(
"startIndex",
),
- ).toBe(String(events.length));
+ ).toBeNull();
expect(
- new URL(fetchMock.mock.calls[1]![0].toString(), "http://localhost").searchParams.get(
+ new URL(fetchMock.mock.calls[0]![0].toString(), "http://localhost").searchParams.get(
"includeTailIndex",
),
).toBe("1");
- expect(publishedEventCounts).not.toContain(1);
- expect(publishedEventCounts).not.toContain(2);
+ expect(publishedEventCounts).toContain(events.length);
expect(store.snapshot.status).toBe("ready");
expect(store.snapshot.events).toEqual(events);
expect(store.snapshot.session).toEqual({ sessionId: "session_1", streamIndex: events.length });
@@ -648,30 +1044,20 @@ describe("EveAgentStore session resume", () => {
createSessionWaitingEvent(),
] as UnstampedMessageStreamEvent[]);
const settled = events.slice(0, 3);
- const [received, started, completed, waiting] = events.slice(3);
- const fetchMock = vi
- .spyOn(globalThis, "fetch")
- .mockResolvedValueOnce(boundedStreamResponse(settled))
- .mockResolvedValueOnce(boundedStreamResponse([received!, started!], settled.length + 1))
- .mockResolvedValueOnce(streamResponse([completed!, waiting!]));
- const store = new EveAgentStore({
+ const live = controlledStreamResponse();
+ live.response.headers.set("x-eve-stream-tail-index", String(settled.length - 1));
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockResolvedValueOnce(live.response);
+ for (const event of settled) live.emit(event);
+ const store = createStore({
initialSession: { sessionId: "session_1", streamIndex: 0 },
reducer: defaultMessageReducer(),
});
await store.resume();
- expect(fetchMock).toHaveBeenCalledTimes(3);
- expect(
- new URL(fetchMock.mock.calls[1]![0].toString(), "http://localhost").searchParams.get(
- "includeTailIndex",
- ),
- ).toBe("1");
- expect(
- new URL(fetchMock.mock.calls[2]![0].toString(), "http://localhost").searchParams.get(
- "includeTailIndex",
- ),
- ).toBeNull();
+ for (const event of events.slice(settled.length)) live.emit(event);
+ await vi.waitFor(() => expect(store.snapshot.events).toHaveLength(events.length));
+ expect(fetchMock).toHaveBeenCalledOnce();
expect(store.snapshot.status).toBe("ready");
expect(store.snapshot.events.slice(settled.length).map((event) => event.type)).toEqual([
"message.received",
@@ -704,23 +1090,21 @@ describe("EveAgentStore session resume", () => {
]);
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockResolvedValueOnce(boundedStreamResponse(events.slice(0, 3)))
- .mockResolvedValueOnce(boundedStreamResponse(events.slice(3, 8), 7))
- .mockResolvedValueOnce(streamResponse(events.slice(8)));
- const store = new EveAgentStore({
+ .mockResolvedValueOnce(boundedStreamResponse(events, 7));
+ const store = createStore({
initialSession: { sessionId: "session_1", streamIndex: 0 },
reducer: defaultMessageReducer(),
});
await store.resume();
- expect(fetchMock).toHaveBeenCalledTimes(3);
+ expect(fetchMock).toHaveBeenCalledTimes(1);
expect(store.snapshot.status).toBe("ready");
expect(store.snapshot.events).toEqual(events);
expect(store.snapshot.session?.streamIndex).toBe(events.length);
});
- it("keeps following when a bounded probe ends with pending authorization", async () => {
+ it("keeps following when catch-up ends with pending authorization", async () => {
const events = stampTestEvents([
createMessageReceivedEvent({ message: "Hello", sequence: 0, turnId: "turn_1" }),
createMessageCompletedEvent({
@@ -751,20 +1135,17 @@ describe("EveAgentStore session resume", () => {
createSessionWaitingEvent(),
] as UnstampedMessageStreamEvent[]);
const settled = events.slice(0, 3);
- const [required, waiting, completed, finalWaiting] = events.slice(3);
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockResolvedValueOnce(boundedStreamResponse(settled))
- .mockResolvedValueOnce(boundedStreamResponse([required!, waiting!], settled.length + 1))
- .mockResolvedValueOnce(streamResponse([completed!, finalWaiting!]));
- const store = new EveAgentStore({
+ .mockResolvedValueOnce(boundedStreamResponse(events, settled.length + 1));
+ const store = createStore({
initialSession: { sessionId: "session_1", streamIndex: 0 },
reducer: defaultMessageReducer(),
});
await store.resume();
- expect(fetchMock).toHaveBeenCalledTimes(3);
+ expect(fetchMock).toHaveBeenCalledTimes(1);
expect(store.snapshot.status).toBe("ready");
expect(store.snapshot.events).toHaveLength(settled.length + 4);
});
@@ -790,7 +1171,7 @@ describe("EveAgentStore session resume", () => {
requests.push(url);
return requests.length === 1 ? boundedStreamResponse([received!, started!]) : live.response;
});
- const store = new EveAgentStore({
+ const store = createStore({
initialSession: { sessionId: "session_1", streamIndex: 0 },
reducer: defaultMessageReducer(),
});
@@ -821,6 +1202,68 @@ describe("EveAgentStore session resume", () => {
});
describe("EveAgentStore steering", () => {
+ it("prepares a steering message once when the active turn finishes during preparation", async () => {
+ const live = controlledStreamResponse();
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockResolvedValueOnce(startedResponse())
+ .mockResolvedValueOnce(live.response)
+ .mockResolvedValueOnce(startedResponse("delivery_2"));
+ const store = createStore({ reducer: defaultMessageReducer() });
+ const first = store.send({ message: "First" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ const preparation = Promise.withResolvers<void>();
+ const prepareSend = vi.fn(async (input) => {
+ await preparation.promise;
+ return input;
+ });
+ store.setCallbacks({ prepareSend });
+ const second = store.send({ message: "Second", turnPolicy: "steer" });
+ for (const event of turnEvents()) live.emit(event);
+ await first;
+ preparation.resolve();
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(3));
+ for (const event of turnEvents())
+ live.emit({
+ ...event,
+ meta: { ...event.meta, id: `second-${event.meta.id}`, deliveryIds: ["delivery_2"] },
+ });
+ await second;
+ expect(prepareSend).toHaveBeenCalledOnce();
+ expect(store.snapshot.status).toBe("ready");
+ });
+
+ it("aborts steering transport on reset without publishing into the new conversation", async () => {
+ const live = controlledStreamResponse();
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockResolvedValueOnce(startedResponse())
+ .mockResolvedValueOnce(live.response)
+ .mockImplementationOnce(
+ (_request, init) =>
+ new Promise((_resolve, reject) => {
+ init?.signal?.addEventListener("abort", () => reject(init.signal?.reason), {
+ once: true,
+ });
+ }),
+ );
+ const store = createStore({ reducer: defaultMessageReducer() });
+ const first = store.send({ message: "First" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ const steering = expect(
+ store.send({ message: "Instead", turnPolicy: "steer" }),
+ ).rejects.toMatchObject({ name: "AbortError" });
+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(3));
+ store.reset();
+ const onChange = vi.fn();
+ store.subscribe(onChange);
+ await Promise.all([first, steering]);
+ expect(onChange).not.toHaveBeenCalled();
+ expect(store.snapshot.status).toBe("ready");
+ expect(store.snapshot.events).toEqual([]);
+ expect(store.snapshot.error).toBeUndefined();
+ });
+
it.each([true, false])(
"completes steering received in the active turn (optimistic=%s)",
async (optimistic) => {
@@ -846,7 +1289,7 @@ describe("EveAgentStore steering", () => {
.mockResolvedValueOnce(startedResponse())
.mockResolvedValueOnce(active.response)
.mockResolvedValueOnce(startedResponse());
- const store = new EveAgentStore({ optimistic, reducer: defaultMessageReducer() });
+ const store = createStore({ optimistic, reducer: defaultMessageReducer() });
const initial = store.send({ message: "First" });
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
active.emit(events[0]!);
@@ -864,7 +1307,6 @@ describe("EveAgentStore steering", () => {
it("follows a late steering delivery after the active turn settles", async () => {
const activeStream = controlledStreamResponse();
- const followUpStream = controlledStreamResponse();
const [
firstReceived,
firstStarted,
@@ -903,9 +1345,8 @@ describe("EveAgentStore steering", () => {
.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(startedResponse())
.mockResolvedValueOnce(activeStream.response)
- .mockResolvedValueOnce(startedResponse())
- .mockResolvedValueOnce(followUpStream.response);
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ .mockResolvedValueOnce(startedResponse());
+ const store = createStore({ reducer: defaultMessageReducer() });
const firstSend = store.send({ message: "First" });
await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
@@ -919,17 +1360,21 @@ describe("EveAgentStore steering", () => {
turnPolicy: "steer",
});
+ let settled = false;
+ const finished = Promise.all([firstSend, steering]).then(() => {
+ settled = true;
+ });
activeStream.emit(firstCompleted!);
activeStream.emit(firstWaiting!);
- activeStream.close();
- await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(4));
- followUpStream.emit(secondReceived!);
- followUpStream.emit(secondStarted!);
- followUpStream.emit(secondCompleted!);
- followUpStream.emit(secondWaiting!);
- followUpStream.close();
-
- await Promise.all([firstSend, steering]);
+ activeStream.emit(secondReceived!);
+ activeStream.emit(secondStarted!);
+ await vi.waitFor(() => expect(store.snapshot.events).toHaveLength(6));
+ expect(settled).toBe(false);
+ expect(store.snapshot.status).toBe("streaming");
+ activeStream.emit(secondCompleted!);
+ activeStream.emit(secondWaiting!);
+ await finished;
+ expect(fetchMock).toHaveBeenCalledTimes(3);
expect(store.snapshot.status).toBe("ready");
expect(store.snapshot.events).toEqual([
firstReceived,
@@ -962,7 +1407,7 @@ describe("EveAgentStore terminal failure", () => {
vi.spyOn(globalThis, "fetch")
.mockResolvedValueOnce(startedResponse())
.mockResolvedValueOnce(streamResponse([failed]));
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
const published: Array<{ eventType: string | undefined; status: string }> = [];
store.subscribe(() => {
published.push({
@@ -993,7 +1438,7 @@ describe("EveAgentStore cancellation", () => {
.mockResolvedValueOnce(startedResponse())
.mockResolvedValueOnce(stream.response)
.mockResolvedValueOnce(acceptedCancellationResponse());
- const store = new EveAgentStore({ optimistic: false, reducer: defaultMessageReducer() });
+ const store = createStore({ optimistic: false, reducer: defaultMessageReducer() });
const sending = store.send({ message: "Hello" });
const cancellation = store.cancel();
@@ -1027,14 +1472,14 @@ describe("EveAgentStore cancellation", () => {
});
it("returns no_active_turn when idle", async () => {
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
await expect(store.cancel()).resolves.toEqual({ status: "no_active_turn" });
});
it("resolves a queued cancellation when reset wins before dispatch", async () => {
const fetchMock = vi.spyOn(globalThis, "fetch");
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
const sending = store.send({ message: "Hello" });
const cancellation = store.cancel();
@@ -1058,7 +1503,7 @@ describe("EveAgentStore cancellation", () => {
);
});
});
- const store = new EveAgentStore({ reducer: defaultMessageReducer() });
+ const store = createStore({ reducer: defaultMessageReducer() });
const sending = store.send({ message: "Hello" });
await vi.waitFor(() => expect(signal).toBeDefined());
diff --git a/packages/eve/src/client/eve-agent-store.ts b/packages/eve/src/client/eve-agent-store.ts
index 3634024c1..1c8eb6dab 100644
--- a/packages/eve/src/client/eve-agent-store.ts
+++ b/packages/eve/src/client/eve-agent-store.ts
@@ -1,121 +1,64 @@
import { Client } from "#client/client.js";
-import type { ActiveTurn, PendingMessageSubmission } from "#client/eve-agent-store-state.js";
-import type { MessageResponse } from "#client/message-response.js";
-import type { EveAgentReducer, EveAgentReducerEvent } from "#client/reducer.js";
+import type {
+ ActiveTurn,
+ EveAgentStoreCallbacks,
+ EveAgentStoreInit,
+ EveAgentStoreSnapshot,
+ EveAgentStoreStatus,
+ PendingMessageSubmission,
+ PrepareSend,
+} from "#client/eve-agent-store-state.js";
+import { consumeMessageResponse, type MessageResponse } from "#client/message-response.js";
+import {
+ SessionEventStream,
+ type SessionEventReader,
+ type SessionEventStreamOptions,
+} from "#client/session-event-stream.js";
+import { EveAgentProjection } from "#client/eve-agent-projection.js";
import type { ClientSession } from "#client/session.js";
import { createEventDeduper } from "#protocol/event-dedupe.js";
import { isCurrentTurnBoundaryEvent, type MessageStreamEvent } from "#protocol/message.js";
import {
assertExclusiveTurnInput,
- collectPendingAuthorizations,
createAbortSignal,
+ createActiveTurn,
+ followSteeredTurns,
createSubmissionId,
isAbortError,
isSettledSessionTail,
summarizeUserContent,
toTerminalStreamFailureError,
- updatePendingAuthorizations,
+ waitWithSignal,
} from "#client/eve-agent-store-helpers.js";
+import { updatePendingAuthorizations } from "#client/session-utils.js";
import { toError } from "#shared/errors.js";
-import type {
- CancelSessionResult,
- ClientAuth,
- HeadersValue,
- SendTurnPayload,
- ClientSessionState,
-} from "#client/types.js";
-
-/**
- * Lifecycle state of an {@link EveAgentStore}: `ready` (idle), `resuming`
- * (checking an attached session for continuation), `submitted` (turn sent,
- * awaiting the first event), `streaming` (events arriving), and `error` (the
- * turn failed). A new turn advances `ready` to `submitted` to `streaming` to
- * `ready` (or `error`).
- */
-export type EveAgentStoreStatus = "error" | "ready" | "resuming" | "streaming" | "submitted";
-
-/**
- * Prepares one outbound turn immediately before the client sends it, e.g. to
- * attach fresh one-turn client state such as page context via `clientContext`.
- */
-export type PrepareSend = (input: SendTurnPayload) => SendTurnPayload | Promise<SendTurnPayload>;
-
-/**
- * Immutable projected state of an {@link EveAgentStore}, read on every render.
- *
- * `data` is the reducer output, `events` is the raw server stream-event log for
- * this session, `session` is the current serializable cursor, `status` is the
- * turn lifecycle state, and `error` is the last failure (or `undefined`).
- */
-export interface EveAgentStoreSnapshot<TData> {
- readonly data: TData;
- readonly error: Error | undefined;
- readonly events: readonly MessageStreamEvent[];
- readonly session: ClientSessionState | undefined;
- readonly status: EveAgentStoreStatus;
-}
+import type { CancelSessionResult, SendTurnPayload } from "#client/types.js";
-/**
- * Hooks invoked while the store processes a turn.
- *
- * `onEvent`, `onError`, `onFinish`, and `onSessionChange` are observe-only.
- * `prepareSend` runs before each turn is sent and may return a modified
- * {@link SendTurnPayload} (for example to attach one-turn client context).
- */
-export interface EveAgentStoreCallbacks<TData> {
- readonly onError?: (error: Error) => void;
- readonly onEvent?: (event: MessageStreamEvent) => void;
- readonly onFinish?: (snapshot: EveAgentStoreSnapshot<TData>) => void;
- readonly onSessionChange?: (session: ClientSessionState | undefined) => void;
- readonly prepareSend?: PrepareSend;
-}
-
-/**
- * Configuration for constructing an {@link EveAgentStore}.
- *
- * Requires a {@link EveAgentReducer | reducer}, plus either connection options
- * (`host`, `auth`, `headers`, `initialSession`) for a
- * store-owned session or an existing {@link ClientSession} via `session`.
- *
- * `optimistic` (default `true`) projects submitted user messages before the
- * server confirms them. `host` defaults to `""`. `initialEvents` and
- * `initialSession` seed prior state on construction. Passing `session` makes
- * `reset()` reuse that external session rather than create a new one.
- * `initialEvents` must be an ordered prefix of the same session's stream; its
- * endpoint may overlap the cursor because repeated ids are applied once.
- */
-export interface EveAgentStoreInit<TData> {
- readonly auth?: ClientAuth;
- readonly headers?: HeadersValue;
- readonly host?: string;
- /** Ordered prefix of the session stream used to rehydrate projected state. */
- readonly initialEvents?: readonly MessageStreamEvent[];
- readonly initialSession?: ClientSessionState;
- readonly optimistic?: boolean;
- readonly reducer: EveAgentReducer<TData>;
- readonly session?: ClientSession;
-}
+export type {
+ EveAgentStoreCallbacks,
+ EveAgentStoreInit,
+ EveAgentStoreSnapshot,
+ EveAgentStoreStatus,
+ PrepareSend,
+} from "#client/eve-agent-store-state.js";
const detachStore = Symbol("detachEveAgentStore");
+const attachStore = Symbol("attachEveAgentStore");
/**
- * Framework-agnostic state machine for an eve agent session.
- *
- * Manages the send/stream lifecycle, optimistic projection, and subscriber
- * notification; framework integrations (React, Vue) wrap it with their own
- * reactivity primitives.
- *
- * Drives one turn at a time: `send` rejects while a turn is active, and
- * concurrent `resume` calls share one replay. Read the latest projection via
- * the `snapshot` getter, observe changes with `subscribe`, register lifecycle
- * hooks with `setCallbacks`, cancel the durable in-flight turn with `cancel`,
- * and discard all state with `reset`.
+ * Owns frontend session creation, continuous streaming, turn submission, and UI projection.
+ * Framework bindings subscribe to snapshots and attach or detach transport with their lifecycle.
+ * Reset clears local state; cancellation remains an explicit durable operation.
*/
export class EveAgentStore<TData> {
readonly #client: Client | undefined;
+ readonly #autoPrewarm: boolean;
+ #attached = false;
+ #stream: SessionEventStream | undefined;
+ readonly #pendingAuthorizations = new Set<string>();
readonly #externalSession: boolean;
readonly #optimistic: boolean;
- readonly #reducer: EveAgentReducer<TData>;
+ readonly #projection: EveAgentProjection<TData>;
readonly #subscribers = new Set<() => void>();
/** Ids already folded into the projection: `initialEvents` and a reconnect can overlap. */
@@ -123,17 +66,19 @@ export class EveAgentStore<TData> {
#activeTurn: ActiveTurn | undefined;
#callbacks: EveAgentStoreCallbacks<TData> = {};
- #data: TData;
#error: Error | undefined;
#events: readonly MessageStreamEvent[];
#pendingMessageSubmissions: readonly PendingMessageSubmission[] = [];
- #projectionEvents: readonly EveAgentReducerEvent[];
+ #prewarmGeneration = 0;
+ #prewarmPromise: Promise<void> | undefined;
+ #prewarmController: AbortController | undefined;
#resumePromise: Promise<void> | undefined;
#session: ClientSession | undefined;
#snapshot: EveAgentStoreSnapshot<TData>;
#status: EveAgentStoreStatus = "ready";
constructor(init: EveAgentStoreInit<TData>) {
+ this.#autoPrewarm = init.prewarm ?? false;
this.#externalSession = init.session !== undefined;
this.#client = this.#externalSession
? undefined
@@ -147,11 +92,11 @@ export class EveAgentStore<TData> {
const initialEvents: MessageStreamEvent[] = [];
for (const event of init.initialEvents ?? []) {
if (this.#seenEvents.admit(event)) initialEvents.push(event);
+ updatePendingAuthorizations(this.#pendingAuthorizations, event);
}
this.#events = initialEvents;
- this.#projectionEvents = [...this.#events];
+ this.#projection = new EveAgentProjection(init.reducer, this.#events);
this.#optimistic = init.optimistic ?? true;
- this.#reducer = init.reducer;
this.#session =
init.session ??
(init.initialSession === undefined
@@ -160,7 +105,6 @@ export class EveAgentStore<TData> {
streamIndex: init.initialSession.streamIndex,
}));
- this.#data = this.#reduceProjectionEvents(this.#projectionEvents);
this.#snapshot = this.#createSnapshot();
}
@@ -179,42 +123,95 @@ export class EveAgentStore<TData> {
};
}
+ /** Creates this store's owned session without starting a turn. */
+ prewarm(): Promise<void> {
+ if (this.#prewarmPromise !== undefined) return this.#prewarmPromise;
+ if (this.#session !== undefined || this.#externalSession) return Promise.resolve();
+ if (this.#activeTurn !== undefined) {
+ return this.#activeTurn.response.then((response) => {
+ if (response === undefined) {
+ throw this.#error ?? new DOMException("Session creation was aborted.", "AbortError");
+ }
+ });
+ }
+ const client = this.#client;
+ if (client === undefined) {
+ return Promise.reject(new Error("This eve agent store does not own a session client."));
+ }
+
+ const generation = this.#prewarmGeneration;
+ const controller = new AbortController();
+ this.#prewarmController = controller;
+ const promise = (async () => {
+ try {
+ const created = await client.sessions.create({ signal: controller.signal });
+ if (generation !== this.#prewarmGeneration) return;
+ this.#session = created.session;
+ this.#error = undefined;
+ if (this.#status === "error") this.#status = "ready";
+ this.#callbacks.onSessionChange?.(created.session.state);
+ this.#publish();
+ this.#ensureStream();
+ } catch (error) {
+ if (
+ generation === this.#prewarmGeneration &&
+ this.#activeTurn === undefined &&
+ this.#error === undefined
+ ) {
+ this.#error = toError(error);
+ this.#status = "error";
+ this.#callbacks.onError?.(this.#error);
+ this.#publish();
+ }
+ throw error;
+ }
+ })();
+ this.#prewarmPromise = promise;
+ const clear = () => {
+ if (this.#prewarmPromise === promise) {
+ this.#prewarmPromise = undefined;
+ this.#prewarmController = undefined;
+ }
+ };
+ void promise.then(clear, clear);
+ return promise;
+ }
+
async send<TOutput = unknown>(input: SendTurnPayload<TOutput>): Promise<void> {
+ return await this.#submit(input, this.#callbacks.prepareSend);
+ }
+
+ async #submit<TOutput>(
+ input: SendTurnPayload<TOutput>,
+ prepareSend?: PrepareSend,
+ ): Promise<void> {
if (this.#activeTurn !== undefined) {
if (this.#status === "resuming") {
throw new Error("eve session is resuming.");
}
- return await this.#sendFollowUp(this.#activeTurn, input);
+ return await this.#sendFollowUp(this.#activeTurn, input, prepareSend);
}
- const response = Promise.withResolvers<MessageResponse | undefined>();
- const completion = Promise.withResolvers<void>();
- const turn: ActiveTurn = {
- abortController: new AbortController(),
- acceptedFollowUps: 0,
- cancel: () =>
- turn.acceptedFollowUps > 0 && this.#session !== undefined
- ? this.#session.cancel()
- : response.promise.then((messageResponse) =>
- messageResponse === undefined
- ? { status: "no_active_turn" }
- : messageResponse.cancel(),
- ),
- completion: completion.promise,
- followUpDispatches: new Set(),
- receivedFollowUps: 0,
- followUpSubmissionIds: new Set(),
- resolveCompletion: completion.resolve,
- response: response.promise,
- resolveResponse: response.resolve,
- };
+ const turn = createActiveTurn((turn) =>
+ turn.acceptedFollowUps > 0 && this.#session !== undefined
+ ? this.#session.cancel()
+ : turn.response.then((response) =>
+ response === undefined ? { status: "no_active_turn" } : response.cancel(),
+ ),
+ );
this.#activeTurn = turn;
this.#error = undefined;
this.#status = "submitted";
this.#publish();
+ let reader: SessionEventReader | undefined;
try {
- const preparedInput = (await this.#callbacks.prepareSend?.(input)) ?? input;
+ const preparedInput = await (prepareSend === undefined
+ ? input
+ : waitWithSignal(
+ Promise.resolve(prepareSend(input)),
+ createAbortSignal(input.signal, turn.abortController.signal),
+ ));
assertExclusiveTurnInput(preparedInput);
if (!this.#isActiveTurn(turn)) {
@@ -229,21 +226,23 @@ export class EveAgentStore<TData> {
...preparedInput,
signal: createAbortSignal(preparedInput.signal, turn.abortController.signal),
};
- const response = await this.#dispatchTurn(turnInput);
+ const dispatched = await this.#dispatchTurn(turnInput);
+ const response = dispatched.response;
+ reader = dispatched.reader;
if (!this.#isActiveTurn(turn)) return;
turn.resolveResponse(response);
- for await (const event of response) {
+ for await (const event of consumeMessageResponse(response, reader)) {
if (!this.#isActiveTurn(turn)) return;
- this.#acceptServerEvent(event);
+ if (turn.receivedFollowUpEvents.delete(event)) turn.receivedFollowUps += 1;
}
if (!this.#isActiveTurn(turn)) {
return;
}
- await this.#followSteeredTurns(turn);
+ await followSteeredTurns(turn, reader, () => this.#isActiveTurn(turn));
if (!this.#isActiveTurn(turn)) return;
this.#status = this.#error === undefined ? "ready" : "error";
} catch (error) {
@@ -255,20 +254,15 @@ export class EveAgentStore<TData> {
this.#status = "ready";
this.#failPendingMessageSubmission(toError(error));
} else {
- this.#error = toError(error);
+ const reported = this.#error !== undefined;
+ this.#error ??= toError(error);
this.#status = "error";
this.#failPendingMessageSubmission(this.#error);
- this.#callbacks.onError?.(this.#error);
+ if (!reported) this.#callbacks.onError?.(this.#error);
}
} finally {
- if (this.#isActiveTurn(turn)) {
- turn.resolveResponse(undefined);
- this.#activeTurn = undefined;
- this.#callbacks.onSessionChange?.(this.#session?.state);
- this.#publish();
- this.#callbacks.onFinish?.(this.#snapshot);
- turn.resolveCompletion();
- }
+ reader?.[Symbol.dispose]();
+ this.#finishTurn(turn);
}
}
@@ -297,21 +291,8 @@ export class EveAgentStore<TData> {
throw new Error("An eve session is required before resuming.");
}
- const response = Promise.withResolvers<MessageResponse | undefined>();
- const completion = Promise.withResolvers<void>();
const session = this.#session;
- const turn: ActiveTurn = {
- abortController: new AbortController(),
- acceptedFollowUps: 0,
- cancel: () => session.cancel(),
- completion: completion.promise,
- followUpDispatches: new Set(),
- receivedFollowUps: 0,
- followUpSubmissionIds: new Set(),
- resolveCompletion: completion.resolve,
- response: response.promise,
- resolveResponse: response.resolve,
- };
+ const turn = createActiveTurn(() => session.cancel());
this.#activeTurn = turn;
turn.resolveResponse(undefined);
this.#error = undefined;
@@ -319,56 +300,28 @@ export class EveAgentStore<TData> {
this.#publish();
try {
- const hasCompleteInitialPrefix = this.#events.length === session.state.streamIndex;
- const replayed: MessageStreamEvent[] = hasCompleteInitialPrefix ? [...this.#events] : [];
- for await (const event of session.stream({
- follow: false,
- signal: turn.abortController.signal,
- startIndex: hasCompleteInitialPrefix ? session.state.streamIndex : 0,
- })) {
- if (!this.#isActiveTurn(turn)) return;
- replayed.push(event);
- this.#acceptServerEvent(event, { publish: false, transitionToStreaming: false });
- }
-
+ const stream = this.#ensureStream({
+ catchUp: true,
+ startIndex:
+ this.#events.length === session.state.streamIndex ? session.state.streamIndex : 0,
+ });
+ using reader = stream.subscribe(turn.abortController.signal);
+ await stream.caughtUp;
if (!this.#isActiveTurn(turn)) return;
- const replayTail = replayed.at(-1);
- if (replayTail !== undefined) this.#applyTerminalStreamFailure(replayTail);
- const replaySettled = isSettledSessionTail(replayed);
- if (this.#error === undefined && (!replaySettled || replayTail?.type === "session.waiting")) {
- const pendingAuthorizations = collectPendingAuthorizations(replayed);
- if (!replaySettled) this.#status = "streaming";
+ reader.discard();
+ const tail = this.#events.at(-1);
+ if (tail !== undefined) this.#applyTerminalStreamFailure(tail);
+ if (tail !== undefined && this.#error === undefined && !isSettledSessionTail(this.#events)) {
+ this.#status = "streaming";
this.#publish();
- let followLive = !replaySettled;
- for (;;) {
- for await (const event of session.stream({
- follow: followLive ? undefined : false,
- signal: turn.abortController.signal,
- })) {
- if (!this.#isActiveTurn(turn)) return;
- replayed.push(event);
- this.#acceptServerEvent(event);
- updatePendingAuthorizations(pendingAuthorizations, event);
- if (
- followLive &&
- isCurrentTurnBoundaryEvent(event) &&
- (event.type !== "session.waiting" || pendingAuthorizations.size === 0)
- )
- break;
- }
+ for await (const event of reader) {
if (!this.#isActiveTurn(turn)) return;
- if (followLive || isSettledSessionTail(replayed)) break;
- followLive = true;
+ if (turn.receivedFollowUpEvents.delete(event)) turn.receivedFollowUps += 1;
+ if (isCurrentTurnBoundaryEvent(event) && this.#pendingAuthorizations.size === 0) break;
}
- } else {
- this.#status = this.#error === undefined ? "ready" : "error";
- this.#publish();
- }
-
- await this.#followSteeredTurns(turn);
- if (this.#isActiveTurn(turn)) {
- this.#status = this.#error === undefined ? "ready" : "error";
}
+ await followSteeredTurns(turn, reader, () => this.#isActiveTurn(turn));
+ if (this.#isActiveTurn(turn)) this.#status = this.#error === undefined ? "ready" : "error";
} catch (error) {
if (!this.#isActiveTurn(turn)) return;
if (isAbortError(error)) {
@@ -379,14 +332,7 @@ export class EveAgentStore<TData> {
this.#callbacks.onError?.(this.#error);
}
} finally {
- if (this.#isActiveTurn(turn)) {
- turn.resolveResponse(undefined);
- this.#activeTurn = undefined;
- this.#callbacks.onSessionChange?.(session.state);
- this.#publish();
- this.#callbacks.onFinish?.(this.#snapshot);
- turn.resolveCompletion();
- }
+ this.#finishTurn(turn);
}
}
@@ -398,63 +344,97 @@ export class EveAgentStore<TData> {
*/
cancel(): Promise<CancelSessionResult> {
const turn = this.#activeTurn;
- if (turn === undefined) return Promise.resolve({ status: "no_active_turn" });
+ if (turn === undefined) {
+ return this.#status === "streaming" && this.#session !== undefined
+ ? this.#session.cancel()
+ : Promise.resolve({ status: "no_active_turn" });
+ }
return turn.cancel();
}
+ [attachStore](): void {
+ this.#attached = true;
+ if (this.#autoPrewarm && this.#session === undefined) void this.prewarm().catch(() => {});
+ }
+
[detachStore](): void {
+ this.#attached = false;
+ this.#stream?.close();
+ this.#stream = undefined;
this.#activeTurn?.abortController.abort();
+ this.#resetPrewarm();
+ this.#resumePromise = undefined;
}
reset(): void {
+ this.#stream?.close();
+ this.#stream = undefined;
+ this.#pendingAuthorizations.clear();
const turn = this.#activeTurn;
this.#activeTurn = undefined;
turn?.resolveResponse(undefined);
turn?.resolveCompletion();
turn?.abortController.abort();
+ this.#resetPrewarm();
+ this.#resumePromise = undefined;
if (!this.#externalSession) this.#session = undefined;
this.#events = [];
this.#seenEvents = createEventDeduper();
this.#pendingMessageSubmissions = [];
- this.#projectionEvents = [];
- this.#data = this.#reducer.initial();
+ this.#projection.reset();
this.#error = undefined;
this.#status = "ready";
this.#callbacks.onSessionChange?.(this.#session?.state);
this.#publish();
+ if (this.#autoPrewarm && this.#attached) void this.prewarm().catch(() => {});
}
- async #sendFollowUp<TOutput>(turn: ActiveTurn, input: SendTurnPayload<TOutput>): Promise<void> {
+ async #sendFollowUp<TOutput>(
+ turn: ActiveTurn,
+ input: SendTurnPayload<TOutput>,
+ prepareSend?: PrepareSend,
+ ): Promise<void> {
if (input.message === undefined || input.turnPolicy !== "steer") {
throw new Error(
'eve session is already processing a turn. Send a message with turnPolicy: "steer" to guide it at the next boundary.',
);
}
- const preparedInput = (await this.#callbacks.prepareSend?.(input)) ?? input;
+ const generation = this.#prewarmGeneration;
+ const signal = createAbortSignal(input.signal, turn.abortController.signal);
+ const preparedInput =
+ (await waitWithSignal(Promise.resolve(prepareSend?.(input)), signal)) ?? input;
+ if (generation !== this.#prewarmGeneration) return;
assertExclusiveTurnInput(preparedInput);
if (preparedInput.message === undefined || preparedInput.turnPolicy !== "steer") {
throw new Error('An in-flight follow-up requires a message with turnPolicy: "steer".');
}
- if (!this.#isActiveTurn(turn)) return await this.send(preparedInput);
+ if (!this.#isActiveTurn(turn)) return await this.#submit(preparedInput);
const submissionId = this.#projectOptimisticMessage(preparedInput);
if (submissionId !== undefined) turn.followUpSubmissionIds.add(submissionId);
this.#publish();
+ this.#ensureStream({
+ headers: preparedInput.headers,
+ streamReconnectPolicy: preparedInput.streamReconnectPolicy,
+ });
let dispatch!: Promise<void>;
dispatch = (async () => {
try {
- await turn.response;
+ const signal = createAbortSignal(preparedInput.signal, turn.abortController.signal);
+ await waitWithSignal(turn.response, signal);
if (!this.#isActiveTurn(turn) || this.#session === undefined) {
throw new Error("The active eve turn ended before the follow-up could be sent.");
}
const { message, ...options } = preparedInput;
- await this.#session.send(message, options);
+ await this.#session.send(message, { ...options, signal });
turn.acceptedFollowUps += 1;
} catch (error) {
- this.#failPendingMessageSubmission(toError(error), submissionId);
- this.#publish();
+ if (this.#isActiveTurn(turn)) {
+ this.#failPendingMessageSubmission(toError(error), submissionId);
+ this.#publish();
+ }
throw error;
} finally {
turn.followUpDispatches.delete(dispatch);
@@ -466,51 +446,101 @@ export class EveAgentStore<TData> {
await turn.completion;
}
- async #followSteeredTurns(turn: ActiveTurn): Promise<void> {
- while (turn.followUpDispatches.size > 0) {
- await Promise.allSettled(turn.followUpDispatches);
+ async #dispatchTurn<TOutput>(
+ input: SendTurnPayload<TOutput>,
+ ): Promise<{ readonly response: MessageResponse<TOutput>; readonly reader: SessionEventReader }> {
+ const streamOptions = {
+ headers: input.headers,
+ streamReconnectPolicy: input.streamReconnectPolicy,
+ };
+ if (this.#prewarmPromise !== undefined) {
+ // A failed speculative create must not prevent the user's message from being sent.
+ await waitWithSignal(
+ this.#prewarmPromise.catch((error: unknown) => {
+ if (this.#session !== undefined) throw error;
+ }),
+ input.signal,
+ );
+ input.signal?.throwIfAborted();
}
- if (turn.receivedFollowUps >= turn.acceptedFollowUps || this.#session === undefined) return;
-
- for await (const event of this.#session.stream({ signal: turn.abortController.signal })) {
- if (!this.#isActiveTurn(turn)) return;
- this.#acceptServerEvent(event);
-
- if (isCurrentTurnBoundaryEvent(event)) {
- while (turn.followUpDispatches.size > 0) {
- await Promise.allSettled(turn.followUpDispatches);
- }
- if (turn.receivedFollowUps >= turn.acceptedFollowUps) return;
+ if (this.#session === undefined) {
+ if (this.#client === undefined) {
+ throw new Error("An external eve session is required before sending.");
+ }
+ if (input.message === undefined) {
+ throw new Error("Cannot answer an input request before the session starts.");
}
+ const created = await this.#client.sessions.create({ ...input, message: input.message });
+ input.signal?.throwIfAborted();
+ this.#session = created.session;
+ this.#callbacks.onSessionChange?.(created.session.state);
+ this.#publish();
+ return {
+ response: created.response,
+ reader: this.#ensureStream(streamOptions).subscribe(input.signal),
+ };
+ }
+ const reader = this.#ensureStream(streamOptions).subscribe(input.signal);
+ try {
+ if (input.inputResponses === undefined) {
+ const { message, ...options } = input;
+ return { response: await this.#session.send(message, options), reader };
+ }
+ const { inputResponses, ...options } = input;
+ return { response: await this.#session.respond(inputResponses, options), reader };
+ } catch (error) {
+ reader[Symbol.dispose]();
+ throw error;
}
}
- async #createFirstTurn<TOutput>(
- input: SendTurnPayload<TOutput>,
- ): Promise<Awaited<ReturnType<ClientSession["send"]>>> {
- if (this.#client === undefined) {
- throw new Error("An external eve session is required before sending.");
+ #ensureStream(
+ options: Omit<SessionEventStreamOptions, "onEvent" | "onError"> = {},
+ ): SessionEventStream {
+ if (this.#stream !== undefined && !this.#stream.ended) {
+ this.#stream.setOptions(options);
+ return this.#stream;
}
- if (input.message === undefined) {
- throw new Error("Cannot answer an input request before the session starts.");
- }
- const created = await this.#client.sessions.create({ ...input, message: input.message });
- this.#session = created.session;
- this.#callbacks.onSessionChange?.(created.session.state);
- this.#publish();
- return created.response;
+ if (this.#session === undefined)
+ throw new Error("A session is required before opening its stream.");
+ const session = this.#session;
+ const generation = this.#prewarmGeneration;
+ this.#stream = new SessionEventStream(session, {
+ ...options,
+ onEvent: (event) => {
+ if (generation === this.#prewarmGeneration) this.#acceptServerEvent(event);
+ },
+ onError: (error) => {
+ if (generation !== this.#prewarmGeneration) return;
+ this.#stream = undefined;
+ if (this.#activeTurn !== undefined) return;
+ this.#error = toError(error);
+ this.#status = "error";
+ this.#callbacks.onError?.(this.#error);
+ this.#publish();
+ },
+ });
+ return this.#stream;
}
- async #dispatchTurn<TOutput>(
- input: SendTurnPayload<TOutput>,
- ): Promise<Awaited<ReturnType<ClientSession["send"]>>> {
- if (this.#session === undefined) return await this.#createFirstTurn(input);
- if (input.inputResponses === undefined) {
- const { message, ...options } = input;
- return await this.#session.send(message, options);
+ #resetPrewarm(): void {
+ this.#prewarmGeneration += 1;
+ this.#prewarmController?.abort();
+ this.#prewarmController = undefined;
+ this.#prewarmPromise = undefined;
+ }
+
+ #finishTurn(turn: ActiveTurn): void {
+ if (!this.#isActiveTurn(turn)) return;
+ turn.resolveResponse(undefined);
+ this.#activeTurn = undefined;
+ try {
+ this.#callbacks.onSessionChange?.(this.#session?.state);
+ this.#publish();
+ this.#callbacks.onFinish?.(this.#snapshot);
+ } finally {
+ turn.resolveCompletion();
}
- const { inputResponses, ...options } = input;
- return await this.#session.respond(inputResponses, options);
}
#isActiveTurn(turn: ActiveTurn): boolean {
@@ -530,7 +560,7 @@ export class EveAgentStore<TData> {
};
this.#pendingMessageSubmissions = [...this.#pendingMessageSubmissions, pending];
if (this.#optimistic)
- this.#appendProjectionEvent({
+ this.#projection.append({
data: {
createdAt: pending.createdAt,
message: pending.message,
@@ -546,7 +576,7 @@ export class EveAgentStore<TData> {
return;
}
- this.#appendProjectionEvent({
+ this.#projection.append({
data: {
createdAt: Date.now(),
responses: input.inputResponses,
@@ -555,17 +585,24 @@ export class EveAgentStore<TData> {
});
}
- #acceptServerEvent(
- event: MessageStreamEvent,
- options: { readonly publish?: boolean; readonly transitionToStreaming?: boolean } = {},
- ): void {
+ #acceptServerEvent(event: MessageStreamEvent): void {
if (!this.#seenEvents.admit(event)) return;
+ const wasStreaming = this.#status === "streaming";
+ updatePendingAuthorizations(this.#pendingAuthorizations, event);
this.#events = [...this.#events, event];
this.#applyServerEvent(event);
this.#callbacks.onEvent?.(event);
- if (options.transitionToStreaming ?? true) this.#status = "streaming";
this.#applyTerminalStreamFailure(event);
- if (options.publish ?? true) this.#publish();
+ const settled = isCurrentTurnBoundaryEvent(event) && this.#pendingAuthorizations.size === 0;
+ if (this.#status !== "resuming" && this.#error === undefined) {
+ if ("data" in event && "turnId" in event.data) this.#status = "streaming";
+ if (this.#activeTurn === undefined && settled) this.#status = "ready";
+ }
+ this.#callbacks.onSessionChange?.(this.#session?.state);
+ this.#publish();
+ if (this.#activeTurn === undefined && wasStreaming && settled) {
+ this.#callbacks.onFinish?.(this.#snapshot);
+ }
}
#applyServerEvent(event: MessageStreamEvent): void {
@@ -573,9 +610,9 @@ export class EveAgentStore<TData> {
if (event.type === "message.received" && pendingSubmission !== undefined) {
const submissionId = pendingSubmission.id;
if (this.#activeTurn?.followUpSubmissionIds.delete(submissionId))
- this.#activeTurn.receivedFollowUps += 1;
+ this.#activeTurn.receivedFollowUpEvents.add(event);
this.#pendingMessageSubmissions = this.#pendingMessageSubmissions.slice(1);
- this.#replaceProjectionEvent(
+ this.#projection.replace(
(candidate) =>
candidate.type === "client.message.submitted" &&
candidate.data.submissionId === submissionId,
@@ -584,7 +621,7 @@ export class EveAgentStore<TData> {
return;
}
- this.#appendProjectionEvent(event);
+ this.#projection.append(event);
}
#applyTerminalStreamFailure(event: MessageStreamEvent): void {
@@ -612,7 +649,7 @@ export class EveAgentStore<TData> {
this.#pendingMessageSubmissions = this.#pendingMessageSubmissions.filter(
(candidate) => candidate.id !== pending.id,
);
- this.#replaceProjectionEvent(
+ this.#projection.replace(
(event) =>
event.type === "client.message.submitted" && event.data.submissionId === pending.id,
{
@@ -629,42 +666,9 @@ export class EveAgentStore<TData> {
);
}
- #appendProjectionEvent(event: EveAgentReducerEvent): void {
- this.#projectionEvents = [...this.#projectionEvents, event];
- this.#data = this.#reducer.reduce(this.#data, event);
- }
-
- #replaceProjectionEvent(
- predicate: (event: EveAgentReducerEvent) => boolean,
- replacement: EveAgentReducerEvent,
- ): void {
- let replaced = false;
- this.#projectionEvents = this.#projectionEvents.map((event) => {
- if (!replaced && predicate(event)) {
- replaced = true;
- return replacement;
- }
- return event;
- });
-
- if (!replaced) {
- this.#projectionEvents = [...this.#projectionEvents, replacement];
- }
-
- this.#data = this.#reduceProjectionEvents(this.#projectionEvents);
- }
-
- #reduceProjectionEvents(events: readonly EveAgentReducerEvent[]): TData {
- let data = this.#reducer.initial();
- for (const event of events) {
- data = this.#reducer.reduce(data, event);
- }
- return data;
- }
-
#createSnapshot(): EveAgentStoreSnapshot<TData> {
return {
- data: this.#data,
+ data: this.#projection.data,
error: this.#error,
events: this.#events,
session: this.#session?.state,
@@ -684,3 +688,8 @@ export class EveAgentStore<TData> {
export function detachEveAgentStore<TData>(store: EveAgentStore<TData>): void {
store[detachStore]();
}
+
+/** @internal Starts mount-scoped session work after the framework commits. */
+export function attachEveAgentStore<TData>(store: EveAgentStore<TData>): void {
+ store[attachStore]();
+}
diff --git a/packages/eve/src/client/index.ts b/packages/eve/src/client/index.ts
index 894fc6340..b9f360d1d 100644
--- a/packages/eve/src/client/index.ts
+++ b/packages/eve/src/client/index.ts
@@ -13,7 +13,11 @@ export { defaultMessageReducer } from "#client/message-reducer.js";
export { createDataUrlFilePart, createTextWithFileContent } from "#client/file-parts.js";
export { MessageResponse } from "#client/message-response.js";
export { ClientSession } from "#client/session.js";
-export { ClientSessions, type CreatedClientSession } from "#client/sessions.js";
+export {
+ ClientSessions,
+ type CreatedClientSession,
+ type CreatedIdleClientSession,
+} from "#client/sessions.js";
// ---------------------------------------------------------------------------
// Client types
@@ -51,6 +55,7 @@ export type {
ClientAuth,
ClientOptions,
ClientRedirectPolicy,
+ CreateSessionOptions,
HeadersValue,
HealthResult,
MessageResult,
diff --git a/packages/eve/src/client/message-response.ts b/packages/eve/src/client/message-response.ts
index 2f4090e34..4be2b6e1b 100644
--- a/packages/eve/src/client/message-response.ts
+++ b/packages/eve/src/client/message-response.ts
@@ -8,10 +8,14 @@ import type { CancelSessionResult, MessageResult } from "#client/types.js";
*/
interface MessageResponseInput {
readonly cancelTurn: (turnId: string) => Promise<CancelSessionResult>;
- readonly createStream: () => AsyncGenerator<MessageStreamEvent>;
+ readonly createStream: (
+ source?: AsyncIterable<MessageStreamEvent>,
+ ) => AsyncGenerator<MessageStreamEvent>;
readonly sessionId: string;
}
+const consumeResponse = Symbol("consumeMessageResponse");
+
/**
* The response from {@link ClientSession.send}.
*
@@ -28,7 +32,7 @@ export class MessageResponse<TOutput = unknown> implements AsyncIterable<Message
readonly #cancelTurn: (turnId: string) => Promise<CancelSessionResult>;
#cancellation: Promise<CancelSessionResult> | undefined;
#consumed = false;
- readonly #createStream: () => AsyncGenerator<MessageStreamEvent>;
+ readonly #createStream: MessageResponseInput["createStream"];
#settled = false;
readonly #turnId = Promise.withResolvers<string | undefined>();
@@ -87,17 +91,25 @@ export class MessageResponse<TOutput = unknown> implements AsyncIterable<Message
* Each response can only be consumed once.
*/
[Symbol.asyncIterator](): AsyncIterator<MessageStreamEvent> {
+ return this[consumeResponse]();
+ }
+
+ [consumeResponse](
+ source?: AsyncIterable<MessageStreamEvent>,
+ ): AsyncGenerator<MessageStreamEvent> {
if (this.#consumed) {
throw new Error("MessageResponse has already been consumed.");
}
this.#consumed = true;
- return this.#observeStream();
+ return this.#observeStream(source);
}
- async *#observeStream(): AsyncGenerator<MessageStreamEvent> {
+ async *#observeStream(
+ source?: AsyncIterable<MessageStreamEvent>,
+ ): AsyncGenerator<MessageStreamEvent> {
try {
- for await (const event of this.#createStream()) {
+ for await (const event of this.#createStream(source)) {
if (event.type === "turn.started") {
this.#turnId.resolve(event.data.turnId);
} else if (isCurrentTurnBoundaryEvent(event)) {
@@ -111,3 +123,11 @@ export class MessageResponse<TOutput = unknown> implements AsyncIterable<Message
}
}
}
+
+/** @internal Observe a turn through the frontend's existing session stream. */
+export function consumeMessageResponse(
+ response: MessageResponse,
+ source: AsyncIterable<MessageStreamEvent>,
+): AsyncIterable<MessageStreamEvent> {
+ return response[consumeResponse](source);
+}
diff --git a/packages/eve/src/client/ndjson.ts b/packages/eve/src/client/ndjson.ts
index 9464ef3ef..8f65e2539 100644
--- a/packages/eve/src/client/ndjson.ts
+++ b/packages/eve/src/client/ndjson.ts
@@ -44,6 +44,7 @@ export async function* readNdjsonStream(
readonly controlVersion?: "1";
readonly idleTimeoutMs?: number;
readonly onLeaseEnded?: () => void;
+ readonly signal?: AbortSignal;
readonly streamVersion: MessageStreamVersion;
},
): AsyncGenerator<MessageStreamEvent> {
@@ -51,10 +52,16 @@ export async function* readNdjsonStream(
const decoder = new TextDecoder();
let buffer = "";
let reachedEof = false;
+ const abort = () => {
+ void reader.cancel().catch(() => {});
+ };
+ options.signal?.addEventListener("abort", abort, { once: true });
try {
while (true) {
+ options.signal?.throwIfAborted();
const result = await readWithIdleTimeout(reader, options?.idleTimeoutMs);
+ options.signal?.throwIfAborted();
if (result.done) {
reachedEof = true;
@@ -97,6 +104,7 @@ export async function* readNdjsonStream(
}
}
} finally {
+ options.signal?.removeEventListener("abort", abort);
if (!reachedEof) {
// A cloned response waits for both branches to cancel. Let the caller
// abort the fetch instead of blocking cleanup on a tracing reader.
diff --git a/packages/eve/src/client/open-stream.test.ts b/packages/eve/src/client/open-stream.test.ts
index 017dcba00..5ec5b5e54 100644
--- a/packages/eve/src/client/open-stream.test.ts
+++ b/packages/eve/src/client/open-stream.test.ts
@@ -76,6 +76,29 @@ describe("openStreamBody", () => {
});
describe("followStreamIterable", () => {
+ it("honors explicit idle retry limits even while following continuously", async () => {
+ const fetchMock = vi.fn(
+ async () =>
+ new Response("\n", {
+ headers: { [EVE_STREAM_VERSION_HEADER]: EVE_MESSAGE_STREAM_VERSION },
+ }),
+ );
+ vi.stubGlobal("fetch", fetchMock);
+ for await (const event of followStreamIterable({
+ host: "https://agent.example",
+ resolveHeaders: () => Promise.resolve(new Headers()),
+ keepAlive: true,
+ resolveReconnectPolicy: () => ({
+ streamIdleReconnectPolicy: { maxAttempts: 1, baseDelayMs: 0 },
+ }),
+ sessionId: "session_1",
+ startIndex: 0,
+ })) {
+ expect.unreachable(`Unexpected event: ${event.type}`);
+ }
+ expect(fetchMock).toHaveBeenCalledTimes(2);
+ });
+
it.each([
{ startIndex: -1, streamReconnectPolicy: undefined },
{ startIndex: 0, streamReconnectPolicy: { reconnect: false } },
diff --git a/packages/eve/src/client/open-stream.ts b/packages/eve/src/client/open-stream.ts
index d34801b22..9deaa48ab 100644
--- a/packages/eve/src/client/open-stream.ts
+++ b/packages/eve/src/client/open-stream.ts
@@ -58,6 +58,7 @@ function resolveRetryPolicy(
function resolveStreamReconnectPolicy(
policy: StreamReconnectPolicy | undefined,
+ keepAlive = false,
): ResolvedStreamReconnectPolicy {
if (policy && "reconnect" in policy && policy.reconnect === false) {
return NO_STREAM_RECONNECT_POLICY;
@@ -68,10 +69,12 @@ function resolveStreamReconnectPolicy(
retryableErrorStatuses: configured?.retryableErrorStatuses
? new Set(configured.retryableErrorStatuses)
: DEFAULT_STREAM_RECONNECT_POLICY.retryableErrorStatuses,
- streamIdleReconnectPolicy: resolveRetryPolicy(
- configured?.streamIdleReconnectPolicy,
- DEFAULT_STREAM_RECONNECT_POLICY.streamIdleReconnectPolicy,
- ),
+ streamIdleReconnectPolicy: resolveRetryPolicy(configured?.streamIdleReconnectPolicy, {
+ ...DEFAULT_STREAM_RECONNECT_POLICY.streamIdleReconnectPolicy,
+ maxAttempts: keepAlive
+ ? Infinity
+ : DEFAULT_STREAM_RECONNECT_POLICY.streamIdleReconnectPolicy.maxAttempts,
+ }),
streamOpenReconnectPolicy: resolveRetryPolicy(
configured?.streamOpenReconnectPolicy,
DEFAULT_STREAM_RECONNECT_POLICY.streamOpenReconnectPolicy,
@@ -83,9 +86,12 @@ function resolveStreamReconnectPolicy(
* Internal configuration for following a durable event stream.
*/
interface FollowStreamInput {
+ /** Called once after consuming the durable tail captured when the connection opens. */
+ readonly onCaughtUp?: () => void;
readonly host: string;
- /** Keep reconnecting after empty streams until the consumer aborts or stops iteration. */
+ /** Keep following empty streams unless the caller configures an idle retry limit. */
readonly keepAlive?: boolean;
+ readonly resolveReconnectPolicy?: () => StreamReconnectPolicy | undefined;
readonly streamReconnectPolicy?: StreamReconnectPolicy;
/** @internal Test override for reconnecting an open stream that stops producing bytes. */
readonly streamReadIdleTimeoutMs?: number;
@@ -125,22 +131,33 @@ export async function* followStreamIterable(
);
}
- const retryPolicy = resolveStreamReconnectPolicy(input.streamReconnectPolicy);
- const idleRetryPolicy = retryPolicy.streamIdleReconnectPolicy;
+ const resolvePolicy = () =>
+ resolveStreamReconnectPolicy(
+ input.resolveReconnectPolicy === undefined
+ ? input.streamReconnectPolicy
+ : input.resolveReconnectPolicy(),
+ input.keepAlive,
+ );
+ let retryPolicy = resolvePolicy();
+ let idleRetryPolicy = retryPolicy.streamIdleReconnectPolicy;
let startIndex = input.startIndex;
let reconnectDelayMs = idleRetryPolicy.baseDelayMs;
let idleReconnects = 0;
let initialConnection = true;
let tailIndex: number | undefined;
+ let caughtUp = false;
while (true) {
+ retryPolicy = resolvePolicy();
+ idleRetryPolicy = retryPolicy.streamIdleReconnectPolicy;
let connection: OpenedStream;
try {
connection = await openStreamBody({
...input,
retryPolicy,
startIndex,
- requestTailIndex: input.follow === false && tailIndex === undefined,
+ requestTailIndex:
+ (input.follow === false || input.onCaughtUp !== undefined) && tailIndex === undefined,
});
} catch (error) {
if (input.signal?.aborted) {
@@ -149,7 +166,7 @@ export async function* followStreamIterable(
throw error;
}
- if (input.follow === false && tailIndex === undefined) {
+ if ((input.follow === false || input.onCaughtUp !== undefined) && tailIndex === undefined) {
tailIndex = connection.tailIndex;
if (tailIndex === undefined) {
connection.close();
@@ -160,7 +177,11 @@ export async function* followStreamIterable(
}
}
- if (tailIndex !== undefined && startIndex > tailIndex) {
+ if (!caughtUp && tailIndex !== undefined && startIndex > tailIndex) {
+ caughtUp = true;
+ input.onCaughtUp?.();
+ }
+ if (input.follow === false && tailIndex !== undefined && startIndex > tailIndex) {
connection.close();
return;
}
@@ -169,6 +190,7 @@ export async function* followStreamIterable(
let leaseEnded = false;
try {
for await (const event of readNdjsonStream(connection.body, {
+ signal: input.signal,
controlVersion: connection.controlVersion,
idleTimeoutMs: input.streamReadIdleTimeoutMs ?? DEFAULT_STREAM_READ_IDLE_TIMEOUT_MS,
onLeaseEnded: () => {
@@ -182,7 +204,11 @@ export async function* followStreamIterable(
idleReconnects = 0;
yield event;
- if (tailIndex !== undefined && startIndex > tailIndex) {
+ if (!caughtUp && tailIndex !== undefined && startIndex > tailIndex) {
+ caughtUp = true;
+ input.onCaughtUp?.();
+ }
+ if (input.follow === false && tailIndex !== undefined && startIndex > tailIndex) {
return;
}
}
@@ -192,6 +218,7 @@ export async function* followStreamIterable(
connection.close();
}
+ idleRetryPolicy = resolvePolicy().streamIdleReconnectPolicy;
if (input.signal?.aborted || input.startIndex < 0 || idleRetryPolicy.maxAttempts === 0) {
return;
}
@@ -201,7 +228,6 @@ export async function* followStreamIterable(
}
if (
- input.keepAlive !== true &&
!deliveredEvent &&
!initialConnection &&
(idleReconnects += 1) >= idleRetryPolicy.maxAttempts
@@ -237,7 +263,7 @@ export async function openStreamBody(
input: OpenStreamInput & { readonly retryPolicy?: ResolvedStreamReconnectPolicy },
): Promise<OpenedStream> {
const retryPolicy =
- input.retryPolicy ?? resolveStreamReconnectPolicy(input.streamReconnectPolicy);
+ input.retryPolicy ?? resolveStreamReconnectPolicy(input.streamReconnectPolicy, input.keepAlive);
const openRetryPolicy = retryPolicy.streamOpenReconnectPolicy;
let lastStatus: number | undefined;
let lastBody: string | undefined;
@@ -260,6 +286,7 @@ export async function openStreamBody(
}
for (let attempt = 0; attempt < openRetryPolicy.maxAttempts; attempt += 1) {
+ input.signal?.throwIfAborted();
const url = createClientUrl(
input.host,
createEveSessionStreamRoutePath(input.sessionId),
@@ -267,6 +294,7 @@ export async function openStreamBody(
);
const headers = await input.resolveHeaders();
+ input.signal?.throwIfAborted();
const connectionController = new AbortController();
const signal = input.signal
? AbortSignal.any([input.signal, connectionController.signal])
diff --git a/packages/eve/src/client/session-event-stream.ts b/packages/eve/src/client/session-event-stream.ts
new file mode 100644
index 000000000..f971225fd
--- /dev/null
+++ b/packages/eve/src/client/session-event-stream.ts
@@ -0,0 +1,135 @@
+import { followClientSession, type ClientSession } from "#client/session.js";
+import type { MessageStreamEvent } from "#protocol/message.js";
+import type { StreamOptions } from "#client/types.js";
+
+export interface SessionEventStreamOptions {
+ readonly startIndex?: number;
+ readonly catchUp?: boolean;
+ readonly headers?: Readonly<Record<string, string>>;
+ readonly streamReconnectPolicy?: StreamOptions["streamReconnectPolicy"];
+ readonly onEvent: (event: MessageStreamEvent) => void;
+ readonly onError: (error: unknown) => void;
+}
+
+/** One transport, with short-lived readers for catch-up and turn completion. */
+export class SessionEventStream {
+ readonly #controller = new AbortController();
+ readonly #readers = new Set<SessionEventReader>();
+ readonly #caughtUp = Promise.withResolvers<void>();
+ readonly caughtUp = this.#caughtUp.promise;
+ #ended = false;
+ #error: unknown;
+ #headers: Readonly<Record<string, string>> | undefined;
+ #reconnectPolicy: StreamOptions["streamReconnectPolicy"];
+
+ constructor(session: ClientSession, options: SessionEventStreamOptions) {
+ this.setOptions(options);
+ void this.caughtUp.catch(() => {});
+ if (!options.catchUp) this.#caughtUp.resolve();
+ void (async () => {
+ try {
+ for await (const event of followClientSession(session, {
+ signal: this.#controller.signal,
+ resolveHeaders: () => this.#headers,
+ resolveReconnectPolicy: () => this.#reconnectPolicy,
+ startIndex: options.startIndex,
+ onCaughtUp: options.catchUp ? () => this.#caughtUp.resolve() : undefined,
+ })) {
+ if (this.#controller.signal.aborted) return;
+ options.onEvent(event);
+ for (const reader of this.#readers) reader.push(event);
+ if (event.type === "session.completed" || event.type === "session.failed") break;
+ }
+ this.#caughtUp.resolve();
+ this.#finish();
+ } catch (error) {
+ this.#caughtUp.reject(error);
+ this.#finish(error);
+ if (!this.#controller.signal.aborted) options.onError(error);
+ }
+ })();
+ }
+
+ subscribe(signal?: AbortSignal): SessionEventReader {
+ const reader = new SessionEventReader(() => this.#readers.delete(reader), signal);
+ if (this.#ended) reader.end(this.#error);
+ else this.#readers.add(reader);
+ return reader;
+ }
+
+ get ended(): boolean {
+ return this.#ended;
+ }
+
+ setOptions(options: Pick<SessionEventStreamOptions, "headers" | "streamReconnectPolicy">): void {
+ if ("headers" in options) this.#headers = options.headers;
+ if ("streamReconnectPolicy" in options) this.#reconnectPolicy = options.streamReconnectPolicy;
+ }
+
+ close(): void {
+ const error = new DOMException("Session stream was detached.", "AbortError");
+ this.#controller.abort(error);
+ this.#caughtUp.reject(error);
+ this.#finish(error);
+ }
+
+ #finish(error?: unknown): void {
+ if (this.#ended) return;
+ this.#ended = true;
+ this.#error = error;
+ for (const reader of this.#readers) reader.end(error);
+ }
+}
+
+export class SessionEventReader implements AsyncIterable<MessageStreamEvent>, Disposable {
+ #events: MessageStreamEvent[] = [];
+ #wake = Promise.withResolvers<void>();
+ #ended = false;
+ #error: unknown;
+ readonly #dispose: () => void;
+
+ constructor(unsubscribe: () => void, signal?: AbortSignal) {
+ const abort = () => this.end(signal?.reason);
+ signal?.addEventListener("abort", abort, { once: true });
+ this.#dispose = () => {
+ signal?.removeEventListener("abort", abort);
+ unsubscribe();
+ };
+ if (signal?.aborted) abort();
+ }
+
+ push(event: MessageStreamEvent): void {
+ if (this.#ended) return;
+ this.#events.push(event);
+ this.#wake.resolve();
+ }
+
+ end(error?: unknown): void {
+ if (this.#ended) return;
+ this.#ended = true;
+ this.#error = error;
+ this.#wake.resolve();
+ }
+
+ [Symbol.dispose](): void {
+ this.end();
+ this.#events = [];
+ this.#dispose();
+ }
+
+ discard(): void {
+ this.#events = [];
+ }
+
+ async *[Symbol.asyncIterator](): AsyncGenerator<MessageStreamEvent> {
+ for (;;) {
+ while (this.#events.length > 0) yield this.#events.shift()!;
+ if (this.#ended) {
+ if (this.#error !== undefined) throw this.#error;
+ return;
+ }
+ this.#wake = Promise.withResolvers<void>();
+ await this.#wake.promise;
+ }
+ }
+}
diff --git a/packages/eve/src/client/session-utils.ts b/packages/eve/src/client/session-utils.ts
index e72d5b1e6..b85eec487 100644
--- a/packages/eve/src/client/session-utils.ts
+++ b/packages/eve/src/client/session-utils.ts
@@ -1,6 +1,7 @@
import type {
AuthorizationRequiredStreamEvent,
MessageCompletedStreamEvent,
+ MessageStreamEvent,
TurnFailureStreamEvent,
UnstampedMessageStreamEvent,
} from "#protocol/message.js";
@@ -80,3 +81,11 @@ function isFinalMessageCompleted(
): event is MessageCompletedStreamEvent {
return event.type === "message.completed" && event.data.finishReason !== "tool-calls";
}
+
+export function updatePendingAuthorizations(pending: Set<string>, event: MessageStreamEvent): void {
+ if (event.type === "authorization.required" && event.data.webhookUrl !== undefined) {
+ pending.add(event.data.name);
+ } else if (event.type === "authorization.completed") {
+ pending.delete(event.data.name);
+ }
+}
diff --git a/packages/eve/src/client/session.test.ts b/packages/eve/src/client/session.test.ts
index 578b54af1..bda857b4c 100644
--- a/packages/eve/src/client/session.test.ts
+++ b/packages/eve/src/client/session.test.ts
@@ -58,11 +58,11 @@ function createAcceptedResponse() {
);
}
-function createSessionNotActiveResponse() {
+function createSessionNotReadyResponse() {
return Response.json(
{
- code: "session_not_active",
- error: "The session is no longer active.",
+ code: "session_not_ready",
+ error: "The session is not ready to accept messages yet.",
ok: false,
},
{ status: 409 },
@@ -296,13 +296,15 @@ describe("ClientSession", () => {
await expect(session.cancel()).rejects.toThrow("Cancel route returned an invalid response");
});
- it("retries session_not_active with exponential backoff", async () => {
+ it("retries session_not_ready beyond three seconds with current headers", async () => {
let headerResolution = 0;
const observedHeaders: string[] = [];
- const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) => {
+ const observedPaths: string[] = [];
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (request, init) => {
+ observedPaths.push(new URL(String(request)).pathname);
observedHeaders.push(new Headers(init?.headers).get("x-attempt") ?? "");
- return fetchMock.mock.calls.length <= 3
- ? createSessionNotActiveResponse()
+ return fetchMock.mock.calls.length <= 4
+ ? createSessionNotReadyResponse()
: createAcceptedResponse();
});
const session = createSession(
@@ -331,19 +333,25 @@ describe("ClientSession", () => {
await vi.advanceTimersByTimeAsync(999);
expect(fetchMock).toHaveBeenCalledTimes(3);
await vi.advanceTimersByTimeAsync(1);
+ expect(fetchMock).toHaveBeenCalledTimes(4);
+
+ await vi.advanceTimersByTimeAsync(1_999);
+ expect(fetchMock).toHaveBeenCalledTimes(4);
+ await vi.advanceTimersByTimeAsync(1);
await expect(sent).resolves.toMatchObject({ sessionId: "session_1" });
} finally {
vi.useRealTimers();
}
- expect(fetchMock).toHaveBeenCalledTimes(4);
- expect(observedHeaders).toEqual(["1", "2", "3", "4"]);
+ expect(fetchMock).toHaveBeenCalledTimes(5);
+ expect(observedHeaders).toEqual(["1", "2", "3", "4", "5"]);
+ expect(new Set(observedPaths)).toEqual(new Set(["/eve/v1/session/session_1"]));
});
- it("exposes the typed session_not_active conflict after three retries", async () => {
+ it("exposes the typed session_not_ready conflict after the readiness deadline", async () => {
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockImplementation(async () => createSessionNotActiveResponse());
+ .mockImplementation(async () => createSessionNotReadyResponse());
const session = createSession({ sessionId: "session_1", streamIndex: 0 });
vi.useFakeTimers();
@@ -358,47 +366,47 @@ describe("ClientSession", () => {
expect(error).toBeInstanceOf(ClientError);
expect(error).toMatchObject({
- code: "session_not_active",
- message: "The session is no longer active.",
+ code: "session_not_ready",
+ message: "The session is not ready to accept messages yet.",
status: 409,
});
- expect(fetchMock).toHaveBeenCalledTimes(4);
+ expect(fetchMock).toHaveBeenCalledTimes(14);
});
- it("does not retry another conflict from send", async () => {
- const fetchMock = vi
- .spyOn(globalThis, "fetch")
- .mockResolvedValue(
- Response.json(
- { code: "another_conflict", error: "The request conflicts.", ok: false },
- { status: 409 },
- ),
- );
- const session = createSession({ sessionId: "session_1", streamIndex: 0 });
+ it.each(["session_not_active", "another_conflict"])(
+ "does not retry %s from send",
+ async (code) => {
+ const fetchMock = vi
+ .spyOn(globalThis, "fetch")
+ .mockResolvedValue(
+ Response.json({ code, error: "The request conflicts.", ok: false }, { status: 409 }),
+ );
+ const session = createSession({ sessionId: "session_1", streamIndex: 0 });
- await expect(session.send("conflict")).rejects.toMatchObject({
- code: "another_conflict",
- status: 409,
- });
- expect(fetchMock).toHaveBeenCalledOnce();
- });
+ await expect(session.send("conflict")).rejects.toMatchObject({
+ code,
+ status: 409,
+ });
+ expect(fetchMock).toHaveBeenCalledOnce();
+ },
+ );
- it("does not retry session_not_active from respond", async () => {
+ it("does not retry session_not_ready from respond", async () => {
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockImplementation(async () => createSessionNotActiveResponse());
+ .mockImplementation(async () => createSessionNotReadyResponse());
const session = createSession({ sessionId: "session_1", streamIndex: 0 });
await expect(
session.respond([{ requestId: "approval_1", optionId: "approve" }]),
- ).rejects.toMatchObject({ code: "session_not_active", status: 409 });
+ ).rejects.toMatchObject({ code: "session_not_ready", status: 409 });
expect(fetchMock).toHaveBeenCalledOnce();
});
- it("stops session_not_active backoff when send is aborted", async () => {
+ it("stops session_not_ready backoff when send is aborted", async () => {
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockImplementation(async () => createSessionNotActiveResponse());
+ .mockImplementation(async () => createSessionNotReadyResponse());
const session = createSession({ sessionId: "session_1", streamIndex: 0 });
const controller = new AbortController();
@@ -740,6 +748,23 @@ describe("ClientSession", () => {
expect(result.inputRequests.map((request) => request.requestId)).toEqual(["approval_1"]);
});
+ it("does not rewind the shared cursor when a historical reader stops early", async () => {
+ vi.spyOn(globalThis, "fetch").mockResolvedValue(
+ createStreamResponse([
+ {
+ type: "session.waiting",
+ data: { continuationToken: "session-id", wait: "next-user-message" },
+ },
+ ]),
+ );
+ const session = createSession({ sessionId: "session_1", streamIndex: 7 });
+ for await (const _event of session.stream({ startIndex: 0 })) {
+ expect(session.state.streamIndex).toBe(7);
+ break;
+ }
+ expect(session.state.streamIndex).toBe(7);
+ });
+
it("opens a one-shot tail-relative stream without advancing the absolute cursor", async () => {
const requests: string[] = [];
vi.spyOn(globalThis, "fetch").mockImplementation(async (request) => {
diff --git a/packages/eve/src/client/session.ts b/packages/eve/src/client/session.ts
index 23f9c7577..e44532640 100644
--- a/packages/eve/src/client/session.ts
+++ b/packages/eve/src/client/session.ts
@@ -1,3 +1,4 @@
+import { updatePendingAuthorizations } from "#client/session-utils.js";
import type { MessageStreamEvent } from "#protocol/message.js";
import { EVE_SESSION_ID_HEADER, isCurrentTurnBoundaryEvent } from "#protocol/message.js";
import { EVE_SESSION_ROUTE_PATH, createEveSessionRoutePath } from "#protocol/routes.js";
@@ -19,6 +20,7 @@ import type {
ClientSessionState,
CompactResult,
ClientRedirectPolicy,
+ CreateSessionOptions,
RespondTurnOptions,
ResetResult,
SendTurnInput,
@@ -28,8 +30,17 @@ import type {
StreamOptions,
} from "#client/types.js";
-const SESSION_SEND_RETRY_COUNT = 3;
const SESSION_SEND_RETRY_BASE_DELAY_MS = 250;
+const SESSION_SEND_RETRY_MAX_DELAY_MS = 2_000;
+const SESSION_SEND_READY_TIMEOUT_MS = 20_000;
+const followSession = Symbol("followClientSession");
+
+interface FollowSessionOptions extends StreamOptions {
+ readonly headers?: Readonly<Record<string, string>>;
+ readonly onCaughtUp?: () => void;
+ readonly resolveReconnectPolicy?: () => StreamOptions["streamReconnectPolicy"];
+ readonly resolveHeaders?: () => Readonly<Record<string, string>> | undefined;
+}
/**
* Internal interface that a {@link ClientSession} uses to access client-level
@@ -67,6 +78,16 @@ export class ClientSession {
};
}
+ /** @internal */
+ static async prewarm(
+ context: ClientSessionContext,
+ options: CreateSessionOptions = {},
+ ): Promise<ClientSession> {
+ const response = await postCreateSession(context, options);
+ const { sessionId } = await readAcceptedMessage(response);
+ return new ClientSession(context, { sessionId, streamIndex: 0 });
+ }
+
/** Current fixed session identity and durable stream cursor. */
get state(): ClientSessionState {
return this.#state;
@@ -113,11 +134,11 @@ export class ClientSession {
async #send<TOutput = unknown>(
input: SendTurnPayload<TOutput>,
- retrySessionNotActive: boolean,
+ retrySessionNotReady: boolean,
): Promise<MessageResponse<TOutput>> {
const initialStreamIndex = this.#state.streamIndex;
const path = createEveSessionRoutePath(this.#state.sessionId);
- const response = retrySessionNotActive
+ const response = retrySessionNotReady
? await postSessionSend(this.#context, path, input)
: await postTurn(this.#context, path, input, false);
const { sessionId: responseSessionId, deliveryId } = await readAcceptedMessage(
@@ -185,6 +206,10 @@ export class ClientSession {
return this.#streamAndAdvance(options);
}
+ [followSession](options: FollowSessionOptions): AsyncIterable<MessageStreamEvent> {
+ return this.#streamAndAdvance({ ...options, keepAlive: true });
+ }
+
#messageResponse<TOutput>(
response: Response,
input: SendTurnPayload,
@@ -194,7 +219,8 @@ export class ClientSession {
response.body?.cancel().catch(() => {});
return new MessageResponse<TOutput>({
cancelTurn: async (turnId) => await this.cancel({ turnId }),
- createStream: () => this.#createEventStream(initialStreamIndex, input, deliveryId),
+ createStream: (source) =>
+ this.#createEventStream(initialStreamIndex, input, deliveryId, source),
sessionId: this.#state.sessionId,
});
}
@@ -203,19 +229,21 @@ export class ClientSession {
initialStreamIndex: number,
input: SendTurnPayload,
deliveryId?: string,
+ source?: AsyncIterable<MessageStreamEvent>,
): AsyncGenerator<MessageStreamEvent> {
let eventCount = 0;
let started = deliveryId === undefined;
let reachedBoundary = false;
const pendingAuthorizations = new Set<string>();
try {
- for await (const event of this.#readStream({
- headers: input.headers,
- keepAlive: shouldKeepActiveTurnAlive(input.streamReconnectPolicy),
- signal: input.signal,
- startIndex: initialStreamIndex,
- streamReconnectPolicy: input.streamReconnectPolicy,
- })) {
+ for await (const event of source ??
+ this.#readStream({
+ headers: input.headers,
+ keepAlive: true,
+ signal: input.signal,
+ startIndex: initialStreamIndex,
+ streamReconnectPolicy: input.streamReconnectPolicy,
+ })) {
eventCount += 1;
if (deliveryId !== undefined) {
const matches = event.meta?.deliveryIds?.includes(deliveryId) === true;
@@ -229,11 +257,7 @@ export class ClientSession {
if (!terminal && event.meta?.deliveryIds !== undefined && !matches) continue;
started = true;
}
- if (event.type === "authorization.required" && event.data.webhookUrl !== undefined) {
- pendingAuthorizations.add(event.data.name);
- } else if (event.type === "authorization.completed") {
- pendingAuthorizations.delete(event.data.name);
- }
+ updatePendingAuthorizations(pendingAuthorizations, event);
reachedBoundary =
isCurrentTurnBoundaryEvent(event) &&
(event.type !== "session.waiting" || pendingAuthorizations.size === 0);
@@ -248,89 +272,116 @@ export class ClientSession {
);
}
} finally {
- this.#state = {
- sessionId: this.#state.sessionId,
- streamIndex: Math.max(this.#state.streamIndex, initialStreamIndex + eventCount),
- };
+ this.#advanceStreamIndex(initialStreamIndex + eventCount);
}
}
- async *#streamAndAdvance(options?: StreamOptions): AsyncGenerator<MessageStreamEvent> {
+ async *#streamAndAdvance(
+ options?: FollowSessionOptions & { readonly keepAlive?: boolean },
+ ): AsyncGenerator<MessageStreamEvent> {
const startIndex = options?.startIndex ?? this.#state.streamIndex;
let eventCount = 0;
- try {
- for await (const event of this.#readStream({
- follow: options?.follow,
- signal: options?.signal,
- startIndex,
- streamReconnectPolicy: options?.streamReconnectPolicy,
- })) {
- eventCount += 1;
- yield event;
- }
- } finally {
- if (startIndex >= 0) {
- this.#state = {
- sessionId: this.#state.sessionId,
- streamIndex: startIndex + eventCount,
- };
- }
+ for await (const event of this.#readStream({
+ follow: options?.follow,
+ headers: options?.headers,
+ keepAlive: options?.keepAlive,
+ onCaughtUp: options?.onCaughtUp,
+ resolveHeaders: options?.resolveHeaders,
+ resolveReconnectPolicy: options?.resolveReconnectPolicy,
+ signal: options?.signal,
+ startIndex,
+ streamReconnectPolicy: options?.streamReconnectPolicy,
+ })) {
+ eventCount += 1;
+ if (startIndex >= 0) this.#advanceStreamIndex(startIndex + eventCount);
+ yield event;
}
}
+ #advanceStreamIndex(streamIndex: number): void {
+ this.#state = {
+ sessionId: this.#state.sessionId,
+ streamIndex: Math.max(this.#state.streamIndex, streamIndex),
+ };
+ }
+
#readStream(input: {
+ readonly onCaughtUp?: () => void;
+ readonly resolveReconnectPolicy?: () => StreamOptions["streamReconnectPolicy"];
readonly follow?: boolean;
readonly headers?: Readonly<Record<string, string>>;
readonly keepAlive?: boolean;
readonly signal?: AbortSignal;
readonly startIndex: number;
readonly streamReconnectPolicy?: StreamOptions["streamReconnectPolicy"];
+ readonly resolveHeaders?: () => Readonly<Record<string, string>> | undefined;
}): AsyncIterable<MessageStreamEvent> {
return followStreamIterable({
+ onCaughtUp: input.onCaughtUp,
follow: input.follow,
host: this.#context.host,
keepAlive: input.keepAlive,
- resolveHeaders: () => this.#context.resolveHeaders(input.headers),
+ resolveHeaders: () => this.#context.resolveHeaders(input.resolveHeaders?.() ?? input.headers),
redirect: this.#context.redirect,
sessionId: this.#state.sessionId,
signal: input.signal,
startIndex: input.startIndex,
streamReconnectPolicy: input.streamReconnectPolicy,
+ resolveReconnectPolicy: input.resolveReconnectPolicy,
});
}
}
+/** @internal Follow continuously while the frontend owns the session. */
+export function followClientSession(
+ session: ClientSession,
+ options: FollowSessionOptions,
+): AsyncIterable<MessageStreamEvent> {
+ return session[followSession](options);
+}
+
async function postSessionSend(
context: ClientSessionContext,
path: string,
input: SendTurnPayload,
): Promise<Response> {
+ const readyDeadline = Date.now() + SESSION_SEND_READY_TIMEOUT_MS;
let retryDelayMs = SESSION_SEND_RETRY_BASE_DELAY_MS;
- for (let retry = 0; ; retry += 1) {
+ for (;;) {
try {
return await postTurn(context, path, input, false);
} catch (error) {
- if (!isSessionNotActive(error) || retry >= SESSION_SEND_RETRY_COUNT) throw error;
+ if (!isSessionNotReady(error)) throw error;
+ const remainingMs = readyDeadline - Date.now();
+ if (remainingMs <= 0) throw error;
+ await sleep(Math.min(retryDelayMs, remainingMs), input.signal);
}
- await sleep(retryDelayMs, input.signal);
input.signal?.throwIfAborted();
- retryDelayMs *= 2;
+ retryDelayMs = Math.min(retryDelayMs * 2, SESSION_SEND_RETRY_MAX_DELAY_MS);
}
}
-function isSessionNotActive(error: unknown): error is ClientError {
- return (
- error instanceof ClientError && error.status === 409 && error.code === "session_not_active"
- );
-}
-
-function shouldKeepActiveTurnAlive(policy: StreamOptions["streamReconnectPolicy"]): boolean {
- if (policy && "reconnect" in policy) {
- return false;
+async function postCreateSession(
+ context: ClientSessionContext,
+ options: CreateSessionOptions,
+): Promise<Response> {
+ const headers = await context.resolveHeaders(options.headers);
+ const response = await fetch(createClientUrl(context.host, EVE_SESSION_ROUTE_PATH), {
+ headers,
+ method: "POST",
+ redirect: context.redirect,
+ signal: options.signal ?? null,
+ });
+ if (!response.ok) {
+ const responseBody = await response.text();
+ throw new ClientError(response.status, responseBody, response.headers);
}
+ return response;
+}
- return policy?.streamIdleReconnectPolicy?.maxAttempts === undefined;
+function isSessionNotReady(error: unknown): error is ClientError {
+ return error instanceof ClientError && error.status === 409 && error.code === "session_not_ready";
}
async function postTurn(
diff --git a/packages/eve/src/client/sessions.test.ts b/packages/eve/src/client/sessions.test.ts
index e03e2e448..f6e672ac3 100644
--- a/packages/eve/src/client/sessions.test.ts
+++ b/packages/eve/src/client/sessions.test.ts
@@ -1,6 +1,8 @@
-import { afterEach, describe, expect, it, vi } from "vitest";
+import { afterEach, describe, expect, expectTypeOf, it, vi } from "vitest";
import { Client } from "#client/client.js";
+import type { CreatedClientSession, CreatedIdleClientSession } from "#client/sessions.js";
+import type { SendTurnInput } from "#client/types.js";
import { EVE_MESSAGE_STREAM_VERSION, EVE_STREAM_VERSION_HEADER } from "#protocol/message.js";
afterEach(() => {
@@ -8,6 +10,27 @@ afterEach(() => {
});
describe("Client.sessions", () => {
+ it("creates a session without starting a turn or opening its stream", async () => {
+ const requests: Array<{ readonly body?: string; readonly url: string }> = [];
+ vi.spyOn(globalThis, "fetch").mockImplementation(async (request, init) => {
+ requests.push({ body: init?.body as string | undefined, url: String(request) });
+ return Response.json(
+ { ok: true, sessionId: "wrun_prewarmer", status: "accepted" },
+ { status: 202 },
+ );
+ });
+ const client = new Client({ host: "https://eve.test" });
+
+ const created = await client.sessions.create();
+ expectTypeOf(created).toEqualTypeOf<CreatedIdleClientSession>();
+ const { session } = created;
+
+ expect(requests).toHaveLength(1);
+ expect(new URL(requests[0]!.url).pathname).toBe("/eve/v1/session");
+ expect(requests[0]!.body).toBeUndefined();
+ expect(session.state).toEqual({ sessionId: "wrun_prewarmer", streamIndex: 0 });
+ });
+
it("returns structured output when fetch instrumentation clones the live stream", async () => {
const events = [
{ type: "result.completed", data: { result: { answer: "child-result" } } },
@@ -82,7 +105,10 @@ describe("Client.sessions", () => {
});
const client = new Client({ host: "https://eve.test" });
- const { response, session } = await client.sessions.create({ message: "hello" });
+ const input: SendTurnInput<{ answer: string }> = { message: "hello" };
+ const created = await client.sessions.create(input);
+ expectTypeOf(created).toEqualTypeOf<CreatedClientSession<{ answer: string }>>();
+ const { response, session } = created;
await response.result();
expect(new URL(requests[0]!.url).pathname).toBe("/eve/v1/session");
diff --git a/packages/eve/src/client/sessions.ts b/packages/eve/src/client/sessions.ts
index 3f654037b..f9668ee4e 100644
--- a/packages/eve/src/client/sessions.ts
+++ b/packages/eve/src/client/sessions.ts
@@ -1,6 +1,6 @@
import { ClientSession, type ClientSessionContext } from "#client/session.js";
import type { MessageResponse } from "#client/message-response.js";
-import type { SendTurnInput } from "#client/types.js";
+import type { CreateSessionOptions, SendTurnInput } from "#client/types.js";
/** Result of explicitly creating an ID-addressed client session. */
export interface CreatedClientSession<TOutput = unknown> {
@@ -8,6 +8,11 @@ export interface CreatedClientSession<TOutput = unknown> {
readonly session: ClientSession;
}
+/** Result of creating a session before its first turn. */
+export interface CreatedIdleClientSession {
+ readonly session: ClientSession;
+}
+
/** Collection surface for explicitly creating or attaching ID-addressed sessions. */
export class ClientSessions {
readonly #context: ClientSessionContext;
@@ -17,11 +22,17 @@ export class ClientSessions {
this.#context = context;
}
- /** Creates a session immediately and returns its fixed handle plus first-turn response. */
+ /** Creates a session and starts its first turn with a message. */
async create<TOutput = unknown>(
input: SendTurnInput<TOutput>,
- ): Promise<CreatedClientSession<TOutput>> {
- return await ClientSession.create(this.#context, input);
+ ): Promise<CreatedClientSession<TOutput>>;
+ /** Creates a conversation session before its first turn. */
+ async create(options?: CreateSessionOptions): Promise<CreatedIdleClientSession>;
+ async create<TOutput = unknown>(
+ input: CreateSessionOptions | SendTurnInput<TOutput> = {},
+ ): Promise<CreatedClientSession<TOutput> | CreatedIdleClientSession> {
+ if ("message" in input) return await ClientSession.create(this.#context, input);
+ return { session: await ClientSession.prewarm(this.#context, input) };
}
/** Attaches a fixed handle to a known session ID without performing I/O. */
diff --git a/packages/eve/src/client/types.ts b/packages/eve/src/client/types.ts
index df8ffb8e3..8e95a00e0 100644
--- a/packages/eve/src/client/types.ts
+++ b/packages/eve/src/client/types.ts
@@ -108,6 +108,14 @@ export interface SendTurnInput<TOutput = unknown> extends SendTurnOptions<TOutpu
readonly message: string | UserContent;
}
+/** Request options for creating a conversation session before its first turn. */
+export interface CreateSessionOptions {
+ /** Abort signal for cancelling the creation request. */
+ readonly signal?: AbortSignal;
+ /** Additional headers for this request only. */
+ readonly headers?: Readonly<Record<string, string>>;
+}
+
/** Options shared by message sends and HITL responses on a client session. */
export interface SendTurnOptions<TOutput = unknown> {
/** Policy for a message sent while the fixed session has an active turn. */
diff --git a/packages/eve/src/compiler/extension-compatibility.ts b/packages/eve/src/compiler/extension-compatibility.ts
index 8421ab7fc..d8905b63e 100644
--- a/packages/eve/src/compiler/extension-compatibility.ts
+++ b/packages/eve/src/compiler/extension-compatibility.ts
@@ -75,8 +75,8 @@ const EXTENSION_CAPABILITY_CONTRACTS = {
},
},
channel: {
- current: 24,
- supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 23, 24],
+ current: 25,
+ supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 23, 24, 25],
dropped: {
12: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",
19: "Continuation rekey was removed; channel extensions must use additive continuation.alias instead.",
@@ -86,8 +86,8 @@ const EXTENSION_CAPABILITY_CONTRACTS = {
},
},
schedule: {
- current: 12,
- supported: [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12],
+ current: 13,
+ supported: [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13],
dropped: {
5: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",
},
diff --git a/packages/eve/src/evals/context.ts b/packages/eve/src/evals/context.ts
index 2c5dcb08e..f8dff7115 100644
--- a/packages/eve/src/evals/context.ts
+++ b/packages/eve/src/evals/context.ts
@@ -1,4 +1,4 @@
-import { EvalSessionManager } from "#evals/session.js";
+import { EvalSessionManager } from "#evals/session-manager.js";
import { AssertionCollector } from "#evals/assertions/collector.js";
import { createScopedAssertions } from "#evals/assertions/scoped.js";
import { buildJudgeContext } from "#evals/judge.js";
@@ -27,62 +27,22 @@ export function createEvalContext(deps: {
readonly log: (message: string) => void;
}): { readonly context: EveEvalContext; readonly collector: AssertionCollector } {
const collector = deps.collector;
- let lastPrompt = "";
-
- const primary = () => deps.manager.primary;
- const replyMessage = () => deps.manager.lastTurnSession()?.lastTurn?.message ?? null;
-
const judge = buildJudgeContext({
collector,
- getReply: replyMessage,
- getInput: () => lastPrompt,
+ getReply: () => deps.manager.lastTurnSession()?.lastTurn?.message ?? null,
+ getInput: () => deps.manager.lastTurnSession()?.lastInput ?? "",
judge: deps.judge,
});
const context: EveEvalContext = {
- // EveEvalSession — drive the primary session.
- get events() {
- return primary().events;
- },
- get transcript() {
- return primary().transcript;
- },
- get pendingInputRequests() {
- return primary().pendingInputRequests;
- },
- get state() {
- return primary().state;
- },
- get sessionId() {
- return primary().sessionId;
- },
- cancel: () => primary().cancel(),
- requireInputRequest: (filter) => primary().requireInputRequest(filter),
- respond: (responses, options) => primary().respond(responses, options),
- startRespond: (responses, options) => primary().startRespond(responses, options),
- respondAll: (optionId) => primary().respondAll(optionId),
- send: (message, options) => {
- lastPrompt = typeof message === "string" ? message : "";
- return primary().send(message, options);
- },
- start: (message, options) => {
- lastPrompt = message;
- return primary().start(message, options);
- },
- sendFile: (text, filePath, mediaType) => {
- lastPrompt = text;
- return primary().sendFile(text, filePath, mediaType);
- },
+ session: (options) => deps.manager.session(options),
+ send: (message, options) => deps.manager.send(message, options),
// Run context.
signal: deps.signal,
target: deps.target,
- get reply() {
- return replyMessage();
- },
log: deps.log,
sleep: (ms) => sleep(ms, deps.signal),
- newSession: () => deps.manager.newSession(),
...createScopedAssertions(collector, { timing: "final", select: (result) => result }),
// Value-level assertion over an explicit value.
@@ -91,7 +51,9 @@ export function createEvalContext(deps: {
skip: (reason) => {
if (reason.trim().length === 0) throw new Error("skip() requires a non-empty reason.");
if (collector.hasEntries || deps.manager.hasActivity()) {
- throw new Error("skip() must be called before sending messages or recording assertions.");
+ throw new Error(
+ "skip() must be called before creating sessions, sending messages, or recording assertions.",
+ );
}
throw new EvalSkipped(reason);
},
diff --git a/packages/eve/src/evals/define-eval.ts b/packages/eve/src/evals/define-eval.ts
index c12f31f09..15d323cbd 100644
--- a/packages/eve/src/evals/define-eval.ts
+++ b/packages/eve/src/evals/define-eval.ts
@@ -2,7 +2,7 @@ import type { EveEvalDefinition, EveEvalInput } from "#evals/types.js";
/**
* Defines one eve eval. Each eval file is exactly one case: an imperative
- * `test(t)` function that drives the agent (`t.send`, `t.respond`, …) and
+ * `test(t)` function that drives the agent (`t.session`, `t.send`, …) and
* asserts on what it produced (`t.succeeded()`, `t.check(...)`,
* `t.judge.autoevals.*`). Organize related evals with directory nesting under
* `evals/`, or default-export an array of evals to fan one file out over a
@@ -59,7 +59,7 @@ function validateEvalInput(input: EveEvalInput): void {
rejectLegacyKey(
input,
"expected",
- "Pass the reference value to the assertion (e.g. `t.check(t.reply, includes(value))`).",
+ "Pass the reference value to the assertion (e.g. `t.check(turn.message, includes(value))`).",
);
rejectLegacyKey(input, "thresholds", "Put the threshold on the assertion: `.atLeast(n)`.");
rejectLegacyKey(
diff --git a/packages/eve/src/evals/judge.ts b/packages/eve/src/evals/judge.ts
index 431c0d2f6..7fd848e5f 100644
--- a/packages/eve/src/evals/judge.ts
+++ b/packages/eve/src/evals/judge.ts
@@ -10,7 +10,7 @@ import type { AssertionHandle, EveEvalJudgeConfig, JudgeContext, JudgeOpts } fro
/**
* Dependencies the judge namespace closes over: where to record assertions,
- * how to resolve the default graded value (`t.reply`) and the original prompt
+ * how to resolve the default graded value (the most recently settled turn) and the original prompt
* (the autoevals `input`), and the eval/config judge model.
*/
export interface JudgeDeps {
diff --git a/packages/eve/src/evals/runner/execute-task.test.ts b/packages/eve/src/evals/runner/execute-task.test.ts
index 04f259ffb..ab8b08405 100644
--- a/packages/eve/src/evals/runner/execute-task.test.ts
+++ b/packages/eve/src/evals/runner/execute-task.test.ts
@@ -42,6 +42,147 @@ function createTestEval(test: (t: EveEvalContext) => unknown, id = "test-eval"):
}
describe("executeTask", () => {
+ it("creates accepted sessions without consuming events and reuses them for sends", async () => {
+ const server = createScriptedServer([
+ { sessionId: "session_1", events: [turnStarted("turn_0", TRACE_A), sessionWaiting()] },
+ ]);
+ let evalSignal: AbortSignal;
+ const fetch = vi.spyOn(globalThis, "fetch").mockImplementation(async (request, init) => {
+ if (new URL(String(request)).pathname === "/eve/v1/session") {
+ expect(init?.body).toBeUndefined();
+ expect(init?.signal).toBe(evalSignal);
+ expect(new Headers(init?.headers).get("authorization")).toBe("Bearer eval-token");
+ expect(new Headers(init?.headers).get("x-create")).toBe("yes");
+ return Response.json({ sessionId: "session_1" }, { status: 202 });
+ }
+ return await server.fetch(request, init);
+ });
+ const onSessionStart = vi.fn();
+ const outcome = await executeTask({
+ client: new Client({ host: target.url, auth: { bearer: "eval-token" } }),
+ target,
+ onSessionStart,
+ evaluation: createTestEval(async (t) => {
+ evalSignal = t.signal;
+ const session = await t.session({ headers: { "x-create": "yes" } });
+ expect(session.sessionId).toBe("session_1");
+ expect(session.state).toEqual({ sessionId: "session_1", streamIndex: 0 });
+ expect(session.events).toEqual([]);
+ expect(fetch).toHaveBeenCalledOnce();
+ expect(onSessionStart).not.toHaveBeenCalled();
+ const turn = await session.send("Hello Alice");
+ expect(turn.session).toBe(session);
+ expect(turn.sessionId).toBe(session.sessionId);
+ }),
+ });
+ expect(outcome.error).toBeUndefined();
+ expect(server.posts.map((post) => new URL(post.url).pathname)).toEqual([
+ "/eve/v1/session/session_1",
+ ]);
+ expect(outcome.result.sessions).toHaveLength(1);
+ expect(onSessionStart).toHaveBeenCalledOnce();
+ });
+
+ it("creates a fresh session for each send in one request and continues through the returned session", async () => {
+ const server = createScriptedServer([
+ {
+ sessionId: "alice",
+ events: [turnStarted("turn_0"), messageCompleted("Alice", "turn_0"), sessionWaiting()],
+ },
+ {
+ sessionId: "bob",
+ events: [turnStarted("turn_0"), messageCompleted("Bob", "turn_0"), sessionWaiting()],
+ },
+ {
+ sessionId: "alice",
+ events: [
+ turnStarted("turn_1"),
+ messageCompleted("Welcome back Alice", "turn_1"),
+ sessionWaiting(),
+ ],
+ },
+ ]);
+ vi.spyOn(globalThis, "fetch").mockImplementation(server.fetch);
+ const outcome = await executeTask({
+ client: new Client({ host: target.url }),
+ target,
+ evaluation: createTestEval(async (t) => {
+ const alice = await t.send("I am Alice");
+ const bob = await t.send("I am Bob");
+ expect(alice.session).not.toBe(bob.session);
+ const followup = await alice.session.send("Hello again");
+ expect(followup.session).toBe(alice.session);
+ expect(alice.message).toBe("Alice");
+ expect(bob.session.events).not.toContainEqual(
+ expect.objectContaining({
+ type: "turn.started",
+ data: expect.objectContaining({ turnId: "turn_1" }),
+ }),
+ );
+ }),
+ });
+ expect(outcome.error).toBeUndefined();
+ expect(server.posts.map((post) => new URL(post.url).pathname)).toEqual([
+ "/eve/v1/session",
+ "/eve/v1/session",
+ "/eve/v1/session/alice",
+ ]);
+ expect(server.posts.map((post) => post.body)).toEqual([
+ { message: "I am Alice" },
+ { message: "I am Bob" },
+ { message: "Hello again", turnPolicy: "queue" },
+ ]);
+ expect(outcome.result.sessions).toHaveLength(2);
+ expect(outcome.result.finalMessage).toBe("Welcome back Alice");
+ });
+
+ it("creates independent sessions concurrently and does not expose failed creations", async () => {
+ const failed = new Error("Create failed");
+ let count = 0;
+ vi.spyOn(globalThis, "fetch").mockImplementation(async () => {
+ if (++count === 1) throw failed;
+ return Response.json({ sessionId: `session_${count}` }, { status: 202 });
+ });
+ const outcome = await executeTask({
+ client: new Client({ host: target.url }),
+ target,
+ evaluation: createTestEval(async (t) => {
+ await expect(t.session()).rejects.toBe(failed);
+ const [first, second] = await Promise.all([t.session(), t.session()]);
+ expect(first.sessionId).not.toBe(second.sessionId);
+ expect(first.events).toEqual([]);
+ expect(second.events).toEqual([]);
+ }),
+ });
+ expect(outcome.error).toBeUndefined();
+ expect(outcome.result.sessions?.map((session) => session.sessionId)).toEqual([
+ "session_2",
+ "session_3",
+ ]);
+ });
+
+ it("cleans up a prewarmed session when the eval times out before sending", async () => {
+ vi.spyOn(globalThis, "fetch").mockResolvedValue(
+ Response.json({ sessionId: "idle-session" }, { status: 202 }),
+ );
+ const reset = vi.spyOn(ClientSession.prototype, "reset").mockResolvedValue({
+ previousSessionId: "idle-session",
+ status: "reset",
+ });
+ const outcome = await executeTask({
+ client: new Client({ host: target.url }),
+ target,
+ evaluation: createTestEval(async (t) => {
+ await t.session();
+ await new Promise<void>(() => {});
+ }),
+ timeoutMs: 50,
+ });
+ expect(outcome.error).toMatch(/timed out|timeout/i);
+ expect(reset).toHaveBeenCalledOnce();
+ expect((reset.mock.contexts[0] as ClientSession).state.sessionId).toBe("idle-session");
+ });
+
it("settles when an eval ignores its timeout signal", async () => {
const outcome = await executeTask({
client: new Client({ host: target.url }),
@@ -180,8 +321,9 @@ describe("executeTask", () => {
target,
evaluation: createTestEval(async (t) => {
const parked = await t.send("run pwd");
+ const session = parked.session;
parked.calledTool("bash", { status: "pending", count: 1 });
- const request = t.requireInputRequest({
+ const request = session.requireInputRequest({
display: "confirmation",
input: { command: "pwd" },
optionIds: ["approve", "cancel"],
@@ -189,7 +331,7 @@ describe("executeTask", () => {
toolName: "bash",
});
expect(request.requestId).toBe("approval_1");
- const approved = await t.respondAll("approve");
+ const approved = await session.respondAll("approve");
approved.calledTool("bash", { status: "completed", count: 1 });
t.calledTool("bash", { status: "completed", count: 1 });
}, "approve"),
@@ -232,8 +374,8 @@ describe("executeTask", () => {
client: new Client({ host: target.url }),
target,
evaluation: createTestEval(async (t) => {
- await t.send("first");
- await t.send("second");
+ const { session } = await t.send("first");
+ await session.send("second");
}, "trace-contexts"),
onSessionStart,
});
@@ -251,7 +393,7 @@ describe("executeTask", () => {
]);
});
- it("exposes the primary session transcript after each turn", async () => {
+ it("exposes the session transcript after each turn", async () => {
const server = createScriptedServer([
{
sessionId: "session_1",
@@ -280,13 +422,13 @@ describe("executeTask", () => {
client: new Client({ host: target.url }),
target,
evaluation: createTestEval(async (t) => {
- await t.send("Remember marigold.");
- expect(t.transcript).toBe(
+ const { session } = await t.send("Remember marigold.");
+ expect(session.transcript).toBe(
"User:\nRemember marigold.\n\nAssistant:\nI will remember marigold.",
);
- await t.send("What word did I ask you to remember?");
- expect(t.transcript).toBe(
+ await session.send("What word did I ask you to remember?");
+ expect(session.transcript).toBe(
[
"User:\nRemember marigold.",
"Assistant:\nI will remember marigold.",
@@ -324,7 +466,7 @@ describe("executeTask", () => {
expect(server.posts[0]?.body).toEqual({ message: "case prompt" });
});
- it("captures independent sessions created by newSession", async () => {
+ it("captures independent sessions created by send", async () => {
const server = createScriptedServer([
{
sessionId: "primary",
@@ -355,8 +497,7 @@ describe("executeTask", () => {
target,
evaluation: createTestEval(async (t) => {
await t.send("primary");
- const secondary = t.newSession();
- await secondary.send("secondary");
+ const { session: secondary } = await t.send("secondary");
secondaryTranscript = secondary.transcript;
}, "multi-session"),
});
@@ -396,7 +537,8 @@ describe("executeTask", () => {
target,
evaluation: createTestEval(async (t) => {
const first = await t.send("first");
- const second = await t.send("second");
+ const session = first.session;
+ const second = await session.send("second");
expect(first.requireToolCall("alpha", { status: "pending" }).name).toBe("alpha");
first.calledTool("alpha", { status: "pending", count: 1 });
second.notCalledTool("alpha");
@@ -435,8 +577,7 @@ describe("executeTask", () => {
client: new Client({ host: target.url }),
target,
evaluation: createTestEval(async (t) => {
- const session = t.newSession();
- await session.send("first");
+ const { session } = await t.send("first");
session.calledTool("alpha", { status: "pending", count: 1 });
session.event("turn.started", { count: 1 });
await session.send("second");
@@ -566,7 +707,8 @@ describe("executeTask", () => {
client: new Client({ host: target.url }),
target,
evaluation: createTestEval(async (t) => {
- const parent = await t.start("delegate");
+ const conversation = await t.session();
+ const parent = await conversation.start("delegate");
expect(parent.sessionId).toBe("parent-session");
const called = await parent.waitForEvent("subagent.called", {
@@ -637,7 +779,8 @@ describe("executeTask", () => {
client: new Client({ host: target.url }),
target,
evaluation: createTestEval(async (t) => {
- const live = await t.start("fail");
+ const session = await t.session();
+ const live = await session.start("fail");
await expect(live.waitForEvent("actions.requested")).rejects.toThrow(/session\.failed/);
await expect(live.result()).resolves.toMatchObject({ status: "failed" });
}, "live-turn-failure"),
@@ -836,6 +979,9 @@ function createScriptedServer(
);
}
+ if (method === "POST" && pathname === "/eve/v1/session" && init?.body === undefined) {
+ return Response.json({ sessionId: pendingTurns[0]?.sessionId }, { status: 202 });
+ }
if (method === "POST") {
const next = pendingTurns.shift();
if (next === undefined) {
diff --git a/packages/eve/src/evals/runner/execute-task.ts b/packages/eve/src/evals/runner/execute-task.ts
index c6823a6d0..561c56dd7 100644
--- a/packages/eve/src/evals/runner/execute-task.ts
+++ b/packages/eve/src/evals/runner/execute-task.ts
@@ -12,7 +12,8 @@ import type {
EveEvalTurn,
} from "#evals/types.js";
import { createEmptyDerivedFacts } from "#evals/runner/derive-run-facts.js";
-import { EvalSessionManager, type EvalSessionStartedEvent } from "#evals/session.js";
+import { EvalSessionManager } from "#evals/session-manager.js";
+import type { EvalSessionStartedEvent } from "#evals/session.js";
import { createEvalContext } from "#evals/context.js";
import { scopeEvalTargetHandle } from "#evals/target.js";
import { AssertionCollector } from "#evals/assertions/collector.js";
diff --git a/packages/eve/src/evals/session-manager.ts b/packages/eve/src/evals/session-manager.ts
new file mode 100644
index 000000000..443f0aa36
--- /dev/null
+++ b/packages/eve/src/evals/session-manager.ts
@@ -0,0 +1,100 @@
+import type { ClientSession } from "#client/session.js";
+import type { CreateSessionOptions, SendTurnInput, SendTurnOptions } from "#client/types.js";
+import type { Client } from "#client/client.js";
+import { AssertionCollector } from "#evals/assertions/collector.js";
+import { EvalSessionDriver, type EvalSessionStartedEvent } from "#evals/session.js";
+import { cleanupEvalSessions } from "#evals/session-cleanup.js";
+import type { EveEvalLiveTurn, EveEvalSessionResult } from "#evals/types.js";
+
+export class EvalSessionManager {
+ readonly #client: Client;
+ readonly #signal: AbortSignal | undefined;
+ readonly #collector: AssertionCollector;
+ readonly #onSessionStart: ((event: EvalSessionStartedEvent) => void) | undefined;
+ readonly #sessions: EvalSessionDriver[] = [];
+ #lastTurnSession: EvalSessionDriver | undefined;
+
+ constructor(input: {
+ readonly client: Client;
+ readonly collector?: AssertionCollector;
+ readonly onSessionStart?: (event: EvalSessionStartedEvent) => void;
+ readonly signal?: AbortSignal;
+ }) {
+ this.#client = input.client;
+ this.#collector = input.collector ?? new AssertionCollector();
+ this.#onSessionStart = input.onSessionStart;
+ this.#signal = input.signal;
+ }
+
+ async session(options: CreateSessionOptions = {}): Promise<EvalSessionDriver> {
+ const { session } = await this.#client.sessions.create({
+ ...options,
+ signal: options.signal ?? this.#signal,
+ });
+ return this.#register(session);
+ }
+
+ async send(message: SendTurnInput["message"], options: SendTurnOptions = {}) {
+ const { session, response } = await this.#client.sessions.create({
+ turnPolicy: "queue",
+ ...options,
+ message,
+ signal: options.signal ?? this.#signal,
+ });
+ const driver = this.#register(session);
+ return await driver.consume(response, message).result();
+ }
+
+ async attachSession(
+ sessionId: string,
+ options?: { readonly startIndex?: number },
+ ): Promise<EvalSessionDriver> {
+ const session = this.#createAttachedSession(sessionId, options);
+ await session.readTurn(options);
+ return session;
+ }
+
+ watchTurn(sessionId: string, options?: { readonly startIndex?: number }): EveEvalLiveTurn {
+ return this.#createAttachedSession(sessionId, options).watchTurn(options);
+ }
+
+ snapshots(): readonly EveEvalSessionResult[] {
+ return this.#sessions.map((session) => session.snapshot());
+ }
+
+ lastTurnSession(): EvalSessionDriver | undefined {
+ return this.#lastTurnSession;
+ }
+
+ hasActivity(): boolean {
+ return this.#sessions.length > 0;
+ }
+ /** @internal */
+ async cleanup(signal: AbortSignal): Promise<readonly PromiseSettledResult<void>[]> {
+ return await cleanupEvalSessions(this.#sessions, signal);
+ }
+
+ #register(session: ClientSession): EvalSessionDriver {
+ const driver = new EvalSessionDriver({
+ collector: this.#collector,
+ onSessionStart: this.#onSessionStart,
+ onTurn: (completed) => {
+ this.#lastTurnSession = completed;
+ },
+ primary: this.#sessions.length === 0,
+ session,
+ signal: this.#signal,
+ });
+ this.#sessions.push(driver);
+ return driver;
+ }
+
+ #createAttachedSession(
+ sessionId: string,
+ options?: { readonly startIndex?: number },
+ ): EvalSessionDriver {
+ return this.#register(
+ this.#client.sessions.attach(sessionId, { streamIndex: options?.startIndex ?? 0 }),
+ );
+ }
+}
diff --git a/packages/eve/src/evals/session.test.ts b/packages/eve/src/evals/session.test.ts
new file mode 100644
index 000000000..4cc1cae34
--- /dev/null
+++ b/packages/eve/src/evals/session.test.ts
@@ -0,0 +1,130 @@
+import { readFile } from "node:fs/promises";
+import { afterEach, describe, expect, it, vi } from "vitest";
+
+import { Client } from "#client/client.js";
+import { MessageResponse } from "#client/message-response.js";
+import { ClientSession } from "#client/session.js";
+import { AssertionCollector } from "#evals/assertions/collector.js";
+import { createEvalContext } from "#evals/context.js";
+import { EvalSessionManager } from "#evals/session-manager.js";
+import { createEvalTargetHandle } from "#evals/target.js";
+import { stampTestEvents } from "#internal/testing/events.js";
+
+const mocks = vi.hoisted(() => ({ closedQA: vi.fn() }));
+
+vi.mock("node:fs/promises", () => ({ readFile: vi.fn() }));
+vi.mock("autoevals", () => ({ ClosedQA: mocks.closedQA }));
+vi.mock("#evals/autoevals-client.js", () => ({ createAutoevalsClient: vi.fn() }));
+
+afterEach(() => {
+ vi.restoreAllMocks();
+ vi.clearAllMocks();
+});
+
+describe("eval judge input", () => {
+ it.each(["respond", "startRespond", "readTurn"] as const)(
+ "preserves the latest prompt after %s",
+ async (operation) => {
+ const { context, session } = await setup();
+ await session.send("Find Alice's order status.");
+ await session.send("Find Bob's order status instead.");
+
+ const responses = [{ requestId: "approval_1", optionId: "approve" }];
+ if (operation === "startRespond") {
+ await (await session.startRespond(responses)).result();
+ } else if (operation === "respond") {
+ await session.respond(responses);
+ } else {
+ await session.readTurn();
+ }
+
+ context.judge.autoevals.closedQA("answers the user's request");
+ expect(mocks.closedQA).toHaveBeenCalledWith(
+ expect.objectContaining({ input: "Find Bob's order status instead." }),
+ );
+ },
+ );
+
+ it("uses the text supplied with a file attachment", async () => {
+ const { context, session } = await setup();
+ vi.mocked(readFile).mockResolvedValue(Buffer.from("invoice"));
+
+ await session.sendFile("Summarize Alice's invoice.", "/invoice.txt", "text/plain");
+ context.judge.autoevals.closedQA("summarizes the invoice");
+
+ expect(mocks.closedQA).toHaveBeenCalledWith(
+ expect.objectContaining({ input: "Summarize Alice's invoice." }),
+ );
+ });
+
+ it("combines multipart text and clears the previous prompt for a file-only message", async () => {
+ const { context, session } = await setup();
+ const file = {
+ type: "file",
+ data: "data:text/plain;base64,SGk=",
+ mediaType: "text/plain",
+ } as const;
+
+ await session.send([
+ { type: "text", text: "Review Alice's invoice." },
+ file,
+ { type: "text", text: "List the line items." },
+ ]);
+ context.judge.autoevals.closedQA("lists the line items");
+ expect(mocks.closedQA).toHaveBeenLastCalledWith(
+ expect.objectContaining({ input: "Review Alice's invoice.\nList the line items." }),
+ );
+
+ await session.send([file]);
+ context.judge.autoevals.closedQA("describes the attachment");
+ expect(mocks.closedQA).toHaveBeenLastCalledWith(expect.objectContaining({ input: "" }));
+ });
+});
+
+async function setup() {
+ mocks.closedQA.mockResolvedValue({ name: "ClosedQA", score: 1 });
+ vi.spyOn(globalThis, "fetch").mockResolvedValue(
+ Response.json({ sessionId: "session_1" }, { status: 202 }),
+ );
+ vi.spyOn(ClientSession.prototype, "send").mockImplementation(async () => response());
+ vi.spyOn(ClientSession.prototype, "respond").mockImplementation(async () => response());
+ vi.spyOn(ClientSession.prototype, "stream").mockImplementation(events);
+ const client = new Client({ host: "https://eve.test" });
+ const collector = new AssertionCollector();
+ const manager = new EvalSessionManager({ client, collector });
+ const { context } = createEvalContext({
+ collector,
+ manager,
+ target: createEvalTargetHandle({
+ capabilities: { devRoutes: true },
+ client,
+ kind: "local",
+ url: "https://eve.test",
+ }),
+ signal: new AbortController().signal,
+ judge: { model: "openai/gpt-5.4-mini" },
+ log: () => {},
+ });
+ return { context, session: await manager.session() };
+}
+
+function response() {
+ return new MessageResponse({
+ sessionId: "session_1",
+ cancelTurn: async () => ({ status: "no_active_turn" }),
+ createStream: events,
+ });
+}
+
+async function* events() {
+ yield* stampTestEvents([
+ {
+ type: "message.completed",
+ data: { finishReason: "stop", message: "Done.", sequence: 1, stepIndex: 0, turnId: "turn_1" },
+ },
+ {
+ type: "session.waiting",
+ data: { continuationToken: "session_1", wait: "next-user-message" },
+ },
+ ]);
+}
diff --git a/packages/eve/src/evals/session.ts b/packages/eve/src/evals/session.ts
index 3928529e7..0ddbdcdef 100644
--- a/packages/eve/src/evals/session.ts
+++ b/packages/eve/src/evals/session.ts
@@ -2,7 +2,6 @@ import { readFile } from "node:fs/promises";
import { basename } from "node:path";
import { createTextWithFileContent } from "#client/file-parts.js";
-import type { Client } from "#client/client.js";
import type { ClientSession } from "#client/session.js";
import type {
CancelSessionResult,
@@ -21,7 +20,6 @@ import { summarizeTurnEvents } from "#client/session-utils.js";
import { extractCompletedResult } from "#client/output-schema.js";
import type { InputRequest, InputResponse } from "#shared/input.js";
import { deriveRunFacts } from "#evals/runner/derive-run-facts.js";
-import { cleanupEvalSessions } from "#evals/session-cleanup.js";
import { formatEvalTranscript, inferMediaType } from "#evals/session-content.js";
import { AssertionCollector } from "#evals/assertions/collector.js";
import { createOutputAssertions, createScopedAssertions } from "#evals/assertions/scoped.js";
@@ -73,8 +71,9 @@ export interface EvalSessionStartedEvent {
}
export class EvalSessionDriver implements EveEvalSession {
- readonly #client: Client;
- #session: ClientSession | undefined;
+ readonly #session: ClientSession;
+ readonly #onTurn: (session: EvalSessionDriver) => void;
+ #lastInput = "";
readonly #signal: AbortSignal | undefined;
readonly #collector: AssertionCollector;
readonly #events: MessageStreamEvent[] = [];
@@ -87,18 +86,18 @@ export class EvalSessionDriver implements EveEvalSession {
#pendingInputRequests: readonly InputRequest[] = [];
constructor(input: {
- readonly client: Client;
readonly collector: AssertionCollector;
readonly onSessionStart?: (event: EvalSessionStartedEvent) => void;
readonly primary: boolean;
- readonly session?: ClientSession;
+ readonly session: ClientSession;
+ readonly onTurn: (session: EvalSessionDriver) => void;
readonly signal?: AbortSignal;
}) {
- this.#client = input.client;
this.#collector = input.collector;
this.#onSessionStart = input.onSessionStart;
this.#primary = input.primary;
this.#session = input.session;
+ this.#onTurn = input.onTurn;
this.#signal = input.signal;
Object.assign(
this,
@@ -121,6 +120,10 @@ export class EvalSessionDriver implements EveEvalSession {
return formatEvalTranscript(this.#events);
}
+ get lastInput(): string {
+ return this.#lastInput;
+ }
+
get lastTurn(): EveEvalTurn | undefined {
return this.#lastTurn;
}
@@ -129,22 +132,20 @@ export class EvalSessionDriver implements EveEvalSession {
return this.#pendingInputRequests;
}
- get sessionId(): string | undefined {
- return this.#session?.state.sessionId ?? this.#lastTurn?.sessionId;
+ get sessionId(): string {
+ return this.#session.state.sessionId;
}
- get state(): ClientSessionState | undefined {
- return this.#session?.state;
+ get state(): ClientSessionState {
+ return this.#session.state;
}
async cancel(): Promise<CancelSessionResult> {
- if (this.#session === undefined) throw new Error("Eval session has not started.");
return await this.#session.cancel();
}
/** @internal */
async cleanup(signal: AbortSignal): Promise<void> {
- if (this.#session === undefined) return;
await this.#session.reset({ reason: "Eval timed out", signal });
}
@@ -218,31 +219,37 @@ export class EvalSessionDriver implements EveEvalSession {
}
async #start(input: SendTurnPayload): Promise<EveEvalLiveTurn> {
- const turnInput = attachSignal(input, this.#signal);
- let response;
- if (this.#session === undefined) {
- if (turnInput.message === undefined) {
- throw new Error("Eval session has not started.");
- }
- const created = await this.#client.sessions.create({
- ...turnInput,
- message: turnInput.message,
- });
- this.#session = created.session;
- response = created.response;
- } else {
- const { inputResponses, message, ...options } = turnInput;
- response =
- inputResponses === undefined
- ? await this.#session.send(message!, options)
- : await this.#session.respond(inputResponses, options);
- }
+ const { inputResponses, message, ...options } = attachSignal(input, this.#signal);
+ const response =
+ inputResponses === undefined
+ ? await this.#session.send(message!, options)
+ : await this.#session.respond(inputResponses, options);
+ return this.consume(response, message);
+ }
+
+ /** @internal */
+ consume(
+ events: AsyncIterable<MessageStreamEvent>,
+ message?: SendTurnInput["message"],
+ ): EveEvalLiveTurn {
+ const sessionId = this.sessionId;
return new EvalLiveTurn({
- events: response,
- observe: (event) => this.#observeEvent(response.sessionId, event),
- record: (events) => this.#recordObservedTurn(response.sessionId, events),
+ events,
+ observe: (event) => this.#observeEvent(sessionId, event),
+ record: (observed) => {
+ if (message !== undefined) {
+ this.#lastInput =
+ typeof message === "string"
+ ? message
+ : message
+ .filter((part) => part.type === "text")
+ .map((part) => part.text)
+ .join("\n");
+ }
+ return this.#recordObservedTurn(sessionId, observed);
+ },
session: this,
- sessionId: response.sessionId,
+ sessionId,
});
}
@@ -258,25 +265,13 @@ export class EvalSessionDriver implements EveEvalSession {
}
async readTurn(options?: { readonly startIndex?: number }): Promise<EveEvalTurn> {
- const sessionId = this.sessionId;
- return await this.watchTurn(options, requireSessionId(sessionId)).result();
+ return await this.watchTurn(options).result();
}
- watchTurn(
- options?: { readonly startIndex?: number },
- sessionId = requireSessionId(this.sessionId),
- ): EveEvalLiveTurn {
- if (this.#session === undefined) throw new Error("Eval session has not started.");
- return new EvalLiveTurn({
- events: this.#session.stream({
- signal: this.#signal,
- startIndex: options?.startIndex,
- }),
- observe: (event) => this.#observeEvent(sessionId, event),
- record: (events) => this.#recordObservedTurn(sessionId, events),
- session: this,
- sessionId,
- });
+ watchTurn(options?: { readonly startIndex?: number }): EveEvalLiveTurn {
+ return this.consume(
+ this.#session.stream({ signal: this.#signal, startIndex: options?.startIndex }),
+ );
}
snapshot(): EveEvalSessionResult {
@@ -286,7 +281,7 @@ export class EvalSessionDriver implements EveEvalSession {
events: [...this.#events],
primary: this.#primary,
sessionId,
- state: this.#session?.state,
+ state: this.#session.state,
traceContexts: [...this.#traceContexts],
};
}
@@ -330,11 +325,13 @@ export class EvalSessionDriver implements EveEvalSession {
events: input.events,
inputRequests: input.inputRequests,
message: input.message,
+ session: this,
sessionId: input.sessionId,
status: input.status,
toolCalls: derived.toolCalls,
});
this.#lastTurn = turn;
+ this.#onTurn(this);
return turn;
}
@@ -494,6 +491,7 @@ class EvalTurn implements EveEvalTurn {
readonly events: readonly MessageStreamEvent[];
readonly inputRequests: readonly InputRequest[];
readonly message: string | undefined;
+ readonly session: EveEvalSession;
readonly sessionId: string;
readonly status: "completed" | "failed" | "waiting";
readonly toolCalls: readonly EveEvalToolCall[];
@@ -507,6 +505,7 @@ class EvalTurn implements EveEvalTurn {
readonly events: readonly MessageStreamEvent[];
readonly inputRequests: readonly InputRequest[];
readonly message: string | undefined;
+ readonly session: EveEvalSession;
readonly sessionId: string;
readonly status: "completed" | "failed" | "waiting";
readonly toolCalls: readonly EveEvalToolCall[];
@@ -516,6 +515,7 @@ class EvalTurn implements EveEvalTurn {
this.inputRequests = input.inputRequests;
this.message = input.message;
this.sessionId = input.sessionId;
+ this.session = input.session;
this.status = input.status;
this.toolCalls = input.toolCalls;
this.#collector = input.collector;
@@ -566,99 +566,6 @@ class EvalTurn implements EveEvalTurn {
}
}
-export class EvalSessionManager {
- readonly #client: Client;
- readonly #signal: AbortSignal | undefined;
- readonly #collector: AssertionCollector;
- readonly #onSessionStart: ((event: EvalSessionStartedEvent) => void) | undefined;
- readonly #sessions: EvalSessionDriver[] = [];
- #primary: EvalSessionDriver | undefined;
-
- constructor(input: {
- readonly client: Client;
- readonly collector?: AssertionCollector;
- readonly onSessionStart?: (event: EvalSessionStartedEvent) => void;
- readonly signal?: AbortSignal;
- }) {
- this.#client = input.client;
- this.#collector = input.collector ?? new AssertionCollector();
- this.#onSessionStart = input.onSessionStart;
- this.#signal = input.signal;
- }
-
- get primary(): EvalSessionDriver {
- this.#primary ??= this.#createSession(true);
- return this.#primary;
- }
-
- newSession(): EvalSessionDriver {
- return this.#createSession(false);
- }
-
- async attachSession(
- sessionId: string,
- options?: { readonly startIndex?: number },
- ): Promise<EvalSessionDriver> {
- const session = this.#createAttachedSession(sessionId, options);
- await session.readTurn(options);
- return session;
- }
-
- watchTurn(sessionId: string, options?: { readonly startIndex?: number }): EveEvalLiveTurn {
- return this.#createAttachedSession(sessionId, options).watchTurn(options, sessionId);
- }
-
- snapshots(): readonly EveEvalSessionResult[] {
- return this.#sessions.map((session) => session.snapshot());
- }
-
- lastTurnSession(): EvalSessionDriver | undefined {
- if (this.#primary?.lastTurn !== undefined) {
- return this.#primary;
- }
-
- return this.#sessions.find((session) => session.lastTurn !== undefined);
- }
-
- hasActivity(): boolean {
- return this.#sessions.length > 0;
- }
- /** @internal */
- async cleanup(signal: AbortSignal): Promise<readonly PromiseSettledResult<void>[]> {
- return await cleanupEvalSessions(this.#sessions, signal);
- }
-
- #createSession(primary: boolean): EvalSessionDriver {
- const session = new EvalSessionDriver({
- client: this.#client,
- collector: this.#collector,
- onSessionStart: this.#onSessionStart,
- primary,
- signal: this.#signal,
- });
- this.#sessions.push(session);
- return session;
- }
-
- #createAttachedSession(
- sessionId: string,
- options?: { readonly startIndex?: number },
- ): EvalSessionDriver {
- const session = new EvalSessionDriver({
- client: this.#client,
- collector: this.#collector,
- onSessionStart: this.#onSessionStart,
- primary: false,
- session: this.#client.sessions.attach(sessionId, {
- streamIndex: options?.startIndex ?? 0,
- }),
- signal: this.#signal,
- });
- this.#sessions.push(session);
- return session;
- }
-}
-
function attachSignal(input: SendTurnPayload, signal: AbortSignal | undefined): SendTurnPayload {
if (signal === undefined) return input;
return input.signal === undefined ? { ...input, signal } : input;
@@ -682,13 +589,6 @@ function outputOf(turn: EveEvalTurn | undefined): unknown {
return turn.data === undefined ? (turn.message ?? null) : turn.data;
}
-function requireSessionId(sessionId: string | undefined): string {
- if (sessionId === undefined) {
- throw new Error("Eval session produced a turn without a session id.");
- }
- return sessionId;
-}
-
function assertRequestHasOption(request: InputRequest, optionId: string): void {
if (request.options === undefined || request.options.length === 0) {
throw new Error(`Input request "${request.requestId}" has no selectable options.`);
diff --git a/packages/eve/src/evals/target.ts b/packages/eve/src/evals/target.ts
index 920bd68b1..59b33db26 100644
--- a/packages/eve/src/evals/target.ts
+++ b/packages/eve/src/evals/target.ts
@@ -5,7 +5,7 @@ import type { AgentInfoResult } from "#client/types.js";
import { createEveDevDispatchSchedulePath } from "#protocol/routes.js";
import { toErrorMessage } from "#shared/errors.js";
import { stripNpmPackageScope } from "#shared/package-name.js";
-import { EvalSessionManager } from "#evals/session.js";
+import { EvalSessionManager } from "#evals/session-manager.js";
import type {
EveEvalScheduleDispatchResult,
EveEvalSession,
diff --git a/packages/eve/src/evals/types.ts b/packages/eve/src/evals/types.ts
index c850aea51..14658674d 100644
--- a/packages/eve/src/evals/types.ts
+++ b/packages/eve/src/evals/types.ts
@@ -9,6 +9,7 @@ import type {
import type {
CancelSessionResult,
ClientSessionState,
+ CreateSessionOptions,
SendTurnInput,
SendTurnOptions,
} from "#client/types.js";
@@ -287,7 +288,7 @@ export interface EveEvalLiveTurn {
): Promise<EveEvalStreamEvent<TType>>;
}
-/** Operations and state shared by the primary eval context and independent sessions. */
+/** Operations and state belonging to one accepted session. */
export interface EveEvalSessionDriver {
/** All events observed on this session so far. */
readonly events: readonly MessageStreamEvent[];
@@ -299,9 +300,9 @@ export interface EveEvalSessionDriver {
/** Input requests left pending by the last parked turn. */
readonly pendingInputRequests: readonly InputRequest[];
/** Serializable cursor for resuming this session. */
- readonly state: ClientSessionState | undefined;
- /** eve session id after the first successful send. */
- readonly sessionId: string | undefined;
+ readonly state: ClientSessionState;
+ /** Durable session id, available when the session is returned. */
+ readonly sessionId: string;
/** Request cooperative cancellation of this session's active turn. */
cancel(): Promise<CancelSessionResult>;
/** Require exactly one pending input request matching `filter`, or abort dependent control flow. */
@@ -323,7 +324,7 @@ export interface EveEvalSessionDriver {
sendFile(text: string, filePath: string, mediaType?: string): Promise<EveEvalTurn>;
}
-/** Driver for one independent session, exposed by `t.newSession()` and target attachment helpers. */
+/** One accepted session, exposed by `t.session()`, turns, and target attachment helpers. */
export interface EveEvalSession
extends EveEvalSessionDriver, EveEvalAssertions, EveEvalOutputAssertions {}
@@ -335,6 +336,8 @@ export interface EveEvalTurn extends EveEvalAssertions, EveEvalOutputAssertions
readonly events: readonly MessageStreamEvent[];
readonly inputRequests: readonly InputRequest[];
readonly message: string | undefined;
+ /** Session that owns this turn; use it for follow-up messages. */
+ readonly session: EveEvalSession;
readonly sessionId: string;
readonly status: "completed" | "failed" | "waiting";
readonly toolCalls: readonly EveEvalToolCall[];
@@ -365,7 +368,7 @@ export interface EveEvalJudgeConfig {
* Per-call options for `t.judge.autoevals.*` assertions.
*/
export interface JudgeOpts {
- /** Value to grade. Defaults to the final assistant message (`t.reply`). */
+ /** Value to grade. Defaults to the most recently settled turn’s assistant message. */
readonly on?: unknown;
/** Judge model for this call only; overrides the eval/config judge model. */
readonly model?: LanguageModel;
@@ -394,26 +397,28 @@ export interface JudgeContext {
}
/**
- * The single context passed to an eval's `test(t)` function. It drives the
- * primary session, carries the run-level
- * and value-level assertion vocabulary, and exposes `judge` for LLM-as-judge.
+ * The context passed to `test(t)`. Creates independent sessions and carries
+ * run-level assertions, value-level assertions, and model-backed judges.
*
* Scoped assertions (`succeeded`, `calledTool`, …) record an entry evaluated
* after the test body; `check`, `require`, and `judge` evaluate explicit values.
*/
-export interface EveEvalContext extends EveEvalSessionDriver, EveEvalAssertions {
+export interface EveEvalContext extends EveEvalAssertions {
/** Eval timeout signal. */
readonly signal: AbortSignal;
/** Current target under test. */
readonly target: EveEvalTargetHandle;
- /** The primary session's last assistant message, or null. */
- readonly reply: string | null;
/** Structured eval log hook. */
log(message: string): void;
/** Pause the eval task, defaulting to 1 second, while respecting the eval timeout signal. */
sleep(ms?: number): Promise<void>;
- /** Create an additional independent session against the same target. */
- newSession(): EveEvalSession;
+ /**
+ * Create a new session without starting a turn or reading events. Resolves on
+ * acceptance with a definite session id and cursor. Every call creates a session.
+ */
+ session(options?: CreateSessionOptions): Promise<EveEvalSession>;
+ /** Create a new session with its first message and wait for the turn to settle. */
+ send(message: SendTurnInput["message"], options?: SendTurnOptions): Promise<EveEvalTurn>;
/** Apply a value-level assertion (from `eve/evals/expect`) to a value. */
check(value: unknown, assertion: Assertion): AssertionHandle;
diff --git a/packages/eve/src/eve-channel/create-request.ts b/packages/eve/src/eve-channel/create-request.ts
new file mode 100644
index 000000000..b9ad052a1
--- /dev/null
+++ b/packages/eve/src/eve-channel/create-request.ts
@@ -0,0 +1,61 @@
+import type { UserContent } from "ai";
+
+import type {
+ ActivityObserverConfig,
+ SessionCallback,
+ SessionCapabilities,
+} from "#channel/types.js";
+import type { JsonObject } from "#shared/json.js";
+import type { RunMode } from "#shared/run-mode.js";
+
+export interface ParsedCreateBody {
+ activityObserver?: ActivityObserverConfig;
+ callback?: SessionCallback;
+ capabilities?: SessionCapabilities;
+ message?: string | UserContent;
+ mode?: RunMode;
+ context?: readonly string[];
+ operationId?: string;
+ outputSchema?: JsonObject;
+}
+
+/** Enforces the fields that only make sense when creation also starts a turn. */
+export function validateMessageFreeCreate(input: {
+ readonly activityObserver: ActivityObserverConfig | undefined;
+ readonly callback: SessionCallback | undefined;
+ readonly hasClientContext: boolean;
+ readonly hasMessageField: boolean;
+ readonly message: string | UserContent | undefined;
+ readonly mode: RunMode | undefined;
+ readonly outputSchema: JsonObject | undefined;
+}): Response | undefined {
+ if (input.hasMessageField && input.message === undefined) {
+ return Response.json(
+ { error: "Expected 'message' to be non-empty when provided.", ok: false },
+ { status: 400 },
+ );
+ }
+ if (input.message !== undefined) return undefined;
+ if (input.mode === "task") {
+ return Response.json(
+ { error: "Task sessions require a non-empty 'message'.", ok: false },
+ { status: 400 },
+ );
+ }
+ if (
+ input.hasClientContext ||
+ input.callback !== undefined ||
+ input.activityObserver !== undefined ||
+ input.outputSchema !== undefined
+ ) {
+ return Response.json(
+ {
+ error:
+ "Creating a session without a message does not accept 'clientContext', 'callback', 'activityObserver', or 'outputSchema'.",
+ ok: false,
+ },
+ { status: 400 },
+ );
+ }
+ return undefined;
+}
diff --git a/packages/eve/src/eve-channel/index.ts b/packages/eve/src/eve-channel/index.ts
index c4e5c488c..d3cac1aef 100644
--- a/packages/eve/src/eve-channel/index.ts
+++ b/packages/eve/src/eve-channel/index.ts
@@ -69,6 +69,7 @@ import {
parseCreateBody,
parseIncludeTailIndex,
parseJsonRequest,
+ parseOptionalJsonRequest,
parseResetBody,
parseSessionControlBody,
parseSessionMessageBody,
@@ -150,7 +151,7 @@ export function eveChannel(input: EveChannelInput): EveChannel {
const authResult = await routeAuth(req, input.auth);
if (authResult instanceof Response) return authResult;
- const payload = await parseJsonRequest(req);
+ const payload = await parseOptionalJsonRequest(req);
if (payload instanceof Response) return payload;
const tokenRejection = rejectSessionContinuationToken(payload);
if (tokenRejection !== null) return tokenRejection;
@@ -286,12 +287,15 @@ export function eveChannel(input: EveChannelInput): EveChannel {
}
}
- const messageResult = await resolveOnMessage({
- auth: forwarded.auth,
- config: input,
- message: body.message,
- request: req,
- });
+ const messageResult =
+ body.message === undefined
+ ? { auth: forwarded.auth }
+ : await resolveOnMessage({
+ auth: forwarded.auth,
+ config: input,
+ message: body.message,
+ request: req,
+ });
if (messageResult instanceof Response) return messageResult;
const createSession = readRouteSessionCreator(args);
if (createSession === undefined) {
@@ -370,6 +374,7 @@ export function eveChannel(input: EveChannelInput): EveChannel {
if (policyRejection !== null) return policyRejection;
let context: readonly string[] | undefined;
+ let title: string | undefined;
let dispatchAuth: SessionAuthContext | null = forwarded.auth;
if (body.message !== undefined) {
const messageResult = await resolveOnMessage({
@@ -381,6 +386,7 @@ export function eveChannel(input: EveChannelInput): EveChannel {
});
if (messageResult instanceof Response) return messageResult;
context = messageResult.context;
+ title = messageResult.title;
dispatchAuth = messageResult.auth;
}
@@ -395,6 +401,7 @@ export function eveChannel(input: EveChannelInput): EveChannel {
context,
outputSchema: body.outputSchema,
turnPolicy: body.turnPolicy,
+ title,
},
body.context,
);
@@ -409,11 +416,14 @@ export function eveChannel(input: EveChannelInput): EveChannel {
{ status: 500 },
);
}
- if (result.status === "session_not_active") {
+ if (result.status !== "accepted") {
return Response.json(
{
- code: "session_not_active",
- error: "The session is no longer active.",
+ code: result.retryable ? "session_not_ready" : "session_not_active",
+ error:
+ result.retryable === true
+ ? "The session is not ready to accept messages yet."
+ : "The session is no longer active.",
ok: false,
},
{ headers: { "cache-control": "no-store" }, status: 409 },
diff --git a/packages/eve/src/eve-channel/request.test.ts b/packages/eve/src/eve-channel/request.test.ts
new file mode 100644
index 000000000..c821580e7
--- /dev/null
+++ b/packages/eve/src/eve-channel/request.test.ts
@@ -0,0 +1,40 @@
+import { describe, expect, it } from "vitest";
+
+import { parseCreateBody } from "#eve-channel/request.js";
+
+describe("parseCreateBody", () => {
+ it("accepts a conversation session without a message", () => {
+ expect(parseCreateBody({})).toEqual({
+ activityObserver: undefined,
+ callback: undefined,
+ capabilities: undefined,
+ context: undefined,
+ mode: undefined,
+ outputSchema: undefined,
+ });
+ });
+
+ it("rejects an explicitly empty message", async () => {
+ const response = parseCreateBody({ message: "" });
+ expect(response).toBeInstanceOf(Response);
+ await expect((response as Response).json()).resolves.toMatchObject({
+ error: "Expected 'message' to be non-empty when provided.",
+ });
+ });
+
+ it("rejects turn-only fields without a message", async () => {
+ const response = parseCreateBody({ clientContext: "page context" });
+ expect(response).toBeInstanceOf(Response);
+ await expect((response as Response).json()).resolves.toMatchObject({
+ error: expect.stringContaining("does not accept"),
+ });
+ });
+
+ it("requires a message for task mode", async () => {
+ const response = parseCreateBody({ mode: "task" });
+ expect(response).toBeInstanceOf(Response);
+ await expect((response as Response).json()).resolves.toMatchObject({
+ error: "Task sessions require a non-empty 'message'.",
+ });
+ });
+});
diff --git a/packages/eve/src/eve-channel/request.ts b/packages/eve/src/eve-channel/request.ts
index 2a9c4c885..752fa3f4d 100644
--- a/packages/eve/src/eve-channel/request.ts
+++ b/packages/eve/src/eve-channel/request.ts
@@ -34,21 +34,11 @@ import {
import { isInputResponse, type ValidatedInputResponse } from "#shared/input.js";
import { parseJsonObject, type JsonObject } from "#shared/json.js";
import type { RunMode } from "#shared/run-mode.js";
+import { type ParsedCreateBody, validateMessageFreeCreate } from "#eve-channel/create-request.js";
const SESSION_STREAM_HEARTBEAT_MS = 10_000;
const SESSION_STREAM_LEASE_MS = 60_000;
-interface ParsedCreateBody {
- activityObserver?: ActivityObserverConfig;
- callback?: SessionCallback;
- capabilities?: SessionCapabilities;
- message: string | UserContent;
- mode?: RunMode;
- context?: readonly string[];
- operationId?: string;
- outputSchema?: JsonObject;
-}
-
/** Replay-stable identity for one authenticated create operation. */
export async function deriveOperationContinuationToken(input: {
readonly auth: SessionAuthContext;
@@ -101,12 +91,16 @@ export function parseCreateBody(payload: Record<string, unknown>): ParsedCreateB
const outputSchema = parseOutputSchemaField(payload.outputSchema);
if (outputSchema instanceof Response) return outputSchema;
- if (message === undefined) {
- return Response.json(
- { error: "Missing or empty 'message' field.", ok: false },
- { status: 400 },
- );
- }
+ const messageFreeRejection = validateMessageFreeCreate({
+ activityObserver,
+ callback,
+ hasClientContext: payload.clientContext !== undefined,
+ hasMessageField: "message" in payload,
+ message,
+ mode,
+ outputSchema,
+ });
+ if (messageFreeRejection !== undefined) return messageFreeRejection;
const rawOperationId = payload.operationId;
if (rawOperationId !== undefined && (typeof rawOperationId !== "string" || !rawOperationId)) {
@@ -120,11 +114,11 @@ export function parseCreateBody(payload: Record<string, unknown>): ParsedCreateB
activityObserver,
callback,
capabilities,
- message,
mode,
context,
outputSchema,
};
+ if (message !== undefined) result.message = message;
if (typeof rawOperationId === "string") result.operationId = rawOperationId;
return result;
}
@@ -270,7 +264,9 @@ export async function parseSessionControlBody(
return rejectSessionContinuationToken(payload) ?? payload;
}
-async function parseOptionalJsonRequest(req: Request): Promise<Record<string, unknown> | Response> {
+export async function parseOptionalJsonRequest(
+ req: Request,
+): Promise<Record<string, unknown> | Response> {
let text: string;
try {
text = await req.text();
diff --git a/packages/eve/src/eve-channel/types.ts b/packages/eve/src/eve-channel/types.ts
index ef4793552..8e2860a94 100644
--- a/packages/eve/src/eve-channel/types.ts
+++ b/packages/eve/src/eve-channel/types.ts
@@ -70,7 +70,7 @@ export interface EveMessageContext {
export type EveMessageResult = {
readonly auth: SessionAuthContext | null;
readonly context?: readonly string[];
- /** Overrides the workflow run title without changing the message sent to the model. */
+ /** Sets the title when creating a workflow or sending its first message after prewarming. */
readonly title?: string;
};
@@ -146,7 +146,9 @@ export interface EveChannelInput {
readonly turnPolicy?: TurnPolicy;
/**
* Pre-dispatch hook for inbound eve HTTP messages. Runs after route auth and body
- * parsing, before runtime dispatch.
+ * parsing, before runtime dispatch. Message-free creation skips this hook and
+ * parks before session initialization. The first message supplies auth and context
+ * for initialization and its first turn.
*/
readonly onMessage?: (
ctx: EveMessageContext,
diff --git a/packages/eve/src/execution/legacy-session/turn-workflow.ts b/packages/eve/src/execution/legacy-session/turn-workflow.ts
index 7fa622cbc..8f4a6ca6f 100644
--- a/packages/eve/src/execution/legacy-session/turn-workflow.ts
+++ b/packages/eve/src/execution/legacy-session/turn-workflow.ts
@@ -63,6 +63,7 @@ export async function turnWorkflow(rawInput: unknown): Promise<void> {
prepared.input.delivery === undefined
? undefined
: { ...prepared.input.delivery, caller: undefined },
+ awaitFirstMessage: false,
mode,
retention: prepared.input.retention,
serializedContext: interrupted.serializedContext,
diff --git a/packages/eve/src/execution/runtime-context.test.ts b/packages/eve/src/execution/runtime-context.test.ts
index 1d8b6fd0c..039698c3f 100644
--- a/packages/eve/src/execution/runtime-context.test.ts
+++ b/packages/eve/src/execution/runtime-context.test.ts
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
import { ContextContainer, contextStorage, loadContext } from "#context/container.js";
import {
AuthKey,
+ InitiatorAuthKey,
ChannelInstrumentationKey,
ContinuationHookTokensKey,
ContinuationTokenKey,
@@ -154,6 +155,25 @@ function createMinimalBundle(): Parameters<typeof buildRunContext>[0]["bundle"]
}
describe("buildRunContext", () => {
+ it.each([undefined, testAuth])(
+ "defers prewarm initiator identity unless explicitly forwarded (%s)",
+ (initiatorAuth) => {
+ const ctx = buildRunContext({
+ bundle: createMinimalBundle(),
+ run: {
+ auth: testAuth,
+ initiatorAuth,
+ adapter: { kind: "http" },
+ input: {},
+ mode: "conversation",
+ },
+ });
+ expect(ctx.get(AuthKey)).toEqual(testAuth);
+ expect(ctx.has(InitiatorAuthKey)).toBe(initiatorAuth !== undefined);
+ expect(ctx.get(InitiatorAuthKey)).toEqual(initiatorAuth);
+ },
+ );
+
it("seeds auth from the run input", () => {
const ctx = buildRunContext({
bundle: createMinimalBundle(),
diff --git a/packages/eve/src/execution/runtime-context.ts b/packages/eve/src/execution/runtime-context.ts
index cec27d17f..d120743bc 100644
--- a/packages/eve/src/execution/runtime-context.ts
+++ b/packages/eve/src/execution/runtime-context.ts
@@ -68,7 +68,9 @@ export function buildRunContext(input: {
}
ctx.set(ModeKey, run.mode);
ctx.set(AuthKey, auth);
- ctx.set(InitiatorAuthKey, run.initiatorAuth ?? auth);
+ if (run.initiatorAuth !== undefined || run.input.message !== undefined) {
+ ctx.set(InitiatorAuthKey, run.initiatorAuth ?? auth);
+ }
if (input.dynamicSubagentAgentConfig !== undefined) {
ctx.set(DynamicSubagentAgentConfigKey, input.dynamicSubagentAgentConfig);
diff --git a/packages/eve/src/execution/session-inbox/protocol.ts b/packages/eve/src/execution/session-inbox/protocol.ts
index dcb46541b..127dfdfd5 100644
--- a/packages/eve/src/execution/session-inbox/protocol.ts
+++ b/packages/eve/src/execution/session-inbox/protocol.ts
@@ -31,6 +31,7 @@ export function decodeSessionInboxPayload(value: unknown): DecodedSessionInbox {
const command = value as Extract<SessionCommand, { readonly kind: "send" }>;
return {
auth: command.auth,
+ title: command.title,
caller: command.caller,
deliveryMetadata:
command.delivery === undefined ? undefined : [{ ...command.delivery, payloadIndex: 0 }],
diff --git a/packages/eve/src/execution/session/entry.integration.test.ts b/packages/eve/src/execution/session/entry.integration.test.ts
index 816e5a4aa..df51bda28 100644
--- a/packages/eve/src/execution/session/entry.integration.test.ts
+++ b/packages/eve/src/execution/session/entry.integration.test.ts
@@ -219,6 +219,102 @@ function expectSingleTurn(events: readonly MessageStreamEvent[], turnId: string)
}
describe("workflowEntry integration", () => {
+ it("parks before initialization and initializes with the first message identity and title", async () => {
+ let initializedSessions = 0;
+ let initializedAuth: unknown;
+ let initializedInitiator: unknown;
+ const runtime = await createTestRuntime({
+ agent: { name: "workflow-entry-prewarm" },
+ modules: [
+ {
+ logicalPath: "hooks/initialize-session.ts",
+ loadNamespace: async () => ({
+ default: defineHook({
+ events: {
+ async "session.started"(_event, ctx) {
+ await ctx.getSandbox();
+ initializedSessions += 1;
+ initializedAuth = ctx.session.auth.current;
+ initializedInitiator = ctx.session.auth.initiator;
+ },
+ },
+ }),
+ }),
+ },
+ ],
+ });
+
+ await runtime.run(async () => {
+ const run = await start(workflowEntry, [
+ {
+ kind: "initial",
+ ownerDeploymentId: "dpl_inline",
+ input: {},
+ serializedContext: buildSerializedContext({
+ auth: { authenticator: "test", principalId: "mount", principalType: "user" },
+ channelKind: "http",
+ mode: "conversation",
+ }),
+ },
+ ]);
+ const stream = captureTurnEvents(run);
+
+ try {
+ await waitForHook(
+ { runId: run.runId },
+ { token: sessionInboxHookToken(sessionCommandHookToken(run.runId)) },
+ );
+ await expectHookClaims(run.runId, [sessionCommandHookToken(run.runId)], {
+ turnStarted: false,
+ });
+
+ const sessionRuntime = createWorkflowRuntime({
+ compiledArtifactsSource: createBundledRuntimeCompiledArtifactsSource(),
+ });
+ expect(await sessionRuntime.getStreamTailIndex(run.runId)).toBe(-1);
+ expect(initializedSessions).toBe(0);
+ for (const kind of ["clear", "compact", "cancel"] as const) {
+ await resumeHook(sessionInboxHookToken(sessionCommandHookToken(run.runId)), { kind });
+ }
+ const firstAuth = { authenticator: "test", principalId: "alice", principalType: "user" };
+ await resumeHook(sessionInboxHookToken(sessionCommandHookToken(run.runId)), {
+ auth: firstAuth,
+ title: "Alice’s first chat",
+ kind: "send",
+ payload: { message: "Say hello to Alice." },
+ turnPolicy: "steer",
+ });
+
+ const firstTurn = await stream.nextTurn();
+ expect(filterEventsByType(firstTurn, "session.started")).toHaveLength(1);
+ expect(initializedSessions).toBe(1);
+ expect(initializedAuth).toEqual(firstAuth);
+ expect(initializedInitiator).toEqual(firstAuth);
+ expect((await (await getWorld()).runs.get(run.runId)).attributes?.["$eve.title"]).toBe(
+ "Alice’s first chat",
+ );
+ expectSingleTurn(firstTurn, "turn_0");
+ expect(firstTurn.at(-1)?.type).toBe("session.waiting");
+ await resumeHook(sessionInboxHookToken(sessionCommandHookToken(run.runId)), {
+ auth: { ...firstAuth, principalId: "bob" },
+ title: "Do not rename",
+ kind: "send",
+ payload: { message: "Bob joins the conversation. Greet him briefly." },
+ });
+ const secondTurn = await stream.nextTurn();
+ expect(filterEventsByType(secondTurn, "session.started")).toHaveLength(0);
+ expectSingleTurn(secondTurn, "turn_1");
+ expect(initializedSessions).toBe(1);
+ expect((await (await getWorld()).runs.get(run.runId)).attributes?.["$eve.title"]).toBe(
+ "Alice’s first chat",
+ );
+ } finally {
+ stream.dispose();
+ await run.cancel();
+ }
+ });
+ });
+
it("persists model output before settlement when a stream append exceeds the SDK flush window", async () => {
const runtime = await createTestRuntime({ agent: { name: "workflow-stream-order" } });
const world = await getWorld();
@@ -1298,12 +1394,16 @@ describe("workflowEntry integration", () => {
kind: "session-timeout",
ownerRunId: anchor.runId,
});
- const anchorHooks = await world.hooks.list({ runId: anchor.runId });
- expect(
- anchorHooks.data
- .map((hook) => hook.token)
- .filter((token) => !token.startsWith("abrt_")),
- ).toEqual([`${anchor.runId}:anchor`]);
+ // The successor can finish its turn before the old owner resumes
+ // from activation and disposes its temporary handoff hook.
+ await vi.waitFor(async () => {
+ const anchorHooks = await world.hooks.list({ runId: anchor.runId });
+ expect(
+ anchorHooks.data
+ .map((hook) => hook.token)
+ .filter((token) => !token.startsWith("abrt_")),
+ ).toEqual([`${anchor.runId}:anchor`]);
+ });
// A third delivery through the stable session id reaches the successor
// and still streams on the original run.
@@ -2228,7 +2328,11 @@ async function withTimeout<T>(promise: Promise<T>, label: string): Promise<T> {
}
}
-async function expectHookClaims(runId: string, tokens: string[]): Promise<void> {
+async function expectHookClaims(
+ runId: string,
+ tokens: string[],
+ options: { readonly turnStarted?: boolean } = {},
+): Promise<void> {
const events = await (
await getWorld()
).events.list({
@@ -2240,7 +2344,7 @@ async function expectHookClaims(runId: string, tokens: string[]): Promise<void>
event.eventType === "hook_created" ? [event.eventData.token] : [],
);
const signals = claims.filter((token) => token.startsWith("abrt_"));
- expect(signals).toHaveLength(2);
+ expect(signals).toHaveLength(options.turnStarted === false ? 0 : 2);
expect(claims.filter((token) => !token.startsWith("abrt_")).sort()).toEqual(
tokens.map(sessionInboxHookToken).sort(),
);
diff --git a/packages/eve/src/execution/session/entry.ts b/packages/eve/src/execution/session/entry.ts
index 26953a681..02c66406b 100644
--- a/packages/eve/src/execution/session/entry.ts
+++ b/packages/eve/src/execution/session/entry.ts
@@ -90,6 +90,9 @@ async function bootInitialOwner(
nodeId?: string;
};
try {
+ if (input.input.message === undefined && mode !== "conversation") {
+ throw new Error("A message-free session must use conversation mode.");
+ }
const [sessionCreation, stableClaim, aliasClaim] = await Promise.allSettled([
createSessionStep({
compiledArtifactsSource: serializedBundle.source,
@@ -134,6 +137,7 @@ async function bootInitialOwner(
capabilities: serializedContext["eve.capabilities"] as SessionCapabilities | undefined,
deploymentId: input.ownerDeploymentId,
initialInput: createInitialDelivery(input, serializedContext),
+ awaitFirstMessage: input.input.message === undefined,
mode,
retention: input.retention,
serializedContext,
@@ -192,6 +196,7 @@ async function bootHandoffOwner(
capabilities: checkpoint.capabilities,
deploymentId: input.ownerDeploymentId,
initialInput: input.delivery,
+ awaitFirstMessage: false,
mode: checkpoint.mode,
retention: checkpoint.retention,
serializedContext,
@@ -215,7 +220,8 @@ function hasDelegatedCallerContext(serializedContext: Record<string, unknown>):
function createInitialDelivery(
input: InitialWorkflowEntryInput,
serializedContext: Record<string, unknown>,
-): DeliverHookPayload {
+): DeliverHookPayload | undefined {
+ if (input.input.message === undefined) return undefined;
return {
deliveryMetadata:
serializedContext["eve.channelDelivery"] === undefined
diff --git a/packages/eve/src/execution/session/program.ts b/packages/eve/src/execution/session/program.ts
index 9a7c46252..e46c18710 100644
--- a/packages/eve/src/execution/session/program.ts
+++ b/packages/eve/src/execution/session/program.ts
@@ -48,6 +48,8 @@ export interface SessionBoot {
readonly capabilities?: SessionCapabilities;
readonly deploymentId: string;
readonly initialInput: DeliverHookPayload | undefined;
+ /** Parks on the inbox before any session-scoped lifecycle work. */
+ readonly awaitFirstMessage: boolean;
readonly mode: RunMode;
readonly retention?: AgentWorkflowRetentionDefinition;
readonly serializedContext: Record<string, unknown>;
@@ -62,6 +64,10 @@ type SessionLoopOutcome =
| { readonly kind: "terminal"; readonly outcome: SessionTerminalOutcome }
| { readonly kind: "transferred" };
+type SessionActionResult =
+ | { readonly action: TurnOutcome; readonly kind: "action" }
+ | SessionLoopOutcome;
+
/** Mutable facts the crash path needs that do not live in the state cursor. */
interface SessionProgress {
/** Delegated caller whose awaited reply is still unsettled. */
@@ -236,6 +242,17 @@ async function runSessionLoop(
progress.turnId = `turn_${String(turnIndex++)}`;
return await execution.runTurn(payload);
};
+ const runDeliveredTurn = async (
+ next: Extract<NextTurnInstruction, { kind: "turn" }>,
+ ): Promise<SessionActionResult> => {
+ const transfer = await handoff.tryTransfer(next, {
+ serializedContext: cursor.serializedContext,
+ sessionState: cursor.sessionState,
+ });
+ if (transfer.kind === "transferred") return transfer;
+ if (next.delivery.caller !== undefined) progress.caller = next.delivery.caller;
+ return { action: await runTurn({ delivery: next.delivery }), kind: "action" };
+ };
const settleCancelledTurn = async () => {
const settled = await settleCancelledTurnStep({
sessionWritable: boot.sessionWritable,
@@ -246,15 +263,43 @@ async function runSessionLoop(
progress.caller = undefined;
return settled;
};
+ const awaitPrewarmedAction = async (): Promise<SessionActionResult> => {
+ while (true) {
+ const next = await nextParkedActivity(new Set());
+ switch (next.kind) {
+ case "expired":
+ case "reset":
+ case "closed":
+ return { kind: "terminal", outcome: { kind: "expired" } };
+ case "clear":
+ case "compact":
+ continue;
+ case "turn":
+ return await runDeliveredTurn(next);
+ case "cancel-turn":
+ case "authorization-resume":
+ continue;
+ }
+ }
+ };
+ const runInitialAction = async (): Promise<SessionActionResult> => {
+ if (boot.awaitFirstMessage) return await awaitPrewarmedAction();
+ const action = await runTurn(
+ boot.initialInput === undefined ? undefined : { delivery: boot.initialInput },
+ );
+ return { action, kind: "action" };
+ };
try {
const [actionResult, timerResult] = await Promise.allSettled([
- runTurn(boot.initialInput === undefined ? undefined : { delivery: boot.initialInput }),
+ runInitialAction(),
sessionTimeout?.start(),
]);
if (timerResult.status === "rejected") throw timerResult.reason;
if (actionResult.status === "rejected") throw actionResult.reason;
- let action: TurnOutcome = actionResult.value;
+ const initial = actionResult.value;
+ if (initial.kind !== "action") return initial;
+ let action = initial.action;
while (true) {
if (action.kind === "done") {
@@ -311,13 +356,9 @@ async function runSessionLoop(
action = { ...action, settled: undefined };
continue;
case "turn": {
- const transfer = await handoff.tryTransfer(next, {
- serializedContext: cursor.serializedContext,
- sessionState: cursor.sessionState,
- });
- if (transfer.kind === "transferred") return { kind: "transferred" };
- if (next.delivery.caller !== undefined) progress.caller = next.delivery.caller;
- action = await runTurn({ delivery: next.delivery });
+ const result = await runDeliveredTurn(next);
+ if (result.kind !== "action") return result;
+ action = result.action;
continue;
}
}
diff --git a/packages/eve/src/execution/session/turn-step.test.ts b/packages/eve/src/execution/session/turn-step.test.ts
index 44fd4f4a4..d1ca85f83 100644
--- a/packages/eve/src/execution/session/turn-step.test.ts
+++ b/packages/eve/src/execution/session/turn-step.test.ts
@@ -2285,6 +2285,38 @@ describe("turnStep", () => {
});
});
+ it("keeps a settled turn when cancellation arrives after its waiting boundary", async () => {
+ const controller = new AbortController();
+ const session = createStubSession();
+ installSessionStoreMocks([session]);
+ vi.mocked(createExecutionNodeStep).mockImplementation(() => {
+ return async (stepSession): Promise<StepResult> => {
+ controller.abort(new TurnCancelledError());
+ return {
+ next: null,
+ session: stepSession,
+ settledTurn: { output: "settled answer" },
+ };
+ };
+ });
+
+ const result = await turnStep({
+ abortSignal: controller.signal,
+ input: {
+ kind: "deliver",
+ payloads: [{ message: "hello" }],
+ },
+ sessionWritable: createTestWritable(),
+ serializedContext: createSerializedContext(),
+ sessionState: createStubSessionState(),
+ });
+
+ expect(result).toMatchObject({
+ action: "park",
+ settled: { output: "settled answer" },
+ });
+ });
+
it("reports each settled turn's usage as a delta, not the cumulative session totals", async () => {
const usageStateAfterTurn = (
totals: Readonly<Record<string, number>>,
diff --git a/packages/eve/src/execution/session/turn-step.ts b/packages/eve/src/execution/session/turn-step.ts
index 1b5b0da6c..c32fe0cc6 100644
--- a/packages/eve/src/execution/session/turn-step.ts
+++ b/packages/eve/src/execution/session/turn-step.ts
@@ -1,3 +1,5 @@
+import { deriveSessionTitle } from "#execution/eve-workflow-attributes.js";
+import { setEveAttributes } from "#runtime/attributes/emit.js";
import { defaultDeliverResult } from "#channel/adapter.js";
import { contextStorage } from "#context/container.js";
import {
@@ -12,6 +14,9 @@ import {
} from "#context/dynamic-tool-lifecycle.js";
import {
AuthKey,
+ InitiatorAuthKey,
+ SessionTitleKey,
+ ParentSessionKey,
CapabilitiesKey,
ChannelDeliveryKey,
HandleEventKey,
@@ -159,6 +164,7 @@ async function runSessionStep(input: TurnStepInput): Promise<DurableStepResult>
// input has no auth; it was seeded by buildRunContext).
if (delivery?.auth !== undefined) {
ctx.set(AuthKey, delivery.auth ?? null);
+ if (!ctx.has(InitiatorAuthKey)) ctx.set(InitiatorAuthKey, delivery.auth ?? null);
}
const backgroundTaskDelivery = getBackgroundTaskDelivery(delivery);
const initialSession = hydrateDurableSession({
@@ -176,6 +182,18 @@ async function runSessionStep(input: TurnStepInput): Promise<DurableStepResult>
sessionId: initialSession.sessionId,
});
const initialEmissionState = getHarnessEmissionState(initialSession.state);
+ if (
+ !initialEmissionState.sessionStarted &&
+ !ctx.has(SessionTitleKey) &&
+ !ctx.has(ParentSessionKey)
+ ) {
+ const message = rawDelivery?.payloads.find((payload) => payload.message !== undefined)?.message;
+ const title = deriveSessionTitle(rawDelivery?.title ?? message);
+ if (title !== undefined) {
+ ctx.set(SessionTitleKey, title);
+ await setEveAttributes({ "$eve.title": title });
+ }
+ }
if (rawDelivery !== undefined) {
await contextStorage.run(ctx, () =>
@@ -520,7 +538,9 @@ async function runSessionStep(input: TurnStepInput): Promise<DurableStepResult>
return runHarnessStep(schemaSession, stepInput);
});
- throwIfTurnAborted(input.abortSignal);
+ // The waiting boundary may reach the client before this step returns.
+ // Its settled result wins over a cancellation of that completed turn.
+ if (result.settledTurn === undefined) throwIfTurnAborted(input.abortSignal);
completedModelCall = { result, serializedContext: serializeContext(ctx) };
return result;
},
diff --git a/packages/eve/src/execution/session/turn.test.ts b/packages/eve/src/execution/session/turn.test.ts
index 133f0cd1d..98d41895b 100644
--- a/packages/eve/src/execution/session/turn.test.ts
+++ b/packages/eve/src/execution/session/turn.test.ts
@@ -524,6 +524,71 @@ describe("SessionExecution background task checkpoints", () => {
expect(queue.pendingCount).toBe(1);
});
+ it.each([false, true])(
+ "keeps a settled turn when a late cancellation races its checkpoint (background tasks: %s)",
+ async (backgroundTasks) => {
+ const followUp: DeliverHookPayload = {
+ kind: "deliver",
+ payloads: [{ message: "Follow up after completion." }],
+ };
+ const cancel = { kind: "cancel", turnId: "turn_0" } as const;
+ let interrupt: (payload: SessionInboxPayload) => void = () => {};
+ const queue = new SessionInputQueue();
+ const inbox: SessionInbox = {
+ claimedTokens: [],
+ claimSessionHook: vi.fn(),
+ claimSessionHooks: vi.fn(),
+ drain: vi.fn().mockReturnValueOnce([cancel, followUp]).mockReturnValue([]),
+ hasPending: () => false,
+ next: vi.fn(),
+ restore: vi.fn(),
+ onDelivery: () => () => {},
+ onInterrupt: (handler) => {
+ interrupt = handler;
+ return () => {};
+ },
+ };
+ const completedState = state("http:completed");
+ const execution = createExecution({ inbox, queue, sessionState: state("") });
+ vi.mocked(cancelDescendantTurnsStep).mockClear();
+ vi.mocked(acknowledgeDelegatedTasksStep).mockReset();
+ vi.mocked(turnStep)
+ .mockReset()
+ .mockImplementationOnce(async (input) => {
+ interrupt(cancel);
+ expect(input.abortSignal?.aborted).toBe(true);
+ return {
+ action: "park",
+ hasPendingAuthorization: false,
+ hasPendingInputBatch: false,
+ serializedContext: input.serializedContext,
+ sessionState: completedState,
+ settled: { output: "Done." },
+ ...(backgroundTasks
+ ? {
+ backgroundTaskState: state("http:background"),
+ backgroundTasks: [
+ {
+ callId: "call-1",
+ taskId: "task-1",
+ taskInboxToken: "inbox-1",
+ taskRunId: "run-1",
+ },
+ ],
+ }
+ : {}),
+ };
+ });
+ await expect(execution.runTurn(undefined)).resolves.toMatchObject({
+ kind: "park",
+ settled: { output: "Done." },
+ });
+ expect(execution.cursor.sessionState).toBe(completedState);
+ expect(cancelDescendantTurnsStep).not.toHaveBeenCalled();
+ expect(queue.pendingCount).toBe(1);
+ },
+ );
+
it("routes a task-owned answer to a descendant while waiting for runtime results", async () => {
const sessionState = { ...state(""), hasProxyInputRequests: true };
const taskAnswer: DeliverHookPayload = {
diff --git a/packages/eve/src/execution/session/turn.ts b/packages/eve/src/execution/session/turn.ts
index e7967bc1c..c8ebaa394 100644
--- a/packages/eve/src/execution/session/turn.ts
+++ b/packages/eve/src/execution/session/turn.ts
@@ -94,6 +94,7 @@ export class SessionExecution {
const pendingCallIds =
result.action === "park" ? result.pendingCoordinationCallIds : undefined;
const hasBackgroundTasks = (result.backgroundTasks?.length ?? 0) > 0;
+ const settled = result.action === "park" && result.settled !== undefined;
if (hasBackgroundTasks) {
if (result.backgroundTaskState === undefined) {
@@ -109,7 +110,7 @@ export class SessionExecution {
await cursor.apply({
serializedContext: result.serializedContext,
sessionState:
- result.action === "cancelled" || turn.signal.aborted
+ result.action === "cancelled" || (!settled && turn.signal.aborted)
? (result.backgroundTaskState ?? result.sessionState)
: result.sessionState,
});
@@ -117,7 +118,7 @@ export class SessionExecution {
turn.resetSteering();
if (result.action === "cancelled") return await this.finishCancelledTurn();
- if (turn.signal.aborted && (pendingCallIds === undefined || hasBackgroundTasks)) {
+ if (!settled && turn.signal.aborted && (pendingCallIds === undefined || hasBackgroundTasks)) {
return await this.finishCancelledTurn();
}
diff --git a/packages/eve/src/execution/turn-cancellation.integration.test.ts b/packages/eve/src/execution/turn-cancellation.integration.test.ts
index 6f0f37b6a..7bc70171b 100644
--- a/packages/eve/src/execution/turn-cancellation.integration.test.ts
+++ b/packages/eve/src/execution/turn-cancellation.integration.test.ts
@@ -830,9 +830,12 @@ describe("turn cancellation integration", () => {
expect(firstTurn.at(-1)?.type).toBe("session.waiting");
expect(filterEventsByType(firstTurn, "turn.completed")).toHaveLength(1);
- // The stable inbox accepts a late cancel and the parked owner consumes it as a no-op.
+ // Hook aliases can be observed out of order. Target the completed turn so
+ // a delayed cancellation cannot apply to the follow-up, as in the client.
+ const completedTurnId = filterEventsByType(firstTurn, "turn.started")[0]!.data.turnId;
await resumeHook(sessionInboxHookToken(sessionCommandHookToken(run.runId)), {
kind: "cancel",
+ turnId: completedTurnId,
});
await waitForHook(
diff --git a/packages/eve/src/execution/workflow-runtime.test.ts b/packages/eve/src/execution/workflow-runtime.test.ts
index 4f5c55a46..8282ac1bf 100644
--- a/packages/eve/src/execution/workflow-runtime.test.ts
+++ b/packages/eve/src/execution/workflow-runtime.test.ts
@@ -64,6 +64,7 @@ beforeEach(() => {
return currentSessionHook(token);
});
getWorldMock.mockResolvedValue(world);
+ getRunMock.mockReturnValue({ status: Promise.resolve("completed") });
resumeHookMock.mockImplementation(async (token: string) => currentSessionHook(token));
});
@@ -235,6 +236,25 @@ describe("createWorkflowRuntime command dispatch", () => {
expect(getHookByTokenMock).not.toHaveBeenCalled();
});
+ it.each(["pending", "running", "completed", "failed", "cancelled"])(
+ "distinguishes an unclaimed inbox for a %s workflow",
+ async (status) => {
+ resumeHookMock.mockRejectedValue(new HookNotFoundError("missing"));
+ getRunMock.mockReturnValue({ status: Promise.resolve(status) });
+ await expect(
+ buildRuntime().dispatchSession({
+ sessionId: "session-1",
+ command: { kind: "send", payload: { message: "Hello" } },
+ }),
+ ).resolves.toEqual(
+ status === "pending" || status === "running"
+ ? { status: "session_not_active", retryable: true }
+ : { status: "session_not_active" },
+ );
+ expect(startMock).not.toHaveBeenCalled();
+ },
+ );
+
it("dispatches commands through the stable session inbox", async () => {
resumeHookMock.mockResolvedValue({ runId: "session-1" });
diff --git a/packages/eve/src/execution/workflow-runtime.ts b/packages/eve/src/execution/workflow-runtime.ts
index 50397e298..2ad04ee15 100644
--- a/packages/eve/src/execution/workflow-runtime.ts
+++ b/packages/eve/src/execution/workflow-runtime.ts
@@ -397,6 +397,19 @@ async function dispatchWorkflowCommand<TCommand extends SessionCommand>(
hook = { runId: resumed.ownerRunId, sessionId: await resumed.sessionId };
} catch (error) {
if (isInactiveCommandTarget(error)) {
+ if (command.kind === "send" && typeof token !== "string") {
+ try {
+ const status = await getRun(token.sessionId).status;
+ if (status === "pending" || status === "running") {
+ return {
+ status: "session_not_active",
+ retryable: true,
+ } as SessionCommandResult<TCommand>;
+ }
+ } catch (statusError) {
+ if (!isInactiveCommandTarget(statusError)) throw statusError;
+ }
+ }
return inactiveCommandResult(command);
}
logError(log, "failed to dispatch session command", error, {
diff --git a/packages/eve/src/instrumentation/prepare-trace-context.test.ts b/packages/eve/src/instrumentation/prepare-trace-context.test.ts
index c351e0541..4ee6eefdc 100644
--- a/packages/eve/src/instrumentation/prepare-trace-context.test.ts
+++ b/packages/eve/src/instrumentation/prepare-trace-context.test.ts
@@ -5,6 +5,23 @@ import { SessionTraceSeedKey } from "#context/keys.js";
import { prepareTurnTraceContext } from "#instrumentation/prepare-trace-context.js";
describe("prepareTurnTraceContext", () => {
+ it("prepares only the session trace when turn zero does not exist yet", async () => {
+ const seed = { spanId: "2".repeat(16), traceFlags: 1, traceId: "1".repeat(32) };
+ const prepareSessionTrace = vi.fn(async (_event: unknown) => seed);
+ const prepareTurnTrace = vi.fn(async (_event: unknown) => seed);
+
+ const trace = await prepareTurnTraceContext({
+ instrumentation: { prepareSessionTrace, prepareTurnTrace },
+ session: { rootSessionId: "session-1", sessionId: "session-1" },
+ sequence: 0,
+ sessionStarted: false,
+ });
+
+ expect(trace).toEqual(seed);
+ expect(prepareSessionTrace).toHaveBeenCalledOnce();
+ expect(prepareTurnTrace).not.toHaveBeenCalled();
+ });
+
it("reuses session parent metadata without copying the seed onto a turn event", async () => {
const seed = { spanId: "2".repeat(16), traceFlags: 1, traceId: "1".repeat(32) };
const parentLineage = {
diff --git a/packages/eve/src/instrumentation/prepare-trace-context.ts b/packages/eve/src/instrumentation/prepare-trace-context.ts
index 2c89fdd01..3a73c589f 100644
--- a/packages/eve/src/instrumentation/prepare-trace-context.ts
+++ b/packages/eve/src/instrumentation/prepare-trace-context.ts
@@ -29,7 +29,8 @@ export interface PrepareTurnTraceContextInput {
readonly sequence: number;
readonly sessionStarted: boolean;
readonly traceContext?: RuntimeTraceContext;
- readonly turnId: string;
+ /** Omit when preparing the session boundary before the first turn exists. */
+ readonly turnId?: string;
}
export async function prepareTurnTraceContext(
@@ -54,7 +55,7 @@ export async function prepareTurnTraceContext(
}
}
- if (input.instrumentation?.prepareTurnTrace !== undefined) {
+ if (input.turnId !== undefined && input.instrumentation?.prepareTurnTrace !== undefined) {
try {
prepared = await input.instrumentation.prepareTurnTrace({
...session,
diff --git a/packages/eve/src/public/channels/eve-session-routes.test.ts b/packages/eve/src/public/channels/eve-session-routes.test.ts
index a355415fb..f1e2b29c2 100644
--- a/packages/eve/src/public/channels/eve-session-routes.test.ts
+++ b/packages/eve/src/public/channels/eve-session-routes.test.ts
@@ -47,6 +47,69 @@ function createArgs(session = createFixedSession()): RouteHandlerArgs {
}
describe("eve ID-addressed session routes", () => {
+ it.each([
+ ["without a request body", undefined],
+ ["with an empty JSON object", "{}"],
+ ])("creates and parks a conversation %s", async (_description, body) => {
+ const createSession = vi.fn().mockResolvedValue({
+ events: new ReadableStream(),
+ sessionId: "wrun_A",
+ });
+ const args = attachRouteSessionCreator(createArgs(), createSession);
+ const onMessage = vi.fn();
+
+ const response = await route("POST", "/eve/v1/session", {
+ auth: none(),
+ onMessage,
+ })(
+ new Request("https://eve.test/eve/v1/session", {
+ body,
+ headers: body === undefined ? undefined : { "content-type": "application/json" },
+ method: "POST",
+ }),
+ args,
+ );
+
+ expect(response.status).toBe(202);
+ await expect(response.json()).resolves.toEqual({
+ ok: true,
+ sessionId: "wrun_A",
+ status: "accepted",
+ });
+ expect(onMessage).not.toHaveBeenCalled();
+ expect(createSession).toHaveBeenCalledWith(
+ expect.objectContaining({
+ capabilities: { requestInput: true },
+ input: {
+ context: undefined,
+ message: undefined,
+ outputSchema: undefined,
+ },
+ mode: "conversation",
+ }),
+ );
+ });
+
+ it.each([
+ ["malformed JSON", "{"],
+ ["a non-object JSON value", "[]"],
+ ])("rejects %s when creating a session", async (_description, body) => {
+ const createSession = vi.fn();
+ const args = attachRouteSessionCreator(createArgs(), createSession);
+
+ const response = await route("POST", "/eve/v1/session")(
+ new Request("https://eve.test/eve/v1/session", {
+ body,
+ headers: { "content-type": "application/json" },
+ method: "POST",
+ }),
+ args,
+ );
+
+ expect(response.status).toBe(400);
+ expect(createSession).not.toHaveBeenCalled();
+ });
+
it("creates a session without a continuation token", async () => {
const createSession = vi.fn().mockResolvedValue({
events: new ReadableStream(),
diff --git a/packages/eve/src/public/channels/eve.test.ts b/packages/eve/src/public/channels/eve.test.ts
index d16e397f5..80c017e38 100644
--- a/packages/eve/src/public/channels/eve.test.ts
+++ b/packages/eve/src/public/channels/eve.test.ts
@@ -693,6 +693,22 @@ describe("eveChannel — onMessage", () => {
expect(options.title).toBe("HTTP run");
});
+ it("passes first-message auth, context, and title to a prewarmed session", async () => {
+ const handler = createEveContinueHandler({
+ auth: () => ACCEPTED_AUTH,
+ onMessage: () => ({ auth: OVERRIDE_AUTH, context: ["first context"], title: "First chat" }),
+ });
+ expect((await handler.fetch(createJsonMessageRequest({ message: "Hello" }))).status).toBe(202);
+ expect(handler.send).toHaveBeenCalledWith(
+ "Hello",
+ expect.objectContaining({
+ auth: OVERRIDE_AUTH,
+ context: ["first context"],
+ title: "First chat",
+ }),
+ );
+ });
+
it("uses auth returned from onMessage for create requests", async () => {
const handler = createEveCreateHandler({
auth: () => ACCEPTED_AUTH,
@@ -885,6 +901,18 @@ describe("eveChannel — onMessage", () => {
expect(handler.send).not.toHaveBeenCalled();
});
+ it("exposes a retryable code while the session inbox is starting", async () => {
+ const handler = createEveContinueHandler({ auth: none() });
+ handler.send.mockResolvedValueOnce({ status: "session_not_active", retryable: true });
+ const response = await handler.fetch(createJsonMessageRequest({ message: "Hello" }));
+ expect(response.status).toBe(409);
+ await expect(response.json()).resolves.toEqual({
+ code: "session_not_ready",
+ error: "The session is not ready to accept messages yet.",
+ ok: false,
+ });
+ });
+
it("returns session_not_active instead of starting a replacement session", async () => {
const handler = createEveContinueHandler({ auth: none() });
handler.send.mockResolvedValueOnce({ status: "session_not_active" });
diff --git a/packages/eve/src/react/use-eve-agent.test.ts b/packages/eve/src/react/use-eve-agent.test.ts
index 30e1ef2c1..d9eed06d0 100644
--- a/packages/eve/src/react/use-eve-agent.test.ts
+++ b/packages/eve/src/react/use-eve-agent.test.ts
@@ -1,4 +1,4 @@
-import { createElement } from "react";
+import { createElement, StrictMode } from "react";
import { act, create as createRenderer } from "react-test-renderer";
import { afterEach, describe, expect, it, vi } from "vitest";
@@ -150,12 +150,63 @@ afterEach(async () => {
});
describe("useEveAgent", () => {
+ it("prewarms in Strict Mode when explicitly enabled", async () => {
+ const streams: Array<{ signal: AbortSignal; cancel: ReturnType<typeof vi.fn> }> = [];
+ let creates = 0;
+ const fetchMock = vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) => {
+ if (init?.method === "POST") return createStartedMessageResponse(`session_${++creates}`, "");
+ const cancel = vi.fn();
+ const body = new ReadableStream<Uint8Array>({
+ start(controller) {
+ const [waiting] = stampTestEvents([createSessionWaitingEvent()]);
+ controller.enqueue(new TextEncoder().encode(`${JSON.stringify(waiting)}\n`));
+ },
+ cancel,
+ });
+ streams.push({ signal: init!.signal!, cancel });
+ return new Response(body, {
+ headers: { [EVE_STREAM_VERSION_HEADER]: EVE_MESSAGE_STREAM_VERSION },
+ });
+ });
+ let agent: UseEveAgentHelpers<EveMessageData> | undefined;
+ function Chat() {
+ agent = useEveAgent({ prewarm: true });
+ return null;
+ }
+ let root: ReturnType<typeof create> | undefined;
+ await act(async () => {
+ root = create(createElement(StrictMode, null, createElement(Chat)));
+ });
+ await act(async () => {
+ await vi.waitFor(() => expect(agent?.events).toHaveLength(1));
+ });
+ expect(creates).toBe(1);
+ expect(fetchMock).toHaveBeenCalledTimes(2);
+ expect(agent?.status).toBe("ready");
+
+ await act(async () => {
+ agent?.reset();
+ });
+ await act(async () => {
+ await vi.waitFor(() => expect(agent?.session?.sessionId).toBe("session_2"));
+ });
+ expect(creates).toBe(2);
+ expect(streams[0]?.signal.aborted).toBe(true);
+ expect(streams[0]?.cancel).toHaveBeenCalledOnce();
+ expect(streams[1]?.signal.aborted).toBe(false);
+ await act(async () => {
+ root?.unmount();
+ });
+ expect(streams[1]?.signal.aborted).toBe(true);
+ expect(streams[1]?.cancel).toHaveBeenCalledOnce();
+ });
+
it("keeps the helpers object stable across renders without store changes", async () => {
const seenHelpers: Array<UseEveAgentHelpers<EveMessageData>> = [];
let root: ReturnType<typeof create> | undefined;
function TestComponent({ label: _label }: { readonly label: string }) {
- seenHelpers.push(useEveAgent());
+ seenHelpers.push(useEveAgent({ prewarm: false }));
return null;
}
@@ -196,7 +247,7 @@ describe("useEveAgent", () => {
let root: ReturnType<typeof create> | undefined;
function TestComponent() {
- helpers = useEveAgent();
+ helpers = useEveAgent({ prewarm: false });
return null;
}
@@ -223,7 +274,7 @@ describe("useEveAgent", () => {
});
});
- it("sends a message and projects streamed events with the default reducer", async () => {
+ it.each([undefined, false])("defers creation until send (prewarm=%s)", async (prewarm) => {
const events = [
createMessageReceivedEvent({
message: "Hello",
@@ -252,6 +303,7 @@ describe("useEveAgent", () => {
function TestComponent() {
helpers = useEveAgent({
+ prewarm,
onEvent(event) {
lifecycle.push(`event:${event.type}`);
seenEvents.push(event);
@@ -268,6 +320,15 @@ describe("useEveAgent", () => {
create(createElement(TestComponent));
});
+ await act(async () => {
+ await new Promise((resolve) => setTimeout(resolve, 10));
+ helpers?.reset();
+ });
+ expect(fetchMock).not.toHaveBeenCalled();
+ expect(helpers?.session).toBeUndefined();
+ lifecycle.length = 0;
+ seenSessions.length = 0;
+
let sendPromise: Promise<void> | undefined;
await act(async () => {
sendPromise = helpers?.send("Hello");
@@ -286,16 +347,7 @@ describe("useEveAgent", () => {
expect(fetchMock).toHaveBeenCalledTimes(2);
expect(lifecycle[0]).toBe("session:0");
expect(seenEvents).toEqual(stampTestEvents(events));
- expect(seenSessions).toEqual([
- {
- sessionId: "session_1",
- streamIndex: 0,
- },
- {
- sessionId: "session_1",
- streamIndex: 3,
- },
- ]);
+ expect(seenSessions.map((session) => session?.streamIndex)).toEqual([0, 1, 2, 3, 3]);
expect(helpers?.status).toBe("ready");
expect(helpers?.session).toEqual({
sessionId: "session_1",
@@ -324,7 +376,7 @@ describe("useEveAgent", () => {
let helpers: UseEveAgentHelpers<EveMessageData> | undefined;
function TestComponent() {
- helpers = useEveAgent();
+ helpers = useEveAgent({ prewarm: false });
return null;
}
@@ -355,14 +407,18 @@ describe("useEveAgent", () => {
const startResponse = createDeferred<Response>();
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockReturnValueOnce(startResponse.promise)
- .mockResolvedValueOnce(createEagerStreamResponse([createSessionWaitingEvent()]));
+ .mockImplementation(async (_request, init) =>
+ init?.method === "POST"
+ ? await startResponse.promise
+ : createEagerStreamResponse([createSessionWaitingEvent()]),
+ );
let randomWord = "jazz";
let helpers: UseEveAgentHelpers<EveMessageData> | undefined;
function TestComponent() {
helpers = useEveAgent({
+ prewarm: false,
prepareSend(input) {
return {
...input,
@@ -408,13 +464,17 @@ describe("useEveAgent", () => {
const startResponse = createDeferred<Response>();
const fetchMock = vi
.spyOn(globalThis, "fetch")
- .mockReturnValueOnce(startResponse.promise)
- .mockResolvedValueOnce(createEagerStreamResponse([createSessionWaitingEvent()]));
+ .mockImplementation(async (_request, init) =>
+ init?.method === "POST"
+ ? await startResponse.promise
+ : createEagerStreamResponse([createSessionWaitingEvent()]),
+ );
let helpers: UseEveAgentHelpers<EveMessageData> | undefined;
function TestComponent() {
helpers = useEveAgent({
+ prewarm: false,
agent: "support",
});
return null;
@@ -449,6 +509,7 @@ describe("useEveAgent", () => {
function TestComponent() {
helpers = useEveAgent({
+ prewarm: false,
onError(error) {
seenErrors.push(error);
},
@@ -498,6 +559,7 @@ describe("useEveAgent", () => {
function TestComponent() {
helpers = useEveAgent({
+ prewarm: false,
onFinish(snapshot) {
seenFinishes.push(snapshot.data);
},
@@ -580,6 +642,7 @@ describe("useEveAgent", () => {
function TestComponent() {
helpers = useEveAgent({
+ prewarm: false,
onError(error) {
seenErrors.push(error);
},
@@ -648,6 +711,7 @@ describe("useEveAgent", () => {
function TestComponent() {
helpers = useEveAgent({
+ prewarm: false,
onError(error) {
seenErrors.push(error);
},
@@ -678,7 +742,7 @@ describe("useEveAgent", () => {
it("requires a session when automatic resume is enabled", async () => {
function TestComponent() {
- useEveAgent({ resume: true });
+ useEveAgent({ prewarm: false, resume: true });
return null;
}
@@ -724,7 +788,7 @@ describe("useEveAgent", () => {
await vi.waitFor(() => expect(helpers?.status).toBe("ready"));
});
- expect(fetchMock).toHaveBeenCalledTimes(2);
+ expect(fetchMock).toHaveBeenCalledTimes(1);
expect(statuses[0]).toBe("resuming");
expect(statuses).not.toContain("submitted");
expect(statuses).not.toContain("streaming");
@@ -740,9 +804,13 @@ describe("useEveAgent", () => {
it("projects input responses before the resumed stream returns", async () => {
const startResponse = createDeferred<Response>();
- vi.spyOn(globalThis, "fetch")
- .mockReturnValueOnce(startResponse.promise)
- .mockResolvedValueOnce(createEagerStreamResponse([createSessionWaitingEvent()]));
+ vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) =>
+ init?.method === "POST"
+ ? await startResponse.promise
+ : await startResponse.promise.then(() =>
+ createEagerStreamResponse([createSessionWaitingEvent()]),
+ ),
+ );
let helpers: UseEveAgentHelpers<readonly string[]> | undefined;
diff --git a/packages/eve/src/react/use-eve-agent.ts b/packages/eve/src/react/use-eve-agent.ts
index accc7e07e..b685894ee 100644
--- a/packages/eve/src/react/use-eve-agent.ts
+++ b/packages/eve/src/react/use-eve-agent.ts
@@ -1,6 +1,7 @@
import { useCallback, useEffect, useMemo, useRef, useState, useSyncExternalStore } from "react";
import {
+ attachEveAgentStore,
detachEveAgentStore,
EveAgentStore,
type EveAgentStoreCallbacks,
@@ -51,6 +52,8 @@ export interface UseEveAgentHelpers<TData> extends UseEveAgentSnapshot<TData> {
readonly cancel: () => Promise<CancelSessionResult>;
/** Replays the attached durable session and follows its in-flight turn, if any. */
readonly resume: () => Promise<void>;
+ /** Creates the session without starting its first turn. */
+ readonly prewarm: () => Promise<void>;
/** Resets the session: detaches any local stream, recreates the owned session, and clears events and projected data. */
readonly reset: () => void;
/** Sends a message. While a turn is active, pass `turnPolicy: "steer"` to replace it. */
@@ -108,6 +111,8 @@ export interface UseEveAgentOptions<TData> extends EveAgentStoreCallbacks<TData>
* @default true
*/
readonly optimistic?: boolean;
+ /** Prewarm an owned session on mount and after reset. @default false */
+ readonly prewarm?: boolean;
readonly reducer?: EveAgentReducer<TData>;
/**
* Replay the attached durable session after mount and follow its in-flight
@@ -131,13 +136,13 @@ export function useEveAgent<TData>(
* React hook that drives an eve session and projects its event stream into UI data.
*
* Returns the current snapshot (`data`, `events`, `session`, `status`, `error`)
- * plus the commands `send`, `respond`, `resume`, `cancel`, and `reset`. With no reducer, `data` is the
+ * plus the commands `prewarm`, `send`, `respond`, `resume`, `cancel`, and `reset`. With no reducer, `data` is the
* built-in `UIMessage` projection from {@link defaultMessageReducer} (`TData`
* is {@link EveMessageData}); pass a reducer to project into your own shape and
* infer `TData`.
*
* Session-shaping options (`host`, `reducer`, `session`, `initialEvents`,
- * `initialSession`, `auth`, `headers`, `optimistic`, `resume`) are
+ * `initialSession`, `auth`, `headers`, `optimistic`, `prewarm`, `resume`) are
* read once when the store is created; remount to change them. Lifecycle
* callbacks (`onError`, `onEvent`, `onFinish`, `onSessionChange`, `prepareSend`)
* refresh on every render.
@@ -165,6 +170,7 @@ export function useEveAgent<TData>(
initialEvents: options.initialEvents,
initialSession: options.initialSession,
optimistic: options.optimistic,
+ prewarm: options.prewarm,
reducer,
session: options.session,
});
@@ -189,7 +195,13 @@ export function useEveAgent<TData>(
() => store.snapshot,
);
- useEffect(() => () => detachEveAgentStore(store), [store]);
+ useEffect(() => {
+ const timeout = setTimeout(() => attachEveAgentStore(store), 0);
+ return () => {
+ clearTimeout(timeout);
+ detachEveAgentStore(store);
+ };
+ }, [store]);
useEffect(() => {
if (!resumeOnMountRef.current) return;
let active = true;
@@ -205,6 +217,7 @@ export function useEveAgent<TData>(
const cancel = useCallback(() => store.cancel(), [store]);
const reset = useCallback(() => store.reset(), [store]);
+ const prewarm = useCallback(() => store.prewarm(), [store]);
const resume = useCallback(() => store.resume(), [store]);
const send = useCallback(
<TOutput = unknown>(message: string | UserContent, options?: SendTurnOptions<TOutput>) => {
@@ -228,11 +241,12 @@ export function useEveAgent<TData>(
() => ({
...visibleSnapshot,
cancel,
+ prewarm,
reset,
respond,
resume,
send,
}),
- [cancel, reset, respond, resume, send, visibleSnapshot],
+ [cancel, prewarm, reset, respond, resume, send, visibleSnapshot],
);
}
diff --git a/packages/eve/src/subagents/handle-dispatch.ts b/packages/eve/src/subagents/handle-dispatch.ts
index 431afdac3..bf52aa207 100644
--- a/packages/eve/src/subagents/handle-dispatch.ts
+++ b/packages/eve/src/subagents/handle-dispatch.ts
@@ -236,11 +236,11 @@ async function deliverToAgentAddress(input: {
},
sessionId: address.sessionId,
});
- if (result.status === "session_not_active") {
+ if (result.status !== "accepted") {
return err({
- cause: new Error(`Agent session "${address.sessionId}" is no longer active.`),
+ cause: new Error(`Agent session "${address.sessionId}" is not available.`),
deliveryAmbiguous: false,
- permanent: true,
+ permanent: result.retryable !== true,
});
}
} catch (error) {
diff --git a/packages/eve/src/svelte/use-eve-agent.test.ts b/packages/eve/src/svelte/use-eve-agent.test.ts
index 63171ed87..b41b0fd6a 100644
--- a/packages/eve/src/svelte/use-eve-agent.test.ts
+++ b/packages/eve/src/svelte/use-eve-agent.test.ts
@@ -50,7 +50,15 @@ function createBoundedStreamResponse(
return response;
}
+const cleanupMounts: Array<() => void> = [];
+vi.mock("svelte", () => ({
+ onMount: (callback: () => () => void) => {
+ cleanupMounts.push(callback());
+ },
+}));
+
afterEach(() => {
+ for (const cleanup of cleanupMounts.splice(0)) cleanup();
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
@@ -118,7 +126,7 @@ describe("useEveAgent (Svelte rune binding)", () => {
});
await vi.waitFor(() => expect(seenEvents).toEqual(stampTestEvents(events)));
- await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));
+ expect(fetchMock).toHaveBeenCalledTimes(1);
});
it("sends messages and notifies lifecycle callbacks from the shared store", async () => {
@@ -133,6 +141,7 @@ describe("useEveAgent (Svelte rune binding)", () => {
const seenEvents: UnstampedMessageStreamEvent[] = [];
const agent = useEveAgent({
+ prewarm: false,
onEvent(event) {
seenEvents.push(event);
},
diff --git a/packages/eve/src/svelte/use-eve-agent.ts b/packages/eve/src/svelte/use-eve-agent.ts
index 437290e34..b179d0b2e 100644
--- a/packages/eve/src/svelte/use-eve-agent.ts
+++ b/packages/eve/src/svelte/use-eve-agent.ts
@@ -1,7 +1,9 @@
+import { onMount } from "svelte";
import { createSubscriber } from "svelte/reactivity";
import type { UserContent } from "ai";
import {
+ attachEveAgentStore,
detachEveAgentStore,
EveAgentStore,
type EveAgentStoreCallbacks,
@@ -57,6 +59,8 @@ export interface UseEveAgentReturn<TData> {
readonly events: readonly MessageStreamEvent[];
/** Replay the attached durable session and follow its in-flight turn, if any. */
readonly resume: () => Promise<void>;
+ /** Create the session without starting its first turn. */
+ readonly prewarm: () => Promise<void>;
/** Clear all state and start a new session. */
readonly reset: () => void;
/** Send a message with optional turn settings. */
@@ -126,6 +130,8 @@ export interface UseEveAgentOptions<TData> extends EveAgentStoreCallbacks<TData>
* @default true
*/
readonly optimistic?: boolean;
+ /** Prewarm an owned session on mount and after reset. @default false */
+ readonly prewarm?: boolean;
/**
* Projects stream events into `TData`. Defaults to {@link defaultMessageReducer},
* which fixes `TData` to {@link EveMessageData}.
@@ -158,7 +164,6 @@ class SvelteEveAgent<TData> implements UseEveAgentReturn<TData> {
return () => {
unsubscribe();
- detachEveAgentStore(store);
};
});
}
@@ -196,6 +201,10 @@ class SvelteEveAgent<TData> implements UseEveAgentReturn<TData> {
this.#store.reset();
};
+ prewarm = (): Promise<void> => {
+ return this.#store.prewarm();
+ };
+
resume = (): Promise<void> => {
return this.#store.resume();
};
@@ -246,6 +255,7 @@ export function useEveAgent<TData>(
initialEvents: options.initialEvents,
initialSession: options.initialSession,
optimistic: options.optimistic,
+ prewarm: options.prewarm,
reducer,
session: options.session,
});
@@ -257,7 +267,12 @@ export function useEveAgent<TData>(
onSessionChange: options.onSessionChange,
prepareSend: options.prepareSend,
});
- if ("window" in globalThis && options.resume) void store.resume();
+ if ("window" in globalThis)
+ onMount(() => {
+ attachEveAgentStore(store);
+ if (options.resume) void store.resume();
+ return () => detachEveAgentStore(store);
+ });
return new SvelteEveAgent(store);
}
diff --git a/packages/eve/src/vue/use-eve-agent.test.ts b/packages/eve/src/vue/use-eve-agent.test.ts
index 1614bf5fc..5525c99c3 100644
--- a/packages/eve/src/vue/use-eve-agent.test.ts
+++ b/packages/eve/src/vue/use-eve-agent.test.ts
@@ -1,7 +1,11 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { effectScope } from "vue";
-import { EveAgentStore, type EveAgentStoreSnapshot } from "#client/eve-agent-store.js";
+import {
+ detachEveAgentStore,
+ EveAgentStore,
+ type EveAgentStoreSnapshot,
+} from "#client/eve-agent-store.js";
import { useEveAgent } from "#vue/use-eve-agent.js";
import type { EveMessageData } from "#client/message-reducer.js";
import {
@@ -107,14 +111,24 @@ function completedTurnData(input: {
};
}
+const cleanupStores: Array<() => void> = [];
+function createStore<TData>(
+ init: ConstructorParameters<typeof EveAgentStore<TData>>[0],
+): EveAgentStore<TData> {
+ const store = new EveAgentStore<TData>(init);
+ cleanupStores.push(() => detachEveAgentStore(store));
+ return store;
+}
+
afterEach(() => {
+ for (const cleanup of cleanupStores.splice(0)) cleanup();
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
describe("EveAgentStore (Vue composable backing store)", () => {
it("starts in ready status with empty data", () => {
- const store = new EveAgentStore({
+ const store = createStore({
reducer: defaultMessageReducer(),
});
@@ -125,7 +139,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {
});
it("notifies subscribers on state changes", async () => {
- const store = new EveAgentStore({
+ const store = createStore({
reducer: defaultMessageReducer(),
});
@@ -163,7 +177,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {
.mockReturnValueOnce(startResponse.promise)
.mockResolvedValueOnce(createEagerStreamResponse(events));
- const store = new EveAgentStore({
+ const store = createStore({
reducer: defaultMessageReducer(),
});
@@ -205,22 +219,13 @@ describe("EveAgentStore (Vue composable backing store)", () => {
userMessage: "Hello",
}),
);
- expect(seenSessions).toEqual([
- {
- sessionId: "session_1",
- streamIndex: 0,
- },
- {
- sessionId: "session_1",
- streamIndex: 3,
- },
- ]);
+ expect(seenSessions.map((session) => session?.streamIndex)).toEqual([0, 1, 2, 3, 3]);
});
it("surfaces transport errors", async () => {
vi.spyOn(globalThis, "fetch").mockRejectedValueOnce(new Error("Network failed"));
- const store = new EveAgentStore({
+ const store = createStore({
reducer: defaultMessageReducer(),
});
@@ -257,7 +262,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {
.mockReturnValueOnce(startResponse.promise)
.mockResolvedValueOnce(createEagerStreamResponse(events));
- const store = new EveAgentStore({
+ const store = createStore({
reducer: defaultMessageReducer(),
});
@@ -280,7 +285,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {
it("resets state and creates a new session", async () => {
vi.spyOn(globalThis, "fetch").mockRejectedValueOnce(new Error("fail"));
- const store = new EveAgentStore({
+ const store = createStore({
reducer: defaultMessageReducer(),
});
@@ -295,7 +300,7 @@ describe("EveAgentStore (Vue composable backing store)", () => {
});
it("unsubscribe removes the listener", () => {
- const store = new EveAgentStore({
+ const store = createStore({
reducer: defaultMessageReducer(),
});
@@ -314,11 +319,13 @@ describe("EveAgentStore (Vue composable backing store)", () => {
it("projects input responses before the resumed stream returns", async () => {
const startResponse = createDeferred<Response>();
- vi.spyOn(globalThis, "fetch")
- .mockReturnValueOnce(startResponse.promise)
- .mockResolvedValueOnce(createEagerStreamResponse([createSessionWaitingEvent()]));
+ vi.spyOn(globalThis, "fetch").mockImplementation(async (_request, init) =>
+ init?.method === "POST"
+ ? await startResponse.promise
+ : createEagerStreamResponse([createSessionWaitingEvent()]),
+ );
- const store = new EveAgentStore<readonly string[]>({
+ const store = createStore<readonly string[]>({
initialSession: {
sessionId: "session_1",
streamIndex: 0,
@@ -407,7 +414,7 @@ describe("useEveAgent (Vue composable wiring)", () => {
.mockResolvedValueOnce(createEagerStreamResponse(events));
const scope = effectScope();
- const agent = scope.run(() => useEveAgent());
+ const agent = scope.run(() => useEveAgent({ prewarm: false }));
if (agent === undefined) throw new Error("effect scope did not run");
expect(agent.status.value).toBe("ready");
@@ -444,7 +451,7 @@ describe("useEveAgent (Vue composable wiring)", () => {
);
const scope = effectScope();
- const agent = scope.run(() => useEveAgent());
+ const agent = scope.run(() => useEveAgent({ prewarm: false }));
if (agent === undefined) throw new Error("effect scope did not run");
const dataBeforeDispose = agent.data.value;
diff --git a/packages/eve/src/vue/use-eve-agent.ts b/packages/eve/src/vue/use-eve-agent.ts
index 8ef130961..e9d82989b 100644
--- a/packages/eve/src/vue/use-eve-agent.ts
+++ b/packages/eve/src/vue/use-eve-agent.ts
@@ -2,6 +2,7 @@ import { shallowRef, computed, onScopeDispose, type ComputedRef } from "vue";
import type { UserContent } from "ai";
import {
+ attachEveAgentStore,
detachEveAgentStore,
EveAgentStore,
type EveAgentStoreCallbacks,
@@ -55,6 +56,8 @@ export interface UseEveAgentReturn<TData> {
readonly events: ComputedRef<readonly MessageStreamEvent[]>;
/** Replay the attached durable session and follow its in-flight turn, if any. */
readonly resume: () => Promise<void>;
+ /** Create the session without starting its first turn. */
+ readonly prewarm: () => Promise<void>;
/** Clear all state and start a new session. */
readonly reset: () => void;
/** Send a message with optional turn settings. */
@@ -125,6 +128,8 @@ export interface UseEveAgentOptions<TData> extends EveAgentStoreCallbacks<TData>
* @default true
*/
readonly optimistic?: boolean;
+ /** Prewarm an owned session on mount and after reset. @default false */
+ readonly prewarm?: boolean;
/**
* Projects stream events into `TData`.
*
@@ -156,7 +161,7 @@ export function useEveAgent<TData>(
* Without a `reducer`, events project into `EveMessageData` via
* `defaultMessageReducer()`; pass `reducer` to project into a custom `TData`.
* Returns reactive refs (`data`, `error`, `events`, `session`, `status`) plus
- * `send`, `respond`, `resume`, `cancel`, and `reset`. Configuration is read once on store creation;
+ * `prewarm`, `send`, `respond`, `resume`, `cancel`, and `reset`. Configuration is read once on store creation;
* remount to change it. On scope dispose, the in-flight request is detached and
* the store unsubscribed.
*/
@@ -175,6 +180,7 @@ export function useEveAgent<TData>(
initialEvents: options.initialEvents,
initialSession: options.initialSession,
optimistic: options.optimistic,
+ prewarm: options.prewarm,
reducer,
session: options.session,
});
@@ -193,6 +199,7 @@ export function useEveAgent<TData>(
const unsubscribe = store.subscribe(() => {
snapshot.value = store.snapshot;
});
+ attachEveAgentStore(store);
if (options.resume) void store.resume();
onScopeDispose(() => {
@@ -206,6 +213,7 @@ export function useEveAgent<TData>(
data: computed(() => snapshot.value.data),
error: computed(() => snapshot.value.error),
events: computed(() => snapshot.value.events),
+ prewarm: () => store.prewarm(),
reset: () => store.reset(),
respond: <TOutput = unknown>(
inputResponses: Parameters<ClientSession["respond"]>[0],
diff --git a/research/session-prewarming.md b/research/session-prewarming.md
new file mode 100644
index 000000000..a443c8ea5
--- /dev/null
+++ b/research/session-prewarming.md
@@ -0,0 +1,70 @@
+---
+issue: https://github.com/vercel/eve/issues/1476
+status: implemented
+last_updated: "2026-09-16"
+---
+
+# Session prewarming
+
+## Authoring API
+
+`POST /eve/v1/session` with `{}` and `client.sessions.create()` start a conversation workflow
+without a message. They return an accepted session ID. `session.send(message)` sends the first
+message to that ID; `create({ message })` still creates and starts a turn in one request.
+
+React, Vue, and Svelte accept `useEveAgent({ prewarm: true })` to prewarm on mount and after
+reset. The default is `false`. The explicit `prewarm()` method resolves on `202 Accepted`;
+concurrent calls share the same request. Applications must have auth, headers, and any chat-row
+binding ready before opting into automatic prewarming.
+
+Eval contexts expose `t.session(options?)` to create an accepted session without a turn,
+and `t.send(message, options?)` to create a fresh session with its first message in one
+request. Both are factories: follow-ups use `session.send()` or `turn.session.send()`.
+Session IDs and cursors are always available on returned handles. `t` retains run-level
+assertions and judges, while session operations and conversation state live on the handle.
+This breaks the implicit primary-session API (`t.newSession`, `t.prewarm`, and session
+fields/methods on `t`). Every accepted session participates in reporting and timeout cleanup.
+Turn output and default judges follow the most recently settled turn; the first registered
+session remains the primary ID in report metadata.
+
+## Runtime boundary
+
+Prewarming starts Workflow, creates the initial durable snapshot, and claims the existing inbox.
+It parks before channel delivery, session hooks, dynamic definitions, sandbox setup, and tracing
+of the first turn. No initial `session.started` or `session.waiting` event is emitted.
+
+The first message runs `onMessage`, then resumes the inbox. The normal turn step initializes the
+session with that message's auth and context and starts `turn_0`. The initiator is established
+from that first message unless explicitly forwarded at creation. The workflow title is set from
+`onMessage.title` or the first message; later messages do not rename it. No separate initialization
+step or harness mode is needed.
+
+```mermaid
+flowchart LR
+ Mount["mount with prewarm: true"] --> Create["start workflow + establish inbox"]
+ Create --> Wait["wait before initialization"]
+ Send["send message"] --> Channel["onMessage"]
+ Channel --> Wait
+ Wait --> Init["normal turn step: initialize + turn_0"]
+ Init --> Stream["continuous session stream"]
+```
+
+## Acceptance and retries
+
+UI `ready` means the composer accepts input, not that the inbox is already registered. A send
+waits for an in-flight create response, then posts immediately without a stream-event barrier.
+An unclaimed inbox on a pending or running workflow returns `409 session_not_ready`. The client
+retries sends with bounded exponential backoff for up to 20 seconds, respecting cancellation.
+Unknown and terminal sessions return `session_not_active` without retries or replacement.
+
+The store consumes one session stream across turns. Resume uses its initial durable tail index
+to finish catch-up. Disconnects and renewable leases reconnect from the cursor; reset and unmount
+close the local transport without cancelling durable execution. External sessions are never
+replaced automatically. Strict Mode effect replay does not create duplicate sessions.
+
+## Boundaries
+
+Message-free creation is conversation-only and rejects turn-scoped client context, output schemas,
+callbacks, and activity observers. Audience and the session timeout are selected at creation.
+The timeout also bounds unused prewarmed sessions. No activation/reservation API or special proxy
+binding protocol is introduced. Controls that require history do not initialize an empty session.