Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

refactor(eve): remove child task_update progress callbacks (#3261)

Ruib27e9f285c1c
diff --git a/.changeset/remove-child-task-update.md b/.changeset/remove-child-task-update.md
new file mode 100644
index 000000000..6911cc0db
--- /dev/null
+++ b/.changeset/remove-child-task-update.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Remove the `task_update` tool and its child-to-parent progress callbacks; use the child session's stream to follow progress. Successful results from overlapping background tasks now reach the parent together across launch turns, after child settlement updates usage and handles; user input, failures, and cancellation remain responsive.
diff --git a/.changeset/report-content-filter.md b/.changeset/report-content-filter.md
new file mode 100644
index 000000000..3eef72e4f
--- /dev/null
+++ b/.changeset/report-content-filter.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Report content-filtered model responses distinctly, including their finish reason and Gateway generation ID when available, instead of retrying them as empty responses. Filtered partial text is no longer delivered as a completed reply.
diff --git a/.changeset/tui-idle-background-completions.md b/.changeset/tui-idle-background-completions.md
new file mode 100644
index 000000000..8c7a19213
--- /dev/null
+++ b/.changeset/tui-idle-background-completions.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Fix `eve dev` silently stopping session updates during long periods of background work. The terminal now keeps listening while the prompt is open, so completion reports appear without another user message.
diff --git a/apps/docs/registry.json b/apps/docs/registry.json
index ffa985919..3ccc24765 100644
--- a/apps/docs/registry.json
+++ b/apps/docs/registry.json
@@ -427,19 +427,6 @@
}
]
},
- {
- "name": "tool/task_update",
- "type": "registry:item",
- "title": "Task Update",
- "description": "Report background-task progress to the parent agent.",
- "files": [
- {
- "path": "registry/tools/task_update.ts",
- "type": "registry:file",
- "target": "agent/tools/task_update.ts"
- }
- ]
- },
{
"name": "tool/todo",
"type": "registry:item",
diff --git a/apps/docs/registry/tools/task_update.ts b/apps/docs/registry/tools/task_update.ts
deleted file mode 100644
index 6dbdff314..000000000
--- a/apps/docs/registry/tools/task_update.ts
+++ /dev/null
@@ -1 +0,0 @@
-export { default } from "eve/tools/task_update";
diff --git a/docs/concepts/built-in-tools.md b/docs/concepts/built-in-tools.md
index 01eb2aa06..b90fe958a 100644
--- a/docs/concepts/built-in-tools.md
+++ b/docs/concepts/built-in-tools.md
@@ -319,26 +319,6 @@ import { disableTool } from "eve/tools";
export default disableTool();
```
-### `task_update`
-
-`task_update` lets a background task report progress to its parent. It appears only in delegated task sessions.
-
-```sh
-eve add tool/task_update
-```
-
-```ts title="agent/tools/task_update.ts"
-export { default } from "eve/tools/task_update";
-```
-
-The framework behavior cannot be overridden. Re-export the definition above to restore it, or disable it:
-
-```ts title="agent/tools/task_update.ts"
-import { disableTool } from "eve/tools";
-
-export default disableTool();
-```
-
### `load_skill`
`load_skill` pulls an on-demand [skill](../skills)'s instructions into the current turn. It appears only when the agent declares skills and adds no execution surface by itself.
diff --git a/docs/concepts/sessions-runs-and-streaming.md b/docs/concepts/sessions-runs-and-streaming.md
index e3f851309..632f17194 100644
--- a/docs/concepts/sessions-runs-and-streaming.md
+++ b/docs/concepts/sessions-runs-and-streaming.md
@@ -97,10 +97,18 @@ Note: consider the privacy, confidentiality, and user-experience implications fo
When a task explicitly requires conditional delivery and there is nothing new to report, the agent can finish with exactly `<eve-empty-delivery/>`. eve emits `message.completed` with `message: null` for that intentional silence. The marker must be the entire response, apart from surrounding whitespace; the HTML-escaped form `&lt;eve-empty-delivery/&gt;` is also accepted. A response that quotes the marker in prose or code is delivered normally.
-A delegated subagent publishes progress on its own child-session stream. The parent emits `subagent.called` with a `childSessionId`, which a client uses to attach. `subagent.completed` carries a working task receipt after admission; later updates and outcomes arrive as task-triggered `message.received` notifications.
+A delegated subagent publishes progress on its own child-session stream. The parent emits `subagent.called` with a `childSessionId`, which a client uses to attach. `subagent.completed` carries a working task receipt after admission; terminal outcomes arrive as task-triggered `message.received` notifications.
`step.failed` and `turn.failed` carry `{ code, message, details? }` for the failed fragment or turn, and `session.failed` is the terminal session-level variant. `turn.cancelled` is not a failure: the cancelled turn ends without any failure event, `session.waiting` follows, and the session accepts the next message normally. Whatever the turn streamed before cancellation stays on the stream. Durable history keeps the accepted user input and previously settled work, but discards incomplete assistant output and unfinished tool state. When a turn requested an output schema, the finalized payload lands on `result.completed` as `data.result` before the turn boundary. `authorization.required` carries the sign-in challenge (`data.authorization` may include `url`, `userCode`, `expiresAt`, `instructions`), and `authorization.completed` carries `data.outcome` (`"authorized" | "declined" | "failed" | "timed-out"`).
+A provider response ending with `content-filter` fails with `MODEL_CALL_FAILED`,
+`details.semanticErrorId: "model-response-content-filtered"`, and
+`details.finishReason: "content-filter"`. Details also include the Gateway
+`generationId` when available. eve does not retry the filtered response or emit
+`message.completed` for its partial text; deltas already streamed remain visible.
+Conversation sessions wait for another user message, while task-mode runs return
+a failed result.
+
## The event envelope
Alongside `type` and `data`, every event carries a `meta` envelope:
diff --git a/docs/guides/frontend/overview.mdx b/docs/guides/frontend/overview.mdx
index 189767ada..ec4850908 100644
--- a/docs/guides/frontend/overview.mdx
+++ b/docs/guides/frontend/overview.mdx
@@ -87,7 +87,7 @@ Most chat UIs only need `data.messages` and `status`. Drop down to `events` when
`data.messages` are eve-owned `EveMessage[]`. Common text, reasoning, file, and dynamic-tool parts follow the [AI SDK `UIMessage`](https://ai-sdk.dev/docs/reference/ai-sdk-core/ui-message) rendering convention, but the types are not interchangeable. eve also exposes authorization and HITL metadata, and a file part's URL can be absent. Adapt those parts before passing messages to an API typed as `UIMessage[]`.
-When the root agent delegates, its stream emits `subagent.called` with the child's `childSessionId`, then `subagent.completed` after admission with a working task receipt. Later task notifications wake the parent with updates or the final result. Detailed child progress lives on the child session's stream instead of being flattened into the root `data.messages`. Use the lower-level [TypeScript client](../client/overview#sessions) to attach to that ID when your UI needs live subagent activity. See [What the parent sees](../../subagents#what-the-parent-sees) for the complete contract.
+When the root agent delegates, its stream emits `subagent.called` with the child's `childSessionId`, then `subagent.completed` after admission with a working task receipt. Later task notifications wake the parent with completion, failure, or cancellation. Detailed child progress lives on the child session's stream instead of being flattened into the root `data.messages`. Use the lower-level [TypeScript client](../client/overview#sessions) to attach to that ID when your UI needs live subagent activity. See [What the parent sees](../../subagents#what-the-parent-sees) for the complete contract.
## Sending and streaming
diff --git a/docs/subagents/index.mdx b/docs/subagents/index.mdx
index 0703b8be6..f70f8949e 100644
--- a/docs/subagents/index.mdx
+++ b/docs/subagents/index.mdx
@@ -7,13 +7,18 @@ eve supports two ways to delegate work: the root-only built-in `agent` tool, whi
## Completion batching
-eve automatically combines adjacent successful completions already queued from
-tasks launched in the same parent turn. Every result is retained, and the parent
-responds to the batch in one turn. No configuration is required.
-
-There is no debounce timer or waiting for unfinished tasks. Failures, input
-requests, explicit updates, user messages, and different task cohorts remain
-separate delivery boundaries.
+Overlapping background work owned by a session forms a cohort. Tasks launched in
+later user turns join the open cohort while earlier work is still pending or its
+successful results await delivery. eve holds successful completion notifications
+until every task in that cohort has completed, failed, or been cancelled, then
+delivers the successful results together in one parent turn. Partial completions
+do not invoke the parent model. Work started after the cohort settles forms a new
+cohort. No configuration or debounce timer is required.
+
+User messages, input requests, authorization events, failures, and cancellation
+are handled without waiting for the cohort. Child lifecycle events are processed
+before completion delivery so usage accounting and handle cleanup stay ordered.
+To follow work in progress, subscribe to the child session streams.
## The built-in `agent` tool
@@ -27,7 +32,7 @@ The root session receives `agent` by default. The model calls it to delegate a t
}
```
-The copy uses the root's instructions, connections, auth, and sandbox. It receives the same tools except for the root-only `agent` and `Workflow`, and starts with fresh conversation history and fresh state. Its file writes are immediately visible to the root. The built-in `agent` always runs in the background and needs no configuration: each call returns `{ status: "working", taskId, agentId }`, then task notifications wake the parent with updates or the final result. Give parallel children non-overlapping write scopes.
+The copy uses the root's instructions, connections, auth, and sandbox. It receives the same tools except for the root-only `agent` and `Workflow`, and starts with fresh conversation history and fresh state. Its file writes are immediately visible to the root. The built-in `agent` always runs in the background and needs no configuration: each call returns `{ status: "working", taskId, agentId }`, then task notifications wake the parent with completion, failure, or cancellation. Give parallel children non-overlapping write scopes.
`agent` is intentionally root-only. Copies created by it cannot call `agent`, and declared subagents never receive the built-in tool. If a stale or forced recursive call reaches execution, eve rejects it instead of starting another child session.
@@ -56,9 +61,9 @@ export default defineAgent({
});
```
-`description` is required. The parent reads it to decide whether to delegate, so the compiler rejects any subagent whose `agent.ts` leaves it out. Every declared local or remote subagent runs as a durable background task: the call returns `{ status: "working", taskId, agentId }` immediately, then task notifications wake the parent with updates, completion, failure, or cancellation. Human input requests surface separately on the parent session.
+`description` is required. The parent reads it to decide whether to delegate, so the compiler rejects any subagent whose `agent.ts` leaves it out. Every declared local or remote subagent runs as a durable background task: the call returns `{ status: "working", taskId, agentId }` immediately, then task notifications wake the parent with completion, failure, or cancellation. Human input requests surface separately on the parent session.
-A child running as a background task receives the framework `task_update` tool. It uses `task_update` to report progress to its parent. `task_cancel` is available to sessions that own background tasks, including a child that starts nested background work.
+`task_cancel` is available to sessions that own background tasks, including a child that starts nested background work.
A mounted extension can also contribute declared subagents from `extension/subagents/`. The mount namespace prefixes the subagent visible to the consuming agent node: mounting an extension as `crm` exposes its `reviewer` subagent as `crm__reviewer`. The contributed subagent keeps its own isolated tools, connections, skills, hooks, instructions, sandbox, and nested subagents, and its modules can read configuration from the extension handle. See [Extensions](./extensions#add-a-subagent) for the authoring and override behavior.
diff --git a/e2e/fixtures/agent-compaction-regressions/agent/instructions.md b/e2e/fixtures/agent-compaction-regressions/agent/instructions.md
index b296ae3dc..ca03e53fe 100644
--- a/e2e/fixtures/agent-compaction-regressions/agent/instructions.md
+++ b/e2e/fixtures/agent-compaction-regressions/agent/instructions.md
@@ -1,14 +1,13 @@
-# Compaction regression fixture
+You help Alice and Bob review a small reading-list application and prepare handoff notes.
-Follow the user's requested regression case exactly.
+Each review and handoff step is performed once. A tool result with
+`completed: true` or a `completionMarker` is its completion receipt. Use that
+recorded work when preparing the final report, including after a conversation
+summary.
-When a tool reports `completed: true` or returns a `completionMarker`, that work
-is complete. Call that tool no more than once. Report the marker without
-restarting the work after context compaction.
+Bob maintains the shared checklist separately. An entry can still say pending
+while Alice's review is complete. Use the completed findings for the handoff
+rather than repeating the review while Bob updates the checklist.
-Completed evidence takes precedence over a pending todo. If a compacted summary
-says a work unit is complete but the todo still shows it as pending, do not run
-the work again.
-
-If a tool returns `hardStop: true`, call no more tools and report its
-`completionMarker` immediately.
+A result with `hardStop: true` ends the scheduled review. Provide the final
+report with its `completionMarker` rather than starting another step.
diff --git a/e2e/fixtures/agent-compaction-regressions/agent/tools/advance-checkpoint.ts b/e2e/fixtures/agent-compaction-regressions/agent/tools/advance-checkpoint.ts
index 59331a082..1f9b09246 100644
--- a/e2e/fixtures/agent-compaction-regressions/agent/tools/advance-checkpoint.ts
+++ b/e2e/fixtures/agent-compaction-regressions/agent/tools/advance-checkpoint.ts
@@ -22,16 +22,16 @@ export default defineTool({
regressionCase: input.regressionCase,
attempt,
handoffNotes: [
- "Alice has finished the initial review of the online shop. Bob will use these notes to prepare the team handoff. The completed review remains the source for the findings; preparing the handoff does not require another inspection.",
- "The catalog, cart, and checkout form the main customer journey. The handoff groups the findings in that order so Bob can follow a product from the catalog through the cart and into a saved order without jumping between unrelated topics.",
- "Catalog notes cover product identifiers, names, prices, and availability. Bob can use that section to explain which values the cart displays and which values checkout reads again when it calculates the final order total.",
- "Cart notes cover adding items, changing quantities, and removing the last item. An empty cart is an ordinary state. The handoff distinguishes an empty cart from an invalid quantity so the examples remain clear to the next reviewer.",
- "Checkout notes separate the subtotal from delivery charges and describe amounts in cents. The recorded order contains both the submitted items and the calculated total, giving the confirmation page a consistent saved result to display.",
- "The confirmation section explains that the saved order supplies the displayed quantities, prices, and delivery details. Notification delivery is a separate follow-up to saving the order, so a delayed notification does not mean the purchase was lost.",
- "Inventory and address validation are listed with the checkout findings. When a customer needs to correct either one, the cart remains available. Bob can refer to these notes when describing the expected correction flow to the team.",
- "The order history section records how stored orders are listed and paginated. The cancellation section records how repeated cancellation requests use the saved order status. These are existing review findings, not additional work requests.",
- "Bob maintains the shared task list independently of Alice's review. A pending review entry means that the list has not been updated yet. The successful tool result records that the review itself is complete, and Bob will reconcile the list during handoff.",
- "The handoff notes are now recorded and the checkpoint marker identifies this completed step. The remaining response should report the review completion marker and this checkpoint marker so Alice and Bob can locate both completed results.",
+ "Alice has finished the initial review of the reading-list application. Bob will use these completed findings for the team handoff. The review and the handoff are separate steps, each with its own saved reference.",
+ "The catalog, reading lists, and preview pages form the main sequence. The handoff follows that order so Bob can describe how a book is found in the catalog, added to a list, and shown in a saved preview.",
+ "Catalog notes cover book identifiers, titles, authors, and shelf labels. Bob can use that section to explain which details appear in a search result and which details the preview looks up for a selected book.",
+ "Reading-list notes cover adding books, arranging their order, and keeping an empty list as a starting point. The examples show how a saved selection can be opened again with the same titles in the same order.",
+ "Preview notes describe the list title, optional description, and total book count. Each book entry shows its title and author together, followed by the shelf label. The recorded selection supplies the order of those entries.",
+ "The print-view section explains how a saved list becomes a plain page for the library noticeboard. Opening that page leaves the reading list unchanged. Bob can return to the regular preview after preparing the printed copy.",
+ "Display preferences cover page size and the compact or spacious layout. An optional description can remain blank. Bob can refer to these examples when explaining how the same list appears in its preview and printable forms.",
+ "The saved-lists section describes recent lists and next-page links. The archive section describes how an archived list remains available on its own page. These paragraphs summarize the modules that Alice has already reviewed.",
+ "Bob maintains the shared checklist separately from Alice's review. Its pending entry records an earlier checklist state, while the completed findings record the finished review. Bob will bring the checklist up to date during handoff.",
+ "The handoff notes are saved, and the checkpoint marker is their reference. Alice and Bob can identify the two completed steps from the review completion marker and the checkpoint marker included in the final report.",
],
};
},
diff --git a/e2e/fixtures/agent-compaction-regressions/agent/tools/emit-compaction-content.ts b/e2e/fixtures/agent-compaction-regressions/agent/tools/emit-compaction-content.ts
index 7d085b3af..180a058f0 100644
--- a/e2e/fixtures/agent-compaction-regressions/agent/tools/emit-compaction-content.ts
+++ b/e2e/fixtures/agent-compaction-regressions/agent/tools/emit-compaction-content.ts
@@ -15,8 +15,7 @@ const INLINE_FILE_BASE64 = `${"A".repeat(400)}${CONTENT_OUTPUT_PAYLOAD_CANARY}${
export default defineTool({
description:
- "Compaction regression tool. Emits text, a large inline file, and trailing text the " +
- "checkpoint must preserve.",
+ "Collect Alice's review note, its attachment, and the closing note for Bob's reading-list handoff.",
inputSchema: z.object({}),
async execute() {
return { completed: true };
@@ -26,7 +25,7 @@ export default defineTool({
// The lead marker is spelled ONLY here — never in the tail's preserve
// instructions — so it can reach the checkpoint solely through this
// part's own rendering.
- toolOutputPart.text(`Work log (preserve this marker): ${CONTENT_OUTPUT_LEAD_MARKER}`),
+ toolOutputPart.text(`Alice's completed review note: ${CONTENT_OUTPUT_LEAD_MARKER}`),
toolOutputPart.file(INLINE_FILE_BASE64, {
filename: CONTENT_OUTPUT_FILENAME,
mediaType: "application/octet-stream",
@@ -34,9 +33,9 @@ export default defineTool({
// "The attachment's filename" is deliberately not spelled out: the
// checkpoint can only carry it by reading the rendered stub.
toolOutputPart.text(
- "Completed content-output work. Preserve every CONTENT_OUTPUT_TEXT_* marker visible " +
- "in this conversation and the attachment's exact filename. " +
- `Preserve this exact marker: ${CONTENT_OUTPUT_TAIL_MARKER}`,
+ "Bob's handoff uses the note references labelled CONTENT_OUTPUT_TEXT_* " +
+ "and the attachment's filename. " +
+ `The closing note reference is ${CONTENT_OUTPUT_TAIL_MARKER}`,
),
]);
},
diff --git a/e2e/fixtures/agent-compaction-regressions/agent/tools/inspect-repository.ts b/e2e/fixtures/agent-compaction-regressions/agent/tools/inspect-repository.ts
index be1372094..85f191741 100644
--- a/e2e/fixtures/agent-compaction-regressions/agent/tools/inspect-repository.ts
+++ b/e2e/fixtures/agent-compaction-regressions/agent/tools/inspect-repository.ts
@@ -7,7 +7,7 @@ const invocationCount = defineState("compaction-regression.inspect-repository",
export default defineTool({
description:
- "Inspect the online shop repository and return an overview of its modules for the review handoff.",
+ "Review the reading-list repository and return an overview of its modules for Alice and Bob's handoff.",
inputSchema: z.object({
scope: z.literal("repository"),
}),
@@ -22,18 +22,18 @@ export default defineTool({
hardStop: attempt >= 10,
attempt,
findings: [
- "The repository contains a small online shop. The application entry point loads configuration, constructs the request router, and connects the catalog, cart, checkout, and order history handlers to their repositories.",
- "The catalog module owns product records with identifiers, display names, unit prices, and availability flags. Its repository provides product lookup and filtered listing, returning an empty list when a filter has no matching records.",
- "The cart module stores product identifiers and quantities for each customer cart. Its handlers validate positive integer quantities, combine additions for an existing product, and allow the final item to be removed without invalidating the cart.",
- "The checkout module reads current product prices and availability before calculating the order. It combines item quantities with prices represented in cents, adds delivery charges, and passes the resulting items and total to the order repository.",
- "The order repository stores an order and its submitted items together. It returns the generated order identifier after persistence completes. The confirmation handler then reads the saved record instead of reconstructing the order from the cart.",
- "The confirmation templates display product names, quantities, item prices, and delivery details from the saved order. Product names are rendered as text. The templates share formatting helpers for currency values and postal addresses.",
- "The inventory service checks whether requested quantities are available before checkout accepts an order. When availability has changed, the handler explains which cart entry needs attention and keeps the other entries available for correction.",
- "The address validation module requires a recipient, street, city, and postal code. An additional address line is optional. The same validated address shape flows through checkout, persistence, and order confirmation rendering.",
- "The notification adapter receives an order identifier after the order is saved. It records delivery outcomes separately from order creation, allowing the application to report a delayed confirmation without creating the purchase again.",
- "The order history module reads saved orders in reverse creation order. Its repository uses bounded page sizes and a continuation cursor. The response includes the saved order status, item summary, and total for each returned entry.",
- "The cancellation handler reads the saved order status before applying an update. An already cancelled order returns its existing state. Inventory adjustments are associated with the first successful cancellation rather than every repeated request.",
- "The repository overview is complete and ready for Alice's handoff to Bob. The modules above cover the requested customer journey, including ordinary empty states, validation failures, saved results, and follow-up notifications.",
+ "The repository contains a small reading-list application. Its entry point loads the display settings and connects the book catalog, reading lists, previews, and saved-lists page to their repositories.",
+ "The catalog module owns book records with identifiers, titles, authors, and shelf labels. Its repository supports title searches and author filters, returning an empty list when a search has no matching records.",
+ "The reading-list module stores selected book identifiers in a chosen order. Adding a book already on the list keeps its existing position. A list with no books is a normal starting point for a new selection.",
+ "The preview module reads current titles and shelf labels from the catalog. It combines those details with the saved list order, then gives the template the list title, description, and selected books.",
+ "The list repository saves the list title and selected books together. It returns a list reference after saving. The preview page then reads that saved record so it displays the same selection when reopened.",
+ "Preview templates display book titles and author names as plain text. Each entry includes its shelf label on a separate line. The templates share simple helpers for spacing, headings, and the total book count.",
+ "The catalog helper checks that the selected books still have catalog entries. If a title has moved, the preview shows a note beside that entry and leaves the reading list available for a later update.",
+ "The preferences module stores the page size and the choice of a compact or spacious layout. A list description is optional. These settings are used consistently on the preview page and the printable version.",
+ "The print-view helper receives a saved list reference and prepares a plain page with the same titles. Opening that view leaves the saved reading list unchanged, making it easy to return to the regular preview.",
+ "The saved-lists page displays recently prepared lists first. Its repository uses bounded page sizes and a next-page reference. Each entry shows the saved title, book count, and a link to open its preview.",
+ "The archive action reads a list's current state before updating it. Selecting the action again leaves an already archived list in the same state. The list can still be found through the archive page.",
+ "Alice's repository overview is complete and ready for Bob's handoff. These notes cover finding books, arranging a reading list, viewing saved results, choosing display settings, and preparing a printable copy.",
],
};
},
diff --git a/e2e/fixtures/agent-compaction-regressions/agent/tools/perform-source-analysis.ts b/e2e/fixtures/agent-compaction-regressions/agent/tools/perform-source-analysis.ts
index fda805b13..bffa8a795 100644
--- a/e2e/fixtures/agent-compaction-regressions/agent/tools/perform-source-analysis.ts
+++ b/e2e/fixtures/agent-compaction-regressions/agent/tools/perform-source-analysis.ts
@@ -8,7 +8,7 @@ const invocationCount = defineState("compaction-regression.perform-source-analys
export default defineTool({
description:
- "Compaction regression tool. Complete source analysis exactly once when the user requests the stale-todo-work case.",
+ "Review the reading-list application's modules and return Alice's completed findings for Bob's handoff.",
inputSchema: z.object({
approach: z.string().min(1),
}),
@@ -30,18 +30,18 @@ export default defineTool({
attempt,
approach: input.approach,
findings: [
- "The application entry point creates the request router and registers the catalog, cart, and checkout endpoints. Configuration is loaded before the first request is accepted.",
- "Catalog records contain a product identifier, display name, unit price, and availability flag. The repository returns an empty list when the catalog has no matching entries.",
- "Cart quantities are validated as positive integers. Adding an existing product updates its quantity, while removing the final item leaves a valid empty cart.",
- "Checkout calculates its total from the current catalog prices and cart quantities. Delivery charges are added after the subtotal, and currency values are represented in cents.",
- "The order repository stores the submitted items and total together. A generated order identifier is returned only after the write completes successfully.",
- "Order confirmation rendering uses the saved order data. Product names are rendered as text, and the template includes a summary of quantities, prices, and delivery details.",
- "The inventory service checks availability before accepting an order. If a product is unavailable, checkout returns an explanation and preserves the cart for correction.",
- "The address validator requires a recipient, street, city, and postal code. Optional address lines remain optional throughout validation and persistence.",
- "The notification adapter receives the order identifier after persistence. Delivery failures are recorded separately so they do not create a second order or discard the first one.",
- "The order history endpoint reads stored orders in reverse creation order. Pagination uses a bounded page size and returns a continuation cursor when more results are available.",
- "The cancellation handler checks the saved order status before updating it. Repeating a cancellation returns the existing cancelled state without repeating inventory adjustments.",
- "The reviewed modules cover the requested source analysis. The task list entry has not yet been updated, but no further source inspection is needed to report these findings.",
+ "The application entry point loads the display settings and connects the book catalog, reading lists, and preview pages. The catalog is ready before the first page is shown.",
+ "Catalog records contain a book identifier, title, author, and shelf label. A search with no matching books returns an empty list and a friendly note beside the search box.",
+ "Reading lists store book identifiers in the order chosen by the reader. Adding a book already on the list keeps its existing position, while an empty list remains a normal starting point.",
+ "The list preview combines the saved book identifiers with titles from the catalog. It shows the title and author together, followed by the shelf label on the next line.",
+ "The list repository saves the list title and its selected books together. It returns a list reference after saving, allowing the preview page to show the recorded selection.",
+ "Preview templates use the saved list data. Book titles appear as plain text, and the template includes a short count of the books followed by their ordered entries.",
+ "The catalog helper checks whether a selected book is still listed before preparing a preview. If a title has moved, the page keeps the reading list available for a later update.",
+ "Display preferences include the page size and the choice of a compact or spacious layout. Optional descriptions can remain blank when the reader prefers a simple list of titles.",
+ "The print-view helper receives the saved list reference. Opening the print view leaves the list unchanged, so the reader can return to the same selection afterward.",
+ "The saved-lists page shows recently prepared lists first. It uses a bounded page size and provides a next-page link when there are more lists to display.",
+ "The archive action reads the saved list state before updating it. A list already in the archive remains there when the same action is selected again.",
+ "Alice's source review is complete and these findings are ready for Bob. The shared checklist still shows the earlier pending entry because Bob updates it separately during the handoff.",
],
};
},
diff --git a/e2e/fixtures/agent-compaction-regressions/evals/content-output-file-stub.eval.ts b/e2e/fixtures/agent-compaction-regressions/evals/content-output-file-stub.eval.ts
index 2d3e5a8c4..18662c5ec 100644
--- a/e2e/fixtures/agent-compaction-regressions/evals/content-output-file-stub.eval.ts
+++ b/e2e/fixtures/agent-compaction-regressions/evals/content-output-file-stub.eval.ts
@@ -31,8 +31,8 @@ export default defineEval({
const turn = await t.send(
[
"[case: content-output-file-stub]",
- "Call emit-compaction-content exactly once.",
- "After compaction, report whether its completion evidence survived.",
+ "Alice is preparing a reading-list handoff for Bob. Please collect one review note and its attachment with emit-compaction-content.",
+ "After the conversation is summarized, confirm that the completed note and its attachment reference are still available.",
].join("\n"),
);
diff --git a/e2e/fixtures/agent-compaction-regressions/evals/redundant-tool-calls.eval.ts b/e2e/fixtures/agent-compaction-regressions/evals/redundant-tool-calls.eval.ts
index a12285542..717b21863 100644
--- a/e2e/fixtures/agent-compaction-regressions/evals/redundant-tool-calls.eval.ts
+++ b/e2e/fixtures/agent-compaction-regressions/evals/redundant-tool-calls.eval.ts
@@ -9,9 +9,9 @@ export default defineEval({
const turn = await t.send(
[
"[case: redundant-tool-calls]",
- "Alice is reviewing a small catalog application and preparing a handoff for Bob.",
- "Run inspect-repository once with scope repository, then use advance-checkpoint to record the handoff notes.",
- "Report REPOSITORY_INSPECTION_COMPLETE and the returned checkpoint marker. Carry the completed inspection into the handoff without repeating it.",
+ "Alice is reviewing a small reading-list application for the library and preparing a handoff for Bob.",
+ "Please complete one review with inspect-repository using scope repository, then use advance-checkpoint to record the handoff notes.",
+ "Use the completed findings for the handoff. Include REPOSITORY_INSPECTION_COMPLETE and the returned checkpoint marker in the final report so Alice and Bob can find both records.",
].join("\n"),
);
diff --git a/e2e/fixtures/agent-compaction-regressions/evals/stale-todo-work.eval.ts b/e2e/fixtures/agent-compaction-regressions/evals/stale-todo-work.eval.ts
index bc4b2ab1f..286e4b933 100644
--- a/e2e/fixtures/agent-compaction-regressions/evals/stale-todo-work.eval.ts
+++ b/e2e/fixtures/agent-compaction-regressions/evals/stale-todo-work.eval.ts
@@ -9,10 +9,10 @@ export default defineEval({
const turn = await t.send(
[
"[case: stale-todo-work]",
- "Alice and Bob are reviewing a small catalog application.",
- "Use perform-source-analysis once with approach initial to prepare Alice's findings.",
- "Bob updates their shared checklist separately, so it may still say pending after the analysis is complete.",
- "Then use advance-checkpoint to record the handoff notes. Report SOURCE_ANALYSIS_COMPLETE and the returned checkpoint marker. The completed analysis does not need to be repeated while Bob updates the list.",
+ "Alice and Bob are reviewing a small reading-list application for their library.",
+ "Alice needs one source review. Please prepare her findings with perform-source-analysis using approach initial.",
+ "Bob updates their shared checklist separately, so it may still say pending after Alice's review is complete.",
+ "Please use advance-checkpoint to record the handoff from those completed findings. Include SOURCE_ANALYSIS_COMPLETE and the returned checkpoint marker in the final report so Alice and Bob can find both records.",
].join("\n"),
);
diff --git a/e2e/fixtures/agent-compaction-regressions/evals/task-survival.eval.ts b/e2e/fixtures/agent-compaction-regressions/evals/task-survival.eval.ts
index 5cb913c13..4688615aa 100644
--- a/e2e/fixtures/agent-compaction-regressions/evals/task-survival.eval.ts
+++ b/e2e/fixtures/agent-compaction-regressions/evals/task-survival.eval.ts
@@ -9,7 +9,7 @@ export default defineEval({
const turn = await t.send(
[
"[case: task-survival]",
- "Inspect the repository, then confirm the requirements for the change.",
+ "Alice is updating the library's reading-list application. Please review the repository and confirm these requirements for Bob's handoff.",
// Sized to a narrow window: long enough that a 280-char summarizer
// input cap would destroy the tail sentinel, but short enough that
// the task alone cannot cross the fixture's ~640-token threshold —
diff --git a/e2e/fixtures/agent-prompt-cache/package.json b/e2e/fixtures/agent-prompt-cache/package.json
index bf39a7993..f51440c8e 100644
--- a/e2e/fixtures/agent-prompt-cache/package.json
+++ b/e2e/fixtures/agent-prompt-cache/package.json
@@ -21,6 +21,6 @@
"typescript": "catalog:"
},
"e2e": {
- "modelMatrix": "full"
+ "modelMatrix": "default"
}
}
diff --git a/e2e/fixtures/agent-task-reporting/agent/instructions.md b/e2e/fixtures/agent-task-reporting/agent/instructions.md
index bce87b044..32f06789d 100644
--- a/e2e/fixtures/agent-task-reporting/agent/instructions.md
+++ b/e2e/fixtures/agent-task-reporting/agent/instructions.md
@@ -1,3 +1,17 @@
-# Identity
+You help Alice and Bob prepare an inventory checklist.
-You are a helpful assistant.
+When coordinating the full checklist, use the built-in `agent` tool to assign
+each entry to a separate background assistant. All three entries use `agent`,
+including the third entry. The coordinator collects their results rather than
+performing the lookups itself.
+
+When you receive an assignment for one entry, handle that entry only. For
+`check=first` or `check=second`, use `probe`. For `check=third`, use
+`warehouse_lookup`; that workflow contacts `warehouse-worker` and returns its
+answer. The specialist is a nested part of the third assistant's work, not a
+replacement for that assistant.
+
+Keep assignments brief and process-oriented, including the check reference and
+lookup tool. Return the item from the lookup. Use the three returned items for
+the coordinator's final checklist, and answer separate user questions normally
+while the lookups are pending.
diff --git a/e2e/fixtures/agent-task-reporting/agent/subagents/warehouse-worker/agent.ts b/e2e/fixtures/agent-task-reporting/agent/subagents/warehouse-worker/agent.ts
new file mode 100644
index 000000000..344de53a2
--- /dev/null
+++ b/e2e/fixtures/agent-task-reporting/agent/subagents/warehouse-worker/agent.ts
@@ -0,0 +1,8 @@
+import { e2eSubagentConfig } from "@eve-e2e/config";
+import { defineAgent } from "eve";
+
+export default defineAgent({
+ ...e2eSubagentConfig(),
+ description:
+ "Nested specialist contacted by warehouse_lookup to find the third entry's inventory item. The checklist coordinator assigns all entries through agent instead.",
+});
diff --git a/e2e/fixtures/agent-task-reporting/agent/subagents/warehouse-worker/tools/probe.ts b/e2e/fixtures/agent-task-reporting/agent/subagents/warehouse-worker/tools/probe.ts
new file mode 100644
index 000000000..819cb200d
--- /dev/null
+++ b/e2e/fixtures/agent-task-reporting/agent/subagents/warehouse-worker/tools/probe.ts
@@ -0,0 +1 @@
+export { default } from "../../../tools/probe.js";
diff --git a/e2e/fixtures/agent-task-reporting/agent/tools/probe.ts b/e2e/fixtures/agent-task-reporting/agent/tools/probe.ts
index 2e948fe9d..6a1a64785 100644
--- a/e2e/fixtures/agent-task-reporting/agent/tools/probe.ts
+++ b/e2e/fixtures/agent-task-reporting/agent/tools/probe.ts
@@ -1,20 +1,14 @@
import { defineTool } from "eve/tools";
+import { once } from "eve/tools/approval";
import { z } from "zod";
-const PROBES = {
- first: { delayMs: 10_000, result: "oranges" },
- second: { delayMs: 40_000, result: "pears" },
- third: { delayMs: 70_000, result: "apples" },
-} as const;
+const PROBES = { first: "oranges", second: "pears", third: "apples" } as const;
export default defineTool({
- description: "Look up the inventory item at one sample warehouse after its configured delay.",
+ description: "Look up the warehouse inventory item for a checklist entry.",
inputSchema: z.strictObject({
check: z.enum(["first", "second", "third"]),
}),
- async execute({ check }) {
- const probe = PROBES[check];
- await new Promise((resolve) => setTimeout(resolve, probe.delayMs));
- return { result: probe.result };
- },
+ approval: once(),
+ execute: ({ check }) => ({ result: PROBES[check] }),
});
diff --git a/e2e/fixtures/agent-task-reporting/agent/tools/warehouse_lookup.ts b/e2e/fixtures/agent-task-reporting/agent/tools/warehouse_lookup.ts
new file mode 100644
index 000000000..79c190caa
--- /dev/null
+++ b/e2e/fixtures/agent-task-reporting/agent/tools/warehouse_lookup.ts
@@ -0,0 +1,14 @@
+import { defineWorkflowTool } from "eve/tools";
+import { z } from "zod";
+
+export default defineWorkflowTool({
+ description: "Ask the warehouse specialist for the third inventory item and wait for its result.",
+ inputSchema: z.strictObject({ check: z.literal("third") }),
+ async execute({ check }, ctx) {
+ "use workflow";
+ // Blocking delegation keeps the outer task open until the nested lookup finishes.
+ return await ctx.agent("warehouse-worker", {
+ message: `Alice is preparing an inventory checklist for Bob's warehouse handoff. Your entry is check=${check}. Please use the inventory lookup tool (probe) for this entry and share the item it returns.`,
+ });
+ },
+});
diff --git a/e2e/fixtures/agent-task-reporting/evals/README.md b/e2e/fixtures/agent-task-reporting/evals/README.md
index 0a79bca28..094c454a5 100644
--- a/e2e/fixtures/agent-task-reporting/evals/README.md
+++ b/e2e/fixtures/agent-task-reporting/evals/README.md
@@ -1,42 +1,47 @@
# Background task reporting
-This fixture measures whether pending-cohort guidance suppresses a required user
-response after an intermediate task wake.
-
-`consolidated-report.eval.ts` runs eight reporting/compaction trials.
-`pending-response.eval.ts` runs 20 trials with the pending instruction and 20
-without it, interleaved. Each comparison trial
-starts the same three warehouse lookups, observes an intermediate completion,
-asks an unrelated arithmetic question, and waits for the combined inventory
-report. The comparison trials do not request compaction.
-
-The fixture model strips the trial marker from both variants before inference.
-The off variant also removes the current pending-cohort instruction. Task state,
-task results, launch guidance, settled guidance, model settings, and the runtime
-delivery policy are preserved. Both arms use the same 128,000-token fixture
-context budget and assert that no compaction occurred. Unit tests compare the requests and check that
-the original history is unchanged. This is a prompt ablation, not a simulation
-of cohort settlement or wake coalescing.
-
-Assertions record three outcomes separately:
-
-- Whether intermediate wakes produce user-facing text. This is a gate for the
- on variant and a recorded score with threshold zero for the off control.
-- Whether the user receives the arithmetic answer, without tools, before the
- last child turn completes. Server event timestamps establish ordering; seeing
- a reply before the final parent notification alone is insufficient.
-- Whether the final parent report includes all inventory results.
-
-A separate timing assertion requires the question to reach its parent turn before the
-last child completes. A trial that misses that window cannot establish a prompt
-effect. Provider errors, failed delegation, and missing intermediate wakes are
-also setup/runtime failures rather than evidence of unwanted silence.
-
-The variants share the same scenario and configured probe delays; stochastic
-model decisions and observed completion times can differ. Inspect per-trial
-outcomes and timing logs before attributing a difference to the instruction.
-Passing this small sample does not establish absence of interference in other
-conversations or models.
-
-Run the fixture's `test:unit` script for the request-transformation checks. The
-real-model comparison runs in the `agent-task-reporting` model-suite CI jobs.
+This fixture checks the full-cohort completion barrier with real models. Both
+retained evals carry `real-model`, so world suites exclude them. The root agent,
+its built-in agent copies, and the declared warehouse worker all resolve the CI
+model through `@eve-e2e/config`; the fixture retains its full model matrix.
+
+Alice is preparing an inventory checklist for Bob's warehouse handoff. Each trial
+launches exactly three independent warehouse tasks in one root turn, acknowledges
+the accepted assignments, and collects the results into one report for Bob.
+The test-only tool keeps its `probe` filename, but its model-facing description
+presents an inventory lookup without asking the model to arrange approval. Its
+`approval: once()` policy requires a runtime approval in each lookup session instead
+of delaying execution with a timer or relying on an `ask_question` exchange. The eval
+waits for all three approval requests, approves the first, and reads that child's
+completed result from its own session stream. The other two requests remain
+unapproved during the partial-state checks.
+
+The third child uses the blocking `warehouse_lookup` workflow tool to invoke a
+nested `warehouse-worker`. Blocking delegation keeps the outer task open until
+the nested lookup returns, rather than completing it with a background launch
+acknowledgement. Its approval is answered through the root session. The eval
+checks the nested session identity, real model selection, probe call, and completion
+before the outer child's result.
+
+`pending-response.eval.ts` retains 20 trials. After the first child completes,
+Alice asks an unrelated arithmetic question. The parent must answer before the
+last child completes, without consuming any held completion notifications. Server
+event timestamps and the unreleased approvals establish the partial-state window.
+Only that user answer and one final cohort report may invoke the parent model
+after setup.
+
+`consolidated-report.eval.ts` retains eight trials. It requests compaction in the
+same gated partial state, then releases the remaining lookups. Whether compaction
+produces a checkpoint or declines to summarize, the parent must retain the cohort
+and report every result. No ordinary parent model step may run before settlement.
+Successful checkpoint production remains a separately recorded soft assertion.
+
+Both drivers submit approval responses without awaiting a parent model turn.
+They retain the parent stream cursor from before the releases and inspect every
+subsequent boundary, so an intermediate model step fails even if it emits no text.
+The final notification must contain every original task ID exactly once, with no
+unknown IDs, and the single report must contain all three distinct inventory items.
+There are no prompt-ablation variants, inference delays, or mock silence responders.
+
+E2E execution is CI-only. Focused static checking uses
+`pnpm --filter agent-task-reporting exec tsc --noEmit`.
diff --git a/e2e/fixtures/agent-task-reporting/evals/consolidated-report.eval.ts b/e2e/fixtures/agent-task-reporting/evals/consolidated-report.eval.ts
index 23490794e..d655556da 100644
--- a/e2e/fixtures/agent-task-reporting/evals/consolidated-report.eval.ts
+++ b/e2e/fixtures/agent-task-reporting/evals/consolidated-report.eval.ts
@@ -1,9 +1,9 @@
import { defineEval } from "eve/evals";
-import { satisfies } from "eve/evals/expect";
+import { equals } from "eve/evals/expect";
import {
- completeReport,
- requireStreamIndex,
+ compactWhilePending,
+ modelSteps,
startWarehouseLookups,
waitForPartialCompletion,
waitForReport,
@@ -12,39 +12,16 @@ import {
function reportingEval() {
return defineEval({
description:
- "A stock eve agent acknowledges accepted background work and reports all results after compaction and task settlement.",
+ "A real parent retains a completed child's result across compaction while its siblings are gated, then reports the whole cohort including a nested lookup.",
tags: ["real-model"],
async test(t) {
const run = await startWarehouseLookups(t);
await waitForPartialCompletion(t, run);
-
- const compaction = t.target.watchTurn(run.sessionId, {
- startIndex: requireStreamIndex(run.session),
- });
- const response = await t.target.fetch(
- `/eve/v1/session/${encodeURIComponent(run.sessionId)}/compact`,
- {
- body: "{}",
- headers: { "content-type": "application/json" },
- method: "POST",
- },
- );
- await t.require(
- response.status,
- satisfies((status: number) => status === 202, "parent session accepts compaction"),
+ await compactWhilePending(t, run);
+ await waitForReport(t, run);
+ t.check(modelSteps(run.parentTurns), equals(1)).label(
+ "compaction does not admit partial successes; only the settled cohort invokes the parent model",
);
- const compactedTurn = await compaction.result();
- compactedTurn.event("compaction.requested", { count: 1 });
- // A declined summary preserves history; the caller still needs the complete report.
- compactedTurn
- .event("compaction.completed", { count: 1 })
- .soft()
- .label("successful checkpoint");
- compactedTurn.noFailedActions();
- run.session = compaction.session;
-
- const report = await waitForReport(t, run);
- await t.require(report, completeReport());
t.noFailedActions();
},
});
diff --git a/e2e/fixtures/agent-task-reporting/evals/event-matching.test.mjs b/e2e/fixtures/agent-task-reporting/evals/event-matching.test.mjs
new file mode 100644
index 000000000..4354a3ec5
--- /dev/null
+++ b/e2e/fixtures/agent-task-reporting/evals/event-matching.test.mjs
@@ -0,0 +1,264 @@
+import assert from "node:assert/strict";
+import { test } from "node:test";
+
+import {
+ childActivations,
+ checkForTask,
+ eventsForSession,
+ requireOriginalTasksHealthy,
+ toolEvidence,
+} from "./event-matching.ts";
+
+function event(id, type, data) {
+ return { meta: { id, at: "2026-09-11T00:00:00.000Z" }, type, data };
+}
+
+function approval(taskId, check, callId = "probe-call") {
+ return {
+ requestId: `${taskId}:approval-id`,
+ kind: "tool-approval",
+ action: { kind: "tool-call", toolName: "probe", callId, input: { check } },
+ };
+}
+
+function requested(id, callId = "probe-call", check = "first") {
+ return event(id, "actions.requested", {
+ actions: [{ kind: "tool-call", toolName: "probe", callId, input: { check } }],
+ turnId: "turn_0",
+ });
+}
+
+function completed(
+ id,
+ callId = "probe-call",
+ status = "completed",
+ output = { result: "oranges" },
+) {
+ return event(id, "action.result", {
+ result: { kind: "tool-result", toolName: "probe", callId, output },
+ status,
+ turnId: "turn_1",
+ });
+}
+
+const expected = [
+ {
+ callId: "probe-call",
+ inputs: [{ check: "first" }],
+ results: [{ status: "completed", output: { result: "oranges" } }],
+ },
+];
+
+function snapshots() {
+ return [
+ {
+ sessionId: "parent",
+ events: [event("proxy", "input.requested", { requests: [approval("task_a", "first")] })],
+ },
+ {
+ sessionId: "child",
+ events: [
+ requested("request"),
+ event("approval", "input.requested", { requests: [approval("task_a", "first")] }),
+ ],
+ },
+ { sessionId: "child", events: [completed("result")] },
+ ];
+}
+
+test("repeated activation events with new event IDs still identify one child", () => {
+ const child = {
+ callId: "call-a",
+ childSessionId: "child-a",
+ agentId: "agent-a",
+ name: "agent",
+ sessionId: "parent",
+ turnId: "turn_1",
+ };
+ const snapshots = [
+ {
+ sessionId: "parent",
+ events: [
+ event("activation-1", "subagent.called", child),
+ event("activation-2", "subagent.called", { ...child }),
+ ],
+ },
+ ];
+ assert.deepEqual(childActivations(snapshots, "parent"), [child]);
+ assert.deepEqual(childActivations(snapshots, "another-parent"), []);
+ for (const changed of [
+ { childSessionId: "child-b" },
+ { agentId: "agent-b" },
+ { name: "other" },
+ { sessionId: "other" },
+ { turnId: "turn_2" },
+ ]) {
+ snapshots[0].events[1] = event("activation-2", "subagent.called", { ...child, ...changed });
+ assert.throws(() => childActivations(snapshots, "parent"), /changed its identity/);
+ }
+ snapshots[0].events[1] = event("activation-2", "subagent.called", { ...child, callId: "call-b" });
+ assert.equal(childActivations(snapshots, "parent").length, 2);
+});
+
+test("joins a paused request to its result in a later approval turn", () => {
+ assert.deepEqual(toolEvidence(snapshots(), "child", "probe"), expected);
+ assert.deepEqual(toolEvidence(snapshots(), "parent", "probe"), []);
+});
+
+test("proxy approvals in both ancestors do not count as nested probe executions", () => {
+ const turns = snapshots();
+ turns[1].sessionId = "leaf";
+ turns[2].sessionId = "leaf";
+ turns.push({ sessionId: "child", events: turns[0].events });
+ assert.deepEqual(toolEvidence(turns, "leaf", "probe"), expected);
+ assert.deepEqual(toolEvidence(turns, "child", "probe"), []);
+});
+
+test("only deduplicates event IDs within the owning session, including overlapping watches", () => {
+ const turns = snapshots();
+ turns.push(...snapshots());
+ turns.push({
+ sessionId: "sibling",
+ events: [
+ requested("request", "probe-call", "second"),
+ completed("result", "probe-call", "completed", { result: "pears" }),
+ ],
+ });
+ assert.deepEqual(toolEvidence(turns, "child", "probe"), expected);
+ assert.equal(eventsForSession(turns, "child").length, 3);
+ assert.deepEqual(toolEvidence(turns, "sibling", "probe")[0].inputs, [{ check: "second" }]);
+});
+
+test("does not join a same-call-ID request and result from different sessions", () => {
+ const turns = snapshots();
+ turns[2].sessionId = "sibling";
+ assert.deepEqual(toolEvidence(turns, "child", "probe")[0].results, []);
+ assert.deepEqual(toolEvidence(turns, "sibling", "probe")[0].inputs, []);
+});
+
+for (const [name, extra] of [
+ [
+ "a second execution with a new call ID",
+ [requested("request-2", "second-call"), completed("result-2", "second-call")],
+ ],
+ ["a duplicate result event for the same call ID", [completed("result-2")]],
+ ["a duplicate request event for the same call ID", [requested("request-2")]],
+ ["an orphan completion", [completed("orphan", "unknown-call")]],
+ ["a pending extra call", [requested("pending", "pending-call")]],
+]) {
+ test(`exact evidence rejects ${name}`, () => {
+ const turns = snapshots();
+ turns.push({ sessionId: "child", events: extra });
+ assert.notDeepEqual(toolEvidence(turns, "child", "probe"), expected);
+ });
+}
+
+for (const [name, result] of [
+ ["a failed result", completed("result", "probe-call", "failed")],
+ ["a rejected result", completed("result", "probe-call", "rejected")],
+ ["the wrong output", completed("result", "probe-call", "completed", { result: "pears" })],
+ ["the wrong call ID", completed("result", "other-call")],
+]) {
+ test(`exact evidence rejects ${name}`, () => {
+ const turns = snapshots();
+ turns[2].events = [result];
+ assert.notDeepEqual(toolEvidence(turns, "child", "probe"), expected);
+ });
+}
+
+test("exact evidence rejects the wrong check or a missing completion", () => {
+ const turns = snapshots();
+ turns[1].events = [requested("request", "probe-call", "second")];
+ assert.notDeepEqual(toolEvidence(turns, "child", "probe"), expected);
+ assert.notDeepEqual(toolEvidence(snapshots().slice(0, 2), "child", "probe"), expected);
+});
+
+test("reports a filtered original task before attempting to join replacement approvals", () => {
+ const message =
+ 'Background task task_original (agent) failed.\n\nError:\n{"code":"SUBAGENT_EXECUTION_FAILED","message":"The model provider filtered this response."}';
+ const turns = [
+ {
+ sessionId: "parent",
+ events: [
+ event("failure", "message.received", { message }),
+ event("replacement-approval", "input.requested", {
+ requests: [approval("task_replacement", "third")],
+ }),
+ ],
+ },
+ ];
+ assert.throws(
+ () => requireOriginalTasksHealthy(turns, "parent", ["task_original"]),
+ /replacement tasks cannot satisfy this eval.*task_original[\s\S]*provider filtered this response/,
+ );
+ assert.throws(() => checkForTask("task_original", [approval("task_replacement", "third")]));
+});
+
+test("reports cancellation of an original task before an approval join", () => {
+ const turns = [
+ {
+ sessionId: "parent",
+ events: [
+ event("cancel", "message.received", {
+ message: "Background task task_original (agent) is cancelled.",
+ }),
+ ],
+ },
+ ];
+ assert.throws(
+ () => requireOriginalTasksHealthy(turns, "parent", ["task_original"]),
+ /is cancelled/,
+ );
+});
+
+test("failure detection is scoped to the original task and its parent session", () => {
+ const failure = "Background task task_original (agent) failed.";
+ const turns = [
+ {
+ sessionId: "other-parent",
+ events: [event("other-failure", "message.received", { message: failure })],
+ },
+ {
+ sessionId: "parent",
+ events: [
+ event("completion", "message.received", {
+ message: "Background task task_original (agent) is completed.",
+ }),
+ event("other-task", "message.received", {
+ message: "Background task task_original_other (agent) failed.",
+ }),
+ event("quotation", "message.received", {
+ message: `Alice asks about this earlier note: ${failure}`,
+ }),
+ event("answer", "message.completed", { message: failure }),
+ ],
+ },
+ ];
+ assert.doesNotThrow(() => requireOriginalTasksHealthy(turns, "parent", ["task_original"]));
+});
+
+test("maps checks by task-scoped runtime approval, independent of assignment wording or order", () => {
+ const requests = [
+ approval("task_second", "second"),
+ approval("task_third", "third"),
+ approval("task_first", "first"),
+ ];
+ assert.equal(checkForTask("task_first", requests), "first");
+ assert.equal(checkForTask("task_second", requests), "second");
+ assert.equal(checkForTask("task_third", requests), "third");
+ assert.throws(() => checkForTask("task_fir", requests));
+});
+
+test("task joins reject missing, ambiguous, invalid, non-approval, and wrong-tool requests", () => {
+ const valid = approval("task_a", "first");
+ for (const requests of [
+ [],
+ [approval("task_b", "first")],
+ [valid, { ...valid, requestId: "task_a:other-approval" }],
+ [approval("task_a", "fourth")],
+ [{ ...valid, kind: "question" }],
+ [{ ...valid, action: { ...valid.action, toolName: "other-tool" } }],
+ ]) {
+ assert.throws(() => checkForTask("task_a", requests));
+ }
+});
diff --git a/e2e/fixtures/agent-task-reporting/evals/event-matching.ts b/e2e/fixtures/agent-task-reporting/evals/event-matching.ts
new file mode 100644
index 000000000..fc187e739
--- /dev/null
+++ b/e2e/fixtures/agent-task-reporting/evals/event-matching.ts
@@ -0,0 +1,120 @@
+import type { EveEvalTurn, InputRequest } from "eve/evals";
+
+type Event = EveEvalTurn["events"][number];
+export type SessionEvents = Pick<EveEvalTurn, "sessionId" | "events">;
+export const CHECKS = ["first", "second", "third"] as const;
+export type Check = (typeof CHECKS)[number];
+
+export function requireOriginalTasksHealthy(
+ snapshots: readonly SessionEvents[],
+ sessionId: string,
+ taskIds: readonly string[],
+): void {
+ for (const event of eventsForSession(snapshots, sessionId)) {
+ if (event.type !== "message.received") continue;
+ for (const taskId of taskIds) {
+ const prefix = `Background task ${taskId} (agent) `;
+ const message = event.data.message;
+ if (message.startsWith(`${prefix}failed.`) || message.startsWith(`${prefix}is cancelled.`)) {
+ throw new Error(
+ `Original warehouse task failed or was cancelled; replacement tasks cannot satisfy this eval. ${message.slice(0, 1000)}`,
+ );
+ }
+ }
+ }
+}
+
+export function checkForTask(taskId: string, requests: readonly InputRequest[]): Check {
+ // The parent proxy prefixes the original approval ID with its owning task ID.
+ const owned = requests.filter((request) => request.requestId.startsWith(`${taskId}:`));
+ const request = owned[0];
+ const check = CHECKS.find((candidate) => candidate === request?.action.input.check);
+ if (
+ owned.length !== 1 ||
+ request?.kind !== "tool-approval" ||
+ request.action.toolName !== "probe" ||
+ check === undefined
+ ) {
+ throw new Error(
+ `Warehouse task ${taskId} needs exactly one probe approval with a valid check.`,
+ );
+ }
+ return check;
+}
+
+export function eventsForSession(snapshots: readonly SessionEvents[], sessionId: string): Event[] {
+ const events = new Map<string, Event>();
+ for (const snapshot of snapshots) {
+ if (snapshot.sessionId !== sessionId) continue;
+ for (const event of snapshot.events) {
+ // Only repeated delivery of the same event is replay, not another result for the same call.
+ if (!events.has(event.meta.id)) events.set(event.meta.id, event);
+ }
+ }
+ return [...events.values()];
+}
+
+export function childActivations(snapshots: readonly SessionEvents[], sessionId: string) {
+ type Called = Extract<Event, { type: "subagent.called" }>["data"];
+ const calls = new Map<string, Called>();
+ for (const event of eventsForSession(snapshots, sessionId)) {
+ if (event.type !== "subagent.called") continue;
+ const current = event.data;
+ const previous = calls.get(current.callId);
+ if (previous !== undefined) {
+ // A retried activation notification must still identify the very same child.
+ if (
+ previous.childSessionId !== current.childSessionId ||
+ previous.agentId !== current.agentId ||
+ previous.name !== current.name ||
+ previous.sessionId !== current.sessionId ||
+ previous.turnId !== current.turnId
+ ) {
+ throw new Error(`Child activation ${current.callId} changed its identity.`);
+ }
+ } else {
+ calls.set(current.callId, current);
+ }
+ }
+ return [...calls.values()];
+}
+
+export function toolEvidence(
+ snapshots: readonly SessionEvents[],
+ sessionId: string,
+ toolName: string,
+) {
+ type Requested = Extract<Event, { type: "actions.requested" }>["data"]["actions"][number];
+ type Result = Extract<Event, { type: "action.result" }>["data"];
+ const calls = new Map<
+ string,
+ {
+ callId: string;
+ inputs: Requested["input"][];
+ results: { status: Result["status"]; output: Result["result"]["output"] }[];
+ }
+ >();
+ function call(callId: string) {
+ let entry = calls.get(callId);
+ if (entry === undefined) {
+ entry = { callId, inputs: [], results: [] };
+ calls.set(callId, entry);
+ }
+ return entry;
+ }
+ for (const event of eventsForSession(snapshots, sessionId)) {
+ if (event.type === "actions.requested") {
+ for (const action of event.data.actions) {
+ if (action.kind === "tool-call" && action.toolName === toolName) {
+ call(action.callId).inputs.push(action.input);
+ }
+ }
+ } else if (event.type === "action.result") {
+ const { result, status } = event.data;
+ if (result.kind === "tool-result" && result.toolName === toolName) {
+ call(result.callId).results.push({ status, output: result.output });
+ }
+ }
+ }
+ return [...calls.values()];
+}
diff --git a/e2e/fixtures/agent-task-reporting/evals/pending-response.eval.ts b/e2e/fixtures/agent-task-reporting/evals/pending-response.eval.ts
index dedbdf9b0..e7627a7f6 100644
--- a/e2e/fixtures/agent-task-reporting/evals/pending-response.eval.ts
+++ b/e2e/fixtures/agent-task-reporting/evals/pending-response.eval.ts
@@ -1,12 +1,12 @@
-import { defineEval, type EveEvalTurn } from "eve/evals";
-import { satisfies } from "eve/evals/expect";
+import { defineEval } from "eve/evals";
+import { equals } from "eve/evals/expect";
import {
- completeReport,
+ completedAt,
+ modelSteps,
QUESTION,
sendQuestion,
startWarehouseLookups,
- TASK_COUNT,
waitForPartialCompletion,
waitForReport,
} from "./reporting.js";
@@ -14,62 +14,44 @@ import {
function pendingResponseEval() {
return defineEval({
description:
- "After a partial completion, the parent answers a user while sibling tasks are running, then reports all results.",
+ "A real parent answers Alice after one child completes while sibling approvals remain pending, then reports the whole cohort including a nested lookup.",
tags: ["real-model", "pending-response"],
async test(t) {
const run = await startWarehouseLookups(t);
await waitForPartialCompletion(t, run);
-
const question = await sendQuestion(t, run);
- question.messageIncludes(/\b56\b/u);
- question.usedNoTools();
-
const report = await waitForReport(t, run);
- t.check(report, completeReport());
- t.notEvent("compaction.requested");
- await t.require(
- run.childSessionIds.size,
- satisfies(
- (count) => count === TASK_COUNT,
- "all child sessions are available for timing checks",
- ),
- );
- const children = await Promise.all(
- [...run.childSessionIds].map((sessionId) => t.target.watchTurn(sessionId).result()),
- );
- const settledAt = Math.max(...children.map(completedAt));
+ const first = run.completedChildren.get("first");
+ const last = run.completedChildren.get("third");
+ if (first === undefined || last === undefined)
+ throw new Error("Missing completed warehouse children.");
const received = question.events.find(
- (event) => event.type === "message.received" && event.data.message.includes(QUESTION),
+ (event) => event.type === "message.received" && event.data.message === QUESTION,
);
const answered = question.events.find(
- (event) =>
- event.type === "message.completed" &&
- event.data.finishReason !== "tool-calls" &&
- /\b56\b/u.test(event.data.message ?? ""),
+ (event) => event.type === "message.completed" && event.data.message?.trim() === "56",
);
+ const reportStep = report.events.find((event) => event.type === "step.started");
const askedAt = Date.parse(received?.meta.at ?? "");
const answeredAt = Date.parse(answered?.meta.at ?? "");
- t.check(
- askedAt < settledAt,
- satisfies(Boolean, "the user question reached the parent before cohort settlement"),
+ const reportAt = Date.parse(reportStep?.meta.at ?? "");
+ t.check(completedAt(first) <= askedAt, equals(true)).label(
+ "the user question follows actual child completion",
);
- t.check(
- answeredAt < settledAt,
- satisfies(Boolean, "the user received an answer before cohort settlement"),
+ t.check(answeredAt < completedAt(last), equals(true)).label(
+ "the user answer precedes the nested child's completion",
);
- t.log(
- `user reply=${JSON.stringify(question.message)} asked=${askedAt} answered=${answeredAt} settled=${settledAt}`,
+ t.check(completedAt(last) <= reportAt, equals(true)).label(
+ "the cohort model turn starts only after the last child completes",
);
+ t.check(modelSteps(run.parentTurns), equals(2)).label(
+ "only the user answer and final cohort report invoke the parent model",
+ );
+ t.notEvent("compaction.requested");
t.noFailedActions();
},
});
}
export default Array.from({ length: 20 }, pendingResponseEval);
-
-function completedAt(turn: EveEvalTurn): number {
- const event = turn.events.find((entry) => entry.type === "turn.completed");
- if (event === undefined) throw new Error("Missing child completion event for timing check.");
- return Date.parse(event.meta.at);
-}
diff --git a/e2e/fixtures/agent-task-reporting/evals/reporting.ts b/e2e/fixtures/agent-task-reporting/evals/reporting.ts
index 1692c701a..4a316d0fc 100644
--- a/e2e/fixtures/agent-task-reporting/evals/reporting.ts
+++ b/e2e/fixtures/agent-task-reporting/evals/reporting.ts
@@ -1,174 +1,438 @@
-import type { EveEvalContext, EveEvalLiveTurn, EveEvalSession, EveEvalTurn } from "eve/evals";
-import { satisfies } from "eve/evals/expect";
+import { e2eModel } from "@eve-e2e/config";
+import type { EveEvalContext, EveEvalSession, EveEvalTurn, InputRequest } from "eve/evals";
+import { equals, satisfies } from "eve/evals/expect";
+
+import {
+ CHECKS,
+ childActivations,
+ checkForTask,
+ eventsForSession,
+ requireOriginalTasksHealthy,
+ toolEvidence,
+ type Check,
+} from "./event-matching.js";
export const TASK_COUNT = 3;
export const QUESTION =
"Alice is packing seven boxes with eight jars in each. How many jars is that? Reply with just the number.";
-const MIN_BACKGROUND_TASKS = 2;
-const RESULTS = [/\boranges\b/iu, /\bpears\b/iu, /\bapples\b/iu];
+const RESULTS = { first: "oranges", second: "pears", third: "apples" } as const;
const COMPLETION = /Background task (task_[a-z0-9]+) \([^)]+\) is completed\./giu;
-interface ReportingRun {
+interface Child {
+ readonly check: Check;
+ readonly sessionId: string;
+ readonly taskId: string;
+}
+
+export interface ReportingRun {
readonly sessionId: string;
readonly taskIds: readonly string[];
- readonly observed: Set<string>;
- readonly childSessionIds: Set<string>;
- readonly wakes: { turn: EveEvalTurn; completed: number; settled: boolean }[];
+ readonly children: readonly Child[];
+ readonly requests: ReadonlyMap<Check, InputRequest>;
+ readonly completedChildren: Map<Check, EveEvalTurn>;
+ readonly childTurns: EveEvalTurn[];
+ readonly probeSessions: Map<Check, string>;
+ readonly parentTurns: EveEvalTurn[];
+ readonly modelId: string;
session: EveEvalSession | EveEvalContext;
}
export async function startWarehouseLookups(t: EveEvalContext): Promise<ReportingRun> {
+ const modelId = e2eModel();
+ if (typeof modelId !== "string") throw new Error("Warehouse reporting requires a real CI model.");
const started =
- await t.send(`Please find the inventory item at each of our three sample warehouses using the built-in agent tool. Start all three lookups in the background without waiting for their results. Delegate the lookups instead of calling probe yourself.
+ await t.send(`Alice is preparing an inventory checklist for Bob's warehouse handoff. Please delegate these three entries to three separate background assistants using the built-in agent tool, so the checks can proceed independently. Include the entry's check reference in each assistant's assignment. Keep this small checklist in the conversation rather than creating a separate todo list.
-1. "Call probe with check=first and report its result value."
-2. "Call probe with check=second and report its result value."
-3. "Call probe with check=third and report its result value."`);
+1. check=first: Find the inventory item for the first entry using the inventory lookup tool (probe), and share the item it returns.
+2. check=second: Find the inventory item for the second entry using the inventory lookup tool (probe), and share the item it returns.
+3. check=third: Assign a third background assistant with agent, just like the first two entries. Ask that assistant to use warehouse_lookup; the workflow contacts the warehouse specialist and returns its item.
+
+Once all three assignments are accepted, let Alice know the checks are underway. When all three results are ready, reply directly from the returned items without further tool calls or task-list updates. Bob needs only three checklist entries, with each item listed once and no separate summary.`);
started.expectOk();
- started.calledSubagent("agent", { count: TASK_COUNT }).soft().label("no repeated delegation");
+ started.calledSubagent("agent", { count: TASK_COUNT });
+ started.notCalledTool("probe");
+ started.notCalledTool("warehouse_lookup");
+ assertModel(started, modelId);
await t.require(
started,
- satisfies(
- hasPostReceiptAcknowledgement,
- "an acknowledgement follows all background task receipts",
- ),
+ satisfies(hasPostReceiptAcknowledgement, "acknowledgement follows all three task receipts"),
);
await t.require(
started.message,
satisfies(
- (message: unknown) =>
+ (message) =>
typeof message === "string" &&
message.trim().length > 0 &&
- RESULTS.every((result) => !result.test(message)),
- "the initiating turn acknowledges work without claiming results",
+ Object.values(RESULTS).every((result) => !message.toLowerCase().includes(result)),
+ "the initiating turn acknowledges work without claiming inventory results",
),
);
- const taskIds = [
- ...new Set(
- started.events.flatMap((event) =>
- event.type === "subagent.completed" &&
- event.data.subagentName === "agent" &&
- event.data.backgroundTask !== undefined
- ? [event.data.backgroundTask.taskId]
- : [],
- ),
- ),
- ];
- await t.require(
- taskIds,
- satisfies(
- (ids: readonly string[]) => ids.length >= MIN_BACKGROUND_TASKS,
- "multiple independent background tasks exercise partial wakes",
- ),
+
+ const receipts = started.events.flatMap((event) =>
+ event.type === "subagent.completed" && event.data.backgroundTask !== undefined
+ ? [{ callId: event.data.callId, taskId: event.data.backgroundTask.taskId }]
+ : [],
);
- return {
+ const actions = started.events.flatMap((event) =>
+ event.type === "actions.requested" ? event.data.actions : [],
+ );
+ const setupTurns = [started];
+ let calls = childActivations(setupTurns, started.sessionId);
+ const children: Child[] = [];
+ const taskIds = receipts.map((receipt) => receipt.taskId);
+ const requests = new Map<Check, InputRequest>();
+ const run: ReportingRun = {
session: t,
sessionId: started.sessionId,
taskIds,
- observed: new Set(),
- childSessionIds: new Set(childSessions(started)),
- wakes: [],
+ children,
+ requests,
+ modelId,
+ completedChildren: new Map(),
+ childTurns: [],
+ probeSessions: new Map(),
+ parentTurns: [],
};
+ requireOriginalTasksHealthy([started], run.sessionId, run.taskIds);
+ collectRequests(started);
+ // Task receipts precede child startup; called events can arrive after the parent parks.
+ for (
+ let attempt = 0;
+ (calls.length < TASK_COUNT || requests.size < TASK_COUNT) && attempt < 8;
+ attempt += 1
+ ) {
+ const turn = await nextParentTurn(t, run);
+ setupTurns.push(turn);
+ calls = childActivations(setupTurns, started.sessionId);
+ collectRequests(turn);
+ }
+ children.push(
+ ...calls.map((call): Child => {
+ const assignments = actions.filter(
+ (action) =>
+ action.callId === call.callId &&
+ action.kind === "tool-call" &&
+ action.toolName === "agent",
+ );
+ const matchingReceipts = receipts.filter((entry) => entry.callId === call.callId);
+ const receipt = matchingReceipts[0];
+ if (
+ call.sessionId !== started.sessionId ||
+ call.name !== "agent" ||
+ assignments.length !== 1 ||
+ matchingReceipts.length !== 1 ||
+ receipt === undefined
+ )
+ throw new Error("Each warehouse child needs one assignment and a matching task receipt.");
+ return {
+ check: checkForTask(receipt.taskId, [...requests.values()]),
+ sessionId: call.childSessionId,
+ taskId: receipt.taskId,
+ };
+ }),
+ );
+ await t.require(
+ {
+ receipts: taskIds.length,
+ tasks: new Set(taskIds).size,
+ children: children.length,
+ sessions: new Set(children.map((child) => child.sessionId)).size,
+ checks: children.map((child) => child.check).sort(),
+ mappedTasks: children.map((child) => child.taskId).sort(),
+ // Child startup may be observed in later turns; assignment events own launch provenance.
+ creatingTurns: new Set(
+ started.events.flatMap((event) =>
+ event.type === "actions.requested" &&
+ event.data.actions.some(
+ (action) => action.kind === "tool-call" && action.toolName === "agent",
+ )
+ ? [event.data.turnId]
+ : [],
+ ),
+ ).size,
+ },
+ equals({
+ receipts: TASK_COUNT,
+ tasks: TASK_COUNT,
+ children: TASK_COUNT,
+ sessions: TASK_COUNT,
+ checks: [...CHECKS].sort(),
+ mappedTasks: [...taskIds].sort(),
+ creatingTurns: 1,
+ }),
+ );
+
+ await t.require(
+ {
+ checks: [...requests.keys()].sort(),
+ ids: new Set([...requests.values()].map((request) => request.requestId)).size,
+ },
+ equals({ checks: [...CHECKS].sort(), ids: TASK_COUNT }),
+ );
+ // Setup includes prompt/approval wakes; completion admission is measured from this cursor onward.
+ run.parentTurns.length = 0;
+ return run;
+
+ function collectRequests(turn: EveEvalTurn) {
+ for (const request of turn.inputRequests) {
+ if (request.action.toolName !== "probe")
+ throw new Error("Unexpected warehouse approval tool.");
+ const check = CHECKS.find((candidate) => candidate === request.action.input.check);
+ if (
+ check === undefined ||
+ (requests.has(check) && requests.get(check)!.requestId !== request.requestId)
+ ) {
+ throw new Error("Each warehouse must have exactly one approval request.");
+ }
+ requests.set(check, request);
+ }
+ }
}
+/** Establish partial completion on the child stream, without waiting for a withheld parent wake. */
export async function waitForPartialCompletion(
t: EveEvalContext,
run: ReportingRun,
): Promise<void> {
- for (let attempt = 0; attempt < 8 && run.observed.size === 0; attempt += 1) {
- await readTurn(t, run, nextTurn(t, run));
- }
- await t.require(
- run.observed.size,
- satisfies(
- (count: number) => count > 0 && count < run.taskIds.length,
- "an intermediate completion leaves sibling tasks pending",
- ),
- );
+ await releaseCheck(t, run, "first");
+ await t.require([...run.completedChildren.keys()], equals(["first"]));
}
export async function sendQuestion(t: EveEvalContext, run: ReportingRun): Promise<EveEvalTurn> {
- let live = await run.session.start(QUESTION);
+ await post(t, run, "", { message: QUESTION, turnPolicy: "queue" });
for (let attempt = 0; attempt < 8; attempt += 1) {
- const turn = await readTurn(t, run, live);
- if (receivedQuestion(turn)) return turn;
- live = nextTurn(t, run);
+ const turn = await nextParentTurn(t, run);
+ await t.require(completedTaskIds(turn), equals([]));
+ if (
+ turn.events.some(
+ (event) => event.type === "message.received" && event.data.message === QUESTION,
+ )
+ ) {
+ await t.require(turn.message?.trim(), equals("56"));
+ turn.usedNoTools();
+ await t.require(modelSteps(run.parentTurns), equals(1));
+ return turn;
+ }
+ turn.notEvent("step.started");
}
- throw new Error("No user turn followed the intermediate wake.");
+ throw new Error("No independent user answer while the remaining warehouses were gated.");
}
-export async function waitForReport(
- t: EveEvalContext,
- run: ReportingRun,
-): Promise<string | undefined> {
- for (let attempt = 0; attempt < 8 && run.observed.size < run.taskIds.length; attempt += 1) {
- await readTurn(t, run, nextTurn(t, run));
+export async function compactWhilePending(t: EveEvalContext, run: ReportingRun): Promise<void> {
+ await post(t, run, "/compact", {});
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const turn = await nextParentTurn(t, run);
+ await t.require(completedTaskIds(turn), equals([]));
+ turn.notEvent("step.started");
+ if (turn.events.some((event) => event.type === "compaction.requested")) {
+ turn.event("compaction.requested", { count: 1 });
+ // A declined summary preserves history; either path must retain the held cohort.
+ turn.event("compaction.completed", { count: 1 }).soft().label("successful checkpoint");
+ await t.require(modelSteps(run.parentTurns), equals(0));
+ return;
+ }
}
- await t.require(
- run.observed.size,
- satisfies((count: number) => count === run.taskIds.length, "all task wakes observed"),
- );
- return run.wakes
- .filter((wake) => wake.completed > 0 && wake.settled && !receivedQuestion(wake.turn))
- .at(-1)?.turn.message;
+ throw new Error("No compaction boundary while the remaining warehouses were gated.");
+}
+
+export async function waitForReport(t: EveEvalContext, run: ReportingRun): Promise<EveEvalTurn> {
+ await releaseCheck(t, run, "second");
+ await releaseCheck(t, run, "third");
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const turn = await nextParentTurn(t, run);
+ const taskIds = completedTaskIds(turn);
+ if (taskIds.length > 0) {
+ await t.require(taskIds.sort(), equals([...run.taskIds].sort()));
+ await t.require(
+ run.parentTurns.flatMap(completedTaskIds).sort(),
+ equals([...run.taskIds].sort()),
+ );
+ turn.usedNoTools();
+ turn.event("step.started", { count: 1, data: { modelId: run.modelId } });
+ turn.event("message.completed", {
+ count: 1,
+ data: (data) => data.finishReason !== "tool-calls" && data.message !== null,
+ });
+ turn.notEvent("message.completed", { data: (data) => data.message === null });
+ await t.require(turn.message, completeReport());
+ t.calledSubagent("agent", { count: TASK_COUNT });
+ const probeCalls = [...new Set(run.childTurns.map((child) => child.sessionId))].flatMap(
+ (sessionId) => toolEvidence(run.childTurns, sessionId, "probe"),
+ );
+ await t.require(probeCalls.length, equals(TASK_COUNT));
+ for (const check of CHECKS) {
+ const sessionId = run.probeSessions.get(check);
+ if (sessionId === undefined) throw new Error(`Missing probe session for ${check}.`);
+ await t.require(
+ toolEvidence(run.childTurns, sessionId, "probe"),
+ equals([
+ {
+ callId: run.requests.get(check)!.action.callId,
+ inputs: [{ check }],
+ results: [{ output: { result: RESULTS[check] }, status: "completed" }],
+ },
+ ]),
+ );
+ }
+ return turn;
+ }
+ turn.notEvent("step.started");
+ }
+ throw new Error("No single report containing every original warehouse task.");
}
export function completeReport() {
return satisfies(
- (message: unknown) =>
- typeof message === "string" && RESULTS.every((result) => result.test(message)),
- "settled tasks produce a complete user-facing report",
+ (message) =>
+ typeof message === "string" &&
+ JSON.stringify(
+ message
+ .toLowerCase()
+ .match(/\b(?:oranges|pears|apples)\b/gu)
+ ?.sort(),
+ ) === JSON.stringify(Object.values(RESULTS).sort()),
+ "one report contains each inventory item exactly once",
);
}
+export function modelSteps(turns: readonly EveEvalTurn[]): number {
+ return turns.flatMap((turn) => turn.events).filter((event) => event.type === "step.started")
+ .length;
+}
+
+export function completedAt(turn: EveEvalTurn): number {
+ const event = turn.events.find((entry) => entry.type === "turn.completed");
+ if (event === undefined) throw new Error("Missing child completion event.");
+ return Date.parse(event.meta.at);
+}
+
export function requireStreamIndex(session: EveEvalSession | EveEvalContext): number {
if (session.state === undefined) throw new Error("Task reporting session has no stream index.");
return session.state.streamIndex;
}
-function nextTurn(t: EveEvalContext, run: ReportingRun): EveEvalLiveTurn {
- return t.target.watchTurn(run.sessionId, { startIndex: requireStreamIndex(run.session) });
+async function releaseCheck(t: EveEvalContext, run: ReportingRun, check: Check): Promise<void> {
+ const request = run.requests.get(check);
+ const child = run.children.find((entry) => entry.check === check);
+ if (request === undefined || child === undefined || run.completedChildren.has(check)) {
+ throw new Error(`No unreleased warehouse child for ${check}.`);
+ }
+ await post(t, run, "", {
+ inputResponses: [{ requestId: request.requestId, optionId: "approve" }],
+ });
+ const turns = await readCompletedChild(t, child.sessionId, check, run.modelId);
+ run.completedChildren.set(check, turns.at(-1)!);
+ run.childTurns.push(...turns);
+ const nested = childActivations(turns, child.sessionId);
+ if (check !== "third") {
+ await t.require(nested, equals([]));
+ run.probeSessions.set(check, child.sessionId);
+ return;
+ }
+ await t.require(nested.length, equals(1));
+ const delegate = nested[0]!;
+ const lookup = toolEvidence(turns, child.sessionId, "warehouse_lookup");
+ await t.require(lookup.length, equals(1));
+ await t.require(
+ {
+ name: delegate.name,
+ sessionId: delegate.sessionId,
+ ownedCall: delegate.callId.startsWith(`${lookup[0]!.callId}:`),
+ distinctSession:
+ delegate.childSessionId !== run.sessionId &&
+ !run.children.some((entry) => entry.sessionId === delegate.childSessionId),
+ },
+ equals({
+ name: "warehouse-worker",
+ sessionId: child.sessionId,
+ ownedCall: true,
+ distinctSession: true,
+ }),
+ );
+ const leaf = await readCompletedChild(t, delegate.childSessionId, check, run.modelId);
+ run.childTurns.push(...leaf);
+ run.probeSessions.set(check, delegate.childSessionId);
+ await t.require(
+ eventsForSession(leaf, delegate.childSessionId).filter(
+ (event) => event.type === "subagent.called",
+ ),
+ equals([]),
+ );
+ // ctx.agent completes through its owning workflow tool, not a subagent.completed event.
+ await t.require(
+ lookup,
+ equals([
+ {
+ callId: lookup[0]!.callId,
+ inputs: [{ check }],
+ results: [{ status: "completed", output: leaf.at(-1)!.message }],
+ },
+ ]),
+ );
+ await t.require(completedAt(leaf.at(-1)!) <= completedAt(turns.at(-1)!), equals(true));
}
-async function readTurn(
+async function readCompletedChild(
t: EveEvalContext,
- run: ReportingRun,
- live: EveEvalLiveTurn,
-): Promise<EveEvalTurn> {
- const turn = await live.result();
- turn.expectOk();
+ sessionId: string,
+ check: Check,
+ modelId: string,
+): Promise<EveEvalTurn[]> {
+ const turns: EveEvalTurn[] = [];
+ let startIndex = 0;
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const live = t.target.watchTurn(sessionId, { startIndex });
+ const turn = (await live.result()).expectOk();
+ assertModel(turn, modelId);
+ turns.push(turn);
+ startIndex = requireStreamIndex(live.session);
+ if (
+ turn.events.some((event) => event.type === "turn.completed") &&
+ turn.message?.toLowerCase().includes(RESULTS[check])
+ )
+ return turns;
+ }
+ throw new Error(`Warehouse child ${check} did not complete with its inventory item.`);
+}
+
+async function nextParentTurn(t: EveEvalContext, run: ReportingRun): Promise<EveEvalTurn> {
+ const live = t.target.watchTurn(run.sessionId, { startIndex: requireStreamIndex(run.session) });
+ const turn = (await live.result()).expectOk();
+ requireOriginalTasksHealthy([turn], run.sessionId, run.taskIds);
turn.noFailedActions();
- const completed = turn.events
- .flatMap((event) => {
- if (event.type !== "message.received") return [];
- return [...event.data.message.matchAll(COMPLETION)].map((match) => match[1] as string);
- })
- .filter((taskId) => run.taskIds.includes(taskId));
- for (const taskId of completed) run.observed.add(taskId);
- for (const sessionId of childSessions(turn)) run.childSessionIds.add(sessionId);
+ assertModel(turn, run.modelId);
run.session = live.session;
- run.wakes.push({
- turn,
- completed: completed.length,
- settled: run.observed.size === run.taskIds.length,
- });
- t.log(
- `wake ${run.wakes.length}: completed=${run.observed.size}/${run.taskIds.length} message=${JSON.stringify(turn.message)}`,
- );
+ run.parentTurns.push(turn);
return turn;
}
-function childSessions(turn: EveEvalTurn): string[] {
+function assertModel(turn: EveEvalTurn, modelId: string) {
+ turn.eventsSatisfy("every model step uses the real CI model", (events) =>
+ events.every((event) => event.type !== "step.started" || event.data.modelId === modelId),
+ );
+}
+
+function completedTaskIds(turn: EveEvalTurn): string[] {
return turn.events.flatMap((event) =>
- event.type === "subagent.called" ? [event.data.childSessionId] : [],
+ event.type === "message.received"
+ ? [...event.data.message.matchAll(COMPLETION)].map((match) => match[1]!)
+ : [],
);
}
-function receivedQuestion(turn: EveEvalTurn): boolean {
- return turn.events.some(
- (event) => event.type === "message.received" && event.data.message.includes(QUESTION),
+async function post(t: EveEvalContext, run: ReportingRun, suffix: "" | "/compact", body: unknown) {
+ const response = await t.target.fetch(
+ `/eve/v1/session/${encodeURIComponent(run.sessionId)}${suffix}`,
+ {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify(body),
+ signal: t.signal,
+ },
);
+ await response.body?.cancel();
+ await t.require(response.status, equals(202));
}
function hasPostReceiptAcknowledgement(turn: EveEvalTurn): boolean {
@@ -179,12 +443,11 @@ function hasPostReceiptAcknowledgement(turn: EveEvalTurn): boolean {
? [index]
: [],
);
- const lastReceiptIndex = Math.max(...receiptIndexes);
return (
- receiptIndexes.length >= MIN_BACKGROUND_TASKS &&
+ receiptIndexes.length === TASK_COUNT &&
turn.events.some(
(event, index) =>
- index > lastReceiptIndex &&
+ index > Math.max(...receiptIndexes) &&
event.type === "message.completed" &&
event.data.finishReason !== "tool-calls" &&
event.data.message !== null &&
diff --git a/e2e/fixtures/agent-task-reporting/package.json b/e2e/fixtures/agent-task-reporting/package.json
index b42d86d14..48950c95f 100644
--- a/e2e/fixtures/agent-task-reporting/package.json
+++ b/e2e/fixtures/agent-task-reporting/package.json
@@ -8,7 +8,8 @@
"dev": "eve dev",
"start": "eve start",
"typecheck": "eve build && tsc",
- "test:e2e": "eve eval --strict"
+ "test:e2e": "eve eval --strict",
+ "test:unit": "node --test evals/event-matching.test.mjs"
},
"dependencies": {
"@eve-e2e/config": "workspace:*",
@@ -22,6 +23,6 @@
"typescript": "catalog:"
},
"e2e": {
- "modelMatrix": "full"
+ "modelMatrix": "default"
}
}
diff --git a/e2e/fixtures/fixture-tasks/agent/agent.ts b/e2e/fixtures/fixture-tasks/agent/agent.ts
index e7b092e42..41dd2873d 100644
--- a/e2e/fixtures/fixture-tasks/agent/agent.ts
+++ b/e2e/fixtures/fixture-tasks/agent/agent.ts
@@ -7,28 +7,25 @@ import {
type MockModelToolResult,
} from "eve/evals";
+import { lifecycleModel } from "./lib/lifecycle-model.js";
+
const TASK_ID_PATTERN = /task_[a-z0-9]+/iu;
-const EMPTY_DELIVERY_SENTINEL = "<eve-empty-delivery/>";
-const REDUNDANT_REVIEW_SCENARIO = "TASK-WAKE-REDUNDANT-REVIEW";
-const REDUNDANT_REVIEW_FINDING = "blocker: task admission can discard deferred user input.";
const TASK_STATE_LABEL = "[Task state]\n";
+const CHILD_TOOL_SURFACE_SCENARIO =
+ "Alice asks Bob to summarize the available tools for a background task.";
-function respond(
- request: MockModelRequest,
-): MockModelResponse | Promise<MockModelResponse | string> | string {
+function respond(request: MockModelRequest): MockModelResponse | string {
+ const lifecycle = lifecycleModel(request);
+ if (lifecycle !== undefined) return lifecycle;
+ if (request.userMessages.includes(CHILD_TOOL_SURFACE_SCENARIO)) {
+ return childToolSurfaceReport(request);
+ }
if (request.userMessages.includes("TASK-BATCHING-BENCHMARK")) {
return batchingBenchmark(request);
}
- if (request.userMessages.includes(REDUNDANT_REVIEW_SCENARIO)) {
- const taskState = latestTaskState(request.userMessages);
- if (taskState !== undefined) return handleRedundantReviewWake(taskState);
- }
// Framework announcements are model context, not scenario turns.
const message = [...request.userMessages].reverse().find(isScenarioMessage) ?? "";
- if (request.userMessages.some((entry) => entry.includes("TASK-UPDATE-PROGRESS"))) {
- return "TASK-UPDATE-RECEIVED";
- }
if (message.includes("TASK-FANOUT-INTERACTIVE-CHECK")) return "TASK-FANOUT-INTERACTIVE-OK";
if (message.includes("TASK-CANCEL-NOW")) return cancelWorkerTask(request);
if (message.includes("CHILD-TASK-EXCLUSIVITY-RACE")) return raceBusyWorker(request);
@@ -103,9 +100,8 @@ function respond(
if (message === "TASK-FANOUT-PARENT-UPDATES") return fanoutTasks(request, 10);
if (message === "TASK-PARENT-WAKE-UPDATES") return fanoutTasks(request, 3);
- if (message === REDUNDANT_REVIEW_SCENARIO) return startRedundantReviewers(request);
if (message === "TASK-FAN-IN") return fanInTasks(request);
- if (message === "TASK-UPDATE-SETUP") return startTaskUpdateChild(request);
+ if (message === "TASK-FAN-IN-STATUS") return fanInNotification(request);
if (message === "TASK-CANCEL-SETUP") return setupCancelWorker(request);
if (message.startsWith("TASK-CANCEL-VERIFY ")) {
return inspectTerminalTask(
@@ -172,61 +168,30 @@ function respond(
return `Mock reply: ${message}`;
}
-function startTaskUpdateChild(request: MockModelRequest): MockModelResponse | string {
- if (resultById(request, "task-update-child") === undefined) {
+function childToolSurfaceReport(request: MockModelRequest): MockModelResponse | string {
+ const task = latestTaskState(request.userMessages)?.tasks.find(
+ (entry) => entry.status === "completed",
+ );
+ if (task !== undefined) {
+ if (task.output?.type !== "result" || typeof task.output.data !== "string") {
+ throw new Error("The completed child did not return a tool report.");
+ }
+ return task.output.data;
+ }
+ if (resultById(request, "task-child-tool-surface") === undefined) {
return {
toolCalls: [
{
- id: "task-update-child",
- input: { message: "TASK-UPDATE-CHILD" },
- name: "update-worker",
+ id: "task-child-tool-surface",
+ input: {
+ message: "Bob, list your available tool names and return your final report to Alice.",
+ },
+ name: "tool-surface-worker",
},
],
};
}
- return "TASK-UPDATE-STARTED";
-}
-
-function startRedundantReviewers(request: MockModelRequest): MockModelResponse | string {
- const reviewers = [
- {
- id: "task-redundant-review-fast",
- message: `Review PR #2277. Return this finding: ${REDUNDANT_REVIEW_FINDING}`,
- },
- {
- id: "task-redundant-review-late",
- message: `BUSY-WORKER-A Review PR #2277. Return this finding: ${REDUNDANT_REVIEW_FINDING}`,
- },
- ] as const;
- const pending = reviewers.filter(({ id }) => resultById(request, id) === undefined);
- if (pending.length > 0) {
- return {
- toolCalls: pending.map(({ id, message }) => ({
- id,
- input: { message },
- name: "busy-worker",
- })),
- };
- }
- return "TASK-REDUNDANT-REVIEWERS-STARTED";
-}
-
-function handleRedundantReviewWake(taskState: TaskState): string {
- if (taskState.tasks.some((task) => task.status === "pending")) {
- return EMPTY_DELIVERY_SENTINEL;
- }
- const outputs = taskState.tasks.flatMap((task) =>
- task.output?.type === "result" && typeof task.output.data === "string"
- ? [task.output.data]
- : [],
- );
- if (
- outputs.length !== taskState.tasks.length ||
- !outputs.every((output) => output.includes(REDUNDANT_REVIEW_FINDING))
- ) {
- throw new Error("The settled reviewer cohort did not contain every expected finding.");
- }
- return `request changes on PR #2277.\n\n- ${REDUNDANT_REVIEW_FINDING}`;
+ return "TASK-CHILD-TOOL-SURFACE-STARTED";
}
interface TaskState {
@@ -266,26 +231,21 @@ function fanoutTasks(request: MockModelRequest, size: number): MockModelResponse
return "TASK-FANOUT-STARTED";
}
-async function batchingBenchmark(request: MockModelRequest): Promise<MockModelResponse | string> {
+function batchingBenchmark(request: MockModelRequest): MockModelResponse | string {
const message = [...request.userMessages]
.reverse()
- .find((entry) => entry.startsWith("TASK-BATCHING-") || entry.startsWith("Background task "));
+ .find(
+ (entry) => entry.startsWith("TASK-BATCHING-") || entry.startsWith("Background task task_"),
+ );
if (message === "TASK-BATCHING-QUESTION") return "56";
if (message === "TASK-BATCHING-BENCHMARK") return fanoutTasks(request, 10);
if (message?.endsWith("needs input.")) return "TASK-NOTIFICATION-ACK";
- // Keep the first completion's model call active while the burst arrives.
- // This is fixture inference time, not a runtime debounce or settlement wait.
- const completionMessages = request.userMessages.filter(
- (entry) => entry.startsWith("Background task ") && entry.includes(" is completed."),
- );
- if (completionMessages.length === 1) {
- await new Promise((resolve) => setTimeout(resolve, 10_000));
- }
const state = latestTaskState(request.userMessages);
if (state === undefined) return "TASK-FANOUT-STARTED";
- // Script perfect compliance so the eval measures delivery cost, not model obedience.
- if (state.tasks.some((task) => task.status === "pending")) return EMPTY_DELIVERY_SENTINEL;
+ if (state.tasks.some((task) => task.status === "pending")) {
+ throw new Error("A completion invoked the parent model before its cohort settled.");
+ }
const results = state.tasks.flatMap((task) =>
task.output?.type === "result" && typeof task.output.data === "string"
? [task.output.data]
@@ -662,12 +622,11 @@ function findString(value: unknown, prefix: string): string | undefined {
return undefined;
}
-const base = e2eAgentConfig();
-
export default defineAgent({
- ...base,
- // These evals target orchestration, not model planning. Keep every suite
- // deterministic while retaining the workflow-world override from `base`.
+ ...e2eAgentConfig(),
+ // Script orchestration checks here; real-model coverage lives in agent-task-reporting.
model: mockModel(respond),
- modelContextWindowTokens: 1_000_000,
+ // The lifecycle eval checks child usage through the runtime's budget gate.
+ limits: { maxInputTokensPerSession: 1_000_000 },
+ modelContextWindowTokens: 10_000_000,
});
diff --git a/e2e/fixtures/fixture-tasks/agent/channels/lifecycle.ts b/e2e/fixtures/fixture-tasks/agent/channels/lifecycle.ts
new file mode 100644
index 000000000..6d0525ac5
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/channels/lifecycle.ts
@@ -0,0 +1,134 @@
+import { defineChannel, POST } from "eve/channels";
+import { getHookByToken, getRun, getWorld, resumeHook } from "workflow/api";
+import { z } from "zod";
+import { lifecycleNamespace, type LifecycleControlEvent } from "../lib/lifecycle-control.js";
+
+const bodySchema = z.object({
+ key: z.string().uuid(),
+ index: z.number().int().min(0).max(20).optional(),
+ token: z.string().min(1).max(512).optional(),
+ marker: z.enum(["A", "B"]).optional(),
+});
+
+// Fixture-only capability routes: a per-eval random key addresses only its own
+// control stream. Only fixture gate tokens are exposed, never runtime routing credentials.
+export default defineChannel({
+ routes: [
+ POST("/eve/v1/task-lifecycle/:sessionId/:action", async (request, { params }) => {
+ const signal = AbortSignal.any([request.signal, AbortSignal.timeout(60_000)]);
+ const body = bodySchema.parse(JSON.parse(await boundedBody(request)));
+ const sessionId = params.sessionId;
+ const read = async (index: number) => {
+ const reader = getRun(sessionId)
+ .getReadable<LifecycleControlEvent>({
+ namespace: lifecycleNamespace(body.key),
+ startIndex: index,
+ })
+ .getReader();
+ const abort = () => {
+ void reader.cancel("Lifecycle observation aborted");
+ };
+ signal.addEventListener("abort", abort, { once: true });
+ try {
+ signal.throwIfAborted();
+ const next = await reader.read();
+ if (next.done) throw new Error("Lifecycle control stream ended before acknowledgment.");
+ return next.value;
+ } finally {
+ signal.removeEventListener("abort", abort);
+ await reader.cancel();
+ reader.releaseLock();
+ }
+ };
+ // Reading this namespace proves that this eval key was registered by the
+ // addressed parent. Unknown keys cannot inspect or release another eval.
+ const first = await read(0);
+ if (first.kind !== "owner" || first.sessionId !== sessionId)
+ return new Response(null, { status: 403 });
+ if (params.action === "next") return Response.json(await read(body.index ?? 0));
+ if ((params.action === "release" || params.action === "active") && body.token !== undefined) {
+ const hook = await getHookByToken(body.token);
+ const metadata = await hook.metadata;
+ if (
+ metadata === null ||
+ typeof metadata !== "object" ||
+ Reflect.get(metadata, "key") !== body.key ||
+ Reflect.get(metadata, "parentSessionId") !== sessionId
+ )
+ return new Response(null, { status: 403 });
+ if (params.action === "active")
+ return Response.json({ status: await getRun(hook.runId).status });
+ await resumeHook(body.token, undefined);
+ return Response.json({ released: true });
+ }
+ if (params.action === "settled" && body.marker !== undefined) {
+ for (let index = 0; index < 6; index += 1) {
+ const event = index === 0 ? first : await read(index);
+ if (event.kind !== "owner" || event.marker !== body.marker) continue;
+ const run = getRun(event.runId);
+ signal.throwIfAborted();
+ let rejectWait: ((reason: Error) => void) | undefined;
+ const aborted = new Promise<never>((_, reject) => {
+ rejectWait = reject;
+ });
+ const abort = () => rejectWait?.(new Error("Task owner did not settle."));
+ signal.addEventListener("abort", abort, { once: true });
+ try {
+ await Promise.race([run.returnValue, aborted]);
+ } finally {
+ signal.removeEventListener("abort", abort);
+ }
+ return Response.json({
+ marker: event.marker,
+ status: await run.status,
+ deliveries: await ownerDeliveries(event.runId),
+ });
+ }
+ }
+ return new Response(null, { status: 400 });
+ }),
+ ],
+});
+
+async function ownerDeliveries(runId: string) {
+ const world = await getWorld();
+ const [steps, events] = await Promise.all([
+ world.steps.list({ runId, pagination: { limit: 100 }, resolveData: "none" }),
+ world.events.list({
+ runId,
+ pagination: { limit: 100, sortOrder: "asc" },
+ resolveData: "none",
+ }),
+ ]);
+ if (steps.hasMore || events.hasMore) throw new Error("Lifecycle owner audit exceeded its bound.");
+ const names = new Map(steps.data.map((step) => [step.stepId, step.stepName.split("//").at(-1)]));
+ return events.data.flatMap((event) => {
+ if (event.eventType !== "step_completed") return [];
+ const name = names.get(event.correlationId);
+ return name === "wakeTaskAgentRequestParentStep"
+ ? ["agent-request"]
+ : name === "wakeTaskParentStep"
+ ? ["completed"]
+ : [];
+ });
+}
+
+async function boundedBody(request: Request) {
+ const reader = request.body?.getReader();
+ if (reader === undefined) throw new Error("Missing lifecycle request.");
+ let text = "";
+ let bytes = 0;
+ const decoder = new TextDecoder();
+ try {
+ while (true) {
+ const next = await reader.read();
+ if (next.done) return text + decoder.decode();
+ bytes += next.value.byteLength;
+ if (bytes > 4096) throw new Error("Lifecycle request exceeds 4096 bytes.");
+ text += decoder.decode(next.value, { stream: true });
+ }
+ } finally {
+ await reader.cancel();
+ reader.releaseLock();
+ }
+}
diff --git a/e2e/fixtures/fixture-tasks/agent/instructions.md b/e2e/fixtures/fixture-tasks/agent/instructions.md
index f3275e7f0..e2cad432c 100644
--- a/e2e/fixtures/fixture-tasks/agent/instructions.md
+++ b/e2e/fixtures/fixture-tasks/agent/instructions.md
@@ -1 +1 @@
-Follow the scripted model's task-control calls exactly. This fixture validates durable background-task orchestration, not model planning.
+Help the user delegate background work and report the results accurately. Use the requested specialist and preserve the result identifiers in your report.
diff --git a/e2e/fixtures/fixture-tasks/agent/lib/lifecycle-control.ts b/e2e/fixtures/fixture-tasks/agent/lib/lifecycle-control.ts
new file mode 100644
index 000000000..048787703
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/lib/lifecycle-control.ts
@@ -0,0 +1,82 @@
+import { createHook, getWorkflowMetadata } from "workflow";
+import { getRun } from "workflow/api";
+
+export interface LifecycleControlEvent {
+ readonly kind: "gate" | "owner" | "gate-cancelled";
+ readonly marker: "A" | "B" | "parent";
+ readonly runId: string;
+ readonly sessionId: string;
+ readonly turnId: string;
+ readonly token?: string;
+}
+
+export const lifecycleNamespace = (key: string) => `fixture-tasks.lifecycle.${key}`;
+
+export async function publishLifecycleControl(
+ parentSessionId: string,
+ key: string,
+ event: LifecycleControlEvent,
+) {
+ "use step";
+ const ops: Promise<void>[] = [];
+ const writer = getRun(parentSessionId)
+ .getWritable<LifecycleControlEvent>({ namespace: lifecycleNamespace(key), ops })
+ .getWriter();
+ try {
+ await writer.write(event);
+ } finally {
+ writer.releaseLock();
+ }
+ await Promise.all(ops);
+}
+
+export async function lifecycleGate(
+ input: {
+ parentSessionId: string;
+ key: string;
+ marker: "A" | "B" | "parent";
+ sessionId: string;
+ turnId: string;
+ },
+ signal?: AbortSignal,
+) {
+ const gate = createHook<void>({
+ metadata: { parentSessionId: input.parentSessionId, key: input.key },
+ });
+ await publishLifecycleControl(input.parentSessionId, input.key, {
+ kind: "gate",
+ marker: input.marker,
+ token: gate.token,
+ runId: getWorkflowMetadata().workflowRunId,
+ sessionId: input.sessionId,
+ turnId: input.turnId,
+ });
+ if (signal === undefined) {
+ await gate;
+ return;
+ }
+
+ let rejectAbort: ((reason: unknown) => void) | undefined;
+ const aborted = new Promise<never>((_, reject) => {
+ rejectAbort = reject;
+ });
+ const abort = () => rejectAbort?.(signal.reason);
+ signal.addEventListener("abort", abort, { once: true });
+ try {
+ signal.throwIfAborted();
+ await Promise.race([gate, aborted]);
+ } finally {
+ signal.removeEventListener("abort", abort);
+ // Do not await the unresolved hook iterator during cancellation teardown.
+ await gate.dispose();
+ if (signal.aborted) {
+ await publishLifecycleControl(input.parentSessionId, input.key, {
+ kind: "gate-cancelled",
+ marker: input.marker,
+ runId: getWorkflowMetadata().workflowRunId,
+ sessionId: input.sessionId,
+ turnId: input.turnId,
+ });
+ }
+ }
+}
diff --git a/e2e/fixtures/fixture-tasks/agent/lib/lifecycle-model.ts b/e2e/fixtures/fixture-tasks/agent/lib/lifecycle-model.ts
new file mode 100644
index 000000000..ed3854001
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/lib/lifecycle-model.ts
@@ -0,0 +1,65 @@
+import type { MockModelRequest, MockModelResponse } from "eve/evals";
+
+export const LIFECYCLE_SCENARIO = "Alice coordinates Bob's lifecycle ordering check.";
+export const CROSS_TURN_SCENARIO = "Alice launches two pieces of work in separate user turns.";
+const ZERO_USAGE = { inputTokens: 0, outputTokens: 0 };
+const COMPLETION = /^Background task task_[a-z0-9]+ \([^)]+\) is completed\./u;
+
+export function lifecycleModel(request: MockModelRequest): MockModelResponse | undefined {
+ const setup = request.userMessages.find(
+ (message) => message.startsWith(LIFECYCLE_SCENARIO) || message.startsWith(CROSS_TURN_SCENARIO),
+ );
+ if (setup === undefined) return;
+ const key = setup.split(" ").at(-1)!;
+ const crossTurn = setup.startsWith(CROSS_TURN_SCENARIO);
+ const message =
+ [...request.userMessages]
+ .reverse()
+ .find((entry) => entry.startsWith("Alice ") || COMPLETION.test(entry)) ?? "";
+ const result = (id: string) => request.toolResults.find((entry) => entry.id === id);
+ const call = (marker: "A" | "B") => ({
+ id: `lifecycle-${marker}`,
+ name: "lifecycle_task",
+ input: { key, marker, child: !crossTurn && marker === "B" },
+ });
+ const response = (value: Omit<MockModelResponse, "usage">) => ({ ...value, usage: ZERO_USAGE });
+ if (message === "Alice checks that Bob's terminal session has no retained handle.") {
+ const agents =
+ [...request.messages].reverse().find((entry) => entry.text.startsWith("[Agents]"))?.text ??
+ "";
+ return response({
+ text: JSON.stringify([...agents.matchAll(/<agent id="([^"]+)"/gu)].map((match) => match[1])),
+ });
+ }
+ if (COMPLETION.test(message)) {
+ // Report only what the runtime actually delivered, without model-side batching.
+ const notifications = request.userMessages.filter((entry) => COMPLETION.test(entry));
+ return response({ text: JSON.stringify({ report: "LIFECYCLE-REPORT", notifications }) });
+ }
+ if (message === "Alice keeps the parent active while Bob finishes.") {
+ return result("lifecycle-hold") === undefined
+ ? response({ toolCalls: [{ id: "lifecycle-hold", name: "lifecycle_hold", input: { key } }] })
+ : response({ text: "PARENT-RELEASED" });
+ }
+ if (message === "Alice checks that the other piece of work is still pending.") {
+ return response({ text: "PENDING-CHECK-ACK" });
+ }
+ if (message === "Alice performs the final metered accounting check.") {
+ return { text: "ACCOUNTING-CHECK", usage: { inputTokens: 999_790, outputTokens: 0 } };
+ }
+ if (message === "Alice checks the remaining session budget.") {
+ return response({ text: "B-USAGE-WAS-NOT-ACCUMULATED" });
+ }
+ if (message === "Alice confirms that all completion deliveries have been observed.") {
+ return response({ text: "DRAIN-ACK" });
+ }
+ const markers: ("A" | "B")[] = crossTurn
+ ? [request.userMessages.includes("Alice now launches the second piece of work.") ? "B" : "A"]
+ : ["A", "B"];
+ const pending = markers.filter((marker) => result(`lifecycle-${marker}`) === undefined);
+ return response(
+ pending.length > 0
+ ? { toolCalls: pending.map(call) }
+ : { text: `LAUNCHED:${markers.join(",")}` },
+ );
+}
diff --git a/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/agent.ts b/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/agent.ts
index f10500c82..5ec22c8cf 100644
--- a/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/agent.ts
+++ b/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/agent.ts
@@ -4,14 +4,17 @@ import { mockModel, type MockModelRequest, type MockModelResponse } from "eve/ev
const FAN_IN_MARKER_PATTERN = /TASK-FAN-IN-[12]/u;
function respond(request: MockModelRequest): MockModelResponse | string {
- const fanInMarker = FAN_IN_MARKER_PATTERN.exec(request.lastUserMessage ?? "")?.[0];
+ const message = request.lastUserMessage ?? "";
+ const fanInMarker = FAN_IN_MARKER_PATTERN.exec(message)?.[0];
+ const fanoutMarker = /FANOUT-WORKER-\d+/u.exec(message)?.[0];
const released = request.toolResults.find((result) => result.name === "release");
if (released === undefined) {
- return { toolCalls: [{ input: { marker: fanInMarker ?? "RELEASE" }, name: "release" }] };
+ return {
+ toolCalls: [{ input: { marker: fanInMarker ?? fanoutMarker ?? "RELEASE" }, name: "release" }],
+ };
}
if (fanInMarker !== undefined) return `FANOUT-COMPLETE:${fanInMarker}`;
- const message = request.lastUserMessage ?? "";
- const marker = /FANOUT-WORKER-\d+/u.exec(message)?.[0] ?? message;
+ const marker = fanoutMarker ?? message;
return `FANOUT-COMPLETE:${marker}`;
}
diff --git a/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/tools/release.ts b/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/tools/release.ts
index a15076a81..ade1bf2c6 100644
--- a/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/tools/release.ts
+++ b/e2e/fixtures/fixture-tasks/agent/subagents/fanout-worker/tools/release.ts
@@ -4,7 +4,12 @@ import { z } from "zod";
export default defineTool({
description: "Release one fanout task after its parent remains interactive.",
- inputSchema: z.object({ marker: z.enum(["RELEASE", "TASK-FAN-IN-1", "TASK-FAN-IN-2"]) }),
+ inputSchema: z.object({
+ marker: z.union([
+ z.enum(["RELEASE", "TASK-FAN-IN-1", "TASK-FAN-IN-2"]),
+ z.string().regex(/^FANOUT-WORKER-(?:[1-9]|10)$/u),
+ ]),
+ }),
approval: once(),
execute: async ({ marker }) => ({ marker, released: true }),
});
diff --git a/e2e/fixtures/fixture-tasks/agent/subagents/lifecycle-worker/agent.ts b/e2e/fixtures/fixture-tasks/agent/subagents/lifecycle-worker/agent.ts
new file mode 100644
index 000000000..aebe788d9
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/subagents/lifecycle-worker/agent.ts
@@ -0,0 +1,20 @@
+import { defineAgent } from "eve";
+import { mockModel } from "eve/evals";
+
+export default defineAgent({
+ description: "Bob performs metered work before an explicitly gated, terminal model rejection.",
+ model: mockModel((request) => {
+ if (request.toolResults.some((result) => result.name === "release")) {
+ // A permanent provider rejection takes the real delegated-session terminal path.
+ throw Object.assign(new Error("LIFECYCLE-TERMINAL-REJECTION"), { statusCode: 400 });
+ }
+ const message = request.userMessages.find((entry) => entry.includes('"parentSessionId"'));
+ if (message === undefined) throw new Error("Missing lifecycle worker coordinates.");
+ const input = JSON.parse(message.slice(message.indexOf("{")));
+ return {
+ toolCalls: [{ id: "metered-release", name: "release", input }],
+ usage: { inputTokens: 211, outputTokens: 37 },
+ };
+ }),
+ modelContextWindowTokens: 1_000_000,
+});
diff --git a/e2e/fixtures/fixture-tasks/agent/subagents/lifecycle-worker/tools/release.ts b/e2e/fixtures/fixture-tasks/agent/subagents/lifecycle-worker/tools/release.ts
new file mode 100644
index 000000000..8c048d31d
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/subagents/lifecycle-worker/tools/release.ts
@@ -0,0 +1,27 @@
+import {
+ defineWorkflowTool,
+ type WorkflowToolContext,
+ type WorkflowToolDefinition,
+} from "eve/tools";
+import { z } from "zod";
+import { lifecycleGate } from "../../../lib/lifecycle-control.js";
+
+type Input = { parentSessionId: string; key: string; marker: "B" };
+
+async function execute(input: Input, ctx: WorkflowToolContext): Promise<string> {
+ "use workflow";
+ await lifecycleGate({ ...input, sessionId: ctx.session.id, turnId: ctx.session.turn.id });
+ return "BOB-RELEASED";
+}
+
+const tool: WorkflowToolDefinition<Input, string> = defineWorkflowTool({
+ description: "Acknowledge Bob's metered work and wait for the coordinator's release.",
+ inputSchema: z.object({
+ parentSessionId: z.string().min(1),
+ key: z.string().uuid(),
+ marker: z.literal("B"),
+ }),
+ execute,
+});
+
+export default tool;
diff --git a/e2e/fixtures/fixture-tasks/agent/subagents/tool-surface-worker/agent.ts b/e2e/fixtures/fixture-tasks/agent/subagents/tool-surface-worker/agent.ts
new file mode 100644
index 000000000..2f52b2c01
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/subagents/tool-surface-worker/agent.ts
@@ -0,0 +1,13 @@
+import { defineAgent } from "eve";
+import { mockModel } from "eve/evals";
+
+export default defineAgent({
+ description: "Return the child's advertised tool names in its final report to the parent.",
+ model: mockModel(({ tools }) =>
+ JSON.stringify({
+ result: "TASK-CHILD-FINAL-RESULT",
+ tools: tools.map((tool) => tool.name).sort(),
+ }),
+ ),
+ modelContextWindowTokens: 1_000_000,
+});
diff --git a/e2e/fixtures/fixture-tasks/agent/subagents/update-worker/agent.ts b/e2e/fixtures/fixture-tasks/agent/subagents/update-worker/agent.ts
deleted file mode 100644
index dbf48c00d..000000000
--- a/e2e/fixtures/fixture-tasks/agent/subagents/update-worker/agent.ts
+++ /dev/null
@@ -1,31 +0,0 @@
-import { defineAgent } from "eve";
-import { mockModel, type MockModelRequest, type MockModelResponse } from "eve/evals";
-
-function respond(request: MockModelRequest): MockModelResponse | string {
- const result = request.toolResults.find((entry) => entry.id === "task-update-progress");
- if (result === undefined) {
- return {
- toolCalls: [
- {
- id: "task-update-progress",
- input: { message: "TASK-UPDATE-PROGRESS" },
- name: "task_update",
- },
- ],
- };
- }
- if (
- result.output === null ||
- typeof result.output !== "object" ||
- Reflect.get(result.output, "status") !== "sent"
- ) {
- throw new Error("task_update did not confirm delivery.");
- }
- return "TASK-UPDATE-CHILD-DONE";
-}
-
-export default defineAgent({
- description: "Report one deterministic task update to the parent.",
- model: mockModel(respond),
- modelContextWindowTokens: 1_000_000,
-});
diff --git a/e2e/fixtures/fixture-tasks/agent/tools/lifecycle_hold.ts b/e2e/fixtures/fixture-tasks/agent/tools/lifecycle_hold.ts
new file mode 100644
index 000000000..33465d0a7
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/tools/lifecycle_hold.ts
@@ -0,0 +1,30 @@
+import {
+ defineWorkflowTool,
+ type WorkflowToolContext,
+ type WorkflowToolDefinition,
+} from "eve/tools";
+import { z } from "zod";
+import { lifecycleGate } from "../lib/lifecycle-control.js";
+
+async function execute({ key }: { key: string }, ctx: WorkflowToolContext): Promise<string> {
+ "use workflow";
+ await lifecycleGate(
+ {
+ key,
+ parentSessionId: ctx.session.id,
+ marker: "parent",
+ sessionId: ctx.session.id,
+ turnId: ctx.session.turn.id,
+ },
+ ctx.abortSignal,
+ );
+ return "PARENT-RELEASED";
+}
+
+const tool: WorkflowToolDefinition<{ key: string }, string> = defineWorkflowTool({
+ description: "Keep Alice's unrelated parent turn active until its session cancellation.",
+ inputSchema: z.object({ key: z.string().uuid() }),
+ execute,
+});
+
+export default tool;
diff --git a/e2e/fixtures/fixture-tasks/agent/tools/lifecycle_task.ts b/e2e/fixtures/fixture-tasks/agent/tools/lifecycle_task.ts
new file mode 100644
index 000000000..1b8d370da
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/agent/tools/lifecycle_task.ts
@@ -0,0 +1,57 @@
+import {
+ defineWorkflowTool,
+ type WorkflowToolContext,
+ type WorkflowToolDefinition,
+} from "eve/tools";
+import { getWorkflowMetadata } from "workflow";
+import { z } from "zod";
+import { lifecycleGate, publishLifecycleControl } from "../lib/lifecycle-control.js";
+
+type Input = { key: string; marker: "A" | "B"; child: boolean };
+
+async function execute({ key, marker, child }: Input, ctx: WorkflowToolContext): Promise<string> {
+ "use workflow";
+ const parentSessionId = ctx.session.id;
+ await publishLifecycleControl(parentSessionId, key, {
+ kind: "owner",
+ marker,
+ runId: getWorkflowMetadata().workflowRunId,
+ sessionId: parentSessionId,
+ turnId: ctx.session.turn.id,
+ });
+ if (child) {
+ try {
+ await ctx.agent("lifecycle-worker", {
+ message: JSON.stringify({ parentSessionId, key, marker }),
+ });
+ throw new Error("Bob's terminal model rejection was not observed.");
+ } catch (error) {
+ if (
+ error === null ||
+ typeof error !== "object" ||
+ Reflect.get(error, "code") !== "SUBAGENT_EXECUTION_FAILED" ||
+ !String(Reflect.get(error, "message")).includes("LIFECYCLE-TERMINAL-REJECTION")
+ )
+ throw error;
+ }
+ } else {
+ await lifecycleGate({
+ parentSessionId,
+ key,
+ marker,
+ sessionId: ctx.session.id,
+ turnId: ctx.session.turn.id,
+ });
+ }
+ return `LIFECYCLE:${marker}`;
+}
+
+const tool: WorkflowToolDefinition<Input, string> = defineWorkflowTool({
+ description:
+ "Coordinate Alice and Bob's independently released background work for lifecycle checks.",
+ execution: "background",
+ inputSchema: z.object({ key: z.string().uuid(), marker: z.enum(["A", "B"]), child: z.boolean() }),
+ execute,
+});
+
+export default tool;
diff --git a/e2e/fixtures/fixture-tasks/evals/README.md b/e2e/fixtures/fixture-tasks/evals/README.md
index 79651baff..c559ec94f 100644
--- a/e2e/fixtures/fixture-tasks/evals/README.md
+++ b/e2e/fixtures/fixture-tasks/evals/README.md
@@ -1,39 +1,77 @@
# Task eval transitions
-## Completion batching measurement
-
-`task.parent.wake.emitted-ready.batching.eval.ts` launches ten children behind
-approval gates. The burst case releases nine together; the staggered case releases
-each child only after the parent has finished responding to the previous one.
-The last child stays blocked while the eval verifies silence and sends a user
-question, then completes to produce a report containing all ten distinct results.
-
-Each case logs a JSON `task-batching` record with completion-driven parent turns,
-model steps, silent and visible messages, and completions per turn. Setup wakes
-and the user question are excluded. The scripted model always obeys the silence
-policy, so this measures the runtime cost of perfect compliance; the real-model
-prompt ablation lives in `agent-task-reporting`.
-
-Batching is automatic. Both schedules run against the same runtime.
-The first completion's mock model call takes ten seconds, allowing later burst
-completions to enter the active parent's buffer. This delay belongs to the test
-model; the runtime adds no timer. The cases allow three minutes because the
-Vercel staggered run exceeded the previous two-minute timeout.
-
-At `76a18ee1`, the same burst workload with the previous option off/on took
-10/4 parent model steps in the local world. The staggered control took 10/10.
-Those measurements are retained in PR #3144; future runs exercise the default
-behavior and log their actual counts.
-
-Callback timing still varies across workflow worlds. Compare the observed batch
-sizes and model-step counts, not an assumed ten-to-one gain. The unit test at the
-delivery boundary separately proves that 100 buffered sibling completions become
-one parent turn with every payload and its metadata preserved.
-Counts are measurements, not fixed assertions that would prohibit improvements.
-The staggered case is a control for active-parent coalescing: it intentionally
-leaves no opportunity to merge adjacent completions. A policy that withholds all
-intermediate deliveries until cohort settlement needs a different driver because
-this case waits for each delivery before releasing the next child.
+## Full-cohort completion barrier
+
+This fixture uses a static mock model for scripted orchestration checks. Real-model
+coverage lives separately in [`agent-task-reporting`](../../agent-task-reporting/evals/README.md),
+whose agent uses `defineAgent(e2eAgentConfig())` without scenario-based routing.
+
+`task.parent.wake.emitted-ready.batching.eval.ts` releases ten scripted children
+in burst and staggered schedules. Each child needs a separate approval before
+returning its marked result. The driver sends approval responses without waiting
+for a parent model turn and observes child completion before advancing. It retains
+the parent stream cursor from before the releases, so an intermediate completion-driven
+model turn fails even if it emits no text. Both schedules require zero intermediate
+completion model steps and one full-cohort turn. Every known task ID must be
+delivered once, and the report must contain each distinct output once. Both cases
+also require an unrelated user answer while the last child remains blocked. The
+mock neither delays inference nor returns a silence sentinel; a completion-driven
+model invocation before settlement is an error. Each case still logs its
+`task-batching` metrics.
+
+`task.join.evaluate.observed-partial.eval.ts` establishes partial completion on
+one child's stream while the other still awaits approval. An independent user
+status request receives WAITING without a completion-driven model step. Releasing
+the remaining sibling then delivers both task IDs in one COMPLETE turn. The
+transition metadata separates lifecycle readiness from parent admission: successes
+wait for all original siblings, even across unrelated user turns and cohorts;
+input, authorization, failures, and cancellation bypass that hold.
+
+The separate `agent-task-reporting` fixture uses the same approval-gated approach
+with real models for the parent, children, and a nested warehouse lookup. It covers
+an intervening user answer and compaction without waiting for partial parent wakes.
+
+## Lifecycle ordering and overlapping launches
+
+`task.parent.wake.emitted-ready.lifecycle-order.eval.ts` holds an unrelated parent
+turn in a blocking Workflow hook. Task A returns successfully before B's child is
+released. The fixture waits for A's task owner (not its child stream) to finish:
+that owner awaits its real terminal inbox wake before returning. B then receives a
+permanent mock-provider rejection after a successful metered model step. Its
+background workflow catches the expected child failure and completes successfully.
+The task owner's recorded forwarding steps must show invocation, settlement, then
+completion, and the parent hook must still be active after both owners finish.
+This establishes A completed → B agent-settled → B completed in the real queue.
+
+The report must contain exactly both task receipts and outputs, once, through a
+subsequent user-message acknowledgment. A separate follow-up turn echoes the actual
+framework-rendered handle announcement: B's terminal child must not remain as an
+available handle. Another check crosses the parent's configured one-million-input-token
+budget: B supplies 211 input tokens, all ordinary parent responses supply zero,
+and an accounting turn supplies 999,790. The next request must produce the
+runtime-authored session-limit prompt with exactly 1,000,001 used tokens. Dropping
+B's settlement therefore fails usage accounting even if the union report looks
+correct. The enlarged fixture context window prevents this metered probe from
+triggering compaction. B's terminal failure and original 211/37 token step are
+also asserted on its real child stream.
+
+`task.parent.wake.emitted-ready.cross-turn-cohort.eval.ts` launches A, acknowledges
+its pending gate, then launches B in a different user turn. Both task-owner
+creating-turn IDs must match their respective launch events and be distinct. The
+two cases release A first and B first respectively. After the first owner's
+completion acknowledgment, an independent user checkpoint must run without any
+completion delivery while the other gate remains active. The final delivery is
+exactly the union, never one report per creating turn. The mock reports received
+notifications immediately; it does not implement its own cohort barrier.
+
+The `/eve/v1/task-lifecycle/:sessionId/:action` channel is fixture-only and stays
+inside the fixture's Vercel service route prefix. Random per-eval
+keys scope control-stream reads; releases validate gate metadata against that key
+and parent session. Reads, bodies, and owner-log audits are bounded. The route
+only releases authored hooks and reads actual Workflow owner metadata; it never
+inserts task lifecycle payloads, edits session state, or calls production routing
+helpers. No production hook or internal/testing package export is needed. These
+evals run only in CI, without timing sleeps or external services.
## Remote callback routing regression
@@ -52,6 +90,14 @@ separately checks that this service route adds no extra public prefix. Local and
Postgres runs exercise the remote round trip, but only the Vercel build exercises
service-prefix inference.
+## Child tool surface and completion
+
+`task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts` uses a
+mock child in every suite to report its actual advertised tool names. It checks
+that `task_update` is absent and the final report reaches the parent with the
+correct task identity. Tool and workflow progress remain covered separately by
+`agent-background-tools` and `agent-workflow-tools`.
+
## Transition declarations
These evals are executable evidence for the background-task contract. They do
diff --git a/e2e/fixtures/fixture-tasks/evals/batching.ts b/e2e/fixtures/fixture-tasks/evals/batching.ts
index 4e88310e3..de228d876 100644
--- a/e2e/fixtures/fixture-tasks/evals/batching.ts
+++ b/e2e/fixtures/fixture-tasks/evals/batching.ts
@@ -1,74 +1,177 @@
import { type EveEvalContext, type EveEvalTurn, type InputRequest } from "eve/evals";
import { equals } from "eve/evals/expect";
-import { requireSessionStreamIndex, type TaskEvalSessionDriver } from "./shared.js";
+import {
+ requireSessionStreamIndex,
+ waitForChildResult,
+ type TaskEvalSessionDriver,
+} from "./shared.js";
-/** Start the benchmark and wait until every child is blocked on its release tool. */
+/** Release children independently of parent wakes, retaining the entire parent stream suffix. */
export async function startBlockedFanout(t: EveEvalContext, count: number) {
let session: TaskEvalSessionDriver = t;
const started = await t.send("TASK-BATCHING-BENCHMARK");
started.expectOk();
+ started.noFailedActions();
+ started.messageIncludes("TASK-FANOUT-STARTED");
started.calledSubagent("fanout-worker", { count });
- const taskIds = started.events.flatMap((event) =>
+ const receipts = started.events.flatMap((event) =>
event.type === "subagent.completed" && event.data.backgroundTask !== undefined
- ? [event.data.backgroundTask.taskId]
+ ? [{ callId: event.data.callId, taskId: event.data.backgroundTask.taskId }]
: [],
);
+ const taskIds = receipts.map(({ taskId }) => taskId);
await t.require(
{ receipts: taskIds.length, distinct: new Set(taskIds).size },
equals({ receipts: count, distinct: count }),
);
+ const setupEvents: EveEvalTurn["events"][number][] = [];
const requests = new Map<string, InputRequest>();
collectRequests(started);
for (let attempt = 0; requests.size < count && attempt < count; attempt += 1) {
collectRequests(await nextTurn());
}
- await t.require({ releaseRequests: requests.size }, equals({ releaseRequests: count }));
- const remaining = [...requests.values()];
- // Only completion turns contribute to measurements; setup and user turns do not.
+ const children = setupEvents.flatMap((event) => {
+ if (event.type !== "subagent.called") return [];
+ const action = setupEvents
+ .flatMap((entry) => (entry.type === "actions.requested" ? entry.data.actions : []))
+ .find((entry) => entry.callId === event.data.callId);
+ const marker =
+ typeof action?.input.message === "string"
+ ? /FANOUT-WORKER-\d+/u.exec(action.input.message)?.[0]
+ : undefined;
+ const receipt = receipts.find(({ callId }) => callId === event.data.callId);
+ if (marker === undefined || receipt === undefined) {
+ throw new Error("Fanout child has no marked delegation and matching task receipt.");
+ }
+ return [
+ {
+ marker,
+ taskId: receipt.taskId,
+ sessionId: event.data.childSessionId,
+ turnId: event.data.turnId,
+ },
+ ];
+ });
+ await t.require(
+ {
+ children: children.length,
+ sessions: new Set(children.map((child) => child.sessionId)).size,
+ tasks: children.map((child) => child.taskId).sort(),
+ markers: children.map((child) => child.marker).sort(),
+ creatingTurns: new Set(children.map((child) => child.turnId)).size,
+ },
+ equals({
+ children: count,
+ sessions: count,
+ tasks: [...taskIds].sort(),
+ markers: Array.from({ length: count }, (_, index) => `FANOUT-WORKER-${index + 1}`).sort(),
+ creatingTurns: 1,
+ }),
+ );
+
+ await t.require(
+ {
+ markers: [...requests.keys()].sort(),
+ requestIds: new Set([...requests.values()].map((request) => request.requestId)).size,
+ },
+ equals({ markers: children.map((child) => child.marker).sort(), requestIds: count }),
+ );
+ const remaining = [...children];
+ const completedChildren: EveEvalTurn[] = [];
+ const parentTurns: EveEvalTurn[] = [];
const completionTurns: EveEvalTurn[] = [];
- const deliveredCount = () => completionTurns.flatMap(completedTaskIds).length;
+
return {
+ started,
taskIds,
+ completedChildren,
+ parentTurns,
completionTurns,
async completeNext(batchSize: number) {
const batch = remaining.splice(0, batchSize);
- await t.require({ releaseRequests: batch.length }, equals({ releaseRequests: batchSize }));
- observe(
- await session.respond(batch.map(({ requestId }) => ({ requestId, optionId: "approve" }))),
- );
- const expected = count - remaining.length;
- for (let attempt = 0; deliveredCount() < expected && attempt < count; attempt += 1) {
- observe(await nextTurn());
+ await t.require(batch.length, equals(batchSize));
+ // respond() waits for a parent boundary that the cohort barrier deliberately withholds.
+ await post({
+ inputResponses: batch.map(({ marker }) => ({
+ requestId: requests.get(marker)!.requestId,
+ optionId: "approve",
+ })),
+ });
+ for (const child of batch) {
+ completedChildren.push(
+ await waitForChildResult(t, child.sessionId, `FANOUT-COMPLETE:${child.marker}`),
+ );
+ }
+ if (remaining.length === 0) {
+ for (let attempt = 0; attempt < count + 3; attempt += 1) {
+ const turn = observe(await nextTurn());
+ if (completedTaskIds(turn).length > 0) {
+ await t.require(completedTaskIds(turn).sort(), equals([...taskIds].sort()));
+ return;
+ }
+ turn.notEvent("step.started");
+ }
+ throw new Error("The settled cohort produced no completion report.");
}
- await t.require(
- { completedDeliveries: deliveredCount() },
- equals({ completedDeliveries: expected }),
- );
},
async send(message: string) {
- return observe(await session.send(message));
+ await post({ message, turnPolicy: "queue" });
+ for (let attempt = 0; attempt < count + 3; attempt += 1) {
+ const turn = observe(await nextTurn());
+ await t.require(completedTaskIds(turn), equals([]));
+ if (
+ turn.events.some(
+ (event) => event.type === "message.received" && event.data.message === message,
+ )
+ )
+ return turn;
+ turn.notEvent("step.started");
+ }
+ throw new Error("The user message did not run while a sibling was blocked.");
},
};
function collectRequests(turn: EveEvalTurn) {
+ setupEvents.push(...turn.events);
for (const request of turn.inputRequests) {
- if (request.action.toolName === "release") requests.set(request.requestId, request);
+ if (request.action.toolName !== "release") continue;
+ const marker = request.action.input.marker;
+ if (
+ typeof marker !== "string" ||
+ (requests.has(marker) && requests.get(marker)!.requestId !== request.requestId)
+ ) {
+ throw new Error("Fanout release requests must have distinct child markers.");
+ }
+ requests.set(marker, request);
}
}
function observe(turn: EveEvalTurn) {
- turn.expectOk();
+ parentTurns.push(turn);
if (completedTaskIds(turn).length > 0) completionTurns.push(turn);
return turn;
}
+ async function post(body: unknown) {
+ const response = await t.target.fetch(
+ `/eve/v1/session/${encodeURIComponent(started.sessionId)}`,
+ {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify(body),
+ signal: t.signal,
+ },
+ );
+ await t.require(response.status, equals(202));
+ await response.body?.cancel();
+ }
+
async function nextTurn() {
- if (session.sessionId === undefined) throw new Error("Fanout has no parent session id.");
- const live = t.target.watchTurn(session.sessionId, {
- startIndex: requireSessionStreamIndex(session, "Batching benchmark"),
+ const live = t.target.watchTurn(started.sessionId, {
+ startIndex: requireSessionStreamIndex(session, "Cohort parent"),
});
const turn = await live.result();
session = live.session;
diff --git a/e2e/fixtures/fixture-tasks/evals/lifecycle-model.test.mjs b/e2e/fixtures/fixture-tasks/evals/lifecycle-model.test.mjs
new file mode 100644
index 000000000..df69f752a
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/evals/lifecycle-model.test.mjs
@@ -0,0 +1,77 @@
+import assert from "node:assert/strict";
+import { test } from "node:test";
+import {
+ CROSS_TURN_SCENARIO,
+ LIFECYCLE_SCENARIO,
+ lifecycleModel,
+} from "../agent/lib/lifecycle-model.ts";
+
+const key = "00000000-0000-4000-8000-000000000001";
+const instruction =
+ "Background task reporting: launch acknowledgement\nAcknowledge the accepted work.";
+const taskState = '[Task state]\n{"tasks":[{"taskId":"task_a","status":"pending"}]}';
+
+function request(userMessages, markers = []) {
+ return {
+ userMessages,
+ messages: userMessages.map((text) => ({ role: "user", text })),
+ toolResults: markers.map((marker) => ({
+ id: `lifecycle-${marker}`,
+ name: "lifecycle_task",
+ output: { status: "working", taskId: `task_${marker.toLowerCase()}` },
+ })),
+ };
+}
+
+for (const [scenario, markers] of [
+ [CROSS_TURN_SCENARIO, ["A"]],
+ [LIFECYCLE_SCENARIO, ["A", "B"]],
+]) {
+ test(`${scenario} acknowledges receipts behind framework instructions`, () => {
+ const messages = [`${scenario} ${key}`, taskState, instruction];
+ assert.deepEqual(lifecycleModel(request(messages, markers)), {
+ text: `LAUNCHED:${markers.join(",")}`,
+ usage: { inputTokens: 0, outputTokens: 0 },
+ });
+ });
+}
+
+test("second-turn receipts are acknowledged without a premature report", () => {
+ const messages = [
+ `${CROSS_TURN_SCENARIO} ${key}`,
+ taskState,
+ instruction,
+ "Alice now launches the second piece of work.",
+ taskState,
+ instruction,
+ ];
+ assert.equal(lifecycleModel(request(messages, ["A", "B"])).text, "LAUNCHED:B");
+});
+
+test("reports every actual completion, not framework reporting instructions", () => {
+ const a = "Background task task_a (lifecycle_task) is completed.\n\nResult:\nLIFECYCLE:A";
+ const b = "Background task task_b (lifecycle_task) is completed.\n\nResult:\nLIFECYCLE:B";
+ const messages = [`${LIFECYCLE_SCENARIO} ${key}`, instruction, a, b, taskState, instruction];
+ const response = lifecycleModel(request(messages, ["A", "B"]));
+ assert.deepEqual(JSON.parse(response.text), {
+ report: "LIFECYCLE-REPORT",
+ notifications: [a, b],
+ });
+});
+
+test("does not implement a model-side barrier that could hide a partial runtime delivery", () => {
+ const a = "Background task task_a (lifecycle_task) is completed.\n\nResult:\nLIFECYCLE:A";
+ const response = lifecycleModel(
+ request([`${CROSS_TURN_SCENARIO} ${key}`, a, instruction], ["A", "B"]),
+ );
+ assert.deepEqual(JSON.parse(response.text).notifications, [a]);
+});
+
+test("continues to answer ordinary user messages after a completed cohort", () => {
+ const messages = [
+ `${LIFECYCLE_SCENARIO} ${key}`,
+ "Background task task_a (lifecycle_task) is completed.\n\nResult:\nLIFECYCLE:A",
+ "Alice confirms that all completion deliveries have been observed.",
+ ];
+ assert.equal(lifecycleModel(request(messages, ["A", "B"])).text, "DRAIN-ACK");
+});
diff --git a/e2e/fixtures/fixture-tasks/evals/lifecycle.ts b/e2e/fixtures/fixture-tasks/evals/lifecycle.ts
new file mode 100644
index 000000000..8660411b4
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/evals/lifecycle.ts
@@ -0,0 +1,200 @@
+import type { EveEvalContext, EveEvalTurn } from "eve/evals";
+import { equals } from "eve/evals/expect";
+import type { LifecycleControlEvent } from "../agent/lib/lifecycle-control.js";
+import { completedTaskIds } from "./batching.js";
+import { requireSessionStreamIndex } from "./shared.js";
+
+export function lifecycleDriver(t: EveEvalContext, key: string) {
+ let index = 0;
+ const controls: LifecycleControlEvent[] = [];
+ const turns: EveEvalTurn[] = [];
+ let streamIndex = 0;
+ let sessionId: string;
+ const taskIds = new Map<string, string>();
+
+ async function post<T>(action: string, body: object): Promise<T> {
+ const response = await t.target.fetch(
+ `/eve/v1/task-lifecycle/${encodeURIComponent(sessionId)}/${action}`,
+ {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify({ key, ...body }),
+ signal: t.signal,
+ },
+ );
+ await t.require(response.status, equals(200));
+ return (await response.json()) as T;
+ }
+
+ function record(turn: EveEvalTurn) {
+ turns.push(turn);
+ for (const call of turn.toolCalls) {
+ if (call.name !== "lifecycle_task") continue;
+ const marker = call.input.marker;
+ const output = call.output;
+ if (
+ typeof marker !== "string" ||
+ output === null ||
+ typeof output !== "object" ||
+ !("taskId" in output) ||
+ typeof output.taskId !== "string"
+ )
+ throw new Error("Missing lifecycle task receipt.");
+ taskIds.set(marker, output.taskId);
+ }
+ return turn;
+ }
+
+ async function nextTurn() {
+ const live = t.target.watchTurn(sessionId, { startIndex: streamIndex });
+ const turn = (await live.result()).expectOk();
+ streamIndex = requireSessionStreamIndex(live.session, "Lifecycle parent");
+ return record(turn);
+ }
+
+ return {
+ turns,
+ controls,
+ taskIds,
+ async start(message: string) {
+ const turn = (await t.send(message)).expectOk();
+ sessionId = turn.sessionId;
+ streamIndex = requireSessionStreamIndex(t, "Lifecycle setup");
+ return record(turn);
+ },
+ async gate(marker: "A" | "B" | "parent") {
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const found = controls.find((event) => event.kind === "gate" && event.marker === marker);
+ if (found !== undefined) return found;
+ controls.push(await post<LifecycleControlEvent>("next", { index: index++ }));
+ }
+ throw new Error(`No ${marker} lifecycle gate acknowledgment.`);
+ },
+ async release(gate: LifecycleControlEvent) {
+ if (gate.token === undefined) throw new Error("Missing fixture gate token.");
+ await t.require(await post("release", { token: gate.token }), equals({ released: true }));
+ },
+ async settled(marker: "A" | "B", agent = false) {
+ await t.require(
+ await post("settled", { marker }),
+ equals({
+ marker,
+ status: "completed",
+ deliveries: agent ? ["agent-request", "agent-request", "completed"] : ["completed"],
+ }),
+ );
+ },
+ async active(gate: LifecycleControlEvent) {
+ await t.require(await post("active", { token: gate.token }), equals({ status: "running" }));
+ },
+ async cancelHeldTurn(parent: LifecycleControlEvent) {
+ if (parent.kind !== "gate" || parent.marker !== "parent" || parent.sessionId !== sessionId) {
+ throw new Error("Cancellation must address this session's acknowledged parent gate.");
+ }
+ // Use the same FIFO session inbox as task wakes, never a private turn/gate hook.
+ // Omit tasks: cancellation must not retire either background task's ownership.
+ const response = await t.target.fetch(
+ `/eve/v1/session/${encodeURIComponent(sessionId)}/cancel`,
+ {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify({ turnId: parent.turnId }),
+ signal: t.signal,
+ },
+ );
+ await t.require(response.status, equals(202));
+ await t.require(await response.json(), equals({ ok: true, status: "accepted", sessionId }));
+ },
+ async parentGateCancelled(parent: LifecycleControlEvent) {
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const acknowledgement = controls.find(
+ (event) => event.kind === "gate-cancelled" && event.marker === "parent",
+ );
+ if (acknowledgement !== undefined) {
+ await t.require(
+ acknowledgement,
+ equals({
+ kind: "gate-cancelled",
+ marker: "parent",
+ runId: parent.runId,
+ sessionId: parent.sessionId,
+ turnId: parent.turnId,
+ }),
+ );
+ return;
+ }
+ controls.push(await post<LifecycleControlEvent>("next", { index: index++ }));
+ }
+ throw new Error("The cancelled parent gate did not acknowledge hook disposal.");
+ },
+ async enqueue(message: string) {
+ const response = await t.target.fetch(`/eve/v1/session/${encodeURIComponent(sessionId)}`, {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify({ message, turnPolicy: "queue" }),
+ signal: t.signal,
+ });
+ await t.require(response.status, equals(202));
+ await response.body?.cancel();
+ },
+ nextTurn,
+ async through(message: string, allowCompletions = false) {
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const turn = await nextTurn();
+ if (!allowCompletions) await t.require(completedTaskIds(turn), equals([]));
+ if (
+ turn.events.some(
+ (event) => event.type === "message.received" && event.data.message === message,
+ )
+ )
+ return turn;
+ }
+ throw new Error("Lifecycle checkpoint message was not acknowledged.");
+ },
+ async report() {
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const turn = await nextTurn();
+ if (completedTaskIds(turn).length > 0) return turn;
+ }
+ throw new Error("Lifecycle cohort did not deliver a completion report.");
+ },
+ };
+}
+
+export function assertLifecycleUnion(
+ t: EveEvalContext,
+ driver: ReturnType<typeof lifecycleDriver>,
+) {
+ const completions = driver.turns.filter((turn) => completedTaskIds(turn).length > 0);
+ const expected = [...driver.taskIds.values()].sort();
+ t.check(
+ driver.turns.flatMap((turn) => turn.toolCalls).filter((call) => call.name === "lifecycle_task")
+ .length,
+ equals(2),
+ ).label("exactly two background launches, without replacement or duplicate tasks");
+ t.check(
+ completions.map((turn) => completedTaskIds(turn).sort()),
+ equals([expected]),
+ ).label("one completion turn containing exactly the union of both launch receipts");
+ t.check(
+ completions.flatMap((turn) => turn.events.filter((event) => event.type === "step.started"))
+ .length,
+ equals(1),
+ ).label("no extra completion-driven model steps");
+ const messages = completions.flatMap((turn) =>
+ turn.events.filter(
+ (event) => event.type === "message.completed" && event.data.message !== null,
+ ),
+ );
+ t.check(messages.length, equals(1)).label("exactly one visible final report");
+ const report = completions[0]?.message;
+ if (report === undefined) throw new Error("Missing union report.");
+ const parsed = JSON.parse(report) as { report: string; notifications: string[] };
+ t.check(parsed.report, equals("LIFECYCLE-REPORT"));
+ for (const marker of ["A", "B"]) {
+ t.check(
+ parsed.notifications.join("\n").split(`LIFECYCLE:${marker}`).length - 1,
+ equals(1),
+ ).label(`${marker} output appears exactly once in the final union`);
+ }
+}
diff --git a/e2e/fixtures/fixture-tasks/evals/shared.ts b/e2e/fixtures/fixture-tasks/evals/shared.ts
index 769312255..a4b5c9f45 100644
--- a/e2e/fixtures/fixture-tasks/evals/shared.ts
+++ b/e2e/fixtures/fixture-tasks/evals/shared.ts
@@ -1,5 +1,5 @@
import type { EveEvalContext, EveEvalSession, EveEvalTurn, InputRequest } from "eve/evals";
-import { satisfies } from "eve/evals/expect";
+import { equals, satisfies } from "eve/evals/expect";
export type TaskEvalSessionDriver = Pick<
EveEvalSession,
@@ -31,6 +31,25 @@ export function requireSessionStreamIndex(
return state.streamIndex;
}
+/** Observe completion on a child's stream without requiring a parent notification. */
+export async function waitForChildResult(
+ t: EveEvalContext,
+ sessionId: string,
+ expected: string,
+): Promise<EveEvalTurn> {
+ let startIndex = 0;
+ for (let attempt = 0; attempt < 4; attempt += 1) {
+ const live = t.target.watchTurn(sessionId, { startIndex });
+ const turn = (await live.result()).expectOk();
+ startIndex = requireSessionStreamIndex(live.session, "Child completion");
+ if (turn.message === undefined || !turn.events.some((event) => event.type === "turn.completed"))
+ continue;
+ await t.require(turn.message, equals(expected));
+ return turn;
+ }
+ throw new Error(`Child ${sessionId} did not complete with its expected result.`);
+}
+
/** Waits across server-initiated parent turns for one task-owned input request. */
export async function waitForTaskInput(
t: EveEvalContext,
diff --git a/e2e/fixtures/fixture-tasks/evals/task-transition.ts b/e2e/fixtures/fixture-tasks/evals/task-transition.ts
index 0caa15df6..7da4176fc 100644
--- a/e2e/fixtures/fixture-tasks/evals/task-transition.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task-transition.ts
@@ -12,6 +12,7 @@ interface StatePattern {
readonly parent?: "active" | "parked" | "finalizing";
readonly ownership?: "owned" | "unowned";
readonly usage?: "absent" | "retained";
+ readonly cohort?: "pending" | "terminal";
}
type SemanticInput =
@@ -50,6 +51,7 @@ type SemanticSideEffect =
| "child-abort"
| "parent-model-step"
| "parent-wake"
+ | "completion-hold"
| "task-view-read"
| "authorization-request";
@@ -88,7 +90,8 @@ export const TASK_TRANSITIONS = {
outcome: "accepted",
postState: { lifecycle: "completed", outstandingInput: "none" },
events: { emitted: ["task-ready-notification"] },
- sideEffects: { executed: ["task-view-append", "parent-wake"] },
+ // Lifecycle readiness is distinct from admitting the notification to a parent turn.
+ sideEffects: { executed: ["task-view-append"] },
},
}),
"task.lifecycle.fail.accepted-nonterminal": transition({
@@ -386,7 +389,10 @@ export const TASK_TRANSITIONS = {
"task.parent.wake.emitted-ready": transition({
preState: { lifecycle: ["input_required", "completed", "failed", "cancelled"] },
input: "task-notification",
- guards: ["task-entered-a-ready-status"],
+ guards: [
+ "task-entered-a-ready-status",
+ "status-is-not-completed-or-all-same-createdByTurnId-siblings-are-terminal",
+ ],
expected: {
outcome: "emitted",
postState: { lifecycle: ["input_required", "completed", "failed", "cancelled"] },
@@ -394,6 +400,24 @@ export const TASK_TRANSITIONS = {
sideEffects: { executed: ["parent-wake"] },
},
}),
+ "task.parent.wake.noop-pending-cohort": transition({
+ preState: { lifecycle: "completed", cohort: "pending" },
+ input: "task-notification",
+ guards: [
+ "a-same-createdByTurnId-sibling-is-nonterminal",
+ "unrelated-user-turns-and-cohorts-do-not-release-held-successes",
+ "input-authorization-failure-and-cancellation-bypass-the-hold",
+ ],
+ expected: {
+ outcome: "noop",
+ postState: { lifecycle: "completed", cohort: "pending" },
+ events: {},
+ sideEffects: {
+ executed: ["completion-hold"],
+ suppressed: ["parent-wake", "parent-model-step"],
+ },
+ },
+ }),
"task.parent-interaction.send.accepted-live-children": transition({
preState: { agent: "busy", parent: "active" },
input: "parent-message",
@@ -406,25 +430,25 @@ export const TASK_TRANSITIONS = {
},
}),
"task.join.evaluate.observed-partial": transition({
- preState: { lifecycle: ["completed", "input_required"] },
- input: "task-peek",
- guards: ["at-least-one-task-is-nonterminal"],
+ preState: { lifecycle: ["completed", "input_required"], cohort: "pending" },
+ input: "parent-message",
+ guards: ["one-child-has-completed", "a-sibling-still-awaits-input", "independent-user-message"],
expected: {
outcome: "observed",
- postState: { lifecycle: ["completed", "input_required"] },
+ postState: { lifecycle: ["completed", "input_required"], cohort: "pending" },
events: {},
- sideEffects: { executed: ["task-view-read", "parent-model-step"] },
+ sideEffects: { executed: ["parent-model-step"] },
},
}),
"task.join.evaluate.observed-all-terminal": transition({
- preState: { lifecycle: "completed" },
- input: "task-peek",
- guards: ["all-joined-tasks-are-terminal"],
+ preState: { lifecycle: "completed", cohort: "terminal" },
+ input: "task-notification",
+ guards: ["all-joined-tasks-are-terminal", "every-result-is-in-the-cohort-notification"],
expected: {
outcome: "observed",
- postState: { lifecycle: "completed" },
+ postState: { lifecycle: "completed", cohort: "terminal" },
events: {},
- sideEffects: { executed: ["task-view-read", "parent-model-step"] },
+ sideEffects: { executed: ["parent-model-step"] },
},
}),
} as const satisfies Record<string, TransitionSpec>;
diff --git a/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts
index f76fd0bb6..7a23c0109 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.join.evaluate.observed-partial.eval.ts
@@ -1,22 +1,20 @@
-import { type EveEvalContext, type EveEvalTurn, type InputRequest } from "eve/evals";
-import { satisfies } from "eve/evals/expect";
+import type { EveEvalTurn, InputRequest } from "eve/evals";
+import { equals } from "eve/evals/expect";
+import { completedTaskIds, completionMetrics } from "./batching.js";
import { defineTaskEval } from "./task-transition.js";
-import { requireSessionStreamIndex, type TaskEvalSessionDriver } from "./shared.js";
+import {
+ requireSessionStreamIndex,
+ waitForChildResult,
+ type TaskEvalSessionDriver,
+} from "./shared.js";
-const FAN_IN_SIZE = 2;
-const MAX_WAKE_TURNS = 8;
-const FAN_IN_CALLS = [
- { callId: "task-fan-in-1", marker: "TASK-FAN-IN-1" },
- { callId: "task-fan-in-2", marker: "TASK-FAN-IN-2" },
-] as const;
+const STATUS_REQUEST = "TASK-FAN-IN-STATUS";
+const MARKERS = ["TASK-FAN-IN-1", "TASK-FAN-IN-2"] as const;
-type FanInMarker = (typeof FAN_IN_CALLS)[number]["marker"];
-
-/** A join remains waiting when its exact task set contains one nonterminal task. */
export default defineTaskEval({
description:
- "A join observes one completed and one input-required task, answers WAITING, and does not answer COMPLETE.",
+ "An independent user turn observes an incomplete join after one child completes while its sibling still awaits input; only the settled cohort triggers a completion turn.",
transition: {
primary: "task.join.evaluate.observed-partial",
setup: [
@@ -24,176 +22,148 @@ export default defineTaskEval({
"task.input.require.accepted-valid-batch",
"task.input.answer.accepted-complete",
"task.lifecycle.complete.accepted-nonterminal",
- "task.parent.wake.emitted-ready",
+ "task.parent.wake.noop-pending-cohort",
],
- dimensions: { transport: "local", parentPhase: "active" },
+ dimensions: { transport: "local", parentPhase: "parked" },
},
async test(t) {
- const started = await t.send("TASK-FAN-IN");
- started.expectOk();
+ const started = (await t.send("TASK-FAN-IN")).expectOk();
started.messageIncludes("TASK-FAN-IN-STARTED");
- started.calledSubagent("fanout-worker", { count: FAN_IN_SIZE });
-
- const tasksByMarker = backgroundTasksByMarker(started);
- const taskIds = [...tasksByMarker.values()];
- await requireDistinctTasks(t, taskIds);
- const blocked = await waitForReleaseRequests(t, t, started);
-
- const marker2 = "TASK-FAN-IN-2";
- const marker2TaskId = requireMappedValue(tasksByMarker, marker2, "background task");
- const marker2Request = requireMappedValue(blocked.requestsByMarker, marker2, "release request");
-
- const firstReleased = await blocked.session.respond([
- { optionId: "approve", requestId: marker2Request.requestId },
- ]);
- firstReleased.expectOk();
-
- const waiting = await waitForTurnMessage(
- t,
- blocked.session,
- "TASK-FAN-IN-WAITING",
- firstReleased,
+ started.calledSubagent("fanout-worker", { count: MARKERS.length });
+ let session: TaskEvalSessionDriver = t;
+ const requests = new Map<string, InputRequest>();
+ const setupEvents: EveEvalTurn["events"][number][] = [];
+ collectRequests(started);
+ for (let attempt = 0; requests.size < 2 && attempt < 8; attempt += 1)
+ collectRequests(await nextTurn());
+ await t.require([...requests.keys()].sort(), equals([...MARKERS]));
+ await t.require(
+ new Set([...requests.values()].map((request) => request.requestId)).size,
+ equals(2),
);
+
+ const children = MARKERS.map((marker) => {
+ const callId = marker.toLowerCase();
+ const called = setupEvents.find(
+ (event) => event.type === "subagent.called" && event.data.callId === callId,
+ );
+ const receipt = started.events.find(
+ (event) => event.type === "subagent.completed" && event.data.callId === callId,
+ );
+ if (
+ called?.type !== "subagent.called" ||
+ receipt?.type !== "subagent.completed" ||
+ receipt.data.backgroundTask === undefined
+ ) {
+ throw new Error(`No child session and task receipt for ${marker}.`);
+ }
+ return {
+ marker,
+ sessionId: called.data.childSessionId,
+ taskId: receipt.data.backgroundTask.taskId,
+ turnId: called.data.turnId,
+ };
+ });
+ const taskIds = children.map((child) => child.taskId);
await t.require(
- waiting.observedTurns,
- satisfies(
- (turns: readonly EveEvalTurn[]) =>
- turns.some((turn) =>
- hasCompletedNotification(turn, marker2TaskId, `FANOUT-COMPLETE:${marker2}`),
- ),
- "the join receives the completed task's result in its notification",
- ),
+ {
+ tasks: new Set(taskIds).size,
+ sessions: new Set(children.map((child) => child.sessionId)).size,
+ creatingTurns: new Set(children.map((child) => child.turnId)).size,
+ },
+ equals({ tasks: 2, sessions: 2, creatingTurns: 1 }),
);
- for (const turn of waiting.observedTurns) {
- await t.require(
- turn.message ?? "",
- satisfies(
- (message) => !String(message).includes("TASK-FAN-IN-COMPLETE"),
- "no COMPLETE answer while one joined task is nonterminal",
+
+ await release("TASK-FAN-IN-2");
+ await waitForChildResult(t, children[1]!.sessionId, "FANOUT-COMPLETE:TASK-FAN-IN-2");
+ await post({ message: STATUS_REQUEST, turnPolicy: "queue" });
+ const partial: EveEvalTurn[] = [];
+ let answered = false;
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const turn = await nextTurn();
+ partial.push(turn);
+ await t.require(completedTaskIds(turn), equals([]));
+ if (
+ turn.events.some(
+ (event) => event.type === "message.received" && event.data.message === STATUS_REQUEST,
+ )
+ ) {
+ await t.require(turn.message, equals("TASK-FAN-IN-WAITING"));
+ turn.usedNoTools();
+ answered = true;
+ break;
+ }
+ turn.notEvent("step.started");
+ }
+ await t.require(answered, equals(true));
+ await t.require(completionMetrics(partial).modelSteps, equals(1));
+ const pendingRequestId = requests.get("TASK-FAN-IN-1")!.requestId;
+ t.check(
+ partial
+ .flatMap((turn) => turn.events)
+ .some(
+ (event) =>
+ event.type === "input.resolved" &&
+ event.data.resolutions.some((resolution) => resolution.requestId === pendingRequestId),
),
- );
+ equals(false),
+ ).label("the other sibling still awaits its own approval during the user answer");
+
+ await release("TASK-FAN-IN-1");
+ await waitForChildResult(t, children[0]!.sessionId, "FANOUT-COMPLETE:TASK-FAN-IN-1");
+ let reported = false;
+ for (let attempt = 0; attempt < 8; attempt += 1) {
+ const turn = await nextTurn();
+ if (completedTaskIds(turn).length > 0) {
+ await t.require(completedTaskIds(turn).sort(), equals([...taskIds].sort()));
+ await t.require(turn.message, equals("TASK-FAN-IN-COMPLETE"));
+ turn.event("step.started", { count: 1 });
+ reported = true;
+ break;
+ }
+ turn.notEvent("step.started");
}
+ await t.require(reported, equals(true));
+ t.calledSubagent("fanout-worker", { count: 2 });
t.notCalledTool("task_peek");
t.noFailedActions();
- },
-});
-
-async function requireDistinctTasks(t: EveEvalContext, taskIds: readonly string[]): Promise<void> {
- await t.require(
- taskIds,
- satisfies(
- (ids: readonly string[]) => ids.length === FAN_IN_SIZE && new Set(ids).size === FAN_IN_SIZE,
- `${FAN_IN_SIZE} distinct background task receipts`,
- ),
- );
-}
-function hasCompletedNotification(turn: EveEvalTurn, taskId: string, result: string): boolean {
- return turn.events.some(
- (event) =>
- event.type === "message.received" &&
- typeof event.data.message === "string" &&
- event.data.message.includes(`Background task ${taskId} (`) &&
- event.data.message.includes(" is completed.") &&
- event.data.message.includes(result),
- );
-}
-
-interface BlockedFanIn {
- readonly requestsByMarker: ReadonlyMap<FanInMarker, InputRequest>;
- readonly session: TaskEvalSessionDriver;
-}
-
-async function waitForReleaseRequests(
- t: EveEvalContext,
- initialSession: TaskEvalSessionDriver,
- initialTurn: EveEvalTurn,
-): Promise<BlockedFanIn> {
- const requestsByMarker = new Map<FanInMarker, InputRequest>();
- let session = initialSession;
- collectReleaseRequests(initialTurn.inputRequests, requestsByMarker);
- for (
- let attempt = 0;
- attempt < MAX_WAKE_TURNS && requestsByMarker.size < FAN_IN_SIZE;
- attempt += 1
- ) {
- const sessionId = session.sessionId;
- if (sessionId === undefined) throw new Error("Fan-in has no parent session id.");
- const live = t.target.watchTurn(sessionId, {
- startIndex: requireSessionStreamIndex(session, "Fan-in release wait"),
- });
- const turn = await live.result();
- collectReleaseRequests(turn.inputRequests, requestsByMarker);
- session = live.session;
- }
- if (requestsByMarker.size !== FAN_IN_SIZE) {
- throw new Error(
- `Expected ${FAN_IN_SIZE} marked release requests; received ${requestsByMarker.size}.`,
- );
- }
- return { requestsByMarker, session };
-}
-
-function collectReleaseRequests(
- inputRequests: readonly InputRequest[],
- requestsByMarker: Map<FanInMarker, InputRequest>,
-): void {
- for (const request of inputRequests) {
- if (request.action.toolName !== "release") continue;
- const marker = request.action.input.marker;
- if (isFanInMarker(marker)) requestsByMarker.set(marker, request);
- }
-}
-
-function isFanInMarker(value: unknown): value is FanInMarker {
- return FAN_IN_CALLS.some(({ marker }) => marker === value);
-}
-
-interface ObservedTurnMessage {
- readonly observedTurns: readonly EveEvalTurn[];
- readonly turn: EveEvalTurn;
-}
+ function collectRequests(turn: EveEvalTurn) {
+ setupEvents.push(...turn.events);
+ for (const request of turn.inputRequests) {
+ const marker = request.action.input.marker;
+ if (request.action.toolName === "release" && typeof marker === "string")
+ requests.set(marker, request);
+ }
+ }
-async function waitForTurnMessage(
- t: EveEvalContext,
- initialSession: TaskEvalSessionDriver,
- token: string,
- initialTurn: EveEvalTurn,
-): Promise<ObservedTurnMessage> {
- const observedTurns: EveEvalTurn[] = [initialTurn];
- let session = initialSession;
- if ((initialTurn.message ?? "").includes(token)) return { observedTurns, turn: initialTurn };
- for (let attempt = 0; attempt < MAX_WAKE_TURNS; attempt += 1) {
- const sessionId = session.sessionId;
- if (sessionId === undefined) throw new Error("Fan-in has no parent session id.");
- const live = t.target.watchTurn(sessionId, {
- startIndex: requireSessionStreamIndex(session, "Fan-in message wait"),
- });
- const turn = await live.result();
- observedTurns.push(turn);
- session = live.session;
- if ((turn.message ?? "").includes(token)) return { observedTurns, turn };
- }
- throw new Error(`No turn carried "${token}" after ${MAX_WAKE_TURNS} turns.`);
-}
+ async function release(marker: string) {
+ await post({
+ inputResponses: [{ optionId: "approve", requestId: requests.get(marker)!.requestId }],
+ });
+ }
-function backgroundTasksByMarker(turn: EveEvalTurn): ReadonlyMap<FanInMarker, string> {
- const tasksByMarker = new Map<FanInMarker, string>();
- for (const event of turn.events) {
- if (event.type !== "subagent.completed" || event.data.backgroundTask === undefined) continue;
- const fanInCall = FAN_IN_CALLS.find(({ callId }) => callId === event.data.callId);
- if (fanInCall !== undefined)
- tasksByMarker.set(fanInCall.marker, event.data.backgroundTask.taskId);
- }
- return tasksByMarker;
-}
+ async function post(body: unknown) {
+ const response = await t.target.fetch(
+ `/eve/v1/session/${encodeURIComponent(started.sessionId)}`,
+ {
+ method: "POST",
+ headers: { "content-type": "application/json" },
+ body: JSON.stringify(body),
+ signal: t.signal,
+ },
+ );
+ await response.body?.cancel();
+ await t.require(response.status, equals(202));
+ }
-function requireMappedValue<T>(
- values: ReadonlyMap<FanInMarker, T>,
- marker: FanInMarker,
- description: string,
-): T {
- const value = values.get(marker);
- if (value === undefined) throw new Error(`Fan-in has no ${description} for ${marker}.`);
- return value;
-}
+ async function nextTurn() {
+ const live = t.target.watchTurn(started.sessionId, {
+ startIndex: requireSessionStreamIndex(session, "Partial join"),
+ });
+ const turn = (await live.result()).expectOk();
+ session = live.session;
+ return turn;
+ }
+ },
+});
diff --git a/e2e/fixtures/fixture-tasks/evals/task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts
new file mode 100644
index 000000000..2c5494f29
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/evals/task.lifecycle.complete.accepted-nonterminal.child-tool-surface.eval.ts
@@ -0,0 +1,63 @@
+import { satisfies } from "eve/evals/expect";
+
+import { defineTaskEval } from "./task-transition.js";
+import { requireBackgroundTaskId, waitForTaskNotification } from "./shared.js";
+
+export default defineTaskEval({
+ description:
+ "A background child no longer advertises task_update, and its final tool report still reaches the parent.",
+ transition: {
+ primary: "task.lifecycle.complete.accepted-nonterminal",
+ setup: ["task.dispatch.start.accepted-acknowledged"],
+ dimensions: { transport: "local" },
+ },
+ async test(t) {
+ const started = await t.send(
+ "Alice asks Bob to summarize the available tools for a background task.",
+ );
+ started.expectOk();
+ started.calledSubagent("tool-surface-worker", { count: 1 });
+ started.event("subagent.completed", {
+ count: 1,
+ data: {
+ backgroundTask: { status: "working" },
+ callId: "task-child-tool-surface",
+ subagentName: "tool-surface-worker",
+ },
+ });
+ const taskId = requireBackgroundTaskId(started);
+
+ const completed = await waitForTaskNotification(t, t, taskId, "completed", [started]);
+ completed.turn.expectOk();
+ const report = completed.turn.message;
+ if (report === undefined) throw new Error("Parent did not return the child's tool report.");
+ await t.require(
+ JSON.parse(report) as unknown,
+ satisfies((value) => {
+ if (value === null || typeof value !== "object") return false;
+ const tools = Reflect.get(value, "tools");
+ return (
+ Reflect.get(value, "result") === "TASK-CHILD-FINAL-RESULT" &&
+ Array.isArray(tools) &&
+ tools.every((tool) => typeof tool === "string") &&
+ !tools.includes("task_update")
+ );
+ }, "the child reports its advertised tools without task_update and a final result"),
+ );
+ completed.turn.eventsSatisfy(
+ "the final child report reaches its owning parent task",
+ (events) =>
+ events.some(
+ (event) =>
+ event.type === "message.received" &&
+ typeof event.data.message === "string" &&
+ event.data.message.includes(
+ `Background task ${taskId} (tool-surface-worker) is completed.`,
+ ) &&
+ event.data.message.includes(report),
+ ),
+ );
+ t.notCalledTool("task_update");
+ t.noFailedActions();
+ },
+});
diff --git a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.batching.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.batching.eval.ts
index 953a88699..3d9487bc6 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.batching.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.batching.eval.ts
@@ -26,19 +26,23 @@ export default [
"task.input.require.accepted-valid-batch",
"task.input.answer.accepted-complete",
"task.lifecycle.complete.accepted-nonterminal",
+ "task.parent.wake.noop-pending-cohort",
],
dimensions: { transport: "local", parentPhase: "parked" },
},
async test(t) {
const children = await startBlockedFanout(t, FANOUT_SIZE);
- // Complete nine children, keeping the last child blocked.
- // Each batch waits for the parent response before releasing the next.
+ // Child streams, not parent replies, acknowledge each release batch.
for (const count of batches) await children.completeNext(count);
- const intermediate = completionMetrics(children.completionTurns);
- t.check(intermediate.visibleMessages, equals(0)).label("intermediate completions are silent");
+ await t.require(children.completedChildren.length, equals(FANOUT_SIZE - 1));
const question = await children.send("TASK-BATCHING-QUESTION");
+ const intermediate = completionMetrics(children.completionTurns);
+ t.check(intermediate.modelSteps, equals(0)).label("no intermediate completion model steps");
+ t.check(completionMetrics(children.parentTurns).modelSteps, equals(1)).label(
+ "only the independent user question invokes the parent before settlement",
+ );
t.check(question.message, equals("56")).label(
"user question answered while a child is blocked",
);
@@ -54,7 +58,16 @@ export default [
);
const total = completionMetrics(children.completionTurns);
- t.check(total.visibleMessages, equals(1)).label("one visible final report");
+ t.check(
+ total,
+ equals({
+ parentTurns: 1,
+ modelSteps: 1,
+ silentMessages: 0,
+ visibleMessages: 1,
+ completionsPerTurn: [FANOUT_SIZE],
+ }),
+ ).label("one full-cohort model turn and visible report");
t.log(
`task-batching ${JSON.stringify({ schedule, children: FANOUT_SIZE, intermediate, total })}`,
);
diff --git a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.conditional-delivery.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.conditional-delivery.eval.ts
deleted file mode 100644
index f23547db5..000000000
--- a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.conditional-delivery.eval.ts
+++ /dev/null
@@ -1,108 +0,0 @@
-import type { EveEvalTurn } from "eve/evals";
-import { satisfies } from "eve/evals/expect";
-
-import { defineTaskEval } from "./task-transition.js";
-import { requireSessionStreamIndex } from "./shared.js";
-
-const REVIEW_FINDING = "blocker: task admission can discard deferred user input.";
-
-/** Related reviewer results stay silent until their same-turn cohort settles. */
-export default defineTaskEval({
- description:
- "Related reviewer wakes remain silent while pending, then produce one settled report.",
- transition: {
- primary: "task.parent.wake.emitted-ready",
- setup: [
- "task.dispatch.start.accepted-acknowledged",
- "task.lifecycle.complete.accepted-nonterminal",
- ],
- dimensions: { transport: "local", parentPhase: "parked" },
- },
- async test(t) {
- const started = await t.send("TASK-WAKE-REDUNDANT-REVIEW");
- started.expectOk();
- started.calledSubagent("busy-worker", { count: 2 });
-
- const taskIds = started.events.flatMap((event) =>
- event.type === "subagent.completed" && event.data.backgroundTask !== undefined
- ? [event.data.backgroundTask.taskId]
- : [],
- );
- await t.require(
- taskIds,
- satisfies(
- (ids: readonly string[]) => ids.length === 2 && new Set(ids).size === 2,
- "two distinct reviewer task receipts",
- ),
- );
-
- const sessionId = started.sessionId;
- const firstLive = t.target.watchTurn(sessionId, {
- startIndex: requireSessionStreamIndex(t, "First reviewer wake"),
- });
- const firstWake = await firstLive.result();
-
- firstWake.expectOk();
- const firstNotification = requireTaskNotification(firstWake);
- await t.require(
- firstNotification,
- satisfies(
- (notification: TaskNotification) =>
- taskIds.includes(notification.taskId) && notification.message.includes(REVIEW_FINDING),
- "the first wake carries one reviewer's result",
- ),
- );
- firstWake.notCalledTool("task_peek");
- firstWake.event("message.completed", {
- count: 1,
- data: (data) => data.finishReason !== "tool-calls" && data.message === null,
- });
- firstWake.notEvent("message.completed", {
- data: (data) => data.finishReason !== "tool-calls" && data.message !== null,
- });
- await t.require(
- firstWake.message,
- satisfies((message) => message === undefined, "the pending cohort wake is silent"),
- );
-
- const secondLive = t.target.watchTurn(sessionId, {
- startIndex: requireSessionStreamIndex(firstLive.session, "Late reviewer wake"),
- });
- const wake = await secondLive.result();
-
- wake.expectOk();
- const lateNotification = requireTaskNotification(wake);
- await t.require(
- lateNotification,
- satisfies(
- (notification: TaskNotification) =>
- taskIds.includes(notification.taskId) &&
- notification.taskId !== firstNotification.taskId &&
- notification.message.includes(REVIEW_FINDING),
- "the late wake carries the other reviewer's redundant result",
- ),
- );
- wake.notCalledTool("task_peek");
- wake.messageIncludes(`request changes on PR #2277.\n\n- ${REVIEW_FINDING}`);
- wake.event("message.completed", {
- count: 1,
- data: (data) => data.finishReason !== "tool-calls" && data.message !== null,
- });
- t.notCalledTool("task_peek");
- t.noFailedActions();
- },
-});
-
-interface TaskNotification {
- readonly message: string;
- readonly taskId: string;
-}
-
-function requireTaskNotification(turn: EveEvalTurn): TaskNotification {
- for (const event of turn.events) {
- if (event.type !== "message.received" || typeof event.data.message !== "string") continue;
- const taskId = /Background task (task_[a-z0-9]+)/iu.exec(event.data.message)?.[1];
- if (taskId !== undefined) return { message: event.data.message, taskId };
- }
- throw new Error("Reviewer wake has no completed-task notification.");
-}
diff --git a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.cross-turn-cohort.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.cross-turn-cohort.eval.ts
new file mode 100644
index 000000000..40db0e64a
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.cross-turn-cohort.eval.ts
@@ -0,0 +1,74 @@
+import { equals } from "eve/evals/expect";
+import { CROSS_TURN_SCENARIO } from "../agent/lib/lifecycle-model.js";
+import { completedTaskIds } from "./batching.js";
+import { assertLifecycleUnion, lifecycleDriver } from "./lifecycle.js";
+import { defineTaskEval } from "./task-transition.js";
+
+export default (["A", "B"] as const).map((first) =>
+ defineTaskEval({
+ description: `Join overlapping launches in distinct user turns; ${first} completes while the other task is still gated.`,
+ timeoutMs: 180_000,
+ transition: {
+ primary: "task.parent.wake.emitted-ready",
+ setup: [
+ "task.parent.wake.noop-pending-cohort",
+ "task.lifecycle.complete.accepted-nonterminal",
+ ],
+ dimensions: { transport: "local", parentPhase: "parked" },
+ },
+ async test(t) {
+ const key = crypto.randomUUID();
+ const driver = lifecycleDriver(t, key);
+ const started = await driver.start(`${CROSS_TURN_SCENARIO} ${key}`);
+ started.messageIncludes("LAUNCHED:A");
+ const a = await driver.gate("A");
+ await t.require([...driver.taskIds.keys()], equals(["A"]));
+ await driver.active(a);
+
+ const launchB = "Alice now launches the second piece of work.";
+ await driver.enqueue(launchB);
+ const second = await driver.through(launchB);
+ second.messageIncludes("LAUNCHED:B");
+ const b = await driver.gate("B");
+ await driver.active(a);
+ await driver.active(b);
+ await t.require([...driver.taskIds.keys()].sort(), equals(["A", "B"]));
+ await t.require(new Set(driver.taskIds.values()).size, equals(2));
+ const owners = driver.controls.filter((event) => event.kind === "owner");
+ await t.require(
+ owners.map((event) => event.marker),
+ equals(["A", "B"]),
+ );
+ await t.require(new Set(owners.map((event) => event.turnId)).size, equals(2));
+ const creatingTurns = [started, second].map(
+ (turn) => turn.events.find((event) => event.type === "actions.requested")?.data.turnId,
+ );
+ await t.require(
+ owners.map((event) => event.turnId),
+ equals(creatingTurns),
+ );
+
+ const remaining = first === "A" ? "B" : "A";
+ await driver.release(first === "A" ? a : b);
+ await driver.settled(first);
+ await driver.active(remaining === "A" ? a : b);
+ const checkpoint = "Alice checks that the other piece of work is still pending.";
+ await driver.enqueue(checkpoint);
+ const pending = await driver.through(checkpoint);
+ pending.messageIncludes("PENDING-CHECK-ACK");
+ t.check(driver.turns.flatMap(completedTaskIds), equals([])).label(
+ `${first} cannot report while ${remaining} remains pending in another creating turn`,
+ );
+
+ await driver.release(remaining === "A" ? a : b);
+ await driver.settled(remaining);
+ await driver.report();
+ const drain = "Alice confirms that all completion deliveries have been observed.";
+ await driver.enqueue(drain);
+ (await driver.through(drain, true)).messageIncludes("DRAIN-ACK");
+ assertLifecycleUnion(t, driver);
+ t.noFailedActions();
+ t.notEvent("compaction.requested");
+ },
+ }),
+);
diff --git a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts
index 8a23cbedf..d4a569169 100644
--- a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts
+++ b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.eval.ts
@@ -7,10 +7,10 @@ import { requireSessionStreamIndex, type TaskEvalSessionDriver } from "./shared.
const FANOUT_SIZE = 3;
const COMPLETED_NOTIFICATION = /Background task (task_[a-z0-9]+) \([^)]+\) is completed\./giu;
-/** Every completed child emits one task-addressed ready notification to its parent. */
+/** The final child releases every sibling's result in one parent turn. */
export default defineTaskEval({
description:
- "Three completed children emit exactly one parent notification each, with no duplicate or unknown task ids.",
+ "Three completed children wake their parent once with all task notifications and no duplicate or unknown task ids.",
transition: {
primary: "task.parent.wake.emitted-ready",
setup: [
@@ -45,11 +45,10 @@ export default defineTaskEval({
);
released.expectOk();
- const notifiedTaskIds = [
- ...completedNotificationTaskIds(started),
- ...blocked.observedTurns.flatMap(completedNotificationTaskIds),
- ...completedNotificationTaskIds(released),
- ];
+ const completionTurns = [...blocked.observedTurns, released].filter(
+ (turn) => completedNotificationTaskIds(turn).length > 0,
+ );
+ const notifiedTaskIds = completionTurns.flatMap(completedNotificationTaskIds);
const observedKnownTaskIds = new Set(
notifiedTaskIds.filter((taskId) => taskIds.includes(taskId)),
);
@@ -65,6 +64,7 @@ export default defineTaskEval({
startIndex: requireSessionStreamIndex(session, "Task fanout notification wait"),
});
const turn = await live.result();
+ if (completedNotificationTaskIds(turn).length > 0) completionTurns.push(turn);
for (const taskId of completedNotificationTaskIds(turn)) {
notifiedTaskIds.push(taskId);
if (taskIds.includes(taskId)) observedKnownTaskIds.add(taskId);
@@ -81,6 +81,11 @@ export default defineTaskEval({
"exactly one completed notification for every known task and none for unknown task ids",
),
);
+ await t.require(
+ completionTurns.length,
+ satisfies((count) => count === 1, "only the last sibling triggers a completion turn"),
+ );
+ completionTurns[0]!.event("step.started", { count: 1 });
t.noFailedActions();
},
});
diff --git a/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.lifecycle-order.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.lifecycle-order.eval.ts
new file mode 100644
index 000000000..6a4433348
--- /dev/null
+++ b/e2e/fixtures/fixture-tasks/evals/task.parent.wake.emitted-ready.lifecycle-order.eval.ts
@@ -0,0 +1,118 @@
+import { equals } from "eve/evals/expect";
+import { LIFECYCLE_SCENARIO } from "../agent/lib/lifecycle-model.js";
+import { completedTaskIds } from "./batching.js";
+import { assertLifecycleUnion, lifecycleDriver } from "./lifecycle.js";
+import { defineTaskEval } from "./task-transition.js";
+
+export default defineTaskEval({
+ description:
+ "Process A completion, B terminal agent settlement, and B completion in FIFO order after an active parent turn.",
+ timeoutMs: 180_000,
+ transition: {
+ primary: "task.parent.wake.emitted-ready",
+ setup: ["task.lifecycle.complete.accepted-nonterminal", "task.parent.wake.noop-pending-cohort"],
+ dimensions: { transport: "local", parentPhase: "active" },
+ },
+ async test(t) {
+ const key = crypto.randomUUID();
+ const driver = lifecycleDriver(t, key);
+ const started = await driver.start(`${LIFECYCLE_SCENARIO} ${key}`);
+ started.messageIncludes("LAUNCHED:A,B");
+ const a = await driver.gate("A");
+ const b = await driver.gate("B");
+ await t.require([...driver.taskIds.keys()].sort(), equals(["A", "B"]));
+ await t.require(new Set(driver.taskIds.values()).size, equals(2));
+ const owners = driver.controls.filter((event) => event.kind === "owner");
+ await t.require(owners.map((event) => event.marker).sort(), equals(["A", "B"]));
+ await t.require(new Set(owners.map((event) => event.turnId)).size, equals(1));
+
+ const holding = "Alice keeps the parent active while Bob finishes.";
+ await driver.enqueue(holding);
+ const parent = await driver.gate("parent");
+ await driver.active(parent);
+
+ await driver.release(a);
+ // Owner completion includes the awaited wakeTaskParentStep, unlike child stream completion.
+ await driver.settled("A");
+ await driver.active(parent);
+ await driver.release(b);
+ // This owner had exactly one agent invocation. Its second agent-request
+ // forwarding step is settlement, and must precede its successful task wake.
+ await driver.settled("B", true);
+ const child = await t.target.watchTurn(b.sessionId, { startIndex: 0 }).result();
+ child.event("session.failed");
+ t.check(
+ child.events.flatMap((event) =>
+ event.type === "step.completed"
+ ? [
+ {
+ inputTokens: event.data.usage?.inputTokens,
+ outputTokens: event.data.usage?.outputTokens,
+ },
+ ]
+ : [],
+ ),
+ equals([{ inputTokens: 211, outputTokens: 37 }]),
+ ).label("B performed real metered work before its terminal provider rejection");
+ await driver.active(parent);
+ // Owner completion proves enqueue, not driver receipt. A normal cancellation
+ // behind B's wake must traverse the same FIFO inbox before ending this turn.
+ // Releasing the unrelated fixture hook here would let turn-result win early.
+ await driver.cancelHeldTurn(parent);
+ const cancelled = await driver.through(holding);
+ await t.require(
+ cancelled.events.flatMap((event) =>
+ event.type === "turn.cancelled" ? [event.data.turnId] : [],
+ ),
+ equals([parent.turnId]),
+ );
+ cancelled.event("session.waiting");
+ cancelled.notEvent("turn.completed");
+ await t.require(completedTaskIds(cancelled), equals([]));
+ await driver.parentGateCancelled(parent);
+ t.log(
+ "FIFO cancellation acknowledged: A completed -> B agent-settled -> B completed were buffered before the held turn ended; its fixture hook was disposed.",
+ );
+ await driver.report();
+
+ const drain = "Alice confirms that all completion deliveries have been observed.";
+ await driver.enqueue(drain);
+ (await driver.through(drain, true)).messageIncludes("DRAIN-ACK");
+ assertLifecycleUnion(t, driver);
+ // This is the actual framework handle announcement in the model request,
+ // not a status inferred from the successful background-task report.
+ const handlesMessage = "Alice checks that Bob's terminal session has no retained handle.";
+ await driver.enqueue(handlesMessage);
+ const handles = await driver.through(handlesMessage);
+ t.check(JSON.parse(handles.message!), equals([])).label(
+ "B's terminal child has no retained available or parked handle",
+ );
+ handles.usedNoTools();
+
+ await driver.enqueue("Alice performs the final metered accounting check.");
+ (await driver.through("Alice performs the final metered accounting check.")).messageIncludes(
+ "ACCOUNTING-CHECK",
+ );
+ await driver.enqueue("Alice checks the remaining session budget.");
+ const budget = await driver.nextTurn();
+ await t.require(budget.inputRequests.length, equals(1));
+ const request = budget.inputRequests[0]!;
+ t.check(
+ { kind: request.kind, tool: request.action.toolName, input: request.action.input },
+ equals({
+ kind: "session-limit",
+ tool: "session_limit_continuation",
+ input: { kind: "input", limit: 1_000_000, usedTokens: 1_000_001 },
+ }),
+ ).label(
+ "runtime session total includes exactly B's 211 input tokens plus 999790 parent tokens",
+ );
+ budget.notEvent("step.completed");
+ t.check(
+ budget.toolCalls.map((call) => call.name),
+ equals(["session_limit_continuation"]),
+ ).label("only the runtime's session-limit control runs, not an authored tool");
+ t.notEvent("compaction.requested");
+ // No blanket noFailedActions: Bob's provider rejection is intentional.
+ },
+});
diff --git a/e2e/fixtures/fixture-tasks/evals/task.update.emitted-working.eval.ts b/e2e/fixtures/fixture-tasks/evals/task.update.emitted-working.eval.ts
deleted file mode 100644
index 5a097d980..000000000
--- a/e2e/fixtures/fixture-tasks/evals/task.update.emitted-working.eval.ts
+++ /dev/null
@@ -1,61 +0,0 @@
-import { satisfies } from "eve/evals/expect";
-
-import { defineTaskEval } from "./task-transition.js";
-import { requireBackgroundTaskId, requireSessionStreamIndex } from "./shared.js";
-
-const UPDATE = "TASK-UPDATE-PROGRESS";
-
-export default defineTaskEval({
- description: "A running child sends one intermediate update to its parent through task_update.",
- transition: {
- primary: "task.update.emitted-working",
- setup: ["task.dispatch.start.accepted-acknowledged"],
- dimensions: { transport: "local", parentPhase: "parked" },
- },
- async test(t) {
- const started = await t.send("TASK-UPDATE-SETUP");
- started.expectOk();
- started.messageIncludes("TASK-UPDATE-STARTED");
- const taskId = requireBackgroundTaskId(started);
-
- const sessionId = t.sessionId;
- if (sessionId === undefined) throw new Error("Task update eval has no parent session id.");
- const live = t.target.watchTurn(sessionId, {
- startIndex: requireSessionStreamIndex(t, "Task update wait"),
- });
- const updateTurn = await live.result();
- updateTurn.expectOk();
- updateTurn.messageIncludes("TASK-UPDATE-RECEIVED");
- await t.require(
- updateTurn.events,
- satisfies(
- (events: typeof updateTurn.events) =>
- events.some(
- (event) =>
- event.type === "message.received" &&
- messageText(event.data.message).includes(
- `Background task ${taskId} (update-worker) update: ${UPDATE}`,
- ),
- ),
- "parent receives the child update with its owning task identity",
- ),
- );
- updateTurn.notEvent("subagent.completed");
- t.noFailedActions();
- },
-});
-
-function messageText(message: unknown): string {
- if (typeof message === "string") return message;
- if (!Array.isArray(message)) return "";
- return message
- .flatMap((part) =>
- part !== null &&
- typeof part === "object" &&
- Reflect.get(part, "type") === "text" &&
- typeof Reflect.get(part, "text") === "string"
- ? [Reflect.get(part, "text") as string]
- : [],
- )
- .join("\n");
-}
diff --git a/e2e/fixtures/fixture-tasks/package.json b/e2e/fixtures/fixture-tasks/package.json
index c7c9ae730..055a0faa1 100644
--- a/e2e/fixtures/fixture-tasks/package.json
+++ b/e2e/fixtures/fixture-tasks/package.json
@@ -9,7 +9,8 @@
"start": "eve start",
"typecheck": "pnpm validate:transitions && eve build && tsc",
"validate:transitions": "node scripts/validate-transitions.mjs",
- "test:e2e": "eve eval --strict"
+ "test:e2e": "eve eval --strict",
+ "test:unit": "node --test evals/lifecycle-model.test.mjs"
},
"dependencies": {
"@eve-e2e/config": "workspace:*",
diff --git a/e2e/fixtures/fixture-tasks/tsconfig.json b/e2e/fixtures/fixture-tasks/tsconfig.json
index 85e3eceb8..032878901 100644
--- a/e2e/fixtures/fixture-tasks/tsconfig.json
+++ b/e2e/fixtures/fixture-tasks/tsconfig.json
@@ -11,7 +11,7 @@
"forceConsistentCasingInFileNames": true,
"declaration": true,
"noEmit": true,
- "types": ["node"]
+ "types": ["node", "eve/workflow-modules"]
},
"include": ["agent/**/*.ts", "evals/**/*.ts"]
}
diff --git a/packages/eve/package.json b/packages/eve/package.json
index 25d232039..701826e56 100644
--- a/packages/eve/package.json
+++ b/packages/eve/package.json
@@ -186,11 +186,6 @@
"import": "./dist/src/public/tools/task-cancel.js",
"default": "./dist/src/public/tools/task-cancel.js"
},
- "./tools/task_update": {
- "types": "./dist/src/public/tools/task-update.d.ts",
- "import": "./dist/src/public/tools/task-update.js",
- "default": "./dist/src/public/tools/task-update.js"
- },
"./tools/glob": {
"types": "./dist/src/public/tools/glob.d.ts",
"import": "./dist/src/public/tools/glob.js",
diff --git a/packages/eve/src/cli/dev/tui/runner.test.ts b/packages/eve/src/cli/dev/tui/runner.test.ts
index 34d015aad..7977b8470 100644
--- a/packages/eve/src/cli/dev/tui/runner.test.ts
+++ b/packages/eve/src/cli/dev/tui/runner.test.ts
@@ -892,6 +892,120 @@ describe("EveTUIRunner idle session follow", () => {
expect(renderIdleStream).not.toHaveBeenCalled();
});
+ it("renders background completion after two quiet minutes without another user message", async () => {
+ vi.useFakeTimers();
+ const session = stubSession();
+ const prompt = createDeferred<string | undefined>();
+ const idleEvents: AgentTUIStreamEvent[] = [];
+ const wakeEvents = [
+ { type: "turn.started", data: { sequence: 1, turnId: "wake-turn" } },
+ {
+ type: "step.started",
+ data: { modelId: "test-model", sequence: 1, stepIndex: 0, turnId: "wake-turn" },
+ },
+ {
+ type: "message.completed",
+ data: {
+ finishReason: "stop",
+ message: "Alice's background review is ready for Bob.",
+ sequence: 1,
+ stepIndex: 0,
+ turnId: "wake-turn",
+ },
+ },
+ { type: "turn.completed", data: { sequence: 1, turnId: "wake-turn" } },
+ { type: "session.waiting", data: { wait: "next-user-message" } },
+ ].map((event, index) => stampTestEvent(event as UnstampedMessageStreamEvent, index));
+ let connections = 0;
+ const signals: AbortSignal[] = [];
+ vi.stubGlobal(
+ "fetch",
+ vi.fn(async (_url: unknown, init?: RequestInit) => {
+ connections += 1;
+ if (init?.signal) signals.push(init.signal);
+ if (connections === 8) return messageStreamResponseOf(wakeEvents);
+ let removeAbortListener: (() => void) | undefined;
+ return new Response(
+ new ReadableStream<Uint8Array>({
+ start(controller) {
+ const abort = () => controller.close();
+ init?.signal?.addEventListener("abort", abort, { once: true });
+ removeAbortListener = () => init?.signal?.removeEventListener("abort", abort);
+ },
+ cancel() {
+ removeAbortListener?.();
+ },
+ }),
+ { headers: { [EVE_STREAM_VERSION_HEADER]: EVE_MESSAGE_STREAM_VERSION } },
+ );
+ }),
+ );
+ const send = vi.spyOn(session, "send");
+ const runner = new EveTUIRunner({
+ session,
+ name: "Research Agent",
+ renderer: fakeRenderer({
+ readPrompt: () => prompt.promise,
+ renderIdleStream: async (result) => {
+ for await (const event of result.events) idleEvents.push(event);
+ },
+ }),
+ });
+ const running = runner.run();
+ try {
+ await vi.advanceTimersByTimeAsync(130_000);
+ expect(connections).toBeGreaterThanOrEqual(8);
+ expect(idleEvents).toContainEqual({
+ type: "assistant-complete",
+ id: "text:wake-turn:0",
+ text: "Alice's background review is ready for Bob.",
+ });
+ expect(send).not.toHaveBeenCalled();
+ } finally {
+ prompt.resolve(undefined);
+ await running;
+ }
+ expect(signals.every((signal) => signal.aborted)).toBe(true);
+ expect(session.state.streamIndex).toBe(wakeEvents.length);
+ });
+
+ it.each(["session.completed", "session.failed"] as const)(
+ "stops idle following at %s even while the prompt remains open",
+ async (type) => {
+ vi.useFakeTimers();
+ const prompt = createDeferred<string | undefined>();
+ const fetch = vi.fn(async () =>
+ messageStreamResponseOf([
+ stampTestEvent({
+ type,
+ data: { sessionId: "session_test", code: "SESSION_FAILED", message: "Session ended." },
+ } as UnstampedMessageStreamEvent),
+ ]),
+ );
+ vi.stubGlobal("fetch", fetch);
+ const runner = new EveTUIRunner({
+ session: stubSession(),
+ name: "Research Agent",
+ renderer: fakeRenderer({
+ readPrompt: () => prompt.promise,
+ renderIdleStream: async (result) => {
+ for await (const event of result.events) {
+ void event;
+ }
+ },
+ }),
+ });
+ const running = runner.run();
+ try {
+ await vi.advanceTimersByTimeAsync(130_000);
+ expect(fetch).toHaveBeenCalledOnce();
+ } finally {
+ prompt.resolve(undefined);
+ await running;
+ }
+ },
+ );
+
it("renders a wake turn while prompting, then stops before send without replay", async () => {
const client = stubClient();
const session = client.sessions.attach("session_test");
diff --git a/packages/eve/src/cli/dev/tui/tool-presentation.test.ts b/packages/eve/src/cli/dev/tui/tool-presentation.test.ts
index 52f839761..9de62e179 100644
--- a/packages/eve/src/cli/dev/tui/tool-presentation.test.ts
+++ b/packages/eve/src/cli/dev/tui/tool-presentation.test.ts
@@ -177,7 +177,6 @@ describe("presentTool", () => {
load_skill: { skill: "commit" },
read_file: { filePath: "/workspace/a.ts" },
task_cancel: { taskIds: ["task_abc"] },
- task_update: { message: "Finished the next region." },
todo: { todos: [] },
web_fetch: { url: "https://example.com" },
web_search: { query: "eve framework" },
@@ -200,6 +199,18 @@ describe("presentTool", () => {
expect(presentTool("task_sleep", { seconds: 30 }).title).toBe("task_sleep");
});
+ it("does not retain semantic copy for the removed task_update tool", () => {
+ const presentation = presentTool("task_update", { message: "Checking the next region." });
+ expect(presentation.title).toBe("task_update");
+ expect(presentation.subtitle).toContain('message="Checking the next region."');
+ expect(presentation.group).toBeUndefined();
+ expect(presentation.doneTitle).toBeUndefined();
+ expect(presentPreparingTool("task_update")).toMatchObject({
+ title: "task_update",
+ subtitle: "preparing…",
+ });
+ });
+
it("presents a named subagent dispatch as a delegation", () => {
const parsed = presentTool(
"stock-price",
diff --git a/packages/eve/src/cli/dev/tui/tool-presentation.ts b/packages/eve/src/cli/dev/tui/tool-presentation.ts
index c528a4f91..f699cb158 100644
--- a/packages/eve/src/cli/dev/tui/tool-presentation.ts
+++ b/packages/eve/src/cli/dev/tui/tool-presentation.ts
@@ -131,13 +131,6 @@ const BUILTIN_TOOL_COPY: Readonly<Record<string, BuiltinToolCopy>> = {
singularNoun: "task",
pluralNoun: "tasks",
},
- task_update: {
- verb: "Update",
- pastVerb: "Updated",
- argKey: "message",
- singularNoun: "parent",
- pluralNoun: "parents",
- },
web_fetch: {
verb: "Fetch",
pastVerb: "Fetched",
diff --git a/packages/eve/src/client/agent-info-schema.ts b/packages/eve/src/client/agent-info-schema.ts
index b585ead80..6f2dbcff7 100644
--- a/packages/eve/src/client/agent-info-schema.ts
+++ b/packages/eve/src/client/agent-info-schema.ts
@@ -254,9 +254,7 @@ const remoteAgent = entry
const kernelEffect = z
.object({
- action: z
- .enum(["subagent-call", "task-update", "task-cancel", "workflow-tool-call"])
- .optional(),
+ action: z.enum(["subagent-call", "task-cancel", "workflow-tool-call"]).optional(),
audience: z.array(z.enum(["root-session", "delegated-task-child", "requires-request-input"])),
kind: z.enum(["request-input", "dispatch", "provider-tool"]),
sourceId: z.string(),
diff --git a/packages/eve/src/compiler/default-tool-policy.ts b/packages/eve/src/compiler/default-tool-policy.ts
index 4c839ae05..80a08bd58 100644
--- a/packages/eve/src/compiler/default-tool-policy.ts
+++ b/packages/eve/src/compiler/default-tool-policy.ts
@@ -22,7 +22,6 @@ export function assertFrameworkToolPolicy(
const closedDispatchSlots = {
"tools/agent": "self-agent",
"tools/task_cancel": "task-cancel",
- "tools/task_update": "task-update",
} as const;
const expectedAction = closedDispatchSlots[slot as keyof typeof closedDispatchSlots];
if (
diff --git a/packages/eve/src/compiler/manifest.ts b/packages/eve/src/compiler/manifest.ts
index 4d1564d7f..2831ef237 100644
--- a/packages/eve/src/compiler/manifest.ts
+++ b/packages/eve/src/compiler/manifest.ts
@@ -833,7 +833,7 @@ const compiledToolBehaviorSchema: z.ZodType<CompiledToolBehavior> = z
.discriminatedUnion("kind", [
z
.object({
- action: z.enum(["self-agent", "task-cancel", "task-update"]),
+ action: z.enum(["self-agent", "task-cancel"]),
kind: z.literal("dispatch"),
})
.strict(),
diff --git a/packages/eve/src/compiler/normalize-manifest.test.ts b/packages/eve/src/compiler/normalize-manifest.test.ts
index 20b8740bd..f369e9cd4 100644
--- a/packages/eve/src/compiler/normalize-manifest.test.ts
+++ b/packages/eve/src/compiler/normalize-manifest.test.ts
@@ -205,7 +205,17 @@ describe("compileAgentManifest source graph", () => {
);
});
- it.each(["agent", "task_cancel", "task_update"])(
+ it("does not install task_update from the framework registry", async () => {
+ const compiled = await compileAgentManifest(manifest());
+
+ expect(compiled.tools.map((tool) => tool.name)).toContain("task_cancel");
+ expect(compiled.tools.map((tool) => tool.name)).not.toContain("task_update");
+ expect(Object.values(compiled.bindings).map((binding) => binding.logicalPath)).not.toContain(
+ "tools/task_update.ts",
+ );
+ });
+
+ it.each(["agent", "task_cancel"])(
"rejects overriding closed framework tool %s",
async (toolName) => {
const sourceRegistry = registry([
diff --git a/packages/eve/src/compiler/subagent-execution.scenario.test.ts b/packages/eve/src/compiler/subagent-execution.scenario.test.ts
index 92877b150..f870ac884 100644
--- a/packages/eve/src/compiler/subagent-execution.scenario.test.ts
+++ b/packages/eve/src/compiler/subagent-execution.scenario.test.ts
@@ -28,9 +28,9 @@ describe("subagent compilation", () => {
expect(child).not.toHaveProperty("execution");
if (child?.configResolver !== undefined) throw new Error("Expected a static child.");
- expect(child?.agent.tools.map((tool) => tool.name)).toEqual(
- expect.arrayContaining(["task_cancel", "task_update"]),
- );
+ const toolNames = child?.agent.tools.map((tool) => tool.name);
+ expect(toolNames).toContain("task_cancel");
+ expect(toolNames).not.toContain("task_update");
});
it("compiles a subagent without root configuration", async () => {
diff --git a/packages/eve/src/execution/coordination-dispatch-step.ts b/packages/eve/src/execution/coordination-dispatch-step.ts
index ae819352e..e75c9cc00 100644
--- a/packages/eve/src/execution/coordination-dispatch-step.ts
+++ b/packages/eve/src/execution/coordination-dispatch-step.ts
@@ -12,7 +12,6 @@ import { hydrateDurableSession } from "#execution/session.js";
import { executeTaskControlAction } from "#execution/tasks/parent/dispatch.js";
import type { BackgroundTask } from "#execution/tasks/parent/delegate.js";
import { cancelBackgroundAgentTask } from "#execution/tools/subagent/task-cancel.js";
-import { deliverTaskUpdate } from "#execution/tasks/child/update.js";
import { startWorkflowTask } from "#execution/tools/workflow/start.js";
import { setPendingCoordinationBatch } from "#harness/coordination.js";
import {
@@ -84,12 +83,7 @@ export async function dispatchCoordinationStep(
if (entry.kind === "task-control") {
const control = await executeTaskControlAction({
action: entry.action,
- adapter: prepared.adapter,
- bundle: prepared.bundle,
cancelOwnedWork: cancelBackgroundAgentTask,
- deliverUpdate: deliverTaskUpdate,
- parentStepIndex: batch.event.stepIndex,
- parentTurnId: batch.event.turnId,
serializedContext: prepared.serializedContext,
session: nextSession,
});
diff --git a/packages/eve/src/execution/create-session-step.test.ts b/packages/eve/src/execution/create-session-step.test.ts
index 51bd8688e..f83393703 100644
--- a/packages/eve/src/execution/create-session-step.test.ts
+++ b/packages/eve/src/execution/create-session-step.test.ts
@@ -3,7 +3,6 @@ import { describe, expect, it, vi } from "vitest";
import { getCompiledRuntimeAgentBundle } from "#runtime/sessions/compiled-agent-cache.js";
import { DEFAULT_ROOT_MAX_INPUT_TOKENS_PER_SESSION } from "#execution/session.js";
import { createSessionStep } from "#execution/create-session-step.js";
-import { TASK_UPDATE_TOOL_NAME } from "#tools/framework/task-contract.js";
import type { RuntimeTurnAgent } from "#runtime/agent/bootstrap.js";
vi.mock("#runtime/sessions/compiled-agent-cache.js", () => ({
@@ -19,45 +18,7 @@ const TestTurnAgent: RuntimeTurnAgent = {
};
describe("createSessionStep", () => {
- it("adds task_update guidance to a task-owned session system prompt", async () => {
- vi.mocked(getCompiledRuntimeAgentBundle).mockResolvedValue({
- resolvedAgent: {
- config: {},
- },
- turnAgent: {
- ...TestTurnAgent,
- tools: [
- {
- behavior: {
- availability: ["delegated-task-child"],
- handling: { kind: "dispatch", target: { kind: "task-update" } },
- },
- description: "Report task progress.",
- inputSchema: null,
- kind: "authored-tool",
- logicalPath: `tools/${TASK_UPDATE_TOOL_NAME}.ts`,
- name: TASK_UPDATE_TOOL_NAME,
- owner: { feature: "tasks", kind: "framework" },
- sourceId: `framework:tools/${TASK_UPDATE_TOOL_NAME}.ts`,
- },
- ],
- },
- } as never);
-
- const { state } = await createSessionStep({
- compiledArtifactsSource: { kind: "bundled" },
- continuationToken: "subagent:test",
- sessionId: "sess-child",
- taskId: "task-1",
- });
-
- expect(state.snapshot?.session.agent.system).toContain("Background task updates");
- expect(state.snapshot?.session.agent.system).toContain("what you are currently doing");
- expect(state.snapshot?.session.taskId).toBe("task-1");
- expect(state.snapshot?.session.state).toBeUndefined();
- });
-
- it("does not add task_update guidance to a task-owned node without the tool", async () => {
+ it("preserves task ownership without injecting progress-reporting instructions", async () => {
vi.mocked(getCompiledRuntimeAgentBundle).mockResolvedValue({
resolvedAgent: {
config: {},
@@ -73,6 +34,9 @@ describe("createSessionStep", () => {
});
expect(state.snapshot?.session.agent.system).not.toContain("Background task updates");
+ expect(state.snapshot?.session.agent.system).not.toContain("task_update");
+ expect(state.snapshot?.session.taskId).toBe("task-1");
+ expect(state.snapshot?.session.state).toBeUndefined();
});
it("defaults root sessions to the root input token budget", async () => {
diff --git a/packages/eve/src/execution/create-session-step.ts b/packages/eve/src/execution/create-session-step.ts
index 2aae3078c..e8a107608 100644
--- a/packages/eve/src/execution/create-session-step.ts
+++ b/packages/eve/src/execution/create-session-step.ts
@@ -13,7 +13,6 @@ import type { RunSessionLimits } from "#channel/types.js";
import type { JsonObject } from "#shared/json.js";
import { resolveEffectiveAgentRuntimeFromConfig } from "#execution/effective-agent-config.js";
import type { DynamicSubagentAgentConfig } from "#runtime/subagents/dynamic-agent-config.js";
-import { TASK_UPDATE_SESSION_INSTRUCTION } from "#tools/framework/task-update.js";
/**
* Result returned by {@link createSessionStep}.
@@ -52,14 +51,6 @@ export async function createSessionStep(input: {
bundle,
input.dynamicSubagentAgentConfig,
);
- const taskUpdatesEnabled =
- input.taskId !== undefined &&
- effectiveAgent.turnAgent.tools.some(
- (tool) =>
- tool.kind === "authored-tool" &&
- tool.behavior?.handling?.kind === "dispatch" &&
- tool.behavior.handling.target.kind === "task-update",
- );
// Both token axes resolve tighter-wins against the cap inherited from the
// delegating parent: a child may narrow what its parent granted, never widen
@@ -89,7 +80,6 @@ export async function createSessionStep(input: {
outputSchema: input.outputSchema,
rootSessionId: input.rootSessionId,
sessionId: input.sessionId,
- systemPromptAdditions: taskUpdatesEnabled ? [TASK_UPDATE_SESSION_INSTRUCTION] : undefined,
taskId: input.taskId,
turnAgent: effectiveAgent.turnAgent,
workflowMaxSubagents: bundle.resolvedAgent.workflowTool?.maxSubagents,
diff --git a/packages/eve/src/execution/multi-agent-callback-routing.scenario.test.ts b/packages/eve/src/execution/multi-agent-callback-routing.scenario.test.ts
index 62b89249a..d09d27d7c 100644
--- a/packages/eve/src/execution/multi-agent-callback-routing.scenario.test.ts
+++ b/packages/eve/src/execution/multi-agent-callback-routing.scenario.test.ts
@@ -13,7 +13,7 @@ import {
notifyTurnCallerStep,
resolveInitialTurnCallerStep,
} from "#subagents/parent-notification.js";
-import { fireSessionCallbackStep, fireTaskUpdateCallbackStep } from "#subagents/callback-step.js";
+import { fireSessionCallbackStep } from "#subagents/callback-step.js";
import { startRemoteAgentSession } from "#subagents/remote-dispatch.js";
import { resolveWorkflowCallbackBaseUrl } from "#execution/workflow-callback-url.js";
import { authHookToken, CallbackBaseUrlKey, getHookUrl } from "#harness/authorization.js";
@@ -398,7 +398,7 @@ describe("multi-agent callback routing", () => {
}
});
- it.each(["session.completed", "task.update", "turn.completed", "turn.failed"] as const)(
+ it.each(["session.completed", "turn.completed", "turn.failed"] as const)(
"logs %s delivery failures when the public route prefix is absent",
async (kind) => {
stubAgentRuntimeEnvironment(undefined);
@@ -423,15 +423,7 @@ describe("multi-agent callback routing", () => {
[SessionCallbackKey.name]: sessionCallback,
};
const deliver = async () => {
- if (kind === "task.update") {
- await fireTaskUpdateCallbackStep({
- callback: sessionCallback,
- callId: "call-support",
- message: "private progress",
- updateEpoch: "turn-child",
- updateIndex: 1,
- });
- } else if (kind === "session.completed") {
+ if (kind === "session.completed") {
await fireSessionCallbackStep({
output: "report done",
serializedContext,
diff --git a/packages/eve/src/execution/node-step.test.ts b/packages/eve/src/execution/node-step.test.ts
index d51996c31..eb178c256 100644
--- a/packages/eve/src/execution/node-step.test.ts
+++ b/packages/eve/src/execution/node-step.test.ts
@@ -254,16 +254,12 @@ describe("createNodeHarnessTools", () => {
expect(agentTool?.execute).toBeDefined();
});
- it("lowers compiled task-control tools from their framework definitions", async () => {
- const node = await createNodeWithSourceOwnedTools({
- names: ["task_cancel", "task_update"],
- });
+ it("lowers task_cancel from its framework definition", async () => {
+ const node = await createNodeWithSourceOwnedTools({ names: ["task_cancel"] });
const tools = createNodeHarnessTools({ node });
- for (const name of ["task_cancel", "task_update"]) {
- expect(tools.get(name)?.runtimeAction).toEqual({ kind: "task-control" });
- expect(tools.get(name)?.execute).toBeUndefined();
- }
+ expect(tools.get("task_cancel")?.runtimeAction).toEqual({ kind: "task-control" });
+ expect(tools.get("task_cancel")?.execute).toBeUndefined();
expect(tools.has("task_sleep")).toBe(false);
});
@@ -307,10 +303,10 @@ describe("createNodeHarnessTools", () => {
it("does not recreate task tools absent from the compiled graph", async () => {
const tools = createNodeHarnessTools({
- node: await createNodeWithSourceOwnedTools({ names: ["task_update"] }),
+ node: await createNodeWithSourceOwnedTools({ names: [] }),
});
- expect(tools.has("task_update")).toBe(true);
+ expect(tools.has("task_update")).toBe(false);
expect(tools.has("task_cancel")).toBe(false);
});
});
diff --git a/packages/eve/src/execution/parked-delivery-wait.test.ts b/packages/eve/src/execution/parked-delivery-wait.test.ts
index b6492a647..fb7944ca7 100644
--- a/packages/eve/src/execution/parked-delivery-wait.test.ts
+++ b/packages/eve/src/execution/parked-delivery-wait.test.ts
@@ -10,6 +10,8 @@ import type {
} from "#execution/session-command-inbox.js";
import type { DurableSessionState } from "#execution/durable-session-store.js";
import { SessionStateCursor } from "#execution/session-state-cursor.js";
+import { cacheTerminalTaskView } from "#tasks/session-index.js";
+import type { TaskView } from "#tasks/types.js";
vi.mock("./route-child-delivery.js", () => ({
routeDeliverToChildren: vi.fn(),
@@ -448,7 +450,7 @@ function completion(taskId: string): DeliverHookPayload {
};
}
-function batchingInput() {
+function batchingInput(count = 100, crossTurn = false) {
const input = waitInput(createMockInbox([]));
input.stateCursor.adoptState({
sessionState: {
@@ -463,15 +465,17 @@ function batchingInput() {
state: {
"eve.tasks": {
version: 2,
- tasks: Array.from({ length: 100 }, (_, index) => ({
+ tasks: Array.from({ length: count }, (_, index) => ({
taskId: `task_${index}`,
+ cohortId: "task_0",
taskRunId: `run-${index}`,
taskInboxToken: `inbox-${index}`,
- createdByTurnId: "turn-1",
+ createdByTurnId: crossTurn ? `turn-${index + 1}` : "turn-1",
metadata: { kind: "subagent", name: "worker" },
})).concat([
{
taskId: "other-cohort",
+ cohortId: "other-cohort",
taskRunId: "other-run",
taskInboxToken: "other-inbox",
createdByTurnId: "turn-2",
@@ -492,7 +496,7 @@ function batchingInput() {
serializedContext,
}),
);
- return input;
+ return { ...input };
}
describe("buffered task completion batching", () => {
@@ -541,7 +545,7 @@ describe("buffered task completion batching", () => {
},
],
] satisfies readonly (readonly [string, DeliverHookPayload])[])(
- "stops at a %s without reaching later siblings",
+ "services a %s while retaining incomplete cohorts",
async (_name, boundary) => {
const input = batchingInput();
const first = completion("task_0");
@@ -549,22 +553,215 @@ describe("buffered task completion batching", () => {
const later = completion("task_3");
const bufferedDeliveries = [first, second, boundary, later];
const next = await nextTurnDelivery({ ...input, bufferedDeliveries });
- expect(next).toMatchObject({
+ expect(next).toEqual({ kind: "turn", delivery: boundary });
+ expect(bufferedDeliveries).toEqual([first, second, later]);
+ },
+ );
+
+ it.each([false, true])(
+ "keeps the cohort across a user turn (cross-turn launches: %s)",
+ async (crossTurn) => {
+ const input = batchingInput(3, crossTurn);
+ const first = completion("task_0");
+ const second = completion("task_1");
+ const last = completion("task_2");
+ input.bufferedDeliveries.push(first);
+ input.commandInbox = createMockInbox([
+ { result: { done: false, value: second }, source: "session" },
+ messageRead("user question"),
+ { result: { done: false, value: last }, source: "session" },
+ ]);
+
+ await expect(nextTurnDelivery(input)).resolves.toMatchObject({
+ kind: "turn",
+ delivery: { payloads: [{ message: "user question" }] },
+ });
+ expect(input.bufferedDeliveries).toEqual([first, second]);
+ expect(routeDeliverToChildren).toHaveBeenCalledTimes(1);
+
+ await expect(nextTurnDelivery(input)).resolves.toMatchObject({
kind: "turn",
- delivery: { payloads: [...first.payloads, ...second.payloads] },
+ delivery: { payloads: [...first.payloads, ...second.payloads, ...last.payloads] },
});
- expect(bufferedDeliveries).toEqual([boundary, later]);
+ expect(input.bufferedDeliveries).toEqual([]);
+ expect(routeDeliverToChildren).toHaveBeenCalledTimes(2);
},
);
- it("delivers an available completion without waiting for unfinished siblings", async () => {
- const input = batchingInput();
- const first = completion("task_0");
- const next = await nextTurnDelivery({ ...input, bufferedDeliveries: [first] });
- expect(next).toMatchObject({ kind: "turn", delivery: { payloads: first.payloads } });
+ it("batches a whole cohort when every completion arrives after the active turn", async () => {
+ const input = batchingInput(3);
+ const deliveries = [completion("task_0"), completion("task_1"), completion("task_2")];
+ input.commandInbox = createMockInbox(
+ deliveries.map((value) => ({ result: { done: false, value }, source: "session" })),
+ );
+ await expect(nextTurnDelivery(input)).resolves.toMatchObject({
+ kind: "turn",
+ delivery: { payloads: deliveries.flatMap((delivery) => delivery.payloads) },
+ });
expect(routeDeliverToChildren).toHaveBeenCalledTimes(1);
});
+ it("routes intervening child settlement before releasing the completion cohort", async () => {
+ const input = batchingInput(2);
+ const first = completion("task_0");
+ const last = completion("task_1");
+ const settlement: DeliverHookPayload = {
+ kind: "deliver",
+ payloads: [
+ {
+ task: {
+ agentRequests: [
+ {
+ taskId: "task_1",
+ replyTo: "child-reply",
+ request: {
+ kind: "agent-settled",
+ result: {
+ callId: "child-call",
+ kind: "subagent-result",
+ origin: "child",
+ subagentName: "worker",
+ output: "done",
+ outcome: {
+ kind: "terminal",
+ result: { kind: "succeeded", output: "done" },
+ usageDelta: {
+ inputTokens: 211,
+ outputTokens: 37,
+ cacheReadTokens: 0,
+ cacheWriteTokens: 0,
+ },
+ },
+ },
+ },
+ },
+ ],
+ },
+ },
+ ],
+ };
+ input.bufferedDeliveries.push(first, settlement, last);
+ vi.mocked(routeDeliverToChildren).mockImplementation(
+ async ({ delivery, sessionState, serializedContext }) => ({
+ kind: "continue",
+ remainder: delivery.payloads.some((payload) => payload.task?.agentRequests !== undefined)
+ ? undefined
+ : delivery,
+ sessionState,
+ serializedContext,
+ }),
+ );
+
+ await expect(nextTurnDelivery(input)).resolves.toMatchObject({
+ kind: "turn",
+ delivery: { payloads: [...first.payloads, ...last.payloads] },
+ });
+ expect(vi.mocked(routeDeliverToChildren).mock.calls.map(([call]) => call.delivery)).toEqual([
+ settlement,
+ expect.objectContaining({ payloads: [...first.payloads, ...last.payloads] }),
+ ]);
+ expect(input.bufferedDeliveries).toEqual([]);
+ });
+
+ it("combines siblings across buffered deliveries from another cohort", async () => {
+ const input = batchingInput(2);
+ const first = completion("task_0");
+ const last = completion("task_1");
+ const other = completion("other-cohort");
+ input.bufferedDeliveries.push(first, other, last);
+ await expect(nextTurnDelivery(input)).resolves.toMatchObject({
+ kind: "turn",
+ delivery: { payloads: [...first.payloads, ...last.payloads] },
+ });
+ expect(input.bufferedDeliveries).toEqual([other]);
+ await expect(nextTurnDelivery(input)).resolves.toEqual({ kind: "turn", delivery: other });
+ });
+
+ it.each(["clear", "compact", "reset", "session-timeout"] as const)(
+ "services %s without releasing an incomplete cohort",
+ async (kind) => {
+ const input = batchingInput(2);
+ const first = completion("task_0");
+ input.bufferedDeliveries.push(first);
+ input.commandInbox = createMockInbox([
+ { result: { done: false, value: { kind } }, source: "session" },
+ ]);
+ await expect(nextTurnDelivery(input)).resolves.toEqual({
+ kind: kind === "session-timeout" ? "expired" : kind,
+ });
+ expect(input.bufferedDeliveries).toEqual([first]);
+ expect(routeDeliverToChildren).not.toHaveBeenCalled();
+ },
+ );
+
+ it("does not wait for a cancelled sibling whose notifications are discarded", async () => {
+ const input = batchingInput(2);
+ const first = completion("task_0");
+ input.bufferedDeliveries.push(first);
+ input.commandInbox = createMockInbox([cancelRead({ taskId: "task_1" })]);
+ await expect(nextTurnDelivery(input)).resolves.toEqual({ kind: "turn", delivery: first });
+ });
+
+ it("ignores duplicate notifications while waiting for the last sibling", async () => {
+ const input = batchingInput(2);
+ const first = completion("task_0");
+ const last = completion("task_1");
+ input.commandInbox = createMockInbox(
+ [first, first, last].map((value) => ({
+ result: { done: false, value },
+ source: "session",
+ })),
+ );
+ await expect(nextTurnDelivery(input)).resolves.toMatchObject({
+ kind: "turn",
+ delivery: { payloads: [...first.payloads, ...last.payloads] },
+ });
+ });
+
+ it.each(["failed", "cancelled"] as const)(
+ "delivers %s immediately, then releases successful siblings from the settled cohort",
+ async (status) => {
+ const input = batchingInput(2);
+ const first = completion("task_0");
+ const view: TaskView = {
+ taskId: "task_1",
+ metadata: { kind: "subagent", name: "worker" },
+ ...(status === "failed"
+ ? { status, lastOutput: { type: "error", data: "failed" } }
+ : { status }),
+ };
+ const terminal: DeliverHookPayload = {
+ kind: "deliver",
+ taskDeliveryId: `task_1:ready:${status}`,
+ payloads: [{ message: status, task: { views: [view] } }],
+ };
+ input.bufferedDeliveries.push(first, terminal);
+ vi.mocked(routeDeliverToChildren).mockImplementation(
+ async ({ delivery, sessionState, serializedContext }) => {
+ const snapshot = sessionState.snapshot!;
+ return {
+ kind: "continue",
+ remainder: delivery,
+ serializedContext,
+ sessionState: {
+ ...sessionState,
+ snapshot: {
+ ...snapshot,
+ session: {
+ ...snapshot.session,
+ state: cacheTerminalTaskView(snapshot.session.state, view),
+ },
+ },
+ },
+ };
+ },
+ );
+ await expect(nextTurnDelivery(input)).resolves.toEqual({ kind: "turn", delivery: terminal });
+ expect(input.bufferedDeliveries).toEqual([first]);
+ await expect(nextTurnDelivery(input)).resolves.toEqual({ kind: "turn", delivery: first });
+ },
+ );
+
it("services ready authorization before a buffered completion batch", async () => {
const input = batchingInput();
const bufferedDeliveries = [completion("task_0"), completion("task_1")];
diff --git a/packages/eve/src/execution/parked-delivery-wait.ts b/packages/eve/src/execution/parked-delivery-wait.ts
index d955af7be..a2c76aab7 100644
--- a/packages/eve/src/execution/parked-delivery-wait.ts
+++ b/packages/eve/src/execution/parked-delivery-wait.ts
@@ -156,27 +156,16 @@ async function waitForNextSessionAction(input: {
return { kind: pendingSessionControl };
}
- while (
- input.bufferedDeliveries[0] !== undefined &&
- isCancelledTaskDelivery(input.bufferedDeliveries[0], input.cancelledTaskIds)
- ) {
- input.bufferedDeliveries.shift();
- }
- if (
- input.deferDeliveries !== true &&
- !input.commandInbox.hasReadyAuthorization() &&
- input.bufferedDeliveries.length > 0
- ) {
- return {
- delivery: takeBufferedTurnDelivery(
+ while (true) {
+ if (input.deferDeliveries !== true && !input.commandInbox.hasReadyAuthorization()) {
+ const delivery = takeBufferedTurnDelivery(
input.bufferedDeliveries,
getSessionTaskCohorts(input.stateCursor.sessionState.snapshot?.session.state),
- ),
- kind: "delivery",
- };
- }
+ input.cancelledTaskIds,
+ );
+ if (delivery !== undefined) return { delivery, kind: "delivery" };
+ }
- while (true) {
const { result: first, source } = await input.commandInbox.nextWithSource();
input.commandInbox.consumeNext();
@@ -247,11 +236,7 @@ async function waitForNextSessionAction(input: {
input.seenTaskDeliveries.add(deliveryId);
}
- if (input.deferDeliveries === true) {
- input.bufferedDeliveries.push(decoded);
- continue;
- }
- return { delivery: decoded, kind: "delivery" };
+ input.bufferedDeliveries.push(decoded);
}
}
@@ -274,31 +259,76 @@ function isCancelledTaskDeliveryId(
function takeBufferedTurnDelivery(
bufferedDeliveries: DeliverHookPayload[],
- cohorts: ReadonlyMap<string, string>,
-): DeliverHookPayload {
- const first = bufferedDeliveries.shift();
- if (first === undefined) {
- throw new Error("Cannot take a turn delivery from an empty buffer.");
- }
+ cohorts: ReturnType<typeof getSessionTaskCohorts>,
+ cancelledTaskIds: ReadonlySet<string>,
+): DeliverHookPayload | undefined {
+ const retained = bufferedDeliveries.filter(
+ (delivery) => !isCancelledTaskDelivery(delivery, cancelledTaskIds),
+ );
+ bufferedDeliveries.splice(0, bufferedDeliveries.length, ...retained);
+ const completed = new Set(
+ bufferedDeliveries.flatMap((delivery) => {
+ const taskId = completionTaskId(delivery);
+ return taskId === undefined ? [] : [taskId];
+ }),
+ );
+ const pendingCohorts = new Set<string>();
+ for (const [taskId, cohort] of cohorts) {
+ if (!cohort.settled && !completed.has(taskId) && !cancelledTaskIds.has(taskId)) {
+ pendingCohorts.add(cohort.cohortId);
+ }
+ }
+ let index = bufferedDeliveries.findIndex((delivery) => {
+ const cohort = completionCohort(delivery, cohorts);
+ return cohort === undefined || !pendingCohorts.has(cohort);
+ });
+ if (index < 0) return undefined;
+ const readyCohort = completionCohort(bufferedDeliveries[index]!, cohorts);
+ if (readyCohort !== undefined) {
+ const lastSibling = bufferedDeliveries.findLastIndex(
+ (delivery) => completionCohort(delivery, cohorts) === readyCohort,
+ );
+ // Settlement must apply before terminal views release its claimed handle.
+ // Process intervening deliveries first, then re-evaluate the cohort.
+ const boundary = bufferedDeliveries.findIndex(
+ (delivery, position) =>
+ position > index &&
+ position < lastSibling &&
+ completionCohort(delivery, cohorts) === undefined,
+ );
+ if (boundary >= 0) index = boundary;
+ }
+ const first = bufferedDeliveries.splice(index, 1)[0]!;
const cohort = completionCohort(first, cohorts);
+ if (cohort !== undefined) {
+ const siblings = bufferedDeliveries.filter(
+ (delivery) => completionCohort(delivery, cohorts) === cohort,
+ );
+ const remaining = bufferedDeliveries.filter(
+ (delivery) => completionCohort(delivery, cohorts) !== cohort,
+ );
+ bufferedDeliveries.splice(0, bufferedDeliveries.length, ...remaining);
+ return coalesceDeliveries([first, ...siblings]);
+ }
+
const authenticated = first.auth != null && first.auth.principalType !== "anonymous";
const turnDeliveries = [first];
let caller = first.caller;
- while (bufferedDeliveries.length > 0) {
- const next = bufferedDeliveries[0];
+ while (bufferedDeliveries.length > index) {
+ const next = bufferedDeliveries[index];
if (
next === undefined ||
- ((first.taskDeliveryId !== undefined || next.taskDeliveryId !== undefined) &&
- (cohort === undefined || completionCohort(next, cohorts) !== cohort)) ||
- (first.taskDeliveryId === undefined &&
- (!authenticated || !jsonValuesEqual(first.auth, next.auth))) ||
+ first.taskDeliveryId !== undefined ||
+ next.taskDeliveryId !== undefined ||
+ !authenticated ||
+ !jsonValuesEqual(first.auth, next.auth) ||
(caller !== undefined && next.caller !== undefined)
) {
break;
}
- const delivery = bufferedDeliveries.shift();
+ const delivery = bufferedDeliveries.splice(index, 1)[0];
if (delivery === undefined) {
throw new Error("Buffered turn delivery disappeared while partitioning.");
}
@@ -312,9 +342,14 @@ function takeBufferedTurnDelivery(
/** Only successful sibling notifications can share their existing cohort context. */
function completionCohort(
delivery: DeliverHookPayload,
- cohorts: ReadonlyMap<string, string>,
+ cohorts: ReturnType<typeof getSessionTaskCohorts>,
): string | undefined {
+ const taskId = completionTaskId(delivery);
+ return taskId === undefined ? undefined : cohorts.get(taskId)?.cohortId;
+}
+
+function completionTaskId(delivery: DeliverHookPayload): string | undefined {
const suffix = ":ready:completed";
if (delivery.caller !== undefined || !delivery.taskDeliveryId?.endsWith(suffix)) return undefined;
- return cohorts.get(delivery.taskDeliveryId.slice(0, -suffix.length));
+ return delivery.taskDeliveryId.slice(0, -suffix.length);
}
diff --git a/packages/eve/src/execution/session-callback-request.test.ts b/packages/eve/src/execution/session-callback-request.test.ts
index 26fe8053a..d20a0a0c5 100644
--- a/packages/eve/src/execution/session-callback-request.test.ts
+++ b/packages/eve/src/execution/session-callback-request.test.ts
@@ -21,7 +21,7 @@ describe("postSessionCallbackRequest", () => {
vi.restoreAllMocks();
});
- it.each(["task.update", "turn.completed", "turn.failed"])(
+ it.each(["session.completed", "turn.completed", "turn.failed"])(
"logs %s HTTP failures with correlation fields and a redacted destination",
async (kind) => {
const response = new Response("private response body", { status: 404 });
@@ -78,7 +78,7 @@ describe("postSessionCallbackRequest", () => {
await expect(
postSessionCallbackRequest({
- body: { kind: "task.update", taskId: "task-1" },
+ body: { kind: "session.completed", taskId: "task-1" },
url: callbackUrl,
timeoutMs: 123,
}),
diff --git a/packages/eve/src/execution/session.test.ts b/packages/eve/src/execution/session.test.ts
index 67cdda9a7..beaf04650 100644
--- a/packages/eve/src/execution/session.test.ts
+++ b/packages/eve/src/execution/session.test.ts
@@ -428,25 +428,6 @@ describe("mintSubagentContinuationToken", () => {
});
describe("refreshSessionFromTurnAgent", () => {
- it("keeps session-specific system additions alongside refreshed authored instructions", () => {
- const session = createSession({
- continuationToken: "child-token",
- sessionId: "child-session",
- systemPromptAdditions: ["Background task updates"],
- turnAgent: createTestTurnAgent({ instructions: ["Original instructions."] }),
- });
- expect(session.agent.system).toBe("Original instructions.\n\nBackground task updates");
-
- const refreshed = refreshSessionFromTurnAgent({
- session,
- systemPromptAdditions: ["Background task updates"],
- turnAgent: createTestTurnAgent({ instructions: ["Updated instructions."] }),
- });
-
- expect(refreshed.agent.system).toBe("Updated instructions.\n\nBackground task updates");
- expect(refreshed.state).toBeUndefined();
- });
-
it("refreshes the current agent configuration while preserving history", () => {
const session = createSession({
continuationToken: "root-token",
diff --git a/packages/eve/src/execution/session.ts b/packages/eve/src/execution/session.ts
index e3457942f..41cb5889c 100644
--- a/packages/eve/src/execution/session.ts
+++ b/packages/eve/src/execution/session.ts
@@ -74,7 +74,6 @@ export interface CreateSessionInput {
readonly turnAgent: RuntimeTurnAgent;
readonly limits?: AuthoredSessionLimits;
readonly outputSchema?: HarnessSession["outputSchema"];
- readonly systemPromptAdditions?: readonly string[];
readonly taskId?: string;
readonly workflowMaxSubagents?: number;
}
@@ -87,11 +86,7 @@ export function createSession(input: CreateSessionInput): HarnessSession {
const session: {
-readonly [K in keyof HarnessSession]: HarnessSession[K];
} = {
- agent: createSessionAgent(
- turnAgent,
- createSessionSystemPrompt({ additions: input.systemPromptAdditions, turnAgent }),
- tools,
- ),
+ agent: createSessionAgent(turnAgent, createSessionSystemPrompt(turnAgent), tools),
compaction: createCompactionConfig({
contextWindowTokens: turnAgent.model?.contextWindowTokens,
thresholdPercent: input.compactionOverrides?.thresholdPercent,
@@ -146,7 +141,6 @@ function createSessionAgent(
*/
export function refreshSessionFromTurnAgent(input: {
readonly session: HarnessSession;
- readonly systemPromptAdditions?: readonly string[];
readonly turnAgent: RuntimeTurnAgent;
readonly compactionOverrides?: {
readonly thresholdPercent?: number;
@@ -156,10 +150,7 @@ export function refreshSessionFromTurnAgent(input: {
...input.session,
agent: createSessionAgent(
input.turnAgent,
- createSessionSystemPrompt({
- additions: input.systemPromptAdditions,
- turnAgent: input.turnAgent,
- }),
+ createSessionSystemPrompt(input.turnAgent),
createSessionToolDefinitions(input.turnAgent),
),
compaction: createCompactionConfig({
@@ -171,16 +162,11 @@ export function refreshSessionFromTurnAgent(input: {
};
}
-function createSessionSystemPrompt(input: {
- readonly additions?: readonly string[];
- readonly turnAgent: RuntimeTurnAgent;
-}): string {
- const skillSection = formatAvailableSkillsSection(input.turnAgent.availableSkills ?? []);
+function createSessionSystemPrompt(turnAgent: RuntimeTurnAgent): string {
+ const skillSection = formatAvailableSkillsSection(turnAgent.availableSkills ?? []);
const blocks =
- skillSection === null
- ? input.turnAgent.instructions
- : [...input.turnAgent.instructions, skillSection];
- return [...blocks, ...(input.additions ?? [])].join("\n\n");
+ skillSection === null ? turnAgent.instructions : [...turnAgent.instructions, skillSection];
+ return blocks.join("\n\n");
}
/**
diff --git a/packages/eve/src/execution/tasks/child/update-step.ts b/packages/eve/src/execution/tasks/child/update-step.ts
deleted file mode 100644
index 301c13c4b..000000000
--- a/packages/eve/src/execution/tasks/child/update-step.ts
+++ /dev/null
@@ -1,12 +0,0 @@
-import { resumeHook } from "#internal/workflow/runtime.js";
-import type { TaskInboundUpdate } from "#tasks/types.js";
-
-/** Forwards a local task-owned child's progress update to its active parent hook. */
-export async function forwardLocalTaskUpdateStep(input: {
- readonly parentContinuationToken: string;
- readonly update: TaskInboundUpdate;
-}): Promise<void> {
- "use step";
-
- await resumeHook(input.parentContinuationToken, input.update);
-}
diff --git a/packages/eve/src/execution/tasks/child/update.ts b/packages/eve/src/execution/tasks/child/update.ts
deleted file mode 100644
index 066fe464b..000000000
--- a/packages/eve/src/execution/tasks/child/update.ts
+++ /dev/null
@@ -1,32 +0,0 @@
-import type { ChannelAdapter } from "#channel/adapter.js";
-import { forwardLocalTaskUpdateStep } from "#execution/tasks/child/update-step.js";
-import { fireTaskUpdateCallbackStep } from "#subagents/callback-step.js";
-import { isSubagentAdapterState, SUBAGENT_ADAPTER_KIND } from "#subagents/adapter-state.js";
-import type { TaskInboundUpdate } from "#tasks/types.js";
-
-/** Delivers a task update through the local child adapter or remote callback. */
-export async function deliverTaskUpdate(input: {
- readonly adapter?: ChannelAdapter;
- readonly callback: unknown;
- readonly update: TaskInboundUpdate;
-}): Promise<string | undefined> {
- const state = input.adapter?.state;
- if (
- input.adapter?.kind === SUBAGENT_ADAPTER_KIND &&
- isSubagentAdapterState(state) &&
- state.taskId !== undefined
- ) {
- await forwardLocalTaskUpdateStep({
- parentContinuationToken: state.parentContinuationToken,
- update: input.update,
- });
- return state.taskId;
- }
- return await fireTaskUpdateCallbackStep({
- callback: input.callback,
- callId: input.update.callId,
- message: input.update.message,
- updateEpoch: input.update.updateEpoch,
- updateIndex: input.update.updateIndex,
- });
-}
diff --git a/packages/eve/src/execution/tasks/parent/dispatch.test.ts b/packages/eve/src/execution/tasks/parent/dispatch.test.ts
index 162a5d3db..3956ea987 100644
--- a/packages/eve/src/execution/tasks/parent/dispatch.test.ts
+++ b/packages/eve/src/execution/tasks/parent/dispatch.test.ts
@@ -1,6 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
-import { cancelOwnedTask, executeTaskControlAction } from "#execution/tasks/parent/dispatch.js";
+import { cancelOwnedTask, isTaskControlAction } from "#execution/tasks/parent/dispatch.js";
import { readLatestTaskView, sendTaskCommand } from "#execution/tasks/parent/run-parent.js";
import { cancelWorkflowToolRun } from "#execution/tools/workflow/cancel.js";
import { resumeSessionInbox } from "#execution/wire/session-inbox-resume.js";
@@ -192,77 +192,13 @@ describe("task cancellation", () => {
});
});
-describe("task updates", () => {
- beforeEach(() => vi.resetAllMocks());
-
- it("forwards a task-owned update and confirms delivery", async () => {
- const deliverUpdate = vi.fn(async () => "task-1");
-
- const result = await executeTaskControlAction({
- deliverUpdate,
- action: {
- callId: "call-update",
- input: { message: "Working" },
- kind: "tool-call",
- toolName: "task_update",
- },
- bundle: {} as never,
- parentStepIndex: 2,
- parentTurnId: "turn-child",
- serializedContext: { "eve.sessionCallback": { taskId: "task-1" } },
- session: {} as never,
- });
-
- expect(deliverUpdate).toHaveBeenCalledWith(
- expect.objectContaining({
- update: expect.objectContaining({
- message: "Working",
- updateEpoch: "turn-child",
- updateIndex: 2,
- }),
- }),
+describe("task control actions", () => {
+ it.each([
+ ["task_cancel", true],
+ ["task_update", false],
+ ])("recognizes %s as a task control: %s", (toolName, expected) => {
+ expect(isTaskControlAction({ callId: "call-1", input: {}, kind: "tool-call", toolName })).toBe(
+ expected,
);
- expect(result.result).toMatchObject({ output: { status: "sent", taskId: "task-1" } });
- });
-
- it("forwards a local task-owned child update through the subagent adapter hook", async () => {
- const deliverUpdate = vi.fn(async () => "task-1");
- const result = await executeTaskControlAction({
- deliverUpdate,
- action: {
- callId: "call-update",
- input: { message: "Working" },
- kind: "tool-call",
- toolName: "task_update",
- },
- adapter: {
- kind: "subagent",
- state: {
- callId: "call-agent",
- parentContinuationToken: "agent-reply-hook",
- parentSessionId: "session-parent",
- subagentName: "worker",
- taskId: "task-1",
- },
- },
- bundle: {} as never,
- parentStepIndex: 2,
- parentTurnId: "turn-child",
- serializedContext: {},
- session: {} as never,
- });
-
- expect(deliverUpdate).toHaveBeenCalledWith({
- adapter: expect.objectContaining({ kind: "subagent" }),
- callback: undefined,
- update: {
- callId: "call-update",
- kind: "task-update",
- message: "Working",
- updateEpoch: "turn-child",
- updateIndex: 2,
- },
- });
- expect(result.result).toMatchObject({ output: { status: "sent", taskId: "task-1" } });
});
});
diff --git a/packages/eve/src/execution/tasks/parent/dispatch.ts b/packages/eve/src/execution/tasks/parent/dispatch.ts
index 421e28a58..d50e62554 100644
--- a/packages/eve/src/execution/tasks/parent/dispatch.ts
+++ b/packages/eve/src/execution/tasks/parent/dispatch.ts
@@ -1,4 +1,3 @@
-import type { ChannelAdapter } from "#channel/adapter.js";
import type { HarnessSession as RuntimeSession } from "#harness/types.js";
import {
createTaskControlError,
@@ -17,34 +16,19 @@ import {
} from "#execution/tasks/parent/task-cancel.js";
import type { RuntimeActionResult, RuntimeToolCallActionRequest } from "#shared/action-types.js";
import type { SessionTaskIndexEntry } from "#tasks/session-index.js";
-import { isTerminalTaskStatus, type TaskInboundUpdate, type TaskView } from "#tasks/types.js";
-import {
- TASK_CANCEL_TOOL_NAME,
- TASK_TOOL_NAMES,
- TASK_UPDATE_TOOL_NAME,
-} from "#tools/framework/task-contract.js";
+import { isTerminalTaskStatus, type TaskView } from "#tasks/types.js";
+import { TASK_CANCEL_TOOL_NAME, TASK_TOOL_NAMES } from "#tools/framework/task-contract.js";
const CANCEL_COMMIT_POLL_ATTEMPTS = 10;
const CANCEL_COMMIT_POLL_DELAY_MS = 250;
-export type DeliverTaskUpdate = (input: {
- readonly adapter?: ChannelAdapter;
- readonly callback: unknown;
- readonly update: TaskInboundUpdate;
-}) => Promise<string | undefined>;
-
export function isTaskControlAction(action: RuntimeToolCallActionRequest): boolean {
return action.kind === "tool-call" && TASK_TOOL_NAMES.has(action.toolName);
}
export async function executeTaskControlAction(input: {
readonly action: RuntimeToolCallActionRequest;
- readonly adapter?: ChannelAdapter;
- readonly bundle: import("#runtime/sessions/runtime-context-keys.js").CompiledBundle;
readonly cancelOwnedWork?: TaskExecutorCancel;
- readonly deliverUpdate?: DeliverTaskUpdate;
- readonly parentStepIndex?: number;
- readonly parentTurnId: string;
readonly serializedContext?: Record<string, unknown>;
readonly session: RuntimeSession;
}): Promise<{
@@ -53,39 +37,6 @@ export async function executeTaskControlAction(input: {
readonly pendingTask?: BackgroundTask;
}> {
const { action, session } = input;
- if (action.toolName === TASK_UPDATE_TOOL_NAME) {
- const message = action.input.message;
- if (typeof message !== "string" || message.trim() === "") {
- return {
- result: createTaskControlError(action, "Provide a non-empty `message`."),
- session,
- };
- }
- const update = {
- callId: action.callId,
- kind: "task-update",
- message,
- updateEpoch: input.parentTurnId,
- updateIndex: input.parentStepIndex ?? 0,
- } satisfies TaskInboundUpdate;
- const taskId = await input.deliverUpdate?.({
- adapter: input.adapter,
- callback: input.serializedContext?.["eve.sessionCallback"],
- update,
- });
- return {
- result:
- taskId === undefined
- ? createTaskControlError(action, "task_update requires a task-owned session.")
- : {
- callId: action.callId,
- kind: "tool-result",
- output: { status: "sent", taskId },
- toolName: action.toolName,
- },
- session,
- };
- }
const taskIds = readTaskIds(action.input);
if (taskIds === undefined || taskIds.length === 0) {
return {
diff --git a/packages/eve/src/execution/tools/subagent/invoke-agent.test.ts b/packages/eve/src/execution/tools/subagent/invoke-agent.test.ts
index bd91dc6bd..61dc69f43 100644
--- a/packages/eve/src/execution/tools/subagent/invoke-agent.test.ts
+++ b/packages/eve/src/execution/tools/subagent/invoke-agent.test.ts
@@ -556,64 +556,4 @@ describe("background agent invocation routing", () => {
expect.objectContaining({ kind: "report" }),
);
});
-
- it("forwards task-owned child updates to the workflow tool report channel", async () => {
- const replies: AgentInvocationReply[] = [
- {
- callId: "task-update-call",
- kind: "task-update",
- message: "Still working",
- updateEpoch: "turn-child",
- updateIndex: 0,
- },
- {
- kind: "runtime-action-result",
- results: [
- {
- callId: "call-1:agent-reply",
- kind: "subagent-result",
- origin: "child",
- output: "done",
- subagentName: "research",
- } as never,
- ],
- },
- ];
- mocks.createHook.mockReturnValue({
- [Symbol.asyncIterator]: () => ({
- next: async () =>
- replies.length > 0
- ? { done: false as const, value: replies.shift()! }
- : { done: true as const, value: undefined },
- }),
- token: "agent-reply",
- });
- mocks.resumeHook.mockImplementation(async () => undefined);
- const from: WorkflowToolRunRef = {
- callId: "call-1",
- execution: "background",
- input: { message: "Find it", target: "research" },
- runId: "run-1",
- sequence: 0,
- stepIndex: 2,
- toolName: "research",
- turnId: "turn-1",
- };
- const ctx = { callId: "call-1" } as ToolContext;
- attachWorkflowToolRunContext(ctx, {
- admission: Promise.resolve({ status: "accepted" }),
- from,
- owner: {
- inbox: "owner-inbox",
- },
- });
-
- await expect(agent(ctx, "research", { message: "Find it" })).resolves.toBe("done");
-
- expect(mocks.resumeHook).toHaveBeenCalledWith("owner-inbox", {
- kind: "report",
- from,
- update: "Still working",
- });
- });
});
diff --git a/packages/eve/src/execution/tools/subagent/invoke-agent.ts b/packages/eve/src/execution/tools/subagent/invoke-agent.ts
index 304706cf4..d52496376 100644
--- a/packages/eve/src/execution/tools/subagent/invoke-agent.ts
+++ b/packages/eve/src/execution/tools/subagent/invoke-agent.ts
@@ -18,7 +18,6 @@ import { disposeHook } from "#execution/hook-ownership.js";
import { sessionCommandHookToken } from "#execution/session-command-token.js";
import type { AgentInput } from "#tools/workflow-definition.js";
import type { ToolContext } from "#tools/definition.js";
-import type { TaskInboundUpdate } from "#tasks/types.js";
export type InternalAgentInput = {
readonly agentId?: string;
@@ -51,10 +50,7 @@ export type AgentInvocationEvent =
| SubagentAuthorizationEventHookPayload
| SubagentInputRequestHookPayload;
-export type AgentInvocationReply =
- | AgentInvocationEvent
- | RuntimeActionResultHookPayload
- | TaskInboundUpdate;
+export type AgentInvocationReply = AgentInvocationEvent | RuntimeActionResultHookPayload;
/** Invokes an agent from a workflow tool. */
export async function agent(
@@ -151,14 +147,6 @@ export async function invokeAgent(
});
continue;
}
- if (reply.kind === "task-update") {
- await resumeHookStep(owner.inbox, {
- kind: "report",
- from: run,
- update: reply.message,
- });
- continue;
- }
await resumeHookStep(owner.inbox, {
kind: "request",
from: run,
diff --git a/packages/eve/src/execution/tools/tasks.ts b/packages/eve/src/execution/tools/tasks.ts
index 1d936a72f..40f4e6b0d 100644
--- a/packages/eve/src/execution/tools/tasks.ts
+++ b/packages/eve/src/execution/tools/tasks.ts
@@ -2,44 +2,20 @@ import type { HarnessToolDefinition } from "#harness/execute-tool.js";
import {
TASK_CANCEL_INPUT_SCHEMA,
TASK_CANCEL_TOOL_NAME,
- TASK_UPDATE_INPUT_SCHEMA,
- TASK_UPDATE_TOOL_NAME,
TASK_VIEWS_OUTPUT_SCHEMA,
} from "#tools/framework/task-contract.js";
-/** Framework controls for durable background tasks. */
-
-const TASK_CANCEL_DESCRIPTION =
- "Request cooperative cancellation of one or more background tasks. " +
- "Cancellation is final: a task that finishes after you cancel it stays cancelled. Cancelling an already-finished task changes nothing.";
-
-const TASK_UPDATE_DESCRIPTION =
- "Briefly tell the parent agent what this background task is currently doing. " +
- "Report activity, not preliminary findings or results.";
-
-/**
- * Builds harness definitions for the compiled framework task tools.
- */
+/** Builds the harness definition for the compiled framework task control. */
export function createTaskToolHarnessDefinitions(): readonly HarnessToolDefinition[] {
return [
{
- description: TASK_CANCEL_DESCRIPTION,
+ description:
+ "Request cooperative cancellation of one or more background tasks. " +
+ "Cancellation is final: a task that finishes after you cancel it stays cancelled. Cancelling an already-finished task changes nothing.",
inputSchema: TASK_CANCEL_INPUT_SCHEMA,
name: TASK_CANCEL_TOOL_NAME,
outputSchema: TASK_VIEWS_OUTPUT_SCHEMA,
runtimeAction: { kind: "task-control" },
},
- {
- description: TASK_UPDATE_DESCRIPTION,
- inputSchema: TASK_UPDATE_INPUT_SCHEMA,
- name: TASK_UPDATE_TOOL_NAME,
- runtimeAction: { kind: "task-control" },
- },
];
}
-
-/**
- * Authored tools with the same name shadow these framework definitions,
- * and `disableTool(name)` removes individual controls. advertised-tools
- * evaluates their selected availability against persisted session metadata.
- */
diff --git a/packages/eve/src/execution/workflow-steps.test.ts b/packages/eve/src/execution/workflow-steps.test.ts
index e27c3cde2..196bd5481 100644
--- a/packages/eve/src/execution/workflow-steps.test.ts
+++ b/packages/eve/src/execution/workflow-steps.test.ts
@@ -961,7 +961,7 @@ describe("dispatchCoordinationStep", () => {
callId: "call-1",
input: {},
kind: "tool-call",
- toolName: "task_update",
+ toolName: "task_cancel",
},
],
tasks: [],
diff --git a/packages/eve/src/execution/workflow-steps.ts b/packages/eve/src/execution/workflow-steps.ts
index ac8d8a193..c81d1def6 100644
--- a/packages/eve/src/execution/workflow-steps.ts
+++ b/packages/eve/src/execution/workflow-steps.ts
@@ -87,7 +87,6 @@ import {
readRetainedBackgroundToolResult,
runBackgroundStep,
} from "#execution/tasks/parent/tool-execution.js";
-import { TASK_UPDATE_SESSION_INSTRUCTION } from "#tools/framework/task-update.js";
import { prepareWorkflowPreambleTrace } from "#execution/workflow-trace-context.js";
import { resolveEffectiveAgentRuntime } from "#execution/effective-agent-config.js";
import { reconcileSessionContinuationToken } from "#execution/reconcile-session-continuation-token.js";
@@ -134,14 +133,6 @@ export async function turnStep(rawInput: TurnStepInput): Promise<DurableStepResu
const adapter = ctx.require(ChannelKey);
const bundle = ctx.require(BundleKey);
const effectiveAgent = resolveEffectiveAgentRuntime(bundle, ctx);
- const taskUpdatesEnabled =
- durableSession.taskId !== undefined &&
- effectiveAgent.turnAgent.tools.some(
- (tool) =>
- tool.kind === "authored-tool" &&
- tool.behavior?.handling?.kind === "dispatch" &&
- tool.behavior.handling.target.kind === "task-update",
- );
// Populate the callback base URL so getHookUrl() works during tool
// execution, preferring eve's active local origin over metadata fallback.
@@ -473,7 +464,6 @@ export async function turnStep(rawInput: TurnStepInput): Promise<DurableStepResu
thresholdPercent: effectiveAgent.thresholdPercent,
},
session: lifecycleSession,
- systemPromptAdditions: taskUpdatesEnabled ? [TASK_UPDATE_SESSION_INSTRUCTION] : undefined,
turnAgent: effectiveAgent.turnAgent,
});
const modelSession = refreshedSession;
diff --git a/packages/eve/src/framework/sources/registry.ts b/packages/eve/src/framework/sources/registry.ts
index 2b7a77969..58e601604 100644
--- a/packages/eve/src/framework/sources/registry.ts
+++ b/packages/eve/src/framework/sources/registry.ts
@@ -9,7 +9,7 @@ import {
type ProgrammaticModuleNamespace,
} from "#compiler/source-graph.js";
-const revision = `eve@${resolveInstalledPackageInfo().version}:compiled-manifest-v47`;
+const revision = `eve@${resolveInstalledPackageInfo().version}:compiled-manifest-v48`;
const localDefaults = defineProgrammaticAgentSource({
id: "eve:defaults",
@@ -53,10 +53,6 @@ const localDefaults = defineProgrammaticAgentSource({
logicalPath: "tools/ask_question.ts",
loadNamespace: () => import("#tools/framework/ask-question.js"),
},
- {
- logicalPath: "tools/task_update.ts",
- loadNamespace: () => import("#tools/framework/task-update.js"),
- },
{
logicalPath: "tools/task_cancel.ts",
loadNamespace: () => import("#tools/framework/task-cancel.js"),
diff --git a/packages/eve/src/harness/advertised-tools.test.ts b/packages/eve/src/harness/advertised-tools.test.ts
index 3003886d9..baae58d9a 100644
--- a/packages/eve/src/harness/advertised-tools.test.ts
+++ b/packages/eve/src/harness/advertised-tools.test.ts
@@ -103,23 +103,23 @@ describe("getAdvertisedTools for definition arrays", () => {
expect(advertisedTools.map((tool) => tool.name)).toEqual(["add", "delegate"]);
});
- it("keeps the task tools in the root session", () => {
+ it("keeps root-session tools in the root session", () => {
const tools = new Map([
["add", createTool("add")],
- ["task_cancel", createTaskControlTool("task_cancel", ["root-session"])],
- ["task_update", createTaskControlTool("task_update", ["delegated-task-child"])],
+ ["root_only", createAvailableTool("root_only", ["root-session"])],
+ ["child_only", createAvailableTool("child_only", ["delegated-task-child"])],
]) satisfies HarnessToolMap;
const advertisedTools = getAdvertisedTools({ session: {}, tools });
- expect([...advertisedTools.keys()]).toEqual(["add", "task_cancel"]);
+ expect([...advertisedTools.keys()]).toEqual(["add", "root_only"]);
});
it("exposes delegated-task-child tools from persisted session ownership", () => {
const tools = new Map([
["add", createTool("add")],
- ["task_cancel", createTaskControlTool("task_cancel", ["root-session"])],
- ["task_update", createTaskControlTool("task_update", ["delegated-task-child"])],
+ ["root_only", createAvailableTool("root_only", ["root-session"])],
+ ["child_only", createAvailableTool("child_only", ["delegated-task-child"])],
]) satisfies HarnessToolMap;
const advertisedTools = getAdvertisedTools({
@@ -127,19 +127,22 @@ describe("getAdvertisedTools for definition arrays", () => {
tools,
});
- expect([...advertisedTools.keys()]).toEqual(["add", "task_update"]);
+ expect([...advertisedTools.keys()]).toEqual(["add", "child_only"]);
});
- it("removes task_update from sessions without task ownership", () => {
- const tools = new Map([
- ["add", createTool("add")],
- ["task_update", createTaskControlTool("task_update", ["delegated-task-child"])],
- ]) satisfies HarnessToolMap;
+ it.each([{}, { rootSessionId: "root-session" }])(
+ "removes delegated-task-child tools from sessions without task ownership (%j)",
+ (session) => {
+ const tools = new Map([
+ ["add", createTool("add")],
+ ["child_only", createAvailableTool("child_only", ["delegated-task-child"])],
+ ]) satisfies HarnessToolMap;
- const advertisedTools = getAdvertisedTools({ session: {}, tools });
+ const advertisedTools = getAdvertisedTools({ session, tools });
- expect([...advertisedTools.keys()]).toEqual(["add"]);
- });
+ expect([...advertisedTools.keys()]).toEqual(["add"]);
+ },
+ );
});
function createTool(name: string): HarnessToolDefinition {
@@ -165,14 +168,13 @@ function createBuiltInAgentTool(): HarnessToolDefinition {
};
}
-function createTaskControlTool(
+function createAvailableTool(
name: string,
availability: NonNullable<HarnessToolDefinition["behavior"]>["availability"],
): HarnessToolDefinition {
return {
...createTool(name),
behavior: { availability },
- runtimeAction: { kind: "task-control" },
};
}
diff --git a/packages/eve/src/harness/coordination.test.ts b/packages/eve/src/harness/coordination.test.ts
index c3ea1a0be..64de7e1b7 100644
--- a/packages/eve/src/harness/coordination.test.ts
+++ b/packages/eve/src/harness/coordination.test.ts
@@ -228,14 +228,14 @@ describe("createCoordinationRequestFromToolCall", () => {
it("reserves runtime actions for task controls", () => {
expect(
createCoordinationRequestFromToolCall({
- toolCall: { ...toolCall, toolName: "task_update" },
+ toolCall: { ...toolCall, input: { taskIds: ["task-1"] }, toolName: "task_cancel" },
tools: new Map([
[
- "task_update",
+ "task_cancel",
{
- description: "Update a task.",
+ description: "Cancel tasks.",
inputSchema: jsonSchema({ type: "object" }),
- name: "task_update",
+ name: "task_cancel",
runtimeAction: { kind: "task-control" as const },
},
],
@@ -245,9 +245,9 @@ describe("createCoordinationRequestFromToolCall", () => {
kind: "runtime-action",
request: {
callId: "call-1",
- input: { message: "research this" },
+ input: { taskIds: ["task-1"] },
kind: "tool-call",
- toolName: "task_update",
+ toolName: "task_cancel",
},
});
});
diff --git a/packages/eve/src/harness/emission.ts b/packages/eve/src/harness/emission.ts
index 79450e377..3a08e8599 100644
--- a/packages/eve/src/harness/emission.ts
+++ b/packages/eve/src/harness/emission.ts
@@ -688,7 +688,11 @@ async function consumeStreamContent(
// Channel adapters deliver terminal completions, so the reserved marker
// becomes a null completion without delaying normal streaming deltas.
- if (finishReason !== "tool-calls" && hasEmptyDeliverySentinel(currentMessage)) {
+ if (
+ finishReason !== "content-filter" &&
+ finishReason !== "tool-calls" &&
+ hasEmptyDeliverySentinel(currentMessage)
+ ) {
await emitFn(
createMessageCompletedEvent({
finishReason,
@@ -698,7 +702,7 @@ async function consumeStreamContent(
turnId: state.turnId,
}),
);
- } else if (currentMessage.trim().length > 0) {
+ } else if (finishReason !== "content-filter" && currentMessage.trim().length > 0) {
await emitFn(
createMessageCompletedEvent({
finishReason,
diff --git a/packages/eve/src/harness/execute-tool.ts b/packages/eve/src/harness/execute-tool.ts
index 721cbe51f..2f5d3dc01 100644
--- a/packages/eve/src/harness/execute-tool.ts
+++ b/packages/eve/src/harness/execute-tool.ts
@@ -9,10 +9,8 @@ import type { PreparedToolBehavior } from "#tools/behavior.js";
/**
* Runtime-owned action metadata attached to one harness-visible tool.
*
- * `task-control` marks the parent-side background task tools
- * (`task_cancel`, `task_update`): they carry no child
- * address of their own — the dispatch step resolves targets through the
- * session task index by tool name.
+ * `task-control` marks `task_cancel`: it carries no child address of its
+ * own — the dispatch step resolves targets through the session task index.
*/
export type HarnessRuntimeActionDefinition = { readonly kind: "task-control" };
diff --git a/packages/eve/src/harness/model-call-error.test.ts b/packages/eve/src/harness/model-call-error.test.ts
index 80fde4c6b..49c1f1daf 100644
--- a/packages/eve/src/harness/model-call-error.test.ts
+++ b/packages/eve/src/harness/model-call-error.test.ts
@@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest";
import {
classifyModelCallError,
+ ContentFilteredModelResponseError,
EmptyModelResponseError,
extractModelCallErrorDetails,
extractUnsupportedProviderToolTypes,
@@ -130,6 +131,20 @@ describe("EmptyModelResponseError", () => {
});
});
+describe("ContentFilteredModelResponseError", () => {
+ it("reports safe filtering diagnostics without classifying the response as retryable", () => {
+ const error = new ContentFilteredModelResponseError("gen_filtered");
+ expect(classifyModelCallError(error)).toBe("recoverable");
+ expect(extractModelCallErrorDetails(error)).toEqual({
+ finishReason: "content-filter",
+ generationId: "gen_filtered",
+ });
+ expect(extractModelCallErrorDetails(new ContentFilteredModelResponseError())).toEqual({
+ finishReason: "content-filter",
+ });
+ });
+});
+
describe("normalizeModelStreamError", () => {
it("retains a plain provider payload for classification", () => {
const raw = { message: "Overloaded", type: "overloaded_error" };
diff --git a/packages/eve/src/harness/model-call-error.ts b/packages/eve/src/harness/model-call-error.ts
index fc157a50d..88eedb543 100644
--- a/packages/eve/src/harness/model-call-error.ts
+++ b/packages/eve/src/harness/model-call-error.ts
@@ -167,6 +167,10 @@ function collectUnsupportedToolTypesFromValue(value: unknown, out: Set<string>):
export function extractModelCallErrorDetails(error: unknown): JsonObject {
const signals = readModelCallErrorSignals(error);
const details: Record<string, JsonValue> = {};
+ if (error instanceof ContentFilteredModelResponseError) {
+ details.finishReason = "content-filter";
+ appendJsonField(details, "generationId", error.generationId);
+ }
appendJsonField(details, "apiErrorMessage", signals.apiErrorMessage);
appendJsonField(details, "gatewayName", signals.gatewayName);
@@ -221,6 +225,17 @@ export class EmptyModelResponseError extends Error {
}
}
+/** Provider filtering is a completed rejection, not an empty response to reissue. */
+export class ContentFilteredModelResponseError extends Error {
+ readonly generationId?: string;
+
+ constructor(generationId?: string) {
+ super("The model provider filtered this response.");
+ this.name = "ContentFilteredModelResponseError";
+ this.generationId = generationId;
+ }
+}
+
/**
* Coerces a streamed provider failure into an Error while retaining the raw
* payload so provider discriminators remain visible to error classification.
diff --git a/packages/eve/src/harness/semantic-errors/rules/model-provider.ts b/packages/eve/src/harness/semantic-errors/rules/model-provider.ts
index af5a5d157..0f8fc516e 100644
--- a/packages/eve/src/harness/semantic-errors/rules/model-provider.ts
+++ b/packages/eve/src/harness/semantic-errors/rules/model-provider.ts
@@ -32,6 +32,14 @@ export const MODEL_PROVIDER_RULES: readonly SemanticErrorRule[] = [
when: nameIs("EmptyModelResponseError"),
message: (link) => link.message,
},
+ {
+ id: "model-response-content-filtered",
+ name: "Model response filtered",
+ tags: ["model-provider", "recoverable"],
+ when: nameIs("ContentFilteredModelResponseError"),
+ message: "The model provider filtered this response.",
+ hint: "Review the request and its context against your model provider's content policy.",
+ },
{
id: "model-capability-unsupported",
name: "Model capability not supported",
diff --git a/packages/eve/src/harness/semantic-errors/semantic-errors.test.ts b/packages/eve/src/harness/semantic-errors/semantic-errors.test.ts
index c1b72929e..9a6da4856 100644
--- a/packages/eve/src/harness/semantic-errors/semantic-errors.test.ts
+++ b/packages/eve/src/harness/semantic-errors/semantic-errors.test.ts
@@ -102,6 +102,14 @@ describe("summarizeKnownError (catalog table)", () => {
),
id: "empty-model-response",
},
+ {
+ title: "content-filtered model response",
+ error: named(
+ "ContentFilteredModelResponseError",
+ "The model provider filtered this response.",
+ ),
+ id: "model-response-content-filtered",
+ },
{
title: "unsupported model capability",
error: named("AI_UnsupportedFunctionalityError", "tool type not supported"),
diff --git a/packages/eve/src/harness/step-hooks.ts b/packages/eve/src/harness/step-hooks.ts
index 246d4729f..e771d8199 100644
--- a/packages/eve/src/harness/step-hooks.ts
+++ b/packages/eve/src/harness/step-hooks.ts
@@ -479,7 +479,7 @@ function extractGatewayCostUsd(providerMetadata: ProviderMetadata | undefined):
return undefined;
}
-function readGatewayGenerationId(
+export function readGatewayGenerationId(
providerMetadata: ProviderMetadata | undefined,
): string | undefined {
const generationId = readGatewayMetadata(providerMetadata)?.generationId;
diff --git a/packages/eve/src/harness/tool-loop-content-filter.integration.test.ts b/packages/eve/src/harness/tool-loop-content-filter.integration.test.ts
new file mode 100644
index 000000000..ce920b123
--- /dev/null
+++ b/packages/eve/src/harness/tool-loop-content-filter.integration.test.ts
@@ -0,0 +1,111 @@
+import { simulateReadableStream } from "ai";
+import { MockLanguageModelV4 } from "ai/test";
+import { describe, expect, it } from "vitest";
+
+import { createToolLoopHarness } from "#harness/tool-loop.js";
+import type { HarnessSession } from "#harness/types.js";
+import type { UnstampedMessageStreamEvent } from "#protocol/message.js";
+
+const usage = {
+ inputTokens: { total: 10, noCache: 10, cacheRead: undefined, cacheWrite: undefined },
+ outputTokens: { total: 0, text: 0, reasoning: undefined },
+};
+const providerMetadata = { gateway: { generationId: "gen_filtered" } };
+const finishReason = { unified: "content-filter", raw: "content_filter" } as const;
+type StreamResult = Awaited<ReturnType<MockLanguageModelV4["doStream"]>>;
+type StreamPart = StreamResult["stream"] extends ReadableStream<infer Part> ? Part : never;
+
+function session(): HarnessSession {
+ return {
+ agent: { modelReference: { id: "filtered-model" }, system: "Help Alice and Bob.", tools: [] },
+ compaction: { recentWindowSize: 10, threshold: 100_000 },
+ continuationToken: "http:filtered-session",
+ history: [],
+ sessionId: "filtered-session",
+ };
+}
+
+describe("content-filter reporting (real AI SDK)", () => {
+ it.each(["", "Alice's inventory list is"])(
+ "reports a filtered stream without retry or successful delivery: %j",
+ async (text) => {
+ let calls = 0;
+ const model = new MockLanguageModelV4({
+ doStream: async () => {
+ calls++;
+ const chunks: StreamPart[] = [{ type: "stream-start", warnings: [] }];
+ if (text !== "")
+ chunks.push(
+ { type: "text-start", id: "answer" },
+ { type: "text-delta", id: "answer", delta: text },
+ { type: "text-end", id: "answer" },
+ );
+ chunks.push({ type: "finish", finishReason, usage, providerMetadata });
+ return { stream: simulateReadableStream({ chunks }) };
+ },
+ });
+ const events: UnstampedMessageStreamEvent[] = [];
+ const step = createToolLoopHarness({
+ mode: "conversation",
+ resolveModel: async () => model,
+ tools: new Map(),
+ handleEvent: async (event) => {
+ events.push(event);
+ },
+ });
+ const result = await step(session(), { message: "Help Alice prepare Bob's inventory list." });
+ expect(calls).toBe(1);
+ expect(result.next).toBeNull();
+ expect(events.find((event) => event.type === "step.failed")).toMatchObject({
+ data: {
+ message: "The model provider filtered this response.",
+ details: {
+ finishReason: "content-filter",
+ generationId: "gen_filtered",
+ semanticErrorId: "model-response-content-filtered",
+ },
+ },
+ });
+ expect(
+ events.some(
+ (event) =>
+ event.type === "message.completed" ||
+ event.type === "step.completed" ||
+ event.type === "turn.completed",
+ ),
+ ).toBe(false);
+ expect(events.some((event) => event.type === "session.waiting")).toBe(true);
+ },
+ );
+
+ it.each(["", "Alice's inventory list is"])(
+ "rejects a filtered generated response without retry: %j",
+ async (text) => {
+ let calls = 0;
+ const model = new MockLanguageModelV4({
+ doGenerate: async () => {
+ calls++;
+ return {
+ content: text === "" ? [] : [{ type: "text", text }],
+ finishReason,
+ usage,
+ providerMetadata,
+ warnings: [],
+ };
+ },
+ });
+ const step = createToolLoopHarness({
+ mode: "task",
+ resolveModel: async () => model,
+ tools: new Map(),
+ });
+ await expect(
+ step(session(), { message: "Help Alice prepare Bob's inventory list." }),
+ ).rejects.toMatchObject({
+ name: "ContentFilteredModelResponseError",
+ generationId: "gen_filtered",
+ });
+ expect(calls).toBe(1);
+ },
+ );
+});
diff --git a/packages/eve/src/harness/tool-loop.test.ts b/packages/eve/src/harness/tool-loop.test.ts
index cab4d7594..9a4148a7d 100644
--- a/packages/eve/src/harness/tool-loop.test.ts
+++ b/packages/eve/src/harness/tool-loop.test.ts
@@ -3403,6 +3403,87 @@ describe("createToolLoopHarness", () => {
});
});
+ it.each([
+ ["conversation", ""],
+ ["conversation", "Alice's inventory list is"],
+ ["task", ""],
+ ["task", "Alice's inventory list is"],
+ ] as const)(
+ "reports content-filter without retrying in %s mode (text: %j)",
+ async (mode, text) => {
+ setupMockAgent({
+ finishReason: "content-filter",
+ providerMetadata: {
+ gateway: { generationId: "gen_filtered", privateDetail: "not-for-clients" },
+ },
+ response: { messages: text === "" ? [] : [{ content: text, role: "assistant" }] },
+ text,
+ toolCalls: [],
+ toolResults: [],
+ });
+ const { emit, events } = createEventCollector();
+ const result = await createToolLoopHarness(createTestConfig(mode, emit))(
+ createTestSession(),
+ { message: "Help Alice prepare Bob's inventory list." },
+ );
+
+ expect(ToolLoopAgent).toHaveBeenCalledTimes(1);
+ expect(events.find((event) => event.type === "step.failed")).toMatchObject({
+ data: {
+ code: "MODEL_CALL_FAILED",
+ message: "The model provider filtered this response.",
+ details: {
+ finishReason: "content-filter",
+ generationId: "gen_filtered",
+ semanticErrorId: "model-response-content-filtered",
+ },
+ },
+ });
+ expect(
+ events.some(
+ (event) =>
+ event.type === "step.completed" ||
+ event.type === "turn.completed" ||
+ event.type === "message.completed",
+ ),
+ ).toBe(false);
+ expect(JSON.stringify(events.filter((event) => event.type === "step.failed"))).not.toContain(
+ "not-for-clients",
+ );
+ if (mode === "task") {
+ expect(result.next).toMatchObject({
+ done: true,
+ isError: true,
+ output: expect.stringContaining("filtered this response"),
+ });
+ } else {
+ expect(result.next).toBeNull();
+ expect(events.some((event) => event.type === "session.waiting")).toBe(true);
+ }
+ },
+ );
+
+ it("throws a distinct content-filter error on the non-streaming path without reissue", async () => {
+ setupMockAgent({
+ finishReason: "content-filter",
+ providerMetadata: { gateway: { generationId: "gen_filtered" } },
+ response: { messages: [] },
+ text: "",
+ toolCalls: [],
+ toolResults: [],
+ });
+ await expect(
+ createToolLoopHarness(createTestConfig())(createTestSession(), {
+ message: "Help Alice prepare Bob's inventory list.",
+ }),
+ ).rejects.toMatchObject({
+ name: "ContentFilteredModelResponseError",
+ message: "The model provider filtered this response.",
+ generationId: "gen_filtered",
+ });
+ expect(ToolLoopAgent).toHaveBeenCalledTimes(1);
+ });
+
it("emits session.completed instead of session.waiting in task mode", async () => {
setupMockAgent({
finishReason: "stop",
diff --git a/packages/eve/src/harness/tool-loop.ts b/packages/eve/src/harness/tool-loop.ts
index 9aebf29b3..d34b86ab4 100644
--- a/packages/eve/src/harness/tool-loop.ts
+++ b/packages/eve/src/harness/tool-loop.ts
@@ -188,6 +188,7 @@ import {
} from "#protocol/message.js";
import {
classifyModelCallError,
+ ContentFilteredModelResponseError,
EmptyModelResponseError,
extractModelCallErrorDetails,
extractUnsupportedProviderToolTypes,
@@ -223,7 +224,12 @@ import {
getInvalidToolCallInputError,
isInvalidToolCall,
} from "#harness/tool-call-input-errors.js";
-import { buildStepHooks, emitStepActions, type HarnessStepResult } from "#harness/step-hooks.js";
+import {
+ buildStepHooks,
+ emitStepActions,
+ readGatewayGenerationId,
+ type HarnessStepResult,
+} from "#harness/step-hooks.js";
import { mergeProviderSafetyIdentifier } from "#harness/provider-safety.js";
import {
buildToolApproval,
@@ -1509,6 +1515,11 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {
hooks.stepResult,
streamResult.responseMessages,
]);
+ if (stepResult.finishReason === "content-filter") {
+ throw new ContentFilteredModelResponseError(
+ readGatewayGenerationId(stepResult.providerMetadata),
+ );
+ }
if (
isEmptyModelResponse(stepResult) &&
extractToolResultCallIds(accumulatedResponseMessages).size === 0 &&
@@ -1547,6 +1558,11 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {
});
throwIfTurnAborted(config.abortSignal);
const stepResult = await hooks.stepResult;
+ if (stepResult.finishReason === "content-filter") {
+ throw new ContentFilteredModelResponseError(
+ readGatewayGenerationId(stepResult.providerMetadata),
+ );
+ }
if (
isEmptyModelResponse(stepResult) &&
extractToolResultCallIds(generateResult.responseMessages).size === 0
@@ -1765,12 +1781,14 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {
if (config.mode === "task") {
if (
classification === "recoverable" &&
- !(finalError instanceof EmptyModelResponseError)
+ !(finalError instanceof EmptyModelResponseError) &&
+ !(finalError instanceof ContentFilteredModelResponseError)
) {
// Task runs cannot park for user-driven recovery. Let the durable
// step retry from committed session state, but only for errors
// that did not already consume the in-process transient budget or
- // the dedicated empty-response reissue.
+ // the dedicated empty-response reissue. Filtering is an explicit
+ // rejection, so it must not receive durable retries either.
log.warn(
upstreamRejection?.message ??
"model call failed recoverably in task mode — rethrowing for durable step retry",
diff --git a/packages/eve/src/public/tools/task-update.ts b/packages/eve/src/public/tools/task-update.ts
deleted file mode 100644
index 520bc31c6..000000000
--- a/packages/eve/src/public/tools/task-update.ts
+++ /dev/null
@@ -1 +0,0 @@
-export { taskUpdate, taskUpdate as default } from "#tools/framework/task-update.js";
diff --git a/packages/eve/src/shared/dispatch-action.ts b/packages/eve/src/shared/dispatch-action.ts
index a088fe343..94d2afe25 100644
--- a/packages/eve/src/shared/dispatch-action.ts
+++ b/packages/eve/src/shared/dispatch-action.ts
@@ -20,10 +20,7 @@ export type PendingAgentDispatchAction = PendingDispatchAction & {
/** Pending action that mutates the durable task lifecycle. */
export type PendingTaskControlAction = PendingDispatchAction & {
- readonly target: Extract<
- PreparedDispatchTarget,
- { readonly kind: "task-cancel" | "task-update" }
- >;
+ readonly target: Extract<PreparedDispatchTarget, { readonly kind: "task-cancel" }>;
};
/** Pending action that runs a tool's execute body as a durable workflow. */
diff --git a/packages/eve/src/subagents/callback-route.test.ts b/packages/eve/src/subagents/callback-route.test.ts
index 9011d2d32..b8ba39832 100644
--- a/packages/eve/src/subagents/callback-route.test.ts
+++ b/packages/eve/src/subagents/callback-route.test.ts
@@ -164,32 +164,32 @@ describe("session callback route", () => {
});
});
- it("retains the generic task progress callback", async () => {
- resumeHookMock.mockResolvedValue(undefined);
- const response = await handleSessionCallbackRequest(
- new Request(`https://app.example.com/eve/v1/callback/${TASK_TOKEN}`, {
- body: JSON.stringify({
- callId: "update-call",
- updateIndex: 2,
- updateEpoch: "turn-child",
- kind: "task.update",
- message: "Found three matching records.",
- taskId: TASK_ID,
+ it.each([TASK_TOKEN, "invocation-reply"])(
+ "rejects obsolete child task.update callbacks on %s",
+ async (token) => {
+ const response = await handleSessionCallbackRequest(
+ new Request(`https://app.example.com/eve/v1/callback/${token}`, {
+ body: JSON.stringify({
+ callId: "update-call",
+ updateIndex: 2,
+ updateEpoch: "turn-child",
+ kind: "task.update",
+ message: "Found three matching records.",
+ taskId: TASK_ID,
+ }),
+ method: "POST",
}),
- method: "POST",
- }),
- createRouteContext({ token: TASK_TOKEN }),
- );
-
- expect(response.status).toBe(202);
- expect(resumeHookMock).toHaveBeenCalledWith(TASK_TOKEN, {
- callId: "update-call",
- updateIndex: 2,
- updateEpoch: "turn-child",
- kind: "task-update",
- message: "Found three matching records.",
- });
- });
+ createRouteContext({ token }),
+ );
+
+ expect(response.status).toBe(400);
+ await expect(response.json()).resolves.toEqual({
+ error: "Unsupported callback kind.",
+ ok: false,
+ });
+ expect(resumeHookMock).not.toHaveBeenCalled();
+ },
+ );
it.each([
["parent turn token", "turn-inbox", TASK_ID],
diff --git a/packages/eve/src/subagents/callback-route.ts b/packages/eve/src/subagents/callback-route.ts
index a3e9d425c..7e5ff1263 100644
--- a/packages/eve/src/subagents/callback-route.ts
+++ b/packages/eve/src/subagents/callback-route.ts
@@ -13,7 +13,6 @@ import { jsonValueSchema } from "#shared/json-schemas.js";
import type { JsonValue } from "#shared/json.js";
import { isInputRequest } from "#shared/input.js";
import { tokenUsageWithCostSchema, type TokenUsage } from "#shared/token-usage.js";
-import type { TaskInboundUpdate } from "#tasks/types.js";
import { readTaskIdFromInboxToken } from "#tasks/task-inbox-token.js";
const ZERO_TOKEN_USAGE: TokenUsage = {
@@ -111,15 +110,6 @@ const taskTurnStartedCallbackSchema = z.object({
turnId: z.string().min(1),
});
-const taskUpdateCallbackSchema = z.object({
- callId: z.string().min(1),
- kind: z.literal("task.update"),
- message: z.string().min(1),
- taskId: z.string().min(1),
- updateEpoch: z.string().min(1),
- updateIndex: eventCoordinateSchema,
-});
-
/**
* Turn callbacks must carry the explicit `AgentTurnOutcome` envelope:
* the receiving parent settles the child's handle from `outcome.kind`, so
@@ -187,17 +177,6 @@ export async function handleSessionCallbackRequest(
return Response.json({ ok: true }, { status: 202 });
}
- const update = projectTaskUpdate(body, token);
- if (update instanceof Response) return update;
- if (update !== undefined) {
- try {
- await resumeHook(token, update);
- } catch {
- return Response.json({ error: "Session callback not pending.", ok: false }, { status: 404 });
- }
- return Response.json({ ok: true }, { status: 202 });
- }
-
const started = rejectDirectTaskTurnStarted(body, token);
if (started instanceof Response) return started;
if (started !== undefined) {
@@ -285,28 +264,6 @@ function rejectDirectTaskTurnStarted(value: unknown, token: string): Response |
);
}
-function projectTaskUpdate(
- value: unknown,
- token: string,
-): TaskInboundUpdate | Response | undefined {
- if (callbackKind(value) !== "task.update") return undefined;
- const parsed = taskUpdateCallbackSchema.safeParse(value);
- if (!parsed.success) {
- return Response.json({ error: "Invalid task update callback.", ok: false }, { status: 400 });
- }
- if (readTaskIdFromInboxToken(token) !== undefined) {
- const tokenRejection = rejectMismatchedTaskToken(token, parsed.data.taskId);
- if (tokenRejection !== undefined) return tokenRejection;
- }
- return {
- callId: parsed.data.callId,
- kind: "task-update",
- message: parsed.data.message,
- updateEpoch: parsed.data.updateEpoch,
- updateIndex: parsed.data.updateIndex,
- };
-}
-
function rejectMismatchedTaskToken(token: string, taskId: string): Response | undefined {
return readTaskIdFromInboxToken(token) === taskId
? undefined
diff --git a/packages/eve/src/subagents/callback-step.test.ts b/packages/eve/src/subagents/callback-step.test.ts
index e06446605..9af7c3045 100644
--- a/packages/eve/src/subagents/callback-step.test.ts
+++ b/packages/eve/src/subagents/callback-step.test.ts
@@ -1,6 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
-import { fireSessionCallbackStep, fireTaskUpdateCallbackStep } from "#subagents/callback-step.js";
+import { fireSessionCallbackStep } from "#subagents/callback-step.js";
const USAGE = { cacheReadTokens: 10, cacheWriteTokens: 5, inputTokens: 100, outputTokens: 50 };
@@ -298,37 +298,6 @@ describe("fireSessionCallbackStep", () => {
});
});
-describe("fireTaskUpdateCallbackStep", () => {
- it("posts a progress update over a task-owned callback", async () => {
- const fetchMock = vi.fn().mockResolvedValue(new Response(null, { status: 202 }));
- vi.stubGlobal("fetch", fetchMock);
-
- const taskId = await fireTaskUpdateCallbackStep({
- callback: {
- callId: "parent-call",
- subagentName: "research",
- taskId: "task_abc",
- token: "task-token",
- url: "https://caller.example.com/eve/v1/callback/task-token",
- },
- callId: "update-call",
- updateIndex: 2,
- updateEpoch: "turn-child",
- message: "Found three matching records.",
- });
-
- expect(taskId).toBe("task_abc");
- expect(parsePostedBody(fetchMock)).toEqual({
- callId: "update-call",
- updateIndex: 2,
- updateEpoch: "turn-child",
- kind: "task.update",
- message: "Found three matching records.",
- taskId: "task_abc",
- });
- });
-});
-
function parsePostedBody(fetchMock: ReturnType<typeof vi.fn>): Record<string, unknown> {
const call = fetchMock.mock.calls[0];
if (call === undefined) {
diff --git a/packages/eve/src/subagents/callback-step.ts b/packages/eve/src/subagents/callback-step.ts
index 3dfc4cc81..5d82ece1b 100644
--- a/packages/eve/src/subagents/callback-step.ts
+++ b/packages/eve/src/subagents/callback-step.ts
@@ -42,35 +42,6 @@ export async function fireTaskEventCallbackStep(input: {
}
}
-/** Sends one remote task progress update over its existing parent callback. */
-export async function fireTaskUpdateCallbackStep(input: {
- readonly callback: unknown;
- readonly callId: string;
- readonly updateIndex: number;
- readonly updateEpoch: string;
- readonly message: string;
-}): Promise<string | undefined> {
- "use step";
-
- const callback = parseSerializedSessionCallback(input.callback);
- if (callback.taskId === undefined) return undefined;
- const response = await postSessionCallbackRequest({
- body: {
- callId: input.callId,
- updateIndex: input.updateIndex,
- updateEpoch: input.updateEpoch,
- kind: "task.update",
- message: input.message,
- taskId: callback.taskId,
- },
- url: callback.url,
- });
- if (!response.ok) {
- throw new Error(`Task update callback failed with HTTP ${response.status}.`);
- }
- return callback.taskId;
-}
-
/**
* Sends the configured session terminal callback.
*
diff --git a/packages/eve/src/tasks/delivery-context.test.ts b/packages/eve/src/tasks/delivery-context.test.ts
index 9ecc5f65d..220bcdaf1 100644
--- a/packages/eve/src/tasks/delivery-context.test.ts
+++ b/packages/eve/src/tasks/delivery-context.test.ts
@@ -11,7 +11,7 @@ import {
TASK_DELIVERY_INITIATING_INSTRUCTION,
TASK_DELIVERY_SETTLED_INSTRUCTION,
} from "#tasks/delivery-context.js";
-import { SESSION_TASKS_STATE_KEY } from "#tasks/session-index.js";
+import { SESSION_TASKS_STATE_KEY, type SessionTaskIndexEntry } from "#tasks/session-index.js";
import type { TaskView } from "#tasks/types.js";
const metadata = { kind: "report-probe", name: "report_probe" } as const;
@@ -76,7 +76,10 @@ describe("resolveInitiatingTaskContext", () => {
resolveInitiatingTaskContext({
state: taskState([
taskEntry("task_1", "turn_1", undefined, { data: {}, kind: "workflow-tool" }),
- taskEntry("task_2", "turn_2", undefined, { data: {}, kind: "workflow-tool" }),
+ {
+ ...taskEntry("task_2", "turn_2", undefined, { data: {}, kind: "workflow-tool" }),
+ cohortId: "task_1",
+ },
]),
turnId: "turn_1",
}),
@@ -98,7 +101,7 @@ describe("resolveInitiatingTaskContext", () => {
});
describe("resolveTaskDeliveryContext", () => {
- it("projects terminal and pending siblings from the delivered task's parent turn", () => {
+ it("includes completed siblings while a cross-turn cohort is pending", () => {
const completed = {
lastOutput: { data: { result: "first" }, type: "result" },
metadata,
@@ -107,8 +110,8 @@ describe("resolveTaskDeliveryContext", () => {
} satisfies TaskView;
const state = taskState([
taskEntry("task_1", "turn_1", completed),
- taskEntry("task_2", "turn_1"),
- taskEntry("task_3", "turn_2"),
+ { ...taskEntry("task_2", "turn_2"), cohortId: "task_1" },
+ taskEntry("task_3", "turn_1"),
]);
expect(resolveTaskDeliveryContext({ state, taskDeliveryId: "task_1:ready:completed" })).toEqual(
@@ -139,7 +142,7 @@ describe("resolveTaskDeliveryContext", () => {
resolveTaskDeliveryContext({
state: taskState([
taskEntry("task_1", "turn_1", first),
- taskEntry("task_2", "turn_1", second),
+ { ...taskEntry("task_2", "turn_2", second), cohortId: "task_1" },
]),
taskDeliveryId: "task_2:ready:completed",
}),
@@ -147,10 +150,46 @@ describe("resolveTaskDeliveryContext", () => {
context:
'[Task state]\n{"tasks":[{"name":"report_probe","output":{"data":{"result":"first"},"type":"result"},"status":"completed","taskId":"task_1"},{"name":"report_probe","output":{"data":{"result":"second"},"type":"result"},"status":"completed","taskId":"task_2"}]}',
phase: "settled",
- rootTurnId: "turn_1",
+ rootTurnId: "turn_2",
});
});
+ it("includes failure and cancellation in the terminal cohort, but not an earlier settled report", () => {
+ const failed: TaskView = {
+ metadata,
+ taskId: "task_failed",
+ status: "failed",
+ lastOutput: { type: "error", data: "Worker failed" },
+ };
+ const cancelled: TaskView = { metadata, taskId: "task_cancelled", status: "cancelled" };
+ const previous: TaskView = {
+ metadata,
+ taskId: "task_previous",
+ status: "completed",
+ lastOutput: { type: "result", data: "Already reported" },
+ };
+ const state = taskState([
+ taskEntry("task_previous", "turn_1", previous),
+ taskEntry("task_failed", "turn_1", failed),
+ { ...taskEntry("task_cancelled", "turn_2", cancelled), cohortId: "task_failed" },
+ ]);
+ const result = resolveTaskDeliveryContext({
+ state,
+ taskDeliveryId: "task_cancelled:ready:cancelled",
+ });
+ expect(result).toMatchObject({ phase: "settled", rootTurnId: "turn_2" });
+ expect(JSON.parse(result!.context.slice(`${TASK_DELIVERY_CONTEXT_LABEL}\n`.length))).toEqual({
+ tasks: [
+ { name: metadata.name, output: failed.lastOutput, status: "failed", taskId: "task_failed" },
+ { name: metadata.name, status: "cancelled", taskId: "task_cancelled" },
+ ],
+ });
+ expect(result!.context).not.toContain("Already reported");
+ expect(result!.context).not.toContain("inbox-");
+ expect(result!.context).not.toContain("cohortId");
+ expect(result!.context).not.toContain("createdByTurnId");
+ });
+
it("returns no context when the delivery is not owned by the session task index", () => {
expect(
resolveTaskDeliveryContext({
@@ -166,7 +205,7 @@ function taskEntry(
createdByTurnId: string,
terminalView?: TaskView,
executor?: { readonly data: Record<string, never>; readonly kind: string },
-) {
+): SessionTaskIndexEntry {
return {
createdByTurnId,
executor,
diff --git a/packages/eve/src/tasks/delivery-context.ts b/packages/eve/src/tasks/delivery-context.ts
index 92f6280bc..e0f32c401 100644
--- a/packages/eve/src/tasks/delivery-context.ts
+++ b/packages/eve/src/tasks/delivery-context.ts
@@ -3,6 +3,7 @@ import { markFrameworkStepInput } from "#harness/messages.js";
import type { SessionStateMap, StepInput } from "#harness/types.js";
import { EMPTY_DELIVERY_SENTINEL } from "#shared/empty-delivery.js";
import { getSessionTaskIndex, type SessionTaskIndexEntry } from "#tasks/session-index.js";
+import { getTaskCohortId } from "#tasks/session-task-cohorts.js";
export const TASK_DELIVERY_CONTEXT_LABEL = "[Task state]";
@@ -11,7 +12,7 @@ The latest ${TASK_DELIVERY_CONTEXT_LABEL} message is runtime-authored and lists
Continue carrying out the user's request, including starting any remaining background work. When no further tool calls are needed in this turn, send one brief user-facing acknowledgement that the background work has started. Do not wait for results or report results that are not available yet. End the turn after the acknowledgement.`;
-export const TASK_DELIVERY_SETTLED_INSTRUCTION = `Background task reporting\nThis turn was triggered by background task activity. The accompanying ${TASK_DELIVERY_CONTEXT_LABEL} message is runtime-authored and lists tasks started by the same parent turn, all settled, with every available terminal output. Do not reply with ${EMPTY_DELIVERY_SENTINEL}. Send one user-facing response that combines their useful results.`;
+export const TASK_DELIVERY_SETTLED_INSTRUCTION = `Background task reporting\nThis turn was triggered by background task activity. The accompanying ${TASK_DELIVERY_CONTEXT_LABEL} message is runtime-authored and lists overlapping background tasks in the same cohort, potentially started across different user turns, all settled, with every available terminal output. Do not reply with ${EMPTY_DELIVERY_SENTINEL}. Send one user-facing response that combines their useful results.`;
type BackgroundTaskDelivery = DeliverHookPayload & {
readonly taskDeliveryId: string;
@@ -33,7 +34,7 @@ export function markBackgroundTaskStepInput(input: StepInput): StepInput {
: markFrameworkStepInput(input, "execution.background_task");
}
-/** Returns model context and cohort phase for tasks started by the same parent turn as this delivery. */
+/** Groups overlapping tasks without changing the delivered task's activity root. */
export function resolveTaskDeliveryContext(input: {
readonly state: SessionStateMap | undefined;
readonly taskDeliveryId: string;
@@ -48,7 +49,7 @@ export function resolveTaskDeliveryContext(input: {
const delivered = entries.find((entry) => input.taskDeliveryId.startsWith(`${entry.taskId}:`));
if (delivered === undefined) return undefined;
- const cohort = entries.filter((entry) => entry.createdByTurnId === delivered.createdByTurnId);
+ const cohort = entries.filter((entry) => getTaskCohortId(entry) === getTaskCohortId(delivered));
return { ...projectTaskCohort(cohort), rootTurnId: delivered.createdByTurnId };
}
diff --git a/packages/eve/src/tasks/session-index.test.ts b/packages/eve/src/tasks/session-index.test.ts
index 9d14670e3..7e2024323 100644
--- a/packages/eve/src/tasks/session-index.test.ts
+++ b/packages/eve/src/tasks/session-index.test.ts
@@ -3,11 +3,15 @@ import { describe, expect, it } from "vitest";
import type { HarnessSession } from "#harness/types.js";
import {
SESSION_TASKS_STATE_KEY,
+ cacheTerminalTaskView,
findSessionTaskEntry,
getSessionTaskIndex,
recordSessionTask,
} from "#tasks/session-index.js";
+import { getTaskCohortId, getSessionTaskCohorts } from "#tasks/session-task-cohorts.js";
import { deriveTaskId } from "#tasks/task-id.js";
+import type { TaskView } from "#tasks/types.js";
+
function createSession(state?: HarnessSession["state"]): HarnessSession {
return {
agent: {
@@ -92,6 +96,127 @@ describe("session task index", () => {
expect(entries[0]?.taskRunId).toBe("run-2");
});
+ function task(taskId: string, createdByTurnId: string) {
+ return {
+ createdByStepIndex: 0,
+ createdByTurnId,
+ metadata,
+ taskId,
+ taskInboxToken: `inbox-${taskId}`,
+ taskRunId: `run-${taskId}`,
+ };
+ }
+
+ function terminal(taskId: string, status: "completed" | "failed" | "cancelled"): TaskView {
+ if (status === "cancelled") return { metadata, status, taskId };
+ return status === "completed"
+ ? { metadata, status, taskId, lastOutput: { type: "result", data: "done" } }
+ : { metadata, status, taskId, lastOutput: { type: "error", data: "failed" } };
+ }
+
+ it("durably joins overlapping work across turns and executor kinds", () => {
+ const first = task("task_a", "turn-1");
+ const initial = recordSessionTask(createSession(), first);
+ const second = {
+ ...task("task_b", "turn-2"),
+ executor: { kind: "workflow-tool", data: {} },
+ };
+ const session = recordSessionTask(initial, second);
+ const entries = getSessionTaskIndex(session.state);
+ expect(entries.map(getTaskCohortId)).toEqual(["task_a", "task_a"]);
+ expect(entries.map((entry) => entry.createdByTurnId)).toEqual(["turn-1", "turn-2"]);
+ expect(entries[0]?.cohortId).toBeUndefined();
+ expect(entries[1]?.cohortId).toBe("task_a");
+ const restored = createSession(JSON.parse(JSON.stringify(initial.state)));
+ expect(recordSessionTask(restored, second).state).toEqual(session.state);
+ expect(getSessionTaskIndex(initial.state)).toHaveLength(1);
+ });
+
+ it.each(["completed", "failed", "cancelled"] as const)(
+ "keeps a %s sibling in a pending cohort, then starts a new cohort after settlement",
+ (status) => {
+ let session = recordSessionTask(createSession(), task("task_a", "turn-1"));
+ session = recordSessionTask(session, task("task_b", "turn-1"));
+ session = {
+ ...session,
+ state: cacheTerminalTaskView(session.state, terminal("task_a", status)),
+ };
+ session = recordSessionTask(session, task("task_c", "turn-2"));
+ expect(getSessionTaskIndex(session.state).map(getTaskCohortId)).toEqual([
+ "task_a",
+ "task_a",
+ "task_a",
+ ]);
+ expect([...getSessionTaskCohorts(session.state).values()]).toEqual([
+ { cohortId: "task_a", settled: true },
+ { cohortId: "task_a", settled: false },
+ { cohortId: "task_a", settled: false },
+ ]);
+ for (const taskId of ["task_b", "task_c"]) {
+ session = {
+ ...session,
+ state: cacheTerminalTaskView(session.state, terminal(taskId, status)),
+ };
+ }
+ // Even another creation in the same turn must not reopen a settled cohort.
+ session = recordSessionTask(session, task("task_d", "turn-2"));
+ expect(getSessionTaskIndex(session.state).map(getTaskCohortId)).toEqual([
+ "task_a",
+ "task_a",
+ "task_a",
+ "task_d",
+ ]);
+ },
+ );
+
+ it("preserves replayed membership, creation provenance, order, and settlement", () => {
+ let session = recordSessionTask(createSession(), task("task_a", "turn-1"));
+ session = recordSessionTask(session, task("task_b", "turn-2"));
+ for (const taskId of ["task_a", "task_b"]) {
+ session = {
+ ...session,
+ state: cacheTerminalTaskView(session.state, terminal(taskId, "completed")),
+ };
+ }
+ session = recordSessionTask(session, task("task_c", "turn-3"));
+ session = recordSessionTask(session, {
+ ...task("task_a", "turn-replay"),
+ createdByStepIndex: 9,
+ taskRunId: "run-replayed",
+ });
+ session = recordSessionTask(session, {
+ ...task("task_b", "turn-replay"),
+ createdByStepIndex: 9,
+ });
+ expect(
+ getSessionTaskIndex(session.state).map((entry) => ({
+ taskId: entry.taskId,
+ cohortId: getTaskCohortId(entry),
+ turnId: entry.createdByTurnId,
+ stepIndex: entry.createdByStepIndex,
+ settled: entry.terminalView !== undefined,
+ })),
+ ).toEqual([
+ { taskId: "task_a", cohortId: "task_a", turnId: "turn-1", stepIndex: 0, settled: true },
+ { taskId: "task_b", cohortId: "task_a", turnId: "turn-2", stepIndex: 0, settled: true },
+ { taskId: "task_c", cohortId: "task_c", turnId: "turn-3", stepIndex: 0, settled: false },
+ ]);
+ expect(findSessionTaskEntry(session.state, "task_a")?.taskRunId).toBe("run-replayed");
+ session = recordSessionTask(session, task("task_d", "turn-4"));
+ expect(findSessionTaskEntry(session.state, "task_d")?.cohortId).toBe("task_c");
+ });
+
+ it.each(["", null, 42])("rejects an invalid additive cohort identity: %j", (cohortId) => {
+ expect(() =>
+ getSessionTaskIndex({
+ [SESSION_TASKS_STATE_KEY]: {
+ tasks: [{ ...task("task_a", "turn-1"), cohortId }],
+ version: 2,
+ },
+ }),
+ ).toThrow(/Corrupt task index/u);
+ });
+
it("retains only terminal views as expired-run fallbacks", () => {
const base = {
taskInboxToken: "task:token-1",
diff --git a/packages/eve/src/tasks/session-index.ts b/packages/eve/src/tasks/session-index.ts
index 657121223..cbd4c8743 100644
--- a/packages/eve/src/tasks/session-index.ts
+++ b/packages/eve/src/tasks/session-index.ts
@@ -5,6 +5,7 @@ import type { JsonValue } from "#shared/json.js";
import type { TaskExecutorBinding } from "#tools/task.js";
import { sameTaskMetadata, type TaskMetadata, type TaskView } from "#tasks/types.js";
import {
+ getTaskCohortId,
SESSION_TASKS_STATE_KEY,
SESSION_TASKS_STATE_VERSION,
} from "#tasks/session-task-cohorts.js";
@@ -37,6 +38,8 @@ export interface SessionTaskIndexEntry {
readonly taskInboxToken: string;
readonly createdByStepIndex?: number;
readonly createdByTurnId: string;
+ /** Immutable join target; absent on the task that starts a cohort. */
+ readonly cohortId?: string;
readonly executor?: TaskExecutorBinding;
readonly metadata: TaskMetadata;
}
@@ -93,6 +96,7 @@ const sessionTaskIndexEntrySchema: z.ZodType<SessionTaskIndexEntry> = z.strictOb
taskInboxToken: z.string().min(1),
createdByStepIndex: z.number().int().nonnegative().optional(),
createdByTurnId: z.string().min(1),
+ cohortId: z.string().min(1).optional(),
executor: z
.strictObject({
data: z.record(z.string(), z.custom<JsonValue>()),
@@ -206,15 +210,32 @@ export function findSessionTaskEntry(
}
/**
- * Records one task, replacing any entry with the same id so replayed
- * creation for the same originating call stays idempotent.
+ * Joins the indexed cohort that still has unreported/nonterminal work. Cached
+ * terminal siblings remain members until the whole cohort settles. Membership
+ * and creation provenance survive replay, even after that cohort has settled.
*/
export function recordSessionTask(
session: HarnessSession,
- entry: SessionTaskIndexEntry,
+ entry: Omit<SessionTaskIndexEntry, "cohortId">,
): HarnessSession {
- const existing = getSessionTaskIndex(session.state);
- const tasks = [...existing.filter((candidate) => candidate.taskId !== entry.taskId), entry];
+ const tasks = [...getSessionTaskIndex(session.state)];
+ const index = tasks.findIndex((candidate) => candidate.taskId === entry.taskId);
+ const previous = tasks[index];
+ if (previous !== undefined) {
+ tasks[index] = {
+ ...entry,
+ cohortId: previous.cohortId,
+ createdByStepIndex: previous.createdByStepIndex,
+ createdByTurnId: previous.createdByTurnId,
+ terminalView: previous.terminalView ?? entry.terminalView,
+ };
+ } else {
+ const pending = tasks.find((candidate) => candidate.terminalView === undefined);
+ tasks.push({
+ ...entry,
+ cohortId: pending === undefined ? undefined : getTaskCohortId(pending),
+ });
+ }
return {
...session,
state: {
diff --git a/packages/eve/src/tasks/session-task-cohorts.test.ts b/packages/eve/src/tasks/session-task-cohorts.test.ts
index e622bce01..cce4e7779 100644
--- a/packages/eve/src/tasks/session-task-cohorts.test.ts
+++ b/packages/eve/src/tasks/session-task-cohorts.test.ts
@@ -1,14 +1,19 @@
import { describe, expect, it } from "vitest";
import { getSessionTaskIndex } from "#tasks/session-index.js";
-import { getSessionTaskCohorts, SESSION_TASKS_STATE_KEY } from "#tasks/session-task-cohorts.js";
+import {
+ getTaskCohortId,
+ getSessionTaskCohorts,
+ SESSION_TASKS_STATE_KEY,
+} from "#tasks/session-task-cohorts.js";
describe("workflow task cohort lookup", () => {
it("projects the same identities as the full task index", () => {
const state = {
[SESSION_TASKS_STATE_KEY]: {
version: 2,
- tasks: ["turn-1", "turn-1", "turn-2"].map((createdByTurnId, index) => ({
+ tasks: ["turn-1", "turn-2", "turn-2"].map((createdByTurnId, index) => ({
+ cohortId: index === 1 ? "task_0" : undefined,
taskId: `task_${index}`,
taskRunId: `run-${index}`,
taskInboxToken: `inbox-${index}`,
@@ -18,7 +23,10 @@ describe("workflow task cohort lookup", () => {
},
};
expect([...getSessionTaskCohorts(state)]).toEqual(
- getSessionTaskIndex(state).map((task) => [task.taskId, task.createdByTurnId]),
+ getSessionTaskIndex(state).map((task) => [
+ task.taskId,
+ { cohortId: getTaskCohortId(task), settled: task.terminalView !== undefined },
+ ]),
);
});
@@ -36,6 +44,10 @@ describe("workflow task cohort lookup", () => {
{ version: 2, tasks: [{ taskId: "", createdByTurnId: "turn-1" }] },
{ version: 2, tasks: [{ taskId: "task_1", createdByTurnId: "" }] },
{ version: 2, tasks: [{ taskId: "task_1", createdByTurnId: 1 }] },
+ ...["", null, 42].map((cohortId) => ({
+ version: 2,
+ tasks: [{ taskId: "task_1", createdByTurnId: "turn-1", cohortId }],
+ })),
{
version: 2,
tasks: [
diff --git a/packages/eve/src/tasks/session-task-cohorts.ts b/packages/eve/src/tasks/session-task-cohorts.ts
index 79e61aef3..ce3ac0e38 100644
--- a/packages/eve/src/tasks/session-task-cohorts.ts
+++ b/packages/eve/src/tasks/session-task-cohorts.ts
@@ -5,15 +5,23 @@ import { isNonEmptyString, isObject } from "#shared/guards.js";
export const SESSION_TASKS_STATE_KEY = "eve.tasks";
export const SESSION_TASKS_STATE_VERSION = 2;
+/** An entry without a join target starts its own cohort. */
+export function getTaskCohortId(task: {
+ readonly taskId: string;
+ readonly cohortId?: string;
+}): string {
+ return task.cohortId ?? task.taskId;
+}
+
/**
- * Reads only the identities needed for workflow-side completion batching.
+ * Reads cohort identities and settlement for workflow-side completion batching.
* Full task validation stays in getSessionTaskIndex on the step side, so
* the workflow bundle does not retain the task schemas and their dependencies.
*/
export function getSessionTaskCohorts(
state: SessionStateMap | undefined,
-): ReadonlyMap<string, string> {
- const cohorts = new Map<string, string>();
+): ReadonlyMap<string, { readonly cohortId: string; readonly settled: boolean }> {
+ const cohorts = new Map<string, { readonly cohortId: string; readonly settled: boolean }>();
const raw = state?.[SESSION_TASKS_STATE_KEY];
if (raw === undefined) return cohorts;
@@ -32,13 +40,17 @@ export function getSessionTaskCohorts(
!isObject(task) ||
!isNonEmptyString(task.taskId) ||
!isNonEmptyString(task.createdByTurnId) ||
+ (task.cohortId !== undefined && !isNonEmptyString(task.cohortId)) ||
cohorts.has(task.taskId)
) {
throw new Error(
`Corrupt task index under session state key "${SESSION_TASKS_STATE_KEY}": invalid task cohort identity.`,
);
}
- cohorts.set(task.taskId, task.createdByTurnId);
+ cohorts.set(task.taskId, {
+ cohortId: getTaskCohortId({ taskId: task.taskId, cohortId: task.cohortId }),
+ settled: task.terminalView !== undefined,
+ });
}
return cohorts;
}
diff --git a/packages/eve/src/tools/behavior.ts b/packages/eve/src/tools/behavior.ts
index dbd64ca18..e4bc11b59 100644
--- a/packages/eve/src/tools/behavior.ts
+++ b/packages/eve/src/tools/behavior.ts
@@ -8,7 +8,7 @@ export type ToolAvailabilityCondition =
/** Native behavior declared by a selected compiled tool. */
export type CompiledToolHandling =
- | { readonly kind: "dispatch"; readonly action: "self-agent" | "task-cancel" | "task-update" }
+ | { readonly kind: "dispatch"; readonly action: "self-agent" | "task-cancel" }
| { readonly kind: "provider-tool"; readonly provider: WebSearchProvider }
| { readonly kind: "request-input"; readonly request: "question" }
| { readonly kind: "workflow-tool"; readonly workflowId: string };
@@ -44,7 +44,6 @@ export type PreparedDispatchTarget =
readonly subagentName: string;
}
| { readonly kind: "task-cancel" }
- | { readonly kind: "task-update" }
| { readonly kind: "workflow-tool-call"; readonly workflowId: string };
/** Runtime-prepared handling consumed by the harness and execution boundary. */
diff --git a/packages/eve/src/tools/framework/task-contract.ts b/packages/eve/src/tools/framework/task-contract.ts
index 7090353df..d6972320e 100644
--- a/packages/eve/src/tools/framework/task-contract.ts
+++ b/packages/eve/src/tools/framework/task-contract.ts
@@ -2,13 +2,9 @@ import { z } from "#compiled/zod/index.js";
import { jsonValueSchema } from "#shared/json-schemas.js";
export const TASK_CANCEL_TOOL_NAME = "task_cancel";
-export const TASK_UPDATE_TOOL_NAME = "task_update";
/** Framework task-control tool names lowered by the runtime. */
-export const TASK_TOOL_NAMES: ReadonlySet<string> = new Set([
- TASK_CANCEL_TOOL_NAME,
- TASK_UPDATE_TOOL_NAME,
-]);
+export const TASK_TOOL_NAMES: ReadonlySet<string> = new Set([TASK_CANCEL_TOOL_NAME]);
const TASK_IDS_SCHEMA = z
.array(z.string().min(1))
@@ -17,10 +13,6 @@ const TASK_IDS_SCHEMA = z
export const TASK_CANCEL_INPUT_SCHEMA = z.strictObject({ taskIds: TASK_IDS_SCHEMA });
-export const TASK_UPDATE_INPUT_SCHEMA = z.strictObject({
- message: z.string().min(1).describe("Brief description of what this task is currently doing."),
-});
-
const taskMetadataJsonSchema = z.object({
agentId: z.string().optional(),
kind: z.string(),
diff --git a/packages/eve/src/tools/framework/task-update.ts b/packages/eve/src/tools/framework/task-update.ts
deleted file mode 100644
index 8ea8ec512..000000000
--- a/packages/eve/src/tools/framework/task-update.ts
+++ /dev/null
@@ -1,19 +0,0 @@
-import { TASK_UPDATE_INPUT_SCHEMA } from "#tools/framework/task-contract.js";
-import { defineNativeTool } from "#tools/native-definition.js";
-
-export const TASK_UPDATE_SESSION_INSTRUCTION =
- "Background task updates\nYou are running as a background task. For multi-step work, use `task_update` at meaningful milestones to briefly state what you are currently doing. Keep updates terse and activity-focused; do not include preliminary findings or results. Do not wait for a response, and return your final result normally.";
-
-export const taskUpdate = defineNativeTool(
- {
- description:
- "Briefly tell the parent agent what this background task is currently doing. Report activity, not preliminary findings or results.",
- inputSchema: TASK_UPDATE_INPUT_SCHEMA,
- },
- {
- availability: ["delegated-task-child"],
- handling: { action: "task-update", kind: "dispatch" },
- },
-);
-
-export default taskUpdate;
diff --git a/packages/eve/test/runtime-tool-registry.test.ts b/packages/eve/test/runtime-tool-registry.test.ts
index 2efa2f3b8..1eb9b86f8 100644
--- a/packages/eve/test/runtime-tool-registry.test.ts
+++ b/packages/eve/test/runtime-tool-registry.test.ts
@@ -46,24 +46,23 @@ describe("createRuntimeToolRegistry", () => {
});
});
- it("carries task_update availability into the prepared tool descriptor", async () => {
+ it("carries delegated-task-child availability into the prepared tool descriptor", async () => {
const registry = await createRuntimeToolRegistry({
tools: [
createResolvedToolDefinition({
behavior: {
availability: ["delegated-task-child"],
- handling: { action: "task-update", kind: "dispatch" },
},
- logicalPath: "tools/task_update.ts",
- name: "task_update",
- sourceId: "framework:tools/task_update.ts",
+ logicalPath: "tools/child_only.ts",
+ name: "child_only",
+ sourceId: "tools/child_only.ts",
}),
],
});
expect(registry.preparedTools[0]?.behavior).toEqual({
availability: ["delegated-task-child"],
- handling: { kind: "dispatch", target: { kind: "task-update" } },
+ handling: undefined,
presentation: undefined,
});
});