The Open Framework for Building Agents
docs(eve): channels - define steering and cancellation plans (#866)
Andrew Barba42d3166a7a78
diff --git a/research/channel-session-reset.md b/research/channel-session-reset.mddeleted file mode 100644index f124f5caf..000000000--- a/research/channel-session-reset.md+++ /dev/null@@ -1,227 +0,0 @@-----issue: https://github.com/vercel/eve/issues/216-last_updated: "2026-06-23"-status: proposed------# Channel session reset and scoped cancellation--## Summary--Telegram private chats and Twilio conversations reuse a stable continuation identity, so one durable-session can accumulate history and state forever. eve should recognize `/new` on those channels and-terminate the current session before the identity is reused.--Cancellation has two explicit scopes:--- **turn:** stop the active turn and its descendants; keep the entry session resumable;-- **session:** stop the entry session and its complete execution tree; the next message starts with- empty history and authored state.--Slash commands are one consumer of a general cancellation API. The same semantics must be available-to the eve HTTP channel, TypeScript client, custom channels, higher-level channel handlers, session-callbacks, and evals.--## Authoring API--### eve HTTP channel--Expose one authenticated route:--`POST /eve/v1/session/:sessionId/cancel`--The body is a strict union with no default scope:--```json-{ "scope": "turn", "cancelToken": "<active-turn capability>" }-```--```json-{ "scope": "session", "continuationToken": "<current session capability>" }-```--The route authenticates first, then verifies that the capability belongs to `:sessionId`. Invalid-bodies return `400`; stale or mismatched capabilities return a non-disclosing `409`; accepted-cancellation returns `202`.--Every request that starts a turn returns a fresh `cancelToken` alongside `sessionId` and the current-`continuationToken`. A cancel token is valid only for that active turn. It cannot cancel the entry-session or a later turn.--### TypeScript client--- `MessageResponse.cancel()` cancels the turn represented by that response.-- `ClientSession.cancel()` cancels the current entry session.-- Both use the client's normal auth, headers, redirects, and error handling.-- Session cancellation clears the client's resumable cursor so a later send cannot accidentally- target the cancelled entry.-- Aborting a request or stream with `AbortSignal` remains local transport cancellation; it does not- request server-side cancellation.--### Channel authors--Custom `defineChannel` route handlers receive separate operations to:--- cancel a turn using its session id and turn cancel token;-- cancel a session using its channel-local continuation token;-- restart a session with replacement input after the old session releases its identity.--The operations own token namespacing and ordering. Authors do not call workflow APIs or manually-compose “cancel, then hope delivery starts fresh.”--Higher-level channel handlers can return a `reset-session` decision with auth, optional context, and-optional replacement content. This lets a Slack mention or direct-message handler request the same-behavior without receiving low-level route operations.--In-session callbacks and event subscribers use an explicit cancellation operation with a required-`turn` or `session` scope. Requesting cancellation exits normal authored execution; code after the-request must not continue.--### Evals--Evals expose first-class controls for cancelling an active turn and its entry session. An eval can-retain a cancellable handle while work is active, request cancellation, and continue observing the-resulting event boundary. Expected cancellation is assertable behavior, not an automatic eval-failure.--Eval cancellation uses the TypeScript client and public channel APIs. It must work for sessions-created by custom channels as well as the built-in eve channel.--## Semantics--### Turn cancellation--The cancel token is minted per turn and bound to `(sessionId, turnId)`.--```text-TURN START--ClientSession.send()-`-- POST message- `-- eve channel / runtime- |-- resume entry session S1 through continuation C1- |-- start turn T7- |-- bind cancel token K7 -> (S1, T7)- `-- return { sessionId: S1, continuationToken: C1, cancelToken: K7 }- `-- MessageResponse stores K7--TURN CANCEL--MessageResponse.cancel()-`-- POST /eve/v1/session/S1/cancel- `-- { scope: "turn", cancelToken: K7 }- `-- eve channel / runtime- |-- authenticate the request- |-- resolve K7 -> (S1, T7)- |-- verify the URL session is S1- |-- durably accept cancellation and return 202- |-- cancel T7 and its descendants- |-- retire K7 when T7 settles- `-- keep C1 -> S1 and emit session.waiting-```--When T7 completes, fails, or is cancelled, K7 becomes stale. The next turn receives a new token K8.-K7 can never cancel K8 or session S1.--### Session cancellation--Session cancellation binds the current continuation token to the session id in the request. The-runtime releases the continuation before tearing down the tree; that release is the reset-linearization point.--```text-SESSION CANCEL--ClientSession.cancel() or authenticated /new handler-`-- request session cancellation for (S1, C1)- `-- channel / runtime- |-- verify C1 currently belongs to S1- |-- mark S1 as closing- |-- release C1 from S1 [reset linearization point]- |-- acknowledge accepted cancellation- |-- cancel the complete S1 tree- | `-- active turn- | |-- model and tools- | `-- local and remote delegates- |-- wait for every branch to settle- `-- emit session.cancelled and close S1--IDENTITY REUSE--Stable Telegram / Twilio identity R-`-- channel continuation C1- |-- before reset: C1 -> S1- |-- after release: C1 -> no active session- `-- next message: C1 -> new session S2 with empty history and state--Old cleanup remains bound to S1.-`-- a stale request naming S1 cannot cancel S2, even though S2 reuses C1-```--For bare `/new`, the flow ends after S1 is cancelled. For `/new <message>` or `/new` with an-attachment, restart waits for the release barrier and creates S2 with that replacement content.--### Runtime guarantees--- Cancellation follows ownership from entry session → turn → tools and delegated agents, including- remote descendants.-- A cancelling session cannot accept input, launch work, or reclaim its continuation.-- Cooperative abort reaches models, tools, sandboxes, and delegates; the runtime remains responsible- for terminating work that does not cooperate.-- Late descendant results cannot resume a cancelled ancestor or mutate a replacement session.-- Turn cancellation ends at `session.waiting`; session cancellation ends at `session.cancelled`.-- Cancellation is intentional control flow and does not trigger generic user-facing failure output.-- Completed external side effects remain recorded and are not rolled back.--## `/new` behavior--Telegram and Twilio expose:--```ts-resetCommands?: false | readonly string[];-```--- omitted: `['new']`;-- `false`: disabled;-- array: replace the default names.--Inbound handling verifies the provider request and runs authored gating/auth before matching the-command. A `null` authored result drops the input without cancellation.--- Bare `/new` silently cancels the session and creates no empty workflow.-- `/new <message>` starts a fresh session with the stripped message.-- Telegram attachments on `/new` belong to the fresh session.-- `/new@botname` matches only the configured Telegram bot.-- Prefixes such as `/newspaper`, unknown commands, and disabled commands remain normal model input.-- The command itself is never added to history and successful reset sends no confirmation.-- Slack has no default reset command; authors opt in through their inbound handler.--## Delivery and verification--Implementation must cover the HTTP contract, cancellation outcomes and events, ownership-tree-propagation, TypeScript client, eval driver, custom-channel operations, Telegram and Twilio `/new`,-docs, and a patch changeset.--Add a new `e2e/fixtures/agent-cancellation` fixture. Its custom `defineChannel` must use only public-channel APIs—no runtime/workflow imports and no proxy through the built-in eve cancellation route.-The fixture includes independent evals for:--- **turn cancellation:** cancel active work, observe the turn boundary and `session.waiting`, then- prove a follow-up resumes the same session and context;-- **session cancellation:** cancel active work, observe `session.cancelled`, reuse the same channel- identity, then prove the new session id, history, and authored state are fresh.--The evals use the new eval cancellation controls, assert events and identities directly, work-locally and against deployed targets, and require no external service beyond model credentials.--Also cover malformed/scoped HTTP requests, auth before capability inspection, stale tokens, command-parsing, attachments, duplicate cancellation, delivery/completion races, nested local and remote-delegates, client cursor updates, and the distinction between `AbortSignal` and server cancellation.--Run the repository's required unit, integration, scenario, typecheck, lint, format, invariant, docs,-and build checks, plus:--```sh-cd e2e/fixtures/agent-cancellation-pnpm exec eve eval --strict-```diff --git a/research/channel-turn-steering.md b/research/channel-turn-steering.mdnew file mode 100644index 000000000..072e4a2f2--- /dev/null+++ b/research/channel-turn-steering.md@@ -0,0 +1,422 @@+---+issue: https://github.com/vercel/eve/issues/867+status: proposed+last_updated: "2026-07-16"+---++# Channel turn steering++## Summary++Chat-style channels often receive corrective or additional input while an+agent is still working. Waiting for the next turn can let the agent finish an+obsolete answer; hard-cancelling the turn throws away useful work and provides+no replacement input.++Expose one input policy to channel authors:++```ts+turnPolicy: "queue" | "steer";+```++`"queue"` is the existing default: hold input for the next turn. `"steer"`+asks eve to apply the input to the active logical turn at its next safe step+boundary. There is no public `"interrupt"` policy. Whether eve can stop an+obsolete provider or tool operation early is an implementation detail of+steering, not a distinct end-user intent.++Turn cancellation remains separate: it stops active work without replacement+input and emits `turn.cancelled`. Steering keeps the same turn alive and does+not emit a cancellation boundary.++## Public authoring API++### Custom channels++```ts+export type TurnPolicy = "queue" | "steer";++export interface SendOptions<TState> {+ // Existing fields omitted.+ turnPolicy?: TurnPolicy;+}+```++`TurnPolicy` is exported from `eve/channels`. `SendOptions.turnPolicy` is+optional and defaults to `"queue"`.++```ts+export default defineChannel({+ routes: [+ POST("/threads/:threadId/messages", async (request, { params, send }) => {+ const body = await request.json();+ const session = await send(body.text, {+ auth: await authenticate(request),+ continuationToken: params.threadId,+ turnPolicy: "steer",+ });++ return Response.json({ sessionId: session.id });+ }),+ ],+});+```++Use the default queue policy when every response should finish in order:++```ts+await send(message, {+ auth,+ continuationToken: threadId,+});+```++Use cancellation for a stop button with no replacement message:++```ts+await cancel({ continuationToken: threadId, turnId });+```++The continuation-addressed cancellation helper is specified separately in+[Turn cancellation for custom channel routes](./turn-cancellation-channel-ops.md).++### Other channel surfaces++- `ChatSdkSendOptions` exposes the same optional `turnPolicy` and passes it+ unchanged through `bridge.send()`.+- Low-level `Agent.deliver()` and runtime `DeliverInput` expose the same policy+ so authored filesystem routes do not need to recreate `send()`.+- Built-in channels continue to use `"queue"` unless their documented inbound+ behavior explicitly opts into steering. Adding the API must not silently+ change Slack, Teams, Telegram, Twilio, Discord, GitHub, or Linear behavior.+- `send()` continues to return `Session`. A successful return means the input+ was durably admitted, not that a particular steering step has already run.+ The event stream is the observable record of whether it joined the active or+ next turn.++No new public steering event or result union is required.++## Observable semantics++### State matrix++| Session state | `queue` | `steer` |+| -------------------------------------------- | --------------------------------- | -------------------------------------------------------------------------- |+| No session owns the continuation token | Start a new turn. | Start a new turn; there is nothing to steer yet. |+| Session is parked and waiting for input | Start the next turn. | Start the next turn; it becomes ordinary input. |+| A steer-capable conversation turn is active | Preserve input for the next turn. | Apply input to the active turn at its next safe boundary. |+| Active pinned turn lacks steering capability | Preserve input for the next turn. | Degrade safely to queue; never drop, fail open, or start a second session. |+| Active turn settles before steering commits | Preserve input for the next turn. | Preserve it in original admission order as queue input for the next turn. |++This feature initially targets resumable conversation sessions. An active+task-mode session cannot promise a next-turn fallback, so `"steer"` must be+rejected with a typed unsupported-policy error until task successors have a+durable design. `send()` must propagate that error and must not interpret it as+permission to start a second session.++### Turn identity and events++Successfully applied steering remains inside the active turn:++- the turn retains its existing `turnId`;+- no second `turn.started` event is emitted;+- the replacement input emits `message.received` under that turn;+- later step, message, tool, and terminal events continue the normal sequence;+- the turn eventually completes, fails, cancels, or parks exactly once.++If the input falls back to the queue, it is observed under the next turn and+that turn emits its own `turn.started` normally.++### Safe boundary++A steering request does not rewrite an executing atomic step. eve applies it+after the current model/tool boundary has settled or has been cooperatively+stopped safely. The implementation may use a soft abort notification to stop+obsolete provider work sooner, but authors cannot depend on whether a specific+provider or tool call finishes or aborts.++The public guarantee is:++1. completed side effects are never rolled back;+2. a steering delivery is applied before the active turn is allowed to settle,+ or it remains queued for a later turn;+3. it is never applied twice.++### Ordering++Every admitted delivery receives a durable sequence. Queue fallback preserves+that admission order.++Steering intentionally bypasses older queued input while an active turn can+accept it. For example, queued input A may remain for the next turn while a+later steering input B joins the current turn. If B cannot be applied and also+falls back to the queue, the next turn observes A before B.++Multiple steering deliveries are forwarded single-flight and applied in their+admission order. eve may coalesce adjacent steering deliveries at one safe+boundary as long as payload order and exactly-once behavior are preserved.+Internal acknowledgement timing must never change their fallback order.++### Delivery and continuation races++- Once `send()` resolves a continuation hook to a session, a concurrent+ `setContinuationToken()` cannot make that delivery fall through to+ `runtime.run()` or create another history.+- `send()` may start a session only after receiving the typed+ no-active-session outcome. Hook conflicts, steering failures, unsupported+ policy, serialization errors, and other runtime failures propagate.+- A terminal race has two valid outcomes: input is consumed by the active turn+ or remains queued. There is no gap in which neither side owns it.+- Retry and replay are idempotent by delivery id. Duplicate hook or step+ execution cannot duplicate model input.++### Input requests and descendants++Structured `inputResponses` retain their addressing semantics:++- a response naming an existing request routes to that request's owner;+- input queued before a later request cannot accidentally answer it;+- steering received while a request is open does not close the request unless+ it contains the matching response;+- a delivery containing both addressed responses and parent-local content+ routes the addressed parts first, then applies `turnPolicy` to the remainder;+- responses for delegated children continue through existing child routing;+ any parent-local remainder retains the source delivery id and sequence.++Steering a parent does not cancel delegated children. Existing child results,+authorization state, and runtime-action reads remain pending. Replacement+input may resolve a child request or continue the parent according to the same+routing rules used for ordinary delivery.++### Adapter projection++`turnPolicy` is runtime routing metadata and is not part of the authored+adapter payload. The adapter still projects the delivery into a harness+`StepInput`.++Explicit steering cannot be silently swallowed because an adapter returns+`undefined`: eve falls back to the standard message/input-response projection+for that delivery. Structurally empty steering is normalized consistently with+new-session input. The harness marks the projected replacement input with its+existing internal `steering: true` marker so deferred input, compaction, and+history logic can distinguish it from an ordinary continuation.++Any future adapter contract that distinguishes “no custom projection” from+“handled without model input” should be designed separately. It is not an+additional `turnPolicy` value.++### Cancellation and side effects++Steering and cancellation are independent operations:++| Operation | Replacement input | Turn identity | Terminal boundary |+| --------------------- | ----------------- | ------------- | --------------------------------------- |+| `turnPolicy: "queue"` | Next turn | New turn | Current turn settles normally |+| `turnPolicy: "steer"` | Active turn | Same turn | No cancellation event |+| `cancel(...)` | None | Ends turn | `turn.cancelled` then `session.waiting` |++If cancellation wins before a steering delivery is consumed, that delivery+remains queued for the next conversation turn. If the turn consumes the input+first and cancellation arrives afterward, the input belongs to the cancelled+turn and is not replayed automatically. Durable command ordering, not process+timing, determines which outcome occurred.++Partial model output already emitted remains on the stream. Completed tool,+channel, sandbox, and external effects remain completed. Steering and+cancellation are not rollback mechanisms.++## Clean-slate ownership model++The session driver must be the sole durable owner of admitted input. The active+turn receives a lease; ownership does not move merely because a private inbox+received the payload.++```text+public continuation hook+ │+ ▼+session driver admission ledger+ id + sequence + payload + requested policy+ │+ ├── queue ─────────────────────────────► next turn+ │+ └── lease steering delivery by id+ │+ ▼+ per-turn command inbox+ │+ ▼+ active turn+ │+ └── consumed | released | remainder+ │+ ▼+ update the original ledger entry+```++The ledger preserves position while an entry is leased:++- `consumed` removes the entry after the turn durably incorporates it into a+ step or routes it to the addressed child;+- `released` leaves it in place and changes effective policy to `queue`;+- `remainder` replaces the entry payload while retaining its id and sequence;+- terminal settlement releases every unresolved lease before selecting the+ next turn.++There are no “return to front/back” operations and no reconstruction with+`push()` or `unshift()`. The session driver is the only component that changes+effective policy from steering to queue.++The turn may expose one neutral, per-turn command inbox for input-bearing+commands such as steering and requested delivery. This does not mean steering+reuses the cancellation hook: hard cancellation and soft steering retain+different controllers, reasons, settlement behavior, and payload types. Share+low-level hook mechanics only where their replay and disposal rules are truly+identical.++### Step notification state++The active step needs to distinguish steering that was already pending when+the step began from steering that arrived while it was running:++```ts+interface TurnSteeringState {+ readonly pending: boolean;+ readonly signal?: AbortSignal;+}++interface TurnStepInput {+ readonly steeringState?: TurnSteeringState;+}+```++`pending` prevents settlement before already-admitted replacement input is+applied. `signal.aborted` wakes settlement checks when input arrives during the+step. Keep `StepInput.steering?: true` as the separate harness marker meaning+“this concrete input is replacement input.” Naming the control object+`steeringState` avoids giving two adjacent layers incompatible fields named+`steering`.++Workflow serializes nested abort signals across step boundaries, but eve must+retain a focused live-signal integration test so an upstream serializer change+cannot silently turn the signal into a plain object.++### Single-flight and terminal authority++- At most one input-bearing lease is outstanding to a turn at a time. The+ driver may continue admitting queue entries but does not forward another+ command until the current lease has a disposition.+- Receipt is not consumption. The turn reports `consumed` only at the durable+ boundary where the input is incorporated or routed.+- A stale disposition id cannot clear a current lease.+- The turn owns its normal result; the driver owns unresolved ledger entries.+ A terminal result therefore carries dispositions or remainders, never an+ untyped array to splice back into a buffer.+- Turn teardown disposes pending reads without awaiting iterator shutdown that+ can remain suspended forever.++### Durable versions and rollout++Separate the durable driver-to-turn seed from runtime-only step input. Abort+signals and `steeringState` are constructed inside the pinned turn workflow;+they do not belong in the durable migration type unless they actually cross+the driver-to-turn wire.++Any change to the durable turn seed or control protocol must update its version+and migration tests. Rollout uses additive capability negotiation:++- a latest driver can identify a steer-capable turn before leasing steering;+- a pinned driver or turn that omits the capability safely treats steering as+ queued input;+- no deployment combination interprets a new control payload as authored user+ input.++## Implementation sequence++Land this through independently reviewable boundaries.++1. **Workflow spike:** prove one per-turn input command can arrive while+ `turnStep` is active, drive the soft notification signal live, replay once,+ and settle a terminal race without a late disposition.+2. **Admission ledger:** replace raw driver buffering with id/sequence entries+ and lease/disposition operations while preserving existing queue and HITL+ behavior. No public steering API yet.+3. **Internal steering:** add the private command, `steeringState`, adapter+ projection rule, capability negotiation, and same-turn continuation tests.+ Existing cancellation remains unchanged.+4. **Public API:** expose `TurnPolicy`, `SendOptions.turnPolicy`, low-level+ delivery plumbing, Chat SDK propagation, custom-channel docs, and a patch+ changeset.+5. **E2E:** exercise the feature from a custom channel using only public APIs,+ then enable steering selectively in built-in channels only through separate+ product decisions.++Each implementation PR must be independently reviewable and leave queue-only+behavior complete.++## Verification++### Public behavior++- default queue behavior is unchanged;+- steering keeps one `turnId`, one `turn.started`, and emits replacement+ `message.received` before settlement;+- no-session, parked-session, pinned-old-turn, terminal-race, and task-mode+ outcomes match the state matrix;+- cancellation remains a hard stop with no replacement input;+- built-in channels do not opt in accidentally.++### Ownership and ordering++- queue A followed by steering B yields A then B if B falls back, regardless+ of whether the turn received B before terminating;+- multiple steering deliveries apply or fall back exactly once in admission+ order;+- stale and duplicate dispositions cannot clear or duplicate the active lease;+- continuation rekey, hook disposal, retry, and replay cannot create a second+ session or lose an admitted delivery;+- partial child routing preserves the source entry id and sequence.++### Step and harness integration++- nested live steering signal crosses the Workflow step boundary;+- already-pending state prevents park or completion without manufacturing a+ pre-aborted signal;+- adapter `undefined` falls back to the original steering input;+- steering during model, tool, authorization, input-request, runtime-action,+ and delegated-child waits preserves their pending state;+- structurally empty input follows new-session normalization;+- no late steering acknowledgement or disposition is emitted after terminal+ teardown.++### Repository checks++Run focused unit and integration tests for the admission ledger, turn command+receiver, workflow step, turn workflow, tool loop, HITL, cancellation, and+descendant routing. Public exposure also requires typecheck, lint, invariant+guard, docs validation, the full unit tier, and a deterministic hosted e2e+fixture.++## Out of scope++- Whole-session cancellation or `/new`.+- Retraction of streamed output or completed side effects.+- A public `interrupt`, `replace`, `supersede`, or priority enum.+- Multiple concurrent leases to one turn.+- Automatic debounce or latest-message policy in built-in channels.+- Task-mode successor sessions.+- An adapter handled/suppressed result protocol beyond the steering fallback+ rule above.++## Success criteria++- Channel authors choose only between standard `queue` and `steer` input+ policies and use `cancel()` for a stop action.+- Every successfully admitted delivery is either durably consumed by the+ active turn or retained by the driver for a later turn, never both and never+ neither.+- Internal acknowledgement timing cannot affect ordering.+- Steering does not require cancellation-hook semantics or expose another+ cancellation name.+- Public behavior is documented independently of Workflow hook layout, and+ queue-only behavior remains correct after every landing stage.diff --git a/research/dev-revision-isolation.md b/research/dev-revision-isolation.mddeleted file mode 100644index 75ff48a71..000000000--- a/research/dev-revision-isolation.md+++ /dev/null@@ -1,179 +0,0 @@-----issue: TBD-status: in-review-last_updated: "2026-07-13"------# Dev generation isolation--## Summary--eve already reloads most authored edits during local development. Changing a tool body,-instructions, or a connection usually works without restarting the server. The remaining problems-appear when an edit removes something, changes server structure, fails partway through, or overlaps-with work that is already running.--One rebuild currently updates several independent pieces of state: compiled artifacts, copied-runtime source, Nitro build inputs, routes, the running worker, and Workflow data. Those pieces do-not share one commit point. A rebuild can therefore publish some new state before another step-fails, leaving the server with a mixture of old and new behavior.--The goal is to prepare one complete immutable generation of an agent privately and expose it only-after everything needed to run it is ready. New turns use the latest successfully promoted-generation. A turn already in progress stays on the generation selected when its child Workflow-started, including queue retries and resumptions, but a parked session is not pinned to historical-authored code.--## Problems being solved--### Builds share mutable application paths--Production builds have historically written compiler, host, Nitro, Workflow, and output artifacts-into application-owned directories also used by other builds or `eve dev`. Concurrent builds can-consume each other's intermediate files, and a failed build can disturb a healthy dev server or-replace part of the last successful output.--Each production build needs private working directories. Only the final, completed output should be-published to the application, and that short publication step must preserve the last-good output if-it fails.--### Dev reloads can expose partial state--Ordinary runtime edits generally work, but removals reveal stale dependencies retained by the-long-lived Nitro host. Structural changes such as channel routes or instrumentation also require a-worker rebuild. Today the runtime pointer, host inputs, routes, and worker lifecycle are not one-transaction, so a late failure can leave earlier mutations active.--A runtime-only edit should publish a complete generation without replacing the worker. A structural-edit should keep serving the old worker until a replacement has built and reported ready. Failure-at any point should leave the old pointer, routes, worker, and watcher state unchanged.--### A generation is not yet a complete executable unit--A copied runtime snapshot can still depend on authored files or packages outside the snapshot. If-those originals change or disappear, an admitted request or active turn may no longer be able to-load the behavior it selected. Dependency resolution must also follow the same Node ESM rules used-when the code executes.--Each generation therefore needs the complete authored module and dependency closure required to run-that version, including workspace resources, instrumentation, configured externals, and transitive-packages. Framework and deployment runtime packages remain outside the authored generation.--### Worker replacement can interrupt live traffic--The current dev listener proxies to a reloadable Nitro worker. Replacing that worker can reset an-admitted HTTP request, interrupt a stream, or briefly make a dev control or Workflow queue endpoint-unavailable. It also makes cancellation and shutdown ownership unclear.--A stable parent server should own the listener, local Workflow World, queue ingress, and dev control-endpoints. It admits each request or queue delivery to one worker and generation, keeps that-ownership until the operation finishes or disconnects, and does not retire the old worker while-admitted work still depends on it.--### Parked sessions and active turns need different lifetimes--Production resolves `latest` when a Workflow starts and then records the selected deployment on-that run. The local World instead exposes one package-version deployment identity and currently-registers queue handlers from the reloadable worker. It cannot reliably route separate child-Workflows to the generations they selected.--The long-lived session driver should remain generation-neutral. Each child turn Workflow resolves-the latest promoted generation once, and all of that run's workflow, step, retry, and resume-deliveries return to the recorded generation. When the child Workflow becomes terminal, it releases-that reference; the next turn in the same session resolves latest again.--## Goals--- Preserve the fast path that already works: tools, connections, skills, instructions, and similar- runtime behavior reload without replacing the Nitro worker.-- Make an authored generation complete and immutable so one request or active child Workflow never- observes a mixture of versions.-- Keep parked sessions on the latest successful authored behavior without changing an active turn- during replay, retry, or suspension.-- Keep the last working server available while a structural candidate is prepared and discard the- candidate cleanly if preparation fails.-- Give requests, queue deliveries, workers, and active child Workflows explicit generation- ownership so shutdown and pruning decisions are safe.-- Keep parent-owned dev and Workflow endpoints available through reloads and prevent planned worker- replacement from surfacing connection resets.-- Preserve Nitro's selected route identity, including overlapping static and parameter routes.--## Ownership model--```text-authored edit- │- ▼-immutable generation ── runtime-only ─────────► publish latest pointer- │- └── structural ─► ready Nitro candidate ─► atomically swap pointer, routes, and worker--stable parent server- ├── owns the listener, local World, queue ingress, and dev control endpoints- ├── leases one worker and generation per admitted request or queue delivery- └── retires old workers and generations only after their references are released--generation-neutral session driver- └── starts child turn with latest ─► generation G for that child Workflow only-```--A generation contains the complete executable authored dependency closure. Nitro host inputs-retained for the server lifetime live outside the prunable generation store. The stable parent does-not reinterpret which authored channel matched a request; it preserves the route Nitro selected and-dispatches that route against the leased generation.--The parent starts exactly one local World. Workers use an eve-owned private World transport rather-than starting competing Worlds or replacing a global direct queue handler. The parent records the-resolved generation on each child Workflow and routes later deliveries by that structured run-record. Public requests cannot select or spoof the generation.--## Delivery--1. **Production build isolation — stop builds from sharing work in progress.** Give each build- private compiler, host, Nitro, Workflow, and output workspaces, then serialize only final- publication. This prevents concurrent or failed builds from disturbing each other, the running- dev server, or the last-good output. This is the current PR.-2. **Immutable generations — make one version of an agent runnable on its own.** Materialize the- complete authored behavior and dependency closure into a path-independent generation. This keeps- admitted requests and active turns executable after source files, tools, or packages are changed- or removed.-3. **Parent worker and World transport — separate stable ownership from reloadable workers.** Put- readiness, request and generation leases, queue delivery, cancellation, trusted client metadata,- and bounded shutdown behind a parent-owned listener. This prevents planned worker replacement- from resetting admitted traffic or leaving the local World bound to a retired worker.-4. **Transactional dev rebuilds — make a reload one complete promotion.** Connect the watcher,- generation pointer, routes, and candidate worker through one coordinator with complete rollback.- This preserves fast runtime-only reloads while preventing failed structural changes from leaving- mixed routes, pointers, handlers, fingerprints, or workers active.-5. **Latest-turn Workflow delivery and pruning — select latest at each child Workflow boundary.**- Keep the parked session driver generation-neutral, record one generation for each active child- Workflow, route its retries and resumptions consistently, and prune from structured active-run- references once the child becomes terminal.--Each stage remains independently valid and does not expose a partially connected lifecycle.--## User-visible result--- Concurrent production builds cannot interfere with each other or a running dev server.-- Adding, editing, or removing an authored tool continues to work without restarting `eve dev`.-- A failed compile, bundle, worker start, route change, or pointer publication keeps the previous- server fully usable.-- Structural reloads move new traffic only after the replacement worker is ready. Existing requests- and streams finish on their original worker; disconnect and shutdown cancellation follow an- explicit owned lifecycle.-- Parent-owned dev and Workflow queue endpoints remain available during reload.-- Channel changes preserve the route Nitro selected, including overlapping routes.-- Advancing and pruning a runtime generation cannot remove Nitro inputs retained by the dev server.-- An active child Workflow completes on its selected generation after a promotion. Once the session- parks, its next turn uses the newly promoted generation.--## Non-goals--- Replacing Nitro as eve's development or production bundler.-- Restarting the worker for every authored edit.-- Pinning a parked session to the authored generation that originally created it.-- Switching an active child Workflow to newer code during retry, replay, or suspension.-- Keeping every historical generation after no request, worker, candidate, or active child Workflow- can reference it.--The gray-matter vendoring change is independent of this lifecycle work.diff --git a/research/dynamic-model.md b/research/dynamic-model.mddeleted file mode 100644index 06932e824..000000000--- a/research/dynamic-model.md+++ /dev/null@@ -1,150 +0,0 @@-----issue: https://github.com/vercel/eve/issues/577-status: in-review-last_updated: "2026-07-07"------# Dynamic model resolution--## Summary--Tools, skills, and instructions can all be resolved dynamically per session or turn through-`defineDynamic`, with session, channel, and conversation context. The agent's model cannot: the-`model` field on `defineAgent` is evaluated once at compile time and frozen into the manifest. That-blocks routing cheap channels to a smaller model, picking a model per tenant from session auth,-escalating to a larger-context model as a conversation grows, and per-session A/B tests.--This plan extends the existing `model` field to accept the same `defineDynamic` sentinel used-everywhere else, with one addition: a required `fallback` model. The fallback is the compiled-static model, which keeps every compile-time surface (routing classification, gateway credential-checks, catalog context-window lookup, agent-info) working unchanged. The runtime seam already-exists — the harness resolves `session.agent.modelReference` immediately before every model call-(`harness/tool-loop.ts`) — so dynamic selection slots into the established dispatch → durable-context → tool-loop pipeline that dynamic instructions use today.--Implementation: vercel/eve#581.--## Authoring API--`model` becomes a union: a static value (unchanged) or `defineDynamic({ fallback, events })`.--```ts-// agent/agent.ts-import { defineAgent, defineDynamic } from "eve";--export default defineAgent({- model: defineDynamic({- fallback: "anthropic/claude-sonnet-5",- events: {- "session.started": (event, ctx) => {- if (ctx.channel.kind === "slack") return "anthropic/claude-haiku-4.5";- if (isEnterprise(ctx.session.auth)) return "anthropic/claude-opus-4.8";- return null; // use the fallback- },- },- }),-});-```--- `fallback` (required) is the build-time model: compiled metadata, compaction defaults, and the- active model whenever no scoped selection is set.-- Handlers receive the standard `(event, ctx)` pair with the shared `DynamicResolveContext`- (`session.id`, `session.auth`, `channel`, `messages`).-- Handlers return one of:- - a gateway model id string (`"anthropic/claude-haiku-4.5"`);- - a live AI SDK `LanguageModel` instance (`step.started` scope only);- - a selection object `{ model, modelContextWindowTokens?, modelOptions? }`, mirroring the- agent-level field names, for selections that need their own metadata;- - `null` — leave this scope's slot unset.-- `defineDynamic` and the `DynamicResolveContext` type are exported from the root `eve`- entrypoint, next to `defineAgent`.-- `compaction.model` stays static-only and rejects the sentinel with a dedicated error. When the- primary model is dynamic, compaction falls back to the active selection exactly as it falls back- to the static model today.--## Semantics--- **Events and scopes.** `session.started`, `turn.started`, and `step.started`. Precedence: step >- turn > session > `fallback`; `null` leaves a scope unset so resolution falls through to the next- scope. `session.started` is the documented default — model identity is almost always a function- of session-stable facts, and **every model switch invalidates the provider prompt cache** (caches- are per model). The docs carry an explicit warning that turn/step switching re-ingests the whole- conversation at uncached prices on every flip.-- **Serialization boundary.** Session- and turn-scoped selections are stored durably (context keys- `Session/TurnDynamicModelReferenceKey`) and must be serializable — a resolver returning a live- provider instance from those scopes logs an error and leaves the scope unset. Live instances are- step-scoped only: held in virtual (non-serialized) context and re-resolved every step, the same- pattern as live step-scoped dynamic tools.-- **Degrade to fallback, never fail the turn.** A resolver that throws or returns an invalid- selection logs an error and clears its scope; the turn proceeds on the next scope or `fallback`.- This matches the dynamic tools/skills/instructions failure posture and is documented — a broken- resolver silently serves the fallback, so resolvers backed by external services need monitoring.-- **Context window.** `modelContextWindowTokens` on a selection sets the compaction trigger for- that scope. It is **never inherited from the fallback** — a different model's window is not a- safe guess. A selection without it has no known window and the compaction threshold keeps using- the last known window. The threshold rescales against the reference it was computed from (the- session's current model reference), so repeated steps on one selection are idempotent, and- clearing a selection restores the fallback-derived threshold.-- **Mocks keep precedence.** String selections are stored reference-only and resolve through- `resolveRuntimeModelReference`, where the bootstrap and eval mock adapters short-circuit first.- In mock mode (`NODE_ENV=test`, `EVE_MOCK_AUTHORED_MODELS=1`), live step-scoped instances are- also stripped to reference-only so `eve eval` mock runs never hit real providers.-- **Static form unchanged.** `model: "anthropic/claude-sonnet-5"` and `model: anthropic(...)`- compile, route, and resolve exactly as before. The TUI `/model` command rewrites only plain- string literals; on a `defineDynamic` model it bails to the manual-edit path (it never rewrites- the `fallback` literal).-- **Observability.** The `session.started` runtime identity reports a dynamic agent's model as- `dynamic:<fallback id>` — per-scope selections happen after that identity is built. Build-time- validation (routing, credentials, catalog) applies to the fallback only; a resolver can select a- model the deployment has no credentials for, which fails at request time.--## Data flow--```text-compile agent.ts model = defineDynamic({ fallback, events })- ├─ manifest config.model = compiled fallback (id, routing, window — unchanged path)- └─ manifest config.dynamicModel = { eventNames, source ref to agent.ts }--boot resolve-agent- └─ dynamicModel source ref → RuntimeTurnAgent.dynamicModel--run workflow-steps handleEvent (session.started / turn.started)- └─ dispatchDynamicModelEvent → re-import agent.ts, run handler- ├─ string / selection object → durable Session/TurnDynamicModelReferenceKey- └─ live instance → error at these scopes (scope cleared)--step tool-loop (before compaction and the model call)- ├─ dispatch synthetic step.started → virtual LiveStepDynamicModelSelectionKey- └─ getActiveDynamicModelSelection: step > turn > session > fallback- ├─ live instance → used directly (stripped in mock mode)- └─ reference → resolveModel (bootstrap/mock adapters first)- └─ session.agent.modelReference + compaction threshold updated per step-```--The updated session reference flows to the other consumers unchanged: web-search backend-selection, compaction model fallback, per-step `providerOptions`, and gateway attribution headers-derive from the active reference or the resolved model object.--## Out of scope--- Dynamic `compaction.model`.-- A runtime model catalog (context windows for dynamic selections come from the author).-- Per-selection routing/credential validation at build time.-- Remote subagents; local subagent nodes compile their own `agent.ts` and get dynamic models- through the same path.--## Delivery and verification--Shipped in one PR (vercel/eve#581) with a patch changeset: shared types + normalization, compiler +-manifest (`config.dynamicModel`, schema v33), runtime resolver reattachment, dispatch lifecycle-(`context/dynamic-model-lifecycle.ts`), harness consumption, docs-(`docs/agent-config.md`, `docs/guides/dynamic-capabilities.md`), and tests:--- unit: lifecycle (scope pinning, precedence, null fallback, live-instance rejection at- session/turn, mock-mode stripping, throwing resolvers, unknown-key selections), normalization- (missing `fallback`, dynamic `compaction.model` rejection), compile (fallback compiled as the- static model, resolver source preserved), harness (selection reaches the model call, compaction- threshold stable across steps of one selection and restored when cleared), `/model` rewrite- bails on `defineDynamic`.-- e2e: `e2e/fixtures/agent-model` evals — fallback turn with `dynamic:` runtime identity, per-turn- selection plus null fallback in one session, and throwing-resolver degradation.diff --git a/research/turn-cancellation-channel-ops.md b/research/turn-cancellation-channel-ops.mdindex 9495e82ae..76e655489 100644--- a/research/turn-cancellation-channel-ops.md+++ b/research/turn-cancellation-channel-ops.md@@ -1,37 +1,30 @@---issue: https://github.com/vercel/eve/issues/483status: proposed-last_updated: "2026-07-15"+last_updated: "2026-07-16"----# Turn cancellation for custom channels+# Turn cancellation for custom channel routes## Summary-Chat-style webhooks often receive several short messages while an agent is-still responding. Plain `send()` queues those messages, so the agent can emit-an obsolete answer before it processes the newer context. Custom channels need-two operations:+eve already supports cooperative turn cancellation through the runtime, the+authenticated eve HTTP route, `ClientSession.cancel()`, eval controls, and+recursive cancellation of active descendants. The remaining custom-channel+gap is addressing: a `defineChannel` route normally knows its channel-local+continuation token, not the runtime session id required by `Agent.cancelTurn`.-- `send(payload, { supersede: "turn" })` queues the payload and cancels only- the turn that was already active;-- `cancelTurn({ continuationToken, turnId? })` supports explicit stop commands.--The layer-2 runtime operation cancels by session id. Channel operations also-need a race-safe continuation-token lookup and delivery primitive. They cannot-be composed safely from today's independent hook resumes.+Add one `cancel` route helper that resolves the token without starting a+session and then delegates to the existing cancellation primitive. Steering+replacement input is separate and is specified in+[Channel turn steering](./channel-turn-steering.md).## Authoring API```ts-export interface SendOptions<TState> {- // Existing fields omitted.- supersede?: "turn";-}-export interface RouteHandlerArgs<TState> {- // Existing fields omitted.- cancelTurn(options: { continuationToken: string; turnId?: string }): Promise<CancelTurnResult>;+ // Existing helpers omitted.+ cancel(options: { continuationToken: string; turnId?: string }): Promise<CancelTurnResult>;}export interface CancelTurnResult {@@ -39,137 +32,86 @@ export interface CancelTurnResult {}```-`"accepted"` means a registered cancellation hook accepted the request, not-that cancellation has settled or necessarily matched the caller's guarded-turn. `"no_active_turn"` covers unknown, idle, parked, swept, and otherwise-uncancellable sessions. Both are successful outcomes.--The one-member `supersede` union leaves room for a future-`supersede: "session"` reset operation without defining its semantics here.--## Example+The continuation token uses the same channel-local, unprefixed format as+`send()`. eve applies the channel namespace internally.```tsexport default defineChannel({routes: [- POST("/webhooks/imessage", async (req, { send, waitUntil }) => {- const inbound = await verifyAndParse(req);- if (inbound instanceof Response) return inbound;-- waitUntil(- send(- { message: inbound.text },- {- auth: null,- continuationToken: inbound.conversationId,- supersede: "turn",- },- ),- );- return new Response(null, { status: 200 });+ POST("/threads/:threadId/stop", async (_request, { cancel, params }) => {+ const result = await cancel({ continuationToken: params.threadId });+ return Response.json(result);}),],- events: {- "message.completed": async (data, channel) => {- if (data.finishReason !== "tool-calls" && data.message) {- await channel.imessage.sendText(data.message);- }- },- },});```-Filtering `finishReason: "tool-calls"` avoids sending pre-tool narration as a-final reply. Cancellation does not retract outbound side effects: if a handler-has already sent content, eve cannot take it back. A channel that requires a-strict no-superseded-output guarantee must buffer output until it observes the-turn's terminal event.--## Required semantics--### A superseding send must not cancel itself--The payload must be durably attached to the existing session before the-operation reports success. Cancellation must target the turn observed before-that delivery, using its `turnId`. If the session advances before cancellation-commits, the guard turns the cancellation into a no-op instead of cancelling-the turn that contains the new payload.--### Rekeying must not create a second session--`ctx.session.setContinuationToken()` can replace the delivery hook while a-webhook is in flight. Once the supersede operation resolves a continuation-token to an existing session, a concurrent rekey must not make delivery fall-through to `runtime.run()` and start a new history. The operation must either-deliver to the resolved session or return an explicit retryable conflict.--### Concurrent superseding sends must compose--Every accepted payload is delivered exactly once. Several sends may cancel the-same observed turn, and their payloads may be coalesced into the next turn in-the order defined by the existing delivery queue. No request may silently-retarget a newer turn.--### Existing cancellation limits remain--Task-mode and hook-conflict-degraded turns may have no current cancellation-hook. Superseding delivery still succeeds in that case and behaves like plain-queueing. Partial model output remains on the event stream, while durable model-history keeps only content that had already settled. Already-executed tool or-channel side effects remain; cancellation is not rollback.+Authors who already have a session id can continue using the public+`RouteContext.agent.cancelTurn({ sessionId, turnId? })` primitive in authored+filesystem routes. The new `cancel` helper is the continuation-addressed+convenience surface for `defineChannel`. The lower-level agent method keeps the+precise `cancelTurn` name because it is addressed by runtime session id.++## Semantics++- `"accepted"` means the existing turn-cancellation hook accepted the request.+ Settlement remains asynchronous and is confirmed by `turn.cancelled`+ followed by `session.waiting` on the event stream.+- `"no_active_turn"` covers an unknown token, an idle or parked session, a+ completed session, and a turn without a live cancellation target. Both+ statuses are successful outcomes.+- `turnId` is an optional stale-request guard. A mismatch is consumed as a+ benign no-op and cannot cancel a newer turn.+- The helper never starts a session, sends input, clears history, or falls back+ to `runtime.run()`.+- Cancellation stops work without replacement input. It is not an alias for+ `turnPolicy: "steer"`.+- Existing cancellation behavior is unchanged: partial streamed output and+ completed side effects remain observable, durable settled history is kept,+ and active local or remote descendants receive cancellation recursively.## Runtime boundary-Neither simple ordering is sufficient:--- cancel then deliver leaves a gap where the channel token can rekey; the old- delivery can miss and incorrectly start a fresh session;-- deliver then unguarded cancel can stop the new turn that already includes the- payload.--The implementation therefore needs one runtime-owned composite operation:-```text-channel send- │- ▼-resolve token ──► existing session + observed active turn- │- ├─ durably deliver payload to that session- └─ cancel only the observed turn id- │- └─ session advanced? guard mismatch, benign no-op+channel-local continuation token+ │+ ▼+resolve current session owner without delivery+ │+ ▼+existing cancelTurn({ sessionId, turnId? })```-The exact World implementation may be transactional or use a stable-session-addressed inbox, but the observable invariants above are required.-`Runtime.cancelTurn({ sessionId, turnId? })` remains the primitive for explicit-session-id cancellation; channel token resolution belongs above it.--## Testing--- Unit-test result mapping and continuation-token namespacing.-- Prove delivery commits before the guarded cancellation is issued.-- Race `setContinuationToken()` against superseding delivery and assert the- payload stays on the original session with no fallback run.-- Race turn settlement against superseding delivery and assert a newer turn is- never cancelled.-- Exercise concurrent superseding sends and verify exactly-once delivery and- deterministic coalescing.-- Cover idle, parked HITL, task-mode, stale `turnId`, and degraded turns.-- Add an e2e custom-channel fixture that supersedes a hanging turn and answers- the complete coalesced input once.--## Out of scope--- Session reset, `/new`, and `supersede: "session"`.-- Descendant cancellation cascade.-- Cancellation from in-band event handlers.-- Built-in channel debounce policy.-- Retraction of already-sent channel messages or completed tool side effects.--## Delivery--Land the runtime composite and its race tests before exposing either public-channel API. The public additions require a patch changeset and channel-authoring documentation.+Resolution and cancellation need not be one transaction because the optional+`turnId` guard protects the turn boundary. They do need these invariants:++1. Resolution returns the session that owned the token at its linearization+ point; a concurrent rekey cannot retarget cancellation to another session.+2. A missing or retired token maps to `no_active_turn`, not an exception and+ never a new run.+3. Runtime or Workflow failures other than an inactive target remain errors;+ they must not be collapsed into `no_active_turn`.+4. The route author remains responsible for authenticating the inbound stop+ request, exactly as for `send()`.++The internal implementation may use a non-delivery session command to resolve+the continuation hook's owning run id. That command must be consumed without+entering adapter delivery or changing the input queue.++## Verification and delivery++- Unit-test channel namespacing, result mapping, and route-helper injection for+ HTTP and WebSocket routes.+- Cover unknown, parked, active, already-settled, guarded-match, and+ guarded-mismatch cases.+- Race continuation rekeying and turn settlement against resolution; assert+ that no newer session or turn is cancelled.+- Assert unexpected resolution errors propagate and never fall back to session+ creation.+- Add a custom-channel e2e route that cancels a running tool using only the+ public helper, observes the cancellation boundary, then resumes the same+ session normally.++This is an additive public channel API and requires custom-channel docs, API+tests, and a patch changeset. It does not change the already-landed cancellation+protocol or events.diff --git a/research/turn-cancellation-descendant-cascade.md b/research/turn-cancellation-descendant-cascade.mddeleted file mode 100644index b5829469d..000000000--- a/research/turn-cancellation-descendant-cascade.md+++ /dev/null@@ -1,53 +0,0 @@-----issue: https://github.com/vercel/eve/issues/483-status: in-review-last_updated: "2026-07-15"------# Turn cancellation, layer 3: descendant cascade--## Summary--Cancelling a parent turn now cancels every successfully started child in its-pending runtime-action batch before the parent settles. Local children receive-a resume on their stable `{childSessionId}:cancel` hook. Remote children receive-an authenticated `POST /eve/v1/session/:childSessionId/cancel`. Each child runs-the same cascade, so cancellation follows nested local and remote delegation.--```text-parent cancel- │- ├─ wait for dispatch adoption- │- ├─ local child ── resume {sessionId}:cancel- └─ remote child ── POST /eve/v1/session/:id/cancel- │- └─ child repeats the cascade-```--## Semantics--- Dispatch remains non-abortable until every start attempt has returned. The- parent durably records each successful child session id before it observes- cancellation, closing the start-versus-cancel adoption race.-- Child cancellation requests run in parallel. The parent waits for requests- to be accepted, not for descendant turns to finish settling.-- A newly started child may return before its cancel hook is registered. Local- and remote `no_active_turn` responses retry up to 12 attempts at 250 ms.- Remote network errors and HTTP 408, 425, 429, and 5xx responses use the same- window, resolving outbound auth again for every attempt.-- Descendant cancellation cannot turn a parent cancellation into a failure.- Permanent and exhausted request errors are logged; the parent still emits- `turn.cancelled` followed by `session.waiting` and drops the pending batch.-- Completed, parked, swept, or cancellation-degraded children are benign- no-ops. This layer cancels active turns only; terminal task cancellation and- parked-session reset remain separate work.--## Rejected mechanisms--Workflow `Run.cancel()` is not the primary mechanism because it only stops a-workflow at a suspension point and does not abort the in-flight `turnStep`.-Minting a Workflow webhook would duplicate eve's authenticated cancellation-route and expose an unnecessary public capability URL. Reusing the turn hook-and HTTP route keeps local and remote cancellation on the same cooperative-settlement path.diff --git a/research/turn-cancellation-harness-abort.md b/research/turn-cancellation-harness-abort.mddeleted file mode 100644index ce98122ab..000000000--- a/research/turn-cancellation-harness-abort.md+++ /dev/null@@ -1,171 +0,0 @@-----issue: https://github.com/vercel/eve/issues/483-status: complete-last_updated: "2026-07-06"------# Turn cancellation, layer 0: harness abort propagation--## Summary--Five cancellation attempts (#118/#127/#128/#135, #230, #347) failed to land, and every blocking-failure lived in the _trigger and cross-process_ layers: raced hooks in the non-terminating session-driver (#230), deterministic hook-token reclaim conflicts and duplicate finalizer-step execution-(upstream Workflow bugs blocking #347), and dispatch-adoption races with delegates. The abort-_propagation_ layer, by contrast, was rewritten three times and converged on the same shape each-time. It has never shipped only because it was always chained to the layers that break.--This plan ships the propagation layer alone: the turn harness accepts an optional `AbortSignal` and-honors it end-to-end — model calls, retries, recovery, compaction, and tool execution. No production-code creates or aborts the signal yet, so runtime behavior is unchanged; the layer is exercised by-tests. Each later layer then adds exactly one trigger concern.--## The cancellation stack--Cancellation lands bottom-up. Each layer is independently shippable and reviewable:--```text-layer 4 clients + channels + evals MessageResponse.cancel(), channel ops, /new, eval controls-layer 3 descendants subagent inbox cascade, remote cancel POST, adoption races-layer 2 trigger surface POST /eve/v1/session/:id/cancel resuming the stable- `${sessionId}:cancel` hook (optional turnId guard)-layer 1 turn-workflow ownership durable AbortController + cancel race in the terminating- turnWorkflow, cancelled-turn finalizer,- turn.cancelled → session.waiting-layer 0 harness abort propagation ← this plan-```--Layers 1–2 stay blocked on the two documented upstream Workflow issues (hook dispose-before-reclaim-determinism; duplicate step execution under one correlation id). Layer 0 involves **no hooks, no-routes, no events, and no durable state**, so it has no upstream dependency. Layer 4 semantics-remain governed by `research/channel-session-reset.md`.--## Authoring API--Two small public additions; everything else is internal plumbing.--### `ToolContext.abortSignal`--```ts-export type ToolContext = SessionContext & {- /** Aborts when the active turn is cancelled. */- readonly abortSignal: AbortSignal;- // ...existing members-};-```--Always present. When no upstream signal exists (all of production until layer 1), the harness-supplies a signal that never aborts, so authored code is written once and works unchanged when-triggers arrive.--### Tool execute options--`ToolDefinition.execute` gains an optional second parameter, the AI SDK's per-call tool options-(minus `context`), carrying `toolCallId` and `abortSignal`:--```ts-export type ToolExecuteOptions = Omit<ToolExecutionOptions<unknown>, "context">;-execute?: (input, options?: ToolExecuteOptions) => Promise<unknown> | unknown;-```--## Semantics--- **Absent or never-aborted signal: byte-for-byte unchanged behavior.** The signal is optional at- every internal seam.-- **Abort during a model call:** the AI SDK call rejects; the harness rethrows `signal.reason`- without retrying (`runModelCallWithRetries` checks the signal before each attempt and after each- error) and without entering the empty-response/provider-tool recovery pipeline.-- **Abort during stream consumption:** a stream that ends quietly after abort is caught by an- explicit aborted-check after stream drain, so the turn can never complete "successfully" from a- truncated stream.-- **Abort during compaction:** the compaction `generateText` call receives the same signal.-- **Abort during a tool call:** best effort. The AI SDK forwards the call-level signal into every- tool's execute options; framework tools honor it (sandbox commands, MCP/OpenAPI fetches), authored- tools receive it via `ctx.abortSignal` and `options.abortSignal` and may ignore it. The harness- does not wait for a non-cooperating tool: the surrounding model-call promise settles on abort and- the straggler's result is discarded.-- **Abort is not failure.** A new `isTurnCancellation(error)` classifier (matching the canonical- serializable cancellation reason, `TurnCancelledError`) is checked before retry/recovery- classification. `classifyModelCallError` must never return `"retry"` or `"recoverable"` for a- cancellation. This is the invariant #347's Windows leak violated (`AbortError` escaping into- `session.failed`); it is pinned by unit tests now so later layers inherit it.-- The cancellation reason must round-trip Workflow step serialization, because layer 1 will observe- it across a `"use step"` boundary. `TurnCancelledError` is a plain, cause-free error with a stable- `name` checked by name (not `instanceof`), like `isNoOutputGeneratedError`- (`harness/model-call-error.ts:279`).--## Implementation seams--The signal enters at `TurnStepInput` and flows one direction. Workflow-core already serializes-`AbortSignal` across `"use step"` boundaries and propagates aborts to running steps in real time-(`.generated/compiled/@workflow/core/workflow/abort-controller.d.ts`,-`serialization/types.d.ts:240`), so this typing is forward-compatible with layer 1's durable-controller.--```text-TurnStepInput.abortSignal? execution/durable-session-migrations/turn-workflow.ts- └─ turnStep execution/workflow-steps.ts:108- └─ CreateExecutionNodeStepInput execution/node-step.ts:48- └─ ToolLoopHarnessConfig harness/types.ts:184- ├─ agent.stream / generate harness/tool-loop.ts:750, 813- ├─ post-drain aborted check harness/tool-loop.ts:760 (after emitStreamContent)- ├─ runModelCallWithRetries harness/tool-loop.ts:2150- ├─ recovery pipeline guards harness/tool-loop.ts:~830 catch block- ├─ maybeCompact → compactMessages harness/tool-loop.ts:2098, compaction.ts:128- └─ AI SDK ToolCallOptions.abortSignal (automatic once passed to stream/generate)- ├─ wrapToolExecute forwards options harness/tools.ts:176 (execute(input, options))- ├─ HarnessToolDefinition.execute harness/execute-tool.ts:25- ├─ authored ToolContext.abortSignal execution/tool-auth.ts:53, node-step.ts- ├─ sandbox sessions bound to the signal execution/sandbox/abort-bound-session.ts- │ (requireSandboxSession(signal) + ctx.getSandbox(); per-call signals compose- │ via AbortSignal.any, so callers can never opt out of turn cancellation)- └─ MCP/OpenAPI executeTool(name, args, { abortSignal })- runtime/connections/mcp-client.ts:119, openapi-client.ts:98-```--Notes:--- `wrapToolExecute` is the single splice point for all tool kinds; today it drops the AI SDK- options (`harness/tools.ts:176`). Forwarding them reaches framework, authored, MCP, and dynamic- connection tools uniformly.-- The sandbox stack already plumbs `abortSignal` end-to-end through every backend (Vercel, Docker,- just-bash, microsandbox); the framework sandbox tools simply never pass one. Layer 0 turns that- dead plumbing live, which is where "truly stopping work" (killing a running `bash` command)- materializes first.-- `harness/tool-loop.ts` has drifted since #347 (e.g. hidden runtime-action tool names in the- stream branch, `rethrowNoOutputAsEmptyResponse`). Re-derive the diff on main; do not cherry-pick.-- The mock model adapter (`runtime/agent/mock-model-adapter.ts`) gains abort support (#118 had a- 19-line version) so unit tests need no real provider.--## Out of scope for layer 0--- No AbortController is created in production code; no hook, route, event (`turn.cancelled`), or- client API exists yet.-- No cancelled-turn epilogue: with no trigger, a turn can never abort in production. The epilogue- (settle → `turn.cancelled` → `session.waiting`) is layer 1, where the failure classification- seam (`isTurnCancellation`) built here gets consumed.-- No delegate/remote cascade; pending runtime-action batches are untouched.--## Delivery and verification--One PR with a patch changeset covering both halves:--1. **Core propagation** — signal threading (`TurnStepInput` → harness), model-call/retry/recovery/- compaction handling, `TurnCancelledError` + `isTurnCancellation`, `wrapToolExecute` options- forwarding, `ToolContext.abortSignal`, docs for the public additions.-2. **Framework tool executors** — sandbox tools pass `options.abortSignal` into `sandbox.run` and- file I/O; `web_fetch` composes the signal with its timeout; MCP/OpenAPI `executeTool` accepts- and threads a signal into the underlying request.--Tests (no hooks → unit + integration tiers only):--- unit: retries never resume after abort; recovery pipeline never runs on abort;- `classifyModelCallError` interplay with `isTurnCancellation`; `wrapToolExecute` forwards options;- compaction receives the signal; post-drain aborted check.-- integration (`tool-loop-cancellation.integration.test.ts`): a real `ToolLoopAgent` over a- cancellable `MockLanguageModelV3` — abort mid-stream settles with the canonical cancellation and- no failure events; an executing tool observes a live signal and its straggler result is- discarded; an inert signal produces output identical to no signal.-- #347's harness test additions (`tool-loop.test.ts` +45, `tools.test.ts` +50) were the starting- corpus; its e2e fixture (`agent-cancellation`, `wait-for-cancellation` tool) stays shelved until- layer 2 provides a trigger.diff --git a/research/turn-cancellation-trigger-surface.md b/research/turn-cancellation-trigger-surface.mddeleted file mode 100644index 159dab43c..000000000--- a/research/turn-cancellation-trigger-surface.md+++ /dev/null@@ -1,94 +0,0 @@-----issue: https://github.com/vercel/eve/issues/483-status: in-review-last_updated: "2026-07-15"------# Turn cancellation, layer 2: HTTP trigger--## Summary--Layer 1 (#573) made each parkable turn claim a stable-`{sessionId}:cancel` hook and settle cancellation as `turn.cancelled` followed-by `session.waiting`. Layer 2 exposes that behavior through-`POST /eve/v1/session/:sessionId/cancel` on `eveChannel()` and adds hosted e2e-coverage.--The route delegates to `Agent.cancelTurn({ sessionId, turnId? })`; hook-addressing and workflow error normalization stay behind `Runtime`. No new-durable state, events, or workflow behavior is introduced.--## HTTP contract--- Auth uses the channel's existing `routeAuth` policy.-- The optional body is `{ "turnId"?: string }`. An empty body targets whichever- turn owns the hook. A `turnId` limits the request to the observed turn; a- mismatch is consumed as a no-op.-- Responses include `cache-control: no-store` and `EVE_SESSION_ID_HEADER`.--| Status | Body | Meaning |-| ------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------- |-| 202 | `{ ok: true, sessionId, status: "accepted" }` | A registered cancellation hook accepted the request. Observe the stream for the effective outcome. |-| 202 | `{ ok: true, sessionId, status: "no_active_turn" }` | No resumable cancellation target exists, including unknown and already-settled sessions. |-| 400 | `{ ok: false, error }` | Malformed body or missing session id. |-| 401/403 | — | Authentication rejected the request. |-| 500 | `{ ok: false, error, errorId }` | An unexpected runtime failure occurred. |--Both 202 outcomes are successful so clients can safely retry or issue a late-stop request. Task-mode sessions have no turn cancellation hook and therefore-return `no_active_turn`.--## Semantics--- `"accepted"` is acceptance, not settlement. Confirm an effective cancel by- observing `turn.cancelled` and `session.waiting`.-- A guarded mismatch returns `"accepted"` because the hook accepted and- discarded the payload.-- A hook-conflict-degraded current turn is uncancellable. A stale prior owner- may accept the request, so HTTP status alone cannot identify the affected- turn.-- Missing hooks, missing owner runs, terminal-run expiry, and terminal-state- conflicts map to `"no_active_turn"`; other errors become 500s.-- Cancellation leaves partial content on the event stream and retains completed- side effects; durable model history keeps only content that had already- settled. It emits no failure event and leaves the session ready for a- follow-up.--## Data flow--```text-POST /eve/v1/session/:sessionId/cancel { turnId? }- │- ├─ routeAuth- ├─ parse body- └─ agent.cancelTurn- │- └─ runtime.cancelTurn- └─ resumeHook(`${sessionId}:cancel`, { turnId? })- ├─ missing / terminal target → no_active_turn- └─ accepted payload → accepted-```--The token helper lives in `execution/turn-cancellation-token.ts`; the HTTP-route does not import workflow hook APIs.--## Coverage--- Unit tests cover parsing, auth, Agent delegation, result mapping, headers,- terminal races, and unexpected errors.-- World-local integration cancels a hanging tool through the real route,- verifies stale guards are no-ops, treats unknown and late requests as- `no_active_turn`, and proves the next turn succeeds.-- The `agent-cancellation` e2e fixture waits for `actions.requested`, cancels- over HTTP, checks `turn.cancelled` → `session.waiting`, then sends a follow-up.- Its stream open retries the same short propagation window as the client.--## Out of scope--Task-mode terminal cancellation, caller-supplied reasons, descendant cascade,-session reset, and client/channel convenience APIs remain separate work.--## Delivery--Ship the route, Runtime/Agent operation, docs, tests, e2e fixture, and patch-changeset together.diff --git a/research/turn-cancellation-turn-workflow.md b/research/turn-cancellation-turn-workflow.mddeleted file mode 100644index dfe4e62ad..000000000--- a/research/turn-cancellation-turn-workflow.md+++ /dev/null@@ -1,337 +0,0 @@-----issue: https://github.com/vercel/eve/issues/483-status: in-review-last_updated: "2026-07-13"------# Turn cancellation, layer 1: turn-workflow ownership--> **Ship status: unblocked** (eve#573). The `e2e-local` stress suite had-> exposed that serializing the durable `AbortSignal` into `turnStep` cost-> ~0.5 s fixed per step on world-local and opened an abort-stream tail-> reader whose 100 ms polling did a full `readdir` of the world-global-> chunks directory — a 100-turn session compounded from ~0.9 s/turn into-> a 240 s replay timeout (the Vercel world was unaffected). Filed as-> [workflow#2795](https://github.com/vercel/workflow/issues/2795) (fixed-> cost) and-> [workflow#2797](https://github.com/vercel/workflow/issues/2797)-> (scaling); both fixed upstream by-> [workflow#2807](https://github.com/vercel/workflow/pull/2807)-> (`executeStep` cancels abort readers post-step; chunks sharded-> per-stream), shipped in `@workflow/core` 5.0.0-beta.28 /-> `@workflow/world-local` 5.0.0-beta.24 and vendored into eve by #612.-> Re-benched on the fixed runtime: 40 sequential signal-bearing turns run-> at the ~385 ms no-signal floor with no compounding.--## Summary--Layer 0 (#494) made the harness honor an `AbortSignal` end to end but left-`TurnStepInput.abortSignal` unpopulated. Layer 1 makes the turn cancellable-in-process: the turn-owned `turnWorkflow` claims a durable session-scoped-cancel hook (`{sessionId}:cancel`) and a durable `AbortController`, and-resuming the hook mid-turn settles the turn as `turn.cancelled` →-`session.waiting` — never as a failure. The payload's optional `turnId`-guard scopes a cancel to the turn the caller observed. No HTTP route,-client API, or channel surface exists yet (layer 2); the only trigger is-resuming the cancel hook, which tests do directly.--## Cancellation semantics (as shipped)--- **Trigger**: `resumeHook("${sessionId}:cancel", { turnId? })`. The cancel- token is **session-scoped and stable**, so a layer-2 trigger derives it- from the session id alone — no per-turn token discovery, no client-carried- hook token. Each turn workflow run creates and claims the token at turn- start and disposes it before publishing its turn result, so at most one- live cancel hook exists per session and the next turn's claim never races- the prior run's teardown. A claim conflict (residue of a crashed prior- turn whose hooks are not yet swept) degrades that turn to uncancellable- rather than failing it. The hook payload carries no caller-supplied- reason: layer 1 uses one canonical cancellation reason (the layer-0- `TurnCancelledError`), and caller reasons arrive with the trigger surface- in layer 2.-- **Turn guard**: the payload's optional `turnId` scopes the cancel to the- turn the caller observed (every stream event is stamped with its- `turnId`). A guard naming any other turn is consumed as a benign no-op —- a cancel that races a turn boundary can never kill a turn the caller- never saw. Omitting `turnId` cancels whatever turn is currently running,- the right default for a plain stop button. Guard reads are durable hook- reads, so the skip sequence replays deterministically.-- **Cancel during model/tool work**: the durable signal (threaded through- `TurnStepInput.abortSignal`, the layer-0 seam) aborts the in-flight harness- work in real time. The turn settles with `turn.cancelled` followed by- `session.waiting` on the session stream (stream version 19), and the- session accepts the next message normally.-- **Cancel is not failure.** A cancelled turn never emits `turn.failed`,- `step.failed`, or `session.failed`, and the aborted `turnStep` is never- retried as a failure. Pinned by tests; this is the invariant that leaked- in #347.-- **Cancel during an in-line runtime-action wait** (subagent / dynamic- workflow results): the wait stops and the turn settles cancelled the same- way. The pending runtime-action batch and workflow interrupt are dropped- at settle time (their tool-call exchange lives inside the batch and never- reached history; replaying them would re-dispatch the actions). Layer 3 now- cancels successfully adopted descendants before this settle.-- **Cancel after the turn settled (or before it starts)**: benign no-op —- the settle disposed the session cancel hook, so a late resume rejects- with `HookNotFoundError`. Duplicate and same-instant cancels are also- safe without trigger-side single-flighting: a duplicate resume neither- re-dispatches the in-flight step (workflow#2848) nor corrupts replay- when racing disposal (workflow#2808). Layer 2 only needs to treat- "already resumed/disposed" as success.-- **Partial content is kept.** Whatever the harness emitted before the abort- stays on the stream, and durable history persists exactly what the harness- had settled at abort time — no rollback, no synthesis.-- **Cancellation is driver-negotiated and parkable-only.** The pinned driver- body advertises `cancelledTurnSettle` through the dispatch input; the turn- workflow registers the cancel hook only when it is set _and_ the session- can park (conversation mode, or an anchored continuation token — delegated- children always have one). Drivers pinned before the capability get no- cancel hook, so a layer-2 trigger cannot strand them with an unsettled- cancelled park; unparkable root task runs get no hook, so cancellation can- never decay into `session.failed`. Task-mode terminal semantics arrive- with layer 2.-- **Cancel during a descendant HITL wait**: the proxy epilogue has already- streamed the turn's waiting boundary, so the settle emits nothing extra- (no fabricated turn id, no duplicate `session.waiting`) and clears the- proxy input-request map — later HITL answers stay with the parent instead- of routing to the orphaned child.-- **The settle epilogue runs the authored-hook pipeline.** `turn.cancelled`- and its `session.waiting` dispatch stream-event hooks like any turn-step- emission.-- **Parked sessions cannot be cancelled** in this layer: parking terminates- the turn workflow, so there is no turn to cancel. The session-scoped story- stays in `research/channel-session-reset.md`.-- **The legacy (non-turn-inbox) workflow path is untouched.**--## Data flow (as shipped)--```text-test / (layer 2 route) resumeHook(`${sessionId}:cancel`, { turnId? })- │-turnWorkflow execution/turn-workflow.ts- control = createTurnCancellationControl({ sessionId, expectedTurnId })- │ claims the stable session token; a conflict (stale prior-turn- │ claim) degrades the turn to uncancellable instead of failing- │ mismatched turnId guards are consumed as durable no-op reads- │ the hook-read continuation aborts the durable controller —- │ replay-deterministic (keyed to the hook_received event), no- │ promise race decides whether abort() runs- ▼- turnStep (awaited plainly) execution/workflow-steps.ts- harness aborts (layer 0) → TurnCancelledError- → returns { action: "cancelled" } pure marker, no side effects- │- cursor.finish(…, { cancelled: true, kind: "park" })- │ park arm reused: new NextDriverAction arms- │ break pinned drivers; optional fields don't- ▼-workflowEntry driver execution/workflow-entry.ts- settleCancelledTurnStep execution/settle-cancelled-turn-step.ts- emits turn.cancelled → session.waiting, clears pending batch/interrupt,- persists the between-turns session- then the normal park playbook: rekey, await the next message-```--Within `waitForRuntimeActionResults`, the inbox read is raced against the-cancel-hook read; a cancel releases any raced public delivery back to the-driver (`turn-delivery-cancelled`) and loops into one final `turnStep`,-which observes the aborted signal at entry (before the park-resume stages,-which would otherwise re-park on the still-pending batch) and settles-through the same cancelled arm.--## Where the shipped design deviates from the proposal--Integration against the real runtime (vendored `@workflow/core`-5.0.0-beta.26) surfaced behaviors that reshaped the epilogue path. This-section is historical: the runtime defects cited below (#2780, #2781)-have since been fixed upstream — see "Runtime findings" for the fixes-and which mitigations remain load-bearing:--1. **No workflow-side race; abort in the hook-read continuation.** The- durable `abort()` writes a hook event, and the upstream contract- ("`abort()` becomes a no-op on replay") presumes the call site is- reached deterministically on every replay — gating it behind a- `Promise.race` winner makes that reachability replay-dependent. The- abort now fires in the `.then` of the cancel-hook read itself, keyed to- the `hook_received` event. (The corruption initially attributed to this- shape was later traced to workflow#2781; the deterministic call site is- kept as contract hygiene.)-2. **`turnStep`'s cancelled result is a pure marker.** The runtime can- supersede an aborted step attempt and re-dispatch it under the same- correlation id, with both attempts running to completion in-process- (at-least-once inline execution, workflow#2780). Any cancel-path side- effect inside `turnStep` — including the epilogue — can therefore- duplicate.-3. **The epilogue runs in the _driver_, not the turn run.** Queued- cancel-payload and abort-hook wakes re-dispatch in-flight steps of the- turn run; the driver's wake sources exclude the cancel hook. The driver- recognizes `park` + `cancelled: true` and runs `settleCancelledTurnStep`- before the normal park playbook. (Old pinned drivers ignore the marker- and simply park — harmless, since no cancel trigger predates them.)-4. **Turn control hooks are disposed one turn late.** The turn run's final- control send is at-least-once (workflow#2780); a late duplicate resume- on a _disposed_ hook is accepted and logged by the world and then- diverges the driver's replay (workflow#2781).- `dispatchAndAwaitTurn` now returns a deferred `dispose()` that- the driver invokes when the _next_ turn settles (or the session ends),- by which time the previous turn's run has completed and cannot re-send.-5. **Turn-run teardown is dispose-only — never `iterator.return()`.** The- cancel hook always has an outstanding durable read (the abort- continuation), and an async generator honors `return()` only after its- in-flight `await` settles — never, for a turn that wasn't cancelled.- Closing the iterator therefore suspends the turn run forever: it never- reaches `run_completed`, so the world never sweeps its hooks, leaking- one running run and three hooks (inbox, cancel, durable-abort) per- turn, and `getByToken`'s O(live hooks) scan slows every resume in the- session. Hook disposal alone is the runtime's sanctioned pattern:- pending reads are dropped and the run proceeds to completion, where- terminal-run cleanup deletes the run's hooks.--## Runtime findings (filed upstream)--Observed against `@workflow/world-local` during integration testing and-re-validated with standalone spike repros before filing:--- **[workflow#2780](https://github.com/vercel/workflow/issues/2780)** —- at-least-once inline step execution under wakes: a `hook_received` that- lands while a step attempt is in flight re-dispatches the step under the- same correlation id; both attempts run concurrently in-process and both- flush side effects (only one `step_completed` is recorded). Standalone- repro deterministic (4/4). Complement to workflow#2777/#2778.- **Fixed** by [workflow#2848](https://github.com/vercel/workflow/pull/2848)- (`@workflow/core` 5.0.0-beta.30): inline steps are stamped with the- owning queue message id, wake replays schedule a delayed backstop- (lease-bounded) instead of immediately re-dispatching an in-flight step,- and an upstream in-process single-flight collapses racing attempts. The- residual is the queue's ordinary at-least-once envelope (redelivery of- the owning message while the step is alive; cross-instance duplicates- post-lease).-- **[workflow#2781](https://github.com/vercel/workflow/issues/2781)** —- a `resumeHook` racing hook disposal is accepted and journaled after- `hook_disposed`; the owning run's replay then deterministically diverges- at that event (`REPLAY_DIVERGENCE`, escalating to- `CorruptedEventLogError`). The duplicate resume that hit the window came- from a #2780-style re-executed step. The acceptance window is- timing-dependent (a simplified standalone repro produced the duplicate- resume but landed it after disposal enforcement); the journal-level- evidence is unambiguous.- **Fixed** by [workflow#2808](https://github.com/vercel/workflow/pull/2808)- (`@workflow/world-local` 5.0.0-beta.24): `hook_received` acceptance is- serialized against the per-hook dispose lock and re-validated before- the event append, so a resume ordering after disposal is rejected with- `HookNotFoundError` instead of corrupting the owner's replay. Two- caveats: the fix is acceptance-side only (a bad order already journaled- before it still diverges on replay), and it ships in world-local — the- hosted Vercel world's parity is unconfirmed.-- **Not filed — falsified in isolation**: an earlier hypothesis that- reaching `abort()` behind a promise-race winner corrupts the event log- did not reproduce standalone (5/5 clean); every corruption we observed- traced to #2781. The abort-in-hook-continuation shape is kept anyway as- determinism hygiene: the upstream abort-controller contract ("`abort()`- becomes a no-op on replay") presumes the call site is reached on every- replay, and the duplicate `abrt` `hook_received` we logged under wake- storms adds wakes that feed #2780's preconditions.--Upstream status (all four findings are now fixed in the runtime this-branch vendors — `@workflow/core` 5.0.0-beta.31, `@workflow/world-local`-5.0.0-beta.27):--- [workflow#2779](https://github.com/vercel/workflow/pull/2779) fixed- #2777/#2778 (suspension-handler per-token ordering; world-local claim- release).-- [workflow#2808](https://github.com/vercel/workflow/pull/2808) closed- #2781's remaining acceptance window (world-local).-- [workflow#2848](https://github.com/vercel/workflow/pull/2848) fixed- #2780 (inline step message ownership).--Consequences for layer 1's mitigations:--- The `settleCancelledTurnStep` in-process single-flight it originally- carried became redundant with the upstream one and was removed; the- queue's residual at-least-once envelope can duplicate the epilogue,- like every stream emission under crash retry.-- Deferred control-hook disposal stays: `sendTurnControlStep` does not- treat `HookNotFoundError` as benign, so a rare crash-retry duplicate of- the final control send against an already-disposed hook would surface- as a turn error, and hosted-world parity for the #2808 acceptance fix- is unconfirmed. Both are addressable in a follow-up if immediate- disposal is worth it.-- The driver-owned epilogue, the pure `cancelled` marker, the- abort-in-hook-continuation shape, and dispose-only teardown all stand- on rationale independent of the fixed bugs (wake-source isolation,- at-least-once side-effect hygiene, replay determinism, async-generator- semantics) and are unchanged.--Consequences for layer 2 (relaxations from the earlier guidance):--- Strict per-turn single-flighting of cancel resumes is no longer- required: a duplicate resume neither spawns a concurrent step attempt- (#2848) nor corrupts replay when racing disposal (#2808). Treating- "already resumed/disposed" as success is sufficient.-- These fixes made a stable session-scoped cancel token viable (#2779- makes the dispose-then-recreate claim shape safe across consecutive- runs; #2808 removes the disposal race), and layer 1 **adopted it**:- the shipped token is `{sessionId}:cancel` with an optional `turnId`- payload guard. Layer 2's route is therefore pure derivation —- `POST /eve/v1/session/:id/cancel` resumes `{sessionId}:cancel`,- forwards the caller's optional `turnId`, and maps `HookNotFoundError`- (no turn in flight) to a "nothing to cancel" success.--## Invariants (pinned by tests)--1. An aborted `turnStep` settles by return value; no thrown cancellation- crosses the step boundary — the turn workflow run records no- `step_failed`/`step_retrying` events and at most one `step_completed`- per correlation id.-2. `turn.cancelled` is emitted once per cancelled turn under normal- operation (at-least-once under crash retry, like every stream- emission), always followed by- `session.waiting`; zero failure events on the cancelled path; the aborted- tool executes once; the cancelled turn streams one `step.started`.-3. The cancel hook token is session-scoped and never doubly live: each- turn workflow run claims `{sessionId}:cancel` at turn start and- disposes it before publishing its turn result, so the next turn's- claim never races the prior run's teardown. A conflicting stale claim- degrades the turn to uncancellable, never to a failure.-4. A cancelled subagent wait is not re-dispatched: the next turn runs- normally with no `subagent.called`.-5. With no cancel resumption, behavior is unchanged (all pre-existing unit- and integration suites pass).--## Testing--- **Integration** (`execution/turn-cancellation.integration.test.ts`): real- `workflowEntry` + `turnWorkflow` over world-local with a hanging- `wait_for_cancel` tool; covers mid-tool cancel (with a matching `turnId`- guard) + follow-up turn, a stale-guard cancel consumed as a no-op with- the turn still cancellable afterwards, cancel during an in-flight- subagent wait (via the mock model's `Delegate to a subagent: …`- directive) + no re-dispatch, late/duplicate cancel no-ops, the no-retry- canary, and cancel-hook sweep after each settle.-- **Unit**: `createTurnCancellationControl` (token derivation, claim- conflict → uncancellable, guard matching and stale-guard skip,- idempotent disposal), cancelled-turn arm of `turnWorkflow` (park +- `cancelled` marker, `canPark` bypass, signal threading,- dispose-before-result ordering), deferred control-hook disposal,- `activeTurnId`, `emitCancelledTurn` (event order, turn-id- reconstruction, state advance), `createTurnCancelledEvent`,- stream-version pin.-- **E2E stays shelved** until layer 2 provides an HTTP trigger.--## Out of scope--- `POST /eve/v1/session/:id/cancel`, caller-supplied cancellation reasons,- channel/client/eval APIs (layers 2 and 4). No token discovery is needed:- the route derives `{sessionId}:cancel` directly.-- Descendant cascade — local subagent inbox propagation and remote cancel- (layer 3). Layer 1 only discards their late results.-- Cancelling parked sessions or session-scoped cancellation- (`research/channel-session-reset.md`).-- The legacy non-turn-inbox workflow path.--## Delivery--Shipped in one PR with a **patch** changeset (additive `turn.cancelled`-protocol event, stream version 18 → 19; no breaking public API). Scope-decisions settled at review: the runtime-action wait arm is **in**, and the-cancellation reason is canonical-only until layer 2.diff --git a/research/turn-cancellation-upstream-spike.md b/research/turn-cancellation-upstream-spike.mddeleted file mode 100644index b0ff72d9b..000000000--- a/research/turn-cancellation-upstream-spike.md+++ /dev/null@@ -1,163 +0,0 @@-----issue: https://github.com/vercel/eve/issues/483-last_updated: "2026-07-02"-status: complete------# Turn cancellation: upstream Workflow spike (layers 1–2 de-risk)--## Verdict--| Blocker | Status on vendored `@workflow/core` 5.0.0-beta.26 | Consequence |-| --------------------------------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------ |-| Duplicate step execution under one correlation id (layer-1 finalizer shape) | **Not reproduced — 5/5 clean** | **Layer 1 is unblocked. Proceed.** |-| Hook token reuse after dispose, within one run | **Reproduced — 5/5 deterministic self-conflict** | Layer 2 blocked for naive same-token designs |-| Hook token reuse after dispose, across sequential runs | **Reproduced — 4/5 flaky conflict** | Layer 2 blocked for session-scoped cancel tokens |--Layer 2 has two viable paths that do not wait on upstream: a verified-in-workflow barrier, and eve's existing indexed-token pattern (see-"Paths forward"). Upstream fixes remain worth filing; the event-log-evidence below is report-ready.--## Method--Integration-tier repros against a real `@workflow/world-local` world and the-production workflow bundle pipeline (same harness as-`workflow-entry.integration.test.ts`):--- fixtures: `packages/eve/src/internal/testing/cancellation-spike-workflow.ts`-- tests: `packages/eve/src/internal/testing/cancellation-spike.integration.test.ts`-- run: `pnpm --filter eve exec vitest run --config vitest.integration.config.ts src/internal/testing/cancellation-spike.integration.test.ts`--Versions: `@workflow/core` 5.0.0-beta.26, `@workflow/world-local`-5.0.0-beta.22 (test world), `@workflow/world` 5.0.0-beta.14. Results below are-from 5+ consecutive suite invocations.--## Finding 1: the layer-1 shape works end to end (5/5)--The exact mechanics layer 1 needs, in one workflow:--```ts-const controller = new AbortController(); // durable, hook-backed in the workflow VM-const cancelHook = createHook({ token: `${workflowRunId}:spike-cancel` });-const stepPromise = spikeWaitStep({ abortSignal: controller.signal, ... });-const winner = await Promise.race([stepPromise, cancelIterator.next()]);-// hook wins → controller.abort(reason) → in-flight step's signal aborts in real time-await stepPromise; // settles (return or FatalError throw)-await spikeFinalizerStep(); // the cancelled-turn epilogue stand-in-```--Verified, deterministically, in both step-settlement modes (return-on-abort and-throw-`FatalError`-on-abort):--- the serialized `AbortSignal` crosses the `"use step"` boundary and aborts the- **in-flight** step in real time (abort stream, not just replay state);-- the raced-and-abandoned hook read does not corrupt the run;-- the finalizer step executes **exactly once** (asserted two ways: step- side-effect log lines, and an event-log scan proving no correlation id- records more than one `step_started`);-- the workflow completes with the expected return value; no retries fire for- the aborted step.--The #347-era "duplicate finalizer-step execution" does not reproduce under the-planned layer-1 shape. Residual risk: #347's failure was observed in the full-turnWorkflow (runtime-action waits, world-postgres in CI); layer 1's PR should-keep the `expectNoDuplicateStepStarts`-style assertion in its integration-tests as a canary.--Implementation notes captured for layer 1:--- `FatalError` prefixes its name into the propagated message- (`"FatalError: <msg>"`); match cancellation by a stable marker, never by- exact message equality (consistent with layer 0's name-based- `TurnCancelledError` matching).-- Workflow-VM errors fail `instanceof Error` in test code (cross-realm);- inspect `error.name` via property access.--## Finding 2: same-token hook reuse within one run self-conflicts (5/5)--Shape: `createHook(T)` → receive → `iterator.return()` → `dispose()` →-`createHook(T)` again in the next loop iteration (the naive per-turn-`${continuationToken}:cancel` cycle). Round 1 fails every time:--```text-HookConflictError: Hook token "wrun_…:spike-reuse" is already in use by-another workflow (run "wrun_…") ← conflictingRunId = the same run-```--Event-log smoking gun (one run, chronological):--```text-hook_created corr=…RM01 token …:spike-reuse-hook_received corr=…RM01 token …:spike-reuse-hook_conflict corr=…RM02 token …:spike-reuse conflictingRunId=<same run>-hook_disposed corr=…RM01 token …:spike-reuse-```--Workflow code called `dispose()` on hook RM01 **before** creating RM02, yet-the engine validated RM02's registration before persisting RM01's disposal —-`hook_conflict` lands ahead of `hook_disposed` in the same run's log. The-suspension flush orders same-suspension hook creations ahead of pending-disposals (or storage claims are checked before disposal intents drain).--This is distinct from (and survives the fix for) upstream-[workflow#2283](https://github.com/vercel/workflow/issues/2283), which made-duplicate processing of the _same_ `hook_created` idempotent. Here two-_different_ hooks reuse one token across a dispose that the engine reorders.--## Finding 3: cross-run token reclaim races dispose persistence (4/5)--Shape: run N claims token T (via `getConflict()`), receives one payload,-disposes, completes; run N+1 starts immediately and claims T. Five rounds per-test; failed 4 of 5 invocations, at varying rounds:--```text-run A: hook_created → hook_received → hook_disposed (all persisted, in order)-run B: hook_conflict token T conflictingRunId=<run A>-```--Run A's `hook_disposed` event exists, but run B's claim still conflicts-against A — the world's token-claim constraint is not released in the same-ordering domain as the disposal event, so a fast next claimant observes a-stale claim. This is the exact "fast descendant resumes" timing eve already-documents at `packages/eve/src/execution/workflow-entry.ts:162-166`.--## Paths forward--**Layer 1 (proceed now).** Durable `AbortController` in `turnWorkflow`, cancel-race, cancelled-turn finalizer, `turn.cancelled` → `session.waiting`. No-upstream dependency. Plug into the layer-0 seam (`TurnStepInput.abortSignal`).-Layer 1 creates its cancel observation hook once per turnWorkflow run with a-run-scoped token — no token reuse, so findings 2–3 do not apply.--**Layer 2 (two upstream-independent options, in preference order):**--1. **Indexed per-turn cancel tokens** — `${sessionId}:cancel:${turnIndex}`,- mirroring the existing turn-control token workaround. No token is ever- reused, sidestepping both repros entirely. The HTTP cancel route must- discover the active index; the driver already stamps `$eve.*` run- attributes, which can carry the live turn index/token.-2. **Barrier after dispose** — a `"use step"` suspension between `dispose()`- and the next same-token `createHook` flushes the disposal (verified 3/3- green in the spike's `interposeStep` variant). Works within one run;- does **not** help the cross-run race, so it only suits designs where one- long-lived run owns the token.--**Upstream (filed; do not block on them):**--1. [workflow#2777](https://github.com/vercel/workflow/issues/2777) — same-run- suspension flush must order `hook_disposed` ahead of subsequent same-token- `hook_created` validation (finding 2).-2. [workflow#2778](https://github.com/vercel/workflow/issues/2778) —- token-claim release must be atomic/ordered with `hook_disposed`- persistence so a new claimant cannot conflict against an already-disposed- hook (finding 3).--## Spike artifacts--Throwaway; not merged. If kept, convert findings 2–3 tests to `it.fails` so-they self-signal when upstream fixes land, and keep the layer-1 tests as-permanent regression coverage for the layer-1 PR.--- `packages/eve/src/internal/testing/cancellation-spike-workflow.ts`-- `packages/eve/src/internal/testing/cancellation-spike.integration.test.ts`