The Open Framework for Building Agents
feat(eve): cancellation - settle cancelled turns through the turn workflow (#573)
Andrew Barba159d674fd2f5
diff --git a/.changeset/turn-cancellation-turn-workflow.md b/.changeset/turn-cancellation-turn-workflow.mdnew file mode 100644index 000000000..e6ac4270a--- /dev/null+++ b/.changeset/turn-cancellation-turn-workflow.md@@ -0,0 +1,5 @@+---+"eve": patch+---++Turns are now cancellable: resuming a session's durable cancel hook (`{sessionId}:cancel`, with an optional `turnId` guard) aborts in-flight work and settles the turn as a new `turn.cancelled` stream event followed by `session.waiting` — never as a failure. Channels and stream-event hooks can handle `turn.cancelled`, and `eve/client` finalizes partially streamed messages. The HTTP cancellation API ships in a following release.diff --git a/docs/concepts/sessions-runs-and-streaming.md b/docs/concepts/sessions-runs-and-streaming.mdindex 4640a5641..c4095319e 100644--- a/docs/concepts/sessions-runs-and-streaming.md+++ b/docs/concepts/sessions-runs-and-streaming.md@@ -58,6 +58,7 @@ The stream is newline-delimited JSON (NDJSON), one event per line:| `step.failed` | A model step failed; carries `{ code, message, details? }`. || `turn.completed` | The turn finished. || `turn.failed` | The turn failed; carries `{ code, message, details? }`. |+| `turn.cancelled` | The turn was cancelled before finishing; always followed by `session.waiting`. || `session.waiting` | The session parked, waiting for the next input (a message, an answer). || `session.failed` | The session failed. || `session.completed` | The session reached a terminal end. |@@ -70,7 +71,7 @@ Note: consider the privacy, confidentiality, and user-experience implications foA delegated subagent publishes progress on its own child-session stream. The parent only emits `subagent.called` with a `childSessionId`, which a client uses to attach.-`step.failed` and `turn.failed` carry `{ code, message, details? }` for the failed fragment or turn, and `session.failed` is the terminal session-level variant. When a turn requested an output schema, the finalized payload lands on `result.completed` as `data.result` before the turn boundary. `authorization.required` carries the sign-in challenge (`data.authorization` may include `url`, `userCode`, `expiresAt`, `instructions`), and `authorization.completed` carries `data.outcome` (`"authorized" | "declined" | "failed" | "timed-out"`).+`step.failed` and `turn.failed` carry `{ code, message, details? }` for the failed fragment or turn, and `session.failed` is the terminal session-level variant. `turn.cancelled` is not a failure: the cancelled turn ends without any failure event, `session.waiting` follows, and the session accepts the next message normally — whatever the turn streamed before cancellation stays on the stream, while durable history keeps only what had already settled. When a turn requested an output schema, the finalized payload lands on `result.completed` as `data.result` before the turn boundary. `authorization.required` carries the sign-in challenge (`data.authorization` may include `url`, `userCode`, `expiresAt`, `instructions`), and `authorization.completed` carries `data.outcome` (`"authorized" | "declined" | "failed" | "timed-out"`).## Send a follow-up messagediff --git a/packages/eve/src/client/index.ts b/packages/eve/src/client/index.tsindex a56dee426..57bdad743 100644--- a/packages/eve/src/client/index.ts+++ b/packages/eve/src/client/index.ts@@ -111,6 +111,7 @@ export type {SubagentChildEventStreamEvent,SubagentCompletedStreamEvent,SubagentStartedStreamEvent,+ TurnCancelledStreamEvent,TurnCompletedStreamEvent,TurnFailedStreamEvent,TurnStartedStreamEvent,diff --git a/packages/eve/src/client/message-reducer.test.ts b/packages/eve/src/client/message-reducer.test.tsindex d3516300c..d8e3576a5 100644--- a/packages/eve/src/client/message-reducer.test.ts+++ b/packages/eve/src/client/message-reducer.test.ts@@ -9,9 +9,11 @@ import {createInputRequestedEvent,createMessageAppendedEvent,createMessageCompletedEvent,+ createReasoningAppendedEvent,createReasoningCompletedEvent,createResultCompletedEvent,createStepStartedEvent,+ createTurnCancelledEvent,} from "#protocol/message.js";describe("defaultMessageReducer", () => {@@ -649,6 +651,57 @@ describe("defaultMessageReducer", () => {]);});+ it("finalizes partial streamed message and reasoning when the turn is cancelled", () => {+ const reducer = defaultMessageReducer();+ let data = reducer.reduce(+ reducer.initial(),+ createReasoningAppendedEvent({+ reasoningDelta: "Thinking",+ reasoningSoFar: "Thinking",+ sequence: 0,+ stepIndex: 0,+ turnId: "turn_1",+ }),+ );+ data = reducer.reduce(+ data,+ createMessageAppendedEvent({+ messageDelta: "Partial",+ messageSoFar: "Partial",+ sequence: 1,+ stepIndex: 0,+ turnId: "turn_1",+ }),+ );+ data = reducer.reduce(data, createTurnCancelledEvent({ sequence: 2, turnId: "turn_1" }));++ expect(data.messages).toEqual([+ {+ id: "turn_1:assistant",+ metadata: {+ status: "complete",+ turnId: "turn_1",+ },+ parts: [+ { type: "step-start" },+ {+ state: "done",+ stepIndex: 0,+ text: "Thinking",+ type: "reasoning",+ },+ {+ state: "done",+ stepIndex: 0,+ text: "Partial",+ type: "text",+ },+ ],+ role: "assistant",+ },+ ]);+ });+it("removes streamed text for a null message completion", () => {const reducer = defaultMessageReducer();let data = reducer.reduce(diff --git a/packages/eve/src/client/message-reducer.ts b/packages/eve/src/client/message-reducer.tsindex 2d0be5fce..a5f5a6e75 100644--- a/packages/eve/src/client/message-reducer.ts+++ b/packages/eve/src/client/message-reducer.ts@@ -266,6 +266,19 @@ function reduceMessageData(data: EveMessageData, event: EveAgentReducerEvent): Ecase "turn.completed":return updateAssistantMetadata(data, event.data.turnId, { status: "complete" });+ case "turn.cancelled":+ // Finalize whatever the cancelled turn streamed: no message.completed+ // or reasoning.completed will follow a partial append.+ return updateAssistantMessage(data, event.data.turnId, (message) => ({+ ...message,+ metadata: { ...message.metadata, status: "complete" },+ parts: message.parts.map((part) =>+ (part.type === "text" || part.type === "reasoning") && part.state === "streaming"+ ? { ...part, state: "done" }+ : part,+ ),+ }));+case "turn.failed":case "session.failed":return data;diff --git a/packages/eve/src/execution/durable-session-migrations/turn-workflow.test.ts b/packages/eve/src/execution/durable-session-migrations/turn-workflow.test.tsindex b8e248e62..23f73b004 100644--- a/packages/eve/src/execution/durable-session-migrations/turn-workflow.test.ts+++ b/packages/eve/src/execution/durable-session-migrations/turn-workflow.test.ts@@ -50,7 +50,7 @@ describe("turn workflow wire migrations", () => {).toEqual({capabilities: undefined,completionToken: "turn-token",- driverCapabilities: { turnInbox: true },+ driverCapabilities: { cancelledTurnSettle: true, turnInbox: true },mode: "conversation",stepInput: {input: delivery,diff --git a/packages/eve/src/execution/durable-session-migrations/turn-workflow.ts b/packages/eve/src/execution/durable-session-migrations/turn-workflow.tsindex fdca10a6b..c5d546cf7 100644--- a/packages/eve/src/execution/durable-session-migrations/turn-workflow.ts+++ b/packages/eve/src/execution/durable-session-migrations/turn-workflow.ts@@ -38,6 +38,7 @@ export interface TurnWorkflowInput {*/readonly driverCapabilities?: {readonly turnInbox?: true;+ readonly cancelledTurnSettle?: true;};readonly mode: RunMode;readonly stepInput: TurnStepInput;@@ -59,7 +60,7 @@ export function createTurnWorkflowInput(input: TurnWorkflowDispatchInput): TurnWreturn {capabilities: input.capabilities,completionToken: input.completionToken,- driverCapabilities: { turnInbox: true },+ driverCapabilities: { cancelledTurnSettle: true, turnInbox: true },mode: input.mode,stepInput: {input: input.delivery,diff --git a/packages/eve/src/execution/next-driver-action.ts b/packages/eve/src/execution/next-driver-action.tsindex accaa1d04..9c2f84298 100644--- a/packages/eve/src/execution/next-driver-action.ts+++ b/packages/eve/src/execution/next-driver-action.ts@@ -30,6 +30,13 @@ export type NextDriverAction =readonly sessionState: DurableSessionState;readonly serializedContext: Record<string, unknown>;readonly authorizationNames?: readonly string[];+ /**+ * Set when the parked turn was cancelled: the driver runs+ * `settleCancelledTurnStep` before the normal park playbook. An+ * optional field rather than a new arm so pinned drivers keep+ * working.+ */+ readonly cancelled?: true;}| {readonly kind: "dispatch-runtime-actions";diff --git a/packages/eve/src/execution/settle-cancelled-turn-step.ts b/packages/eve/src/execution/settle-cancelled-turn-step.tsnew file mode 100644index 000000000..027bc0cd6--- /dev/null+++ b/packages/eve/src/execution/settle-cancelled-turn-step.ts@@ -0,0 +1,107 @@+import { buildAdapterContext } from "#channel/adapter-context.js";+import { callAdapterEventHandler } from "#channel/adapter.js";+import { dispatchStreamEventHooks } from "#context/hook-lifecycle.js";+import { withContextScope } from "#context/run-step.js";+import { deserializeContext, serializeContext } from "#context/serialize.js";+import { setChannelContext } from "#execution/channel-context.js";+import {+ createDurableSessionState,+ type DurableSessionState,+ readDurableSession,+} from "#execution/durable-session-store.js";+import { hydrateDurableSession } from "#execution/session.js";+import { reconcileSessionContinuationToken } from "#execution/reconcile-session-continuation-token.js";+import { emitCancelledTurn } from "#harness/cancelled-turn-emission.js";+import { getHarnessEmissionState, setHarnessEmissionState } from "#harness/emission.js";+import {+ clearAllProxyInputRequests,+ hasProxyInputRequests,+} from "#harness/proxy-input-requests.js";+import { clearPendingRuntimeActionBatch } from "#harness/runtime-actions.js";+import { clearPendingWorkflowInterrupt } from "#harness/workflow-interrupt-state.js";+import {+ encodeMessageStreamEvent,+ type HandleMessageStreamEvent,+ timestampHandleMessageStreamEvent,+} from "#protocol/message.js";+import { BundleKey, ChannelKey } from "#runtime/sessions/runtime-context-keys.js";++export interface CancelledTurnSettleResult {+ readonly serializedContext: Record<string, unknown>;+ readonly sessionState: DurableSessionState;+}++/**+ * Settles one cancelled turn: emits `turn.cancelled` → `session.waiting`,+ * drops pending runtime-action state, and persists the between-turns+ * session. Runs in the *driver* run, whose wake sources exclude the+ * cancel hook, so a queued cancel wake cannot re-dispatch it.+ */+export async function settleCancelledTurnStep(input: {+ readonly parentWritable: WritableStream<Uint8Array>;+ readonly serializedContext: Record<string, unknown>;+ readonly sessionState: DurableSessionState;+}): Promise<CancelledTurnSettleResult> {+ "use step";++ const durableSession = await readDurableSession(input.sessionState);+ const ctx = await deserializeContext(input.serializedContext);+ const adapter = ctx.require(ChannelKey);+ const adapterCtx = buildAdapterContext(adapter, ctx);+ const bundle = ctx.require(BundleKey);++ let session = hydrateDurableSession({+ compactionOverrides: {+ thresholdPercent: bundle.resolvedAgent.config.compaction?.thresholdPercent,+ },+ durable: durableSession,+ turnAgent: bundle.turnAgent,+ });++ let emissionState = getHarnessEmissionState(durableSession.state);+ // A descendant HITL wait already streamed this turn's waiting boundary+ // (the proxy epilogue clears the turn id); re-emitting would fabricate+ // a turn id and duplicate the boundary.+ const alreadyEpilogued =+ emissionState.turnId === "" && hasProxyInputRequests(durableSession.state);++ if (!alreadyEpilogued) {+ const writer = input.parentWritable.getWriter();+ try {+ const scoped = await withContextScope(ctx, session, async (enrichedSession) => {+ const emit = async (event: HandleMessageStreamEvent): Promise<void> => {+ const transformed = await callAdapterEventHandler(adapter, event, adapterCtx);+ setChannelContext(ctx, { ...adapter, state: { ...adapterCtx.state } });+ await writer.write(+ encodeMessageStreamEvent(timestampHandleMessageStreamEvent(transformed)),+ );+ await dispatchStreamEventHooks({+ ctx,+ event: transformed,+ registry: bundle.hookRegistry,+ });+ };+ return { result: await emitCancelledTurn(emit, emissionState), session: enrichedSession };+ });+ emissionState = scoped.result;+ session = scoped.session;+ } finally {+ writer.releaseLock();+ }+ }++ const cancelledSession = reconcileSessionContinuationToken(+ ctx,+ setHarnessEmissionState(+ clearAllProxyInputRequests(+ clearPendingWorkflowInterrupt(clearPendingRuntimeActionBatch(session)),+ ),+ emissionState,+ ),+ );++ return {+ serializedContext: serializeContext(ctx),+ sessionState: createDurableSessionState({ session: cancelledSession }),+ };+}diff --git a/packages/eve/src/execution/terminal-session-failure-step.ts b/packages/eve/src/execution/terminal-session-failure-step.tsnew file mode 100644index 000000000..c9a0e1080--- /dev/null+++ b/packages/eve/src/execution/terminal-session-failure-step.ts@@ -0,0 +1,72 @@+import { buildAdapterContext } from "#channel/adapter-context.js";+import { callAdapterEventHandler } from "#channel/adapter.js";+import { deserializeContext } from "#context/serialize.js";+import { createLogger, formatError } from "#internal/logging.js";+import {+ createSessionFailedEvent,+ encodeMessageStreamEvent,+ timestampHandleMessageStreamEvent,+} from "#protocol/message.js";+import { ChannelKey } from "#runtime/sessions/runtime-context-keys.js";++const log = createLogger("execution.workflow-entry");++/** Emits a terminal `session.failed` to the adapter and durable stream. */+export async function emitTerminalSessionFailureStep(input: {+ readonly error: unknown;+ readonly parentWritable: WritableStream<Uint8Array>;+ readonly serializedContext: Record<string, unknown>;+}): Promise<void> {+ "use step";++ const details = formatError(input.error);+ const code = typeof details.name === "string" ? details.name : "WORKFLOW_EXECUTION_FAILED";+ const message = typeof details.message === "string" ? details.message : String(input.error);+ const sessionId = (input.serializedContext["eve.sessionId"] as string | undefined) ?? "";++ log.error("workflow loop threw — emitting terminal session.failed", {+ sessionId,+ errorId: typeof details.errorId === "string" ? details.errorId : undefined,+ code,+ message,+ detail: typeof details.detail === "string" ? details.detail : undefined,+ });++ const event = createSessionFailedEvent({ code, details, message, sessionId });++ // Best-effort: invoke the adapter handler so channels surface the+ // failure. Errors are logged, never rethrown — the outer workflow+ // throw must still reach the run handle.+ try {+ const ctx = await deserializeContext(input.serializedContext);+ const adapter = ctx.get(ChannelKey);+ if (adapter !== undefined) {+ const adapterCtx = buildAdapterContext(adapter, ctx);+ await callAdapterEventHandler(adapter, event, adapterCtx);+ }+ } catch (notificationError) {+ log.error("adapter failed to handle terminal session.failed event", {+ errorId: typeof details.errorId === "string" ? details.errorId : undefined,+ sessionId,+ error: notificationError,+ });+ }++ // Always write the event to the durable stream so downstream+ // consumers see a canonical terminal event instead of an abrupt+ // stream close.+ try {+ const writer = input.parentWritable.getWriter();+ try {+ await writer.write(encodeMessageStreamEvent(timestampHandleMessageStreamEvent(event)));+ } finally {+ writer.releaseLock();+ }+ } catch (writeError) {+ log.error("failed to write terminal session.failed event to durable stream", {+ errorId: typeof details.errorId === "string" ? details.errorId : undefined,+ sessionId,+ error: writeError,+ });+ }+}diff --git a/packages/eve/src/execution/turn-cancellation-control.test.ts b/packages/eve/src/execution/turn-cancellation-control.test.tsnew file mode 100644index 000000000..879ea1950--- /dev/null+++ b/packages/eve/src/execution/turn-cancellation-control.test.ts@@ -0,0 +1,131 @@+import { afterEach, describe, expect, it, vi } from "vitest";++import {+ createTurnCancellationControl,+ sessionCancelHookToken,+} from "#execution/turn-cancellation-control.js";+import { TurnCancelledError } from "#harness/turn-cancellation.js";++const createHookMock = vi.fn();++vi.mock("#compiled/@workflow/core/index.js", () => ({+ createHook: (...args: unknown[]) => createHookMock(...args),+}));++function installCancelHook(options: {+ readonly conflict?: { readonly runId: string } | null;+ readonly payloads?: readonly unknown[];+}): { dispose: ReturnType<typeof vi.fn> } {+ const queue = [...(options.payloads ?? [])];+ const dispose = vi.fn();+ createHookMock.mockReturnValue({+ token: "session-1:cancel",+ getConflict: vi.fn(async () => options.conflict ?? null),+ dispose,+ [Symbol.asyncIterator](): AsyncIterator<unknown> {+ return {+ next: () => {+ const value = queue.shift();+ return value === undefined+ ? new Promise<IteratorResult<unknown>>(() => {})+ : Promise.resolve({ done: false, value });+ },+ return: vi.fn(async () => ({ done: true, value: undefined })),+ };+ },+ });+ return { dispose };+}++async function settles(promise: Promise<unknown>): Promise<boolean> {+ return await Promise.race([+ promise.then(() => true),+ new Promise<boolean>((resolve) => setTimeout(() => resolve(false), 25)),+ ]);+}++describe("sessionCancelHookToken", () => {+ it("derives the stable token from the session id", () => {+ expect(sessionCancelHookToken("wrun_abc")).toBe("wrun_abc:cancel");+ });+});++describe("createTurnCancellationControl", () => {+ afterEach(() => {+ createHookMock.mockReset();+ });++ it("returns undefined when the token is claimed by another run", async () => {+ installCancelHook({ conflict: { runId: "wrun_stale" } });++ const control = await createTurnCancellationControl({+ expectedTurnId: "turn_0",+ sessionId: "session-1",+ });++ expect(control).toBeUndefined();+ });++ it("aborts the turn signal on a cancel without a turn guard", async () => {+ installCancelHook({ payloads: [{}] });++ const control = await createTurnCancellationControl({+ expectedTurnId: "turn_0",+ sessionId: "session-1",+ });++ await expect(control!.requested).resolves.toBe("cancel");+ expect(control!.signal.aborted).toBe(true);+ expect(control!.signal.reason).toBeInstanceOf(TurnCancelledError);+ });++ it("aborts on a cancel whose guard matches the active turn", async () => {+ installCancelHook({ payloads: [{ turnId: "turn_2" }] });++ const control = await createTurnCancellationControl({+ expectedTurnId: "turn_2",+ sessionId: "session-1",+ });++ await expect(control!.requested).resolves.toBe("cancel");+ expect(control!.signal.aborted).toBe(true);+ });++ it("consumes a stale turn guard as a no-op and honors the next matching cancel", async () => {+ installCancelHook({ payloads: [{ turnId: "turn_99" }, { turnId: "turn_2" }] });++ const control = await createTurnCancellationControl({+ expectedTurnId: "turn_2",+ sessionId: "session-1",+ });++ await expect(control!.requested).resolves.toBe("cancel");+ expect(control!.signal.aborted).toBe(true);+ });++ it("never aborts when only mismatched guards arrive", async () => {+ installCancelHook({ payloads: [{ turnId: "turn_99" }] });++ const control = await createTurnCancellationControl({+ expectedTurnId: "turn_2",+ sessionId: "session-1",+ });++ expect(await settles(control!.requested)).toBe(false);+ expect(control!.signal.aborted).toBe(false);+ });++ it("disposes idempotently", async () => {+ const { dispose } = installCancelHook({});++ const control = await createTurnCancellationControl({+ expectedTurnId: "turn_0",+ sessionId: "session-1",+ });++ await control!.dispose();+ await control!.dispose();++ expect(dispose).toHaveBeenCalledTimes(1);+ });+});diff --git a/packages/eve/src/execution/turn-cancellation-control.ts b/packages/eve/src/execution/turn-cancellation-control.tsnew file mode 100644index 000000000..c7c438c57--- /dev/null+++ b/packages/eve/src/execution/turn-cancellation-control.ts@@ -0,0 +1,105 @@+import { createHook } from "#compiled/@workflow/core/index.js";++import { claimHookOwnership, disposeHook, isHookConflictError } from "#execution/hook-ownership.js";+import { TurnCancelledError } from "#harness/turn-cancellation.js";++/**+ * Derives the session-scoped cancel hook token. Stable for the session's+ * lifetime, so a cancel trigger can address it from the session id alone.+ */+export function sessionCancelHookToken(sessionId: string): string {+ return `${sessionId}:cancel`;+}++/**+ * Payload accepted by the session cancel hook. The optional `turnId`+ * guard scopes the cancel to the turn the caller observed; a mismatch is+ * consumed as a benign no-op. Omitting it cancels the current turn.+ */+export interface TurnCancelPayload {+ readonly turnId?: string;+}++/**+ * Owns one turn's cancellation surface inside the turn workflow: the+ * session-scoped cancel hook and the durable `AbortController` whose+ * signal is serialized into every `turnStep`. Must be created inside a+ * `"use workflow"` body.+ */+export interface TurnCancellationControl {+ /** Turn signal to serialize into each `turnStep` input. */+ readonly signal: AbortSignal;+ /**+ * Resolves `"cancel"` once a matching cancel payload is consumed and+ * the signal aborted. Race it against turn-owned awaits — never+ * `await` it alone.+ */+ readonly requested: Promise<"cancel">;+ /** Disposes the hook, abandoning any outstanding read. Idempotent. */+ dispose(): Promise<void>;+}++/**+ * Creates and claims the session cancel hook for one turn workflow run.+ * Returns `undefined` when the token is still claimed by a crashed prior+ * run — the turn then runs uncancellable rather than failing.+ */+export async function createTurnCancellationControl(input: {+ readonly expectedTurnId: string;+ readonly sessionId: string;+}): Promise<TurnCancellationControl | undefined> {+ const hook = createHook<TurnCancelPayload>({+ token: sessionCancelHookToken(input.sessionId),+ });+ // Hook promises and iterators share one durable cursor. Create the+ // iterator before claiming so conflict replay is consumed by+ // getConflict(), not a later iterator read.+ const iterator = hook[Symbol.asyncIterator]();++ try {+ await claimHookOwnership(hook);+ } catch (error) {+ if (isHookConflictError(error)) return undefined;+ throw error;+ }++ const controller = new AbortController();+ // The durable abort fires in the read's continuation so its call site+ // is reached deterministically on every replay.+ const requested = consumeMatchingCancel(iterator, input.expectedTurnId).then(() => {+ controller.abort(new TurnCancelledError());+ return "cancel" as const;+ });++ let disposed = false;+ return {+ signal: controller.signal,+ requested,+ async dispose(): Promise<void> {+ if (disposed) return;+ disposed = true;+ // Never `iterator.return()`: it would await the pending durable+ // read forever, leaving the run `running` and its hooks unswept.+ await disposeHook(hook);+ },+ };+}++// Mismatched turn guards are consumed as no-ops; each read is durable,+// so the skip sequence replays deterministically.+async function consumeMatchingCancel(+ iterator: AsyncIterator<TurnCancelPayload>,+ expectedTurnId: string,+): Promise<void> {+ while (true) {+ const next = await iterator.next();+ if (next.done) return await new Promise<never>(() => {});+ if (matchesActiveTurn(next.value, expectedTurnId)) return;+ }+}++function matchesActiveTurn(payload: unknown, expectedTurnId: string): boolean {+ if (typeof payload !== "object" || payload === null) return true;+ const guard = (payload as TurnCancelPayload).turnId;+ return guard === undefined || guard === expectedTurnId;+}diff --git a/packages/eve/src/execution/turn-cancellation.integration.test.ts b/packages/eve/src/execution/turn-cancellation.integration.test.tsnew file mode 100644index 000000000..762756740--- /dev/null+++ b/packages/eve/src/execution/turn-cancellation.integration.test.ts@@ -0,0 +1,574 @@+import { describe, expect, it } from "vitest";+import { getWorld, resumeHook, start } from "#internal/workflow/runtime.js";++import { createTestRuntime, type TestRuntime } from "#internal/testing/app-harness.js";+import {+ captureTurnEvents,+ containsEventSequence,+ filterEventsByType,+} from "#internal/testing/events.js";+import { waitForHook } from "#internal/testing/workflow-test-helpers.js";+import { ROOT_COMPILED_AGENT_NODE_ID } from "#compiler/manifest.js";+import { createBundledRuntimeCompiledArtifactsSource } from "#runtime/compiled-artifacts-source.js";+import { sessionCancelHookToken } from "#execution/turn-cancellation-control.js";+import { workflowEntry } from "#execution/workflow-entry.js";+import type { HandleMessageStreamEvent } from "#protocol/message.js";+import type { ToolContext } from "#public/definitions/tool.js";+import type { ResolvedToolDefinition } from "#runtime/types.js";++/**+ * Layer-1 turn cancellation: resuming a session's `{sessionId}:cancel`+ * hook mid-turn settles the turn as `turn.cancelled` → `session.waiting`+ * with zero failure events, no step retries, and a session that accepts+ * the next message normally. The payload may carry a `turnId` guard+ * scoping the cancel to one observed turn. (The HTTP trigger arrives in+ * layer 2; these tests resume the hook directly.)+ */++const FAILURE_EVENT_TYPES = ["step.failed", "turn.failed", "session.failed"] as const;+const WAIT_TOOL_NAME = "wait_for_cancel";++function buildSerializedContext(overrides: {+ channelKind: string;+ continuationToken: string;+ mode: string;+}): Record<string, unknown> {+ return {+ "eve.auth": null,+ "eve.bundle": { source: createBundledRuntimeCompiledArtifactsSource() },+ "eve.channel": { kind: overrides.channelKind, state: {} },+ "eve.continuationToken": overrides.continuationToken,+ "eve.mode": overrides.mode,+ };+}++/**+ * Builds an authored tool that hangs until the layer-0 turn signal+ * aborts, then rejects with the signal's reason — the deterministic+ * mid-turn anchor for cancellation tests.+ */+function buildWaitForCancelTool(onStart: () => void, onAbort: () => void): ResolvedToolDefinition {+ return {+ description: "Waits until the turn is cancelled.",+ execute: (_input: unknown, rawCtx: unknown) => {+ const ctx = rawCtx as ToolContext;+ onStart();+ return new Promise((_resolve, reject) => {+ const abort = (): void => {+ onAbort();+ reject(ctx.abortSignal.reason);+ };+ if (ctx.abortSignal.aborted) {+ abort();+ return;+ }+ ctx.abortSignal.addEventListener("abort", abort, { once: true });+ });+ },+ inputSchema: { additionalProperties: false, properties: {}, type: "object" },+ logicalPath: `tools/${WAIT_TOOL_NAME}.ts`,+ name: WAIT_TOOL_NAME,+ sourceId: `tools/${WAIT_TOOL_NAME}.ts`,+ sourceKind: "module",+ };+}++interface WaitToolFixture {+ readonly runtime: TestRuntime;+ readonly toolStarted: Promise<void>;+ toolAborts(): number;+ toolStarts(): number;+}++function createWaitToolRuntime(agentName: string): WaitToolFixture {+ let aborts = 0;+ let starts = 0;+ let resolveStarted: (() => void) | undefined;+ const toolStarted = new Promise<void>((resolve) => {+ resolveStarted = resolve;+ });+ const waitTool = buildWaitForCancelTool(+ () => {+ starts += 1;+ resolveStarted?.();+ },+ () => {+ aborts += 1;+ },+ );+ const runtime = createTestRuntime({ agent: { name: agentName }, tools: [waitTool] });+ const manifestTool = runtime.manifest.tools.find((tool) => tool.name === WAIT_TOOL_NAME);+ if (manifestTool === undefined) {+ throw new Error(`Expected ${WAIT_TOOL_NAME} to be present in the test manifest.`);+ }+ runtime.moduleMap.nodes[ROOT_COMPILED_AGENT_NODE_ID]!.modules[manifestTool.sourceId] = {+ default: { execute: waitTool.execute },+ };+ return { runtime, toolStarted, toolAborts: () => aborts, toolStarts: () => starts };+}++/** Polls the world until the given run reaches `completed`. */+async function waitForRunCompletion(runId: string, timeout = 15_000): Promise<void> {+ const world = await getWorld();+ const deadline = Date.now() + timeout;+ while (Date.now() < deadline) {+ const page = await world.runs.list({ pagination: { limit: 100 } });+ const row = page.data.find((entry: { runId?: string }) => entry.runId === runId);+ if (row?.status === "completed") return;+ await new Promise((resolve) => setTimeout(resolve, 50));+ }+ throw new Error(`Timed out waiting for run "${runId}" to complete.`);+}++/** Polls the world for a hook row by token (hooks are per-run; the token is global). */+async function waitForHookByToken(token: string, timeout = 15_000): Promise<{ runId: string }> {+ const world = await getWorld();+ const deadline = Date.now() + timeout;+ while (Date.now() < deadline) {+ try {+ const hook = await world.hooks.getByToken(token);+ if (hook !== null && hook !== undefined) {+ return hook;+ }+ } catch {+ // Not registered yet.+ }+ await new Promise((resolve) => setTimeout(resolve, 50));+ }+ throw new Error(`Timed out waiting for hook token "${token}".`);+}++/**+ * The retry canary: an aborted `turnStep` settles by *returning*, so the+ * turn workflow run must record no `step_failed`/`step_retrying` events+ * (nothing thrown ever crosses the step boundary) and at most one+ * `step_completed` per correlation id. Duplicate `step_started` entries+ * are allowed: the runtime may supersede an aborted attempt and+ * re-dispatch the step under the same correlation id — the entry abort+ * check makes the superseding attempt side-effect free.+ */+async function expectNoStepRetries(runId: string): Promise<void> {+ const world = await getWorld();+ const completions = new Map<string, number>();+ const failureEvents: string[] = [];+ let cursor: string | undefined;++ do {+ const pagination: { cursor?: string; limit: number } = { limit: 1000 };+ if (cursor !== undefined) {+ pagination.cursor = cursor;+ }+ const page = await world.events.list({ pagination, resolveData: "none", runId });+ const events: readonly { correlationId?: string | null; eventType?: string }[] = page.data;+ for (const event of events) {+ if (event.eventType === "step_failed" || event.eventType === "step_retrying") {+ failureEvents.push(`${event.eventType}:${String(event.correlationId ?? "?")}`);+ }+ if (event.eventType === "step_completed") {+ const correlationId = String(event.correlationId ?? "?");+ completions.set(correlationId, (completions.get(correlationId) ?? 0) + 1);+ }+ }+ cursor = page.hasMore === true && page.cursor !== null ? page.cursor : undefined;+ } while (cursor !== undefined);++ expect(failureEvents).toEqual([]);+ expect([...completions.entries()].filter(([, count]) => count > 1)).toEqual([]);+}++function expectNoFailureEvents(events: readonly HandleMessageStreamEvent[]): void {+ const types = events.map((event) => event.type);+ for (const failureType of FAILURE_EVENT_TYPES) {+ expect(types).not.toContain(failureType);+ }+}++describe("turn cancellation integration", () => {+ it("cancels a turn mid-tool and accepts the next message normally", async () => {+ const fixture = createWaitToolRuntime("turn-cancel-tool");+ const continuationToken = "http:turn-cancel-tool";++ await fixture.runtime.run(async () => {+ const run = await start(workflowEntry, [+ {+ input: { message: `Use the ${WAIT_TOOL_NAME} tool.` },+ serializedContext: buildSerializedContext({+ channelKind: "http",+ continuationToken,+ mode: "conversation",+ }),+ },+ ]);+ const stream = captureTurnEvents(run);++ try {+ const cancelToken = sessionCancelHookToken(run.runId);+ const cancelHook = await waitForHookByToken(cancelToken);+ await fixture.toolStarted;+ // A matching turn guard cancels the observed turn (the first+ // turn's id is `turn_0`).+ await resumeHook(cancelToken, { turnId: "turn_0" });++ const cancelledTurn = await stream.nextTurn();++ // A duplicate cancel after the turn settled lands on a+ // consumed/disposed hook and must not disturb the session.+ await resumeHook(cancelToken, {}).catch(() => undefined);++ expect(cancelledTurn.at(-1)?.type).toBe("session.waiting");+ expect(+ containsEventSequence(cancelledTurn, [+ "turn.started",+ "turn.cancelled",+ "session.waiting",+ ]),+ ).toBe(true);+ expect(filterEventsByType(cancelledTurn, "turn.started")).toHaveLength(1);+ expect(filterEventsByType(cancelledTurn, "turn.cancelled")).toHaveLength(1);+ // The superseding step attempt settles before any model work, so+ // the cancelled turn streams exactly one step.+ expect(filterEventsByType(cancelledTurn, "step.started")).toHaveLength(1);+ expectNoFailureEvents(cancelledTurn);+ expect(fixture.toolStarts()).toBe(1);++ await expectNoStepRetries(cancelHook.runId);++ await waitForHook({ runId: run.runId }, { token: continuationToken });+ await resumeHook(continuationToken, {+ kind: "deliver",+ payloads: [{ message: "follow up after cancel" }],+ });++ const followUpTurn = await stream.nextTurn();++ expect(followUpTurn.at(-1)?.type).toBe("session.waiting");+ expect(filterEventsByType(followUpTurn, "turn.cancelled")).toHaveLength(0);+ expectNoFailureEvents(followUpTurn);+ expect(+ followUpTurn.some(+ (event) =>+ event.type === "message.completed" &&+ event.data.message?.includes("follow up after cancel") === true,+ ),+ ).toBe(true);+ } finally {+ stream.dispose();+ await run.cancel();+ }+ });+ });++ it("cancels a turn waiting on an in-flight subagent and does not re-dispatch it", async () => {+ const fixture = createWaitToolRuntime("turn-cancel-subagent");+ const continuationToken = "http:turn-cancel-subagent";++ await fixture.runtime.run(async () => {+ const run = await start(workflowEntry, [+ {+ input: { message: `Delegate to a subagent: use the ${WAIT_TOOL_NAME} tool.` },+ serializedContext: buildSerializedContext({+ channelKind: "http",+ continuationToken,+ mode: "conversation",+ }),+ },+ ]);+ const stream = captureTurnEvents(run);++ try {+ // The child (a fresh copy of the same agent) hangs on the wait+ // tool, holding the parent in `waitForRuntimeActionResults`.+ await fixture.toolStarted;++ const cancelToken = sessionCancelHookToken(run.runId);+ await waitForHookByToken(cancelToken);+ await resumeHook(cancelToken, {});++ const cancelledTurn = await stream.nextTurn();++ expect(cancelledTurn.at(-1)?.type).toBe("session.waiting");+ expect(filterEventsByType(cancelledTurn, "turn.cancelled")).toHaveLength(1);+ expect(filterEventsByType(cancelledTurn, "subagent.called")).toHaveLength(1);+ expectNoFailureEvents(cancelledTurn);++ // Unblock the orphaned child (no cascade in layer 1 — its late+ // result lands on the parent's disposed inbox and is dropped).+ const childSessionId = filterEventsByType(cancelledTurn, "subagent.called")[0]?.data+ .childSessionId;+ expect(childSessionId).toBeDefined();+ const childCancelToken = sessionCancelHookToken(childSessionId ?? "");+ await waitForHookByToken(childCancelToken);+ await resumeHook(childCancelToken, {}).catch(() => undefined);++ // The cleared pending batch must not re-dispatch on the next turn.+ await waitForHook({ runId: run.runId }, { token: continuationToken });+ await resumeHook(continuationToken, {+ kind: "deliver",+ payloads: [{ message: "follow up after subagent cancel" }],+ });++ const followUpTurn = await stream.nextTurn();++ expect(followUpTurn.at(-1)?.type).toBe("session.waiting");+ expect(filterEventsByType(followUpTurn, "subagent.called")).toHaveLength(0);+ expect(filterEventsByType(followUpTurn, "turn.cancelled")).toHaveLength(0);+ expectNoFailureEvents(followUpTurn);+ expect(+ followUpTurn.some(+ (event) =>+ event.type === "message.completed" &&+ event.data.message?.includes("follow up after subagent cancel") === true,+ ),+ ).toBe(true);+ } finally {+ stream.dispose();+ await run.cancel();+ }+ });+ }, 60_000);++ it("cancels a turn parked on a child HITL request without corrupting the stream", async () => {+ const runtime = createTestRuntime({ agent: { name: "turn-cancel-hitl" } });+ const continuationToken = "http:turn-cancel-hitl";++ await runtime.run(async () => {+ const run = await start(workflowEntry, [+ {+ input: { message: "Delegate to a subagent: Use the ask_question tool exactly once." },+ serializedContext: {+ ...buildSerializedContext({+ channelKind: "http",+ continuationToken,+ mode: "conversation",+ }),+ "eve.capabilities": { requestInput: true },+ },+ },+ ]);+ const stream = captureTurnEvents(run);++ try {+ // The child asks a question; the proxy epilogue emits this turn's+ // waiting boundary while the parent keeps waiting on the child.+ const hitlTurn = await stream.nextTurn();+ expect(hitlTurn.at(-1)?.type).toBe("session.waiting");+ const requested = filterEventsByType(hitlTurn, "input.requested");+ expect(requested).toHaveLength(1);+ const requestId = requested[0]?.data.requests[0]?.requestId;+ expect(requestId).toBeDefined();+ const childSessionId = filterEventsByType(hitlTurn, "subagent.called")[0]?.data+ .childSessionId;+ expect(childSessionId).toBeDefined();++ const cancelToken = sessionCancelHookToken(run.runId);+ const cancelHook = await waitForHookByToken(cancelToken);+ await resumeHook(cancelToken, {});+ // Barrier: the answer must not race the cancel — a delivery that+ // beats the cancel is legitimately routed to the still-live child.+ await waitForRunCompletion(cancelHook.runId);++ // The boundary is already on the stream: settling must not emit a+ // fabricated turn.cancelled or a second session.waiting.+ const world = await getWorld();+ const answer = {+ kind: "deliver",+ payloads: [+ {+ inputResponses: [{ requestId: requestId ?? "", text: "blue" }],+ message: "answer after hitl cancel",+ },+ ],+ };+ await waitForHook({ runId: run.runId }, { token: continuationToken });+ await resumeHook(continuationToken, answer);++ const followUpTurn = await stream.nextTurn();++ expect(followUpTurn.at(-1)?.type).toBe("session.waiting");+ expect(filterEventsByType(followUpTurn, "turn.cancelled")).toHaveLength(0);+ expect(filterEventsByType(followUpTurn, "turn.started")).toHaveLength(1);+ expect(filterEventsByType(followUpTurn, "session.waiting")).toHaveLength(1);+ expectNoFailureEvents(followUpTurn);+ expect(+ followUpTurn.some(+ (event) =>+ event.type === "message.completed" &&+ event.data.message?.includes("answer after hitl cancel") === true,+ ),+ ).toBe(true);++ // The cleared proxy map must not route the answer to the orphaned+ // child: the child never starts a second turn.+ const childSecondTurnInbox = `${childSessionId ?? ""}:turn-control:1:inbox`;+ const deadline = Date.now() + 1_500;+ while (Date.now() < deadline) {+ const hook = await world.hooks.getByToken(childSecondTurnInbox).catch(() => null);+ expect(hook ?? null).toBeNull();+ await new Promise((resolve) => setTimeout(resolve, 250));+ }+ } finally {+ stream.dispose();+ await run.cancel();+ }+ });+ }, 60_000);++ it("consumes a cancel with a stale turn guard as a no-op and keeps the turn running", async () => {+ const fixture = createWaitToolRuntime("turn-cancel-stale-guard");+ const continuationToken = "http:turn-cancel-stale-guard";++ await fixture.runtime.run(async () => {+ const run = await start(workflowEntry, [+ {+ input: { message: `Use the ${WAIT_TOOL_NAME} tool.` },+ serializedContext: buildSerializedContext({+ channelKind: "http",+ continuationToken,+ mode: "conversation",+ }),+ },+ ]);+ const stream = captureTurnEvents(run);++ try {+ const cancelToken = sessionCancelHookToken(run.runId);+ await waitForHookByToken(cancelToken);+ await fixture.toolStarted;++ // A guard naming a turn the session has never run: the payload is+ // consumed as a no-op — the caller's cancel must never leak onto+ // a turn they did not observe.+ await resumeHook(cancelToken, { turnId: "turn_99" });++ // The turn must still be cancellable afterwards: the skip loop+ // re-arms the durable read rather than consuming the hook.+ await new Promise((resolve) => setTimeout(resolve, 250));+ expect(fixture.toolAborts()).toBe(0);++ await resumeHook(cancelToken, { turnId: "turn_0" });++ const cancelledTurn = await stream.nextTurn();++ expect(cancelledTurn.at(-1)?.type).toBe("session.waiting");+ expect(filterEventsByType(cancelledTurn, "turn.cancelled")).toHaveLength(1);+ expectNoFailureEvents(cancelledTurn);+ expect(fixture.toolAborts()).toBe(1);+ } finally {+ stream.dispose();+ await run.cancel();+ }+ });+ }, 60_000);++ it("treats a cancel after the turn settled as a benign no-op", async () => {+ const runtime = createTestRuntime({ agent: { name: "turn-cancel-late" } });+ const continuationToken = "http:turn-cancel-late";++ await runtime.run(async () => {+ const run = await start(workflowEntry, [+ {+ input: { message: "hello there" },+ serializedContext: buildSerializedContext({+ channelKind: "http",+ continuationToken,+ mode: "conversation",+ }),+ },+ ]);+ const stream = captureTurnEvents(run);++ try {+ const firstTurn = await stream.nextTurn();+ expect(firstTurn.at(-1)?.type).toBe("session.waiting");+ expect(filterEventsByType(firstTurn, "turn.completed")).toHaveLength(1);++ // The turn workflow has settled and disposed its cancel hook; a+ // late cancel either rejects (hook gone) or lands unconsumed.+ await resumeHook(sessionCancelHookToken(run.runId), {}).catch(() => undefined);++ await waitForHook({ runId: run.runId }, { token: continuationToken });+ await resumeHook(continuationToken, {+ kind: "deliver",+ payloads: [{ message: "follow up after late cancel" }],+ });++ const secondTurn = await stream.nextTurn();++ expect(secondTurn.at(-1)?.type).toBe("session.waiting");+ expect(filterEventsByType(secondTurn, "turn.cancelled")).toHaveLength(0);+ expectNoFailureEvents(secondTurn);+ expect(+ secondTurn.some(+ (event) =>+ event.type === "message.completed" &&+ event.data.message?.includes("follow up after late cancel") === true,+ ),+ ).toBe(true);+ } finally {+ stream.dispose();+ await run.cancel();+ }+ });+ });++ it("completes settled turn runs so the world sweeps their hooks", async () => {+ const runtime = createTestRuntime({ agent: { name: "turn-cancel-sweep" } });+ const continuationToken = "http:turn-cancel-sweep";++ await runtime.run(async () => {+ const run = await start(workflowEntry, [+ {+ input: { message: "first turn" },+ serializedContext: buildSerializedContext({+ channelKind: "http",+ continuationToken,+ mode: "conversation",+ }),+ },+ ]);+ const stream = captureTurnEvents(run);++ try {+ expect((await stream.nextTurn()).at(-1)?.type).toBe("session.waiting");++ await waitForHook({ runId: run.runId }, { token: continuationToken });+ await resumeHook(continuationToken, {+ kind: "deliver",+ payloads: [{ message: "second turn" }],+ });+ expect((await stream.nextTurn()).at(-1)?.type).toBe("session.waiting");++ // Teardown must not await the cancel hook's outstanding read: a+ // turn run that never returns stays `running` forever and its+ // hooks are never swept. With the session parked, no cancel hook+ // may remain in the world.+ const world = await getWorld();+ const deadline = Date.now() + 15_000;+ let completedTurnRuns = 0;+ let cancelHooks = 0;+ while (Date.now() < deadline) {+ const cancelToken0 = sessionCancelHookToken(run.runId);+ const cancelHook0 = await world.hooks.getByToken(cancelToken0).catch(() => null);+ cancelHooks = cancelHook0 === null ? 0 : 1;++ const runsPage = await world.runs.list({ pagination: { limit: 100 } });+ completedTurnRuns = runsPage.data.filter(+ (row: { status?: string; workflowName?: string }) =>+ row.workflowName?.includes("turnWorkflow") === true && row.status === "completed",+ ).length;++ if (completedTurnRuns >= 1 && cancelHooks === 0) break;+ await new Promise((resolve) => setTimeout(resolve, 100));+ }++ // The first turn settled a full turn ago: its run must have+ // completed and its cancel hook must be gone from the world.+ expect(completedTurnRuns).toBeGreaterThanOrEqual(1);+ expect(cancelHooks).toBe(0);+ } finally {+ stream.dispose();+ await run.cancel();+ }+ });+ }, 60_000);+});diff --git a/packages/eve/src/execution/turn-dispatch.test.ts b/packages/eve/src/execution/turn-dispatch.test.tsindex 45c6bea15..5a20d8cb3 100644--- a/packages/eve/src/execution/turn-dispatch.test.ts+++ b/packages/eve/src/execution/turn-dispatch.test.ts@@ -170,7 +170,7 @@ describe("dispatchAndAwaitTurn", () => {]);const bufferedDeliveries: DeliverHookPayload[] = [delivery];- const action = await dispatchAndAwaitTurn({+ const turn = await dispatchAndAwaitTurn({bufferedDeliveries,controlToken: "turn-control",delivery: { kind: "deliver", payloads: [{ message: "start" }] },@@ -182,9 +182,37 @@ describe("dispatchAndAwaitTurn", () => {});expect(forwardTurnDeliveryStep).toHaveBeenCalledOnce();- expect(action.kind).toBe("park");+ expect(turn.action.kind).toBe("park");expect(bufferedDeliveries).toEqual([delivery]);});++ it("defers control-hook disposal until the caller invokes dispose()", async () => {+ const state = createState("http:test");+ installControlHook([+ {+ action: { kind: "park", serializedContext: {}, sessionState: state },+ kind: "turn-result",+ },+ ]);++ const turn = await dispatchAndAwaitTurn({+ bufferedDeliveries: [],+ controlToken: "turn-control",+ delivery: { kind: "deliver", payloads: [{ message: "start" }] },+ deliveryHook: createDeliveryHook(),+ mode: "conversation",+ parentWritable: new WritableStream<Uint8Array>(),+ serializedContext: {},+ sessionState: state,+ });++ // The turn run's final control send is at-least-once; a late+ // duplicate resume must land on a live hook (see DispatchedTurn).+ const hook = createHookMock.mock.results[0]?.value as { dispose: ReturnType<typeof vi.fn> };+ expect(hook.dispose).not.toHaveBeenCalled();+ await turn.dispose();+ expect(hook.dispose).toHaveBeenCalledOnce();+ });});function createDeliveryHook(overrides: Partial<SessionDeliveryHook> = {}): SessionDeliveryHook {diff --git a/packages/eve/src/execution/turn-dispatch.ts b/packages/eve/src/execution/turn-dispatch.tsindex b2aa35d59..dc69fc771 100644--- a/packages/eve/src/execution/turn-dispatch.ts+++ b/packages/eve/src/execution/turn-dispatch.ts@@ -6,6 +6,20 @@ import type { SessionDeliveryHook } from "#execution/session-delivery-hook.js";import { dispatchTurnStep } from "#execution/workflow-steps.js";import type { RunMode } from "#shared/run-mode.js";+/** One settled turn: its terminal driver action plus deferred hook cleanup. */+export interface DispatchedTurn {+ readonly action: NextDriverAction;+ /**+ * Disposes the turn's control hook. Deferred until the *next* turn+ * settles (or the session ends): the turn run's final control send is+ * at-least-once, and `sendTurnControlStep` does not treat+ * `HookNotFoundError` as benign, so a late duplicate resume must land+ * on a live hook. By the next settle, the previous run has completed+ * and can no longer re-send.+ */+ dispose(): Promise<void>;+}+/** Dispatches one turn and services its private-inbox control protocol until it terminates. */export async function dispatchAndAwaitTurn(input: {readonly bufferedDeliveries: DeliverHookPayload[];@@ -17,7 +31,7 @@ export async function dispatchAndAwaitTurn(input: {readonly parentWritable: WritableStream<Uint8Array>;readonly serializedContext: Record<string, unknown>;readonly sessionState: DurableSessionState;-}): Promise<NextDriverAction> {+}): Promise<DispatchedTurn> {const control = new TurnControlReceiver({bufferedDeliveries: input.bufferedDeliveries,deliveryHook: input.deliveryHook,@@ -35,8 +49,10 @@ export async function dispatchAndAwaitTurn(input: {sessionState: input.sessionState,});- return await control.waitForAction();- } finally {+ const action = await control.waitForAction();+ return { action, dispose: () => control.dispose() };+ } catch (error) {await control.dispose();+ throw error;}}diff --git a/packages/eve/src/execution/turn-execution-cursor.ts b/packages/eve/src/execution/turn-execution-cursor.tsindex 935796e1d..bcb40ac87 100644--- a/packages/eve/src/execution/turn-execution-cursor.ts+++ b/packages/eve/src/execution/turn-execution-cursor.ts@@ -19,6 +19,7 @@ type TurnTerminalAction =}| {readonly authorizationNames?: readonly string[];+ readonly cancelled?: true;readonly kind: "park";};@@ -66,8 +67,9 @@ export class TurnExecutionCursor {}/** Builds the next atomic turn-step input from the cursor's current state. */- createStepInput(input: HookPayload | undefined): TurnStepInput {+ createStepInput(input: HookPayload | undefined, abortSignal?: AbortSignal): TurnStepInput {return {+ abortSignal,input,parentWritable: this.parentWritable,serializedContext: this.currentSerializedContext,diff --git a/packages/eve/src/execution/turn-workflow.test.ts b/packages/eve/src/execution/turn-workflow.test.tsindex feba1d975..388c7f63a 100644--- a/packages/eve/src/execution/turn-workflow.test.ts+++ b/packages/eve/src/execution/turn-workflow.test.ts@@ -252,13 +252,141 @@ describe("turnWorkflow", () => {});});+ it("reports a cancelled turn as a park with the cancelled marker", async () => {+ const sessionState = createSessionState();+ installInbox([]);+ vi.mocked(turnStep).mockResolvedValueOnce({+ action: "cancelled",+ serializedContext: { state: "cancelled" },+ sessionState,+ });++ // Task mode on purpose: cancellation bypasses the `canPark` gate.+ const { input } = createInput({+ driverCapabilities: { cancelledTurnSettle: true, turnInbox: true },+ mode: "task",+ sessionState,+ });+ await turnWorkflow(input);++ expect(vi.mocked(turnStep).mock.calls[0]?.[0].abortSignal).toBeInstanceOf(AbortSignal);+ // The cancel token is session-scoped so a trigger can derive it from+ // the session id alone.+ expect(cancelHookTokens()).toEqual(["wrun_test_123:cancel"]);+ // The cancelled result is a pure marker: the control payload carries+ // the cursor's last settled state, not the aborted step's echo.+ expect(resumeHookMock).toHaveBeenCalledWith("turn-token", {+ action: {+ cancelled: true,+ kind: "park",+ serializedContext: { state: "start" },+ sessionState,+ },+ kind: "turn-result",+ });+ expect(resumeHookMock.mock.calls.filter((call) => call[1]?.kind === "turn-error")).toEqual([]);+ });++ it("runs uncancellable when the session cancel token is claimed by another run", async () => {+ const sessionState = createSessionState();+ installInbox([], { cancelConflict: { runId: "wrun_stale_prior_turn" } });+ vi.mocked(turnStep).mockResolvedValueOnce({+ action: "done",+ output: "ok",+ serializedContext: { state: "done" },+ sessionState,+ });++ const { input } = createInput({+ driverCapabilities: { cancelledTurnSettle: true, turnInbox: true },+ sessionState,+ });+ await turnWorkflow(input);++ // The stale claim degrades cancellation instead of failing the turn.+ expect(vi.mocked(turnStep).mock.calls[0]?.[0].abortSignal).toBeUndefined();+ expect(resumeHookMock).toHaveBeenCalledWith(+ "turn-token",+ expect.objectContaining({ kind: "turn-result" }),+ );+ expect(resumeHookMock.mock.calls.filter((call) => call[1]?.kind === "turn-error")).toEqual([]);+ });++ it("disposes the session cancel hook before publishing the turn result", async () => {+ const sessionState = createSessionState();+ installInbox([]);+ vi.mocked(turnStep).mockResolvedValueOnce({+ action: "done",+ output: "ok",+ serializedContext: { state: "done" },+ sessionState,+ });++ const { input } = createInput({+ driverCapabilities: { cancelledTurnSettle: true, turnInbox: true },+ sessionState,+ });+ await turnWorkflow(input);++ // The token is stable across turns: the next turn's claim must never+ // race this run's teardown, so disposal precedes the terminal send.+ const cancelHook = createHookMock.mock.results.find(+ (result) => (result.value as { token?: string }).token === "wrun_test_123:cancel",+ )?.value as { dispose: ReturnType<typeof vi.fn> };+ const resultCall = resumeHookMock.mock.calls.findIndex(+ (call) => call[1]?.kind === "turn-result",+ );+ expect(cancelHook.dispose).toHaveBeenCalled();+ expect(cancelHook.dispose.mock.invocationCallOrder[0]).toBeLessThan(+ resumeHookMock.mock.invocationCallOrder[resultCall]!,+ );+ });++ it("registers no cancel hook when the driver cannot settle cancelled parks", async () => {+ const sessionState = createSessionState();+ installInbox([]);+ vi.mocked(turnStep).mockResolvedValueOnce({+ action: "done",+ output: "ok",+ serializedContext: { state: "done" },+ sessionState,+ });++ const { input } = createInput({ driverCapabilities: { turnInbox: true }, sessionState });+ await turnWorkflow(input);++ expect(vi.mocked(turnStep).mock.calls[0]?.[0].abortSignal).toBeUndefined();+ expect(cancelHookTokens()).toEqual([]);+ });++ it("registers no cancel hook when the session cannot park", async () => {+ const sessionState = createSessionState({ continuationToken: "" });+ installInbox([]);+ vi.mocked(turnStep).mockResolvedValueOnce({+ action: "done",+ output: "ok",+ serializedContext: { state: "done" },+ sessionState,+ });++ const { input } = createInput({+ driverCapabilities: { cancelledTurnSettle: true, turnInbox: true },+ mode: "task",+ sessionState,+ });+ await turnWorkflow(input);++ expect(vi.mocked(turnStep).mock.calls[0]?.[0].abortSignal).toBeUndefined();+ expect(cancelHookTokens()).toEqual([]);+ });+it("deduplicates concurrent turn workflows through inbox ownership", async () => {const sessionState = createSessionState();const ownerInbox = createInboxMock([]);const duplicateInbox = createInboxMock([], {conflict: { runId: "wrun_owner" },});- createHookMock.mockReturnValueOnce(ownerInbox.hook).mockReturnValueOnce(duplicateInbox.hook);+ installHookDispatch([ownerInbox, duplicateInbox]);vi.mocked(turnStep).mockResolvedValueOnce({action: "done",output: "ok",@@ -754,15 +882,73 @@ interface InboxMock {function installInbox(values: readonly unknown[],options: {+ readonly cancelConflict?: { readonly runId: string } | null;+ readonly cancelPayloads?: readonly unknown[];readonly claimError?: unknown;readonly conflict?: { readonly runId: string } | null;} = {},): InboxMock {const inbox = createInboxMock(values, options);- createHookMock.mockReturnValue(inbox.hook);+ installHookDispatch([inbox], {+ conflict: options.cancelConflict ?? null,+ payloads: options.cancelPayloads ?? [],+ });return inbox;}+/**+ * Routes inbox tokens to the queued inbox mocks and `:cancel` tokens to+ * cancel hooks (inert by default — their reads never resolve, so no+ * cancellation is ever observed unless a test provides payloads or a+ * claim conflict via `cancelOptions`).+ */+function installHookDispatch(+ inboxes: readonly InboxMock[],+ cancelOptions: {+ readonly conflict?: { readonly runId: string } | null;+ readonly payloads?: readonly unknown[];+ } = {},+): void {+ const queue = [...inboxes];+ createHookMock.mockImplementation((input: { token: string }) =>+ input.token.endsWith(":cancel")+ ? createCancelHookMock(input.token, cancelOptions)+ : queue.shift()?.hook,+ );+}++function cancelHookTokens(): string[] {+ return createHookMock.mock.calls+ .map((call) => (call[0] as { token: string }).token)+ .filter((token) => token.endsWith(":cancel"));+}++function createCancelHookMock(+ token: string,+ options: {+ readonly conflict?: { readonly runId: string } | null;+ readonly payloads?: readonly unknown[];+ } = {},+): unknown {+ const queue = [...(options.payloads ?? [])];+ return {+ token,+ getConflict: vi.fn(async () => options.conflict ?? null),+ dispose: vi.fn(),+ [Symbol.asyncIterator](): AsyncIterator<unknown> {+ return {+ next: () => {+ const value = queue.shift();+ return value === undefined+ ? new Promise<IteratorResult<unknown>>(() => {})+ : Promise.resolve({ done: false, value });+ },+ return: vi.fn(async () => ({ done: true, value: undefined })),+ };+ },+ };+}+function createInboxMock(values: readonly unknown[],options: {diff --git a/packages/eve/src/execution/turn-workflow.ts b/packages/eve/src/execution/turn-workflow.tsindex c98c8e373..d754272e0 100644--- a/packages/eve/src/execution/turn-workflow.ts+++ b/packages/eve/src/execution/turn-workflow.ts@@ -9,27 +9,45 @@ import {type TurnStepInput,type TurnWorkflowInput,} from "#execution/durable-session-migrations/turn-workflow.js";-import {- claimHookOwnership,- closeHookIterator,- disposeHook,- isHookConflictError,-} from "#execution/hook-ownership.js";+import { claimHookOwnership, disposeHook, isHookConflictError } from "#execution/hook-ownership.js";import type { NextDriverAction } from "#execution/next-driver-action.js";import { routeDeliverToChildren } from "#execution/route-child-delivery.js";import { runProxySubagentEventStep } from "#execution/subagent-event-proxy-step.js";+import {+ createTurnCancellationControl,+ type TurnCancellationControl,+} from "#execution/turn-cancellation-control.js";import { TurnExecutionCursor } from "#execution/turn-execution-cursor.js";import { resolveWorkflowCallbackBaseUrl } from "#execution/workflow-callback-url.js";import { normalizeSerializableError } from "#execution/workflow-errors.js";import { turnStep } from "#execution/workflow-steps.js";+import { activeTurnId } from "#harness/active-turn-id.js";import { resolveRuntimeActionResultsForKeys } from "#harness/runtime-actions.js";import type { RuntimeActionResult } from "#runtime/actions/types.js";const TASK_MODE_WAIT_ERROR_MESSAGE = "Task mode cannot wait for follow-up input (`next: null`).";+// A cancelled turn settles by parking the session, so the cancel hook is+// only claimed where a park can land: conversation sessions always accept+// follow-up input, and task sessions can park only when a continuation+// token anchors them to a waiting parent (delegated subagents always have+// one). A root task run without one is unparkable, so it stays+// uncancellable rather than settling a cancel as `session.failed`.+function canSettleCancelledTurnAsPark(input: TurnWorkflowInput): boolean {+ return input.mode === "conversation" || input.stepInput.sessionState.continuationToken !== "";+}+export type { TurnWorkflowInput };-/** Runs one complete logical turn, including child-agent waits when supported. */+/**+ * Runs one complete logical turn, including child-agent waits when supported.+ *+ * The turn-owned path also owns turn cancellation: resuming the+ * session-scoped cancel hook (`{sessionId}:cancel`) mid-turn aborts the+ * signal serialized into every `turnStep` and settles the turn as+ * `turn.cancelled` → `session.waiting` — never as a failure. A late or+ * guard-mismatched cancel is a benign no-op.+ */export async function turnWorkflow(rawInput: unknown): Promise<void> {"use workflow";@@ -62,6 +80,7 @@ async function runTurnOwnedWorkflow(input: TurnWorkflowInput): Promise<void> {const bufferedDeliveries: DeliverHookPayload[] = [];let nextStepInput = input.stepInput.input;let ownsInbox = false;+ let cancellation: TurnCancellationControl | undefined;try {try {@@ -72,10 +91,40 @@ async function runTurnOwnedWorkflow(input: TurnWorkflowInput): Promise<void> {throw error;}+ // Claimed after the inbox claim so a losing duplicate run never+ // contends for the session cancel token.+ if (+ input.driverCapabilities?.cancelledTurnSettle === true &&+ canSettleCancelledTurnAsPark(input)+ ) {+ cancellation = await createTurnCancellationControl({+ expectedTurnId: activeTurnId(input.stepInput.sessionState.emissionState),+ sessionId: input.stepInput.sessionState.sessionId,+ });+ }+while (true) {- const result = await turnStep(cursor.createStepInput(nextStepInput));+ const result = await turnStep(cursor.createStepInput(nextStepInput, cancellation?.signal));++ if (result.action === "cancelled") {+ // No `canPark` check here: that gate rejects model-authored waits+ // (`next: null`) in task mode, whereas a cancelled turn parks by+ // design and its parkability was already established when the+ // cancel hook was claimed (`canSettleCancelledTurnAsPark`). The+ // epilogue runs in the driver (`settleCancelledTurnStep`), not as+ // a step in this run, where queued cancel wakes could re-dispatch+ // it.+ await cancellation?.dispose();+ await cursor.finish(+ { sessionState: cursor.sessionState },+ { cancelled: true, kind: "park" },+ bufferedDeliveries,+ );+ return;+ }if (result.action === "done") {+ await cancellation?.dispose();await cursor.finish(result,{@@ -114,6 +163,7 @@ async function runTurnOwnedWorkflow(input: TurnWorkflowInput): Promise<void> {const results = await waitForRuntimeActionResults({bufferedDeliveries,+ cancellation,cursor,inboxToken: inbox.token,initialResults: dispatchResult.results,@@ -121,6 +171,12 @@ async function runTurnOwnedWorkflow(input: TurnWorkflowInput): Promise<void> {nextDeliveryRequestId,pendingActionKeys,});+ if (results === "cancelled") {+ // The next turnStep observes the aborted signal and settles+ // through the `cancelled` arm above.+ nextStepInput = undefined;+ continue;+ }nextStepInput = { kind: "runtime-action-result", results };continue;}@@ -133,6 +189,7 @@ async function runTurnOwnedWorkflow(input: TurnWorkflowInput): Promise<void> {if (!canPark) throw new Error(TASK_MODE_WAIT_ERROR_MESSAGE);+ await cancellation?.dispose();await cursor.finish(result,{@@ -151,20 +208,30 @@ async function runTurnOwnedWorkflow(input: TurnWorkflowInput): Promise<void> {await cursor.send({ error: normalizeSerializableError(error), kind: "turn-error" });throw error;} finally {- await closeHookIterator(iterator);+ // Dispose-only teardown: `iterator.return()` would await a pending+ // durable read that never settles, leaving this run `running` forever+ // and its hooks unswept. The cancel token is disposed *before* each+ // terminal result publishes so the next turn's claim never races this+ // run's teardown; this backstop covers the error path.+ if (cancellation !== undefined) await cancellation.dispose();if (ownsInbox) await disposeHook(inbox);}}+// `"cancelled"` stays a sentinel rather than a `RuntimeActionResult`+// variant: that union is the schema-validated wire type projected into+// harness resume calls, while cancellation is a control-flow outcome of+// this wait that never leaves the workflow.async function waitForRuntimeActionResults(input: {readonly bufferedDeliveries: DeliverHookPayload[];+ readonly cancellation: TurnCancellationControl | undefined;readonly cursor: TurnExecutionCursor;readonly inboxToken: string;readonly initialResults: readonly RuntimeActionResult[];readonly iterator: AsyncIterator<TurnInboxPayload>;readonly nextDeliveryRequestId: () => string;readonly pendingActionKeys: readonly string[];-}): Promise<readonly RuntimeActionResult[]> {+}): Promise<readonly RuntimeActionResult[] | "cancelled"> {let pendingDeliveryRequest: string | undefined;const results: RuntimeActionResult[] = [...input.initialResults];@@ -195,7 +262,25 @@ async function waitForRuntimeActionResults(input: {});}- const next = await input.iterator.next();+ const nextPromise = input.iterator.next();+ // When a cancel wins the race, the dangling inbox `next()` is dropped+ // by disposal in teardown; pre-attach a handler so a late rejection+ // never surfaces as unhandled.+ nextPromise.catch(() => {});+ const next = await (input.cancellation === undefined+ ? nextPromise+ : Promise.race([nextPromise, input.cancellation.requested]));+ if (next === "cancel") {+ if (pendingDeliveryRequest !== undefined) {+ // Release the raced public input back to the driver so it stays+ // available for the next turn.+ await input.cursor.send({+ kind: "turn-delivery-cancelled",+ requestId: pendingDeliveryRequest,+ });+ }+ return "cancelled";+ }if (next.done) throw new Error("Turn inbox closed before runtime actions completed.");const value = next.value;diff --git a/packages/eve/src/execution/workflow-entry.test.ts b/packages/eve/src/execution/workflow-entry.test.tsindex 12baa441d..b1dd420af 100644--- a/packages/eve/src/execution/workflow-entry.test.ts+++ b/packages/eve/src/execution/workflow-entry.test.ts@@ -11,7 +11,9 @@ import { fireSessionCallbackStep } from "#execution/session-callback-step.js";import type { TurnControlPayload } from "#execution/turn-control-protocol.js";import { workflowEntry } from "#execution/workflow-entry.js";import { routeDeliverToChildren } from "#execution/route-child-delivery.js";-import { dispatchTurnStep, emitTerminalSessionFailureStep } from "#execution/workflow-steps.js";+import { settleCancelledTurnStep } from "#execution/settle-cancelled-turn-step.js";+import { dispatchTurnStep } from "#execution/workflow-steps.js";+import { emitTerminalSessionFailureStep } from "#execution/terminal-session-failure-step.js";vi.mock("#compiled/@workflow/core/index.js", () => ({createHook: vi.fn(),@@ -52,9 +54,16 @@ vi.mock("./delegated-parent-notification.js", () => ({vi.mock("./workflow-steps.js", () => ({dispatchTurnStep: vi.fn().mockImplementation(async () => ({ runId: "turn-run" })),+}));++vi.mock("./terminal-session-failure-step.js", () => ({emitTerminalSessionFailureStep: vi.fn().mockResolvedValue(undefined),}));+vi.mock("./settle-cancelled-turn-step.js", () => ({+ settleCancelledTurnStep: vi.fn().mockResolvedValue(undefined),+}));+function createSessionStateForMock(overrides: Partial<DurableSessionState> = {},): DurableSessionState {@@ -470,6 +479,71 @@ describe("workflowEntry", () => {expect(resumeHook).not.toHaveBeenCalled();});+ it("settles a cancelled turn in the driver and starts the next turn from the settled state", async () => {+ const sessionState = createBaseSessionState();+ const settledState = createBaseSessionState({+ emissionState: { sequence: 1, sessionStarted: true, stepIndex: 0, turnId: "" },+ });+ vi.mocked(createSessionStep).mockResolvedValue(createSessionStepResultForMock(sessionState));+ vi.mocked(settleCancelledTurnStep).mockResolvedValue({+ serializedContext: { "eve.sessionId": "wrun_test_123", settled: true },+ sessionState: settledState,+ });+ installHookMocks({+ deliveryHooks: [+ {+ token: "http:test",+ values: [{ kind: "deliver", payloads: [{ message: "after cancel" }] }],+ },+ ],+ turnControls: [+ {+ action: {+ cancelled: true,+ kind: "park",+ serializedContext: { "eve.sessionId": "wrun_test_123" },+ sessionState,+ },+ kind: "turn-result",+ },+ turnResult({ action: "done", output: "ok", sessionState: settledState }),+ ],+ });++ const result = await workflowEntry({+ input: { message: "hello" },+ serializedContext: createSerializedContext(),+ });++ expect(result).toEqual({ output: "ok" });+ expect(settleCancelledTurnStep).toHaveBeenCalledExactlyOnceWith({+ parentWritable: expect.any(WritableStream),+ serializedContext: { "eve.sessionId": "wrun_test_123" },+ sessionState,+ });+ expect(vi.mocked(dispatchTurnStep).mock.calls[1]?.[0]).toMatchObject({+ serializedContext: { settled: true },+ sessionState: settledState,+ });+ });++ it("does not settle an ordinary park as cancelled", async () => {+ const sessionState = createBaseSessionState();+ vi.mocked(createSessionStep).mockResolvedValue(createSessionStepResultForMock(sessionState));+ installHookMocks({+ deliveryHooks: [{ token: "http:test", values: [] }],+ turnControls: [turnResult({ action: "park", sessionState })],+ });++ const result = await workflowEntry({+ input: { message: "hello" },+ serializedContext: createSerializedContext(),+ });++ expect(result).toEqual({ output: "" });+ expect(settleCancelledTurnStep).not.toHaveBeenCalled();+ });+it("skips child routing when a turn completes without yielding to a delivery", async () => {const sessionState = createBaseSessionState();vi.mocked(createSessionStep).mockResolvedValue(createSessionStepResultForMock(sessionState));diff --git a/packages/eve/src/execution/workflow-entry.ts b/packages/eve/src/execution/workflow-entry.tsindex 29e60be99..4f0147dce 100644--- a/packages/eve/src/execution/workflow-entry.ts+++ b/packages/eve/src/execution/workflow-entry.ts@@ -22,7 +22,8 @@ import { routeDeliverToChildren } from "#execution/route-child-delivery.js";import { dispatchAndAwaitTurn } from "#execution/turn-dispatch.js";import { normalizeSerializableError } from "#execution/workflow-errors.js";import { createSessionStep } from "#execution/create-session-step.js";-import { emitTerminalSessionFailureStep } from "#execution/workflow-steps.js";+import { settleCancelledTurnStep } from "#execution/settle-cancelled-turn-step.js";+import { emitTerminalSessionFailureStep } from "#execution/terminal-session-failure-step.js";import { fireSessionCallbackStep } from "#execution/session-callback-step.js";import { closeHookIterator, disposeHook } from "#execution/hook-ownership.js";import {@@ -167,19 +168,36 @@ async function runDriverLoop(input: {const bufferedDeliveries: DeliverHookPayload[] = [];const deliveryHook = createSessionDeliveryHook(bufferedDeliveries);- try {- if (input.sessionState.continuationToken) {- await deliveryHook.rekey(input.sessionState.continuationToken);- }-- let action: NextDriverAction = await dispatchAndAwaitTurn({+ // Control-hook disposal is deferred one turn — see DispatchedTurn.+ let disposeSettledTurnControl: (() => Promise<void>) | undefined;+ const runTurn = async (args: {+ readonly delivery: HookPayload;+ readonly serializedContext: Record<string, unknown>;+ readonly sessionState: DurableSessionState;+ }): Promise<NextDriverAction> => {+ const turn = await dispatchAndAwaitTurn({bufferedDeliveries,capabilities: input.capabilities,controlToken: nextTurnControlToken(),- delivery: input.initialInput,+ delivery: args.delivery,deliveryHook,mode: input.mode,parentWritable: input.driverWritable,+ serializedContext: args.serializedContext,+ sessionState: args.sessionState,+ });+ await disposeSettledTurnControl?.();+ disposeSettledTurnControl = turn.dispose;+ return turn.action;+ };++ try {+ if (input.sessionState.continuationToken) {+ await deliveryHook.rekey(input.sessionState.continuationToken);+ }++ let action: NextDriverAction = await runTurn({+ delivery: input.initialInput,serializedContext: input.serializedContext,sessionState: input.sessionState,});@@ -199,6 +217,19 @@ async function runDriverLoop(input: {throw new Error(`Driver received unexpected turn action "${action.kind}".`);}+ if (action.cancelled === true) {+ const settled = await settleCancelledTurnStep({+ parentWritable: input.driverWritable,+ serializedContext: action.serializedContext,+ sessionState: action.sessionState,+ });+ action = {+ ...action,+ serializedContext: settled.serializedContext,+ sessionState: settled.sessionState,+ };+ }+if (!action.sessionState.continuationToken) {throw new Error("Cannot park: no continuation token available. The channel must " +@@ -224,17 +255,11 @@ async function runDriverLoop(input: {}}- action = await dispatchAndAwaitTurn({- bufferedDeliveries,- capabilities: input.capabilities,- controlToken: nextTurnControlToken(),+ action = await runTurn({delivery: {kind: "deliver",payloads: allPayloads,},- deliveryHook,- mode: input.mode,- parentWritable: input.driverWritable,serializedContext: action.serializedContext,sessionState: action.sessionState,});@@ -262,24 +287,19 @@ async function runDriverLoop(input: {continue;}- action = await dispatchAndAwaitTurn({- bufferedDeliveries,- capabilities: input.capabilities,- controlToken: nextTurnControlToken(),+ action = await runTurn({delivery: {auth: nextDeliver.auth,kind: "deliver",payloads: [remainder],requestId: nextDeliver.requestId,},- deliveryHook,- mode: input.mode,- parentWritable: input.driverWritable,serializedContext: action.serializedContext,sessionState: action.sessionState,});}} finally {+ await disposeSettledTurnControl?.();await deliveryHook.dispose();await closeHookIterator(authIterator);await disposeHook(authHook);diff --git a/packages/eve/src/execution/workflow-steps.test.ts b/packages/eve/src/execution/workflow-steps.test.tsindex 379aa9b12..988299680 100644--- a/packages/eve/src/execution/workflow-steps.test.ts+++ b/packages/eve/src/execution/workflow-steps.test.ts@@ -24,9 +24,9 @@ import { projectToDurableSession } from "#execution/session.js";import { createExecutionNodeStep } from "#execution/node-step.js";import { dispatchRuntimeActionsStep } from "#execution/dispatch-runtime-actions-step.js";import { runProxySubagentEventStep } from "#execution/subagent-event-proxy-step.js";+import { emitTerminalSessionFailureStep } from "#execution/terminal-session-failure-step.js";import {dispatchTurnStep,- emitTerminalSessionFailureStep,resolveEffectiveOutputSchema,turnStep,} from "#execution/workflow-steps.js";diff --git a/packages/eve/src/execution/workflow-steps.ts b/packages/eve/src/execution/workflow-steps.tsindex aaabdaa4b..185dab3b0 100644--- a/packages/eve/src/execution/workflow-steps.ts+++ b/packages/eve/src/execution/workflow-steps.ts@@ -11,6 +11,7 @@ import { BundleKey, ChannelKey } from "#runtime/sessions/runtime-context-keys.jsimport { runStep } from "#context/run-step.js";import { deserializeContext, serializeContext } from "#context/serialize.js";import { getHarnessEmissionState } from "#harness/emission.js";+import { isTurnCancellation, throwIfTurnAborted } from "#harness/turn-cancellation.js";import { setChannelContext } from "#execution/channel-context.js";import { hasPendingInputBatch } from "#harness/input-requests.js";import { coalesceTurnInputs } from "#harness/messages.js";@@ -26,10 +27,8 @@ import type { TokenUsage } from "#shared/token-usage.js";import type { JsonObject } from "#shared/json.js";import type { RunMode } from "#shared/run-mode.js";import { getRuntimeActionRequestKey } from "#runtime/actions/keys.js";-import { createLogger, formatError } from "#internal/logging.js";import {createAuthorizationCompletedEvent,- createSessionFailedEvent,encodeMessageStreamEvent,type HandleMessageStreamEvent,timestampHandleMessageStreamEvent,@@ -76,6 +75,10 @@ import { resumeHook } from "#internal/workflow/runtime.js";* `pendingRuntimeActionKeys` so the turn workflow can pick the right* {@link import("#execution/next-driver-action.js").NextDriverAction}* arm without re-reading the session.+ *+ * `cancelled` converts the harness's cancellation throw into a *returned*+ * result so workflow-core never classifies the abort as a step failure or+ * retries it; the epilogue runs in `settleCancelledTurnStep`.*/export type DurableStepResult =| {@@ -87,6 +90,11 @@ export type DurableStepResult =/** Session-total token usage; set on `done` when the session spent any. */readonly usage?: TokenUsage;}+ | {+ readonly action: "cancelled";+ readonly serializedContext: Record<string, unknown>;+ readonly sessionState: DurableSessionState;+ }| {readonly action: "park";readonly authorizationNames?: readonly string[];@@ -297,66 +305,81 @@ export async function turnStep(rawInput: TurnStepInput): Promise<DurableStepResuconst mode = ctx.require(ModeKey);- let stepResult = await runStep(ctx, initialSession, async (enrichedSession) => {- const schemaSession = resolveEffectiveOutputSchema({- agentOutputSchema: bundle.turnAgent.outputSchema,- input: resolved,- mode,- session: enrichedSession,- });- if (completedAuths) {- const emissionState = getHarnessEmissionState(schemaSession.state);- for (const { name, authorization } of completedAuths) {- await handleEvent(- createAuthorizationCompletedEvent({- authorization,- name,- outcome: "authorized",- sequence: emissionState.sequence,- stepIndex: emissionState.stepIndex,- turnId: emissionState.turnId,- }),- );+ let stepResult: StepResult;+ try {+ // A signal already aborted at entry (cancellation during an in-line+ // runtime-action wait) must settle before the park-resume stages run,+ // or the pending batch would re-park and later re-dispatch.+ throwIfTurnAborted(input.abortSignal);+ stepResult = await runStep(ctx, initialSession, async (enrichedSession) => {+ const schemaSession = resolveEffectiveOutputSchema({+ agentOutputSchema: bundle.turnAgent.outputSchema,+ input: resolved,+ mode,+ session: enrichedSession,+ });+ if (completedAuths) {+ const emissionState = getHarnessEmissionState(schemaSession.state);+ for (const { name, authorization } of completedAuths) {+ await handleEvent(+ createAuthorizationCompletedEvent({+ authorization,+ name,+ outcome: "authorized",+ sequence: emissionState.sequence,+ stepIndex: emissionState.stepIndex,+ turnId: emissionState.turnId,+ }),+ );+ }}- }-- const capabilities = ctx.get(CapabilitiesKey);- const runHarnessStep = async (- lifecycleSession: HarnessSession,- stepInput: StepInput | undefined,- ): Promise<StepResult> => {- const skillRoot = await resolveSessionSkillRoot({- ctx,- turnAgent: bundle.turnAgent,- });- const refreshedSession = refreshSessionFromTurnAgent({- compactionOverrides: {- thresholdPercent: bundle.resolvedAgent.config.compaction?.thresholdPercent,- },- session: lifecycleSession,- skillRoot,- turnAgent: bundle.turnAgent,- });+ const capabilities = ctx.get(CapabilitiesKey);++ const runHarnessStep = async (+ lifecycleSession: HarnessSession,+ stepInput: StepInput | undefined,+ ): Promise<StepResult> => {+ const skillRoot = await resolveSessionSkillRoot({+ ctx,+ turnAgent: bundle.turnAgent,+ });+ const refreshedSession = refreshSessionFromTurnAgent({+ compactionOverrides: {+ thresholdPercent: bundle.resolvedAgent.config.compaction?.thresholdPercent,+ },+ session: lifecycleSession,+ skillRoot,+ turnAgent: bundle.turnAgent,+ });++ const step = createExecutionNodeStep({+ abortSignal: input.abortSignal,+ capabilities,+ createRuntime: createWorkflowRuntime,+ handleEvent,+ mode,+ modelResolutionScope: {+ moduleMap: bundle.moduleMap,+ nodeId: bundle.nodeId,+ },+ node: bundle.graph.root,+ workflowMaxSubagents: refreshedSession.workflowMaxSubagents,+ });+ return step(refreshedSession, stepInput);+ };- const step = createExecutionNodeStep({- abortSignal: input.abortSignal,- capabilities,- createRuntime: createWorkflowRuntime,- handleEvent,- mode,- modelResolutionScope: {- moduleMap: bundle.moduleMap,- nodeId: bundle.nodeId,- },- node: bundle.graph.root,- workflowMaxSubagents: refreshedSession.workflowMaxSubagents,- });- return step(refreshedSession, stepInput);+ return runHarnessStep(schemaSession, resolved);+ });+ } catch (error) {+ if (!isTurnCancellation(error)) throw error;+ writer.releaseLock();+ return {+ action: "cancelled",+ serializedContext: input.serializedContext,+ sessionState: input.sessionState,};-- return runHarnessStep(schemaSession, resolved);- });+ }// Re-stamp the in-memory session's continuation token in case a// handler called `setContinuationToken(...)` (eg. Slack auto-anchor).@@ -474,68 +497,6 @@ export function resolveEffectiveOutputSchema(input: {return session;}-const log = createLogger("execution.workflow-entry");--/** Emits a terminal `session.failed` to the adapter and durable stream. */-export async function emitTerminalSessionFailureStep(input: {- readonly error: unknown;- readonly parentWritable: WritableStream<Uint8Array>;- readonly serializedContext: Record<string, unknown>;-}): Promise<void> {- "use step";-- const details = formatError(input.error);- const code = typeof details.name === "string" ? details.name : "WORKFLOW_EXECUTION_FAILED";- const message = typeof details.message === "string" ? details.message : String(input.error);- const sessionId = (input.serializedContext["eve.sessionId"] as string | undefined) ?? "";-- log.error("workflow loop threw — emitting terminal session.failed", {- sessionId,- errorId: typeof details.errorId === "string" ? details.errorId : undefined,- code,- message,- detail: typeof details.detail === "string" ? details.detail : undefined,- });-- const event = createSessionFailedEvent({ code, details, message, sessionId });-- // Best-effort: invoke the adapter handler so channels surface the- // failure. Errors are logged, never rethrown — the outer workflow- // throw must still reach the run handle.- try {- const ctx = await deserializeContext(input.serializedContext);- const adapter = ctx.get(ChannelKey);- if (adapter !== undefined) {- const adapterCtx = buildAdapterContext(adapter, ctx);- await callAdapterEventHandler(adapter, event, adapterCtx);- }- } catch (notificationError) {- log.error("adapter failed to handle terminal session.failed event", {- errorId: typeof details.errorId === "string" ? details.errorId : undefined,- sessionId,- error: notificationError,- });- }-- // Always write the event to the durable stream so downstream- // consumers see a canonical terminal event instead of an abrupt- // stream close.- try {- const writer = input.parentWritable.getWriter();- try {- await writer.write(encodeMessageStreamEvent(timestampHandleMessageStreamEvent(event)));- } finally {- writer.releaseLock();- }- } catch (writeError) {- log.error("failed to write terminal session.failed event to durable stream", {- errorId: typeof details.errorId === "string" ? details.errorId : undefined,- sessionId,- error: writeError,- });- }-}-export interface RoutedDeliverResult {/** `undefined` when the entire payload was routed to descendants. */readonly remainder: DeliverPayload | undefined;diff --git a/packages/eve/src/harness/active-turn-id.test.ts b/packages/eve/src/harness/active-turn-id.test.tsnew file mode 100644index 000000000..7cf7eef32--- /dev/null+++ b/packages/eve/src/harness/active-turn-id.test.ts@@ -0,0 +1,17 @@+import { describe, expect, it } from "vitest";++import { activeTurnId } from "#harness/active-turn-id.js";++describe("activeTurnId", () => {+ it("returns the in-flight turn id when a turn is active", () => {+ expect(+ activeTurnId({ sessionStarted: true, sequence: 3, stepIndex: 1, turnId: "turn_3" }),+ ).toBe("turn_3");+ });++ it("mints the next turn id from the sequence between turns", () => {+ expect(activeTurnId({ sessionStarted: true, sequence: 4, stepIndex: 0, turnId: "" })).toBe(+ "turn_4",+ );+ });+});diff --git a/packages/eve/src/harness/active-turn-id.ts b/packages/eve/src/harness/active-turn-id.tsnew file mode 100644index 000000000..858f28ce2--- /dev/null+++ b/packages/eve/src/harness/active-turn-id.ts@@ -0,0 +1,14 @@+import type { HarnessEmissionState } from "#harness/emission.js";++/**+ * Returns the id of the in-flight turn, or — between turns — the id the+ * next turn will mint (`turn_${sequence}`, the same formula+ * `emitTurnPreamble` uses).+ *+ * Lives in a leaf module so `"use workflow"` bodies can import it without+ * pulling harness emission (and its Node builtins) into the workflow+ * bundle.+ */+export function activeTurnId(state: HarnessEmissionState): string {+ return state.turnId === "" ? `turn_${state.sequence}` : state.turnId;+}diff --git a/packages/eve/src/harness/cancelled-turn-emission.test.ts b/packages/eve/src/harness/cancelled-turn-emission.test.tsnew file mode 100644index 000000000..eb21bbb8e--- /dev/null+++ b/packages/eve/src/harness/cancelled-turn-emission.test.ts@@ -0,0 +1,46 @@+import { describe, expect, it } from "vitest";++import { emitCancelledTurn } from "#harness/cancelled-turn-emission.js";+import type { HandleMessageStreamEvent } from "#protocol/message.js";++describe("emitCancelledTurn", () => {+ it("emits turn.cancelled → session.waiting and no failure events", async () => {+ const events: HandleMessageStreamEvent[] = [];+ const next = await emitCancelledTurn(+ async (event) => {+ events.push(event);+ },+ { sessionStarted: true, sequence: 3, stepIndex: 2, turnId: "turn_3" },+ );++ expect(events.map((event) => event.type)).toEqual(["turn.cancelled", "session.waiting"]);+ expect(events[0]).toMatchObject({+ data: { sequence: 3, turnId: "turn_3" },+ type: "turn.cancelled",+ });+ expect(next).toEqual({ sessionStarted: true, sequence: 4, stepIndex: 0, turnId: "" });+ });++ it("reconstructs the turn id when the cancelled step began the turn", async () => {+ // A first-step cancellation aborts before the preamble's state update+ // is persisted: the persisted state still has the between-turns+ // turnId "", but `turn.started` for `turn_${sequence}` is already on+ // the stream.+ const events: HandleMessageStreamEvent[] = [];+ const next = await emitCancelledTurn(+ async (event) => {+ events.push(event);+ },+ { sessionStarted: false, sequence: 0, stepIndex: 0, turnId: "" },+ );++ expect(events[0]).toMatchObject({+ data: { sequence: 0, turnId: "turn_0" },+ type: "turn.cancelled",+ });+ // `session.started` was already emitted by the preamble.+ expect(next.sessionStarted).toBe(true);+ expect(next.sequence).toBe(1);+ expect(next.turnId).toBe("");+ });+});diff --git a/packages/eve/src/harness/cancelled-turn-emission.ts b/packages/eve/src/harness/cancelled-turn-emission.tsnew file mode 100644index 000000000..948779cb9--- /dev/null+++ b/packages/eve/src/harness/cancelled-turn-emission.ts@@ -0,0 +1,33 @@+import { createSessionWaitingEvent, createTurnCancelledEvent } from "#protocol/message.js";+import type { HarnessEmitFn } from "#harness/types.js";++import { activeTurnId } from "#harness/active-turn-id.js";+import type { HarnessEmissionState } from "#harness/emission.js";++/**+ * Emits the cancelled-turn epilogue: `turn.cancelled` → `session.waiting`+ * (never a failure event) and returns the between-turns emission state.+ *+ * `state` is the last *persisted* emission state, which may predate the+ * cancelled turn's preamble — the turn id is reconstructed via+ * {@link activeTurnId} and `sessionStarted` is stamped `true`.+ */+export async function emitCancelledTurn(+ emitFn: HarnessEmitFn,+ state: HarnessEmissionState,+): Promise<HarnessEmissionState> {+ await emitFn(+ createTurnCancelledEvent({+ sequence: state.sequence,+ turnId: activeTurnId(state),+ }),+ );+ await emitFn(createSessionWaitingEvent());++ return {+ sessionStarted: true,+ sequence: state.sequence + 1,+ stepIndex: 0,+ turnId: "",+ };+}diff --git a/packages/eve/src/harness/proxy-input-requests.ts b/packages/eve/src/harness/proxy-input-requests.tsindex 175e6fca0..b093e5119 100644--- a/packages/eve/src/harness/proxy-input-requests.ts+++ b/packages/eve/src/harness/proxy-input-requests.ts@@ -79,6 +79,17 @@ export function clearProxyInputRequestsForChild(return writeMap(session, next);}+/**+ * Removes every proxy entry. Called when a cancelled turn orphans its+ * descendants so stale HITL responses no longer route to them.+ */+export function clearAllProxyInputRequests(session: HarnessSession): HarnessSession {+ if (!hasProxyInputRequests(session.state)) {+ return session;+ }+ return writeMap(session, {});+}+/*** Projects a {@link SubagentInputRequestHookPayload} into the* `(requestId, childContinuationToken)` tuples the session stores.diff --git a/packages/eve/src/protocol/message.test.ts b/packages/eve/src/protocol/message.test.tsindex 29c27f3ee..9d056d544 100644--- a/packages/eve/src/protocol/message.test.ts+++ b/packages/eve/src/protocol/message.test.ts@@ -11,6 +11,7 @@ import {createMessageReceivedEvent,createResultCompletedEvent,createStepStartedEvent,+ createTurnCancelledEvent,encodeMessageStreamEvent,timestampHandleMessageStreamEvent,} from "#protocol/message.js";@@ -18,7 +19,14 @@ import { createEveConnectionCallbackRoutePath } from "#protocol/routes.js";describe("message stream protocol", () => {it("pins the stream version for timed session events", () => {- expect(EVE_MESSAGE_STREAM_VERSION).toBe("18");+ expect(EVE_MESSAGE_STREAM_VERSION).toBe("19");+ });++ it("creates turn.cancelled events", () => {+ expect(createTurnCancelledEvent({ sequence: 2, turnId: "turn_2" })).toEqual({+ data: { sequence: 2, turnId: "turn_2" },+ type: "turn.cancelled",+ });});it("creates result.completed events", () => {diff --git a/packages/eve/src/protocol/message.ts b/packages/eve/src/protocol/message.tsindex eff239f31..810b4ee3d 100644--- a/packages/eve/src/protocol/message.ts+++ b/packages/eve/src/protocol/message.ts@@ -16,7 +16,7 @@ export const EVE_STREAM_FORMAT_HEADER = "x-eve-stream-format";export const EVE_STREAM_VERSION_HEADER = "x-eve-stream-version";export const EVE_MESSAGE_STREAM_CONTENT_TYPE = "application/x-ndjson; charset=utf-8";export const EVE_MESSAGE_STREAM_FORMAT = "ndjson";-export const EVE_MESSAGE_STREAM_VERSION = "18";+export const EVE_MESSAGE_STREAM_VERSION = "19";/*** eve-owned finish reason for one completed assistant step.@@ -448,6 +448,20 @@ export interface TurnFailedStreamEvent {type: "turn.failed";}+/**+ * Stream event emitted when one turn is cancelled before reaching a+ * terminal outcome. Cancellation is not failure: the turn ends without+ * `turn.failed`/`session.failed`, is followed by `session.waiting`, and+ * the session accepts the next message normally.+ */+export interface TurnCancelledStreamEvent {+ data: {+ sequence: number;+ turnId: string;+ };+ type: "turn.cancelled";+}+/*** Stream event emitted when the workflow decides to compact the current* visible session history before the next model fragment runs.@@ -590,6 +604,7 @@ export type HandleMessageStreamEvent = (| StepCompletedStreamEvent| StepFailedStreamEvent| StepStartedStreamEvent+ | TurnCancelledStreamEvent| TurnCompletedStreamEvent| TurnFailedStreamEvent| TurnStartedStreamEvent@@ -1286,6 +1301,20 @@ export function createTurnFailedEvent(input: {};}+/** Creates the `turn.cancelled` event for one cancelled turn. */+export function createTurnCancelledEvent(input: {+ readonly sequence: number;+ readonly turnId: string;+}): TurnCancelledStreamEvent {+ return {+ data: {+ sequence: input.sequence,+ turnId: input.turnId,+ },+ type: "turn.cancelled",+ };+}+/*** Creates the `compaction.requested` event for one runtime compaction pass.*/diff --git a/packages/eve/src/public/channels/discord/discordChannel.ts b/packages/eve/src/public/channels/discord/discordChannel.tsindex 637b730ad..1e0386357 100644--- a/packages/eve/src/public/channels/discord/discordChannel.ts+++ b/packages/eve/src/public/channels/discord/discordChannel.ts@@ -134,6 +134,7 @@ export interface DiscordChannelEvents {readonly "input.requested"?: DiscordEventHandler<"input.requested">;readonly "turn.failed"?: DiscordEventHandler<"turn.failed">;readonly "turn.completed"?: DiscordEventHandler<"turn.completed">;+ readonly "turn.cancelled"?: DiscordEventHandler<"turn.cancelled">;readonly "session.failed"?: DiscordSessionFailedHandler;readonly "session.completed"?: DiscordEventHandler<"session.completed">;readonly "session.waiting"?: DiscordEventHandler<"session.waiting">;diff --git a/packages/eve/src/public/channels/github/githubChannel.ts b/packages/eve/src/public/channels/github/githubChannel.tsindex 436554cb8..bf41a6397 100644--- a/packages/eve/src/public/channels/github/githubChannel.ts+++ b/packages/eve/src/public/channels/github/githubChannel.ts@@ -142,6 +142,7 @@ export interface GitHubChannelEvents {readonly "session.failed"?: GitHubSessionFailedHandler;readonly "session.waiting"?: GitHubEventHandler<"session.waiting">;readonly "turn.completed"?: GitHubEventHandler<"turn.completed">;+ readonly "turn.cancelled"?: GitHubEventHandler<"turn.cancelled">;readonly "turn.failed"?: GitHubEventHandler<"turn.failed">;readonly "turn.started"?: GitHubEventHandler<"turn.started">;}diff --git a/packages/eve/src/public/channels/linear/linearChannel.ts b/packages/eve/src/public/channels/linear/linearChannel.tsindex 32b5bb278..56b2cfaf6 100644--- a/packages/eve/src/public/channels/linear/linearChannel.ts+++ b/packages/eve/src/public/channels/linear/linearChannel.ts@@ -146,6 +146,7 @@ export interface LinearChannelEvents {readonly "input.requested"?: LinearEventHandler<"input.requested">;readonly "turn.failed"?: LinearEventHandler<"turn.failed">;readonly "turn.completed"?: LinearEventHandler<"turn.completed">;+ readonly "turn.cancelled"?: LinearEventHandler<"turn.cancelled">;readonly "session.failed"?: LinearSessionFailedHandler;readonly "session.completed"?: LinearEventHandler<"session.completed">;readonly "session.waiting"?: LinearEventHandler<"session.waiting">;diff --git a/packages/eve/src/public/channels/slack/slackChannel.ts b/packages/eve/src/public/channels/slack/slackChannel.tsindex af6f012b6..4b2374b3d 100644--- a/packages/eve/src/public/channels/slack/slackChannel.ts+++ b/packages/eve/src/public/channels/slack/slackChannel.ts@@ -332,6 +332,7 @@ export interface SlackChannelEvents {readonly "input.requested"?: SlackEventHandler<"input.requested">;readonly "turn.failed"?: SlackEventHandler<"turn.failed">;readonly "turn.completed"?: SlackEventHandler<"turn.completed">;+ readonly "turn.cancelled"?: SlackEventHandler<"turn.cancelled">;readonly "session.failed"?: SlackSessionFailedHandler;readonly "session.completed"?: SlackEventHandler<"session.completed">;readonly "session.waiting"?: SlackEventHandler<"session.waiting">;diff --git a/packages/eve/src/public/channels/teams/teamsChannel.ts b/packages/eve/src/public/channels/teams/teamsChannel.tsindex 406dd7de6..520ab7ce5 100644--- a/packages/eve/src/public/channels/teams/teamsChannel.ts+++ b/packages/eve/src/public/channels/teams/teamsChannel.ts@@ -164,6 +164,7 @@ export interface TeamsChannelEvents {readonly "input.requested"?: TeamsEventHandler<"input.requested">;readonly "turn.failed"?: TeamsEventHandler<"turn.failed">;readonly "turn.completed"?: TeamsEventHandler<"turn.completed">;+ readonly "turn.cancelled"?: TeamsEventHandler<"turn.cancelled">;readonly "session.failed"?: TeamsSessionFailedHandler;readonly "session.completed"?: TeamsEventHandler<"session.completed">;readonly "session.waiting"?: TeamsEventHandler<"session.waiting">;diff --git a/packages/eve/src/public/channels/telegram/telegramChannel.ts b/packages/eve/src/public/channels/telegram/telegramChannel.tsindex 046a21c2a..9cc497679 100644--- a/packages/eve/src/public/channels/telegram/telegramChannel.ts+++ b/packages/eve/src/public/channels/telegram/telegramChannel.ts@@ -134,6 +134,7 @@ export interface TelegramChannelEvents {readonly "input.requested"?: TelegramEventHandler<"input.requested">;readonly "turn.failed"?: TelegramEventHandler<"turn.failed">;readonly "turn.completed"?: TelegramEventHandler<"turn.completed">;+ readonly "turn.cancelled"?: TelegramEventHandler<"turn.cancelled">;readonly "session.failed"?: TelegramSessionFailedHandler;readonly "session.completed"?: TelegramEventHandler<"session.completed">;readonly "session.waiting"?: TelegramEventHandler<"session.waiting">;diff --git a/packages/eve/src/public/channels/twilio/twilioChannel.ts b/packages/eve/src/public/channels/twilio/twilioChannel.tsindex ee8412e88..1ce70452f 100644--- a/packages/eve/src/public/channels/twilio/twilioChannel.ts+++ b/packages/eve/src/public/channels/twilio/twilioChannel.ts@@ -165,6 +165,7 @@ export interface TwilioChannelEvents {readonly "input.requested"?: TwilioEventHandler<"input.requested">;readonly "turn.failed"?: TwilioEventHandler<"turn.failed">;readonly "turn.completed"?: TwilioEventHandler<"turn.completed">;+ readonly "turn.cancelled"?: TwilioEventHandler<"turn.cancelled">;readonly "session.failed"?: TwilioSessionFailedHandler;readonly "session.completed"?: TwilioEventHandler<"session.completed">;readonly "session.waiting"?: TwilioEventHandler<"session.waiting">;diff --git a/packages/eve/src/public/definitions/channel.test.ts b/packages/eve/src/public/definitions/channel.test.tsindex 0c617b6b4..f7f42abdc 100644--- a/packages/eve/src/public/definitions/channel.test.ts+++ b/packages/eve/src/public/definitions/channel.test.ts@@ -214,6 +214,21 @@ describe("defineChannel", () => {void invalid;});+ it("wires a turn.cancelled handler onto the built adapter", () => {+ const channel = defineChannel({+ routes: [POST("/x", async () => new Response("ok"))],+ events: {+ "turn.cancelled"(event) {+ const turnId: string = event.turnId;+ void turnId;+ },+ },+ });++ const adapter = getAdapter(channel) as Record<string, unknown>;+ expect(typeof adapter["turn.cancelled"]).toBe("function");+ });+it("falls back to open metadata for channels without metadata()", () => {const channel = defineChannel({routes: [POST("/x", async () => new Response("ok"))],diff --git a/packages/eve/src/public/definitions/channel.ts b/packages/eve/src/public/definitions/channel.tsindex 298c5a944..403dbc7cd 100644--- a/packages/eve/src/public/definitions/channel.ts+++ b/packages/eve/src/public/definitions/channel.ts@@ -228,6 +228,7 @@ export interface ChannelEvents<TCtx = void> {readonly "input.requested"?: ChannelEventHandler<"input.requested", TCtx>;readonly "turn.failed"?: ChannelEventHandler<"turn.failed", TCtx>;readonly "turn.completed"?: ChannelEventHandler<"turn.completed", TCtx>;+ readonly "turn.cancelled"?: ChannelEventHandler<"turn.cancelled", TCtx>;readonly "session.failed"?: ChannelSessionFailedHandler<TCtx>;readonly "session.completed"?: ChannelEventHandler<"session.completed", TCtx>;readonly "session.waiting"?: ChannelEventHandler<"session.waiting", TCtx>;@@ -317,6 +318,29 @@ export function defineChannel<return compiled;}+// The Record type fails to compile if this map drifts from the ChannelEvents+// keys in either direction.+const channelEventTypes: Record<keyof ChannelEvents, null> = {+ "turn.started": null,+ "actions.requested": null,+ "action.result": null,+ "message.completed": null,+ "message.appended": null,+ "reasoning.appended": null,+ "reasoning.completed": null,+ "input.requested": null,+ "turn.failed": null,+ "turn.completed": null,+ "turn.cancelled": null,+ "session.failed": null,+ "session.completed": null,+ "session.waiting": null,+ "authorization.required": null,+ "authorization.completed": null,+};++const eventTypes = Object.keys(channelEventTypes) as readonly (keyof ChannelEvents)[];+function buildAdapter<TState, TCtx, TReceiveTarget, TMetadata extends Record<string, unknown>>(definition: ChannelDefinition<TState, TCtx, TReceiveTarget, TMetadata>,): ChannelAdapter<any> {@@ -330,24 +354,6 @@ function buildAdapter<TState, TCtx, TReceiveTarget, TMetadata extends Record<strconst eventHandlers: Record<string, unknown> = {};let hasEventHandlers = false;- const eventTypes = [- "turn.started",- "actions.requested",- "action.result",- "message.completed",- "message.appended",- "reasoning.appended",- "reasoning.completed",- "input.requested",- "turn.failed",- "turn.completed",- "session.failed",- "session.completed",- "session.waiting",- "authorization.required",- "authorization.completed",- ] as const;-const events = definition.events;for (const eventType of eventTypes) {const userHandler = events?.[eventType];diff --git a/packages/eve/src/runtime/agent/mock-model-adapter.ts b/packages/eve/src/runtime/agent/mock-model-adapter.tsindex 98a648462..46201b538 100644--- a/packages/eve/src/runtime/agent/mock-model-adapter.ts+++ b/packages/eve/src/runtime/agent/mock-model-adapter.ts@@ -106,6 +106,7 @@ function createMockModelResult(}} else {const toolCallResult =+ createSubagentDelegationResult(options, modelId) ??createSkillLoadResult(options.prompt, modelId) ??createAuthoredToolCallResult(options, modelId);if (toolCallResult !== null) {@@ -203,6 +204,49 @@ function createSkillLoadResult(});}+const SUBAGENT_TOOL_NAME = "agent";+const SUBAGENT_DELEGATION_DIRECTIVE = /\bdelegate\s+to\s+a\s+subagent\s*:\s*(.+)$/iu;++/**+ * Emits one built-in `agent` tool call when the current user message uses+ * the explicit directive `Delegate to a subagent: <message>`, letting+ * tests exercise a real runtime-action wait. Fires only before the+ * delegated call resolves; then the reply path takes over.+ */+function createSubagentDelegationResult(+ options: BootstrapGenerateOptions,+ modelId: string,+): BootstrapGenerateResult | null {+ const lastUserMessage = getLastUserPromptText(options.prompt);++ if (lastUserMessage === null) {+ return null;+ }++ const directive = SUBAGENT_DELEGATION_DIRECTIVE.exec(lastUserMessage);++ if (directive?.[1] === undefined) {+ return null;+ }++ const tool = getAvailableTools(options).find((entry) => entry.name === SUBAGENT_TOOL_NAME);++ if (tool === undefined) {+ return null;+ }++ const toolInput = { message: directive[1].trim() };++ return createToolCallGenerateResult({+ input: toolInput,+ inputTokens: estimateTokenCount(getPromptText(options.prompt)),+ modelId,+ outputTokens: estimateTokenCount(toolInput.message),+ toolCallId: createToolCallId(SUBAGENT_TOOL_NAME),+ toolName: SUBAGENT_TOOL_NAME,+ });+}+function createAuthoredToolCallResult(options: BootstrapGenerateOptions,modelId: string,diff --git a/research/turn-cancellation-harness-abort.md b/research/turn-cancellation-harness-abort.mdindex 478f14833..ce98122ab 100644--- a/research/turn-cancellation-harness-abort.md+++ b/research/turn-cancellation-harness-abort.md@@ -1,7 +1,7 @@---issue: https://github.com/vercel/eve/issues/483-status: proposed-last_updated: "2026-07-02"+status: complete+last_updated: "2026-07-06"---# Turn cancellation, layer 0: harness abort propagation@@ -27,8 +27,8 @@ Cancellation lands bottom-up. Each layer is independently shippable and reviewab```textlayer 4 clients + channels + evals MessageResponse.cancel(), channel ops, /new, eval controlslayer 3 descendants subagent inbox cascade, remote cancel POST, adoption races-layer 2 trigger surface POST /eve/v1/session/:id/cancel, `${continuationToken}:cancel`- hook, best-effort 202/409+layer 2 trigger surface POST /eve/v1/session/:id/cancel resuming the stable+ `${sessionId}:cancel` hook (optional turnId guard)layer 1 turn-workflow ownership durable AbortController + cancel race in the terminatingturnWorkflow, cancelled-turn finalizer,turn.cancelled → session.waitingdiff --git a/research/turn-cancellation-turn-workflow.md b/research/turn-cancellation-turn-workflow.mdnew file mode 100644index 000000000..ed79d9fd7--- /dev/null+++ b/research/turn-cancellation-turn-workflow.md@@ -0,0 +1,338 @@+---+issue: https://github.com/vercel/eve/issues/483+status: in-review+last_updated: "2026-07-13"+---++# Turn cancellation, layer 1: turn-workflow ownership++> **Ship status: unblocked** (eve#573). The `e2e-local` stress suite had+> exposed that serializing the durable `AbortSignal` into `turnStep` cost+> ~0.5 s fixed per step on world-local and opened an abort-stream tail+> reader whose 100 ms polling did a full `readdir` of the world-global+> chunks directory — a 100-turn session compounded from ~0.9 s/turn into+> a 240 s replay timeout (the Vercel world was unaffected). Filed as+> [workflow#2795](https://github.com/vercel/workflow/issues/2795) (fixed+> cost) and+> [workflow#2797](https://github.com/vercel/workflow/issues/2797)+> (scaling); both fixed upstream by+> [workflow#2807](https://github.com/vercel/workflow/pull/2807)+> (`executeStep` cancels abort readers post-step; chunks sharded+> per-stream), shipped in `@workflow/core` 5.0.0-beta.28 /+> `@workflow/world-local` 5.0.0-beta.24 and vendored into eve by #612.+> Re-benched on the fixed runtime: 40 sequential signal-bearing turns run+> at the ~385 ms no-signal floor with no compounding.++## Summary++Layer 0 (#494) made the harness honor an `AbortSignal` end to end but left+`TurnStepInput.abortSignal` unpopulated. Layer 1 makes the turn cancellable+in-process: the turn-owned `turnWorkflow` claims a durable session-scoped+cancel hook (`{sessionId}:cancel`) and a durable `AbortController`, and+resuming the hook mid-turn settles the turn as `turn.cancelled` →+`session.waiting` — never as a failure. The payload's optional `turnId`+guard scopes a cancel to the turn the caller observed. No HTTP route,+client API, or channel surface exists yet (layer 2); the only trigger is+resuming the cancel hook, which tests do directly.++## Cancellation semantics (as shipped)++- **Trigger**: `resumeHook("${sessionId}:cancel", { turnId? })`. The cancel+ token is **session-scoped and stable**, so a layer-2 trigger derives it+ from the session id alone — no per-turn token discovery, no client-carried+ hook token. Each turn workflow run creates and claims the token at turn+ start and disposes it before publishing its turn result, so at most one+ live cancel hook exists per session and the next turn's claim never races+ the prior run's teardown. A claim conflict (residue of a crashed prior+ turn whose hooks are not yet swept) degrades that turn to uncancellable+ rather than failing it. The hook payload carries no caller-supplied+ reason: layer 1 uses one canonical cancellation reason (the layer-0+ `TurnCancelledError`), and caller reasons arrive with the trigger surface+ in layer 2.+- **Turn guard**: the payload's optional `turnId` scopes the cancel to the+ turn the caller observed (every stream event is stamped with its+ `turnId`). A guard naming any other turn is consumed as a benign no-op —+ a cancel that races a turn boundary can never kill a turn the caller+ never saw. Omitting `turnId` cancels whatever turn is currently running,+ the right default for a plain stop button. Guard reads are durable hook+ reads, so the skip sequence replays deterministically.+- **Cancel during model/tool work**: the durable signal (threaded through+ `TurnStepInput.abortSignal`, the layer-0 seam) aborts the in-flight harness+ work in real time. The turn settles with `turn.cancelled` followed by+ `session.waiting` on the session stream (stream version 19), and the+ session accepts the next message normally.+- **Cancel is not failure.** A cancelled turn never emits `turn.failed`,+ `step.failed`, or `session.failed`, and the aborted `turnStep` is never+ retried as a failure. Pinned by tests; this is the invariant that leaked+ in #347.+- **Cancel during an in-line runtime-action wait** (subagent / dynamic+ workflow results): the wait stops and the turn settles cancelled the same+ way. The pending runtime-action batch and workflow interrupt are dropped+ at settle time (their tool-call exchange lives inside the batch and never+ reached history; replaying them would re-dispatch the actions).+ Descendants are _not_ cascaded to (layer 3); their late results land on+ the dead turn inbox and are dropped.+- **Cancel after the turn settled (or before it starts)**: benign no-op —+ the settle disposed the session cancel hook, so a late resume rejects+ with `HookNotFoundError`. Duplicate and same-instant cancels are also+ safe without trigger-side single-flighting: a duplicate resume neither+ re-dispatches the in-flight step (workflow#2848) nor corrupts replay+ when racing disposal (workflow#2808). Layer 2 only needs to treat+ "already resumed/disposed" as success.+- **Partial content is kept.** Whatever the harness emitted before the abort+ stays on the stream, and durable history persists exactly what the harness+ had settled at abort time — no rollback, no synthesis.+- **Cancellation is driver-negotiated and parkable-only.** The pinned driver+ body advertises `cancelledTurnSettle` through the dispatch input; the turn+ workflow registers the cancel hook only when it is set _and_ the session+ can park (conversation mode, or an anchored continuation token — delegated+ children always have one). Drivers pinned before the capability get no+ cancel hook, so a layer-2 trigger cannot strand them with an unsettled+ cancelled park; unparkable root task runs get no hook, so cancellation can+ never decay into `session.failed`. Task-mode terminal semantics arrive+ with layer 2.+- **Cancel during a descendant HITL wait**: the proxy epilogue has already+ streamed the turn's waiting boundary, so the settle emits nothing extra+ (no fabricated turn id, no duplicate `session.waiting`) and clears the+ proxy input-request map — later HITL answers stay with the parent instead+ of routing to the orphaned child.+- **The settle epilogue runs the authored-hook pipeline.** `turn.cancelled`+ and its `session.waiting` dispatch stream-event hooks like any turn-step+ emission.+- **Parked sessions cannot be cancelled** in this layer: parking terminates+ the turn workflow, so there is no turn to cancel. The session-scoped story+ stays in `research/channel-session-reset.md`.+- **The legacy (non-turn-inbox) workflow path is untouched.**++## Data flow (as shipped)++```text+test / (layer 2 route) resumeHook(`${sessionId}:cancel`, { turnId? })+ │+turnWorkflow execution/turn-workflow.ts+ control = createTurnCancellationControl({ sessionId, expectedTurnId })+ │ claims the stable session token; a conflict (stale prior-turn+ │ claim) degrades the turn to uncancellable instead of failing+ │ mismatched turnId guards are consumed as durable no-op reads+ │ the hook-read continuation aborts the durable controller —+ │ replay-deterministic (keyed to the hook_received event), no+ │ promise race decides whether abort() runs+ ▼+ turnStep (awaited plainly) execution/workflow-steps.ts+ harness aborts (layer 0) → TurnCancelledError+ → returns { action: "cancelled" } pure marker, no side effects+ │+ cursor.finish(…, { cancelled: true, kind: "park" })+ │ park arm reused: new NextDriverAction arms+ │ break pinned drivers; optional fields don't+ ▼+workflowEntry driver execution/workflow-entry.ts+ settleCancelledTurnStep execution/settle-cancelled-turn-step.ts+ emits turn.cancelled → session.waiting, clears pending batch/interrupt,+ persists the between-turns session+ then the normal park playbook: rekey, await the next message+```++Within `waitForRuntimeActionResults`, the inbox read is raced against the+cancel-hook read; a cancel releases any raced public delivery back to the+driver (`turn-delivery-cancelled`) and loops into one final `turnStep`,+which observes the aborted signal at entry (before the park-resume stages,+which would otherwise re-park on the still-pending batch) and settles+through the same cancelled arm.++## Where the shipped design deviates from the proposal++Integration against the real runtime (vendored `@workflow/core`+5.0.0-beta.26) surfaced behaviors that reshaped the epilogue path. This+section is historical: the runtime defects cited below (#2780, #2781)+have since been fixed upstream — see "Runtime findings" for the fixes+and which mitigations remain load-bearing:++1. **No workflow-side race; abort in the hook-read continuation.** The+ durable `abort()` writes a hook event, and the upstream contract+ ("`abort()` becomes a no-op on replay") presumes the call site is+ reached deterministically on every replay — gating it behind a+ `Promise.race` winner makes that reachability replay-dependent. The+ abort now fires in the `.then` of the cancel-hook read itself, keyed to+ the `hook_received` event. (The corruption initially attributed to this+ shape was later traced to workflow#2781; the deterministic call site is+ kept as contract hygiene.)+2. **`turnStep`'s cancelled result is a pure marker.** The runtime can+ supersede an aborted step attempt and re-dispatch it under the same+ correlation id, with both attempts running to completion in-process+ (at-least-once inline execution, workflow#2780). Any cancel-path side+ effect inside `turnStep` — including the epilogue — can therefore+ duplicate.+3. **The epilogue runs in the _driver_, not the turn run.** Queued+ cancel-payload and abort-hook wakes re-dispatch in-flight steps of the+ turn run; the driver's wake sources exclude the cancel hook. The driver+ recognizes `park` + `cancelled: true` and runs `settleCancelledTurnStep`+ before the normal park playbook. (Old pinned drivers ignore the marker+ and simply park — harmless, since no cancel trigger predates them.)+4. **Turn control hooks are disposed one turn late.** The turn run's final+ control send is at-least-once (workflow#2780); a late duplicate resume+ on a _disposed_ hook is accepted and logged by the world and then+ diverges the driver's replay (workflow#2781).+ `dispatchAndAwaitTurn` now returns a deferred `dispose()` that+ the driver invokes when the _next_ turn settles (or the session ends),+ by which time the previous turn's run has completed and cannot re-send.+5. **Turn-run teardown is dispose-only — never `iterator.return()`.** The+ cancel hook always has an outstanding durable read (the abort+ continuation), and an async generator honors `return()` only after its+ in-flight `await` settles — never, for a turn that wasn't cancelled.+ Closing the iterator therefore suspends the turn run forever: it never+ reaches `run_completed`, so the world never sweeps its hooks, leaking+ one running run and three hooks (inbox, cancel, durable-abort) per+ turn, and `getByToken`'s O(live hooks) scan slows every resume in the+ session. Hook disposal alone is the runtime's sanctioned pattern:+ pending reads are dropped and the run proceeds to completion, where+ terminal-run cleanup deletes the run's hooks.++## Runtime findings (filed upstream)++Observed against `@workflow/world-local` during integration testing and+re-validated with standalone spike repros before filing:++- **[workflow#2780](https://github.com/vercel/workflow/issues/2780)** —+ at-least-once inline step execution under wakes: a `hook_received` that+ lands while a step attempt is in flight re-dispatches the step under the+ same correlation id; both attempts run concurrently in-process and both+ flush side effects (only one `step_completed` is recorded). Standalone+ repro deterministic (4/4). Complement to workflow#2777/#2778.+ **Fixed** by [workflow#2848](https://github.com/vercel/workflow/pull/2848)+ (`@workflow/core` 5.0.0-beta.30): inline steps are stamped with the+ owning queue message id, wake replays schedule a delayed backstop+ (lease-bounded) instead of immediately re-dispatching an in-flight step,+ and an upstream in-process single-flight collapses racing attempts. The+ residual is the queue's ordinary at-least-once envelope (redelivery of+ the owning message while the step is alive; cross-instance duplicates+ post-lease).+- **[workflow#2781](https://github.com/vercel/workflow/issues/2781)** —+ a `resumeHook` racing hook disposal is accepted and journaled after+ `hook_disposed`; the owning run's replay then deterministically diverges+ at that event (`REPLAY_DIVERGENCE`, escalating to+ `CorruptedEventLogError`). The duplicate resume that hit the window came+ from a #2780-style re-executed step. The acceptance window is+ timing-dependent (a simplified standalone repro produced the duplicate+ resume but landed it after disposal enforcement); the journal-level+ evidence is unambiguous.+ **Fixed** by [workflow#2808](https://github.com/vercel/workflow/pull/2808)+ (`@workflow/world-local` 5.0.0-beta.24): `hook_received` acceptance is+ serialized against the per-hook dispose lock and re-validated before+ the event append, so a resume ordering after disposal is rejected with+ `HookNotFoundError` instead of corrupting the owner's replay. Two+ caveats: the fix is acceptance-side only (a bad order already journaled+ before it still diverges on replay), and it ships in world-local — the+ hosted Vercel world's parity is unconfirmed.+- **Not filed — falsified in isolation**: an earlier hypothesis that+ reaching `abort()` behind a promise-race winner corrupts the event log+ did not reproduce standalone (5/5 clean); every corruption we observed+ traced to #2781. The abort-in-hook-continuation shape is kept anyway as+ determinism hygiene: the upstream abort-controller contract ("`abort()`+ becomes a no-op on replay") presumes the call site is reached on every+ replay, and the duplicate `abrt` `hook_received` we logged under wake+ storms adds wakes that feed #2780's preconditions.++Upstream status (all four findings are now fixed in the runtime this+branch vendors — `@workflow/core` 5.0.0-beta.31, `@workflow/world-local`+5.0.0-beta.27):++- [workflow#2779](https://github.com/vercel/workflow/pull/2779) fixed+ #2777/#2778 (suspension-handler per-token ordering; world-local claim+ release).+- [workflow#2808](https://github.com/vercel/workflow/pull/2808) closed+ #2781's remaining acceptance window (world-local).+- [workflow#2848](https://github.com/vercel/workflow/pull/2848) fixed+ #2780 (inline step message ownership).++Consequences for layer 1's mitigations:++- The `settleCancelledTurnStep` in-process single-flight it originally+ carried became redundant with the upstream one and was removed; the+ queue's residual at-least-once envelope can duplicate the epilogue,+ like every stream emission under crash retry.+- Deferred control-hook disposal stays: `sendTurnControlStep` does not+ treat `HookNotFoundError` as benign, so a rare crash-retry duplicate of+ the final control send against an already-disposed hook would surface+ as a turn error, and hosted-world parity for the #2808 acceptance fix+ is unconfirmed. Both are addressable in a follow-up if immediate+ disposal is worth it.+- The driver-owned epilogue, the pure `cancelled` marker, the+ abort-in-hook-continuation shape, and dispose-only teardown all stand+ on rationale independent of the fixed bugs (wake-source isolation,+ at-least-once side-effect hygiene, replay determinism, async-generator+ semantics) and are unchanged.++Consequences for layer 2 (relaxations from the earlier guidance):++- Strict per-turn single-flighting of cancel resumes is no longer+ required: a duplicate resume neither spawns a concurrent step attempt+ (#2848) nor corrupts replay when racing disposal (#2808). Treating+ "already resumed/disposed" as success is sufficient.+- These fixes made a stable session-scoped cancel token viable (#2779+ makes the dispose-then-recreate claim shape safe across consecutive+ runs; #2808 removes the disposal race), and layer 1 **adopted it**:+ the shipped token is `{sessionId}:cancel` with an optional `turnId`+ payload guard. Layer 2's route is therefore pure derivation —+ `POST /eve/v1/session/:id/cancel` resumes `{sessionId}:cancel`,+ forwards the caller's optional `turnId`, and maps `HookNotFoundError`+ (no turn in flight) to a "nothing to cancel" success.++## Invariants (pinned by tests)++1. An aborted `turnStep` settles by return value; no thrown cancellation+ crosses the step boundary — the turn workflow run records no+ `step_failed`/`step_retrying` events and at most one `step_completed`+ per correlation id.+2. `turn.cancelled` is emitted once per cancelled turn under normal+ operation (at-least-once under crash retry, like every stream+ emission), always followed by+ `session.waiting`; zero failure events on the cancelled path; the aborted+ tool executes once; the cancelled turn streams one `step.started`.+3. The cancel hook token is session-scoped and never doubly live: each+ turn workflow run claims `{sessionId}:cancel` at turn start and+ disposes it before publishing its turn result, so the next turn's+ claim never races the prior run's teardown. A conflicting stale claim+ degrades the turn to uncancellable, never to a failure.+4. A cancelled subagent wait is not re-dispatched: the next turn runs+ normally with no `subagent.called`.+5. With no cancel resumption, behavior is unchanged (all pre-existing unit+ and integration suites pass).++## Testing++- **Integration** (`execution/turn-cancellation.integration.test.ts`): real+ `workflowEntry` + `turnWorkflow` over world-local with a hanging+ `wait_for_cancel` tool; covers mid-tool cancel (with a matching `turnId`+ guard) + follow-up turn, a stale-guard cancel consumed as a no-op with+ the turn still cancellable afterwards, cancel during an in-flight+ subagent wait (via the mock model's `Delegate to a subagent: …`+ directive) + no re-dispatch, late/duplicate cancel no-ops, the no-retry+ canary, and cancel-hook sweep after each settle.+- **Unit**: `createTurnCancellationControl` (token derivation, claim+ conflict → uncancellable, guard matching and stale-guard skip,+ idempotent disposal), cancelled-turn arm of `turnWorkflow` (park ++ `cancelled` marker, `canPark` bypass, signal threading,+ dispose-before-result ordering), deferred control-hook disposal,+ `activeTurnId`, `emitCancelledTurn` (event order, turn-id+ reconstruction, state advance), `createTurnCancelledEvent`,+ stream-version pin.+- **E2E stays shelved** until layer 2 provides an HTTP trigger.++## Out of scope++- `POST /eve/v1/session/:id/cancel`, caller-supplied cancellation reasons,+ channel/client/eval APIs (layers 2 and 4). No token discovery is needed:+ the route derives `{sessionId}:cancel` directly.+- Descendant cascade — local subagent inbox propagation and remote cancel+ (layer 3). Layer 1 only discards their late results.+- Cancelling parked sessions or session-scoped cancellation+ (`research/channel-session-reset.md`).+- The legacy non-turn-inbox workflow path.++## Delivery++Shipped in one PR with a **patch** changeset (additive `turn.cancelled`+protocol event, stream version 18 → 19; no breaking public API). Scope+decisions settled at review: the runtime-action wait arm is **in**, and the+cancellation reason is canonical-only until layer 2.diff --git a/research/turn-cancellation-upstream-spike.md b/research/turn-cancellation-upstream-spike.mdnew file mode 100644index 000000000..b0ff72d9b--- /dev/null+++ b/research/turn-cancellation-upstream-spike.md@@ -0,0 +1,163 @@+---+issue: https://github.com/vercel/eve/issues/483+last_updated: "2026-07-02"+status: complete+---++# Turn cancellation: upstream Workflow spike (layers 1–2 de-risk)++## Verdict++| Blocker | Status on vendored `@workflow/core` 5.0.0-beta.26 | Consequence |+| --------------------------------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------ |+| Duplicate step execution under one correlation id (layer-1 finalizer shape) | **Not reproduced — 5/5 clean** | **Layer 1 is unblocked. Proceed.** |+| Hook token reuse after dispose, within one run | **Reproduced — 5/5 deterministic self-conflict** | Layer 2 blocked for naive same-token designs |+| Hook token reuse after dispose, across sequential runs | **Reproduced — 4/5 flaky conflict** | Layer 2 blocked for session-scoped cancel tokens |++Layer 2 has two viable paths that do not wait on upstream: a verified+in-workflow barrier, and eve's existing indexed-token pattern (see+"Paths forward"). Upstream fixes remain worth filing; the event-log+evidence below is report-ready.++## Method++Integration-tier repros against a real `@workflow/world-local` world and the+production workflow bundle pipeline (same harness as+`workflow-entry.integration.test.ts`):++- fixtures: `packages/eve/src/internal/testing/cancellation-spike-workflow.ts`+- tests: `packages/eve/src/internal/testing/cancellation-spike.integration.test.ts`+- run: `pnpm --filter eve exec vitest run --config vitest.integration.config.ts src/internal/testing/cancellation-spike.integration.test.ts`++Versions: `@workflow/core` 5.0.0-beta.26, `@workflow/world-local`+5.0.0-beta.22 (test world), `@workflow/world` 5.0.0-beta.14. Results below are+from 5+ consecutive suite invocations.++## Finding 1: the layer-1 shape works end to end (5/5)++The exact mechanics layer 1 needs, in one workflow:++```ts+const controller = new AbortController(); // durable, hook-backed in the workflow VM+const cancelHook = createHook({ token: `${workflowRunId}:spike-cancel` });+const stepPromise = spikeWaitStep({ abortSignal: controller.signal, ... });+const winner = await Promise.race([stepPromise, cancelIterator.next()]);+// hook wins → controller.abort(reason) → in-flight step's signal aborts in real time+await stepPromise; // settles (return or FatalError throw)+await spikeFinalizerStep(); // the cancelled-turn epilogue stand-in+```++Verified, deterministically, in both step-settlement modes (return-on-abort and+throw-`FatalError`-on-abort):++- the serialized `AbortSignal` crosses the `"use step"` boundary and aborts the+ **in-flight** step in real time (abort stream, not just replay state);+- the raced-and-abandoned hook read does not corrupt the run;+- the finalizer step executes **exactly once** (asserted two ways: step+ side-effect log lines, and an event-log scan proving no correlation id+ records more than one `step_started`);+- the workflow completes with the expected return value; no retries fire for+ the aborted step.++The #347-era "duplicate finalizer-step execution" does not reproduce under the+planned layer-1 shape. Residual risk: #347's failure was observed in the full+turnWorkflow (runtime-action waits, world-postgres in CI); layer 1's PR should+keep the `expectNoDuplicateStepStarts`-style assertion in its integration+tests as a canary.++Implementation notes captured for layer 1:++- `FatalError` prefixes its name into the propagated message+ (`"FatalError: <msg>"`); match cancellation by a stable marker, never by+ exact message equality (consistent with layer 0's name-based+ `TurnCancelledError` matching).+- Workflow-VM errors fail `instanceof Error` in test code (cross-realm);+ inspect `error.name` via property access.++## Finding 2: same-token hook reuse within one run self-conflicts (5/5)++Shape: `createHook(T)` → receive → `iterator.return()` → `dispose()` →+`createHook(T)` again in the next loop iteration (the naive per-turn+`${continuationToken}:cancel` cycle). Round 1 fails every time:++```text+HookConflictError: Hook token "wrun_…:spike-reuse" is already in use by+another workflow (run "wrun_…") ← conflictingRunId = the same run+```++Event-log smoking gun (one run, chronological):++```text+hook_created corr=…RM01 token …:spike-reuse+hook_received corr=…RM01 token …:spike-reuse+hook_conflict corr=…RM02 token …:spike-reuse conflictingRunId=<same run>+hook_disposed corr=…RM01 token …:spike-reuse+```++Workflow code called `dispose()` on hook RM01 **before** creating RM02, yet+the engine validated RM02's registration before persisting RM01's disposal —+`hook_conflict` lands ahead of `hook_disposed` in the same run's log. The+suspension flush orders same-suspension hook creations ahead of pending+disposals (or storage claims are checked before disposal intents drain).++This is distinct from (and survives the fix for) upstream+[workflow#2283](https://github.com/vercel/workflow/issues/2283), which made+duplicate processing of the _same_ `hook_created` idempotent. Here two+_different_ hooks reuse one token across a dispose that the engine reorders.++## Finding 3: cross-run token reclaim races dispose persistence (4/5)++Shape: run N claims token T (via `getConflict()`), receives one payload,+disposes, completes; run N+1 starts immediately and claims T. Five rounds per+test; failed 4 of 5 invocations, at varying rounds:++```text+run A: hook_created → hook_received → hook_disposed (all persisted, in order)+run B: hook_conflict token T conflictingRunId=<run A>+```++Run A's `hook_disposed` event exists, but run B's claim still conflicts+against A — the world's token-claim constraint is not released in the same+ordering domain as the disposal event, so a fast next claimant observes a+stale claim. This is the exact "fast descendant resumes" timing eve already+documents at `packages/eve/src/execution/workflow-entry.ts:162-166`.++## Paths forward++**Layer 1 (proceed now).** Durable `AbortController` in `turnWorkflow`, cancel+race, cancelled-turn finalizer, `turn.cancelled` → `session.waiting`. No+upstream dependency. Plug into the layer-0 seam (`TurnStepInput.abortSignal`).+Layer 1 creates its cancel observation hook once per turnWorkflow run with a+run-scoped token — no token reuse, so findings 2–3 do not apply.++**Layer 2 (two upstream-independent options, in preference order):**++1. **Indexed per-turn cancel tokens** — `${sessionId}:cancel:${turnIndex}`,+ mirroring the existing turn-control token workaround. No token is ever+ reused, sidestepping both repros entirely. The HTTP cancel route must+ discover the active index; the driver already stamps `$eve.*` run+ attributes, which can carry the live turn index/token.+2. **Barrier after dispose** — a `"use step"` suspension between `dispose()`+ and the next same-token `createHook` flushes the disposal (verified 3/3+ green in the spike's `interposeStep` variant). Works within one run;+ does **not** help the cross-run race, so it only suits designs where one+ long-lived run owns the token.++**Upstream (filed; do not block on them):**++1. [workflow#2777](https://github.com/vercel/workflow/issues/2777) — same-run+ suspension flush must order `hook_disposed` ahead of subsequent same-token+ `hook_created` validation (finding 2).+2. [workflow#2778](https://github.com/vercel/workflow/issues/2778) —+ token-claim release must be atomic/ordered with `hook_disposed`+ persistence so a new claimant cannot conflict against an already-disposed+ hook (finding 3).++## Spike artifacts++Throwaway; not merged. If kept, convert findings 2–3 tests to `it.fails` so+they self-signal when upstream fixes land, and keep the layer-1 tests as+permanent regression coverage for the layer-1 PR.++- `packages/eve/src/internal/testing/cancellation-spike-workflow.ts`+- `packages/eve/src/internal/testing/cancellation-spike.integration.test.ts`