The Open Framework for Building Agents
fix(eve): withdraw a serve tool's pending questions when it replies (#3953)
Andrew Barbae167dd3ed355
diff --git a/.changeset/serve-reply-withdraws-questions.md b/.changeset/serve-reply-withdraws-questions.mdnew file mode 100644index 000000000..5409cdb33--- /dev/null+++ b/.changeset/serve-reply-withdraws-questions.md@@ -0,0 +1,5 @@+---+"eve": patch+---++`ctx.reply()` in a `serve` workflow tool now withdraws the `ctx.ask()` questions still pending for the calls it settles, so they resolve as `cancelled` and channels stop offering them. Before, such a question stayed pending after its call had a result: it kept steering from interrupting a model step and could take the person's next plain message as its answer.diff --git a/docs/tools/tasks.md b/docs/tools/tasks.mdindex 9387c0e1a..84c7db695 100644--- a/docs/tools/tasks.md+++ b/docs/tools/tasks.md@@ -104,7 +104,8 @@ A workflow tool body receives its signals from its context or, in a `serve` bodyevery call in one stretch of work shares one signal, and the next stretch gets a new one. Pass itto the steps and `ctx.agent` sends that should stop.- **`ctx.ask(request)`** withdraws the question when the call's `abortSignal` aborts, and- `ctx.ask(request, { signal })` also when `signal` does. The answer resolves as+ `ctx.ask(request, { signal })` also when `signal` does. In a `serve` body, `ctx.reply()` also+ withdraws the questions still pending for the calls it settles. The answer resolves as`{ status: "cancelled" }`, and the stream reports `input.resolved` with `outcome: "cancelled"` sochannels stop offering the question. A person's answer that reached the session first still wins:the ask resolves as `answered`, so branch on `status`, not on the signal.diff --git a/docs/tools/workflows.mdx b/docs/tools/workflows.mdxindex 4d3c39838..d5ddae045 100644--- a/docs/tools/workflows.mdx+++ b/docs/tools/workflows.mdx@@ -346,6 +346,8 @@ described in [Tasks](/docs/tools/tasks). What differs:Each settled call reports its own `task.settled`, and the model receives the reply once. The taskthen goes idle until its next call. A reply withno call left to settle is dropped, and `ctx.ask` throws while no call is waiting for a result.+ A reply also withdraws the questions still pending for the calls it settles, but doesn't abort+ their `abortSignal`, so work the body continues after replying keeps running.Returning settles the calls still waiting and ends the task; a throw fails them. `serve` is nota generator, so it can't `yield` progress.- **Working and idle:** a resumable task is working while one of its calls has no result, and idle@@ -448,7 +450,8 @@ The answer's `status` tells the body what happened:- `answered`: the person picked an option (`optionId`) or typed a reply (`text`).- `cancelled`: the request was withdrawn before anyone answered it, because the `signal` passed to- `ctx.ask` or the call's `abortSignal` aborted. The session emits `input.resolved` with+ `ctx.ask` or the call's `abortSignal` aborted, or because a `serve` body replied to the call. The+ session emits `input.resolved` with`outcome: "cancelled"`, so clients stop offering the question.- `unavailable`: the session cannot request input, such as a scheduled run. `ctx.ask` returns thisimmediately instead of waiting for an answer no one can give.diff --git a/packages/eve/src/execution/tools/workflow/ask.ts b/packages/eve/src/execution/tools/workflow/ask.tsindex a5e28687e..1d72e747b 100644--- a/packages/eve/src/execution/tools/workflow/ask.ts+++ b/packages/eve/src/execution/tools/workflow/ask.ts@@ -78,6 +78,8 @@ const UNAVAILABLE: ToolInputResponse = { status: "unavailable" };interface PendingAsk {readonly resolve: (response: ToolInputResponse) => void;readonly reject: (error: unknown) => void;+ /** Sends the session the request to withdraw the question; unset once sent. */+ withdraw: (() => Promise<void>) | undefined;}/**@@ -88,24 +90,59 @@ interface PendingAsk {export class WorkflowToolRunAsks {private opened = 0;private readonly pending = new Map<string, PendingAsk>();+ /** Withdrawal requests on their way to the run's inbox. */+ private readonly withdrawing = new Set<Promise<void>>();private readonly runId: string;constructor(runId: string) {this.runId = runId;}- /** Opens a question under a request ID that replays deterministically with the run. */- open(): { readonly answer: Promise<ToolInputResponse>; readonly requestId: string } {+ /**+ * Opens a question under a request ID that replays deterministically with+ * the run. `send` sends the request and returns how to send its withdrawal.+ */+ open(send: (requestId: string) => () => Promise<void>): {+ readonly answer: Promise<ToolInputResponse>;+ readonly requestId: string;+ } {this.opened += 1;const requestId = `${this.runId}-ask-${String(this.opened)}`;+ let entry!: PendingAsk;const answer = new Promise<ToolInputResponse>((resolve, reject) => {- this.pending.set(requestId, { reject, resolve });+ entry = { reject, resolve, withdraw: undefined };});+ this.pending.set(requestId, entry);+ entry.withdraw = send(requestId);return { answer, requestId };}- isPending(requestId: string): boolean {- return this.pending.has(requestId);+ /**+ * Asks the session to withdraw a pending question, at most once. The+ * question stays pending until the session decides it, so an answer the+ * session accepted first still wins.+ */+ withdraw(requestId: string): void {+ const entry = this.pending.get(requestId);+ const withdraw = entry?.withdraw;+ if (entry === undefined || withdraw === undefined) return;+ entry.withdraw = undefined;+ const sending = withdraw().catch((error: unknown) => this.fail(requestId, error));+ this.withdrawing.add(sending);+ void sending.then(() => this.withdrawing.delete(sending));+ }++ /** Asks the session to withdraw every pending question not already withdrawn. */+ withdrawAll(): void {+ for (const requestId of this.pending.keys()) this.withdraw(requestId);+ }++ /**+ * Waits until every withdrawal asked for has reached the run's inbox, so the+ * run's outcome, which drops its questions silently, cannot overtake one.+ */+ async flush(): Promise<void> {+ while (this.withdrawing.size > 0) await Promise.all(this.withdrawing);}/** Applies the session's decision. The first one wins; a repeated one is dropped. */@@ -153,25 +190,21 @@ export function ask(if (signals.some((signal) => signal.aborted)) return Promise.resolve(CANCELLED);const { asks, control, owner } = context;- const { answer, requestId } = asks.open();// A `serve` task's current call changes; the question stays the call's that asked it.const from = context.from;- const sent = owner.send({- kind: "request",- from,- replyTo: requestId,- request: { control, kind: "ask", request },+ const { answer, requestId } = asks.open((replyTo) => {+ const sent = owner.send({+ kind: "request",+ from,+ replyTo,+ request: { control, kind: "ask", request },+ });+ sent.catch((error: unknown) => asks.fail(replyTo, error));+ // The withdrawal must not overtake the request it withdraws.+ return () => sent.then(() => owner.send({ control, from, kind: "withdraw", replyTo }));});- sent.catch((error: unknown) => asks.fail(requestId, error));- const requestWithdrawal = (): void => {- if (!asks.isPending(requestId)) return;- // The withdrawal must not overtake the request it withdraws.- const withdrawal = sent.then(() =>- owner.send({ control, from, kind: "withdraw", replyTo: requestId }),- );- withdrawal.catch((error: unknown) => asks.fail(requestId, error));- };+ const requestWithdrawal = (): void => asks.withdraw(requestId);for (const signal of signals) {signal.addEventListener("abort", requestWithdrawal, { once: true });}diff --git a/packages/eve/src/execution/tools/workflow/serve.test.ts b/packages/eve/src/execution/tools/workflow/serve.test.tsindex 60119f2dc..83117fcf7 100644--- a/packages/eve/src/execution/tools/workflow/serve.test.ts+++ b/packages/eve/src/execution/tools/workflow/serve.test.ts@@ -13,6 +13,7 @@ import type {import { startServeBody } from "#execution/tools/workflow/serve.js";import { workflowToolRunWorkflow } from "#execution/tools/workflow/workflow.js";import type { JsonValue } from "#shared/json.js";+import type { ToolInputResponse } from "#tools/definition.js";import type {WorkflowServeCall,WorkflowServeContext,@@ -127,6 +128,13 @@ function aborted(signal: AbortSignal): Promise<void> {return new Promise((resolve) => signal.addEventListener("abort", () => resolve()));}+/** The messages of one kind the run sent its inbox, oldest first. */+function sent<K extends WorkflowToolRunMessage["kind"]>(kind: K) {+ return mocks.deliver.mock.calls.flatMap(([, message]) =>+ message.kind === kind ? [message as Extract<WorkflowToolRunMessage, { readonly kind: K }>] : [],+ );+}+it("serves every call to its task, one stretch of work at a time, until the session ends", async () => {const received: WorkflowServeCall<JsonValue>[] = [];const pendings: Promise<WorkflowServeCall<JsonValue>>[] = [];@@ -248,6 +256,67 @@ it("serves every call to its task, one stretch of work at a time, until the sessexpect(fourth?.aborted).toBe(true);});+it("withdraws a question once its call has a reply, without aborting the stretch", async () => {+ mocks.deliver.mockClear();+ mocks.deliver.mockResolvedValue(undefined);+ const replyNow = Promise.withResolvers<void>();+ let stretch: AbortSignal | undefined;+ let answer: PromiseLike<ToolInputResponse> | undefined;+ mocks.serve.mockImplementation(+ async (receive: WorkflowServeReceive<JsonValue>, ctx: WorkflowServeContext<JsonValue>) => {+ stretch = (await receive()).abortSignal;+ answer = ctx.ask({ prompt: "Should the plan include a rollback drill?" });+ await replyNow.promise;+ ctx.reply("plan without a rollback drill");+ await answer;+ return await receive();+ },+ );+ const started = startServeBody(input);++ await vi.waitFor(() => expect(sent("request")).toHaveLength(1));+ const requestId = sent("request")[0]?.replyTo ?? "";+ // Alice's second call arrives before the reply, so it joins the stretch and the reply leaves it working.+ started.control.apply(call("call-2", "Add a rollback step."));+ replyNow.resolve();+ await vi.waitFor(() => expect(sent("withdraw")).toHaveLength(1));++ expect(sent("withdraw")).toMatchObject([{ from: { callId: "call-1" }, replyTo: requestId }]);+ expect(stretch?.aborted).toBe(false);+ // Cancelling the second call aborts the stretch, and the question's withdrawal was already asked for.+ started.control.apply({ kind: "cancel", reason: "The task was cancelled." });+ expect(stretch?.aborted).toBe(true);+ started.asks.settle({ kind: "withdrawn", requestId });+ await expect(answer).resolves.toEqual({ status: "cancelled" });+ expect(sent("withdraw")).toHaveLength(1);+ started.control.apply({ kind: "end", reason: "The session ended." });+ await started.outcome;+});++it("holds the outcome of a body that returns right after replying until its withdrawals reach the run", async () => {+ mocks.deliver.mockClear();+ const withdrawalLands = Promise.withResolvers<void>();+ mocks.deliver.mockImplementation(async (_inbox, message) => {+ if (message.kind === "withdraw") await withdrawalLands.promise;+ });+ mocks.serve.mockImplementation(+ async (receive: WorkflowServeReceive<JsonValue>, ctx: WorkflowServeContext<JsonValue>) => {+ await receive();+ void ctx.ask({ prompt: "Should the plan include a rollback drill?" });+ ctx.reply("plan without a rollback drill");+ return null;+ },+ );+ const started = startServeBody(input);+ let finished = false;+ void started.outcome.then(() => (finished = true));++ await vi.waitFor(() => expect(sent("withdraw")).toHaveLength(1));+ expect(finished).toBe(false);+ withdrawalLands.resolve();+ await expect(started.outcome).resolves.toEqual({ output: null, status: "completed" });+});+it.each([{body: "ignores the cancel",diff --git a/packages/eve/src/execution/tools/workflow/serve.ts b/packages/eve/src/execution/tools/workflow/serve.tsindex 406d04032..ad465b2fa 100644--- a/packages/eve/src/execution/tools/workflow/serve.ts+++ b/packages/eve/src/execution/tools/workflow/serve.ts@@ -78,6 +78,8 @@ class WorkflowServeCalls implements WorkflowBodyControl {/*** The body's questions. A cancel keeps the run, so it withdraws them* through the stretch's `abortSignal`, and the session decides each one.+ * A reply withdraws them too, but leaves the signal alone: it means+ * cancelled, and the body may keep working after it replies.*/readonly asks: WorkflowToolRunAsks;private readonly run = new AbortController();@@ -188,6 +190,10 @@ class WorkflowServeCalls implements WorkflowBodyControl {if (settled.length === 0) return;this.waiting = [];if (this.arrived.length === 0) this.stretch = undefined;+ // `ctx.ask()` needs a waiting call, and a cancel already withdrew the+ // questions of the calls it settled, so every question not yet withdrawn+ // was asked for a call this settles.+ this.asks.withdrawAll();const previous = this.replies;this.replies = previous.then(() => this.deliverReply(settled, output));}@@ -199,8 +205,9 @@ class WorkflowServeCalls implements WorkflowBodyControl {);}- /** Waits for in-flight replies, so the outcome can never overtake them. */+ /** Waits for in-flight replies and withdrawals, so the outcome can never overtake them. */async flush(): Promise<void> {+ await this.asks.flush();await this.replies;}