The Open Framework for Building Agents
fix(eve): settle idle approval response deliveries (#3762)
OwenKephartbd71df906502
diff --git a/.changeset/approval-response-idle-settlement.md b/.changeset/approval-response-idle-settlement.mdnew file mode 100644index 000000000..b2d7e67ce--- /dev/null+++ b/.changeset/approval-response-idle-settlement.md@@ -0,0 +1,5 @@+---+"eve": patch+---++Finish refused approval-response deliveries without leaving clients streaming, and let the dev TUI retry refused approvals. The default reducer now keeps approval, question, and session-limit prompts pending until server confirmation; frontends should disable response controls while the store is `submitted`, `streaming`, or `resuming` instead of relying on the prompt disappearing immediately.diff --git a/docs/guides/frontend/overview.mdx b/docs/guides/frontend/overview.mdxindex 024e1d23c..b0db1f2da 100644--- a/docs/guides/frontend/overview.mdx+++ b/docs/guides/frontend/overview.mdx@@ -229,8 +229,10 @@ const pendingRequests = agent.data.messagesreturn request ? [request] : [];});+const inputDisabled = ["submitted", "streaming", "resuming"].includes(agent.status);+return pendingRequests.map((request) => (- <fieldset key={request.requestId}>+ <fieldset disabled={inputDisabled} key={request.requestId}><legend>{request.kind === "tool-approval"? "Approval required"@@ -252,7 +254,7 @@ return pendingRequests.map((request) => ());```-For a question with `allowFreeform`, render a text input and send `{ requestId, text }`. The default reducer marks each matching part as responded immediately. Approved tools update again when eve streams their result.+For a question with `allowFreeform`, render a text input and send `{ requestId, text }`. The default reducer keeps each request pending until the server confirms its resolution through `input.resolved` or `approval.settled`. Disable input controls while the response is being processed; a refused approval attempt leaves the request answerable. Approved tools update again when eve streams their result.## Authorization prompts@@ -348,7 +350,7 @@ const agent = useEveAgent({ reducer: toolCounter });// agent.data is ToolLog```-`reduce(data, event)` receives both authoritative eve stream events and client projection events (`client.message.submitted`, `client.message.failed`, `client.input.responded`). `client.input.responded` updates the submitting UI immediately; the durable `input.resolved` event later confirms the server-accepted outcome and lets replayed history rebuild the same HITL state. Return `data` unchanged for events your reducer does not handle.+`reduce(data, event)` receives both authoritative eve stream events and client projection events (`client.message.submitted`, `client.message.failed`, `client.input.responded`). `client.input.responded` notifies custom reducers that an answer was submitted, not that it was accepted. The default reducer waits for server settlement; the durable `input.resolved` event records the server-accepted outcome and lets replayed history rebuild the same HITL state. Return `data` unchanged for events your reducer does not handle.## Resumable sessionsdiff --git a/docs/tools/human-in-the-loop.md b/docs/tools/human-in-the-loop.mdindex b85147033..32bedf709 100644--- a/docs/tools/human-in-the-loop.md+++ b/docs/tools/human-in-the-loop.md@@ -121,6 +121,8 @@ The `response` policy receives:Return `{ status: "allowed" }` to accept the approval. Return `{ status: "rejected", reason }` to leave the shared request pending so another eligible responder can approve it.+When a response is refused without starting a turn, the session returns to `session.waiting`. The client finishes the submission and keeps the approval prompt answerable. Submitting an answer does not confirm approval: `approval.settled` or `input.resolved` records the server's decision. You can inspect `approval.candidate` events for the response policy's refusal reason.+### Skipping approval for schedule-dispatched turns`session.auth.current` identifies the caller of this turn. Markdown schedules use the app principal (`authenticator: "app"`, `principalId: "eve:app"`, `principalType: "runtime"`) automatically. A `run` schedule must pass its `appAuth` to `send(...)` for the child session to use that principal. Match all three fields to skip approval for automated turns while still prompting when a person calls the same tool:@@ -197,6 +199,8 @@ Channels turn requests into native UI: the Slack adapter renders approvals as buFrom your own frontend, scan all messages for pending requests and answer through the same session — see [Building a frontend](/docs/guides/frontend/overview#human-in-the-loop-prompts) for the client-side reducer and `inputResponses` shape.+The default message reducer waits for server confirmation before marking any input request answered. Use the store's `submitted` or `streaming` status to show that a response is being processed; submitting a response alone does not resolve an approval, question, or session-limit prompt. The `client.input.responded` event remains a submission notification for custom reducers, not confirmation that the server accepted the answer.+## What to read next- [Tools](/docs/tools): define the typed actions an approval gatesdiff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/approval-response-settlement.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/approval-response-settlement.eval.tsnew file mode 100644index 000000000..6fc6ea6dd--- /dev/null+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/approval-response-settlement.eval.ts@@ -0,0 +1,219 @@+import {+ EveAgentStore,+ defaultMessageReducer,+ type EveDynamicToolPart,+ type EveMessageData,+} from "eve/client";+import { defineEval, type EveEvalContext } from "eve/evals";+import { equals } from "eve/evals/expect";++const ALICE = {+ "x-eve-fixture-user": "alice",+ "x-eve-fixture-model": "continuation",+};+const TOOL = "authorized-change";++type Store = EveAgentStore<EveMessageData>;++function toolPart(store: Store, callId: string): EveDynamicToolPart {+ const parts = store.snapshot.data.messages.flatMap((message) => message.parts);+ const matches = parts.filter(+ (part): part is EveDynamicToolPart =>+ part.type === "dynamic-tool" && part.toolCallId === callId,+ );+ if (matches.length !== 1) throw new Error(`Expected one projected tool part for ${callId}.`);+ return matches[0]!;+}++async function settles<T>(label: string, promise: Promise<T>): Promise<T> {+ let timer: ReturnType<typeof setTimeout> | undefined;+ try {+ return await Promise.race([+ promise,+ new Promise<never>((_, reject) => {+ timer = setTimeout(() => reject(new Error(`${label} did not settle within 20s.`)), 20_000);+ }),+ ]);+ } finally {+ clearTimeout(timer);+ }+}++async function expectAnswerable(+ t: EveEvalContext,+ store: Store,+ callId: string,+ requestId: string,+) {+ await t.require(store.snapshot.status, equals("ready"));+ await t.require(store.snapshot.error, equals(undefined));+ const part = toolPart(store, callId);+ await t.require(part.state, equals("approval-requested"));+ await t.require(part.approval?.id, equals(requestId));+ await t.require(part.toolMetadata?.eve?.inputRequest?.requestId, equals(requestId));+ await t.require(part.toolMetadata?.eve?.inputResponse, equals(undefined));+ const cancel = await settles("Idle store cancellation", store.cancel());+ await t.require(cancel.status, equals("no_active_turn"));+ await t.require(store.snapshot.status, equals("ready"));+}++async function refuse(t: EveEvalContext, store: Store, callId: string, requestId: string) {+ const start = store.snapshot.events.length;+ await settles(+ "Refused approval submission",+ store.send({+ headers: ALICE,+ inputResponses: [{ requestId, optionId: "approve" }],+ signal: t.signal,+ }),+ );+ const events = store.snapshot.events.slice(start);+ const candidates = events+ .filter((event) => event.type === "approval.candidate")+ .filter((event) => event.data.requestId === requestId);+ await t.require(+ candidates.map((event) => event.data.outcome),+ equals(["pending", "rejected"]),+ );+ await t.require(candidates.at(-1)?.data.reason, equals("Wrong responder."));+ await t.require(+ events+ .filter((event) => event.type === "approval.candidate" || event.type === "session.waiting")+ .map((event) => event.type),+ equals(["approval.candidate", "approval.candidate", "session.waiting"]),+ );+ await t.require(events.at(-1)?.type, equals("session.waiting"));+ await t.require(+ events.filter(+ (event) =>+ event.type === "turn.started" ||+ event.type === "input.resolved" ||+ event.type === "approval.settled" ||+ event.type === "action.result",+ ).length,+ equals(0),+ );+ await expectAnswerable(t, store, callId, requestId);+ return candidates.at(-1)?.data.candidateId;+}++export default defineEval({+ description:+ "A refused approval returns the frontend to an answerable idle state, including after reload (#3757).",+ tags: ["hitl", "regression", "authorization", "client"],+ timeoutMs: 120_000,+ async test(t) {+ const host = `https://${crypto.randomUUID()}.invalid`;+ const originalFetch = globalThis.fetch;+ const stores: Store[] = [];+ // Keep deployment credentials in the eval target while exercising the real client transport.+ globalThis.fetch = async (input, init) => {+ const url = new URL(input instanceof Request ? input.url : String(input));+ return url.origin === host+ ? t.target.fetch(`${url.pathname}${url.search}`, init)+ : originalFetch(input, init);+ };++ try {+ for (const decision of ["approve", "cancel"] as const) {+ t.log(+ `Alice prepares a change; after a refused answer and reload, finish with ${decision}.`,+ );+ const session = await t.session({ headers: ALICE });+ const page = new EveAgentStore({+ host,+ headers: ALICE,+ initialSession: session.state,+ reducer: defaultMessageReducer(),+ });+ stores.push(page);+ await settles(+ "Initial approval request",+ page.send({ message: "Prepare an authorized change.", signal: t.signal }),+ );+ const parked = (await t.target.watchTurn(session.sessionId).result()).expectOk();+ const approval = parked.session.requireInputRequest({ toolName: TOOL });+ const { requestId } = approval;+ const callId = approval.action.callId;+ await expectAnswerable(t, page, callId, requestId);++ const firstCandidate = await refuse(t, page, callId, requestId);+ // Retrying on the same page must reach the policy again, not reuse the refused candidate.+ const retryCandidate = await refuse(t, page, callId, requestId);+ await t.require(retryCandidate !== firstCandidate, equals(true));+ page.reset();++ const reload = new EveAgentStore({+ host,+ headers: ALICE,+ initialSession: { sessionId: session.sessionId, streamIndex: 0 },+ reducer: defaultMessageReducer(),+ });+ stores.push(reload);+ await settles("Reload resume", reload.resume());+ await expectAnswerable(t, reload, callId, requestId);+ await t.require(+ reload.snapshot.events.filter(+ (event) =>+ event.type === "approval.candidate" &&+ event.data.requestId === requestId &&+ event.data.outcome === "rejected",+ ).length,+ equals(2),+ );+ const reloadedCandidate = await refuse(t, reload, callId, requestId);+ await t.require(reloadedCandidate !== retryCandidate, equals(true));++ t.log(+ decision === "approve"+ ? "The authorized responder approves Alice's still-pending change."+ : "Alice cancels her still-pending change instead of approving it.",+ );+ const startIndex = reload.snapshot.session!.streamIndex;+ await settles(+ "Final approval decision",+ reload.send({+ headers:+ decision === "approve" ? { "x-eve-fixture-user": "e2e-approval-responder" } : ALICE,+ inputResponses: [{ requestId, optionId: decision }],+ signal: t.signal,+ }),+ );+ await t.require(reload.snapshot.status, equals("ready"));+ await t.require(reload.snapshot.error, equals(undefined));+ const finished = (+ await t.target.watchTurn(session.sessionId, { startIndex }).result()+ ).expectOk();+ finished.event("approval.settled", {+ data: { requestId, outcome: decision === "approve" ? "approved" : "cancelled" },+ count: 1,+ });+ finished.event("input.resolved", {+ data: {+ resolutions: (items) =>+ items.some(+ (item) =>+ item.requestId === requestId &&+ item.outcome === (decision === "approve" ? "approved" : "denied"),+ ),+ },+ count: 1,+ });+ if (decision === "approve") {+ finished.calledTool(TOOL, { status: "completed", output: { executions: 1 }, count: 1 });+ } else {+ finished.event("action.result", {+ data: { result: { callId, toolName: TOOL }, status: "rejected" },+ count: 1,+ });+ finished.calledTool(TOOL, { status: "completed", count: 0 });+ }+ await t.require(toolPart(reload, callId).state === "approval-requested", equals(false));+ reload.reset();+ }+ } finally {+ for (const store of stores) store.reset();+ globalThis.fetch = originalFetch;+ }+ },+});diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.tsindex 78367ae8c..af138d1b3 100644--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.ts+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/text-approve.eval.ts@@ -23,9 +23,12 @@ export default defineEval({data: { outcome: "rejected", requestId: approval.requestId },});- const approved = await rejectedTurn.session.respond([- { optionId: "approve", requestId: approval.requestId },- ]);+ // Finish consuming the refusal boundary before opening the next response reader.+ (await rejectedTurn.result()).expectOk();+ const approved = await rejectedTurn.session.respond(+ [{ optionId: "approve", requestId: approval.requestId }],+ { headers: { "x-eve-fixture-user": "e2e-approval-responder" } },+ );approved.expectOk();approved.event("approval.settled", {count: 1,diff --git a/packages/eve/src/cli/dev/tui/runner.test.ts b/packages/eve/src/cli/dev/tui/runner.test.tsindex efd22aae2..fe45b0c9a 100644--- a/packages/eve/src/cli/dev/tui/runner.test.ts+++ b/packages/eve/src/cli/dev/tui/runner.test.ts@@ -2242,6 +2242,65 @@ describe("EveTUIRunner initial input", () => {});describe("EveTUIRunner native continuation state", () => {+ it.each(["rejected", "failed", "timed-out"])(+ "offers a retry after a %s approval attempt",+ async (outcome) => {+ const prompts: Array<string | undefined> = ["Prepare Alice's task", undefined];+ const session = sessionYieldingTurns([+ [+ {+ type: "input.requested",+ data: {+ requests: [+ {+ action: { callId: "call-1", input: {}, kind: "tool-call", toolName: "save_note" },+ kind: "tool-approval",+ requestId: "request-1",+ prompt: "Approve Alice's note?",+ options: [+ { id: "approve", label: "Approve" },+ { id: "cancel", label: "Cancel" },+ ],+ },+ ],+ },+ },+ { type: "session.waiting" },+ ],+ [+ { type: "approval.candidate", data: { requestId: "request-1", outcome } },+ { type: "session.waiting" },+ ],+ [+ { type: "approval.settled", data: { requestId: "request-1", outcome: "cancelled" } },+ {+ type: "input.resolved",+ data: { resolutions: [{ requestId: "request-1", outcome: "cancelled" }] },+ },+ { type: "session.waiting" },+ ],+ ]);+ const readToolApproval = vi+ .fn()+ .mockResolvedValueOnce({ approved: true })+ .mockResolvedValueOnce({ approved: false });+ const renderer = fakeRenderer({+ readPrompt: vi.fn(async () => prompts.shift()),+ readToolApproval,+ renderStream: vi.fn(async (result) => {+ for await (const event of result.events) void event;+ }),+ });+ await new EveTUIRunner({ session, renderer }).run();+ expect(readToolApproval).toHaveBeenCalledTimes(2);+ expect(session.respond).toHaveBeenNthCalledWith(+ 2,+ [{ requestId: "request-1", optionId: "cancel" }],+ { signal: expect.any(AbortSignal) },+ );+ },+ );+it("continues an input request from eve-native turn state", async () => {const prompts: Array<string | undefined> = ["approve this", undefined];const session = sessionYieldingTurns([diff --git a/packages/eve/src/cli/dev/tui/runner.ts b/packages/eve/src/cli/dev/tui/runner.tsindex 56311c4dd..44792bbdb 100644--- a/packages/eve/src/cli/dev/tui/runner.ts+++ b/packages/eve/src/cli/dev/tui/runner.ts@@ -1104,7 +1104,6 @@ export class EveTUIRunner {requestId: request.approvalId,optionId: response.approved ? "approve" : "cancel",});- this.#pendingInputRequests.delete(request.approvalId);}}@@ -1125,7 +1124,6 @@ export class EveTUIRunner {if (response.optionId !== undefined) inputResponse.optionId = response.optionId;if (response.text !== undefined) inputResponse.text = response.text;responses.push(inputResponse);- this.#pendingInputRequests.delete(inputRequest.requestId);}}@@ -2422,6 +2420,30 @@ async function* eveEventsToTUIStream(break;}+ case "approval.candidate": {+ if (event.data.outcome === "pending") break;+ const request = pendingInputRequests.get(event.data.requestId);+ if (request !== undefined) upsertPendingApproval(turnState, request);+ break;+ }++ case "approval.settled":+ case "input.resolved": {+ const requestIds = new Set(+ event.type === "input.resolved"+ ? event.data.resolutions.map((resolution) => resolution.requestId)+ : [event.data.requestId],+ );+ for (const requestId of requestIds) pendingInputRequests.delete(requestId);+ turnState.pendingApprovals = turnState.pendingApprovals.filter(+ (request) => !requestIds.has(request.approvalId),+ );+ turnState.pendingQuestions = turnState.pendingQuestions.filter(+ (request) => !requestIds.has(request.requestId),+ );+ break;+ }+case "action.result": {const resultEvent = event as ActionResultStreamEvent;const result = resultEvent.data.result;diff --git a/packages/eve/src/client/eve-agent-store.test.ts b/packages/eve/src/client/eve-agent-store.test.tsindex 045ea7129..8e7ded0d9 100644--- a/packages/eve/src/client/eve-agent-store.test.ts+++ b/packages/eve/src/client/eve-agent-store.test.ts@@ -4,6 +4,8 @@ import { detachEveAgentStore, EveAgentStore } from "#client/eve-agent-store.js";import { defaultMessageReducer } from "#client/message-reducer.js";import { stampTestEvents } from "#internal/testing/events.js";import {+ createApprovalCandidateEvent,+ createInputRequestedEvent,createAuthorizationCompletedEvent,createAuthorizationRequiredEvent,createMessageAppendedEvent,@@ -443,6 +445,94 @@ describe("EveAgentStore prewarming", () => {expect(store.snapshot.session?.streamIndex).toBe(6);});+ it("keeps following background completion after a refused approval returns to waiting", async () => {+ const live = controlledStreamResponse();+ const fetchMock = vi+ .spyOn(globalThis, "fetch")+ .mockResolvedValueOnce(live.response)+ .mockResolvedValueOnce(startedResponse("approval-delivery"));+ const initialEvents = stampTestEvents([+ createInputRequestedEvent({+ requests: [+ {+ action: { kind: "tool-call", callId: "save-note", toolName: "save_note", input: {} },+ kind: "tool-approval",+ requestId: "approval-1",+ prompt: "Approve Alice's note?",+ },+ ],+ sequence: 0,+ stepIndex: 0,+ turnId: "turn-note",+ }),+ createSessionWaitingEvent(),+ ]);+ const store = createStore({+ initialSession: { sessionId: "session_1", streamIndex: initialEvents.length },+ initialEvents,+ reducer: defaultMessageReducer(),+ });+ const onEvent = vi.fn();+ store.setCallbacks({ onEvent });+ const sending = store.send({+ inputResponses: [{ requestId: "approval-1", optionId: "approve" }],+ });+ await vi.waitFor(() => expect(fetchMock).toHaveBeenCalledTimes(2));+ const refusal = stampTestEvents([+ createApprovalCandidateEvent({+ candidateId: "alice-attempt",+ requestId: "approval-1",+ responderPrincipalId: "alice",+ outcome: "rejected",+ sequence: 0,+ stepIndex: 0,+ turnId: "turn-note",+ }),+ createSessionWaitingEvent(),+ ]).map((event) => ({+ ...event,+ meta: { ...event.meta, id: `refusal-${event.meta.id}`, deliveryIds: ["approval-delivery"] },+ }));+ for (const event of refusal) live.emit(event);+ await sending;+ expect(store.snapshot.status).toBe("ready");+ expect(store.snapshot.data.messages.flatMap((message) => message.parts)).toEqual(+ expect.arrayContaining([+ expect.objectContaining({ type: "dynamic-tool", state: "approval-requested" }),+ ]),+ );++ const background = stampTestEvents([+ createTurnStartedEvent({ sequence: 1, turnId: "turn-report" }),+ createMessageCompletedEvent({+ finishReason: "stop",+ message: "Bob's background report is complete.",+ sequence: 1,+ stepIndex: 0,+ turnId: "turn-report",+ }),+ createSessionWaitingEvent(),+ ]).map((event) => ({+ ...event,+ meta: { ...event.meta, id: `background-${event.meta.id}`, deliveryIds: ["report-delivery"] },+ }));+ live.emit(background[0]!);+ await vi.waitFor(() => expect(store.snapshot.status).toBe("streaming"));+ for (const event of background.slice(1)) live.emit(event);+ await vi.waitFor(() =>+ expect(store.snapshot.events).toEqual([...initialEvents, ...refusal, ...background]),+ );+ expect(store.snapshot.status).toBe("ready");+ expect(store.snapshot.data.messages.flatMap((message) => message.parts)).toEqual(+ expect.arrayContaining([+ expect.objectContaining({ type: "text", text: "Bob's background report is complete." }),+ ]),+ );+ expect(onEvent).toHaveBeenCalledTimes(5);+ expect(fetchMock).toHaveBeenCalledTimes(2);+ expect(fetchMock.mock.calls[0]![1]?.signal?.aborted).toBe(false);+ });+it("projects a background turn that arrives while another message is being accepted", async () => {const live = controlledStreamResponse();const accepted = Promise.withResolvers<Response>();diff --git a/packages/eve/src/client/message-reducer.test.ts b/packages/eve/src/client/message-reducer.test.tsindex 23126605b..dd0c00bf4 100644--- a/packages/eve/src/client/message-reducer.test.ts+++ b/packages/eve/src/client/message-reducer.test.ts@@ -803,9 +803,58 @@ describe("defaultMessageReducer", () => {type: "client.input.responded",});- expect(findToolPart(data, "call_1")).toMatchObject({ state: "approval-responded" });+ expect(findToolPart(data, "call_1")).toMatchObject({ state: "approval-requested" });});+ it.each(["tool-approval", "question", "session-limit"] as const)(+ "waits for authoritative resolution of a submitted %s response",+ (kind) => {+ const reducer = defaultMessageReducer();+ const requested = reduceServerEvents(reducer, reducer.initial(), [+ createInputRequestedEvent({+ requests: [+ {+ action: { callId: "call_1", input: {}, kind: "tool-call", toolName: "ask_question" },+ kind,+ prompt: "Continue Alice's task?",+ requestId: "request_1",+ },+ ],+ sequence: 0,+ stepIndex: 0,+ turnId: "turn_1",+ }),+ ]);+ const response = { requestId: "request_1", optionId: "continue" };+ const submitted = reducer.reduce(requested, {+ type: "client.input.responded",+ data: { createdAt: 1, responses: [response] },+ });+ expect(submitted).toBe(requested);+ expect(findToolPart(submitted, "call_1")).toMatchObject({ state: "approval-requested" });+ expect(findToolPart(submitted, "call_1")?.toolMetadata?.eve?.inputResponse).toBeUndefined();+ const resolved = reduceServerEvents(reducer, submitted, [+ createInputResolvedEvent({+ resolutions: [+ {+ kind,+ outcome: kind === "tool-approval" ? "approved" : "answered",+ requestId: "request_1",+ response,+ },+ ],+ sequence: 0,+ stepIndex: 0,+ turnId: "turn_1",+ }),+ ]);+ expect(findToolPart(resolved, "call_1")).toMatchObject({+ state: "approval-responded",+ toolMetadata: { eve: { inputResponse: response } },+ });+ },+ );+it("projects authoritative input resolutions from replayed server events", () => {const reducer = defaultMessageReducer();const data = reduceServerEvents(reducer, reducer.initial(), [@@ -916,85 +965,70 @@ describe("defaultMessageReducer", () => {});});- it("marks input requests as responded when the client submits a response", () => {- const reducer = defaultMessageReducer();- let data = reduceServerEvents(reducer, reducer.initial(), [- createInputRequestedEvent({- requests: [- {- action: {- callId: "call_1",- input: { command: "pwd" },- kind: "tool-call",- toolName: "bash",+ it.each(["rejected", "failed", "timed-out", "stale"] as const)(+ "keeps an approval answerable after a %s candidate",+ (outcome) => {+ const reducer = defaultMessageReducer();+ let data = reduceServerEvents(reducer, reducer.initial(), [+ createInputRequestedEvent({+ requests: [+ {+ action: { callId: "call_1", input: {}, kind: "tool-call", toolName: "save_note" },+ display: "confirmation",+ kind: "tool-approval",+ options: [+ { id: "approve", label: "Approve" },+ { id: "cancel", label: "Cancel" },+ ],+ prompt: "Save the note?",+ requestId: "approval_1",},- display: "confirmation",- kind: "tool-approval",- options: [- { id: "approve", label: "Yes", style: "primary" },- { id: "deny", label: "No", style: "danger" },- ],- prompt: "Approve tool call: bash",+ ],+ sequence: 0,+ stepIndex: 0,+ turnId: "turn_1",+ }),+ ]);+ data = reducer.reduce(data, {+ data: { createdAt: 1, responses: [{ optionId: "approve", requestId: "approval_1" }] },+ type: "client.input.responded",+ });+ expect(findToolPart(data, "call_1")).toMatchObject({ state: "approval-requested" });+ data = reduceServerEvents(reducer, data, [+ {+ type: "approval.candidate",+ data: {+ candidateId: "candidate_1",requestId: "approval_1",+ responderPrincipalId: "alice",+ outcome,+ sequence: 0,+ stepIndex: 0,+ turnId: "turn_1",},- ],- sequence: 0,- stepIndex: 0,- turnId: "turn_1",- }),- ]);-- data = reducer.reduce(data, {- data: {- createdAt: 1,- responses: [{ optionId: "deny", requestId: "approval_1" }],- },- type: "client.input.responded",- });-- expect(data.messages).toEqual([- {- id: "turn_1:assistant",- metadata: {- status: "streaming",- turnId: "turn_1",},- parts: [- { type: "step-start" },- {- approval: {- id: "approval_1",- },- input: { command: "pwd" },- state: "approval-responded",+ ]);+ expect(findToolPart(data, "call_1")).toMatchObject({ state: "approval-requested" });+ expect(findToolPart(data, "call_1")?.toolMetadata?.eve?.inputResponse).toBeUndefined();+ data = reduceServerEvents(reducer, data, [+ {+ type: "approval.settled",+ data: {+ requestId: "approval_1",+ responderPrincipalId: "bob",+ outcome: "approved",+ sequence: 0,stepIndex: 0,- toolCallId: "call_1",- toolMetadata: {- eve: {- inputRequest: {- allowFreeform: undefined,- display: "confirmation",- kind: "tool-approval",- options: [- { id: "approve", label: "Yes", style: "primary" },- { id: "deny", label: "No", style: "danger" },- ],- prompt: "Approve tool call: bash",- requestId: "approval_1",- },- inputResponse: { optionId: "deny", requestId: "approval_1" },- kind: "tool-call",- name: "bash",- },- },- toolName: "bash",- type: "dynamic-tool",+ turnId: "turn_1",},- ],- role: "assistant",- },- ]);- });+ },+ ]);+ expect(findToolPart(data, "call_1")).toMatchObject({+ state: "approval-responded",+ approval: { approved: true },+ });+ },+ );it("merges resumed approval results back into the requested tool part", () => {const reducer = defaultMessageReducer();@@ -1032,6 +1066,19 @@ describe("defaultMessageReducer", () => {type: "client.input.responded",});data = reduceServerEvents(reducer, data, [+ createInputResolvedEvent({+ resolutions: [+ {+ kind: "tool-approval",+ outcome: "approved",+ requestId: "approval_1",+ response: { optionId: "approve", requestId: "approval_1" },+ },+ ],+ sequence: 1,+ stepIndex: 0,+ turnId: "turn_0",+ }),createStepStartedEvent({modelId: "openai/gpt-5.5",sequence: 1,@@ -1389,5 +1436,6 @@ function findToolPart() {return data.messages.flatMap((message) => message.parts)- .find((part) => part.type === "dynamic-tool" && part.toolCallId === toolCallId);+ .filter((part) => part.type === "dynamic-tool")+ .find((part) => part.toolCallId === toolCallId);}diff --git a/packages/eve/src/client/message-reducer.ts b/packages/eve/src/client/message-reducer.tsindex 874553f85..8b5779721 100644--- a/packages/eve/src/client/message-reducer.ts+++ b/packages/eve/src/client/message-reducer.ts@@ -95,14 +95,6 @@ function reduceMessageData(data: EveMessageData, event: EveAgentReducerEvent): Erole: "user",});- case "client.input.responded": {- let next = data;- for (const response of event.data.responses) {- next = respondToInputRequest(next, response);- }- return next;- }-case "input.resolved": {let next = data;for (const resolution of event.data.resolutions) {diff --git a/packages/eve/src/harness/approval-delivery-coordinator.test.ts b/packages/eve/src/harness/approval-delivery-coordinator.test.tsindex 690dff48c..46e4de063 100644--- a/packages/eve/src/harness/approval-delivery-coordinator.test.ts+++ b/packages/eve/src/harness/approval-delivery-coordinator.test.ts@@ -1,4 +1,13 @@-import { describe, expect, it } from "vitest";+import { describe, expect, it, vi } from "vitest";+import { jsonSchema } from "ai";+import type { ApprovalResponsePolicy } from "#approval/definition.js";+import { ContextContainer, contextStorage } from "#context/container.js";+import { SessionKey } from "#context/keys.js";+import {+ getApprovalAuditState,+ markApprovalCandidateAuthorizationRequired,+} from "#harness/approval-candidates.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";import type { SessionAuthContext } from "#channel/types.js";import { settleDirectApprovalResponse } from "#harness/approval-candidates.js";@@ -46,6 +55,125 @@ function parkedSession(): HarnessSession {}describe("coordinateApprovalDelivery", () => {+ function authorize(session: HarnessSession, response: ApprovalResponsePolicy) {+ const ctx = new ContextContainer();+ ctx.set(SessionKey, {+ auth: { current: responder, initiator: responder },+ sessionId: session.sessionId,+ turn: { id: "turn-1", sequence: 1 },+ });+ const tool: HarnessToolDefinition = {+ name: "gate",+ description: "A gated tool.",+ inputSchema: jsonSchema({ type: "object" }),+ execute: async () => "done",+ approval: { request: () => "user-approval", response },+ };+ return contextStorage.run(ctx, () =>+ coordinateApprovalDelivery({+ now: 101,+ session,+ tools: new Map([["gate", tool]]),+ }),+ );+ }++ async function ingest(session = parkedSession()) {+ return coordinateApprovalDelivery({+ now: 100,+ session,+ stepInput: {+ attributedInputResponses: [+ { auth: responder, response: { requestId: request.requestId, optionId: "approve" } },+ ],+ },+ tools: new Map(),+ });+ }++ it.each(["rejected", "failed", "allowed"] as const)(+ "explicitly completes %s candidates only after ingestion",+ async (status) => {+ const ingested = await ingest();+ expect(ingested.kind).toBe("continue-coordination");+ expect(getApprovalAuditState(ingested.session.state).activeCandidates).toHaveLength(1);+ const response = vi.fn(() => {+ if (status === "failed") throw new Error("policy unavailable");+ return { status, reason: "Not an eligible responder." };+ });+ const result = await authorize(ingested.session, response);+ expect(response).toHaveBeenCalledOnce();+ expect(result.kind).toBe("responses-completed");+ expect(getApprovalAuditState(result.session.state).activeCandidates).toEqual([]);+ expect(getApprovalAuditState(result.session.state).candidateHistory[0]?.status).toBe(status);+ expect(result.stepInput?.inputResponses ?? []).toEqual(+ status === "allowed" ? [{ optionId: "approve", requestId: request.requestId }] : [],+ );+ const repeated = await authorize(result.session, response);+ expect(repeated.kind).toBe("continue");+ expect(response).toHaveBeenCalledOnce();+ },+ );++ it("completes an expired candidate without executing policy", async () => {+ const ingested = await ingest();+ const result = await coordinateApprovalDelivery({+ now: 600_100,+ session: ingested.session,+ tools: new Map(),+ });+ expect(result.kind).toBe("responses-completed");+ expect(getApprovalAuditState(result.session.state).candidateHistory[0]?.status).toBe(+ "timed-out",+ );+ });++ it("completes a failed authorizer timeout", async () => {+ vi.useFakeTimers();+ try {+ const ingested = await ingest();+ const pending = authorize(ingested.session, () => new Promise(() => {}));+ await vi.advanceTimersByTimeAsync(10_000);+ const result = await pending;+ expect(result.kind).toBe("responses-completed");+ expect(getApprovalAuditState(result.session.state).candidateHistory[0]?.status).toBe(+ "failed",+ );+ } finally {+ vi.useRealTimers();+ }+ });++ it("does not complete a duplicate while its candidate is active", async () => {+ const ingested = await ingest();+ const duplicate = await ingest(ingested.session);+ expect(duplicate.kind).toBe("continue");+ expect(getApprovalAuditState(duplicate.session.state).activeCandidates).toHaveLength(1);+ });++ it("keeps authorization-required candidates parked instead of completing", async () => {+ const ingested = await ingest();+ const candidate = getApprovalAuditState(ingested.session.state).activeCandidates[0]!;+ const session = {+ ...ingested.session,+ state: markApprovalCandidateAuthorizationRequired({+ candidateId: candidate.candidateId,+ state: ingested.session.state,+ authorizationChallenges: [+ {+ name: "provider",+ hookUrl: "https://example.com/callback",+ challenge: { url: "https://example.com/login" },+ },+ ],+ }),+ };+ const response = vi.fn(() => ({ status: "allowed" as const }));+ const result = await authorize(session, response);+ expect(result.kind).toBe("authorization-required");+ expect(response).not.toHaveBeenCalled();+ });+it("recovers an allowed settlement before its synthetic response is consumed", async () => {const parked = parkedSession();const settled = settleDirectApprovalResponse({diff --git a/packages/eve/src/harness/approval-delivery-coordinator.ts b/packages/eve/src/harness/approval-delivery-coordinator.tsindex 151c0251d..9339623f9 100644--- a/packages/eve/src/harness/approval-delivery-coordinator.ts+++ b/packages/eve/src/harness/approval-delivery-coordinator.ts@@ -38,7 +38,12 @@ const APPROVAL_CANDIDATE_TTL_MS = 10 * 60_000;export interface ApprovalDeliveryResult {readonly challenges: readonly AuthorizationChallenge[];readonly feedback: readonly string[];- readonly kind: "continue" | "continue-coordination" | "authorization-required" | "park";+ readonly kind:+ | "continue"+ | "continue-coordination"+ | "authorization-required"+ | "responses-completed"+ | "park";readonly session: HarnessSession;readonly stepInput?: StepInput;}@@ -146,15 +151,19 @@ export async function coordinateApprovalDelivery(input: {readonly tools: HarnessToolMap;}): Promise<ApprovalDeliveryResult> {const now = input.now ?? Date.now();- const expiredChallengeNames = getApprovalAuditState(input.session.state)- .activeCandidates.filter((candidate) => candidate.expiresAt <= now)- .flatMap(- (candidate) => candidate.authorizationChallenges?.map((challenge) => challenge.name) ?? [],- );+ const expiredCandidates = getApprovalAuditState(input.session.state).activeCandidates.filter(+ (candidate) => candidate.expiresAt <= now,+ );+ const expiredChallengeIds = expiredCandidates.flatMap(+ (candidate) =>+ candidate.authorizationChallenges?.map(+ (challenge) => challenge.attemptId ?? challenge.candidateId ?? challenge.name,+ ) ?? [],+ );const expiredState = expireApprovalCandidates({ now, state: input.session.state });let session: HarnessSession = {...input.session,- state: clearPendingAuthorization(expiredState, expiredChallengeNames),+ state: clearPendingAuthorization(expiredState, expiredChallengeIds),};const audit = getApprovalAuditState(session.state);const batches = getPendingInputBatches(session.state);@@ -331,6 +340,14 @@ export async function coordinateApprovalDelivery(input: {}const resumedStepInput = appendSettledResponses(remainingStepInput, pendingSettlements);+ // Only a terminal candidate pass completes response processing. Ingestion must+ // still commit before policy work, and live candidates still own their park.+ if (+ (didCommit || expiredCandidates.length > 0) &&+ getApprovalAuditState(session.state).activeCandidates.length === 0+ ) {+ return deliveryResult(session, resumedStepInput, "responses-completed");+ }if (pendingSettlements.length > 0) {return deliveryResult(session, resumedStepInput, "continue");}diff --git a/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts b/packages/eve/src/harness/issue-3494-adversarial.integration.test.tsindex b9b27fec6..f1486beb6 100644--- a/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts+++ b/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts@@ -1,10 +1,17 @@import { jsonSchema, simulateReadableStream } from "ai";import { MockLanguageModelV4 } from "ai/test";-import { afterAll, expect, it } from "vitest";+import { afterAll, expect, it, vi } from "vitest";+import type { ApprovalResponsePolicy } from "#approval/definition.js";+import {+ getApprovalAuditState,+ markApprovalCandidateAuthorizationRequired,+} from "#harness/approval-candidates.js";+import { setPendingAuthorization } from "#harness/authorization.js";import { z } from "zod";import { ContextContainer, contextStorage } from "#context/container.js";import { SessionKey } from "#context/keys.js";import { BackgroundToolExecutorKey } from "#harness/background-tools.js";+import { getBackgroundTasks, registerWorkflowToolRun } from "#harness/workflow-tool-runs.js";import { getHarnessEmissionState } from "#harness/emission.js";import { getPendingCoordinationBatch } from "#harness/coordination.js";import { getPendingInputBatches } from "#harness/pending-input-batches.js";@@ -26,9 +33,10 @@ afterAll(() => {function fixture(name: string,- responseAuthorized = false,+ responseAuthorized: boolean | ApprovalResponsePolicy = false,outputLimit?: number,outputSchema?: HarnessSession["outputSchema"],+ executorHasPendingTasks = false,) {const script: Reply[] = [];const events: UnstampedMessageStreamEvent[] = [];@@ -108,7 +116,13 @@ function fixture(...(toolName.startsWith("gate")? {approval: responseAuthorized- ? { request: always(), response: async () => ({ status: "allowed" as const }) }+ ? {+ request: always(),+ response:+ typeof responseAuthorized === "function"+ ? responseAuthorized+ : async () => ({ status: "allowed" as const }),+ }: always(),}: {}),@@ -161,6 +175,7 @@ function fixture(turn: { id: emission.turnId || `turn_${emission.sequence}`, sequence: emission.sequence },});ctx.set(BackgroundToolExecutorKey, {+ hasPendingTasks: () => executorHasPendingTasks,async execute({ batch, options }) {expect(batch.calls.some((call) => call.callId === options.toolCallId)).toBe(true);executions.push("background-admitted");@@ -208,6 +223,9 @@ function fixture(get session() {return session;},+ updateSession(update: (session: HarnessSession) => HarnessSession) {+ session = update(session);+ },pending: () => getPendingInputBatches(session.state).flatMap((b) => b.requests),async gate(...names: string[]) {script.push(calls(...names));@@ -534,6 +552,203 @@ it("continues after responder-authorized approval of the older batch", async ()expect(result.settledTurn?.output).toBe("FINAL");});+it.each(["rejected", "failed", "timed-out"] as const)(+ "finishes a %s response attempt without starting a turn and permits retry",+ async (outcome) => {+ let allowed = false;+ const policy = vi.fn(() => {+ if (allowed) return { status: "allowed" as const };+ if (outcome === "failed") throw new Error("Policy unavailable");+ return { status: "rejected" as const, reason: "Alice needs Bob's approval." };+ });+ const f = fixture(`response-${outcome}`, policy);+ await f.gate("gateA");+ const input = {+ attributedInputResponses: f.respond("gateA").inputResponses!.map((response) => ({+ response,+ auth: {+ attributes: {},+ authenticator: "test",+ issuer: "test",+ principalId: "alice",+ principalType: "user" as const,+ },+ })),+ };+ const start = f.events.length;+ const ingested = await f.step(input);+ expect(typeof ingested.next).toBe("function");+ expect(f.events.slice(start).map((event) => event.type)).toEqual(["approval.candidate"]);+ expect(policy).not.toHaveBeenCalled();+ const clock =+ outcome === "timed-out"+ ? vi.spyOn(Date, "now").mockReturnValue(Date.now() + 600_001)+ : undefined;+ try {+ await f.drive();+ } finally {+ clock?.mockRestore();+ }+ const events = f.events.slice(start);+ expect(events.at(-1)?.type).toBe("session.waiting");+ expect(events.filter((event) => event.type === "session.waiting")).toHaveLength(1);+ expect(events.some((event) => event.type === "turn.started")).toBe(false);+ expect(getApprovalAuditState(f.session.state).candidateHistory.at(-1)?.status).toBe(outcome);+ expect(f.pending()).toHaveLength(1);+ expect(f.executions).toEqual([]);++ allowed = true;+ f.script.push("Bob approved the task.");+ const retryStart = f.events.length;+ await f.drive(input);+ expect(f.pending()).toEqual([]);+ expect(f.executions).toEqual(["gateA"]);+ expect(+ f.events.slice(retryStart).filter((event) => event.type === "session.waiting"),+ ).toHaveLength(1);+ },+);++it.each(["persisted", "executor"] as const)(+ "finishes a refusal while unrelated background work is %s, like an ordinary conversation turn",+ async (source) => {+ const f = fixture(+ `refusal-with-background-${source}`,+ () => ({ status: "rejected", reason: "Bob must approve Alice's note." }),+ undefined,+ undefined,+ source === "executor",+ );+ await f.gate("gateA");+ if (source === "persisted") {+ f.updateSession((session) =>+ registerWorkflowToolRun(session, {+ callId: "report-call",+ toolName: "report",+ lifetime: "session",+ origin: { turnId: "report-turn", stepIndex: 0 },+ address: { runId: "report-run", hookToken: "report-hook" },+ task: {+ taskId: "report-task",+ metadata: { kind: "workflow", name: "report" },+ dispatchContext: { auth: { current: null, initiator: null } },+ },+ }),+ );+ }+ const ordinaryStart = f.events.length;+ f.script.push("The report is still running.");+ await f.drive({ message: "What is the report's status?" });+ expect(f.events.slice(ordinaryStart).at(-1)?.type).toBe("session.waiting");++ const responseStart = f.events.length;+ await f.drive({+ attributedInputResponses: f.respond("gateA").inputResponses!.map((response) => ({+ response,+ auth: {+ attributes: {},+ authenticator: "test",+ issuer: "test",+ principalId: "alice",+ principalType: "user" as const,+ },+ })),+ });+ expect(f.events.slice(responseStart).map((event) => event.type)).toEqual([+ "approval.candidate",+ "approval.candidate",+ "session.waiting",+ ]);+ expect(f.pending()).toHaveLength(1);+ expect(f.executions).toEqual([]);+ if (source === "persisted") {+ expect(getBackgroundTasks(f.session.state).query({ state: "working" })).toHaveLength(1);+ }+ },+);++it("does not announce waiting while a candidate needs sign-in, then completes on expiry", async () => {+ const policy = vi.fn(() => ({ status: "allowed" as const }));+ const f = fixture("candidate-sign-in", policy);+ await f.gate("gateA");+ await f.step({+ attributedInputResponses: f.respond("gateA").inputResponses!.map((response) => ({+ response,+ auth: {+ attributes: {},+ authenticator: "test",+ issuer: "test",+ principalId: "alice",+ principalType: "user" as const,+ },+ })),+ });+ const candidate = getApprovalAuditState(f.session.state).activeCandidates[0]!;+ const challenges = [+ {+ candidateId: candidate.candidateId,+ name: "notes",+ hookUrl: "https://example.com/callback",+ challenge: { url: "https://example.com/sign-in" },+ },+ ];+ f.updateSession((session) => ({+ ...session,+ state: setPendingAuthorization(+ markApprovalCandidateAuthorizationRequired({+ state: session.state,+ candidateId: candidate.candidateId,+ authorizationChallenges: challenges,+ }),+ { challenges },+ ),+ }));+ const start = f.events.length;+ await f.drive();+ expect(f.events.slice(start)).toEqual([]);+ expect(policy).not.toHaveBeenCalled();+ const clock = vi.spyOn(Date, "now").mockReturnValue(candidate.expiresAt + 1);+ try {+ await f.drive();+ } finally {+ clock.mockRestore();+ }+ expect(f.events.slice(start).map((event) => event.type)).toEqual([+ "authorization.completed",+ "approval.candidate",+ "session.waiting",+ ]);+ expect(f.pending()).toHaveLength(1);+});++it("uses the normal turn boundary for a mixed accepted and refused delivery", async () => {+ const f = fixture("mixed-response", ({ request }) =>+ request.toolName === "gateA"+ ? { status: "allowed" }+ : { status: "rejected", reason: "Bob must approve this task." },+ );+ await f.gate("gateA");+ f.script.push(calls("gateB"));+ await f.drive({ message: "Prepare Bob's independent task." });+ const start = f.events.length;+ f.script.push("Alice's task is complete.");+ await f.drive({+ attributedInputResponses: f.pending().map((request) => ({+ response: { requestId: request.requestId, optionId: "approve" },+ auth: {+ attributes: {},+ authenticator: "test",+ issuer: "test",+ principalId: "alice",+ principalType: "user" as const,+ },+ })),+ });+ expect(f.executions).toEqual(["gateA"]);+ expect(f.pending().map((request) => request.action.toolName)).toEqual(["gateB"]);+ expect(f.events.slice(start).filter((event) => event.type === "session.waiting")).toHaveLength(1);+});+it("resumes a complete batch while another batch has only a partial approval [control]", async () => {const f = fixture("complete-and-partial-approvals");// Given A and B need approval together, and a second independent B is pending.diff --git a/packages/eve/src/harness/tool-loop.ts b/packages/eve/src/harness/tool-loop.tsindex 9deae03ab..bc6d10c13 100644--- a/packages/eve/src/harness/tool-loop.ts+++ b/packages/eve/src/harness/tool-loop.ts@@ -163,6 +163,7 @@ import {} from "#harness/messages.js";import { normalizeProviderToolHistory } from "#harness/provider-tool-history.js";import {+ getPendingAuthorization,getSupersededAuthorizationChallenges,setPendingAuthorization,} from "#harness/authorization.js";@@ -717,6 +718,7 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {context: approvalContext,}): config.tools;+ const pendingApprovalChallenges = getPendingAuthorization(session.state)?.challenges ?? [];const coordinated = await coordinateApprovalDelivery({session,stepInput: effectiveStepInput,@@ -735,6 +737,30 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {);}const audit = getApprovalAuditState(session.state);+ for (const challenge of pendingApprovalChallenges) {+ if (+ !audit.candidateHistory.some(+ (candidate) =>+ candidate.candidateId === challenge.candidateId &&+ candidate.status === "timed-out" &&+ candidate.eventEmitted !== true,+ )+ )+ continue;+ await emit(+ createAuthorizationCompletedEvent({+ attemptId: challenge.attemptId,+ authorization: challenge.challenge,+ candidateId: challenge.candidateId,+ name: challenge.name,+ outcome: "failed",+ reason: "The approval response expired. Please submit a new response.",+ sequence: emissionState.sequence,+ stepIndex: emissionState.stepIndex,+ turnId: emissionState.turnId,+ }),+ );+ }for (const candidate of audit.activeCandidates.filter((entry) => entry.pendingEventEmitted !== true,)) {@@ -940,6 +966,15 @@ export function createToolLoopHarness(config: ToolLoopHarnessConfig): StepFn {return { next: null, session: parkedSession };}+ if (+ coordinated.kind === "responses-completed" &&+ config.mode === "conversation" &&+ isHarnessBetweenTurns(pending.session) &&+ getPendingAuthorization(pending.session.state) === undefined+ ) {+ // Like a conversation turn, a completed response may wait while background work continues.+ await emit?.(createSessionWaitingEvent());+ }return { next: null, session: pending.session };}