Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

fix(eve): hold over-budget turns until the continuation answer (#4499)

Ruif3dbd0c01340
diff --git a/.changeset/held-budget-question.md b/.changeset/held-budget-question.md
new file mode 100644
index 000000000..91a93fd05
--- /dev/null
+++ b/.changeset/held-budget-question.md
@@ -0,0 +1,5 @@
+---
+"eve": patch
+---
+
+Keep an over-budget turn waiting for input until its continuation question is answered. Continue resumes the same turn, queued messages wait for the answer, and cancellation withdraws the question.
diff --git a/docs/agent-config.md b/docs/agent-config.md
index be095607e..9a123f38d 100644
--- a/docs/agent-config.md
+++ b/docs/agent-config.md
@@ -220,16 +220,24 @@ timeout. Expiration does not delete stored session data.
Input tokens, output tokens, and model token cost are checked independently.
The model call that crosses a limit is allowed to finish because exact usage
arrives after the call completes. Before the next model call, eve pauses the
-session and sends a deterministic continuation prompt with two options:
-**Approve** grants a fresh window of each configured size, and **Stop**
+turn (`turn.waiting`) and sends a deterministic continuation prompt with two options:
+**Continue** grants a fresh window of each configured size and resumes the same
+turn, and **Stop**
cancels the in-flight turn through the standard cancellation path
(`turn.cancelled` → `session.waiting`) — a user decision, not an error. The session stays resumable; because it is
-still over budget, the next message re-raises the prompt. Declining a
+still over budget after Stop, the next message raises a new prompt. Declining a
delegated child's prompt cancels the root turn, which cascades to the whole
delegation tree — the delegating parent never receives an error result it
could retry against a fresh quota share. A reply that answers neither option
is queued while the existing prompt stays pending; eve does not raise another
-copy. The reply is processed once the budget is granted.
+copy or start another turn. The reply is received (`message.received`) and
+processed in the held turn once the budget is granted.
+
+For sessions already waiting on a continuation prompt from an older deployment,
+the previous turn has completed. A non-answer message still queues behind the
+existing prompt, but emits no event until Continue is chosen. Continue starts a
+new turn and receives the queued message once; it does not reopen the completed
+turn or duplicate the prompt.
Sessions that cannot request input from a human, such as markdown schedules and
delegated runs without input proxying, skip the prompt and fail the next model
diff --git a/docs/guides/dynamic-capabilities.md b/docs/guides/dynamic-capabilities.md
index 30ebe31e3..e4d169dc8 100644
--- a/docs/guides/dynamic-capabilities.md
+++ b/docs/guides/dynamic-capabilities.md
@@ -377,7 +377,7 @@ A dynamic connection, tool, or skill whose name matches an **authored** one **ov
At `turn.started`, model, tool, skill, and subagent resolvers receive the visible conversation history and incoming message in `ctx.messages`, oldest first. Request context is included, and history projection still applies. Read these messages from the handler's second argument; the event itself contains turn metadata. Instruction resolvers use the separate snapshot described under [Dynamic instructions](#dynamic-instructions).
-A message that steers a held turn joins it without another `turn.started`, as does a completed sign-in, so `turn.started` resolvers do not see it; `step.started` resolvers do. A message sent while a session-limit prompt waits starts its own turn, so `turn.started` resolvers see it, and that turn waits for the prompt's answer before the model runs. When memory recall runs, its results appear in the projected snapshot before incoming input.
+A message that steers a held turn joins it without another `turn.started`, as does a completed sign-in, so `turn.started` resolvers do not see it; `step.started` resolvers do. A message sent while a session-limit prompt waits queues in the held turn without another `turn.started`. After Continue grants the budget, `step.started` resolvers see the queued message before the model runs. When memory recall runs, its results appear in the projected snapshot before incoming input.
### Execution order
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/README.md b/e2e/fixtures/agent-tools-hitl/evals/hitl/README.md
index 0b5606206..843be9cef 100644
--- a/e2e/fixtures/agent-tools-hitl/evals/hitl/README.md
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/README.md
@@ -40,6 +40,11 @@ and [`agent-workflow-tools/evals/hitl`](../../../agent-workflow-tools/evals/hitl
| A rejected responder leaves the approval open | [rejected-responder-leaves-approval-open](./response-policies/rejected-responder-leaves-approval-open.eval.ts) |
| An allowed responder settles the approval | [allowed-responder-settles](./response-policies/allowed-responder-settles.eval.ts) |
| A responder's authorization, then settlement | [responder-authorization-then-settles](./response-policies/responder-authorization-then-settles.eval.ts) |
+| Budget question holds the turn | [budget-question-holds-the-turn](./budget/budget-question-holds-the-turn.eval.ts) |
+| Continue resumes the same turn | [continue-runs-the-same-call](./budget/continue-runs-the-same-call.eval.ts) |
+| Message waits behind the budget question | [message-waits-behind-budget](./budget/message-waits-behind-budget.eval.ts) |
+| Typed Continue answers the budget question | [typed-continue-answers](./budget/typed-continue-answers.eval.ts) |
+| Cancel withdraws the budget question | [cancel-withdraws-budget-question](./budget/cancel-withdraws-budget-question.eval.ts) |
| Stop cancels the turn, resolving once | [stop-cancels-once](./budget/stop-cancels-once.eval.ts) |
| No step starts behind the budget question | [no-step-behind-budget](./budget/no-step-behind-budget.eval.ts) |
| A late budget answer is dropped | [stale-budget-answer-dropped](./budget/stale-budget-answer-dropped.eval.ts) |
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/budget-question-holds-the-turn.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/budget-question-holds-the-turn.eval.ts
new file mode 100644
index 000000000..7d4d75b10
--- /dev/null
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/budget-question-holds-the-turn.eval.ts
@@ -0,0 +1,29 @@
+import { defineEval } from "eve/evals";
+
+import { budgetQuestion, expectHeld, expectNoModelCallWhileOpen } from "../helpers.ts";
+
+/**
+ * Alice's session is out of output budget when she asks for a status note.
+ * Before the model call, the turn asks whether to continue and holds: it
+ * neither completes nor calls the model.
+ */
+export default defineEval({
+ description: "Running out of budget before a model call asks the person and holds the turn.",
+ tags: ["hitl", "budget"],
+ timeoutMs: 60_000,
+ async test(t) {
+ const { held, request, session } = await budgetQuestion(t);
+ expectHeld(held);
+ held.event("input.requested", {
+ count: 1,
+ data: {
+ requests: (requests) =>
+ requests.length === 1 &&
+ requests[0]?.kind === "session-limit" &&
+ requests[0]?.requestId === request.requestId,
+ },
+ });
+ held.notEvent("message.completed");
+ expectNoModelCallWhileOpen(session, request.requestId);
+ },
+});
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/cancel-withdraws-budget-question.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/cancel-withdraws-budget-question.eval.ts
new file mode 100644
index 000000000..2d93c8e15
--- /dev/null
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/cancel-withdraws-budget-question.eval.ts
@@ -0,0 +1,21 @@
+import { defineEval } from "eve/evals";
+
+import { budgetQuestion, expectResolved, follow } from "../helpers.ts";
+
+/** Alice cancels her turn while the budget question waits: it closes unanswered, as cancelled. */
+export default defineEval({
+ description: "Cancelling the turn withdraws its budget question.",
+ tags: ["hitl", "budget", "cancel"],
+ timeoutMs: 60_000,
+ async test(t) {
+ const { request, session } = await budgetQuestion(t);
+ const startIndex = session.state.streamIndex;
+
+ await session.cancel();
+ const cancelled = await follow(t, session, startIndex);
+ expectResolved(cancelled, request, "cancelled");
+ cancelled.event("input.resolved", { count: 1 });
+ cancelled.event("turn.cancelled", { count: 1 });
+ cancelled.notEvent("step.started");
+ },
+});
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/continue-runs-the-same-call.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/continue-runs-the-same-call.eval.ts
new file mode 100644
index 000000000..c443aae4e
--- /dev/null
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/continue-runs-the-same-call.eval.ts
@@ -0,0 +1,27 @@
+import { defineEval } from "eve/evals";
+
+import { REPLY, answers, asAlice, budgetQuestion, expectResolved } from "../helpers.ts";
+
+/**
+ * Alice chooses Continue. That grants a fresh budget window, and the model
+ * call the budget stopped runs in the same turn.
+ */
+export default defineEval({
+ description: "Continue grants a fresh budget and runs the stopped model call in the same turn.",
+ tags: ["hitl", "budget"],
+ timeoutMs: 60_000,
+ async test(t) {
+ const { held, request, session } = await budgetQuestion(t);
+ const turnId = held.events.find((event) => event.type === "turn.started")?.data.turnId;
+
+ const resumed = (await session.respond(answers("continue", request), asAlice)).expectOk();
+ expectResolved(resumed, request, "answered");
+ resumed.notEvent("turn.started");
+ resumed.eventOrder([
+ { type: "input.resolved" },
+ { type: "step.started", data: { turnId } },
+ { type: "message.completed", data: { message: REPLY.statusNote, turnId } },
+ { type: "turn.completed", data: { turnId } },
+ ]);
+ },
+});
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/message-waits-behind-budget.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/message-waits-behind-budget.eval.ts
new file mode 100644
index 000000000..899f8234b
--- /dev/null
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/message-waits-behind-budget.eval.ts
@@ -0,0 +1,33 @@
+import { defineEval } from "eve/evals";
+
+import { REPLY, SAY, answers, asAlice, budgetQuestion, expectHeld } from "../helpers.ts";
+
+/**
+ * While the budget question waits, Alice adds that the note should mention
+ * the deadline. Her message is queued and does not answer the
+ * question, so the turn stays held without calling the model; after Continue
+ * the model reads it with her original request.
+ *
+ * `no-step-behind-budget.eval.ts` covers the stricter rule that no step even
+ * starts meanwhile.
+ */
+export default defineEval({
+ description: "A message behind the budget question is received and read after Continue.",
+ tags: ["hitl", "budget", "steer"],
+ timeoutMs: 60_000,
+ async test(t) {
+ const { request, session } = await budgetQuestion(t);
+
+ const added = await session.send(SAY.deadline, asAlice);
+ added.notEvent("message.received");
+ expectHeld(added);
+ added.notEvent("input.resolved");
+ added.notEvent("step.completed");
+ added.notEvent("message.completed");
+
+ const resumed = (await session.respond(answers("continue", request), asAlice)).expectOk();
+ resumed.event("message.received", { count: 1, data: { message: SAY.deadline } });
+ resumed.event("message.completed", { data: { message: REPLY.statusNoteWithDeadline } });
+ resumed.notEvent("turn.started");
+ },
+});
diff --git a/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/typed-continue-answers.eval.ts b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/typed-continue-answers.eval.ts
new file mode 100644
index 000000000..37d2f0cdb
--- /dev/null
+++ b/e2e/fixtures/agent-tools-hitl/evals/hitl/budget/typed-continue-answers.eval.ts
@@ -0,0 +1,21 @@
+import { defineEval } from "eve/evals";
+
+import { REPLY, asAlice, budgetQuestion, expectResolved } from "../helpers.ts";
+
+/** Alice types "continue": it names the question's option, so it answers it like the button. */
+export default defineEval({
+ description: "A typed reply naming a budget option answers the budget question.",
+ tags: ["hitl", "budget", "text-reply"],
+ timeoutMs: 60_000,
+ async test(t) {
+ const { request, session } = await budgetQuestion(t);
+
+ const typed = (await session.send("continue", asAlice)).expectOk();
+ expectResolved(typed, request, "answered");
+ typed.event("input.resolved", {
+ data: { resolutions: [{ requestId: request.requestId, response: { optionId: "continue" } }] },
+ });
+ typed.notEvent("turn.started");
+ typed.event("message.completed", { data: { message: REPLY.statusNote } });
+ },
+});
diff --git a/packages/eve/src/execution/session-limit-cancellation.integration.test.ts b/packages/eve/src/execution/session-limit-cancellation.integration.test.ts
index 25e5c4371..a0cda52ae 100644
--- a/packages/eve/src/execution/session-limit-cancellation.integration.test.ts
+++ b/packages/eve/src/execution/session-limit-cancellation.integration.test.ts
@@ -111,7 +111,7 @@ describe("session-limit continuation decline integration", () => {
// Turn 2 parks on the continuation prompt before any model call.
await deliver(continuationToken, { message: "keep going please" });
const promptTurn = await stream.nextTurn();
- expect(promptTurn.at(-1)?.type).toBe("session.waiting");
+ expect(promptTurn.at(-1)?.type).toBe("turn.waiting");
const requestId = requestIdFromPromptTurn(promptTurn);
// Declining settles the turn as cancelled — a user decision, not an
@@ -124,7 +124,7 @@ describe("session-limit continuation decline integration", () => {
expect(declinedTurn.at(-1)?.type).toBe("session.waiting");
expect(
containsEventSequence(declinedTurn, [
- "turn.started",
+ "input.resolved",
"turn.cancelled",
"session.waiting",
]),
@@ -138,7 +138,7 @@ describe("session-limit continuation decline integration", () => {
await deliver(continuationToken, { message: "try again" });
const repromptTurn = await stream.nextTurn();
- expect(repromptTurn.at(-1)?.type).toBe("session.waiting");
+ expect(repromptTurn.at(-1)?.type).toBe("turn.waiting");
expect(filterEventsByType(repromptTurn, "input.requested")).toHaveLength(1);
expect(filterEventsByType(repromptTurn, "turn.cancelled")).toHaveLength(0);
expectNoFailureEvents(repromptTurn);
diff --git a/packages/eve/src/execution/session/next-input.ts b/packages/eve/src/execution/session/next-input.ts
index d82faf6ca..d18ca12b7 100644
--- a/packages/eve/src/execution/session/next-input.ts
+++ b/packages/eve/src/execution/session/next-input.ts
@@ -1,3 +1,5 @@
+import { storedProjection } from "#harness/session-machine/view.js";
+import { openInputs } from "#protocol/session-projection.js";
import { routeSelectedDelivery } from "#execution/session/route-selected-delivery.js";
import type {
SessionControl,
@@ -56,6 +58,13 @@ export async function nextTurnDelivery(input: {
freshSequence =
wasIdle && admitted.kind === "delivery" ? admitted.admission.sequence : undefined;
if (admitted.kind === "workflow") return { kind: "workflow", message: admitted.message };
+ if (
+ admitted.kind === "cancel" &&
+ openInputs(storedProjection(cursor.sessionState.snapshot.session.state)).some(
+ (input) => input.request.kind === "session-limit",
+ )
+ )
+ return { kind: "cancel-turn" };
if (admitted.kind === "cancel" && input.hasWorkingTasks()) {
return { kind: "cancel-working-tasks" };
}
diff --git a/packages/eve/src/harness/hitl/approvals.ts b/packages/eve/src/harness/hitl/approvals.ts
index 0c76e77ba..d27177620 100644
--- a/packages/eve/src/harness/hitl/approvals.ts
+++ b/packages/eve/src/harness/hitl/approvals.ts
@@ -45,17 +45,14 @@ import { signInWithdrawn } from "#harness/session-machine/events.js";
import {
canonicalize,
compactInput,
- hasInput,
isEmptyInput,
resolveTextInput,
withoutResponses,
- withoutTurnInput,
type ResolvedStepInput,
} from "./delivery.js";
import type { StepCoordinates, SuspendedStep, TurnState } from "#harness/session-machine/view.js";
import { withoutCalls } from "#harness/inline-tool-authorization.js";
import {
- finishTurn,
hold,
ownOpenRequestIds,
settle,
@@ -189,10 +186,9 @@ export interface Answered extends Transition {
/**
* `continue`: the step goes on. `park`: nothing can run until more input arrives.
* `repeat`: a policy pass committed, so the next pass runs in a fresh step. `sign-in`: a
- * responder must sign in first. `defer-message`: a message arrived while the session-limit
- * prompt is open; it waits in its turn for the prompt.
+ * responder must sign in first.
*/
- readonly next: "continue" | "park" | "repeat" | "sign-in" | "defer-message";
+ readonly next: "continue" | "park" | "repeat" | "sign-in";
/** The turn's input, without the answers `answer` consumed. */
readonly input?: StepInput;
/** Whether a plain-text answer consumed the message. */
@@ -286,16 +282,7 @@ export function answer(
if (limit !== undefined) {
const response = byId.get(limit.request.requestId);
- if (response === undefined) {
- if (!hasInput(delivery)) {
- queue(compactInput(resolved));
- return done({ next: "park" });
- }
- // The message is received now, into the turn that holds for the prompt: only answers to
- // other requests wait for the grant. What waits in the queue was never received.
- queue(withoutTurnInput(resolved));
- return done({ input: resolved, next: "defer-message" });
- }
+ if (response === undefined) return park();
const batch: ResolvedInputBatch = {
event: { sequence: limit.sequence, stepIndex: limit.stepIndex, turnId: limit.turnId },
inputs: [
@@ -496,6 +483,13 @@ function reportApprovalProgress(
export function hasRunnableQueue(view: SessionView): boolean {
const queued = view.turn.queued;
if (queued === undefined) return false;
+ const responses = [
+ ...(queued.inputResponses ?? []),
+ ...(queued.attributedInputResponses ?? []).map(({ response }) => response),
+ ];
+ const answered = new Set(responses.map((response) => response.requestId));
+ const limit = openInputs(view.projection).find((open) => open.request.kind === "session-limit");
+ if (limit !== undefined) return answered.has(limit.request.requestId);
if (
queued.message !== undefined ||
(queued.context?.length ?? 0) > 0 ||
@@ -505,14 +499,7 @@ export function hasRunnableQueue(view: SessionView): boolean {
) {
return true;
}
- const responses = [
- ...(queued.inputResponses ?? []),
- ...(queued.attributedInputResponses ?? []).map(({ response }) => response),
- ];
if (responses.length === 0) return false;
- const answered = new Set(responses.map((response) => response.requestId));
- const limit = openInputs(view.projection).find((open) => open.request.kind === "session-limit");
- if (limit !== undefined) return answered.has(limit.request.requestId);
return view.turn.suspended.some(
(step) =>
step.requests.length > 0 && step.requests.every((request) => answered.has(request.requestId)),
@@ -583,7 +570,7 @@ export function withdrawSignIns(view: SessionView, reason: string): Transition {
/**
* The session spent its budget: it asks whether to continue instead of calling the model, and
- * the turn ends. The prompt is the projection's open request; `answer` grants a
+ * the turn holds for input. The prompt is the projection's open request; `answer` grants a
* fresh budget or declines it.
*/
export function requestLimit(
@@ -599,7 +586,7 @@ export function requestLimit(
stepIndex: position.stepIndex,
turnId: position.turnId,
}),
- ...finishTurn(view).events,
+ ...hold(view, { on: "input" }).events,
],
turn: view.turn,
};
diff --git a/packages/eve/src/harness/hitl/delivery.ts b/packages/eve/src/harness/hitl/delivery.ts
index fa8c5edfd..fad10d6f8 100644
--- a/packages/eve/src/harness/hitl/delivery.ts
+++ b/packages/eve/src/harness/hitl/delivery.ts
@@ -1,4 +1,5 @@
import type { UserContent } from "ai";
+import { openInputs } from "#protocol/session-projection.js";
import { resolveTextToResponse, resolveTextToResponses } from "#channel/resolve-text.js";
import { coalesceTurnInputs } from "#harness/messages.js";
@@ -35,8 +36,18 @@ export function deliver(
readonly takeQueued: boolean;
} {
const queued = options.takeQueued ? view.turn.queued : undefined;
+ const limit = openInputs(view.projection).find((entry) => entry.request.kind === "session-limit");
+ // Match the new typed answer before queued messages join it.
+ const incoming =
+ queued !== undefined && limit !== undefined
+ ? resolveTextInput({ requests: [limit.request] }, input)
+ : input;
const merged =
- queued === undefined ? input : input === undefined ? queued : coalesceTurnInputs(queued, input);
+ queued === undefined
+ ? incoming
+ : incoming === undefined
+ ? queued
+ : coalesceTurnInputs(queued, incoming);
// An approval settles on its own answer, before its batch resolves: the answer still counts
// while its batch waits for the rest.
const pendingRequestIds = new Set([
@@ -129,10 +140,6 @@ export function canonicalize(responses: readonly InputResponse[]): readonly Inpu
return [...byRequestId.values()];
}
-export function hasInput(input: StepInput | undefined): boolean {
- return input?.message !== undefined || (input?.inputResponses?.length ?? 0) > 0;
-}
-
export function isEmptyInput(input: StepInput): boolean {
return Object.keys(compactInput(input)).length === 0;
}
diff --git a/packages/eve/src/harness/hitl/intake.ts b/packages/eve/src/harness/hitl/intake.ts
index 1edd6dbbd..161428761 100644
--- a/packages/eve/src/harness/hitl/intake.ts
+++ b/packages/eve/src/harness/hitl/intake.ts
@@ -10,10 +10,7 @@ import {
settle,
} from "#harness/session-machine/transitions.js";
import type { SuspendedStep } from "#harness/session-machine/view.js";
-import { validateHarnessModelMessages } from "#harness/messages.js";
-import { openTurn, type Step } from "#harness/step/context.js";
-import { prepareTurnInput } from "#harness/step/intake.js";
-import { placeTurnInput } from "#harness/step/prompt.js";
+import type { Step } from "#harness/step/context.js";
import { SessionLimitDeclinedError, throwIfTurnAborted } from "#harness/turn-cancellation.js";
import { bumpSessionRuntimeUsageLimits } from "#harness/turn-tag-state.js";
import type { HarnessToolLookup, StepInput, StepResult } from "#harness/types.js";
@@ -178,27 +175,6 @@ export async function acceptHumanInput(
}),
);
return stop(held(step));
- case "defer-message": {
- // Approved calls wait behind the budget prompt, and the message waits behind them.
- if (approvedCalls(step.view().turn).length > 0) {
- const deferred = turnInputOnly(decision.input);
- if (deferred !== undefined) await step.apply(deferInput(step.view(), deferred));
- return stop({ next: null, session: step.session });
- }
- // The message is received now, into a turn that holds for the budget prompt: the grant
- // resumes that turn, and the model reads the message from history.
- const turn = await prepareTurnInput(step, decision.input, { consumedMessage: false });
- const failed = await openTurn(step, {
- input: [...turn.ephemeral, ...turn.messages],
- message: delivered.displayMessage ?? decision.input?.message,
- });
- if (failed !== undefined) return stop(failed);
- step.session = {
- ...step.session,
- history: validateHarnessModelMessages(placeTurnInput(step, step.session.history, turn)),
- };
- return stop(await holdForInput(step));
- }
case "continue":
break;
}
diff --git a/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts b/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts
index 8d6f26495..27e6fad01 100644
--- a/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts
+++ b/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts
@@ -402,12 +402,8 @@ it("keeps a message waiting behind a budget prompt and runs it after the grant",
const waitingStart = f.events.length;
expect((await f.drive({ message: "Also say goodbye." })).held).toEqual({ kind: "request" });
- // The message is received into a turn that holds on the prompt; the grant resumes it.
- expect(f.events.slice(waitingStart).map((event) => event.type)).toEqual([
- "turn.started",
- "message.received",
- "turn.waiting",
- ]);
+ // The message waits for the grant in the held turn.
+ expect(f.events.slice(waitingStart).map((event) => event.type)).toEqual(["turn.waiting"]);
expect(f.pending().map((r) => r.kind)).toEqual(["session-limit"]);
f.script.push(calls("read"), "FINAL");
diff --git a/packages/eve/src/harness/session-machine/machine.test.ts b/packages/eve/src/harness/session-machine/machine.test.ts
index 50a267227..8e571f7f9 100644
--- a/packages/eve/src/harness/session-machine/machine.test.ts
+++ b/packages/eve/src/harness/session-machine/machine.test.ts
@@ -580,14 +580,10 @@ describe("answers", () => {
return machine;
}
- it("asks whether to continue and ends the turn", async () => {
+ it("asks whether to continue and holds the turn", async () => {
const machine = await exhausted();
- expect(machine.events.slice(-3)).toEqual([
- "input.requested",
- "turn.completed",
- "session.waiting",
- ]);
+ expect(machine.events.slice(-2)).toEqual(["input.requested", "turn.waiting"]);
expect(machine.projection.inputs[request.requestId]?.status).toBe("open");
});
@@ -605,14 +601,14 @@ describe("answers", () => {
}
});
- it("receives a message now, and queues only what waits for the answer", async () => {
+ it("queues a message until the budget answer", async () => {
const machine = await exhausted();
const decision = await respond(machine, { message: "Any update?" });
- expect(decision.next).toBe("defer-message");
- expect(decision.input?.message).toBe("Any update?");
- expect(machine.view().turn.queued).toBeUndefined();
+ expect(decision.next).toBe("park");
+ expect(decision.input).toBeUndefined();
+ expect(machine.view().turn.queued?.message).toBe("Any update?");
});
it("takes the prompt's answer from text while an approval is also open", async () => {
diff --git a/packages/eve/src/harness/tool-loop.test.ts b/packages/eve/src/harness/tool-loop.test.ts
index dfe8c1671..4ab870faf 100644
--- a/packages/eve/src/harness/tool-loop.test.ts
+++ b/packages/eve/src/harness/tool-loop.test.ts
@@ -61,7 +61,7 @@ import {
import { applyTransition, sessionView } from "#harness/session-machine/commit.js";
import { requireSignIn } from "#harness/hitl/approvals.js";
import { createAuthorizationRequiredEvent } from "#protocol/message.js";
-import { ownOpenRequestIds } from "#harness/session-machine/transitions.js";
+import { finishTurn, ownOpenRequestIds } from "#harness/session-machine/transitions.js";
import { runtimeWait, storedProjection } from "#harness/session-machine/view.js";
import {
foldingHandler,
@@ -2054,8 +2054,7 @@ describe("createToolLoopHarness", () => {
"message.received",
"step.started",
"input.requested",
- "turn.completed",
- "session.waiting",
+ "turn.waiting",
]);
const requested = events.find((event) => event.type === "input.requested");
expect(requested?.data).toMatchObject({
@@ -2243,12 +2242,8 @@ describe("createToolLoopHarness", () => {
const beforeQueued = events.length;
const reparked = await runStep(parked.session, { message: "also do this other thing" });
- // The message is received into a real turn, which holds for the prompt.
- expect(events.slice(beforeQueued).map((event) => event.type)).toEqual([
- "turn.started",
- "message.received",
- "turn.waiting",
- ]);
+ // The message waits behind the question without starting another turn.
+ expect(events.slice(beforeQueued).map((event) => event.type)).toEqual(["turn.waiting"]);
const waiting = events.at(-1);
expect(waiting).toMatchObject({ data: { on: "input" }, type: "turn.waiting" });
const turnId = waiting?.type === "turn.waiting" ? waiting.data.turnId : undefined;
@@ -2257,17 +2252,17 @@ describe("createToolLoopHarness", () => {
expect(vi.mocked(ToolLoopAgent)).not.toHaveBeenCalled();
expect(events.filter((event) => event.type === "input.requested")).toHaveLength(1);
const asked = { content: "also do this other thing", kind: "user" as const, role: "user" };
- expect(reparked.session.history).toContainEqual(asked);
+ expect(reparked.session.history).not.toContainEqual(asked);
const beforeGrant = events.length;
const resumed = await runStep(reparked.session, {
inputResponses: [{ optionId: "continue", requestId: LIMIT_REQUEST_ID }],
});
- // The grant resumes that turn; the message was received once, and the model reads it once.
+ // The grant resumes that turn and receives the queued message once.
const resumedEvents = events.slice(beforeGrant);
expect(resumedEvents.some((event) => event.type === "turn.started")).toBe(false);
- expect(resumedEvents.some((event) => event.type === "message.received")).toBe(false);
+ expect(resumedEvents.filter((event) => event.type === "message.received")).toHaveLength(1);
expect(resumedEvents.find((event) => event.type === "step.started")?.data.turnId).toBe(turnId);
expect(vi.mocked(ToolLoopAgent)).toHaveBeenCalledTimes(1);
expect(
@@ -2275,6 +2270,62 @@ describe("createToolLoopHarness", () => {
).toHaveLength(1);
});
+ it("continues a persisted pre-upgrade budget prompt after a new message", async () => {
+ setupMockAgent({
+ finishReason: "stop",
+ response: { messages: [{ content: "Hello!", role: "assistant" }] },
+ text: "Hello!",
+ toolCalls: [],
+ toolResults: [],
+ usage: { inputTokens: 7, outputTokens: 3 },
+ });
+ const { emit, events } = createEventCollector();
+ const runStep = createToolLoopHarness(createTestConfig(emit));
+ const parked = await runStep(createLimitReachedSession(), { message: "Hi again" });
+
+ // Before this change requestLimit published input.requested followed by
+ // finishTurn: turn.completed and session.waiting, retaining an open prompt.
+ const oldSession = withPublished(
+ parked.session,
+ finishTurn(sessionView(storedProjection(parked.session.state), parked.session.state)).events,
+ );
+ const restored: HarnessSession = JSON.parse(JSON.stringify(oldSession));
+ const projection = storedProjection(restored.state);
+ expect(projection.activeTurnId).toBeUndefined();
+ expect(projection.inputs[LIMIT_REQUEST_ID]?.status).toBe("open");
+
+ const beforeMessage = events.length;
+ const queued = await runStep(restored, { message: "also do this other thing" });
+ const messageEvents = events.slice(beforeMessage);
+ // The old turn is already closed: queuing emits no event until the grant.
+ expect(messageEvents.map((event) => event.type)).toEqual([]);
+ expect(queued.held).toBeUndefined();
+ expect(vi.mocked(ToolLoopAgent)).not.toHaveBeenCalled();
+ const turnId = storedProjection(queued.session.state).activeTurnId;
+ expect(turnId).toBeUndefined();
+
+ const beforeGrant = events.length;
+ const resumed = await runStep(queued.session, {
+ inputResponses: [{ optionId: "continue", requestId: LIMIT_REQUEST_ID }],
+ });
+ const resumedEvents = events.slice(beforeGrant);
+ expect(resumedEvents.filter((event) => event.type === "turn.started")).toHaveLength(1);
+ expect(resumedEvents.filter((event) => event.type === "message.received")).toHaveLength(1);
+ expect(resumedEvents.find((event) => event.type === "step.started")?.data.turnId).toBe(
+ resumedEvents.find((event) => event.type === "turn.started")?.data.turnId,
+ );
+ expect(events.filter((event) => event.type === "input.requested")).toHaveLength(1);
+ expect(storedProjection(resumed.session.state).inputs[LIMIT_REQUEST_ID]).toBeUndefined();
+ expect(resumedEvents.find((event) => event.type === "input.resolved")).toMatchObject({
+ data: { resolutions: [{ requestId: LIMIT_REQUEST_ID, outcome: "answered" }] },
+ });
+ expect(vi.mocked(ToolLoopAgent)).toHaveBeenCalledTimes(1);
+ expect(getSessionUsageLimitViolation(resumed.session)).toBeNull();
+ expect(
+ resumed.session.history.filter((message) => message.content === "also do this other thing"),
+ ).toHaveLength(1);
+ });
+
it("preserves approval gates on step-scoped dynamic tools", async () => {
setupMockAgent({
finishReason: "stop",
diff --git a/packages/eve/test/scenarios/turn-cancellation.scenario.test.ts b/packages/eve/test/scenarios/turn-cancellation.scenario.test.ts
index 10b380a48..945d5305a 100644
--- a/packages/eve/test/scenarios/turn-cancellation.scenario.test.ts
+++ b/packages/eve/test/scenarios/turn-cancellation.scenario.test.ts
@@ -289,8 +289,11 @@ describe("turn cancellation descendant cascade", () => {
message:
"Use workflow exactly once to call local-sleeper only with message Use wait-for-cancel.",
});
+ // The limit prompt holds the root turn, so its stream pauses at
+ // `turn.waiting` rather than settling the session.
const events = await readThroughBoundary({
iterator: response[Symbol.asyncIterator](),
+ isBoundary: (event) => event.type === "turn.waiting",
label: "root session-limit prompt",
});
// The workflow program runs as a task, so the root reaches its limit
@@ -382,8 +385,10 @@ async function readUntil(input: {
async function readThroughBoundary(input: {
readonly iterator: AsyncIterator<MessageStreamEvent>;
+ readonly isBoundary?: (event: MessageStreamEvent) => boolean;
readonly label: string;
}): Promise<readonly MessageStreamEvent[]> {
+ const isBoundary = input.isBoundary ?? isCurrentTurnBoundaryEvent;
const events: MessageStreamEvent[] = [];
await withinEventDeadline(
(async () => {
@@ -391,7 +396,7 @@ async function readThroughBoundary(input: {
const next = await input.iterator.next();
if (next.done) throw new Error(`Stream ended before ${input.label}.`);
events.push(next.value);
- if (isCurrentTurnBoundaryEvent(next.value)) return;
+ if (isBoundary(next.value)) return;
}
})(),
input.label,