The Open Framework for Building Agents
feat(eve): deferred tools and skills (#4400)
Andrew Barba8c55ea7c132f
diff --git a/.changeset/catalog-call-labels.md b/.changeset/catalog-call-labels.mdnew file mode 100644index 000000000..1db415fd5--- /dev/null+++ b/.changeset/catalog-call-labels.md@@ -0,0 +1,5 @@+---+"eve": patch+---++Catalog calls now read as what happened: `eve__search` shows `Search tools for “linear issues”` and then how much it found, a skill load shows `Load skill <name>` and then `Loaded skill <name>`, and a connection's sign-in entry shows `Sign in to Linear`. Linear activities, Chat SDK typing indicators, and ACP tool calls now show each call's label instead of its raw tool name, and `eve/client` tool parts carry that text as `toolMetadata.eve.label`.diff --git a/.changeset/deferred-skills.md b/.changeset/deferred-skills.mdnew file mode 100644index 000000000..6a9931e1b--- /dev/null+++ b/.changeset/deferred-skills.md@@ -0,0 +1,10 @@+---+"eve": minor+---++Skills now load through `eve__skill({ name })`, which every agent with a skill has in place of `load_skill`, and a skill with `deferred: true` (in `SKILL.md` frontmatter or on `defineSkill`, static or dynamic) leaves the system prompt and is found with `eve__search` instead.++- The `load_skill` tool and the `eve/tools/load_skill` export are removed. Delete any `agent/tools/load_skill.ts` override or approval policy that names it; an authored `agent/tools/load_skill.ts` is now an ordinary tool.+- A skill load now reports a `load-skill-result` named for its skill instead of a `tool-result` named `load_skill`, and its client message part is named for the skill.+- Eval facts report skill loads in `skillLoads`, which `t.loadedSkill` reads. They no longer count toward `toolCallCount`, `usedNoTools`, or `maxToolCalls`.+- Skill names must use only ASCII letters, digits, underscores, and dashes, start with a letter or digit, and have at most 64 characters. The compiler rejects a static skill with another name, and eve skips a dynamic skill resolver that returns one.diff --git a/.changeset/deferred-tools-catalog.md b/.changeset/deferred-tools-catalog.mdnew file mode 100644index 000000000..a11b5f53b--- /dev/null+++ b/.changeset/deferred-tools-catalog.md@@ -0,0 +1,16 @@+---+"eve": minor+---++An agent with deferred entries or connections now has `eve__search` to find tools marked `deferred: true`, agents marked `tool: "deferred"`, and every connection tool, and `eve__tool` to call them, without growing the model's tool list. An agent with skills has `eve__skill` to load them, and `eve__search` also finds deferred skills. An agent with none of these gets no catalog tools, so its tool list is unchanged apart from `load_skill` becoming `eve__skill`. They replace `connection_search` and `connection_execute`, and stream events, approvals, hooks, and traces now name each connection tool call `<connection>__<tool>`. `eve__search` never asks the user to sign in: it lists a connection that needs sign-in under the connection's own name, and `eve__tool` on that name, such as `eve__tool({ name: "linear" })`, asks the user to sign in. `eve__search` takes a required `query`; a query such as `linear__` searches only that connection's tools.++Breaking changes:++- The tools eve adds itself now live in the `eve` namespace: `task_wait` is `eve__task_wait`, `task_cancel` is `eve__task_cancel`, and `final_output`, the tool a session with an output schema calls to deliver its structured result, is `eve__reply`. The catalog tools are `eve__search`, `eve__tool`, and `eve__skill`. Update evals, approval policies, hooks, and channel code that match `task_wait`, `task_cancel`, or `final_output` by name. A turn held on `task_wait` when you upgrade stays held until the next message or a cancel.+- eve reserves the `eve` namespace. A tool, subagent, skill, connection, or extension mount named `eve` or starting with `eve__` now fails to build, or fails when the agent loads or a dynamic entry resolves; rename it. `search`, `execute`, `task_wait`, `task_cancel`, and `final_output` are ordinary names again.+- A connection owns its name and every name starting with `<name>__`. A tool, subagent, or connection under another connection's prefix is rejected at build time, and a dynamic one when it resolves.+- Dynamic tool map keys must be legal tool names: ASCII letters, digits, `_`, and `-`, starting with a letter, up to 64 characters.+- Connection approvals are keyed by `<connection>__<tool>` instead of a `[connection, tool]` pair, so saved "always approve" decisions for connection tools reset.+- A `connection_execute` call still waiting for approval when you upgrade fails when someone answers it, with "The tool "connection_execute" is no longer available, so the call didn't run."+- Eval tool-name helpers (`calledTool`, `notCalledTool`, `toolOrder`, and `requireToolCall`) now throw, saying what to assert instead, for `eve__tool` and `eve__skill`, whose calls are reported under the entry they reach and as skill loads, and for a retired eve tool name (`load_skill`, `task_wait`, `task_cancel`, `final_output`, `connection_search`, `connection_execute`) unless the root agent declares a tool by that name or has a dynamic tool resolver. Before, `notCalledTool` on one of these names always passed. `usedNoTools` and `maxToolCalls` count `eve__search` calls; skill loads no longer do.+- Removed: the `parentCallId` field on `tool-call` actions in `actions.requested` events, and the `agent.action.parent_call_id` span attribute. The subagent `parentCallId` on `session.started` invocation metadata is unchanged.diff --git a/apps/docs/registry.json b/apps/docs/registry.jsonindex 2c5e5732e..fde7719c7 100644--- a/apps/docs/registry.json+++ b/apps/docs/registry.json@@ -359,19 +359,6 @@}]},- {- "name": "tool/load_skill",- "type": "registry:item",- "title": "Load Skill",- "description": "Load instructions from an agent skill.",- "files": [- {- "path": "registry/tools/load_skill.ts",- "type": "registry:file",- "target": "agent/tools/load_skill.ts"- }- ]- },{"name": "tool/read_file","type": "registry:item",diff --git a/apps/docs/registry/channel/web/app/_components/agent-message.tsx b/apps/docs/registry/channel/web/app/_components/agent-message.tsxindex 6a124ba72..6997049e8 100644--- a/apps/docs/registry/channel/web/app/_components/agent-message.tsx+++ b/apps/docs/registry/channel/web/app/_components/agent-message.tsx@@ -150,7 +150,7 @@ function AgentMessagePart({<Tool><ToolHeaderstate={part.state}- title={part.toolName}+ title={part.toolMetadata?.eve?.label ?? part.toolName}toolName={part.toolName}type="dynamic-tool"/>diff --git a/apps/docs/registry/tools/load_skill.ts b/apps/docs/registry/tools/load_skill.tsdeleted file mode 100644index 1bd6aaaa5..000000000--- a/apps/docs/registry/tools/load_skill.ts+++ /dev/null@@ -1 +0,0 @@-export { default } from "eve/tools/load_skill";diff --git a/apps/frameworks/next/app/_chat/trace.ts b/apps/frameworks/next/app/_chat/trace.tsindex 670ec4eba..983781806 100644--- a/apps/frameworks/next/app/_chat/trace.ts+++ b/apps/frameworks/next/app/_chat/trace.ts@@ -198,7 +198,7 @@ function getTraceActionNameFromRequest(action: Record<string, unknown>): stringconst kind = getTraceActionKindFromRequest(action);switch (kind) {case "load-skill":- return "load_skill";+ return isRecord(action.input) ? readString(action.input.skill) : undefined;case "subagent-call":return readString(action.subagentName) ?? readString(action.name);case "tool-call":@@ -214,7 +214,7 @@ function getTraceActionNameFromResult(result: Record<string, unknown>): string |const kind = getTraceActionKindFromResult(result);switch (kind) {case "load-skill":- return readString(result.name) ?? "load_skill";+ return readString(result.name);case "subagent-call":return readString(result.subagentName);case "tool-call":diff --git a/apps/frameworks/nuxt/app/components/ToolBlock.vue b/apps/frameworks/nuxt/app/components/ToolBlock.vueindex 426660934..55aa326ca 100644--- a/apps/frameworks/nuxt/app/components/ToolBlock.vue+++ b/apps/frameworks/nuxt/app/components/ToolBlock.vue@@ -20,7 +20,12 @@ const isOpen = ref(props.part.state === "approval-requested" || props.part.state === "approval-responded",);-const toolName = computed(() => props.part.toolMetadata?.eve?.name ?? props.part.toolName);+const title = computed(+ () =>+ props.part.toolMetadata?.eve?.label ??+ props.part.toolMetadata?.eve?.name ??+ props.part.toolName,+);const stateLabel = computed(() => {const labels: Record<string, string> = {@@ -75,7 +80,7 @@ const formattedOutput = computed(() => {d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>- <span class="text-sm font-medium">{{ toolName }}</span>+ <span class="text-sm font-medium">{{ title }}</span><span class="rounded-full bg-secondary px-2 py-0.5 text-xs" :class="stateColor">{{ stateLabel }}</span>diff --git a/apps/frameworks/sveltekit/src/lib/ToolBlock.svelte b/apps/frameworks/sveltekit/src/lib/ToolBlock.svelteindex b5e76b109..b43006fe3 100644--- a/apps/frameworks/sveltekit/src/lib/ToolBlock.svelte+++ b/apps/frameworks/sveltekit/src/lib/ToolBlock.svelte@@ -23,7 +23,7 @@);let isOpen = $derived(userOpen ?? initiallyOpen);- let toolName = $derived(part.toolMetadata?.eve?.name ?? part.toolName);+ let title = $derived(part.toolMetadata?.eve?.label ?? part.toolMetadata?.eve?.name ?? part.toolName);let stateLabel = $derived.by(() => {const labels: Record<string, string> = {"input-streaming": "Pending",@@ -82,7 +82,7 @@d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>- <span class="text-sm font-medium">{toolName}</span>+ <span class="text-sm font-medium">{title}</span><span class="rounded-full bg-secondary px-2 py-0.5 text-xs {stateColor}">{stateLabel}</span>diff --git a/apps/templates/eve-chat-template/apps/web/components/chat/message.tsx b/apps/templates/eve-chat-template/apps/web/components/chat/message.tsxindex c69f5df45..aa60fc2a9 100644--- a/apps/templates/eve-chat-template/apps/web/components/chat/message.tsx+++ b/apps/templates/eve-chat-template/apps/web/components/chat/message.tsx@@ -362,7 +362,7 @@ function ToolGroup({const shouldOpen = parts.some(needsInputResponse);const [open, setOpen] = useState(shouldOpen);const status = getSettledToolStatus(getToolGroupStatus(parts), isSettled && !shouldOpen);- const label = summarizeToolGroup(parts, status);+ const label = summarizeToolGroup(parts);const canExpand = parts.length > 1 ? parts.some(hasToolDetails) : hasToolDetails(parts[0]!);useEffect(() => {@@ -453,7 +453,7 @@ function ToolCallItem({><ToolStatusIcon status={status} /><ToolNameLabel part={part} />- <span className="truncate text-foreground/80">{describeToolAction(part, status)}</span>+ <span className="truncate text-foreground/80">{describeToolAction(part)}</span>{canExpand ? (<ChevronRightIconclassName={cn(@@ -731,9 +731,9 @@ function toolStatusLabel(status: ToolStatus) {}}-function summarizeToolGroup(parts: readonly EveDynamicToolPart[], status: ToolStatus) {+function summarizeToolGroup(parts: readonly EveDynamicToolPart[]) {if (parts.length === 1) {- return describeToolAction(parts[0]!, status);+ return describeToolAction(parts[0]!);}const counts = new Map<string, number>();@@ -780,7 +780,12 @@ function toolCategory(name: string) {return "ran";}-function describeToolAction(part: EveDynamicToolPart, status = getToolStatus(part)) {+function describeToolAction(part: EveDynamicToolPart) {+ const label = toolLabel(part);+ if (label) {+ return label;+ }+const name = resolveToolName(part);const normalized = normalizeToolName(name);const input = asRecord(part.input);@@ -788,22 +793,6 @@ function describeToolAction(part: EveDynamicToolPart, status = getToolStatus(parconst path = readString(input, ["path", "filePath", "filename"]);const command = readString(input, ["command", "cmd"]);const url = readString(input, ["url", "href"]);- const connection = readString(input, ["connection", "connectionName", "connector", "source"]);-- if (normalized.includes("connection") && normalized.includes("search")) {- const verb = status === "running" ? "Searching" : "Searched";- const connectionName = resolveConnectionName(name, connection);-- if (connectionName) {- return `${verb} ${formatDisplayName(connectionName)}`;- }-- if (query && query !== "*") {- return `${verb} ${truncateInline(query, 72)}`;- }-- return `${verb} connections`;- }if (normalized.includes("search") || normalized.includes("grep")) {return query ? `Searched ${truncateInline(query, 72)}` : `Searched ${formatToolName(name)}`;@@ -836,57 +825,25 @@ function describeToolAction(part: EveDynamicToolPart, status = getToolStatus(parreturn `Used ${formatToolName(name)}`;}+/** eve versions that set `toolMetadata.eve.label` say how a call reads; older ones fall back to its name. */+function toolLabel(part: EveDynamicToolPart): string | undefined {+ const eve: Readonly<Record<string, unknown>> | undefined = part.toolMetadata?.eve;+ return typeof eve?.label === "string" ? eve.label : undefined;+}+function resolveToolName(part: EveDynamicToolPart) {const metadataName = part.toolMetadata?.eve?.name;return metadataName && metadataName !== "unknown" ? metadataName : part.toolName;}function formatToolName(name: string) {- return normalizeToolName(name)- .replace(/^connection search$/, "connection search")- .replace(/\s+/g, " ")- .trim();+ return normalizeToolName(name).replace(/\s+/g, " ").trim();}function normalizeToolName(name: string) {return name.replace(/__/g, " ").replace(/[_-]/g, " ").trim().toLowerCase();}-function formatDisplayName(value: string) {- const cleaned = value- .replace(/^mcp\./, "")- .replace(/\.com(?:\/.*)?$/, "")- .replace(/[_-]/g, " ");-- return cleaned- .split(/\s+/)- .filter(Boolean)- .map((part) => part.charAt(0).toUpperCase() + part.slice(1))- .join(" ");-}--function resolveConnectionName(toolName: string, inputConnection?: string | null) {- if (inputConnection && inputConnection !== "*") {- return inputConnection;- }-- const tokens = normalizeToolName(toolName).split(/\s+/).filter(Boolean);-- if (tokens[0] !== "connection" || tokens.length <= 2) {- return null;- }-- const connectionTokens = tokens- .slice(1)- .filter((token) => token !== "search" && token !== "tool" && token !== "tools");-- if (connectionTokens.length === 0) {- return null;- }-- return [...new Set(connectionTokens)].join(" ");-}-function shortenPath(filepath: string) {const parts = filepath.split("/").filter(Boolean);diff --git a/apps/templates/personal-agent-template/apps/web/app/utils/chat/eve.ts b/apps/templates/personal-agent-template/apps/web/app/utils/chat/eve.tsindex ed7e1a16c..924c5244d 100644--- a/apps/templates/personal-agent-template/apps/web/app/utils/chat/eve.ts+++ b/apps/templates/personal-agent-template/apps/web/app/utils/chat/eve.ts@@ -25,5 +25,11 @@ export function getToolDisplayName(part: EveDynamicToolPart): string {if (part.toolName === "ask_question") {return part.toolMetadata?.eve?.inputRequest?.prompt ?? "Question";}- return part.toolName;+ return toolLabel(part) ?? part.toolName;+}++/** eve versions that set `toolMetadata.eve.label` say how a call reads; older ones fall back to its name. */+function toolLabel(part: EveDynamicToolPart): string | undefined {+ const eve: Readonly<Record<string, unknown>> | undefined = part.toolMetadata?.eve;+ return typeof eve?.label === "string" ? eve.label : undefined;}diff --git a/docs/agent-config.md b/docs/agent-config.mdindex bb3441d94..43d2c4d99 100644--- a/docs/agent-config.md+++ b/docs/agent-config.md@@ -358,14 +358,14 @@ it falls back to the World's default retention period.`defineAgent` takes a few more fields, all optional. For the exported types, see the [TypeScript API Reference](./reference/typescript-api).-| Field | Type | Default | Description |-| -------------- | ------------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| `reasoning` | `AgentReasoningDefinition` | provider default | Provider-agnostic reasoning effort forwarded to the agent's turn model calls. |-| `modelOptions` | `AgentModelOptionsDefinition` | none | Provider option overrides forwarded to the model call. |-| `limits` | `AgentLimitsDefinition` | field-specific | Framework-owned runtime limits. Sessions complete after 30 days by default; usage-limit defaults and inheritance are described above. Set a limit to `false` to disable it. |-| `experimental` | `AgentExperimentalDefinition` | unset | Unstable opt-ins. `workflow.world` selects the Workflow world package on the root agent; `workflow.modelCallsPerStep` batches sequential model calls into a wider replay unit; `workflow.retention` controls how long the durable runtime keeps run data. |-| `build` | `{ externalDependencies?: string[] }` | none | Hosted-build packaging controls. `externalDependencies` keeps listed packages external while eve compiles authored modules such as tools and channels, and traces those packages into the hosted output. |-| `tool` | `boolean` | `true` | Exposes this agent to its parent model as a tool. On the root agent, controls the built-in `agent` tool. A subagent with `tool: false` remains callable from authored workflow tools through `ctx.agent()`. |+| Field | Type | Default | Description |+| -------------- | ------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |+| `reasoning` | `AgentReasoningDefinition` | provider default | Provider-agnostic reasoning effort forwarded to the agent's turn model calls. |+| `modelOptions` | `AgentModelOptionsDefinition` | none | Provider option overrides forwarded to the model call. |+| `limits` | `AgentLimitsDefinition` | field-specific | Framework-owned runtime limits. Sessions complete after 30 days by default; usage-limit defaults and inheritance are described above. Set a limit to `false` to disable it. |+| `experimental` | `AgentExperimentalDefinition` | unset | Unstable opt-ins. `workflow.world` selects the Workflow world package on the root agent; `workflow.modelCallsPerStep` batches sequential model calls into a wider replay unit; `workflow.retention` controls how long the durable runtime keeps run data. |+| `build` | `{ externalDependencies?: string[] }` | none | Hosted-build packaging controls. `externalDependencies` keeps listed packages external while eve compiles authored modules such as tools and channels, and traces those packages into the hosted output. |+| `tool` | `boolean \| "deferred"` | `true` | Exposes this agent to its parent model as a tool. `"deferred"` keeps it out of the model's tool list; the model finds it with `eve__search` and calls it with `eve__tool`. On the root agent, controls the built-in `agent` tool. A subagent with `tool: false` remains callable from authored workflow tools through `ctx.agent()`. |`externalDependencies` is a packaging control only. It keeps selected packages as runtime dependencies in the hosted output; it does not authorize, configure, or review any third-party service those packages may call.diff --git a/docs/channels/chat-sdk.mdx b/docs/channels/chat-sdk.mdxindex 3a0b64c9e..125d94d3a 100644--- a/docs/channels/chat-sdk.mdx+++ b/docs/channels/chat-sdk.mdx@@ -151,7 +151,7 @@ chatSdkChannel({});```-Typing indicators post automatically where the adapter supports them: `Working…` on `turn.started`, and tool status on `actions.requested`.+Typing indicators post automatically where the adapter supports them: `Working…` on `turn.started`, and on `actions.requested`, each call's [label](/docs/tools#label-tool-activity) or a readable form of its name. They leave out the model's own `eve__task_wait` and `eve__task_cancel` calls.### Optional capabilities degrade gracefullydiff --git a/docs/channels/slack.mdx b/docs/channels/slack.mdxindex c37cd040b..3849e472c 100644--- a/docs/channels/slack.mdx+++ b/docs/channels/slack.mdx@@ -476,7 +476,7 @@ A model step never resets the status to a generic line. The status keeps namingSlack removes a thread status after two minutes without a message from the app, and clears it whenever the app posts in the thread. eve sets the current status again at each model step, after it posts a new task card, and as a tool reports progress. A tool that runs longer than two minutes without reporting progress loses its status. Report progress with [`label.delta`](/docs/tools#label-tool-activity), or run the work as a [task](/docs/tools/tasks), whose card shows it is still going.-Action labels follow the same rule as the task card titles described below. They leave out the model's own `task_wait` and `task_cancel` calls. Reasoning shows one piece at a time: a heading when the model titles its reasoning, as reasoning summaries do, otherwise the latest finished sentence. Each piece of reasoning, and each progress label, stays up for at least three seconds, so the status changes at a readable pace instead of sending a Slack request per token. Every status is cut to 50 characters at a word boundary.+Action labels follow the same rule as the task card titles described below. They leave out the model's own `eve__task_wait` and `eve__task_cancel` calls. A label can include part of the call's input, such as a search query or a `bash` command, as labels always could. Reasoning shows one piece at a time: a heading when the model titles its reasoning, as reasoning summaries do, otherwise the latest finished sentence. Each piece of reasoning, and each progress label, stays up for at least three seconds, so the status changes at a readable pace instead of sending a Slack request per token. Every status is cut to 50 characters at a word boundary.Replies up to 12,000 characters render inline. Longer replies are uploaded unchanged as a Markdown snippet named `eve-response.md` with a short in-thread note, which requires the `files:write` bot scope.@@ -498,9 +498,9 @@ Each row shows:- **Title:** the tool's [`label.start`](/docs/tools#label-tool-activity), or a readable form of the tool name. eve drops the extension or memory namespace before `__` and splits the rest into words, so `crm__sync_accounts` and `syncAccounts` both show as `Sync accounts`. A connection call shows the connection and its tool, as in `Linear: List issues`. An agent task shows the agent name and the first sentence of its brief (`Researcher: Find the incidents behind the checkout spike`). A task the agent hands to its own copy shows the brief alone. eve can't infer brand casing from a name, so `github` shows as `Github`; set `label.start` when the exact wording matters.- **Waiting:** while the task waits on an approval, an answer, or a sign-in, the row says so, as in `Waiting for approval`. In a DM or private channel it also shows the request's prompt; elsewhere it shows only what kind of request is waiting, since `approvalChannel` can send the request to one person. The request itself posts wherever `approvalChannel` sends it, and the row goes back to working once the request resolves.-- **How it ended:** the first sentence of the result or the agent's reply, cut to about 100 characters, `Failed` for a failure, or a line for a cancelled task that says why it stopped. A task the model cancelled with `task_cancel` shows `Stopped early since it was no longer needed` with a success check, since the model no longer needed that work. A task stopped by [cancelling the turn](#message-hooks) shows `Stopped by request`, one stopped because its turn ended shows `Stopped when the turn ended`, and one whose run stopped on its own shows `Stopped`. Those rows and failed rows never show a success check. In a DM or private channel, a failure also shows the first sentence of its error, as in `Failed: Rate limited`. Error text can carry internal details, so everywhere else, including proactive sessions without a known `audience`, shows `Failed` alone. A completed or failed task that ran for a second or more also shows how long it took: `Done in 1m 14s: Found three incidents.` or `Failed after 3m`.+- **How it ended:** the first sentence of the result or the agent's reply, cut to about 100 characters, `Failed` for a failure, or a line for a cancelled task that says why it stopped. A task the model cancelled with `eve__task_cancel` shows `Stopped early since it was no longer needed` with a success check, since the model no longer needed that work. A task stopped by [cancelling the turn](#message-hooks) shows `Stopped by request`, one stopped because its turn ended shows `Stopped when the turn ended`, and one whose run stopped on its own shows `Stopped`. Those rows and failed rows never show a success check. In a DM or private channel, a failure also shows the first sentence of its error, as in `Failed: Rate limited`. Error text can carry internal details, so everywhere else, including proactive sessions without a known `audience`, shows `Failed` alone. A completed or failed task that ran for a second or more also shows how long it took: `Done in 1m 14s: Found three incidents.` or `Failed after 3m`.-The plan title summarizes the turn: `Working on 2 tasks`, `1 of 2 tasks done`, `Waiting for approval`, then `Finished 2 tasks in 3m` or `Finished 3 tasks in 5m 12s: 1 failed, 1 stopped`. When every task calls a declared subagent or remote agent, and the turn calls three or fewer of them, the title names them instead of counting: `Asking researcher and reviewer`, then `Waiting on reviewer · 1 of 2 tasks done`. Tasks the model cancelled with `task_cancel` count as finished, not stopped. A resumable task continued in a later turn appears in that turn's card.+The plan title summarizes the turn: `Working on 2 tasks`, `1 of 2 tasks done`, `Waiting for approval`, then `Finished 2 tasks in 3m` or `Finished 3 tasks in 5m 12s: 1 failed, 1 stopped`. When every task calls a declared subagent or remote agent, and the turn calls three or fewer of them, the title names them instead of counting: `Asking researcher and reviewer`, then `Waiting on reviewer · 1 of 2 tasks done`. Tasks the model cancelled with `eve__task_cancel` count as finished, not stopped. A resumable task continued in a later turn appears in that turn's card.A turn stays open while its tasks work, so messages you send during that time continue the same turn. When that turn starts a task after every task on its card has settled, eve marks that card finished and posts a new card for the new work, so the new tasks don't update a card further up the thread.diff --git a/docs/concepts/built-in-tools.md b/docs/concepts/built-in-tools.mdindex 6c467ba25..93a1cfc90 100644--- a/docs/concepts/built-in-tools.md+++ b/docs/concepts/built-in-tools.md@@ -24,7 +24,7 @@ export default defineAgent({This turns off the optional defaults described below. Add back only the tools the agent needs with the command in each tool's section. Existing files under `agent/tools/` remain available, including same-name replacements such as `agent/tools/bash.ts`.-`connection_search` and `connection_execute` stay available when the agent has connections because they provide access to connection tools.+`defaultTools` doesn't affect [the catalog tools](#eve__search-eve__tool-and-eve__skill): eve adds them from what the agent declares, as listed in that section.### `bash`@@ -254,7 +254,7 @@ eve add tool/agentexport { default } from "eve/tools/agent";```-An authored tool at `agent/tools/agent.ts` replaces the framework behavior. Re-export the definition above to restore direct root-copy delegation, export another tool such as `agentRouter()` to change the model-facing behavior, or disable the slot. `agentRouter()` runs each call as a [task](/docs/tools/workflows#run-calls-as-tasks-task), which adds `task_wait` and `task_cancel`:+An authored tool at `agent/tools/agent.ts` replaces the framework behavior. Re-export the definition above to restore direct root-copy delegation, export another tool such as `agentRouter()` to change the model-facing behavior, or disable the slot. `agentRouter()` runs each call as a [task](/docs/tools/workflows#run-calls-as-tasks-task), which adds `eve__task_wait` and `eve__task_cancel`:```ts title="agent/tools/agent.ts"import { disableTool } from "eve/tools";@@ -262,56 +262,36 @@ import { disableTool } from "eve/tools";export default disableTool();```-### `load_skill`+### `eve__search`, `eve__tool`, and `eve__skill`-`load_skill` pulls an on-demand [skill](../skills)'s instructions into the current turn. It appears only when the agent declares skills and adds no execution surface by itself.+These tools let the model reach what isn't in its tool list. `eve__search` finds tools defined with `deferred: true`, agents defined with `tool: "deferred"`, every tool from the agent's [connections](../connections), and deferred [skills](../skills). `eve__tool` calls a deferred tool, agent, or connection tool by name, and `eve__skill` loads a skill, deferred or not. There is no add command; eve adds each one from what the agent declares, even when `defaultTools` is `false`:-```sh-eve add tool/load_skill-```--```ts title="agent/tools/load_skill.ts"-export { default } from "eve/tools/load_skill";-```--Override it:--```ts title="agent/tools/load_skill.ts"-import { defineTool } from "eve/tools";-import { loadSkill } from "eve/tools/load_skill";--export default defineTool({- ...loadSkill,- description: "Load instructions for an available skill.",-});-```--Disable it:--```ts title="agent/tools/load_skill.ts"-import { disableTool } from "eve/tools";--export default disableTool();-```+- `eve__tool` comes with anything it can call: a deferred tool or agent, a connection, or a dynamic resolver in `agent/tools/`, `agent/subagents/`, or `agent/connections/` that may add one at runtime.+- `eve__skill` comes with any skill, static or from a dynamic skill resolver, so an agent whose only catalog entries are listed skills gets `eve__skill` alone.+- `eve__search` comes with anything it could find: whatever `eve__tool` can call, plus deferred skills and dynamic skill resolvers.+- An agent with none of these gets no catalog tools.-### `connection_search`+The decision depends only on what the agent declares, never on what its resolvers return, so the tool list stays the same for a deployment and never changes within a session.-`connection_search` and `connection_execute` give the model every tool from the agent's [connections](../connections) without adding each tool to the model's tool list. eve adds both when the agent has a static connection or a dynamic connection resolver, even when `defaultTools` is `false`, so there is no add command.+- `eve__search({ query, limit? })` returns the best matches, up to `limit` (default 10, at most 50). `query` is required and must contain a word. A tool match has its exact `tool` name, its `description`, and a TypeScript `signature` rendered from its schemas. A deferred skill's match has its `skill` name, its `description`, and the `path` of its `SKILL.md` when it has supporting files. A result with `tool` is called with `eve__tool`, and one with `skill` is loaded with `eve__skill`. Tools and skills rank together, in tiers: an entry whose name is exactly the query, then one whose name after its last `__` is the query, such as `linear__create_issue` or `sre__create_issue` for `create_issue`, then the tools of a connection named by the query, then entries whose names, or names after their last `__`, start with the query, then keyword matches in names, parameters, and descriptions. So `eve__search({ query: "linear" })` lists the `linear` connection's tools first. One-letter and filler words such as `a` and `the` don't count toward keyword matches unless the query has nothing else.+- A query whose first word contains `__` searches one namespace: everything before its last `__`, without trailing underscores. The namespace only filters. `eve__search` keeps names under `<namespace>__` and anything named exactly the namespace, such as a connection's sign-in entry, then ranks them by the whole query, so `eve__search({ query: "linear__" })` returns only the `linear` connection's tools, or its sign-in result, and an exact name that contains `__` still ranks first. eve lists only the connections that can own names in that namespace, so other connections make no network call and don't appear in `unavailable`. A namespace that matches nothing fails with the closest connection names. Characters that can't appear in a name, such as a leading `^`, are ignored; `query` isn't a regular expression.+- `eve__search` never asks the user to sign in. A connection whose server won't list its tools until the user signs in appears as one result named after the connection, such as `linear`; a connection whose server lists its tools without a token is searched like any other. A connection whose tools fail to load, or don't load within 10 seconds, appears under `unavailable` with its `error`, such as `"crm" did not list its tools within 10s. Try again later.` Matches from everything else are still returned, so a slow server doesn't hold up the search. An `eve__tool` call on that connection's tools fails with the same error.+- `eve__tool({ name: "linear" })`, with a connection's own name, asks the user to sign in to that connection when its tools need it, and returns once they can be listed, so the next `eve__search` finds them. On a server that lists its tools without sign-in, it confirms the tools are available without asking.+- `eve__tool({ name, input? })` calls the entry named `name` with `input`, which defaults to `{}`. A connection tool's name is `<connection>__<tool>`, such as `linear__list_issues`. eve checks `input` against the entry's input schema. An invalid input fails with the entry's signature, an unknown name fails with the closest names, a skill's name fails with a reminder to load it with `eve__skill`, and a tool already in the model's tool list fails with a reminder to call it directly. When the server asks for the user's authorization, the call asks the user to sign in and parks until sign-in completes.+- `eve__skill({ name })` loads the named skill, deferred or not, and returns its instructions. An unknown name fails with the closest skill names, a tool's name fails with a reminder to call it with `eve__tool` or directly, and a name that matches a connection says to find its tools with `eve__search({ query: "<connection>__" })`.-- `connection_search({ query?, connection?, signIn?, limit?, offset? })` returns matching tools with their connection, name, description, and a TypeScript signature rendered from the tool's schemas. Omit `query` to list every tool, or pair it with `connection` to list one connection's tools. A plain search never asks the user to sign in. For a connection whose server will not list its tools until the user signs in, the result tells the model that sign-in is needed: the connection appears under `unavailable` with `requiresSignIn: true` and an error that points to `signIn: true`. Tools from the other connections are still returned.-- `connection_search({ connection, signIn: true, query? })` asks the user to sign in to that one connection when they have not yet, then returns its matching tools. Without `connection` it fails, so the user is asked about one service at a time.-- `connection_execute({ connection, tool, input })` checks `input` against the tool's input schema, calls the tool, and returns its result. When the server asks for the user's authorization, it also asks the user to sign in and the call parks until sign-in completes. The stream reports the call as a nested action named `<connection>__<tool>`, such as `linear__list_issues`, whose `parentCallId` is the `connection_execute` call id.+After `eve__tool` or `eve__skill` resolves its entry, the call runs exactly like a direct call to that entry. A skill load reports a `load-skill` action named for its skill and an `execute_tool eve:load-skill` span. Approval policies and `approvedTools`, workflow tools and tasks, agents, `endsTurn`, `toModelOutput`, hooks, and stream events all see the entry's own name and input, such as `linear__list_issues`. Only model history records the call as `eve__tool` or `eve__skill`.-The definitions of both tools never change during a session. eve lists connection names and descriptions in append-only context messages rather than in the system prompt, so finding a tool, signing in, or resolving a dynamic connection keeps the cached prompt prefix.+eve tells the model what it can reach in an append-only context message rather than in the system prompt: which kinds of deferred entries exist, up to 20 namespaces (the first `__` segment of deferred names, such as `sre` for `sre__list_alerts`), and up to 20 connections with their descriptions. It never names a deferred entry, so deferred entries stay out of context until `eve__search` finds them. A later step appends the listing again only when that changes; a deferred entry added to a listed namespace, or without one, changes nothing. The definitions of the catalog tools never change within a deployment, so adding a deferred entry, signing in, or resolving a dynamic connection keeps the cached prompt prefix.-The tools cannot be replaced or disabled. The compiler rejects an authored `agent/tools/connection_search.ts`, `agent/tools/connection_execute.ts`, or `agent/tools/connection_tools.ts`. An agent without connections has neither tool.+The tools eve adds itself all live in the `eve` namespace: `eve__search`, `eve__tool`, `eve__skill`, `eve__task_wait`, `eve__task_cancel`, and `eve__reply`. Nothing you author or resolve may be named `eve` or start with `eve__`: tools, subagents, skills, connections, and extension mounts, static or dynamic, since a connection or mount named `eve` would own every `eve__` name. The compiler rejects an authored tool, skill, connection, or mount, such as `agent/tools/eve__search.ts`; eve rejects an authored subagent when the agent loads, and a dynamic entry when its resolver returns it. Any other name is free, including `search` and `execute`. A connection's own name is also the entry that signs the user in to it.-### `task_wait` and `task_cancel`+### `eve__task_wait` and `eve__task_cancel`-eve adds `task_wait` and `task_cancel` when the agent has a tool that runs its calls as [tasks](/docs/tools/tasks): any agent tool, including the built-in `agent` tool, declared subagents, and remote agents, or a tool such as `agentRouter()`, the `workflow` tool, or an authored workflow tool that defines `task(input, ctx)` or `serve(receive, ctx)`. There is no add command, and the tools are not workflow tools. Both names are reserved: the compiler rejects an authored `agent/tools/task_wait.ts` or `agent/tools/task_cancel.ts`.+eve adds `eve__task_wait` and `eve__task_cancel` when the agent has a tool that runs its calls as [tasks](/docs/tools/tasks): any agent tool, including the built-in `agent` tool, declared subagents, and remote agents, or a tool such as `agentRouter()`, the `workflow` tool, or an authored workflow tool that defines `task(input, ctx)` or `serve(receive, ctx)`. There is no add command, and the tools are not workflow tools. Like every `eve__` name, both are reserved.-- `task_wait({ timeoutSeconds? })` parks the turn until any task has a result, a new message arrives, or `timeoutSeconds` pass, and returns at once when a result is already waiting. While it waits, the stream reports `turn.waiting` for the open turn. Results arrive in a `<task_result>` message right after it returns. Waiting never stops a task.-- `task_cancel({ taskId })` stops a task's current work and says so, or says the task had no work to stop when it already finished or is an idle [resumable task](/docs/tools/workflows#resumable-tasks-serve). An id that names no task fails with `UNKNOWN_TASK`. A resumable task stays available after a cancel.+- `eve__task_wait({ timeoutSeconds? })` parks the turn until any task has a result, a new message arrives, or `timeoutSeconds` pass, and returns at once when a result is already waiting. While it waits, the stream reports `turn.waiting` for the open turn. Results arrive in a `<task_result>` message right after it returns. Waiting never stops a task.+- `eve__task_cancel({ taskId })` stops a task's current work and says so, or says the task had no work to stop when it already finished or is an idle [resumable task](/docs/tools/workflows#resumable-tasks-serve). An id that names no task fails with `UNKNOWN_TASK`. A resumable task stays available after a cancel.Review these tools before production use. Disable, wrap, restrict, or require approval for any tool that can access the filesystem, network, shell, or sensitive data.diff --git a/docs/concepts/context-control.md b/docs/concepts/context-control.mdindex 05cc428c9..1e15d24ea 100644--- a/docs/concepts/context-control.md+++ b/docs/concepts/context-control.md@@ -28,7 +28,7 @@ Use `instructions.ts` when you need typed helpers, build-time composition, or a## Load procedures on demand with `skills/`-Use skills for optional procedures that would otherwise make the always-on prompt unnecessarily large. eve advertises each skill's description and loads the full instructions only when the model calls `load_skill`.+Use skills for optional procedures that would otherwise make the always-on prompt unnecessarily large. eve advertises each skill's description and loads the full instructions only when the model calls `eve__skill({ name })`. A deferred skill isn't named in context, so even its description stays out of context until the model searches for it.### Flat skilldiff --git a/docs/concepts/execution-model-and-durability.mdx b/docs/concepts/execution-model-and-durability.mdxindex 95f9b90e8..ae1c6d714 100644--- a/docs/concepts/execution-model-and-durability.mdx+++ b/docs/concepts/execution-model-and-durability.mdx@@ -125,7 +125,7 @@ Some work has to wait, including a human approving a [tool](../tools) or an inteA workflow tool can suspend too, with its original tool call still pending and the turn parkeduntil the workflow settles. `defineTool` always executes inside its initiating step. A workflowtool that defines `task(input, ctx)` or `serve(receive, ctx)` doesn't park the turn: its call returns a receipt, the run keeps working,-and the turn parks only when the model calls `task_wait` or tries to end the turn while the task+and the turn parks only when the model calls `eve__task_wait` or tries to end the turn while the taskworks. See [tasks](../tools/workflows#run-calls-as-tasks-task).An awaited workflow operation provides the durable wait. A generator's `yield` reports progress.diff --git a/docs/concepts/sessions-runs-and-streaming.md b/docs/concepts/sessions-runs-and-streaming.mdindex 3abce8dcf..46d94907b 100644--- a/docs/concepts/sessions-runs-and-streaming.md+++ b/docs/concepts/sessions-runs-and-streaming.md@@ -113,7 +113,7 @@ The client validates the `x-eve-stream-version` header on every connection. It aWhen a streamed tool input becomes a validated call, its `action.input.appended` events precede the matching `actions.requested` event. The default client reducer projects the potentially incomplete JSON as a `dynamic-tool` part with `state: "input-streaming"` and cumulative text in `inputText`. `actions.requested` replaces that part with `state: "input-available"` and the validated `input`. Excluded internal actions never publish their input stream.-A nested action is a call a tool makes on the model's behalf, such as each connection tool call `connection_execute` makes. eve reports it as its own `actions.requested` and `action.result` pair right before the calling tool's `action.result`. The nested tool call's `parentCallId` holds the calling tool's `callId`. Nested actions never enter model history, and clients that ignore `parentCallId` render them as ordinary tool calls.+A call the model makes through [`eve__tool` or `eve__skill`](/docs/concepts/built-in-tools#eve__search-eve__tool-and-eve__skill) is reported as a call to the entry it names. Its `actions.requested` and `action.result` events carry that entry's name and input, such as `linear__list_issues`, never the catalog tool's. Such a call publishes no `action.input.appended` events, because its entry is known only once its input is complete.`action.partial` carries one complete preliminary output snapshot from an authored async-generator tool. A later partial for the same `callId` replaces it, and `action.result` is the final snapshot. When the durable writer is busy, eve may keep only the newest adjacent partial for a call. Treat partials as last-write-wins: a durable step can retry and replay overlapping event runs. Provider-executed tool progress and MCP progress notifications are not projected as `action.partial` events.@@ -129,7 +129,7 @@ A question or sign-in from inside a running call does not end the turn. This covA call to a tool that runs as a task, such as an agent tool, `agentRouter()`, the `workflow` tool, or a workflow tool that defines [`task(input, ctx)`](/docs/tools/workflows#run-calls-as-tasks-task) or [`serve(receive, ctx)`](/docs/tools/workflows#resumable-tasks-serve), returns a receipt as its `action.result` and keeps working. `task.started` reports the task's `taskId` for the call's `callId` and `turnId`, both when a call starts a task and when a call reaches a resumable task by its `taskId`, before any event the task's run publishes for that call. `task.settled` comes once per call, with the same `turnId`, `name`, and `kind` as its `task.started`, `status` `"completed"`, `"failed"`, or `"cancelled"`, and the call's `output` when it completed or `error.message` when it failed. Events recorded by earlier eve versions omit `name` and `kind` on `task.settled`, so match them to their `task.started` by `callId` when those fields are missing. Results reach the model as a message in its history, not as a stream event, so read outcomes from `task.settled`. A session the task's run opens with `ctx.agent` is announced with `agent.started` carrying the `taskId`. An `input.requested`, `authorization.required`, or `authorization.completed` event from a task's run carries its `taskId`. Each of these events names the call the task is serving when it happens: a resumable task's later call brings its own `callId` and `turnId`, which may belong to a later turn than the call that started the task. An `input.requested` relayed from a subagent or a workflow tool run also carries `callId`, the call in this session that the request serves. The request keeps the coordinates it was raised with, and a relayed approval's `request.action.callId` still names the asking run's call, not one of this session's.-A turn doesn't end while tasks are working. When the model ends its text early, eve holds the turn and the stream emits `turn.waiting` for it; the turn continues once a result arrives or its caller writes. In a root session, that text completes as an ordinary `message.completed`. In a child session or a schedule's session, it reports `finishReason: "tool-calls"`, so channels post only the final reply. When the model calls `task_wait` and no result is ready yet, the stream also emits `turn.waiting`. Either way, the next `step.started` for the same `turnId` means the turn resumed, and `session.waiting` comes only after the turn ends. See [Tasks](/docs/tools/tasks#turns-wait-for-their-tasks).+A turn doesn't end while tasks are working. When the model ends its text early, eve holds the turn and the stream emits `turn.waiting` for it; the turn continues once a result arrives or its caller writes. In a root session, that text completes as an ordinary `message.completed`. In a child session or a schedule's session, it reports `finishReason: "tool-calls"`, so channels post only the final reply. When the model calls `eve__task_wait` and no result is ready yet, the stream also emits `turn.waiting`. Either way, the next `step.started` for the same `turnId` means the turn resumed, and `session.waiting` comes only after the turn ends. See [Tasks](/docs/tools/tasks#turns-wait-for-their-tasks).`step.failed` and `turn.failed` carry `{ code, message, details? }` for the failed fragment or turn, and `session.failed` is the terminal session-level variant. `turn.cancelled` is not a failure: the cancelled turn ends without any failure event, `session.waiting` follows, and the session accepts the next message normally. Whatever the turn streamed before cancellation stays on the stream. Durable history keeps the accepted user input and previously settled work, and discards incomplete assistant output. Tool calls the cancellation stopped stay in history, each answered as cancelled, so the model sees that the work started and stopped instead of a request left unanswered. Before `turn.cancelled`, the stream reports requests the cancel strands as `cancelled` in `input.resolved`. These include the turn's own questions, the session-limit prompt, and requests relayed from the work it stops. When a turn requested an output schema, the finalized payload lands on `result.completed` as `data.result` before the turn boundary. `authorization.required` carries the sign-in challenge (`data.authorization` may include `url`, `userCode`, `expiresAt`, `instructions`), and `authorization.completed` carries `data.outcome` (`"authorized" | "declined" | "failed" | "timed-out"`). Both carry `data.principalId` when a session principal started the sign-in. It holds the same value as `responderPrincipalId` on approval events, so a channel can deliver the challenge privately to that person even after someone else starts a later turn.@@ -215,7 +215,7 @@ curl -X POST http://127.0.0.1:2000/eve/v1/session/<sessionId> \-d '{"inputResponses":[{"requestId":"req_A","optionId":"approve"}]}'```-Message sends default to `"steer"`. Before assistant output begins, eve interrupts pending model generation and continues the same turn with the correction. Reasoning and provider search progress do not count as assistant output. An executing eve tool finishes safely, and its result is preserved before the correction reaches the next model call. A steering message aborts the [`ctx.abortSignal`](/docs/tools/workflows#stop-early-for-a-new-message) of each `execute` workflow tool call the turn is waiting on: its questions are withdrawn, `sleep` and `ask_question` stop early, and the call settles with what its body returns. A steering message ends a `task_wait` but never interrupts a task. Only the turn's own caller steers it; another caller's message waits for the turn to end. After assistant output starts, steering applies at the next committed workflow boundary; text already streamed remains visible. Channels and TypeScript `Session.send(...)` calls can select `turnPolicy: "queue"` when the active turn should finish first. Structured `inputResponses` answer their addressed requests.+Message sends default to `"steer"`. Before assistant output begins, eve interrupts pending model generation and continues the same turn with the correction. Reasoning and provider search progress do not count as assistant output. An executing eve tool finishes safely, and its result is preserved before the correction reaches the next model call. A steering message aborts the [`ctx.abortSignal`](/docs/tools/workflows#stop-early-for-a-new-message) of each `execute` workflow tool call the turn is waiting on: its questions are withdrawn, `sleep` and `ask_question` stop early, and the call settles with what its body returns. A steering message ends an `eve__task_wait` but never interrupts a task. Only the turn's own caller steers it; another caller's message waits for the turn to end. After assistant output starts, steering applies at the next committed workflow boundary; text already streamed remains visible. Channels and TypeScript `Session.send(...)` calls can select `turnPolicy: "queue"` when the active turn should finish first. Structured `inputResponses` answer their addressed requests.If the session is waiting on a human-in-the-loop approval, respond with the channel’s Approve or Cancel controls. Text that doesn't match an option doesn't approve the call. From the person the turn serves, it steers the turn and cancels the approval, even when sent with `turnPolicy: "queue"`, since the turn can't end until that person acts; from anyone else, it waits until the turn ends. If they had already approved some calls in the batch, those calls still run. Cancelling the turn withdraws the approval, and a later answer to it approves nothing.diff --git a/docs/connections/mcp.mdx b/docs/connections/mcp.mdxindex 4be2a65fb..f6878088e 100644--- a/docs/connections/mcp.mdx+++ b/docs/connections/mcp.mdx@@ -3,7 +3,7 @@ title: "MCP Connections"description: "Connect an eve agent to a remote MCP server, authorize it with Vercel Connect or static credentials, and control which tools the model can discover."----MCP connections point eve at a remote MCP server you do not author. The server publishes its tools and schemas, and the model finds and calls them through `connection_search` and `connection_execute`.+MCP connections point eve at a remote MCP server you do not author. The server publishes its tools and schemas, and the model finds them with `eve__search` and calls them with `eve__tool`.Use MCP when the service already has an MCP server, when the server owns tool schemas dynamically, or when one connection should expose a family of related remote tools. Use an [OpenAPI connection](./openapi) instead when the service publishes an HTTP API contract and you want eve to generate one tool per operation.@@ -48,10 +48,10 @@ For OAuth-backed MCP servers, use the shared [Vercel Connect flow](../connectioneve asks the user to sign in only when the MCP server asks for it. When a connection has `auth` but no token yet, because its `getToken` throws `ConnectionAuthorizationRequiredError`, eve connects without an `Authorization` header and lists the server's tools. Any other error from `getToken` still fails the connection.-- **The server rejects every request without a token.** Most hosted OAuth servers, such as Linear, GitHub, and Notion, answer `initialize` or `tools/list` with HTTP `401`. A plain `connection_search` lists the connection under `unavailable` with `requiresSignIn: true`. Calling `connection_search({ connection, signIn: true })` or any of its tools with `connection_execute` asks the user to sign in.-- **The server lists tools without a token.** `connection_search` returns its tools with no prompt, and `connection_execute` runs tools the server answers anonymously. Sign-in starts only when a tool call comes back with HTTP `401`, or with an error result that carries a `WWW-Authenticate` challenge in `_meta["mcp/www_authenticate"]`.+- **The server rejects every request without a token.** Most hosted OAuth servers, such as Linear, GitHub, and Notion, answer `initialize` or `tools/list` with HTTP `401`. `eve__search` lists the connection as one result named after it, such as `linear`. Calling that name with `eve__tool({ name: "linear" })`, or any of its tools with `eve__tool`, asks the user to sign in.+- **The server lists tools without a token.** `eve__search` returns its tools with no prompt, and `eve__tool` runs tools the server answers anonymously. Sign-in starts only when a tool call comes back with HTTP `401`, or with an error result that carries a `WWW-Authenticate` challenge in `_meta["mcp/www_authenticate"]`.-After the user signs in, the call that asked for it runs again. eve reconnects with the token and reloads the tool list, which may now include tools the server hides from anonymous callers. A connection that already has a token sends it on every request, as before.+After the user signs in, the turn resumes and the model makes the call again. eve reconnects with the token and reloads the tool list, which may now include tools the server hides from anonymous callers. A connection that already has a token sends it on every request, as before.A server that leaves protected tools out of its anonymous `tools/list` never triggers sign-in, because the model cannot find those tools. Servers that support anonymous use should list every tool and refuse the protected calls.@@ -180,7 +180,7 @@ Authenticate the connection as this deployment, with a stable service credential## Control result size-`connection_execute` returns an MCP tool's `structuredContent` when the server sends it. Otherwise it returns the result's text, parsed as JSON when the tool declares no output schema. Image, audio, and resource content reaches the model as file parts, and a result with `isError: true` fails the call with its text. MCP connections do not expose a per-result transform equivalent to an authored tool's [`toModelOutput`](/docs/tools#shape-what-the-model-sees-with-tomodeloutput).+An MCP tool call returns the tool's `structuredContent` when the server sends it. Otherwise it returns the result's text, parsed as JSON when the tool declares no output schema. Image, audio, and resource content reaches the model as file parts, and a result with `isError: true` fails the call with its text. MCP connections do not expose a per-result transform equivalent to an authored tool's [`toModelOutput`](/docs/tools#shape-what-the-model-sees-with-tomodeloutput).When a remote tool returns more data than the model needs, narrow the result at the MCP server. Prefer a purpose-built search or summary tool, or return a stable handle that another call can use to fetch a smaller slice. If you do not control the server and an equivalent upstream API is available, replace only that operation with an authored tool that stores the full payload outside model context and projects the needed fields through `toModelOutput`. Block the original MCP operation through [`tools.block`](#tool-filters) so the model does not see duplicate tools.diff --git a/docs/connections/openapi.mdx b/docs/connections/openapi.mdxindex 734c96bd1..cdcf6355a 100644--- a/docs/connections/openapi.mdx+++ b/docs/connections/openapi.mdx@@ -24,7 +24,7 @@ export default defineOpenAPIConnection({});```-Each operation becomes a tool named by its `operationId`, reported as `<connection>__<operationId>`, for example `petstore__getInventory`. When an operation has no `operationId`, eve derives a deterministic `<method>_<sanitized-path>` name instead. The model calls it with `connection_execute({ connection: "petstore", tool: "getInventory", input })` and receives `{ status, statusText, body }`.+Each operation becomes a tool named by its `operationId`, reported as `<connection>__<operationId>`, for example `petstore__getInventory`. When an operation has no `operationId`, eve derives a deterministic `<method>_<sanitized-path>` name instead. The model calls it with `eve__tool({ name: "petstore__getInventory", input })` and receives `{ status, statusText, body }`.`spec` can be a URL that eve fetches at runtime, or an inline parsed OpenAPI object. A spec URL must use `https` (plain `http` is allowed only for loopback hosts such as `localhost` during local development), and eve re-checks the transport after any redirects. Prefer a URL when the provider owns the contract and updates it; prefer an inline object for private APIs, generated specs you pin in source control, or small hand-authored contracts.diff --git a/docs/connections/overview.mdx b/docs/connections/overview.mdxindex bf792213c..fbce413e8 100644--- a/docs/connections/overview.mdx+++ b/docs/connections/overview.mdx@@ -6,7 +6,9 @@ url: /connectionsA connection wires an agent into an external server you don't author, either an MCP server (Linear, GitHub, a warehouse) or any HTTP API with an OpenAPI document. eve handles the parts you'd otherwise hand-roll, discovering the remote tools, surfacing them to the model, and brokering auth. To instead let eve communicate with users through an external service, use a [channel](/docs/channels/overview).-Connections live under `agent/connections/`. A static connection's runtime name comes from the filename, so `agent/connections/linear.ts` registers as `"linear"`. A dynamic connection file can instead resolve a caller-specific connection set at session or turn boundaries. The model never sees a connection's URL or credentials. It finds tools with the built-in `connection_search` and calls them with `connection_execute`. Stream events, approval policies, and hooks name each call by its qualified name, `<connection>__<tool>` (e.g. `linear__list_issues`).+Connections live under `agent/connections/`. A static connection's runtime name comes from the filename, so `agent/connections/linear.ts` registers as `"linear"`. A dynamic connection file can instead resolve a caller-specific connection set at session or turn boundaries. The model never sees a connection's URL or credentials. It finds tools with the built-in [`eve__search`](/docs/concepts/built-in-tools#eve__search-eve__tool-and-eve__skill), where a query such as `linear__` searches one connection's tools, and calls them with `eve__tool`. Stream events, approval policies, and hooks name each call by its qualified name, `<connection>__<tool>` (e.g. `linear__list_issues`).++A connection owns its name and every name that starts with `<name>__`, so `linear__list_issues` always means the `linear` connection's `list_issues` tool. The compiler rejects a tool, subagent, or connection whose name is a static connection's name or starts with its `<name>__` prefix. eve applies the same rule to dynamic connections, tools, and subagents when they resolve. Both errors name the two sides of the conflict; rename either one.## MCP connections@@ -227,7 +229,7 @@ export default defineMcpClientConnection({Use the connector UID returned by `vercel connect create`; the display name passed to `--name` is not the UID. `connect("linear/myagent")` is shorthand for a user-scoped interactive OAuth connection: eve resolves a token for the active user before each tool call, emits `authorization.required` when that user has not authorized yet, and resumes the parked turn after the callback completes.-Discovering tools never posts a sign-in prompt for a service the answer does not need. Before the user signs in, eve connects to an MCP server without a token, so a server that lists its tools anonymously is searched like any other connection, and sign-in starts only when one of its tools asks for it (see [When eve asks for sign-in](/docs/connections/mcp#when-eve-asks-for-sign-in)). Most OAuth MCP servers reject even tool listing without the user's token. A plain `connection_search` lists those connections under `unavailable` with `requiresSignIn: true`, and the model asks for sign-in explicitly, one connection at a time, with `connection_search({ connection, signIn: true })`, which returns that connection's tools once the user signs in. Calling a tool with `connection_execute` on such a connection also asks for sign-in.+Discovering tools never posts a sign-in prompt for a service the answer does not need: `eve__search` never asks for sign-in. Before the user signs in, eve connects to an MCP server without a token, so a server that lists its tools anonymously is searched like any other connection, and sign-in starts only when one of its tools asks for it (see [When eve asks for sign-in](/docs/connections/mcp#when-eve-asks-for-sign-in)). Most OAuth MCP servers reject even tool listing without the user's token, so `search` lists such a connection as one result named after the connection, such as `linear`. The model asks for sign-in explicitly, one connection at a time, by calling that name with `eve__tool({ name: "linear" })`; once the user signs in, `eve__search({ query: "linear__" })` returns the connection's tools. Calling a tool with `eve__tool` on such a connection also asks for sign-in.A sign-in holds the turn open until the user connects, the turn is cancelled, or the user sends another message. That message steers the turn: eve cancels the sign-in, reports `authorization.completed` with `outcome: "declined"`, and the model reads the message next. Messages from other people in a shared thread wait until the turn ends, and the completed sign-in resumes the turn as the person who started it.diff --git a/docs/evals/assertions.mdx b/docs/evals/assertions.mdxindex 7c66502df..a8bf20118 100644--- a/docs/evals/assertions.mdx+++ b/docs/evals/assertions.mdx@@ -16,11 +16,11 @@ Scoped assertions take no explicit value and gate by default. Assertions on `t`| `t.messageIncludes(token)` | Joined assistant text contains `token` (string or RegExp) || `turn.outputEquals(value)` / `.outputMatches(schema)` | Deep equality or Standard Schema validation of turn/session structured output || `t.calledTool(name, opts?)` | A matching tool call completed (`input`, `output`, `status`, `count`) |-| `t.loadedSkill(skill, opts?)` | Sugar for `t.calledTool("load_skill", { input: { skill }, ...opts })` |+| `t.loadedSkill(skill, opts?)` | A load of `skill` completed (`output`, `status`, `count`) || `t.notCalledTool(name)` | No request for `name` in any lifecycle state || `t.toolOrder([...names])` | Tool requests appear in order |-| `t.usedNoTools()` | No tool calls at all |-| `t.maxToolCalls(n)` | At most `n` tool calls |+| `t.usedNoTools()` | No tool calls at all; skill loads don't count |+| `t.maxToolCalls(n)` | At most `n` tool calls; skill loads don't count || `t.noFailedActions()` | No tool, subagent, or skill action reported a failure || `t.calledSubagent(name, opts?)` | A subagent delegation happened (identity, remote, output, status, count) || `t.event(type, opts?)` / `t.notEvent(type, opts?)` | Typed event presence, data, and count matching |@@ -48,6 +48,8 @@ attached.messageIncludes("Sunny");`t.calledTool` and `t.usedNoTools` are mutually exclusive; assert one or the other, never both in the same run.+A call through [`eve__tool`](../concepts/built-in-tools#eve__search-eve__tool-and-eve__skill) is reported under the name and input of the entry it reaches, such as `linear__list_issues`, so assert on that entry; `eve__search` calls count as tool calls in `t.usedNoTools()` and `t.maxToolCalls(n)`, and skill loads don't. The tool-name assertions and `turn.requireToolCall` throw for `eve__tool` and `eve__skill`, whose calls are reported as their entries and as skill loads, and for a retired name such as `load_skill` unless the root agent declares a tool by that name or has a dynamic tool resolver, and say what to assert instead.+## Value assertions with `t.check``t.check(value, assertion)` grades an explicit value against a builder from `eve/evals/expect`. The value can be a turn’s `.message`, parsed JSON, or any local you computed:diff --git a/docs/evals/tool-stubs.mdx b/docs/evals/tool-stubs.mdxindex f01f35d16..a8e3541d6 100644--- a/docs/evals/tool-stubs.mdx+++ b/docs/evals/tool-stubs.mdx@@ -213,7 +213,9 @@ Repeated subagent sessions at the same path share that rule's sequence. DifferenUse the qualified operation name, such as `linear__list_issues` or `researcher/linear__list_issues`. Put operation arguments in `match` and its JSON result in `outcome.response`, or in each sequence entry's `response`.-Connection authentication and discovery stay live so eve can validate the operation and its arguments. Stubs do not provide an offline connection. The framework's `connection_search` and `connection_execute` tools cannot themselves be replaced.+Connection discovery stays live: eve lists the connection's tools to validate the operation and its arguments before a stub replaces the call, so stubs do not provide an offline connection.++A rule names the tool itself, whether the model calls it directly or through [`eve__tool`](/docs/concepts/built-in-tools#eve__search-eve__tool-and-eve__skill), as it does for deferred tools and connection operations. A call through `eve__tool` matches by the name and `input` of the tool it reaches. `eve__search`, `eve__tool`, and `eve__skill` cannot themselves be stubbed.Tools executed by the model provider, such as provider-hosted web search, have no eve executor to replace. A search tool implemented in your own agent can be stubbed.diff --git a/docs/extensions.md b/docs/extensions.mdindex fb5dab4a5..94833acc6 100644--- a/docs/extensions.md+++ b/docs/extensions.md@@ -198,6 +198,8 @@ Set `CRM_API_KEY` in the consumer's environment, such as `.env.local` for localThe mount adds `crm__` to named contributions: `tools/search.ts` becomes `crm__search`, `channels/webhook.ts` becomes `crm__webhook`, `schedules/sync.ts` becomes `crm__sync`, `connections/api.ts` becomes `crm__api`, and `subagents/reviewer/` becomes `crm__reviewer`. Channels keep their declared route paths, and schedules keep their cron expressions.+Do not mount an extension under the name of one of the agent's connections. A [connection owns](/docs/connections) every name that starts with `<name>__`, so the compiler rejects the mounted tools and subagents. Nor can a mount be named `eve`: eve reserves that namespace for its [built-in tools](/docs/concepts/built-in-tools#eve__search-eve__tool-and-eve__skill).+For an extension with no configuration, mount its default export directly:```ts title="agent/extensions/gizmo.ts"diff --git a/docs/guides/dev-tui.md b/docs/guides/dev-tui.mdindex f91701f77..bd3e169b3 100644--- a/docs/guides/dev-tui.md+++ b/docs/guides/dev-tui.md@@ -98,9 +98,9 @@ Type a message and press `Enter` to send it. When the agent asks a question or rThe activity line shows **Thinking** while the model reasons or waits to respond, **Generating** while it writes a response or tool input, and **Running** while tools execute. A blinking dot and elapsed time indicate progress, with token counts shown when available. While [tasks](../tools/tasks) work, the drawer header replaces this line: it shows the parent's current activity, task count, and turn elapsed time without token counts. **Waiting** means the parent is waiting on work; **Working** appears without a turn timer when tasks remain active between turns.-A tool call shows the tool's own `label`, or eve's copy for built-in tools such as `Read README.md`. Any other call shows a readable tool name without its extension or connection prefix, such as `List issues` for `linear__list_issues`. Approval prompts show the exact tool name unless the tool provides a title.+A tool call shows the tool's own `label`, or eve's copy for built-in tools such as `Read README.md`. Any other call shows a readable tool name without its extension or connection prefix, such as `List issues` for `linear__list_issues`. Approval prompts name the call the same way, such as `Approve Linear: List issues?`.-A task, such as a call to a subagent, writes one line to the transcript when it starts and one when it finishes, fails, or is stopped. The finished line names how long the task took and, for a subagent, what it did, such as `Read 3 files, Ran 2 commands`. Task starts use the same tool-call styling as synchronous calls. While tasks work, an activity drawer above the prompt shows each task's name, current activity, and elapsed time without repeating its launch prompt. Subagents use readable labels, such as `subagent(stock price)` for a `stock-price` subagent or `subagent(worker)` for the `code__worker` subagent of the code extension, with elapsed time beside the name and the latest observed tool grouped beneath an elbow. The tool remains visible between calls rather than switching to a starting placeholder. During questions and approvals, activity stays visible above the request with a shared divider when screen space permits; the request takes priority on small terminals. Background work started by a followed subagent appears beneath that subagent; deeper work uses an ownership path instead of further indentation. The panel caps its height and prioritizes tasks needing approval when work overflows. The UI does not show the model's own `task_wait` and `task_cancel` calls; their effect appears as the **Waiting** header and a stopped task's line.+A task, such as a call to a subagent, writes one line to the transcript when it starts and one when it finishes, fails, or is stopped. The finished line names how long the task took and, for a subagent, what it did, such as `Read 3 files, Ran 2 commands`. Task starts use the same tool-call styling as synchronous calls. While tasks work, an activity drawer above the prompt shows each task's name, current activity, and elapsed time without repeating its launch prompt. Subagents use readable labels, such as `subagent(stock price)` for a `stock-price` subagent or `subagent(worker)` for the `code__worker` subagent of the code extension, with elapsed time beside the name and the latest observed tool grouped beneath an elbow. The tool remains visible between calls rather than switching to a starting placeholder. During questions and approvals, activity stays visible above the request with a shared divider when screen space permits; the request takes priority on small terminals. Background work started by a followed subagent appears beneath that subagent; deeper work uses an ownership path instead of further indentation. The panel caps its height and prioritizes tasks needing approval when work overflows. The UI does not show the model's own `eve__task_wait` and `eve__task_cancel` calls; their effect appears as the **Waiting** header and a stopped task's line.The prompt stays open while the agent works, including turns started elsewhere, such as a scheduled run. While a turn is running, `Enter` sends your message immediately as steering. Before assistant output begins, the runtime interrupts pending model generation and continues the same turn with your correction. Executing tools finish safely. After output begins, steering applies at the next workflow boundary and preserves streamed text.diff --git a/docs/guides/dynamic-capabilities.md b/docs/guides/dynamic-capabilities.mdindex ec00cf123..30ebe31e3 100644--- a/docs/guides/dynamic-capabilities.md+++ b/docs/guides/dynamic-capabilities.md@@ -195,7 +195,7 @@ arguments, and approval options as static [MCP](../connections/mcp) and[OpenAPI](../connections/openapi) connections. Each resolved connection joinsthe per-step connection registry. eve announces it to the model in anappend-only context message, and the model reaches its tools through-`connection_search` and `connection_execute`, so the tool list never changes.+`eve__search` and `eve__tool`, so the tool list never changes.Set `instanceKey` on every authenticated dynamic connection. Use a stable,non-secret account or tenant identifier, and change it whenever the endpoint,@@ -217,6 +217,12 @@ eve does not prefix them with the file slug. A dynamic connection overrides asame-named static connection. Two effective dynamic resolvers cannot emit thesame name; namespace one map key to remove the ambiguity.+A connection owns its name and every name that starts with `<name>__`. When a+dynamic connection resolves, eve rejects it if its name is, or is the `__`+prefix of, one of the agent's tools, subagents, current dynamic tools, or+other connections. Dynamic tools and dynamic subagents get the same check when+they resolve. The error names both sides.+### Events and recoveryDynamic connections support `session.started` and `turn.started`. A turn result@@ -353,9 +359,11 @@ A recovery rebind is not a new lifecycle event, but it can run resolver code agaA single return produces one tool named after the file slug, identical to a static tool. A map names each entry by its **bare key** — there is no automatic slug prefix. If a bare name might collide, namespace the key yourself by including the prefix in the key (e.g. return `{ "tenant__export": … }` to get `tenant__export`).+A map key must be a legal tool name: ASCII letters, digits, underscores, and dashes, starting with a letter, up to 64 characters. A key cannot be `eve` or start with `eve__`, which eve [reserves](../concepts/built-in-tools#eve__search-eve__tool-and-eve__skill) for its built-in tools, and cannot be a connection's name or start with its `<name>__` prefix. eve rejects such a key when the resolver returns it.+### Conflicts-A dynamic connection, tool, or skill whose name matches an **authored** one **overrides** it — a per-caller resolver can replace a static capability by name. Two **dynamic** resolvers of the same capability type emitting the same name is a genuine ambiguity and throws; namespace one of the keys manually to resolve it.+A dynamic connection, tool, or skill whose name matches an **authored** one **overrides** it — a per-caller resolver can replace a static capability by name. Workflow tools and subagents are the exception: a dynamic tool can't take one of their names, and eve logs the error and skips that resolver's result. Two **dynamic** resolvers of the same capability type emitting the same name is a genuine ambiguity and throws; namespace one of the keys manually to resolve it.### Events@@ -432,9 +440,11 @@ export default defineDynamic({The caller's team gets its own playbook advertised as a loadable skill; everyone else gets nothing.-Skills follow the same naming rule as tools: a single `defineSkill(...)` is named after the file slug, while a map names each entry by its bare key (namespace the key yourself if it might collide). A dynamic skill overrides a same-named authored one; two dynamic resolvers emitting the same name throws.+Skills follow the same naming rule as tools: a single `defineSkill(...)` is named after the file slug, while a map names each entry by its bare key (namespace the key yourself if it might collide). Every name must use only ASCII letters, digits, underscores, and dashes, start with a letter, and have at most 64 characters; eve logs a resolver that returns another name and skips its result. A dynamic skill overrides a same-named authored one; two dynamic resolvers emitting the same name throws.++A dynamic skill with `deferred: true` is left out of the dynamic skill announcement and isn't named in context. The model finds it with `eve__search` and loads it with `eve__skill({ name })`, like a static deferred skill. When it overrides a static skill that isn't deferred, the system prompt still lists the static skill's description, and loading returns the dynamic body.-A dynamic skill that returns only `markdown` never starts a sandbox: eve keeps its instructions in session state and serves them from `load_skill`. When the skill also returns `files`, eve writes the package to the sandbox skill root when the resolver first returns it, and again only when its contents change or the session gets a new sandbox. A changed package replaces the previous directory, so files omitted from the new result are removed.+A dynamic skill that returns only `markdown` never starts a sandbox: eve keeps its instructions in session state and returns them when the model loads the skill. When the skill also returns `files`, eve writes the package to the sandbox skill root when the resolver first returns it, and again only when its contents change or the session gets a new sandbox. A changed package replaces the previous directory, so files omitted from the new result are removed.## Dynamic instructionsdiff --git a/docs/guides/frontend/overview.mdx b/docs/guides/frontend/overview.mdxindex 61fb6b06d..4d95dc01b 100644--- a/docs/guides/frontend/overview.mdx+++ b/docs/guides/frontend/overview.mdx@@ -172,7 +172,7 @@ supplied `session` remains externally owned and is never replaced by prewarming`data.messages` are eve-owned `EveMessage[]`. Assistant text and reasoning parts have a stable `id` from the event that created the part. Appends and completion retain that ID. Use `part.id` to key streamed assistant content, including successive text parts within one step. Older events without an ID get a best-effort fallback. User text parts may not have an ID.-Common text, reasoning, file, and dynamic-tool parts follow the [AI SDK `UIMessage`](https://ai-sdk.dev/docs/reference/ai-sdk-core/ui-message) rendering convention, but the types are not interchangeable. eve also exposes authorization and HITL metadata, and a file part's URL can be absent. Adapt those parts before passing messages to an API typed as `UIMessage[]`.+Common text, reasoning, file, and dynamic-tool parts follow the [AI SDK `UIMessage`](https://ai-sdk.dev/docs/reference/ai-sdk-core/ui-message) rendering convention, but the types are not interchangeable. eve also exposes authorization and HITL metadata, and a file part's URL can be absent. A tool part's `toolMetadata.eve.label` is how the call reads to people, the same text eve's channels show: the tool's [label](/docs/tools#label-tool-activity) or a readable form of its name, replaced by its completion label once the call settles. Show it instead of `toolName`, which can be an internal name such as `eve:load-skill`. Adapt those parts before passing messages to an API typed as `UIMessage[]`.A tool part's `state` comes from the shared lifecycle interpretation of the event stream. Its events supply input, output, and error text. A call waiting on an approval or a question reads `approval-requested` until the stream settles the request. An answer you submit doesn't change the part until then. `toolCallState(conversation, callId)` reads tool-call status, including task cancellation and calls interrupted without a result. The full event projection remains internal; `ConversationState` retains its messages, turns, inputs, tasks, and agent sessions.diff --git a/docs/guides/remote-agents.md b/docs/guides/remote-agents.mdindex 53885498a..12e9c6224 100644--- a/docs/guides/remote-agents.md+++ b/docs/guides/remote-agents.md@@ -87,7 +87,7 @@ The function may be async and must return a non-empty string. `auth` and `header## Calling a remote agent-By default, a remote agent is another subagent tool to the model. The model calls it the same way it calls a local subagent, with a `message` and an optional `taskId`. Set `tool: false` when an authored workflow tool should be the only model-facing routing surface; the workflow can still call the remote agent by its path-derived name through `ctx.agent()`. The message must carry the full task, including any context the remote agent needs, because it never receives the parent's conversation history.+By default, a remote agent is another subagent tool to the model. The model calls it the same way it calls a local subagent, with a `message` and an optional `taskId`. Set `tool: false` when an authored workflow tool should be the only model-facing routing surface; the workflow can still call the remote agent by its path-derived name through `ctx.agent()`. Set `tool: "deferred"` to keep the remote agent out of the model's tool list while the model can still find it with `eve__search` and call it with `eve__tool`. The message must carry the full task, including any context the remote agent needs, because it never receives the parent's conversation history.To require structured output, open a session with the remote agent from an authored workflow tool with `ctx.agent(name)` and pass `outputSchema` to `send()`. The schema applies to that turn, the structured value is the response's `data`, and the session accepts follow-up messages until the workflow run finishes. See [Delegate work: `ctx.agent`](../tools/workflows#delegate-work-ctxagent).@@ -211,7 +211,7 @@ The parent stream carries the same `task.started`, `agent.started`, and `task.seClients follow a remote child through the parent. [`session.agent(started).stream()`](./client/streaming#follow-a-subagent) reads the `agent.started` event's `streamPath`, a route on the parent deployment. The parent verifies that its session recorded the child for that tool call, resolves the remote agent's `auth` and `headers`, and relays the child's stream. A browser never calls the remote deployment or holds its credentials; it only needs access to the parent session.-Cancelling the parent turn or the agent task with `task_cancel` also cancels the remote child's current turn. eve resolves the remote's `headers` and `auth` again for every cancellation attempt, so rotating credentials work the same way as they do for session creation. Cancellation always uses the standard eve cancel path on `url`, even when `path` customizes only the create-session endpoint. The remote child reports `turn.cancelled` → `session.waiting` on its own stream; an older or unreachable remote is logged but cannot turn the parent's cancellation into a failure.+Cancelling the parent turn or the agent task with `eve__task_cancel` also cancels the remote child's current turn. eve resolves the remote's `headers` and `auth` again for every cancellation attempt, so rotating credentials work the same way as they do for session creation. Cancellation always uses the standard eve cancel path on `url`, even when `path` customizes only the create-session endpoint. The remote child reports `turn.cancelled` → `session.waiting` on its own stream; an older or unreachable remote is logged but cannot turn the parent's cancellation into a failure.When the workflow run that opened a remote child finishes, eve sends an authenticated `POST /eve/v1/session/:childSessionId/reset` for it. An agent task's run finishes when the parent session ends. Reset retires the parked remote session and recursively cleans up its descendants. The request uses freshly resolved `headers` and `auth`; failures are logged so an unreachable remote cannot block parent finalization.diff --git a/docs/instructions.mdx b/docs/instructions.mdxindex 2aecfa3ae..5910f324d 100644--- a/docs/instructions.mdx+++ b/docs/instructions.mdx@@ -62,11 +62,11 @@ A flat `agent/instructions.md` (or `.ts`) at the agent root and the directory caInstructions and [skills](./skills) both feed text into the model's context. The difference is timing:-| | Loaded | Use for |-| ------------------------ | -------------------------------------------------- | -------------------------------------------------------- |-| System-role instructions | Outside history on every model call | Permanent identity and standing rules |-| User-role instructions | Once at their static or dynamic lifecycle boundary | Application context that belongs in conversation history |-| `agent/skills/*` | On demand, when the model calls `load_skill` | Optional procedures that should not bloat every turn |+| | Loaded | Use for |+| ------------------------ | ----------------------------------------------------- | -------------------------------------------------------- |+| System-role instructions | Outside history on every model call | Permanent identity and standing rules |+| User-role instructions | Once at their static or dynamic lifecycle boundary | Application context that belongs in conversation history |+| `agent/skills/*` | On demand, when the model loads one with `eve__skill` | Optional procedures that should not bloat every turn |Keep instructions short and stable. Long or situational procedures belong in [skills](./skills), where they only enter context when the request calls for them.diff --git a/docs/reference/cli.md b/docs/reference/cli.mdindex 581fbe4f1..5ed67c1fa 100644--- a/docs/reference/cli.md+++ b/docs/reference/cli.md@@ -199,7 +199,7 @@ Run this first when something behaves unexpectedly. It confirms a file was disco}```-`toolInputSchemas` covers only tools compiled from tool files. It does not include provider-managed tools such as `web_search`, dynamic tools, connection tools, the tools eve generates at runtime for each subagent and remote agent, or the `task_wait` and `task_cancel` tools. Model providers can also transform a schema before the model reads it; those changes are not reflected here. To compute the same form for an input schema in your own code, such as in a test, call `serializeModelInputSchema(schema)` from `eve/tools`; it returns JSON Schema data and does not add `taskId`.+`toolInputSchemas` covers only tools compiled from tool files. It does not include provider-managed tools such as `web_search`, dynamic tools, connection tools, the tools eve generates at runtime for each subagent and remote agent, or the `eve__task_wait` and `eve__task_cancel` tools. Model providers can also transform a schema before the model reads it; those changes are not reflected here. To compute the same form for an input schema in your own code, such as in a test, call `serializeModelInputSchema(schema)` from `eve/tools`; it returns JSON Schema data and does not add `taskId`.## `eve build`@@ -397,7 +397,7 @@ Each workflow tool call has one `execute_tool <tool>` span. The first turn of eaOutbound MCP `tools/call` requests add MCP semantic attributes to the matching `execute_tool` span. When eve has no tool span to enrich, it creates a `CLIENT` `tools/call <tool>` span. Each `tools/list` discovery also has a `CLIENT` span. Configured OpenTelemetry propagation fields are injected into MCP `params._meta` for JSON-RPC bodies up to 1 MiB and propagation metadata up to 8 KiB; larger requests are sent unchanged. eve removes its audience and session-lineage baggage before forwarding, and keeps the input/output content-capture policy local.-`connection_execute` records the connector call as a child of its outer `execute_tool` span, with `agent.action.parent_call_id` identifying the outer call. Both spans retain `agent.action.kind=tool-call`; consumers can distinguish nested calls by the parent call ID. The nested span's `gen_ai.tool.call.arguments` holds the validated connector input, including schema defaults, when input capture is enabled. Its duration covers the reporting of the nested result, not the connector request; use the outer call's `execute_tool` span for execution timing.+A call the model makes through `eve__tool` or `eve__skill` is traced exactly like a direct call to the entry it names: its `execute_tool` span carries that entry's name and input, such as `execute_tool linear__list_issues`, and a workflow tool or agent reached this way gets the same span its direct call would. A skill load has an `execute_tool eve:load-skill` span. The AI SDK's own tracing channel, which eve does not filter, still reports the `eve__tool` or `eve__skill` tool running for each such call.Each root agent turn starts a new trace. With local caller trace context, that trace includes the first turn of each local subagent it starts. Remote child turns and later local child turns start separate traces. If a worker is replaced, the new worker uses the prepared trace context for the same turn. Supply the conversation ID to show all related traces, oldest first.diff --git a/docs/reference/typescript-api.md b/docs/reference/typescript-api.mdindex c5798373c..30f9eee77 100644--- a/docs/reference/typescript-api.md+++ b/docs/reference/typescript-api.md@@ -110,7 +110,7 @@ import template from "../../prompts/template.txt?raw";| `ctx.getToken(provider)` | Resolve a bearer token for an inline auth provider such as `connect("...")` || `ctx.requireAuth(provider)` | Evict and re-authorize an inline provider, commonly after a downstream `401` |-Tool definitions accept `availableInSubagents: false` to restrict the tool to top-level root sessions. `defineTool` definitions accept `endsTurn: true`, or a function that receives the `execute` output, to end the turn without a reply after a successful call; see [Tools](../tools#end-the-turn-after-a-tool-call). Authored workflow bodies also receive `ctx.agents`, a replay-stable map of callable-agent metadata, and `ctx.agent(name)`, which returns an `AgentSession` whose `send(message, options?)` resolves to a response with `result()`. A `"use step"` helper that receives the context directly should type it as `WorkflowStepToolContext`, which excludes `agents`, `agent`, and `ask`. The root-copy `agent` entry has an empty description when the root omits `description`. See [Workflows as tools](../tools/workflows#delegate-work-ctxagent) for the workflow-only context.+Tool definitions accept `availableInSubagents: false` to restrict the tool to top-level root sessions, and `deferred: true` to keep the tool out of the model's tool list so the model finds it with `eve__search` and calls it with `eve__tool`; see [Tools](../tools#keep-a-tool-out-of-the-tool-list). `defineTool` definitions accept `endsTurn: true`, or a function that receives the `execute` output, to end the turn without a reply after a successful call; see [Tools](../tools#end-the-turn-after-a-tool-call). Authored workflow bodies also receive `ctx.agents`, a replay-stable map of callable-agent metadata, and `ctx.agent(name)`, which returns an `AgentSession` whose `send(message, options?)` resolves to a response with `result()`. A `"use step"` helper that receives the context directly should type it as `WorkflowStepToolContext`, which excludes `agents`, `agent`, and `ask`. The root-copy `agent` entry has an empty description when the root omits `description`. See [Workflows as tools](../tools/workflows#delegate-work-ctxagent) for the workflow-only context.## Imports at a glance@@ -118,7 +118,7 @@ Tool definitions accept `availableInSubagents: false` to restrict the tool to to| --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ || `eve` | `defineAgent`, `defineRemoteAgent`, `defineDynamic`, agent config types || `eve/tools` | `defineTool`, `defineWorkflowTool`, `defineDurableCallback`, `defineDurableSchema`, `defineDynamic`, `disableTool`, tool types |-| `eve/tools/{bash,read_file,write_file,web_fetch,load_skill,glob,grep}` | Individual reusable tool definitions |+| `eve/tools/{bash,read_file,write_file,web_fetch,glob,grep}` | Individual reusable tool definitions || `eve/tools/approval` | Approval types and `auto`, `always`, `once`, `never` || `eve/tools/ask_question` | Opt-in `askQuestion` tool factory || `eve/tools/web_search` | Provider-managed `webSearch` configuration |diff --git a/docs/skills.mdx b/docs/skills.mdxindex 82f198c83..3b676c323 100644--- a/docs/skills.mdx+++ b/docs/skills.mdx@@ -1,15 +1,15 @@---title: "Skills"-description: "Author load-on-demand procedures the model pulls into context with load_skill."+description: "Author load-on-demand procedures the model pulls into context with eve__skill."---A skill is a model-loadable procedure that follows the `SKILL.md` convention. It is a markdown document, optionally a packaged directory with supporting files, that the model pulls into context on demand rather than carrying on every turn. eve advertises each skill's description, and the model loads the full body only when a turn calls for it. This is progressive disclosure, the same model the broader Agent Skills standard uses, so a skill authored against that standard ports over as-is.## How loading works-eve scans the files under `agent/skills/` and exposes each one's description to the model alongside a framework-owned `load_skill` tool. When a request matches a skill's description (or you name the skill outright), the model calls `load_skill`, and eve appends that skill's markdown to the active turn's context.+eve scans the files under `agent/skills/` and lists each one's name and description in the system prompt. When a request matches a skill's description (or you name the skill outright), the model loads it with `eve__skill({ name })`, and the skill's markdown becomes the call's result in the active turn. An agent with skills always has the [`eve__skill` tool](./concepts/built-in-tools#eve__search-eve__tool-and-eve__skill). A load reports a `load-skill` action named for the skill, and `t.loadedSkill` asserts it in [evals](./evals/assertions).-Listing skills and loading their instructions do not require a sandbox: `load_skill` returns static instructions from the compiled agent and dynamic instructions from session state. Only access to supporting package files requires a sandbox. Builds may also prewarm sandbox templates that contain static package files. Supporting files are available under `$HOME/.agents/skills/<skill>/`, with `/workspace/skills/<skill>/` as the fallback when `$HOME` is unavailable. Sibling references inside `SKILL.md`, such as `references/checklist.md`, are relative to the directory containing that specific `SKILL.md`.+Listing skills and loading their instructions do not require a sandbox: loading returns static instructions from the compiled agent and dynamic instructions from session state. Only access to supporting package files requires a sandbox. Builds may also prewarm sandbox templates that contain static package files. Supporting files are available under `$HOME/.agents/skills/<skill>/`, with `/workspace/skills/<skill>/` as the fallback when `$HOME` is unavailable. Sibling references inside `SKILL.md`, such as `references/checklist.md`, are relative to the directory containing that specific `SKILL.md`.The description is a routing hint, not a label. Write it as the task that should trigger activation:@@ -19,6 +19,19 @@ Use when the user needs a release checklist or changelog workflow.Loading a skill adds instructions, never a new execution surface. Tools stay visible whether a skill is loaded or not. If you need typed runtime behavior, reach for a [tool](./tools) instead.+## Deferred skills++An agent with many skills can keep most of them out of the system prompt. Set `deferred: true` in the frontmatter or on `defineSkill`:++```md title="agent/skills/pdf-forms/SKILL.md"+---+description: Fill and validate PDF forms.+deferred: true+---+```++A deferred skill isn't named in context; the catalog listing only says that deferred skills exist, and their namespaces. The model finds a deferred skill with `eve__search`, which ranks skills and tools together, and loads it with `eve__skill({ name })`. Its markdown, supporting files, and eval assertions work as they do for any other skill. Dynamic skills accept `deferred` too; see [Dynamic capabilities](./guides/dynamic-capabilities#dynamic-skills).+## Markdown vs `defineSkill`The smallest skill is a flat markdown file. The content is the procedure, and the name comes from the path.@@ -27,6 +40,8 @@ The smallest skill is a flat markdown file. The content is the procedure, and thUse the weather tool before answering forecast or temperature questions.```+A skill's name comes from its path and must use only ASCII letters, digits, underscores, and dashes, start with a letter or digit, and have at most 64 characters. The compiler rejects any other name.+A flat markdown skill can skip the `description` frontmatter. When it does, eve advertises the first non-empty, non-code-fence line of the body with any leading `#`, `>`, `*`, or `-` marker stripped. If the body has no such line, eve falls back to the literal `Instructions for the <name> skill.`, which is a weak routing hint, so add a `description` when you want the model to route on intent.A packaged skill is a directory with a `SKILL.md` plus sibling files like `references/`, `assets/`, and `scripts/`. The packaged `SKILL.md` must carry `description` frontmatter; it has no filename slug to fall back on.@@ -40,7 +55,7 @@ When the task is novel or ambiguous, gather evidence first, then answer with thekey facts and the remaining uncertainty.```-eve reads `description`, optional `license`, and string `metadata`; other `SKILL.md` frontmatter is accepted as a no-op.+eve reads `description`, optional `license`, string `metadata`, and `deferred`; other `SKILL.md` frontmatter is accepted as a no-op.When markdown can't express what you need (typed values, generated content, or inline sibling files), author the skill in TypeScript with `defineSkill` from `eve/skills`:diff --git a/docs/subagents/index.mdx b/docs/subagents/index.mdxindex 4b5bdba65..aaa7e5f19 100644--- a/docs/subagents/index.mdx+++ b/docs/subagents/index.mdx@@ -74,7 +74,9 @@ export default defineAgent({The parent can alternatively export `disableTool()` from a same-named tool path, such as `agent/tools/researcher.ts`. This suppresses the derived model tool without removing `researcher` from the workflow invocation registry. A same-named authored tool replaces the model-facing surface while the workflow can still call the underlying subagent with `ctx.agent("researcher", ...)`.-A declared local or remote subagent call returns a receipt, and the child's reply arrives as the task's result. Human input requests from the child surface on the parent session while the task works. `task_cancel` with the task's `taskId` cancels the child's current turn, and so does cancelling the calling turn.+Set `tool: "deferred"` to keep the subagent out of the parent model's tool list while the model can still call it. eve doesn't name the subagent in the parent's context; the model finds it with [`eve__search`](/docs/concepts/built-in-tools#eve__search-eve__tool-and-eve__skill) and calls it with `eve__tool`. The call then runs exactly like a direct subagent call. Use this when a parent has many specialists and you want to keep its tool list short.++A declared local or remote subagent call returns a receipt, and the child's reply arrives as the task's result. Human input requests from the child surface on the parent session while the task works. `eve__task_cancel` with the task's `taskId` cancels the child's current turn, and so does cancelling the calling turn.A mounted extension can also contribute declared subagents from `extension/subagents/`. The mount namespace prefixes the subagent visible to the consuming agent node: mounting an extension as `crm` exposes its `reviewer` subagent as `crm__reviewer`. The contributed subagent keeps its own isolated tools, connections, skills, hooks, instructions, sandbox, and nested subagents, and its modules can read configuration from the extension handle. See [Extensions](./extensions#add-a-subagent) for the authoring and override behavior.@@ -225,7 +227,7 @@ A directly declared subagent's tool name is the bare path-derived name, with noThe tool's description is the subagent's `description` followed by a sentence telling the model that the agent does not see the conversation and that `taskId` continues an agent task.-By default, a subagent named `researcher` collides with an authored tool named `researcher`. Set the subagent's `tool` to `false` when the authored tool intentionally wraps that subagent; the authored tool becomes the model-facing capability while `ctx.agent("researcher", ...)` continues to address the child. A same-named `disableTool()` removes the model-facing capability entirely. Other static and active dynamic collisions remain errors.+By default, a subagent named `researcher` collides with an authored tool named `researcher`. Set the subagent's `tool` to `false` when the authored tool intentionally wraps that subagent; the authored tool becomes the model-facing capability while `ctx.agent("researcher", ...)` continues to address the child. A same-named `disableTool()` removes the model-facing capability entirely. The same holds for dynamic subagents and dynamic tools: a dynamic subagent whose resolved config sets `tool: false` can be wrapped by an authored tool, and a dynamic tool can wrap a static subagent hidden either way. Any other collision is an error: eve rejects a static one when the deployment builds, and logs a dynamic one and skips that resolver's result.Do not rely on subagent delegation by itself as an approval boundary. Put sensitive tools behind `approval`, connection approval, route/session authorization, or other controls wherever those tools can be called.@@ -241,7 +243,7 @@ Subagent model calls automatically retry classified transient provider failures,A child parks after answering instead of terminating, keeping its session and conversation history alive. Pass the task's `taskId` to the same subagent tool with a new `message` to continue that session. Omitting `taskId` always starts a new child. When a child's turn fails, the call fails and the task ends. A `taskId` that names no task of that subagent tool, including one that ended, fails with `UNKNOWN_TASK`. Any later turn can continue the child, and each continuation runs with the auth of the caller that sends it.-A call with the `taskId` of a child that is still working joins the child's current turn, and that turn's final reply settles both calls. If the turn ends before the child reads the message, the message starts the child's next turn, and that turn's reply settles both calls. Use it to correct or add to work in progress. `task_cancel` with the task's `taskId` cancels the child's current turn and keeps its conversation, so a later call with the same `taskId` continues where it left off.+A call with the `taskId` of a child that is still working joins the child's current turn, and that turn's final reply settles both calls. If the turn ends before the child reads the message, the message starts the child's next turn, and that turn's reply settles both calls. Use it to correct or add to work in progress. `eve__task_cancel` with the task's `taskId` cancels the child's current turn and keeps its conversation, so a later call with the same `taskId` continues where it left off.eve adds a framework-injected `[Tasks]` note to the conversation that lists the working and idle tasks by `taskId` and tool, so the model can find a child to continue. See [Tasks](./tools/tasks) for how the model waits for, corrects, and cancels agent tasks.diff --git a/docs/tools/overview.mdx b/docs/tools/overview.mdxindex 46c515e74..74bf9b49a 100644--- a/docs/tools/overview.mdx+++ b/docs/tools/overview.mdx@@ -58,6 +58,25 @@ export default defineTool({The option defaults to `true` and works with `defineTool` and `defineWorkflowTool`. A root copy does not receive a tool with this option, and a tool authored inside a declared subagent is unavailable because that agent runs in a delegated session. This controls tool availability only; it does not add or remove entries from a workflow's `ctx.agents` registry.+### Keep a tool out of the tool list++Set `deferred: true` when the model needs a tool only occasionally, or when an agent has more tools than you want in every model request:++```ts title="agent/tools/export_report.ts"+export default defineTool({+ deferred: true,+ description: "Export a finished report as a PDF.",+ inputSchema: z.object({ reportId: z.string() }),+ execute({ reportId }) {+ return exportReport(reportId);+ },+});+```++A deferred tool is not in the model's tool list, and eve doesn't name it in context. The model finds its name, description, and signature with [`eve__search`](/docs/concepts/built-in-tools#eve__search-eve__tool-and-eve__skill) and calls it with `eve__tool({ name: "export_report", input })`. After that, the call runs exactly like a direct call: approvals, `endsTurn`, `toModelOutput`, hooks, and stream events see `export_report`, not `execute`.++The option works with `defineTool`, `defineWorkflowTool`, and dynamic tool entries. Provider tools such as `webSearch()` cannot be deferred, because the provider has to see their definition; the compiler rejects `deferred` on them. To defer an agent, set [`tool: "deferred"`](/docs/subagents#declared-subagents) on its `defineAgent` definition.+Compiled tools preserve Zod schema composition, including input and output schemas imported from helper modules. Durable replay metadata stays separate from the original schema, so adding it does not mutate shared schema objects.### End the turn after a tool call@@ -128,7 +147,9 @@ export default defineTool({The label is presentation only. It does not replace the path-derived tool name, enter model input, or change execution. eve records the bounded label on the corresponding public action event so activity renderers and other authorized stream consumers can use it without interpreting raw tool data. Built-in channel activity can show `Deploy storefront to production` instead of `Deploy`; if the callback throws or returns an empty label, activity falls back to a readable form of the tool name, such as `Sync accounts` for `sync_accounts`.-Provided tools define labels for `bash`, `glob`, `grep`, `load_skill`, `read_file`, `web_fetch`, `web_search`, and `write_file`.+Provided tools define labels for `bash`, `glob`, `grep`, `read_file`, `web_fetch`, `web_search`, and `write_file`.++Calls the model makes through the [catalog](/docs/concepts/built-in-tools#eve__search-eve__tool-and-eve__skill) read the same way. A deferred tool, agent, or connection tool called through `eve__tool` shows its own label, exactly as a direct call would. An `eve__search` call shows `Search tools for “linear issues”`, then `Searched tools for “linear issues”, found 5` when it returns. A skill load shows `Load skill release-notes`, then `Loaded skill release-notes`, and a connection's sign-in entry shows `Sign in to Linear`.### Stream preliminary tool resultsdiff --git a/docs/tools/tasks-upgrade.md b/docs/tools/tasks-upgrade.mdindex d13a9fdfb..b8b74a2aa 100644--- a/docs/tools/tasks-upgrade.md+++ b/docs/tools/tasks-upgrade.md@@ -148,7 +148,7 @@ The eval assertion `t.calledSubagent(name, { status })` keeps its API and readsA turn doesn't end while tasks are working, so one turn can span a wait. The streamreports each wait with the new `turn.waiting` event, which carries the open turn's `turnId`:-- **Held turns.** When the model ends its text while its tasks work, and when `task_wait` has no+- **Held turns.** When the model ends its text while its tasks work, and when `eve__task_wait` has noresult yet, the stream emits `turn.waiting`. The next `step.started` for the same `turnId` meansthe turn resumed. In a root session, the text before the wait completes as a normal`message.completed`; in child sessions and a schedule's sessions, that step reportsdiff --git a/docs/tools/tasks.md b/docs/tools/tasks.mdindex 168134180..67683b9c5 100644--- a/docs/tools/tasks.md+++ b/docs/tools/tasks.md@@ -31,7 +31,7 @@ Defining none of the three, or more than one, throws when the tool is defined.| `execute` workflow tool, including `sleep` and `ask_question` | Tool call; the turn waits | The tool result | Aborts the call's `abortSignal` || `task` workflow tool, including `agentRouter()` and the `workflow` tool | Task | A receipt, then a `task.result` message | Nothing || `serve` workflow tool, including every agent tool | Resumable task | A receipt, then a `task.result` message per reply | Nothing |-| `task_wait` | Wait inside the turn | Which tasks settled | Ends the wait |+| `eve__task_wait` | Wait inside the turn | Which tasks settled | Ends the wait || Plain and MCP tools | Inside the model step | The tool result | Applied at the next step boundary |Waiting alone doesn't need a task. A question, a `sleep`, or an approval inside an `execute` call@@ -97,7 +97,7 @@ conversation should continue while the review runs.A workflow tool body receives its signals from its context or, in a `serve` body, from each call`receive()` resolves:-- **`abortSignal`** aborts when the call's work should stop: by `task_cancel`, `session.cancel()`,+- **`abortSignal`** aborts when the call's work should stop: by `eve__task_cancel`, `session.cancel()`,a cancelled or failed turn, or the end of the session, and, for an `execute` call, by a steeringmessage that arrives while the turn waits on it. `execute` and `task` bodies read it as`ctx.abortSignal`. In a `serve` body, each call from `receive()` carries its own `abortSignal`;@@ -134,12 +134,12 @@ provided `sleep` tool races its timer against the signal.## What the model sees-eve adds `task_wait`, `task_cancel`, and a short system prompt block that explains tasks whenever+eve adds `eve__task_wait`, `eve__task_cancel`, and a short system prompt block that explains tasks wheneverthe agent has an agent tool or a `task` or `serve` workflow tool. The block tells the model to call-`task_wait` when it has nothing to say until a result arrives, and to reply when the person should+`eve__task_wait` when it has nothing to say until a result arrives, and to reply when the person shouldhear from it first, such as a confirmation that work is underway. In a child session or aschedule's session, where only the final reply reaches the caller, it tells the model to call-`task_wait` instead of replying while tasks work.+`eve__task_wait` instead of replying while tasks work.**Receipts.** A call that starts a task returns a receipt as its tool result. Task ids are the toolname and six characters. A resumable task's receipt tells the model how to reach it again, and a@@ -152,7 +152,7 @@ Sent to task researcher-7k2m9q. Its reply will arrive in a <task_result> message```**Results.** Each result arrives once, as a `<task_result>` block in a `task.result` message that-eve appends at the next step boundary, right after `task_wait` returns, or when a held turn+eve appends at the next step boundary, right after `eve__task_wait` returns, or when a held turnresumes. The body is the tool's `toModelOutput` projection of the output, or the error message fora failed call. All results in one message share a budget of 50 KB and 2,000 lines, after which thetext is cut and marked `[truncated]`.@@ -161,7 +161,7 @@ text is cut and marked `[truncated]`.<task_result id="deploy-4hd8sa" tool="deploy" status="completed">{"url":"https://…"}</task_result>```-**`task_wait({ timeoutSeconds? })`** controls when the model replies. The model should call it+**`eve__task_wait({ timeoutSeconds? })`** controls when the model replies. The model should call itsparingly, only when it deliberately wants to withhold a message from the user while waiting fora task result. Tasks keep running and their results reach the model without this call; the modelcan reply now if the user should hear from it.@@ -175,9 +175,9 @@ model reads which tasks settled and which are still working, for example:deploy-4hd8sa completed; its result follows. 1 task is still working: researcher-7k2m9q.```-**`task_cancel({ taskId })`** stops a task's current work and says so. A resumable task's answer+**`eve__task_cancel({ taskId })`** stops a task's current work and says so. A resumable task's answeralso tells the model how to give it new work. When the task has no work to stop, because it already-finished or is an idle resumable task, `task_cancel` says that instead:+finished or is an idle resumable task, `eve__task_cancel` says that instead:```textStopped deploy-4hd8sa; it won't report back.@@ -201,22 +201,22 @@ lists the working tasks and the 10 most recently used idle resumable tasks. The**Errors.** Two error codes are specific to tasks:-| Code | When |-| ---------------- | ---------------------------------------------------------------------------------- |-| `UNKNOWN_TASK` | A call's `taskId` names no unfinished resumable task that the same tool started |-| `UNKNOWN_TASK` | `task_cancel`'s `taskId` names no task; a finished or idle one has no work to stop |-| `TOO_MANY_TASKS` | A call would start a task, or make an idle one work, while 32 tasks are working |+| Code | When |+| ---------------- | --------------------------------------------------------------------------------------- |+| `UNKNOWN_TASK` | A call's `taskId` names no unfinished resumable task that the same tool started |+| `UNKNOWN_TASK` | `eve__task_cancel`'s `taskId` names no task; a finished or idle one has no work to stop |+| `TOO_MANY_TASKS` | A call would start a task, or make an idle one work, while 32 tasks are working |-A `final_output` call made while tasks are working returns an error that names them.+An `eve__reply` call made while tasks are working returns an error that names them.## Turns wait for their tasksNo turn ends while any task is working. When the model ends a turn early, eve waits as-`task_wait` does: it parks until one of the tasks settles or the turn's own caller steers it,+`eve__task_wait` does: it parks until one of the tasks settles or the turn's own caller steers it,appends the results, and calls the model again in the same turn. No result ever starts a turn onits own. An idle resumable task isn't working, so it doesn't hold the turn.-A held turn stays open. Each time it parks, when eve holds it or when `task_wait` has no result+A held turn stays open. Each time it parks, when eve holds it or when `eve__task_wait` has no resultyet, the stream emits `turn.waiting` with the turn's `turnId`. The next `step.started` for thesame `turnId` means the turn resumed, and `session.waiting` comes only after the turn ends. Themodel's text before the wait depends on the session:@@ -240,14 +240,14 @@ ordinary reply, then post the reply after the results as another message. Slackturn's tasks in a live [task card](/docs/channels/slack#task-card).A steering message from the turn's own caller, one sent with `turnPolicy: "steer"`, the default,-ends a `task_wait` and aborts the `abortSignal` of any `execute` call the turn waits on, but it+ends an `eve__task_wait` and aborts the `abortSignal` of any `execute` call the turn waits on, but itnever interrupts a task. The model reads the message and decides whether to keep each task,-correct an agent by calling it again with its `taskId`, or stop a task with `task_cancel`. A+correct an agent by calling it again with its `taskId`, or stop a task with `eve__task_cancel`. A`"queue"` message, or a message from another caller, waits for the turn to end.## Cancel a task-- **`task_cancel`** stops one task's current work.+- **`eve__task_cancel`** stops one task's current work.- **`session.cancel()`** cancels the turn, the `execute` calls it waits on, and every working taskin the session.- **A failed turn** cancels every working task.@@ -263,12 +263,12 @@ same `taskId` continues where it left off. A `serve` body that doesn't return to## Stream events-| Event | When | Data |-| --------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |-| `task.started` | A call starts a task, or reaches a resumable task by its `taskId` | `taskId`, `callId`, `turnId`, the tool `name`, and `kind` |-| `task.settled` | A reply, return, failure, or cancel settles one call | `taskId`, `callId`, `turnId`, `name`, `kind`, `status`, and `output`, `error`, or `cancel` |-| `agent.started` | A workflow run, including an agent tool's, opens a session with an agent | `callId`, `turnId`, `taskId`, `name`, `sessionId`, `streamPath` |-| `turn.waiting` | An open turn parks on its tasks, a `task_wait`, a question, a sign-in, or an approval | `turnId`, `sequence`, and `on`: `"input"` when a person must act, otherwise `"tasks"` |+| Event | When | Data |+| --------------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |+| `task.started` | A call starts a task, or reaches a resumable task by its `taskId` | `taskId`, `callId`, `turnId`, the tool `name`, and `kind` |+| `task.settled` | A reply, return, failure, or cancel settles one call | `taskId`, `callId`, `turnId`, `name`, `kind`, `status`, and `output`, `error`, or `cancel` |+| `agent.started` | A workflow run, including an agent tool's, opens a session with an agent | `callId`, `turnId`, `taskId`, `name`, `sessionId`, `streamPath` |+| `turn.waiting` | An open turn parks on its tasks, an `eve__task_wait`, a question, a sign-in, or an approval | `turnId`, `sequence`, and `on`: `"input"` when a person must act, otherwise `"tasks"` |`task.started` and `task.settled` come once each per call, and `(taskId, callId)` identifies thecall. Both carry that call's `turnId`, which for a resumable task's later call can be a later turn@@ -276,7 +276,7 @@ than the one that started the task, and the task's tool `name` and `kind`. `kindan agent tool's call and `"tool"` otherwise. Events recorded by earlier eve versions omit `name`and `kind` on `task.settled`; match those to their `task.started` by `callId`. `status` is `"completed"`, `"failed"`, or `"cancelled"`. A completed call carries`output`, a failed call carries `error`, and a cancelled call carries `cancel.reason`:-`"task_cancel"` when the model called `task_cancel`, `"turn_cancelled"` when someone cancelled+`"task_cancel"` when the model called `eve__task_cancel`, `"turn_cancelled"` when someone cancelledthe turn, or the working tasks between turns, or `"turn_ended"` when the turn ended, such as byfailing, while the task still worked. `cancel` is absent when the task's run stopped on its own, and on events recorded byearlier eve versions.@@ -284,9 +284,9 @@ earlier eve versions.when an `execute` call opened it. Results reach the model as a message in its history, not as astream event, so read outcomes from `task.settled`. An `input.requested`,`authorization.required`, or `authorization.completed` event from a task's run carries its-`taskId`. Hooks subscribe to the same events. The stream also carries the model's `task_wait` and-`task_cancel` calls as ordinary `actions.requested` tool calls, so evals can assert on them. The-`eve dev` terminal UI and Slack typing indicators leave those calls out; the terminal UI shows each+`taskId`. Hooks subscribe to the same events. The stream also carries the model's `eve__task_wait` and+`eve__task_cancel` calls as ordinary `actions.requested` tool calls, so evals can assert on them. The+`eve dev` terminal UI, channel activity such as Slack's status line, and ACP clients leave those calls out; the terminal UI shows eachtask's start and end instead. See[Sessions, runs, and streaming](/docs/concepts/sessions-runs-and-streaming#task-events) and[Follow a subagent](/docs/guides/client/streaming#follow-a-subagent).@@ -294,12 +294,12 @@ task's start and end instead. See## Ownership and limits- **Ownership.** Tasks belong to the session's open turn, not to the caller that started them. A- session has at most one open turn, so `task_wait`, `task_cancel`, task results, and the `[Tasks]`+ session has at most one open turn, so `eve__task_wait`, `eve__task_cancel`, task results, and the `[Tasks]`note cover every task in the session. Any later turn can continue an idle resumable task by its`taskId`, and that call runs with its own caller's auth. Only the turn's own caller steers it;anonymous callers share one identity.eve doesn't check which caller started a task: in a session with several people, such as a- shared Slack thread, the model working for one person can continue or `task_cancel` a task+ shared Slack thread, the model working for one person can continue or `eve__task_cancel` a taskanother person started. A continued `serve` task keeps the state its body built for earliercalls, so [key per-caller data on the principal](/docs/tools/workflows#resumable-tasks-serve)and enforce per-person access inside the tool.diff --git a/docs/tools/workflows.mdx b/docs/tools/workflows.mdxindex 3a1502acb..eccfd22b7 100644--- a/docs/tools/workflows.mdx+++ b/docs/tools/workflows.mdx@@ -273,8 +273,8 @@ Each call is a task with its own id, `<tool>-<6 characters>`:`task.result` message, which eve appends at the next step boundary. `toModelOutput` projects theresult, not the receipt.-The model waits for results with `task_wait` and stops a task with `task_cancel`, and a turn can't-end while tasks are working. `task_cancel`, `session.cancel()`, a failed turn, and the+The model waits for results with `eve__task_wait` and stops a task with `eve__task_cancel`, and a turn can't+end while tasks are working. `eve__task_cancel`, `session.cancel()`, a failed turn, and theend of the session abort the call's `abortSignal`; the run waits up to 30 seconds for the body tofinish unwinding, then ends as cancelled. See [Tasks](/docs/tools/tasks) for what the model sees,how held turns look on the stream, steering, events, and limits.@@ -370,9 +370,9 @@ described in [Tasks](/docs/tools/tasks). What differs:- **One `abortSignal` per stretch of work:** a stretch starts when a call reaches an idle task andends when a reply or a cancel settles its calls. Calls that arrive during a stretch share itssignal, and the next stretch gets a new one, so a task can be cancelled any number of times.-- **Cancellation:** `task_cancel`, `session.cancel()`, and a failed turn settle the task's waiting+- **Cancellation:** `eve__task_cancel`, `session.cancel()`, and a failed turn settle the task's waitingcalls as `cancelled` and abort the stretch's `abortSignal`. The run keeps waiting for more calls,- so the task becomes idle and stays available. `task_cancel` on an idle task says the task had no+ so the task becomes idle and stays available. `eve__task_cancel` on an idle task says the task had nowork to stop. To stay available, a body must catch its stretch's abort andreturn to `receive()`, as the example does; an abort that escapes the body finishes the task. Abody that ignores the signal keeps working and its reply is dropped, but a body that doesn't@@ -408,7 +408,7 @@ async function readRepository(ctx: WorkflowStepToolContext, repository: string)}```-Connection tools called through `connection_execute`, ordinary tools, and workflow steps use the same+Connection tools, ordinary tools, and workflow steps use the sameauthorization machinery for requester identity, token caching, callback completion, and rejectionafter sign-in. The execution runtime owns the wait: an agent returns to its model after authorization,while a workflow retries the interrupted step and continues the authored body.@@ -709,7 +709,7 @@ Use [`ctx.ask`](#ask-a-human-ctxask) when the workflow needs a human answer. See## Cancel and clean up: `ctx.abortSignal``ctx.abortSignal` aborts when the call is cancelled: by `session.cancel()`, or, for a task, by-`task_cancel`, a failed turn, or the end of the session. A steering message also aborts it for an+`eve__task_cancel`, a failed turn, or the end of the session. A steering message also aborts it for an`execute` call; see [Stop early for a new message](#stop-early-for-a-new-message). It is durable:it survives replay, and a step that receives it observes the abort. Pass it into the steps thatshould stop, and clean up in `try/finally`:diff --git a/docs/tutorial/team-playbooks.mdx b/docs/tutorial/team-playbooks.mdxindex 366eec95d..90e535694 100644--- a/docs/tutorial/team-playbooks.mdx+++ b/docs/tutorial/team-playbooks.mdx@@ -5,7 +5,7 @@ description: "Part 6 of the Build an Agent tutorial. Load the caller's team playThe glossary from [Step 5](./remember-definitions) is per-session. But your teams have standing analysis conventions for the analytics assistant (Growth groups revenue by date, Finance reconciles customer totals), and those shouldn't bleed across tenants. Load the right team's playbook for whoever is asking.-A skill is an on-demand procedure. The model pulls it in with `load_skill` only when a turn needs it. Make it dynamic and the skill gets decided at runtime instead of baked in. A `defineDynamic` resolver reads the session and returns a `defineSkill` (or nothing). Here you key that decision on the caller's identity in `ctx.session.auth`.+A skill is an on-demand procedure. The model pulls it in with `eve__skill({ name })` only when a turn needs it. Make it dynamic and the skill gets decided at runtime instead of baked in. A `defineDynamic` resolver reads the session and returns a `defineSkill` (or nothing). Here you key that decision on the caller's identity in `ctx.session.auth`.## A playbook per principal@@ -75,7 +75,7 @@ export default eveChannel({});```-Restart with `npm run dev` and ask "Summarize May 2026 revenue using our team's playbook." The model calls `load_skill`, queries the sample data, and applies Growth's daily and plan groupings. Switch `team` to `"finance"`, restart, and start a new session before asking again. The resolver runs at `session.started`, so changing the team does not replace the playbook in an existing session. The same question in the new session uses Finance's reconciliation rules.+Restart with `npm run dev` and ask "Summarize May 2026 revenue using our team's playbook." The model loads the skill with `eve__skill`, queries the sample data, and applies Growth's daily and plan groupings. Switch `team` to `"finance"`, restart, and start a new session before asking again. The resolver runs at `session.started`, so changing the team does not replace the playbook in an existing session. The same question in the new session uses Finance's reconciliation rules.Because the team comes from authenticated claims, not from the message, one tenant can't borrow another's playbook through the message content.diff --git a/e2e/README.md b/e2e/README.mdindex e1309d643..474ac4649 100644--- a/e2e/README.md+++ b/e2e/README.md@@ -159,6 +159,21 @@ message, and ordering a side effect after the result it depends on. Every evalin it is tagged `real-model`. Deterministic task coverage lives with thesurfaces it exercises, in `agent-workflow-tools` and `agent-subagents`.+`agent-deferred-tools` owns the catalog: about 40 deferred tools (20 of them+under the `ledger` namespace), agents, skills, and a connection, reached only+through `search` and `execute`. Its mock-model evals cover dispatch through+`execute`; its `real-model` evals gate prompt-cache reuse after discovery and+that web questions go to `web_search`.++`agent-tool-discovery` is the real-model gate for how models use the catalog+tools: a catalog of 70 deferred tools (60 in the `sre`, `d0`, `index`, and+`support` namespaces), a deferred skill, and a deferred agent, with no+connections, beside the default `web_search` and `bash`. Its evals check that+models find deferred tools with `eve__search` and run them with `eve__tool`,+send web and shell requests to `web_search` and `bash` instead, and leave the+catalog alone for questions they can answer themselves. Every eval in it is+tagged `real-model`.+E2E fixtures live under `e2e/fixtures/*`. Fixture discovery also accepts`apps/fixtures/*` apps with an `evals/` directory, but shared development appsshould stay out of the e2e matrix unless they intentionally own evals.diff --git a/e2e/fixtures/agent-basic-runtime/agent/agent.ts b/e2e/fixtures/agent-basic-runtime/agent/agent.tsindex ebe1b3cf2..35983af8c 100644--- a/e2e/fixtures/agent-basic-runtime/agent/agent.ts+++ b/e2e/fixtures/agent-basic-runtime/agent/agent.ts@@ -3,7 +3,7 @@ import { defineAgent, defineDynamic } from "eve";import { mockModel } from "eve/evals";const DISABLED_AGENT_TOOL_REQUEST = "E2E_DISABLED_ROOT_AGENT_TOOL";-const CHILD_REQUEST = 'Call final_output exactly once with {"answer":"client-recursion-ok"}.';+const CHILD_REQUEST = 'Call eve__reply exactly once with {"answer":"client-recursion-ok"}.';const disabledAgentToolModel = mockModel({modelId: "disabled-root-agent-tool",respond: ({ tools }) => {@@ -16,7 +16,7 @@ const disabledAgentToolModel = mockModel({const childModel = mockModel({modelId: "recursive-client-result-child",respond: () => ({- toolCalls: [{ name: "final_output", input: { answer: "client-recursion-ok" } }],+ toolCalls: [{ name: "eve__reply", input: { answer: "client-recursion-ok" } }],}),});@@ -42,7 +42,7 @@ const config = e2eAgentConfig({}if (lastUserMessage === CHILD_REQUEST) {return {- toolCalls: [{ name: "final_output", input: { answer: "client-recursion-ok" } }],+ toolCalls: [{ name: "eve__reply", input: { answer: "client-recursion-ok" } }],};}return `Mock reply: ${lastUserMessage ?? ""}`;diff --git a/e2e/fixtures/agent-basic-runtime/agent/tools/call_child.ts b/e2e/fixtures/agent-basic-runtime/agent/tools/call_child.tsindex 533bb45d0..2f4d22bf2 100644--- a/e2e/fixtures/agent-basic-runtime/agent/tools/call_child.ts+++ b/e2e/fixtures/agent-basic-runtime/agent/tools/call_child.ts@@ -34,7 +34,7 @@ export default defineDynamic({},});const { response } = await client.sessions.create({- message: 'Call final_output exactly once with {"answer":"client-recursion-ok"}.',+ message: 'Call eve__reply exactly once with {"answer":"client-recursion-ok"}.',outputSchema: {type: "object",properties: { answer: { type: "string", const: "client-recursion-ok" } },diff --git a/e2e/fixtures/agent-cancellation/agent/agent.ts b/e2e/fixtures/agent-cancellation/agent/agent.tsindex 803e42caf..c6b35ee32 100644--- a/e2e/fixtures/agent-cancellation/agent/agent.ts+++ b/e2e/fixtures/agent-cancellation/agent/agent.ts@@ -56,13 +56,13 @@ async function respond(request: MockModelRequest): Promise<MockModelResponse | sif (message.includes("call the sleeper subagent")) {const hitl = message.includes(HITL_REQUEST);// The workflow tool runs as a task: its call returns a receipt, and the- // program's result arrives in a <task_result> message after task_wait.+ // program's result arrives in a <task_result> message after eve__task_wait.const taskResult = request.messages.find((entry) => entry.role === "user" && entry.text.startsWith("<task_result"),);if (taskResult !== undefined) return taskResult.text;if (request.toolResults.some((entry) => entry.name === "workflow")) {- return { toolCalls: [{ id: "wait-for-sleeper", input: {}, name: "task_wait" }] };+ return { toolCalls: [{ id: "wait-for-sleeper", input: {}, name: "eve__task_wait" }] };}return {toolCalls: [@@ -120,11 +120,11 @@ function cancelAndContinueSleeper(request: MockModelRequest): MockModelResponse}const taskId = /Started task (\S+)\./u.exec(String(started.output))?.[1];if (taskId === undefined) throw new Error("The sleeper call returned no task receipt.");- if (calls("task_wait").length === 0) {- return { toolCalls: [{ input: { timeoutSeconds: 2 }, name: "task_wait" }] };+ if (calls("eve__task_wait").length === 0) {+ return { toolCalls: [{ input: { timeoutSeconds: 2 }, name: "eve__task_wait" }] };}- if (calls("task_cancel").length === 0) {- return { toolCalls: [{ input: { taskId }, name: "task_cancel" }] };+ if (calls("eve__task_cancel").length === 0) {+ return { toolCalls: [{ input: { taskId }, name: "eve__task_cancel" }] };}if (continued === undefined) {return { toolCalls: [{ input: { message: SLEEPER_FOLLOW_UP, taskId }, name: "sleeper" }] };@@ -132,7 +132,7 @@ function cancelAndContinueSleeper(request: MockModelRequest): MockModelResponseconst result = [...request.messages].reverse().find((entry) => entry.role === "user" && entry.text.startsWith("<task_result"));- return result?.text ?? { toolCalls: [{ input: {}, name: "task_wait" }] };+ return result?.text ?? { toolCalls: [{ input: {}, name: "eve__task_wait" }] };}const base = e2eAgentConfig({ mock: respond });diff --git a/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-agent-task.eval.ts b/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-agent-task.eval.tsindex 707f8a6e4..57344d4f8 100644--- a/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-agent-task.eval.ts+++ b/e2e/fixtures/agent-cancellation/evals/cancellation/cancel-agent-task.eval.ts@@ -10,7 +10,7 @@ export default defineEval({["Alice is checking that a paused helper keeps its notes. AGENT-TASK-CANCEL",'Ask the sleeper agent with the message "Please wait for cancellation."',- "Once it is waiting, stop that sleeper task with task_cancel.",+ "Once it is waiting, stop that sleeper task with eve__task_cancel.",'Then continue the same sleeper task with its taskId and the message "SLEEPER-FOLLOW-UP", and report its reply.',].join("\n"),);diff --git a/e2e/fixtures/agent-compaction-regressions/agent/agent.ts b/e2e/fixtures/agent-compaction-regressions/agent/agent.tsindex b35dc6d3b..9ce1fc68a 100644--- a/e2e/fixtures/agent-compaction-regressions/agent/agent.ts+++ b/e2e/fixtures/agent-compaction-regressions/agent/agent.ts@@ -16,10 +16,16 @@ import { assistantHasReport } from "../report-evidence";import { HANDOFF_REFERENCE, REVIEW_REFERENCE } from "../release-reports";const TEST_CONTEXT_WINDOW_TOKENS = 32_000;-// The compiled fixture's instructions, task system block, and 11 advertised-// tools occupy ~2,163 tokens, including `read_file`'s image guidance. Reserve-// them in addition to the summarizer's history budget.-const TEST_REQUEST_ENVELOPE_TOKENS = 2_163;+// eve estimates this fixture's request envelope (instructions, task system+// block, and 10 advertised tools; with nothing deferred and no skills or+// connections, the agent has no catalog tools) at 2,273+// tokens. This allowance sits 230.25 tokens below that estimate, the offset the+// cases have always run with, so the threshold leaves them a ~670-token history+// budget. When eve's fixed instructions or tools change, re-measure: log what+// `estimateRequestEnvelope` (packages/eve/src/harness/request-envelope.ts)+// returns while `EVE_E2E_MODEL=mock pnpm exec eve eval --strict+// content-output-file-stub` runs in this fixture, subtract 230.25, and round up.+const TEST_REQUEST_ENVELOPE_TOKENS = 2_043;// Fit the capped file-output exchange, while forcing the larger review and// handoff reports into the assistant checkpoint consumed by the script.const TEST_HISTORY_BUDGET_TOKENS = 900;diff --git a/e2e/fixtures/agent-deferred-tools/.gitignore b/e2e/fixtures/agent-deferred-tools/.gitignorenew file mode 100644index 000000000..eb095f0f8--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/.gitignore@@ -0,0 +1,10 @@+node_modules+.env*+.eve+.vercel+.next+.output+.nitro+dist+.DS_Store+*.tsbuildinfodiff --git a/e2e/fixtures/agent-deferred-tools/.vercelignore b/e2e/fixtures/agent-deferred-tools/.vercelignorenew file mode 100644index 000000000..357664320--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/.vercelignore@@ -0,0 +1,6 @@+node_modules+.eve+.next+.output+.nitro+distdiff --git a/e2e/fixtures/agent-deferred-tools/agent/agent.ts b/e2e/fixtures/agent-deferred-tools/agent/agent.tsnew file mode 100644index 000000000..b6522355c--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/agent.ts@@ -0,0 +1,130 @@+import { e2eAgentConfig } from "@eve-e2e/config";+import { CALL_TOOL, SEARCH_TOOL, SKILL_TOOL, TASK_WAIT_TOOL } from "@eve-e2e/config/catalog-tools";+import {+ callTool,+ latestTaskResult,+ loadSkills,+ outputOf,+ playScript,+} from "@eve-e2e/config/mock-script";+import { defineAgent } from "eve";+import type { MockModelRequest, MockModelResponse } from "eve/evals";++import { LEDGER_REGIONS } from "./lib/ledger-regions";++/** Every catalog listing says this, whatever kinds it names. */+const LISTING_MARKER = `look for one with ${SEARCH_TOOL}, which searches your own catalog`;++/** Entries that must reach the model only through `eve__search` and `eve__tool`. */+const DEFERRED_SAMPLES = [+ "refund_invoice",+ "deploy_service",+ "research_report",+ "billing_specialist",+ `ledger__${LEDGER_REGIONS[0]}`,+];++/** The first tool `eve__tool` suggested in its error for the call with `id`. */+function suggestedTool(request: MockModelRequest, id: string): string | undefined {+ return /Closest tools: ([A-Za-z0-9_-]+)/u.exec(outputOf(request, id))?.[1];+}++/** Waits for a task's result after its receipt, then reports it. */+function reportTask(+ request: MockModelRequest,+ tool: string,+ label: string,+): MockModelResponse | string {+ const result = latestTaskResult(request, tool);+ return result === undefined+ ? { toolCalls: [{ name: TASK_WAIT_TOOL, input: {} }] }+ : `${label} ${result}`;+}++const SCENARIOS: Record<string, (request: MockModelRequest) => MockModelResponse | string> = {+ /** Reports what the model was told: deferred names in its tool list, and the listing. */+ "DEFERRED-CATALOG": (request) => {+ const listed = DEFERRED_SAMPLES.filter((name) =>+ request.tools.some((tool) => tool.name === name),+ );+ const listing = request.messages.find(+ (message) => message.role === "user" && message.text.includes(LISTING_MARKER),+ );+ return [+ `DEFERRED-IN-TOOLS: ${listed.length === 0 ? "none" : listed.join(", ")}`,+ `CATALOG-TOOLS: ${[SEARCH_TOOL, CALL_TOOL, SKILL_TOOL].filter((name) => request.tools.some((tool) => tool.name === name)).join(", ")}`,+ listing?.text ?? "NO-LISTING",+ ].join("\n");+ },+ "DEFERRED-DEPLOY": (request) =>+ playScript(+ request,+ [callTool("deploy", "deploy_service", { service: "billing-api" })],+ (done) => `DEPLOY-RESULT ${outputOf(done, "deploy")}`,+ ),+ "DEFERRED-RESEARCH": (request) =>+ playScript(request, [callTool("research", "research_report", { topic: "refunds" })], (done) =>+ reportTask(done, "research_report", "RESEARCH-RESULT"),+ ),+ "DEFERRED-SPECIALIST": (request) =>+ playScript(+ request,+ [callTool("specialist", "billing_specialist", { message: "Review Bob's dispute DSP-17." })],+ (done) => reportTask(done, "billing_specialist", "SPECIALIST-RESULT"),+ ),+ "DEFERRED-SKILLS": (request) =>+ loadSkills(request, ["pdf-forms", "release_notes", "tenant-playbook"]),+ /** Calls a misspelled tool, then the first name the error suggests. */+ "DEFERRED-MISSPELLED": (request) =>+ playScript(+ request,+ [+ callTool("misspelled", "refund_invoce", { invoiceId: "INV-2041" }),+ {+ id: "corrected",+ input: (current) => ({+ name: suggestedTool(current, "misspelled"),+ input: { invoiceId: "INV-2041" },+ }),+ name: CALL_TOOL,+ },+ ],+ (done) => `REFUND-RESULT ${outputOf(done, "corrected")}`,+ ),+ /** Finds a dynamic ledger tool with eve__search and calls it, then calls the connection. */+ "DEFERRED-LEDGER": (request) =>+ playScript(+ request,+ [+ { id: "ledger-search", input: () => ({ query: "west ledger" }), name: SEARCH_TOOL },+ {+ id: "ledger",+ input: (current) => ({+ name: /"tool":"(ledger__[a-z_]+)"/u.exec(outputOf(current, "ledger-search"))?.[1],+ input: { month: "2026-09" },+ }),+ name: CALL_TOOL,+ },+ callTool("inventory", "petstore__getInventory"),+ ],+ (done) =>+ `LEDGER-RESULT ${outputOf(done, "ledger")} INVENTORY-RESULT ${outputOf(done, "inventory")}`,+ ),+};++function respond(request: MockModelRequest): MockModelResponse | string {+ const directive = (request.lastUserMessage ?? "").trim().split(/\s+/u)[0] ?? "";+ return SCENARIOS[directive]?.(request) ?? `Mock reply: ${request.lastUserMessage ?? ""}`;+}++const config = e2eAgentConfig({ mock: respond });++export default defineAgent({+ ...config,+ // Measure Anthropic cache reuse through its native provider (see cache-after-discovery.eval.ts).+ ...(typeof config.model === "string" && config.model.startsWith("anthropic/")+ ? { modelOptions: { providerOptions: { gateway: { only: ["anthropic"] } } } }+ : {}),+ reasoning: "high",+ limits: { maxInputTokensPerSession: 300_000 },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/channels/petstore.ts b/e2e/fixtures/agent-deferred-tools/agent/channels/petstore.tsnew file mode 100644index 000000000..28ed18f0c--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/channels/petstore.ts@@ -0,0 +1,13 @@+import { defineChannel, GET } from "eve/channels";++import { PETSTORE_SPEC } from "../../petstore";++// A fixture-owned OpenAPI service: eve fetches the spec and calls the operation over HTTP.+export default defineChannel({+ routes: [+ GET("/fixture-petstore/swagger", async () => Response.json(PETSTORE_SPEC)),+ GET("/fixture-petstore/store/inventory", async () =>+ Response.json({ available: 7, pending: 2, sold: 3 }),+ ),+ ],+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/connections/petstore.ts b/e2e/fixtures/agent-deferred-tools/agent/connections/petstore.tsnew file mode 100644index 000000000..9c720d3f5--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/connections/petstore.ts@@ -0,0 +1,17 @@+import { defineDynamic, defineOpenAPIConnection } from "eve/connections";++import { petstoreBaseUrl, petstoreHeaders, petstoreSpecUrl } from "../../petstore";++// Resolved per session because the fixture's own URL is known only at runtime.+export default defineDynamic({+ events: {+ "session.started": () => ({+ petstore: defineOpenAPIConnection({+ baseUrl: petstoreBaseUrl(),+ spec: petstoreSpecUrl(),+ headers: petstoreHeaders(),+ description: "Pet store inventory API.",+ }),+ }),+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/instructions.md b/e2e/fixtures/agent-deferred-tools/agent/instructions.mdnew file mode 100644index 000000000..f65419e61--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/instructions.md@@ -0,0 +1,3 @@+# Identity++You help Alice and Bob run a small company's billing desk. Use the desk's own tools for billing work and the web for public information.diff --git a/e2e/fixtures/agent-deferred-tools/agent/lib/ledger-regions.ts b/e2e/fixtures/agent-deferred-tools/agent/lib/ledger-regions.tsnew file mode 100644index 000000000..5ec01b947--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/lib/ledger-regions.ts@@ -0,0 +1,23 @@+/** The regions the session-scoped ledger resolver returns a deferred tool for. */+export const LEDGER_REGIONS = [+ "us_east",+ "us_west",+ "us_central",+ "canada",+ "mexico",+ "brazil",+ "argentina",+ "uk",+ "ireland",+ "france",+ "germany",+ "spain",+ "italy",+ "nordics",+ "india",+ "singapore",+ "japan",+ "korea",+ "australia",+ "new_zealand",+] as const;diff --git a/e2e/fixtures/agent-deferred-tools/agent/skills/pdf-forms/SKILL.md b/e2e/fixtures/agent-deferred-tools/agent/skills/pdf-forms/SKILL.mdnew file mode 100644index 000000000..2d3b24987--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/skills/pdf-forms/SKILL.md@@ -0,0 +1,10 @@+---+description: Fill and validate the billing desk's PDF forms, such as refund authorizations.+deferred: true+---++# PDF forms++Read `references/fields.md` next to this file for the field list before filling a form.++pdf-forms-skill-ok-R7K2diff --git a/e2e/fixtures/agent-deferred-tools/agent/skills/pdf-forms/references/fields.md b/e2e/fixtures/agent-deferred-tools/agent/skills/pdf-forms/references/fields.mdnew file mode 100644index 000000000..6afbb1860--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/skills/pdf-forms/references/fields.md@@ -0,0 +1,5 @@+# Refund authorization fields++- Invoice id+- Refund amount+- Approverdiff --git a/e2e/fixtures/agent-deferred-tools/agent/skills/release_notes.ts b/e2e/fixtures/agent-deferred-tools/agent/skills/release_notes.tsnew file mode 100644index 000000000..a05719052--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/skills/release_notes.ts@@ -0,0 +1,13 @@+import { defineSkill } from "eve/skills";++export default defineSkill({+ description: "Write customer-facing release notes for billing changes.",+ deferred: true,+ markdown: [+ "# Release notes",+ "",+ "Lead with what customers will notice.",+ "",+ "release-notes-skill-ok-L3M8",+ ].join("\n"),+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/skills/tenant.ts b/e2e/fixtures/agent-deferred-tools/agent/skills/tenant.tsnew file mode 100644index 000000000..d9cb9f87a--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/skills/tenant.ts@@ -0,0 +1,20 @@+import { defineDynamic, defineSkill } from "eve/skills";++/** A deferred dynamic skill resolved for each session. */+export default defineDynamic({+ events: {+ "session.started": () => ({+ "tenant-playbook": defineSkill({+ description: "The tenant's escalation playbook for billing disputes.",+ deferred: true,+ markdown: [+ "# Tenant playbook",+ "",+ "Escalate disputes over $500.",+ "",+ "tenant-playbook-ok-T9P4",+ ].join("\n"),+ }),+ }),+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/subagents/billing_specialist/agent.ts b/e2e/fixtures/agent-deferred-tools/agent/subagents/billing_specialist/agent.tsnew file mode 100644index 000000000..255a7fd04--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/subagents/billing_specialist/agent.ts@@ -0,0 +1,14 @@+import { e2eSubagentConfig } from "@eve-e2e/config";+import { defineAgent } from "eve";++/** A deferred subagent: the root reaches it through search and execute. */+export default defineAgent({+ ...e2eSubagentConfig({+ mock: ({ userMessages }) =>+ userMessages.some((message) => message.includes("Review Bob's dispute DSP-17."))+ ? "SPECIALIST-REVIEW: refund DSP-17"+ : "SPECIALIST-REVIEW: no dispute named",+ }),+ description: "Review billing disputes and recommend a resolution.",+ tool: "deferred",+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/subagents/billing_specialist/instructions.md b/e2e/fixtures/agent-deferred-tools/agent/subagents/billing_specialist/instructions.mdnew file mode 100644index 000000000..fa9d31220--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/subagents/billing_specialist/instructions.md@@ -0,0 +1 @@+You are a billing specialist. Review the dispute you are given and recommend a resolution in two sentences.diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/account_overview.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/account_overview.tsnew file mode 100644index 000000000..a8244400f--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/account_overview.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++// The desk's everyday tool stays in the model's tool list; the long tail is deferred.+export default defineTool({+ description: "Show a customer's plan, balance, and open invoices.",+ inputSchema: z.strictObject({ customer: z.string() }),+ async execute({ customer }) {+ return { balance: 120, customer, openInvoices: ["INV-2041"], plan: "team" };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/apply_discount.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/apply_discount.tsnew file mode 100644index 000000000..e786b8f72--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/apply_discount.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Apply a percentage discount to a customer's next invoice.",+ deferred: true,+ inputSchema: z.strictObject({ customer: z.string(), percent: z.number() }),+ async execute({ customer, percent }) {+ return { applied: true, customer, percent };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/close_account.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/close_account.tsnew file mode 100644index 000000000..d0026273e--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/close_account.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Close a customer's billing account after a final invoice.",+ deferred: true,+ inputSchema: z.strictObject({ customer: z.string() }),+ async execute({ customer }) {+ return { closed: true, customer };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/deploy_service.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/deploy_service.tsnew file mode 100644index 000000000..4fb6bba60--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/deploy_service.ts@@ -0,0 +1,16 @@+import { defineWorkflowTool } from "eve/tools";+import { always } from "eve/tools/approval";+import { z } from "zod";++/** A deferred workflow tool that asks for approval; its turn parks until the run returns. */+export default defineWorkflowTool({+ description: "Deploy the billing service after a person approves it.",+ deferred: true,+ approval: always(),+ inputSchema: z.strictObject({ service: z.string() }),+ async execute({ service }, ctx) {+ "use workflow";++ return { deployed: service, tool: ctx.toolName };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/export_ledger.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/export_ledger.tsnew file mode 100644index 000000000..d0f1bdc45--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/export_ledger.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Export the general ledger for a month as CSV.",+ deferred: true,+ inputSchema: z.strictObject({ month: z.string() }),+ async execute({ month }) {+ return { month, rows: 318 };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/ledger.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/ledger.tsnew file mode 100644index 000000000..1e08a434f--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/ledger.ts@@ -0,0 +1,28 @@+import { defineDynamic, defineTool } from "eve/tools";++import { LEDGER_REGIONS } from "../lib/ledger-regions";++/**+ * A session-scoped resolver that returns one deferred ledger tool per region,+ * all under the `ledger` namespace.+ */+export default defineDynamic({+ events: {+ "session.started": () =>+ Object.fromEntries(+ LEDGER_REGIONS.map((region) => [+ `ledger__${region}`,+ defineTool({+ description: `Read the ${region.replaceAll("_", " ")} regional ledger balance for a month.`,+ deferred: true,+ inputSchema: {+ type: "object",+ properties: { month: { type: "string" } },+ required: ["month"],+ },+ execute: (input: { month: string }) => ({ balance: 1000, month: input.month, region }),+ }),+ ]),+ ),+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/list_disputes.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/list_disputes.tsnew file mode 100644index 000000000..22de8be12--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/list_disputes.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "List open card disputes for a customer.",+ deferred: true,+ inputSchema: z.strictObject({ customer: z.string() }),+ async execute({ customer }) {+ return { customer, disputes: ["DSP-17"] };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/lookup_invoice.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/lookup_invoice.tsnew file mode 100644index 000000000..182de821a--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/lookup_invoice.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Look up an invoice's amount, status, and line items.",+ deferred: true,+ inputSchema: z.strictObject({ invoiceId: z.string() }),+ async execute({ invoiceId }) {+ return { amount: 120, invoiceId, status: "paid" };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/refund_invoice.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/refund_invoice.tsnew file mode 100644index 000000000..4b1382e7c--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/refund_invoice.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Refund a paid invoice in full.",+ deferred: true,+ inputSchema: z.strictObject({ invoiceId: z.string() }),+ async execute({ invoiceId }) {+ return { invoiceId, refunded: true };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/research_report.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/research_report.tsnew file mode 100644index 000000000..14cc73f2b--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/research_report.ts@@ -0,0 +1,14 @@+import { defineWorkflowTool } from "eve/tools";+import { z } from "zod";++/** A deferred background workflow tool: each call starts a task and returns its receipt. */+export default defineWorkflowTool({+ description: "Research a billing topic in the background and report the findings.",+ deferred: true,+ inputSchema: z.strictObject({ topic: z.string() }),+ async task({ topic }) {+ "use workflow";++ return `RESEARCH-FINDINGS:${topic}`;+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/resend_receipt.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/resend_receipt.tsnew file mode 100644index 000000000..18571ac23--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/resend_receipt.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Email a payment receipt to the customer again.",+ deferred: true,+ inputSchema: z.strictObject({ invoiceId: z.string() }),+ async execute({ invoiceId }) {+ return { invoiceId, resent: true };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/schedule_payout.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/schedule_payout.tsnew file mode 100644index 000000000..1a6167f89--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/schedule_payout.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Schedule a payout of a customer's credit balance.",+ deferred: true,+ inputSchema: z.strictObject({ customer: z.string() }),+ async execute({ customer }) {+ return { customer, scheduled: true };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/summarize_usage.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/summarize_usage.tsnew file mode 100644index 000000000..ca2ad130f--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/summarize_usage.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Summarize a customer's metered usage for a month.",+ deferred: true,+ inputSchema: z.strictObject({ customer: z.string(), month: z.string() }),+ async execute({ customer, month }) {+ return { customer, month, units: 4200 };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/tag_customer.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/tag_customer.tsnew file mode 100644index 000000000..11dbf8363--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/tag_customer.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Add a support tag to a customer record.",+ deferred: true,+ inputSchema: z.strictObject({ customer: z.string(), tag: z.string() }),+ async execute({ customer, tag }) {+ return { customer, tag, tagged: true };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/update_billing_email.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/update_billing_email.tsnew file mode 100644index 000000000..c179cde9f--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/update_billing_email.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Change the email address that receives a customer's invoices.",+ deferred: true,+ inputSchema: z.strictObject({ customer: z.string(), email: z.string() }),+ async execute({ customer, email }) {+ return { customer, email, updated: true };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/agent/tools/void_invoice.ts b/e2e/fixtures/agent-deferred-tools/agent/tools/void_invoice.tsnew file mode 100644index 000000000..d281cf6ab--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/agent/tools/void_invoice.ts@@ -0,0 +1,11 @@+import { defineTool } from "eve/tools";+import { z } from "zod";++export default defineTool({+ description: "Void an unpaid invoice so it can no longer be paid.",+ deferred: true,+ inputSchema: z.strictObject({ invoiceId: z.string() }),+ async execute({ invoiceId }) {+ return { invoiceId, voided: true };+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/billing-specialist.subagent.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/billing-specialist.subagent.eval.tsnew file mode 100644index 000000000..43165860f--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/billing-specialist.subagent.eval.ts@@ -0,0 +1,27 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++export default defineEval({+ description:+ "A deferred subagent called through eve__tool starts a child session that records the model's call id.",++ async test(t) {+ requireMockModel(t);++ const turn = await t.send("DEFERRED-SPECIALIST Alice asks the specialist about Bob's dispute.");++ turn.expectOk();+ turn.noFailedActions();+ t.calledSubagent("billing_specialist", {+ callId: "specialist",+ count: 1,+ output: "SPECIALIST-REVIEW: refund DSP-17",+ status: "completed",+ });+ turn.event("agent.started", {+ count: 1,+ data: { callId: "specialist", name: "billing_specialist" },+ });+ t.messageIncludes("SPECIALIST-RESULT SPECIALIST-REVIEW: refund DSP-17");+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/cache-after-discovery.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/cache-after-discovery.eval.tsnew file mode 100644index 000000000..ea0a92f2f--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/cache-after-discovery.eval.ts@@ -0,0 +1,31 @@+import { expectCacheReuse, expectHealthyTurn } from "@eve-e2e/config/cache-checks";+import { defineEval } from "eve/evals";++import { ledgerNotes } from "../ledger-notes";++// Discovering and calling deferred entries never adds a definition, so every+// request after the first reads the preceding one from the provider cache:+// an agent tool and a connection tool alike.+export default defineEval({+ tags: ["real-model"],+ description: "Finding and calling a deferred tool and a connection tool keeps the prompt cache.",++ async test(t) {+ const turn = await t.send(+ [+ "Alice is reconciling the billing desk's weekly ledger notes below.",+ "Invoice INV-2041 was paid twice by mistake, so please refund it in full with the desk's tools.",+ "Bob is adding the pet store's current stock to the same report, so please also check the pet store inventory.",+ "Then tell Alice in two sentences that the refund is done and what the inventory shows.",+ "",+ ledgerNotes(),+ ].join("\n"),+ );++ expectHealthyTurn(turn);+ turn.calledTool("refund_invoice", { count: 1, input: { invoiceId: "INV-2041" } });+ turn.calledTool("petstore__getInventory");+ turn.notEvent("compaction.completed");+ expectCacheReuse(t, [turn]);+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/catalog.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/catalog.eval.tsnew file mode 100644index 000000000..2ffd46cfb--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/catalog.eval.ts@@ -0,0 +1,43 @@+import { CALL_TOOL, SEARCH_TOOL, SKILL_TOOL } from "@eve-e2e/config/catalog-tools";+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";+import { satisfies } from "eve/evals/expect";++/** Deferred entries of each kind, none of which the listing may name. */+const DEFERRED = [+ "refund_invoice",+ "deploy_service",+ "research_report",+ "billing_specialist",+ "pdf-forms",+ "release_notes",+ "tenant-playbook",+ "ledger__us_west",+];++export default defineEval({+ description:+ "Deferred entries stay out of the tool list, which keeps eve__search, eve__tool, and eve__skill, and one listing names their kinds, namespaces, and connections, but no entry.",++ async test(t) {+ requireMockModel(t);++ const turn = await t.send("DEFERRED-CATALOG");++ turn.expectOk();+ turn.usedNoTools();+ t.messageIncludes("DEFERRED-IN-TOOLS: none");+ t.messageIncludes(`CATALOG-TOOLS: ${SEARCH_TOOL}, ${CALL_TOOL}, ${SKILL_TOOL}`);+ t.messageIncludes("You have more tools, agents, and skills than are loaded here.");+ // The dynamic ledger tools share one namespace.+ t.messageIncludes(/^Namespaces, .*: ledger$/mu);+ t.messageIncludes("\n- petstore: Pet store inventory API.");+ t.check(+ turn.message ?? "",+ satisfies(+ (message: string) => DEFERRED.every((name) => !message.includes(name)),+ "the listing names no deferred entry",+ ),+ );+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/deferred-skills.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/deferred-skills.eval.tsnew file mode 100644index 000000000..9d8191f7e--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/deferred-skills.eval.ts@@ -0,0 +1,22 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++export default defineEval({+ description:+ "eve__skill({ name }) loads deferred skills from SKILL.md frontmatter, defineSkill, and a dynamic resolver.",++ async test(t) {+ requireMockModel(t);++ const turn = await t.send("DEFERRED-SKILLS Alice asks how to fill a refund form.");++ turn.expectOk();+ turn.noFailedActions();+ t.loadedSkill("pdf-forms", { count: 1, output: /pdf-forms-skill-ok-R7K2/u });+ t.loadedSkill("release_notes", { count: 1, output: /release-notes-skill-ok-L3M8/u });+ t.loadedSkill("tenant-playbook", { count: 1, output: /tenant-playbook-ok-T9P4/u });+ t.messageIncludes(+ "pdf-forms-skill-ok-R7K2 release-notes-skill-ok-L3M8 tenant-playbook-ok-T9P4",+ );+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/deploy-service.approval.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/deploy-service.approval.eval.tsnew file mode 100644index 000000000..ca3e112b3--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/deploy-service.approval.eval.ts@@ -0,0 +1,40 @@+import { CALL_TOOL } from "@eve-e2e/config/catalog-tools";+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++export default defineEval({+ description:+ "A deferred workflow tool called through eve__tool asks for approval under its own name, then parks its turn until the run returns.",++ async test(t) {+ requireMockModel(t);++ const parked = await t.send("DEFERRED-DEPLOY Alice asks to deploy the billing service.");+ const session = parked.session;+ parked.expectOk();+ session.requireInputRequest({ optionIds: ["approve", "cancel"], toolName: "deploy_service" });+ parked.calledTool("deploy_service", { count: 1, status: "pending" });+ // notCalledTool rejects CALL_TOOL because no call is reported under it, so check the raw actions.+ parked.eventsSatisfy(`no action is reported under ${CALL_TOOL}`, (events) =>+ events.every(+ (event) =>+ event.type !== "actions.requested" ||+ event.data.actions.every(+ (action) => !("toolName" in action) || action.toolName !== CALL_TOOL,+ ),+ ),+ );++ const approved = await session.respondAll("approve");++ approved.expectOk();+ approved.noFailedActions();+ // The run sees the entry's name as its tool name, not eve__tool.+ approved.calledTool("deploy_service", {+ count: 1,+ output: { deployed: "billing-api", tool: "deploy_service" },+ status: "completed",+ });+ approved.messageIncludes('DEPLOY-RESULT {"deployed":"billing-api","tool":"deploy_service"}');+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/evals.config.ts b/e2e/fixtures/agent-deferred-tools/evals/evals.config.tsnew file mode 100644index 000000000..822467682--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/evals.config.ts@@ -0,0 +1,4 @@+import { defineEvalConfig } from "eve/evals";++// Bounds each eval, as agent-prompt-cache does for its long real-model turns.+export default defineEvalConfig({ timeoutMs: 240_000 });diff --git a/e2e/fixtures/agent-deferred-tools/evals/ledger-and-connection.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/ledger-and-connection.eval.tsnew file mode 100644index 000000000..719e9fcd7--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/ledger-and-connection.eval.ts@@ -0,0 +1,26 @@+import { SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++export default defineEval({+ description:+ "eve__search finds a deferred tool a session-scoped resolver returned, eve__tool runs it, and eve__tool calls an OpenAPI connection tool.",++ async test(t) {+ requireMockModel(t);++ const turn = await t.send("DEFERRED-LEDGER Alice asks for the west ledger and pet inventory.");++ turn.expectOk();+ turn.noFailedActions();+ t.toolOrder([SEARCH_TOOL, "ledger__us_west", "petstore__getInventory"]);+ turn.calledTool("ledger__us_west", {+ count: 1,+ output: { balance: 1000, month: "2026-09", region: "us_west" },+ });+ turn.calledTool("petstore__getInventory", {+ count: 1,+ output: { body: { available: 7, pending: 2, sold: 3 }, status: 200 },+ });+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/misspelled-name.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/misspelled-name.eval.tsnew file mode 100644index 000000000..e02b7125b--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/misspelled-name.eval.ts@@ -0,0 +1,22 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++export default defineEval({+ description:+ "eve__tool rejects a misspelled tool name with the closest names, and the suggested name runs the tool.",++ async test(t) {+ requireMockModel(t);++ const turn = await t.send("DEFERRED-MISSPELLED Alice asks to refund invoice INV-2041.");++ turn.expectOk();+ // The misspelled call never reached a tool; only the corrected one ran.+ turn.calledTool("refund_invoice", {+ count: 1,+ input: { invoiceId: "INV-2041" },+ output: { invoiceId: "INV-2041", refunded: true },+ });+ t.messageIncludes('REFUND-RESULT {"invoiceId":"INV-2041","refunded":true}');+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/research-report.task.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/research-report.task.eval.tsnew file mode 100644index 000000000..cd49e511b--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/research-report.task.eval.ts@@ -0,0 +1,20 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++export default defineEval({+ description:+ "A deferred background workflow tool called through eve__tool returns a task receipt, and its result arrives as a task result.",++ async test(t) {+ requireMockModel(t);++ const turn = await t.send("DEFERRED-RESEARCH Alice asks for research on refunds.");++ turn.expectOk();+ turn.noFailedActions();+ turn.calledTool("research_report", { count: 1, output: /^Started task research_report-/u });+ turn.event("task.started", { count: 1 });+ turn.event("task.settled", { count: 1, data: { output: "RESEARCH-FINDINGS:refunds" } });+ t.messageIncludes("RESEARCH-RESULT RESEARCH-FINDINGS:refunds");+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/evals/web-search-routing.eval.ts b/e2e/fixtures/agent-deferred-tools/evals/web-search-routing.eval.tsnew file mode 100644index 000000000..7a96ea435--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/evals/web-search-routing.eval.ts@@ -0,0 +1,31 @@+import { defineEval } from "eve/evals";++// `eve__search` finds the agent's own tools, not web pages, so a question about+// the world goes to `web_search` even while the catalog lists many entries.+// Calls through `eve__tool` are reported under their entry's name, so the+// check is that `web_search` is the only tool called.+export default defineEval({+ tags: ["real-model"],+ description: "A web question goes to web_search, not the catalog.",++ async test(t) {+ const turn = await t.send(+ [+ "Bob is writing the team newsletter and wants one line of sports news.",+ "Please look up on the web who won the 2026 NBA Finals and tell him the winning team.",+ ].join(" "),+ );++ turn.expectOk();+ turn.eventsSatisfy("web_search is the only tool called", (events) => {+ const called = events.flatMap((event) =>+ event.type === "actions.requested"+ ? event.data.actions.flatMap((action) =>+ action.kind === "tool-call" ? [action.toolName] : [],+ )+ : [],+ );+ return called.length > 0 && called.every((name) => name === "web_search");+ });+ },+});diff --git a/e2e/fixtures/agent-deferred-tools/ledger-notes.ts b/e2e/fixtures/agent-deferred-tools/ledger-notes.tsnew file mode 100644index 000000000..f5083423e--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/ledger-notes.ts@@ -0,0 +1,28 @@+/**+ * Deterministic weekly ledger notes, long enough that the first request clears+ * every provider's prompt-cache minimum.+ */+export function ledgerNotes(): string {+ const customers = [+ "Acme",+ "Globex",+ "Initech",+ "Umbrella",+ "Hooli",+ "Vandelay",+ "Stark",+ "Wayne",+ ];+ const lines: string[] = [];+ for (let week = 1; week <= 12; week += 1) {+ lines.push(`Week ${week}`);+ for (const [index, customer] of customers.entries()) {+ const invoice = 2000 + week * 10 + index;+ lines.push(+ `- INV-${invoice}: ${customer} paid ${100 + index * 15} dollars for the team plan; ` ++ `receipt sent, no dispute open, ledger balanced against the processor's settlement report.`,+ );+ }+ }+ return lines.join("\n");+}diff --git a/e2e/fixtures/agent-deferred-tools/package.json b/e2e/fixtures/agent-deferred-tools/package.jsonnew file mode 100644index 000000000..13f82f35e--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/package.json@@ -0,0 +1,29 @@+{+ "name": "agent-deferred-tools",+ "version": "0.0.0",+ "private": true,+ "type": "module",+ "scripts": {+ "build": "eve build",+ "dev": "eve dev",+ "start": "eve start",+ "typecheck": "eve build --skip-sandbox-prewarm && tsc",+ "test:e2e": "eve eval --strict"+ },+ "dependencies": {+ "@eve-e2e/config": "workspace:*",+ "@workflow/world-postgres": "catalog:",+ "eve": "workspace:*",+ "zod": "catalog:"+ },+ "devDependencies": {+ "@types/node": "catalog:",+ "typescript": "catalog:"+ },+ "e2e": {+ "modelMatrix": "full",+ "optionalModels": [+ "anthropic-opus"+ ]+ }+}diff --git a/e2e/fixtures/agent-deferred-tools/petstore.ts b/e2e/fixtures/agent-deferred-tools/petstore.tsnew file mode 100644index 000000000..e784f68d7--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/petstore.ts@@ -0,0 +1,42 @@+/** A fixture-owned Swagger 2.0 document, served by `agent/channels/petstore.ts`. */+export const PETSTORE_SPEC = {+ swagger: "2.0",+ info: { title: "Sample Petstore", version: "1.0.0" },+ produces: ["application/json"],+ paths: {+ "/store/inventory": {+ get: {+ operationId: "getInventory",+ summary: "Return inventory counts by pet status.",+ responses: {+ 200: {+ description: "Inventory counts.",+ schema: { type: "object", additionalProperties: { type: "integer" } },+ },+ },+ },+ },+ },+};++export function petstoreBaseUrl(): string {+ const deploymentHost = process.env.VERCEL_URL;+ const host = deploymentHost+ ? `https://${deploymentHost}`+ : (process.env.WORKFLOW_LOCAL_BASE_URL ?? "http://127.0.0.1:3000");+ return `${host}${process.env.EVE_PUBLIC_ROUTE_PREFIX ?? ""}/fixture-petstore`;+}++/** The spec URL, carrying the preview protection bypass because spec fetches send no connection headers. */+export function petstoreSpecUrl(): string {+ const url = new URL(`${petstoreBaseUrl()}/swagger`);+ const bypass = process.env.VERCEL_AUTOMATION_BYPASS_SECRET;+ if (bypass) url.searchParams.set("x-vercel-protection-bypass", bypass);+ return url.href;+}++/** Lets operation calls reach a protected preview deployment of this fixture. */+export function petstoreHeaders(): Record<string, string> {+ const bypass = process.env.VERCEL_AUTOMATION_BYPASS_SECRET;+ return bypass ? { "x-vercel-protection-bypass": bypass } : {};+}diff --git a/e2e/fixtures/agent-deferred-tools/tsconfig.json b/e2e/fixtures/agent-deferred-tools/tsconfig.jsonnew file mode 100644index 000000000..85e3eceb8--- /dev/null+++ b/e2e/fixtures/agent-deferred-tools/tsconfig.json@@ -0,0 +1,17 @@+{+ "compilerOptions": {+ "target": "ES2022",+ "module": "esnext",+ "moduleResolution": "bundler",+ "outDir": "dist",+ "rootDir": ".",+ "strict": true,+ "esModuleInterop": true,+ "skipLibCheck": true,+ "forceConsistentCasingInFileNames": true,+ "declaration": true,+ "noEmit": true,+ "types": ["node"]+ },+ "include": ["agent/**/*.ts", "evals/**/*.ts"]+}diff --git a/e2e/fixtures/agent-mcp/agent/agent.ts b/e2e/fixtures/agent-mcp/agent/agent.tsindex b8be7496a..a456db1c0 100644--- a/e2e/fixtures/agent-mcp/agent/agent.ts+++ b/e2e/fixtures/agent-mcp/agent/agent.ts@@ -1,4 +1,5 @@import { e2eAgentConfig } from "@eve-e2e/config";+import { CALL_TOOL } from "@eve-e2e/config/catalog-tools";import { defineAgent } from "eve";import type { MockModelRequest, MockModelResponse, MockModelToolCall } from "eve/evals";@@ -11,16 +12,16 @@ function directiveCall(message: string): MockModelToolCall | undefined {if (message.includes("MCP_WHOAMI")) {return {id: "mcp-whoami",- input: { connection: LOOPBACK_CONNECTION, input: {}, tool: "whoami" },- name: "connection_execute",+ input: { input: {}, name: `${LOOPBACK_CONNECTION}__whoami` },+ name: CALL_TOOL,};}const notice = PUBLISH_DIRECTIVE.exec(message)?.[1];if (notice !== undefined) {return {id: "mcp-publish",- input: { connection: LOOPBACK_CONNECTION, input: { notice }, tool: "publish_notice" },- name: "connection_execute",+ input: { input: { notice }, name: `${LOOPBACK_CONNECTION}__publish_notice` },+ name: CALL_TOOL,};}return undefined;diff --git a/e2e/fixtures/agent-openapi-swagger/agent/agent.ts b/e2e/fixtures/agent-openapi-swagger/agent/agent.tsindex afdb52544..8239211c1 100644--- a/e2e/fixtures/agent-openapi-swagger/agent/agent.ts+++ b/e2e/fixtures/agent-openapi-swagger/agent/agent.ts@@ -1,107 +1,112 @@import { e2eAgentConfig } from "@eve-e2e/config";+import { CALL_TOOL, SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { callTool, outputOf, playScript, type ScriptedCall } from "@eve-e2e/config/mock-script";import { defineAgent } from "eve";import type { MockModelRequest, MockModelResponse } from "eve/evals";-/** Whether eve announced `name` in its connection listing. */+/** Whether eve announced `name` in its catalog listing. */function announced({ messages }: MockModelRequest, name: string): boolean {return messages.some((message) => message.role === "user" && message.text.includes(`- ${name}: `),);}-const kennelCall = (id: string, tool: string, input: Record<string, unknown>) => ({- id,- input: { connection: "kennel", tool, input },- name: "connection_execute",-});-const BISCUIT_VISIT = {petId: 4217,visit: { kind: "grooming", date: "2026-10-14" },contacts: [{ name: "Alice", phone: "555-0100" }],};-/** Walks every kennel result shape, retrying the misshapen call from its signature. */-function kennelResponse({ toolResults }: MockModelRequest): MockModelResponse | string {- const byId = new Map(toolResults.map((result) => [result.id, result]));- if (!byId.has("kennel-bad-input")) {- return {- toolCalls: [- kennelCall("kennel-find", "find_pet", { name: "Biscuit" }),- kennelCall("kennel-feedings", "list_feedings", { petId: 4217 }),- kennelCall("kennel-photo", "pet_photo", { petId: 4217 }),- kennelCall("kennel-discharge", "discharge_pet", { petId: 4217 }),- kennelCall("kennel-bad-input", "book_visit", {- petId: "4217",- visit: { kind: "bath" },- }),- kennelCall("kennel-unknown", "find_pets", { name: "Biscuit" }),+const KENNEL_CALLS: readonly ScriptedCall[] = [+ callTool("kennel-find", "kennel__find_pet", { name: "Biscuit" }),+ callTool("kennel-feedings", "kennel__list_feedings", { petId: 4217 }),+ callTool("kennel-photo", "kennel__pet_photo", { petId: 4217 }),+ callTool("kennel-discharge", "kennel__discharge_pet", { petId: 4217 }),+ callTool("kennel-bad-input", "kennel__book_visit", { petId: "4217", visit: { kind: "bath" } }),+ callTool("kennel-unknown", "kennel__find_pets"),+ {+ id: "kennel-book",+ // Corrects the misshapen booking from the signature its error returned.+ input: (request) => ({+ input: outputOf(request, "kennel-bad-input").includes("Signature: kennel__book_visit(")+ ? BISCUIT_VISIT+ : {},+ name: "kennel__book_visit",+ }),+ name: CALL_TOOL,+ },+];++/** Calls each kennel tool, then reports the photo's image parts and the suggested name. */+function kennelResponse(request: MockModelRequest): MockModelResponse | string {+ return playScript(request, KENNEL_CALLS, (finished) => {+ const photo = finished.toolResults.find((result) => result.id === "kennel-photo")?.output;+ const imageParts = Array.isArray(photo)+ ? photo.filter((part) => JSON.stringify(part).includes('"image/png"')).length+ : 0;+ const suggested = outputOf(finished, "kennel-unknown").includes(+ "Closest tools: kennel__find_pet",+ );+ return `KENNEL_MCP_DONE photo-image-parts=${imageParts} suggestion=${suggested ? "yes" : "no"}`;+ });+}++function respond(request: MockModelRequest): MockModelResponse | string {+ const message = request.lastUserMessage ?? "";+ if (message.includes("DYNAMIC_MCP_CONNECTION_E2E")) {+ return announced(request, "dynamic-mcp")+ ? "DYNAMIC_MCP_CONNECTION_FOUND"+ : "DYNAMIC_MCP_CONNECTION_MISSING";+ }+ if (message.includes("DYNAMIC_CONNECTION_E2E")) {+ if (!announced(request, "dynamic-catalog")) return "DYNAMIC_CONNECTION_MISSING";+ return playScript(+ request,+ [+ {+ id: "dynamic-search",+ input: () => ({ query: "dynamic-catalog__" }),+ name: SEARCH_TOOL,+ },+ ],+ (finished) =>+ outputOf(finished, "dynamic-search").includes('"tool":"dynamic-catalog__getStatus"')+ ? "DYNAMIC_CONNECTION_FOUND"+ : "DYNAMIC_CONNECTION_MISSING",+ );+ }+ if (message.includes("PETSTORE_EXECUTE_E2E")) {+ return playScript(+ request,+ [+ {+ id: "petstore-search",+ input: () => ({ query: "petstore__" }),+ name: SEARCH_TOOL,+ },+ callTool("petstore-inventory", "petstore__getInventory", {}),],- };+ (finished) =>+ outputOf(finished, "petstore-inventory").includes('"available":7')+ ? "inventory received"+ : "inventory failed",+ );}- if (!byId.has("kennel-book")) {- const error = byId.get("kennel-bad-input");- return error?.isError === true && JSON.stringify(error.output).includes("book_visit(input:")- ? { toolCalls: [kennelCall("kennel-book", "book_visit", BISCUIT_VISIT)] }- : "KENNEL_SIGNATURE_MISSING";+ if (message.includes("PETSTORE_APPROVAL_E2E")) {+ return playScript(+ request,+ [callTool("approval-inventory", "petstore-approval__getInventory")],+ (finished) =>+ outputOf(finished, "approval-inventory").includes('"available":7')+ ? "inventory received"+ : "inventory failed",+ );}- const photo = byId.get("kennel-photo")?.output;- const imageParts = Array.isArray(photo)- ? photo.filter((part) => JSON.stringify(part).includes('"image/png"')).length- : 0;- return `KENNEL_MCP_DONE photo-image-parts=${imageParts}`;+ if (message.includes("KENNEL_MCP_E2E")) return kennelResponse(request);+ return `Mock reply: ${message}`;}-const config = e2eAgentConfig({- mock: (request) => {- const { lastUserMessage, toolResults } = request;- if (lastUserMessage?.includes("KENNEL_MCP_E2E")) return kennelResponse(request);- if (lastUserMessage?.includes("DYNAMIC_MCP_CONNECTION_E2E")) {- return announced(request, "dynamic-mcp")- ? "DYNAMIC_MCP_CONNECTION_FOUND"- : "DYNAMIC_MCP_CONNECTION_MISSING";- }- if (lastUserMessage?.includes("PETSTORE_EXECUTE_E2E")) {- const result = toolResults.find((entry) => entry.name === "connection_execute");- if (result === undefined) {- return {- toolCalls: [- {- id: "petstore-inventory",- input: { connection: "petstore", tool: "getInventory", input: {} },- name: "connection_execute",- },- ],- };- }- return result.isError ? "inventory failed" : "inventory received";- }- if (!lastUserMessage?.includes("DYNAMIC_CONNECTION_E2E")) {- return `Mock reply: ${lastUserMessage ?? ""}`;- }- if (toolResults.some((result) => result.name === "connection_search")) {- return "DYNAMIC_CONNECTION_FOUND";- }- if (announced(request, "dynamic-catalog")) {- return {- toolCalls: [- {- id: "dynamic-connection-search",- input: { connection: "dynamic-catalog", query: "status" },- name: "connection_search",- },- ],- };- }- return "DYNAMIC_CONNECTION_MISSING";- },-});-export default defineAgent({- ...config,- // Measure Anthropic cache reuse through its native provider (see connection-cache.eval.ts).- ...(typeof config.model === "string" && config.model.startsWith("anthropic/")- ? { modelOptions: { providerOptions: { gateway: { only: ["anthropic"] } } } }- : {}),+ ...e2eAgentConfig({ mock: respond }),reasoning: "high",});diff --git a/e2e/fixtures/agent-openapi-swagger/evals/connection-cache.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/connection-cache.eval.tsdeleted file mode 100644index 3bd62f243..000000000--- a/e2e/fixtures/agent-openapi-swagger/evals/connection-cache.eval.ts+++ /dev/null@@ -1,59 +0,0 @@-import assert from "node:assert/strict";-import { defineEval } from "eve/evals";-import { satisfies } from "eve/evals/expect";--export default defineEval({- tags: ["real-model"],- description: "Searching for and calling a connection tool keeps the provider prompt cache.",-- async test(t) {- const turn = await t.send(- [- "Alice is reconciling the pet store's weekly stock notes below against the live inventory.",- "Use the `connection_search` tool to find the inventory operation in the `petstore` connection,",- "then call it with `connection_execute` and an empty input.",- "Reply with the exact words `inventory received` if the result contains inventory counts.",- "",- stockNotes(),- ].join("\n"),- );-- turn.expectOk();- turn.noFailedActions();- t.toolOrder(["connection_search", "connection_execute"]);- t.messageIncludes(/\binventory received\b/iu);-- const steps = turn.events.filter((event) => event.type === "step.completed");- assert(steps.length >= 3, "search, execute, and reply each call the model");- assert(- (steps[0]?.data.usage?.inputTokens ?? 0) >= 4_096,- "the notes are large enough for provider caching",- );- for (let index = 1; index < steps.length; index += 1) {- const previousInput = steps[index - 1]!.data.usage?.inputTokens;- const usage = steps[index]!.data.usage;- assert(previousInput !== undefined, "preceding input token usage is present");- assert(usage?.cacheReadTokens !== undefined, "provider cache-read usage is present");- t.log(`Request ${index + 1}: ${JSON.stringify({ previousInput, ...usage })}`);- // Discovery used to add tool definitions, which invalidated the whole prefix.- t.check(- usage.cacheReadTokens / previousInput,- satisfies(- (ratio: number) => ratio >= 0.9,- `request ${index + 1} reads at least 90% of the preceding input from the provider cache`,- ),- );- }- },-});--const STATUSES = ["available", "pending", "sold"] as const;--/** Deterministic, benign bulk text that keeps the first request above the cache minimum. */-function stockNotes(): string {- return Array.from({ length: 160 }, (_, index) => {- const day = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"][index % 5];- const status = STATUSES[index % STATUSES.length];- return `Note ${index + 1}: On ${day}, Bob recorded shelf ${(index % 12) + 1} with ${(index * 7) % 23} items marked ${status}, and Alice asked to compare that count with the live inventory before the weekly order.`;- }).join("\n");-}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/connection-execute.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/connection-execute.eval.tsdeleted file mode 100644index 5ff3f9d2e..000000000--- a/e2e/fixtures/agent-openapi-swagger/evals/connection-execute.eval.ts+++ /dev/null@@ -1,36 +0,0 @@-import { defineEval } from "eve/evals";--const PETSTORE_INVENTORY_TOOL = "petstore__getInventory";--export default defineEval({- description:- "connection_execute calls a connection tool over HTTP and reports it as a nested action.",-- async test(t) {- if (process.env.EVE_E2E_MODEL !== "mock") {- t.skip("Requires the deterministic mock model to issue the exact call.");- }-- const turn = await t.send("PETSTORE_EXECUTE_E2E");-- turn.expectOk();- turn.noFailedActions();- turn.calledTool("connection_execute", { count: 1, output: hasInventoryCounts });- turn.calledTool(PETSTORE_INVENTORY_TOOL, { count: 1, output: hasInventoryCounts });- turn.event("actions.requested", {- count: 1,- data: {- actions: [{ parentCallId: "petstore-inventory", toolName: PETSTORE_INVENTORY_TOOL }],- },- });- t.succeeded();- t.messageIncludes("inventory received");- },-});--function hasInventoryCounts(value: unknown): boolean {- if (typeof value !== "object" || value === null) return false;- const body = (value as { body?: unknown }).body;- if (typeof body !== "object" || body === null || Array.isArray(body)) return false;- return Object.values(body).some((count) => typeof count === "number");-}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/connection-input-shape.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/connection-input-shape.eval.tsindex 0adfa3f42..e0d8152fb 100644--- a/e2e/fixtures/agent-openapi-swagger/evals/connection-input-shape.eval.ts+++ b/e2e/fixtures/agent-openapi-swagger/evals/connection-input-shape.eval.ts@@ -1,12 +1,15 @@-import { defineEval, type EveEvalTurn } from "eve/evals";+import { SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { defineEval } from "eve/evals";import { satisfies } from "eve/evals/expect";+import { noCallRejectedWholesale } from "./first-try";+const ADD_PET_TOOL = "petstore__addPet";export default defineEval({tags: ["real-model"],description:- "A real model builds a nested connection_execute input from the connection_search signature.",+ "A real model builds a nested connection tool input from the TypeScript signature search returns.",async test(t) {const turn = await t.send(@@ -14,24 +17,21 @@ export default defineEval({"Alice runs the Maple Street pet store, whose store id is `maple-street`.","This morning she took in a golden retriever named Biscuit. He is ready for adoption,","belongs in the Dogs category, and his photo is at https://example.com/photos/biscuit.jpg.",- "Use `connection_search` to find how to add a pet in the `petstore` connection, then add",- "Biscuit with `connection_execute`. Reply with the id the store assigns to him.",+ "Please add Biscuit to the store's catalog through the petstore connection,",+ "then tell Alice the id the store assigns to him.",].join(" "),);turn.expectOk();- t.toolOrder(["connection_search", "connection_execute"]);- // The nested action carries the operation's HTTP result.+ t.toolOrder([SEARCH_TOOL, ADD_PET_TOOL]);t.calledTool(ADD_PET_TOOL, { output: isStoredBiscuit });t.messageIncludes("4217");- // Tracked, not gated: how often the first attempt already had the right shape.+ // Tracked, not gated: no step had every tool call rejected, a proxy for building+ // the nested input right on the first try.t.check(- firstAddPetSucceeded(turn),- satisfies(- (succeeded: boolean) => succeeded,- "the first addPet call has a valid input shape",- ).soft(),+ noCallRejectedWholesale(turn),+ satisfies((firstTry: boolean) => firstTry, "every step that requested tools ran one").soft(),);},});@@ -52,21 +52,3 @@ function isStoredBiscuit(value: unknown): boolean {body.photoUrls.includes("https://example.com/photos/biscuit.jpg"));}--/** Whether the first connection_execute call for addPet completed with a 200 response. */-function firstAddPetSucceeded(turn: EveEvalTurn): boolean {- const first = turn.events- .flatMap((event) => (event.type === "actions.requested" ? event.data.actions : []))- .find(- (action) =>- action.kind === "tool-call" &&- action.toolName === "connection_execute" &&- (action.input as { tool?: unknown }).tool === "addPet",- );- if (first === undefined) return false;- const result = turn.events.find(- (event) => event.type === "action.result" && event.data.result.callId === first.callId,- );- if (result?.type !== "action.result" || result.data.status !== "completed") return false;- return (result.data.result as { output?: { status?: unknown } }).output?.status === 200;-}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/dynamic-connection.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/dynamic-connection.eval.tsindex 802a4ff5f..1e91c4cc8 100644--- a/e2e/fixtures/agent-openapi-swagger/evals/dynamic-connection.eval.ts+++ b/e2e/fixtures/agent-openapi-swagger/evals/dynamic-connection.eval.ts@@ -1,32 +1,25 @@+import { SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { requireMockModel } from "@eve-e2e/config/mock-script";import { defineEval } from "eve/evals";export default defineEval({- description: "A session-scoped dynamic OpenAPI connection is announced and searchable.",+ description:+ "A session-scoped dynamic OpenAPI connection is announced in the catalog listing and searchable by name.",async test(t) {- if (process.env.EVE_E2E_MODEL !== "mock") {- t.skip("Requires the deterministic mock model; the fixture API endpoint is non-routable.");- }+ requireMockModel(+ t,+ "Requires the deterministic mock model; the fixture API endpoint is non-routable.",+ );- await t.send("DYNAMIC_CONNECTION_E2E");+ const turn = await t.send("DYNAMIC_CONNECTION_E2E");- t.succeeded();- t.toolOrder(["connection_search"]);- t.calledTool("connection_search", { count: 1, output: hasDynamicStatusTool });+ turn.expectOk();+ turn.calledTool(SEARCH_TOOL, {+ count: 1,+ input: { query: "dynamic-catalog__" },+ output: (value) => JSON.stringify(value).includes('"tool":"dynamic-catalog__getStatus"'),+ });t.messageIncludes("DYNAMIC_CONNECTION_FOUND");},});--function hasDynamicStatusTool(value: unknown): boolean {- const tools = typeof value === "object" && value !== null ? Reflect.get(value, "tools") : [];- return (- Array.isArray(tools) &&- tools.some(- (entry) =>- typeof entry === "object" &&- entry !== null &&- Reflect.get(entry, "connection") === "dynamic-catalog" &&- Reflect.get(entry, "tool") === "getStatus",- )- );-}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/dynamic-mcp-connection.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/dynamic-mcp-connection.eval.tsindex 407c7f61a..9154dad9a 100644--- a/e2e/fixtures/agent-openapi-swagger/evals/dynamic-mcp-connection.eval.ts+++ b/e2e/fixtures/agent-openapi-swagger/evals/dynamic-mcp-connection.eval.ts@@ -1,12 +1,14 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";import { defineEval } from "eve/evals";export default defineEval({description: "A session-scoped dynamic MCP connection is exposed to the model.",async test(t) {- if (process.env.EVE_E2E_MODEL !== "mock") {- t.skip("Requires the deterministic mock model; the fixture MCP endpoint is non-routable.");- }+ requireMockModel(+ t,+ "Requires the deterministic mock model; the fixture MCP endpoint is non-routable.",+ );await t.send("DYNAMIC_MCP_CONNECTION_E2E");diff --git a/e2e/fixtures/agent-openapi-swagger/evals/first-try.ts b/e2e/fixtures/agent-openapi-swagger/evals/first-try.tsnew file mode 100644index 000000000..aa9260ad6--- /dev/null+++ b/e2e/fixtures/agent-openapi-swagger/evals/first-try.ts@@ -0,0 +1,23 @@+import type { EveEvalTurn } from "eve/evals";++/**+ * Whether every model step that asked for tools ran at least one of them. A+ * call through `eve__tool` whose input fails validation never becomes an action,+ * so a step that requested tools but produced no action had every call+ * rejected and the model had to try again.+ */+export function noCallRejectedWholesale(turn: EveEvalTurn): boolean {+ const stepsWithActions = new Set(+ turn.events.flatMap((event) =>+ event.type === "actions.requested" || event.type === "action.result"+ ? [`${event.data.turnId}:${event.data.stepIndex}`]+ : [],+ ),+ );+ return turn.events.every(+ (event) =>+ event.type !== "step.completed" ||+ event.data.finishReason !== "tool-calls" ||+ stepsWithActions.has(`${event.data.turnId}:${event.data.stepIndex}`),+ );+}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp-results.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp-results.eval.tsindex c691dffd4..15cba6949 100644--- a/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp-results.eval.ts+++ b/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp-results.eval.ts@@ -1,3 +1,4 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";import { defineEval } from "eve/evals";/** Failed tool outputs carry the error message; match its text whatever its wrapping. */@@ -5,23 +6,20 @@ const mentions = (text: string) => (value: unknown) => JSON.stringify(value).incexport default defineEval({description:- "connection_execute returns each MCP result shape, and its errors let the model correct a call.",+ "execute returns each MCP result shape, validates input before a call reaches the server, and suggests the closest tool for an unknown name.",async test(t) {- if (process.env.EVE_E2E_MODEL !== "mock") {- t.skip("Requires the deterministic mock model to issue the exact calls.");- }+ requireMockModel(t);const turn = await t.send("KENNEL_MCP_E2E");turn.expectOk();- t.succeeded();// structuredContent wins over the text copy of the same result.turn.calledTool("kennel__find_pet", {count: 1,output: { petId: 4217, name: "Biscuit", kennel: "B3" },});- // Text-only results without an output schema are parsed as JSON, and the+ // A text result without an output schema is parsed as JSON, and the// omitted `day` reaches the server as its schema default.turn.calledTool("kennel__list_feedings", {count: 1,@@ -32,42 +30,24 @@ export default defineEval({value.every((entry) => (entry as { day?: unknown }).day === "today"),});// Image content stays MCP content, and the model receives it as a file part.- turn.calledTool("kennel__pet_photo", {- count: 1,- output: mentions('"type":"image"'),- });+ turn.calledTool("kennel__pet_photo", { count: 1, output: mentions('"type":"image"') });t.messageIncludes("photo-image-parts=1");-- // isError results fail both the outer call and the nested action.- turn.calledTool("connection_execute", {+ // An isError result fails the call.+ turn.calledTool("kennel__discharge_pet", {count: 1,- status: "failed",- input: { tool: "discharge_pet" },output: mentions("adoption is in progress"),- });- turn.calledTool("kennel__discharge_pet", { count: 1, status: "failed" });-- // Invalid input never reaches the server and returns the tool's signature.- turn.calledTool("connection_execute", {- count: 1,status: "failed",- input: { tool: "book_visit", input: { petId: "4217" } },- output: (value) =>- mentions("Invalid input for")(value) && mentions("book_visit(input:")(value),});- // The corrected call is the only one the server sees.++ // Invalid input and an unknown name fail validation, so neither becomes a+ // connection call. The model reads the signature from the first error and+ // corrects the booking, and the second error suggests the closest tool.+ t.messageIncludes("suggestion=yes");+ // The corrected booking is the only one the server sees.turn.calledTool("kennel__book_visit", {count: 1,output: { visitId: "V-88", petId: 4217, visit: { kind: "grooming" } },});-- // An unknown tool name points at the closest real one.- turn.calledTool("connection_execute", {- count: 1,- status: "failed",- input: { tool: "find_pets" },- output: mentions("Closest tools: find_pet"),- });t.messageIncludes("KENNEL_MCP_DONE");},});diff --git a/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp.eval.tsindex a96219ec1..a403ae232 100644--- a/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp.eval.ts+++ b/e2e/fixtures/agent-openapi-swagger/evals/kennel-mcp.eval.ts@@ -1,24 +1,27 @@-import { defineEval, type EveEvalTurn } from "eve/evals";+import { SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { defineEval } from "eve/evals";import { satisfies } from "eve/evals/expect";+import { noCallRejectedWholesale } from "./first-try";+export default defineEval({tags: ["real-model"],description:- "A real model reads an MCP tool's structured result and builds a nested MCP tool input from its signature.",+ "A real model finds an MCP connection's tools with eve__search, reads a tool's structured result, and builds a nested tool input from its signature.",async test(t) {const turn = await t.send(["Bob works the front desk at the Maple Street kennel. Biscuit, a boarding golden retriever,","needs a grooming visit on October 14, 2026. Alice (555-0100) should be notified about it.",- "Use `connection_search` on the `kennel` connection to find the tools you need, look up",- "Biscuit's pet id, then book the visit with `connection_execute`.",+ "Use `eve__search` to find the `kennel` connection's tools, look up Biscuit's pet id,",+ "then book the visit with `eve__tool`.","Reply with the visit id the kennel returns.",].join(" "),);turn.expectOk();- t.calledTool("connection_search");+ t.toolOrder([SEARCH_TOOL, "kennel__find_pet", "kennel__book_visit"]);// The pet id comes from find_pet's structured result, not the prompt.t.calledTool("kennel__find_pet", { output: { petId: 4217 } });t.calledTool("kennel__book_visit", {@@ -32,29 +35,11 @@ export default defineEval({});t.messageIncludes("V-88");- // Tracked, not gated: whether the model built the nested input on its first try.+ // Tracked, not gated: no step had every tool call rejected, a proxy for building+ // the nested input right on the first try.t.check(- firstBookingSucceeded(turn),- satisfies(- (succeeded: boolean) => succeeded,- "the first book_visit call has a valid input shape",- ).soft(),+ noCallRejectedWholesale(turn),+ satisfies((firstTry: boolean) => firstTry, "every step that requested tools ran one").soft(),);},});--function firstBookingSucceeded(turn: EveEvalTurn): boolean {- const first = turn.events- .flatMap((event) => (event.type === "actions.requested" ? event.data.actions : []))- .find(- (action) =>- action.kind === "tool-call" &&- action.toolName === "connection_execute" &&- (action.input as { tool?: unknown }).tool === "book_visit",- );- if (first === undefined) return false;- const result = turn.events.find(- (event) => event.type === "action.result" && event.data.result.callId === first.callId,- );- return result?.type === "action.result" && result.data.status === "completed";-}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/petstore-approval.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/petstore-approval.eval.tsnew file mode 100644index 000000000..865fb2637--- /dev/null+++ b/e2e/fixtures/agent-openapi-swagger/evals/petstore-approval.eval.ts@@ -0,0 +1,32 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++const INVENTORY_TOOL = "petstore-approval__getInventory";++export default defineEval({+ description:+ "A connection's approval asks about the connection tool by its own name, and the call runs once approved.",++ async test(t) {+ requireMockModel(t);++ const parked = await t.send("PETSTORE_APPROVAL_E2E");+ const session = parked.session;+ parked.expectOk();++ session.requireInputRequest({+ display: "confirmation",+ optionIds: ["approve", "cancel"],+ toolName: INVENTORY_TOOL,+ });+ parked.calledTool(INVENTORY_TOOL, { count: 1, status: "pending" });+ parked.notEvent("action.result", { data: { result: { toolName: INVENTORY_TOOL } } });++ const approved = await session.respondAll("approve");++ approved.expectOk();+ approved.noFailedActions();+ approved.calledTool(INVENTORY_TOOL, { count: 1, status: "completed" });+ t.messageIncludes("inventory received");+ },+});diff --git a/e2e/fixtures/agent-openapi-swagger/evals/petstore-execute.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/petstore-execute.eval.tsnew file mode 100644index 000000000..c97680a06--- /dev/null+++ b/e2e/fixtures/agent-openapi-swagger/evals/petstore-execute.eval.ts@@ -0,0 +1,35 @@+import { CALL_TOOL, SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++const INVENTORY_TOOL = "petstore__getInventory";++export default defineEval({+ description:+ "search lists an OpenAPI connection's operations, and execute calls one over HTTP under its full name.",++ async test(t) {+ requireMockModel(t);++ const turn = await t.send("PETSTORE_EXECUTE_E2E");++ turn.expectOk();+ turn.noFailedActions();+ t.toolOrder([SEARCH_TOOL, INVENTORY_TOOL]);+ turn.calledTool(SEARCH_TOOL, {+ count: 1,+ input: { query: "petstore__" },+ output: (value) => JSON.stringify(value).includes(`"tool":"${INVENTORY_TOOL}"`),+ });+ // The call is reported under the entry's name, never as a nested eve__tool action.+ turn.calledTool(INVENTORY_TOOL, { count: 1, output: hasInventoryCounts });+ turn.notEvent("actions.requested", { data: { actions: [{ toolName: CALL_TOOL }] } });+ t.messageIncludes("inventory received");+ },+});++function hasInventoryCounts(value: unknown): boolean {+ const body = (value as { body?: unknown } | null)?.body;+ if (typeof body !== "object" || body === null || Array.isArray(body)) return false;+ return Object.values(body).some((count) => typeof count === "number");+}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger-approval.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger-approval.eval.tsdeleted file mode 100644index 5c878cabe..000000000--- a/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger-approval.eval.ts+++ /dev/null@@ -1,58 +0,0 @@-import { defineEval } from "eve/evals";--const SEARCH_TOOL = "connection_search";-const PETSTORE_APPROVAL_INVENTORY_TOOL = "petstore-approval__getInventory";--export default defineEval({- tags: ["real-model"],- description:- "An approval-gated operation from a fixture-owned Swagger 2.0 document parks before execution.",-- async test(t) {- const parked = await t.send(- [- "Use the `connection_search` tool with connection `petstore-approval` to find the inventory operation.",- "Then call it with `connection_execute` and an empty input.",- "Wait for approval if requested.",- "After the tool runs, reply with the exact words `inventory received` if the tool result contains inventory counts.",- ].join("\n"),- );- const session = parked.session;- parked.expectOk();-- // The approval belongs to the model's connection_execute call.- session.requireInputRequest({- display: "confirmation",- optionIds: ["approve", "cancel"],- toolName: "connection_execute",- });- parked.calledTool("connection_execute", { status: "pending", count: 1 });- parked.notEvent("action.result", {- data: { result: { toolName: PETSTORE_APPROVAL_INVENTORY_TOOL } },- });-- const approved = await session.respondAll("approve");- approved.expectOk();-- approved.event("action.result", {- data: {- result: { kind: "tool-result", toolName: PETSTORE_APPROVAL_INVENTORY_TOOL },- status: "completed",- },- });-- t.succeeded();- t.calledTool(SEARCH_TOOL);- t.calledTool(PETSTORE_APPROVAL_INVENTORY_TOOL, {- output: hasInventoryCounts,- });- t.messageIncludes(/\binventory received\b/iu);- },-});--function hasInventoryCounts(value: unknown): boolean {- if (typeof value !== "object" || value === null) return false;- const body = (value as { body?: unknown }).body;- if (typeof body !== "object" || body === null || Array.isArray(body)) return false;- return Object.values(body).some((count) => typeof count === "number");-}diff --git a/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger.eval.ts b/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger.eval.tsdeleted file mode 100644index c3e4ad89e..000000000--- a/e2e/fixtures/agent-openapi-swagger/evals/petstore-swagger.eval.ts+++ /dev/null@@ -1,34 +0,0 @@-import { defineEval } from "eve/evals";--const PETSTORE_INVENTORY_TOOL = "petstore__getInventory";--export default defineEval({- tags: ["real-model"],- description:- "A fixture-owned Swagger 2.0 document is searched and its getInventory called over HTTP.",-- async test(t) {- const turn = await t.send(- [- "Use the `connection_search` tool to find the inventory operation in the `petstore` connection.",- "Then call it with `connection_execute` and an empty input.",- "Reply with the exact words `inventory received` if the tool result contains inventory counts.",- ].join("\n"),- );-- turn.calledTool(PETSTORE_INVENTORY_TOOL, { output: hasInventoryCounts });-- t.succeeded();- t.toolOrder(["connection_search", "connection_execute"]);- t.calledTool("connection_search");- t.calledTool(PETSTORE_INVENTORY_TOOL, { output: hasInventoryCounts });- t.messageIncludes(/\binventory received\b/iu);- },-});--function hasInventoryCounts(value: unknown): boolean {- if (typeof value !== "object" || value === null) return false;- const body = (value as { body?: unknown }).body;- if (typeof body !== "object" || body === null || Array.isArray(body)) return false;- return Object.values(body).some((count) => typeof count === "number");-}diff --git a/e2e/fixtures/agent-openapi-swagger/kennel.ts b/e2e/fixtures/agent-openapi-swagger/kennel.tsindex c36219657..59e614160 100644--- a/e2e/fixtures/agent-openapi-swagger/kennel.ts+++ b/e2e/fixtures/agent-openapi-swagger/kennel.ts@@ -1,9 +1,9 @@import { petstoreHeaders } from "./petstore";/**- * A fixture-owned MCP server whose tools cover each shape of MCP result that- * `connection_execute` returns: structured content, JSON text, images, and- * tool errors, plus a nested input the model has to build from a signature.+ * A fixture-owned MCP server whose tools cover each shape of MCP result a+ * connection tool returns: structured content, JSON text, images, and tool+ * errors, plus a nested input the model has to build from a signature.*/export const KENNEL_TOOLS = [{diff --git a/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.ts b/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.tsindex 62423fd3f..e25167a06 100644--- a/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.ts+++ b/e2e/fixtures/agent-prompt-cache/evals/parallel-review.eval.ts@@ -1,7 +1,7 @@import assert from "node:assert/strict";import { defineEval, type EveEvalContext, type EveEvalTurn } from "eve/evals";import { z } from "zod";-import { expectCacheReuse, expectHealthyTurn } from "../cache-checks";+import { expectCacheReuse, expectHealthyTurn } from "@eve-e2e/config/cache-checks";import { EVENT_OVERVIEW } from "../event-overview";import { purchasingSheets } from "../purchasing-sheets";diff --git a/e2e/fixtures/agent-prompt-cache/evals/tool-image-cache.eval.ts b/e2e/fixtures/agent-prompt-cache/evals/tool-image-cache.eval.tsindex 2aec12874..09acbabd6 100644--- a/e2e/fixtures/agent-prompt-cache/evals/tool-image-cache.eval.ts+++ b/e2e/fixtures/agent-prompt-cache/evals/tool-image-cache.eval.ts@@ -1,5 +1,5 @@import { defineEval } from "eve/evals";-import { expectCacheReuse, expectHealthyTurn } from "../cache-checks";+import { expectCacheReuse, expectHealthyTurn } from "@eve-e2e/config/cache-checks";import { EVENT_OVERVIEW } from "../event-overview";import { purchasingSheets } from "../purchasing-sheets";diff --git a/e2e/fixtures/agent-router/agent/agent.ts b/e2e/fixtures/agent-router/agent/agent.tsindex 775472e01..7b26da444 100644--- a/e2e/fixtures/agent-router/agent/agent.ts+++ b/e2e/fixtures/agent-router/agent/agent.ts@@ -22,13 +22,13 @@ export default defineAgent({return "AGENT-ROUTER-ROOT-COPY-OK";}// agentRouter() runs each call as a task: a receipt now, the result in a- // later <task_result> message once task_wait returns.+ // later <task_result> message once eve__task_wait returns.const taskResult = request.messages.find((message) => message.role === "user" && message.text.startsWith("<task_result"),);if (taskResult !== undefined) return taskResult.text;if (request.toolResults.some((entry) => entry.name === "agent")) {- return { toolCalls: [{ name: "task_wait", input: {} }] };+ return { toolCalls: [{ name: "eve__task_wait", input: {} }] };}return {toolCalls: [{ name: "agent", input: { message: "Return the root-copy marker." } }],diff --git a/e2e/fixtures/agent-router/evals/agent-router.eval.ts b/e2e/fixtures/agent-router/evals/agent-router.eval.tsindex 5f040e669..6447ba667 100644--- a/e2e/fixtures/agent-router/evals/agent-router.eval.ts+++ b/e2e/fixtures/agent-router/evals/agent-router.eval.ts@@ -2,14 +2,14 @@ import { defineEval } from "eve/evals";export default defineEval({description:- "agentRouter runs as a task that invokes the root-copy agent, and task_wait delivers its result.",+ "agentRouter runs as a task that invokes the root-copy agent, and eve__task_wait delivers its result.",async test(t) {const turn = await t.send("Route this task.");turn.expectOk();turn.messageIncludes("AGENT-ROUTER-ROOT-COPY-OK");turn.calledTool("agent", { count: 1 });- turn.calledTool("task_wait", { count: 1 });+ turn.calledTool("eve__task_wait", { count: 1 });turn.event("task.started", { count: 1, data: { kind: "tool", name: "agent" } });turn.event("task.settled", {count: 1,diff --git a/e2e/fixtures/agent-session-limits/agent/agent.ts b/e2e/fixtures/agent-session-limits/agent/agent.tsindex 141729ba4..75eef2a76 100644--- a/e2e/fixtures/agent-session-limits/agent/agent.ts+++ b/e2e/fixtures/agent-session-limits/agent/agent.ts@@ -9,6 +9,6 @@ export default defineAgent({limits: {maxInputTokensPerSession: 1,},- // Without the built-in agent tool there are no task tools, so the one-call budget can't go to task_wait.+ // Without the built-in agent tool there are no task tools, so the one-call budget can't go to eve__task_wait.tool: false,});diff --git a/e2e/fixtures/agent-skills/agent/agent.ts b/e2e/fixtures/agent-skills/agent/agent.tsindex 0089ad0ec..5696e5fdf 100644--- a/e2e/fixtures/agent-skills/agent/agent.ts+++ b/e2e/fixtures/agent-skills/agent/agent.ts@@ -1,11 +1,19 @@import { e2eAgentConfig } from "@eve-e2e/config";+import { loadSkills } from "@eve-e2e/config/mock-script";import { defineAgent, defineDynamic } from "eve";import { PREFIX_REQUEST, prefixModel } from "./lib/prompt-prefix";-import type { MockModelRequest } from "eve/evals";+import type { MockModelRequest, MockModelResponse } from "eve/evals";const DYNAMIC_INSTRUCTIONS_TOKEN = "dynamic-instructions-ok-M3K8";+const SKILL_LOAD_DIRECTIVE = "SKILL-LOAD";-function respond(request: MockModelRequest): string {+function respond(request: MockModelRequest): MockModelResponse | string {+ const directive = request.userMessages.find((message) =>+ message.startsWith(SKILL_LOAD_DIRECTIVE),+ );+ if (directive !== undefined) {+ return loadSkills(request, directive.slice(SKILL_LOAD_DIRECTIVE.length).trim().split(/\s+/u));+ }const hasDynamicUserInstruction = request.userMessages.some((message) =>message.includes(DYNAMIC_INSTRUCTIONS_TOKEN),);diff --git a/e2e/fixtures/agent-skills/agent/skills/house-rules-override.ts b/e2e/fixtures/agent-skills/agent/skills/house-rules-override.tsindex a0b155955..47988828d 100644--- a/e2e/fixtures/agent-skills/agent/skills/house-rules-override.ts+++ b/e2e/fixtures/agent-skills/agent/skills/house-rules-override.ts@@ -4,7 +4,7 @@ export const HOUSE_RULES_OVERRIDE_TOKEN = "house-rules-dynamic-ok-M5T8";// Emits a skill named `house-rules`, the same name as the authored skill// (house-rules.md). A dynamic skill overrides a same-named authored one, so-// load_skill returns this body, not the authored one.+// loading it returns this body, not the authored one.export default defineDynamic({events: {"session.started": async () => {diff --git a/e2e/fixtures/agent-skills/evals/skills/auto-invocation.eval.ts b/e2e/fixtures/agent-skills/evals/skills/auto-invocation.eval.tsindex e06a1de11..663b3f952 100644--- a/e2e/fixtures/agent-skills/evals/skills/auto-invocation.eval.ts+++ b/e2e/fixtures/agent-skills/evals/skills/auto-invocation.eval.ts@@ -5,12 +5,12 @@ const ECHO_MARKER_TOKEN = "skill-echo-marker-ok-V8Y2";/*** Skill smoke eval:* a flat markdown skill (skills/echo-marker.md) is advertised, loaded on- * demand through the framework-owned `load_skill` tool, and its body shapes- * the reply: the skill instructs an exact-token response.+ * demand with `eve__skill({ name })`, and its body shapes the reply: the skill+ * instructs an exact-token response.*/export default defineEval({tags: ["real-model"],- description: "Skills smoke: markdown skill auto-invocation via load_skill.",+ description: "Skills smoke: markdown skill auto-invocation.",async test(t) {await t.send("Please use the echo marker skill and follow its instructions exactly.");diff --git a/e2e/fixtures/agent-skills/evals/skills/dynamic-skill-map.eval.ts b/e2e/fixtures/agent-skills/evals/skills/dynamic-skill-map.eval.tsindex 18446fa02..053fd5496 100644--- a/e2e/fixtures/agent-skills/evals/skills/dynamic-skill-map.eval.ts+++ b/e2e/fixtures/agent-skills/evals/skills/dynamic-skill-map.eval.ts@@ -6,7 +6,7 @@ const DYNAMIC_MULTI_ALPHA_TOKEN = "dynamic-multi-alpha-Q8V3";* Skill smoke eval:* a `defineDynamic` multi-skill resolver (skills/dynamic-multi.ts) exposes* map entries under derived ids (`alpha`); the alpha body- * must land in the load_skill result and the reply.+ * must land in the skill load's result and the reply.*/export default defineEval({tags: ["real-model"],diff --git a/e2e/fixtures/agent-skills/evals/skills/dynamic-skill.eval.ts b/e2e/fixtures/agent-skills/evals/skills/dynamic-skill.eval.tsindex 4c427ab90..d65c03b42 100644--- a/e2e/fixtures/agent-skills/evals/skills/dynamic-skill.eval.ts+++ b/e2e/fixtures/agent-skills/evals/skills/dynamic-skill.eval.ts@@ -5,7 +5,7 @@ const DYNAMIC_SKILL_TOKEN = "dynamic-skill-ok-P4K9";/*** Skill smoke eval:* a `defineDynamic` single-skill resolver (skills/dynamic-tenant-policy.ts)- * resolves at session start; the load_skill result must carry the resolved+ * resolves at session start; the skill load's result must carry the resolved* markdown body, which then shapes the reply.*/export default defineEval({diff --git a/e2e/fixtures/agent-skills/evals/skills/skill-loads.eval.ts b/e2e/fixtures/agent-skills/evals/skills/skill-loads.eval.tsnew file mode 100644index 000000000..9456e8cab--- /dev/null+++ b/e2e/fixtures/agent-skills/evals/skills/skill-loads.eval.ts@@ -0,0 +1,35 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++import { DYNAMIC_MULTI_ALPHA_TOKEN } from "../../agent/skills/dynamic-multi";+import { DYNAMIC_SKILL_TOKEN } from "../../agent/skills/dynamic-tenant-policy";+import { HOUSE_RULES_OVERRIDE_TOKEN } from "../../agent/skills/house-rules-override";++/**+ * eve__skill({ name }) loads dynamic skills by name: a single resolved skill+ * under its file slug, a map entry under its bare key, and a dynamic skill+ * that overrides the authored skill of the same name.+ */+export default defineEval({+ description: "eve__skill({ name }) loads dynamic skills, map entries, and dynamic overrides.",+ async test(t) {+ requireMockModel(t);++ const turn = await t.send("SKILL-LOAD dynamic-tenant-policy alpha house-rules");++ turn.expectOk();+ turn.noFailedActions();+ t.loadedSkill("dynamic-tenant-policy", {+ count: 1,+ output: new RegExp(DYNAMIC_SKILL_TOKEN, "u"),+ });+ t.loadedSkill("alpha", { count: 1, output: new RegExp(DYNAMIC_MULTI_ALPHA_TOKEN, "u") });+ t.loadedSkill("house-rules", {+ count: 1,+ output: new RegExp(HOUSE_RULES_OVERRIDE_TOKEN, "u"),+ });+ t.messageIncludes(+ `${DYNAMIC_SKILL_TOKEN} ${DYNAMIC_MULTI_ALPHA_TOKEN} ${HOUSE_RULES_OVERRIDE_TOKEN}`,+ );+ },+});diff --git a/e2e/fixtures/agent-subagents-hitl/agent/agent.ts b/e2e/fixtures/agent-subagents-hitl/agent/agent.tsindex e7b520f24..fa81db70f 100644--- a/e2e/fixtures/agent-subagents-hitl/agent/agent.ts+++ b/e2e/fixtures/agent-subagents-hitl/agent/agent.ts@@ -98,7 +98,7 @@ function hasReceipt(request: MockModelRequest, tool: string): boolean {}function waitForTasks(): MockModelResponse {- return { toolCalls: [{ input: {}, name: "task_wait" }] };+ return { toolCalls: [{ input: {}, name: "eve__task_wait" }] };}export default defineAgent({diff --git a/e2e/fixtures/agent-subagents/agent/lib/notebook.ts b/e2e/fixtures/agent-subagents/agent/lib/notebook.tsindex f6e415e87..ad93aebf2 100644--- a/e2e/fixtures/agent-subagents/agent/lib/notebook.ts+++ b/e2e/fixtures/agent-subagents/agent/lib/notebook.ts@@ -67,7 +67,7 @@ function parentTurn(return {calls: [{ id: "notebook-remember", input: () => ({ message: REMEMBER_ENTRY }), name: tool },- { id: "notebook-remember-wait", name: "task_wait" },+ { id: "notebook-remember-wait", name: "eve__task_wait" },],finish: (request) => latestTaskResult(request, tool),};@@ -79,11 +79,15 @@ function parentTurn(input: (request) => ({ message: REVIEW_ENTRY, taskId: keeperTask(request) }),name: tool,},- { id: "notebook-review-wait", input: () => ({ timeoutSeconds: 2 }), name: "task_wait" },+ {+ id: "notebook-review-wait",+ input: () => ({ timeoutSeconds: 2 }),+ name: "eve__task_wait",+ },{id: "notebook-cancel",input: (request) => ({ taskId: keeperTask(request) }),- name: "task_cancel",+ name: "eve__task_cancel",},],finish: (request) => outputOf(request, "notebook-cancel"),@@ -96,7 +100,7 @@ function parentTurn(input: (request) => ({ message: RECALL_ENTRY, taskId: keeperTask(request) }),name: tool,},- { id: "notebook-recall-wait", name: "task_wait" },+ { id: "notebook-recall-wait", name: "eve__task_wait" },],finish: (request) => latestTaskResult(request, tool),};@@ -112,7 +116,7 @@ function parentTurn(}),name: tool,},- { id: "notebook-correction-wait", name: "task_wait" },+ { id: "notebook-correction-wait", name: "eve__task_wait" },],finish: (request) => latestTaskResult(request, tool),};diff --git a/e2e/fixtures/agent-subagents/agent/lib/remote-question-script.ts b/e2e/fixtures/agent-subagents/agent/lib/remote-question-script.tsindex ce55c36f6..a5f4c6f80 100644--- a/e2e/fixtures/agent-subagents/agent/lib/remote-question-script.ts+++ b/e2e/fixtures/agent-subagents/agent/lib/remote-question-script.ts@@ -17,7 +17,7 @@ export function respondToRemoteQuestion(request: MockModelRequest): MockModelResconst answer = latestTaskResult(request, "remote-loopback");if (answer !== undefined) return `PARENT-QUESTION-COMPLETE: ${answer}`;if (request.toolResults.some((entry) => entry.name === "remote-loopback")) {- return { toolCalls: [{ name: "task_wait", input: {} }] };+ return { toolCalls: [{ name: "eve__task_wait", input: {} }] };}return {toolCalls: [diff --git a/e2e/fixtures/agent-subagents/agent/lib/survey.ts b/e2e/fixtures/agent-subagents/agent/lib/survey.tsindex 4a419d9de..ae57be825 100644--- a/e2e/fixtures/agent-subagents/agent/lib/survey.ts+++ b/e2e/fixtures/agent-subagents/agent/lib/survey.ts@@ -34,7 +34,7 @@ export function respondAsSurveyParent(request: MockModelRequest): MockModelRespoinput: () => ({ message: "Please count the tide survey stations for Alice." }),name: "survey-worker",},- { id: "survey-wait", name: "task_wait" },+ { id: "survey-wait", name: "eve__task_wait" },],(finished) => `SURVEY-REPLY ${latestTaskResult(finished, "survey-worker") ?? "no result"}`,);diff --git a/e2e/fixtures/agent-subagents/evals/agent-continuation-local.eval.ts b/e2e/fixtures/agent-subagents/evals/agent-continuation-local.eval.tsindex 4fd33ee67..b572b4bc2 100644--- a/e2e/fixtures/agent-subagents/evals/agent-continuation-local.eval.ts+++ b/e2e/fixtures/agent-subagents/evals/agent-continuation-local.eval.ts@@ -5,7 +5,7 @@ import { continueKeeperAcrossTurns } from "./agent-continuation.shared";/** Deterministic counterpart of `agent-messaging-local`, with a cancel between the turns. */export default defineEval({description:- "A local agent task is continued by taskId across turns and after task_cancel with its conversation intact.",+ "A local agent task is continued by taskId across turns and after eve__task_cancel with its conversation intact.",timeoutMs: 120_000,async test(t) {await continueKeeperAcrossTurns(t, "notebook-keeper");diff --git a/e2e/fixtures/agent-subagents/evals/agent-continuation-remote.eval.ts b/e2e/fixtures/agent-subagents/evals/agent-continuation-remote.eval.tsindex 01612d67d..9dedf0888 100644--- a/e2e/fixtures/agent-subagents/evals/agent-continuation-remote.eval.ts+++ b/e2e/fixtures/agent-subagents/evals/agent-continuation-remote.eval.ts@@ -9,7 +9,7 @@ import { continueKeeperAcrossTurns } from "./agent-continuation.shared";*/export default defineEval({description:- "A remote agent task is continued by taskId across turns and after task_cancel with its conversation intact.",+ "A remote agent task is continued by taskId across turns and after eve__task_cancel with its conversation intact.",timeoutMs: 120_000,async test(t) {await continueKeeperAcrossTurns(t, "remote-loopback");diff --git a/e2e/fixtures/agent-subagents/evals/agent-continuation.shared.ts b/e2e/fixtures/agent-subagents/evals/agent-continuation.shared.tsindex de7ac4c60..2aaff67d2 100644--- a/e2e/fixtures/agent-subagents/evals/agent-continuation.shared.ts+++ b/e2e/fixtures/agent-subagents/evals/agent-continuation.shared.ts@@ -20,7 +20,7 @@ export async function continueKeeperAcrossTurns(t: EveEvalContext, tool: string)`NOTEBOOK-REVIEW ${tool} Alice asks for a review, then changes her mind.`,);reviewed.expectOk();- reviewed.calledTool("task_cancel", { count: 1, output: /^Stopped \S+'s current work;/u });+ reviewed.calledTool("eve__task_cancel", { count: 1, output: /^Stopped \S+'s current work;/u });reviewed.event("task.settled", {count: 1,data: { callId: "notebook-review", status: "cancelled" },diff --git a/e2e/fixtures/agent-tasks/evals/fans-out-then-waits.eval.ts b/e2e/fixtures/agent-tasks/evals/fans-out-then-waits.eval.tsindex c6fada137..a317f0012 100644--- a/e2e/fixtures/agent-tasks/evals/fans-out-then-waits.eval.ts+++ b/e2e/fixtures/agent-tasks/evals/fans-out-then-waits.eval.ts@@ -19,7 +19,7 @@ export default defineEval({const started = taskStarts(turn.events, "compile_report");t.check(started.length, equals(2)).label("each report is compiled once");turn.eventsSatisfy("both reports start before the first wait", (events) => {- const firstWait = firstRequestOf(events, "task_wait");+ const firstWait = firstRequestOf(events, "eve__task_wait");const starts = events.flatMap((event, index) =>event.type === "task.started" && event.data.name === "compile_report" ? [index] : [],);diff --git a/e2e/fixtures/agent-tasks/evals/keeps-tasks-after-unrelated-message.eval.ts b/e2e/fixtures/agent-tasks/evals/keeps-tasks-after-unrelated-message.eval.tsindex ab9b566a2..72a759356 100644--- a/e2e/fixtures/agent-tasks/evals/keeps-tasks-after-unrelated-message.eval.ts+++ b/e2e/fixtures/agent-tasks/evals/keeps-tasks-after-unrelated-message.eval.ts@@ -29,7 +29,7 @@ export default defineEval({const answered = await question.result();turn.expectOk();- turn.notCalledTool("task_cancel");+ turn.notCalledTool("eve__task_cancel");const started = taskStarts(turn.events, "compile_report");t.check(started.length, equals(1)).label("the report is compiled once");const settled = settlementsOf(diff --git a/e2e/fixtures/agent-tasks/evals/no-rush.eval.ts b/e2e/fixtures/agent-tasks/evals/no-rush.eval.tsindex ff7437d91..91372ec94 100644--- a/e2e/fixtures/agent-tasks/evals/no-rush.eval.ts+++ b/e2e/fixtures/agent-tasks/evals/no-rush.eval.ts@@ -18,7 +18,7 @@ export default defineEval({);turn.expectOk();- turn.notCalledTool("task_wait");+ turn.notCalledTool("eve__task_wait");turn.eventsSatisfy("eve holds the turn after the reply", heldTurn);turn.eventsSatisfy("the model replies before the report is ready", (events) => {const callIds = taskStarts(events, "compile_report").map((call) => call.callId);diff --git a/e2e/fixtures/agent-tasks/evals/redirects-an-agent.eval.ts b/e2e/fixtures/agent-tasks/evals/redirects-an-agent.eval.tsindex 6a6f66fe9..f50ebb5f5 100644--- a/e2e/fixtures/agent-tasks/evals/redirects-an-agent.eval.ts+++ b/e2e/fixtures/agent-tasks/evals/redirects-an-agent.eval.ts@@ -6,7 +6,7 @@ import { taskStarts } from "./task-events";/*** While the researcher looks into EMEA, Alice corrects the region. The model* either corrects the running task by calling the researcher again with its- * taskId, or stops it with task_cancel and asks again; either way the answer+ * taskId, or stops it with eve__task_cancel and asks again; either way the answer* is about APAC.*/export default defineEval({@@ -34,7 +34,7 @@ export default defineEval({(call) => call.taskId === first.data.taskId && call.callId !== first.data.callId,);const cancelled = turn.toolCalls.some(- (call) => call.name === "task_cancel" && call.input.taskId === first.data.taskId,+ (call) => call.name === "eve__task_cancel" && call.input.taskId === first.data.taskId,);return corrected || cancelled;},diff --git a/e2e/fixtures/agent-tasks/evals/task-events.ts b/e2e/fixtures/agent-tasks/evals/task-events.tsindex 526e35065..6836eb6d9 100644--- a/e2e/fixtures/agent-tasks/evals/task-events.ts+++ b/e2e/fixtures/agent-tasks/evals/task-events.ts@@ -7,7 +7,7 @@ import type {/*** Whether eve held the turn: the model ended a step with a reply while its* tasks worked, and the turn parked with `turn.waiting` instead of ending. A- * `turn.waiting` after a `task_wait` call is the model waiting, not a hold.+ * `turn.waiting` after a `eve__task_wait` call is the model waiting, not a hold.*/export function heldTurn(events: readonly MessageStreamEvent[]): boolean {let replied = false;diff --git a/e2e/fixtures/agent-tasks/evals/waits-when-needed.eval.ts b/e2e/fixtures/agent-tasks/evals/waits-when-needed.eval.tsindex 47d3c008b..7a16ae189 100644--- a/e2e/fixtures/agent-tasks/evals/waits-when-needed.eval.ts+++ b/e2e/fixtures/agent-tasks/evals/waits-when-needed.eval.ts@@ -5,7 +5,7 @@ import { heldTurn } from "./task-events";/*** The answer depends on the researcher's result, so the model waits for it- * with `task_wait` instead of ending its step while the task works.+ * with `eve__task_wait` instead of ending its step while the task works.*/export default defineEval({description: "The model waits for an agent's result when its answer depends on it.",@@ -17,7 +17,7 @@ export default defineEval({turn.expectOk();t.calledSubagent("researcher", { status: "completed" });- turn.calledTool("task_wait");+ turn.calledTool("eve__task_wait");turn.eventsSatisfy("the model never replies while the researcher works",(events) => !heldTurn(events),diff --git a/e2e/fixtures/agent-tool-discovery/.gitignore b/e2e/fixtures/agent-tool-discovery/.gitignorenew file mode 100644index 000000000..eb095f0f8--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/.gitignore@@ -0,0 +1,10 @@+node_modules+.env*+.eve+.vercel+.next+.output+.nitro+dist+.DS_Store+*.tsbuildinfodiff --git a/e2e/fixtures/agent-tool-discovery/.vercelignore b/e2e/fixtures/agent-tool-discovery/.vercelignorenew file mode 100644index 000000000..357664320--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/.vercelignore@@ -0,0 +1,6 @@+node_modules+.eve+.next+.output+.nitro+distdiff --git a/e2e/fixtures/agent-tool-discovery/agent/agent.ts b/e2e/fixtures/agent-tool-discovery/agent/agent.tsnew file mode 100644index 000000000..b2175fb59--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/agent/agent.ts@@ -0,0 +1,12 @@+import { e2eAgentConfig } from "@eve-e2e/config";+import { defineAgent } from "eve";++/**+ * Real-model gate for discovery: a large deferred catalog with no connections,+ * beside the default `web_search` and `bash`. Every eval here is tagged+ * `real-model`, because it checks which tools a live model reaches for.+ */+export default defineAgent({+ ...e2eAgentConfig(),+ description: "Help a platform team with operations, support, and data questions.",+});diff --git a/e2e/fixtures/agent-tool-discovery/agent/instructions.md b/e2e/fixtures/agent-tool-discovery/agent/instructions.mdnew file mode 100644index 000000000..248dc537c--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/agent/instructions.md@@ -0,0 +1,5 @@+# Identity++You help Alice and Bob's platform team with operations, support, sales, and data+questions. Keep replies short, and quote ids, names, and times exactly as you+receive them.diff --git a/e2e/fixtures/agent-tool-discovery/agent/lib/catalog.ts b/e2e/fixtures/agent-tool-discovery/agent/lib/catalog.tsnew file mode 100644index 000000000..1c201185b--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/agent/lib/catalog.ts@@ -0,0 +1,392 @@+/**+ * The fixture's deferred tools: a large catalog shaped like a real agent's,+ * with extensions mounted as `sre`, `d0`, and `index`, vendored `support__*`+ * tools, and a few tools without a namespace. Every tool returns fixed data,+ * so a reply that carries a tool's reference proves the model ran it.+ */++export interface CatalogTool {+ readonly description: string;+ readonly name: string;+ /** Optional string inputs, by name and description. */+ readonly inputs?: Readonly<Record<string, string>>;+ /** What the tool returns, beyond its `reference`. */+ readonly result?: Readonly<Record<string, unknown>>;+}++/** The open incidents `sre__status_page_incidents_list` returns. */+export const OPEN_INCIDENTS = [+ { id: "INC-4821", status: "investigating", title: "Elevated API latency in iad1" },+ { id: "INC-4790", status: "monitoring", title: "Delayed webhook deliveries" },+];++/** The next mobile release train `release_train_schedule` returns. */+export const NEXT_RELEASE_TRAIN = {+ captain: "Priya Natarajan",+ departs: "2026-10-14T17:00:00Z",+ train: "mobile-2026.42",+};++const NAMESPACED: readonly CatalogTool[] = [+ // sre: incident response, status page, and observability.+ {+ name: "sre__status_page_incidents_list",+ description: "List incidents on the company status page, optionally filtered by status.",+ inputs: { status: "open, resolved, or all" },+ result: { incidents: OPEN_INCIDENTS },+ },+ {+ name: "sre__status_page_incident_get",+ description: "Get one status page incident by id.",+ inputs: { incidentId: "Incident id" },+ },+ {+ name: "sre__status_page_incident_create",+ description: "Open a new status page incident.",+ inputs: { title: "Incident title", impact: "none, minor, major, or critical" },+ },+ {+ name: "sre__status_page_incident_update",+ description: "Post an update to a status page incident.",+ inputs: { incidentId: "Incident id", message: "Update text" },+ },+ {+ name: "sre__status_page_components_list",+ description: "List status page components and their current status.",+ },+ {+ name: "sre__status_page_component_get",+ description: "Get one status page component by id.",+ inputs: { componentId: "Component id" },+ },+ {+ name: "sre__lookup_ownership",+ description: "Find the team that owns a service, repository, or alert.",+ inputs: { subject: "Service, repository, or alert name" },+ },+ {+ name: "sre__build_datadog_link",+ description: "Build a Datadog dashboard link for a service and time range.",+ inputs: { service: "Service name", range: "Time range" },+ },+ {+ name: "sre__build_cdn_dashboard_link",+ description: "Build a CDN analytics dashboard link for a project.",+ inputs: { project: "Project name" },+ },+ {+ name: "sre__evidence_record",+ description: "Record a piece of evidence on an incident timeline.",+ inputs: { incidentId: "Incident id", note: "Evidence" },+ },+ {+ name: "sre__evidence_list",+ description: "List the evidence recorded on an incident timeline.",+ inputs: { incidentId: "Incident id" },+ },+ {+ name: "sre__evidence_remove",+ description: "Remove a piece of evidence from an incident timeline.",+ inputs: { evidenceId: "Evidence id" },+ },+ {+ name: "sre__oncall_schedule_get",+ description: "Get who is on call for a team this week.",+ inputs: { team: "Team name" },+ },+ {+ name: "sre__alert_rules_list",+ description: "List alerting rules for a service.",+ inputs: { service: "Service name" },+ },+ { name: "sre__deploy_freeze_status", description: "Check whether a deploy freeze is in effect." },+ {+ name: "sre__error_budget_get",+ description: "Get a service's remaining error budget for its SLO.",+ inputs: { service: "Service name" },+ },+ // d0: the data warehouse and business metrics.+ {+ name: "d0__run_sql",+ description: "Run a read-only SQL query against the data warehouse.",+ inputs: { sql: "SQL query" },+ },+ {+ name: "d0__query_metrics",+ description: "Query a named business metric over a time range.",+ inputs: { metric: "Metric name", range: "Time range" },+ },+ {+ name: "d0__query_cost",+ description: "Query cloud infrastructure cost by team or service.",+ inputs: { groupBy: "team or service" },+ },+ {+ name: "d0__find_customer",+ description: "Find a customer account by name or domain.",+ inputs: { query: "Customer name or domain" },+ },+ {+ name: "d0__find_data",+ description: "Find warehouse tables and columns that hold a kind of data.",+ inputs: { topic: "What the data is about" },+ },+ {+ name: "d0__profile_site",+ description: "Profile a customer site's traffic and framework usage.",+ inputs: { domain: "Site domain" },+ },+ {+ name: "d0__export_sheet",+ description: "Export query results to a spreadsheet.",+ inputs: { queryId: "Query id" },+ },+ {+ name: "d0__query_partnerships",+ description: "Query revenue influenced by partnerships.",+ inputs: { partner: "Partner name" },+ },+ {+ name: "d0__usage_summary",+ description: "Summarize a customer's product usage for a month.",+ inputs: { customer: "Customer", month: "Month as YYYY-MM" },+ },+ {+ name: "d0__revenue_by_plan",+ description: "Break down monthly recurring revenue by plan.",+ inputs: { month: "Month as YYYY-MM" },+ },+ {+ name: "d0__churn_cohorts",+ description: "Report customer churn by signup cohort.",+ inputs: { cohort: "Signup quarter" },+ },+ {+ name: "d0__funnel_report",+ description: "Report conversion through the signup funnel.",+ inputs: { range: "Time range" },+ },+ {+ name: "d0__data_glossary_lookup",+ description: "Look up the definition of a business metric or term.",+ inputs: { term: "Term" },+ },+ {+ name: "d0__dashboard_link",+ description: "Get the link to a saved analytics dashboard.",+ inputs: { dashboard: "Dashboard name" },+ },+ {+ name: "d0__query_latency",+ description: "Query request latency percentiles for a region.",+ inputs: { region: "Region" },+ },+ // index: accounts, sales activity, and internal knowledge.+ {+ name: "index__resolve_account",+ description: "Resolve a company name to its CRM account.",+ inputs: { company: "Company name" },+ },+ {+ name: "index__list_opportunities",+ description: "List open sales opportunities for an account.",+ inputs: { account: "Account" },+ },+ {+ name: "index__get_opportunity_metrics",+ description: "Get pipeline metrics for an opportunity.",+ inputs: { opportunityId: "Opportunity id" },+ },+ {+ name: "index__get_call_insights",+ description: "Summarize insights from recent sales calls with an account.",+ inputs: { account: "Account" },+ },+ {+ name: "index__get_meeting_transcript",+ description: "Get the transcript of a recorded meeting.",+ inputs: { meetingId: "Meeting id" },+ },+ {+ name: "index__list_upcoming_meetings",+ description: "List upcoming meetings with an account.",+ inputs: { account: "Account" },+ },+ {+ name: "index__search_meetings",+ description: "Search recorded meetings by topic.",+ inputs: { topic: "Topic" },+ },+ {+ name: "index__search_docs",+ description: "Search the company's internal documentation.",+ inputs: { query: "Words to find" },+ },+ {+ name: "index__search_customer_stories",+ description: "Search published customer stories by industry or product.",+ inputs: { query: "Words to find" },+ },+ {+ name: "index__search_slack",+ description: "Search the company's Slack messages.",+ inputs: { query: "Words to find" },+ },+ {+ name: "index__get_sales_guidance",+ description: "Get the sales playbook guidance for a product.",+ inputs: { product: "Product" },+ },+ {+ name: "index__list_implementations",+ description: "List customer implementation projects.",+ inputs: { account: "Account" },+ },+ {+ name: "index__get_implementation_metrics",+ description: "Get progress metrics for an implementation project.",+ inputs: { implementationId: "Implementation id" },+ },+ {+ name: "index__get_activity_metrics",+ description: "Get a sales rep's activity metrics for a quarter.",+ inputs: { rep: "Rep name", quarter: "Quarter" },+ },+ {+ name: "index__list_deliverables",+ description: "List deliverables owed to a customer.",+ inputs: { account: "Account" },+ },+ {+ name: "index__present_estimate",+ description: "Prepare a pricing estimate for a customer.",+ inputs: { account: "Account", plan: "Plan" },+ },+ // support: vendored support desk tools.+ {+ name: "support__search_cases",+ description: "Search support cases by keyword or customer.",+ inputs: { query: "Words to find" },+ },+ {+ name: "support__usage_cases",+ description: "List support cases about usage limits.",+ inputs: { customer: "Customer" },+ },+ {+ name: "support__support_billing",+ description: "Look up a customer's billing details for support.",+ inputs: { customer: "Customer" },+ },+ {+ name: "support__diagnose_infrastructure",+ description: "Run infrastructure diagnostics for a customer project.",+ inputs: { project: "Project" },+ },+ {+ name: "support__link_incident",+ description: "Link a support case to an incident.",+ inputs: { caseId: "Case id", incidentId: "Incident id" },+ },+ {+ name: "support__get_email_content",+ description: "Get the content of a support email thread.",+ inputs: { threadId: "Thread id" },+ },+ {+ name: "support__latest_chrome_versions",+ description: "List the latest Chrome versions support has verified.",+ inputs: {},+ },+ {+ name: "support__case_get",+ description: "Get one support case by id.",+ inputs: { caseId: "Case id" },+ },+ {+ name: "support__case_assign",+ description: "Assign a support case to an agent.",+ inputs: { caseId: "Case id", assignee: "Agent" },+ },+ {+ name: "support__case_reply_draft",+ description: "Draft a reply to a support case.",+ inputs: { caseId: "Case id" },+ },+ {+ name: "support__refund_request_status",+ description: "Check the status of a customer's refund request.",+ inputs: { requestId: "Request id" },+ },+ {+ name: "support__sla_breaches_list",+ description: "List support cases that breached their SLA.",+ inputs: { range: "Time range" },+ },+ {+ name: "support__macro_lookup",+ description: "Look up a saved support reply macro.",+ inputs: { macro: "Macro name" },+ },+];++const PLAIN: readonly CatalogTool[] = [+ {+ name: "expense_policy_lookup",+ description: "Look up the company's expense policy for a category.",+ inputs: { category: "Expense category" },+ },+ {+ name: "holiday_calendar",+ description: "List an office's company holidays for a month.",+ inputs: { office: "Office", month: "Month" },+ },+ {+ name: "meeting_room_book",+ description: "Book a meeting room in an office.",+ inputs: { office: "Office", time: "Start time" },+ },+ {+ name: "travel_request_status",+ description: "Check the status of a business travel request.",+ inputs: { requestId: "Request id" },+ },+ {+ name: "laptop_order_status",+ description: "Check the status of a laptop order from IT.",+ inputs: { orderId: "Order id" },+ },+ {+ name: "parking_permit_status",+ description: "Check the status of an office parking permit.",+ inputs: { employee: "Employee" },+ },+ {+ name: "team_directory_lookup",+ description: "Look up a person's team, manager, and location.",+ inputs: { person: "Name" },+ },+ {+ name: "timezone_overlap",+ description: "Find working-hour overlap between team members' time zones.",+ inputs: { people: "Names, comma separated" },+ },+ {+ name: "release_train_schedule",+ description: "Get the departure time and release captain of the next release train for an app.",+ inputs: { app: "web, mobile, or desktop" },+ result: NEXT_RELEASE_TRAIN,+ },+ {+ name: "office_floor_plan",+ description: "Get the floor plan of an office.",+ inputs: { office: "Office" },+ },+];++export const CATALOG_TOOLS: readonly CatalogTool[] = [...NAMESPACED, ...PLAIN];++/** A stable reference per tool, so a reply quoting it proves which tool ran. */+export function referenceOf(name: string): string {+ return `REF-${name.replaceAll("_", "-").toUpperCase()}`;+}diff --git a/e2e/fixtures/agent-tool-discovery/agent/skills/support__escalation-policy/SKILL.md b/e2e/fixtures/agent-tool-discovery/agent/skills/support__escalation-policy/SKILL.mdnew file mode 100644index 000000000..780bd5b02--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/agent/skills/support__escalation-policy/SKILL.md@@ -0,0 +1,9 @@+---+description: When and how to escalate a support case to engineering.+deferred: true+---++# Escalation policy++Escalate a case to engineering when a customer's production traffic is affected+for more than 30 minutes, and link the case to the matching incident.diff --git a/e2e/fixtures/agent-tool-discovery/agent/subagents/account_researcher/agent.ts b/e2e/fixtures/agent-tool-discovery/agent/subagents/account_researcher/agent.tsnew file mode 100644index 000000000..2b5050073--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/agent/subagents/account_researcher/agent.ts@@ -0,0 +1,9 @@+import { e2eSubagentConfig } from "@eve-e2e/config";+import { defineAgent } from "eve";++/** A deferred agent: the root reaches it only through eve__search and eve__tool. */+export default defineAgent({+ ...e2eSubagentConfig(),+ description: "Research a customer account's history and summarize it for a meeting.",+ tool: "deferred",+});diff --git a/e2e/fixtures/agent-tool-discovery/agent/subagents/account_researcher/instructions.md b/e2e/fixtures/agent-tool-discovery/agent/subagents/account_researcher/instructions.mdnew file mode 100644index 000000000..755ef4e62--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/agent/subagents/account_researcher/instructions.md@@ -0,0 +1,3 @@+# Identity++You research customer accounts and summarize their history in a few sentences.diff --git a/e2e/fixtures/agent-tool-discovery/agent/tools/catalog.ts b/e2e/fixtures/agent-tool-discovery/agent/tools/catalog.tsnew file mode 100644index 000000000..685fe6699--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/agent/tools/catalog.ts@@ -0,0 +1,33 @@+import { defineDynamic, defineTool } from "eve/tools";++import { CATALOG_TOOLS, referenceOf } from "../lib/catalog";++/**+ * Every catalog tool, deferred, for the whole session. One resolver serves the+ * table so the evals and the agent agree on names and results; the model sees+ * the same deferred entries as authored files would give it.+ */+export default defineDynamic({+ events: {+ "session.started": () =>+ Object.fromEntries(+ CATALOG_TOOLS.map(({ description, inputs = {}, name, result }) => [+ name,+ defineTool({+ deferred: true,+ description,+ inputSchema: {+ type: "object",+ properties: Object.fromEntries(+ Object.entries(inputs).map(([input, about]) => [+ input,+ { type: "string", description: about },+ ]),+ ),+ },+ execute: () => ({ reference: referenceOf(name), ...result }),+ }),+ ]),+ ),+ },+});diff --git a/e2e/fixtures/agent-tool-discovery/evals/discover-namespaced-tool.eval.ts b/e2e/fixtures/agent-tool-discovery/evals/discover-namespaced-tool.eval.tsnew file mode 100644index 000000000..61f60287b--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/evals/discover-namespaced-tool.eval.ts@@ -0,0 +1,30 @@+import { SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { defineEval } from "eve/evals";++import { OPEN_INCIDENTS } from "../agent/lib/catalog";++const TOOL = "sre__status_page_incidents_list";++/**+ * A request for a deferred tool's capability: the model searches its catalog+ * with eve__search, then runs the tool through eve__tool.+ */+export default defineEval({+ tags: ["real-model"],+ description:+ "A request matching a namespaced deferred tool finds it with eve__search and runs it.",++ async test(t) {+ const turn = await t.send(+ [+ "Alice is starting her on-call shift and wants to know which incidents are open on our status page right now.",+ "Please check and give her each incident's id and title.",+ ].join(" "),+ );++ turn.expectOk();+ t.toolOrder([SEARCH_TOOL, TOOL]);+ t.calledTool(TOOL, { status: "completed" });+ t.messageIncludes(OPEN_INCIDENTS[0]!.id);+ },+});diff --git a/e2e/fixtures/agent-tool-discovery/evals/discover-plain-tool.eval.ts b/e2e/fixtures/agent-tool-discovery/evals/discover-plain-tool.eval.tsnew file mode 100644index 000000000..787af7b15--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/evals/discover-plain-tool.eval.ts@@ -0,0 +1,31 @@+import { SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { defineEval } from "eve/evals";++import { NEXT_RELEASE_TRAIN } from "../agent/lib/catalog";++const TOOL = "release_train_schedule";++/**+ * A deferred tool without a namespace, which the request describes but never+ * names: the model searches its catalog with eve__search, then runs the tool+ * through eve__tool.+ */+export default defineEval({+ tags: ["real-model"],+ description:+ "A request matching a deferred tool without a namespace finds it with eve__search and runs it.",++ async test(t) {+ const turn = await t.send(+ [+ "Bob is planning his week around the mobile app release.",+ "When does the next mobile release train depart, and who is its release captain?",+ ].join(" "),+ );++ turn.expectOk();+ t.toolOrder([SEARCH_TOOL, TOOL]);+ t.calledTool(TOOL, { status: "completed" });+ t.messageIncludes(NEXT_RELEASE_TRAIN.captain);+ },+});diff --git a/e2e/fixtures/agent-tool-discovery/evals/evals.config.ts b/e2e/fixtures/agent-tool-discovery/evals/evals.config.tsnew file mode 100644index 000000000..895f61fe3--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/evals/evals.config.ts@@ -0,0 +1,4 @@+import { defineEvalConfig } from "eve/evals";++// A shell eval may start the session's first sandbox, which can take a while.+export default defineEvalConfig({ timeoutMs: 240_000 });diff --git a/e2e/fixtures/agent-tool-discovery/evals/no-catalog-for-known-answers.eval.ts b/e2e/fixtures/agent-tool-discovery/evals/no-catalog-for-known-answers.eval.tsnew file mode 100644index 000000000..069ad9a2d--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/evals/no-catalog-for-known-answers.eval.ts@@ -0,0 +1,21 @@+import { defineEval } from "eve/evals";++import { calledTools, usesCatalog } from "./tool-use";++/** A question the model can answer itself doesn't send it to the catalog. */+export default defineEval({+ tags: ["real-model"],+ description: "A plain question is answered without the catalog tools.",++ async test(t) {+ const turn = await t.send(+ "Alice is converting a recipe for Bob. How many grams are in 3 ounces, rounded to the nearest gram?",+ );++ turn.expectOk();+ turn.eventsSatisfy("no catalog tool or entry is called", (events) =>+ calledTools(events).every((name) => !usesCatalog(name)),+ );+ t.messageIncludes(/\b85\b/u);+ },+});diff --git a/e2e/fixtures/agent-tool-discovery/evals/shell-routing.eval.ts b/e2e/fixtures/agent-tool-discovery/evals/shell-routing.eval.tsnew file mode 100644index 000000000..3ca98ccbe--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/evals/shell-routing.eval.ts@@ -0,0 +1,24 @@+import { defineEval } from "eve/evals";++import { calledTools, usesCatalog } from "./tool-use";++/** A shell request goes to `bash`; `eve__tool` only runs catalog entries. */+export default defineEval({+ tags: ["real-model"],+ description: "A request to inspect files with a shell command goes to bash, not eve__tool.",++ async test(t) {+ const turn = await t.send(+ [+ "Bob wants a quick look at the files in your working directory.",+ "Please run `ls -la` in a shell and tell him how many entries it lists.",+ ].join(" "),+ );++ turn.expectOk();+ turn.eventsSatisfy("bash is called, and the catalog isn't", (events) => {+ const called = calledTools(events);+ return called.includes("bash") && !called.some(usesCatalog);+ });+ },+});diff --git a/e2e/fixtures/agent-tool-discovery/evals/tool-use.ts b/e2e/fixtures/agent-tool-discovery/evals/tool-use.tsnew file mode 100644index 000000000..da55d4614--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/evals/tool-use.ts@@ -0,0 +1,36 @@+import { CALL_TOOL, SEARCH_TOOL, SKILL_TOOL } from "@eve-e2e/config/catalog-tools";+import type { EveEvalTurn } from "eve/evals";++import { CATALOG_TOOLS } from "../agent/lib/catalog";++type Events = Parameters<Parameters<EveEvalTurn["eventsSatisfy"]>[1]>[0];++/** Entries only the catalog tools reach: the deferred tools, skill, and agent. */+const CATALOG_ENTRIES = new Set([+ ...CATALOG_TOOLS.map(({ name }) => name),+ "support__escalation-policy",+ "account_researcher",+]);++/**+ * The tools the model called, in order. A call through `eve__tool` is reported+ * under the entry it reaches, and an `eve__skill` load as `load-skill`.+ */+export function calledTools(events: Events): string[] {+ return events.flatMap((event) =>+ event.type === "actions.requested"+ ? event.data.actions.map((action) => ("toolName" in action ? action.toolName : action.kind))+ : [],+ );+}++/** Whether a called tool belongs to the catalog: a catalog tool, or an entry they reach. */+export function usesCatalog(name: string): boolean {+ return (+ name === SEARCH_TOOL ||+ name === CALL_TOOL ||+ name === SKILL_TOOL ||+ name === "load-skill" ||+ CATALOG_ENTRIES.has(name)+ );+}diff --git a/e2e/fixtures/agent-tool-discovery/evals/web-routing.eval.ts b/e2e/fixtures/agent-tool-discovery/evals/web-routing.eval.tsnew file mode 100644index 000000000..b50c66eb6--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/evals/web-routing.eval.ts@@ -0,0 +1,30 @@+import { defineEval } from "eve/evals";++import { calledTools, usesCatalog } from "./tool-use";++const WEB_TOOLS = new Set(["web_search", "web_fetch"]);++/**+ * A question about the world goes to the web tools; `eve__search` only finds the agent's own tools.+ * Either web tool is a correct route: models answer this with `web_search` or by fetching a release index with `web_fetch`.+ */+export default defineEval({+ tags: ["real-model"],+ description:+ "A question that needs current public information goes to web_search or web_fetch, not eve__search.",++ async test(t) {+ const turn = await t.send(+ [+ "Alice is preparing a short briefing for Bob and needs current public information.",+ "Please look up on the web the most recent stable release of Node.js and when it came out.",+ ].join(" "),+ );++ turn.expectOk();+ turn.eventsSatisfy("a web tool is called, and the catalog isn't", (events) => {+ const called = calledTools(events);+ return called.some((name) => WEB_TOOLS.has(name)) && !called.some(usesCatalog);+ });+ },+});diff --git a/e2e/fixtures/agent-tool-discovery/package.json b/e2e/fixtures/agent-tool-discovery/package.jsonnew file mode 100644index 000000000..57fb036b2--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/package.json@@ -0,0 +1,29 @@+{+ "name": "agent-tool-discovery",+ "version": "0.0.0",+ "private": true,+ "type": "module",+ "scripts": {+ "build": "eve build",+ "dev": "eve dev",+ "start": "eve start",+ "typecheck": "eve build --skip-sandbox-prewarm && tsc",+ "test:e2e": "eve eval --strict"+ },+ "dependencies": {+ "@eve-e2e/config": "workspace:*",+ "@workflow/world-postgres": "catalog:",+ "eve": "workspace:*",+ "zod": "catalog:"+ },+ "devDependencies": {+ "@types/node": "catalog:",+ "typescript": "catalog:"+ },+ "e2e": {+ "modelMatrix": "full",+ "optionalModels": [+ "anthropic-opus"+ ]+ }+}diff --git a/e2e/fixtures/agent-tool-discovery/tsconfig.json b/e2e/fixtures/agent-tool-discovery/tsconfig.jsonnew file mode 100644index 000000000..85e3eceb8--- /dev/null+++ b/e2e/fixtures/agent-tool-discovery/tsconfig.json@@ -0,0 +1,17 @@+{+ "compilerOptions": {+ "target": "ES2022",+ "module": "esnext",+ "moduleResolution": "bundler",+ "outDir": "dist",+ "rootDir": ".",+ "strict": true,+ "esModuleInterop": true,+ "skipLibCheck": true,+ "forceConsistentCasingInFileNames": true,+ "declaration": true,+ "noEmit": true,+ "types": ["node"]+ },+ "include": ["agent/**/*.ts", "evals/**/*.ts"]+}diff --git a/e2e/fixtures/agent-tools-sandbox/agent/lib/mock-responder.ts b/e2e/fixtures/agent-tools-sandbox/agent/lib/mock-responder.tsindex 75a3e475a..068cc37db 100644--- a/e2e/fixtures/agent-tools-sandbox/agent/lib/mock-responder.ts+++ b/e2e/fixtures/agent-tools-sandbox/agent/lib/mock-responder.ts@@ -1,3 +1,4 @@+import { SKILL_TOOL } from "@eve-e2e/config/catalog-tools";import type { MockModelRequest, MockModelResponse } from "eve/evals";const SUBAGENT_DIRECTIVE = /ask the `([^`]+)` subagent with message:\s*([\s\S]+)/iu;@@ -53,7 +54,9 @@ export function respond(request: MockModelRequest): MockModelResponse | string {return { toolCalls: [{ input: { message: subagent[2] }, name: subagent[1] }] };}// The agent call returned a receipt; its result arrives in a <task_result> message.- return taskResultOf(request, subagent[1]) ?? { toolCalls: [{ input: {}, name: "task_wait" }] };+ return (+ taskResultOf(request, subagent[1]) ?? { toolCalls: [{ input: {}, name: "eve__task_wait" }] }+ );}const bash = BASH_DIRECTIVE.exec(message);@@ -70,11 +73,11 @@ export function respond(request: MockModelRequest): MockModelResponse | string {const skill = SKILL_DIRECTIVE.exec(message);if (skill?.[1] !== undefined) {if (!turnHasToolResult) {- return { toolCalls: [{ input: { skill: skill[1] }, name: "load_skill" }] };+ return { toolCalls: [{ input: { name: skill[1] }, name: SKILL_TOOL }] };}// Loaded skills instruct an exact reply whose text is the skill body's// final line (see the redeploy eval's deploy-note skill).- return lastNonEmptyLine(toolOutput(request, "load_skill"));+ return lastNonEmptyLine(toolOutput(request, SKILL_TOOL));}return `Mock reply: ${message}`;diff --git a/e2e/fixtures/agent-workflow-tools/agent/agent.ts b/e2e/fixtures/agent-workflow-tools/agent/agent.tsindex 06ac109e6..19604b735 100644--- a/e2e/fixtures/agent-workflow-tools/agent/agent.ts+++ b/e2e/fixtures/agent-workflow-tools/agent/agent.ts@@ -1,5 +1,6 @@import { e2eAgentConfig } from "@eve-e2e/config";-import { latestTaskResult } from "@eve-e2e/config/mock-script";+import { CALL_TOOL, SEARCH_TOOL, SKILL_TOOL } from "@eve-e2e/config/catalog-tools";+import { latestTaskResult, outputOf, playScript } from "@eve-e2e/config/mock-script";import { defineAgent } from "eve";import { mockModel, type MockModelRequest, type MockModelResponse } from "eve/evals";@@ -18,16 +19,24 @@ const HOOK_SCENARIO_CALLS = {} as const;/**- * public-catalog lists its tools without sign-in: run its public tool, then its- * protected one, which asks the user to sign in and resumes once they have.+ * public-catalog lists its tools without sign-in: connect it, which needs no+ * sign-in, run its public tool, then its protected one, which asks the user to+ * sign in and resumes once they have.*/function respondToPublicCatalog(request: MockModelRequest): MockModelResponse | string {const call = (id: string, tool: string) => ({id,- name: "connection_execute",- input: { connection: "public-catalog", tool, input: {} },+ name: CALL_TOOL,+ input: { name: `public-catalog__${tool}`, input: {} },});const byId = new Map(request.toolResults.map((entry) => [entry.id, entry]));+ if (!byId.has("public-catalog-connect")) {+ return {+ toolCalls: [+ { id: "public-catalog-connect", name: CALL_TOOL, input: { name: "public-catalog" } },+ ],+ };+ }if (!byId.has("public-catalog-items")) {return { toolCalls: [call("public-catalog-items", "list_items")] };}@@ -49,7 +58,7 @@ async function respond(request: MockModelRequest): Promise<MockModelResponse | sconst skillCallId = auditing ? "audit-policy" : "initial-policy";if (!request.toolResults.some((entry) => entry.id === skillCallId)) {return {- toolCalls: [{ id: skillCallId, name: "load_skill", input: { skill: "delegation-policy" } }],+ toolCalls: [{ id: skillCallId, name: SKILL_TOOL, input: { name: "delegation-policy" } }],};}const mode = (/SUBAGENT-HOOKS:(direct|background|waiting)/u.exec(hookScenario)?.[1] ??@@ -68,7 +77,9 @@ async function respond(request: MockModelRequest): Promise<MockModelResponse | sif (call === undefined) return { toolCalls: [{ name: tool, input }] };// An agent call or task returned a receipt; its result arrives in a <task_result> message.if (mode === "direct") {- return latestTaskResult(request, tool) ?? { toolCalls: [{ name: "task_wait", input: {} }] };+ return (+ latestTaskResult(request, tool) ?? { toolCalls: [{ name: "eve__task_wait", input: {} }] }+ );}// Ends the step with text while the task works, which holds the turn. The// step takes a few seconds, so the task's helper most likely opens while it@@ -87,50 +98,28 @@ async function respond(request: MockModelRequest): Promise<MockModelResponse | sreturn respondToPublicCatalog(request);}const message =- [...request.userMessages]- .reverse()- .find((entry) => entry.startsWith("WORKFLOW-") || entry.includes("private-catalog")) ?? "";+ [...request.userMessages].reverse().find((entry) => entry.startsWith("WORKFLOW-")) ?? "";const scenario = respondToTaskScenario(request, directiveOf(message));if (scenario !== undefined) return scenario;- if (message.includes("private-catalog")) {- const search = [...request.toolResults]- .reverse()- .find((entry) => entry.name === "connection_search");- if (search === undefined) {- return {- toolCalls: [- {- name: "connection_search",- input: { connection: "private-catalog", query: "items" },- },- ],- };- }- // A plain search reports the connection as requiring sign-in; `signIn` asks the user.- if (JSON.stringify(search.output).includes('"requiresSignIn":true')) {- return {- toolCalls: [- {- name: "connection_search",- input: { connection: "private-catalog", query: "items", signIn: true },- },- ],- };- }- const execute = request.toolResults.find((entry) => entry.name === "connection_execute");- if (execute === undefined) {- return {- toolCalls: [- {- name: "connection_execute",- input: { connection: "private-catalog", tool: "list_items", input: {} },- },- ],- };- }- return JSON.stringify(execute.output);+ if (message.startsWith("WORKFLOW-CATALOG-SIGN-IN")) {+ // A plain search finds the catalog as its sign-in entry; executing it asks+ // the user. Sign-in drops that interrupted call from history, so the script+ // makes it again once the turn resumes, then searches and calls a tool.+ return playScript(+ request,+ [+ { id: "catalog-search", name: SEARCH_TOOL, input: () => ({ query: "private-catalog" }) },+ { id: "catalog-sign-in", name: CALL_TOOL, input: () => ({ name: "private-catalog" }) },+ { id: "catalog-tools", name: SEARCH_TOOL, input: () => ({ query: "private-catalog__" }) },+ {+ id: "catalog-items",+ name: CALL_TOOL,+ input: () => ({ name: "private-catalog__list_items" }),+ },+ ],+ (finished) => outputOf(finished, "catalog-items"),+ );}-const stepAuth = /WORKFLOW-STEP-AUTH-(IMPLICIT|EXPLICIT|REJECTED)/u.exec(message);if (stepAuth !== null) {const result = request.toolResults.find((entry) => entry.name === "authorize_service");diff --git a/e2e/fixtures/agent-workflow-tools/agent/lib/task-scenarios.ts b/e2e/fixtures/agent-workflow-tools/agent/lib/task-scenarios.tsindex f1fe861fb..0092e5e5b 100644--- a/e2e/fixtures/agent-workflow-tools/agent/lib/task-scenarios.ts+++ b/e2e/fixtures/agent-workflow-tools/agent/lib/task-scenarios.ts@@ -28,7 +28,7 @@ const SCENARIOS: Readonly<Record<string, Scenario>> = {request,[{ id: "stage", input: apiService, name: "stage_deploy" },- { id: "stage-wait", name: "task_wait" },+ { id: "stage-wait", name: "eve__task_wait" },],() => report("WORKFLOW-STAGE-RESULT", latestTaskResult(request, "stage_deploy")),),@@ -53,7 +53,7 @@ const SCENARIOS: Readonly<Record<string, Scenario>> = {request,[{ id: "canary", input: apiService, name: "canary_deploy" },- { id: "canary-cancel", input: taskOf("canary"), name: "task_cancel" },+ { id: "canary-cancel", input: taskOf("canary"), name: "eve__task_cancel" },],() => report("WORKFLOW-CANARY-RESULT", outputOf(request, "canary-cancel")),),@@ -63,7 +63,7 @@ const SCENARIOS: Readonly<Record<string, Scenario>> = {request,[{ id: "plan-draft", input: () => ({ request: "draft" }), name: "revise_plan" },- { id: "plan-draft-wait", name: "task_wait" },+ { id: "plan-draft-wait", name: "eve__task_wait" },],() => report("WORKFLOW-PLAN-RESULT", latestTaskResult(request, "revise_plan")),),@@ -74,10 +74,10 @@ const SCENARIOS: Readonly<Record<string, Scenario>> = {request,[{ id: "plan-hold", input: revisionOf(HOLD_REQUEST), name: "revise_plan" },- { id: "plan-hold-wait", input: () => ({ timeoutSeconds: 1 }), name: "task_wait" },- { id: "plan-cancel", input: taskOf("plan-draft"), name: "task_cancel" },+ { id: "plan-hold-wait", input: () => ({ timeoutSeconds: 1 }), name: "eve__task_wait" },+ { id: "plan-cancel", input: taskOf("plan-draft"), name: "eve__task_cancel" },{ id: "plan-final", input: revisionOf("final"), name: "revise_plan" },- { id: "plan-final-wait", name: "task_wait" },+ { id: "plan-final-wait", name: "eve__task_wait" },],() => report("WORKFLOW-PLAN-RESULT", latestTaskResult(request, "revise_plan")),),@@ -101,7 +101,7 @@ const SCENARIOS: Readonly<Record<string, Scenario>> = {playScript(request,[- { id: "signoff-cancel", input: taskOf("signoff"), name: "task_cancel" },+ { id: "signoff-cancel", input: taskOf("signoff"), name: "eve__task_cancel" },{id: "signoff-note",input: (current) => ({@@ -110,7 +110,7 @@ const SCENARIOS: Readonly<Record<string, Scenario>> = {}),name: "sign_off_plan",},- { id: "signoff-note-wait", name: "task_wait" },+ { id: "signoff-note-wait", name: "eve__task_wait" },],() => report("WORKFLOW-SIGNOFF-RESULT", latestTaskResult(request, "sign_off_plan")),),@@ -129,7 +129,7 @@ const SCENARIOS: Readonly<Record<string, Scenario>> = {}),name: "collect_notes",},- { id: "notes-wait", name: "task_wait" },+ { id: "notes-wait", name: "eve__task_wait" },],() =>report(diff --git a/e2e/fixtures/agent-workflow-tools/evals/canary-deploy.cancel.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/canary-deploy.cancel.eval.tsindex e13eb069d..ea1cdc996 100644--- a/e2e/fixtures/agent-workflow-tools/evals/canary-deploy.cancel.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/canary-deploy.cancel.eval.ts@@ -2,17 +2,20 @@ import { defineEval } from "eve/evals";/*** `canary_deploy` is a task that would watch its canary for an hour.- * `task_cancel` stops it at once: the call settles as cancelled, no result is+ * `eve__task_cancel` stops it at once: the call settles as cancelled, no result is* ever delivered, and with nothing left working the turn ends.*/export default defineEval({- description: "task_cancel stops a working task, which then never reports a result.",+ description: "eve__task_cancel stops a working task, which then never reports a result.",async test(t) {const turn = await t.send("WORKFLOW-CANARY-CANCEL");turn.expectOk();turn.event("task.started", { count: 1, data: { callId: "canary", name: "canary_deploy" } });- turn.calledTool("task_cancel", { count: 1, output: /^Stopped \S+; it won't report back\.$/u });+ turn.calledTool("eve__task_cancel", {+ count: 1,+ output: /^Stopped \S+; it won't report back\.$/u,+ });turn.event("task.settled", { count: 1, data: { callId: "canary", status: "cancelled" } });turn.notEvent("task.settled", { data: { status: "completed" } });turn.event("turn.completed", { count: 1 });diff --git a/e2e/fixtures/agent-workflow-tools/evals/collect-notes.reply.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/collect-notes.reply.eval.tsindex 2cbbbccaf..f7ac20409 100644--- a/e2e/fixtures/agent-workflow-tools/evals/collect-notes.reply.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/collect-notes.reply.eval.ts@@ -15,7 +15,7 @@ export default defineEval({const turn = await t.send("WORKFLOW-NOTES-BATCH");turn.expectOk();turn.calledTool("collect_notes", { count: 2 });- turn.calledTool("task_wait", {+ turn.calledTool("eve__task_wait", {count: 1,output: /^collect_notes-\w{6} completed; its result follows\.$/u,});diff --git a/e2e/fixtures/agent-workflow-tools/evals/connection-tools.authorization.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/connection-tools.authorization.eval.tsdeleted file mode 100644index 2bf0657a7..000000000--- a/e2e/fixtures/agent-workflow-tools/evals/connection-tools.authorization.eval.ts+++ /dev/null@@ -1,83 +0,0 @@-import { defineEval } from "eve/evals";-import { z } from "zod";--import { fixtureAuthorizationCallback } from "../agent/lib/fake-service.ts";--const unavailableResult = z.object({- tools: z.array(z.unknown()).length(0),- unavailable: z.array(- z.object({ connection: z.string(), error: z.string(), requiresSignIn: z.literal(true) }),- ),-});-const toolsResult = z.object({ tools: z.array(z.object({ tool: z.string() })) });-const executeResult = z.object({ items: z.array(z.string()) });--export default defineEval({- description:- "A plain connection search reports that private-catalog requires sign-in without prompting; searching it with signIn prompts, then lists and runs its tools over authenticated HTTP.",- timeoutMs: 90_000,-- async test(t) {- // A fresh principal per run: the fixture provider remembers each user's token.- const alice = { "x-eve-forwarded-principal-id": `catalog-alice-${crypto.randomUUID()}` };- const started = await t.send(- "Alice wants to see the items in private-catalog. Search that connection for its items tools, then list the items and report them.",- { headers: alice },- );- const session = started.session;- started.expectOk();- started.calledTool("connection_search", {- count: 1,- input: { connection: "private-catalog" },- output: (value) =>- unavailableResult- .safeParse(value)- .data?.unavailable.some((entry) => entry.connection === "private-catalog") === true,- });- started.event("authorization.required", { count: 1 });- started.notEvent("authorization.completed");- // The sign-in holds the turn; the callback resumes it.- started.event("turn.waiting", { count: 1 });- started.notEvent("session.waiting");-- const required = started.events.find((event) => event.type === "authorization.required");- if (required?.type !== "authorization.required") {- throw new Error("Connection search with signIn did not produce an authorization challenge.");- }- const callback = fixtureAuthorizationCallback(t.target.url, required.data.authorization?.url);- if (session.sessionId === undefined || session.state === undefined) {- throw new Error("Connection search did not create a session.");- }-- const resumed = t.target.watchTurn(session.sessionId, {- startIndex: session.state.streamIndex,- });- const response = await fetch(callback);- if (!response.ok) {- throw new Error(`Authorization callback failed (${response.status}).`);- }-- const completed = await resumed.result();- completed.expectOk();- completed.notEvent("turn.started");- completed.noFailedActions();- completed.notEvent("authorization.required");- completed.event("authorization.completed", {- count: 1,- data: { candidateId: required.data.candidateId, outcome: "authorized" },- });- completed.calledTool("connection_search", {- count: 1,- input: { connection: "private-catalog", signIn: true },- output: (value) =>- toolsResult.safeParse(value).data?.tools.some((entry) => entry.tool === "list_items") ===- true,- });- completed.calledTool("connection_execute", {- count: 1,- output: (value) =>- executeResult.safeParse(value).data?.items.includes("Alice's lamp") === true,- });- completed.messageIncludes("Alice's lamp");- },-});diff --git a/e2e/fixtures/agent-workflow-tools/evals/dynamic-skill-context.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/dynamic-skill-context.eval.tsindex 3bb177aa1..dacb93202 100644--- a/e2e/fixtures/agent-workflow-tools/evals/dynamic-skill-context.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/dynamic-skill-context.eval.ts@@ -52,7 +52,6 @@ export default (["direct", "waiting"] as const).map((mode) =>const started = await send(t.target, threadId, "alice", firstMessage);const initial = await t.target.watchTurn(started.sessionId).result();initial.expectOk();- initial.calledTool("load_skill", { count: 1, status: "completed" });const marker = mode === "direct" ? "Alice's hook audit" : "hook-audit:blocking";const childOutput = readChildOutput(initial, mode);t.check(childOutput.startsWith("WORKFLOW-CHILD:"), equals(true)).label(@@ -68,7 +67,6 @@ export default (["direct", "waiting"] as const).map((mode) =>);const audit = await t.target.watchTurn(resumed.sessionId, { startIndex }).result();audit.expectOk();- audit.calledTool("load_skill", { count: 1, status: "completed" });audit.calledTool("read_dynamic_skill_context", { count: 1, status: "completed" });const output = audit.toolCalls.find((call) => call.name === "read_dynamic_skill_context",diff --git a/e2e/fixtures/agent-workflow-tools/evals/held-turn.root.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/held-turn.root.eval.tsindex 6d4c433e1..de5319d9f 100644--- a/e2e/fixtures/agent-workflow-tools/evals/held-turn.root.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/held-turn.root.eval.ts@@ -32,7 +32,7 @@ export default defineEval({turn.event("turn.started", { count: 1 });turn.event("turn.completed", { count: 1 });turn.eventsSatisfy("the held turn resumes under its own id", staysInOneTurn);- turn.notCalledTool("task_wait");+ turn.notCalledTool("eve__task_wait");t.check(turn.message, includes(/^WORKFLOW-STAGE-RESULT \{.*"plan":"deploy api"/u)).label("the turn's result is the final reply",);diff --git a/e2e/fixtures/agent-workflow-tools/evals/private-catalog.sign-in.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/private-catalog.sign-in.eval.tsnew file mode 100644index 000000000..107ae0962--- /dev/null+++ b/e2e/fixtures/agent-workflow-tools/evals/private-catalog.sign-in.eval.ts@@ -0,0 +1,85 @@+import { SEARCH_TOOL } from "@eve-e2e/config/catalog-tools";+import { defineEval } from "eve/evals";+import { z } from "zod";++import { fixtureAuthorizationCallback } from "../agent/lib/fake-service.ts";++const CATALOG = "private-catalog";+const ITEMS_TOOL = "private-catalog__list_items";++const results = z.object({+ results: z.array(z.object({ description: z.string(), tool: z.string().optional() })),+});+const items = z.object({ items: z.array(z.string()) });++/** The tools a search returned. */+const found = (value: unknown) => results.safeParse(value).data?.results ?? [];++export default defineEval({+ description:+ "A plain eve__search finds private-catalog as its sign-in entry without prompting; running it through eve__tool holds the turn for sign-in, and the resumed turn signs in, then searches the private-catalog__ namespace and calls its tool over authenticated HTTP.",+ timeoutMs: 90_000,++ async test(t) {+ // A fresh principal per run: the fixture provider remembers each user's token.+ const alice = { "x-eve-forwarded-principal-id": `catalog-alice-${crypto.randomUUID()}` };+ const started = await t.send(+ "WORKFLOW-CATALOG-SIGN-IN Alice wants to see the items in her private catalog.",+ { headers: alice },+ );+ const session = started.session;+ started.expectOk();+ started.calledTool(SEARCH_TOOL, {+ count: 1,+ input: { query: CATALOG },+ output: (value) => {+ const [first] = found(value);+ return first?.tool === CATALOG && first.description.startsWith("Sign in to use the");+ },+ });+ started.event("authorization.required", { count: 1, data: { name: CATALOG } });+ started.notEvent("authorization.completed");+ // The sign-in holds the turn; the callback resumes it.+ started.event("turn.waiting", { count: 1 });+ started.notEvent("session.waiting");++ const required = started.events.find((event) => event.type === "authorization.required");+ if (required?.type !== "authorization.required") {+ throw new Error("eve__tool did not produce an authorization challenge.");+ }+ if (session.sessionId === undefined || session.state === undefined) {+ throw new Error("The sign-in turn did not create a session.");+ }+ const resumed = t.target.watchTurn(session.sessionId, {+ startIndex: session.state.streamIndex,+ });+ const response = await fetch(+ fixtureAuthorizationCallback(t.target.url, required.data.authorization?.url),+ );+ if (!response.ok) throw new Error(`Authorization callback failed (${response.status}).`);++ const completed = await resumed.result();+ completed.expectOk();+ completed.notEvent("turn.started");+ completed.noFailedActions();+ completed.notEvent("authorization.required");+ completed.event("authorization.completed", {+ count: 1,+ data: { candidateId: required.data.candidateId, outcome: "authorized" },+ });+ completed.calledTool(CATALOG, {+ count: 1,+ output: (value) => typeof value === "string" && value.startsWith("Signed in to "),+ });+ completed.calledTool(SEARCH_TOOL, {+ count: 1,+ input: { query: `${CATALOG}__` },+ output: (value) => found(value).some((entry) => entry.tool === ITEMS_TOOL),+ });+ completed.calledTool(ITEMS_TOOL, {+ count: 1,+ output: (value) => items.safeParse(value).data?.items.includes("Alice's lamp") === true,+ });+ completed.messageIncludes("Alice's lamp");+ },+});diff --git a/e2e/fixtures/agent-workflow-tools/evals/connection-tools.lazy-authorization.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/public-catalog.lazy-authorization.eval.tssimilarity index 68%rename from e2e/fixtures/agent-workflow-tools/evals/connection-tools.lazy-authorization.eval.tsrename to e2e/fixtures/agent-workflow-tools/evals/public-catalog.lazy-authorization.eval.tsindex 75ab0ccf3..7f938cec9 100644--- a/e2e/fixtures/agent-workflow-tools/evals/connection-tools.lazy-authorization.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/public-catalog.lazy-authorization.eval.ts@@ -6,7 +6,7 @@ const mentions = (text: string) => (value: unknown) => JSON.stringify(value).incexport default defineEval({description:- "An MCP server that lists its tools without a token runs its public tool with no sign-in; its protected tool asks the user to sign in, then resumes over authenticated HTTP.",+ "An MCP server that lists its tools without a token connects and runs its public tool with no sign-in; its protected tool asks the user to sign in, then resumes over authenticated HTTP.",timeoutMs: 90_000,async test(t) {@@ -24,11 +24,30 @@ export default defineEval({);const session = started.session;started.expectOk();+ // Connecting a server that lists its tools anonymously needs no sign-in.+ started.calledTool("public-catalog", {+ count: 1,+ status: "completed",+ output: (value) =>+ value ===+ 'The Public catalog tools are available. Find them with eve__search({ query: "public-catalog__" }).',+ });+ started.eventsSatisfy("no sign-in is requested before the protected call", (events) => {+ const protectedCall = events.findIndex(+ (event) =>+ event.type === "actions.requested" &&+ event.data.actions.some(+ (action) =>+ action.kind === "tool-call" && action.toolName === "public-catalog__list_orders",+ ),+ );+ const signIn = events.findIndex((event) => event.type === "authorization.required");+ return protectedCall >= 0 && signIn > protectedCall;+ });// The public tool runs before anyone signs in.- started.calledTool("connection_execute", {+ started.calledTool("public-catalog__list_items", {count: 1,status: "completed",- input: { connection: "public-catalog", tool: "list_items" },output: mentions("Lamp"),});// Only the protected tool asks for sign-in, and the turn holds on it.@@ -64,9 +83,8 @@ export default defineEval({data: { candidateId: required.data.candidateId, outcome: "authorized" },});// The server answers list_orders only when the request carries the bearer.- completed.calledTool("connection_execute", {+ completed.calledTool("public-catalog__list_orders", {count: 1,- input: { connection: "public-catalog", tool: "list_orders" },output: mentions('"signedIn":true'),});completed.messageIncludes("PUBLIC_CATALOG_DONE");diff --git a/e2e/fixtures/agent-workflow-tools/evals/revise-plan.serve.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/revise-plan.serve.eval.tsindex 3e72128a7..9fa3cccdd 100644--- a/e2e/fixtures/agent-workflow-tools/evals/revise-plan.serve.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/revise-plan.serve.eval.ts@@ -23,8 +23,8 @@ export default defineEval({const revised = await drafted.session.send("WORKFLOW-PLAN-REVISE");revised.expectOk();revised.calledTool("revise_plan", { count: 2, output: /^Sent to task revise_plan-\w{6}\. /u });- revised.calledTool("task_wait", { output: /^Stopped waiting after/u });- revised.calledTool("task_cancel", {+ revised.calledTool("eve__task_wait", { output: /^Stopped waiting after/u });+ revised.calledTool("eve__task_cancel", {count: 1,output: /^Stopped revise_plan-\w{6}'s current work;/u,});diff --git a/e2e/fixtures/agent-workflow-tools/evals/sign-off-plan.cancel.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/sign-off-plan.cancel.eval.tsindex 6a23073d1..e4254b2f2 100644--- a/e2e/fixtures/agent-workflow-tools/evals/sign-off-plan.cancel.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/sign-off-plan.cancel.eval.ts@@ -23,7 +23,7 @@ export default defineEval({const reworked = await parked.session.send(ALICE_REWORK, { turnPolicy: "steer" });reworked.expectOk();- reworked.calledTool("task_cancel", {+ reworked.calledTool("eve__task_cancel", {count: 1,output: /^Stopped sign_off_plan-\w{6}'s current work;/u,});diff --git a/e2e/fixtures/agent-workflow-tools/evals/stage-deploy.task.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/stage-deploy.task.eval.tsindex b3caea782..2d4fa7eba 100644--- a/e2e/fixtures/agent-workflow-tools/evals/stage-deploy.task.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/stage-deploy.task.eval.ts@@ -1,13 +1,14 @@import { defineEval } from "eve/evals";/**- * `stage_deploy` defines `task()`. Its call returns a receipt, `task_wait`+ * `stage_deploy` defines `task()`. Its call returns a receipt, `eve__task_wait`* parks the turn with `turn.waiting` until the task settles, and clients read* the outcome from `task.settled` while the result reaches the model before* the final reply.*/export default defineEval({- description: "A task tool starts a task, task_wait waits on it, and its result is delivered.",+ description:+ "A task tool starts a task, eve__task_wait waits on it, and its result is delivered.",async test(t) {const turn = await t.send("WORKFLOW-STAGE-WAIT");turn.expectOk();@@ -17,7 +18,7 @@ export default defineEval({output: /^Started task stage_deploy-\w{6}\. Its result will arrive/u,});turn.event("task.started", { count: 1, data: { callId: "stage", name: "stage_deploy" } });- turn.calledTool("task_wait", { count: 1, output: /stage_deploy-\w{6} completed/u });+ turn.calledTool("eve__task_wait", { count: 1, output: /stage_deploy-\w{6} completed/u });turn.event("task.settled", {count: 1,data: { callId: "stage", output: { plan: "deploy api" }, status: "completed" },diff --git a/e2e/fixtures/agent-workflow-tools/evals/subagent-hook-context.eval.ts b/e2e/fixtures/agent-workflow-tools/evals/subagent-hook-context.eval.tsindex 767da1af6..9fb214460 100644--- a/e2e/fixtures/agent-workflow-tools/evals/subagent-hook-context.eval.ts+++ b/e2e/fixtures/agent-workflow-tools/evals/subagent-hook-context.eval.ts@@ -36,26 +36,21 @@ export default (["direct", "background"] as const).map((mode) =>const scenario = SCENARIOS[mode];const initial = await t.send(scenario.request);initial.expectOk();- initial.calledTool("load_skill", { count: 1, status: "completed" });initial.messageIncludes("Alice's hook audit");if (mode === "background") {initial.event("message.completed", {count: 1,data: { message: RESEARCH_INTERIM_MESSAGE },});- initial.notCalledTool("task_wait");+ initial.notCalledTool("eve__task_wait");}const audit = await initial.session.send(scenario.audit);audit.expectOk();- audit.calledTool("load_skill", { count: 1, status: "completed" });- t.eventsSatisfy("the parent's dynamic skill remains loadable after delegation", () =>- [initial, audit].every((turn) =>- String(turn.toolCalls.find((call) => call.name === "load_skill")?.output).includes(- "DELEGATION-POLICY:",- ),- ),- );+ // The parent's dynamic skill stays loadable after delegation.+ for (const turn of [initial, audit]) {+ turn.loadedSkill("delegation-policy", { count: 1, output: /DELEGATION-POLICY:/u });+ }audit.calledTool("read_subagent_hooks", { count: 1, status: "completed" });const records = readHookAudit(audit);t.eventsSatisfy(diff --git a/e2e/fixtures/e2e-config/package.json b/e2e/fixtures/e2e-config/package.jsonindex c61cc8721..9450ef25e 100644--- a/e2e/fixtures/e2e-config/package.json+++ b/e2e/fixtures/e2e-config/package.json@@ -5,6 +5,8 @@"type": "module","exports": {".": "./src/index.ts",+ "./cache-checks": "./src/cache-checks.ts",+ "./catalog-tools": "./src/catalog-tools.ts","./instrumentation": "./src/instrumentation.ts","./instrumentation-otel": "./src/instrumentation-otel.ts","./mock-script": "./src/mock-script.ts"diff --git a/e2e/fixtures/agent-prompt-cache/cache-checks.ts b/e2e/fixtures/e2e-config/src/cache-checks.tssimilarity index 100%rename from e2e/fixtures/agent-prompt-cache/cache-checks.tsrename to e2e/fixtures/e2e-config/src/cache-checks.tsdiff --git a/e2e/fixtures/e2e-config/src/catalog-tools.ts b/e2e/fixtures/e2e-config/src/catalog-tools.tsnew file mode 100644index 000000000..9aafa3f6a--- /dev/null+++ b/e2e/fixtures/e2e-config/src/catalog-tools.ts@@ -0,0 +1,14 @@+// The names eve's built-in tools ship under. eve doesn't export them, so the+// fixtures' scripts and evals share these to stay on the shipped names.++/** Searches the agent's own catalog of deferred tools, agents, and skills. */+export const SEARCH_TOOL = "eve__search";++/** Calls a deferred tool, agent, or connection tool by name. */+export const CALL_TOOL = "eve__tool";++/** Loads a skill by name. */+export const SKILL_TOOL = "eve__skill";++/** Waits for running tasks. */+export const TASK_WAIT_TOOL = "eve__task_wait";diff --git a/e2e/fixtures/e2e-config/src/mock-script.ts b/e2e/fixtures/e2e-config/src/mock-script.tsindex 0e0eb7f9a..ce7e3686f 100644--- a/e2e/fixtures/e2e-config/src/mock-script.ts+++ b/e2e/fixtures/e2e-config/src/mock-script.ts@@ -1,6 +1,8 @@// Helpers for scripted mock responders: a scenario is a fixed sequence of tool// calls with stable ids, played one model step at a time.-import type { MockModelRequest, MockModelResponse } from "eve/evals";+import type { EveEvalContext, MockModelRequest, MockModelResponse } from "eve/evals";++import { CALL_TOOL, SKILL_TOOL } from "./catalog-tools";/** One tool call of a scripted scenario. Its fixed id marks it as done once its result is in the prompt. */export interface ScriptedCall {@@ -24,6 +26,37 @@ export function playScript(return { toolCalls: [{ id: next.id, input: next.input?.(request) ?? {}, name: next.name }] };}+/** A scripted `eve__tool` call: a catalog tool by name, with its input. */+export function callTool(id: string, name: string, input?: object): ScriptedCall {+ return { id, input: () => (input === undefined ? { name } : { input, name }), name: CALL_TOOL };+}++/** A scripted `eve__skill` call that loads the named skill. */+export function loadSkill(id: string, name: string): ScriptedCall {+ return { id, input: () => ({ name }), name: SKILL_TOOL };+}++/** Loads each named skill with `eve__skill({ name })`, then replies with each skill's last line. */+export function loadSkills(+ request: MockModelRequest,+ names: readonly string[],+): MockModelResponse | string {+ return playScript(+ request,+ names.map((skill) => loadSkill(`load-${skill}`, skill)),+ (finished) =>+ names.map((skill) => outputOf(finished, `load-${skill}`).trim().split("\n").at(-1)).join(" "),+ );+}++/** Skips an eval whose scripted calls only the deterministic mock model makes. */+export function requireMockModel(+ t: EveEvalContext,+ reason = "Requires the deterministic mock model to issue the exact calls.",+): void {+ if (process.env.EVE_E2E_MODEL !== "mock") t.skip(reason);+}+/** The output of the call with `id`, as the model sees it. */export function outputOf(request: MockModelRequest, id: string): string {const result = request.toolResults.find((entry) => entry.id === id);diff --git a/e2e/fixtures/extensions/agent/mock-responder.ts b/e2e/fixtures/extensions/agent/mock-responder.tsindex 37616e45e..3be840966 100644--- a/e2e/fixtures/extensions/agent/mock-responder.ts+++ b/e2e/fixtures/extensions/agent/mock-responder.ts@@ -1,11 +1,17 @@+import { loadSkills } from "@eve-e2e/config/mock-script";import type { MockModelRequest, MockModelResponse } from "eve/evals";const GIZMO_INSTRUCTIONS_TOKEN = "gizmo-instructions-ok-7K2M";const JAVASCRIPT_INSTRUCTIONS_TOKEN = "javascript-instructions-ok-9P4R";const LAYOUT_TOOL = "gizmo__gizmo_layout";+const SKILL_LOAD_DIRECTIVE = "SKILL-LOAD";+export function respond(request: MockModelRequest): MockModelResponse | string {const message = request.lastUserMessage ?? "";+ if (message.startsWith(SKILL_LOAD_DIRECTIVE)) {+ return loadSkills(request, message.slice(SKILL_LOAD_DIRECTIVE.length).trim().split(/\s+/u));+ }if (message.includes("Report both extension instruction tokens")) {const instructions = request.messages.filter((entry) => entry.role === "system")diff --git a/e2e/fixtures/extensions/evals/dynamic-skill.eval.ts b/e2e/fixtures/extensions/evals/dynamic-skill.eval.tsdeleted file mode 100644index 177bacde3..000000000--- a/e2e/fixtures/extensions/evals/dynamic-skill.eval.ts+++ /dev/null@@ -1,26 +0,0 @@-import { defineEval } from "eve/evals";--const TOOLKIT_INCIDENT_TOKEN = "toolkit-incident-dynamic-ok-7T2X";--/**- * A map-produced dynamic skill authored in an extension must be namespaced by- * the mount (`toolkit__incident`), not exposed under its bare map key- * (`incident`). Loading it by the namespaced name proves the mount prefix- * reaches map entries the way it already does for the extension's static and- * dynamic tools.- */-export default defineEval({- tags: ["real-model"],- description: "Map-produced dynamic skill from an extension is namespaced (toolkit__incident).",- async test(t) {- await t.send(- "Call `load_skill` exactly once with the skill `toolkit__incident`. " +- "After it succeeds, reply with the verification token from the loaded skill. " +- "Do not ask any follow-up questions or call any other tools.",- );-- t.succeeded();- t.loadedSkill("toolkit__incident", { output: new RegExp(TOOLKIT_INCIDENT_TOKEN, "u") });- t.messageIncludes(TOOLKIT_INCIDENT_TOKEN);- },-});diff --git a/e2e/fixtures/extensions/evals/extension-skills.eval.ts b/e2e/fixtures/extensions/evals/extension-skills.eval.tsnew file mode 100644index 000000000..65eb6a408--- /dev/null+++ b/e2e/fixtures/extensions/evals/extension-skills.eval.ts@@ -0,0 +1,32 @@+import { requireMockModel } from "@eve-e2e/config/mock-script";+import { defineEval } from "eve/evals";++const TOOLKIT_INCIDENT_TOKEN = "toolkit-incident-dynamic-ok-7T2X";++/**+ * Extension skills load through eve__skill({ name }) under their mount prefix:+ * a packaged static skill, and a dynamic map entry that composes under each+ * mount (`toolkit__incident`, `toolkit-alt__incident`) instead of its bare key.+ */+export default defineEval({+ description: "eve__skill({ name }) loads static and dynamic extension skills under their mounts.",+ async test(t) {+ requireMockModel(t);++ const turn = await t.send(+ "SKILL-LOAD toolkit__toolkit-guide toolkit__incident toolkit-alt__incident local-guide",+ );++ turn.expectOk();+ turn.noFailedActions();+ t.loadedSkill("toolkit__toolkit-guide", { count: 1, output: /# Toolkit triage/u });+ for (const mount of ["toolkit", "toolkit-alt"]) {+ t.loadedSkill(`${mount}__incident`, {+ count: 1,+ output: new RegExp(TOOLKIT_INCIDENT_TOKEN, "u"),+ });+ }+ t.loadedSkill("local-guide", { count: 1, output: /belongs to the consuming agent/u });+ t.messageIncludes(`${TOOLKIT_INCIDENT_TOKEN} ${TOOLKIT_INCIDENT_TOKEN}`);+ },+});diff --git a/packages/eve-code/extension/subagents/worker/tools/load_skill.ts b/packages/eve-code/extension/subagents/worker/tools/load_skill.tsdeleted file mode 100644index 04bd05447..000000000--- a/packages/eve-code/extension/subagents/worker/tools/load_skill.ts+++ /dev/null@@ -1,3 +0,0 @@-import { disableTool } from "eve/tools";--export default disableTool();diff --git a/packages/eve-code/test/scenario/worker-manifest.test.ts b/packages/eve-code/test/scenario/worker-manifest.test.tsindex 0ff7023ec..b09a52351 100644--- a/packages/eve-code/test/scenario/worker-manifest.test.ts+++ b/packages/eve-code/test/scenario/worker-manifest.test.ts@@ -71,10 +71,6 @@ test("built worker keeps explicit read tools without shell or write capabilitiesassert.deepEqual(toolNames, ["glob", "grep", "read_file"]);for (const tool of worker.agent.tools) assert.equal(tool.hasExecute, true);assert.deepEqual(worker.agent.connections, []);- assert.deepEqual(- worker.agent.dynamicTools.map((tool) => tool.slug),- ["connection_tools"],- );} finally {await rm(root, { recursive: true, force: true });}diff --git a/packages/eve/extension-contracts/compatibility/channel/v54.ts b/packages/eve/extension-contracts/compatibility/channel/v54.tsnew file mode 100644index 000000000..b1fce1c66--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/channel/v54.ts@@ -0,0 +1,15 @@+import { defineChannel } from "#public/channels/index.js";++// Epoch 54 tool-call action requests could carry `parentCallId` for nested+// actions; epoch 55 drops it, since connection tools no longer report them.+export default defineChannel({+ routes: [],+ events: {+ "actions.requested"(event) {+ for (const action of event.actions) {+ if (action.kind !== "tool-call") continue;+ console.info("tool requested", { callId: action.callId, toolName: action.toolName });+ }+ },+ },+});diff --git a/packages/eve/extension-contracts/compatibility/connection/v43.ts b/packages/eve/extension-contracts/compatibility/connection/v43.tsnew file mode 100644index 000000000..7616554f3--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/connection/v43.ts@@ -0,0 +1,11 @@+import { defineMcpClientConnection } from "#public/connections/index.js";++// Epoch 43 tool-call action requests could carry `parentCallId` for nested+// actions; epoch 44 drops it, since connection tools no longer report them.+export default defineMcpClientConnection({+ description: "Search the incident tracker.",+ url: "https://incidents.example.com/mcp",+ approval({ toolName }) {+ return toolName.endsWith("__search") ? "approved" : "user-approval";+ },+});diff --git a/packages/eve/extension-contracts/compatibility/dynamicInstructions/v34.ts b/packages/eve/extension-contracts/compatibility/dynamicInstructions/v34.tsnew file mode 100644index 000000000..28ac09cde--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/dynamicInstructions/v34.ts@@ -0,0 +1,12 @@+import { defineDynamic, defineInstructions } from "#public/instructions/index.js";++// Epoch 34 tool-call action requests could carry `parentCallId` for nested+// actions; epoch 35 drops it, since connection tools no longer report them.+export default defineDynamic({+ events: {+ "turn.started": (_event, ctx) =>+ defineInstructions({+ markdown: `Triage the incident reported in session ${ctx.session.id}.`,+ }),+ },+});diff --git a/packages/eve/extension-contracts/compatibility/dynamicSkill/v33.ts b/packages/eve/extension-contracts/compatibility/dynamicSkill/v33.tsnew file mode 100644index 000000000..4019d67e4--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/dynamicSkill/v33.ts@@ -0,0 +1,13 @@+import { defineDynamic, defineSkill } from "#public/skills/index.js";++// Epoch 33 tool-call action requests could carry `parentCallId` for nested+// actions; epoch 34 drops it, since connection tools no longer report them.+export default defineDynamic({+ events: {+ "turn.started": (_event, ctx) =>+ defineSkill({+ description: "Triage the active incident.",+ markdown: `Triage the incident reported in session ${ctx.session.id}.`,+ }),+ },+});diff --git a/packages/eve/extension-contracts/compatibility/dynamicTool/v75.ts b/packages/eve/extension-contracts/compatibility/dynamicTool/v75.tsnew file mode 100644index 000000000..be710f4da--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/dynamicTool/v75.ts@@ -0,0 +1,14 @@+import { defineDynamic, defineTool } from "#public/tools/index.js";++// Epoch 75 dynamic tool entries had no `deferred`; epoch 76 adds it as optional.+export default defineDynamic({+ events: {+ "turn.started": (_event, ctx) => ({+ session: defineTool({+ description: "Return the active session identifier.",+ inputSchema: { type: "object", properties: {} },+ execute: () => ({ sessionId: ctx.session.id }),+ }),+ }),+ },+});diff --git a/packages/eve/extension-contracts/compatibility/hook/v41.ts b/packages/eve/extension-contracts/compatibility/hook/v41.tsnew file mode 100644index 000000000..7922bd388--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/hook/v41.ts@@ -0,0 +1,15 @@+import { defineHook } from "#public/hooks/index.js";++// Epoch 41 tool-call action requests could carry `parentCallId` for nested+// actions; epoch 42 drops it, since connection tools no longer report them.+// Hooks that read tool calls keep working.+export default defineHook({+ events: {+ "actions.requested"(event) {+ for (const action of event.data.actions) {+ if (action.kind !== "tool-call") continue;+ console.info("tool requested", { callId: action.callId, toolName: action.toolName });+ }+ },+ },+});diff --git a/packages/eve/extension-contracts/compatibility/schedule/v27.ts b/packages/eve/extension-contracts/compatibility/schedule/v27.tsnew file mode 100644index 000000000..fcb2ed408--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/schedule/v27.ts@@ -0,0 +1,23 @@+import { defineChannel } from "#public/channels/index.js";+import { defineSchedule } from "#public/schedules/index.js";++// Epoch 27 tool-call action requests could carry `parentCallId` for nested+// actions; epoch 28 drops it, since connection tools no longer report them.+const reports = defineChannel({+ routes: [],+ events: {+ "actions.requested"(event) {+ console.info("report actions", { count: event.actions.length });+ },+ },+ receive(input, { from }) {+ return from("nightly-report").send(input.message, { auth: input.auth });+ },+});++export default defineSchedule({+ cron: "0 2 * * *",+ async run({ to, appAuth, waitUntil }) {+ waitUntil(to(reports, {}).send("Prepare the nightly report.", { auth: appAuth }));+ },+});diff --git a/packages/eve/extension-contracts/compatibility/skill/v2.ts b/packages/eve/extension-contracts/compatibility/skill/v2.tsnew file mode 100644index 000000000..007269b37--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/skill/v2.ts@@ -0,0 +1,7 @@+import { defineSkill } from "#public/skills/index.js";++export default defineSkill({+ description: "Follow the incident response checklist.",+ markdown: "# Incident response\n\nRead `references/checklist.md` before paging anyone.\n",+ files: { "references/checklist.md": "1. Confirm impact.\n2. Page the owner.\n" },+});diff --git a/packages/eve/extension-contracts/compatibility/subagent/v31.ts b/packages/eve/extension-contracts/compatibility/subagent/v31.tsnew file mode 100644index 000000000..ca5086198--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/subagent/v31.ts@@ -0,0 +1,8 @@+import { defineAgent } from "#public/index.js";++// Epoch 31 `tool` accepted only a boolean; epoch 32 also accepts `"deferred"`.+export default defineAgent({+ description: "Research the incidents behind a metric change.",+ model: "openai/gpt-5.6-sol",+ tool: false,+});diff --git a/packages/eve/extension-contracts/compatibility/tool/v80.ts b/packages/eve/extension-contracts/compatibility/tool/v80.tsnew file mode 100644index 000000000..0023de936--- /dev/null+++ b/packages/eve/extension-contracts/compatibility/tool/v80.ts@@ -0,0 +1,11 @@+import { z } from "zod";+import { defineTool } from "#public/tools/index.js";++// Epoch 80 tools had no `deferred`; epoch 81 adds it as optional.+// Tools that leave it unset stay in the model's tool list.+export default defineTool({+ description: "Summarize the open incidents for a service.",+ inputSchema: z.object({ service: z.string() }),+ outputSchema: z.object({ service: z.string(), summary: z.string() }),+ execute: ({ service }) => ({ service, summary: `No open incidents for ${service}.` }),+});diff --git a/packages/eve/extension-contracts/reports/channel/v55.json b/packages/eve/extension-contracts/reports/channel/v55.jsonnew file mode 100644index 000000000..e0fde820f--- /dev/null+++ b/packages/eve/extension-contracts/reports/channel/v55.json@@ -0,0 +1,21 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "channel",+ "epoch": 55,+ "sha256": "d9e254fe08e47dbac7d9751c6099615eac151fe8ff90301280cb6013003d3d22",+ "exports": [+ "DELETE",+ "GET",+ "HEAD",+ "OPTIONS",+ "PATCH",+ "POST",+ "PUT",+ "WS",+ "createWebSocketUpgradeServer",+ "defineChannel",+ "disableRoute",+ "isChannel",+ "isDisabledRouteSentinel"+ ]+}diff --git a/packages/eve/extension-contracts/reports/connection/v44.json b/packages/eve/extension-contracts/reports/connection/v44.jsonnew file mode 100644index 000000000..6e17104f2--- /dev/null+++ b/packages/eve/extension-contracts/reports/connection/v44.json@@ -0,0 +1,16 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "connection",+ "epoch": 44,+ "sha256": "70ab219d89758b026bdfbdc1d58c1f31b9b77f4292ccfabfee49a8ae008a30cd",+ "exports": [+ "ConnectionAuthorizationFailedError",+ "ConnectionAuthorizationRequiredError",+ "defineDynamic",+ "defineInteractiveAuthorization",+ "defineMcpClientConnection",+ "defineOpenAPIConnection",+ "isConnectionAuthorizationFailedError",+ "isConnectionAuthorizationRequiredError"+ ]+}diff --git a/packages/eve/extension-contracts/reports/dynamicInstructions/v35.json b/packages/eve/extension-contracts/reports/dynamicInstructions/v35.jsonnew file mode 100644index 000000000..3835ce4c7--- /dev/null+++ b/packages/eve/extension-contracts/reports/dynamicInstructions/v35.json@@ -0,0 +1,7 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "dynamicInstructions",+ "epoch": 35,+ "sha256": "4d51d92d22246dd516829a816b4b7783cad9c8e09f106e64c49e8ec045bacc05",+ "exports": ["defineDynamic"]+}diff --git a/packages/eve/extension-contracts/reports/dynamicSkill/v34.json b/packages/eve/extension-contracts/reports/dynamicSkill/v34.jsonnew file mode 100644index 000000000..62c088474--- /dev/null+++ b/packages/eve/extension-contracts/reports/dynamicSkill/v34.json@@ -0,0 +1,7 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "dynamicSkill",+ "epoch": 34,+ "sha256": "80dd677e56bd308bcff35d3997d5a6b6996edab93344d8e65e7e41bdbd60cdba",+ "exports": ["defineDynamic"]+}diff --git a/packages/eve/extension-contracts/reports/dynamicTool/v76.json b/packages/eve/extension-contracts/reports/dynamicTool/v76.jsonnew file mode 100644index 000000000..18bd2e938--- /dev/null+++ b/packages/eve/extension-contracts/reports/dynamicTool/v76.json@@ -0,0 +1,16 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "dynamicTool",+ "epoch": 76,+ "sha256": "43682f6a101028ba64ae13fc2c5a9575f9394b039479395bfbef8d44b254159b",+ "exports": [+ "DynamicToolEntry",+ "DynamicToolEvents",+ "DynamicToolResult",+ "DynamicToolSet",+ "auto",+ "defineDurableCallback",+ "defineDurableSchema",+ "defineDynamic"+ ]+}diff --git a/packages/eve/extension-contracts/reports/hook/v42.json b/packages/eve/extension-contracts/reports/hook/v42.jsonnew file mode 100644index 000000000..c927da10a--- /dev/null+++ b/packages/eve/extension-contracts/reports/hook/v42.json@@ -0,0 +1,7 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "hook",+ "epoch": 42,+ "sha256": "73183c48614dc9ad29f1d9057952a5e8799da1fc04cabfc773b4fb9772189326",+ "exports": ["defineHook"]+}diff --git a/packages/eve/extension-contracts/reports/schedule/v28.json b/packages/eve/extension-contracts/reports/schedule/v28.jsonnew file mode 100644index 000000000..3d1969932--- /dev/null+++ b/packages/eve/extension-contracts/reports/schedule/v28.json@@ -0,0 +1,15 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "schedule",+ "epoch": 28,+ "sha256": "b4290c7b7d95b19c7a579fd73eac0dc5413db3d59e0249eda198901edfbdd930",+ "exports": [+ "ScheduleDefinition",+ "ScheduleHandlerArgs",+ "ScheduleRunHandler",+ "ScheduleToFn",+ "TypedReceiveTarget",+ "defineSchedule",+ "isScheduleAuth"+ ]+}diff --git a/packages/eve/extension-contracts/reports/skill/v3.json b/packages/eve/extension-contracts/reports/skill/v3.jsonnew file mode 100644index 000000000..f60f3c64e--- /dev/null+++ b/packages/eve/extension-contracts/reports/skill/v3.json@@ -0,0 +1,7 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "skill",+ "epoch": 3,+ "sha256": "e8b139bace8199513ced8e438c9031f73b3b4dbd7faf6e0aff79e32e0e061cf7",+ "exports": ["NamedSkillDefinition", "defineSkill"]+}diff --git a/packages/eve/extension-contracts/reports/subagent/v32.json b/packages/eve/extension-contracts/reports/subagent/v32.jsonnew file mode 100644index 000000000..ae9a2da45--- /dev/null+++ b/packages/eve/extension-contracts/reports/subagent/v32.json@@ -0,0 +1,23 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "subagent",+ "epoch": 32,+ "sha256": "c82d7c360c8e8f75b2b71621869412fdaa69a92d604ad6f8f5f75d9810c6a66a",+ "exports": [+ "AgentCompactionDefinition",+ "AgentDefinition",+ "AgentModelDefinition",+ "AgentStaticModelDefinition",+ "DefinedAgent",+ "DynamicLocalSubagentDefinition",+ "DynamicSubagentDefinition",+ "RemoteAgentDefinition",+ "RemoteAgentDefinitionInput",+ "WorkspaceAgentDefinition",+ "WorkspaceAgentTransport",+ "defineAgent",+ "defineDynamic",+ "defineRemoteAgent",+ "defineWorkspaceAgent"+ ]+}diff --git a/packages/eve/extension-contracts/reports/tool/v81.json b/packages/eve/extension-contracts/reports/tool/v81.jsonnew file mode 100644index 000000000..4e73a6f47--- /dev/null+++ b/packages/eve/extension-contracts/reports/tool/v81.json@@ -0,0 +1,29 @@+{+ "kind": "eve-extension-capability-contract",+ "capability": "tool",+ "epoch": 81,+ "sha256": "c86eab0b5768943b32d54198742ab1e3ca1919ae877468b7f9b000f3cebe3000",+ "exports": [+ "AgentRouterInput",+ "AgentRouterTool",+ "WorkflowStepToolContext",+ "WorkflowTool",+ "WorkflowToolDefinition",+ "WorkflowToolInput",+ "WorkflowToolOptions",+ "agentRouter",+ "decide",+ "defaultWebSearch",+ "defineTool",+ "defineWorkflowTool",+ "disableTool",+ "isDisabledToolSentinel",+ "isWebSearchToolDefinition",+ "serializeModelInputSchema",+ "toolOutput",+ "toolOutputPart",+ "toolResultFrom",+ "webSearch",+ "workflow"+ ]+}diff --git a/packages/eve/package.json b/packages/eve/package.jsonindex 804510c43..59b810313 100644--- a/packages/eve/package.json+++ b/packages/eve/package.json@@ -182,11 +182,6 @@"import": "./dist/src/public/tools/web-search.js","default": "./dist/src/public/tools/web-search.js"},- "./tools/load_skill": {- "types": "./dist/src/public/tools/load-skill.d.ts",- "import": "./dist/src/public/tools/load-skill.js",- "default": "./dist/src/public/tools/load-skill.js"- },"./tools/glob": {"types": "./dist/src/public/tools/glob.d.ts","import": "./dist/src/public/tools/glob.js",diff --git a/packages/eve/src/acp/adapter.test.ts b/packages/eve/src/acp/adapter.test.tsindex f2304eb56..cca5cb1cf 100644--- a/packages/eve/src/acp/adapter.test.ts+++ b/packages/eve/src/acp/adapter.test.ts@@ -161,6 +161,7 @@ describe("EveAcpAdapter", () => {workflowId: "publish-workflow",},],+ presentation: { "call-1": { label: "Check the weather in SF" } },sequence: 3,stepIndex: 0,turnId: "t1",@@ -169,6 +170,7 @@ describe("EveAcpAdapter", () => {{type: "action.result",data: {+ presentation: { "call-1": { label: "Sunny in SF" } },result: {callId: "call-1",kind: "tool-result",@@ -206,7 +208,12 @@ describe("EveAcpAdapter", () => {expect(notifications.slice(0, 2).map(({ params }) => (params as any).update.messageId)).toEqual(["t1:message:0", "t1:thought:0"],);- expect((notifications[3]!.params as any).update.title).toBe("publish");+ // A call reads as its label, or a readable form of its name, and a result can retitle it.+ expect(notifications.slice(2).map(({ params }) => (params as any).update.title)).toEqual([+ "Check the weather in SF",+ "Publish",+ "Sunny in SF",+ ]);expect(JSON.stringify(notifications)).not.toContain("secret");});diff --git a/packages/eve/src/acp/adapter.ts b/packages/eve/src/acp/adapter.tsindex 019a1d461..4312184e3 100644--- a/packages/eve/src/acp/adapter.ts+++ b/packages/eve/src/acp/adapter.ts@@ -18,7 +18,7 @@ import {} from "#compiled/@agentclientprotocol/sdk/index.js";import { Client, ClientError } from "#client/index.js";import type { ClientOptions, SendTurnInput, SendTurnPayload } from "#client/types.js";-import type { HandleMessageStreamEvent } from "#protocol/message.js";+import type { ActionPresentationByCallId, HandleMessageStreamEvent } from "#protocol/message.js";import {callStatus,foldSession,@@ -26,6 +26,7 @@ import {type SessionCallStatus,type SessionProjection,} from "#protocol/session-projection.js";+import { actionLabel, visibleActions } from "#shared/action-label.js";import type { RuntimeActionRequest, RuntimeActionResult } from "#shared/action-types.js";import type { InputRequest, InputResponse } from "#shared/input.js";@@ -378,6 +379,7 @@ export class EveAcpAdapter {toolCallId: callId,status: ACP_TOOL_STATUS[status],...callOutput(event, callId),+ ...resultTitle(event, callId),});}}@@ -399,6 +401,7 @@ export class EveAcpAdapter {toolCallId: resultCallId,status: ACP_TOOL_STATUS[status],...output,+ ...resultTitle(event, resultCallId),});}@@ -443,8 +446,8 @@ export class EveAcpAdapter {});return;case "actions.requested":- for (const action of event.data.actions) {- const toolCall = toolCallForAction(action);+ for (const action of visibleActions(event.data.actions)) {+ const toolCall = toolCallForAction(action, event.data.presentation);session.tools.set(action.callId, toolCall);await notifyUpdate(client, sessionId, { sessionUpdate: "tool_call", ...toolCall });}@@ -571,16 +574,13 @@ function promptContent(params: PromptRequest): Array<{ type: "text"; text: strinreturn content;}-function toolCallForAction(action: RuntimeActionRequest): ToolCall {- const title =- action.kind === "tool-call" || action.kind === "workflow-tool-call"- ? action.toolName- : action.kind === "load-skill"- ? "Load skill"- : action.name;+function toolCallForAction(+ action: RuntimeActionRequest,+ presentation?: ActionPresentationByCallId,+): ToolCall {return {toolCallId: action.callId,- title,+ title: actionLabel(action, presentation),kind: "other",status: "pending",rawInput: action.input,@@ -642,6 +642,13 @@ function errorMessage(error: unknown): string {return error instanceof Error ? error.message : String(error);}+/** A result's completion label retitles its call, such as a search reporting what it found. */+function resultTitle(event: HandleMessageStreamEvent, callId: string): { title?: string } {+ const label =+ event.type === "action.result" ? event.data.presentation?.[callId]?.label : undefined;+ return label === undefined ? {} : { title: label };+}+/** The output an event carries for a call, if it reports one. */function callOutput(event: HandleMessageStreamEvent, callId: string) {const output =diff --git a/packages/eve/src/channel/agent-description.test.ts b/packages/eve/src/channel/agent-description.test.tsindex 5f35dd8d7..ec4edfe3d 100644--- a/packages/eve/src/channel/agent-description.test.ts+++ b/packages/eve/src/channel/agent-description.test.ts@@ -27,7 +27,7 @@ it("describes only the tools invokeTool can run, sorted by name", () => {bindings: {"source:deploy": { owner: { kind: "application" } },"source:lookup": { owner: { kind: "extension" } },- "source:load_skill": { owner: { feature: "skills", kind: "framework" } },+ "source:web_fetch": { owner: { feature: "eve:defaults", kind: "framework" } },"source:plan": { owner: { kind: "application" } },"source:remote": { owner: { kind: "application" } },},@@ -39,7 +39,7 @@ it("describes only the tools invokeTool can run, sorted by name", () => {tools: [tool("lookup", { outputSchema: { type: "object" } }),tool("deploy", { requiresApproval: true }),- tool("load_skill"),+ tool("web_fetch"),tool("plan", { behavior: { handling: { kind: "workflow-tool" } } }),tool("remote", { hasExecute: false }),],diff --git a/packages/eve/src/channel/task-card.test.ts b/packages/eve/src/channel/task-card.test.tsdeleted file mode 100644index 17bf402d1..000000000--- a/packages/eve/src/channel/task-card.test.ts+++ /dev/null@@ -1,67 +0,0 @@-import { describe, expect, it } from "vitest";--import { createActionResultEvent, type UnstampedMessageStreamEvent } from "#protocol/message.js";--import { taskCardView, trackTaskCardEvent, type TaskCardTurn } from "./task-card.js";--const AT = "2026-09-30T00:00:00.000Z";--function fold(events: readonly UnstampedMessageStreamEvent[]): TaskCardTurn {- let turns: Readonly<Record<string, TaskCardTurn>> = {};- for (const event of events) turns = { ...turns, ...trackTaskCardEvent(turns, event, AT) };- return turns["turn-1"]!;-}--describe("task card", () => {- it("shows a connection call as one row, not also its nested connection tool", () => {- const turn = fold([- {- type: "actions.requested",- data: {- turnId: "turn-1",- sequence: 0,- stepIndex: 0,- actions: [- {- kind: "tool-call",- callId: "call-1",- toolName: "connection_execute",- input: { connection: "kennel", tool: "book_visit", input: {} },- },- ],- presentation: { "call-1": { label: "kennel__book_visit" } },- },- },- {- type: "actions.requested",- data: {- turnId: "turn-1",- sequence: 1,- stepIndex: 0,- actions: [- {- kind: "tool-call",- callId: "call-1:nested",- parentCallId: "call-1",- toolName: "kennel__book_visit",- input: {},- },- ],- },- },- ...["call-1:nested", "call-1"].map((callId, index) =>- createActionResultEvent({- result: { kind: "tool-result", callId, toolName: "connection_execute", output: {} },- sequence: 2 + index,- stepIndex: 0,- turnId: "turn-1",- }),- ),- ]);-- const { actions } = taskCardView("turn-1", turn, { audience: "public" });- expect(actions).toHaveLength(1);- expect(actions[0]).toMatchObject({ id: "call-1", status: "completed" });- expect(JSON.stringify(actions[0])).toContain("kennel__book_visit");- });-});diff --git a/packages/eve/src/channel/task-card.ts b/packages/eve/src/channel/task-card.tsindex b8eabdfc4..9397c81e4 100644--- a/packages/eve/src/channel/task-card.ts+++ b/packages/eve/src/channel/task-card.ts@@ -1,5 +1,9 @@-import { isTaskControlTool } from "#protocol/task-tools.js";-import type { TaskCancelReason, UnstampedMessageStreamEvent } from "#protocol/message.js";+import type {+ ActionPresentationByCallId,+ TaskCancelReason,+ UnstampedMessageStreamEvent,+} from "#protocol/message.js";+import { actionLabel, visibleActions } from "#shared/action-label.js";import { actionRequestName } from "#shared/action-request-name.js";import type { RuntimeActionRequest } from "#shared/action-types.js";import type { ChannelAudience } from "#shared/channel-audience.js";@@ -213,20 +217,11 @@ function trackTurnEvent(const { turnId } = event.data;const current = turns[turnId] ?? { calls: [], ended: false };const known = new Set(current.calls.map((call) => call.callId));- // Nested actions (such as a connection tool run by connection_execute)- // already appear as their parent call's row.- const requested = event.data.actions.filter(- (action) =>- !known.has(action.callId) &&- !(- action.kind === "tool-call" &&- (isTaskControlTool(action.toolName) || action.parentCallId !== undefined)- ),+ const requested = visibleActions(event.data.actions).filter(+ (action) => !known.has(action.callId),);if (requested.length === 0) return undefined;- const calls = requested.map((action) =>- requestedCall(action, event.data.presentation?.[action.callId]?.label, at),- );+ const calls = requested.map((action) => requestedCall(action, event.data.presentation, at));return { turn: { ...current, calls: bounded([...current.calls, ...calls]) }, turnId };}case "action.result": {@@ -339,7 +334,7 @@ export function workingTaskNames(turn: TaskCardTurn | undefined): readonly strinfunction requestedCall(action: RuntimeActionRequest,- label: string | undefined,+ presentation: ActionPresentationByCallId | undefined,at: string,): TrackedCall {const name = actionRequestName(action);@@ -348,7 +343,7 @@ function requestedCall(name,startedAt: at,status: "working",- title: presentationText(label) ?? displayTitle(name),+ title: actionLabel(action, presentation),};const input = boundedInput(action.input);if (input !== undefined) call.input = input;diff --git a/packages/eve/src/channel/tool-eligibility.ts b/packages/eve/src/channel/tool-eligibility.tsindex 061a042a0..930b5ebca 100644--- a/packages/eve/src/channel/tool-eligibility.ts+++ b/packages/eve/src/channel/tool-eligibility.ts@@ -20,8 +20,7 @@ export interface CompiledToolBindings {* conservative phase-1 policy, stricter than the mechanical predicate in* the research plan (`hasExecute` minus handled tools and framework* actions). It is not a claim that every framework tool needs a turn: some,- * such as `load_skill` and `agent`, depend on the turn's harness or- * session, while others, such as `web_fetch`, could run without one. They+ * such as `agent`, depend on the turn's harness or session, while others, such as `web_fetch`, could run without one. They* are all withheld so that phase 1 publishes only tools the application or* an extension authored; exposing individual framework tools is a later,* explicit decision. An application or extension tool that overrides adiff --git a/packages/eve/src/cli/dev/tui/terminal-renderer.test.ts b/packages/eve/src/cli/dev/tui/terminal-renderer.test.tsindex fed0f3bf9..d5bb24d8f 100644--- a/packages/eve/src/cli/dev/tui/terminal-renderer.test.ts+++ b/packages/eve/src/cli/dev/tui/terminal-renderer.test.ts@@ -3839,7 +3839,7 @@ describe("TerminalRenderer conversation", () => {stamped(createTaskStartedEvent(call)),stamped(createActionsRequestedEvent({- actions: [{ callId: "wait_1", input: {}, kind: "tool-call", toolName: "task_wait" }],+ actions: [{ callId: "wait_1", input: {}, kind: "tool-call", toolName: "eve__task_wait" }],sequence: 2,stepIndex: 1,turnId: "turn_1",@@ -3859,7 +3859,7 @@ describe("TerminalRenderer conversation", () => {expect(during).not.toContain("Waiting for");expect(during).not.toContain("↑");expect(during).not.toContain("↓");- expect(during).not.toContain("task_wait");+ expect(during).not.toContain("eve__task_wait");const approval = renderer.readToolApproval({approvalId: "approval_1",diff --git a/packages/eve/src/cli/dev/tui/tool-presentation.test.ts b/packages/eve/src/cli/dev/tui/tool-presentation.test.tsindex b0180d0ac..2b19cb06a 100644--- a/packages/eve/src/cli/dev/tui/tool-presentation.test.ts+++ b/packages/eve/src/cli/dev/tui/tool-presentation.test.ts@@ -8,7 +8,7 @@ describe("presentPreparingTool", () => {expect(presentPreparingTool("bash").title).toBe("Run …");expect(presentPreparingTool("write_file").title).toBe("Write …");expect(presentPreparingTool("eve.web_search").title).toBe("Search …");- expect(presentPreparingTool("final_output").title).toBe("Return final output");+ expect(presentPreparingTool("eve__reply").title).toBe("Reply");});it("leads unknown tools with their name", () => {@@ -60,7 +60,6 @@ describe("presentTool", () => {expect(presentTool("bash", { command: "pnpm test" }).title).toBe("Run pnpm test");expect(presentTool("glob", { pattern: "**/*.ts" }).title).toBe("Glob **/*.ts");expect(presentTool("grep", { pattern: "useEve" }).title).toBe("Grep useEve");- expect(presentTool("load_skill", { skill: "commit" }).title).toBe("Load commit");expect(presentTool("read_file", { filePath: "/workspace/agent.ts" })).toMatchObject({title: "Read /workspace/agent.ts",group: { verb: "Read", singularNoun: "file", pluralNoun: "files" },@@ -146,13 +145,7 @@ describe("presentTool", () => {expect(presentTool("agent", { message: "Audit the auth flow.\nDetails…" }).title).toBe("Delegate Audit the auth flow.",);- expect(presentTool("connection_search", { query: "linear issues" }).title).toBe(- "Search linear issues",- );- expect(- presentTool("connection_execute", { connection: "linear", tool: "list_issues" }).title,- ).toBe("Call linear.list_issues");- expect(presentTool("final_output", { anything: true }).title).toBe("Return final output");+ expect(presentTool("eve__reply", { anything: true }).title).toBe("Reply");});it("covers the builtin presentation table with semantic copy", () => {@@ -162,7 +155,6 @@ describe("presentTool", () => {bash: { command: "ls" },glob: { pattern: "**/*.ts" },grep: { pattern: "useEve" },- load_skill: { skill: "commit" },read_file: { filePath: "/workspace/a.ts" },web_fetch: { url: "https://example.com" },web_search: { query: "eve framework" },diff --git a/packages/eve/src/cli/dev/tui/tool-presentation.ts b/packages/eve/src/cli/dev/tui/tool-presentation.tsindex 6b8dff2ff..2ce0a84f1 100644--- a/packages/eve/src/cli/dev/tui/tool-presentation.ts+++ b/packages/eve/src/cli/dev/tui/tool-presentation.ts@@ -1,6 +1,7 @@import { diffWriteDetail, type ToolDetailLine } from "./line-diff.js";import { stripTerminalControls } from "#cli/ui/terminal-text.js";import { displayName, displayTitle } from "#shared/display-name.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import { summarizeToolArgs, summarizeToolResult } from "./tool-format.js";/** Renderer-ready copy derived from a tool call without owning its lifecycle. */@@ -64,7 +65,7 @@ interface BuiltinToolCopy {/** Copy shared by the full presenters and their preparing placeholders. */const WRITE_FILE_VERB = "Write";const DELEGATE_VERB = "Delegate";-const FINAL_OUTPUT_TITLE = "Return final output";+const REPLY_TITLE = "Reply";/*** Builtin tools whose calls read as one verb plus one argument. Runs group@@ -93,26 +94,6 @@ const BUILTIN_TOOL_COPY: Readonly<Record<string, BuiltinToolCopy>> = {singularNoun: "command",pluralNoun: "commands",},- connection_execute: {- verb: "Call",- pastVerb: "Called",- argKey: "",- extractItem: (input) => {- const connection = salientArg(input, "connection");- const tool = salientArg(input, "tool");- return connection === undefined || tool === undefined ? undefined : `${connection}.${tool}`;- },- singularNoun: "connection tool",- pluralNoun: "connection tools",- },- connection_search: {- verb: "Search",- pastVerb: "Searched",- argKey: "query",- extractItem: (input) => salientArg(input, "connection"),- singularNoun: "tool search",- pluralNoun: "tool searches",- },glob: {verb: "Glob",pastVerb: "Globbed",@@ -127,13 +108,6 @@ const BUILTIN_TOOL_COPY: Readonly<Record<string, BuiltinToolCopy>> = {singularNoun: "pattern",pluralNoun: "patterns",},- load_skill: {- verb: "Load",- pastVerb: "Loaded",- argKey: "skill",- singularNoun: "skill",- pluralNoun: "skills",- },read_file: {verb: "Read",pastVerb: "Read",@@ -245,10 +219,10 @@ export function presentTool(summarizeResult: () => undefined,};}- if (baseName === "final_output") {+ if (baseName === REPLY_TOOL_NAME) {// Structured-output terminal signal: its input is the// structured result itself, kept behind the expanded `--tools full` view.- return { title: FINAL_OUTPUT_TITLE, subtitle: "", summarizeResult: () => undefined };+ return { title: REPLY_TITLE, subtitle: "", summarizeResult: () => undefined };}const copy = BUILTIN_TOOL_COPY[baseName];@@ -352,8 +326,8 @@ export function presentPreparingTool(context?: ToolPresentationContext,): ToolPresentation {const baseName = toolBaseName(toolName);- if (baseName === "final_output") {- return { title: FINAL_OUTPUT_TITLE, subtitle: "", summarizeResult: () => undefined };+ if (baseName === REPLY_TOOL_NAME) {+ return { title: REPLY_TITLE, subtitle: "", summarizeResult: () => undefined };}if (context?.isSubagent === true) {// A named subagent's tool carries the delegation target in its name —diff --git a/packages/eve/src/cli/dev/tui/traces/trace-view.test.ts b/packages/eve/src/cli/dev/tui/traces/trace-view.test.tsindex 22a041832..d75bf9504 100644--- a/packages/eve/src/cli/dev/tui/traces/trace-view.test.ts+++ b/packages/eve/src/cli/dev/tui/traces/trace-view.test.ts@@ -312,7 +312,7 @@ describe("renderTraceViewer", () => {// and inject a phantom blank line into the frame.const turn = span("a".repeat(16), "agent.turn", 0, 0);const selected = span("f".repeat(16), "agent.action", 100, 200, turn.spanId, {- "agent.action.name": "load_skill",+ "agent.action.name": "lookup","gen_ai.tool.call.result": "When the user asks about weather, call the tool.\n",});const frame = render(viewerState([turn, selected], { panelOpen: true }), 100, 24);diff --git a/packages/eve/src/cli/dev/tui/transcript.test.ts b/packages/eve/src/cli/dev/tui/transcript.test.tsindex e338e7611..a81091d4e 100644--- a/packages/eve/src/cli/dev/tui/transcript.test.ts+++ b/packages/eve/src/cli/dev/tui/transcript.test.ts@@ -305,7 +305,7 @@ describe("ConversationTranscript", () => {toolCall("call_1", "summarize"),taskStarted("call_1", "summarize"),toolResult("call_1", "summarize"),- toolCall("wait_1", "task_wait"),+ toolCall("wait_1", "eve__task_wait"),event(createTurnWaitingEvent({ on: "tasks", usage: TEST_USAGE, sequence: 3, turnId: "turn_1" }),),diff --git a/packages/eve/src/client/message-action-parts.ts b/packages/eve/src/client/message-action-parts.tsindex 8b4484f5b..d230d8b10 100644--- a/packages/eve/src/client/message-action-parts.ts+++ b/packages/eve/src/client/message-action-parts.ts@@ -1,4 +1,6 @@+import { SKILL_ENTRY_NAME } from "#protocol/catalog-tools.js";import { actionRequestName } from "#shared/action-request-name.js";+import { displayTitle } from "#shared/display-name.js";import type { RuntimeActionRequest, RuntimeActionResult } from "#shared/action-types.js";import type { InputRequest } from "#shared/input.js";import type {@@ -34,12 +36,13 @@ export function toMessageInputRequest(request: InputRequest): EveMessageInputReq/** Builds tool metadata for a freshly projected tool part. */export function createToolMetadata(descriptor: ActionDescriptor,- extra?: { readonly inputRequest?: EveMessageInputRequest },+ extra?: { readonly inputRequest?: EveMessageInputRequest; readonly label?: string },): EveMessageToolMetadata {return {eve: {inputRequest: extra?.inputRequest,kind: descriptor.kind,+ label: extra?.label,name: descriptor.name,},};@@ -64,6 +67,7 @@ export function mergeToolMetadata(inputRequest: next.eve?.inputRequest ?? current?.eve?.inputRequest,inputResponse: next.eve?.inputResponse ?? current?.eve?.inputResponse,kind,+ label: next.eve?.label ?? current?.eve?.label,name,},};@@ -74,7 +78,7 @@ export function normalizeActionRequest(action: RuntimeActionRequest): ActionDescconst name = actionRequestName(action);switch (action.kind) {case "load-skill":- return { kind: "load-skill", name, toolName: "eve:load-skill" };+ return { kind: "load-skill", name, toolName: SKILL_ENTRY_NAME };case "tool-call":case "workflow-tool-call":return { kind: "tool-call", name, toolName: name };@@ -90,8 +94,8 @@ export function normalizeActionResult(result: RuntimeActionResult): ActionDescricase "load-skill-result":return {kind: "load-skill",- name: result.name ?? "load_skill",- toolName: "eve:load-skill",+ name: result.name ?? SKILL_ENTRY_NAME,+ toolName: SKILL_ENTRY_NAME,};case "tool-result":return {@@ -108,6 +112,15 @@ export function normalizeActionResult(result: RuntimeActionResult): ActionDescri}}+/** A settling call reads as its update or completion label, else as it already did. */+export function settledLabel(+ current: EveMessageToolMetadata | undefined,+ label: string | undefined,+ descriptor: ActionDescriptor,+): string {+ return label ?? current?.eve?.label ?? displayTitle(descriptor.name);+}+/** Best-effort string rendering of an unknown tool output for error display. */export function stringifyUnknown(value: unknown): string {if (typeof value === "string") return value;diff --git a/packages/eve/src/client/message-reducer-types.ts b/packages/eve/src/client/message-reducer-types.tsindex 16695f3ac..a2917bcad 100644--- a/packages/eve/src/client/message-reducer-types.ts+++ b/packages/eve/src/client/message-reducer-types.ts@@ -227,13 +227,18 @@ export type EveDynamicToolPart = {* classifies the action (`"tool-call"`, `"subagent-call"`, `"load-skill"`, or* `"unknown"`), `eve.name` is the resolved action name, and `eve.inputRequest`* and `eve.inputResponse` store the HITL prompt and submitted response when the- * call required approval.+ * call required approval. `eve.label` is how the call reads to people, the same+ * text eve's channels and terminal UI show: the tool's label, or a readable form+ * of its name, replaced by its completion label once it settles. The default+ * reducer always sets it; it is optional only for parts persisted by older eve+ * versions.*/export interface EveMessageToolMetadata {readonly eve?: {readonly inputRequest?: EveMessageInputRequest;readonly inputResponse?: InputResponse;readonly kind: "load-skill" | "subagent-call" | "tool-call" | "unknown";+ readonly label?: string;readonly name: string;};}diff --git a/packages/eve/src/client/message-reducer.test.ts b/packages/eve/src/client/message-reducer.test.tsindex c59126fa8..f3a2a8b6c 100644--- a/packages/eve/src/client/message-reducer.test.ts+++ b/packages/eve/src/client/message-reducer.test.ts@@ -267,7 +267,9 @@ describe("defaultMessageReducer", () => {state: "input-available",stepIndex: 0,toolCallId: "call_render",- toolMetadata: { eve: { inputRequest: undefined, kind: "tool-call", name: "render" } },+ toolMetadata: {+ eve: { inputRequest: undefined, kind: "tool-call", label: "Render", name: "render" },+ },toolName: "render",type: "dynamic-tool",});@@ -311,13 +313,49 @@ describe("defaultMessageReducer", () => {stepIndex: 0,toolCallId: "call_publish",toolMetadata: {- eve: { inputRequest: undefined, kind: "tool-call", name: "publish" },+ eve: { inputRequest: undefined, kind: "tool-call", label: "Publish", name: "publish" },},toolName: "publish",type: "dynamic-tool",});});+ it("labels a tool part as its call reads, then as its completion label", () => {+ const reducer = defaultMessageReducer();+ const requested = reduceServerEvents(reducer, reducer.initial(), [+ createActionsRequestedEvent({+ actions: [+ {+ callId: "call_search",+ input: { query: "refunds" },+ kind: "tool-call",+ toolName: "find",+ },+ ],+ presentation: { call_search: { label: "Search tools for “refunds”" } },+ sequence: 1,+ stepIndex: 0,+ turnId: "turn_1",+ }),+ ]);+ const settled = reduceServerEvents(reducer, requested, [+ createActionResultEvent({+ presentation: { call_search: { label: "Searched tools for “refunds”, found 3" } },+ result: { callId: "call_search", kind: "tool-result", output: {}, toolName: "find" },+ sequence: 2,+ stepIndex: 0,+ turnId: "turn_1",+ }),+ ]);++ expect(findToolPart(requested, "call_search")?.toolMetadata?.eve?.label).toBe(+ "Search tools for “refunds”",+ );+ expect(findToolPart(settled, "call_search")?.toolMetadata?.eve?.label).toBe(+ "Searched tools for “refunds”, found 3",+ );+ });+it("removes an unfinished streamed tool input when the turn is cancelled", () => {const reducer = defaultMessageReducer();const data = reduceServerEvents(reducer, reducer.initial(), [@@ -749,6 +787,7 @@ describe("defaultMessageReducer", () => {toolMetadata: {eve: {kind: "tool-call",+ label: "Get weather",name: "get_weather",inputRequest: undefined,inputResponse: undefined,@@ -809,6 +848,7 @@ describe("defaultMessageReducer", () => {toolMetadata: {eve: {kind: "subagent-call",+ label: "Research",name: "research",},},@@ -895,6 +935,7 @@ describe("defaultMessageReducer", () => {toolMetadata: {eve: {kind: "tool-call",+ label: "Bash",name: "bash",},},@@ -1139,6 +1180,7 @@ describe("defaultMessageReducer", () => {requestId: "approval_1",},kind: "tool-call",+ label: "Bash",name: "bash",},},diff --git a/packages/eve/src/client/message-reducer.ts b/packages/eve/src/client/message-reducer.tsindex f14904eca..9e1cec8ed 100644--- a/packages/eve/src/client/message-reducer.ts+++ b/packages/eve/src/client/message-reducer.ts@@ -19,6 +19,7 @@ import {mergeToolMetadata,normalizeActionRequest,normalizeActionResult,+ settledLabel,stringifyUnknown,toMessageInputRequest,} from "#client/message-action-parts.js";@@ -30,6 +31,7 @@ import {} from "#client/message-reducer-primitives.js";import { messageRun } from "#client/message-run-parts.js";import type { AuthorizationCompletedStreamEvent } from "#protocol/message.js";+import { actionLabel } from "#shared/action-label.js";import {foldSession,initialSessionProjection,@@ -194,7 +196,9 @@ function reduceContent(data: EveMessageData, event: EveAgentReducerEvent): EveMestate: "input-available",stepIndex: event.data.stepIndex,toolCallId: action.callId,- toolMetadata: createToolMetadata(descriptor),+ toolMetadata: createToolMetadata(descriptor, {+ label: actionLabel(action, event.data.presentation),+ }),toolName: descriptor.toolName,type: "dynamic-tool",}),@@ -224,6 +228,7 @@ function reduceContent(data: EveMessageData, event: EveAgentReducerEvent): EveMetoolCallId: request.action.callId,toolMetadata: createToolMetadata(descriptor, {inputRequest: toMessageInputRequest(request),+ label: existing?.toolMetadata?.eve?.label ?? actionLabel(request.action, undefined),}),toolName: descriptor.toolName,type: "dynamic-tool",@@ -257,7 +262,16 @@ function reduceContent(data: EveMessageData, event: EveAgentReducerEvent): EveMeinput: existing?.input,stepIndex: existing?.stepIndex ?? event.data.stepIndex,toolCallId: event.data.result.callId,- toolMetadata: mergeToolMetadata(existing?.toolMetadata, createToolMetadata(descriptor)),+ toolMetadata: mergeToolMetadata(+ existing?.toolMetadata,+ createToolMetadata(descriptor, {+ label: settledLabel(+ existing?.toolMetadata,+ event.data.presentation?.[event.data.result.callId]?.label,+ descriptor,+ ),+ }),+ ),toolName: existing?.toolName ?? descriptor.toolName,type: "dynamic-tool" as const,};@@ -286,7 +300,16 @@ function reduceContent(data: EveMessageData, event: EveAgentReducerEvent): EveMestate: "output-available",stepIndex: existing?.stepIndex ?? event.data.stepIndex,toolCallId: event.data.result.callId,- toolMetadata: mergeToolMetadata(existing?.toolMetadata, createToolMetadata(descriptor)),+ toolMetadata: mergeToolMetadata(+ existing?.toolMetadata,+ createToolMetadata(descriptor, {+ label: settledLabel(+ existing?.toolMetadata,+ event.data.presentation?.[event.data.result.callId]?.label,+ descriptor,+ ),+ }),+ ),toolName: existing?.toolName ?? descriptor.toolName,type: "dynamic-tool",} as EveDynamicToolPart);diff --git a/packages/eve/src/compiler/connection-ownership.ts b/packages/eve/src/compiler/connection-ownership.tsnew file mode 100644index 000000000..2e32e3720--- /dev/null+++ b/packages/eve/src/compiler/connection-ownership.ts@@ -0,0 +1,25 @@+import { assertNotConnectionOwned } from "#connections/ownership.js";++/** Rejects static names that fall under a static connection's name or `__` prefix. */+export function assertStaticConnectionOwnership(input: {+ readonly connectionNames: readonly string[];+ readonly subagentNames: readonly string[];+ readonly toolNames: readonly string[];+}): void {+ const { connectionNames } = input;+ const checks = [+ ["Connection", connectionNames, "Rename one of the connections."],+ ["Tool", input.toolNames, "Rename the tool file."],+ ["Subagent", input.subagentNames, "Rename the subagent directory."],+ ] as const;+ for (const [subject, names, remedy] of checks) {+ for (const name of names) {+ // A connection owns its own name.+ const owners =+ names === connectionNames+ ? connectionNames.filter((other) => other !== name)+ : connectionNames;+ assertNotConnectionOwned({ connectionNames: owners, name, remedy, subject });+ }+ }+}diff --git a/packages/eve/src/compiler/default-tool-policy.ts b/packages/eve/src/compiler/default-tool-policy.tsindex 965fab3db..c62a9530d 100644--- a/packages/eve/src/compiler/default-tool-policy.ts+++ b/packages/eve/src/compiler/default-tool-policy.ts@@ -3,29 +3,7 @@ import type {FinalizedNodeSourceState,PhaseOneNodeSourceState,} from "#compiler/node-source-state.js";-import {- canonicalSourceSlot,- composeAgentModuleCandidates,- type AgentSourceCandidate,-} from "#compiler/source-graph.js";--const CONNECTION_TOOLS_SLOT = "tools/connection_tools";-/** Slots of the closed connection tools and the framework module that provides them. */-const CONNECTION_TOOL_SLOTS = new Set([- CONNECTION_TOOLS_SLOT,- "tools/connection_search",- "tools/connection_execute",-]);--/** Rejects authored sources that would replace or disable the connection tools. */-export function assertFrameworkToolPolicy(candidate: AgentSourceCandidate): void {- if (candidate.layer === "framework-default") return;- const slot = canonicalSourceSlot(candidate.logicalPath);- if (!CONNECTION_TOOL_SLOTS.has(slot)) return;- throw new Error(- `"agent/${slot}.ts" is reserved. connection_search and connection_execute are framework tools that cannot be replaced or disabled; they exist only while the agent has connections. Rename the file.`,- );-}+import { canonicalSourceSlot, composeAgentModuleCandidates } from "#compiler/source-graph.js";export function applyAgentToolPolicy(phaseOne: PhaseOneNodeSourceState,@@ -73,7 +51,6 @@ export function applyDefaultToolPolicy(return (candidate.layer !== "framework-default" ||!slot.startsWith("tools/") ||- slot === CONNECTION_TOOLS_SLOT ||overriddenSlots.has(slot));}),diff --git a/packages/eve/src/compiler/extension-compatibility.ts b/packages/eve/src/compiler/extension-compatibility.tsindex 66f09b3fa..05ef04a29 100644--- a/packages/eve/src/compiler/extension-compatibility.ts+++ b/packages/eve/src/compiler/extension-compatibility.ts@@ -22,8 +22,10 @@ interface ExtensionCapabilityContract {const EXTENSION_CAPABILITY_CONTRACTS = {extension: { current: 1, supported: [1], dropped: {} },tool: {- current: 80,- supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 29, 30, 31, 32, 34, 35, 76, 77, 78, 80],+ current: 81,+ supported: [+ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 29, 30, 31, 32, 34, 35, 76, 77, 78, 80, 81,+ ],dropped: {14: "TaskExec.delegated was removed; migrate to workflow-backed background tools",15: "TaskExec replaces stageEffect with send",@@ -85,10 +87,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},dynamicTool: {- current: 75,+ current: 76,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 31, 32, 33, 52, 54,- 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 73, 75,+ 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 73, 75, 76,],dropped: {21: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",@@ -129,10 +131,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},channel: {- current: 54,+ current: 55,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 29, 31, 36, 37, 38, 39, 40, 41, 42,- 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54,+ 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 55,],dropped: {12: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",@@ -155,9 +157,9 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},schedule: {- current: 27,+ current: 28,supported: [- 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,+ 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,],dropped: {5: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",@@ -166,9 +168,9 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},subagent: {- current: 31,+ current: 32,supported: [- 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31,+ 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,],dropped: {1: "Persistent subagent sessions are now the default and the experimental opt-in was removed",@@ -182,10 +184,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},connection: {- current: 43,+ current: 44,supported: [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 26, 32, 33, 34,- 35, 36, 37, 38, 39, 40, 41, 43,+ 35, 36, 37, 38, 39, 40, 41, 43, 44,],dropped: {9: "Dynamic connection resolvers no longer receive conversation or channel continuation data",@@ -201,8 +203,8 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},},hook: {- current: 41,- supported: [20, 21, 22, 23, 25, 27, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41],+ current: 42,+ supported: [20, 21, 22, 23, 25, 27, 31, 32, 33, 34, 35, 36, 37, 38, 39, 41, 42],dropped: {1: "Model identity moved from session.started runtime metadata to step.started call attribution.",2: "Model identity moved from session.started runtime metadata to step.started call attribution.",@@ -231,12 +233,12 @@ const EXTENSION_CAPABILITY_CONTRACTS = {40: "Sandbox provider handle hooks were renamed to onSandboxStop and onSandboxDelete.",},},- skill: { current: 2, supported: [1, 2], dropped: {} },+ skill: { current: 3, supported: [1, 2, 3], dropped: {} },dynamicSkill: {- current: 33,+ current: 34,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 25, 26, 27, 28, 29, 30,- 31, 32, 33,+ 31, 32, 33, 34,],dropped: {13: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",@@ -247,10 +249,10 @@ const EXTENSION_CAPABILITY_CONTRACTS = {},instructions: { current: 2, supported: [1, 2], dropped: {} },dynamicInstructions: {- current: 34,+ current: 35,supported: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 26, 27, 28, 29, 30,- 31, 32, 33, 34,+ 31, 32, 33, 34, 35,],dropped: {14: "Message and reasoning append events now expose deltas instead of cumulative snapshots.",diff --git a/packages/eve/src/compiler/manifest.ts b/packages/eve/src/compiler/manifest.tsindex 055621876..5d875f1ee 100644--- a/packages/eve/src/compiler/manifest.ts+++ b/packages/eve/src/compiler/manifest.ts@@ -633,7 +633,7 @@ const compiledAgentConfigBaseFields = {.enum(["provider-default", "none", "minimal", "low", "medium", "high", "xhigh"]).optional(),source: moduleSourceRefSchema,- tool: z.boolean().optional(),+ tool: z.union([z.boolean(), z.literal("deferred")]).optional(),limits: compiledAgentLimitsDefinitionSchema.optional(),};@@ -681,6 +681,7 @@ const compiledInstructionsSchema: z.ZodType<CompiledInstructionsDefinition> = z.const compiledSkillBaseFields = {name: z.string(),+ deferred: z.boolean().optional(),description: z.string(),license: z.string().optional(),markdown: z.string(),@@ -859,7 +860,6 @@ const compiledToolBehaviorSchema: z.ZodType<CompiledToolBehavior> = z.strict(),]).optional(),- presentation: z.literal("load-skill").optional(),shape: z.object({suspend: z.enum(["none", "workflow"]),@@ -873,6 +873,7 @@ const compiledToolDefinitionSchema = z.object({availableInSubagents: z.boolean().optional(),behavior: compiledToolBehaviorSchema.optional(),+ deferred: z.boolean().optional(),description: z.string(),exportName: z.string().optional(),hasExecute: z.boolean(),diff --git a/packages/eve/src/compiler/normalize-agent-config.ts b/packages/eve/src/compiler/normalize-agent-config.tsindex 6b33a103f..134ab0766 100644--- a/packages/eve/src/compiler/normalize-agent-config.ts+++ b/packages/eve/src/compiler/normalize-agent-config.ts@@ -12,6 +12,7 @@ import type { ModuleSourceRef } from "#shared/source-ref.js";import {isDynamicModelDefinition,type PublicAgentStaticModelDefinition,+ type AgentToolExposure,} from "#shared/agent-definition.js";import type { DynamicToolEventName } from "#dynamic/definition.js";import type { CompiledAgentDefinition, CompiledRuntimeModelReference } from "#compiler/manifest.js";@@ -88,7 +89,7 @@ export async function compileAgentConfig(name: string;reasoning?: CompiledAgentDefinition["reasoning"];source: ModuleSourceRef;- tool?: boolean;+ tool?: AgentToolExposure;limits?: CompiledAgentDefinition["limits"];} = {compaction,diff --git a/packages/eve/src/compiler/normalize-connection.ts b/packages/eve/src/compiler/normalize-connection.tsindex 04b1ca85b..2dfe4fc49 100644--- a/packages/eve/src/compiler/normalize-connection.ts+++ b/packages/eve/src/compiler/normalize-connection.ts@@ -1,4 +1,5 @@import type { ConnectionSourceRef } from "#discover/manifest.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import {normalizeMcpClientConnectionDefinition,normalizeOpenApiConnectionDefinition,@@ -82,6 +83,14 @@ export async function compileConnectionDefinition(};}+ // A connection's own name is the entry that signs the user in to it.+ const reservation = eveNamespaceReservation(source.connectionName);+ if (reservation !== undefined) {+ throw new Error(+ `Connection "${source.logicalPath}" uses the reserved name "${source.connectionName}". Rename its path; ${reservation}.`,+ );+ }+const protocol = readConnectionProtocol(loaded);const shared = {diff --git a/packages/eve/src/compiler/normalize-manifest.test.ts b/packages/eve/src/compiler/normalize-manifest.test.tsindex 6b75f17b7..566a488b9 100644--- a/packages/eve/src/compiler/normalize-manifest.test.ts+++ b/packages/eve/src/compiler/normalize-manifest.test.ts@@ -26,6 +26,11 @@ import { defineDynamicSchedules } from "#public/schedules/subscription.js";import { inMemoryScheduleProvider } from "#public/schedules/providers/in-memory.js";import { z } from "#compiled/zod/index.js";import { defineSkill } from "#public/definitions/skill.js";+import {+ EVE_NAMESPACE_NAMES,+ EVE_NAMESPACE_RESERVATION,+ NAMES_OUTSIDE_EVE_NAMESPACE,+} from "#internal/testing/catalog-fixtures.js";import { resolveAgent } from "#runtime/resolve-agent.js";import { resolveRuntimeAgentGraph } from "#runtime/resolve-agent-graph.js";import { compiledAgentManifestSchema } from "#compiler/manifest.js";@@ -217,7 +222,6 @@ describe("compileAgentManifest source graph", () => {expect(compiled.config.defaultTools).toBe(false);expect(compiled.tools.map((tool) => tool.name).sort()).toEqual(["bash", "weather"]);- expect(compiled.dynamicTools.map((tool) => tool.slug)).toEqual(["connection_tools"]);expect(compiled.tools.find((tool) => tool.name === "bash")?.description).toBe("Application-owned shell replacement.",);@@ -232,23 +236,6 @@ describe("compileAgentManifest source graph", () => {).toEqual(["tools/bash.ts"]);});- it.each([- ["tools/connection_search.ts", disableTool()],- [- "tools/connection_execute.ts",- defineTool({ description: "Replacement.", execute: () => null, inputSchema: {} }),- ],- ["tools/connection_tools.ts", disableTool()],- ])("rejects authored %s because the connection tools are closed", async (logicalPath, entry) => {- const sourceRegistry = registry([- { logicalPath, loadNamespace: async () => ({ default: entry }) },- ]);-- await expect(- compileAgentManifest(manifest(), { sourceRegistries: [sourceRegistry] }),- ).rejects.toThrow(`"agent/${logicalPath}" is reserved.`);- });-it("allows disableTool for the root agent tool", async () => {const sourceRegistry = registry([{@@ -782,21 +769,142 @@ describe("compileAgentManifest source graph", () => {);});- it("reserves the task tools' names", async () => {+ describe("eve's namespace", () => {+ const authoredTool = (name: string) => ({+ logicalPath: `tools/${name}.ts`,+ loadNamespace: async () => ({+ default: defineTool({ description: name, execute: () => null, inputSchema: {} }),+ }),+ });+ const authoredSkill = (name: string) => ({+ logicalPath: `skills/${name}.ts`,+ loadNamespace: async () => ({+ default: defineSkill({ description: name, markdown: `# ${name}\n` }),+ }),+ });+ const compile = (sources: Parameters<typeof registry>[0]) =>+ compileAgentManifest(manifest(), { sourceRegistries: [registry(sources)] });++ it.each(EVE_NAMESPACE_NAMES)("rejects an authored tool named %s", async (name) => {+ await expect(compile([authoredTool(name)])).rejects.toThrow(+ `Tool "tools/${name}.ts" uses the reserved name "${name}". Rename its path; ${EVE_NAMESPACE_RESERVATION}.`,+ );+ });++ it("rejects disabling a built-in tool, whose name is in eve's namespace", async () => {+ await expect(+ compile([+ {+ logicalPath: "tools/eve__search.ts",+ loadNamespace: async () => ({ default: disableTool() }),+ },+ ]),+ ).rejects.toThrow(EVE_NAMESPACE_RESERVATION);+ });++ it.each(EVE_NAMESPACE_NAMES)("rejects an authored skill named %s", async (name) => {+ await expect(compile([authoredSkill(name)])).rejects.toThrow(+ `uses the reserved name "${name}". Rename its path; ${EVE_NAMESPACE_RESERVATION}.`,+ );+ });++ it("accepts tools and skills outside it, including the built-in tools' former names", async () => {+ const compiled = await compile([+ ...NAMES_OUTSIDE_EVE_NAMESPACE.map(authoredTool),+ ...NAMES_OUTSIDE_EVE_NAMESPACE.map(authoredSkill),+ ]);++ expect(compiled.tools.map((entry) => entry.name)).toEqual(+ expect.arrayContaining(NAMES_OUTSIDE_EVE_NAMESPACE),+ );+ expect(compiled.skills.map((entry) => entry.name).toSorted()).toEqual(+ NAMES_OUTSIDE_EVE_NAMESPACE.toSorted(),+ );+ });+ });++ it("rejects a deferred provider tool, which the provider has to see", async () => {const sourceRegistry = registry([{- logicalPath: "tools/task_cancel.ts",+ logicalPath: "tools/web_search.ts",loadNamespace: async () => ({- default: defineTool({ description: "Cancel.", inputSchema: {}, execute: () => null }),+ default: { ...webSearch({ provider: "exa" }), deferred: true },}),},]);await expect(compileAgentManifest(manifest(), { sourceRegistries: [sourceRegistry] }),- ).rejects.toThrow(- 'Tool "tools/task_cancel.ts" uses the reserved name "task_cancel". Rename its path; eve reserves "task_cancel" for its built-in task tool.',- );+ ).rejects.toThrow("Provider tools can't be deferred");+ });++ describe("connection name ownership", () => {+ const linear = {+ logicalPath: "connections/linear.ts",+ loadNamespace: async () => ({+ default: defineMcpClientConnection({+ description: "Linear.",+ url: "https://mcp.linear.example",+ }),+ }),+ };+ const tool = (name: string) => ({+ logicalPath: `tools/${name}.ts`,+ loadNamespace: async () => ({+ default: defineTool({ description: name, execute: () => null, inputSchema: {} }),+ }),+ });++ it.each([+ [+ "linear__sync",+ 'Tool "linear__sync" starts with "linear__", which belongs to connection "linear". Rename the tool file.',+ ],+ ["linear", 'Tool "linear" has the same name as connection "linear". Rename the tool file.'],+ ])("rejects a tool named %s beside connection linear", async (name, message) => {+ const sourceRegistry = registry([linear, tool(name)]);++ await expect(+ compileAgentManifest(manifest(), { sourceRegistries: [sourceRegistry] }),+ ).rejects.toThrow(message);+ });++ it("rejects a connection named eve, which would own eve's namespace", async () => {+ const sourceRegistry = registry([{ ...linear, logicalPath: "connections/eve.ts" }]);++ await expect(+ compileAgentManifest(manifest(), { sourceRegistries: [sourceRegistry] }),+ ).rejects.toThrow(+ `Connection "connections/eve.ts" uses the reserved name "eve". Rename its path; ${EVE_NAMESPACE_RESERVATION}.`,+ );+ });++ it("accepts connections outside eve's namespace, including the catalog tools' former names", async () => {+ const names = ["search", "execute", "steve", "eve-tools"];+ const sourceRegistry = registry(+ names.map((name) => ({ ...linear, logicalPath: `connections/${name}.ts` })),+ );++ const compiled = await compileAgentManifest(manifest(), {+ sourceRegistries: [sourceRegistry],+ });++ expect(compiled.connections.map((entry) => entry.connectionName).toSorted()).toEqual(+ names.toSorted(),+ );+ });++ it("allows a convention prefix that no connection owns", async () => {+ const sourceRegistry = registry([linear, tool("linearize"), tool("tenant__export")]);++ const compiled = await compileAgentManifest(manifest(), {+ sourceRegistries: [sourceRegistry],+ });++ expect(compiled.tools.map((entry) => entry.name)).toEqual(+ expect.arrayContaining(["linearize", "tenant__export"]),+ );+ });});it("projects a local subagent node once", async () => {diff --git a/packages/eve/src/compiler/normalize-manifest.ts b/packages/eve/src/compiler/normalize-manifest.tsindex ec795aee4..8567d0711 100644--- a/packages/eve/src/compiler/normalize-manifest.ts+++ b/packages/eve/src/compiler/normalize-manifest.ts@@ -1,6 +1,7 @@import { randomUUID } from "node:crypto";import type { ExtensionCompileMount } from "#compiler/load-binding-namespace.js";import { NodeModuleEvaluationContext } from "#compiler/module-lifecycle.js";+import { assertStaticConnectionOwnership } from "#compiler/connection-ownership.js";import { bindingMountId } from "#compiler/extension-mount-bindings.js";import type { AgentSourceManifest } from "#discover/manifest.js";@@ -39,7 +40,6 @@ import { compileConnectionDefinition } from "#compiler/normalize-connection.js";import {applyAgentToolPolicy,applyDefaultToolPolicy,- assertFrameworkToolPolicy,canDisableToolWithoutSelectedSource,} from "#compiler/default-tool-policy.js";import {@@ -57,7 +57,7 @@ import { compileScheduleDefinition } from "#compiler/normalize-schedule.js";import { compileScheduleCollectionDefinition } from "#compiler/normalize-schedule-collection.js";import { isScheduleCollectionDefinition } from "#shared/schedule-collection-definition.js";import { createScheduleCollectionWrapperCandidates } from "#compiler/schedule-collection-wrapper-candidates.js";-import { compileSkillSource } from "#compiler/normalize-skill.js";+import { assertSkillNameAvailable, compileSkillSource } from "#compiler/normalize-skill.js";import {assertRemoteAgentDefinitionHasNoLocalPackageEntries,normalizeSubagentConfig,@@ -576,15 +576,16 @@ class AgentGraphCompiler {}case "skill": {const result = await compileSkillSource(input.manifest.agentRoot, entry.source, options);- if (result.kind === "skill") skills.push(result.definition);- else {+ if (result.kind === "skill") {+ assertSkillNameAvailable(result.definition);+ skills.push(result.definition);+ } else {dynamicSkills.push(withExtensionNamespace(result.definition, candidate.owner));state.evaluation.requireRuntimeEntry(candidate.sourceId);}break;}case "tool": {- assertFrameworkToolPolicy(candidate);const result = await compileToolEntry(input.manifest.agentRoot, entry.source, {binding: binding!,loadNamespace,@@ -622,6 +623,13 @@ class AgentGraphCompiler {assertUniqueBy(connections, (connection) => connection.connectionName, "connection name");assertUniqueBy(dynamicConnections, (connection) => connection.slug, "dynamic connection slug");assertUniqueBy(skills, (skill) => skill.name, "skill name");+ assertStaticConnectionOwnership({+ connectionNames: connections.map((connection) => connection.connectionName),+ subagentNames: state.projected.subagents+ .filter((subagent) => selectedSourceIds.has(subagent.candidate.sourceId))+ .map((subagent) => subagent.source.subagentId),+ toolNames: tools.map((tool) => tool.name),+ });const channelRoutes = compileChannelRoutes(state, channels, this.diagnostics, input.nodeId);const extensionMounts = compileExtensionMounts(input.manifest, state.composed, input.nodePath);diff --git a/packages/eve/src/compiler/normalize-skill.ts b/packages/eve/src/compiler/normalize-skill.tsindex 8d46e5351..a4839cdfd 100644--- a/packages/eve/src/compiler/normalize-skill.ts+++ b/packages/eve/src/compiler/normalize-skill.ts@@ -3,6 +3,7 @@ import type { SkillSourceRef } from "#discover/manifest.js";import type { SkillPackageSourceRef } from "#shared/source-ref.js";import type { NamedSkillDefinition } from "#shared/skill-definition.js";import { normalizeSkillDefinition } from "#internal/authored-definition/core.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import type {CompiledDynamicSkillDefinition,CompiledSkillDefinition,@@ -28,6 +29,19 @@ type CompiledSkillEntry =| { readonly kind: "skill"; readonly definition: CompiledSkillDefinition }| { readonly kind: "dynamic-skill"; readonly definition: CompiledDynamicSkillDefinition };+/** Throws when an authored skill takes a name in eve's namespace. */+export function assertSkillNameAvailable(skill: {+ readonly logicalPath: string;+ readonly name: string;+}): void {+ const reservation = eveNamespaceReservation(skill.name);+ if (reservation !== undefined) {+ throw new Error(+ `Skill "${skill.logicalPath}" uses the reserved name "${skill.name}". Rename its path; ${reservation}.`,+ );+ }+}+/*** Compiles one authored skill source (markdown, module, or skill* package directory) into the normalized shape stored on the compiled@@ -50,6 +64,7 @@ export async function compileSkillSource(return {kind: "skill",definition: {+ deferred: definition.deferred,description: definition.description,files: definition.files,license: definition.license,@@ -105,6 +120,7 @@ export async function compileSkillSource(return {kind: "skill",definition: {+ deferred: definition.deferred,description: definition.description,files: definition.files,license: definition.license,@@ -129,6 +145,7 @@ function compileSkillPackageSource(): CompiledSkillDefinition {return {assetsPath: source.assetsPath,+ deferred: source.deferred,description: source.description,license: source.license,logicalPath: source.logicalPath,diff --git a/packages/eve/src/compiler/normalize-subagent.ts b/packages/eve/src/compiler/normalize-subagent.tsindex b43b3654c..3cce91155 100644--- a/packages/eve/src/compiler/normalize-subagent.ts+++ b/packages/eve/src/compiler/normalize-subagent.ts@@ -1,4 +1,5 @@import {+ expectAgentToolExposure,expectBoolean,expectFunction,expectObjectRecord,@@ -8,6 +9,7 @@ import {import { EVE_SESSION_ROUTE_PATH } from "#protocol/routes.js";import { isDynamicSentinel, type DynamicToolEventName } from "#dynamic/definition.js";import type { LocalSubagentSourceRef } from "#discover/manifest.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";const ALLOWED_DYNAMIC_SUBAGENT_EVENTS = new Set<DynamicToolEventName>(["session.started",@@ -28,7 +30,7 @@ export type NormalizedSubagentConfig =readonly description: string;readonly kind: "remote";readonly path: string;- readonly tool?: boolean;+ readonly tool?: AgentToolExposure;readonly url?: string;};@@ -75,7 +77,7 @@ export function normalizeSubagentConfig(value: unknown, message: string): Normaldescription: expectString(record.description, message),kind: "remote",path: record.path === undefined ? EVE_SESSION_ROUTE_PATH : expectString(record.path, message),- tool: record.tool === undefined ? undefined : expectBoolean(record.tool, message),+ tool: record.tool === undefined ? undefined : expectAgentToolExposure(record.tool, message),url: typeof record.url === "function" ? undefined : expectString(record.url, message),};}diff --git a/packages/eve/src/compiler/normalize-tool.ts b/packages/eve/src/compiler/normalize-tool.tsindex ec94831ea..01d5b5397 100644--- a/packages/eve/src/compiler/normalize-tool.ts+++ b/packages/eve/src/compiler/normalize-tool.ts@@ -6,7 +6,7 @@ import {loadModuleBackedDefinition,type ModuleBackedDefinitionLoadOptions,} from "#compiler/normalize-helpers.js";-import { TASK_TOOL_NAMES } from "#protocol/task-tools.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";/*** Compiled tool entry produced from one authored `tools/*.ts` file.@@ -54,9 +54,11 @@ export async function compileToolEntry(.replace(/^tools\//, "").replaceAll("/", "-");- if (TASK_TOOL_NAMES.includes(toolName)) {+ // Checked before a disabled tool returns: a runtime tool can't be disabled either.+ const reservation = eveNamespaceReservation(toolName);+ if (reservation !== undefined) {throw new Error(- `Tool "${source.logicalPath}" uses the reserved name "${toolName}". Rename its path; eve reserves "${toolName}" for its built-in task tool.`,+ `Tool "${source.logicalPath}" uses the reserved name "${toolName}". Rename its path; ${reservation}.`,);}@@ -115,6 +117,7 @@ export async function compileToolEntry(kind: "tool",definition: {availableInSubagents: entry.definition.availableInSubagents,+ deferred: entry.definition.deferred,behavior:workflow === undefined? entry.definition.behavior === undefineddiff --git a/packages/eve/src/compiler/remote-agent-node.ts b/packages/eve/src/compiler/remote-agent-node.tsindex 9f81f0e8e..6ae89d4ec 100644--- a/packages/eve/src/compiler/remote-agent-node.ts+++ b/packages/eve/src/compiler/remote-agent-node.ts@@ -4,6 +4,7 @@ import { mountIdSchema } from "#shared/extension-mount.js";import type { Node } from "#shared/node.js";import type { ModuleSourceRef } from "#shared/source-ref.js";import type { AgentSourceOwner, CompiledModuleBinding } from "#compiler/source-graph.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";export interface CompiledDynamicSubagentDefinition extends Readonly<ModuleSourceRef> {readonly build?: {@@ -29,7 +30,7 @@ export type CompiledRemoteAgentNode = Readonly<parentNodeId: string;path: string;rootPath: string;- tool?: boolean;+ tool?: AgentToolExposure;// Absent when the definition's `url` is a function the runtime resolves.url?: string;}@@ -104,7 +105,7 @@ export const compiledRemoteAgentNodeSchema: z.ZodType<CompiledRemoteAgentNode> =parentNodeId: z.string(),path: z.string(),rootPath: z.string(),- tool: z.boolean().optional(),+ tool: z.union([z.boolean(), z.literal("deferred")]).optional(),sourceId: z.string(),sourceKind: z.literal("module"),url: z.string().optional(),diff --git a/packages/eve/src/connections/ownership.ts b/packages/eve/src/connections/ownership.tsnew file mode 100644index 000000000..9627e551b--- /dev/null+++ b/packages/eve/src/connections/ownership.ts@@ -0,0 +1,40 @@+/**+ * A connection owns its name and every name under its `<name>__` prefix, so a+ * catalog name under that prefix always means one of the connection's tools.+ * Names are compared as written; a connection's tools are never listed to+ * check them.+ */++/** The full name of a connection tool: `linear__list_issues`. */+export function connectionToolName(connectionName: string, toolName: string): string {+ return `${connectionName}__${toolName}`;+}++/** The connection that owns `name`, if any. */+function owningConnection(name: string, connectionNames: Iterable<string>): string | undefined {+ for (const connection of connectionNames) {+ if (name === connection || name.startsWith(connectionToolName(connection, ""))) {+ return connection;+ }+ }+ return undefined;+}++/**+ * Throws when `name` belongs to one of `connectionNames`. `subject` names the+ * rejected entry, such as `Dynamic tool`, and `remedy` says how to fix it.+ */+export function assertNotConnectionOwned(input: {+ readonly connectionNames: Iterable<string>;+ readonly name: string;+ readonly remedy: string;+ readonly subject: string;+}): void {+ const connection = owningConnection(input.name, input.connectionNames);+ if (connection === undefined) return;+ const conflict =+ input.name === connection+ ? `has the same name as connection "${connection}"`+ : `starts with "${connection}__", which belongs to connection "${connection}"`;+ throw new Error(`${input.subject} "${input.name}" ${conflict}. ${input.remedy}`);+}diff --git a/packages/eve/src/context/build-dynamic-tools.ts b/packages/eve/src/context/build-dynamic-tools.tsindex ec687083e..7722516ab 100644--- a/packages/eve/src/context/build-dynamic-tools.ts+++ b/packages/eve/src/context/build-dynamic-tools.ts@@ -1,6 +1,5 @@import type { StandardSchemaV1 } from "#compiled/@standard-schema/spec/index.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";-import type { HarnessToolMap } from "#harness/types.js";import type { ContextReader } from "#context/key.js";import {SessionIdKey,@@ -141,6 +140,7 @@ export function replayDynamicTools(-readonly [K in keyof HarnessToolDefinition]: HarnessToolDefinition[K];} = {availableInSubagents: entry.availableInSubagents,+ deferred: entry.deferred,frameworkTool: entry.frameworkTool,description: entry.description,endsTurn: entry.endsTurn,@@ -258,24 +258,7 @@ function requireCurrentDynamicToolMetadata(return metadata as readonly CurrentDynamicToolMetadata[];}-/**- * Builds live dynamic tool definitions. Narrower scopes appear first so they- * win on name collision (the tool loop uses `??=` for deduplication).- */-export function buildResponseAuthorizationTools(input: {- readonly authoredTools: HarnessToolMap;- readonly context?: ContextReader;-}): HarnessToolMap {- const tools = new Map<string, HarnessToolDefinition>();- for (const tool of input.context === undefined ? [] : buildDynamicTools(input.context)) {- if (!tools.has(tool.name)) tools.set(tool.name, tool);- }- for (const [name, tool] of input.authoredTools) {- if (!tools.has(name)) tools.set(name, tool);- }- return tools;-}-+/** Builds live dynamic tool definitions. Narrower scopes appear first so they win on name collision. */export function buildDynamicTools(ctx: ContextReader): readonly HarnessToolDefinition[] {const step = replayDynamicTools(requireCurrentDynamicToolMetadata(ctx.get(StepDynamicToolMetadataKey) ?? []),@@ -291,3 +274,10 @@ export function buildDynamicTools(ctx: ContextReader): readonly HarnessToolDefin);return [...step, ...turn, ...session];}++/** The names of the dynamic tools every scope holds. */+export function dynamicToolNames(ctx: ContextReader): readonly string[] {+ return [StepDynamicToolMetadataKey, TurnDynamicToolMetadataKey, SessionDynamicToolMetadataKey]+ .flatMap((key) => ctx.get(key) ?? [])+ .map((metadata) => metadata.name);+}diff --git a/packages/eve/src/context/dynamic-connection-lifecycle.test.ts b/packages/eve/src/context/dynamic-connection-lifecycle.test.tsindex 01c4461b4..d6aee7be0 100644--- a/packages/eve/src/context/dynamic-connection-lifecycle.test.ts+++ b/packages/eve/src/context/dynamic-connection-lifecycle.test.ts@@ -2,8 +2,13 @@ import { describe, expect, it } from "vitest";import { ContextContainer } from "#context/container.js";import { dispatchDynamicConnectionEvent } from "#context/dynamic-connection-lifecycle.js";-import { AuthKey, SessionIdKey } from "#context/keys.js";+import {+ dispatchDynamicSubagentEvent,+ getDynamicSubagentSelection,+} from "#context/dynamic-subagent-lifecycle.js";+import { AuthKey, SessionIdKey, StaticModelReferenceKey } from "#context/keys.js";import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import { defineAgent } from "#public/definitions/agent.js";import { defineMcpClientConnection } from "#public/definitions/connections/mcp.js";import { defineOpenAPIConnection } from "#public/definitions/connections/openapi.js";import { createSessionStartedEvent, createTurnStartedEvent } from "#protocol/message.js";@@ -13,6 +18,8 @@ import type {ResolvedDynamicConnectionResolver,} from "#runtime/types.js";import { captureLogRecords } from "#internal/testing/log-records.js";+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import type { ResolvedDynamicSubagentResolver } from "#runtime/subagents/registry.js";describe("dynamic connection lifecycle", () => {it("resolves a mixed connection map with bare map-key names", async () => {@@ -303,6 +310,154 @@ describe("dynamic connection lifecycle", () => {);});+ it("fails when a dynamic connection would own an existing tool's name", async () => {+ const { ctx, registry } = createContext();+ ctx.set(BundleKey, agentBundle({ tools: ["billing__refund"] }));+ const resolver = createResolver({+ handler: () => ({+ billing: defineMcpClientConnection({+ description: "Billing.",+ url: "https://mcp.example.com/billing",+ }),+ }),+ });++ await expect(+ dispatchDynamicConnectionEvent({+ ctx,+ event: createSessionStartedEvent(),+ resolvers: [resolver],+ }),+ ).rejects.toThrow(+ 'Tool or subagent "billing__refund" starts with "billing__", which belongs to connection "billing". Rename it, or the dynamic connection that "connections/accounts.ts" returned.',+ );+ expect(registry.getConnectionNames()).toEqual([]);+ });++ it("rejects a turn's dynamic connection named after a dynamic subagent the session selected", async () => {+ const { ctx, registry } = createContext();+ ctx.set(StaticModelReferenceKey, { id: "openai/gpt-root" });+ const billing: ResolvedDynamicSubagentResolver = {+ eventNames: ["session.started"],+ events: {+ "session.started": () =>+ defineAgent({+ description: "Resolve billing disputes.",+ model: "openai/gpt-5.5",+ modelContextWindowTokens: 200_000,+ }),+ },+ kind: "subagent",+ logicalPath: "subagents/billing/agent.ts",+ name: "billing",+ nodeId: "subagents/billing",+ sourceId: "subagents/billing/agent.ts",+ sourceKind: "module",+ };+ ctx.set(BundleKey, agentBundle({ dynamicSubagents: [billing] }));+ await dispatchDynamicSubagentEvent({+ ctx,+ event: createSessionStartedEvent(),+ messages: [],+ resolvers: [billing],+ });+ expect(getDynamicSubagentSelection(ctx, billing.nodeId)).toBeDefined();+ const resolver = createResolver({+ eventNames: ["turn.started"],+ events: {+ "turn.started": () =>+ defineMcpClientConnection({+ description: "Billing.",+ url: "https://mcp.example.com/billing",+ }),+ },+ slug: "billing",+ });++ await expect(+ dispatchDynamicConnectionEvent({+ ctx,+ event: createTurnStartedEvent({ sequence: 1, turnId: "turn_1" }),+ resolvers: [resolver],+ }),+ ).rejects.toThrow(+ 'Tool or subagent "billing" has the same name as connection "billing". Rename it, or the dynamic connection that "connections/billing.ts" returned.',+ );+ expect(registry.getConnectionNames()).toEqual([]);+ });++ it("fails when a dynamic connection is named eve, which would own eve's namespace", async () => {+ const { ctx, registry } = createContext();+ const resolver = createResolver({+ handler: () => ({+ eve: defineMcpClientConnection({+ description: "Runs jobs.",+ url: "https://mcp.example.com/jobs",+ }),+ }),+ });++ await expect(+ dispatchDynamicConnectionEvent({+ ctx,+ event: createSessionStartedEvent(),+ resolvers: [resolver],+ }),+ ).rejects.toThrow(+ 'Dynamic connection resolver "connections/accounts.ts" returned the reserved connection name "eve". eve reserves the "eve" namespace for its built-in tools; rename the connection.',+ );+ expect(registry.getConnectionNames()).toEqual([]);+ });++ it("registers dynamic connections outside eve's namespace, including the catalog tools' former names", async () => {+ const { ctx, registry } = createContext();+ const names = ["search", "execute", "steve", "eve-tools"];+ const resolver = createResolver({+ handler: () =>+ Object.fromEntries(+ names.map((name) => [+ name,+ defineMcpClientConnection({+ description: name,+ url: `https://mcp.example.com/${name}`,+ }),+ ]),+ ),+ });++ await dispatchDynamicConnectionEvent({+ ctx,+ event: createSessionStartedEvent(),+ resolvers: [resolver],+ });++ expect(registry.getConnectionNames().toSorted()).toEqual(names.toSorted());+ });++ it("fails when a dynamic connection nests under another connection's name", async () => {+ const { ctx, registry } = createContext([createStaticConnection("cloud")]);+ const resolver = createResolver({+ extensionNamespace: "cloud",+ handler: () => ({+ production: defineMcpClientConnection({+ description: "Production account.",+ url: "https://mcp.example.com/production",+ }),+ }),+ });++ await expect(+ dispatchDynamicConnectionEvent({+ ctx,+ event: createSessionStartedEvent(),+ resolvers: [resolver],+ }),+ ).rejects.toThrow(+ 'Connection "cloud__production" starts with "cloud__", which belongs to connection "cloud". Rename the dynamic connection "cloud__production".',+ );+ expect(registry.getConnectionNames()).toEqual(["cloud"]);+ });+it("fails closed when a handler returns an unbranded connection", async () => {const logs = captureLogRecords();const { ctx, registry } = createContext();@@ -356,6 +511,20 @@ describe("dynamic connection lifecycle", () => {});});+/** The compiled bundle fields connection ownership reads: authored tools and dynamic subagents. */+function agentBundle(input: {+ readonly dynamicSubagents?: readonly ResolvedDynamicSubagentResolver[];+ readonly tools?: readonly string[];+}) {+ return {+ subagentRegistry: {+ dynamicResolvers: input.dynamicSubagents ?? [],+ subagentsByName: new Map(),+ },+ toolRegistry: { toolsByName: new Map((input.tools ?? []).map((name) => [name, {}])) },+ } as never;+}+function createContext(staticConnections: readonly ResolvedConnectionDefinition[] = []) {const ctx = new ContextContainer();ctx.set(SessionIdKey, "session-1");diff --git a/packages/eve/src/context/dynamic-connection-lifecycle.ts b/packages/eve/src/context/dynamic-connection-lifecycle.tsindex 9c1d37084..0bf539768 100644--- a/packages/eve/src/context/dynamic-connection-lifecycle.ts+++ b/packages/eve/src/context/dynamic-connection-lifecycle.ts@@ -1,15 +1,18 @@import { getAdapterKind } from "#channel/adapter.js";import type { ContextContainer } from "#context/container.js";import { AuthKey, InitiatorAuthKey, SessionIdKey } from "#context/keys.js";+import { assertNotConnectionOwned } from "#connections/ownership.js";+import { dynamicToolNames } from "#context/build-dynamic-tools.js";import { ConnectionRegistryKey } from "#context/providers/connection-key.js";import { ALLOWED_DYNAMIC_CONNECTION_EVENTS } from "#dynamic/definition.js";import { CONNECTION_SLUG_PATTERN } from "#discover/grammar.js";import { createLogger } from "#internal/logging.js";import type { UnstampedMessageStreamEvent } from "#protocol/message.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import { readStampedConnectionProtocol } from "#public/definitions/connections/protocol.js";import type { DynamicConnectionResolveContext } from "#public/definitions/connections/dynamic.js";import { ConnectionRegistryImpl } from "#runtime/connections/registry.js";-import { ChannelKey } from "#runtime/sessions/runtime-context-keys.js";+import { BundleKey, ChannelKey } from "#runtime/sessions/runtime-context-keys.js";import { resolveDynamicConnectionValue } from "#runtime/resolve-connection.js";import type {ResolvedConnectionDefinition,@@ -115,6 +118,7 @@ export async function dispatchDynamicConnectionEvent(input: {})),);const updates = new Map<string, readonly ResolvedConnectionDefinition[]>();+ const entryNames = agentEntryNames(input.ctx);let failedResolver: ResolvedDynamicConnectionResolver | undefined;let failedReason: unknown;for (let index = 0; index < outcomes.length; index += 1) {@@ -131,6 +135,8 @@ export async function dispatchDynamicConnectionEvent(input: {}continue;}+ assertNoReservedNames(outcome.value.connections, resolver);+ assertOwnsNoEntries(outcome.value.connections, resolver, entryNames);updates.set(outcome.value.resolver.slug, outcome.value.connections);}if (failedResolver !== undefined) {@@ -148,6 +154,49 @@ export async function dispatchDynamicConnectionEvent(input: {);}+/**+ * The agent's tool and subagent names, which no connection may own. A dynamic+ * subagent's name counts whether or not its resolver has resolved yet.+ */+function agentEntryNames(ctx: ContextContainer): readonly string[] {+ const bundle = ctx.get(BundleKey);+ return [+ ...(bundle?.toolRegistry.toolsByName.keys() ?? []),+ ...(bundle?.subagentRegistry.subagentsByName.keys() ?? []),+ ...(bundle?.subagentRegistry.dynamicResolvers.map((resolver) => resolver.name) ?? []),+ ...dynamicToolNames(ctx),+ ];+}++function assertNoReservedNames(+ connections: readonly ResolvedConnectionDefinition[],+ resolver: ResolvedDynamicConnectionResolver,+): void {+ for (const { connectionName } of connections) {+ const reservation = eveNamespaceReservation(connectionName);+ if (reservation === undefined) continue;+ throw new Error(+ `Dynamic connection resolver "${resolver.logicalPath}" returned the reserved connection name "${connectionName}". ${reservation}; rename the connection.`,+ );+ }+}++function assertOwnsNoEntries(+ connections: readonly ResolvedConnectionDefinition[],+ resolver: ResolvedDynamicConnectionResolver,+ entryNames: readonly string[],+): void {+ const connectionNames = connections.map((connection) => connection.connectionName);+ for (const name of entryNames) {+ assertNotConnectionOwned({+ connectionNames,+ name,+ remedy: `Rename it, or the dynamic connection that "${resolver.logicalPath}" returned.`,+ subject: "Tool or subagent",+ });+ }+}+function buildConnectionResolveContext(ctx: ContextContainer): DynamicConnectionResolveContext {const channel = ctx.get(ChannelKey);return {diff --git a/packages/eve/src/context/dynamic-skill-lifecycle.test.ts b/packages/eve/src/context/dynamic-skill-lifecycle.test.tsindex 3e1c0f469..7db06729b 100644--- a/packages/eve/src/context/dynamic-skill-lifecycle.test.ts+++ b/packages/eve/src/context/dynamic-skill-lifecycle.test.ts@@ -1,10 +1,7 @@import { describe, expect, it, vi } from "vitest";import { ContextContainer } from "#context/container.js";-import {- PendingSkillAnnouncementKey,- dispatchDynamicSkillEvent,-} from "#context/dynamic-skill-lifecycle.js";+import { dispatchDynamicSkillEvent } from "#context/dynamic-skill-lifecycle.js";import {StaticModelReferenceKey,DynamicSkillManifestKey,@@ -13,6 +10,12 @@ import {SandboxKey,} from "#context/keys.js";import { deserializeContext } from "#context/serialize.js";+import {+ EVE_NAMESPACE_NAMES,+ EVE_NAMESPACE_RESERVATION,+ NAMES_OUTSIDE_EVE_NAMESPACE,+} from "#internal/testing/catalog-fixtures.js";+import { captureLogRecords } from "#internal/testing/log-records.js";import { mockSandbox } from "#internal/testing/mocks/mock-sandbox.js";import type { UnstampedMessageStreamEvent } from "#protocol/message.js";import { defineSkill } from "#public/definitions/skill.js";@@ -118,140 +121,110 @@ function writtenPaths(sandbox: ReturnType<typeof mockSandbox>): string[] {}describe("dispatchDynamicSkillEvent", () => {- it.each(["task.started", "task.settled", "turn.completed"] as const)(- "does not access the sandbox to rebuild announcements on %s",- async (type) => {- const ctx = new ContextContainer();- const manifest = {- policy: [{ name: "policy", description: "Tenant policy", markdown: "Tenant policy" }],- };- ctx.set(DynamicSkillManifestKey, manifest);-- await dispatchDynamicSkillEvent({- ctx,- event: { type, data: {} } as UnstampedMessageStreamEvent,- messages: [],- resolvers: [],- });-- expect(ctx.get(DynamicSkillManifestKey)).toEqual(manifest);- expect(ctx.has(PendingSkillAnnouncementKey)).toBe(false);- },- );-- it("restores the skill announcement at the next model step without resolving skills again", async () => {- const { access, ctx } = createCtx();- const handler = vi.fn(() => makeSkill("Tenant policy"));- const resolver = createResolver("policy", handler);- await dispatchDynamicSkillEvent({- ctx,- event: makeEvent(),- messages: [],- resolvers: [resolver],- });- const announcement = ctx.get(PendingSkillAnnouncementKey);- expect(announcement).toContain("policy: Tenant policy");- ctx.clearVirtualContext();- ctx.setVirtualContext(SandboxKey, access);-- await dispatchDynamicSkillEvent({+ it("stores SKILL.md as authored, including frontmatter", async () => {+ const { ctx } = createCtx();+ await dispatch(ctx,- event: { type: "step.started", data: {} } as UnstampedMessageStreamEvent,- messages: [],- resolvers: [resolver],- });-- expect(ctx.get(PendingSkillAnnouncementKey)).toBe(announcement);- expect(handler).toHaveBeenCalledOnce();- });-- it("announces when all dynamic skills are withdrawn without opening the sandbox", async () => {- const { ctx, get } = createCtx();- let enabled = true;- const resolver = createResolver("tenant", () =>- enabled ? makeSkill("Tenant policy", "Follow tenant policy.") : null,+ createResolver("tenant", () => makeSkill("Tenant", "---\nname: tenant\n---\n# Body\n")),);- await dispatch(ctx, resolver);-- expect(ctx.get(PendingSkillAnnouncementKey)).toContain("- tenant: Tenant policy");- expect(ctx.get(PendingSkillAnnouncementKey)).not.toContain("tenant/SKILL.md");- expect(ctx.get(DynamicSkillManifestKey)).toEqual({- tenant: [{ description: "Tenant policy", markdown: "Follow tenant policy.", name: "tenant" }],- });-- enabled = false;- await dispatch(ctx, resolver);-- expect(ctx.get(DynamicSkillManifestKey)).toEqual({});- expect(ctx.get(PendingSkillAnnouncementKey)).toBe("Available skills: none");- expect(get).not.toHaveBeenCalled();+ expect(ctx.get(DynamicSkillManifestKey)?.tenant?.[0]?.markdown).toBe(+ "---\nname: tenant\n---\n# Body\n",+ );});- it("rebuilds the announcement after a step boundary without opening the sandbox", async () => {- const { ctx, get } = createCtx();+ it("names map entries by their bare key", async () => {+ const { ctx } = createCtx();await dispatch(ctx,- createResolver("tenant", () => makeSkill("Tenant policy", "Body", { "a.md": "a" })),+ createResolver("custom", () => ({+ "talk-like-a-dog": makeSkill("Talk like a dog", "Woof."),+ })),);- get.mockClear();- ctx.clearVirtualContext();- await dispatchDynamicSkillEvent({- ctx,- event: { type: "step.started", data: {} } as UnstampedMessageStreamEvent,- messages: [],- resolvers: [],+ expect(ctx.get(DynamicSkillManifestKey)).toEqual({+ custom: [{ description: "Talk like a dog", markdown: "Woof.", name: "talk-like-a-dog" }],});-- expect(ctx.get(PendingSkillAnnouncementKey)).toContain(- `- tenant: Tenant policy (path: $HOME/.agents/skills/tenant/SKILL.md)`,- );- expect(get).not.toHaveBeenCalled();});- it("stores SKILL.md as authored, including frontmatter", async () => {+ it("accepts a skill name that starts with a digit", async () => {const { ctx } = createCtx();await dispatch(ctx,- createResolver("tenant", () => makeSkill("Tenant", "---\nname: tenant\n---\n# Body\n")),+ createResolver("tenant", () => ({ "3d-modeling": makeSkill("Model in 3D", "Model.") })),);- expect(ctx.get(DynamicSkillManifestKey)?.tenant?.[0]?.markdown).toBe(- "---\nname: tenant\n---\n# Body\n",- );+ expect(ctx.get(DynamicSkillManifestKey)?.tenant?.map((skill) => skill.name)).toEqual([+ "3d-modeling",+ ]);});- it("keeps remaining dynamic skills in the announcement when one resolver removes its skill", async () => {+ it.each(EVE_NAMESPACE_NAMES)(+ "skips a resolver that returns %s, in eve's namespace",+ async (name) => {+ const logs = captureLogRecords();+ const { ctx } = createCtx();+ await dispatch(+ ctx,+ createResolver("tenant", () => ({+ [name]: makeSkill("Reserved"),+ policy: makeSkill("Tenant policy"),+ })),+ );++ expect(ctx.get(DynamicSkillManifestKey)?.tenant).toBeUndefined();+ expect(logs.records).toContainEqual(+ expect.objectContaining({+ fields: {+ error: expect.stringContaining(+ `Dynamic skill resolver "skills/tenant.ts" returned the reserved skill name "${name}". ${EVE_NAMESPACE_RESERVATION}; rename the skill.`,+ ),+ },+ level: "error",+ }),+ );+ },+ );++ it("keeps skills outside eve's namespace, including the built-in tools' former names", async () => {const { ctx } = createCtx();- let tenantEnabled = true;- const tenant = createResolver("tenant", () =>- tenantEnabled ? makeSkill("Tenant policy") : null,+ await dispatch(+ ctx,+ createResolver("tenant", () =>+ Object.fromEntries(NAMES_OUTSIDE_EVE_NAMESPACE.map((name) => [name, makeSkill(name)])),+ ),);- const support = createResolver("support", () => makeSkill("Support policy"));- await dispatch(ctx, tenant, support);- tenantEnabled = false;- await dispatch(ctx, tenant, support);-- const announcement = ctx.get(PendingSkillAnnouncementKey);- expect(announcement).not.toContain("tenant: Tenant policy");- expect(announcement).toContain("support: Support policy");+ expect(ctx.get(DynamicSkillManifestKey)?.tenant?.map((skill) => skill.name)).toEqual(+ NAMES_OUTSIDE_EVE_NAMESPACE,+ );});- it("names map entries by their bare key", async () => {+ it("skips a resolver that returns an illegal skill name", async () => {+ const logs = captureLogRecords();const { ctx } = createCtx();await dispatch(ctx,- createResolver("custom", () => ({- "talk-like-a-dog": makeSkill("Talk like a dog", "Woof."),+ createResolver("tenant", () => ({+ "release notes": makeSkill("Release notes"),+ policy: makeSkill("Tenant policy"),})),+ createResolver("support", () => makeSkill("Support policy")),);expect(ctx.get(DynamicSkillManifestKey)).toEqual({- custom: [{ description: "Talk like a dog", markdown: "Woof.", name: "talk-like-a-dog" }],+ support: [{ description: "Support policy", markdown: "Support policy", name: "support" }],});- expect(ctx.get(PendingSkillAnnouncementKey)).toContain("talk-like-a-dog: Talk like a dog");+ expect(logs.records).toContainEqual(+ expect.objectContaining({+ fields: {+ error: expect.stringContaining(+ 'Dynamic skill resolver "skills/tenant.ts" returned illegal skill name "release notes".',+ ),+ },+ level: "error",+ }),+ );});it("prefixes map entries with the mount namespace for an extension resolver", async () => {@@ -270,7 +243,6 @@ describe("dispatchDynamicSkillEvent", () => {{ description: "Triage an account", markdown: "Triage.", name: "crm__triage" },],});- expect(ctx.get(PendingSkillAnnouncementKey)).toContain("crm__triage: Triage an account");});it("lets a dynamic skill override a same-named authored skill instead of throwing", async () => {@@ -316,7 +288,6 @@ describe("dispatchDynamicSkillEvent", () => {expect(sandbox.writes).toEqual([]);expect(ctx.get(DynamicSkillManifestKey)).toBeUndefined();- expect(ctx.get(PendingSkillAnnouncementKey)).toBeUndefined();});});diff --git a/packages/eve/src/context/dynamic-skill-lifecycle.ts b/packages/eve/src/context/dynamic-skill-lifecycle.tsindex 909750b3b..91c8e6427 100644--- a/packages/eve/src/context/dynamic-skill-lifecycle.ts+++ b/packages/eve/src/context/dynamic-skill-lifecycle.ts@@ -1,5 +1,6 @@import type { ModelMessage } from "ai";+import { SKILL_NAME_PATTERN, SKILL_NAME_RULE } from "#discover/grammar.js";import { ALLOWED_DYNAMIC_SKILL_EVENTS } from "#dynamic/definition.js";import { isBrandedSkillEntry, type SkillPackageDefinition } from "#shared/skill-definition.js";import {@@ -10,14 +11,13 @@ import {writeSkillPackageToSandbox,} from "#shared/skill-package.js";import type { UnstampedMessageStreamEvent } from "#protocol/message.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import type { ResolvedDynamicSkillResolver } from "#runtime/types.js";-import { formatAvailableSkillsSection } from "#execution/skills/instructions.js";import { createLogger } from "#internal/logging.js";import { toErrorMessage } from "#shared/errors.js";import type { ContextContainer } from "#context/container.js";import {type DurableDynamicSkillMetadata,- type DynamicSkillManifest,DynamicSkillManifestKey,DynamicSkillSandboxKey,SandboxKey,@@ -32,31 +32,33 @@ const log = createLogger("dynamic-skills");// ---------------------------------------------------------------------------function qualifyDynamicSkillNames(- resolver: { readonly slug: string; readonly extensionNamespace?: string },+ resolver: ResolvedDynamicSkillResolver,isSingle: boolean,entries: Readonly<Record<string, SkillPackageDefinition>>,-): Array<{ name: string; entryKey: string; entry: SkillPackageDefinition }> {- const keys = Object.keys(entries);- const result: Array<{ name: string; entryKey: string; entry: SkillPackageDefinition }> = [];-- if (keys.length === 0) return result;-+): Array<{ name: string; entry: SkillPackageDefinition }> {// A single returned defineSkill is named after the file slug (already- // namespaced for an extension). A map names each entry by its bare key.- if (isSingle) {- result.push({ name: resolver.slug, entryKey: keys[0]!, entry: entries[keys[0]!]! });- return result;- }-- // Map entries from an extension resolver are prefixed with the mount- // namespace so extension-produced skills are namespaced like the extension's- // static skills; a non-extension resolver's keys stay bare.+ // namespaced for an extension). A map names each entry by its bare key,+ // prefixed with the mount namespace for an extension resolver.const prefix =resolver.extensionNamespace !== undefined ? `${resolver.extensionNamespace}__` : "";- for (const key of keys) {- result.push({ name: `${prefix}${key}`, entryKey: key, entry: entries[key]! });+ const named = Object.entries(entries).map(([key, entry]) => ({+ entry,+ name: isSingle ? resolver.slug : `${prefix}${key}`,+ }));+ for (const { name } of named) {+ const reservation = eveNamespaceReservation(name);+ if (reservation !== undefined) {+ throw new Error(+ `Dynamic skill resolver "${resolver.logicalPath}" returned the reserved skill name "${name}". ${reservation}; rename the skill.`,+ );+ }+ if (!SKILL_NAME_PATTERN.test(name)) {+ throw new Error(+ `Dynamic skill resolver "${resolver.logicalPath}" returned illegal skill name "${name}". ${SKILL_NAME_RULE}`,+ );+ }}- return result;+ return named;}interface DynamicSkillUpdate {@@ -74,19 +76,9 @@ interface SandboxSkillPackage {readonly skill: MaterializableSkillPackage;}-function formatDynamicSkillAnnouncement(manifest: DynamicSkillManifest): string {- const skills = Object.values(manifest)- .flat()- .map(({ description, name, revision }) => ({- description,- hasFiles: revision !== undefined,- name,- }));- return formatAvailableSkillsSection(skills) ?? "Available skills: none";-}-function toDurableSkill(skill: MaterializableSkillPackage): DurableDynamicSkillMetadata {return {+ deferred: skill.deferred === true ? true : undefined,description: skill.description,markdown: skill.markdown,name: skill.name,@@ -146,33 +138,14 @@ async function syncDynamicSkillFiles(input: {ctx.set(DynamicSkillSandboxKey, nextWritten);}-// ----------------------------------------------------------------------------// Single entry detection-// -----------------------------------------------------------------------------// ----------------------------------------------------------------------------// Context key for pending announcements-// -----------------------------------------------------------------------------import { ContextKey } from "#context/key.js";--/**- * Durable pending skill announcement text. Set by- * {@link dispatchDynamicSkillEvent} whenever the dynamic skill manifest- * changes. Read by the tool-loop to inject the announcement into model- * context.- */-export const PendingSkillAnnouncementKey = new ContextKey<string>("eve.pendingSkillAnnouncement");-// ---------------------------------------------------------------------------// Event dispatch// ---------------------------------------------------------------------------/*** Dispatches a stream event to dynamic skill resolvers. On a matching- * event: runs handlers, stores instructions in durable context, syncs- * changed supporting files to the sandbox, and stores a pending- * announcement for the tool-loop to inject.+ * event: runs handlers, stores instructions in durable context, and syncs+ * changed supporting files to the sandbox.*/export async function dispatchDynamicSkillEvent(input: {readonly ctx: ContextContainer;@@ -181,20 +154,6 @@ export async function dispatchDynamicSkillEvent(input: {readonly messages: readonly ModelMessage[];}): Promise<void> {const { ctx, resolvers, event, messages } = input;-- // Subagent event steps do not initialize sandbox access.- // Rebuild announcements only at boundaries that prepare model context.- if (!ALLOWED_DYNAMIC_SKILL_EVENTS.has(event.type) && event.type !== "step.started") return;-- // Build phase: rebuild announcement from durable manifest when the- // virtual key is empty (step boundary crossed).- if (ctx.get(PendingSkillAnnouncementKey) === undefined) {- const manifest = ctx.get(DynamicSkillManifestKey);- if (manifest !== undefined && Object.keys(manifest).length > 0) {- ctx.setVirtualContext(PendingSkillAnnouncementKey, formatDynamicSkillAnnouncement(manifest));- }- }-if (!ALLOWED_DYNAMIC_SKILL_EVENTS.has(event.type)) return;const matching = resolvers.filter((r) => r.eventNames.includes(event.type));@@ -268,7 +227,7 @@ export async function dispatchDynamicSkillEvent(input: {}// A dynamic skill whose name matches an authored skill overrides it:- // load_skill prefers the dynamic body, and supporting files replace the+ // loading prefers the dynamic body, and supporting files replace the// authored package at the same sandbox path. Two dynamic resolvers// emitting the same name is a genuine ambiguity and still throws.const dynamicSkillOwners = new Map<string, string>();@@ -287,5 +246,4 @@ export async function dispatchDynamicSkillEvent(input: {await syncDynamicSkillFiles({ ctx, next, previous });ctx.set(DynamicSkillManifestKey, newManifest);- ctx.setVirtualContext(PendingSkillAnnouncementKey, formatDynamicSkillAnnouncement(newManifest));}diff --git a/packages/eve/src/context/dynamic-subagent-lifecycle.test.ts b/packages/eve/src/context/dynamic-subagent-lifecycle.test.tsindex 0f3f1a8a8..90541cfe5 100644--- a/packages/eve/src/context/dynamic-subagent-lifecycle.test.ts+++ b/packages/eve/src/context/dynamic-subagent-lifecycle.test.ts@@ -16,7 +16,16 @@ import { defineAgent } from "#public/definitions/agent.js";import { defineRemoteAgent } from "#public/definitions/remote-agent.js";import { createSessionStartedEvent, createTurnStartedEvent } from "#protocol/message.js";import type { ResolvedDynamicSubagentResolver } from "#runtime/subagents/registry.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import {+ connectionRegistry,+ fakeConnection,+ EVE_NAMESPACE_NAMES,+ EVE_NAMESPACE_RESERVATION,+ NAMES_OUTSIDE_EVE_NAMESPACE,+} from "#internal/testing/catalog-fixtures.js";import { captureLogRecords } from "#internal/testing/log-records.js";+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";describe("dynamic subagent lifecycle", () => {it("omits a subagent when its resolver returns null", async () => {@@ -452,6 +461,102 @@ describe("dynamic subagent lifecycle", () => {});});+describe("dynamic subagent names", () => {+ /** Resolves a dynamic subagent beside an authored `researcher` tool and a `linear` connection. */+ async function resolveResearcher(input: {+ readonly name?: string;+ readonly tool?: boolean | "deferred";+ }) {+ const logs = captureLogRecords();+ const ctx = createContext();+ ctx.set(BundleKey, {+ toolRegistry: { toolsByName: new Map([["researcher", {}]]) },+ } as never);+ ctx.set(+ ConnectionRegistryKey,+ connectionRegistry([fakeConnection({ name: "linear", tools: [] })]),+ );+ const created = createResolver();+ const resolver: ResolvedDynamicSubagentResolver = {+ ...created.resolver,+ events: {+ "session.started": () =>+ defineAgent({+ description: "Research the request.",+ model: "openai/gpt-5.5",+ modelContextWindowTokens: 200_000,+ tool: input.tool,+ }),+ },+ name: input.name ?? created.resolver.name,+ };+ await dispatchDynamicSubagentEvent({+ ctx,+ event: createSessionStartedEvent(),+ messages: [],+ resolvers: [resolver],+ });+ return {+ errors: logs.records.flatMap((record) =>+ record.level === "error" ? [String(record.fields?.error)] : [],+ ),+ selected: getDynamicSubagentSelection(ctx, resolver.nodeId) !== undefined,+ tools: buildDynamicSubagentTools(ctx).map((tool) => tool.name),+ };+ }++ it.each([true, "deferred"] as const)(+ "omits a subagent with tool %s that takes an authored tool's name",+ async (tool) => {+ expect(await resolveResearcher({ tool })).toEqual({+ errors: [+ 'Dynamic subagent "researcher" from "agent.ts" collides with the tool "researcher". Set the subagent\'s tool to false when that tool wraps it.',+ ],+ selected: false,+ tools: [],+ });+ },+ );++ it("keeps a tool: false subagent that a same-named authored tool wraps", async () => {+ expect(await resolveResearcher({ tool: false })).toEqual({+ errors: [],+ selected: true,+ tools: [],+ });+ });++ it.each(+ EVE_NAMESPACE_NAMES.flatMap((name) => [true, false].map((tool) => [name, tool] as const)),+ )("omits a subagent named %s, in eve's namespace, with tool %s", async (name, tool) => {+ const resolved = await resolveResearcher({ name, tool });++ expect(resolved.selected).toBe(false);+ expect(resolved.errors).toEqual([+ `Dynamic subagent "${name}" from "agent.ts" uses the reserved name "${name}". ${EVE_NAMESPACE_RESERVATION}; rename the subagent.`,+ ]);+ });++ it.each(NAMES_OUTSIDE_EVE_NAMESPACE)(+ "selects a subagent named %s, outside eve's namespace",+ async (name) => {+ expect(await resolveResearcher({ name })).toMatchObject({ errors: [], selected: true });+ },+ );++ it.each([+ [+ "linear__triage",+ 'Dynamic subagent "linear__triage" starts with "linear__", which belongs to connection "linear". Rename the subagent directory.',+ ],+ ])("omits a subagent named %s", async (name, error) => {+ const resolved = await resolveResearcher({ name });++ expect(resolved.selected).toBe(false);+ expect(resolved.errors).toEqual([expect.stringContaining(error)]);+ });+});+function createContext(): ContextContainer {const ctx = new ContextContainer();ctx.set(StaticModelReferenceKey, { id: "openai/gpt-root" });diff --git a/packages/eve/src/context/dynamic-subagent-lifecycle.ts b/packages/eve/src/context/dynamic-subagent-lifecycle.tsindex 0af640f5e..c6639ac7e 100644--- a/packages/eve/src/context/dynamic-subagent-lifecycle.ts+++ b/packages/eve/src/context/dynamic-subagent-lifecycle.ts@@ -1,6 +1,8 @@import type { ModelMessage } from "ai";+import { assertNotConnectionOwned } from "#connections/ownership.js";import type { ContextContainer } from "#context/container.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";import { buildResolveContext } from "#context/dynamic-resolve-context.js";import type { ContextReader } from "#context/key.js";import {@@ -12,11 +14,14 @@ import {import { createPreparedWorkflowToolHarnessDefinition } from "#execution/tools/workflow/harness-definition.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";import { createLogger } from "#internal/logging.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import type { SessionStartedStreamEvent, UnstampedMessageStreamEvent } from "#protocol/message.js";import type { ResolvedDynamicSubagentResolver } from "#runtime/subagents/registry.js";import { createPreparedRuntimeSubagentTool } from "#runtime/subagents/registry.js";import { normalizeDynamicSubagentAgentConfig } from "#runtime/subagents/dynamic-agent-config.js";import { normalizeDynamicRemoteAgentConfig } from "#runtime/subagents/dynamic-remote-agent-config.js";+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";import { toErrorMessage } from "#shared/errors.js";const log = createLogger("dynamic-subagents");@@ -41,6 +46,12 @@ async function resolveSelections(input: {if (result === null || result === undefined) {return [resolver.nodeId, null] as const;}+ assertNotConnectionOwned({+ connectionNames: input.ctx.get(ConnectionRegistryKey)?.getConnectionNames() ?? [],+ name: resolver.name,+ remedy: "Rename the subagent directory.",+ subject: "Dynamic subagent",+ });if (isRemoteAgentDefinition(result)) {const remoteAgent = await normalizeDynamicRemoteAgentConfig({name: resolver.name,@@ -48,6 +59,7 @@ async function resolveSelections(input: {});const effectiveRemoteAgent =resolver.tool === false ? { ...remoteAgent, tool: false as const } : remoteAgent;+ assertToolNameAvailable(input.ctx, resolver, effectiveRemoteAgent.tool);const prepared = createPreparedRuntimeSubagentTool({description: effectiveRemoteAgent.description,kind: "remote",@@ -77,6 +89,7 @@ async function resolveSelections(input: {resolver.tool === false? { ...resolvedAgentConfig, tool: false as const }: resolvedAgentConfig;+ assertToolNameAvailable(input.ctx, resolver, effectiveAgentConfig.tool);const prepared = createPreparedRuntimeSubagentTool({description: effectiveAgentConfig.description,kind: "subagent",@@ -113,6 +126,30 @@ async function resolveSelections(input: {return selections;}+/**+ * No dynamic subagent may take a name in eve's namespace. One the model can+ * call cannot take a tool's name either; with `tool: false` it can, since a+ * same-named authored tool then wraps it.+ */+function assertToolNameAvailable(+ ctx: ContextReader,+ resolver: ResolvedDynamicSubagentResolver,+ tool: AgentToolExposure | undefined,+): void {+ const reservation = eveNamespaceReservation(resolver.name);+ if (reservation !== undefined) {+ throw new Error(+ `Dynamic subagent "${resolver.name}" from "${resolver.logicalPath}" uses the reserved name "${resolver.name}". ${reservation}; rename the subagent.`,+ );+ }+ if (tool === false) return;+ const authoredTools = ctx.get(BundleKey)?.toolRegistry.toolsByName;+ if (authoredTools?.has(resolver.name) !== true) return;+ throw new Error(+ `Dynamic subagent "${resolver.name}" from "${resolver.logicalPath}" collides with the tool "${resolver.name}". Set the subagent's tool to false when that tool wraps it.`,+ );+}+function isRemoteAgentDefinition(value: unknown): boolean {return (typeof value === "object" &&@@ -167,18 +204,11 @@ export function buildDynamicSubagentTools(input: ContextReader): readonly Harnesconst turn = input.get(TurnDynamicSubagentSelectionsKey) ?? {};const effective = { ...session, ...turn };const tools: HarnessToolDefinition[] = [];- const names = new Set<string>();for (const selection of Object.values(effective)) {if (selection === null) {continue;}- if (names.has(selection.prepared.name)) {- throw new Error(- `Found multiple active dynamic subagents named "${selection.prepared.name}". Subagent names must be unique at runtime.`,- );- }- names.add(selection.prepared.name);const modelVisible =selection.kind === "subagent"? selection.agentConfig.tool !== falsediff --git a/packages/eve/src/context/dynamic-tool-lifecycle.test.ts b/packages/eve/src/context/dynamic-tool-lifecycle.test.tsindex c9b28f985..f4b63d78f 100644--- a/packages/eve/src/context/dynamic-tool-lifecycle.test.ts+++ b/packages/eve/src/context/dynamic-tool-lifecycle.test.ts@@ -33,8 +33,7 @@ const {rebindMissingCompiledDynamicToolCallbacks,validateDurableDynamicToolCallbacks,} = await import("#context/dynamic-tool-lifecycle.js");-const { buildDynamicTools, buildResponseAuthorizationTools, replayDynamicTools } =- await import("#context/build-dynamic-tools.js");+const { buildDynamicTools, replayDynamicTools } = await import("#context/build-dynamic-tools.js");import { ContextContainer } from "#context/container.js";import { deserializeContext, serializeContext } from "#context/serialize.js";@@ -58,67 +57,148 @@ import {createStepStartedEvent,type UnstampedMessageStreamEvent,} from "#protocol/message.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import {+ connectionRegistry,+ fakeConnection,+ EVE_NAMESPACE_NAMES,+ EVE_NAMESPACE_RESERVATION,+ NAMES_OUTSIDE_EVE_NAMESPACE,+} from "#internal/testing/catalog-fixtures.js";import { captureLogRecords } from "#internal/testing/log-records.js";--// Re-implement the naming logic here to test it independently-// (the production function is unexported — testing via the public behavior)-function qualifyDynamicToolNames(- slug: string,- isSingle: boolean,- entries: Readonly<Record<string, DynamicToolEntry>>,-): Map<string, DynamicToolEntry> {- const keys = Object.keys(entries);- const result = new Map<string, DynamicToolEntry>();-- if (keys.length === 0) return result;-- // single entry: one tool, named after the file slug.- // map of entries: each named by its bare key.- if (isSingle) {- result.set(slug, entries[keys[0]!]!);- return result;- }-- for (const key of keys) {- result.set(key, entries[key]!);- }- return result;-}+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import { createRuntimeSubagentRegistry } from "#runtime/subagents/registry.js";const executeOptions = { messages: [], toolCallId: "call_1" };-const stubEntry = defineTool({- description: "test",- inputSchema: { type: "object" },- execute: async (): Promise<unknown> => ({}),-});+describe("dynamic tool names", () => {+ const deployService = {+ prepared: {+ behavior: {+ availability: [],+ handling: {+ kind: "dispatch",+ target: {+ entryPoint: "execute",+ kind: "workflow-tool-call",+ workflowId: "workflow//./agent/tools/deploy_service//execute",+ },+ },+ },+ name: "deploy_service",+ },+ };-describe("dynamic tool naming", () => {- it("uses file slug for a single entry", () => {- const names = qualifyDynamicToolNames("analytics", true, {- run: stubEntry,+ /**+ * Resolves `entries` beside a `deploy_service` workflow tool, a `researcher`+ * subagent with the given tool setting, and a `linear` connection.+ */+ async function resolveTools(+ entries: Readonly<Record<string, DynamicToolEntry>>,+ researcherTool?: boolean,+ ) {+ const logs = captureLogRecords();+ const ctx = createCtx();+ ctx.set(BundleKey, {+ subagentRegistry: createRuntimeSubagentRegistry({+ subagents: [+ {+ description: "Research the request.",+ kind: "subagent",+ logicalPath: "subagents/researcher",+ name: "researcher",+ nodeId: "subagents/researcher",+ sourceId: "subagents/researcher",+ sourceKind: "module",+ tool: researcherTool,+ },+ ],+ }),+ toolRegistry: { toolsByName: new Map([["deploy_service", deployService]]) },+ } as never);+ ctx.set(+ ConnectionRegistryKey,+ connectionRegistry([fakeConnection({ name: "linear", tools: [] })]),+ );+ await dispatchDynamicToolEvent({+ ctx,+ event: makeEvent("session.started"),+ messages: [],+ resolvers: [createResolver("tenant", ["session.started"], () => entries)],});- expect([...names.keys()]).toEqual(["analytics"]);- });+ return {+ errors: logs.records.flatMap((record) =>+ record.level === "error" ? [String(record.fields?.error)] : [],+ ),+ tools: buildDynamicTools(ctx).map((tool) => tool.name),+ };+ }- it("uses the bare key for a map entry", () => {- const names = qualifyDynamicToolNames("search", false, {- run: stubEntry,- });- expect([...names.keys()]).toEqual(["run"]);+ it("names map entries by their bare keys", async () => {+ expect(+ await resolveTools({+ tenant__export: createReplayableTool(),+ sync: createReplayableTool(),+ }),+ ).toEqual({ errors: [], tools: ["tenant__export", "sync"] });});- it("uses bare keys for multiple map entries", () => {- const names = qualifyDynamicToolNames("tenant", false, {- export: stubEntry,- query: stubEntry,- });- expect([...names.keys()]).toEqual(["export", "query"]);+ it.each([+ [+ "linear__sync",+ 'Dynamic tool "linear__sync" starts with "linear__", which belongs to connection "linear". Rename the map key.',+ ],+ [+ "linear",+ 'Dynamic tool "linear" has the same name as connection "linear". Rename the map key.',+ ],+ [+ "deploy_service",+ 'Dynamic tool "deploy_service" from resolver "agent/tools/tenant.ts" collides with the workflow tool or agent "deploy_service". Rename the map key.',+ ],+ [+ "researcher",+ 'Dynamic tool "researcher" from resolver "agent/tools/tenant.ts" collides with the workflow tool or agent "researcher". Rename the map key.',+ ],+ [+ "sync now",+ 'Dynamic tool resolver "agent/tools/tenant.ts" returned illegal tool name "sync now".',+ ],+ ])("skips the whole result of a resolver that returns %s", async (name, error) => {+ expect(+ await resolveTools({ [name]: createReplayableTool(), sync: createReplayableTool() }),+ ).toEqual({ errors: [expect.stringContaining(error)], tools: [] });+ });++ it.each(EVE_NAMESPACE_NAMES)(+ "skips a resolver that returns %s, in eve's namespace",+ async (name) => {+ expect(+ await resolveTools({ [name]: createReplayableTool(), sync: createReplayableTool() }),+ ).toEqual({+ errors: [+ `Dynamic tool resolver "agent/tools/tenant.ts" returned the reserved tool name "${name}". ${EVE_NAMESPACE_RESERVATION}; rename the map key.`,+ ],+ tools: [],+ });+ },+ );++ it("keeps tools outside eve's namespace, including the built-in tools' former names", async () => {+ expect(+ await resolveTools(+ Object.fromEntries(+ NAMES_OUTSIDE_EVE_NAMESPACE.map((name) => [name, createReplayableTool()]),+ ),+ ),+ ).toEqual({ errors: [], tools: NAMES_OUTSIDE_EVE_NAMESPACE });});- it("handles empty entries — no tools produced", () => {- const names = qualifyDynamicToolNames("empty", false, {});- expect([...names.keys()]).toEqual([]);+ it("lets a dynamic tool wrap a subagent hidden with tool: false", async () => {+ expect(await resolveTools({ researcher: createReplayableTool() }, false)).toEqual({+ errors: [],+ tools: ["researcher"],+ });});});@@ -1500,7 +1580,7 @@ describe("programmatic dynamic tools (no bundler transform)", () => {const ctx = createCtx();const executeFn = vi.fn(() => ({ data: "from-programmatic" }));const resolver = createResolver("fwk", ["session.started"], () => ({- search: createProgrammaticTool("programmatic search", executeFn),+ lookup: createProgrammaticTool("programmatic search", executeFn),}));await dispatchDynamicToolEvent({@@ -1516,13 +1596,13 @@ describe("programmatic dynamic tools (no bundler transform)", () => {const tools = buildDynamicTools(ctx);expect(tools).toHaveLength(1);- expect(tools[0]!.name).toBe("search");+ expect(tools[0]!.name).toBe("lookup");// Simulate step boundary — virtual context cleared, durable survivesconst replayedTools = buildDynamicTools(ctx);expect(replayedTools).toHaveLength(1);- expect(replayedTools[0]!.name).toBe("search");+ expect(replayedTools[0]!.name).toBe("lookup");// Execute the replayed tool — the original closure is invokedawait replayedTools[0]!.execute!({ query: "test" }, executeOptions);@@ -1559,7 +1639,7 @@ describe("programmatic dynamic tools (no bundler transform)", () => {it("framework and authored tools coexist in session scope", async () => {const ctx = createCtx();const frameworkResolver = createResolver("fwk", ["session.started"], () => ({- search: createProgrammaticTool("programmatic search"),+ lookup: createProgrammaticTool("programmatic search"),}));const authoredResolver = createResolver("authored", ["session.started"], () => ({query: createReplayableTool("authored query"),@@ -1577,7 +1657,7 @@ describe("programmatic dynamic tools (no bundler transform)", () => {const tools = buildDynamicTools(ctx);expect(tools).toHaveLength(2);- expect(tools.map((t) => t.name).sort()).toEqual(["query", "search"]);+ expect(tools.map((t) => t.name).sort()).toEqual(["lookup", "query"]);});it("single-entry programmatic tool uses slug as name", async () => {@@ -1910,53 +1990,6 @@ describe("programmatic dynamic tools (no bundler transform)", () => {getDynamicCallbackRegistry().delete("guarded");});- it("uses the first dynamic definition for response authorization", () => {- const ctx = createCtx();- registerTestCallback("guarded", "execute", () => null);- registerTestCallback("guarded", "approvalRequest", () => "user-approval");- registerTestCallback(- "guarded",- "approvalResponse",- async () => ({ status: "allowed" }) as const,- );- ctx.set(StepDynamicToolMetadataKey, [- {- callbacks: {- approvalRequest: { closure: {} },- approvalResponse: { closure: {} },- execute: { closure: {} },- },- description: "step",- entryKey: "step:guarded",- inputSchema: { type: "object" },- name: "guarded",- resolverSlug: "step",- },- ]);- ctx.set(SessionDynamicToolMetadataKey, [- {- callbacks: {- approvalRequest: { closure: {} },- approvalResponse: { closure: {} },- execute: { closure: {} },- },- description: "session",- entryKey: "session:guarded",- inputSchema: { type: "object" },- name: "guarded",- resolverSlug: "session",- },- ]);-- const tools = buildResponseAuthorizationTools({- authoredTools: new Map(),- context: ctx,- });-- expect(tools.get("guarded")?.description).toBe("step");- getDynamicCallbackRegistry().delete("guarded");- });-it("rejects an untransformed tool atomically without resolver hydration", async () => {const logs = captureLogRecords();const ctx = createCtx();@@ -2121,7 +2154,7 @@ describe("dynamic callback binding isolation", () => {const resolver = createResolver("conditional", ["session.started"], (_event, rawContext) => {const context = rawContext as { session: { id: string } };return {- search: variantTool(+ lookup: variantTool(context.session.id === first.require(SessionIdKey) ? "guarded" : "open","original",),@@ -2140,7 +2173,7 @@ describe("dynamic callback binding isolation", () => {if (cold) {clearDurableDynamicCallbacks(first.require(SessionIdKey));const changedCapture = createResolver("conditional", ["session.started"], () => ({- search: variantTool("guarded", "recaptured"),+ lookup: variantTool("guarded", "recaptured"),}));await refreshDynamicSessionToolsForRuntimeRevision({ctx: restored,@@ -2157,7 +2190,7 @@ describe("dynamic callback binding isolation", () => {});expect(tool!.approvalKey!({})).toBe("guarded");await expect(- resolveApprovalPolicy(tool!.approval!)(createApprovalContext({ toolName: "search" })),+ resolveApprovalPolicy(tool!.approval!)(createApprovalContext({ toolName: "lookup" })),).resolves.toBe("user-approval");expect(await tool!.toModelOutput!({})).toEqual({ type: "text", value: "guarded" });const approval = tool!.approval!;@@ -2181,7 +2214,7 @@ describe("dynamic callback binding isolation", () => {ctx,resolvers: [createResolver(slug, ["session.started"], () => ({- search: variantTool(implementation, slug),+ lookup: variantTool(implementation, slug),})),],event: makeEvent("session.started"),@@ -2201,7 +2234,7 @@ describe("dynamic callback binding isolation", () => {ctx,resolvers: [createResolver("shared-owner", [eventName], () => ({- search: variantTool(scope, scope),+ lookup: variantTool(scope, scope),})),],event: makeEvent(eventName),@@ -2219,7 +2252,7 @@ describe("dynamic callback binding isolation", () => {const first = createCtx();const second = createCtx();const resolver = createResolver("conditional", ["session.started"], () => ({- search: variantTool("guarded", "original"),+ lookup: variantTool("guarded", "original"),}));for (const ctx of [first, second]) {await dispatchDynamicToolEvent({@@ -2247,7 +2280,7 @@ describe("dynamic callback binding isolation", () => {const first = createCtx();const second = createCtx();const resolver = createResolver("conditional", ["session.started"], () => ({- search: variantTool("guarded", "original"),+ lookup: variantTool("guarded", "original"),}));for (const ctx of [first, second]) {await dispatchDynamicToolEvent({@@ -2331,7 +2364,7 @@ describe("dynamic callback cache recovery", () => {const ctx = createCtx();const eventName = `${scope}.started`;const resolver = createResolver("evicted", [eventName], () => ({- search: createReplayableTool(),+ lookup: createReplayableTool(),}));await dispatchDynamicToolEvent({ctx,@@ -2341,7 +2374,7 @@ describe("dynamic callback cache recovery", () => {});ctx.set(SessionDynamicToolRuntimeRevisionKey, "stable");for (let index = 0; index < 1_024; index++) {- registerTestCallback("search", "execute", () => ({ wrong: true }), {+ registerTestCallback("lookup", "execute", () => ({ wrong: true }), {sessionId: `cache-pressure-${index}`,scope,resolverSlug: "evicted",diff --git a/packages/eve/src/context/dynamic-tool-lifecycle.ts b/packages/eve/src/context/dynamic-tool-lifecycle.tsindex 4373a4a86..6e63259a3 100644--- a/packages/eve/src/context/dynamic-tool-lifecycle.ts+++ b/packages/eve/src/context/dynamic-tool-lifecycle.ts@@ -25,6 +25,10 @@ import type {StepStartedStreamEvent,UnstampedMessageStreamEvent,} from "#protocol/message.js";+import { assertNotConnectionOwned } from "#connections/ownership.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import { TOOL_SLUG_PATTERN, TOOL_SLUG_RULE } from "#discover/grammar.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import { ALLOWED_DYNAMIC_TOOL_EVENTS } from "#dynamic/definition.js";import { isBrandedToolEntry, type DynamicToolEntry } from "#tools/dynamic.js";import {@@ -43,6 +47,8 @@ import { toErrorMessage } from "#shared/errors.js";import { parseJsonObject } from "#shared/json.js";import { hasSchemaValidator } from "#tools/durable-schema.js";import { serializeInputSchema, serializeOutputSchema } from "#tools/schema.js";+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import { workflowIdForHandling } from "#runtime/subagents/workflow-reference.js";import type { ResolvedDynamicToolResolver } from "#runtime/types.js";const log = createLogger("dynamic-tools");@@ -61,11 +67,21 @@ function qualifyDynamicToolNames(const prefix =resolver.extensionNamespace === undefined ? "" : `${resolver.extensionNamespace}__`;- return keys.map((entryKey) => ({- entry: entries[entryKey]!,- entryKey,- name: `${prefix}${entryKey}`,- }));+ return keys.map((entryKey) => {+ const name = `${prefix}${entryKey}`;+ if (!TOOL_SLUG_PATTERN.test(name)) {+ throw new Error(+ `Dynamic tool resolver "${resolver.logicalPath}" returned illegal tool name "${name}". ${TOOL_SLUG_RULE}`,+ );+ }+ const reservation = eveNamespaceReservation(name);+ if (reservation !== undefined) {+ throw new Error(+ `Dynamic tool resolver "${resolver.logicalPath}" returned the reserved tool name "${name}". ${reservation}; rename the map key.`,+ );+ }+ return { entry: entries[entryKey]!, entryKey, name };+ });}function durableKeyForEvent(@@ -316,6 +332,7 @@ function createMetadata(input: {}): CurrentDynamicToolMetadata {return {availableInSubagents: input.entry.availableInSubagents,+ deferred: input.entry.deferred,callbacks: validateDurableDynamicToolCallbacks(input.name, input.entry, {sessionId: input.sessionId,scope: input.scope,@@ -344,6 +361,24 @@ function readDynamicEndsTurn(name: string, endsTurn: DynamicToolEntry["endsTurn"return endsTurn === true;}+/**+ * The agent's workflow tools and agents. They run after the model step, so a+ * dynamic tool, which runs in it, cannot take their names. A static subagent+ * hidden with `tool: false` or `disableTool()` can be wrapped by one.+ */+function workflowToolNames(ctx: AlsContext): ReadonlySet<string> {+ const bundle = ctx.get(BundleKey);+ if (bundle === undefined) return new Set();+ const { subagentRegistry, toolRegistry } = bundle;+ return new Set([+ ...[...toolRegistry.toolsByName.values()]+ .filter(({ prepared }) => workflowIdForHandling(prepared.behavior?.handling) !== undefined)+ .map(({ prepared }) => prepared.name),+ ...subagentRegistry.preparedTools.map((tool) => tool.name),+ ...subagentRegistry.dynamicResolvers.map((resolver) => resolver.name),+ ]);+}+interface ResolvedDynamicToolEvent {readonly metadata: readonly CurrentDynamicToolMetadata[];}@@ -366,6 +401,21 @@ async function resolveToolsFromEvent(if (rawResult === null || rawResult === undefined) return null;const { entries, isSingle } = readDynamicToolResult(resolver, rawResult);const named = qualifyDynamicToolNames(resolver, isSingle, entries);+ const connectionNames = ctx.get(ConnectionRegistryKey)?.getConnectionNames() ?? [];+ const workflowNames = workflowToolNames(ctx);+ for (const { name } of named) {+ assertNotConnectionOwned({+ connectionNames,+ name,+ remedy: "Rename the map key.",+ subject: "Dynamic tool",+ });+ if (workflowNames.has(name)) {+ throw new Error(+ `Dynamic tool "${name}" from resolver "${resolver.logicalPath}" collides with the workflow tool or agent "${name}". Rename the map key.`,+ );+ }+ }return {metadata: named.map(({ name, entryKey, entry }) =>createMetadata({ entry, entryKey, name, resolver, sessionId, scope }),diff --git a/packages/eve/src/context/dynamic-tool-metadata.ts b/packages/eve/src/context/dynamic-tool-metadata.tsindex 6014a14aa..7176d5ec3 100644--- a/packages/eve/src/context/dynamic-tool-metadata.ts+++ b/packages/eve/src/context/dynamic-tool-metadata.ts@@ -4,6 +4,7 @@ import type { DurableDynamicToolCallbacks } from "#tools/durable-callbacks.js";interface DynamicToolMetadataBase {readonly frameworkTool?: boolean;readonly availableInSubagents?: boolean;+ readonly deferred?: boolean;readonly name: string;readonly description: string;readonly endsTurn?: boolean;diff --git a/packages/eve/src/context/keys.ts b/packages/eve/src/context/keys.tsindex 2e04676cf..8502a83da 100644--- a/packages/eve/src/context/keys.ts+++ b/packages/eve/src/context/keys.ts@@ -114,7 +114,6 @@ export const ChannelDeliveryKey = new ContextKey<ChannelDeliveryMetadata>("eve.cexport const TurnDeliveryIdsKey = new ContextKey<readonly string[]>("eve.turnDeliveryIds");/** Last framework announcements recorded in the retained session history. */export interface HistoryState {- readonly availableSkills?: string;/** Last announced value per keyed announcement (see `#harness/announcements.js`). */readonly announcements?: Readonly<Record<string, string>>;}@@ -331,8 +330,10 @@ export const DynamicSubagentAgentConfigKey = new ContextKey<DynamicSubagentAgent*/export interface DurableDynamicSkillMetadata {readonly name: string;+ /** Listed in the catalog instead of the dynamic skill announcement. */+ readonly deferred?: true;readonly description: string;- /** `SKILL.md` content as authored; `load_skill` strips any frontmatter. */+ /** `SKILL.md` content as authored; loading strips any frontmatter. */readonly markdown: string;/*** Content hash of the package files. Present only for packages with@@ -345,8 +346,8 @@ export type DynamicSkillManifest = Readonly<Record<string, readonly DurableDynam/*** Durable map from resolver slug to the qualified skills it last produced.- * Used to diff on re-resolution, serve `load_skill`, and rebuild the- * model-visible announcement across turns without a sandbox.+ * Used to diff on re-resolution, load skills, and rebuild the model-visible+ * announcement across turns without a sandbox.*/export const DynamicSkillManifestKey = new ContextKey<DynamicSkillManifest>("eve.dynamicSkillManifest",diff --git a/packages/eve/src/context/memory-tools.ts b/packages/eve/src/context/memory-tools.tsindex 1048e2a7c..cd48ad676 100644--- a/packages/eve/src/context/memory-tools.ts+++ b/packages/eve/src/context/memory-tools.ts@@ -2,7 +2,6 @@ import type { MemoryDefinition, MemoryToolSet, MemoryToolsContext } from "#publiimport { resolveApprovalPolicy } from "#approval/definition.js";import { loadContext } from "#context/container.js";import { TurnMemoryLocksKey } from "#context/keys.js";-import { TOOL_SLUG_PATTERN } from "#discover/grammar.js";import { defineDynamic } from "#dynamic/definition.js";import { markDynamicCallbackRebind } from "#internal/dynamic-tool-rebind.js";import { parseJsonObject, type JsonObject } from "#shared/json.js";@@ -33,11 +32,6 @@ export function createMemoryToolDynamicDefinition(definition: MemoryDefinition,return Object.fromEntries(Object.entries(result).map(([key, tool]) => {const name = `${slot}__${key}`;- if (!TOOL_SLUG_PATTERN.test(name)) {- throw new Error(- `Memory provider tool name "${name}" must start with an ASCII letter, contain only letters, digits, underscores, or dashes, and be at most 64 characters.`,- );- }if (!isBrandedToolEntry(tool)) {throw new Error(`Memory provider tool "${name}" must be created with defineTool().`,diff --git a/packages/eve/src/discover/grammar.test.ts b/packages/eve/src/discover/grammar.test.tsindex 5dd1320be..22fb62bb5 100644--- a/packages/eve/src/discover/grammar.test.ts+++ b/packages/eve/src/discover/grammar.test.ts@@ -1,7 +1,29 @@import { describe, expect, it } from "vitest";+import {+ EVE_NAMESPACE_NAMES,+ EVE_NAMESPACE_RESERVATION,+ NAMES_OUTSIDE_EVE_NAMESPACE,+} from "../internal/testing/catalog-fixtures.js";import { buildMemoryAgentProject } from "../internal/testing/memory-agent-source.js";-import { discoverNamedSourceDirectory, readSortedDirectoryEntries } from "./grammar.js";+import {+ createExtensionNameDiagnostic,+ discoverNamedSourceDirectory,+ readSortedDirectoryEntries,+} from "./grammar.js";++describe("createExtensionNameDiagnostic", () => {+ it.each(EVE_NAMESPACE_NAMES)("rejects an extension mounted as %s, in eve's namespace", (name) => {+ expect(createExtensionNameDiagnostic(name, `extensions/${name}.ts`)).toMatchObject({+ message: `Extension mount "${name}" uses the reserved name "${name}". Rename it; ${EVE_NAMESPACE_RESERVATION}.`,+ sourcePath: `extensions/${name}.ts`,+ });+ });++ it.each(NAMES_OUTSIDE_EVE_NAMESPACE)("accepts an extension mounted as %s", (name) => {+ expect(createExtensionNameDiagnostic(name, `extensions/${name}.ts`)).toBeNull();+ });+});/*** Direct coverage for the unified discovery helper. The channels, tools,diff --git a/packages/eve/src/discover/grammar.ts b/packages/eve/src/discover/grammar.tsindex b8804d5c9..08c02c4e7 100644--- a/packages/eve/src/discover/grammar.ts+++ b/packages/eve/src/discover/grammar.ts@@ -3,6 +3,7 @@ import { join } from "node:path";import type { ModuleSourceRef } from "#shared/source-ref.js";import type { InstructionsDefinition } from "#public/definitions/instructions.js";import { lowerInstructionsMarkdown } from "#internal/helpers/markdown.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import {createDiscoverErrorDiagnostic,createDiscoverWarningDiagnostic,@@ -121,6 +122,22 @@ export const DISCOVER_EXTENSION_NAME_INVALID = "discover/extension-name-invalid"*/export const TOOL_SLUG_PATTERN = /^[a-zA-Z][a-zA-Z0-9_-]{0,63}$/;+/** {@link TOOL_SLUG_PATTERN} in words, for errors about a name that breaks it. */+export const TOOL_SLUG_RULE =+ "Expected ASCII letters, digits, underscores, and dashes only, starting with a letter, up to 64 characters.";++/**+ * Skill name charset: {@link TOOL_SLUG_PATTERN}, except that a name may start+ * with a digit, so a skill such as `3d-modeling` from the Agent Skills+ * standard ports over as-is. A skill is loaded by name, never called as a+ * tool, so tool-name limits on the first character don't apply.+ */+export const SKILL_NAME_PATTERN = /^[a-zA-Z0-9][a-zA-Z0-9_-]{0,63}$/;++/** {@link SKILL_NAME_PATTERN} in words, for errors about a name that breaks it. */+export const SKILL_NAME_RULE =+ "Expected ASCII letters, digits, underscores, and dashes only, starting with a letter or digit, up to 64 characters.";+/*** Connection filename charset. Connections use the same lowercase* kebab-case rule as sandbox since they are not directly exposed to@@ -430,9 +447,7 @@ export function createToolNameDiagnostic(return createDiscoverErrorDiagnostic({code: DISCOVER_TOOL_NAME_INVALID,- message:- `Tool filename "${slotName}" is not a legal tool name. ` +- `Expected ASCII letters, digits, underscores, and dashes only, starting with a letter, up to 64 characters.`,+ message: `Tool filename "${slotName}" is not a legal tool name. ${TOOL_SLUG_RULE}`,sourcePath,});}@@ -519,6 +534,14 @@ export function createExtensionNameDiagnostic(slotName: string,sourcePath: string,): DiscoverDiagnostic | null {+ const reservation = eveNamespaceReservation(slotName);+ if (reservation !== undefined) {+ return createDiscoverErrorDiagnostic({+ code: DISCOVER_EXTENSION_NAME_INVALID,+ message: `Extension mount "${slotName}" uses the reserved name "${slotName}". Rename it; ${reservation}.`,+ sourcePath,+ });+ }if (EXTENSION_SLUG_PATTERN.test(slotName)) {return null;}diff --git a/packages/eve/src/discover/manifest.ts b/packages/eve/src/discover/manifest.tsindex eaf8902b2..ea72bb47a 100644--- a/packages/eve/src/discover/manifest.ts+++ b/packages/eve/src/discover/manifest.ts@@ -285,6 +285,7 @@ export interface CreateAgentSourceManifestInput {*/interface CreateSkillPackageSourceRefInput {assetsPath?: string;+ deferred?: boolean;description: string;license?: string;logicalPath: string;@@ -445,6 +446,7 @@ export function createSkillPackageSourceRef(): NamedSkillDefinition & SkillPackageSourceRef {const skillSourceRef: NamedSkillDefinition & SkillPackageSourceRef = {assetsPath: input.assetsPath,+ deferred: input.deferred,description: input.description,license: input.license,logicalPath: normalizeLogicalPath(input.logicalPath),diff --git a/packages/eve/src/discover/skills.integration.test.ts b/packages/eve/src/discover/skills.integration.test.tsindex f7e5d1fa5..af7abb9c6 100644--- a/packages/eve/src/discover/skills.integration.test.ts+++ b/packages/eve/src/discover/skills.integration.test.ts@@ -9,6 +9,7 @@ import {DISCOVER_SKILL_ENTRY_NOT_DIRECTORY,DISCOVER_SKILL_FRONTMATTER_INVALID,DISCOVER_SKILL_MARKDOWN_MISSING,+ DISCOVER_SKILL_NAME_INVALID,discoverSkills,} from "#discover/skills.js";@@ -240,4 +241,37 @@ describe("discoverSkills (memory)", () => {DISCOVER_SKILL_COLLISION,]);});++ it("accepts a skill name that starts with a digit and rejects one that is not a legal name", async () => {+ const project = buildMemoryAgentProject({+ agentFiles: {+ "skills/2fa-reset/SKILL.md": [+ "---",+ "description: Reset two-factor auth.",+ "---",+ "Reset.",+ ].join("\n"),+ "skills/release notes.md": [+ "---",+ "description: Write release notes.",+ "---",+ "Write.",+ ].join("\n"),+ },+ });++ const result = await discoverSkills({ agentRoot: project.agentRoot, source: project.source });++ expect(+ result.skills.map((skill) => ("skillId" in skill ? skill.skillId : skill.logicalPath)),+ ).toEqual(["2fa-reset"]);+ expect(result.diagnostics).toEqual([+ expect.objectContaining({+ code: DISCOVER_SKILL_NAME_INVALID,+ message: expect.stringContaining(+ 'Skill "skills/release notes.md" has an illegal name "release notes".',+ ),+ }),+ ]);+ });});diff --git a/packages/eve/src/discover/skills.ts b/packages/eve/src/discover/skills.tsindex 32e28efff..497fd1410 100644--- a/packages/eve/src/discover/skills.ts+++ b/packages/eve/src/discover/skills.ts@@ -10,7 +10,11 @@ import {getSupportedModuleBaseName,normalizeLogicalPath,} from "#discover/filesystem.js";-import { readSortedDirectoryEntries } from "#discover/grammar.js";+import {+ readSortedDirectoryEntries,+ SKILL_NAME_PATTERN,+ SKILL_NAME_RULE,+} from "#discover/grammar.js";import {createModuleSourceRef,createPathDerivedSourceId,@@ -27,6 +31,7 @@ export const DISCOVER_SKILL_COLLISION = "discover/skill-collision";export const DISCOVER_SKILL_ENTRY_NOT_DIRECTORY = "discover/skill-entry-not-directory";export const DISCOVER_SKILL_FRONTMATTER_INVALID = "discover/skill-frontmatter-invalid";export const DISCOVER_SKILL_MARKDOWN_MISSING = "discover/skill-markdown-missing";+export const DISCOVER_SKILL_NAME_INVALID = "discover/skill-name-invalid";/*** Input for discovering authored skills.@@ -110,6 +115,17 @@ export async function discoverSkills(input: DiscoverSkillsInput): Promise<Discovcontinue;}+ if (!SKILL_NAME_PATTERN.test(discoveredSkill.skillId)) {+ diagnostics.push(+ createDiscoverErrorDiagnostic({+ code: DISCOVER_SKILL_NAME_INVALID,+ message: `Skill "${discoveredSkill.logicalPath}" has an illegal name "${discoveredSkill.skillId}". ${SKILL_NAME_RULE}`,+ sourcePath: join(skillsDirectoryPath, entry.name),+ }),+ );+ continue;+ }+if (collidedSkillIds.has(discoveredSkill.skillId)) {continue;}@@ -256,6 +272,7 @@ async function discoverPackagedSkill(input: {const packagePaths = await discoverSkillPackagePaths(input.source, input.skillRootPath);const skillSourceRefInput: {assetsPath?: string;+ deferred?: boolean;description: string;license?: string;logicalPath: string;@@ -283,6 +300,10 @@ async function discoverPackagedSkill(input: {skillSourceRefInput.assetsPath = packagePaths.assetsPath;}+ if (definition.deferred !== undefined) {+ skillSourceRefInput.deferred = definition.deferred;+ }+if (definition.license !== undefined) {skillSourceRefInput.license = definition.license;}diff --git a/packages/eve/src/dynamic/definition.ts b/packages/eve/src/dynamic/definition.tsindex 37022f7d3..b88768d98 100644--- a/packages/eve/src/dynamic/definition.ts+++ b/packages/eve/src/dynamic/definition.ts@@ -113,8 +113,9 @@ export type DynamicSentinel<TResult = unknown> = {* handlers. It is shared across tools, skills, connections, and agent definitions;* the directory it is authored in (not this function) decides what each* handler must return and which events are honored. The file's path-derived- * slug names the single-entry case; a `Record<string, ...>` return names- * entries `slug__key`. Return `null` to contribute nothing for that event.+ * slug names the single-entry case; a `Record<string, ...>` return names each+ * entry by its bare key, prefixed with the mount namespace for an extension's+ * resolver. Return `null` to contribute nothing for that event.** Per-slot return shape:* - `agent/tools/`: return a single `defineTool(...)`, a@@ -153,9 +154,10 @@ export type DynamicSentinel<TResult = unknown> = {** A single return is named after the file slug. A map names each entry by its* bare key — there is no automatic slug prefix, so namespace keys yourself- * (e.g. `team__playbook`) when a bare name might collide. A dynamic tool/skill- * whose name matches an authored one overrides it; two dynamic resolvers- * emitting the same name is an error.+ * (e.g. `team__playbook`) when a bare name might collide. Tool names must match+ * the tool filename charset, and a name under a connection's `<name>__` prefix+ * is rejected. A dynamic tool/skill whose name matches an authored one+ * overrides it; two dynamic resolvers emitting the same name is an error.*/export function defineDynamic<const TEvents extends DynamicEvents>(definition: {readonly events: TEvents;diff --git a/packages/eve/src/evals/assertions/collector.ts b/packages/eve/src/evals/assertions/collector.tsindex 4e00a0053..6cf6c3abc 100644--- a/packages/eve/src/evals/assertions/collector.ts+++ b/packages/eve/src/evals/assertions/collector.ts@@ -7,6 +7,7 @@ import type {AssertionSeverity,EveEvalTaskResult,} from "#evals/types.js";+import type { TargetTools } from "#evals/target.js";import type { EveEvalAssertionSubject } from "#evals/assertions/run.js";export type AssertionOutcome = AssertionEvaluation;@@ -46,6 +47,13 @@ export class AssertionCollector {readonly #entries: MutableEntry[] = [];readonly #pending: Promise<void>[] = [];+ /** The target's root agent tools, which tool-name assertions are checked against. */+ readonly tools: TargetTools | undefined;++ constructor(tools?: TargetTools) {+ this.tools = tools;+ }+/** Whether the eval has already recorded an assertion. */get hasEntries(): boolean {return this.#entries.length > 0;diff --git a/packages/eve/src/evals/assertions/run.test.ts b/packages/eve/src/evals/assertions/run.test.tsindex 1455945c8..df1079c22 100644--- a/packages/eve/src/evals/assertions/run.test.ts+++ b/packages/eve/src/evals/assertions/run.test.ts@@ -398,25 +398,6 @@ describe("run assertions", () => {).toBe(0);});- it("loadedSkill matches a load_skill call by skill id", async () => {- const result = makeResult({- derived: {- toolCalls: [- {- ...completedToolCall("load_skill"),- input: { skill: "custom__talk-like-a-dog" },- },- ],- toolCallCount: 1,- },- });- expect((await Run.loadedSkill("custom__talk-like-a-dog").evaluate(result)).score).toBe(1);- expect((await Run.loadedSkill("talk-like-a-dog").evaluate(result)).score).toBe(0);- expect(Run.loadedSkill("custom__talk-like-a-dog").name).toBe(- "loadedSkill(custom__talk-like-a-dog)",- );- });-it("usedNoTools passes only with zero tool calls", async () => {expect((await Run.usedNoTools().evaluate(makeResult({}))).score).toBe(1);expect(diff --git a/packages/eve/src/evals/assertions/run.ts b/packages/eve/src/evals/assertions/run.tsindex 34822d7df..445f3a60e 100644--- a/packages/eve/src/evals/assertions/run.ts+++ b/packages/eve/src/evals/assertions/run.ts@@ -12,7 +12,6 @@ import {type EveEvalToolCallMatchOptions,} from "#evals/match.js";import type { EveEvalEventMatch } from "#evals/match.js";-import { LOAD_SKILL_TOOL_NAME } from "#runtime/skills/fragment-context.js";import type { AssertionOutcome, RunAssertion } from "#evals/assertions/collector.js";/** Minimal captured scope consumed by deterministic eval assertions. */@@ -117,16 +116,28 @@ export function calledTool(name: string, options: EveEvalToolCallMatchOptions =}/**- * Sugar over {@link calledTool} for the framework `load_skill` tool: asserts a- * skill with id `skill` was loaded. `output`/`status`/`count` constrain the- * matching call exactly as for `calledTool`.+ * Asserts the skill `skill` was loaded. `output`/`status`/`count` constrain+ * the matching load exactly as for `calledTool`.*/export function loadedSkill(skill: string,options: EveEvalSkillLoadMatchOptions = {},): RunAssertion {- const base = calledTool(LOAD_SKILL_TOOL_NAME, { ...options, input: { skill } });- return { ...base, name: `loadedSkill(${skill})` };+ validateCount(options.count);+ return {+ name: `loadedSkill(${skill})`,+ evaluate(result) {+ const named = result.derived.skillLoads.filter((load) => load.skill === skill);+ const matching = named.filter((load) => toolCallMatches(load, options));+ if (matchesCount(options.count, matching.length)) {+ return { score: 1, metadata: { matchingLoads: matching.length } };+ }+ const loaded = result.derived.skillLoads.map((load) => load.skill).join(", ");+ return fail(+ `expected ${describeCount(options.count)} matching load(s) of "${skill}", found ${matching.length}; observed skill loads: [${loaded}]`,+ );+ },+ };}/**diff --git a/packages/eve/src/evals/assertions/scoped.test.ts b/packages/eve/src/evals/assertions/scoped.test.tsnew file mode 100644index 000000000..47788ed7f--- /dev/null+++ b/packages/eve/src/evals/assertions/scoped.test.ts@@ -0,0 +1,63 @@+import { describe, expect, it } from "vitest";++import { AssertionCollector } from "#evals/assertions/collector.js";+import { createScopedAssertions } from "#evals/assertions/scoped.js";+import type { TargetTools } from "#evals/target.js";++function assertionsFor(tools: TargetTools | undefined) {+ return createScopedAssertions(new AssertionCollector(tools), {+ timing: "final",+ select: (result) => result,+ });+}++describe("tool-name assertions", () => {+ it.each([+ ["load_skill", 'Tool name "load_skill" was retired; use t.loadedSkill(name).'],+ ["task_wait", 'Tool name "task_wait" was retired; use eve__task_wait.'],+ ["task_cancel", 'Tool name "task_cancel" was retired; use eve__task_cancel.'],+ ["final_output", 'Tool name "final_output" was retired; use eve__reply.'],+ ["connection_search", 'Tool name "connection_search" was retired; use eve__search.'],+ [+ "connection_execute",+ 'Tool name "connection_execute" was retired; assert on the connection tool\'s own name, `<connection>__<tool>`.',+ ],+ ])("reject %s when the root agent has no tool by that name", (name, message) => {+ const t = assertionsFor({ dynamic: false, static: ["lookup"] });++ expect(() => t.calledTool(name)).toThrow(message);+ expect(() => t.notCalledTool(name)).toThrow(message);+ expect(() => t.toolOrder(["lookup", name])).toThrow(message);+ expect(() => t.notCalledTool("lookup")).not.toThrow();+ });++ it("accept a retired name the root agent authors, that a dynamic tool resolver may return, or when its tools are unknown", () => {+ expect(() =>+ assertionsFor({ dynamic: false, static: ["task_wait"] }).notCalledTool("task_wait"),+ ).not.toThrow();+ expect(() =>+ assertionsFor({ dynamic: true, static: [] }).calledTool("load_skill"),+ ).not.toThrow();+ expect(() => assertionsFor(undefined).toolOrder(["final_output"])).not.toThrow();+ });++ it.each([+ [+ "eve__tool",+ "Calls through eve__tool are reported under the entry's name; assert on the tool or agent it reaches.",+ ],+ [+ "eve__skill",+ "Calls through eve__skill are reported as skill loads; use t.loadedSkill(name).",+ ],+ [+ "eve__execute",+ "eve__execute isn't a tool eve registers (it's reserved for code mode); assert on the tool or agent a call reaches, or use t.loadedSkill(name).",+ ],+ ])("always reject %s, which no call is reported under", (name, message) => {+ expect(() => assertionsFor(undefined).notCalledTool(name)).toThrow(message);+ expect(() => assertionsFor({ dynamic: true, static: [name] }).calledTool(name)).toThrow(+ message,+ );+ });+});diff --git a/packages/eve/src/evals/assertions/scoped.ts b/packages/eve/src/evals/assertions/scoped.tsindex b100e86c7..a3f89489e 100644--- a/packages/eve/src/evals/assertions/scoped.ts+++ b/packages/eve/src/evals/assertions/scoped.ts@@ -2,6 +2,7 @@ import { AssertionCollector, type RunAssertion } from "#evals/assertions/collectimport * as RunAssertions from "#evals/assertions/run.js";import type { EveEvalAssertionSubject } from "#evals/assertions/run.js";import type { EveEvalEventMatch } from "#evals/match.js";+import { assertReportedToolName } from "#evals/reported-tool-name.js";import type {EveEvalAssertions,EveEvalOutputAssertions,@@ -24,15 +25,19 @@ export function createScopedAssertions(scope: AssertionScope,): EveEvalAssertions {const record = createRecorder(collector, scope);+ const toolName = (name: string) => {+ assertReportedToolName(name, collector.tools);+ return name;+ };return {succeeded: () => record(RunAssertions.succeeded()),parked: () => record(RunAssertions.parked()),messageIncludes: (token) => record(RunAssertions.messageIncludes(token)),- calledTool: (name, options) => record(RunAssertions.calledTool(name, options)),+ calledTool: (name, options) => record(RunAssertions.calledTool(toolName(name), options)),loadedSkill: (skill, options) => record(RunAssertions.loadedSkill(skill, options)),- notCalledTool: (name) => record(RunAssertions.notCalledTool(name)),- toolOrder: (names) => record(RunAssertions.toolOrder(names)),+ notCalledTool: (name) => record(RunAssertions.notCalledTool(toolName(name))),+ toolOrder: (names) => record(RunAssertions.toolOrder(names.map(toolName))),usedNoTools: () => record(RunAssertions.usedNoTools()),maxToolCalls: (max) => record(RunAssertions.maxToolCalls(max)),calledSubagent: (name, options) => record(RunAssertions.calledSubagent(name, options)),diff --git a/packages/eve/src/evals/index.ts b/packages/eve/src/evals/index.tsindex 7a463e086..24e24cb35 100644--- a/packages/eve/src/evals/index.ts+++ b/packages/eve/src/evals/index.ts@@ -41,6 +41,7 @@ export type {EveEvalSession,EveEvalSessionResult,EveEvalScheduleDispatchResult,+ EveEvalSkillLoad,EveEvalSubagentCall,EveEval,EveEvalConfig,diff --git a/packages/eve/src/evals/match.ts b/packages/eve/src/evals/match.tsindex 6e3b1151b..183b1d302 100644--- a/packages/eve/src/evals/match.ts+++ b/packages/eve/src/evals/match.ts@@ -48,9 +48,9 @@ export interface EveEvalToolCallMatchOptions {}/**- * Constraints applied to a `load_skill` call by `t.loadedSkill`. Identical to- * {@link EveEvalToolCallMatchOptions} without `input`, which the helper fixes to- * the loaded skill id.+ * Constraints applied to a skill load by `t.loadedSkill`. Identical to+ * {@link EveEvalToolCallMatchOptions} without `input`, since the skill name is+ * the whole input.*/export type EveEvalSkillLoadMatchOptions = Omit<EveEvalToolCallMatchOptions, "input">;@@ -129,11 +129,12 @@ export function matchesValue(matcher: unknown, value: unknown): boolean {}/**- * Returns true when one derived tool call satisfies the `input`/`output`/- * lifecycle constraints (the `count` option is the caller's concern).+ * Returns true when one derived tool call or skill load satisfies the+ * `input`/`output`/lifecycle constraints (the `count` option is the caller's+ * concern).*/export function toolCallMatches(- call: EveEvalToolCall,+ call: Pick<EveEvalToolCall, "output" | "status"> & { readonly input?: JsonObject },options: EveEvalToolCallMatchOptions,): boolean {if (options.input !== undefined && !matchesValue(options.input, call.input)) return false;diff --git a/packages/eve/src/evals/reported-tool-name.ts b/packages/eve/src/evals/reported-tool-name.tsnew file mode 100644index 000000000..6fa8845dd--- /dev/null+++ b/packages/eve/src/evals/reported-tool-name.ts@@ -0,0 +1,49 @@+import { CALL_TOOL_NAME, SEARCH_TOOL_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";+import { TASK_CANCEL_TOOL_NAME, TASK_WAIT_TOOL_NAME } from "#protocol/task-tools.js";+import type { TargetTools } from "#evals/target.js";++/**+ * Names in the reserved `eve` namespace that no call is reported under, so an+ * assertion on one always throws: no agent can declare a tool by that name.+ */+const UNREPORTED_TOOL_NAMES = new Map([+ [+ CALL_TOOL_NAME,+ `Calls through ${CALL_TOOL_NAME} are reported under the entry's name; assert on the tool or agent it reaches.`,+ ],+ [+ SKILL_TOOL_NAME,+ `Calls through ${SKILL_TOOL_NAME} are reported as skill loads; use t.loadedSkill(name).`,+ ],+ [+ "eve__execute",+ "eve__execute isn't a tool eve registers (it's reserved for code mode); assert on the tool or agent a call reaches, or use t.loadedSkill(name).",+ ],+]);++/** Framework tool names eve no longer uses, with what to assert instead. */+const RETIRED_TOOL_NAMES = new Map([+ ["connection_execute", "assert on the connection tool's own name, `<connection>__<tool>`"],+ ["connection_search", `use ${SEARCH_TOOL_NAME}`],+ ["final_output", `use ${REPLY_TOOL_NAME}`],+ ["load_skill", "use t.loadedSkill(name)"],+ ["task_cancel", `use ${TASK_CANCEL_TOOL_NAME}`],+ ["task_wait", `use ${TASK_WAIT_TOOL_NAME}`],+]);++/**+ * Throws for a tool name no call is reported under, so an assertion on it can't+ * pass or fail without saying why. A retired name outside the `eve` namespace+ * is still an ordinary tool name, so it passes when the root agent declares a+ * tool by that name, has a dynamic tool resolver that might return one, or its+ * tools are unknown.+ */+export function assertReportedToolName(name: string, tools: TargetTools | undefined): void {+ const unreported = UNREPORTED_TOOL_NAMES.get(name);+ if (unreported !== undefined) throw new TypeError(unreported);+ const replacement = RETIRED_TOOL_NAMES.get(name);+ if (replacement === undefined || tools === undefined) return;+ if (tools.dynamic || tools.static.includes(name)) return;+ throw new TypeError(`Tool name "${name}" was retired; ${replacement}.`);+}diff --git a/packages/eve/src/evals/runner/derive-run-facts.test.ts b/packages/eve/src/evals/runner/derive-run-facts.test.tsindex 97f779e41..5ac1ce04b 100644--- a/packages/eve/src/evals/runner/derive-run-facts.test.ts+++ b/packages/eve/src/evals/runner/derive-run-facts.test.ts@@ -137,6 +137,7 @@ describe("deriveRunFacts", () => {expect(facts).toEqual({toolCalls: [],toolCallCount: 0,+ skillLoads: [],subagentCalls: [],subagentCallCount: 0,inputRequests: [],@@ -187,39 +188,6 @@ describe("deriveRunFacts", () => {expect(facts.toolCallCount).toBe(2);});- it("preserves framework skill input in the eval tool-call view", () => {- const facts = derive([- turnStarted("t1", 0),- {- type: "actions.requested",- data: {- actions: [- {- callId: "skill-1",- input: { skill: "research" },- kind: "load-skill",- },- ],- sequence: 1,- stepIndex: 0,- turnId: "t1",- },- },- actionResult({ callId: "skill-1", toolName: "load_skill", output: "Skill body" }),- ]);-- expect(facts.toolCalls).toEqual([- {- input: { skill: "research" },- name: "load_skill",- output: "Skill body",- sessionId: undefined,- status: "completed",- turnIndex: 0,- },- ]);- });-it("uses the normalized failed lifecycle status for error results", () => {const events: UnstampedMessageStreamEvent[] = [actionsRequested([{ callId: "c1", toolName: "bash" }]),diff --git a/packages/eve/src/evals/runner/derive-run-facts.ts b/packages/eve/src/evals/runner/derive-run-facts.tsindex 38bee80d0..40fdab151 100644--- a/packages/eve/src/evals/runner/derive-run-facts.ts+++ b/packages/eve/src/evals/runner/derive-run-facts.ts@@ -3,11 +3,16 @@ import type {MessageStreamEvent,TaskSettledStreamEvent,} from "#protocol/message.js";-import { LOAD_SKILL_TOOL_NAME } from "#runtime/skills/fragment-context.js";+import { requestedSkill } from "#shared/action-request-name.js";import type { InputRequest } from "#shared/input.js";import type { JsonObject, JsonValue } from "#shared/json.js";import type { TokenUsage } from "#shared/token-usage.js";-import type { EveEvalDerivedFacts, EveEvalSubagentCall, EveEvalToolCall } from "#evals/types.js";+import type {+ EveEvalDerivedFacts,+ EveEvalSkillLoad,+ EveEvalSubagentCall,+ EveEvalToolCall,+} from "#evals/types.js";interface MutableToolCall {name: string;@@ -18,6 +23,8 @@ interface MutableToolCall {sessionId?: string;}+type MutableSkillLoad = { -readonly [K in keyof EveEvalSkillLoad]: EveEvalSkillLoad[K] };+/** One call to an agent task, as `task.started` reports it. */interface AgentCall {readonly callId: string;@@ -30,7 +37,7 @@ interface AgentCall {* Options for {@link deriveRunFacts}.*/export interface DeriveRunFactsOptions {- /** Session id stamped onto every derived tool and subagent call. */+ /** Session id stamped onto every derived tool call, skill load, and subagent call. */readonly sessionId?: string;}@@ -49,8 +56,8 @@ const PARKING_EVENT_TYPES: ReadonlySet<MessageStreamEvent["type"]> = new Set([/*** Extracts derived execution facts from a completed run's stream events.*- * Tool calls pair each `actions.requested` entry with its matching- * `action.result` by call id. Subagent calls are the calls to agent tasks,+ * Tool calls and skill loads pair each `actions.requested` entry with its+ * matching `action.result` by call id. Subagent calls are the calls to agent tasks,* paired with their `task.settled` the same way. These facts power checks,* scorers, and reporters.*/@@ -61,6 +68,7 @@ export function deriveRunFacts(const sessionId = options?.sessionId;const toolCalls: MutableToolCall[] = [];const toolCallsByCallId = new Map<string, MutableToolCall>();+ const skillLoads = new Map<string, MutableSkillLoad>();const agentCalls: AgentCall[] = [];const settledTaskCalls = new Map<string, TaskSettledStreamEvent["data"]>();const agentSessions: AgentStartedStreamEvent["data"][] = [];@@ -101,7 +109,13 @@ export function deriveRunFacts(if (action.kind === "tool-call") {ensureToolCall(action.callId, action.toolName, action.input);} else if (action.kind === "load-skill") {- ensureToolCall(action.callId, LOAD_SKILL_TOOL_NAME, action.input);+ skillLoads.set(action.callId, {+ output: undefined,+ sessionId,+ skill: requestedSkill(action),+ status: "pending",+ turnIndex: Math.max(turnIndex, 0),+ });}}break;@@ -114,6 +128,12 @@ export function deriveRunFacts(call.output = result.output;call.status = status;}+ const load =+ result.kind === "load-skill-result" ? skillLoads.get(result.callId) : undefined;+ if (load !== undefined) {+ load.output = result.output;+ load.status = status;+ }break;}@@ -187,6 +207,7 @@ export function deriveRunFacts(return {toolCalls: toolCalls as readonly EveEvalToolCall[],toolCallCount: toolCalls.length,+ skillLoads: [...skillLoads.values()],subagentCalls,subagentCallCount: subagentCalls.length,inputRequests,@@ -237,6 +258,7 @@ export function createEmptyDerivedFacts(): EveEvalDerivedFacts {return {toolCalls: [],toolCallCount: 0,+ skillLoads: [],subagentCalls: [],subagentCallCount: 0,inputRequests: [],diff --git a/packages/eve/src/evals/runner/execute-task.test.ts b/packages/eve/src/evals/runner/execute-task.test.tsindex 4e1593f29..79bca9246 100644--- a/packages/eve/src/evals/runner/execute-task.test.ts+++ b/packages/eve/src/evals/runner/execute-task.test.ts@@ -801,6 +801,35 @@ describe("executeTask", () => {]);});+ it("fails with an authoring error when an eval requires eve__tool", async () => {+ const server = createScriptedServer([+ {+ sessionId: "session_1",+ events: [+ turnStarted("turn_1"),+ messageCompleted("done", "turn_1"),+ turnCompleted("turn_1"),+ sessionCompleted(),+ ],+ },+ ]);+ vi.spyOn(globalThis, "fetch").mockImplementation(server.fetch);++ const outcome = await executeTask({+ client: new Client({ host: target.url }),+ target,+ evaluation: createTestEval(async (t) => {+ const turn = await t.send("run");+ turn.requireToolCall("eve__tool");+ }, "execute-tool"),+ });++ expect(String(outcome.error)).toContain(+ "Calls through eve__tool are reported under the entry's name",+ );+ expect(outcome.assertions).toEqual([]);+ });+it("captures an explicit skip without an execution error", async () => {const outcome = await executeTask({client: new Client({ host: target.url }),diff --git a/packages/eve/src/evals/runner/execute-task.ts b/packages/eve/src/evals/runner/execute-task.tsindex 9ff257d31..2b2d9a04b 100644--- a/packages/eve/src/evals/runner/execute-task.ts+++ b/packages/eve/src/evals/runner/execute-task.ts@@ -17,7 +17,7 @@ import { createEmptyDerivedFacts } from "#evals/runner/derive-run-facts.js";import { EvalSessionManager } from "#evals/session-manager.js";import type { EvalSessionStartedEvent } from "#evals/session.js";import { createEvalContext } from "#evals/context.js";-import { scopeEvalTargetHandle } from "#evals/target.js";+import { scopeEvalTargetHandle, targetTools } from "#evals/target.js";import { AssertionCollector } from "#evals/assertions/collector.js";import { EvalRequirementFailed, EvalSkipped } from "#evals/control-flow.js";@@ -59,7 +59,7 @@ interface ExecuteTaskResult {export async function executeTask(options: ExecuteTaskOptions): Promise<ExecuteTaskResult> {const { client, evaluation, target, timeoutMs } = options;const signal = timeoutMs !== undefined ? AbortSignal.timeout(timeoutMs) : neverAbortSignal();- const collector = new AssertionCollector();+ const collector = new AssertionCollector(targetTools(target));const manager = new EvalSessionManager({client,collector,@@ -159,6 +159,7 @@ function combineDerivedFacts(sessions: readonly EveEvalSessionResult[]): EveEvalif (sessions.length === 0) return createEmptyDerivedFacts();const toolCalls = sessions.flatMap((session) => session.derived.toolCalls);+ const skillLoads = sessions.flatMap((session) => session.derived.skillLoads);const subagentCalls = sessions.flatMap((session) => session.derived.subagentCalls);const inputRequests = sessions.flatMap((session) => session.derived.inputRequests);const failureCode = sessions.find((session) => session.derived.failureCode !== undefined)?.derived@@ -167,6 +168,7 @@ function combineDerivedFacts(sessions: readonly EveEvalSessionResult[]): EveEvalreturn {toolCalls,toolCallCount: toolCalls.length,+ skillLoads,subagentCalls,subagentCallCount: subagentCalls.length,inputRequests,diff --git a/packages/eve/src/evals/runner/reporters/braintrust.test.ts b/packages/eve/src/evals/runner/reporters/braintrust.test.tsindex 9c4ef1767..07a4498d2 100644--- a/packages/eve/src/evals/runner/reporters/braintrust.test.ts+++ b/packages/eve/src/evals/runner/reporters/braintrust.test.ts@@ -55,6 +55,7 @@ function makeEvalResult(overrides: Partial<EveEvalResult> = {}): EveEvalResult {},],toolCallCount: 1,+ skillLoads: [],subagentCalls: [],subagentCallCount: 0,inputRequests: [],diff --git a/packages/eve/src/evals/runner/reporters/braintrust.ts b/packages/eve/src/evals/runner/reporters/braintrust.tsindex 4dc71b9dd..646640f0d 100644--- a/packages/eve/src/evals/runner/reporters/braintrust.ts+++ b/packages/eve/src/evals/runner/reporters/braintrust.ts@@ -156,6 +156,7 @@ class BraintrustReporter implements EvalReporter {eveSkipReason: result.skipReason,eveToolCalls: result.result.derived.toolCalls.map((call) => call.name),eveSubagentCalls: result.result.derived.subagentCalls.map((call) => call.name),+ eveSkillLoads: result.result.derived.skillLoads.map((load) => load.skill),eveParked: result.result.derived.parked,eveAssertionScores: composeAssertionScoreMetadata(result.assertions),};diff --git a/packages/eve/src/evals/runner/reporters/datadog.ts b/packages/eve/src/evals/runner/reporters/datadog.tsindex 006e9e03a..427c7684c 100644--- a/packages/eve/src/evals/runner/reporters/datadog.ts+++ b/packages/eve/src/evals/runner/reporters/datadog.ts@@ -473,6 +473,7 @@ function resolveResultMetadata(eveSkipReason: result.skipReason,eveToolCalls: result.result.derived.toolCalls.map((call) => call.name),eveSubagentCalls: result.result.derived.subagentCalls.map((call) => call.name),+ eveSkillLoads: result.result.derived.skillLoads.map((load) => load.skill),eveParked: result.result.derived.parked,});if (recordAssertionDetails) {diff --git a/packages/eve/src/evals/session.ts b/packages/eve/src/evals/session.tsindex cb055f02a..287d7a3b7 100644--- a/packages/eve/src/evals/session.ts+++ b/packages/eve/src/evals/session.ts@@ -41,6 +41,7 @@ import type {EveEvalWaitForEventOptions,} from "#evals/types.js";import type { EveEvalInputRequestMatchOptions, EveEvalToolCallMatchOptions } from "#evals/match.js";+import { assertReportedToolName } from "#evals/reported-tool-name.js";/* oxlint-disable typescript/no-unsafe-declaration-merging */@@ -567,6 +568,7 @@ class EvalTurn implements EveEvalTurn {name: string,options: Omit<EveEvalToolCallMatchOptions, "count"> = {},): EveEvalToolCall {+ assertReportedToolName(name, this.#collector.tools);const matching = this.toolCalls.filter((call) => call.name === name && toolCallMatches(call, options),);diff --git a/packages/eve/src/evals/target.test.ts b/packages/eve/src/evals/target.test.tsindex c65f7227e..bb05d7945 100644--- a/packages/eve/src/evals/target.test.ts+++ b/packages/eve/src/evals/target.test.ts@@ -1,7 +1,7 @@import { afterEach, describe, expect, it, vi } from "vitest";import { Client } from "#client/client.js";-import { resolveEvalTargetHandle } from "#evals/target.js";+import { createEvalTargetHandle, resolveEvalTargetHandle, targetTools } from "#evals/target.js";import { createTestAgentInfoResult } from "#internal/testing/agent-info-fixture.js";afterEach(() => {@@ -40,6 +40,17 @@ describe("resolveEvalTargetHandle", () => {expect(target.url).toBe("http://127.0.0.1:3000");expect(target.capabilities).toEqual({ devRoutes: true });+ expect(targetTools(target)).toEqual({ dynamic: false, static: [] });+ expect(+ targetTools(+ createEvalTargetHandle({+ capabilities: { devRoutes: true },+ client,+ kind: "local",+ url: target.url,+ }),+ ),+ ).toBeUndefined();const { dispatchSchedule } = target;await expect(dispatchSchedule("heartbeat")).resolves.toEqual({scheduleId: "heartbeat",diff --git a/packages/eve/src/evals/target.ts b/packages/eve/src/evals/target.tsindex 03e0835c7..793c83237 100644--- a/packages/eve/src/evals/target.ts+++ b/packages/eve/src/evals/target.ts@@ -35,12 +35,30 @@ export async function resolveEvalTargetHandle(input: {);}- return createEvalTargetHandle({+ const target = createEvalTargetHandle({capabilities: capabilitiesFromInfo(info),client: input.client,kind: input.kind,url: input.url,});+ toolsByTarget.set(target, {+ dynamic: info.tools.dynamic.length > 0,+ static: info.tools.static.map((tool) => tool.name),+ });+ return target;+}++/** The root agent's static tool names, and whether it has a dynamic tool resolver. */+export interface TargetTools {+ readonly dynamic: boolean;+ readonly static: readonly string[];+}++const toolsByTarget = new WeakMap<EveEvalTargetHandle, TargetTools>();++/** The tools of a target resolved from `/eve/v1/info`; undefined for a hand-built handle. */+export function targetTools(target: EveEvalTargetHandle): TargetTools | undefined {+ return toolsByTarget.get(target);}export function createEvalTargetHandle(input: {diff --git a/packages/eve/src/evals/types.ts b/packages/eve/src/evals/types.tsindex 9cc457e1f..ec27f98e5 100644--- a/packages/eve/src/evals/types.ts+++ b/packages/eve/src/evals/types.ts@@ -50,6 +50,22 @@ export interface EveEvalToolCall {readonly sessionId?: string;}+/**+ * One skill load extracted from the captured stream, pairing the `load-skill`+ * request with its matching `load-skill-result`.+ */+export interface EveEvalSkillLoad {+ /** The loaded skill's name. */+ readonly skill: string;+ /** The skill's instructions; `undefined` when the load never resolved. */+ readonly output: JsonValue | undefined;+ readonly status: EveEvalActionStatus;+ /** Zero-based index of the turn the load happened in. */+ readonly turnIndex: number;+ /** Owning session id, when the runner knows it. */+ readonly sessionId?: string;+}+/*** One call to an agent task extracted from the captured stream: its* `task.started`, joined with its `task.settled` and the task's@@ -80,6 +96,7 @@ export interface EveEvalSubagentCall {export interface EveEvalDerivedFacts {readonly toolCalls: readonly EveEvalToolCall[];readonly toolCallCount: number;+ readonly skillLoads: readonly EveEvalSkillLoad[];readonly subagentCalls: readonly EveEvalSubagentCall[];readonly subagentCallCount: number;/** Every HITL input request raised during the run (`input.requested`). */@@ -244,7 +261,7 @@ export interface EveEvalAssertions {parked(): AssertionHandle;messageIncludes(token: string | RegExp): AssertionHandle;calledTool(name: string, options?: EveEvalToolCallMatchOptions): AssertionHandle;- /** Sugar for `calledTool("load_skill", { input: { skill }, ... })`. */+ /** Asserts a completed load of `skill`, constrained like `calledTool`. */loadedSkill(skill: string, options?: EveEvalSkillLoadMatchOptions): AssertionHandle;notCalledTool(name: string): AssertionHandle;/** Asserts that tool requests appeared in order, allowing unrelated requests between them. */diff --git a/packages/eve/src/execution/catalog/connection-auth.ts b/packages/eve/src/execution/catalog/connection-auth.tsnew file mode 100644index 000000000..330ca8b08--- /dev/null+++ b/packages/eve/src/execution/catalog/connection-auth.ts@@ -0,0 +1,151 @@+/** Sign-in shared by connection tool calls, searches, and connecting a connection. */++import {+ isConnectionAuthorizationFailedError,+ isConnectionAuthorizationRequiredError,+} from "#connections/errors.js";+import { getAuthorizationResults } from "#harness/authorization.js";+import { createLogger } from "#internal/logging.js";+import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";+import { resolveConnectionAuthorization } from "#runtime/connections/resolve-authorization.js";+import {+ createAuthorizationExecution,+ type ScopedAuthorization,+} from "#runtime/connections/scoped-authorization.js";+import type { ResolvedConnectionDefinition } from "#runtime/types.js";+import {+ supportsInteractiveAuthorization,+ type ConnectionClient,+ type ConnectionToolMetadata,+} from "#shared/connection-types.js";+import { toErrorMessage } from "#shared/errors.js";+import { CALL_TOOL_NAME } from "#protocol/catalog-tools.js";++const log = createLogger("framework.catalog-connections");++/** A connection ready for one call, with any sign-in the user finished completed. */+export interface ConnectionSession {+ /** Completes and starts sign-in for this call, and guards against a sign-in loop. */+ readonly auth: ReturnType<typeof createAuthorizationExecution>;+ readonly client: ConnectionClient;+ /** The connection's interactive sign-in, when it supports one. */+ readonly scoped: ScopedAuthorization | undefined;+}++/**+ * Completes the sign-in the user finished for `connection`, if any, before a+ * search, a sign-in, or a tool call uses its client. It never prompts; a caller+ * that needs sign-in starts it through `auth`.+ */+export async function completeConnectionSignIn(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+): Promise<ConnectionSession> {+ assertPendingSignInInstance(connection);+ const scoped = await interactiveAuthorization(connection);+ const auth = createAuthorizationExecution();+ if (scoped !== undefined) await auth.complete(scoped);+ const client = registry.getClient(connection.connectionName);+ // A client that connected anonymously before sign-in must reconnect with the new token.+ if (scoped !== undefined && auth.isJustAuthorized(scoped)) await client.close();+ return { auth, client, scoped };+}++/** A connection's tools, or why they can't be listed. */+export type ConnectionListing =+ | { readonly tools: readonly ConnectionToolMetadata[] }+ /** Listable once the user signs in; `error` is the listing's request for sign-in. */+ | { readonly authorization: ScopedAuthorization; readonly error: unknown }+ | { readonly failure: string };++export async function listConnectionTools(+ connection: ResolvedConnectionDefinition,+ { auth, client, scoped }: ConnectionSession,+): Promise<ConnectionListing> {+ const name = connection.connectionName;+ try {+ return { tools: await listToolMetadata(name, client) };+ } catch (error) {+ if (!isConnectionAuthorizationRequiredError(error)) {+ log.warn("failed to load connection tools", { connection: name, error });+ return { failure: listingFailureMessage(name, error) };+ }+ if (scoped === undefined) {+ return { failure: `"${name}" requires authorization and cannot start interactive sign-in.` };+ }+ // The token the user just signed in with was refused; asking again would loop.+ if (auth.isJustAuthorized(scoped)) {+ return {+ failure: `Authorization failed for "${name}": the service rejected the token immediately after authorization.`,+ };+ }+ return { authorization: scoped, error };+ }+}++/** How long a connection may take to list its tools before it counts as unavailable. */+const LISTING_TIMEOUT_MS = 10_000;++class ListingTimeoutError extends Error {}++/**+ * A connection's tools, or a timeout error once listing them takes longer than+ * {@link LISTING_TIMEOUT_MS}, so one hung server can't stall a step. Clients+ * share one in-flight listing and take no signal, so this stops waiting rather+ * than cancelling: a late result fills only that client's cache, and the race+ * handles a late rejection.+ */+export async function listToolMetadata(+ connectionName: string,+ client: ConnectionClient,+): Promise<readonly ConnectionToolMetadata[]> {+ let timer: ReturnType<typeof setTimeout> | undefined;+ const timeout = new Promise<never>((_, reject) => {+ timer = setTimeout(() => {+ reject(+ new ListingTimeoutError(+ `"${connectionName}" did not list its tools within ${LISTING_TIMEOUT_MS / 1000}s. Try again later.`,+ ),+ );+ }, LISTING_TIMEOUT_MS);+ });+ try {+ return await Promise.race([client.getToolMetadata(), timeout]);+ } finally {+ clearTimeout(timer);+ }+}++/** Why a connection's tools could not be listed, for a failure other than a needed sign-in. */+export function listingFailureMessage(connectionName: string, error: unknown): string {+ if (error instanceof ListingTimeoutError) return error.message;+ return isConnectionAuthorizationFailedError(error)+ ? `Authorization failed for "${connectionName}": ${error.message}`+ : `Failed to load tools for "${connectionName}": ${toErrorMessage(error)}`;+}++async function interactiveAuthorization(+ connection: ResolvedConnectionDefinition,+): Promise<ScopedAuthorization | undefined> {+ const authorization = await resolveConnectionAuthorization(connection);+ if (authorization === undefined || !supportsInteractiveAuthorization(authorization)) {+ return undefined;+ }+ return {+ scope: connection.connectionName,+ instanceId: connection.instanceId,+ connection: { url: connection.url ?? "" },+ authorization,+ };+}++/** Rejects a finished sign-in for `connection` when its resolved instance changed while pending. */+function assertPendingSignInInstance(connection: ResolvedConnectionDefinition): void {+ for (const result of getAuthorizationResults()) {+ if (result.name !== connection.connectionName || result.instanceId === undefined) continue;+ if (result.instanceId === connection.instanceId) continue;+ throw new Error(+ `Authorization for "${result.name}" cannot complete because its resolved connection changed while sign-in was pending. Sign in again with ${CALL_TOOL_NAME}({ name: "${result.name}" }).`,+ );+ }+}diff --git a/packages/eve/src/execution/catalog/connection-entry.ts b/packages/eve/src/execution/catalog/connection-entry.tsnew file mode 100644index 000000000..2bfbab087--- /dev/null+++ b/packages/eve/src/execution/catalog/connection-entry.ts@@ -0,0 +1,343 @@+/**+ * A connection's catalog entries. `<connection>__<tool>` calls one of its+ * tools: it applies the connection's approval under the entry's name and runs+ * an approved call only against the connection instance it was approved for.+ * The connection's own name signs the user in, so its tools become listable.+ */++import type { StandardSchemaV1 } from "#compiled/@standard-schema/spec/index.js";+import {+ resolveApprovalPolicy,+ type Approval,+ type ApprovalContext,+ type ApprovalPolicy,+ type ApprovalResponseContext,+ type ApprovalResponseDecision,+ type ApprovalResponsePolicy,+ type ApprovalStatus,+} from "#approval/definition.js";+import { isConnectionAuthorizationRequiredError } from "#connections/errors.js";+import { connectionToolName } from "#connections/ownership.js";+import { loadContext } from "#context/container.js";+import { ContextKey } from "#context/key.js";+import { isApprovalRecheck } from "#harness/approval-recheck.js";+import type { AuthorizationSignal } from "#harness/authorization.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";+import {+ connectionToolModelOutput,+ toConnectionToolResult,+} from "#runtime/connections/tool-result.js";+import type { ResolvedConnectionDefinition } from "#runtime/types.js";+import type { ConnectionToolMetadata } from "#shared/connection-types.js";+import { displayProperName, displayTitle } from "#shared/display-name.js";+import { isObject } from "#shared/guards.js";+import type { JsonObject } from "#shared/json.js";+import type { ToolExecuteOptions } from "#tools/definition.js";+import { defineJsonSchema, refineJsonSchema } from "#tools/schema.js";+import { SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";++import {+ completeConnectionSignIn,+ listConnectionTools,+ listingFailureMessage,+ listToolMetadata,+} from "./connection-auth.js";+import { closestNames } from "./rank.js";+import { connectionToolSignature } from "./signatures.js";++/** Bounds approval pins; an evicted pin rejects its call rather than letting it run. */+const MAX_APPROVAL_PINS = 50;++const CONNECTION_CHANGED_MESSAGE =+ "The connection for this tool call changed or is unavailable. Request a new tool call and approval.";++/**+ * Connection instance each parked call was sent for approval against, keyed by+ * call id. When a person approves the call, it runs only if its connection+ * still resolves to that instance; a changed instance or a missing pin (the map+ * is bounded) rejects it.+ */+const ConnectionApprovalPinsKey = new ContextKey<Readonly<Record<string, string>>>(+ "eve.connectionApprovalPins",+);++/** The pin for a connection's instance; one without an id still gets a pin. */+function instanceKey(connection: ResolvedConnectionDefinition): string {+ return connection.instanceId ?? "";+}++/** No instance id matches this, so the call fails when it runs. */+const CHANGED_INSTANCE = "\u0000changed";++/** The entry for one tool of one connection instance in `registry`. */+export function connectionEntry(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+ toolName: string,+): HarnessToolDefinition {+ const label = `${displayProperName(connection.connectionName)}: ${displayTitle(toolName)}`;+ return {+ approval: connectionApproval(connection),+ deferred: true,+ description: "",+ execute: (input: unknown, options: ToolExecuteOptions) =>+ callConnectionTool(registry, connection, toolName, input, options),+ // Checked before approval, so no one is asked to approve a call that cannot run.+ inputSchema: refineJsonSchema({}, (input) => checkInput(registry, connection, toolName, input)),+ label: { start: () => label },+ name: connectionToolName(connection.connectionName, toolName),+ toModelOutput: connectionToolModelOutput,+ };+}++/**+ * The entry for a connection's own name: it signs the user in when listing the+ * connection's tools needs that, then points the model at `eve__search`.+ */+export function connectionSignInEntry(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+): HarnessToolDefinition {+ const displayName = displayProperName(connection.connectionName);+ const signIn = `Sign in to use the ${displayName} tools`;+ return {+ deferred: true,+ description:+ connection.description === "" ? `${signIn}.` : `${signIn}: ${connection.description}`,+ execute: () => connect(registry, connection),+ inputSchema: defineJsonSchema({ type: "object", properties: {}, additionalProperties: false }),+ label: { start: () => `Sign in to ${displayName}` },+ name: connection.connectionName,+ };+}++// ---------------------------------------------------------------------------+// Execution+// ---------------------------------------------------------------------------++async function connect(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+): Promise<string | AuthorizationSignal> {+ const session = await completeConnectionSignIn(registry, connection);+ const listing = await listConnectionTools(connection, session);+ if ("failure" in listing) throw new Error(listing.failure);+ if ("authorization" in listing) {+ return await session.auth.handleError(listing.error, listing.authorization);+ }+ const displayName = displayProperName(connection.connectionName);+ const search = `${SEARCH_TOOL_NAME}({ query: "${connection.connectionName}__" })`;+ // A listable connection may not need sign-in at all, so only a sign-in this call completed counts.+ return session.scoped !== undefined && session.auth.isJustAuthorized(session.scoped)+ ? `Signed in to ${displayName}. Find the ${displayName} tools with ${search}.`+ : `The ${displayName} tools are available. Find them with ${search}.`;+}++async function callConnectionTool(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+ toolName: string,+ rawInput: unknown,+ options: ToolExecuteOptions,+): Promise<unknown> {+ releaseApprovalPin(options.toolCallId, connection);+ const session = await completeConnectionSignIn(registry, connection);+ const { auth, client, scoped } = session;+ const listing = await listConnectionTools(connection, session);+ if ("failure" in listing) throw new Error(listing.failure);+ if ("authorization" in listing) {+ return await auth.handleError(listing.error, listing.authorization);+ }+ // Validation may have run before a sign-in made the tools listable.+ const checked = await checkCall(connection, listing.tools, toolName, rawInput);+ if ("issues" in checked) throw new Error(issuesMessage(checked.issues));+ const { input, tool } = checked;++ let raw: unknown;+ try {+ raw = await client.executeTool(tool.name, input, {+ abortSignal: options.abortSignal,+ callId: options.toolCallId,+ });+ } catch (error) {+ if (isConnectionAuthorizationRequiredError(error)) return await auth.handleError(error, scoped);+ throw error;+ }++ const result = toConnectionToolResult(connection.protocol, tool, raw);+ if (!result.ok) throw new Error(result.error);+ return result.value;+}++async function checkInput(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+ toolName: string,+ input: unknown,+): Promise<StandardSchemaV1.Result<unknown>> {+ let tools: readonly ConnectionToolMetadata[];+ try {+ tools = await listToolMetadata(+ connection.connectionName,+ registry.getClient(connection.connectionName),+ );+ } catch (error) {+ // Only the call itself can start the sign-in that makes its tools listable.+ if (isConnectionAuthorizationRequiredError(error)) return { value: input };+ return { issues: [{ message: listingFailureMessage(connection.connectionName, error) }] };+ }+ const checked = await checkCall(connection, tools, toolName, input);+ return "issues" in checked ? { issues: checked.issues } : { value: checked.input };+}++type CallCheck =+ | { readonly input: JsonObject; readonly tool: ConnectionToolMetadata }+ | { readonly issues: readonly StandardSchemaV1.Issue[] };++/** The tool a call runs and its input with schema defaults filled in, or why it cannot run. */+async function checkCall(+ connection: ResolvedConnectionDefinition,+ tools: readonly ConnectionToolMetadata[],+ toolName: string,+ input: unknown,+): Promise<CallCheck> {+ const tool = tools.find((entry) => entry.name === toolName);+ if (tool === undefined) {+ return { issues: [{ message: unknownToolMessage(connection, toolName, tools) }] };+ }+ const result = await defineJsonSchema(tool.inputSchema as JsonObject)["~standard"].validate(+ isObject(input) ? input : {},+ );+ if (result.issues === undefined) return { input: result.value as JsonObject, tool };+ return {+ issues: [+ ...result.issues,+ { message: `Signature: ${connectionToolSignature(connection, tool)}` },+ ],+ };+}++/** The issues as one error message, each led by the path it is about. */+function issuesMessage(issues: readonly StandardSchemaV1.Issue[]): string {+ return issues+ .map(({ message, path = [] }) => {+ const at = path+ .map((segment) => (typeof segment === "object" ? String(segment.key) : String(segment)))+ .join(".");+ return at.length > 0 ? `${at}: ${message}` : message;+ })+ .join(" ");+}++function unknownToolMessage(+ connection: ResolvedConnectionDefinition,+ toolName: string,+ tools: readonly ConnectionToolMetadata[],+): string {+ const { connectionName } = connection;+ const suggestions = closestNames(toolName, tools).map((name) =>+ connectionToolName(connectionName, name),+ );+ const hint =+ suggestions.length > 0+ ? ` Closest tools: ${suggestions.join(", ")}.`+ : ` Find its tools with ${SEARCH_TOOL_NAME}({ query: "${connectionName}__" }).`;+ return `Connection "${connectionName}" has no tool named "${toolName}".${hint}`;+}++// ---------------------------------------------------------------------------+// Approval+// ---------------------------------------------------------------------------++/**+ * The connection's approval, pinned to this connection instance. The response+ * phase exists only when the connection defines one; without it, the pin is+ * checked when the approved call runs.+ */+function connectionApproval(connection: ResolvedConnectionDefinition): Approval | undefined {+ const { approval } = connection;+ if (approval === undefined) return undefined;+ const policy = resolveApprovalPolicy(approval);+ const request = (context: ApprovalContext) => requestApproval(connection, policy, context);+ const response = typeof approval === "function" ? undefined : approval.response;+ if (response === undefined) return request;+ return {+ request,+ response: (context: ApprovalResponseContext) =>+ authorizeApprovalResponse(connection, response, context),+ };+}++async function requestApproval(+ connection: ResolvedConnectionDefinition,+ policy: ApprovalPolicy,+ context: ApprovalContext,+): Promise<ApprovalStatus> {+ // Denying here would drop the call without a result, so the approved call+ // runs and fails in execution instead, where the failure is reported.+ if (isApprovalRecheck(context) && !matchesApprovalPin(context.callId, connection)) {+ markApprovalPinChanged(context.callId);+ }+ const status = await policy(context);+ if (!isApprovalRecheck(context) && parksForApproval(status)) {+ pinApprovedInstance(context.callId, connection);+ }+ return status;+}++function parksForApproval(status: ApprovalStatus): boolean {+ return (+ status === true ||+ status === "user-approval" ||+ (typeof status === "object" && status.type === "user-approval")+ );+}++async function authorizeApprovalResponse(+ connection: ResolvedConnectionDefinition,+ response: ApprovalResponsePolicy,+ context: ApprovalResponseContext,+): Promise<ApprovalResponseDecision> {+ if (!matchesApprovalPin(context.request.callId, connection)) {+ return { reason: CONNECTION_CHANGED_MESSAGE, status: "rejected" };+ }+ return await response(context);+}++/** Records the instance a parked call was first sent for approval against. */+function pinApprovedInstance(callId: string, connection: ResolvedConnectionDefinition): void {+ const ctx = loadContext();+ if (ctx.get(ConnectionApprovalPinsKey)?.[callId] !== undefined) return;+ ctx.set(ConnectionApprovalPinsKey, (pins = {}) =>+ Object.fromEntries([+ ...Object.entries(pins).slice(-(MAX_APPROVAL_PINS - 1)),+ [callId, instanceKey(connection)],+ ]),+ );+}++/** Fails closed: a parked call whose pin is missing counts as changed. */+function matchesApprovalPin(callId: string, connection: ResolvedConnectionDefinition): boolean {+ const pinned = loadContext().get(ConnectionApprovalPinsKey)?.[callId];+ return pinned !== undefined && pinned === instanceKey(connection);+}++function markApprovalPinChanged(callId: string): void {+ loadContext().set(ConnectionApprovalPinsKey, (pins = {}) => ({+ ...pins,+ [callId]: CHANGED_INSTANCE,+ }));+}++/** Consumes the call's approval pin, rejecting the call if its connection changed. */+function releaseApprovalPin(callId: string, connection: ResolvedConnectionDefinition): void {+ const ctx = loadContext();+ const pins = ctx.get(ConnectionApprovalPinsKey);+ if (pins?.[callId] === undefined) return;+ const { [callId]: pinned, ...rest } = pins;+ ctx.set(ConnectionApprovalPinsKey, rest);+ if (pinned !== instanceKey(connection)) {+ throw new Error(CONNECTION_CHANGED_MESSAGE);+ }+}diff --git a/packages/eve/src/execution/catalog/listing.test.ts b/packages/eve/src/execution/catalog/listing.test.tsnew file mode 100644index 000000000..7d0d27db0--- /dev/null+++ b/packages/eve/src/execution/catalog/listing.test.ts@@ -0,0 +1,266 @@+import { describe, expect, it } from "vitest";++import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import {+ catalogContext,+ fakeConnection,+ inlineTool,+ subagentTool,+ type CatalogSkillSource,+} from "#internal/testing/catalog-fixtures.js";++import { catalogAnnouncements } from "./listing.js";++interface CatalogState {+ readonly connections?: readonly { readonly description?: string; readonly name: string }[];+ readonly skills?: readonly CatalogSkillSource[];+ readonly tools?: readonly HarnessToolDefinition[];+}++function announce(state: CatalogState, announced?: Readonly<Record<string, string>>) {+ const { catalog } = catalogContext({+ connections: (state.connections ?? []).map((connection) =>+ fakeConnection({ ...connection, tools: [] }),+ ),+ skills: state.skills,+ tools: state.tools,+ });+ return catalogAnnouncements(catalog, announced);+}++/** The listing a session's first step appends. */+function baseline(state: CatalogState): string | undefined {+ return announce(state).catalog?.render(undefined);+}++/**+ * What the model reads when the catalog goes from `previous` to `next`. Like+ * the harness, nothing is appended while the recorded value stays the same.+ */+function appended(previous: CatalogState, next: CatalogState): string | undefined {+ const before = announce(previous).catalog!.value;+ const after = announce(next, { catalog: before }).catalog;+ return after === undefined || after.value === before ? undefined : after.render(before);+}++const deferred = (name: string) => inlineTool(name, { deferred: true });++/** `count` deferred tools under `namespace`. */+const namespace = (name: string, count: number) =>+ Array.from({ length: count }, (_, index) => deferred(`${name}__tool_${index}`));++const HEADER_GUIDANCE =+ "than are loaded here. Before saying you have no tool for a task, look for one with eve__search, which searches your own catalog, not the web.";+const NAMESPACES =+ 'Namespaces, whose entries are named <namespace>__<name>; search one with "<namespace>__":';+const CONNECTIONS =+ 'Connections, whose tools are named <connection>__<tool>; search one connection\'s tools with "<connection>__":';++/** A catalog shaped like a real agent's: vendored namespaces, loose entries, and connections. */+const INK: CatalogState = {+ connections: [+ { description: "Pet store inventory API", name: "petstore" },+ { description: "Linear issues and projects", name: "linear" },+ ],+ skills: [+ { deferred: true, name: "sre__incident-runbook" },+ { deferred: true, name: "support__refund-policy" },+ { name: "house-rules" },+ ],+ tools: [+ inlineTool("add"),+ deferred("sre__list_alerts"),+ deferred("sre__page_oncall"),+ deferred("d0__deploy_preview"),+ deferred("index__search_docs"),+ deferred("support__open_ticket"),+ deferred("summarize_usage"),+ subagentTool("billing_specialist", { deferred: true }),+ ],+};++describe("catalogAnnouncements", () => {+ it("names the kinds, namespaces, and connections, but no deferred entry", () => {+ const listing = baseline(INK);++ expect(listing).toBe(+ [+ `You have more tools, agents, and skills ${HEADER_GUIDANCE} Call tools with eve__tool({ name, input }) and load skills with eve__skill({ name }).`,+ `${NAMESPACES} d0, index, sre, support`,+ CONNECTIONS,+ "- linear: Linear issues and projects",+ "- petstore: Pet store inventory API",+ ].join("\n"),+ );+ const deferredNames = [+ ...INK.tools!.filter((tool) => tool.deferred === true).map((tool) => tool.name),+ ...INK.skills!.filter((skill) => skill.deferred === true).map((skill) => skill.name),+ ];+ for (const name of deferredNames) expect(listing).not.toContain(name);+ });++ it.each([+ [+ "tools",+ { tools: [deferred("export_ledger")] },+ `You have more tools ${HEADER_GUIDANCE} Call them with eve__tool({ name, input }).`,+ ],+ [+ "agents",+ { tools: [subagentTool("billing_specialist", { deferred: true })] },+ `You have more agents ${HEADER_GUIDANCE} Call them with eve__tool({ name, input }).`,+ ],+ [+ "skills",+ { skills: [{ deferred: true, name: "pdf-forms" }] },+ `You have more skills ${HEADER_GUIDANCE} Load them with eve__skill({ name }).`,+ ],+ [+ "tools and agents, but no deferred skill",+ {+ skills: [{ name: "house-rules" }],+ tools: [deferred("export_ledger"), subagentTool("researcher", { deferred: true })],+ },+ `You have more tools and agents ${HEADER_GUIDANCE} Call them with eve__tool({ name, input }).`,+ ],+ [+ "skills beside a connection, whose tools eve__tool calls",+ {+ connections: [{ description: "Linear issues", name: "linear" }],+ skills: [{ deferred: true, name: "pdf-forms" }],+ },+ `You have more skills ${HEADER_GUIDANCE} Call tools with eve__tool({ name, input }) and load skills with eve__skill({ name }).`,+ ],+ [+ "only connections",+ { connections: [{ description: "Linear issues", name: "linear" }] },+ `Your connections have more tools ${HEADER_GUIDANCE} Call them with eve__tool({ name, input }).`,+ ],+ ])("names only the kinds present: %s", (_case, state: CatalogState, header) => {+ expect(baseline(state)?.split("\n")[0]).toBe(header);+ });++ it("lists the 20 largest namespaces by name, then 'and more', and a connection's namespace only as the connection", () => {+ const large = Array.from(+ { length: 20 },+ (_, index) => `ns${String(index + 1).padStart(2, "0")}`,+ );+ const listing = baseline({+ connections: [{ description: "Linear issues", name: "linear" }],+ tools: [+ // Smaller than every listed namespace, though first by name.+ ...namespace("aa", 1),+ ...namespace("ab", 1),+ ...large.flatMap((name, index) => namespace(name, 2 + (index % 3))),+ ...namespace("linear", 5),+ ],+ });++ expect(listing?.split("\n")[1]).toBe(`${NAMESPACES} ${large.join(", ")}, and more`);+ expect(listing).toContain("- linear: Linear issues");+ });++ it("caps connections at 20 by name, then 'and more'", () => {+ const connections = Array.from({ length: 21 }, (_, index) => ({+ name: `svc${String(index + 1).padStart(2, "0")}`,+ }));++ const lines = baseline({ connections })!.split("\n");++ expect(lines.slice(2)).toEqual([+ ...connections.slice(0, 20).map(({ name }) => `- ${name}: ${name} service`),+ "- and more",+ ]);+ });++ it("renders the same value for the same catalog, whatever order entries arrive in", () => {+ const reordered = {+ ...INK,+ connections: [...INK.connections!].reverse(),+ tools: [...INK.tools!].reverse(),+ };++ expect(announce(reordered).catalog?.value).toBe(announce(INK).catalog?.value);+ });++ it("announces nothing for a catalog that is empty and was never announced", () => {+ expect(announce({ skills: [{ name: "house-rules" }], tools: [inlineTool("add")] })).toEqual({});+ });++ describe("changes", () => {+ it("appends nothing when an entry joins a listed namespace or has no namespace", () => {+ expect(+ appended(INK, {+ ...INK,+ skills: [...INK.skills!, { deferred: true, name: "sre__postmortem" }],+ tools: [...INK.tools!, deferred("sre__ack_alert"), deferred("export_ledger")],+ }),+ ).toBeUndefined();+ });++ it.each([+ ["a namespace", { ...INK, tools: [...INK.tools!, deferred("billing__refund")] }],+ [+ "a connection",+ { ...INK, connections: [...INK.connections!, { description: "CRM", name: "crm" }] },+ ],+ ])("appends the full listing again when %s appears", (_case, next: CatalogState) => {+ const message = appended(INK, next);++ expect(message).toBe(`The catalog changed.\n${baseline(next)}`);+ });++ it("names a namespace or connection that is gone", () => {+ const next: CatalogState = {+ ...INK,+ connections: INK.connections!.filter(({ name }) => name !== "linear"),+ tools: INK.tools!.filter(({ name }) => !name.startsWith("d0__")),+ };++ expect(appended(INK, next)).toBe(+ `The catalog changed.\n${baseline(next)}\nNo longer available: d0, linear`,+ );+ });++ it("does not call a namespace or connection gone when it only falls past the cap", () => {+ const twenty = Array.from(+ { length: 20 },+ (_, index) => `ns${String(index + 1).padStart(2, "0")}`,+ );+ const previous: CatalogState = {+ connections: twenty.map((name) => ({ name: `svc_${name}` })),+ tools: twenty.flatMap((name) => namespace(name, 2)),+ };+ // A larger namespace and an earlier connection push ns20 and svc_ns20 past their caps.+ const next: CatalogState = {+ connections: [{ name: "svc_ns00" }, ...previous.connections!],+ tools: [...namespace("top", 3), ...previous.tools!],+ };++ const message = appended(previous, next);++ expect(message).toContain(", and more");+ expect(message).toContain("- and more");+ expect(message).not.toContain("No longer available");+ });++ it("says when an announced catalog becomes empty, and lists it in full when entries return", () => {+ const synced = announce({ tools: [deferred("tenant__sync")] }).catalog!.value;+ const empty = announce({ tools: [inlineTool("add")] }, { catalog: synced }).catalog!;++ expect(empty.render(synced)).toBe(+ "The catalog changed. It is empty now: eve__search finds nothing.",+ );+ const returned = { tools: [deferred("tenant__export")] };+ expect(announce(returned, { catalog: empty.value }).catalog?.render(empty.value)).toBe(+ baseline(returned),+ );+ });++ it("treats an unreadable recorded value as a fresh baseline", () => {+ const recorded = "not json";++ expect(announce(INK, { catalog: recorded }).catalog?.render(recorded)).toBe(baseline(INK));+ });+ });+});diff --git a/packages/eve/src/execution/catalog/listing.ts b/packages/eve/src/execution/catalog/listing.tsnew file mode 100644index 000000000..2f59899d8--- /dev/null+++ b/packages/eve/src/execution/catalog/listing.ts@@ -0,0 +1,199 @@+import { isAgentTool } from "#execution/tasks/tool-entry-point.js";+import type { Announcement } from "#harness/announcements.js";+import { CALL_TOOL_NAME, SEARCH_TOOL_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";++import { compareCodeUnits } from "./rank.js";+import type { StepCatalog } from "./step-catalog.js";++const ANNOUNCEMENT_KEY = "catalog";+const MAX_LISTED_NAMESPACES = 20;+const MAX_LISTED_CONNECTIONS = 20;++/** The kinds of deferred entries, as the header names them. */+type Kind = "tools" | "agents" | "skills";++interface ListedConnection {+ readonly description: string;+ readonly name: string;+}++/** A list cut at its cap; `more` says it was cut. */+interface Capped<T> {+ readonly items: readonly T[];+ readonly more: boolean;+}++/**+ * Exactly what the listing says, which is what gets recorded. It names no+ * deferred entry, so deferring keeps entries out of context and the listing+ * doesn't grow with the catalog. It counts nothing, so a dynamic resolver+ * adding or dropping an entry doesn't append a message.+ */+interface CatalogListing {+ readonly connections: Capped<ListedConnection>;+ readonly kinds: readonly Kind[];+ readonly namespaces: Capped<string>;+}++/**+ * Announces the catalog: the kinds of deferred entries, their namespaces, and+ * the connections, then the whole listing again whenever that changes.+ * Connection tools stay unlisted, since listing them needs a network call and+ * maybe a sign-in. `announced` holds the values announced so far.+ */+export function catalogAnnouncements(+ catalog: StepCatalog,+ announced: Readonly<Record<string, string>> | undefined,+): Readonly<Record<string, Announcement>> {+ const current = listCatalog(catalog);+ // An empty catalog the model never heard of has nothing to announce.+ if (announced?.[ANNOUNCEMENT_KEY] === undefined && isEmpty(current)) return {};+ const reachable = reachableNames(catalog);+ return {+ [ANNOUNCEMENT_KEY]: {+ value: JSON.stringify(current),+ render: (previous) => renderCatalogAnnouncement(parseListing(previous), current, reachable),+ },+ };+}++function listCatalog(catalog: StepCatalog): CatalogListing {+ const deferred = [...catalog.deferred.values()];+ const kinds: Kind[] = [];+ if (deferred.some((definition) => !isAgentTool(definition))) kinds.push("tools");+ if (deferred.some(isAgentTool)) kinds.push("agents");+ // Skills that aren't deferred are announced with their descriptions elsewhere.+ if ([...catalog.skills.values()].some(({ deferred }) => deferred)) kinds.push("skills");+ // The largest namespaces make the cut, rendered by name, so an entry joining or leaving one+ // only changes the text when it moves a namespace across the cap.+ const namespaces = capped(+ [...namespaceCounts(catalog)]+ .sort(([a, aCount], [b, bCount]) => bCount - aCount || compareCodeUnits(a, b))+ .map(([namespace]) => namespace),+ MAX_LISTED_NAMESPACES,+ );+ const connections = catalog.connections+ .map(({ connectionName, description }) => ({ description, name: connectionName }))+ .sort((a, b) => compareCodeUnits(a.name, b.name));+ return {+ connections: capped(connections, MAX_LISTED_CONNECTIONS),+ kinds,+ namespaces: { ...namespaces, items: [...namespaces.items].sort(compareCodeUnits) },+ };+}++/** How many deferred entries each namespace holds. */+function namespaceCounts(catalog: StepCatalog): ReadonlyMap<string, number> {+ const connections = new Set(catalog.connections.map(({ connectionName }) => connectionName));+ const counts = new Map<string, number>();+ for (const name of deferredNames(catalog)) {+ const namespace = namespaceOf(name);+ // Only a skill can share a connection's name, since connection ownership doesn't cover skills.+ if (namespace !== "" && !connections.has(namespace)) {+ counts.set(namespace, (counts.get(namespace) ?? 0) + 1);+ }+ }+ return counts;+}++/**+ * The namespaces and connections the session can still reach, deferred or+ * not: a namespace past the cap, or whose entries stopped being deferred, is+ * not gone.+ */+function reachableNames(catalog: StepCatalog): ReadonlySet<string> {+ return new Set([+ ...[...catalog.entries.keys(), ...catalog.skills.keys()].map(namespaceOf),+ ...catalog.connections.map(({ connectionName }) => connectionName),+ ]);+}++/** A name's first `__` segment, or "" when it has none. */+function namespaceOf(name: string): string {+ return name.slice(0, Math.max(0, name.indexOf("__")));+}++function deferredNames(catalog: StepCatalog): readonly string[] {+ return [+ ...[...catalog.deferred.values()].map(({ name }) => name),+ ...[...catalog.skills.values()].filter(({ deferred }) => deferred).map(({ name }) => name),+ ];+}++function capped<T>(items: readonly T[], max: number): Capped<T> {+ return { items: items.slice(0, max), more: items.length > max };+}++function renderCatalogAnnouncement(+ previous: CatalogListing | undefined,+ current: CatalogListing,+ reachable: ReadonlySet<string>,+): string {+ if (isEmpty(current)) {+ return `The catalog changed. It is empty now: ${SEARCH_TOOL_NAME} finds nothing.`;+ }+ if (previous === undefined || isEmpty(previous)) return renderListing(current);+ const gone = [+ ...previous.namespaces.items,+ ...previous.connections.items.map(({ name }) => name),+ ].filter((name) => !reachable.has(name));+ return [+ "The catalog changed.",+ renderListing(current),+ ...(gone.length > 0 ? [`No longer available: ${gone.join(", ")}`] : []),+ ].join("\n");+}++function renderListing({ connections, kinds, namespaces }: CatalogListing): string {+ const subject =+ kinds.length === 0 ? "Your connections have more tools" : `You have more ${joinWords(kinds)}`;+ const call = !kinds.includes("skills")+ ? `Call them with ${CALL_TOOL_NAME}({ name, input }).`+ : kinds.length === 1 && connections.items.length === 0+ ? `Load them with ${SKILL_TOOL_NAME}({ name }).`+ : `Call tools with ${CALL_TOOL_NAME}({ name, input }) and load skills with ${SKILL_TOOL_NAME}({ name }).`;+ const lines = [+ `${subject} than are loaded here. Before saying you have no tool for a task, look for one with ${SEARCH_TOOL_NAME}, which searches your own catalog, not the web. ${call}`,+ ];+ if (namespaces.items.length > 0) {+ const listed = [...namespaces.items, ...(namespaces.more ? ["and more"] : [])];+ lines.push(+ `Namespaces, whose entries are named <namespace>__<name>; search one with "<namespace>__": ${listed.join(", ")}`,+ );+ }+ if (connections.items.length > 0) {+ lines.push(+ 'Connections, whose tools are named <connection>__<tool>; search one connection\'s tools with "<connection>__":',+ ...connections.items.map(formatConnection),+ ...(connections.more ? ["- and more"] : []),+ );+ }+ return lines.join("\n");+}++/** "a", "a and b", or "a, b, and c". */+function joinWords(words: readonly string[]): string {+ return words.length <= 2+ ? words.join(" and ")+ : `${words.slice(0, -1).join(", ")}, and ${words.at(-1)}`;+}++function formatConnection(entry: ListedConnection): string {+ return entry.description.length === 0+ ? `- ${entry.name}`+ : `- ${entry.name}: ${entry.description}`;+}++function isEmpty(listing: CatalogListing): boolean {+ return listing.kinds.length === 0 && listing.connections.items.length === 0;+}++/** The recorded listing; a value that isn't JSON counts as none. */+function parseListing(value: string | undefined): CatalogListing | undefined {+ if (value === undefined) return undefined;+ try {+ return JSON.parse(value) as CatalogListing;+ } catch {+ return undefined;+ }+}diff --git a/packages/eve/src/execution/catalog/mcp-sign-in.test.ts b/packages/eve/src/execution/catalog/mcp-sign-in.test.tsnew file mode 100644index 000000000..74dfc4f91--- /dev/null+++ b/packages/eve/src/execution/catalog/mcp-sign-in.test.ts@@ -0,0 +1,263 @@+import { afterEach, describe, expect, it, vi } from "vitest";+import { CALL_TOOL_NAME, SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";++import { ConnectionAuthorizationRequiredError } from "#connections/errors.js";+import { ContextContainer, contextStorage } from "#context/container.js";+import { AuthKey, SessionIdKey } from "#context/keys.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import {+ CallbackBaseUrlKey,+ isAuthorizationSignal,+ PendingAuthorizationResultKey,+} from "#harness/authorization.js";+import { McpConnectionClient } from "#runtime/connections/mcp-client.js";+import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";+import type { ResolvedConnectionDefinition } from "#runtime/types.js";+import { type ConnectionClient, defineInteractiveAuthorization } from "#shared/connection-types.js";++import { catalogBundle } from "#internal/testing/catalog-fixtures.js";+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import { buildStepCatalog } from "./step-catalog.js";++const URL = "https://kennel.example.com/mcp";+const TOKEN = "signed-in-token";++/** How the fake server treats requests that carry no bearer. */+type ServerMode =+ /** `initialize`/`tools/list` are public; a protected `tools/call` answers HTTP 401. */+ | "public-list-401"+ /** `initialize`/`tools/list` are public; a protected `tools/call` returns an `mcp/www_authenticate` error. */+ | "public-list-meta"+ /** Every request without a bearer answers HTTP 401. */+ | "token-everywhere";++interface SeenRequest {+ readonly authorization: string | null;+ readonly method: string;+}++/**+ * Alice's session with one MCP connection, `kennel`, served by a stubbed+ * `fetch` and reached through the real MCP client, `search`, and `execute`.+ */+function setup(mode: ServerMode) {+ const seen: SeenRequest[] = [];+ vi.stubGlobal("fetch", async (input: string | globalThis.URL | Request, init?: RequestInit) => {+ const request = new Request(input, init);+ if (request.method !== "POST") return new Response(null, { status: 405 });+ const authorization = request.headers.get("authorization");+ const message = (await request.json()) as {+ id?: number;+ method: string;+ params?: { name?: string };+ };+ seen.push({ authorization, method: message.method });+ const signedIn = authorization === `Bearer ${TOKEN}`;+ if (!signedIn && mode === "token-everywhere") {+ return new Response("Unauthorized", { status: 401 });+ }+ if (message.id === undefined) return new Response(null, { status: 202 });+ const reply = (result: unknown) => Response.json({ jsonrpc: "2.0", id: message.id, result });+ switch (message.method) {+ case "initialize":+ return reply({+ capabilities: { tools: {} },+ protocolVersion: "2025-06-18",+ serverInfo: { name: "kennel", version: "1.0.0" },+ });+ case "tools/list":+ return reply({+ tools: [+ { name: "find_pet", description: "Find a pet.", inputSchema: { type: "object" } },+ { name: "pet_invoice", description: "A pet's bill.", inputSchema: { type: "object" } },+ ],+ });+ case "tools/call": {+ if (message.params?.name === "pet_invoice" && !signedIn) {+ if (mode === "public-list-401") return new Response("Unauthorized", { status: 401 });+ return reply({+ _meta: { "mcp/www_authenticate": [`Bearer resource_metadata="${URL}/.well-known"`] },+ content: [{ type: "text", text: "Sign in to see invoices." }],+ isError: true,+ });+ }+ return reply({ content: [{ type: "text", text: `${message.params?.name} ok` }] });+ }+ default:+ return Response.json({+ jsonrpc: "2.0",+ id: message.id,+ error: { code: -32601, message: "Method not found" },+ });+ }+ });++ let token: string | undefined;+ const startAuthorization = vi.fn(async ({ callbackUrl }: { callbackUrl: string }) => ({+ challenge: { url: `https://idp.example.com/authorize?redirect=${callbackUrl}` },+ }));+ const connection = {+ authorization: defineInteractiveAuthorization({+ getToken: async () => {+ if (token === undefined) throw new ConnectionAuthorizationRequiredError("kennel");+ return { token };+ },+ startAuthorization,+ completeAuthorization: async () => {+ token = TOKEN;+ return { token };+ },+ }),+ connectionName: "kennel",+ description: "Pet boarding.",+ protocol: "mcp",+ protocolVersionDiscovery: false,+ url: URL,+ } as ResolvedConnectionDefinition;++ const clients = new Map<string, ConnectionClient>();+ const registry: ConnectionRegistry = {+ dispose: async () => {+ await Promise.all([...clients.values()].map((client) => client.close()));+ },+ getClient: (name) => {+ let client = clients.get(name);+ if (client === undefined) {+ client = new McpConnectionClient(connection);+ clients.set(name, client);+ }+ return client;+ },+ getConnectionApproval: () => undefined,+ getConnectionNames: () => ["kennel"],+ getConnections: () => [connection],+ };+ registries.push(registry);++ const ctx = new ContextContainer();+ ctx.set(AuthKey, {+ attributes: {},+ authenticator: "test",+ principalId: "alice",+ principalType: "user",+ });+ ctx.set(SessionIdKey, "mcp-sign-in");+ ctx.set(CallbackBaseUrlKey, "https://agent.example.com");+ ctx.set(ConnectionRegistryKey, registry);+ ctx.set(BundleKey, catalogBundle({ connections: [connection] }));+ const catalog = buildStepCatalog({ agentTools: new Map(), ctx, endsTurn: true, session: {} });+ const options = { messages: [], toolCallId: "call-1" };++ return {+ seen,+ startAuthorization,+ search: (query: string) =>+ contextStorage.run(ctx, () =>+ catalog.advertised.get(SEARCH_TOOL_NAME)!.execute!({ query }, options),+ ),+ /** Runs `eve__tool({ name, input })` the way the harness runs a resolved call. */+ execute: (tool: string, input: object = {}) => {+ const resolved = catalog.resolve({ input: { input, name: tool }, toolName: CALL_TOOL_NAME });+ if (resolved === undefined) throw new Error(`execute could not resolve "${tool}".`);+ return contextStorage.run(ctx, () =>+ resolved.definition.execute!(resolved.call.input, options),+ );+ },+ /** Delivers the finished sign-in, as the turn step does when its callback arrives. */+ finishSignIn: () =>+ ctx.set(PendingAuthorizationResultKey, [+ {+ callback: { method: "GET", params: { code: "ok" } },+ hookUrl: "https://agent.example.com/callback",+ name: "kennel",+ },+ ]),+ };+}++const registries: ConnectionRegistry[] = [];++afterEach(async () => {+ await Promise.all(registries.splice(0).map((registry) => registry.dispose()));+ vi.unstubAllGlobals();+});++describe("MCP connections sign in only when the server asks", () => {+ it("searches a server that lists its tools anonymously, with no prompt and no sign-in result", async () => {+ const kennel = setup("public-list-401");++ const output = await kennel.search("kennel");++ expect(output).toEqual({+ results: [+ expect.objectContaining({ tool: "kennel__find_pet" }),+ expect.objectContaining({ tool: "kennel__pet_invoice" }),+ ],+ });+ expect(kennel.startAuthorization).not.toHaveBeenCalled();+ expect(kennel.seen.every((request) => request.authorization === null)).toBe(true);+ });++ it.each(["public-list-401", "public-list-meta"] as const)(+ "%s: runs a public tool anonymously, then signs in for a protected tool and calls it with the token",+ async (mode) => {+ const kennel = setup(mode);++ expect(await kennel.execute("kennel__find_pet")).toBe("find_pet ok");+ expect(kennel.startAuthorization).not.toHaveBeenCalled();++ const parked = await kennel.execute("kennel__pet_invoice");+ expect(isAuthorizationSignal(parked)).toBe(true);+ expect(parked).toMatchObject({ challenges: [{ name: "kennel" }] });+ expect(kennel.startAuthorization).toHaveBeenCalledOnce();++ kennel.finishSignIn();+ const before = kennel.seen.length;+ expect(await kennel.execute("kennel__pet_invoice")).toBe("pet_invoice ok");+ const resumed = kennel.seen.slice(before);+ expect(resumed.map((request) => request.method)).toContain("initialize");+ expect(resumed.every((request) => request.authorization === `Bearer ${TOKEN}`)).toBe(true);+ expect(kennel.startAuthorization).toHaveBeenCalledOnce();+ },+ );++ it("reconnects with the token after sign-in, even when a public call reopened an anonymous connection", async () => {+ const kennel = setup("public-list-401");+ await kennel.execute("kennel__pet_invoice");+ // While the sign-in is pending, a public call connects anonymously again.+ expect(await kennel.execute("kennel__find_pet")).toBe("find_pet ok");++ kennel.finishSignIn();+ const before = kennel.seen.length;++ expect(await kennel.execute("kennel__pet_invoice")).toBe("pet_invoice ok");+ const resumed = kennel.seen.slice(before);+ expect(resumed[0]).toEqual({ authorization: `Bearer ${TOKEN}`, method: "initialize" });+ expect(resumed.every((request) => request.authorization === `Bearer ${TOKEN}`)).toBe(true);+ expect(kennel.startAuthorization).toHaveBeenCalledOnce();+ });++ it("finds a server that rejects anonymous listing as its sign-in result, which signs in", async () => {+ const kennel = setup("token-everywhere");++ expect(await kennel.search("kennel__")).toEqual({+ results: [+ {+ description: "Sign in to use the Kennel tools: Pet boarding.",+ signature: "kennel(input: {}): Promise<unknown>",+ tool: "kennel",+ },+ ],+ });+ expect(kennel.startAuthorization).not.toHaveBeenCalled();++ expect(isAuthorizationSignal(await kennel.execute("kennel"))).toBe(true);+ kennel.finishSignIn();+ expect(await kennel.execute("kennel")).toBe(+ 'Signed in to Kennel. Find the Kennel tools with eve__search({ query: "kennel__" }).',+ );+ expect(await kennel.search("kennel__")).toMatchObject({+ results: [{ tool: "kennel__find_pet" }, { tool: "kennel__pet_invoice" }],+ });+ });+});diff --git a/packages/eve/src/execution/catalog/rank.ts b/packages/eve/src/execution/catalog/rank.tsnew file mode 100644index 000000000..7b4e3448f--- /dev/null+++ b/packages/eve/src/execution/catalog/rank.ts@@ -0,0 +1,203 @@+/**+ * Ranking for `eve__search` and for `eve__tool`'s and `eve__skill`'s suggestions when a+ * name is unknown.+ * Query and text split into lowercase words (camelCase, acronyms, and+ * punctuation are boundaries), so `create issue`, `createIssue`, and+ * `create_issue` are the same words. Matches rank in tiers:+ *+ * 1. Exact: the query's words are the full name's words.+ * 2. Exact without namespace: they are the name after its last `__`, so+ * `create_issue` finds `linear__create_issue` and `sre__create_issue`.+ * 3. Connection: they are the connection's name, or its first whole words,+ * so `linear` lists the `linear` connection's tools, while `git` leaves+ * `github`'s tools to the name prefix tier.+ * 4. Name prefix: the full name's words, or those after its last `__`, start+ * with the query's words, the last of which may be partial, so+ * `create_iss` finds `create_issue` and `status page incidents` finds+ * `sre__status_page_incidents_list`.+ * 5. Keyword: any other match. A query word matches a word when either is a+ * prefix of the other, so `repo` finds `repositories` and `issues` finds+ * `issue`, and each query word adds the weight of every field it matches.+ * One-letter and filler words such as `a` and `the` don't score when the+ * query has other words, so `a` doesn't find `assign`.+ *+ * Within a tier, the keyword score orders matches, then connection, then name.+ */++import { connectionToolName } from "#connections/ownership.js";+import { isObject } from "#shared/guards.js";++/** What the ranker reads from one catalog entry. */+export interface RankCandidate {+ /** The entry's own name; a connection tool's name without its connection prefix. */+ readonly name: string;+ readonly description: string;+ readonly inputSchema?: Readonly<Record<string, unknown>>;+ readonly connection?: { readonly description: string; readonly name: string };+}++/** Candidates matching `query`, best first. */+export function rankCandidates<T extends RankCandidate>(+ query: string,+ candidates: readonly T[],+): T[] {+ const terms = tokenize(query);+ const scored = scoringTerms(terms);+ return candidates+ .map((candidate) => ({ candidate, ...rankCandidate(terms, scored, candidate) }))+ .filter((entry) => entry.score > 0)+ .sort(+ (a, b) =>+ a.tier - b.tier ||+ b.score - a.score ||+ compareCodeUnits(a.candidate.connection?.name ?? "", b.candidate.connection?.name ?? "") ||+ compareCodeUnits(a.candidate.name, b.candidate.name),+ )+ .map((entry) => entry.candidate);+}++/** Whether `query` has no searchable words, such as `"^"` or `" "`. */+export function isEmptyQuery(query: string): boolean {+ return tokenize(query).length === 0;+}++const MAX_SUGGESTIONS = 5;++/** Names of the candidates closest to an unknown `name`, best first. */+export function closestNames(name: string, candidates: readonly RankCandidate[]): string[] {+ if (isEmptyQuery(name)) return [];+ return rankCandidates(name, candidates)+ .slice(0, MAX_SUGGESTIONS)+ .map((candidate) => candidate.name);+}++const Tier = {+ exact: 0,+ exactWithoutNamespace: 1,+ connection: 2,+ namePrefix: 3,+ keyword: 4,+} as const;++function rankCandidate(+ terms: readonly string[],+ scored: readonly string[],+ candidate: RankCandidate,+): { readonly score: number; readonly tier: number } {+ const name =+ candidate.connection === undefined+ ? candidate.name+ : connectionToolName(candidate.connection.name, candidate.name);+ const fullName = tokenize(name);+ const at = name.lastIndexOf("__");+ const localName = tokenize(at < 0 ? name : name.slice(at + 2));+ const tier = sameWords(fullName, terms)+ ? Tier.exact+ : sameWords(localName, terms)+ ? Tier.exactWithoutNamespace+ : candidate.connection !== undefined &&+ startsWithWholeWords(tokenize(candidate.connection.name), terms)+ ? Tier.connection+ : startsWithWords(fullName, terms) || startsWithWords(localName, terms)+ ? Tier.namePrefix+ : Tier.keyword;+ return { score: scoreFields(scored, candidateFields(candidate)), tier };+}++/** Words that say nothing about which tool a query wants. */+const FILLER_WORDS = new Set([+ "a",+ "an",+ "and",+ "for",+ "in",+ "is",+ "me",+ "my",+ "of",+ "or",+ "please",+ "the",+ "to",+ "with",+]);++/**+ * The query words that score keyword matches: without one-letter and filler+ * words, which match nearly anything by prefix, unless they are all it has.+ */+function scoringTerms(terms: readonly string[]): readonly string[] {+ const meaningful = terms.filter((term) => term.length > 1 && !FILLER_WORDS.has(term));+ return meaningful.length > 0 ? meaningful : terms;+}++function sameWords(words: readonly string[], terms: readonly string[]): boolean {+ return words.length === terms.length && startsWithWholeWords(words, terms);+}++function startsWithWholeWords(words: readonly string[], terms: readonly string[]): boolean {+ return terms.length <= words.length && terms.every((term, index) => words[index] === term);+}++/** The last term may be a partial word. */+function startsWithWords(words: readonly string[], terms: readonly string[]): boolean {+ return (+ terms.length <= words.length &&+ terms.every((term, index) =>+ index === terms.length - 1 ? words[index]!.startsWith(term) : words[index] === term,+ )+ );+}++type Field = readonly [words: readonly string[], weight: number];++function candidateFields(candidate: RankCandidate): Field[] {+ const properties = isObject(candidate.inputSchema?.properties)+ ? Object.entries(candidate.inputSchema.properties)+ : [];+ const propertyDescriptions = properties.flatMap(([, schema]) =>+ isObject(schema) && typeof schema.description === "string" ? [schema.description] : [],+ );+ return [+ [tokenize(candidate.name), 6],+ [tokenize(candidate.connection?.name ?? ""), 4],+ [properties.flatMap(([key]) => tokenize(key)), 3],+ [proseWords(candidate.description), 2],+ [propertyDescriptions.flatMap(proseWords), 1],+ [proseWords(candidate.connection?.description ?? ""), 1],+ ];+}++function scoreFields(terms: readonly string[], fields: readonly Field[]): number {+ let score = 0;+ for (const term of terms) {+ for (const [words, weight] of fields) {+ if (words.some((word) => matchesWord(word, term))) score += weight;+ }+ }+ return score;+}++function matchesWord(word: string, term: string): boolean {+ return word.startsWith(term) || (word.length >= 3 && term.startsWith(word));+}++/** Every word of a name or query, so a one-letter name can still be searched by name. */+function tokenize(text: string): string[] {+ return text+ .replaceAll(/([A-Z]+)([A-Z][a-z])/gu, "$1 $2")+ .replaceAll(/([a-z0-9])([A-Z])/gu, "$1 $2")+ .toLowerCase()+ .split(/[^a-z0-9]+/u)+ .filter((word) => word !== "");+}++/** A description's words, without one-letter words such as `a`. */+function proseWords(text: string): string[] {+ return tokenize(text).filter((word) => word.length > 1);+}++/** Orders by UTF-16 code unit, the same in every locale. */+export function compareCodeUnits(a: string, b: string): number {+ return a < b ? -1 : a > b ? 1 : 0;+}diff --git a/packages/eve/src/execution/catalog/search.test.ts b/packages/eve/src/execution/catalog/search.test.tsnew file mode 100644index 000000000..01612f182--- /dev/null+++ b/packages/eve/src/execution/catalog/search.test.ts@@ -0,0 +1,562 @@+import { describe, expect, it, vi } from "vitest";+import { SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";++import type { StandardSchemaV1 } from "#compiled/@standard-schema/spec/index.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import {+ catalogContext,+ connectionTool,+ fakeConnection,+ inlineTool,+ subagentTool,+ type CatalogSkillSource,+ type FakeConnection,+} from "#internal/testing/catalog-fixtures.js";+import { captureLogRecords } from "#internal/testing/log-records.js";++interface SearchOutput {+ readonly results: readonly Record<string, unknown>[];+ readonly unavailable?: readonly Record<string, unknown>[];+}++/** The step's `search` tool, run inside its session. */+function searchFor(+ tools: readonly HarnessToolDefinition[],+ options: {+ readonly connections?: readonly FakeConnection[];+ readonly skills?: readonly CatalogSkillSource[];+ } = {},+) {+ const { catalog, run } = catalogContext({ ...options, tools });+ const search = catalog.advertised.get(SEARCH_TOOL_NAME)!;+ return (input: Record<string, unknown>) =>+ run(async () => (await search.execute!(input, OPTIONS)) as SearchOutput);+}++const OPTIONS = { messages: [], toolCallId: "call-1" };++const names = (output: SearchOutput) => output.results.map((result) => result.tool ?? result.skill);++describe("search", () => {+ const tools = [+ inlineTool("add"),+ inlineTool("refund_invoice", {+ deferred: true,+ description: "Refund a paid invoice.",+ schema: {+ type: "object",+ properties: { invoiceId: { type: "string" } },+ required: ["invoiceId"],+ },+ }),+ inlineTool("list_invoices", { deferred: true, description: "List invoices to refund later." }),+ inlineTool("export_ledger", { deferred: true, description: "Export the ledger." }),+ subagentTool("billing_specialist", {+ deferred: true,+ description: "Resolve billing disputes and refunds.",+ }),+ ];+ const skills = [+ { deferred: true, description: "How to refund a disputed charge.", name: "refund-policy" },+ { description: "Refund wording for replies.", name: "refund-voice" },+ ];++ it("ranks tools and skills together by name before description", async () => {+ const output = await searchFor(tools, { skills })({ query: "refund" });++ // Equal matches sort by name, compared by code unit: `-` before `_`.+ expect(names(output)).toEqual([+ "refund-policy",+ "refund_invoice",+ "billing_specialist",+ "list_invoices",+ ]);+ expect(output.results[0]).toEqual({+ description: "How to refund a disputed charge.",+ path: "$HOME/.agents/skills/refund-policy/SKILL.md",+ skill: "refund-policy",+ });+ expect(output.results[1]).toEqual({+ description: "Refund a paid invoice.",+ signature: "refund_invoice(input: { invoiceId: string }): Promise<unknown>",+ tool: "refund_invoice",+ });+ });++ it("labels a search with its query, then with how much it found", async () => {+ const { catalog, run } = catalogContext({ skills, tools });+ const search = catalog.advertised.get(SEARCH_TOOL_NAME)!;+ const label = search.label!;+ const found = await run(() => search.execute!({ query: "refund" }, OPTIONS));++ expect(label.start!({ query: "refund" })).toBe("Search tools for “refund”");+ expect(label.complete!({ query: "refund" }, found)).toBe(+ "Searched tools for “refund”, found 4",+ );+ expect(label.complete!({ query: "zebra" }, { results: [] })).toBe(+ "Searched tools for “zebra”, found nothing",+ );+ const unavailable = (count: number) => ({+ results: [],+ unavailable: Array.from({ length: count }, (_, i) => ({+ connection: `c${i}`,+ error: "down",+ })),+ });+ expect(label.complete!({ query: "zebra" }, unavailable(1))).toBe(+ "Searched tools for “zebra”, found nothing; 1 connection unavailable",+ );+ expect(label.complete!({ query: "zebra" }, unavailable(2))).toBe(+ "Searched tools for “zebra”, found nothing; 2 connections unavailable",+ );+ });++ it("finds only deferred entries; direct tools and listed skills are already in context", async () => {+ const search = searchFor(tools, { skills });++ expect(await search({ query: "add" })).toEqual({ results: [] });+ expect(names(await search({ query: "refund voice" }))).not.toContain("refund-voice");+ });++ it("requires a query with at least one word", async () => {+ const search = searchFor(tools, { skills });+ const schema = catalogContext({ tools }).catalog.advertised.get(SEARCH_TOOL_NAME)!+ .inputSchema as StandardSchemaV1;+ const needsWords =+ 'search needs at least one word in query, such as a capability ("list open issues") or a name or connection prefix ("linear__").';++ expect(await schema["~standard"].validate({ limit: 5 })).toEqual({+ issues: [+ expect.objectContaining({ message: 'Instance does not have required property "query".' }),+ ],+ });+ await expect(search({ query: " ^ " })).rejects.toThrow(needsWords);+ await expect(search({})).rejects.toThrow(needsWords);+ });++ it("returns the best matches up to limit, 10 by default, and takes no paging", async () => {+ const many = Array.from({ length: 60 }, (_, index) =>+ inlineTool(`report_${String(index).padStart(2, "0")}`, { deferred: true }),+ );+ const search = searchFor(many);++ const best = names(await search({ limit: 50, query: "report" }));+ expect(best).toHaveLength(50);+ // A smaller limit keeps the best matches, in rank order.+ expect(names(await search({ limit: 3, query: "report" }))).toEqual(best.slice(0, 3));+ expect(names(await search({ query: "report" }))).toEqual(best.slice(0, 10));++ // The model's calls are checked against the schema, which caps limit at 50 and has no offset.+ const schema = catalogContext({ tools: many }).catalog.advertised.get(SEARCH_TOOL_NAME)!+ .inputSchema as StandardSchemaV1;+ expect(await schema["~standard"].validate({ limit: 51, query: "report" })).toEqual({+ issues: [expect.objectContaining({ path: ["limit"] })],+ });+ expect(await schema["~standard"].validate({ offset: 10, query: "report" })).toEqual({+ issues: [expect.objectContaining({ message: 'Unrecognized key: "offset"' })],+ });+ });++ describe("connections", () => {+ const linear = () =>+ fakeConnection({+ description: "Linear issues and projects",+ name: "linear",+ tools: [+ connectionTool("list_issues", {+ type: "object",+ properties: { team: { type: "string" } },+ required: ["team"],+ }),+ connectionTool("create_project"),+ ],+ });++ it("ranks connection tools with local entries under their full names", async () => {+ const output = await searchFor(tools, { connections: [linear()] })({ query: "issues" });++ // The connection's description matches too, but weighs less than a tool's own name.+ expect(output.results).toEqual([+ {+ description: "list_issues description",+ signature: "linear__list_issues(input: { team: string }): Promise<unknown>",+ tool: "linear__list_issues",+ },+ {+ description: "create_project description",+ signature: "linear__create_project(input: Record<string, unknown>): Promise<unknown>",+ tool: "linear__create_project",+ },+ ]);+ });++ it("lists a connection whose tools need sign-in as one result named after it, and never prompts", async () => {+ const notion = fakeConnection({+ description: "Notion pages and databases",+ listing: "sign-in",+ name: "notion",+ signIn: true,+ tools: [connectionTool("search_pages")],+ });+ const search = searchFor(tools, { connections: [linear(), notion] });++ const signIn = {+ description: "Sign in to use the Notion tools: Notion pages and databases",+ signature: "notion(input: {}): Promise<unknown>",+ tool: "notion",+ };+ expect(await search({ query: "notion" })).toEqual({ results: [signIn] });+ // Found by the connection's description too, while its tools can't be listed.+ expect((await search({ query: "pages" })).results).toEqual([signIn]);+ expect(names(await search({ query: "notion__" }))).toEqual(["notion"]);+ expect(notion.signIns).toEqual([]);+ });++ it("reports a connection that cannot start sign-in, and one whose tools fail to load, as unavailable", async () => {+ const logs = captureLogRecords();+ const vault = fakeConnection({ listing: "sign-in", name: "vault", tools: [] });+ const crm = fakeConnection({+ listing: new Error("upstream returned 502"),+ name: "crm",+ tools: [],+ });++ const output = await searchFor(tools, { connections: [linear(), vault, crm] })({+ query: "issues",+ });++ expect(names(output)).toContain("linear__list_issues");+ expect(names(output)).not.toContain("vault");+ expect(output.unavailable).toEqual([+ {+ connection: "vault",+ error: '"vault" requires authorization and cannot start interactive sign-in.',+ },+ { connection: "crm", error: 'Failed to load tools for "crm": upstream returned 502' },+ ]);+ expect(logs.records).toContainEqual(+ expect.objectContaining({ level: "warn", message: "failed to load connection tools" }),+ );+ });++ it("reports a connection that doesn't list its tools within 10 seconds as unavailable, while other results return, and settles late without an unhandled rejection", async () => {+ const unhandled = vi.fn();+ process.on("unhandledRejection", unhandled);+ vi.useFakeTimers();+ try {+ const crm = fakeConnection({ name: "crm", tools: [] });+ let settleLate!: (error: Error) => void;+ vi.spyOn(crm.client, "getToolMetadata").mockReturnValue(+ new Promise((_, reject) => {+ settleLate = reject;+ }),+ );+ let settled = false;+ const pending = searchFor(tools, { connections: [linear(), crm] })({ query: "issues" });+ void pending.then(() => (settled = true));++ await vi.advanceTimersByTimeAsync(9_999);+ expect(settled).toBe(false);+ await vi.advanceTimersByTimeAsync(1);+ const output = await pending;++ expect(names(output)).toContain("linear__list_issues");+ expect(output.unavailable).toEqual([+ { connection: "crm", error: '"crm" did not list its tools within 10s. Try again later.' },+ ]);+ vi.useRealTimers();+ settleLate(new Error("the server answered after the timeout"));+ await new Promise((resolve) => setTimeout(resolve, 0));+ expect(unhandled).not.toHaveBeenCalled();+ } finally {+ vi.useRealTimers();+ process.off("unhandledRejection", unhandled);+ }+ });+ });++ describe("ranking", () => {+ const search = searchFor(+ [+ inlineTool("pdf", { deferred: true, description: "Open a document." }),+ inlineTool("pdf_fill", {+ deferred: true,+ description: "Fill a PDF form from pdf fields.",+ schema: {+ type: "object",+ properties: { pdf: { type: "string", description: "The pdf to fill." } },+ },+ }),+ inlineTool("create_issue", { deferred: true, description: "File a local ticket." }),+ inlineTool("linear_report", { deferred: true, description: "Summarize Linear usage." }),+ ],+ {+ connections: [+ fakeConnection({+ description: "Linear issues and projects",+ name: "linear",+ tools: [+ connectionTool("create_issue"),+ connectionTool("create_project"),+ connectionTool("list_issues"),+ ],+ }),+ fakeConnection({+ description: "Jira issues",+ name: "jira",+ tools: [connectionTool("create_issue")],+ }),+ fakeConnection({+ description: "Service health",+ name: "ops",+ tools: [connectionTool("getHTTPServerStatus")],+ }),+ fakeConnection({ description: "Posts", name: "x", tools: [connectionTool("post")] }),+ ],+ skills: [{ deferred: true, description: "Fill PDF forms.", name: "pdf-forms" }],+ },+ );+ const ranked = async (query: string) => names(await search({ limit: 50, query }));++ it("ranks an exact name first, above an entry that mentions it in more places", async () => {+ expect((await ranked("pdf")).slice(0, 2)).toEqual(["pdf", "pdf_fill"]);+ });++ it("ranks an exact full name above the same name under a connection's prefix", async () => {+ const results = await ranked("create_issue");++ expect(results[0]).toBe("create_issue");+ expect(results.slice(1, 3).sort()).toEqual(["jira__create_issue", "linear__create_issue"]);+ expect((await ranked("linear__create_issue"))[0]).toBe("linear__create_issue");+ });++ it("lists a connection's tools for its name, before an entry whose name starts with it", async () => {+ const results = await ranked("linear");++ expect(results.slice(0, 3).sort()).toEqual([+ "linear__create_issue",+ "linear__create_project",+ "linear__list_issues",+ ]);+ expect(results[3]).toBe("linear_report");+ });++ it("lists a connection's tools first only for its whole name, not a partial one", async () => {+ const search = searchFor([inlineTool("git_status", { deferred: true })], {+ connections: [+ fakeConnection({+ name: "github",+ tools: [connectionTool("create_issue"), connectionTool("list_pulls")],+ }),+ ],+ });+ const ranked = async (query: string) => names(await search({ query }));++ expect(await ranked("git")).toEqual([+ "git_status",+ "github__create_issue",+ "github__list_pulls",+ ]);+ expect((await ranked("github")).slice(0, 2)).toEqual([+ "github__create_issue",+ "github__list_pulls",+ ]);+ });++ it("matches a name prefix whose last word is partial", async () => {+ expect((await ranked("create_iss")).slice(0, 3).sort()).toEqual([+ "create_issue",+ "jira__create_issue",+ "linear__create_issue",+ ]);+ expect((await ranked("linear__cre")).slice(0, 2).sort()).toEqual([+ "linear__create_issue",+ "linear__create_project",+ ]);+ });++ it("matches the name after any namespace, not only a connection's, as exact or prefix", async () => {+ const search = searchFor([+ inlineTool("sre__status_page_incidents_list", {+ deferred: true,+ description: "List open incidents.",+ }),+ inlineTool("sre__status_page_incident_get", {+ deferred: true,+ description: "Get one status page incident, with its status, page, and incidents linked.",+ }),+ inlineTool("support__refund", { deferred: true, description: "Refund an order." }),+ inlineTool("refund_policy", { deferred: true, description: "Read the refund policy." }),+ ]);+ const ranked = async (query: string) => names(await search({ query }));++ expect((await ranked("status page incidents"))[0]).toBe("sre__status_page_incidents_list");+ expect((await ranked("refund"))[0]).toBe("support__refund");+ });++ it("keeps a name without a namespace whole for exact and prefix matches", async () => {+ const search = searchFor([+ inlineTool("xrefund", { deferred: true, description: "Refund a charge." }),+ inlineTool("refund_policy", { deferred: true, description: "Read the policy." }),+ ]);+ const ranked = async (query: string) => names(await search({ query }));++ expect(await ranked("refund")).toEqual(["refund_policy", "xrefund"]);+ expect((await ranked("xrefund"))[0]).toBe("xrefund");+ expect((await ranked("refund_pol"))[0]).toBe("refund_policy");+ });++ it("leaves one-letter and filler words out of scoring unless the query has nothing else", async () => {+ const search = searchFor([+ inlineTool("support__case_assign", {+ deferred: true,+ description: "Assign a support case to a teammate.",+ }),+ inlineTool("support__case_reply_draft", {+ deferred: true,+ description: "Write a response.",+ }),+ ]);+ const ranked = async (query: string) => names(await search({ query }));++ expect((await ranked("draft a reply to a support ticket"))[0]).toBe(+ "support__case_reply_draft",+ );+ expect(await ranked("a")).toContain("support__case_assign");+ });++ it("finds a one-letter connection by its name", async () => {+ expect((await ranked("x"))[0]).toBe("x__post");+ });++ it.each(["http server", "getHttpServer", "get_http_server_status"])(+ "splits camelCase and acronyms into words, so %s finds getHTTPServerStatus",+ async (query) => {+ expect((await ranked(query))[0]).toBe("ops__getHTTPServerStatus");+ },+ );++ it("lists a bare namespace by name, in a fixed order", async () => {+ expect(await ranked("linear__")).toEqual([+ "linear__create_issue",+ "linear__create_project",+ "linear__list_issues",+ ]);+ });+ });++ describe("namespace queries", () => {+ function namespaced() {+ const crmApi = fakeConnection({+ description: "CRM customers and deals",+ name: "crm__api",+ tools: [+ connectionTool("get_customer"),+ connectionTool("list_customers"),+ connectionTool("list_deals"),+ ],+ });+ const notion = fakeConnection({+ listing: "sign-in",+ name: "notion",+ signIn: true,+ tools: [connectionTool("search_pages")],+ });+ const linear = fakeConnection({+ name: "linear",+ tools: [+ connectionTool("list_issues"),+ connectionTool("issue__create"),+ connectionTool("issue__create_comment"),+ ],+ });+ const broken = fakeConnection({+ listing: new Error("upstream returned 502"),+ name: "jira",+ tools: [],+ });+ const empty = fakeConnection({ name: "archive", tools: [] });+ const outside = [linear, broken].map((connection) =>+ vi.spyOn(connection.client, "getToolMetadata"),+ );+ const search = searchFor(+ [+ // An extension mounted as `crm` adds a tool and a connection under its name.+ inlineTool("crm__export", { deferred: true, description: "Export CRM accounts." }),+ inlineTool("export_ledger", {+ deferred: true,+ description: "Export the ledger to the crm.",+ }),+ ],+ { connections: [crmApi, notion, linear, broken, empty] },+ );+ return { notion, outside, search };+ }++ it("keeps only the namespace, including a connection mounted under it, and never lists other connections", async () => {+ const { outside, search } = namespaced();++ const output = await search({ query: "crm__" });++ expect(names(output).sort()).toEqual([+ "crm__api__get_customer",+ "crm__api__list_customers",+ "crm__api__list_deals",+ "crm__export",+ ]);+ // Connections that can't own the namespace aren't asked for their tools, so the+ // failing one isn't reported either.+ expect(output.unavailable).toBeUndefined();+ for (const getToolMetadata of outside) expect(getToolMetadata).not.toHaveBeenCalled();+ });++ it("ranks the namespace by the whole query, best match first", async () => {+ const { search } = namespaced();++ const found = names(await search({ query: "crm__api__list deals" }));++ expect(found[0]).toBe("crm__api__list_deals");+ expect(found.every((name) => String(name).startsWith("crm__api__"))).toBe(true);+ });++ it("ranks a tool whose own name contains __ first for its exact full name", async () => {+ const { search } = namespaced();++ expect(names(await search({ query: "linear__issue__create" }))[0]).toBe(+ "linear__issue__create",+ );+ });++ it("keeps a connection's sign-in entry, alone or with words after the namespace, without prompting", async () => {+ const { notion, search } = namespaced();++ expect(names(await search({ query: "notion__" }))).toEqual(["notion"]);+ expect(names(await search({ query: "notion__search pages" }))).toEqual(["notion"]);+ expect(notion.signIns).toEqual([]);+ });++ it("ignores a leading ^ and trailing underscores", async () => {+ const { search } = namespaced();+ const linear = await search({ query: "linear__" });++ expect(await search({ query: "^linear__" })).toEqual(linear);+ expect(await search({ query: "linear____" })).toEqual(linear);+ });++ it.each([+ ["Linear__", 'No entries are named "Linear__…". Closest connections: linear.'],+ [+ "nope__x",+ 'No entries are named "nope__…". Find names in the catalog listing or your tool list, or call eve__search with plain words.',+ ],+ ["archive__", 'Connection "archive" has no tools.'],+ ])("fails for %s, a namespace with no entries", async (query, message) => {+ const { outside, search } = namespaced();++ await expect(search({ query })).rejects.toThrow(message);+ if (query === "nope__x") {+ for (const getToolMetadata of outside) expect(getToolMetadata).not.toHaveBeenCalled();+ }+ });+ });+});diff --git a/packages/eve/src/execution/catalog/search.ts b/packages/eve/src/execution/catalog/search.tsnew file mode 100644index 000000000..5cf875545--- /dev/null+++ b/packages/eve/src/execution/catalog/search.ts@@ -0,0 +1,347 @@+/**+ * `eve__search`: finds catalog entries, the agent's deferred tools and skills+ * and every connection tool, by keyword. It never prompts: a connection whose+ * tools need sign-in is found as its sign-in entry instead. Its definition is+ * fixed for each eve version, so the catalog can change without changing the model's+ * tool list.+ */++import { connectionToolName } from "#connections/ownership.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import { CALL_TOOL_NAME, SEARCH_TOOL_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";+import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";+import type { ResolvedConnectionDefinition } from "#runtime/types.js";+import type { ConnectionToolMetadata } from "#shared/connection-types.js";+import { serializeInputSchema, toInputSchema, type ToolSchemaSource } from "#tools/schema.js";++import {+ completeConnectionSignIn,+ listConnectionTools,+ listingFailureMessage,+ type ConnectionListing,+} from "./connection-auth.js";+import { connectionSignInEntry } from "./connection-entry.js";+import { closestNames, isEmptyQuery, rankCandidates, type RankCandidate } from "./rank.js";+import { connectionToolSignature, entrySignature } from "./signatures.js";+import type { CatalogSkill } from "./skills.js";++const DEFAULT_LIMIT = 10;+const MAX_LIMIT = 50;++/** What an agent's search can return, from what it declares. */+export interface SearchReach {+ readonly connections: boolean;+ readonly skills: boolean;+ readonly tools: boolean;+}++/**+ * `eve__search`'s description, saying which tool takes each kind of result.+ * It mentions only the catalog tools the agent has, so it is fixed for a+ * deployment.+ */+function searchDescription({ connections, skills, tools }: SearchReach): string {+ return [+ "Find your own tools, agents, skills, and connected services by keyword; this searches what you can do, not the web.",+ tools+ ? `A result with \`tool\` has the exact name, description, and TypeScript signature: call it with ${CALL_TOOL_NAME}({ name, input }).`+ : undefined,+ skills+ ? `A result with \`skill\` has the name and description: load it with ${SKILL_TOOL_NAME}({ name }).`+ : undefined,+ 'When you already know a name or connection, from the catalog listing, an earlier result, or an error, search it directly: the exact name, or "<connection>__", which is faster and returns only that connection\'s tools.',+ connections+ ? `A connection that needs sign-in appears as a tool named after the connection: calling it with ${CALL_TOOL_NAME}({ name }) asks the user to sign in.`+ : undefined,+ ]+ .filter((sentence) => sentence !== undefined)+ .join(" ");+}++const SEARCH_INPUT_SCHEMA = toInputSchema({+ type: "object",+ properties: {+ query: {+ type: "string",+ minLength: 1,+ description:+ "Words describing the capability, such as 'list open issues'; an exact name; or a namespace prefix such as 'linear__', which searches only that connection's tools.",+ },+ limit: {+ type: "integer",+ minimum: 1,+ maximum: MAX_LIMIT,+ description: `Maximum results, best matches first. Defaults to ${DEFAULT_LIMIT}.`,+ },+ },+ required: ["query"],+ additionalProperties: false,+});++interface SearchInput {+ readonly limit?: number;+ /** The schema requires it; a call that skips validation and omits it fails like a query with no words. */+ readonly query?: string;+}++type SearchResult =+ | { readonly description: string; readonly signature: string; readonly tool: string }+ | { readonly description: string; readonly path?: string; readonly skill: string };++interface UnavailableConnection {+ readonly connection: string;+ readonly error: string;+}++interface SearchOutput {+ readonly results: readonly SearchResult[];+ readonly unavailable?: readonly UnavailableConnection[];+}++/**+ * `fullName` is the name `eve__tool` takes and a namespace query scopes. `result`+ * renders only for the results returned, since signatures cost a render.+ */+type SearchCandidate = RankCandidate & {+ readonly fullName: string;+ readonly result: () => SearchResult;+};++/** People see what the model searched for and, once it returns, how much it found. */+const SEARCH_LABEL = {+ start: (input: unknown) => `Search tools${labelQuery(input)}`,+ complete: (input: unknown, output: unknown) => {+ const { results, unavailable = [] } = output as SearchOutput;+ const searched = `Searched tools${labelQuery(input)}, found ${results.length === 0 ? "nothing" : String(results.length)}`;+ if (results.length > 0 || unavailable.length === 0) return searched;+ const connections = unavailable.length === 1 ? "connection" : "connections";+ return `${searched}; ${String(unavailable.length)} ${connections} unavailable`;+ },+};++function labelQuery(input: unknown): string {+ const query = (input as SearchInput).query?.trim();+ return query ? ` for “${query}”` : "";+}++/**+ * Builds `eve__search` over one step's deferred entries and skills and the+ * connections in `registry`. `describe` returns an entry's description as the+ * model would read it in its tool list.+ */+export function createSearchTool(input: {+ readonly deferred: readonly HarnessToolDefinition[];+ readonly describe: (definition: HarnessToolDefinition) => string;+ readonly reach: SearchReach;+ readonly registry: ConnectionRegistry | undefined;+ readonly skills: readonly CatalogSkill[];+}): HarnessToolDefinition {+ return {+ description: searchDescription(input.reach),+ execute: (rawInput: unknown) => search(input, rawInput as SearchInput),+ frameworkTool: true,+ inputSchema: SEARCH_INPUT_SCHEMA,+ label: SEARCH_LABEL,+ name: SEARCH_TOOL_NAME,+ };+}++async function search(+ catalog: Parameters<typeof createSearchTool>[0],+ input: SearchInput,+): Promise<SearchOutput> {+ const query = input.query ?? "";+ if (isEmptyQuery(query)) {+ throw new Error(+ `${SEARCH_TOOL_NAME} needs at least one word in query, such as a capability ("list open issues") or a name or connection prefix ("linear__").`,+ );+ }+ const namespace = parseNamespace(query);+ const inScope = (name: string) => namespace === undefined || inNamespace(name, namespace);++ const candidates: SearchCandidate[] = [+ ...catalog.deferred.map((definition) => entryCandidate(definition, catalog.describe)),+ ...catalog.skills.map(skillCandidate),+ ].filter((candidate) => inScope(candidate.fullName));+ const unavailable: UnavailableConnection[] = [];+ const { registry } = catalog;+ if (registry !== undefined) {+ const connections = registry+ .getConnections()+ .filter(+ ({ connectionName }) =>+ namespace === undefined || mayOwnNamespace(connectionName, namespace),+ );+ const found = await Promise.all(+ connections.map((connection) => searchConnection(registry, connection)),+ );+ for (const connection of found) {+ if ("unavailable" in connection) unavailable.push(connection.unavailable);+ else+ candidates.push(+ ...connection.candidates.filter((candidate) => inScope(candidate.fullName)),+ );+ }+ }++ if (namespace !== undefined && candidates.length === 0 && unavailable.length === 0) {+ throw new Error(unknownNamespaceMessage(namespace, registry?.getConnections() ?? []));+ }++ const limit = clampInteger(input.limit, 1, MAX_LIMIT, DEFAULT_LIMIT);+ const results = rankCandidates(query, candidates)+ .slice(0, limit)+ .map((candidate) => candidate.result());+ return unavailable.length > 0 ? { results, unavailable } : { results };+}++/**+ * The namespace a query scopes to: when its first word contains `__`,+ * everything before its last `__`, without trailing underscores. It only+ * filters; the whole query still ranks. Characters that can't appear in a+ * name, such as a leading `^`, are ignored; regex isn't supported.+ */+function parseNamespace(query: string): string | undefined {+ const word = (query.trim().split(/\s+/u)[0] ?? "").replace(/^[^A-Za-z0-9_-]+/u, "");+ const namespace = word.slice(0, Math.max(0, word.lastIndexOf("__"))).replace(/_+$/u, "");+ return namespace === "" ? undefined : namespace;+}++/**+ * Names under `<namespace>__`, plus anything named exactly `namespace`, such+ * as a connection's sign-in entry.+ */+function inNamespace(name: string, namespace: string): boolean {+ return name === namespace || name.startsWith(`${namespace}__`);+}++/**+ * Whether a connection can own names in `namespace`. Its names are its own+ * and those under its prefix, so only a connection that is the namespace,+ * contains it, or sits under it (an extension's connection, such as+ * `crm__api` under `crm`) can; every other connection is skipped without+ * listing its tools.+ */+function mayOwnNamespace(connectionName: string, namespace: string): boolean {+ return inNamespace(connectionName, namespace) || namespace.startsWith(`${connectionName}__`);+}++/**+ * Why a namespace query found nothing: the connection it names has no tools,+ * or the closest connection names, or how to find names when none is close.+ * Names are case-sensitive, so `Linear__` misses `linear`.+ */+function unknownNamespaceMessage(+ namespace: string,+ connections: readonly ResolvedConnectionDefinition[],+): string {+ // A connection that lists no tools, such as an empty server or one its tool filter empties.+ if (connections.some(({ connectionName }) => connectionName === namespace)) {+ return `Connection "${namespace}" has no tools.`;+ }+ const closest = closestNames(+ namespace,+ connections.map(({ connectionName, description }) => ({+ description,+ name: connectionName,+ })),+ );+ const hint =+ closest.length > 0+ ? `Closest connections: ${closest.join(", ")}.`+ : `Find names in the catalog listing or your tool list, or call ${SEARCH_TOOL_NAME} with plain words.`;+ return `No entries are named "${namespace}__…". ${hint}`;+}++/** A connection's tools, or its sign-in entry while listing them needs sign-in. */+async function searchConnection(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+): Promise<+ | { readonly candidates: readonly SearchCandidate[] }+ | { readonly unavailable: UnavailableConnection }+> {+ const name = connection.connectionName;+ let listing: ConnectionListing;+ try {+ listing = await listConnectionTools(+ connection,+ await completeConnectionSignIn(registry, connection),+ );+ } catch (error) {+ return { unavailable: { connection: name, error: listingFailureMessage(name, error) } };+ }+ if ("failure" in listing) return { unavailable: { connection: name, error: listing.failure } };+ if ("authorization" in listing) return { candidates: [signInCandidate(registry, connection)] };+ return { candidates: listing.tools.map((tool) => toolCandidate(connection, tool)) };+}++function entryCandidate(+ definition: HarnessToolDefinition,+ describe: (definition: HarnessToolDefinition) => string,+): SearchCandidate {+ const inputSchema = serializeInputSchema(definition.inputSchema as ToolSchemaSource);+ const description = describe(definition);+ return {+ description,+ inputSchema,+ name: definition.name,+ result: () => ({+ description,+ signature: entrySignature(definition, inputSchema),+ tool: definition.name,+ }),+ fullName: definition.name,+ };+}++function skillCandidate({ description, name, path }: CatalogSkill): SearchCandidate {+ return {+ description,+ fullName: name,+ name,+ result: () => ({ description, path, skill: name }),+ };+}++function toolCandidate(+ connection: ResolvedConnectionDefinition,+ tool: ConnectionToolMetadata,+): SearchCandidate {+ const name = connectionToolName(connection.connectionName, tool.name);+ return {+ connection: { description: connection.description, name: connection.connectionName },+ description: tool.description,+ inputSchema: tool.inputSchema,+ name: tool.name,+ result: () => ({+ description: tool.description,+ signature: connectionToolSignature(connection, tool),+ tool: name,+ }),+ fullName: name,+ };+}++/** Found by the connection's own name and description, as its sign-in entry. */+function signInCandidate(+ registry: ConnectionRegistry,+ connection: ResolvedConnectionDefinition,+): SearchCandidate {+ const entry = connectionSignInEntry(registry, connection);+ return {+ description: connection.description,+ name: connection.connectionName,+ result: () => ({+ description: entry.description,+ signature: entrySignature(entry),+ tool: entry.name,+ }),+ fullName: entry.name,+ };+}++function clampInteger(value: unknown, min: number, max: number, fallback: number): number {+ if (typeof value !== "number" || !Number.isFinite(value)) return fallback;+ return Math.min(max, Math.max(min, Math.trunc(value)));+}diff --git a/packages/eve/src/execution/catalog/signatures.ts b/packages/eve/src/execution/catalog/signatures.tsnew file mode 100644index 000000000..7d0a57ef5--- /dev/null+++ b/packages/eve/src/execution/catalog/signatures.ts@@ -0,0 +1,39 @@+/** TypeScript signatures for catalog entries, rendered from the schemas each call sees. */++import { connectionToolName } from "#connections/ownership.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import type { ResolvedConnectionDefinition } from "#runtime/types.js";+import type { ConnectionToolMetadata } from "#shared/connection-types.js";+import {+ serializeInputSchema,+ serializeOutputSchema,+ type ToolSchemaSource,+} from "#tools/schema.js";+import { renderToolSignature } from "#tools/signature.js";++/**+ * An agent entry's signature, from the schemas the model would see for it.+ * A caller that already converted the input schema passes it in.+ */+export function entrySignature(+ definition: HarnessToolDefinition,+ inputSchema = serializeInputSchema(definition.inputSchema as ToolSchemaSource),+): string {+ return renderToolSignature({+ inputSchema,+ name: definition.name,+ outputSchema: serializeOutputSchema(definition.outputSchema as ToolSchemaSource | undefined),+ });+}++/** A connection tool's signature under its full name. */+export function connectionToolSignature(+ connection: ResolvedConnectionDefinition,+ tool: ConnectionToolMetadata,+): string {+ return renderToolSignature({+ inputSchema: tool.inputSchema,+ name: connectionToolName(connection.connectionName, tool.name),+ outputSchema: tool.outputSchema,+ });+}diff --git a/packages/eve/src/execution/catalog/skills.ts b/packages/eve/src/execution/catalog/skills.tsnew file mode 100644index 000000000..095918c35--- /dev/null+++ b/packages/eve/src/execution/catalog/skills.ts@@ -0,0 +1,113 @@+/**+ * Skills in the catalog. Every skill the session can see loads through+ * `eve__skill({ name })`, deferred or not. The call resolves to one loader entry+ * whose name sits outside the tool namespace, because skills and tools have+ * separate names.+ */++import type { ContextReader } from "#context/key.js";+import { DynamicSkillManifestKey } from "#context/keys.js";+import { skillFilePath } from "#execution/skills/instructions.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import { SEARCH_TOOL_NAME, SKILL_ENTRY_NAME } from "#protocol/catalog-tools.js";+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import { skillTarget } from "#shared/action-request-name.js";+import { stripSkillFrontmatter } from "#shared/skill-package.js";+import { toInputSchema } from "#tools/schema.js";++import { closestNames } from "./rank.js";++/** A skill the session can load. */+export interface CatalogSkill {+ readonly deferred: boolean;+ readonly description: string;+ /** The instructions loading returns. */+ readonly markdown: string;+ readonly name: string;+ /** The skill's `SKILL.md` under the skills root, when it has package files. */+ readonly path?: string;+}++/** The session's skills by name; a dynamic skill overrides a same-named authored one. */+export function sessionSkills(ctx: ContextReader | undefined): ReadonlyMap<string, CatalogSkill> {+ const skills = new Map<string, CatalogSkill>();+ for (const skill of ctx?.get(BundleKey)?.resolvedAgent.skills ?? []) {+ skills.set(skill.name, {+ deferred: skill.deferred === true,+ description: skill.description,+ markdown: skill.markdown,+ name: skill.name,+ path: skillFilePath(skill.name),+ });+ }+ for (const skill of Object.values(ctx?.get(DynamicSkillManifestKey) ?? {}).flat()) {+ skills.set(skill.name, {+ deferred: skill.deferred === true,+ description: skill.description,+ markdown: stripSkillFrontmatter(skill.markdown),+ name: skill.name,+ path: skill.revision === undefined ? undefined : skillFilePath(skill.name),+ });+ }+ return skills;+}++const SKILL_LOAD_LABEL = {+ start: (input: unknown) => `Load skill${skillName(input)}`,+ complete: (input: unknown) => `Loaded skill${skillName(input)}`,+};++function skillName(input: unknown): string {+ const skill = skillTarget(input);+ return skill === undefined ? "" : ` ${skill}`;+}++/** The entry every `eve__skill` call resolves to, with its skill as `{ skill }`. */+export function createSkillLoader(+ skills: ReadonlyMap<string, CatalogSkill>,+): HarnessToolDefinition {+ return {+ description: "Loads a skill's instructions.",+ execute: (input: { readonly skill: string }) => {+ const skill = skills.get(input.skill);+ // `eve__skill` validation resolves every skill before the call runs.+ if (skill === undefined) throw new Error(`No skill named "${input.skill}".`);+ return skill.markdown;+ },+ frameworkTool: true,+ inputSchema: SKILL_INPUT_SCHEMA,+ label: SKILL_LOAD_LABEL,+ name: SKILL_ENTRY_NAME,+ };+}++/** Why `eve__skill` can't load `name`: the closest skills, or the connection it names. */+export function unknownSkillMessage(+ name: string,+ skills: ReadonlyMap<string, CatalogSkill>,+ connections: readonly string[],+ searchable: boolean,+): string {+ const suggestions = closestNames(name, [...skills.values()]);+ const hint =+ suggestions.length > 0+ ? ` Closest skills: ${suggestions.join(", ")}.`+ : searchable+ ? ` Find skills with ${SEARCH_TOOL_NAME}.`+ : "";+ const connection = connections.find(+ (connectionName) => connectionName.toLowerCase() === name.toLowerCase(),+ );+ const connectionHint =+ connection === undefined+ ? ""+ : ` "${connection}" is a connection, not a skill. Find its tools with ${SEARCH_TOOL_NAME}({ query: "${connection}__" }).`;+ return `No skill named "${name}".${hint}${connectionHint}`;+}++const SKILL_INPUT_SCHEMA = toInputSchema({+ type: "object",+ properties: { skill: { type: "string" } },+ required: ["skill"],+ additionalProperties: false,+});diff --git a/packages/eve/src/execution/catalog/step-catalog.integration.test.ts b/packages/eve/src/execution/catalog/step-catalog.integration.test.tsnew file mode 100644index 000000000..afbf48537--- /dev/null+++ b/packages/eve/src/execution/catalog/step-catalog.integration.test.ts@@ -0,0 +1,725 @@+import type {+ LanguageModelV4CallOptions,+ LanguageModelV4Prompt,+ LanguageModelV4ToolResultPart,+} from "@ai-sdk/provider";+import { MockLanguageModelV4 } from "ai/test";+import { describe, expect, it, vi } from "vitest";++import { CALL_TOOL_NAME, SEARCH_TOOL_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";+import { TASK_CANCEL_TOOL_NAME, TASK_WAIT_TOOL_NAME } from "#protocol/task-tools.js";++import { ContextContainer, contextStorage } from "#context/container.js";+import { dispatchDynamicSkillEvent } from "#context/dynamic-skill-lifecycle.js";+import { dispatchDynamicSubagentEvent } from "#context/dynamic-subagent-lifecycle.js";+import { dispatchDynamicToolEvent } from "#context/dynamic-tool-lifecycle.js";+import { SessionIdKey, StaticModelReferenceKey } from "#context/keys.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import { mockModel } from "#evals/mock-model.js";+import {+ CallbackBaseUrlKey,+ clearPendingAuthorization,+ getPendingAuthorization,+ PendingAuthorizationResultKey,+} from "#harness/authorization.js";+import { createToolLoopHarness } from "#harness/tool-loop.js";+import type { HarnessSession, HarnessToolMap, StepInput, StepResult } from "#harness/types.js";+import {+ createApprovalContext,+ textStreamResult,+ toolCallsStreamResult,+} from "#internal/testing/approval-resume.js";+import {+ catalogBundle,+ connectionRegistry,+ connectionTool,+ fakeConnection,+ inlineTool,+ subagentTool,+ toolMap,+ workflowTool,+} from "#internal/testing/catalog-fixtures.js";+import { captureLogRecords } from "#internal/testing/log-records.js";+import { parkedSteps } from "#internal/testing/session-machine.js";+import {+ createSessionStartedEvent,+ createTurnStartedEvent,+ type UnstampedMessageStreamEvent,+} from "#protocol/message.js";+import { defineAgent } from "#public/definitions/agent.js";+import { defineSkill } from "#public/definitions/skill.js";+import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import { always } from "#tools/approval/policies.js";+import { defineTool } from "#tools/definition.js";+import { stampDurableDynamicToolCallbacks } from "#tools/durable-callbacks.js";++// The harness runs outside a workflow body here, where run attributes cannot+// be written; the attribute contract is covered by emit.test.ts.+vi.mock("#runtime/attributes/emit.js", () => ({ setEveAttributes: vi.fn(async () => {}) }));++/** The namespaces line once a dynamic `tenant__*` tool joins the static `ops__*` one. */+const NAMESPACES_OPS_TENANT = /^Namespaces, .*: ops, tenant$/mu;++/** Every catalog listing says this, whatever kinds it names. */+const LISTING_MARKER = "look for one with eve__search, which searches your own catalog";++type Reply = ReturnType<typeof textStreamResult>;++function call(toolCallId: string, toolName: string, input: unknown) {+ return { input: JSON.stringify(input), toolCallId, toolName };+}++const text = textStreamResult;+const calls = (...entries: ReturnType<typeof call>[]) => toolCallsStreamResult(entries);++/** Alice's session context, with what sign-in and dynamic subagents read. */+function createSessionContext(): ContextContainer {+ const ctx = createApprovalContext();+ ctx.set(CallbackBaseUrlKey, "https://agent.example.com");+ ctx.set(StaticModelReferenceKey, { id: "catalog-model" });+ return ctx;+}++/** One session on one harness; `drive` runs a step and every continuation it asks for. */+function createDriver(ctx: ContextContainer, tools: HarnessToolMap) {+ const replies: Reply[] = [];+ const model = new MockLanguageModelV4({+ doStream: async () => {+ const reply = replies.shift();+ if (reply === undefined) throw new Error("The model script ran out of replies.");+ return reply;+ },+ modelId: "catalog-model",+ provider: "eve-integration-mock",+ });+ const summary = mockModel("Alice and Bob handled refunds, credits, deploys, and lookups.");+ const events: UnstampedMessageStreamEvent[] = [];+ const harness = createToolLoopHarness({+ capabilities: { requestInput: true },+ handleEvent: async (event) => {+ events.push(event);+ },+ resolveModel: async (reference) => (reference.id === "summary" ? summary : model),+ tools,+ });+ const sessionId = ctx.require(SessionIdKey);+ const driver = {+ events,+ reply: (...next: Reply[]) => replies.push(...next),+ requests: () => model.doStreamCalls as LanguageModelV4CallOptions[],+ session: {+ agent: {+ compactionModelReference: { id: "summary" },+ modelReference: { id: "catalog-model" },+ system: "Help Alice run the billing desk.",+ tools: [],+ },+ compaction: { recentWindowSize: 2, threshold: 1_000_000 },+ continuationToken: `http:${sessionId}`,+ history: [],+ sessionId,+ } as HarnessSession,+ async drive(input?: StepInput): Promise<StepResult> {+ const step = (stepInput?: StepInput) =>+ contextStorage.run(ctx, () => harness(driver.session, stepInput));+ let result = await step(input);+ driver.session = result.session;+ while (typeof result.next === "function") {+ result = await step();+ driver.session = result.session;+ }+ return result;+ },+ };+ return driver;+}++/** A message as the provider receives it, without per-request provider options. */+function messageKey(message: LanguageModelV4Prompt[number]): string {+ return JSON.stringify({ content: message.content, role: message.role });+}++function messageText(message: LanguageModelV4Prompt[number]): string {+ if (typeof message.content === "string") return message.content;+ return message.content+ .map((part) => ("text" in part ? part.text : JSON.stringify(part)))+ .join("\n");+}++const systemText = (request: LanguageModelV4CallOptions) =>+ request.prompt.filter((message) => message.role === "system").map(messageText);+const conversation = (request: LanguageModelV4CallOptions) =>+ request.prompt.filter((message) => message.role !== "system").map(messageKey);++/** The catalog listing and diffs a request carries, in order. */+function catalogMessages(request: LanguageModelV4CallOptions): string[] {+ return request.prompt+ .filter((message) => message.role === "user")+ .map(messageText)+ .filter((entry) => entry.includes(LISTING_MARKER) || entry.startsWith("The catalog changed"));+}++/** What the model read back for `callId`: a JSON value or error text. */+function toolResult(request: LanguageModelV4CallOptions, callId: string): unknown {+ for (const message of request.prompt) {+ if (message.role !== "tool") continue;+ for (const part of message.content) {+ if (part.type === "tool-result" && part.toolCallId === callId && "value" in part.output) {+ return part.output.value;+ }+ }+ }+ throw new Error(`No result for ${callId} in the request.`);+}++/** Every tool-result part for `callId` in the request's prompt. */+function toolResultParts(+ request: LanguageModelV4CallOptions,+ callId: string,+): LanguageModelV4ToolResultPart[] {+ return request.prompt.flatMap((message) =>+ message.role === "tool"+ ? message.content.filter(+ (part): part is LanguageModelV4ToolResultPart =>+ part.type === "tool-result" && part.toolCallId === callId,+ )+ : [],+ );+}++/** A dynamic tool, stamped with the durable callback the bundler adds to authored resolvers. */+function dynamicTool(description: string) {+ const entry = defineTool({+ deferred: true,+ description,+ execute: async () => ({ synced: true }),+ inputSchema: { type: "object" },+ });+ stampDurableDynamicToolCallbacks(entry, {+ execute: { callback: () => ({ synced: true }), closure: {} },+ });+ return entry;+}++describe("step catalog in the harness (real AI SDK)", () => {+ it("keeps the six cache invariants while one session reaches every kind of entry", async () => {+ const ctx = createSessionContext();+ const privateCatalog = fakeConnection({+ description: "Private catalog that needs sign-in.",+ listing: "sign-in",+ name: "private",+ signIn: true,+ tools: [connectionTool("list_items")],+ });+ const connections = [privateCatalog];+ ctx.set(ConnectionRegistryKey, connectionRegistry(connections));+ ctx.set(+ BundleKey,+ catalogBundle({+ skills: [+ { deferred: true, description: "Fill and validate PDF forms.", name: "pdf-forms" },+ { deferred: true, description: "How to write release notes.", name: "release_notes" },+ {+ description: "House rules for replies.",+ markdown: "# House rules",+ name: "house-rules",+ },+ ],+ }),+ );+ const tools = toolMap(+ inlineTool("add"),+ inlineTool("refund_invoice", {+ deferred: true,+ schema: {+ type: "object",+ properties: { invoiceId: { type: "string" } },+ required: ["invoiceId"],+ },+ }),+ inlineTool("issue_credit", { approval: always(), deferred: true }),+ inlineTool("release_notes", { deferred: true, description: "Publish release notes." }),+ inlineTool("ops__restart", { deferred: true, description: "Restart an ops service." }),+ workflowTool("deploy_service", "execute", { deferred: true }),+ workflowTool("research", "task", { deferred: true }),+ subagentTool("billing_specialist", { deferred: true }),+ );+ const driver = createDriver(ctx, tools);+ const { drive, reply } = driver;+ const mark = () => driver.requests().length;+ let tenantSyncAvailable = true;+ const resolveTenantTools = () =>+ dispatchDynamicToolEvent({+ ctx,+ event: createSessionStartedEvent(),+ messages: [],+ resolvers: [+ {+ eventNames: ["session.started"],+ events: {+ "session.started": () =>+ tenantSyncAvailable ? { tenant__sync: dynamicTool("Sync the tenant.") } : null,+ },+ logicalPath: "agent/tools/tenant.ts",+ slug: "tenant",+ sourceId: "tools/tenant.ts",+ sourceKind: "module",+ },+ ],+ });++ // A search, then an inline tool through eve__tool.+ reply(+ calls(call("search-refund", SEARCH_TOOL_NAME, { query: "refund" })),+ calls(+ call("refund", CALL_TOOL_NAME, { input: { invoiceId: "in_1" }, name: "refund_invoice" }),+ ),+ text("Refunded in_1."),+ );+ await drive({ message: "Alice asks for a refund of invoice in_1." });++ // An approval, during which a dynamic deferred tool appears. The catalog+ // changes on the step that runs the approved call.+ reply(calls(call("credit", CALL_TOOL_NAME, { name: "issue_credit" })));+ const awaitingApproval = await drive({ message: "Alice asks for a credit for Bob." });+ const [approval] = parkedSteps(awaitingApproval.session).flatMap((step) => step.requests);+ expect(approval?.action.toolName).toBe("issue_credit");+ await resolveTenantTools();+ const approvalStep = mark();+ reply(+ calls(call("search-sync", SEARCH_TOOL_NAME, { query: "sync" })),+ text("Credited Bob and found the sync tool."),+ );+ await drive({ inputResponses: [{ optionId: "approve", requestId: approval!.requestId }] });++ // A foreground workflow tool parks the turn and resumes with its result.+ reply(+ calls(call("deploy", CALL_TOOL_NAME, { input: { service: "api" }, name: "deploy_service" })),+ );+ const deploying = await drive({ message: "Alice asks to deploy the api service." });+ expect(+ parkedSteps(deploying.session).flatMap((step) => step.tasks.map((task) => task.toolName)),+ ).toEqual(["deploy_service"]);+ reply(text("Deployed api."));+ await drive({+ runtimeActionResults: [+ {+ callId: "deploy",+ kind: "tool-result",+ output: "deployed api",+ toolName: "deploy_service",+ },+ ],+ });++ // A background workflow tool and a deferred subagent each start a task.+ reply(+ calls(+ call("research", CALL_TOOL_NAME, { input: { topic: "refunds" }, name: "research" }),+ call("delegate", CALL_TOOL_NAME, {+ input: { message: "Review Bob's dispute." },+ name: "billing_specialist",+ }),+ ),+ );+ const delegating = await drive({ message: "Alice asks for research and a dispute review." });+ expect(+ parkedSteps(delegating.session).flatMap((step) => step.tasks.map((task) => task.toolName)),+ ).toEqual(["research", "billing_specialist"]);+ reply(text("Both are underway."));+ await drive({+ runtimeActionResults: [+ {+ callId: "research",+ kind: "tool-result",+ output: "Started task research-1.",+ toolName: "research",+ },+ {+ callId: "delegate",+ kind: "tool-result",+ output: "Started task billing_specialist-1.",+ toolName: "billing_specialist",+ },+ ],+ });++ // A connection tool parks the turn for sign-in, then runs once the user signs in.+ reply(calls(call("items", CALL_TOOL_NAME, { name: "private__list_items" })));+ const signingIn = await drive({ message: "Alice wants the items in the private catalog." });+ const [challenge] = getPendingAuthorization(signingIn.session.state)?.challenges ?? [];+ expect(challenge?.name).toBe("private");+ expect(privateCatalog.signIns).toHaveLength(1);+ // Delivers the callback the way the turn step does when the user signs in.+ ctx.set(PendingAuthorizationResultKey, [+ {+ attemptId: challenge!.attemptId,+ callback: { method: "GET", params: { code: "alice" } },+ hookUrl: challenge!.hookUrl,+ instanceId: challenge!.instanceId,+ name: "private",+ },+ ]);+ driver.session = {+ ...driver.session,+ state: clearPendingAuthorization(driver.session.state, [challenge!.attemptId!]),+ };+ reply(+ calls(call("items-after-sign-in", CALL_TOOL_NAME, { name: "private__list_items" })),+ text("The private catalog has Alice's lamp."),+ );+ await drive();+ expect(privateCatalog.calls).toEqual([{ input: {}, tool: "list_items" }]);++ // A dynamic connection resolves.+ connections.push(+ fakeConnection({+ description: "Caller-specific product catalog.",+ name: "products",+ tools: [connectionTool("get_status")],+ }),+ );+ const connectionStep = mark();+ reply(+ calls(call("status", CALL_TOOL_NAME, { name: "products__get_status" })),+ text("The product catalog is up."),+ );+ await drive({ message: "Alice asks whether the product catalog is up." });++ // A dynamic deferred subagent appears and the dynamic tool disappears.+ await dispatchDynamicSubagentEvent({+ ctx,+ event: createSessionStartedEvent(),+ messages: [],+ resolvers: [+ {+ eventNames: ["session.started"],+ events: {+ "session.started": () =>+ defineAgent({+ description: "Answer questions about the tenant's plan.",+ model: "openai/gpt-5.5",+ modelContextWindowTokens: 200_000,+ tool: "deferred",+ }),+ },+ kind: "subagent",+ logicalPath: "subagents/plan_advisor/agent.ts",+ name: "plan_advisor",+ nodeId: "subagents/plan_advisor",+ sourceId: "subagents/plan_advisor/agent.ts",+ sourceKind: "module",+ },+ ],+ });+ tenantSyncAvailable = false;+ await resolveTenantTools();+ const changedStep = mark();+ reply(+ calls(call("sync-again", CALL_TOOL_NAME, { name: "tenant__sync" })),+ text("The sync tool is gone."),+ );+ await drive({ message: "Alice asks to sync the tenant again." });+ // An entry the resolver no longer returns can't be called.+ const gone = toolResult(driver.requests().at(-1)!, "sync-again");+ expect(gone).toContain("No tool named");+ expect(gone).toContain("tenant__sync");++ // A tool and a skill share a name; skills load deferred or not, including+ // a dynamic deferred skill that appears when the turn starts. It joins the+ // listed `ops` namespace, so the listing doesn't change.+ await dispatchDynamicSkillEvent({+ ctx,+ event: createTurnStartedEvent({ sequence: 9, turnId: "turn_9" }),+ messages: [],+ resolvers: [+ {+ eventNames: ["turn.started"],+ events: {+ "turn.started": () => ({+ ops__playbook: defineSkill({+ deferred: true,+ description: "The ops escalation playbook.",+ markdown: "# Ops playbook",+ }),+ }),+ },+ exportName: "default",+ logicalPath: "skills/playbooks.ts",+ slug: "playbooks",+ sourceId: "skills/playbooks.ts",+ sourceKind: "module",+ },+ ],+ });+ const skillStep = mark();+ reply(+ calls(call("search-notes", SEARCH_TOOL_NAME, { query: "release notes" })),+ calls(+ call("notes", SKILL_TOOL_NAME, { name: "release_notes" }),+ call("forms", SKILL_TOOL_NAME, { name: "pdf-forms" }),+ call("playbook", SKILL_TOOL_NAME, { name: "ops__playbook" }),+ call("rules", SKILL_TOOL_NAME, { name: "house-rules" }),+ ),+ text("Loaded the skills."),+ );+ await drive({ message: "Alice asks how to publish release notes and escalate." });+ expect(toolResult(driver.requests()[skillStep + 1]!, "search-notes")).toMatchObject({+ results: expect.arrayContaining([+ expect.objectContaining({ tool: "release_notes" }),+ {+ description: "How to write release notes.",+ path: "$HOME/.agents/skills/release_notes/SKILL.md",+ skill: "release_notes",+ },+ ]),+ });+ const loaded = JSON.stringify(driver.requests().at(-1)!.prompt);+ for (const markdown of ["# release_notes", "# pdf-forms", "# Ops playbook", "# House rules"]) {+ expect(loaded).toContain(markdown);+ }++ const historyBeforeCompaction = driver.session.history;++ // Compaction replaces history; the next request starts a fresh baseline.+ // Any real history exceeds a threshold of 1, so the next step compacts.+ const compactionStep = mark();+ driver.session = {+ ...driver.session,+ compaction: { ...driver.session.compaction, threshold: 1 },+ };+ reply(text("Ready for the next request."));+ await drive({ message: "Alice asks what is next." });++ const requests = driver.requests();+ const listingFor = (index: number) => catalogMessages(requests[index]!);++ // 1. Fixed tools: the same names, descriptions, schemas, and order on every request.+ expect(requests[0]!.tools?.map((tool) => tool.name)).toEqual([+ "add",+ TASK_WAIT_TOOL_NAME,+ TASK_CANCEL_TOOL_NAME,+ SEARCH_TOOL_NAME,+ CALL_TOOL_NAME,+ SKILL_TOOL_NAME,+ ]);+ for (const request of requests) expect(request.tools).toEqual(requests[0]!.tools);++ // 2. No session-specific text in the system prompt or any tool description.+ const fixedText = [+ ...systemText(requests[0]!),+ ...(requests[0]!.tools ?? []).map((tool) => ("description" in tool ? tool.description : "")),+ ].join("\n");+ for (const name of [+ "refund_invoice",+ "deploy_service",+ "billing_specialist",+ "tenant__sync",+ "plan_advisor",+ "pdf-forms",+ "ops__playbook",+ "private",+ "products",+ ]) {+ expect(fixedText).not.toContain(name);+ }+ for (const request of requests) expect(systemText(request)).toEqual(systemText(requests[0]!));++ // 3. Append-only history: each request extends the one before it, until compaction.+ for (let index = 1; index < requests.length; index += 1) {+ if (index === compactionStep) continue;+ const previous = conversation(requests[index - 1]!);+ expect(conversation(requests[index]!).slice(0, previous.length), `request ${index}`).toEqual(+ previous,+ );+ }++ // 4. No system-message fallback, including on the step that runs approved+ // calls: the approved call's result, then the listing, which gained the+ // `tenant` namespace, as an appended message.+ const approvedStep = requests[approvalStep]!;+ expect(toolResult(approvedStep, "credit")).toEqual({ input: {}, ran: "issue_credit" });+ expect(approvedStep.prompt.at(-2)?.role).toBe("tool");+ expect(approvedStep.prompt.at(-1)).toEqual({+ content: [{ text: expect.stringMatching(NAMESPACES_OPS_TENANT), type: "text" }],+ role: "user",+ });+ expect(listingFor(approvalStep).at(-1)).toMatch(/^The catalog changed\./u);+ expect(systemText(approvedStep).join("\n")).not.toContain("tenant");+ expect(listingFor(approvalStep)).toHaveLength(2);++ // 5. Deterministic rendering is owned by listing.test.ts; here each change+ // appends the listing again, and compaction starts a fresh baseline.+ expect(listingFor(0)).toEqual([+ expect.stringContaining("You have more tools, agents, and skills than are loaded here"),+ ]);+ expect(listingFor(connectionStep).at(-1)).toContain(+ "- products: Caller-specific product catalog.",+ );+ expect(listingFor(changedStep).at(-1)).toMatch(/\nNo longer available: tenant$/u);+ expect(listingFor(compactionStep)).toEqual([expect.stringMatching(/^Namespaces, .*: ops$/mu)]);++ // 6. Calling an entry adds nothing: only catalog changes append listing+ // messages. A dynamic agent of a kind already listed, or a skill joining a+ // listed namespace (skillStep), changes nothing the listing says.+ const announcedAt = requests.flatMap((_request, index) =>+ index > 0 && listingFor(index).length > listingFor(index - 1).length ? [index] : [],+ );+ expect(announcedAt).toEqual([approvalStep, connectionStep, changedStep]);++ // The stand-in result for calls the harness dispatches after the step never+ // reaches the model or the protocol: until compaction, each prompt carries+ // exactly one result per call, and it is the dispatched call's real result.+ for (const callId of ["deploy", "research", "delegate"]) {+ const firstResult = requests.findIndex(+ (request) => toolResultParts(request, callId).length > 0,+ );+ expect(firstResult, callId).toBeGreaterThan(0);+ for (const request of requests.slice(firstResult, compactionStep)) {+ expect(+ toolResultParts(request, callId).map((part) => part.output),+ callId,+ ).toEqual([expect.objectContaining({ value: expect.anything() })]);+ }+ }+ for (const callId of ["deploy", "research", "delegate"]) {+ const results = driver.events.filter(+ (event) => event.type === "action.result" && event.data.result.callId === callId,+ );+ expect(results, callId).toHaveLength(1);+ expect(JSON.stringify(results), callId).not.toContain("dispatched");+ }++ // History keeps the model's own catalog calls; actions carry each entry's name.+ const historyCalls = historyBeforeCompaction.flatMap((message) =>+ message.role === "assistant" && Array.isArray(message.content)+ ? message.content.flatMap((part) => (part.type === "tool-call" ? [part.toolName] : []))+ : [],+ );+ expect(new Set(historyCalls)).toEqual(+ new Set([CALL_TOOL_NAME, SEARCH_TOOL_NAME, SKILL_TOOL_NAME]),+ );+ expect(+ driver.events.flatMap((event) =>+ event.type === "actions.requested"+ ? event.data.actions.map((action) =>+ "toolName" in action ? action.toolName : "name" in action ? action.name : action.kind,+ )+ : [],+ ),+ ).toEqual([+ SEARCH_TOOL_NAME,+ "refund_invoice",+ "issue_credit",+ SEARCH_TOOL_NAME,+ "deploy_service",+ "research",+ "billing_specialist",+ // The call that parked for sign-in, then the model's call after it.+ "private__list_items",+ "private__list_items",+ "products__get_status",+ SEARCH_TOOL_NAME,+ "load-skill",+ "load-skill",+ "load-skill",+ "load-skill",+ ]);+ });++ it("names the entry, not eve__tool, when a deferred entry returns a result that isn't JSON", async () => {+ const logs = captureLogRecords();+ const ctx = createSessionContext();+ ctx.set(BundleKey, catalogBundle());+ const circular: Record<string, unknown> = {};+ circular.self = circular;+ const driver = createDriver(+ ctx,+ toolMap(+ inlineTool("add"),+ inlineTool("export_ledger", { deferred: true, execute: async () => circular }),+ ),+ );+ driver.reply(+ calls(call("export", CALL_TOOL_NAME, { name: "export_ledger" })),+ text("The ledger export failed."),+ );++ await driver.drive({ message: "Alice asks for the ledger export." });++ expect(toolResult(driver.requests()[1]!, "export")).toContain(+ 'Tool "export_ledger" call "export" returned a non-JSON-serializable result.',+ );+ expect(logs.records).toContainEqual(+ expect.objectContaining({+ fields: expect.objectContaining({ toolCallId: "export", toolName: "export_ledger" }),+ message: "tool execution failed",+ }),+ );+ });++ it("gives an agent with only listed skills eve__skill to load them, no search, and no listing", async () => {+ const ctx = createSessionContext();+ ctx.set(BundleKey, catalogBundle({ skills: [{ name: "house-rules" }] }));+ const driver = createDriver(ctx, toolMap(inlineTool("add")));+ driver.reply(+ calls(call("load-rules", SKILL_TOOL_NAME, { name: "house-rules" })),+ calls(call("add", "add", {})),+ text("Done, following the house rules."),+ );++ await driver.drive({ message: "Alice asks for a sum, following the house rules." });++ const requests = driver.requests();+ expect(requests).toHaveLength(3);+ for (const request of requests) {+ expect(request.tools?.map((tool) => tool.name)).toEqual(["add", SKILL_TOOL_NAME]);+ expect(catalogMessages(request)).toEqual([]);+ }+ expect(toolResult(requests[1]!, "load-rules")).toBe("# house-rules");+ });++ it("validates a connection tool's input before asking anyone to approve the call", async () => {+ const ctx = createSessionContext();+ const crm = fakeConnection({+ approval: always(),+ name: "crm",+ tools: [+ connectionTool("archive_account", {+ type: "object",+ properties: { accountId: { type: "string" } },+ required: ["accountId"],+ }),+ ],+ });+ ctx.set(ConnectionRegistryKey, connectionRegistry([crm]));+ ctx.set(BundleKey, catalogBundle({ connections: [crm.definition] }));+ const driver = createDriver(ctx, toolMap(inlineTool("add")));+ driver.reply(+ calls(call("archive-unnamed", CALL_TOOL_NAME, { input: {}, name: "crm__archive_account" })),+ calls(+ call("archive", CALL_TOOL_NAME, {+ input: { accountId: "acct_1" },+ name: "crm__archive_account",+ }),+ ),+ );++ const parked = await driver.drive({ message: "Alice asks to archive Bob's account." });++ const requests = parkedSteps(parked.session).flatMap((step) => step.requests);+ expect(requests.map((request) => request.action)).toEqual([+ expect.objectContaining({ callId: "archive", toolName: "crm__archive_account" }),+ ]);+ expect(toolResult(driver.requests()[1]!, "archive-unnamed")).toContain(+ "Signature: crm__archive_account(input: { accountId: string })",+ );+ expect(crm.calls).toEqual([]);++ driver.reply(text("Archived Bob's account."));+ await driver.drive({+ inputResponses: [{ optionId: "approve", requestId: requests[0]!.requestId }],+ });+ expect(crm.calls).toEqual([{ input: { accountId: "acct_1" }, tool: "archive_account" }]);+ });+});diff --git a/packages/eve/src/execution/catalog/step-catalog.test.ts b/packages/eve/src/execution/catalog/step-catalog.test.tsnew file mode 100644index 000000000..c35c68e81--- /dev/null+++ b/packages/eve/src/execution/catalog/step-catalog.test.ts@@ -0,0 +1,651 @@+import { describe, expect, it, vi } from "vitest";+import { CALL_TOOL_NAME, SEARCH_TOOL_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";+import { TASK_CANCEL_TOOL_NAME, TASK_WAIT_TOOL_NAME } from "#protocol/task-tools.js";++import type { StandardSchemaV1 } from "#compiled/@standard-schema/spec/index.js";+import { isAuthorizationSignal, PendingAuthorizationResultKey } from "#harness/authorization.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import { ContextContainer } from "#context/container.js";+import {+ catalogBundle,+ catalogContext,+ connectionTool,+ fakeConnection,+ inlineTool,+ subagentTool,+ toolMap,+ workflowTool,+} from "#internal/testing/catalog-fixtures.js";+import { captureLogRecords } from "#internal/testing/log-records.js";++import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";+import { serializeInputSchema, type ToolSchemaSource } from "#tools/schema.js";++import { buildStepCatalog, type StepCatalog } from "./step-catalog.js";++const CHILD = { rootSessionId: "root-session" };++const names = (tools: ReadonlyMap<string, HarnessToolDefinition>) => [...tools.keys()];++/** Validates `input` against a catalog tool's schema, which every call passes before it runs. */+function validate(catalog: StepCatalog, tool: string, input: unknown) {+ const schema = catalog.advertised.get(tool)!.inputSchema as StandardSchemaV1;+ return schema["~standard"].validate(input);+}+const validateTool = (catalog: StepCatalog, input: unknown) =>+ validate(catalog, CALL_TOOL_NAME, input);+const validateSkill = (catalog: StepCatalog, input: unknown) =>+ validate(catalog, SKILL_TOOL_NAME, input);++/** A catalog tool's description and input fields, as the model reads them. */+function shape(catalog: StepCatalog, tool: string) {+ const definition = catalog.advertised.get(tool)!;+ const schema = serializeInputSchema(definition.inputSchema as ToolSchemaSource) as {+ properties: Record<string, unknown>;+ required?: string[];+ };+ return {+ description: definition.description,+ fields: Object.keys(schema.properties),+ required: schema.required,+ };+}++const LISTED_SKILL_DESCRIPTION =+ "Load a skill's instructions when the request clearly matches one of your listed skills or the user asks for it, then follow them.";+const CALL_DESCRIPTION =+ "Call a tool that isn't in your tool list by its exact name from eve__search, with `input` matching its signature.";++describe("buildStepCatalog", () => {+ it("lists direct entries, then the catalog tools, and keeps deferred entries out of the tool list", () => {+ const { catalog } = catalogContext({+ tools: [inlineTool("add"), inlineTool("refund_invoice", { deferred: true })],+ });++ expect(names(catalog.advertised)).toEqual(["add", SEARCH_TOOL_NAME, CALL_TOOL_NAME]);+ expect(names(catalog.deferred)).toEqual(["refund_invoice"]);+ });++ describe("which catalog tools the agent gets", () => {+ /** The tool list of an agent that declares one dynamic resolver of `kind`. */+ function advertisedWithResolver(+ kind: "dynamicConnectionResolvers" | "dynamicSkillResolvers" | "dynamicToolResolvers",+ ) {+ const bundle = catalogBundle();+ const ctx = new ContextContainer();+ ctx.set(BundleKey, { ...bundle, resolvedAgent: { ...bundle.resolvedAgent, [kind]: [{}] } });+ return names(+ buildStepCatalog({ agentTools: toolMap(), ctx, endsTurn: true, session: {} }).advertised,+ );+ }++ it("counts only the deferred tools the session sees, so a delegated session doesn't search for root-only ones", () => {+ const tools = [inlineTool("archive", { availableInSubagents: false, deferred: true })];++ expect(names(catalogContext({ tools }).catalog.advertised)).toEqual([+ SEARCH_TOOL_NAME,+ CALL_TOOL_NAME,+ ]);+ expect(names(catalogContext({ session: CHILD, tools }).catalog.advertised)).toEqual([]);+ });++ it("gives an agent with nothing to find, call, or load no catalog tools", () => {+ const { catalog } = catalogContext({ tools: [inlineTool("add")] });++ expect(names(catalog.advertised)).toEqual(["add"]);+ });++ it("gives an agent with only listed skills eve__skill alone, which never mentions search", async () => {+ const { catalog } = catalogContext({ skills: [{ name: "house-rules" }] });++ expect(names(catalog.advertised)).toEqual([SKILL_TOOL_NAME]);+ expect(shape(catalog, SKILL_TOOL_NAME)).toEqual({+ description: LISTED_SKILL_DESCRIPTION,+ fields: ["name"],+ required: ["name"],+ });+ expect(await validateSkill(catalog, { name: "house-rules" })).toEqual({+ value: { name: "house-rules" },+ });+ const unknown = JSON.stringify(await validateSkill(catalog, { name: "house_rules" }));+ expect(unknown).toContain('No skill named \\"house_rules\\"');+ expect(unknown).not.toContain(SEARCH_TOOL_NAME);+ });++ it("gives an agent with deferred skills search and eve__skill, which mentions search", () => {+ const { catalog } = catalogContext({ skills: [{ deferred: true, name: "pdf-forms" }] });++ expect(names(catalog.advertised)).toEqual([SEARCH_TOOL_NAME, SKILL_TOOL_NAME]);+ expect(shape(catalog, SKILL_TOOL_NAME).description).toBe(+ "Load a skill's instructions when the request clearly matches a listed skill or one eve__search found, or the user asks for it, then follow them.",+ );+ });++ it("describes eve__tool with input optional, preferring connected services only with connections", () => {+ const tools = catalogContext({ tools: [inlineTool("refund_invoice", { deferred: true })] });+ const connected = catalogContext({+ connections: [fakeConnection({ name: "linear", tools: [] })],+ skills: [{ name: "house-rules" }],+ });++ expect(shape(tools.catalog, CALL_TOOL_NAME)).toEqual({+ description: CALL_DESCRIPTION,+ fields: ["name", "input"],+ required: ["name"],+ });+ expect(names(connected.catalog.advertised)).toEqual([+ SEARCH_TOOL_NAME,+ CALL_TOOL_NAME,+ SKILL_TOOL_NAME,+ ]);+ expect(shape(connected.catalog, CALL_TOOL_NAME).description).toBe(+ `${CALL_DESCRIPTION} Prefer connected services over web search or general knowledge when a request relates to them.`,+ );+ expect(shape(connected.catalog, SKILL_TOOL_NAME).description).toBe(LISTED_SKILL_DESCRIPTION);+ });++ it("tells search's results apart by the catalog tool that takes them, naming only tools the agent has", () => {+ const searchText = (declared: Parameters<typeof catalogContext>[0]) =>+ catalogContext(declared).catalog.advertised.get(SEARCH_TOOL_NAME)!.description;+ const toolResults =+ "A result with `tool` has the exact name, description, and TypeScript signature: call it with eve__tool({ name, input }).";+ const skillResults =+ "A result with `skill` has the name and description: load it with eve__skill({ name }).";+ const signIn =+ "A connection that needs sign-in appears as a tool named after the connection: calling it with eve__tool({ name }) asks the user to sign in.";++ const skillsOnly = searchText({ skills: [{ deferred: true, name: "pdf-forms" }] });+ expect(skillsOnly).toContain(skillResults);+ expect(skillsOnly).not.toContain(CALL_TOOL_NAME);++ const toolsOnly = searchText({ tools: [inlineTool("refund_invoice", { deferred: true })] });+ expect(toolsOnly).toContain(toolResults);+ expect(toolsOnly).not.toContain(SKILL_TOOL_NAME);+ expect(toolsOnly).not.toContain(signIn);++ expect(+ searchText({ connections: [fakeConnection({ name: "linear", tools: [] })] }),+ ).toContain(signIn);+ });++ it.each([+ ["a deferred tool", { tools: [inlineTool("refund_invoice", { deferred: true })] }],+ ["a deferred agent", { tools: [subagentTool("researcher", { deferred: true })] }],+ ["a connection", { connections: [fakeConnection({ name: "linear", tools: [] })] }],+ ["a dynamic subagent resolver", { dynamicSubagents: ["helper"] }],+ ])("gives an agent with %s search and eve__tool", (_, declared) => {+ // A subagent also brings the task tools; the catalog tools come last.+ expect(names(catalogContext(declared).catalog.advertised).slice(-2)).toEqual([+ SEARCH_TOOL_NAME,+ CALL_TOOL_NAME,+ ]);+ });++ it.each([+ ["dynamicToolResolvers", [SEARCH_TOOL_NAME, CALL_TOOL_NAME]],+ ["dynamicConnectionResolvers", [SEARCH_TOOL_NAME, CALL_TOOL_NAME]],+ ["dynamicSkillResolvers", [SEARCH_TOOL_NAME, SKILL_TOOL_NAME]],+ ] as const)(+ "gives an agent with %s its catalog tools, whatever they resolve",+ (kind, tools) => {+ expect(advertisedWithResolver(kind)).toEqual(tools);+ },+ );+ });++ it("hides root-only entries, direct or deferred, from delegated sessions", async () => {+ const tools = [+ inlineTool("add"),+ inlineTool("notify_team", { availableInSubagents: false }),+ inlineTool("billing", { behavior: { availability: ["root-session"] } }),+ inlineTool("archive", { availableInSubagents: false, deferred: true }),+ subagentTool("agent", { rootOnly: true }),+ subagentTool("delegate"),+ subagentTool("researcher", { deferred: true }),+ ];++ const root = catalogContext({ tools }).catalog;+ expect(names(root.advertised)).toEqual([+ "add",+ "notify_team",+ "billing",+ "agent",+ "delegate",+ TASK_WAIT_TOOL_NAME,+ TASK_CANCEL_TOOL_NAME,+ SEARCH_TOOL_NAME,+ CALL_TOOL_NAME,+ ]);+ expect(names(root.deferred)).toEqual(["archive", "researcher"]);++ const child = catalogContext({ session: CHILD, tools }).catalog;+ expect(names(child.advertised)).toEqual([+ "add",+ "delegate",+ TASK_WAIT_TOOL_NAME,+ TASK_CANCEL_TOOL_NAME,+ SEARCH_TOOL_NAME,+ CALL_TOOL_NAME,+ ]);+ expect(names(child.deferred)).toEqual(["researcher"]);+ // A hidden entry can't be reached through eve__tool either.+ expect(await validateTool(child, { name: "archive" })).toEqual({+ issues: [+ { message: 'No tool named "archive". Find tools with eve__search.', path: ["name"] },+ ],+ });+ });++ describe("task tools", () => {+ const offered = (catalog: StepCatalog) => ({+ advertised:+ catalog.advertised.has(TASK_WAIT_TOOL_NAME) &&+ catalog.advertised.has(TASK_CANCEL_TOOL_NAME),+ offersTasks: catalog.offersTasks,+ });+ const offeredFor = (...input: Parameters<typeof catalogContext>) =>+ offered(catalogContext(...input).catalog);+ const OFFERED = { advertised: true, offersTasks: true };+ const NOT_OFFERED = { advertised: false, offersTasks: false };++ it("are offered while a tool the session sees starts tasks, deferred or not", () => {+ expect(offeredFor({ tools: [inlineTool("add")] })).toEqual(NOT_OFFERED);+ expect(offeredFor({ tools: [workflowTool("deploy_service")] })).toEqual(NOT_OFFERED);+ expect(offeredFor({ tools: [workflowTool("research", "task", { deferred: true })] })).toEqual(+ OFFERED,+ );+ expect(offeredFor({ tools: [subagentTool("delegate")] })).toEqual(OFFERED);+ });++ it("are not offered to a leaf subagent session whose only task-starting tool is root-only", () => {+ const tools = [inlineTool("add"), subagentTool("agent", { rootOnly: true })];++ expect(offeredFor({ tools })).toEqual(OFFERED);+ expect(offeredFor({ session: CHILD, tools })).toEqual(NOT_OFFERED);+ });++ it("are offered when the agent resolves subagents at runtime", () => {+ expect(offeredFor({ dynamicSubagents: ["specialist"], tools: [inlineTool("add")] })).toEqual(+ OFFERED,+ );+ });+ });++ describe("resolve", () => {+ const refundInvoice = inlineTool("refund_invoice", { deferred: true });+ const add = inlineTool("add");+ const { catalog } = catalogContext({+ skills: [{ name: "pdf-forms", deferred: true, markdown: "# PDF forms" }],+ tools: [add, refundInvoice, subagentTool("billing_specialist", { deferred: true })],+ });++ it("resolves an eve__tool call to the entry it names, keeping the model's call id", () => {+ const call = {+ input: { input: { invoiceId: "in_1" }, name: "refund_invoice" },+ toolCallId: "call-1",+ toolName: CALL_TOOL_NAME,+ };++ expect(catalog.resolve(call)).toEqual({+ call: { input: { invoiceId: "in_1" }, toolCallId: "call-1", toolName: "refund_invoice" },+ definition: refundInvoice,+ });+ expect(+ catalog.resolve({ input: { name: "billing_specialist" }, toolName: CALL_TOOL_NAME })?.call,+ ).toEqual({ input: {}, toolName: "billing_specialist" });+ });++ it("resolves an eve__skill call to the skill loader, which returns the skill's markdown", async () => {+ const resolved = catalog.resolve({ input: { name: "pdf-forms" }, toolName: SKILL_TOOL_NAME });++ expect(resolved?.call).toEqual({ input: { skill: "pdf-forms" }, toolName: "eve:load-skill" });+ const label = resolved?.definition.label;+ expect(label?.start?.(resolved?.call.input)).toBe("Load skill pdf-forms");+ expect(label?.complete?.(resolved?.call.input, "# PDF forms")).toBe("Loaded skill pdf-forms");+ expect(+ await resolved?.definition.execute?.(resolved.call.input, { messages: [], toolCallId: "" }),+ ).toBe("# PDF forms");+ });++ it("resolves a direct call only to a listed tool", () => {+ expect(catalog.resolve({ input: {}, toolName: "add" })?.definition).toBe(add);+ expect(catalog.resolve({ input: {}, toolName: "refund_invoice" })).toBeUndefined();+ expect(catalog.resolve({ input: { name: "add" }, toolName: CALL_TOOL_NAME })).toBeUndefined();+ expect(+ catalog.resolve({ input: { name: "refund_invoice" }, toolName: SKILL_TOOL_NAME }),+ ).toBeUndefined();+ });+ });++ describe("eve__tool and eve__skill validation", () => {+ const { catalog } = catalogContext({+ connections: [fakeConnection({ name: "linear", tools: [] })],+ skills: [+ { name: "release_notes", deferred: true },+ { name: "release-checklist" },+ { name: "refund_invoice", deferred: true },+ ],+ tools: [+ inlineTool("add"),+ inlineTool("refund_invoice", {+ deferred: true,+ schema: {+ type: "object",+ properties: { invoiceId: { type: "string" } },+ required: ["invoiceId"],+ additionalProperties: false,+ },+ }),+ inlineTool("refund_payment", { deferred: true }),+ inlineTool("stripe_list_disputes", { deferred: true }),+ ],+ });++ it.each([+ [+ "an unknown tool, with the closest names",+ "refund_invoce",+ 'No tool named "refund_invoce". Closest tools: refund_invoice, refund_payment.',+ ],+ [+ "an unknown tool with no close names",+ "weather",+ 'No tool named "weather". Find tools with eve__search.',+ ],+ ["a tool in the model's tool list", "add", '"add" is in your tool list; call it directly.'],+ [+ "eve__tool itself",+ CALL_TOOL_NAME,+ `"${CALL_TOOL_NAME}" is in your tool list; call it directly.`,+ ],+ [+ "a skill named as a tool when no tool has its name",+ "release_notes",+ '"release_notes" is a skill; load it with eve__skill({ name: "release_notes" }).',+ ],+ ])("eve__tool rejects %s", async (_case, name, message) => {+ expect(await validateTool(catalog, { name })).toEqual({+ issues: [{ message, path: ["name"] }],+ });+ });++ it.each([+ [+ "an unknown skill, with the closest names",+ "release",+ 'No skill named "release". Closest skills: release-checklist, release_notes.',+ ],+ [+ "a skill named like a connection",+ "Linear",+ 'No skill named "Linear". Find skills with eve__search. "linear" is a connection, not a skill. Find its tools with eve__search({ query: "linear__" }).',+ ],+ [+ "a deferred tool's name",+ "refund_payment",+ '"refund_payment" is a tool, not a skill; call it with eve__tool({ name: "refund_payment" }).',+ ],+ [+ "a listed tool's name",+ "add",+ '"add" is a tool in your tool list, not a skill; call it directly.',+ ],+ ])("eve__skill rejects %s", async (_case, name, message) => {+ expect(await validateSkill(catalog, { name })).toEqual({+ issues: [{ message, path: ["name"] }],+ });+ });++ it("doesn't point eve__skill at eve__search when search can't find skills", async () => {+ const { catalog: toolSearch } = catalogContext({+ skills: [{ name: "house-rules" }],+ tools: [inlineTool("refund_invoice", { deferred: true })],+ });++ expect(await validateSkill(toolSearch, { name: "weather" })).toEqual({+ issues: [{ message: 'No skill named "weather".', path: ["name"] }],+ });+ });++ it("calls the tool and loads the skill that share a name, since tools and skills have separate names", async () => {+ expect(+ await validateTool(catalog, { input: { invoiceId: "in_1" }, name: "refund_invoice" }),+ ).toEqual({ value: { input: { invoiceId: "in_1" }, name: "refund_invoice" } });+ expect(await validateSkill(catalog, { name: "refund_invoice" })).toEqual({+ value: { name: "refund_invoice" },+ });+ });++ it("reports invalid input as the entry's own issues under input, with its signature", async () => {+ expect(+ await validateTool(catalog, { input: { invoice: 1 }, name: "refund_invoice" }),+ ).toEqual({+ issues: [+ { message: 'Unrecognized key: "invoice"', path: ["input"] },+ {+ message: "Signature: refund_invoice(input: { invoiceId: string }): Promise<unknown>",+ path: ["input"],+ },+ ],+ });+ });+ });++ describe("connection entries", () => {+ const listIssues = connectionTool("list_issues", {+ type: "object",+ properties: { team: { type: "string" }, limit: { type: "integer", default: 20 } },+ required: ["team"],+ additionalProperties: false,+ });++ function linearCatalog(listing?: Parameters<typeof fakeConnection>[0]["listing"]) {+ const linear = fakeConnection({+ listing,+ name: "linear",+ tools: [listIssues, connectionTool("list_projects")],+ });+ return { linear, ...catalogContext({ connections: [linear], tools: [inlineTool("add")] }) };+ }++ it("resolves a name under a connection's prefix to that connection's tool, filling schema defaults", async () => {+ const { catalog, linear, run } = linearCatalog();+ const validated = { input: { limit: 20, team: "core" }, name: "linear__list_issues" };++ expect(+ await validateTool(catalog, { input: { team: "core" }, name: "linear__list_issues" }),+ ).toEqual({ value: validated });+ const resolved = catalog.resolve({ input: validated, toolName: CALL_TOOL_NAME });+ expect(resolved?.call.toolName).toBe("linear__list_issues");+ expect(resolved?.definition.label?.start?.({})).toBe("Linear: List issues");+ await run(() =>+ resolved?.definition.execute?.(resolved.call.input, { messages: [], toolCallId: "call-1" }),+ );+ expect(linear.calls).toEqual([{ input: { limit: 20, team: "core" }, tool: "list_issues" }]);+ });++ it("fails a call whose connection doesn't list its tools within 10 seconds, when validating and when calling", async () => {+ vi.useFakeTimers();+ try {+ const { catalog, linear, run } = linearCatalog();+ const listing = vi.spyOn(linear.client, "getToolMetadata");+ const timedOut = '"linear" did not list its tools within 10s. Try again later.';+ const call = { input: { team: "core" }, name: "linear__list_issues" };++ listing.mockReturnValueOnce(new Promise(() => {}));+ const validating = validateTool(catalog, call);+ await vi.advanceTimersByTimeAsync(10_000);+ expect(await validating).toEqual({+ issues: [expect.objectContaining({ message: timedOut })],+ });++ const { value } = (await validateTool(catalog, call)) as { value: unknown };+ const resolved = catalog.resolve({ input: value, toolName: CALL_TOOL_NAME });+ listing.mockReturnValueOnce(new Promise(() => {}));+ const calling = run(() =>+ resolved?.definition.execute?.(resolved.call.input, {+ messages: [],+ toolCallId: "call-1",+ }),+ );+ const rejected = expect(calling).rejects.toThrow(timedOut);+ await vi.advanceTimersByTimeAsync(10_000);+ await rejected;+ expect(linear.calls).toEqual([]);+ } finally {+ vi.useRealTimers();+ }+ });++ it("validates input against the connection's schema before the call can run", async () => {+ const { catalog, linear } = linearCatalog();++ expect(+ await validateTool(catalog, { input: { team: 7 }, name: "linear__list_issues" }),+ ).toEqual({+ issues: [+ {+ message: 'Instance type "number" is invalid. Expected "string".',+ path: ["input", "team"],+ },+ {+ message:+ "Signature: linear__list_issues(input: { team: string; limit?: number /* default: 20 */ }): Promise<unknown>",+ path: ["input"],+ },+ ],+ });+ expect(linear.calls).toEqual([]);+ });++ it.each([+ [+ "an unknown tool, with the connection's closest tools",+ undefined,+ 'Connection "linear" has no tool named "list_issue". Closest tools: linear__list_issues, linear__list_projects.',+ ],+ [+ "a listing failure that is not a sign-in",+ new Error("upstream returned 502"),+ 'Failed to load tools for "linear": upstream returned 502',+ ],+ ])("rejects %s", async (_case, listing, message) => {+ const { catalog } = linearCatalog(listing);+ const tool = listing === undefined ? "linear__list_issue" : "linear__list_issues";++ expect(await validateTool(catalog, { name: tool })).toEqual({+ issues: [{ message, path: ["input"] }],+ });+ });++ it("accepts a call to a connection that needs sign-in, so the call can start it", async () => {+ const { catalog } = linearCatalog("sign-in");++ expect(+ await validateTool(catalog, { input: { team: 7 }, name: "linear__list_issues" }),+ ).toEqual({ value: { input: { team: 7 }, name: "linear__list_issues" } });+ });+ });++ describe("connection sign-in", () => {+ function notionCatalog(+ options: Pick<+ Parameters<typeof fakeConnection>[0],+ "listing" | "rejectsToken" | "signIn"+ > = {},+ ) {+ const notion = fakeConnection({+ description: "Notion pages",+ listing: "sign-in",+ name: "notion",+ signIn: true,+ ...options,+ tools: [connectionTool("search_pages")],+ });+ const context = catalogContext({ connections: [notion] });+ const signIn = context.catalog.resolve({+ input: { name: "notion" },+ toolName: CALL_TOOL_NAME,+ })!;+ return {+ ...context,+ notion,+ signIn,+ connect: () =>+ context.run(() =>+ signIn.definition.execute!(signIn.call.input, { messages: [], toolCallId: "connect" }),+ ),+ /** Delivers the finished sign-in, as the turn step does when its callback arrives. */+ finishSignIn: () =>+ context.ctx.set(PendingAuthorizationResultKey, [+ {+ callback: { method: "GET", params: { code: "ok" } },+ hookUrl: "https://agent.example.com/callback",+ name: "notion",+ },+ ]),+ };+ }++ it("parks eve__tool({ name: <connection> }) for sign-in, then reports the sign-in and lists the tools", async () => {+ const { catalog, connect, finishSignIn, notion, run, signIn } = notionCatalog();++ expect(signIn.call.toolName).toBe("notion");+ expect(signIn.definition.label?.start?.({})).toBe("Sign in to Notion");+ expect(signIn.definition.approval).toBeUndefined();+ const parked = await connect();+ expect(isAuthorizationSignal(parked)).toBe(true);+ expect(parked).toMatchObject({ challenges: [{ name: "notion" }] });+ expect(notion.signIns).toHaveLength(1);++ finishSignIn();+ expect(await connect()).toBe(+ 'Signed in to Notion. Find the Notion tools with eve__search({ query: "notion__" }).',+ );+ expect(notion.signIns).toHaveLength(1);+ const search = catalog.advertised.get(SEARCH_TOOL_NAME)!;+ expect(+ await run(() =>+ search.execute!({ query: "notion__" }, { messages: [], toolCallId: "find" }),+ ),+ ).toMatchObject({ results: [{ tool: "notion__search_pages" }] });+ });++ it("confirms a connection whose tools are already listable, without prompting", async () => {+ const { connect, notion } = notionCatalog({ listing: "listed" });++ expect(await connect()).toBe(+ 'The Notion tools are available. Find them with eve__search({ query: "notion__" }).',+ );+ expect(notion.signIns).toEqual([]);+ });++ it("fails rather than asking again when the service rejects the token it just issued", async () => {+ const { connect, finishSignIn, notion } = notionCatalog({ rejectsToken: true });++ await connect();+ finishSignIn();++ await expect(connect()).rejects.toThrow(+ 'Authorization failed for "notion": the service rejected the token immediately after authorization.',+ );+ expect(notion.signIns).toHaveLength(1);+ });++ it.each([+ [+ "a connection that cannot start sign-in",+ { signIn: false },+ '"notion" requires authorization and cannot start interactive sign-in.',+ ],+ [+ "a listing failure",+ { listing: new Error("upstream returned 502") },+ 'Failed to load tools for "notion": upstream returned 502',+ ],+ ])("fails for %s", async (_case, options, message) => {+ captureLogRecords();+ await expect(notionCatalog(options).connect()).rejects.toThrow(message);+ });+ });+});diff --git a/packages/eve/src/execution/catalog/step-catalog.ts b/packages/eve/src/execution/catalog/step-catalog.tsnew file mode 100644index 000000000..ebb3205f8--- /dev/null+++ b/packages/eve/src/execution/catalog/step-catalog.ts@@ -0,0 +1,462 @@+/**+ * The step's catalog: one table of every entry a model step can reach. Direct+ * entries go into the model's tool list next to the catalog tools the agent+ * has: `eve__search` finds entries, `eve__tool` calls deferred entries and+ * connection tools, and `eve__skill` loads skills. A call resolves to its entry+ * here and then runs exactly as a direct call would, so only model history+ * ever names a catalog tool.+ */++import type { StandardSchemaV1 } from "#compiled/@standard-schema/spec/index.js";+import { connectionToolName } from "#connections/ownership.js";+import { buildDynamicTools } from "#context/build-dynamic-tools.js";+import { buildDynamicSubagentTools } from "#context/dynamic-subagent-lifecycle.js";+import type { ContextReader } from "#context/key.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import { startsTasks } from "#execution/tasks/tool-entry-point.js";+import { withTaskTools } from "#execution/tasks/model-step.js";+import { runEntryCall } from "#harness/execute-call.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import { checkToolCallInput } from "#harness/tool-call-io.js";+import type {+ CallResolver,+ HarnessSession,+ HarnessToolLookup,+ HarnessToolMap,+ ToolCallLike,+} from "#harness/types.js";+import {+ CALL_TOOL_NAME,+ SEARCH_TOOL_NAME,+ SKILL_ENTRY_NAME,+ SKILL_TOOL_NAME,+} from "#protocol/catalog-tools.js";+import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";+import { BundleKey, type CompiledBundle } from "#runtime/sessions/runtime-context-keys.js";+import type { ResolvedConnectionDefinition } from "#runtime/types.js";+import { isObject } from "#shared/guards.js";+import type { ToolExecuteOptions } from "#tools/definition.js";+import { refineJsonSchema } from "#tools/schema.js";++import { connectionEntry, connectionSignInEntry } from "./connection-entry.js";+import { closestNames } from "./rank.js";+import { createSearchTool } from "./search.js";+import { entrySignature } from "./signatures.js";+import {+ createSkillLoader,+ sessionSkills,+ unknownSkillMessage,+ type CatalogSkill,+} from "./skills.js";++/** Appended to the model-facing description of every tool with `endsTurn: true`. */+const ENDS_TURN_TOOL_NOTE =+ "Calling this tool ends your turn once it succeeds: do not write a reply or call other tools in the same step. If it fails, you will see the error and can continue.";++const CALL_TOOL_DESCRIPTION = `Call a tool that isn't in your tool list by its exact name from ${SEARCH_TOOL_NAME}, with \`input\` matching its signature.`;+const CONNECTIONS_CLAUSE =+ "Prefer connected services over web search or general knowledge when a request relates to them.";+const LISTED_SKILL_DESCRIPTION =+ "Load a skill's instructions when the request clearly matches one of your listed skills or the user asks for it, then follow them.";+const SEARCHABLE_SKILL_DESCRIPTION = `Load a skill's instructions when the request clearly matches a listed skill or one ${SEARCH_TOOL_NAME} found, or the user asks for it, then follow them.`;++/** Which catalog tools an agent gets, and what they reach. */+interface CatalogTools {+ /** A static connection or a dynamic connection resolver. */+ readonly connections: boolean;+ readonly search: boolean;+ /** A deferred skill or a dynamic skill resolver, which `eve__search` can find. */+ readonly searchableSkills: boolean;+ /** Any skill, static or from a dynamic skill resolver: `eve__skill` loads it. */+ readonly skills: boolean;+ /** Anything `eve__tool` can call: a deferred tool or agent, or a connection. */+ readonly tools: boolean;+}++/**+ * Each catalog tool comes with what it reaches: `eve__tool` with a deferred+ * tool or agent, a connection, or a resolver that may add one; `eve__skill`+ * with any skill; and `eve__search` with anything it could find, which is+ * whatever `eve__tool` calls plus deferred and dynamic skills. All follow from+ * what the agent declares, never from what a step's resolvers returned, so the+ * tool list is fixed for a deployment and never changes within a session. An+ * agent with none of these has no catalog tools, and pays nothing for them.+ */+function catalogToolsFor(+ agentTools: readonly HarnessToolDefinition[],+ bundle: CompiledBundle | undefined,+): CatalogTools {+ const agent = bundle?.resolvedAgent;+ const skills = agent?.skills ?? [];+ const dynamicSkills = (agent?.dynamicSkillResolvers.length ?? 0) > 0;+ const connections =+ (agent?.connections ?? []).length > 0 || (agent?.dynamicConnectionResolvers?.length ?? 0) > 0;+ const searchableSkills = dynamicSkills || skills.some((skill) => skill.deferred === true);+ const tools =+ connections ||+ agentTools.some((definition) => definition.deferred === true) ||+ (agent?.dynamicToolResolvers.length ?? 0) > 0 ||+ (bundle?.subagentRegistry.dynamicResolvers.length ?? 0) > 0;+ return {+ connections,+ search: tools || searchableSkills,+ searchableSkills,+ skills: skills.length > 0 || dynamicSkills,+ tools,+ };+}++export interface StepCatalog extends HarnessToolLookup {+ /** The model's tool list: direct entries, then the catalog tools the agent has. */+ readonly advertised: HarnessToolMap;+ readonly connections: readonly ResolvedConnectionDefinition[];+ /** Entries reached only through `eve__tool`, connection tools aside. */+ readonly deferred: HarnessToolMap;+ /** Every direct and deferred entry the session sees, connection tools aside. */+ readonly entries: HarnessToolMap;+ /** Every skill the session can load, deferred or not. */+ readonly skills: ReadonlyMap<string, CatalogSkill>;+ /**+ * Whether the session offers tasks: one of the agent's own tools it sees+ * starts them, or the agent resolves subagents at runtime. Both are fixed by+ * the deployment, so the task tools and system block change only on upgrade.+ */+ readonly offersTasks: boolean;+ /** An entry's description as the model reads it, with the notes eve appends. */+ describe(definition: HarnessToolDefinition): string;+ /**+ * Resolves a model tool call to the entry it runs. An `eve__tool` call that+ * names a deferred entry becomes the call to that entry, and an `eve__skill`+ * call becomes the call to load its skill; any other call runs the listed+ * tool it names. A call that reaches none of these resolves to nothing.+ */+ readonly resolve: CallResolver;+}++/**+ * Builds the catalog for one step from the agent's tools, the dynamic tools and+ * subagents resolved for it, and its skills, keeping only what the session sees.+ */+export function buildStepCatalog(input: {+ readonly agentTools: HarnessToolMap;+ readonly ctx: ContextReader | undefined;+ /** Whether `endsTurn` applies: a root turn without an output schema. */+ readonly endsTurn: boolean;+ readonly session: Pick<HarnessSession, "rootSessionId">;+}): StepCatalog {+ const visible = (definition: HarnessToolDefinition) => isVisible(definition, input.session);+ const bundle = input.ctx?.get(BundleKey);+ // Only the tools this session sees count, so a delegated session of the same+ // agent doesn't search for root-only entries it can't reach.+ const catalogTools = catalogToolsFor([...input.agentTools.values()].filter(visible), bundle);+ const offersTasks =+ (bundle?.subagentRegistry.dynamicResolvers.length ?? 0) > 0 ||+ [...input.agentTools.values()].some(+ (definition) => visible(definition) && startsTasks(definition),+ );+ const ownEntries = stepEntries(input, visible);+ const entries = offersTasks ? withTaskTools(ownEntries) : ownEntries;+ const direct = new Map<string, HarnessToolDefinition>();+ const deferred = new Map<string, HarnessToolDefinition>();+ for (const [name, definition] of entries) {+ (definition.deferred === true ? deferred : direct).set(name, definition);+ }+ // Read once here: validation runs while the SDK parses the stream, where the+ // context store may not be active.+ const registry = input.ctx?.get(ConnectionRegistryKey);+ const connections = registry?.getConnections() ?? [];+ const skills = sessionSkills(input.ctx);+ const skillLoader = createSkillLoader(skills);+ const advertised = new Map(direct);+ // Listed tools include the catalog tools, so `eve__tool` naming one of them+ // says to call it directly rather than that it does not exist.+ const toolEntry = (name: string): HarnessToolDefinition | undefined =>+ entries.get(name) ??+ advertised.get(name) ??+ (registry === undefined ? undefined : connectionEntryNamed(name, registry, connections));+ const describe = (definition: HarnessToolDefinition) => describeEntry(definition, input.endsTurn);+ const catalog: StepCatalog = {+ advertised,+ connections,+ deferred,+ describe,+ entries,+ get: (name) => (name === SKILL_ENTRY_NAME ? skillLoader : toolEntry(name)),+ offersTasks,+ resolve: (toolCall) => {+ if (toolCall.toolName === CALL_TOOL_NAME && catalogTools.tools) {+ const name = targetName(toolCall.input);+ const definition = name === undefined ? undefined : toolEntry(name);+ if (name === undefined || definition?.deferred !== true) return undefined;+ return { call: asEntryCall(toolCall, name), definition };+ }+ if (toolCall.toolName === SKILL_TOOL_NAME && catalogTools.skills) {+ const name = targetName(toolCall.input);+ if (name === undefined || !skills.has(name)) return undefined;+ const call = { ...toolCall, input: { skill: name }, toolName: SKILL_ENTRY_NAME };+ return { call, definition: skillLoader };+ }+ const definition = advertised.get(toolCall.toolName);+ return definition === undefined ? undefined : { call: toolCall, definition };+ },+ skills,+ };+ if (catalogTools.search) {+ const search = createSearchTool({+ deferred: [...deferred.values()],+ describe,+ reach: {+ connections: catalogTools.connections,+ skills: catalogTools.searchableSkills,+ tools: catalogTools.tools,+ },+ registry,+ skills: [...skills.values()].filter((skill) => skill.deferred),+ });+ advertised.set(search.name, search);+ }+ if (catalogTools.tools) {+ advertised.set(CALL_TOOL_NAME, createCallTool(catalog, toolEntry, catalogTools));+ }+ if (catalogTools.skills) {+ advertised.set(SKILL_TOOL_NAME, createSkillTool(catalog, toolEntry, catalogTools));+ }+ return catalog;+}++/** The entry an `eve__tool` or `eve__skill` input names, if it names one. */+function targetName(input: unknown): string | undefined {+ return isObject(input) && typeof input.name === "string" ? input.name : undefined;+}++/** `eve__tool({ name, input })` as the call to `name` with `input`. */+function asEntryCall<T extends ToolCallLike>(toolCall: T, toolName: string): T {+ const input = (toolCall.input as { readonly input?: unknown }).input;+ return { ...toolCall, input: input ?? {}, toolName };+}++function describeEntry(definition: HarnessToolDefinition, endsTurn: boolean): string {+ return endsTurn && definition.endsTurn === true+ ? `${definition.description}\n\n${ENDS_TURN_TOOL_NOTE}`.trimStart()+ : definition.description;+}++function stepEntries(+ input: Parameters<typeof buildStepCatalog>[0],+ visible: (definition: HarnessToolDefinition) => boolean,+): Map<string, HarnessToolDefinition> {+ const entries = new Map([...input.agentTools].filter(([, definition]) => visible(definition)));+ if (input.ctx === undefined) return entries;++ for (const subagent of buildDynamicSubagentTools(input.ctx).filter(visible)) {+ entries.set(subagent.name, subagent);+ }+ // Dynamic tools override a same-named authored tool. The first definition of+ // a name wins: step, then turn, then session.+ const dynamicNames = new Set<string>();+ for (const tool of buildDynamicTools(input.ctx).filter(visible)) {+ if (dynamicNames.has(tool.name)) continue;+ dynamicNames.add(tool.name);+ entries.set(tool.name, tool);+ }+ return entries;+}++/** Root-only tools and tools unavailable in subagents are hidden from delegated sessions. */+function isVisible(+ definition: HarnessToolDefinition,+ session: Pick<HarnessSession, "rootSessionId">,+): boolean {+ const rootOnly =+ definition.rootOnly === true ||+ definition.availableInSubagents === false ||+ definition.behavior?.availability.includes("root-session") === true;+ return !rootOnly || session.rootSessionId === undefined;+}++/** A connection owns its own name, which signs the user in, and every name under its prefix. */+function connectionEntryNamed(+ name: string,+ registry: ConnectionRegistry,+ connections: readonly ResolvedConnectionDefinition[],+): HarnessToolDefinition | undefined {+ for (const connection of connections) {+ if (name === connection.connectionName) return connectionSignInEntry(registry, connection);+ const prefix = connectionToolName(connection.connectionName, "");+ if (name.length > prefix.length && name.startsWith(prefix)) {+ return connectionEntry(registry, connection, name.slice(prefix.length));+ }+ }+ return undefined;+}++// ---------------------------------------------------------------------------+// eve__tool and eve__skill+// ---------------------------------------------------------------------------++/**+ * The model sees one fixed schema. Validation resolves the named entry and+ * checks `input` against that entry's own schema, so a call that reaches+ * `eve__tool` always names a deferred entry with valid input, as a direct call+ * names a listed tool.+ */+function createCallTool(+ catalog: StepCatalog,+ toolEntry: (name: string) => HarnessToolDefinition | undefined,+ tools: CatalogTools,+): HarnessToolDefinition {+ return {+ description: tools.connections+ ? `${CALL_TOOL_DESCRIPTION} ${CONNECTIONS_CLAUSE}`+ : CALL_TOOL_DESCRIPTION,+ execute: (input: unknown, options: ToolExecuteOptions) =>+ runResolved(catalog, { input, toolName: CALL_TOOL_NAME }, options),+ frameworkTool: true,+ inputSchema: refineJsonSchema(+ {+ type: "object",+ properties: {+ name: {+ type: "string",+ description: `The tool's exact name, as ${SEARCH_TOOL_NAME} returns it.`,+ },+ input: {+ type: "object",+ description: "Arguments matching the tool's signature. Defaults to {}.",+ },+ },+ required: ["name"],+ additionalProperties: false,+ },+ (value) => resolveToolInput(catalog, toolEntry, value as ToolInput),+ ),+ name: CALL_TOOL_NAME,+ };+}++/** Loads a skill by name; validation resolves the name, so a call always loads one. */+function createSkillTool(+ catalog: StepCatalog,+ toolEntry: (name: string) => HarnessToolDefinition | undefined,+ tools: CatalogTools,+): HarnessToolDefinition {+ return {+ description: tools.searchableSkills ? SEARCHABLE_SKILL_DESCRIPTION : LISTED_SKILL_DESCRIPTION,+ execute: (input: unknown, options: ToolExecuteOptions) =>+ runResolved(catalog, { input, toolName: SKILL_TOOL_NAME }, options),+ frameworkTool: true,+ inputSchema: refineJsonSchema(+ {+ type: "object",+ properties: {+ name: {+ type: "string",+ description: tools.searchableSkills+ ? `The skill's name, from your listed skills or ${SEARCH_TOOL_NAME}.`+ : "The skill's name, from your listed skills.",+ },+ },+ required: ["name"],+ additionalProperties: false,+ },+ async (value) =>+ resolveSkillInput(catalog, toolEntry, (value as { name: string }).name, tools),+ ),+ name: SKILL_TOOL_NAME,+ };+}++function runResolved(+ catalog: StepCatalog,+ toolCall: ToolCallLike,+ options: ToolExecuteOptions,+): unknown {+ const resolved = catalog.resolve(toolCall);+ // Input validation resolves every call before the SDK runs it.+ if (resolved === undefined) {+ throw new Error(`A ${toolCall.toolName} call ran without a resolved entry.`);+ }+ return runEntryCall(resolved, options);+}++interface ToolInput {+ readonly input?: unknown;+ readonly name: string;+}++async function resolveToolInput(+ catalog: StepCatalog,+ toolEntry: (name: string) => HarnessToolDefinition | undefined,+ { input, name }: ToolInput,+): Promise<StandardSchemaV1.Result<ToolInput>> {+ const definition = toolEntry(name);+ if (definition === undefined) return failure("name", unknownEntryMessage(name, catalog));+ if (definition.deferred !== true) {+ return failure("name", `"${name}" is in your tool list; call it directly.`);+ }+ const checked = await checkToolCallInput(definition, input, "");+ if (checked.kind === "threw") throw checked.error;+ if (checked.kind === "invalid") {+ // A connection tool's validation reports its own signature; every other entry gets one here.+ const issues = isConnectionTool(name, catalog.connections)+ ? checked.issues+ : [...checked.issues, { message: `Signature: ${entrySignature(definition)}` }];+ // The entry's issues rather than its message, so the SDK's report is the only wrapper.+ return {+ issues: issues.map(({ message, path = [] }) => ({ message, path: ["input", ...path] })),+ };+ }+ return { value: { input: checked.value, name } };+}++function resolveSkillInput(+ catalog: StepCatalog,+ toolEntry: (name: string) => HarnessToolDefinition | undefined,+ name: string,+ tools: CatalogTools,+): StandardSchemaV1.Result<{ readonly name: string }> {+ if (catalog.skills.has(name)) return { value: { name } };+ const connections = catalog.connections.map((connection) => connection.connectionName);+ const tool = connections.includes(name) ? undefined : toolEntry(name);+ if (tool !== undefined) {+ return failure(+ "name",+ tool.deferred === true+ ? `"${name}" is a tool, not a skill; call it with ${CALL_TOOL_NAME}({ name: "${name}" }).`+ : `"${name}" is a tool in your tool list, not a skill; call it directly.`,+ );+ }+ return failure(+ "name",+ unknownSkillMessage(name, catalog.skills, connections, tools.searchableSkills),+ );+}++function isConnectionTool(+ name: string,+ connections: readonly ResolvedConnectionDefinition[],+): boolean {+ return connections.some(({ connectionName }) =>+ name.startsWith(connectionToolName(connectionName, "")),+ );+}++function failure(path: string, message: string): StandardSchemaV1.FailureResult {+ return { issues: [{ message, path: [path] }] };+}++function unknownEntryMessage(name: string, catalog: StepCatalog): string {+ if (catalog.skills.has(name)) {+ return `"${name}" is a skill; load it with ${SKILL_TOOL_NAME}({ name: "${name}" }).`;+ }+ const candidates = [...catalog.deferred.values()].map(({ description, name }) => ({+ description,+ name,+ }));+ const suggestions = closestNames(name, candidates);+ const hint =+ suggestions.length > 0+ ? ` Closest tools: ${suggestions.join(", ")}.`+ : ` Find tools with ${SEARCH_TOOL_NAME}.`;+ return `No tool named "${name}".${hint}`;+}diff --git a/packages/eve/src/execution/connection-announcement.test.ts b/packages/eve/src/execution/connection-announcement.test.tsdeleted file mode 100644index 5fd1c58a1..000000000--- a/packages/eve/src/execution/connection-announcement.test.ts+++ /dev/null@@ -1,76 +0,0 @@-import { describe, expect, it } from "vitest";-import { ContextContainer } from "#context/container.js";-import type { HistoryState } from "#context/keys.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { announceConnections } from "#execution/connection-announcement.js";-import { getPendingAnnouncements } from "#harness/announcements.js";-import { createCurrentMessages } from "#harness/current-messages.js";-import type { HarnessModelMessage } from "#harness/messages.js";-import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";-import type { ResolvedConnectionDefinition } from "#runtime/types.js";--function registryOf(connections: Record<string, string>): ConnectionRegistry {- const resolved = Object.entries(connections).map(- ([connectionName, description]) =>- ({ connectionName, description }) as ResolvedConnectionDefinition,- );- return {- dispose: async () => {},- getClient: () => {- throw new Error("unused");- },- getConnectionApproval: () => undefined,- getConnectionNames: () => resolved.map((connection) => connection.connectionName),- getConnections: () => resolved,- };-}--/** Runs one step's announcement pass and returns the messages it appended. */-function step(- connections: Record<string, string>,- state: { history: HarnessModelMessage[]; historyState?: HistoryState },-): string[] {- const ctx = new ContextContainer();- ctx.set(ConnectionRegistryKey, registryOf(connections));- announceConnections(ctx);- const current = createCurrentMessages(state.history, { historyState: state.historyState });- current.addAnnouncements({ keyed: getPendingAnnouncements(ctx) });- const appended = current.history.slice(state.history.length);- state.history = [...current.history];- state.historyState = current.historyState;- return appended.map((message) => String(message.content));-}--describe("connection announcements", () => {- it("appends a baseline, then only changes, and rebaselines after compaction", () => {- const state: { history: HarnessModelMessage[]; historyState?: HistoryState } = {- history: [],- };-- expect(step({}, state)).toEqual([]);- expect(step({ linear: "Linear issues" }, state)).toEqual([- [- "Connections. Find their tools with connection_search and call them with connection_execute.",- "- linear: Linear issues",- ].join("\n"),- ]);- expect(step({ linear: "Linear issues" }, state)).toEqual([]);- expect(step({ github: "GitHub repositories" }, state)).toEqual([- [- "Connections changed.",- "Added or updated:",- "- github: GitHub repositories",- "No longer available, do not call: linear",- ].join("\n"),- ]);-- // Compaction replaces history and clears the announcement record.- const compacted = { history: [], historyState: undefined };- expect(step({ github: "GitHub repositories" }, compacted)).toEqual([- [- "Connections. Find their tools with connection_search and call them with connection_execute.",- "- github: GitHub repositories",- ].join("\n"),- ]);- });-});diff --git a/packages/eve/src/execution/connection-announcement.ts b/packages/eve/src/execution/connection-announcement.tsdeleted file mode 100644index fdf1a6935..000000000--- a/packages/eve/src/execution/connection-announcement.ts+++ /dev/null@@ -1,88 +0,0 @@-import type { ContextContainer } from "#context/container.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { setPendingAnnouncement } from "#harness/announcements.js";--const ANNOUNCEMENT_KEY = "connections";--interface ListedConnection {- readonly description: string;- readonly name: string;-}--/**- * Announces the agent's connections for the current step: a baseline listing- * the first time, then only what changed. Tool names and signatures stay out- * of the listing; `connection_search` returns them.- */-export function announceConnections(ctx: ContextContainer): void {- const registry = ctx.get(ConnectionRegistryKey);- if (registry === undefined) {- setPendingAnnouncement(ctx, ANNOUNCEMENT_KEY, undefined);- return;- }- const current = registry- .getConnections()- .map((connection) => ({- description: connection.description,- name: connection.connectionName,- }))- .sort((a, b) => a.name.localeCompare(b.name));- setPendingAnnouncement(ctx, ANNOUNCEMENT_KEY, {- value: JSON.stringify(current),- render: (previous) => renderConnectionsAnnouncement(parseListing(previous), current),- });-}--function renderConnectionsAnnouncement(- previous: readonly ListedConnection[] | undefined,- current: readonly ListedConnection[],-): string | undefined {- // An empty listing was recorded without a message, so the model has not- // seen the baseline yet.- if (previous === undefined || previous.length === 0) {- return current.length === 0 ? undefined : renderListing(current);- }- if (current.length === 0) {- return "Connections changed. No connections are available now; do not call connection_execute.";- }-- const previousByName = new Map(previous.map((entry) => [entry.name, entry]));- const currentNames = new Set(current.map((entry) => entry.name));- const added = current.filter(- (entry) => previousByName.get(entry.name)?.description !== entry.description,- );- const removed = previous.filter((entry) => !currentNames.has(entry.name));- const parts = ["Connections changed."];- if (added.length > 0) parts.push(["Added or updated:", ...added.map(formatEntry)].join("\n"));- if (removed.length > 0) {- parts.push(- `No longer available, do not call: ${removed.map((entry) => entry.name).join(", ")}`,- );- }- const delta = parts.join("\n");- const replacement = `Connections changed. This list replaces the previous one.\n${renderListing(current)}`;- return delta.length < replacement.length ? delta : replacement;-}--function renderListing(connections: readonly ListedConnection[]): string {- return [- "Connections. Find their tools with connection_search and call them with connection_execute.",- ...connections.map(formatEntry),- ].join("\n");-}--function formatEntry(entry: ListedConnection): string {- return entry.description.length === 0- ? `- ${entry.name}`- : `- ${entry.name}: ${entry.description}`;-}--function parseListing(value: string | undefined): readonly ListedConnection[] | undefined {- if (value === undefined) return undefined;- try {- const parsed = JSON.parse(value) as unknown;- return Array.isArray(parsed) ? (parsed as ListedConnection[]) : undefined;- } catch {- return undefined;- }-}diff --git a/packages/eve/src/execution/dynamic-connections.ts b/packages/eve/src/execution/dynamic-connections.tsindex dec2db2b5..8e3338c0c 100644--- a/packages/eve/src/execution/dynamic-connections.ts+++ b/packages/eve/src/execution/dynamic-connections.ts@@ -1,6 +1,5 @@import type { ContextContainer } from "#context/container.js";import { dispatchDynamicConnectionEvent } from "#context/dynamic-connection-lifecycle.js";-import { announceConnections } from "#execution/connection-announcement.js";import type { TurnPosition } from "#harness/session-machine/view.js";import {sessionStartedForResolvers,@@ -17,7 +16,6 @@ export function bindDynamicConnections(const resolvers = agent.dynamicConnectionResolvers ?? [];const dispatch = async (event: UnstampedMessageStreamEvent): Promise<void> => {await dispatchDynamicConnectionEvent({ ctx, event, resolvers });- if (event.type === "step.started") announceConnections(ctx);};return {diff --git a/packages/eve/src/execution/invoke-tool.test.ts b/packages/eve/src/execution/invoke-tool.test.tsindex 0b62da796..8f774f445 100644--- a/packages/eve/src/execution/invoke-tool.test.ts+++ b/packages/eve/src/execution/invoke-tool.test.ts@@ -236,14 +236,14 @@ describe("invokeTool", () => {it("refuses unknown, framework, and badly typed calls before running anything", async () => {const execute = vi.fn();- const runtime = runtimeWith([tool("note", execute), tool("load_skill", execute)], undefined, {- load_skill: "framework",+ const runtime = runtimeWith([tool("note", execute), tool("web_fetch", execute)], undefined, {+ web_fetch: "framework",});expect(await invokeTool(runtime, "missing", {}, { auth: alice })).toMatchObject({message: 'The agent has no tool named "missing".',status: "failed",});- expect(await invokeTool(runtime, "load_skill", {}, { auth: alice })).toMatchObject({+ expect(await invokeTool(runtime, "web_fetch", {}, { auth: alice })).toMatchObject({message: expect.stringContaining("framework tool"),status: "failed",});diff --git a/packages/eve/src/execution/node-step.test.ts b/packages/eve/src/execution/node-step.test.tsindex 295ef9795..854e3b02c 100644--- a/packages/eve/src/execution/node-step.test.ts+++ b/packages/eve/src/execution/node-step.test.ts@@ -377,6 +377,9 @@ describe("createExecutionNodeStep", () => {ctx.set(InitiatorAuthKey, null);ctx.set(BundleKey, {compiledArtifactsSource: createBundledRuntimeCompiledArtifactsSource(),+ resolvedAgent: { dynamicSkillResolvers: [], dynamicToolResolvers: [], skills: [] },+ subagentRegistry: rootNode.subagentRegistry,+ toolRegistry: rootNode.toolRegistry,} as never);ctx.set(ChannelKey, { kind: "http" });ctx.set(SessionIdKey, "sess-root");@@ -461,6 +464,9 @@ describe("createExecutionNodeStep", () => {ctx.set(InitiatorAuthKey, null);ctx.set(BundleKey, {compiledArtifactsSource: createBundledRuntimeCompiledArtifactsSource(),+ resolvedAgent: { dynamicSkillResolvers: [], dynamicToolResolvers: [], skills: [] },+ subagentRegistry: node.subagentRegistry,+ toolRegistry: node.toolRegistry,} as never);ctx.set(ChannelKey, { kind: "http" });ctx.set(SessionIdKey, "sess-dynamic");diff --git a/packages/eve/src/execution/node-step.ts b/packages/eve/src/execution/node-step.tsindex 0bad10795..e89c73a84 100644--- a/packages/eve/src/execution/node-step.ts+++ b/packages/eve/src/execution/node-step.ts@@ -266,6 +266,7 @@ function createRegisteredHarnessToolDefinition(input: {: undefined),approvalKey: def.approvalKey,behavior: input.behavior,+ deferred: def.deferred,description: def.description,endsTurn: def.endsTurn,executeInput: def.executeInput,@@ -274,7 +275,6 @@ function createRegisteredHarnessToolDefinition(input: {scope: def.name,}),frameworkTool: isFrameworkTool(def) || def.owner.kind === "framework",- frameworkAction: def.behavior?.presentation === "load-skill" ? "load-skill" : undefined,inputSchema: def.inputSchema ?? UNSPECIFIED_INPUT_SCHEMA,name: def.name,approval: def.approval,diff --git a/packages/eve/src/execution/remote-agent-input.scenario.test.ts b/packages/eve/src/execution/remote-agent-input.scenario.test.tsindex 9fe81470c..e59d31beb 100644--- a/packages/eve/src/execution/remote-agent-input.scenario.test.ts+++ b/packages/eve/src/execution/remote-agent-input.scenario.test.ts@@ -6,6 +6,7 @@ import type { ClientSession } from "#client/session.js";import { filterEventsByType } from "#internal/testing/events.js";import { type ScenarioAppDescriptor, useScenarioApp } from "#internal/testing/scenario-app.js";import type { HandleMessageStreamEvent } from "#protocol/message.js";+import { TASK_WAIT_TOOL_NAME } from "#protocol/task-tools.js";const scenarioApp = useScenarioApp();const TOKEN = "remote-input-scenario-token";@@ -79,7 +80,7 @@ function parentDescriptor(url: string, concurrent = false): ScenarioAppDescripto? "REMOTE_HITL_RESULT=alice-lantern,bob-comet": PARENT_RESULT;const parentAgent = `import { defineAgent } from "eve"; import { mockModel } from "eve/evals";-const model = mockModel((request) => { const text = request.messages.map((m) => m.text).join("\\n"); if (text.includes("<task_result") && text.includes("${expectedChildResult}")) return "${expectedParentResult}"; if (request.toolResults.some((r) => r.id === "delegate")) return { toolCalls: [{ id: "wait", name: "task_wait", input: {} }] }; return { toolCalls: [{ id: "delegate", name: "delegate-approval", input: {} }] }; });+const model = mockModel((request) => { const text = request.messages.map((m) => m.text).join("\\n"); if (text.includes("<task_result") && text.includes("${expectedChildResult}")) return "${expectedParentResult}"; if (request.toolResults.some((r) => r.id === "delegate")) return { toolCalls: [{ id: "wait", name: "${TASK_WAIT_TOOL_NAME}", input: {} }] }; return { toolCalls: [{ id: "delegate", name: "delegate-approval", input: {} }] }; });export default defineAgent({ model, modelContextWindowTokens: 32000 });`;const tool = `import { defineWorkflowTool } from "eve/tools";export default defineWorkflowTool({ description: "Delegate approval.", inputSchema: {}, async task(_input, ctx) { "use workflow"; const result = await ctx.agent("remote-hitl-child").send("Ask the parent for approval."); return (await result.result()).message; } });`;@@ -103,7 +104,7 @@ const model = mockModel((request) => {const text = request.messages.map((message) => message.text).join("\\n");if (text.includes("MEMORY_RECALLED=LANTERN-COMET-7319")) return "PARENT_RECALLED=LANTERN-COMET-7319";if (text.includes("MEMORY_RECALLED=CONTEXT_LOST")) return "PARENT_RECALLED=CONTEXT_LOST";- if (request.toolResults.some((result) => result.id === "remember")) return { toolCalls: [{ id: "wait-memory", name: "task_wait", input: {} }] };+ if (request.toolResults.some((result) => result.id === "remember")) return { toolCalls: [{ id: "wait-memory", name: "${TASK_WAIT_TOOL_NAME}", input: {} }] };return { toolCalls: [{ id: "remember", name: "remember", input: {} }] };});export default defineAgent({ model, modelContextWindowTokens: 32000 });`;diff --git a/packages/eve/src/execution/session.test.ts b/packages/eve/src/execution/session.test.tsindex 019e3bbbb..43285bd27 100644--- a/packages/eve/src/execution/session.test.ts+++ b/packages/eve/src/execution/session.test.ts@@ -107,7 +107,7 @@ describe("createSession", () => {continuationToken: "root-token",sessionId: "sess-root",turnAgent: createTestTurnAgent({- availableSkills: [{ description: "Research topics.", name: "research" }],+ listedSkills: [{ description: "Research topics.", name: "research" }],}),});diff --git a/packages/eve/src/execution/session.ts b/packages/eve/src/execution/session.tsindex fe50f2029..5e3be4894 100644--- a/packages/eve/src/execution/session.ts+++ b/packages/eve/src/execution/session.ts@@ -155,7 +155,7 @@ export function refreshSessionFromTurnAgent(input: {}function createSessionSystemPrompt(turnAgent: RuntimeTurnAgent): string {- const skillSection = formatAvailableSkillsSection(turnAgent.availableSkills ?? []);+ const skillSection = formatAvailableSkillsSection(turnAgent.listedSkills ?? []);const blocks =skillSection === null ? turnAgent.instructions : [...turnAgent.instructions, skillSection];return blocks.join("\n\n");diff --git a/packages/eve/src/execution/session/active-turn.ts b/packages/eve/src/execution/session/active-turn.tsindex 129ce8d31..ed5602b50 100644--- a/packages/eve/src/execution/session/active-turn.ts+++ b/packages/eve/src/execution/session/active-turn.ts@@ -25,7 +25,7 @@ export type RuntimeEvent =| { readonly kind: "workflow"; readonly message: WorkflowToolRunMessage }/** A steering message that answered no pending request arrived during the wait. */| { readonly kind: "steering" }- /** A `task_wait` call's timeout passed. */+ /** An `eve__task_wait` call's timeout passed. */| { readonly kind: "timeout"; readonly callId: string }/** A delivery or sign-in callback was admitted; a held request may be answered. */| { readonly kind: "input" }diff --git a/packages/eve/src/execution/session/turn-connection-approval.integration.test.ts b/packages/eve/src/execution/session/turn-connection-approval.integration.test.tsindex f79246a70..eb6b0e8c7 100644--- a/packages/eve/src/execution/session/turn-connection-approval.integration.test.ts+++ b/packages/eve/src/execution/session/turn-connection-approval.integration.test.ts@@ -1,6 +1,7 @@import { simulateReadableStream } from "ai";import { MockLanguageModelV4 } from "ai/test";import { afterEach, describe, expect, it, vi } from "vitest";+import { CALL_TOOL_NAME, SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";import { ContextContainer } from "#context/container.js";import { AuthKey, InitiatorAuthKey, SessionIdKey } from "#context/keys.js";import { serializeContext } from "#context/serialize.js";@@ -29,11 +30,7 @@ import {} from "#runtime/sessions/runtime-context-keys.js";import { createRuntimeHookRegistry } from "#runtime/hooks/registry.js";import { resolveRuntimeModelReference } from "#runtime/agent/resolve-model.js";-import type {- ResolvedDynamicConnectionResolver,- ResolvedDynamicToolResolver,-} from "#runtime/types.js";-import connectionTools from "#tools/framework/connection-tools.js";+import type { ResolvedDynamicConnectionResolver } from "#runtime/types.js";import { clearDurableDynamicCallbacks } from "#tools/durable-callbacks.js";import type { ApprovalResponseContext } from "#approval/definition.js";@@ -61,8 +58,10 @@ const usage = {outputTokens: { reasoning: undefined, text: 1, total: 1 },};const sessionId = "turn-connection-approval";+const TOOL_GONE_RESULT =+ 'The tool "notes__saveNote" is no longer available, so the call didn\'t run. If the task still needs it, find an available tool with eve__search and make a new call.';-/** A model step that saves notes through `connection_execute`, or replies when `callId` is omitted. */+/** A model step that saves notes through `execute`, or replies when `callId` is omitted. */function modelResponse(callId?: string | readonly string[], connection = "notes") {const callIds = callId === undefined ? [] : typeof callId === "string" ? [callId] : callId;return {@@ -73,10 +72,9 @@ function modelResponse(callId?: string | readonly string[], connection = "notes"? callIds.map((toolCallId) => ({type: "tool-call" as const,toolCallId,- toolName: "connection_execute",+ toolName: CALL_TOOL_NAME,input: JSON.stringify({- connection,- tool: "saveNote",+ name: `${connection}__saveNote`,input: { body: { note: "hello" } },}),}))@@ -111,9 +109,10 @@ function setup(: { status: "allowed" as const };});const policyTurns: string[] = [];- let removed = false;+ let connection: "available" | "failing" | "removed" = "available";const resolver = vi.fn((event: unknown) => {- if (removed) return {};+ if (connection === "failing") throw new Error("The notes directory is unreachable.");+ if (connection === "removed") return {};const sequence = (event as { data: { sequence?: number } }).data.sequence ?? 0;return {[(variation === "name" || variation === "unapproved-name") && sequence > 0@@ -179,16 +178,6 @@ function setup(sourceKind: "module",},];- const dynamicToolResolvers: ResolvedDynamicToolResolver[] = [- {- eventNames: ["step.started"],- events: connectionTools.events as ResolvedDynamicToolResolver["events"],- logicalPath: "tools/connection_tools.ts",- slug: "connection_tools",- sourceId: "eve:connection-tools",- sourceKind: "module",- },- ];const adapter = { kind: "test" };const turnAgent = {id: "notes-agent",@@ -201,7 +190,8 @@ function setup(const resolvedAgent: Partial<CompiledBundle["resolvedAgent"]> = {connections: [],dynamicConnectionResolvers,- dynamicToolResolvers,+ dynamicSkillResolvers: [],+ dynamicToolResolvers: [],};const sandboxRegistry: {sandbox: CompiledBundle["graph"]["root"]["sandboxRegistry"]["sandbox"] | null;@@ -302,8 +292,13 @@ function setup(response,step,policyTurns,- removeConnection() {- removed = true;+ /** The session state as last committed. */+ state() {+ return readDurableSession(snapshot.sessionState).state;+ },+ /** What the connection's resolver does from now on. */+ setConnection(state: typeof connection) {+ connection = state;},updateSession(update: (session: HarnessSession) => HarnessSession) {snapshot = {@@ -351,10 +346,7 @@ describe("turn connection approval restoration", () => {);expect(requests.length).toBeGreaterThanOrEqual(2);const [first, ...later] = requests;- expect(first!.tools.map((tool) => tool.name).sort()).toEqual([- "connection_execute",- "connection_search",- ]);+ expect(first!.tools.map((tool) => tool.name)).toEqual([SEARCH_TOOL_NAME, CALL_TOOL_NAME]);const system = (request: (typeof requests)[number]) =>request.prompt.filter((message) => message.role === "system");for (const request of later) {@@ -397,7 +389,6 @@ describe("turn connection approval restoration", () => {],},});- await fixture.step();// A missing pin can't prove the connection is unchanged, so that call fails// rather than re-pinning to whatever the connection resolves to now.const failed = fixture.events.flatMap((event) =>@@ -412,64 +403,130 @@ describe("turn connection approval restoration", () => {expect(fixture.fetch).toHaveBeenCalledTimes(50);});- it("fails an approved call whose connection is gone without a response policy", async () => {+ it("fails an approved call whose connection is gone, and the session continues", async () => {const fixture = setup("turn.started", false, "request-only");await fixture.step({delivery: { kind: "deliver", payloads: [{ message: "Prepare Alice's note." }] },});const parked = await fixture.step();const request = suspendedSteps(readDurableSession(parked.sessionState).state)[0]!.requests[0]!;- fixture.removeConnection();- await fixture.step({+ fixture.setConnection("removed");+ const resumed = await fixture.step({delivery: {kind: "deliver",auth: alice,payloads: [{ inputResponses: [{ requestId: request.requestId, optionId: "approve" }] }],},});- await fixture.step();+expect(fixture.fetch).not.toHaveBeenCalled();- expect(fixture.events).toContainEqual(- expect.objectContaining({- type: "action.result",- data: expect.objectContaining({- status: "failed",- result: expect.objectContaining({- callId: request.action.callId,- output: expect.stringContaining('Connection "notes" is not available'),- }),- }),- }),+ const failed = fixture.events.findIndex(+ (event) =>+ event.type === "action.result" &&+ event.data.status === "failed" &&+ (event.data.result as { callId: string }).callId === request.action.callId,+ );+ expect(fixture.events[failed]?.data.result).toMatchObject({ output: TOOL_GONE_RESULT });+ // The model reads the failure as the call's result, and the turn completes.+ expect(JSON.stringify(fixture.doStream.mock.calls.at(-1)![0])).toContain(+ "is no longer available, so the call didn't run",);+ expect(fixture.events.slice(failed).map((event) => event.type)).toContain("turn.completed");+ expect(suspendedSteps(readDurableSession(resumed.sessionState).state)).toEqual([]);});- it("leaves the request pending when its connection can no longer be reconstructed", async () => {- const fixture = setup();- await fixture.step({- delivery: { kind: "deliver", payloads: [{ message: "Prepare Alice's note." }] },+ describe("a connection with a response policy, while its request waits", () => {+ /** Parks Alice's note for approval and returns its request. */+ async function parkNote(fixture: ReturnType<typeof setup>) {+ await fixture.step({+ delivery: { kind: "deliver", payloads: [{ message: "Prepare Alice's note." }] },+ });+ const parked = await fixture.step();+ return suspendedSteps(readDurableSession(parked.sessionState).state)[0]!.requests[0]!;+ }++ /** Delivers Bob's approval, then runs until the session waits again. */+ async function approveAsBob(fixture: ReturnType<typeof setup>, requestId: string) {+ let result = await fixture.step({+ delivery: {+ kind: "deliver",+ auth: bob,+ payloads: [{ inputResponses: [{ requestId, optionId: "approve" }] }],+ },+ });+ while (result.action === "continue") result = await fixture.step();+ return readDurableSession(result.sessionState).state;+ }++ it("settles the request as unavailable when the connection is gone", async () => {+ const fixture = setup();+ const request = await parkNote(fixture);+ fixture.setConnection("removed");++ const state = await approveAsBob(fixture, request.requestId);++ expect(fixture.response).not.toHaveBeenCalled();+ expect(fixture.fetch).not.toHaveBeenCalled();+ expect(fixture.events).toContainEqual(+ expect.objectContaining({+ type: "approval.candidate",+ data: expect.objectContaining({+ outcome: "failed",+ reason: "The tool this approval was for is no longer available, so the call won't run.",+ requestId: request.requestId,+ responderPrincipalId: "bob",+ }),+ }),+ );+ expect(fixture.events).toContainEqual(+ expect.objectContaining({+ type: "input.resolved",+ data: expect.objectContaining({+ resolutions: [+ expect.objectContaining({ outcome: "denied", requestId: request.requestId }),+ ],+ }),+ }),+ );+ // The model reads the failure under its own eve__tool call.+ expect(modelToolResult(fixture, request.action.callId)).toEqual(+ expect.objectContaining({+ output: { type: "error-text", value: TOOL_GONE_RESULT },+ toolName: CALL_TOOL_NAME,+ }),+ );+ expect(suspendedSteps(state)).toEqual([]);+ expect(fixture.events.at(-1)?.type).toBe("session.waiting");+ expect(fixture.events.map((event) => event.type)).toContain("turn.completed");});- const parked = await fixture.step();- const request = suspendedSteps(readDurableSession(parked.sessionState).state)[0]!.requests[0]!;- fixture.removeConnection();- await fixture.step({- delivery: {- kind: "deliver",- auth: bob,- payloads: [{ inputResponses: [{ requestId: request.requestId, optionId: "approve" }] }],- },++ it("keeps the request pending while the connection's resolver fails, and a retried approval runs it", async () => {+ const fixture = setup();+ const request = await parkNote(fixture);+ fixture.setConnection("failing");+ const before = fixture.events.length;++ // The step restores its connections before it reads the approval, so the+ // failure fails the turn, parks the session, and commits nothing.+ await approveAsBob(fixture, request.requestId);++ expect(fixture.events.slice(before).map((event) => event.type)).toEqual([+ "step.failed",+ "turn.failed",+ "session.waiting",+ ]);+ expect(suspendedSteps(fixture.state())[0]!.requests[0]!.requestId).toBe(request.requestId);+ expect(getApprovalAuditState(fixture.state()).candidateHistory).toEqual([]);+ expect(fixture.response).not.toHaveBeenCalled();+ expect(fixture.fetch).not.toHaveBeenCalled();++ fixture.setConnection("available");+ const approved = await approveAsBob(fixture, request.requestId);++ expect(fixture.response).toHaveBeenCalledOnce();+ expect(fixture.fetch).toHaveBeenCalledOnce();+ expect(suspendedSteps(approved)).toEqual([]);});- clearDurableDynamicCallbacks(sessionId);- const refused = await fixture.step();- expect(fixture.response).not.toHaveBeenCalled();- expect(fixture.fetch).not.toHaveBeenCalled();- const state = readDurableSession(refused.sessionState).state;- expect(suspendedSteps(state)[0]!.requests[0]!.requestId).toBe(request.requestId);- expect(getApprovalAuditState(state).candidateHistory).toEqual([- expect.objectContaining({- status: "rejected",- reason: expect.stringContaining("cannot replay its approvalResponse callback"),- }),- ]);});it("names the responder, not the requester, on a candidate's sign-in events", async () => {@@ -618,7 +675,7 @@ describe("turn connection approval restoration", () => {const parked = await fixture.step();const request = suspendedSteps(readDurableSession(parked.sessionState).state)[0]!.requests[0]!;- expect(request.action.toolName).toBe("connection_execute");+ expect(request.action.toolName).toBe("notes__saveNote");expect(request.prompt).toBe("Approve Notes: Save note?");expect(fixture.events.flatMap((event) =>@@ -651,34 +708,13 @@ describe("turn connection approval restoration", () => {expect(suspendedSteps(readDurableSession(resumed.sessionState).state)).toEqual([]);expect(resumed.serializedContext).not.toHaveProperty("eve.pendingConnectionCalls");expect(fixture.events.filter((event) => event.type === "turn.started")).toHaveLength(1);- // The connection call is reported as a nested action of the approved call.- const nestedCallId = `${request.action.callId}:1`;expect(fixture.events).toContainEqual(expect.objectContaining({- type: "actions.requested",- data: expect.objectContaining({- actions: [- expect.objectContaining({- callId: nestedCallId,- parentCallId: request.action.callId,- toolName: "notes__saveNote",- }),- ],- }),- }),- );- const results = fixture.events.filter((event) => event.type === "action.result");- const resultCallIds = results.map(- (event) => (event.data.result as { callId: string }).callId,- );- expect(resultCallIds.indexOf(nestedCallId)).toBeLessThan(- resultCallIds.indexOf(request.action.callId),- );- expect(results[resultCallIds.indexOf(nestedCallId)]).toEqual(- expect.objectContaining({+ type: "action.result",data: expect.objectContaining({status: "completed",result: expect.objectContaining({+ callId: request.action.callId,output: { body: { saved: true }, status: 200, statusText: "" },toolName: "notes__saveNote",}),@@ -735,6 +771,18 @@ describe("turn connection approval restoration", () => {});});+/** The result the model read back for `callId` on its latest request. */+function modelToolResult(fixture: ReturnType<typeof setup>, callId: string): unknown {+ const [options] = fixture.doStream.mock.calls.at(-1)! as [+ { prompt: { content: unknown; role: string }[] },+ ];+ return options.prompt+ .flatMap((message) =>+ message.role === "tool" ? (message.content as { toolCallId: string }[]) : [],+ )+ .find((part) => part.toolCallId === callId);+}+function callerHeader(call: readonly unknown[] | undefined): string | null {const [input, init] = (call ?? []) as [Request | string | URL, RequestInit | undefined];const headers = new Headers(input instanceof Request ? input.headers : init?.headers);diff --git a/packages/eve/src/execution/session/turn-step.test.ts b/packages/eve/src/execution/session/turn-step.test.tsindex 0ef2a8511..ec3e00bec 100644--- a/packages/eve/src/execution/session/turn-step.test.ts+++ b/packages/eve/src/execution/session/turn-step.test.ts@@ -262,8 +262,8 @@ function createTurnStepTestBundle(modelCallsPerStep?: number) {},hookRegistry: createRuntimeHookRegistry([]),moduleMap: { nodes: {} },- resolvedAgent: { config },- subagentRegistry: {},+ resolvedAgent: { config, dynamicSkillResolvers: [], dynamicToolResolvers: [] },+ subagentRegistry: { dynamicResolvers: [] },toolRegistry: {},turnAgent: TestTurnAgent,} as never;@@ -2013,7 +2013,7 @@ describe("turnStep", () => {serializedContext: {...createSerializedContext(),[TurnDeliveryIdsKey.name]: ["previous-delivery"],- [HistoryStateKey.name]: { availableSkills: announcement },+ [HistoryStateKey.name]: { announcements: { skills: announcement } },},sessionState: createStubSessionState(),});@@ -2022,7 +2022,7 @@ describe("turnStep", () => {action: "cancelled",serializedContext: {[TurnDeliveryIdsKey.name]: ["cancelled-delivery"],- [HistoryStateKey.name]: { availableSkills: announcement },+ [HistoryStateKey.name]: { announcements: { skills: announcement } },[SessionDynamicModelReferenceKey.name]: {id: "anthropic/claude-opus-4.6",contextWindowTokens: 1_000_000,@@ -3005,8 +3005,8 @@ describe("turnStep", () => {config: {},dynamicToolResolvers: [dynamicToolResolver],},- subagentRegistry: {},- toolRegistry: {},+ subagentRegistry: { dynamicResolvers: [], preparedTools: [] },+ toolRegistry: { toolsByName: new Map() },turnAgent: TestTurnAgent,} as never;vi.mocked(getCompiledRuntimeAgentBundle).mockResolvedValue(compiledBundle);diff --git a/packages/eve/src/execution/session/turn.test.ts b/packages/eve/src/execution/session/turn.test.tsindex 85b59012b..5c7cd59c5 100644--- a/packages/eve/src/execution/session/turn.test.ts+++ b/packages/eve/src/execution/session/turn.test.ts@@ -1011,7 +1011,7 @@ describe("SessionExecution checkpoints", () => {action: "park",hasRunsToDispatch: false,pendingCoordinationCallIds: ["wait-call"],- pendingTaskToolCalls: [{ callId: "wait-call", kind: "task_wait" }],+ pendingTaskToolCalls: [{ callId: "wait-call", kind: "eve__task_wait" }],serializedContext: {},sessionState,})),@@ -1035,7 +1035,7 @@ describe("SessionExecution checkpoints", () => {expect.anything(),expect.objectContaining({callId: "wait-call",- toolName: "task_wait",+ toolName: "eve__task_wait",failed: true,startedAtMs: expect.any(Number),completedAtMs: expect.any(Number),@@ -1279,7 +1279,7 @@ describe("SessionExecution checkpoints", () => {action: "park",hasRunsToDispatch: false,pendingCoordinationCallIds: ["wait-call"],- pendingTaskToolCalls: [{ callId: "wait-call", kind: "task_wait" }],+ pendingTaskToolCalls: [{ callId: "wait-call", kind: "eve__task_wait" }],serializedContext: {},sessionState,})),diff --git a/packages/eve/src/execution/session/turn.ts b/packages/eve/src/execution/session/turn.tsindex a0a11dedc..c8be8ab84 100644--- a/packages/eve/src/execution/session/turn.ts+++ b/packages/eve/src/execution/session/turn.ts@@ -305,7 +305,7 @@ export class SessionExecution {/*** The model tried to end the turn while its tasks work. The turn parks as- * `task_wait` would, until one of them settles or the turn is steered.+ * `eve__task_wait` would, until one of them settles or the turn is steered.*/private async waitForHeldTurn(turn: ActiveTurn): Promise<"cancelled" | "woke"> {let interrupted = false;@@ -444,7 +444,7 @@ export class SessionExecution {}}- /** Answers every `task_wait` that can return now; returns the ones still waiting. */+ /** Answers every `eve__task_wait` that can return now; returns the ones still waiting. */private async resolveTaskWaits(waits: readonly TaskWait[],interrupted: boolean,@@ -469,11 +469,11 @@ export class SessionExecution {}/**- * `task_cancel` is answered at once, and so is each `task_wait` that can+ * `eve__task_cancel` is answered at once, and so is each `eve__task_wait` that can* return now: a timeout of 0, a result already waiting, or nothing working.- * Any other `task_wait` parks the open turn, reported once as+ * Any other `eve__task_wait` parks the open turn, reported once as* `turn.waiting`, and the turn resolves it alongside the step's other- * deferred calls.+ * workflow tool calls.*/private async answerTaskToolCalls(calls: readonly TaskToolCall[],@@ -482,7 +482,7 @@ export class SessionExecution {const waits: TaskWait[] = [];const startedAtMs = calls.length === 0 ? 0 : await startTaskToolCallsStep();for (const call of calls) {- if (call.kind === "task_wait") waits.push(startTaskWait(call, startedAtMs));+ if (call.kind === TASK_WAIT_TOOL_NAME) waits.push(startTaskWait(call, startedAtMs));else {let result: RuntimeActionResult;try {@@ -522,7 +522,7 @@ export class SessionExecution {await this.input.cursor.advance((state) =>traceTaskToolCallStep(state, {callId: wait.callId,- toolName: "task_wait",+ toolName: TASK_WAIT_TOOL_NAME,startedAtMs: wait.startedAtMs,completedAtMs: Date.now(),input: wait.input,@@ -549,7 +549,7 @@ export class SessionExecution {}/**- * One `task_wait` call the turn is parked on. Its timeout is a durable sleep+ * One `eve__task_wait` call the turn is parked on. Its timeout is a durable sleep* the turn races against the inbox; nothing polls.*/interface TaskWait {@@ -562,7 +562,7 @@ interface TaskWait {}function startTaskWait(- call: Extract<TaskToolCall, { readonly kind: "task_wait" }>,+ call: Extract<TaskToolCall, { readonly kind: typeof TASK_WAIT_TOOL_NAME }>,startedAtMs: number,): TaskWait {const { callId, timeoutMs } = call;diff --git a/packages/eve/src/execution/skills/instructions.test.ts b/packages/eve/src/execution/skills/instructions.test.tsindex e4b25b8ec..b3d6533b8 100644--- a/packages/eve/src/execution/skills/instructions.test.ts+++ b/packages/eve/src/execution/skills/instructions.test.ts@@ -31,9 +31,6 @@ describe("formatAvailableSkillsSection", () => {expect(result).not.toBeNull();expect(result).toContain("Available skills");- expect(result).toContain(- "If the user names a skill or the request clearly matches one of the descriptions below, call load_skill before proceeding.",- );expect(result).toContain("If multiple skills match, activate the minimal set that covers the task.",);diff --git a/packages/eve/src/execution/skills/instructions.ts b/packages/eve/src/execution/skills/instructions.tsindex af3f13f57..3dbd5ff49 100644--- a/packages/eve/src/execution/skills/instructions.ts+++ b/packages/eve/src/execution/skills/instructions.ts@@ -1,3 +1,6 @@+import type { DynamicSkillManifest } from "#context/keys.js";+import type { Announcement } from "#harness/announcements.js";+import { SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";import { FALLBACK_SKILL_ROOT, MODEL_SKILL_ROOT } from "#shared/skill-paths.js";export interface AvailableSkillDescription {@@ -7,19 +10,21 @@ export interface AvailableSkillDescription {readonly hasFiles?: boolean;}+const ANNOUNCEMENT_KEY = "skills";+/**- * Formats the "Available skills" system prompt section.+ * Formats the "Available skills" section: every skill that isn't deferred,+ * with the instruction to load it. Deferred skills stay out of context until+ * search finds them.*- * All skills are always listed regardless of activation state. Active skill- * instructions are never injected into the system prompt — the model already- * has them from the `load_skill` tool result. The formatter uses the canonical- * symbolic skill root so supporting files remain discoverable without- * provisioning a sandbox.+ * A loaded skill's instructions are never injected into the system prompt;+ * the model has them from the result of loading it. The formatter uses the+ * canonical symbolic skill root so supporting files remain discoverable+ * without provisioning a sandbox.*- * Authored skills call this at graph resolution time so the section is- * part of the turn agent's static instructions. Dynamic skills- * (`defineDynamic` in `agent/skills/`) reuse the same formatter for- * durable context announcements.+ * Authored skills call this at graph resolution time so the section is part of+ * the turn agent's static instructions. Dynamic skills reuse it for their+ * announcement.*/export function formatAvailableSkillsSection(skills: readonly AvailableSkillDescription[],@@ -32,7 +37,7 @@ export function formatAvailableSkillsSection("Available skills","Listed skills are available in this run. Do not claim a listed skill is inaccessible unless activation or workspace inspection actually fails.","Dynamic skill announcements replace earlier dynamic skills and override static skills with the same name. Static skills omitted from a dynamic announcement remain available.",- "If the user names a skill or the request clearly matches one of the descriptions below, call load_skill before proceeding.",+ `If the user names a skill or the request clearly matches one of the descriptions below, load it with ${SKILL_TOOL_NAME}({ name }) before proceeding.`,"If multiple skills match, activate the minimal set that covers the task. After activation, follow the returned instructions instead of improvising around them.","If activation fails, say so briefly and continue with the best available alternative.",`Skill files live under \`${MODEL_SKILL_ROOT}/<skill>/\`, with \`${FALLBACK_SKILL_ROOT}/<skill>/\` as the fallback when \`$HOME\` is unavailable.`,@@ -43,8 +48,37 @@ export function formatAvailableSkillsSection(return lines.join("\n");}+/**+ * Announces the dynamic skills that aren't deferred, in full whenever they+ * change. Nothing is announced until the first one resolves. `announced` holds+ * the values announced so far.+ */+export function dynamicSkillAnnouncements(+ manifest: DynamicSkillManifest | undefined,+ announced: Readonly<Record<string, string>> | undefined,+): Readonly<Record<string, Announcement>> {+ const skills = Object.values(manifest ?? {})+ .flat()+ .filter((skill) => skill.deferred !== true);+ if (skills.length === 0 && announced?.[ANNOUNCEMENT_KEY] === undefined) return {};+ const text =+ formatAvailableSkillsSection(+ skills.map(({ description, name, revision }) => ({+ description,+ hasFiles: revision !== undefined,+ name,+ })),+ ) ?? "Available skills: none";+ return { [ANNOUNCEMENT_KEY]: { value: text, render: () => text } };+}+function formatAvailableSkillLine(skill: AvailableSkillDescription): string {const prefix = `- ${skill.name}: ${skill.description}`;if (skill.hasFiles === false) return prefix;- return `${prefix} (path: ${MODEL_SKILL_ROOT}/${skill.name}/SKILL.md)`;+ return `${prefix} (path: ${skillFilePath(skill.name)})`;+}++/** Where a skill's `SKILL.md` lives under the model's skill root. */+export function skillFilePath(name: string): string {+ return `${MODEL_SKILL_ROOT}/${name}/SKILL.md`;}diff --git a/packages/eve/src/execution/tasks/calls.test.ts b/packages/eve/src/execution/tasks/calls.test.tsindex c344a53e5..2da1bab34 100644--- a/packages/eve/src/execution/tasks/calls.test.ts+++ b/packages/eve/src/execution/tasks/calls.test.ts@@ -9,7 +9,7 @@ it("reads task_wait's timeoutSeconds as the milliseconds the turn waits", () =>{input: { timeoutSeconds: 90 },toolCallId: "wait-1",- toolName: "task_wait",+ toolName: "eve__task_wait",type: "tool-call",},],@@ -17,5 +17,5 @@ it("reads task_wait's timeoutSeconds as the milliseconds the turn waits", () =>},]);- expect(calls).toEqual([{ callId: "wait-1", kind: "task_wait", timeoutMs: 90_000 }]);+ expect(calls).toEqual([{ callId: "wait-1", kind: "eve__task_wait", timeoutMs: 90_000 }]);});diff --git a/packages/eve/src/execution/tasks/calls.ts b/packages/eve/src/execution/tasks/calls.tsindex 222a0e28f..483dce02c 100644--- a/packages/eve/src/execution/tasks/calls.ts+++ b/packages/eve/src/execution/tasks/calls.ts@@ -9,14 +9,22 @@ import { TASK_CANCEL_TOOL_NAME, TASK_WAIT_TOOL_NAME } from "#protocol/task-tools* workflow tool calls; the session answers them instead of a workflow run.*/export type TaskToolCall =- | { readonly callId: string; readonly kind: "task_wait"; readonly timeoutMs?: number }- | { readonly callId: string; readonly kind: "task_cancel"; readonly taskId: string };+ | {+ readonly callId: string;+ readonly kind: typeof TASK_WAIT_TOOL_NAME;+ readonly timeoutMs?: number;+ }+ | {+ readonly callId: string;+ readonly kind: typeof TASK_CANCEL_TOOL_NAME;+ readonly taskId: string;+ };/*** The task tool calls a parked step's response still waits on. The response* is the one record of these calls: a call without a result is pending, and- * a call with invalid input already carries its error result. The names are- * reserved, so no authored tool's call matches.+ * a call with invalid input already carries its error result. The `eve`+ * namespace is reserved, so no authored tool's call matches.*/export function pendingTaskToolCalls(messages: readonly ModelMessage[]): TaskToolCall[] {const answered = new Set<string>();@@ -34,14 +42,18 @@ export function pendingTaskToolCalls(messages: readonly ModelMessage[]): TaskTooconst input = typeof part.input === "object" && part.input !== null ? part.input : {};if (part.toolName === TASK_CANCEL_TOOL_NAME) {const taskId = String(Reflect.get(input, "taskId"));- calls.push({ callId: part.toolCallId, kind: "task_cancel", taskId });+ calls.push({ callId: part.toolCallId, kind: TASK_CANCEL_TOOL_NAME, taskId });} else if (part.toolName === TASK_WAIT_TOOL_NAME) {// Seconds, to match `sleep`.const timeoutSeconds: unknown = Reflect.get(input, "timeoutSeconds");calls.push(typeof timeoutSeconds === "number"- ? { callId: part.toolCallId, kind: "task_wait", timeoutMs: timeoutSeconds * 1_000 }- : { callId: part.toolCallId, kind: "task_wait" },+ ? {+ callId: part.toolCallId,+ kind: TASK_WAIT_TOOL_NAME,+ timeoutMs: timeoutSeconds * 1_000,+ }+ : { callId: part.toolCallId, kind: TASK_WAIT_TOOL_NAME },);}}@@ -49,7 +61,7 @@ export function pendingTaskToolCalls(messages: readonly ModelMessage[]): TaskTooreturn calls;}-/** Why a `task_wait` call, or a held turn, stopped waiting. */+/** Why an `eve__task_wait` call, or a held turn, stopped waiting. */export type TaskWaitResult =| {readonly status: "settled";diff --git a/packages/eve/src/execution/tasks/model-step.ts b/packages/eve/src/execution/tasks/model-step.tsindex c40bf551e..e1f87f163 100644--- a/packages/eve/src/execution/tasks/model-step.ts+++ b/packages/eve/src/execution/tasks/model-step.ts@@ -23,7 +23,7 @@ import {type TaskResult,} from "#execution/tasks/table.js";import { splitTaskIdInput } from "#execution/tasks/task-id-input.js";-import { entryPointOf, isAgentTool, startsTasks } from "#execution/tasks/tool-entry-point.js";+import { entryPointOf, isAgentTool } from "#execution/tasks/tool-entry-point.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";import { createFrameworkUserMessage, type HarnessModelMessage } from "#harness/messages.js";import type { HarnessSession, HarnessToolMap } from "#harness/types.js";@@ -42,17 +42,13 @@ export function isTaskTool(definition: HarnessToolDefinition | undefined): boole);}-/** Whether the agent can start tasks, so the task tools and system block are offered. */-export function offersTasks(tools: HarnessToolMap): boolean {- for (const definition of tools.values()) {- if (startsTasks(definition)) return true;- }- return false;+/** Workflow tools, and the task tools that control them, run after the model step. */+export function isWorkflowTool(definition: HarnessToolDefinition | undefined): boolean {+ return definition?.workflowId !== undefined || isTaskTool(definition);}-/** Adds `task_wait` and `task_cancel` to a tool set that can start tasks. */+/** Adds `eve__task_wait` and `eve__task_cancel` to the tools of a session that offers tasks. */export function withTaskTools(tools: HarnessToolMap): HarnessToolMap {- if (!offersTasks(tools)) return tools;return new Map([...tools,[TASK_WAIT_TOOL_NAME, taskWaitTool],@@ -61,21 +57,21 @@ export function withTaskTools(tools: HarnessToolMap): HarnessToolMap {}/**- * The task block, when the agent can start tasks. `finalReplyOnly` is set for a+ * The task block, when the session offers tasks. `finalReplyOnly` is set for a* child or schedule session, which hides a held turn's text, so a reply before* a result would reach no one.*/export function taskSystemMessages(- tools: HarnessToolMap,+ offers: boolean,options: { readonly finalReplyOnly: boolean },): SystemModelMessage[] {- if (!offersTasks(tools)) return [];+ if (!offers) return [];const content = options.finalReplyOnly ? FINAL_REPLY_TASK_SYSTEM_BLOCK : TASK_SYSTEM_BLOCK;return [{ content, role: "system" }];}-/** A deferred call as the model made it, before the session knows what it enters. */-export interface DeferredCall {+/** A workflow tool call as the model made it, before the session knows what it enters. */+export interface WorkflowCall {readonly callId: string;readonly definition: HarnessToolDefinition | undefined;/** The call's model input. */@@ -84,7 +80,7 @@ export interface DeferredCall {readonly turnId: string;}-/** How a deferred call enters its workflow, with the tool's own input. */+/** How a workflow tool call enters its workflow, with the tool's own input. */export interface CommittedCallEntry {readonly entry: WorkflowToolCallEntry;readonly input: JsonObject;@@ -92,12 +88,12 @@ export interface CommittedCallEntry {}/**- * How a deferred call enters its workflow. A call that starts a task commits+ * How a workflow tool call enters its workflow. A call that starts a task commits* the task's record alongside the call, so the run starts with its id; a* `serve` tool's call with `taskId` goes to that task's `receive()`, which the* session checks when it sends the call.*/-export function commitCallEntry(session: HarnessSession, call: DeferredCall): CommittedCallEntry {+export function commitCallEntry(session: HarnessSession, call: WorkflowCall): CommittedCallEntry {const entryPoint = entryPointOf(call.definition);switch (entryPoint) {case "execute":@@ -114,7 +110,7 @@ export function commitCallEntry(session: HarnessSession, call: DeferredCall): Cofunction commitTask(session: HarnessSession,- call: DeferredCall,+ call: WorkflowCall,entryPoint: "task" | "serve",input: JsonObject,): CommittedCallEntry {diff --git a/packages/eve/src/execution/tasks/render.ts b/packages/eve/src/execution/tasks/render.tsindex fbedec8f1..1082827fa 100644--- a/packages/eve/src/execution/tasks/render.ts+++ b/packages/eve/src/execution/tasks/render.ts@@ -1,10 +1,11 @@import type { TaskCancelResult, TaskWaitResult } from "#execution/tasks/calls.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";+import { TASK_CANCEL_TOOL_NAME, TASK_WAIT_TOOL_NAME } from "#protocol/task-tools.js";import { formatDuration } from "#shared/format-duration.js";// Every string the model reads about tasks lives in this file.-export const TASK_WAIT_DESCRIPTION =- "Call task_wait sparingly, only when you deliberately want to withhold a message from the user while waiting for a task result. Tasks keep running and their results reach you without calling task_wait. If the user should hear from you now, reply instead. Wait silently until any of your tasks has a result, a new message arrives, or timeoutSeconds pass. It returns after the first result, which follows in a <task_result> message; wait again only if you still want to withhold a reply until another result arrives. Waiting never stops a task.";+export const TASK_WAIT_DESCRIPTION = `Call ${TASK_WAIT_TOOL_NAME} sparingly, only when you deliberately want to withhold a message from the user while waiting for a task result. Tasks keep running and their results reach you without calling ${TASK_WAIT_TOOL_NAME}. If the user should hear from you now, reply instead. Wait silently until any of your tasks has a result, a new message arrives, or timeoutSeconds pass. It returns after the first result, which follows in a <task_result> message; wait again only if you still want to withhold a reply until another result arrives. Waiting never stops a task.`;export const TASK_WAIT_TIMEOUT_DESCRIPTION ="Optional. Seconds to wait before returning without a result. Omit it to wait for a result or a new message, which is almost always right. Don't use short timeouts to check on tasks: results reach you without checking.";@@ -25,13 +26,13 @@ const TASK_SYSTEM_BLOCK_OPENING = [const TASK_SYSTEM_BLOCK_CLOSING = ["- Don't redo work you delegated. Never use sleep or shell commands to wait for a task.",- "- To correct or continue an agent, or any task that accepts more input, call its tool again with its taskId. A new message never stops your tasks: answer it if it asks you something, decide whether it changes the work, then keep each task, correct an agent by taskId, or stop a task with task_cancel.",+ `- To correct or continue an agent, or any task that accepts more input, call its tool again with its taskId. A new message never stops your tasks: answer it if it asks you something, decide whether it changes the work, then keep each task, correct an agent by taskId, or stop a task with ${TASK_CANCEL_TOOL_NAME}.`,];/** The task block for a session a person reads, where a reply before a result is posted. */export const TASK_SYSTEM_BLOCK = [...TASK_SYSTEM_BLOCK_OPENING,- "- If you have nothing to tell the person until a result arrives, call task_wait. Start every independent task first, then wait.",+ `- If you have nothing to tell the person until a result arrives, call ${TASK_WAIT_TOOL_NAME}. Start every independent task first, then wait.`,"- If the person should hear from you now, for example to confirm work is underway when they don't need the result yet, reply. When results arrive, report them without repeating yourself.",...TASK_SYSTEM_BLOCK_CLOSING,].join("\n");@@ -39,7 +40,7 @@ export const TASK_SYSTEM_BLOCK = [/** The task block for a child or schedule session, whose caller reads only the final reply. */export const FINAL_REPLY_TASK_SYSTEM_BLOCK = [...TASK_SYSTEM_BLOCK_OPENING,- "- Only your final reply reaches your caller. While tasks work, call task_wait instead of replying. Start every independent task first, then wait.",+ `- Only your final reply reaches your caller. While tasks work, call ${TASK_WAIT_TOOL_NAME} instead of replying. Start every independent task first, then wait.`,...TASK_SYSTEM_BLOCK_CLOSING,].join("\n");@@ -91,7 +92,7 @@ export function renderUnknownCancelTaskError(taskId: string): string {return `No task "${taskId}" is in this session. Copy an id from a "Started task" receipt or the [Tasks] note.`;}-/** What the model reads when `task_cancel` answers. */+/** What the model reads when `eve__task_cancel` answers. */export function renderTaskCancelResult(taskId: string, result: TaskCancelResult): string {if (result.status === "already_finished") return `${taskId} had no work to stop.`;if (!result.resumable) return `Stopped ${taskId}; it won't report back.`;@@ -99,14 +100,14 @@ export function renderTaskCancelResult(taskId: string, result: TaskCancelResult)}export function renderTooManyTasksError(max: number, workingIds: readonly string[]): string {- return `${String(max)} tasks are already working (${workingIds.join(", ")}). Wait with task_wait or stop one with task_cancel, then try again.`;+ return `${String(max)} tasks are already working (${workingIds.join(", ")}). Wait with ${TASK_WAIT_TOOL_NAME} or stop one with ${TASK_CANCEL_TOOL_NAME}, then try again.`;}export function renderFinalOutputWhileWorkingError(workingIds: readonly string[]): string {- return `You can't give your final output while tasks you started are working (${workingIds.join(", ")}). Wait for their results with task_wait, or stop a task with task_cancel, then call final_output.`;+ return `You can't give your final reply while tasks you started are working (${workingIds.join(", ")}). Wait for their results with ${TASK_WAIT_TOOL_NAME}, or stop a task with ${TASK_CANCEL_TOOL_NAME}, then call ${REPLY_TOOL_NAME}.`;}-/** What the model reads when `task_wait` returns after waiting `waitedMs`. */+/** What the model reads when `eve__task_wait` returns after waiting `waitedMs`. */export function renderTaskWaitResult(result: TaskWaitResult, waitedMs: number): string {switch (result.status) {case "settled":@@ -149,7 +150,7 @@ function resultsArriveLater(workingCount: number): string {function interruptChoices(workingCount: number): string {const keep = workingCount === 1 ? "keep the task" : "keep the tasks";- return `${keep}, call an agent again with its taskId to correct it, or stop a task with task_cancel.`;+ return `${keep}, call an agent again with its taskId to correct it, or stop a task with ${TASK_CANCEL_TOOL_NAME}.`;}/** One `<task_result>` block of a `task.result` message. */diff --git a/packages/eve/src/execution/tasks/session.test.ts b/packages/eve/src/execution/tasks/session.test.tsindex 82e05c83f..dae0b0e7f 100644--- a/packages/eve/src/execution/tasks/session.test.ts+++ b/packages/eve/src/execution/tasks/session.test.ts@@ -60,7 +60,7 @@ describe("answerTaskCancel", () => {await answerTaskCancel(cursor, {callId: "cancel-call",- kind: "task_cancel",+ kind: "eve__task_cancel",taskId: research.taskId,});diff --git a/packages/eve/src/execution/tasks/session.ts b/packages/eve/src/execution/tasks/session.tsindex 46eeb517d..29b87dba9 100644--- a/packages/eve/src/execution/tasks/session.ts+++ b/packages/eve/src/execution/tasks/session.ts@@ -48,7 +48,7 @@ export async function cancelWorkingTasks(export async function answerTaskCancel(cursor: SessionStateCursor,- call: Extract<TaskToolCall, { readonly kind: "task_cancel" }>,+ call: Extract<TaskToolCall, { readonly kind: typeof TASK_CANCEL_TOOL_NAME }>,): Promise<RuntimeActionResult> {const result = taskCancelResult(sessionTaskTable(cursor), call.taskId);if (result === undefined) {diff --git a/packages/eve/src/execution/tasks/table.ts b/packages/eve/src/execution/tasks/table.tsindex 0e4b512f5..3d2fe914d 100644--- a/packages/eve/src/execution/tasks/table.ts+++ b/packages/eve/src/execution/tasks/table.ts@@ -17,7 +17,7 @@ const TASK_TABLE_VERSION = 1;/** At most this many tasks work at once in one session: a backstop normal use shouldn't reach. */export const MAX_WORKING_TASKS = 32;-/** Finished records kept so `task_cancel` can still answer `already_finished`. */+/** Finished records kept so `eve__task_cancel` can still answer `already_finished`. */const MAX_FINISHED_RECORDS = 100;/** Idle resumable tasks the `[Tasks]` note lists: the most recently used ones. */@@ -189,7 +189,7 @@ export function taskWaitResult(}/**- * What `task_cancel` answers, or `undefined` for a task that doesn't exist.+ * What `eve__task_cancel` answers, or `undefined` for a task that doesn't exist.* `cancelled` means the caller cancels the working task.*/export function taskCancelResult(table: TaskTable, taskId: string): TaskCancelResult | undefined {diff --git a/packages/eve/src/execution/tasks/tool-entry-point.ts b/packages/eve/src/execution/tasks/tool-entry-point.tsindex 175c5041a..9c97c5ea0 100644--- a/packages/eve/src/execution/tasks/tool-entry-point.ts+++ b/packages/eve/src/execution/tasks/tool-entry-point.ts@@ -3,7 +3,7 @@ import type { PreparedDispatchTarget } from "#tools/behavior.js";import type { WorkflowToolEntryPoint } from "#tools/workflow-entry-point.js";/**- * The entry point a deferred tool's calls run through. Every agent tool is a+ * The entry point a workflow tool's calls run through. Every agent tool is a* `serve` tool; tools that dispatch nowhere are called like `execute`.*/export function entryPointOf(diff --git a/packages/eve/src/execution/tool-stubs/catalog.integration.test.ts b/packages/eve/src/execution/tool-stubs/catalog.integration.test.tsnew file mode 100644index 000000000..bdbfbb461--- /dev/null+++ b/packages/eve/src/execution/tool-stubs/catalog.integration.test.ts@@ -0,0 +1,123 @@+import { describe, expect, it } from "vitest";++import { SessionKey, ToolStubsKey } from "#context/keys.js";+import { toolStubProvider } from "#context/providers/tool-stubs.js";+import { workflowEntry } from "#execution/session/entry.js";+import { buildToolSet } from "#harness/tools.js";+import { createTestRuntime } from "#internal/testing/app-harness.js";+import {+ catalogContext,+ connectionTool,+ fakeConnection,+ inlineTool,+} from "#internal/testing/catalog-fixtures.js";+import { buildSerializedContext } from "#internal/testing/entry-test-helpers.js";+import { waitForHook } from "#internal/testing/workflow-test-helpers.js";+import { start } from "#internal/workflow/runtime.js";+import { CALL_TOOL_NAME } from "#protocol/catalog-tools.js";+import { isAsyncIterable } from "#shared/async-iterable.js";+import { STUB_CONTEXT_KEY, type ToolStub } from "#tool-stubs/types.js";+import type { ToolExecuteOptions } from "#tools/definition.js";++/** The last value a tool execution produced, as the AI SDK records it. */+async function settle(output: unknown): Promise<unknown> {+ if (!isAsyncIterable(output)) return await output;+ let last: unknown;+ for await (const value of output) last = value;+ return last;+}++describe("tool stubs through eve__tool", () => {+ it("match the entry an eve__tool call reaches, by its name and input, and leave other calls live", async () => {+ const rules: readonly ToolStub[] = [+ {+ id: "refund",+ tool: "refund_invoice",+ match: { invoiceId: { const: "inv_1" } },+ outcome: { response: { refunded: "stubbed" } },+ },+ {+ id: "open-issues",+ tool: "linear__list_issues",+ match: { status: { const: "open" } },+ outcome: { response: { issues: ["Fix login"] } },+ },+ ];+ const linear = fakeConnection({+ description: "Linear issues.",+ name: "linear",+ tools: [+ connectionTool("list_issues", {+ type: "object",+ properties: { status: { type: "string" } },+ required: ["status"],+ }),+ ],+ });+ const runtime = await createTestRuntime();+ await runtime.run(async () => {+ const token = "catalog-stub-playback";+ const run = await start(workflowEntry, [+ {+ kind: "initial",+ ownerDeploymentId: "dpl_inline",+ input: {},+ serializedContext: {+ ...buildSerializedContext({ channelKind: "http" }),+ [STUB_CONTEXT_KEY]: { token, rules },+ },+ },+ ]);+ try {+ await waitForHook({ runId: run.runId }, { token });+ const {+ catalog,+ ctx,+ run: inSession,+ } = catalogContext({+ connections: [linear],+ tools: [+ inlineTool("refund_invoice", {+ deferred: true,+ schema: {+ type: "object",+ properties: { invoiceId: { type: "string" } },+ required: ["invoiceId"],+ },+ }),+ ],+ });+ ctx.set(SessionKey, {+ auth: { current: null, initiator: null },+ sessionId: run.runId,+ turn: { id: "turn-0", sequence: 0 },+ });+ ctx.set(ToolStubsKey, { rootSessionId: run.runId, rules, token });+ ctx.setVirtualContext(toolStubProvider.key, toolStubProvider.create(ctx)!.value);+ const execute = buildToolSet({+ describe: (definition) => definition.description,+ resolve: catalog.resolve,+ tools: catalog.advertised,+ })[CALL_TOOL_NAME]!.execute as (input: unknown, options: ToolExecuteOptions) => unknown;+ const call = (toolCallId: string, input: Record<string, unknown>) =>+ inSession(async () => await settle(execute(input, { messages: [], toolCallId })));++ expect(+ await call("refund-1", { name: "refund_invoice", input: { invoiceId: "inv_1" } }),+ ).toEqual({ refunded: "stubbed" });+ expect(+ await call("refund-2", { name: "refund_invoice", input: { invoiceId: "inv_2" } }),+ ).toEqual({ input: { invoiceId: "inv_2" }, ran: "refund_invoice" });++ expect(+ await call("issues-1", { name: "linear__list_issues", input: { status: "open" } }),+ ).toEqual({ issues: ["Fix login"] });+ expect(linear.calls).toEqual([]);+ await call("issues-2", { name: "linear__list_issues", input: { status: "closed" } });+ expect(linear.calls).toEqual([{ input: { status: "closed" }, tool: "list_issues" }]);+ } finally {+ await run.cancel();+ }+ });+ });+});diff --git a/packages/eve/src/execution/tool-stubs/connection.integration.test.ts b/packages/eve/src/execution/tool-stubs/connection.integration.test.tsdeleted file mode 100644index e163dbe07..000000000--- a/packages/eve/src/execution/tool-stubs/connection.integration.test.ts+++ /dev/null@@ -1,217 +0,0 @@-import { toolStubProvider } from "#context/providers/tool-stubs.js";-import { describe, expect, it, vi } from "vitest";-import { ContextContainer, contextStorage } from "#context/container.js";-import { SessionKey, ToolStubsKey } from "#context/keys.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { createTestRuntime } from "#internal/testing/app-harness.js";-import { buildSerializedContext } from "#internal/testing/entry-test-helpers.js";-import { waitForHook } from "#internal/testing/workflow-test-helpers.js";-import { getWorld, start } from "#internal/workflow/runtime.js";-import { workflowEntry } from "#execution/session/entry.js";-import { resolveConnectionTools } from "#execution/tools/connection-tools.js";-import { calledTool } from "#evals/assertions/run.js";-import { deriveRunFacts } from "#evals/runner/derive-run-facts.js";-import type { EveEvalTaskResult } from "#evals/types.js";-import { emitNestedToolActions } from "#harness/nested-actions.js";-import { stampTestEvents } from "#internal/testing/events.js";-import type { UnstampedMessageStreamEvent } from "#protocol/message.js";-import { recordToolStubFailure } from "#tool-stubs/execute.js";-import { readStubFailure } from "#execution/tool-stubs/steps.js";-import { STUB_CONTEXT_KEY, STUB_FAILURE_NAMESPACE, type ToolStub } from "#tool-stubs/types.js";-import type { ConnectionClient } from "#shared/connection-types.js";-import type { ToolContext } from "#tools/definition.js";--describe("connection operation stubs", () => {- it("reports child-scoped failures and recovery, validates input, and executes unmatched calls", async () => {- let liveCalls = 0;- const runtime = await createTestRuntime();- await runtime.run(async () => {- const rules: readonly ToolStub[] = [- {- id: "open",- tool: "researcher/linear__list_issues",- match: { status: { const: "open" } },- outcomes: [- { throw: { name: "TimeoutError", message: "Issue service timed out" } },- { throw: { name: "ConnectionAuthorizationRequiredError", message: "Access denied" } },- { response: { issues: ["milk"] } },- ],- },- ];- const token = "connection-stub-playback";- const run = await start(workflowEntry, [- {- kind: "initial",- ownerDeploymentId: "dpl_inline",- input: {},- serializedContext: {- ...buildSerializedContext({ channelKind: "http" }),- [STUB_CONTEXT_KEY]: { token, rules },- },- },- ]);- try {- await waitForHook({ runId: run.runId }, { token });- const client: ConnectionClient = {- close: async () => {},- connect: async () => {},- getToolMetadata: async () => [- {- name: "list_issues",- description: "List issues.",- inputSchema: {- type: "object",- properties: { status: { type: "string" } },- required: ["status"],- },- },- ],- executeTool: async () => {- liveCalls++;- return { content: [{ type: "text", text: "live" }] };- },- };- const connection = {- sourceId: "test:linear",- sourceKind: "module" as const,- logicalPath: "connections/linear.ts",- connectionName: "linear",- description: "Issues",- protocol: "mcp" as const,- url: "https://linear.example/mcp",- };- const context = new ContextContainer();- context.set(SessionKey, {- sessionId: run.runId,- auth: { current: null, initiator: null },- turn: { id: "turn-0", sequence: 0 },- });- context.set(ToolStubsKey, {- token,- rules,- rootSessionId: run.runId,- agentPath: "researcher",- });- context.set(ConnectionRegistryKey, {- dispose: async () => {},- getClient: () => client,- getConnectionApproval: () => undefined,- getConnectionNames: () => ["linear"],- getConnections: () => [connection],- });- context.setVirtualContext(toolStubProvider.key, toolStubProvider.create(context)!.value);- await contextStorage.run(context, async () => {- const execute = resolveConnectionTools()!.connection_execute!.execute!;- const events: UnstampedMessageStreamEvent[] = [];- const call = async (callId: string, input: unknown) => {- try {- return await execute({ connection: "linear", tool: "list_issues", input }, {- callId,- } as ToolContext);- } finally {- await emitNestedToolActions(- async (event) => {- events.push(event);- },- { sequence: 0, stepIndex: 0, turnId: "turn-0" },- callId,- );- }- };- const evalResult = (): EveEvalTaskResult => {- const stamped = stampTestEvents(events);- return {- events: stamped,- derived: deriveRunFacts(stamped),- status: "completed",- output: null,- finalMessage: null,- traceContexts: [],- };- };- await expect(call("invalid", { status: 42 })).rejects.toThrow(/Invalid input/);- await expect(call("failed", { status: "open" })).rejects.toMatchObject({- name: "TimeoutError",- message: "Issue service timed out",- });- expect(- await calledTool("linear__list_issues", {- status: "failed",- input: { status: "open" },- output: "Issue service timed out",- count: 1,- }).evaluate(evalResult()),- ).toMatchObject({ score: 1 });- expect(events[0]).toMatchObject({- type: "actions.requested",- data: {- actions: [{ parentCallId: "failed", toolName: "linear__list_issues" }],- },- });- expect(await readStubFailure(run.runId)).toBeUndefined();- expect(liveCalls).toBe(0);- await expect(call("auth-named-failure", { status: "open" })).rejects.toMatchObject({- name: "ConnectionAuthorizationRequiredError",- message: "Access denied",- });- expect(- await calledTool("linear__list_issues", {- status: "failed",- output: "Access denied",- count: 1,- }).evaluate(evalResult()),- ).toMatchObject({ score: 1 });- expect(await call("stubbed", { status: "open", limit: 10 })).toEqual({- issues: ["milk"],- });- expect(- await calledTool("linear__list_issues", {- status: "completed",- input: { status: "open", limit: 10 },- output: { issues: ["milk"] },- count: 1,- }).evaluate(evalResult()),- ).toMatchObject({ score: 1 });- expect(liveCalls).toBe(0);- await call("live", { status: "closed" });- expect(liveCalls).toBe(1);- // connection_execute converts the operation's output before the model receives it.- await recordToolStubFailure("connection_execute", "live");- expect(await readStubFailure(run.runId)).toBeUndefined();- const world = await getWorld();- const append = world.streams.write.bind(world.streams);- const failureSuffix = Buffer.from(STUB_FAILURE_NAMESPACE).toString("base64url");- const entered = Promise.withResolvers<void>();- const release = Promise.withResolvers<void>();- const write = vi.spyOn(world.streams, "write").mockImplementation(async (...args) => {- if (args[1].endsWith(`_${failureSuffix}`)) {- entered.resolve();- await release.promise;- }- return await append(...args);- });- let acknowledged = false;- const recording = recordToolStubFailure("connection_execute", "stubbed").then(() => {- acknowledged = true;- });- try {- await entered.promise;- expect(acknowledged).toBe(false);- expect(await readStubFailure(run.runId)).toBeUndefined();- release.resolve();- await recording;- expect(await readStubFailure(run.runId)).toBe(- 'Stubbed tool "connection_execute" failed during output processing.',- );- } finally {- release.resolve();- await recording;- write.mockRestore();- }- });- } finally {- await run.cancel();- }- });- });-});diff --git a/packages/eve/src/execution/tools/connection-approval.ts b/packages/eve/src/execution/tools/connection-approval.tsdeleted file mode 100644index ef649f1ce..000000000--- a/packages/eve/src/execution/tools/connection-approval.ts+++ /dev/null@@ -1,211 +0,0 @@-/**- * Approval for `connection_execute`: each call is approved under the called- * connection's own policy, keyed by connection and tool, and an approved call- * runs only against the connection instance it was approved for.- */--import {- resolveApprovalPolicy,- type Approval,- type ApprovalContext,- type ApprovalResponseContext,- type ApprovalResponseDecision,- type ApprovalStatus,-} from "#approval/definition.js";-import { loadContext } from "#context/container.js";-import { ContextKey } from "#context/key.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { isApprovalRecheck } from "#harness/approval-recheck.js";-import type { ResolvedConnectionDefinition } from "#runtime/types.js";-import { defineDurableCallback } from "#tools/durable-callbacks.js";--import {- CONNECTION_EXECUTE_TOOL_NAME,- findConnection,- qualifiedToolName,- readExecuteTarget,- type ExecuteTarget,-} from "./connection-target.js";--/** Bounds approval pins; an evicted pin rejects its call rather than letting it run. */-const MAX_APPROVAL_PINS = 50;--/**- * Connection instance each parked `connection_execute` call was sent for- * approval against, keyed by call id. When a person approves the call, it runs- * only if its connection still resolves to that instance; a changed instance or- * a missing pin (the map is bounded) rejects it.- */-const ConnectionApprovalPinsKey = new ContextKey<Readonly<Record<string, string>>>(- "eve.connectionApprovalPins",-);--/**- * Delegates approval to the called connection's policy. The request and- * response phases exist only when some registered connection defines them,- * so connections without a response policy keep the default response flow- * unless another connection in the same agent defines one.- */-export function connectionExecuteApproval(approvals: readonly Approval[]) {- if (approvals.length === 0) return {};- const request = defineDurableCallback({ callback: requestConnectionApproval, closure: {} });- const hasResponsePolicy = approvals.some(- (approval) => typeof approval !== "function" && approval.response !== undefined,- );- return {- approvalKey: defineDurableCallback({ callback: connectionApprovalKey, closure: {} }),- approval: hasResponsePolicy- ? {- request,- response: defineDurableCallback({- callback: authorizeConnectionApprovalResponse,- closure: {},- }),- }- : request,- };-}--/**- * The recorded approval identity. `<connection>__<tool>` is not unique (a- * connection `a` with tool `b__c` and a connection `a__b` with tool `c` share- * it), so approvals are keyed on the pair.- */-function connectionApprovalKey(_closure: object, input: unknown): string {- const target = readExecuteTarget(input);- return target === undefined ? CONNECTION_EXECUTE_TOOL_NAME : approvalKeyFor(target);-}--function approvalKeyFor(target: Pick<ExecuteTarget, "connection" | "tool">): string {- return JSON.stringify([target.connection, target.tool]);-}--/**- * Policies see the qualified tool name, so `approvedTools` lists it only when- * this exact connection and tool were approved, never a same-named pair.- */-function policyApprovedTools(- approvedTools: ReadonlySet<string>,- target: Pick<ExecuteTarget, "connection" | "tool">,-): ReadonlySet<string> {- const qualified = qualifiedToolName(target);- const view = new Set([...approvedTools].filter((key) => key !== qualified));- if (approvedTools.has(approvalKeyFor(target))) view.add(qualified);- return view;-}--async function requestConnectionApproval(- _closure: object,- context: ApprovalContext,-): Promise<ApprovalStatus> {- const target = readExecuteTarget(context.toolInput);- const registry = loadContext().get(ConnectionRegistryKey);- if (target === undefined || registry === undefined) return "not-applicable";- const connection = findConnection(registry, target.connection);- const approval =- connection === undefined- ? undefined- : registry.getConnectionApproval(connection.connectionName);- if (connection === undefined || approval === undefined) return "not-applicable";-- // Denying here would drop the call without a result, so the approved call- // runs and fails in execution instead, where the failure is reported.- if (isApprovalRecheck(context) && !matchesApprovalPin(context.callId, connection)) {- markApprovalPinChanged(context.callId);- }- const status = await resolveApprovalPolicy(approval)({- ...context,- approvedTools: policyApprovedTools(context.approvedTools, target),- toolInput: target.input,- toolName: qualifiedToolName(target),- });- if (!isApprovalRecheck(context) && parksForApproval(status)) {- pinApprovedInstance(context.callId, connection);- }- return status;-}--function parksForApproval(status: ApprovalStatus): boolean {- return (- status === true ||- status === "user-approval" ||- (typeof status === "object" && status.type === "user-approval")- );-}--async function authorizeConnectionApprovalResponse(- _closure: object,- context: ApprovalResponseContext,-): Promise<ApprovalResponseDecision> {- const target = readExecuteTarget(context.request.toolInput);- const registry = loadContext().get(ConnectionRegistryKey);- const connection =- target === undefined || registry === undefined- ? undefined- : findConnection(registry, target.connection);- if (target === undefined || registry === undefined || connection === undefined) {- return { reason: "The connection for this tool call is unavailable.", status: "rejected" };- }- if (!matchesApprovalPin(context.request.callId, connection)) {- return { reason: CONNECTION_CHANGED_MESSAGE, status: "rejected" };- }- const approval = registry.getConnectionApproval(connection.connectionName);- const response =- approval === undefined || typeof approval === "function" ? undefined : approval.response;- if (response === undefined) return { status: "allowed" };- return await response({- ...context,- request: {- ...context.request,- toolInput: target.input,- toolName: qualifiedToolName(target),- },- });-}--const CONNECTION_CHANGED_MESSAGE =- "The connection for this tool call changed or is unavailable. Request a new tool call and approval.";--/** Stands in for connections without an instance id, so every parked call has a pin. */-const UNKEYED_INSTANCE = "";--/** Records the instance a parked call was first sent for approval against. */-function pinApprovedInstance(callId: string, connection: ResolvedConnectionDefinition): void {- const instanceId = connection.instanceId ?? UNKEYED_INSTANCE;- const ctx = loadContext();- if (ctx.get(ConnectionApprovalPinsKey)?.[callId] !== undefined) return;- ctx.set(ConnectionApprovalPinsKey, (pins = {}) =>- Object.fromEntries([- ...Object.entries(pins).slice(-(MAX_APPROVAL_PINS - 1)),- [callId, instanceId],- ]),- );-}--/** Fails closed: a parked call whose pin is missing counts as changed. */-function matchesApprovalPin(callId: string, connection: ResolvedConnectionDefinition): boolean {- const pinned = loadContext().get(ConnectionApprovalPinsKey)?.[callId];- return pinned !== undefined && pinned === (connection.instanceId ?? UNKEYED_INSTANCE);-}--/** No instance id matches this, so the call fails when it runs. */-const CHANGED_INSTANCE = "\u0000changed";--function markApprovalPinChanged(callId: string): void {- loadContext().set(ConnectionApprovalPinsKey, (pins = {}) => ({- ...pins,- [callId]: CHANGED_INSTANCE,- }));-}--/** Consumes the call's approval pin, rejecting the call if its connection changed. */-export function releaseApprovalPin(callId: string, connection: ResolvedConnectionDefinition): void {- const ctx = loadContext();- const pins = ctx.get(ConnectionApprovalPinsKey);- if (pins?.[callId] === undefined) return;- const { [callId]: pinned, ...rest } = pins;- ctx.set(ConnectionApprovalPinsKey, rest);- if (pinned !== (connection.instanceId ?? UNKEYED_INSTANCE)) {- throw new Error(CONNECTION_CHANGED_MESSAGE);- }-}diff --git a/packages/eve/src/execution/tools/connection-search-rank.test.ts b/packages/eve/src/execution/tools/connection-search-rank.test.tsdeleted file mode 100644index 892dd8086..000000000--- a/packages/eve/src/execution/tools/connection-search-rank.test.ts+++ /dev/null@@ -1,63 +0,0 @@-import { describe, expect, it } from "vitest";--import type { ConnectionToolMetadata } from "#shared/connection-types.js";--import { closestToolNames, rankConnectionTools } from "./connection-search-rank.js";--function tool(- name: string,- description: string,- properties: Record<string, { readonly description?: string }> = {},-): ConnectionToolMetadata {- return { name, description, inputSchema: { type: "object", properties } };-}--const linear = { connectionName: "linear", description: "Issues, projects, and teams." };-const github = { connectionName: "github", description: "Repositories and pull requests." };-const kennel = { connectionName: "kennel", description: "Boarding pets and visits." };--const candidates = [- { connection: linear, tool: tool("list_issues", "List open issues.", { teamId: {} }) },- { connection: linear, tool: tool("get_team", "Get a team by id.") },- { connection: github, tool: tool("search_repositories", "Find repos by keyword.") },- {- connection: kennel,- tool: tool("book_visit", "Book a care visit.", {- contacts: { description: "People to notify about the visit." },- }),- },-];--const rank = (query: string) =>- rankConnectionTools(query, candidates).map(- ({ connection, tool }) => `${connection.connectionName}/${tool.name}`,- );--describe("rankConnectionTools", () => {- it.each([- { query: "linear", expected: ["linear/get_team", "linear/list_issues"] },- { query: "people", expected: ["kennel/book_visit"] },- // The tool named for issues outranks one matched only by its connection's description.- { query: "issues", expected: ["linear/list_issues", "linear/get_team"] },- { query: "repo", expected: ["github/search_repositories"] },- { query: "isue", expected: [] },- {- query: "",- expected: [- "github/search_repositories",- "kennel/book_visit",- "linear/get_team",- "linear/list_issues",- ],- },- ])("ranks $query", ({ query, expected }) => {- expect(rank(query)).toEqual(expected);- });-});--describe("closestToolNames", () => {- it("suggests from the tools' own fields, not the shared connection", () => {- const tools = [tool("find_pet", "Look up a pet."), tool("list_feedings", "Feeding schedule.")];- expect(closestToolNames("kennel_find_pets", tools, 5)).toEqual(["find_pet"]);- });-});diff --git a/packages/eve/src/execution/tools/connection-search-rank.ts b/packages/eve/src/execution/tools/connection-search-rank.tsdeleted file mode 100644index f8a3410b0..000000000--- a/packages/eve/src/execution/tools/connection-search-rank.ts+++ /dev/null@@ -1,105 +0,0 @@-/**- * Keyword ranking for `connection_search` and for `connection_execute`'s- * suggestions when a tool name is unknown. Query and field text split into- * lowercase words (camelCase and punctuation are boundaries); a query word- * matches a field word when either is a prefix of the other, so `repo` finds- * `repositories` and `issues` finds `issue`. Each query word adds the weight of- * every field it matches, so broad fields rank below a tool's own name.- */--import type { ResolvedConnectionDefinition } from "#runtime/types.js";-import type { ConnectionToolMetadata } from "#shared/connection-types.js";-import { isObject } from "#shared/guards.js";--export interface RankCandidate {- readonly connection: Pick<ResolvedConnectionDefinition, "connectionName" | "description">;- readonly tool: ConnectionToolMetadata;-}--/** Candidates matching `query`, best first. An empty query matches every candidate. */-export function rankConnectionTools<T extends RankCandidate>(- query: string,- candidates: readonly T[],-): T[] {- const terms = tokenize(query);- return candidates- .map((candidate) => ({- candidate,- score:- terms.length === 0- ? 1- : scoreFields(terms, [- ...toolFields(candidate.tool),- [tokenize(candidate.connection.connectionName), 4],- [tokenize(candidate.connection.description), 1],- ]),- }))- .filter((entry) => entry.score > 0)- .sort(- (a, b) =>- b.score - a.score ||- a.candidate.connection.connectionName.localeCompare(- b.candidate.connection.connectionName,- ) ||- a.candidate.tool.name.localeCompare(b.candidate.tool.name),- )- .map((entry) => entry.candidate);-}--/**- * Names of one connection's tools closest to an unknown `name`. Only the tools'- * own fields count, since every candidate shares the connection.- */-export function closestToolNames(- name: string,- tools: readonly ConnectionToolMetadata[],- limit: number,-): string[] {- const terms = tokenize(name);- if (terms.length === 0) return [];- return tools- .map((tool) => ({ name: tool.name, score: scoreFields(terms, toolFields(tool)) }))- .filter((entry) => entry.score > 0)- .sort((a, b) => b.score - a.score || a.name.localeCompare(b.name))- .slice(0, limit)- .map((entry) => entry.name);-}--type Field = readonly [words: readonly string[], weight: number];--function toolFields(tool: ConnectionToolMetadata): Field[] {- const properties = isObject(tool.inputSchema.properties)- ? Object.entries(tool.inputSchema.properties)- : [];- const propertyDescriptions = properties.flatMap(([, schema]) =>- isObject(schema) && typeof schema.description === "string" ? [schema.description] : [],- );- return [- [tokenize(tool.name), 6],- [properties.flatMap(([key]) => tokenize(key)), 3],- [tokenize(tool.description), 2],- [propertyDescriptions.flatMap(tokenize), 1],- ];-}--function scoreFields(terms: readonly string[], fields: readonly Field[]): number {- let score = 0;- for (const term of terms) {- for (const [words, weight] of fields) {- if (words.some((word) => matchesWord(word, term))) score += weight;- }- }- return score;-}--function matchesWord(word: string, term: string): boolean {- return word.startsWith(term) || (word.length >= 3 && term.startsWith(word));-}--function tokenize(text: string): string[] {- return text- .replaceAll(/([a-z0-9])([A-Z])/gu, "$1 $2")- .toLowerCase()- .split(/[^a-z0-9]+/u)- .filter((word) => word.length > 1);-}diff --git a/packages/eve/src/execution/tools/connection-target.ts b/packages/eve/src/execution/tools/connection-target.tsdeleted file mode 100644index aaadcdc45..000000000--- a/packages/eve/src/execution/tools/connection-target.ts+++ /dev/null@@ -1,43 +0,0 @@-/** The `connection_execute` call target, shared by execution and approval. */--import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";-import type { ResolvedConnectionDefinition } from "#runtime/types.js";-import { displayProperName, displayTitle } from "#shared/display-name.js";-import { isObject } from "#shared/guards.js";-import type { JsonObject } from "#shared/json.js";--export const CONNECTION_EXECUTE_TOOL_NAME = "connection_execute";--export interface ExecuteTarget {- readonly connection: string;- readonly input: JsonObject;- readonly tool: string;-}--export function readExecuteTarget(value: unknown): ExecuteTarget | undefined {- if (!isObject(value)) return undefined;- const { connection, input, tool } = value;- if (typeof connection !== "string" || typeof tool !== "string") return undefined;- return { connection, input: isObject(input) ? (input as JsonObject) : {}, tool };-}--export function qualifiedToolName(target: Pick<ExecuteTarget, "connection" | "tool">): string {- return `${target.connection}__${target.tool}`;-}--/**- * How a tool call reads to a person: `Linear: List issues` for a- * `connection_execute` call, otherwise the tool's display title.- */-export function toolCallDisplayName(toolName: string, input: unknown): string {- const target = toolName === CONNECTION_EXECUTE_TOOL_NAME ? readExecuteTarget(input) : undefined;- if (target === undefined) return displayTitle(toolName);- return `${displayProperName(target.connection)}: ${displayTitle(target.tool)}`;-}--export function findConnection(- registry: ConnectionRegistry,- name: string,-): ResolvedConnectionDefinition | undefined {- return registry.getConnections().find((connection) => connection.connectionName === name);-}diff --git a/packages/eve/src/execution/tools/connection-tools.lazy-auth.test.ts b/packages/eve/src/execution/tools/connection-tools.lazy-auth.test.tsdeleted file mode 100644index 54ddaf4ec..000000000--- a/packages/eve/src/execution/tools/connection-tools.lazy-auth.test.ts+++ /dev/null@@ -1,241 +0,0 @@-import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";--import { ConnectionAuthorizationRequiredError } from "#connections/errors.js";-import { ContextContainer, contextStorage } from "#context/container.js";-import { AuthKey, SessionIdKey } from "#context/keys.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import {- CallbackBaseUrlKey,- isAuthorizationSignal,- PendingAuthorizationResultKey,-} from "#harness/authorization.js";-import { McpConnectionClient } from "#runtime/connections/mcp-client.js";-import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";-import type { ResolvedConnectionDefinition } from "#runtime/types.js";-import { type ConnectionClient, defineInteractiveAuthorization } from "#shared/connection-types.js";--import { CONNECTION_EXECUTE_TOOL_NAME } from "./connection-target.js";-import { CONNECTION_SEARCH_TOOL_NAME, resolveConnectionTools } from "./connection-tools.js";--const URL = "https://kennel.example.com/mcp";-const TOKEN = "signed-in-token";--/** How the fake server treats requests that carry no bearer. */-type ServerMode =- /** `initialize`/`tools/list` are public; a protected `tools/call` answers HTTP 401. */- | "public-list-401"- /** `initialize`/`tools/list` are public; a protected `tools/call` returns an `mcp/www_authenticate` error. */- | "public-list-meta"- /** Every request without a bearer answers HTTP 401. */- | "token-everywhere";--interface SeenRequest {- readonly authorization: string | null;- readonly method: string;- readonly tool?: string;-}--function setup(mode: ServerMode) {- const seen: SeenRequest[] = [];- vi.stubGlobal("fetch", async (input: string | globalThis.URL | Request, init?: RequestInit) => {- const request = new Request(input, init);- if (request.method !== "POST") return new Response(null, { status: 405 });- const authorization = request.headers.get("authorization");- const message = (await request.json()) as {- id?: number;- method: string;- params?: { name?: string };- };- seen.push({ authorization, method: message.method, tool: message.params?.name });- const signedIn = authorization === `Bearer ${TOKEN}`;- if (!signedIn && mode === "token-everywhere") {- return new Response("Unauthorized", { status: 401 });- }- if (message.id === undefined) return new Response(null, { status: 202 });- const reply = (result: unknown) => Response.json({ jsonrpc: "2.0", id: message.id, result });- switch (message.method) {- case "initialize":- return reply({- capabilities: { tools: {} },- protocolVersion: "2025-06-18",- serverInfo: { name: "kennel", version: "1.0.0" },- });- case "tools/list":- return reply({- tools: [- { name: "find_pet", description: "Find a pet.", inputSchema: { type: "object" } },- { name: "pet_invoice", description: "A pet's bill.", inputSchema: { type: "object" } },- ],- });- case "tools/call": {- if (message.params?.name === "pet_invoice" && !signedIn) {- if (mode === "public-list-401") return new Response("Unauthorized", { status: 401 });- return reply({- _meta: { "mcp/www_authenticate": [`Bearer resource_metadata="${URL}/.well-known"`] },- content: [{ type: "text", text: "Sign in to see invoices." }],- isError: true,- });- }- return reply({ content: [{ type: "text", text: `${message.params?.name} ok` }] });- }- default:- return Response.json({- jsonrpc: "2.0",- id: message.id,- error: { code: -32601, message: "Method not found" },- });- }- });-- let token: string | undefined;- const startAuthorization = vi.fn(async ({ callbackUrl }: { callbackUrl: string }) => ({- challenge: { url: `https://idp.example.com/authorize?redirect=${callbackUrl}` },- }));- const authorization = defineInteractiveAuthorization({- getToken: async () => {- if (token === undefined) throw new ConnectionAuthorizationRequiredError("kennel");- return { token };- },- startAuthorization,- completeAuthorization: async () => {- token = TOKEN;- return { token };- },- });- const connection = {- authorization,- connectionName: "kennel",- description: "Kennel",- protocol: "mcp",- protocolVersionDiscovery: false,- url: URL,- } as ResolvedConnectionDefinition;-- const clients = new Map<string, ConnectionClient>();- const registry: ConnectionRegistry = {- dispose: async () => {- await Promise.all([...clients.values()].map((client) => client.close()));- },- getClient: (name) => {- let client = clients.get(name);- if (client === undefined) {- client = new McpConnectionClient(connection);- clients.set(name, client);- }- return client;- },- getConnectionApproval: () => undefined,- getConnectionNames: () => ["kennel"],- getConnections: () => [connection],- };-- const ctx = new ContextContainer();- ctx.set(AuthKey, {- attributes: {},- authenticator: "test",- principalId: "alice",- principalType: "user",- });- ctx.set(SessionIdKey, "lazy-auth");- ctx.set(CallbackBaseUrlKey, "https://agent.example.com");- ctx.set(ConnectionRegistryKey, registry);-- async function call(toolName: string, input: unknown): Promise<unknown> {- return await contextStorage.run(ctx, async () => {- const execute = resolveConnectionTools()?.[toolName]?.execute;- if (execute === undefined) throw new Error(`${toolName} is not defined.`);- return await execute(input, { callId: "call-1", messages: [] } as never);- });- }-- /** Simulates the user finishing sign-in: the callback result arrives for the next step. */- function completeSignIn() {- ctx.set(PendingAuthorizationResultKey, [- {- callback: { method: "GET", params: { code: "ok" } },- hookUrl: "https://agent.example.com/callback",- name: "kennel",- },- ]);- }-- return { call, completeSignIn, registry, seen, startAuthorization };-}--describe("MCP connections sign in only when the server asks", () => {- let registry: ConnectionRegistry | undefined;-- beforeEach(() => {- registry = undefined;- });-- afterEach(async () => {- await registry?.dispose();- vi.unstubAllGlobals();- });-- it("lists a public server's tools without a token or a sign-in prompt", async () => {- const env = setup("public-list-401");- registry = env.registry;-- const result = await env.call(CONNECTION_SEARCH_TOOL_NAME, { connection: "kennel" });-- expect(result).toMatchObject({- tools: [{ tool: "find_pet" }, { tool: "pet_invoice" }],- total: 2,- });- expect(result).not.toHaveProperty("unavailable");- expect(env.startAuthorization).not.toHaveBeenCalled();- expect(env.seen.every((request) => request.authorization === null)).toBe(true);- });-- it.each(["public-list-401", "public-list-meta"] as const)(- "%s: runs a public tool anonymously, then prompts on a protected tool and resumes with the token",- async (mode) => {- const env = setup(mode);- registry = env.registry;-- await expect(- env.call(CONNECTION_EXECUTE_TOOL_NAME, { connection: "kennel", tool: "find_pet" }),- ).resolves.toBe("find_pet ok");- expect(env.startAuthorization).not.toHaveBeenCalled();-- const signal = await env.call(CONNECTION_EXECUTE_TOOL_NAME, {- connection: "kennel",- tool: "pet_invoice",- });- expect(isAuthorizationSignal(signal)).toBe(true);- expect(signal).toMatchObject({ challenges: [{ name: "kennel" }] });- expect(env.startAuthorization).toHaveBeenCalledOnce();-- env.completeSignIn();- const seenBefore = env.seen.length;- await expect(- env.call(CONNECTION_EXECUTE_TOOL_NAME, { connection: "kennel", tool: "pet_invoice" }),- ).resolves.toBe("pet_invoice ok");- const resumed = env.seen.slice(seenBefore);- expect(resumed.map((request) => request.method)).toContain("initialize");- expect(resumed.every((request) => request.authorization === `Bearer ${TOKEN}`)).toBe(true);- expect(env.startAuthorization).toHaveBeenCalledOnce();- },- );-- it("keeps reporting requiresSignIn for servers that reject every anonymous request", async () => {- const env = setup("token-everywhere");- registry = env.registry;-- const result = await env.call(CONNECTION_SEARCH_TOOL_NAME, { connection: "kennel" });-- expect(result).toMatchObject({- tools: [],- unavailable: [{ connection: "kennel", requiresSignIn: true }],- });- expect(env.startAuthorization).not.toHaveBeenCalled();-- const signal = await env.call(CONNECTION_EXECUTE_TOOL_NAME, {- connection: "kennel",- tool: "find_pet",- });- expect(signal).toMatchObject({ challenges: [{ name: "kennel" }] });- });-});diff --git a/packages/eve/src/execution/tools/connection-tools.test.ts b/packages/eve/src/execution/tools/connection-tools.test.tsdeleted file mode 100644index dca4badd7..000000000--- a/packages/eve/src/execution/tools/connection-tools.test.ts+++ /dev/null@@ -1,156 +0,0 @@-import { describe, expect, it, vi } from "vitest";--import { ConnectionAuthorizationRequiredError } from "#connections/errors.js";-import { ContextContainer, contextStorage } from "#context/container.js";-import { AuthKey, SessionIdKey } from "#context/keys.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import {- CallbackBaseUrlKey,- isAuthorizationSignal,- PendingAuthorizationResultKey,-} from "#harness/authorization.js";-import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";-import type { ResolvedConnectionDefinition } from "#runtime/types.js";-import {- type ConnectionClient,- type ConnectionToolMetadata,- defineInteractiveAuthorization,-} from "#shared/connection-types.js";--import { CONNECTION_EXECUTE_TOOL_NAME } from "./connection-target.js";-import { CONNECTION_SEARCH_TOOL_NAME, resolveConnectionTools } from "./connection-tools.js";--const listIssues: ConnectionToolMetadata = {- description: "List open issues.",- inputSchema: { type: "object", properties: {} },- name: "list_issues",-};--function setup() {- const startAuthorization = vi.fn(async ({ callbackUrl }: { callbackUrl: string }) => ({- challenge: { url: `https://idp.example.com/authorize?redirect=${callbackUrl}` },- }));- const authorization = {- ...defineInteractiveAuthorization({- getToken: async () => {- throw new ConnectionAuthorizationRequiredError("unused");- },- startAuthorization,- completeAuthorization: async () => ({ token: "token" }),- }),- vercelConnect: { connector: "workspace/agent" },- };- const connections = ["github", "linear", "notion"].map(- (connectionName) =>- ({- authorization: connectionName === "github" ? undefined : authorization,- connectionName,- description: connectionName,- protocol: "mcp",- url: `https://${connectionName}.example.com/mcp`,- }) as ResolvedConnectionDefinition,- );- const client = (connectionName: string): ConnectionClient => ({- close: async () => {},- connect: async () => {},- executeTool: async () => ({ content: [] }),- getToolMetadata: async () => {- if (connectionName === "github") return [listIssues];- throw new ConnectionAuthorizationRequiredError(connectionName);- },- });- const registry: ConnectionRegistry = {- dispose: async () => {},- getClient: client,- getConnectionApproval: () => undefined,- getConnectionNames: () => connections.map((connection) => connection.connectionName),- getConnections: () => connections,- };-- const ctx = new ContextContainer();- ctx.set(AuthKey, {- attributes: {},- authenticator: "test",- principalId: "alice",- principalType: "user",- });- ctx.set(SessionIdKey, "connection-tools");- ctx.set(CallbackBaseUrlKey, "https://agent.example.com");- ctx.set(ConnectionRegistryKey, registry);-- async function call(toolName: string, input: unknown): Promise<unknown> {- return await contextStorage.run(ctx, async () => {- const execute = resolveConnectionTools()?.[toolName]?.execute;- if (execute === undefined) throw new Error(`${toolName} is not defined.`);- return await execute(input, { callId: "call-1", messages: [] } as never);- });- }-- return { call, ctx, startAuthorization };-}--describe("connection tools authorization", () => {- it("prompts only for one named connection with signIn, never for a plain search", async () => {- const { call, startAuthorization } = setup();- const signIn = (connection: string) => ({- connection,- error: `Sign-in required: the user has not signed in to "${connection}", so its tools cannot be listed. If the request needs "${connection}", call connection_search with connection "${connection}" and signIn: true to ask the user to sign in.`,- requiresSignIn: true,- });-- const unscoped = await call(CONNECTION_SEARCH_TOOL_NAME, { query: "open issues" });- expect(isAuthorizationSignal(unscoped)).toBe(false);- expect(unscoped).toMatchObject({- tools: [{ connection: "github", tool: "list_issues" }],- unavailable: [signIn("linear"), signIn("notion")],- });- const scoped = await call(CONNECTION_SEARCH_TOOL_NAME, { connection: "linear" });- expect(scoped).toEqual({ tools: [], total: 0, unavailable: [signIn("linear")] });- await expect(call(CONNECTION_SEARCH_TOOL_NAME, { signIn: true })).rejects.toThrow(- "connection_search with signIn: true requires `connection`.",- );- const signedIn = await call(CONNECTION_SEARCH_TOOL_NAME, {- connection: "github",- query: "issues",- signIn: true,- });- expect(signedIn).toMatchObject({ tools: [{ tool: "list_issues" }], total: 1 });- expect(startAuthorization).not.toHaveBeenCalled();-- const requested = await call(CONNECTION_SEARCH_TOOL_NAME, {- connection: "linear",- query: "issues",- signIn: true,- });- expect(requested).toMatchObject({- challenges: [- { grant: "workspace/agent", name: "linear", requester: { principalId: "alice" } },- ],- });- expect(startAuthorization).toHaveBeenCalledOnce();-- const executed = await call(CONNECTION_EXECUTE_TOOL_NAME, {- connection: "notion",- tool: "list_issues",- });- expect(executed).toMatchObject({ challenges: [{ name: "notion" }] });- expect(startAuthorization).toHaveBeenCalledTimes(2);- });-- it("reports a token refused right after sign-in instead of asking again", async () => {- const { call, ctx, startAuthorization } = setup();- // The user finished signing in to linear, but the service still refuses the token.- ctx.set(PendingAuthorizationResultKey, [- {- callback: { method: "GET", params: { code: "ok" } },- hookUrl: "https://agent.example.com/callback",- name: "linear",- },- ]);-- await expect(call(CONNECTION_SEARCH_TOOL_NAME, { connection: "linear" })).rejects.toThrow(- 'Authorization failed for "linear": the service rejected the token immediately after authorization.',- );- expect(startAuthorization).not.toHaveBeenCalled();- });-});diff --git a/packages/eve/src/execution/tools/connection-tools.ts b/packages/eve/src/execution/tools/connection-tools.tsdeleted file mode 100644index 6a02a5785..000000000--- a/packages/eve/src/execution/tools/connection-tools.ts+++ /dev/null@@ -1,527 +0,0 @@-import { connectionToolStub } from "#tool-stubs/execute.js";-import { toolStubOutput } from "#tool-stubs/output.js";-/**- * `connection_search` and `connection_execute`: fixed tools that reach every- * connection tool without adding definitions to the model's tool list, so- * discovery never changes the cached prompt prefix. Both are rebuilt each- * step from the connection registry with identical model-facing definitions.- */--import type { Approval } from "#approval/definition.js";-import {- isConnectionAuthorizationFailedError,- isConnectionAuthorizationRequiredError,-} from "#connections/errors.js";-import { loadContext } from "#context/container.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { getAuthorizationResults, type AuthorizationSignal } from "#harness/authorization.js";-import { reportNestedToolAction } from "#harness/nested-actions.js";-import { createLogger } from "#internal/logging.js";-import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";-import { resolveConnectionAuthorization } from "#runtime/connections/resolve-authorization.js";-import {- createAuthorizationExecution,- type ScopedAuthorization,-} from "#runtime/connections/scoped-authorization.js";-import {- connectionToolModelOutput,- toConnectionToolResult,-} from "#runtime/connections/tool-result.js";-import { renderToolSignature } from "#runtime/connections/tool-signature.js";-import type { ResolvedConnectionDefinition } from "#runtime/types.js";-import {- supportsInteractiveAuthorization,- type ConnectionToolMetadata,-} from "#shared/connection-types.js";-import { displayProperName } from "#shared/display-name.js";-import { toErrorMessage } from "#shared/errors.js";-import { isObject } from "#shared/guards.js";-import type { JsonObject } from "#shared/json.js";-import { defineDurableCallback } from "#tools/durable-callbacks.js";-import { defineTool, type ToolContext } from "#tools/definition.js";-import type { DynamicToolSet } from "#tools/dynamic.js";-import { defineJsonSchema } from "#tools/schema.js";--import { connectionExecuteApproval, releaseApprovalPin } from "./connection-approval.js";-import {- closestToolNames,- rankConnectionTools,- type RankCandidate,-} from "./connection-search-rank.js";-import {- CONNECTION_EXECUTE_TOOL_NAME,- findConnection,- qualifiedToolName,- readExecuteTarget,- toolCallDisplayName,-} from "./connection-target.js";--const log = createLogger("framework.connection-tools");--export const CONNECTION_SEARCH_TOOL_NAME = "connection_search";--const DEFAULT_SEARCH_LIMIT = 10;-const MAX_SEARCH_LIMIT = 50;-const MAX_SUGGESTIONS = 5;--const CONNECTION_SEARCH_DESCRIPTION = [- "Find tools in your connected services (MCP servers and OpenAPI APIs).",- "Returns each matching tool's connection, name, description, and TypeScript signature.",- "Omit `query` to list a connection's tools. Call a found tool with connection_execute.",- "Connections the user has not signed in to are listed under `unavailable` with `requiresSignIn`;",- "when the request needs one, search it again with `connection` and `signIn: true` to ask the user to sign in.",- "Prefer connected services over web search or general knowledge when a request relates to them.",-].join(" ");--const CONNECTION_EXECUTE_DESCRIPTION = [- "Call one tool from a connected service and return its result.",- "Use the exact `connection` and `tool` names returned by connection_search,",- "and pass `input` matching the tool's signature.",-].join(" ");--const CONNECTION_SEARCH_INPUT_SCHEMA: JsonObject = {- type: "object",- properties: {- query: {- type: "string",- description:- "Words describing the capability, such as 'list open issues'. Omit to list every tool.",- },- connection: { type: "string", description: "Only search this connection." },- signIn: {- type: "boolean",- description:- "Ask the user to sign in to `connection` first, then search it. Requires `connection`. Use only for a connection listed with `requiresSignIn` that the request needs.",- },- limit: {- type: "integer",- minimum: 1,- maximum: MAX_SEARCH_LIMIT,- description: `Maximum results. Defaults to ${DEFAULT_SEARCH_LIMIT}.`,- },- offset: { type: "integer", minimum: 0, description: "Results to skip, for paging." },- },- additionalProperties: false,-};--const CONNECTION_EXECUTE_INPUT_SCHEMA: JsonObject = {- type: "object",- properties: {- connection: { type: "string", description: "Connection name from connection_search." },- tool: { type: "string", description: "Tool name from connection_search." },- input: {- type: "object",- description: "Arguments matching the tool's signature. Defaults to {}.",- },- },- required: ["connection", "tool"],- additionalProperties: false,-};--interface ConnectionSearchInput {- readonly connection?: string;- readonly limit?: number;- readonly offset?: number;- readonly query?: string;- readonly signIn?: boolean;-}--interface ConnectionSearchMatch {- readonly connection: string;- readonly description: string;- readonly signature: string;- readonly tool: string;-}--interface UnavailableConnection {- readonly connection: string;- readonly error: string;- /** Present when `signIn: true` can make the connection available. */- readonly requiresSignIn?: true;- /** Whether the failure is final rather than waiting on authorization. */- readonly terminal: boolean;-}--interface ConnectionSearchOutput {- readonly tools: readonly ConnectionSearchMatch[];- /** Matches across all pages. */- readonly total: number;- readonly unavailable?: readonly Omit<UnavailableConnection, "terminal">[];-}--// ----------------------------------------------------------------------------// Tool definitions-// -----------------------------------------------------------------------------/** Builds both tools for the current step, or `null` when the agent has no connections. */-export function resolveConnectionTools(): DynamicToolSet | null {- const registry = loadContext().get(ConnectionRegistryKey);- if (registry === undefined) return null;-- const approvals = registry- .getConnections()- .map((connection) => registry.getConnectionApproval(connection.connectionName))- .filter((approval): approval is Approval => approval !== undefined);-- return {- [CONNECTION_SEARCH_TOOL_NAME]: frameworkTool(- defineTool({- description: CONNECTION_SEARCH_DESCRIPTION,- inputSchema: CONNECTION_SEARCH_INPUT_SCHEMA,- execute: defineDurableCallback({ callback: searchConnectionTools, closure: {} }),- label: {- start: defineDurableCallback({ callback: connectionSearchLabel, closure: {} }),- },- }),- ),- [CONNECTION_EXECUTE_TOOL_NAME]: frameworkTool(- defineTool({- description: CONNECTION_EXECUTE_DESCRIPTION,- inputSchema: CONNECTION_EXECUTE_INPUT_SCHEMA,- execute: defineDurableCallback({ callback: executeConnectionTool, closure: {} }),- label: {- start: defineDurableCallback({ callback: connectionToolLabel, closure: {} }),- },- toModelOutput: defineDurableCallback({- callback: (_closure: object, output: unknown) => connectionToolModelOutput(output),- closure: {},- }),- ...connectionExecuteApproval(approvals),- }),- ),- };-}--function connectionToolLabel(_closure: object, input: unknown): string {- return toolCallDisplayName(CONNECTION_EXECUTE_TOOL_NAME, input);-}--function connectionSearchLabel(_closure: object, input: unknown): string {- const connection = isObject(input) ? input.connection : undefined;- if (typeof connection !== "string" || connection === "") return "Search connected tools";- return isObject(input) && input.signIn === true- ? `Connect ${displayProperName(connection)}`- : `Search ${displayProperName(connection)} tools`;-}--// ----------------------------------------------------------------------------// connection_search-// -----------------------------------------------------------------------------async function searchConnectionTools(- _closure: object,- rawInput: unknown,-): Promise<ConnectionSearchOutput | AuthorizationSignal> {- const input = (isObject(rawInput) ? rawInput : {}) as ConnectionSearchInput;- const registry = requireRegistry();- const connectionName =- input.connection === undefined || input.connection === "" ? undefined : input.connection;- if (input.signIn === true && connectionName === undefined) {- throw new Error(- "connection_search with signIn: true requires `connection`. Ask the user to sign in to one connection at a time.",- );- }- const targets =- connectionName === undefined- ? registry.getConnections()- : [requireConnection(registry, connectionName)];-- // Finishing a sign-in the user already completed never prompts. Starting one- // is reserved for `signIn: true` on one named connection and connection_execute.- const auth = createAuthorizationExecution();- await completePendingAuthorizations(registry, targets, auth);-- const unavailable: UnavailableConnection[] = [];- const candidates: RankCandidate[] = [];- for (const connection of targets) {- const listed = await listConnectionTools(registry, connection, auth, input.signIn === true);- if ("signIn" in listed) return listed.signIn;- if ("unavailable" in listed) unavailable.push(listed.unavailable);- else {- for (const tool of listed.tools) candidates.push({ connection, tool });- }- }-- const terminal = unavailable.filter((entry) => entry.terminal);- if (targets.length > 0 && terminal.length === targets.length) {- throw new Error(terminal.map((entry) => entry.error).join("\n"));- }-- const ranked = rankConnectionTools(input.query ?? "", candidates);- const limit = clampInteger(input.limit, 1, MAX_SEARCH_LIMIT, DEFAULT_SEARCH_LIMIT);- const offset = clampInteger(input.offset, 0, Number.MAX_SAFE_INTEGER, 0);-- const output: { -readonly [K in keyof ConnectionSearchOutput]: ConnectionSearchOutput[K] } = {- tools: ranked.slice(offset, offset + limit).map(({ connection, tool }) => ({- connection: connection.connectionName,- description: tool.description,- signature: renderToolSignature({- inputSchema: tool.inputSchema,- name: tool.name,- outputSchema: tool.outputSchema,- }),- tool: tool.name,- })),- total: ranked.length,- };- if (unavailable.length > 0) {- output.unavailable = unavailable.map(({ terminal: _terminal, ...entry }) => entry);- }- return output;-}--type ListedConnectionTools =- | { readonly tools: readonly ConnectionToolMetadata[] }- | { readonly signIn: AuthorizationSignal }- | { readonly unavailable: UnavailableConnection };--/** Lists a connection's tools, starting its sign-in instead when `signIn` asks for it. */-async function listConnectionTools(- registry: ConnectionRegistry,- connection: ResolvedConnectionDefinition,- auth: ReturnType<typeof createAuthorizationExecution>,- signIn: boolean,-): Promise<ListedConnectionTools> {- const name = connection.connectionName;- try {- return { tools: await registry.getClient(name).getToolMetadata() };- } catch (error) {- if (isConnectionAuthorizationRequiredError(error)) {- const scoped = await resolveInteractiveAuthorization(registry, name);- // The token the user just signed in with was refused. Asking again would- // loop, so report the failure instead.- if (scoped !== undefined && auth.isJustAuthorized(scoped)) {- return {- unavailable: {- connection: name,- error: `Authorization failed for "${name}": the service rejected the token immediately after authorization.`,- terminal: true,- },- };- }- if (scoped === undefined) {- const cannotSignIn = `"${name}" requires authorization and cannot start interactive sign-in.`;- if (signIn) throw new Error(cannotSignIn);- return { unavailable: { connection: name, error: cannotSignIn, terminal: false } };- }- if (signIn) return { signIn: await auth.handleError(error, scoped) };- return {- unavailable: {- connection: name,- error:- `Sign-in required: the user has not signed in to "${name}", so its tools cannot be listed. ` +- `If the request needs "${name}", call connection_search with connection "${name}" and signIn: true to ask the user to sign in.`,- requiresSignIn: true,- terminal: false,- },- };- }- log.warn("failed to load connection tools", { connection: name, error });- return {- unavailable: {- connection: name,- error: isConnectionAuthorizationFailedError(error)- ? `Authorization failed for "${name}": ${error.message}`- : `Failed to load tools for "${name}": ${toErrorMessage(error)}`,- terminal: true,- },- };- }-}--function clampInteger(value: unknown, min: number, max: number, fallback: number): number {- if (typeof value !== "number" || !Number.isFinite(value)) return fallback;- return Math.min(max, Math.max(min, Math.trunc(value)));-}--// ----------------------------------------------------------------------------// connection_execute-// -----------------------------------------------------------------------------async function executeConnectionTool(- _closure: object,- rawInput: unknown,- ctx: ToolContext,-): Promise<unknown> {- const target = readExecuteTarget(rawInput);- if (target === undefined) {- throw new Error("connection_execute requires string `connection` and `tool` values.");- }- const registry = requireRegistry();- const connection = requireConnection(registry, target.connection);- releaseApprovalPin(ctx.callId, connection);- assertPendingAuthorizationInstances(registry, [connection]);-- const scoped = await resolveInteractiveAuthorization(registry, connection.connectionName);- const auth = createAuthorizationExecution();- if (scoped !== undefined) await auth.complete(scoped);- const client = registry.getClient(connection.connectionName);- // A client that connected anonymously before sign-in must reconnect with the new token.- if (scoped !== undefined && auth.isJustAuthorized(scoped)) await client.close();-- let tools: readonly ConnectionToolMetadata[];- try {- tools = await client.getToolMetadata();- } catch (error) {- return await auth.handleError(error, scoped);- }- const tool = tools.find((entry) => entry.name === target.tool);- if (tool === undefined) throw new Error(unknownToolMessage(connection, target.tool, tools));- const input = await validToolInput(connection, tool, target.input);-- const toolName = qualifiedToolName(target);- const stub = await connectionToolStub(toolName, input, ctx.callId);- if (stub.kind === "error") throw new Error(stub.error);- let raw: unknown;- try {- if (stub.kind === "stub") {- const output = toolStubOutput(stub.outcome);- reportNestedToolAction(ctx.callId, { input, output, toolName });- return output;- }- raw = await client.executeTool(tool.name, input, {- abortSignal: ctx.abortSignal,- callId: ctx.callId,- });- } catch (error) {- if (stub.kind === "real" && isConnectionAuthorizationRequiredError(error))- return await auth.handleError(error, scoped);- reportNestedToolAction(ctx.callId, {- input,- isError: true,- output: toErrorMessage(error),- toolName,- });- throw error;- }-- const result = toConnectionToolResult(connection.protocol, tool, raw);- if (!result.ok) {- reportNestedToolAction(ctx.callId, {- input,- isError: true,- output: result.error,- toolName,- });- throw new Error(result.error);- }- reportNestedToolAction(ctx.callId, { input, output: result.value, toolName });- return result.value;-}--/** Validates `input` against the tool's schema, returning it with schema defaults filled in. */-async function validToolInput(- connection: ResolvedConnectionDefinition,- tool: ConnectionToolMetadata,- input: JsonObject,-): Promise<JsonObject> {- const result = await defineJsonSchema(tool.inputSchema as JsonObject)["~standard"].validate(- input,- );- if (result.issues === undefined) return result.value as JsonObject;- const issues = result.issues- .map((issue) => {- const path = (issue.path ?? [])- .map((segment) => (typeof segment === "object" ? String(segment.key) : String(segment)))- .join(".");- return path.length > 0 ? `${path}: ${issue.message}` : issue.message;- })- .join("; ");- throw new Error(- `Invalid input for "${tool.name}" on connection "${connection.connectionName}": ${issues}. ` +- `Signature: ${renderToolSignature({ inputSchema: tool.inputSchema, name: tool.name, outputSchema: tool.outputSchema })}`,- );-}--function unknownToolMessage(- connection: ResolvedConnectionDefinition,- toolName: string,- tools: readonly ConnectionToolMetadata[],-): string {- const suggestions = closestToolNames(toolName, tools, MAX_SUGGESTIONS);- const hint =- suggestions.length > 0- ? ` Closest tools: ${suggestions.join(", ")}.`- : " Use connection_search to find its tools.";- return `Connection "${connection.connectionName}" has no tool named "${toolName}".${hint}`;-}--// ----------------------------------------------------------------------------// Authorization-// -----------------------------------------------------------------------------async function resolveInteractiveAuthorization(- registry: ConnectionRegistry,- connectionName: string,-): Promise<ScopedAuthorization | undefined> {- const connection = findConnection(registry, connectionName);- if (connection === undefined) return undefined;- const authorization = await resolveConnectionAuthorization(connection);- if (authorization === undefined || !supportsInteractiveAuthorization(authorization)) {- return undefined;- }- return {- scope: connection.connectionName,- instanceId: connection.instanceId,- connection: { url: connection.url ?? "" },- authorization,- };-}--/** Completes sign-in callbacks for the targeted connections only. */-async function completePendingAuthorizations(- registry: ConnectionRegistry,- connections: readonly ResolvedConnectionDefinition[],- auth: ReturnType<typeof createAuthorizationExecution>,-): Promise<void> {- assertPendingAuthorizationInstances(registry, connections);- const results = getAuthorizationResults();- for (const connection of connections) {- if (!results.some((result) => result.name === connection.connectionName)) continue;- const scoped = await resolveInteractiveAuthorization(registry, connection.connectionName);- if (scoped === undefined) continue;- await auth.complete(scoped);- // A client that connected anonymously before sign-in must reconnect with the new token.- if (auth.isJustAuthorized(scoped)) await registry.getClient(connection.connectionName).close();- }-}--/** Rejects sign-in results for the targeted connections whose instance changed while pending. */-function assertPendingAuthorizationInstances(- registry: ConnectionRegistry,- connections: readonly ResolvedConnectionDefinition[],-): void {- const targeted = new Set(connections.map((connection) => connection.connectionName));- for (const result of getAuthorizationResults()) {- if (result.instanceId === undefined || !targeted.has(result.name)) continue;- if (findConnection(registry, result.name)?.instanceId === result.instanceId) continue;- throw new Error(- `Authorization for "${result.name}" cannot complete because its resolved connection changed while sign-in was pending. Start sign-in again.`,- );- }-}--// ----------------------------------------------------------------------------// Shared-// -----------------------------------------------------------------------------function requireRegistry(): ConnectionRegistry {- const registry = loadContext().get(ConnectionRegistryKey);- if (registry === undefined) throw new Error("This agent has no connections.");- return registry;-}--function requireConnection(- registry: ConnectionRegistry,- name: string,-): ResolvedConnectionDefinition {- const connection = findConnection(registry, name);- if (connection !== undefined) return connection;- const available = registry.getConnectionNames();- throw new Error(- available.length === 0- ? `Connection "${name}" is not available. No connections are available right now.`- : `Connection "${name}" is not available. Available connections: ${available.join(", ")}.`,- );-}-import { frameworkTool } from "#tools/provided/framework-tool.js";diff --git a/packages/eve/src/execution/tools/load-skill.test.ts b/packages/eve/src/execution/tools/load-skill.test.tsdeleted file mode 100644index 86610ebd3..000000000--- a/packages/eve/src/execution/tools/load-skill.test.ts+++ /dev/null@@ -1,169 +0,0 @@-import { describe, expect, it, vi } from "vitest";--import { ContextContainer, contextStorage } from "#context/container.js";-import { DynamicSkillManifestKey, SandboxKey } from "#context/keys.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { mockSandbox } from "#internal/testing/mocks/mock-sandbox.js";-import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";-import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";-import type { ResolvedSkillDefinition } from "#runtime/types.js";-import { readToolBehavior } from "#tools/behavior.js";-import { loadSkill } from "#tools/provided/load-skill.js";--function skillToolExecutor(ctx: ContextContainer, skills: readonly ResolvedSkillDefinition[] = []) {- ctx.set(BundleKey, {- resolvedAgent: { skills },- } as never);- const execute = loadSkill.execute;- if (execute === undefined) throw new Error("load_skill tool is missing an execute function");- return execute;-}--describe("loadSkill", () => {- it("is always advertised", () => {- expect(readToolBehavior(loadSkill)).toEqual({- availability: [],- presentation: "load-skill",- });- });-- it("describes when skill loading should be used", () => {- expect(loadSkill.description).toContain("request clearly matches a listed skill description");- expect(loadSkill.description).toContain(- "Loading adds the skill instructions to the current turn.",- );- expect(loadSkill.description).toContain("Available skills block");- expect(loadSkill.description).not.toContain("connection_search");- });-});--describe("load_skill executor", () => {- it("loads an authored markdown skill from the active compiled bundle", async () => {- const ctx = new ContextContainer();- const execute = skillToolExecutor(ctx, [- {- description: "Research a topic systematically",- logicalPath: "skills/research.md",- markdown: "# Research\n\nFollow the evidence.\n",- name: "research",- sourceId: "skills/research.md",- sourceKind: "markdown",- },- {- description: "Write a concise summary",- logicalPath: "skills/summarize.md",- markdown: "# Summarize\n",- name: "summarize",- sourceId: "skills/summarize.md",- sourceKind: "markdown",- },- ]);-- await expect(- contextStorage.run(ctx, () => execute({ skill: "research" }, {} as never)),- ).resolves.toBe("# Research\n\nFollow the evidence.\n");- });-- it("loads an authored package skill without opening the sandbox", async () => {- const get = vi.fn(async () => null);- const access = { captureState: async () => ({ session: null }), get, stop: async () => {} };- const ctx = new ContextContainer();- ctx.set(SandboxKey, access);- const execute = skillToolExecutor(ctx, [- {- assetsPath: "/authored/skills/incident-response/assets",- description: "Run the full incident response procedure",- logicalPath: "skills/incident-response/SKILL.md",- markdown:- "# Incident response\n\nConsult `references/services/api/owners.md` when needed.\n",- name: "incident-response",- referencesPath: "/authored/skills/incident-response/references",- rootPath: "/authored/skills/incident-response",- scriptsPath: "/authored/skills/incident-response/scripts",- skillFilePath: "/authored/skills/incident-response/SKILL.md",- skillId: "incident-response",- sourceId: "skills/incident-response/SKILL.md",- sourceKind: "skill-package",- },- ]);-- await expect(- contextStorage.run(ctx, () => execute({ skill: "incident-response" }, {} as never)),- ).resolves.toBe(- "# Incident response\n\nConsult `references/services/api/owners.md` when needed.\n",- );- expect(get).not.toHaveBeenCalled();- });-- it("loads an active dynamic skill from durable context without frontmatter or the sandbox", async () => {- const get = vi.fn(async () => null);- const ctx = new ContextContainer();- ctx.set(SandboxKey, {- captureState: async () => ({ session: null }),- get,- stop: async () => {},- });- ctx.set(DynamicSkillManifestKey, {- policy: [- {- description: "Apply the dynamic policy",- markdown: "---\nname: policy\n---\n# Dynamic policy\n",- name: "policy",- },- ],- });- const execute = skillToolExecutor(ctx, [- {- description: "Apply the static policy",- logicalPath: "skills/policy.md",- markdown: "# Static policy\n",- name: "policy",- sourceId: "skills/policy.md",- sourceKind: "markdown",- },- ]);-- await expect(- contextStorage.run(ctx, () => execute({ skill: "policy" }, {} as never)),- ).resolves.toBe("# Dynamic policy\n");- expect(get).not.toHaveBeenCalled();- });-- it("surfaces dynamic skill names when the requested id is missing", async () => {- const ctx = new ContextContainer();- ctx.set(SandboxKey, mockSandbox().access);- ctx.set(DynamicSkillManifestKey, {- custom: [- { description: "Talk like a dog", markdown: "# Woof", name: "custom__talk-like-a-dog" },- { description: "Bark", markdown: "# Bark", name: "custom__bark" },- ],- });- const execute = skillToolExecutor(ctx);-- await expect(- contextStorage.run(ctx, () => execute({ skill: "talk-like-a-dog" }, {} as never)),- ).rejects.toThrow("Available skills: custom__bark, custom__talk-like-a-dog.");- });-- it("redirects an installed connection mistakenly passed as a skill", async () => {- const registry = {- dispose: async () => {},- getClient: () => {- throw new Error("Not used by load_skill");- },- getConnectionApproval: () => undefined,- getConnectionNames: () => ["linear"],- getConnections: () => [],- } satisfies ConnectionRegistry;- const ctx = new ContextContainer();- ctx.set(SandboxKey, mockSandbox().access);- ctx.set(ConnectionRegistryKey, registry);- const execute = skillToolExecutor(ctx);-- await expect(- contextStorage.run(ctx, () => execute({ skill: "linear" }, {} as never)),- ).rejects.toThrow(- '"linear" is a connection, not a skill. Find its tools with connection_search and connection "linear", then call them with connection_execute.',- );- });-});diff --git a/packages/eve/src/execution/tools/load-skill.ts b/packages/eve/src/execution/tools/load-skill.tsdeleted file mode 100644index b4fbb27b5..000000000--- a/packages/eve/src/execution/tools/load-skill.ts+++ /dev/null@@ -1,70 +0,0 @@-import { defineJsonSchema } from "#tools/schema.js";--import { loadContext } from "#context/container.js";-import { DynamicSkillManifestKey } from "#context/keys.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { BundleKey } from "#runtime/sessions/runtime-context-keys.js";-import { stripSkillFrontmatter } from "#shared/skill-package.js";--/**- * Typed input accepted by {@link executeLoadSkillTool}.- */-interface LoadSkillInput {- skill: string;-}--/**- * Executes the `load_skill` tool.- *- * Returns instructions from memory without a sandbox: authored skills from the- * resolved agent and dynamic skills from durable context. Dynamic skills take- * precedence over authored skills with the same name.- */-async function executeLoadSkillTool(args: LoadSkillInput): Promise<unknown> {- const ctx = loadContext();- const { skill } = args;- const dynamicSkills = Object.values(ctx.get(DynamicSkillManifestKey) ?? {}).flat();- const authoredSkills = ctx.require(BundleKey).resolvedAgent.skills;- const dynamicSkill = dynamicSkills.find((entry) => entry.name === skill);- if (dynamicSkill !== undefined) return stripSkillFrontmatter(dynamicSkill.markdown);- const authoredSkill = authoredSkills.find((entry) => entry.name === skill);- if (authoredSkill !== undefined) return authoredSkill.markdown;-- const availableSkills = [- ...new Set([...dynamicSkills, ...authoredSkills].map((entry) => entry.name)),- ].sort();- const message = formatSkillNotFoundError(skill, availableSkills);- const connectionName = ctx- .get(ConnectionRegistryKey)- ?.getConnectionNames()- .find((name) => name.toLowerCase() === skill.toLowerCase());- if (connectionName === undefined) throw new Error(message);- throw new Error(- `${message} "${connectionName}" is a connection, not a skill. ` +- `Find its tools with connection_search and connection "${connectionName}", then call them with connection_execute.`,- );-}--function formatSkillNotFoundError(skill: string, availableSkills: readonly string[]): string {- const hint =- availableSkills.length > 0 ? ` Available skills: ${availableSkills.join(", ")}.` : "";- return `No skill named "${skill}".${hint}`;-}--// ----------------------------------------------------------------------------// Tool definition-// -----------------------------------------------------------------------------export const SKILL_INPUT_SCHEMA = defineJsonSchema<LoadSkillInput>({- type: "object",- properties: {- skill: { type: "string", description: "Available skill name or id." },- },- required: ["skill"],- additionalProperties: false,-});-export const SKILL_OUTPUT_SCHEMA = defineJsonSchema<string>({ type: "string" });--export async function executeLoadSkill(input: unknown): Promise<unknown> {- return await executeLoadSkillTool(input as LoadSkillInput);-}diff --git a/packages/eve/src/execution/tools/workflow/harness-definition.ts b/packages/eve/src/execution/tools/workflow/harness-definition.tsindex a683acd52..348fa0248 100644--- a/packages/eve/src/execution/tools/workflow/harness-definition.ts+++ b/packages/eve/src/execution/tools/workflow/harness-definition.ts@@ -42,6 +42,7 @@ export function createPreparedWorkflowToolHarnessDefinition(definition: {frameworkTool: tool.kind !== "authored-tool" || tool.owner.kind === "framework",behavior: tool.behavior,+ deferred: tool.deferred,description: tool.description,inputSchema: toInputSchema(tool.inputSchema) ?? UNSPECIFIED_INPUT_SCHEMA,name: tool.name,diff --git a/packages/eve/src/framework/sources/registry.ts b/packages/eve/src/framework/sources/registry.tsindex 6053c1d67..9676e5b7d 100644--- a/packages/eve/src/framework/sources/registry.ts+++ b/packages/eve/src/framework/sources/registry.ts@@ -9,7 +9,7 @@ import {type ProgrammaticModuleNamespace,} from "#compiler/source-graph.js";-const revision = `eve@${resolveInstalledPackageInfo().version}:compiled-manifest-v52`;+const revision = `eve@${resolveInstalledPackageInfo().version}:compiled-manifest-v53`;const localDefaults = defineProgrammaticAgentSource({id: "eve:defaults",@@ -37,14 +37,6 @@ const localDefaults = defineProgrammaticAgentSource({logicalPath: "tools/web_fetch.ts",loadNamespace: () => import("#tools/provided/web-fetch.js"),},- {- logicalPath: "tools/load_skill.ts",- loadNamespace: () => import("#tools/provided/load-skill.js"),- },- {- logicalPath: "tools/connection_tools.ts",- loadNamespace: () => import("#tools/framework/connection-tools.js"),- },{logicalPath: "tools/web_search.ts",loadNamespace: () => import("#tools/provided/web-search.js"),diff --git a/packages/eve/src/harness/action-presentation.ts b/packages/eve/src/harness/action-presentation.tsindex 937587f17..2693487c0 100644--- a/packages/eve/src/harness/action-presentation.ts+++ b/packages/eve/src/harness/action-presentation.ts@@ -3,7 +3,7 @@ import type { ToolSet, TypedToolCall } from "ai";import type { ActionPresentationByCallId } from "#protocol/message.js";import { createRuntimeActionRequestFromToolCall } from "#harness/coordination.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";-import type { HarnessToolMap } from "#harness/types.js";+import type { HarnessToolLookup } from "#harness/types.js";import type { RuntimeActionRequest } from "#shared/action-types.js";import { normalizePresentationText } from "#shared/presentation-text.js";import { parseJsonObject } from "#shared/json.js";@@ -15,7 +15,7 @@ export interface RuntimeActionRequestProjection {export function createPresentedRuntimeActionRequestFromToolCall(input: {readonly toolCall: TypedToolCall<ToolSet>;- readonly tools: HarnessToolMap;+ readonly tools: HarnessToolLookup;}): RuntimeActionRequestProjection {const action = createRuntimeActionRequestFromToolCall(input);const presentationLabel = projectToolStartLabel(diff --git a/packages/eve/src/harness/action-result-helpers.test.ts b/packages/eve/src/harness/action-result-helpers.test.tsindex ea9506008..e7201df74 100644--- a/packages/eve/src/harness/action-result-helpers.test.ts+++ b/packages/eve/src/harness/action-result-helpers.test.ts@@ -57,16 +57,16 @@ describe("createRuntimeToolResultFromToolError", () => {createRuntimeToolResultFromToolError({error: new Error('No skill named "demo"'),input: { skill: "demo" },- toolCallId: "call_load_skill",- toolName: "load_skill",+ toolCallId: "call_lookup",+ toolName: "lookup",type: "tool-error",}),).toEqual({- callId: "call_load_skill",+ callId: "call_lookup",isError: true,kind: "tool-result",output: 'No skill named "demo"',- toolName: "load_skill",+ toolName: "lookup",});});});diff --git a/packages/eve/src/harness/action-result-helpers.ts b/packages/eve/src/harness/action-result-helpers.tsindex 4f8d5fc16..57c2194be 100644--- a/packages/eve/src/harness/action-result-helpers.ts+++ b/packages/eve/src/harness/action-result-helpers.ts@@ -1,7 +1,9 @@import type { ModelMessage, ToolSet, TypedToolError, TypedToolResult } from "ai";-import type { RuntimeToolResultActionResult } from "#shared/action-types.js";+import { SKILL_ENTRY_NAME } from "#protocol/catalog-tools.js";+import type { RuntimeActionResult, RuntimeToolResultActionResult } from "#shared/action-types.js";import { toError } from "#shared/errors.js";+import { skillTarget } from "#shared/action-request-name.js";import { parseJsonValue, type JsonValue } from "#shared/json.js";import {authorizationPendingAsJsonObject,@@ -101,6 +103,25 @@ export function createRuntimeToolResultFromToolError(});}+/**+ * The action result a call's tool result reports. A skill load reports the+ * `load-skill-result` of the skill its `input` names, as its call reports a+ * `load-skill` action.+ */+export function toActionResult(+ result: RuntimeToolResultActionResult,+ input: unknown,+): RuntimeActionResult {+ if (result.toolName !== SKILL_ENTRY_NAME) return result;+ return {+ callId: result.callId,+ isError: result.isError,+ kind: "load-skill-result",+ name: skillTarget(input),+ output: result.output,+ };+}+/*** Builds the inline tool-result message part that repairs model history after a* local tool execution error.@@ -125,15 +146,17 @@ export function createToolResultMessagePartFromToolError(*/export function createRuntimeToolResultFromMessagePart(part: ToolResultPart,+ /** The entry the result's call ran, which history names after the catalog tool for a call made through one. */+ toolName: string,): RuntimeToolResultActionResult {return createRuntimeToolResultFromValue({callId: part.toolCallId,output: toolResultOutputToJsonValue({output: part.output,toolCallId: part.toolCallId,- toolName: part.toolName,+ toolName,}),- toolName: part.toolName,+ toolName,isError: isToolResultError(part.output),});}diff --git a/packages/eve/src/harness/advertised-tools.test.ts b/packages/eve/src/harness/advertised-tools.test.tsdeleted file mode 100644index fd842feef..000000000--- a/packages/eve/src/harness/advertised-tools.test.ts+++ /dev/null@@ -1,126 +0,0 @@-import { jsonSchema } from "ai";-import { describe, expect, it } from "vitest";--import { getAdvertisedTools } from "#harness/advertised-tools.js";-import type { HarnessToolDefinition } from "#harness/execute-tool.js";-import type { HarnessToolMap } from "#harness/types.js";--describe("getAdvertisedTools", () => {- it("keeps the built-in agent tool in the root session", () => {- const tools = new Map([- ["add", createTool("add")],- ["agent", createBuiltInAgentTool()],- ]) satisfies HarnessToolMap;-- const advertisedTools = getAdvertisedTools({ session: {}, tools });-- expect([...advertisedTools.keys()]).toEqual(["add", "agent"]);- });-- it("keeps declared subagent tools in delegated sessions", () => {- const tools = new Map([- ["add", createTool("add")],- ["delegate", createSubagentTool("delegate")],- ]) satisfies HarnessToolMap;-- const advertisedTools = getAdvertisedTools({- session: { rootSessionId: "root-session" },- tools,- });-- expect([...advertisedTools.keys()]).toEqual(["add", "delegate"]);- });-- it("removes the built-in agent tool from delegated sessions", () => {- const tools = new Map([- ["add", createTool("add")],- ["agent", createBuiltInAgentTool()],- ]) satisfies HarnessToolMap;-- const advertisedTools = getAdvertisedTools({- session: { rootSessionId: "root-session" },- tools,- });-- expect([...advertisedTools.keys()]).toEqual(["add"]);- });-- it("keeps a declared subagent named agent in delegated sessions", () => {- const tools = new Map([- ["add", createTool("add")],- ["agent", createSubagentTool("agent")],- ]) satisfies HarnessToolMap;-- const advertisedTools = getAdvertisedTools({- session: { rootSessionId: "root-session" },- tools,- });-- expect([...advertisedTools.keys()]).toEqual(["add", "agent"]);- });-});--describe("getAdvertisedTools for definition arrays", () => {- it("removes built-in agent tool definitions from delegated sessions", () => {- const advertisedTools = getAdvertisedTools({- session: { rootSessionId: "root-session" },- tools: [createTool("add"), createSubagentTool("delegate"), createBuiltInAgentTool()],- });-- expect(advertisedTools.map((tool) => tool.name)).toEqual(["add", "delegate"]);- });-- it("hides tools marked unavailable in subagents from delegated sessions", () => {- const tools = new Map([- ["root_only", { ...createTool("root_only"), availableInSubagents: false }],- ]) satisfies HarnessToolMap;-- expect([...getAdvertisedTools({ session: {}, tools }).keys()]).toEqual(["root_only"]);- expect([- ...getAdvertisedTools({ session: { rootSessionId: "root-session" }, tools }).keys(),- ]).toEqual([]);- });-- it("keeps root-session tools in the root session", () => {- const tools = new Map([- ["add", createTool("add")],- ["root_only", createAvailableTool("root_only", ["root-session"])],- ]) satisfies HarnessToolMap;-- const advertisedTools = getAdvertisedTools({ session: {}, tools });-- expect([...advertisedTools.keys()]).toEqual(["add", "root_only"]);- });-});--function createTool(name: string): HarnessToolDefinition {- return {- description: `${name} description`,- inputSchema: jsonSchema({ type: "object" }),- name,- };-}--function createSubagentTool(name: string): HarnessToolDefinition {- return {- ...createTool(name),- workflowId: "workflow//./agent/subagents/researcher//execute",- };-}--function createBuiltInAgentTool(): HarnessToolDefinition {- return {- ...createSubagentTool("agent"),- rootOnly: true,- };-}--function createAvailableTool(- name: string,- availability: NonNullable<HarnessToolDefinition["behavior"]>["availability"],-): HarnessToolDefinition {- return {- ...createTool(name),- behavior: { availability },- };-}diff --git a/packages/eve/src/harness/advertised-tools.ts b/packages/eve/src/harness/advertised-tools.tsdeleted file mode 100644index 1c3003e12..000000000--- a/packages/eve/src/harness/advertised-tools.ts+++ /dev/null@@ -1,114 +0,0 @@-import type { ToolSet } from "ai";-import type { HarnessToolDefinition } from "#harness/execute-tool.js";-import type { HarnessSession, HarnessToolMap } from "#harness/types.js";--type AdvertisedToolSession = Pick<HarnessSession, "rootSessionId">;--type AdvertisedToolMapInput = {- readonly session: AdvertisedToolSession;- readonly tools: HarnessToolMap;-};--type AdvertisedToolDefinitionsInput = {- readonly session: AdvertisedToolSession;- readonly tools: readonly HarnessToolDefinition[];-};--type AdvertisedModelToolsInput = {- readonly modelTools: ToolSet;- readonly session: HarnessSession;- readonly tools: HarnessToolMap;-};--type AdvertisedModelTools = {- readonly harnessTools: HarnessToolMap;- readonly modelTools: ToolSet;- readonly session: HarnessSession;-};--type AdvertisedToolsInput =- | AdvertisedModelToolsInput- | AdvertisedToolMapInput- | AdvertisedToolDefinitionsInput;--export function getAdvertisedTools(input: AdvertisedModelToolsInput): Promise<AdvertisedModelTools>;--export function getAdvertisedTools(input: AdvertisedToolMapInput): HarnessToolMap;-export function getAdvertisedTools(- input: AdvertisedToolDefinitionsInput,-): readonly HarnessToolDefinition[];-export function getAdvertisedTools(- input: AdvertisedToolsInput,-): HarnessToolMap | Promise<AdvertisedModelTools> | readonly HarnessToolDefinition[] {- if ("modelTools" in input) {- return getAdvertisedModelTools(input);- }-- if (isToolDefinitionList(input.tools)) {- return filterUnavailableToolDefinitions(input.tools, input.session);- }-- return filterUnavailableToolMap(input.tools, input.session);-}--async function getAdvertisedModelTools(- input: AdvertisedModelToolsInput,-): Promise<AdvertisedModelTools> {- const tools = filterUnavailableToolMap(input.tools, input.session);- return {- harnessTools: tools,- modelTools: input.modelTools,- session: input.session,- };-}--function filterUnavailableToolDefinitions(- tools: readonly HarnessToolDefinition[],- session: AdvertisedToolSession,-): readonly HarnessToolDefinition[] {- const filteredTools: HarnessToolDefinition[] = [];-- for (const tool of tools) {- if (shouldHideTool(tool, session)) {- continue;- }- filteredTools.push(tool);- }- return filteredTools;-}--function filterUnavailableToolMap(- tools: HarnessToolMap,- session: AdvertisedToolSession,-): HarnessToolMap {- const filteredTools = new Map<string, HarnessToolDefinition>();-- for (const [name, tool] of tools) {- if (shouldHideTool(tool, session)) {- continue;- }- filteredTools.set(name, tool);- }- return filteredTools;-}--function shouldHideTool(- definition: HarnessToolDefinition,- session: AdvertisedToolSession,-): boolean {- const delegated = session.rootSessionId !== undefined;- if (- definition.rootOnly === true ||- definition.availableInSubagents === false ||- definition.behavior?.availability.includes("root-session") === true- ) {- return delegated;- }- return false;-}--function isToolDefinitionList(- tools: HarnessToolMap | readonly HarnessToolDefinition[],-): tools is readonly HarnessToolDefinition[] {- return Array.isArray(tools);-}diff --git a/packages/eve/src/harness/announcements.ts b/packages/eve/src/harness/announcements.tsindex 06e9429e7..ccbeb61b2 100644--- a/packages/eve/src/harness/announcements.ts+++ b/packages/eve/src/harness/announcements.ts@@ -1,6 +1,3 @@-import type { ContextContainer } from "#context/container.js";-import { ContextKey } from "#context/key.js";-/*** Framework state announced to the model as an append-only `context.state`* message. The harness appends a message only when `value` differs from the@@ -13,33 +10,7 @@ export interface Announcement {readonly value: string;/*** Renders the message for `value`. `previous` is the last announced value,- * or `undefined` for a baseline. Returning `undefined` records the value- * without adding a message.+ * or `undefined` for a baseline.*/- render(previous: string | undefined): string | undefined;-}--/** Step-local announcements keyed by producer, set before the model call. */-const PendingAnnouncementsKey = new ContextKey<Readonly<Record<string, Announcement>>>(- "eve.pendingAnnouncements",-);--/** Sets or clears the announcement `key` for the current step. */-export function setPendingAnnouncement(- ctx: ContextContainer,- key: string,- announcement: Announcement | undefined,-): void {- const { [key]: _previous, ...rest } = ctx.get(PendingAnnouncementsKey) ?? {};- ctx.setVirtualContext(- PendingAnnouncementsKey,- announcement === undefined ? rest : { ...rest, [key]: announcement },- );-}--/** The announcements set for the current step. */-export function getPendingAnnouncements(- ctx: Pick<ContextContainer, "get"> | undefined,-): Readonly<Record<string, Announcement>> {- return ctx?.get(PendingAnnouncementsKey) ?? {};+ render(previous: string | undefined): string;}diff --git a/packages/eve/src/harness/coordination.test.ts b/packages/eve/src/harness/coordination.test.tsindex ea198f7b3..d77d90e9e 100644--- a/packages/eve/src/harness/coordination.test.ts+++ b/packages/eve/src/harness/coordination.test.ts@@ -63,35 +63,6 @@ describe("createRuntimeActionRequestFromToolCall", () => {});},);- const loadSkillCall = {- input: { skill: "research" },- toolCallId: "call-skill",- toolName: "load_skill",- type: "tool-call" as const,- };-- it("classifies the framework load_skill tool as a skill action", () => {- expect(- createRuntimeActionRequestFromToolCall({- toolCall: loadSkillCall,- tools: new Map([- [- "load_skill",- {- description: "Load a skill.",- frameworkAction: "load-skill" as const,- inputSchema: jsonSchema({ type: "object" }),- name: "load_skill",- },- ],- ]),- }),- ).toEqual({- callId: "call-skill",- input: { skill: "research" },- kind: "load-skill",- });- });it("preserves workflow identity without changing observable action data", () => {const action = createRuntimeActionRequestFromToolCall({@@ -232,20 +203,6 @@ describe("createRuntimeActionRequestFromToolCall", () => {},});});-- it("keeps an authored load_skill override as an ordinary tool action", () => {- expect(- createRuntimeActionRequestFromToolCall({- toolCall: loadSkillCall,- tools: new Map(),- }),- ).toEqual({- callId: "call-skill",- input: { skill: "research" },- kind: "tool-call",- toolName: "load_skill",- });- });});describe("createCoordinationRequestFromToolCall", () => {diff --git a/packages/eve/src/harness/coordination.ts b/packages/eve/src/harness/coordination.tsindex de949477a..dccaf8677 100644--- a/packages/eve/src/harness/coordination.ts+++ b/packages/eve/src/harness/coordination.ts@@ -7,6 +7,7 @@ import type {RuntimeWorkflowTaskRequest,WorkflowToolCallEntry,} from "#shared/action-types.js";+import { SKILL_ENTRY_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";import { markRuntimeWorkflowToolAction } from "#shared/action-types.js";import { parseJsonObject, type JsonObject } from "#shared/json.js";import { getProxyInputRequests } from "#harness/proxy-input-requests.js";@@ -18,7 +19,7 @@ import { normalizeToolModelOutput } from "#harness/tool-model-output.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";import { commitCallEntry, isTaskTool } from "#execution/tasks/model-step.js";import { startsTasks } from "#execution/tasks/tool-entry-point.js";-import type { HarnessSession, HarnessToolMap } from "#harness/types.js";+import type { HarnessSession, HarnessToolLookup } from "#harness/types.js";type ToolResponsePart = Extract<ModelMessage, { role: "tool" }>["content"][number];type ToolResultPart = Extract<ToolResponsePart, { type: "tool-result" }>;@@ -61,14 +62,15 @@ export function forgetFinishedRuns(/** Each runtime result as the model reads it, beside the result the stream reports. */export async function runtimeResultCalls(results: readonly RuntimeActionResult[],- tools: HarnessToolMap | undefined,+ tools: HarnessToolLookup | undefined,): Promise<{ readonly part: ToolResultPart; readonly result: RuntimeActionResult }[]> {const settled: { part: ToolResultPart; result: RuntimeActionResult }[] = [];for (const result of results) {switch (result.kind) {+ // A skill load is always an `eve__skill` call in history.case "load-skill-result":settled.push({- part: toolResult(result, "load_skill", toToolResultOutput(result)),+ part: toolResult(result, SKILL_TOOL_NAME, toToolResultOutput(result)),result,});continue;@@ -103,14 +105,14 @@ function toolResult(}/**- * Turns a step's deferred calls into workflow runs, committing a task record- * for each call that starts a task. Task tool calls stay in the response- * alone: the session reads them from there.+ * Turns a step's workflow tool calls into workflow runs, committing a task+ * record for each call that starts a task. Task tool calls stay in the+ * response alone: the session reads them from there.*/-export function collectDeferredCalls(input: {+export function collectWorkflowCalls(input: {readonly session: HarnessSession;readonly toolCalls: readonly CoordinationToolCall[];- readonly tools: HarnessToolMap;+ readonly tools: HarnessToolLookup;readonly turnId: string;}): {readonly session: HarnessSession;@@ -156,19 +158,15 @@ export interface CoordinationToolCall {*/export function createRuntimeActionRequestFromToolCall(input: {readonly toolCall: CoordinationToolCall;- readonly tools: HarnessToolMap;+ readonly tools: HarnessToolLookup;}): RuntimeActionRequest {const definition = input.tools.get(input.toolCall.toolName);const toolInput = resolveToolCallInputObject(input.toolCall.input, {callId: input.toolCall.toolCallId,toolName: input.toolCall.toolName,});- if (definition?.frameworkAction === "load-skill") {- return {- callId: input.toolCall.toolCallId,- input: toolInput,- kind: "load-skill",- };+ if (input.toolCall.toolName === SKILL_ENTRY_NAME) {+ return { callId: input.toolCall.toolCallId, input: toolInput, kind: "load-skill" };}const handling = definition?.behavior?.handling;if (@@ -198,18 +196,18 @@ export function createRuntimeActionRequestFromToolCall(input: {}/**- * Projects one deferred harness tool call into a workflow run request. The- * input is the tool's own, without anything eve added to its model input.+ * Projects one workflow tool call into a workflow run request. The input is+ * the tool's own, without anything eve added to its model input.*/export function createCoordinationRequestFromToolCall(input: {readonly entry: WorkflowToolCallEntry;readonly input: JsonObject;readonly toolCall: CoordinationToolCall;- readonly tools: HarnessToolMap;+ readonly tools: HarnessToolLookup;}): RuntimeWorkflowTaskRequest {const definition = input.tools.get(input.toolCall.toolName);if (definition?.workflowId === undefined) {- throw new Error(`Deferred tool "${input.toolCall.toolName}" has no workflow.`);+ throw new Error(`Workflow tool "${input.toolCall.toolName}" has no workflow.`);}return {callId: input.toolCall.toolCallId,diff --git a/packages/eve/src/harness/current-messages.test.ts b/packages/eve/src/harness/current-messages.test.tsindex df38072b5..b28589012 100644--- a/packages/eve/src/harness/current-messages.test.ts+++ b/packages/eve/src/harness/current-messages.test.ts@@ -78,34 +78,6 @@ describe("createCurrentMessages", () => {]);});- it("prepares tracked announcements without advancing the recorded baseline", () => {- const recorded = { availableSkills: "working" };- const current = createCurrentMessages([{ role: "user", content: "working", kind: "user" }], {- historyState: recorded,- });-- current.addAnnouncements({ availableSkills: "working" });- current.addAnnouncements({ availableSkills: "completed" });- current.addAnnouncements({ availableSkills: "completed" });-- expect(recorded).toEqual({ availableSkills: "working" });- expect(current.historyState).toEqual({ availableSkills: "completed" });- expect(current.history).toEqual([- { role: "user", content: "working", kind: "user" },- { role: "user", content: "completed", kind: "context.state" },- ]);- });-- it("ignores empty or absent announcements", () => {- const current = createCurrentMessages([]);- current.addAnnouncements({ availableSkills: "skills" });- current.addAnnouncements({ availableSkills: "" });- current.addAnnouncements({});-- expect(current.history).toEqual([{ role: "user", content: "skills", kind: "context.state" }]);- expect(current.historyState).toEqual({ availableSkills: "skills" });- });-it("persists additions without storing client context or replacing projected history", () => {const hidden = {role: "user" as const,diff --git a/packages/eve/src/harness/current-messages.ts b/packages/eve/src/harness/current-messages.tsindex 8a4e12e08..a5cfbc6fd 100644--- a/packages/eve/src/harness/current-messages.ts+++ b/packages/eve/src/harness/current-messages.ts@@ -12,11 +12,6 @@ interface AddCurrentMessageOptions {readonly cacheFriendly?: boolean;}-interface CurrentAnnouncements {- readonly availableSkills?: string;- readonly keyed?: Readonly<Record<string, Announcement>>;-}-interface CurrentMessagesOptions {readonly historyState?: HistoryState;readonly currentTurnMessages?: readonly HarnessModelMessage[];@@ -33,7 +28,7 @@ export function createCurrentMessages(readonly nonSystemMessages: readonly HarnessModelMessage[];readonly systemMessages: readonly SystemModelMessage[];add(message: string, kind: FrameworkMessageKind, options?: AddCurrentMessageOptions): void;- addAnnouncements(announcements: CurrentAnnouncements): void;+ addAnnouncements(announcements: Readonly<Record<string, Announcement>>): void;addSystem(messages: SystemModelMessage | readonly SystemModelMessage[]): void;} {const durableMessages = [...history];@@ -81,18 +76,11 @@ export function createCurrentMessages(return {add,addAnnouncements(announcements) {- const skills = announcements.availableSkills;- if (skills !== undefined && skills.length > 0 && historyState.availableSkills !== skills) {- appendUserMessage(skills, "context.state");- historyState.availableSkills = skills;- }- const keyed = announcements.keyed ?? {};- for (const key of Object.keys(keyed).sort()) {- const announcement = keyed[key]!;+ for (const key of Object.keys(announcements).sort()) {+ const announcement = announcements[key]!;const previous = historyState.announcements?.[key];if (previous === announcement.value) continue;- const message = announcement.render(previous);- if (message !== undefined) appendUserMessage(message, "context.state");+ appendUserMessage(announcement.render(previous), "context.state");historyState.announcements = { ...historyState.announcements, [key]: announcement.value };}},diff --git a/packages/eve/src/harness/emission.ts b/packages/eve/src/harness/emission.tsindex 232f5682e..5d20492eb 100644--- a/packages/eve/src/harness/emission.ts+++ b/packages/eve/src/harness/emission.ts@@ -25,6 +25,7 @@ import type { JsonObject } from "#shared/json.js";import {createRuntimeToolResultFromStepResult,createRuntimeToolResultFromToolError,+ toActionResult,createToolResultMessagePartFromToolError,} from "#harness/action-result-helpers.js";import {@@ -44,9 +45,8 @@ import { normalizeModelStreamError } from "#harness/model-call/errors.js";import { createOrderedStreamEmitter } from "#harness/ordered-stream-emitter.js";import { interruptStreamOnFailure } from "#harness/interruptible-stream.js";import { isInlineAuthorizationToolResult } from "#harness/inline-tool-authorization.js";-import { emitNestedToolActions } from "#harness/nested-actions.js";import type { TurnPosition } from "#harness/session-machine/view.js";-import type { HarnessEmitFn, HarnessToolMap } from "#harness/types.js";+import type { HarnessEmitFn, HarnessToolLookup } from "#harness/types.js";import { normalizeAssistantStepFinishReason } from "#harness/finish-reason.js";/**@@ -71,7 +71,7 @@ interface StreamActionEmissionOptions {* as the reply.*/readonly hidesHeldText?: boolean;- readonly tools: HarnessToolMap;+ readonly tools: HarnessToolLookup;readonly unsettledActionToolNames?: Map<string, string>;}@@ -246,7 +246,6 @@ async function consumeStreamContent(return;}emittedActionResultCallIds.add(result.callId);- await emitNestedToolActions(emitFn, state, result.callId);const resultPresentation =result.isError === true? undefined@@ -259,7 +258,7 @@ async function consumeStreamContent(await emitFn(createActionResultEvent({presentation: resultPresentation,- result,+ result: toActionResult(result, actionInputs.get(result.callId)),sequence: state.sequence,stepIndex: state.stepIndex,turnId: state.turnId,diff --git a/packages/eve/src/harness/execute-call.test.ts b/packages/eve/src/harness/execute-call.test.tsnew file mode 100644index 000000000..695b1f67b--- /dev/null+++ b/packages/eve/src/harness/execute-call.test.ts@@ -0,0 +1,96 @@+import type { Telemetry } from "ai";+import { describe, expect, it } from "vitest";+import { CALL_TOOL_NAME } from "#protocol/catalog-tools.js";++import { catalogContext, inlineTool, workflowTool } from "#internal/testing/catalog-fixtures.js";++import { toEntryTelemetry } from "./execute-call.js";++interface ToolCall {+ readonly input: unknown;+ readonly toolCallId: string;+ readonly toolName: string;+}++/** An integration written as a class, so its methods read their state through `this`. */+class RecordingIntegration {+ readonly records: string[] = [];++ onToolExecutionStart(event: { readonly toolCall: ToolCall }): void {+ this.records.push(`start ${event.toolCall.toolName} ${JSON.stringify(event.toolCall.input)}`);+ }++ onToolExecutionEnd(event: { readonly toolCall: ToolCall }): void {+ this.records.push(`end ${event.toolCall.toolName}`);+ }++ executeTool<T>(options: { readonly execute: () => PromiseLike<T>; readonly toolCallId: string }) {+ this.records.push(`executeTool ${options.toolCallId}`);+ return options.execute();+ }+}++/** Reports one tool execution to `telemetry` in the order the AI SDK does. */+async function reportExecution(telemetry: Telemetry, toolCall: ToolCall): Promise<string> {+ await telemetry.onToolExecutionStart?.({ toolCall } as never);+ const output = await telemetry.executeTool!({+ callId: "generation-1",+ execute: async () => "ran",+ toolCallId: toolCall.toolCallId,+ });+ await telemetry.onToolExecutionEnd?.({ output, success: true, toolCall } as never);+ return output;+}++function setup() {+ const { catalog } = catalogContext({+ tools: [+ inlineTool("add"),+ inlineTool("refund_invoice", { deferred: true }),+ workflowTool("deploy_service", "execute", { deferred: true }),+ ],+ });+ const integration = new RecordingIntegration();+ const telemetry = toEntryTelemetry({ integrations: [integration] }, catalog.resolve);+ const [wrapped] = (telemetry?.integrations ?? []) as Telemetry[];+ return { integration, wrapped: wrapped! };+}++describe("toEntryTelemetry", () => {+ it("hides a workflow entry reached through eve__tool, which the harness dispatches after the step", async () => {+ const { integration, wrapped } = setup();++ const output = await reportExecution(wrapped, {+ input: { input: { service: "api" }, name: "deploy_service" },+ toolCallId: "call-deploy",+ toolName: CALL_TOOL_NAME,+ });++ expect(output).toBe("ran");+ expect(integration.records).toEqual([]);+ });++ it("reports an inline entry reached through eve__tool under its name, with a class integration's this bound", async () => {+ const { integration, wrapped } = setup();++ await reportExecution(wrapped, {+ input: { input: { invoiceId: "in_1" }, name: "refund_invoice" },+ toolCallId: "call-refund",+ toolName: CALL_TOOL_NAME,+ });++ expect(integration.records).toEqual([+ 'start refund_invoice {"invoiceId":"in_1"}',+ "executeTool call-refund",+ "end refund_invoice",+ ]);+ });++ it("reports a direct call unchanged", async () => {+ const { integration, wrapped } = setup();++ await reportExecution(wrapped, { input: { a: 1 }, toolCallId: "call-add", toolName: "add" });++ expect(integration.records).toEqual(['start add {"a":1}', "executeTool call-add", "end add"]);+ });+});diff --git a/packages/eve/src/harness/execute-call.ts b/packages/eve/src/harness/execute-call.tsnew file mode 100644index 000000000..af38dabbe--- /dev/null+++ b/packages/eve/src/harness/execute-call.ts@@ -0,0 +1,231 @@+/**+ * Catalog calls in the harness. Once a step's calls enter the harness, every+ * seam reads an `eve__tool` or `eve__skill` call as the call to the entry it+ * names. Model history keeps the model's own calls.+ */++import type { ModelMessage, Telemetry, TelemetryOptions, TextStreamPart, ToolSet } from "ai";++import { isWorkflowTool } from "#execution/tasks/model-step.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import type { HarnessStepResult } from "#harness/step-hooks.js";+import { toolCallModelOutput } from "#harness/tool-call-io.js";+import type { CallResolver, ResolvedCall, ToolCallLike } from "#harness/types.js";+import { CALL_TOOL_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";+import type { ToolExecuteOptions } from "#tools/definition.js";++/**+ * The name each tool call has in model history, by call id. A result written+ * to history carries its call's name there, which is the catalog tool for a+ * call made through one.+ */+export function historyCallNames(messages: readonly ModelMessage[]): ReadonlyMap<string, string> {+ const names = new Map<string, string>();+ for (const message of messages) {+ if (message.role !== "assistant" || !Array.isArray(message.content)) continue;+ for (const part of message.content) {+ if (part.type === "tool-call") names.set(part.toolCallId, part.toolName);+ }+ }+ return names;+}++/**+ * The AI SDK decides per tool name whether to run a call, and it runs every+ * `eve__tool` call. A direct call to a workflow tool or agent never runs in the+ * SDK: the harness dispatches it after the step. Through `eve__tool`, such a+ * call returns this stand-in instead, which never leaves this module: it is+ * dropped from the stream, the step, and telemetry, and the harness then+ * dispatches the call as it does a direct one.+ */+const DISPATCHED = Object.freeze({ dispatched: true });++/** Whether a call runs as a workflow tool or agent the harness dispatches after the step. */+export function dispatchesAfterStep(resolved: ResolvedCall<ToolCallLike> | undefined): boolean {+ return resolved !== undefined && isWorkflowTool(resolved.definition);+}++/** Runs a resolved catalog call, or stands in for one the harness dispatches. */+export function runEntryCall(+ resolved: ResolvedCall<ToolCallLike>,+ options: ToolExecuteOptions,+): unknown {+ if (dispatchesAfterStep(resolved)) return DISPATCHED;+ const { call, definition } = resolved;+ if (definition.execute === undefined) throw new Error(`"${call.toolName}" cannot run.`);+ return definition.execute(call.input, options);+}++/**+ * The call a tool stub matches. An `eve__tool` call matches as the call to its+ * entry, with the entry's input, so a stub applies to it exactly as to a direct+ * call. A call the harness dispatches after the step has none here: its+ * dispatch applies the stub, as it does for a direct call.+ */+export function stubbedCall(+ call: ToolCallLike,+ resolved: ResolvedCall<ToolCallLike> | undefined,+): ToolCallLike | undefined {+ if (resolved === undefined) return call;+ return dispatchesAfterStep(resolved) ? undefined : resolved.call;+}++/** The model output of a call's result; a stand-in's output is dropped with it. */+export function entryModelOutput(+ definition: Pick<HarnessToolDefinition, "name" | "toModelOutput">,+ output: unknown,+ toolCallId: string | undefined,+): ReturnType<typeof toolCallModelOutput> {+ return output === DISPATCHED+ ? Promise.resolve({ type: "json", value: null })+ : toolCallModelOutput(definition, output, toolCallId);+}++/** Every field of a step, so a new one cannot be left out of {@link toEntryStep}. */+type StepFields = { readonly [K in keyof Required<HarnessStepResult>]: HarnessStepResult[K] };++/**+ * The step as the harness reads it: catalog calls become calls to+ * their entries and stand-in results are dropped. The response messages, which+ * become history, keep the model's own calls.+ */+export function toEntryStep<T extends ModelMessage>(+ step: HarnessStepResult,+ responseMessages: readonly T[],+ resolve: CallResolver,+): readonly [HarnessStepResult, T[]] {+ const dispatched = new Set(+ (step.toolResults ?? [])+ .filter((toolResult) => toolResult.output === DISPATCHED)+ .map((toolResult) => toolResult.toolCallId),+ );+ const kept = (part: { readonly toolCallId?: string; readonly type: string }) =>+ part.type !== "tool-result" || !dispatched.has(part.toolCallId!);+ // SDK step fields are prototype getters, so each one is read explicitly.+ const entryStep: StepFields = {+ content: (step.content ?? []).filter(kept).map((part) => toEntryPart(part, resolve)),+ finishReason: step.finishReason,+ invalidInputToolCallIds: step.invalidInputToolCallIds,+ providerMetadata: step.providerMetadata,+ response: { ...step.response, messages: withoutResults(step.response.messages, kept) },+ text: step.text,+ toolCalls: (step.toolCalls ?? []).map((toolCall) => entryCallOf(toolCall, resolve)),+ toolResults: (step.toolResults ?? [])+ .filter(kept)+ .map((toolResult) => entryCallOf(toolResult, resolve)),+ usage: step.usage,+ };+ return [entryStep, withoutResults(responseMessages, kept)];+}++/**+ * The stream as the harness reads it, as {@link toEntryStep} reads the step.+ * A catalog call's input stream is dropped: its entry is known only once the+ * input is complete.+ */+export async function* toEntryStream(+ stream: AsyncIterable<TextStreamPart<ToolSet>>,+ resolve: CallResolver,+): AsyncIterable<TextStreamPart<ToolSet>> {+ const executeInputs = new Set<string>();+ for await (const part of stream) {+ if (+ part.type === "tool-input-start" &&+ (part.toolName === CALL_TOOL_NAME || part.toolName === SKILL_TOOL_NAME)+ ) {+ executeInputs.add(part.id);+ continue;+ }+ if (part.type === "tool-input-delta" && executeInputs.has(part.id)) continue;+ if (part.type === "tool-input-end" && executeInputs.delete(part.id)) continue;+ if (part.type === "tool-result" && part.output === DISPATCHED) continue;+ yield toEntryPart(part, resolve);+ }+}++/**+ * Telemetry as the harness reports it. Integrations see each catalog call as+ * the call to its entry, and never see a stand-in run: a direct+ * call to the same entry never runs in the SDK.+ */+export function toEntryTelemetry(+ telemetry: TelemetryOptions | undefined,+ resolve: CallResolver,+): TelemetryOptions | undefined {+ const integrations = telemetry?.integrations;+ if (integrations === undefined) return telemetry;+ const list = Array.isArray(integrations) ? integrations : [integrations];+ const hiddenCallIds = new Set<string>();+ return {+ ...telemetry,+ integrations: list.map((integration) =>+ entryToolExecutions(integration, resolve, hiddenCallIds),+ ),+ };+}++function entryToolExecutions(+ integration: Telemetry,+ resolve: CallResolver,+ hiddenCallIds: Set<string>,+): Telemetry {+ const { executeTool, onToolExecutionEnd, onToolExecutionStart } = integration;+ if (!executeTool && !onToolExecutionEnd && !onToolExecutionStart) return integration;+ type ExecuteToolOptions = Parameters<NonNullable<Telemetry["executeTool"]>>[0];+ type ToolExecutionEvent = {+ readonly toolCall: ToolCallLike & { readonly toolCallId: string };+ };+ const reportAsEntry =+ <E extends ToolExecutionEvent>(report: ((event: E) => unknown) | undefined) =>+ (event: E) => {+ const resolved = resolve(event.toolCall);+ if (dispatchesAfterStep(resolved)) {+ hiddenCallIds.add(event.toolCall.toolCallId);+ return undefined;+ }+ return report?.call(integration, { ...event, toolCall: resolved?.call ?? event.toolCall });+ };+ return Object.create(integration, {+ executeTool: {+ value(options: ExecuteToolOptions) {+ return hiddenCallIds.has(options.toolCallId) || executeTool === undefined+ ? options.execute()+ : executeTool.call(integration, options);+ },+ },+ onToolExecutionEnd: { value: reportAsEntry(onToolExecutionEnd) },+ onToolExecutionStart: { value: reportAsEntry(onToolExecutionStart) },+ }) as Telemetry;+}++function entryCallOf<T extends ToolCallLike>(toolCall: T, resolve: CallResolver): T {+ return resolve(toolCall)?.call ?? toolCall;+}++/** A stream or content part, with the call it carries resolved to its entry. */+function toEntryPart<P extends { readonly type: string }>(part: P, resolve: CallResolver): P {+ switch (part.type) {+ case "tool-call":+ case "tool-result":+ case "tool-error":+ return entryCallOf(part as P & ToolCallLike, resolve);+ case "tool-approval-request": {+ const request = part as P & { readonly toolCall?: ToolCallLike };+ if (request.toolCall === undefined) return part;+ return { ...request, toolCall: entryCallOf(request.toolCall, resolve) };+ }+ default:+ return part;+ }+}++function withoutResults<T extends ModelMessage>(+ messages: readonly T[],+ kept: (part: { readonly toolCallId?: string; readonly type: string }) => boolean,+): T[] {+ return messages.flatMap((message) => {+ if (message.role !== "tool") return [message];+ const content = message.content.filter(kept);+ return content.length === 0 ? [] : [{ ...message, content }];+ });+}diff --git a/packages/eve/src/harness/execute-tool.ts b/packages/eve/src/harness/execute-tool.tsindex 4509fdaa6..72ca2bbd8 100644--- a/packages/eve/src/harness/execute-tool.ts+++ b/packages/eve/src/harness/execute-tool.ts@@ -17,6 +17,8 @@ export interface HarnessToolDefinition {readonly label?: InternalToolLabelDefinition;readonly approvalKey?: (toolInput: Readonly<Record<string, unknown>>) => string;readonly behavior?: PreparedToolBehavior;+ /** Kept out of the model's tool list; the model reaches it through `eve__search` and `eve__tool`. */+ readonly deferred?: boolean;readonly description: string;/** A successful call ends the turn without a final reply; see `ToolDefinition.endsTurn`. */readonly endsTurn?: InternalToolDefinition["endsTurn"];@@ -24,7 +26,7 @@ export interface HarnessToolDefinition {/** Optional JSON input substituted when this tool starts its workflow body. */readonly executeInput?: (input: unknown) => JsonValue;readonly frameworkTool?: boolean;- readonly frameworkAction?: "load-skill" | "task-cancel" | "task-wait";+ readonly frameworkAction?: "task-cancel" | "task-wait";readonly inputSchema: FlexibleSchema;readonly name: string;readonly approval?: Approval;diff --git a/packages/eve/src/harness/final-output.ts b/packages/eve/src/harness/final-output.tsindex 55bed9ecf..7a94473de 100644--- a/packages/eve/src/harness/final-output.ts+++ b/packages/eve/src/harness/final-output.ts@@ -1,26 +1,21 @@import type { Tool } from "ai";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import { defineJsonSchema, toModelSchema } from "#tools/schema.js";import type { JsonObject } from "#shared/json.js";-/**- * Stable model-visible name for the kernel structured-output interception.- */-export const FINAL_OUTPUT_TOOL_NAME = "final_output";--const FINAL_OUTPUT_TOOL_DESCRIPTION =- "Deliver your final answer in the required structure by calling this tool. " +- "Call it exactly once, when you are done; do not answer in prose.";+const REPLY_TOOL_DESCRIPTION =+ "Reply with your final answer in the required structure by calling this tool. " ++ "Call it when you are done, as your final step; do not answer in prose.";/*** What the model reads when it gives its final output beside calls whose results it hasn't seen:* the turn can't end on an answer written before them, so it answers again once they arrive.*/-export const FINAL_OUTPUT_BESIDE_PENDING_CALLS =- "Your final output wasn't delivered because other calls in this response were still running. Use their results, then call final_output again.";+export const FINAL_OUTPUT_BESIDE_PENDING_CALLS = `Your reply wasn't delivered because other calls in this response were still running. Use their results, then call ${REPLY_TOOL_NAME} again.`;/**- * Builds the model-facing `final_output` tool from a lowered output schema.+ * Builds the model-facing `eve__reply` tool from a lowered output schema.** The tool has no `execute`: calling it is the terminal signal the harness* intercepts to surface the structured result. Its input is provider-constrained@@ -29,7 +24,7 @@ export const FINAL_OUTPUT_BESIDE_PENDING_CALLS =export function buildFinalOutputTool(schema: JsonObject): Tool {const modelSchema = toModelSchema(defineJsonSchema(schema), "input");return {- description: FINAL_OUTPUT_TOOL_DESCRIPTION,+ description: REPLY_TOOL_DESCRIPTION,inputSchema: modelSchema,outputSchema: modelSchema,};diff --git a/packages/eve/src/harness/hitl/approvals.ts b/packages/eve/src/harness/hitl/approvals.tsindex 4ffe5f6cc..0c76e77ba 100644--- a/packages/eve/src/harness/hitl/approvals.ts+++ b/packages/eve/src/harness/hitl/approvals.ts@@ -11,9 +11,11 @@ import { authorizationEventFields } from "#harness/authorization-event-fields.jsimport { renderPendingApprovalsSnippet } from "#harness/hitl/approval-prompt.js";import { isApprovalRequest } from "#harness/input-request-class.js";import {+ failedCall,resolveApprovalOutcome,resolveInputOutcome,TOOL_EXECUTION_DENIED_MESSAGE,+ unavailableToolMessage,type ResolvedInputBatch,} from "#harness/input-request-resolution.js";import { coalesceTurnInputs, createFrameworkUserMessage } from "#harness/messages.js";@@ -216,6 +218,11 @@ export function answer(/** The delivery merged the queue in, so the queue empties. */readonly takeQueued: boolean;readonly approvalKey: (request: InputRequest) => string | undefined;+ /**+ * Whether the request's restored step has `eve__search`, which a call whose+ * tool is gone can use to find another.+ */+ readonly searchable: (request: InputRequest) => boolean;},): Answered {const { policy } = input;@@ -320,29 +327,45 @@ export function answer(const grants = new Set(turn.grants);const batches: ResolvedInputBatch[] = [];- const rejected: UnstampedMessageStreamEvent[] = [];+ const results: UnstampedMessageStreamEvent[] = [];+ const unavailable = new Set(+ policy.audit.settlements+ .filter((settlement) => settlement.outcome === "unavailable")+ .map((settlement) => settlement.requestId),+ );const suspended = turn.suspended.map((step) => {if (!answered.includes(step)) return step;let messages = step.messages;const approvedRequests: InputRequest[] = [];for (const request of step.requests) {+ const { callId, toolName } = request.action;+ if (unavailable.has(request.requestId)) {+ const failed = failedCall({+ callId,+ message: unavailableToolMessage(toolName, input.searchable(request)),+ toolName,+ });+ messages = withResult(messages, failed.part);+ results.push(createActionResultEvent({ result: failed.result, ...step.event }));+ continue;+ }const { approved, reason, status } = resolveApprovalOutcome(byId.get(request.requestId));if (approved) {- grants.add(input.approvalKey(request) ?? request.action.toolName);+ grants.add(input.approvalKey(request) ?? toolName);approvedRequests.push(request);continue;}messages = withResult(messages, {output: { reason, type: "execution-denied" },- toolCallId: request.action.callId,- toolName: request.action.toolName,+ toolCallId: callId,+ toolName,type: "tool-result",});- rejected.push(+ results.push(createActionResultEvent({rejected: true,result: {- callId: request.action.callId,+ callId,isError: true,kind: "tool-result",output: {@@ -351,7 +374,7 @@ export function answer(message: reason ?? TOOL_EXECUTION_DENIED_MESSAGE,tool: { result: "not_run" },},- toolName: request.action.toolName,+ toolName,},...step.event,}),@@ -367,7 +390,7 @@ export function answer(const approved = [...(step.approved ?? []), ...approvedRequests];return { ...step, messages, requests: [], ...(approved.length > 0 && { approved }) };});- events.push(...batches.map(resolvedEvent), ...rejected);+ events.push(...batches.map(resolvedEvent), ...results);turn = { ...turn, grants: [...grants], suspended };return done({consumedMessage: resolved?.messageConsumed,@@ -455,7 +478,8 @@ function reportApprovalProgress();}for (const settlement of audit.settlements) {- if (!isOpen(settlement.requestId)) continue;+ // An unavailable request's candidate reported it failed, with the reason.+ if (!isOpen(settlement.requestId) || settlement.outcome === "unavailable") continue;events.push(createApprovalSettledEvent({outcome: settlement.outcome === "allowed" ? "approved" : "cancelled",diff --git a/packages/eve/src/harness/hitl/approved-calls.ts b/packages/eve/src/harness/hitl/approved-calls.tsindex bf931eea8..4539ec1b6 100644--- a/packages/eve/src/harness/hitl/approved-calls.ts+++ b/packages/eve/src/harness/hitl/approved-calls.ts@@ -9,18 +9,24 @@ import {import { createRuntimeToolResultFromValue } from "#harness/action-result-helpers.js";import { isInlineAuthorizationToolResult } from "#harness/inline-tool-authorization.js";+import { toolCallModelOutput } from "#harness/tool-call-io.js";import { projectDeltaPresentation, projectResultPresentation } from "#harness/tool-presentation.js";-import { buildToolSet, recheckApprovedCall } from "#harness/tools.js";-import { TOOL_EXECUTION_DENIED_MESSAGE } from "#harness/input-request-resolution.js";+import { recheckApprovedCall, wrapToolExecute } from "#harness/tools.js";+import {+ failedCall,+ TOOL_EXECUTION_DENIED_MESSAGE,+ unavailableToolMessage,+} from "#harness/input-request-resolution.js";import { throwIfTurnAborted } from "#harness/turn-cancellation.js";-import type { HarnessToolMap } from "#harness/types.js";-import { emitNestedToolActions } from "#harness/nested-actions.js";+import type { HarnessToolLookup } from "#harness/types.js";+import { SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";import { createActionPartialEvent, createActionResultEvent } from "#protocol/message.js";import { isAsyncIterable } from "#shared/async-iterable.js";import { toError } from "#shared/errors.js";import { createLogger, logError } from "#internal/logging.js";import type { InputRequest } from "#shared/input.js";import { parseJsonObject } from "#shared/json.js";+import { toModelSchema } from "#tools/schema.js";import type { HandleEventFn } from "#harness/types.js";type ToolResponsePart = Extract<ModelMessage, { role: "tool" }>["content"][number];@@ -41,7 +47,8 @@ export const APPROVED_CALL_INTERRUPTED_MESSAGE =*/export async function runApprovedCalls(input: {readonly requests: readonly InputRequest[];- readonly tools: HarnessToolMap;+ /** The entries of the step that asked, which the approved calls run. */+ readonly tools: HarnessToolLookup;readonly approvedTools?: ReadonlySet<string>;/** The conversation the tools read as `ctx.messages`. */readonly messages: readonly ModelMessage[];@@ -58,67 +65,56 @@ export async function runApprovedCalls(input: {readonly settled: readonly ApprovedCallResult[];readonly toolResults: readonly TypedToolResult<ToolSet>[];}> {- const tools = buildToolSet({ tools: input.tools, approvedTools: input.approvedTools });- const calls = input.requests.map((request) => {- const definition = input.tools.get(request.action.toolName);- const tool = tools[request.action.toolName];- if (- definition?.execute === undefined ||- tool?.execute === undefined ||- tool.toModelOutput === undefined- ) {- throw new Error(- "The approved tool is no longer available. Request a new tool call and approval.",- );- }- return { request, definition, tool };- });const at = {sequence: input.position.sequence,stepIndex: input.position.stepIndex,turnId: input.position.turnId,};+ /** A call that ends without running: the stream reports it failed, and the model reads why. */+ const settleFailure = async (+ callId: string,+ toolName: string,+ message: string,+ ): Promise<ApprovedCallResult> => {+ const failed = failedCall({ callId, message, toolName });+ await input.publish(createActionResultEvent({ ...at, result: failed.result }));+ return { part: failed.part };+ };const executed = await Promise.allSettled(- calls.map(async ({ request, definition, tool }) => {+ input.requests.map(async (request) => {const settled: ApprovedCallResult[] = [];const toolResults: TypedToolResult<ToolSet>[] = [];const { callId, toolName, input: args } = request.action;+ const definition = input.tools.get(toolName);+ const execute = definition === undefined ? undefined : wrapToolExecute(definition);// Once the tool runs, the call settles even if the turn is cancelled: a result reaches the// session, so it never runs a call with side effects a second time.let started = false;let completed: ApprovedCallResult | undefined;try {throwIfTurnAborted(input.abortSignal);- // As in the AI SDK: the stored input revalidates against the tool's own schema and runs- // unchanged, so the call that runs is the one the person approved.- const validation = await asSchema(tool.inputSchema).validate?.(args);- if (validation?.success === false) {- const message = `The approved input is no longer valid for tool "${toolName}". Request a new tool call and approval.`;- await input.publish(- createActionResultEvent({- ...at,- result: createRuntimeToolResultFromValue({- callId,- isError: true,- output: message,- toolName,- }),- }),- );+ // A connection or dynamic tool can go away while its call waits for approval.+ if (definition === undefined || execute === undefined) {return {settled: [- {- part: {- type: "tool-result" as const,- toolCallId: callId,- toolName,- output: { type: "error-text" as const, value: message },- },- },+ await settleFailure(+ callId,+ toolName,+ unavailableToolMessage(toolName, input.tools.get(SEARCH_TOOL_NAME) !== undefined),+ ),],toolResults,};}+ // As in the AI SDK: the stored input revalidates against the tool's own schema and runs+ // unchanged, so the call that runs is the one the person approved.+ const validation = await asSchema(+ toModelSchema(definition.inputSchema, "input"),+ ).validate?.(args);+ if (validation?.success === false) {+ const message = `The approved input is no longer valid for tool "${toolName}". Request a new tool call and approval.`;+ return { settled: [await settleFailure(callId, toolName, message)], toolResults };+ }const recheck = await recheckApprovedCall(definition, {abortSignal: input.abortSignal,callId,@@ -167,9 +163,8 @@ export async function runApprovedCalls(input: {const startedAt = performance.now();try {started = true;- const executed = tool.execute!(args, {+ const executed = execute(args, {abortSignal: input.abortSignal,- context: undefined,messages: [...input.messages],toolCallId: callId,});@@ -225,13 +220,11 @@ export async function runApprovedCalls(input: {} as TypedToolResult<ToolSet>;toolResults.push(result);if (isInlineAuthorizationToolResult(result)) return { settled, toolResults };- // Calls the tool made on the model's behalf report before its result.- await emitNestedToolActions(input.publish, input.position, callId);completed = {part: {output: failed? { type: "error-text", value: String(output) }- : await tool.toModelOutput!({ input: args, output, toolCallId: callId }),+ : await toolCallModelOutput(definition, output, callId),toolCallId: callId,toolName,type: "tool-result",@@ -254,30 +247,9 @@ export async function runApprovedCalls(input: {if (completed !== undefined) return { settled: [completed], toolResults };return { settled: [interruptedResult(callId, toolName)], toolResults };}- const message = toError(error).message;logError(log, "approved tool failed", error, { toolName, toolCallId: callId });- await input.publish(- createActionResultEvent({- ...at,- result: createRuntimeToolResultFromValue({- callId,- isError: true,- output: message,- toolName,- }),- }),- );return {- settled: [- {- part: {- type: "tool-result" as const,- toolCallId: callId,- toolName,- output: { type: "error-text" as const, value: message },- },- },- ],+ settled: [await settleFailure(callId, toolName, toError(error).message)],toolResults,};}diff --git a/packages/eve/src/harness/hitl/candidates.ts b/packages/eve/src/harness/hitl/candidates.tsindex a8753fb4f..26ee28891 100644--- a/packages/eve/src/harness/hitl/candidates.ts+++ b/packages/eve/src/harness/hitl/candidates.ts@@ -37,9 +37,10 @@ export interface ApprovalResponderIdentity {export interface ApprovalSettlementAuditRecord {readonly actor: ApprovalResponderIdentity;- /** The approver's full auth, which the approved call runs as. Absent for cancellations. */+ /** The approver's full auth, which the approved call runs as. Present only when allowed. */readonly approver?: SessionAuthContext;- readonly outcome: "allowed" | "cancelled";+ /** `unavailable`: the request's entry was gone, so its call is reported unavailable, never run. */+ readonly outcome: "allowed" | "cancelled" | "unavailable";readonly requestId: string;readonly settledAt: number;readonly candidateId?: string;@@ -229,6 +230,30 @@ export function settleAllowedCandidate(input: {});}+/**+ * Settles a request whose entry is gone, which no response can approve and no call can run: the+ * candidate fails with `reason`, its competitors go stale, and the call is reported unavailable.+ */+export function settleUnavailableCandidate(input: {+ readonly candidateId: string;+ readonly reason: string;+ readonly settledAt: number;+ readonly state: SessionStateMap | undefined;+}): ApprovalStateTransition {+ const state = expireApprovalCandidates({ now: input.settledAt, state: input.state });+ const candidate = readApprovalState(state).activeCandidates[input.candidateId];+ if (candidate === undefined) return { changed: false, state };+ return settleRequest({+ candidateId: candidate.candidateId,+ outcome: "unavailable",+ reason: input.reason,+ requestId: candidate.requestId,+ responder: candidate.responder,+ settledAt: input.settledAt,+ state,+ });+}+/** Atomically settles a direct authenticated approval response. */export function settleDirectApprovalResponse(input: {readonly actor: SessionAuthContext;@@ -272,6 +297,8 @@ export function getApprovalAuditState(state: SessionStateMap | undefined): {function settleRequest(input: {readonly candidateId?: string;readonly outcome: ApprovalSettlementAuditRecord["outcome"];+ /** Why the settling candidate failed, for an unavailable request. */+ readonly reason?: string;readonly requestId: string;readonly responder: SessionAuthContext;readonly settledAt: number;@@ -298,11 +325,13 @@ function settleRequest(input: {activeCandidates[candidate.candidateId] = candidate;continue;}+ const settling = candidate.candidateId === input.candidateId;candidateHistory.push(toCandidateAuditRecord({candidate,completedAt: input.settledAt,- status: candidate.candidateId === input.candidateId ? "allowed" : "stale",+ reason: settling ? input.reason : undefined,+ status: !settling ? "stale" : input.outcome === "unavailable" ? "failed" : "allowed",}),);}diff --git a/packages/eve/src/harness/hitl/coordinator.test.ts b/packages/eve/src/harness/hitl/coordinator.test.tsindex 7ce8729b9..5a1464b7d 100644--- a/packages/eve/src/harness/hitl/coordinator.test.ts+++ b/packages/eve/src/harness/hitl/coordinator.test.ts@@ -16,6 +16,7 @@ import { approvingSteps } from "#harness/hitl/approvals.js";import { sessionView } from "#harness/session-machine/commit.js";import { resolveTypedApproval } from "#harness/hitl/delivery.js";import { storedProjection } from "#harness/session-machine/view.js";+import { captureLogRecords } from "#internal/testing/log-records.js";import { parkedSteps, withParkedStep, withPublished } from "#internal/testing/session-machine.js";import { createApprovalSettledEvent } from "#protocol/message.js";import type { HarnessSession } from "#harness/types.js";@@ -59,7 +60,11 @@ function parkedSession(requester?: SessionAuthContext): HarnessSession {}describe("coordinateApprovalDelivery", () => {- function authorize(session: HarnessSession, response: ApprovalResponsePolicy) {+ function authorize(+ session: HarnessSession,+ response: ApprovalResponsePolicy,+ prepareTools?: () => Promise<Map<string, HarnessToolDefinition>>,+ ) {const ctx = new ContextContainer();ctx.set(SessionKey, {auth: { current: responder, initiator: responder },@@ -76,6 +81,7 @@ describe("coordinateApprovalDelivery", () => {return contextStorage.run(ctx, () =>coordinateApprovalDelivery({now: 101,+ prepareTools,session,tools: new Map([["gate", tool]]),}),@@ -226,6 +232,31 @@ describe("coordinateApprovalDelivery", () => {}});+ it("fails the candidate but keeps the request parked when its step's tools can't be restored", async () => {+ const logs = captureLogRecords();+ const ingested = await ingest();+ const response = vi.fn<ApprovalResponsePolicy>(() => ({ status: "allowed" }));++ const result = await authorize(ingested.session, response, async () => {+ throw new Error("The notes directory is unreachable.");+ });++ expect(response).not.toHaveBeenCalled();+ expect(getApprovalAuditState(result.session.state).candidateHistory).toEqual([+ expect.objectContaining({+ reason: "Approval authorization is temporarily unavailable. Please try again.",+ status: "failed",+ }),+ ]);+ // A restore failure may be transient, so the request waits for another response.+ expect(parkedSteps(result.session)[0]?.requests.map((entry) => entry.requestId)).toEqual([+ request.requestId,+ ]);+ expect(logs.records.map((record) => record.message)).toContain(+ "approval tools could not be restored",+ );+ });+it("does not complete a duplicate while its candidate is active", async () => {const ingested = await ingest();const duplicate = await ingest(ingested.session);diff --git a/packages/eve/src/harness/hitl/coordinator.ts b/packages/eve/src/harness/hitl/coordinator.tsindex 0619e0829..84347f083 100644--- a/packages/eve/src/harness/hitl/coordinator.ts+++ b/packages/eve/src/harness/hitl/coordinator.ts@@ -14,6 +14,7 @@ import {getApprovalAuditState,markApprovalCandidateAuthorizationRequired,settleAllowedCandidate,+ settleUnavailableCandidate,sameResponder,settleDirectApprovalResponse,type ActiveApprovalCandidate,@@ -29,15 +30,22 @@ import {} from "#harness/authorization.js";import { isApprovalRequest } from "#harness/input-request-class.js";import { suspendedSteps } from "#harness/session-machine/view.js";-import type { HarnessSession, HarnessToolMap, StepInput } from "#harness/types.js";+import type { HarnessSession, HarnessToolLookup, StepInput } from "#harness/types.js";+import { createLogger, logError } from "#internal/logging.js";import type { InputRequest } from "#shared/input.js";const UNAUTHENTICATED_APPROVAL_FEEDBACK = "Authentication is required to respond to this approval.";const REQUESTER_ONLY_APPROVAL_FEEDBACK ="Only the person who requested this action can respond to this approval.";+const UNAVAILABLE_TOOL_APPROVAL_REASON =+ "The tool this approval was for is no longer available, so the call won't run.";+const TEMPORARILY_UNAVAILABLE_APPROVAL_REASON =+ "Approval authorization is temporarily unavailable. Please try again.";const APPROVAL_AUTHORIZER_TIMEOUT_MS = 10_000;const APPROVAL_CANDIDATE_TTL_MS = 10 * 60_000;+const log = createLogger("harness.hitl");+interface ApprovalDeliveryResult {readonly challenges: readonly AuthorizationChallenge[];readonly feedback: readonly string[];@@ -72,8 +80,8 @@ export async function coordinateApprovalDelivery(input: {readonly now?: number;readonly session: HarnessSession;readonly stepInput?: StepInput;- readonly tools: HarnessToolMap;- readonly prepareTools?: (request: InputRequest) => Promise<HarnessToolMap>;+ readonly tools: HarnessToolLookup;+ readonly prepareTools?: (request: InputRequest) => Promise<HarnessToolLookup>;}): Promise<ApprovalDeliveryResult> {const now = input.now ?? Date.now();const expiredCandidates = getApprovalAuditState(input.session.state).activeCandidates.filter(@@ -252,8 +260,8 @@ export async function coordinateApprovalDelivery(input: {now,request,responder: candidate.responder,+ restoreTools: async () => (await input.prepareTools?.(request)) ?? input.tools,session,- tools: (await input.prepareTools?.(request)) ?? input.tools,});session = processed.session;didCommit ||= processed.didCommit;@@ -292,8 +300,9 @@ async function authorizeCandidate(input: {readonly now: number;readonly request: InputRequest;readonly responder: ActiveApprovalCandidate["responder"];+ /** The entries of the step that asked, restored by running its resolvers. */+ readonly restoreTools: () => Promise<HarnessToolLookup>;readonly session: HarnessSession;- readonly tools: HarnessToolMap;}): Promise<{readonly challenges: readonly AuthorizationChallenge[];readonly didCommit: boolean;@@ -308,15 +317,36 @@ async function authorizeCandidate(input: {return { challenges: [], didCommit: false, session };}- const approval = input.tools.get(input.request.action.toolName)?.approval;+ let tools: HarnessToolLookup;+ try {+ tools = await input.restoreTools();+ } catch (error) {+ // A resolver or its infrastructure failed: the entry may still exist, so the request waits.+ logError(log, "approval tools could not be restored", error, {+ requestId: input.request.requestId,+ });+ return failCandidate({ ...input, reason: TEMPORARILY_UNAVAILABLE_APPROVAL_REASON, session });+ }+ const definition = tools.get(input.request.action.toolName);+ // Gone from the restored catalog: no response can approve the call, and it can't run.+ if (definition === undefined) {+ const settled = settleUnavailableCandidate({+ candidateId: input.candidateId,+ reason: UNAVAILABLE_TOOL_APPROVAL_REASON,+ settledAt: input.now,+ state: session.state,+ });+ return {+ challenges: [],+ didCommit: settled.changed,+ session: { ...session, state: settled.state },+ };+ }+ const { approval } = definition;const responsePolicy =approval !== undefined && typeof approval !== "function" ? approval.response : undefined;if (responsePolicy === undefined) {- return failCandidate({- ...input,- reason: "Approval authorization is temporarily unavailable. Please try again.",- session,- });+ return failCandidate({ ...input, reason: TEMPORARILY_UNAVAILABLE_APPROVAL_REASON, session });}try {@@ -471,6 +501,7 @@ function appendSettledResponses(...stepInput,inputResponses: [...(stepInput?.inputResponses ?? []),+ // An unavailable request answers as cancel, so nothing runs; `answer` reports it unavailable....missingSettlements.map((settlement) => ({optionId: settlement.outcome === "allowed" ? "approve" : "cancel",requestId: settlement.requestId,diff --git a/packages/eve/src/harness/hitl/index.ts b/packages/eve/src/harness/hitl/index.tsindex 2945d3ef9..522aef459 100644--- a/packages/eve/src/harness/hitl/index.ts+++ b/packages/eve/src/harness/hitl/index.ts@@ -1,7 +1,6 @@import type { ModelMessage } from "ai";import type { SessionAuthContext } from "#channel/types.js";-import { buildResponseAuthorizationTools } from "#context/build-dynamic-tools.js";import { AuthKey, SessionKey } from "#context/keys.js";import { clearPendingAuthorization } from "#harness/authorization.js";import type { resolveInlineAuthorizationInterrupt } from "#harness/inline-tool-authorization.js";@@ -10,7 +9,12 @@ import { fail } from "#harness/session-machine/transitions.js";import { readTurnState, writeTurnState } from "#harness/session-machine/state.js";import type { StepCoordinates } from "#harness/session-machine/view.js";import type { Step } from "#harness/step/context.js";-import type { HarnessSessionBase, HarnessToolMap, StepInput, StepResult } from "#harness/types.js";+import type {+ HarnessSessionBase,+ HarnessToolLookup,+ StepInput,+ StepResult,+} from "#harness/types.js";import type { RuntimeWorkflowTaskRequest } from "#shared/action-types.js";import type { InputRequest } from "#shared/input.js";import { renderPendingApprovalsInstruction } from "./approval-prompt.js";@@ -56,6 +60,8 @@ export async function parkOnApprovals(readonly messages: readonly ModelMessage[];readonly requests: readonly InputRequest[];readonly tasks: readonly RuntimeWorkflowTaskRequest[];+ /** The entries the step's calls ran, whose approvals the requests ask for. */+ readonly tools: HarnessToolLookup;/** The step made calls the runtime runs, so the turn waits on them instead. */readonly waitsOnRuntime: boolean;},@@ -66,7 +72,7 @@ export async function parkOnApprovals(requests: input.requests,tasks: input.tasks,requester: currentRequester(step),- responseAuthRequiredRequestIds: responsePolicyRequestIds(step, input.requests),+ responseAuthRequiredRequestIds: responsePolicyRequestIds(input.tools, input.requests),});await step.apply(transition, [...step.session.history, ...(transition.commit ?? [])]);if (input.waitsOnRuntime) return { next: null, session: step.session };@@ -125,15 +131,17 @@ export async function enforceBudget(}/**- * What a model call needs from the approvals: the keys `once()` approvals granted, and a note on- * the calls still awaiting approval, which a message may revise.+ * What a model call needs from the approvals: the keys `once()` approvals of `tools` granted, and+ * a note on the calls still awaiting approval, which a message may revise.*/-export function humanInputContext(step: Step): {+export function humanInputContext(+ step: Step,+ tools: HarnessToolLookup,+): {readonly approvedTools: ReadonlySet<string>;readonly pendingApprovalsNote?: string;} {const view = step.view();- const tools = responseTools(step);return {approvedTools: grantedApprovalKeys(view, (request) =>tools.get(request.action.toolName)?.approvalKey?.(request.action.input),@@ -156,19 +164,14 @@ export function discardClearedHumanInput<T extends HarnessSessionBase>(session:return { ...session, state: Object.keys(state).length > 0 ? state : undefined };}-function responseTools(step: Step): HarnessToolMap {- return buildResponseAuthorizationTools({ authoredTools: step.config.tools, context: step.ctx });-}-/*** The approvals whose tool defines a response policy. Every park records them, so no Approve or* Cancel of such an approval skips the policy.*/function responsePolicyRequestIds(- step: Step,+ tools: HarnessToolLookup,requests: readonly InputRequest[],): readonly string[] {- const tools = responseTools(step);return requests.filter((request) => {const approval = tools.get(request.action.toolName)?.approval;diff --git a/packages/eve/src/harness/hitl/intake.ts b/packages/eve/src/harness/hitl/intake.tsindex 2d9d4ad1a..59555a56c 100644--- a/packages/eve/src/harness/hitl/intake.ts+++ b/packages/eve/src/harness/hitl/intake.ts@@ -1,5 +1,6 @@-import { buildResponseAuthorizationTools } from "#context/build-dynamic-tools.js";-import { collectDeferredCalls } from "#harness/coordination.js";+import { buildStepCatalog } from "#execution/catalog/step-catalog.js";+import { SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";+import { collectWorkflowCalls } from "#harness/coordination.js";import { resolveInlineAuthorizationInterrupt } from "#harness/inline-tool-authorization.js";import { stepStartedForResolvers } from "#harness/session-machine/resolver-events.js";import {@@ -15,7 +16,7 @@ import { prepareTurnInput } from "#harness/step/intake.js";import { placeTurnInput } from "#harness/step/prompt.js";import { SessionLimitDeclinedError, throwIfTurnAborted } from "#harness/turn-cancellation.js";import { bumpSessionRuntimeUsageLimits } from "#harness/turn-tag-state.js";-import type { HarnessToolMap, StepInput, StepResult } from "#harness/types.js";+import type { HarnessToolLookup, StepInput, StepResult } from "#harness/types.js";import type { RuntimeWorkflowTaskRequest } from "#shared/action-types.js";import {answer,@@ -61,8 +62,8 @@ export type HumanInputIntake =*/export interface ApprovedWork {readonly limit?: { readonly granted: boolean };- /** The tools a parked step's calls run with: those of the step that asked. */- readonly toolsOf: (step: SuspendedStep | undefined) => HarnessToolMap;+ /** The entries a parked step's calls run with: those of the step that asked. */+ readonly toolsOf: (step: SuspendedStep | undefined) => HarnessToolLookup;}/**@@ -91,9 +92,9 @@ export async function acceptHumanInput(: typed;// Restoring a turn's tools runs its resolvers, so a step's tools are restored once, and again// only after another step's.- const restoredTools = new Map<string, HarnessToolMap>();+ const restoredTools = new Map<string, HarnessToolLookup>();let restoredStep: string | undefined;- const restoreTools = async (parked: SuspendedStep | undefined): Promise<HarnessToolMap> => {+ const restoreTools = async (parked: SuspendedStep | undefined): Promise<HarnessToolLookup> => {const at = parked?.event ?? step.position();const key = `${at.turnId}:${at.stepIndex}`;const restored = restoredTools.get(key);@@ -111,7 +112,12 @@ export async function acceptHumanInput(messages: step.projectHistory(step.session.history),});}- const tools = buildResponseAuthorizationTools({ authoredTools: config.tools, context: ctx });+ const tools = buildStepCatalog({+ agentTools: config.tools,+ ctx,+ endsTurn: false,+ session: step.session,+ });restoredTools.set(key, tools);restoredStep = key;return tools;@@ -138,6 +144,8 @@ export async function acceptHumanInput(.get(request.action.toolName)?.approvalKey?.(request.action.input),delivery: coordinated.stepInput,+ searchable: (request) =>+ toolsOf(stepForRequest(step.view(), request.requestId)).get(SEARCH_TOOL_NAME) !== undefined,policy: {...coordinated,audit: getApprovalAuditState(step.session.state),@@ -257,7 +265,7 @@ export async function dispatchApprovedWorkflows(step: Step, work: ApprovedWork):);if (requests.length === 0) continue;approved.push(...requests);- const deferred = collectDeferredCalls({+ const dispatched = collectWorkflowCalls({session: step.session,toolCalls: requests.map(({ action }) => ({input: action.input,@@ -267,8 +275,8 @@ export async function dispatchApprovedWorkflows(step: Step, work: ApprovedWork):tools,turnId: parked.event.turnId,});- step.session = deferred.session;- tasks.push(...deferred.workflowRequests);+ step.session = dispatched.session;+ tasks.push(...dispatched.workflowRequests);}if (tasks.length === 0) return false;await step.apply(@@ -297,8 +305,10 @@ export async function runApprovedLocalCalls(});const approved = local.flatMap(({ requests }) => requests);step.frameworkToolNames = new Set(- local.flatMap(({ tools }) =>- [...tools].filter(([, tool]) => tool.frameworkTool === true).map(([name]) => name),+ local.flatMap(({ requests, tools }) =>+ requests+ .map((request) => request.action.toolName)+ .filter((name) => tools.get(name)?.frameworkTool === true),),);setApprovedCallCallers(approved, step.session.state);diff --git a/packages/eve/src/harness/input-extraction.integration.test.ts b/packages/eve/src/harness/input-extraction.integration.test.tsindex dd8d45910..4115ede59 100644--- a/packages/eve/src/harness/input-extraction.integration.test.ts+++ b/packages/eve/src/harness/input-extraction.integration.test.ts@@ -66,7 +66,12 @@ describe("AI SDK 7 automatic approval extraction", () => {expect(step.content).toContainEqual(expect.objectContaining({ isAutomatic: true, type: "tool-approval-request" }),);- expect(extractToolApprovalInputRequests({ content: step.content })).toEqual([]);+ expect(+ extractToolApprovalInputRequests({+ tools: new Map(),+ content: step.content,+ }),+ ).toEqual([]);});it("keeps automatic denial records out of the human input queue", async () => {@@ -83,14 +88,24 @@ describe("AI SDK 7 automatic approval extraction", () => {type: "tool-approval-response",}),);- expect(extractToolApprovalInputRequests({ content: step.content })).toEqual([]);+ expect(+ extractToolApprovalInputRequests({+ tools: new Map(),+ content: step.content,+ }),+ ).toEqual([]);});it("continues to queue unresolved user approval requests", async () => {const { execute, step } = await generateToolCall("user-approval");expect(execute).not.toHaveBeenCalled();- expect(extractToolApprovalInputRequests({ content: step.content })).toEqual([+ expect(+ extractToolApprovalInputRequests({+ tools: new Map(),+ content: step.content,+ }),+ ).toEqual([expect.objectContaining({action: expect.objectContaining({ callId: "call-1", toolName: "bash" }),requestId: expect.any(String),diff --git a/packages/eve/src/harness/input-extraction.test.ts b/packages/eve/src/harness/input-extraction.test.tsindex 5d0b6ca79..a1496433b 100644--- a/packages/eve/src/harness/input-extraction.test.ts+++ b/packages/eve/src/harness/input-extraction.test.ts@@ -5,6 +5,7 @@ import { extractToolApprovalInputRequests } from "#harness/input-extraction.js";describe("extractToolApprovalInputRequests", () => {it("extracts a tool approval request from content parts", () => {const result = extractToolApprovalInputRequests({+ tools: new Map(),content: [{approvalId: "approval-1",@@ -42,6 +43,7 @@ describe("extractToolApprovalInputRequests", () => {it("extracts an approval request from a sibling tool call", () => {const result = extractToolApprovalInputRequests({+ tools: new Map(),content: [{input: { command: "rm -rf /tmp" },@@ -80,6 +82,7 @@ describe("extractToolApprovalInputRequests", () => {it("skips automatic approval decisions", () => {const result = extractToolApprovalInputRequests({+ tools: new Map(),content: [{approvalId: "approval-1",@@ -100,6 +103,7 @@ describe("extractToolApprovalInputRequests", () => {it("skips approval requests without matching tool-call data", () => {const result = extractToolApprovalInputRequests({+ tools: new Map(),content: [{approvalId: "approval-1",@@ -114,6 +118,7 @@ describe("extractToolApprovalInputRequests", () => {it("skips approval requests for excluded tool calls before parsing input", () => {const result = extractToolApprovalInputRequests({+ tools: new Map(),content: [{input: [],@@ -135,6 +140,7 @@ describe("extractToolApprovalInputRequests", () => {it("skips non-approval content parts", () => {const result = extractToolApprovalInputRequests({+ tools: new Map(),content: [{ text: "Some text.", type: "text" },{@@ -150,6 +156,11 @@ describe("extractToolApprovalInputRequests", () => {});it("returns an empty array for empty content", () => {- expect(extractToolApprovalInputRequests({ content: [] })).toEqual([]);+ expect(+ extractToolApprovalInputRequests({+ tools: new Map(),+ content: [],+ }),+ ).toEqual([]);});});diff --git a/packages/eve/src/harness/input-extraction.ts b/packages/eve/src/harness/input-extraction.tsindex 25dfa6d40..2642231a1 100644--- a/packages/eve/src/harness/input-extraction.ts+++ b/packages/eve/src/harness/input-extraction.ts@@ -1,9 +1,11 @@import type { ContentPart, ToolSet } from "ai";import { z } from "#compiled/zod/index.js";-import { toolCallDisplayName } from "#execution/tools/connection-target.js";+import { projectToolStartLabel } from "#harness/action-presentation.js";import type { InputRequest } from "#shared/input.js";import { createRuntimeToolCallActionFromToolCall } from "#harness/tool-call-action.js";+import type { HarnessToolLookup } from "#harness/types.js";+import { displayTitle } from "#shared/display-name.js";// Persisted history parts lose AI SDK typing on the storage round trip. The// schemas are the single source for the runtime narrowing and the static@@ -33,11 +35,13 @@ const ToolApprovalRequestSchema = z.object({/*** Extracts tool approval input requests from AI SDK content parts that- * contain `tool-approval-request` entries.+ * contain `tool-approval-request` entries. Each prompt names the call by the+ * label its entry in `tools` gives it.*/export function extractToolApprovalInputRequests(input: {readonly content: readonly ContentPart<ToolSet>[];readonly excludedCallIds?: ReadonlySet<string>;+ readonly tools: HarnessToolLookup;}): InputRequest[] {return extractApprovalRequests(input);}@@ -48,6 +52,7 @@ export function extractToolApprovalInputRequests(input: {function extractApprovalRequests(input: {readonly content: readonly unknown[];readonly excludedCallIds?: ReadonlySet<string>;+ readonly tools: HarnessToolLookup;}): InputRequest[] {const requests: InputRequest[] = [];const toolCallsById = new Map<string, ToolCallDescriptor>();@@ -83,8 +88,12 @@ function extractApprovalRequests(input: {continue;}+ const action = createRuntimeToolCallActionFromToolCall({ toolCall });+ const label =+ projectToolStartLabel(input.tools.get(action.toolName), action.input) ??+ displayTitle(action.toolName);requests.push({- action: createRuntimeToolCallActionFromToolCall({ toolCall }),+ action,allowFreeform: false,display: "confirmation",kind: "tool-approval",@@ -92,7 +101,7 @@ function extractApprovalRequests(input: {{ id: "approve", label: "Approve" },{ id: "cancel", label: "Cancel" },],- prompt: `Approve ${toolCallDisplayName(toolCall.toolName, toolCall.input)}?`,+ prompt: `Approve ${label}?`,requestId: approval.approvalId,});}diff --git a/packages/eve/src/harness/input-request-resolution.test.ts b/packages/eve/src/harness/input-request-resolution.test.tsnew file mode 100644index 000000000..3b53023ad--- /dev/null+++ b/packages/eve/src/harness/input-request-resolution.test.ts@@ -0,0 +1,14 @@+import { describe, expect, it } from "vitest";++import { unavailableToolMessage } from "#harness/input-request-resolution.js";++describe("unavailableToolMessage", () => {+ it("points at eve__search only for an agent that has it", () => {+ expect(unavailableToolMessage("deploy", true)).toBe(+ 'The tool "deploy" is no longer available, so the call didn\'t run. If the task still needs it, find an available tool with eve__search and make a new call.',+ );+ expect(unavailableToolMessage("deploy", false)).toBe(+ 'The tool "deploy" is no longer available, so the call didn\'t run. If the task still needs it, make a new call with an available tool.',+ );+ });+});diff --git a/packages/eve/src/harness/input-request-resolution.ts b/packages/eve/src/harness/input-request-resolution.tsindex f3ba00e24..17e8c66e3 100644--- a/packages/eve/src/harness/input-request-resolution.ts+++ b/packages/eve/src/harness/input-request-resolution.ts@@ -1,4 +1,7 @@+import { createRuntimeToolResultFromValue } from "#harness/action-result-helpers.js";+import type { SettledCall } from "#harness/session-machine/transitions.js";import type { StepCoordinates as PendingInputBatchEvent } from "#harness/session-machine/view.js";+import { SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";import type { InputRequest, InputResponse } from "#shared/input.js";const IGNORED_INPUT_REASON = "Ignored because the user continued without responding.";@@ -64,3 +67,35 @@ export function resolveApprovalOutcome(response: InputResponse | undefined): {status: "invalid",};}++/**+ * What the model reads when a call's tool went away before the call could run.+ * `searchable` says whether the agent has `eve__search` to find another.+ */+export function unavailableToolMessage(toolName: string, searchable: boolean): string {+ const next = searchable+ ? `find an available tool with ${SEARCH_TOOL_NAME} and make a new call`+ : "make a new call with an available tool";+ return `The tool "${toolName}" is no longer available, so the call didn't run. If the task still needs it, ${next}.`;+}++/**+ * A call that ends without running: its failed runtime result, which the lifecycle reports at the+ * step's coordinates, and the `error-text` result the model reads.+ */+export function failedCall(call: {+ readonly callId: string;+ readonly message: string;+ readonly toolName: string;+}): Required<SettledCall> {+ const { callId, message, toolName } = call;+ return {+ part: {+ output: { type: "error-text", value: message },+ toolCallId: callId,+ toolName,+ type: "tool-result",+ },+ result: createRuntimeToolResultFromValue({ callId, isError: true, output: message, toolName }),+ };+}diff --git a/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts b/packages/eve/src/harness/issue-3494-adversarial.integration.test.tsindex a4d1be99e..8d6f26495 100644--- a/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts+++ b/packages/eve/src/harness/issue-3494-adversarial.integration.test.ts@@ -13,6 +13,7 @@ import { SessionKey } from "#context/keys.js";import { createProjectionRecorder } from "#internal/testing/session-projection-recorder.js";import { runtimeWait, storedProjection, suspendedSteps } from "#harness/session-machine/view.js";import { openInputs } from "#protocol/session-projection.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import { createToolLoopHarness } from "#harness/tool-loop.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";import type { HarnessSession, StepInput, StepResult } from "#harness/types.js";@@ -286,9 +287,9 @@ for (const beside of ["workflow", "gateA"] as const) {type: "object",};const f = fixture(`final-output-beside-${beside}`, false, undefined, schema);- f.script.push([{ toolName: beside }, { toolName: "final_output", input: { title: "Early" } }]);+ f.script.push([{ toolName: beside }, { toolName: REPLY_TOOL_NAME, input: { title: "Early" } }]);await f.drive({ message: "Run it, then report." });- f.script.push([{ toolName: "final_output", input: { title: "Done" } }]);+ f.script.push([{ toolName: REPLY_TOOL_NAME, input: { title: "Done" } }]);const result =beside === "workflow" ? await f.finishRuntime() : await f.drive(f.respond("gateA"));@@ -296,7 +297,7 @@ for (const beside of ["workflow", "gateA"] as const) {expect(suspendedSteps(f.session.state)).toEqual([]);const rereads = JSON.stringify(f.prompts.at(-1));expect(rereads).toContain(beside === "workflow" ? "runtime-RESULT" : "gateA-RESULT");- expect(rereads).toContain("Your final output wasn't delivered");+ expect(rereads).toContain("Your reply wasn't delivered");});}diff --git a/packages/eve/src/harness/model-call/call.ts b/packages/eve/src/harness/model-call/call.tsindex 63c4aa893..c3828752c 100644--- a/packages/eve/src/harness/model-call/call.ts+++ b/packages/eve/src/harness/model-call/call.ts@@ -10,13 +10,15 @@ import {} from "ai";import { AuthKey, HistoryStateKey } from "#context/keys.js";+import type { StepCatalog } from "#execution/catalog/step-catalog.js";import { workingTaskIds } from "#execution/tasks/model-step.js";import {hydrateSandboxAttachments,moveToolResultFilesToUserMessages,} from "#harness/attachment-staging.js";import { emitStreamContent } from "#harness/emission.js";-import { FINAL_OUTPUT_TOOL_NAME } from "#harness/final-output.js";+import { toEntryStep, toEntryStream, toEntryTelemetry } from "#harness/execute-call.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import type { GenerationSteering } from "#harness/generation-steering.js";import { interruptStreamOnFailure } from "#harness/interruptible-stream.js";import type { HarnessModelMessage, UserModelMessage } from "#harness/messages.js";@@ -65,7 +67,7 @@ import {withAccumulatedResponseMessages,} from "./response.js";import { runModelCallWithRetries } from "./retry.js";-import { logToolExecutionError, type ModelTools, prepareModelTools } from "./tools.js";+import { logToolExecutionError, prepareModelTools } from "./tools.js";import { extractGatewayCostUsd, extractTokenUsageDelta } from "./usage.js";const environment = process.env.NODE_ENV ?? "unknown";@@ -83,6 +85,8 @@ export interface ModelCallOptions {/** What a model step's caller needs from the step. */interface ModelCallerInput {+ /** The step's catalog: what the model can call, and what each call runs. */+ readonly catalog: StepCatalog;readonly model: LanguageModel;readonly generation: GenerationSteering;readonly hidesHeldText: boolean;@@ -107,8 +111,6 @@ export class ModelCaller {private projectedMessages: HarnessModelMessage[];/** The latest attempt's request; its history is what the step commits. */request: RequestMessages;- /** The latest attempt's tools, which decide how the step's calls defer and end the turn. */- tools: ModelTools | undefined;/** The messages the latest attempt sent, after attachments were hydrated. */modelMessages: ModelMessage[] = [];requestEnvelopeTokens = 0;@@ -132,7 +134,7 @@ export class ModelCaller {step.config.runtimeIdentity,);this.projectedMessages = input.projectedMessages;- this.request = this.buildRequest(step.config.tools);+ this.request = this.buildRequest();}/** Calls the model, retrying transient failures. */@@ -207,9 +209,9 @@ export class ModelCaller {this.interruptedUsage = undefined;}- private buildRequest(coordinationTools: ModelTools["coordinationTools"]): RequestMessages {+ private buildRequest(): RequestMessages {return requestMessages(this.step, {- coordinationTools,+ catalog: this.input.catalog,hidesHeldText: this.input.hidesHeldText,messages: this.prompt.messages,pendingApprovalsNote: this.input.pendingApprovalsNote,@@ -227,26 +229,25 @@ export class ModelCaller {});}- private async prepare(options: ModelCallOptions): Promise<ModelTools> {+ private async prepare(options: ModelCallOptions): Promise<ToolSet> {const tools = await prepareModelTools(this.step, {- approvedTools: this.input.approvedTools,+ catalog: this.input.catalog,disabledProviderTools: options.disabledProviderTools,generation: this.input.generation,profile: this.profile,});- this.tools = tools;- this.request = this.buildRequest(tools.coordinationTools);+ this.request = this.buildRequest();this.requestEnvelopeTokens = await estimateRequestEnvelope({history: this.projectedMessages,instructions: this.instructions(options.extraSystemNote),messages: withTrailingUserNote(this.request.nonSystemMessages, options.trailingUserNote),- tools: tools.effectiveTools,+ tools,});return tools;}/** Compacts the prompt when it's over the threshold, then rebuilds what depends on it. */- private async compact(options: ModelCallOptions, tools: ModelTools): Promise<ModelTools> {+ private async compact(options: ModelCallOptions, tools: ToolSet): Promise<ToolSet> {const { step, prompt } = this;const { config } = step;let compaction: Awaited<ReturnType<typeof maybeCompact>>;@@ -292,7 +293,7 @@ export class ModelCaller {unsettledActionToolNames: Map<string, string>,): Promise<HarnessStepResult> {const { step } = this;- const { generation, model } = this.input;+ const { catalog, generation, model } = this.input;const tools = await this.compact(options, await this.prepare(options));// New announcements join durable history, so they must not inflate the// envelope baseline and hide instruction growth on the next step.@@ -348,7 +349,8 @@ export class ModelCaller {usage: event.usage,});},- onToolExecutionEnd: logToolExecutionError,+ onToolExecutionEnd: (event: Parameters<typeof logToolExecutionError>[0]) =>+ logToolExecutionError(event, catalog.resolve),// Replaces the AI SDK's default `console.error`; the harness reports failures as events.onError(event: { error: unknown }) {if (generation.interrupted) return;@@ -363,9 +365,13 @@ export class ModelCaller {reasoning: step.session.agent.modelReference?.reasoning ?? step.session.agent.reasoning,runtimeContext,stopWhen: isStepCount(1),- telemetry: attempt?.telemetry,- toolApproval: buildToolApproval(tools.modelTools, generation.signal),- tools: tools.effectiveTools,+ telemetry: toEntryTelemetry(attempt?.telemetry, catalog.resolve),+ toolApproval: buildToolApproval({+ abortSignal: generation.signal,+ approvedTools: this.input.approvedTools,+ resolve: catalog.resolve,+ }),+ tools,};const agent = new ToolLoopAgent(settings);@@ -373,13 +379,7 @@ export class ModelCaller {const result =step.emit === undefined? await this.generate(agent, callMessages, hooks.stepResult)- : await this.stream(- agent,- callMessages,- hooks.stepResult,- tools,- unsettledActionToolNames,- );+ : await this.stream(agent, callMessages, hooks.stepResult, unsettledActionToolNames);await attempt?.complete();return result;} catch (error) {@@ -395,12 +395,11 @@ export class ModelCaller {agent: ToolLoopAgent,messages: ModelMessage[],stepResultPromise: Promise<HarnessStepResult>,- tools: ModelTools,unsettledActionToolNames: Map<string, string>,): Promise<HarnessStepResult> {const { step } = this;- const { generation } = this.input;- const excludedActionToolNames = new Set([FINAL_OUTPUT_TOOL_NAME]);+ const { catalog, generation } = this.input;+ const excludedActionToolNames = new Set([REPLY_TOOL_NAME]);const streamResult = await agent.stream({ abortSignal: generation.signal, messages });const {emittedActionCallIds,@@ -411,26 +410,29 @@ export class ModelCaller {} = await emitStreamContent(step.publish,step.position(),- interruptStreamOnFailure(streamResult.fullStream, generation.signal),+ toEntryStream(+ interruptStreamOnFailure(streamResult.fullStream, generation.signal),+ catalog.resolve,+ ),{excludedActionToolNames,hidesHeldText: this.input.hidesHeldText && workingTaskIds(step.session).length > 0,- tools: tools.presentationTools,+ tools: catalog,unsettledActionToolNames,},);throwIfTurnAborted(step.config.abortSignal);generation.check();- const [stepResult, accumulatedResponseMessages] = await Promise.all([- stepResultPromise,- streamResult.responseMessages,- ]);+ const [stepResult, accumulatedResponseMessages] = toEntryStep(+ ...(await Promise.all([stepResultPromise, streamResult.responseMessages])),+ catalog.resolve,+ );assertUsableResponse(stepResult,accumulatedResponseMessages,inlineAuthorizationResults.length > 0 || trailingInlineToolResultParts.length > 0,);- const skipped = answerSkippedToolCalls(stepResult, tools.effectiveTools);+ const skipped = answerSkippedToolCalls(stepResult, catalog);await emitStepActions(step.publish,step.position(),@@ -448,7 +450,7 @@ export class ModelCaller {excludedActionCallIds: invalidInputToolCallIds,excludedActionToolNames,handledInlineToolResultCallIds,- tools: tools.presentationTools,+ tools: catalog,},);const toolResultsByCallId = new Map(@@ -463,10 +465,7 @@ export class ModelCaller {return withAccumulatedResponseMessages({invalidInputToolCallIds,responseMessages: appendMissingToolResultMessages({- append: [- ...trailingInlineToolResultParts,- ...answerSkippedToolCalls(stepResult, tools.effectiveTools),- ],+ append: [...trailingInlineToolResultParts, ...answerSkippedToolCalls(stepResult, catalog)],responseMessages: accumulatedResponseMessages,}),stepResult,@@ -480,16 +479,20 @@ export class ModelCaller {messages: ModelMessage[],stepResultPromise: Promise<HarnessStepResult>,): Promise<HarnessStepResult> {- const { generation } = this.input;+ const { catalog, generation } = this.input;const generateResult = await agent.generate({ abortSignal: generation.signal, messages });throwIfTurnAborted(this.step.config.abortSignal);generation.check();- const stepResult = await stepResultPromise;- assertUsableResponse(stepResult, generateResult.responseMessages, false);+ const [stepResult, responseMessages] = toEntryStep(+ await stepResultPromise,+ generateResult.responseMessages,+ catalog.resolve,+ );+ assertUsableResponse(stepResult, responseMessages, false);return withAccumulatedResponseMessages({responseMessages: appendMissingToolResultMessages({- append: answerSkippedToolCalls(stepResult, this.tools!.effectiveTools),- responseMessages: generateResult.responseMessages,+ append: answerSkippedToolCalls(stepResult, catalog),+ responseMessages,}),stepResult,});diff --git a/packages/eve/src/harness/model-call/request.ts b/packages/eve/src/harness/model-call/request.tsindex 7849776ca..111d1d26b 100644--- a/packages/eve/src/harness/model-call/request.ts+++ b/packages/eve/src/harness/model-call/request.ts@@ -1,10 +1,11 @@import type { ModelMessage, SystemModelMessage } from "ai";import { buildDynamicInstructionMessages } from "#context/dynamic-instruction-lifecycle.js";-import { PendingSkillAnnouncementKey } from "#context/dynamic-skill-lifecycle.js";-import { HistoryStateKey } from "#context/keys.js";+import { DynamicSkillManifestKey, HistoryStateKey } from "#context/keys.js";+import { catalogAnnouncements } from "#execution/catalog/listing.js";+import type { StepCatalog } from "#execution/catalog/step-catalog.js";+import { dynamicSkillAnnouncements } from "#execution/skills/instructions.js";import { taskSystemMessages } from "#execution/tasks/model-step.js";-import { getPendingAnnouncements } from "#harness/announcements.js";import { createCurrentMessages } from "#harness/current-messages.js";import {createFrameworkUserMessage,@@ -13,7 +14,7 @@ import {} from "#harness/messages.js";import { applySystemCacheBreakpoint } from "#harness/prompt-cache.js";import type { Step } from "#harness/step/context.js";-import type { HarnessSession, HarnessToolMap } from "#harness/types.js";+import type { HarnessSession } from "#harness/types.js";export type RequestMessages = ReturnType<typeof createCurrentMessages>;@@ -28,7 +29,7 @@ export function requestMessages(readonly messages: readonly HarnessModelMessage[];readonly projectedMessages: readonly HarnessModelMessage[];readonly turnMessages: readonly UserModelMessage[];- readonly coordinationTools: HarnessToolMap;+ readonly catalog: StepCatalog;readonly hidesHeldText: boolean;readonly pendingApprovalsNote: string | undefined;},@@ -41,11 +42,12 @@ export function requestMessages(});if (ctx !== undefined) messages.addSystem(buildDynamicInstructionMessages(ctx));messages.addSystem(- taskSystemMessages(input.coordinationTools, { finalReplyOnly: input.hidesHeldText }),+ taskSystemMessages(input.catalog.offersTasks, { finalReplyOnly: input.hidesHeldText }),);+ const announced = ctx?.get(HistoryStateKey)?.announcements;messages.addAnnouncements({- availableSkills: ctx?.get(PendingSkillAnnouncementKey),- keyed: getPendingAnnouncements(ctx),+ ...catalogAnnouncements(input.catalog, announced),+ ...dynamicSkillAnnouncements(ctx?.get(DynamicSkillManifestKey), announced),});if (input.pendingApprovalsNote !== undefined) {messages.add(input.pendingApprovalsNote, "context.state", { cacheFriendly: false });diff --git a/packages/eve/src/harness/model-call/response.ts b/packages/eve/src/harness/model-call/response.tsindex 743e4091d..718be573e 100644--- a/packages/eve/src/harness/model-call/response.ts+++ b/packages/eve/src/harness/model-call/response.ts@@ -4,7 +4,9 @@ import {} from "#harness/tool-call-input-errors.js";import type { ModelMessage, ToolSet, TypedToolCall, TypedToolResult } from "ai";+import { historyCallNames } from "#harness/execute-call.js";import type { HarnessStepResult } from "#harness/step-hooks.js";+import type { HarnessToolLookup } from "#harness/types.js";type StepResponseMessage = HarnessStepResult["response"]["messages"][number];type ToolResponsePart = Extract<ModelMessage, { role: "tool" }>["content"][number];@@ -75,7 +77,10 @@ export function appendMissingToolResultMessages(input: {readonly responseMessages: readonly StepResponseMessage[];}): StepResponseMessage[] {const existingCallIds = extractToolResultCallIds(input.responseMessages);- const append = input.append.filter((part) => !existingCallIds.has(part.toolCallId));+ const callNames = historyCallNames(input.responseMessages);+ const append = input.append+ .filter((part) => !existingCallIds.has(part.toolCallId))+ .map((part) => ({ ...part, toolName: callNames.get(part.toolCallId) ?? part.toolName }));return [...input.responseMessages,@@ -108,7 +113,10 @@ export function extractToolResultCallIds(messages: readonly ModelMessage[]): Reareturn callIds;}-export function answerSkippedToolCalls(step: HarnessStepResult, tools: ToolSet): ToolResultPart[] {+export function answerSkippedToolCalls(+ step: HarnessStepResult,+ tools: HarnessToolLookup,+): ToolResultPart[] {const { finishReason } = step;if (finishReason === "stop" || finishReason === "tool-calls") return [];@@ -124,7 +132,7 @@ export function answerSkippedToolCalls(step: HarnessStepResult, tools: ToolSet):return ((step.toolCalls ?? []) as TypedToolCall<ToolSet>[]).filter((toolCall) =>- tools[toolCall.toolName]?.execute !== undefined &&+ tools.get(toolCall.toolName)?.execute !== undefined &&toolCall.providerExecuted !== true &&!isInvalidToolCall(toolCall) &&getInvalidToolCallInputError({ toolCall }) === undefined &&diff --git a/packages/eve/src/harness/model-call/run.ts b/packages/eve/src/harness/model-call/run.tsindex 5f0921b0a..7f818401e 100644--- a/packages/eve/src/harness/model-call/run.ts+++ b/packages/eve/src/harness/model-call/run.ts@@ -1,6 +1,7 @@import type { LanguageModel, ModelMessage } from "ai";import { HistoryStateKey } from "#context/keys.js";+import { buildStepCatalog, type StepCatalog } from "#execution/catalog/step-catalog.js";import type { GenerationSteering } from "#harness/generation-steering.js";import { type HarnessModelMessage, validateHarnessModelMessages } from "#harness/messages.js";import {@@ -25,14 +26,10 @@ import {} from "#harness/step/prompt.js";import type { HarnessStepResult } from "#harness/step-hooks.js";import { throwIfTurnAborted } from "#harness/turn-cancellation.js";-import {- type HarnessToolMap,- requireSessionModelReference,- type StepResult,-} from "#harness/types.js";+import { requireSessionModelReference, type StepResult } from "#harness/types.js";import type { InstrumentationAttempt } from "#instrumentation/runtime.js";import { ModelCaller } from "./call.js";-import type { EndsTurnTools } from "./tools.js";+import { type EndsTurnTools, endsTurnTools, frameworkToolNames } from "./tools.js";import { reportModelCallFailure } from "./failure.js";import { resolveEffectiveRuntimeModel } from "./model.js";import { recoverModelCall } from "./recovery.js";@@ -40,7 +37,8 @@ import { recordModelUsage } from "./usage.js";/** A model step's response, for the session to act on. */export interface ModelResponse {- readonly coordinationTools: HarnessToolMap;+ /** The step's catalog, which decides what each of the response's calls runs. */+ readonly catalog: StepCatalog;/** How many prompt messages the model read, when no client context was spliced in. */readonly durableModelPromptMessageCount?: number;/** Tools that can end the turn in this step, with their `endsTurn` option. */@@ -102,9 +100,20 @@ export async function runModelStep(prompt = await buildPrompt(step, input.turn);projectedMessages = projectPrompt(step, prompt);}- const { approvedTools, pendingApprovalsNote } = humanInputContext(step);+ // Dynamic tools and subagents resolved when the step started, so the step's calls resolve once,+ // against this catalog.+ const endsTurn = !step.hasDelegatedCaller && step.session.outputSchema === undefined;+ const catalog = buildStepCatalog({+ agentTools: step.config.tools,+ ctx: step.ctx,+ endsTurn,+ session: step.session,+ });+ step.frameworkToolNames = frameworkToolNames(catalog);+ const { approvedTools, pendingApprovalsNote } = humanInputContext(step, catalog);const caller = new ModelCaller(step, prompt, {approvedTools,+ catalog,generation,hidesHeldText: input.hidesHeldText,model: model.model,@@ -142,13 +151,13 @@ export async function runModelStep(try {generation.check();stepResult = await input.onResponse({- coordinationTools: caller.tools?.coordinationTools ?? step.config.tools,+ catalog,// Usage measures what the model read only when no client context was spliced in.durableModelPromptMessageCount:prompt.clientContext === undefined || prompt.clientContext.messages.length === 0? caller.modelMessages.length: undefined,- endsTurnTools: caller.tools?.endsTurnTools ?? new Map(),+ endsTurnTools: endsTurnTools(catalog, endsTurn),promptMessages: caller.request.history,requestEnvelopeTokens: caller.requestEnvelopeTokens,result,diff --git a/packages/eve/src/harness/model-call/tools.ts b/packages/eve/src/harness/model-call/tools.tsindex 45e1e01ee..0f2e714ea 100644--- a/packages/eve/src/harness/model-call/tools.ts+++ b/packages/eve/src/harness/model-call/tools.ts@@ -1,127 +1,52 @@import type { ToolSet } from "ai";-import { buildDynamicTools } from "#context/build-dynamic-tools.js";-import { buildDynamicSubagentTools } from "#context/dynamic-subagent-lifecycle.js";-import { withTaskTools } from "#execution/tasks/model-step.js";-import { getAdvertisedTools } from "#harness/advertised-tools.js";+import type { StepCatalog } from "#execution/catalog/step-catalog.js";+import { dispatchesAfterStep } from "#harness/execute-call.js";+import { SKILL_ENTRY_NAME } from "#protocol/catalog-tools.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";-import { buildFinalOutputTool, FINAL_OUTPUT_TOOL_NAME } from "#harness/final-output.js";+import { buildFinalOutputTool } from "#harness/final-output.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import type { GenerationSteering } from "#harness/generation-steering.js";import type { ModelProfile } from "#harness/model-profile.js";import { applyLastToolCacheBreakpoint } from "#harness/prompt-cache.js";import type { Step } from "#harness/step/context.js";-import { buildToolSetFromDefinitions, buildToolSetWithProviderTools } from "#harness/tools.js";+import { buildToolSetWithProviderTools } from "#harness/tools.js";import { isTurnCancellation } from "#harness/turn-cancellation.js";-import type { HarnessToolMap } from "#harness/types.js";+import type { CallResolver } from "#harness/types.js";import { createLogger, logError } from "#internal/logging.js";import { toModelSchema } from "#tools/schema.js";const log = createLogger("harness.tool-loop");-/** The tools one model call offers, and the views of them the step reads afterwards. */-export interface ModelTools {- /** What the AI SDK receives. */- readonly effectiveTools: ToolSet;- readonly modelTools: ToolSet;- /** Every tool the model can call, by name, for presenting its calls. */- readonly presentationTools: HarnessToolMap;- /** The advertised harness tools; they decide which calls defer to the runtime. */- readonly coordinationTools: HarnessToolMap;- /** Tools that can end the turn, with their `endsTurn` option. */- readonly endsTurnTools: EndsTurnTools;-}-/**- * Assembles the tools one model call offers: authored and dynamic subagent tools, task tools,- * provider tools, dynamic tools (which override a same-named authored tool), and `final_output`- * when the turn asks for a structured result.+ * Assembles the tools one model call offers: the catalog's listed entries with provider tools in+ * place, and `eve__reply` when the turn asks for a structured result.*/export async function prepareModelTools(step: Step,input: {- readonly approvedTools: ReadonlySet<string>;+ readonly catalog: StepCatalog;readonly disabledProviderTools?: ReadonlySet<string>;readonly generation: GenerationSteering;readonly profile: ModelProfile;},-): Promise<ModelTools> {- const { ctx } = step;- const { approvedTools, disabledProviderTools, profile } = input;- const coordinationTools = withTaskTools(- getAdvertisedTools({- session: step.session,- tools: buildHarnessToolsWithDynamicSubagents(step.config.tools, ctx),- }),- );-- const flatTools = await buildToolSetWithProviderTools({- approvedTools,- disabledProviderTools,+): Promise<ToolSet> {+ const { catalog, profile } = input;+ const modelTools = await buildToolSetWithProviderTools({+ describe: catalog.describe,+ disabledProviderTools: input.disabledProviderTools,profile,- tools: coordinationTools,+ resolve: catalog.resolve,+ tools: catalog.advertised,});-- // Stream emitters resolve label callbacks by tool name, so they need the- // dynamic tools the model can call, not only the authored ones.- const presentationTools = new Map(coordinationTools);- if (ctx !== undefined) {- const dynamicTools = getAdvertisedTools({- session: step.session,- tools: buildDynamicTools(ctx),- });- const dynamicToolSet = buildToolSetFromDefinitions({- approvedTools,- disabledProviderTools,- tools: dynamicTools,- });- for (const [name, toolDefinition] of Object.entries(dynamicToolSet)) {- if (coordinationTools.get(name)?.workflowId !== undefined) {- throw new Error(- `Dynamic tool "${name}" collides with a coordination-visible deferred tool.`,- );- }- flatTools[name] = toolDefinition;- }- // Match `buildToolSetFromDefinitions`: the first dynamic definition of a- // name (step, then turn, then session) wins, and still overrides authored.- const presentedDynamicNames = new Set<string>();- for (const tool of dynamicTools) {- if (presentedDynamicNames.has(tool.name)) continue;- presentedDynamicNames.add(tool.name);- presentationTools.set(tool.name, tool);- }- }-if (step.session.outputSchema !== undefined) {- flatTools[FINAL_OUTPUT_TOOL_NAME] = buildFinalOutputTool(step.session.outputSchema);- }-- const advertised = await getAdvertisedTools({- modelTools: flatTools,- session: step.session,- tools: coordinationTools,- });- step.session = advertised.session;- const { modelTools } = advertised;- // `endsTurn` applies only in root, unstructured turns. Only a literal- // `true` is described to the model; a function decides from each result.- const endsTurnTools = new Map<string, NonNullable<HarnessToolDefinition["endsTurn"]>>();- if (!step.hasDelegatedCaller && step.session.outputSchema === undefined) {- for (const [name, modelTool] of Object.entries(modelTools)) {- const endsTurn = presentationTools.get(name)?.endsTurn;- if (modelTool.type === "provider" || endsTurn === undefined || endsTurn === false) continue;- endsTurnTools.set(name, endsTurn);- if (endsTurn === true) {- modelTool.description =- `${modelTool.description ?? ""}\n\n${ENDS_TURN_TOOL_NOTE}`.trimStart();- }- }+ modelTools[REPLY_TOOL_NAME] = buildFinalOutputTool(step.session.outputSchema);}const effectiveTools = profile.anthropicCache? applyLastToolCacheBreakpoint(modelTools): modelTools;- for (const tool of Object.values(effectiveTools)) {+ for (const [name, tool] of Object.entries(effectiveTools)) {// Whatever produced this tool, the AI SDK must only receive its own// schema type; see toModelSchema.tool.inputSchema = toModelSchema(tool.inputSchema, "input");@@ -131,16 +56,39 @@ export async function prepareModelTools(const execute = tool.execute;if (execute !== undefined) {tool.execute = (...args) => {- input.generation.protectToolExecution();+ // A call the harness dispatches after the step has no effects yet, so steering can still+ // interrupt it, as it can the same call made directly.+ if (!dispatchesAfterStep(catalog.resolve({ input: args[0], toolName: name }))) {+ input.generation.protectToolExecution();+ }return execute(...args);};}}+ return effectiveTools;+}++/** The entries that can end the turn, with their `endsTurn` option, when `endsTurn` applies. */+export function endsTurnTools(catalog: StepCatalog, applies: boolean): EndsTurnTools {+ if (!applies) return new Map();+ return new Map(+ [...catalog.entries].flatMap(([name, definition]) =>+ definition.endsTurn === undefined || definition.endsTurn === false+ ? []+ : [[name, definition.endsTurn] as const],+ ),+ );+}- step.frameworkToolNames = new Set(- [...presentationTools].filter(([, tool]) => tool.frameworkTool === true).map(([name]) => name),+/** The names of the entries eve provides, which tracing marks as framework tools. */+export function frameworkToolNames(catalog: StepCatalog): ReadonlySet<string> {+ return new Set(+ [+ ...catalog.advertised.values(),+ ...catalog.entries.values(),+ catalog.get(SKILL_ENTRY_NAME),+ ].flatMap((definition) => (definition?.frameworkTool === true ? [definition.name] : [])),);- return { coordinationTools, effectiveTools, endsTurnTools, modelTools, presentationTools };}/**@@ -148,40 +96,26 @@ export async function prepareModelTools(* the `error` is still the original throwable (stack/cause intact),* unlike the message-only `tool-error` part the model later sees.*/-export function logToolExecutionError(event: {- readonly toolCall: { readonly toolName: string; readonly toolCallId: string };- readonly toolOutput: { readonly type: string; readonly error?: unknown };-}): void {+export function logToolExecutionError(+ event: {+ readonly toolCall: {+ readonly input: unknown;+ readonly toolName: string;+ readonly toolCallId: string;+ };+ readonly toolOutput: { readonly type: string; readonly error?: unknown };+ },+ resolve: CallResolver,+): void {// A tool unwinding because its turn was cancelled is the expected outcome// of a user action, not a failure worth an error log.if (event.toolOutput.type !== "tool-error" || isTurnCancellation(event.toolOutput.error)) {return;}logError(log, "tool execution failed", event.toolOutput.error, {- toolName: event.toolCall.toolName,+ toolName: resolve(event.toolCall)?.definition.name ?? event.toolCall.toolName,toolCallId: event.toolCall.toolCallId,});}-function buildHarnessToolsWithDynamicSubagents(- tools: HarnessToolMap,- ctx: Parameters<typeof buildDynamicSubagentTools>[0] | undefined,-): HarnessToolMap {- const effectiveTools = new Map(tools);- if (ctx === undefined) return effectiveTools;- for (const dynamicSubagent of buildDynamicSubagentTools(ctx)) {- if (effectiveTools.has(dynamicSubagent.name)) {- throw new Error(- `Dynamic subagent "${dynamicSubagent.name}" collides with another runtime-visible tool name.`,- );- }- effectiveTools.set(dynamicSubagent.name, dynamicSubagent);- }- return effectiveTools;-}--/** Appended to the model-facing description of every tool with `endsTurn: true`. */-export const ENDS_TURN_TOOL_NOTE =- "Calling this tool ends your turn once it succeeds: do not write a reply or call other tools in the same step. If it fails, you will see the error and can continue.";-export type EndsTurnTools = ReadonlyMap<string, NonNullable<HarnessToolDefinition["endsTurn"]>>;diff --git a/packages/eve/src/harness/nested-actions.ts b/packages/eve/src/harness/nested-actions.tsdeleted file mode 100644index 7b6935009..000000000--- a/packages/eve/src/harness/nested-actions.ts+++ /dev/null@@ -1,92 +0,0 @@-import { contextStorage, type ContextContainer } from "#context/container.js";-import { ContextKey } from "#context/key.js";-import type { TurnPosition } from "#harness/session-machine/view.js";-import type { HarnessEmitFn } from "#harness/types.js";-import { createRuntimeToolResultFromValue } from "#harness/action-result-helpers.js";-import { createActionResultEvent, createActionsRequestedEvent } from "#protocol/message.js";-import type { JsonObject } from "#shared/json.js";--/**- * A call a tool made on the model's behalf, reported on the protocol as a- * nested action of the calling tool's action. Nested actions never enter- * model history; the model sees only the parent call and its result.- */-export interface NestedToolAction {- readonly input: JsonObject;- readonly isError?: boolean;- /** The call's result, or its error message when `isError` is set. */- readonly output: unknown;- readonly toolName: string;-}--/**- * Step-local nested actions keyed by parent call id. Written while the parent- * executes and drained when the harness emits the parent's result, so nested- * actions always follow the parent's request and precede its result.- */-const NestedToolActionsKey = new ContextKey<Readonly<Record<string, readonly NestedToolAction[]>>>(- "eve.nestedToolActions",-);--/** Records a nested action of the tool call `parentCallId`. */-export function reportNestedToolAction(parentCallId: string, action: NestedToolAction): void {- const ctx = contextStorage.getStore() as ContextContainer | undefined;- if (ctx === undefined) return;- const pending = ctx.get(NestedToolActionsKey) ?? {};- ctx.setVirtualContext(NestedToolActionsKey, {- ...pending,- [parentCallId]: [...(pending[parentCallId] ?? []), action],- });-}--/**- * Emits the nested actions recorded for `parentCallId`, each as an- * `actions.requested` and `action.result` pair. Call ids derive from the- * parent's, so a re-run step reports the same ids.- */-export async function emitNestedToolActions(- emitFn: HarnessEmitFn,- state: Pick<TurnPosition, "sequence" | "stepIndex" | "turnId">,- parentCallId: string,-): Promise<void> {- const ctx = contextStorage.getStore() as ContextContainer | undefined;- const pending = ctx?.get(NestedToolActionsKey);- const actions = pending?.[parentCallId];- if (ctx === undefined || pending === undefined || actions === undefined) return;- const { [parentCallId]: _drained, ...rest } = pending;- ctx.setVirtualContext(NestedToolActionsKey, rest);-- for (const [index, action] of actions.entries()) {- const callId = `${parentCallId}:${index + 1}`;- const position = {- sequence: state.sequence,- stepIndex: state.stepIndex,- turnId: state.turnId,- };- await emitFn(- createActionsRequestedEvent({- actions: [- {- callId,- input: action.input,- kind: "tool-call",- parentCallId,- toolName: action.toolName,- },- ],- ...position,- }),- );- await emitFn(- createActionResultEvent({- result: createRuntimeToolResultFromValue({- callId,- isError: action.isError,- output: action.output,- toolName: action.toolName,- }),- ...position,- }),- );- }-}diff --git a/packages/eve/src/harness/session-machine/machine.test.ts b/packages/eve/src/harness/session-machine/machine.test.tsindex fa05d379f..50a267227 100644--- a/packages/eve/src/harness/session-machine/machine.test.ts+++ b/packages/eve/src/harness/session-machine/machine.test.ts@@ -146,6 +146,7 @@ async function respond(machine: Machine, input: StepInput) {approvalKey: () => undefined,delivery: typed,policy: noPolicy(typed),+ searchable: () => false,takeQueued: delivered.takeQueued,}),);@@ -535,6 +536,7 @@ describe("answers", () => {approvalKey: (request) => `deploy:${String(request.action.input.service)}`,delivery: { inputResponses: [{ optionId: "approve", requestId: "approval-call-1" }] },policy: noPolicy(),+ searchable: () => false,takeQueued: false,}),);diff --git a/packages/eve/src/harness/session-machine/transitions.ts b/packages/eve/src/harness/session-machine/transitions.tsindex d4a59b388..8bcd276b2 100644--- a/packages/eve/src/harness/session-machine/transitions.ts+++ b/packages/eve/src/harness/session-machine/transitions.ts@@ -1,4 +1,4 @@-import type { ModelMessage, UserContent } from "ai";+import type { ModelMessage, ToolCallPart, UserContent } from "ai";import type {SubagentAuthorizationEventHookPayload,@@ -370,29 +370,46 @@ function isComplete(step: SuspendedStep): boolean {return [...stepCallIds(step)].every((callId) => answered.has(callId));}-/** Places a result right after the message that made its call. */+/**+ * Places a result right after the message that made its call, under the call's name: a call+ * made through `eve__tool` or `eve__skill` keeps that name in history, whatever entry it ran.+ */export function withResult(messages: readonly ModelMessage[],- part: ToolResultPart,+ result: ToolResultPart,): ModelMessage[] {const next = [...messages];- const asking = next.findIndex(- (message) =>- message.role === "assistant" &&- Array.isArray(message.content) &&- message.content.some(- (content) => content.type === "tool-call" && content.toolCallId === part.toolCallId,- ),- );- const following = next[asking + 1];- if (asking >= 0 && following?.role === "tool") {- next[asking + 1] = { ...following, content: [...following.content, part] };+ const asking = findCall(next, result.toolCallId);+ if (asking === undefined) {+ next.push({ content: [result], role: "tool" });+ return next;+ }+ const part = { ...result, toolName: asking.call.toolName };+ const following = next[asking.index + 1];+ if (following?.role === "tool") {+ next[asking.index + 1] = { ...following, content: [...following.content, part] };} else {- next.splice(asking >= 0 ? asking + 1 : next.length, 0, { content: [part], role: "tool" });+ next.splice(asking.index + 1, 0, { content: [part], role: "tool" });}return next;}+/** The call with this id, and the index of the message that made it. */+function findCall(+ messages: readonly ModelMessage[],+ callId: string,+): { readonly call: ToolCallPart; readonly index: number } | undefined {+ for (const [index, message] of messages.entries()) {+ if (message.role !== "assistant" || typeof message.content === "string") continue;+ const call = message.content.find(+ (content): content is ToolCallPart =>+ content.type === "tool-call" && content.toolCallId === callId,+ );+ if (call !== undefined) return { call, index };+ }+ return undefined;+}+// ---------------------------------------------------------------------------// Tasks// ---------------------------------------------------------------------------diff --git a/packages/eve/src/harness/step-hooks.ts b/packages/eve/src/harness/step-hooks.tsindex 4d3e4ac34..16a868617 100644--- a/packages/eve/src/harness/step-hooks.ts+++ b/packages/eve/src/harness/step-hooks.ts@@ -21,6 +21,7 @@ import {createRuntimeToolResultFromToolError,createRuntimeToolResultFromMessagePart,createRuntimeToolResultFromStepResult,+ toActionResult,} from "#harness/action-result-helpers.js";import type { TurnPosition } from "#harness/session-machine/view.js";import { normalizeAssistantStepFinishReason } from "#harness/finish-reason.js";@@ -37,13 +38,12 @@ import type { RuntimeToolResultActionResult } from "#shared/action-types.js";import {type HarnessEmitFn,type HarnessSession,+ type HarnessToolLookup,requireSessionModelReference,- type ToolLoopHarnessConfig,} from "#harness/types.js";import { contextStorage } from "#context/container.js";import { isAuthorizationSignal, isPendingAuthorizationToolOutput } from "#harness/authorization.js";import { readToolInterrupt } from "#harness/tool-interrupts.js";-import { emitNestedToolActions } from "#harness/nested-actions.js";import { AuthKey } from "#context/keys.js";import { resolveConversationId } from "#shared/conversation-identity.js";@@ -232,7 +232,7 @@ export async function emitStepActions(readonly excludedActionCallIds?: ReadonlySet<string>;readonly excludedActionToolNames: ReadonlySet<string>;readonly handledInlineToolResultCallIds?: ReadonlySet<string>;- readonly tools: ToolLoopHarnessConfig["tools"];+ readonly tools: HarnessToolLookup;},): Promise<void> {const providerExecutedCallIds = new Set(@@ -246,6 +246,7 @@ export async function emitStepActions(...extractToolApprovalInputRequests({content: (step.content ?? []) as ContentPart<ToolSet>[],excludedCallIds: options.excludedActionCallIds,+ tools: options.tools,}).map((request) => request.action.callId),...(step.toolCalls as TypedToolCall<ToolSet>[]).filter(isInvalidToolCall)@@ -289,6 +290,12 @@ export async function emitStepActions(toolResult.output,]),);+ const inputByCallId = new Map<string, unknown>(+ (step.toolCalls as TypedToolCall<ToolSet>[]).map((toolCall) => [+ toolCall.toolCallId,+ toolCall.input,+ ]),+ );for (const result of reconcileToolResults(step)) {if (isExcluded(result.callId, result.toolName)) {@@ -304,10 +311,9 @@ export async function emitStepActions(continue;}- await emitNestedToolActions(emitFn, state, result.callId);await emitFn(createActionResultEvent({- result,+ result: toActionResult(result, inputByCallId.get(result.callId)),sequence: state.sequence,stepIndex: state.stepIndex,turnId: state.turnId,@@ -362,6 +368,12 @@ function reconcileToolResults(step: HarnessStepResult): readonly RuntimeToolResuresultsByCallId.set(part.toolCallId, createRuntimeToolResultFromToolError(part));}+ const entryNames = new Map(+ (step.toolCalls as TypedToolCall<ToolSet>[]).map((toolCall) => [+ toolCall.toolCallId,+ toolCall.toolName,+ ]),+ );for (const part of extractToolResultParts(step.response.messages)) {if ((part as { readonly providerExecuted?: boolean }).providerExecuted === true) {continue;@@ -371,7 +383,13 @@ function reconcileToolResults(step: HarnessStepResult): readonly RuntimeToolResucontinue;}- resultsByCallId.set(part.toolCallId, createRuntimeToolResultFromMessagePart(part));+ resultsByCallId.set(+ part.toolCallId,+ createRuntimeToolResultFromMessagePart(+ part,+ entryNames.get(part.toolCallId) ?? part.toolName,+ ),+ );}return [...resultsByCallId.values()];diff --git a/packages/eve/src/harness/step/after-model.ts b/packages/eve/src/harness/step/after-model.tsindex c5b6c753a..3b7f5aa5f 100644--- a/packages/eve/src/harness/step/after-model.ts+++ b/packages/eve/src/harness/step/after-model.ts@@ -2,17 +2,14 @@ import { stageToolResultMedia } from "#harness/attachment-staging.js";import type { ModelMessage, ToolSet, TypedToolCall, TypedToolError } from "ai";import type { CompactionConfig, StepResult } from "#harness/types.js";-import {- FINAL_OUTPUT_BESIDE_PENDING_CALLS,- FINAL_OUTPUT_TOOL_NAME,-} from "#harness/final-output.js";+import { FINAL_OUTPUT_BESIDE_PENDING_CALLS } from "#harness/final-output.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import {type HarnessModelMessage,resolveAssistantStepText,validateHarnessModelMessages,} from "#harness/messages.js";import type { HarnessStepResult } from "#harness/step-hooks.js";-import type { HarnessToolDefinition } from "#harness/execute-tool.js";import {appendMissingToolResultMessages,extractToolResultCallIds,@@ -30,7 +27,7 @@ import {withResult,} from "#harness/session-machine/transitions.js";import { clearTurnClientContextState } from "#harness/turn-client-context.js";-import { collectDeferredCalls } from "#harness/coordination.js";+import { collectWorkflowCalls } from "#harness/coordination.js";import {createToolResultMessagePartFromToolError,isToolResultError,@@ -41,12 +38,11 @@ import {parkOnApprovals,stopForToolSignIn,} from "#harness/hitl/index.js";-import { getAdvertisedTools } from "#harness/advertised-tools.js";import {getInvalidToolCallInputError,isInvalidToolCall,} from "#harness/tool-call-input-errors.js";-import { isTaskTool, workingTaskIds } from "#execution/tasks/model-step.js";+import { isWorkflowTool, workingTaskIds } from "#execution/tasks/model-step.js";import { normalizeProviderToolHistory } from "#harness/provider-tool-history.js";import { renderFinalOutputWhileWorkingError } from "#execution/tasks/render.js";import { resolveInlineAuthorizationInterrupt } from "#harness/inline-tool-authorization.js";@@ -61,7 +57,7 @@ function getInvalidToolCallInputErrors(input: {const errors: TypedToolError<ToolSet>[] = [];for (const toolCall of input.toolCalls) {- if (toolCall.toolName === FINAL_OUTPUT_TOOL_NAME) {+ if (toolCall.toolName === REPLY_TOOL_NAME) {continue;}@@ -131,57 +127,42 @@ export async function handleStepResult(step: Step, input: ModelResponse): Promisconst approvalRequests = extractToolApprovalInputRequests({content: result.content ?? [],excludedCallIds: invalidInputToolCallIds,- });- const advertisedCoordinationTools = getAdvertisedTools({- session: step.session,- tools: input.coordinationTools,+ tools: input.catalog,});// Only unanswered calls can dispatch: automatic denials already have results.const blockedCallIds = new Set([...approvalRequests.map((request) => request.action.callId),...extractToolResultCallIds(responseMessages),]);- const unadvertised: TypedToolCall<ToolSet>[] = [];- const deferredToolCalls = ((result.toolCalls ?? []) as TypedToolCall<ToolSet>[])+ const workflowToolCalls = ((result.toolCalls ?? []) as TypedToolCall<ToolSet>[]).filter((toolCall) => !invalidInputToolCallIds.has(toolCall.toolCallId)).filter((toolCall) => !blockedCallIds.has(toolCall.toolCallId))- .filter((toolCall) => isDeferredHarnessTool(input.coordinationTools.get(toolCall.toolName)))- .filter((toolCall) => {- if (isDeferredHarnessTool(advertisedCoordinationTools.get(toolCall.toolName))) {- return true;- }- log.warn("deferred tool call blocked because tool is not advertised", {- callId: toolCall.toolCallId,- sessionId: step.session.sessionId,- toolName: toolCall.toolName,- });- unadvertised.push(toolCall);- return false;- });+ .filter((toolCall) => isWorkflowTool(input.catalog.get(toolCall.toolName)));// --- Park on approvals or runtime calls ----------------------------------- if (deferredToolCalls.length > 0 || approvalRequests.length > 0) {+ if (workflowToolCalls.length > 0 || approvalRequests.length > 0) {const { sequence, stepIndex, turnId } = step.position();- const deferred = collectDeferredCalls({+ const dispatched = collectWorkflowCalls({session: step.session,- toolCalls: deferredToolCalls,- tools: advertisedCoordinationTools,+ toolCalls: workflowToolCalls,+ tools: input.catalog,turnId,});- step.session = { ...deferred.session, history: validateHarnessModelMessages(promptMessages) };+ step.session = { ...dispatched.session, history: validateHarnessModelMessages(promptMessages) };const parked = {event: { sequence, stepIndex, turnId },// A suspended response commits once every call it made has a result, so a call that will// never get one is answered now.- messages: answerCallsThatWontRun(responseMessages, result, unadvertised),- tasks: deferred.workflowRequests,+ messages: answerCallsThatWontRun(responseMessages, result),+ tasks: dispatched.workflowRequests,};if (approvalRequests.length > 0) {return parkOnApprovals(step, {...parked,requests: approvalRequests,- waitsOnRuntime: deferredToolCalls.length > 0,+ tools: input.catalog,+ waitsOnRuntime: workflowToolCalls.length > 0,});}await step.apply(suspendStep(step.view(), parked));@@ -207,7 +188,7 @@ export async function handleStepResult(step: Step, input: ModelResponse): Promishistory: validateHarnessModelMessages([...promptMessages, ...responseMessages]),};- // A `final_output` call is terminal even when the model emits it alongside+ // An `eve__reply` call is terminal even when the model emits it alongside// executing tools: continuing the loop would leave the no-execute call as a// dangling tool_use the next provider call rejects, and drop the result.const calledFinalOutput =@@ -247,35 +228,22 @@ export async function handleStepResult(step: Step, input: ModelResponse): Promis/*** Answers the calls of a response that waits on others which will never get a result: a- * `final_output` written before the results it waits beside, and a deferred call to a tool this- * step didn't advertise.+ * `eve__reply` written before the results it waits beside.*/function answerCallsThatWontRun(messages: readonly ModelMessage[],result: HarnessStepResult,- unadvertised: readonly TypedToolCall<ToolSet>[],): ModelMessage[] {const answered = extractToolResultCallIds(messages);const finalOutputs = ((result.toolCalls ?? []) as TypedToolCall<ToolSet>[]).filter(- (call) => call.toolName === FINAL_OUTPUT_TOOL_NAME,+ (call) => call.toolName === REPLY_TOOL_NAME,);- const answers: ToolResultPart[] = [- ...finalOutputs.map((call) => ({- output: { type: "error-text" as const, value: FINAL_OUTPUT_BESIDE_PENDING_CALLS },- toolCallId: call.toolCallId,- toolName: call.toolName,- type: "tool-result" as const,- })),- ...unadvertised.map((call) => ({- output: {- type: "error-text" as const,- value: `The tool "${call.toolName}" isn't available in this step, so the call didn't run.`,- },- toolCallId: call.toolCallId,- toolName: call.toolName,- type: "tool-result" as const,- })),- ];+ const answers: ToolResultPart[] = finalOutputs.map((call) => ({+ output: { type: "error-text" as const, value: FINAL_OUTPUT_BESIDE_PENDING_CALLS },+ toolCallId: call.toolCallId,+ toolName: call.toolName,+ type: "tool-result" as const,+ }));return answers.filter((part) => !answered.has(part.toolCallId)).reduce<ModelMessage[]>((next, part) => withResult(next, part), [...messages]);@@ -350,24 +318,18 @@ async function callEndsTurn(}}-function isDeferredHarnessTool(tool: HarnessToolDefinition | undefined): boolean {- return tool?.workflowId !== undefined || isTaskTool(tool);-}--/** Answers a `final_output` call made while tasks work with an error naming them. */+/** Answers an `eve__reply` call made while tasks work with an error naming them. */function rejectFinalOutput(responseMessages: readonly ModelMessage[],result: HarnessStepResult,workingTasks: readonly string[],): ModelMessage[] {- const call = (result.toolCalls ?? []).find(- (toolCall) => toolCall.toolName === FINAL_OUTPUT_TOOL_NAME,- );+ const call = (result.toolCalls ?? []).find((toolCall) => toolCall.toolName === REPLY_TOOL_NAME);if (call === undefined) return [...responseMessages];const rejection: ToolResultPart = {output: { type: "error-text", value: renderFinalOutputWhileWorkingError(workingTasks) },toolCallId: call.toolCallId,- toolName: FINAL_OUTPUT_TOOL_NAME,+ toolName: REPLY_TOOL_NAME,type: "tool-result",};return [...responseMessages, { content: [rejection], role: "tool" }];@@ -380,11 +342,11 @@ const OUTPUT_SCHEMA_NOT_FULFILLED = {/*** The structured value the model delivered by calling the framework- * `final_output` tool, or `undefined` when the terminal turn ended in prose.+ * `eve__reply` tool, or `undefined` when the terminal turn ended in prose.*/function extractFinalOutput(result: HarnessStepResult): JsonValue | undefined {return (result.toolCalls ?? []).find(- (call) => call.toolName === FINAL_OUTPUT_TOOL_NAME && !isInvalidToolCall(call),+ (call) => call.toolName === REPLY_TOOL_NAME && !isInvalidToolCall(call),)?.input as JsonValue | undefined;}@@ -392,7 +354,7 @@ function extractFinalOutput(result: HarnessStepResult): JsonValue | undefined {* Closes a terminal turn. An unmet output schema fails the turn recoverably;* otherwise the structured value (or prose) ends the turn and the session* waits for the next message. The structured value replaces the un-executed- * `final_output` call, which would be a dangling tool_use on the next turn,+ * `eve__reply` call, which would be a dangling tool_use on the next turn,* and the schema, scoped to the turn, clears.*/async function settleTurn(diff --git a/packages/eve/src/harness/tool-call-input-errors.ts b/packages/eve/src/harness/tool-call-input-errors.tsindex 58f646056..bbcda313b 100644--- a/packages/eve/src/harness/tool-call-input-errors.ts+++ b/packages/eve/src/harness/tool-call-input-errors.ts@@ -5,7 +5,7 @@ import {type RuntimeActionRequestProjection,} from "#harness/action-presentation.js";import { resolveToolCallInputObject } from "#harness/coordination.js";-import type { HarnessToolMap } from "#harness/types.js";+import type { HarnessToolLookup } from "#harness/types.js";/*** Returns true when the AI SDK marked the tool call `invalid` (typically@@ -75,7 +75,7 @@ export function resolveProviderToolCallRequest(readonly toolCallId: string;readonly toolName: string;},- tools: HarnessToolMap,+ tools: HarnessToolLookup,):| { readonly request: RuntimeActionRequestProjection; readonly toolError?: undefined }| { readonly request?: undefined; readonly toolError: TypedToolError<ToolSet> } {diff --git a/packages/eve/src/harness/tool-call-io.ts b/packages/eve/src/harness/tool-call-io.tsindex 8e81f4d57..1d9771cce 100644--- a/packages/eve/src/harness/tool-call-io.ts+++ b/packages/eve/src/harness/tool-call-io.ts@@ -1,6 +1,7 @@-import { asSchema } from "ai";+import { asSchema, TypeValidationError } from "ai";import { observeToolOutput } from "#tool-stubs/execute.js";+import type { StandardSchemaV1 } from "#compiled/@standard-schema/spec/index.js";import {authorizationPendingModelText,isAuthorizationPendingModelOutput,@@ -9,13 +10,19 @@ import { resolveToolCallInputObject } from "#harness/coordination.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";import { normalizeToolModelOutput, type ToolModelOutputValue } from "#harness/tool-model-output.js";import { toErrorMessage } from "#shared/errors.js";+import { isObject } from "#shared/guards.js";import type { JsonObject } from "#shared/json.js";import { toModelSchema } from "#tools/schema.js";/** The result of checking one tool call's input the way a model-issued call is checked. */export type ToolCallInputCheck =| { readonly kind: "valid"; readonly value: JsonObject }- | { readonly kind: "invalid"; readonly message: string }+ | {+ readonly kind: "invalid";+ /** What is wrong with the input, for a caller that reports it in its own words. */+ readonly issues: readonly StandardSchemaV1.Issue[];+ readonly message: string;+ }| { readonly kind: "threw"; readonly error: unknown };/**@@ -32,10 +39,7 @@ export async function checkToolCallInput(try {value = resolveToolCallInputObject(input, { callId, toolName: definition.name });} catch (error) {- return {- kind: "invalid",- message: `Invalid input for tool "${definition.name}": ${toErrorMessage(error)}`,- };+ return invalidInput(definition.name, error);}let result: Awaited<ReturnType<NonNullable<ReturnType<typeof asSchema>["validate"]>>>;try {@@ -49,10 +53,27 @@ export async function checkToolCallInput(}return result.success? { kind: "valid", value: result.value as JsonObject }- : {- kind: "invalid",- message: `Invalid input for tool "${definition.name}": ${toErrorMessage(result.error)}`,- };+ : invalidInput(definition.name, result.error);+}++function invalidInput(toolName: string, error: unknown): ToolCallInputCheck {+ return {+ issues: schemaIssues(error),+ kind: "invalid",+ message: `Invalid input for tool "${toolName}": ${toErrorMessage(error)}`,+ };+}++/** The schema's own issues; any other failure is reported as one issue. */+function schemaIssues(error: unknown): readonly StandardSchemaV1.Issue[] {+ const cause = TypeValidationError.isInstance(error) ? error.cause : undefined;+ return Array.isArray(cause) && cause.every(isIssue)+ ? cause+ : [{ message: toErrorMessage(error) }];+}++function isIssue(value: unknown): value is StandardSchemaV1.Issue {+ return isObject(value) && typeof value.message === "string";}/**diff --git a/packages/eve/src/harness/tool-input-validation.integration.test.ts b/packages/eve/src/harness/tool-input-validation.integration.test.tsindex 51515019f..60988e05c 100644--- a/packages/eve/src/harness/tool-input-validation.integration.test.ts+++ b/packages/eve/src/harness/tool-input-validation.integration.test.ts@@ -3,6 +3,7 @@ import { convertArrayToReadableStream, MockLanguageModelV4 } from "ai/test";import { describe, expect, it, vi } from "vitest";import { createToolLoopHarness } from "#harness/tool-loop.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import type { HarnessSession, ToolLoopHarnessConfig } from "#harness/types.js";// The harness runs outside a workflow body here, where run attributes cannot@@ -35,7 +36,7 @@ function findToolResult(messages: readonly ModelMessage[], toolCallId: string):}describe("framework tool input validation (real AI SDK)", () => {- it("rejects invalid final_output input instead of settling the turn", async () => {+ it("rejects invalid eve__reply input instead of settling the turn", async () => {const invalidCallId = "final-invalid";const validCallId = "final-valid";const outputSchema = {@@ -51,7 +52,7 @@ describe("framework tool input validation (real AI SDK)", () => {{input: JSON.stringify({ answer: 42 }),toolCallId: invalidCallId,- toolName: "final_output",+ toolName: REPLY_TOOL_NAME,type: "tool-call",},],@@ -64,7 +65,7 @@ describe("framework tool input validation (real AI SDK)", () => {{input: JSON.stringify({ answer: "done" }),toolCallId: validCallId,- toolName: "final_output",+ toolName: REPLY_TOOL_NAME,type: "tool-call",},],@@ -99,7 +100,7 @@ describe("framework tool input validation (real AI SDK)", () => {expect(typeof invalidStep.next).toBe("function");expect(findToolResult(invalidStep.session.history, invalidCallId)).toMatchObject({output: expect.objectContaining({ type: "error-text" }),- toolName: "final_output",+ toolName: REPLY_TOOL_NAME,});if (typeof invalidStep.next !== "function") {diff --git a/packages/eve/src/harness/tool-loop-compaction-accounting.test.ts b/packages/eve/src/harness/tool-loop-compaction-accounting.test.tsindex fa31ed58f..965452394 100644--- a/packages/eve/src/harness/tool-loop-compaction-accounting.test.ts+++ b/packages/eve/src/harness/tool-loop-compaction-accounting.test.ts@@ -288,7 +288,7 @@ describe("tool-loop structured compaction accounting", () => {},],usage: {- inputTokens: 100,+ inputTokens: 600,},},{@@ -314,7 +314,7 @@ describe("tool-loop structured compaction accounting", () => {createTestSession({compaction: {recentWindowSize: 10,- threshold: 500,+ threshold: 1000,},}),{ message: "Compute something" },@@ -322,7 +322,7 @@ describe("tool-loop structured compaction accounting", () => {expect(first.next).toBe(runStep);expect(first.session.compaction).toMatchObject({- lastKnownInputTokens: 100,+ lastKnownInputTokens: 600,lastKnownPromptMessageCount: 1,});diff --git a/packages/eve/src/harness/tool-loop-generate-approval-resume.integration.test.ts b/packages/eve/src/harness/tool-loop-generate-approval-resume.integration.test.tsindex 4622d8620..c146be3aa 100644--- a/packages/eve/src/harness/tool-loop-generate-approval-resume.integration.test.ts+++ b/packages/eve/src/harness/tool-loop-generate-approval-resume.integration.test.ts@@ -4,7 +4,6 @@ import {toolCallStreamResult,usage,} from "#internal/testing/approval-resume.js";-import { setTurnClientContextState } from "#harness/turn-client-context.js";import { jsonSchema, type LanguageModel, type ModelMessage, type Telemetry } from "ai";import { MockLanguageModelV4 } from "ai/test";import type { InstrumentationAttemptScope } from "#instrumentation/lifecycle.js";@@ -15,10 +14,8 @@ import {preparePersistedStepDynamicToolMetadata,} from "#context/dynamic-tool-lifecycle.js";import type { OldSourceOffsetDynamicToolMetadata } from "#context/dynamic-tool-metadata.js";-import { PendingSkillAnnouncementKey } from "#context/dynamic-skill-lifecycle.js";import {AuthKey,- HistoryStateKey,SessionKey,SessionIdKey,SessionDynamicToolMetadataKey,@@ -961,96 +958,6 @@ describe("tool loop generate approval resume (real AI SDK)", () => {});});- // Regression: turn-local context (a dynamic skill announcement) was- // appended as a user message after the approval response. The AI SDK reads- // approvals only from the tail tool message, so the approved tool never ran- // and the provider rejected the prompt with a tool call that had no output.- it.each([false, true])(- "executes the approved tool when a dynamic skill announcement is injected on the resume step (restored anchor: %s)",- async (restoredAnchor) => {- const siblingCall = {- input: { command: "whoami" },- toolCallId: "call-sibling",- toolName: "bash",- type: "tool-call" as const,- };- const siblingResult = {- output: { type: "text" as const, value: "eve" },- toolCallId: siblingCall.toolCallId,- toolName: "bash",- type: "tool-result" as const,- };- // The parked shape when a gated call shares a step with an ungated one.- const session = withParkedStep(createBaseSession(), {- messages: [- { content: [toolCall, approvalRequest, siblingCall], role: "assistant" },- { content: [siblingResult], role: "tool" },- ],- requests: [pendingApprovalInputRequest],- });- const ctx = new ContextContainer();- const runtimeContextAnnouncement = "Available skills\n- policy: Tenant policy";- ctx.set(PendingSkillAnnouncementKey, runtimeContextAnnouncement);- const execute = vi.fn(async () => "/workspace");- const model = createModel();- const runStep = createToolLoopHarness(createConfig(model, execute));-- const result = await contextStorage.run(ctx, () =>- runStep(- withOpenTurn(- restoredAnchor- ? setTurnClientContextState(session, {- insertionIndex: 0,- messages: [],- turnId: "turn-1",- })- : session,- { sequence: 1, stepIndex: 1, turnId: "turn-1" },- ),- { inputResponses: [{ optionId: "approve", requestId: approvalRequest.approvalId }] },- ),- );-- expect(execute).toHaveBeenCalledExactlyOnceWith(- toolCall.input,- expect.objectContaining({ toolCallId: toolCall.toolCallId }),- );-- const providerPrompt = model.doStreamCalls[0]?.prompt ?? [];- const answered = new Set<string>();- const called: string[] = [];- for (const message of providerPrompt) {- if (!Array.isArray(message.content)) continue;- for (const part of message.content) {- if (part.type === "tool-call") called.push(part.toolCallId);- if (part.type === "tool-result") answered.add(part.toolCallId);- }- }- expect(called).toEqual([toolCall.toolCallId, siblingCall.toolCallId]);- expect(called.filter((id) => !answered.has(id))).toEqual([]);- expect(JSON.stringify(providerPrompt)).toContain("Tenant policy");- expect(ctx.get(HistoryStateKey)).toMatchObject({- availableSkills: runtimeContextAnnouncement,- });- expect(result.session.history.at(-1)).toMatchObject({- content: [{ text: "The command returned /workspace.", type: "text" }],- role: "assistant",- });-- const continued = await contextStorage.run(ctx, () =>- runStep(result.session, { message: "Continue." }),- );- expect(- continued.session.history.filter(- (message) => message.content === runtimeContextAnnouncement,- ),- ).toHaveLength(1);- expect(ctx.get(HistoryStateKey)).toMatchObject({- availableSkills: runtimeContextAnnouncement,- });- },- );-// Acceptance gate for the HITL non-blocking plan (research/hitl-request-lifecycle.md):// once messages run as normal turns while an approval is open, the approval batch is// restored *after* that intervening exchange. This proves the AI SDK accepts thediff --git a/packages/eve/src/harness/tool-loop-request-envelope.integration.test.ts b/packages/eve/src/harness/tool-loop-request-envelope.integration.test.tsindex 86ec399e2..1551bfbd3 100644--- a/packages/eve/src/harness/tool-loop-request-envelope.integration.test.ts+++ b/packages/eve/src/harness/tool-loop-request-envelope.integration.test.ts@@ -3,12 +3,14 @@ import type { LanguageModel } from "ai";import { describe, expect, it, vi } from "vitest";import { ContextContainer, contextStorage } from "#context/container.js";-import { PendingSkillAnnouncementKey } from "#context/dynamic-skill-lifecycle.js";-import { SessionDynamicInstructionsKey } from "#context/keys.js";+import { dispatchDynamicSkillEvent } from "#context/dynamic-skill-lifecycle.js";+import { SessionDynamicInstructionsKey, StaticModelReferenceKey } from "#context/keys.js";import { mockModel, type MockModelRequest } from "#evals/mock-model.js";import { createToolLoopHarness } from "#harness/tool-loop.js";import type { HarnessSession } from "#harness/types.js";import { captureLogRecords } from "#internal/testing/log-records.js";+import { createSessionStartedEvent } from "#protocol/message.js";+import { defineSkill } from "#public/definitions/skill.js";// The harness runs outside a workflow body here, where run attributes cannot// be written; the attribute contract is covered by emit.test.ts.@@ -83,9 +85,32 @@ describe("model request envelope accounting", () => {);});- it("does not let a persisted announcement mask later instruction growth", async () => {+ it("does not let a new announcement mask later instruction growth", async () => {const ctx = new ContextContainer();- ctx.set(PendingSkillAnnouncementKey, "Available skill description ".repeat(600));+ // Dynamic resolvers read the session's model.+ ctx.set(StaticModelReferenceKey, { id: "task" });+ await dispatchDynamicSkillEvent({+ ctx,+ event: createSessionStartedEvent(),+ messages: [],+ resolvers: [+ {+ eventNames: ["session.started"],+ events: {+ "session.started": () =>+ defineSkill({+ description: "Available skill description ".repeat(600),+ markdown: "# Tenant policy",+ }),+ },+ exportName: "default",+ logicalPath: "skills/tenant-policy.ts",+ slug: "tenant-policy",+ sourceId: "skills/tenant-policy.ts",+ sourceKind: "module",+ },+ ],+ });let summaries = 0;const task = mockModel({respond: () => ({ text: "Done.", usage: { inputTokens: 8_000 } }),@@ -105,6 +130,7 @@ describe("model request envelope accounting", () => {runStep(session(), { message: "First task." }),);expect(summaries).toBe(0);+ // The skills announcement joined durable history on the first step.expect(JSON.stringify(first.session.history)).toContain("Available skill description");setInstructions(ctx, 600);await contextStorage.run(ctx, () => runStep(first.session, { message: "Second task." }));diff --git a/packages/eve/src/harness/tool-loop-steering.integration.test.ts b/packages/eve/src/harness/tool-loop-steering.integration.test.tsindex 8705e0098..f7399763e 100644--- a/packages/eve/src/harness/tool-loop-steering.integration.test.ts+++ b/packages/eve/src/harness/tool-loop-steering.integration.test.ts@@ -1,8 +1,20 @@import { jsonSchema } from "ai";import { MockLanguageModelV3 } from "ai/test";import { describe, expect, it, vi } from "vitest";+import { CALL_TOOL_NAME } from "#protocol/catalog-tools.js";import { createToolLoopHarness } from "#harness/tool-loop.js";-import { foldingHandler, positionOf, withOpenTurn } from "#internal/testing/session-machine.js";+import {+ inlineTool,+ subagentTool,+ toolMap,+ workflowTool,+} from "#internal/testing/catalog-fixtures.js";+import {+ foldingHandler,+ parkedSteps,+ positionOf,+ withOpenTurn,+} from "#internal/testing/session-machine.js";import {createFrameworkUserMessage,createUserMessage,@@ -472,4 +484,99 @@ describe("generation steering with the real AI SDK", () => {expect(providerSignal?.aborted).toBe(false);expect(events.filter((event) => event.type === "turn.completed")).toHaveLength(1);});++ it.each([+ ["a workflow tool directly", workflowTool("deploy_service"), "deploy_service", {}],+ [+ "a workflow tool through eve__tool",+ workflowTool("deploy_service", "execute", { deferred: true }),+ CALL_TOOL_NAME,+ { name: "deploy_service" },+ ],+ [+ "an agent through eve__tool",+ subagentTool("billing_specialist", { deferred: true }),+ CALL_TOOL_NAME,+ { input: { message: "Review Bob's dispute." }, name: "billing_specialist" },+ ],+ ] as const)(+ "interrupts a step that calls %s, since it runs only after the step",+ async (_case, entry, toolName, input) => {+ const steering = new AbortController();+ const model = new MockLanguageModelV3({+ doStream: async () => ({+ stream: new ReadableStream<Part>({+ start(controller) {+ controller.enqueue({+ type: "tool-call",+ toolCallId: "call-1",+ toolName,+ input: JSON.stringify(input),+ });+ controller.enqueue({+ type: "finish",+ finishReason: { unified: "tool-calls", raw: undefined },+ usage,+ });+ controller.close();+ },+ }),+ }),+ });+ const result = await createToolLoopHarness({+ steeringSignal: steering.signal,+ tools: toolMap(entry),+ resolveModel: async () => model,+ handleEvent: async (event) => {+ if (event.type === "actions.requested") steering.abort();+ },+ })(session(), { message: "Alice asks for the work to start" });++ expect(result.steered).toBe(true);+ expect(parkedSteps(result.session)).toEqual([]);+ },+ );++ it("does not interrupt a step once an inline entry called through eve__tool is running", async () => {+ const steering = new AbortController();+ const executing = Promise.withResolvers<void>();+ const release = Promise.withResolvers<void>();+ const refund = vi.fn(async () => {+ executing.resolve();+ await release.promise;+ return { refunded: true };+ });+ const model = new MockLanguageModelV3({+ doStream: async () => ({+ stream: new ReadableStream<Part>({+ start(controller) {+ controller.enqueue({+ type: "tool-call",+ toolCallId: "refund-1",+ toolName: CALL_TOOL_NAME,+ input: JSON.stringify({ input: {}, name: "refund_invoice" }),+ });+ controller.enqueue({+ type: "finish",+ finishReason: { unified: "tool-calls", raw: undefined },+ usage,+ });+ controller.close();+ },+ }),+ }),+ });+ const running = createToolLoopHarness({+ steeringSignal: steering.signal,+ tools: toolMap(inlineTool("refund_invoice", { deferred: true, execute: refund })),+ resolveModel: async () => model,+ handleEvent: async () => {},+ })(session(), { message: "Alice asks for a refund of invoice in_1" });+ await executing.promise;+ steering.abort();+ release.resolve();++ expect((await running).steered).toBeUndefined();+ expect(refund).toHaveBeenCalledOnce();+ });});diff --git a/packages/eve/src/harness/tool-loop.test.ts b/packages/eve/src/harness/tool-loop.test.tsindex 7234ae71d..b3f6aae86 100644--- a/packages/eve/src/harness/tool-loop.test.ts+++ b/packages/eve/src/harness/tool-loop.test.ts@@ -71,8 +71,6 @@ import {withPublished,withParkedStep,} from "#internal/testing/session-machine.js";-import { PendingSkillAnnouncementKey } from "#context/dynamic-skill-lifecycle.js";-import { deserializeContext, serializeContext } from "#context/serialize.js";import { stashToolInterrupt } from "#harness/tool-interrupts.js";import { appendMissingToolResultMessages } from "#harness/model-call/response.js";import { createToolLoopHarness } from "#harness/tool-loop.js";@@ -704,7 +702,7 @@ function finalOutputResult(text: string, structured: unknown): Record<string, unfinishReason: "stop",response: { messages: [{ content: text, role: "assistant" }] },text,- toolCalls: [{ input: structured, toolCallId: "final-output-1", toolName: "final_output" }],+ toolCalls: [{ input: structured, toolCallId: "final-output-1", toolName: "eve__reply" }],toolResults: [],};}@@ -1763,13 +1761,13 @@ describe("createToolLoopHarness", () => {const waitToolCall = {input: {},toolCallId: "wait-1",- toolName: "task_wait",+ toolName: "eve__task_wait",type: "tool-call" as const,};const invalidCancelToolCall = {input: "not an object",toolCallId: "cancel-1",- toolName: "task_cancel",+ toolName: "eve__task_cancel",type: "tool-call" as const,};const assistantContent = [@@ -2406,7 +2404,7 @@ describe("createToolLoopHarness", () => {]);expect(result.session.outputSchema).toBeUndefined();expect(vi.mocked(ToolLoopAgent).mock.calls[0]?.[0]).toMatchObject({- tools: expect.objectContaining({ final_output: expect.anything() }),+ tools: expect.objectContaining({ eve__reply: expect.anything() }),});});@@ -2427,7 +2425,7 @@ describe("createToolLoopHarness", () => {expect(vi.mocked(ToolLoopAgent).mock.calls).toHaveLength(1);expect(vi.mocked(ToolLoopAgent).mock.calls[0]?.[0]).not.toMatchObject({- tools: expect.objectContaining({ final_output: expect.anything() }),+ tools: expect.objectContaining({ eve__reply: expect.anything() }),});});@@ -2450,7 +2448,7 @@ describe("createToolLoopHarness", () => {{type: "tool-call",toolCallId: "final-output-1",- toolName: "final_output",+ toolName: "eve__reply",input: { title: "Done" },},],@@ -2465,7 +2463,7 @@ describe("createToolLoopHarness", () => {text: "",toolCalls: [{ input: {}, toolCallId: "add-1", toolName: "add" },- { input: { title: "Done" }, toolCallId: "final-output-1", toolName: "final_output" },+ { input: { title: "Done" }, toolCallId: "final-output-1", toolName: "eve__reply" },],toolResults: [{ toolCallId: "add-1", toolName: "add", output: "42" }],});@@ -4016,146 +4014,6 @@ describe("createToolLoopHarness", () => {});});- it("continues the tool loop when load_skill fails during local tool execution", async () => {- setupMockAgent({- content: [- {- input: { skill: "missing-demo-skill" },- toolCallId: "call-load-skill",- toolName: "load_skill",- type: "tool-call",- },- ],- finishReason: "tool-calls",- fullStreamParts: [- {- input: { skill: "missing-demo-skill" },- toolCallId: "call-load-skill",- toolName: "load_skill",- type: "tool-call",- },- {- error: new Error(- 'No skill named "missing-demo-skill" at /workspace/skills/missing-demo-skill/SKILL.md.',- ),- input: { skill: "missing-demo-skill" },- toolCallId: "call-load-skill",- toolName: "load_skill",- type: "tool-error",- },- { finishReason: "tool-calls", type: "finish-step" },- ],- response: {- messages: [- {- content: [- {- input: { skill: "missing-demo-skill" },- toolCallId: "call-load-skill",- toolName: "load_skill",- type: "tool-call",- },- ],- role: "assistant",- },- ],- },- text: "",- toolCalls: [- {- input: { skill: "missing-demo-skill" },- toolCallId: "call-load-skill",- toolName: "load_skill",- type: "tool-call",- },- ],- toolResults: [],- });-- const { emit, events } = createEventCollector();- const config = createTestConfig(emit, {- tools: new Map([- [- "load_skill",- {- description: "Load a skill.",- execute: vi.fn(),- inputSchema: jsonSchema({ type: "object" }),- name: "load_skill",- },- ],- ]),- });- const session = createTestSession({- agent: {- modelReference: { id: "test-model" },- system: "You are a test assistant.",- tools: [- {- description: "Load a skill.",- inputSchema: { type: "object" },- name: "load_skill",- },- ],- },- });-- const result = await createToolLoopHarness(config)(session, {- message: "Use the missing demo skill.",- });-- expect(typeof result.next).toBe("function");- expect(events.find((event) => event.type === "action.result")?.data).toEqual({- error: {- code: "ACTION_RESULT_FAILED",- message:- 'No skill named "missing-demo-skill" at /workspace/skills/missing-demo-skill/SKILL.md.',- },- result: {- callId: "call-load-skill",- isError: true,- kind: "tool-result",- output:- 'No skill named "missing-demo-skill" at /workspace/skills/missing-demo-skill/SKILL.md.',- toolName: "load_skill",- },- sequence: 0,- stepIndex: 0,- status: "failed",- turnId: "turn_0",- });- expect(result.session.history.slice(-2)).toEqual([- {- content: [- {- input: { skill: "missing-demo-skill" },- toolCallId: "call-load-skill",- toolName: "load_skill",- type: "tool-call",- },- ],- role: "assistant",- },- {- content: [- {- output: {- type: "error-text",- value:- 'No skill named "missing-demo-skill" at /workspace/skills/missing-demo-skill/SKILL.md.',- },- toolCallId: "call-load-skill",- toolName: "load_skill",- type: "tool-result",- },- ],- role: "tool",- },- ]);- expect(events.some((event) => event.type === "turn.failed")).toBe(false);- expect(events.some((event) => event.type === "session.failed")).toBe(false);- });-it("prefers toolResults over response messages when the stream omits the result", async () => {setupMockAgent({finishReason: "tool-calls",@@ -8399,13 +8257,13 @@ describe("createToolLoopHarness", () => {history: [{ content: "old message", kind: "user" as const, role: "user" }],});const ctx = new ContextContainer();- ctx.set(HistoryStateKey, { availableSkills: "old message" });+ ctx.set(HistoryStateKey, { announcements: { skills: "old message" } });const result = await contextStorage.run(ctx, () => runStep(session));expect(result.next).toBeNull();// Only the lifecycle the step published joins the session.expect({ ...result.session, state: session.state }).toEqual(session);- expect(ctx.get(HistoryStateKey)).toEqual({ availableSkills: "old message" });+ expect(ctx.get(HistoryStateKey)).toEqual({ announcements: { skills: "old message" } });expect(getCompatibilityEventTypes(events)).toEqual(["compaction.requested", "session.waiting"]);expect(ToolLoopAgent).not.toHaveBeenCalled();expect(logs.records).toContainEqual(@@ -8938,20 +8796,6 @@ describe("createToolLoopHarness", () => {"You are a test assistant.","You are a test assistant.",]);- expect(firstPrompt.tools).toEqual([- {- description: "Adds numbers",- inputSchema: { type: "object" },- name: "add",- providerOptions: undefined,- },- {- description: "Looks up a saved result",- inputSchema: { type: "object" },- name: "lookup",- providerOptions: undefined,- },- ]);expect(modelCalls.map((call) => call.tools)).toEqual([firstPrompt.tools,firstPrompt.tools,@@ -11023,287 +10867,6 @@ describe("createToolLoopHarness", () => {});});- it.each(- ["plain", "client context", "compaction", "projected history"].flatMap((scenario) =>- [false, true].map((changed) => ({ scenario, changed })),- ),- )(- "persists framework context across durable steps ($scenario, changed: $changed)",- async ({ scenario, changed }) => {- const withClientContext = scenario === "client context";- if (scenario === "compaction") {- vi.mocked(shouldCompact).mockReturnValueOnce(true);- vi.mocked(compactMessages).mockImplementationOnce(async (messages) => messages.slice(2));- }- const toolCall = {- type: "tool-call" as const,- toolCallId: "cache-call",- toolName: "add",- input: { a: 20, b: 22 },- };- const toolResult = {- type: "tool-result" as const,- toolCallId: "cache-call",- toolName: "add",- output: "42",- };- setupMockAgent({- finishReason: "tool-calls",- response: {- messages: [- { role: "assistant", content: [toolCall] },- { role: "tool", content: [toolResult] },- ],- },- text: "",- toolCalls: [toolCall],- toolResults: [{ ...toolResult, input: toolCall.input }],- });- const announcement = "Available skills\n- policy: Tenant policy";- const ctx = new ContextContainer();- ctx.set(PendingSkillAnnouncementKey, announcement);- const runStep = createToolLoopHarness(- createTestConfig(undefined, {- historyProjector:- scenario === "projected history"- ? ({ messages }) =>- messages.filter((message) => message.content !== "Previous answer")- : undefined,- }),- );- const initial = withOpenTurn(- createTestSession({- history: [- { role: "user", content: "Previous request", kind: "user" as const },- { role: "assistant", content: "Previous answer" },- ],- }),- { sequence: 1, stepIndex: 0, turnId: "turn_1" },- );- const input = { context: ["Current channel context"], message: "Add 20 and 22." };- const first = await contextStorage.run(ctx, () =>- runStep(- initial,- withClientContext ? attachClientContext(input, ["Client context"]) : input,- ),- );- expect(first.next).toBe(runStep);- const firstPrompt = structuredClone(getLastAgentSettings().messages);- expect(first.session.history).toContainEqual({- role: "user",- content: announcement,- kind: "context.state",- });- const nextState = changed- ? "Available skills\n- policy: Tenant policy\n- billing: Billing policy"- : announcement;- const nextContext = await deserializeContext(- JSON.parse(JSON.stringify(serializeContext(ctx))),- );- nextContext.set(PendingSkillAnnouncementKey, nextState);- setupMockAgent(defaultModelResult());- const restored = JSON.parse(JSON.stringify(first.session)) as HarnessSession;- await contextStorage.run(nextContext, () => runStep(restored));- const nextPrompt = getLastAgentSettings().messages;- expect(nextPrompt.slice(0, firstPrompt.length)).toEqual(firstPrompt);- expect(nextPrompt.filter((message) => message.content === announcement)).toHaveLength(1);- expect(nextContext.get(HistoryStateKey)).toMatchObject({ availableSkills: nextState });- if (changed) {- expect(nextPrompt.at(-1)).toEqual({- role: "user",- content: nextState,- kind: "context.state",- });- } else {- expect(nextPrompt.at(-1)?.role).toBe("tool");- }- },- );-- it("does not advance history state when the model call fails", async () => {- const logs = captureLogRecords();- const ctx = new ContextContainer();- ctx.set(PendingSkillAnnouncementKey, "Available skills\n- policy: Tenant policy");- const { emit } = createEventCollector();- const runStep = createToolLoopHarness(createTestConfig(emit));- setupMockAgentError(new Error("Model unavailable"));-- const failed = await contextStorage.run(ctx, () =>- runStep(createTestSession(), { message: "Check progress." }),- );- expect(ctx.get(HistoryStateKey)).toBeUndefined();- expect(failed.session.history).not.toContainEqual({- kind: "user" as const,- role: "user",- content: "Available skills\n- policy: Tenant policy",- });-- setupMockAgent(defaultModelResult());- const retried = await contextStorage.run(ctx, () =>- runStep(failed.session, { message: "Try again." }),- );- expect(retried.session.history).toContainEqual({- role: "user",- content: "Available skills\n- policy: Tenant policy",- kind: "context.state",- });- expect(ctx.get(HistoryStateKey)).toEqual({- availableSkills: "Available skills\n- policy: Tenant policy",- });- expect(logs.records).toContainEqual(- expect.objectContaining({- level: "error",- message: "model call failed — parking session for retry by the user",- }),- );- });-- it("does not advance history state when recording the step result fails", async () => {- const ctx = new ContextContainer();- ctx.set(PendingSkillAnnouncementKey, "Available skills\n- policy: Tenant policy");- const failure = new Error("Failed to finish the turn");- const runStep = createToolLoopHarness(- createTestConfig(async (event) => {- if (event.type === "session.waiting") throw failure;- }),- );- setupMockAgent(defaultModelResult());-- await expect(- contextStorage.run(ctx, () => runStep(createTestSession(), { message: "Check progress." })),- ).rejects.toBe(failure);- expect(getLastAgentSettings().messages).toContainEqual({- role: "user",- content: "Available skills\n- policy: Tenant policy",- kind: "context.state",- });- expect(ctx.get(HistoryStateKey)).toBeUndefined();- });-- it("retains completed compaction when the next model call fails", async () => {- const logs = captureLogRecords();- const announcement = "Available skills\n- policy: Tenant policy";- const ctx = new ContextContainer();- ctx.set(PendingSkillAnnouncementKey, announcement);- ctx.set(HistoryStateKey, { availableSkills: announcement });- const session = createTestSession({- compaction: {- recentWindowSize: 10,- threshold: 100_000,- lastKnownInputTokens: 50_000,- lastKnownPromptMessageCount: 1,- },- history: [{ role: "user", content: announcement, kind: "user" as const }],- });- const { emit } = createEventCollector();- const runStep = createToolLoopHarness(- createTestConfig(emit, {- resolveModel: vi.fn().mockResolvedValue({ modelId: "test-model", provider: "openai" }),- }),- );- vi.mocked(shouldCompact).mockReturnValueOnce(true);- vi.mocked(compactMessages).mockResolvedValueOnce([- createFrameworkUserMessage("context.compaction", "Conversation summary"),- ]);- setupMockAgentError(new Error("Model unavailable"));-- const failed = await contextStorage.run(ctx, () =>- runStep(session, { message: "Continue." }),- );- expect(failed.session.history).toEqual([- createFrameworkUserMessage("context.compaction", "Conversation summary"),- ]);- expect(ctx.get(HistoryStateKey)).toBeUndefined();- expect(failed.session.compaction).not.toHaveProperty("lastKnownInputTokens");- expect(failed.session.compaction).not.toHaveProperty("lastKnownPromptMessageCount");-- setupMockAgent(defaultModelResult());- const restoredContext = await deserializeContext(- JSON.parse(JSON.stringify(serializeContext(ctx))),- );- restoredContext.set(PendingSkillAnnouncementKey, announcement);- const restoredSession = JSON.parse(JSON.stringify(failed.session)) as HarnessSession;- const retried = await contextStorage.run(restoredContext, () =>- runStep(restoredSession, { message: "Try again." }),- );- expect(- retried.session.history.filter((message) => message.content === announcement),- ).toHaveLength(1);- expect(restoredContext.get(HistoryStateKey)).toEqual({ availableSkills: announcement });- expect(logs.records).toContainEqual(- expect.objectContaining({- level: "error",- message: "model call failed — parking session for retry by the user",- }),- );- });-- it.each(["clear", "manual compaction", "automatic compaction"])(- "reannounces unchanged context after %s replaces history",- async (replacement) => {- const ctx = new ContextContainer();- const availableSkills = "Available skills\n- policy: Tenant policy";- ctx.set(PendingSkillAnnouncementKey, availableSkills);- const runStep = createToolLoopHarness(createTestConfig());- setupMockAgent(defaultModelResult());- const first = await contextStorage.run(ctx, () =>- runStep(createTestSession(), { message: "Check progress." }),- );- const expectedState = { availableSkills };- expect(ctx.get(HistoryStateKey)).toEqual(expectedState);-- vi.mocked(compactMessages).mockResolvedValue([- createFrameworkUserMessage("context.compaction", "Conversation summary"),- ]);- let session = first.session;- if (replacement === "automatic compaction") {- vi.mocked(shouldCompact).mockReturnValueOnce(true);- } else {- const replaceHistory = createToolLoopHarness(- createTestConfig(undefined, {- clearOnly: replacement === "clear",- compactOnly: replacement === "manual compaction",- }),- );- session = (await contextStorage.run(ctx, () => replaceHistory(session))).session;- expect(ctx.get(HistoryStateKey)).toBeUndefined();- }-- setupMockAgent(defaultModelResult());- const next = await contextStorage.run(ctx, () =>- runStep(session, { message: "Continue." }),- );- for (const content of Object.values(expectedState)) {- expect(- next.session.history.filter((message) => message.content === content),- ).toHaveLength(1);- }- expect(ctx.get(HistoryStateKey)).toEqual(expectedState);- },- );-- it("skips empty skill announcements without rewriting earlier history", async () => {- const ctx = new ContextContainer();- ctx.set(PendingSkillAnnouncementKey, "Available skills\n- policy: Tenant policy");- const runStep = createToolLoopHarness(createTestConfig());- setupMockAgent(defaultModelResult());- const first = await contextStorage.run(ctx, () =>- runStep(createTestSession(), { message: "Check the policy." }),- );- ctx.set(PendingSkillAnnouncementKey, "");- setupMockAgent(defaultModelResult());- const next = await contextStorage.run(ctx, () =>- runStep(first.session, { message: "Continue." }),- );- expect(next.session.history.slice(0, first.session.history.length)).toEqual(- first.session.history,- );- expect(getLastAgentSettings().messages).toEqual([- ...first.session.history,- { kind: "user" as const, role: "user", content: "Continue." },- ]);- });-it("keeps ephemeral client context out of compaction and its token baseline", async () => {vi.mocked(shouldCompact).mockReturnValueOnce(true);vi.mocked(compactMessages).mockImplementationOnce(async (messages) => [...messages]);diff --git a/packages/eve/src/harness/tools-schema.integration.test.ts b/packages/eve/src/harness/tools-schema.integration.test.tsindex 79bffdb1b..6cece01ee 100644--- a/packages/eve/src/harness/tools-schema.integration.test.ts+++ b/packages/eve/src/harness/tools-schema.integration.test.ts@@ -73,18 +73,24 @@ describe("tool schemas at the model provider boundary", () => {},}).responses("gpt-5.4");const execute = vi.fn(async (_input: unknown) => ({ issues: [] }));+ const definitions = new Map([+ [+ "linear__list_issues",+ {+ name: "linear__list_issues",+ description: "Search issues by query or saved custom view.",+ inputSchema: schema(),+ execute,+ },+ ],+ ]);const tools = buildToolSet({- tools: new Map([- [- "linear__list_issues",- {- name: "linear__list_issues",- description: "Search issues by query or saved custom view.",- inputSchema: schema(),- execute,- },- ],- ]),+ describe: (definition) => definition.description,+ resolve: (call) => {+ const definition = definitions.get(call.toolName);+ return definition === undefined ? undefined : { call, definition };+ },+ tools: definitions,});await generateText({diff --git a/packages/eve/src/harness/tools.test.ts b/packages/eve/src/harness/tools.test.tsindex 24cf1d7f9..c0fe46b44 100644--- a/packages/eve/src/harness/tools.test.ts+++ b/packages/eve/src/harness/tools.test.ts@@ -20,7 +20,7 @@ import type { JsonObject } from "#shared/json.js";import { isAsyncIterable } from "#shared/async-iterable.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";import { buildToolApproval, buildToolSet, buildToolSetWithProviderTools } from "#harness/tools.js";-import type { HarnessToolMap } from "#harness/types.js";+import type { HarnessToolMap, ToolCallLike } from "#harness/types.js";import { createToolExecuteWithAuth } from "#execution/tool-auth.js";import type { ApprovalContext } from "#approval/definition.js";import type { ToolContext } from "#tools/definition.js";@@ -40,14 +40,31 @@ function getOutputJsonSchema(tool: unknown): unknown {return (tool as { outputSchema: { jsonSchema: unknown } }).outputSchema.jsonSchema;}+function catalogOf(tools: HarnessToolMap) {+ return {+ describe: (definition: HarnessToolDefinition) => definition.description,+ resolve: <T extends ToolCallLike>(call: T) => {+ const definition = tools.get(call.toolName);+ return definition === undefined ? undefined : { call, definition };+ },+ };+}+async function resolveApproval(- tools: ReturnType<typeof buildToolSet>,+ tools: HarnessToolMap,toolName: string,input: unknown,session?: Session,- options: { readonly abortSignal?: AbortSignal } = {},+ options: {+ readonly abortSignal?: AbortSignal;+ readonly approvedTools?: ReadonlySet<string>;+ } = {},): Promise<unknown> {- const approval = buildToolApproval(tools, options.abortSignal);+ const approval = buildToolApproval({+ abortSignal: options.abortSignal,+ approvedTools: options.approvedTools ?? new Set(),+ ...catalogOf(tools),+ });const activeSession = session ?? {auth: { current: null, initiator: null },sessionId: "session-1",@@ -61,7 +78,7 @@ async function resolveApproval(messages: [],runtimeContext: {},toolCall: { input, toolCallId: "call_1", toolName } as never,- tools,+ tools: buildToolSet({ ...catalogOf(tools), tools }),toolsContext: {} as never,}),);@@ -131,7 +148,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await executeSdkTool({abortSignal: abortController.signal,tool: result.observe_options,@@ -169,7 +186,7 @@ describe("buildToolSet", () => {turn: { id: "turn-1", sequence: 0 },});- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await contextStorage.run(ctx, () =>executeSdkTool({abortSignal: abortController.signal,@@ -206,7 +223,7 @@ describe("buildToolSet", () => {});await contextStorage.run(ctx, async () => {- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });const execute = (result.stream_progress as {readonly execute?: (@@ -254,7 +271,7 @@ describe("buildToolSet", () => {turn: { id: "turn-1", sequence: 0 },});- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await contextStorage.run(ctx, () => executeSdkTool({ tool: result.observe_signal }));expect(receivedSignal).toBeInstanceOf(AbortSignal);@@ -287,7 +304,7 @@ describe("buildToolSet", () => {turn: { id: "turn-1", sequence: 0 },});- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await contextStorage.run(ctx, () =>executeSdkTool({ tool: result.observe_call_id, toolCallId: "call_observe" }),);@@ -325,7 +342,7 @@ describe("buildToolSet", () => {{ content: "Let me check.", role: "assistant" },];- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await contextStorage.run(ctx, () =>executeSdkTool({ messages, tool: result.observe_messages }),);@@ -351,7 +368,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });expect(getJsonSchema(result.echo_city)).toEqual(schema);});@@ -375,7 +392,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });expect(getOutputJsonSchema(result.summarize)).toEqual(outputSchema);});@@ -421,7 +438,7 @@ describe("buildToolSet", () => {]),);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });for (const tool of Object.values(result)) {for (const schema of [tool.inputSchema, tool.outputSchema]) {@@ -461,7 +478,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });expect(getJsonSchema(result.pick_color)).toEqual(schema);});@@ -496,6 +513,7 @@ describe("buildToolSet", () => {const result = buildToolSet({disabledProviderTools: new Set(["web_search"]),+ ...catalogOf(tools),tools,});@@ -530,6 +548,7 @@ describe("buildToolSet", () => {const result = await buildToolSetWithProviderTools({profile: resolveModelProfile(model),+ ...catalogOf(tools),tools,});@@ -555,6 +574,7 @@ describe("buildToolSet", () => {const result = await buildToolSetWithProviderTools({profile: resolveModelProfile("openai/gpt-5.4"),+ ...catalogOf(tools),tools,});@@ -578,6 +598,7 @@ describe("buildToolSet", () => {]);const result = await buildToolSetWithProviderTools({profile: resolveModelProfile("openai/gpt-5.4"),+ ...catalogOf(tools),tools,});const search = result.web_search!;@@ -597,6 +618,7 @@ describe("buildToolSet", () => {const disabled = await buildToolSetWithProviderTools({profile: resolveModelProfile("openai/gpt-5.4"),+ ...catalogOf(tools),tools,disabledProviderTools: new Set(["web_search"]),});@@ -604,6 +626,7 @@ describe("buildToolSet", () => {const direct = await buildToolSetWithProviderTools({profile: resolveModelProfile(directModel("openai.chat")),+ ...catalogOf(tools),tools,});expect(direct.web_search).toMatchObject({ id: "openai.web_search" });@@ -627,6 +650,7 @@ describe("buildToolSet", () => {const result = await buildToolSetWithProviderTools({profile: resolveModelProfile(directModel("some-provider")),+ ...catalogOf(tools),tools,});@@ -646,11 +670,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({- tools,- });-- await expect(resolveApproval(result, "dangerous_tool", {})).resolves.toBeUndefined();+ await expect(resolveApproval(tools, "dangerous_tool", {})).resolves.toBeUndefined();});it("forwards toModelOutput to the SDK tool", () => {@@ -671,7 +691,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });const sdkTool = result.report as { toModelOutput?: (...args: unknown[]) => unknown };expect(sdkTool.toModelOutput).toBeTypeOf("function");@@ -690,7 +710,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });const sdkTool = result.echo as { toModelOutput?: unknown };expect(sdkTool.toModelOutput).toBeTypeOf("function");@@ -714,7 +734,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });const sdkTool = result.report as {toModelOutput?: (options: { toolCallId: string; input: unknown; output: unknown }) => unknown;};@@ -742,7 +762,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await expect(executeSdkTool({@@ -768,7 +788,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await expect(executeSdkTool({ tool: result.report })).resolves.toBe(output);});@@ -786,7 +806,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await expect(executeSdkTool({ tool: result.maybe_empty })).resolves.toBeNull();});@@ -808,7 +828,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await expect(projectSdkToolOutput({@@ -848,7 +868,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await expect(projectSdkToolOutput({ output: { ok: true }, tool: result.screenshot }),@@ -880,7 +900,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });+ const result = buildToolSet({ ...catalogOf(tools), tools });await expect(projectSdkToolOutput({@@ -918,9 +938,8 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });- await expect(resolveApproval(result, "dangerous", {})).resolves.toBe("user-approval");- await expect(resolveApproval(result, "safe", {})).resolves.toBe("not-applicable");+ await expect(resolveApproval(tools, "dangerous", {})).resolves.toBe("user-approval");+ await expect(resolveApproval(tools, "safe", {})).resolves.toBe("not-applicable");});it("preserves async AI SDK 7 approval statuses", async () => {@@ -937,8 +956,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });- await expect(resolveApproval(result, "delete_account", {})).resolves.toEqual({+ await expect(resolveApproval(tools, "delete_account", {})).resolves.toEqual({type: "denied",reason: "Account is protected.",});@@ -958,10 +976,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({- tools,- });- await expect(resolveApproval(result, "bash", {})).resolves.toBe("user-approval");+ await expect(resolveApproval(tools, "bash", {})).resolves.toBe("user-approval");});it("never() skips approval", async () => {@@ -978,10 +993,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({- tools,- });- await expect(resolveApproval(result, "bash", {})).resolves.toBe("not-applicable");+ await expect(resolveApproval(tools, "bash", {})).resolves.toBe("not-applicable");});it("once() requires approval when tool not yet approved", async () => {@@ -998,10 +1010,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({- tools,- });- await expect(resolveApproval(result, "bash", {})).resolves.toBe("user-approval");+ await expect(resolveApproval(tools, "bash", {})).resolves.toBe("user-approval");});it("once() skips approval when tool already approved", async () => {@@ -1018,11 +1027,9 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({- approvedTools: new Set(["bash"]),- tools,- });- await expect(resolveApproval(result, "bash", {})).resolves.toBe("not-applicable");+ await expect(+ resolveApproval(tools, "bash", {}, undefined, { approvedTools: new Set(["bash"]) }),+ ).resolves.toBe("not-applicable");});it("tool without approval defaults to false when another tool has an override", async () => {@@ -1048,11 +1055,8 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({- tools,- });- await expect(resolveApproval(result, "bash", {})).resolves.toBe("user-approval");- await expect(resolveApproval(result, "write_file", {})).resolves.toBeUndefined();+ await expect(resolveApproval(tools, "bash", {})).resolves.toBe("user-approval");+ await expect(resolveApproval(tools, "write_file", {})).resolves.toBeUndefined();});it("passes toolInput from the AI SDK into approval", async () => {@@ -1073,9 +1077,8 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });const toolInput = { teamId: "team_abc", limit: 20 };- await resolveApproval(result, "vercel__list_projects", toolInput);+ await resolveApproval(tools, "vercel__list_projects", toolInput);expect(capturedInput).toEqual(toolInput);});@@ -1098,8 +1101,7 @@ describe("buildToolSet", () => {],]);- const result = buildToolSet({ tools });- await resolveApproval(result, "vercel__list_projects", {});+ await resolveApproval(tools, "vercel__list_projects", {});expect(capturedCallId).toBe("call_1");});@@ -1123,8 +1125,7 @@ describe("buildToolSet", () => {]);const abortSignal = new AbortController().signal;- const result = buildToolSet({ tools });- await resolveApproval(result, "deploy", {}, undefined, { abortSignal });+ await resolveApproval(tools, "deploy", {}, undefined, { abortSignal });expect(capturedSignal).toBe(abortSignal);});@@ -1173,8 +1174,7 @@ describe("buildToolSet", () => {turn: { id: "turn_current", sequence: 2 },};- const result = buildToolSet({ tools });- await expect(resolveApproval(result, "delete_project", {}, session)).resolves.toBe(+ await expect(resolveApproval(tools, "delete_project", {}, session)).resolves.toBe("user-approval",);@@ -1224,12 +1224,11 @@ describe("buildToolSet", () => {sessionId: "schedule-session",turn: { id: "human-turn", sequence: 1 },};- const result = buildToolSet({ tools });- await expect(resolveApproval(result, "refund", {}, scheduleSession)).resolves.toBe(+ await expect(resolveApproval(tools, "refund", {}, scheduleSession)).resolves.toBe("not-applicable",);- await expect(resolveApproval(result, "refund", {}, humanResumedSession)).resolves.toBe(+ await expect(resolveApproval(tools, "refund", {}, humanResumedSession)).resolves.toBe("user-approval",);});@@ -1253,22 +1252,30 @@ describe("buildToolSet", () => {],]);- const withCompoundKey = buildToolSet({- approvedTools: new Set(["vercel__list_projects:team_abc"]),- tools,- });await expect(- resolveApproval(withCompoundKey, "vercel__list_projects", {- teamId: "team_abc",- limit: 10,- }),+ resolveApproval(+ tools,+ "vercel__list_projects",+ {+ teamId: "team_abc",+ limit: 10,+ },+ undefined,+ { approvedTools: new Set(["vercel__list_projects:team_abc"]) },+ ),).resolves.toBe("not-applicable");await expect(- resolveApproval(withCompoundKey, "vercel__list_projects", {- teamId: "team_xyz",- limit: 10,- }),+ resolveApproval(+ tools,+ "vercel__list_projects",+ {+ teamId: "team_xyz",+ limit: 10,+ },+ undefined,+ { approvedTools: new Set(["vercel__list_projects:team_abc"]) },+ ),).resolves.toBe("user-approval");});});diff --git a/packages/eve/src/harness/tools.ts b/packages/eve/src/harness/tools.tsindex 16a91669a..b2ee0f3f0 100644--- a/packages/eve/src/harness/tools.ts+++ b/packages/eve/src/harness/tools.ts@@ -3,9 +3,10 @@ import { type ToolApprovalConfiguration, type ToolApprovalStatus, type ToolSet,import type { ModelProfile } from "#harness/model-profile.js";import { isObject } from "#shared/guards.js";import type { HarnessToolDefinition } from "#harness/execute-tool.js";-import { resolveApprovalPolicy, type ApprovalStatus } from "#approval/definition.js";+import { resolveApprovalPolicy } from "#approval/definition.js";import { resolveWebSearchBackend, resolveWebSearchProviderTool } from "#harness/provider-tools.js";-import type { HarnessToolMap } from "#harness/types.js";+import { entryModelOutput, stubbedCall } from "#harness/execute-call.js";+import type { CallResolver, HarnessToolMap } from "#harness/types.js";import { buildCallbackContext } from "#context/build-callback-context.js";import { loadContext } from "#context/container.js";import { isAuthorizationSignal, modelFacingAuthorizationOutput } from "#harness/authorization.js";@@ -13,25 +14,15 @@ import { stashToolInterrupt } from "#harness/tool-interrupts.js";import { isApprovedToolCall, markApprovalRecheck } from "#harness/approval-recheck.js";import { toModelSchema } from "#tools/schema.js";import { normalizeToolJsonOutput } from "#harness/tool-model-output.js";-import { toolCallModelOutput } from "#harness/tool-call-io.js";import type { ToolExecuteOptions } from "#tools/definition.js";import { isAsyncIterable } from "#shared/async-iterable.js";import { executeWithToolStub } from "#tool-stubs/execute.js";import { iterateAsApprover, runAsApprover } from "#harness/hitl/approved-call-callers.js";-type NativeApprovalStatus = Exclude<ApprovalStatus, boolean>;--type ApprovalFn = (- toolInput: unknown,- callId: string,- abortSignal: AbortSignal | undefined,- recheck: boolean,-) => Promise<NativeApprovalStatus>;--const toolApprovals = new WeakMap<object, ApprovalFn>();-/**- * Builds an AI SDK `ToolSet` from unified harness tool definitions.+ * Builds an AI SDK `ToolSet` from unified harness tool definitions, described+ * by `describe`. Each call's output is projected by the entry `resolve`+ * resolves it to.** Tools without `execute` are surfaced to the model as client-side tools* (no server execution).@@ -42,8 +33,9 @@ const toolApprovals = new WeakMap<object, ApprovalFn>();* retry call so the request can proceed without it.*/export function buildToolSet(input: {- readonly approvedTools?: ReadonlySet<string>;+ readonly describe: (definition: HarnessToolDefinition) => string;readonly disabledProviderTools?: ReadonlySet<string>;+ readonly resolve: CallResolver;readonly tools: HarnessToolMap;}): ToolSet {const tools: Record<string, ToolSet[string]> = {};@@ -54,77 +46,64 @@ export function buildToolSet(input: {continue;}- const authorToModelOutput = definition.toModelOutput;- const approval = buildApprovalFn(definition, input);- const aiTool = tool({- description: definition.description,- execute: wrapToolExecute(definition),+ tools[definition.name] = tool({+ description: input.describe(definition),+ execute: wrapToolExecute(definition, input.resolve),inputSchema: toModelSchema(definition.inputSchema, "input"),strict: false,outputSchema: toModelSchema(definition.outputSchema, "output"),- ...(definition.execute !== undefined || authorToModelOutput !== undefined+ ...(definition.execute !== undefined || definition.toModelOutput !== undefined? {toModelOutput: ({+ input: callInput,output,toolCallId,}: {+ readonly input: unknown;readonly output: unknown;readonly toolCallId?: string;- }) => toolCallModelOutput(definition, output, toolCallId),+ }) =>+ entryModelOutput(+ input.resolve({ input: callInput, toolName: definition.name })?.definition ??+ definition,+ output,+ toolCallId,+ ),}: {}),});- tools[definition.name] = aiTool;- if (definition.approval !== undefined) {- toolApprovals.set(aiTool, approval);- }}return tools as ToolSet;}-/**- * Builds a ToolSet from an ordered list of harness definitions.- *- * The first definition for a name wins, matching the dynamic-tool scope- * ordering where step tools override turn/session tools.- */-export function buildToolSetFromDefinitions(input: {- readonly approvedTools?: ReadonlySet<string>;- readonly disabledProviderTools?: ReadonlySet<string>;- readonly tools: readonly HarnessToolDefinition[];-}): ToolSet {- const tools = new Map<string, HarnessToolDefinition>();- for (const definition of input.tools) {- if (!tools.has(definition.name)) {- tools.set(definition.name, definition);- }- }- return buildToolSet({- approvedTools: input.approvedTools,- disabledProviderTools: input.disabledProviderTools,- tools,- });-}-/*** Wraps a tool's `execute` so a returned {@link AuthorizationSignal} is* stashed out-of-band ({@link stashToolInterrupt}) for the park detector while* the AI SDK records an opaque {@link AuthorizationPendingModelOutput} that* omits OAuth URLs, user codes, and hook URLs from model-facing history.- * Returns `undefined` for client-side tools (no `execute`).+ * Returns `undefined` for client-side tools (no `execute`). With `resolve`,+ * output is normalized under the name of the entry each call resolves to, so+ * an error from a call through `execute` names the entry, and a tool stub+ * matches that entry and its input, as it would a direct call.*/export function wrapToolExecute(definition: HarnessToolDefinition,+ resolve?: CallResolver,): ((input: any, options: ToolExecuteOptions) => Promise<any> | AsyncIterable<any>) | undefined {const execute = definition.execute;if (execute === undefined) return undefined;return (input, options) => {+ const self = { input, toolName: definition.name };+ const resolved = resolve?.(self);+ const toolName = resolved?.definition.name ?? definition.name;+ const call = stubbedCall(self, resolved);+ const run = () => execute(input, options);let output: unknown;try {output = runAsApprover(options.toolCallId, () =>- executeWithToolStub(definition.name, input, options, () => execute(input, options)),+ call === undefined ? run() : executeWithToolStub(call.toolName, call.input, options, run),);} catch (error) {return Promise.reject(error);@@ -133,13 +112,13 @@ export function wrapToolExecute(if (isAsyncIterable(output)) {return normalizeToolExecuteIterable(iterateAsApprover(options.toolCallId, output),- definition.name,+ toolName,options,);}return Promise.resolve(output).then((value) =>- normalizeToolExecuteOutput(value, definition.name, options),+ normalizeToolExecuteOutput(value, toolName, options),);};}@@ -191,19 +170,14 @@ function normalizeToolExecuteOutput(* a gateway fallback provider has rejected a provider-specific tool.*/export async function buildToolSetWithProviderTools(input: {- readonly approvedTools?: ReadonlySet<string>;+ readonly describe: (definition: HarnessToolDefinition) => string;readonly disabledProviderTools?: ReadonlySet<string>;readonly profile: ModelProfile;+ readonly resolve: CallResolver;readonly tools: HarnessToolMap;}): Promise<ToolSet> {const disabled = input.disabledProviderTools;- const tools: ToolSet = {- ...buildToolSet({- approvedTools: input.approvedTools,- disabledProviderTools: disabled,- tools: input.tools,- }),- };+ const tools: ToolSet = { ...buildToolSet(input) };for (const definition of input.tools.values()) {const handling = definition.behavior?.handling;@@ -224,27 +198,25 @@ export async function buildToolSetWithProviderTools(input: {return tools;}-function buildApprovalFn(- definition: HarnessToolDefinition,- input: { readonly approvedTools?: ReadonlySet<string> },-): ApprovalFn {- return async (toolInput, callId, abortSignal, recheck) => {- if (definition.approval === undefined) return undefined;-- const toolInputRecord = isObject(toolInput) ? toolInput : undefined;- const context = {- ...buildCallbackContext(),- abortSignal: abortSignal ?? new AbortController().signal,- approvedTools: input.approvedTools ?? new Set<string>(),- callId,- toolInput: toolInputRecord,- toolName: definition.name,- };-- const status = await resolveApprovalPolicy(definition.approval)(- recheck ? markApprovalRecheck(context) : context,- );- return typeof status === "boolean" ? (status ? "user-approval" : "not-applicable") : status;+/**+ * Builds the AI SDK 7 call-level approval policy: each call is approved by the+ * entry it runs, under that entry's name and with its own input.+ */+export function buildToolApproval(input: {+ readonly abortSignal?: AbortSignal;+ readonly approvedTools: ReadonlySet<string>;+ readonly resolve: CallResolver;+}): ToolApprovalConfiguration<ToolSet, Record<string, unknown>> {+ return async ({ toolCall, messages }) => {+ const resolved = input.resolve(toolCall);+ if (resolved === undefined) return undefined;+ return await evaluateApproval(resolved.definition, {+ abortSignal: input.abortSignal,+ approvedTools: input.approvedTools,+ callId: toolCall.toolCallId,+ input: resolved.call.input,+ recheck: isApprovedToolCall(messages, toolCall.toolCallId),+ });};}@@ -261,12 +233,7 @@ export async function recheckApprovedCall(readonly approvedTools?: ReadonlySet<string>;},): Promise<{ readonly denied: boolean; readonly reason?: string }> {- const status = await buildApprovalFn(definition, { approvedTools: call.approvedTools })(- call.input,- call.callId,- call.abortSignal,- true,- );+ const status = await evaluateApproval(definition, { ...call, recheck: true });if (status === "denied") return { denied: true };if (typeof status === "object" && status !== null && status.type === "denied") {return { denied: true, reason: status.reason };@@ -274,21 +241,30 @@ export async function recheckApprovedCall(return { denied: false };}-/** Builds the AI SDK 7 call-level approval policy for an assembled tool set. */-export function buildToolApproval(- tools: ToolSet,- abortSignal?: AbortSignal,-): ToolApprovalConfiguration<ToolSet, Record<string, unknown>> {- return async ({ toolCall, messages }) => {- const toolDefinition = tools[toolCall.toolName];- if (toolDefinition === undefined) return undefined;-- const approval = toolApprovals.get(toolDefinition);- return (await approval?.(- toolCall.input,- toolCall.toolCallId,- abortSignal,- isApprovedToolCall(messages, toolCall.toolCallId),- )) as ToolApprovalStatus;+/** Runs `definition`'s approval policy for one call, in the status form the AI SDK reads. */+async function evaluateApproval(+ definition: HarnessToolDefinition,+ call: {+ readonly abortSignal?: AbortSignal;+ readonly approvedTools?: ReadonlySet<string>;+ readonly callId: string;+ readonly input: unknown;+ readonly recheck: boolean;+ },+): Promise<ToolApprovalStatus | undefined> {+ if (definition.approval === undefined) return undefined;+ const context = {+ ...buildCallbackContext(),+ abortSignal: call.abortSignal ?? new AbortController().signal,+ approvedTools: call.approvedTools ?? new Set<string>(),+ callId: call.callId,+ toolInput: isObject(call.input) ? call.input : undefined,+ toolName: definition.name,};+ const status = await resolveApprovalPolicy(definition.approval)(+ call.recheck ? markApprovalRecheck(context) : context,+ );+ return (+ typeof status === "boolean" ? (status ? "user-approval" : "not-applicable") : status+ ) as ToolApprovalStatus;}diff --git a/packages/eve/src/harness/types.ts b/packages/eve/src/harness/types.tsindex d55b52c23..73169cde7 100644--- a/packages/eve/src/harness/types.ts+++ b/packages/eve/src/harness/types.ts@@ -252,6 +252,24 @@ export type StepFn = (session: HarnessSession, input?: StepInput) => Promise<Ste*/export type HarnessToolMap = ReadonlyMap<string, HarnessToolDefinition>;+/** Looks up the definition a call runs by its entry name. */+export type HarnessToolLookup = Pick<HarnessToolMap, "get">;++/** The parts of a model tool call that name what it runs. */+export interface ToolCallLike {+ readonly input: unknown;+ readonly toolName: string;+}++/** A model tool call as the call to its entry, with the entry it runs. */+export interface ResolvedCall<T extends ToolCallLike> {+ readonly call: T;+ readonly definition: HarnessToolDefinition;+}++/** Resolves a model tool call to the entry it runs, if it reaches one. */+export type CallResolver = <T extends ToolCallLike>(toolCall: T) => ResolvedCall<T> | undefined;+/*** Callback that writes one event to the event stream.*diff --git a/packages/eve/src/instrumentation/lifecycle.ts b/packages/eve/src/instrumentation/lifecycle.tsindex 4e997975d..f9ce0999e 100644--- a/packages/eve/src/instrumentation/lifecycle.ts+++ b/packages/eve/src/instrumentation/lifecycle.ts@@ -461,7 +461,6 @@ export type InstrumentationToolCallKind =export interface InstrumentationToolCallStartedEvent {/** Durable dispatch metadata, when this call is owned by eve's tool loop. */readonly kind?: InstrumentationToolCallKind;- readonly parentCallId?: string;readonly isWorkflowTool?: boolean;readonly frameworkTool?: boolean;readonly startedAtMs?: number;diff --git a/packages/eve/src/instrumentation/native-events.test.ts b/packages/eve/src/instrumentation/native-events.test.tsindex 7ed6ef408..3074d328d 100644--- a/packages/eve/src/instrumentation/native-events.test.ts+++ b/packages/eve/src/instrumentation/native-events.test.ts@@ -259,7 +259,6 @@ describe("createInstrumentationHandleEvent", () => {nodeId: "workers",subagentName: "worker",},- { callId: "skill-1", input: { name: "research" }, kind: "load-skill" },{callId: "remote-1",description: "Call a remote agent.",@@ -388,7 +387,7 @@ describe("createInstrumentationHandleEvent", () => {);});- expect(events.slice(0, 5)).toEqual([+ expect(events.slice(0, 4)).toEqual([{callId: "delegate-1",frameworkTool: false,@@ -401,17 +400,6 @@ describe("createInstrumentationHandleEvent", () => {scope,type: "tool.call.started",},- {- callId: "skill-1",- frameworkTool: false,- startedAtMs: 1_000,- idempotencyKey: actionIdempotencyKey("session-1", "turn-1", "skill-1"),- input: { name: "research" },- kind: "load-skill",- toolName: "load_skill",- scope,- type: "tool.call.started",- },{callId: "remote-1",frameworkTool: false,@@ -489,7 +477,7 @@ describe("createInstrumentationHandleEvent", () => {type: "tool.call.completed",usage: undefined,});- expect(events).toHaveLength(8);+ expect(events).toHaveLength(7);expect(events.every(Object.isFrozen)).toBe(true);});diff --git a/packages/eve/src/instrumentation/native-events.ts b/packages/eve/src/instrumentation/native-events.tsindex 3d17a93d3..f2f59e6ea 100644--- a/packages/eve/src/instrumentation/native-events.ts+++ b/packages/eve/src/instrumentation/native-events.ts@@ -35,6 +35,7 @@ import {type RuntimeActionResult,} from "#shared/action-types.js";import type { ChannelAudience } from "#shared/channel-audience.js";+import { requestedSkill } from "#shared/action-request-name.js";export interface CreateInstrumentationHandleEventInput {readonly isFrameworkTool?: (name: string) => boolean;@@ -221,7 +222,7 @@ async function publishActionStarts(type: "tool.call.started",callId: action.callId,toolName: actionName(action),- frameworkTool: input.isFrameworkTool?.(actionName(action)) === true,+ frameworkTool: isFrameworkAction(action, input),scope,idempotencyKey: toolCallIdempotencyKey(scope, action.callId, 0),startedAtMs: Date.now(),@@ -232,16 +233,13 @@ async function publishActionStarts(await hooks.publish(Object.freeze({callId: action.callId,- ...(action.kind === "tool-call" && action.parentCallId !== undefined- ? { parentCallId: action.parentCallId }- : undefined),startedAtMs,idempotencyKey,input: capturesInputs ? action.input : undefined,...(deferred ? { isWorkflowTool: true } : undefined),kind: action.kind === "workflow-tool-call" ? "tool-call" : action.kind,toolName: actionName(action),- frameworkTool: input.isFrameworkTool?.(actionName(action)) === true,+ frameworkTool: isFrameworkAction(action, input),scope,type: "tool.call.started",} satisfies InstrumentationToolCallStartedEvent),@@ -326,9 +324,17 @@ function actionUsage(result: RuntimeActionResult): InstrumentationUsage | undefireturn usage;}+/** A skill load runs eve's skill loader; any other call is eve's when its tool is. */+function isFrameworkAction(+ action: RuntimeActionRequest,+ input: Pick<CreateInstrumentationHandleEventInput, "isFrameworkTool">,+): boolean {+ return action.kind === "load-skill" || input.isFrameworkTool?.(actionName(action)) === true;+}+function actionName(action: RuntimeActionRequest): string {if (action.kind === "tool-call" || action.kind === "workflow-tool-call") return action.toolName;- if (action.kind === "load-skill") return "load_skill";+ if (action.kind === "load-skill") return requestedSkill(action);return action.name;}diff --git a/packages/eve/src/instrumentation/runtime.ts b/packages/eve/src/instrumentation/runtime.tsindex b97ee4628..7d3eb21e6 100644--- a/packages/eve/src/instrumentation/runtime.ts+++ b/packages/eve/src/instrumentation/runtime.ts@@ -75,6 +75,7 @@ export { getInstrumentationRuntime, registerInstrumentationRuntime };export { initializeSessionInstrumentation };const TURN_TRACE_STATE_KEY = "eve.harness.turnTrace";import type { SessionTraceSeed } from "#context/keys.js";+import type { TASK_CANCEL_TOOL_NAME, TASK_WAIT_TOOL_NAME } from "#protocol/task-tools.js";interface InstrumentedStepSession {readonly sessionId: string;@@ -177,7 +178,7 @@ export interface SessionInstrumentation {export interface ExecutionInstrumentation {readonly instrumentTaskToolCall: (input: {readonly callId: string;- readonly toolName: "task_wait" | "task_cancel";+ readonly toolName: typeof TASK_WAIT_TOOL_NAME | typeof TASK_CANCEL_TOOL_NAME;readonly startedAtMs: number;readonly completedAtMs: number;readonly input: unknown;diff --git a/packages/eve/src/internal/authored-definition/core.ts b/packages/eve/src/internal/authored-definition/core.tsindex 4dc81e52e..abfb86db6 100644--- a/packages/eve/src/internal/authored-definition/core.ts+++ b/packages/eve/src/internal/authored-definition/core.ts@@ -7,6 +7,7 @@ import type {import type { ScheduleDefinition, ScheduleRunHandler } from "#public/definitions/schedule.js";import type { SkillDefinition, SkillFileContent } from "#public/definitions/skill.js";import {+ expectAgentToolExposure,expectBoolean,expectFunction,expectObjectRecord,@@ -118,7 +119,7 @@ export function normalizeAgentDefinition(}if (record.tool !== undefined) {- definition.tool = expectBoolean(record.tool, message);+ definition.tool = expectAgentToolExposure(record.tool, message);}if (record.limits !== undefined) {@@ -425,11 +426,18 @@ export function normalizeInstructionsDefinition(*/export function normalizeSkillDefinition(value: unknown, message: string): SkillDefinition {const record = expectObjectRecord(value, message);- expectOnlyKnownKeys(record, ["description", "files", "license", "markdown", "metadata"], message);+ expectOnlyKnownKeys(+ record,+ ["deferred", "description", "files", "license", "markdown", "metadata"],+ message,+ );const definition: Mutable<SkillDefinition> = {description: expectString(record.description, message),markdown: expectString(record.markdown, message),};+ if (record.deferred !== undefined) {+ definition.deferred = expectBoolean(record.deferred, message);+ }const license = record.license;const metadata = getOptionalStringRecordProperty(record, "metadata", message);diff --git a/packages/eve/src/internal/authored-definition/schema-backed.ts b/packages/eve/src/internal/authored-definition/schema-backed.tsindex ed47013b0..3ff2ba976 100644--- a/packages/eve/src/internal/authored-definition/schema-backed.ts+++ b/packages/eve/src/internal/authored-definition/schema-backed.ts@@ -106,6 +106,11 @@ export function normalizeToolDefinition(value: unknown, message: string): Normal}if (isWebSearchToolDefinition(value)) {const record = expectObjectRecord(value, message);+ if (record.deferred !== undefined) {+ throw new Error(+ `${message} Provider tools can't be deferred: the provider has to see their definition. Remove "deferred".`,+ );+ }expectOnlyKnownKeys(record, ["kind", "provider"], message);const provider = expectString(record.provider, message);if (provider !== "exa" && provider !== "parallel" && provider !== "browserbase") {@@ -132,6 +137,7 @@ export function normalizeToolDefinition(value: unknown, message: string): Normalrecord,["availableInSubagents",+ "deferred","endsTurn","label","auth",@@ -164,6 +170,7 @@ export function normalizeToolDefinition(value: unknown, message: string): Normalrecord.availableInSubagents === undefined? undefined: expectBoolean(record.availableInSubagents, message),+ deferred: record.deferred === undefined ? undefined : expectBoolean(record.deferred, message),description: expectString(record.description, message),endsTurn:record.endsTurn === undefined || typeof record.endsTurn === "boolean"diff --git a/packages/eve/src/internal/authored-module.ts b/packages/eve/src/internal/authored-module.tsindex 89099e895..8f0663c23 100644--- a/packages/eve/src/internal/authored-module.ts+++ b/packages/eve/src/internal/authored-module.ts@@ -1,7 +1,7 @@import type { ModuleDefinitionExport } from "#public/definitions/source.js";import type { JsonObject } from "#shared/json.js";import type { ModuleSourceRef } from "#shared/source-ref.js";-import type { AgentModelOptionsDefinition } from "#shared/agent-definition.js";+import type { AgentModelOptionsDefinition, AgentToolExposure } from "#shared/agent-definition.js";/*** Returns the selected authored module export from one namespace.@@ -60,6 +60,11 @@ export function expectBoolean(value: unknown, message: string): boolean {return value;}+export function expectAgentToolExposure(value: unknown, message: string): AgentToolExposure {+ if (value === "deferred") return value;+ return expectBoolean(value, `${message} Expected "tool" to be true, false, or "deferred".`);+}+/*** Returns one positive integer or throws.*/diff --git a/packages/eve/src/internal/helpers/markdown.ts b/packages/eve/src/internal/helpers/markdown.tsindex e050fe688..47365a54e 100644--- a/packages/eve/src/internal/helpers/markdown.ts+++ b/packages/eve/src/internal/helpers/markdown.ts@@ -178,6 +178,14 @@ function applyOptionalSkillFrontmatter(if (metadata !== undefined) {rawDefinition.metadata = metadata;}++ const deferred = frontmatter.deferred;+ if (deferred !== undefined && deferred !== null) {+ if (typeof deferred !== "boolean") {+ throw new Error('Expected "deferred" frontmatter to be true or false.');+ }+ rawDefinition.deferred = deferred;+ }}function toOptionalString(value: unknown, fieldName: string): string | undefined {diff --git a/packages/eve/src/internal/testing/catalog-fixtures.ts b/packages/eve/src/internal/testing/catalog-fixtures.tsnew file mode 100644index 000000000..38bc08462--- /dev/null+++ b/packages/eve/src/internal/testing/catalog-fixtures.ts@@ -0,0 +1,307 @@+/**+ * Building blocks for tests of the step catalog: agent entries of each kind,+ * a session bundle with skills, in-memory connections, and a session context+ * that holds them.+ */++import { ConnectionAuthorizationRequiredError } from "#connections/errors.js";+import { ContextContainer, contextStorage } from "#context/container.js";+import { AuthKey, SessionIdKey } from "#context/keys.js";+import { ConnectionRegistryKey } from "#context/providers/connection-key.js";+import { buildStepCatalog } from "#execution/catalog/step-catalog.js";+import { CallbackBaseUrlKey } from "#harness/authorization.js";+import type { HarnessToolDefinition } from "#harness/execute-tool.js";+import type { ConnectionRegistry } from "#runtime/connections/registry-types.js";+import { BundleKey, type CompiledBundle } from "#runtime/sessions/runtime-context-keys.js";+import type { ResolvedConnectionDefinition } from "#runtime/types.js";+import {+ type ConnectionClient,+ type ConnectionToolMetadata,+ defineInteractiveAuthorization,+} from "#shared/connection-types.js";+import type { JsonObject } from "#shared/json.js";+import { toInputSchema } from "#tools/schema.js";++const OBJECT_SCHEMA: JsonObject = { type: "object", additionalProperties: false };++/** A tool with an inline `execute` that echoes its name and input. */+export function inlineTool(+ name: string,+ overrides: Partial<HarnessToolDefinition> & { readonly schema?: JsonObject } = {},+): HarnessToolDefinition {+ const { schema = OBJECT_SCHEMA, ...rest } = overrides;+ return {+ description: `${name} description`,+ execute: async (input: unknown) => ({ ran: name, input }),+ inputSchema: toInputSchema(schema),+ name,+ ...rest,+ };+}++/** A workflow tool; `task` runs each call as a background task. */+export function workflowTool(+ name: string,+ entryPoint: "execute" | "task" = "execute",+ overrides: Partial<HarnessToolDefinition> = {},+): HarnessToolDefinition {+ const workflowId = `workflow//./agent/tools/${name}//${entryPoint}`;+ return {+ behavior: {+ availability: [],+ handling: {+ kind: "dispatch",+ target: { entryPoint, kind: "workflow-tool-call", workflowId },+ },+ },+ description: `${name} description`,+ inputSchema: toInputSchema({ type: "object" }),+ name,+ workflowId,+ ...overrides,+ };+}++/** A declared subagent; every call to it starts a child session. */+export function subagentTool(+ name: string,+ overrides: Partial<HarnessToolDefinition> = {},+): HarnessToolDefinition {+ return {+ behavior: {+ availability: [],+ handling: {+ kind: "dispatch",+ target: { kind: "subagent-call", nodeId: `subagents/${name}`, subagentName: name },+ },+ },+ description: `${name} description`,+ inputSchema: toInputSchema({+ type: "object",+ properties: { message: { type: "string" } },+ required: ["message"],+ }),+ name,+ workflowId: `workflow//./agent/subagents/${name}//execute`,+ ...overrides,+ };+}++export function toolMap(...tools: readonly HarnessToolDefinition[]) {+ return new Map(tools.map((tool) => [tool.name, tool]));+}++export interface CatalogSkillSource {+ readonly deferred?: boolean;+ readonly description?: string;+ readonly markdown?: string;+ readonly name: string;+}++/** The parts of a session bundle the catalog reads: authored skills and dynamic subagent resolvers. */+export function catalogBundle(+ input: {+ /** The connections the agent declares. */+ readonly connections?: readonly ResolvedConnectionDefinition[];+ readonly dynamicSubagents?: readonly string[];+ readonly skills?: readonly CatalogSkillSource[];+ } = {},+): CompiledBundle {+ return {+ adapterRegistry: undefined as never,+ compiledArtifactsSource: undefined as never,+ graph: undefined as never,+ hookRegistry: undefined as never,+ moduleMap: undefined as never,+ nodeId: undefined,+ resolvedAgent: {+ config: { name: "test-agent" },+ connections: input.connections ?? [],+ dynamicSkillResolvers: [],+ dynamicToolResolvers: [],+ skills: (input.skills ?? []).map((skill) => ({+ deferred: skill.deferred,+ description: skill.description ?? `${skill.name} skill`,+ markdown: skill.markdown ?? `# ${skill.name}`,+ name: skill.name,+ })),+ } as never,+ subagentRegistry: {+ dynamicResolvers: (input.dynamicSubagents ?? []).map((name) => ({ name })),+ preparedTools: [],+ subagentsByName: new Map(),+ } as never,+ toolRegistry: { toolsByName: new Map() } as never,+ turnAgent: undefined as never,+ };+}++/** How a fake connection answers a request to list its tools. */+export type ConnectionListing = "listed" | "sign-in" | Error;++export interface FakeConnection {+ /** Each tool call the connection received, in order. */+ readonly calls: { readonly input: unknown; readonly tool: string }[];+ readonly client: ConnectionClient;+ readonly definition: ResolvedConnectionDefinition;+ listing: ConnectionListing;+ /** Callback URLs of the interactive sign-ins it started, in order. */+ readonly signIns: string[];+}++/**+ * An in-memory connection. With `signIn`, it supports interactive sign-in and+ * lists its tools once a sign-in completes, unless it `rejectsToken`.+ */+export function fakeConnection(input: {+ readonly approval?: ResolvedConnectionDefinition["approval"];+ readonly description?: string;+ readonly listing?: ConnectionListing;+ readonly name: string;+ readonly rejectsToken?: boolean;+ readonly signIn?: boolean;+ readonly tools: readonly ConnectionToolMetadata[];+}): FakeConnection {+ const calls: FakeConnection["calls"] = [];+ const signIns: string[] = [];+ const definition: ResolvedConnectionDefinition = {+ approval: input.approval,+ connectionName: input.name,+ description: input.description ?? `${input.name} service`,+ logicalPath: `connections/${input.name}.ts`,+ protocol: "mcp",+ sourceId: `connections/${input.name}.ts`,+ sourceKind: "module",+ url: `https://${input.name}.example.com/mcp`,+ };+ const connection: FakeConnection = {+ calls,+ client: {+ close: async () => {},+ connect: async () => {},+ executeTool: async (tool, args) => {+ calls.push({ input: args, tool });+ return { content: [{ type: "text", text: JSON.stringify({ tool, args }) }] };+ },+ getToolMetadata: async () => {+ if (connection.listing === "sign-in") {+ throw new ConnectionAuthorizationRequiredError(input.name);+ }+ if (connection.listing instanceof Error) throw connection.listing;+ return input.tools;+ },+ },+ definition:+ input.signIn === true+ ? {+ ...definition,+ authorization: defineInteractiveAuthorization({+ getToken: async () => {+ throw new ConnectionAuthorizationRequiredError(input.name);+ },+ startAuthorization: async ({ callbackUrl }) => {+ signIns.push(callbackUrl);+ return {+ challenge: { url: `https://idp.example.com/authorize?redirect=${callbackUrl}` },+ };+ },+ completeAuthorization: async () => {+ if (input.rejectsToken !== true) connection.listing = "listed";+ return { token: `${input.name}-token` };+ },+ }),+ }+ : definition,+ listing: input.listing ?? "listed",+ signIns,+ };+ return connection;+}++/** A registry over `connections`; the array may change to model dynamic connections. */+export function connectionRegistry(connections: readonly FakeConnection[]): ConnectionRegistry {+ const find = (name: string) => {+ const connection = connections.find((entry) => entry.definition.connectionName === name);+ if (connection === undefined) throw new Error(`No connection named "${name}".`);+ return connection;+ };+ return {+ dispose: async () => {},+ getClient: (name) => find(name).client,+ getConnectionApproval: (name) => find(name).definition.approval,+ getConnectionNames: () => connections.map((entry) => entry.definition.connectionName),+ getConnections: () => connections.map((entry) => entry.definition),+ };+}++/** Connection tool metadata with a JSON Schema input. */+export function connectionTool(+ name: string,+ inputSchema: JsonObject = { type: "object", properties: {} },+ description = `${name} description`,+): ConnectionToolMetadata {+ return { description, inputSchema, name };+}++/**+ * Alice's session with `tools`, `skills`, and `connections`, and the step+ * catalog built from it. `run` runs code inside the session's context.+ */+export function catalogContext(+ input: {+ readonly connections?: readonly FakeConnection[];+ readonly dynamicSubagents?: readonly string[];+ readonly session?: { readonly rootSessionId?: string };+ readonly skills?: readonly CatalogSkillSource[];+ readonly tools?: readonly HarnessToolDefinition[];+ } = {},+) {+ const ctx = new ContextContainer();+ ctx.set(AuthKey, {+ attributes: {},+ authenticator: "test",+ principalId: "alice",+ principalType: "user",+ });+ ctx.set(SessionIdKey, "catalog-session");+ ctx.set(CallbackBaseUrlKey, "https://agent.example.com");+ ctx.set(+ BundleKey,+ catalogBundle({+ connections: input.connections?.map((connection) => connection.definition),+ dynamicSubagents: input.dynamicSubagents,+ skills: input.skills,+ }),+ );+ if (input.connections !== undefined) {+ ctx.set(ConnectionRegistryKey, connectionRegistry(input.connections));+ }+ const catalog = buildStepCatalog({+ agentTools: toolMap(...(input.tools ?? [])),+ ctx,+ endsTurn: true,+ session: input.session ?? {},+ });+ return { catalog, ctx, run: <T>(fn: () => T) => contextStorage.run(ctx, fn) };+}++/** Names in eve's namespace, which nothing authored or dynamic may take. */+export const EVE_NAMESPACE_NAMES = ["eve", "eve__search", "eve__anything"];++/**+ * Names outside eve's namespace: the built-in tools' former names, which are+ * free again, and names that only contain "eve".+ */+export const NAMES_OUTSIDE_EVE_NAMESPACE = [+ "search",+ "execute",+ "task_wait",+ "task_cancel",+ "final_output",+ "steve",+ "eve_tool",+ "my__eve__x",+];++/** How every reserved-name error explains the rule. */+export const EVE_NAMESPACE_RESERVATION = 'eve reserves the "eve" namespace for its built-in tools';diff --git a/packages/eve/src/internal/vercel-agent-summary.ts b/packages/eve/src/internal/vercel-agent-summary.tsindex 7fb8741bf..4d32b36be 100644--- a/packages/eve/src/internal/vercel-agent-summary.ts+++ b/packages/eve/src/internal/vercel-agent-summary.ts@@ -171,8 +171,8 @@ export interface VercelEveSandboxEntry {/*** One authored skill exposed to the dashboard.*- * Skills are markdown procedures the model loads on demand through the- * framework-owned `load_skill` tool. The summary lists the available+ * Skills are markdown procedures the model loads on demand with+ * `eve__skill({ name })`. The summary lists the available* skills by `name` + `description` so the dashboard can render the* surface without pulling each skill's full markdown body. Consumers* that need the full content can drill in through a dedicated endpointdiff --git a/packages/eve/src/protocol/catalog-tools.ts b/packages/eve/src/protocol/catalog-tools.tsnew file mode 100644index 000000000..a9f0fdfa2--- /dev/null+++ b/packages/eve/src/protocol/catalog-tools.ts@@ -0,0 +1,13 @@+export const SEARCH_TOOL_NAME = "eve__search";+/** Calls a deferred entry or connection tool by name. */+export const CALL_TOOL_NAME = "eve__tool";+/** Loads a skill by name. */+export const SKILL_TOOL_NAME = "eve__skill";++/**+ * The name an `eve__skill` call takes once it resolves inside the harness,+ * where it reports as a `load-skill` action, and the tool name of a skill+ * load's client message part. The colon keeps it out of the tool namespace,+ * because skills and tools have separate names.+ */+export const SKILL_ENTRY_NAME = "eve:load-skill";diff --git a/packages/eve/src/protocol/message.ts b/packages/eve/src/protocol/message.tsindex a3147452a..89e2d8a50 100644--- a/packages/eve/src/protocol/message.ts+++ b/packages/eve/src/protocol/message.ts@@ -459,7 +459,7 @@ export interface TaskSettledStreamEvent {}/**- * Why the session cancelled a task call: the model called `task_cancel`,+ * Why the session cancelled a task call: the model called `eve__task_cancel`,* someone cancelled the turn (or, between turns, the tasks still working), or* the turn ended while the task still worked.*/diff --git a/packages/eve/src/protocol/reply-tool.ts b/packages/eve/src/protocol/reply-tool.tsnew file mode 100644index 000000000..0abdb7430--- /dev/null+++ b/packages/eve/src/protocol/reply-tool.ts@@ -0,0 +1,2 @@+/** The model-visible name of the tool a session with an output schema calls to reply with its structured result. */+export const REPLY_TOOL_NAME = "eve__reply";diff --git a/packages/eve/src/protocol/runtime-tools.ts b/packages/eve/src/protocol/runtime-tools.tsnew file mode 100644index 000000000..13f729ef8--- /dev/null+++ b/packages/eve/src/protocol/runtime-tools.ts@@ -0,0 +1,16 @@+/**+ * The namespace of the tools eve adds to sessions itself, such as+ * `eve__search` and `eve__task_wait`.+ */+const EVE_NAMESPACE = "eve";++/**+ * Why `name` is taken, for errors to quote; undefined when the name is free.+ * Nothing authored or dynamic may be named `eve` or start with `eve__`: a+ * connection or extension named `eve` would own every `eve__` name.+ */+export function eveNamespaceReservation(name: string): string | undefined {+ return name === EVE_NAMESPACE || name.startsWith(`${EVE_NAMESPACE}__`)+ ? `eve reserves the "${EVE_NAMESPACE}" namespace for its built-in tools`+ : undefined;+}diff --git a/packages/eve/src/protocol/session-projection.ts b/packages/eve/src/protocol/session-projection.tsindex d9a0c3350..c69a8a385 100644--- a/packages/eve/src/protocol/session-projection.ts+++ b/packages/eve/src/protocol/session-projection.ts@@ -5,6 +5,8 @@ import type {InputResolutionOutcome,UnstampedMessageStreamEvent,} from "#protocol/message.js";+import { SKILL_ENTRY_NAME } from "#protocol/catalog-tools.js";+import { requestedSkill } from "#shared/action-request-name.js";import type { RuntimeActionRequest, RuntimeActionResult } from "#shared/action-types.js";import type { InputRequest, InputResponse } from "#shared/input.js";import type { JsonValue } from "#shared/json.js";@@ -235,7 +237,7 @@ function settleCall(call: SessionCall, status: SessionCallStatus, error?: Actionfunction actionName(action: RuntimeActionRequest): string {switch (action.kind) {case "load-skill":- return "load_skill";+ return requestedSkill(action);case "subagent-call":case "remote-agent-call":return action.name;@@ -248,7 +250,7 @@ function actionName(action: RuntimeActionRequest): string {function resultName(result: RuntimeActionResult): string {switch (result.kind) {case "load-skill-result":- return "load_skill";+ return result.name ?? SKILL_ENTRY_NAME;case "subagent-result":return result.subagentName;default:diff --git a/packages/eve/src/protocol/task-tools.ts b/packages/eve/src/protocol/task-tools.tsindex bd91f8c77..9fe9d4d3e 100644--- a/packages/eve/src/protocol/task-tools.ts+++ b/packages/eve/src/protocol/task-tools.ts@@ -1,16 +1,16 @@import type { ActionResultStreamEvent } from "#protocol/message.js";-export const TASK_WAIT_TOOL_NAME = "task_wait";-export const TASK_CANCEL_TOOL_NAME = "task_cancel";+export const TASK_WAIT_TOOL_NAME = "eve__task_wait";+export const TASK_CANCEL_TOOL_NAME = "eve__task_cancel";-/** The task tools' names; authored tools cannot use them. */+/** The task tools' names. */export const TASK_TOOL_NAMES: readonly string[] = [TASK_WAIT_TOOL_NAME, TASK_CANCEL_TOOL_NAME];/*** Whether a tool call is the model managing its own tasks. The stream keeps* these calls so evals and traces can see them, but activity surfaces hide* them: people see the tasks, not the model waiting on or stopping them. The- * names are reserved, so no authored tool matches.+ * `eve` namespace is reserved, so no authored tool matches.*/export function isTaskControlTool(toolName: string): boolean {return TASK_TOOL_NAMES.includes(toolName);diff --git a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.ts b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.tsindex 7c94237fd..516599fb3 100644--- a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.ts+++ b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.test.ts@@ -9,7 +9,9 @@ import { ContextContainer, contextStorage } from "#context/container.js";import { enterSessionProjection, recordPublishedEvent } from "#harness/session-machine/current.js";import { SessionKey } from "#context/keys.js";import { mockChannelContext } from "#internal/testing/mocks/mock-channel-operations.js";+import { SEARCH_TOOL_NAME } from "#protocol/catalog-tools.js";import type { UnstampedMessageStreamEvent } from "#protocol/message.js";+import { TASK_WAIT_TOOL_NAME } from "#protocol/task-tools.js";import {chatSdkChannel,isNotImplemented,@@ -656,6 +658,49 @@ describe("chatSdkChannel", () => {expect(adapter.typingStatuses).toEqual(["Working..."]);});+ it("types a step's calls by their labels, leaving out the model's own task waits", async () => {+ const adapter = testAdapter();+ const bridge = chatSdkChannel({+ adapters: { test: adapter },+ state: memoryState(),+ userName: "bot",+ });+ const channelAdapter = withState(getAdapter(bridge.channel), {+ thread: serializedThread(),+ });+ const ctx = buildAdapterContext(channelAdapter, stubAccessor());+ const requested = (actions: readonly Record<string, unknown>[], presentation?: unknown) =>+ makeEvent("actions.requested", {+ actions,+ presentation,+ sequence: 1,+ stepIndex: 0,+ turnId: "turn-1",+ });+ const wait = {+ callId: "call-wait",+ input: {},+ kind: "tool-call",+ toolName: TASK_WAIT_TOOL_NAME,+ };++ await callEvent(+ channelAdapter,+ requested(+ [+ { callId: "call-search", input: {}, kind: "tool-call", toolName: SEARCH_TOOL_NAME },+ { callId: "call-refund", input: {}, kind: "tool-call", toolName: "refund_invoice" },+ wait,+ ],+ { "call-search": { label: "Search tools for “refunds”" } },+ ),+ ctx,+ );+ await callEvent(channelAdapter, requested([wait]), ctx);++ expect(adapter.typingStatuses).toEqual(["Search tools for “refunds”, Refund invoice..."]);+ });+it("streams assistant deltas by posting an anchor then editing it", async () => {const adapter = testAdapter();const bridge = chatSdkChannel({diff --git a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.ts b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.tsindex c33a3be31..f4d101f64 100644--- a/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.ts+++ b/packages/eve/src/public/channels/chat-sdk/chatSdkChannel.ts@@ -16,6 +16,7 @@ import { assertWithinLimit } from "#internal/attachments/limited-read.js";import { createLogger, extractErrorId, formatErrorHint } from "#internal/logging.js";import type { InputResolution, UnstampedMessageStreamEvent } from "#protocol/message.js";import type { FetchFileResult } from "#shared/channel-definition.js";+import { actionLabel, visibleActions } from "#shared/action-label.js";import type { InputRequest } from "#shared/input.js";import {type InputResponse,@@ -447,10 +448,11 @@ function defaultEvents<TAdapters extends ChatSdkAdapters>(await safeStartTyping(channel.thread, truncate(buffered));return;}- const labels = event.actions.map((action) =>- action.kind === "tool-call" ? action.toolName : action.kind,+ const labels = visibleActions(event.actions).map((action) =>+ actionLabel(action, event.presentation),);- await safeStartTyping(channel.thread, truncate(`Running ${labels.join(", ")}...`));+ if (labels.length === 0) return;+ await safeStartTyping(channel.thread, truncate(`${labels.join(", ")}...`));},async "message.appended"(event, channel, _ctx) {if (!channel.thread || !canStream(channel)) return;diff --git a/packages/eve/src/public/channels/linear/defaults.ts b/packages/eve/src/public/channels/linear/defaults.tsindex 10e514cb6..04ee1db0e 100644--- a/packages/eve/src/public/channels/linear/defaults.ts+++ b/packages/eve/src/public/channels/linear/defaults.ts@@ -10,6 +10,7 @@ import {} from "#public/channels/linear/hitl.js";import type { LinearAgentSessionEvent, LinearUser } from "#public/channels/linear/inbound.js";import type { SessionContext } from "#public/definitions/callback-context.js";+import { actionLabel, visibleActions } from "#shared/action-label.js";import type { InputRequest } from "#shared/input.js";import type {LinearChannelEvents,@@ -110,14 +111,15 @@ export function createDefaultEvents(options: LinearDefaultEventOptions = {}): Lireturn;}- if (event.actions.length === 0) return;- if (event.actions.length > 1) {+ const actions = visibleActions(event.actions);+ if (actions.length === 0) return;+ if (actions.length > 1) {await postActivity(channel,options,{action: "Running",- parameter: event.actions.map(actionLabel).join(", "),+ parameter: actions.map((action) => actionLabel(action, event.presentation)).join(", "),type: "action",},{@@ -127,12 +129,12 @@ export function createDefaultEvents(options: LinearDefaultEventOptions = {}): Lireturn;}- for (const action of event.actions) {+ for (const action of actions) {await postActivity(channel,options,{- action: actionLabel(action),+ action: actionLabel(action, event.presentation),parameter: actionParameter(action),type: "action",},@@ -311,10 +313,6 @@ function firstNonEmptyLine(text: string): string | undefined {return undefined;}-function actionLabel(action: { readonly kind: string; readonly toolName?: string }): string {- return action.kind === "tool-call" && action.toolName ? action.toolName : action.kind;-}-function actionParameter(action: {readonly description?: string;readonly input?: unknown;diff --git a/packages/eve/src/public/channels/linear/linearChannel.test.ts b/packages/eve/src/public/channels/linear/linearChannel.test.tsindex 07afea3e6..6da767e81 100644--- a/packages/eve/src/public/channels/linear/linearChannel.test.ts+++ b/packages/eve/src/public/channels/linear/linearChannel.test.ts@@ -440,7 +440,7 @@ describe("linearChannel default event handlers", () => {variables: {input: {agentSessionId: "agent_session_1",- content: { action: "search", parameter: "{}", type: "action" },+ content: { action: "Search", parameter: "{}", type: "action" },ephemeral: true,},},diff --git a/packages/eve/src/public/channels/slack/action-status.ts b/packages/eve/src/public/channels/slack/action-status.tsindex 9dcd4d5d6..71ec76ab6 100644--- a/packages/eve/src/public/channels/slack/action-status.ts+++ b/packages/eve/src/public/channels/slack/action-status.ts@@ -1,22 +1,11 @@/**- * Typing-indicator text for the turn's calls. It uses each call's start label,- * the same text its task card row shows, so both surfaces name a call the same- * way: `Run pnpm test`, `researcher: Find the March incidents`.+ * Typing-indicator text for the turn's calls. A call reads as its+ * `actionLabel`, the same text its task card row shows, so both surfaces name a+ * call the same way: `Run pnpm test`, `researcher: Find the March incidents`.*/-import type { ActionPresentationByCallId } from "#protocol/message.js";-import { actionRequestName } from "#shared/action-request-name.js";-import type { RuntimeActionRequest } from "#shared/action-types.js";-import { displayName, displayTitle } from "#shared/display-name.js";+import { displayName } from "#shared/display-name.js";import { AGENT_TOOL_NAME } from "#tools/framework/agent-contract.js";-/** The call's start label, or its tool or agent display title. */-export function actionLabel(- action: RuntimeActionRequest,- presentation: ActionPresentationByCallId | undefined,-): string {- return presentation?.[action.callId]?.label ?? displayTitle(actionRequestName(action));-}-/** The first call's label, plus `+N more` when the model made several calls in one step. */export function withMoreCalls(label: string, count: number): string {return count <= 1 ? label : `${label} +${String(count - 1)} more`;diff --git a/packages/eve/src/public/channels/slack/defaults.ts b/packages/eve/src/public/channels/slack/defaults.tsindex 671f47711..594a9ad6d 100644--- a/packages/eve/src/public/channels/slack/defaults.ts+++ b/packages/eve/src/public/channels/slack/defaults.ts@@ -3,12 +3,11 @@ import type { SessionAuthContext } from "#channel/types.js";import { createLogger, extractErrorId, formatErrorHint } from "#internal/logging.js";import {- actionLabel,reviewingResults,waitingOnTasks,withMoreCalls,} from "#public/channels/slack/action-status.js";-import { isTaskControlTool } from "#protocol/task-tools.js";+import { actionLabel, visibleActions } from "#shared/action-label.js";import { buildSlackAuthContext, slackUserIdForPrincipal } from "#public/channels/slack/auth.js";import {buildAuthCompletedText,@@ -304,16 +303,11 @@ export const defaultEvents: SlackChannelInternalEvents = {},// Calls in one step stream in one at a time, so the step keeps its first- // label, or the model's narration, and counts the rest. Calls a tool makes- // on the model's behalf, such as a connection tool, belong to their parent.+ // label, or the model's narration, and counts the rest.async "actions.requested"(event, channel, _ctx) {const narration = channel.state.pendingToolCallMessage;channel.state.pendingToolCallMessage = null;- const actions = event.actions.filter(- (action) =>- action.kind !== "tool-call" ||- (!isTaskControlTool(action.toolName) && action.parentCallId === undefined),- );+ const actions = visibleActions(event.actions);if (!narration && actions.length === 0) return;const stream = stepStream(channel.state, event.turnId, event.stepIndex);const calls =diff --git a/packages/eve/src/public/channels/slack/slackChannel.test.ts b/packages/eve/src/public/channels/slack/slackChannel.test.tsindex e2f7d4879..7cfa44ef4 100644--- a/packages/eve/src/public/channels/slack/slackChannel.test.ts+++ b/packages/eve/src/public/channels/slack/slackChannel.test.ts@@ -958,7 +958,7 @@ describe("slackChannel() default event handlers", () => {expect(ctx.state.pendingToolCallMessage).toBeNull();});- it("shows a step's first call label, or its display title, and counts the step's other calls, without the model's own task calls or calls its tools make", async () => {+ it("shows a step's first call label, or its display title, and counts the step's other calls, without the model's own task calls", async () => {const adapter = withState(getAdapter(slackChannel({ credentials: { botToken: "xoxb-test" } })),THREAD_STATE,@@ -978,31 +978,17 @@ describe("slackChannel() default event handlers", () => {turnId: "t1",});- await callEvent(adapter, requested(["task_wait"], 0), ctx);+ await callEvent(adapter, requested(["eve__task_wait"], 0), ctx);expect(fetchMock).not.toHaveBeenCalled();- await callEvent(adapter, requested(["task_cancel", "search", "ops__deploy_preview"], 0), ctx);- // Streamed calls arrive one at a time; the step keeps its first label.- await callEvent(adapter, requested(["lookup"], 0), ctx);await callEvent(adapter,- makeEvent("actions.requested", {- actions: [- {- callId: "call_lookup:1",- input: {},- kind: "tool-call",- parentCallId: "call_lookup",- toolName: "linear__search_issues",- },- ],- sequence: 0,- stepIndex: 0,- turnId: "t1",- }),+ requested(["eve__task_cancel", "search", "ops__deploy_preview"], 0),ctx,);- await callEvent(adapter, requested(["task_cancel", "ops__deploy_preview"], 1), ctx);+ // Streamed calls arrive one at a time; the step keeps its first label.+ await callEvent(adapter, requested(["lookup"], 0), ctx);+ await callEvent(adapter, requested(["eve__task_cancel", "ops__deploy_preview"], 1), ctx);expect(slackStatuses(fetchMock)).toEqual(["Search checkout incidents +1 more","Search checkout incidents +2 more",diff --git a/packages/eve/src/public/channels/slack/task-card.ts b/packages/eve/src/public/channels/slack/task-card.tsindex a1be16d0d..3a7652323 100644--- a/packages/eve/src/public/channels/slack/task-card.ts+++ b/packages/eve/src/public/channels/slack/task-card.ts@@ -56,7 +56,7 @@ interface SlackTaskObject {* eve's default task card: a `task_card` block for one task, or a `plan` block* for several, and no card for a turn that started none. A settled task shows* one line about how it ended and how long it took. A task the model cancelled- * with `task_cancel` shows as done, since it no longer needed the work; a+ * with `eve__task_cancel` shows as done, since it no longer needed the work; a* failed task, or one stopped any other way, shows as an error.*/export function renderDefaultSlackTaskCard(view: TaskCardView): SlackTaskCard | null {@@ -138,7 +138,7 @@ function shownDuration(ms: number): string | undefined {return Number.isFinite(ms) && ms >= MIN_SHOWN_DURATION_MS ? formatDuration(ms) : undefined;}-/** A task stopped before it finished for any reason but the model's own `task_cancel`. */+/** A task stopped before it finished for any reason but the model's own `eve__task_cancel`. */function isInterrupted(task: TaskCardTask): boolean {return task.status === "cancelled" && task.cancelReason !== "task_cancel";}diff --git a/packages/eve/src/public/definitions/connections/mcp.ts b/packages/eve/src/public/definitions/connections/mcp.tsindex f70ce5e3e..f54d5ea38 100644--- a/packages/eve/src/public/definitions/connections/mcp.ts+++ b/packages/eve/src/public/definitions/connections/mcp.ts@@ -48,8 +48,8 @@ export interface McpClientConnectionDefinition {/*** Human-readable summary of the connection and its tools.*- * The model sees it in the connection listing eve announces, so it- * can choose which connection to search with `connection_search`.+ * The model sees it in the catalog listing eve announces, so it can+ * choose which connection to search with `eve__search`.*/readonly description: string;/**@@ -107,8 +107,8 @@ export interface McpClientConnectionDefinition {*/toolCall?: ConnectionToolCallDefinition;/**- * Client-side tool filter. When set, `connection_search` returns and- * `connection_execute` calls only tools whose names pass the filter.+ * Client-side tool filter. When set, `eve__search` returns and `eve__tool`+ * calls only tools whose names pass the filter.** Specify exactly one of `allow` or `block`.*/diff --git a/packages/eve/src/public/definitions/connections/openapi.ts b/packages/eve/src/public/definitions/connections/openapi.tsindex ffa61fbb5..faa416dc3 100644--- a/packages/eve/src/public/definitions/connections/openapi.ts+++ b/packages/eve/src/public/definitions/connections/openapi.ts@@ -26,8 +26,8 @@ export type OpenAPISpecSource = string | Record<string, unknown>;* `"vercel"`.** Each operation in the document becomes a connection tool the model can- * find with `connection_search` and call with `connection_execute`; events- * report the call as `vercel__getProjects`. The tool name is the operation's+ * find with `eve__search` and call with `eve__tool` as `vercel__getProjects`, the+ * name events report the call under. The tool name is the operation's* `operationId`; operations without one get a deterministic synthesized* name (`<method>_<sanitized-path>`).*@@ -55,8 +55,8 @@ export interface OpenAPIConnectionDefinition {/*** Human-readable summary of the connection and its operations.*- * The model sees it in the connection listing eve announces, so it- * can choose which connection to search with `connection_search`.+ * The model sees it in the catalog listing eve announces, so it can+ * choose which connection to search.*/readonly description: string;/**@@ -109,9 +109,8 @@ export interface OpenAPIConnectionDefinition {*/toolCall?: ConnectionToolCallDefinition;/**- * Operation filter keyed on `operationId`. When set,- * `connection_search` returns and `connection_execute` calls only- * operations whose id passes the filter.+ * Operation filter keyed on `operationId`. When set, the model can find+ * and call only operations whose id passes the filter.** Specify exactly one of `allow` or `block`. Mirrors `tools` on MCP* connections, but names operations rather than tools.diff --git a/packages/eve/src/public/definitions/remote-agent.ts b/packages/eve/src/public/definitions/remote-agent.tsindex 82279b930..369015fd8 100644--- a/packages/eve/src/public/definitions/remote-agent.ts+++ b/packages/eve/src/public/definitions/remote-agent.ts@@ -1,6 +1,7 @@import type { HeadersValue } from "#client/types.js";import type { OutboundAuthFn } from "#public/agents/auth.js";import { EVE_SESSION_ROUTE_PATH } from "#protocol/routes.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";/*** Base URL of a remote eve deployment, either a static string or a function@@ -48,10 +49,12 @@ export interface RemoteAgentDefinition {*/readonly path: string;/**- * Whether eve exposes this remote agent to the parent model as a tool.- * Defaults to `true`; `false` keeps it callable from workflow tools.+ * How eve exposes this remote agent to the parent model: `true` lists it as+ * a tool, `"deferred"` makes it a catalog entry the model finds with+ * `eve__search` and calls with `eve__tool`, and `false` keeps it callable only from+ * workflow tools. Defaults to `true`.*/- readonly tool?: boolean;+ readonly tool?: AgentToolExposure;/*** Base URL of the remote eve deployment to call. Accepts a static string* (baked at compile time) or a function resolved at runtime — use thediff --git a/packages/eve/src/public/definitions/workspace-agent.ts b/packages/eve/src/public/definitions/workspace-agent.tsindex 9ef79611b..458667b5a 100644--- a/packages/eve/src/public/definitions/workspace-agent.ts+++ b/packages/eve/src/public/definitions/workspace-agent.ts@@ -6,6 +6,7 @@ import {type RemoteAgentUrl,} from "#public/definitions/remote-agent.js";import { normalizePublicRoutePrefix } from "#shared/public-route-prefix.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";const WORKSPACE_AGENT_NAME = Symbol.for("eve.workspace-agent.name");@@ -28,10 +29,12 @@ export interface WorkspaceAgentDefinition {/** Name of the peer workspace member, such as `research`. */readonly name: string;/**- * Whether eve exposes this workspace peer to the parent model as a tool.- * Defaults to `true`; `false` keeps it callable from workflow tools.+ * How eve exposes this workspace peer to the parent model: `true` lists it+ * as a tool, `"deferred"` makes it a catalog entry the model finds with+ * `eve__search` and calls with `eve__tool`, and `false` keeps it callable only from+ * workflow tools. Defaults to `true`.*/- readonly tool?: boolean;+ readonly tool?: AgentToolExposure;/** Overrides environment-aware workspace routing and service authentication. */readonly transport?: WorkspaceAgentTransport;}diff --git a/packages/eve/src/public/tools/load-skill.ts b/packages/eve/src/public/tools/load-skill.tsdeleted file mode 100644index 05caf970b..000000000--- a/packages/eve/src/public/tools/load-skill.ts+++ /dev/null@@ -1,5 +0,0 @@-import loadSkillDefinition from "#tools/provided/load-skill.js";--export { loadSkill } from "#tools/provided/load-skill.js";--export default loadSkillDefinition;diff --git a/packages/eve/src/runtime/agent/bootstrap.ts b/packages/eve/src/runtime/agent/bootstrap.tsindex 47ebe2167..dfa956548 100644--- a/packages/eve/src/runtime/agent/bootstrap.ts+++ b/packages/eve/src/runtime/agent/bootstrap.ts@@ -39,7 +39,8 @@ export type RuntimeDynamicModelReference = Readonly<* Minimal runtime-owned agent shape prepared for one harness turn.*/interface RuntimeTurnAgentBase {- readonly availableSkills?: readonly AvailableSkillDescription[];+ /** The skills the system prompt lists: every authored skill that isn't deferred. */+ readonly listedSkills?: readonly AvailableSkillDescription[];readonly id: string;readonly instructions: readonly string[];readonly initialMessages?: readonly HarnessModelMessage[];@@ -97,10 +98,9 @@ export function createResolvedRuntimeTurnAgent(input: {throw new Error("Expected a path-derived agent id while resolving agent resources.");}const base: RuntimeTurnAgentBase = {- availableSkills: agent.skills.map((skill) => ({- description: skill.description,- name: skill.name,- })),+ listedSkills: agent.skills+ .filter((skill) => skill.deferred !== true)+ .map((skill) => ({ description: skill.description, name: skill.name })),id,initialMessages: agent.instructions.filter((entry) => entry.role === "user" && entry.content.trim().length > 0)diff --git a/packages/eve/src/runtime/agent/mock-model-adapter.test.ts b/packages/eve/src/runtime/agent/mock-model-adapter.test.tsindex 6a5dcf394..6bb89dfce 100644--- a/packages/eve/src/runtime/agent/mock-model-adapter.test.ts+++ b/packages/eve/src/runtime/agent/mock-model-adapter.test.ts@@ -73,33 +73,6 @@ describe("createMockAuthoredRuntimeModel", () => {]);});- it("activates a matching skill when the available skill line includes a skill path", async () => {- const result = await generateWithPrompt([- {- content: [- "Available skills\n",- "Listed skills are available in this run.\n",- "- weather-skill: Use the weather tool before answering forecast or temperature questions. (path: /home/agent/.agents/skills/weather-skill/SKILL.md)",- ].join(""),- role: "system",- },- {- content: "What is the weather in Brooklyn?",- role: "user",- },- ]);-- expect(result.finishReason).toEqual({ raw: undefined, unified: "tool-calls" });- expect(result.content).toEqual([- {- input: JSON.stringify({ skill: "weather-skill" }),- toolCallId: "call_load_skill",- toolName: "load_skill",- type: "tool-call",- },- ]);- });-it("does not treat the available skills menu as a prompt-layer label", async () => {const result = await generateWithPrompt([{@@ -125,154 +98,6 @@ describe("createMockAuthoredRuntimeModel", () => {]);});- it("discovers skills announced in later system history messages", async () => {- const result = await generateWithPrompt([- {- content: [- "Available skills\n",- "Listed skills are available in this run.\n",- "- release: Use for release checklist requests. (path: /home/agent/.agents/skills/release/SKILL.md)",- ].join(""),- role: "system",- },- {- content: [- "Available skills\n",- "Listed skills are available in this run.\n",- "- tenant-weather: Use tenant weather policy before answering forecast questions. (path: /home/agent/.agents/skills/tenant-weather/SKILL.md)",- ].join(""),- role: "system",- },- {- content: "What is the weather in Brooklyn?",- role: "user",- },- ]);-- expect(result.finishReason).toEqual({ raw: undefined, unified: "tool-calls" });- expect(result.content).toEqual([- {- input: JSON.stringify({ skill: "tenant-weather" }),- toolCallId: "call_load_skill",- toolName: "load_skill",- type: "tool-call",- },- ]);- });-- it("discovers skills advertised inside larger static instruction text", async () => {- const result = await generateWithPrompt([- {- content: [- "# Identity",- "",- "You are a helpful assistant.",- "",- "Available skills",- "Listed skills are available in this run.",- "- echo-marker: Use when the user asks for the echo marker. (path: /home/agent/.agents/skills/echo-marker/SKILL.md)",- "",- "Another section that must not be parsed as skills.",- ].join("\n"),- role: "system",- },- {- content: "Please use the echo marker skill and follow its instructions exactly.",- role: "user",- },- ]);-- expect(result.finishReason).toEqual({ raw: undefined, unified: "tool-calls" });- expect(result.content).toEqual([- {- input: JSON.stringify({ skill: "echo-marker" }),- toolCallId: "call_load_skill",- toolName: "load_skill",- type: "tool-call",- },- ]);- });-- it("does not reload a skill already loaded earlier in the session", async () => {- const result = await generateWithPrompt([- {- content: [- "Available skills",- "Listed skills are available in this run.",- "- echo-marker: Use when the user asks for the echo marker. (path: /home/agent/.agents/skills/echo-marker/SKILL.md)",- ].join("\n"),- role: "system",- },- {- content: "Please use the echo marker skill and follow its instructions exactly.",- role: "user",- },- {- content: [- {- input: JSON.stringify({ skill: "echo-marker" }),- toolCallId: "call_load_skill",- toolName: "load_skill",- type: "tool-call",- },- ],- role: "assistant",- },- {- content: [- {- output: {- type: "json",- value: "Reply with exactly the following text and nothing else:\nskill-echo-ok-V1",- },- toolCallId: "call_load_skill",- toolName: "load_skill",- type: "tool-result",- },- ],- role: "tool",- },- ]);-- expect(result.finishReason).toEqual({ raw: undefined, unified: "stop" });- expect(result.content).toEqual([- {- text: "skill-echo-ok-V1",- type: "text",- },- ]);- });-- it("never matches load_skill by explicit name in the user message", async () => {- const result = await generateWithPrompt(- [- {- content: 'Call the load_skill tool with skill "echo-marker".',- role: "user",- },- ],- [- {- inputSchema: {- properties: { skill: { type: "string" } },- required: ["skill"],- type: "object",- },- name: "load_skill",- type: "function",- },- ],- );-- expect(result.finishReason).toEqual({ raw: undefined, unified: "stop" });- expect(result.content).toEqual([- {- text: 'Bootstrap reply: Call the load_skill tool with skill "echo-marker".',- type: "text",- },- ]);- });-it("builds ask_question input from question text and option labels", async () => {const result = await generateWithPrompt([@@ -510,46 +335,6 @@ describe("createMockAuthoredRuntimeModel", () => {]);});- it("prefers loaded skill exact text over ambient instruction tokens", async () => {- const result = await generateWithPrompt([- {- content:- "When you reply to the next user message, include the exact token ambient-ok-M3K8 verbatim.",- role: "system",- },- {- content: [- {- output: {- type: "text",- value: [- "Skill (dynamic-tenant-policy)",- "Reply with exactly the following text and nothing else:",- "loaded-skill-ok-P4K9",- ].join("\n"),- },- toolCallId: "call_load_skill",- toolName: "load_skill",- type: "tool-result",- },- ],- role: "tool",- },- {- content: "Please use the dynamic tenant policy skill.",- role: "user",- },- ]);-- expect(result.finishReason).toEqual({ raw: undefined, unified: "stop" });- expect(result.content).toEqual([- {- text: "loaded-skill-ok-P4K9",- type: "text",- },- ]);- });-it("honors exact-token directives delivered as trailing user context", async () => {const result = await generateWithPrompt([{@@ -824,7 +609,7 @@ describe("createMockAuthoredRuntimeModel", () => {[{ content: "Summarize this", role: "user" }],[{- name: "final_output",+ name: "eve__reply",type: "function",description: "Deliver your final answer.",inputSchema: {@@ -844,7 +629,7 @@ describe("createMockAuthoredRuntimeModel", () => {{input: JSON.stringify({ title: "structured-output", count: 1 }),toolCallId: expect.any(String),- toolName: "final_output",+ toolName: "eve__reply",type: "tool-call",},]);diff --git a/packages/eve/src/runtime/agent/mock-model-adapter.ts b/packages/eve/src/runtime/agent/mock-model-adapter.tsindex 52a5eef61..734474cd3 100644--- a/packages/eve/src/runtime/agent/mock-model-adapter.ts+++ b/packages/eve/src/runtime/agent/mock-model-adapter.ts@@ -31,14 +31,15 @@ import {findRelevantSkill,getActivatedSkillIds,getAvailableSkills,+ getSkillLoads,} from "#runtime/agent/mock-model-skill-selection.js";import { createJsonSchemaSample } from "#runtime/agent/mock-structured-output.js";-import { FINAL_OUTPUT_TOOL_NAME } from "#harness/final-output.js";+import { REPLY_TOOL_NAME } from "#protocol/reply-tool.js";import { readTaskResults } from "#execution/tasks/render.js";-import { LOAD_SKILL_TOOL_NAME } from "#runtime/skills/fragment-context.js";+import { SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";const MOCK_RUNTIME_MODEL_PROVIDER = "eve-runtime-mock";-const LOAD_SKILL_TOOL_CALL_ID = "call_load_skill";+const SKILL_LOAD_CALL_ID = "call_execute_skill";const MOCK_AUTHORED_MODELS_ENV = "EVE_MOCK_AUTHORED_MODELS";type BootstrapGenerateOptions = Parameters<MockLanguageModelV3["doGenerate"]>[0];@@ -119,7 +120,7 @@ function createMockModelResult(}}- // The model is ready to answer. With the framework `final_output` tool+ // The model is ready to answer. With the framework `eve__reply` tool// offered, deliver the answer by calling it with a schema-derived sample;// otherwise reply in prose.const finalOutput = createFinalOutputResult(options, modelId);@@ -141,7 +142,7 @@ function createMockModelResult(}/**- * When the framework `final_output` tool is offered, returns a tool-call result+ * When the framework `eve__reply` tool is offered, returns a tool-call result* carrying a schema-derived sample — the structured-output analogue of a final* text reply. Returns `null` when the tool is absent.*/@@ -149,7 +150,7 @@ function createFinalOutputResult(options: BootstrapGenerateOptions,modelId: string,): BootstrapGenerateResult | null {- const tool = getAvailableTools(options).find((entry) => entry.name === FINAL_OUTPUT_TOOL_NAME);+ const tool = getAvailableTools(options).find((entry) => entry.name === REPLY_TOOL_NAME);if (tool === undefined) {return null;@@ -162,8 +163,8 @@ function createFinalOutputResult(inputTokens: estimateTokenCount(getPromptText(options.prompt)),modelId,outputTokens: estimateTokenCount(JSON.stringify(sample)),- toolCallId: createToolCallId(FINAL_OUTPUT_TOOL_NAME),- toolName: FINAL_OUTPUT_TOOL_NAME,+ toolCallId: createToolCallId(REPLY_TOOL_NAME),+ toolName: REPLY_TOOL_NAME,});}@@ -199,13 +200,13 @@ function createSkillLoadResult(return createToolCallGenerateResult({input: {- skill: skill.name,+ name: skill.name,},inputTokens: estimateTokenCount(getPromptText(prompt)),modelId,outputTokens: estimateTokenCount(skill.name),- toolCallId: LOAD_SKILL_TOOL_CALL_ID,- toolName: LOAD_SKILL_TOOL_NAME,+ toolCallId: SKILL_LOAD_CALL_ID,+ toolName: SKILL_TOOL_NAME,});}@@ -492,6 +493,7 @@ function getAvailableTools(options: BootstrapGenerateOptions): AvailableBootstra}function getLastAuthoredToolResult(prompt: BootstrapPrompt): BootstrapToolResult | null {+ const skillLoads = getSkillLoads(prompt);for (const message of [...prompt].reverse()) {if (message.role === "user") {const text = getPromptContentText(message.content).trim();@@ -516,7 +518,7 @@ function getLastAuthoredToolResult(prompt: BootstrapPrompt): BootstrapToolResultcontinue;}- if (part.toolName === LOAD_SKILL_TOOL_NAME) {+ if (skillLoads.has(part.toolCallId)) {continue;}@@ -640,14 +642,9 @@ function findRelevantTool(message: string,): AvailableBootstrapTool | null {const normalizedMessage = normalizeText(message);- // `load_skill` is reachable only through skill-relevance selection- // (createSkillLoadResult); matching it by name here would re-call it on- // every step, because its results are invisible to the tool-result check.+ // Skills load through `eve__skill` only by skill-relevance selection.const explicitTool = tools.find(- (tool) =>- tool.name !== "agent" &&- tool.name !== LOAD_SKILL_TOOL_NAME &&- normalizedMessage.includes(normalizeText(tool.name)),+ (tool) => tool.name !== "agent" && normalizedMessage.includes(normalizeText(tool.name)),);if (explicitTool !== undefined) {return explicitTool;diff --git a/packages/eve/src/runtime/agent/mock-model-fixtures.ts b/packages/eve/src/runtime/agent/mock-model-fixtures.tsindex a85d1c677..59d46a182 100644--- a/packages/eve/src/runtime/agent/mock-model-fixtures.ts+++ b/packages/eve/src/runtime/agent/mock-model-fixtures.ts@@ -5,7 +5,7 @@ import {} from "#runtime/agent/bootstrap-model-utils.js";import { TASK_ID_INPUT } from "#execution/tasks/task-id-input.js";import { createJsonSchemaSample } from "#runtime/agent/mock-structured-output.js";-import { LOAD_SKILL_TOOL_NAME } from "#runtime/skills/fragment-context.js";+import { getSkillLoads } from "#runtime/agent/mock-model-skill-selection.js";export interface AvailableBootstrapTool {readonly description?: string;@@ -161,6 +161,7 @@ function getTrailingUserText(prompt: BootstrapPrompt): string {}function getLoadedSkillResultTexts(prompt: BootstrapPrompt): string[] {+ const skillLoads = getSkillLoads(prompt);return prompt.flatMap((message) => {if (message.role !== "tool" && message.role !== "assistant") {return [];@@ -173,7 +174,7 @@ function getLoadedSkillResultTexts(prompt: BootstrapPrompt): string[] {return [];}- if (part.toolName !== LOAD_SKILL_TOOL_NAME || part.output.type === "execution-denied") {+ if (!skillLoads.has(part.toolCallId) || part.output.type === "execution-denied") {return [];}diff --git a/packages/eve/src/runtime/agent/mock-model-skill-selection.ts b/packages/eve/src/runtime/agent/mock-model-skill-selection.tsindex 878464a03..69cc62b85 100644--- a/packages/eve/src/runtime/agent/mock-model-skill-selection.ts+++ b/packages/eve/src/runtime/agent/mock-model-skill-selection.ts@@ -1,3 +1,4 @@+import { SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";import type { BootstrapPrompt } from "#runtime/agent/bootstrap-model-utils.js";import { getPromptContentText } from "#runtime/agent/bootstrap-model-utils.js";@@ -10,14 +11,15 @@ export function getAvailableSkills(prompt: BootstrapPrompt): AvailableBootstrapSconst skillsById = new Map<string, AvailableBootstrapSkill>();for (const message of prompt) {- if (message.role !== "system") {+ if (message.role !== "system" && message.role !== "user") {continue;}// The "Available skills" section may be a standalone announcement// (dynamic skills) or embedded inside the agent's static instructions// (authored skills); parse bullet lines from the section header to the- // first blank line either way.+ // first blank line either way. Deferred skills aren't listed, so only+ // these are offered.const lines = getPromptContentText(message.content).split("\n");const headerIndex = lines.findIndex((line) => line.trim() === "Available skills");@@ -80,14 +82,15 @@ export function getActivatedSkillIds(prompt: BootstrapPrompt): string[] {});// Static skill adverts stay in the prompt for the whole session (prompt- // caching), so activation must also be derived from `load_skill` calls- // already present in the history — otherwise the mock would re-load the- // same skill on every step.- return [...fromSystemLabels, ...getLoadedSkillIdsFromHistory(prompt)];+ // caching), so activation must also be derived from skill loads already+ // present in the history — otherwise the mock would re-load the same skill+ // on every step.+ return [...fromSystemLabels, ...getSkillLoads(prompt).values()];}-function getLoadedSkillIdsFromHistory(prompt: BootstrapPrompt): string[] {- const ids: string[] = [];+/** Skill loads in the history: each `eve__skill({ name })` call's id and skill. */+export function getSkillLoads(prompt: BootstrapPrompt): ReadonlyMap<string, string> {+ const loads = new Map<string, string>();for (const message of prompt) {if (message.role !== "assistant" || typeof message.content === "string") {@@ -95,18 +98,22 @@ function getLoadedSkillIdsFromHistory(prompt: BootstrapPrompt): string[] {}for (const part of message.content) {- if (typeof part === "string" || part.type !== "tool-call" || part.toolName !== "load_skill") {+ if (+ typeof part === "string" ||+ part.type !== "tool-call" ||+ part.toolName !== SKILL_TOOL_NAME+ ) {continue;}const skill = readSkillFromToolInput(part.input);if (skill !== undefined) {- ids.push(skill);+ loads.set(part.toolCallId, skill);}}}- return ids;+ return loads;}function readSkillFromToolInput(input: unknown): string | undefined {@@ -115,10 +122,10 @@ function readSkillFromToolInput(input: unknown): string | undefined {if (typeof parsed === "object" &&parsed !== null &&- "skill" in parsed &&- typeof parsed.skill === "string"+ "name" in parsed &&+ typeof parsed.name === "string") {- return parsed.skill;+ return parsed.name;}return undefined;diff --git a/packages/eve/src/runtime/agent/mock-structured-output.ts b/packages/eve/src/runtime/agent/mock-structured-output.tsindex 93adcd6be..e80305b5b 100644--- a/packages/eve/src/runtime/agent/mock-structured-output.ts+++ b/packages/eve/src/runtime/agent/mock-structured-output.ts@@ -1,6 +1,6 @@/*** Builds a deterministic sample value satisfying a JSON Schema, used by the- * mock model to populate a `final_output` tool call in tests.+ * mock model to populate an `eve__reply` tool call in tests.*/export function createJsonSchemaSample(schema: unknown): unknown {if (!isRecord(schema)) {diff --git a/packages/eve/src/runtime/connections/registry.ts b/packages/eve/src/runtime/connections/registry.tsindex bcb67c436..8d1f0ffba 100644--- a/packages/eve/src/runtime/connections/registry.ts+++ b/packages/eve/src/runtime/connections/registry.ts@@ -1,4 +1,5 @@import type { Approval } from "#approval/definition.js";+import { assertNotConnectionOwned } from "#connections/ownership.js";import type { ResolvedConnectionDefinition } from "#runtime/types.js";import { McpConnectionClient } from "#runtime/connections/mcp-client.js";import { OpenApiConnectionClient } from "#runtime/connections/openapi-client.js";@@ -74,6 +75,7 @@ export class ConnectionRegistryImpl implements ConnectionRegistry {connections.set(definition.connectionName, definition);}}+ for (const name of dynamicNames.keys()) assertNoNestedConnection(name, connections.keys());return [...connections.values()];}@@ -132,3 +134,19 @@ export class ConnectionRegistryImpl implements ConnectionRegistry {this.#clients.clear();}}++/** A dynamic connection may neither own nor be owned by another connection's name. */+function assertNoNestedConnection(dynamicName: string, connectionNames: Iterable<string>): void {+ for (const other of connectionNames) {+ if (other === dynamicName) continue;+ // Only the shorter of two names can own the other.+ const [name, owner] =+ other.length < dynamicName.length ? [dynamicName, other] : [other, dynamicName];+ assertNotConnectionOwned({+ connectionNames: [owner],+ name,+ remedy: `Rename the dynamic connection "${dynamicName}".`,+ subject: "Connection",+ });+ }+}diff --git a/packages/eve/src/runtime/connections/tool-result.ts b/packages/eve/src/runtime/connections/tool-result.tsindex 688d790b9..ae61714ff 100644--- a/packages/eve/src/runtime/connections/tool-result.ts+++ b/packages/eve/src/runtime/connections/tool-result.ts@@ -1,6 +1,6 @@/**- * The value a connection tool call returns: what `connection_execute` hands- * the model today and what a code mode program receives later.+ * The value a connection tool call returns: what the model receives for the+ * call and what a code mode program receives later.** - MCP results return `structuredContent` when present, otherwise their text,* parsed as JSON when the tool declares no output schema. Results with image,diff --git a/packages/eve/src/runtime/resolve-agent-graph.ts b/packages/eve/src/runtime/resolve-agent-graph.tsindex 3fe8e4b63..70caeabd0 100644--- a/packages/eve/src/runtime/resolve-agent-graph.ts+++ b/packages/eve/src/runtime/resolve-agent-graph.ts@@ -18,7 +18,6 @@ import { resolveAgent } from "#runtime/resolve-agent.js";import { resolveDynamicSubagentDefinition } from "#runtime/resolve-dynamic-subagent.js";import { loadResolvedModuleExport } from "#runtime/resolve-helpers.js";import { createRuntimeSandboxRegistry } from "#runtime/sandbox/registry.js";-import { LOAD_SKILL_TOOL_NAME } from "#runtime/skills/fragment-context.js";import { createRuntimeSubagentRegistry } from "#runtime/subagents/registry.js";import { createRuntimeToolRegistry } from "#runtime/tools/registry.js";import { createWorkspacePromptSection } from "#runtime/workspace/spec.js";@@ -28,6 +27,7 @@ import type {ResolvedRuntimeRemoteAgentNode,ResolvedRuntimeSubagentNode,} from "#runtime/types.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";/*** Input for resolving the compiled authored manifest and flattened module graph@@ -135,7 +135,10 @@ async function resolveRuntimeAgentNode(moduleMap: input.moduleMap,nodeId: input.nodeId,});- const toolRegistry = await createRuntimeToolRegistry({ tools: agent.tools }, { nodeId });+ const toolRegistry = await createRuntimeToolRegistry(+ { tools: agent.tools },+ { deferSelfAgent: agent.config?.tool === "deferred", nodeId },+ );const sandboxRegistry = createRuntimeSandboxRegistry({sandbox: agent.sandbox,@@ -150,10 +153,7 @@ async function resolveRuntimeAgentNode(.replaceAll("/", "-"),];}),- reservedToolNames: [- LOAD_SKILL_TOOL_NAME,- ...toolRegistry.preparedTools.map((tool) => tool.name),- ],+ reservedToolNames: toolRegistry.preparedTools.map((tool) => tool.name),subagents: await resolveRuntimeSubagents({childNodeIdsByParentNodeId: input.childNodeIdsByParentNodeId,manifest: input.manifest,@@ -241,7 +241,7 @@ async function resolveRuntimeSubagent(input: {readonly subagentNodesById: ReadonlyMap<string, CompiledSubagentNode>;}): Promise<ResolvedRuntimeSubagentNode> {const variant:- | { readonly description: string; readonly dynamic?: never; readonly tool?: boolean }+ | { readonly description: string; readonly dynamic?: never; readonly tool?: AgentToolExposure }| {readonly description?: never;readonly dynamic: ResolvedDynamicSubagentDefinition;@@ -307,7 +307,7 @@ async function resolveRuntimeRemoteAgent(input: {path: string;sourceId: string;sourceKind: "module";- tool?: boolean;+ tool?: AgentToolExposure;url: string;} = {description: input.sourceRef.description,diff --git a/packages/eve/src/runtime/resolve-agent.ts b/packages/eve/src/runtime/resolve-agent.tsindex c1697f84e..12c60e535 100644--- a/packages/eve/src/runtime/resolve-agent.ts+++ b/packages/eve/src/runtime/resolve-agent.ts@@ -23,6 +23,7 @@ import type {ResolvedSkillDefinition,ResolvedInstructionsDefinition,} from "#runtime/types.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";/*** Input for resolving one compiled authored agent into a runtime-owned model.@@ -185,7 +186,7 @@ function createResolvedAgentConfig(name: string;reasoning?: NonNullable<ResolvedAgent["config"]>["reasoning"];source?: NonNullable<ResolvedAgent["config"]>["source"];- tool?: boolean;+ tool?: AgentToolExposure;limits?: NonNullable<ResolvedAgent["config"]>["limits"];} = {name: manifest.config.name,diff --git a/packages/eve/src/runtime/resolve-tool.ts b/packages/eve/src/runtime/resolve-tool.tsindex a3e35572d..075229888 100644--- a/packages/eve/src/runtime/resolve-tool.ts+++ b/packages/eve/src/runtime/resolve-tool.ts@@ -30,6 +30,7 @@ export async function resolveToolDefinition(const resolved: ResolvedToolDefinition = {availableInSubagents: definition.availableInSubagents,behavior: definition.behavior,+ deferred: definition.deferred,description: definition.description,inputSchema: toInputSchema(definition.inputSchema),logicalPath: definition.logicalPath,@@ -84,6 +85,7 @@ export async function resolveToolDefinition(const resolved: ResolvedToolDefinition = {availableInSubagents: definition.availableInSubagents,behavior: definition.behavior,+ deferred: definition.deferred,description: definition.description,execute,executeInput,diff --git a/packages/eve/src/runtime/skills/fragment-context.ts b/packages/eve/src/runtime/skills/fragment-context.tsdeleted file mode 100644index 640da094a..000000000--- a/packages/eve/src/runtime/skills/fragment-context.ts+++ /dev/null@@ -1,4 +0,0 @@-/**- * Framework-owned tool name used to load one available compiled skill.- */-export const LOAD_SKILL_TOOL_NAME = "load_skill";diff --git a/packages/eve/src/runtime/subagents/dynamic-agent-config.ts b/packages/eve/src/runtime/subagents/dynamic-agent-config.tsindex 719462180..8ca35bea5 100644--- a/packages/eve/src/runtime/subagents/dynamic-agent-config.ts+++ b/packages/eve/src/runtime/subagents/dynamic-agent-config.ts@@ -7,6 +7,7 @@ import {isDynamicModelDefinition,type AgentLimitsDefinition,type AgentReasoningDefinition,+ type AgentToolExposure,} from "#shared/agent-definition.js";export interface DynamicSubagentAgentConfig {@@ -18,7 +19,7 @@ export interface DynamicSubagentAgentConfig {readonly limits?: AgentLimitsDefinition;readonly model: DynamicSubagentModelReference;readonly reasoning?: AgentReasoningDefinition;- readonly tool?: boolean;+ readonly tool?: AgentToolExposure;}export type DynamicSubagentModelReference = RuntimeModelReference;@@ -54,7 +55,7 @@ export async function normalizeDynamicSubagentAgentConfig(input: {limits?: AgentLimitsDefinition;model: DynamicSubagentModelReference;reasoning?: AgentReasoningDefinition;- tool?: boolean;+ tool?: AgentToolExposure;} = {description: definition.description,model: await normalizeDurableModelSelection({diff --git a/packages/eve/src/runtime/subagents/dynamic-remote-agent-config.ts b/packages/eve/src/runtime/subagents/dynamic-remote-agent-config.tsindex 05bd6e74c..7d652d2dd 100644--- a/packages/eve/src/runtime/subagents/dynamic-remote-agent-config.ts+++ b/packages/eve/src/runtime/subagents/dynamic-remote-agent-config.ts@@ -1,4 +1,5 @@import {+ expectAgentToolExposure,expectBoolean,expectFunction,expectObjectRecord,@@ -8,13 +9,14 @@ import {} from "#internal/authored-module.js";import type { OutboundAuthFn } from "#public/agents/auth.js";import { EVE_SESSION_ROUTE_PATH } from "#protocol/routes.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";export interface DynamicRemoteAgentConfig {readonly credentialsStepId?: string;readonly description: string;readonly forwardPrincipal?: boolean;readonly path: string;- readonly tool?: boolean;+ readonly tool?: AgentToolExposure;readonly url: string;}@@ -50,7 +52,7 @@ export async function normalizeDynamicRemoteAgentConfig(input: {description: string;forwardPrincipal?: boolean;path: string;- tool?: boolean;+ tool?: AgentToolExposure;url: string;} = {description: expectString(record.description, message),@@ -65,7 +67,7 @@ export async function normalizeDynamicRemoteAgentConfig(input: {config.forwardPrincipal = expectBoolean(record.forwardPrincipal, message);}if (record.tool !== undefined) {- config.tool = expectBoolean(record.tool, message);+ config.tool = expectAgentToolExposure(record.tool, message);}return config;diff --git a/packages/eve/src/runtime/subagents/registry.test.ts b/packages/eve/src/runtime/subagents/registry.test.tsindex c3fec23ce..296dc52bf 100644--- a/packages/eve/src/runtime/subagents/registry.test.ts+++ b/packages/eve/src/runtime/subagents/registry.test.ts@@ -3,14 +3,14 @@ import { describe, expect, it } from "vitest";import { createRuntimeSubagentRegistry } from "#runtime/subagents/registry.js";import type { ResolvedRuntimeSubagentNode } from "#runtime/types.js";-function subagent(tool?: boolean): ResolvedRuntimeSubagentNode {+function subagent(tool?: boolean, name = "researcher"): ResolvedRuntimeSubagentNode {return {description: "Research difficult questions.",kind: "subagent",- logicalPath: "subagents/researcher",- name: "researcher",- nodeId: "subagents/researcher",- sourceId: "subagents/researcher",+ logicalPath: `subagents/${name}`,+ name,+ nodeId: `subagents/${name}`,+ sourceId: `subagents/${name}`,sourceKind: "module",tool,};@@ -53,4 +53,18 @@ describe("createRuntimeSubagentRegistry", () => {}),).toThrow('Subagent "researcher" collides with another runtime-visible tool name.');});++ it.each([true, false])("rejects a subagent in eve's namespace with tool %s", (tool) => {+ expect(() =>+ createRuntimeSubagentRegistry({ subagents: [subagent(tool, "eve__task_wait")] }),+ ).toThrow(+ 'Subagent "subagents/eve__task_wait" uses the reserved name "eve__task_wait". Rename its path; eve reserves the "eve" namespace for its built-in tools.',+ );+ });++ it("lets a model-visible subagent take a built-in tool's former name", () => {+ const registry = createRuntimeSubagentRegistry({ subagents: [subagent(true, "task_wait")] });++ expect(registry.preparedTools.map((tool) => tool.name)).toEqual(["task_wait"]);+ });});diff --git a/packages/eve/src/runtime/subagents/registry.ts b/packages/eve/src/runtime/subagents/registry.tsindex 82f14a322..3b9dcfd76 100644--- a/packages/eve/src/runtime/subagents/registry.ts+++ b/packages/eve/src/runtime/subagents/registry.ts@@ -1,4 +1,5 @@import { RuntimeRegistry, RuntimeRegistryError } from "#internal/runtime-registry.js";+import { eveNamespaceReservation } from "#protocol/runtime-tools.js";import type { PreparedRuntimeDelegationTool } from "#runtime/sessions/turn.js";import type {ResolvedDynamicSubagentDefinition,@@ -7,6 +8,7 @@ import type {import type { JsonObject } from "#shared/json.js";import { serializeInputSchema } from "#tools/schema.js";import { SUBAGENT_TOOL_INPUT_SCHEMA } from "#tools/framework/agent-contract.js";+import type { AgentToolExposure } from "#shared/agent-definition.js";/*** One runtime-owned subagent tracked by the prepared registry.@@ -20,7 +22,7 @@ export interface ResolvedDynamicSubagentResolver extends ResolvedDynamicSubagentreadonly kind: "subagent";readonly name: string;readonly nodeId: string;- readonly tool?: boolean;+ readonly tool?: AgentToolExposure;}/**@@ -85,10 +87,18 @@ export function createRuntimeSubagentRegistry(input: {};registry.register(subagentDefinition.name, registeredSubagent, {location,- duplicateMessage: `Found multiple subagents named "${subagentDefinition.name}". Subagent names must be unique at runtime.`,+ duplicateMessage: duplicateSubagentMessage(subagentDefinition.name),});const modelVisible =subagentDefinition.tool !== false && !disabledToolNames.has(subagentDefinition.name);+ const reservation = eveNamespaceReservation(subagentDefinition.name);+ if (reservation !== undefined) {+ throw new RuntimeRegistryError(+ "subagent",+ `Subagent "${subagentDefinition.logicalPath}" uses the reserved name "${subagentDefinition.name}". Rename its path; ${reservation}.`,+ { ...location, entryName: subagentDefinition.name },+ );+ }if (modelVisible && reservedToolNames.has(subagentDefinition.name)) {throw new RuntimeRegistryError("subagent",@@ -115,6 +125,7 @@ export function createRuntimeSubagentRegistry(input: {}subagentsByNodeId.set(subagentDefinition.nodeId, registeredSubagent);}+ assertUniqueDynamicSubagentNames(dynamicResolvers, registry.asMap());return {dynamicNodeIds,@@ -151,6 +162,7 @@ export function createPreparedRuntimeSubagentTool(},},},+ deferred: definition.tool === "deferred" || undefined,description: definition.description,inputSchema,kind: definition.kind,@@ -160,3 +172,25 @@ export function createPreparedRuntimeSubagentTool(sourceId: definition.sourceId,};}++function duplicateSubagentMessage(name: string): string {+ return `Found multiple subagents named "${name}". Subagent names must be unique at runtime.`;+}++/** Dynamic subagents resolve later, so their names are checked here, against every subagent. */+function assertUniqueDynamicSubagentNames(+ dynamicResolvers: readonly ResolvedDynamicSubagentResolver[],+ subagentsByName: ReadonlyMap<string, unknown>,+): void {+ const names = new Set(subagentsByName.keys());+ for (const { logicalPath, name, sourceId } of dynamicResolvers) {+ if (names.has(name)) {+ throw new RuntimeRegistryError("subagent", duplicateSubagentMessage(name), {+ entryName: name,+ logicalPath,+ sourceId,+ });+ }+ names.add(name);+ }+}diff --git a/packages/eve/src/runtime/tools/registry.ts b/packages/eve/src/runtime/tools/registry.tsindex d959ab6aa..b5392b0ac 100644--- a/packages/eve/src/runtime/tools/registry.ts+++ b/packages/eve/src/runtime/tools/registry.ts@@ -34,6 +34,8 @@ export async function createRuntimeToolRegistry(readonly tools: readonly ResolvedToolDefinition[];},input: {+ /** Set by `tool: "deferred"` on the agent, which defers its built-in `agent` tool. */+ readonly deferSelfAgent?: boolean;readonly nodeId?: string;readonly reservedToolNames?: readonly string[];} = {},@@ -45,7 +47,7 @@ export async function createRuntimeToolRegistry();for (const toolDefinition of definitions.tools) {- const prepared = await createPreparedRuntimeTool(toolDefinition, input.nodeId);+ const prepared = await createPreparedRuntimeTool(toolDefinition, input);registry.register(toolDefinition.name,{ definition: toolDefinition, prepared },@@ -79,14 +81,15 @@ export function findRegisteredRuntimeTool(async function createPreparedRuntimeTool(definition: ResolvedToolDefinition,- nodeId: string | undefined,+ input: { readonly deferSelfAgent?: boolean; readonly nodeId?: string },): Promise<PreparedRuntimeAuthoredTool> {const isSelfAgent =definition.behavior?.handling?.kind === "dispatch" &&definition.behavior.handling.action === "self-agent";return {availableInSubagents: definition.availableInSubagents,- behavior: prepareToolBehavior(definition.behavior, nodeId),+ behavior: prepareToolBehavior(definition.behavior, input.nodeId),+ deferred: definition.deferred ?? ((isSelfAgent && input.deferSelfAgent) || undefined),description: definition.description,inputSchema: serializeInputSchema(definition.inputSchema),kind: "authored-tool",@@ -130,6 +133,5 @@ function prepareToolBehavior(return {availability: behavior.availability,handling,- presentation: behavior.presentation,};}diff --git a/packages/eve/src/runtime/types.ts b/packages/eve/src/runtime/types.tsindex 25ae1b2bc..78c70fcf0 100644--- a/packages/eve/src/runtime/types.ts+++ b/packages/eve/src/runtime/types.ts@@ -31,7 +31,7 @@ import type {MarkdownSourceRef,} from "#shared/source-ref.js";import type { NamedSkillDefinition } from "#shared/skill-definition.js";-import type { InternalAgentDefinition } from "#shared/agent-definition.js";+import type { InternalAgentDefinition, AgentToolExposure } from "#shared/agent-definition.js";import type { RuntimeDynamicModelReference } from "#runtime/agent/bootstrap.js";import type { InternalToolDefinitionWithExecuteFn } from "#tools/definition.js";import type { CompiledToolBehavior } from "#tools/behavior.js";@@ -276,7 +276,7 @@ export type ResolvedRuntimeSubagentNode = Readonly<Node & {kind: "subagent";name: string;- tool?: boolean;+ tool?: AgentToolExposure;} & (| {description: string;@@ -303,7 +303,7 @@ export type ResolvedRuntimeRemoteAgentNode = Readonly<kind: "remote";name: string;path: string;- tool?: boolean;+ tool?: AgentToolExposure;url: string;}>;diff --git a/packages/eve/src/setup/scaffold/create/web-template.ts b/packages/eve/src/setup/scaffold/create/web-template.tsindex 1460f3b03..66882c236 100644--- a/packages/eve/src/setup/scaffold/create/web-template.ts+++ b/packages/eve/src/setup/scaffold/create/web-template.ts@@ -5,7 +5,7 @@ export const WEB_APP_TEMPLATE_FILES = {"app/_components/agent-chat.tsx":'"use client";\n\nimport type { UserContent } from "ai";\nimport { useEveAgent } from "eve/react";\nimport { AlertCircleIcon, BrainIcon, PlusIcon, SquareIcon } from "lucide-react";\nimport { useState } from "react";\nimport {\n Conversation,\n ConversationContent,\n ConversationScrollButton,\n ConversationTopFade,\n} from "@/components/ai-elements/conversation";\nimport { Message, MessageContent } from "@/components/ai-elements/message";\nimport {\n PromptInput,\n PromptInputButton,\n type PromptInputMessage,\n PromptInputSubmit,\n PromptInputTextarea,\n usePromptInputAttachments,\n} from "@/components/ai-elements/prompt-input";\nimport { Shimmer } from "@/components/ai-elements/shimmer";\nimport { Button } from "@/components/ui/button";\nimport { cn } from "@/lib/utils";\nimport { AgentMessage } from "./agent-message";\nimport { WEB_CHAT_AGENT } from "@/app/eve-agent";\n\nconst DEFAULT_AGENT_NAME = "__EVE_INIT_APP_NAME__";\nconst AGENT_NAME = WEB_CHAT_AGENT ?? DEFAULT_AGENT_NAME;\n\nexport function AgentChat({\n sessionId,\n sessionless = false,\n}: {\n readonly sessionId?: string;\n readonly sessionless?: boolean;\n}) {\n const [cancellationError, setCancellationError] = useState<string>();\n const [hasInputText, setHasInputText] = useState(false);\n const agent = useEveAgent({\n agent: WEB_CHAT_AGENT,\n initialSession:\n sessionId === undefined\n ? undefined\n : {\n sessionId,\n streamIndex: 0,\n },\n resume: sessionId !== undefined,\n onSessionChange(session) {\n if (sessionId === undefined && session !== undefined) {\n // Next patches window.history to navigate, which would detach the active stream.\n History.prototype.replaceState.call(\n window.history,\n window.history.state,\n "",\n `/s/${encodeURIComponent(session.sessionId)}`,\n );\n }\n },\n });\n\n const isBusy = agent.status === "submitted" || agent.status === "streaming";\n const isResuming = agent.status === "resuming";\n const isEmpty = agent.data.messages.length === 0;\n const lastMessage = agent.data.messages.at(-1);\n const isPendingAssistantShell =\n lastMessage?.role === "assistant" &&\n lastMessage.parts.every((part) => part.type === "step-start");\n const showPendingThinking =\n isBusy &&\n (agent.status === "submitted" || lastMessage?.role !== "assistant" || isPendingAssistantShell);\n const turnFailure = isBusy || isResuming ? undefined : getLatestTurnFailure(agent.events);\n const errorMessage = cancellationError ?? agent.error?.message ?? turnFailure;\n const hasConversationContent = sessionless || !isEmpty || errorMessage !== undefined;\n const showConversationLayout = isResuming || hasConversationContent;\n const activeSessionId = sessionId ?? agent.session?.sessionId;\n // Answers are accepted while work runs; the server\'s settlement closes each request.\n const canRespond = (requestId: string) =>\n !isResuming && agent.data.inputs[requestId]?.status === "open";\n // A tool call can ask several questions at once, and its message part holds only the latest.\n const questionsFor = (callId: string) =>\n Object.values(agent.data.inputs).filter(\n (input) => input.request.kind === "question" && input.request.action.callId === callId,\n );\n\n const requestCancellation = () => {\n setCancellationError(undefined);\n void agent.cancel().catch((error: unknown) => {\n setCancellationError(toErrorMessage(error));\n });\n };\n\n const handleSubmit = async (message: PromptInputMessage) => {\n const text = message.text.trim();\n if ((text.length === 0 && message.files.length === 0) || isResuming) return;\n\n setHasInputText(false);\n setCancellationError(undefined);\n const options = isBusy ? { turnPolicy: "steer" as const } : undefined;\n\n if (message.files.length === 0) {\n await agent.send(text, options);\n return;\n }\n\n const parts: UserContent = [];\n if (text.length > 0) {\n parts.push({ text, type: "text" });\n }\n for (const file of message.files) {\n parts.push({\n data: file.url,\n filename: file.filename,\n mediaType: file.mediaType,\n type: "file",\n });\n }\n\n await agent.send(parts, options);\n };\n\n const composer = (\n <PromptInput onSubmit={handleSubmit}>\n <PromptInputTextarea\n disabled={isResuming}\n onChange={(event) => setHasInputText(event.currentTarget.value.trim().length > 0)}\n placeholder="Send a message…"\n />\n <ComposerAction\n hasInputText={hasInputText}\n isBusy={isBusy}\n isResuming={isResuming}\n onCancel={requestCancellation}\n />\n </PromptInput>\n );\n\n return (\n <main className="flex h-dvh flex-col overflow-hidden bg-background text-foreground">\n {showConversationLayout ? (\n <ChatHeader canStartNewChat={activeSessionId !== undefined} />\n ) : null}\n\n {showConversationLayout ? (\n <Conversation\n className="min-h-0 flex-1"\n initial={sessionId === undefined ? undefined : false}\n resize={activeSessionId === undefined ? "smooth" : "instant"}\n scrollRestorationKey={\n isEmpty || activeSessionId === undefined\n ? undefined\n : `eve:web-chat-scroll:${activeSessionId}`\n }\n >\n <ConversationTopFade className="top-14" />\n <ConversationContent className="mx-auto w-full max-w-3xl gap-6 px-4 pt-20 pb-36 sm:px-6">\n {agent.data.messages.map((message, index) =>\n showPendingThinking &&\n isPendingAssistantShell &&\n message.id === lastMessage.id ? null : (\n <AgentMessage\n canRespond={canRespond}\n isStreaming={\n agent.status === "streaming" && index === agent.data.messages.length - 1\n }\n key={message.id}\n message={message}\n questionsFor={questionsFor}\n onInputResponses={(inputResponses) => {\n setCancellationError(undefined);\n return agent.respond(inputResponses);\n }}\n />\n ),\n )}\n {showPendingThinking ? <PendingThinking /> : null}\n {errorMessage ? <ErrorMessage message={errorMessage} /> : null}\n </ConversationContent>\n <ConversationScrollButton />\n </Conversation>\n ) : null}\n\n <div\n className={cn(\n "mx-auto w-full px-4 sm:px-6",\n showConversationLayout\n ? "fixed bottom-0 left-1/2 z-20 max-w-3xl -translate-x-1/2 bg-gradient-to-t from-background via-background to-transparent pt-4 pb-6"\n : "flex max-w-xl flex-1 flex-col items-center justify-center gap-8 pb-[10vh]",\n )}\n >\n {showConversationLayout ? null : (\n <div className="flex flex-col items-center gap-3 text-center">\n <h1 className="font-medium text-5xl tracking-tighter">{AGENT_NAME}</h1>\n </div>\n )}\n <div className="w-full">{composer}</div>\n </div>\n </main>\n );\n}\n\nfunction ComposerAction({\n hasInputText,\n isBusy,\n isResuming,\n onCancel,\n}: {\n readonly hasInputText: boolean;\n readonly isBusy: boolean;\n readonly isResuming: boolean;\n readonly onCancel: () => void;\n}) {\n const attachments = usePromptInputAttachments();\n const canSubmit = hasInputText || attachments.files.length > 0;\n\n if (!isBusy || canSubmit) {\n return <PromptInputSubmit disabled={isResuming} />;\n }\n\n return (\n <PromptInputButton\n aria-label="Stop"\n className="absolute right-2.5 bottom-2.5"\n onClick={onCancel}\n variant="outline"\n >\n <SquareIcon className="size-3 fill-current" />\n </PromptInputButton>\n );\n}\n\nfunction ErrorMessage({ message }: { readonly message: string }) {\n return (\n <Message className="max-w-full" from="assistant">\n <MessageContent>\n <div\n className="flex w-full items-start gap-3 rounded-lg border border-destructive/30 bg-destructive/5 px-3 py-2.5 text-sm"\n role="alert"\n >\n <AlertCircleIcon className="mt-0.5 size-4 shrink-0 text-destructive" />\n <div>\n <p className="font-medium">Request failed</p>\n <p className="mt-0.5 text-muted-foreground">{message}</p>\n </div>\n </div>\n </MessageContent>\n </Message>\n );\n}\n\nfunction ChatHeader({ canStartNewChat }: { readonly canStartNewChat: boolean }) {\n return (\n <header className="pointer-events-none fixed top-0 right-0 left-0 z-20 h-14">\n <div className="relative mx-auto flex h-full w-full max-w-3xl items-center justify-center bg-background px-24">\n <span className="truncate text-muted-foreground text-sm">{AGENT_NAME}</span>\n {canStartNewChat ? (\n <Button\n aria-label="Start a new chat"\n className="pointer-events-auto fixed top-3 right-6 pr-4"\n onClick={() => window.location.assign("/s")}\n size="sm"\n type="button"\n variant="ghost"\n >\n <PlusIcon className="size-4" />\n <span className="hidden font-normal text-sm sm:inline">New chat</span>\n </Button>\n ) : null}\n </div>\n </header>\n );\n}\n\nfunction PendingThinking() {\n return (\n <Message aria-live="polite" from="assistant">\n <MessageContent>\n <div className="mb-4 flex w-full items-center gap-2 text-muted-foreground text-sm">\n <BrainIcon className="size-4" />\n <Shimmer duration={1}>Thinking</Shimmer>\n </div>\n </MessageContent>\n </Message>\n );\n}\n\nfunction toErrorMessage(error: unknown): string {\n return error instanceof Error ? error.message : "Unable to cancel the response.";\n}\n\nfunction getLatestTurnFailure(\n events: ReturnType<typeof useEveAgent>["events"],\n): string | undefined {\n for (let index = events.length - 1; index >= 0; index -= 1) {\n const event = events[index];\n\n if (event.type === "turn.failed") {\n return event.data.code === "MODEL_CALL_FAILED"\n ? "The model is temporarily unavailable. Please try again."\n : event.data.message;\n }\n\n if (event.type === "turn.completed" || event.type === "turn.cancelled") {\n return undefined;\n }\n\n if (event.type === "message.received") {\n return undefined;\n }\n }\n\n return undefined;\n}\n',"app/_components/agent-message.tsx":- '"use client";\n\nimport type {\n ConversationInput,\n EveAuthorizationPart,\n EveDynamicToolPart,\n EveMessage,\n EveMessageInputRequest,\n EveMessagePart,\n} from "eve/react";\nimport type { FormEvent } from "react";\nimport {\n CheckCircleIcon,\n ExternalLinkIcon,\n FileIcon,\n ImageIcon,\n KeyRoundIcon,\n XCircleIcon,\n} from "lucide-react";\nimport { Message, MessageContent, MessageResponse } from "@/components/ai-elements/message";\nimport { Reasoning, ReasoningContent, ReasoningTrigger } from "@/components/ai-elements/reasoning";\nimport {\n BashToolContent,\n Tool,\n ToolContent,\n ToolHeader,\n ToolInput,\n ToolOutput,\n} from "@/components/ai-elements/tool";\nimport { Button } from "@/components/ui/button";\nimport {\n Questionnaire,\n QuestionnaireActions,\n QuestionnaireChoice,\n QuestionnaireChoiceDescription,\n QuestionnaireChoices,\n QuestionnaireError,\n QuestionnaireInput,\n QuestionnaireItem,\n QuestionnaireSubmit,\n QuestionnaireTitle,\n} from "@/components/ui/questionnaire";\nimport { cn } from "@/lib/utils";\n\nexport type AgentInputResponse = {\n readonly optionId?: string;\n readonly requestId: string;\n readonly text?: string;\n};\n\ntype EveFilePart = Extract<EveMessagePart, { type: "file" }>;\n\nexport function AgentMessage({\n canRespond,\n isStreaming,\n message,\n onInputResponses,\n questionsFor,\n}: {\n readonly canRespond: (requestId: string) => boolean;\n readonly isStreaming: boolean;\n readonly message: EveMessage;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n readonly questionsFor: (callId: string) => readonly ConversationInput[];\n}) {\n const lastTextIndex = message.parts.reduce(\n (last, part, index) => (part.type === "text" ? index : last),\n -1,\n );\n const hasAssistantText =\n message.role === "assistant" &&\n message.parts.some((part) => part.type === "text" && part.text.length > 0);\n\n return (\n <Message\n data-optimistic={message.metadata?.optimistic ? "true" : undefined}\n from={message.role}\n >\n <MessageContent>\n {message.parts.map((part, index) =>\n hasAssistantText && part.type === "reasoning" ? null : (\n <AgentMessagePart\n canRespond={canRespond}\n key={partKey(part, index)}\n onInputResponses={onInputResponses}\n part={part}\n questionsFor={questionsFor}\n showCaret={isStreaming && message.role === "assistant" && index === lastTextIndex}\n />\n ),\n )}\n </MessageContent>\n </Message>\n );\n}\n\nfunction AgentMessagePart({\n canRespond,\n onInputResponses,\n part,\n questionsFor,\n showCaret,\n}: {\n readonly canRespond: (requestId: string) => boolean;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n readonly part: EveMessagePart;\n readonly questionsFor: (callId: string) => readonly ConversationInput[];\n readonly showCaret: boolean;\n}) {\n switch (part.type) {\n case "step-start":\n return null;\n case "text":\n return (\n <MessageResponse caret="block" isAnimating={showCaret}>\n {part.text}\n </MessageResponse>\n );\n case "reasoning":\n return (\n <Reasoning defaultOpen isStreaming={part.state === "streaming"}>\n <ReasoningTrigger />\n <ReasoningContent>{part.text}</ReasoningContent>\n </Reasoning>\n );\n case "file":\n return <AttachmentPart part={part} />;\n case "authorization":\n return <AuthorizationPrompt part={part} />;\n case "dynamic-tool": {\n const questions = questionsFor(part.toolCallId);\n if (questions.length > 0) {\n return (\n <div className="space-y-4">\n {questions.map(({ request, response }) => (\n <QuestionRequest\n canRespond={canRespond(request.requestId)}\n inputRequest={request}\n inputResponse={response}\n key={request.requestId}\n onInputResponses={onInputResponses}\n />\n ))}\n </div>\n );\n }\n\n return (\n <>\n <Tool>\n <ToolHeader\n state={part.state}\n title={part.toolName}\n toolName={part.toolName}\n type="dynamic-tool"\n />\n <ToolContent>\n {part.toolName === "bash" ? (\n <BashToolContent\n errorText={part.errorText}\n input={part.input}\n output={part.output}\n />\n ) : (\n <>\n <ToolInput input={part.input} />\n <ToolOutput errorText={part.errorText} output={part.output} />\n </>\n )}\n </ToolContent>\n </Tool>\n {/* Kept outside the collapsed card so a pending approval is always visible. */}\n <InputRequestActions\n canRespond={canRespond}\n part={part}\n onInputResponses={onInputResponses}\n />\n </>\n );\n }\n }\n}\n\nfunction QuestionRequest({\n canRespond,\n inputRequest,\n inputResponse,\n onInputResponses,\n}: {\n readonly canRespond: boolean;\n readonly inputRequest: EveMessageInputRequest;\n readonly inputResponse?: AgentInputResponse;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n}) {\n const options = inputRequest.options ?? [];\n const acceptsFreeform = inputRequest.allowFreeform === true || options.length === 0;\n const disabled = !canRespond || inputResponse !== undefined;\n\n const submitResponse = (event: FormEvent<HTMLFormElement>) => {\n event.preventDefault();\n const answer = new FormData(event.currentTarget).get(inputRequest.requestId);\n if (typeof answer !== "string") {\n return;\n }\n\n const option = options.find(({ id }) => id === answer);\n void onInputResponses([\n option === undefined\n ? { requestId: inputRequest.requestId, text: answer.trim() }\n : { optionId: option.id, requestId: inputRequest.requestId },\n ]);\n };\n\n return (\n <Questionnaire\n className="gap-4 rounded-xl border bg-card p-4"\n // Validation reads disabled answers as missing, so locking a submitted form in place would\n // flag it invalid. Remounting restores the answer from the response instead.\n key={disabled ? "locked" : "open"}\n onSubmit={submitResponse}\n shortcuts="numbers"\n >\n <QuestionnaireItem name={inputRequest.requestId} required>\n <QuestionnaireTitle className="text-sm font-medium">\n {inputRequest.prompt}\n </QuestionnaireTitle>\n <QuestionnaireChoices>\n {options.map((option) => (\n <QuestionnaireChoice\n className={cn(inputResponse?.optionId === option.id && "data-disabled:opacity-100")}\n defaultChecked={inputResponse?.optionId === option.id}\n disabled={disabled}\n key={option.id}\n value={option.id}\n >\n {option.label}\n {option.description ? (\n <QuestionnaireChoiceDescription>\n {option.description}\n </QuestionnaireChoiceDescription>\n ) : null}\n </QuestionnaireChoice>\n ))}\n {acceptsFreeform ? (\n <QuestionnaireInput\n aria-label="Answer"\n className={cn(inputResponse?.text !== undefined && "disabled:opacity-100")}\n defaultValue={inputResponse?.text}\n disabled={disabled}\n placeholder="Type your answer…"\n />\n ) : null}\n </QuestionnaireChoices>\n <QuestionnaireError />\n </QuestionnaireItem>\n {disabled ? null : (\n <QuestionnaireActions>\n <QuestionnaireSubmit size="sm">Answer</QuestionnaireSubmit>\n </QuestionnaireActions>\n )}\n </Questionnaire>\n );\n}\n\nfunction AttachmentPart({ part }: { readonly part: EveFilePart }) {\n const label = part.filename ?? "Attachment";\n const detail = [part.mediaType, formatBytes(part.size)].filter(Boolean).join(" - ");\n const isImage = part.mediaType.startsWith("image/") && part.url !== undefined;\n const Icon = isImage ? ImageIcon : FileIcon;\n const body = (\n <span className="flex max-w-sm items-center gap-3 rounded-md border bg-background/60 p-2 text-sm">\n {isImage ? (\n <img alt={label} className="size-12 shrink-0 rounded-sm object-cover" src={part.url} />\n ) : (\n <span className="flex size-10 shrink-0 items-center justify-center rounded-sm bg-muted text-muted-foreground">\n <Icon className="size-4" />\n </span>\n )}\n <span className="min-w-0 flex-1">\n <span className="block truncate font-medium">{label}</span>\n {detail ? <span className="block truncate text-muted-foreground">{detail}</span> : null}\n </span>\n {part.url ? <ExternalLinkIcon className="size-4 shrink-0 text-muted-foreground" /> : null}\n </span>\n );\n\n return part.url ? (\n <a href={part.url} rel="noreferrer" target="_blank">\n {body}\n </a>\n ) : (\n body\n );\n}\n\nfunction AuthorizationPrompt({ part }: { readonly part: EveAuthorizationPart }) {\n const isAuthorized = part.state === "completed" && part.outcome === "authorized";\n const isCompleted = part.state === "completed";\n const Icon = isAuthorized ? CheckCircleIcon : isCompleted ? XCircleIcon : KeyRoundIcon;\n const instructions = part.authorization?.instructions;\n const shouldShowInstructions = instructions !== undefined && instructions !== part.description;\n\n return (\n <div\n className={cn(\n "space-y-3 rounded-md border p-3",\n isAuthorized\n ? "border-emerald-500/30 bg-emerald-500/5"\n : isCompleted\n ? "border-destructive/30 bg-destructive/5"\n : "border-blue-500/30 bg-blue-500/5",\n )}\n >\n <div className="flex items-start gap-3">\n <span\n className={cn(\n "mt-0.5 flex size-8 shrink-0 items-center justify-center rounded-full",\n isAuthorized\n ? "bg-emerald-500/10 text-emerald-700 dark:text-emerald-300"\n : isCompleted\n ? "bg-destructive/10 text-destructive"\n : "bg-blue-500/10 text-blue-700 dark:text-blue-300",\n )}\n >\n <Icon className="size-4" />\n </span>\n <div className="min-w-0 flex-1 space-y-2">\n <p className="font-medium text-sm">{authorizationTitle(part)}</p>\n <p className="text-muted-foreground text-sm">{authorizationDescription(part)}</p>\n {shouldShowInstructions ? (\n <p className="text-muted-foreground text-sm">{instructions}</p>\n ) : null}\n {part.state === "required" && part.authorization?.userCode ? (\n <div className="flex flex-wrap items-center gap-2 text-sm">\n <span className="text-muted-foreground">Code</span>\n <code className="rounded-md bg-background px-2 py-1 font-mono">\n {part.authorization.userCode}\n </code>\n </div>\n ) : null}\n {part.state === "required" && part.authorization?.url ? (\n <Button asChild size="sm">\n <a href={part.authorization.url} rel="noreferrer" target="_blank">\n <ExternalLinkIcon className="size-4" />\n Sign in with {part.displayName}\n </a>\n </Button>\n ) : null}\n </div>\n </div>\n </div>\n );\n}\n\nfunction authorizationTitle(part: EveAuthorizationPart): string {\n if (part.state === "required") {\n return `Connect ${part.displayName}`;\n }\n if (part.outcome === "authorized") {\n return `${part.displayName} connected`;\n }\n return `${part.displayName} authorization ${formatAuthorizationOutcome(part.outcome)}`;\n}\n\nfunction authorizationDescription(part: EveAuthorizationPart): string {\n if (part.state === "required") {\n return part.description;\n }\n if (part.outcome === "authorized") {\n return `${part.displayName} connected.`;\n }\n const tail = part.reason !== undefined ? ` (${part.reason})` : "";\n return `${part.displayName} authorization ${formatAuthorizationOutcome(part.outcome)}${tail}.`;\n}\n\nfunction formatAuthorizationOutcome(outcome: NonNullable<EveAuthorizationPart["outcome"]>): string {\n switch (outcome) {\n case "authorized":\n return "authorized";\n case "declined":\n return "declined";\n case "failed":\n return "failed";\n case "timed-out":\n return "timed out";\n }\n}\n\nfunction formatBytes(size: number | undefined): string | undefined {\n if (size === undefined) {\n return undefined;\n }\n if (size < 1024) {\n return `${size} B`;\n }\n if (size < 1024 * 1024) {\n return `${(size / 1024).toFixed(1)} KB`;\n }\n return `${(size / (1024 * 1024)).toFixed(1)} MB`;\n}\n\nfunction InputRequestActions({\n canRespond,\n onInputResponses,\n part,\n}: {\n readonly canRespond: (requestId: string) => boolean;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n readonly part: EveDynamicToolPart;\n}) {\n const inputRequest = part.toolMetadata?.eve?.inputRequest;\n if (!inputRequest) {\n return null;\n }\n\n const inputResponse = part.toolMetadata?.eve?.inputResponse;\n const selectedOption = inputRequest.options?.find(\n (option) => option.id === inputResponse?.optionId,\n );\n // An approval can settle before its batch resolves, without an input response.\n const settledApproval =\n part.approval?.approved === undefined\n ? undefined\n : part.approval.approved\n ? "Approved"\n : "Denied";\n\n return (\n <div className="space-y-3 rounded-md border border-yellow-500/30 bg-yellow-500/5 p-3">\n <p className="text-muted-foreground text-sm">{inputRequest.prompt}</p>\n {inputResponse || settledApproval ? (\n <p className="font-medium text-sm">\n Responded:{" "}\n {selectedOption?.label ??\n inputResponse?.text ??\n inputResponse?.optionId ??\n settledApproval}\n </p>\n ) : (\n <div className="flex flex-wrap gap-2">\n {inputRequest.options?.map((option) => (\n <Button\n disabled={!canRespond(inputRequest.requestId)}\n key={option.id}\n onClick={() => {\n void onInputResponses([\n {\n optionId: option.id,\n requestId: inputRequest.requestId,\n },\n ]);\n }}\n size="sm"\n type="button"\n variant={option.style === "danger" ? "destructive" : "default"}\n >\n {option.label}\n </Button>\n ))}\n </div>\n )}\n </div>\n );\n}\n\nfunction partKey(part: EveMessagePart, index: number): string {\n switch (part.type) {\n case "authorization":\n return part.attemptId === undefined\n ? `authorization:${part.turnId}:${part.stepIndex}:${part.name}`\n : `authorization:${part.attemptId}`;\n case "dynamic-tool":\n return part.toolCallId;\n case "reasoning":\n case "text":\n return `${part.type}:${part.id ?? index}`;\n default:\n return `${part.type}:${index}`;\n }\n}\n',+ '"use client";\n\nimport type {\n ConversationInput,\n EveAuthorizationPart,\n EveDynamicToolPart,\n EveMessage,\n EveMessageInputRequest,\n EveMessagePart,\n} from "eve/react";\nimport type { FormEvent } from "react";\nimport {\n CheckCircleIcon,\n ExternalLinkIcon,\n FileIcon,\n ImageIcon,\n KeyRoundIcon,\n XCircleIcon,\n} from "lucide-react";\nimport { Message, MessageContent, MessageResponse } from "@/components/ai-elements/message";\nimport { Reasoning, ReasoningContent, ReasoningTrigger } from "@/components/ai-elements/reasoning";\nimport {\n BashToolContent,\n Tool,\n ToolContent,\n ToolHeader,\n ToolInput,\n ToolOutput,\n} from "@/components/ai-elements/tool";\nimport { Button } from "@/components/ui/button";\nimport {\n Questionnaire,\n QuestionnaireActions,\n QuestionnaireChoice,\n QuestionnaireChoiceDescription,\n QuestionnaireChoices,\n QuestionnaireError,\n QuestionnaireInput,\n QuestionnaireItem,\n QuestionnaireSubmit,\n QuestionnaireTitle,\n} from "@/components/ui/questionnaire";\nimport { cn } from "@/lib/utils";\n\nexport type AgentInputResponse = {\n readonly optionId?: string;\n readonly requestId: string;\n readonly text?: string;\n};\n\ntype EveFilePart = Extract<EveMessagePart, { type: "file" }>;\n\nexport function AgentMessage({\n canRespond,\n isStreaming,\n message,\n onInputResponses,\n questionsFor,\n}: {\n readonly canRespond: (requestId: string) => boolean;\n readonly isStreaming: boolean;\n readonly message: EveMessage;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n readonly questionsFor: (callId: string) => readonly ConversationInput[];\n}) {\n const lastTextIndex = message.parts.reduce(\n (last, part, index) => (part.type === "text" ? index : last),\n -1,\n );\n const hasAssistantText =\n message.role === "assistant" &&\n message.parts.some((part) => part.type === "text" && part.text.length > 0);\n\n return (\n <Message\n data-optimistic={message.metadata?.optimistic ? "true" : undefined}\n from={message.role}\n >\n <MessageContent>\n {message.parts.map((part, index) =>\n hasAssistantText && part.type === "reasoning" ? null : (\n <AgentMessagePart\n canRespond={canRespond}\n key={partKey(part, index)}\n onInputResponses={onInputResponses}\n part={part}\n questionsFor={questionsFor}\n showCaret={isStreaming && message.role === "assistant" && index === lastTextIndex}\n />\n ),\n )}\n </MessageContent>\n </Message>\n );\n}\n\nfunction AgentMessagePart({\n canRespond,\n onInputResponses,\n part,\n questionsFor,\n showCaret,\n}: {\n readonly canRespond: (requestId: string) => boolean;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n readonly part: EveMessagePart;\n readonly questionsFor: (callId: string) => readonly ConversationInput[];\n readonly showCaret: boolean;\n}) {\n switch (part.type) {\n case "step-start":\n return null;\n case "text":\n return (\n <MessageResponse caret="block" isAnimating={showCaret}>\n {part.text}\n </MessageResponse>\n );\n case "reasoning":\n return (\n <Reasoning defaultOpen isStreaming={part.state === "streaming"}>\n <ReasoningTrigger />\n <ReasoningContent>{part.text}</ReasoningContent>\n </Reasoning>\n );\n case "file":\n return <AttachmentPart part={part} />;\n case "authorization":\n return <AuthorizationPrompt part={part} />;\n case "dynamic-tool": {\n const questions = questionsFor(part.toolCallId);\n if (questions.length > 0) {\n return (\n <div className="space-y-4">\n {questions.map(({ request, response }) => (\n <QuestionRequest\n canRespond={canRespond(request.requestId)}\n inputRequest={request}\n inputResponse={response}\n key={request.requestId}\n onInputResponses={onInputResponses}\n />\n ))}\n </div>\n );\n }\n\n return (\n <>\n <Tool>\n <ToolHeader\n state={part.state}\n title={part.toolMetadata?.eve?.label ?? part.toolName}\n toolName={part.toolName}\n type="dynamic-tool"\n />\n <ToolContent>\n {part.toolName === "bash" ? (\n <BashToolContent\n errorText={part.errorText}\n input={part.input}\n output={part.output}\n />\n ) : (\n <>\n <ToolInput input={part.input} />\n <ToolOutput errorText={part.errorText} output={part.output} />\n </>\n )}\n </ToolContent>\n </Tool>\n {/* Kept outside the collapsed card so a pending approval is always visible. */}\n <InputRequestActions\n canRespond={canRespond}\n part={part}\n onInputResponses={onInputResponses}\n />\n </>\n );\n }\n }\n}\n\nfunction QuestionRequest({\n canRespond,\n inputRequest,\n inputResponse,\n onInputResponses,\n}: {\n readonly canRespond: boolean;\n readonly inputRequest: EveMessageInputRequest;\n readonly inputResponse?: AgentInputResponse;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n}) {\n const options = inputRequest.options ?? [];\n const acceptsFreeform = inputRequest.allowFreeform === true || options.length === 0;\n const disabled = !canRespond || inputResponse !== undefined;\n\n const submitResponse = (event: FormEvent<HTMLFormElement>) => {\n event.preventDefault();\n const answer = new FormData(event.currentTarget).get(inputRequest.requestId);\n if (typeof answer !== "string") {\n return;\n }\n\n const option = options.find(({ id }) => id === answer);\n void onInputResponses([\n option === undefined\n ? { requestId: inputRequest.requestId, text: answer.trim() }\n : { optionId: option.id, requestId: inputRequest.requestId },\n ]);\n };\n\n return (\n <Questionnaire\n className="gap-4 rounded-xl border bg-card p-4"\n // Validation reads disabled answers as missing, so locking a submitted form in place would\n // flag it invalid. Remounting restores the answer from the response instead.\n key={disabled ? "locked" : "open"}\n onSubmit={submitResponse}\n shortcuts="numbers"\n >\n <QuestionnaireItem name={inputRequest.requestId} required>\n <QuestionnaireTitle className="text-sm font-medium">\n {inputRequest.prompt}\n </QuestionnaireTitle>\n <QuestionnaireChoices>\n {options.map((option) => (\n <QuestionnaireChoice\n className={cn(inputResponse?.optionId === option.id && "data-disabled:opacity-100")}\n defaultChecked={inputResponse?.optionId === option.id}\n disabled={disabled}\n key={option.id}\n value={option.id}\n >\n {option.label}\n {option.description ? (\n <QuestionnaireChoiceDescription>\n {option.description}\n </QuestionnaireChoiceDescription>\n ) : null}\n </QuestionnaireChoice>\n ))}\n {acceptsFreeform ? (\n <QuestionnaireInput\n aria-label="Answer"\n className={cn(inputResponse?.text !== undefined && "disabled:opacity-100")}\n defaultValue={inputResponse?.text}\n disabled={disabled}\n placeholder="Type your answer…"\n />\n ) : null}\n </QuestionnaireChoices>\n <QuestionnaireError />\n </QuestionnaireItem>\n {disabled ? null : (\n <QuestionnaireActions>\n <QuestionnaireSubmit size="sm">Answer</QuestionnaireSubmit>\n </QuestionnaireActions>\n )}\n </Questionnaire>\n );\n}\n\nfunction AttachmentPart({ part }: { readonly part: EveFilePart }) {\n const label = part.filename ?? "Attachment";\n const detail = [part.mediaType, formatBytes(part.size)].filter(Boolean).join(" - ");\n const isImage = part.mediaType.startsWith("image/") && part.url !== undefined;\n const Icon = isImage ? ImageIcon : FileIcon;\n const body = (\n <span className="flex max-w-sm items-center gap-3 rounded-md border bg-background/60 p-2 text-sm">\n {isImage ? (\n <img alt={label} className="size-12 shrink-0 rounded-sm object-cover" src={part.url} />\n ) : (\n <span className="flex size-10 shrink-0 items-center justify-center rounded-sm bg-muted text-muted-foreground">\n <Icon className="size-4" />\n </span>\n )}\n <span className="min-w-0 flex-1">\n <span className="block truncate font-medium">{label}</span>\n {detail ? <span className="block truncate text-muted-foreground">{detail}</span> : null}\n </span>\n {part.url ? <ExternalLinkIcon className="size-4 shrink-0 text-muted-foreground" /> : null}\n </span>\n );\n\n return part.url ? (\n <a href={part.url} rel="noreferrer" target="_blank">\n {body}\n </a>\n ) : (\n body\n );\n}\n\nfunction AuthorizationPrompt({ part }: { readonly part: EveAuthorizationPart }) {\n const isAuthorized = part.state === "completed" && part.outcome === "authorized";\n const isCompleted = part.state === "completed";\n const Icon = isAuthorized ? CheckCircleIcon : isCompleted ? XCircleIcon : KeyRoundIcon;\n const instructions = part.authorization?.instructions;\n const shouldShowInstructions = instructions !== undefined && instructions !== part.description;\n\n return (\n <div\n className={cn(\n "space-y-3 rounded-md border p-3",\n isAuthorized\n ? "border-emerald-500/30 bg-emerald-500/5"\n : isCompleted\n ? "border-destructive/30 bg-destructive/5"\n : "border-blue-500/30 bg-blue-500/5",\n )}\n >\n <div className="flex items-start gap-3">\n <span\n className={cn(\n "mt-0.5 flex size-8 shrink-0 items-center justify-center rounded-full",\n isAuthorized\n ? "bg-emerald-500/10 text-emerald-700 dark:text-emerald-300"\n : isCompleted\n ? "bg-destructive/10 text-destructive"\n : "bg-blue-500/10 text-blue-700 dark:text-blue-300",\n )}\n >\n <Icon className="size-4" />\n </span>\n <div className="min-w-0 flex-1 space-y-2">\n <p className="font-medium text-sm">{authorizationTitle(part)}</p>\n <p className="text-muted-foreground text-sm">{authorizationDescription(part)}</p>\n {shouldShowInstructions ? (\n <p className="text-muted-foreground text-sm">{instructions}</p>\n ) : null}\n {part.state === "required" && part.authorization?.userCode ? (\n <div className="flex flex-wrap items-center gap-2 text-sm">\n <span className="text-muted-foreground">Code</span>\n <code className="rounded-md bg-background px-2 py-1 font-mono">\n {part.authorization.userCode}\n </code>\n </div>\n ) : null}\n {part.state === "required" && part.authorization?.url ? (\n <Button asChild size="sm">\n <a href={part.authorization.url} rel="noreferrer" target="_blank">\n <ExternalLinkIcon className="size-4" />\n Sign in with {part.displayName}\n </a>\n </Button>\n ) : null}\n </div>\n </div>\n </div>\n );\n}\n\nfunction authorizationTitle(part: EveAuthorizationPart): string {\n if (part.state === "required") {\n return `Connect ${part.displayName}`;\n }\n if (part.outcome === "authorized") {\n return `${part.displayName} connected`;\n }\n return `${part.displayName} authorization ${formatAuthorizationOutcome(part.outcome)}`;\n}\n\nfunction authorizationDescription(part: EveAuthorizationPart): string {\n if (part.state === "required") {\n return part.description;\n }\n if (part.outcome === "authorized") {\n return `${part.displayName} connected.`;\n }\n const tail = part.reason !== undefined ? ` (${part.reason})` : "";\n return `${part.displayName} authorization ${formatAuthorizationOutcome(part.outcome)}${tail}.`;\n}\n\nfunction formatAuthorizationOutcome(outcome: NonNullable<EveAuthorizationPart["outcome"]>): string {\n switch (outcome) {\n case "authorized":\n return "authorized";\n case "declined":\n return "declined";\n case "failed":\n return "failed";\n case "timed-out":\n return "timed out";\n }\n}\n\nfunction formatBytes(size: number | undefined): string | undefined {\n if (size === undefined) {\n return undefined;\n }\n if (size < 1024) {\n return `${size} B`;\n }\n if (size < 1024 * 1024) {\n return `${(size / 1024).toFixed(1)} KB`;\n }\n return `${(size / (1024 * 1024)).toFixed(1)} MB`;\n}\n\nfunction InputRequestActions({\n canRespond,\n onInputResponses,\n part,\n}: {\n readonly canRespond: (requestId: string) => boolean;\n readonly onInputResponses: (responses: readonly AgentInputResponse[]) => void | Promise<void>;\n readonly part: EveDynamicToolPart;\n}) {\n const inputRequest = part.toolMetadata?.eve?.inputRequest;\n if (!inputRequest) {\n return null;\n }\n\n const inputResponse = part.toolMetadata?.eve?.inputResponse;\n const selectedOption = inputRequest.options?.find(\n (option) => option.id === inputResponse?.optionId,\n );\n // An approval can settle before its batch resolves, without an input response.\n const settledApproval =\n part.approval?.approved === undefined\n ? undefined\n : part.approval.approved\n ? "Approved"\n : "Denied";\n\n return (\n <div className="space-y-3 rounded-md border border-yellow-500/30 bg-yellow-500/5 p-3">\n <p className="text-muted-foreground text-sm">{inputRequest.prompt}</p>\n {inputResponse || settledApproval ? (\n <p className="font-medium text-sm">\n Responded:{" "}\n {selectedOption?.label ??\n inputResponse?.text ??\n inputResponse?.optionId ??\n settledApproval}\n </p>\n ) : (\n <div className="flex flex-wrap gap-2">\n {inputRequest.options?.map((option) => (\n <Button\n disabled={!canRespond(inputRequest.requestId)}\n key={option.id}\n onClick={() => {\n void onInputResponses([\n {\n optionId: option.id,\n requestId: inputRequest.requestId,\n },\n ]);\n }}\n size="sm"\n type="button"\n variant={option.style === "danger" ? "destructive" : "default"}\n >\n {option.label}\n </Button>\n ))}\n </div>\n )}\n </div>\n );\n}\n\nfunction partKey(part: EveMessagePart, index: number): string {\n switch (part.type) {\n case "authorization":\n return part.attemptId === undefined\n ? `authorization:${part.turnId}:${part.stepIndex}:${part.name}`\n : `authorization:${part.attemptId}`;\n case "dynamic-tool":\n return part.toolCallId;\n case "reasoning":\n case "text":\n return `${part.type}:${part.id ?? index}`;\n default:\n return `${part.type}:${index}`;\n }\n}\n',"app/apple-icon.tsx":'import { ImageResponse } from "next/og";\n\nexport const size = {\n width: 180,\n height: 180,\n};\n\nexport const contentType = "image/png";\n\nexport default function AppleIcon() {\n return new ImageResponse(\n <svg fill="none" viewBox="0 0 102 102" xmlns="http://www.w3.org/2000/svg">\n <path d="M0 0h102v102H0z" fill="#000" />\n <path\n d="M49.28 66.94 75.03 34.96h-6.89L47.91 60.11l-5.49 6.83h6.86ZM0 34.96h42.4v5.11H0zm0 13.32h27.66v5.11H0zm0 13.54h27.66v5.11H0zm69.63-26.86H102v5.11H69.63zm4.71 13.32H102v5.11H74.34zm0 13.54H102v5.11H74.34z"\n fill="#fff"\n />\n </svg>,\n size,\n );\n}\n',"app/eve-agent.ts":diff --git a/packages/eve/src/shared/action-label.ts b/packages/eve/src/shared/action-label.tsnew file mode 100644index 000000000..d0dce2b1c--- /dev/null+++ b/packages/eve/src/shared/action-label.ts@@ -0,0 +1,24 @@+import type { ActionPresentationByCallId } from "#protocol/message.js";+import { isTaskControlTool } from "#protocol/task-tools.js";+import { actionRequestName } from "#shared/action-request-name.js";+import type { RuntimeActionRequest } from "#shared/action-types.js";+import { displayTitle } from "#shared/display-name.js";++/**+ * How a call reads to people on every surface: the label its entry's definition+ * gives it, or the entry's display title. A call made through `eve__tool` or+ * `eve__skill` arrives as the call to its entry, so it reads like a direct call.+ */+export function actionLabel(+ action: RuntimeActionRequest,+ presentation: ActionPresentationByCallId | undefined,+): string {+ return presentation?.[action.callId]?.label ?? displayTitle(actionRequestName(action));+}++/** The calls people see: the model waiting on or stopping its own tasks stays out of view. */+export function visibleActions<T extends RuntimeActionRequest>(actions: readonly T[]): T[] {+ return actions.filter(+ (action) => !(action.kind === "tool-call" && isTaskControlTool(action.toolName)),+ );+}diff --git a/packages/eve/src/shared/action-request-name.ts b/packages/eve/src/shared/action-request-name.tsindex 7b263a6f5..bb4134a51 100644--- a/packages/eve/src/shared/action-request-name.ts+++ b/packages/eve/src/shared/action-request-name.ts@@ -1,10 +1,12 @@+import { SKILL_ENTRY_NAME } from "#protocol/catalog-tools.js";import type { RuntimeActionRequest } from "#shared/action-types.js";+import { isObject } from "#shared/guards.js";-/** The tool, agent, or `load_skill` a request targets. */+/** The tool, agent, or skill a request targets. */export function actionRequestName(action: RuntimeActionRequest): string {switch (action.kind) {case "load-skill":- return "load_skill";+ return requestedSkill(action);case "subagent-call":return action.subagentName;case "remote-agent-call":@@ -14,3 +16,15 @@ export function actionRequestName(action: RuntimeActionRequest): string {return action.toolName;}}++/** The skill a `load-skill` request loads, or the loader itself when its input names none. */+export function requestedSkill(+ action: Extract<RuntimeActionRequest, { readonly kind: "load-skill" }>,+): string {+ return skillTarget(action.input) ?? SKILL_ENTRY_NAME;+}++/** The skill a skill loader's input names, if it names one. */+export function skillTarget(input: unknown): string | undefined {+ return isObject(input) && typeof input.skill === "string" ? input.skill : undefined;+}diff --git a/packages/eve/src/shared/action-types.ts b/packages/eve/src/shared/action-types.tsindex 45b6bbce5..28fad57c3 100644--- a/packages/eve/src/shared/action-types.ts+++ b/packages/eve/src/shared/action-types.ts@@ -18,11 +18,6 @@ export const runtimeToolCallActionRequestSchema = zcallId: z.string(),input: jsonObjectSchema,kind: z.literal("tool-call"),- /**- * Set on a nested action: a call a tool made on the model's behalf. Names- * the call id of the tool call that made it.- */- parentCallId: z.string().optional(),toolName: z.string(),}).strict();@@ -128,7 +123,7 @@ export type WorkflowToolRunEntry = Exclude<>;/**- * How a deferred call enters its tool's workflow: through the entry point of+ * How a workflow tool call enters its tool's workflow: through the entry point of* a new run, or, for a call with `taskId` to a `serve` tool, through the* `receive()` of the running task that id names.*/diff --git a/packages/eve/src/shared/agent-definition.ts b/packages/eve/src/shared/agent-definition.tsindex f0acbc502..0731c7397 100644--- a/packages/eve/src/shared/agent-definition.ts+++ b/packages/eve/src/shared/agent-definition.ts@@ -295,6 +295,13 @@ export interface AgentWorkflowDefinition {readonly world?: AgentWorkflowWorldDefinition;}+/**+ * How eve exposes an agent to its parent model: `true` as a tool in the+ * model's tool list, `"deferred"` as a catalog entry the model finds with+ * `eve__search` and calls with `eve__tool`, and `false` not at all.+ */+export type AgentToolExposure = boolean | "deferred";+/*** Compiled-side agent definition. Carries a `name` because the compiler* stamps the path-derived `agentId` onto every compiled agent node.@@ -309,7 +316,7 @@ export type InternalAgentDefinition = {model: InternalAgentModelDefinition;reasoning?: AgentReasoningDefinition;source?: ModuleSourceRef;- tool?: boolean;+ tool?: AgentToolExposure;limits?: AgentLimitsDefinition;};@@ -350,13 +357,15 @@ type PublicAgentDefinitionBase = {*/readonly limits?: AgentLimitsDefinition;/**- * Whether eve exposes this agent to its parent model as a tool. On the root+ * How eve exposes this agent to its parent model. `true` lists it as a+ * tool, `"deferred"` makes it a catalog entry the model finds with `eve__search`+ * and calls with `eve__tool`, and `false` hides it from the model. On the root* agent, this controls the built-in `agent` tool. Defaults to `true`.** A subagent with this set to `false` remains callable through `ctx.agent()`* in workflow tools.*/- readonly tool?: boolean;+ readonly tool?: AgentToolExposure;};/**diff --git a/packages/eve/src/shared/skill-definition.ts b/packages/eve/src/shared/skill-definition.tsindex 09ae9cec6..b2244f812 100644--- a/packages/eve/src/shared/skill-definition.ts+++ b/packages/eve/src/shared/skill-definition.ts@@ -9,6 +9,12 @@ export type SkillFileContent = string | Uint8Array;* resolvers.*/export interface SkillPackageDefinition {+ /**+ * Keeps this skill out of the system prompt and the dynamic skill+ * announcement. The model finds it with `eve__search` and loads it with+ * `eve__skill({ name })`. Defaults to `false`.+ */+ readonly deferred?: boolean;readonly description: string;readonly license?: string;readonly markdown: string;diff --git a/packages/eve/src/shared/skill-package.ts b/packages/eve/src/shared/skill-package.tsindex 660ff44af..dfcdb9e31 100644--- a/packages/eve/src/shared/skill-package.ts+++ b/packages/eve/src/shared/skill-package.ts@@ -13,6 +13,7 @@ export interface NormalizedSkillPackageFile {}export interface MaterializableSkillPackage {+ readonly deferred?: boolean;readonly description: string;readonly files: readonly NormalizedSkillPackageFile[];readonly license?: string;@@ -59,6 +60,7 @@ export function normalizeSkillPackage(input: NamedSkillDefinition): Materializabfiles.sort((left, right) => comparePaths(left.relativePath, right.relativePath));return {+ deferred: input.deferred,description: input.description,files,license: input.license,diff --git a/packages/eve/src/tool-stubs/execute.ts b/packages/eve/src/tool-stubs/execute.tsindex 362ba2613..de5f6845e 100644--- a/packages/eve/src/tool-stubs/execute.ts+++ b/packages/eve/src/tool-stubs/execute.ts@@ -3,7 +3,6 @@ import { toolStubOutput } from "#tool-stubs/output.js";import { contextStorage } from "#context/container.js";import { SessionKey, ToolStubsKey } from "#context/keys.js";import { ToolStubPlaybackKey, type ToolStubPlayback } from "#context/providers/tool-stubs-key.js";-import type { StubResult } from "#tool-stubs/types.js";import type { ToolExecuteOptions } from "#tools/definition.js";import { isAsyncIterable } from "#shared/async-iterable.js";@@ -37,16 +36,6 @@ async function* executeStubbedTool(else yield output;}-/** The connection wrapper checks operation inputs and approvals before requesting a stub. */-export async function connectionToolStub(- tool: string,- input: unknown,- callId: string,-): Promise<StubResult> {- const stub = contextStubCall(tool, input, callId);- return stub === undefined ? { kind: "real" } : await stub.playback.call(stub.call);-}-function contextStubCall(tool: string, input: unknown, callId: string) {const context = contextStorage.getStore();if (context === undefined) return undefined;@@ -72,8 +61,7 @@ export async function recordToolStubFailure(const context = contextStorage.getStore();const scope = context?.get(ToolStubsKey);if (scope === undefined || callId === undefined) return;- // Connection output conversion runs on the wrapper, after a qualified operation was stubbed.- if (tool !== "connection_execute" && findStubTarget(scope, tool) === undefined) return;+ if (findStubTarget(scope, tool) === undefined) return;const session = context!.require(SessionKey);await context!.require(ToolStubPlaybackKey)diff --git a/packages/eve/src/tool-stubs/rules.test.ts b/packages/eve/src/tool-stubs/rules.test.tsindex 605bbfa3a..739ecad96 100644--- a/packages/eve/src/tool-stubs/rules.test.ts+++ b/packages/eve/src/tool-stubs/rules.test.ts@@ -14,6 +14,14 @@ describe("tool stubs", () => {).toThrow(/prototype/i);});+ it("rejects stubbing the catalog tools and points at the tool a call reaches", () => {+ for (const tool of ["eve__search", "researcher/eve__tool", "eve__skill"]) {+ expect(() => parseToolStubs([{ id: "a", tool, outcome: { response: null } }])).toThrow(+ "Cannot stub eve__search, eve__tool, or eve__skill. Stub the tool an eve__tool call reaches, such as linear__list_issues.",+ );+ }+ });+it("selects a response by partial input without changing the arguments", () => {const playback = new StubPlayback(parseToolStubs([diff --git a/packages/eve/src/tool-stubs/rules.ts b/packages/eve/src/tool-stubs/rules.tsindex 3749a137a..98412821b 100644--- a/packages/eve/src/tool-stubs/rules.ts+++ b/packages/eve/src/tool-stubs/rules.ts@@ -1,4 +1,5 @@import { z } from "#compiled/zod/index.js";+import { CALL_TOOL_NAME, SEARCH_TOOL_NAME, SKILL_TOOL_NAME } from "#protocol/catalog-tools.js";import { parseJsonValue } from "#shared/json.js";import { createStubValidator, validateStubConstraint } from "#tool-stubs/schema.js";import type { ToolStub } from "#tool-stubs/types.js";@@ -33,9 +34,11 @@ export function parseToolStubs(value: unknown): readonly ToolStub[] {for (const rule of rules) {if (ids.has(rule.id)) throw new Error(`Duplicate tool stub id "${rule.id}".`);ids.add(rule.id);- if (["connection_execute", "connection_search"].includes(rule.tool.split("/").at(-1)!)) {+ if (+ [SEARCH_TOOL_NAME, CALL_TOOL_NAME, SKILL_TOOL_NAME].includes(rule.tool.split("/").at(-1)!)+ ) {throw new Error(- "Cannot stub connection_execute or connection_search. Stub a specific connection operation, such as linear__list_issues.",+ `Cannot stub ${SEARCH_TOOL_NAME}, ${CALL_TOOL_NAME}, or ${SKILL_TOOL_NAME}. Stub the tool an ${CALL_TOOL_NAME} call reaches, such as linear__list_issues.`,);}for (const [property, schema] of Object.entries(rule.match ?? {})) {diff --git a/packages/eve/src/tool-stubs/validate-targets.ts b/packages/eve/src/tool-stubs/validate-targets.tsindex be3360cbc..f9d518f52 100644--- a/packages/eve/src/tool-stubs/validate-targets.ts+++ b/packages/eve/src/tool-stubs/validate-targets.ts@@ -25,12 +25,10 @@ function hasToolPath(graph: ResolvedAgentGraphBundle, path: readonly string[]):if (index === path.length - 1) {if (tool !== undefined) return tool.behavior?.handling?.kind !== "provider-tool";return (- // The reserved connection resolver only supplies the two framework wrappers.node.agent.dynamicToolResolvers.some((resolver) =>- resolver.slug !== "connection_tools" &&- (resolver.extensionNamespace === undefined ||- name.startsWith(`${resolver.extensionNamespace}__`)),+ resolver.extensionNamespace === undefined ||+ name.startsWith(`${resolver.extensionNamespace}__`),) ||node.subagentRegistry.dynamicResolvers.some((resolver) => resolver.name === name) ||(name.includes("__") &&diff --git a/packages/eve/src/tools/behavior.ts b/packages/eve/src/tools/behavior.tsindex 1e704207d..464306a1a 100644--- a/packages/eve/src/tools/behavior.ts+++ b/packages/eve/src/tools/behavior.ts@@ -22,7 +22,6 @@ export interface ToolExecutionShape {export interface CompiledToolBehavior {readonly availability: readonly ToolAvailabilityCondition[];readonly handling?: CompiledToolHandling;- readonly presentation?: "load-skill";readonly shape?: ToolExecutionShape;}@@ -58,7 +57,6 @@ export type PreparedToolHandling =export interface PreparedToolBehavior {readonly availability: readonly ToolAvailabilityCondition[];readonly handling?: PreparedToolHandling;- readonly presentation?: CompiledToolBehavior["presentation"];}// Framework definitions and the compiler may come from different bundled copies.diff --git a/packages/eve/src/tools/definition.ts b/packages/eve/src/tools/definition.tsindex 009163572..4f56af3a1 100644--- a/packages/eve/src/tools/definition.ts+++ b/packages/eve/src/tools/definition.ts@@ -33,6 +33,11 @@ export type ToolExecuteFn<TInput = unknown, TOutput = unknown> = (interface ToolDefinitionBase {/** Whether delegated agent sessions receive this tool. Defaults to `true`. */readonly availableInSubagents?: boolean;+ /**+ * Keeps this tool out of the model's tool list. The model finds it with+ * `eve__search` and calls it with `eve__tool`. Defaults to `false`.+ */+ readonly deferred?: boolean;readonly description: string;}diff --git a/packages/eve/src/tools/dynamic.ts b/packages/eve/src/tools/dynamic.tsindex 91a010a6a..8b8bffc5a 100644--- a/packages/eve/src/tools/dynamic.ts+++ b/packages/eve/src/tools/dynamic.ts@@ -13,7 +13,8 @@ import type { ToolModelOutput } from "#tools/model-output.js";** Identity comes from context: a single returned entry is named after* the file slug; entries in a returned `Record<string, DynamicToolEntry>`- * are each named `slug__key`.+ * are each named by their bare key, prefixed with the mount namespace for an+ * extension's resolver.** `TInput` defaults to `Record<string, unknown>` but is inferred when* `inputSchema` is a Standard Schema (e.g. Zod) via the `defineTool`@@ -23,6 +24,8 @@ import type { ToolModelOutput } from "#tools/model-output.js";// eslint-disable-next-line @typescript-eslint/no-explicit-anyexport interface DynamicToolEntry<TInput = Record<string, unknown>, TOutput = any> {readonly availableInSubagents?: boolean;+ /** Keeps this entry out of the model's tool list, as `deferred` on `defineTool`. */+ readonly deferred?: boolean;readonly label?: ToolLabelDefinition<TInput, TOutput>;readonly description: string;/**@@ -58,7 +61,7 @@ export type DynamicToolSet = Readonly<Record<string, DynamicToolEntry<any, any>>/*** Return type for a `defineDynamic` event handler: a single tool entry- * (named after the file slug), a map of entries (named `slug__key`), or+ * (named after the file slug), a map of entries (named by their keys), or* `null` for no tools.*/// eslint-disable-next-line @typescript-eslint/no-explicit-anydiff --git a/packages/eve/src/tools/framework/connection-tools.ts b/packages/eve/src/tools/framework/connection-tools.tsdeleted file mode 100644index 421492213..000000000--- a/packages/eve/src/tools/framework/connection-tools.ts+++ /dev/null@@ -1,11 +0,0 @@-import { defineDynamic } from "#dynamic/definition.js";-import { resolveConnectionTools } from "#execution/tools/connection-tools.js";--/** Provides `connection_search` and `connection_execute` while the agent has connections. */-export const connectionTools = defineDynamic({- events: {- "step.started": resolveConnectionTools,- },-});--export default connectionTools;diff --git a/packages/eve/src/tools/provided/load-skill.ts b/packages/eve/src/tools/provided/load-skill.tsdeleted file mode 100644index cdca58c47..000000000--- a/packages/eve/src/tools/provided/load-skill.ts+++ /dev/null@@ -1,37 +0,0 @@-import {- executeLoadSkill,- SKILL_INPUT_SCHEMA,- SKILL_OUTPUT_SCHEMA,-} from "#execution/tools/load-skill.js";-import { displayTitle } from "#shared/display-name.js";-import { defineTool } from "#tools/definition.js";-import { attachToolBehavior } from "#tools/behavior.js";--export const loadSkill = attachToolBehavior(- frameworkTool(- defineTool({- label: {- start: (input) => {- const skill =- typeof input === "object" && input !== null ? Reflect.get(input, "skill") : undefined;- return typeof skill === "string" && skill.trim() !== ""- ? `Load skill: ${displayTitle(skill.trim())}`- : "Load skill";- },- },- description: [- "Load the full instructions for one available skill by name or id.",- "Use this tool when the request clearly matches a listed skill description or when the user explicitly asks for that skill.",- "Loading adds the skill instructions to the current turn.",- 'Choose the "skill" value from the Available skills block.',- ].join(" "),- execute: executeLoadSkill,- inputSchema: SKILL_INPUT_SCHEMA,- outputSchema: SKILL_OUTPUT_SCHEMA,- }),- ),- { availability: [], presentation: "load-skill" },-);--export default loadSkill;-import { frameworkTool } from "./framework-tool.js";diff --git a/packages/eve/src/tools/provided/task-cancel.ts b/packages/eve/src/tools/provided/task-cancel.tsindex ca28ada64..a5bfcfa79 100644--- a/packages/eve/src/tools/provided/task-cancel.ts+++ b/packages/eve/src/tools/provided/task-cancel.ts@@ -10,7 +10,7 @@ export interface TaskCancelInput {readonly taskId: string;}-/** `task_cancel`, offered to agents that can start tasks; the session answers each call. */+/** `eve__task_cancel`, offered to agents that can start tasks; the session answers each call. */export const taskCancelTool: HarnessToolDefinition = {frameworkTool: true,description: TASK_CANCEL_DESCRIPTION,diff --git a/packages/eve/src/tools/provided/task-wait.ts b/packages/eve/src/tools/provided/task-wait.tsindex 23aade637..ac6272e6c 100644--- a/packages/eve/src/tools/provided/task-wait.ts+++ b/packages/eve/src/tools/provided/task-wait.ts@@ -8,7 +8,7 @@ export interface TaskWaitInput {}/**- * `task_wait`, offered to agents that can start tasks. It is not a workflow tool: the+ * `eve__task_wait`, offered to agents that can start tasks. It is not a workflow tool: the* call defers out of the model step and the session parks the turn itself.*/export const taskWaitTool: HarnessToolDefinition = {diff --git a/packages/eve/src/tools/schema.ts b/packages/eve/src/tools/schema.tsindex 476b8659a..956b507d1 100644--- a/packages/eve/src/tools/schema.ts+++ b/packages/eve/src/tools/schema.ts@@ -150,6 +150,28 @@ export function defineJsonSchema<T = unknown>(/** Schemas built by {@link defineJsonSchema}, which reach the model exactly as written. */const plainJsonSchemas = new WeakSet<object>();+/**+ * A JSON Schema advertised exactly as written whose accepted values `refine`+ * then checks further, rejecting them or returning the value to use.+ */+export function refineJsonSchema(+ schema: JsonObject,+ refine: (value: unknown) => Promise<StandardSchemaV1.Result<unknown>>,+): ToolSchema {+ const base = defineJsonSchema(schema);+ const refined = {+ "~standard": {+ ...base["~standard"],+ validate: async (value: unknown) => {+ const result = await base["~standard"].validate(value);+ return result.issues === undefined ? await refine(result.value) : result;+ },+ },+ } as ToolSchema;+ plainJsonSchemas.add(refined);+ return refined;+}+/** An optional string property added to an input schema. */export interface OptionalStringProperty {readonly description: string;diff --git a/packages/eve/src/runtime/connections/tool-signature.test.ts b/packages/eve/src/tools/signature.test.tssimilarity index 98%rename from packages/eve/src/runtime/connections/tool-signature.test.tsrename to packages/eve/src/tools/signature.test.tsindex d56b97c83..bf7fb79cf 100644--- a/packages/eve/src/runtime/connections/tool-signature.test.ts+++ b/packages/eve/src/tools/signature.test.ts@@ -1,6 +1,6 @@import { describe, expect, it } from "vitest";-import { renderToolSignature } from "./tool-signature.js";+import { renderToolSignature } from "./signature.js";describe("renderToolSignature", () => {it.each([diff --git a/packages/eve/src/runtime/connections/tool-signature.ts b/packages/eve/src/tools/signature.tssimilarity index 98%rename from packages/eve/src/runtime/connections/tool-signature.tsrename to packages/eve/src/tools/signature.tsindex 313b923ba..a65db2bf6 100644--- a/packages/eve/src/runtime/connections/tool-signature.ts+++ b/packages/eve/src/tools/signature.ts@@ -1,6 +1,6 @@/**- * Renders a connection tool's input and output JSON Schemas as a compact- * TypeScript signature, e.g.+ * Renders a tool's input and output JSON Schemas as a compact TypeScript+ * signature, e.g.* `list_issues(input: { teamId: string; first?: number /* maximum: 50 *\/ }): Promise<unknown>`.** Constraints the model must respect (`minimum`, `maximum`, `pattern`, …) staydiff --git a/packages/eve/src/tools/tool-label.test.ts b/packages/eve/src/tools/tool-label.test.tsindex 944a7c303..ca27ca341 100644--- a/packages/eve/src/tools/tool-label.test.ts+++ b/packages/eve/src/tools/tool-label.test.ts@@ -3,7 +3,6 @@ import { describe, expect, it } from "vitest";import { bash } from "#tools/provided/bash.js";import { glob } from "#tools/provided/glob.js";import { grep } from "#tools/provided/grep.js";-import { loadSkill } from "#tools/provided/load-skill.js";import { readFile } from "#tools/provided/read-file.js";import { webFetch } from "#tools/provided/web-fetch.js";import { resolveWebSearchActivityLabel } from "#harness/provider-tool-schemas.js";@@ -16,9 +15,6 @@ describe("provided tool labels", () => {expect(grep.label?.start({ pattern: "slackActivityMessage" })).toBe("Search slackActivityMessage",);- expect(loadSkill.label?.start({ skill: "technical-writing" })).toBe(- "Load skill: Technical writing",- );expect(readFile.label?.start({ filePath: "channels/slack/activity.ts" })).toBe("Read channels/slack/activity.ts",);diff --git a/packages/eve/src/tools/workflow-definition.ts b/packages/eve/src/tools/workflow-definition.tsindex 1068f4414..5389332bf 100644--- a/packages/eve/src/tools/workflow-definition.ts+++ b/packages/eve/src/tools/workflow-definition.ts@@ -104,7 +104,7 @@ export type WorkflowToolContext = WorkflowSharedContext &Pick<ToolContext, "callId"> & {/*** Aborts when the call's work should stop: the call is cancelled, by- * `task_cancel`, `session.cancel()`, a failed turn, or the end of the+ * `eve__task_cancel`, `session.cancel()`, a failed turn, or the end of the* session, or, for an `execute` call, a steering message arrives while the* turn waits on it. Stop and return what you have. After a steering* message the call settles with what the body returns, or withdiff --git a/packages/eve/src/tracing/agent-otel-provider.test.ts b/packages/eve/src/tracing/agent-otel-provider.test.tsindex 0e8ec64db..f4ca74c7e 100644--- a/packages/eve/src/tracing/agent-otel-provider.test.ts+++ b/packages/eve/src/tracing/agent-otel-provider.test.ts@@ -2066,7 +2066,7 @@ describe("createAgentOtelInstrumentation", () => {idempotencyKey: uncorrelatedToolKey,input: {},scope,- toolName: "final_output",+ toolName: "eve__reply",type: "tool.call",},() => Promise.resolve("done"),@@ -2080,7 +2080,7 @@ describe("createAgentOtelInstrumentation", () => {const spans = runtime.exporter.getFinishedSpans();const tool = byName(spans, "execute_tool weather")[0]!;- const uncorrelatedTool = byName(spans, "execute_tool final_output")[0]!;+ const uncorrelatedTool = byName(spans, "execute_tool eve__reply")[0]!;expect(tool.parentSpanContext?.spanId).toBe(byName(spans, "agent.step")[0]!.spanContext().spanId,);diff --git a/packages/eve/src/tracing/agent-telemetry-contract.integration.test.ts b/packages/eve/src/tracing/agent-telemetry-contract.integration.test.tsindex 81fec5364..fa27bc804 100644--- a/packages/eve/src/tracing/agent-telemetry-contract.integration.test.ts+++ b/packages/eve/src/tracing/agent-telemetry-contract.integration.test.ts@@ -48,12 +48,8 @@ import { buildConversationItems } from "#cli/dev/tui/traces/trace-conversation.jimport { contentFilteringProcessor } from "#tracing/content-span-processor.js";import { ConversationContextKey } from "#shared/conversation-context.js";import { captureLogRecords } from "#internal/testing/log-records.js";-import { ConnectionRegistryKey } from "#context/providers/connection-key.js";-import { resolveConnectionTools } from "#execution/tools/connection-tools.js";-import { emitNestedToolActions } from "#harness/nested-actions.js";import { createInstrumentationHandleEvent } from "#instrumentation/native-events.js";import { createActionResultEvent, createActionsRequestedEvent } from "#protocol/message.js";-import type { JsonValue } from "#shared/json.js";import { createPresentedRuntimeActionRequestFromToolCall } from "#harness/action-presentation.js";import { taskWaitTool } from "#tools/provided/task-wait.js";import { taskCancelTool } from "#tools/provided/task-cancel.js";@@ -136,161 +132,6 @@ function scopeFor(sessionId: string, audience: "public" | "private"): Instrument}describe("exported agent telemetry contract", () => {- it.each(["public", "private"] as const)(- "parents nested connector actions and preserves resolved arguments under the %s content policy",- async (audience) => {- const runtime = createRuntime();- const ctx = contextFor(audience);- const scope = scopeFor("parent", audience);- const hooks = runtime.hooks.forTrace!(traceContext("parent", audience));- const executeTool = vi.fn(async () => ({ structuredContent: { title: "Issue" } }));- ctx.set(ConnectionRegistryKey, {- dispose: async () => {},- getConnectionApproval: () => undefined,- getConnectionNames: () => ["linear"],- getConnections: () => [- {- connectionName: "linear",- protocol: "mcp",- url: "https://linear.example.com/mcp",- description: "Issues",- sourceId: "linear",- sourceKind: "module",- logicalPath: "connections/linear",- },- ],- getClient: () => ({- close: async () => {},- connect: async () => {},- executeTool,- getToolMetadata: async () => [- {- name: "get_issue",- description: "Read an issue",- inputSchema: {- type: "object",- properties: {- id: { type: "string" },- includeRelations: { type: "boolean", default: false },- },- required: ["id"],- },- },- ],- }),- });- const accepted = vi.fn(async () => {});- const handleEvent = createInstrumentationHandleEvent({- getAttemptScope: () => scope,- handleEvent: accepted,- hooks,- sessionId: "parent",- })!;- const input = { connection: "linear", tool: "get_issue", input: { id: "ISSUE-1" } };- const toolKey = toolCallIdempotencyKey(scope, "call-1", 0);- try {- await contextStorage.run(ctx, async () => {- await bindInstrumentationRuntime(runtime, ctx, {- agentName: "parent",- rootSessionId: "parent",- sessionId: "parent",- })!.preparePreamble({ sequence: 0, sessionStarted: false, turnId: "turn_0" });- await hooks.publish({- type: "step.attempt.started",- idempotencyKey: attemptIdempotencyKey(scope),- scope,- operation: { modelId: "test", operationId: "ai.streamText", provider: "test" },- });- await handleEvent(- createActionsRequestedEvent({- actions: [- { callId: "call-1", kind: "tool-call", toolName: "connection_execute", input },- ],- sequence: 0,- stepIndex: 0,- turnId: "turn_0",- }),- );- const output = await runtime.runInContext(- {- idempotencyKey: toolKey,- scope,- type: "tool.call",- callId: "call-1",- toolName: "connection_execute",- input,- },- () =>- resolveConnectionTools()!.connection_execute!.execute!(input, {- callId: "call-1",- messages: [],- } as never),- );- await emitNestedToolActions(- handleEvent,- { sequence: 0, stepIndex: 0, turnId: "turn_0" },- "call-1",- );- await handleEvent(- createActionResultEvent({- result: {- callId: "call-1",- kind: "tool-result",- toolName: "connection_execute",- output: output as JsonValue,- },- sequence: 0,- stepIndex: 0,- turnId: "turn_0",- }),- );- await hooks.publish({- type: "step.attempt.completed",- idempotencyKey: attemptIdempotencyKey(scope),- scope,- });- });- await runtime.forceFlush();- expect(executeTool.mock.calls[0]).toEqual([- "get_issue",- { id: "ISSUE-1", includeRelations: false },- { abortSignal: undefined, callId: "call-1" },- ]);- expect(accepted.mock.calls).toHaveLength(4);- const actions = runtime.exporter- .getFinishedSpans()- .filter((span) => span.attributes["gen_ai.operation.name"] === "execute_tool");- expect(actions).toHaveLength(2);- const outer = actions.find((span) => span.attributes["gen_ai.tool.call.id"] === "call-1")!;- const nested = actions.find(- (span) => span.attributes["gen_ai.tool.call.id"] === "call-1:1",- )!;- expect(outer.attributes["agent.action.kind"]).toBe("tool-call");- expect(nested.parentSpanContext?.spanId).toBe(outer.spanContext().spanId);- expect(nested.attributes).toMatchObject({- "agent.action.kind": "tool-call",- "agent.action.parent_call_id": "call-1",- "agent.action.name": "linear__get_issue",- });- const tool = runtime.exporter- .getFinishedSpans()- .find((span) => span.name === "execute_tool connection_execute")!;- expect(tool.spanContext().spanId).toBe(outer.spanContext().spanId);- expect(nested.attributes["gen_ai.tool.call.arguments"]).toBe(- audience === "public" ? '{"id":"ISSUE-1","includeRelations":false}' : undefined,- );- expect(tool.attributes["gen_ai.tool.call.arguments"]).toBe(- audience === "public" ? JSON.stringify(input) : undefined,- );- const metadataNested = runtime.metadata- .getFinishedSpans()- .find((span) => span.attributes["gen_ai.tool.call.id"] === "call-1:1")!;- expect(metadataNested.attributes).not.toHaveProperty("gen_ai.tool.call.arguments");- } finally {- await runtime.shutdown();- }- },- );it.each([{tool: taskWaitTool,@@ -363,7 +204,7 @@ describe("exported agent telemetry contract", () => {})!;await binding.instrumentTaskToolCall({callId: "control",- toolName: tool.name as "task_wait" | "task_cancel",+ toolName: tool.name as "eve__task_wait" | "eve__task_cancel",startedAtMs,completedAtMs,input,diff --git a/packages/eve/src/tracing/agent-tool-instrumentation.ts b/packages/eve/src/tracing/agent-tool-instrumentation.tsindex 8b5250e46..0d1634e4c 100644--- a/packages/eve/src/tracing/agent-tool-instrumentation.ts+++ b/packages/eve/src/tracing/agent-tool-instrumentation.ts@@ -125,15 +125,10 @@ export function createAgentToolInstrumentation(input: {name: event.toolName,toolAttributes: { "agent.tool.is_framework": event.frameworkTool === true },parent: {- spanId: input.idGenerator.deriveSpanId(- event.parentCallId === undefined- ? attemptIdempotencyKey(event.scope)- : `action:${actionIdempotencyKey(event.scope.sessionId, event.scope.turnId, event.parentCallId)}`,- ),+ spanId: input.idGenerator.deriveSpanId(attemptIdempotencyKey(event.scope)),traceFlags: traceContext.traceFlags,traceId: traceContext.traceId,},- parentCallId: event.parentCallId,rootSessionId: event.scope.rootSessionId ?? event.scope.sessionId,traceSessionId: traceSessionIdOf(event.scope),sessionId: event.scope.sessionId,@@ -204,9 +199,6 @@ export function createAgentToolInstrumentation(input: {attributes: {"agent.action.kind": state.kind,"agent.action.name": state.name,- ...(state.parentCallId === undefined- ? undefined- : { "agent.action.parent_call_id": state.parentCallId }),"gen_ai.operation.name": "execute_tool","gen_ai.tool.call.id": state.callId,"gen_ai.tool.name": state.name,diff --git a/packages/eve/src/tracing/agent-trace-context-codec.ts b/packages/eve/src/tracing/agent-trace-context-codec.tsindex 15ee140ec..1dac78e93 100644--- a/packages/eve/src/tracing/agent-trace-context-codec.ts+++ b/packages/eve/src/tracing/agent-trace-context-codec.ts@@ -177,7 +177,6 @@ function deserializeAction(value: unknown): AgentActionTraceState | undefined {kind: value.kind,name: value.name,parent: value.parent,- parentCallId: typeof value.parentCallId === "string" ? value.parentCallId : undefined,rootSessionId: value.rootSessionId,traceSessionId: decodeTraceSessionId(value),sessionId: value.sessionId,diff --git a/packages/eve/src/tracing/agent-trace-state.ts b/packages/eve/src/tracing/agent-trace-state.tsindex 1e151c1dd..82189757e 100644--- a/packages/eve/src/tracing/agent-trace-state.ts+++ b/packages/eve/src/tracing/agent-trace-state.ts@@ -61,7 +61,6 @@ export interface AgentActionTraceState {readonly kind: InstrumentationToolCallKind;readonly name: string;readonly parent: InstrumentationTraceContext;- readonly parentCallId?: string;readonly rootSessionId: string;readonly sessionId: string;readonly spanId: string;diff --git a/packages/eve/src/tracing/invoke-tool-parity.integration.test.ts b/packages/eve/src/tracing/invoke-tool-parity.integration.test.tsindex 5ad588cc1..8c2493259 100644--- a/packages/eve/src/tracing/invoke-tool-parity.integration.test.ts+++ b/packages/eve/src/tracing/invoke-tool-parity.integration.test.ts@@ -262,7 +262,14 @@ async function runInConversation(toolName: string, modelInput: string): Promise<prompt: "Use the tool.",stopWhen: stepCountIs(1),telemetry: attempt.telemetry,- tools: buildToolSet({ tools }),+ tools: buildToolSet({+ describe: (definition) => definition.description,+ resolve: (call) => {+ const definition = tools.get(call.toolName);+ return definition === undefined ? undefined : { call, definition };+ },+ tools,+ }),});const output = result.content.find((part) => part.type === "tool-result" || part.type === "tool-error",diff --git a/packages/eve/test/runtime-tool-registry.test.ts b/packages/eve/test/runtime-tool-registry.test.tsindex 4549ef057..221e1aac2 100644--- a/packages/eve/test/runtime-tool-registry.test.ts+++ b/packages/eve/test/runtime-tool-registry.test.ts@@ -169,14 +169,14 @@ describe("createRuntimeToolRegistry", () => {{tools: [createResolvedToolDefinition({- logicalPath: "tools/load-skill.ts",- name: "load_skill",- sourceId: "tools/load-skill.ts",+ logicalPath: "tools/search.ts",+ name: "search",+ sourceId: "tools/search.ts",}),],},{- reservedToolNames: ["load_skill"],+ reservedToolNames: ["search"],},),).rejects.toBeInstanceOf(RuntimeRegistryError);diff --git a/packages/eve/test/scenarios/nested-tool-stubs.scenario.test.ts b/packages/eve/test/scenarios/nested-tool-stubs.scenario.test.tsindex 7f54a1dcb..b4e16ce3d 100644--- a/packages/eve/test/scenarios/nested-tool-stubs.scenario.test.ts+++ b/packages/eve/test/scenarios/nested-tool-stubs.scenario.test.ts@@ -1,5 +1,6 @@import { expect, it } from "vitest";import { Client } from "../../src/client/client.js";+import { TASK_WAIT_TOOL_NAME } from "../../src/protocol/task-tools.js";import { useScenarioApp } from "../../src/internal/testing/scenario-app.js";import { startEveDev } from "./dev-server-harness.js";@@ -96,7 +97,7 @@ export default defineAgent({const completed = request.messages.find(message => message.role === "user" && message.text.startsWith("<task_result"));if (completed) return completed.text.slice(completed.text.indexOf(">") + 1, completed.text.lastIndexOf("</task_result>"));const result = request.toolResults.find(result => result.name === ${JSON.stringify(tool)});- if (result && ${tool !== "lookup"}) return { toolCalls: [{ name: "task_wait", input: {} }] };+ if (result && ${tool !== "lookup"}) return { toolCalls: [{ name: ${JSON.stringify(TASK_WAIT_TOOL_NAME)}, input: {} }] };if (result) return typeof result.output === "string" ? result.output : JSON.stringify(result.output);return { toolCalls: [{ name: ${JSON.stringify(tool)}, input: ${tool === "lookup" ? "{}" : '{ message: "Look up Alice\'s assigned tasks." }'diff --git a/packages/eve/test/scenarios/runtime-loaders.scenario.test.ts b/packages/eve/test/scenarios/runtime-loaders.scenario.test.tsindex 3926b10f4..3504231c6 100644--- a/packages/eve/test/scenarios/runtime-loaders.scenario.test.ts+++ b/packages/eve/test/scenarios/runtime-loaders.scenario.test.ts@@ -181,7 +181,7 @@ async function writeRuntimeLoaderSubagentFixture(agentRoot: string): Promise<voi].join("\n"),);await writeFile(- join(researcherRoot, "tools", "search.mjs"),+ join(researcherRoot, "tools", "lookup.mjs"),['import { createSearchResponse } from "../lib/search-response.mjs";',"",@@ -551,7 +551,7 @@ describe("runtime compiled artifact loaders", () => {]);const researcherModuleIds = Object.keys(moduleMap.nodes["subagents/researcher"]?.modules ?? {});expect(researcherModuleIds.filter((moduleId) => !moduleId.startsWith("eve:defaults:"))).toEqual(- ["sandbox/sandbox.mjs", "tools/search.mjs"],+ ["sandbox/sandbox.mjs", "tools/lookup.mjs"],);expect(researcherModuleIds.some((moduleId) => moduleId.startsWith("eve:defaults:"))).toBe(true);expect(researcherNode?.agent.instructions).toEqual([@@ -580,7 +580,7 @@ describe("runtime compiled artifact loaders", () => {).toBe(false);await expect(researcherNode?.agent.tools- .find((tool) => tool.name === "search")+ .find((tool) => tool.name === "lookup")?.execute?.({ query: "climate" }, { messages: [], toolCallId: "call_1" }),).resolves.toEqual({query: "climate",diff --git a/packages/eve/test/scenarios/tool-schemas-app-zod.scenario.test.ts b/packages/eve/test/scenarios/tool-schemas-app-zod.scenario.test.tsindex 4c0b06f93..2b1ea3056 100644--- a/packages/eve/test/scenarios/tool-schemas-app-zod.scenario.test.ts+++ b/packages/eve/test/scenarios/tool-schemas-app-zod.scenario.test.ts@@ -72,7 +72,7 @@ export default defineAgent({{ id: "authored", name: "tag_page", input: { id: "page-1", tags: { status: "done" } } },] };}- return { toolCalls: [{ id: "final", name: "final_output", input: {+ return { toolCalls: [{ id: "final", name: "eve__reply", input: {advertised: tools.find((tool) => tool.name === "file_pages")?.inputSchema,results: toolResults.map(({ id, isError, output }) => ({ id, isError, output })),} }] };diff --git a/packages/eve/test/scenarios/turn-cancellation.scenario.test.ts b/packages/eve/test/scenarios/turn-cancellation.scenario.test.tsindex ef18710eb..10b380a48 100644--- a/packages/eve/test/scenarios/turn-cancellation.scenario.test.ts+++ b/packages/eve/test/scenarios/turn-cancellation.scenario.test.ts@@ -79,7 +79,7 @@ const model = mockModel((request) => {if (message.includes("Use workflow exactly once")) {// The workflow tool runs as a task: after its receipt, wait on it.if (request.toolResults.some((entry) => entry.name === "workflow")) {- return { toolCalls: [{ name: "task_wait", input: {} }] };+ return { toolCalls: [{ name: "eve__task_wait", input: {} }] };}const localOnly = message.includes("local-sleeper only");return {diff --git a/packages/eve/test/tui-client/tui-subagents.ts b/packages/eve/test/tui-client/tui-subagents.tsindex 4c66367da..c74905372 100644--- a/packages/eve/test/tui-client/tui-subagents.ts+++ b/packages/eve/test/tui-client/tui-subagents.ts@@ -72,7 +72,7 @@ run({ app: "agent-tui-client", kind: "local-build" }, async (target) => {const finalSnapshot = screen.snapshot();// The model's own task bookkeeping never reaches the transcript.- for (const internal of ["task_wait", "task_cancel", "<task_result"]) {+ for (const internal of ["eve__task_wait", "eve__task_cancel", "<task_result"]) {if (finalSnapshot.includes(internal)) {throw new Error(`Final screen shows internal task text "${internal}":\n${finalSnapshot}`);}diff --git a/pnpm-lock.yaml b/pnpm-lock.yamlindex 0fdc2a311..a36c67fef 100644--- a/pnpm-lock.yaml+++ b/pnpm-lock.yaml@@ -968,6 +968,28 @@ importers:specifier: 'catalog:'version: 7.0.2+ e2e/fixtures/agent-deferred-tools:+ dependencies:+ '@eve-e2e/config':+ specifier: workspace:*+ version: link:../e2e-config+ '@workflow/world-postgres':+ specifier: 'catalog:'+ version: 5.0.3(@opentelemetry/[email protected])(@types/[email protected])(@upstash/[email protected])([email protected])([email protected])([email protected])+ eve:+ specifier: workspace:*+ version: link:../../../packages/eve+ zod:+ specifier: 'catalog:'+ version: 4.5.4+ devDependencies:+ '@types/node':+ specifier: 'catalog:'+ version: 24.13.3+ typescript:+ specifier: 'catalog:'+ version: 7.0.2+e2e/fixtures/agent-dynamic-model-session:dependencies:'@eve-e2e/config':@@ -1339,6 +1361,28 @@ importers:specifier: 'catalog:'version: 7.0.2+ e2e/fixtures/agent-tool-discovery:+ dependencies:+ '@eve-e2e/config':+ specifier: workspace:*+ version: link:../e2e-config+ '@workflow/world-postgres':+ specifier: 'catalog:'+ version: 5.0.3(@opentelemetry/[email protected])(@types/[email protected])(@upstash/[email protected])([email protected])([email protected])([email protected])+ eve:+ specifier: workspace:*+ version: link:../../../packages/eve+ zod:+ specifier: 'catalog:'+ version: 4.5.4+ devDependencies:+ '@types/node':+ specifier: 'catalog:'+ version: 24.13.3+ typescript:+ specifier: 'catalog:'+ version: 7.0.2+e2e/fixtures/agent-tool-stubs:dependencies:'@eve-e2e/config':diff --git a/research/connection-execute.md b/research/connection-execute.mdindex bfabf84cc..f2a1c5dec 100644--- a/research/connection-execute.md+++ b/research/connection-execute.md@@ -1,11 +1,19 @@---issue: TBD-status: proposed-last_updated: "2026-10-01"+status: superseded+last_updated: "2026-10-06"---# Cache-stable connection tools+Superseded by [deferred tools](./deferred-tools.md). `search` and `execute`+replace `connection_search` and `connection_execute` with one catalog that also+covers deferred tools, subagents, and skills. Connection tools keep their+`<connection>__<tool>` names, and connection calls no longer report nested+actions. The result format, sign-in, approval, instance+pinning, and cache invariants below carry over. The model surface below+records the shipped connection tools, not current guidance.+## Summary`connection_search` breaks the prompt cache. Every hit addsdiff --git a/research/deferred-tools.md b/research/deferred-tools.mdnew file mode 100644index 000000000..a515fd48c--- /dev/null+++ b/research/deferred-tools.md@@ -0,0 +1,1070 @@+---+issue: TBD+status: proposed+last_updated: "2026-10-08"+---++# Deferred tools and skills++## Summary++Agents tend to score better on evals when every tool and skill lives on the+main agent, so the model doesn't have to hand each specialized task to a+subagent. Large teams push that until the tool list and skill list fill the+context. A few hundred tool definitions cost tens of thousands of tokens on+every request, and the model starts picking the wrong tool. One popular MCP server alone,+Playwright, spends 13.7k tokens on 21 tools+([source](https://mariozechner.at/posts/2025-11-02-what-if-you-dont-need-mcp/)).++This doc proposes a catalog of deferred capabilities that the model reaches+through three fixed tools, `eve__search`, `eve__tool`, and `eve__skill`:++```ts+eve__search(opts: {+ query: string;+ limit?: number; // default 10, at most 50+}): Promise<{+ results: Array<+ | { tool: string; description: string; signature: string } // call with eve__tool+ | { skill: string; description: string; path?: string } // load with eve__skill+ >;+ unavailable?: Array<{ connection: string; error: string }>;+}>;++eve__tool(opts: { name: string; input?: object }): Promise<unknown>;++eve__skill(opts: { name: string }): Promise<string>;+```++- **One opt-in.**+ - A tool sets `deferred: true` on `defineTool` or `defineWorkflowTool`.+ - A subagent sets `tool: "deferred"` on its `defineAgent`.+ - A skill sets `deferred: true` in its `SKILL.md` frontmatter or on+ `defineSkill`.++ Each works for static and dynamic entries. Connection tools are always+ deferred.++- **`eve__search` and `eve__tool` replace `connection_search` and+ `connection_execute`, and `eve__skill` replaces `load_skill`.** There is one+ catalog for everything, and no `skill_search`.+- **One dispatch path.** `eve__tool` and `eve__skill` look the entry up and+ run it exactly like a direct call:+ - an inline tool runs inline;+ - a workflow tool starts its run;+ - a subagent starts its child session;+ - a connection tool calls its connection;+ - `eve__skill` loads a skill, deferred or not, with today's `load-skill`+ action.+- **Registered from declarations.** Each tool comes with what it reaches:+ `eve__tool` with a deferred tool or agent, a connection, or a dynamic tool,+ subagent, or connection resolver; `eve__skill` with any skill, static or+ dynamic; and `eve__search` with anything it could find, which is whatever+ `eve__tool` calls plus deferred skills and dynamic skill resolvers. An agent+ with only listed skills gets `eve__skill` alone, `main`'s `load_skill`+ renamed; with none of these, no catalog tools. The rule reads what the agent+ declares, never what its resolvers return, so entries, static or dynamic,+ can join or leave the catalog at any step without changing the `tools`+ array.+- **Cache stable.** Catalog entries never enter the `tools` array, the system+ prompt, or a tool description. They are announced through one append-only+ listing, under the invariants of+ [cache-stable connection tools](./connection-execute.md).+- **Ready for code mode.** Code mode will be a separate tool,+ `eve__execute({ code })`, built on the workflow program runtime, with each+ call dispatched through the catalog. The `eve` namespace reserves the name.++```ts+export default defineTool({+ description: "Refund a paid Stripe invoice.",+ deferred: true,+ inputSchema: z.object({ invoiceId: z.string() }),+ async execute({ invoiceId }) {+ /* ... */+ },+});+```++```md title="agent/skills/pdf-forms/SKILL.md"+---+description: Fill and validate PDF forms.+deferred: true+---+```++This supersedes the model surface in `connection-execute.md`: the two+connection tools, their nested actions, and the connection listing. The+connection result format, sign-in, approval, and instance pinning carry over.++## Current state++- **Every tool is advertised.** Each static tool and each subagent tool the+ session can use goes into the provider `tools` array on every step+ (`harness/advertised-tools.ts`). The only filters are subagent visibility+ (`availableInSubagents`, `rootOnly`) and `tool: false`.+- **The harness already knows each tool's kind.** Every entry in the+ `HarnessToolMap` carries `behavior.handling` (`tools/behavior.ts`):+ - `dispatch` to `workflow-tool-call`, `subagent-call`,+ `remote-agent-call`, or `self-agent-call`;+ - `provider-tool`;+ - nothing, for a plain tool with an inline `execute`.++ Coordination reads it to decide whether a call runs inline, starts a+ workflow run, or starts a child session (`harness/coordination.ts`).+ Dynamic subagents join the same map each step+ (`buildHarnessToolsWithDynamicSubagents`).++- **Connection tools are the exception.** They aren't harness tools.+ `connection_execute` (`execution/tools/connection-tools.ts`) is a+ `defineTool` closure that calls the connection and reports a nested+ `<connection>__<tool>` action. That pattern can't reach workflow tools or+ subagents, because the harness has to see their call to park the turn and+ start the run.+- **Connections already have the cache machinery this doc reuses.**+ - The listing is a `context.state` announcement+ (`execution/connection-announcement.ts`, `harness/announcements.ts`).+ - Search returns TypeScript signatures+ (`runtime/connections/tool-signature.ts`), ranked by+ `execution/tools/connection-search-rank.ts`.+- **Every skill is listed.**+ - Static skills appear in the system prompt's "Available skills" section,+ with name, description, and file path (`formatAvailableSkillsSection` in+ `execution/skills/instructions.ts`).+ - Dynamic skills are re-announced as a full `context.state` list whenever+ they change (`context/dynamic-skill-lifecycle.ts`).+ - The model loads one with `load_skill({ skill })`, a framework action+ (`frameworkAction: "load-skill"`) that returns the skill's markdown.+ `load_skill` exists only when the agent declares skills.+- **Name clash.** Inside the harness, "deferred tool" currently means a+ workflow-backed tool, as in `Deferred tool "…" has no workflow.`+ (`harness/coordination.ts`, `harness/tool-loop.ts`,+ `execution/tasks/tool-entry-point.ts`). That internal term gets renamed to+ "workflow tool" (8 occurrences).++## Prior art++### AI SDK++Read in the versions eve pins: `[email protected]`, `@ai-sdk/[email protected]`,+`@ai-sdk/[email protected]`, and `@ai-sdk/[email protected]`.++- **`deferLoading` and `toolSearch()`.** `BaseTool.deferLoading` hides a tool+ until `toolSearch()` finds it. `createToolSearchState` (`ai/dist/index.js`)+ keeps a `discovered` set in memory for one generation. Each step filters+ `activeTools` down to tools that aren't deferred plus the ones discovered so+ far. Search is word overlap on name and description, top five results, and+ returns `{ name, description }` with no schema.+- **It breaks the cache.** A tool the model finds is added to `tools` on the+ next step, and the whole cached prefix is lost. The only cache-stable path+ is code mode with `toolDiscovery: 'conversation'`. It also needs every+ definition before the generation starts and drops discoveries when the+ generation ends, which doesn't fit eve's durable step-by-step execution.+- **Provider-native search.**+ - Anthropic: `toolSearchRegex_20251119` and `toolSearchBm25_20251119` with+ `providerOptions.anthropic.deferLoading`. The `toolChanges` option on+ mid-conversation system messages also adds tools without losing the cache+ (beta `mid-conversation-tool-changes-2026-07-01`).+ - OpenAI: `openai.tools.toolSearch()` with+ `providerOptions.openai.deferLoading` and tool namespaces.+ - In all of these the provider runs discovery and the model calls the tool+ directly by name. All of it is specific to one provider and some of it is+ beta.++### opencode++Sources:+[`v2@b5c43a4`](https://github.com/anomalyco/opencode/tree/b5c43a4) for the+current tool names, and the earlier+[`v2@6bffe79`](https://github.com/anomalyco/opencode/tree/6bffe7932efa9adc36b74e389598daecb4b17ac1)+for catalog delivery, read but not run. Neither has `deferLoading`,+`tool_search`, or provider-native search.++- **Code mode by default.** Tools set `codemode: false` to stay native: shell,+ read, edit, `skill({ id })`, `subagent`, and MCP servers configured that+ way. Everything else is reachable only through `execute({ code })`, whose+ programs call `search()` and `tools.<ns>.<tool>()`. There is no tool for a+ single call outside code, and calling a code-mode tool by name fails with+ "No tool named … is currently available".+- **Fixed description, appended catalog.** The `execute` description is+ fixed. The catalog arrives as an instruction baseline followed by appended+ diffs (`codemode/instructions.ts`, `session/instruction-state.ts`). Each+ child call runs the same permission check as a direct call (`tool/mcp.ts`).+- **What carries over:**+ - one catalog over local and MCP tools;+ - skills loaded by their own tool, by id;+ - `execute` as the name of the code tool, with `search()` inside programs;+ - an append-only catalog;+ - the same permission checks whichever way a tool is called.++### pi++Sources:+[`earendil-works/pi@6fb2e78`](https://github.com/earendil-works/pi/tree/6fb2e78)+for the current tool names (the repository moved from `badlogic/pi-mono`), and+the earlier+[`pi-mono@636703a`](https://github.com/badlogic/pi-mono/tree/636703a0a4f2f4d8558d08f2308cb41109585bf5)+for exposure and caching. Built-in MCP, `tool_search`, and code mode shipped in+0.99.0, which reverses pi's earlier "no MCP" position.++- **Per-tool exposure.** `ToolExposure` is `direct`, `model-only`,+ `codemode`, `deferred`, or `hidden`. MCP tools default to `codemode`, run+ through `codemode({ code })`. The `deferred` exposure routes a tool through+ `tool_search`.+- **Search activates matches.** `tool_search({ query, limit? })`, with a+ default `limit` of 8, uses BM25 over name, description, schema text, and+ namespace, and adds its matches to the active tools. The model then calls+ them directly by name. To keep the cache, pi depends on provider support:+ - Anthropic: inline `tool_addition` blocks plus a `defer_loading`+ placeholder tool. Without the placeholder, pi measured a full cache miss on+ the first tool change.+ - OpenAI: `additional_tools`, or a synthetic `tool_search_call`.+ - Other providers: pi's own docs say the prefix can be invalidated.+- **Fixed search description.** It never lists the searchable tools, so it+ doesn't change as servers connect.+- **No skill tool.** Skills are listed by name, description, and path in the+ system prompt and read with `read`. They aren't searchable.+- **Community adapters.** `pi-mcp-adapter` uses a single proxy tool,+ `mcp({ search })` then `mcp({ tool, args })`.++### What we take++| Question | AI SDK | opencode v2 | pi | This design |+| -------------------------- | ------------------------- | ------------------------ | ----------------------------------- | --------------------------------------- |+| Opt-in | `deferLoading` | `codemode` (default on) | `exposure: "deferred"` | `deferred: true` |+| One catalog with MCP | No | Yes | Yes | Yes |+| Search | `toolSearch()`, top 5 | `search()` inside code | `tool_search({ query, limit? })`, 8 | `eve__search({ query, limit? })`, 10 |+| How a found tool is called | Directly, next step | `tools.x()` in `execute` | Directly, next step | `eve__tool({ name, input })`, same step |+| Skills | None | `skill({ id })` | `read` on `SKILL.md` | `eve__skill({ name })` |+| Code mode | `toolDiscovery` option | `execute({ code })` | `codemode({ code })` | Later, `eve__execute({ code })` |+| `tools` array mid-session | Grows | Fixed | Grows, except on 2 providers | Fixed |+| Catalog delivery | Search result only | Baseline + diffs | Search result only | Baseline + diffs |+| Discovery state | In memory, one generation | None | Session transcript | None; search is stateless |++## Design++```text+ tools array (fixed for every session)+ ┌──────────────────────────────────────────────┐+ │ direct tools... eve__search eve__tool eve__skill │+ └──────────────────────────────────────────────┘+messages: [system prompt, no catalog names]+ [context.state: catalog baseline]+ ... turns ...+ [context.state: catalog diff] ← append only+ ... compaction ...+ [context.state: new baseline]++eve__tool({ name, input }) ← model history only+ └─ resolve entry by name → dispatch as a direct call+ inline tool → execute+ workflow tool → durable run (foreground parks, background returns a task)+ subagent → child session (local, remote, or root copy)+ connection → connection client ("linear__list_issues")++eve__skill({ name }) ← model history only+ └─ resolve skill by name → load-skill action → the skill's markdown+```++### Authoring API++**Tools.** `deferred?: boolean` sits next to `availableInSubagents` in+`ToolDefinitionBase`, so `defineTool` and `defineWorkflowTool` both accept it.+It defaults to `false`.++```ts title="agent/tools/deploy_service.ts"+import { defineWorkflowTool } from "eve/tools";+import { always } from "eve/tools/approval";+import { z } from "zod";++export default defineWorkflowTool({+ description: "Review and deploy a service.",+ deferred: true,+ approval: always(),+ inputSchema: z.object({ service: z.string() }),+ async execute({ service }, ctx) {+ "use workflow";+ /* ... */+ },+});+```++**Subagents.** A subagent's `tool` setting already controls whether the+subagent appears as a tool. It widens from `boolean` to+`boolean | "deferred"`. This applies to `defineAgent`, remote agents,+workspace agents, and dynamic subagent configs.++```ts title="agent/subagents/billing_specialist/agent.ts"+export default defineAgent({+ description: "Resolve billing disputes and refunds.",+ model: "anthropic/claude-sonnet-4.6",+ tool: "deferred",+});+```++- **What each value means:** `true` is a direct tool, `"deferred"` is a+ catalog entry, and `false` is not a tool at all.+- **On the root agent,** `tool: "deferred"` defers the built-in `agent`+ self-delegation tool.+- **Why `tool` and not a separate flag.** A separate `deferred` flag on+ `defineAgent` would allow the contradiction `tool: false, deferred: true`.+- **`ctx.agent(name)` is unchanged.** It resolves from the full registry+ whatever `tool` says.++**Built-in and extension tools** are deferred by spreading them, the same way+they're overridden today:++```ts title="agent/tools/bash.ts"+import { defineTool } from "eve/tools";+import { bash } from "eve/tools/bash";++export default defineTool({ ...bash, deferred: true });+```++**Dynamic entries.** `DynamicToolEntry` accepts `deferred`, and dynamic+subagent configs accept `tool: "deferred"`. Teams use this for catalogs that+depend on the tenant or the user:++```ts title="agent/tools/tenant.ts"+import { defineDynamic } from "eve";+import { defineTool } from "eve/tools";++export default defineDynamic({+ events: {+ "session.started": async (_event, ctx) => {+ const actions = await loadTenantActions(ctx.session);+ return Object.fromEntries(+ actions.map((action) => [+ action.id,+ defineTool({+ description: action.description,+ deferred: true,+ inputSchema: action.inputSchema,+ execute: (input) => runTenantAction(action.id, input),+ }),+ ]),+ );+ },+ },+});+```++- **Same rules as direct dynamic entries.**+ - A map entry is named by its bare key, with the mount prefix added for an+ extension's resolver. Authors add their own prefix when they want one,+ such as `tenant__export`.+ - A dynamic tool still can't be a workflow tool.+ - An entry the resolver no longer returns can't be called: `eve__tool`+ reports it as unknown and suggests the closest names.+- **Keys are validated.** Every dynamic entry name, direct or deferred, must+ match `TOOL_SLUG_PATTERN`. Today a bad key fails only when the provider+ rejects it, and a deferred name never reaches a provider. Checking it when+ the resolver returns it means any entry that works deferred also works+ direct.+- **Prefer session- or turn-scoped resolvers.** A `step.started` resolver+ whose set of kinds or namespaces changes every step appends a listing every+ step. The cache stays intact, but history grows.++**Skills.** `deferred?: boolean` joins the skill definition. It defaults to+`false` and is set in any of three places:++```md title="agent/skills/pdf-forms/SKILL.md"+---+description: Fill and validate PDF forms.+deferred: true+---+```++```ts title="agent/skills/release_notes.ts"+import { defineSkill } from "eve/skills";++export default defineSkill({+ description: "Write release notes from merged pull requests.",+ deferred: true,+ markdown: "...",+});+```++```ts title="agent/skills/tenant.ts"+import { defineDynamic } from "eve";+import { defineSkill } from "eve/skills";++export default defineDynamic({+ events: {+ "session.started": async (_event, ctx) => {+ const playbooks = await loadTenantPlaybooks(ctx.session);+ return Object.fromEntries(+ playbooks.map((playbook) => [+ playbook.id,+ defineSkill({+ description: playbook.description,+ deferred: true,+ markdown: playbook.markdown,+ }),+ ]),+ );+ },+ },+});+```++- **What `deferred` changes.** A deferred skill leaves the system prompt's+ skill section and the dynamic skill announcement. It is found with+ `eve__search` and loaded with `eve__skill({ name })`. Everything else is the same+ as today: its markdown, its package files in the sandbox, its activation,+ and its eval facts.+- **Dynamic skills keep their rules.** They resolve on `session.started` and+ `turn.started` only. A map entry is named by its bare key, with the mount+ prefix added for an extension's resolver. A dynamic skill still overrides a+ same-named authored skill.+- **Names are validated.** Skill names, static and dynamic, must match+ `TOOL_SLUG_PATTERN`. There is no check today, and dynamic skill keys+ aren't checked at all.++**Connection tools** are always catalog entries. There is no switch to make+them direct; this matches what `connection_search` does today.++**Provider tools can't be deferred.** These are tools with+`behavior.handling.kind === "provider-tool"`, such as native web search. The+provider has to see their definition, so deferring one is a compile error.++### The catalog++Every entry has one flat name: the same name it has, or would have, as a+direct tool. `__` is the only namespace separator, as it is everywhere in eve+today.++| Entry | Source | Name | Lookup |+| ------------- | --------------------------------------------------------------------- | ----------------------------------------------- | -------------------------------------------------------------------------------------- |+| Inline tool | `deferred: true` on a static or dynamic tool with an inline `execute` | `refund_invoice`, `crm__search`, `tenant__sync` | Harness tools for the step, no dispatch handling |+| Workflow tool | `deferred: true` on `defineWorkflowTool` | `deploy_service` | Harness tools for the step, `workflow-tool-call` |+| Subagent | `tool: "deferred"` on a local, remote, dynamic, or root-copy agent | `billing_specialist`, `crm__reviewer` | Harness tools for the step, `subagent-call`, `remote-agent-call`, or `self-agent-call` |+| Connection | Every tool of every connection, after `tools.allow` and `tools.block` | `linear__list_issues`, `crm__api__list_issues` | The owning connection, then its `getToolMetadata()` |+| Skill | `deferred: true` on a static or dynamic skill | `pdf-forms`, `crm__playbook` | Authored skills and the dynamic skill manifest, under the `skill` key |++- **Owners.** A `__` prefix names the entry's owner:+ - an extension mount: `crm__search`;+ - a connection: `linear__list_issues`;+ - a memory slot: `<slot>__<key>`.++ Owners nest. An extension mounted as `crm` with `connections/api.ts`+ contributes connection `crm__api`, whose tools are `crm__api__list_issues`.+ A prefix an author adds by convention, such as `tenant__export`, has no+ owner.++- **A connection owns its name and its `__` prefix.** No tool, subagent,+ memory tool, extension contribution, or other connection may be named+ `linear` or start with `linear__` while connection `linear` exists.+ - **Only connections need the rule.** They are the only entries eve can't+ list up front, because listing their tools needs a network call and maybe+ a sign-in. Every other name is known, and exact duplicates are already+ errors.+ - **Checked on names alone.** eve never lists a connection's tools to check+ it.+ - **When it's checked.** Static names are checked at compile time. Dynamic+ connections, tools, subagents, and memory tools are checked when they+ resolve. The entry that introduces the conflict is rejected with an error+ naming both sides, for example:++ ```text+ Dynamic tool "linear__sync" starts with "linear__", which belongs to connection "linear". Rename the map key.+ ```++ - **Nested connections.** Connection `acme__crm` conflicts with connection+ `acme`. Without the rule, `acme__crm__list` could mean connection `acme`+ with tool `crm__list`, or connection `acme__crm` with tool `list`.+- **Routing.** `eve__tool({ name })` first looks for an exact match among the+ step's harness tools: authored and dynamic tools plus static and dynamic+ subagents, in the precedence direct calls use today. Otherwise it finds the+ connection whose `<name>__` starts `name`, and the rest is that+ connection's tool name, checked against its metadata.+ - The ownership rule allows at most one candidate, so the order of the+ checks never changes the result.+ - The rest of the name is never split again, so MCP tool names work+ whatever characters they contain.+- **Dispatch is decided before anything runs.** For harness entries, the+ dispatch handling is on the definition, which the harness already has. A+ name under a connection's prefix is always a connection call. So+ coordination can decide whether a call parks without any network call.+- **Deferring never renames.** The model sees the same name in its tool list,+ in `eve__search` results, in `eve__tool` input, and in history. All of these keep+ working whether a tool is direct or deferred: protocol action names,+ approvals and `approvalKey`, evals, hooks, `toolResultFrom`, display titles,+ and extension overrides such as `agent/tools/crm__search.ts`.+- **Namespaces are never parsed.** The ownership check compares names+ against connection names, and a connection entry knows its connection,+ which gives labels such as `Linear: List issues`. Nothing splits a name at+ `__`. Code mode will record owners on each entry for program paths such as+ `tools.crm.api.list_issues`; a convention prefix with no owner stays part+ of the name, as in `tools.tenant__export`.+- **Scope.** The catalog holds only entries advertised to the current+ session, so a child session never sees a tool with+ `availableInSubagents: false`.+- **Skills have their own key.** A skill is addressed as `{ skill }`, never+ `{ tool }`. Skills and tools keep separate name spaces, as they have today,+ so a skill can share a name with the tool it documents. The connection+ ownership rule applies only to `tool` names.++### Model surface++**`eve__search`**++- **Input:** `{ query, limit? }`.+ - `limit` defaults to 10 and is capped at 50. Results stay in history for+ the rest of the session, so the default stays small: at 20, results+ averaged about 660 tokens, and 15 of 32 benchmark queries returned all 20,+ mostly weak matches, while every expected tool ranked in the top 5.+ - `query` is required and must contain a word. Deferred entries are found+ by what they do or by namespace, not browsed. A connection's name as the+ query, such as `linear`, lists that connection's tools first.+ - **Namespace queries.** A query whose first word contains `__` searches one+ namespace: everything before its last `__`, with trailing underscores+ trimmed. The namespace only filters. `eve__search` keeps names under+ `<namespace>__` and anything named exactly the namespace, such as a+ connection's sign-in entry, then ranks them by the whole query. So+ `linear__list_issues` still finds the `linear` sign-in entry when Linear+ needs sign-in, and an exact name that contains `__` still ranks first.+ Connection ownership means no tool outside a connection uses its+ prefix, so `eve__search` lists only the connections that can own names in+ the namespace (`linear`, or `crm__api` under an extension mounted as+ `crm`). Other connections make no network call and don't appear in+ `unavailable`. A namespace that matches nothing fails with the closest+ connection names instead of returning an empty list. Characters that+ can't appear in a name, such as a leading `^`, are ignored, and regex+ isn't supported. The description tells the model to use this when it+ already knows a name or namespace, from the listing, a result, or an+ error.+ - There is no connection filter and no sign-in flag. The query already+ selects a connection, and sign-in is an `eve__tool` call.+ - There is no paging and no match count. `eve__search` returns the best matches+ up to `limit`, and the model narrows the query or raises `limit` for more.+ Paging and counts assume a fixed, countable match list, which a future+ search that asks a model to choose entries wouldn't have.+- **Async.** `eve__search` returns a promise, so the way it picks results can+ change, for example to a decision model, without changing its signature.+- **Result:** `{ results, unavailable? }`.+ - A tool entry is `{ tool, description, signature }`. `tool` is the exact+ name to pass to `eve__tool`.+ - `signature` is TypeScript rendered from the input and output schemas by+ `renderToolSignature`, which moves to a shared module. It is more compact+ than JSON Schema, includes the output type, and is what a code-mode+ program calls.+ - Results don't say whether a tool is inline, a workflow, a subagent, or a+ connection tool. The call is the same for all of them. Notes eve appends+ to a direct tool's description, such as the `endsTurn` sentence or+ background-task guidance, go at the end of `description` instead.+ - A skill entry is `{ skill, description, path? }`. `path` is the skill's+ `SKILL.md` under the skills root, present when the skill has package+ files. It is the same path the skill section shows for a skill that isn't+ deferred, so relative references inside the skill still resolve.+ - Tools and skills rank together, so one query such as "fill a PDF form"+ can return a `pdf_fill` tool and a `pdf-forms` skill.+ - A connection whose tools can't be listed until the user signs in appears+ as a tool result named after the connection, such as+ `{ tool: "linear", description: "Sign in to use the Linear tools: …" }`.+ - `unavailable` reports only connections that failed to list or can't+ start an interactive sign-in.+ - Listing a connection's tools stops waiting after 10 seconds. Every+ unscoped search lists every connection, so without a bound a search for+ a local tool would wait on the slowest server, and a hung one would stall+ the step. The connection is reported in `unavailable` with+ `"<connection>" did not list its tools within 10s. Try again later.`, and+ an `eve__tool` call on its tools fails with the same error. The+ bound covers listing only, not sign-in or tool calls. Clients share one+ in-flight listing and take no signal, so eve stops waiting instead of+ cancelling; a late result fills only that step's client, and the+ registry is rebuilt every step.+- **Ranking.** Matches rank in tiers, and an exact name always ranks first:+ 1. the query's words equal the entry's full name, such as+ `linear__create_issue`;+ 2. they equal the name after its last `__`, whatever the namespace, such+ as `create_issue` for `linear__create_issue` or `sre__create_issue`;+ 3. they start the entry's connection name and the full name, so `linear`+ returns the `linear` connection's tools;+ 4. they start the entry's full name or its name after the last `__`, with+ the last word allowed to be partial, such as `create_iss`;+ 5. keyword matches, weighted by field: the name, then the connection name,+ input property names, the description, and last property descriptions+ and the connection description. One-letter and filler words such as+ `a`, `to`, and `the` don't score when the query has other words, since+ they match nearly any name by prefix.++ A score breaks ties within a tier, then the connection name, then the+ entry name. A skill has only a name and a description to match. `eve__tool`'s and `eve__skill`'s+ closest-name suggestions use the same ranking.++ Measured on the `agent-tool-discovery` catalog plus two connections+ (Linear and GitHub) with 32 queries phrased as models send them, matching+ names after any namespace and ignoring filler words raised recall@1 from+ 27/32 to 29/32 and MRR from 0.907 to 0.939; recall@5 stayed at 32/32.++- **Sign-in.** `eve__search` never prompts. It lists a connection that needs+ sign-in as a result, and `eve__tool({ name: "<connection>" })` asks the user+ to sign in.+- **Description.** Fixed for each eve version. It names no entry. It says+ that `eve__search` finds the agent's own tools and services, not web pages, so+ the model doesn't use it in place of `web_search`.+- **Label.** People see `Search tools for “<query>”`, then+ `Searched tools for “<query>”, found <n>` when it returns, never the tool's+ name.++**`eve__tool`**++- **Input:** `{ name: string, input?: object }`. `input` defaults to `{}`.+- **Wire schema.** One flat object schema with `name` required. Skills have+ their own tool, so there is no `tool`-or-`skill` union to work around:+ Anthropic rejects `oneOf`, `anyOf`, and `allOf` at the top of a tool's+ input schema ([example](https://github.com/anthropics/claude-code/issues/4886)).+- **Any entry, no prior search.** It calls any catalog entry by name, whether+ or not it was searched for. There is no discovered set to persist or replay.+- **Validation.** eve checks `input` against the entry's input schema before+ dispatching, because the provider never saw that schema. Failures come back+ as data the model can act on:+ - An unknown name lists the closest tool names.+ - A skill's name says to load it with `eve__skill`.+ - A direct tool returns `"<name>" is in your tool list; call it directly.`+ - Invalid input returns the entry's `signature`.+- **Sign-in.** A connection's own name is an entry.+ `eve__tool({ name: "linear" })` finishes a pending sign-in or starts one, the+ same way a call to one of its tools does, and parks until the user signs+ in. It then says the connection's tools can be searched. When the tools+ are already listable, it only confirms. It has no approval, and its label+ is `Sign in to Linear`. Every path that finishes a sign-in, whether in+ `eve__search`, a tool call, or this entry, goes through one helper.+- **Description.** Fixed for a deployment: "Call a tool that isn't in your+ tool list by its exact name from eve__search, with `input` matching its+ signature." Only with connections, it adds the preference for connected+ services over web search or general knowledge, a sentence that moves over+ from `connection_search`.++**`eve__skill`**++- **Input:** `{ name: string }`, required.+- **Loads every skill.** It loads any skill the session can see, deferred or+ not, and replaces `load_skill`. An agent whose only catalog entries are+ listed skills gets `eve__skill` alone, which is `load_skill` renamed.+- **Validation.** An unknown name lists the closest skill names. A tool's+ name says to call it with `eve__tool`, or directly when it's in the tool+ list. A name that matches a connection says to find its tools with+ `eve__search({ query: "<connection>__" })`, a hint that moves over from+ `load_skill`.+- **Description.** `load_skill`'s trigger, fixed for a deployment: "Load a+ skill's instructions when the request clearly matches a listed skill or the+ user asks for it, then follow them." It mentions `eve__search` only when+ search can find skills. An audit found that a skill clause placed after a+ generic call-a-tool text left only the system prompt prompting skill loads.+ The skill section of the system prompt and the dynamic skill announcement+ tell the model to load listed skills the same way.++**All three**++- **Present from what the agent declares.** Each agent, root or subagent,+ decides from its own declarations, once per deployment:+ - `eve__tool` with a deferred tool or agent the session sees, a connection,+ or a dynamic tool, subagent, or connection resolver;+ - `eve__skill` with any skill, static or from a dynamic skill resolver;+ - `eve__search` with anything it could find: whatever `eve__tool` reaches,+ plus deferred skills and dynamic skill resolvers;+ - none otherwise, so such an agent's tool list is `main`'s minus+ `load_skill`.++ A rule like "present while the catalog is non-empty" would flip the array+ the first time a resolver returns a deferred entry; reading declarations+ keeps it fixed for the deployment, so parked approvals and restored steps+ see the same tools. No text points at a catalog tool the agent lacks.++- **Why not always.** The first design registered its catalog tools for every+ agent. A pre-merge audit measured about 290 extra tokens per request (net+ of the removed `load_skill`) for agents with nothing to find, where+ `eve__search` always returned `{ results: [] }` and the call tool described+ nothing the agent had. gpt-6-sol also serialized parallel tool calls+ whenever the catalog tools were present.+- **One reserved namespace.** Every tool eve adds at runtime is named+ `eve__<name>`: `eve__search`, `eve__tool`, `eve__skill`, `eve__task_wait`,+ `eve__task_cancel`, and `eve__reply`, with `eve__execute` held for code+ mode. Nothing authored or dynamic may be named `eve` or start with `eve__`:+ tools, subagents, skills, connections, and extension mounts, since a+ connection or mount named `eve` would own every `eve__` name. One rule in+ one helper replaces a list of reserved names, so `search` and `execute`+ stay ordinary names an author can use. The prefix also tells the model+ these tools are eve's and can't be replaced. Static entries fail at compile+ time or when the agent loads, and dynamic ones when they resolve, with an+ error that tells the author to rename.+- **`defaultTools: false`** doesn't remove them.++### Dispatch++An `eve__tool` or `eve__skill` call is dispatched as if the model had called+its entry directly. The harness resolves the entry once per call+(`resolveCatalogEntry(toolCall, tools)`) and uses it everywhere the harness+looks a call up by name. Model history is the only place that keeps the+catalog tool's name. Both are thin fronts on one resolver and one dispatch+path.++| Concern | Harness entries (inline, workflow, subagent) | Connection entries |+| ------------------------------ | ----------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |+| Action name on the protocol | The entry's name | `<connection>__<tool>`, as today |+| Approval | The entry's `approval` and `approvalKey` | The connection's `approval`, keyed `<connection>__<tool>`, with instance pinning |+| Execution | Inline `execute`, a workflow run, or a child session, chosen by `behavior.handling` | The connection client, with result conversion and auth parking unchanged |+| `toModelOutput`, `endsTurn` | The entry's | The connection's file-part projection |+| `ctx.toolName`, `ctx.callId` | The entry's name and the model's call id | n/a |+| Labels, hooks, audience policy | The entry | `Linear: List issues`, as today |+| Eval assertions | `t.calledTool("deploy_service")`, `t.calledSubagent(...)` | `t.calledTool("linear__list_issues")`, unchanged |+| Model history | `eve__tool` call and result, same call id | Same |++- **One name everywhere.** The name the model passes to `eve__tool` is also the+ protocol action name, the approval `toolName` and key, and the eval+ assertion name. For connection tools that is today's `<connection>__<tool>`.+ So existing approval policies, recorded "always approve" decisions,+ `toolResultFrom`, and eval assertions keep working.+- **Harness entries need nothing new.** Once a call resolves to its entry,+ the harness dispatches it exactly like a direct call, because the dispatch+ handling is already on the definition.+ - A foreground workflow tool parks the turn and starts its run.+ - A background workflow tool returns its task receipt, and `eve__task_wait` and+ `eve__task_cancel` work on the `taskId`.+ - A subagent starts its child session, and the child's `session.started`+ records the model's call id as `parentCallId`.+- **Connection entries become harness definitions.** At resolve time, eve+ builds a harness definition from the connection's metadata that is never+ advertised: an `execute` that calls the client, the connection's approval,+ and its model-output projection. The code is today's+ `executeConnectionTool`, `connection-approval.ts`, and `tool-result.ts`,+ moved behind that definition. The definition is built only after the call+ is already known to be a connection call, so the network lookup it needs+ never delays the decision to park.+- **Skill entries run the `load-skill` action.** `eve__skill({ name })` becomes+ the same runtime action `load_skill` produces today+ (`kind: "load-skill"`). So protocol events, activation, package files+ under the skills root, and `t.loadedSkill(...)` are unchanged. Its label+ reads `Load skill <name>`, then `Loaded skill <name>` once it loads. History holds an `eve__skill` call whose result is the skill's+ markdown. Loading needs no approval, as today.+- **No nested actions.** Every entry, including a connection tool, is+ reported as the call itself. The nested-action helper and the tool-call+ action's `parentCallId` field lose their only producer and are removed.+ Code mode adds the field back for calls made from a program.+- **`eve__tool` and `eve__skill` inside the AI SDK.** The AI SDK decides per+ tool name whether to run a tool, so both always have a run function. For an+ inline entry or a skill it runs the entry's own `execute`. For a workflow tool or subagent it+ returns a dispatch marker that the harness strips from the step, then+ dispatches the resolved call through the existing workflow and+ child-session path. The marker is hidden from telemetry and lives in one+ module.+- **Harness seams.** These are the places that look up by tool name today:+ - `buildToolApproval` and `buildToolSet` (`harness/tools.ts`);+ - `createRuntimeActionRequestFromToolCall`,+ `createCoordinationRequestFromToolCall`, subagent dispatch, and the+ `load-skill` mapping that `frameworkAction` drives today+ (`harness/coordination.ts`);+ - label and `endsTurn` lookup in `prepareModelTools`+ (`harness/tool-loop.ts`);+ - the history writer, which must record the catalog tool rather than+ `result.toolName`, `subagentName`, or `load_skill`.++ This is a change to the harness, and it is needed: without it, workflow+ tools and subagents can't be deferred, and skills can't load without+ `load_skill`.++- **Never AI SDK `deferLoading`.** eve builds its AI SDK tools itself and+ never sets `deferLoading` or passes `toolSearch()`. Either one would bring+ back the growing `tools` array.++### Catalog listing++One announcement under the key `catalog` replaces the connection listing.++- **Baseline.** On a session's first model step, eve appends one+ `context.state` message. It says which kinds of deferred entries exist+ (tools, agents, skills) and tells the model to look for a tool with+ `eve__search`, which searches its own catalog rather than the web, before saying+ it has no tool for a task. It then lists namespaces and connections, each+ capped.+- **No deferred names.** A deferred entry is meant to stay out of context+ until a search finds it, so the listing never names one. It doesn't count+ entries either: counts would change, and append a message, whenever a+ dynamic resolver adds or drops one.+- **Namespaces.** The first `__` segment of deferred tool, agent, and skill+ names, such as `sre` for `sre__list_alerts`, whether the entries come from+ an extension mount or are vendored under that prefix. A namespace query+ (`eve__search({ query: "sre__" })`) searches exactly that set. At most 20 are+ listed: the 20 with the most entries, rendered by name, so a namespace+ growing or shrinking only changes the listing at the cap. A longer list+ ends with "and more". A namespace that is also a connection's name is listed only+ as the connection.+- **Connections** are listed by name and description, capped at 20 the same+ way. Listing their tools would mean connecting to every server, and+ possibly asking for sign-in, at session start.+- **Skills that aren't deferred stay where they are:** static skills in the+ system prompt's skill section, dynamic skills in their announcement. Only+ the load instruction in both changes, to `eve__skill({ name })`.+- **Changes.** The listing only changes when the kinds present, the listed+ namespaces, or the listed connections change: when a dynamic resolver adds+ the first entry of a kind or namespace, a dynamic connection resolves, or+ the session upgrades to a new deployment. eve then appends the full+ listing again, prefixed with "The catalog changed.", and names any+ namespace or connection that is gone. A deferred entry added to a listed+ namespace, or without one, appends nothing; `eve__search` finds it.+- **After compaction,** the next step appends a fresh baseline.+- **Empty catalog,** no message.++```text+You have more tools, agents, and skills than are loaded here. Before saying you have no tool for a task, look for one with eve__search, which searches your own catalog, not the web. Call tools with eve__tool({ name, input }) and load skills with eve__skill({ name }).+Namespaces, whose entries are named <namespace>__<name>; search one with "<namespace>__": d0, index, sre, support+Connections, whose tools are named <connection>__<tool>; search one connection's tools with "<connection>__":+- linear: Linear issues and projects+- petstore: Pet store inventory API+```++## Cache invariants++1. **Fixed tools.** The `tools` array is identical on every step of a+ session: same names, descriptions, schemas, and order. Catalog changes,+ static or dynamic, never touch it, because which catalog tools are present+ follows from what the agent declares, never from what its resolvers+ return.+2. **No session-specific text** in the system prompt or any tool description.+3. **Append-only history.** Listing changes only append messages, and earlier+ messages are never rewritten.+4. **No system-message fallback.** Announcements are always messages+ appended to history. They never fall back to the system prompt, including+ on the step that runs approved calls.+5. **Deterministic rendering.** Listings are sorted, and a signature is a+ pure function of its schema, so the same entry always renders the same+ text. `eve__search` renders signatures only for the results it returns.+6. **Calling an entry adds nothing.** An `eve__tool` call, skill load,+ approval, park, sign-in, child session, or resume never adds a definition. That is exactly+ where AI SDK `toolSearch()` and pi's activation path lose the cache.++## Implementation++### Principles++The result must read as if eve had been designed around the catalog from the+start.++1. **Expect a net deletion.** This work replaces three model surfaces+ (`connection_search`, `connection_execute`, `load_skill`) and the special+ cases each one grew. It should remove more code than it adds.+2. **One path per job.** Each job in the table below has exactly one+ implementation. A direct call and an `eve__tool` call share everything after+ name resolution. No adapter, fallback, or second branch does the same work+ another way.+3. **No compatibility layer.** There are no aliases for removed tools, no+ flags that restore old behavior, and no code or comments that mention the+ old surface. Pre-1.0, eve prefers the breaking change.+4. **Old tests are deleted, not ported.** A test that covers a removed path+ is deleted along with the path. Tests for the new design are written fresh+ in the testing PR, against the new surface. Old assertions are never+ adjusted to fit.+5. **Generalize, then delete the original.** Code that survives, such as the+ ranker, the signature renderer, connection approval, and instance pinning,+ moves to where the unified path needs it. No copy stays behind under its+ old name.++### One path per job++| Job | Today | After |+| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |+| Find a capability by name | Authored tool map, dynamic tool list, dynamic subagent merge, presentation map (`prepareModelTools`), and `connection_execute`'s own lookup | One table of the step's entries, built once per step and used both to advertise direct tools and to resolve `eve__tool` and `eve__skill` |+| Run a call | AI SDK `execute`, workflow dispatch, subagent dispatch, the `load_skill` `frameworkAction`, and the `connection_execute` closure | Resolve the entry, then dispatch on its handling, the same for direct and catalog calls |+| Approve a call | Per-tool approval functions on AI SDK tool objects, plus connection approval delegated through `connection_execute` | The resolved entry's approval, for every call |+| Announce state | A dedicated `availableSkills` field and the keyed announcements (`harness/current-messages.ts`) | Keyed announcements only: `catalog`, plus skills that aren't deferred |+| Load a skill | `load_skill`, the `frameworkAction` marker, and reserved-name handling in the subagent registry | `eve__skill({ name })` to the `load-skill` action |+| Report a call | An action, plus nested actions for connection calls | One action per call, named for the entry |+| Label a call | Entry labels, plus `connection_execute` special cases in `toolCallDisplayName`, the dev TUI, and the channel task card | Entry labels |+| Mock model | Special handling and skill selection for `load_skill` (`runtime/agent/mock-model-*`) | The mock model calls `eve__search`, `eve__tool`, and `eve__skill` like a real one |++### Deletion inventory++- **Connection tools.** `execution/tools/connection-tools.ts`,+ `execution/tools/connection-target.ts`, `tools/framework/connection-tools.ts`,+ `execution/connection-announcement.ts`, and the connection slots in+ `compiler/default-tool-policy.ts`.+ - `connection-search-rank.ts` and `runtime/connections/tool-signature.ts`+ move to the catalog.+ - `connection-approval.ts` moves behind the connection entry's definition.+- **Nested actions.** `harness/nested-actions.ts` and its callers in+ `harness/emission.ts` and `harness/step-hooks.ts`, plus their rendering in+ the dev TUI trace view and the channel task card, and the tool-call+ action's `parentCallId` field with its span attribute.+- **`load_skill`.** `execution/tools/load-skill.ts`,+ `tools/provided/load-skill.ts`, `public/tools/load-skill.ts`, the+ `eve/tools/load_skill` export, its framework source registration, the+ `load-skill` value of `frameworkAction`, `LOAD_SKILL_TOOL_NAME`, and its+ reserved-name entry in `runtime/resolve-agent-graph.ts`. An authored+ `agent/tools/load_skill.ts` becomes an ordinary tool. The `load-skill`+ action kind stays, because the protocol, traces, and `t.loadedSkill` report+ skill loads with it.+- **Skill announcement channel.** `PendingSkillAnnouncementKey`, the+ `availableSkills` announcement field, and its history state. Dynamic skills+ use a keyed announcement instead.+- **Special cases.** Every branch that names `connection_search`,+ `connection_execute`, or `load_skill`, including the TUI tool+ presentation, the channel task card, `load_skill`'s connection hint (which+ moves to `eve__tool`), and the mock model.+- **Internal naming.** The harness's "deferred tool" for workflow-backed+ tools becomes "workflow tool".+- **Tests.** Every unit, integration, and e2e test for the paths above,+ deleted in the PR that removes the path. That is 18 unit and integration+ test files and 23 tracked e2e files today, led by+ `execution/tools/connection-tools.test.ts`,+ `execution/tools/load-skill.test.ts`,+ `execution/connection-announcement.test.ts`, the connection evals in+ `agent-openapi-swagger` and `agent-workflow-tools`, and the `load_skill`+ evals in `agent-skills` and `extensions`. A test file that also covers+ surviving behavior loses only the removed cases.++### Pull requests++Three stacked PRs. The first two hold all of the implementation and add no+tests. The third holds all of the new tests.++| PR | Scope |+| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| [1/3] Catalog | Naming groundwork (connection prefix ownership, key and name validation, the internal rename); `deferred` on tools and workflow tools; `tool: "deferred"` on subagents; dynamic entries; `eve__search` and `eve__tool`; the step entry table and call resolution; the catalog listing; connection tools as entries; removal of the connection tools and nested actions |+| [2/3] Skills | `deferred` in frontmatter and `defineSkill`, static and dynamic; skill results in `eve__search`; `eve__skill({ name })`; removal of `load_skill` and the skill announcement channel |+| [3/3] Tests | Every new test, listed below. Fixes the tests surface go back into [1/3] or [2/3] before the stack merges. |++- **The stack merges together.** [1/3] and [2/3] delete the old e2e evals+ with the paths they cover, and [3/3] adds the new ones. So [1/3] and [2/3]+ pass unit tests on their own, but the e2e suites are only meaningful with+ [3/3] stacked on top.+- **Docs and changesets go with the implementation.** [1/3] and [2/3] each+ update the docs for the behavior they change and carry a `minor`+ changeset.++## Tests and rollout++Everything in this section lands in [3/3].++- **Captured-request unit test.** It drives one session through each entry:+ - an `eve__search`, then `eve__tool` of an inline tool;+ - a foreground workflow tool that parks and resumes;+ - a background workflow tool;+ - a deferred subagent;+ - a connection tool that parks for sign-in;+ - a connection listed for sign-in, then `eve__tool` of its name;+ - a dynamic connection resolving;+ - a dynamic deferred tool and a dynamic deferred subagent appearing+ mid-session, then one of them disappearing;+ - a `eve__search` that returns a tool and a skill with the same name, then+ `eve__skill({ name })` for a deferred static skill with package files;+ - a deferred dynamic skill appearing on `turn.started`, then loading;+ - `eve__skill({ name })` for a skill that isn't deferred;+ - a catalog change on the step that runs an approved call;+ - compaction.++ It asserts all six invariants. A second case covers an agent with only+ listed skills: `eve__skill` is its only catalog tool and no listing is+ appended.++- **E2E: a new `agent-deferred-tools` fixture.** About 40 deferred entries:+ inline tools, a workflow tool with approval, a background workflow tool, a+ deferred subagent, deferred static and dynamic skills, a session-scoped+ dynamic resolver returning deferred tools, and the self-contained petstore+ OpenAPI connection.+ - World suites with the mock model: approval keyed to the entry, the+ workflow tool parking and resuming, the background task receipt, the+ subagent's child session, a deferred skill loaded with `t.loadedSkill`,+ and a misspelled name corrected from the suggestion.+ - Real-model suite: cache reads on the steps after discovery, following+ `agent-prompt-cache`.+- **New evals in the existing fixtures.** `agent-openapi-swagger`,+ `agent-workflow-tools`, `agent-skills`, and `extensions` get new evals+ written against `eve__search`, `eve__tool`, and `eve__skill`. They cover what the deleted evals+ covered: connection approval, sign-in, dynamic connections, MCP result+ content, input validation, dynamic skills, skill overrides, and extension+ skills. They are written from the behavior, not ported from the old files.+- **Unit coverage for rules whose old tests were deleted.** Subagent+ visibility in the step catalog, the task tools offered per session,+ connection prefix ownership at compile and resolve time, dynamic tool and+ subagent collisions (including the `tool: false` wrapper pattern),+ connection input validated before approval, and listing failures that+ aren't sign-in.+- **Measurements.** Run one task set three ways:+ 1. all tools and skills direct,+ 2. long-tail tools and skills deferred,+ 3. long-tail tools and skills in subagents.++ Compare task success, input tokens, cache read ratio, and model calls, at a+ moderate tool count and at a large one.++- **Ship gate.**+ - Arm 2 matches arm 1 on success at the moderate count and beats it at the+ large count.+ - Arm 2 beats arm 3 on success.+ - Connection and skill evals don't regress.+ - Agents with nothing to find get neither tool, so they behave as on+ `main`.+ - The model doesn't call `eve__search` for web questions on agents that have+ `web_search`.+ - No cache regression after discovery.+- **Later, gated by evals.** A real-model eval of discovery with a large+ catalog and no connections, which decides whether the listing needs more+ than kinds and namespaces.+- **Docs,** updated in [1/3] and [2/3]:+ - `tools/overview`: deferring a tool, and which tools to defer (keep+ frequently used tools direct and defer the long tail).+ - `tools/workflows`, `subagents/*` (`tool: "deferred"`), and+ `concepts/built-in-tools` (`eve__search`, `eve__tool`, and `eve__skill` instead of the+ connection tools and `load_skill`).+ - `skills`, `instructions`, `concepts/context-control`,+ `evals/assertions`, `reference/typescript-api`, and the team playbooks+ tutorial: deferring a skill and loading with `eve__skill({ name })`.+ - `connections/overview`, `connections/mcp`, `guides/dynamic-capabilities`+ (bare map keys, and the connection prefix rule), and `extensions` (the+ reserved names and how mount prefixes nest with connections).+- **Changesets,** one `minor` in each of [1/3] and [2/3]. Together they:+ - remove `connection_search`, `connection_execute`, and `load_skill`;+ - reserve the `eve` namespace in every agent;+ - reject names that start with a connection's prefix, which an existing+ agent may already have, such as connection `linear` beside a tool named+ `linear__sync`.++## Decisions and alternatives considered++| Decision | Chosen | Rejected |+| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |+| Tool names | `eve__search`, `eve__tool`, and `eve__skill`, under the `eve__` namespace eve reserves for every tool it adds | `search` and `execute` unprefixed (reserves two common words authors want, and nothing marks them as eve's); `$search` (`$` isn't legal in tool names for eve or some providers); `_search` (tool names start with a letter, and a leading underscore reads as private); `tool_search` and `tool_execute` (still reserves names outside any namespace) |+| Search surface | One `eve__search` over every entry, connections included | Keeping `connection_search` next to it (two places to look, two listings); a separate `skill_search` |+| Split by kind | `eve__tool({ name, input? })` for tools and `eve__skill({ name })` for skills, each with a flat schema that requires `name` | One `eve__execute` taking `tool` or `skill` (providers reject a top-level `oneOf`, so it needs both keys optional plus an exactly-one check; loading a skill reads as a clause of a call-a-tool description and loses `load_skill`'s trigger; one tool registered for two unrelated reasons); `eve__connection` for connection tools (connection tools are tools, and the model would have to classify a tool by where it comes from); markers such as `skill:pdf` in one name space |+| Presence | From what the agent declares, per tool: `eve__tool` with anything it can call, `eve__skill` with any skill, `eve__search` with anything it could find | Always (about 290 tokens per request with nothing to find, and serialized parallel calls on gpt-6-sol); only while the catalog is non-empty (flips `tools` when a dynamic entry appears); an agent-level flag |+| Dynamic entries | Deferrable, like static ones | Static only (rules out per-tenant catalogs) |+| Opt-in | `deferred: true` on tools and skills; `tool: "deferred"` on agents | An agent-level list of names; `deferLoading` (eve never loads the definition); pi's `exposure` enum |+| Discovery mechanism | Fixed catalog tools | AI SDK `toolSearch()` or activation (grows `tools`); provider-native search (one provider at a time, partly beta) |+| Dispatch | Resolve to an entry, then dispatch it like a direct call | A `defineTool` proxy (can't reach workflow tools or subagents; duplicates approval) |+| Entry names | One flat name, `__` as the only namespace separator, and connections own their prefix | `<connection>.<tool>` (a second separator for one owner, and it renames connection tools); dots everywhere, encoded as `__` for providers (deferring would rename a tool); a separate `connection` argument (a two-part key) |+| Owners | Recorded on each entry as data, added with code mode | Recovered by splitting names at `__` (can't tell an owner from a convention prefix) |+| Dispatch marker | `eve__tool` returns a marker for workflow and subagent entries; the harness strips it | Renaming calls in a provider middleware (connection tools can't join the tool set without listing every connection, so they would need a second path) |+| Search output | TypeScript `signature` | Raw JSON Schema (larger, and code mode needs the signature anyway) |+| Protocol shape | Actions carry the entry's name; only history says `eve__tool` or `eve__skill` | An outer catalog-tool action with a nested entry action |+| Search state | Stateless; any entry can be executed | A durable discovered set that execution checks |+| Listing | One append-only listing: the kinds present, capped namespaces, and capped connections | Nothing (the model can't tell when to search); every deferred name (defeats deferring and grows without bound); counts (churn on dynamic changes) |+| Old paths and tests | Deleted with no compatibility layer; old tests deleted, and new tests written fresh in [3/3] | Aliases or flags for removed tools; porting old tests to the new surface; tests spread across the implementation PRs |+| Loading skills | `eve__skill({ name })` for every skill, registered with any skill; `load_skill` is removed | Keeping `load_skill` beside `eve__skill` (two ways to load); `load_skill` only for skills that aren't deferred (the model must know which kind it has) |++## Later: code mode++Code mode will be a separate tool, `eve__execute({ code })`, built on the+workflow program runtime. Each call a program makes is dispatched through the+catalog, as an `eve__tool` call is. The rest of this design carries over:++| This design | Code mode |+| ----------------------------------- | ---------------------------------------------------------------- |+| `eve__search(opts)` tool | `search(opts)` inside the program, same result shape |+| `eve__tool({ name, input })` | `tools.deploy_service(input)`, `tools.linear.list_issues(input)` |+| `eve__skill({ name })` | Loading a skill from the program |+| One action per `eve__tool` call | One nested action per call, with `parentCallId` added back |+| Approval on the model's call | Approval on the nested action, parking mid-program |+| Catalog listing | Unchanged |+| Signatures in `eve__search` results | The types the program is written against |++- `eve__tool({ name, input })` is a one-call program, so both forms share one+ dispatch path.+- Each entry records its owners, and program paths come from that record,+ not from splitting names: `linear__list_issues` is `tools.linear.list_issues`,+ `crm__api__list_issues` is `tools.crm.api.list_issues`, and+ `tenant__export`, which has no owner, is `tools.tenant__export`.+- `eve__execute` appears only when code mode is on. That is fixed per agent+ and eve version, never per step, so the `tools` array stays stable.+- Names with `-` aren't JavaScript identifiers. A kebab-case connection such+ as `google-drive` is called as `tools["google-drive"].list_files(input)`.diff --git a/scripts/guard-invariants-baseline.json b/scripts/guard-invariants-baseline.jsonindex f726dd26c..bd55a80cd 100644--- a/scripts/guard-invariants-baseline.json+++ b/scripts/guard-invariants-baseline.json@@ -44,7 +44,6 @@"packages/eve/src/tools/provided/bash.ts","packages/eve/src/tools/provided/glob.ts","packages/eve/src/tools/provided/grep.ts",- "packages/eve/src/tools/provided/load-skill.ts","packages/eve/src/tools/provided/read-file.ts","packages/eve/src/tools/provided/web-fetch.ts","packages/eve/src/tools/provided/web-search.ts",diff --git a/scripts/guard-invariants.mjs b/scripts/guard-invariants.mjsindex 873ee05dd..e0f5e5c32 100644--- a/scripts/guard-invariants.mjs+++ b/scripts/guard-invariants.mjs@@ -393,7 +393,7 @@ const PROVIDED_TOOL_EXTRA_IMPORTS = new Set(["#tools/schema.js",]);-// `task_wait` and `task_cancel` belong to the session, not to authors: they+// `eve__task_wait` and `eve__task_cancel` belong to the session, not to authors: they// take their model text from execution/tasks/render.ts directly.const PROVIDED_TASK_TOOL_FILES = new Set(["packages/eve/src/tools/provided/task-cancel.ts",@@ -1223,10 +1223,6 @@ async function checkFrameworkActionIdentity() {for (const root of roots) {for await (const file of walkFiles(join(REPO_ROOT, root))) files.push(file);}- files.push({- absPath: join(REPO_ROOT, "packages/eve/src/execution/tools/connection-tools.ts"),- relPath: "packages/eve/src/execution/tools/connection-tools.ts",- });for (const { absPath, relPath } of files) {if (!absPath.endsWith(".ts") || absPath.endsWith(".test.ts")) continue;const source = await readFile(absPath, "utf8");