Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

docs: dedicated human-in-the-loop page (#23)

Timo Linsed97fda52dec
diff --git a/docs/README.md b/docs/README.md
index 148b4e2d2..64fe1e164 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -40,7 +40,7 @@ Read in this order:
5. [TypeScript API](./reference/typescript-api.md)
6. [Context Control](./concepts/context-control.md)
7. [Skills](./skills.mdx)
-8. [Tools](./tools.mdx)
+8. [Tools](./tools/overview.mdx)
9. [Connections](./connections.mdx)
10. [Sandboxes](./sandbox.mdx)
11. [Channels](./channels/overview.mdx)
diff --git a/docs/concepts/default-harness.md b/docs/concepts/default-harness.md
index 33f08d830..53447a8ef 100644
--- a/docs/concepts/default-harness.md
+++ b/docs/concepts/default-harness.md
@@ -24,20 +24,20 @@ Compaction also preserves the framework's own tool state automatically. It reset
These ship with every agent, no imports. The harness shows the model the tool descriptors first, then executes only what the model actually calls; discovery never runs them. The shell and file tools (`bash`, `read_file`, `write_file`, `glob`, `grep`) live in the app runtime and proxy their work into the agent's single [sandbox](../sandbox); the rest run in the app runtime. The "Where it runs" column below names where each tool's effect lands.
-| Tool | Does | Where it runs |
-| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
-| `bash` | Run a shell command. | Sandbox |
-| `read_file` | Read a text file with line-numbered output (enables read-before-write). | Sandbox FS |
-| `write_file` | Write a complete file; enforces read-before-write and stale-read detection. | Sandbox FS |
-| `glob` | Find files by glob pattern. | Sandbox FS |
-| `grep` | Search file contents by regex. | Sandbox FS |
-| `web_fetch` | Fetch a URL. | App runtime |
-| `web_search` | Search the web (provider-managed; resolved from the model provider). | Provider |
-| `todo` | Maintain a durable per-session todo list. | App runtime |
-| `ask_question` | Ask the user a clarifying question or a choice mid-turn and park until they answer. No `execute`; the model calls it with `{ prompt, options?, allowFreeform? }`. | App runtime |
-| `agent` | Delegate a subtask to a copy of itself (shares the parent sandbox + tools, fresh history/state). | App runtime |
-| `load_skill` | Pull an on-demand [skill](../skills)'s instructions into the current turn. Present only when the agent declares skills. | App runtime |
-| `connection_search` | Discover tools across declared [connections](../connections); matched tools become directly callable. Present only when the agent declares connections. | App runtime |
+| Tool | Does | Where it runs |
+| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
+| `bash` | Run a shell command. | Sandbox |
+| `read_file` | Read a text file with line-numbered output (enables read-before-write). | Sandbox FS |
+| `write_file` | Write a complete file; enforces read-before-write and stale-read detection. | Sandbox FS |
+| `glob` | Find files by glob pattern. | Sandbox FS |
+| `grep` | Search file contents by regex. | Sandbox FS |
+| `web_fetch` | Fetch a URL. | App runtime |
+| `web_search` | Search the web (provider-managed; resolved from the model provider). | Provider |
+| `todo` | Maintain a durable per-session todo list. | App runtime |
+| `ask_question` | Ask the user a clarifying question or a choice mid-turn and park until they answer. No `execute`; the model calls it with `{ prompt, options?, allowFreeform? }`. See [Human-in-the-loop](../tools/human-in-the-loop). | App runtime |
+| `agent` | Delegate a subtask to a copy of itself (shares the parent sandbox + tools, fresh history/state). | App runtime |
+| `load_skill` | Pull an on-demand [skill](../skills)'s instructions into the current turn. Present only when the agent declares skills. | App runtime |
+| `connection_search` | Discover tools across declared [connections](../connections); matched tools become directly callable. Present only when the agent declares connections. | App runtime |
Notes:
diff --git a/docs/concepts/sessions-runs-and-streaming.md b/docs/concepts/sessions-runs-and-streaming.md
index 358f30e27..7993ec14a 100644
--- a/docs/concepts/sessions-runs-and-streaming.md
+++ b/docs/concepts/sessions-runs-and-streaming.md
@@ -34,33 +34,33 @@ curl http://127.0.0.1:3000/eve/v1/session/<sessionId>/stream
The stream is newline-delimited JSON (NDJSON), one event per line:
-| Event | Meaning |
-| ------------------------- | ---------------------------------------------------------------------------------------------- |
-| `session.started` | A durable session was created. |
-| `turn.started` | A new turn began. |
-| `message.received` | An inbound user message was accepted. |
-| `step.started` | A model step began. |
-| `actions.requested` | The model requested tool calls. |
-| `action.result` | A tool call returned. |
-| `input.requested` | The run paused for human input (HITL approval or `ask_question`); carries `requests`. |
-| `subagent.called` | A subagent was delegated; carries `childSessionId` to attach to. |
-| `subagent.completed` | A delegated subagent finished. |
-| `reasoning.appended` | A reasoning delta (incremental, with cumulative text so far). |
-| `reasoning.completed` | The finalized reasoning block. |
-| `message.appended` | An assistant text delta (incremental, with cumulative text so far). |
-| `message.completed` | A finalized assistant text block. |
-| `result.completed` | The finalized structured result for a turn that requested an output schema; carries `result`. |
-| `compaction.requested` | Context-window compaction began; carries `modelId`, `sessionId`, `turnId`, `usageInputTokens`. |
-| `compaction.completed` | A compaction checkpoint was written to durable history. |
-| `authorization.required` | A connection needs OAuth; carries `name`, `description`, and an `authorization` challenge. |
-| `authorization.completed` | A connection's authorization resolved; carries `outcome`. |
-| `step.completed` | A model step finished; carries `finishReason` and usage. |
-| `step.failed` | A model step failed; carries `{ code, message, details? }`. |
-| `turn.completed` | The turn finished. |
-| `turn.failed` | The turn failed; carries `{ code, message, details? }`. |
-| `session.waiting` | The session parked, waiting for the next input (a message, an answer). |
-| `session.failed` | The session failed. |
-| `session.completed` | The session reached a terminal end. |
+| Event | Meaning |
+| ------------------------- | ------------------------------------------------------------------------------------------------------------------- |
+| `session.started` | A durable session was created. |
+| `turn.started` | A new turn began. |
+| `message.received` | An inbound user message was accepted. |
+| `step.started` | A model step began. |
+| `actions.requested` | The model requested tool calls. |
+| `action.result` | A tool call returned. |
+| `input.requested` | The run paused for human input ([HITL](../tools/human-in-the-loop) approval or `ask_question`); carries `requests`. |
+| `subagent.called` | A subagent was delegated; carries `childSessionId` to attach to. |
+| `subagent.completed` | A delegated subagent finished. |
+| `reasoning.appended` | A reasoning delta (incremental, with cumulative text so far). |
+| `reasoning.completed` | The finalized reasoning block. |
+| `message.appended` | An assistant text delta (incremental, with cumulative text so far). |
+| `message.completed` | A finalized assistant text block. |
+| `result.completed` | The finalized structured result for a turn that requested an output schema; carries `result`. |
+| `compaction.requested` | Context-window compaction began; carries `modelId`, `sessionId`, `turnId`, `usageInputTokens`. |
+| `compaction.completed` | A compaction checkpoint was written to durable history. |
+| `authorization.required` | A connection needs OAuth; carries `name`, `description`, and an `authorization` challenge. |
+| `authorization.completed` | A connection's authorization resolved; carries `outcome`. |
+| `step.completed` | A model step finished; carries `finishReason` and usage. |
+| `step.failed` | A model step failed; carries `{ code, message, details? }`. |
+| `turn.completed` | The turn finished. |
+| `turn.failed` | The turn failed; carries `{ code, message, details? }`. |
+| `session.waiting` | The session parked, waiting for the next input (a message, an answer). |
+| `session.failed` | The session failed. |
+| `session.completed` | The session reached a terminal end. |
`reasoning.appended` and `message.appended` stream deltas as they arrive, and each one carries both the new delta and the cumulative text for the current block. The finalized block shows up on `message.completed` and `reasoning.completed`, which is the compatibility path for clients that don't render incremental streaming.
diff --git a/docs/guides/frontend/overview.mdx b/docs/guides/frontend/overview.mdx
index 181df4119..b32e37092 100644
--- a/docs/guides/frontend/overview.mdx
+++ b/docs/guides/frontend/overview.mdx
@@ -100,7 +100,7 @@ Assistant text, reasoning, tool calls, and tool results stream into `data` as th
## Human-in-the-loop prompts
-Tools opt into approval with `needsApproval` on the [server side](../../tools), and the model can also ask a question with `ask_question`. Either way the stream emits an `input.requested` event, and the pending request rides on a `dynamic-tool` part of the latest message at `part.toolMetadata?.eve?.inputRequest`. Read it, then answer through the same session with `send()`:
+Tools opt into approval with `needsApproval`, and the model can also ask a question with `ask_question` — see [Human-in-the-loop](../../tools/human-in-the-loop) for the server-side model. Either way the stream emits an `input.requested` event, and the pending request rides on a `dynamic-tool` part of the latest message at `part.toolMetadata?.eve?.inputRequest`. Read it, then answer through the same session with `send()`:
```tsx
const request = agent.data.messages
diff --git a/docs/tools/human-in-the-loop.md b/docs/tools/human-in-the-loop.md
new file mode 100644
index 000000000..f387aaafd
--- /dev/null
+++ b/docs/tools/human-in-the-loop.md
@@ -0,0 +1,83 @@
+---
+title: "Human-in-the-loop"
+description: "Pause a run for a person — gate a tool on approval or have the agent ask a question — and resume durably when they answer."
+url: /human-in-the-loop
+---
+
+Human-in-the-loop (HITL) is any point where the agent durably pauses and waits for a person. Two things trigger it, and both ride the same pause-and-resume protocol:
+
+- **Approvals** — a tool requires a person to sign off before (or instead of) running. The agent decides to call the tool; a human decides whether it does.
+- **Questions** — the agent itself asks the user a clarifying question or a choice mid-turn, and parks until they answer.
+
+Either way the run parks at `session.waiting`, durably, for as long as it takes — seconds or days — and picks back up exactly where it left off once the answer arrives. Channels render the request for you.
+
+## Approvals
+
+Approval is a property of a [tool](./overview) that pauses for a person before it runs. Gate a tool with `needsApproval` and the helpers from `eve/tools/approval`:
+
+```ts title="agent/tools/refund_charge.ts"
+import { defineTool } from "eve/tools";
+import { always } from "eve/tools/approval";
+import { z } from "zod";
+
+export default defineTool({
+ description: "Refund a charge.",
+ inputSchema: z.object({ chargeId: z.string(), amount: z.number() }),
+ needsApproval: always(), // or once() / never() / a predicate
+ async execute(input) {
+ return refund(input);
+ },
+});
+```
+
+| Helper | Behavior |
+| ---------- | ---------------------------------------------------------------------------------- |
+| `never()` | Never require approval (the default when omitted). |
+| `once()` | Require approval only the first time the tool runs in a session; auto-allow after. |
+| `always()` | Require approval before every call. |
+
+By default, omitted `needsApproval` behaves like `never()`, so tool calls may execute without human approval. Require human approval or other safeguards for sensitive, irreversible, regulated, financial, healthcare, employment, housing, legal, safety-impacting, user-impacting, or external side-effecting actions.
+
+When the decision depends on the input, pass your own predicate instead of a helper. It receives `{ toolName, toolInput, approvedTools }` and returns a boolean. `toolInput` can be undefined, so guard the access. To require approval only when an amount crosses a threshold:
+
+```ts
+needsApproval: ({ toolInput }) => (toolInput?.amount ?? 0) > 1000,
+```
+
+Gating a side effect on approval is also how you make non-idempotent work safe across replays: a charge or email that sits behind `always()` can't fire from a re-run step without a fresh human decision.
+
+## Questions
+
+The built-in `ask_question` tool lets the model pause and ask the user, rather than guessing. It has no `execute` — the model calls it with `{ prompt, options?, allowFreeform? }`:
+
+- `prompt`: the question to put to the user.
+- `options`: an optional list of choices to offer. Channels render these as buttons or a select menu.
+- `allowFreeform`: whether the user may answer with free text instead of picking an option.
+
+`ask_question` is part of the [default harness](../concepts/default-harness), so it is available without you defining anything. It produces the same `input.requested` pause as an approval, and resumes the same way.
+
+## How pause and resume works
+
+Approvals and questions share one protocol:
+
+1. The model requests input (an approval, or an `ask_question`).
+2. Eve emits an `input.requested` stream event carrying the pending requests.
+3. The turn parks at `session.waiting`, durably, for as long as it takes.
+4. The client answers with `inputResponses` (structured, keyed by `requestId`) or a normal follow-up `message`. A follow-up whose text matches an option label (case-insensitive) resolves automatically.
+
+The run picks back up exactly where it parked. Because the pause is durable, nothing is held in memory while it waits — the process can restart and the parked turn survives.
+
+See [Sessions, runs & streaming](../concepts/sessions-runs-and-streaming) for the full event and resume contract that this builds on.
+
+## Answering from a client or channel
+
+Channels turn requests into native UI: the Slack adapter renders approvals as buttons and questions as select menus, and writes the user's choice back as the answer. You get this for free on every [channel](../channels).
+
+From your own frontend, read the pending request off the latest message and answer through the same session — see [Building a frontend](../guides/frontend/overview#human-in-the-loop-prompts) for the client-side reducer and `inputResponses` shape.
+
+## What to read next
+
+- [Tools](./overview): define the typed actions an approval gates
+- [Default harness](../concepts/default-harness): the built-in tools, including `ask_question`
+- [Sessions, runs & streaming](../concepts/sessions-runs-and-streaming): the event and resume contract behind the pause
+- [Building a frontend](../guides/frontend/overview): render and answer requests from your own UI
diff --git a/docs/tools/meta.json b/docs/tools/meta.json
new file mode 100644
index 000000000..d1d3d7c10
--- /dev/null
+++ b/docs/tools/meta.json
@@ -0,0 +1,4 @@
+{
+ "title": "Tools",
+ "pages": ["overview", "human-in-the-loop"]
+}
diff --git a/docs/tools.mdx b/docs/tools/overview.mdx
similarity index 54%
rename from docs/tools.mdx
rename to docs/tools/overview.mdx
index 5f96a61a7..98536c797 100644
--- a/docs/tools.mdx
+++ b/docs/tools/overview.mdx
@@ -1,9 +1,10 @@
---
title: "Tools"
description: "Define typed actions the agent can call, and gate sensitive ones on human approval."
+url: /tools
---
-A tool is a typed action the agent can call, such as hitting an API, running a query, or writing a file. The action stays in code you control. Tools run in your app runtime with full access to `process.env`, not in the [sandbox](./sandbox).
+A tool is a typed action the agent can call, such as hitting an API, running a query, or writing a file. The action stays in code you control. Tools run in your app runtime with full access to `process.env`, not in the [sandbox](../sandbox).
## Define a tool
@@ -36,16 +37,16 @@ When a tool returns structured data, add an optional `outputSchema`. With Zod or
`execute` gets a `ctx` carrying the runtime accessors:
- `ctx.session`: session metadata, turn, auth, parent lineage.
-- `ctx.getSandbox()`: the live [sandbox](./sandbox) handle.
-- `ctx.getSkill(id)`: read a packaged [skill](./skills)'s metadata and files.
+- `ctx.getSandbox()`: the live [sandbox](../sandbox) handle.
+- `ctx.getSkill(id)`: read a packaged [skill](../skills)'s metadata and files.
Running in the app runtime is what lets a tool import shared code from `lib/`, read `process.env`, and take part in Eve’s durable pause/resume model.
Eve never runs authored tools during discovery. The model sees descriptors first, and only what it actually calls gets executed. Completed steps never re-run; Eve replays the recorded result. A step interrupted mid-execution re-runs, so make non-idempotent side effects like charges or emails idempotent, or gate them with approval.
-## Human-in-the-loop
+## Gate a tool on human approval
-Human-in-the-loop (HITL) approval is a property of a tool that pauses for a person before (or instead of) running. Gate a tool on approval with `needsApproval` and the helpers from `eve/tools/approval`:
+A tool can require a person to sign off before it runs. Set `needsApproval` with the helpers from `eve/tools/approval`:
```ts title="agent/tools/refund_charge.ts"
import { defineTool } from "eve/tools";
@@ -62,32 +63,7 @@ export default defineTool({
});
```
-| Helper | Behavior |
-| ---------- | ---------------------------------------------------------------------------------- |
-| `never()` | Never require approval (the default when omitted). |
-| `once()` | Require approval only the first time the tool runs in a session; auto-allow after. |
-| `always()` | Require approval before every call. |
-
-By default, omitted `needsApproval` behaves like `never()`, so tool calls may execute without human approval. Require human approval or other safeguards for sensitive, irreversible, regulated, financial, healthcare, employment, housing, legal, safety-impacting, user-impacting, or external side-effecting actions.
-
-When the decision depends on the input, pass your own predicate instead of a helper. It receives `{ toolName, toolInput, approvedTools }` and returns a boolean. `toolInput` can be undefined, so guard the access. To require approval only when an amount crosses a threshold:
-
-```ts
-needsApproval: ({ toolInput }) => (toolInput?.amount ?? 0) > 1000,
-```
-
-### How approval pause and resume works
-
-Approvals and questions share one protocol:
-
-1. The model requests input (an approval or an `ask_question`).
-2. Eve emits an `input.requested` stream event carrying the pending requests.
-3. The turn parks at `session.waiting`, durably, for as long as it takes.
-4. The client answers with `inputResponses` (structured, keyed by `requestId`) or a normal follow-up `message`. A follow-up whose text matches an option label (case-insensitive) resolves automatically.
-
-The run picks back up exactly where it parked, and channels render the request for you. The Slack adapter, for example, turns approvals into buttons and questions into select menus.
-
-See [Sessions, runs & streaming](./concepts/sessions-runs-and-streaming) for the event and resume contract.
+Approval is one half of Eve's [human-in-the-loop](./human-in-the-loop) model — the page covers the `always/once/never` helpers, input-dependent predicates, and how a gated call pauses and resumes durably.
## Shape what the model sees with `toModelOutput`
@@ -105,7 +81,8 @@ Do not return secrets, credentials, unnecessary personal data, or unbounded sens
## What to read next
-- [Skills](./skills): on-demand procedures the model loads when relevant
-- [Default harness](./concepts/default-harness): the built-in tools and how to override or disable them
-- [Dynamic capabilities](./guides/dynamic-capabilities): tools whose set is resolved per session with `defineDynamic`
-- [Auth & route protection](./guides/auth-and-route-protection): authenticate a tool to an external service
+- [Human-in-the-loop](./human-in-the-loop): gate a tool on approval, or have the agent ask a question
+- [Skills](../skills): on-demand procedures the model loads when relevant
+- [Default harness](../concepts/default-harness): the built-in tools and how to override or disable them
+- [Dynamic capabilities](../guides/dynamic-capabilities): tools whose set is resolved per session with `defineDynamic`
+- [Auth & route protection](../guides/auth-and-route-protection): authenticate a tool to an external service