Skip to content

CubeLoop — a Pythonic, async-native agent harness core designed for high performance, readability, and production-grade persistence.

docs(deferred): document dispatch vs inject strategies; hooks page gains resolve_tool_call (en + zh-Hans)

xf gongd7f28edf6700
diff --git a/website/docs/guides/middleware/composition.md b/website/docs/guides/middleware/composition.md
index 4b669b4..d9ed7d0 100644
--- a/website/docs/guides/middleware/composition.md
+++ b/website/docs/guides/middleware/composition.md
@@ -181,5 +181,5 @@ their relative order within the injected list.
## See also
-- [The 8 Hooks](./hooks) — what each hook does and when it fires.
+- [The 9 Hooks](./hooks) — what each hook does and when it fires.
- [Examples](./examples) — composition in practice.
diff --git a/website/docs/guides/middleware/deferred-tools.md b/website/docs/guides/middleware/deferred-tools.md
index 8687e91..9789eb7 100644
--- a/website/docs/guides/middleware/deferred-tools.md
+++ b/website/docs/guides/middleware/deferred-tools.md
@@ -1,6 +1,6 @@
---
title: Deferred Tool Groups
-description: "Hide MCP tool schemas from the model by default, expanding them on demand to reduce context bloat."
+description: "Hide MCP tool schemas from the model by default, expanding them on demand — without invalidating the prompt cache."
---
# Deferred Tool Groups
@@ -9,22 +9,52 @@ When an agent connects to many MCP servers, their combined tool schemas
can consume thousands of tokens of context on every turn — even if the
model only needs one or two groups for the current task.
`DeferredToolGroup` solves this by replacing full schemas with a compact
-catalog, letting the model expand groups on demand.
-
-## How it works
-
-1. At construction time, the agent's system prompt includes a short
- catalog — one line per group with a description and tool list.
-2. The model sees a built-in `load_tools` tool it can call to load a
- group (or specific tools within a group).
-3. On expansion, the loader runs once, the tools are injected into the
- live tool set, and their schemas are appended to the system prompt.
+catalog, letting the model load groups on demand.
+
+## Two strategies
+
+Deferred tools support two strategies, chosen with
+`deferred_tool_strategy` (default: `"dispatch"`):
+
+| | `tools` param | system prompt | cache cost per expansion | calling path |
+|---|---|---|---|---|
+| **`dispatch`** (default) | static | static | **zero** — schemas are message-suffix appends | `deferred_tool_call` dispatcher, engine-unwrapped |
+| **`inject`** | grows per expansion | catalog counts change | full re-read of system + history | native tool calls |
+
+**Why dispatch is the default.** Tool definitions render at the very
+front of the prompt on every prefix-cached provider. Injecting a tool
+mid-conversation inserts bytes *before* the entire history, so each
+expansion re-reads the whole conversation at uncached rates. Dispatch
+mode never touches the tools array or the system prompt after the first
+request — schemas travel in `load_tools` tool results, which append to
+the end of the message history and cache incrementally like any other
+turn.
+
+**When to pick `inject`.** Native tool calls get provider-side schema
+validation and the calling ergonomics models are trained on. If your
+tool arguments are complex and your conversations are short (so the
+per-expansion cache cost is small), `inject` trades cache efficiency for
+calling reliability.
+
+## How dispatch mode works
+
+1. The system prompt carries a short, **static** catalog — one line per
+ group with a description and tool names. It never changes.
+2. The model calls the built-in `load_tools(group_id)` and receives the
+ group's **full schemas in the tool result**.
+3. The model invokes a loaded tool through the built-in dispatcher:
+ `deferred_tool_call(tool_name=..., arguments=...)`.
+4. The engine unwraps the dispatcher call before anything else sees it:
+ validation, `before_tool_call`/`after_tool_call` hooks, permission
+ systems, emitted events, and tracing all observe the **real** tool
+ name and arguments — never the envelope.
```
# Deferred tool groups
These tool groups are available but not yet loaded. Call `load_tools(group_id)`
-to load a group's tools for the rest of this conversation.
+to get their full schemas, then invoke them via
+`deferred_tool_call(tool_name=..., arguments=...)`.
- `mcp:github` — GitHub: Issues, PRs, repos, code search (4 tools)
create_issue, search_repos, create_pr, list_comments
@@ -32,6 +62,19 @@ to load a group's tools for the rest of this conversation.
create_issue, update_issue, list_projects, ...
```
+A few properties worth knowing:
+
+- **Implicit loading.** If the model calls `deferred_tool_call` for a
+ tool it never explicitly loaded, the middleware loads it on the fly
+ and validates the arguments. On a validation failure the error result
+ includes the full schema, so the model self-corrects in one round
+ trip.
+- **Compaction self-rescue.** `load_tools` is idempotent — if context
+ compaction drops an old result, the model can simply call it again
+ and get byte-identical schemas back.
+- **Forks.** Forked agents (`fork_once`) inherit the dispatch resolver,
+ so tools the parent loaded remain invocable inside forks.
+
## Basic setup
Pass `deferred_tool_groups` to `Agent`. The middleware is created
@@ -65,6 +108,7 @@ agent = Agent(
model=provider.model("claude-sonnet-4-6"),
tools=[search_tool, calculator], # always-available tools
deferred_tool_groups=[github_group, linear_group],
+ # deferred_tool_strategy="inject", # opt into v1 behavior
)
```
@@ -156,83 +200,62 @@ unexpanded.
The model calls `load_tools` to load a group's tools. Two modes:
```
-# Expand everything in the group
+# Load everything in the group
load_tools(group_id="mcp:github")
-# Expand specific tools only
+# Load specific tools only
load_tools(group_id="mcp:github", tool_names=["create_issue", "search_repos"])
```
-The tool returns a structured result:
+In dispatch mode the result carries the full schemas:
```json
{
"group_id": "mcp:github",
"expanded": true,
- "tool_names": ["create_issue", "search_repos", "create_pr", "list_comments"],
- "remaining": 0
+ "tool_names": ["create_issue", "search_repos"],
+ "remaining": 2,
+ "schemas": [
+ {"name": "create_issue", "description": "...", "parameters": {"...": "..."}},
+ {"name": "search_repos", "description": "...", "parameters": {"...": "..."}}
+ ]
}
```
-After expansion, the tools are immediately available for the model to
-call in the same turn (via the `after_tool_call` hook).
-
-### Selective expansion
-
-The model can expand a group incrementally — requesting one or two tools
-now and more later:
-
-```
-load_tools(group_id="mcp:github", tool_names=["create_issue"])
-# → remaining: 3
-
-# later...
-load_tools(group_id="mcp:github", tool_names=["search_repos"])
-# → remaining: 2
-```
+(In `inject` mode `schemas` is omitted — the definitions join the
+model-visible tools array instead.)
-Already-expanded tools are idempotent — re-requesting them is a no-op.
+After loading, the tools are immediately available in the same turn.
### Loader caching
The `loader` callback is invoked exactly **once per group per run**.
-The first `load_tools` call triggers it; subsequent selective
-expansions filter from the cached result. If the loader fails, the
-error is returned to the model and the group remains unexpanded.
-
-## Prompt-cache stability
-
-The system prompt is designed for prompt-cache prefix stability:
-
-- **Catalog** is sorted by `group_id` alphabetically — input order
- doesn't matter, the rendered text is byte-stable.
-- **Expanded schemas** are appended in expansion order (the order the
- model called `load_tools`), never reordered. Each new expansion
- appends to the end, preserving the existing prefix.
-
-This means the LLM API's prompt cache remains valid across turns: the
-system prompt only grows, and only at the end.
+The first load triggers it; subsequent selective loads filter from the
+cached result. If the loader fails, the error is returned to the model
+and the group remains unloaded. Already-loaded tools are idempotent —
+re-requesting them is a no-op (and in dispatch mode re-serves the same
+schemas).
## Expansion state
-The middleware tracks which groups are expanded in `ctx.extra`:
+The middleware tracks which groups are loaded in `ctx.extra`:
```python
ctx.extra["expanded_groups"] = {
- "mcp:github": None, # fully expanded (None = all tools)
- "mcp:linear": ["create_issue"], # partially expanded
- # mcp:slack not present = unexpanded
+ "mcp:github": None, # fully loaded (None = all tools)
+ "mcp:linear": ["create_issue"], # partially loaded
+ # mcp:slack not present = unloaded
}
```
-This state survives checkpointing and can be used for cross-run replay
-(see below).
+This state survives checkpointing and drives cross-run replay.
## Cross-run replay
-When resuming a conversation from a previous run, you need to restore
-the expansion state so the model has the same tools available.
-`prepare_resumed_state` handles this:
+When resuming a conversation from a previous run, restore the expansion
+state so dispatched calls resolve immediately. `prepare_resumed_state`
+handles this — the `strategy` argument is **required** and must match
+the middleware's strategy:
```python
from cubepi.deferred import DeferredToolsMiddleware
@@ -241,6 +264,7 @@ from cubepi.deferred import DeferredToolsMiddleware
resumed = await DeferredToolsMiddleware.prepare_resumed_state(
groups=all_groups,
expanded=saved_extra["expanded_groups"],
+ strategy="dispatch",
)
agent = Agent(
@@ -254,41 +278,19 @@ agent = Agent(
| Field | Description |
|---|---|
-| `pre_loaded_tools` | Tools from previously-expanded groups, ready to use |
-| `remaining_groups` | Groups that were never expanded or only partially expanded |
-| `expanded_schemas` | Schema data for the system prompt (pass to `resumed_schemas` for advanced use) |
+| `pre_loaded_tools` | Tools from previously-loaded groups, ready to resolve (hidden from the payload in dispatch mode) |
+| `remaining_groups` | Groups still loadable via `load_tools` |
| `loader_cache` | Pre-loaded tool cache (pass to `resumed_loader_cache` to avoid redundant loader calls) |
-Fully expanded groups are loaded and removed from the deferred set.
-Partially expanded groups load the selected tools but stay deferrable
-(the model can still expand the rest).
-
-### Restoring schema text
-
-The `Agent(deferred_tool_groups=...)` shorthand handles the common case.
-For full prompt-cache continuity — where the resumed run's system prompt
-must match the previous run's final state byte-for-byte — construct the
-middleware directly with `resumed_schemas`:
-
-```python
-mw = DeferredToolsMiddleware(
- groups=resumed.remaining_groups,
- extra_ref=lambda: agent_extra,
- resumed_schemas=resumed.expanded_schemas,
- resumed_loader_cache=resumed.loader_cache,
-)
-
-agent = Agent(
- model=model,
- tools=[*builtin_tools, *resumed.pre_loaded_tools],
- middleware=[mw],
-)
-```
+In dispatch mode there is nothing else to restore: the schemas the model
+saw live in the message history, and the checkpointer brings them back
+with the conversation. In `inject` mode, fully loaded groups leave the
+deferred set (as in v1).
## Advanced: constructing the middleware directly
-For full control over the catalog header, cross-run schema seeding, or
-other middleware options, construct `DeferredToolsMiddleware` yourself:
+For full control over the catalog header or resume seeding, construct
+`DeferredToolsMiddleware` yourself:
```python
from cubepi.deferred import DeferredToolsMiddleware
@@ -296,8 +298,8 @@ from cubepi.deferred import DeferredToolsMiddleware
mw = DeferredToolsMiddleware(
groups=[github_group, linear_group],
extra_ref=lambda: agent_extra,
- catalog_header="# Available integrations\n\nExpand with load_tools().",
- resumed_schemas=None, # or pass schemas from a previous run
+ strategy="dispatch",
+ catalog_header="# Available integrations\n\nLoad with load_tools().",
)
agent = Agent(
@@ -313,14 +315,32 @@ agent = Agent(
|---|---|---|---|
| `groups` | `list[DeferredToolGroup]` | required | Groups to defer |
| `extra_ref` | `() -> dict` | required | Returns the live `ctx.extra` dict |
-| `catalog_header` | `str` | *(built-in)* | Header text for the catalog section |
-| `resumed_schemas` | `list[tuple[str, list[dict]]] \| None` | `None` | Schema data to seed from a previous run |
+| `strategy` | `"dispatch" \| "inject"` | `"dispatch"` | Disclosure strategy (see above) |
+| `catalog_header` | `str \| None` | *(strategy-specific built-in)* | Header text for the catalog section |
| `resumed_loader_cache` | `dict[str, list[AgentTool]] \| None` | `None` | Pre-loaded tool cache from a previous run (avoids re-calling loaders on resume) |
-| `on_tools_expanded` | `(list[AgentTool]) -> None \| None` | `None` | Called after new tools are expanded (used internally for cross-turn persistence) |
+| `on_tools_expanded` | `(list[AgentTool]) -> None \| None` | `None` | Called after new tools are loaded (used internally for cross-turn persistence) |
When using the `Agent(deferred_tool_groups=...)` shorthand, `extra_ref`
is automatically bound to `self._extra`.
+## Migrating from 0.10
+
+Deferred tool groups shipped in CubePi 0.10 with what is now the
+`inject` strategy. Upgrading changes behavior:
+
+- **The default strategy is now `dispatch`.** The catalog wording
+ changes, a `deferred_tool_call` builtin appears, and loaded tools no
+ longer join the model-visible tools array. Restore the 0.10 behavior
+ with `Agent(deferred_tool_strategy="inject")` or
+ `DeferredToolsMiddleware(strategy="inject")`.
+- **`inject` mode no longer renders schemas into the system prompt.**
+ The definitions were already in the tools array; the duplicate
+ rendering (and its double token billing) is gone. As a consequence,
+ the `resumed_schemas` constructor parameter and
+ `ResumedState.expanded_schemas` no longer exist.
+- **`prepare_resumed_state` requires `strategy=`** so a resume can't
+ silently mismatch the middleware's strategy.
+
## When to use it
**Good fit:**
@@ -340,7 +360,7 @@ is automatically bound to `self._extra`.
- [Loading MCP Tools](../mcp/loading) — how to get `AgentTool` lists from
MCP servers.
-- [The 8 Hooks](./hooks) — the middleware hooks that power deferred tools
- (`transform_system_prompt`, `after_tool_call`).
+- [The 9 Hooks](./hooks) — the middleware hooks that power deferred tools
+ (`transform_system_prompt`, `after_tool_call`, `resolve_tool_call`).
- [Composition](./composition) — how middleware composes when stacked with
other middleware.
diff --git a/website/docs/guides/middleware/hooks.md b/website/docs/guides/middleware/hooks.md
index bde1e27..f2159d1 100644
--- a/website/docs/guides/middleware/hooks.md
+++ b/website/docs/guides/middleware/hooks.md
@@ -1,11 +1,11 @@
---
-title: The 8 Hooks
-description: "Reference for the 8 middleware hooks in CubePi — transform_context, convert_to_llm, before_tool_call, after_tool_call, on_run_end, and more."
+title: The 9 Hooks
+description: "Reference for the 9 middleware hooks in CubePi — transform_context, convert_to_llm, resolve_tool_call, before_tool_call, after_tool_call, on_run_end, and more."
---
-# The 8 Hooks
+# The 9 Hooks
-`Middleware` is a class with up to eight optional async methods. Each
+`Middleware` is a class with up to nine optional async methods. Each
hook fires at a precise point in the agent loop. Implement only the
ones you need — CubePi only wires in the ones you override.
@@ -90,6 +90,30 @@ Fires **before each model call**, on the system prompt string. Use to:
Composition: chained.
+## `resolve_tool_call`
+
+```python
+async def resolve_tool_call(self, tool_call: ToolCall, *, context: AgentContext, signal=None) -> ToolCall | None:
+ ...
+```
+
+Fires **per tool call, before everything else** — argument validation,
+`before_tool_call`, execution, events, and tracing all operate on
+whatever this hook returns. Return a rewritten `ToolCall` to redirect
+the call, or `None` to pass through unchanged.
+
+The returned `ToolCall` **must keep the original `id`** — the tool
+result message is keyed by it on the wire.
+
+Use for: dispatcher unwrapping (this is how
+[deferred tool groups](./deferred-tools) route `deferred_tool_call` to
+the real tool), tool aliasing, version redirection.
+
+Composition: **first non-`None` wins** — the first middleware to return
+a rewritten call short-circuits the chain. This differs from
+`before_tool_call`, which chains sequentially; a resolver never sees
+another resolver's output.
+
## `before_tool_call`
```python
diff --git a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/composition.md b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/composition.md
index 7cd66c7..8e746a4 100644
--- a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/composition.md
+++ b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/composition.md
@@ -160,5 +160,5 @@ class JustTransform(Middleware):
## 另请参阅
-- [8 个 Hook](./hooks) —— 每个 hook 的作用及触发时机。
+- [9 个 Hook](./hooks) —— 每个 hook 的作用及触发时机。
- [示例](./examples) —— 组合在实践中的应用。
diff --git a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/deferred-tools.md b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/deferred-tools.md
index e0a3d30..8b71845 100644
--- a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/deferred-tools.md
+++ b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/deferred-tools.md
@@ -1,29 +1,53 @@
---
title: 延迟工具组
-description: "默认对模型隐藏 MCP 工具 schema,按需展开,减少上下文膨胀。"
+description: "默认对模型隐藏 MCP 工具 schema,按需加载——且不破坏 prompt 缓存。"
---
# 延迟工具组
当一个 agent 接入多个 MCP 服务器时,它们合起来的 tool schema 在每一轮都
能吃掉数千个 token 上下文——即使模型这一回合只需要其中一两组。
-`DeferredToolGroup` 用一份紧凑的目录替代完整 schema,让模型按需展开
+`DeferredToolGroup` 用一份紧凑的目录替代完整 schema,让模型按需加载
工具组。
-## 工作方式
+## 两种策略
-1. 构造时,agent 的 system prompt 里包含一份简短目录——每个组一行,
- 带描述和工具列表。
-2. 模型看到一个内置的 `load_tools` 工具,可以调用它来加载一组(或
- 组内的某些工具)。
-3. 展开时,loader 跑一次,工具被注入到运行中的 tool 集,schema 被
- 追加到 system prompt 末尾。
+延迟工具支持两种策略,由 `deferred_tool_strategy` 选择(默认
+`"dispatch"`):
+
+| | `tools` 参数 | system prompt | 每次加载的缓存代价 | 调用路径 |
+|---|---|---|---|---|
+| **`dispatch`**(默认) | 静态 | 静态 | **零**——schema 以消息尾部追加交付 | `deferred_tool_call` 转发器,引擎解包 |
+| **`inject`** | 随加载增长 | 目录计数变化 | system + 全部历史重读一次 | 原生工具调用 |
+
+**为什么默认 dispatch。** 在所有前缀缓存的 provider 上,工具定义渲染在
+prompt 最前端。会话中途注入工具等于在整个历史**之前**插入字节,每次加载
+都要按未缓存价重读全部对话。dispatch 模式在首个请求之后再不触碰 tools
+数组和 system prompt——schema 通过 `load_tools` 的工具结果交付,追加在
+消息历史末尾,像普通轮次一样增量缓存。
+
+**何时选 `inject`。** 原生工具调用享有 provider 侧 schema 校验,也是模型
+训练中最熟悉的调用方式。如果你的工具参数复杂、会话较短(单次加载的缓存
+代价小),`inject` 用缓存效率换取调用可靠性。
+
+## dispatch 模式如何工作
+
+1. system prompt 携带一份简短的**静态**目录——每组一行,含描述和工具
+ 名。它永不改变。
+2. 模型调用内置的 `load_tools(group_id)`,在**工具结果里**拿到该组的
+ 完整 schema。
+3. 模型通过内置转发器调用已加载的工具:
+ `deferred_tool_call(tool_name=..., arguments=...)`。
+4. 引擎在一切流程之前解包转发调用:参数校验、
+ `before_tool_call`/`after_tool_call` 钩子、权限系统、事件和 tracing
+ 看到的都是**真实**工具名和参数——而非信封。
```
# Deferred tool groups
These tool groups are available but not yet loaded. Call `load_tools(group_id)`
-to load a group's tools for the rest of this conversation.
+to get their full schemas, then invoke them via
+`deferred_tool_call(tool_name=..., arguments=...)`.
- `mcp:github` — GitHub: Issues, PRs, repos, code search (4 tools)
create_issue, search_repos, create_pr, list_comments
@@ -31,10 +55,19 @@ to load a group's tools for the rest of this conversation.
create_issue, update_issue, list_projects, ...
```
-## 基础用法
+几个值得了解的性质:
+
+- **隐式加载。** 模型对从未显式加载的工具直接调用
+ `deferred_tool_call` 时,中间件会即时加载并校验参数。校验失败时,
+ 错误结果附带完整 schema,模型一个来回即可自我纠正。
+- **压缩自救。** `load_tools` 幂等——若上下文压缩丢掉了旧结果,模型
+ 再调一次即可拿回字节相同的 schema。
+- **Fork。** fork 出的 agent(`fork_once`)继承 dispatch resolver,
+ 父 agent 已加载的工具在 fork 内仍可调用。
-把 `deferred_tool_groups` 传给 `Agent`。中间件会自动创建——不需要手动
-拼装:
+## 基本用法
+
+向 `Agent` 传入 `deferred_tool_groups`,中间件自动创建,无需手动接线:
```python
from cubepi import Agent
@@ -62,8 +95,9 @@ linear_group = DeferredToolGroup(
agent = Agent(
model=provider.model("claude-sonnet-4-6"),
- tools=[search_tool, calculator], # 始终可用的 tool
+ tools=[search_tool, calculator], # 始终可用的工具
deferred_tool_groups=[github_group, linear_group],
+ # deferred_tool_strategy="inject", # 选用 v1 行为
)
```
@@ -71,11 +105,11 @@ agent = Agent(
| 字段 | 类型 | 说明 |
|---|---|---|
-| `group_id` | `str` | 模型在 `load_tools` 调用里使用的唯一 ID(如 `"mcp:github"`) |
-| `display_name` | `str` | 目录里展示的人类可读名称 |
+| `group_id` | `str` | 模型在 `load_tools` 调用中使用的唯一标识(如 `"mcp:github"`) |
+| `display_name` | `str` | 目录中展示的人类可读标签 |
| `description` | `str` | 该组能力的一行摘要 |
| `tool_names` | `list[str]` | 目录里列出的 tool 名。**必须和 loader 返回的每个工具的 `AgentTool.name` 完全一致** —— 选择性展开(`load_tools(group_id, tool_names=[…])`)就靠这个字段匹配。 |
-| `loader` | `async () -> list[AgentTool]` | 返回该组完整 tool 集的回调 |
+| `loader` | `async () -> list[AgentTool]` | 返回该组完整工具集的回调 |
### 编写 loader
@@ -148,88 +182,70 @@ github_group = DeferredToolGroup(
## `load_tools` 工具
-模型通过 `load_tools` 加载一组的 tool。两种模式:
+模型调用 `load_tools` 加载一组工具,两种模式:
```
-# 展开整组
+# 加载整组
load_tools(group_id="mcp:github")
-# 只展开指定的 tool
+# 只加载指定工具
load_tools(group_id="mcp:github", tool_names=["create_issue", "search_repos"])
```
-工具返回结构化结果:
+dispatch 模式下结果携带完整 schema:
```json
{
"group_id": "mcp:github",
"expanded": true,
- "tool_names": ["create_issue", "search_repos", "create_pr", "list_comments"],
- "remaining": 0
+ "tool_names": ["create_issue", "search_repos"],
+ "remaining": 2,
+ "schemas": [
+ {"name": "create_issue", "description": "...", "parameters": {"...": "..."}},
+ {"name": "search_repos", "description": "...", "parameters": {"...": "..."}}
+ ]
}
```
-展开后这些 tool 同一轮就可以被模型调用(通过 `after_tool_call` hook)。
-
-### 选择性展开
-
-模型可以分批次展开一个组——现在要一两个,稍后再要更多:
-
-```
-load_tools(group_id="mcp:github", tool_names=["create_issue"])
-# → remaining: 3
-
-# 稍后……
-load_tools(group_id="mcp:github", tool_names=["search_repos"])
-# → remaining: 2
-```
+(`inject` 模式下省略 `schemas`——定义直接加入模型可见的 tools 数组。)
-已经展开过的 tool 是幂等的——再请求一次是 no-op。
+加载后工具在同一轮内即刻可用。
### Loader 缓存
-`loader` 回调在**每个组、每次 run** 里只会被调用**一次**。第一次
-`load_tools` 调用触发它;后续的选择性展开从缓存结果里筛选。如果 loader
-失败,错误会回给模型,组保持未展开。
-
-## Prompt 缓存稳定性
-
-System prompt 的设计目标是 prompt 缓存的前缀稳定:
-
-- **目录**按 `group_id` 字典序排序——输入顺序无关,渲染出来的文本
- 字节稳定。
-- **展开后的 schema** 按展开顺序(模型调用 `load_tools` 的顺序)追加
- 到末尾,从不重排。每次新展开只在尾部追加,保留已有的前缀。
+`loader` 回调每组每 run 恰好调用**一次**。首次加载触发;后续的选择性
+加载从缓存结果中过滤。loader 失败时错误返回给模型,该组保持未加载。
+已加载的工具幂等——重复请求是 no-op(dispatch 模式下会重新返回相同的
+schema)。
-这意味着 LLM API 的 prompt 缓存在轮次间一直有效:system prompt 只增长,
-而且只在末尾增长。
+## 加载状态
-## 展开状态
-
-中间件把哪些组已展开记录在 `ctx.extra` 里:
+中间件在 `ctx.extra` 里记录各组的加载情况:
```python
ctx.extra["expanded_groups"] = {
- "mcp:github": None, # 完全展开(None = 全部 tool)
- "mcp:linear": ["create_issue"], # 部分展开
- # mcp:slack 不在 = 未展开
+ "mcp:github": None, # 全量加载(None = 全部工具)
+ "mcp:linear": ["create_issue"], # 部分加载
+ # mcp:slack 不存在 = 未加载
}
```
-这份状态会跟着 checkpoint 一起持久化,可用于跨 run 恢复(见下文)。
+该状态随 checkpoint 持久化,驱动跨 run 重放。
-## 跨 run 恢复
+## 跨 run 重放
-从上一次 run 恢复对话时,你需要把展开状态还原,让模型有相同的工具
-可用。`prepare_resumed_state` 负责这件事:
+从上一个 run 恢复会话时,需要还原加载状态,让转发调用立即可解析。
+`prepare_resumed_state` 负责这件事——`strategy` 参数**必填**,且必须与
+中间件的 strategy 一致:
```python
from cubepi.deferred import DeferredToolsMiddleware
-# saved_extra 是上一次 run 持久化下来的 ctx.extra
+# saved_extra 是上一个 run 持久化的 ctx.extra
resumed = await DeferredToolsMiddleware.prepare_resumed_state(
groups=all_groups,
expanded=saved_extra["expanded_groups"],
+ strategy="dispatch",
)
agent = Agent(
@@ -239,42 +255,21 @@ agent = Agent(
)
```
-`prepare_resumed_state` 返回一个 `ResumedState`:
+`prepare_resumed_state` 返回的 `ResumedState` 包含:
| 字段 | 说明 |
|---|---|
-| `pre_loaded_tools` | 此前已展开组的 tool,已就绪可用 |
-| `remaining_groups` | 未展开或部分展开的组 |
-| `expanded_schemas` | 用于 system prompt 的 schema 数据(高级用法时传入 `resumed_schemas`) |
-| `loader_cache` | 已加载的 tool 缓存(传给 `resumed_loader_cache` 可避免重复调用 loader) |
-
-完全展开的组会被加载并从延迟集合中移除。部分展开的组会加载已选择的
-tool,但仍保留为可延迟(模型仍可展开余下部分)。
-
-### 还原 schema 文本
-
-`Agent(deferred_tool_groups=...)` 这种简写覆盖常见用例。如果要做完整的
-prompt 缓存连续性——也就是恢复后的 run 的 system prompt 必须和上一次
-最终状态字节一致——直接构造中间件并传 `resumed_schemas`:
-
-```python
-mw = DeferredToolsMiddleware(
- groups=resumed.remaining_groups,
- extra_ref=lambda: agent_extra,
- resumed_schemas=resumed.expanded_schemas,
- resumed_loader_cache=resumed.loader_cache,
-)
+| `pre_loaded_tools` | 之前已加载组的工具,随时可被解析(dispatch 模式下对 payload 隐藏) |
+| `remaining_groups` | 仍可通过 `load_tools` 加载的组 |
+| `loader_cache` | 预载的工具缓存(传给 `resumed_loader_cache` 避免重复调用 loader) |
-agent = Agent(
- model=model,
- tools=[*builtin_tools, *resumed.pre_loaded_tools],
- middleware=[mw],
-)
-```
+dispatch 模式下没有其他要恢复的东西:模型见过的 schema 留在消息历史
+里,由 checkpointer 随会话带回。`inject` 模式下,全量加载的组退出延迟
+集合(与 v1 相同)。
## 进阶:直接构造中间件
-如果要完全控制目录头、跨 run schema 种子或其他中间件参数,直接构造
+需要完全控制目录 header 或恢复种子时,自行构造
`DeferredToolsMiddleware`:
```python
@@ -283,8 +278,8 @@ from cubepi.deferred import DeferredToolsMiddleware
mw = DeferredToolsMiddleware(
groups=[github_group, linear_group],
extra_ref=lambda: agent_extra,
- catalog_header="# Available integrations\n\nExpand with load_tools().",
- resumed_schemas=None, # 或者传入上一次 run 的 schema
+ strategy="dispatch",
+ catalog_header="# Available integrations\n\nLoad with load_tools().",
)
agent = Agent(
@@ -296,36 +291,51 @@ agent = Agent(
### 构造参数
-| 参数 | 类型 | 默认 | 说明 |
+| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
| `groups` | `list[DeferredToolGroup]` | 必填 | 要延迟的组 |
-| `extra_ref` | `() -> dict` | 必填 | 返回当前的 `ctx.extra` dict |
-| `catalog_header` | `str` | *(内置)* | 目录区段的头文本 |
-| `resumed_schemas` | `list[tuple[str, list[dict]]] \| None` | `None` | 用于种子的 schema 数据 |
-| `resumed_loader_cache` | `dict[str, list[AgentTool]] \| None` | `None` | 上一次 run 的 tool 缓存(恢复时避免重复调 loader) |
-| `on_tools_expanded` | `(list[AgentTool]) -> None \| None` | `None` | 新 tool 展开后回调(内部用于跨轮持久化) |
-
-使用 `Agent(deferred_tool_groups=...)` 这种简写时,`extra_ref` 会自动
-绑到 `self._extra`。
+| `extra_ref` | `() -> dict` | 必填 | 返回实时的 `ctx.extra` 字典 |
+| `strategy` | `"dispatch" \| "inject"` | `"dispatch"` | 披露策略(见上文) |
+| `catalog_header` | `str \| None` | *(按策略内置)* | 目录段的 header 文本 |
+| `resumed_loader_cache` | `dict[str, list[AgentTool]] \| None` | `None` | 上一个 run 的工具缓存(避免恢复时重复调用 loader) |
+| `on_tools_expanded` | `(list[AgentTool]) -> None \| None` | `None` | 新工具加载后回调(内部用于跨轮持久化) |
+
+使用 `Agent(deferred_tool_groups=...)` 简写时,`extra_ref` 自动绑定到
+`self._extra`。
+
+## 从 0.10 迁移
+
+延迟工具组在 CubePi 0.10 发布时的行为即现在的 `inject` 策略。升级后
+行为变化:
+
+- **默认策略现在是 `dispatch`。** 目录措辞改变,出现 `deferred_tool_call`
+ 内置工具,加载的工具不再加入模型可见的 tools 数组。用
+ `Agent(deferred_tool_strategy="inject")` 或
+ `DeferredToolsMiddleware(strategy="inject")` 恢复 0.10 行为。
+- **`inject` 模式不再把 schema 渲染进 system prompt。** 定义本来就在
+ tools 数组里;重复渲染(及其双倍 token 计费)已移除。因此
+ `resumed_schemas` 构造参数和 `ResumedState.expanded_schemas` 不复存在。
+- **`prepare_resumed_state` 要求显式传 `strategy=`**,避免恢复时与中间件
+ 的策略静默错配。
## 何时使用
-**适用:**
+**适合:**
-- Agent 接入 5+ 个 MCP 服务器,但每次对话通常只用 1–2 组。
-- Tool schema 很大(参数多、描述长)。
+- agent 接入 5 个以上 MCP 服务器,但每次会话通常只用 1–2 个。
+- tool schema 很大(参数多、描述长)。
- 你希望跨轮保持高 prompt 缓存命中率。
-**不适用:**
+**不适合:**
-- Agent 只有少量 tool——目录和 `load_tools` 调用的开销不划算。
-- 每一轮都要全部 tool——延迟只多一次往返。
-- Tool schema 本身很小——上下文节省微乎其微。
+- agent 只有少量工具——目录和 `load_tools` 调用的开销不值得。
+- 每一轮都需要全部工具——延迟只是徒增一个来回。
+- tool schema 很小——上下文节省微乎其微。
-## 参见
+## 另请参阅
-- [加载 MCP 工具](../mcp/loading)——如何从 MCP 服务器拿到
- `AgentTool` 列表。
-- [8 个 Hook](./hooks)——驱动延迟工具的两个中间件 hook
- (`transform_system_prompt`、`after_tool_call`)。
-- [组合](./composition)——多个中间件叠加时怎么组合。
+- [加载 MCP 工具](../mcp/loading)——如何从 MCP 服务器获得 `AgentTool`
+ 列表。
+- [9 个 Hook](./hooks)——驱动延迟工具的中间件 hook
+ (`transform_system_prompt`、`after_tool_call`、`resolve_tool_call`)。
+- [组合](./composition)——与其他中间件叠加时的组合行为。
diff --git a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/hooks.md b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/hooks.md
index d2265c9..e2e6965 100644
--- a/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/hooks.md
+++ b/website/i18n/zh-Hans/docusaurus-plugin-content-docs/current/guides/middleware/hooks.md
@@ -1,11 +1,11 @@
---
-title: 8 个 Hook
-description: "CubePi 的 8 个中间件 hook 参考——transform_context、before_tool_call、after_tool_call、on_run_end 等。"
+title: 9 个 Hook
+description: "CubePi 的 9 个中间件 hook 参考——transform_context、resolve_tool_call、before_tool_call、after_tool_call、on_run_end 等。"
---
-# 8 个 Hook
+# 9 个 Hook
-`Middleware` 是一个最多包含八个可选异步方法的类。每个 hook 在 agent
+`Middleware` 是一个最多包含九个可选异步方法的类。每个 hook 在 agent
循环中的精确位置触发。只实现你需要的——CubePi 只会连接你重写了的方法。
```python
@@ -84,6 +84,26 @@ async def transform_system_prompt(
组合方式:链式。
+## `resolve_tool_call`
+
+```python
+async def resolve_tool_call(self, tool_call: ToolCall, *, context: AgentContext, signal=None) -> ToolCall | None:
+ ...
+```
+
+**每个工具调用时、先于其他一切**触发——参数校验、`before_tool_call`、
+执行、事件和 tracing 都基于这个 hook 的返回值运作。返回改写后的
+`ToolCall` 以重定向调用,返回 `None` 则原样通过。
+
+返回的 `ToolCall` **必须保留原始 `id`**——工具结果消息在线上以它为键。
+
+用于:dispatcher 解包([延迟工具组](./deferred-tools) 正是用它把
+`deferred_tool_call` 路由到真实工具)、工具别名、版本重定向。
+
+组合方式:**第一个非 `None` 返回值胜出**——第一个返回改写结果的中间件
+会短路整个链。这与顺序链式的 `before_tool_call` 不同;一个 resolver
+永远看不到另一个 resolver 的输出。
+
## `before_tool_call`
```python