Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

docs(brain): two AI agent gotchas from the task-agent and eval fixes (#16445)

Hazem Adel36f41f234a18
diff --git a/brain/knowledge/ai-intelligence/ai-agents.md b/brain/knowledge/ai-intelligence/ai-agents.md
index 80d125f20d6..d31ed35a2b9 100644
--- a/brain/knowledge/ai-intelligence/ai-agents.md
+++ b/brain/knowledge/ai-intelligence/ai-agents.md
@@ -102,6 +102,8 @@ Paths verified 2026-07-17. An earlier version pointed at `packages/core/shared/s
- **Read a chat or saved-agent tool name only through `agentToolSkills.effectiveToolCall`.** Those surfaces run with skills (`agentToolSkills.surfaceFor`), so most calls arrive as `ap_lazy_tool({ tool, input })` and any check on `toolName` sees the wrapper. Billing (`completedToolCalls`), the build-phase switch, `buildStepParts`, `collapseStaleToolOutputs` pinning and the live stream (`unwrapLazyToolChunk`) all unwrap first. A new name-keyed check that skips this silently treats every deferred tool as `ap_lazy_tool`. Never change `tools` or `activeTools` mid-conversation either, or the prompt cache is rebuilt from scratch.
- **The catalog of deferred tools is rendered in the worker, not the API.** Configured and MCP tool names are only final after `agentToolPolicy.withValidNames`, so a catalog the API built from raw names would point the model at tools that do not exist.
+- **`ap_load_skill` takes the skill's name, not its guide topic.** The build skill is `flow_building` and its guide topic is `build_flow`; a prompt that tells the model to load `build_flow` gets the call rejected and the build guide never arrives. Name skills by `AGENT_SKILLS[].name` in every prompt, including the task-agent prompt.
+- **Every agent turn fetches the model catalog over the network unless it is already cached.** Production warms it at worker start; tests do not, so an eval that runs a turn waits on the CDN and times out at random when it is slow. Worker tests point `AP_MODEL_CATALOG_URL` at an `.invalid` host in `vitest.setup.ts`, so run any new agent-turn test under the worker's vitest config.
- **A tool whose arguments the model guesses must be sent directly, not behind `ap_lazy_tool`.** Through the wrapper the provider no longer checks a tool's input, so a model that never loaded the schema guesses it: the connection picker was called with `label` instead of `displayName`, which cost retries and crashed the web card that read `piece` from the input before the dispatcher rejected it. The connection cards, auth discovery, the question card, piece props and dropdown resolution are therefore in `CORE_TOOL_NAMES`; only rarely used tools stay deferred.
- **Paid tools are never sent directly in skills mode; they always go through `ap_lazy_tool`.** `buildSkillSurface` drops every `chatBilling.isPaidTool` name from `coreToolNames`, even for a small tool set, and `CORE_TOOL_NAMES` holds none (a test pins it). An unaffordable call is then refused in the wrapper's `validate`, which neither runs nor bills it. Hiding the tool from `activeTools` once credits ran low changed the tool list mid-turn and lost the prompt cache, and refusing inside `execute` by returning `{ error }` still counted as a completed paid call, so billing charged for work that never ran.
- **A rule that must hold in every chat belongs in the system prompt, not a guide.** Guides arrive only when their skill is loaded. When "never publish without a yes" lived only in `build_flow`, a follow-up chat ("Nice, thanks.") published the flow and called it live.