Self-improving AI Employees for everyone
Tool Building Pipeline
Canonical home for how AI Agents discover connected tool nodes, build
provider-neutral AgentToolSpec / ToolDef values, expose them through the
native ChatUnifier, and dispatch tool calls back to the right handler.
Replaces the partial explanations previously scattered across
agent_architecture.md and agent_delegation.md.
Related docs:
- agent_architecture.md — overall agent loop that owns the current tool surface
- agent_delegation.md —
delegate_to_*tools + taskTrigger / input-task patterns- TEMPORAL_ARCHITECTURE.md — per-type Temporal activity dispatch (F4.A) that backs tool calls when the flag is on
- node_creation.md — recipe for adding a new tool node
- pi_coding_agent.md — Pi tool callbacks, automatic workspace bindings and receipt reconciliation
Coding Agent exposes this same canonical tool catalogue to Pi. SDK callbacks reference recorded call IDs and frozen bindings; the trusted dispatcher reads the original arguments and applies existing validation/policy. Workspace mode fills missing filesystem/shell types through virtual bindings and a dedicated Activity, without creating canvas nodes or node output records.
1. Five stages
1. DISCOVER plugin/edge_walker.py:collect_agent_connections
→ reads edges → tool_data list
│
▼
2. BUILD services/ai.py:AIService._build_tool_from_node
→ tool_data entry → (AgentToolSpec, config_dict)
│
▼
3. EXPOSE services/agent_runtime.py:run_native_agent_loop
→ ToolDef list → ChatUnifier → provider schema compiler
│
▼
4. INVOKE LLM emits tool_calls in its response
→ loop dispatches each via tool_executor
│
▼
5. DISPATCH services/handlers/tools.py:execute_tool
→ owns node lifecycle broadcasts
→ _dispatch_tool routes by node_type to a handler
2. Stage 1 — Discovery (plugin/edge_walker.py)
collect_agent_connections(node_id, context, database) walks the workflow edges and returns a 5-tuple per agent:
(context_data, skill_data, tool_data, input_data, task_data)
(edge_walker.py:181-199 — the first element is the Context descriptor from
input-context, or the legacy Memory descriptor on input-memory graphs.)
tool_datais a list of dicts. Each entry:{node_id, node_type, parameters, label}. (Theconnected_servicesextra belonged to the retiredandroidToolgateway tool — see §7.)- MasterSkill expansion runs at discovery time: a single
masterSkillconnection withskillsConfigexpands into N individual skill entries, one per enabled skill key. - Team-lead expansion (
orchestrator_agent/ai_employee): agents connected toinput-teammatesbecome authorized descriptors. Internaldelegate_to_*identities remain available to the coordinator, while the LLM delegates via Task Manager.
3. Stage 2 — Build (services/ai.py:_build_tool_from_node)
AIService._build_tool_from_node(tool_info)
returns (AgentToolSpec, config_dict) or (None, None) on failure.
3a. Tool name resolution
The LLM-visible name and description come from the plugin rather than a central map:
- A per-node
delegate_tool_namewins for expanded teammate/delegation identities. - A database
ToolSchemaoverride wins when present. - Otherwise declared node parameters (
tool_name/tool_description) may override the defaults. - Plugin class variables
tool_name/tool_descriptionprovide the normal default;tool_descriptionfalls back to the plugin's human-facingdescription. - Built-in pseudo-tools such as
Skillandcheck_delegated_tasksuse the small_PSEUDO_TOOL_FALLBACKtable. - The last resort is
tool_<label>/Execute <label>.
The resolved name is normalized to alphanumeric characters and underscores. Duplicate visible names are rejected before the first billed model call.
3b. Schema resolution
schema = self._get_tool_schema(node_type, schema_params)
_get_tool_schema returns a Pydantic BaseModel
subclass.
| Path | Source | When |
|---|---|---|
| Database override (preferred) | ToolSchema table (server/models/database.py) via db.get_tool_schema(node_id) |
User customized the schema in the Tool Schema Editor UI |
| Plugin schema (normal) | Registered plugin class's Pydantic Params model |
No DB override |
| Special built-in schema | Delegation, Skill, and check_delegated_tasks definitions in _get_tool_schema |
The LLM invocation contract intentionally differs from a node's configuration model, or no plugin exists |
| Generic fallback | One required string input field |
Unknown non-plugin node type |
ToolSchema.schema_config is JSON of {description, fields: {name: {type, description, required}}}. CRUD handlers in routers/websocket.py: get_tool_schema, save_tool_schema, delete_tool_schema, get_all_tool_schemas.
3c. AgentToolSpec construction
config = {
"node_id": node_id,
"node_type": node_type,
"parameters": node_params,
"label": node_label,
"connected_services": connected_services,
}
parameters = inline_schema_refs(schema.model_json_schema())
tool = AgentToolSpec(
definition=ToolDef(
name=tool_name,
description=tool_description,
parameters=parameters,
),
args_schema=schema,
execution=config,
)
ToolDef is the provider-facing declaration. AgentToolSpec retains the
Pydantic validation model and local execution metadata. The returned config
is augmented by the agent's callback with services and graph context before it
reaches execute_tool.
4. Stage 3 — Expose (run_native_agent_loop)
services/ai.py:execute_agent calls
run_native_agent_loop(ChatUnifier, ..., tools=tools). On each LLM step,
run_native_llm_step passes only the ToolDef declarations to
ChatUnifier.chat. The selected native provider compiles
ToolDef.parameters into its SDK's tool schema; there is no mutable
chat-model binding object.
Provider-specific handling for nullable values, unions, enums, nested objects, and unsupported keywords remains inside the native provider layer. See native_llm_sdk.md for the OpenAI / Anthropic / Gemini differences.
5. Stage 4 — Invoke (LLM response handling)
When the native LLMResponse contains tool calls,
run_native_agent_loop first appends the exact replayable
assistant_message, then validates and dispatches each call through the
supplied tool_executor callback. Inside the callback:
- Broadcast
executing_toolto the parent agent (phase broadcast, not the tool node's lifecycle). from services.handlers.tools import execute_toolresult = await execute_tool(tool_name, tool_args, config)— this owns the tool node's lifecycle.- Broadcast
tool_completedto the parent agent. - Return
resultto the loop → serialized into a nativeMessage(role="tool", tool_call_id=...), appended tomessages, and fed back to the LLM on the next turn.
Malformed JSON arguments retain raw_arguments plus a parse error. The loop
returns a deterministic tool-error result to the model instead of invoking the
tool or crashing.
Single-source-of-truth rule. execute_tool owns executing / success / error broadcasts for the tool node. The parent-agent closure in services/ai.py only emits executing_tool / tool_completed phase events for the parent agent. Don't duplicate either.
6. Stage 5 — Dispatch (handlers/tools.py)
execute_tool(tool_name, tool_args, config) in services/handlers/tools.py:
1. broadcaster.update_node_status(node_id, "executing", ...)
2. result = await _dispatch_tool(tool_name, tool_args, config)
3a. If result.status in {"delegated", "ALREADY_DELEGATED"}: handler owns
its own success/error lifecycle (delegated agents broadcast from
inside their background task). Skip terminal broadcast.
3b. Else: broadcaster.update_node_status(node_id, "success", {result})
4. Return result. Exceptions trigger an "error" broadcast and re-raise.
_dispatch_tool is pure routing — no broadcasting. It checks config["node_type"] in this order:
| Order | Pattern | Handler |
|---|---|---|
| 1 | _builtin_skill |
services.skill_runtime.execute_skill_tool |
| 2 | Any registered plugin that is not a registered agent (get_node_class(node_type)) |
The plugin fast path: plugin_cls().execute_as_tool(tool_args, node_params, ctx). This covers every tool plugin, dual-purpose usable_as_tool nodes, the Android services, taskManager and proxyConfig. A plugin that needs to adjust the model's arguments overrides execute_as_tool (see §12). |
| 3 | ANDROID_SERVICE_NODE_TYPES, taskManager, proxyConfig |
Legacy branches (execute_android_service_tool, _execute_task_manager, execute_proxy_config). Each of these types is a registered plugin, so step 2 handles it first and these branches are not reached today. |
| 4 | _builtin_check_delegated_tasks |
_execute_check_delegated_tasks |
| 5 | A registered agent (is_registered_agent: AI_AGENT_TYPES or supports_delegation) |
_execute_delegated_agent (fire-and-forget background task) |
| 6 | Anything else | _execute_generic (catch-all, logs a warning) |
7. Gateway-tool pattern (retired)
androidTool was the only gateway tool — a single LLM-visible
android_device declaration that fanned out to multiple connected Android
service nodes via a connected_services list. The node no longer exists:
Android service plugins expose their own tool_name class variables and
connect directly to input-tools. Legacy
service -> androidTool -> agent graphs are rewritten on load by
services/workflow_migrations.normalize_legacy_android_toolkit, and sub-node
exclusion keys solely on the AI-agent config handles (input-context /
input-memory (legacy compatibility) / input-tools / input-skill /
input-teammates; ExecutionContext.create in execution/models.py) — the
TOOLKIT_NODE_TYPES constant is gone.
8. Internal delegation identities (delegate_to_*)
For team leads these identities are hidden from the model. Task Manager first persists and authorizes an assignment, then the Temporal or legacy coordinator uses the matching identity to dispatch the child. Direct calls described below apply only to compatibility/non-team-lead paths.
Specialized agents connected to an agent's input-tools handle become delegation tools. Schema is fixed:
class DelegateToAgentSchema(BaseModel):
task: str
context: Optional[str] = None
_execute_delegated_agent in services/handlers/tools.py:
- Generate a
task_id(uuid4().hex). - Spawn
asyncio.create_task(<child agent execute>)— fire and forget. - Track in
_active_delegated_nodesso repeated calls during in-flight delegations short-circuit. - Return
{"status": "delegated", "task_id": task_id}IMMEDIATELY (parent agent continues without waiting). - From inside the background task: broadcast the child's
executing/success/errortimeline + emit atask_completedCloudEvents event when the child finishes (success or error).
Parent agents see results via three paths — see agent_delegation.md for the full pattern matrix.
9. Per-type Temporal dispatch (F4.A)
When TEMPORAL_PER_TYPE_DISPATCH=true and the run is happening inside a Temporal workflow, tool calls do NOT go through execute_tool directly. Instead:
agent.prepare_payloadbuilds the payload; there is one wire standard with nollm_engine/message_wire_versiondiscriminators (locked bytests/llm/test_single_wire_standard.py). New controlled generations also record resolved tool bindings and plugin Activity policies here.AgentWorkflow(the F4.B child workflow) gets the LLM's tool-calls list back fromagent.execute_llm_step, whose native branch sendsToolDefdeclarations throughChatUnifier.- For each tool call, the workflow schedules
f"node.{node_type}.v{version}"as a Temporal activity on the plugin's declaredtask_queuewhen the run's frozen worker-pool setting is enabled; otherwise the default worker handles the Activity. - The per-type activity body in
server/services/plugin/base.py:as_activityruns the same pipelineexecute_toolwould have — broadcasts, pluginexecute()call, terminal broadcast — but inside the activity boundary. - The result returns to the workflow, is appended to the native message history, and the next LLM step sees it.
Tool calls inside an agent workflow have their own regular Activity boundary.
For new generations (execution_control_version=1), ordinary plugin tools use
the recorded start_to_close_timeout, heartbeat_timeout, retry_policy,
and queue declaration. Heartbeats enforce liveness; they do not extend an
attempt's start_to_close_timeout or checkpoint an opaque provider call.
The parent is a workflow, so it has no Activity start_to_close_timeout.
Existing histories retain their previous policy decisions. See
Temporal Architecture and
Node creation.
Stop at the next pending tool
The workflow closes admission before each model request, tool, compaction, refresh, or child start. An admitted tool finishes under its recorded retry policy, and its result/transcript bookkeeping completes even while stopped. For a recorded response requesting A then B, Stop during A preserves A's result and B's pending call; Resume executes B once without repeating that response or A. Already-admitted parallel tools drain concurrently. Permit waits and child-result waits are excluded from local drain acknowledgement; the attached child is controlled through the execution tree instead.
Clean Continue-As-New boundaries carry the prepared bindings, transcript,
thinking, iteration, usage, execution identity, and control revision. Stopped
agents park before rollover. If existing compaction cannot bring the complete
continuation input under the limit, the versioned path raises
AgentContinuationTooLarge rather than returning to the opening prompt.
Do not implement a second tool pause flag, blanket heartbeat cursor, or
rescheduling loop for already-recorded results. External effects before an
Activity's completion is recorded still depend on the tool's idempotency and
retry contract. Opaque Claude Code and RLM agents have a
whole-Activity Stop boundary; independently approved sends and Workspace
tasks retain their own lifecycle. The full topology and messaging protocol
is in Workflow control.
The v1/v2 wire duality was purged; there is no engine-marker refusal path.
10. Auto-skill edges
Some tools bundle a default skill — writeTodos ships with write-todos-skill, the WhatsApp tools ship with their respective skills. The frontend useAutoSkillEdges.ts hook detects these connections at canvas-edit time and auto-creates a phantom skill connection so the LLM gets both the tool schema (for invocation) and the skill instructions (for usage guidance) without the user having to wire two edges.
The isMasterSkillNode() helper in useAutoSkillEdges.ts resolves via getCachedNodeSpec(type)?.uiHints?.isMasterSkillEditor === true, not a hardcoded string match.
11. Pytest invariants
server/tests/
├── test_status_broadcasts.py # locks execute_tool's lifecycle broadcast contract
├── credentials/test_credential_broadcasts.py # auth tools' CloudEvents envelope
├── nodes/test_ai_agents.py # end-to-end agent + tool exercise
└── temporal/test_dispatch.py # F4.A per-type activity dispatch (when flag on)
_LEGACY_RAW_DICT_CALLSITES in test_status_broadcasts.py is closed — new tool implementations cannot emit raw-dict broadcasts. Use WorkflowEvent factories + StatusBroadcaster wrappers; see status_broadcaster.md.
12. Adding a new tool node
The shortest path:
- Create the plugin folder:
server/nodes/tool/<plugin>/__init__.pyextendingToolNode(orActionNodewithusable_as_tool = Truefor dual-purpose). - Declare
type,display_name,Params(the schema the LLM sees),Output, and an@Operation-decorated method (for example@Operation("search") async def search(self, ctx, params));BaseNode.executeandexecute_as_tooldispatch to it. - Declare the plugin's
tool_nameand, when the LLM-facing wording differs,tool_descriptionclass variables. - (Optional) Add a special schema clause to
_get_tool_schemaonly when the plugin'sParamsmodel is not the correct invocation contract. - (Optional) List settings the model must never choose (an account, a
profile, a mount) in
server_controlled_fields. The framework keeps them out of the model's reach on every path:execute_as_toolstrips them from the model's arguments, andNodeExecutorrestores the saved value on the Temporal path. Overrideexecute_as_toolonly when the model's arguments need some other adjustment. Do not add a clause to_dispatch_tool: its plugin fast path already runsexecute_as_toolfor every registered plugin.
Steps 4-5 are usually unnecessary — most new tools are fully described by their plugin class.
See node_creation.md for the broader recipe and server/nodes/README.md for the 5-minute walkthrough.
13. What NOT to do
- Don't broadcast
executing/success/errorfrom inside_dispatch_toolor a handler —execute_toolowns the tool node's lifecycle. Returning{"status": "delegated"}is the documented opt-out. - Don't bypass
_build_tool_from_nodeto hand-craft anAgentToolSpecorToolDef. Schema dereferencing, validation metadata, execution routing, and duplicate-name checks are load-bearing. - Don't expose a tool without a real Pydantic invocation schema. An empty schema makes argument generation unreliable.
- Don't catch exceptions inside the tool callback and return them as
{"error": ...}— re-raise soexecute_toolcan broadcasterror. The LLM-sees-tool-errors-and-continues pattern depends on this. - Don't store per-execution state on
AIService(it's a singleton). Useconfigor per-call closures.