Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs(internal): fix Wave-11 path drift, stale counts, and broken links across docs-internal
trohitgcbac075cc507
diff --git a/docs-internal/credentials_scaling/architecture.md b/docs-internal/ARCHIVE/credentials_scaling/architecture.mdsimilarity index 97%rename from docs-internal/credentials_scaling/architecture.mdrename to docs-internal/ARCHIVE/credentials_scaling/architecture.mdindex 4628351f..dbcac027 100644--- a/docs-internal/credentials_scaling/architecture.md+++ b/docs-internal/ARCHIVE/credentials_scaling/architecture.md@@ -1,3 +1,5 @@+> **ARCHIVED / HISTORICAL (superseded).** This is a point-in-time research/exploration record from before the antd removal. It describes keeping antd inputs via a react-hook-form `Controller`, which never shipped — the live UI is fully shadcn/ui (antd was removed in `ui_migration_plan.md` Phase 7). Kept for rationale only; do not treat as current state.+# Credentials Panel Architecture (Scalable to 5000 Providers)The architecture for the MachinaOs credentials panel after scaling work.@@ -6,9 +8,9 @@ layers: server-owned registry, bulk-fetch cache, command palette, lazy panels,and react-hook-form detail panels.See also:-- [research_production_platforms.md](./research_production_platforms.md) — evidence base from n8n, Nango, Pipedream, Zapier, Supabase+- `research_production_platforms.md` (not in repo) — evidence base from n8n, Nango, Pipedream, Zapier, Supabase- [research_react_stack.md](./research_react_stack.md) — library choices and benchmarks-- [typed-splashing-crown.md](../../../../.claude/plans/typed-splashing-crown.md) — execution plan+- `.claude/plans/typed-splashing-crown.md` (developer-local, not in repo) — execution plan---@@ -290,7 +292,7 @@ MachinaOs uses JSON for all backend config (`email_providers.json`, `llm_default## Files touched-See [typed-splashing-crown.md](../../../../.claude/plans/typed-splashing-crown.md)+See `.claude/plans/typed-splashing-crown.md` (developer-local, not in repo)section "Critical files to modify/create" for the exhaustive list.Summary:diff --git a/docs-internal/credentials_scaling/research_react_stack.md b/docs-internal/ARCHIVE/credentials_scaling/research_react_stack.mdsimilarity index 98%rename from docs-internal/credentials_scaling/research_react_stack.mdrename to docs-internal/ARCHIVE/credentials_scaling/research_react_stack.mdindex 1dbc79c2..dcf7ea7d 100644--- a/docs-internal/credentials_scaling/research_react_stack.md+++ b/docs-internal/ARCHIVE/credentials_scaling/research_react_stack.md@@ -1,3 +1,5 @@+> **ARCHIVED / HISTORICAL (superseded).** This is a point-in-time research/exploration record from before the antd removal. It describes keeping antd inputs via a react-hook-form `Controller`, which never shipped — the live UI is fully shadcn/ui (antd was removed in `ui_migration_plan.md` Phase 7). Kept for rationale only; do not treat as current state.+# Research: React Stack for a Scalable Credentials PanelBenchmarks, bundle sizes, and recommendations for the React libraries that powerdiff --git a/docs-internal/agent_architecture.md b/docs-internal/agent_architecture.mdindex 1364a5d1..db98c0cc 100644--- a/docs-internal/agent_architecture.md+++ b/docs-internal/agent_architecture.md@@ -345,7 +345,7 @@ Patterns covered in the canonical doc:## Memory Integration-Memory load + save lives in [memory_lifecycle.md](./memory_lifecycle.md). The agent loop reads `memory_data` from `_collect_agent_connections()` (via the `input-memory` edge), prepends parsed history to the system message + current prompt, runs `_run_agent_loop`, then appends the turn + trims the window + archives trimmed text to the vector store. The markdown helpers (`parse_memory_markdown`, `append_to_memory_markdown`, `trim_markdown_window`) and the vector store (`InMemoryVectorStore` with HuggingFace embeddings) are the load-bearing surface — see the canonical doc for signatures, the markdown format, and the engine-specific adapter table (aiAgent / rlm_agent / claude_code_agent native session resume bridge).+Memory load + save lives in [memory_lifecycle.md](./memory_lifecycle.md). The agent loop reads `memory_data` from `collect_agent_connections()` (`server/services/plugin/edge_walker.py`, via the `input-memory` edge), prepends parsed history to the system message + current prompt, runs `_run_agent_loop`, then appends the turn + trims the window + archives trimmed text to the vector store. The markdown helpers (`parse_memory_markdown`, `append_to_memory_markdown`, `trim_markdown_window`) and the vector store (`InMemoryVectorStore` with HuggingFace embeddings) are the load-bearing surface — see the canonical doc for signatures, the markdown format, and the engine-specific adapter table (aiAgent / rlm_agent / claude_code_agent native session resume bridge).---@@ -362,26 +362,47 @@ Both methods live in `server/services/ai.py` and follow the same general pattern### Specialized Agent Routing-There are **11 specialized agents** plus 2 team leads. Most route to `handle_chat_agent`; `rlm_agent` and `claude_code_agent` have dedicated handlers:+There is no hardcoded routing table for agents. Wave 11.C deleted the+`SPECIALIZED_AGENT_TYPES → partial(handle_chat_agent, ...)` registry that+node_executor used to build by hand. Every agent variant is now a+self-contained plugin folder under `server/nodes/agent/<plugin>/` whose+`BaseNode` subclass registers itself into `services.node_registry`+(`_HANDLER_REGISTRY`) at import time via `register_node(...)`. `NodeExecutor._dispatch()`+does a single registry lookup — `self._handlers.get(node_type)` — and the+plugin handler that won the merge runs the node's `@Operation("execute")`+`execute_op` method (`server/services/node_executor.py:_dispatch`,+`_build_handler_registry`).-```python-# server/services/node_executor.py-SPECIALIZED_AGENT_TYPES = {- 'android_agent', 'coding_agent', 'web_agent', 'task_agent', 'social_agent',- 'travel_agent', 'tool_agent', 'productivity_agent', 'payments_agent', 'consumer_agent',- 'autonomous_agent', 'orchestrator_agent', 'ai_employee',- # rlm_agent and claude_code_agent are handled by dedicated handlers, not handle_chat_agent-}--# Most specialized agents map to handle_chat_agent (uses _run_agent_loop)-for agent_type in SPECIALIZED_AGENT_TYPES:- registry[agent_type] = partial(handle_chat_agent, ai_service=self.ai_service, database=self.database)+Each agent's `execute_op` calls the matching `AIService` method:-# Dedicated handlers for agents with externalised session state-registry['rlm_agent'] = partial(handle_rlm_agent, ai_service=self.ai_service, database=self.database)-registry['claude_code_agent'] = partial(handle_claude_code_agent, ...)+```python+# server/nodes/agent/_specialized.py — every specialized agent + team lead+from ._inline import prepare_agent_call+kwargs = await prepare_agent_call(...) # collect_agent_connections + prompt/task prep+response = await ai_service.execute_chat_agent(ctx.node_id, **kwargs)++# server/nodes/agent/ai_agent/__init__.py -> ai_service.execute_agent(...)+# server/nodes/agent/chat_agent/__init__.py -> ai_service.execute_chat_agent(...)+# server/nodes/agent/rlm_agent/__init__.py -> ai_service.rlm_service.execute(...)+# server/nodes/agent/claude_code_agent/__init__.py / codex_agent/__init__.py+# -> AICliService (CLI agent runtime) via the plugin's own execute_op```+`prepare_agent_call` (in `server/nodes/agent/_inline.py`) chooses+`execute_agent` vs `execute_chat_agent` based on `self.type` and shapes the+shared keyword bundle; `_specialized.py` is the single body shared by all+chat-agent-path specialized agents. Glob `server/nodes/agent/` for the+authoritative agent list, or read `AI_AGENT_TYPES` in+[`server/constants.py`](../server/constants.py) — do not hand-maintain a+count here. As of this writing `AI_AGENT_TYPES` lists 17 entries: the two+base agents (`aiAgent`, `chatAgent`), 11 specialized agents+(`android_agent`, `coding_agent`, `web_agent`, `task_agent`, `social_agent`,+`travel_agent`, `tool_agent`, `productivity_agent`, `payments_agent`,+`consumer_agent`, `autonomous_agent`) plus the 2 team leads+(`orchestrator_agent`, `ai_employee`), and the 2 CLI/REPL agents that bypass+the standard loop (`rlm_agent`, `claude_code_agent`). `codex_agent` is a+sibling plugin folder on the same CLI-agent path.+### Temporal dispatch routingTwo settings flags route agent execution through different Temporal paths (see [TEMPORAL_ARCHITECTURE.md](TEMPORAL_ARCHITECTURE.md) for the full matrix):@@ -393,7 +414,12 @@ Two settings flags route agent execution through different Temporal paths (see [Both flags default to `true` in `.env.template`.-**Team leads** (`orchestrator_agent`, `ai_employee`) use the same `handle_chat_agent` routing but add an `input-teammates` handle. Connected agents become `delegate_to_<type>` tools automatically via `_collect_teammate_connections()`. See [agent_teams.md](agent_teams.md).+**Team leads** (`orchestrator_agent`, `ai_employee`) run the same+`execute_chat_agent` path as the other specialized agents but add an+`input-teammates` handle. Connected agents become `delegate_to_<type>` tools+automatically via `collect_teammate_connections()` (in+`server/services/plugin/edge_walker.py`, called from+`server/nodes/agent/_inline.py:prepare_agent_call`). See [agent_teams.md](agent_teams.md).### RLM Agent Pattern@@ -414,14 +440,10 @@ Exposed helpers inside the REPL:| `rlm_query(prompt)` | Recursively invoke the same RLM agent || `FINAL(answer)` | Signal completion and return the final answer |-Routing:--```python-# server/services/node_executor.py-registry['rlm_agent'] = partial(handle_rlm_agent, ai_service=self.ai_service, database=self.database)-```--`handle_rlm_agent` delegates to `RLMService` in `server/services/rlm_service.py`. See [rlm_service.md](rlm_service.md) for full details.+Routing: `rlm_agent` is a plugin folder like every other agent. Its+`execute_op` (`server/nodes/agent/rlm_agent/__init__.py`) delegates to+`ai_service.rlm_service.execute(...)` rather than `_run_agent_loop`. See+[rlm_service.md](rlm_service.md) for full details.### Chat Agent Conditional Loop@@ -491,9 +513,9 @@ Zeenie loads SKILL.md instructions from connected skill nodes, builds tools from## Handle Topology-Handle layouts are declared in `server/nodes/agent/_handles.py` (local to `nodes/agent/`, not a global lookup). Two layouts cover the 20 agent variants:+Handle layouts are declared in `server/nodes/agent/_handles.py` (local to `nodes/agent/`, not a global lookup). Two layouts cover every agent variant (glob `server/nodes/agent/` for the live count):-**`std_agent_handles()`** — shared by 18 of 20 agent variants:+**`std_agent_handles()`** — shared by every agent variant except the two team leads:| Handle | Kind | Position | Offset | Role ||---|---|---|---|---|@@ -505,7 +527,7 @@ Handle layouts are declared in `server/nodes/agent/_handles.py` (local to `nodes| `output-main` | output | right | 50% | main || `output-top` | output | top | — | main |-**`team_lead_agent_handles()`** — `orchestrator_agent` and `ai_employee` only; adds an `input-teammates` handle (bottom, 80%) and shifts the bottom skill/tool offsets to 20%/50%. Connected agents become `delegate_to_<type>` tools via `_collect_teammate_connections()`. Team leads are also listed in `TEAM_LEAD_TYPES` in [`client/src/components/TeamMonitorNode.tsx`](../client/src/components/TeamMonitorNode.tsx) (frontend tribal array, flagged as tech debt). See [agent_teams.md](agent_teams.md).+**`team_lead_agent_handles()`** — `orchestrator_agent` and `ai_employee` only; adds an `input-teammates` handle (bottom, 80%) and shifts the bottom skill/tool offsets to 20%/50%. Connected agents become `delegate_to_<type>` tools via `collect_teammate_connections()` (in `server/services/plugin/edge_walker.py`). Team leads are also listed in `TEAM_LEAD_TYPES` in [`client/src/components/TeamMonitorNode.tsx`](../client/src/components/TeamMonitorNode.tsx) (frontend tribal array, flagged as tech debt). See [agent_teams.md](agent_teams.md).`AIAgentNode.tsx` is type-agnostic: it calls `useNodeSpec(type)` and renders whatever handles / icon / color the spec returns — there is no `AGENT_CONFIGS` map (retired Wave 10.D). Specialized-agent visuals all come from the NodeSpec; per-type display name / subtitle / description live on the plugin class, icon from `<plugin>/icon.svg` (visuals.json emoji fallback), color from `<plugin>/meta.json`. Glob `server/nodes/agent/` for the authoritative agent list — do not hand-maintain one.diff --git a/docs-internal/agent_delegation.md b/docs-internal/agent_delegation.mdindex d59fd142..6ef574df 100644--- a/docs-internal/agent_delegation.md+++ b/docs-internal/agent_delegation.md@@ -24,17 +24,19 @@ Parent Agent (chatAgent / aiAgent / specialized)|vtool_executor callback (closure in ai.py)- | Injects: ai_service, database, nodes, edges, workflow_id+ | Injects: ai_service, database, nodes, edges, workflow_id,+ | parent_node_id, execution_id|vexecute_tool() dispatcher (handlers/tools.py)- | Routes agent node_types to _execute_delegated_agent()+ | Routes node_type in AI_AGENT_TYPES to _execute_delegated_agent()|v_execute_delegated_agent() (handlers/tools.py)| 1. Fetches child_params from DATABASE- | 2. Injects API key from credential store- | 3. OVERWRITES prompt with delegated task+ | 2. Injects API key + default model from credential store+ | 3. task -> system_message (directive),+ | context-or-task -> prompt| 4. Builds child_context with all nodes/edges| 5. Spawns asyncio.create_task()| 6. Returns immediately: {status: "delegated", task_id: "..."}@@ -43,17 +45,21 @@ _execute_delegated_agent() (handlers/tools.py)Parent continues reasoning Child runs independently(non-blocking) |v- handle_ai_agent() or handle_chat_agent()+ plugin_cls().execute(child_node_id, child_params, child_ctx)+ | (BaseNode.execute -> the agent plugin's+ | @Operation method in nodes/agent/<plugin>/)|- _collect_agent_connections(child_node_id)+ collect_agent_connections(child_node_id)+ | (services/plugin/edge_walker.py)| Filters edges by child's ID- | Finds child's own memory/skills/tools+ | Finds child's own memory/skills/tools/task|- execute_agent() or execute_chat_agent()+ AIService.execute_agent() / execute_chat_agent()| Uses child's provider/model/system_message| Uses child's memory (isolated session)| Uses child's skills + tools- | Prompt = delegated task from parent+ | system_message = delegated task,+ | prompt = delegated context|Result broadcast via WebSocket(NOT returned to parent LLM)@@ -63,7 +69,7 @@ Parent continues reasoning Child runs independently### Step 1: Parent Builds Tool from Child Agent Node-**File:** `server/services/ai.py`, `_build_tool_from_node()` (line 2041)+**File:** `server/services/ai.py`, `_build_tool_from_node()`When the parent agent starts executing and finds a child agent connected to its `input-tools` handle, `_build_tool_from_node()` is called during tool setup:@@ -76,28 +82,21 @@ for tool_info in tool_data:For agent node types, the method:-1. Looks up the tool name from `DEFAULT_TOOL_NAMES` (line 2054):- ```python- 'aiAgent': 'delegate_to_ai_agent',- 'chatAgent': 'delegate_to_chat_agent',- 'android_agent': 'delegate_to_android_agent',- 'coding_agent': 'delegate_to_coding_agent',- # ... all 12 agent types have entries- ```+1. Resolves the tool name via `_resolve_default_tool_name_description(node_type)` — a closure inside `_build_tool_from_node()` that reads the plugin class's `tool_name` ClassVar (each agent plugin declares `tool_name = "delegate_to_<x>"`). There is no central `DEFAULT_TOOL_NAMES` dict; the name comes from the plugin class via `services.node_registry.get_node_class`. The delegation branch is gated by `_get_tool_schema()`'s `_AGENT_DELEGATION_TYPES` tuple — the **15-agent delegatable set** (`aiAgent`, `chatAgent`, `android_agent`, `coding_agent`, `web_agent`, `task_agent`, `social_agent`, `travel_agent`, `tool_agent`, `productivity_agent`, `payments_agent`, `consumer_agent`, `autonomous_agent`, `orchestrator_agent`, `ai_employee`) plus the two bypass-loop agents `rlm_agent` and `claude_code_agent`. (`AI_AGENT_TYPES` in `server/constants.py` mirrors this same set; `codex_agent` is not currently in it.)-2. Creates a `DelegateToAgentSchema` Pydantic model (line 2622) with two fields:+2. Creates a `DelegateToAgentSchema` Pydantic model in `_get_tool_schema()` with two fields (`task` -> the child's mission directive / system message, `context` -> the child's input data / prompt):```pythonclass DelegateToAgentSchema(BaseModel):task: str = Field(- description=f"The task/instruction to delegate to '{agent_label}'."+ description=f"The mission directive for '{agent_label}'. ...")context: Optional[str] = Field(default=None,- description="Additional context or data the child agent needs"+ description="Input data or specific details the agent needs to work with, ...")```-3. Returns a config dict (line 2202):+3. Returns a config dict:```pythonconfig = {'node_type': node_type, # e.g., 'aiAgent'@@ -114,7 +113,7 @@ The LLM sees a tool like `delegate_to_ai_agent(task="...", context="...")`. It d### Step 2: Tool Executor Callback Injects Services-**File:** `server/services/ai.py`, `tool_executor` (line 1406) / `chat_tool_executor` (line 1835)+**File:** `server/services/ai.py`, `tool_executor` / `chat_tool_executor` closuresBoth `execute_agent()` and `execute_chat_agent()` define nearly identical tool executor callbacks as closures. When the parent's LLM decides to call the delegation tool, the callback fires.@@ -128,15 +127,18 @@ The callback captures from outer scope:| `workflow_id` | Function parameter | Scopes broadcasts to workflow || `context` | Function parameter | Contains `nodes` and `edges` arrays |-Before calling `execute_tool()`, the callback injects 5 fields into the config (line 1434-1443):+Before calling `execute_tool()`, the callback injects the service + graph fields into the config:```python-config['workflow_id'] = workflow_id-config['ai_service'] = self # AIService instance (shared)-config['database'] = self.database # Database instance (shared)+config["workflow_id"] = workflow_id+config["ai_service"] = self # AIService instance (shared)+config["database"] = self.database # Database instance (shared)+config["parent_node_id"] = node_id # parent's node id (duplicate tracking)if context:- config['nodes'] = context.get('nodes', []) # ALL workflow nodes- config['edges'] = context.get('edges', []) # ALL workflow edges+ config["nodes"] = context.get("nodes", []) # ALL workflow nodes+ config["edges"] = context.get("edges", []) # ALL workflow edges+ config["workspace_dir"] = context.get("workspace_dir", "")+ config["execution_id"] = context.get("execution_id") # stable per-run id```These injected fields are what enable the child agent to discover its own connections and execute independently. Without them, the child would have no access to the workflow graph or services.@@ -145,84 +147,91 @@ These injected fields are what enable the child agent to discover its own connec### Step 3: Dispatch to Delegation Handler-**File:** `server/services/handlers/tools.py`, `execute_tool()` (line 27)+**File:** `server/services/handlers/tools.py`, `execute_tool()`-The dispatcher checks `node_type` and routes all 12 agent types to `_execute_delegated_agent()` (line 120):+The dispatcher checks `node_type` against the `AI_AGENT_TYPES` frozenset (`server/constants.py`) and routes every agent type to `_execute_delegated_agent()`:```python-if node_type in ('aiAgent', 'chatAgent', 'android_agent', 'coding_agent',- 'web_agent', 'task_agent', 'social_agent', 'travel_agent',- 'tool_agent', 'productivity_agent', 'payments_agent',- 'consumer_agent'):+from constants import AI_AGENT_TYPES++if node_type == "_builtin_check_delegated_tasks":+ return await _execute_check_delegated_tasks(tool_args, config)++if node_type in AI_AGENT_TYPES:return await _execute_delegated_agent(tool_args, config)```+`AI_AGENT_TYPES` is the single source of truth: the 15 standard delegatable agents plus `rlm_agent` and `claude_code_agent`.+---### Step 4: Delegated Agent Execution (Fire-and-Forget)-**File:** `server/services/handlers/tools.py`, `_execute_delegated_agent()` (line 1126)+**File:** `server/services/handlers/tools.py`, `_execute_delegated_agent()`This is where the actual parameter assembly happens.-**4a. Extract injected services from config** (line 1149-1153):+**4a. Extract injected services from config:**```python-ai_service = config.get('ai_service')-database = config.get('database')-nodes = config.get('nodes', [])-edges = config.get('edges', [])-workflow_id = config.get('workflow_id')+ai_service = config.get("ai_service")+database = config.get("database")+nodes = config.get("nodes", [])+edges = config.get("edges", [])+workflow_id = config.get("workflow_id")```-**4b. Fetch child's OWN parameters from database** (line 1165):+A `(parent_node_id, node_id, task_hash)` duplicate guard (`_active_delegations`) short-circuits with `status: "ALREADY_DELEGATED"` if the LLM calls the same delegation twice. A per-child counter (`_active_delegated_nodes`) keeps the child's canvas glow alive past the parent run's end (the background task outlives the `delegate_to_<x>` tool return).++**4b. Fetch child's OWN parameters from database:**```pythonchild_params = await database.get_node_parameters(node_id) or {}```The child's stored parameters (provider, model, system message, etc.) come from the database -- whatever the user configured in the child node's parameter panel. They are NOT inherited from the parent.-**4c. Inject API key if missing** (line 1169-1175):+**4c. Inject API key if missing:**```python-if not child_params.get('api_key') and not child_params.get('apiKey'):+if not child_params.get("api_key"):provider = detect_ai_provider(node_type, child_params)key = await ai_service.auth.get_api_key(provider, "default")if key:- child_params['api_key'] = key+ child_params["api_key"] = key```The child gets its API key from the credential store based on its own provider setting. If the child is configured for Anthropic and the parent uses OpenAI, the child gets the Anthropic key.-**4d. Inject default model if not set** (line 1178-1183):+**4d. Inject default model if not set:**```python-if not child_params.get('model'):+if not child_params.get("model"):provider = detect_ai_provider(node_type, child_params)models = await ai_service.auth.get_stored_models(provider, "default")if models:- child_params['model'] = models[0]+ child_params["model"] = models[0]```-**4e. OVERWRITE prompt with delegated task** (line 1186-1189):+**4e. Map the delegated task/context onto the child's params:**```python-full_prompt = task_description-if task_context:- full_prompt = f"{task_description}\n\nContext:\n{task_context}"-child_params['prompt'] = full_prompt+# task -> mission directive (system message), context -> input data (prompt)+child_params["system_message"] = task_description+child_params.pop("systemMessage", None) # drop pre-migration camelCase mirror+child_params["prompt"] = task_context if task_context else task_description```-This is the **only parameter the parent passes to the child** -- the task instruction. Everything else comes from the child's own stored configuration.+These two strings are the **only inputs the parent passes to the child**. `task` becomes the child's `system_message` (its mission directive); `context` becomes its `prompt` (its input data), falling back to the task text when no context is given. Everything else comes from the child's own stored configuration.-**4f. Build child execution context** (line 1192-1198):+**4f. Build child execution context:**```pythonchild_context = {- 'nodes': nodes, # ALL workflow nodes (not just child's)- 'edges': edges, # ALL workflow edges (not just child's)- 'workflow_id': workflow_id, # Parent's workflow ID (for status scoping)- 'outputs': {}, # Empty -- child starts fresh- 'parent_task_id': task_id # Link back to parent's delegation task+ "nodes": nodes, # ALL workflow nodes (not just child's)+ "edges": edges, # ALL workflow edges (not just child's)+ "workflow_id": workflow_id, # Parent's workflow ID (for status scoping)+ "outputs": {}, # Empty -- child starts fresh+ "parent_task_id": task_id, # Link back to parent's delegation task+ "execution_id": config.get("execution_id"), # shared per-run id (session-keyed tools)}```-**4g. Spawn as background task and return immediately** (line 1269-1281):+**4g. Spawn as background task and return immediately:**```pythontask = asyncio.create_task(run_child_agent())_delegated_tasks[task_id] = task@@ -233,28 +242,38 @@ return {"task_id": task_id,"agent_node_id": node_id,"agent_name": agent_label,- "message": f"Task delegated to '{agent_label}'. Agent is now working independently..."+ "message": f"Task delegated to '{agent_label}'. Agent is now working independently...",}```-Inside `run_child_agent()` (line 1208-1267), the child handler is selected based on node_type:-- `aiAgent` -> `handle_ai_agent()` (agent loop with tools)-- All others -> `handle_chat_agent()` (direct LLM invoke)+Inside `run_child_agent()`, the child executes through its own plugin class (Wave 11.E.3 — the legacy `handle_ai_agent` / `handle_chat_agent` imports are gone). Every agent type owns an `@Operation` method that wraps `prepare_agent_call` + `AIService` dispatch, so delegation just goes through `BaseNode.execute`:++```python+from services.node_registry import get_node_class+from services.plugin import NodeContext++plugin_cls = get_node_class(node_type)+instance = plugin_cls()+child_ctx = NodeContext.from_legacy(+ node_id=node_id, node_type=node_type, context=child_context,+)+result = await instance.execute(node_id, child_params, child_ctx)+```---### Step 5: Child Handler Collects Its Own Connections-**File:** `server/services/handlers/ai.py`, `handle_ai_agent()` (line 225) / `handle_chat_agent()` (line 280)+**File:** `server/nodes/agent/_inline.py`, `prepare_agent_call()` (called by every agent plugin's `@Operation`)-The spawned child calls `_collect_agent_connections()` with **its own node_id**:+The spawned child's `execute()` runs `prepare_agent_call()`, which calls `collect_agent_connections()` (from `server/services/plugin/edge_walker.py`) with **its own node_id**. The function returns a **5-tuple** — `(memory_data, skill_data, tool_data, input_data, task_data)`:```python-memory_data, skill_data, tool_data, input_data = await _collect_agent_connections(+memory_data, skill_data, tool_data, input_data, task_data = await collect_agent_connections(node_id, # CHILD's node IDchild_context, # Contains ALL nodes/edgesdatabase,- log_prefix="[AI Agent]"+ log_prefix="[AI Agent]",)```@@ -262,27 +281,28 @@ memory_data, skill_data, tool_data, input_data = await _collect_agent_connection### Step 6: Connection Filtering by Node ID-**File:** `server/services/handlers/ai.py`, `_collect_agent_connections()` (line 14)+**File:** `server/services/plugin/edge_walker.py`, `collect_agent_connections()`-This function receives ALL nodes/edges but filters by the child's `node_id` (line 65-66):+This function receives ALL nodes/edges but filters by the child's `node_id`:```pythonfor edge in edges:- if edge.get('target') != node_id: # Only edges pointing TO this child+ if edge.get("target") != node_id: # Only edges pointing TO this childcontinue- target_handle = edge.get('targetHandle')- source_node_id = edge.get('source')+ target_handle = edge.get("targetHandle")+ source_node_id = edge.get("source")```For each matching edge, it checks the target handle:| Handle | Action ||--------|--------|-| `input-memory` | Loads markdown memory content from child's connected `simpleMemory` node (line 77-93) |-| `input-skill` | Loads skill instructions; expands masterSkill into individual enabled skills (line 97-153) |-| `input-tools` | Discovers tool nodes; for androidTool, finds connected Android services (line 156-205) |-| `input-main` / `input-chat` | Reads upstream node output for auto-prompt fallback (line 209-213) |+| `input-memory` | Loads markdown memory content from child's connected `simpleMemory` node |+| `input-skill` | Loads skill instructions; expands masterSkill into individual enabled skills |+| `input-tools` | Discovers tool nodes; for androidTool, finds connected Android services |+| `input-main` / `input-chat` | Reads upstream node output for auto-prompt fallback |+| `input-task` | Collects `taskTrigger` output for the conversational task-report pattern |The child only gets connections that are physically wired to it in the workflow graph. If the child has no memory node connected, `memory_data` is `None`. If the child has its own tools, it gets those tools -- and can delegate further.@@ -496,9 +516,11 @@ All broadcasts include `workflow_id` for UI scoping. The frontend uses these to| File | Role ||------|------|-| `server/services/ai.py` | `_build_tool_from_node()`, `_get_tool_schema()`, `DelegateToAgentSchema`, `tool_executor` callback, `DEFAULT_TOOL_NAMES/DESCRIPTIONS` |-| `server/services/handlers/ai.py` | `_collect_agent_connections()`, `handle_ai_agent()`, `handle_chat_agent()` |-| `server/services/handlers/tools.py` | `execute_tool()` dispatcher, `_execute_delegated_agent()`, `_delegated_tasks` tracking |+| `server/services/ai.py` | `_build_tool_from_node()`, `_get_tool_schema()` (incl. `_AGENT_DELEGATION_TYPES` + `DelegateToAgentSchema`), `_resolve_default_tool_name_description()`, `tool_executor` / `chat_tool_executor` closures, `execute_agent()` / `execute_chat_agent()` |+| `server/services/plugin/edge_walker.py` | `collect_agent_connections()` (5-tuple: memory / skill / tool / input / task) |+| `server/nodes/agent/_inline.py` | `prepare_agent_call()` — the 3-step pre-dispatch flow (connection collection + param prep) every agent plugin's `@Operation` runs |+| `server/nodes/agent/<plugin>/__init__.py` | Per-agent plugin classes (`ai_agent`, `chat_agent`, the specialized agents) with their delegation `tool_name` ClassVar + `@Operation` execute method |+| `server/services/handlers/tools.py` | `execute_tool()` dispatcher, `_execute_delegated_agent()`, `_execute_check_delegated_tasks()`, `_delegated_tasks` / `_delegation_results` / `_active_delegations` tracking || `server/constants.py` | `AI_AGENT_TYPES` frozenset, `detect_ai_provider()` || `client/src/components/AIAgentNode.tsx` | Frontend rendering of execution phase animations |---@@ -511,7 +533,7 @@ All broadcasts include `workflow_id` for UI scoping. The frontend uses these to3. **Memory is fully isolated.** Parent and child have separate simpleMemory nodes with separate session IDs and separate vector stores.-4. **The child sees the full workflow graph** via `nodes`/`edges`, but `_collect_agent_connections()` filters by the child's own `node_id`, so it only gets resources physically connected to it.+4. **The child sees the full workflow graph** via `nodes`/`edges`, but `collect_agent_connections()` filters by the child's own `node_id`, so it only gets resources physically connected to it.5. **Services are shared, not duplicated.** `ai_service` and `database` are passed by reference. They share the same credential store and DB connection.@@ -536,13 +558,15 @@ tools.py: _execute_delegated_agent.run_child_agent()↓broadcaster.send_custom_event('task_completed', event_data)↓-event_waiter.dispatch_async('task_completed', event_data)+event_waiter dispatch ('task_completed', event_data)↓Matching taskTrigger nodes resolve their Futures↓Downstream workflow nodes execute with delegation result```+> Note (Wave 12/13): `taskTrigger` is one of the canary-registered trigger types. Under the canary path the producer's CloudEvents envelope routes via `dispatch.emit`; the legacy `event_waiter.register` / `send_custom_event` path is retained for the in-process fallback. See [event_framework.md](./event_framework.md) and the trigger-architecture notes in CLAUDE.md.+### Event Data SchemaThe `task_completed` event carries:@@ -608,7 +632,7 @@ All approaches can coexist. The LLM-driven approach is better for dynamic reason## Agent Task Input Handle (input-task)-All AI agents (aiAgent, chatAgent, and 10 specialized agents) have an `input-task` handle on the left side that can receive task completion events from `taskTrigger` nodes. This enables a conversational pattern where the parent agent reports delegated task results to the user.+All AI agents (`aiAgent`, `chatAgent`, and the specialized agents — live count via `glob server/nodes/agent/*/__init__.py`) have an `input-task` handle on the left side that can receive task completion events from `taskTrigger` nodes. This enables a conversational pattern where the parent agent reports delegated task results to the user.### Architecture@@ -624,9 +648,9 @@ Agent generates conversational response about completed task### How It Works-1. **Backend Detection**: `_collect_agent_connections()` in `handlers/ai.py` detects nodes connected to `input-task` handle-2. **Task Data Collection**: Collects output from connected `taskTrigger` node-3. **Context Injection**: `_format_task_context()` formats task data as prompt context+1. **Backend Detection**: `collect_agent_connections()` in `server/services/plugin/edge_walker.py` detects nodes connected to the `input-task` handle and returns the collected `task_data` (5th element of the tuple)+2. **Task Data Collection**: Collects output from the connected `taskTrigger` node+3. **Context Injection**: `prepare_agent_call` (`server/nodes/agent/_inline.py`) calls `format_task_context()` (from `edge_walker.py`) to render `task_data` as prompt context4. **Prompt Prepending**: Task context is prepended to the agent's prompt before execution### Task Context Format@@ -657,8 +681,9 @@ Please report this error to the user and suggest next steps if appropriate.| File | Changes ||------|---------|-| `client/src/components/AIAgentNode.tsx` | `input-task` handle in leftHandles for all 12 agent configs |-| `server/services/handlers/ai.py` | `task_data` collection, `_format_task_context()`, prompt injection |+| `client/src/components/AIAgentNode.tsx` | Spec-driven; renders the `input-task` handle from the backend NodeSpec (no per-agent config map) |+| `server/services/plugin/edge_walker.py` | `task_data` collection via `collect_agent_connections()`, `format_task_context()` |+| `server/nodes/agent/_inline.py` | `prepare_agent_call()` injects the formatted task context into the prompt |### Usage Example@@ -676,30 +701,12 @@ Please report this error to the user and suggest next steps if appropriate.Here's what was found: ..."```-### Shared Input/Output Constants+### Shared Input/Output Handles (backend SSOT)-Specialized agents use shared constants for consistency:+Post-Wave-11 the frontend `specializedAgentNodes.ts` definition file no longer exists — handle topology is declared on the backend plugin classes and the frontend renders it from each NodeSpec via `useNodeSpec(type)`. Every specialized agent subclasses `SpecializedAgentBase` (`server/nodes/agent/_specialized.py`), which provides the standard handle set through `std_agent_handles()` (declared in `server/nodes/agent/_handles.py`):-```typescript-// In specializedAgentNodes.ts-export const AI_AGENT_INPUTS = [- { name: 'main', displayName: 'Input', ... },- { name: 'skill', displayName: 'Skill', ... },- { name: 'memory', displayName: 'Memory', ... },- { name: 'tools', displayName: 'Tool', ... },- { name: 'task', displayName: 'Task', ... }, // Task completion events-];+- **Left**: `input-main` (Input), `input-memory` (Memory), `input-task` (Task)+- **Bottom**: `input-skill` (Skill), `input-tools` (Tool)+- **Top**: `output-top` (Output)-export const AI_AGENT_OUTPUTS = [- { name: 'main', displayName: 'Output', ... },-];-```--All 10 specialized agents reference these constants:-```typescript-android_agent: {- inputs: AI_AGENT_INPUTS,- outputs: AI_AGENT_OUTPUTS,- properties: AI_AGENT_PROPERTIES-}-```+Team-lead agents (`orchestrator_agent`, `ai_employee`) add an extra `input-teammates` handle for delegation. `AIAgentNode.tsx` is type-agnostic — it reads `handles` / `icon` / `color` / `displayName` / `uiHints` straight from the spec, with no `AGENT_CONFIGS` map.diff --git a/docs-internal/agent_teams.md b/docs-internal/agent_teams.mdindex 60d1302e..07409bae 100644--- a/docs-internal/agent_teams.md+++ b/docs-internal/agent_teams.md@@ -15,7 +15,7 @@ Two node types support the Agent Teams pattern:| `orchestrator_agent` | Orchestrator Agent | Coordinates multiple agents for complex workflows || `ai_employee` | AI Employee | Team lead for intelligent task delegation |-Both are defined in `TEAM_LEAD_TYPES` in `server/services/handlers/ai.py`.+Both are defined in `TEAM_LEAD_TYPES` in [`server/nodes/agent/_inline.py`](../server/nodes/agent/_inline.py).## Architecture@@ -36,7 +36,7 @@ Both are defined in `TEAM_LEAD_TYPES` in `server/services/handlers/ai.py`.1. **User sends prompt** to AI Employee (e.g., "Write a Python script and search the web for documentation")-2. **Team lead collects teammates** via `input-teammates` handle using `_collect_teammate_connections()`+2. **Team lead collects teammates** via the `input-teammates` handle using `collect_teammate_connections()` ([`server/services/plugin/edge_walker.py`](../server/services/plugin/edge_walker.py)), invoked from `prepare_agent_call` in [`_inline.py`](../server/nodes/agent/_inline.py)3. **Teammates become delegation tools** - each connected agent becomes a `delegate_to_*` tool:- `coding_agent` → `delegate_to_coding_agent` tool@@ -65,77 +65,93 @@ Team lead agents have an additional handle compared to standard specialized agen## Backend Implementation-### Handler Logic (`server/services/handlers/ai.py`)+Wave 11 deleted `server/services/handlers/ai.py` (and `handle_chat_agent`). Agent execution now flows through each agent plugin's `execute_op` under `server/nodes/agent/<plugin>/__init__.py` (`ai_agent`, `chat_agent`) or [`server/nodes/agent/_specialized.py`](../server/nodes/agent/_specialized.py) (the 13 specialized agents share one execution body). All of them call the shared pre-dispatch helper `prepare_agent_call` in [`server/nodes/agent/_inline.py`](../server/nodes/agent/_inline.py), which (1) collects standard connections via `collect_agent_connections`, (2) injects task context, (3) auto-prompts from upstream input, and (4) for team-lead types, appends teammates as delegation tools. The prepared kwargs are then splatted into `AIService.execute_chat_agent(...)`.++### Pre-dispatch Logic ([`server/nodes/agent/_inline.py`](../server/nodes/agent/_inline.py))```python-TEAM_LEAD_TYPES = {'orchestrator_agent', 'ai_employee'}+# Team-lead agent types where teammates become delegation tools.+TEAM_LEAD_TYPES = frozenset({"orchestrator_agent", "ai_employee"})-async def handle_chat_agent(...):- # Collect standard connections- memory_data, skill_data, tool_data, input_data, task_data = await _collect_agent_connections(...)+async def prepare_agent_call(*, node_id, node_type, parameters, context, database, ...):+ # 5-tuple standard connection collection+ memory_data, skill_data, tool_data, input_data, task_data = (+ await collect_agent_connections(node_id, context, database, log_prefix=log_prefix)+ )- # Team lead detection - add teammates as delegation tools- if node_type in TEAM_LEAD_TYPES:- teammates = await _collect_teammate_connections(node_id, context, database)+ # ... task-context injection + auto-prompt fallback ...+ # Team-lead detection - add teammates as delegation tools+ if node_type in TEAM_LEAD_TYPES:+ teammates = await collect_teammate_connections(node_id, context, database)if teammates:tool_data = tool_data or []for tm in teammates:+ # Each teammate's own input-tools edges are walked into+ # `child_tools` so the delegation tool's description lists+ # what that teammate can actually do.tool_data.append({- 'node_id': tm['node_id'],- 'node_type': tm['node_type'],- 'label': tm['label'],- 'parameters': tm.get('parameters', {}),+ "node_id": tm["node_id"],+ "node_type": tm["node_type"],+ "label": tm["label"],+ "parameters": tm.get("parameters", {}),+ "child_tools": child_tools,})logger.info(f"[Teams] Added {len(teammates)} teammates as delegation tools")- # Standard execution - AI has delegate_to_* tools available- return await ai_service.execute_chat_agent(...)+ return {"parameters": parameters, "tool_data": tool_data or None, ...}```-### Teammate Collection (`_collect_teammate_connections`)+Note: `collect_agent_connections` ([`server/services/plugin/edge_walker.py`](../server/services/plugin/edge_walker.py)) returns a **5-tuple** — `(memory_data, skill_data, tool_data, input_data, task_data)`.++### Teammate Collection (`collect_teammate_connections`)++Lives in [`server/services/plugin/edge_walker.py`](../server/services/plugin/edge_walker.py):```python-async def _collect_teammate_connections(node_id, context, database):- """Collect agents connected via input-teammates handle."""+async def collect_teammate_connections(node_id, context, database):+ """Walk input-teammates edges and return connected agents.++ Used by orchestrator_agent / ai_employee.+ """+ nodes = context.get("nodes", [])+ edges = context.get("edges", [])teammates = []for edge in edges:- if edge.get('target') != node_id:+ if edge.get("target") != node_id or edge.get("targetHandle") != "input-teammates":continue- if edge.get('targetHandle') != 'input-teammates':+ source_id = edge.get("source")+ source_node = next((n for n in nodes if n.get("id") == source_id), None)+ if not source_node:continue-- source_node = next((n for n in nodes if n.get('id') == edge.get('source')), None)- if source_node and source_node.get('type') in AI_AGENT_TYPES:- params = await database.get_node_parameters(source_node['id'])- teammates.append({- 'node_id': source_node['id'],- 'node_type': source_node['type'],- 'label': source_node.get('data', {}).get('label'),- 'parameters': params- })+ node_type = source_node.get("type", "")+ if node_type not in AI_AGENT_TYPES:+ continue+ params = await database.get_node_parameters(source_id) or {}+ teammates.append({+ "node_id": source_id,+ "node_type": node_type,+ "label": source_node.get("data", {}).get("label", node_type),+ "parameters": params,+ })return teammates```## Delegation Tools-When teammates are connected, the AI service builds delegation tools via `_build_tool_from_node()`:+When teammates are connected, the AI service builds delegation tools via `_build_tool_from_node()` in [`server/services/ai.py`](../server/services/ai.py).++The `delegate_to_<type>` tool name is **auto-derived**, not maintained as a static map. `BaseNode.__init_subclass__` ([`server/services/plugin/base.py`](../server/services/plugin/base.py)) sets `cls.tool_name = f"delegate_to_{cls.type}"` for every plugin whose `component_kind == "agent"` that doesn't declare its own (e.g. `coding_agent` -> `delegate_to_coding_agent`). Subclasses with a distinct delegation contract (`autonomous_agent`, `orchestrator_agent`, `ai_employee`, `rlm_agent`, `claude_code_agent`) override `tool_description` on the class.++When `_build_tool_from_node` sees an agent node type listed in `_AGENT_DELEGATION_TYPES`, it exposes the `(task, context)` delegation schema instead of the agent's own Params (which would leak provider/model/prompt into the parent LLM):```python-# Tool name mapping in ai.py-DEFAULT_TOOL_NAMES = {- 'coding_agent': 'delegate_to_coding_agent',- 'web_agent': 'delegate_to_web_agent',- 'task_agent': 'delegate_to_task_agent',- # ... etc-}--# Schema for delegation+# server/services/ai.pyclass DelegateToAgentSchema(BaseModel):- task: str = Field(description="The task to delegate")- context: Optional[str] = Field(description="Additional context")+ task: str = Field(description="The mission directive for the agent (becomes its system message)")+ context: Optional[str] = Field(default=None, description="Input data / details the agent needs (becomes its user prompt)")```The AI receives tools like:@@ -206,12 +222,14 @@ Connect Team Monitor to a team lead's output to visualize team activity.| File | Description ||------|-------------|-| `server/services/handlers/ai.py` | `TEAM_LEAD_TYPES`, `_collect_teammate_connections()`, team mode detection |-| `server/services/agent_team.py` | `AgentTeamService` for team tracking |-| `server/services/handlers/tools.py` | `_execute_delegated_agent()` for actual delegation |-| `server/services/ai.py` | `_build_tool_from_node()` builds delegate_to_* tools |-| `client/src/components/AIAgentNode.tsx` | Agent node rendering with `input-teammates` handle |-| `client/src/components/TeamMonitorNode.tsx` | Team monitoring UI |+| [`server/nodes/agent/_inline.py`](../server/nodes/agent/_inline.py) | `TEAM_LEAD_TYPES`, `prepare_agent_call()` pre-dispatch (teammate -> delegation-tool injection) |+| [`server/services/plugin/edge_walker.py`](../server/services/plugin/edge_walker.py) | `collect_teammate_connections()` (input-teammates walk) + `collect_agent_connections()` (5-tuple) |+| [`server/services/plugin/base.py`](../server/services/plugin/base.py) | `BaseNode.__init_subclass__` auto-derives `tool_name = f"delegate_to_{type}"` for `component_kind=="agent"` |+| [`server/services/agent_team.py`](../server/services/agent_team.py) | `AgentTeamService` for team tracking |+| [`server/services/handlers/tools.py`](../server/services/handlers/tools.py) | `_execute_delegated_agent()` for actual delegation |+| [`server/services/ai.py`](../server/services/ai.py) | `_build_tool_from_node()` builds delegate_to_* tools, `DelegateToAgentSchema` |+| [`client/src/components/AIAgentNode.tsx`](../client/src/components/AIAgentNode.tsx) | Agent node rendering with `input-teammates` handle |+| [`client/src/components/TeamMonitorNode.tsx`](../client/src/components/TeamMonitorNode.tsx) | Team monitoring UI |## Design Decisions@@ -221,6 +239,6 @@ Connect Team Monitor to a team lead's output to visualize team activity.3. **Fire-and-Forget**: Delegated agents run as background tasks; team lead can check status via `check_delegated_tasks`-4. **API Key Inheritance**: Teammates inherit provider/model from team lead if not configured; API keys injected via standard `_inject_api_keys` flow+4. **API Key Inheritance**: Teammates inherit provider/model from team lead if not configured; API keys injected via the standard `_inject_api_keys` flow in [`server/services/node_executor.py`](../server/services/node_executor.py)5. **Standard Execution Path**: Team leads use the same `execute_chat_agent` flow as other agents, just with additional delegation toolsdiff --git a/docs-internal/ci_cd.md b/docs-internal/ci_cd.mdindex d2c62613..f4a803b8 100644--- a/docs-internal/ci_cd.md+++ b/docs-internal/ci_cd.md@@ -1,6 +1,8 @@# CI/CD Pipeline-Internal reference for MachinaOs's GitHub Actions setup. Workflow inventory, security posture, release flow.+Internal reference for MachinaOs's GitHub Actions setup. Workflow inventory, release flow, and the composite setup action.++The repo currently ships **exactly 4 workflows** plus one composite action. A number of hardening / security workflows described in earlier revisions of this doc (CodeQL, zizmor, rollback, reusable PyPI publish, cross-platform test-install) are **not yet implemented** — see [Planned (not yet implemented)](#planned-not-yet-implemented) at the end.---@@ -15,36 +17,18 @@ Internal reference for MachinaOs's GitHub Actions setup. Workflow inventory, sec+--> | predeploy.yml || | (workflow_call) |v*.*.* tag push -------> +------------------+ | | |- manual dispatch -------> | release.yml |---+ | - plan |- | (dry-run by | | - pre-commit |- | default for | | - build-and-lint |- | manual runs) | | - backend-tests |- +------------------+ | (3 shards) |- | | - cli-tests |- +-- predeploy | - test-build-start |- +-- audit | (3 OS) |- +-- build-for-publish| - ci-passed (alls) |- +-- publish-npm +---------------------++ manual dispatch -------> | release.yml |---+ | - build-and-lint |+ | | | - backend-tests |+ | predeploy gate, | | - cli-tests |+ | then publish | | - test-build-start |+ +------------------+ | (3 OS) |+ | +---------------------++ +-- publish-npm+-- publish-github-packages- +-- publish-pypi --> publish-pypi.yml (reusable)- +-- create-github-release- +-- test-install --> test-install.yml-- manual dispatch -------> +------------------+- | rollback.yml | npm deprecate + (optional) revert PR- +------------------+-- weekly + on PR --------> +------------------+- | codeql.yml | Python + JS/TS SAST (security-extended)- +------------------+-- on .github changes ----> +------------------+- | check-zizmor.yml| workflow-security linter (SARIF)- +------------------+- push to docs-MachinaOs/ +------------------+- | docs.yml | Mintlify documentation deploy- +------------------++ push to main under +------------------++ docs-MachinaOs/ ------> | docs.yml | Mintlify documentation deploy+ +------------------+```### Workflow summary@@ -52,33 +36,23 @@ Internal reference for MachinaOs's GitHub Actions setup. Workflow inventory, sec| Workflow | File | Triggers | Purpose ||----------|------|----------|---------|| CI | `.github/workflows/ci.yml` | push/PR → main | Delegates to predeploy.yml |-| Predeploy | `.github/workflows/predeploy.yml` | `workflow_call` | Plan + pre-commit + build/lint + sharded tests + OS-matrix build/start + alls-green |-| Release | `.github/workflows/release.yml` | `v*.*.*` tag, manual (dry-run default) | predeploy → audit → publish (npm + GH Packages + PyPI) → SLSA attest → release → test-install |-| Publish PyPI | `.github/workflows/publish-pypi.yml` | `workflow_call` | OIDC trusted publishing for `machinaos-server` |-| Test Install | `.github/workflows/test-install.yml` | manual, called from release | Cross-platform install smoke (3 OS × 3 paths) |-| Rollback | `.github/workflows/rollback.yml` | manual (`workflow_dispatch`) | npm deprecate + optional revert PR |-| CodeQL | `.github/workflows/codeql.yml` | push/PR + weekly Sun 20:30 UTC | Python + JS/TS SAST |-| Zizmor | `.github/workflows/check-zizmor.yml` | `.github/**` changes + weekly Mon 07:00 UTC | Workflow-security linter |-| Docs | `.github/workflows/docs.yml` | push to `docs-MachinaOs/**` | Mintlify deploy |-| Setup | `.github/actions/setup/action.yml` | (composite) | pnpm 9 + Node 22 + Python (from `.python-version`) + uv |--### Supply chain--- **All action `uses:` references are pinned to commit SHAs.** Enforced by [zizmor](https://github.com/woodruffw/zizmor) on every PR touching `.github/`. Dependabot rewrites pins via the `github-actions` ecosystem group.-- **`.github/dependabot.yml`** — weekly grouped updates for npm + pip (root + server/) + github-actions ecosystems. Each grouped PR is one ecosystem × one dependency-type.-- **`.pre-commit-config.yaml`** — ruff (format + lint), prettier (client/), eslint, actionlint, stdlib hygiene. Mirrored as a CI job in predeploy.yml so missed local installs still gate merges.-- **`.python-version`** — single source of truth (`3.12`). Consumed via `python-version-file:` in `setup-python` everywhere except the `test-install.yml` jobs which run after `actions/checkout` so the file is available.+| Predeploy | `.github/workflows/predeploy.yml` | `workflow_call` | build/lint + backend tests + CLI tests + cross-OS build/start smoke |+| Release | `.github/workflows/release.yml` | `v*.*.*` tag, `workflow_dispatch` | predeploy gate → publish (npm + GitHub Packages) |+| Docs | `.github/workflows/docs.yml` | push to `docs-MachinaOs/**` on main + manual | Mintlify deploy |+| Setup | `.github/actions/setup/action.yml` | (composite) | pnpm 9 + Node 22 + Python 3.12 + uv v8 + editable CLI install |++### Toolchain pin++- **`.python-version`** — single source of truth (`3.12`). The composite setup action currently hard-codes `python-version: '3.12'` in `setup-python` rather than reading the file via `python-version-file:`; keep the two values in sync when bumping.### Required secrets| Secret | Used by | Description ||--------|---------|-------------|-| `NPM_TOKEN` | release.yml, rollback.yml | npm registry publish + deprecate |-| `GITHUB_TOKEN` | release.yml, rollback.yml | GitHub Packages publish + revert-PR creation (auto-provided) |+| `NPM_TOKEN` | release.yml | npm registry publish (`publish-npm`) |+| `GITHUB_TOKEN` | release.yml | GitHub Packages publish (auto-provided) || `MINTLIFY_TOKEN` | docs.yml | Mintlify documentation deployment |-PyPI publishing uses **OIDC trusted publishing** (no PYPI_TOKEN secret needed). One-time setup: configure MachinaOs as a trusted publisher on PyPI with workflow=`publish-pypi.yml`, environment=`pypi`.----## Composite setup action@@ -89,17 +63,19 @@ PyPI publishing uses **OIDC trusted publishing** (no PYPI_TOKEN secret needed).- uses: ./.github/actions/setup- uses: ./.github/actions/setupwith:- node-version: '20' # override Node only; Python tracks .python-version+ node-version: '20' # override Node only; Python is fixed at 3.12```-| Tool | Version source | Action (pinned SHA) |-|------|---------------|---------------------|-| pnpm | hard-coded `9` (default) | `pnpm/action-setup` |-| Node.js | `node-version` input, default `22` | `actions/setup-node` |-| Python | `.python-version` file (`3.12`) | `actions/setup-python` |-| uv | latest in v8 line | `astral-sh/setup-uv` |+| Tool | Version source | Action |+|------|---------------|--------|+| pnpm | `pnpm/action-setup@v4` default | `pnpm/action-setup@v4` |+| Node.js | `node-version` input, default `22` | `actions/setup-node@v4` |+| Python | hard-coded `3.12` (matches `.python-version`) | `actions/setup-python@v5` |+| uv | `astral-sh/[email protected]`, cache disabled | `astral-sh/[email protected]` |-After tool install, the composite installs the supervisor CLI editably (`uv pip install --system -e .`) and runs a verify step that prints each tool's version.+`setup-node`'s `cache:` is intentionally omitted — its post-job cache save fails with a `Path Validation Error` in lanes that didn't run pnpm (CLI / backend test jobs), which would mark the whole job red even when every test passed.++After tool install the composite installs the supervisor CLI editably (`uv pip install --system -e .`).---@@ -107,20 +83,12 @@ After tool install, the composite installs the supervisor CLI editably (`uv pip**File:** [.github/workflows/predeploy.yml](../.github/workflows/predeploy.yml)-Six jobs gated by a `plan` job that emits change-detection booleans (lifted from [astral-sh/uv's `ci.yml`](https://github.com/astral-sh/uv/blob/main/.github/workflows/ci.yml)):--- `plan` — `dorny/paths-filter` decides which downstream jobs need to run. Boolean outputs: `backend_changed`, `frontend_changed`, `cli_changed`, `workflows_changed`, `docs_only`.-- `pre-commit` — runs `pre-commit run --all-files`. Catches lint/format issues for contributors who haven't installed the hook locally. Skipped for doc-only PRs (unless workflows also changed).-- `build-and-lint` — full `pnpm run build` + frontend tests (vitest, 148 cases) + ESLint + tsgo type-check. Skipped for doc-only PRs.-- `backend-tests` — pytest sharded by domain (prefect pattern):- - `nodes` — `tests/nodes/`- - `services` — `tests/services/` + `tests/temporal/` + `tests/llm/` + `tests/credentials/`- - `root` — everything else (cross-cutting invariants)+Reusable `workflow_call` workflow with four independent jobs (no plan/change-detection gate, no aggregator — every job runs on every call):- Gated on `backend_changed || workflows_changed`. Each shard runs the same `uv sync` + `uv run pytest` pattern.-- `cli-tests` — 106 CLI tests via `python -m pytest cli/tests/`. Gated on `cli_changed || workflows_changed`.-- `test-build-start` — cross-OS (`ubuntu-latest`, `macos-latest`, `windows-latest`) build + start smoke. Verifies `client/dist/` + `server/.venv/` materialise and the supervisor brings up the backend reachable at `localhost:3010/health`. Skipped for doc-only PRs.-- `ci-passed` — single aggregator job using [`re-actors/alls-green`](https://github.com/re-actors/alls-green) (hatch pattern). Branch protection targets this one job name; matrix changes don't break protection rules. `allowed-skips` lets the plan-job optimisations skip downstream jobs without failing the aggregator.+- `build-and-lint` — `pnpm install --frozen-lockfile` + `pnpm run build`, then client lint (`pnpm --filter react-flow-client run lint`), TypeScript check (`... run typecheck`), and frontend tests (`... run test`, vitest). Runs on `ubuntu-latest`.+- `backend-tests` — `uv sync` + `uv run pytest tests/ -v` in `server/`. Whole suite, unsharded. Runs on `ubuntu-latest`.+- `cli-tests` — `uv pip install --system pytest pytest-asyncio pyyaml` + `python -m pytest cli/tests/ -v`. Runs on `ubuntu-latest`.+- `test-build-start` — cross-OS matrix (`ubuntu-latest`, `macos-latest`, `windows-latest`, `fail-fast: false`). Runs `pnpm run build`, then a start smoke test. On Unix it backgrounds `pnpm run start`, polls `http://localhost:3010/health` for up to ~30 s, then `pnpm run stop`. On Windows it starts the supervisor as a background job, waits 15 s, and fails if the job already exited.---@@ -129,116 +97,37 @@ Six jobs gated by a `plan` job that emits change-detection booleans (lifted from**File:** [.github/workflows/release.yml](../.github/workflows/release.yml)### Triggers+| Trigger | Behaviour ||---------|-----------|-| Push of `v*.*.*` tag | Full release (npm + GH Packages + PyPI + SLSA attest + GitHub release + test-install) |-| `workflow_dispatch` | Dry-run by default; explicit `dry_run=false` required to publish (gemini-cli pattern) |+| Push of `v*.*.*` tag | predeploy gate, then publish to npm + GitHub Packages |+| `workflow_dispatch` | Same job graph (manual run; there is no dry-run switch) |++`permissions:` — `contents: read`, `id-token: write`, `packages: write`.### Job graph```predeploy (uses predeploy.yml)|- v-audit -- pnpm audit --prod --audit-level moderate (BLOCKING)- |- v-build-for-publish -- once-per-release client build, uploaded as artifact- | | |- v v v-publish-npm publish-github-packages publish-pypi- | | (uses publish-pypi.yml)- +- SLSA +- npm scope rewrite +- uv build --no-sources- +- attest-build-provenance@v2 +- attest-build-provenance@v2- +- conditional on dry_run +- pypa/gh-action-pypi-publish- +- OIDC, no token- | | |- v v v-create-github-release -- generate-notes + verify-tag (skipped on dry-run)- |- v-test-install (uses test-install.yml)+ +-------------------++ v v+publish-npm publish-github-packages+ | |+ +- build +- build+ +- cli version sync +- cli version sync+ +- npm publish +- rewrite name -> @zeenie-ai/machinaos+ --access public +- npm publish (registry: npm.pkg.github.com)+ --provenance NODE_AUTH_TOKEN = GITHUB_TOKEN+ NODE_AUTH_TOKEN =+ NPM_TOKEN```-### Hardening (lifted from gh CLI + vercel + wrangler + uv)--- `NPM_CONFIG_PROVENANCE=true` set at workflow env-level — supersedes per-command `--provenance` flag.-- `pnpm audit --prod --audit-level moderate` is **blocking** (previous `|| true` masked vulnerabilities).-- `workflow_dispatch` defaults `dry_run=true`; explicit override required to publish.-- `cache: ''` on `setup-node` in publish jobs — prevents poisoned-cache supply-chain contamination (wrangler pattern).-- `actions/attest-build-provenance@v2` on every published artifact (npm tarball + Python wheel + sdist). SLSA Level 3. Verifiable via `gh attestation verify <artifact> --repo zeenie-ai/MachinaOS`.-------## Reusable: publish-pypi.yml--**File:** [.github/workflows/publish-pypi.yml](../.github/workflows/publish-pypi.yml)--Called via `workflow_call` from release.yml. Three jobs:--1. `build` — `uv build --no-sources` in `server/`, produces wheel + sdist, uploads artifact.-2. `attest` — `actions/attest-build-provenance@v2` signs each artifact via OIDC.-3. `publish` — `pypa/gh-action-pypi-publish` with `environment: pypi` for the OIDC subject claim. `skip-existing: true` so re-runs after partial failures don't error.--Inputs:-- `package_dir` (default `server`)-- `dry_run` (boolean) — skips the publish job-------## Rollback workflow--**File:** [.github/workflows/rollback.yml](../.github/workflows/rollback.yml)--Manual `workflow_dispatch` with inputs:-- `version` (required, e.g. `v0.0.72`)-- `deprecate_msg` (default: security-conservative template)-- `open_revert_pr` (boolean, default `false`)--Three jobs:-1. `deprecate-npm` — `npm deprecate` on both `machinaos@VERSION` (npm) and `@zeenie-ai/machinaos@VERSION` (GitHub Packages).-2. `yank-pypi-notice` — prints PyPI yank instructions (PyPI yank is UI-only, no API).-3. `open-revert-pr` — opens a `rollback/<version>` branch + revert PR (skipped unless `open_revert_pr=true`).+- Both publish jobs `needs: predeploy`, run on `ubuntu-latest`, and share the same prefix: `actions/checkout` → composite setup → `actions/setup-node@v4` (with the target `registry-url`) → `pnpm install --frozen-lockfile` → `pnpm run build` → `python -m cli version sync`.+- `publish-npm` — `npm publish --access public --provenance` with `NODE_AUTH_TOKEN=secrets.NPM_TOKEN`. The `--provenance` flag emits an npm provenance attestation (backed by the workflow's `id-token: write`).+- `publish-github-packages` — rewrites `package.json` `name` to `@zeenie-ai/machinaos` and sets `publishConfig.registry = https://npm.pkg.github.com`, then `npm publish` with `NODE_AUTH_TOKEN=secrets.GITHUB_TOKEN`.-A `summary` job posts a step-summary table of what ran.-------## Security workflows--### CodeQL--**File:** [.github/workflows/codeql.yml](../.github/workflows/codeql.yml)--Two-language matrix:-- `python` — covers cli/, server/, scripts/.-- `javascript-typescript` — covers client/, server/nodejs/.--Triggers: push to main + PR to main (both with `paths-ignore` excluding docs) + weekly schedule (Sun 20:30 UTC). Query pack: `security-extended` (broader than default, narrower than `security-and-quality` which adds noisy code-quality queries).--### Zizmor--**File:** [.github/workflows/check-zizmor.yml](../.github/workflows/check-zizmor.yml)--Lints workflows for over-granted permissions, unpinned action `uses:`, insecure `${{ ... }}` template expansion, and `pull_request_target` + checkout-of-PR-ref combinations. Triggers on `.github/**` changes + weekly Mon 07:00 UTC (before the Mon 06:00 Dependabot batch).--Outputs SARIF to the Security tab.-------## Test-install workflow--**File:** [.github/workflows/test-install.yml](../.github/workflows/test-install.yml)--Validates the end-user install paths after a release publishes (or on manual dispatch). Three jobs × three OS = nine matrix cells:--| Job | What it does |-|-----|--------------|-| `test-npm-install` | `npm install -g machinaos`, run `machina --help` / `--version`, smoke-start the backend |-| `test-git-clone` | `git clone` the public repo, `pnpm run build`, smoke-start |-| `test-install-script-unix` / `test-install-script-windows` | Curl/iwr the install script from the repo, validate the resulting `machina` is on PATH |--Each job runs an `actions/checkout` first so `.python-version` is available for `setup-python` — independent of the install path being tested.+There is currently no audit gate, no PyPI publish, no SLSA `attest-build-provenance` step, and no `create-github-release` / test-install stage — see [Planned](#planned-not-yet-implemented).---@@ -246,7 +135,7 @@ Each job runs an `actions/checkout` first so `.python-version` is available for**File:** [.github/workflows/docs.yml](../.github/workflows/docs.yml)-Triggers on push to `docs-MachinaOs/**` on main + manual dispatch. Single job: install Mintlify CLI, run `mintlify deploy` with `MINTLIFY_TOKEN`. No predeploy gate — docs are static, independent of the application's build.+Triggers on push to `main` touching `docs-MachinaOs/**` + manual dispatch. Single `deploy` job: `actions/checkout` → `actions/setup-node@v4` (Node 22) → `npm install -g mintlify` → `mintlify deploy` from `docs-MachinaOs/` with `MINTLIFY_TOKEN`. No predeploy gate — docs are static, independent of the application build.---@@ -254,16 +143,26 @@ Triggers on push to `docs-MachinaOs/**` on main + manual dispatch. Single job: i| File | Purpose ||------|---------|-| `.github/workflows/ci.yml` | CI entry point |-| `.github/workflows/predeploy.yml` | Reusable validation (plan + pre-commit + build/lint + tests + OS matrix + alls-green) |-| `.github/workflows/release.yml` | Tag-triggered release pipeline (dry-run default on manual) |-| `.github/workflows/publish-pypi.yml` | Reusable: build wheel + sdist + OIDC trusted publish |-| `.github/workflows/rollback.yml` | Manual deprecate + revert PR |-| `.github/workflows/codeql.yml` | SAST (Python + JS/TS) |-| `.github/workflows/check-zizmor.yml` | Workflow-security linter |-| `.github/workflows/test-install.yml` | Cross-platform user-install smoke |+| `.github/workflows/ci.yml` | CI entry point (delegates to predeploy.yml) |+| `.github/workflows/predeploy.yml` | Reusable validation (build/lint + backend tests + CLI tests + OS matrix build/start) |+| `.github/workflows/release.yml` | Tag / manual release: predeploy gate → publish npm + GitHub Packages || `.github/workflows/docs.yml` | Mintlify documentation deploy |-| `.github/actions/setup/action.yml` | Composite: pnpm + Node + Python + uv |-| `.github/dependabot.yml` | Weekly grouped updates (npm + pip × 2 + actions) |-| `.pre-commit-config.yaml` | ruff + prettier + eslint + actionlint hooks |+| `.github/actions/setup/action.yml` | Composite: pnpm + Node + Python + uv + editable CLI install || `.python-version` | Toolchain pin (`3.12`) — single source of truth |++---++## Planned (not yet implemented)++The following existed in earlier drafts of this doc but are **not present in the current repo**. Listed here so the intent is preserved without misrepresenting the shipped pipeline:++- **`predeploy.yml` change-detection + aggregator** — a `plan` job (`dorny/paths-filter`) gating downstream jobs, a `pre-commit` job, pytest sharding by domain, and a `ci-passed` aggregator (`re-actors/alls-green`) as the single branch-protection target. Today every predeploy job runs unconditionally and there is no aggregator job.+- **`release.yml` hardening** — `workflow_dispatch` dry-run default, a blocking `pnpm audit`, a once-per-release `build-for-publish` artifact, `actions/attest-build-provenance` SLSA attestations, and a `create-github-release` job.+- **`publish-pypi.yml`** — reusable PyPI publish (OIDC trusted publishing, `uv build --no-sources`, `pypa/gh-action-pypi-publish`). No PyPI distribution is published today.+- **`test-install.yml`** — cross-platform end-user install smoke (npm install, git clone, install script) across 3 OS.+- **`rollback.yml`** — manual `npm deprecate` + optional revert PR.+- **`codeql.yml`** — Python + JS/TS SAST (`security-extended`).+- **`check-zizmor.yml`** — workflow-security linter (SARIF to the Security tab).+- **`.github/dependabot.yml`** — weekly grouped npm + pip + github-actions updates.+- **`.pre-commit-config.yaml`** — ruff / prettier / eslint / actionlint hooks (note: the project rule is to verify with pytest + tsgo/eslint, not ruff).+- **Commit-SHA pinning of action `uses:`** — the shipped workflows currently pin to major-version tags (`@v4`, `@v5`, `@v8.1.0`), not commit SHAs.diff --git a/docs-internal/claude_code_cli_reference.md b/docs-internal/claude_code_cli_reference.mdindex 631bcf48..c6ffab82 100644--- a/docs-internal/claude_code_cli_reference.md+++ b/docs-internal/claude_code_cli_reference.md@@ -205,4 +205,4 @@ Quick cross-reference for the MachinaOs `claude_code_agent` integration:| `--disallowedTools <csv>` | optional per-task override | remove tools from context || `--agent <name>` | optional per-task override | use a specific subagent |-See [`services/cli_agent/providers/anthropic_claude.py:headless_argv`](../server/services/cli_agent/providers/anthropic_claude.py) for the argv-construction code.+See [`server/nodes/agent/claude_code_agent/_provider.py`](../server/nodes/agent/claude_code_agent/_provider.py) for the argv-construction code.diff --git a/docs-internal/claude_code_permission_modes_reference.md b/docs-internal/claude_code_permission_modes_reference.mdindex 1abaa381..cd76421e 100644--- a/docs-internal/claude_code_permission_modes_reference.md+++ b/docs-internal/claude_code_permission_modes_reference.md@@ -7,7 +7,7 @@> (`default`, `acceptEdits`, `plan`, `auto`, `dontAsk`,> `bypassPermissions`) on `ClaudeTaskSpec.permission_mode`. See> `services/cli_agent/types.py` and-> `services/cli_agent/providers/anthropic_claude.py:headless_argv`.+> `server/nodes/agent/claude_code_agent/_provider.py`.---diff --git a/docs-internal/cli_agent_canonical_patterns_rfc.md b/docs-internal/cli_agent_canonical_patterns_rfc.mdindex 886233ef..19a1e72f 100644--- a/docs-internal/cli_agent_canonical_patterns_rfc.md+++ b/docs-internal/cli_agent_canonical_patterns_rfc.md@@ -5,7 +5,7 @@| Status | Draft (research, revision 2) || Date | 2026-05-07 || Scope | `services/cli_agent` framework — Claude Code, Codex, Gemini-CLI agents spawned by MachinaOs |-| Companion code | [`server/services/cli_agent/`](../server/services/cli_agent/), [`server/nodes/agent/claude_code_agent.py`](../server/nodes/agent/claude_code_agent.py) |+| Companion code | [`server/services/cli_agent/`](../server/services/cli_agent/), [`server/nodes/agent/claude_code_agent/`](../server/nodes/agent/claude_code_agent/) || Companion docs | [cli_agent_framework.md](./cli_agent_framework.md), [claude_code_agent_architecture.md](./claude_code_agent_architecture.md) |## Abstract@@ -276,20 +276,26 @@ File:line evidence from a focused read of `services/cli_agent/`:### 4.1 Argv flags emitted by the Claude provider-[`providers/anthropic_claude.py:74-220`](../server/services/cli_agent/providers/anthropic_claude.py):+[`claude_code_agent/_provider.py::interactive_argv`](../server/nodes/agent/claude_code_agent/_provider.py)+(the Anthropic/Claude provider moved into the plugin folder; the generic+`services/cli_agent/providers/` now holds only `google_gemini.py` ++`openai_codex.py`). Post-cutover the pool path drives claude in+interactive stream-json mode, not `claude -p` headless — so the argv is+built by `interactive_argv`, not a `-p`-prefixed print-mode command:-**Always-on:** `-p <prompt>`, `--output-format stream-json`, `--verbose`,-`--include-partial-messages`, `--include-hook-events`, `--model`,-`--max-turns`, `--permission-mode`, `--allowedTools <csv>`.+**Always-on:** `--output-format stream-json`, `--input-format+stream-json`, `--verbose`, `--ide`, `--model`, `--permission-mode`,+`--allowedTools <csv>`.**Conditional:** `--mcp-config <json>` + `--strict-mcp-config` (when-`mcp_endpoint_url` and `mcp_bearer_token` set), `--session-id`/`--resume`,-`--max-budget-usd`, `--append-system-prompt`, `--effort`,-`--fallback-model`, `--add-dir`, `--disallowedTools`, `--agent`.+`mcp_endpoint_url` and `mcp_bearer_token` set), `--continue`/`--resume`+(memory-bound runs, mutually exclusive), `--max-budget-usd`,+`--append-system-prompt`, `--effort`, `--fallback-model`, `--add-dir`,+`--disallowedTools`, `--agent`.### 4.2 `--mcp-config` JSON shape-[`providers/anthropic_claude.py:112-123`](../server/services/cli_agent/providers/anthropic_claude.py):+[`claude_code_agent/_provider.py::interactive_argv`](../server/nodes/agent/claude_code_agent/_provider.py):```json{@@ -297,26 +303,28 @@ File:line evidence from a focused read of `services/cli_agent/`:"machinaos": {"type": "http","url": "<mcp_endpoint_url>",- "headers": { "Authorization": "Bearer <mcp_bearer_token>" }+ "headers": { "Authorization": "Bearer <mcp_bearer_token>" },+ "alwaysLoad": true}}}```-**Spec gap.** Per §2.4 (I-4), with `ENABLE_TOOL_SEARCH` unset (the default-for spawned subprocesses), the agent **defers all `mcp__machinaos__*`-tools**. Only `ToolSearch`-driven discovery loads them — and the spawned-agent doesn't always make that call. The fix is one line: add-`"alwaysLoad": true` to the entry. Result: tools enter context at-session start; startup blocks ≤5s waiting for connection.+**Closed (I-4).** Per §2.4, with `ENABLE_TOOL_SEARCH` unset (the default+for spawned subprocesses), the agent would otherwise **defer all+`mcp__machinaos__*` tools** and only load them via `ToolSearch`-driven+discovery — which the spawned agent doesn't always call. The+`"alwaysLoad": true` entry (present in `interactive_argv`'s emitted+`--mcp-config`) forces the tools into context at session start; startup+blocks ≤5s waiting for connection.### 4.3 `--allowedTools` value-[`providers/anthropic_claude.py:176-194`](../server/services/cli_agent/providers/anthropic_claude.py):+[`claude_code_agent/_provider.py::interactive_argv`](../server/nodes/agent/claude_code_agent/_provider.py):```-Read,Edit,Bash,Glob,Grep,Write,mcp__machinaos__<each connected node_type>,+Skill # only when >=1 skill is wiredmcp__machinaos__getWorkspaceFiles,mcp__machinaos__listSkills,mcp__machinaos__getSkill,@@ -324,10 +332,20 @@ mcp__machinaos__getCredential,mcp__machinaos__broadcastLog```-**Spec gap.** `Skill` is missing → claude can't execute auto-discovered-skills (when we materialise them per I-1). `WebSearch` and `WebFetch`-are missing → agent gets denied when wiring is incomplete (the user's-runtime log shows exactly this).+**Post-cutover: strict MCP-only allowlist (gated by `--permission-mode+dontAsk`).** Claude's built-in escape hatches (`Read`, `Edit`, `Bash`,+`Glob`, `Grep`, `Write`, `WebSearch`, `WebFetch`) are intentionally NOT+in the default allowlist — equivalent capability is wired explicitly via+the `fileRead` / `fileModify` / `fsSearch` / `shell` / `browser` /+`perplexitySearch` workflow tools. The built-in `Skill` tool is added+**conditionally**, only when at least one skill is connected through+`input-skill` (paired with SKILL.md materialisation under+`<cwd>/.claude/skills/`). Callers can opt specific built-ins back in+per-task via `ClaudeTaskSpec.allowed_tools`. This reverses the earlier+"WebSearch/WebFetch missing → agent gets denied" gap: the framework now+routes every capability through explicitly wired MCP tools rather than+built-in fallbacks. Locked by+`test_no_claude_builtins_in_default_allowlist`.`ToolSearch` is **not** needed in the allowlist (permission-free per §2.1).@@ -358,8 +376,8 @@ path today.### 4.5 FastMCP server tools-[`mcp_server.py:224-427`](../server/services/cli_agent/mcp_server.py),-[`workflow_tools.py:40-72`](../server/services/cli_agent/workflow_tools.py):+[`mcp_server.py`](../server/services/cli_agent/mcp_server.py),+[`workflow_tools.py`](../server/services/cli_agent/workflow_tools.py):5 built-in: `getWorkspaceFiles`, `listSkills`, `getSkill`,`getCredential`, `broadcastLog`.@@ -374,7 +392,7 @@ the agent never calls them.### 4.6 Spawn env-[`session.py:138-155`](../server/services/cli_agent/session.py):+[`session.py`](../server/services/cli_agent/session.py):`PYTHONUNBUFFERED=1`, `CLAUDE_CONFIG_DIR=<MACHINA_CLAUDE_DIR>` (claude only),`<provider.ide_lock_env_var>=<lockfile_path>`,`MACHINA_PARENT_RUN_ID=<workflow_id>:<node_id>:<token[:8]>`.@@ -386,9 +404,18 @@ Either works.### 4.8 Memory bridge — `simpleMemory` → `claude_code_agent` (DONE in `ecbe69b`)-**Status:** native claude session resume works end-to-end via stable-cwd + UUID5/last_session_id round-trip. Documented in+**Status:** native claude session continuity works end-to-end. The+mechanism was later simplified from the UUID5 / `--session-id`+round-trip described in this section to `--continue` (first cold spawn+for a memory-wired run) + intra-process stream-json multi-turn against a+warm subprocess + `--resume <UUID>` for crash recovery — all on a stable+`cwd=repo_root`. The entry point is+[`claude_code_agent/__init__.py::execute_op`](../server/nodes/agent/claude_code_agent/__init__.py)+(sets `continue_session = bool(memory_data)`) and the warm-subprocess+pool at [`claude_code_agent/_pool.py`](../server/nodes/agent/claude_code_agent/_pool.py).+Current canonical description:[cli_agent_framework.md → Memory bridge](./cli_agent_framework.md#memory-bridge--simplememory--claude_code_agent).+The UUID5/`--session-id` prose below is retained for historical context.**Project-key derivation verified empirically** by listing`<DATA_DIR>/claude/projects/` on disk and reproducing each name from@@ -455,14 +482,14 @@ documented project-instruction surface.| Invariant | Status | Action ||---|---|---|-| **I-1** Skills as files | **DONE (`b40011e`).** [`session.py:_materialise_skills`](../server/services/cli_agent/session.py) writes connected skills to `<cwd>/.claude/skills/<name>/` on `_pre_spawn`. `mcp__machinaos__listSkills` / `getSkill` retained as a transitional fallback. | Drop `getSkill`/`listSkills` MCP tools after one release. |+| **I-1** Skills as files | **DONE.** [`claude_code_agent/_skills.py::materialise_skills`](../server/nodes/agent/claude_code_agent/_skills.py) writes connected skills to `<cwd>/.claude/skills/<name>/`, invoked from both the pool spawn ([`_pool.py`](../server/nodes/agent/claude_code_agent/_pool.py)) and the non-pool `session.py::_pre_spawn` (via the `get_skill_materialiser` registry in [`services/cli_agent/factory.py`](../server/services/cli_agent/factory.py)). `mcp__machinaos__listSkills` / `getSkill` retained as a transitional fallback. | Drop `getSkill`/`listSkills` MCP tools after one release. || **I-2** MCP transport | **Aligned.** Streamable-HTTP via `--mcp-config`. | None. || **I-3** `list_changed` notification | **DONE (`b40011e`).** [`workflow_tools._schedule_list_changed_notify`](../server/services/cli_agent/workflow_tools.py) fires after each `add_tool` / `remove_tool` since FastMCP doesn't emit it automatically. | Optional: unit test asserting `session.send_tool_list_changed` is called. |-| **I-4** Tool-search deferral | **DONE (`b40011e`).** `"alwaysLoad": true` added to the `machinaos` server entry in [`providers/anthropic_claude.py:111-124`](../server/services/cli_agent/providers/anthropic_claude.py). | None. |+| **I-4** Tool-search deferral | **DONE.** `"alwaysLoad": true` set on the `machinaos` server entry in [`claude_code_agent/_provider.py::interactive_argv`](../server/nodes/agent/claude_code_agent/_provider.py). | None. || **I-5** Visible-tool filtering | **Gap.** All 5 built-in MachinaOs MCP tools (including `getCredential`, `broadcastLog`) are visible to the model. | Mark internal-only tools `_meta["anthropic/alwaysLoad"]: false` or filter via FastMCP middleware. **Defer** — not breaking today. |-| **I-6** Native session continuity | **DONE (`ecbe69b`).** [`session.py`](../server/services/cli_agent/session.py) accepts `memory_bound=True` → `cwd()` returns `repo_root`, `_pre_spawn` / `cleanup` skip the worktree. [`claude_code_agent.py`](../server/nodes/agent/claude_code_agent.py) derives UUID5 for first-run `--session-id`, reads `last_session_id` for `--resume`. [`service.py:_persist_memory`](../server/services/cli_agent/service.py) saves `last_session_id`, broadcasts `node_parameters_updated`, auto-clears stale UUIDs. See §4.8. | Drop the markdown `--append-system-prompt` injection block from `claude_code_agent` once SDK migration lands (the markdown surface remains as a UI mirror, not a context channel). Pending. |+| **I-6** Native session continuity | **DONE.** [`session.py`](../server/services/cli_agent/session.py) keeps a stable cwd for memory-bound spawns; the warm-subprocess pool at [`claude_code_agent/_pool.py`](../server/nodes/agent/claude_code_agent/_pool.py) preserves the session across turns. [`claude_code_agent/__init__.py`](../server/nodes/agent/claude_code_agent/__init__.py) sets `continue_session = bool(memory_data)` → argv emits `--continue` (first cold spawn) with `--resume <UUID>` reserved for crash recovery. [`service.py:_persist_memory`](../server/services/cli_agent/service.py) appends turns to `memory_content`, saves `last_session_id` (display-only), broadcasts `node_parameters_updated`, and auto-clears stale UUIDs via `_clear_stale_session_id`. See §4.8. | Markdown `memory_content` remains the UI mirror, not the resume channel. || **System-prompt directive** (Cursor / `CLAUDE.md` pattern) | **DONE (`b40011e`).** Second `--append-system-prompt` listing connected `mcp__machinaos__*` tools. | None. |-| `--allowedTools` includes `Skill,WebSearch,WebFetch` | **DONE (`b40011e`).** [`ai_cli_providers.json`](../server/config/ai_cli_providers.json) + hardcoded fallback both updated. | None. |+| `--allowedTools` strict MCP-only allowlist | **DONE (superseded R3).** Built-in escape hatches (`Read`/`Edit`/`Bash`/`Glob`/`Grep`/`Write`/`WebSearch`/`WebFetch`) are NOT in the default allowlist; `Skill` is added conditionally (only when a skill is wired). `default_allowed_tools: ""` in [`ai_cli_providers.json`](../server/config/ai_cli_providers.json); allowlist assembled in [`_provider.py::interactive_argv`](../server/nodes/agent/claude_code_agent/_provider.py). Gated by `--permission-mode dontAsk`. | None. || Composio-style server-side credentials | **Aligned.** `getCredential` allowlist + `auth_service.get_api_key`. | None. || Hermes-style `provider_data` envelope | **Aligned.** [`protocol.py:SessionResult.provider_data`](../server/services/cli_agent/protocol.py). | None. || Composio-style parent-run-id | **Aligned.** `MACHINA_PARENT_RUN_ID` env var. | None. |@@ -481,12 +508,18 @@ entry in `--mcp-config` (Closes I-4).** Per §2.4: *"…also blocks startupuntil that server connects (5s cap)."* The 5s wait is acceptable — theFastMCP server is already up before spawn.-**R3 (DONE in `b40011e`). `--allowedTools` defaults extended with-`Skill,WebSearch,WebFetch`.** Updated in both-[`ai_cli_providers.json`](../server/config/ai_cli_providers.json) and-the hardcoded fallback in-[`anthropic_claude.py`](../server/services/cli_agent/providers/anthropic_claude.py).-`ToolSearch` intentionally NOT added — it's permission-free.+**R3 (DONE, then superseded by the strict-allowlist cutover).** An+earlier revision extended `--allowedTools` with `Skill,WebSearch,WebFetch`.+The current design instead ships a strict **MCP-only** allowlist:+`default_allowed_tools: ""` in+[`ai_cli_providers.json`](../server/config/ai_cli_providers.json), and+[`_provider.py::interactive_argv`](../server/nodes/agent/claude_code_agent/_provider.py)+assembles `mcp__machinaos__<node_type>` per wired tool + the 5 infra MCP+tools + the built-in `Skill` (conditionally, when a skill is wired),+gated by `--permission-mode dontAsk`. `WebSearch`/`WebFetch`/`Bash`/etc.+are intentionally excluded — equivalent capability is wired explicitly+via workflow tools. `ToolSearch` intentionally NOT added — it's+permission-free.**R4 (DONE in `b40011e`). System-prompt directive when MCP tools arewired (the `CLAUDE.md` / Cursor-rules pattern).** Second@@ -505,13 +538,20 @@ runs (Closes I-6).** Three coupled mechanisms:skips `git worktree add`; `cleanup` skips `git worktree remove`.Confirms claude's `project_key` is constant across runs.-2. **UUID5 first-run / `--resume` thereafter.**- [`claude_code_agent.py`](../server/nodes/agent/claude_code_agent.py)- derives `uuid5(NAMESPACE_OID, f"{memory_node_id}:{simpleMemory.session_id}")`- for the very first run and passes it as `--session-id <UUID5>`.- Subsequent runs read `simpleMemory.last_session_id` and pass- `--resume <UUID>`. Mutually exclusive in argv emission- (`providers/anthropic_claude.py:141-146`).+2. **`--continue` first-run / warm-subprocess multi-turn / `--resume`+ crash recovery.**+ [`claude_code_agent/__init__.py`](../server/nodes/agent/claude_code_agent/__init__.py)+ sets `continue_session = bool(memory_data)`; the argv-builder+ [`_provider.py::interactive_argv`](../server/nodes/agent/claude_code_agent/_provider.py)+ emits `--continue` on the first cold spawn for a memory-wired run+ (claude auto-loads the most recent conversation under the stable+ cwd's `project_key`). Subsequent turns are written as stream-json to+ the same warm subprocess held by+ [`_pool.py`](../server/nodes/agent/claude_code_agent/_pool.py), which+ preserves the session UUID in-process. If the subprocess dies between+ batches, the pool respawns with `--resume <current_session_uuid>`.+ `--continue` and `--resume` are mutually exclusive in argv emission.+ (This supersedes the earlier UUID5 / `--session-id` first-run dance.)3. **Auto-clear stale `last_session_id`.**[`_persist_memory`](../server/services/cli_agent/service.py)@@ -536,8 +576,8 @@ runs (Closes I-6).** Three coupled mechanisms:origin; locked by `tests/test_cloudevents_node_parameters.py`.5. **Parallel-batch guard.** `len(tasks) > 1` with memory wired raises- `NodeUserError` at handler entry — concurrent `--resume <UUID>`- spawns against one JSONL would race.+ `NodeUserError` at handler entry — concurrent `--continue` spawns+ against one project_key would race claude's session resolution.Markdown is the UI mirror. The `simpleMemory.memory_content` fieldkeeps mirroring conversation turns via@@ -551,7 +591,7 @@ claude's on-disk JSONL.## 7. Open questions- **Concurrent batches sharing the FastMCP registry.** Refcount in- [`workflow_tools.py:51-52`](../server/services/cli_agent/workflow_tools.py)+ [`workflow_tools.py`](../server/services/cli_agent/workflow_tools.py)is unlocked. Theoretical race; today's call ordering is synchronous.Add `threading.Lock` for safety.- **Verifying FastMCP emits `notifications/tools/list_changed`.**diff --git a/docs-internal/email_service.md b/docs-internal/email_service.mdindex 94bb3f02..9f16b5d6 100644--- a/docs-internal/email_service.md+++ b/docs-internal/email_service.md@@ -8,9 +8,9 @@ IMAP/SMTP email integration via the [Himalaya CLI](https://github.com/pimalaya/h┌──────────────────────────────────────────────┐│ EmailService (singleton) ││ │- handler ─────────► resolve_credentials(params) -> dict │- (email.py) │ params > preset (email_providers.json) │- │ > stored API keys (AuthService) │+ plugin ──────────► resolve_credentials(params) -> dict │+ execute_op │ params > preset (email_providers.json) │+ (nodes/email/...) │ > stored API keys (AuthService) ││ ││ send(params) -> dict ││ read(params) -> dict (7 operations) │@@ -30,13 +30,18 @@ IMAP/SMTP email integration via the [Himalaya CLI](https://github.com/pimalaya/hcalls IMAP/SMTP backends```+`EmailService` and `HimalayaService` are both singletons living in the+self-contained email plugin folder (`server/nodes/email/_service.py` and+`server/nodes/email/_himalaya.py`). `EmailService` exposes `HimalayaService`+via its `himalaya` property (lazy import).+### Request Flow (emailSend)```emailSend node│▼-handle_email_send(node_id, type, params, ctx)+EmailSendNode.execute_op (server/nodes/email/email_send/__init__.py)│▼EmailService.send(params)@@ -72,28 +77,32 @@ Custom/self-hosted providers rely on stored custom keys because their presets ar## Key Files+Self-contained plugin folder (Wave 11.I) — everything email-specific lives under `server/nodes/email/`.+| File | Description ||------|-------------|-| `server/services/himalaya_service.py` | CLI wrapper. Subprocess-based invocation, temp TOML config generation, JSON output parsing. Singleton via `get_himalaya_service()`. |-| `server/services/email_service.py` | Orchestration layer. Credential resolution, operation dispatch, polling helpers. Singleton via `get_email_service()`. |+| `server/nodes/email/__init__.py` | Self-registration: `register_filter_builder("emailReceive", build_email_filter)` + `register_canary_trigger_type("emailReceive", "com.machinaos.email.message.received")` + re-export of `dispatch_email_received`. |+| `server/nodes/email/_himalaya.py` | `HimalayaService` (`ServiceSingleton`). Subprocess-based invocation, temp TOML config generation, JSON output parsing. Singleton via `get_himalaya_service()`. |+| `server/nodes/email/_service.py` | `EmailService` (`ServiceSingleton`). Credential resolution, operation dispatch, polling helpers. Exposes `HimalayaService` via the `himalaya` property. Singleton via `get_email_service()`. |+| `server/nodes/email/email_send/__init__.py` | `EmailSendNode(ActionNode)` — dual-purpose send plugin (`group = ("email", "tool")`, `usable_as_tool = True`). |+| `server/nodes/email/email_read/__init__.py` | `EmailReadNode(ActionNode)` — dual-purpose read/search/manage plugin (7 operations). |+| `server/nodes/email/email_receive/__init__.py` | `EmailReceiveNode(PollingTriggerNode)` — polling trigger; baseline + diff loop with `poll_ids` / `fetch_detail` hooks. |+| `server/nodes/email/_filters.py` | `build_filter` (registered as `build_email_filter`) — server-side filter closure for the `emailReceive` trigger. |+| `server/nodes/email/_events.py` | `email_message_received` `WorkflowEvent` factory + `dispatch_email_received` (single `dispatch.emit`, CloudEvents type `com.machinaos.email.message.received`). |+| `server/nodes/email/email_{send,read,receive}/icon.svg` + `meta.json` | Per-plugin icon (served at `/api/schemas/nodes/<type>/icon`) + color metadata. || `server/config/email_providers.json` | Provider presets (IMAP/SMTP host/port/encryption per provider) + defaults + polling config. Cached on first load. |-| `server/services/handlers/email.py` | Three thin node handlers: `handle_email_send`, `handle_email_read`, `handle_email_receive`. |-| `server/services/node_executor.py` | Handler registry entries: `emailSend`, `emailRead`, `emailReceive`. |-| `server/services/event_waiter.py` | `TRIGGER_REGISTRY['emailReceive']` + `build_email_filter`. |-| `server/services/deployment/manager.py` | `_create_email_poll_coroutine` factory for deployment-mode polling. |-| `server/services/ai.py` | `EmailSendSchema`, `EmailReadSchema` Pydantic schemas for AI tool calling + `DEFAULT_TOOL_NAMES` / `DEFAULT_TOOL_DESCRIPTIONS` entries. |-| `server/services/handlers/tools.py` | `_execute_email_tool` dispatch for `emailSend` / `emailRead` when called by an AI Agent. || `server/constants.py` | `EMAIL_TYPES`, `EMAIL_TOOL_TYPES`, plus `emailReceive` in `POLLING_TRIGGER_TYPES` and `WORKFLOW_TRIGGER_TYPES`. |-| `client/src/assets/icons/email/` | `send.svg`, `read.svg`, `receive.svg` + `index.ts` registering them as data URIs via `?raw` import. || `client/src/components/CredentialsModal.tsx` | Email credentials panel (provider dropdown, email/password inputs, conditional custom IMAP/SMTP section). |+**AI tool schema** is derived automatically from each plugin's `Params` Pydantic model — there is no hand-written `EmailSendSchema` / `EmailReadSchema`. Dual-purpose dispatch goes through the generic plugin fast-path in `server/services/handlers/tools.py` (`instance.execute_as_tool(...)`), not a per-email `_execute_email_tool` branch.+## EmailService API-**File:** `server/services/email_service.py`+**File:** `server/nodes/email/_service.py`-### `resolve_credentials(params: Dict) -> Dict`+### `async resolve_credentials(params: Dict) -> Dict`-Builds the credentials dict consumed by `HimalayaService`. Required before every operation. Raises `ValueError` if `email_address` or `email_password` is not set (neither in params nor stored).+Async. Builds the credentials dict consumed by `HimalayaService` (reads stored keys via `AuthService.get_api_key`). Required before every operation. Raises `ValueError` if the email address or password is not set (neither in params nor stored).Returned keys:- `email` (str, required) — account address, used as IMAP/SMTP login@@ -135,7 +144,7 @@ Reads polling config from `email_providers.json` (`polling.interval`, `polling.m## HimalayaService API-**File:** `server/services/himalaya_service.py`+**File:** `server/nodes/email/_himalaya.py`### CLI execution model@@ -192,7 +201,7 @@ Send email via SMTP. Group: `['email', 'tool']`. Two outputs (`main`, `tool`).| `bcc` | string | no | || `body_type` | options | no | `text` (default) or `html` |-**AI tool schema:** `EmailSendSchema` (same fields as node params, `body_type` default `"text"`, `cc`/`bcc` optional). When the LLM invokes `email_send`, `_execute_email_tool` routes to `EmailService.send()` with the LLM args merged over the node params.+**AI tool schema:** derived from `EmailSendParams` (same fields as the node, `body_type` default `"text"`, `cc`/`bcc` optional). When the LLM invokes the tool, the generic plugin tool fast-path (`instance.execute_as_tool` in `handlers/tools.py`) runs `EmailSendNode`'s send operation with the LLM args merged over the node params.### emailRead — Dual-purpose (workflow node + AI tool)@@ -211,7 +220,7 @@ Read/search/manage emails via IMAP. Group: `['email', 'tool']`. Two outputs (`ma| `page` | list | default 1 || `page_size` | list | default 20, max 100 |-**AI tool schema:** `EmailReadSchema` exposes every operation and all operation-specific fields as optional. The LLM picks an `operation` and fills the relevant subset.+**AI tool schema:** derived from `EmailReadParams`, exposing every operation and all operation-specific fields. The LLM picks an `operation` and fills the relevant subset.### emailReceive — Polling trigger@@ -228,34 +237,27 @@ Group: `['email', 'trigger']`. No inputs. Single output `main` with the new emai**Baseline detection:** On first execution the handler calls `poll_ids(creds, folder)` to capture the set of currently-existing envelope IDs. The poll loop then diffs against this baseline -- only newly appearing IDs trigger the workflow. This avoids firing on existing historical mail.-**Event dispatch:** When a new message arrives, the handler:+**Event dispatch:** When a new message arrives, `EmailReceiveNode.execute_op`:1. Fetches full detail via `fetch_detail`2. Optionally flags it as read-3. Dispatches `event_waiter.dispatch("email_received", email_data)` so deployment-mode listeners see it-4. Returns the result (single-shot in standalone mode; the deployment manager's poll coroutine keeps looping)+3. Calls `dispatch_email_received(email_data)` (in `_events.py`), which builds a `WorkflowEvent` (`type="com.machinaos.email.message.received"`) and routes it via a single `dispatch.emit` so Temporal-durable `TriggerListenerWorkflow` consumers fire+4. Returns the first new email's result (single-shot in standalone mode; the `PollingTriggerNode` deployment loop keeps polling)## Deployment Mode (Continuous Polling)-**File:** `server/services/deployment/manager.py`+When a workflow containing `emailReceive` is **deployed** (not just run once), the trigger runs as a continuous polling loop owned by its `PollingTriggerNode` base class — not by a hand-written factory in the deployment manager.-When a workflow containing `emailReceive` is **deployed** (not just run once), the `DeploymentManager` routes the trigger to a polling coroutine instead of a one-shot execution:--```python-def _create_poll_coroutine(self, node_type, node_id, params):- if node_type == 'emailReceive':- return self._create_email_poll_coroutine(node_id, params)-```+`EmailReceiveNode(PollingTriggerNode)` declares the polling hooks (`poll_ids`, `fetch_detail`); the base class's poll-coroutine factory self-registers via `services.deployment.poll_registry.register_poll_coroutine_factory`. The deployment manager's per-type factories (`_create_poll_coroutine` / `_create_gmail_poll_coroutine` / `_create_email_poll_coroutine`) were **removed in Wave 11.I, milestone L** — the dispatch now flows through `DeploymentManager._setup_event_trigger`, which looks up the registered factory.-`_create_email_poll_coroutine` returns an async poll loop that:-1. Resolves credentials once+The poll loop:+1. Resolves credentials once via `EmailService.resolve_credentials`2. Establishes the baseline via `svc.poll_ids(creds, folder)`3. On each iteration (clamped interval from `resolve_poll_params`):- - Polls for current IDs- - Computes `new_ids = current - seen`- - For each new ID: fetches detail, optionally marks read, enqueues via `await queue.put(email_data)` for the per-run workflow executor+ - Polls for current IDs and computes `new_ids = current - seen`+ - For each new ID: fetches detail, optionally marks read, dispatches the event- Handles `asyncio.CancelledError` cleanly on teardown-The continuous loop delegates entirely to `EmailService` — no credential resolution or polling logic is duplicated in the deployment manager.+All credential resolution and IMAP access still delegate to `EmailService` — nothing is duplicated in the deployment manager. (Post-2026-05-15, with the event framework enabled, deployed `emailReceive` triggers run as a Temporal-durable `PollingTriggerWorkflow` via the canary registration in `nodes/email/__init__.py`.)## Credentials Storage@@ -349,7 +351,7 @@ Status is shown via `getSpecialStatus(item)` returning `{ connected: !!emailStor}```-Loaded lazily by `EmailService._load_config()` and cached in a module-level `_CONFIG` variable. To add a new provider, edit only this JSON — no code changes required.+Loaded lazily by the module-level `_load_config()` in `_service.py` and cached in a module-level `_CONFIG` variable (surfaced through `EmailService.config` / `.defaults` / `.polling` properties). To add a new provider, edit only this JSON — no code changes required.**Zero magic numbers:** every default, clamp, and preset lives in this file. `resolve_poll_params` reads the polling defaults; `EmailService.read()` reads `defaults.page_size`, `defaults.flag`, etc.; the provider dropdown options in `CredentialsModal.tsx` mirror the `providers` keys.diff --git a/docs-internal/new_service_integration.md b/docs-internal/new_service_integration.mdindex 224200ed..c28fd9d3 100644--- a/docs-internal/new_service_integration.md+++ b/docs-internal/new_service_integration.md@@ -1,7 +1,15 @@# New Service Integration Guide> **⚠️ Pre-Wave-11 — historical reference only.**-> Node authoring now happens on the backend: each node is a Python plugin under `server/nodes/<category>/<node>.py` that emits a `NodeSpec`. The frontend reads specs via [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) + [adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts). See [plugin_system.md](./plugin_system.md) and [server/nodes/README.md](../server/nodes/README.md) for the current model. The snippets below that reference `client/src/nodeDefinitions/*` are kept for historical context.+> Node authoring now happens on the backend: each node is a Python plugin in a self-contained folder under `server/nodes/<category>/<plugin>/__init__.py` that emits a `NodeSpec`. The frontend reads specs via [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) + [adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts). See [plugin_system.md](./plugin_system.md) and [server/nodes/README.md](../server/nodes/README.md) for the current model. The snippets below that reference `client/src/nodeDefinitions/*` are kept for historical context.+>+> **Path-correction note (post-extraction).** Several code locations named below moved when each plugin became a self-contained folder. When following this guide, substitute:+> - `server/services/{service}_oauth.py` → `server/nodes/<plugin>/_oauth.py` (e.g. [server/nodes/google/_oauth.py](../server/nodes/google/_oauth.py), [server/nodes/twitter/_oauth.py](../server/nodes/twitter/_oauth.py))+> - `server/services/handlers/{service}.py` → the plugin's own folder (Google's handler bodies are inlined into the six service plugins under [server/nodes/google/](../server/nodes/google/) plus shared helpers `_base.py` / `_gmail.py` / `_auth_helper.py`)+> - `server/services/node_option_loaders/` → per-plugin `_option_loaders.py` registered via `register_option_loader(...)` (e.g. [server/nodes/google/_option_loaders.py](../server/nodes/google/_option_loaders.py), [server/nodes/android/_option_loaders.py](../server/nodes/android/_option_loaders.py), [server/nodes/whatsapp/_option_loaders.py](../server/nodes/whatsapp/_option_loaders.py))+> - `server/routers/{service}.py` → the plugin's `_router.py`, mounted via `register_router(...)` (e.g. [server/nodes/google/_router.py](../server/nodes/google/_router.py))+> - Handler registration in `server/services/node_executor.py` → automatic via `BaseNode.__init_subclass__`; no manual registry entry.+> `server/services/handlers/` now contains only cross-cutting orchestration — `tools.py`, `triggers.py`, `todo.py`.This guide provides a comprehensive walkthrough for integrating new external services (like Google Workspace, Slack, Notion, etc.) into MachinaOs. It covers OAuth authentication, database models, API handlers, frontend nodes, and AI Agent tool integration.@@ -15,7 +23,7 @@ A complete service integration includes:2. **Database Models** - Store OAuth tokens and connection state3. **API Handlers** - Execute service operations (CRUD)4. **Pydantic input model + NODE_METADATA entry** - The Wave 6 recipe replaces the bulk of the legacy frontend node definition work. See [node_creation.md Wave 6 section](./node_creation.md#wave-6-recommended-recipe-backend-first).-5. **Dynamic-option loaders** - If your service has dropdown fields needing live data (label list, channel list, calendar list), add a loader to [server/services/node_option_loaders/](../server/services/node_option_loaders/) and register in `LOAD_OPTIONS_REGISTRY`. One-line registration; see Google Workspace loaders as the reference (`gmailLabels`, `googleCalendarList`, `googleDriveFolders`, `googleTasklists`).+5. **Dynamic-option loaders** - If your service has dropdown fields needing live data (label list, channel list, calendar list), add a loader to your plugin folder's `_option_loaders.py` and register it from the plugin's `__init__.py` via `register_option_loader(method_name, fn)` (from [server/services/ws_handler_registry.py](../server/services/ws_handler_registry.py)). One-line registration; see [server/nodes/google/_option_loaders.py](../server/nodes/google/_option_loaders.py) as the reference (`gmailLabels`, `googleCalendarList`, `googleDriveFolders`, `googleTasklists`).6. **Frontend visual-component routing** - Add to the appropriate `*_NODE_TYPES` list or routing branch in [client/src/Dashboard.tsx](../client/src/Dashboard.tsx) so React Flow knows which component to render.7. **AI Tool Schemas** - Enable LLM tool calling8. **Credentials Modal** - UI for managing connections@@ -68,7 +76,7 @@ The sections below detail each step using Google Workspace as the reference inte## Step 1: OAuth Service-Create `server/services/{service}_oauth.py`:+Create `server/nodes/{service}/_oauth.py` (post-extraction location; the pre-Wave-11 path was `server/services/{service}_oauth.py`):```python"""@@ -263,7 +271,7 @@ if old_exists:## Step 3: API Router-Create `server/routers/{service}.py`:+Create `server/nodes/{service}/_router.py` and register it from the plugin's `__init__.py` via `register_router(router, name="{service}")` (post-extraction location; the pre-Wave-11 path was `server/routers/{service}.py`):```python"""@@ -360,34 +368,29 @@ def _callback_html(success: bool, email: str = None, error: str = None) -> str:### Register Router-In `server/main.py`:+**No `main.py` edit needed (post-extraction).** Register the router from the plugin's `__init__.py`; `main.py` mounts every plugin-registered router via its plugin-router loop:```python-# Add to router imports-from routers.{service} import router as {service}_router+# server/nodes/{service}/__init__.py+from services.ws_handler_registry import register_router+from . import _router-# Add to app-app.include_router({service}_router)--# Add to container wiring-container.wire(modules=[- # ... existing modules- "routers.{service}",-])+register_router(_router.router, name="{service}")```+For OAuth callback paths also call `register_oauth_callback_path("{service}", "/api/{service}/callback")` (see [server/nodes/google/__init__.py](../server/nodes/google/__init__.py) for the full self-registration set).+---## Step 4: WebSocket Handlers-Add handlers in `server/routers/websocket.py`:+Add handlers in your plugin folder's `_handlers.py` and register them from `__init__.py` via `register_ws_handlers(WS_HANDLERS)` (post-extraction; the pre-Wave-11 path was `server/routers/websocket.py`). See [server/nodes/google/_handlers.py](../server/nodes/google/_handlers.py) for the `google_oauth_login` / `google_oauth_status` / `google_logout` reference:```python-@ws_handler()async def handle_{service}_oauth_login(data: Dict[str, Any], websocket: WebSocket) -> Dict[str, Any]:"""Initiate OAuth flow."""import webbrowser- from services.{service}_oauth import ServiceOAuth+ from ._oauth import ServiceOAuthauth_service = container.auth_service()client_id = await auth_service.get_api_key("{service}_client_id")@@ -415,19 +418,26 @@ async def handle_{service}_oauth_status(data: Dict[str, Any], websocket: WebSockreturn {"connected": True, "email": "..."}-# Add to MESSAGE_HANDLERS dict-MESSAGE_HANDLERS = {- # ... existing handlers+# Plugin-owned dict, registered from __init__.py+WS_HANDLERS = {"{service}_oauth_login": handle_{service}_oauth_login,"{service}_oauth_status": handle_{service}_oauth_status,}```+```python+# server/nodes/{service}/__init__.py+from services.ws_handler_registry import register_ws_handlers+from ._handlers import WS_HANDLERS++register_ws_handlers(WS_HANDLERS)+```+---## Step 5: Service Handlers-Create `server/services/handlers/{service}.py`:+**Post-Wave-11 the operation logic is inlined into the plugin class** as `@Operation`-decorated methods on the `ActionNode` / `ToolNode` subclass in `server/nodes/{service}/{plugin}/__init__.py`; shared helpers live alongside as `_base.py` / `_<service>.py`. The standalone `server/services/handlers/{service}.py` file no longer exists (Google's six operations, for example, are inlined into the six service plugins under [server/nodes/google/](../server/nodes/google/) with shared helpers `_base.py` / `_gmail.py` / `_auth_helper.py`). The template below shows the pre-Wave-11 handler shape for historical reference:```python"""@@ -501,20 +511,7 @@ async def handle_{operation1}(### Register Handlers-In `server/services/node_executor.py`:--```python-# Add imports-from services.handlers.{service} import (- handle_{operation1},- handle_{operation2},- # etc.-)--# Add to handler registry in _build_handler_registry()-'{operation1}Node': handle_{operation1},-'{operation2}Node': handle_{operation2},-```+**No `node_executor.py` edit needed (post-Wave-11).** The plugin class auto-registers its handler via `BaseNode.__init_subclass__` on import — dispatch flows through `BaseNode.execute()`. The pre-Wave-11 manual registry entry in `server/services/node_executor.py::_build_handler_registry()` is gone.---@@ -674,38 +671,36 @@ case '{service}_oauth_complete':The Google Workspace integration demonstrates this pattern with:-### Files Created+**Note:** post-extraction, all Google Workspace code lives inside the self-contained plugin folder [server/nodes/google/](../server/nodes/google/). The former `server/services/google_oauth.py`, `server/services/handlers/{calendar,drive,gmail,sheets,tasks,contacts}.py`, and `server/services/handlers/google_auth.py` files are gone — their bodies moved into the plugin folder (OAuth client → `_oauth.py`, shared credential builder → `_auth_helper.py`, per-service operations inlined into the service plugins). The pre-Wave-11 `client/src/nodeDefinitions/*.ts` files no longer exist at all.++### Files (current layout)| File | Purpose ||------|---------|-| `server/services/google_oauth.py` | OAuth 2.0 with google-auth-oauthlib |-| `server/services/handlers/calendar.py` | Calendar CRUD handlers |-| `server/services/handlers/drive.py` | Drive file handlers |-| `server/services/handlers/sheets.py` | Sheets data handlers |-| `server/services/handlers/tasks.py` | Tasks CRUD handlers |-| `server/services/handlers/contacts.py` | Contacts handlers |+| `server/nodes/google/__init__.py` | Self-registration: WS handlers, router, OAuth callback path, service-refresh, filter builder, 4 option loaders, canary trigger |+| `server/nodes/google/_oauth.py` | OAuth 2.0 client with google-auth-oauthlib (was `services/google_oauth.py`) |+| `server/nodes/google/_auth_helper.py` | Shared credential builder + proactive refresh (was `services/handlers/google_auth.py`) |+| `server/nodes/google/_credentials.py` | `GoogleCredential` (OAuth2) — referenced by all 6 service plugins |+| `server/nodes/google/_handlers.py` | WS handlers: `google_oauth_login` / `google_oauth_status` / `google_logout` |+| `server/nodes/google/_router.py` | HTTP OAuth callback (`/api/google/callback`) |+| `server/nodes/google/_option_loaders.py` | `gmailLabels` / `googleCalendarList` / `googleDriveFolders` / `googleTasklists` loaders |+| `server/nodes/google/_base.py`, `_gmail.py` | Shared helpers (`build_google_service`, `run_sync`, `track_google_usage`, Gmail-specific) |+| `server/nodes/google/{gmail,calendar,drive,sheets,tasks,contacts}/__init__.py` | 6 service plugins (operation logic inlined) |+| `server/nodes/google/gmail_receive/__init__.py` | Polling trigger plugin || `server/config/google_apis.json` | API endpoints and scopes |-| `client/src/nodeDefinitions/calendarNodes.ts` | Calendar node definitions |-| `client/src/nodeDefinitions/driveNodes.ts` | Drive node definitions |-| `client/src/nodeDefinitions/sheetsNodes.ts` | Sheets node definitions |-| `client/src/nodeDefinitions/tasksNodes.ts` | Tasks node definitions |-| `client/src/nodeDefinitions/contactsNodes.ts` | Contacts node definitions |-### Files Modified+### Files Modified (cross-cutting)| File | Changes ||------|---------|| `server/models/database.py` | Added `GoogleConnection` model || `server/core/database.py` | Added CRUD methods, migration logic |-| `server/services/node_executor.py` | Added handler registry entries |-| `server/services/ai.py` | Added Pydantic schemas for AI tools |+| `server/services/ai.py` | Added Pydantic schemas for AI tools (dual-purpose nodes) || `server/services/handlers/tools.py` | Added tool dispatchers |+| `server/services/node_output_schemas.py` | Added per-operation output schemas || `server/config/pricing.json` | Added Google API pricing |-| `client/src/nodeDefinitions.ts` | Imported all Google node definitions |-| `client/src/components/CredentialsModal.tsx` | Added Google Workspace panel |-| `client/src/components/parameterPanel/InputSection.tsx` | Added output schemas |-| `client/src/Dashboard.tsx` | Added node type mappings |-| `client/src/contexts/WebSocketContext.tsx` | Added OAuth complete handler |++No frontend edits: the node definitions, credentials panel, output schemas, and Dashboard component routing are all now driven by the backend `NodeSpec` + `credential_providers.json` catalogue.### Combined OAuth Scopes@@ -741,7 +736,7 @@ GOOGLE_WORKSPACE_SCOPES = [Use this checklist when adding a new service:-- [ ] **OAuth Service** (`server/services/{service}_oauth.py`)+- [ ] **OAuth Service** (`server/nodes/{service}/_oauth.py`)- [ ] Define scopes- [ ] Implement `generate_authorization_url()`- [ ] Implement `exchange_code()`@@ -754,44 +749,34 @@ Use this checklist when adding a new service:- [ ] Add CRUD methods (save, get, delete, update_last_used)- [ ] Add migration for table rename if needed-- [ ] **API Router** (`server/routers/{service}.py`)+- [ ] **API Router** (`server/nodes/{service}/_router.py`)- [ ] Add callback endpoint- [ ] Add status endpoint- [ ] Add logout endpoint- [ ] Add customer endpoints if needed- - [ ] Register in `main.py`- - [ ] Add to container wiring+ - [ ] Register from `__init__.py` via `register_router(...)` + `register_oauth_callback_path(...)`-- [ ] **WebSocket Handlers** (`server/routers/websocket.py`)+- [ ] **WebSocket Handlers** (`server/nodes/{service}/_handlers.py`)- [ ] Add `handle_{service}_oauth_login`- [ ] Add `handle_{service}_oauth_status`- - [ ] Add to MESSAGE_HANDLERS dict+ - [ ] Collect into a `WS_HANDLERS` dict, register from `__init__.py` via `register_ws_handlers(...)`-- [ ] **Service Handlers** (`server/services/handlers/{service}.py`)- - [ ] Add `_get_service()` helper- - [ ] Add handler for each operation- - [ ] Register in node_executor.py+- [ ] **Operation Logic** (plugin classes in `server/nodes/{service}/{plugin}/__init__.py`)+ - [ ] Add `@Operation`-decorated methods for each operation+ - [ ] Shared helpers in `_base.py` / `_<service>.py`, credential builder in `_auth_helper.py`+ - [ ] Auto-registers via `BaseNode.__init_subclass__` — no `node_executor.py` edit-- [ ] **Frontend Nodes** (`client/src/nodeDefinitions/{service}Nodes.ts`)- - [ ] Define common properties- - [ ] Add node definition for each operation- - [ ] Export node types array- - [ ] Import in `nodeDefinitions.ts`- - [ ] Add to Dashboard.tsx- - [ ] **Add to `executionService.ts` `isNodeTypeSupported()`** (CRITICAL - enables Run button)+- [ ] **Frontend Nodes** — none. The `NodeSpec` emitted by the plugin drives the palette, canvas component, and parameter panel automatically (see [node_creation.md](./node_creation.md)).- [ ] **AI Tool Integration** (`server/services/ai.py`, `server/services/handlers/tools.py`)- [ ] Add Pydantic schemas in `_get_tool_schema()`- [ ] Add dispatchers in `execute_tool()`- [ ] Add handler functions-- [ ] **Output Schemas** (`client/src/components/parameterPanel/InputSection.tsx`)- - [ ] Add output schema for drag-and-drop variables+- [ ] **Output Schemas** (`server/services/node_output_schemas.py`)+ - [ ] Add a Pydantic model per node type, register in `NODE_OUTPUT_SCHEMAS`-- [ ] **Credentials Modal** (`client/src/components/CredentialsModal.tsx`)- - [ ] Add to CATEGORIES- - [ ] Add icon component- - [ ] Add OAuth panel rendering+- [ ] **Credentials Modal** — no React edits. Add an entry to `server/config/credential_providers.json`, drop the brand SVG at `server/credentials/icons/<id>.svg`, and implement the `Credential` subclass in the plugin's `_credentials.py` (see Step 9).- [ ] **WebSocket Context** (`client/src/contexts/WebSocketContext.tsx`)- [ ] Add OAuth complete handlerdiff --git a/docs-internal/node-logic-flows/android/_pattern.md b/docs-internal/node-logic-flows/android/_pattern.mdindex 39f9978e..bdae38c0 100644--- a/docs-internal/node-logic-flows/android/_pattern.md+++ b/docs-internal/node-logic-flows/android/_pattern.md@@ -190,7 +190,7 @@ preserved.## Related-- Android service infrastructure: [`server/services/android/`](../../../server/services/android/)+- Android service infrastructure: [`server/nodes/android/`](../../../server/nodes/android/)- Status broadcaster (two-state model): [Status Broadcaster](../../status_broadcaster.md)- Android-agent skills: [`server/skills/android_agent/`](../../../server/skills/android_agent/)- AI-tool dispatch (`androidTool` aggregator + direct service tools):diff --git a/docs-internal/node_panels.md b/docs-internal/node_panels.mdindex d5f31127..c3045fc2 100644--- a/docs-internal/node_panels.md+++ b/docs-internal/node_panels.md@@ -128,11 +128,9 @@ than `success`/`error` (e.g. `running`) are ignored.Locked in by the test suite at:- [client/src/hooks/__tests__/useDragVariable.test.ts](../client/src/hooks/__tests__/useDragVariable.test.ts)-- [client/src/hooks/__tests__/useParameterPanel.test.ts](../client/src/hooks/__tests__/useParameterPanel.test.ts)- [client/src/components/parameterPanel/__tests__/MiddleSection.test.tsx](../client/src/components/parameterPanel/__tests__/MiddleSection.test.tsx)- [client/src/components/parameterPanel/__tests__/InputSection.test.tsx](../client/src/components/parameterPanel/__tests__/InputSection.test.tsx)- [client/src/components/parameterPanel/__tests__/OutputSection.test.tsx](../client/src/components/parameterPanel/__tests__/OutputSection.test.tsx)-- [client/src/components/__tests__/OutputPanel.test.tsx](../client/src/components/__tests__/OutputPanel.test.tsx)1. **Defaults loaded** from `nodeDefinition.properties[].default`; missing default ⇒ `null`.2. **Saved params win** over defaults when merged (DB is source of truth).@@ -165,9 +163,7 @@ Locked in by the test suite at:cd clientnpm installnpm run test:run -- src/hooks/__tests__/useDragVariable.test.ts \- src/hooks/__tests__/useParameterPanel.test.ts \- src/components/parameterPanel/__tests__ \- src/components/__tests__/OutputPanel.test.tsx+ src/components/parameterPanel/__tests__```Or use the dedicated script (added in `client/package.json`):diff --git a/docs-internal/plugin_system.md b/docs-internal/plugin_system.mdindex ebe98156..261b17d1 100644--- a/docs-internal/plugin_system.md+++ b/docs-internal/plugin_system.md@@ -636,30 +636,30 @@ moving 754 lines out of `services/telegram_service.py`.```server/-├── nodes/ # One file per node+├── nodes/ # One self-contained folder per plugin (Wave 11.H)│ ├── __init__.py # pkgutil.walk_packages discovery│ ├── groups.py # Palette group metadata-│ ├── agent/ # AI agents (aiAgent, chatAgent, 13 specialized)+│ ├── agent/ # AI agents (aiAgent, chatAgent + 16 specialized/variant)│ │ ├── _handles.py # Shared handle topology helpers│ │ ├── _inline.py # prepare_agent_call()│ │ ├── _specialized.py # SpecializedAgentBase-│ │ └── <agent>.py-│ ├── model/ # AI chat models (9 providers)+│ │ └── <agent>/__init__.py # one folder per agent+│ ├── model/ # AI chat models (11 providers; xai native-chat-only)│ │ ├── _base.py # ChatModelBase + ChatModelParams/Output-│ │ └── <provider>_chat_model.py+│ │ └── <provider>_chat_model/__init__.py│ ├── android/ # 16 Android service nodes│ │ ├── _base.py # AndroidServiceBase-│ │ └── <service>.py+│ │ └── <service>/__init__.py│ ├── code/ # python/js/ts executors│ │ ├── _base.py # CodeExecutorBase│ │ ├── _nodejs.py # Shared NodeJSClient singleton-│ │ └── <lang>_executor.py+│ │ └── <lang>_executor/__init__.py│ ├── filesystem/ # file_read / file_modify / shell / fs_search│ │ ├── _backend.py # Shared LocalShellBackend helper-│ │ └── <op>.py+│ │ └── <op>/__init__.py│ ├── document/ # http_scraper / parser / chunker / embedding / vector│ │ ├── _helpers.py # delegate() wrapper-│ │ └── <stage>.py+│ │ └── <stage>/__init__.py│ ├── google/ # gmail / calendar / drive / sheets / tasks / contacts│ ├── proxy/ # proxy_request / proxy_config / proxy_status│ │ └── _usage.py # Shared track_proxy_usagediff --git a/docs-internal/pricing_service.md b/docs-internal/pricing_service.mdindex 44e3cebe..117803ef 100644--- a/docs-internal/pricing_service.md+++ b/docs-internal/pricing_service.md@@ -41,9 +41,11 @@ The pricing service provides centralized cost tracking for both LLM tokens and e▼ ▼ ▼┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐│ Manual Tracking │ │ HTTPX Event Hooks │ │ AI Service │-│ _track_*_usage() │ │ tracked_http.py │ │ _track_token_ │-│ maps.py │ │ │ │ usage() ai.py │-│ twitter.py │ │ │ │ │+│ track_*_usage() │ │ tracked_http.py │ │ _track_token_ │+│ location/_service │ │ │ │ usage() ai.py │+│ twitter/_base │ │ │ │ │+│ google/_base │ │ │ │ │+│ proxy/_usage │ │ │ │ │└───────────────────┘ └───────────────────┘ └───────────────────┘│ │ │└─────────────────┼─────────────────┘@@ -218,7 +220,7 @@ For services that make API calls directly (not through HTTPX), use manual tracki### Example: Maps Service```python-# server/services/maps.py+# server/nodes/location/_service.pyasync def _track_maps_usage(node_id: str,@@ -228,12 +230,12 @@ async def _track_maps_usage(session_id: str = "default") -> Dict[str, float]:"""Track Google Maps API usage."""- from core.container import container+ from services.plugin.deps import get_databasepricing = get_pricing_service()cost_data = pricing.calculate_api_cost('google_maps', action, resource_count)- db = container.database()+ db = get_database()await db.save_api_usage_metric({'session_id': session_id,'node_id': node_id,@@ -247,44 +249,56 @@ async def _track_maps_usage(return cost_data-# Usage in handler:-async def geocode_location(self, node_id: str, parameters: Dict, context: Dict):+# Usage inside the maps plugins (server/nodes/location/{gmaps_locations,gmaps_nearby_places,...}):# ... API call ...- await _track_maps_usage(node_id, 'geocode', 1, context.get('workflow_id'))+ await _track_maps_usage(node_id, 'geocode', 1, context.get('workflow_id'), context.get('session_id', 'default'))```-### Example: Twitter Handler+### Example: Twitter Plugin++The Twitter plugins (`server/nodes/twitter/twitter_send`, `twitter_search`, `twitter_user`) share one tracker defined on `server/nodes/twitter/_base.py`. It takes the raw execution context dict (not separate `workflow_id` / `session_id` args).```python-# server/services/handlers/twitter.py+# server/nodes/twitter/_base.py-async def _track_twitter_usage(node_id, action, resource_count=1, workflow_id=None, session_id="default"):+async def track_twitter_usage(node_id, action, resource_count, context):# Same pattern as maps+ from services.plugin.deps import get_databasepricing = get_pricing_service()cost_data = pricing.calculate_api_cost('twitter', action, resource_count)- # ... save to database ...+ # ... save to database (reads session_id / workflow_id from `context`) ...-# Usage:-await _track_twitter_usage(node_id, 'tweet', 1, workflow_id, session_id)+# Usage (ctx_raw is the plugin's raw NodeContext dict):+await track_twitter_usage(node_id, 'tweet', 1, ctx_raw)+await track_twitter_usage(node_id, 'search', len(tweets), ctx_raw)```-### Example: Search Handler+### Example: Google Workspace Plugins++The 6 Google service plugins (`server/nodes/google/{gmail,calendar,drive,sheets,tasks,contacts}`) plus `gmail_receive` call `track_google_usage()` on `server/nodes/google/_base.py`. Google Workspace APIs are free within rate limits, so this records resource counts at `cost=$0` for analytics.```python-# server/services/handlers/search.py+# server/nodes/google/_base.py-async def _track_search_usage(node_id, service, action, resource_count=1, workflow_id=None, session_id="default"):- # Same pattern as maps/twitter+async def track_google_usage(node_id, service, action, resource_count, context):pricing = get_pricing_service()cost_data = pricing.calculate_api_cost(service, action, resource_count)# ... save to database ...+```++### Example: Search Plugins (declarative cost)-# Usage in each handler:-await _track_search_usage(node_id, 'brave_search', 'web_search', 1, workflow_id, session_id)-await _track_search_usage(node_id, 'serper', 'web_search', 1, workflow_id, session_id)-await _track_search_usage(node_id, 'perplexity', 'sonar_search', 1, workflow_id, session_id)+The search plugins (`server/nodes/search/{brave_search,serper_search,perplexity_search}`) no longer call a `_track_search_usage()` helper. Each declares its cost inline on the `@Operation` decorator instead:++```python+# server/nodes/search/brave_search/__init__.py++@Operation("search", cost={"service": "brave_search", "action": "web_search", "count": 1})+async def search(self, ctx, params): ...```+The `cost=` metadata is attached to the operation's `OperationSpec` (`server/services/plugin/operation.py`); `duckduckgo_search` is free and carries no cost block.+## Automatic HTTPX TrackingFor services using `httpx.AsyncClient`, use the tracked client for transparent tracking.@@ -443,14 +457,14 @@ await getApiUsage('twitter');### 2. Add Manual Tracking (if not using HTTPX)```python-# server/services/handlers/new_service.py+# server/nodes/<group>/<plugin>/_base.py (or the plugin __init__.py)-async def _track_new_service_usage(node_id, action, count=1, workflow_id=None, session_id="default"):- from core.container import container+async def track_new_service_usage(node_id, action, count=1, workflow_id=None, session_id="default"):+ from services.plugin.deps import get_databasepricing = get_pricing_service()cost_data = pricing.calculate_api_cost('new_service', action, count)- db = container.database()+ db = get_database()await db.save_api_usage_metric({'session_id': session_id,'node_id': node_id,@@ -463,10 +477,12 @@ async def _track_new_service_usage(node_id, action, count=1, workflow_id=None, s})return cost_data-# Call in handler:-await _track_new_service_usage(node_id, 'get_data', len(results), workflow_id)+# Call inside the plugin's execute_op / operation method:+await track_new_service_usage(node_id, 'get_data', len(results), workflow_id)```+Alternatively, for a service billed per operation, skip the helper entirely and declare the cost inline via `@Operation("get_data", cost={"service": "new_service", "action": "read_operation", "count": 1})` (the search-plugin pattern).+### 3. Add URL Patterns (if using HTTPX)```json@@ -497,9 +513,12 @@ await _track_new_service_usage(node_id, 'get_data', len(results), workflow_id)| `server/config/pricing.json` | Pricing configuration (user-editable) || `server/services/pricing.py` | PricingService with LLM and API cost calculation || `server/services/tracked_http.py` | HTTPX event hooks for automatic tracking |-| `server/services/maps.py` | Manual tracking example (`_track_maps_usage`) |-| `server/services/handlers/twitter.py` | Manual tracking example (`_track_twitter_usage`) |-| `server/services/handlers/search.py` | Manual tracking example (`_track_search_usage`) |+| `server/nodes/location/_service.py` | Maps manual tracker (`_track_maps_usage`) |+| `server/nodes/twitter/_base.py` | Twitter manual tracker (`track_twitter_usage`) |+| `server/nodes/google/_base.py` | Google Workspace manual tracker (`track_google_usage`, cost $0) |+| `server/nodes/proxy/_usage.py` | Proxy manual tracker (`track_proxy_usage`, per-GB) |+| `server/nodes/search/{brave_search,serper_search,perplexity_search}/__init__.py` | Search plugins — declarative `@Operation(cost={...})` (no `_track_search_usage`) |+| `server/services/plugin/operation.py` | `@Operation` decorator + `OperationSpec.cost` metadata || `server/models/database.py` | `APIUsageMetric`, `TokenUsageMetric` models || `server/core/database.py` | `save_api_usage_metric()`, `get_api_usage_summary()` || `client/src/components/CredentialsModal.tsx` | `renderApiUsagePanel()` UI component |diff --git a/docs-internal/proxy_service.md b/docs-internal/proxy_service.mdindex 005c2940..162b16f5 100644--- a/docs-internal/proxy_service.md+++ b/docs-internal/proxy_service.md@@ -25,10 +25,10 @@ Residential proxy provider management with template-based URL formatting, health### Request Flow```-httpRequest(useProxy=true)+httpRequest(use_proxy=true)│▼-_get_proxy_url_if_enabled()+_resolve_proxy_url() (nodes/utility/http_request)│▼ProxyService.get_proxy_url(url, parameters)@@ -53,19 +53,22 @@ httpx.AsyncClient(proxy=proxy_url)| File | Description ||------|-------------|-| `server/services/proxy/__init__.py` | Exports `get_proxy_service`, `init_proxy_service`, `ProxyService` |+| `server/services/proxy/__init__.py` | Exports `ProxyService`, `get_proxy_service` (`init_proxy_service` lives in `service.py` and is imported there directly) || `server/services/proxy/service.py` | ProxyService singleton - provider selection, URL generation, health scoring || `server/services/proxy/providers.py` | TemplateProxyProvider - formats proxy URLs from JSON template config || `server/services/proxy/models.py` | Pydantic models: ProviderConfig, RoutingRule, SessionType, GeoTarget, ProxyResult, ProviderStats |-| `server/services/proxy/exceptions.py` | Custom exceptions: NoHealthyProviderError, BudgetExceededError, ProxyConfigError, ProviderError |-| `server/services/handlers/proxy.py` | Node handlers: handle_proxy_request, handle_proxy_status, handle_proxy_config |-| `server/services/handlers/http.py` | `_get_proxy_url_if_enabled()` shared helper for transparent proxy injection |-| `server/services/handlers/tools.py` | `_execute_proxy_config()`, `_execute_proxy_request()`, `_execute_proxy_status()` AI tool handlers |-| `server/services/handlers/document.py` | httpScraper handler with proxy injection |+| `server/services/proxy/exceptions.py` | Custom exceptions: ProxyError, ProviderError, NoHealthyProviderError, BudgetExceededError, ProxyConfigError |+| `server/nodes/proxy/proxy_request/__init__.py` | `proxyRequest` plugin - direct proxy HTTP request with retry/failover + usage tracking |+| `server/nodes/proxy/proxy_config/__init__.py` | `proxyConfig` plugin (dual-purpose) - `execute_proxy_config()` + the per-operation helpers (`_add_provider`, `_set_credentials`, `_test_provider`, routing-rule ops, ...) |+| `server/nodes/proxy/proxy_status/__init__.py` | `proxyStatus` plugin - provider health stats |+| `server/nodes/proxy/_usage.py` | `track_proxy_usage()` - shared cost tracker (computes cost from `pricing.json:proxy.<provider>.cost_per_gb`, writes `APIUsageMetric`) |+| `server/nodes/utility/http_request/__init__.py` | `httpRequest` plugin - `_resolve_proxy_url()` transparent proxy injection helper (`use_proxy` param) |+| `server/nodes/document/http_scraper/__init__.py` | `httpScraper` plugin - inline proxy injection via `get_proxy_service().get_proxy_url()` (`use_proxy` param) |+| `server/nodes/scraper/crawlee_scraper/__init__.py` | `crawleeScraper` plugin - `_get_proxy_config()` proxy injection |+| `server/services/handlers/tools.py` | AI-tool dispatch - the `proxyConfig` tool merges params then calls `nodes.proxy.proxy_config.execute_proxy_config()` || `server/core/database.py` | CRUD: get/save/delete proxy providers and routing rules || `server/models/database.py` | SQLModel tables: ProxyProvider, ProxyRoutingRule |-| `server/core/container.py` | DI: `proxy_service()` factory |-| `server/main.py` | Startup: `proxy_svc.startup()` after database init |+| `server/main.py` | Startup: `init_proxy_service(...)` then `proxy_svc.startup()` after database init (not wired through the DI container) || `server/constants.py` | `PROXY_NODE_TYPES` frozenset || `server/skills/web_agent/proxy-config-skill/SKILL.md` | AI skill: proxy provider setup and management || `server/skills/web_agent/http-request-skill/SKILL.md` | AI skill: HTTP requests with `useProxy: true` |@@ -96,17 +99,24 @@ View provider health stats. Optionally filter by provider name.## Transparent Proxy on HTTP Nodes-The `httpRequest` and `httpScraper` nodes support `useProxy: true`. When set, `_get_proxy_url_if_enabled()` calls `ProxyService.get_proxy_url()` and passes the result to `httpx.AsyncClient(proxy=...)`. If proxy lookup fails, the request proceeds without proxy (graceful degradation).+The `httpRequest`, `httpScraper`, and `crawleeScraper` plugins support a `use_proxy` parameter. When set, the plugin calls `ProxyService.get_proxy_url()` and passes the result to `httpx.AsyncClient(proxy=...)`. `httpRequest` routes through `_resolve_proxy_url()` (which swallows exceptions so the request proceeds without proxy on failure); `httpScraper` inlines the same lookup directly. Graceful degradation either way.-The AI Agent's `http_request` tool schema includes `useProxy: bool`. The LLM only sets the flag -- the proxy service handles provider selection, geo-targeting, and session type from its own configuration.+`httpRequest` is also a dual-purpose AI tool (`tool_name = "http_request"`, `group = ("utility", "tool")`). Its Pydantic `Params` is the AI-tool schema, so `use_proxy` is exposed to the LLM. The LLM only sets the flag -- the proxy service handles provider selection, geo-targeting, and session type from its own configuration.```python-# In server/services/ai.py-class HttpRequestSchema(BaseModel):- url: str- method: str = "GET"- body: Optional[Dict[str, Any]] = None- useProxy: bool = False # LLM sets this, proxy service handles the rest+# In server/nodes/utility/http_request/__init__.py+class HttpRequestNode(ActionNode):+ type = "httpRequest"+ tool_name = "http_request"++ class Params(BaseModel):+ url: str+ method: str = "GET"+ body: Optional[Any] = None+ use_proxy: bool = False # LLM sets this, proxy service handles the rest+ proxy_provider: str = "auto"+ proxy_country: str = ""+ session_type: Literal["rotating", "sticky"] = "rotating"```## Template System@@ -213,12 +223,12 @@ Rules are matched first-match against the target URL's hostname.Proxy credentials are stored via AuthService using the pattern `proxy_{name}_username` and `proxy_{name}_password`:```python-# In _execute_proxy_config set_credentials operation:-await auth_service.store_api_key(f"proxy_{name}_username", username, [])-await auth_service.store_api_key(f"proxy_{name}_password", password, [])+# In nodes/proxy/proxy_config _set_credentials operation:+await auth_svc.store_api_key(f"proxy_{name}_username", username, [])+await auth_svc.store_api_key(f"proxy_{name}_password", password, [])```-On startup, `ProxyService._load_providers()` reads credentials from AuthService for each provider.+On startup, `ProxyService.startup()` loads providers from the DB and `_load_providers()` reads credentials from AuthService for each provider.## Database Tables@@ -268,14 +278,22 @@ PROXY_BUDGET_DAILY_USD= # Daily spend limit (empty = unlimited)```python# In server/main.py startup-proxy_svc = container.proxy_service() # Creates singleton via init_proxy_service()-await proxy_svc.startup() # Loads providers from DB + credentials from AuthService+from services.proxy.service import init_proxy_service+proxy_svc = init_proxy_service( # Builds + stores the module-level singleton+ auth_service=container.auth_service(),+ database=container.database(),+ settings=settings,+)+await proxy_svc.startup() # Loads providers from DB + credentials from AuthService# In server/main.py shutdown-await proxy_svc.shutdown() # Clears in-memory state+from services.proxy.service import get_proxy_service+_proxy_svc = get_proxy_service()+if _proxy_svc:+ await _proxy_svc.shutdown() # Clears in-memory state```-The proxy service always initializes regardless of `PROXY_ENABLED`. Providers are managed dynamically by the LLM via the `proxy_config` tool. If no providers are configured, `get_proxy_url()` raises `NoHealthyProviderError`.+`init_proxy_service` / `get_proxy_service` live in `services/proxy/service.py` (the latter is re-exported from `services/proxy/__init__.py`); the singleton is module-level, not wired through the DI container. The proxy service always initializes regardless of `PROXY_ENABLED`. Providers are managed dynamically by the LLM via the `proxyConfig` tool. If no providers are configured, `get_proxy_url()` raises `NoHealthyProviderError`.## Adding a New Provider (AI Workflow)@@ -289,6 +307,6 @@ If the provider uses an unfamiliar URL format, the LLM can use `python_code` to## Cost Tracking-Proxy usage is tracked via the pricing service. When a proxied request completes, `_track_proxy_usage()` in `handlers/proxy.py` calculates cost based on bytes transferred and the provider's `cost_per_gb` from `server/config/pricing.json`. The cost is persisted to the `APIUsageMetric` table.+Proxy usage is tracked via the pricing service. When a proxied request completes, `track_proxy_usage()` in `server/nodes/proxy/_usage.py` (called by the `proxyRequest` plugin) calculates cost based on bytes transferred and the provider's `cost_per_gb` from `server/config/pricing.json` (`proxy.<provider>.cost_per_gb`). The cost is persisted to the `APIUsageMetric` table.The daily spend is also tracked in-memory on `ProxyService._daily_spend_usd` and checked against `PROXY_BUDGET_DAILY_USD` before each request.diff --git a/docs-internal/schema_source_of_truth_rfc.md b/docs-internal/schema_source_of_truth_rfc.mdindex 26a4b208..c91124c1 100644--- a/docs-internal/schema_source_of_truth_rfc.md+++ b/docs-internal/schema_source_of_truth_rfc.md@@ -17,11 +17,11 @@ Wave 3 proved the pattern for output schemas. Wave 6 generalises it to the **ful**Shipped contract:**-- **Input schemas** — [server/services/node_input_schemas.py](../server/services/node_input_schemas.py) registers Pydantic input models across every node type — live count via `len(_DIRECT_MODELS)` — auto-expanding `SpecializedAgentParams` and `AndroidServiceParams` via Literal union introspection.+- **Input schemas** — [server/services/node_input_schemas.py](../server/services/node_input_schemas.py) holds the `_DIRECT_MODELS` registry (aliased `NODE_INPUT_MODELS`) of Pydantic input models across every node type — live count via `len(_DIRECT_MODELS)`. Post-Wave-11 each plugin's `Params` class self-registers here at import (the earlier Wave 6 Literal-union introspection of `SpecializedAgentParams` / `AndroidServiceParams` is gone — specialized agents and Android service nodes are now individual plugin folders under `server/nodes/agent/` and `server/nodes/android/`).- **NodeSpec envelope** — [server/services/node_spec.py](../server/services/node_spec.py) assembles `{type, displayName, icon, group, description, version, inputs (JSON Schema), outputs (JSON Schema, reuses Wave 3), credentials, uiHints}` fusing three sources of truth.-- **Display metadata** — [server/models/node_metadata.py](../server/models/node_metadata.py) carries one entry per input-modeled type — live size via `len(NODE_METADATA)`. The only new server-side data file.+- **Display metadata** — [server/models/node_metadata.py](../server/models/node_metadata.py) defines the `NodeMetadata` / `NodeHandle` TypedDicts and the `NODE_METADATA` dict — live size via `len(NODE_METADATA)`. Post-Wave-11 it starts empty at import and is plugin-populated via `services.node_registry.register_node` (called from `BaseNode.__init_subclass__`), rather than being hand-maintained.- **Endpoints** — alongside the Wave 3 `/nodes/{type}.json` output endpoint: `GET /api/schemas/nodes/{type}/spec.json`, `/nodes/{type}/input.json`, `/nodes/specs` (list), all with 24h `Cache-Control`. WS mirrors: `get_node_spec`, `list_node_specs`. **Icon endpoints (RFC §6.5 + F7):** `GET /api/schemas/nodes/{type}/icon` serves the plugin's co-located `icon.svg` (or per-node-type `icon_<nodeType>.svg` for multi-node folders like whatsapp); `GET /api/schemas/credentials/{provider}/icon` serves the credential brand icon from `server/credentials/icons/<provider>.svg`. Both with long cache headers.-- **loadOptionsMethod dispatch** — [server/services/node_option_loaders/](../server/services/node_option_loaders/) generalises the WhatsApp-only pattern to a registry. Adding a new dynamic-option loader = one-line registration. `POST /api/schemas/nodes/options/{method}` + WS `load_options`.+- **loadOptionsMethod dispatch** — per-plugin `_option_loaders.py` files ([server/nodes/android/_option_loaders.py](../server/nodes/android/_option_loaders.py), [server/nodes/google/_option_loaders.py](../server/nodes/google/_option_loaders.py), [server/nodes/whatsapp/_option_loaders.py](../server/nodes/whatsapp/_option_loaders.py)) self-register into a shared loader registry, generalising the WhatsApp-only pattern. Adding a new dynamic-option loader = one registration inside the owning plugin folder. `POST /api/schemas/nodes/options/{method}` + WS `load_options`.- **Node-groups index** — `GET /api/schemas/nodes/groups` returns `{group: [node_type, ...]}` derived from every NodeSpec's `group` array — live count via `len(NODE_GROUPS)` — replacing the hand-rolled `*_NODE_TYPES` arrays scattered across 6 frontend files.- **Pydantic Field hints** — authors encode `displayOptions.show`/`.hide`, `loadOptionsMethod`, `placeholder`, `validation`, `typeOptions` via `Field(json_schema_extra={...})`. Currently ~36 rules encoded across TwitterSend/User, TelegramSend, HttpRequest, WhatsAppSend, SocialSend, Gmail, Calendar.@@ -30,14 +30,16 @@ Wave 3 proved the pattern for output schemas. Wave 6 generalises it to the **ful- `VITE_NODESPEC_BACKEND` flag at [client/src/lib/featureFlags.ts](../client/src/lib/featureFlags.ts). Defaults OFF — legacy `nodeDefinitions/*.ts` wins; ON routes rendering through backend NodeSpec via the adapter.- [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) — shared `fetchNodeSpec`/`prefetchAllNodeSpecs`/`resolveNodeDescription` helpers following the Wave 3 colocation pattern (no new hook file).- [client/src/adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts) — NodeSpec wire shape → legacy `INodeTypeDescription` bridge. Handles `required`, JSON Schema `format` (dateTime/file/binary), `displayOptions`, `validation`, typeOptions lift (`loadOptionsMethod`, `password`, `rows`, `editor`, `editorLanguage`, `accept`, …), and reads hints from both top-level and nested `uiHints` wrapper.-- Dashboard idle-time prefetch warms all 110 NodeSpecs after WebSocket connect (no-op when flag off).+- Dashboard idle-time prefetch (`prefetchAllNodeSpecs` in [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts)) warms every NodeSpec after WebSocket connect. (The `VITE_NODESPEC_BACKEND` flag is now the default-on path — backend NodeSpec is the SSOT; the legacy `nodeDefinitions/*.ts` files were deleted in the Wave 7/10 reductions.)-**Adding a new node type's parameter surface:**+**Adding a new node type's parameter surface** (Wave 6 recipe; superseded post-Wave-11 — see the plugin recipe in [10.C](#10c--register_node-decorator) and [Wave 11](#wave-11--plugin-first-class-hierarchy)):-1. Define `XyzParams(BaseNodeParams)` in [server/models/nodes.py](../server/models/nodes.py) with `type: Literal["xyz"]` discriminator and Field constraints.-2. Register in `KnownNodeParams` discriminated union.-3. Register in `_DIRECT_MODELS` dict in `node_input_schemas.py`.-4. Add `NODE_METADATA["xyz"] = {displayName, icon, group, description, version}` in `node_metadata.py`.+At Wave 6 the surface was three separate data files:++1. Define `XyzParams(BaseNodeParams)` with a `type: Literal["xyz"]` discriminator and Field constraints, register it in the `KnownNodeParams` discriminated union, and register it in the `_DIRECT_MODELS` dict in `node_input_schemas.py`.+2. Add `NODE_METADATA["xyz"] = {displayName, icon, group, description, version}` in `node_metadata.py`.++Post-Wave-11 all three collapse onto the plugin class: `Params` is declared on the `BaseNode` subclass in `server/nodes/<group>/<plugin>/__init__.py`, and `register_node` (via `__init_subclass__`) writes the class's `Params` into `_DIRECT_MODELS` (aliased as `NODE_INPUT_MODELS`) and its metadata into `NODE_METADATA` at import time. There is no `server/models/nodes.py` and no `KnownNodeParams` union any more — plugin `Params` extend `BaseModel` directly (no `type:` discriminator).Zero frontend change for parameter rendering. Icons that are SVG assets still need frontend import until SVG payload migration is its own follow-up.@@ -55,7 +57,9 @@ The editor's Input panel (`client/src/components/parameterPanel/InputSection.tsx1. A 350-line `sampleSchemas` map inline in `InputSection.tsx` (59 entries).2. As-of Wave 3 Phase 1 Batch 1 (commit `f1b2813`), a new `outputSchema` field on each `INodeTypeDescription` that the frontend consults before the legacy map.-Both are **frontend duplications of what the backend already owns** — the handler's return type (Pydantic models in `server/services/handlers/*`). Every node addition requires a frontend edit to keep schemas in sync.+Both are **frontend duplications of what the backend already owns** — the handler's return type (Pydantic models colocated with each node handler). Every node addition requires a frontend edit to keep schemas in sync.++> **Historical note.** This RFC was written pre-Wave-11 when handler bodies lived under `server/services/handlers/*`. Post-Wave-11 those bodies inlined into per-plugin folders under `server/nodes/<group>/<plugin>/__init__.py`; only `todo.py` / `tools.py` / `triggers.py` remain under `server/services/handlers/`. Path references throughout the earlier sections have been updated to their current locations; the design decisions still hold.## How this scales to 1000+ nodes elsewhere@@ -177,14 +181,17 @@ Each field flows through `get_node_spec()` into the `/api/schemas/nodes/{type}/senvelope, so every consumer (React Flow dispatch, parameter panel, palette)reads from one source.-### 10.C — `@register_node` decorator+### 10.C — `register_node` registry writer[server/services/node_registry.register_node(...)](../server/services/node_registry.py)-writes to four registries atomically: `NODE_METADATA`, `_DIRECT_MODELS`,-`NODE_OUTPUT_SCHEMAS`, `_HANDLER_REGISTRY`. `server/nodes/__init__.py` walks-`server/nodes/*.py` submodules at import time via `pkgutil.iter_modules`,-so plugin registration is side-effect at startup. 106/111 node types migrated-to this path (the remaining 5 are output-only legacy aliases).+writes to four registries in one call: `NODE_METADATA`, `_DIRECT_MODELS`,+`NODE_OUTPUT_SCHEMAS`, `_HANDLER_REGISTRY` (and invalidates the per-type+NodeSpec + input-schema caches). `server/nodes/__init__.py` recursively+imports every submodule at import time via `pkgutil.walk_packages`, so+plugin registration is a side effect at startup. Post-Wave-11 the caller is+`BaseNode.__init_subclass__` (via `register_node_class`), so every plugin+folder self-registers — live migrated count via+`glob server/nodes/**/__init__.py`.**New node checklist (post-Wave-10):**@@ -235,7 +242,7 @@ Icons live as `icon.svg` (or `icon_<nodeType>.svg` for multi-node folders like wbackend is sole declaration site. `INodeTypeDescription.icon` narrowedto `icon?: string`.-### Contract invariants (108 pytest in `tests/test_node_spec.py`)+### Contract invariants (`tests/test_node_spec.py` — live count via `pytest --collect-only`)`TestWave10GContractInvariants` enforces:@@ -289,7 +296,7 @@ API around a single `BaseNode` hierarchy. See### Contract invariants-Adds 16 invariants to the 108 Wave 10 suite → **124 total**:+Adds a further set of invariants on top of the Wave 10 suite (live total via `pytest --collect-only` on `tests/test_node_spec.py`):- Every `BaseNode` subclass has `type`, `display_name`, `group`.- `Params` + `Output` are Pydantic `BaseModel` subclasses.@@ -307,12 +314,20 @@ Plugins (live count via `glob server/nodes/**/__init__.py`) live across 9 Tempor`ai-heavy`, `code-exec`, `triggers-poll`, `triggers-event`, `android`,`browser`, `messaging`, `machina-default`). Handler bodies are fullyinlined into plugin files — `services/handlers/` shrank from-**12.8K LOC / 16 files → 1.1K LOC / 4 files**. Only cross-cutting-orchestration remains: `tools.py` (AI-tool dispatch + agent-delegation through `BaseNode.execute()` via the node registry),-`google_auth.py`, `triggers.py`. Sunset of the empty-`nodes/{agents,services,triggers,tools,utilities}.py` bulk files +-dead dispatcher fallbacks is complete (Wave 11.D.13).+**12.8K LOC / 16 files → a few files** (live set: `todo.py` / `tools.py` /+`triggers.py` + `__init__.py`). Only cross-cutting orchestration remains+there: `tools.py` (AI-tool dispatch + agent delegation through+`BaseNode.execute()` via the node registry — the live delegation handler is+`_execute_delegated_agent`), `triggers.py` (generic event-trigger handler),+and `todo.py`. The shared Google credential helper moved out of handlers+into its owning plugin folder at+[server/nodes/google/_auth_helper.py](../server/nodes/google/_auth_helper.py).+Agent connection collection lives at+[server/services/plugin/edge_walker.py::collect_agent_connections](../server/services/plugin/edge_walker.py)+(a 5-tuple: memory, skill, tool, input, task), and per-plugin pre-dispatch+prep is [server/nodes/agent/_inline.py::prepare_agent_call](../server/nodes/agent/_inline.py).+Sunset of the empty `nodes/{agents,services,triggers,tools,utilities}.py`+bulk files + dead dispatcher fallbacks is complete (Wave 11.D.13).### Sub-waves shippeddiff --git a/docs-internal/status_broadcaster.md b/docs-internal/status_broadcaster.mdindex 008a6ba0..28cf92eb 100644--- a/docs-internal/status_broadcaster.md+++ b/docs-internal/status_broadcaster.md@@ -2,11 +2,11 @@MachinaOS uses WebSocket as the primary communication channel between the React frontend and the FastAPI backend. A single `StatusBroadcaster` service manages all active WebSocket connections and broadcasts real-time state updates (node status, workflow progress, Android device status, variable changes, etc.). This replaces REST polling with push-based updates and is the reason the UI can animate node execution, show live tool calls, and update Android status without reloading.-This document covers the broadcaster architecture, the 89 WebSocket message handlers, the broadcast message types, and the Android two-state connection model.+This document covers the broadcaster architecture, the WebSocket message handlers, the broadcast message types, and the Android two-state connection model.Source files:- `server/services/status_broadcaster.py` - `StatusBroadcaster` singleton-- `server/routers/websocket.py` - WebSocket endpoint and 89 message handlers+- `server/routers/websocket.py` - WebSocket endpoint and message handlers- `client/src/contexts/WebSocketContext.tsx` - frontend WebSocket provider with hooks## Why WebSocket-First@@ -93,7 +93,7 @@ async def handle_get_node_parameters(data: Dict, websocket: WebSocket) -> Dict:The decorator populates a module-level `_HANDLERS: Dict[str, Callable]` map from the function name (`handle_get_node_parameters` -> `"get_node_parameters"`). The dispatcher reads the incoming message's `type` field and looks up the handler.-Current total: **89 WebSocket handlers** in `server/routers/websocket.py`.+Live total = `len(MESSAGE_HANDLERS) + len(get_ws_handlers())` -- the `MESSAGE_HANDLERS` dict in `server/routers/websocket.py` plus plugin-registered handlers via `services.ws_handler_registry`. Do not hand-maintain a fixed count here.### Handler Categoriesdiff --git a/docs-internal/theme_system.md b/docs-internal/theme_system.mdindex 9afd282d..b0889a18 100644--- a/docs-internal/theme_system.md+++ b/docs-internal/theme_system.md@@ -535,6 +535,6 @@ cd client && npx eslint <files>| [client/src/components/ui/DropCap.tsx](../client/src/components/ui/DropCap.tsx) | `v-display drop-cap` wrapper for Renaissance ornament rule — Wave 25 || [client/src/types/NodeTypes.ts](../client/src/types/NodeTypes.ts) | `NodeStyle` helper type for inline `--*` custom props — Wave 21 || [client/index.html](../client/index.html) | Google Fonts loading (all 10 themes' typefaces) |-| [design_handoff_machinaos_themes/](../design_handoff_machinaos_themes/) | Reference HTML mocks + token spec |-| [design_handoff_machinaos_themes/app/icons.js](../design_handoff_machinaos_themes/app/icons.js) | Upstream icon source (ported by Wave 23) |-| [design_handoff_machinaos_themes/MIGRATION_PLAYBOOK.md](../design_handoff_machinaos_themes/MIGRATION_PLAYBOOK.md) | Upstream recipe — all originally-deferred items now landed (W14–W25) |+| `design_handoff_machinaos_themes/` (external design handoff, not in repo) | Reference HTML mocks + token spec |+| `design_handoff_machinaos_themes/app/icons.js` (external) | Upstream icon source (ported by Wave 23) |+| `design_handoff_machinaos_themes/MIGRATION_PLAYBOOK.md` (external) | Upstream recipe — all originally-deferred items now landed (W14–W25) |diff --git a/docs-internal/ui_migration_plan.md b/docs-internal/ui_migration_plan.mdindex 289d94a2..ab6f2c2b 100644--- a/docs-internal/ui_migration_plan.md+++ b/docs-internal/ui_migration_plan.md@@ -26,7 +26,7 @@## Post-antd cleanup (April 2026)-A second audit (3 parallel sub-agents) flagged tribal patterns that survived the antd retirement and didn't match the schema-driven design system. A focused 5-phase follow-up plan ([typed-splashing-crown](../../../.claude/plans/typed-splashing-crown.md)) addresses them:+A second audit (3 parallel sub-agents) flagged tribal patterns that survived the antd retirement and didn't match the schema-driven design system. A focused 5-phase follow-up plan (`.claude/plans/typed-splashing-crown.md` (developer-local plan, not in repo)) addresses them:| Follow-up phase | Status | Commit | What it removes ||---|---|---|---|@@ -131,7 +131,7 @@ Plan: [C:\\Users\\Tgroh\\.claude\\plans\\typed-splashing-crown.md].| 3c — Backend parity: agents + chat models (28 types) | ✅ done | `8a8a413` | SpecializedAgentParams promoted to mirror AIAgentParams full surface (temperature/max_tokens/thinking/reasoning). 28 metadata entries. || 3d.i — Backend parity: location + scheduler + chat + Android (26 types) | ✅ done | `f3664e7` | 26 metadata entries. `test_input_model_coverage_complete` invariant locks "every input model has metadata". || 3d.ii — Backend parity: long-tail integrations (28 output-only types) | ✅ done | `2b65a7f` | 28 new Pydantic models across search, browser/scraping, email, Google Workspace, document/RAG, filesystem, proxy. `TestWave6FullCoverage` invariants. |-| 4 — Generalized loadOptionsMethod dispatch | ✅ done | `a3c1ac2` | `server/services/node_option_loaders/` package with registry + `dispatch_load_options()` + 3 WhatsApp loaders (groups/channels/members). `POST /api/schemas/nodes/options/{method}` + WS `load_options`. One-line registration for future Gmail/Calendar/Telegram loaders. |+| 4 — Generalized loadOptionsMethod dispatch | ✅ done | `a3c1ac2` | `server/services/node_option_loaders/` package with registry (later refactored to per-plugin `_option_loaders.py`) + `dispatch_load_options()` + 3 WhatsApp loaders (groups/channels/members). `POST /api/schemas/nodes/options/{method}` + WS `load_options`. One-line registration for future Gmail/Calendar/Telegram loaders. || 5.a — Backend node-groups index | ✅ done | `6ef271d` | `GET /api/schemas/nodes/groups` returns `{group: [node_type, ...]}` derived from every NodeSpec's group array. 25 groups with 110 entries (tool=34, agent=19, android=16, trigger=10, …). Replaces 34 frontend `*_NODE_TYPES` arrays once consumers migrate. || 5.b wave 1 — SquareNode + MiddleSection group membership | ✅ done | `15df605` | 5 call sites migrated to `isNodeInBackendGroup()` helper with legacy array fallback chain. Also seeded 9 uiHints entries on NODE_METADATA (chatTrigger / console / teamMonitor / simpleMemory / 3 code executors / gmaps_create / start). || 5.b wave 2 — OutputPanel / ParameterRenderer / InputSection / ToolSchemaEditor | ✅ done | `0a6a259` | 4 more Android/agent call sites migrated. |@@ -243,7 +243,7 @@ a shared `resolveIcon` + `resolveLibraryIcon` pair.## Wave 12 — tech-debt cleanup (May 2026)-Closes the surviving pre-Wave-11 tribal patterns and the perf hotspots six parallel exploration passes turned up. Plan: [`.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`](../../.claude/plans/properly-fix-the-tech-dreamy-tarjan.md). Four batches, all four shipped.+Closes the surviving pre-Wave-11 tribal patterns and the perf hotspots six parallel exploration passes turned up. Plan: `.claude/plans/properly-fix-the-tech-dreamy-tarjan.md` (developer-local plan, not in repo). Four batches, all four shipped.| Batch | What it delivers | Where ||---|---|---|@@ -268,7 +268,7 @@ Closes the surviving pre-Wave-11 tribal patterns and the perf hotspots six paral## Wave 13 — Credentials: DB as single source of truth + symmetric broadcasts + cache dedup (May 2026)-Driven by three sub-agent audits (broadcast asymmetry, duplicated caches, parallel sources of truth) plus a research pass against modern (2024–2025) standards (OWASP, RFC 9700, CloudEvents 1.0). Plan: [`.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`](../../.claude/plans/properly-fix-the-tech-dreamy-tarjan.md). Commit `c94a610`. 17 files changed, +698 / -308.+Driven by three sub-agent audits (broadcast asymmetry, duplicated caches, parallel sources of truth) plus a research pass against modern (2024–2025) standards (OWASP, RFC 9700, CloudEvents 1.0). Plan: `.claude/plans/properly-fix-the-tech-dreamy-tarjan.md` (developer-local plan, not in repo). Commit `c94a610`. 17 files changed, +698 / -308.| Section | What it delivers | Where ||---|---|---|@@ -302,7 +302,7 @@ Driven by three sub-agent audits (broadcast asymmetry, duplicated caches, parall## Wave 14 — Theme system: 10 themes + decorative + sound + canvas overlays (May 2026)-Driven by the upstream `design_handoff_machinaos_themes/` bundle expanding from 2 → 10 designed themes (5 utopian + 5 dystopian) and shipping a formal MIGRATION_PLAYBOOK for the four areas the prior pass deferred. Plan: [`.claude/plans/deploy-multiple-parallel-subagents-wild-turtle.md`](../../.claude/plans/deploy-multiple-parallel-subagents-wild-turtle.md). Two commits: `52c5229` (foundation: 4 themes + chrome migration + StatusBar + CommandPalette + playbook doc) and the subsequent push (Waves W1–W7 below). Full architecture lives in [theme_system.md](./theme_system.md) — that's the source of truth for the contract.+Driven by the upstream `design_handoff_machinaos_themes/` bundle expanding from 2 → 10 designed themes (5 utopian + 5 dystopian) and shipping a formal MIGRATION_PLAYBOOK for the four areas the prior pass deferred. Plan: `.claude/plans/deploy-multiple-parallel-subagents-wild-turtle.md` (developer-local plan, not in repo). Two commits: `52c5229` (foundation: 4 themes + chrome migration + StatusBar + CommandPalette + playbook doc) and the subsequent push (Waves W1–W7 below). Full architecture lives in [theme_system.md](./theme_system.md) — that's the source of truth for the contract.| Wave | What it delivers | Where ||---|---|---|@@ -351,7 +351,7 @@ The MachinaOs frontend was coupled to Ant Design (40 files, 187-line theme file,## Codebase facts (from audit, 2026-04-13)- **antd usage:** 40 files. Top imports: `Space` (20), `Button` (16), `Flex` (13), `Tag` (11), `Spin` (10), `Alert` (10), `Typography` (9), `InputNumber` (7), `Collapse` (7), `Input` (6), `Card` (6), `Form` (5), `Statistic` (4), `Select` (4), `Switch` (3).-- **ConfigProvider:** only in [client/src/App.tsx](../client/src/App.tsx); theme in [client/src/config/antdTheme.ts](../client/src/config/antdTheme.ts) mirrors [client/src/styles/theme.ts](../client/src/styles/theme.ts).+- **ConfigProvider:** only in [client/src/App.tsx](../client/src/App.tsx); theme in `client/src/config/antdTheme.ts` (deleted in Phase 7) mirrors [client/src/styles/theme.ts](../client/src/styles/theme.ts).- **Already installed:** Tailwind 4.1.13, `@radix-ui/react-dialog`, `@radix-ui/react-collapsible`, `react-hook-form`, `[email protected]` (scoped to `components/credentials/`), `@uiw/react-json-view`, `class-variance-authority`, `clsx`, `tailwind-merge`, `@radix-ui/react-slot`, `sonner`.- **styled-components:** exactly 1 file — [client/src/components/shared/JSONTreeRenderer.tsx](../client/src/components/shared/JSONTreeRenderer.tsx).- **Hand-written code to be deleted in corrected Phase 0:** `client/src/design-system/primitives/*` (8 files) and `client/src/design-system/lib/toast.ts` (introduced by commits `2209dba` and `7ac69fe` — the previous mistake).@@ -506,7 +506,7 @@ Dependency order: 0 → {1, 2, 3} parallelizable → 4 → 5 → 7; 6 can overla- Verify zero `from 'antd'` imports remain.- Migrate [client/src/components/shared/JSONTreeRenderer.tsx](../client/src/components/shared/JSONTreeRenderer.tsx) — rewrite styled-components as Tailwind classes.-- Delete: [client/src/config/antdTheme.ts](../client/src/config/antdTheme.ts); `ConfigProvider` wrapper in [client/src/App.tsx](../client/src/App.tsx); antd reset CSS import in `main.tsx`. Inline remaining [client/src/styles/theme.ts](../client/src/styles/theme.ts) usages against CSS vars then delete.+- Delete: `client/src/config/antdTheme.ts` (deleted in Phase 7); `ConfigProvider` wrapper in [client/src/App.tsx](../client/src/App.tsx); antd reset CSS import in `main.tsx`. Inline remaining [client/src/styles/theme.ts](../client/src/styles/theme.ts) usages against CSS vars then delete.- Remove from [client/package.json](../client/package.json): `antd`, `@ant-design/icons`, `styled-components`, `@types/styled-components`.- Broaden React Compiler scope to whole `src/`.@@ -530,7 +530,7 @@ No test runner. Before Phase 5, decide:### Critical files- [client/src/App.tsx](../client/src/App.tsx) — `ConfigProvider` removal in Phase 7-- [client/src/config/antdTheme.ts](../client/src/config/antdTheme.ts) — theme migration source+- `client/src/config/antdTheme.ts` (deleted in Phase 7) — theme migration source- [client/src/styles/theme.ts](../client/src/styles/theme.ts) — token source for Phase 0- [client/tailwind.config.js](../client/tailwind.config.js) — Phase 0 rewiring- [client/tsconfig.json](../client/tsconfig.json) + [client/vite.config.js](../client/vite.config.js) — `@/*` alias for shadcndiff --git a/docs-internal/workflow-schema.md b/docs-internal/workflow-schema.mdindex 2c28e8b8..246c9495 100644--- a/docs-internal/workflow-schema.md+++ b/docs-internal/workflow-schema.md@@ -66,7 +66,9 @@ A workflow JSON document contains:}```-## Supported Node Types (96 Total)+## Supported Node Types++> Authoritative count = glob `server/nodes/**/__init__.py` (live total via `pytest --collect-only`). The breakdown below is illustrative and grouped by category; do not rely on a hand-maintained total.> The canonical list of nodes lives in the backend plugin tree at `server/nodes/<category>/<node>.py`; this section is a human-readable index.