Skip to content

Self-improving AI Employees for everyone

docs: restructure CLAUDE.md to lean index, dedupe canonical docs, audit node-flow cards

trohitg0b5ad7fc108f
diff --git a/CLAUDE.md b/CLAUDE.md
index cff0e490..7e452204 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -46,7 +46,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire
| **[Credentials Encryption](./docs-internal/credentials_encryption.md)** | Fernet + PBKDF2 encryption pipeline, separate credentials.db, two credential systems (OAuth vs API keys), multi-backend abstraction |
| **[Status Broadcaster](./docs-internal/status_broadcaster.md)** | WebSocket-first communication: StatusBroadcaster singleton, live count via `len(MESSAGE_HANDLERS) + len(get_ws_handlers())`, broadcast message types, Android two-state model |
| **[RLM Service](./docs-internal/rlm_service.md)** | Recursive Language Model agent with REPL-based execution (llm_query, rlm_query, FINAL) |
-| **[Claude Code Agent](./docs-internal/claude_code_agent_architecture.md)** | Claude Code integration internals (LangGraph reference; companion to `cli_agent_framework.md`) |
+| **[Claude Code Agent](./docs-internal/claude_code_agent.md)** | Hub for the Claude Code agent — routes to architecture, interactive mode, CLI framework, canonical-patterns RFC, and the 5 CLI snapshot references. |
| **[CLI Agent Framework](./docs-internal/cli_agent_framework.md)** | Multi-provider CLI runtime (Claude Code / Codex / Gemini): `AICliService.run_batch`, per-task worktree isolation, FastMCP bridge, **memory bridge** (`--continue` for first run + intra-process stream-json multi-turn + `--resume <UUID>` for crash recovery, all on a stable `cwd=repo_root`; `node_parameters_updated` broadcast on every successful turn). **Plugin-folder layout** (post-cutover): all claude-specific code lives in [`server/nodes/agent/claude_code_agent/`](./server/nodes/agent/claude_code_agent/) — `_provider.py`, `_pool.py`, `_skills.py`, `_oauth.py`, `_handlers.py` — and self-registers via three `factory.py` registries (`register_provider` / `register_session_pool` / `register_skill_materialiser`) plus `register_ws_handlers`. The generic framework at `services/cli_agent/` imports nothing from `nodes/`. |
| **[Claude Code Interactive Mode](./docs-internal/claude_code_interactive_mode.md)** | The interactive-mode cutover: MachinaOs no longer uses `claude -p` headless. `ClaudeSessionPool` (keyed by `simpleMemory.node_id`) spawns `claude` as a plain subprocess with stdio pipes — **no PTY** — and drives it over the VSCode-extension protocol: `--output-format stream-json --input-format stream-json --verbose --ide`. User prompts written as JSON to `proc.stdin` (`{"type":"user","message":{"role":"user","content":"..."}}\n`); `system/init` / `assistant` / `result` / `system/compact_boundary` events stream back on `proc.stdout` (parsed by a background `stdout_reader_task` — the on-disk JSONL is persistence-only, not the runtime contract, because `result` events are stdout-only in stream-json mode). Cross-platform via plain pipes (the earlier PTY pattern was broken on Windows because pywinpty/ConPTY's emulated stdin never reached claude's Ink TUI keystroke handler). Stays in interactive billing — entrypoint `claude-vscode`, NOT `sdk-cli`. Multi-turn within one warm subprocess preserves the session UUID (verified end-to-end); cross-batch continuity via `--continue` (first run) or `--resume <UUID>` (crash recovery). Four typed CloudEvents (`claude.session.{spawned,cleared,terminated,usage}`) fire from the pool. `/compact` events forward to `CompactionService`. The non-pooled `AICliSession` path (one-shot prompt-in-argv runs) still uses PTY (out of scope for this refactor; works on POSIX). |
| **[CLI Agent Canonical Patterns RFC](./docs-internal/cli_agent_canonical_patterns_rfc.md)** | Audit of MachinaOs's `services/cli_agent` against the official Claude Code spec — six invariants (skills-as-files, MCP-only transport, `list_changed`, deferral, visible-tool filtering, native-session-continuity-via-stable-cwd) with current implementation status. |
@@ -57,6 +57,12 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire
| **[Claude Code Skills (snapshot)](./docs-internal/claude_code_skills_reference.md)** | Verbatim [code.claude.com/docs/en/skills](https://code.claude.com/docs/en/skills) — `SKILL.md` frontmatter spec, discovery paths, content lifecycle, `context: fork`, dynamic context injection via `` !`<command>` ``. The spec MachinaOs materialises connected skills against in `_pre_spawn`. |
| **[Autonomous Agent Creation](./docs-internal/autonomous_agent_creation.md)** | Creating autonomous agents with Code Mode patterns and agentic loops |
| **[Polyglot Server](../polyglot-server/ARCHITECTURE.md)** | Plugin registry microservice with MCP gateway (optional integration) |
+| **[Authentication](./docs-internal/authentication.md)** | JWT/cookie auth: toggle, single/multi mode, backend + frontend, middleware, startup retry. |
+| **[Credentials Panel](./docs-internal/credentials_panel.md)** | CredentialsModal.tsx logic-flow reference (state taxonomy, per-provider panels, invariants). |
+| **[Memory Lifecycle](./docs-internal/memory_lifecycle.md)** | Canonical conversation-memory lifecycle: load/append/trim/archive/clear/resume across all agents. |
+| **[Node Parameter Panel](./docs-internal/node_panels.md)** | Three-section node config UI (Input / Parameters / Output) logic-flow reference. |
+| **[Deployment (legacy reference)](./docs-internal/deployment_legacy.md)** | machina deploy CLI summary + the historical Docker Compose topology. |
+| **[GCP VM Deploy Runbook](./docs-internal/gcp_vm_deploy_runbook.md)** | Manual gcloud + Cloudflare runbook for deploying the released npm package (non-Terraform path). |
## Design Principles & Standards
@@ -196,91 +202,7 @@ server/nodejs/ # Persistent Node.js server for JS/TS execution
```
### Polyglot Server Integration (Optional)
-MachinaOs can optionally integrate with **polyglot-server** - a centralized plugin registry microservice that exposes integrations through REST API, MCP (Model Context Protocol), and WebSocket.
-
-**Architecture:**
-```
-┌─────────────────────────────────────────────────────────────────┐
-│ MachinaOs │
-│ ┌────────────────┐ ┌────────────────┐ │
-│ │ React Flow │───▶│ FastAPI Backend│ │
-│ │ Frontend │ │ (port 3010) │ │
-│ └────────────────┘ └───────┬────────┘ │
-│ │ │
-│ ┌─────────────────┴─────────────────┐ │
-│ │ NodeExecutor │ │
-│ │ (registry-based dispatch) │ │
-│ └─────────────────┬─────────────────┘ │
-└────────────────────────────────│────────────────────────────────┘
- │ HTTP (aiohttp)
- ▼
-┌─────────────────────────────────────────────────────────────────┐
-│ Polyglot Server (port 8080) │
-│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
-│ │ REST API │ │ MCP │ │ WebSocket │ │
-│ │ Gateway │ │ Server │ │ Handler │ │
-│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
-│ └────────────────┼────────────────┘ │
-│ ▼ │
-│ ┌───────────────────────┐ │
-│ │ Plugin Registry │ │
-│ │ Discord, Telegram, │ │
-│ │ Notion, GitHub, etc. │ │
-│ └───────────────────────┘ │
-└─────────────────────────────────────────────────────────────────┘
-```
-
-**Files:**
-```
-server/services/
-├── polyglot_client.py # HTTP client for polyglot-server
-└── handlers/
- └── polyglot.py # Standalone handler (not wired to NodeExecutor)
-```
-
-**PolyglotClient** (`server/services/polyglot_client.py`):
-```python
-class PolyglotClient:
- """HTTP client for polyglot-server plugin registry."""
-
- async def list_plugins(self) -> List[Dict[str, Any]]:
- """List all available plugins."""
-
- async def get_schema(self, plugin_name: str) -> Optional[Dict[str, Any]]:
- """Get plugin input/output schema for workflow node integration."""
-
- async def execute(self, plugin_name: str, action: str, params: Dict) -> Dict:
- """Execute a plugin action."""
-```
-
-**Polyglot Handler** (`server/services/handlers/polyglot.py`):
-```python
-async def handle_polyglot_node(
- node_id: str,
- node_type: str,
- parameters: Dict[str, Any],
- context: Dict[str, Any],
- polyglot_client, # Injected via functools.partial
-) -> Dict[str, Any]:
- """Execute a workflow node via polyglot-server plugin registry."""
- plugin_name = node_type.replace("Node", "").lower()
- result = await polyglot_client.execute(plugin_name, action, params)
- return {"success": True, "result": result.get("result", {}), ...}
-
-# Node types that can be routed to polyglot-server
-POLYGLOT_NODE_TYPES = frozenset([
- "discordNode", "telegramNode", "slackNode", "notionNode",
- "todoistNode", "gmailNode", "twitterNode", "githubNode", ...
-])
-```
-
-**Configuration** (when enabled):
-```bash
-# In server/.env
-POLYGLOT_SERVER_URL=http://localhost:8080 # polyglot-server address
-```
-
-**Current Status**: Standalone integration files created. Not wired into NodeExecutor to avoid disturbing existing workflow execution flow. Future integration will add polyglot node types to handler registry via `functools.partial` pattern.
+MachinaOs can optionally integrate with the sibling **polyglot-server** repo (a plugin-registry microservice exposing REST + MCP + WebSocket). NOTE: the MachinaOs-side client/handler (`polyglot_client.py`, `handlers/polyglot.py`) are not currently present in the tree — this is a possible future integration, not wired. See [Polyglot Server](../polyglot-server/ARCHITECTURE.md).
### Node.js Code Executor
Persistent Node.js server for JavaScript/TypeScript code execution, replacing subprocess spawning per execution.
@@ -672,28 +594,7 @@ AI model nodes route through `SquareNode` via `Dashboard.tsx`'s `COMPONENT_BY_KI
### Theme System
-Token VALUES are **hex + `color-mix()`** (not HSL triplets) and live in the per-theme files: [light.css](./client/src/themes/light.css) / [dark.css](./client/src/themes/dark.css) own the shadcn + dracula + node + action colour hex; [base.css](./client/src/themes/base.css) owns the shared scale / chrome / shadow / type tokens **plus the `--tint-*` alpha scale — the single home for `color-mix` percentages** (consumers reference `var(--tint-X)`; skins keep only their bespoke decorative alphas inline). [index.css](./client/src/index.css) is **plumbing-only**: its `@theme inline` bridge maps `--color-X: var(--X)` (NO `hsl()` wrapper). Tailwind v4 still composes alpha (`bg-primary/50`, `text-foreground/80`) by wrapping any colour in `color-mix()`, so utilities work against hex. Action intents also expose an **`--action-X-ink`** readable-text variant (light: a darkened accent; dark: the raw accent) — `ActionButton` reads `text-action-X-ink`. The **animation system** — per-theme `--pulse-keyframe` / `--pulse-duration` / `--pulse-timing` tokens, the executing + trigger keyframes, and the `.machina-pulse` / `.machina-trigger-armed` / `.machina-bolt` / `.machina-crt` / `.machina-scanline` helpers — lives in [animations.css](./client/src/themes/animations.css); base.css's `.executing` rule reads `var(--pulse-keyframe)` so a theme switches its pulse by overriding one token. Trigger nodes show a distinct continuous `.machina-trigger-armed` "listening" heartbeat (waiting state) vs the one-shot execution pulse, with the ⚡ badge pulsing via `.machina-bolt`.
-
-**Token tiers** (pick the most specific that fits):
-
-1. **shadcn semantic tokens** — `background`, `foreground`, `card`, `popover`, `primary`, `secondary`, `muted`, `accent`, `destructive`, `success`, `warning`, `info`, `border`, `input`, `ring`. Each rotates per theme.
-2. **Node-type role tokens** — `--node-agent`, `--node-model`, `--node-skill`, `--node-tool`, `--node-trigger`, `--node-workflow`. Each exposes three variants:
- - `bg-node-X` / `text-node-X` — solid (icons, badges, accents)
- - `bg-node-X-soft` — tinted surface (cards, tinted backgrounds)
- - `border-node-X-border` — tinted outline
- Themes redefine these in their own scope without touching call sites. **Never use opacity arithmetic at the call site** (`bg-node-agent/10`); add a new `-soft` / `-border` variant if you need a different opacity.
-3. **Action role tokens** — `--action-run`, `--action-stop`, `--action-save`, `--action-config`, `--action-secret`, `--action-tools`. `base / -soft / -hover / -border / -ink` set (e.g. `bg-action-run-soft text-action-run-ink border-action-run-border`, plus `hover:bg-action-run-hover`). `-ink` is the readable label colour (light: darkened accent; dark: raw accent). Used for toolbar icon buttons, File menu items, and as the tokens behind `<ActionButton>`'s `intent` variants. Themes redefine these without touching call sites; ActionButton composes hover via `-hover` (no opacity arithmetic).
-4. **Dracula raw accents** — `--dracula-green/purple/pink/cyan/red/orange/yellow`. Same value across light + dark themes. Used as the underlying palette that `--action-X` and `--node-X` reference; do not consume directly in components — go through the semantic role token instead.
-
-**Action buttons** — use [ActionButton](./client/src/components/ui/action-button.tsx) (CVA primitive with semantic `intent` variants: `run | stop | save | config | secret | tools`). Each intent reads the matching `--action-X` set (`-soft` for resting bg, `-hover` for hover bg, `-border` for outline, `-ink` for the label text). Disabled state is the shadcn-idiomatic `disabled:opacity-50` on the base class (one rule, all intents) — never per-token opacity arithmetic. Replaces the old `actionButtonStyle()` helper. Credential-modal panels (`OAuthConnect`, `EmailPanel`, `QrPairingPanel`, `ActionBar`) consume `<ActionButton>` directly; their `ActionDef` carries an `intent` key, not a free-form colour.
-
-**Canvas-wide animations** — [client/src/styles/canvasAnimations.ts](./client/src/styles/canvasAnimations.ts) owns the `@keyframes` + `.react-flow__edge.{status}` + `.react-flow__node.{status}` rules injected once into Dashboard's `<style>` tag. Three named groups (`KEYFRAMES`, `edgeStatusStyles`, `nodeStatusStyles`) -- adding a new keyframe or status visual is a single-file change. Per-node inline animations (border pulse on `isExecuting`, etc.) live in their components and read theme tokens directly. Light/dark distinction lives entirely in `theme.ts` — `buildCanvasStyles(colors)` is single-arg and the file ships zero hardcoded hex colours; `CanvasStatusColors` carries `edgeDefault | edgeSelected | edgeExecuting | edgeCompleted | edgeError | edgePending | edgeMemoryActive | edgeToolActive`. The `nodeGlow` keyframe consumes scoped `--node-glow` / `--node-glow-soft` vars so one keyframe serves both themes.
-
-**shadcn primitives** — full set under `client/src/components/ui/` (Button, Badge, Alert, Dialog, DropdownMenu, Select, Popover, Tooltip, Tabs, Card, Input, Textarea, Switch, Checkbox, Slider, Label, Form, Sonner, Skeleton, Collapsible, Accordion, AlertDialog). New code composes these with Tailwind classes.
-
-**`useAppTheme()`** ([src/styles/theme.ts](./client/src/styles/theme.ts), [src/hooks/useAppTheme.ts](./client/src/hooks/useAppTheme.ts)) is grandfathered for the canvas node components (`AIAgentNode`, `SquareNode`, `TriggerNode`, `StartNode`, `ToolkitNode`, `TeamMonitorNode`, `GenericNode`) and `EdgeConditionEditor` — they interpolate per-definition `nodeColor` into gradients, borders, and React Flow `<Handle>` styles. Credential-modal panels (`OAuthConnect`, `EmailPanel`, `QrPairingPanel`) and the skill/tool editors (`SkillEditorModal`, `ToolSchemaEditor`) no longer call it — they compose `<ActionButton intent="...">` plus shadcn semantic tokens (`bg-warning/10`, `bg-accent/10`, `<Alert variant="destructive">`). Every other surface uses Tailwind + the tokens above.
-
-**Theme switching** — `[data-theme="dark"]` + Tailwind's `.dark` class, both set by `App.tsx`'s `useEffect` on `isDarkMode`. See [docs-internal/frontend_architecture.md](./docs-internal/frontend_architecture.md#tokens--theming) for the full token table and migration patterns.
+10-way visual theme system (5 utopian: light, dark, renaissance, greek, edo, steampunk, atomic + 5 dystopian: cyber, wasteland, rot, plague, surveillance) driven by `<html data-theme>` (set by [ThemeContext.tsx](./client/src/contexts/ThemeContext.tsx), which also toggles `.dark` for DARK_FAMILY themes) + per-theme CSS in `client/src/themes/`. Token VALUES are hex + `color-mix()` (never HSL): per-theme files own shadcn/dracula/node/action hex, `base.css` owns the shared `--tint-*` scale, `index.css` is plumbing (`@theme inline` maps `--color-X: var(--X)`). Six token tiers, the per-theme `--pulse-keyframe` animation system, decorative-layer wrappers, the 10-pack WebAudio sound system, and the canvas-wide edge/node status rules (`canvasAnimations.ts`) are all documented in **[Theme System](./docs-internal/theme_system.md)** — read it before adding a theme or a canvas-node component. The strict frontend theme RULES remain normative under "Frontend Design + Theme System (strict)" above.
### WebSocket-First Architecture
The project uses WebSocket as the primary communication method between frontend and backend, replacing most REST API calls:
@@ -707,884 +608,11 @@ The project uses WebSocket as the primary communication method between frontend
> **Authoritative source: backend plugin registry.** Glob [`server/nodes/**/*.py`](./server/nodes/) (excluding `_*.py` helpers and `__init__.py`) for the live count. The per-node descriptions below are reference material — they drift on every plugin add and should be cross-checked against [`server/nodes/README.md`](./server/nodes/README.md), the per-domain docs in `docs-internal/`, and the actual plugin classes before relying on any specific detail.
-### AI Chat Models (9 nodes)
-- **openaiChatModel**: OpenAI GPT models with response format options. O-series models (o1, o3, o4) support reasoning effort parameter.
-- **anthropicChatModel**: Claude models (Fable 5, Opus 4.8/4.7, Sonnet 4.6, Haiku 4.5) with extended thinking support
-- **geminiChatModel**: Google Gemini models with multimodal capabilities, safety settings, and thinking support for 2.5/Flash Thinking models
-- **openrouterChatModel**: OpenRouter unified API - access 200+ models from OpenAI, Anthropic, Google, Meta, Mistral, and more through a single API. Features free/paid model grouping in dropdown.
-- **groqChatModel**: Groq ultra-fast inference with Llama, Qwen3, and GPT-OSS models. Qwen3-32b supports reasoning_format.
-- **cerebrasChatModel**: Cerebras ultra-fast inference on custom AI hardware with Llama and Qwen models
-- **deepseekChatModel**: DeepSeek V4 models (deepseek-v4-flash, deepseek-v4-pro); `deepseek-chat` / `deepseek-reasoner` remain as legacy aliases (deprecate 2026-07-24). Up to 1M context (V4), up to 64K output.
-- **kimiChatModel**: Kimi models by Moonshot AI (kimi-k2.6, kimi-k2.5, kimi-k2.7-code). 256K context, 96K output. Thinking on by default (explicitly disabled for tool-calling agent compatibility). Fixed temperature 0.6.
-- **mistralChatModel**: Mistral AI models (mistral-large-latest, mistral-medium-latest, mistral-small-latest, codestral-latest). Up to 256K context. No thinking support. Temperature 0-1.5.
-
-### AI Agents & Memory (3 nodes)
-- **aiAgent**: Advanced AI agent with tool calling, memory input handle, and iterative reasoning. Uses the plain-async `_run_agent_loop` for structured execution. Parameters: Provider, Model, Prompt, System Message, Options.
-- **chatAgent**: Conversational AI agent with memory and skill support for multi-turn chat interactions. Parameters: Provider, Model, Prompt (supports `{{chatTrigger.message}}` template or auto-fallback from connected input), System Message. Behavior extended by connected skills.
-### AI Agent Tool Nodes (6 dedicated + 10 dual-purpose)
-Tool nodes connect to AI Agent's `input-tools` handle to provide capabilities the agent can invoke during reasoning. Both `masterSkill` and `simpleMemory` are in the AI Tools category.
-
-#### Dedicated Tool Nodes (passive, tool-only)
-- **masterSkill**: Master Skill (icon: target) - Aggregates multiple skills with enable/disable toggles. Split-panel UI: left panel shows skill list with checkboxes, right panel shows selected skill's markdown editor. Supports both built-in skills (from `server/skills/` folders) and user-created skills (stored in database). User skills can be created/edited/deleted inline via the "+" button.
-- **simpleMemory**: Markdown-based conversation memory with editable UI, window-based trimming, and optional vector DB for long-term semantic retrieval
-
-#### Skill Node Architecture
-Skills are organized in subfolders under `server/skills/`. Each top-level folder appears as an option in the Master Skill node's folder dropdown. See **[Skill Creation Guide](./server/skills/GUIDE.md)** for full documentation.
-
-```
-server/skills/
-├── GUIDE.md # Skill creation guide
-├── assistant/ # General-purpose assistant skills
-│ ├── agent-builder-skill/SKILL.md # 5 canvas-mutation ops for live tool growth
-│ ├── assistant-personality/SKILL.md
-│ ├── compaction-skill/SKILL.md
-│ ├── humanify-skill/SKILL.md
-│ ├── memory-skill/SKILL.md
-│ ├── subagent-skill/SKILL.md
-│ └── write-todos-skill/SKILL.md # Task planning with plan-work-update loop
-├── android_agent/ # Android device control skills
-│ ├── personality/SKILL.md
-│ ├── battery-skill/SKILL.md
-│ ├── wifi-skill/SKILL.md
-│ └── ... (12 skills total)
-├── autonomous/ # Autonomous agent patterns
-│ ├── code-mode-skill/SKILL.md
-│ ├── agentic-loop-skill/SKILL.md
-│ ├── progressive-discovery-skill/SKILL.md
-│ ├── error-recovery-skill/SKILL.md
-│ └── multi-tool-orchestration-skill/SKILL.md
-├── coding_agent/ # Code execution & filesystem skills
-│ ├── python-skill/SKILL.md
-│ ├── javascript-skill/SKILL.md
-│ ├── file-read-skill/SKILL.md
-│ ├── file-modify-skill/SKILL.md
-│ └── fs-search-skill/SKILL.md
-├── productivity_agent/ # Google Workspace skills
-│ ├── gmail-skill/SKILL.md # Send, search, read emails
-│ ├── calendar-skill/SKILL.md # Create, list, update, delete events
-│ ├── drive-skill/SKILL.md # Upload, download, list, share files
-│ ├── sheets-skill/SKILL.md # Read, write, append spreadsheet data
-│ ├── tasks-skill/SKILL.md # Create, list, complete tasks
-│ └── contacts-skill/SKILL.md # Create, list, search contacts
-├── social_agent/ # Social media platform skills
-│ ├── twitter-send-skill/SKILL.md
-│ ├── twitter-search-skill/SKILL.md
-│ ├── twitter-user-skill/SKILL.md
-│ ├── whatsapp-send-skill/SKILL.md
-│ └── whatsapp-db-skill/SKILL.md
-├── task_agent/ # Task management skills
-│ ├── timer-skill/SKILL.md
-│ ├── cron-scheduler-skill/SKILL.md
-│ └── task-manager-skill/SKILL.md
-├── travel_agent/ # Location and maps skills
-│ ├── geocoding-skill/SKILL.md
-│ └── nearby-places-skill/SKILL.md
-├── rlm_agent/ # Recursive Language Model agent skills
-│ └── rlm-reasoning-skill/SKILL.md
-├── terminal/ # Shell, process management, OS-specific terminals
-│ ├── shell-skill/SKILL.md # Sandboxed shell (no system PATH)
-│ ├── process-manager-skill/SKILL.md # Long-running processes (full PATH, streams output)
-│ ├── bash-skill/SKILL.md # Linux/macOS terminal patterns
-│ ├── powershell-skill/SKILL.md # Windows terminal patterns
-│ └── wsl-skill/SKILL.md # WSL bridge for Linux tools on Windows
-└── web_agent/ # Web automation skills
- ├── http-request-skill/SKILL.md
- ├── proxy-config-skill/SKILL.md
- ├── apify-skill/SKILL.md
- ├── crawlee-scraper-skill/SKILL.md
- ├── browser-skill/SKILL.md
- ├── duckduckgo-search-skill/SKILL.md
- ├── brave-search-skill/SKILL.md
- ├── serper-search-skill/SKILL.md
- └── perplexity-search-skill/SKILL.md
-```
-
-**SKILL.md Format:**
-```yaml
----
-name: skill-name
-description: Brief description for LLM visibility
-allowed-tools: tool1 tool2
-metadata:
- author: machina
- version: "1.0"
- category: general
- icon: "🔧"
- color: "#6366F1"
----
-
-# Skill Instructions (Markdown)
-Detailed instructions loaded when skill is activated.
-```
-
-**Skill Content Lifecycle:**
-1. First load reads from SKILL.md on disk, seeds to database
-2. Database is source of truth after first activation
-3. Users edit instructions in UI, edits saved to DB only
-4. "Reset to Default" reloads from original SKILL.md file
-
-#### Skill Content Editor
-Skill nodes display an Ant Design `Input.TextArea` (markdown content) to view and edit instructions:
-- Instructions loaded automatically when skill node is selected
-- Save button writes changes back to database
-- Uses `get_skill_content` / `save_skill_content` WebSocket handlers
-- **No Input/Output panels**: Skill nodes only show the middle section (parameters + editor) in the parameter panel
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `client/src/hooks/useParameterPanel.ts` | Loads/saves skill content for skill nodes |
-| `server/services/skill_loader.py` | SkillLoader for filesystem and database skills (DB-wired via `get_skill_loader()`) |
-| `server/services/skills/handlers.py` | `get_skill_content`, `save_skill_content`, `list_skill_folders`, `scan_skill_folder` handlers |
-
-#### Master Skill Editor
-The Master Skill node uses a custom split-panel editor (`MasterSkillEditor.tsx`) instead of standard parameters:
-
-**UI Layout:**
-```
-+----------------------------------------------------------+
-| [Folder Dropdown: assistant v] |
-+----------------------------------------------------------+
-| +--------------------+ +--------------------------------+ |
-| | SKILLS LIST | | SKILL INSTRUCTIONS | |
-| | [Badge: 3 enabled] | | | |
-| | [Search skills...] | | # WhatsApp Skill | |
-| | | | | |
-| | [x] WhatsApp | | This skill provides WhatsApp | |
-| | [x] Memory | | messaging capabilities... | |
-| | [ ] Android | | | |
-| | [x] Maps | | [Reset to Default] | |
-| | [ ] HTTP | | | |
-| +--------------------+ +--------------------------------+ |
-+----------------------------------------------------------+
-```
-
-**Folder Dropdown:**
-- Ant Design `Select` listing available skill folders from backend via `list_skill_folders` WebSocket handler
-- Shows loading/disabled state while folders are being fetched
-- Uses `getPopupContainer` to ensure dropdown renders correctly regardless of parent overflow
-- Default folder: `assistant`
-
-**Icon Resolution Priority:**
-Skill icons resolve through the same chain as the teaching node (mirrors `BaseNode._metadata_dict`): per-plugin `<plugin>/icon.svg` (or `<plugin>/icon_<type>.svg`) served as `/api/schemas/nodes/<type>/icon` → [`server/nodes/visuals.json`](./server/nodes/visuals.json) entry (emoji or `lobehub:<brand>`). Color falls through per-plugin `<plugin>/meta.json` then `visuals.json`. Resolution lives in [`server/services/skill_loader.py::_parse_skill_metadata`](./server/services/skill_loader.py) which calls `nodes._visuals.get_plugin_icon_path` / `get_plugin_meta` / `get_icon` / `get_color`. The pre-Wave-10 `skillNodes.ts` + `getNodeDefaults()` helper that did per-skill icon overrides on the frontend was retired; icons + colours now live exclusively on disk (per-plugin folder) with `visuals.json` as the fallback registry, and SKILL.md frontmatter is read only for orphan skills (no node target).
-
-**Keyboard Handling:**
-The editor uses a native DOM `addEventListener('keydown')` on a wrapper div (via `useRef`) to `stopPropagation()` for Ctrl/Meta key events. This prevents React Flow's document-level `useKeyPress` hook from intercepting Ctrl+A (select all) and other Ctrl shortcuts inside the textarea. React synthetic `stopPropagation()` is insufficient because React Flow uses native `document.addEventListener`.
-
-**Data Structure:**
-```typescript
-// skillsConfig parameter stored in node parameters
-interface MasterSkillConfig {
- [skillName: string]: {
- enabled: boolean; // Whether skill is active
- instructions: string; // Custom or default SKILL.md content
- isCustomized: boolean; // True if user modified instructions
- };
-}
-```
-
-**skillsConfig Persistence:**
-- `skillsConfig` persists skills from **all folders**, not just the currently selected one. Switching folders does not remove previously enabled skills from config.
-- The backend (`handlers/ai.py`) handles stale config entries gracefully at execution time -- it checks `enabled`, tries to load instructions, and logs a warning if a skill file is missing. Stale disabled entries are simply skipped.
-- No frontend cleanup of stale skills is performed to avoid race conditions with async skill loading.
-
-**Custom Skill Creation:**
-- The `create_user_skill` WebSocket handler requires `name`, `display_name`, and `instructions`. The `description` field is optional (defaults to empty string). Skill handlers live in `server/services/skills/handlers.py` (self-registered via `services.ws_handler_registry`).
-- On create, `MasterSkillEditor` persists the updated `skills_config` to the node via `save_node_parameters` (mirroring the delete path) so the new skill's enabled state survives panel close / reload, then `invalidateUserSkills()` + the `skill_lifecycle` broadcast refresh the list.
-- User skills are stored in the database and resolve through the database-wired `get_skill_loader().load_skill_async()` (so `get_skill_content` returns their instructions the same way it does for built-in skills).
-
-**Backend Expansion:**
-When AI Agent executes with a connected Master Skill node, `_collect_agent_connections()` in `handlers/ai.py` expands the skillsConfig into individual skill entries:
-```python
-if skill_type == 'masterSkill':
- skills_config = skill_params.get('skillsConfig', {})
- for skill_key, skill_cfg in skills_config.items():
- if skill_cfg.get('enabled', False):
- # Load from customized or skill folder
- instructions = skill_cfg.get('instructions') if skill_cfg.get('isCustomized') else skill_loader.load_skill(skill_key).instructions
- skill_data.append({
- 'node_id': f"{source_node_id}_{skill_key}",
- 'node_type': 'masterSkill',
- 'skill_name': skill_key,
- 'parameters': {'instructions': instructions},
- 'label': skill_key
- })
-```
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `client/src/components/parameterPanel/MasterSkillEditor.tsx` | Split-panel skill aggregator UI with inline user skill CRUD |
-| `server/services/skills/handlers.py` | User skill CRUD: `get_user_skills`, `create_user_skill`, `update_user_skill`, `delete_user_skill` |
-| `server/core/database.py` | UserSkill model and database CRUD methods |
-| `server/services/handlers/ai.py` | Expands masterSkill into individual skills at execution |
-| `server/skills/GUIDE.md` | Skill creation guide for built-in skills |
-
-#### Other Dedicated Tool Nodes
-- **calculatorTool**: Mathematical operations (add, subtract, multiply, divide, power, sqrt, mod, abs)
-- **currentTimeTool**: Get current date/time with timezone support
-- **duckduckgoSearch**: DuckDuckGo web search (free, uses `ddgs` library, no API key required)
-- **taskManager**: Task management tool for AI agents to create, track, and manage tasks
-- **writeTodos**: Structured task list planning for complex multi-step operations. Connects to any agent's `input-tools` handle. Dual output (`tool` + `main`), dracula purple `#bd93f9`. Backed by `TodoService` singleton (`server/services/todo_service.py`) with JSON-based per-session state keyed by workflow_id. Schema: `WriteTodosSchema` with `TodoItem`/`TodoStatus` Pydantic enum (`pending` | `in_progress` | `completed`). Handler broadcasts `phase: "todo_update"` via WebSocket for real-time UI; `formatTodoOutput()` in `OutputDisplayPanel.tsx` renders as a checklist. Skill: `server/skills/assistant/write-todos-skill/SKILL.md` teaches the plan-work-update loop. **Parameter panel = editable Current Todos manager** — `uiHints.isTodoEditor` routes `MiddleSection` to [`client/src/components/parameterPanel/TodoEditor.tsx`](./client/src/components/parameterPanel/TodoEditor.tsx) instead of the empty generic-params card. It reads/writes the LIVE TodoService list (keyed `workflow_id` → `node_id`, so all writeTodos nodes in one workflow share one list) via the `get_todos` / `set_todos` WS handlers, with add / remove / edit-text / click-to-cycle-status. Both the tool's `write` op and manual panel edits emit the typed `todos_updated` CloudEvent through the centralized `services.events.dispatch.emit`; the FE handler refreshes the `['todos', session_key]` query live. Self-contained plugin folder `server/nodes/tool/write_todos/` (`__init__.py` + `_handlers.py` + `_events.py`).
-
-#### Dual-Purpose Search Nodes (workflow node + AI tool)
-Search API nodes that work BOTH as standalone workflow nodes AND as AI Agent tools. When connected to `input-tools`, the LLM fills the node's parameter schema.
-- **braveSearch**: **Dual-purpose node** - Search the web using Brave Search API. Returns web results with titles, snippets, and URLs. Group: `['search', 'tool']`. Parameters: query, maxResults, country, searchLang, safeSearch.
-- **serperSearch**: **Dual-purpose node** - Search the web using Google via Serper API. Supports web, news, images, and places search types with knowledge graph. Group: `['search', 'tool']`. Parameters: query, searchType, maxResults, country, language.
-- **perplexitySearch**: **Dual-purpose node** - AI-powered search using Perplexity Sonar. Returns a markdown-formatted AI answer with inline citation references and source URLs. Group: `['search', 'tool']`. Parameters: query, model (sonar/sonar-pro/sonar-reasoning/sonar-reasoning-pro), searchRecencyFilter, returnImages, returnRelatedQuestions.
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/handlers/search.py` | 3 handler functions with API key fetch + usage tracking |
-| `server/services/handlers/tools.py` | Tool dispatch wrappers for AI Agent tool calling |
-| `server/services/ai.py` | Tool names, descriptions, and Pydantic schemas |
-| `server/constants.py` | `SEARCH_NODE_TYPES` and `SEARCH_TOOL_TYPES` constants |
-| `client/src/assets/icons/search/` | SVG icons for Brave, Serper, Perplexity |
-
-**Search API Authentication:**
-| Provider | Credential Key | Header | API Endpoint |
-|----------|---------------|--------|-------------|
-| Brave Search | `brave_search` | `X-Subscription-Token` | `GET https://api.search.brave.com/res/v1/web/search` |
-| Serper | `serper` | `X-API-KEY` | `POST https://google.serper.dev/search` |
-| Perplexity | `perplexity` | `Authorization: Bearer` | `POST https://api.perplexity.ai/chat/completions` |
-
-**Credentials Modal Layout:**
-- Brave Search and Perplexity in **Search** category
-- Serper in **Scrapers** category (Google SERP scraping)
-
-**Perplexity validation note:** `PerplexityCredential._probe` short-circuits to `ProbeResult(valid=True)` without an HTTP call — Perplexity exposes no cheap auth-gated endpoint (`/v1/models` returns 200 for any key including garbage; every other endpoint charges tokens, and `chat/completions` rejects `max_tokens<16` for `sonar` with HTTP 400 before checking auth). Runtime calls will surface 401 if the key is wrong. `ProbeResult` is exported from `services.plugin` for `_probe` overrides like this.
-
-### Specialized AI Agents (15 nodes)
-Specialized agents are AI Agents pre-configured for specific domains. They inherit full AI Agent functionality (provider, model, prompt, system message, thinking/reasoning) while being tailored for specific capabilities. All specialized agents dispatch through `BaseNode.execute()` via the node registry (Wave 11) and support the same input handles. Node colors use centralized dracula theme constants imported from `client/src/styles/theme.ts`.
-
-**Input Handles:**
-- `input-main` - Main data input (auto-prompting fallback)
-- `input-skill` - Skill nodes (including Master Skill for aggregated skills)
-- `input-memory` - Memory node for conversation history
-- `input-tools` - Tool nodes for LLM tool calling
-- `input-task` - Task completion events from taskTrigger nodes
-
-**Specialized Agent Types:**
-- **android_agent**: Android Control Agent - AI agent for Android device control. Connect Android service nodes (battery, wifi, bluetooth, apps, location, camera, sensors) as tools.
-- **coding_agent**: Coding Agent - AI agent for code execution. Connect code executor nodes (Python, JavaScript) as tools.
-- **web_agent**: Web Control Agent - AI agent for web automation. Connect web/browser nodes (scraper, HTTP, browser) as tools.
-- **task_agent**: Task Management Agent - AI agent for task automation. Connect scheduling nodes (scheduler, reminders) as tools.
-- **social_agent**: Social Media Agent - AI agent for social messaging. Connect messaging nodes (WhatsApp, Telegram) as tools.
-- **travel_agent**: Travel Agent - AI agent for travel planning. Connect location, maps, and scheduling nodes as tools.
-- **tool_agent**: Tool Agent - AI agent for tool orchestration. Connect any combination of tool nodes for flexible automation.
-- **productivity_agent**: Productivity Agent - AI agent for productivity workflows. Connect scheduling, task, and utility nodes as tools.
-- **payments_agent**: Payments Agent - AI agent for payment processing. Connect payment, invoice, and financial tool nodes.
-- **consumer_agent**: Consumer Agent - AI agent for consumer interactions. Connect customer support, product, and order management tools.
-- **autonomous_agent**: Autonomous Agent - AI agent for autonomous operations using Code Mode patterns. Uses agentic loops, progressive discovery, error recovery, and multi-tool orchestration for 81-98% token savings. Connect autonomous skills via Master Skill.
-- **orchestrator_agent**: Orchestrator Agent - Team lead agent for coordinating multiple agents. Connect specialized agents via `input-teammates` handle; they become `delegate_to_*` tools the AI can invoke.
-- **ai_employee**: AI Employee - Team lead agent similar to orchestrator_agent. Connect specialized agents via `input-teammates` handle for intelligent task delegation.
-- **rlm_agent**: RLM Agent - Recursive Language Model agent using REPL-based code execution with recursive LM calls. Replaces the standard tool-calling loop with RLM's `exec()` REPL loop (`llm_query()`, `rlm_query()`, `FINAL()`). Routes to dedicated `handle_rlm_agent` handler and `RLMService` (not `handle_chat_agent`). Connect AI chat model nodes as small LMs for depth>=1 calls. See `docs-internal/rlm_service.md`.
-
-**Backend Routing:**
-Specialized agents are detected by `SPECIALIZED_AGENT_TYPES` and dispatched through `BaseNode.execute()` via the node registry (Wave 11). Dedicated engine: `rlm_agent` -> `RLMService`.
-```python
-SPECIALIZED_AGENT_TYPES = {
- 'android_agent', 'coding_agent', 'web_agent', 'task_agent', 'social_agent',
- 'travel_agent', 'tool_agent', 'productivity_agent', 'payments_agent', 'consumer_agent',
- 'autonomous_agent', 'orchestrator_agent', 'ai_employee',
-}
-```
-
-**Team Lead Types (Agent Teams Pattern):**
-Team leads (`orchestrator_agent`, `ai_employee`) have a special `input-teammates` handle. Agents connected to this handle become delegation tools:
-```python
-# In handlers/ai.py
-TEAM_LEAD_TYPES = {'orchestrator_agent', 'ai_employee'}
-
-# Teammates become delegate_to_* tools automatically
-if node_type in TEAM_LEAD_TYPES:
- teammates = await _collect_teammate_connections(node_id, context, database)
- if teammates:
- for tm in teammates:
- tool_data.append({
- 'node_id': tm['node_id'],
- 'node_type': tm['node_type'], # e.g., 'coding_agent'
- 'label': tm['label'],
- })
- # AI now has delegate_to_coding_agent, delegate_to_web_agent, etc.
-```
-
-**Direct Android Service Tools:**
-Android service nodes (batteryMonitor, wifiAutomation, etc.) can be connected directly to any agent's `input-tools` handle. The backend maps camelCase node types to snake_case service IDs:
-```python
-# In handlers/tools.py
-service_id_map = {
- 'batteryMonitor': 'battery',
- 'wifiAutomation': 'wifi_automation',
- 'bluetoothAutomation': 'bluetooth_automation',
- # ... etc
-}
-```
-
-#### Dual-Purpose Tool Nodes (workflow node + AI tool)
-Nodes that work BOTH as standalone workflow nodes AND as AI Agent tools. When connected to `input-tools`, the LLM fills the node's parameter schema.
-- **whatsappSend**: Send WhatsApp messages (text, media, location, contact) to contacts, groups, or newsletter channels. Full schema for all message types.
-- **whatsappDb**: Query WhatsApp database - chat history, contacts, groups, and newsletter channels with filtering and pagination.
-- **pythonExecutor**: Execute Python code for calculations, data processing, and automation. Tool name: `python_code`. Available: math, json, datetime, Counter, defaultdict, random.
-- **montyExecutor**: Sandboxed Python via Pydantic Monty (`pydantic-monty`). Tool name: `sandboxed_python`. Deny-by-default with ENFORCED time/memory limits + dynamic `capabilities` (`http_get`/`workspace_read`/`workspace_write`). Supported subset only (no class/yield/with/match/random/collections).
-- **javascriptExecutor**: Execute JavaScript code via persistent Node.js server for calculations, data processing, and JSON manipulation. Tool name: `javascript_code`.
-- **typescriptExecutor**: Execute TypeScript code via persistent Node.js server with type safety. Tool name: `typescript_code`.
-- **gmaps_locations**: Google Maps Geocoding service for address-to-coordinates conversion. Tool name: `geocode`.
-- **gmaps_nearby_places**: Google Places API nearbySearch. Tool name: `nearby_places`.
-
-See [plugin_system.md](./docs-internal/plugin_system.md) for the dual-purpose pattern (mark `group: ['category', 'tool']` and set `usable_as_tool = True` on the plugin class); whatsapp / twitter / google plugin folders are the live examples.
-
-### Location Services (3 nodes)
-- **gmaps_create**: Google Maps creation with customizable center, zoom, and map type (display only, not a tool)
-- **gmaps_locations**: **Dual-purpose** - Google Maps Geocoding service for address-to-coordinates conversion
-- **gmaps_nearby_places**: **Dual-purpose** - Google Places API nearbySearch with detailed place information
-
-#### Google Maps API Key Resolution
-The MapsSection component fetches the Google Maps API key from backend credentials:
-```typescript
-// In MapsSection.tsx
-const { getStoredApiKey, isConnected } = useApiKeys();
-const [apiKey, setApiKey] = useState<string | undefined>(() => getGoogleMapsApiKey()); // Env fallback
-const hasFetchedRef = useRef(false);
-
-useEffect(() => {
- if (!isConnected || hasFetchedRef.current) return;
- const fetchApiKey = async () => {
- hasFetchedRef.current = true;
- const storedKey = await getStoredApiKey('google_maps');
- if (storedKey) setApiKey(storedKey);
- };
- fetchApiKey();
-}, [isConnected, getStoredApiKey]);
-```
-- Falls back to environment variable if no stored key
-- Uses `hasFetchedRef` to prevent multiple fetches
-- Only fetches when WebSocket is connected
-
-### Android Services (16 nodes)
-Android device connection is configured via the Credentials Modal (Android panel), not via workflow nodes.
-
-#### System Monitoring (4 nodes)
-- **batteryMonitor**: Monitor battery status, level, charging state, temperature, and health
-- **networkMonitor**: Monitor network connectivity, type, and internet availability
-- **systemInfo**: Get device and OS information including Android version, API level, memory, and hardware details
-- **location**: GPS location tracking with latitude, longitude, accuracy, and provider information
-
-#### App Management (2 nodes)
-- **appLauncher**: Launch applications by package name
-- **appList**: Get list of installed applications with package names, versions, and metadata
-
-#### Device Automation (6 nodes)
-- **wifiAutomation**: WiFi control and scanning - enable, disable, get status, scan for networks
-- **bluetoothAutomation**: Bluetooth control - enable, disable, get status, and paired devices
-- **audioAutomation**: Volume and audio control - get/set volume, mute, unmute
-- **deviceStateAutomation**: Device state control - airplane mode, screen on/off, power save mode, brightness
-- **screenControlAutomation**: Screen control - brightness adjustment, wake screen, auto-brightness, screen timeout
-- **airplaneModeControl**: Airplane mode status monitoring and control
-
-#### Sensors (2 nodes)
-- **motionDetection**: Accelerometer and gyroscope data - detect motion, shake gestures, device orientation
-- **environmentalSensors**: Environmental sensors - temperature, humidity, pressure, light level
-
-#### Media (2 nodes)
-- **cameraControl**: Camera control - get camera info, take photos, camera capabilities
-- **mediaControl**: Media playback control - volume control, playback control, play media files
-
-### WhatsApp Nodes (3 nodes)
-- **whatsappSend**: **Dual-purpose node** - Send WhatsApp messages (text, image, video, audio, document, sticker, location, contact) to contacts, groups, or newsletter channels. Works as workflow node OR AI Agent tool. Group: `['whatsapp', 'tool']`. Recipient types: Self (connected phone), Phone Number, Group, Channel (newsletter). Channel messages only support text, image, video, audio, document (NOT sticker, location, contact). **Format Markdown** toggle (default: true): converts GFM markdown to WhatsApp-native formatting via `markdown_formatter.to_whatsapp()`. Full parameter schema for message type, media URL, location coordinates, contact vCard, channel JID.
-- **whatsappDb**: **Dual-purpose node** - Comprehensive WhatsApp database query node with 18 operations. Works as workflow node OR AI Agent tool. Group: `['whatsapp', 'tool']`. Operations:
- - `chat_history`: Retrieve messages from individual or group chats with filtering, pagination, and optional media download
- - `search_groups`: Search groups by name
- - `get_group_info`: Get group details with participant names and phone numbers
- - `get_contact_info`: Get full contact info (name, phone, profile picture) for sending/replying
- - `list_contacts`: List all contacts with saved names
- - `check_contacts`: Check WhatsApp registration status for phone numbers
- - `list_channels`: List subscribed newsletter channels with optional server refresh
- - `get_channel_info`: Get channel details (name, description, subscribers)
- - `channel_messages`: Get channel message history with pagination, date range, media type filter, text search, and optional media download
- - `channel_stats`: Get channel subscriber/view statistics
- - `channel_follow`: Follow/subscribe to a newsletter channel
- - `channel_unfollow`: Unfollow/unsubscribe from a newsletter channel
- - `channel_create`: Create a new newsletter channel (with optional picture)
- - `channel_mute`: Mute/unmute a newsletter channel
- - `channel_mark_viewed`: Mark channel messages as viewed
- - `newsletter_react`: React to a channel message with an emoji
- - `newsletter_live_updates`: Subscribe to live view/reaction counts for channel messages
- - `contact_profile_pic`: Get profile picture for a contact or group
-- **whatsappReceive**: Event-driven trigger that waits for incoming WhatsApp messages with filters (message type, sender, group, channel, keywords, forwarded status). Marked with `['whatsapp', 'trigger']` group for n8n-style trigger identification. Stores group/sender names alongside JID/phone for display persistence. The Go RPC resolves LIDs to phone numbers before sending events - `sender_phone` field is already resolved. Filter options: All Messages, From Self (notes to self chat only), From Any Contact (Non-Group), From Specific Contact, From Specific Group, From Channel (Newsletter), Contains Keywords
-
-### Social Nodes (2 nodes)
-Unified social messaging nodes for multi-platform communication. Supports WhatsApp, Telegram, Discord, Slack, Signal, SMS, Webchat, Email, Matrix, Teams.
-
-- **socialReceive**: Normalizes messages from platform triggers into unified format. Multiple outputs: Message, Media, Contact, Metadata. Filters by channel, message type, sender.
-- **socialSend**: **Dual-purpose node** - Send messages to any supported platform. Works as workflow node OR AI Agent tool. Supports text, image, video, audio, document, sticker, location, contact, poll, buttons, list message types.
-
-### Twitter/X Nodes (4 nodes)
-Twitter/X integration using the official XDK Python SDK with OAuth 2.0 PKCE authentication. All sync XDK calls wrapped in `asyncio.to_thread()` to avoid blocking the event loop. Lazy token refresh on 401/403 errors instead of validating on every call.
-
-- **twitterSend**: **Dual-purpose node** - Post tweets, reply, retweet, like/unlike, and delete tweets. Works as workflow node OR AI Agent tool. Group: `['social', 'tool']`. Actions: `tweet`, `reply`, `retweet`, `like`, `unlike`, `delete`. Parameters: action, text (280 char max), tweet_id, reply_to_id.
-- **twitterSearch**: **Dual-purpose node** - Search recent tweets with rich data via X API v2 expansions. Returns enriched tweets with `display_text` (expanded URLs), `author` profile, `public_metrics` (likes/retweets/replies/quotes/bookmarks/impressions), `media` attachments, `referenced_tweets` (quoted/replied), `note_tweet` for long-form content. Works as workflow node OR AI Agent tool. Group: `['social', 'tool']`. `max_results` clamped to 10-100 (X API v2 minimum is 10). Supports query operators: keywords, hashtags (#), mentions (@), from:user, to:user, -exclude, OR, lang:, has:links, has:media, has:images, has:videos, is:retweet, -is:retweet, is:reply, is:quote, url:.
-- **twitterUser**: **Dual-purpose node** - Look up user profiles and social connections with description and created_at. Works as workflow node OR AI Agent tool. Group: `['social', 'tool']`. Operations: `me` (get authenticated user), `by_username`, `by_id`, `followers` (max_results 1-1000), `following` (max_results 1-1000).
-- **twitterReceive**: Event-driven trigger that waits for incoming Twitter events (mentions, DMs, timeline updates). Group: `['social', 'trigger']`. Polling-based since X API free tier lacks webhooks.
-
-#### Twitter OAuth 2.0 Authentication
-Authentication is handled via OAuth 2.0 PKCE flow in the Credentials Modal:
-1. User clicks "Login with Twitter" button
-2. Backend generates PKCE code challenge and authorization URL
-3. Browser opens Twitter authorization page
-4. User grants permission, Twitter redirects with auth code
-5. Backend exchanges code for access_token + refresh_token
-6. Tokens stored in database via auth_service
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/twitter_oauth.py` | OAuth 2.0 PKCE flow implementation |
-| `server/services/oauth_utils.py` | Runtime OAuth redirect URI derivation from request context |
-| `server/routers/twitter.py` | OAuth callback endpoint, token exchange |
-| `server/services/handlers/twitter.py` | Node handlers using XDK SDK |
-| `client/src/components/CredentialsModal.tsx` | Twitter panel with OAuth button |
-| `server/skills/social_agent/twitter-*-skill/` | 3 Twitter skills for AI agents |
-
-**Handler Architecture:**
-- All sync XDK calls wrapped in `asyncio.to_thread()` (XDK uses sync `requests` internally)
-- No `get_me()` validation on every call -- lazy token refresh on 401/403 via `_refresh_and_get_client()`
-- Search uses full X API v2 expansions and returns enriched data with expanded URLs, author profiles, media, metrics, referenced tweets
-- `max_results` clamped: search 10-100, followers/following 1-1000
-
-**XDK SDK API Patterns:**
-```python
-from xdk import Client
-client = Client(access_token=access_token)
-
-client.posts.create(body={"text": "Hello world!"})
-client.posts.create(body={"text": "Reply", "reply": {"in_reply_to_tweet_id": "123"}})
-client.users.repost_post(user_id, body={"tweet_id": "123"})
-client.users.like_post(user_id, body={"tweet_id": "123"})
-client.users.unlike_post(user_id, tweet_id="123")
-client.posts.delete(tweet_id)
-client.posts.search_recent(query="...", max_results=100,
- tweet_fields=[...], expansions=[...], media_fields=[...], user_fields=[...])
-client.users.get_me(user_fields=["created_at", "description"])
-client.users.get_by_usernames(usernames=["user1"], user_fields=["description"])
-client.users.get_followers(user_id, max_results=100, user_fields=["created_at"])
-```
-
-**Environment Variables:**
-```bash
-TWITTER_CLIENT_ID=your_client_id
-TWITTER_CLIENT_SECRET=your_client_secret
-# TWITTER_REDIRECT_URI is derived at runtime from request context (no env var needed)
-```
-
-### Telegram Nodes (2 nodes) — reference for self-contained plugin folders
-Telegram bot integration using `python-telegram-bot` SDK with long-polling for incoming messages. **All telegram code lives in `server/nodes/telegram/`** — the plugin folder owns its service, WebSocket handlers, event filter, lifecycle hooks, and credentials. Read this folder first before adding any plugin that needs more than a single `BaseNode` subclass; see [docs-internal/plugin_system.md → "Self-contained plugin folders"](./docs-internal/plugin_system.md#self-contained-plugin-folders).
-
-- **telegramSend**: Workflow-only ActionNode (dropped dual-tool support in `190dbb9` — `usable_as_tool=True` removed, group reduced from `('social', 'tool')` to `('social',)`). Send text, photo, document, location, or contact messages via Telegram bot. Recipient types: Self (bot owner), User/Chat ID, Group. Parameters: recipient_type, chat_id, message_type, text, media_url, caption, parse_mode (Auto/HTML/Markdown/MarkdownV2/None), silent, reply_to_message_id. **Auto parse_mode** (default, recommended): converts GFM markdown to Telegram HTML via `markdown_formatter.to_telegram_html()`. MarkdownV2/Markdown text auto-escaped via `_escape_text()`; falls back to plain text on `BadRequest`. "Self" restores `owner_chat_id` from credentials DB if not in memory.
-- **telegramReceive**: Event-driven trigger that waits for incoming Telegram messages. Group: `['social', 'trigger']`. Uses `senderFilter` dropdown: All Messages, From Self (Bot Owner) (zero-config), Private/Group/Supergroup/Channel Only, Specific Chat, Specific User, Keywords. Content type filter and ignore bots option. "From Self" uses lazy `_get_owner_chat_id()` lookup at match time. Auto-reconnects on server restart via the registered service-refresh callback.
-
-**Key Files** (all under `server/nodes/telegram/`):
-| File | Description |
-|------|-------------|
-| `__init__.py` | Imports + 6 self-registration calls — the only wiring needed (zero logic). |
-| `_credentials.py` | `TelegramCredential(ApiKeyCredential)` — bot token + `telegram_owner_chat_id` extra field. |
-| `_service.py` | `TelegramService` singleton: connect/disconnect/send/poll lifecycle, idempotent connect, parse-mode fallback helper. |
-| `_handlers.py` | 7 WebSocket handlers (`telegram_connect` / `_disconnect` / `_status` / `_send` / `_reconnect` / `_get_me` / `_get_chat`) + `WS_HANDLERS` dict registered into `services.ws_handler_registry`. |
-| `_filters.py` | `build_telegram_filter` — registered into `event_waiter.FILTER_BUILDERS`. |
-| `_refresh.py` | `refresh_telegram_status` (registered into `status_broadcaster._SERVICE_REFRESH_CALLBACKS`) + `precheck_telegram_trigger` (registered into `event_waiter._TRIGGER_PRECHECKS`). |
-| `telegram_send.py` | `TelegramSendNode(ActionNode)` + `TelegramSendOutput` — the workflow + AI-tool plugin. |
-| `telegram_receive.py` | `TelegramReceiveNode(TriggerNode)` + `TelegramReceiveOutput` — the trigger plugin (declares `event_type = "telegram_message_received"`). |
-| `client/src/components/CredentialsModal.tsx` | Telegram bot token panel (frontend; identifies commands by WebSocket message-type strings, not Python paths). |
-
-**Authentication:** Bot token from @BotFather on Telegram. Stored via `auth_service.store_api_key('telegram', token, models=[])` (renamed from `telegram_bot_token` — class id, catalogue key, and co-located SVG basename now all align with the brand name). **Owner detection has three independent layers** (commit f746ddf): (1) explicit `Your Chat ID (optional)` text input in the Telegram credentials modal — declared as a secondary `FieldDef` in `credential_providers.json` and rendered by `ApiKeyPanel`'s `SecondaryFieldRow`; saves directly to `telegram_owner_chat_id` via `auth_service.store_api_key`; (2) pre-poll peek in `connect()` via `_capture_owner_from_pending_updates()` — `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue; (3) atomic write-through in `_on_message_received` — persists FIRST, sets in-memory only on success, ERROR-level logging on failure. Lazy fallback in `telegram_send.py` reads DB and calls `service.set_owner` if in-memory is empty. Credentials panel: Save (store only), Connect (store + connect), Reconnect. All credential access uses `from core.container import container; container.auth_service()` (NOT `from services.auth import get_auth_service`).
-
-**Environment Variables:**
-```bash
-# Optional: Set owner chat ID for "Self" recipient type
-TELEGRAM_OWNER_CHAT_ID=your_chat_id
-```
-
-### Markdown Formatter Service
-Platform-specific markdown formatting using `markdown-it-py` (Python port of `markdown-it`, 18K+ GitHub stars). Converts GFM markdown (as produced by LLMs) to platform-native formats.
-
-**Key File:** `server/services/markdown_formatter.py`
-
-**Public Functions:**
-| Function | Target | Description |
-|----------|--------|-------------|
-| `to_telegram_html(text)` | Telegram | Renders GFM to HTML, converts unsupported tags (`<h1>`-`<h6>` -> `<b>`, `<ul>/<li>` -> bullets, strips `<p>`) |
-| `to_whatsapp(text)` | WhatsApp | Walks markdown-it token stream, maps to WhatsApp syntax (`*bold*`, `_italic_`, `~strike~`, `` ```code``` ``, `> quote`) |
-| `to_plain(text)` | Any | Renders to HTML, strips all tags |
-
-**Usage:**
-- Telegram Send node: Auto parse_mode (default) calls `to_telegram_html()` via `TelegramService._format_auto()`
-- WhatsApp Send node: `format_markdown` toggle (default: true) calls `to_whatsapp()` in handler
-- No new dependencies -- `markdown-it-py` v4.0.0 already installed as transitive dependency
-
-### Google Workspace Nodes (7 nodes)
-Consolidated Google Workspace integration with 6 unified operation-based nodes + 1 polling trigger. Each service node uses an `operation` parameter to select the action (e.g., gmail with operation: send/search/read). All services share a single OAuth connection with combined scopes.
-
-#### Consolidated Service Nodes (6 nodes)
-- **gmail**: **Dual-purpose node** - Operations: `send`, `search`, `read`. Handler: `handle_google_gmail()` dispatcher.
-- **calendar**: **Dual-purpose node** - Operations: `create`, `list`, `update`, `delete`.
-- **drive**: **Dual-purpose node** - Operations: `upload`, `download`, `list`, `share`.
-- **sheets**: **Dual-purpose node** - Operations: `read`, `write`, `append`.
-- **tasks**: **Dual-purpose node** - Operations: `create`, `list`, `complete`, `update`, `delete`.
-- **contacts**: **Dual-purpose node** - Operations: `create`, `list`, `search`, `get`, `update`, `delete`.
-
-#### Trigger Nodes (1 node)
-- **gmailReceive**: Polling-based trigger for incoming emails. Polls Gmail API at configurable interval (10-3600s). Parameters: `filter_query` (default: `is:unread`), `label_filter` (default: `INBOX`), `mark_as_read`, `poll_interval` (default: 60s). In deployment mode, uses `setup_polling_trigger` with baseline detection to avoid triggering on existing emails.
-
-#### Google Workspace OAuth 2.0 Authentication
-Authentication is handled via OAuth 2.0 flow in the Credentials Modal:
-1. User enters Google Cloud Client ID and Secret (OAuth 2.0 credentials)
-2. User clicks "Login with Google"
-3. Consent screen shows all requested scopes (Gmail, Calendar, Drive, etc.)
-4. User grants permission, Google redirects with auth code
-5. Backend exchanges code for access_token + refresh_token
-6. Tokens stored via auth_service (owner mode) or google_connections table (customer mode)
-
-**Combined OAuth Scopes:**
-```python
-GOOGLE_WORKSPACE_SCOPES = [
- # User Info
- "openid",
- "https://www.googleapis.com/auth/userinfo.email",
- "https://www.googleapis.com/auth/userinfo.profile",
- # Gmail
- "https://www.googleapis.com/auth/gmail.send",
- "https://www.googleapis.com/auth/gmail.readonly",
- "https://www.googleapis.com/auth/gmail.modify",
- # Calendar
- "https://www.googleapis.com/auth/calendar",
- "https://www.googleapis.com/auth/calendar.events",
- # Drive
- "https://www.googleapis.com/auth/drive",
- "https://www.googleapis.com/auth/drive.file",
- # Sheets
- "https://www.googleapis.com/auth/spreadsheets",
- # Tasks
- "https://www.googleapis.com/auth/tasks",
- # Contacts
- "https://www.googleapis.com/auth/contacts",
- "https://www.googleapis.com/auth/contacts.readonly",
-]
-```
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/google_oauth.py` | OAuth 2.0 flow (authorization URL, token exchange, `get_callback_paths()`) |
-| `server/services/oauth_utils.py` | Runtime OAuth redirect URI derivation from request context |
-| `server/services/handlers/google_auth.py` | Shared credential helper (`get_google_credentials()`) used by all 6 handlers |
-| `server/config/google_apis.json` | API endpoints, scopes, and OAuth callback paths |
-| `server/routers/google.py` | OAuth callback and status endpoints |
-| `server/services/handlers/gmail.py` | Gmail handlers + `handle_gmail_receive` polling trigger |
-| `server/services/handlers/calendar.py` | Calendar handlers |
-| `server/services/handlers/drive.py` | Drive handlers |
-| `server/services/handlers/sheets.py` | Sheets handlers |
-| `server/services/handlers/tasks.py` | Tasks handlers |
-| `server/services/handlers/contacts.py` | Contacts handlers |
-| `server/models/database.py` | `GoogleConnection` model |
-| `server/skills/productivity_agent/` | Google Workspace skills for AI agents |
-| `client/src/assets/icons/google/` | Official Google service SVG icons (n8n pattern) |
-| `client/src/components/CredentialsModal.tsx` | Google Workspace panel |
-
-**Token Storage (Two Separate Systems):**
-
-Client ID and Secret are stored as API keys (user enters manually):
-| Key | Storage Method | Description |
-|-----|---------------|-------------|
-| `google_client_id` | `auth_service.get_api_key()` / `EncryptedAPIKey` table | OAuth 2.0 Client ID |
-| `google_client_secret` | `auth_service.get_api_key()` / `EncryptedAPIKey` table | OAuth 2.0 Client Secret |
-
-Access and refresh tokens are stored via OAuth system (after Google login):
-| Field | Storage Method | Description |
-|-------|---------------|-------------|
-| `access_token` | `auth_service.get_oauth_tokens("google")` / `EncryptedOAuthToken` table | Access token for API calls |
-| `refresh_token` | `auth_service.get_oauth_tokens("google")` / `EncryptedOAuthToken` table | Refresh token for renewal |
-| `email`, `name` | `auth_service.get_oauth_tokens("google")` / `EncryptedOAuthToken` table | Connected user info |
-
-**IMPORTANT**: All handlers use `get_google_credentials()` from `google_auth.py`, which reads tokens via `get_oauth_tokens()`. Never use `get_api_key("google_access_token")` — that reads from the wrong table.
-
-**AI Agent Skills (productivity_agent folder):**
-| Skill | Tools | Description |
-|-------|-------|-------------|
-| `gmail-skill` | `gmail_send`, `gmail_search`, `gmail_read` | Send, search, read emails |
-| `calendar-skill` | `calendar_create`, `calendar_list`, `calendar_update`, `calendar_delete` | Manage calendar events |
-| `drive-skill` | `drive_upload`, `drive_download`, `drive_list`, `drive_share` | Manage Drive files |
-| `sheets-skill` | `sheets_read`, `sheets_write`, `sheets_append` | Read/write spreadsheet data |
-| `tasks-skill` | `tasks_create`, `tasks_list`, `tasks_complete` | Manage Google Tasks |
-| `contacts-skill` | `contacts_create`, `contacts_list`, `contacts_search` | Manage contacts |
-
-**OAuth Redirect URIs:**
-OAuth redirect URIs are derived at runtime from the request/WebSocket context via `server/services/oauth_utils.py`. No environment variable needed -- works automatically in dev (`http://localhost:3010`) and production (`https://domain.com`).
-
-**Google API Pricing:** All Google Workspace APIs are free with rate limits. See `server/config/pricing.json` for configured limits.
-
-### Email Nodes (3 nodes)
-IMAP/SMTP email integration via the [Himalaya CLI](https://github.com/pimalaya/himalaya). Supports any IMAP/SMTP provider -- Gmail, Outlook/Office 365, Yahoo, iCloud, ProtonMail (Bridge), Fastmail, and custom/self-hosted. Credentials are stored via `auth_service.store_api_key()` and TOML config files are generated on the fly for each operation.
-
-- **emailSend**: **Dual-purpose node** - Send emails via SMTP. Works as workflow node OR AI Agent tool. Group: `['email', 'tool']`. Parameters: provider, to, subject, body, cc, bcc, body_type (text/html).
-- **emailRead**: **Dual-purpose node** - Read and manage emails via IMAP. Works as workflow node OR AI Agent tool. Group: `['email', 'tool']`. Operations:
- - `list`: List envelopes in a folder with pagination
- - `search`: Search emails by query string (e.g., `from:john subject:meeting`)
- - `read`: Read full message content by ID
- - `folders`: List all mailbox folders
- - `move`: Move message to target folder
- - `delete`: Delete message
- - `flag`: Add/remove flag (Seen, Answered, Flagged, Draft, Deleted)
-- **emailReceive**: Polling-based trigger that fires on new emails. Group: `['email', 'trigger']`. Parameters: provider, folder (default `INBOX`), poll_interval (30-3600s), filter_query, mark_as_read. Polls IMAP via Himalaya at the configured interval.
-
-**Architecture:**
-```
-emailSend/Read/Receive node
- |
- v
-handle_email_send/read/receive (handlers/email.py)
- |
- v
-EmailService (email_service.py) <-- credential resolution + provider defaults
- |
- v
-HimalayaService (himalaya_service.py) <-- CLI wrapper
- |
- v
-himalaya CLI binary <-- generates TOML config, calls IMAP/SMTP
-```
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `client/src/assets/icons/email/` | Email icons (send, read, receive) loaded via `?raw` + `svgToDataUri` |
-| `client/src/components/CredentialsModal.tsx` | Email credentials panel (provider dropdown, email/password, conditional custom IMAP/SMTP) |
-| `server/services/himalaya_service.py` | Himalaya CLI wrapper -- send, list_envelopes, search, read, move, delete, flag, list_folders |
-| `server/services/email_service.py` | EmailService orchestrator (credential resolution, operation dispatch, polling helpers) |
-| `server/services/handlers/email.py` | handle_email_send, handle_email_read, handle_email_receive (thin handlers) |
-| `server/services/deployment/manager.py` | `_create_email_poll_coroutine` - deployment-mode continuous polling |
-| `server/services/event_waiter.py` | `TRIGGER_REGISTRY['emailReceive']` + `build_email_filter` |
-| `server/config/email_providers.json` | IMAP/SMTP provider presets, defaults, polling config (zero magic numbers) |
-
-**Credential Storage** (via `auth_service.store_api_key()`):
-
-| Key | Scope | Description |
-|-----|-------|-------------|
-| `email_provider` | Always | One of `gmail`, `outlook`, `yahoo`, `icloud`, `protonmail`, `fastmail`, `custom` |
-| `email_address` | Always | Account email (IMAP/SMTP login) |
-| `email_password` | Always | Password or App Password (stored as secret) |
-| `email_imap_host` | Custom only | Fallback when preset empty -- required for `custom` provider |
-| `email_imap_port` | Custom only | Stored as string, coerced to int via `_coerce_port` |
-| `email_imap_encryption` | Custom only | `tls` / `start-tls` / `none` |
-| `email_smtp_host` | Custom only | Fallback SMTP hostname |
-| `email_smtp_port` | Custom only | |
-| `email_smtp_encryption` | Custom only | |
-
-**Credential Resolution Precedence** (per field, in `EmailService.resolve_credentials`):
-
-1. **Node parameter** (per-node override on the workflow node)
-2. **Provider preset** from `email_providers.json` (e.g., `gmail` preset populates `imap_host = "imap.gmail.com"`)
-3. **Stored custom API key** (`email_imap_host`, etc.) -- only reached when the preset is empty, i.e. `provider == 'custom'`
-
-For named providers the preset always wins before the custom-key fallback. Credentials panel saves the custom IMAP/SMTP keys only when the user selects "Custom / Self-hosted".
-
-**Installation Requirement:** The `himalaya` CLI binary must be installed and on PATH. Install via `cargo install himalaya`, `brew install himalaya`, or download from https://github.com/pimalaya/himalaya/releases. `HimalayaService.ensure_binary()` caches the resolved path on the singleton after first detection; missing binary raises `RuntimeError` with install instructions.
-
-**Authentication:** Email/password (or App Password for Gmail/Outlook/Yahoo) stored per provider. ProtonMail requires running the ProtonMail Bridge locally (IMAP: `localhost:1143`, SMTP: `localhost:1025`, encryption: `none`).
-
-See **[Email Service](./docs-internal/email_service.md)** for the full architecture, API reference, and operational details.
-
----
-
-### Browser Nodes (1 node)
-Interactive browser automation via the [agent-browser](https://www.npmjs.com/package/agent-browser) CLI binary. Wraps a persistent headless Chromium browser with an accessibility-tree-based interaction model designed for AI agents.
-
-- **browser**: **Dual-purpose node** - Interactive browser automation. Works as workflow node OR AI Agent tool. Group: `['browser', 'tool']`. Session persistence across chained operations via `session` parameter (auto-derived as `machina_{execution_id}` if not specified; execution_id is stable per workflow/agent run — threaded through MachinaWorkflow node contexts, AgentWorkflow tool payloads, the legacy tool dispatch, and delegated child agents — so every browser call in one run, including delegated sub-agents, reuses ONE browser instance while separate runs stay isolated). **Instance cap**: each distinct session = one browser instance; `BrowserService` gates new sessions against agent-browser's own registry (`session list --json`) and closes the oldest via per-session `close` when `BROWSER_MAX_INSTANCES` (default 3) would be exceeded; idle browsers self-reap via `AGENT_BROWSER_IDLE_TIMEOUT_MS` injected from `BROWSER_IDLE_TIMEOUT_MS` (default 10 min). Canonical values for both knobs live in `.env.template`. Operations:
- - `navigate`: Open a URL
- - `click`: Click an element (CSS selector or `@eN` ref from snapshot)
- - `type`: Type text keystroke-by-keystroke
- - `fill`: Clear and fill an input field
- - `screenshot`: Take visible/full-page screenshot
- - `snapshot`: Get accessibility tree with `@eN` element refs (AI-optimized, stable within session)
- - `get_text`: Extract text from element
- - `get_html`: Extract innerHTML from element
- - `eval`: Execute JavaScript in page context
- - `wait`: Wait for element to appear
- - `scroll`: Scroll the page (up/down/left/right)
- - `select`: Select dropdown option
- - `batch`: Execute multiple commands at once
-
-**Recommended Workflow (AI agents):** `navigate` -> `snapshot` -> `click`/`type`/`fill` using `@eN` refs -> `snapshot` again to verify. Prefer `@eN` refs over CSS selectors (more stable), and prefer `snapshot` + `click`/`fill` over raw `eval`.
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `client/src/assets/icons/browser/` | Chrome browser icon (from @ant-design/icons-svg) |
-| `server/nodes/browser/_service.py` | BrowserService singleton -- thin subprocess wrapper, reads first JSON line from stdout; instance-cap gate via agent-browser's session registry |
-| `server/nodes/browser/browser/__init__.py` | BrowserNode plugin mapping operation+params to agent-browser CLI args; derives the per-run session |
-| `server/skills/web_agent/browser-skill/SKILL.md` | AI agent skill documenting snapshot-act-snapshot loop + session semantics |
-
-**Installation Requirement:** `agent-browser` is MachinaOs-managed (NOT a `package.json` dep). On first browser-node invocation, [`server/nodes/browser/_install.py::agent_browser_binary_path`](../server/nodes/browser/_install.py) runs `npm install agent-browser@latest --prefix <packages_dir()>` then `<bin> install` for the Chromium runtime. Same shape as [`claude_binary_path`](../server/nodes/agent/claude_code_agent/_oauth.py) (the precedent) — both land in the single shared npm tree at [`core.paths.packages_dir()`](../server/core/paths.py) = `<DATA_DIR>/packages/` (= `~/.machina/packages/` by default), resolving to `~/.machina/packages/node_modules/.bin/agent-browser[.cmd]`. Pre-fix `packages_dir()` routed through `platformdirs.user_cache_path` (`~/.cache/MachinaOs/`, `~/Library/Caches/MachinaOs/`, `%LOCALAPPDATA%\MachinaOs\Cache\`); it now sits under DATA_DIR with everything else MachinaOs owns.
+### Node Catalogue (collapsed)
-**Implementation Detail:** `nodes/browser/_service.py` invokes the resolved binary directly via `subprocess.Popen([binary, ...args], shell=False)`. Reads only the first JSON output line from agent-browser (the daemon holds stdout open), truncates at 100KB, and kills the process tree via `psutil` (the Rust daemon detaches and persists between commands). `shell=False` with list argv avoids BatBadBut (CVE-2024-1874) on Windows; `npm` itself is located via `shutil.which("npm")` which honours `PATHEXT` for the `.cmd` shim. The `[Browser]` info log line carries `session=` so per-run session reuse is visible in the operator log.
+> The authoritative, per-node reference is the backend plugin registry plus the per-node "logic-flow" cards under [docs-internal/node-logic-flows/](./docs-internal/node-logic-flows/) (one card per node, grouped by category, with handles / params / outputs / side-effects / edge-cases). Live node list = glob `server/nodes/**/__init__.py` (~138 today); live total via `pytest --collect-only`. Do NOT maintain a per-node catalogue here — it drifts on every plugin add.
----
-
-### Crawlee Nodes (1 node)
-Python-based web scraping via the [crawlee](https://github.com/apify/crawlee-python) library. Supports static HTML scraping (BeautifulSoup) and JS-rendered content (Playwright) with built-in concurrency, retries, and anti-bot handling.
-
-- **crawleeScraper**: **Dual-purpose node** - Web scraper. Works as workflow node OR AI Agent tool. Group: `['scraper', 'tool']`. Configuration:
- - **Crawler Type**: `beautifulsoup` (static HTML), `playwright` (full browser), `adaptive` (auto-detect)
- - **Mode**: `single` (scrape one URL) or `crawl` (follow links with pattern matching)
- - **Parameters**: url, cssSelector, extractLinks, linkSelector, urlPattern, maxPages, maxDepth, waitForSelector, waitTimeout, takeScreenshot
-- Useful for JavaScript-rendered SPAs where a simple HTTP scraper fails, and for multi-page crawls where concurrency matters.
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/handlers/crawlee.py` | Handler using `BeautifulSoupCrawler` + `PlaywrightCrawler` from crawlee library |
-| `server/skills/web_agent/crawlee-scraper-skill/SKILL.md` | AI agent skill for web scraping |
-
-**Installation Requirement:** The `crawleeScraper` node requires `playwright` chromium for JS-rendered content. The Docker image pre-installs Playwright chromium; for local dev run `playwright install chromium`.
-
----
-
-### Apify Nodes (1 node)
-Web scraping service for social media, search engines, and websites using pre-built actors.
-
-- **apifyActor**: **Dual-purpose node** - Run Apify actors (web scrapers) for Instagram, TikTok, Twitter/X, LinkedIn, Facebook, YouTube, Google Search, Google Maps, and website crawling. Works as workflow node OR AI Agent tool. Group: `['api', 'scraper', 'tool']`. Pre-built actor dropdown with quick input helpers per actor type. Parameters: actorId, actorInput (JSON), maxResults, timeout, memory.
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/handlers/apify.py` | Actor execution via apify-client SDK |
-| `server/skills/web_agent/apify-skill/SKILL.md` | AI agent skill for web scraping |
-| `client/src/components/CredentialsModal.tsx` | Apify API token panel |
-
-**Authentication:** Single API token (Personal or Organization) from Apify Console -> Settings -> Integrations.
-
-### Proxy Nodes (3 nodes)
-Residential proxy provider management with geo-targeting, session control, and automatic failover.
-
-- **proxyRequest**: Make HTTP requests through residential proxy providers with geo-targeting and failover. Parameters: method, url, headers, body, timeout, proxyProvider (auto-select or specific), proxyCountry (ISO code), sessionType (rotating/sticky), stickyDuration, maxRetries, followRedirects.
-- **proxyConfig**: **Dual-purpose node** - Configure proxy providers and routing rules. Works as workflow node OR AI Agent tool. Operations: list_providers, add_provider, update_provider, remove_provider, set_credentials, test_provider, get_stats, add_routing_rule, list_routing_rules, remove_routing_rule.
-- **proxyStatus**: View proxy provider health, scores, and usage statistics. Filter by specific provider name or view all.
-
-**Key Architecture:**
-- **TemplateProxyProvider**: Single class using JSON `url_template` to format proxy URLs for any provider (username-based, password-based, or no encoding)
-- **Auto-selection**: ProxyService ranks providers by health score (success rate, latency) and selects the best one
-- **Transparent proxy on HTTP nodes**: `httpRequest` and `httpScraper` nodes support `useProxy: true` flag. The proxy service handles provider selection, geo-targeting, and session type automatically.
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/proxy/service.py` | ProxyService singleton with provider selection and URL generation |
-| `server/services/proxy/providers.py` | TemplateProxyProvider for JSON url_template formatting |
-| `server/services/proxy/models.py` | ProxyProvider, RoutingRule, SessionType dataclasses |
-| `server/nodes/proxy/proxy_config.py`, `proxy_request.py`, `proxy_status.py` | Plugin nodes (Wave 11); ProxyConfig 10-op matrix lives in `proxy_config.py` |
-| `server/skills/web_agent/proxy-config-skill/SKILL.md` | AI agent skill for proxy configuration |
-| `server/skills/web_agent/http-request-skill/SKILL.md` | HTTP request skill with proxy usage docs |
-
-### Workflow Nodes (2 nodes)
-- **start**: Manual workflow trigger to start workflow execution
-- **taskTrigger**: Event-driven trigger that fires when a delegated child agent completes its task (success or error). Filters by task_id, agent_name, status (all/completed/error), and parent_node_id. Output includes task_id, status, agent_name, result/error, workflow_id.
-
-### Code Nodes (4 nodes)
-- **pythonExecutor**: **Dual-purpose node** - Execute Python code with syntax-highlighted editor, input_data access, and console output. Works as workflow node OR AI Agent tool (`python_code`). Available libraries: math, json, datetime, timedelta, re, random, Counter, defaultdict. CPython `exec()` with a restricted-builtins namespace; its `timeout` param is NOT enforced (only Temporal's CODE_EXEC activity timeout bounds it).
-- **montyExecutor**: **Dual-purpose node** - Execute Python in a hard sandbox via Pydantic's [Monty](https://github.com/pydantic/monty) interpreter (`pydantic-monty==0.0.18`, pinned alpha). Works as workflow node OR AI Agent tool (`sandboxed_python`). Deny-by-default: no filesystem/network/env access unless the LLM requests it via the dynamic `capabilities` param (`http_get` → SSRF-guarded host fetch; `workspace_read`/`workspace_write` → `MountDir` over the per-workflow workspace at `/workspace`). **ENFORCED** `timeout` (→ `ResourceLimits.max_duration_secs`) and `max_memory_mb` (→ `max_memory` bytes) — the headline win over pythonExecutor. Supported subset: def/closures/lambda, comprehensions, f-strings, try/except, async def/await, and `import math|json|re`; NOT supported: class, yield/generators, with, match, `import random|collections|os`. Reads `input_data`; returns the last expression as `output` + captured `print()` as `console_output`. Lazy-imports `pydantic_monty` inside the op (graceful `NodeUserError` if the wheel is missing); plugin at [server/nodes/code/monty_executor/](./server/nodes/code/monty_executor/). Skill: `server/skills/coding_agent/monty-skill/SKILL.md`.
-- **javascriptExecutor**: **Dual-purpose node** - Execute JavaScript code via persistent Node.js server with syntax-highlighted editor and console output. Works as workflow node OR AI Agent tool (`javascript_code`).
-- **typescriptExecutor**: **Dual-purpose node** - Execute TypeScript code via persistent Node.js server with type safety, syntax-highlighted editor and console output. Works as workflow node OR AI Agent tool (`typescript_code`).
-
-### Filesystem & Shell Nodes (4 nodes)
-Dual-purpose tool nodes wrapping `deepagents.backends.LocalShellBackend`. Per-workflow workspace from execution context (`context["workspace_dir"]` = `~/.machina/workspaces/<workflow_slug>/` post-Wave-14; the slug is the human-readable form `AI_Assistant_1`, see "Workflow Naming" section). Fallback uses `Settings().workspace_base_dir` (never `os.getcwd()`).
-
-**Path safety**: `nodes/filesystem/_backend.py` exposes `normalize_virtual_path()` which uses `pathlib.PureWindowsPath` (host-OS independent) to strip Windows drives, POSIX root, and UNC anchors uniformly, then delegates to `deepagents.backends.utils.validate_path` for `..`/`~` rejection. Wired into `file_read`, `file_modify`, and `fs_search` so LLM-emitted paths in any flavour (`C:\foo`, `/tmp/foo`, `\\server\share\x`, `foo\bar`) all map to a virtual path under the workspace. `virtual_mode=True` only sandboxes filesystem ops — `execute()` itself is never path-restricted (deepagents documents this).
-
-**Shell backend is Nushell** (`NushellBackend` subclassing `LocalShellBackend`). Cross-platform parity — same grammar on Windows/macOS/Linux. `inherit_env=True` so `npm`, `node`, `python`, `git`, etc. are reachable on PATH. Nu stdlib is loaded (no `--no-std-lib`). Falls back to upstream `LocalShellBackend.execute()` (POSIX `sh` / cmd.exe) when `nu` isn't on PATH. **Bash idioms (`&&`, `||`, `$VAR`, backticks, `>`) do not work** — see `server/skills/terminal/shell-skill/SKILL.md` for the Nu equivalents.
-
-- **fileRead**: **Dual-purpose node** - Read file contents with line numbers and pagination. Works as workflow node OR AI Agent tool (`file_read`). Parameters: file_path, offset, limit.
-- **fileModify**: **Dual-purpose node** - Write new files or edit existing files with string replacement. Works as workflow node OR AI Agent tool (`file_modify`). Operations: write (wholesale create-or-replace, no overwrite flag), edit (find and replace with old_string/new_string/replace_all).
-- **shell**: **Dual-purpose node** - Execute Nushell commands with timeout. Works as workflow node OR AI Agent tool (`shell_execute`). PATH inherited; external tools (npm, node, python, git, ...) work. All backend calls wrapped in `asyncio.to_thread()` to avoid blocking the event loop. Returns stdout, exit_code, truncated flag.
-- **fsSearch**: **Dual-purpose node** - Search the filesystem with three modes. Works as workflow node OR AI Agent tool (`fs_search`). Modes: ls (list directory), glob (pattern match), grep (search file contents).
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/nodes/filesystem/_backend.py` | `NushellBackend`, `get_backend()`, `normalize_virtual_path()` |
-| `server/skills/coding_agent/` | Skills: file-read-skill, file-modify-skill, fs-search-skill |
-| `server/skills/terminal/` | Skills: shell-skill (Nushell), process-manager-skill, bash-skill, powershell-skill, wsl-skill |
-
-### Process Manager Node (1 node)
-Cross-platform process manager for long-running subprocesses (dev servers, watchers, build tools). Uses `asyncio.create_subprocess_exec` with full system PATH (`env={**os.environ}`). Output streams to Terminal tab via `broadcast_terminal_log()` and persists to log files in the workspace. **Captured lines are ANSI-stripped at the source** — `process_service._read_stream` runs each decoded line through `core.ansi.strip_ansi` (a thin wrapper over **`click.unstyle`**, a declared dep) before the log-file write / Terminal broadcast / `get_output` / `line_handler`, so colour + cursor/erase codes from `vite`/`npm`/`git`/etc. don't render as garbage (`[36m…[39m`). The `shell` node strips its `stdout` the same way. `click.unstyle` was chosen over `rich.Text.from_ansi` because rich line-normalizes and drops trailing newlines; `click.unstyle` is byte-faithful except for the stripped escapes.
-
-- **processManager**: **Dual-purpose node** - Start, stop, restart, and manage long-running processes. Works as workflow node OR AI Agent tool (`process_manager`). Operations: start, stop, restart, list, send_input, get_output. Max concurrent processes configurable in Settings (default: 10).
-
-**Key Architecture:**
-- **ProcessService** singleton (`server/services/process_service.py`) tracks running processes per workflow
-- Each process writes stdout/stderr to `{workspace}/{agent_node_id}/.processes/{name}/stdout.log` and `stderr.log`
-- AI agents read output selectively via `get_output(name, stream, tail)` with pagination
-- Process tree cleanup via `psutil.children(recursive=True)` on stop and server shutdown
-- `PYTHONUNBUFFERED=1` injected for line-buffered Python output
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/process_service.py` | ProcessService singleton with start/stop/restart/list/send_input/get_output/shutdown |
-| `server/services/handlers/process.py` | Dual-purpose handler (workflow + AI tool) |
-| `server/skills/terminal/process-manager-skill/` | Skill for AI agents |
-
-### Utility Nodes (6 nodes)
-- **httpRequest**: Make HTTP requests to external APIs (GET, POST, PUT, DELETE, PATCH) with configurable headers, body, timeout, and optional proxy support (`useProxy: true` routes through configured residential proxy)
-- **webhookTrigger**: Event-driven trigger that waits for incoming HTTP requests at `/webhook/{path}` with method filtering and authentication options
-- **webhookResponse**: Send custom response back to webhook caller with configurable status code, body, and content type
-- **chatTrigger**: Trigger node that receives messages from the Console Panel chat interface
-- **console**: Console output node for logging workflow execution data
-- **teamMonitor**: Real-time monitoring of Agent Team operations. Connect to AI Employee or Orchestrator to display team status, active tasks, and event stream
-
-### Document Processing Nodes (6 nodes)
-RAG pipeline nodes for document ingestion, processing, and vector storage. Supports multiple providers and backends.
-
-- **httpScraper**: Scrape links from web pages with date/page pagination support. Modes: single request, date range iteration, page pagination. Outputs: items array with URLs. Supports `useProxy: true` for proxy routing.
-- **fileDownloader**: Download files from URLs in parallel using semaphore-based concurrency. Parameters: output directory, max workers (1-32), skip existing, timeout.
-- **documentParser**: Parse documents to text using configurable parsers. Parsers: PyPDF (fast), Marker (GPU OCR), Unstructured (multi-format), BeautifulSoup (HTML).
-- **textChunker**: Split text into overlapping chunks for embedding. Strategies: recursive (recommended), markdown, token. Parameters: chunk size (100-8000), overlap (0-1000).
-- **embeddingGenerator**: Generate vector embeddings from text chunks. Providers: HuggingFace (local), OpenAI, Ollama. Default model: BAAI/bge-small-en-v1.5.
-- **vectorStore**: Store and query vector embeddings. Operations: store, query, delete. Backends: ChromaDB (local), Qdrant (production), Pinecone (cloud).
-
-### Chat Nodes (2 nodes)
-- **chatSend**: Send messages to chat conversations
-- **chatHistory**: Retrieve chat conversation history
-
-### Scheduler Nodes (2 nodes)
-- **timer**: Timer-based trigger with configurable delay
-- **cronScheduler**: Cron expression-based scheduling trigger
-
-#### Document Processing Dependencies
-```
-# Required (in server/requirements.txt)
-beautifulsoup4>=4.12.0
-langchain-text-splitters>=0.3.0
-langchain-huggingface>=0.1.0
-chromadb>=0.5.0
-qdrant-client>=1.12.0
-sentence-transformers>=3.0.0
-pypdf>=4.0.0
-
-# Optional (GPU OCR and multi-format parsing)
-# marker-pdf>=1.0.0 # Requires CUDA
-# unstructured>=0.16.0 # Multi-format document parsing
-```
-
-#### Document Node Architecture
-```
-server/nodes/document/
-├── http_scraper.py # each plugin emits its own NodeSpec
-├── file_downloader.py
-├── document_parser.py
-├── text_chunker.py
-├── embedding_generator.py
-└── vector_store.py
-```
+Node groups (palette categories): agent, model, skill, tool, trigger, workflow, search, google, android, whatsapp, telegram, twitter, social, email, proxy, chat, scheduler, text, code, document, location, utility, browser, scraper, filesystem, stripe. See [docs-internal/node-logic-flows/](./docs-internal/node-logic-flows/) for the card index.
## Backend Services
@@ -1618,7 +646,7 @@ server/nodes/document/
- `GET /webhook/` - Webhook endpoint info and usage documentation
#### Workflow Services (`server/services/workflow.py`)
-- Node execution handlers for all 27 node types (including httpRequest, webhookResponse)
+- Node execution dispatches every registered plugin via the `BaseNode` registry (authoritative count = glob `server/nodes/**/__init__.py`, ~138 node files today; live total via `pytest --collect-only`)
- Parameter resolution and template variable substitution
- Result formatting and error handling
@@ -1653,89 +681,7 @@ server/nodes/document/
### Temporal Distributed Execution
-Workflows execute via Temporal for durability and horizontal scaling. Three dispatch paths (legacy `execute_node_activity` / per-type `node.{type}.v{version}` (F4.A) / Agent-as-child-workflow (F4.B)) gated by `TEMPORAL_PER_TYPE_DISPATCH` and `TEMPORAL_AGENT_WORKFLOW_ENABLED` settings flags.
-
-Full architecture, dispatch matrix, per-node lifecycle, heartbeat semantics, the 14 migrating agent types + 2 bypass agents (rlm_agent / claude_code_agent), the 6 F4.B agent activities (`prepare_payload.v1` / `execute_llm_step.v1` / `persist_turn.v1` / `compact_memory.v1` / `store_output.v1` / `broadcast_progress.v1`), the **`delegate_to_<x>` Temporal-native dispatch** (child agent type in `AGENT_WORKFLOW_TYPES` → `workflow.execute_child_workflow` with `parent_node_id` propagation + parent-mirror in `_emit_phase` so the parent's canvas badge advances live; falls through to `execute_activity` otherwise), and the future `TemporalWorkerPool` per-queue routing live in [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md). Tool-call dispatch under F4.A documented at [docs-internal/tool_building_pipeline.md §9](./docs-internal/tool_building_pipeline.md). **Delegation input contract**: the LLM's `{task, context}` args travel as the child workflow input's `invocation` field (per-invocation input vs node config); `prepare_agent_payload` applies it AFTER config resolution so stored node parameters never override the delegated task — empty-task calls are rejected at the parent's call boundary, and `execute_llm_step` raises `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)` on system-only message lists (providers require ≥1 non-system message).
-
-**Configuration** (`.env.template` — canonical defaults; `.env` layers user overrides):
-```env
-TEMPORAL_ENABLED=true
-TEMPORAL_SERVER_ADDRESS=localhost:7233
-TEMPORAL_NAMESPACE=default
-TEMPORAL_TASK_QUEUE=machina-tasks
-TEMPORAL_PER_TYPE_DISPATCH=true # F4.A flag — per-type activity dispatch
-TEMPORAL_AGENT_WORKFLOW_ENABLED=true # F4.B flag — agent-as-child-workflow
-TEMPORAL_FRONTEND_GRPC_PORT=7233
-TEMPORAL_UI_PORT=8080 # Web UI (override of CLI default 8233)
-TEMPORAL_SQLITE_PATH=temporal.db # Flat under DATA_DIR (~/.machina/temporal.db)
-TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS=30
-TEMPORAL_SERVER_READY_TIMEOUT_SECONDS=120
-TEMPORAL_TERMINATE_RUNNING_ON_STARTUP=true # see "Resumption" below
-```
-
-**Temporal binary + persistence** are managed in-process by the plugin-folder pattern under [`server/services/temporal/`](./server/services/temporal/). Single supervised process — the official `temporal` CLI's `server start-dev` mode (per [docs.temporal.io/develop/python/set-up-your-local-python](https://docs.temporal.io/develop/python/set-up-your-local-python)):
-
-- **`_install.py`** — [`pooch`](https://pypi.org/project/pooch/) downloads the official `temporal` CLI archive from `https://temporal.download/cli/archive/latest?platform=<os>&arch=<arch>`. Cross-platform (Windows zip / macOS+Linux tar.gz), cached at `<DATA_DIR>/packages/temporal/`. Download uses an explicit `HTTPDownloader(timeout=300)` — the timeout is per-socket-read, not total transfer, so slow links can finish a ~114 MB fetch (pooch's 30 s default aborted them). Standalone entry: `python -m services.temporal._install` — invoked by `machina build` step [6/6] (fatal there, locked by `test_temporal_install_is_fatal_on_failure`) and by npm postinstall (**non-fatal** there: `TemporalServerRuntime._pre_spawn()` re-downloads lazily on first `machina start`, so a failed eager fetch must never fail `npm install -g`). Note dev mode resolves `DATA_DIR` to repo-local `.machina/`, so dev and `~/.machina` installs keep separate binary caches.
-- **`_runtime.py`** — `TemporalServerRuntime(BaseProcessSupervisor)` spawns `temporal server start-dev` with `--port`, `--ui-port`, `--db-filename`, `--namespace`. Same singleton + lifecycle surface every other supervised service uses ([`nodes/whatsapp/_runtime.py`](./server/nodes/whatsapp/_runtime.py) is the template).
-- **`_handlers.py` + `_refresh.py`** — `temporal_status` / `_start` / `_stop` WS commands and the WS-connect status snapshot, registered through `services.ws_handler_registry` and `services.status_broadcaster.register_service_refresh`.
-- **`client.py`** — `TemporalClientWrapper.connect()` + `terminate_running_workflows()` (resumption toggle, see below).
-
-**Ports:**
-| Service | Port | URL |
-|---------|------|------------------------|
-| gRPC | 7233 | localhost:7233 |
-| Web UI | 8080 | http://localhost:8080 |
-
-Both ports are bound by the same `temporal.exe` process (per docs.temporal.io/cli/server — "all running in a single process").
-
-**Resumption** — controlled by `TEMPORAL_TERMINATE_RUNNING_ON_STARTUP` (default `true`):
-- Temporal SQLite is always persisted (history preserved across restarts; UI keeps showing every workflow that ever ran).
-- On server boot, after the Temporal client connects but BEFORE the worker starts, `TemporalClientWrapper.terminate_running_workflows()` queries Visibility for every `Running` workflow and calls `handle.terminate(reason="MachinaOS startup: auto-resumption disabled")`. Workflows show as `Terminated` (not deleted) in the Temporal UI.
-- Why: `DeploymentManager` has no boot-time reconcile against Temporal Visibility yet — resumed workflows would otherwise keep executing but be invisible to the MachinaOS UI. Flip to `false` once the reconcile lands.
-
-**Start script integration** ([`cli/commands/start.py`](./cli/commands/start.py) + [`cli/commands/dev.py`](./cli/commands/dev.py)): both call `temporal_specs(root, cfg)` ([`cli/commands/_temporal_specs.py`](./cli/commands/_temporal_specs.py)) which returns the single ServiceSpec for the supervised Temporal dev server. The supervised-runtime shim lives at [`server/services/temporal/_supervised_runtime.py`](./server/services/temporal/_supervised_runtime.py) so the spawned `uv run python -m services.temporal._supervised_runtime` resolves out of the workspace `.venv` without any cross-tree path plumbing. The supervisor's `_temporal_running()` probe short-circuits if Temporal is already running on 7233 (lets you run `temporal server start-dev` standalone for debugging).
-
-**Docker**: Not used. The Python supervisor handles cross-platform lifecycle via the pooch-downloaded CLI — single binary, single process, single SQLite file.
-
-**Execution Routing** (`workflow.py`):
-1. If `TEMPORAL_ENABLED=true` and Temporal configured → `_execute_temporal()`
-2. Else if Redis available → `_execute_parallel()` (local parallel)
-3. Else → `_execute_sequential()` (fallback)
-
-**Running with Temporal**:
-Temporal server and embedded worker start automatically with all launch scripts:
-```bash
-npm run start # Starts Temporal server + all services
-npm run dev # Starts Temporal server + all services (dev mode)
-npm run stop # Stops all services including Temporal
-```
-The embedded Temporal worker runs inside the Python backend (`main.py` lifespan via `TemporalWorkerManager`), not as a separate process.
-
-**Standalone Worker** (horizontal scaling):
-```bash
-cd server
-python -m services.temporal.worker
-```
-
-**Node Filtering** (`workflow.py`):
-- Config nodes filtered by handle: `CONFIG_HANDLES = {input-tools, input-memory, input-model, input-skill, input-task, input-teammates}`
-- Trigger nodes (`TRIGGER_NODE_TYPES`) auto-completed if not pre-executed (prevents blocking on event waiter)
-- Android service nodes and skill nodes filtered by type
-
-**Key Files**:
-- `services/temporal/workflow.py` - MachinaWorkflow orchestrator (FIRST_COMPLETED pattern, `_resolve_activity` dispatch)
-- `services/temporal/activities.py` - Legacy `execute_node_activity` (WebSocket round-trip path)
-- `services/temporal/plugin_activities.py` - `collect_plugin_activities` → per-type activities for F4.A
-- `services/temporal/agent_workflow.py` - `AgentWorkflow` child workflow for F4.B agent loops; its first step calls `agent.prepare_payload.v1` to resolve the DB-backed payload, then loops LLM steps + tool dispatch + per-turn persist + compaction
-- `services/temporal/agent_activities.py` - F4.B activities (6 total): `agent.prepare_payload.v1` (resolves DB params + runs `_param_resolver.resolve` so `{{templates}}` work inside the agent's own prompt), `agent.execute_llm_step.v1` (rebuilds tools via `ai_service._build_tool_from_node`; uses LangChain `messages_to_dict`/`messages_from_dict` so Gemini `thought_signature` survives), `agent.persist_turn.v1`, `agent.compact_memory.v1`, `agent.store_output.v1` (wraps `workflow_service.store_node_output` so downstream nodes can resolve `{{aiAgent.response}}`), `agent.broadcast_progress.v1` (emits CloudEvents-shaped `agent_progress` for every phase: `starting` / `llm_step` / `executing_tool` / `tool_completed` / `completed`, optionally driving raw-dict `update_node_status` for canvas-glow color)
-- `services/temporal/worker.py` - TemporalWorkerManager (registers MachinaWorkflow + AgentWorkflow + activities) + `run_standalone_worker()` + `TemporalWorkerPool` (multi-queue, future enhancement)
-- `services/temporal/executor.py` - TemporalExecutor interface matching WorkflowExecutor
-- `services/temporal/client.py` - Client wrapper with runtime heartbeat disabled
-- `services/plugin/base.py::BaseNode.as_activity()` - Per-type activity body for F4.A (mirrors the legacy pipeline)
-
-**Runtime Configuration**: Worker heartbeating is disabled via `Runtime(worker_heartbeat_interval=None)` to avoid warnings on older Temporal server versions.
-
-**Temporal UI**: http://localhost:8080 (Web UI), http://localhost:8233 (HTTP API)
+Workflows execute via Temporal for durability and horizontal scaling, gated by `TEMPORAL_ENABLED`. Three dispatch paths exist — legacy single `execute_node_activity` (default), per-type `node.{type}.v{version}` activities (F4.A, `TEMPORAL_PER_TYPE_DISPATCH=true`), and Agent-as-child-workflow (F4.B, `TEMPORAL_AGENT_WORKFLOW_ENABLED=true`) — with `rlm_agent`/`claude_code_agent` always bypassing AgentWorkflow. Execution routing falls back Temporal → Redis-parallel → sequential; the Temporal dev server + embedded worker are managed in-process under `server/services/temporal/` (pooch-downloaded CLI, single `temporal server start-dev` process, gRPC 7233 + Web UI 8080, SQLite at `~/.machina/temporal.db`). Full architecture, dispatch matrix, per-node + agent-loop lifecycle, the 7 F4.B `agent.*.v1` activities, heartbeat semantics, delegation input contract, resumption toggle (`TEMPORAL_TERMINATE_RUNNING_ON_STARTUP`), and all `.env` tunables live in [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md).
## Development Commands
@@ -1811,7 +757,7 @@ See **[Scripts Reference](./docs-internal/SCRIPTS.md)** for full documentation.
✅ **Authentication System**: n8n-style JWT authentication with HttpOnly cookies, single-owner and multi-user modes
✅ **Cache System**: n8n-pattern cache with Redis (production) / SQLite (local dev) / Memory fallback hierarchy
✅ **AI Thinking/Reasoning**: Extended thinking for Claude, Gemini 2.5/3, OpenAI GPT-5/o-series, Groq Qwen3 with output available in Input Data & Variables for downstream nodes
-✅ **Onboarding Service**: 5-step welcome wizard with Ant Design UI, database persistence, skip/resume/replay support
+✅ **Onboarding Service**: 5-step welcome wizard with shadcn UI, database persistence, skip/resume/replay support
✅ **Proxy System**: Residential proxy provider management with template-based URL formatting, auto-selection by health score, transparent proxy injection on httpRequest/httpScraper nodes via `useProxy: true`
✅ **Markdown Formatter**: GFM markdown to platform-native formatting (Telegram HTML, WhatsApp syntax, plain text) using markdown-it-py
@@ -2148,495 +1094,77 @@ deploy_workflow() -> Sets up triggers, returns immediately
- `server/services/execution/executor.py`: Continuous scheduling with `asyncio.wait(FIRST_COMPLETED)`
### AI Chat Model System (5-Layer Architecture)
-- **Visual Components**: Circular node design with real-time status indicators (green/yellow/red)
-- **Node Definitions**: Backend NodeSpec (`server/nodes/models/<provider>.py`) is the single source of truth; the frontend renders via `lib/nodeSpec.ts` + `adapters/nodeSpecToDescription.ts`
-- **Parameter System**: Universal renderer with drag-and-drop template variables (`{{variable}}`)
-- **API Key Management**: Secure localStorage with base64 encryption and LangChain validation
-- **Execution Engine**: Routes AI nodes to Python Flask backend with auto-injection of API keys
-
-#### Supported AI Providers & Models
-9 providers are available for aiAgent, chatAgent (Zeenie), and all specialized agents: OpenAI, Anthropic, Google (Gemini), DeepSeek, Kimi (Moonshot), Mistral, OpenRouter, **Ollama**, **LM Studio**. Groq and Cerebras are available as standalone chat model nodes only. Model parameters (max output, context length, thinking type, temperature range) are managed by `ModelRegistryService` (`server/services/model_registry.py`) which fetches from OpenRouter for cloud models, and from the user's running local server (via `ollama.AsyncClient.ps()` / `lmstudio.AsyncClient.llm.list_loaded()`) for Ollama / LM Studio. Falls back to `server/config/llm_defaults.json` only when neither source is available.
-
-**Dual-path execution architecture:**
-- **Native SDK path** (`execute_chat()`, `fetch_models()`): OpenAI, Anthropic, Gemini, OpenRouter, xAI, DeepSeek, Kimi, Mistral, Ollama, LM Studio use the `openai` Python SDK via `services/llm/` layer with config-driven `base_url`. Local providers (Ollama, LM Studio) ride the same OpenAI-compat path — `OpenAIProvider` is constructed with `base_url={user-stored URL}` so requests stay on `http://localhost:…` and never hit api.openai.com. Factory: `create_provider(name, api_key)` with lazy imports.
-- **LangChain agent path** (`execute_agent()`, `execute_chat_agent()`): All providers use `ChatOpenAI` with `base_url` from `llm_defaults.json` + `chat_model.bind_tools` for tool calling, driven by the plain-async `_run_agent_loop`. OpenAI-compatible providers (DeepSeek, Kimi, Mistral) pass `max_tokens` via `extra_body` to bypass LangChain's `max_completion_tokens` conversion.
-- **LangChain fallback** (`execute_chat()`): Groq, Cerebras still use LangChain `create_model()` + `chat_model.invoke()`.
-- Native types aliased in ai.py: `NativeMessage`, `NativeThinkingConfig`, `LLMResponse` to avoid naming conflicts with LangChain's `ThinkingConfig`.
-- Provider configs (base_url, models_endpoint, supported_params, temperature constraints) are driven by `server/config/llm_defaults.json` -- no hardcoded URLs in Python code.
-
-**Local LLM SDKs (Ollama, LM Studio):** the validator at [`server/nodes/model/_local_validator.py`](./server/nodes/model/_local_validator.py) probes the user's running server through the official Python SDKs (`ollama>=0.6.0`, `lmstudio>=1.5.0`) and reads only typed fields — no Modelfile-parameters parsing, no `/api/show` modelinfo dict-key hunting. `ollama.AsyncClient.ps()` returns a typed `ProcessResponse.Model` per loaded model with typed `context_length` + typed `ModelDetails` (family, parameter_size, quantization_level, format). `lmstudio.AsyncClient.llm.list_loaded()` returns `AsyncModelHandle` objects whose `get_info()` is a typed `LlmInstanceInfo` (`context_length`, `max_context_length`, `vision`, `trained_for_tool_use`, `architecture`, `params_string`, `format`). The validator persists every probed field into `EncryptedAPIKey.models["model_params"]` (the same JSON column as the model list) and calls `model_registry.register_local_model()` so the sync `get_context_length()` / `get_max_output_tokens()` lookups return real values. The registry also writes through to `model_registry.json` so per-model params survive process restart — the user doesn't have to re-click "Fetch" every time the server bounces.
-
-**Open-world providers** (`openrouter`, `ollama`, `lmstudio`) skip the model-name pattern check in `is_model_valid_for_provider` — local model names like `qwen/qwen3.6-27b` don't contain provider substrings, and the cloud-style filter would have produced a "invalid for provider … using default: <same name>" log line on every chat call. The upstream server still rejects genuinely missing models with a clear 404.
-
-**Provider detection** (`detect_ai_provider` in `server/constants.py`) substring-matches the node type's lowercase form. Local-server providers MUST be listed there — without `ollama` / `lmstudio` branches, `lmstudioChatModel` falls through to `'openai'` and `execute_chat` ends up calling api.openai.com with the local-server placeholder key. Same fix is required in the agent `provider` Literal in `ai_agent.py` / `chat_agent.py` / `_specialized.py`: the dropdown only offers the values listed there, and any local-LLM user picking a missing entry silently falls back to OpenAI cloud.
-
-| Provider | Key Models | Context | Max Output | Thinking | Temp Range |
-|----------|-----------|---------|-----------|----------|------------|
-| **OpenAI** | GPT-5.5/5.4/5.2 | 400K–1.05M | 128K | effort (hybrid) | 0-2 |
-| **OpenAI** | GPT-4.1/4.1-mini/4.1-nano | 1M | 32K | none | 0-2 |
-| **OpenAI** | o3, o4-mini | 200K | 100K | effort (reasoning) | fixed 1.0 |
-| **OpenAI** | GPT-4o/4o-mini | 128K | 16K | none | 0-2 |
-| **Anthropic** | Claude Fable 5 | 1M | 128K | budget | 0-1 |
-| **Anthropic** | Claude Opus 4.8/4.7 | 1M | 128K | budget | 0-1 |
-| **Anthropic** | Claude Sonnet 4.6 | 1M | 64K | budget | 0-1 |
-| **Anthropic** | Claude Haiku 4.5 | 200K | 64K | budget | 0-1 |
-| **Google** | Gemini 3.5-flash, 3.1-pro/flash-lite, 3-flash, 2.5-pro/flash/flash-lite | 1M | 64K | budget | 0-2 |
-| **DeepSeek** | deepseek-v4-flash, deepseek-v4-pro (deepseek-chat/reasoner legacy) | 1M | 64K | thinking modes | 0-2 |
-| **Kimi** | kimi-k2.6, kimi-k2.5, kimi-k2.7-code | 256K | 96K | on by default (disabled for agents) | fixed 0.6 |
-| **Mistral** | mistral-large/medium/small-latest, codestral-latest | 256K | 131K | none | 0-1.5 |
-| **Groq** | Llama 3.3-70b, Llama 3.1-8b, GPT-OSS-120b/20b, Qwen3-32b | 131K | 8-131K | format (Qwen3) | 0-2 |
-| **OpenRouter** | 200+ models from multiple providers | varies | varies | varies | 0-2 |
-| **Cerebras** | GPT-OSS-120b, Qwen-3-235b, GLM-4.7 | 131K | 65K | budget (Qwen) | 0-1.5 |
-| **Ollama** | Whatever the user has pulled (qwen2.5, llama3.x, phi-3, deepseek-r1, ...) | per-loaded-model (typed via `ps()`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |
-| **LM Studio** | Whatever the user has loaded in the LM Studio UI | per-loaded-model (typed via `LlmInstanceInfo.context_length`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |
-
-`_resolve_max_tokens()` in `ai.py` clamps user-requested max_tokens to the model's actual limit.
-
-#### Key Features
-- Visual configuration with status indicators and parameter buttons
-- Template variable system for dynamic parameter binding from connected nodes
-- Provider-specific parameter sets (temperature, max tokens, penalties, safety settings)
-- Secure API key validation with automatic model discovery and 30-day expiration
-- Execution routing to Python Flask backend for AI model processing
-- **Proxy-based authentication** for routing through local servers (Ollama pattern)
-- **Provider default parameters** configurable in Credentials Modal (temperature, max_tokens, thinking settings)
-
-#### Provider Default Parameters
-Users can configure default parameter values per LLM provider in the Credentials Modal. These defaults are applied to new AI nodes using that provider.
-
-**Configurable Parameters:**
-- `temperature`: Controls randomness (range varies: Anthropic 0-1, Cerebras 0-1.5, others 0-2; o-series fixed at 1.0)
-- `max_tokens` (1-200000): Maximum response length (clamped to model's actual limit by `_resolve_max_tokens()`)
-- `thinking_enabled`: Enable extended thinking for supported models
-- `thinking_budget` (1024-16000): Token budget for thinking (Claude, Gemini)
-- `reasoning_effort` (low/medium/high): For OpenAI o-series and GPT-5 hybrid reasoning
-- `reasoning_format` (parsed/hidden): For Groq Qwen3 models
-
-**Database Model** (`server/models/database.py`):
-```python
-class ProviderDefaults(SQLModel, table=True):
- provider: str # openai, anthropic, gemini, groq, openrouter, cerebras
- temperature: float
- max_tokens: int
- thinking_enabled: bool
- thinking_budget: int
- reasoning_effort: str # low, medium, high
- reasoning_format: str # parsed, hidden
-```
-
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/models/database.py` | `ProviderDefaults` SQLModel |
-| `server/core/database.py` | `get_provider_defaults()`, `save_provider_defaults()` CRUD |
-| `server/routers/websocket.py` | `get_provider_defaults`, `save_provider_defaults` handlers |
-| `client/src/hooks/useApiKeys.ts` | `getProviderDefaults()`, `saveProviderDefaults()` methods |
-| `client/src/components/CredentialsModal.tsx` | Default Parameters UI section |
-#### Proxy-Based Authentication (Ollama Pattern)
-AI providers support optional proxy-based authentication, allowing requests to route through a local proxy server that handles authentication. This follows the [Ollama Claude Code integration](https://docs.ollama.com/integrations/claude-code) pattern.
+Chat-model nodes (`openaiChatModel`, `anthropicChatModel`, ...) render through `SquareNode` from the backend NodeSpec. Dual-path execution: native SDK (`ChatUnifier` in `services/llm/`) for chat, LangChain (`_run_agent_loop`) for agents. 11 providers are wired for agents (see the provider list above); model params (max output, context length, thinking type, temperature range) come from `ModelRegistryService`. The 5-layer architecture, the native-vs-LangChain split, proxy/local-LLM (Ollama, LM Studio) routing, the per-provider model + thinking/reasoning matrix (budget / effort / format), and `extract_thinking_from_response` all live in **[Native LLM SDK](./docs-internal/native_llm_sdk.md)**. The runtime output schema (`thinking` field for downstream nodes) is backend-served per [Schema Source of Truth RFC](./docs-internal/schema_source_of_truth_rfc.md).
-**How it works:**
-1. User configures a proxy URL in the Credentials Modal (e.g., `http://localhost:11434`)
-2. Requests route through the proxy instead of directly to the provider API
-3. Proxy handles authentication (token set to "ollama" automatically)
-4. No API key storage needed in MachinaOs - auth delegated to proxy
+## AI Agent Node Architecture
-**Configuration:**
-- Proxy URLs stored in database via `{provider}_proxy` pattern (e.g., `anthropic_proxy`, `openai_proxy`)
-- Configured in Credentials Modal under each AI provider
-- Falls back to direct API key if no proxy configured
+AI Agent (`aiAgent`) and Chat Agent / Zeenie (`chatAgent`) run the same plain-async `_run_agent_loop` (`server/services/ai.py`) and support memory / skills / tools / task input; differences are the backend method (`execute_agent()` vs `execute_chat_agent()`) and error-handling softness. The loop routes purely on `response.tool_calls`, hot-rebinds the tool surface mid-run after canvas mutations, and treats compaction (token-based, post-turn) as the real termination signal. Connection collection is `collect_agent_connections` in `server/services/plugin/edge_walker.py` (5-tuple: memory, skill, tool, input, task); the pre-Wave-11 `handle_ai_agent`/`handle_chat_agent` handlers are gone (dispatch is per-plugin `execute_op` under `server/nodes/agent/<plugin>/__init__.py`).
-**Key Files:**
-| File | Description |
-|------|-------------|
-| `server/services/ai.py` | Native path: `create_provider(name, api_key, proxy_url=url)`. LangChain path: `create_model()` with `base_url` kwarg. |
-| `client/src/components/CredentialsModal.tsx` | Proxy URL input for AI providers |
+**`max_iterations` precedence (highest->lowest): per-node `parameters.max_iterations` > `UserSettings.agent_recursion_limit` > env `AGENT_RECURSION_LIMIT` (default 200) > `llm_defaults.json:agent.recursion_limit`.**
-**Backend Implementation** (`server/services/ai.py`):
-```python
-def create_model(self, provider: str, api_key: str, model: str,
- temperature: float, max_tokens: int,
- thinking: Optional[ThinkingConfig] = None,
- proxy_url: Optional[str] = None):
- # ...
- if proxy_url:
- kwargs['base_url'] = proxy_url
- kwargs[config.api_key_param] = "ollama" # Ollama-style token
-```
+Full reference — agent loop, skill injection, tool building, input/auto-prompt fallback (message>text>content>str), handle topology, spec-driven `AIAgentNode`, async delegation (F4.B child-workflow + legacy fire-and-forget), specialized-agent routing — in [docs-internal/agent_architecture.md](./docs-internal/agent_architecture.md); delegation deep-dive in [agent_delegation.md](./docs-internal/agent_delegation.md), team leads in [agent_teams.md](./docs-internal/agent_teams.md).
-**Use Cases:**
-- Claude Code CLI proxy for Anthropic models
-- **Native Ollama / LM Studio support** — these are first-class providers (see "Local LLM SDKs" above). The user enters their server URL (e.g. `http://localhost:11434/v1`) in the Credentials Modal, the validator probes via the official SDK, and `{provider}_proxy` carries the URL into `OpenAIProvider`'s `base_url` at runtime. Identical mechanism, but the `base_url` resolves to the user's machine, not OpenAI cloud.
-- Custom authentication proxies
-- Development/testing with mock servers
+## Architecture Patterns
+- **Plugin-first (Wave 11).** One folder per plugin under [`server/nodes/<group>/<plugin>/`](./server/nodes/) rooted at `__init__.py`, subclassing `BaseNode` / `ActionNode` / `TriggerNode` / `ToolNode`. Auto-registers via `__init_subclass__`. See [`docs-internal/plugin_system.md`](./docs-internal/plugin_system.md).
+- **Backend NodeSpec is the SSOT** for icon, colour, handles, params, output schema, uiHints, palette group. Frontend consumes via `useNodeSpec(type)` and adapts the JSON Schema → `INodeTypeDescription` shape through [`adapters/nodeSpecToDescription.ts`](./client/src/adapters/nodeSpecToDescription.ts) (legacy interface kept as a render contract; not a parallel schema system).
+- **Component-driven frontend.** shadcn/ui primitives + Tailwind tokens; canvas nodes are spec-driven (see "Spec-driven component design" above).
+- **State management.** TanStack Query owns server-backed data; Zustand (`useAppStore`, `nodeStatusStore`) owns UI state and slice-subscribed high-frequency push state. Slice-selector reads only — never whole-store destructure (see "Frontend Performance Architecture").
+- **Execution pipeline.** Temporal-distributed activities for plugin execution; `WorkflowExecutor` for parallel orchestration; per-node retry / timeout / heartbeat declared on the plugin class.
-## AI Chat Model Implementation Details
+## File Structure Cleanup
+**Removed Files:**
+- `src/nodeDefinitions.ts` + `src/nodeDefinitions/` (27 files) — superseded by backend NodeSpec SSOT; frontend resolves specs via `lib/nodeSpec.ts` + `adapters/nodeSpecToDescription.ts`
+- `src/nodeDefinitions.backup.ts` (backup file)
+- `src/schemas/` directory (unused schema system)
+- `src/utils/schemaParser.ts` (legacy parser)
+- `src/utils/nodeSchemaParser.ts` (unused modern parser)
+- `src/types/NodeSchema.ts` (legacy schema types)
-### Component Architecture
-AI chat model nodes (`openaiChatModel`, `anthropicChatModel`, `geminiChatModel`, etc.) render through `SquareNode` via the spec-driven `COMPONENT_BY_KIND['model']` dispatch in `Dashboard.tsx`. Per-provider visual config (icon, color, displayName, parameter schema) comes from the backend `NodeSpec` declared in `server/nodes/model/<provider>_chat_model.py`. Provider icons use `lobehub:<brand>` strings resolved by `resolveLibraryIcon` against `@lobehub/icons`; brand-specific SVGs (deepseek/kimi/mistral) live under `client/src/assets/icons/llm/` and are addressed via `asset:<key>`. There are no per-provider wrapper components anymore — adding a new provider takes a single Python file under `server/nodes/model/`.
+**Cleaned Code:**
+- Removed unused imports and dead functions
+- Eliminated legacy NodeDefinition interface
+- Streamlined parameter handling logic
+- Maintained backward compatibility only where actively used
-### API Key Management (`src/services/apiKeyManager.ts`)
-- **Validation**: Uses LangChain for real API testing with provider-specific chat models
-- **Storage**: localStorage with base64 encryption and key hashing for security
-- **Models**: Automatic discovery and caching of available models per provider
-- **Expiration**: 30-day validation period with automatic cleanup
+## Testing & Validation
+```bash
+# Development server test
+curl -I http://localhost:3001
-### Execution Flow (`src/services/executionService.ts`)
-- **Detection**: `isAIModelNode()` identifies AI chat models for Python routing
-- **Enhancement**: `injectStoredApiKeys()` auto-injects stored credentials and models
-- **Routing**: AI nodes → Python Flask backend, other nodes → Node.js backend
-- **Logging**: Comprehensive debug output for API key injection and model selection
-- **Supported Types**: `isNodeTypeSupported()` controls which nodes show Run button - includes AI models, agents, Android, WhatsApp, Twitter, Google Workspace, code executors, schedulers, utilities, and document processing nodes
+# TypeScript validation
+npx tsc --noEmit
-### Parameter System Integration
-- **Template Variables**: Support for `{{nodeId.output}}` syntax in all text parameters
-- **Drag-and-Drop**: Visual parameter mapping from connected node outputs
-- **Type-Specific**: Provider-specific parameters (OpenAI response format, Gemini safety settings)
-- **Validation**: Real-time parameter validation with visual feedback
+# Build verification
+npm run build
-### AI Thinking/Reasoning System
-Extended thinking and reasoning capabilities for supported AI models. When enabled, the model's internal reasoning process is captured and available for downstream nodes.
+# Test WebSocket connection for remote Android devices
+python test_websocket.py
+```
-#### Supported Providers & Configuration
+### WebSocket Testing (`test_websocket.py`)
+- Tests health endpoint configured via `WEBSOCKET_URL` environment variable
+- Tests stats endpoint
+- Tests WebSocket connection
+- Validates connection establishment and ping/pong messaging
+- Comprehensive test results with pass/fail status
-| Provider | Models | Parameter | Thinking Type | Notes |
-|----------|--------|-----------|---------------|-------|
-| **Claude** | All Claude 4.x/3.5 | `thinkingBudget` (1024-16000 tokens) | budget | Requires `max_tokens > budget_tokens`. Temperature auto-set to 1. |
-| **Gemini** | gemini-3.x, gemini-2.5-pro/flash | `thinkingBudget` (token count) | budget | Uses `thinking_budget` API parameter |
-| **OpenAI** | o1, o3, o3-mini, o4-mini | `reasoningEffort` (low/medium/high) | effort | Reasoning-only models. Temperature fixed at 1.0. |
-| **OpenAI** | GPT-5.2/5.1/5/5-mini/5-nano | `reasoningEffort` (low/medium/high/xhigh) | effort | Hybrid reasoning: can operate with or without thinking. |
-| **Groq** | qwen3-32b | `reasoningFormat` ('parsed' or 'hidden') | format | 'parsed' returns reasoning, 'hidden' returns only final answer |
-| **Cerebras** | qwen-3-235b | `reasoningFormat` ('parsed' or 'hidden') | format | Same format-based reasoning as Groq Qwen |
+## Production Deployment
-#### Thinking/Reasoning Parameters
-The thinking/reasoning fields (`thinkingEnabled`, `thinkingBudget`, `reasoningEffort`, `reasoningFormat`) live in the backend NodeSpec for each chat model (`server/nodes/models/<provider>.py`) and are surfaced through `AIChatModelParams` in `server/models/nodes.py`. The frontend renders them automatically via the universal parameter panel.
+### Self-Deploy CLI (`machina deploy`) — current path
+One command provisions a login-gated MachinaOs VM on a cloud provider. Two stages: the
+operator's **cloud CLI** (gcloud; aws planned) handles auth + project/region/zone resolution +
+ADC verification + API enablement, then **Terraform** (`cli/terraform/gcp/`) owns all resources —
+VM (always named `machinaos`), firewall, artifact bucket (local `npm pack` source), service
+account, and a cloud-init startup script that installs Node 22 + uv + the package and runs
+`machina serve` under systemd. Login gate = built-in auth (`VITE_AUTH_ENABLED=true`,
+`AUTH_MODE=single`) with the owner credential generated at deploy time and seeded on first boot.
-#### Backend Implementation (`server/services/ai.py`)
-The AI service extracts thinking content from LangChain response objects:
-
-```python
-def extract_thinking_from_response(response, provider: str) -> Optional[str]:
- """Extract thinking/reasoning from AI response based on provider."""
- # Claude: content_blocks with type='thinking'
- # Gemini: response_metadata.candidates[0].content.parts with thought=True
- # Groq: additional_kwargs.reasoning or response_metadata.reasoning
- # OpenAI o-series: requires organization verification
-```
-
-**Response Structure:**
-```python
-{
- "success": True,
- "result": {
- "response": "The final answer text",
- "thinking": "The model's internal reasoning (if available)",
- "model": "claude-3-5-sonnet-20241022",
- "provider": "anthropic",
- "finish_reason": "stop",
- "timestamp": "2025-01-23T..."
- }
-}
-```
-
-#### Output Schema for Connected Nodes
-The `thinking` field is available in Input Data & Variables for downstream nodes. This schema applies to all AI nodes including chat models and specialized agents.
-
-**Source of truth (Wave 3, April 2026):** runtime output shapes live on the **backend**, not the frontend. Declared in Pydantic models at `server/services/node_output_schemas.py` and served lazy via `GET /api/schemas/nodes/{node_type}.json` / the `get_node_output_schema` WebSocket handler. The frontend's InputSection prefers real execution data, falls back to the backend schema, then an empty state. See [docs-internal/schema_source_of_truth_rfc.md](./docs-internal/schema_source_of_truth_rfc.md).
-
-```python
-# server/services/node_output_schemas.py
-class AIAgentOutput(_OutputBase):
- response: Optional[str] = None
- thinking: Optional[str] = None # Available for drag-and-drop mapping
- model: Optional[str] = None
- provider: Optional[str] = None
- finish_reason: Optional[str] = None
- timestamp: Optional[str] = None
-
-# Shared across every LLM-backed agent + chat model:
-_AGENT_TYPES = ['aiAgent', 'chatAgent', 'android_agent', 'coding_agent',
- 'web_agent', 'task_agent', 'social_agent', 'travel_agent',
- 'tool_agent', 'productivity_agent', 'payments_agent',
- 'consumer_agent', 'autonomous_agent', 'orchestrator_agent',
- 'ai_employee', 'rlm_agent', 'claude_code_agent']
-_CHAT_MODEL_TYPES = ['openaiChatModel', 'anthropicChatModel', 'geminiChatModel', ...]
-NODE_OUTPUT_SCHEMAS = {
- **{t: AIAgentOutput for t in _AGENT_TYPES},
- **{t: AIAgentOutput for t in _CHAT_MODEL_TYPES},
- # ...
-}
-```
-
-Adding a new node type's output shape: define one Pydantic model, register it in `NODE_OUTPUT_SCHEMAS`. Zero frontend change.
-
-#### UI Display (`client/src/components/ui/NodeOutputPanel.tsx`)
-- **ThinkingBlock Component**: Collapsible display for thinking content
-- **Default Expanded**: Thinking block is expanded by default when present
-- **Provider-Aware**: Shows appropriate label based on provider (e.g., "Claude Extended Thinking")
-
-#### Limitations
-- **OpenAI o-series**: Reasoning summaries are only available to organizations that have completed verification at platform.openai.com. Without verification, `thinking` will be `null`.
-- **Claude**: `max_tokens` must be greater than `thinkingBudget`. Temperature is automatically set to 1 when thinking is enabled.
-- **Groq**: Only Qwen3-32b supports reasoning (QwQ removed from Groq). Format 'hidden' suppresses reasoning output.
-- **Cerebras**: Qwen-3-235b supports format-based reasoning (same as Groq Qwen).
-
-## AI Agent Node Architecture
-
-### Agent Loop Termination
-The standard agent path (`_run_agent_loop` in `services/ai.py`) routes purely on `response.tool_calls` — no custom iteration counter beyond `max_iterations`. Resolution precedence (highest → lowest):
-
-1. Per-agent-node `parameters.max_iterations` (set on the agent node itself).
-2. Per-user `UserSettings.agent_recursion_limit` (Settings tab, DB-backed).
-3. Env `Settings.agent_recursion_limit` from `AGENT_RECURSION_LIMIT` (default **200**).
-4. JSON fallback `llm_defaults.json:agent.recursion_limit` (legacy SSOT, kept as last-resort).
-
-The token-based compaction threshold (`compaction_ratio` × model context_length, default 80% × ~200K for claude-sonnet-4-6 ≈ 160K) is the real termination signal: `_track_token_usage` runs after each agent turn, and when cumulative tokens cross the threshold it invokes `CompactionService.compact_context` to summarise the transcript before the next turn. Ratio precedence mirrors the recursion limit chain — per-session `custom_threshold` > `UserSettings.compaction_ratio` > `Settings.compaction_ratio` (env `COMPACTION_RATIO`, default 0.8) > JSON fallback.
-
-If the iteration cap is ever hit anyway, `_run_agent_loop` appends a terminal `AIMessage` carrying a truncation note and returns `truncated=True` so `_extract_text_content` returns a usable partial response and the workflow continues — `_track_token_usage` and any post-loop persistence still run.
-
-### Hot rebind after canvas mutation
-
-`_run_agent_loop` accepts a `rebind_from_operations: Callable[[List[Dict]], Awaitable[List[StructuredTool]]]` callback. After each iteration's tool calls return, the loop inspects every result for an `operations` field (workflow_ops batch, today only `agentBuilder` emits this), invokes the callback, extends the local `current_tools` list, and re-calls `chat_model.bind_tools(current_tools)` so the LLM sees the new tools in the NEXT iteration — no Run-stop-Run cycle. The callback is wired by `execute_agent` / `execute_chat_agent` as a closure that calls `_build_tool_from_node` for each `add_node` op with `component_kind == "tool"` OR `usable_as_tool=True` (excluding chat models). Gated by `UserSettings.auto_rebind_tools_after_canvas_change` (default True); when off the closure is `None` and the LLM is told "Available on your next turn" in the operation summary.
-
-For the F4.B Temporal path, the same rebind happens via the `agent.refresh_tools.v1` activity called from `AgentWorkflow.run` between iterations. See [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md).
-
-### Spec-driven component design (Wave 10.D)
-
-`AIAgentNode.tsx` knows nothing about specific agent types. It calls `useNodeSpec(type)` once and reads `handles`, `color`, `displayName`, `subtitle`, and `uiHints` from the cached `NodeSpec` ([`AIAgentNode.tsx:55-68`](./client/src/components/AIAgentNode.tsx#L55-L68)). The earlier 60-line `AGENT_CONFIGS` map (one entry per agent type, each with hardcoded `themeColorKey` + handle topology) was retired in Wave 10.D.
-
-Component dispatch lives in [`Dashboard.tsx:74-105`](./client/src/Dashboard.tsx#L74-L105):
-
-```typescript
-const COMPONENT_BY_KIND: Record<string, React.ComponentType<any>> = {
- start: StartNode,
- trigger: TriggerNode,
- agent: AIAgentNode, // every plugin with component_kind="agent" routes here
- chat: AIAgentNode,
- model: SquareNode,
- square: SquareNode,
- tool: SquareNode,
- generic: SquareNode,
-};
-
-const createNodeTypes = () => {
- const types: Record<string, React.ComponentType<any>> = {};
- listCachedNodeSpecs().forEach(spec => {
- const kind = spec.componentKind;
- if (kind && COMPONENT_BY_KIND[kind]) {
- types[spec.type] = COMPONENT_BY_KIND[kind];
- } else if (spec.type === 'teamMonitor') {
- types[spec.type] = TeamMonitorNode;
- } else if ((spec.uiHints as any)?.isMasterSkillEditor === true) {
- types[spec.type] = ToolkitNode;
- } else {
- types[spec.type] = SquareNode;
- }
- });
- return types;
-};
-```
-
-The dispatch keys off `spec.componentKind` (a backend-declared string), never `spec.type`. Specialized agent visuals (icon, color, handles, subtitle, width, height) all come from the spec. Icons live as `icon.svg` in the plugin folder (served via `/api/schemas/nodes/<type>/icon`); colors live as `meta.json` in the plugin folder (post-F2). [`server/nodes/visuals.json`](./server/nodes/visuals.json) is the fallback registry for emoji / `lobehub:<brand>` icons + the skill reverse-map.
-
-### AI Agent vs Zeenie
-
-| Feature | AI Agent | Zeenie |
-|---------|----------|------------|
-| Tool Calling | Yes (agent loop) | Yes (agent loop) |
-| Memory Support | Yes | Yes |
-| Skill Support | Yes | Yes |
-| Task Input | Yes (input-task) | Yes (input-task) |
-| Bottom Handles | Skill, Tools | Skill, Tools |
-| Left Handles | Input, Memory, Task | Input, Memory, Task |
-| Backend Method | `execute_agent()` | `execute_chat_agent()` |
-| Async Delegation | Yes (fire-and-forget) | Yes (fire-and-forget) |
-
-### Unified Tool Calling Architecture
-
-Both AI Agent and Zeenie use the **same tool calling pattern**:
-
-1. **Tool Building**: Both use `_build_tool_from_node()` to create schema-only tools
-2. **Tool Execution**: Both use `execute_tool()` from `handlers/tools.py`
-3. **Supported Tools**: `calculatorTool`, `currentTimeTool`, `duckduckgoSearch`, `androidTool`, `httpRequest`, `braveSearch`, `serperSearch`, `perplexitySearch`
-
-**Tool Execution Flow:**
-```
-Tool Node (connected to input-tools)
- ↓
-Handler collects tool_data: {node_id, node_type, parameters, label, connected_services}
- ↓
-AIService._build_tool_from_node() → creates schema-only StructuredTool
- ↓
-_run_agent_loop binds tools via chat_model.bind_tools(tools)
- ↓
-LLM decides to call tool with arguments
- ↓
-tool_executor callback → execute_tool() from handlers/tools.py
- ↓
-Dispatch to handler: _execute_http_request(), _execute_calculator(), etc.
-```
-
-**Key Files:**
-| File | Purpose |
-|------|---------|
-| `server/services/ai.py` | `_build_tool_from_node()` - builds schema-only tools for both agents |
-| `server/services/handlers/tools.py` | `execute_tool()` - dispatches to specific handlers |
-| `server/services/handlers/ai.py` | Collects `tool_data` from `input-tools` handle |
-
-### Zeenie Input Methods
-
-Zeenie accepts input in two ways:
-
-1. **Template Variable (Explicit)**: Set the Prompt field to `{{chatTrigger.message}}` or `{{whatsappReceive.text}}`
- - Templates are resolved by `ParameterResolver` before handler execution
- - Supports nested paths: `{{nodeName.nested.field}}`
-
-2. **Auto-Fallback (Implicit)**: Leave the Prompt field empty
- - Handler detects nodes connected to `input-main` handle
- - Reads output from `context.get('outputs', {}).get(source_node_id)`
- - Extracts text from `message`, `text`, `content` fields (in order)
- - Falls back to string representation of entire output
-
-**Example Workflow:**
-```
-Chat Trigger → Zeenie ← HTTP Skill (SKILL.md context)
- ← HTTP Request (tool node)
-```
-
-The Zeenie will:
-- Load SKILL.md instructions from connected skill nodes
-- Build tools from connected tool nodes (httpRequest, calculatorTool, etc.)
-- Use `_run_agent_loop` for tool execution when tools are connected
-
-### Backend Handlers
-Wave 11: `handle_ai_agent` / `handle_chat_agent` were deleted. Agent execution now flows through `BaseNode.execute()` + `NodeContext.from_legacy()` via the node registry (`server/nodes/agent/`). Connection collection is handled by the agent node classes, which internally call `_collect_agent_connections()` to:
-- Scans edges for nodes connected to `input-memory`, `input-skill`, `input-tools`, `input-main`/`input-chat` handles
-- Returns a 4-tuple: `(memory_data, skill_data, tool_data, input_data)`
-- Handles MasterSkill expansion into individual skill entries
-
-Both call corresponding AI service methods:
-- `AIService.execute_agent()` - Runs `_run_agent_loop` with tool binding
-- `AIService.execute_chat_agent()` - Runs `_run_agent_loop` (when tools) or single `ainvoke` (no tools), with skills providing context + tools
-
-### Async Agent Delegation (Nested Agents)
-
-AI Agents can delegate tasks to other agents connected to their `input-tools` handle. This enables hierarchical agent architectures where a parent agent can spawn child agents that work independently.
-
-**Two execution paths** depending on whether the parent runs under Temporal F4.B:
-
-- **F4.B (default)**: parent `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow` when the child type is in `AGENT_WORKFLOW_TYPES`. Parent's `node_id` is passed in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent's canvas badge (parent badge advances live alongside child iteration counter instead of freezing). The LLM's `{task, context}` args travel as `child_context["invocation"]` — per-invocation input applied AFTER config resolution in `prepare_agent_payload` (`task` → system_message, `context`-or-`task` → prompt, same semantics as the legacy path), so stored node parameters never override the delegated task; empty-task calls are rejected at the call boundary without spawning. Deterministic child workflow id `f"{parent_workflow_id}-delegate-{child_node_id}-{iteration}"`. Parent waits for child result through the child-workflow handle — synchronous within the parent's loop.
-- **Legacy / F4.A-only**: the fire-and-forget pattern below. Still the path for `rlm_agent` / `claude_code_agent` (excluded from `AGENT_WORKFLOW_TYPES`) and any deployment with `TEMPORAL_AGENT_WORKFLOW_ENABLED=false`.
-
-**Architecture: Fire-and-Forget Pattern (legacy / F4.A)**
-```
-Parent Agent calls "delegate_to_ai_agent" tool
- |
-Tool handler spawns asyncio.Task for Child Agent
- |
-Returns immediately: {"status": "delegated", "task_id": "..."}
- |
-Parent Agent continues working
- |
-Child Agent executes independently in background
- |
-Child broadcasts its own status updates (executing, success, error)
-```
-
-**How It Works (legacy / F4.A):**
-1. Connect a Child Agent (aiAgent/chatAgent) to Parent Agent's `input-tools` handle
-2. Parent sees a tool like `delegate_to_ai_agent` with schema `{task: string, context?: string}`
-3. When Parent calls the tool, handler spawns Child as `asyncio.create_task()`
-4. Tool returns immediately with `{"status": "delegated", "task_id": "..."}`
-5. Parent continues without waiting
-6. Child executes with its own connected tools, skills, and memory
-7. Both agents can execute simultaneously with independent status indicators
-
-**Key Files:**
-| File | Purpose |
-|------|---------|
-| `server/services/ai.py` | `DelegateToAgentSchema` in `_get_tool_schema()`, injects `ai_service`, `database`, `nodes`, `edges` into tool config |
-| `server/services/handlers/tools.py` | `_execute_delegated_agent()` - spawns child as background task, `get_delegated_task_status()` utility |
-| `server/services/handlers/ai.py` | Passes `context` to `execute_agent()`/`execute_chat_agent()` for nested delegation |
-
-**Design Decisions:**
-- **Memory Isolation**: Child uses its own connected memory, not shared with Parent
-- **Error Isolation**: Child errors don't propagate to Parent - logged and broadcast independently
-- **Task Tracking**: Background tasks tracked in `_delegated_tasks` dict, cleaned up on completion
-
-### Specialized AI Agents
-
-The system ships specialized agent variants — each is a folder under [`server/nodes/agent/`](./server/nodes/agent/) with `__init__.py` declaring a `SpecializedAgentBase` subclass. Authoritative list: glob that folder. Per-type display name / subtitle / description live on the plugin class; icon comes from `<plugin>/icon.svg` (or visuals.json emoji fallback); color comes from `<plugin>/meta.json` (post-F2). Do not maintain a hand-list of agent types in this doc — it drifts on every plugin add.
-
-Standard handle topology (declared on `SpecializedAgentBase` via `std_agent_handles()`):
-- **Left**: `input-main` (Input, 30%), `input-memory` (Memory, 55%), `input-task` (Task, 85%)
-- **Bottom**: `input-skill` (Skill, 25%), `input-tools` (Tool, 75%)
-- **Top**: `output-top` (Output)
-
-**Team Lead Agents** (`orchestrator_agent`, `ai_employee`) declare an extra `input-teammates` handle on the bottom (50%) for delegation. They are listed in `TEAM_LEAD_TYPES` in [`client/src/components/TeamMonitorNode.tsx`](./client/src/components/TeamMonitorNode.tsx) (frontend tribal array — flagged as tech debt; see "remaining tribal arrays" note).
-
-`AIAgentNode.tsx` is type-agnostic: it calls `useNodeSpec(type)` and renders whatever handles/icon/color the spec returns — no `AGENT_CONFIGS` map.
-
-## Architecture Patterns
-- **Plugin-first (Wave 11).** One folder per plugin under [`server/nodes/<group>/<plugin>/`](./server/nodes/) rooted at `__init__.py`, subclassing `BaseNode` / `ActionNode` / `TriggerNode` / `ToolNode`. Auto-registers via `__init_subclass__`. See [`docs-internal/plugin_system.md`](./docs-internal/plugin_system.md).
-- **Backend NodeSpec is the SSOT** for icon, colour, handles, params, output schema, uiHints, palette group. Frontend consumes via `useNodeSpec(type)` and adapts the JSON Schema → `INodeTypeDescription` shape through [`adapters/nodeSpecToDescription.ts`](./client/src/adapters/nodeSpecToDescription.ts) (legacy interface kept as a render contract; not a parallel schema system).
-- **Component-driven frontend.** shadcn/ui primitives + Tailwind tokens; canvas nodes are spec-driven (see "Spec-driven component design" above).
-- **State management.** TanStack Query owns server-backed data; Zustand (`useAppStore`, `nodeStatusStore`) owns UI state and slice-subscribed high-frequency push state. Slice-selector reads only — never whole-store destructure (see "Frontend Performance Architecture").
-- **Execution pipeline.** Temporal-distributed activities for plugin execution; `WorkflowExecutor` for parallel orchestration; per-node retry / timeout / heartbeat declared on the plugin class.
-
-## File Structure Cleanup
-**Removed Files:**
-- `src/nodeDefinitions.ts` + `src/nodeDefinitions/` (27 files) — superseded by backend NodeSpec SSOT; frontend resolves specs via `lib/nodeSpec.ts` + `adapters/nodeSpecToDescription.ts`
-- `src/nodeDefinitions.backup.ts` (backup file)
-- `src/schemas/` directory (unused schema system)
-- `src/utils/schemaParser.ts` (legacy parser)
-- `src/utils/nodeSchemaParser.ts` (unused modern parser)
-- `src/types/NodeSchema.ts` (legacy schema types)
-
-**Cleaned Code:**
-- Removed unused imports and dead functions
-- Eliminated legacy NodeDefinition interface
-- Streamlined parameter handling logic
-- Maintained backward compatibility only where actively used
-
-## Testing & Validation
-```bash
-# Development server test
-curl -I http://localhost:3001
-
-# TypeScript validation
-npx tsc --noEmit
-
-# Build verification
-npm run build
-
-# Test WebSocket connection for remote Android devices
-python test_websocket.py
-```
-
-### WebSocket Testing (`test_websocket.py`)
-- Tests health endpoint configured via `WEBSOCKET_URL` environment variable
-- Tests stats endpoint
-- Tests WebSocket connection
-- Validates connection establishment and ping/pong messaging
-- Comprehensive test results with pass/fail status
-
-## Production Deployment
-
-### Self-Deploy CLI (`machina deploy`) — current path
-One command provisions a login-gated MachinaOs VM on a cloud provider. Two stages: the
-operator's **cloud CLI** (gcloud; aws planned) handles auth + project/region/zone resolution +
-ADC verification + API enablement, then **Terraform** (`cli/terraform/gcp/`) owns all resources —
-VM (always named `machinaos`), firewall, artifact bucket (local `npm pack` source), service
-account, and a cloud-init startup script that installs Node 22 + uv + the package and runs
-`machina serve` under systemd. Login gate = built-in auth (`VITE_AUTH_ENABLED=true`,
-`AUTH_MODE=single`) with the owner credential generated at deploy time and seeded on first boot.
-
-```bash
-machina deploy up --provider gcp --owner-email [email protected] # provision + install + print URL/creds
-machina deploy status # URL + /health
-machina deploy destroy # terraform destroy + clear state
-```
+```bash
+machina deploy up --provider gcp --owner-email [email protected] # provision + install + print URL/creds
+machina deploy status # URL + /health
+machina deploy destroy # terraform destroy + clear state
+```
Key files: `cli/commands/serve.py` (single-port runtime: uvicorn fronts API + WS + built SPA, plus
the node sidecar), `cli/commands/deploy/` (verbs, secrets, Terraform driver, provider CLI adapters),
@@ -2649,644 +1177,20 @@ The legacy `deploy.sh` (docker-compose images over SCP to a GCE box) was removed
Compose notes below are retained for reference for the historical container topology.
### Docker Deployment (legacy reference)
-The project previously deployed using Docker Compose with nginx reverse proxy.
-
-#### Docker Configuration (4-Container Stack)
-
-**Services:**
-| Container | Image | Port | Description |
-|-----------|-------|------|-------------|
-| redis | redis:7-alpine | 6379 | Cache and pub/sub for workflows |
-| backend | machinaos-backend | 3010 | FastAPI Python backend |
-| frontend | machinaos-frontend | 3000 | React app via nginx |
-| whatsapp | machinaos-whatsapp | 5000 | Go WhatsApp bridge service |
-
-**Frontend (`client/Dockerfile`):**
-- Multi-stage build: Node.js builder → nginx:alpine production
-- Serves static files via nginx on port 80 (mapped to 3000)
-- Size: ~54 MB
-
-**Backend (`server/Dockerfile`):**
-- Python 3.12-slim base with Node.js 22.x for JS/TS execution
-- Includes Playwright chromium for JS-rendered web scraping (crawleeScraper node)
-- Includes persistent Node.js server (Express + tsx) on port 3020
-- Optimized bytecode compilation (`python -O -m compileall`)
-- Health check endpoint on port 3010
-- Startup script (`start.sh`) runs both Python and Node.js servers (must have LF line endings, not CRLF)
-- Depends on: redis, whatsapp
-- Size: ~800 MB (includes Playwright chromium)
-
-**WhatsApp (`docker/Dockerfile.whatsapp`):**
-- Uses npm package `whatsapp-rpc` with pre-built binaries
-- Node.js 20-alpine base with `npx whatsapp-rpc api --foreground`
-- Binary downloaded from GitHub releases during npm postinstall
-- Exposed on port 9400 (configurable via `PORT`, `WHATSAPP_RPC_PORT` env vars, or `--port` CLI flag)
-- QR codes generated as base64 PNG in memory (no file I/O)
-- Also published to PyPI as `whatsapp-rpc` (async Python client)
-- Size: ~150 MB (includes Node.js runtime)
-
-**Redis:**
-- Official redis:7-alpine image
-- Healthcheck: `redis-cli ping`
-- Persistent volume: `redis_data`
-- No authentication (internal network only)
-
-**Development Compose (`docker-compose.yml`):**
-```yaml
-services:
- # Redis uses profiles - only starts when REDIS_ENABLED=true
- redis:
- image: redis:7-alpine
- profiles:
- - redis # Only starts with --profile redis flag
- ports: ["${REDIS_PORT:-6379}:6379"]
- healthcheck:
- test: ["CMD", "redis-cli", "ping"]
-
- backend:
- build: ./server
- ports: ["${PYTHON_BACKEND_PORT:-3010}:${PYTHON_BACKEND_PORT:-3010}"]
- volumes:
- - ./server:/app
- - /app/nodejs/node_modules # Preserve Linux binaries (prevents Windows esbuild conflict)
- depends_on:
- whatsapp: { condition: service_healthy } # No Redis dependency
- environment:
- - REDIS_ENABLED=${REDIS_ENABLED:-false}
- - REDIS_URL=redis://redis:6379
-
- frontend:
- build: ./client
- ports: ["${VITE_CLIENT_PORT:-3000}:${VITE_CLIENT_PORT:-3000}"]
-
- whatsapp:
- build:
- context: .
- dockerfile: docker/Dockerfile.whatsapp
- ports: ["${WHATSAPP_RPC_PORT:-9400}:${WHATSAPP_RPC_PORT:-9400}"]
-```
-
-**Docker Scripts Wrapper (`scripts/docker.js`):**
-Auto-detects `REDIS_ENABLED` in `.env` and adds `--profile redis` flag when enabled:
-```javascript
-// Reads .env and checks REDIS_ENABLED value
-function isRedisEnabled() {
- const content = readFileSync(resolve(ROOT, '.env'), 'utf8');
- const match = content.match(/^REDIS_ENABLED\s*=\s*(.+)$/m);
- const value = match?.[1].trim().toLowerCase();
- return value === 'true' || value === '1' || value === 'yes';
-}
-
-// Adds --profile redis when enabled
-if (isRedisEnabled()) {
- composeArgs.push('--profile', 'redis');
-}
-```
-
-**Production Compose (`docker-compose.prod.yml`):**
-```yaml
-services:
- redis:
- image: redis:7-alpine
- ports: ["6379:6379"]
- volumes: ["redis_data:/data"]
- healthcheck:
- test: ["CMD", "redis-cli", "ping"]
-
- whatsapp:
- build:
- context: .
- dockerfile: docker/Dockerfile.whatsapp
- ports: ["9400:9400"]
- healthcheck:
- test: ["CMD", "wget", "-q", "--spider", "http://localhost:9400/health"]
-
- backend:
- build: ./server
- ports: ["3010:3010"]
- depends_on:
- redis: { condition: service_healthy }
- whatsapp: { condition: service_healthy }
- environment:
- - REDIS_ENABLED=true
- - REDIS_URL=redis://redis:6379
-
- frontend:
- build: ./client
- ports: ["3000:80"]
-```
-
-#### Nginx Configuration
-Located at `/etc/nginx/sites-available/flow.zeenie.xyz`:
-- Frontend: `/` → `http://127.0.0.1:3000`
-- Backend API: `/api/` → `http://127.0.0.1:3010/api/`
-- WebSocket: `/ws/` → `http://127.0.0.1:3010/ws/` (with upgrade headers)
-- Webhook: `/webhook/` → `http://127.0.0.1:3010/webhook/`
-- Health: `/health` → `http://127.0.0.1:3010/health`
-- SSL via Let's Encrypt certbot
-
-#### Environment Configuration
-
-**Development** (`server/.env`):
-- `DEBUG=true`
-- `CORS_ORIGINS` includes localhost ports
-- `REDIS_ENABLED=false` (uses SQLite cache for local dev)
-
-**Production** (Docker environment variables):
-- `DEBUG=false`
-- `CORS_ORIGINS=["https://your-domain.com"]`
-- `REDIS_ENABLED=true` (Docker Redis container)
-- `REDIS_URL=redis://redis:6379`
-- Environment set in `docker-compose.prod.yml`, not `.env` file
-
-#### Frontend API URL Resolution
-The frontend automatically detects production vs development:
-
-```typescript
-// client/src/config/api.ts
-const isProduction = typeof window !== 'undefined' &&
- !window.location.hostname.includes('localhost') &&
- !window.location.hostname.includes('127.0.0.1');
-
-return {
- PYTHON_BASE_URL: isProduction ? '' : 'http://localhost:3010',
-};
-```
-
-- **Production**: Empty base URL = relative URLs (same origin)
-- **Development**: Explicit `http://localhost:3010`
-
-WebSocket URL derived from base URL:
-```typescript
-// client/src/contexts/WebSocketContext.tsx
-if (!baseUrl) {
- // Production: use current origin
- const wsProtocol = window.location.protocol === 'https:' ? 'wss' : 'ws';
- return `${wsProtocol}://${window.location.host}/ws/status`;
-}
-```
-
-#### Resource Usage (GCP e2-micro)
-| Resource | Value |
-|----------|-------|
-| CPU | 2 cores (Intel Xeon @ 2.20GHz) |
-| RAM | 1.9 GB total, ~820 MB used |
-| Disk | 14 GB total, ~9.2 GB used |
-| Backend Memory | ~144 MB |
-| Frontend Memory | ~3.4 MB |
-
-#### Useful Commands
-```bash
-# View logs (all containers)
-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f'
-
-# View specific service logs
-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f backend'
-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f whatsapp'
-
-# Restart all services
-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart'
-# Restart specific service
-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart backend'
-
-# Check container status
-ssh $DEPLOY_HOST 'docker ps'
-
-# Check resource usage
-ssh $DEPLOY_HOST 'docker stats --no-stream'
-
-# Check Redis connection
-ssh $DEPLOY_HOST 'docker exec machinaos-redis-1 redis-cli ping'
-
-# Check backend health (shows redis_enabled status)
-curl -s https://$DEPLOY_DOMAIN/health | jq
-
-# Clean up Docker resources (if disk full)
-ssh $DEPLOY_HOST 'docker system prune -af && docker builder prune -af'
-```
-
-### Local Docker Development
-For testing the full production stack locally:
-
-```bash
-# Build and start all containers
-docker-compose -f docker-compose.prod.yml up --build
-
-# Access locally
-# Frontend: http://localhost:3000
-# Backend API: http://localhost:3010
-# WhatsApp RPC: http://localhost:9400
-# Redis: localhost:6379
-
-# Stop all containers
-docker-compose -f docker-compose.prod.yml down
-
-# Remove volumes (clean slate)
-docker-compose -f docker-compose.prod.yml down -v
-```
-
-### Local Development Build
-```bash
-# Create optimized build
-npm run build
-
-# Serve built files locally
-npm run preview
-```
+The historical Docker Compose topology (4-container stack: redis / backend / frontend / whatsapp, nginx reverse proxy, dev + prod compose files, env config, resource usage, useful commands) and the `npm run build` / `npm run preview` local-build commands are preserved in **[Deployment (legacy reference)](./docs-internal/deployment_legacy.md)**.
## Authentication System
-### Overview
-n8n-inspired authentication system with JWT tokens stored in HttpOnly cookies. Authentication can be completely disabled for development or supports two deployment modes for different use cases.
-
-### Authentication Toggle
-| Setting | Environment Variable | Description |
-|---------|---------------------|-------------|
-| **Enabled** | `VITE_AUTH_ENABLED=true` | Require login (default) |
-| **Disabled** | `VITE_AUTH_ENABLED=false` | Bypass authentication, anonymous access |
-
-When `VITE_AUTH_ENABLED=false`:
-- Frontend skips login page entirely
-- User is set to anonymous with owner privileges
-- No backend auth API calls are made
-- Useful for local development and testing
-
-### Deployment Modes (when auth enabled)
-| Mode | Environment Variable | Description |
-|------|---------------------|-------------|
-| **Single Owner** | `AUTH_MODE=single` | First user becomes owner, registration disabled after |
-| **Multi User** | `AUTH_MODE=multi` | Open registration for cloud deployments |
-
-### Architecture
-```
-Frontend (LoginPage.tsx) → AuthContext → Backend (/api/auth/*) → JWT Cookie
- ↓
- AuthMiddleware
- ↓
- Protected Routes
-```
-
-### Backend Implementation
-
-#### User Model (`server/models/auth.py`)
-```python
-class User(SQLModel, table=True):
- __tablename__ = "users"
- id: Optional[int] = Field(default=None, primary_key=True)
- email: str = Field(unique=True, index=True)
- password_hash: str
- display_name: str
- is_owner: bool = Field(default=False)
- is_active: bool = Field(default=True)
- created_at: datetime
- last_login: Optional[datetime]
-
- def set_password(self, password: str) -> None:
- # Uses bcrypt for secure hashing
-
- def verify_password(self, password: str) -> bool:
- # Verifies against bcrypt hash
-```
-
-#### Auth Service (`server/services/user_auth.py`)
-- `register_user()` - Creates new user, sets as owner if first user in single mode
-- `authenticate_user()` - Validates credentials, returns user
-- `create_token()` - Generates JWT token
-- `verify_token()` - Validates JWT token
-- `get_auth_status()` - Returns mode, registration availability, user count
-
-#### Auth Router (`server/routers/auth.py`)
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/api/auth/status` | GET | Get auth mode and registration status |
-| `/api/auth/register` | POST | Register new user |
-| `/api/auth/login` | POST | Login and set cookie |
-| `/api/auth/logout` | POST | Clear auth cookie |
-| `/api/auth/me` | GET | Get current user info |
-
-#### Auth Middleware (`server/middleware/auth.py`)
-Protects all routes except public paths:
-```python
-PUBLIC_PATHS = frozenset([
- "/health", "/docs", "/openapi.json", "/redoc",
- "/api/auth/status", "/api/auth/login", "/api/auth/register", "/api/auth/logout",
-])
-PUBLIC_PREFIXES = ("/webhook/",)
-```
-
-### Frontend Implementation
-
-#### Auth Context (`client/src/contexts/AuthContext.tsx`)
-```typescript
-interface AuthContextValue {
- user: User | null;
- isAuthenticated: boolean;
- isLoading: boolean;
- error: string | null;
- authMode: 'single' | 'multi';
- canRegister: boolean;
- login: (email: string, password: string) => Promise<boolean>;
- register: (email: string, password: string, displayName: string) => Promise<boolean>;
- logout: () => Promise<void>;
- checkAuth: () => Promise<void>;
-}
-```
-
-#### Protected Route (`client/src/components/auth/ProtectedRoute.tsx`)
-Wraps protected content, shows LoginPage if not authenticated:
-```typescript
-const ProtectedRoute: React.FC<{ children: React.ReactNode }> = ({ children }) => {
- const { isAuthenticated, isLoading } = useAuth();
- if (isLoading) return <LoadingSpinner />;
- if (!isAuthenticated) return <LoginPage />;
- return <>{children}</>;
-};
-```
+n8n-style JWT auth in HttpOnly cookies. `VITE_AUTH_ENABLED=false` bypasses login (anonymous owner, dev); when enabled, `AUTH_MODE=single` (first user = owner, registration then closed) or `multi` (open registration). Backend: `User` (bcrypt) + `UserAuthService` + `/api/auth/*` router + `AuthMiddleware` (public-path allowlist + `/webhook/` prefix). Frontend: `AuthContext` (TanStack-Query bootstrap with full-jitter backoff) + `ProtectedRoute` + `LoginPage`; all API calls send `credentials: 'include'`, and the WebSocket refuses to connect without the cookie.
-#### Login Page (`client/src/components/auth/LoginPage.tsx`)
-- Dracula-themed login/register form
-- Switches between login and register based on `canRegister`
-- Displays errors from auth context
-
-### Configuration
-Environment variables in `.env`:
-```bash
-# Authentication Toggle (frontend - Vite)
-VITE_AUTH_ENABLED=true # 'true' or 'false' - disable to bypass login
-
-# Authentication Mode (backend)
-AUTH_MODE=single # 'single' or 'multi'
-JWT_SECRET_KEY=your-secret-key-32 # Min 32 chars
-JWT_EXPIRE_MINUTES=10080 # 7 days
-JWT_COOKIE_NAME=machina_token
-JWT_COOKIE_SECURE=false # true for HTTPS
-JWT_COOKIE_SAMESITE=lax
-```
-
-### Race Condition Handling
-The AuthContext includes retry logic with exponential backoff to handle the case where frontend starts before backend is ready:
-- 5 retries with exponential backoff (1s, 2s, 4s, 8s, 16s)
-- Shows "Failed to connect to server" only after all retries exhausted
-- Prevents false "not authenticated" errors during startup
-
-### Cookie-Based Auth for API Calls
-All API calls must include `credentials: 'include'` for HttpOnly cookie:
-```typescript
-// In workflowApi.ts, all fetch calls include:
-fetch(url, { credentials: 'include' })
-```
-
-### WebSocket Authentication
-WebSocket checks cookie before accepting connection:
-```python
-# In websocket.py
-token = websocket.cookies.get(settings.jwt_cookie_name)
-if not token:
- await websocket.close(code=4001, reason="Not authenticated")
- return
-```
-
-WebSocketProvider only connects when authenticated:
-```typescript
-// In WebSocketContext.tsx
-const { isAuthenticated, isLoading: authLoading } = useAuth();
-
-useEffect(() => {
- if (authLoading || !isAuthenticated) {
- // Disconnect if logged out
- return;
- }
- connect();
-}, [isAuthenticated, authLoading]);
-```
-
-### Key Files
-| File | Description |
-|------|-------------|
-| `client/src/config/api.ts` | API config with AUTH_ENABLED toggle |
-| `client/src/contexts/AuthContext.tsx` | React auth state with retry logic |
-| `client/src/components/auth/LoginPage.tsx` | Login UI |
-| `client/src/components/auth/ProtectedRoute.tsx` | Route guard |
-| `server/models/auth.py` | User SQLModel with bcrypt |
-| `server/services/user_auth.py` | JWT creation/verification |
-| `server/routers/auth.py` | REST endpoints |
-| `server/middleware/auth.py` | Route protection |
-| `server/core/config.py` | Settings with vite_auth_enabled field |
-
-### Dependencies
-```
-# server/pyproject.toml
-bcrypt>=4.1.0
-pyjwt>=2.13.0
-email-validator>=2.0.0
-```
-
-JWT handling uses **PyJWT** (`import jwt`, `jwt.encode` / `jwt.decode`, catch `jwt.PyJWTError`) — HS256 with `Settings.jwt_secret_key`. Do not reintroduce `python-jose`: it drags in pure-Python `ecdsa`, which carries an unpatchable Minerva timing-attack advisory (GHSA-wj6h-64fc-37mp).
+**Load-bearing rule:** JWT handling uses **PyJWT** (`import jwt`, HS256 with `Settings.jwt_secret_key`). Do NOT reintroduce `python-jose` — it drags in pure-Python `ecdsa` with the unpatchable Minerva timing-attack advisory (GHSA-wj6h-64fc-37mp). Full reference (models, router, middleware, config, startup retry, key files, deps) in **[Authentication](./docs-internal/authentication.md)**.
## Encrypted Credentials System
-### Overview
-API keys and OAuth tokens are stored in a separate encrypted database (`credentials.db`) using Fernet encryption (AES-128-CBC + HMAC-SHA256). Following the n8n pattern, the encryption key is derived from a server-scoped config key (`API_KEY_ENCRYPTION_KEY` in `.env`) using PBKDF2, initialized at startup and persisting across restarts.
-
-### Security Architecture
-```
-Server Startup
- ↓
-API_KEY_ENCRYPTION_KEY (from .env) + Salt (from credentials.db)
- ↓
-PBKDF2HMAC (SHA256, 600K iterations)
- ↓
-Fernet Key (in-memory for application lifetime)
- ↓
-EncryptionService.encrypt()/decrypt()
- ↓
-credentials.db (encrypted ciphertext)
-```
-
-**Key Security Properties:**
-- Server-scoped encryption key from `API_KEY_ENCRYPTION_KEY` in `.env` (n8n pattern)
-- Key initialized at startup, persists across application lifetime
-- Not tied to user sessions -- survives server restarts with valid JWT
-- Salt stored in credentials database (not the main database)
-- OWASP 2024 compliant: 600,000 PBKDF2 iterations
-
-### Single Point of Access Pattern
-**IMPORTANT**: All credential operations MUST go through `AuthService`. Routers should NEVER access `CredentialsDatabase` directly.
-
-```python
-# Correct: Use auth_service
-auth_service = get_auth_service()
-await auth_service.store_oauth_tokens(provider="google", ...)
-tokens = await auth_service.get_oauth_tokens("google", customer_id="owner")
-
-# Wrong: Direct database access
-credentials_db = get_credentials_db() # Don't do this in routers
-await credentials_db.save_oauth_tokens(...) # Don't do this
-```
-
-### Backend Implementation
-
-#### EncryptionService (`server/core/encryption.py`)
-```python
-class EncryptionService:
- """Fernet encryption with PBKDF2 key derivation."""
-
- def initialize(self, password: str, salt: bytes) -> None:
- """Derive key from password using PBKDF2HMAC."""
- kdf = PBKDF2HMAC(
- algorithm=hashes.SHA256(),
- length=32,
- salt=salt,
- iterations=600_000, # OWASP 2024 recommendation
- )
- key = base64.urlsafe_b64encode(kdf.derive(password.encode()))
- self._fernet = Fernet(key)
-
- def encrypt(self, plaintext: str) -> str:
- """Encrypt and return base64 ciphertext."""
-
- def decrypt(self, ciphertext: str) -> str:
- """Decrypt base64 ciphertext."""
-
- def clear(self) -> None:
- """Clear encryption key from memory."""
-
- def is_initialized(self) -> bool:
- """Check if encryption is ready."""
-```
-
-#### CredentialsDatabase (`server/core/credentials_database.py`)
-```python
-class CredentialsDatabase:
- """Async SQLite database for encrypted credentials."""
-
- async def initialize(self) -> bytes:
- """Initialize database, create tables, return salt."""
-
- async def save_api_key(self, provider: str, key: str, metadata: Dict = None) -> bool
- async def get_api_key(self, provider: str) -> Optional[str]
- async def delete_api_key(self, provider: str) -> bool
-
- async def save_oauth_tokens(self, provider, access_token, refresh_token, ...) -> bool
- async def get_oauth_tokens(self, provider, customer_id="owner") -> Optional[Dict]
- async def delete_oauth_tokens(self, provider, customer_id="owner") -> bool
-```
-
-#### AuthService OAuth Methods (`server/services/auth.py`)
-```python
-class AuthService:
- """Single point of access for all credentials."""
-
- # Memory-only cache for decrypted credentials
- _api_key_cache: Dict[str, str] = {}
- _oauth_cache: Dict[str, Dict[str, Any]] = {}
+API keys and OAuth tokens live in a separate encrypted database (`credentials.db`): Fernet (AES-128-CBC + HMAC-SHA256) with the key derived via PBKDF2HMAC-SHA256 (600K iterations, OWASP-2024) from the server-scoped `API_KEY_ENCRYPTION_KEY` (`.env`) + a salt stored in `credentials.db`, initialized at startup and held for the process lifetime. Two distinct systems that never cross: API keys (`store_api_key`/`get_api_key` → `EncryptedAPIKey`) and OAuth tokens (`store_oauth_tokens`/`get_oauth_tokens` → `EncryptedOAuthToken`). Multi-backend (Fernet default / Keyring / AWS Secrets Manager) via `CREDENTIAL_BACKEND`.
- async def store_api_key(self, provider: str, key: str) -> bool
- async def get_api_key(self, provider: str) -> Optional[str]
- async def delete_api_key(self, provider: str) -> bool
-
- async def store_oauth_tokens(self, provider, access_token, refresh_token, ...) -> bool
- async def get_oauth_tokens(self, provider, customer_id="owner") -> Optional[Dict]
- async def remove_oauth_tokens(self, provider, customer_id="owner") -> bool
-
- def clear_cache(self) -> None:
- """Clear all cached credentials."""
-```
-
-#### UserAuthService Integration (`server/services/user_auth.py`)
-```python
-async def login(self, email: str, password: str):
- # ... verify credentials ...
- # Initialize encryption with user's password
- await self._initialize_encryption(password)
- return user, None
-
-async def _initialize_encryption(self, password: str) -> None:
- salt = await self.credentials_db.initialize()
- self.encryption.initialize(password, salt)
-
-def logout(self) -> None:
- self.encryption.clear() # Clear key from memory
-```
-
-### Multi-Backend Support
-
-For deployment flexibility, credentials can be stored in different backends:
-
-#### Backend Options
-| Backend | Use Case | Configuration |
-|---------|----------|---------------|
-| **Fernet** (default) | Local development, single-server | `CREDENTIAL_BACKEND=fernet` |
-| **Keyring** | Desktop apps (OS-native storage) | `CREDENTIAL_BACKEND=keyring` |
-| **AWS Secrets Manager** | Cloud deployments | `CREDENTIAL_BACKEND=aws` |
-
-#### credential_backends.py
-```python
-class CredentialBackend(ABC):
- async def store(self, key: str, value: str, metadata: Dict = None) -> bool
- async def retrieve(self, key: str) -> Optional[str]
- async def delete(self, key: str) -> bool
- def is_available(self) -> bool
-
-class FernetBackend(CredentialBackend):
- """Default: Fernet-encrypted SQLite."""
-
-class KeyringBackend(CredentialBackend):
- """OS-native: Windows Credential Locker, macOS Keychain, Linux Secret Service."""
- SERVICE_NAME = "MachinaOS"
-
-class AWSSecretsBackend(CredentialBackend):
- """AWS Secrets Manager for cloud deployments."""
-
-def create_backend(settings, credentials_db=None) -> CredentialBackend:
- """Factory with automatic fallback to Fernet."""
-```
-
-### Configuration
-
-Environment variables in `.env`:
-```bash
-# Credentials Database
-CREDENTIALS_DB_PATH=credentials.db
-
-# Backend Selection
-CREDENTIAL_BACKEND=fernet # fernet, keyring, or aws
-
-# AWS Secrets Manager (when CREDENTIAL_BACKEND=aws)
-AWS_SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789:secret:machinaos-creds
-AWS_REGION=us-east-1
-```
-
-### Key Files
-| File | Description |
-|------|-------------|
-| `server/core/encryption.py` | Fernet encryption with PBKDF2 key derivation |
-| `server/core/credentials_database.py` | Async SQLite for encrypted credentials |
-| `server/core/credential_backends.py` | Multi-backend abstraction (Fernet, Keyring, AWS) |
-| `server/services/auth.py` | AuthService with OAuth methods (single point of access) |
-| `server/services/user_auth.py` | Encryption initialization on login/logout |
-| `server/core/config.py` | credential_backend, aws_secret_arn settings |
-
-### Dependencies
-```toml
-# server/pyproject.toml
-[project]
-dependencies = [
- "cryptography>=44.0.0", # Fernet encryption
-]
-
-[project.optional-dependencies]
-keyring = ["keyring>=25.0.0"] # OS-native credential storage
-aws = ["boto3>=1.34.0"] # AWS Secrets Manager
-```
-
-### Design Decisions
-- **No Migration**: Users re-enter API keys after upgrade (simpler, more secure)
-- **Memory-Only Cache**: Decrypted credentials never written to disk/Redis
-- **Separate Database**: `credentials.db` isolated from main `machina.db`
-- **Password-Derived Key**: Encryption key not stored anywhere
-- **Single Point of Access**: AuthService prevents direct database access from routers
+**Load-bearing rule:** every credential operation MUST go through `AuthService` (`from core.container import container; container.auth_service()`); routers must NEVER touch `CredentialsDatabase` directly. Full pipeline, cache contract, backends, config, key files, and design decisions are in **[Credentials Encryption](./docs-internal/credentials_encryption.md)**.
## Example Workflows
@@ -3407,507 +1311,23 @@ if "examples_loaded" not in columns:
## Onboarding Service
-### Overview
-Multi-step welcome wizard that appears after first launch, guiding users through platform capabilities. Database-backed, skippable, resumable, and replayable from Settings.
-
-See **[Onboarding Service](./docs-internal/onboarding.md)** for full documentation.
-
-### Architecture
-- **5-step wizard** using existing `Modal` component (autoHeight, maxWidth 580px, maxHeight 70vh) + Ant Design `Steps`, `Card`, `Button`, `Typography`, `Tag`
-- **Database persistence** via `UserSettings.onboarding_completed` + `UserSettings.onboarding_step`
-- **No new WebSocket handlers** -- reuses `get_user_settings` / `save_user_settings`
-- **Existing users** auto-skip via migration (`examples_loaded=1` -> `onboarding_completed=1`)
-
-### Steps
-
-| Step | Component | Title | Purpose |
-|------|-----------|-------|---------|
-| 0 | `WelcomeStep` | Welcome to MachinaOs | Platform intro + feature highlights |
-| 1 | `ConceptsStep` | Key Concepts | Nodes, Edges, Agents, Skills, Normal/Dev Mode |
-| 2 | `ApiKeyStep` | API Key Setup | Provider list + "Open Credentials" button |
-| 3 | `CanvasStep` | Canvas Tour | Visual UI layout diagram + keyboard shortcuts |
-| 4 | `GetStartedStep` | Get Started | Example workflows, quick recipe, tips |
-
-### Key Files
-| File | Description |
-|------|-------------|
-| `client/src/hooks/useOnboarding.ts` | State hook with WebSocket persistence |
-| `client/src/components/onboarding/OnboardingWizard.tsx` | Main wizard with Ant Design Steps |
-| `client/src/components/onboarding/steps/*.tsx` | 5 step components using antd + @ant-design/icons |
-| `client/src/Dashboard.tsx` | Renders wizard + passes `reopenTrigger` |
-| `client/src/components/ui/SettingsPanel.tsx` | "Replay Welcome Guide" button in Help section |
-| `server/models/database.py` | `UserSettings.onboarding_completed`, `onboarding_step` |
-| `server/core/database.py` | Migration + CRUD for onboarding fields |
-
-### Replay from Settings
-- SettingsPanel has a "Replay Welcome Guide" button in the Help section
-- Clicking it: closes Settings, increments `onboardingReopenTrigger` in Dashboard
-- `useOnboarding` detects trigger change, resets state, reopens wizard from step 0
-
-### Adding New Steps
-1. Create `client/src/components/onboarding/steps/NewStep.tsx` using Ant Design components
-2. Import in `OnboardingWizard.tsx`, add to `renderStep()` switch and `stepItems` array
-3. Update `TOTAL_STEPS` in `useOnboarding.ts`
+Multi-step welcome wizard shown on first launch — database-backed (`UserSettings.onboarding_completed` + `onboarding_step`), skippable, resumable, replayable from Settings, and auto-skipped for existing users (migration sets `onboarding_completed=1` where `examples_loaded=1`). Built with shadcn primitives (no antd). Full step list, key files, replay flow, and how to add a step are in **[Onboarding Service](./docs-internal/onboarding.md)**.
## AI Chat Model Development Guide
-### Adding a new AI provider (post-Wave-11)
-
-A new chat-model provider is **a self-contained folder** under `server/nodes/model/<provider>_chat_model/` with `__init__.py` declaring a `ChatModelBase` subclass. The plugin auto-registers via `BaseNode.__init_subclass__`; the frontend renders it through `SquareNode` from the emitted NodeSpec without a single TS change.
-
-```python
-# server/nodes/model/openrouter_chat_model/__init__.py
-class OpenRouterChatModel(ChatModelBase):
- type = "openrouterChatModel"
- metadata = NodeMetadata(
- display_name="OpenRouter",
- icon="lobehub:openrouter", # asset:<key>, lobehub:<brand>, or emoji
- color="#6366F1",
- component_kind="model", # routes to SquareNode in Dashboard.tsx
- )
-
- class Params(ChatModelBase.Params):
- # provider-specific overrides; everything else inherits from ChatModelBase
- ...
-```
-
-The native chat path lives in `server/services/llm/providers/<provider>.py` (Protocol-based, see [Native LLM SDK](./docs-internal/native_llm_sdk.md)). The LangChain agent path uses `ChatOpenAI` with a custom `base_url` from `server/config/llm_defaults.json` -- no Python branching needed for OpenAI-compatible APIs.
-
-Credentials live in `server/nodes/model/_credentials.py` (one `Credential` subclass per provider) and surface in the Credentials Modal automatically. There is no `nodeDefinitions/` to edit, no `ModelNode.tsx` to update, no `Dashboard.tsx` switch to extend.
-
-### Key implementation files
-
-| File | Purpose |
-|---|---|
-| `server/nodes/model/<provider>_chat_model.py` | Plugin entry — metadata + Params + auto-registers |
-| `server/nodes/model/_credentials.py` | `Credential` subclass per provider |
-| `server/config/llm_defaults.json` | base_url + supported_params + temperature constraints (no hardcoded URLs in Python) |
-| `server/services/llm/providers/<provider>.py` | Native SDK provider (Protocol-based) |
-| `server/services/ai.py` | Routing: native chat path + LangChain agent path |
-| `client/src/Dashboard.tsx` | Generic `COMPONENT_BY_KIND` dispatch — no per-provider entry needed |
+A new chat-model provider is one self-contained folder under `server/nodes/model/<provider>_chat_model/` with `__init__.py` declaring a `ChatModelBase` subclass (auto-registers via `BaseNode.__init_subclass__`; the frontend renders it through `SquareNode` from the NodeSpec with zero TS changes). Credentials live in `server/nodes/model/_credentials.py`. The full recipe — native chat path (`services/llm/providers/`), LangChain agent path, `llm_defaults.json` config, the dual-path routing, and the per-provider implementation-file map — is in **[Native LLM SDK](./docs-internal/native_llm_sdk.md)**.
## Simple Memory System
-### Overview
-The Simple Memory node provides markdown-based conversation history storage for AI agents. It connects to the AI Agent's `input-memory` handle to provide context from previous conversations. Memory is stored in markdown format, visible and editable directly in the parameter panel, with optional long-term vector storage for semantic retrieval.
-
-### Architecture
-```
-Simple Memory Node → (memory output) → AI Agent (input-memory handle)
- ↓ ↓
- Markdown editor Parses markdown, saves new exchanges
- (visible in UI) Trims to window, archives to vector DB
-```
-
-### Memory Format
-Conversation history is stored in markdown format with timestamps:
-```markdown
-# Conversation History
-
-### **Human** (2025-01-30 14:23:45)
-What is the weather like today?
-
-### **Assistant** (2025-01-30 14:23:48)
-I don't have access to real-time weather data...
-```
-
-### Key Features
-- **Editable UI**: Conversation history visible in markdown editor in parameter panel
-- **Window-Based Trimming**: Keeps last N message pairs, archives old messages
-- **Long-Term Memory**: Optional vector DB storage for semantic retrieval of archived messages
-- **Uses LangChain's InMemoryVectorStore**: Per-session vector stores with HuggingFaceEmbeddings (BAAI/bge-small-en-v1.5)
-
-### Key Files
-- **Node Definition**: `server/nodes/skill/simple_memory.py` - simpleMemory plugin (NodeSpec + execute). Carries an internal `last_session_id` field (hidden in UI) used by the claude_code_agent bridge for native session resume.
-- **Memory Package**: `server/services/memory/` — promoted from the old `services/memory.py` so each concern owns its own module:
- - `services/memory/markdown.py` — `parse_memory_markdown`, `append_to_memory_markdown`, `trim_markdown_window` (the canonical helpers every agent uses; **previously inlined into `services/ai.py`**, now imported from here)
- - `services/memory/jsonl.py` — standalone JSONL `parse_jsonl` / `append_message` / `trim_window` (Anthropic Messages API shape, kept for future SDK migration; not used by any agent bridge today)
- - `services/memory/vector_store.py` — `get_memory_vector_store` + per-session `InMemoryVectorStore` cache
- - `services/memory/state.py` — `clear_agent_session_state(session_id, workflow_id, clear_long_term, memory_node_id)`; when `memory_node_id` is supplied it resets `memory_content` to the default placeholder AND wipes `last_session_id` server-side (single DB write)
- - `services/memory/__init__.py` — re-exports the full surface so existing `from services.memory import …` callers keep working
-- **AI Integration (aiAgent / chatAgent / rlm_agent)**: `server/services/ai.py` — `execute_agent` / `execute_chat_agent` consume `memory_data`, call the markdown helpers via the package re-exports
-- **CLI bridge (`claude_code_agent`)**: `server/services/cli_agent/service.py:_persist_memory` — appends turns to `memory_content` via the same markdown helpers AND saves `simpleMemory.last_session_id` from the most recent successful run's `r.session_id`. Always broadcasts `node_parameters_updated` so the simpleMemory parameter panel refetches live (no page reload needed).
-- **Edge walker**: `server/services/plugin/edge_walker.py:_build_memory_entry` — surfaces `memory_content` + `window_size` + `long_term_enabled` + `retrieval_count` + `last_session_id` to every consuming agent
-
-### Node Properties
-| Property | Type | Default | Description |
-|----------|------|---------|-------------|
-| sessionId | string | 'default' | Unique identifier for conversation session |
-| windowSize | number | 10 | Number of message pairs to keep in short-term memory |
-| memoryContent | string (markdown) | Initial template | Editable conversation history in markdown format |
-| longTermEnabled | boolean | false | Archive old messages to vector DB for semantic retrieval |
-| retrievalCount | number | 3 | Number of relevant memories to retrieve from long-term storage (shown when longTermEnabled=true) |
-
-### Memory Flow
-1. **Load**: AI Agent reads `memoryContent` markdown from connected Simple Memory node
-2. **Parse**: `_parse_memory_markdown()` converts markdown to LangChain messages
-3. **Retrieve** (if enabled): Semantic search of vector store for relevant context
-4. **Execute**: AI processes prompt with conversation history
-5. **Append**: New human/AI messages appended to markdown
-6. **Trim**: `_trim_markdown_window()` keeps last N pairs, returns removed texts
-7. **Archive**: Removed messages stored in InMemoryVectorStore (if longTermEnabled)
-8. **Save**: Updated markdown saved back to node parameters
-
-### Usage Flow
-1. Add Simple Memory node to workflow
-2. Connect its output to AI Agent's `input-memory` handle (bottom-left diamond)
-3. Configure session ID (use different IDs for separate conversations)
-4. Optionally enable long-term memory for semantic retrieval
-5. Run AI Agent - it automatically reads/updates memory content
-6. View/edit conversation history in the markdown editor
-
-### Design Decisions
-- **Passive Node**: Memory node has no Run button - AI Agent reads its configuration directly when executed
-- **Markdown Storage**: Human-readable format, editable in UI, easy to debug
-- **Per-Session Vector Stores**: `_memory_vector_stores` dict keyed by session_id
-- **Window + Archive**: Short-term (recent messages) + long-term (semantic retrieval) memory pattern
-- **Auto-Save**: AI Agent automatically saves updated markdown content to node parameters
-
-### claude_code_agent bridge — `--continue` + warm subprocess pool (NOT markdown injection)
-
-When `simpleMemory` is connected to `claude_code_agent`, memory continuity is handled by claude itself — the agent does NOT re-inject markdown as a system prompt. Claude stores per-session transcripts under `<CLAUDE_CONFIG_DIR>/projects/<project_key>/<session_id>.jsonl` where `project_key = re.sub(r"[^a-zA-Z0-9.-]", "-", str(cwd))` (verified against the on-disk `<DATA_DIR>/claude/projects/` listing; `CLAUDE_CONFIG_DIR = data_path("claude")`).
-
-**The three coupled mechanisms** (see [docs-internal/cli_agent_framework.md → Memory bridge](./docs-internal/cli_agent_framework.md#memory-bridge--simplememory--claude_code_agent) for full plumbing):
-
-1. **Stable cwd.** The pool keeps `cwd = repo_root` for every memory-bound spawn (no per-task worktree carving). Same cwd every run → claude's `project_key` is constant → `--continue` / `--resume <UUID>` resolve against the same JSONL across batches.
-2. **`--continue` first, intra-process thereafter.** [`claude_code_agent.execute_op`](./server/nodes/agent/claude_code_agent/__init__.py) sets `spec.continue_session = bool(memory_data)` so the first cold spawn for a memory-wired run emits `--continue` (claude auto-loads the most recent conversation under the cwd's `project_key`, per [code.claude.com/docs/en/cli-reference](https://code.claude.com/docs/en/cli-reference)). Subsequent turns are written as stream-json to the SAME warm subprocess's `proc.stdin` — claude keeps the conversation in-process with the same `session_id` (verified: turn 1 + turn 2 share UUID within one process).
-3. **`--resume <UUID>` crash recovery.** If the warm subprocess dies between batches, [`ClaudeSessionPool.acquire`](./server/nodes/agent/claude_code_agent/_pool.py) detects `process.returncode is not None`, captures the dead session's `current_session_uuid` (last seen on a `result` event), and respawns with `--resume <captured_uuid>` so the same on-disk JSONL keeps growing. Mutually exclusive with `--continue` in `nodes/agent/claude_code_agent/_provider.py:interactive_argv`.
-
-**`memory_content` is the UI mirror, not the resume channel.** Claude reads its own JSONL via `--continue` / `--resume`. The markdown in `memory_content` is appended every successful run (same `append_to_memory_markdown` + `trim_markdown_window` helpers aiAgent uses) so the human reader sees the conversation. User edits to `memory_content` do not influence claude's next response.
-
-**`simpleMemory.last_session_id` is display-only.** Pre-cutover code persisted the session UUID back to the memory node and emitted `--session-id <UUID5>` / `--resume <UUID>` on subsequent runs. That dance is gone — `claude_code_agent` no longer reads `last_session_id` from the memory data. The field is kept on the Pydantic params for back-compat and diagnostic display; clearing the memory wipes it.
-
-**Parallel-batch guard.** When memory is wired AND `len(tasks) > 1`, `claude_code_agent` raises `NodeUserError` at handler entry — concurrent `--continue` spawns against one project_key would race claude's session-resolution.
-
-**Live UI refresh — CloudEvents v1.0 envelope.** `_persist_memory` calls `broadcaster.broadcast_node_parameters_updated(...)` after `save_node_parameters`. The broadcast carries a `WorkflowEvent` envelope (`type: "com.machinaos.node.parameters.updated"`, `subject: <memory_node_id>`, `data: {node_id, parameters, version, source: "cli"}`) wrapped in the wire-key `{type: "node_parameters_updated", data: <envelope>}`. The FE handler in `client/src/contexts/WebSocketContext.tsx` casts to `WorkflowEvent<{node_id, parameters, version, source}>`, reads `inner.node_id || envelope.subject`, and routes the inner payload to `setNodeParameters` + `queryClient.setQueryData` so the simpleMemory parameter panel auto-refreshes the moment the turn completes. Pre-cutover the FE read the old flat top-level keys (`message.parameters`, `message.node_id`) and silently dropped every broadcast — symptom was "memory only updates after a page reload."
-
-**Workspace dir routed via `--add-dir`.** The per-workflow workspace (`~/.machina/workspaces/<workflow_slug>/`, injected into `ctx.raw` by `workflow.py:_get_workspace_dir`) is spliced into each task's `add_dir` list in `AICliService.run_batch` so the spawned claude can read files dropped by upstream nodes (`fileDownloader`, `documentParser`, code executors). Without this the workspace is invisible: memory-bound runs spawn with `cwd=repo_root` (stable for `--continue`'s `project_key`) and non-memory runs with `cwd=worktree`, neither of which sees the workspace files. Mirrors the ai_agent pattern (`services/ai.py:1899` — `config['workspace_dir'] = context.get('workspace_dir', '')`) but uses claude's native `--add-dir` instead of MCP-tool-config injection because claude has its own filesystem tools.
-
-**Stale-model coercion.** `ClaudeCodeAgentParams.model` is a `Literal` of supported model IDs + aliases — strict validation would reject legacy saved values like `"claude-sonnet-4.6"` (dot-spelled), `"claude-3-5-sonnet-20241022"` (date-suffixed), `""`. A `field_validator("model", "fallback_model", mode="before")` coerces unknown values to the default (`"claude-sonnet-4-6"` for `model`, `None` for `fallback_model`) so old workflows keep loading; the UI dropdown still constrains new edits.
-
-**Strict tool allowlist — no claude built-ins (except conditional `Skill`), gated by `--permission-mode dontAsk`.** The spawn argv emits `--permission-mode dontAsk` (documented as *"only pre-approved tools, no prompts"* per [code.claude.com/docs/en/permission-modes](https://code.claude.com/docs/en/permission-modes)) so `--allowedTools` is actually enforced. The previous default `bypassPermissions` *"skips the permission layer entirely"* — same doc — which turned the allowlist into documentation-only and let claude's built-in `Read` / `Edit` / `Bash` / `Glob` / `Grep` / `Write` / `Skill` / `WebSearch` / `WebFetch` fire regardless of wiring. The new `--allowedTools` list contains exactly: (a) one `mcp__machinaos__<node_type>` per node wired through the agent's `input-tools` handle, (b) the claude built-in `Skill` **conditionally** — only when at least one skill is wired through `input-skill` (paired with the materialisation helper below; never enabled when no skill is connected), (c) the five MachinaOs MCP infrastructure tools (`getWorkspaceFiles`, `listSkills`, `getSkill`, `getCredential`, `broadcastLog`). Every filesystem / shell / web operation flows through wired MCP tools (`fileRead` / `fileModify` / `fsSearch` / `shell` / `browser` / `perplexitySearch`) — explicit wiring, no built-in escape hatches. Defaults live in [`server/config/ai_cli_providers.json`](./server/config/ai_cli_providers.json) (`default_allowed_tools: ""`, `default_permission_mode: "dontAsk"`); per-task override via `ClaudeTaskSpec.allowed_tools` is honored verbatim (no auto-merge).
-
-**Skill materialisation — per-workflow workspace, live-watched, diff-based.** [`nodes/agent/claude_code_agent/_skills.py::materialise_skills`](./server/nodes/agent/claude_code_agent/_skills.py) writes connected-AND-enabled SKILL.md trees under `<workspace_dir>/.claude/skills/<name>/` where `workspace_dir = <DATA_DIR>/workspaces/<workflow_slug>/`. The workspace is passed via `--add-dir`, and claude scans `.claude/skills/` inside every `--add-dir` path per [the skills spec's "Automatic discovery from parent and nested directories" rule](https://code.claude.com/docs/en/skills#automatic-discovery-from-parent-and-nested-directories) — gives us per-workflow isolation (workflow A's skills can't bleed into workflow B's subprocess even when both spawn with `cwd=repo_root`). Pool tracks the materialised set in `PooledClaudeSession.materialised_skills: frozenset[str]`; on warm reuse, `acquire` calls `materialise_skills(workspace_dir, new_set, previous_skill_names=session.materialised_skills)` and the helper applies only the diff — `rmtree`'d skills disappear from claude's registry, new ones become invocable, all without respawning. Claude's [live filesystem watcher](https://code.claude.com/docs/en/skills#live-change-detection) picks up both edge events. Same helper drives both pool path and `AICliSession._pre_spawn` (non-pool) — uniform behaviour across transports. **Migration note**: pre-cutover the pool path wrote into `<repo_root>/.claude/skills/` so SKILL.md trees accumulated in the user's actual repo (gitignored but visible). Run `rm -rf .claude/skills/` once from the repo root to clean accumulated trees; future runs only write into per-workflow workspaces. We deliberately do NOT auto-prune because the repo's `.claude/` may contain user-authored skills outside the MachinaOs registry. **Architecture rationale**: surveyed alternatives — MCP `resources`/`prompts` require `@mention`/`/command` (not auto-loaded), hooks fire after skill discovery, `settings.json` doesn't expose per-session paths. Anthropic's own [Agent SDK Skills](https://code.claude.com/docs/en/agent-sdk/skills) uses the same filesystem-first pattern. Workspace-dir + live-watch is canonical.
-
-**Warm-subprocess tool-surface rebind — no respawn cost.** Claude bakes the MCP bearer token into argv (`--mcp-config`) at spawn time and can't rotate without respawning. The pool stashes the spawn-time token on `PooledClaudeSession.batch_token` so subsequent batches can rebind the same persistent `BatchContext` in place. On warm reuse, `ClaudeSessionPool.acquire` calls [`rebind_batch(existing.batch_token, connected_tools=new_ctx.connected_tools, ...)`](./server/services/cli_agent/mcp_server.py) — diffs old vs new tool lists, calls `unexpose_workflow_tools(removed)` to decrement FastMCP refcounts (so a disconnected `duckduckgoSearch` falls off the tool list when its refcount hits zero), `expose_workflow_tools(added)` for newly-wired ones, then unregisters the redundant new token. `_terminate_locked` calls `unregister_batch(session.batch_token)` so refcounts drain on pool eviction / `clear` / `shutdown_all` (closes the long-standing leak where every pooled run permanently incremented every wired tool's refcount). The per-handler scope check in `workflow_tools._build_handler` reads `ctx.connected_tools` at call time, so even if claude's frozen `--allowedTools` argv still references a now-disconnected tool, the handler returns `{"error": "...not connected to this batch", "status": 403}` — double-lock against tool leakage across warm-reuse turns.
-
-**Logs to grep on a live run:**
-- `[Claude Code memory] memory_node=<id> -> --continue (claude auto-finds latest session under cwd)`
-- `[ClaudeSessionPool] spawned new session memory_node=<id> pid=<N>` (cold spawn) or `warm reuse memory_node=<id> pid=<N> uuid=<UUID>` (intra-process turn)
-- `[CC-Agent _persist_memory] saved memory_node=<id> last_session_id=<UUID> appended_turns=1 ... content_length=<N>`
-- On tool-surface change between batches: `[CC-Agent MCP rebind_batch] node=<id> wf=<id> token=<8hex>... +<added> -<removed> kept=<n> (now <total> tools)` — confirms the in-place rebind ran instead of a respawn.
-- On crash recovery: `[ClaudeSessionPool] dropping dead session memory_node=<id> ... — will respawn` followed by spawn with `--resume <UUID>` argv.
+The Simple Memory node (`simpleMemory`, plugin `server/nodes/skill/simple_memory.py`) provides markdown-based conversation history on the agent's `input-memory` handle: window-based trimming, an editable markdown view, and optional long-term vector retrieval. Memory is loaded/parsed/appended/trimmed via the helpers in `server/services/memory/`. The full lifecycle (two storage formats, the markdown helper API, vector store, the `claude_code_agent` native-session bridge, clearing/resume) is the canonical **[Memory Lifecycle](./docs-internal/memory_lifecycle.md)**; token tracking + compaction thresholds are in **[Memory Compaction](./docs-internal/memory_compaction.md)**.
## Memory Compaction, Token Tracking, and Cost Calculation
-### Overview
-The compaction service enables automatic memory compaction, token tracking, and **cost calculation** for all LLM providers. Cost is calculated using official pricing (per 1M tokens) and stored alongside token metrics. The Credentials Modal displays per-provider usage and costs.
-
-### Architecture
-```
-AI Agent Execution
- ↓
-CompactionService.track() → PricingService.calculate_cost()
- ↓ → Save token metrics + cost to DB
- ↓ → Update cumulative state + cost
- ↓ → Check if threshold exceeded
- ↓
-If needs_compaction:
- - Anthropic: Native compaction via context_management API
- - OpenAI: Native compaction via compact_threshold
- - Others: Client-side summarization fallback
- ↓
-CompactionService.record() → Save compaction event to DB
- → Reset cumulative token count
-```
-
-### Native Provider APIs
-
-**Threshold strategy:** per-session `custom_threshold` > per-user `UserSettings.compaction_ratio` × context_length > env `Settings.compaction_ratio` (`COMPACTION_RATIO`, default 0.8) × context_length > JSON `llm_defaults.json` ratio × context_length.
-
-**Anthropic SDK (tool_runner):**
-```python
-# Model-aware: threshold = compaction_ratio × model's context window
-compaction_control = svc.anthropic_config(model="claude-opus-4.6", provider="anthropic")
-# Returns: {"enabled": True, "context_token_threshold": 800000} (0.8 × 1M with default ratio)
-```
-
-**Anthropic Messages API:**
-```python
-api_config = svc.anthropic_api_config(model="claude-sonnet-4.5", provider="anthropic")
-# Returns: {"betas": ["compact-2026-01-12"], "context_management": {"edits": [...]}}
-# Threshold auto-computed from model's context window × compaction_ratio.
-```
-
-**OpenAI:**
-```python
-openai_config = svc.openai_config(model="gpt-5.2", provider="openai")
-# Returns: {"context_management": {"compact_threshold": 320000}} (0.8 × 400K with default ratio)
-```
-
-### Key Files
-| File | Description |
-|------|-------------|
-| `server/services/compaction.py` | CompactionService with model-aware thresholds, track(), record(), stats(), configure(), compact_context() methods |
-| `server/services/model_registry.py` | ModelRegistryService providing context_length for model-aware threshold computation |
-| `server/services/pricing.py` | PricingService with official pricing registry for all 6 providers |
-| `server/services/ai.py` | `_track_token_usage()` with automatic compaction triggering |
-| `server/models/database.py` | TokenUsageMetric, CompactionEvent, SessionTokenState tables (with cost fields) |
-| `server/core/database.py` | CRUD methods for token metrics, compaction events, and `get_provider_usage_summary()` |
-| `server/core/config.py` | compaction_enabled, compaction_threshold settings |
-| `server/core/container.py` | Dependency injection for compaction_service |
-| `server/main.py` | Wires AI service to compaction service at startup |
-| `server/routers/websocket.py` | get_compaction_stats, configure_compaction, get_provider_usage_summary handlers |
-| `client/src/hooks/useApiKeys.ts` | `getProviderUsageSummary()` hook method |
-| `client/src/components/CredentialsModal.tsx` | Usage & Costs collapsible section per provider |
-
-### Database Models
-
-**TokenUsageMetric** - Per-execution token usage and cost:
-```python
-class TokenUsageMetric(SQLModel, table=True):
- session_id: str # Memory session identifier
- node_id: str # Agent node ID
- provider: str # openai, anthropic, gemini, groq, cerebras, openrouter
- model: str # Model name
- input_tokens: int # Input token count
- output_tokens: int # Output token count
- total_tokens: int # Total tokens
- cache_creation_tokens: int # Anthropic cache creation
- cache_read_tokens: int # Anthropic cache read
- reasoning_tokens: int # OpenAI o-series reasoning
- # Cost fields (USD)
- input_cost: float # Cost for input tokens
- output_cost: float # Cost for output tokens
- cache_cost: float # Cost for cache tokens
- total_cost: float # Total cost
-```
-
-**SessionTokenState** - Cumulative state per session:
-```python
-class SessionTokenState(SQLModel, table=True):
- session_id: str # Unique session identifier
- cumulative_total: int # Running total tokens
- custom_threshold: int # Per-session threshold override
- compaction_count: int # Number of compactions
- last_compaction_at: datetime # Last compaction timestamp
- # Cumulative cost fields (USD)
- cumulative_input_cost: float
- cumulative_output_cost: float
- cumulative_total_cost: float
-```
-
-**CompactionEvent** - Compaction history:
-```python
-class CompactionEvent(SQLModel, table=True):
- session_id: str
- trigger_reason: str # "native" or "threshold"
- tokens_before: int
- tokens_after: int
- summary_content: str # Compacted summary (if available)
-```
-
-### CompactionService API
-
-```python
-from services.compaction import get_compaction_service
-
-svc = get_compaction_service()
-
-# Get model-aware provider config (threshold = compaction_ratio × context window)
-anthropic_cfg = svc.anthropic_config(model="claude-opus-4.6", provider="anthropic")
-# threshold: 800000 (0.8 × 1M context with default ratio)
-openai_cfg = svc.openai_config(model="gpt-5.2", provider="openai")
-# threshold: 320000 (0.8 × 400K context with default ratio)
-
-# Or override with explicit threshold
-anthropic_cfg = svc.anthropic_config(threshold=100000)
-
-# Track token usage after AI execution (threshold auto-computed from model)
-result = await svc.track(
- session_id="user-123",
- node_id="agent-1",
- provider="anthropic",
- model="claude-opus-4.6",
- usage={"input_tokens": 5000, "output_tokens": 1000, "total_tokens": 6000}
-)
-# result: {"total": 6000, "total_cost": 0.021, "threshold": 800000, "needs_compaction": False} (80% of 1M)
-
-# Record compaction event after native API handles it
-await svc.record(
- session_id="user-123",
- node_id="agent-1",
- provider="anthropic",
- model="claude-opus-4.6",
- tokens_before=805000,
- tokens_after=15000,
- summary="Compacted conversation summary..."
-)
-
-# Get session statistics (model-aware threshold when model/provider given)
-stats = await svc.stats("user-123", model="claude-opus-4.6", provider="anthropic")
-# {"session_id": "user-123", "total": 15000, "threshold": 800000, "count": 1} (80% of 1M)
-
-# Configure per-session custom threshold (overrides model-aware threshold)
-await svc.configure("user-123", threshold=50000, enabled=True)
-```
-
-### WebSocket Handlers
-
-| Handler | Description |
-|---------|-------------|
-| `get_compaction_stats` | Get token statistics for a session |
-| `configure_compaction` | Update threshold/enabled settings for a session |
-| `get_provider_usage_summary` | Get aggregated usage and cost by provider for Credentials Modal |
-
-### PricingService
-
-Calculates cost based on official pricing (USD per 1M tokens):
-
-```python
-from services.pricing import get_pricing_service
-
-pricing = get_pricing_service()
-
-# Calculate cost for token usage
-cost = pricing.calculate_cost(
- provider="anthropic",
- model="claude-3-5-sonnet",
- input_tokens=5000,
- output_tokens=1000,
- cache_read_tokens=500
-)
-# cost: {"input_cost": 0.015, "output_cost": 0.015, "cache_cost": 0.00015, "total_cost": 0.03015}
-
-# Get pricing for a model (supports partial matching)
-pricing_info = pricing.get_pricing("anthropic", "claude-3-5-sonnet-20241022")
-# pricing_info: ModelPricing(input_per_mtok=3.0, output_per_mtok=15.0, cache_read_per_mtok=0.30)
-```
-
-**Supported Providers & Pricing** (February 2026):
-| Provider | Example Models | Input $/MTok | Output $/MTok |
-|----------|---------------|--------------|---------------|
-| OpenAI | gpt-5, gpt-4o, o3 | 1.25-15.00 | 10.00-60.00 |
-| Anthropic | claude-opus-4.6, claude-sonnet-4 | 3.00-5.00 | 15.00-25.00 |
-| Gemini | gemini-2.5-pro, gemini-2.0-flash | 0.10-1.25 | 0.40-10.00 |
-| Groq | llama-3.3-70b, gpt-oss-120b | 0.05-0.59 | 0.08-0.79 |
-| Cerebras | llama-3.1-70b | 0.10-0.60 | 0.10-0.60 |
-| OpenRouter | Pass-through | Varies | Varies |
-
-### Configuration
-
-The compaction threshold + agent recursion limit are env-driven via `.env` (read by `core.config.Settings`); `server/config/llm_defaults.json` is the legacy SSOT and acts as last-resort fallback when Settings can't load:
-
-```bash
-# .env (mirrors .env.template)
-COMPACTION_ENABLED=true # Enable/disable compaction globally
-COMPACTION_RATIO=0.8 # Fraction of context window that triggers compaction (0.05-0.99)
-AGENT_RECURSION_LIMIT=200 # Agent loop hard step cap (>=1)
-```
-
-```json
-// server/config/llm_defaults.json — last-resort fallback only
-"agent": {
- "recursion_limit": 200,
- "default_temperature": 0.7,
- "compaction": { "ratio": 0.8 }
-}
-```
-
-Effective threshold = `providers.<provider>.context_length.<model>` × `compaction_ratio` (e.g. claude-sonnet-4-6 → 1M × 0.8 ≈ 838K input tokens). When model/provider are unknown, `model_registry.get_context_length` falls through to the provider's `_default` entry in the same JSON.
-
-**Threshold priority** (highest → lowest):
-1. Per-session `SessionTokenState.custom_threshold` (WebSocket override).
-2. Per-user `UserSettings.compaction_ratio` × context_length (Settings tab override).
-3. Env `Settings.compaction_ratio` × context_length.
-4. JSON `llm_defaults.json:agent.compaction.ratio` × context_length.
+Automatic memory compaction, token tracking, and cost calculation for all LLM providers. After each agent turn, `CompactionService.track()` records token metrics + cost (via `PricingService`) and, when cumulative tokens cross the threshold, summarises the transcript (native Anthropic/OpenAI context-management APIs where available, client-side summarisation otherwise). Effective threshold = `compaction_ratio` x the model's context window.
-Same precedence chain applies to `recursion_limit`: per-node `parameters.max_iterations` > `UserSettings.agent_recursion_limit` > `Settings.agent_recursion_limit` > JSON.
-
-### Client-Side Compaction
-
-When the token threshold is exceeded, the service automatically triggers compaction using the AI service to generate a structured summary:
-
-```python
-# Automatic compaction in _track_token_usage()
-if tracking.get('needs_compaction') and memory_content and api_key:
- result = await svc.compact_context(
- session_id=session_id,
- node_id=node_id,
- memory_content=memory_content, # Current conversation markdown
- provider=provider,
- api_key=api_key,
- model=model
- )
- # result: {"success": True, "summary": "# Conversation Summary (Compacted)...", "tokens_before": 105000, "tokens_after": 0}
-```
-
-**Summary Structure** (Claude Code pattern):
-```markdown
-# Conversation Summary (Compacted)
-*Generated: 2025-02-12T10:30:00Z*
-
-## Task Overview
-What the user is trying to accomplish.
-
-## Current State
-What's been completed and what's in progress.
-
-## Important Discoveries
-Key findings, decisions, or problems encountered.
-
-## Next Steps
-What needs to happen next.
-
-## Context to Preserve
-Details that must be retained for continuity.
-```
-
-### WebSocket Broadcasts
-
-Real-time updates broadcast to frontend:
-
-| Event | Description |
-|-------|-------------|
-| `token_usage_update` | After each AI execution: `{session_id, data: {total, threshold, needs_compaction}}` |
-| `compaction_starting` | Before compaction begins: `{session_id, node_id}` |
-| `compaction_completed` | After compaction: `{session_id, success, tokens_before, tokens_after, error}` |
-
-### Token Usage UI
-
-The MiddleSection displays a Token Usage panel for memory nodes with:
-- **Progress bar**: Visual tokens used vs threshold
-- **Statistics**: Current token count, threshold, compaction count
-- **Editable threshold**: Click edit icon to change per-session threshold
-
-```typescript
-// In client/src/components/parameterPanel/MiddleSection.tsx
-<Collapse.Panel header="Token Usage" key="tokenUsage">
- <Progress percent={Math.round((total / threshold) * 100)} />
- <Statistic title="Tokens Used" value={`${total} / ${threshold}`} />
- <Statistic title="Compactions" value={count} />
- <InputNumber onChange={updateThreshold} /> {/* Edit threshold */}
-</Collapse.Panel>
-```
-
-### Service Wiring
-
-The compaction service requires AI service for summarization, wired at startup:
-
-```python
-# In server/main.py
-from services.compaction import get_compaction_service
-compaction_svc = container.compaction_service()
-compaction_svc.set_ai_service(container.ai_service())
-```
+**Compaction-ratio precedence (highest->lowest): per-session `SessionTokenState.custom_threshold` > per-user `UserSettings.compaction_ratio` > env `COMPACTION_RATIO` (default 0.8) > `llm_defaults.json:agent.compaction.ratio`.**
-### Design Decisions
-- **Hybrid Approach**: Native APIs for Anthropic/OpenAI configs, client-side summarization for actual compaction
-- **5-Section Summary**: Follows Claude Code's structured summary format for continuity
-- **Automatic Triggering**: Compaction triggered in `_track_token_usage()` when threshold exceeded
-- **Per-Session State**: Each memory session has independent token tracking and thresholds
-- **Model-Aware Threshold**: `compaction_ratio` × model's context window. Default ratio 0.8 (env `COMPACTION_RATIO`, was 0.5 pre-2026.06). E.g. 800K for Claude Opus 4.8 with 1M context, 320K for GPT-5.2 with 400K context. Falls back to JSON `llm_defaults.json:agent.compaction.ratio` when Settings can't load. Per-session `custom_threshold` > per-user `UserSettings.compaction_ratio` > env Settings.
-- **Singleton Pattern**: Service accessible via `get_compaction_service()`
+Full reference — the `CompactionService` API, native vs client-side compaction, the 5-section summary format, DB models (`TokenUsageMetric` / `SessionTokenState` / `CompactionEvent`), WS handlers, and the broadcast events — in **[Memory Compaction](./docs-internal/memory_compaction.md)**; per-service API cost tracking is in **[Pricing Service](./docs-internal/pricing_service.md)**.
## API Cost Tracking
@@ -5306,7 +2726,7 @@ This function:
- **Workflow Export/Import with Parameters**: Exported workflow JSON includes a `nodeParameters` field containing all node configuration (provider, model, prompt, skillsConfig, etc.) fetched from the database at export time. On import, embedded `nodeParameters` are saved back to the database. `sanitizeNodes()` in `workflowExport.ts` still strips `node.data` to UI-only fields (`label`, `disabled`, `condition`). A `parameterSanitizer.ts` utility exists for credential stripping but is currently disabled (pass-through). Old exports without `nodeParameters` import cleanly (backward compatible). Key files: `client/src/utils/workflowExport.ts`, `client/src/utils/parameterSanitizer.ts`, `client/src/Dashboard.tsx` (export/import handlers), `server/services/example_loader.py`.
- **Skill System Architecture**: Skills organized in `server/skills/<folder>/` subfolders. Each folder appears in Master Skill dropdown. DB is source of truth for skill instructions (seeded from SKILL.md on first load). Icon resolution mirrors `BaseNode._metadata_dict`: per-plugin `<plugin>/icon.svg` (served as `/api/schemas/nodes/<type>/icon`) → `visuals.json` (emoji / `lobehub:<brand>`); color: per-plugin `<plugin>/meta.json` → `visuals.json`. Lives in [`skill_loader.py::_parse_skill_metadata`](./server/services/skill_loader.py). Native DOM keydown handler prevents React Flow from intercepting Ctrl shortcuts in skill editor. **Mutation broadcasts use the CloudEvents-typed `skill_lifecycle` wire key** with stages `created` / `updated` / `deleted` / `content_saved` — see "Plugin-folder location for plugin-specific CloudEvents factories" below.
- **Example Workflows**: Auto-load example workflow seeds from `<repo>/.machina/workflows/` on first use (git-tracked; the only non-ignored content under `<repo>/.machina/`). Path resolved by `core.paths.example_workflows_dir()` — fixed, NOT under `DATA_DIR`, preserved by `machina clean` via `_MACHINA_KEEP`. Uses `UserSettings.examples_loaded` flag; supports anonymous users (`user_id="default"`); embedded `nodeParameters` saved to DB on import.
-- **Onboarding Service**: 5-step welcome wizard (Welcome, Concepts, API Keys, Canvas Tour, Get Started) using Ant Design Steps/Card/Button/Typography. Database-backed via `UserSettings.onboarding_completed` + `onboarding_step`. Existing users auto-skip (migration marks `examples_loaded=1` as completed). Replayable from Settings "Help" section. No new WebSocket handlers needed. See [Onboarding Service](./docs-internal/onboarding.md) for details.
+- **Onboarding Service**: 5-step welcome wizard (Welcome, Concepts, API Keys, Canvas Tour, Get Started) using shadcn primitives + lucide icons. Database-backed via `UserSettings.onboarding_completed` + `onboarding_step`. Existing users auto-skip (migration marks `examples_loaded=1` as completed). Replayable from Settings "Help" section. No new WebSocket handlers needed. See [Onboarding Service](./docs-internal/onboarding.md) for details.
- **Node.js Code Executor**: Persistent Node.js server (Express + tsx) at port 3020 for JavaScript/TypeScript execution, replacing subprocess spawning per execution. Handlers in `server/services/handlers/code.py` call `NodeJSClient` which makes HTTP requests to the Node.js server. All config via environment variables (`NODEJS_EXECUTOR_URL`, `NODEJS_EXECUTOR_PORT`, etc.).
- **writeTodos Tool Node**: Dedicated AI tool for task planning connecting to any agent's `input-tools` handle. `TodoService` singleton (`server/services/todo_service.py`) stores JSON-based per-session todo state keyed by workflow_id. Handler broadcasts `phase: "todo_update"` via WebSocket on each update for real-time UI; `formatTodoOutput()` in `OutputDisplayPanel.tsx` renders the result as a checklist with `[ ]` / `[~]` / `[x]` icons. Schema uses `TodoItem`/`TodoStatus` Pydantic enum. Skill at `server/skills/assistant/write-todos-skill/SKILL.md` teaches the plan-work-update loop.
- **Temporal Activity Heartbeats**: Activities send `activity.heartbeat()` on every non-matching WebSocket broadcast inside the read loop in `services/temporal/activities.py`. This keeps long-running browser and claude_code_agent operations alive past the 2-minute `heartbeat_timeout`. Start/end heartbeats alone were causing `TIMEOUT_TYPE_HEARTBEAT` failures on ops taking 5-10 minutes. Connection config: `heartbeat=30`, `receive_timeout=540` (fits within 10-min `start_to_close_timeout`).
diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.md
index 6ba9a57f..2f19017a 100644
--- a/docs-internal/TEMPORAL_ARCHITECTURE.md
+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md
@@ -12,6 +12,33 @@ Each workflow node executes as a **Temporal activity** with its own isolated con
`rlm_agent`, `claude_code_agent` are intentionally excluded from AgentWorkflow — their externalised loops (RLM REPL / Claude CLI `--resume`) require single-process state continuity.
+## Execution Routing & Running
+
+`WorkflowService.execute_workflow` picks one of three executors in priority order (`server/services/workflow.py`):
+
+1. If `TEMPORAL_ENABLED=true` and Temporal is configured → `_execute_temporal()` (the distributed path documented in this file).
+2. Else if Redis is available → `_execute_parallel()` (local parallel orchestration via `WorkflowExecutor`).
+3. Else → `_execute_sequential()` (single-threaded fallback).
+
+**Running with Temporal** — the Temporal server and the embedded worker start automatically with every launch script:
+
+```bash
+npm run start # Starts Temporal server + all services
+npm run dev # Starts Temporal server + all services (dev mode)
+npm run stop # Stops all services including Temporal
+```
+
+The embedded Temporal worker runs **inside the Python backend** — registered in the `main.py` lifespan via `TemporalWorkerManager`, not as a separate process.
+
+**Standalone worker** (for horizontal scaling — add more pollers against the same task queue):
+
+```bash
+cd server
+python -m services.temporal.worker
+```
+
+This invokes `run_standalone_worker()` from `services/temporal/worker.py`.
+
## System Architecture
```
@@ -426,9 +453,12 @@ Trigger nodes that aren't the firing trigger are:
```
server/services/temporal/
├── __init__.py # Exports TemporalExecutor, TemporalClientWrapper
-├── activities.py # NodeExecutionActivities class
+├── activities.py # NodeExecutionActivities class (legacy WebSocket round-trip path)
│ ├── execute_node_activity() # Main activity method
│ └── _execute_via_websocket() # WebSocket execution
+├── plugin_activities.py # collect_plugin_activities() -> per-type node.{type}.v{ver} activities (F4.A)
+├── agent_workflow.py # AgentWorkflow child workflow (F4.B agent loop)
+├── agent_activities.py # collect_agent_activities() -> the 7 agent.*.v1 activities (F4.B)
├── workflow.py # MachinaWorkflow class
│ ├── run() # Main orchestrator
│ ├── _filter_executable_graph() # Config node filtering
@@ -525,6 +555,8 @@ One supervisor-build-time env var (read inside `_temporal_specs.py`, not in `Set
## Debugging
+The Web UI is at http://localhost:8080; the UI's HTTP API is served at http://localhost:8233.
+
```bash
# Temporal Web UI
open http://localhost:8080
diff --git a/docs-internal/agent_architecture.md b/docs-internal/agent_architecture.md
index e413f747..1364a5d1 100644
--- a/docs-internal/agent_architecture.md
+++ b/docs-internal/agent_architecture.md
@@ -41,19 +41,24 @@ WorkflowService.execute_node() server/services/workflow.py
v
NodeExecutor._dispatch() server/services/node_executor.py
Handler registry lookup via functools.partial
- Dispatches to handle_ai_agent() or handle_chat_agent()
+ Dispatches via BaseNode.execute() to the agent plugin's execute_op()
+ (server/nodes/agent/<plugin>/__init__.py — Wave 11 deleted handle_ai_agent /
+ handle_chat_agent; agent logic moved into the plugin folder)
|
v
-_collect_agent_connections() server/services/handlers/ai.py
+collect_agent_connections() server/services/plugin/edge_walker.py
+ (called by nodes/agent/_inline.prepare_agent_call)
Scans edges where target == node_id
- Groups by targetHandle into 4 buckets:
- input-memory -> memory_data
- input-skill -> skill_data[]
- input-tools -> tool_data[]
- input-main -> input_data
+ Groups by targetHandle into 5 buckets (returns a 5-tuple
+ memory_data, skill_data, tool_data, input_data, task_data):
+ input-memory -> memory_data
+ input-skill -> skill_data[]
+ input-tools -> tool_data[]
+ input-main / input-chat -> input_data
+ input-task -> task_data
|
v
-AIService.execute_agent() server/services/ai.py
+AIService.execute_agent() / execute_chat_agent() server/services/ai.py
1. Inject skills into system message
2. Build LangChain StructuredTools from tool_data
3. Call _run_agent_loop() (plain async while loop)
@@ -76,7 +81,9 @@ Result broadcast via WebSocket
| `server/routers/websocket.py` | WebSocket handler `handle_execute_node()` |
| `server/services/workflow.py` | Facade, builds context, delegates to NodeExecutor |
| `server/services/node_executor.py` | Handler registry, dispatches via `functools.partial` |
-| `server/services/handlers/ai.py` | `_collect_agent_connections()`, `handle_ai_agent()`, `handle_chat_agent()` |
+| `server/services/plugin/edge_walker.py` | `collect_agent_connections()` (the renamed `_collect_agent_connections`), `format_task_context()` |
+| `server/nodes/agent/_inline.py` | `prepare_agent_call()` — task-context injection + auto-prompt fallback + teammate collection; calls `collect_agent_connections` then `ai_service.execute_[chat_]agent` |
+| `server/nodes/agent/<plugin>/__init__.py` | Per-plugin `execute_op()` (Wave 11 replaced `handle_ai_agent` / `handle_chat_agent`) |
| `server/services/ai.py` | `AIService` -- `_run_agent_loop`, skill injection, tool building |
| `server/services/handlers/tools.py` | `execute_tool()` -- dispatch router for all tool types |
| `server/services/skill_loader.py` | `SkillLoader` -- filesystem/DB skill discovery and loading |
@@ -159,7 +166,7 @@ Both build `initial_messages` (system + memory history + current prompt + skill
### 1. Edge Scanning
-In `_collect_agent_connections()` (`server/services/handlers/ai.py:14-49`), all edges targeting the agent node are scanned:
+In `collect_agent_connections()` (`server/services/plugin/edge_walker.py`; the renamed Wave-11 successor to `_collect_agent_connections` — the old `handlers/ai.py` was deleted), all edges targeting the agent node are scanned:
```python
for edge in edges:
@@ -435,3 +442,82 @@ else:
```
This optimisation means tool-less Chat Agent conversations skip `_run_agent_loop` entirely for faster response times.
+
+---
+
+## AI Agent vs Chat Agent (Zeenie)
+
+`aiAgent` and `chatAgent` (display name "Zeenie") are near-identical in capability — both run the same `_run_agent_loop`, support memory / skills / tools / task input, and can delegate asynchronously. The differences are which backend method they call and the softness of error handling.
+
+| Feature | AI Agent (`aiAgent`) | Zeenie (`chatAgent`) |
+|---------|----------------------|----------------------|
+| Tool Calling | Yes (agent loop) | Yes (agent loop) |
+| Memory Support | Yes | Yes |
+| Skill Support | Yes | Yes |
+| Task Input | Yes (`input-task`) | Yes (`input-task`) |
+| Bottom Handles | Skill, Tools | Skill, Tools |
+| Left Handles | Input, Memory, Task | Input, Memory, Task |
+| Backend Method | `execute_agent()` | `execute_chat_agent()` |
+| No-tool path | Always uses `_run_agent_loop` | Single `await chat_model.ainvoke(messages)` (skips loop overhead) |
+| Tool-failure handling | Re-raises (loop wraps as `{"error": ...}` ToolMessages) | Returns `{"error": str(e)}` (softer) |
+| Async Delegation | Yes (fire-and-forget) | Yes (fire-and-forget) |
+
+See the full method comparison in [execute_agent vs execute_chat_agent](#execute_agent-vs-execute_chat_agent) above.
+
+---
+
+## Agent Input Methods
+
+Both agents resolve their prompt the same way (logic in `server/nodes/agent/_inline.py:prepare_agent_call`):
+
+1. **Template Variable (Explicit)** — set the Prompt field to a template like `{{chatTrigger.message}}` or `{{whatsappReceive.text}}`.
+ - Templates are resolved by `ParameterResolver` before the plugin's `execute_op` runs.
+ - Supports nested paths: `{{nodeName.nested.field}}`.
+
+2. **Auto-Fallback (Implicit)** — leave the Prompt field empty (`prepare_agent_call` Step 2, lines ~85-93):
+ - When `not parameters.get("prompt") and input_data`, the agent reads the output of the node wired to its `input-main` / `input-chat` handle (`input_data`, surfaced by `collect_agent_connections`).
+ - Extraction order: `input_data["message"]` → `input_data["text"]` → `input_data["content"]` → `str(input_data)` (whole-output fallback).
+
+**Task-context injection (Step 1)** runs before the prompt fallback: when an `input-task` edge supplies `task_data`, `format_task_context(task_data)` is prepended to the prompt and tools may be stripped (the agent is reacting to a completed delegation, not starting fresh).
+
+**Example workflow:**
+```
+Chat Trigger → Zeenie ← HTTP Skill (SKILL.md context)
+ ← HTTP Request (tool node)
+```
+Zeenie loads SKILL.md instructions from connected skill nodes, builds tools from connected tool nodes (httpRequest, calculatorTool, …), and uses `_run_agent_loop` for tool execution when tools are connected.
+
+---
+
+## Handle Topology
+
+Handle layouts are declared in `server/nodes/agent/_handles.py` (local to `nodes/agent/`, not a global lookup). Two layouts cover the 20 agent variants:
+
+**`std_agent_handles()`** — shared by 18 of 20 agent variants:
+
+| Handle | Kind | Position | Offset | Role |
+|---|---|---|---|---|
+| `input-skill` | input | bottom | 25% | skill |
+| `input-tools` | input | bottom | 75% | tools |
+| `input-main` | input | left | 25% | main |
+| `input-memory` | input | left | 50% | memory |
+| `input-task` | input | left | 75% | task |
+| `output-main` | output | right | 50% | main |
+| `output-top` | output | top | — | main |
+
+**`team_lead_agent_handles()`** — `orchestrator_agent` and `ai_employee` only; adds an `input-teammates` handle (bottom, 80%) and shifts the bottom skill/tool offsets to 20%/50%. Connected agents become `delegate_to_<type>` tools via `_collect_teammate_connections()`. Team leads are also listed in `TEAM_LEAD_TYPES` in [`client/src/components/TeamMonitorNode.tsx`](../client/src/components/TeamMonitorNode.tsx) (frontend tribal array, flagged as tech debt). See [agent_teams.md](agent_teams.md).
+
+`AIAgentNode.tsx` is type-agnostic: it calls `useNodeSpec(type)` and renders whatever handles / icon / color the spec returns — there is no `AGENT_CONFIGS` map (retired Wave 10.D). Specialized-agent visuals all come from the NodeSpec; per-type display name / subtitle / description live on the plugin class, icon from `<plugin>/icon.svg` (visuals.json emoji fallback), color from `<plugin>/meta.json`. Glob `server/nodes/agent/` for the authoritative agent list — do not hand-maintain one.
+
+---
+
+## Async Agent Delegation (overview)
+
+AI Agents delegate to other agents wired to their `input-tools` handle, enabling hierarchical agent trees. There are two execution paths depending on whether the parent runs under Temporal F4.B:
+
+- **F4.B (default, `TEMPORAL_AGENT_WORKFLOW_ENABLED=true`)**: the parent `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow` when the child type is in `AGENT_WORKFLOW_TYPES`. The parent's `node_id` rides in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent's canvas badge. The LLM's `{task, context}` args travel as `child_context["invocation"]`, applied AFTER config resolution in `prepare_agent_payload` (so stored node parameters never override the delegated task; empty-task calls are rejected at the boundary without spawning). Deterministic child id `f"{parent_workflow_id}-delegate-{child_node_id}-{iteration}"`. The parent waits for the child result synchronously through the child-workflow handle.
+- **Legacy / F4.A-only**: fire-and-forget via `asyncio.create_task`. Still the path for `rlm_agent` / `claude_code_agent` (excluded from `AGENT_WORKFLOW_TYPES`) and any deployment with `TEMPORAL_AGENT_WORKFLOW_ENABLED=false`. The `delegate_to_*` tool spawns the child as a background task and returns immediately with `{"status": "delegated", "task_id": "..."}`; the parent keeps working while the child executes independently and broadcasts its own status (executing / success / error).
+
+Design decisions (legacy path): **memory isolation** (child uses its own connected memory, not the parent's), **error isolation** (child errors are logged + broadcast, never propagated to the parent), **task tracking** (background tasks live in `_delegated_tasks`, cleaned up on completion).
+
+Key files: `server/services/ai.py` (`DelegateToAgentSchema` in `_get_tool_schema()`, injects `ai_service` / `database` / `nodes` / `edges` into tool config), `server/services/handlers/tools.py` (`_execute_delegated_agent()`, `get_delegated_task_status()`). Full plumbing — memory / parameter / execution-context flow — is in [agent_delegation.md](agent_delegation.md); the multi-agent team-lead variant is in [agent_teams.md](agent_teams.md).
diff --git a/docs-internal/authentication.md b/docs-internal/authentication.md
new file mode 100644
index 00000000..8a4ab2e9
--- /dev/null
+++ b/docs-internal/authentication.md
@@ -0,0 +1,239 @@
+# Authentication System
+
+## Overview
+n8n-inspired authentication system with JWT tokens stored in HttpOnly cookies. Authentication can be completely disabled for development or supports two deployment modes for different use cases.
+
+## Authentication Toggle
+| Setting | Environment Variable | Description |
+|---------|---------------------|-------------|
+| **Enabled** | `VITE_AUTH_ENABLED=true` | Require login (default) |
+| **Disabled** | `VITE_AUTH_ENABLED=false` | Bypass authentication, anonymous access |
+
+When `VITE_AUTH_ENABLED=false`:
+- Frontend skips login page entirely
+- User is set to anonymous with owner privileges
+- No backend auth API calls are made
+- Useful for local development and testing
+
+## Deployment Modes (when auth enabled)
+| Mode | Environment Variable | Description |
+|------|---------------------|-------------|
+| **Single Owner** | `AUTH_MODE=single` | First user becomes owner, registration disabled after |
+| **Multi User** | `AUTH_MODE=multi` | Open registration for cloud deployments |
+
+## Architecture
+```
+Frontend (LoginPage.tsx) → AuthContext → Backend (/api/auth/*) → JWT Cookie
+ ↓
+ AuthMiddleware
+ ↓
+ Protected Routes
+```
+
+## Backend Implementation
+
+### User Model (`server/models/auth.py`)
+```python
+class User(SQLModel, table=True):
+ __tablename__ = "users"
+ id: Optional[int] = Field(default=None, primary_key=True)
+ email: str = Field(unique=True, index=True)
+ password_hash: str
+ display_name: str
+ is_owner: bool = Field(default=False)
+ is_active: bool = Field(default=True)
+ created_at: datetime
+ last_login: Optional[datetime]
+
+ def set_password(self, password: str) -> None:
+ # Uses bcrypt for secure hashing
+
+ def verify_password(self, password: str) -> bool:
+ # Verifies against bcrypt hash
+```
+
+### Auth Service (`server/services/user_auth.py`)
+- `register_user()` - Creates new user, sets as owner if first user in single mode
+- `authenticate_user()` - Validates credentials, returns user
+- `create_token()` - Generates JWT token
+- `verify_token()` - Validates JWT token
+- `get_auth_status()` - Returns mode, registration availability, user count
+
+`UserAuthService` is also the integration point for the encrypted-credentials
+system: `login()` calls `_initialize_encryption(password)` (derives the Fernet
+key via PBKDF2 from the user's password + the credentials DB salt) and
+`logout()` calls `self.encryption.clear()` to wipe the key from memory. See
+[Credentials Encryption](./credentials_encryption.md) for the full pipeline.
+
+### Auth Router (`server/routers/auth.py`)
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/auth/status` | GET | Get auth mode and registration status |
+| `/api/auth/register` | POST | Register new user |
+| `/api/auth/login` | POST | Login and set cookie |
+| `/api/auth/logout` | POST | Clear auth cookie |
+| `/api/auth/me` | GET | Get current user info |
+
+`routers/auth.py` defines its own local `get_auth_service()` helper
+(`return container.auth_service()`) — `get_auth_service` is NOT exported from
+`services/auth.py`. The same local-helper pattern is repeated in
+`routers/twitter.py`, `routers/google.py`, and `routers/websocket.py`. In
+handlers/services use `from core.container import container; auth = container.auth_service()`.
+
+### Auth Middleware (`server/middleware/auth.py`)
+Protects all routes except public paths:
+```python
+PUBLIC_PATHS = frozenset([
+ "/health", "/docs", "/openapi.json", "/redoc",
+ "/api/auth/status", "/api/auth/login", "/api/auth/register", "/api/auth/logout",
+ "/ws/internal", # Internal WebSocket for Temporal workers
+])
+
+# Path prefixes that are public. ``/mcp/`` is the CLI-agent MCP server, which
+# enforces its own per-batch bearer-token auth (cookies don't apply to it), so
+# it bypasses this cookie gate.
+PUBLIC_PREFIXES = ("/webhook/", "/mcp/")
+```
+
+The middleware also lets the static SPA shell, built assets, and client-side
+routes load BEFORE login (the SPA renders the login page itself) when the
+container serves the client on a single port.
+
+## Frontend Implementation
+
+### Auth Context (`client/src/contexts/AuthContext.tsx`)
+```typescript
+interface AuthContextValue {
+ user: User | null;
+ isAuthenticated: boolean;
+ isLoading: boolean;
+ error: string | null;
+ authMode: 'single' | 'multi';
+ canRegister: boolean;
+ login: (email: string, password: string) => Promise<boolean>;
+ register: (email: string, password: string, displayName: string) => Promise<boolean>;
+ logout: () => Promise<void>;
+ checkAuth: () => Promise<void>;
+}
+```
+
+### Protected Route (`client/src/components/auth/ProtectedRoute.tsx`)
+Wraps protected content, shows LoginPage if not authenticated:
+```typescript
+const ProtectedRoute: React.FC<{ children: React.ReactNode }> = ({ children }) => {
+ const { isAuthenticated, isLoading } = useAuth();
+ if (isLoading) return <LoadingSpinner />;
+ if (!isAuthenticated) return <LoginPage />;
+ return <>{children}</>;
+};
+```
+
+### Login Page (`client/src/components/auth/LoginPage.tsx`)
+- Dracula-themed login/register form
+- Switches between login and register based on `canRegister`
+- Displays errors from auth context
+
+## Configuration
+Environment variables in `.env`:
+```bash
+# Authentication Toggle (frontend - Vite)
+VITE_AUTH_ENABLED=true # 'true' or 'false' - disable to bypass login
+
+# Authentication Mode (backend)
+AUTH_MODE=single # 'single' or 'multi'
+JWT_SECRET_KEY=your-secret-key-32 # Min 32 chars
+JWT_EXPIRE_MINUTES=10080 # 7 days
+JWT_COOKIE_NAME=machina_token
+JWT_COOKIE_SECURE=false # true for HTTPS
+JWT_COOKIE_SAMESITE=lax
+```
+
+`core/config.py` carries the `vite_auth_enabled` field (required because
+Pydantic Settings uses `extra="forbid"`).
+
+## Race Condition Handling (TanStack Query bootstrap)
+The frontend starts before the backend is ready during cold launch, so the
+auth-status check must tolerate transient failures.
+
+The `AuthContext` bootstraps the auth-status check through TanStack Query
+(`useQuery({ queryKey: AUTH_STATUS_QUERY_KEY, queryFn: fetchAuthStatus, retry, retryDelay, signal })`),
+which replaced the previous recursive `setTimeout` retry chain. Behaviour:
+
+- **Full-jitter exponential backoff** per the AWS Architecture Blog formula:
+ `random(0, min(CAP_MS, BASE_MS * 2^attempt))`. Constants live in
+ [`client/src/lib/connectionConfig.ts`](../client/src/lib/connectionConfig.ts)
+ under `AUTH_RETRY`: `BASE_MS = 50`, `CAP_MS = 4000`, `MAX_ATTEMPTS = 7`.
+ Cumulative budget is ~10 s (vs. the old ~31 s); sub-second granularity early
+ covers the typical 4 s backend cold-start window in 4–5 attempts.
+- **401/403 short-circuit the retry chain** — those are valid responses meaning
+ "auth disabled / not logged in", not "backend unavailable", so no retry
+ budget is burned (`authShouldRetry` returns `false` when the wrapped error
+ message contains `HTTP 401` / `HTTP 403`).
+- **AbortController `signal`** is plumbed through `queryFn` so unmount + React
+ Strict Mode cleanup cancel in-flight requests automatically.
+- `login` / `register` / `logout` invalidate the cache via
+ `queryClient.invalidateQueries({ queryKey: AUTH_STATUS_QUERY_KEY })`
+ (`AUTH_STATUS_QUERY_KEY = ['auth', 'status']`).
+
+> Historical note: the original implementation used 5 fixed retries with
+> exponential backoff (1 s, 2 s, 4 s, 8 s, 16 s) and a recursive `setTimeout`
+> chain, surfacing "Failed to connect to server" only after all retries were
+> exhausted. This was superseded by the TanStack Query bootstrap above.
+
+## Cookie-Based Auth for API Calls
+All API calls must include `credentials: 'include'` for the HttpOnly cookie:
+```typescript
+// In workflowApi.ts, all fetch calls include:
+fetch(url, { credentials: 'include' })
+```
+
+## WebSocket Authentication
+WebSocket checks the cookie before accepting the connection:
+```python
+# In websocket.py
+token = websocket.cookies.get(settings.jwt_cookie_name)
+if not token:
+ await websocket.close(code=4001, reason="Not authenticated")
+ return
+```
+
+`WebSocketProvider` only connects when authenticated:
+```typescript
+// In WebSocketContext.tsx
+const { isAuthenticated, isLoading: authLoading } = useAuth();
+
+useEffect(() => {
+ if (authLoading || !isAuthenticated) {
+ // Disconnect if logged out
+ return;
+ }
+ connect();
+}, [isAuthenticated, authLoading]);
+```
+
+## Key Files
+| File | Description |
+|------|-------------|
+| `client/src/config/api.ts` | API config with AUTH_ENABLED toggle |
+| `client/src/contexts/AuthContext.tsx` | React auth state with TanStack Query bootstrap + retry logic |
+| `client/src/lib/connectionConfig.ts` | `AUTH_RETRY` backoff constants (`BASE_MS` / `CAP_MS` / `MAX_ATTEMPTS`) |
+| `client/src/components/auth/LoginPage.tsx` | Login UI |
+| `client/src/components/auth/ProtectedRoute.tsx` | Route guard |
+| `server/models/auth.py` | User SQLModel with bcrypt |
+| `server/services/user_auth.py` | JWT creation/verification + encryption init on login/logout |
+| `server/routers/auth.py` | REST endpoints |
+| `server/middleware/auth.py` | Route protection (`PUBLIC_PATHS` / `PUBLIC_PREFIXES`) |
+| `server/core/config.py` | Settings with `vite_auth_enabled` field |
+
+## Dependencies
+```
+# server/pyproject.toml
+bcrypt>=4.1.0
+pyjwt>=2.13.0
+email-validator>=2.0.0
+```
+
+JWT handling uses **PyJWT** (`import jwt`, `jwt.encode` / `jwt.decode`, catch
+`jwt.PyJWTError`) — HS256 with `Settings.jwt_secret_key`. Do **not** reintroduce
+`python-jose`: it drags in pure-Python `ecdsa`, which carries an unpatchable
+Minerva timing-attack advisory (GHSA-wj6h-64fc-37mp).
diff --git a/docs-internal/claude_code_agent.md b/docs-internal/claude_code_agent.md
new file mode 100644
index 00000000..48229772
--- /dev/null
+++ b/docs-internal/claude_code_agent.md
@@ -0,0 +1,49 @@
+# Claude Code Agent — Documentation Hub
+
+`claude_code_agent` is a MachinaOs agent node that drives the real `claude` CLI as a
+long-lived **interactive** subprocess (stdio pipes, no PTY) over the VSCode-extension
+stream-json protocol — not the `claude -p` headless path. A warm `ClaudeSessionPool`
+(keyed by the connected `simpleMemory.node_id`) preserves the session across turns;
+memory continuity is claude-native via `--continue` / `--resume <UUID>` on a stable
+`cwd=repo_root`. Connected tools and skills reach claude through an MCP bridge; the spawn
+runs under `--permission-mode dontAsk` with a strict `--allowedTools` allowlist and stays
+in interactive billing (entrypoint `claude-vscode`, not `sdk-cli`).
+
+This page is a **router**. The detail lives in the documents below — start here, then jump.
+
+## Where to read what
+
+| Your question | Read |
+|---|---|
+| How does the whole integration work end-to-end? (architecture, memory & context compaction, sub-agent creation, data structures) | [claude_code_agent_architecture.md](./claude_code_agent_architecture.md) |
+| How does interactive mode work? (stdio pipes vs PTY, stream-json events, `system/init` / `assistant` / `result`, session continuity, the four `claude.session.*` CloudEvents) | [claude_code_interactive_mode.md](./claude_code_interactive_mode.md) |
+| The generic multi-provider runtime (`AICliService.run_batch`, worktree isolation, FastMCP bridge, the memory bridge `--continue`/`--resume`, plugin-folder layout) | [cli_agent_framework.md](./cli_agent_framework.md) |
+| Does our implementation match the official Claude Code spec? (six invariants + status) | [cli_agent_canonical_patterns_rfc.md](./cli_agent_canonical_patterns_rfc.md) |
+| Which CLI subcommands/flags exist, and which subset we emit | [claude_code_cli_reference.md](./claude_code_cli_reference.md) (snapshot) |
+| Environment variables (`CLAUDE_CONFIG_DIR`, `MAX_MCP_OUTPUT_TOKENS`, `ENABLE_TOOL_SEARCH`, Bedrock/Vertex, telemetry) | [claude_code_env_vars_reference.md](./claude_code_env_vars_reference.md) (snapshot) |
+| Permission-mode semantics (`default` / `acceptEdits` / `plan` / `dontAsk` / `bypassPermissions`); why MachinaOs uses `dontAsk` | [claude_code_permission_modes_reference.md](./claude_code_permission_modes_reference.md) (snapshot) |
+| Headless / print mode (`claude -p`, `--output-format`, stream-json event schema) — the schema the pool parses off stdout | [claude_code_headless_reference.md](./claude_code_headless_reference.md) (snapshot) |
+| SKILL.md frontmatter spec, discovery paths, `context: fork` — what we materialise skills against | [claude_code_skills_reference.md](./claude_code_skills_reference.md) (snapshot) |
+
+The four `cli-reference` / `env-vars` / `permission-modes` / `headless` / `skills`
+documents are **verbatim snapshots** of code.claude.com docs (fetched 2026-05-11), kept so
+the contract the pool parses is pinned even if upstream changes.
+
+## Plugin folder — [server/nodes/agent/claude_code_agent/](../server/nodes/agent/claude_code_agent/)
+
+All claude-specific code is self-contained here; the generic framework at
+`server/services/cli_agent/` imports nothing from `nodes/`.
+
+| File | Responsibility |
+|---|---|
+| `__init__.py` | `ClaudeCodeAgentNode(SpecializedAgentBase)` + `Params`/`Output`; self-registers via `factory.py` (`register_provider` / `register_session_pool` / `register_skill_materialiser`) + `register_ws_handlers`. |
+| `_provider.py` | Builds the spawn argv (`--output-format stream-json --input-format stream-json --verbose --ide`, `--permission-mode dontAsk`, `--allowedTools`, `--continue`/`--resume`). |
+| `_pool.py` | `ClaudeSessionPool` — warm subprocess keyed by `simpleMemory.node_id`; stdout reader, session-UUID capture, crash-recovery respawn. |
+| `_skills.py` | `materialise_skills` — writes connected SKILL.md trees under `<workspace>/.claude/skills/`, diff-based on warm reuse. |
+| `_oauth.py` | Isolated `CLAUDE_CONFIG_DIR` (`<DATA_DIR>/claude/`), browser-OAuth bridge, binary install. |
+| `_handlers.py` | WebSocket handlers (`cli_login` / `cli_auth_status`). |
+
+## See also
+
+- Memory bridge specifics (how `simpleMemory` feeds `--continue`, why `last_session_id` is display-only): [cli_agent_framework.md → Memory bridge](./cli_agent_framework.md).
+- Generic CLI agent runtime shared with Codex / Gemini providers: `server/services/cli_agent/`.
diff --git a/docs-internal/credentials_encryption.md b/docs-internal/credentials_encryption.md
index 376a4332..35d7ca3f 100644
--- a/docs-internal/credentials_encryption.md
+++ b/docs-internal/credentials_encryption.md
@@ -54,6 +54,8 @@ API_KEY_ENCRYPTION_KEY (from .env)
The derived Fernet key lives only in `EncryptionService._fernet` in process memory. It is never written to disk or to Redis.
+`EncryptionService` (`server/core/encryption.py`) exposes exactly five methods: `initialize(password, salt)` (derive + store the Fernet cipher), `encrypt(plaintext) -> str` (base64 ciphertext), `decrypt(ciphertext) -> str`, `clear()` (drop the in-memory key), and `is_initialized() -> bool`. `CredentialsDatabase` (`server/core/credentials_database.py`) backs both systems with `initialize() -> bytes` (creates tables, returns the salt), `save_api_key` / `get_api_key` / `delete_api_key`, and `save_oauth_tokens` / `get_oauth_tokens` / `delete_oauth_tokens(provider, customer_id="owner")`.
+
## Lifecycle
```
@@ -147,12 +149,17 @@ class CredentialBackend(ABC):
The factory `create_backend(settings, credentials_db)` returns the selected backend with automatic fallback to Fernet if the requested backend is unavailable (e.g. `boto3` not installed for AWS).
-Optional dependencies for alternate backends:
+Dependencies (`server/pyproject.toml`) — the core `cryptography` package is always required for the default Fernet backend; the keyring / AWS packages are optional extras:
```toml
+[project]
+dependencies = [
+ "cryptography>=44.0.0", # Fernet encryption (always required)
+]
+
[project.optional-dependencies]
-keyring = ["keyring>=25.0.0"]
-aws = ["boto3>=1.34.0"]
+keyring = ["keyring>=25.0.0"] # OS-native credential storage
+aws = ["boto3>=1.34.0"] # AWS Secrets Manager
```
## Configuration
diff --git a/docs-internal/credentials_panel.md b/docs-internal/credentials_panel.md
index 4c458526..5cca188f 100644
--- a/docs-internal/credentials_panel.md
+++ b/docs-internal/credentials_panel.md
@@ -376,8 +376,8 @@ These are the non-negotiable behaviours the test suite enforces. The refactor is
| Credential service | [server/services/auth.py](../server/services/auth.py) | Single point of access; memory caches |
| Encryption | [server/core/encryption.py](../server/core/encryption.py) | Fernet + PBKDF2 |
| DB | [server/core/credentials_database.py](../server/core/credentials_database.py) | Two encrypted tables + metadata |
-| OAuth (Google) | [server/services/google_oauth.py](../server/services/google_oauth.py) | Flow, PKCE, refresh, service builders |
-| OAuth (Twitter) | [server/services/twitter_oauth.py](../server/services/twitter_oauth.py) | Manual PKCE, token exchange/refresh/revoke |
+| OAuth (Google) | [server/nodes/google/_oauth.py](../server/nodes/google/_oauth.py) | Flow, PKCE, refresh, service builders |
+| OAuth (Twitter) | [server/nodes/twitter/_oauth.py](../server/nodes/twitter/_oauth.py) | Manual PKCE, token exchange/refresh/revoke |
| OAuth (Claude) | [server/nodes/agent/claude_code_agent/_oauth.py](../server/nodes/agent/claude_code_agent/_oauth.py) + [_handlers.py](../server/nodes/agent/claude_code_agent/_handlers.py) | CLI-managed `claude auth login` (subprocess, `stdin=PIPE`), `CLAUDE_CONFIG_DIR=<DATA_DIR>/claude/` |
| OAuth utils | [server/services/oauth_utils.py](../server/services/oauth_utils.py) | Runtime redirect URI derivation |
| Container | [server/core/container.py](../server/core/container.py) | DI wiring for `auth_service` etc. |
diff --git a/docs-internal/deployment_legacy.md b/docs-internal/deployment_legacy.md
new file mode 100644
index 00000000..5f4d47d8
--- /dev/null
+++ b/docs-internal/deployment_legacy.md
@@ -0,0 +1,311 @@
+# Deployment — Current Self-Deploy CLI + Legacy Docker Reference
+
+This doc is the single home for MachinaOS deployment topology. The **current** path is the
+`machina deploy` self-deploy CLI (top section). The **Docker Compose** path below it is a
+**legacy reference only** — the compose files, `docker/` directory, and `scripts/docker.js`
+wrapper have all been removed from the repo. It is retained solely as documentation of the
+historical container topology.
+
+---
+
+## Self-Deploy CLI (`machina deploy`) — current path
+
+One command provisions a login-gated MachinaOS VM on a cloud provider. Two stages:
+
+1. **Operator's cloud CLI** (gcloud; aws planned) handles auth + project/region/zone resolution +
+ ADC verification + API enablement.
+2. **Terraform** (`cli/terraform/gcp/`) owns all resources — VM (always named `machinaos`),
+ firewall, artifact bucket (local `npm pack` source), service account, and a cloud-init startup
+ script that installs Node 22 + uv + the package and runs `machina serve` under systemd.
+
+Login gate = built-in auth (`VITE_AUTH_ENABLED=true`, `AUTH_MODE=single`) with the owner
+credential generated at deploy time and seeded on first boot.
+
+```bash
+machina deploy up --provider gcp --owner-email [email protected] # provision + install + print URL/creds
+machina deploy status # URL + /health
+machina deploy destroy # terraform destroy + clear state
+```
+
+### Key files
+
+| Path | Responsibility |
+|------|----------------|
+| `cli/commands/serve.py` | Single-port runtime: uvicorn fronts API + WS + built SPA, plus the node sidecar |
+| `cli/commands/deploy/` | Verbs (`up.py` / `status.py` / `destroy.py`), `_secrets.py`, `_state.py`, `_terraform.py` (Terraform driver), `providers/` (`gcp.py` / `aws.py` provider CLI adapters) |
+| `cli/terraform/gcp/` | HCL module (`main.tf` / `variables.tf` / `outputs.tf`) + `startup.sh.tftpl` cloud-init template |
+
+### State + delinking
+
+- Deployment state lives at `<user-data>/deploy/machinaos/` — preserved by `machina clean`
+ (see `_MACHINA_KEEP` in `cli/commands/clean.py`); only `machina deploy destroy` removes it.
+- The deploy code is fully delinked from `machina build` / `machina clean`: lazy verb stubs in
+ `cli/cli.py` mean nothing in the build pipeline imports it.
+
+The legacy `deploy.sh` (docker-compose images over SCP to a GCE box) was removed.
+
+---
+
+## Docker Deployment (LEGACY REFERENCE — removed from repo)
+
+> **Legacy reference only.** The compose files (`docker-compose.yml`, `docker-compose.prod.yml`),
+> the `docker/` directory (incl. `docker/Dockerfile.whatsapp`), `client/Dockerfile`,
+> `server/Dockerfile`, and `scripts/docker.js` are **no longer in the repository**. This section
+> documents the historical 4-container topology for reference; it is not a runnable path today.
+
+The project previously deployed using Docker Compose with an nginx reverse proxy.
+
+### Docker Configuration (4-Container Stack)
+
+**Services:**
+
+| Container | Image | Port | Description |
+|-----------|-------|------|-------------|
+| redis | redis:7-alpine | 6379 | Cache and pub/sub for workflows |
+| backend | machinaos-backend | 3010 | FastAPI Python backend |
+| frontend | machinaos-frontend | 3000 | React app via nginx |
+| whatsapp | machinaos-whatsapp | 5000 | Go WhatsApp bridge service |
+
+**Frontend (`client/Dockerfile`):**
+- Multi-stage build: Node.js builder -> nginx:alpine production
+- Serves static files via nginx on port 80 (mapped to 3000)
+- Size: ~54 MB
+
+**Backend (`server/Dockerfile`):**
+- Python 3.12-slim base with Node.js 22.x for JS/TS execution
+- Includes Playwright chromium for JS-rendered web scraping (crawleeScraper node)
+- Includes persistent Node.js server (Express + tsx) on port 3020
+- Optimized bytecode compilation (`python -O -m compileall`)
+- Health check endpoint on port 3010
+- Startup script (`start.sh`) runs both Python and Node.js servers (must have LF line endings, not CRLF)
+- Depends on: redis, whatsapp
+- Size: ~800 MB (includes Playwright chromium)
+
+**WhatsApp (`docker/Dockerfile.whatsapp`):**
+- Uses npm package `whatsapp-rpc` with pre-built binaries
+- Node.js 20-alpine base with `npx whatsapp-rpc api --foreground`
+- Binary downloaded from GitHub releases during npm postinstall
+- Exposed on port 9400 (configurable via `PORT`, `WHATSAPP_RPC_PORT` env vars, or `--port` CLI flag)
+- QR codes generated as base64 PNG in memory (no file I/O)
+- Also published to PyPI as `whatsapp-rpc` (async Python client)
+- Size: ~150 MB (includes Node.js runtime)
+
+**Redis:**
+- Official redis:7-alpine image
+- Healthcheck: `redis-cli ping`
+- Persistent volume: `redis_data`
+- No authentication (internal network only)
+
+**Development Compose (`docker-compose.yml`):**
+```yaml
+services:
+ # Redis uses profiles - only starts when REDIS_ENABLED=true
+ redis:
+ image: redis:7-alpine
+ profiles:
+ - redis # Only starts with --profile redis flag
+ ports: ["${REDIS_PORT:-6379}:6379"]
+ healthcheck:
+ test: ["CMD", "redis-cli", "ping"]
+
+ backend:
+ build: ./server
+ ports: ["${PYTHON_BACKEND_PORT:-3010}:${PYTHON_BACKEND_PORT:-3010}"]
+ volumes:
+ - ./server:/app
+ - /app/nodejs/node_modules # Preserve Linux binaries (prevents Windows esbuild conflict)
+ depends_on:
+ whatsapp: { condition: service_healthy } # No Redis dependency
+ environment:
+ - REDIS_ENABLED=${REDIS_ENABLED:-false}
+ - REDIS_URL=redis://redis:6379
+
+ frontend:
+ build: ./client
+ ports: ["${VITE_CLIENT_PORT:-3000}:${VITE_CLIENT_PORT:-3000}"]
+
+ whatsapp:
+ build:
+ context: .
+ dockerfile: docker/Dockerfile.whatsapp
+ ports: ["${WHATSAPP_RPC_PORT:-9400}:${WHATSAPP_RPC_PORT:-9400}"]
+```
+
+**Docker Scripts Wrapper (`scripts/docker.js`):**
+Auto-detected `REDIS_ENABLED` in `.env` and added the `--profile redis` flag when enabled:
+```javascript
+// Reads .env and checks REDIS_ENABLED value
+function isRedisEnabled() {
+ const content = readFileSync(resolve(ROOT, '.env'), 'utf8');
+ const match = content.match(/^REDIS_ENABLED\s*=\s*(.+)$/m);
+ const value = match?.[1].trim().toLowerCase();
+ return value === 'true' || value === '1' || value === 'yes';
+}
+
+// Adds --profile redis when enabled
+if (isRedisEnabled()) {
+ composeArgs.push('--profile', 'redis');
+}
+```
+
+**Production Compose (`docker-compose.prod.yml`):**
+```yaml
+services:
+ redis:
+ image: redis:7-alpine
+ ports: ["6379:6379"]
+ volumes: ["redis_data:/data"]
+ healthcheck:
+ test: ["CMD", "redis-cli", "ping"]
+
+ whatsapp:
+ build:
+ context: .
+ dockerfile: docker/Dockerfile.whatsapp
+ ports: ["9400:9400"]
+ healthcheck:
+ test: ["CMD", "wget", "-q", "--spider", "http://localhost:9400/health"]
+
+ backend:
+ build: ./server
+ ports: ["3010:3010"]
+ depends_on:
+ redis: { condition: service_healthy }
+ whatsapp: { condition: service_healthy }
+ environment:
+ - REDIS_ENABLED=true
+ - REDIS_URL=redis://redis:6379
+
+ frontend:
+ build: ./client
+ ports: ["3000:80"]
+```
+
+### Nginx Configuration
+
+Located at `/etc/nginx/sites-available/flow.zeenie.xyz`:
+- Frontend: `/` -> `http://127.0.0.1:3000`
+- Backend API: `/api/` -> `http://127.0.0.1:3010/api/`
+- WebSocket: `/ws/` -> `http://127.0.0.1:3010/ws/` (with upgrade headers)
+- Webhook: `/webhook/` -> `http://127.0.0.1:3010/webhook/`
+- Health: `/health` -> `http://127.0.0.1:3010/health`
+- SSL via Let's Encrypt certbot
+
+### Environment Configuration
+
+**Development** (`server/.env`):
+- `DEBUG=true`
+- `CORS_ORIGINS` includes localhost ports
+- `REDIS_ENABLED=false` (uses SQLite cache for local dev)
+
+**Production** (Docker environment variables):
+- `DEBUG=false`
+- `CORS_ORIGINS=["https://your-domain.com"]`
+- `REDIS_ENABLED=true` (Docker Redis container)
+- `REDIS_URL=redis://redis:6379`
+- Environment set in `docker-compose.prod.yml`, not the `.env` file
+
+### Frontend API URL Resolution
+
+The frontend automatically detects production vs development. This logic still ships in
+`client/src/config/api.ts` (it is independent of the deployment mechanism):
+
+```typescript
+// client/src/config/api.ts
+const isProduction = typeof window !== 'undefined' &&
+ !window.location.hostname.includes('localhost') &&
+ !window.location.hostname.includes('127.0.0.1');
+
+return {
+ // Python FastAPI backend (port 3010 in dev, same origin in prod)
+ PYTHON_BASE_URL: viteEnv.VITE_PYTHON_SERVICE_URL || (isProduction ? '' : 'http://localhost:3010'),
+};
+```
+
+- **Production**: Empty base URL = relative URLs (same origin)
+- **Development**: Explicit `http://localhost:3010`
+
+WebSocket URL derived from base URL:
+```typescript
+// client/src/contexts/WebSocketContext.tsx
+if (!baseUrl) {
+ // Production: use current origin
+ const wsProtocol = window.location.protocol === 'https:' ? 'wss' : 'ws';
+ return `${wsProtocol}://${window.location.host}/ws/status`;
+}
+```
+
+### Resource Usage (GCP e2-micro, legacy Docker deployment)
+
+| Resource | Value |
+|----------|-------|
+| CPU | 2 cores (Intel Xeon @ 2.20GHz) |
+| RAM | 1.9 GB total, ~820 MB used |
+| Disk | 14 GB total, ~9.2 GB used |
+| Backend Memory | ~144 MB |
+| Frontend Memory | ~3.4 MB |
+
+### Useful Commands (legacy Docker deployment)
+
+```bash
+# View logs (all containers)
+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f'
+
+# View specific service logs
+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f backend'
+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f whatsapp'
+
+# Restart all services
+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart'
+
+# Restart specific service
+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart backend'
+
+# Check container status
+ssh $DEPLOY_HOST 'docker ps'
+
+# Check resource usage
+ssh $DEPLOY_HOST 'docker stats --no-stream'
+
+# Check Redis connection
+ssh $DEPLOY_HOST 'docker exec machinaos-redis-1 redis-cli ping'
+
+# Check backend health (shows redis_enabled status)
+curl -s https://$DEPLOY_DOMAIN/health | jq
+
+# Clean up Docker resources (if disk full)
+ssh $DEPLOY_HOST 'docker system prune -af && docker builder prune -af'
+```
+
+### Local Docker Development (legacy)
+
+For testing the full production stack locally (when the compose files existed):
+
+```bash
+# Build and start all containers
+docker-compose -f docker-compose.prod.yml up --build
+
+# Access locally
+# Frontend: http://localhost:3000
+# Backend API: http://localhost:3010
+# WhatsApp RPC: http://localhost:9400
+# Redis: localhost:6379
+
+# Stop all containers
+docker-compose -f docker-compose.prod.yml down
+
+# Remove volumes (clean slate)
+docker-compose -f docker-compose.prod.yml down -v
+```
+
+---
+
+## Local Development Build (current, non-Docker)
+
+```bash
+# Create optimized build
+npm run build
+
+# Serve built files locally
+npm run preview
+```
diff --git a/docs-internal/event_framework.md b/docs-internal/event_framework.md
index 9b790566..6c72d01e 100644
--- a/docs-internal/event_framework.md
+++ b/docs-internal/event_framework.md
@@ -2,7 +2,7 @@
Temporal-native event-routing layer for MachinaOs. Implements RFC sections
6.3 (Temporal worker contract) + 6.4 (CloudEvents broadcast contract) from
-[plugin_authoring_rfc.md](./plugin_authoring_rfc.md).
+[plugin_authoring_rfc.md](./ARCHIVE/plugin_authoring_rfc.md).
This doc is the operator + plugin-author reference. The design rationale +
phase plan lives in `~/.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`.
@@ -267,6 +267,6 @@ Locked by `TestCancelSweepsStuckNodeStatuses` in [`test_deployment_canary_listen
## References
- Plan: `~/.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`
-- RFC: [`plugin_authoring_rfc.md`](./plugin_authoring_rfc.md)
+- RFC: [`plugin_authoring_rfc.md`](./ARCHIVE/plugin_authoring_rfc.md)
- Temporal: [Search Attributes](https://docs.temporal.io/search-attribute) · [Signals](https://docs.temporal.io/develop/python/message-passing) · [Schedules](https://docs.temporal.io/develop/python/schedules) · [Retry Policies](https://docs.temporal.io/encyclopedia/retry-policies)
- CloudEvents: [v1.0 spec](https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md)
diff --git a/docs-internal/memory_lifecycle.md b/docs-internal/memory_lifecycle.md
index 377a4cd7..66aa5528 100644
--- a/docs-internal/memory_lifecycle.md
+++ b/docs-internal/memory_lifecycle.md
@@ -91,7 +91,7 @@ await clear_agent_session_state(
Three stores are cleared in one call:
1. **Vector store** (when `clear_long_term=True`) — `del services.ai._memory_vector_stores[session_id]`.
-2. **TodoService** — every candidate key (`workflow_id`, `session_id`, `"default"`) is cleared because [`server/nodes/tool/write_todos.py`](../server/nodes/tool/write_todos.py) uses `ctx.workflow_id or ctx.node_id or "default"` as the storage key and we want to clear whichever fallback the write path actually used.
+2. **TodoService** — every candidate key (`workflow_id`, `session_id`, `"default"`) is cleared because [`server/nodes/tool/write_todos/__init__.py`](../server/nodes/tool/write_todos/__init__.py) uses `ctx.workflow_id or ctx.node_id or "default"` as the storage key and we want to clear whichever fallback the write path actually used.
3. **simpleMemory node fields** (when `memory_node_id` provided) — `memory_content` → `DEFAULT_MEMORY_CONTENT`; `last_session_id` → `None` (so claude_code_agent starts fresh next run instead of `--resume`-ing into a wiped transcript); orphan `memory_jsonl` field popped if present.
Frontend `clear_memory` WS handler ([`routers/websocket.py:2167`](../server/routers/websocket.py)) calls this; UI presents a Reset Memory button on the simpleMemory parameter panel.
diff --git a/docs-internal/native_llm_sdk.md b/docs-internal/native_llm_sdk.md
index d503676b..cfc61f91 100644
--- a/docs-internal/native_llm_sdk.md
+++ b/docs-internal/native_llm_sdk.md
@@ -1,7 +1,6 @@
# Native LLM SDK Architecture
-> **⚠️ Pre-Wave-11 — historical reference only.**
-> Node authoring now happens on the backend: each node is a Python plugin under `server/nodes/<category>/<node>.py` that emits a `NodeSpec`. The frontend reads specs via [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) + [adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts). See [plugin_system.md](./plugin_system.md) and [server/nodes/README.md](../server/nodes/README.md) for the current model. The snippets below that reference `client/src/nodeDefinitions/*` are kept for historical context.
+> **Authoring model (post-Wave-11):** each chat-model node is a self-contained Python plugin folder under `server/nodes/model/<provider>_chat_model/` that emits a `NodeSpec`. The frontend reads specs via [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) + [adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts) and renders through `SquareNode` with zero TS edits. See [plugin_system.md](./plugin_system.md) and [server/nodes/README.md](../server/nodes/README.md) for the plugin model, and "Adding a New Provider" below for the chat-model recipe.
MachinaOS uses a hybrid LLM architecture: a native SDK layer in `server/services/llm/` for direct chat completions and model fetching, and a LangChain-backed agent loop in `server/services/ai.py` (the plain `_run_agent_loop` async function) for agent tool-calling. This document describes the native layer, its design, and how both paths coexist.
@@ -53,6 +52,38 @@ The native layer currently supports **12 providers**, grouped by implementation:
Source of truth for this list: `server/config/llm_defaults.json` (the `providers` dict) and `server/services/llm/factory.py` (`NATIVE_PROVIDERS` constant).
+### Native chat path vs agent dropdown — two different counts
+
+- **Native chat path (`execute_chat` / `fetch_models`) supports 12 providers** (the 10 OpenAI-compat + Anthropic/Gemini natives above; Groq/Cerebras fall back to LangChain inside `execute_chat`). `xai` lives here.
+- **The agent dropdown exposes 11 providers** for `aiAgent`, `chatAgent` (Zeenie), and all specialized agents — the `provider` Literal in [`nodes/agent/ai_agent/__init__.py`](../server/nodes/agent/ai_agent/__init__.py), [`chat_agent.py`](../server/nodes/agent/chat_agent/__init__.py), and [`_specialized.py`](../server/nodes/agent/_specialized.py): `openai`, `anthropic`, `gemini`, `openrouter`, `groq`, `cerebras`, `deepseek`, `kimi`, `mistral`, `ollama`, `lmstudio`. **`xai` is native-chat-only and is NOT in the agent Literal** (no agent dropdown entry).
+- Groq and Cerebras are available as standalone chat-model nodes AND in the agent dropdown, but use the LangChain path for both chat and agent execution.
+
+### Provider / model reference table
+
+`ModelRegistryService` (`server/services/model_registry.py`) manages the per-model constraints below — fetching from OpenRouter for cloud models and from the user's running local server (`ollama.AsyncClient.ps()` / `lmstudio.AsyncClient.llm.list_loaded()`) for Ollama / LM Studio. Falls back to `llm_defaults.json` only when neither source is available.
+
+| Provider | Key Models | Context | Max Output | Thinking | Temp Range |
+|----------|-----------|---------|-----------|----------|------------|
+| **OpenAI** | GPT-5.5/5.4/5.2 | 400K–1.05M | 128K | effort (hybrid) | 0-2 |
+| **OpenAI** | GPT-4.1/4.1-mini/4.1-nano | 1M | 32K | none | 0-2 |
+| **OpenAI** | o3, o4-mini | 200K | 100K | effort (reasoning) | fixed 1.0 |
+| **OpenAI** | GPT-4o/4o-mini | 128K | 16K | none | 0-2 |
+| **Anthropic** | Claude Fable 5 | 1M | 128K | budget | 0-1 |
+| **Anthropic** | Claude Opus 4.8/4.7 | 1M | 128K | budget | 0-1 |
+| **Anthropic** | Claude Sonnet 4.6 | 1M | 64K | budget | 0-1 |
+| **Anthropic** | Claude Haiku 4.5 | 200K | 64K | budget | 0-1 |
+| **Google** | Gemini 3.5-flash, 3.1-pro/flash-lite, 3-flash, 2.5-pro/flash/flash-lite | 1M | 64K | budget | 0-2 |
+| **DeepSeek** | deepseek-v4-flash, deepseek-v4-pro (deepseek-chat/reasoner legacy) | 1M | 64K | thinking modes | 0-2 |
+| **Kimi** | kimi-k2.6, kimi-k2.5, kimi-k2.7-code | 256K | 96K | on by default (disabled for agents) | fixed 0.6 |
+| **Mistral** | mistral-large/medium/small-latest, codestral-latest | 256K | 131K | none | 0-1.5 |
+| **Groq** | Llama 3.3-70b, Llama 3.1-8b, GPT-OSS-120b/20b, Qwen3-32b | 131K | 8-131K | format (Qwen3) | 0-2 |
+| **OpenRouter** | 200+ models from multiple providers | varies | varies | varies | 0-2 |
+| **Cerebras** | GPT-OSS-120b, Qwen-3-235b, GLM-4.7 | 131K | 65K | budget (Qwen) | 0-1.5 |
+| **Ollama** | Whatever the user has pulled (qwen2.5, llama3.x, phi-3, deepseek-r1, ...) | per-loaded-model (typed via `ps()`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |
+| **LM Studio** | Whatever the user has loaded in the LM Studio UI | per-loaded-model (typed via `LlmInstanceInfo.context_length`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |
+
+`_resolve_max_tokens()` in `server/services/ai.py` (a thin wrapper over `services/llm/config.py::resolve_max_tokens`) clamps user-requested `max_tokens` to the model's actual limit.
+
## Local LLM Providers (Ollama, LM Studio)
Ollama and LM Studio expose an OpenAI-compatible `/v1` HTTP API, so they ride the same `OpenAIProvider` runtime path used by every other OpenAI-compat backend (DeepSeek, Kimi, Mistral). The differences are the **base URL** (the user enters their server's address, e.g. `http://localhost:11434/v1`) and the **per-model parameters** (which depend on what the user has loaded in the local server's UI, not a JSON default).
@@ -71,7 +102,7 @@ Ollama and LM Studio expose an OpenAI-compatible `/v1` HTTP API, so they ride th
**Runtime path** — `execute_chat()` and `execute_agent()` read `{provider}_proxy` via `auth.get_api_key()` and pass it as `proxy_url` to `create_provider()`. `OpenAIProvider.__init__` overrides `kwargs["base_url"]` with the user's URL and forces `api_key="ollama"` (the documented placeholder for unauthenticated local servers). The openai SDK then sends `POST {user_url}/chat/completions` — **traffic stays on the user's machine, never reaches api.openai.com**.
-**Provider detection** ([`server/constants.py:detect_ai_provider`](../server/constants.py)) MUST list `ollama` / `lmstudio` substrings, and the agent dropdown's `provider` Literal in [`ai_agent.py`](../server/nodes/agent/ai_agent.py) / [`chat_agent.py`](../server/nodes/agent/chat_agent.py) / [`_specialized.py`](../server/nodes/agent/_specialized.py) MUST include `"ollama"` / `"lmstudio"` — otherwise the chat-model node silently falls through to `'openai'` and the runtime calls the OpenAI cloud with the local-server placeholder key.
+**Provider detection** ([`server/constants.py:detect_ai_provider`](../server/constants.py)) MUST list `ollama` / `lmstudio` substrings, and the agent dropdown's `provider` Literal in [`ai_agent.py`](../server/nodes/agent/ai_agent/__init__.py) / [`chat_agent.py`](../server/nodes/agent/chat_agent/__init__.py) / [`_specialized.py`](../server/nodes/agent/_specialized.py) MUST include `"ollama"` / `"lmstudio"` — otherwise the chat-model node silently falls through to `'openai'` and the runtime calls the OpenAI cloud with the local-server placeholder key.
**Open-world skip in `is_model_valid_for_provider`** — local model names like `qwen/qwen3.6-27b` don't contain provider substrings, so the cloud-style pattern check would always reject them. The function returns `True` for `openrouter` / `ollama` / `lmstudio` without consulting `detection_patterns`. The upstream server still rejects genuinely missing models with a clear 404.
@@ -227,6 +258,57 @@ class ThinkingConfig:
Each provider's `chat()` method reads only the fields it supports. The extracted thinking text is returned in `LLMResponse.thinking` so downstream nodes can display or drag it into parameters.
+### Supported thinking/reasoning providers
+
+| Provider | Models | Parameter | Thinking Type | Notes |
+|----------|--------|-----------|---------------|-------|
+| **Claude** | All Claude 4.x/3.5 | `thinkingBudget` (1024-16000 tokens) | budget | Requires `max_tokens > budget_tokens`. Temperature auto-set to 1. |
+| **Gemini** | gemini-3.x, gemini-2.5-pro/flash | `thinkingBudget` (token count) | budget | Uses `thinking_budget` API parameter |
+| **OpenAI** | o1, o3, o3-mini, o4-mini | `reasoningEffort` (low/medium/high) | effort | Reasoning-only models. Temperature fixed at 1.0. |
+| **OpenAI** | GPT-5.2/5.1/5/5-mini/5-nano | `reasoningEffort` (low/medium/high/xhigh) | effort | Hybrid reasoning: can operate with or without thinking. |
+| **Groq** | qwen3-32b | `reasoningFormat` ('parsed' or 'hidden') | format | 'parsed' returns reasoning, 'hidden' returns only final answer |
+| **Cerebras** | qwen-3-235b | `reasoningFormat` ('parsed' or 'hidden') | format | Same format-based reasoning as Groq Qwen |
+
+The thinking/reasoning fields (`thinkingEnabled`, `thinkingBudget`, `reasoningEffort`, `reasoningFormat`) live in the backend NodeSpec for each chat model (`server/nodes/model/<provider>_chat_model/`) and are surfaced through `AIChatModelParams` in `server/models/nodes.py`. The frontend renders them automatically via the universal parameter panel.
+
+### Thinking extraction (LangChain path)
+
+On the LangChain path, `extract_thinking_from_response(response, provider)` in `server/services/ai.py` pulls reasoning text out of provider-specific response shapes:
+
+```python
+def extract_thinking_from_response(response, provider: str) -> Optional[str]:
+ """Extract thinking/reasoning from AI response based on provider."""
+ # Claude: content_blocks with type='thinking'
+ # Gemini: response_metadata.candidates[0].content.parts with thought=True
+ # Groq: additional_kwargs.reasoning or response_metadata.reasoning
+ # OpenAI o-series: requires organization verification
+```
+
+The agent/chat result envelope carries `thinking` alongside the answer:
+
+```python
+{
+ "success": True,
+ "result": {
+ "response": "The final answer text",
+ "thinking": "The model's internal reasoning (if available)",
+ "model": "claude-3-5-sonnet-20241022",
+ "provider": "anthropic",
+ "finish_reason": "stop",
+ "timestamp": "2025-01-23T...",
+ }
+}
+```
+
+The `thinking` field is exposed to downstream nodes via the backend output schema (`AIAgentOutput` in `server/services/node_output_schemas.py`, shared across every LLM-backed agent + chat model) and rendered in a collapsible `ThinkingBlock` (`client/src/components/ui/NodeOutputPanel.tsx`, default-expanded, provider-aware label).
+
+### Thinking limitations
+
+- **OpenAI o-series**: Reasoning summaries are only available to organizations that have completed verification at platform.openai.com. Without verification, `thinking` is `null`.
+- **Claude**: `max_tokens` must be greater than `thinkingBudget`. Temperature is automatically set to 1 when thinking is enabled.
+- **Groq**: Only Qwen3-32b supports reasoning (QwQ removed from Groq). Format `hidden` suppresses reasoning output.
+- **Cerebras**: Qwen-3-235b supports format-based reasoning (same as Groq Qwen).
+
See [memory_compaction.md](memory_compaction.md) for how thinking token counts are tracked separately from output tokens.
## Model Max Tokens Resolution
@@ -255,23 +337,112 @@ Paired with `ModelRegistryService` (`server/services/model_registry.py`), which
- `supports_thinking`
- `thinking_type`
+## Proxy-Based Authentication (Ollama Pattern)
+
+AI providers support optional proxy-based authentication — requests route through a local proxy server that handles auth, following the [Ollama Claude Code integration](https://docs.ollama.com/integrations/claude-code) pattern.
+
+**How it works:**
+1. User configures a proxy URL in the Credentials Modal (e.g., `http://localhost:11434`).
+2. Requests route through the proxy instead of directly to the provider API.
+3. Proxy handles authentication (token set to `"ollama"` automatically).
+4. No API key storage needed in MachinaOs — auth delegated to proxy.
+
+**Configuration:** proxy URLs are stored in the credentials DB under the `{provider}_proxy` pattern (e.g., `anthropic_proxy`, `openai_proxy`). Falls back to direct API key if no proxy configured. This is the SAME mechanism the native Ollama / LM Studio path uses (see "Local LLM Providers" above) — the validator persists the user's server URL under `{provider}_proxy`, and at runtime it carries into `OpenAIProvider`'s `base_url`.
+
+**Native path:** `create_provider(name, api_key, proxy_url=url)`. **LangChain path:** `create_model()` sets `base_url`:
+
+```python
+def create_model(self, provider: str, api_key: str, model: str,
+ temperature: float, max_tokens: int,
+ thinking: Optional[ThinkingConfig] = None,
+ proxy_url: Optional[str] = None):
+ # ...
+ if proxy_url:
+ kwargs['base_url'] = proxy_url
+ kwargs[config.api_key_param] = "ollama" # Ollama-style placeholder token
+```
+
+**Use cases:** Claude Code CLI proxy for Anthropic models; native Ollama / LM Studio support; custom auth proxies; dev/testing with mock servers.
+
+## Provider Default Parameters
+
+Users configure default parameter values per LLM provider in the Credentials Modal; defaults apply to new AI nodes using that provider.
+
+**Configurable parameters:**
+- `temperature` — range varies by provider (Anthropic 0-1, Cerebras 0-1.5, others 0-2; o-series fixed 1.0)
+- `max_tokens` (1-200000) — clamped to the model's actual limit by `_resolve_max_tokens()`
+- `thinking_enabled` — extended thinking toggle
+- `thinking_budget` (1024-16000) — token budget for thinking (Claude, Gemini)
+- `reasoning_effort` (low/medium/high) — OpenAI o-series and GPT-5 hybrid reasoning
+- `reasoning_format` (parsed/hidden) — Groq Qwen3 models
+
+```python
+# server/models/database.py
+class ProviderDefaults(SQLModel, table=True):
+ provider: str # openai, anthropic, gemini, groq, openrouter, cerebras
+ temperature: float
+ max_tokens: int
+ thinking_enabled: bool
+ thinking_budget: int
+ reasoning_effort: str # low, medium, high
+ reasoning_format: str # parsed, hidden
+```
+
+| File | Description |
+|------|-------------|
+| `server/models/database.py` | `ProviderDefaults` SQLModel |
+| `server/core/database.py` | `get_provider_defaults()`, `save_provider_defaults()` CRUD |
+| `server/routers/websocket.py` | `get_provider_defaults`, `save_provider_defaults` handlers |
+| `client/src/hooks/useApiKeys.ts` | `getProviderDefaults()`, `saveProviderDefaults()` methods |
+| `client/src/components/CredentialsModal.tsx` | Default Parameters UI section |
+
## Adding a New Provider
+> **Post-Wave-11 authoring.** A chat-model provider is a self-contained folder under `server/nodes/model/<provider>_chat_model/` with `__init__.py` declaring a `ChatModelBase` subclass. It auto-registers via `BaseNode.__init_subclass__`; the frontend renders it through `SquareNode` from the emitted NodeSpec with **zero TypeScript changes**. There is no `client/src/nodeDefinitions/`, no `ModelNode.tsx`, no `Dashboard.tsx` switch to edit.
+
+```python
+# server/nodes/model/openrouter_chat_model/__init__.py
+class OpenRouterChatModel(ChatModelBase):
+ type = "openrouterChatModel"
+ metadata = NodeMetadata(
+ display_name="OpenRouter",
+ icon="lobehub:openrouter", # asset:<key>, lobehub:<brand>, or emoji
+ color="#6366F1",
+ component_kind="model", # routes to SquareNode in Dashboard.tsx
+ )
+
+ class Params(ChatModelBase.Params):
+ # provider-specific overrides; everything else inherits from ChatModelBase
+ ...
+```
+
+**Backend steps:**
+
1. **OpenAI-compatible provider** (DeepSeek, Kimi, Mistral pattern):
- Add an entry to `llm_defaults.json` with `base_url`, `default_model`, `detection_patterns`, `max_output_tokens._default`, `context_length._default`, `temperature_range`.
- Add the provider name to `NATIVE_PROVIDERS` in `factory.py` if you want `execute_chat()` to route natively.
- - Add to `PROVIDER_CONFIGS` in `services/ai.py` for LangChain agent path (reuse `ChatOpenAI`, set `base_url` in `create_model()`).
+ - The LangChain agent path needs no Python branching — `create_model()` reuses `ChatOpenAI` with `base_url` from `llm_defaults.json`.
2. **Custom-SDK provider** (Anthropic, Gemini pattern):
- - Create `services/llm/providers/<name>.py` implementing `LLMProvider` protocol.
+ - Create `services/llm/providers/<name>.py` implementing the `LLMProvider` protocol.
- Add a branch in `create_provider()` in `factory.py`.
- - Add the provider name to `_DEDICATED_PROVIDERS` and `NATIVE_PROVIDERS`.
- - Add config entry in `llm_defaults.json` (no `base_url` needed if the SDK handles URLs itself).
-
-3. **Frontend wiring** (in either case):
- - Add node definition in `client/src/nodeDefinitions/aiModelNodes.ts` using `createBaseChatModel()`.
- - Add to `CREDENTIAL_TO_PROVIDER` map in `ModelNode.tsx`.
- - Add credential entry in `CredentialsModal.tsx`.
+ - Add the provider name to the dedicated-provider set and `NATIVE_PROVIDERS`.
+ - Add a config entry in `llm_defaults.json` (no `base_url` needed if the SDK handles URLs itself).
+
+3. **Credentials + agent exposure:**
+ - Add a `Credential` subclass in `server/nodes/model/_credentials.py` — surfaces in the Credentials Modal automatically.
+ - To expose the provider in the **agent dropdown**, add its name to the `provider` Literal in `nodes/agent/ai_agent/__init__.py`, `chat_agent.py`, AND `_specialized.py`, and add the substring to `detect_ai_provider` in `server/constants.py` — otherwise an agent using it silently falls back to `'openai'`.
+
+### Key implementation files
+
+| File | Purpose |
+|---|---|
+| `server/nodes/model/<provider>_chat_model/__init__.py` | Plugin entry — metadata + Params + auto-registers |
+| `server/nodes/model/_credentials.py` | `Credential` subclass per provider |
+| `server/config/llm_defaults.json` | base_url + supported_params + temperature constraints (no hardcoded URLs in Python) |
+| `server/services/llm/providers/<provider>.py` | Native SDK provider (Protocol-based) |
+| `server/services/ai.py` | Routing: native chat path + LangChain agent path |
+| `client/src/Dashboard.tsx` | Generic `COMPONENT_BY_KIND` dispatch — no per-provider entry needed |
## Related Docs
diff --git a/docs-internal/node-logic-flows/README.md b/docs-internal/node-logic-flows/README.md
index be3cac92..a4f4dd82 100644
--- a/docs-internal/node-logic-flows/README.md
+++ b/docs-internal/node-logic-flows/README.md
@@ -12,8 +12,8 @@ match its doc here, and each doc must still describe what the code does.
- **Refactoring a node**: update the matching contract test in
`server/tests/nodes/test_<category>.py` *first*, then change the handler,
then update this doc.
-- **Rebuilding the index below**: run `node scripts/build-node-docs-index.js`
- (also runs in CI to fail builds when a node is missing a doc).
+- **The index below is maintained by hand** (there is no generator script
+ currently). When you add, rename, or remove a card, update its entry below.
## Conventions
@@ -26,7 +26,7 @@ match its doc here, and each doc must still describe what the code does.
## Index
-<!-- AUTO-GENERATED-INDEX-START -->
+<!-- INDEX-START (manually maintained; filenames are camelCase of the plugin `type`) -->
### ai_agents
@@ -42,12 +42,15 @@ match its doc here, and each doc must still describe what the code does.
- [Gemini Chat Model (`geminiChatModel`)](./ai_chat_models/geminiChatModel.md)
- [Groq Chat Model (`groqChatModel`)](./ai_chat_models/groqChatModel.md)
- [Kimi Chat Model (`kimiChatModel`)](./ai_chat_models/kimiChatModel.md)
+- [LM Studio Chat Model (`lmstudioChatModel`)](./ai_chat_models/lmstudioChatModel.md)
- [Mistral Chat Model (`mistralChatModel`)](./ai_chat_models/mistralChatModel.md)
+- [Ollama Chat Model (`ollamaChatModel`)](./ai_chat_models/ollamaChatModel.md)
- [OpenAI Chat Model (`openaiChatModel`)](./ai_chat_models/openaiChatModel.md)
- [OpenRouter Chat Model (`openrouterChatModel`)](./ai_chat_models/openrouterChatModel.md)
### ai_tools
+- [Agent Builder (`agentBuilder`)](./ai_tools/agentBuilder.md)
- [Calculator Tool (`calculatorTool`)](./ai_tools/calculatorTool.md)
- [Current Time Tool (`currentTimeTool`)](./ai_tools/currentTimeTool.md)
- [DuckDuckGo Search (`duckduckgoSearch`)](./ai_tools/duckduckgoSearch.md)
@@ -73,12 +76,53 @@ match its doc here, and each doc must still describe what the code does.
- [System Info (`systemInfo`)](./android/systemInfo.md)
- [WiFi Automation (`wifiAutomation`)](./android/wifiAutomation.md)
+### chat_utility
+
+- [Chat History (`chatHistory`)](./chat_utility/chatHistory.md)
+- [Chat Send (`chatSend`)](./chat_utility/chatSend.md)
+- [Console (`console`)](./chat_utility/console.md)
+- [Create Map (`gmaps_create`)](./chat_utility/gmaps_create.md)
+- [File Handler (`fileHandler`)](./chat_utility/fileHandler.md)
+- [Team Monitor (`teamMonitor`)](./chat_utility/teamMonitor.md)
+- [Text Generator (`textGenerator`)](./chat_utility/textGenerator.md)
+
+### code_fs_process
+
+- [File Modify (`fileModify`)](./code_fs_process/fileModify.md)
+- [File Read (`fileRead`)](./code_fs_process/fileRead.md)
+- [FS Search (`fsSearch`)](./code_fs_process/fsSearch.md)
+- [JavaScript Executor (`javascriptExecutor`)](./code_fs_process/javascriptExecutor.md)
+- [Monty Executor (`montyExecutor`)](./code_fs_process/montyExecutor.md)
+- [Process Manager (`processManager`)](./code_fs_process/processManager.md)
+- [Python Executor (`pythonExecutor`)](./code_fs_process/pythonExecutor.md)
+- [Shell (`shell`)](./code_fs_process/shell.md)
+- [TypeScript Executor (`typescriptExecutor`)](./code_fs_process/typescriptExecutor.md)
+
+### document
+
+- [Document Parser (`documentParser`)](./document/documentParser.md)
+- [Embedding Generator (`embeddingGenerator`)](./document/embeddingGenerator.md)
+- [File Downloader (`fileDownloader`)](./document/fileDownloader.md)
+- [HTTP Scraper (`httpScraper`)](./document/httpScraper.md)
+- [Text Chunker (`textChunker`)](./document/textChunker.md)
+- [Vector Store (`vectorStore`)](./document/vectorStore.md)
+
### email
- [Email Read (`emailRead`)](./email/emailRead.md)
- [Email Receive (`emailReceive`)](./email/emailReceive.md)
- [Email Send (`emailSend`)](./email/emailSend.md)
+### google_workspace
+
+- [Calendar (`googleCalendar`)](./google_workspace/googleCalendar.md)
+- [Contacts (`googleContacts`)](./google_workspace/googleContacts.md)
+- [Drive (`googleDrive`)](./google_workspace/googleDrive.md)
+- [Gmail (`googleGmail`)](./google_workspace/googleGmail.md)
+- [Gmail Receive (`googleGmailReceive`)](./google_workspace/googleGmailReceive.md)
+- [Sheets (`googleSheets`)](./google_workspace/googleSheets.md)
+- [Tasks (`googleTasks`)](./google_workspace/googleTasks.md)
+
### http_proxy
- [HTTP Request (`httpRequest`)](./http_proxy/httpRequest.md)
@@ -98,6 +142,7 @@ match its doc here, and each doc must still describe what the code does.
- [Android Control Agent (`android_agent`)](./specialized_agents/androidAgent.md)
- [Autonomous Agent (`autonomous_agent`)](./specialized_agents/autonomousAgent.md)
- [Claude Code Agent (`claude_code_agent`)](./specialized_agents/claudeCodeAgent.md)
+- [Codex (`codex_agent`)](./specialized_agents/codexAgent.md)
- [Coding Agent (`coding_agent`)](./specialized_agents/codingAgent.md)
- [Consumer Agent (`consumer_agent`)](./specialized_agents/consumerAgent.md)
- [Orchestrator Agent (`orchestrator_agent`)](./specialized_agents/orchestratorAgent.md)
@@ -146,4 +191,4 @@ match its doc here, and each doc must still describe what the code does.
- [Webhook Response (`webhookResponse`)](./workflow_triggers/webhookResponse.md)
- [Webhook Trigger (`webhookTrigger`)](./workflow_triggers/webhookTrigger.md)
-<!-- AUTO-GENERATED-INDEX-END -->
+<!-- INDEX-END -->
diff --git a/docs-internal/node-logic-flows/ai_agents/aiAgent.md b/docs-internal/node-logic-flows/ai_agents/aiAgent.md
index 4e912f5b..ebe4d230 100644
--- a/docs-internal/node-logic-flows/ai_agents/aiAgent.md
+++ b/docs-internal/node-logic-flows/ai_agents/aiAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | ai_agents / agent |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_agent`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/agent/ai_agent/__init__.py`](../../../server/nodes/agent/ai_agent/__init__.py) — dispatched via `BaseNode.execute()` + the `@Operation("execute")` method (`execute_op`). Pre-dispatch (edge walk + task inject + prompt fallback) in [`server/nodes/agent/_inline.py::prepare_agent_call`](../../../server/nodes/agent/_inline.py); LLM loop in `AIService.execute_agent`. |
| **Tests** | [`server/tests/nodes/test_ai_agents.py`](../../../server/tests/nodes/test_ai_agents.py) |
| **Skill (if any)** | n/a (the agent consumes skills via `input-skill`) |
| **Dual-purpose tool** | no |
@@ -16,8 +16,9 @@ and system message, optionally merges conversation history from a connected
via `chat_model.bind_tools`, and runs the tool-calling loop until the LLM produces a
final answer. All of the heavy lifting (LLM invocation, tool execution, memory
persistence) lives in `AIService.execute_agent`; the handler's job is purely
-to gather the connected-node payloads via `_collect_agent_connections` and
-forward them.
+to gather the connected-node payloads via
+`edge_walker.collect_agent_connections` (a 5-tuple: memory, skill, tool,
+input, task) and forward them.
## Inputs (handles)
@@ -31,17 +32,18 @@ forward them.
## Parameters
-| Name | Type | Default | Required | displayOptions.show | Description |
-|------|------|---------|----------|---------------------|-------------|
-| `provider` | options | `openai` | no | - | AI provider (openai, anthropic, gemini, groq, cerebras, openrouter, deepseek, kimi, mistral, xai). |
-| `model` | string | `""` | **yes** | - | Model id; loaded dynamically from the provider. |
-| `prompt` | string | `{{ $json.chatInput }}` | **yes** | - | User prompt; may reference upstream node outputs via templates. |
-| `systemMessage` | string | `You are a helpful assistant` | no | - | System prompt prepended to the conversation. |
-| `options.temperature` | number | `0.7` | no | - | 0-2. |
-| `options.maxTokens` | number | `4096` | no | - | 1-200000; clamped to model context in `AIService._resolve_max_tokens`. |
-| `options.thinkingEnabled` | boolean | `false` | no | - | Enables extended thinking / reasoning. |
-| `options.thinkingBudget` | number | `2048` | no | `thinkingEnabled=true` | Token budget (Claude, Gemini, Cerebras). |
-| `options.reasoningEffort` | options | `medium` | no | `thinkingEnabled=true` | `low` / `medium` / `high` (OpenAI o-series, Groq). |
+Source: `AIAgentParams` in [`ai_agent/__init__.py`](../../../server/nodes/agent/ai_agent/__init__.py).
+
+| Name | Type | Default | Required | Group | Description |
+|------|------|---------|----------|-------|-------------|
+| `prompt` | string (textarea, rows 4) | `""` | no | - | User prompt; may reference upstream node outputs via templates. Empty -> auto-prompt fallback from `input-main`. |
+| `provider` | Literal | `openai` | no | - | One of openai, anthropic, gemini, openrouter, groq, cerebras, deepseek, kimi, mistral, ollama, lmstudio. |
+| `model` | string | `""` | no | - | Model id; loaded dynamically from the provider. |
+| `system_message` | string (rows 3) | `You are a helpful assistant` | no | - | System prompt prepended to the conversation. |
+| `temperature` | number (optional) | `None` | no | options | 0-2, step 0.1. Unset falls through to `agent.default_temperature` in `llm_defaults.json`. |
+| `max_tokens` | int (optional) | `None` | no | options | 1-200000. Unset falls through to the per-model default; clamped in `AIService._resolve_max_tokens`. |
+
+> Note: `api_key` is **not** a declared field — credentials are auto-injected at execution time by `node_executor._inject_api_keys`. The thinking/reasoning controls are resolved by the LLM service layer from provider/model metadata, not exposed as `AIAgentParams` fields.
## Outputs (handles)
@@ -68,7 +70,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution
```mermaid
flowchart TD
- A[handle_ai_agent] --> B[_collect_agent_connections]
+ A[execute_op -> prepare_agent_call] --> B[edge_walker.collect_agent_connections]
B --> C{task_data?}
C -- yes --> D[Format task_context<br/>prepend to prompt]
D --> E{task status in<br/>completed/error AND<br/>tool_data?}
@@ -85,7 +87,7 @@ flowchart TD
## Decision Logic
-- **`_collect_agent_connections`** scans `context.edges` for edges whose target
+- **`edge_walker.collect_agent_connections`** scans `context.edges` for edges whose target
is this node, then routes each by `targetHandle`:
- `input-memory` + source type `simpleMemory` -> build `memory_data` dict.
`session_id` is auto-set to the agent's `node_id` unless the memory node
@@ -106,30 +108,36 @@ flowchart TD
- **Task completion short-circuit**: if `task_data` exists, `_format_task_context`
produces a plain-English wrapper which is prepended to the prompt. If the
task status is `completed` or `error` and any tools are present, **all tools
- are stripped** to prevent the LLM from delegating again.
+ are stripped** to prevent the LLM from delegating again. (See
+ `prepare_agent_call` in `_inline.py`.)
- **Auto-prompt fallback**: if `parameters.prompt` is empty after the task
- block, the handler extracts `input_data.message`, `input_data.text`, or
- `input_data.content` (first truthy wins) and assigns it to `prompt`. If the
+ block, `prepare_agent_call` extracts `input_data.message`, `input_data.text`,
+ or `input_data.content` (first truthy wins) and assigns it to `prompt`. If the
input is not a dict the whole value is stringified.
-- **Delegation to AIService**: the handler always returns whatever
- `ai_service.execute_agent` returns; success/error envelope shape is
- owned by the service, not the handler.
+- **Team-lead delegation**: for `orchestrator_agent` / `ai_employee`,
+ teammates connected via `input-teammates` are appended to `tool_data` as
+ `delegate_to_*` tools (not applicable to `aiAgent` itself).
+- **Delegation to AIService**: `execute_op` returns `response.get("result")`
+ on success; a `success=False` envelope raises `RuntimeError`. Typed openai
+ SDK failures surface as `NodeUserError` from inside `execute_agent`.
## Side Effects
-- **Database writes**: none directly in the handler. `AIService.execute_agent`
+- **Database writes**: none directly in `execute_op`. `AIService.execute_agent`
writes `token_usage_metrics` rows (via `CompactionService.track`), and
`memory_content` markdown changes persist back through the
`simpleMemory` node's `save_node_parameters` path owned by the service.
-- **Broadcasts**: `handle_ai_agent` acquires `StatusBroadcaster` and passes
+- **Broadcasts**: `prepare_agent_call` resolves `StatusBroadcaster` and passes
it into `execute_agent`. The service then emits `update_node_status`
(`thinking`, `executing_tool`, `success`, ...) plus `token_usage_update`
and potentially `compaction_starting` / `compaction_completed` events.
+ `BaseNode.execute()` additionally wraps the body in a `node.aiAgent.execute`
+ OpenTelemetry span + `log_context(node_id, node_type, workflow_id)`.
- **External API calls**: delegated to the native LLM provider SDKs (Anthropic,
OpenAI, Gemini, OpenRouter, xAI, DeepSeek, Kimi, Mistral) or the LangChain
fallback (Groq, Cerebras). Tool nodes may spawn their own HTTP or subprocess
calls via `execute_tool`.
-- **File I/O**: none in the handler. Filesystem tool nodes may read/write the
+- **File I/O**: none in `execute_op`. Filesystem tool nodes may read/write the
per-workflow workspace (`context.workspace_dir`).
- **Subprocess**: none directly. Tool executors (shell, process manager,
browser, code executors) spawn subprocesses.
@@ -143,7 +151,7 @@ flowchart TD
`PricingService`, `SkillLoader` (for `masterSkill` expansion).
- **Python packages**: `langchain-core`, `langchain-openai`,
`langchain-anthropic`, provider SDKs (`anthropic`, `openai`, `google-genai`).
-- **Environment variables**: none read directly by the handler.
+- **Environment variables**: none read directly by `execute_op`.
## Edge cases & known limits
@@ -157,10 +165,11 @@ flowchart TD
`writeTodos`) are removed when any task completes. This was introduced as a
"CRITICAL FIX" because binding tools while instructing the LLM not to use
them confused Gemini into returning empty tool call arrays.
-- **No pre-flight token budget check**: `_collect_agent_connections` can
- produce an arbitrarily long prompt (task context + memory + skills). The
- handler does **not** trim before calling `execute_agent`; the provider may
- reject with `prompt_too_long`. See `docs-internal/memory_compaction.md`.
+- **No pre-flight token budget check**: `collect_agent_connections` can
+ produce an arbitrarily long prompt (task context + memory + skills).
+ `prepare_agent_call` does **not** trim before calling `execute_agent`; the
+ provider may reject with `prompt_too_long`. See
+ [`../../memory_compaction.md`](../../memory_compaction.md).
- **Input fallback is field-ordered**: `message > text > content > str(dict)`.
An upstream node that outputs both `message` and `text` will never reach
`text`.
@@ -169,8 +178,9 @@ flowchart TD
## Related
-- **Shared collection helper**: `_collect_agent_connections` is used by every
- specialized-agent type (android_agent, coding_agent, web_agent, ...). See
+- **Shared collection helper**: `edge_walker.collect_agent_connections` +
+ `_inline.prepare_agent_call` are used by every specialized-agent type
+ (android_agent, coding_agent, web_agent, ...). See
[`chatAgent.md`](./chatAgent.md) for the same contract with a different
system prompt.
- **Memory node**: [`simpleMemory.md`](./simpleMemory.md)
diff --git a/docs-internal/node-logic-flows/ai_agents/chatAgent.md b/docs-internal/node-logic-flows/ai_agents/chatAgent.md
index 73157d65..99e8a85b 100644
--- a/docs-internal/node-logic-flows/ai_agents/chatAgent.md
+++ b/docs-internal/node-logic-flows/ai_agents/chatAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | ai_agents / agent |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/agent/chat_agent/__init__.py`](../../../server/nodes/agent/chat_agent/__init__.py) — dispatched via `BaseNode.execute()` + `@Operation("execute")` (`execute_op`). Pre-dispatch in [`_inline.py::prepare_agent_call`](../../../server/nodes/agent/_inline.py); LLM loop in `AIService.execute_chat_agent`. |
| **Tests** | [`server/tests/nodes/test_ai_agents.py`](../../../server/tests/nodes/test_ai_agents.py) |
| **Skill (if any)** | n/a (consumes skills via `input-skill`) |
| **Dual-purpose tool** | no |
@@ -11,16 +11,18 @@
## Purpose
`chatAgent` (display name **Zeenie**) is the conversational variant of
-`aiAgent`. Same connection model, same `_collect_agent_connections` helper,
-same tool-calling stack - it differs from `aiAgent` in the frontend icon /
-system prompt and in the service method it calls
-(`AIService.execute_chat_agent` instead of `AIService.execute_agent`). It is
-also the handler used by every **specialized agent** node
+`aiAgent`. Same connection model, same `edge_walker.collect_agent_connections`
+helper + `_inline.prepare_agent_call` pre-dispatch, same tool-calling stack -
+it differs from `aiAgent` in the frontend icon / default system message and in
+the service method it calls (`AIService.execute_chat_agent` instead of
+`AIService.execute_agent`). Every **specialized agent** node
(`android_agent`, `coding_agent`, `web_agent`, `task_agent`, `social_agent`,
`travel_agent`, `tool_agent`, `productivity_agent`, `payments_agent`,
-`consumer_agent`, `autonomous_agent`, `orchestrator_agent`, `ai_employee`) -
-the node registry wires all of those to `handle_chat_agent` via
-`functools.partial`.
+`consumer_agent`, `autonomous_agent`, `orchestrator_agent`, `ai_employee`)
+takes the **same execution path** — they subclass
+[`SpecializedAgentBase`](../../../server/nodes/agent/_specialized.py) whose
+`execute_op` also runs `prepare_agent_call` + `execute_chat_agent`. Each is its
+own plugin folder; there is no `functools.partial` wiring anymore.
## Inputs (handles)
@@ -35,13 +37,15 @@ the node registry wires all of those to `handle_chat_agent` via
## Parameters
-Identical to [`aiAgent`](./aiAgent.md) (`provider`, `model`, `prompt`,
-`systemMessage`, `options.*`). The only frontend differences:
+`ChatAgentParams` mirrors [`aiAgent`](./aiAgent.md)'s `AIAgentParams`
+(`provider` Literal, `model`, `prompt`, `system_message`, `temperature` /
+`max_tokens` in the `options` group). Differences:
-- `prompt` default is `""` (empty) rather than `{{ $json.chatInput }}` - so
- Zeenie relies on the auto-prompt fallback when wired to a chat trigger.
-- `systemMessage` default is `"You are a helpful assistant."` (Zeenie persona
- is layered on top by connected skills).
+- `prompt` default `""` (empty) — Zeenie relies on the auto-prompt fallback
+ when wired to a chat trigger (placeholder: "Optional: leave empty to use
+ connected input").
+- `system_message` default `""` (vs `aiAgent`'s "You are a helpful assistant")
+ — the Zeenie persona is layered on top by connected skills.
## Outputs (handles)
@@ -53,7 +57,7 @@ Identical to [`aiAgent`](./aiAgent.md) (`provider`, `model`, `prompt`,
```mermaid
flowchart TD
- A[handle_chat_agent] --> B[_collect_agent_connections]
+ A[execute_op -> prepare_agent_call] --> B[edge_walker.collect_agent_connections]
B --> C{task_data?}
C -- yes --> D[Prepend task_context to prompt]
D --> E{status completed/error<br/>AND tool_data?}
@@ -64,7 +68,7 @@ flowchart TD
G -- yes --> H[prompt := input_data.message<br/>or .text or .content or str]
G -- no --> I
H --> I{node_type in<br/>TEAM_LEAD_TYPES?}
- I -- yes --> J[_collect_teammate_connections<br/>-> append each teammate to tool_data]
+ I -- yes --> J[collect_teammate_connections<br/>-> append each teammate to tool_data]
I -- no --> K
J --> K[await ai_service.execute_chat_agent<br/>memory/skill/tool/broadcaster/context/db]
K --> L[Return envelope]
@@ -72,21 +76,22 @@ flowchart TD
## Decision Logic
-- **Connection collection** is delegated to `_collect_agent_connections`; same
- rules as `aiAgent` (see that doc for memory session derivation,
- `masterSkill` expansion, Android toolkit, child-agent tool discovery).
-- **Task context injection** mirrors `aiAgent`: `_format_task_context` wraps
+- **Connection collection** is delegated to
+ `edge_walker.collect_agent_connections`; same rules as `aiAgent` (see that doc
+ for memory session derivation, `masterSkill` expansion, Android toolkit,
+ child-agent tool discovery).
+- **Task context injection** mirrors `aiAgent`: `format_task_context` wraps
the task result as a plain-English instruction that the LLM must "report
naturally", then all tools are stripped if the task has already completed
- or errored.
+ or errored. (In `_inline.prepare_agent_call`.)
- **Auto-prompt fallback** is identical to `aiAgent` - `message` wins over
`text` which wins over `content`, falling back to `str(input_data)`.
-- **Team mode** (`orchestrator_agent`, `ai_employee`): after collection, the
- handler calls `_collect_teammate_connections(node_id, context, database)`
- to find nodes wired to `input-teammates`. Each teammate whose `node_type`
- is in `AI_AGENT_TYPES` is appended to `tool_data` as a synthetic tool
- entry. The AIService then exposes them to the LLM as `delegate_to_<type>`
- tools (see [Agent Teams](../../agent_teams.md)).
+- **Team mode** (`orchestrator_agent`, `ai_employee`): after collection,
+ `prepare_agent_call` calls `collect_teammate_connections(node_id, context,
+ database)` to find nodes wired to `input-teammates`. Each teammate is
+ appended to `tool_data` as a synthetic entry (with `child_tools` describing
+ its own `input-tools` neighbours). The AIService then exposes them to the LLM
+ as `delegate_to_<type>` tools (see [Agent Teams](../../agent_teams.md)).
## Side Effects
@@ -111,8 +116,9 @@ flowchart TD
## Edge cases & known limits
-- **Specialized agents inherit every quirk**: any bug in `handle_chat_agent`
- (e.g. the blanket tool-strip on task completion) affects 13 node types.
+- **Specialized agents inherit every quirk**: any bug in `prepare_agent_call`
+ or `execute_chat_agent` (e.g. the blanket tool-strip on task completion)
+ affects all 13 specialized node types plus the two team leads.
- **Team-lead teammates are appended after existing tools**: if a parent
wires both regular tool nodes and teammates, tool order depends on edge
scan order (not stable across clients).
@@ -122,11 +128,12 @@ flowchart TD
at delegate-time, not at collection-time.
- **Input fallback field order is identical** to `aiAgent`: `message` >
`text` > `content` > `str(dict)`.
-- **Empty `teammates` on a team-lead is benign** - the handler simply runs
- without delegation tools, so an `orchestrator_agent` with nothing wired to
- `input-teammates` behaves exactly like `chatAgent`.
-- **`rlm_agent` and `claude_code_agent` do NOT route here** - they have dedicated
- handlers. All other specialized agents do.
+- **Empty `teammates` on a team-lead is benign** - `prepare_agent_call` simply
+ runs without delegation tools, so an `orchestrator_agent` with nothing wired
+ to `input-teammates` behaves exactly like `chatAgent`.
+- **`rlm_agent`, `claude_code_agent`, `codex_agent` do NOT take this path** -
+ they have their own plugin execute methods (RLMService / CLI agent runtime).
+ The other 13 specialized agents share `execute_chat_agent`.
## Related
diff --git a/docs-internal/node-logic-flows/ai_agents/simpleMemory.md b/docs-internal/node-logic-flows/ai_agents/simpleMemory.md
index 0c821ac3..0c440a68 100644
--- a/docs-internal/node-logic-flows/ai_agents/simpleMemory.md
+++ b/docs-internal/node-logic-flows/ai_agents/simpleMemory.md
@@ -2,8 +2,8 @@
| Field | Value |
|------|-------|
-| **Category** | ai_agents / memory (UI group: `['tool', 'memory']`) |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_simple_memory`](../../../server/services/handlers/ai.py) |
+| **Category** | ai_agents / memory (group: `('tool', 'memory')`, `component_kind = "model"`) |
+| **Backend handler** | [`server/nodes/skill/simple_memory/__init__.py`](../../../server/nodes/skill/simple_memory/__init__.py) — dispatched via `BaseNode.execute()` + the `@Operation("read")` method (`read`). Agent-side consumption via [`edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) / `_build_memory_entry`. |
| **Tests** | [`server/tests/nodes/test_ai_agents.py`](../../../server/tests/nodes/test_ai_agents.py) |
| **Skill (if any)** | n/a |
| **Dual-purpose tool** | no |
@@ -12,19 +12,20 @@
`simpleMemory` is a **passive configuration node** that holds conversation
history for connected agents. It has no inputs and is not meant to be run on
-its own - it has no Run button in the UI. When an `aiAgent` / `chatAgent`
-(or any specialized agent) executes, `_collect_agent_connections` reads this
-node's saved parameters (`memoryContent`, `windowSize`, `longTermEnabled`,
-etc.) and forwards them to `AIService.execute_agent` /
-`execute_chat_agent`. The service parses the markdown, feeds the messages to
-the LLM, appends the new exchange, trims to window, and saves the updated
-markdown back to the memory node via `database.save_node_parameters`.
-
-The actual handler (`handle_simple_memory`) is only invoked when a user
-explicitly hits Run on the node; it returns an inspector-style snapshot of
-the `services.memory_store` in-memory session, **which is a separate legacy
-store from the markdown pipeline used by the agents**. See "Edge cases"
-below.
+its own — `ui_hints.hideRunButton` is set. When an `aiAgent` / `chatAgent`
+(or any specialized agent) executes, `edge_walker.collect_agent_connections`
+reads this node's saved parameters (`memory_content`, `window_size`,
+`long_term_enabled`, etc.) via `_build_memory_entry` and forwards them to
+`AIService.execute_agent` / `execute_chat_agent`. The service parses the
+markdown, feeds the messages to the LLM, appends the new exchange, trims to
+window, and saves the updated markdown back to the memory node via
+`database.save_node_parameters`.
+
+The plugin's own `@Operation("read")` method is only invoked when a user
+explicitly hits Run (or via direct dispatch); it returns an inspector-style
+snapshot of the `services.memory_store` in-memory session, **which is a
+separate store from the markdown `memory_content` pipeline used by the
+agents**. See "Edge cases" below.
## Inputs (handles)
@@ -32,103 +33,108 @@ _None._ `simpleMemory` has no incoming handles.
## Parameters
+Source: `SimpleMemoryParams` in
+[`simple_memory/__init__.py`](../../../server/nodes/skill/simple_memory/__init__.py).
+All field names are snake_case (Pydantic params are the post-Wave-11 source of
+truth; the old `memoryType` / `clearOnRun` / buffer-vs-window split was removed
+— trimming is always windowed and the UI exposes a manual "Clear Memory"
+button).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `sessionId` | string | `""` | no | - | Session override. Empty or `default` -> agent node_id is used instead (see Decision Logic). |
-| `windowSize` | number | `100` | no | - | Message *pairs* to keep in short-term memory; 1-100. |
-| `memoryContent` | string | `# Conversation History\n\n*No messages yet.*\n` | no | - | Markdown body (`### **Human** (ts)` / `### **Assistant** (ts)` blocks). Edited in-place in the UI. |
-| `longTermEnabled` | boolean | `false` | no | - | When true, trimmed messages are archived to an `InMemoryVectorStore`. |
-| `retrievalCount` | number | `3` | no | `longTermEnabled=true` | Relevant messages to retrieve at query time (1-10). |
+| `session_id` | string | `""` | no | - | Session override. Empty -> agent node_id is used by the agent-side collection; the `read` op resolves empty to `"default"`. |
+| `window_size` | int | `100` | no | - | Message *pairs* to keep in short-term memory; 1-100. |
+| `memory_content` | string (code editor, markdown, rows 15) | `# Conversation History\n\n*No messages yet.*\n` | no | - | Markdown body (`### **Human** (ts)` / `### **Assistant** (ts)` blocks). Edited in-place in the UI. |
+| `long_term_enabled` | boolean | `false` | no | - | When true, trimmed messages are archived to an `InMemoryVectorStore`. |
+| `retrieval_count` | int | `3` | no | `long_term_enabled=[true]` | Relevant messages to retrieve at query time (1-10). |
+| `last_session_id` | string (optional, **hidden**) | `None` | no | - | Internal/display-only: last Claude session UUID. `claude_code_agent` no longer reads it; cleared when memory is wiped. |
Parameters consumed when an agent pulls the memory:
| Name | Where it comes from | Where it is read |
|------|---------------------|------------------|
-| `sessionId`, `windowSize`, `memoryContent`, `longTermEnabled`, `retrievalCount` | `database.get_node_parameters(source_node_id)` inside `_collect_agent_connections` | Packed into `memory_data` dict and forwarded to `AIService`. |
+| `memory_content`, `window_size`, `long_term_enabled`, `retrieval_count`, `last_session_id` | `database.get_node_parameters(source_node_id)` inside `edge_walker._build_memory_entry` | Packed into the `memory_data` dict and forwarded to `AIService`. |
-Parameters consumed when the node is run directly:
+Parameters consumed when the node is run directly (`@Operation("read")`):
-| Name | Type | Default | Description |
-|------|------|---------|-------------|
-| `sessionId` | string | `default` | Session key used for `services.memory_store`. |
-| `memoryType` | string | `buffer` | Either `buffer` (all messages) or `window`. |
-| `windowSize` | number | `10` | Window size; only applied when `memoryType == 'window'`. |
-| `clearOnRun` | boolean | `false` | When true, clears the session before returning. |
+| Name | Default | Description |
+|------|---------|-------------|
+| `session_id` | resolves to `"default"` when empty | Session key used for `services.memory_store.get_messages`. |
+| `window_size` | `100` | Windowing applied to the message log. |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-memory` (`main` type) | object | Snapshot of the in-memory session (see below). |
+| `output-memory` (output, top, role `memory`) | object | Snapshot of the in-memory session (see below). |
-### Output payload (from `handle_simple_memory`)
+### Output payload (`SimpleMemoryOutput` from the `read` op)
```ts
{
- session_id: string;
- messages: Array<{ role: 'human' | 'ai'; content: string; timestamp: string }>;
- message_count: number;
- memory_type: 'buffer' | 'window';
- window_size: number | null;
+ memory_content?: string; // echoes the editable markdown param
+ message_count?: number;
+ session_id?: string;
+ messages?: any[]; // from services.memory_store.get_messages
+ window_size?: number;
+ // model_config extra="allow"
}
```
-Wrapped in the standard envelope: `{ success: true, result: <payload> }`.
+Serialized through `BaseNode._serialize_result` and wrapped in the standard
+envelope: `{ success: true, result: <payload>, execution_time }`.
## Logic Flow
```mermaid
flowchart TD
subgraph passiveUse["Passive use (agent collection)"]
- P1[Agent handler scans edges] --> P2{target=input-memory<br/>source type=simpleMemory?}
+ P1[edge_walker scans edges] --> P2{target=input-memory<br/>source type=simpleMemory?}
P2 -- yes --> P3[database.get_node_parameters source_id]
- P3 --> P4[memory_session_id = node_id<br/>unless sessionId explicit]
- P4 --> P5[Build memory_data dict<br/>memoryContent/windowSize/...]
+ P3 --> P4[memory_session_id = node_id<br/>unless session_id explicit]
+ P4 --> P5[_build_memory_entry: memory_data dict<br/>memory_content/window_size/...]
P5 --> P6[Forward to AIService]
end
- subgraph directRun["Direct run (handle_simple_memory)"]
- D1[handle_simple_memory] --> D2[Read sessionId/memoryType/windowSize/clearOnRun]
- D2 --> D3{clearOnRun?}
- D3 -- yes --> D4[memory_store.clear_session]
- D3 -- no --> D5
- D4 --> D5[memory_store.get_messages<br/>session_id window_size]
- D5 --> D6[Return success envelope]
+ subgraph directRun["Direct run (Operation read)"]
+ D1[read op] --> D2[session_id = session_id or default<br/>window_size]
+ D2 --> D5[memory_store.get_messages<br/>session_id window_size]
+ D5 --> D6[Return SimpleMemoryOutput envelope]
end
```
## Decision Logic
-- **Passive collection session key**: if the memory node's `sessionId` is
- empty or literally `"default"`, the agent's own `node_id` is used as the
- session key. This is what makes two agents wired to the same memory node
- keep **separate** histories unless the user explicitly overrides
- `sessionId`.
-- **Instruction source of truth**: `memoryContent` in the DB is authoritative.
+- **Passive collection session key**: if the memory node's `session_id` is
+ empty (or `"default"`), the agent's own `node_id` is used as the session key.
+ This is what makes two agents wired to the same memory node keep **separate**
+ histories unless the user explicitly overrides `session_id`.
+- **Instruction source of truth**: `memory_content` in the DB is authoritative.
UI edits persist there via `save_node_parameters`. The agent side
- re-serialises after each turn through `AIService._append_to_memory_markdown`
- + `_trim_markdown_window`.
-- **Window semantics**: `windowSize` counts *pairs* (Human + Assistant), so
- `_trim_markdown_window` keeps `window_size * 2` blocks. Trimmed blocks are
+ re-serialises after each turn through
+ `services.memory.append_to_memory_markdown` + `trim_markdown_window`.
+- **Window semantics**: `window_size` counts *pairs* (Human + Assistant), so
+ `trim_markdown_window` keeps `window_size * 2` blocks. Trimmed blocks are
returned as raw text for optional vector-store archival.
-- **Long-term retrieval**: `_get_memory_vector_store(session_id)` lazily
- creates an `InMemoryVectorStore` with `HuggingFaceEmbeddings(BAAI/bge-small-en-v1.5)`
- the first time it is requested for that session. On `ImportError` (e.g.
- `langchain_huggingface` missing) the function returns `None` and the agent
- silently skips archival - **no error is surfaced**.
-- **Direct-run memory store is different**: `handle_simple_memory` pulls
- from `services.memory_store._sessions`, **not** from the markdown
- `memoryContent`. A user who clicks Run on a `simpleMemory` node that is
- actively used by an agent will see the `memory_store` view, which may be
- empty even though `memoryContent` contains history.
+- **Long-term retrieval**: `services.memory.get_memory_vector_store(session_id)`
+ lazily creates an `InMemoryVectorStore` with
+ `HuggingFaceEmbeddings(BAAI/bge-small-en-v1.5)` the first time it is requested
+ for that session. On `ImportError` (e.g. `langchain_huggingface` missing) the
+ function returns `None` and the agent silently skips archival - **no error is
+ surfaced**.
+- **Direct-run memory store is different**: the `read` op pulls from
+ `services.memory_store` (via `get_messages`), **not** from the markdown
+ `memory_content`. A user who clicks Run on a `simpleMemory` node actively
+ used by an agent sees the `memory_store` view, which may be empty even though
+ `memory_content` contains history.
## Side Effects
-- **Database writes**: none in the handler. The agent pipeline writes the
+- **Database writes**: none in the `read` op. The agent pipeline writes the
updated markdown back to this node's `node_parameters` row via
`database.save_node_parameters` after each turn.
- **Broadcasts**: none. The node is passive.
-- **External API calls**: none in the handler. When `longTermEnabled` is set
+- **External API calls**: none in the `read` op. When `long_term_enabled` is set
and the agent archives, `HuggingFaceEmbeddings` loads the
`BAAI/bge-small-en-v1.5` model locally (no network once cached).
- **File I/O**: none. The vector store is `InMemoryVectorStore` - not
@@ -146,33 +152,32 @@ flowchart TD
## Edge cases & known limits
-- **Two storage systems for one node**. The markdown `memoryContent` (used
- by agents) and the `services.memory_store._sessions` dict (used by the
- direct-run handler) are **independent**. Clicking Run on the node does not
+- **Two storage systems for one node**. The markdown `memory_content` (used
+ by agents) and the `services.memory_store` session log (used by the
+ `read` op) are **independent**. Clicking Run on the node does not
reflect what the agents see, and vice-versa. This is a known architectural
seam noted in the CLAUDE.md overview.
-- **Vector store is in-memory only**. `_memory_vector_stores` is a module-
- global dict keyed by session id. A server restart empties it; long-term
- memory is not actually durable without external persistence.
+- **Vector store is in-memory only**. The per-session cache in
+ `services.memory.vector_store` is a module-global dict keyed by session id.
+ A server restart empties it; long-term memory is not actually durable
+ without external persistence.
- **`HuggingFaceEmbeddings` import is silent-catch**. If the dependency is
- missing at runtime, `_get_memory_vector_store` logs a warning and returns
+ missing at runtime, `get_memory_vector_store` logs a warning and returns
`None`; the agent continues without archival and the user never learns.
-- **No token budgeting**. `windowSize` is a message-pair count, not a token
+- **No token budgeting**. `window_size` is a message-pair count, not a token
count. A long history below the window threshold can still exceed the
model's context window; see the "Prompt Too Long" note in CLAUDE.md.
-- **`clearOnRun=true` only affects direct-run**, not the markdown used by
- connected agents.
-- **Markdown parser is regex-based**. `_parse_memory_markdown` matches
+- **Markdown parser is regex-based**. `parse_memory_markdown` matches
`### **Human**` / `### **Assistant**` exactly. If a user edits the UI
content to a non-conforming header the messages will silently be dropped.
-- **`windowSize` UI cap is 100**, but the handler accepts any int. The
- agent-side `_trim_markdown_window` trusts whatever the parameter contains.
+- **`window_size` is Pydantic-clamped 1-100** by the field constraint; the
+ agent-side `trim_markdown_window` trusts whatever value is passed.
- **Passive node, no envelope on read paths**. When consumed by an agent,
this node never produces its own envelope - it only supplies parameters.
## Related
- **Consumer nodes**: [`aiAgent`](./aiAgent.md), [`chatAgent`](./chatAgent.md),
- every specialized agent that routes to `handle_chat_agent`.
+ every specialized agent that routes through `execute_chat_agent`.
- **Architecture docs**: [Memory Compaction](../../memory_compaction.md),
[Agent Architecture](../../agent_architecture.md)
diff --git a/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.md
index 0382ce65..47c410b1 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/anthropic_chat_model/__init__.py`](../../../server/nodes/model/anthropic_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | yes - tool name `anthropic_chat_model` (advisor; `usable_as_tool = True`, group `('model', 'tool')`) |
## Purpose
-Single-turn chat completion against the Anthropic Messages API (Claude family). Shares the same handler as all other chat-model nodes (`handle_ai_chat_model`); provider-specific branching lives inside `AIService.execute_chat` and the native `services/llm/providers/anthropic.py`.
+Single-turn chat completion against the Anthropic Messages API (Claude family). The `ChatModelBase.chat` operation (in `_base.py`) calls `AIService.execute_chat` for every chat-model node; provider-specific branching lives inside `execute_chat` and the native `services/llm/providers/anthropic.py`. Also wired as an AI-agent advisor tool (`usable_as_tool = True`) so an agent can consult a stronger model mid-task.
## Inputs (handles)
@@ -24,21 +24,23 @@ Single-turn chat completion against the Anthropic Messages API (Claude family).
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes (non-empty) | - | User message |
-| `systemMessage` / `systemPrompt` / `system_prompt` | string | `""` | no | - | System prompt |
-| `model` | string | injected from stored models, fallback `gpt-3.5-turbo` (Anthropic override via stored models list) | no | - | Anthropic model ID - **hyphenated** form (e.g. `claude-sonnet-4-6`, NOT `claude-sonnet-4.6`) |
-| `temperature` | number | 0-1 range | no | - | Clamped to 0-1 for Claude; forced to 1 when thinking is enabled |
-| `maxTokens` | number | model default | no | - | **Must be greater than `thinkingBudget`** when thinking is enabled |
-| `topP` | number | - | no | - | Nucleus sampling |
-| `thinkingEnabled` | boolean | false | no | - | Enable extended thinking |
-| `thinkingBudget` | number | 2048 | no | `thinkingEnabled=[true]` | 1024-16000 tokens allocated to internal reasoning |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('anthropic', 'default')` |
-| `options` | object | `{}` | no | - | Flattened into top-level params |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected from stored models) | no | - | Anthropic model ID - **hyphenated** form (e.g. `claude-sonnet-4-6`, NOT `claude-sonnet-4.6`) |
+| `temperature` | number\|null | `null` -> falls through to `agent.default_temperature` | no | - | Clamped to 0-1 for Claude; forced to 1 when thinking is enabled |
+| `max_tokens` | number\|null | `null` -> per-model default | no | - | 1-200000. **Must be greater than `thinking_budget`** when thinking is enabled |
+| `top_p` | number\|null | `1.0` | no | - | Nucleus sampling (0-1) |
+| `top_k` | number\|null | `40` | no | - | 1-100; Anthropic-specific |
+| `thinking_enabled` | boolean | `false` | no | - | Enable extended thinking |
+| `thinking_budget` | number\|null | `2048` | no | `thinking_enabled=[true]` | 1024-16000 tokens allocated to internal reasoning |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('anthropic', 'default')` |
+
+(Field names are snake_case on the `AnthropicChatModelParams` Pydantic model; `model_config = ConfigDict(extra="ignore")` drops unknown keys.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload |
### Output payload
@@ -61,8 +63,8 @@ Wrapped in `{ success, node_id, node_type, result, execution_time }`.
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[_prepare_parameters]
- B --> C[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
+ B --> C[get_ai_service]
C --> D[AIService.execute_chat]
D --> E{api_key + non-empty prompt?}
E -- no --> X[ValueError -> error envelope]
diff --git a/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md
index 99bc2bac..385cff46 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/cerebras_chat_model/__init__.py`](../../../server/nodes/model/cerebras_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | no (group `('model',)`) |
## Purpose
-Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1, GPT-OSS-120b, Qwen-3-235b. Shares `handle_ai_chat_model`. Like Groq, routes through the **LangChain fallback** (`is_native_provider('cerebras')` is False).
+Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1, GPT-OSS-120b, Qwen-3-235b. The `ChatModelBase.chat` operation calls `AIService.execute_chat`. Like Groq, routes through the **LangChain fallback** (`is_native_provider('cerebras')` is False).
## Inputs (handles)
@@ -24,20 +24,23 @@ Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1,
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes | - | User message |
-| `systemMessage` | string | `""` | no | - | System prompt |
-| `model` | string | injected | no | - | e.g. `llama3.1-8b`, `gpt-oss-120b`, `qwen-3-235b-a22b` |
-| `temperature` | number | 0-1.5 | no | - | Narrower range than OpenAI (0-1.5 rather than 0-2) |
-| `maxTokens` | number | up to 8K | no | - | |
-| `topP` | number | - | no | - | |
-| `thinkingEnabled` | boolean | false | no | - | Only Qwen-3-235b supports format-based reasoning |
-| `reasoningFormat` | options | `parsed` | no | `thinkingEnabled=[true]` | `parsed` / `hidden` - same semantics as Groq Qwen |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('cerebras', 'default')` |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | e.g. `llama3.1-8b`, `gpt-oss-120b`, `qwen-3-235b-a22b` |
+| `temperature` | number\|null | `null` | no | - | Narrower range than OpenAI (0-1.5 rather than 0-2) |
+| `max_tokens` | number\|null | `null` (up to 8K) | no | - | 1-200000 |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `thinking_enabled` | boolean | `false` | no | - | Only Qwen-3-235b supports format-based reasoning |
+| `thinking_budget` | number\|null | `2048` | no | `thinking_enabled=[true]` | 1024-16000 (Cerebras Qwen budget) |
+| `reasoning_format` | enum | `parsed` | no | - | `parsed` / `hidden` - same semantics as Groq Qwen (inherited base field) |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('cerebras', 'default')` |
+
+(Field names are snake_case on `CerebrasChatModelParams`; unknown keys ignored.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output; standard envelope payload |
### Output payload
@@ -58,7 +61,7 @@ Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1,
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
diff --git a/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.md
index f381b595..4c5ae5c7 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/deepseek_chat_model/__init__.py`](../../../server/nodes/model/deepseek_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | no (group `('model',)`) |
## Purpose
-DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / `deepseek-reasoner` remain as legacy aliases (deprecate 2026-07-24). Uses the OpenAI-compatible DeepSeek endpoint via the `services/llm/providers` layer (native path). Shares `handle_ai_chat_model`.
+DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / `deepseek-reasoner` remain as legacy aliases (deprecate 2026-07-24). Uses the OpenAI-compatible DeepSeek endpoint via the `services/llm/providers` layer (native path). The `ChatModelBase.chat` operation calls `AIService.execute_chat`. The plugin docstring describes `deepseek-chat`/`deepseek-reasoner` (V3) but the registry description in `__init__.py` predates the V4 rename - the card's V4 listing reflects current `llm_defaults.json`.
## Inputs (handles)
@@ -24,18 +24,22 @@ DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / `
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes | - | User message |
-| `systemMessage` | string | `""` | no | - | System prompt |
-| `model` | string | injected | no | - | `deepseek-v4-flash` / `deepseek-v4-pro`; `deepseek-chat` / `deepseek-reasoner` legacy aliases (reasoner = always-on CoT) |
-| `temperature` | number | 0-2 | no | - | |
-| `maxTokens` | number | 8-64K | no | - | |
-| `topP` | number | - | no | - | |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('deepseek', 'default')` |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | `deepseek-v4-flash` / `deepseek-v4-pro`; `deepseek-chat` / `deepseek-reasoner` legacy aliases (reasoner = always-on CoT) |
+| `temperature` | number\|null | `null` | no | - | 0-2 |
+| `max_tokens` | number\|null | `null` (8-64K) | no | - | 1-200000 |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `frequency_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 (DeepSeek-specific) |
+| `presence_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 (DeepSeek-specific) |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('deepseek', 'default')` |
+
+(Field names are snake_case on `DeepseekChatModelParams`; unknown keys ignored.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output; standard envelope payload |
### Output payload
@@ -56,7 +60,7 @@ DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / `
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
diff --git a/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md
index 65366c6e..9df2c6c1 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/gemini_chat_model/__init__.py`](../../../server/nodes/model/gemini_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | yes - tool name `gemini_chat_model` (advisor; `usable_as_tool = True`, group `('model', 'tool')`) |
## Purpose
-Single-turn chat completion against Google's Gemini API (google-genai native SDK). Shares the common `handle_ai_chat_model` handler.
+Single-turn chat completion against Google's Gemini API (google-genai native SDK). The `ChatModelBase.chat` operation calls `AIService.execute_chat`. Also wired as an AI-agent advisor tool (`usable_as_tool = True`) so an agent can consult Gemini mid-task.
## Inputs (handles)
@@ -24,22 +24,24 @@ Single-turn chat completion against Google's Gemini API (google-genai native SDK
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes (non-empty) | - | User message |
-| `systemMessage` | string | `""` | no | - | System prompt |
-| `model` | string | injected | no | - | e.g. `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-3-pro` |
-| `temperature` | number | 0-2 | no | - | 0-2 range |
-| `maxTokens` | number | clamped to model ceiling | no | - | Up to 65K output |
-| `topP` | number | - | no | - | |
-| `thinkingEnabled` | boolean | false | no | - | Enable thinking (2.5 Pro/Flash, 3.x) |
-| `thinkingBudget` | number | 2048 | no | `thinkingEnabled=[true]` | Token budget for internal reasoning |
-| `safetySettings` | object | provider defaults | no | - | Gemini safety category thresholds |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('gemini', 'default')` |
-| `options` | object | `{}` | no | - | Flattened |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | e.g. `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-3-pro` |
+| `temperature` | number\|null | `null` | no | - | 0-2 range |
+| `max_tokens` | number\|null | `null` (clamped to ceiling) | no | - | 1-200000; up to 65K output |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `top_k` | number\|null | `40` | no | - | 1-100; Gemini-specific |
+| `thinking_enabled` | boolean | `false` | no | - | Enable thinking (2.5 Pro/Flash, 3.x) |
+| `thinking_budget` | number\|null | `2048` | no | `thinking_enabled=[true]` | 1024-16000 token budget for internal reasoning |
+| `safety_settings` | enum | `default` | no | - | `default` / `strict` / `permissive` |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('gemini', 'default')` |
+
+(Field names are snake_case on `GeminiChatModelParams`; unknown keys ignored.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output; standard envelope payload |
### Output payload
@@ -60,8 +62,8 @@ Single-turn chat completion against Google's Gemini API (google-genai native SDK
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[_prepare_parameters]
- B --> C[handle_ai_chat_model -> AIService.execute_chat]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
+ B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
D -- yes --> E[detect_ai_provider -> 'gemini']
diff --git a/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md
index 4e46edf2..70268fbb 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/groq_chat_model/__init__.py`](../../../server/nodes/model/groq_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | no (group `('model',)`) |
## Purpose
-Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen3-32b, GPT-OSS. Shares `handle_ai_chat_model`; unlike OpenAI/Anthropic/Gemini, Groq routes through the **LangChain fallback** (`create_model` + `chat_model.invoke`) because `is_native_provider('groq')` returns False.
+Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen3-32b, GPT-OSS. The `ChatModelBase.chat` operation calls `AIService.execute_chat`; unlike OpenAI/Anthropic/Gemini, Groq routes through the **LangChain fallback** (`create_model` + `chat_model.invoke`) because `is_native_provider('groq')` returns False.
## Inputs (handles)
@@ -24,20 +24,22 @@ Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes | - | User message |
-| `systemMessage` | string | `""` | no | - | System prompt |
-| `model` | string | injected | no | - | e.g. `llama-3.1-70b-versatile`, `qwen/qwen3-32b`, `groq/compound-beta` |
-| `temperature` | number | 0-2 | no | - | |
-| `maxTokens` | number | 8-131K per model | no | - | |
-| `topP` | number | - | no | - | |
-| `thinkingEnabled` | boolean | false | no | - | Only Qwen3-32b supports reasoning |
-| `reasoningFormat` | options | `parsed` | no | `thinkingEnabled=[true]` | `parsed` (returns reasoning) or `hidden` (suppresses it) |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('groq', 'default')` |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | e.g. `llama-3.1-70b-versatile`, `qwen/qwen3-32b`, `groq/compound-beta` |
+| `temperature` | number\|null | `null` | no | - | 0-2 |
+| `max_tokens` | number\|null | `null` (8-131K per model) | no | - | 1-200000 |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `thinking_enabled` | boolean | `false` | no | - | Only Qwen3-32b supports reasoning |
+| `reasoning_format` | enum | `parsed` | no | `thinking_enabled=[true]` | `parsed` (returns reasoning) or `hidden` (suppresses it) |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('groq', 'default')` |
+
+(Field names are snake_case on `GroqChatModelParams`; unknown keys ignored.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output; standard envelope payload |
### Output payload
@@ -58,7 +60,7 @@ Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
diff --git a/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.md
index 7c6038dd..2e62dea1 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/kimi_chat_model/__init__.py`](../../../server/nodes/model/kimi_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | no (group `('model',)`) |
## Purpose
-Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K context, 96K output. Uses OpenAI-compatible Moonshot endpoint via the native path. Shares `handle_ai_chat_model`.
+Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K context, 96K output. Uses OpenAI-compatible Moonshot endpoint via the native path. `KimiChatModelNode` uses the shared `ChatModelParams` unchanged (no provider override). The `ChatModelBase.chat` operation calls `AIService.execute_chat`.
## Inputs (handles)
@@ -24,18 +24,21 @@ Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K co
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes | - | User message |
-| `systemMessage` | string | `""` | no | - | System prompt |
-| `model` | string | injected | no | - | `kimi-k2.6`, `kimi-k2.5`, or `kimi-k2.7-code` |
-| `temperature` | number | **fixed** | no | - | Fixed at 0.6 (instant) or 1.0 (thinking). Input is ignored. |
-| `maxTokens` | number | up to 96K | no | - | |
-| `thinkingEnabled` | boolean | true for k2.5 | no | - | ON by default for k2.5; explicitly disabled for tool-calling agents |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('kimi', 'default')` |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | `kimi-k2.6`, `kimi-k2.5`, or `kimi-k2.7-code` |
+| `temperature` | number\|null | `null` (**ignored**) | no | - | Provider forces 0.6 (instant) or 1.0 (thinking); user input ignored |
+| `max_tokens` | number\|null | `null` (up to 96K) | no | - | 1-200000 |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `thinking_enabled` | boolean | `false` (Params default) | no | - | Base default is `false`; the provider treats k2.5 as thinking-on unless explicitly disabled |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('kimi', 'default')` |
+
+(Kimi uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output; standard envelope payload |
### Output payload
@@ -56,7 +59,7 @@ Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K co
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
diff --git a/docs-internal/node-logic-flows/ai_chat_models/lmstudioChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/lmstudioChatModel.md
new file mode 100644
index 00000000..438a089f
--- /dev/null
+++ b/docs-internal/node-logic-flows/ai_chat_models/lmstudioChatModel.md
@@ -0,0 +1,107 @@
+# LM Studio Chat Model (`lmstudioChatModel`)
+
+| Field | Value |
+|------|-------|
+| **Category** | ai_chat_models |
+| **Backend handler** | [`server/nodes/model/lmstudio_chat_model/__init__.py`](../../../server/nodes/model/lmstudio_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
+| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
+| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
+| **Skill (if any)** | n/a |
+| **Dual-purpose tool** | no (group `('model',)`) |
+
+## Purpose
+
+Run local LLMs through LM Studio's OpenAI-compatible server (default `http://localhost:1234/v1`). The existing OpenAI-compatible fallback in `services/llm/factory.py` routes it through `OpenAIProvider` with `base_url` from `llm_defaults.json` — same path as deepseek/kimi/mistral/ollama. The user's custom server URL is stored as the `lmstudio_proxy` credential. `LMStudioChatModelNode` uses the shared `ChatModelParams` unchanged. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.
+
+## Inputs (handles)
+
+| Handle | Connection type | Required | Purpose |
+|--------|-----------------|----------|---------|
+| `input-main` | main | no | Upstream data; not consumed directly |
+
+## Parameters
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `prompt` | string | `""` | yes | - | User message |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | Whatever the user has loaded in the LM Studio UI. Open-world: name not pattern-checked by `is_model_valid_for_provider` |
+| `temperature` | number\|null | `null` | no | - | 0-2 |
+| `max_tokens` | number\|null | `null` | no | - | 1-200000; default per-loaded-model ctx ÷ 4 (capped 4096) |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `api_key` | string\|null | `null` -> placeholder `"ollama"` | no | - | Optional; local servers usually run with no auth. `LMStudioCredential.resolve()` returns `"ollama"` when none stored |
+
+(LM Studio uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)
+
+## Outputs (handles)
+
+| Handle | Shape | Description |
+|--------|-------|-------------|
+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload |
+
+### Output payload
+
+```ts
+{
+ response: string;
+ thinking: string | null; // per-model
+ thinking_enabled: boolean;
+ model: string;
+ provider: 'lmstudio';
+ finish_reason: string;
+ timestamp: string;
+ input: { prompt: string; system_prompt: string };
+}
+```
+
+Wrapped in `{ success, node_id, node_type, result, execution_time }`.
+
+## Logic Flow
+
+```mermaid
+flowchart TD
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
+ B --> C[AIService.execute_chat]
+ C --> D{valid key + prompt?}
+ D -- no --> X[error envelope]
+ D -- yes --> E[detect_ai_provider -> 'lmstudio']
+ E --> F[Lookup lmstudio_proxy credential -> base_url override]
+ F --> G[create_provider lmstudio -> OpenAIProvider<br/>base_url=user server, api_key='ollama']
+ G --> H[provider.chat]
+ H --> I[success envelope]
+ G -- Exception --> X
+```
+
+## Decision Logic
+
+- **Validation**: empty prompt -> error envelope. `api_key` is never the blocker — `LMStudioCredential.resolve()` returns the placeholder `"ollama"` when no key is stored, so the central "API key required" check in `execute_chat` passes.
+- **Provider routing**: `detect_ai_provider` MUST list `lmstudio` (in `server/constants.py`) or the node falls through to `'openai'` and `execute_chat` hits api.openai.com with the placeholder key.
+- **Open-world model name**: `is_model_valid_for_provider` returns `True` for `lmstudio` so local model names are not rejected by the cloud-style pattern check.
+- **Base URL routing**: `lmstudio_proxy` credential carries the user's server URL into `OpenAIProvider.base_url`; traffic stays on `localhost`.
+
+## Side Effects
+
+- **Database writes**: per-model context params persist in `EncryptedAPIKey.models["model_params"]` and `model_registry.json` at credential-validation time (via `_local_validator.py` + `model_registry.register_local_model()`), not on the bare chat path.
+- **Broadcasts**: none on the bare chat path.
+- **External API calls**: `POST {user_server}/v1/chat/completions` via the `openai` SDK with overridden `base_url` (default `http://localhost:1234/v1`).
+- **File I/O**: none.
+- **Subprocess**: none.
+
+## External Dependencies
+
+- **Credentials**: optional `auth_service.get_api_key('lmstudio')`; user server URL stored as `lmstudio_proxy`.
+- **Services**: `services/llm/providers/openai.py` (reused with LM Studio base_url); `nodes/model/_local_validator.py` (SDK probe via `lmstudio.AsyncClient.llm.list_loaded()`).
+- **Python packages**: `openai`, `lmstudio>=1.5.0` (validation only).
+- **Environment variables**: none.
+
+## Edge cases & known limits
+
+- **Server must be running with a model loaded**: LM Studio reports loaded models via `list_loaded()`; if nothing is loaded the probe returns no models.
+- **Provider routing dependency**: `lmstudio` must be present in `detect_ai_provider` AND in each agent's `provider` Literal, or the node silently falls back to OpenAI cloud.
+- **Max output default**: ctx ÷ 4, capped at 4096, unless the user overrides `max_tokens`. Typed `LlmInstanceInfo.context_length` drives this.
+- **Errors swallowed into envelope** on the chat path.
+
+## Related
+
+- **Peer nodes**: [`ollamaChatModel`](./ollamaChatModel.md) (other local-server provider), and the cloud chat-model docs in this folder.
+- **Architecture docs**: [Native LLM SDK](../../native_llm_sdk.md).
diff --git a/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.md
index 7db34193..ab423a24 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/mistral_chat_model/__init__.py`](../../../server/nodes/model/mistral_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | no (group `('model',)`) |
## Purpose
-Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest`). Up to 256K context, 131K output. No thinking/reasoning support. Uses OpenAI-compatible Mistral endpoint via native path. Shares `handle_ai_chat_model`.
+Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest`). Up to 256K context, 131K output. No thinking/reasoning support. Uses OpenAI-compatible Mistral endpoint via native path. `MistralChatModelNode` uses the shared `ChatModelParams` unchanged. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.
## Inputs (handles)
@@ -24,18 +24,20 @@ Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-sma
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes | - | User message |
-| `systemMessage` | string | `""` | no | - | System prompt |
-| `model` | string | injected | no | - | `mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest` |
-| `temperature` | number | 0-1.5 | no | - | Narrower range than OpenAI (0-1.5) |
-| `maxTokens` | number | up to 131K | no | - | |
-| `topP` | number | - | no | - | |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('mistral', 'default')` |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | `mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest` |
+| `temperature` | number\|null | `null` | no | - | Narrower range than OpenAI (0-1.5) |
+| `max_tokens` | number\|null | `null` (up to 131K) | no | - | 1-200000 |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('mistral', 'default')` |
+
+(Mistral uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output; standard envelope payload |
### Output payload
@@ -56,7 +58,7 @@ Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-sma
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D{valid key + prompt?}
D -- no --> X[error envelope]
diff --git a/docs-internal/node-logic-flows/ai_chat_models/ollamaChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/ollamaChatModel.md
new file mode 100644
index 00000000..54c22253
--- /dev/null
+++ b/docs-internal/node-logic-flows/ai_chat_models/ollamaChatModel.md
@@ -0,0 +1,109 @@
+# Ollama Chat Model (`ollamaChatModel`)
+
+| Field | Value |
+|------|-------|
+| **Category** | ai_chat_models |
+| **Backend handler** | [`server/nodes/model/ollama_chat_model/__init__.py`](../../../server/nodes/model/ollama_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
+| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
+| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
+| **Skill (if any)** | n/a |
+| **Dual-purpose tool** | no (group `('model',)`) |
+
+## Purpose
+
+Run local LLMs (llama, mistral, qwen, deepseek-r1, ...) through a locally-running Ollama server. Ollama exposes an OpenAI-shaped `/v1` endpoint, so the existing OpenAI-compatible fallback in `services/llm/factory.py` hands it to `OpenAIProvider` with `base_url` from `llm_defaults.json` — same path as deepseek/kimi/mistral. The user's custom server URL (if not localhost) is stored as the `ollama_proxy` credential and flows through the same `proxy_url` parameter cloud providers already use for Ollama-style auth delegation. `OllamaChatModelNode` uses the shared `ChatModelParams` unchanged. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.
+
+## Inputs (handles)
+
+| Handle | Connection type | Required | Purpose |
+|--------|-----------------|----------|---------|
+| `input-main` | main | no | Upstream data; not consumed directly |
+
+## Parameters
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `prompt` | string | `""` | yes | - | User message |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | Whatever the user has pulled, e.g. `qwen2.5`, `llama3.x`, `deepseek-r1`. Open-world: name not pattern-checked by `is_model_valid_for_provider` |
+| `temperature` | number\|null | `null` | no | - | 0-2 |
+| `max_tokens` | number\|null | `null` | no | - | 1-200000; default per-loaded-model ctx ÷ 4 (capped 4096) |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `api_key` | string\|null | `null` -> placeholder `"ollama"` | no | - | Optional; local servers usually run with no auth. `OllamaCredential.resolve()` returns `"ollama"` when none stored |
+
+(Ollama uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)
+
+## Outputs (handles)
+
+| Handle | Shape | Description |
+|--------|-------|-------------|
+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload |
+
+### Output payload
+
+```ts
+{
+ response: string;
+ thinking: string | null; // per-model; Ollama has no generic thinking knob
+ thinking_enabled: boolean;
+ model: string;
+ provider: 'ollama';
+ finish_reason: string;
+ timestamp: string;
+ input: { prompt: string; system_prompt: string };
+}
+```
+
+Wrapped in `{ success, node_id, node_type, result, execution_time }`.
+
+## Logic Flow
+
+```mermaid
+flowchart TD
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
+ B --> C[AIService.execute_chat]
+ C --> D{valid key + prompt?}
+ D -- no --> X[error envelope]
+ D -- yes --> E[detect_ai_provider -> 'ollama']
+ E --> F[Lookup ollama_proxy credential -> base_url override]
+ F --> G[create_provider ollama -> OpenAIProvider<br/>base_url=user server, api_key='ollama']
+ G --> H[provider.chat]
+ H --> I[success envelope]
+ G -- Exception --> X
+```
+
+## Decision Logic
+
+- **Validation**: empty prompt -> error envelope. `api_key` is never the blocker — `OllamaCredential.resolve()` returns the placeholder `"ollama"` when no key is stored, so the central "API key required" check in `execute_chat` passes.
+- **Provider routing**: `detect_ai_provider` MUST list `ollama` (in `server/constants.py`) or the node falls through to `'openai'` and `execute_chat` hits api.openai.com with the placeholder key.
+- **Open-world model name**: `is_model_valid_for_provider` returns `True` for `ollama` so local model names like `qwen/qwen3.6-27b` are not rejected by the cloud-style pattern check.
+- **Base URL routing**: `ollama_proxy` credential carries the user's server URL into `OpenAIProvider.base_url`; traffic stays on `localhost`.
+- **No thinking knob**: the shared `thinking_enabled` field is present but generic Ollama has no per-call thinking parameter; reasoning is per-model.
+
+## Side Effects
+
+- **Database writes**: per-model context params persist in `EncryptedAPIKey.models["model_params"]` and `model_registry.json` at credential-validation time (via `_local_validator.py` + `model_registry.register_local_model()`), not on the bare chat path.
+- **Broadcasts**: none on the bare chat path.
+- **External API calls**: `POST {user_server}/v1/chat/completions` via the `openai` SDK with overridden `base_url` (default `http://localhost:11434/v1`).
+- **File I/O**: none.
+- **Subprocess**: none.
+
+## External Dependencies
+
+- **Credentials**: optional `auth_service.get_api_key('ollama')`; user server URL stored as `ollama_proxy`.
+- **Services**: `services/llm/providers/openai.py` (reused with Ollama base_url); `nodes/model/_local_validator.py` (SDK probe via `ollama.AsyncClient.ps()`).
+- **Python packages**: `openai`, `ollama>=0.6.0` (validation only).
+- **Environment variables**: none.
+
+## Edge cases & known limits
+
+- **Server must be running**: requests go to the user's local Ollama server; if it is down the request surfaces as a connection error in the envelope.
+- **Provider routing dependency**: `ollama` must be present in `detect_ai_provider` AND in each agent's `provider` Literal, or the node silently falls back to OpenAI cloud.
+- **Max output default**: ctx ÷ 4, capped at 4096, unless the user overrides `max_tokens`.
+- **Per-model params survive restart**: written through to `model_registry.json` so context length is known without re-clicking Fetch.
+- **Errors swallowed into envelope** on the chat path.
+
+## Related
+
+- **Peer nodes**: [`lmstudioChatModel`](./lmstudioChatModel.md) (other local-server provider), and the cloud chat-model docs in this folder.
+- **Architecture docs**: [Native LLM SDK](../../native_llm_sdk.md).
diff --git a/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.md
index fe2cc4fd..5248b23d 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/openai_chat_model/__init__.py`](../../../server/nodes/model/openai_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
-| **Skill (if any)** | n/a - chat models are model config nodes, not tools |
-| **Dual-purpose tool** | no - pure chat completion node |
+| **Skill (if any)** | n/a |
+| **Dual-purpose tool** | yes - tool name `openai_chat_model` (advisor; `usable_as_tool = True`, group `('model', 'tool')`) |
## Purpose
-Single-turn chat completion against the OpenAI REST API. Used either as a standalone workflow node (user enters a prompt, the node produces a response) or as a model config node wired into an AI Agent via the `input-model` handle. All 9 chat-model node types share the same handler (`handle_ai_chat_model`) and the same dispatch path; per-provider differences live inside `AIService.execute_chat` (native SDK for OpenAI) and in the clamp / temperature helpers.
+Single-turn chat completion against the OpenAI REST API. Used either as a standalone workflow node (user enters a prompt, the node produces a response), wired into an AI Agent via the `input-model` handle, or connected as an advisor tool (`usable_as_tool = True`). Every chat-model node runs through the shared `ChatModelBase.chat` operation (in `_base.py`), which calls `AIService.execute_chat`; per-provider differences live inside `execute_chat` (native SDK for OpenAI) and in the clamp / temperature helpers.
## Inputs (handles)
@@ -24,24 +24,25 @@ Single-turn chat completion against the OpenAI REST API. Used either as a standa
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes (non-empty) | - | User message content sent to the model |
-| `systemMessage` / `systemPrompt` / `system_prompt` | string | `""` | no | - | System prompt; any of the three naming conventions accepted |
-| `model` | string | auto-populated from stored models, fallback `gpt-3.5-turbo` | no | - | Model ID. `[FREE] ` prefix is stripped before the API call |
-| `temperature` | number | provider default | no | - | 0-2 for most OpenAI models; o-series (o1/o3/o4) force temperature=1 |
-| `maxTokens` / `max_tokens` | number | provider default | no | - | Clamped via `_resolve_max_tokens` to the model's actual context limit |
-| `topP` | number | 1.0 | no | - | Nucleus sampling |
-| `frequencyPenalty` | number | 0 | no | - | OpenAI penalty |
-| `presencePenalty` | number | 0 | no | - | OpenAI penalty |
-| `responseFormat` | options | `text` | no | - | `text` or `json_object` |
-| `thinkingEnabled` | boolean | false | no | - | Turn on extended reasoning (GPT-5 hybrid / o-series) |
-| `reasoningEffort` | options | `medium` | no | `thinkingEnabled=[true]` | `minimal` / `low` / `medium` / `high` |
-| `apiKey` / `api_key` | string | injected by executor | no | - | Auto-injected by `NodeExecutor._inject_api_keys` via `auth_service.get_api_key('openai', 'default')` |
-| `options` | object | `{}` | no | - | Collection bag; flattened into the top-level params dict before use |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected from stored models) | no | - | Model ID. `[FREE] ` prefix is stripped before the API call |
+| `temperature` | number\|null | `null` -> `agent.default_temperature` | no | - | 0-2 for most OpenAI models; o-series (o1/o3/o4) force temperature=1 |
+| `max_tokens` | number\|null | `null` -> per-model default | no | - | 1-200000; clamped via `_resolve_max_tokens` to the model's actual ceiling |
+| `top_p` | number\|null | `1.0` | no | - | Nucleus sampling |
+| `frequency_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 |
+| `presence_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 |
+| `response_format` | enum | `text` | no | - | `text` or `json_object` |
+| `thinking_enabled` | boolean | `false` | no | - | Turn on extended reasoning (GPT-5 hybrid / o-series) |
+| `reasoning_effort` | enum | `medium` | no | `thinking_enabled=[true]` | `minimal` / `low` / `medium` / `high` (OpenAI override adds `minimal`) |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('openai', 'default')` |
+
+(Field names are snake_case on `OpenAIChatModelParams`; `model_config = ConfigDict(extra="ignore")` drops unknown keys.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload (see below) |
+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload (see below) |
### Output payload
@@ -64,8 +65,8 @@ Wrapped in the standard envelope: `{ success: true, node_id, node_type, result:
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[_prepare_parameters:<br/>merge DB params, validate, inject api_key + model]
- B --> C[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation<br/>params validated by Pydantic]
+ B --> C[get_ai_service]
C --> D[AIService.execute_chat]
D --> E[Flatten options into flat params dict]
E --> F{api_key present?}
diff --git a/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.md
index e9c9bf8c..6eb02bef 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | ai_chat_models |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |
+| **Backend handler** | [`server/nodes/model/openrouter_chat_model/__init__.py`](../../../server/nodes/model/openrouter_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |
| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |
| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |
| **Skill (if any)** | n/a |
-| **Dual-purpose tool** | no |
+| **Dual-purpose tool** | no (group `('model',)`) |
## Purpose
-Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google, Meta, Mistral, etc.) through a single OpenAI-compatible API. Shares the common `handle_ai_chat_model` handler.
+Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google, Meta, Mistral, etc.) through a single OpenAI-compatible API. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.
## Inputs (handles)
@@ -24,23 +24,25 @@ Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | yes (non-empty) | - | User message |
-| `systemMessage` | string | `""` | no | - | System prompt |
-| `model` | string | injected | no | - | `provider/model-id` format, e.g. `anthropic/claude-opus-4.6`, `openai/gpt-4o`. `[FREE] ` prefix stripped before the API call but the prefix IS preserved for the dropdown grouping |
-| `temperature` | number | 0-2 | no | - | |
-| `maxTokens` | number | varies per model | no | - | |
-| `topP` | number | - | no | - | |
-| `frequencyPenalty` | number | 0 | no | - | Forwarded to downstream provider |
-| `presencePenalty` | number | 0 | no | - | Forwarded |
-| `thinkingEnabled` | boolean | false | no | - | Only honored if the routed model supports it |
-| `thinkingBudget` | number | 2048 | no | `thinkingEnabled=[true]` | |
-| `reasoningEffort` | options | `medium` | no | `thinkingEnabled=[true]` | |
-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('openrouter', 'default')` |
+| `system_prompt` | string | `""` | no | - | System prompt |
+| `model` | string | `""` (injected) | no | - | `provider/model-id` format, e.g. `anthropic/claude-opus-4.6`, `openai/gpt-4o`. `[FREE] ` prefix stripped before the API call but preserved for dropdown grouping |
+| `temperature` | number\|null | `null` | no | - | 0-2 |
+| `max_tokens` | number\|null | `null` (varies per model) | no | - | 1-200000 |
+| `top_p` | number\|null | `1.0` | no | - | |
+| `frequency_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0; forwarded to downstream provider |
+| `presence_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0; forwarded |
+| `thinking_enabled` | boolean | `false` | no | - | Only honored if the routed model supports it |
+| `thinking_budget` | number\|null | `2048` | no | - | Inherited base field (no displayOptions); 1024-16000 |
+| `reasoning_effort` | enum\|null | `null` | no | - | Inherited base field (no displayOptions); `low`/`medium`/`high` |
+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('openrouter', 'default')` |
+
+(Only `frequency_penalty` / `presence_penalty` are OpenRouter overrides; the thinking/reasoning fields come from the shared base `ChatModelParams` with no displayOptions. Field names are snake_case, unknown keys ignored.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
+| `output-model` | object | Model output; standard envelope payload |
### Output payload
@@ -61,7 +63,7 @@ Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google
```mermaid
flowchart TD
- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]
+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]
B --> C[AIService.execute_chat]
C --> D[Strip '[FREE] ' prefix from model]
D --> E{valid key + prompt?}
diff --git a/docs-internal/node-logic-flows/ai_tools/agentBuilder.md b/docs-internal/node-logic-flows/ai_tools/agentBuilder.md
new file mode 100644
index 00000000..65b15e2a
--- /dev/null
+++ b/docs-internal/node-logic-flows/ai_tools/agentBuilder.md
@@ -0,0 +1,132 @@
+# Agent Builder (`agentBuilder`)
+
+| Field | Value |
+|------|-------|
+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |
+| **Backend handler** | [`server/nodes/tool/agent_builder/__init__.py`](../../../server/nodes/tool/agent_builder/__init__.py) — `AgentBuilderNode`, dispatched via `BaseNode.execute()` + one `@Operation` per canvas mutation (`inspect_canvas` / `add_tool` / `add_skill` / `add_subagent` / `create_workflow`) |
+| **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) |
+| **Skill (if any)** | [`server/skills/assistant/agent-builder-skill/SKILL.md`](../../../server/skills/assistant/agent-builder-skill/SKILL.md) |
+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `agent_builder` (`tool_name` class attr) |
+
+## Purpose
+
+Runtime canvas-mutation tool. Wired to an AI agent's `input-tools` handle, it
+lets the LLM inspect the live workflow canvas and grow its own toolset mid-run:
+spawn tool nodes, enable skills on a Master Skill, add delegate agents (team
+leads only), or create a fresh workflow. One node, one `Params` model with an
+`operation: Literal[...]` discriminator, five `@Operation` methods — the LLM
+sees ONE tool with a select-style `operation` field. Each mutation persists to
+the DB and pushes a `workflow_ops_apply` event so the React Flow canvas updates
+live.
+
+## Inputs (handles)
+
+| Handle | Connection type | Required | Purpose |
+|--------|-----------------|----------|---------|
+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools`. Operations walk `ctx.edges` (via `needs_canvas = True`) to resolve the calling agent and mutate the canvas. |
+
+## Parameters
+
+The `AgentBuilderParams` model fields ARE the LLM-provided tool args (no
+separate `toolName` / `toolDescription` node params — those live on the class
+as `tool_name` / `tool_description`). `model_config = ConfigDict(extra="ignore")`.
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `operation` | enum | `inspect_canvas` | no | - | One of `inspect_canvas`, `add_tool`, `add_skill`, `add_subagent`, `create_workflow` |
+| `node_type` | string | `""` | no | `operation == add_tool` | Tool node type to spawn (must be a registered `component_kind="tool"` or `usable_as_tool=True` node, minus the allowlist blocklist) |
+| `skill_folder` | string | `""` | no | `operation == add_skill` | Skill folder name under `server/skills/**` |
+| `agent_type` | string | `""` | no | `operation == add_subagent` | Agent node type to spawn as a teammate (caller must be a team lead) |
+| `workflow_name` | string | `""` | no | `operation == create_workflow` | Display name (create_workflow is currently disabled) |
+| `workflow_description` | string | `""` | no | `operation == create_workflow` | Optional one-line description |
+
+## Outputs (handles)
+
+| Handle | Shape | Description |
+|--------|-------|-------------|
+| `output-tool` | object | `AgentBuilderOutput` model (`extra="allow"`), serialized per `BaseNode._serialize_result` |
+
+### Output payload (TypeScript shape)
+
+```ts
+{
+ operation?: string;
+ summary?: string;
+ operations?: Array<Record<string, unknown>>; // workflow-ops batch applied
+ // inspect_canvas extras:
+ nodes?: Array<{ id; type; label; key_params }>;
+ edges?: Array<{ source; target; source_handle; target_handle }>;
+ you?: { node_id: string; incoming: unknown[]; outgoing: unknown[] };
+ available_tools?: Array<{ type; display_name; description }>;
+ available_agents?: Array<{ type; display_name; description }>;
+ available_skills?: Array<{ folder; name; description }>;
+ // create_workflow extras:
+ workflow_id?: string;
+}
+```
+
+## Logic Flow
+
+```mermaid
+flowchart TD
+ A[BaseNode.execute -> operation method] --> R{operation}
+ R -- inspect_canvas --> I[reload canvas from DB<br/>resolve caller via input-tools edge<br/>build node/edge summaries + tool/agent/skill catalogues]
+ R -- add_tool --> T{node_type allowed + not already wired?}
+ T -- no --> Tn[Return summary, operations=empty]
+ T -- yes --> Top[mint node id + add_node + add_edge ops]
+ R -- add_skill --> S{masterSkill wired?}
+ S -- yes --> Sy[toggle skill in skills_config via set_node_parameters]
+ S -- no --> Sn[spawn masterSkill node + edge with skill enabled]
+ R -- add_subagent --> G{caller is team lead + agent_type allowed + not dup?}
+ G -- no --> Gn[Return summary, operations=empty]
+ G -- yes --> Gop[mint node id + add_node + add_edge to input-teammates]
+ R -- create_workflow --> C[disabled: return summary, no-op]
+ Top --> B[_broadcast: persist to DB + broadcast_workflow_ops]
+ Sy --> B
+ Sn --> B
+ Gop --> B
+ B --> Z[Return AgentBuilderOutput summary + operations]
+ I --> Z
+ Tn --> Z
+ Gn --> Z
+ C --> Z
+```
+
+## Decision Logic
+
+- **Caller resolution**: walks edges for `source == self_id && targetHandle == input-tools`; falls back to self as caller (standalone Run).
+- **Live canvas reload**: `_load_live_canvas` re-reads `workflow.data` from the DB so successive ops in the same run see each other's mutations (the per-tool `ctx` snapshot is frozen at MachinaWorkflow start).
+- **Allowlist gating**: `_allowed_tool_types` / `_allowed_subagent_types` honor `node_allowlist.json` (`disabled_nodes` / `disabled_groups`); `_DENIED_TOOL_TYPES` (`agentBuilder`, `masterSkill`) are never spawnable (recursion guard).
+- **Idempotency**: `add_tool` / `add_subagent` / `add_skill` are no-ops (empty `operations`) when the target is already wired/enabled, surfacing the existing node id in the summary.
+- **add_subagent gate**: only team leads (`orchestrator_agent`, `ai_employee`) may spawn delegates; cannot spawn another team lead.
+- **create_workflow disabled**: `_CREATE_WORKFLOW_ENABLED = False` short-circuits with a "temporarily disabled" summary; the implementation body is intact for one-line re-enable.
+- **Summary suffix**: depends on `ctx.raw["auto_rebind_tools"]` (UserSettings) — "Available immediately" when auto-rebind is on, else "Available on your next turn" (the agent loop binds tools at turn start, so a mid-run mutation is callable next turn unless auto-rebind rebinds it in place).
+
+## Side Effects
+
+- **Database writes**: `_persist_canvas_mutation` applies `add_node` / `add_edge` / `set_node_parameters` ops to `workflow.data` via `database.get_workflow` + `database.save_workflow` (adopts the BE-minted `minted_id` so FE/BE node ids align). `create_workflow` (when enabled) persists a fresh workflow row.
+- **Broadcasts**: `broadcast_workflow_ops` (in `_events.py`) emits a flat `workflow_ops_apply` WS frame consumed by the frontend `useWorkflowOpsListener` hook. NOT a `dispatch.emit` event (no Temporal/event_waiter consumer for canvas mutations). A `WorkflowEvent` envelope (`com.machinaos.workflow.ops.applied`) is constructed for parity but not put on the wire today.
+- **External API calls**: none.
+- **File I/O**: reads `server/skills/**` to validate skill folders (`_skill_folder_exists`) and catalogue skills.
+- **Subprocess**: none.
+
+## External Dependencies
+
+- **Credentials**: none.
+- **Services**: `database` (via `services.plugin.deps.get_database`), `NodeAllowlistService`, `SkillLoader`, `services.node_registry.registered_node_classes`, `services.workflow_ops`, `services.workflow_naming`.
+- **Python packages**: stdlib only.
+- **Environment variables**: none.
+
+## Edge cases & known limits
+
+- `create_workflow` is disabled behind `_CREATE_WORKFLOW_ENABLED`; the operation returns a guidance summary and performs no mutation.
+- Mid-run mutations are not callable in the SAME turn unless the user's "Auto-Rebind Tools After Canvas Changes" toggle is on (default) — otherwise the LLM must wait for its next turn.
+- `inspect_canvas` is read-only and emits no `operations`.
+- Persist failures are logged at WARN (`exc_info=True`) and do not abort the broadcast — the canvas may diverge from the DB if persistence fails mid-mutation.
+- The `workflow_ops_apply` wire format is flat (`{workflow_id, caller_node_id, operations}`) for FE back-compat, not the CloudEvents envelope.
+
+## Related
+
+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)
+- **Skill using this tool**: [`agent-builder-skill/SKILL.md`](../../../server/skills/assistant/agent-builder-skill/SKILL.md)
+- **Architecture docs**: [Workflow Ops Protocol](../../workflow_ops_protocol.md), [Agent Architecture](../../agent_architecture.md), [Node Creation Guide](../../node_creation.md)
diff --git a/docs-internal/node-logic-flows/ai_tools/calculatorTool.md b/docs-internal/node-logic-flows/ai_tools/calculatorTool.md
index e53ccdb1..1d66c473 100644
--- a/docs-internal/node-logic-flows/ai_tools/calculatorTool.md
+++ b/docs-internal/node-logic-flows/ai_tools/calculatorTool.md
@@ -2,85 +2,71 @@
| Field | Value |
|------|-------|
-| **Category** | ai_tools (dedicated AI tool) |
-| **Backend handler** | [`server/services/handlers/tools.py::_execute_calculator`](../../../server/services/handlers/tools.py) |
+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |
+| **Backend handler** | [`server/nodes/tool/calculator_tool/__init__.py`](../../../server/nodes/tool/calculator_tool/__init__.py) — `CalculatorToolNode`, dispatched via `BaseNode.execute()` + the `@Operation("calculate")` method |
| **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) |
| **Skill (if any)** | None (referenced directly by AI agents via tool-calling) |
-| **Dual-purpose tool** | tool-only - LLM invokes as `calculator` (configurable via `toolName` param) |
+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `calculator` (`tool_name` class attr) |
## Purpose
Exposes basic arithmetic as a structured tool that an AI Agent can call during
-reasoning. The LLM fills the `operation` + operand arguments and the handler
+reasoning. The LLM fills the `operation` + operand arguments and the operation
returns the numeric result. Supports the eight operations listed below; no
-external calls, no state.
+external calls, no state. The `CalculatorParams` Pydantic model is the
+LLM-visible tool schema (via `ToolNode.as_tool_schema()`).
## Inputs (handles)
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| (none) | - | - | Passive node - connect `output-tool` to an AI Agent's `input-tools` |
+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |
## Parameters
+The `CalculatorParams` model fields ARE the LLM-provided tool args (no separate
+`toolName` / `toolDescription` node params — those live on the class as
+`tool_name` / `tool_description`).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `calculator` | yes | - | Name the LLM uses to invoke the tool |
-| `toolDescription` | string | (see frontend) | no | - | LLM-visible tool description |
-
-### LLM-provided tool args (at invocation time)
-
-| Arg | Type | Description |
-|-----|------|-------------|
-| `operation` | string | One of `add`, `subtract`, `multiply`, `divide`, `power`, `sqrt`, `mod`, `abs` |
-| `a` | number | First operand (defaults to `0` if missing) |
-| `b` | number | Second operand (defaults to `0`; unused for `sqrt` / `abs`) |
+| `operation` | enum | (required) | yes | - | One of `add`, `subtract`, `multiply`, `divide`, `power`, `sqrt`, `mod`, `abs` |
+| `a` | float | (required) | yes | - | First operand (or the sole input for `sqrt` / `abs`) |
+| `b` | float? | `None` | no | - | Second operand; required for add/subtract/multiply/divide/power/mod, unused for `sqrt` / `abs` |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-tool` | object | Raw dict returned to the LLM (no envelope wrapper at the tool level) |
+| `output-tool` | object | `CalculatorOutput` model, serialized per `BaseNode._serialize_result` |
### Output payload (TypeScript shape)
-On success:
+On success (matches the `CalculatorOutput` model):
```ts
{
operation: string;
a: number;
- b: number;
+ b: number | null;
result: number; // float('inf') for x/0, 0 for x mod 0
}
```
-On unsupported operation:
-```ts
-{
- error: string; // "Unknown operation: <op>"
- supported_operations: string[];
-}
-```
-
-On any raised exception inside the op lambda:
-```ts
-{ error: string } // str(exception)
-```
+Errors are not produced inside the op (the `Literal` schema rejects unknown
+operations at Pydantic validation before the op runs; a validation failure
+surfaces as the framework's standard error envelope from `BaseNode.execute()`).
## Logic Flow
```mermaid
flowchart TD
- A[_execute_calculator args] --> B[operation = args.operation.lower<br/>a = float args.a default 0<br/>b = float args.b default 0]
- B --> C{operation in<br/>supported map?}
- C -- no --> Eunknown[Return error + supported_operations list]
- C -- yes --> D{Invoke op lambda}
+ A[BaseNode.execute -> calculate ctx params] --> B[fn = _OPERATIONS params.operation<br/>b_value = params.b if not None else 0.0]
+ B --> D{Invoke op lambda}
D -- divide with b==0 --> Finf[result = float inf]
D -- mod with b==0 --> Fmod[result = 0]
D -- sqrt --> Fsqrt[result = sqrt of abs a]
- D -- other --> Fok[result = a op b]
- D -- exception --> Eexc[Return error: str exception]
- Finf --> R[Return operation / a / b / result]
+ D -- other --> Fok[result = a op b_value]
+ Finf --> R[Return CalculatorOutput operation/a/b/result]
Fmod --> R
Fsqrt --> R
Fok --> R
@@ -88,18 +74,16 @@ flowchart TD
## Decision Logic
-- **Unknown operation**: returns `{error, supported_operations}` and does not
- raise. `operations.keys()` is the authoritative supported list.
+- **Unknown operation**: cannot occur — `operation` is a `Literal[...]` so an
+ out-of-set value fails Pydantic validation before the op runs.
- **Division by zero** (`divide`): returns `float('inf')` instead of erroring.
- **Mod by zero** (`mod`): returns `0` instead of erroring.
-- **Negative sqrt**: handler silently calls `math.sqrt(abs(a))`, so `sqrt(-9)`
- returns `3.0` (not an error).
-- **`power` overflow**: raised as `OverflowError` by `math.pow`, caught by the
- outer `except Exception` and returned as `{error: str}`.
-- **Tool mode only**: when dropped on the canvas and run directly, the node
- type is not registered in `NodeExecutor._handlers`, so the executor falls
- through to the generic success fallback. The real contract is the tool
- invocation path via `execute_tool`.
+- **Negative sqrt**: silently calls `math.sqrt(abs(a))`, so `sqrt(-9)` returns
+ `3.0` (not an error).
+- **Missing `b`**: when `params.b is None`, `b_value` defaults to `0.0` before
+ the lambda runs (so `add`/`multiply`/etc. with no `b` operate against 0).
+- **`power` overflow**: raised as `OverflowError` by `math.pow`; bubbles up to
+ `BaseNode.execute()`'s generic exception path (full traceback envelope).
## Side Effects
@@ -121,14 +105,14 @@ flowchart TD
- `divide` returns `float('inf')` for division by zero - downstream JSON
serializers may choke on `Infinity` (not strict JSON).
- `mod` returns `0` for modulo-by-zero, which is mathematically wrong but
- matches the handler. Document this for agents that might reason about it.
+ matches the op lambda. Document this for agents that might reason about it.
- `sqrt` of a negative number silently returns the sqrt of the absolute value
rather than erroring or returning NaN.
-- `float(args.get('a', 0))` will raise `ValueError` if the LLM returns
- non-numeric strings; that error is caught and returned as `{error: ...}`.
-- Operation names are case-insensitive (`.lower()` applied).
+- `a` is typed `float` (required) — a non-numeric LLM argument fails Pydantic
+ coercion and produces a validation error the LLM can correct.
+- Operation names are case-sensitive (`Literal` exact match; no `.lower()`).
## Related
-- **Sibling tools**: [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)
+- **Sibling tools**: [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)
- **Architecture docs**: [Agent Architecture](../../agent_architecture.md), [Tool Building Pipeline](../../tool_building_pipeline.md), [Node Creation Guide](../../node_creation.md)
diff --git a/docs-internal/node-logic-flows/ai_tools/currentTimeTool.md b/docs-internal/node-logic-flows/ai_tools/currentTimeTool.md
index 797d5da1..bdc06cbf 100644
--- a/docs-internal/node-logic-flows/ai_tools/currentTimeTool.md
+++ b/docs-internal/node-logic-flows/ai_tools/currentTimeTool.md
@@ -2,84 +2,71 @@
| Field | Value |
|------|-------|
-| **Category** | ai_tools (dedicated AI tool) |
-| **Backend handler** | [`server/services/handlers/tools.py::_execute_current_time`](../../../server/services/handlers/tools.py) |
+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |
+| **Backend handler** | [`server/nodes/tool/current_time_tool/__init__.py`](../../../server/nodes/tool/current_time_tool/__init__.py) — `CurrentTimeToolNode`, dispatched via `BaseNode.execute()` + the `@Operation("now")` method |
| **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) |
| **Skill (if any)** | None |
-| **Dual-purpose tool** | tool-only - LLM invokes as `get_current_time` (configurable via `toolName` param) |
+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `get_current_time` (`tool_name` class attr) |
## Purpose
-Provides the current date, time, day-of-week, and unix timestamp in a
-caller-specified timezone. Intended for AI Agents that need temporal grounding
-(e.g., "what day is it today?" before scheduling something). Pure wrapper
-around `datetime.now(tz)` with `pytz` for tz resolution.
+Provides the current ISO timestamp and unix epoch in a caller-specified
+timezone. Intended for AI Agents that need temporal grounding (e.g., "what day
+is it today?" before scheduling something). Pure wrapper around
+`datetime.now(tz)` with `zoneinfo.ZoneInfo` for tz resolution.
## Inputs (handles)
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| (none) | - | - | Passive node |
+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |
## Parameters
+The `CurrentTimeParams` model field IS the LLM-provided tool arg (no separate
+`toolName` / `toolDescription` node params — those live on the class as
+`tool_name` / `tool_description`).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `get_current_time` | yes | - | LLM-visible tool name |
-| `toolDescription` | string | (see frontend) | no | - | LLM-visible description |
-| `timezone` | string | `UTC` | no | - | Default timezone used if the LLM does not pass one |
-
-### LLM-provided tool args (at invocation time)
-
-| Arg | Type | Description |
-|-----|------|-------------|
-| `timezone` | string? | IANA name (`UTC`, `America/New_York`, `Europe/London`, ...). When omitted or empty, handler falls back to the node's `timezone` parameter. |
+| `timezone` | string | `UTC` | no | - | IANA name (`UTC`, `America/New_York`, `Europe/London`, ...). Invalid values silently fall back to `UTC`. |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-tool` | object | Raw dict returned to the LLM |
+| `output-tool` | object | `CurrentTimeOutput` model, serialized per `BaseNode._serialize_result` |
### Output payload (TypeScript shape)
-On success:
+On success (matches the `CurrentTimeOutput` model):
```ts
{
- datetime: string; // ISO 8601 with offset, e.g. "2026-04-15T12:00:00+00:00"
- date: string; // "YYYY-MM-DD"
- time: string; // "HH:MM:SS"
- timezone: string; // echoed back (the resolved string, not the tz object)
- day_of_week: string; // "Monday" ... "Sunday"
- timestamp: number; // unix seconds, integer
+ iso: string; // ISO 8601 with offset, e.g. "2026-04-15T12:00:00+00:00"
+ timezone: string; // echoed back (the requested input string)
+ unix: number; // unix seconds, integer
}
```
-On invalid timezone:
-```ts
-{ error: string } // "Invalid timezone: <tz>. Error: <details>"
-```
-
## Logic Flow
```mermaid
flowchart TD
- A[_execute_current_time args node_params] --> B[tz_str = args.timezone OR node_params.timezone OR UTC]
- B --> C{pytz.timezone tz_str}
- C -- raises UnknownTimeZoneError --> E[Return error: Invalid timezone ...]
- C -- ok --> D[now = datetime.now tz]
- D --> F[Format ISO / date / time / day_of_week / timestamp]
- F --> R[Return dict]
+ A[BaseNode.execute -> now ctx params] --> B{ZoneInfo params.timezone}
+ B -- raises any exception --> C[tz = ZoneInfo UTC]
+ B -- ok --> D[tz resolved]
+ C --> E[now = datetime.now tz]
+ D --> E
+ E --> R[Return CurrentTimeOutput iso/timezone/unix]
```
## Decision Logic
-- **Timezone resolution**: `args.timezone` wins if truthy; otherwise
- `node_params.timezone`; otherwise the string literal `'UTC'`.
-- **Invalid timezone**: caught by the outer `except Exception` and returned
- as `{error: "Invalid timezone: <tz>. Error: <str>"}` (no raise).
-- **Empty string**: an empty `timezone` arg is falsy so falls back to
- `node_params.timezone`; an empty node param then falls back to `'UTC'`.
+- **Timezone resolution**: `ZoneInfo(params.timezone)`; `timezone` defaults to
+ `"UTC"` when the LLM omits it.
+- **Invalid timezone**: any exception constructing `ZoneInfo` is caught and
+ silently falls back to `ZoneInfo("UTC")` (the result still echoes the
+ requested `params.timezone` string in the `timezone` field).
## Side Effects
@@ -93,20 +80,20 @@ flowchart TD
- **Credentials**: none.
- **Services**: none.
-- **Python packages**: `pytz`, `datetime` (stdlib).
+- **Python packages**: `zoneinfo`, `datetime` (stdlib).
- **Environment variables**: none.
## Edge cases & known limits
-- `timestamp` is truncated to integer seconds via `int(now.timestamp())` -
+- `unix` is truncated to integer seconds via `int(now.timestamp())` -
millisecond precision is lost.
-- `timezone` field in the output is the input string, not the canonical
- zone name `pytz` may have normalised to (e.g. aliases are not rewritten).
+- `timezone` field in the output is the requested input string, not the
+ canonical zone name (and stays the input even when the lookup falls back to
+ UTC, so an invalid tz is silently masked).
- If the server system clock is wrong, output is wrong. No NTP sync.
-- `pytz.UnknownTimeZoneError` is the typical failure mode; any other
- unexpected exception (e.g. strftime locale issues) is also swallowed.
+- A bad/unknown IANA name does NOT error — it falls back to UTC silently.
## Related
-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)
+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)
- **Architecture docs**: [Agent Architecture](../../agent_architecture.md)
diff --git a/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.md b/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.md
index b6eca692..2a71cca5 100644
--- a/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.md
+++ b/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.md
@@ -2,11 +2,11 @@
| Field | Value |
|------|-------|
-| **Category** | ai_tools (dedicated AI tool) |
-| **Backend handler** | [`server/services/handlers/tools.py::_execute_duckduckgo_search`](../../../server/services/handlers/tools.py) |
+| **Category** | ai_tools (dedicated AI tool; plugin lives under `server/nodes/search/`, group `("tool", "ai", "search")`) |
+| **Backend handler** | [`server/nodes/search/duckduckgo_search/__init__.py`](../../../server/nodes/search/duckduckgo_search/__init__.py) — `DuckDuckGoSearchNode`, dispatched via `BaseNode.execute()` + the `@Operation("search")` method |
| **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) |
| **Skill (if any)** | [`server/skills/web_agent/duckduckgo-search-skill/SKILL.md`](../../../server/skills/web_agent/duckduckgo-search-skill/SKILL.md) |
-| **Dual-purpose tool** | tool-only - LLM invokes as `web_search` (configurable via `toolName` param) |
+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `web_search` (`tool_name` class attr) |
## Purpose
@@ -19,75 +19,62 @@ default web search tool for AI Agents, complementing the keyed alternatives
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| (none) | - | - | Passive node |
+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |
## Parameters
+The `DuckDuckGoSearchParams` model fields ARE the LLM-provided tool args (no
+separate `toolName` / `toolDescription` node params — those live on the class
+as `tool_name` / `tool_description`).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `web_search` | yes | - | LLM-visible tool name |
-| `toolDescription` | string | (see frontend) | no | - | LLM-visible description |
-| `maxResults` | number | `5` | no | - | Result cap (frontend clamps `1..10`); cast to `int` in handler |
-
-### LLM-provided tool args (at invocation time)
-
-| Arg | Type | Description |
-|-----|------|-------------|
-| `query` | string | Search query; empty string short-circuits with an error |
+| `query` | string | (required) | yes | - | Search query; `min_length=1` (empty fails Pydantic validation) |
+| `max_results` | int | `5` | no | - | Result cap; Pydantic-clamped `1..20` (`ge=1, le=20`) |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-tool` | object | Raw dict returned to the LLM |
+| `output-tool` | object | `DuckDuckGoSearchOutput` model, serialized per `BaseNode._serialize_result` |
### Output payload (TypeScript shape)
-On success:
+On success (matches the `DuckDuckGoSearchOutput` model):
```ts
{
query: string;
results: Array<{ title: string; snippet: string; url: string }>;
- provider: 'duckduckgo';
+ provider: 'duckduckgo'; // fixed default
}
```
-On empty query:
-```ts
-{ error: 'No search query provided' }
-```
+On a `DDGSException` (rate-limit / network / bot detection) the op raises
+`NodeUserError("DuckDuckGo search failed: <e>")`, which `BaseNode.execute()`
+catches into a single WARN line + `{success: false, error_type: "NodeUserError",
+error: ...}` envelope (no traceback).
## Logic Flow
```mermaid
flowchart TD
- A[_execute_duckduckgo_search args node_params] --> B{query non-empty?}
- B -- no --> Eq[Return error: No search query provided]
- B -- yes --> C[provider = node_params.provider OR duckduckgo<br/>max_results = int node_params.maxResults OR 5]
- C --> D{provider == duckduckgo}
- D -- yes --> E{Try import ddgs}
- E -- ok --> F[run_in_executor DDGS.text query max_results]
- F --> G[Map title / body-&gt;snippet / href-&gt;url]
- G --> R[Return query / results / provider]
- E -- ImportError --> H[Fallback: httpx GET api.duckduckgo.com Instant Answer API]
- H --> I[Map AbstractText + RelatedTopics into results]
- I --> R
- F -- Exception --> Eexc[Logged; propagates to outer generic handler]
+ A[BaseNode.execute -> search ctx params] --> B[import ddgs DDGS + DDGSException]
+ B --> F[run_in_executor: DDGS.text query max_results]
+ F -- DDGSException --> Eq[raise NodeUserError: DuckDuckGo search failed]
+ F -- ok --> G[Map title / body-&gt;snippet / href-&gt;url]
+ G --> R[Return DuckDuckGoSearchOutput query/results/provider]
```
## Decision Logic
-- **Empty query**: `if not query: return {error: ...}` - no library call.
-- **Provider switch**: `node_params.provider` defaults to `'duckduckgo'` and
- the handler only implements that branch. Any other value falls through
- without raising (returns whatever the outer try returns, typically nothing
- useful). The frontend node does not expose `provider`, so in practice this
- branch is always taken.
-- **`ddgs` ImportError fallback**: the handler quietly falls back to the
- DuckDuckGo Instant Answer API (`api.duckduckgo.com`) via `httpx`, which
- returns a very different shape (abstracts + related topics, not ranked web
- results). Shown logs: `"ddgs not installed, falling back to Instant Answer API"`.
-- **Sync-in-async**: `DDGS().text(...)` is synchronous; handler wraps it in
+- **Empty query**: rejected at Pydantic validation (`query` has `min_length=1`)
+ before the op runs — no library call.
+- **No provider switch / no fallback**: the plugin always uses the `ddgs`
+ library `DDGS().text(...)`. The old Instant Answer API (`api.duckduckgo.com`)
+ httpx fallback was removed; `ddgs` is a hard dependency now.
+- **`DDGSException`**: caught and re-raised as `NodeUserError` so the LLM can
+ retry with a different query (one WARN line, no traceback).
+- **Sync-in-async**: `DDGS().text(...)` is synchronous; the op wraps it in
`loop.run_in_executor(None, ...)` to avoid blocking the event loop.
## Side Effects
@@ -96,11 +83,8 @@ flowchart TD
this handler does **not** write to `api_usage_metrics` - it is free and
tracking cost is skipped.)
- **Broadcasts**: none.
-- **External API calls**:
- - Primary: `ddgs` library (HTTPS to `duckduckgo.com`; the library manages
- its own networking, timeout, and rate limits).
- - Fallback: `GET https://api.duckduckgo.com/?q=<query>&format=json&no_html=1`
- (timeout 10s) via `httpx.AsyncClient`.
+- **External API calls**: `ddgs` library (HTTPS to `duckduckgo.com`; the
+ library manages its own networking, timeout, and rate limits).
- **File I/O**: none.
- **Subprocess**: none.
@@ -108,27 +92,22 @@ flowchart TD
- **Credentials**: none.
- **Services**: none.
-- **Python packages**: `ddgs` (preferred), `httpx` (fallback).
+- **Python packages**: `ddgs` (hard dependency; lazy-imported inside the op).
- **Environment variables**: none.
## Edge cases & known limits
-- Handler does **not** wrap network errors from `ddgs` - they propagate up to
- `execute_tool`, which will bubble them into the tool-call result as an
- uncaught exception. (Mocking the library in tests is therefore required.)
-- Missing `ddgs` silently switches to Instant Answer API, which returns a
- noticeably smaller and differently-structured result set (abstract +
- related topics, not ranked web results). The `provider` field still says
- `"duckduckgo"`, masking the fallback.
-- `maxResults` clamp (`1..10`) is enforced only by the frontend; an LLM could
- in theory pass `node_params.maxResults = 999` via stored defaults and the
- handler would honour it.
-- `results[].url` uses the `href` field from `ddgs`; older library versions
- used a different key - the handler fills missing keys with `""` to stay
- defensive.
+- A `DDGSException` (the only caught error) becomes a `NodeUserError`; other
+ unexpected exceptions bubble up to `BaseNode.execute()`'s generic handler
+ with a full traceback.
+- `max_results` clamp (`1..20`) is enforced by Pydantic on the model
+ (`ge=1, le=20`), so an out-of-range LLM value fails validation rather than
+ being silently honoured.
+- `results[].url` uses the `href` field from `ddgs`; missing keys default to
+ `""` to stay defensive.
## Related
-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)
+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)
- **Companion search nodes** (API-keyed, track cost): [`braveSearch`](../search/braveSearch.md), [`serperSearch`](../search/serperSearch.md), [`perplexitySearch`](../search/perplexitySearch.md)
- **Skill using this tool**: [`duckduckgo-search-skill/SKILL.md`](../../../server/skills/web_agent/duckduckgo-search-skill/SKILL.md)
diff --git a/docs-internal/node-logic-flows/ai_tools/taskManager.md b/docs-internal/node-logic-flows/ai_tools/taskManager.md
index 54da3352..69df9b55 100644
--- a/docs-internal/node-logic-flows/ai_tools/taskManager.md
+++ b/docs-internal/node-logic-flows/ai_tools/taskManager.md
@@ -2,42 +2,53 @@
| Field | Value |
|------|-------|
-| **Category** | ai_tools (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/tools.py::handle_task_manager`](../../../server/services/handlers/tools.py) (workflow-node entry) -> `_execute_task_manager` |
+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |
+| **Backend handler** | [`server/nodes/tool/task_manager/__init__.py`](../../../server/nodes/tool/task_manager/__init__.py) — `TaskManagerNode`, dispatched via `BaseNode.execute()` + the `@Operation("manage")` method, which calls the inlined `_execute_task_manager` in the same file |
| **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) |
| **Skill (if any)** | [`server/skills/task_agent/task-manager-skill/SKILL.md`](../../../server/skills/task_agent/task-manager-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `task_manager`; handler registered under both registry key `taskManager` and `execute_tool` dispatch |
+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `task_manager` (`tool_name` class attr) |
## Purpose
-Inspects the in-process registry of delegated agent tasks (`_delegated_tasks`
-and `_delegation_results` dicts in `tools.py`). Lets a parent agent (or a
-workflow node run) list currently-running / completed delegations, fetch
-details for a specific `task_id`, or clear a finished task from tracking.
-Does **not** spawn work itself; it only reads/mutates the delegation state
-populated by `_execute_delegated_agent`.
+Inspects the in-process registry of delegated agent tasks. The
+`_execute_task_manager` body lives in the plugin file but reads **through** to
+the delegation registry owned by `services/handlers/tools.py` (the
+`_delegated_tasks` + `_delegation_results` dicts + `get_delegated_task_status`
+3-layer lookup) — that lifecycle state stays in `tools.py` intentionally. Lets a
+parent agent list currently-running / completed delegations, fetch details for a
+specific `task_id`, or clear a finished task from tracking. Does **not** spawn
+work itself; it only reads/mutates the delegation state populated by
+`_execute_delegated_agent`.
## Inputs (handles)
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| (none) | - | - | Passive node |
+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |
## Parameters
+The declared `TaskManagerParams` model fields are below. The model carries
+`model_config = ConfigDict(extra="allow")`, so the LLM-driven `operation` /
+`status_filter` args (which `_execute_task_manager` reads) pass through even
+though they are not declared fields.
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `task_manager` | no | - | LLM-visible tool name |
-| `toolDescription` | string | (see frontend) | no | - | LLM-visible description |
-| `operation` | options | `list_tasks` | no | - | One of `list_tasks`, `get_task`, `mark_done` (workflow-node mode) |
-| `task_id` | string | `""` | no | `operation in [get_task, mark_done]` | Target task id |
-| `status_filter` | options | `""` | no | `operation == list_tasks` | One of `""`, `running`, `completed`, `error` |
+| `action` | enum | `create` | no | - | Declared field: one of `create`, `list`, `complete`, `delete`, `update` (not the dispatch discriminator — see below) |
+| `task_id` | string? | `None` | no | - | Target task id |
+| `title` | string? | `None` | no | - | Declared but unused by the inlined op |
+| `description` | string? | `None` | no | - | Declared but unused by the inlined op |
+| `status` | string? | `None` | no | - | Declared but unused by the inlined op |
+
+### LLM-provided tool args (the operations the op actually dispatches on)
-### LLM-provided tool args (when used as a tool)
+The `manage` op passes `params.model_dump()` (with extras) into
+`_execute_task_manager`, which reads `operation` / `task_id` / `status_filter`:
| Arg | Type | Description |
|-----|------|-------------|
-| `operation` | string | Same three values; falls back to node param, then to `'list_tasks'` |
+| `operation` | string | One of `list_tasks`, `get_task`, `mark_done`; defaults to `'list_tasks'` |
| `task_id` | string | Required for `get_task` / `mark_done`; otherwise ignored |
| `status_filter` | string | Optional filter applied to `list_tasks` |
@@ -45,8 +56,7 @@ populated by `_execute_delegated_agent`.
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-tool` | object | Raw dict returned to the LLM |
-| `output-main` | object | Same payload, available when run as a workflow node |
+| `output-tool` | object | Raw dict returned to the LLM (the `TaskManagerOutput` model has `extra="allow"`, so the full operation dict passes through) |
### Output payload (TypeScript shape)
@@ -99,7 +109,8 @@ Unknown operation: `{success: false, error: "Unknown operation: <op>"}`.
```mermaid
flowchart TD
- A[_execute_task_manager args config] --> B[Merge operation / task_id / status_filter<br/>args wins, then node params, then defaults]
+ A[BaseNode.execute -> manage ctx params] --> A2[_execute_task_manager params.model_dump config]
+ A2 --> B[Read operation / task_id / status_filter<br/>tool_args wins, then params, then defaults]
B --> C{operation}
C -- list_tasks --> D[Iterate _delegated_tasks -&gt; derive running/completed/cancelled/error]
D --> E[Merge _delegation_results entries not already in list]
@@ -171,11 +182,12 @@ flowchart TD
three aggregate counters.
- Unknown `operation` returns a failed envelope rather than raising; the
agent might keep retrying with the same bad operation string.
-- `handle_task_manager` (workflow-node entry) passes an empty `tool_args`
- dict and relies entirely on node parameters.
+- The dispatch discriminator is `operation` (read from the extra-allowed
+ tool args), NOT the declared `action` field — the `action` / `title` /
+ `description` / `status` model fields are vestigial and unused by the op.
## Related
-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`writeTodos`](./writeTodos.md)
+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)
- **Skill using this tool**: [`task-manager-skill/SKILL.md`](../../../server/skills/task_agent/task-manager-skill/SKILL.md)
- **Architecture docs**: [Agent Delegation](../../agent_delegation.md)
diff --git a/docs-internal/node-logic-flows/ai_tools/writeTodos.md b/docs-internal/node-logic-flows/ai_tools/writeTodos.md
index a092c7e3..da1ec5a1 100644
--- a/docs-internal/node-logic-flows/ai_tools/writeTodos.md
+++ b/docs-internal/node-logic-flows/ai_tools/writeTodos.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | ai_tools (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/todo.py::handle_write_todos`](../../../server/services/handlers/todo.py) (workflow) / `execute_write_todos` (tool) |
+| **Backend handler** | [`server/nodes/tool/write_todos/__init__.py`](../../../server/nodes/tool/write_todos/__init__.py) — `WriteTodosNode`, dispatched via `BaseNode.execute()` + the `@Operation("write")` method (logic inlined; the standalone `services/handlers/todo.py::handle_write_todos` / `execute_write_todos` still exist but are no longer the dispatch path) |
| **Service** | [`server/services/todo_service.py::TodoService`](../../../server/services/todo_service.py) |
| **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) |
| **Skill (if any)** | [`server/skills/assistant/write-todos-skill/SKILL.md`](../../../server/skills/assistant/write-todos-skill/SKILL.md) |
@@ -51,51 +51,49 @@ generic-params card. `uiHints.isTodoEditor` (declared on the plugin) routes
## Parameters
+The `WriteTodosParams` model field IS the LLM-provided tool arg (no separate
+`toolName` / `toolDescription` node params — those live on the class as
+`tool_name` / `tool_description`).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `write_todos` | no | - | LLM-visible tool name |
-| `toolDescription` | string | (see frontend, 3-row hint) | no | - | LLM-visible description |
-
-### LLM-provided tool args (at invocation time)
-
-| Arg | Type | Description |
-|-----|------|-------------|
-| `todos` | `Array<{content: string, status: 'pending' \| 'in_progress' \| 'completed'}>` | Full replacement snapshot of the todo list |
+| `todos` | `Array<TodoItem>` | `[]` | no | - | Full replacement snapshot. `TodoItem = {id?: string, content: string, status: 'pending' \| 'in_progress' \| 'completed'}`. A JSON-encoded string is coerced to a list by the `_coerce_todos` `field_validator(mode="before")` (handles Gemini stringified-array args). |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-tool` | object | Tool result returned to the LLM |
-| `output-main` | object | Same payload, available when run as a workflow node |
+| `output-tool` | object | Tool result returned to the LLM (`WriteTodosOutput` has `extra="allow"`, so the dict below passes through) |
### Output payload (TypeScript shape)
```ts
{
- success: true;
message: string; // "Updated todo list (<N> items)"
- todos: string; // JSON string of the validated list
+ todos: Array<{ id?: string; content: string; status: string }>; // the validated list (NOT a JSON string)
count: number; // len(stored)
}
```
+Note: the op returns the validated `todos` LIST, not a JSON string. The
+pre-fix `format_for_llm()` call that stringified this leaked a raw JSON string
+into the key, which the `WriteTodosOutput` contract now rejects — the
+LLM-facing serialization happens downstream.
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_write_todos node_id / parameters / context] --> B[Build config: node_id + workflow_id + broadcaster]
- B --> C[execute_write_todos tool_args=parameters config]
- C --> D[session_key = workflow_id OR node_id OR default]
+ A[BaseNode.execute -> write ctx params] --> D[session_key = ctx.workflow_id OR ctx.node_id OR default]
D --> E[service = get_todo_service singleton]
- E --> F[service.write session_key todos_input]
- F --> G[Per-item validation: dict? content stripped non-empty? status in VALID_STATUSES?]
- G --> H[Serialize validated list to JSON, store in _store session_key]
- H --> I{broadcaster + node_id?}
- I -- yes --> J[await broadcaster.update_node_status node_id executing<br/>data.phase = todo_update<br/>data.todos = stored<br/>workflow_id = workflow_id]
+ E --> F[service.write session_key todos dumped from params.todos]
+ F --> G[Per-item validation inside TodoService]
+ G --> I{broadcaster on ctx.raw?}
+ I -- yes --> J[await broadcaster.update_node_status node_id executing<br/>data.phase = todo_update<br/>data.todos = stored<br/>workflow_id]
I -- no --> K[skip]
- J --> L[Return success + message + todos JSON + count]
- K --> L
+ J --> M[dispatch_todos_updated CloudEvent via services.events.dispatch.emit]
+ K --> M
+ M --> L[Return message + todos list + count]
```
## Decision Logic
@@ -123,9 +121,14 @@ flowchart TD
- **In-memory state writes**: `TodoService._store[session_key] = json.dumps(validated)`.
State persists for the lifetime of the Python process only.
- **Database writes**: none.
-- **Broadcasts**: `StatusBroadcaster.update_node_status(node_id, "executing",
- {"phase": "todo_update", "todos": [...]}, workflow_id=workflow_id)` - one
- per successful write, when a broadcaster is present in context.
+- **Broadcasts** (two per successful write):
+ - `StatusBroadcaster.update_node_status(node_id, "executing",
+ {"phase": "todo_update", "todos": [...]}, workflow_id=...)` — drives the
+ canvas glow; only when a broadcaster is present on `ctx.raw`.
+ - `dispatch_todos_updated(...)` — typed `todos_updated` CloudEvent
+ (`com.machinaos.todos.updated`) via `services.events.dispatch.emit`
+ (`server/nodes/tool/write_todos/_events.py`), refreshing the open Current
+ Todos panel's `['todos', session_key]` query. Fires unconditionally.
- **External API calls**: none.
- **File I/O**: none.
- **Subprocess**: none.
@@ -148,16 +151,17 @@ flowchart TD
- **All validation silent**: dropping an item with empty content or coercing
an invalid status never produces a warning in the return payload - only a
DEBUG log line.
-- **Broadcast failures swallowed by caller**: `handle_write_todos` awaits
+- **Broadcast failures bubble up**: the `write` op awaits
`update_node_status`; if the broadcaster raises, the exception bubbles up
- to the executor (no `try/except` inside the handler).
-- **`todos` returned as a JSON string, not an object**: the LLM must parse
- it again. This matches Claude Code's `TodoWrite` tool convention.
+ to `BaseNode.execute()` (no `try/except` inside the op).
+- **`todos` returned as a list, not a JSON string**: the Output contract
+ declares `todos: Optional[list]`; returning a stringified list is rejected
+ by `BaseNode._serialize_result`.
- **Content is trimmed** (`.strip()`) but otherwise not sanitised - markdown
or HTML in content survives untouched.
## Related
-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md)
+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`agentBuilder`](./agentBuilder.md)
- **Skill using this tool**: [`write-todos-skill/SKILL.md`](../../../server/skills/assistant/write-todos-skill/SKILL.md)
- **Architecture docs**: [Status Broadcaster](../../status_broadcaster.md), [Agent Architecture](../../agent_architecture.md)
diff --git a/docs-internal/node-logic-flows/android/_pattern.md b/docs-internal/node-logic-flows/android/_pattern.md
index b7273cff..39f9978e 100644
--- a/docs-internal/node-logic-flows/android/_pattern.md
+++ b/docs-internal/node-logic-flows/android/_pattern.md
@@ -5,25 +5,26 @@ factory, and a single execution path. This document is the authoritative
description of that shared behaviour. Individual per-node docs link back here
for everything that is not node-specific.
-## Single handler, registry-driven dispatch
+## Single plugin base, registry-driven dispatch
+
+Wave 11.C migrated all 16 Android nodes to self-contained plugin folders
+under `server/nodes/android/<service>/__init__.py`. Each is a trivial
+subclass of the shared `AndroidServiceBase` (`server/nodes/android/_base.py`)
+that sets `type` / `display_name` / `description` / `tool_name` only. The old
+`server/services/handlers/android.py::handle_android_service` and
+`server/services/android_service.py` files are **deleted**. Dispatch flows
+through `BaseNode.execute()` -> the `@Operation("invoke")` method on
+`AndroidServiceBase`.
| Concern | Location |
|---------|----------|
-| Backend handler | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) |
-| Underlying service | [`server/services/android_service.py::AndroidService.execute_service`](../../../server/services/android_service.py) |
-| Registry binding | [`server/services/node_executor.py`](../../../server/services/node_executor.py) lines 228-230 |
-| Output flatten | [`server/services/node_executor.py`](../../../server/services/node_executor.py) lines 277-285 |
-
-The `NodeExecutor._build_handler_registry()` method binds every entry in
-`ANDROID_SERVICE_NODE_TYPES` to the same partial:
-
-```python
-for node_type in ANDROID_SERVICE_NODE_TYPES:
- registry[node_type] = partial(handle_android_service, android_service=self.android_service)
-```
+| Plugin base | [`server/nodes/android/_base.py::AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
+| Underlying service | [`server/nodes/android/_dispatcher.py::AndroidService.execute_service`](../../../server/nodes/android/_dispatcher.py) |
+| Registry binding | plugins self-register via `BaseNode.__init_subclass__`; merged into `NodeExecutor` via `_PLUGIN_HANDLERS` ([`server/services/node_executor.py`](../../../server/services/node_executor.py) line 124) |
+| Output flatten | [`server/services/node_executor.py`](../../../server/services/node_executor.py) lines 173-181 |
-The handler then maps the incoming `node_type` (camelCase) to a backend
-`service_id` (snake_case) via a hard-coded dict **inside the handler**:
+The `@Operation("invoke")` method maps the incoming `node_type` (camelCase) to a
+backend `service_id` (snake_case) via the `SERVICE_ID_MAP` dict in `_base.py`:
```python
SERVICE_ID_MAP = {
@@ -47,7 +48,8 @@ SERVICE_ID_MAP = {
```
The frontend hidden `service_id` param is **ignored** for known node types;
-the handler's own map wins. See "Known inconsistencies" below.
+the plugin's own map wins (`SERVICE_ID_MAP.get(self.type, ...)`). See "Known
+inconsistencies" below.
## Shared parameter set
@@ -61,12 +63,13 @@ the handler's own map wins. See "Known inconsistencies" below.
| `parameters` | dict or JSON string | `{}` | no | Per-action nested parameters |
| `package_name` | string | `""` | conditional | Only surfaced by `appLauncher`; handler promotes it into `parameters` automatically |
-Any extra root-level key in the `additional_param_keys` list is hoisted into
-`parameters` before dispatch. Currently the only key in that list is
-`package_name`.
+Root-level `package_name` is hoisted into `parameters` before dispatch by the
+`invoke` method (the only key promoted today).
-If `parameters` is received as a string the handler calls `json.loads()` on it;
-on `JSONDecodeError` the value is silently replaced with `{}`.
+If `parameters` is received as a string it is coerced via `json.loads()` —
+first in `AndroidServiceParams._coerce_parameters` (a `field_validator(mode="before")`)
+and again defensively inside `invoke`; on `JSONDecodeError` the value is
+silently replaced with `{}`.
## Execution paths
@@ -82,7 +85,7 @@ on `JSONDecodeError` the value is silently replaced with `{}`.
```mermaid
flowchart TD
- A[handle_android_service] --> B[Look up service_id via SERVICE_ID_MAP<br/>fallback to parameters.service_id then 'battery']
+ A[AndroidServiceBase.invoke @Operation] --> B[Look up service_id via SERVICE_ID_MAP self.type<br/>fallback to parameters.service_id then 'battery']
B --> C[Parse parameters<br/>coerce JSON string, promote package_name]
C --> D[android_service.execute_service<br/>node_id, service_id, action, parameters, host, port]
D --> E{relay client paired?}
@@ -163,13 +166,14 @@ preserved.
- `screenControlAutomation` -> `screen_control`
- `airplaneModeControl` -> `airplane_mode`
- The handler map wins because it is looked up first, but any consumer that
+ The plugin map wins because it is looked up first, but any consumer that
reads the hidden param directly would see a different value. The same
- discrepancy exists in `handlers/tools.py::_execute_android_service`.
+ discrepancy exists in the AI-tool path
+ `server/nodes/android/_base.py::execute_android_service_tool`.
2. **JSON parse failure is silent**. A `parameters` string that fails to parse
is replaced with `{}` and logged only at debug level.
3. **No validation of required action params**. `appLauncher` needs
- `package_name`, but the handler never checks for it - an empty value is
+ `package_name`, but the plugin never checks for it - an empty value is
passed through and the Android side is expected to error.
4. **`default_timeout` is service-instance-wide**. All sensor / camera / media
calls share the same timeout; there is no per-action override.
@@ -177,15 +181,20 @@ preserved.
dict whose keys collide with `result` metadata keys (`action`,
`service_id`, `timestamp`, ...), the flatten merges `data` **after**
metadata, so the data wins and the metadata is lost from the promoted view.
-6. **`execute_service` never raises**. Every exception path returns an envelope
- with `success=false` and the error string, so the handler itself never
- raises past `NodeExecutor`.
+6. **`execute_service` never raises, but `invoke` does on failure**.
+ `AndroidService.execute_service` returns a `success=false` envelope for
+ every exception path. The `@Operation("invoke")` wrapper then re-raises a
+ `RuntimeError` when `result["success"] is False` (using `result["error"]`),
+ so `BaseNode.execute()` produces the standard error envelope. On success
+ `invoke` returns `result["result"]` (or the whole dict if no `result` key).
## Related
- Android service infrastructure: [`server/services/android/`](../../../server/services/android/)
- Status broadcaster (two-state model): [Status Broadcaster](../../status_broadcaster.md)
- Android-agent skills: [`server/skills/android_agent/`](../../../server/skills/android_agent/)
-- Toolkit sub-node execution: see `TOOLKIT_NODE_TYPES` and
- `_execute_android_service` in
- [`server/services/handlers/tools.py`](../../../server/services/handlers/tools.py)
+- AI-tool dispatch (`androidTool` aggregator + direct service tools):
+ `execute_android_toolkit` and `execute_android_service_tool` in
+ [`server/nodes/android/_base.py`](../../../server/nodes/android/_base.py),
+ called from
+ [`server/services/handlers/tools.py::execute_tool`](../../../server/services/handlers/tools.py)
diff --git a/docs-internal/node-logic-flows/android/airplaneModeControl.md b/docs-internal/node-logic-flows/android/airplaneModeControl.md
index 49cde069..2a34ea56 100644
--- a/docs-internal/node-logic-flows/android/airplaneModeControl.md
+++ b/docs-internal/node-logic-flows/android/airplaneModeControl.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / automation |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/airplane_mode_control/__init__.py`](../../../server/nodes/android/airplane_mode_control/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/appLauncher.md b/docs-internal/node-logic-flows/android/appLauncher.md
index c3b27f1a..0c13aa61 100644
--- a/docs-internal/node-logic-flows/android/appLauncher.md
+++ b/docs-internal/node-logic-flows/android/appLauncher.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / apps |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/app_launcher/__init__.py`](../../../server/nodes/android/app_launcher/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/app-launcher-skill/SKILL.md`](../../../server/skills/android_agent/app-launcher-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/appList.md b/docs-internal/node-logic-flows/android/appList.md
index 095610ab..5bf97111 100644
--- a/docs-internal/node-logic-flows/android/appList.md
+++ b/docs-internal/node-logic-flows/android/appList.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / apps |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/app_list/__init__.py`](../../../server/nodes/android/app_list/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/app-list-skill/SKILL.md`](../../../server/skills/android_agent/app-list-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/audioAutomation.md b/docs-internal/node-logic-flows/android/audioAutomation.md
index 087be70f..a6fcd004 100644
--- a/docs-internal/node-logic-flows/android/audioAutomation.md
+++ b/docs-internal/node-logic-flows/android/audioAutomation.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / automation |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/audio_automation/__init__.py`](../../../server/nodes/android/audio_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/audio-skill/SKILL.md`](../../../server/skills/android_agent/audio-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/batteryMonitor.md b/docs-internal/node-logic-flows/android/batteryMonitor.md
index fc08e420..5026224b 100644
--- a/docs-internal/node-logic-flows/android/batteryMonitor.md
+++ b/docs-internal/node-logic-flows/android/batteryMonitor.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / monitoring |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/battery_monitor/__init__.py`](../../../server/nodes/android/battery_monitor/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/battery-skill/SKILL.md`](../../../server/skills/android_agent/battery-skill/SKILL.md) |
| **Dual-purpose tool** | direct sub-node of `androidTool` - can also connect straight to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/bluetoothAutomation.md b/docs-internal/node-logic-flows/android/bluetoothAutomation.md
index f8304da5..3bd4f8ce 100644
--- a/docs-internal/node-logic-flows/android/bluetoothAutomation.md
+++ b/docs-internal/node-logic-flows/android/bluetoothAutomation.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / automation |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/bluetooth_automation/__init__.py`](../../../server/nodes/android/bluetooth_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/bluetooth-skill/SKILL.md`](../../../server/skills/android_agent/bluetooth-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/cameraControl.md b/docs-internal/node-logic-flows/android/cameraControl.md
index a5b04d9c..e54f2286 100644
--- a/docs-internal/node-logic-flows/android/cameraControl.md
+++ b/docs-internal/node-logic-flows/android/cameraControl.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / media |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/camera_control/__init__.py`](../../../server/nodes/android/camera_control/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/camera-skill/SKILL.md`](../../../server/skills/android_agent/camera-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/deviceStateAutomation.md b/docs-internal/node-logic-flows/android/deviceStateAutomation.md
index 00d072ad..1d221095 100644
--- a/docs-internal/node-logic-flows/android/deviceStateAutomation.md
+++ b/docs-internal/node-logic-flows/android/deviceStateAutomation.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / automation |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/device_state_automation/__init__.py`](../../../server/nodes/android/device_state_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/environmentalSensors.md b/docs-internal/node-logic-flows/android/environmentalSensors.md
index b8fb7807..44634ce6 100644
--- a/docs-internal/node-logic-flows/android/environmentalSensors.md
+++ b/docs-internal/node-logic-flows/android/environmentalSensors.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / sensors |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/environmental_sensors/__init__.py`](../../../server/nodes/android/environmental_sensors/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/environmental-skill/SKILL.md`](../../../server/skills/android_agent/environmental-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/location.md b/docs-internal/node-logic-flows/android/location.md
index d2c49391..145dd4b6 100644
--- a/docs-internal/node-logic-flows/android/location.md
+++ b/docs-internal/node-logic-flows/android/location.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / monitoring |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/location/__init__.py`](../../../server/nodes/android/location/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/location-skill/SKILL.md`](../../../server/skills/android_agent/location-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/mediaControl.md b/docs-internal/node-logic-flows/android/mediaControl.md
index cfa877e5..acd120bb 100644
--- a/docs-internal/node-logic-flows/android/mediaControl.md
+++ b/docs-internal/node-logic-flows/android/mediaControl.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / media |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/media_control/__init__.py`](../../../server/nodes/android/media_control/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/motionDetection.md b/docs-internal/node-logic-flows/android/motionDetection.md
index 1508a207..63e9ebef 100644
--- a/docs-internal/node-logic-flows/android/motionDetection.md
+++ b/docs-internal/node-logic-flows/android/motionDetection.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / sensors |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/motion_detection/__init__.py`](../../../server/nodes/android/motion_detection/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/motion-skill/SKILL.md`](../../../server/skills/android_agent/motion-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/networkMonitor.md b/docs-internal/node-logic-flows/android/networkMonitor.md
index 794e743d..34ba5a6f 100644
--- a/docs-internal/node-logic-flows/android/networkMonitor.md
+++ b/docs-internal/node-logic-flows/android/networkMonitor.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / monitoring |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/network_monitor/__init__.py`](../../../server/nodes/android/network_monitor/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | sub-node of `androidTool`; also connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/screenControlAutomation.md b/docs-internal/node-logic-flows/android/screenControlAutomation.md
index 37056c8e..e8d150c2 100644
--- a/docs-internal/node-logic-flows/android/screenControlAutomation.md
+++ b/docs-internal/node-logic-flows/android/screenControlAutomation.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / automation |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/screen_control_automation/__init__.py`](../../../server/nodes/android/screen_control_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/screen-control-skill/SKILL.md`](../../../server/skills/android_agent/screen-control-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/systemInfo.md b/docs-internal/node-logic-flows/android/systemInfo.md
index 2b38ed0a..fdf23370 100644
--- a/docs-internal/node-logic-flows/android/systemInfo.md
+++ b/docs-internal/node-logic-flows/android/systemInfo.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / monitoring |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/system_info/__init__.py`](../../../server/nodes/android/system_info/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/android/wifiAutomation.md b/docs-internal/node-logic-flows/android/wifiAutomation.md
index 458c2820..1dccb5f5 100644
--- a/docs-internal/node-logic-flows/android/wifiAutomation.md
+++ b/docs-internal/node-logic-flows/android/wifiAutomation.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | android / automation |
-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |
+| **Backend handler** | plugin [`server/nodes/android/wifi_automation/__init__.py`](../../../server/nodes/android/wifi_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |
| **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) |
| **Skill (if any)** | [`server/skills/android_agent/wifi-skill/SKILL.md`](../../../server/skills/android_agent/wifi-skill/SKILL.md) |
| **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |
diff --git a/docs-internal/node-logic-flows/chat_utility/chatHistory.md b/docs-internal/node-logic-flows/chat_utility/chatHistory.md
index bd7ebdda..c1bacdd3 100644
--- a/docs-internal/node-logic-flows/chat_utility/chatHistory.md
+++ b/docs-internal/node-logic-flows/chat_utility/chatHistory.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | chat_utility |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_chat_history`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | [`server/nodes/chat/chat_history/__init__.py`](../../../server/nodes/chat/chat_history/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("read")` |
| **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) |
| **Skill (if any)** | - |
| **Dual-purpose tool** | no |
@@ -25,20 +25,21 @@ workflows that need to read a conversation from a remote chat host.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `host` | string | `localhost` | no | - | Chat backend host |
-| `port` | number | `8080` | no | - | Chat backend port |
+| `port` | number | `8080` | no | - | Chat backend port (1..65535) |
| `session_id` | string | `default` | no | - | Chat session identifier |
-| `api_key` | string | `""` | no | - | Auth token forwarded to the chat backend |
-| `limit` | number | `50` | no | - | Maximum messages to return |
+| `api_key` | string (password) | `""` | no | - | Auth token forwarded to the chat backend |
+| `limit` | number | `50` | no | - | Maximum messages to return (1..500) |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | `{ messages: Array<ChatMessage> }` |
+| `output-main` | object | `{ messages: Array<ChatMessage>, count: number }` |
### Output payload (TypeScript shape)
```ts
+// ChatHistoryOutput (model_config extra="allow")
{
messages: Array<{
role: string; // e.g. "user" | "assistant"
@@ -46,6 +47,7 @@ workflows that need to read a conversation from a remote chat host.
created_at: string;
[key: string]: unknown;
}>;
+ count: number; // len(messages)
}
```
@@ -57,7 +59,7 @@ flowchart TD
B -- JSON-RPC error --> E[Return error envelope]
B -- RPC ok --> F{result.success?}
F -- false --> E
- F -- true --> G[Wrap result.messages into envelope]
+ F -- true --> G[Wrap result.messages + count into ChatHistoryOutput]
```
## Decision Logic
@@ -65,8 +67,9 @@ flowchart TD
- **Validation**: none; no parameter is required.
- **Branches**: success vs error branch on RPC response `success` flag.
- **Fallbacks**: `host`/`port`/`session_id`/`api_key`/`limit` all have defaults.
-- **Error paths**: every exception logged via `logger.error` and returned as
- `{success: false, error: str(e)}`.
+- **Error paths**: `not result.success` raises `RuntimeError(result.error or "chatHistory fetch failed")`;
+ `BaseNode.execute()` wraps the raised exception into the standard error envelope
+ (`success=false`, `error`, plus full traceback at `logger.exception` for non-`NodeUserError`).
## Side Effects
diff --git a/docs-internal/node-logic-flows/chat_utility/chatSend.md b/docs-internal/node-logic-flows/chat_utility/chatSend.md
index 7da8d7d9..d4447582 100644
--- a/docs-internal/node-logic-flows/chat_utility/chatSend.md
+++ b/docs-internal/node-logic-flows/chat_utility/chatSend.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | chat_utility |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_chat_send`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | [`server/nodes/chat/chat_send/__init__.py`](../../../server/nodes/chat/chat_send/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("send")` |
| **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) |
| **Skill (if any)** | - |
| **Dual-purpose tool** | no |
@@ -28,23 +28,27 @@ chat), `chatSend` pushes into a remote chat host.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `host` | string | `localhost` | no | - | Chat backend host |
-| `port` | number | `8080` | no | - | Chat backend port |
+| `port` | number | `8080` | no | - | Chat backend port (1..65535) |
| `session_id` | string | `default` | no | - | Chat session identifier |
-| `api_key` | string | `""` | no | - | Auth token forwarded to the chat backend |
-| `content` | string | `""` | yes | - | Message body |
+| `api_key` | string (password) | `""` | no | - | Auth token forwarded to the chat backend |
+| `content` | string | `""` | conditional | - | Message body. Defaults to `""` at the Param level; the op raises if both `content` and `message` are empty |
+| `message` | string | `""` | no | - | Legacy alias for `content` (used as fallback when `content` is empty) |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | The `result` payload returned by the chat backend |
+| `output-main` | object | `{ sent: true, message_id, ...rest }` from the chat backend |
### Output payload (TypeScript shape)
```ts
+// ChatSendOutput (model_config extra="allow")
{
- // Pass-through of whatever the chat backend's send_message RPC returned,
- // e.g. { message_id: string, timestamp: string, ... }
+ sent: true; // hard-set on success
+ message_id: string | null; // from RPC result.message_id
+ // plus every other key from the RPC result payload (extra="allow"),
+ // e.g. timestamp
[key: string]: unknown;
}
```
@@ -53,24 +57,26 @@ chat), `chatSend` pushes into a remote chat host.
```mermaid
flowchart TD
- A[Receive params] --> B{content non-empty?}
- B -- no --> E[Return error envelope: content required]
+ A[Receive params] --> B{content or message non-empty?}
+ B -- no --> E[Raise RuntimeError: Message content is required]
B -- yes --> C[Call services.chat_client.send_chat_message]
C -- JSON-RPC error --> E
C -- RPC ok --> F{result.success?}
F -- false --> E
- F -- true --> G[Return success envelope with result.result]
+ F -- true --> G[Return ChatSendOutput sent=true + message_id + rest]
```
## Decision Logic
-- **Validation**: empty `content` raises `ValueError("Message content is required")`
- which is caught and returned as `success=false`.
-- **Branches**: success vs error branch on RPC response `success` flag.
+- **Validation**: empty `content` AND empty `message` raises
+ `RuntimeError("Message content is required")`.
+- **Branches**: success vs error branch on RPC response `success` flag;
+ `content = params.content or params.message` (legacy alias fallback).
- **Fallbacks**: `host`/`port`/`session_id` default to `localhost:8080/default`,
`api_key` defaults to empty string.
-- **Error paths**: every exception is logged via `logger.error` and returned as
- `{success: false, error: str(e)}` with `node_id`, `node_type`, timing fields.
+- **Error paths**: `not result.success` raises `RuntimeError(result.error or "chatSend failed")`;
+ `BaseNode.execute()` wraps any raised exception into the standard error envelope
+ (`success=false`, `error`, `node_id`, `node_type`, timing fields).
## Side Effects
@@ -96,8 +102,8 @@ flowchart TD
- Any exception from `send_chat_message` (connection refused, timeout, malformed
RPC response) is swallowed and surfaced as `success=false` with the stringified
error. No retry is performed.
-- `port` is coerced via `int(parameters.get('port', 8080))`; a non-numeric port
- raises `ValueError` and is returned as `success=false`.
+- `port` is validated by the Pydantic Param (`int`, `ge=1, le=65535`); a non-coercible
+ or out-of-range value fails Param validation before the op runs.
- Templates in `content` (e.g. `{{aiAgent.response}}`) are resolved upstream by
`ParameterResolver`; this handler never sees unresolved `{{...}}`.
- `session_id` here is the chat-backend session, not the MachinaOs workflow
diff --git a/docs-internal/node-logic-flows/chat_utility/console.md b/docs-internal/node-logic-flows/chat_utility/console.md
index 72037213..32c3d9d9 100644
--- a/docs-internal/node-logic-flows/chat_utility/console.md
+++ b/docs-internal/node-logic-flows/chat_utility/console.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | chat_utility |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_console`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | [`server/nodes/utility/console/__init__.py`](../../../server/nodes/utility/console/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("log")` (logic fully inlined in the plugin) |
| **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) |
| **Skill (if any)** | - |
| **Dual-purpose tool** | no |
@@ -23,35 +23,39 @@ spliced into a pipeline without breaking data flow.
|--------|-----------------|----------|---------|
| `input-main` | main | no | Any number of upstream outputs merged into `input_data` |
-`console` is dispatched via `NodeExecutor._dispatch`'s special-handlers branch,
-which resolves upstream outputs via `_get_connected_outputs_with_info` and
-passes `connected_outputs` (keyed by source node type) and `source_nodes`
-(ids/types/labels of source nodes) to the handler.
+`console` declares its type in `NodeExecutor._NEEDS_CONNECTED_OUTPUTS`, so the
+executor injects `connected_outputs` (keyed by source node type) and
+`source_nodes` (ids/types/labels of source nodes) into `ctx.raw` before
+dispatching. The op reads `ctx.raw.get("connected_outputs")` /
+`ctx.raw.get("source_nodes")`. The node has `hide_output_handle = True` and
+`ui_hints = {"isConsoleSink": True}`, so it renders as a sink with only an
+`input-main` handle on the canvas.
## Parameters
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `label` | string | `""` | no | - | Display label for this console entry |
-| `logMode` | string | `all` | no | - | `all`, `field`, or `expression` |
+| `log_mode` | string | `all` | no | - | `all`, `field`, or `expression` |
+| `field_path` | string | `""` | no | `log_mode=field` | Dot path like `data.items[0].name` |
+| `expression` | string | `""` | no | `log_mode=expression` | Pre-resolved template output |
| `format` | string | `json` | no | - | `json`, `json_compact`, `text`, or `table` |
-| `fieldPath` | string | `""` | no | `logMode=field` | Dot path like `data.items[0].name` |
-| `expression` | string | `""` | no | `logMode=expression` | Pre-resolved template output |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | `{label, logged_at, format, data, formatted, ...input_data}` |
+| (no output handle) | object | `hide_output_handle = True` — renders as a sink, but the op still RETURNS the pass-through dict below so the output store can serve `{{console.*}}` downstream |
### Output payload (TypeScript shape)
```ts
+// ConsoleOutput (model_config extra="allow")
{
label: string; // user label or "Console (<nodeid[:8]>)"
logged_at: string; // ISO timestamp
format: string; // format parameter (json/text/table/json_compact)
- data: unknown; // the logged value (respects logMode)
+ data: unknown; // the logged value (respects log_mode)
formatted: string; // stringified representation for display
// plus every key from the merged upstream input so downstream nodes
// can keep templating against the upstream shape
@@ -63,12 +67,12 @@ passes `connected_outputs` (keyed by source node type) and `source_nodes`
```mermaid
flowchart TD
- A[Receive params + connected_outputs + source_nodes] --> B[Merge upstream dicts into input_data]
- B --> C{logMode}
+ A[Receive params + ctx.raw connected_outputs + source_nodes] --> B[Merge upstream dicts into input_data]
+ B --> C{log_mode}
C -- all --> D[log_value = input_data]
- C -- field --> E{fieldPath has curly braces OR key exists?}
- E -- resolved template --> F[log_value = fieldPath as-is]
- E -- literal path --> G[log_value = _navigate_field_path]
+ C -- field --> E{first segment of field_path is a key in input_data?}
+ E -- no --> F[log_value = field_path as-is resolved scalar]
+ E -- yes --> G[log_value = _navigate_field_path]
C -- expression --> H[log_value = expression or input_data]
C -- unknown --> D
D --> I[_format_console_output]
@@ -83,25 +87,25 @@ flowchart TD
- **Validation**: none; all parameters optional.
- **Branches**:
- - `logMode=all` - logs the full merged `input_data`.
- - `logMode=field` + `fieldPath` without `{{` and not a key in `input_data` ->
- treat `fieldPath` as a pre-resolved template value.
- - `logMode=field` + valid literal path -> `_navigate_field_path` returns
- nested value or `None` on mismatch.
- - `logMode=expression` -> use resolved `expression` or fall back to
+ - `log_mode=all` - logs the full merged `input_data`.
+ - `log_mode=field`: takes the first segment of `field_path` (before `.` / `[`).
+ If that segment matches a top-level key in `input_data` -> `_navigate_field_path`.
+ Otherwise the resolver already replaced `{{...}}` with a scalar, so log
+ `field_path` as-is. (Empty `field_path` -> `input_data`.)
+ - `log_mode=expression` -> use resolved `expression` or fall back to
`input_data` when empty.
- - Any other `logMode` falls through to `input_data` (via `match _`).
+ - Any other `log_mode` falls through to `input_data` (via `match _`).
- **Fallbacks**: non-dict upstream output is wrapped as `{"value": output}` in
`input_data`; label defaults to `Console (<first 8 chars of node_id>)`.
-- **Error paths**: top-level try/except returns `success=false` with the
- stringified exception.
+- **Error paths**: no in-op try/except; any raised exception is wrapped by
+ `BaseNode.execute()` into the standard error envelope.
## Side Effects
- **Database writes**: none (console logs are transient; they are NOT persisted
via `database.add_console_log` in this handler - persistence happens via the
WebSocket broadcast consumer in the frontend/DB layer).
-- **Broadcasts**: `StatusBroadcaster.broadcast_console_log` with
+- **Broadcasts**: `get_status_broadcaster().broadcast_console_log(...)` with
`{node_id, label, timestamp, data, formatted, format, workflow_id,
source_node_id, source_node_type, source_node_label}`.
- **External API calls**: none.
@@ -119,10 +123,10 @@ flowchart TD
- `_navigate_field_path` returns `None` for any missing key or out-of-range
index, so `log_value` silently becomes `null` rather than raising.
-- `fieldPath` heuristic: the handler treats a field path as "already a resolved
- template value" when it has no `{{` AND is not a key in `input_data`. This
- can misfire if the user's literal path happens not to match a top-level key
- even though the intent was to navigate.
+- `field_path` heuristic: the op treats a field path as "already a resolved
+ template scalar" when its first segment is NOT a top-level key in `input_data`.
+ This can misfire if the user's literal path happens not to match a top-level
+ key even though the intent was to navigate.
- `format=table` falls back to indented JSON for non-tabular data (not a list
of dicts).
- JSON formatting uses `default=str` so non-JSON-serialisable types become
diff --git a/docs-internal/node-logic-flows/chat_utility/fileHandler.md b/docs-internal/node-logic-flows/chat_utility/fileHandler.md
index 4aa4f205..5f70f78a 100644
--- a/docs-internal/node-logic-flows/chat_utility/fileHandler.md
+++ b/docs-internal/node-logic-flows/chat_utility/fileHandler.md
@@ -3,18 +3,19 @@
| Field | Value |
|------|-------|
| **Category** | chat_utility |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_file_handler`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | [`server/nodes/text/file_handler/__init__.py`](../../../server/nodes/text/file_handler/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("wrap")` → [`server/services/text.py::TextService.execute_file_handler`](../../../server/services/text.py) |
| **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) |
| **Skill (if any)** | - |
| **Dual-purpose tool** | no |
## Purpose
-Legacy file-shape wrapper retained from the Node.js-era engine. Does NOT touch
-the filesystem - it only composes a descriptive dict (`fileName`, `fileType`,
-`content`, `size`) around an already-in-memory string. For real filesystem
-access use the `fileRead`, `fileModify`, `fsSearch`, and `fileDownloader`
-nodes in the `code_fs_process` and `document` categories.
+Content-metadata wrapper (NOT file I/O), retained from the Node.js-era engine.
+It only composes a descriptive dict (`fileName`, `fileType`, `content`, `size`)
+around an already-in-memory string and returns a `{type: "file", data: {...}}`
+envelope for downstream nodes. For real filesystem access use the `fileRead`,
+`fileModify`, `fsSearch`, and `fileDownloader` nodes in the `code_fs_process`
+and `document` categories.
## Inputs (handles)
@@ -26,9 +27,13 @@ nodes in the `code_fs_process` and `document` categories.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `fileType` | string | `generic` | no | - | Arbitrary tag stored in output |
-| `content` | string | `""` | no | - | In-memory file contents |
-| `fileName` | string | `untitled.txt` | no | - | Display file name |
+| `file_type` | enum | `generic` | no | - | One of `generic`, `markdown`, `text`, `json`, `csv`, `html`, `xml` — content-type hint stored in output |
+| `file_name` | string | `untitled.txt` | no | - | Filename label attached to the wrapped content |
+| `content` | string | `""` | no | - | In-memory text content to wrap (8 rows) |
+
+Note: parameters in are snake_case (`file_type` / `file_name` / `content`), but
+the service's output dict keeps camelCase keys (`fileName` / `fileType`) by
+historical wire contract — the frontend reads them that way.
## Outputs (handles)
@@ -39,13 +44,15 @@ nodes in the `code_fs_process` and `document` categories.
### Output payload (TypeScript shape)
```ts
+// FileHandlerOutput (model_config extra="allow"); the service returns
+// result = { type, data, nodeId, timestamp }, which the op returns verbatim.
{
type: "file";
data: {
fileName: string;
fileType: string;
content: string;
- size: number; // len(content), bytes-of-str length
+ size: number; // len(content) — character count, not byte count
processed: true;
processingType: string; // === fileType
nodeId: string;
@@ -84,8 +91,7 @@ flowchart TD
## External Dependencies
- **Credentials**: none.
-- **Services**: `TextService` injected via `functools.partial` in
- `NodeExecutor._build_handler_registry`.
+- **Services**: `TextService` obtained via `services.plugin.deps.get_text_service()`.
- **Python packages**: stdlib only.
- **Environment variables**: none.
diff --git a/docs-internal/node-logic-flows/chat_utility/gmaps_create.md b/docs-internal/node-logic-flows/chat_utility/gmaps_create.md
index cc5f597c..dcec5aa0 100644
--- a/docs-internal/node-logic-flows/chat_utility/gmaps_create.md
+++ b/docs-internal/node-logic-flows/chat_utility/gmaps_create.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | chat_utility (grouping) / location (functional domain) |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_create_map`](../../../server/services/handlers/utility.py) -> [`server/services/maps.py::MapsService.create_map`](../../../server/services/maps.py) |
+| **Backend handler** | [`server/nodes/location/gmaps_create/__init__.py`](../../../server/nodes/location/gmaps_create/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("create")` → [`server/nodes/location/_service.py::MapsService.create_map`](../../../server/nodes/location/_service.py) |
| **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) |
| **Skill (if any)** | - |
| **Dual-purpose tool** | no (display-only; `gmaps_locations` and `gmaps_nearby_places` are the dual-purpose pair) |
@@ -23,13 +23,20 @@ the Maps API - it assembles the URL from parameters and tracks a
## Parameters
+Params model (`GmapsCreateParams`):
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `api_key` | string | - | yes (auto-injected) | - | Google Maps API key; auto-injected from `auth_service.get_api_key('google_maps')` or `settings.google_maps_api_key` |
-| `lat` | number | `40.7128` | no | - | Latitude, -90 .. 90 |
-| `lng` | number | `-74.0060` | no | - | Longitude, -180 .. 180 |
-| `zoom` | number | `13` | no | - | Zoom level, 0 .. 21 |
-| `map_type_id` | string | `ROADMAP` | no | - | One of `ROADMAP`, `SATELLITE`, `HYBRID`, `TERRAIN` |
+| `center_lat` | number | `0.0` | no | - | Map center latitude |
+| `center_lng` | number | `0.0` | no | - | Map center longitude |
+| `zoom` | number | `10` | no | - | Zoom level, `ge=1, le=20` (1=world, 20=street) |
+| `map_type` | enum | `roadmap` | no | - | One of `roadmap`, `satellite`, `hybrid`, `terrain` |
+| `options` | object | `{}` | no | - | Map customization (`disable_default_ui`, `zoom_control`, `street_view_control`, `map_type_control`, `fullscreen_control`); 4 rows |
+
+The Google Maps API key is NOT a node Param — `MapsService.create_map` reads it
+from `parameters.get("api_key")` (absent here) falling back to
+`settings.google_maps_api_key`. The `GoogleMapsCredential` (`google_maps`) is
+declared on the node for credential-modal visibility.
## Outputs (handles)
@@ -66,15 +73,18 @@ flowchart TD
## Decision Logic
-- **Validation**:
- - `api_key` required (from params, else `settings.google_maps_api_key`).
+- **Validation** (inside `MapsService.create_map`):
+ - `api_key` required (from `parameters.get("api_key")`, else `settings.google_maps_api_key`).
- `lat`/`lng` validated via `validate_coordinates`.
- `zoom` validated via `validate_zoom_level` (0-21).
- - `map_type` restricted to the four Google Maps MapTypeId values.
+ - `map_type` restricted to `["ROADMAP", "SATELLITE", "HYBRID", "TERRAIN"]` (uppercase).
- **Branches**: success envelope vs error envelope.
-- **Fallbacks**: defaults centre the map on New York City.
-- **Error paths**: any raised `ValueError` bubbles into the generic
- `except Exception` and returns `success=false`.
+- **Fallbacks**: when `lat`/`lng`/`zoom`/`map_type_id` keys are absent in the
+ dumped params, the service centres on New York City (40.7128, -74.0060), zoom 13,
+ `ROADMAP` (see the key-mismatch note under Edge cases).
+- **Error paths**: on `not response.success` the op raises
+ `NodeUserError(response.error or "Map create failed")` (single WARN line,
+ no traceback); `BaseNode.execute()` builds the structured error envelope.
## Side Effects
@@ -88,21 +98,26 @@ flowchart TD
## External Dependencies
-- **Credentials**: Google Maps API key via `auth_service.get_api_key('google_maps')`
- or `settings.google_maps_api_key` (env `GOOGLE_MAPS_API_KEY`).
-- **Services**: `MapsService` injected via `functools.partial` in
- `NodeExecutor._build_handler_registry`.
+- **Credentials**: `GoogleMapsCredential` (`google_maps`); the service resolves
+ the key from `parameters.get("api_key")` else `settings.google_maps_api_key`
+ (env `GOOGLE_MAPS_API_KEY`).
+- **Services**: `MapsService` obtained via `services.plugin.deps.get_maps_service()`.
- **Python packages**: stdlib only for this path (googlemaps is used by other
map handlers, not this one).
- **Environment variables**: `GOOGLE_MAPS_API_KEY` (fallback).
## Edge cases & known limits
+- Param/service key mismatch: the plugin Params are `center_lat` / `center_lng`
+ / `map_type`, but `MapsService.create_map` reads `lat` / `lng` / `map_type_id`.
+ With the current `params.model_dump()` wiring the service falls back to its
+ NYC / ROADMAP defaults regardless of the configured center / map type.
+ (Documented, not fixed here.)
- The static map URL embeds the API key in plain text - any downstream node or
log that captures the output leaks the key. Treat the output as sensitive.
- Size is hard-coded to `600x400`; there is no parameter to override.
-- `map_type` comparison is case-sensitive; `roadmap` (lowercase) is rejected as
- invalid even though Google Maps accepts it.
+- `map_type` validation is case-sensitive (uppercase `ROADMAP`...); the lowercase
+ enum values the Params expose would be rejected if they reached the service.
- Cost tracking is optimistic: the $0.002 is charged even if the frontend
never actually fetches the static map URL.
diff --git a/docs-internal/node-logic-flows/chat_utility/teamMonitor.md b/docs-internal/node-logic-flows/chat_utility/teamMonitor.md
index e2f5667a..e1f0b095 100644
--- a/docs-internal/node-logic-flows/chat_utility/teamMonitor.md
+++ b/docs-internal/node-logic-flows/chat_utility/teamMonitor.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | chat_utility |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_team_monitor`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | [`server/nodes/utility/team_monitor/__init__.py`](../../../server/nodes/utility/team_monitor/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("monitor")` |
| **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) |
| **Skill (if any)** | - |
| **Dual-purpose tool** | no |
@@ -26,7 +26,12 @@ architecture.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `maxHistoryItems` | number | `50` | no | - | Cap on `recent_events` slice length |
+| `team_id` | string | `""` | no | - | Explicit team id (usually resolved from upstream output instead) |
+| `auto_refresh` | boolean | `true` | no | - | UI flag for periodic refresh of the monitor panel |
+| `max_history_items` | number | `50` | no | - | Cap on `recent_events` slice length (`ge=1`) |
+
+The node carries `ui_hints = {"isMonitorPanel": True, "hideInputSection": True,
+"hideOutputSection": True}` — it renders as a dedicated monitor panel.
## Outputs (handles)
@@ -58,12 +63,12 @@ architecture.
```mermaid
flowchart TD
- A[Receive params + context] --> B{team_id in context?}
+ A[Receive params + ctx.raw] --> B{ctx.raw.team_id set?}
B -- yes --> E[team_service.get_team_status]
- B -- no --> C[Scan context.outputs for dict with team_id]
+ B -- no --> C[Scan ctx.raw.outputs for dict with team_id]
C -- found --> E
C -- not found --> D[Return empty envelope with message=No team connected]
- E --> F[Slice recent_events by maxHistoryItems]
+ E --> F[Slice recent_events by max_history_items]
F --> G[Return envelope with counters + lists]
```
@@ -71,14 +76,14 @@ flowchart TD
- **Validation**: none.
- **Branches**:
- - `team_id` resolved via `context['team_id']` first, else the first connected
- output that is a dict containing `team_id`.
+ - `team_id` resolved via `ctx.raw["team_id"]` first, else the first value in
+ `ctx.raw["outputs"]` that is a dict containing a truthy `team_id`.
- When no `team_id` is found, returns an intentional success envelope with
zero counters - the node does not error.
- **Fallbacks**: counters default to 0 for any missing key; `members`,
`active_tasks`, `recent_events` default to empty lists.
-- **Error paths**: any exception from `team_service.get_team_status` is caught
- and returned as `success=false` with the stringified error.
+- **Error paths**: no in-op try/except; any exception from
+ `get_team_status` is wrapped by `BaseNode.execute()` into the error envelope.
## Side Effects
@@ -100,8 +105,7 @@ flowchart TD
## Edge cases & known limits
-- `maxHistoryItems` is used as a negative slice (`[-max:]`); passing a
- non-integer triggers a `TypeError` which surfaces as `success=false`.
+- `max_history_items` is used as a negative slice (`[-max:]`) on `recent_events`.
- The "find team_id in outputs" loop stops at the first dict it sees containing
a truthy `team_id`; if two upstream outputs carry different team ids, behaviour
is order-dependent.
diff --git a/docs-internal/node-logic-flows/chat_utility/textGenerator.md b/docs-internal/node-logic-flows/chat_utility/textGenerator.md
index 72308e90..09d735c4 100644
--- a/docs-internal/node-logic-flows/chat_utility/textGenerator.md
+++ b/docs-internal/node-logic-flows/chat_utility/textGenerator.md
@@ -3,18 +3,18 @@
| Field | Value |
|------|-------|
| **Category** | chat_utility |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_text_generator`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | [`server/nodes/text/text_generator/__init__.py`](../../../server/nodes/text/text_generator/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("generate")` → [`server/services/text.py::TextService.execute_text_generator`](../../../server/services/text.py) |
| **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) |
| **Skill (if any)** | - |
| **Dual-purpose tool** | no |
## Purpose
-Trivial text factory node retained from the original Node.js-era workflow
-engine. Emits a text string and its length (optionally with a timestamp). Used
-by legacy workflows and as a scaffolding node during tests. Not surfaced in the
-current React component palette but still reachable through the backend
-registry.
+Text factory node retained from the original Node.js-era workflow engine.
+Emits a text string plus its length and an ISO timestamp. The Params expose a
+`source` selector (`static` / `ai` / `file` / `api`) for future expansion, but
+the current `TextService.execute_text_generator` only consumes `text` and
+returns the static-text envelope regardless of `source`.
## Inputs (handles)
@@ -24,10 +24,18 @@ registry.
## Parameters
+Params model (`TextGeneratorParams`, `model_config extra="allow"`):
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `text` | string | `Hello World` | no | - | Text payload to emit |
-| `includeTimestamp` | boolean | `true` | no | - | Include ISO timestamp in `data` |
+| `source` | enum | `static` | no | - | One of `static`, `ai`, `file`, `api` (only `static` behaviour is implemented today) |
+| `text` | string | `""` | no | - | Text payload to emit (4 rows) |
+| `file_path` | string | `""` | no | - | Reserved for `source=file` (unused by the current service) |
+| `api_url` | string | `""` | no | - | Reserved for `source=api` (unused by the current service) |
+
+Note: the service reads `include_timestamp` (default `True`) but no Param exposes
+it, so the ISO `timestamp` is always attached to `data`. If `text` is omitted the
+service substitutes `"Hello World"`.
## Outputs (handles)
@@ -38,13 +46,16 @@ registry.
### Output payload (TypeScript shape)
```ts
+// Service returns result = { type, data, nodeId, timestamp }, returned verbatim.
+// The TextGeneratorOutput model (extra="allow") declares text/source but the
+// op returns the service result dict, so the runtime shape is:
{
type: "text";
data: {
text: string;
- length: number;
+ length: number; // len(text) — character count
nodeId: string;
- timestamp?: string; // only when includeTimestamp=true
+ timestamp: string; // always present (include_timestamp defaults True)
};
nodeId: string;
timestamp: string;
@@ -57,20 +68,18 @@ registry.
flowchart TD
A[Receive params] --> B[TextService.execute_text_generator]
B --> C[Compose data with text/length/nodeId]
- C --> D{includeTimestamp?}
+ C --> D{include_timestamp default True}
D -- yes --> E[Attach isoformat timestamp]
- D -- no --> F[Skip timestamp]
- E --> G[Return success envelope]
- F --> G
+ E --> G[Return service result envelope]
```
## Decision Logic
-- **Validation**: none; both params have defaults.
-- **Branches**: `includeTimestamp` toggles the nested timestamp.
-- **Fallbacks**: `text` defaults to `Hello World`.
-- **Error paths**: any exception inside `TextService.execute_text_generator`
- is caught there and returned as `success=false`.
+- **Validation**: none; all params have defaults.
+- **Branches**: `source` is accepted but ignored; only the static-text path runs.
+- **Fallbacks**: empty `text` falls back to `"Hello World"` inside the service.
+- **Error paths**: the service catches exceptions and returns `success=false`;
+ on `not response.success` the op raises `RuntimeError(response.error or "Text generator failed")`.
## Side Effects
@@ -83,8 +92,7 @@ flowchart TD
## External Dependencies
- **Credentials**: none.
-- **Services**: `TextService` injected via `functools.partial` in
- `NodeExecutor._build_handler_registry`.
+- **Services**: `TextService` obtained via `services.plugin.deps.get_text_service()`.
- **Python packages**: stdlib only.
- **Environment variables**: none.
diff --git a/docs-internal/node-logic-flows/code_fs_process/fileModify.md b/docs-internal/node-logic-flows/code_fs_process/fileModify.md
index 1bd45aba..c70dc0e4 100644
--- a/docs-internal/node-logic-flows/code_fs_process/fileModify.md
+++ b/docs-internal/node-logic-flows/code_fs_process/fileModify.md
@@ -3,8 +3,8 @@
| Field | Value |
|------|-------|
| **Category** | code_fs_process / filesystem |
-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_file_modify`](../../../server/services/handlers/filesystem.py) |
-| **Backend** | [`deepagents.backends.LocalShellBackend`](https://github.com/langchain-ai/deepagents) |
+| **Backend handler** | [`server/nodes/filesystem/file_modify/__init__.py::FileModifyNode.modify`](../../../server/nodes/filesystem/file_modify/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("modify")`) |
+| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/file-modify-skill/SKILL.md`](../../../server/skills/coding_agent/file-modify-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `file_modify` |
@@ -13,8 +13,12 @@
Writes a new file or edits an existing one inside the per-workflow workspace.
Two operations: `write` (create/overwrite) and `edit` (string find-and-replace).
-Delegates to `LocalShellBackend.write()` / `backend.edit()` with
-`virtual_mode=True` sandboxing, invoked through `asyncio.to_thread()`.
+Delegates to `backend.write()` / `backend.edit()` on `NushellBackend`
+(`virtual_mode=True`, `inherit_env=True`) via `get_backend()`, invoked through
+`asyncio.to_thread()`. Paths are normalised with `normalize_virtual_path()`
+first. Because deepagents' `backend.write()` refuses to overwrite, the `write`
+op first unlinks any pre-existing file at the resolved path (wholesale
+create-or-replace), guarding against `IsADirectoryError`.
## Inputs (handles)
@@ -26,20 +30,21 @@ Delegates to `LocalShellBackend.write()` / `backend.edit()` with
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `operation` | options | `write` | no | - | `write` or `edit` |
+| `operation` | `write` \| `edit` (Literal) | `write` | no | - | `write` or `edit` |
| `file_path` | string | `""` | yes | - | Target file path |
| `content` | string | `""` | yes (when `operation=write`) | `operation=write` | File content |
| `old_string` | string | `""` | yes (when `operation=edit`) | `operation=edit` | Text to find |
| `new_string` | string | `""` | no | `operation=edit` | Replacement text |
| `replace_all` | boolean | `false` | no | `operation=edit` | Replace every occurrence; when `false` the backend insists `old_string` is unique |
-| `working_directory` | string | `""` | no | - | Overrides context workspace |
+
+`FileModifyParams` uses `extra="ignore"` — `working_directory` is NOT exposed by
+this node (the model drops unknown keys before `model_dump()`).
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `file_modify` tool result) |
### Output payload
@@ -53,46 +58,52 @@ Edit:
{ operation: "edit"; file_path: string; occurrences: number }
```
+`node_output_schemas.FileModifyOutput` declares `operation` / `file_path` /
+`occurrences` (the model's declared `Output` has `written` / `replacements`,
+but the operation returns the dicts above; `_OutputBase`/`extra="allow"` keeps
+them).
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_file_modify] --> B{file_path empty?}
- B -- yes --> E[Return error:<br/>file_path is required]
- B -- no --> C[_get_backend root=workspace]
+ A[modify] --> B{file_path empty?}
+ B -- yes --> E[raise NodeUserError:<br/>file_path is required]
+ B -- no --> C[get_backend + normalize_virtual_path]
C --> D{operation?}
- D -- write --> W[to_thread backend.write file_path, content]
- W --> W1{result.error?}
- W1 -- yes --> Eenv[Return error result.error]
- W1 -- no --> Wok[Return success<br/>operation=write, file_path=result.path]
+ D -- write --> W[to_thread _do_write:<br/>unlink existing if file,<br/>raise IsADir if dir, backend.write]
+ W -- OSError/ValueError --> Eenv[raise NodeUserError str e]
+ W -- result.error --> Eenv
+ W -- ok --> Wok[Return dict<br/>operation=write, file_path=result.path]
D -- edit --> Ei{old_string empty?}
- Ei -- yes --> Ereq[Return error:<br/>old_string is required for edit]
+ Ei -- yes --> Ereq[raise NodeUserError:<br/>old_string is required for edit]
Ei -- no --> Ed[to_thread backend.edit file_path,<br/>old_string, new_string, replace_all]
- Ed --> Ed1{result.error?}
- Ed1 -- yes --> Eenv
- Ed1 -- no --> Edok[Return success<br/>occurrences=result.occurrences]
- D -- other --> Unk[Return error:<br/>Unknown operation: <op>]
+ Ed -- result.error --> Eenv
+ Ed -- ok --> Edok[Return dict<br/>occurrences=result.occurrences]
+ D -- other --> Unk[raise NodeUserError:<br/>Unknown operation: <op>]
```
## Decision Logic
- **Validation**:
- - empty `file_path` -> `"file_path is required"`.
- - `operation=edit` with empty `old_string` -> `"old_string is required for
- edit"`.
- - Anything other than `write` / `edit` -> `"Unknown operation: <op>"`.
-- **Backend-level errors**: both `backend.write()` and `backend.edit()`
- return a result object with an `error` field. A non-empty `error` short-
- circuits before the success envelope is built. Common backend errors:
- non-unique `old_string` when `replace_all=False`, path escape with
- `virtual_mode=True`.
-- **`operations.edit` uniqueness constraint**: when `replace_all=False`, the
- backend REQUIRES `old_string` to appear exactly once. Zero or multiple
- matches trigger a backend-level error.
+ - empty `file_path` -> `raise NodeUserError("file_path is required")`.
+ - `operation=edit` with empty `old_string` -> `raise NodeUserError("old_string
+ is required for edit")`.
+ - Anything other than `write` / `edit` -> `raise NodeUserError("Unknown
+ operation: <op>")` (Pydantic Literal already constrains it).
+- **`write` overwrite handling**: `_do_write` resolves the path, raises
+ `IsADirectoryError` if it's a directory, `unlink()`s an existing file, then
+ calls `backend.write()` (which itself refuses to overwrite). `OSError` /
+ `ValueError` are caught and re-raised as `NodeUserError`.
+- **Backend-level errors**: a non-empty `result.error` from `write`/`edit`
+ short-circuits with `raise NodeUserError(result.error)`. Common: non-unique
+ `old_string` when `replace_all=False`, path escape with `virtual_mode=True`.
+- **`edit` uniqueness constraint**: when `replace_all=False`, the backend
+ REQUIRES `old_string` to appear exactly once. Zero or multiple matches
+ trigger a backend-level error.
- **`file_path` echoed from `result.path`**: on success the returned
`file_path` comes from the backend's normalised path, falling back to the
- user-supplied path only if `result.path` is empty.
-- **Broad `except Exception`**: catches unexpected backend or I/O errors.
+ normalised `file_path` only if `result.path` is empty.
## Side Effects
@@ -100,26 +111,27 @@ flowchart TD
- **Broadcasts**: none.
- **External API calls**: none.
- **File I/O**:
- - `os.makedirs(root, exist_ok=True)` on the workspace root.
- - Writes/edits `<root>/<file_path>`.
+ - `get_backend` ensures the workspace root exists.
+ - For `write`: `unlink()` an existing file then writes `<root>/<file_path>`.
+ - For `edit`: rewrites `<root>/<file_path>` in place.
- **Subprocess**: none.
## External Dependencies
-- **Python packages**: `deepagents`.
+- **Python packages**: `deepagents` (via `NushellBackend`).
- **Environment variables**: `WORKSPACE_BASE_DIR`.
## Edge cases & known limits
- **Silent overwrite on `write`**: there is no "fail if exists" flag - every
- `write` call replaces the file.
-- **`edit` without a match**: returns an error envelope (backend surfaces the
+ `write` call unlinks then replaces the file.
+- **`edit` without a match**: raises `NodeUserError` (backend surfaces the
message). The handler never distinguishes "not found" from "multiple
- matches" - both are string errors.
+ matches" - both are `NodeUserError` strings.
- **No encoding option**: the backend opens files in text mode with the
platform default encoding.
-- **`working_directory` escape**: as in `fileRead`, the node trusts the
- parameter which can bypass the workspace sandbox.
+- **`working_directory` not exposed**: `extra="ignore"` means the sandbox
+ cannot be widened via a node param on this node.
- **`replace_all=true` with empty `new_string`**: effectively deletes every
occurrence of `old_string`. No safeguard.
- **Atomicity**: the backend does not do atomic writes - a crash mid-write
diff --git a/docs-internal/node-logic-flows/code_fs_process/fileRead.md b/docs-internal/node-logic-flows/code_fs_process/fileRead.md
index 871d1c2c..1812d8cb 100644
--- a/docs-internal/node-logic-flows/code_fs_process/fileRead.md
+++ b/docs-internal/node-logic-flows/code_fs_process/fileRead.md
@@ -3,20 +3,24 @@
| Field | Value |
|------|-------|
| **Category** | code_fs_process / filesystem |
-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_file_read`](../../../server/services/handlers/filesystem.py) |
-| **Backend** | [`deepagents.backends.LocalShellBackend`](https://github.com/langchain-ai/deepagents) (third-party) |
+| **Backend handler** | [`server/nodes/filesystem/file_read/__init__.py::FileReadNode.read`](../../../server/nodes/filesystem/file_read/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("read")`) |
+| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/file-read-skill/SKILL.md`](../../../server/skills/coding_agent/file-read-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `file_read` |
## Purpose
-Reads a file within the per-workflow workspace. Delegates to
-`LocalShellBackend.read()` from the `deepagents` package. The backend is
-instantiated with `virtual_mode=True` and a `root_dir` pinned to
-`<DATA_DIR>/workspaces/<workflow_slug>/` (or the node-level `working_directory`
-override), so path traversal outside the workspace is rejected by the
-backend itself.
+Reads a file within the per-workflow workspace. Delegates to `backend.read()`
+on `NushellBackend` (a `LocalShellBackend` subclass) via `get_backend()` in
+[`_backend.py`](../../../server/nodes/filesystem/_backend.py). The backend is
+instantiated with `virtual_mode=True`, `inherit_env=True`, and a `root_dir`
+pinned to `<DATA_DIR>/workspaces/<workflow_slug>/` (resolution:
+`working_directory` param > `ctx.workspace_dir` >
+`Settings().workspace_base_resolved/default`), so path traversal outside the
+workspace is rejected by the backend. The plugin first runs the requested
+`file_path` through `normalize_virtual_path()`, which strips Windows drives /
+POSIX root / UNC anchors before the backend's `..`/`~` rejection.
All calls run inside `asyncio.to_thread()` because the underlying backend
APIs are synchronous file I/O.
@@ -32,53 +36,64 @@ APIs are synchronous file I/O.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `file_path` | string | `""` | yes | - | File path relative to the workspace root |
-| `offset` | number | `0` | no | - | 0-indexed starting line |
-| `limit` | number | `100` | no | - | Max lines to read (1-10000 in UI) |
-| `working_directory` | string | `""` | no | - | Overrides the context workspace; resolved before backend creation |
+| `offset` | number | `0` (ge=0) | no | - | 0-indexed starting line |
+| `limit` | number | `2000` (ge=1, le=10000) | no | - | Max lines to read |
+
+`FileReadParams` uses `extra="ignore"`, so `working_directory` is NOT exposed by
+this node — `get_backend()` reads it from the params dict, but the model drops
+unknown keys before `model_dump()`. Workspace resolution effectively uses
+`ctx.workspace_dir`.
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `file_read` tool result) |
### Output payload
```ts
{
- content: string; // File content returned by LocalShellBackend.read()
- file_path: string; // Echo of the requested path
+ content: string; // File content from backend.read()
+ line_count: number; // len(content.splitlines())
+ file_path: string; // Normalised path
}
```
+The plugin returns a `FileReadOutput(content, line_count, file_path)`.
+`node_output_schemas.FileReadOutput` declares `content` / `file_path` /
+`encoding` (extra fields like `line_count` allowed by `_OutputBase`).
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_file_read] --> B{file_path empty?}
- B -- yes --> E[Return error:<br/>file_path is required]
- B -- no --> C[_get_backend:<br/>root = param.working_directory OR<br/>context.workspace_dir OR<br/>Settings().workspace_base_resolved/default]
- C --> D[os.makedirs root, exist_ok=True]
- D --> F[LocalShellBackend root_dir=root, virtual_mode=True]
- F --> G[to_thread backend.read file_path, offset, limit]
- G -- exception --> H[Return error envelope str e]
- G -- ok --> I[Return success envelope<br/>content, file_path]
+ A[read] --> B{file_path empty?}
+ B -- yes --> E[raise NodeUserError:<br/>file_path is required]
+ B -- no --> C[get_backend params, ctx.raw:<br/>root = ctx.workspace_dir OR<br/>Settings().workspace_base_resolved/default]
+ C --> D[NushellBackend root_dir=root,<br/>virtual_mode=True, inherit_env=True]
+ D --> N[file_path = normalize_virtual_path]
+ N --> G[to_thread backend.read file_path, offset, limit]
+ G -- FileNotFound/IsADir/ValueError --> H[raise NodeUserError str e]
+ G -- result.error --> H2[raise NodeUserError result.error]
+ G -- ok --> I[Return FileReadOutput<br/>content, line_count, file_path]
```
## Decision Logic
-- **Validation**: empty `file_path` -> `"file_path is required"` error.
-- **Workspace resolution precedence**: node param `working_directory` >
- `context["workspace_dir"]` > `Settings().workspace_base_resolved/default`.
-- **Directory creation**: the resolved root is `os.makedirs(... exist_ok=True)`
- BEFORE the backend is built, so read against a fresh workflow works.
-- **`virtual_mode=True`**: the backend rejects absolute paths and `..`
- traversal that escape the root. Violations raise an exception which is
- caught by the broad `except` and returned as a string.
-- **Broad `except Exception`**: missing files, permission errors, decoding
- errors all collapse to the same shape - users must parse the `error` string
- to distinguish them.
+- **Validation**: empty `file_path` -> `raise NodeUserError("file_path is required")`.
+- **Workspace resolution precedence** (in `get_backend`): `working_directory`
+ param (not exposed here, `extra="ignore"`) > `ctx.workspace_dir` >
+ `Settings().workspace_base_resolved/default`.
+- **Directory creation**: `get_backend` ensures the resolved root exists before
+ building the backend, so a read against a fresh workflow works.
+- **Path normalization**: `normalize_virtual_path()` strips drive/root/UNC
+ anchors; `virtual_mode=True` then rejects `..`/`~` traversal that escapes the
+ root.
+- **Error paths**: `FileNotFoundError` / `IsADirectoryError` / `ValueError`
+ (bad offset, path escape) are caught and re-raised as `NodeUserError`; a
+ non-empty `result.error` from the backend also raises `NodeUserError`. Both
+ produce a single WARN line + structured envelope from `BaseNode.execute()`.
## Side Effects
@@ -86,7 +101,7 @@ flowchart TD
- **Broadcasts**: none.
- **External API calls**: none.
- **File I/O**:
- - `os.makedirs(root, exist_ok=True)` on the resolved workspace root.
+ - `get_backend` ensures the resolved workspace root exists.
- Reads `<root>/<file_path>` via the backend.
- **Subprocess**: none.
@@ -94,25 +109,24 @@ flowchart TD
- **Credentials**: none.
- **Services**: none.
-- **Python packages**: `deepagents` (imported lazily inside `_get_backend`),
+- **Python packages**: `deepagents` (via `NushellBackend` in `_backend.py`),
`asyncio`.
- **Environment variables**: `WORKSPACE_BASE_DIR` (read by `core.config.Settings`).
## Edge cases & known limits
-- **Errors are strings, not structured**: "file not found", "permission
- denied", "path escapes workspace" all return `{success: false, error: <str>}`
- with no error code.
-- **Binary files**: `LocalShellBackend.read()` decodes as text; a binary file
- raises a `UnicodeDecodeError` that surfaces as a vague error string.
-- **`offset`/`limit` typed as numbers**: negative offsets are NOT validated by
- the handler - they are forwarded to the backend which may return an empty
- string or raise.
-- **`working_directory` can point outside the workspace root**: the handler
- trusts it, so setting `working_directory="/"` effectively disables the
- `virtual_mode` sandbox. Dangerous when the node is exposed as an AI tool.
-- **No line-count cap**: `limit` default is 100 but the UI allows up to
- 10000; very large files with `limit=10000` block the thread longer.
+- **Errors are `NodeUserError`**: "file not found", "is a directory", "path
+ escapes workspace", "bad offset" all become `error_type="NodeUserError"`
+ envelopes with a string message (no distinct error code per case).
+- **Binary files**: `backend.read()` decodes as text; a binary file raises a
+ `UnicodeDecodeError` (uncaught here, so it surfaces with a full traceback as a
+ generic error envelope, not `NodeUserError`).
+- **`offset`/`limit` Pydantic-validated**: `offset>=0`, `limit` clamped 1-10000.
+ Negative inputs are rejected at validation, not forwarded.
+- **`working_directory` not exposed**: `FileReadParams` uses `extra="ignore"`,
+ so the sandbox cannot be widened via a node param on this node.
+- **`limit` default 2000**: very large files with `limit=10000` block the
+ thread longer.
## Related
diff --git a/docs-internal/node-logic-flows/code_fs_process/fsSearch.md b/docs-internal/node-logic-flows/code_fs_process/fsSearch.md
index 25565156..a477007e 100644
--- a/docs-internal/node-logic-flows/code_fs_process/fsSearch.md
+++ b/docs-internal/node-logic-flows/code_fs_process/fsSearch.md
@@ -3,8 +3,8 @@
| Field | Value |
|------|-------|
| **Category** | code_fs_process / filesystem |
-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_fs_search`](../../../server/services/handlers/filesystem.py) |
-| **Backend** | [`deepagents.backends.LocalShellBackend`](https://github.com/langchain-ai/deepagents) (`ls_info`, `glob_info`, `grep_raw`) |
+| **Backend handler** | [`server/nodes/filesystem/fs_search/__init__.py::FsSearchNode.search`](../../../server/nodes/filesystem/fs_search/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("search")`) |
+| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`; `ls_info`, `glob_info`, `grep_raw`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/fs-search-skill/SKILL.md`](../../../server/skills/coding_agent/fs-search-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `fs_search` |
@@ -12,8 +12,9 @@
## Purpose
Three-mode filesystem query node confined to the per-workflow workspace.
-Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on
-`LocalShellBackend` (`virtual_mode=True`) depending on `mode`.
+Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on `NushellBackend`
+(`virtual_mode=True`, `inherit_env=True`) via `get_backend()` depending on
+`mode`. The `path` is run through `normalize_virtual_path()` first.
## Inputs (handles)
@@ -25,18 +26,20 @@ Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `mode` | options | `ls` | no | - | `ls`, `glob`, or `grep` |
+| `mode` | `ls` \| `glob` \| `grep` (Literal) | `ls` | no | - | `ls`, `glob`, or `grep` |
| `path` | string | `.` | no | - | Directory path to search in (workspace-relative) |
-| `pattern` | string | `""` | yes (when `mode != ls`) | `mode=glob`, `mode=grep` | Glob pattern or grep regex |
-| `file_filter` | string | `""` | no | `mode=grep` | Glob pattern restricting which files are searched |
-| `working_directory` | string | `""` | no | - | Overrides context workspace |
+| `pattern` | string | `""` | yes (when `mode != ls`) | - | Glob pattern or grep regex |
+
+`FsSearchParams` uses `extra="ignore"` — there are NO `file_filter` or
+`working_directory` params; the model drops unknown keys. The grep mode passes
+only `pattern` + `path` to `backend.grep_raw()`. (displayOptions are not
+declared on these fields in the current Params model.)
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `fs_search` tool result) |
### Output payload
@@ -51,43 +54,43 @@ Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on
`FileInfo` and `GrepMatch` are the dataclass shapes returned by the
`deepagents` backend, converted via `dict(entry)`.
+`node_output_schemas.FsSearchOutput` declares `path` / `entries` / `matches` /
+`pattern` / `count` (extra fields allowed by `_OutputBase`).
## Logic Flow
```mermaid
flowchart TD
- A[handle_fs_search] --> C[_get_backend virtual_mode=True]
+ A[search] --> C[get_backend + normalize_virtual_path]
C --> D{mode?}
D -- ls --> L[to_thread backend.ls_info path]
- L --> Lok[Return success entries, count]
+ L --> Lok[Return dict path, entries, count]
D -- glob --> G{pattern empty?}
- G -- yes --> Gp[Return error:<br/>pattern is required for glob mode]
+ G -- yes --> Gp[raise NodeUserError:<br/>pattern is required for glob mode]
G -- no --> Gr[to_thread backend.glob_info pattern, path]
- Gr --> Grok[Return success matches, count]
+ Gr --> Grok[Return dict path, pattern, matches, count]
D -- grep --> Gg{pattern empty?}
- Gg -- yes --> Ggp[Return error:<br/>pattern is required for grep mode]
- Gg -- no --> Gx[to_thread backend.grep_raw pattern, path, glob=file_filter]
+ Gg -- yes --> Ggp[raise NodeUserError:<br/>pattern is required for grep mode]
+ Gg -- no --> Gx[to_thread backend.grep_raw pattern, path]
Gx --> Gxt{result type?}
- Gxt -- str error --> Gxerr[Return error<br/>error=result]
- Gxt -- list --> Gxok[Return success matches, count]
- D -- other --> U[Return error:<br/>Unknown mode: <mode>]
+ Gxt -- str error --> Gxerr[raise NodeUserError result]
+ Gxt -- list --> Gxok[Return dict path, pattern, matches, count]
+ D -- other --> U[raise NodeUserError:<br/>Unknown mode: <mode>]
```
## Decision Logic
- **`mode=ls`** requires nothing; defaults `path='.'`.
-- **`mode=glob`** requires `pattern`. Missing pattern -> error envelope.
-- **`mode=grep`** requires `pattern`. Optional `file_filter` is forwarded as
- the backend's `glob` kwarg.
+- **`mode=glob`** requires `pattern`. Missing pattern -> `raise NodeUserError`.
+- **`mode=grep`** requires `pattern`. No file-filter kwarg is passed (only
+ `pattern` + `path`).
- **`grep_raw` polymorphic return**: the backend returns either a list of
`GrepMatch` objects (success) OR a plain string (error description). The
- handler checks `isinstance(result, str)` and turns a string into an error
- envelope verbatim. This is the only place error details flow through as a
- direct backend string rather than the generic `str(Exception)` path.
-- **`file_filter` empty-string handling**: `file_filter or None` -> empty
- string is coerced to `None` before forwarding.
-- **Unknown modes**: returns error envelope `"Unknown mode: <mode>"`.
-- **Broad `except Exception`**: catches backend-level surprises.
+ plugin checks `isinstance(result, str)` and `raise NodeUserError(result)`.
+- **Unknown modes**: `raise NodeUserError("Unknown mode: <mode>")` (Pydantic
+ Literal already constrains it).
+- All raises become single-WARN-line `error_type="NodeUserError"` envelopes via
+ `BaseNode.execute()`.
## Side Effects
@@ -95,29 +98,29 @@ flowchart TD
- **Broadcasts**: none.
- **External API calls**: none.
- **File I/O**:
- - `os.makedirs(root, exist_ok=True)` on the workspace root.
+ - `get_backend` ensures the workspace root exists.
- Reads directory listings and file contents under `<root>/<path>`.
- **Subprocess**: none (`grep_raw` is a Python regex walk in recent
`deepagents`; not a `grep(1)` exec).
## External Dependencies
-- **Python packages**: `deepagents`.
+- **Python packages**: `deepagents` (via `NushellBackend`).
- **Environment variables**: `WORKSPACE_BASE_DIR`.
## Edge cases & known limits
- **`grep_raw` string-error path is unique**: unlike ls/glob which raise,
- grep can "succeed" at the Python level but return a string. Frontends
- expecting a consistent error shape across modes may miss this.
+ grep can "succeed" at the Python level but return a string; the plugin
+ converts that to a `NodeUserError` so the final envelope shape is consistent.
- **Glob patterns match the backend's semantics**: `**` globs work, but
the backend's behaviour for symlinks, hidden files, and case sensitivity
is inherited from `pathlib`.
- **No maxResults cap**: a `**/*` glob over a huge workspace returns every
match; the handler does not paginate. The caller must truncate.
- **`path='.'` resolves against the workspace root, not the server CWD**.
-- **`working_directory` escape**: same caveat as other filesystem nodes -
- a caller can bypass the sandbox.
+- **`working_directory` not exposed**: `extra="ignore"` means the sandbox
+ cannot be widened via a node param on this node.
- **Dataclass `dict()` conversion**: relies on `FileInfo` / `GrepMatch`
being dataclasses (they implement `__iter__` via dataclass). A future
backend upgrade that returns `TypedDict`s could break this.
diff --git a/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.md b/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.md
index 55f5b37f..4478d163 100644
--- a/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.md
+++ b/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.md
@@ -3,8 +3,8 @@
| Field | Value |
|------|-------|
| **Category** | code_fs_process / code |
-| **Backend handler** | [`server/services/handlers/code.py::handle_javascript_executor`](../../../server/services/handlers/code.py) |
-| **Node.js client** | [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py) |
+| **Backend handler** | [`server/nodes/code/javascript_executor/__init__.py::JavaScriptExecutorNode.execute_op`](../../../server/nodes/code/javascript_executor/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`; base in [`_base.py`](../../../server/nodes/code/_base.py)) |
+| **Node.js client** | [`server/nodes/code/_nodejs.py::get_nodejs_client`](../../../server/nodes/code/_nodejs.py) (singleton over [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py)) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/javascript-skill/SKILL.md`](../../../server/skills/coding_agent/javascript-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `javascript_code` |
@@ -12,17 +12,15 @@
## Purpose
Runs user JavaScript through the persistent Node.js executor server (Express +
-tsx) that the Python backend spawns on startup. The handler does **not** spawn
-`node` per call - it POSTs the code to `http://localhost:3020/execute` via an
-async `aiohttp` client. The Node.js server evaluates the script inside a VM
-sandbox and returns `{success, output, console_output, execution_time_ms}`.
+tsx) that the Python backend spawns on startup. The plugin does **not** spawn
+`node` per call - it POSTs the code to `http://localhost:3020/execute` via the
+shared `get_nodejs_client()` singleton (an async `aiohttp` client). The Node.js
+server evaluates the script and returns `{success, output, console_output, ...}`.
-The handler merges the caller's upstream outputs into an `input_data` object,
-injects the workflow's `workspace_dir` as a key, then forwards the payload.
-
-Dispatched through the special-handlers branch at
-[`node_executor.py:367-381`](../../../server/services/node_executor.py) so it
-receives `connected_outputs`.
+The plugin merges the caller's upstream outputs (`ctx.raw["connected_outputs"]`)
+into an `input_data` object, injects the workflow's `workspace_dir`
+(`ctx.workspace_dir`) as a key, then forwards the payload with `language="javascript"`.
+`connected_outputs` is injected by the executor for code-executor node types.
## Inputs (handles)
@@ -34,22 +32,23 @@ receives `connected_outputs`.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `code` | string (code editor) | (boilerplate stub) | yes | - | JavaScript source. User must assign to `output` |
-| `timeout` | number | `30` | no | - | Seconds - multiplied by 1000 and forwarded as millisecond timeout to the Node server |
+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | JavaScript source. User must assign to `output` |
+| `timeout` | number | `30` (ge=1, le=600) | no | - | Seconds - multiplied by 1000 and forwarded as millisecond timeout to the Node server |
+
+`CodeExecutorParams` uses `extra="allow"` (extra params persist but are unread).
-## Handler-level kwargs (bound via partial at registry setup)
+## Node.js client singleton
-| Name | Default | Description |
-|------|---------|-------------|
-| `nodejs_url` | `http://localhost:3020` | Overridable via `NODEJS_EXECUTOR_URL` at container wiring |
-| `nodejs_timeout` | `30` | Seconds; overridable via `NODEJS_EXECUTOR_TIMEOUT` |
+`get_nodejs_client()` in [`_nodejs.py`](../../../server/nodes/code/_nodejs.py) lazily
+constructs one `NodeJSClient(base_url="http://localhost:3020", timeout=30)` shared
+across the JS + TS plugins. The defaults are hard-coded in the helper signature
+(no per-call kwargs from the plugin).
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `javascript_code` tool result) |
### Output payload
@@ -57,44 +56,42 @@ receives `connected_outputs`.
{
output: any; // Value the Node.js server parsed from `output` in the script
console_output: string; // Captured console.log/.error etc.
- timestamp: string; // ISO8601
}
```
+`node_output_schemas.CodeExecutorOutput` declares only `output` (the
+`_OutputBase` base allows extra fields like `console_output`).
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_javascript_executor] --> B{code strip empty?}
- B -- yes --> E[Return error envelope:<br/>No code provided]
- B -- no --> C[timeout_ms = int timeout * 1000]
- C --> D[input_data = connected_outputs or {}<br/>inject workspace_dir]
+ A[execute_op] --> B{code strip empty?}
+ B -- yes --> E[raise NodeUserError:<br/>No code provided]
+ B -- no --> C[timeout_ms = timeout * 1000]
+ C --> D[input_data = ctx.raw connected_outputs or {}<br/>inject workspace_dir]
D --> F[get_nodejs_client<br/>lazy singleton]
F --> G[client.execute<br/>POST /execute<br/>language=javascript]
- G -- exception --> H[Return error envelope<br/>error=str e]
- G -- ok + success=true --> I[Return success envelope<br/>output + console_output]
- G -- ok + success=false --> J[Return error envelope<br/>preserve console_output]
+ G -- ClientConnectorError --> H1[raise NodeUserError<br/>JS executor not running on :3020]
+ G -- success=false --> H2[raise NodeUserError<br/>error=result.error]
+ G -- success=true --> I[Return dict<br/>output, console_output]
```
## Decision Logic
-- **Validation**: `code.strip() == ""` -> immediate error envelope
- `"No code provided"`.
-- **Timeout unit mismatch**: UI accepts seconds, handler multiplies by 1000
- before forwarding. Default 30 -> 30000 ms. No upper cap enforced in Python.
+- **Validation**: `code.strip() == ""` -> `raise NodeUserError("No code provided")`.
+- **Timeout unit mismatch**: UI accepts seconds, plugin multiplies by 1000
+ before forwarding. Default 30 -> 30000 ms. Pydantic clamps the input to 1-600 s.
- **`workspace_dir` injection**: unconditionally sets
`input_data["workspace_dir"]`, shadowing any upstream node that happened to
produce a key with that name.
-- **Client reuse**: `_nodejs_client` is a **module-global** singleton. First
- call fixes the `base_url` and `timeout`; subsequent calls with different
- kwargs silently reuse the first one. See Edge cases.
-- **Node server error propagation**: if the Node.js server returns
- `{success: false, error, console_output}`, the handler forwards the `error`
- string and **keeps** `console_output` so users still see logs from the failed
- run.
-- **Broad `except Exception`**: any networking error (aiohttp.ClientError,
- TimeoutError, ConnectionRefusedError) is caught, logged at ERROR, and
- surfaced as an error envelope with `console_output=""`.
+- **Client reuse**: `get_nodejs_client()`'s `_client` is a **module-global**
+ singleton in `_nodejs.py`, fixed at the hard-coded `base_url`/`timeout`.
+- **Node server error propagation**: if `result["success"]` is falsey, the
+ plugin raises `NodeUserError(result["error"] or "JavaScript executor failed")`.
+- **Sidecar down**: `aiohttp.ClientConnectorError` is caught and re-raised as a
+ `NodeUserError` telling the LLM the Node executor is unreachable on
+ localhost:3020 and to fall back to `python_executor`.
## Side Effects
@@ -107,8 +104,8 @@ flowchart TD
packages at `server/nodejs/user-packages/`.
- **Subprocess**: none directly. The Node.js server itself is a long-lived
subprocess started by `main.py` lifespan.
-- **Module-level state**: the `_nodejs_client` module global is created on
- first use and never reset.
+- **Module-level state**: the `_client` module global in `_nodejs.py` is created
+ on first use and never reset.
## External Dependencies
@@ -122,10 +119,9 @@ flowchart TD
## Edge cases & known limits
-- **Module-level client singleton**: `_nodejs_client` is cached on first call.
- If the injected `nodejs_url` / `nodejs_timeout` kwargs change mid-process
- (e.g., tests re-wiring the container), the cached instance keeps the
- original values. Reset by `services.handlers.code._nodejs_client = None`.
+- **Module-level client singleton**: `_client` in `_nodejs.py` is cached on
+ first call at the hard-coded `base_url`/`timeout`. Reset by setting
+ `services.code._nodejs._client = None` (or `nodes.code._nodejs._client`).
- **Node server down**: connection refused surfaces as
`error="Cannot connect to host localhost:3020 ssl:default [Connect call
failed]"` or similar aiohttp message. No automatic retry.
@@ -146,5 +142,5 @@ flowchart TD
## Related
- **Skills using this as a tool**: [`javascript-skill/SKILL.md`](../../../server/skills/coding_agent/javascript-skill/SKILL.md)
-- **Sibling nodes**: [`typescriptExecutor`](./typescriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md)
-- **Architecture docs**: [DESIGN.md](../../DESIGN.md)
+- **Sibling nodes**: [`typescriptExecutor`](./typescriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md), [`montyExecutor`](./montyExecutor.md)
+- **Architecture docs**: [DESIGN.md](../../DESIGN.md), [Plugin System](../../plugin_system.md)
diff --git a/docs-internal/node-logic-flows/code_fs_process/montyExecutor.md b/docs-internal/node-logic-flows/code_fs_process/montyExecutor.md
new file mode 100644
index 00000000..6c051b32
--- /dev/null
+++ b/docs-internal/node-logic-flows/code_fs_process/montyExecutor.md
@@ -0,0 +1,148 @@
+# Monty Executor (`montyExecutor`)
+
+| Field | Value |
+|------|-------|
+| **Category** | code_fs_process / code |
+| **Backend handler** | [`server/nodes/code/monty_executor/__init__.py::MontyExecutorNode.execute_op`](../../../server/nodes/code/monty_executor/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`; base in [`_base.py`](../../../server/nodes/code/_base.py)) |
+| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
+| **Skill (if any)** | [`server/skills/coding_agent/monty-skill/SKILL.md`](../../../server/skills/coding_agent/monty-skill/SKILL.md) |
+| **Dual-purpose tool** | yes - tool name `sandboxed_python` |
+
+## Purpose
+
+A deny-by-default, hard-sandboxed alternative to [`pythonExecutor`](./pythonExecutor.md).
+Runs AI-generated/untrusted Python through Pydantic's
+[Monty](https://github.com/pydantic/monty) interpreter (`pydantic-monty==0.0.18`,
+a Python subset implemented in Rust) with **ENFORCED** wall-clock + memory limits
+and zero host access unless explicitly granted. The headline win over
+`pythonExecutor` is that `timeout` and `max_memory_mb` are actually enforced.
+
+Capabilities are dynamic: the caller (an LLM, when wired as the
+`sandboxed_python` tool) selects from a fixed menu via the `capabilities` param.
+Each selection wires a real Monty grant — `http_get` (SSRF-guarded host fetch
+via an `external_functions` entry) and/or `workspace_read` / `workspace_write`
+(a `MountDir` over the per-workflow workspace at `/workspace`). The default
+(empty list) is pure deny-by-default. The program's last expression becomes
+`output`; captured `print()` becomes `console_output`.
+
+`pydantic_monty` is lazy-imported inside the op so the server still boots and
+registers every other node if no wheel exists for the platform.
+
+## Inputs (handles)
+
+| Handle | Connection type | Required | Purpose |
+|--------|-----------------|----------|---------|
+| `input-main` | main | no | Upstream outputs exposed as the `input_data` dict inside the sandbox (from `ctx.raw["connected_outputs"]`, injected by the executor for code-executor node types) |
+
+## Parameters
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | Python (Monty subset) source. Last expression is returned as `output` |
+| `timeout` | number | `30` (ge=1, le=600) | no | - | Wall-clock limit in seconds, ENFORCED via `ResourceLimits.max_duration_secs` |
+| `max_memory_mb` | number | `256` (ge=16, le=2048) | no | - | Memory limit in MB, ENFORCED via `ResourceLimits.max_memory` (converted to bytes) |
+| `capabilities` | string[] (enum: `http_get`, `workspace_read`, `workspace_write`) | `[]` | no | - | Opt-in grants; empty = deny-by-default |
+
+`MontyExecutorParams` uses `extra="allow"` (extra params persist but are unread).
+
+## Outputs (handles)
+
+| Handle | Shape | Description |
+|--------|-------|-------------|
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `sandboxed_python` tool result) |
+
+### Output payload
+
+```ts
+{
+ output: any; // The program's last expression value
+ console_output: string; // Captured print() output (Monty CollectString)
+}
+```
+
+`montyExecutor` is mapped to `node_output_schemas.CodeExecutorOutput` (declares
+only `output`; the `_OutputBase` base allows extra fields like `console_output`).
+
+## Logic Flow
+
+```mermaid
+flowchart TD
+ A[execute_op] --> B{code strip empty?}
+ B -- yes --> E[raise NodeUserError:<br/>No code provided]
+ B -- no --> I{import pydantic_monty ok?}
+ I -- no --> Ie[raise NodeUserError:<br/>Monty unavailable, use python_code]
+ I -- yes --> C[input_data = ctx.raw connected_outputs or {}]
+ C --> Cap[_build_capabilities:<br/>http_get -> external_functions,<br/>workspace_* -> MountDir /workspace]
+ Cap -- unknown cap --> Ee[raise NodeUserError:<br/>Unknown capability]
+ Cap -- workspace cap & no workspace_dir --> Ew[raise NodeUserError]
+ Cap -- ok --> L[ResourceLimits max_duration_secs, max_memory]
+ L --> R[Monty code parse + to_thread m.run<br/>inputs, limits, external_functions,<br/>print_callback, mount]
+ R -- MontyError --> Cl[classify by message ->raise NodeUserError]
+ R -- ok --> O[output = result.output<br/>console_output = collector.output]
+```
+
+## Decision Logic
+
+- **Validation**: `code.strip() == ""` -> `raise NodeUserError("No code provided")`.
+- **Missing wheel**: `ImportError` on `import pydantic_monty` -> `NodeUserError`
+ telling the caller Monty is unavailable and to use `python_code` instead.
+- **Capability resolution** (`_build_capabilities`): unknown capability ->
+ `NodeUserError`. `http_get` wires an SSRF-guarded `http_get(url)` host
+ function. `workspace_read` / `workspace_write` require `ctx.workspace_dir`
+ (else `NodeUserError`) and mount it at `/workspace` (`read-only` /
+ `read-write`).
+- **Enforced limits**: `timeout` -> `max_duration_secs`, `max_memory_mb * 1024
+ * 1024` -> `max_memory`. Both are enforced by the Rust interpreter.
+- **Error classification**: all Monty failures subclass `MontyError`; the op
+ classifies by message into: unsupported-feature (`does not yet support` /
+ `notimplementederror`), time-limit, memory-limit, `MontySyntaxError`, and
+ plain runtime errors (incl. exceptions raised by a capability host function
+ like a blocked `http_get`). Each becomes a `NodeUserError` with an actionable
+ hint. `class` surfaces a `MontyRuntimeError` at construct time and is caught
+ by the same handler.
+- **Result unwrap**: `result.output` if the result is a `MontyComplete`, else
+ the raw result.
+
+## Side Effects
+
+- **Database writes**: none.
+- **Broadcasts**: none from this op.
+- **External API calls**: only when `http_get` is granted — an outbound
+ `httpx.get(url)` (follow-redirects, 15 s timeout) to a publicly-routable host.
+- **File I/O**: only when `workspace_read` / `workspace_write` is granted — the
+ sandbox sees `/workspace` mapped to `ctx.workspace_dir` (read-only or
+ read-write).
+- **Subprocess**: none. Monty runs in-process; `m.run` is offloaded via
+ `asyncio.to_thread` so the CPU-bound Rust execution doesn't block the loop.
+
+## External Dependencies
+
+- **Credentials**: none.
+- **Services**: none.
+- **Python packages**: `pydantic-monty==0.0.18` (lazy-imported), `httpx` (only
+ for the `http_get` capability), `ipaddress` / `socket` (SSRF guard).
+- **Environment variables**: none.
+
+## Edge cases & known limits
+
+- **Python SUBSET only**: supports def/closures/lambda, if/for/while,
+ try/except, comprehensions, f-strings, async def/await, and `import math|json|re`.
+ NOT supported: `class`, yield/generators, `with`, `match`, `import
+ random|collections|os`. Unsupported features raise `NodeUserError` pointing the
+ caller to `python_code`.
+- **Enforced limits** (unlike `pythonExecutor`): time + memory breaches are
+ caught and surfaced as `NodeUserError`.
+- **SSRF guard is best-effort**: `_is_public_url` re-resolves DNS and rejects
+ loopback / private / link-local / reserved / multicast / unspecified
+ addresses, but httpx re-resolves at request time so it is not TOCTOU-proof.
+- **Workspace capability requires a workspace**: requesting `workspace_read` /
+ `workspace_write` without a `ctx.workspace_dir` raises `NodeUserError`.
+- **Missing wheel degrades gracefully**: a platform without a `pydantic-monty`
+ wheel returns a `NodeUserError`, not a server crash — every other node still
+ registers.
+
+## Related
+
+- **Skills using this as a tool**: [`monty-skill/SKILL.md`](../../../server/skills/coding_agent/monty-skill/SKILL.md)
+- **Sibling nodes**: [`pythonExecutor`](./pythonExecutor.md), [`javascriptExecutor`](./javascriptExecutor.md), [`typescriptExecutor`](./typescriptExecutor.md), [`shell`](./shell.md), [`processManager`](./processManager.md)
+- **Architecture docs**: [DESIGN.md](../../DESIGN.md), [Plugin System](../../plugin_system.md)
diff --git a/docs-internal/node-logic-flows/code_fs_process/processManager.md b/docs-internal/node-logic-flows/code_fs_process/processManager.md
index 1c6b65c3..cff52fef 100644
--- a/docs-internal/node-logic-flows/code_fs_process/processManager.md
+++ b/docs-internal/node-logic-flows/code_fs_process/processManager.md
@@ -2,8 +2,8 @@
| Field | Value |
|------|-------|
-| **Category** | code_fs_process / process |
-| **Backend handler** | [`server/services/handlers/process.py::handle_process_manager`](../../../server/services/handlers/process.py) |
+| **Category** | code_fs_process / process (plugin lives under `server/nodes/utility/`) |
+| **Backend handler** | [`server/nodes/utility/process_manager/__init__.py::ProcessManagerNode.dispatch`](../../../server/nodes/utility/process_manager/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("dispatch")`) |
| **Service** | [`server/services/process_service.py::ProcessService`](../../../server/services/process_service.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/terminal/process-manager-skill/SKILL.md`](../../../server/skills/terminal/process-manager-skill/SKILL.md) |
@@ -17,10 +17,12 @@ workflow. Unlike [`shell`](./shell.md), processes spawned here inherit the
the Terminal tab via `broadcast_terminal_log()` and persists to per-process log
files at `<workspace>/<agent_node_id>/.processes/<name>/{stdout,stderr}.log`.
-Six operations: `start`, `stop`, `restart`, `send_input`, `list`, `get_output`.
-The handler is a thin dispatcher over the `ProcessService` singleton; the
-singleton owns a `{(workflow_id, name): ManagedProcess}` dict, the streaming
-tasks, and the cleanup scheduler.
+Six operations: `start`, `stop`, `restart`, `send_input`, `list`, `get_output`
+(default `list`). The plugin's `dispatch` op is a thin wrapper over the
+`ProcessService` singleton; the singleton owns a `{(workflow_id, name):
+ManagedProcess}` dict, the streaming tasks, and the cleanup scheduler. The
+wrapper's `_unwrap` raises `NodeUserError` on any service `{"success": False}`
+envelope so the framework returns a clean failure.
## Inputs (handles)
@@ -32,38 +34,45 @@ tasks, and the cleanup scheduler.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `process_manager` | no | - | Tool name exposed to AI agents |
-| `toolDescription` | string | (see frontend) | no | - | Description shown to LLM |
-| `operation` | options | `start` | no | - | One of `start`, `stop`, `restart`, `send_input`, `list`, `get_output` |
-| `name` | string | `""` | yes (all except `list`) | `operation in [start, stop, restart, send_input, get_output]` | Unique process name within the workflow |
-| `command` | string | `""` | yes (`start` only) | `operation=start` | Shell command (parsed with `shlex.split`) |
-| `working_directory` | string | `""` | no | `operation=start` | Defaults to `<workspace>/<node_id>` |
-| `text` | string | `""` | yes (`send_input`) | `operation=send_input` | Text to write to stdin (newline auto-appended) |
-| `stream` | string | `"stdout"` | no | - (tool-only) | `stdout` or `stderr` for `get_output` |
-| `tail` | number | `50` | no | - (tool-only) | Tail lines; 0 means all lines from `offset` |
-| `offset` | number | `0` | no | - (tool-only) | Skip N lines (only when `tail=0`) |
+| `operation` | `start`/`stop`/`restart`/`list`/`send_input`/`get_output` (Literal) | `list` | no | - | Operation to run |
+| `name` | string | `""` | yes (all except `list`) | - | Unique process name within the workflow (LLM `"None"` coerced to empty by `_clean`) |
+| `command` | string | `""` | yes (`start` only) | `operation=start` | Command (`ProcessService` parses with `shlex.split`) |
+| `cwd` | string | `""` | no | `operation=start` | Working directory; defaults to `<workspace>/<node_id>` |
+| `env` | object (Dict[str,str]) | `{}` | no | `operation=start` | Declared param, but the `dispatch` op does NOT forward it to `svc.start` — currently inert |
+| `input_text` | string | `""` | yes (`send_input`) | `operation=send_input` | Text to write to stdin (newline auto-appended by the service) |
+| `stream` | `stdout`/`stderr` (Literal) | `stdout` | no | `operation=get_output` | Stream to read for `get_output` |
+| `tail` | number | `100` (ge=1, le=10000) | no | `operation=get_output` | Tail N lines |
+
+`ProcessManagerParams` uses `extra="ignore"`. There are NO `toolName` /
+`toolDescription` / `working_directory` / `text` / `offset` params (the old
+card listed pre-Wave-11 frontend fields). `get_output` is always called with
+`offset=0`.
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `process_manager` tool result) |
### Output payload per operation
-- `start` / `stop` / `restart`: `{name, command, pid, status, started_at,
- exit_code, working_directory, stdout_lines, stderr_lines, log_dir}`.
-- `send_input`: `{sent: <text>}`.
+The shapes below are whatever `ProcessService` returns (passed through `_unwrap`).
+`node_output_schemas.ProcessManagerOutput` declares `operation` / `pid` /
+`status` / `output` / `processes` (extra fields allowed by `_OutputBase`).
+
+- `start` / `stop` / `restart`: the `ProcessInfo` dict from the service
+ (`{name, command, pid, status, started_at, exit_code, working_directory,
+ log_dir, ...}`).
+- `send_input`: the service's send-input result dict.
- `list`: `{processes: Array<ProcessInfo>}` for the current workflow.
-- `get_output`: `{lines: string[], total: number, file: string}`.
+- `get_output`: the service's `{lines, total, file, ...}` dict.
## Logic Flow
```mermaid
flowchart TD
- A[handle_process_manager] --> B[Build tool_args:<br/>clean 'None' strings,<br/>default working_directory to workspace/node_id]
- B --> C[execute_process_manager]
+ A[dispatch] --> B[name = _clean params.name<br/>agent_dir = workspace/node_id]
+ B --> C[route on params.operation]
C --> D{operation}
D -- start --> S1{command blocked?<br/>rm, format, mkfs...}
S1 -- yes --> Sblk[Return error:<br/>Destructive commands blocked]
@@ -83,18 +92,22 @@ flowchart TD
St2 --> St3[wait exit, schedule 60s cleanup]
St3 --> St4[Return success]
D -- restart --> R[stop then start with same cmd+cwd]
- D -- send_input --> I1[write text+newline to stdin.drain]
- D -- list --> L[return info for workflow_id processes]
- D -- get_output --> O[read log file, tail or offset slice]
- D -- unknown --> U[Return error:<br/>Unknown operation]
+ D -- send_input --> I1[svc.send_input name, wf, input_text]
+ D -- list --> L[svc.list_processes workflow_id]
+ D -- get_output --> O[svc.get_output name, wf, stream, tail, 0]
+ D -- unknown --> U[raise NodeUserError:<br/>Unknown operation]
```
+Each service call (except `list`) is wrapped in `_unwrap`, which raises
+`NodeUserError` if the service returned `{"success": False}`.
+
## Decision Logic
-- **`_clean_arg()`**: LLMs sometimes pass the literal string `"None"` for
- missing fields; the handler coerces `""` and `"None"` to empty.
-- **`working_directory` fallback**: if the param is empty the handler uses
- `<workspace>/<node_id>` (each agent node gets its own subfolder). If the
+- **`_clean()`**: LLMs sometimes pass the literal string `"None"` for missing
+ fields; the wrapper coerces `""` and `"None"` to empty (applied to `name`,
+ `command`, `cwd`, `input_text`).
+- **`cwd` fallback**: if `cwd` is empty the wrapper uses `agent_dir =
+ <workspace>/<node_id>` (each agent node gets its own subfolder). If the
workspace is also empty, `ProcessService.start` falls back to
`<workspace_base>/default` and mkdir's it.
- **Workspace guardrail**: `cwd` must resolve inside `workspace_base_resolved`
diff --git a/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.md b/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.md
index 65288e66..3269247e 100644
--- a/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.md
+++ b/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | code_fs_process / code |
-| **Backend handler** | [`server/services/handlers/code.py::handle_python_executor`](../../../server/services/handlers/code.py) |
+| **Backend handler** | [`server/nodes/code/python_executor/__init__.py::PythonExecutorNode.execute_op`](../../../server/nodes/code/python_executor/__init__.py) (dispatched via `BaseNode.execute()` + the `@Operation("execute")` method; base in [`_base.py`](../../../server/nodes/code/_base.py)) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/python-skill/SKILL.md`](../../../server/skills/coding_agent/python-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `python_code` |
@@ -19,9 +19,10 @@ are intentionally excluded - only `math`, `json`, `datetime`, `re`, `random`,
redirected to an `io.StringIO` buffer so stdout becomes the `console_output`
return field.
-Dispatched through the special-handlers branch at
-[`node_executor.py:367-381`](../../../server/services/node_executor.py) so it
-receives `connected_outputs` (dict of upstream outputs keyed by source node id).
+The plugin reads upstream outputs from `ctx.raw["connected_outputs"]` (dict of
+upstream outputs keyed by source node id), which the executor injects for
+code-executor node types, and `ctx.workspace_dir` for the per-workflow scratch
+directory.
## Inputs (handles)
@@ -33,15 +34,17 @@ receives `connected_outputs` (dict of upstream outputs keyed by source node id).
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `code` | string (code editor) | (boilerplate stub) | yes | - | Python source. Must assign to the free variable `output` to emit a value |
-| `timeout` | number | `30` | no | - | Read and coerced to int but **not enforced** - `exec()` is synchronous with no wall-clock guard |
+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | Python source. Must assign to the free variable `output` to emit a value |
+| `timeout` | number | `30` (ge=1, le=600) | no | - | Validated/clamped but **not enforced** - `exec()` is synchronous with no wall-clock guard. Use `montyExecutor` for an enforced limit |
+
+`CodeExecutorParams` uses `extra="allow"`, so any extra params persist but are
+not read by the operation.
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (the node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `python_code` tool result) |
### Output payload
@@ -49,48 +52,52 @@ receives `connected_outputs` (dict of upstream outputs keyed by source node id).
{
output: any; // Value assigned to `output` in the user code (None by default)
console_output: string; // Captured stdout from redirected print()
- timestamp: string; // ISO8601 local time
}
```
-On failure the envelope is `{success: false, error, console_output: "", ...}`.
-Note that `console_output` is **only** preserved on the success path; when an
-exception fires mid-execution the already-captured buffer is discarded because
-the `except` branch hard-codes `"console_output": console_output` but
-`console_output` is only assigned after `exec()` finishes. See Edge cases.
+On failure the operation raises `NodeUserError` (not an error envelope). The
+framework's `BaseNode.execute()` catches it and returns
+`{success: false, error_type: "NodeUserError", error, ...}` with a single WARN
+line (no traceback). The error message preserves any captured stdout via a
+`stdout before error:` suffix, so prints before the exception are NOT lost.
+`node_output_schemas.CodeExecutorOutput` declares only `output` (the
+`_OutputBase` base allows extra fields like `console_output`).
## Logic Flow
```mermaid
flowchart TD
- A[handle_python_executor] --> B{code strip empty?}
- B -- yes --> E[Return error envelope:<br/>No code provided]
- B -- no --> C[Read timeout<br/>int-coerce only, unused]
+ A[execute_op] --> B{code strip empty?}
+ B -- yes --> E[raise NodeUserError:<br/>No code provided]
+ B -- no --> C[timeout validated by Pydantic, unused at runtime]
C --> D[Build safe_builtins dict:<br/>abs..zip, math, json, captured_print]
- D --> F[Build namespace:<br/>input_data=connected_outputs or {},<br/>workspace_dir=context,<br/>output=None]
+ D --> F[Build namespace:<br/>input_data=ctx.raw connected_outputs or {},<br/>workspace_dir=ctx.workspace_dir,<br/>output=None]
F --> G[exec code in namespace]
- G -- exception --> H[Return error envelope<br/>error=str e, console_output=empty]
+ G -- ImportError __import__ --> H1[raise NodeUserError<br/>import not allowed, list sandbox names]
+ G -- other exception --> H2[raise NodeUserError<br/>ErrType at line N: msg + stdout before error]
G -- ok --> I[output = namespace.get output<br/>console_output = StringIO.getvalue]
- I --> J[Return success envelope<br/>output, console_output, timestamp]
+ I --> J[Return dict<br/>output, console_output]
```
## Decision Logic
-- **Validation**: `code.strip() == ""` -> immediate error envelope with message
- `"No code provided"`.
+- **Validation**: `code.strip() == ""` -> `raise NodeUserError("No code provided")`.
- **Namespace seeding**: `input_data` defaults to `{}` if `connected_outputs` is
- None. `workspace_dir` is pulled from `context["workspace_dir"]` but never
+ None. `workspace_dir` is pulled from `ctx.workspace_dir` but never
validated; an empty string is fine.
- **Print redirection**: `safe_builtins["print"]` is a wrapper that forces
`file=stdout_capture`, so any user-supplied `file=` kwarg is **overwritten**.
- **Output extraction**: `namespace.get("output", None)` - the user writing
`output = <value>` is the only way to emit data. A top-level expression is
evaluated but not captured.
-- **No timeout enforcement**: the `timeout` parameter is int-coerced only as a
- sanity check (raises on non-numeric strings) but never passed to a watchdog.
- Long loops will block the async event loop for the whole backend.
-- **Error path**: broad `except Exception` catches everything, logs via
- `logger.error`, and returns `{success: false, error: str(e)}`.
+- **No timeout enforcement**: the `timeout` parameter is Pydantic-validated
+ (1-600) but never passed to a watchdog. Long loops will block the async event
+ loop for the whole backend.
+- **Error path**: `except Exception` re-raises as `NodeUserError`. `import X`
+ (which hits the sandboxed builtins' missing `__import__`) gets a dedicated
+ message listing the pre-injected names; other exceptions are formatted as
+ `<ErrType> at line N: <msg>` (line walked from the `<string>` traceback frame)
+ with any captured stdout appended.
## Side Effects
@@ -121,10 +128,9 @@ flowchart TD
`math` and `json` are module objects, and user code can re-import anything via
`().__class__.__base__.__subclasses__()` or similar well-known escapes. Treat
this node as trusted-input only.
-- **`console_output` swallowed on error**: the `except` path hard-codes
- `"console_output": console_output`, but `console_output` is initialised to
- `""` and only overwritten **after** `exec()` succeeds. Any print()s before an
- exception are therefore lost.
+- **`console_output` preserved on error**: the `NodeUserError` message appends a
+ `stdout before error:` block with whatever the StringIO captured before the
+ exception, so prints before a failure are surfaced (not lost) in the error.
- **`output=None` is indistinguishable from a user explicitly returning
`None`**: both surface as `{"output": None}` in the envelope.
- **`input_data` mutation is visible to later siblings**: the handler passes
@@ -139,5 +145,5 @@ flowchart TD
## Related
- **Skills using this as a tool**: [`python-skill/SKILL.md`](../../../server/skills/coding_agent/python-skill/SKILL.md)
-- **Sibling nodes**: [`javascriptExecutor`](./javascriptExecutor.md), [`typescriptExecutor`](./typescriptExecutor.md), [`shell`](./shell.md), [`processManager`](./processManager.md)
-- **Architecture docs**: [DESIGN.md](../../DESIGN.md)
+- **Sibling nodes**: [`montyExecutor`](./montyExecutor.md), [`javascriptExecutor`](./javascriptExecutor.md), [`typescriptExecutor`](./typescriptExecutor.md), [`shell`](./shell.md), [`processManager`](./processManager.md)
+- **Architecture docs**: [DESIGN.md](../../DESIGN.md), [Plugin System](../../plugin_system.md)
diff --git a/docs-internal/node-logic-flows/code_fs_process/shell.md b/docs-internal/node-logic-flows/code_fs_process/shell.md
index 31f92ffc..5e92026d 100644
--- a/docs-internal/node-logic-flows/code_fs_process/shell.md
+++ b/docs-internal/node-logic-flows/code_fs_process/shell.md
@@ -3,20 +3,31 @@
| Field | Value |
|------|-------|
| **Category** | code_fs_process / filesystem |
-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_shell`](../../../server/services/handlers/filesystem.py) |
-| **Backend** | [`deepagents.backends.LocalShellBackend.execute`](https://github.com/langchain-ai/deepagents) |
+| **Backend handler** | [`server/nodes/filesystem/shell/__init__.py::ShellNode.execute_op`](../../../server/nodes/filesystem/shell/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`) |
+| **Backend** | `NushellBackend.execute` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/terminal/shell-skill/SKILL.md`](../../../server/skills/terminal/shell-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `shell_execute` |
## Purpose
-Runs a short-lived shell command inside the per-workflow workspace. Delegates
-to `LocalShellBackend.execute()`, which invokes the command via `subprocess`
-but intentionally **scrubs the system PATH**, so tools like `npm`, `node`,
-`python`, and `git` are NOT resolvable unless supplied with an absolute path.
-For long-running processes or tools that need the real PATH, users are
-directed to [`processManager`](./processManager.md).
+Runs a short-lived shell command inside the per-workflow workspace. The backend
+is **Nushell** (`NushellBackend`, a `LocalShellBackend` subclass) constructed
+with `inherit_env=True`, so external tools like `npm`, `node`, `python`, and
+`git` ARE reachable on PATH (it falls back to upstream `LocalShellBackend.execute()`
+/ POSIX `sh` / cmd.exe only when `nu` isn't installed). The command grammar is
+Nushell, not bash — `&&` / `||` / `$VAR` / backticks / `>` do not work; see
+`shell-skill/SKILL.md` for the Nu equivalents. For long-running processes
+(dev servers, watchers) users are directed to [`processManager`](./processManager.md);
+this node kills the command at `timeout`.
+
+NOTE: the node's `description` metadata still reads "sandboxed; no system PATH",
+which no longer matches the `inherit_env=True` backend — a code-side copy
+mismatch (out of scope for this card).
+
+A pre-flight regex (`_BASH_CHAIN_RE`) rejects bash chain operators (` && ` / ` || `)
+up-front with a corrective message before Nushell's parser would emit a cryptic
+`shell_andand` / `shell_oror` error.
Runs synchronously under `asyncio.to_thread()` so the event loop keeps
servicing other requests while the command executes.
@@ -31,16 +42,18 @@ servicing other requests while the command executes.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `command` | string | `""` | yes | - | Shell command |
-| `timeout` | number | `30` | no | - | Max seconds (UI clamps 1-300) |
-| `working_directory` | string | `""` | no | - | Overrides context workspace |
+| `command` | string | (required, `min_length=1`) | yes | - | Nushell command |
+| `cwd` | string | `""` | no | - | Declared param, but `get_backend` resolves the working dir from `working_directory` (not exposed) / `ctx.workspace_dir`, so `cwd` does NOT change the backend root |
+| `timeout` | number | `30` (ge=1, le=600) | no | - | Max seconds |
+
+`ShellParams` uses `extra="ignore"` — `working_directory` is NOT exposed; the
+backend root is `ctx.workspace_dir`.
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `shell_execute` tool result) |
### Output payload
@@ -53,29 +66,34 @@ servicing other requests while the command executes.
}
```
+`node_output_schemas.ShellOutput` declares `stdout` / `exit_code` / `truncated`
+/ `command`.
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_shell] --> B{command empty?}
- B -- yes --> E[Return error:<br/>command is required]
- B -- no --> C[_get_backend workspace root<br/>virtual_mode=True]
+ A[execute_op] --> P{_BASH_CHAIN_RE matches<br/> && / || ?}
+ P -- yes --> Pe[raise NodeUserError:<br/>use ; or try{}catch{}]
+ P -- no --> C[get_backend workspace root<br/>NushellBackend virtual_mode, inherit_env]
C --> D[to_thread backend.execute command, timeout]
- D -- exception --> H[Return error envelope]
- D -- ok --> I{exit_code?}
+ D --> S[strip_ansi result.output]
+ S --> I{exit_code?}
I -- 124 --> I1[Log warning:<br/>Timed out]
I -- nonzero --> I2[Log warning:<br/>Non-zero exit]
I -- 0 --> I3[Log info: Completed]
- I1 & I2 & I3 --> J[Return success envelope<br/>stdout, exit_code, truncated, command]
+ I1 & I2 & I3 --> J[Return dict<br/>stdout, exit_code, truncated, command]
```
## Decision Logic
-- **Validation**: empty `command` -> `"command is required"` error.
-- **Sandbox PATH**: `LocalShellBackend` strips the system PATH before spawn,
- so by default only built-ins and absolute paths work. This is a deliberate
- safety choice; AI agents are steered toward `process_manager` for any
- install/build tool invocation.
+- **Validation**: empty `command` is rejected by Pydantic `min_length=1` (no
+ manual check). A ` && ` / ` || ` in the command -> `raise NodeUserError` with
+ a Nushell-equivalent hint (`;` or `try { … } catch { … }`).
+- **Inherited PATH**: `NushellBackend(inherit_env=True)` — external tools
+ (`npm`, `node`, `python`, `git`, …) ARE reachable. The previous "scrubbed
+ PATH" framing no longer holds. AI agents are still steered toward
+ `process_manager` for long-running daemons (the shell kills at `timeout`).
- **Timeout handling**: `timeout` is forwarded to
`backend.execute(command, timeout=timeout)`. On timeout the backend kills
the process and sets `exit_code=124` (POSIX convention). Logged at WARNING.
@@ -90,8 +108,10 @@ flowchart TD
before the operator-log line), so colour/cursor codes from tools like
`vite`/`npm` don't render as garbage in the Output panel. `click.unstyle` is
byte-faithful apart from the stripped escapes.
-- **Broad `except Exception`**: backend failures (missing binary,
- `virtual_mode` violations) become an error envelope.
+- **No catch-all error wrapper**: the op does not wrap `backend.execute` in a
+ try/except — non-zero exit codes are returned as a normal success envelope
+ with `exit_code != 0`. An unexpected backend exception surfaces with a full
+ traceback via `BaseNode.execute()`'s generic path.
## Side Effects
@@ -99,23 +119,22 @@ flowchart TD
- **Broadcasts**: none (but the process stdout/stderr is NOT streamed to
Terminal - only `processManager` does that).
- **External API calls**: none.
-- **File I/O**: `os.makedirs(root, exist_ok=True)` on the workspace root. The
- command itself may read/write under that root.
-- **Subprocess**: one per call, via the backend's `subprocess.run`
- (synchronous, scrubbed PATH). Finished before the handler returns.
+- **File I/O**: `get_backend` ensures the workspace root exists. The command
+ itself may read/write under that root.
+- **Subprocess**: one per call, via the backend's subprocess (synchronous,
+ inherited PATH). Finished before the operation returns.
## External Dependencies
-- **Python packages**: `deepagents`.
+- **Python packages**: `deepagents` (via `NushellBackend`), `core.ansi`.
- **Environment variables**: `WORKSPACE_BASE_DIR`.
-- **OS utilities**: whatever the command references - but PATH is empty, so
- absolute paths only.
+- **OS utilities**: `nu` (Nushell) on PATH for the primary path; whatever the
+ command references is reachable because PATH is inherited.
## Edge cases & known limits
-- **Missing binaries masquerade as command failures**: because PATH is
- scrubbed, a plain `npm test` gives "command not found" rather than a
- permissions error. Users often confuse this for a broken node.
+- **Nushell grammar, not bash**: `&&` / `||` are pre-flight-rejected; `$VAR`,
+ backticks, and `>` redirection differ from bash. See `shell-skill/SKILL.md`.
- **Truncation corrupts JSON output**: if a command prints JSON and the
backend truncates, the `stdout` string becomes invalid JSON with no
trailing marker beyond `truncated=true`.
@@ -124,10 +143,10 @@ flowchart TD
SIGTERM-on-timeout.
- **`timeout` is best-effort**: the backend uses `subprocess.run(..., timeout=)`.
A child process that ignores SIGTERM may keep running until SIGKILL.
-- **Stdin is empty**: the handler exposes no way to pipe data into the
+- **Stdin is empty**: the node exposes no way to pipe data into the
command - `input_data` from upstream nodes does not reach it.
- **No environment variable override**: users cannot inject env vars via
- parameters; the backend sets its own minimal env.
+ parameters; the backend inherits the server's env (`inherit_env=True`).
- **Windows path traversal check is string-based**: `virtual_mode=True`
normalises paths but trust-boundary violations depend on the backend
implementation.
diff --git a/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.md b/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.md
index 511d83d9..79c6c5fb 100644
--- a/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.md
+++ b/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.md
@@ -3,8 +3,8 @@
| Field | Value |
|------|-------|
| **Category** | code_fs_process / code |
-| **Backend handler** | [`server/services/handlers/code.py::handle_typescript_executor`](../../../server/services/handlers/code.py) |
-| **Node.js client** | [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py) |
+| **Backend handler** | [`server/nodes/code/typescript_executor/__init__.py::TypeScriptExecutorNode.execute_op`](../../../server/nodes/code/typescript_executor/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`; base in [`_base.py`](../../../server/nodes/code/_base.py)) |
+| **Node.js client** | [`server/nodes/code/_nodejs.py::get_nodejs_client`](../../../server/nodes/code/_nodejs.py) (singleton over [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py)) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | - (shared with `javascript-skill`) |
| **Dual-purpose tool** | yes - tool name `typescript_code` |
@@ -18,9 +18,9 @@ All inputs, outputs, and error paths match the JS variant - the only
user-facing difference is that TypeScript type annotations parse without
error.
-Dispatched through the special-handlers branch at
-[`node_executor.py:367-381`](../../../server/services/node_executor.py) so it
-receives `connected_outputs`.
+The plugin reads upstream outputs from `ctx.raw["connected_outputs"]` (injected
+by the executor for code-executor node types) and the workflow's
+`ctx.workspace_dir`, then POSTs with `language="typescript"`.
## Inputs (handles)
@@ -32,22 +32,22 @@ receives `connected_outputs`.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `code` | string (code editor) | (boilerplate stub with `interface Result { ... }`) | yes | - | TypeScript source. User must assign to `output` |
-| `timeout` | number | `30` | no | - | Seconds - multiplied by 1000 before forwarding to the Node server |
+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | TypeScript source. User must assign to `output` |
+| `timeout` | number | `30` (ge=1, le=600) | no | - | Seconds - multiplied by 1000 before forwarding to the Node server |
-## Handler-level kwargs
+`CodeExecutorParams` uses `extra="allow"` (extra params persist but are unread).
-| Name | Default | Description |
-|------|---------|-------------|
-| `nodejs_url` | `http://localhost:3020` | |
-| `nodejs_timeout` | `30` | |
+## Node.js client singleton
+
+Shares `get_nodejs_client()` in [`_nodejs.py`](../../../server/nodes/code/_nodejs.py)
+with the JS plugin — one `NodeJSClient(base_url="http://localhost:3020",
+timeout=30)` instance, hard-coded defaults.
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `typescript_code` tool result) |
### Output payload
@@ -55,23 +55,25 @@ receives `connected_outputs`.
{
output: any;
console_output: string;
- timestamp: string;
}
```
+`node_output_schemas.CodeExecutorOutput` declares only `output` (extra fields
+like `console_output` allowed by `_OutputBase`).
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_typescript_executor] --> B{code strip empty?}
- B -- yes --> E[Return error envelope:<br/>No code provided]
- B -- no --> C[timeout_ms = int timeout * 1000]
- C --> D[input_data = connected_outputs or {}<br/>inject workspace_dir]
+ A[execute_op] --> B{code strip empty?}
+ B -- yes --> E[raise NodeUserError:<br/>No code provided]
+ B -- no --> C[timeout_ms = timeout * 1000]
+ C --> D[input_data = ctx.raw connected_outputs or {}<br/>inject workspace_dir]
D --> F[get_nodejs_client<br/>lazy singleton]
F --> G[client.execute<br/>POST /execute<br/>language=typescript]
- G -- exception --> H[Return error envelope]
- G -- ok + success=true --> I[Return success envelope]
- G -- ok + success=false --> J[Return error envelope<br/>preserve console_output]
+ G -- ClientConnectorError --> H1[raise NodeUserError<br/>TS executor not running on :3020]
+ G -- success=false --> H2[raise NodeUserError<br/>error=result.error]
+ G -- success=true --> I[Return dict<br/>output, console_output]
```
## Decision Logic
@@ -113,4 +115,4 @@ wire-level difference is the POST body's `language` field.
## Related
- **Shared skill**: [`javascript-skill/SKILL.md`](../../../server/skills/coding_agent/javascript-skill/SKILL.md)
-- **Sibling nodes**: [`javascriptExecutor`](./javascriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md)
+- **Sibling nodes**: [`javascriptExecutor`](./javascriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md), [`montyExecutor`](./montyExecutor.md)
diff --git a/docs-internal/node-logic-flows/document/documentParser.md b/docs-internal/node-logic-flows/document/documentParser.md
index e7998946..36d70da4 100644
--- a/docs-internal/node-logic-flows/document/documentParser.md
+++ b/docs-internal/node-logic-flows/document/documentParser.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | document |
-| **Backend handler** | [`server/services/handlers/document.py::handle_document_parser`](../../../server/services/handlers/document.py) |
+| **Backend handler** | [`server/nodes/document/document_parser/__init__.py`](../../../server/nodes/document/document_parser/__init__.py) — `DocumentParserNode`; dispatch via `BaseNode.execute()` + `@Operation("parse")` |
| **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -21,16 +21,18 @@ glob pattern.
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| `input-main` | main | no | Provides `files[]` upstream (e.g. from `fileDownloader`) |
+| `input-main` | main | no | Upstream data resolved into params (e.g. `input_dir` pointing at `fileDownloader`'s `output_dir`) |
## Parameters
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `files` | array | `[]` | no | - | List of `{path: string}` or raw path strings |
-| `inputDir` | string | `""` | no | - | Directory to glob for files |
-| `filePattern` | string | `*.pdf` | no | - | Glob pattern applied inside `inputDir` |
| `parser` | options | `pypdf` | no | - | `pypdf` / `marker` / `unstructured` / `beautifulsoup` |
+| `file_path` | string | `""` | no | - | Single file path (takes precedence over `input_dir`) |
+| `input_dir` | string | `""` | no | - | Directory to glob for files |
+| `file_pattern` | string | `*.pdf` | no | - | Glob pattern applied inside `input_dir` |
+
+Params are snake_case (`Params = DocumentParserParams`, `extra="ignore"`). There is no `files` param — paths come from `file_path` and/or the `input_dir` glob.
## Outputs (handles)
@@ -60,9 +62,9 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_
```mermaid
flowchart TD
- A[handle_document_parser] --> B[Normalize files param<br/>dict -> path, str -> path]
- B --> C{inputDir exists?}
- C -- yes --> C1[extend paths with inputDir.glob(filePattern)]
+ A[DocumentParserNode.parse] --> B[Collect paths from file_path]
+ B --> C{input_dir exists?}
+ C -- yes --> C1[extend paths with input_dir.glob(file_pattern)]
C -- no --> D{paths empty?}
C1 --> D
D -- yes --> Ret0[Return success=true<br/>documents=[], parsed_count=0]
@@ -86,7 +88,7 @@ flowchart TD
- **No paths**: returns success with empty `documents[]`, no error.
- **Per-file failure**: any exception inside `_parse_file_sync` is captured into `failed` with `{file, error}`; other files continue.
-- **Unknown parser**: raised by `_parse_file_sync`, recorded per-file in `failed` (NOT a top-level failure - every file fails identically).
+- **Unknown parser**: `_parse_file_sync` raises `ValueError`, but `parser` is a `Literal` so Pydantic rejects unknown values before dispatch; if it ever reaches the loop it is recorded per-file in `failed` (every file fails identically).
- **Lazy imports**: each parser imports its backend only when selected, so unused heavy deps (marker CUDA stack, unstructured) are not loaded.
- **BeautifulSoup special handling**: strips `<script>` and `<style>` tags before extracting text.
@@ -114,9 +116,9 @@ flowchart TD
- `marker`: downloads models on first call and requires CUDA; will fail on CPU-only hosts.
- `unstructured`: classifies file type from extension/content; no explicit type param.
- BeautifulSoup branch is the only one that handles HTML via `read_text`, so passing a binary `.pdf` there returns garbled `get_text`.
-- `files` and `inputDir` can be combined - directory glob is appended to the explicit list.
-- `filePattern` is applied via `Path.glob` (non-recursive). Use `**/*.pdf` for recursive.
-- The handler returns `success=true` even when every file failed - consumers must inspect `failed` and `parsed_count`.
+- `file_path` and `input_dir` can be combined - directory glob is appended to the explicit single path.
+- `file_pattern` is applied via `Path.glob` (non-recursive). Use `**/*.pdf` for recursive.
+- Returns `success=true` even when every file failed - consumers must inspect `failed` and `parsed_count`.
## Related
diff --git a/docs-internal/node-logic-flows/document/embeddingGenerator.md b/docs-internal/node-logic-flows/document/embeddingGenerator.md
index 9382daa7..4458d63b 100644
--- a/docs-internal/node-logic-flows/document/embeddingGenerator.md
+++ b/docs-internal/node-logic-flows/document/embeddingGenerator.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | document |
-| **Backend handler** | [`server/services/handlers/document.py::handle_embedding_generator`](../../../server/services/handlers/document.py) |
+| **Backend handler** | [`server/nodes/document/embedding_generator/__init__.py`](../../../server/nodes/document/embedding_generator/__init__.py) — `EmbeddingGeneratorNode`; dispatch via `BaseNode.execute()` + `@Operation("embed")` |
| **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -28,7 +28,10 @@ thread pool via `asyncio.to_thread` so the event loop is not blocked.
| `chunks` | array | `[]` | yes | - | List of `{content, source, chunk_index}` dicts |
| `provider` | options | `huggingface` | no | - | `huggingface` / `openai` / `ollama` |
| `model` | string | `BAAI/bge-small-en-v1.5` | no | - | Model name for the chosen provider |
-| `apiKey` | string | `""` | no (required for openai) | - | OpenAI API key |
+| `batch_size` | number | `32` | no | - | Texts per embedding batch (1-256). Declared but not currently used by the embed op. |
+| `api_key` | string (password) | `""` | no (required for openai) | `provider=openai` | OpenAI API key |
+
+Params are snake_case (`Params = EmbeddingGeneratorParams`).
## Outputs (handles)
@@ -61,16 +64,16 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_
```mermaid
flowchart TD
- A[handle_embedding_generator] --> B{chunks empty?}
+ A[EmbeddingGeneratorNode.embed] --> B{chunks empty?}
B -- yes --> Ret0[Return success=true<br/>zero-valued payload]
B -- no --> C[Extract text list: dict.content or str(chunk)]
C --> D{provider}
- D -- huggingface --> Dh[Import langchain_huggingface<br/>raise ImportError with install hint if missing]
+ D -- huggingface --> Dh[Import langchain_huggingface<br/>raise NodeUserError with install hint if missing]
D -- openai --> Do[Import langchain_openai.OpenAIEmbeddings]
D -- ollama --> Dl[Import langchain_ollama.OllamaEmbeddings]
- D -- other --> Dx[raise ValueError Unknown provider]
+ D -- other --> Dx[raise NodeUserError Unknown provider]
Dh --> E[HuggingFaceEmbeddings(model_name=model)]
- Do --> F[OpenAIEmbeddings(model=model, api_key=apiKey)]
+ Do --> F[OpenAIEmbeddings(model=model, api_key=api_key)]
Dl --> G[OllamaEmbeddings(model=model)]
E --> H[asyncio.to_thread embedder.embed_documents(texts)]
F --> H
@@ -82,11 +85,10 @@ flowchart TD
## Decision Logic
- **Empty chunks**: short-circuits to the empty-payload success envelope.
-- **Provider dispatch**: string match on `provider`, else `ValueError`.
+- **Provider dispatch**: string match on `provider`; `provider` is a `Literal` so Pydantic rejects unknown values before dispatch (the `else: NodeUserError` is a dead path).
- **Embedding call**: always runs in a worker thread via `asyncio.to_thread`.
- **Dimensions**: inferred from first embedding; `0` if the call returned `[]`.
-- **Unknown provider**: raised inside the try block -> caught by outer `except` -> `success=false`.
-- **Missing huggingface package**: re-raised with an install hint (`pip install langchain-huggingface sentence-transformers`).
+- **Missing huggingface package**: raises `NodeUserError` with an install hint (`pip install langchain-huggingface sentence-transformers`); `BaseNode.execute()` logs one WARN line (no traceback) and returns `{success: false, error_type: "NodeUserError"}`.
## Side Effects
@@ -101,14 +103,14 @@ flowchart TD
## External Dependencies
-- **Credentials**: `apiKey` param (OpenAI only). Not fetched from `auth_service` - must be passed as a node parameter.
+- **Credentials**: `api_key` param (OpenAI only). Not fetched from `auth_service` - must be passed as a node parameter.
- **Python packages**: `langchain-huggingface` + `sentence-transformers` (HF), `langchain-openai` (OpenAI), `langchain-ollama` (Ollama).
- **Environment variables**: `OLLAMA_HOST` respected by `langchain_ollama`.
## Edge cases & known limits
-- OpenAI `apiKey` is read from the node parameter, NOT from `auth_service.get_api_key('openai')`. This is inconsistent with other OpenAI-using nodes (documented gotcha).
-- No batching controls - everything is passed in one `embed_documents` call; provider-side batch limits apply.
+- OpenAI `api_key` is read from the node parameter, NOT from `auth_service.get_api_key('openai')`. This is inconsistent with other OpenAI-using nodes (documented gotcha).
+- `batch_size` is a declared param but the embed op passes everything in one `embed_documents` call; provider-side batch limits apply.
- HF first-run model download can take minutes; the node will appear to hang.
- Model name default (`BAAI/bge-small-en-v1.5`) is HF-specific; you must override `model` when switching provider.
- No cost tracking (unlike LLM chat nodes).
diff --git a/docs-internal/node-logic-flows/document/fileDownloader.md b/docs-internal/node-logic-flows/document/fileDownloader.md
index fd7abe76..6a07282c 100644
--- a/docs-internal/node-logic-flows/document/fileDownloader.md
+++ b/docs-internal/node-logic-flows/document/fileDownloader.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | document |
-| **Backend handler** | [`server/services/handlers/document.py::handle_file_downloader`](../../../server/services/handlers/document.py) |
+| **Backend handler** | [`server/nodes/document/file_downloader/__init__.py`](../../../server/nodes/document/file_downloader/__init__.py) — `FileDownloaderNode`; dispatch via `BaseNode.execute()` + `@Operation("download")` |
| **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -26,11 +26,13 @@ downloaded files land in the per-workflow workspace and can be picked up by
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `items` | array | `[]` | yes | - | List of `{url: string, ...}` dicts or raw URL strings |
-| `outputDir` | string | `""` | no | - | Falls back to `<workspace_dir>/downloads` or `downloads` |
-| `maxWorkers` | number | `8` | no | - | Semaphore width, 1-32 |
-| `skipExisting` | boolean | `true` | no | - | Skip when destination file already exists |
-| `timeout` | number | `60` | no | - | Per-request timeout (seconds) |
+| `urls` | array | `[]` | no | - | Declared Params field; wrapped into `items` as `[{url}]` if `items` not present |
+| `output_dir` | string | `downloads` | no | - | Declared Params field |
+| `max_workers` | number | `4` | no | - | Declared Params field (1-32) |
+| `skip_existing` | boolean | `true` | no | - | Declared Params field |
+| `timeout` | number | `60` | no | - | Declared Params field (1-600) |
+
+**Quirk**: `Params = FileDownloaderParams` declares snake_case fields (above), but `extra="allow"` lets camelCase keys pass through, and the `download` op reads camelCase off `model_dump()`: `items`, `outputDir`, `maxWorkers` (default `8` here, not `4`), `skipExisting`, plus `urls`/`timeout`. So the operative keys consumed at runtime are `items`/`urls`, `outputDir`, `maxWorkers`, `skipExisting`, `timeout`.
## Outputs (handles)
@@ -62,7 +64,7 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_
```mermaid
flowchart TD
- A[handle_file_downloader] --> B{items empty?}
+ A[FileDownloaderNode.download] --> B{items empty?}
B -- yes --> Ret0[Return success=true<br/>counts all zero]
B -- no --> C[Resolve output_dir from outputDir or context.workspace_dir or ./downloads]
C --> D[mkdir parents exist_ok]
diff --git a/docs-internal/node-logic-flows/document/httpScraper.md b/docs-internal/node-logic-flows/document/httpScraper.md
index 54291e2d..babe1077 100644
--- a/docs-internal/node-logic-flows/document/httpScraper.md
+++ b/docs-internal/node-logic-flows/document/httpScraper.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | document |
-| **Backend handler** | [`server/services/handlers/document.py::handle_http_scraper`](../../../server/services/handlers/document.py) |
+| **Backend handler** | [`server/nodes/document/http_scraper/__init__.py`](../../../server/nodes/document/http_scraper/__init__.py) — `HttpScraperNode`; dispatch via `BaseNode.execute()` + `@Operation("scrape")` |
| **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -28,15 +28,22 @@ elements become items. Typically the first stage of an ingestion pipeline:
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `url` | string | `""` | **yes** | - | URL with optional `{date}` / `{page}` placeholder |
-| `iterationMode` | options | `single` | no | - | `single` / `date` / `page` |
-| `startDate` | string | `""` | yes (date mode) | `iterationMode=date` | `YYYY-MM-DD` |
-| `endDate` | string | `""` | yes (date mode) | `iterationMode=date` | `YYYY-MM-DD`, inclusive |
-| `datePlaceholder` | string | `{date}` | no | `iterationMode=date` | Substring replaced with formatted date |
-| `startPage` | number | `1` | no | `iterationMode=page` | Inclusive |
-| `endPage` | number | `10` | no | `iterationMode=page` | Inclusive; `{page}` literal replaced |
-| `linkSelector` | string | `a[href$=".pdf"]` | no | - | BeautifulSoup CSS selector |
+| `iteration_mode` | options | `single` | no | - | `single` / `date` / `page` |
+| `link_selector` | string | `a[href$=".pdf"]` | no | - | BeautifulSoup CSS selector |
| `headers` | string (JSON) | `{}` | no | - | Extra HTTP headers as JSON object |
-| `useProxy` | boolean | `false` | no | - | Route through proxy service if enabled |
+| `start_date` | string | `""` | yes (date mode) | `iteration_mode=date` | `YYYY-MM-DD` |
+| `end_date` | string | `""` | yes (date mode) | `iteration_mode=date` | `YYYY-MM-DD`, inclusive |
+| `date_placeholder` | string | `{date}` | no | `iteration_mode=date` | Substring replaced with formatted date |
+| `start_page` | number | `1` | no | `iteration_mode=page` | Inclusive |
+| `end_page` | number | `10` | no | `iteration_mode=page` | Inclusive; `{page}` literal replaced |
+| `max_pages` | number | `10` | no | - | Safety cap on pages fetched (1-1000) |
+| `use_proxy` | boolean | `false` | no | - | Route through proxy service if enabled |
+| `proxy_provider` | string | `auto` | no | `use_proxy=true` | Provider name |
+| `proxy_country` | string | `""` | no | `use_proxy=true` | ISO country code |
+| `session_type` | options | `rotating` | no | `use_proxy=true` | `rotating` / `sticky` |
+| `sticky_duration` | number | `600` | no | `use_proxy=true, session_type=sticky` | Sticky session seconds |
+
+Params are snake_case (`Params = HttpScraperParams`, `extra="ignore"`).
## Outputs (handles)
@@ -66,12 +73,12 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_
```mermaid
flowchart TD
- A[handle_http_scraper] --> B{url provided?}
- B -- no --> Eret[Return success=false<br/>error: URL is required]
- B -- yes --> C[Parse headers JSON]
- C --> D{iterationMode}
- D -- date --> D1[Iterate start_date..end_date days<br/>replace datePlaceholder per URL]
- D -- page --> D2[Iterate startPage..endPage<br/>replace `{page}` literal per URL]
+ A[HttpScraperNode.scrape] --> B{url provided?}
+ B -- no --> Eret[raise NodeUserError URL is required<br/>-> success=false envelope]
+ B -- yes --> C[json.loads headers]
+ C --> D{iteration_mode}
+ D -- date --> D1[Iterate start_date..end_date days<br/>replace date_placeholder per URL]
+ D -- page --> D2[Iterate start_page..end_page<br/>replace `{page}` literal per URL]
D -- single --> D3[Single URL, meta=`{}`]
D1 --> E[Build urls_to_fetch list]
D2 --> E
@@ -89,10 +96,10 @@ flowchart TD
## Decision Logic
-- **Validation**: missing `url` raises `ValueError("URL is required")`; caught by outer `except` -> `success=false`.
-- **Date mode**: missing `startDate` or `endDate` raises `ValueError` -> `success=false`.
-- **Iteration mode fallback**: any value other than `date` or `page` takes the single-URL branch (no validation of unknown modes).
-- **Per-URL errors**: collected into `errors` list; the handler still returns `success=true` even if every URL failed (only top-level setup errors fail the envelope).
+- **Validation**: missing `url` raises `NodeUserError("URL is required")` (single WARN line, no traceback) -> `success=false` envelope.
+- **Date mode**: missing `start_date` or `end_date` raises `NodeUserError` -> `success=false`.
+- **Iteration mode**: `iteration_mode` is a `Literal`, so unknown values are rejected by Pydantic; `single` is the only non-date/page branch.
+- **Per-URL errors**: collected into `errors` list; the op still returns `success=true` even if every URL failed (only top-level setup errors fail the envelope).
- **Proxy failure**: logged at warning, request proceeds without proxy.
## Side Effects
@@ -112,12 +119,13 @@ flowchart TD
## Edge cases & known limits
-- `datePlaceholder` is a plain `str.replace`, not a token boundary - a URL containing the placeholder substring twice is replaced twice.
+- `date_placeholder` is a plain `str.replace`, not a token boundary - a URL containing the placeholder substring twice is replaced twice.
- Date mode formats dates as `YYYY-MM-DD`; placeholder in URL is replaced with this exact format regardless of what the user wrote.
- `{page}` in page mode is hard-coded, not configurable (unlike the date placeholder).
-- `headers` is parsed with `json.loads` with no try/except - invalid JSON raises at the top level and fails the envelope.
+- `headers` is parsed with `json.loads` with no try/except - invalid JSON raises (caught as a bare `Exception` -> full traceback) and fails the envelope.
+- `max_pages` is declared but not enforced in the page-iteration loop (the loop runs `start_page..end_page`).
- Per-URL failures are silently aggregated into `errors` but the envelope is still `success=true`; downstream nodes must check `errors`.
-- `endPage`/`endDate` are inclusive.
+- `end_page`/`end_date` are inclusive.
## Related
diff --git a/docs-internal/node-logic-flows/document/textChunker.md b/docs-internal/node-logic-flows/document/textChunker.md
index 0a39dcf9..3deb6a56 100644
--- a/docs-internal/node-logic-flows/document/textChunker.md
+++ b/docs-internal/node-logic-flows/document/textChunker.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | document |
-| **Backend handler** | [`server/services/handlers/document.py::handle_text_chunker`](../../../server/services/handlers/document.py) |
+| **Backend handler** | [`server/nodes/document/text_chunker/__init__.py`](../../../server/nodes/document/text_chunker/__init__.py) — `TextChunkerNode`; dispatch via `BaseNode.execute()` + `@Operation("chunk")` |
| **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -26,10 +26,12 @@ pipeline.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `documents` | array | `[]` | yes | - | List of `{content, source}` dicts (or raw strings) |
-| `chunkSize` | number | `1024` | no | - | Max chars per chunk (100-8000) |
-| `chunkOverlap` | number | `200` | no | - | Char overlap between adjacent chunks (0-1000) |
-| `strategy` | options | `recursive` | no | - | `recursive` / `markdown` (token strategy is documented but falls through to recursive) |
+| `text` | string | `""` | no | - | Declared Params field; wrapped into `documents` as `[{content: text}]` if `documents` absent |
+| `strategy` | options | `recursive` | no | - | `recursive` / `markdown` / `token` (token falls through to recursive) |
+| `chunk_size` | number | `1000` | no | - | Declared Params field (100-8000) |
+| `overlap` | number | `200` | no | - | Declared Params field (0-1000) |
+
+**Quirk**: `Params = TextChunkerParams` declares `text` / `strategy` / `chunk_size` / `overlap` (snake_case), but `extra="allow"` passes camelCase through and the `chunk` op reads `documents`, `chunkSize` (default `1000`), `chunkOverlap` or `overlap`, and `strategy` off `model_dump()`. So the operative inputs at runtime are `documents`/`text`, `chunkSize`, `chunkOverlap`/`overlap`, `strategy`.
## Outputs (handles)
@@ -57,7 +59,7 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_
```mermaid
flowchart TD
- A[handle_text_chunker] --> B{documents empty?}
+ A[TextChunkerNode.chunk] --> B{documents empty?}
B -- yes --> Ret0[Return success=true<br/>chunks=[], chunk_count=0]
B -- no --> C{strategy}
C -- markdown --> Cm[MarkdownTextSplitter]
diff --git a/docs-internal/node-logic-flows/document/vectorStore.md b/docs-internal/node-logic-flows/document/vectorStore.md
index 59067e5e..89ff2fe2 100644
--- a/docs-internal/node-logic-flows/document/vectorStore.md
+++ b/docs-internal/node-logic-flows/document/vectorStore.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | document |
-| **Backend handler** | [`server/services/handlers/document.py::handle_vector_store`](../../../server/services/handlers/document.py) |
+| **Backend handler** | [`server/nodes/document/vector_store/__init__.py`](../../../server/nodes/document/vector_store/__init__.py) — `VectorStoreNode`; dispatch via `BaseNode.execute()` + `@Operation("dispatch")` |
| **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -26,16 +26,18 @@ pipeline and/or the retrieval stage at query time.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `operation` | options | `store` | no | - | `store` / `query` / `delete` |
-| `backend` | options | `chroma` | no | - | `chroma` / `qdrant` / `pinecone` |
-| `collectionName` | string | `documents` | no | - | Collection / index name |
-| `embeddings` | array | `[]` | yes (store) | `operation=store` | Vectors from `embeddingGenerator` |
-| `chunks` | array | `[]` | no (store) | `operation=store` | Metadata pairs for the vectors |
-| `queryEmbedding` | array | `[]` | yes (query) | `operation=query` | Single vector to query |
-| `topK` | number | `5` | no | `operation=query` | Max matches to return |
-| `ids` | array | `[]` | yes (delete) | `operation=delete` | Vector IDs to delete |
-| `persistDir` | string | `./data/vectors` | no | `backend=chroma` | ChromaDB persistence dir |
-| `qdrantUrl` | string | `http://localhost:6333` | no | `backend=qdrant` | Qdrant URL |
-| `pineconeApiKey` | string | `""` | yes | `backend=pinecone` | Pinecone API key |
+| `backend` | options | `chroma` | no | - | `chroma` / `chromadb` / `qdrant` / `pinecone` (`chromadb` aliased to `chroma`) |
+| `collection_name` | string | `documents` | no | - | Collection / index name |
+| `embeddings` | array | `null` | yes (store) | - | Vectors from `embeddingGenerator` |
+| `chunks` | array | `null` | no (store) | `operation=store` | Metadata pairs for the vectors |
+| `query_embedding` | array | `null` | yes (query) | `operation=query` | Single vector to query |
+| `top_k` | number | `5` | no | `operation=query` | Max matches to return (1-100) |
+| `ids` | array | `null` | yes (delete) | `operation=delete` | Vector IDs to delete |
+| `persist_dir` | string | `./data/vectors` | no | `backend=chroma,chromadb` | ChromaDB persistence dir |
+| `qdrant_url` | string | `http://localhost:6333` | no | `backend=qdrant` | Qdrant URL |
+| `pinecone_api_key` | string | `""` | yes | `backend=pinecone` | Pinecone API key |
+
+Params are snake_case (`Params = VectorStoreParams`, `extra="ignore"`).
## Outputs (handles)
@@ -76,16 +78,16 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_
```mermaid
flowchart TD
- A[handle_vector_store] --> B{backend}
- B -- chroma --> C1[_chroma_op]
+ A[VectorStoreNode.dispatch] --> B{backend}
+ B -- chroma/chromadb --> C1[_chroma_op]
B -- qdrant --> C2[_qdrant_op]
B -- pinecone --> C3[_pinecone_op]
- B -- other --> Ex[raise ValueError Unknown backend]
+ B -- other --> Ex[raise NodeUserError Unknown backend]
C1 --> D{operation}
C2 --> D
C3 --> D
D -- store --> Ds[Auto-create collection if needed<br/>generate uuid ids + metas<br/>upsert via asyncio.to_thread]
- D -- query --> Dq[query / search with queryEmbedding + topK<br/>normalize matches across backends]
+ D -- query --> Dq[query / search with query_embedding + top_k<br/>normalize matches across backends]
D -- delete --> Dd[delete by ids, count = len(ids)]
Ds --> E[Return partial result]
Dq --> E
@@ -96,9 +98,10 @@ flowchart TD
## Decision Logic
-- **Backend dispatch**: `chroma` / `qdrant` / `pinecone` - anything else raises `ValueError` caught as `success=false`.
+- **Backend dispatch**: `backend` is a `Literal` (`chroma` / `chromadb` / `qdrant` / `pinecone`); `chromadb` is normalized to `chroma`. Unknown values are rejected by Pydantic before dispatch (the `else: NodeUserError` is a dead path).
+- **Pinecone without key**: `_pinecone_op` raises `NodeUserError("Pinecone API key required")` when `pinecone_api_key` is empty.
- **Empty embeddings (store)**: returns `stored_count=0` without raising; for Chroma also returns current `collection_count`.
-- **Empty queryEmbedding (query)**: returns `matches=[]` without hitting the backend.
+- **Empty query_embedding (query)**: returns `matches=[]` without hitting the backend.
- **Empty ids (delete)**: returns `deleted=true, count=0` without hitting the backend.
- **Collection auto-creation**:
- Chroma: `get_or_create_collection` always.
@@ -120,7 +123,7 @@ flowchart TD
## External Dependencies
-- **Credentials**: `pineconeApiKey` node parameter (Pinecone only). Not fetched from `auth_service`.
+- **Credentials**: `pinecone_api_key` node parameter (Pinecone only). Not fetched from `auth_service`.
- **Python packages**: `chromadb>=0.5.0`, `qdrant-client>=1.12.0`, `pinecone` (all lazy-imported per backend).
- **Environment variables**: none (Qdrant URL is a param, Pinecone key is a param).
@@ -130,7 +133,7 @@ flowchart TD
- Chroma is the only backend that returns `collection_count` after a store.
- Qdrant collection creation infers the vector size from the first embedding; mixed-size batches will be rejected by Qdrant on upsert.
- Pinecone `pc.Index(collection)` does not create the index - it must exist, or the op fails with Pinecone's own error.
-- Pinecone `apiKey` is a node parameter, inconsistent with other cloud services that pull from `auth_service`.
+- Pinecone `pinecone_api_key` is a node parameter, inconsistent with other cloud services that pull from `auth_service`.
- Delete is by ID only - no metadata-filter delete.
- Query match shape differs per backend: Chroma returns `distance`, Qdrant and Pinecone return `score`. Downstream consumers must handle both.
diff --git a/docs-internal/node-logic-flows/email/emailRead.md b/docs-internal/node-logic-flows/email/emailRead.md
index 408e0cb8..e618a738 100644
--- a/docs-internal/node-logic-flows/email/emailRead.md
+++ b/docs-internal/node-logic-flows/email/emailRead.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | email / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/email.py::handle_email_read`](../../../server/services/handlers/email.py) |
+| **Backend handler** | [`server/nodes/email/email_read/__init__.py`](../../../server/nodes/email/email_read/__init__.py) — dispatched via `BaseNode.execute()` -> `@Operation("query")` -> `EmailService.read` ([`_service.py`](../../../server/nodes/email/_service.py)) |
| **Tests** | [`server/tests/nodes/test_email.py`](../../../server/tests/nodes/test_email.py) |
| **Skill (if any)** | none shipped |
| **Dual-purpose tool** | yes - connect to `input-tools` as the node's own name |
@@ -25,15 +25,22 @@ node exposes seven operations selected by the `operation` parameter.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `provider` | options | `gmail` | no | - | Preset key in `email_providers.json` |
-| `operation` | options | `list` | **yes** | - | `list` / `search` / `read` / `folders` / `move` / `delete` / `flag` |
-| `folder` | string | `INBOX` | no | show for all except `folders` | IMAP folder name |
-| `query` | string | `""` | **yes** (`search`) | `operation: ['search']` | Himalaya search expression, e.g. `from:x subject:y` |
-| `message_id` | string | `""` | **yes** for `read`/`move`/`delete`/`flag` | `operation: ['read','move','delete','flag']` | IMAP message UID |
-| `target_folder` | string | `""` | **yes** (`move`) | `operation: ['move']` | Destination folder for `move` |
-| `flag` | options | `Seen` | no | `operation: ['flag']` | One of `Seen` / `Answered` / `Flagged` / `Draft` / `Deleted` |
+| `operation` | options | `list` | no | - | `list` / `search` / `read` / `folders` / `move` / `delete` / `flag` |
+| `folder` | string | `INBOX` | no | `operation: ['list','search']` | IMAP folder name |
+| `query` | string | `""` | no (`search`) | `operation: ['search']` | Himalaya search expression, e.g. `from:x subject:y` |
+| `message_id` | string | `""` | no (used by `read`/`move`/`delete`/`flag`) | `operation: ['read','move','delete','flag']` | IMAP message UID |
+| `target_folder` | string | `""` | no (`move`) | `operation: ['move']` | Destination folder for `move` |
+| `flag` | options | `""` | no | `operation: ['flag']` | One of `Seen` / `Answered` / `Flagged` / `Draft` / `Deleted` (empty default) |
| `flag_action` | options | `add` | no | `operation: ['flag']` | `add` or `remove` |
-| `page` | number | `1` | no | `operation: ['list']` | 1-based page number |
-| `page_size` | number | `20` | no | `operation: ['list']` | 1-100 envelopes per page |
+| `limit` | number | `20` | no | `operation: ['list','search']` | Max envelopes per page (1-500) |
+| `page` | number | `1` | no | `operation: ['list','search']` | 1-based page number |
+| `page_size` | number | `20` | no | `operation: ['list','search']` | Items per page (1-500); overrides `limit` when paginating |
+| `offset` | number | `0` | no | `operation: ['list','search']` | Skip this many messages (alternative to page-based pagination) |
+
+> Note: no parameter is Pydantic-required (`operation` defaults to `list`); the
+> operation router in `EmailService.read` passes empty strings straight to
+> Himalaya, which fails with a non-zero exit for ops that genuinely need
+> `message_id` / `query` / `target_folder`.
Credential params (`email`, `password`, `imap_host`, etc.) follow the same
resolution rules as [`emailSend`](./emailSend.md#decision-logic).
@@ -43,7 +50,8 @@ resolution rules as [`emailSend`](./emailSend.md#decision-logic).
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Operation result |
-| `output-tool` | object | Same payload, used when wired as an AI tool |
+
+When wired to an AI agent's `input-tools` handle (`usable_as_tool = True`, tool name `email_read`), the same payload is returned to the LLM via the tool-dispatch path — there is no separate `output-tool` handle.
### Output payload
@@ -66,7 +74,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution
```mermaid
flowchart TD
- A[handle_email_read] --> B[EmailService.read params]
+ A[BaseNode.execute -> Operation query] --> B[EmailService.read params]
B --> C[resolve_credentials]
C -->|missing email/password| Ec[ValueError -> error envelope]
C --> D{operation in router?}
@@ -127,9 +135,10 @@ flowchart TD
## Edge cases & known limits
-- **Page-size clamp is frontend-only**: the node def caps `page_size` at 100,
- but the backend passes whatever value arrives. Requesting 10000 will hit
- the IMAP server's own limits.
+- **Pagination uses `page`/`page_size` only**: the router in `EmailService.read`
+ reads `page` + `page_size`; the `limit` and `offset` params exist on the model
+ but are not forwarded to Himalaya for `list`/`search`. `page_size` is Pydantic-
+ validated to `1-500`; the IMAP server's own limits still apply.
- **No pagination metadata**: `list` returns whatever Himalaya gives; there
is no `has_more` / `total` helper.
- **Deletion is permanent** (no undo), and the handler does not prompt or
diff --git a/docs-internal/node-logic-flows/email/emailReceive.md b/docs-internal/node-logic-flows/email/emailReceive.md
index 8cf00aaa..95b98ba8 100644
--- a/docs-internal/node-logic-flows/email/emailReceive.md
+++ b/docs-internal/node-logic-flows/email/emailReceive.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | email / trigger |
-| **Backend handler** | [`server/services/handlers/email.py::handle_email_receive`](../../../server/services/handlers/email.py) |
+| **Backend handler** | [`server/nodes/email/email_receive/__init__.py`](../../../server/nodes/email/email_receive/__init__.py) — Run-button path is the `execute()` override; deployment-mode polling uses the `PollingTriggerNode` hooks (`setup_service` / `fetch_ids` / `fetch_detail` / `post_emit`) draining via the canary `TriggerListenerWorkflow` path |
| **Tests** | [`server/tests/nodes/test_email.py`](../../../server/tests/nodes/test_email.py) |
| **Skill (if any)** | none shipped |
| **Dual-purpose tool** | no (trigger only) |
@@ -24,9 +24,9 @@ None. Triggers have no inputs.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `provider` | options | `googleGmail` | no | - | Preset key in `email_providers.json` |
+| `provider` | options | `gmail` | no | - | Preset key in `email_providers.json` |
| `folder` | string | `INBOX` | no | - | IMAP folder to watch |
-| `poll_interval` | number | `60` | no | - | Seconds between polls. Clamped to `[min_interval, max_interval]` from config (30-3600) |
+| `poll_interval` | number | `60` | no | - | Seconds between polls. Pydantic-validated `30-3600` (`poll_interval_clamp = (30, 3600)`) and re-clamped against config `min_interval`/`max_interval` |
| `filter_query` | string | `""` | no | - | Reserved, currently not applied during polling |
| `mark_as_read` | boolean | `false` | no | - | When true, adds the `Seen` flag to the new message after fetch |
@@ -57,7 +57,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution
```mermaid
flowchart TD
- A[handle_email_receive] --> B[resolve_credentials]
+ A[EmailReceiveNode.execute] --> B[resolve_credentials]
B -->|ValueError| Ec[error envelope]
B --> C[resolve_poll_params]
C --> D[broadcaster.update_node_status waiting]
@@ -70,7 +70,7 @@ flowchart TD
I --> J{mark_as_read?}
J -- yes --> K[himalaya flag add Seen id]
J -- no --> L[skip]
- K --> M[event_waiter.dispatch email_received]
+ K --> M[dispatch_email_received -> dispatch.emit<br/>CloudEvents WorkflowEvent]
L --> M
M --> N[Return success envelope]
```
@@ -83,9 +83,10 @@ flowchart TD
- **Interval clamping** in `EmailService.resolve_poll_params`:
`max(min_interval, min(max_interval, requested))`. Defaults come from
`email_providers.json -> polling` block.
-- **One event per handler run**: as soon as any new IDs are seen, the handler
- fetches the *first* of them, dispatches a single `email_received` event,
- and returns. Remaining new IDs are added to `seen` but not processed;
+- **One event per Run-button `execute()` run**: as soon as any new IDs are seen,
+ it fetches the *first* of them, dispatches a single `email_received` event via
+ `dispatch_email_received`, and returns. Remaining new IDs are added to `seen`
+ but not processed;
they will be skipped on the next run because they are now considered
baseline.
- **`filter_query` is unused in polling**: the parameter appears in the UI
@@ -99,13 +100,20 @@ flowchart TD
## Side Effects
-- **Database writes**: none directly. (The `event_waiter.dispatch` side is
- in-memory only - no DB row is written for the event.)
+- **Database writes**: none directly.
- **Broadcasts**: one `update_node_status(..., "waiting", ...)` when polling
starts, with `workflow_id` from context.
-- **Event bus**: `event_waiter.dispatch("email_received", email_data)` fires
- once per handler run. Downstream trigger consumers (e.g. another
- `emailReceive` listener) receive this payload.
+- **Event bus (canary path)**: `dispatch_email_received(email_data)` ->
+ `services.events.dispatch.emit` with a CloudEvents `WorkflowEvent`
+ (`type = "com.machinaos.email.message.received"`, `subject = message_id`,
+ outer wire-routing key `email_received`). `emailReceive` is canary-registered
+ via `register_canary_trigger_type("emailReceive", "com.machinaos.email.message.received")`
+ in [`nodes/email/__init__.py`](../../../server/nodes/email/__init__.py), so the
+ deployment manager skips the legacy `setup_event_trigger` and a Temporal-durable
+ `TriggerListenerWorkflow` consumes the event; the same call also broadcasts the
+ envelope to the FE on the `email_received` wire key. The legacy
+ `event_waiter.dispatch` path has zero consumers in canary mode and is no longer
+ called (`_events.py`).
- **External API calls**: none direct - IMAP traffic flows through Himalaya.
- **File I/O**: one `himalaya_*.toml` tempfile per subprocess call (baseline,
each poll, fetch, flag). Each is deleted in `finally`.
@@ -117,7 +125,8 @@ flowchart TD
- **Binary**: `himalaya` on `PATH`.
- **Credentials**: same as [`emailSend`](./emailSend.md#external-dependencies).
- **Services**: `StatusBroadcaster` (for the waiting status),
- `event_waiter` (for dispatch).
+ `services.events.dispatch` (canary CloudEvents emit via `_events.py`);
+ `event_waiter.register_filter_builder` only for the `emailReceive` filter.
- **Config**: `server/config/email_providers.json -> polling`.
## Edge cases & known limits
diff --git a/docs-internal/node-logic-flows/email/emailSend.md b/docs-internal/node-logic-flows/email/emailSend.md
index debb2d51..6a1c5c8a 100644
--- a/docs-internal/node-logic-flows/email/emailSend.md
+++ b/docs-internal/node-logic-flows/email/emailSend.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | email / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/email.py::handle_email_send`](../../../server/services/handlers/email.py) |
+| **Backend handler** | [`server/nodes/email/email_send/__init__.py`](../../../server/nodes/email/email_send/__init__.py) — dispatched via `BaseNode.execute()` -> `@Operation("send")` -> `EmailService.send` ([`_service.py`](../../../server/nodes/email/_service.py)) |
| **Tests** | [`server/tests/nodes/test_email.py`](../../../server/tests/nodes/test_email.py) |
| **Skill (if any)** | none shipped |
| **Dual-purpose tool** | yes - connect to `input-tools` as the node's own name |
@@ -29,7 +29,7 @@ presets (see `server/config/email_providers.json`). When wired to an AI agent's
| `provider` | options | `gmail` | no | - | One of `gmail` / `outlook` / `yahoo` / `icloud` / `protonmail` / `fastmail` / `custom`. Picks the preset in `email_providers.json` |
| `to` | string | `""` | **yes** | - | Comma-separated recipient list |
| `subject` | string | `""` | **yes** | - | Email subject line |
-| `body` | string | `""` | **yes** | - | Email body (plain text or HTML, see `body_type`) |
+| `body` | string | `""` | no | - | Email body (plain text or HTML, see `body_type`); `rows: 6` textarea hint |
| `cc` | string | `""` | no | - | CC recipients, only added if truthy |
| `bcc` | string | `""` | no | - | BCC recipients, only added if truthy |
| `body_type` | options | `text` | no | - | `text` or `html`. HTML sends a `multipart/alternative` MIME message |
@@ -43,7 +43,8 @@ Also resolved at runtime (not node-exposed):
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Himalaya send result merged with `from` (sender address) |
-| `output-tool` | object | Same payload, used when wired as an AI tool |
+
+When wired to an AI agent's `input-tools` handle (`usable_as_tool = True`, tool name `email_send`), the same payload is returned to the LLM via the tool-dispatch path — there is no separate `output-tool` handle.
### Output payload
@@ -61,7 +62,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution
```mermaid
flowchart TD
- A[handle_email_send] --> B[EmailService.send params]
+ A[BaseNode.execute -> Operation send] --> B[EmailService.send params]
B --> C[resolve_credentials]
C -->|no email/password| Ee[ValueError -> error envelope]
C --> D[Merge with provider preset from email_providers.json]
@@ -88,8 +89,9 @@ flowchart TD
`provider == 'custom'`.
- **Port coercion**: stored ports are strings in the API-key store; `_coerce_port`
casts them to `int` and silently returns `None` on invalid values.
-- **Missing email/password**: `resolve_credentials` raises `ValueError` before
- any subprocess is spawned -> `handle_email_send` catches and returns
+- **Missing email/password**: `resolve_credentials` raises `ValueError`
+ (`"Email address not configured"` / `"Email password not configured"`) before
+ any subprocess is spawned -> `BaseNode.execute()` catches and returns
`success=false`.
- **HTML vs plain**: `body_type == 'html'` -> `multipart/alternative` with an
HTML part (no plain-text fallback part is attached). Anything else -> a
@@ -101,8 +103,8 @@ flowchart TD
`HimalayaService._account_name` (`[email protected]` -> `john_doe`). Dots and
`+` are replaced with `_`.
- **Subprocess errors**: non-zero returncode raises `RuntimeError("himalaya
- error: <stderr>")`; the handler's `except Exception` catches and emits an
- error envelope.
+ error: <stderr>")`; `BaseNode.execute()`'s `except Exception` catches and
+ emits the error envelope (with full traceback logging).
## Side Effects
diff --git a/docs-internal/node-logic-flows/google_workspace/googleCalendar.md b/docs-internal/node-logic-flows/google_workspace/googleCalendar.md
index 6e82bfb8..633bcf07 100644
--- a/docs-internal/node-logic-flows/google_workspace/googleCalendar.md
+++ b/docs-internal/node-logic-flows/google_workspace/googleCalendar.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | google_workspace / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/calendar.py::handle_google_calendar`](../../../server/services/handlers/calendar.py) |
+| **Backend handler** | [`server/nodes/google/calendar/__init__.py`](../../../server/nodes/google/calendar/__init__.py) (`CalendarNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) |
| **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) |
| **Skill (if any)** | [`server/skills/productivity_agent/google-calendar-skill/SKILL.md`](../../../server/skills/productivity_agent/google-calendar-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `googleCalendar` |
+| **Dual-purpose tool** | yes - tool name `google_calendar` |
## Purpose
@@ -57,8 +57,11 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `update`, `delete`).
| `title` / `start_time` / `end_time` / `description` / `location` | string | `""` | no | Patch fields |
| `calendar_id` | string | `primary` | no | - |
-Also accepts `update_title`, `update_start_time`, etc. which the dispatcher
-aliases onto the standard names.
+Also accepts the optional `update_title`, `update_start_time`,
+`update_end_time`, `update_description`, `update_location` fields. The dispatch
+method does NOT mutate the params dict — it reads `params.update_title or
+params.title` (etc.) as fallback chains so either the `update_*` field or the
+plain field works.
### `operation = delete`
@@ -70,10 +73,13 @@ aliases onto the standard names.
## Outputs (handles)
+The node declares only `input-main` and `output-main`. Tool mode
+(`usable_as_tool = True`, tool name `google_calendar`) returns the same
+`output-main` payload — there is no separate `output-tool` handle.
+
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Operation-specific payload |
-| `output-tool` | object | Same, for AI tool wiring |
+| `output-main` | object | Operation-specific `CalendarOutput` payload |
- `create` / `update`: `{event_id, title, start, end, html_link, status, created?/updated?}`
- `list`: `{events: [{event_id, title, start, end, description, location, status, html_link, attendees}], count, time_range: {start, end}}`
@@ -83,58 +89,48 @@ aliases onto the standard names.
```mermaid
flowchart TD
- A[handle_google_calendar] --> AL{operation == update?}
- AL -- yes --> AM[Alias update_* params<br/>to standard names]
- AL -- no --> B
- AM --> B{operation?}
- B -- create --> C1[handle_calendar_create]
- B -- list --> C2[handle_calendar_list]
- B -- update --> C3[handle_calendar_update]
- B -- delete --> C4[handle_calendar_delete]
- B -- unknown --> Eret[success=false<br/>Unknown Calendar operation]
- C1 --> G[get_google_credentials + build v3]
- C2 --> G
- C3 --> G
- C4 --> G
- G -- ValueError --> Eret
- C1 --> V1{title & start & end present?}
- V1 -- no --> Eret
- V1 -- yes --> R1[events.insert sendUpdates=all]
- C2 --> DP[Parse date shortcuts:<br/>today / today+Nd]
+ A[BaseNode.execute -> CalendarNode.dispatch] --> G[build_google_service calendar v3]
+ G --> B{operation?}
+ B -- create --> V1{title & start & end present?}
+ V1 -- no --> Eret[raise RuntimeError]
+ V1 -- yes --> R1[events.insert sendUpdates]
+ B -- list --> DP[Parse date shortcuts:<br/>today / today+Nd]
DP --> R2[events.list timeMin/timeMax]
- C3 --> V3{event_id?}
+ B -- update --> V3{event_id?}
V3 -- no --> Eret
- V3 -- yes --> G3[events.get -> merge updates -> events.update]
- C4 --> V4{event_id?}
+ V3 -- yes --> G3[events.get -> merge update_* fallbacks -> events.update]
+ B -- delete --> V4{event_id?}
V4 -- no --> Eret
V4 -- yes --> R4[events.delete sendUpdates]
- R1 --> T[_track_calendar_usage]
+ B -- unknown --> Eret
+ R1 --> T[track_google_usage google_calendar]
R2 --> T
G3 --> T
R4 --> T
- T --> OUT[Return success envelope]
+ T --> OUT[Return CalendarOutput; BaseNode serializes envelope]
```
## Decision Logic
+- **Operation branch**: one `@Operation("dispatch")` method branches on `params.operation` (`create` / `list` / `update` / `delete`); unknown raises `RuntimeError`.
- **Date shortcut parsing** (list): `start_date` of `today` or empty -> today 00:00 UTC; `end_date` starting with `today+` is `today + Nd`; otherwise the raw string is used and `Z` is appended if missing.
- **Timezone fallback** (update): when updating start/end, the existing event's `timeZone` is reused; if absent falls back to `UTC`.
-- **Update parameter aliasing**: the dispatcher copies `update_title`, `update_start_time`, `update_end_time`, `update_description`, `update_location` onto their plain names before delegating - this is irreversible mutation of the incoming `parameters` dict.
+- **Update fallback chains**: `params.update_title or params.title` (etc.) — no mutation of the params dict; either field works.
- **Description / location**: treated as `is not None` rather than truthy - empty string is a valid clear.
- **Attendees**: empty strings after `split(',')` are filtered; the whole `attendees` field is omitted if the list is empty.
-- **`order_by`**: only sent when `single_events=True`; otherwise `None`, which the Google client library strips.
+- **`order_by`**: only sent when `single_events=True`.
## Side Effects
-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_calendar'`.
-- **Broadcasts**: none.
-- **External API calls**: Calendar API v3 - `events().insert/list/update/delete/get`. `sendUpdates='all'` on create/update sends email invitations to attendees.
+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_calendar'`.
+- **Broadcasts**: none from the operation; executor emits standard `node_status`.
+- **External API calls**: Calendar API v3 - `events().insert/list/update/delete/get`. `sendUpdates` on create/update/delete drives email invitations/cancellations.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.
+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.
- **Services**: Google Calendar API, `PricingService`, `Database`.
- **Python packages**: `google-api-python-client`.
- **Environment variables**: none.
@@ -144,7 +140,6 @@ flowchart TD
- `max_results` clamped to 250 silently.
- `update` performs a read-modify-write cycle: two API calls per update. A concurrent modification between the `get` and `update` is overwritten without ETag checking.
- `delete` with an invalid event_id will surface the `HttpError 404` message into the envelope as a string.
-- The dispatcher mutates the incoming `parameters` dict for update aliasing; callers that reuse the dict will see the aliased fields.
- `description=""` is treated as a real clear; `description=None` or missing leaves the field untouched. Subtle but load-bearing for update callers.
## Related
diff --git a/docs-internal/node-logic-flows/google_workspace/googleContacts.md b/docs-internal/node-logic-flows/google_workspace/googleContacts.md
index f845edc8..92a2841f 100644
--- a/docs-internal/node-logic-flows/google_workspace/googleContacts.md
+++ b/docs-internal/node-logic-flows/google_workspace/googleContacts.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | google_workspace / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/contacts.py::handle_google_contacts`](../../../server/services/handlers/contacts.py) |
+| **Backend handler** | [`server/nodes/google/contacts/__init__.py`](../../../server/nodes/google/contacts/__init__.py) (`ContactsNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) |
| **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) |
| **Skill (if any)** | [`server/skills/productivity_agent/google-contacts-skill/SKILL.md`](../../../server/skills/productivity_agent/google-contacts-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `googleContacts` |
+| **Dual-purpose tool** | yes - tool name `google_contacts` |
## Purpose
@@ -64,8 +64,10 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `search`, `get`, `up
| `resource_name` | string | `""` | **yes** | - |
| `first_name` / `last_name` / `email` / `phone` / `company` / `job_title` | string | `""` | no | Any provided field triggers its person field group to be replaced. **At least one patch field required.** |
-Also accepts `update_first_name`, `update_last_name`, `update_email`,
-`update_phone`, `update_company`, `update_job_title` aliases.
+Also accepts the optional `update_first_name`, `update_last_name`,
+`update_email`, `update_phone`, `update_company`, `update_job_title` fields. The
+dispatch method reads `params.update_first_name or params.first_name` (etc.) as
+fallback chains — it does NOT mutate the params dict.
### `operation = delete`
@@ -75,10 +77,13 @@ Also accepts `update_first_name`, `update_last_name`, `update_email`,
## Outputs (handles)
+The node declares only `input-main` and `output-main`. Tool mode
+(`usable_as_tool = True`, tool name `google_contacts`) returns the same
+`output-main` payload — there is no separate `output-tool` handle.
+
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Formatted contact(s) payload |
-| `output-tool` | object | Same, for AI tool wiring |
+| `output-main` | object | Formatted contact(s) `ContactsOutput` payload |
### Formatted contact shape (`_format_contact`)
@@ -106,72 +111,57 @@ Also accepts `update_first_name`, `update_last_name`, `update_email`,
```mermaid
flowchart TD
- A[handle_google_contacts] --> AL{operation == update?}
- AL -- yes --> AM[Alias update_* params<br/>to plain names]
- AL -- no --> B
- AM --> B{operation?}
- B -- create --> C1[handle_contacts_create]
- B -- list --> C2[handle_contacts_list]
- B -- search --> C3[handle_contacts_search]
- B -- get --> C4[handle_contacts_get]
- B -- update --> C5[handle_contacts_update]
- B -- delete --> C6[handle_contacts_delete]
- B -- unknown --> Eret[success=false<br/>Unknown Contacts operation]
- C1 --> G[get_google_credentials + build people v1]
- C2 --> G
- C3 --> G
- C4 --> G
- C5 --> G
- C6 --> G
- G -- ValueError --> Eret
- C1 --> V1{first_name?}
- V1 -- no --> Eret
+ A[BaseNode.execute -> ContactsNode.dispatch] --> G[build_google_service people v1]
+ G --> B{operation?}
+ B -- create --> V1{first_name?}
+ V1 -- no --> Eret[raise RuntimeError]
V1 -- yes --> BB[Build names/emails/phones<br/>organizations/biographies]
BB --> R1[people.createContact]
- C2 --> R2[people.connections.list<br/>personFields names emails phones<br/>organizations photos biographies]
- C3 --> V3{query?}
+ B -- list --> R2[people.connections.list<br/>personFields names emails phones<br/>organizations photos biographies]
+ B -- search --> V3{query?}
V3 -- no --> Eret
V3 -- yes --> R3[people.searchContacts]
- C4 --> V4{resource_name?}
+ B -- get --> V4{resource_name?}
V4 -- no --> Eret
V4 -- yes --> R4[people.get]
- C5 --> V5{resource_name?}
+ B -- update --> V5{resource_name?}
V5 -- no --> Eret
- V5 -- yes --> G5[people.get etag -> build update body -> check updatePersonFields non-empty]
- G5 -- no fields --> Eret2[raise<br/>At least one field to update required]
+ V5 -- yes --> G5[people.get etag -> build update body via update_* fallbacks -> check fields]
+ G5 -- no fields --> Eret2[raise RuntimeError<br/>At least one field to update required]
G5 -- ok --> R5[people.updateContact updatePersonFields=csv]
- C6 --> V6{resource_name?}
+ B -- delete --> V6{resource_name?}
V6 -- no --> Eret
V6 -- yes --> R6[people.deleteContact]
- R1 --> T[_track_contacts_usage]
+ B -- unknown --> Eret
+ R1 --> T[track_google_usage google_contacts]
R2 --> T
R3 --> T
R4 --> T
R5 --> T
R6 --> T
- T --> OUT[Return success envelope]
+ T --> OUT[Return ContactsOutput; BaseNode serializes envelope]
```
## Decision Logic
-- **Dispatcher aliasing**: `update_*` params are copied over their plain counterparts (mutating input) before delegation.
-- **Update requires etag**: `update` does a `people.get` first to fetch the current etag, then submits `updateContact` with `updatePersonFields` set to the comma-joined list of person field groups touched. If no field group was modified, raises `ValueError("At least one field to update is required")`.
-- **Search requires query**: empty query -> `ValueError` captured by outer except.
+- **Update fallback chains**: `params.update_first_name or params.first_name` (etc.) — no mutation of the params dict.
+- **Update requires etag**: `update` does a `people.get` first to fetch the current etag, then submits `updateContact` with `updatePersonFields` set to the comma-joined list of person field groups touched. If no field group was modified, raises `RuntimeError("At least one field to update is required")`.
+- **Search requires query**: empty query -> `RuntimeError` surfaced by `BaseNode.execute`.
- **Notes field on create**: non-empty `notes` writes a `biographies[0]` entry; empty notes skip the field. Cannot be cleared via update (not wired).
- **Format fallbacks**: `_format_contact` returns `""` for missing headers rather than `None`, so callers should not distinguish absence from empty.
- **`list` resource_name** is hard-coded to `people/me` - this node cannot list another user's contacts.
## Side Effects
-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_contacts'`. `list` DOES call `_track_contacts_usage`.
-- **Broadcasts**: none.
+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_contacts'`. `list` DOES call `track_google_usage`.
+- **Broadcasts**: none from the operation; executor emits standard `node_status`.
- **External API calls**: People API v1 - `people().createContact/get/updateContact/deleteContact/searchContacts`, `people().connections().list`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.
+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.
- **Services**: Google People API, `PricingService`, `Database`.
- **Python packages**: `google-api-python-client`.
- **Environment variables**: none.
@@ -182,7 +172,6 @@ flowchart TD
- Update replaces entire field groups: passing `email` wipes any additional emails on the contact. The handler does not merge.
- `search` returns at most `page_size` results; there is no pagination wrapper.
- `delete` does not prompt or verify - the deletion is immediate and final on Google's side.
-- An unused local statement `person.get('addresses', [])` exists in `_format_contact` without assignment - noted as dead code, not a bug.
- `update` requires at least one patch field; however the handler only treats first_name/last_name/email/phone/company/job_title. Notes and other fields cannot be updated here.
## Related
diff --git a/docs-internal/node-logic-flows/google_workspace/googleDrive.md b/docs-internal/node-logic-flows/google_workspace/googleDrive.md
index a2a4300d..cc8e9e75 100644
--- a/docs-internal/node-logic-flows/google_workspace/googleDrive.md
+++ b/docs-internal/node-logic-flows/google_workspace/googleDrive.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | google_workspace / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/drive.py::handle_google_drive`](../../../server/services/handlers/drive.py) |
+| **Backend handler** | [`server/nodes/google/drive/__init__.py`](../../../server/nodes/google/drive/__init__.py) (`DriveNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) |
| **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) |
| **Skill (if any)** | [`server/skills/productivity_agent/google-drive-skill/SKILL.md`](../../../server/skills/productivity_agent/google-drive-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `googleDrive` |
+| **Dual-purpose tool** | yes - tool name `google_drive` |
## Purpose
@@ -65,10 +65,13 @@ Top-level dispatcher: `operation` (one of `upload`, `download`, `list`, `share`)
## Outputs (handles)
+The node declares only `input-main` and `output-main`. Tool mode
+(`usable_as_tool = True`, tool name `google_drive`) returns the same
+`output-main` payload — there is no separate `output-tool` handle.
+
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Operation-specific payload |
-| `output-tool` | object | Same, for AI tool wiring |
+| `output-main` | object | Operation-specific `DriveOutput` payload |
- `upload`: `{file_id, name, mime_type, size, web_link, download_link, created_time}`
- `download` (base64): `{file_id, name, mime_type, size, content_base64}`
@@ -80,17 +83,13 @@ Top-level dispatcher: `operation` (one of `upload`, `download`, `list`, `share`)
```mermaid
flowchart TD
- A[handle_google_drive] --> B{operation?}
- B -- upload --> U[handle_drive_upload]
- B -- download --> D[handle_drive_download]
- B -- list --> L[handle_drive_list]
- B -- share --> S[handle_drive_share]
- B -- unknown --> Eret[success=false<br/>Unknown Drive operation]
- U --> G[get_google_credentials + build v3]
- D --> G
- L --> G
- S --> G
- G -- ValueError --> Eret
+ A[BaseNode.execute -> DriveNode.dispatch] --> G[build_google_service drive v3]
+ G --> B{operation?}
+ B -- upload --> U[upload branch]
+ B -- download --> D[download branch]
+ B -- list --> L[list branch]
+ B -- share --> S[share branch]
+ B -- unknown --> Eret[raise RuntimeError]
U --> V1{filename &<br/>file_url OR file_content?}
V1 -- no --> Eret
V1 -- yes --> SRC{file_url set?}
@@ -110,12 +109,12 @@ flowchart TD
S --> V3{file_id & email?}
V3 -- no --> Eret
V3 -- yes --> R4[permissions.create +<br/>files.get for name/link]
- R1 --> T[_track_drive_usage]
+ R1 --> T[track_google_usage google_drive]
DL --> T
OUT --> T
R3 --> T
R4 --> T
- T --> OE[Return success envelope]
+ T --> OE[Return DriveOutput; BaseNode serializes envelope]
```
## Decision Logic
@@ -128,15 +127,15 @@ flowchart TD
## Side Effects
-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_drive'`.
-- **Broadcasts**: none.
+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_drive'`.
+- **Broadcasts**: none from the operation; executor emits standard `node_status`.
- **External API calls**: Drive API v3 - `files().create/get/list`, `permissions().create`, `files().get_media`. Plus `httpx.AsyncClient.get(file_url)` on upload-from-URL.
- **File I/O**: in-memory `io.BytesIO` buffers only; no disk writes.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.
+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.
- **Services**: Google Drive API, `PricingService`, `Database`.
- **Python packages**: `google-api-python-client`, `httpx`.
- **Environment variables**: none.
diff --git a/docs-internal/node-logic-flows/google_workspace/googleGmail.md b/docs-internal/node-logic-flows/google_workspace/googleGmail.md
index bb5b6e2d..fa275f3a 100644
--- a/docs-internal/node-logic-flows/google_workspace/googleGmail.md
+++ b/docs-internal/node-logic-flows/google_workspace/googleGmail.md
@@ -3,18 +3,18 @@
| Field | Value |
|------|-------|
| **Category** | google_workspace / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/gmail.py::handle_google_gmail`](../../../server/services/handlers/gmail.py) |
+| **Backend handler** | [`server/nodes/google/gmail/__init__.py`](../../../server/nodes/google/gmail/__init__.py) (`GmailNode`; dispatched via `BaseNode.execute()` -> per-operation `@Operation` methods `send` / `search` / `read`) |
| **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) |
| **Skill (if any)** | [`server/skills/productivity_agent/google-gmail-skill/SKILL.md`](../../../server/skills/productivity_agent/google-gmail-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `googleGmail` |
+| **Dual-purpose tool** | yes - tool name `google_gmail` |
## Purpose
Consolidated Gmail node covering send, search, and read operations. Uses the
Google Gmail API v1 via the official `google-api-python-client`. One node, three
-operations switched via the `operation` parameter. Authenticates through the
-shared `get_google_credentials()` helper that reads OAuth tokens stored via
-`auth_service.store_oauth_tokens("google", ...)`.
+operations, each its own `@Operation` method selected by the `operation` Param.
+Authenticates through `GoogleCredential.build_credentials()` (via
+`build_google_service`) which reads OAuth tokens stored under provider `google`.
## Inputs (handles)
@@ -52,76 +52,75 @@ Top-level dispatcher: `operation` (one of `send`, `search`, `read`).
| `message_id` | string | `""` | **yes** | Gmail message ID |
| `format` | options | `full` | no | `full` / `minimal` / `raw` / `metadata` |
-Shared: `account_mode` (default `owner`), `customer_id` (required when
-`account_mode == 'customer'`).
+Params model: `GmailParams` (`model_config extra="ignore"`). All non-`operation`
+fields carry `displayOptions.show` keyed on the active operation. There is no
+`account_mode` / `customer_id` field — credential scoping is resolved inside
+`GoogleCredential.build_credentials`.
## Outputs (handles)
+The node declares only `input-main` and `output-main`. When wired to an AI
+agent's `input-tools` handle (`usable_as_tool = True`, tool name `google_gmail`)
+the same `output-main` payload is returned to the LLM — there is no separate
+`output-tool` handle.
+
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Operation-specific payload (see below) |
-| `output-tool` | object | Same payload, used when wired to an AI agent `input-tools` |
+| `output-main` | object | Operation-specific `GmailOutput` payload (see below) |
-### Output payloads
+### Output payloads (`GmailOutput`, `model_config extra="allow"`)
-- `send`: `{message_id, thread_id, label_ids, to, subject}`
-- `search`: `{messages: [...], count, query, result_size_estimate}` where each
- message is `{message_id, thread_id, from, to, cc, subject, date, snippet,
- labels, size_estimate, body?, attachments?}`
-- `read`: formatted message (same shape as a single entry in `search`)
+- `send`: `{operation: "send", message_id, thread_id, label_ids, to, subject}`
+- `search`: `{operation: "search", messages: [...], count, query, result_size_estimate}`
+ where each message comes from `format_message`
+- `read`: `{operation: "read", ...format_message(result)}` (the `size_estimate`
+ key is stripped out)
## Logic Flow
```mermaid
flowchart TD
- A[handle_google_gmail] --> B{operation?}
- B -- send --> S1[handle_gmail_send]
- B -- search --> S2[handle_gmail_search]
- B -- read --> S3[handle_gmail_read]
- B -- unknown --> Eret[Return success=false<br/>Unknown Gmail operation: X]
- S1 --> C1[get_google_credentials]
- S2 --> C2[get_google_credentials]
- S3 --> C3[get_google_credentials]
- C1 -- ValueError --> Eret
- C2 -- ValueError --> Eret
- C3 -- ValueError --> Eret
- C1 --> V1{to/subject/body<br/>all present?}
- V1 -- no --> Eret
+ A[BaseNode.execute -> @Operation method] --> B{operation?}
+ B -- send --> S1[GmailNode.send]
+ B -- search --> S2[GmailNode.search]
+ B -- read --> S3[GmailNode.read]
+ S1 --> V1{to/subject/body<br/>all present?}
+ V1 -- no --> Eret[raise RuntimeError]
V1 -- yes --> B1[Build MIMEText/MIMEMultipart<br/>base64 urlsafe encode]
- B1 --> R1[service.users.messages.send<br/>run_in_executor]
- C2 --> V2{query non-empty?}
+ B1 --> SVC1[_service -> build_google_service gmail v1]
+ SVC1 --> R1[users.messages.send via run_sync]
+ S2 --> V2{query non-empty?}
V2 -- no --> Eret
- V2 -- yes --> R2[service.users.messages.list q=query]
+ V2 -- yes --> R2[users.messages.list q=query]
R2 --> L2[Loop: users.messages.get per id<br/>format=full if include_body else metadata]
- C3 --> V3{message_id?}
+ S3 --> V3{message_id?}
V3 -- no --> Eret
- V3 -- yes --> R3[service.users.messages.get id=message_id]
- R1 --> T[_track_gmail_usage -> save_api_usage_metric]
+ V3 -- yes --> R3[users.messages.get id=message_id]
+ R1 --> T[track_google_usage gmail -> save_api_usage_metric]
L2 --> T
R3 --> T
- T --> OUT[Return success envelope]
+ T --> OUT[Return GmailOutput; BaseNode.execute serializes envelope]
```
## Decision Logic
-- **Operation dispatch**: `if/elif` chain in `handle_google_gmail`. Unknown operation returns an error envelope (does NOT raise).
-- **Credential resolution** (`get_google_credentials`): owner mode reads from `auth_service.get_oauth_tokens("google", customer_id="owner")`; customer mode reads from `google_connections` table. Raises `ValueError` if no tokens found.
-- **Proactive refresh**: if the Credentials object reports `expired and not valid` and a `refresh_token` is present, a refresh is attempted in a thread executor; result is persisted via `auth_service.store_oauth_tokens`. Refresh failures are swallowed to DEBUG.
-- **Search body fetching**: `include_body=False` uses `format=metadata` with a fixed header list (`From`, `To`, `Subject`, `Date`); `True` uses `format=full`. Body extraction recurses into multipart payloads; prefers `text/plain` over `text/html`.
+- **Operation dispatch**: the `@Operation`-decorated method (`send` / `search` / `read`) is selected by the framework from `params.operation`. Missing-field validation in each method raises `RuntimeError` (surfaced by `BaseNode.execute` as an error envelope).
+- **Credential resolution**: `build_google_service` -> `GoogleCredential.build_credentials(params, ctx.raw)` reads stored OAuth tokens for provider `google` (owner / customer scoping handled inside the credential class). The blocking `build(...)` runs in an executor.
+- **Search body fetching**: `include_body=False` uses `format=metadata` with a fixed header list (`From`, `To`, `Subject`, `Date`); `True` uses `format=full`. `format_message` (in `_gmail.py`) extracts the body.
- **Send body type**: `body_type=='html'` -> `MIMEMultipart('alternative')` with an HTML part; otherwise `MIMEText(body, 'plain')`. Message is `urlsafe_b64encode`d to base64.
-- **Error paths**: every sub-handler wraps the whole body in `try/except Exception` and returns `{success: false, error: str(e), execution_time: ...}`.
+- **Error paths**: each method raises `RuntimeError` for missing required fields; `BaseNode.execute()` wraps the body in `log_context` + a span and converts exceptions into the standard error envelope.
## Side Effects
-- **Database writes**: one row per API call in `api_usage_metrics` via `database.save_api_usage_metric` with `service='googleGmail'`, `operation=<action>`, `endpoint=<action>`, `resource_count`, `cost` (always 0 for Google APIs).
-- **Broadcasts**: none from the handler itself; `NodeExecutor` emits standard `node_status`.
-- **External API calls**: Gmail API v1 via `googleapiclient.discovery.build("googleGmail", "v1", creds)` - `users().messages().send/list/get`. Proactive token refresh hits `https://oauth2.googleapis.com/token`.
+- **Database writes**: one row per API call in `api_usage_metrics` via `track_google_usage` (`_base.py`) -> `database.save_api_usage_metric` with `service='gmail'`, `endpoint=<action>`, `resource_count`, `cost` (always 0 for Google APIs).
+- **Broadcasts**: none from the operation itself; `BaseNode.execute()` / the executor emit standard `node_status`.
+- **External API calls**: Gmail API v1 via `googleapiclient.discovery.build("gmail", "v1", creds)` - `users().messages().send/list/get`. Token refresh hits `https://oauth2.googleapis.com/token`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: OAuth tokens via `auth_service.get_oauth_tokens("google", customer_id="owner")`; `google_client_id` / `google_client_secret` via `auth_service.get_api_key(...)`.
+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google` (`access_token` / `refresh_token`); `google_client_id` / `google_client_secret` via the API-key store.
- **Services**: Google Gmail API, `PricingService`, `Database`.
- **Python packages**: `google-auth`, `google-auth-oauthlib`, `google-api-python-client`.
- **Environment variables**: none (OAuth redirect URI derived from request at auth time).
@@ -130,8 +129,7 @@ flowchart TD
- `max_results` is silently clamped to 100 (hard ceiling).
- Search fetches every matching message individually in sequence - with `max_results=100` this is 100 round trips. No batching.
-- The proactive refresh path swallows exceptions - if the refresh token has been revoked the handler continues and subsequently fails with a 401-shaped error surfaced as `"<status>"` in the envelope.
-- `_format_message` returns empty strings for missing headers rather than omitting them; callers should not treat `""` as "header was present and empty".
+- `format_message` (in `_gmail.py`) returns empty strings for missing headers rather than omitting them; callers should not treat `""` as "header was present and empty".
- `body` is returned as a UTF-8 decoded string with `errors='ignore'` - binary parts may produce partial output without raising.
## Related
diff --git a/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.md b/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.md
index 000341b1..7b40e72e 100644
--- a/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.md
+++ b/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | google_workspace / trigger (polling) |
-| **Backend handler** | [`server/services/handlers/gmail.py::handle_gmail_receive`](../../../server/services/handlers/gmail.py) |
+| **Backend handler** | [`server/nodes/google/gmail_receive/__init__.py`](../../../server/nodes/google/gmail_receive/__init__.py) (`GmailReceiveNode`, a `PollingTriggerNode`; inline-Run path uses the `execute()` override, deployment path uses the `setup_service` / `fetch_ids` / `fetch_detail` / `post_emit` hooks driven by `PollingTriggerWorkflow`) |
| **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -12,13 +12,18 @@
Polling-based trigger that fires when a new Gmail message matching a filter
query arrives. Unlike push-based triggers (webhook, WhatsApp, Telegram) Gmail
-has no official push without extra Pub/Sub setup, so this handler polls
+has no official push without extra Pub/Sub setup, so this node polls
`users.messages.list` at a configurable interval, diffs IDs against a
baseline, fetches full details for new messages, and optionally marks them
as read.
-Dispatches a `gmail_email_received` event via `event_waiter.dispatch` so
-deployment-mode listeners can react in parallel.
+`event_type = "gmail_email_received"`. In deployment (canary) mode the
+`PollingTriggerWorkflow` owns event fan-out directly: it calls the per-cycle
+activity built from `PollingTriggerNode.as_poll_activity()` and spawns the
+child `MachinaWorkflow` per new email — there is no `event_waiter.dispatch`
+step. The `_events.py` `gmail_message_received` CloudEvents factory is kept for
+parity / future ad-hoc emits; the legacy `dispatch_gmail_received` shim was
+removed in Wave 13.
## Inputs (handles)
diff --git a/docs-internal/node-logic-flows/google_workspace/googleSheets.md b/docs-internal/node-logic-flows/google_workspace/googleSheets.md
index 419e8e0d..28275448 100644
--- a/docs-internal/node-logic-flows/google_workspace/googleSheets.md
+++ b/docs-internal/node-logic-flows/google_workspace/googleSheets.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | google_workspace / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/sheets.py::handle_google_sheets`](../../../server/services/handlers/sheets.py) |
+| **Backend handler** | [`server/nodes/google/sheets/__init__.py`](../../../server/nodes/google/sheets/__init__.py) (`SheetsNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) |
| **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) |
| **Skill (if any)** | [`server/skills/productivity_agent/google-sheets-skill/SKILL.md`](../../../server/skills/productivity_agent/google-sheets-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `googleSheets` |
+| **Dual-purpose tool** | yes - tool name `google_sheets` |
## Purpose
@@ -29,7 +29,7 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).
| Name | Type | Default | Required | Description |
|------|------|---------|----------|-------------|
| `spreadsheet_id` | string | `""` | **yes** | Sheet ID (from URL) |
-| `range` | string | `""` | **yes** | A1 notation, e.g. `Sheet1!A1:D10` |
+| `range` | string | `A1:Z1000` | **yes** | A1 notation, e.g. `Sheet1!A1:D10` |
| `value_render_option` | options | `FORMATTED_VALUE` | no | `FORMATTED_VALUE` / `UNFORMATTED_VALUE` / `FORMULA` |
| `major_dimension` | options | `ROWS` | no | `ROWS` / `COLUMNS` |
@@ -38,7 +38,7 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).
| Name | Type | Default | Required | Description |
|------|------|---------|----------|-------------|
| `spreadsheet_id` | string | `""` | **yes** | - |
-| `range` | string | `""` | **yes** | e.g. `Sheet1!A1` |
+| `range` | string | `A1:Z1000` | **yes** | e.g. `Sheet1!A1` |
| `values` | array/string | `[]` | **yes** | 2D array, or JSON string that parses to 2D array, or 1D auto-wrapped to 2D |
| `value_input_option` | options | `USER_ENTERED` | no | `RAW` or `USER_ENTERED` |
@@ -47,17 +47,20 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).
| Name | Type | Default | Required | Description |
|------|------|---------|----------|-------------|
| `spreadsheet_id` | string | `""` | **yes** | - |
-| `range` | string | `""` | **yes** | e.g. `Sheet1!A:D` |
+| `range` | string | `A1:Z1000` | **yes** | e.g. `Sheet1!A:D` |
| `values` | array/string | `[]` | **yes** | Same coercion as write |
| `value_input_option` | options | `USER_ENTERED` | no | - |
| `insert_data_option` | options | `INSERT_ROWS` | no | `INSERT_ROWS` / `OVERWRITE` |
## Outputs (handles)
+The node declares only `input-main` and `output-main`. Tool mode
+(`usable_as_tool = True`, tool name `google_sheets`) returns the same
+`output-main` payload — there is no separate `output-tool` handle.
+
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Operation-specific payload |
-| `output-tool` | object | Same, for AI tool wiring |
+| `output-main` | object | Operation-specific `SheetsOutput` payload |
- `read`: `{values: [[...],...], range, rows, columns, major_dimension}`
- `write` / `append`: `{updated_range, updated_rows, updated_columns, updated_cells, table_range?}`
@@ -66,50 +69,44 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).
```mermaid
flowchart TD
- A[handle_google_sheets] --> B{operation?}
- B -- read --> R[handle_sheets_read]
- B -- write --> W[handle_sheets_write]
- B -- append --> AP[handle_sheets_append]
- B -- unknown --> Eret[success=false<br/>Unknown Sheets operation]
- R --> G[get_google_credentials + build v4]
- W --> G
- AP --> G
- G -- ValueError --> Eret
- R --> V1{spreadsheet_id & range?}
- V1 -- no --> Eret
- V1 -- yes --> R1[spreadsheets.values.get<br/>valueRenderOption majorDimension]
- W --> V2{spreadsheet_id & range & values?}
- V2 -- no --> Eret
- V2 -- yes --> C2[Coerce values:<br/>str -> json.loads<br/>1D -> wrap to 2D]
- C2 --> R2[spreadsheets.values.update<br/>valueInputOption body]
- AP --> V3{spreadsheet_id & range & values?}
- V3 -- no --> Eret
- V3 -- yes --> C3[Same coercion as write]
- C3 --> R3[spreadsheets.values.append<br/>valueInputOption insertDataOption]
- R1 --> T[_track_sheets_usage]
+ A[BaseNode.execute -> SheetsNode.dispatch] --> V0{spreadsheet_id & range present?}
+ V0 -- no --> Eret[raise RuntimeError]
+ V0 -- yes --> G[build_google_service sheets v4]
+ G --> B{operation?}
+ B -- read --> R1[spreadsheets.values.get<br/>valueRenderOption majorDimension]
+ B -- write --> CW[Coerce values:<br/>str -> json.loads, 1D -> wrap to 2D]
+ CW --> VW{values non-empty?}
+ VW -- no --> Eret
+ VW -- yes --> R2[spreadsheets.values.update<br/>valueInputOption body]
+ B -- append --> CA[Same coercion as write]
+ CA --> VA{values non-empty?}
+ VA -- no --> Eret
+ VA -- yes --> R3[spreadsheets.values.append<br/>valueInputOption insertDataOption]
+ B -- unknown --> Eret
+ R1 --> T[track_google_usage google_sheets]
R2 --> T
R3 --> T
- T --> OUT[Return success envelope]
+ T --> OUT[Return SheetsOutput; BaseNode serializes envelope]
```
## Decision Logic
- **Values coercion**: if `values` is a `str`, parsed via `json.loads`. If the (possibly parsed) sequence's first element is not a list, the whole thing is wrapped into `[values]` to guarantee 2D.
-- **Required fields**: each operation has its own validation block; all three require `spreadsheet_id` and `range`, write/append additionally require a non-empty `values`.
+- **Required fields**: `spreadsheet_id` and `range` are validated once at the top of `dispatch` (before the operation branch); write/append additionally require a non-empty `values`.
- **Usage tracking count**: read reports `len(values)` rows; write reports `updatedCells`; append reports `updates.updatedCells`. If any field is missing in the response, usage count is 0 (no exception).
- **JSON parse errors** on `values` propagate into the outer `except Exception` and become `error: "<ValueError str>"`.
## Side Effects
-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_sheets'`.
-- **Broadcasts**: none.
+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_sheets'`.
+- **Broadcasts**: none from the operation; executor emits standard `node_status`.
- **External API calls**: Sheets API v4 - `spreadsheets().values().get/update/append`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.
+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.
- **Services**: Google Sheets API, `PricingService`, `Database`.
- **Python packages**: `google-api-python-client`.
- **Environment variables**: none.
diff --git a/docs-internal/node-logic-flows/google_workspace/googleTasks.md b/docs-internal/node-logic-flows/google_workspace/googleTasks.md
index 08292cb5..6e34a58f 100644
--- a/docs-internal/node-logic-flows/google_workspace/googleTasks.md
+++ b/docs-internal/node-logic-flows/google_workspace/googleTasks.md
@@ -3,15 +3,15 @@
| Field | Value |
|------|-------|
| **Category** | google_workspace / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/tasks.py::handle_google_tasks`](../../../server/services/handlers/tasks.py) |
+| **Backend handler** | [`server/nodes/google/tasks/__init__.py`](../../../server/nodes/google/tasks/__init__.py) (`TasksNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) |
| **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) |
| **Skill (if any)** | [`server/skills/productivity_agent/google-tasks-skill/SKILL.md`](../../../server/skills/productivity_agent/google-tasks-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `googleTasks` |
+| **Dual-purpose tool** | yes - tool name `google_tasks` |
## Purpose
Consolidated Google Tasks node for managing personal to-do items. Uses Google
-Tasks API v1 (`tasklists` + `googleTasks` collections). One node, five operations
+Tasks API v1 (`tasklists` + `tasks` collections). One node, five operations
switched via the `operation` parameter.
## Inputs (handles)
@@ -28,10 +28,11 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `complete`, `update`
| Name | Type | Default | Required | Description |
|------|------|---------|----------|-------------|
-| `title` | string | `""` | **yes** | Task title |
-| `notes` | string | `""` | no | Task description |
-| `due_date` | string | `""` | no | RFC 3339; date-only is upgraded to `T00:00:00.000Z` |
-| `tasklist_id` | string | `@default` | no | Which tasklist |
+| `title` | string | `null` | **yes** | Task title |
+| `notes` | string | `null` | no | Task description |
+| `due_date` | string | `null` | no | RFC 3339; date-only is upgraded to `T00:00:00.000Z` |
+| `status` | string | `null` | no | Task status (Param exists; not written by the `create` branch) |
+| `tasklist_id` | string | `@default` | no | Which tasklist (`loadOptionsMethod: googleTasklists`) |
### `operation = list`
@@ -57,8 +58,9 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `complete`, `update`
| `tasklist_id` | string | `@default` | no | - |
| `title` / `notes` / `due_date` / `status` | string | `""` | no | Patch fields |
-Also accepts `update_title`, `update_notes`, `update_due_date`, `update_status`
-aliases (the dispatcher rewrites them before delegating).
+Also accepts the optional `update_title`, `update_notes`, `update_due_date`,
+`update_status` fields. The dispatch method reads `params.update_title or
+params.title` (etc.) as fallback chains — it does NOT mutate the params dict.
### `operation = delete`
@@ -69,10 +71,13 @@ aliases (the dispatcher rewrites them before delegating).
## Outputs (handles)
+The node declares only `input-main` and `output-main`. Tool mode
+(`usable_as_tool = True`, tool name `google_tasks`) returns the same
+`output-main` payload — there is no separate `output-tool` handle.
+
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Operation-specific payload |
-| `output-tool` | object | Same, for AI tool wiring |
+| `output-main` | object | Operation-specific `TasksOutput` payload |
- `create` / `complete` / `update`: `{task_id, title, notes?, due, status, completed?, self_link?}`
- `list`: `{tasks: [{task_id, title, notes, due, status, completed, position}], count}`
@@ -82,73 +87,59 @@ aliases (the dispatcher rewrites them before delegating).
```mermaid
flowchart TD
- A[handle_google_tasks] --> AL{operation == update?}
- AL -- yes --> AM[Alias update_* params<br/>to plain names]
- AL -- no --> B
- AM --> B{operation?}
- B -- create --> C1[handle_tasks_create]
- B -- list --> C2[handle_tasks_list]
- B -- complete --> C3[handle_tasks_complete]
- B -- update --> C4[handle_tasks_update]
- B -- delete --> C5[handle_tasks_delete]
- B -- unknown --> Eret[success=false<br/>Unknown Tasks operation]
- C1 --> G[get_google_credentials + build v1]
- C2 --> G
- C3 --> G
- C4 --> G
- C5 --> G
- G -- ValueError --> Eret
- C1 --> V1{title?}
- V1 -- no --> Eret
+ A[BaseNode.execute -> TasksNode.dispatch] --> G[build_google_service tasks v1]
+ G --> B{operation?}
+ B -- create --> V1{title?}
+ V1 -- no --> Eret[raise RuntimeError]
V1 -- yes --> DF[Normalize due_date<br/>append T00:00:00.000Z if missing]
DF --> R1[tasks.insert]
- C2 --> R2[tasks.list]
- C3 --> V3{task_id?}
+ B -- list --> R2[tasks.list]
+ B -- complete --> V3{task_id?}
V3 -- no --> Eret
V3 -- yes --> G3[tasks.get -> set status=completed -> tasks.update]
- C4 --> V4{task_id?}
+ B -- update --> V4{task_id?}
V4 -- no --> Eret
- V4 -- yes --> G4[tasks.get -> merge patch -> tasks.update]
- C5 --> V5{task_id?}
+ V4 -- yes --> G4[tasks.get -> merge update_* fallbacks -> tasks.update]
+ B -- delete --> V5{task_id?}
V5 -- no --> Eret
V5 -- yes --> R5[tasks.delete]
- R1 --> T[_track_tasks_usage]
+ B -- unknown --> Eret
+ R1 --> T[track_google_usage google_tasks]
R2 --> T
G3 --> T
G4 --> T
R5 --> T
- T --> OUT[Return success envelope]
+ T --> OUT[Return TasksOutput; BaseNode serializes envelope]
```
## Decision Logic
-- **Update parameter aliasing** (dispatcher): the dispatcher mutates the incoming `parameters` dict to promote `update_*` keys onto their plain counterparts BEFORE delegating. Irreversible.
+- **Operation branch**: one `@Operation("dispatch")` method branches on `params.operation`; `complete`/`update`/`delete` share a `task_id`-required block.
+- **Update fallback chains**: `params.update_title or params.title` (etc.) — no mutation of the params dict; either field works.
- **Due-date upgrade**: any `due_date` that lacks a `T` gets `T00:00:00.000Z` appended. There is no strict ISO validation otherwise.
- **Complete** is a convenience op - it does a read-modify-write (`tasks.get` then `tasks.update`) with `status='completed'`. Two API calls per completion.
-- **Update** merges only truthy fields (uses `parameters.get('title')` etc. which skips empty strings). Callers cannot clear `notes` via the update path - use an explicit `status='needsAction'` or direct API.
+- **Update** merges only truthy fields (skips empty strings). Callers cannot clear `notes` via the update path.
- **`tasks.list`** does NOT reject `max_results > 100` - the API itself caps silently.
## Side Effects
-- **Database writes**: `api_usage_metrics` row per API call via `save_api_usage_metric` with `service='google_tasks'`. `list` records one row with `resource_count = len(tasks)`, but `_track_tasks_usage` is NOT called on the list path (see bug below).
-- **Broadcasts**: none.
+- **Database writes**: `api_usage_metrics` row per API call via `track_google_usage` -> `save_api_usage_metric` with `service='google_tasks'`. The `list` branch DOES call `track_google_usage` (`resource_count = len(tasks)`).
+- **Broadcasts**: none from the operation; executor emits standard `node_status`.
- **External API calls**: Tasks API v1 - `tasks().insert/list/get/update/delete`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.
+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.
- **Services**: Google Tasks API, `PricingService`, `Database`.
- **Python packages**: `google-api-python-client`.
- **Environment variables**: none.
## Edge cases & known limits
-- **`handle_tasks_list` does not call `_track_tasks_usage`** (all other operations do). The list operation therefore skips the `api_usage_metrics` row. Handler bug noted, not fixed.
- Update cannot clear a field - empty strings are filtered out by truthiness. To clear notes or title you must bypass this node.
- Complete is non-idempotent on the server side: re-completing a completed task overwrites the `completed` timestamp.
-- Dispatcher mutates `parameters` in place - callers that reuse the dict will see `title`/`notes`/`due_date`/`status` filled in from `update_*` aliases.
- `tasklist_id='@default'` only resolves for authenticated users who have created at least one list; freshly authenticated users may get a 404 until Google Tasks provisions their default list.
## Related
diff --git a/docs-internal/node-logic-flows/http_proxy/httpRequest.md b/docs-internal/node-logic-flows/http_proxy/httpRequest.md
index e49caf72..6af9872c 100644
--- a/docs-internal/node-logic-flows/http_proxy/httpRequest.md
+++ b/docs-internal/node-logic-flows/http_proxy/httpRequest.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | utility / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/http.py::handle_http_request`](../../../server/services/handlers/http.py) |
+| **Backend handler** | [`server/nodes/utility/http_request/__init__.py`](../../../server/nodes/utility/http_request/__init__.py) — `HttpRequestNode.request` (dispatched via `BaseNode.execute()` + `@Operation("request")`; the legacy `handlers/http.py` was deleted in Wave 11.D.3) |
| **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) |
| **Skill (if any)** | [`server/skills/web_agent/http-request-skill/SKILL.md`](../../../server/skills/web_agent/http-request-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - exposed to agents as a generic HTTP tool |
+| **Dual-purpose tool** | yes - tool name `http_request` (`usable_as_tool = True`) |
## Purpose
@@ -26,14 +26,15 @@ configured residential proxy provider via `ProxyService`.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `method` | options | `GET` | no | - | One of `GET` / `POST` / `PUT` / `DELETE` / `PATCH` |
-| `url` | string | `""` | **yes** | - | Target URL; empty value short-circuits with `URL is required` |
-| `headers` | string (JSON) | `"{}"` | no | - | JSON object; invalid JSON silently falls back to `{}` |
-| `body` | string | `""` | no | `method in [POST, PUT, PATCH]` | Sent as JSON when parseable, else as raw content |
-| `timeout` | number | `30` | no | - | Seconds; coerced via `float(...)` |
-| `useProxy` | boolean | `false` | no | - | Route through `ProxyService.get_proxy_url` |
-| `proxyProvider` | string | `""` | no | `useProxy=true` | Specific provider name (empty -> auto-select) |
-| `proxyCountry` | string | `""` | no | `useProxy=true` | ISO country code for geo-targeting |
-| `sessionType` | options | `rotating` | no | `useProxy=true` | `rotating` or `sticky` |
+| `url` | string | `""` | **yes** | - | Target URL; empty value raises `RuntimeError('URL is required')` |
+| `headers` | object (`Dict[str,str]`) | `{}` | no | - | Request headers passed straight to httpx |
+| `body` | `Optional[Any]` | `null` | no | `method in [POST, PUT, PATCH]` | JSON object or raw string; ignored for GET/DELETE. String parsed as JSON when possible, else sent as raw content |
+| `timeout` | number | `30` | no | - | Seconds (1-600); coerced via `float(...)` |
+| `use_proxy` | boolean | `false` | no | - | Route through `ProxyService.get_proxy_url` |
+| `proxy_provider` | string | `auto` | no | `use_proxy=true` | Specific provider name (`auto` selects by health score) |
+| `proxy_country` | string | `""` | no | `use_proxy=true` | ISO country code for geo-targeting |
+| `session_type` | options | `rotating` | no | `use_proxy=true` | `rotating` or `sticky` |
+| `sticky_duration` | number | `600` | no | `use_proxy=true` and `session_type=sticky` | Sticky session duration in seconds |
## Outputs (handles)
@@ -54,50 +55,48 @@ configured residential proxy provider via `ProxyService`.
}
```
-Wrapped in the standard envelope: `{ success: <status<400>, node_id, node_type: 'httpRequest', result, execution_time, timestamp }`.
+The `request` op returns this dict directly; `BaseNode.execute()` wraps it in the standard success envelope. A status `>= 400` raises `RuntimeError(f"HTTP {status}: ...")` instead of returning, so it lands in the error envelope. The declared `HttpRequestOutput` model only pins `status` / `headers` / `body` (`extra="allow"`), so the extra `data` / `url` / `method` / `proxied` keys pass through unvalidated.
## Logic Flow
```mermaid
flowchart TD
- A[handle_http_request] --> B[Read method/url/headers/body/timeout]
+ A[HttpRequestNode.request] --> B[Read params method/url/headers/body/timeout]
B --> C{url truthy?}
- C -- no --> Eerr[Return success=false<br/>error: URL is required]
- C -- yes --> D[Parse headers JSON<br/>fallback {} on JSONDecodeError]
- D --> E{useProxy?}
- E -- no --> G[httpx.AsyncClient timeout=timeout]
- E -- yes --> F[_get_proxy_url_if_enabled<br/>-> proxy_svc.get_proxy_url]
- F -- returns None --> G
- F -- returns url --> G2[httpx.AsyncClient proxy=proxy_url timeout=timeout]
- F -- raises --> G3[Log warning, proceed without proxy -> G]
- G --> H{method in POST/PUT/PATCH<br/>and body truthy?}
+ C -- no --> Eerr[raise RuntimeError<br/>URL is required]
+ C -- yes --> D[_resolve_proxy_url use_proxy params]
+ D -- use_proxy false --> G[httpx.AsyncClient timeout=timeout]
+ D -- returns url --> G2[httpx.AsyncClient proxy=proxy_url timeout=timeout]
+ D -- service off / raises --> G3[Log warning, proceed without proxy -> G]
+ G --> H{method in POST/PUT/PATCH<br/>and body is not None?}
G2 --> H
G3 --> H
- H -- yes JSON parseable --> I[kwargs.json = parsed]
- H -- yes non-JSON --> I2[kwargs.content = body]
+ H -- yes str JSON parseable --> I[kwargs.json = parsed]
+ H -- yes str non-JSON --> I2[kwargs.content = body]
+ H -- yes non-str --> I4[kwargs.json = body]
H -- no --> I3[no body]
I --> J[await client.request]
I2 --> J
I3 --> J
- J -- response --> K[response.json fallback text]
- K --> L[Return success=(status<400)<br/>with status/data/headers/url/method/proxied]
- J -- TimeoutException --> T[Return success=false<br/>error: Request timed out]
- J -- Exception --> Z[Return success=false<br/>error: str(e)]
+ I4 --> J
+ J --> K[response.json fallback text]
+ K --> L{status >= 400?}
+ L -- yes --> M[raise RuntimeError HTTP status]
+ L -- no --> N[return dict status/data/headers/url/method/proxied]
```
## Decision Logic
-- **Validation**: `url` empty -> raises `ValueError('URL is required')` caught by outer `try`.
+- **Validation**: `url` empty -> raises `RuntimeError('URL is required')` (surfaced via the error envelope by `BaseNode.execute()`).
- **Branches**:
- - `useProxy=true` -> attempt proxy lookup; swallow proxy errors and fall back to direct call.
- - Body handling depends on `method` and JSON-parseability of `body`.
+ - `use_proxy=true` -> `_resolve_proxy_url` attempts a lookup; proxy errors are swallowed and the request falls back to a direct call.
+ - Body handling depends on `method` and the type / JSON-parseability of `body` (str-JSON -> `json`, str-non-JSON -> `content`, non-str -> `json`).
- **Fallbacks**:
- - Invalid headers JSON -> `{}`.
- - Proxy service disabled / `get_proxy_url` raises -> proceed without proxy.
+ - Proxy service disabled / not enabled / `get_proxy_url` raises -> proceed without proxy (returns `None`).
- Non-JSON response body -> plain `text` string in `data`.
- **Error paths**:
- - `TimeoutException` (class name match) -> user-friendly `Request timed out after N seconds`.
- - Any other exception -> `str(e)` verbatim in envelope.
+ - Status `>= 400` -> `raise RuntimeError(f"HTTP {status}: {body!r}")`.
+ - Any httpx exception (timeout, connection error, etc.) propagates uncaught; `BaseNode.execute()` produces the error envelope with the traceback. There is no timeout special-casing.
## Side Effects
@@ -116,10 +115,9 @@ flowchart TD
## Edge cases & known limits
-- Proxy lookup errors are **swallowed** (`_get_proxy_url_if_enabled` logs a warning and returns `None`); the request then proceeds directly, silently losing the proxy. Downstream nodes can detect this by checking `result.proxied`.
-- Timeout detection is by `type(e).__name__ == 'TimeoutException'` (string match) rather than `isinstance`; subclass hierarchy changes in httpx could bypass this branch.
-- `success` in the envelope reflects `status < 400`, so non-2xx 3xx responses (e.g. unfollowed 302) count as success.
-- The handler always wraps with `execution_time = time.time() - start_time`; no caching, no retry (retry is only implemented in `proxyRequest`).
+- Proxy lookup errors are **swallowed** (`_resolve_proxy_url` logs a warning and returns `None`); the request then proceeds directly, silently losing the proxy. Downstream nodes can detect this by checking `result.proxied`.
+- A `4xx`/`5xx` response raises `RuntimeError` rather than returning the body, so the error envelope carries the status; `3xx` are followed by httpx default and only surface if final status is `>= 400`.
+- No caching, no retry (retry / failover is only implemented in `proxyRequest`).
- `response.url` (final URL) is stringified; for non-proxied requests with redirects disabled this equals the request URL.
## Related
diff --git a/docs-internal/node-logic-flows/http_proxy/proxyConfig.md b/docs-internal/node-logic-flows/http_proxy/proxyConfig.md
index 48b988bf..99cc7ead 100644
--- a/docs-internal/node-logic-flows/http_proxy/proxyConfig.md
+++ b/docs-internal/node-logic-flows/http_proxy/proxyConfig.md
@@ -3,17 +3,18 @@
| Field | Value |
|------|-------|
| **Category** | proxy / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/proxy.py::handle_proxy_config`](../../../server/services/handlers/proxy.py) (delegates to [`services/handlers/tools.py::_execute_proxy_config`](../../../server/services/handlers/tools.py)) |
+| **Backend handler** | [`server/nodes/proxy/proxy_config/__init__.py`](../../../server/nodes/proxy/proxy_config/__init__.py) — `ProxyConfigNode.dispatch` (via `BaseNode.execute()` + `@Operation("dispatch")`) calls the module-level `execute_proxy_config(...)` 10-op dispatcher. The AI-tool fast path in [`server/services/handlers/tools.py`](../../../server/services/handlers/tools.py) imports the same `execute_proxy_config`. The legacy `handlers/proxy.py` was deleted in Wave 11. |
| **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) |
| **Skill (if any)** | [`server/skills/web_agent/proxy-config-skill/SKILL.md`](../../../server/skills/web_agent/proxy-config-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `proxy_config` |
+| **Dual-purpose tool** | yes - tool name `proxy_config` (`destructive = True`, `usable_as_tool = True`) |
## Purpose
Admin-style CRUD node for managing proxy providers, credentials, and routing
rules. Dispatches on the `operation` param. Works the same as both a workflow
-node and an AI-agent tool: the node handler simply forwards its parameters as
-both `args` and `node_params` to the tool handler `_execute_proxy_config`.
+node and an AI-agent tool: `ProxyConfigNode.dispatch` calls the module-level
+`execute_proxy_config(params.model_dump())`, and the AI-tool fast path in
+`tools.py` imports the same function — there is no separate handler shim.
## Inputs (handles)
@@ -31,15 +32,15 @@ both `args` and `node_params` to the tool handler `_execute_proxy_config`.
| `gateway_port` | number | `0` | no | `operation in [add/update]` | Proxy port |
| `url_template` | string (JSON) | `"{}"` | no | `operation in [add/update]` | JSON template config |
| `cost_per_gb` | number | `0` | no | `operation in [add/update]` | USD per GB |
-| `priority` | number | `50` | no | `operation in [add/update]` | Lower = preferred |
-| `enabled` | boolean | `true` | no | `operation in [add/update]` | Provider enabled |
+| `priority` | number | `50` | no | `operation in [add/update]` | Selection priority 0-100, higher ranks first |
+| `enabled` | `Optional[bool]` | `null` | no | `operation in [add/update]` | Whether the provider is active (None = unchanged on update) |
| `username` | string | `""` | yes (op=set_credentials) | `operation=set_credentials` | Proxy username |
-| `password` | string | `""` | yes (op=set_credentials) | `operation=set_credentials` | Proxy password |
-| `domain_pattern` | string | `""` | yes (op=add_routing_rule) | `operation=add_routing_rule` | Glob like `*.linkedin.com` |
-| `preferred_providers` | string (JSON array) | `"[]"` | no | `operation=add_routing_rule` | `["provider1"]` |
+| `password` | string (password widget) | `""` | yes (op=set_credentials) | `operation=set_credentials` | Proxy password |
+| `domain_pattern` | string | `""` | yes (op=add_routing_rule) | `operation=add_routing_rule` | Glob like `*.example.com` |
+| `preferred_providers` | `Any` (JSON array string) | `"[]"` | no | `operation=add_routing_rule` | `["provider1"]` |
| `required_country` | string | `""` | no | `operation=add_routing_rule` | ISO country |
| `session_type` | options | `rotating` | no | `operation=add_routing_rule` | `rotating` or `sticky` |
-| `rule_id` | number | `0` | yes (op=remove_routing_rule) | `operation=remove_routing_rule` | Rule PK |
+| `rule_id` | `Optional[int]` | `null` | yes (op=remove_routing_rule) | `operation=remove_routing_rule` | Routing rule ID to remove |
### Operation index
@@ -65,19 +66,22 @@ both `args` and `node_params` to the tool handler `_execute_proxy_config`.
### Output payload (success)
```ts
-// The node handler wraps the tool result under result = {...}
+// execute_proxy_config returns this dict; dispatch returns it directly,
+// then BaseNode.execute() wraps it once in the standard success envelope.
{
success: true,
operation: string,
- // op-specific fields: providers / rules / stats / name / rule_id / ip+latency_ms / updated_fields
+ // op-specific fields: providers / rules / stats / name / rule_id / ip+latency_ms / status_code / updated_fields / domain_pattern
}
```
+On `success=false`, `dispatch` raises `NodeUserError(result.error)` so the failure surfaces as an error envelope (a single WARN line), NOT a nested `result.result.success=false` payload.
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_proxy_config] --> B[_execute_proxy_config args=params node_params=params]
+ A[ProxyConfigNode.dispatch] --> B[execute_proxy_config params.model_dump]
B --> C{operation}
C -- list_providers --> LP[proxy_svc.get_providers -> model_dump list]
C -- get_stats --> GS[proxy_svc.get_stats]
@@ -108,7 +112,7 @@ flowchart TD
RR -- no --> Erid[success=false]
RR -- yes --> RR2[db.delete_proxy_routing_rule int(rule_id) + reload]
C -- unknown --> Eunk[success=false error Unknown operation]
- LP --> W[handle_proxy_config wraps tool_result<br/>under result=]
+ LP --> W[dispatch: success? return dict<br/>else raise NodeUserError]
GS --> W
LR --> W
AP3 --> W
@@ -133,7 +137,7 @@ flowchart TD
- **Validation**: each op checks its own required fields before any service call.
- **Branches**: 10 operations dispatched by string match; unknown op returns `Unknown operation: <op>`.
- **Fallbacks**: `preferred_providers` JSON parse error -> `[]`; `url_template` JSON parse error -> early return with error (no fallback).
-- **Error paths**: validation failures return `{success: false, error: "..."}` *without* setting `operation`. The node handler wraps the tool result in its own envelope and copies the tool's `success` into the outer envelope, so validation errors look like `{success: false, result: {success: false, error: "..."}}`.
+- **Error paths**: per-op validation failures return `{success: false, error: "..."}` from `execute_proxy_config`. `dispatch` checks `result.get("success")` and, when false, `raise NodeUserError(result.error or "Proxy config failed")`, so the failure surfaces as a single-WARN error envelope rather than a nested success-false payload.
## Side Effects
@@ -154,9 +158,9 @@ flowchart TD
## Edge cases & known limits
-- `handle_proxy_config` passes the same dict for both `args` and `node_params`, so the `args.get(..., node_params.get(...))` fallback resolves to the same source. This is harmless but tautological.
-- The outer node handler copies the tool's `success` into the envelope, yet still wraps the tool payload under `result`; tests must inspect both `result["success"]` AND `result["result"]["success"]`.
-- `test_provider` does a **real** HTTP call to `httpbin.org/ip` when the handler is invoked - there is no offline / dry-run path; this call is un-mocked in production.
+- The workflow-node path and the AI-tool path call the same `execute_proxy_config` function, so behaviour is identical between them.
+- A failed op no longer double-wraps: `dispatch` raises `NodeUserError` on `success=false`, so tests inspect the error envelope rather than `result["result"]["success"]`.
+- `test_provider` does a **real** HTTP call to `httpbin.org/ip` when invoked - there is no offline / dry-run path; this call is un-mocked in production.
- `update_provider` silently returns `success=true` with `updated_fields=[]` if no fields were changed.
- `remove_provider` / `remove_routing_rule` do not verify the row existed before issuing the delete; they always return `success=true` unless the DB raises.
- `url_template` is re-serialized via `json.dumps(...)` before being saved, so whitespace in the caller's JSON is normalised.
diff --git a/docs-internal/node-logic-flows/http_proxy/proxyRequest.md b/docs-internal/node-logic-flows/http_proxy/proxyRequest.md
index 66bc5613..3c52e7fc 100644
--- a/docs-internal/node-logic-flows/http_proxy/proxyRequest.md
+++ b/docs-internal/node-logic-flows/http_proxy/proxyRequest.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | proxy / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/proxy.py::handle_proxy_request`](../../../server/services/handlers/proxy.py) |
+| **Backend handler** | [`server/nodes/proxy/proxy_request/__init__.py`](../../../server/nodes/proxy/proxy_request/__init__.py) — `ProxyRequestNode.request` (dispatched via `BaseNode.execute()` + `@Operation("request")`; the legacy `handlers/proxy.py` was deleted in Wave 11.D.3) |
| **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) |
-| **Skill (if any)** | [`server/skills/web_agent/http-request-skill/SKILL.md`](../../../server/skills/web_agent/http-request-skill/SKILL.md) (covers useProxy + proxyRequest) |
-| **Dual-purpose tool** | yes - workflow node and AI tool |
+| **Skill (if any)** | [`server/skills/web_agent/http-request-skill/SKILL.md`](../../../server/skills/web_agent/http-request-skill/SKILL.md) (covers use_proxy + proxyRequest) |
+| **Dual-purpose tool** | yes - tool name `proxy_request` (`usable_as_tool = True`) |
## Purpose
@@ -28,16 +28,17 @@ latency per attempt and feeds them back into the provider's health score.
|------|------|---------|----------|---------------------|-------------|
| `method` | options | `GET` | no | - | `GET` / `POST` / `PUT` / `DELETE` / `PATCH` |
| `url` | string | `""` | **yes** | - | Target URL |
-| `headers` | string (JSON) | `"{}"` | no | - | JSON object, falls back to `{}` on parse error |
-| `body` | string | `""` | no | `method in [POST, PUT, PATCH]` | JSON if parseable else raw content |
-| `timeout` | number | `30` | no | - | Seconds |
-| `proxyProvider` | string | `""` | no | - | Specific provider name, empty for auto-select |
-| `proxyCountry` | string | `""` | no | - | ISO country code |
-| `sessionType` | options | `rotating` | no | - | `rotating` or `sticky` |
-| `stickyDuration` | number | `300` | no | `sessionType=sticky` | Seconds (30-3600) |
-| `maxRetries` | number | `3` | no | - | Number of retry attempts (`proxyMaxRetries` is also read) |
-| `proxyFailover` | boolean | `true` | no | - | On failure, request a new proxy URL and retry |
-| `followRedirects` | boolean | `true` | no | - | Declared in frontend; handler does not currently forward to httpx |
+| `headers` | object (`Dict[str,str]`) | `{}` | no | - | Request headers passed straight to httpx |
+| `body` | `Optional[Any]` | `null` | no | `method in [POST, PUT, PATCH]` | JSON object or raw string; string parsed as JSON if possible else raw content |
+| `timeout` | number | `30` | no | - | Seconds (1-600) |
+| `proxy_provider` | string | `auto` | no | - | Specific provider name; `auto` for health-based auto-select |
+| `proxy_country` | string | `""` | no | - | ISO country code |
+| `session_type` | options | `rotating` | no | - | `rotating` or `sticky` |
+| `sticky_duration` | number | `600` | no | - | Sticky session duration in seconds (`ge=1`) |
+| `max_retries` | number | `3` | no | - | Retry attempts (0-10); loop runs `max_retries + 1` times |
+| `follow_redirects` | boolean | `true` | no | - | Declared field; handler does not currently forward it to httpx |
+
+Params uses `extra="allow"`, so the retry loop also reads an undeclared `proxy_failover` key from the raw params (default `true`) — when `false` it breaks after the first failure.
## Outputs (handles)
@@ -63,57 +64,55 @@ latency per attempt and feeds them back into the provider's health score.
### Output payload (failure)
-`{ success: false, error: "All N attempts failed. Last error: <str>", ... }` or a short-circuit error if proxy service is unavailable.
+All retries exhausted -> `raise RuntimeError("All N attempts failed. Last error: <str>")`. Short-circuit raises are `NodeUserError` ("Proxy service not initialized..." when the service is disabled; "No proxy provider available" when `get_proxy_url` returns `None`). A status `>= 400` on an attempt raises `NodeUserError(f"HTTP {status}: ...")`. `BaseNode.execute()` maps `NodeUserError` to a single WARN line + structured envelope, and the bare `RuntimeError` to a full-traceback error envelope.
## Logic Flow
```mermaid
flowchart TD
- A[handle_proxy_request] --> B{proxy_svc.is_enabled?}
- B -- no --> Edis[Return success=false<br/>error: Proxy service not initialized]
- B -- yes --> C[Read method/url/headers/body/timeout/maxRetries/failover]
- C --> D{url truthy?}
- D -- no --> Eval[Return success=false<br/>error: URL is required]
- D -- yes --> E[Parse headers JSON; fallback {}]
- E --> F[await proxy_svc.get_proxy_url]
- F -- None --> Enop[Return success=false<br/>error: No proxy provider available]
+ A[ProxyRequestNode.request] --> B{proxy_svc enabled?}
+ B -- no --> Edis[raise NodeUserError<br/>Proxy service not initialized]
+ B -- yes --> F[await proxy_svc.get_proxy_url url raw]
+ F -- None --> Enop[raise NodeUserError<br/>No proxy provider available]
F -- url --> G[attempt = 0]
G --> H[httpx.AsyncClient proxy=proxy_url timeout=timeout]
- H --> I{method in POST/PUT/PATCH and body?}
+ H --> I{method in POST/PUT/PATCH and body is not None?}
I -- yes JSON --> J[kwargs.json]
I -- yes non-JSON --> J2[kwargs.content]
I -- no --> J3[no body]
J --> K[await client.request]
J2 --> K
J3 --> K
- K -- response --> L[proxy_svc.report_result success=status<400]
- L --> M[_track_proxy_usage -> db.save_api_usage_metric]
- M --> N[Return success=(status<400) with status/data/headers/url/method/provider/latency_ms/bytes/attempt]
- K -- Exception --> R[proxy_svc.report_result success=False<br/>log warning]
- R --> S{failover and attempt<maxRetries?}
+ K --> L[proxy_svc.report_result success=status<400]
+ L --> M[track_proxy_usage -> db.save_api_usage_metric]
+ M --> Lq{status >= 400?}
+ Lq -- yes --> Lerr[raise NodeUserError HTTP status]
+ Lq -- no --> N[return dict status/data/headers/url/method/provider/latency_ms/bytes/attempt]
+ H -- Exception --> R[proxy_svc.report_result success=False<br/>log warning]
+ Lerr -. caught by attempt loop .-> R
+ R --> S{failover and attempt<max_retries?}
S -- yes --> T[await get_proxy_url again<br/>on exception break]
T --> H
- S -- no --> U[Return success=false<br/>error: All N attempts failed]
+ S -- no --> U[raise RuntimeError<br/>All N attempts failed]
```
## Decision Logic
- **Validation**:
- - Service disabled -> short-circuit error envelope.
- - Empty `url` -> `ValueError` caught -> error envelope.
- - `get_proxy_url` returns `None` -> `No proxy provider available` error.
+ - Service disabled / not enabled -> `raise NodeUserError("Proxy service not initialized...")`.
+ - `get_proxy_url` returns `None` -> `raise NodeUserError("No proxy provider available")`.
+ - (`url` is a required Pydantic field; an empty value is rejected at param validation, not in the op body.)
- **Branches**:
- - Retry loop runs up to `maxRetries + 1` times; `failover=false` breaks after first failure.
- - Body JSON-parseable -> `kwargs.json`, else `kwargs.content`.
-- **Fallbacks**: invalid headers JSON -> `{}`.
+ - Retry loop runs up to `max_retries + 1` times; `proxy_failover=false` breaks after the first failure.
+ - Body str-JSON-parseable -> `kwargs.json`, str-non-JSON -> `kwargs.content`, non-str -> `kwargs.json`.
- **Error paths**:
+ - A `>= 400` status raises `NodeUserError(HTTP ...)` *inside* the try block, so it is caught by the attempt loop and triggers retry/failover.
- Per-attempt exception logged at `warning` and reported via `ProxyResult(success=False, error=...)`.
- - Final error string: `All <maxRetries+1> attempts failed. Last error: <last_error>`.
- - Outer unexpected exception -> `error: str(e)`.
+ - After all attempts: `raise RuntimeError("All <max_retries+1> attempts failed. Last error: <last_error>")`.
## Side Effects
-- **Database writes**: one row per successful attempt in `api_usage_metrics` via `database.save_api_usage_metric`, with `service=proxy_<provider_name>`, `operation=proxy_request`, `cost` computed as `bytes / 1GB * cost_per_gb` from `config/pricing.json`.
+- **Database writes**: one row per attempt that returned a response in `api_usage_metrics` via `database.save_api_usage_metric` (called from `track_proxy_usage` in [`server/nodes/proxy/_usage.py`](../../../server/nodes/proxy/_usage.py)), with `service=proxy_<provider_name>` (or `proxy` when empty), `operation=proxy_request`, `cost` computed as `bytes / 1GB * cost_per_gb` from `config/pricing.json`.
- **In-memory writes**: `proxy_svc.report_result(...)` mutates the provider's rolling history deque (length 100), which feeds `compute_score()`. The service also increments `_daily_spend_usd`.
- **Broadcasts**: none.
- **External API calls**: the target URL, tunneled through the selected proxy URL.
@@ -123,15 +122,15 @@ flowchart TD
## External Dependencies
- **Credentials**: `auth_service` stores proxy username/password under keys `proxy_<name>_username` / `proxy_<name>_password` (read by ProxyService when loading providers).
-- **Services**: `ProxyService` (must be enabled), `PricingService` (read through `_track_proxy_usage` -> `pricing._config["proxy"]`), `Database`.
+- **Services**: `ProxyService` (must be enabled), `PricingService` (read through `track_proxy_usage` -> `pricing._config["proxy"]`), `Database`.
- **Python packages**: `httpx`.
- **Environment variables**: none directly; ProxyService reads daily budget from `Settings.proxy_budget_daily_usd`.
## Edge cases & known limits
- Failed attempts are **not** tracked in `api_usage_metrics` - only successful requests that returned a response. If every attempt raises, the DB sees nothing for that call.
-- `followRedirects` is defined in the frontend but never forwarded to `httpx`; httpx default follows redirects.
-- `provider_name` in the result echoes the `proxyProvider` param verbatim; when left empty (auto-select), the envelope shows `""` even though a specific provider was used.
+- `follow_redirects` is a declared field but never forwarded to `httpx`; httpx default follows redirects.
+- `provider_name` in the result echoes `params.proxy_provider or ""` verbatim; with the default `auto` it shows `"auto"` even though `ProxyService` picked a concrete provider.
- Retry loop re-requests a new proxy URL via `get_proxy_url` on each retry; if that call raises (e.g. `BudgetExceededError`), the retry loop silently `break`s and the envelope reports the previous network error rather than the budget exception.
- Cost tracking reads `pricing._config["proxy"]` directly - touching a private attribute. If the pricing config is missing a `proxy` section, `cost_per_gb` falls back to `0.0` and cost rows are still written with `cost=0`.
diff --git a/docs-internal/node-logic-flows/http_proxy/proxyStatus.md b/docs-internal/node-logic-flows/http_proxy/proxyStatus.md
index c04fa10e..6243611f 100644
--- a/docs-internal/node-logic-flows/http_proxy/proxyStatus.md
+++ b/docs-internal/node-logic-flows/http_proxy/proxyStatus.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | proxy / tool |
-| **Backend handler** | [`server/services/handlers/proxy.py::handle_proxy_status`](../../../server/services/handlers/proxy.py) |
+| **Backend handler** | [`server/nodes/proxy/proxy_status/__init__.py`](../../../server/nodes/proxy/proxy_status/__init__.py) — `ProxyStatusNode.status` (dispatched via `BaseNode.execute()` + `@Operation("status")`; the legacy `handlers/proxy.py` was deleted in Wave 11.D.3) |
| **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) |
| **Skill (if any)** | [`server/skills/web_agent/proxy-config-skill/SKILL.md`](../../../server/skills/web_agent/proxy-config-skill/SKILL.md) |
-| **Dual-purpose tool** | yes |
+| **Dual-purpose tool** | yes - tool name `proxy_status` (`readonly = True`, `usable_as_tool = True`) |
## Purpose
@@ -25,7 +25,7 @@ provider to target.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `providerFilter` | string | `""` | no | - | Declared in frontend; handler currently **ignores** it and returns all providers |
+| `provider_name` | string | `""` | no | - | Declared field; the op currently **ignores** it and always returns all providers |
## Outputs (handles)
@@ -51,20 +51,19 @@ provider to target.
```mermaid
flowchart TD
- A[handle_proxy_status] --> B{proxy_svc and<br/>proxy_svc.is_enabled?}
- B -- no --> C[Return success=true<br/>result = enabled:false providers:[] stats:{}]
- B -- yes --> D[stats = proxy_svc.get_stats]
- D --> E[providers = proxy_svc.get_providers -> model_dump each]
- E --> F[Return success=true<br/>result = enabled:true providers stats]
- A -- Exception --> Eerr[Return success=false<br/>error: str(e)]
+ A[ProxyStatusNode.status] --> B{proxy_svc and<br/>proxy_svc.is_enabled?}
+ B -- no --> C[return enabled:false providers:[] stats:{}]
+ B -- yes --> D[providers = proxy_svc.get_providers -> model_dump each]
+ D --> E[stats = proxy_svc.get_stats]
+ E --> F[return enabled:true providers stats]
```
## Decision Logic
- **Validation**: none - no required params.
-- **Branches**: service-disabled short-circuit returns `enabled=false` with empty collections; still `success=true`.
-- **Fallbacks**: `providerFilter` is ignored - the handler always returns the full list.
-- **Error paths**: only the outer `Exception` catch; returns `{success: false, error: str(e)}`.
+- **Branches**: service-disabled short-circuit returns `enabled=false` with empty collections (still a success envelope from `BaseNode.execute()`).
+- **Fallbacks**: `provider_name` is ignored - the op always returns the full list.
+- **Error paths**: the op has no try/except; any exception propagates to `BaseNode.execute()`, which produces the error envelope.
## Side Effects
@@ -84,8 +83,8 @@ flowchart TD
## Edge cases & known limits
-- Even when proxy is disabled, the handler returns `success=true` with empty collections - callers need to inspect `result.enabled` rather than `success`.
-- `providerFilter` is defined in the frontend UI but never applied in the backend; filtering must happen in the consumer.
+- Even when proxy is disabled, the op returns a success envelope with `enabled=false` and empty collections - callers need to inspect `result.enabled` rather than `success`.
+- `provider_name` is a declared field but never applied in the op; filtering must happen in the consumer.
- `ProviderStats.model_dump()` exposes every field including computed `score`; the shape is governed by `services/proxy/models.py::ProviderStats`.
- No locking around the read: if another request is mutating `ProxyService._providers` (e.g. `reload_providers` mid-flight), the returned list may reflect a partial state.
diff --git a/docs-internal/node-logic-flows/search/braveSearch.md b/docs-internal/node-logic-flows/search/braveSearch.md
index 9ac04cd2..04419c16 100644
--- a/docs-internal/node-logic-flows/search/braveSearch.md
+++ b/docs-internal/node-logic-flows/search/braveSearch.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | search / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/search.py::handle_brave_search`](../../../server/services/handlers/search.py) |
+| **Backend handler** | [`server/nodes/search/brave_search/__init__.py`](../../../server/nodes/search/brave_search/__init__.py) - `BraveSearchNode`; dispatched via `BaseNode.execute()` + the `@Operation("search")` method (Wave 11; the old `server/services/handlers/search.py::handle_brave_search` is deleted) |
| **Tests** | [`server/tests/nodes/test_search.py`](../../../server/tests/nodes/test_search.py) |
| **Skill (if any)** | [`server/skills/web_agent/brave-search-skill/SKILL.md`](../../../server/skills/web_agent/brave-search-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `brave_search` |
+| **Dual-purpose tool** | yes - tool name `brave_search` (`usable_as_tool = True`) |
## Purpose
@@ -23,25 +23,28 @@ the LLM fills the same parameter schema documented below.
## Parameters
+(Pydantic `BraveSearchParams`, `model_config = {"extra": "ignore"}`.)
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `brave_search` | no | - | Name of this node when exposed as an AI tool |
-| `toolDescription` | string | (see frontend) | no | - | Description shown to LLM in tool schema |
-| `query` | string | `""` | **yes** | - | Search query |
-| `maxResults` | number | `10` | no | - | 1-100; clamped via `min(maxResults, 100)` before API call |
+| `tool_name` | string | `brave_search` | no | - | Name of this node when exposed as an AI tool |
+| `tool_description` | string | (see Params) | no | - | Description shown to LLM in tool schema |
+| `query` | string | (none) | **yes** | - | Search query (`min_length=1`) |
+| `max_results` | int | `10` | no | - | `ge=1, le=20`; clamped via `min(max_results, 20)` before API call |
| `country` | string | `""` | no | - | ISO country code, e.g. `US` - only sent when truthy |
-| `searchLang` | string | `""` | no | - | ISO language code, e.g. `en` - only sent when truthy |
-| `safeSearch` | options | `moderate` | no | - | One of `off` / `moderate` / `strict` |
+| `search_lang` | string | `"en"` | no | - | ISO language code - only sent when truthy |
+| `safe_search` | enum | `moderate` | no | - | One of `off` / `moderate` / `strict` |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard search payload (see below) |
-| `output-tool` | object | Same payload, used when this node is wired to an AI agent's `input-tools` |
+| `output-main` | object | Standard search payload (see below). When wired to an AI agent's `input-tools`, the same payload is returned to the LLM. |
### Output payload
+`BraveSearchOutput`:
+
```ts
{
query: string;
@@ -51,56 +54,53 @@ the LLM fills the same parameter schema documented below.
}
```
-Wrapped in the standard envelope: `{ success: true, result: <payload>, execution_time: number }`.
+Wrapped by `BaseNode._serialize_result` in the standard envelope: `{ success: true, result: <payload>, ... }`. Shared runtime schema: `SearchOutput` in [`server/services/node_output_schemas.py`](../../../server/services/node_output_schemas.py).
## Logic Flow
```mermaid
flowchart TD
- A[handle_brave_search] --> B{query stripped<br/>non-empty?}
- B -- no --> Eret[Return success=false<br/>error: Search query is required]
- B -- yes --> C[Read maxResults / country / searchLang / safeSearch from params]
- C --> D[await _get_api_key brave_search]
- D -- ValueError --> Eapi[Return success=false<br/>error: API key not configured]
- D -- ok --> E[Build params dict; only include country/searchLang/safeSearch if truthy]
- E --> F[GET api.search.brave.com/res/v1/web/search<br/>headers X-Subscription-Token + Accept: application/json<br/>timeout 30s]
- F -- HTTPStatusError --> Estat[Return success=false<br/>error: Brave Search API error STATUS]
- F -- other Exception --> Egen[Return success=false<br/>error: Brave Search failed STR]
- F -- 2xx --> G[data.web.results -> map first maxResults to title/snippet/url]
- G --> H[await _track_search_usage<br/>service=brave_search action=web_search count=1]
- H --> I[Return success=true with query / results / result_count / provider]
+ A[BaseNode.execute -> BraveSearchNode.search] --> B{Pydantic validates params}
+ B -- query missing / max_results out of range --> Eret[NodeUserError -> success=false<br/>error: invalid parameters]
+ B -- ok --> C[Build qs dict: q + count=min max_results,20<br/>add country/search_lang/safesearch when truthy]
+ C --> D[ctx.connection 'brave_search'<br/>injects X-Subscription-Token header]
+ D --> F[GET api.search.brave.com/res/v1/web/search]
+ F -- raise_for_status fails --> Estat[Exception -> success=false envelope]
+ F -- 2xx --> G[data.web.results -> map first max_results to title/snippet/url]
+ G --> H[Operation cost spec: service=brave_search action=web_search count=1]
+ H --> I[Return BraveSearchOutput]
```
## Decision Logic
-- **Empty query**: `query.strip() == ""` -> immediate failure envelope, no API call.
-- **Param trimming**: `country`, `searchLang`, `safeSearch` only added to the request when truthy. Note: `safeSearch` defaults to `'moderate'` in the frontend so it is effectively always sent.
-- **maxResults clamp**: API receives `min(maxResults, 100)` regardless of the requested value; the response is then sliced to the original `maxResults` again before mapping.
+- **Validation**: handled by Pydantic on `BraveSearchParams`. Empty / missing `query` (`min_length=1`) and out-of-range `max_results` (`le=20`) are rejected before the operation body runs; `BaseNode.execute` surfaces an `invalid parameters` failure envelope.
+- **Param trimming**: `country`, `search_lang`, `safe_search` only added to the request querystring when truthy. `search_lang` defaults to `'en'` and `safe_search` defaults to `'moderate'`, so both are effectively always sent.
+- **max_results clamp**: API receives `min(max_results, 20)`; the response list is then sliced to `max_results` before mapping.
- **Result mapping**: missing fields fall back to empty string (`title`, `snippet`, `url`).
## Side Effects
-- **Database writes**: one row in `api_usage_metrics` (via `database.save_api_usage_metric`) with `service='brave_search'`, `operation` from PricingService, `cost` from PricingService, plus `session_id` / `node_id` / `workflow_id`.
-- **Broadcasts**: none.
-- **External API calls**: `GET https://api.search.brave.com/res/v1/web/search` (timeout 30s).
+- **Database writes**: one row in `api_usage_metrics` via the framework's cost-tracking on the `@Operation("search", cost={...})` spec (`service='brave_search'`, `action='web_search'`, `count=1`).
+- **Broadcasts**: per-node status via `BaseNode.execute` (`update_node_status` executing/success/error); no plugin-specific broadcasts.
+- **External API calls**: `GET https://api.search.brave.com/res/v1/web/search` via `ctx.connection`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: `auth_service.get_api_key('brave_search')` - looked up at execution time; stored under `EncryptedAPIKey` table.
-- **Services**: `PricingService` for cost calc, `Database` for usage row.
-- **Python packages**: `httpx`.
+- **Credentials**: `BraveSearchCredential` (`ApiKeyCredential`, id `brave_search`, header `X-Subscription-Token`) - resolved by the Connection facade at execution time; stored in `EncryptedAPIKey`.
+- **Services**: `ctx.connection` (Connection facade), framework cost tracking.
+- **Python packages**: `httpx` (via Connection facade).
- **Environment variables**: none.
## Edge cases & known limits
-- API hard-caps `count` at 100; values above 100 are silently capped.
-- Network errors are caught by the generic `Exception` handler and returned as `Brave Search failed: <str>` - i.e. the handler never raises; downstream nodes always see an envelope.
-- The handler does *not* validate `safeSearch` values - the API rejects unknown values with 4xx, which becomes the `Brave Search API error: 400` envelope.
+- `max_results` is hard-bounded at 20 by Pydantic (`le=20`); values above 20 fail validation rather than being silently capped (test `test_clamps_max_results_to_100` asserts `success=false`).
+- `response.raise_for_status()` propagates non-2xx as an exception; `BaseNode.execute` converts it into a failure envelope, so downstream nodes always see an envelope.
+- The handler does not validate `safe_search` enum at runtime beyond Pydantic - the API rejects unknown values with 4xx.
## Related
- **Skills using this as a tool**: [`brave-search-skill/SKILL.md`](../../../server/skills/web_agent/brave-search-skill/SKILL.md)
-- **Companion nodes**: [`serperSearch`](./serperSearch.md), [`perplexitySearch`](./perplexitySearch.md), [`duckduckgoSearch`](../tools/duckduckgoSearch.md)
-- **Architecture docs**: [Pricing Service](../../pricing_service.md), [Credentials Encryption](../../credentials_encryption.md)
+- **Companion nodes**: [`serperSearch`](./serperSearch.md), [`perplexitySearch`](./perplexitySearch.md), `duckduckgoSearch` (tool category)
+- **Architecture docs**: [Plugin System](../../plugin_system.md), [Pricing Service](../../pricing_service.md), [Credentials Encryption](../../credentials_encryption.md)
diff --git a/docs-internal/node-logic-flows/search/perplexitySearch.md b/docs-internal/node-logic-flows/search/perplexitySearch.md
index 986319ab..2e467ad9 100644
--- a/docs-internal/node-logic-flows/search/perplexitySearch.md
+++ b/docs-internal/node-logic-flows/search/perplexitySearch.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | search / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/search.py::handle_perplexity_search`](../../../server/services/handlers/search.py) |
+| **Backend handler** | [`server/nodes/search/perplexity_search/__init__.py`](../../../server/nodes/search/perplexity_search/__init__.py) - `PerplexitySearchNode`; dispatched via `BaseNode.execute()` + the `@Operation("search")` method (Wave 11; the old `server/services/handlers/search.py::handle_perplexity_search` is deleted) |
| **Tests** | [`server/tests/nodes/test_search.py`](../../../server/tests/nodes/test_search.py) |
| **Skill (if any)** | [`server/skills/web_agent/perplexity-search-skill/SKILL.md`](../../../server/skills/web_agent/perplexity-search-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `perplexity_search` |
+| **Dual-purpose tool** | yes - tool name `perplexity_search` (`usable_as_tool = True`) |
## Purpose
@@ -22,98 +22,99 @@ downstream nodes can render references or follow the sources.
## Parameters
+(Pydantic `PerplexitySearchParams`, `model_config = {"extra": "ignore"}`.)
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `perplexity_search` | no | - | Tool name when exposed to AI |
-| `toolDescription` | string | (see frontend) | no | - | Tool description for AI |
-| `query` | string | `""` | **yes** | - | Question for the model |
-| `model` | options | `sonar` | no | - | One of `sonar` / `sonar-pro` / `sonar-reasoning` / `sonar-reasoning-pro` |
-| `searchRecencyFilter` | options | `""` | no | - | One of `""` / `month` / `week` / `day` / `hour` |
-| `returnImages` | boolean | `false` | no | - | When true, include images array in payload |
-| `returnRelatedQuestions` | boolean | `false` | no | - | When true, include related_questions array |
+| `tool_name` | string | `perplexity_search` | no | - | Tool name when exposed to AI |
+| `tool_description` | string | (see Params) | no | - | Tool description for AI |
+| `query` | string | (none) | **yes** | - | Question for the model (`min_length=1`) |
+| `model` | enum | `sonar` | no | - | One of `sonar` / `sonar-pro` / `sonar-reasoning` / `sonar-reasoning-pro` |
+| `search_recency_filter` | enum | `all` | no | - | One of `all` / `month` / `week` / `day` / `hour` (`all` = off, not sent) |
+| `return_images` | boolean | `false` | no | - | When true, request images from the API |
+| `return_related_questions` | boolean | `false` | no | - | When true, request related questions from the API |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Answer + citations payload |
-| `output-tool` | object | Same payload, AI-tool wiring |
### Output payload
+`PerplexitySearchOutput`:
+
```ts
{
query: string;
answer: string; // markdown content from choices[0].message.content
citations: string[]; // URLs the model cited
results: Array<{ url: string }>; // citations remapped as result objects
+ images: dict[] | null; // data.images verbatim (null when API omits it)
+ related_questions: string[] | null; // data.related_questions verbatim (null when API omits it)
model: string; // echoes the model param
provider: 'perplexity';
- images?: string[]; // present only if requested AND returned
- related_questions?: string[]; // present only if requested AND returned
}
```
-Wrapped in the standard envelope.
+Wrapped by `BaseNode._serialize_result` in the standard envelope. Shared runtime schema: `SearchOutput` in [`server/services/node_output_schemas.py`](../../../server/services/node_output_schemas.py).
## Logic Flow
```mermaid
flowchart TD
- A[handle_perplexity_search] --> B{query stripped<br/>non-empty?}
- B -- no --> Eret[Return error: Search query is required]
- B -- yes --> C[Read model, searchRecencyFilter, returnImages,<br/>returnRelatedQuestions]
- C --> D[await _get_api_key 'perplexity']
- D -- ValueError --> Eapi[Return error: API key not configured]
- D -- ok --> E[Build body:<br/>model + messages role=user content=query]
- E --> F{searchRecencyFilter truthy?}
- F -- yes --> G[Add search_recency_filter to body]
- F -- no --> H{returnImages?}
+ A[BaseNode.execute -> PerplexitySearchNode.search] --> B{Pydantic validates params}
+ B -- query missing --> Eret[NodeUserError -> success=false<br/>error: invalid parameters]
+ B -- ok --> C[Build body: model + messages role=user content=query]
+ C --> D{search_recency_filter != 'all'?}
+ D -- yes --> E[Add search_recency_filter to body]
+ D -- no --> F{return_images?}
+ E --> F
+ F -- yes --> G[body.return_images=true]
+ F --> H{return_related_questions?}
G --> H
- H -- yes --> I[body.return_images=true]
- H -- no --> J{returnRelatedQuestions?}
+ H -- yes --> I[body.return_related_questions=true]
+ H --> J[ctx.connection 'perplexity'<br/>injects Authorization: Bearer]
I --> J
- J -- yes --> K[body.return_related_questions=true]
- J -- no --> L[POST api.perplexity.ai/chat/completions<br/>headers Authorization Bearer + JSON<br/>timeout 60s]
- K --> L
- L -- HTTPStatusError --> Estat[Return error: Perplexity API error STATUS]
- L -- other Exception --> Egen[Return error: Perplexity search failed STR]
- L -- 2xx --> M[answer = data.choices 0 .message.content or empty]
- M --> N[citations = data.citations or empty list]
- N --> O[results = list of url-only objects]
- O --> P[await _track_search_usage 'perplexity' 'sonar_search' 1]
- P --> Q[Return success with query / answer / citations / results / model / provider<br/>plus images and related_questions when present]
+ J --> K[POST api.perplexity.ai/chat/completions]
+ K -- raise_for_status fails --> Estat[Exception -> success=false envelope]
+ K -- 2xx --> L[answer = data.choices 0 .message.content or empty]
+ L --> M[citations = data.citations or empty list]
+ M --> N[results = list of url-only objects]
+ N --> O[images/related_questions = data.get verbatim]
+ O --> P[Operation cost spec: service=perplexity action=sonar_search count=1]
+ P --> Q[Return PerplexitySearchOutput]
```
## Decision Logic
-- **Empty query**: short-circuit failure envelope.
-- **Optional body fields** (`search_recency_filter`, `return_images`, `return_related_questions`) only added when the corresponding param is truthy. `searchRecencyFilter=''` is the explicit "off" value.
+- **Validation**: Pydantic on `PerplexitySearchParams`. Empty / missing `query` (`min_length=1`) is rejected before the operation body (`invalid parameters` failure envelope).
+- **Optional body fields**: `search_recency_filter` is added only when `!= 'all'`; `return_images` / `return_related_questions` are added only when truthy. `search_recency_filter='all'` is the explicit "off" value.
- **Empty `choices`**: `answer` falls back to `''` rather than raising.
-- **Optional response fields** (`images`, `related_questions`) only added to the output payload when present *and non-empty* (`if images:` / `if related_questions:`).
+- **Optional response fields**: `images` / `related_questions` are set verbatim from `data.get(...)` - they are `None` when the API omits them (no `if x:` gating; this differs from the deleted pre-Wave-11 handler which only included them when non-empty).
## Side Effects
-- **Database writes**: one `api_usage_metrics` row per call (`service='perplexity'`, `operation='sonar_search'`).
-- **Broadcasts**: none.
-- **External API calls**: `POST https://api.perplexity.ai/chat/completions` (timeout 60s - longer than the other search nodes because Sonar Reasoning models can be slow).
+- **Database writes**: one `api_usage_metrics` row per call via the `@Operation` cost spec (`service='perplexity'`, `action='sonar_search'`, `count=1`).
+- **Broadcasts**: per-node status via `BaseNode.execute`; no plugin-specific broadcasts.
+- **External API calls**: `POST https://api.perplexity.ai/chat/completions` via `ctx.connection`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: `auth_service.get_api_key('perplexity')`.
-- **Services**: `PricingService`, `Database`.
-- **Python packages**: `httpx`.
+- **Credentials**: `PerplexityCredential` (`ApiKeyCredential`, id `perplexity`, bearer auth). Its `_probe` short-circuits to `ProbeResult(valid=True)` - Perplexity has no cheap auth-gated validation endpoint, so a bad key only surfaces as a runtime 401.
+- **Services**: `ctx.connection`, framework cost tracking.
+- **Python packages**: `httpx` (via Connection facade).
## Edge cases & known limits
-- 60s timeout - reasoning-pro models can occasionally exceed this and surface as `Perplexity search failed: ReadTimeout`.
-- Citations field shape can vary across Perplexity response versions (`list[str]` historically). The handler assumes `list[str]`; a list-of-objects response will pass through to the output unchanged but break the `results` mapping (`{'url': <obj>}`).
+- Sonar Reasoning models can be slow; the Connection facade's default timeout governs (no per-call override in the plugin body, unlike the old handler's explicit 60s).
+- Citations field shape can vary across Perplexity response versions (`list[str]` historically). The plugin assumes `list[str]`; a list-of-objects response passes through to the output unchanged but breaks the `results` mapping (`{'url': <obj>}`).
- The `results` list is purely a citations remap; downstream nodes expecting `title` / `snippet` will not find them - use `braveSearch` or `serperSearch` for those.
## Related
- **Skills using this as a tool**: [`perplexity-search-skill/SKILL.md`](../../../server/skills/web_agent/perplexity-search-skill/SKILL.md)
- **Companion nodes**: [`braveSearch`](./braveSearch.md), [`serperSearch`](./serperSearch.md)
-- **Architecture docs**: [Pricing Service](../../pricing_service.md)
+- **Architecture docs**: [Plugin System](../../plugin_system.md), [Pricing Service](../../pricing_service.md)
diff --git a/docs-internal/node-logic-flows/search/serperSearch.md b/docs-internal/node-logic-flows/search/serperSearch.md
index 93dac436..fc2e399c 100644
--- a/docs-internal/node-logic-flows/search/serperSearch.md
+++ b/docs-internal/node-logic-flows/search/serperSearch.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | search / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/search.py::handle_serper_search`](../../../server/services/handlers/search.py) |
+| **Backend handler** | [`server/nodes/search/serper_search/__init__.py`](../../../server/nodes/search/serper_search/__init__.py) - `SerperSearchNode`; dispatched via `BaseNode.execute()` + the `@Operation("search")` method (Wave 11; the old `server/services/handlers/search.py::handle_serper_search` is deleted) |
| **Tests** | [`server/tests/nodes/test_search.py`](../../../server/tests/nodes/test_search.py) |
| **Skill (if any)** | [`server/skills/web_agent/serper-search-skill/SKILL.md`](../../../server/skills/web_agent/serper-search-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `serper_search` |
+| **Dual-purpose tool** | yes - tool name `serper_search` (`usable_as_tool = True`) |
## Purpose
@@ -22,100 +22,104 @@ default web search. Used as a workflow node and as an AI agent tool.
## Parameters
+(Pydantic `SerperSearchParams`, `model_config = {"extra": "ignore"}`.)
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `serper_search` | no | - | Tool name when exposed to AI |
-| `toolDescription` | string | (see frontend) | no | - | Tool description for AI |
-| `query` | string | `""` | **yes** | - | Search query |
-| `searchType` | options | `search` | no | - | One of `search` / `news` / `images` / `places` |
-| `maxResults` | number | `10` | no | - | 1-100; clamped via `min(maxResults, 100)` before API call |
+| `tool_name` | string | `serper_search` | no | - | Tool name when exposed to AI |
+| `tool_description` | string | (see Params) | no | - | Tool description for AI |
+| `query` | string | (none) | **yes** | - | Search query (`min_length=1`) |
+| `search_type` | enum | `search` | no | - | One of `search` / `news` / `images` / `places` |
+| `max_results` | int | `10` | no | - | `ge=1, le=100`; clamped via `min(max_results, 100)` before API call |
| `country` | string | `""` | no | - | Sent as `gl` only when truthy |
-| `language` | string | `""` | no | - | Sent as `hl` only when truthy |
+| `language` | string | `"en"` | no | - | Sent as `hl` only when truthy |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Search payload (shape varies by `searchType`) |
-| `output-tool` | object | Same payload, AI-tool wiring |
+| `output-main` | object | Search payload (`results` field shape varies by `search_type`). |
### Output payload
+`SerperSearchOutput`. Every result is the unified `SerperSearchResult` model
+(`title` / `snippet` / `url` / `position?`); fields not populated for a given
+vertical are left at their defaults (empty string / null).
+
```ts
{
query: string;
- results: Array<SearchResult | NewsResult | ImageResult | PlaceResult>;
+ results: Array<{ title: string; snippet: string; url: string; position: number | null }>;
result_count: number;
search_type: 'search' | 'news' | 'images' | 'places';
+ knowledge_graph: object | null; // only set when API returns knowledgeGraph
provider: 'serper';
- knowledge_graph?: object; // only when API returns it
}
-
-type SearchResult = { title: string; snippet: string; url: string; position?: number };
-type NewsResult = { title: string; snippet: string; url: string; date: string; source: string };
-type ImageResult = { title: string; imageUrl: string; url: string };
-type PlaceResult = { title: string; address: string; rating?: number; url: string };
```
-Wrapped in the standard envelope.
+Per vertical (mapping in the operation body):
+- `search`: `title`, `snippet`, `url` (from `organic[].link`), `position`.
+- `news`: `title`, `snippet`, `url` (from `news[].link`). No `position`.
+- `images`: `title`, `url` (from `images[].imageUrl` or `link`). No `snippet`.
+- `places`: `title`, `url` (from `places[].website`). No `snippet`.
+
+Wrapped by `BaseNode._serialize_result` in the standard envelope. Shared runtime schema: `SearchOutput` in [`server/services/node_output_schemas.py`](../../../server/services/node_output_schemas.py).
## Logic Flow
```mermaid
flowchart TD
- A[handle_serper_search] --> B{query stripped non-empty?}
- B -- no --> Eret[Return error: Search query is required]
- B -- yes --> C[Read maxResults, searchType, country, language]
- C --> D[await _get_api_key 'serper']
- D -- ValueError --> Eapi[Return error: API key not configured]
- D -- ok --> E[Build body q + num; add gl/hl when truthy]
- E --> F[Pick endpoint by searchType<br/>fallback to /search]
- F --> G[POST endpoint headers X-API-KEY + Content-Type: application/json<br/>timeout 30s]
- G -- HTTPStatusError --> Estat[Return error: Serper API error STATUS]
- G -- other Exception --> Egen[Return error: Serper search failed STR]
- G -- 2xx --> H{searchType branch}
+ A[BaseNode.execute -> SerperSearchNode.search] --> B{Pydantic validates params}
+ B -- query missing / max_results out of range --> Eret[NodeUserError -> success=false<br/>error: invalid parameters]
+ B -- ok --> C[Build body: q + num=min max_results,100<br/>add gl/hl when truthy]
+ C --> F[Pick endpoint by search_type<br/>fallback to /search]
+ F --> D[ctx.connection 'serper'<br/>injects X-API-KEY header]
+ D --> G[POST endpoint]
+ G -- raise_for_status fails --> Estat[Exception -> success=false envelope]
+ G -- 2xx --> H{search_type branch}
H -- search --> J1[Map data.organic -> title/snippet/url/position]
- H -- news --> J2[Map data.news -> title/snippet/url/date/source]
- H -- images --> J3[Map data.images -> title/imageUrl/url]
- H -- places --> J4[Map data.places -> title/address/rating/url]
+ H -- news --> J2[Map data.news -> title/snippet/url]
+ H -- images --> J3[Map data.images -> title/url]
+ H -- places --> J4[Map data.places -> title/url]
J1 --> K[knowledge_graph = data.knowledgeGraph or None]
J2 --> K
J3 --> K
J4 --> K
- K --> L[await _track_search_usage 'serper' 'web_search' 1]
- L --> M[Return success with query results result_count search_type provider plus knowledge_graph if present]
+ K --> L[Operation cost spec: service=serper action=web_search count=1]
+ L --> M[Return SerperSearchOutput]
```
## Decision Logic
-- **Empty query**: short-circuit failure envelope.
-- **searchType dispatch**: hard-coded `endpoint_map`; unknown values fall through to `https://google.serper.dev/search` *and* take the `if search_type == 'search'` branch... wait no - they hit the default endpoint but skip every branch in the result loop, returning `results: []`. Document this as a known gotcha.
-- **knowledge_graph**: only added to the payload when the API includes a non-falsy `knowledgeGraph` field.
-- **maxResults clamp**: API receives `min(maxResults, 100)`; results list is then sliced again to `maxResults`.
-- **gl/hl trimming**: `country` -> `gl`, `language` -> `hl`. Only sent when truthy.
+- **Validation**: Pydantic on `SerperSearchParams`. Empty / missing `query` and out-of-range `max_results` are rejected before the operation body (`invalid parameters` failure envelope).
+- **search_type dispatch**: `_ENDPOINTS` map keyed by `search_type`; unknown values fall back to `https://google.serper.dev/search` for the request BUT match no result-mapping branch, so they return `results: []` (known gotcha; locked by `test_unknown_search_type_falls_back_to_web_endpoint_but_returns_empty_results`). Note `search_type` is a `Literal`, so frontend/agent inputs are constrained; an out-of-Literal value only reaches the body via the AI-tool path if validation is bypassed.
+- **knowledge_graph**: only set when the API includes a non-falsy `knowledgeGraph` field.
+- **max_results clamp**: API receives `min(max_results, 100)`; each branch slices its source list to `max_results`.
+- **gl/hl trimming**: `country` -> `gl`, `language` -> `hl`. Only sent when truthy (`language` defaults to `'en'`).
## Side Effects
-- **Database writes**: one `api_usage_metrics` row per call (`service='serper'`, `operation` from PricingService).
-- **Broadcasts**: none.
-- **External API calls**: `POST` to one of `https://google.serper.dev/{search,news,images,places}` (timeout 30s).
+- **Database writes**: one `api_usage_metrics` row per call via the `@Operation` cost spec (`service='serper'`, `action='web_search'`, `count=1`).
+- **Broadcasts**: per-node status via `BaseNode.execute`; no plugin-specific broadcasts.
+- **External API calls**: `POST` to one of `https://google.serper.dev/{search,news,images,places}` via `ctx.connection`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: `auth_service.get_api_key('serper')`.
-- **Services**: `PricingService`, `Database`.
-- **Python packages**: `httpx`.
+- **Credentials**: `SerperCredential` (`ApiKeyCredential`, id `serper`, header `X-API-KEY`) - resolved by the Connection facade.
+- **Services**: `ctx.connection`, framework cost tracking.
+- **Python packages**: `httpx` (via Connection facade).
## Edge cases & known limits
-- An unrecognised `searchType` value silently returns `results: []` (no error). Refactors must preserve this or update the test to assert the new behaviour.
-- 4xx/5xx API errors are translated into `Serper API error: <status>` - the response body is logged but not exposed.
+- An unrecognised `search_type` value returns `results: []` (no error). Refactors must preserve this or update the contract test.
+- `response.raise_for_status()` propagates non-2xx as an exception; `BaseNode.execute` converts it into a failure envelope.
- The handler trusts the API to return the expected nested keys (`organic`, `news`, etc.); a malformed response yields an empty results list rather than an exception.
+- Result models are unified - `news`/`images`/`places` do NOT carry `date` / `source` / `address` / `rating` fields; only `title` / `snippet` / `url` / `position` exist.
## Related
- **Skills using this as a tool**: [`serper-search-skill/SKILL.md`](../../../server/skills/web_agent/serper-search-skill/SKILL.md)
- **Companion nodes**: [`braveSearch`](./braveSearch.md), [`perplexitySearch`](./perplexitySearch.md)
-- **Architecture docs**: [Pricing Service](../../pricing_service.md)
+- **Architecture docs**: [Plugin System](../../plugin_system.md), [Pricing Service](../../pricing_service.md)
diff --git a/docs-internal/node-logic-flows/specialized_agents/_pattern.md b/docs-internal/node-logic-flows/specialized_agents/_pattern.md
index b077c7e4..0547ad16 100644
--- a/docs-internal/node-logic-flows/specialized_agents/_pattern.md
+++ b/docs-internal/node-logic-flows/specialized_agents/_pattern.md
@@ -1,23 +1,30 @@
-# Generic Specialized Agent Pattern (`handle_chat_agent`)
+# Generic Specialized Agent Pattern (`SpecializedAgentBase`)
This document describes the shared behavioural contract for the 13 specialized
-agent nodes that route to the same backend handler:
+agent nodes that share one plugin base class:
`android_agent`, `coding_agent`, `web_agent`, `task_agent`, `social_agent`,
`travel_agent`, `tool_agent`, `productivity_agent`, `payments_agent`,
`consumer_agent`, `autonomous_agent`, `orchestrator_agent`, `ai_employee`.
-All specialized nodes use the `handle_chat_agent` handler wired in
-`server/services/node_executor.py`. They differ only in frontend
-presentation (icon, theme color, default skills) and intended use case.
-The two agents with **dedicated** handlers (`rlm_agent`,
+Post-Wave-11 there is no `handle_chat_agent` handler. Each agent is a
+self-contained plugin folder under `server/nodes/agent/<agent>/__init__.py`
+declaring a `SpecializedAgentBase` subclass; the body lives in
+[`server/nodes/agent/_specialized.py`](../../../server/nodes/agent/_specialized.py).
+Dispatch is via `BaseNode.execute()` -> the `@Operation("execute")` method
+(`SpecializedAgentBase.execute_op`), which calls
+[`prepare_agent_call`](../../../server/nodes/agent/_inline.py) for the shared
+pre-dispatch flow then `AIService.execute_chat_agent`. The agents differ only
+in frontend presentation (icon, color, subtitle) and intended use case. The
+two agents that DON'T use `execute_chat_agent` (`rlm_agent`,
`claude_code_agent`) are documented separately.
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
-| **Registry binding** | `partial(handle_chat_agent, ai_service=..., database=...)` |
+| **Plugin** | [`server/nodes/agent/<agent>/__init__.py`](../../../server/nodes/agent/) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) |
+| **Dispatch** | `BaseNode.execute()` -> `@Operation("execute")` -> `prepare_agent_call` -> `AIService.execute_chat_agent` |
+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) (5-tuple: memory, skill, tool, input, task) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
| **Architecture docs** | [Agent Architecture](../../agent_architecture.md), [Agent Delegation](../../agent_delegation.md), [Agent Teams](../../agent_teams.md) |
@@ -25,15 +32,17 @@ The two agents with **dedicated** handlers (`rlm_agent`,
A specialized agent is an `aiAgent` pre-configured for a specific domain
(Android control, coding, web automation, etc.). From the backend's
-perspective it is identical to `chatAgent`: same handler, same inputs, same
-output envelope, same delegation semantics. The specialization is conveyed
-only through:
+perspective it is identical to `chatAgent`: same `execute_chat_agent` path,
+same inputs, same output envelope, same delegation semantics. The
+specialization is conveyed only through:
1. Connected skills (usually via a Master Skill node seeded with the
domain-specific folder in `server/skills/`).
2. Connected tools (e.g. `coding_agent` typically has `pythonExecutor` wired
to `input-tools`).
-3. Frontend cosmetics (icon, theme color, AGENT_CONFIGS label).
+3. Frontend cosmetics (icon + color from the plugin folder / `visuals.json`,
+ plus `display_name` / `subtitle` on the plugin class). The old
+ `AGENT_CONFIGS` map was retired -- `AIAgentNode.tsx` reads the NodeSpec.
## Inputs (handles)
@@ -41,30 +50,33 @@ All 13 agents share these input handles.
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| `input-main` | main | no | Upstream data; auto-prompt fallback when `prompt` param is empty |
-| `input-skill` | main | no | Skill node(s) providing SKILL.md context |
-| `input-memory` | main | no | `simpleMemory` node for conversation history |
-| `input-tools` | main | no | Tool nodes exposed to the LLM via `chat_model.bind_tools` |
-| `input-task` | main | no | `taskTrigger` events from delegated child agents |
-| `input-teammates` | main | no | **Only on `orchestrator_agent` and `ai_employee`** -- connected agents become `delegate_to_*` tools |
+| `input-main` | input/main | no | Upstream data; auto-prompt fallback when `prompt` param is empty |
+| `input-skill` | input/skill | no | Skill node(s) providing SKILL.md context |
+| `input-memory` | input/memory | no | `simpleMemory` node for conversation history |
+| `input-tools` | input/tools | no | Tool nodes exposed to the LLM via `chat_model.bind_tools` |
+| `input-task` | input/task | no | `taskTrigger` events from delegated child agents |
+| `input-teammates` | input/teammates | no | **Only on `orchestrator_agent` and `ai_employee`** (via `team_lead_agent_handles()`) -- connected agents become `delegate_to_*` tools |
+
+Handle topology comes from `std_agent_handles()` (or `team_lead_agent_handles()`
+for the two team leads) in
+[`server/nodes/agent/_handles.py`](../../../server/nodes/agent/_handles.py).
## Parameters
-All specialized agents share the `AI_AGENT_PROPERTIES` schema from
-`specializedAgentNodes.ts` plus node-specific extras (e.g. orchestrator has
-`teamMode`, `ai_employee` has `teamMode` + `maxConcurrent`).
-
-| Name | Type | Default | Description |
-|------|------|---------|-------------|
-| `provider` | options | `openai` | AI provider (openai / anthropic / gemini / ...) |
-| `model` | string | `""` | Model ID; required |
-| `prompt` | string | `""` | User prompt; falls back to upstream output when empty |
-| `systemMessage` | string | `""` | Additional system instructions (prepended to skill content) |
-| `options.temperature` | number | `0.7` | Sampling temperature |
-| `options.maxTokens` | number | `4096` | Max response tokens |
-| `options.thinkingEnabled` | boolean | `false` | Extended thinking / reasoning |
-| `options.thinkingBudget` | number | `2048` | Thinking budget tokens (Claude/Gemini) |
-| `options.reasoningEffort` | options | `medium` | For OpenAI o-series and GPT-5 |
+All 13 specialized agents share the `SpecializedAgentParams` Pydantic model in
+[`server/nodes/agent/_specialized.py`](../../../server/nodes/agent/_specialized.py).
+There are **no** per-agent parameter extras: `orchestrator_agent` and
+`ai_employee` reuse the same `Params` (they override only `handles` +
+`tool_description`), so there is no `teamMode` / `maxConcurrent` field.
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `prompt` | string | `""` | no | - | User prompt; falls back to upstream output when empty (4 rows) |
+| `provider` | enum | `openai` | no | - | `openai` / `anthropic` / `gemini` / `openrouter` / `groq` / `cerebras` / `deepseek` / `kimi` / `mistral` / `ollama` / `lmstudio` |
+| `model` | string | `""` | no | - | Model ID (resolved against the provider; empty falls through to default) |
+| `system_message` | string\|null | `"You are a helpful assistant"` | no | - | System instructions (3 rows) |
+| `temperature` | float\|null | `None` | no | group `options` | 0.0-2.0; `None` -> `agent.default_temperature` in `llm_defaults.json` |
+| `max_tokens` | int\|null | `None` | no | group `options` | 1-200000; `None` -> per-model default (not silently capped) |
## Outputs (handles)
@@ -92,7 +104,8 @@ Wrapped in the standard envelope:
```mermaid
flowchart TD
- A[handle_chat_agent] --> B[_collect_agent_connections]
+ A[BaseNode.execute -> execute_op] --> P[prepare_agent_call]
+ P --> B[collect_agent_connections]
B --> B1[memory_data from input-memory]
B --> B2[skill_data from input-skill<br/>masterSkill expanded to N skills]
B --> B3[tool_data from input-tools<br/>androidTool sub-nodes discovered]
@@ -103,16 +116,16 @@ flowchart TD
B3 --> C
B4 --> C
B5 --> C
- C -- yes + completed/error --> D[_format_task_context<br/>prepend to prompt<br/>STRIP ALL tools]
+ C -- yes + completed/error --> D[format_task_context<br/>prepend to prompt<br/>STRIP ALL tools]
C -- no --> E{prompt empty<br/>and input_data?}
D --> E
E -- yes --> F[prompt = input_data.message/text/content/str]
E -- no --> G{team lead?<br/>orchestrator / ai_employee}
F --> G
- G -- yes --> H[_collect_teammate_connections<br/>append as delegate_to_* tools]
+ G -- yes --> H[collect_teammate_connections<br/>append as delegate_to_* tools]
G -- no --> I[ai_service.execute_chat_agent]
H --> I
- I --> J[Return envelope]
+ I --> J[Return result payload<br/>or raise RuntimeError]
```
## Decision Logic
@@ -126,11 +139,18 @@ flowchart TD
`input_data.message` -> `text` -> `content` -> `str(input_data)` in that
order.
- **Team-lead detection**: `TEAM_LEAD_TYPES = {'orchestrator_agent',
- 'ai_employee'}`. Teammates are collected only for these two node types
- and appended to `tool_data` as delegation targets.
+ 'ai_employee'}` (in `_inline.py`). Teammates are collected only for these
+ two node types and appended to `tool_data` as delegation targets; each
+ teammate's own `input-tools` edges are walked to populate `child_tools` so
+ the delegation tool description lists what the teammate can do.
- **Master Skill expansion**: `masterSkill` source nodes are expanded in
- `_collect_agent_connections` - each enabled entry in `skillsConfig`
+ `collect_agent_connections` - each enabled entry in `skillsConfig`
becomes a separate skill entry with `node_id = f"{source_id}_{skill_key}"`.
+- **api_key recovery**: `prepare_agent_call` reads the resolved key from
+ `context["_raw_parameters"]["api_key"]` (injected before Pydantic strips
+ it) and re-attaches it to `parameters` so `execute_chat_agent` sees it.
+- **Failure**: `execute_op` raises `RuntimeError(response["error"])` when
+ `execute_chat_agent` returns `success=False`.
## Side Effects
@@ -144,7 +164,7 @@ flowchart TD
invokes them.
- **External API calls**: one or more calls to the configured LLM provider
(see `services/ai.py::execute_chat_agent` -> LangChain chat model + `_run_agent_loop`).
-- **Subprocess / file I/O**: none directly in the handler; tools invoked by
+- **Subprocess / file I/O**: none directly in the plugin; tools invoked by
the LLM may trigger those themselves.
## External Dependencies
@@ -159,7 +179,7 @@ flowchart TD
- Empty `prompt` with no `input-main` connection still calls
`execute_chat_agent` -- the LLM may produce a generic response. There is
- no early-return error envelope for missing prompts in this handler.
+ no early-return error envelope for missing prompts in this path.
- When a `masterSkill` has zero enabled skills, no skill context is
injected and the agent runs without domain-specific instructions.
- `input-teammates` is silently ignored for all non-team-lead node types
@@ -169,7 +189,7 @@ flowchart TD
## Related
-- **Dedicated-handler siblings**: [`deepAgent`](./deepAgent.md), [`rlmAgent`](./rlmAgent.md), [`claudeCodeAgent`](./claudeCodeAgent.md)
+- **Dedicated-path siblings**: [`rlmAgent`](./rlmAgent.md), [`claudeCodeAgent`](./claudeCodeAgent.md)
- **Per-node variants** (short link-style docs):
[`androidAgent`](./androidAgent.md), [`codingAgent`](./codingAgent.md),
[`webAgent`](./webAgent.md), [`taskAgent`](./taskAgent.md),
diff --git a/docs-internal/node-logic-flows/specialized_agents/aiEmployee.md b/docs-internal/node-logic-flows/specialized_agents/aiEmployee.md
index 4dd05e02..c131c587 100644
--- a/docs-internal/node-logic-flows/specialized_agents/aiEmployee.md
+++ b/docs-internal/node-logic-flows/specialized_agents/aiEmployee.md
@@ -3,28 +3,27 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
-| **Theme color** | `dracula.purple` |
-| **Icon** | people / briefcase (U+1F465) |
-| **Team lead** | **yes** -- `input-teammates` handle enabled |
+| **Plugin** | [`server/nodes/agent/ai_employee/__init__.py`](../../../server/nodes/agent/ai_employee/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
+| **Team lead** | **yes** -- `input-teammates` handle enabled (`team_lead_agent_handles()`) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
## Purpose
Team-lead agent identical in behaviour to `orchestrator_agent`. Distinct
-only in frontend presentation (different icon, theme color, default
-labels) and an extra `maxConcurrent` parameter. Backend routing is the
-same: `handle_chat_agent` -> `_collect_teammate_connections` ->
-teammates appended to `tool_data` as `delegate_to_*` tools.
+only in frontend presentation (display name "AI Employee", subtitle
+"Team Orchestration"). Backend routing is the same:
+`SpecializedAgentBase.execute_op` -> `prepare_agent_call`
+(`collect_teammate_connections`) -> teammates appended to `tool_data` as
+`delegate_to_*` tools -> `execute_chat_agent`.
## What is unique to this node
-- **`input-teammates` handle** (same as `orchestrator_agent`).
-- **`teamMode` parameter** (`parallel` / `sequential`, default `parallel`;
- no `""` option).
-- **`maxConcurrent` parameter** (number, default `5`, range 1-20)
- controlling max concurrent task executions.
-- **Frontend theming**: purple dracula accent.
+- **`input-teammates` handle** (same as `orchestrator_agent`, via
+ `team_lead_agent_handles()`).
+- **No extra parameters**: reuses `SpecializedAgentParams` verbatim. There
+ is **no** `teamMode` or `maxConcurrent` field on the current Pydantic
+ model — those claims were stale.
+- **`tool_description`** declared for when a parent team lead delegates to it.
## Behaviour
@@ -34,10 +33,9 @@ for the shared contract.
## Edge cases
-- Same as `orchestrator_agent`: non-agent teammates are silently skipped,
- and `maxConcurrent` / `teamMode` are passed through as parameters but
- not enforced by `handle_chat_agent` itself -- the `delegate_to_*`
- implementation owns concurrency.
+- Same as `orchestrator_agent`: non-agent teammates are silently skipped;
+ concurrency of `delegate_to_*` calls is owned by the downstream tool /
+ Temporal child-workflow dispatch, not by any node parameter.
## Related
diff --git a/docs-internal/node-logic-flows/specialized_agents/androidAgent.md b/docs-internal/node-logic-flows/specialized_agents/androidAgent.md
index 751dd20d..85a3a631 100644
--- a/docs-internal/node-logic-flows/specialized_agents/androidAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/androidAgent.md
@@ -3,10 +3,8 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) (routed via registry) |
-| **Theme color** | `dracula.green` |
-| **Icon** | phone (U+1F4F1) |
-| **AGENT_CONFIGS entry** | `AIAgentNode.tsx` key `android_agent` |
+| **Plugin** | [`server/nodes/agent/android_agent/__init__.py`](../../../server/nodes/agent/android_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
+| **Subtitle** | Device Control |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
## Purpose
@@ -29,8 +27,9 @@ the `android_agent` folder via a Master Skill.
See **[Generic Specialized Agent Pattern](./_pattern.md)** for the full
contract: inputs, parameters, outputs, logic flow, decision logic, side
-effects, edge cases. This node routes to `handle_chat_agent` with no
-behavioural differences from the other 12 variants.
+effects, edge cases. This node routes through `SpecializedAgentBase.execute_op`
+-> `execute_chat_agent` with no behavioural differences from the other 12
+variants.
## Related
diff --git a/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.md b/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.md
index 520e69ac..cb72c117 100644
--- a/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/autonomous_agent/__init__.py`](../../../server/nodes/agent/autonomous_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.purple` |
| **Icon** | target (U+1F3AF) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
@@ -26,9 +26,11 @@ the autonomous skill pack.
## Behaviour
-See **[Generic Specialized Agent Pattern](./_pattern.md)**. Routes to the
-same `handle_chat_agent` -- the "autonomous" behaviour comes entirely from
-the attached skill content, not a different execution engine.
+See **[Generic Specialized Agent Pattern](./_pattern.md)**. Routes through the
+same `SpecializedAgentBase.execute_op` -> `execute_chat_agent` -- the
+"autonomous" behaviour comes entirely from the attached skill content, not a
+different execution engine. (The plugin declares a `tool_description` for when
+it is delegated to by a team lead.)
## Related
diff --git a/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.md b/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.md
index a4fbb8c7..7547e0f5 100644
--- a/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.md
@@ -3,104 +3,154 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/claude_code.py::handle_claude_code_agent`](../../../server/services/handlers/claude_code.py) |
-| **Backend service** | [`server/services/claude_code_service.py::ClaudeCodeService.execute`](../../../server/services/claude_code_service.py) |
+| **Plugin** | [`server/nodes/agent/claude_code_agent/__init__.py::ClaudeCodeAgentNode.execute_op`](../../../server/nodes/agent/claude_code_agent/__init__.py) (dispatch via `BaseNode.execute()`) |
+| **Backend service** | [`server/services/cli_agent/service.py::AICliService.run_batch`](../../../server/services/cli_agent/service.py) (provider `"claude"`); pool [`_pool.py`](../../../server/nodes/agent/claude_code_agent/_pool.py), argv [`_provider.py`](../../../server/nodes/agent/claude_code_agent/_provider.py), skill materialiser [`_skills.py`](../../../server/nodes/agent/claude_code_agent/_skills.py) |
+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py::TestClaudeCodeAgent`](../../../server/tests/nodes/test_specialized_agents.py) |
## Purpose
-Runs a Claude Code CLI subprocess (`claude -p <prompt> --output-format json
-...`) and returns the parsed JSON. Keeps a per-`node_id` session map so
-follow-up invocations resume with `--resume <session_id>`. This is the
-only specialized-agent node that shells out; the others stay in-process.
+Runs N parallel Claude Code CLI sessions over a list of `ClaudeTaskSpec`
+tasks via `AICliService.run_batch("claude", ...)`. The pool spawns `claude`
+as a plain subprocess driven over the VSCode-extension stream-json protocol
+(`--output-format stream-json --input-format stream-json --verbose --ide`) —
+NOT the old headless `claude -p`. The common case is a single visible
+`prompt` that synthesises a one-task batch. Memory continuity is handled by
+claude's own on-disk session JSONL via `--continue` / `--resume` over a
+stable `cwd`, not by re-injecting markdown. This is the only
+specialized-agent node that shells out; the others stay in-process.
## Inputs (handles)
+Standard `std_agent_handles()` topology (same as the generic agents).
+
| Handle | Purpose |
|--------|---------|
-| `input-main` | Auto-prompt fallback (reads `source.message / text / content`) |
-| `input-skill` | Skill instructions appended to `--append-system-prompt` |
-
-All other handles (`input-memory`, `input-tools`, `input-task`,
-`input-teammates`) are **ignored** by this handler.
+| `input-main` | Auto-prompt fallback (reads `source.message / text / content / str`) |
+| `input-skill` | Connected skill names collected; SKILL.md trees materialised under `<workspace>/.claude/skills/` and exposed to the CLI (not injected into a system prompt) |
+| `input-memory` | When wired, sets `continue_session=True` so claude emits `--continue` and resumes its on-disk session under the stable cwd |
+| `input-tools` | Connected tool nodes exposed to the CLI as `mcp__machinaos__<type>` MCP tools |
+| `input-task` | Collected by the edge-walker; `task_data` is unpacked but the 5th element is ignored (`_`) here |
## Parameters
-| Name | Type | Default | Description |
-|------|------|---------|-------------|
-| `prompt` | string | `""` | User prompt passed via `-p` |
-| `systemPrompt` | string | `""` | Prepended to any skill content for `--append-system-prompt` |
-| `model` | string | `claude-sonnet-4-6` | Passed via `--model` |
-| `workingDirectory` | string | (default workspace) | `cwd` for the subprocess |
-| `allowedTools` | string | `Read,Edit,Bash,Glob,Grep,Write` | Passed via `--allowedTools` |
-| `maxTurns` | number | `10` | `--max-turns` |
-| `maxBudgetUsd` | number | `5.0` | **`--max-budget-usd`** (note: historical bug fixed to use this exact flag name) |
+`ClaudeCodeAgentParams` — only `prompt` is visible; every other field carries
+`json_schema_extra.hidden=True` and feeds defaults into the per-task
+`ClaudeTaskSpec`.
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `prompt` | string | `""` | no | - | The prompt sent to Claude Code (only visible field, 4 rows) |
+| `tasks` | `ClaudeTaskSpec[]` | `[]` | no | hidden | Advanced multi-task batch (max 5 concurrent); `null` coerced to `[]` |
+| `model` | enum (`ClaudeCodeModel`) | `claude-sonnet-4-6` | no | hidden | `--model`; aliases `sonnet`/`opus`/`haiku` or pinned IDs; unknown values coerced to default |
+| `system_prompt` | string\|null | `None` | no | hidden | System prompt (3 rows) |
+| `working_directory` | string\|null | `None` | no | hidden | Git repo root; defaults to workflow workspace dir |
+| `max_parallel` | int | `5` | no | hidden | 1-20 concurrency cap |
+| `allowed_credentials` | string[] | `[]` | no | hidden | Credential names the CLI may fetch via MCP; `null` coerced to `[]` |
+| `effort` | enum\|null (`ClaudeCodeEffort`) | `None` | no | hidden | `low`/`medium`/`high`/`xhigh`/`max` via `--effort` (thinking models) |
+| `fallback_model` | enum\|null | `None` | no | hidden | `--fallback-model`; unknown values coerced to `None` |
## Outputs (handles)
+`Output = ClaudeCodeAgentOutput` (`extra="forbid"`) — aggregated batch shape:
+
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | `{ response, model, provider: 'anthropic', session_id, usage, timestamp }` |
+| `output-main` / `output-top` | object | Batch result (see payload below) |
+
+### Output payload
+
+```ts
+{
+ success: boolean; // result.n_failed == 0
+ n_tasks: number;
+ n_succeeded: number;
+ n_failed: number;
+ total_cost_usd: number | null;
+ wall_clock_ms: number;
+ tasks: SessionResultModel[]; // per-task result models
+ provider: "claude";
+ timestamp: string;
+ // Legacy single-task convenience (populated when len(tasks) == 1):
+ response: string | null;
+ session_id: string | null;
+ cost_usd: number | null;
+}
+```
+
+`node_output_schemas.py` registers `claude_code_agent` as `AIAgentOutput`
+(`response`/`thinking`/`model`/`provider`/`finish_reason`/`timestamp`) for the
+frontend Input Data panel — a separate, looser schema from the plugin's strict
+`ClaudeCodeAgentOutput`.
## Logic Flow
```mermaid
flowchart TD
- A[handle_claude_code_agent] --> B[broadcast executing]
- B --> C{prompt param set?}
- C -- no --> D[Scan input-main edges<br/>pull source.message/text/content]
- C -- yes --> E[_collect_agent_connections for skills]
- D --> F{prompt still empty?}
- F -- yes --> G[Return envelope success=false<br/>error: No prompt provided]
- F -- no --> E
- E --> H[Build system_parts<br/>systemPrompt + skill.instructions]
- H --> I[get_claude_code_service]
- I --> J[service.execute<br/>spawns claude subprocess<br/>300s timeout]
- J -- returncode != 0 --> K[RuntimeError<br/>caught by outer try/except<br/>returns success=false envelope]
- J -- returncode == 0 --> L[Parse JSON stdout<br/>or fallback to raw text]
- L --> M[Store session_id in service._session_map]
- M --> N[broadcast success]
- N --> O[Return envelope success=true]
+ A[ClaudeCodeAgentNode.execute_op] --> B[broadcast executing]
+ B --> C[collect_agent_connections<br/>memory/skill/tool/input]
+ C --> D{memory_data wired?}
+ D -- yes --> D1[continue_session = True]
+ D -- no --> E{params.tasks empty?}
+ D1 --> E
+ E -- yes --> F[Resolve prompt:<br/>params.prompt else input_data<br/>message/text/content/str]
+ F --> G{prompt empty?}
+ G -- yes --> H[raise NodeUserError<br/>no prompt]
+ G -- no --> I[Build single ClaudeTaskSpec]
+ E -- no --> J[Apply node defaults to each task<br/>model/system_prompt/effort/fallback/continue]
+ I --> K{memory + len tasks > 1?}
+ J --> K
+ K -- yes --> L[raise NodeUserError<br/>memory batch must be serial]
+ K -- no --> M[Resolve workspace_dir + repo_root]
+ M --> N[AICliService.run_batch claude]
+ N --> O[Aggregate result -> dict payload]
+ O --> P[broadcast success/warning]
```
## Decision Logic
- **Prompt resolution**: explicit `prompt` param wins; otherwise scans
- `input-main` edges and reads `source.message / text / content / str`.
-- **Skill collection**: uses `_collect_agent_connections` only to gather
- skills. Memory, tools, teammates, task triggers are all discarded.
-- **System prompt assembly**: `systemPrompt` param + every enabled skill's
- `instructions`, joined by `\n\n`, passed as
- `--append-system-prompt`.
-- **Session resume**: `ClaudeCodeService._session_map[node_id]` is
- consulted; when set, `--resume <session_id>` is added to argv.
-- **Working directory**: if `workingDirectory` is empty, the service
- defaults to `{workspace_base_resolved}/default` and creates it if
- missing.
+ `input_data` and reads `message / text / content / str` (same chain as
+ `_inline.py`). Empty -> `NodeUserError`.
+- **Single-task synthesis**: when `params.tasks` is empty, one
+ `ClaudeTaskSpec` is built from `prompt` + `model` + `system_prompt` +
+ `effort` + `fallback_model` + `continue_session`.
+- **Batch defaulting**: when `params.tasks` is non-empty, node-level
+ `model`/`system_prompt`/`effort`/`fallback_model` fill any task that
+ didn't override; `continue_session=True` is auto-set only when memory is
+ wired AND the task didn't explicitly opt in or pick a resume UUID.
+- **Memory bridge**: `continue_session = bool(memory_data)`; the argv builder
+ emits `--continue` (first run is a benign no-op, later runs resume the
+ on-disk JSONL). NO markdown re-injection.
+- **Serial-memory guard**: memory wired AND `len(tasks) > 1` -> `NodeUserError`
+ (parallel `--resume` against one JSONL would corrupt it).
+- **Workspace**: `ctx.raw["workspace_dir"]` (injected by workflow.py) or
+ `working_directory`; falls back to `{workspace_base_resolved}/{workflow_id|default}`.
## Side Effects
-- **Subprocess spawn**: `asyncio.create_subprocess_exec('claude', '-p',
- <prompt>, '--output-format', 'json', '--model', <model>, '--max-turns',
- N, '--allowedTools', <list>, '--max-budget-usd', <usd>, [--resume
- <session_id>,] [--append-system-prompt <system>,] ...)` with a 300s
- `asyncio.wait_for` timeout. Falls back to `npx -y @anthropic-ai/claude-code`
- when the `claude` binary is not on PATH.
+- **Subprocess spawn**: `AICliService.run_batch` -> `ClaudeSessionPool` spawns
+ `claude` as a plain-pipe subprocess (no PTY) with `--output-format
+ stream-json --input-format stream-json --verbose --ide
+ --permission-mode dontAsk` + `--allowedTools <mcp tools + conditional Skill +
+ 5 infra tools>` + `--mcp-config <bearer>` + `--add-dir <workspace>`. Warm
+ subprocesses are reused across turns/batches; crash recovery via
+ `--resume <UUID>`.
- **Broadcasts**: `StatusBroadcaster.update_node_status` -- executing
- ("Starting Claude Code..." and "Running Claude Code (<model>)..."),
- then success on completion.
-- **Database**: none directly; usage tracking happens only if the caller
- wires it in (handler signature accepts `database=None` and never uses
- it).
-- **File I/O**: creates `{workspace_base_resolved}/default` when the
- default cwd does not exist.
+ ("Starting Claude Code batch..."), then `success`/`warning` on completion
+ (`n_succeeded`/`n_tasks`). The pool emits 4 typed CloudEvents
+ (`claude.session.{spawned,cleared,terminated,usage}`); memory turns emit
+ `node_parameters_updated` so the simpleMemory panel refreshes live.
+- **File I/O**: materialises connected SKILL.md trees under
+ `<workspace>/.claude/skills/<name>/` (diff-based on warm reuse). Claude
+ keeps its own session JSONL under `<CLAUDE_CONFIG_DIR>/projects/<cwd-key>/`.
- **External API**: indirect -- the CLI calls Anthropic internally.
## External Dependencies
- **Binaries**: `@anthropic-ai/claude-code`, MachinaOs-managed — auto-installed on first use into the shared npm tree at `<DATA_DIR>/packages/` (binary at `<DATA_DIR>/packages/node_modules/.bin/claude[.cmd]`); requires `npm` on PATH.
-- **Python packages**: standard library only
- (`asyncio.create_subprocess_exec`).
+- **Python packages**: standard library + the `services/cli_agent/` framework
+ (FastMCP bridge, session pool).
- **Credentials**: Claude Code handles its own authentication via
`CLAUDE_CONFIG_DIR=<DATA_DIR>/claude/` (= `~/.machina/claude/` by
default; see [Claude OAuth](../../claude_code_agent_architecture.md));
@@ -108,26 +158,29 @@ flowchart TD
## Edge cases & known limits
-- **No prompt -> early failure**: unlike `handle_chat_agent`, this
- handler explicitly returns an error envelope when neither `prompt`
- nor a connected `input-main` output yields text.
-- **`max_budget_usd <= 0`**: the `--max-budget-usd` flag is omitted
- entirely (see `claude_code_service.py:50-51`). This silently disables
- budget enforcement rather than passing `0`.
-- **Non-zero exit**: `ClaudeCodeService.execute` raises `RuntimeError`,
- the handler's outer `except Exception` converts it to
- `success=false` with the error message as the envelope's `error` field.
-- **300-second timeout**: `asyncio.wait_for(proc.communicate(),
- timeout=300)` is hard-coded. Long CLI runs beyond 5 minutes abort and
- surface as `TimeoutError` in the envelope.
-- **Tools/memory/teammates ignored**: connecting memory, tools, or
- teammates has no effect. The CLI's own tool set is controlled by
- `allowedTools`.
-- **JSON parsing fallback**: if stdout is not valid JSON, the service
- returns `{"result": <raw text>}` -- the handler still reports
- `success=true` and `response` contains the raw text.
+- **No prompt -> `NodeUserError`**: when `tasks` is empty AND neither
+ `prompt` nor a connected `input-main` output yields text, `execute_op`
+ raises `NodeUserError` (single WARN line, structured envelope).
+- **Memory-bound batches must be serial**: memory wired AND `len(tasks) > 1`
+ raises `NodeUserError` (parallel `--resume` would corrupt one JSONL).
+- **Stale-model coercion**: legacy saved `model` / `fallback_model` strings
+ (dot-spelled, date-suffixed, empty) coerce to the default / `None` via a
+ `mode="before"` field validator so old workflows still load.
+- **Output is strict** (`extra="forbid"`): a malformed CLI envelope raises a
+ `ValidationError` at the Output boundary instead of emitting an
+ unparseable shape downstream.
+- **Strict tool allowlist**: `--permission-mode dontAsk` enforces
+ `--allowedTools` (only wired `mcp__machinaos__*` tools, conditional `Skill`,
+ 5 infra tools); claude's own built-in Read/Edit/Bash/etc. do NOT fire
+ unless wired.
+- **Per-workflow skill isolation**: SKILL.md trees materialise under the
+ workspace's `.claude/skills/`, not the repo, so workflow A's skills can't
+ leak into workflow B's subprocess.
## Related
-- **Pattern siblings**: [`deepAgent`](./deepAgent.md), [`rlmAgent`](./rlmAgent.md)
-- **Architecture**: [Claude Code Agent](../../claude_code_agent_architecture.md)
+- **Dedicated-path siblings**: [`rlmAgent`](./rlmAgent.md)
+- **Generic pattern**: [`_pattern.md`](./_pattern.md)
+- **Architecture**: [CLI Agent Framework](../../cli_agent_framework.md),
+ [Claude Code Interactive Mode](../../claude_code_interactive_mode.md),
+ [Claude Code Agent](../../claude_code_agent_architecture.md)
diff --git a/docs-internal/node-logic-flows/specialized_agents/codexAgent.md b/docs-internal/node-logic-flows/specialized_agents/codexAgent.md
new file mode 100644
index 00000000..d45a6276
--- /dev/null
+++ b/docs-internal/node-logic-flows/specialized_agents/codexAgent.md
@@ -0,0 +1,142 @@
+# Codex (`codex_agent`)
+
+| Field | Value |
+|------|-------|
+| **Category** | specialized_agents |
+| **Plugin** | [`server/nodes/agent/codex_agent/__init__.py::CodexAgentNode.execute_op`](../../../server/nodes/agent/codex_agent/__init__.py) (dispatch via `BaseNode.execute()`) |
+| **Backend service** | [`server/services/cli_agent/service.py::AICliService.run_batch`](../../../server/services/cli_agent/service.py) (provider `"codex"`) |
+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) |
+| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
+| **Dual-purpose tool** | no |
+
+## Purpose
+
+Runs N parallel OpenAI Codex CLI sessions over a list of `CodexTaskSpec`
+tasks via `AICliService.run_batch("codex", ...)`. Sandbox-first companion to
+`claude_code_agent`: the sandbox is enforced by Codex itself (not MachinaOs),
+and each task runs in its own git worktree. Codex exposes no
+session/resume/budget/turns surface — only `sandbox` + `ask_for_approval` are
+task-level overrides. The common case is a single `prompt` that synthesises a
+one-task batch.
+
+## Inputs (handles)
+
+Standard `std_agent_handles()` topology.
+
+| Handle | Connection type | Required | Purpose |
+|--------|-----------------|----------|---------|
+| `input-main` | input/main | no | Prompt fallback: `_infer_prompt_from_inputs` reads `outputs[source].message/text/content/prompt/str` |
+| `input-skill` | input/skill | no | Connected skill names collected and passed to `run_batch` |
+| `input-memory` | input/memory | no | Collected by edge-walker but unused (Codex has no resume surface) |
+| `input-tools` | input/tools | no | Collected by edge-walker (5-tuple) but discarded here |
+| `input-task` | input/task | no | Collected but discarded |
+
+## Parameters
+
+`CodexAgentParams` (`extra="ignore"`):
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `tasks` | `CodexTaskSpec[]` | `[]` | no | - | List of Codex tasks (max 5 concurrent) |
+| `prompt` | string | `""` | no | - | Single-prompt fallback used only when `tasks` is empty (4 rows) |
+| `model` | string | `gpt-5.2-codex` | no | - | Default model for tasks that don't override |
+| `sandbox` | string | `workspace-write` | no | - | `read-only` / `workspace-write` / `danger-full-access` |
+| `ask_for_approval` | string | `never` | no | - | `untrusted` / `on-request` / `never` |
+| `system_prompt` | string\|null | `None` | no | - | System prompt (3 rows) |
+| `working_directory` | string\|null | `None` | no | - | Git repo root; defaults to workflow workspace dir |
+| `max_parallel` | int | `5` | no | - | 1-20 concurrency cap |
+| `allowed_credentials` | string[] | `[]` | no | - | Credential names the CLI may fetch via MCP |
+
+## Outputs (handles)
+
+`Output = CodexAgentOutput` (`extra="allow"`) — aggregated batch shape.
+
+| Handle | Shape | Description |
+|--------|-------|-------------|
+| `output-main` / `output-top` | object | Batch result (see payload below) |
+
+### Output payload
+
+```ts
+{
+ success: boolean; // result.n_failed == 0
+ n_tasks: number;
+ n_succeeded: number;
+ n_failed: number;
+ total_cost_usd: null; // always None for Codex
+ wall_clock_ms: number;
+ tasks: SessionResultModel[];
+ provider: "codex";
+ timestamp: string;
+ response: string | null; // populated when len(tasks) == 1
+}
+```
+
+Note: `codex_agent` is **not** registered in
+`server/services/node_output_schemas.py::_AGENT_TYPES`, so the frontend Input
+Data panel has no declared output schema for it (falls back to live execution
+data or empty state).
+
+## Logic Flow
+
+```mermaid
+flowchart TD
+ A[CodexAgentNode.execute_op] --> B[broadcast executing]
+ B --> C{params.tasks empty?}
+ C -- yes --> D[prompt = params.prompt<br/>else _infer_prompt_from_inputs]
+ D --> E{prompt empty?}
+ E -- yes --> F[raise RuntimeError<br/>provide tasks or prompt]
+ E -- no --> G[Build single CodexTaskSpec]
+ C -- no --> H[Apply node defaults<br/>model/system_prompt per task]
+ G --> I[collect_agent_connections<br/>skill names]
+ H --> I
+ I --> J[Resolve workspace_dir + repo_root]
+ J --> K[AICliService.run_batch codex]
+ K --> L[Aggregate result -> dict payload]
+ L --> M[broadcast success/warning]
+```
+
+## Decision Logic
+
+- **Prompt resolution**: `params.prompt` wins; otherwise
+ `_infer_prompt_from_inputs` scans `input-main` (or untagged) edges and reads
+ `outputs[source].message/text/content/prompt/str`. Empty -> `RuntimeError`.
+- **Single-task synthesis**: empty `tasks` builds one `CodexTaskSpec` from
+ `prompt` + `model` + `sandbox` + `ask_for_approval` + `system_prompt`.
+- **Batch defaulting**: non-empty `tasks` get node-level `model` /
+ `system_prompt` filled where the task didn't override.
+- **No memory/resume/budget/turns**: Codex has no session surface; memory /
+ tools / task data are collected by the edge-walker but discarded.
+
+## Side Effects
+
+- **Subprocess spawn**: `AICliService.run_batch("codex", ...)` spawns the
+ Codex CLI per task (per-task git worktree isolation; sandbox enforced by
+ Codex).
+- **Broadcasts**: `StatusBroadcaster.update_node_status` -- executing
+ ("Starting Codex batch..."), then `success`/`warning` on completion.
+- **File I/O**: per-task git worktrees under the workflow workspace dir.
+- **External API**: indirect -- the Codex CLI calls OpenAI internally.
+
+## External Dependencies
+
+- **Binaries**: the OpenAI Codex CLI (provider `"codex"` in the cli_agent
+ framework).
+- **Python packages**: the `services/cli_agent/` framework.
+- **Credentials**: Codex handles its own authentication; MachinaOs does not
+ inject an API key here. `allowed_credentials` gates what the CLI may fetch
+ via the MachinaOs MCP bridge.
+
+## Edge cases & known limits
+
+- **No prompt / tasks -> `RuntimeError`**: when `tasks` is empty AND no input
+ yields a prompt, `execute_op` raises `RuntimeError` (full traceback path —
+ not `NodeUserError` like `claude_code_agent`).
+- **`total_cost_usd` is always `None`**: Codex does not surface per-run cost.
+- **No output schema registration**: see Outputs note above.
+
+## Related
+
+- **Dedicated-path siblings**: [`claudeCodeAgent`](./claudeCodeAgent.md), [`rlmAgent`](./rlmAgent.md)
+- **Generic pattern**: [`_pattern.md`](./_pattern.md)
+- **Architecture**: [CLI Agent Framework](../../cli_agent_framework.md)
diff --git a/docs-internal/node-logic-flows/specialized_agents/codingAgent.md b/docs-internal/node-logic-flows/specialized_agents/codingAgent.md
index b5e89f9e..3f07c6ed 100644
--- a/docs-internal/node-logic-flows/specialized_agents/codingAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/codingAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/coding_agent/__init__.py`](../../../server/nodes/agent/coding_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.cyan` |
| **Icon** | laptop (U+1F4BB) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
diff --git a/docs-internal/node-logic-flows/specialized_agents/consumerAgent.md b/docs-internal/node-logic-flows/specialized_agents/consumerAgent.md
index 4f55140d..9a6f575c 100644
--- a/docs-internal/node-logic-flows/specialized_agents/consumerAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/consumerAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/consumer_agent/__init__.py`](../../../server/nodes/agent/consumer_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.purple` |
| **Icon** | shopping cart (U+1F6D2) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
@@ -12,7 +12,7 @@
AI agent pre-configured for customer support and consumer interactions
(order status, product recommendations). Like `payments_agent`, this is a
-naming / theming shell around `handle_chat_agent`.
+naming / theming shell around `SpecializedAgentBase` -> `execute_chat_agent`.
## What is unique to this node
diff --git a/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.md b/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.md
index 4a557388..f3ab3810 100644
--- a/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.md
@@ -3,10 +3,8 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
-| **Theme color** | `dracula.cyan` |
-| **Icon** | musical score (U+1F3BC) |
-| **Team lead** | **yes** -- `input-teammates` handle enabled |
+| **Plugin** | [`server/nodes/agent/orchestrator_agent/__init__.py`](../../../server/nodes/agent/orchestrator_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
+| **Team lead** | **yes** -- `input-teammates` handle enabled (`team_lead_agent_handles()`) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
## Purpose
@@ -17,18 +15,21 @@ are wired via the extra `input-teammates` handle and become
## What is unique to this node
-- **`input-teammates` handle** (extra input beyond the shared 5).
-- **`teamMode` parameter** (`""` / `parallel` / `sequential`) controlling
- how delegate_to_* calls are executed.
-- **Team lead detection**: `handle_chat_agent` checks `node_type in
- TEAM_LEAD_TYPES = {'orchestrator_agent', 'ai_employee'}` and calls
- `_collect_teammate_connections` to expand teammates into tools.
-- **Frontend theming**: cyan dracula accent.
+- **`input-teammates` handle** (extra input beyond the shared 5; declared
+ via `team_lead_agent_handles()` in `_handles.py`).
+- **No extra parameters**: uses the same `SpecializedAgentParams` as every
+ other agent. There is **no** `teamMode` / `maxConcurrent` field — those
+ were removed (or never existed on the current Pydantic model).
+- **`tool_description`** declared so a parent team lead delegating to this
+ node gets a meaningful tool string ("ONE-SHOT delegation ...").
+- **Team lead detection**: `prepare_agent_call` (in `_inline.py`) checks
+ `node_type in TEAM_LEAD_TYPES = {'orchestrator_agent', 'ai_employee'}` and
+ calls `collect_teammate_connections` to expand teammates into tools.
## Teammate collection
-`_collect_teammate_connections(node_id, context, database)` in
-`handlers/ai.py`:
+`collect_teammate_connections(node_id, context, database)` in
+[`server/services/plugin/edge_walker.py`](../../../server/services/plugin/edge_walker.py):
1. Scans `context.edges` for `edge.target == node_id` and
`edge.targetHandle == 'input-teammates'`.
@@ -37,8 +38,11 @@ are wired via the extra `input-teammates` handle and become
4. Loads `database.get_node_parameters(source_id)` for each teammate.
5. Returns a list of `{node_id, node_type, label, parameters}` dicts.
-The teammates are then appended to `tool_data` before
-`execute_chat_agent` is called, so the LLM sees them as ordinary tools.
+`prepare_agent_call` then walks each teammate's own `input-tools` edges to
+populate `child_tools` (so the delegation tool description lists what each
+teammate can do) and appends the teammates to `tool_data` before
+`execute_chat_agent` is called, so the LLM sees them as ordinary
+`delegate_to_*` tools.
## Behaviour
@@ -50,10 +54,10 @@ expansion above.
- Non-`AI_AGENT_TYPES` nodes wired to `input-teammates` are silently
skipped.
-- `teamMode` is collected by `execute_chat_agent` but the actual
- parallel / sequential scheduling is not enforced at this layer -- it
- depends on how the downstream `delegate_to_*` tool implementation handles
- concurrent invocations.
+- Concurrency of `delegate_to_*` calls is not controlled by any node
+ parameter (no `teamMode`); it depends on how the downstream
+ `delegate_to_*` tool implementation / Temporal child-workflow dispatch
+ handles concurrent invocations.
- When zero teammates are connected, the node behaves identically to any
other specialized agent.
diff --git a/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.md b/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.md
index e9f53867..78c44b0b 100644
--- a/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/payments_agent/__init__.py`](../../../server/nodes/agent/payments_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.green` |
| **Icon** | credit card (U+1F4B3) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
@@ -19,7 +19,7 @@ wire `httpRequest` against a payment provider (Stripe, Razorpay) as a tool.
- **Intended tool set**: `httpRequest` against payment APIs.
- **Intended skills**: no dedicated `server/skills/payments_agent/` folder
ships today -- the node is a naming / theming shell around
- `handle_chat_agent`.
+ `SpecializedAgentBase` -> `execute_chat_agent`.
- **Frontend theming**: green dracula accent.
## Behaviour
diff --git a/docs-internal/node-logic-flows/specialized_agents/productivityAgent.md b/docs-internal/node-logic-flows/specialized_agents/productivityAgent.md
index ba720152..9e759cd2 100644
--- a/docs-internal/node-logic-flows/specialized_agents/productivityAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/productivityAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/productivity_agent/__init__.py`](../../../server/nodes/agent/productivity_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.cyan` |
| **Icon** | clock (U+23F0) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
diff --git a/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md b/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md
index a85ddf0a..be5dc4ac 100644
--- a/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md
@@ -3,10 +3,9 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/rlm.py::handle_rlm_agent`](../../../server/services/handlers/rlm.py) |
+| **Plugin** | [`server/nodes/agent/rlm_agent/__init__.py::RLMAgentNode.execute_op`](../../../server/nodes/agent/rlm_agent/__init__.py) (dispatch via `BaseNode.execute()`) |
| **Backend service** | `AIService.rlm_service` -> `RLMService.execute` |
-| **Theme color** | `dracula.orange` |
-| **Icon** | brain (U+1F9E0) |
+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py::TestRLMAgent`](../../../server/tests/nodes/test_specialized_agents.py) |
## Purpose
@@ -23,23 +22,35 @@ Same 5 shared handles as the generic specialized agents. **No
## Parameters
-Standard `AI_AGENT_PROPERTIES` plus RLM extras:
-
-| Name | Type | Default | Description |
-|------|------|---------|-------------|
-| `maxIterations` | number | `30` | Hard cap on REPL iterations before forcing `FINAL(...)` |
+`RLMAgentNode.Params = SpecializedAgentParams` — the **same** model as the
+generic specialized agents (`prompt`, `provider`, `model`, `system_message`,
+`temperature`, `max_tokens`). There is **no** `maxIterations` field on the node
+params; REPL iteration bounding lives inside `RLMService` /
+`server/config/llm_defaults.json`, not on this plugin.
+
+| Name | Type | Default | Required | displayOptions.show | Description |
+|------|------|---------|----------|---------------------|-------------|
+| `prompt` | string | `""` | no | - | Reasoning task; falls back to upstream input |
+| `provider` | enum | `openai` | no | - | Big-LM provider |
+| `model` | string | `""` | no | - | Big-LM model ID |
+| `system_message` | string\|null | `"You are a helpful assistant"` | no | - | System instructions |
+| `temperature` | float\|null | `None` | no | group `options` | 0.0-2.0 |
+| `max_tokens` | int\|null | `None` | no | group `options` | 1-200000 |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | `{ response, model, provider, iterations, timestamp }` |
+| `output-main` / `output-top` | object | `SpecializedAgentOutput` (`extra="allow"`): `{ response, thinking?, model, provider, finish_reason?, timestamp }` plus any RLM extras the service emits |
+
+`node_output_schemas.py` registers `rlm_agent` as `AIAgentOutput` (same field
+set) for the frontend Input Data panel.
## Logic Flow
```mermaid
flowchart TD
- A[handle_rlm_agent] --> B[_collect_agent_connections]
+ A[RLMAgentNode.execute_op] --> B[collect_agent_connections]
B --> C{task_data present?}
C -- completed/error --> D[prepend task context<br/>strip tools]
C -- no --> E{prompt empty<br/>and input_data?}
@@ -48,18 +59,22 @@ flowchart TD
E -- no --> G[ai_service.rlm_service.execute]
F --> G
G --> H[RLMService REPL loop<br/>llm_query / rlm_query / FINAL]
- H --> I[Return envelope]
+ H --> I[Return result payload<br/>or raise RuntimeError]
```
## Decision Logic
-- **Same preprocessing as `handle_chat_agent`**: task-completion tool
- strip, auto-prompt fallback.
-- **No teammate expansion**: `_collect_teammate_connections` is not
- called.
+- **Same preprocessing as the generic agent path**: task-completion tool
+ strip (when `task_data.status in {completed, error}`), auto-prompt
+ fallback. Implemented inline in `RLMAgentNode.execute_op` (NOT via
+ `prepare_agent_call` — RLM has its own copy of the 3-step preamble).
+- **No teammate expansion**: `collect_teammate_connections` is not called;
+ `rlm_agent` is not in `TEAM_LEAD_TYPES`.
- **No workspace injection**: RLM executes code in an in-memory REPL
- (`exec()`), not on a real filesystem; the handler does not pass
+ (`exec()`), not on a real filesystem; the plugin does not pass
`workspace_dir`.
+- **Failure**: `execute_op` raises `RuntimeError` when the service returns
+ `success=False`; `BaseNode.execute()` wraps it into the error envelope.
## Side Effects
@@ -89,8 +104,9 @@ flowchart TD
process with no sandboxing. The skill prompt is expected to constrain
the LLM, but nothing prevents a misbehaving LLM from importing
`subprocess` or `os` and doing I/O.
-- **No per-step timeout**: `maxIterations` is the only bound; a single
- REPL step can hang the handler if user code blocks.
+- **No per-step timeout**: the RLM iteration cap (configured in
+ `RLMService` / `llm_defaults.json`, not on the node) is the only bound;
+ a single REPL step can hang the operation if user code blocks.
- **No team-lead support**: `input-teammates` is not read. Connecting
agents there has no effect.
- **LLM output parsing**: the REPL parser expects a specific fenced-code
@@ -99,5 +115,6 @@ flowchart TD
## Related
-- **Pattern siblings**: [`deepAgent`](./deepAgent.md), [`claudeCodeAgent`](./claudeCodeAgent.md)
+- **Dedicated-path siblings**: [`claudeCodeAgent`](./claudeCodeAgent.md)
+- **Generic pattern**: [`_pattern.md`](./_pattern.md)
- **Architecture**: [RLM Service](../../rlm_service.md)
diff --git a/docs-internal/node-logic-flows/specialized_agents/socialAgent.md b/docs-internal/node-logic-flows/specialized_agents/socialAgent.md
index 0d1f001c..c5542f47 100644
--- a/docs-internal/node-logic-flows/specialized_agents/socialAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/socialAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/social_agent/__init__.py`](../../../server/nodes/agent/social_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.green` |
| **Icon** | phone (U+1F4F1) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
diff --git a/docs-internal/node-logic-flows/specialized_agents/taskAgent.md b/docs-internal/node-logic-flows/specialized_agents/taskAgent.md
index 8de427ef..61251c60 100644
--- a/docs-internal/node-logic-flows/specialized_agents/taskAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/taskAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/task_agent/__init__.py`](../../../server/nodes/agent/task_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.purple` |
| **Icon** | clipboard (U+1F4CB) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
diff --git a/docs-internal/node-logic-flows/specialized_agents/toolAgent.md b/docs-internal/node-logic-flows/specialized_agents/toolAgent.md
index 26016bc1..bbf93a15 100644
--- a/docs-internal/node-logic-flows/specialized_agents/toolAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/toolAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/tool_agent/__init__.py`](../../../server/nodes/agent/tool_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.yellow` |
| **Icon** | wrench (U+1F527) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
diff --git a/docs-internal/node-logic-flows/specialized_agents/travelAgent.md b/docs-internal/node-logic-flows/specialized_agents/travelAgent.md
index 0a21d5e3..ddbf442d 100644
--- a/docs-internal/node-logic-flows/specialized_agents/travelAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/travelAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/travel_agent/__init__.py`](../../../server/nodes/agent/travel_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.orange` |
| **Icon** | airplane (U+2708) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
diff --git a/docs-internal/node-logic-flows/specialized_agents/webAgent.md b/docs-internal/node-logic-flows/specialized_agents/webAgent.md
index 23e1472c..96c75762 100644
--- a/docs-internal/node-logic-flows/specialized_agents/webAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/webAgent.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | specialized_agents |
-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |
+| **Plugin** | [`server/nodes/agent/web_agent/__init__.py`](../../../server/nodes/agent/web_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |
| **Theme color** | `dracula.pink` |
| **Icon** | globe (U+1F310) |
| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |
diff --git a/docs-internal/node-logic-flows/telegram_social/socialReceive.md b/docs-internal/node-logic-flows/telegram_social/socialReceive.md
index 243abebf..c750bee9 100644
--- a/docs-internal/node-logic-flows/telegram_social/socialReceive.md
+++ b/docs-internal/node-logic-flows/telegram_social/socialReceive.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | social |
-| **Backend handler** | [`server/services/handlers/social.py::handle_social_receive`](../../../server/services/handlers/social.py) |
+| **Backend handler** | plugin [`server/nodes/social/social_receive/__init__.py`](../../../server/nodes/social/social_receive/__init__.py) (`SocialReceiveNode`, custom `execute` override) delegating to [`server/nodes/social/_base.py::handle_social_receive`](../../../server/nodes/social/_base.py) |
| **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -16,37 +16,44 @@ Applies channel / message-type / sender / keyword filters, then fans the
result out across four dedicated handles (`message`, `media`, `contact`,
`metadata`) so downstream agents can grab just the slice they need.
-The dispatcher in `NodeExecutor._dispatch` special-cases `socialReceive`
-(node_executor.py:367-374) to collect upstream outputs via
-`_get_connected_outputs_with_info` and inject them as `outputs` / `source_nodes`.
+`socialReceive` is in the executor's `_NEEDS_CONNECTED_OUTPUTS` set, so the
+executor injects upstream outputs as `connected_outputs` / `source_nodes` into
+the execution context. `SocialReceiveNode.execute` overrides the default
+envelope wrap (reading `context.raw["connected_outputs"]` and
+`context.raw["source_nodes"]`) so the `filtered` / `reason` keys the handler
+emits survive to the top-level result dict.
## Inputs (handles)
-| Handle | Connection type | Required | Purpose |
-|--------|-----------------|----------|---------|
-| `input-main` | main | yes | Upstream trigger output (`whatsappReceive`, `telegramReceive`, `chatTrigger`, ...) - this is what gets normalized |
-
-The executor resolves upstream outputs keyed by source node **type**, so the
-handler walks `source_nodes` and looks up `outputs[source_type]`.
+The plugin declares **no explicit input handle** (`component_kind = "agent"`,
+the `handles` tuple lists only the four right-side output handles). Upstream
+trigger outputs reach the node via the executor's `connected_outputs` injection
+(`socialReceive` is in `_NEEDS_CONNECTED_OUTPUTS`), keyed by source node
+**type**, so the handler walks `source_nodes` and looks up
+`connected_outputs[source_type]`. Typical sources: `whatsappReceive`,
+`telegramReceive`, `chatTrigger`.
## Parameters
+All field names are snake_case (no camelCase aliases on `SocialReceiveParams`).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `channelFilter` | options | `all` | no | - | Only allow messages whose detected `channel` matches |
-| `messageTypeFilter` | options | `all` | no | - | Filter by `message_type` |
-| `senderFilter` | options | `all` | no | - | `all` / `any_contact` / `contact` / `group` / `keywords` |
-| `contactPhone` | string | `""` | yes when `senderFilter=contact` | `senderFilter: ['contact']` | Exact phone compare |
-| `groupId` | string | `""` | yes when `senderFilter=group` | `senderFilter: ['group']` | Exact chat_id compare |
-| `keywords` | string | `""` | no | `senderFilter: ['keywords']` | Comma-separated, lower-case substring match |
-| `ignoreOwnMessages` | boolean | `true` | no | - | Drop messages where `is_from_me=True` |
-| `ignoreBots` | boolean | `false` | no | - | Drop messages where `is_bot=True` |
+| `channel_filter` | options | `all` | no | - | `all`/`whatsapp`/`telegram`/`discord`/`slack`/`signal`/`sms`/`webchat`/`email`/`matrix`/`teams` - exact match against detected `channel` |
+| `message_type_filter` | options | `all` | no | - | `all`/`text`/`image`/`video`/`audio`/`document`/`sticker`/`location`/`contact`/`poll`/`reaction` |
+| `sender_filter` | options | `all` | no | - | `all`/`any_contact`/`contact`/`group`/`keywords` |
+| `contact_phone` | string | `""` | yes when `sender_filter=contact` | `sender_filter: ['contact']` | Exact phone compare |
+| `group_id` | string | `""` | yes when `sender_filter=group` | `sender_filter: ['group']` | Exact chat_id compare |
+| `keywords` | string | `""` | no | `sender_filter: ['keywords']` | Comma-separated, lower-case substring match |
+| `ignore_own_messages` | boolean | `true` | no | - | Drop messages where `is_from_me=True` |
+| `ignore_bots` | boolean | `false` | no | - | Drop messages where `is_bot=True` |
+| `include_media_data` | boolean | `false` | no | - | Include base64 media payload (memory-intensive) |
## Outputs (handles)
`NodeExecutor` stores the handler's `result.result` under `output_main`
**and** splits it into four additional output-store keys so different
-downstream handles can consume different slices (node_executor.py:292-296):
+downstream handles can consume different slices (node_executor.py:188-192):
| Handle / key | Shape | Description |
|--------------|-------|-------------|
@@ -105,7 +112,7 @@ When a message is filtered out the handler returns
```mermaid
flowchart TD
- A[_dispatch specialcase] --> A1[_get_connected_outputs_with_info]
+ A[SocialReceiveNode.execute] --> A1[read context.raw connected_outputs<br/>+ source_nodes]
A1 --> B[handle_social_receive]
B --> C[Scan source_nodes,<br/>pick first output with<br/>message_id / text / message]
C -- none --> C2[Fallback: iterate outputs dict]
@@ -141,16 +148,16 @@ flowchart TD
- **Chat type**: `_determine_chat_type` returns `data.chat_type` if set, else
`"group"` if `is_group` else `"dm"`.
- **Filters** (`_apply_filters`):
- - `channelFilter`: exact match against normalized `channel`.
- - `messageTypeFilter`: exact match against `message_type`.
- - `senderFilter=any_contact`: rejects group messages only.
- - `senderFilter=contact`: rejects when `sender_phone != contactPhone` (only
- if `contactPhone` is truthy).
- - `senderFilter=group`: rejects when `chat_id != groupId` (only if `groupId` is truthy).
- - `senderFilter=keywords`: lower-cased substring match on the `text`; empty
+ - `channel_filter`: exact match against normalized `channel`.
+ - `message_type_filter`: exact match against `message_type`.
+ - `sender_filter=any_contact`: rejects group messages only.
+ - `sender_filter=contact`: rejects when `sender_phone != contact_phone` (only
+ if `contact_phone` is truthy).
+ - `sender_filter=group`: rejects when `chat_id != group_id` (only if `group_id` is truthy).
+ - `sender_filter=keywords`: lower-cased substring match on the `text`; empty
`keywords` accepts all.
- - `ignoreOwnMessages` (default `true`): drops `is_from_me`.
- - `ignoreBots`: drops `is_bot`.
+ - `ignore_own_messages` (default `true`): drops `is_from_me`.
+ - `ignore_bots`: drops `is_bot`.
- **Message id synthesis**: If upstream omits `message_id`, a random 8-char
hex (`uuid4().hex[:8]`) is assigned.
- **Webchat defaults**: When `source_channel == 'webchat'`, missing `sender` is
@@ -187,7 +194,7 @@ flowchart TD
envelope is `success=true, result=null, filtered=true`. Downstream nodes
receive `null` via the output store - anything that expects a dict must
handle this.
-- **`_apply_filters` default for `ignoreOwnMessages` is `True`**: Loopback
+- **`_apply_filters` default for `ignore_own_messages` is `True`**: Loopback
messages (echo bots, self-tests) are silently dropped unless explicitly
turned off in the UI.
- **Top-level spread**: `{ ...unified_message }` overwrites the dedicated
@@ -199,13 +206,15 @@ flowchart TD
- **UUID `message_id` on fallback**: Not cryptographically unique across
sessions; two workflows receiving chatTrigger messages at the same time
can generate colliding ids.
-- **`source_nodes` parameter typo risk**: Handler accepts `outputs=None` and
- `source_nodes=None` and falls back to `context['outputs']` / `[]` - useful
- for direct unit tests but also means a broken executor path would silently
+- **Empty connected outputs**: `execute` reads `context.raw["connected_outputs"]`
+ and `["source_nodes"]`, defaulting to `{}` / `[]`. The handler still accepts
+ `outputs=None` / `source_nodes=None` and falls back to `context["outputs"]` -
+ useful for direct unit tests, but a broken executor injection would silently
return "No message data received" instead of failing fast.
## Related
- **Sibling nodes**: [`socialSend`](./socialSend.md), [`telegramReceive`](./telegramReceive.md)
- **Upstream triggers**: [`telegramReceive`](./telegramReceive.md); WhatsApp/Chat triggers live in other categories
-- **Dispatcher special-case**: `server/services/node_executor.py:287-296, 367-374`
+- **Shared handler body**: [`server/nodes/social/_base.py`](../../../server/nodes/social/_base.py)
+- **Output handle split**: `server/services/node_executor.py:188-192` (`_NEEDS_CONNECTED_OUTPUTS` injection + 4-key split)
diff --git a/docs-internal/node-logic-flows/telegram_social/socialSend.md b/docs-internal/node-logic-flows/telegram_social/socialSend.md
index da96a8c3..ee786b82 100644
--- a/docs-internal/node-logic-flows/telegram_social/socialSend.md
+++ b/docs-internal/node-logic-flows/telegram_social/socialSend.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | social / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/social.py::handle_social_send`](../../../server/services/handlers/social.py) |
+| **Backend handler** | plugin [`server/nodes/social/social_send/__init__.py`](../../../server/nodes/social/social_send/__init__.py) (`SocialSendNode`, dispatch via `BaseNode.execute()` -> `@Operation("send")`) delegating to [`server/nodes/social/_base.py::handle_social_send`](../../../server/nodes/social/_base.py) |
| **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) |
| **Skill (if any)** | none |
-| **Dual-purpose tool** | yes - works as a workflow node and as an AI Agent tool |
+| **Dual-purpose tool** | yes - `group = ("social", "tool")`, `usable_as_tool = True`; works as a workflow node and as an AI Agent tool |
## Purpose
@@ -18,69 +18,96 @@ and surfaces as a failed envelope.
## Inputs (handles)
+`component_kind = "agent"` (multi-handle layout). Four left-side input handles
+(all `role: main`):
+
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| `input-main` | main | no | Upstream data; consumed only via template resolution on individual params (e.g. `message`) |
+| `input-message` | main | no | Upstream message text (consumed via template resolution on `message`) |
+| `input-media` | main | no | Upstream media payload |
+| `input-contact` | main | no | Upstream contact payload |
+| `input-metadata` | main | no | Upstream metadata payload |
## Parameters
-| Name | Type | Default | Required | displayOptions.show | Description |
-|------|------|---------|----------|---------------------|-------------|
-| `channel` | options | `whatsapp` | no | - | Destination platform - only `whatsapp` is wired today |
-| `recipientType` | options | `phone` | no | - | `phone` / `group` / `channel` / `user` / `chat` |
-| `phone` | string | `""` | yes when `recipientType=phone` | `recipientType: ['phone']` | Destination phone (WhatsApp style incl. country code) |
-| `groupId` | string | `""` | yes when `recipientType=group` | `recipientType: ['group']` | Group JID |
-| `channelId` | string | `""` | yes when `recipientType=channel` | `recipientType: ['channel']` | Newsletter/channel JID |
-| `userId` | string | `""` | yes when `recipientType=user` | `recipientType: ['user']` | Platform user id |
-| `chatId` | string | `""` | yes when `recipientType=chat` | `recipientType: ['chat']` | Generic chat id |
-| `messageType` | options | `text` | no | - | `text`/`image`/`video`/`audio`/`document`/`sticker`/`location`/`contact`/`poll`/`buttons`/`list` |
-| `message` | string | `""` | yes when `messageType=text` | `messageType: ['text']` | Message body |
-| `mediaSource` | options | `url` | no | `messageType: ['image','video','audio','document','sticker']` | One of `url`/`base64`/`file` |
-| `mediaUrl` / `mediaData` / `filePath` | string | `""` | depends on `mediaSource` | (see above) | Media payload |
-| `mimeType`, `caption`, `filename` | string | `""` | no | media types | Extra media fields |
-| `latitude`, `longitude`, `locationName`, `address` | mixed | - | yes for location | `messageType: ['location']` | Location fields |
-| `contactName`, `vcard` | string | `""` | yes for contact | `messageType: ['contact']` | vCard fields |
-| `replyToMessage` | boolean | `false` | no | - | Send as reply |
-| `replyMessageId` | string | `""` | yes when `replyToMessage=true` | - | Message id to reply to |
+All field names are snake_case (the `SocialSendParams` model uses no camelCase
+aliases). 40 fields total; abbreviated below by group.
+
+| Name | Type | Default | displayOptions.show | Description |
+|------|------|---------|---------------------|-------------|
+| `channel` | options | `whatsapp` | - | `whatsapp`/`telegram`/`discord`/`slack`/`signal`/`sms`/`webchat`/`email`/`matrix`/`teams` - only `whatsapp` is wired |
+| `recipient_type` | options | `phone` | - | `phone`/`group`/`channel`/`user`/`chat` |
+| `phone` | string | `""` | `recipient_type: ['phone']` | Destination phone (no `+` prefix) |
+| `group_id` | string | `""` | `recipient_type: ['group']` | Group identifier |
+| `channel_id` | string | `""` | `recipient_type: ['channel']` | Channel identifier |
+| `user_id` | string | `""` | `recipient_type: ['user']` | User identifier |
+| `chat_id` | string | `""` | `recipient_type: ['chat']` | Generic chat identifier |
+| `thread_id` | string | `""` | - | Thread to reply in (optional) |
+| `message_type` | options | `text` | - | `text`/`image`/`video`/`audio`/`document`/`sticker`/`location`/`contact`/`poll`/`buttons`/`list` |
+| `message` | string | `""` | `message_type: ['text']` | Message body |
+| `format` | options | `plain` | `message_type: ['text']` | `plain`/`markdown`/`html` |
+| `media_source` | options | `url` | media types | `url`/`base64`/`file` |
+| `media_url` / `media_data` / `file_path` | string | `""` | media types + matching `media_source` | Media payload |
+| `mime_type`, `caption`, `filename` | string | `""` | media types | Extra media fields |
+| `latitude`, `longitude` | float | `0.0` | `message_type: ['location']` | Location coords |
+| `location_name`, `address` | string | `""` | `message_type: ['location']` | Location text |
+| `contact_name`, `contact_phone`, `vcard` | string | `""` | `message_type: ['contact']` | Contact fields |
+| `poll_question`, `poll_options` | string | `""` | `message_type: ['poll']` | Poll text |
+| `poll_allow_multiple` | boolean | `false` | `message_type: ['poll']` | Multi-select |
+| `button_text`, `buttons` | string | `""` / `"[]"` | `message_type: ['buttons']` | Buttons (JSON array) |
+| `list_title`, `list_button_text`, `list_sections` | string | `""` / `"View Options"` / `"[]"` | `message_type: ['list']` | List message |
+| `reply_to_message` | boolean | `false` | - | Quote an existing message |
+| `reply_message_id` | string | `""` | `reply_to_message: [true]` | Message id to reply to |
+| `reply_to_current` | boolean | `false` | `reply_to_message: [true]` | Reply to the triggering message |
+| `audio_as_voice` | boolean | `false` | `message_type: ['audio']` | Send audio as voice |
+| `disable_preview` | boolean | `false` | `message_type: ['text']` | Disable link preview |
+| `silent` | boolean | `false` | - | Send without notification |
+| `protect_content` | boolean | `false` | - | Prevent forwarding/saving |
## Outputs (handles)
-| Handle | Shape | Description |
-|--------|-------|-------------|
-| `output-main` | object | Send result envelope |
-| `output-tool` | object | Same payload, emitted when wired to an AI agent's `input-tools` |
+The plugin declares no explicit output handle (the `handles` tuple lists only
+the four input handles); the operation result flows out through the default
+`output_main` store key. When wired into an agent's `input-tools` the same
+payload is returned to the LLM (`usable_as_tool = True`).
+
+| Handle / key | Shape | Description |
+|--------------|-------|-------------|
+| `output_main` | object | Send result envelope (validated against `SocialSendOutput`: `sent`/`message_id` + `extra="allow"`) |
-### Output payload
+### Output payload (`result`)
```ts
{
success: true;
message_id: string | null;
channel: string; // e.g. 'whatsapp'
- recipient: string; // resolved phone / groupId / channelId / userId / chatId
+ recipient: string; // resolved phone / group_id / channel_id / user_id / chat_id
recipient_type: string;
message_type: string;
timestamp: string; // ISO
}
```
-Wrapped in the standard envelope `{ success, node_id, node_type, result, execution_time, timestamp }`.
+`handle_social_send` already returns the full `{ success, node_id, node_type,
+result, execution_time, timestamp }` envelope; the plugin's `send` op returns
+`response["result"]` on success and raises `RuntimeError(error)` otherwise.
## Logic Flow
```mermaid
flowchart TD
- A[handle_social_send] --> B[Read channel / recipientType / messageType]
- B --> C{recipientType}
+ A[SocialSendNode.send op<br/>-> handle_social_send] --> B[Read channel / recipient_type / message_type]
+ B --> C{recipient_type}
C -- phone --> C1[recipient = params.phone]
- C -- group --> C2[recipient = params.groupId]
- C -- channel --> C3[recipient = params.channelId]
- C -- user --> C4[recipient = params.userId]
- C -- chat --> C5[recipient = params.chatId]
+ C -- group --> C2[recipient = params.group_id]
+ C -- channel --> C3[recipient = params.channel_id]
+ C -- user --> C4[recipient = params.user_id]
+ C -- chat --> C5[recipient = params.chat_id]
C1 & C2 & C3 & C4 & C5 --> D{recipient truthy?}
D -- no --> Eval[raise ValueError<br/>caught -> success=false]
D -- yes --> E{channel}
- E -- whatsapp --> W[_send_via_whatsapp<br/>-> routers.whatsapp.handle_whatsapp_send]
+ E -- whatsapp --> W[_send_via_whatsapp<br/>-> get_social_send_handler 'whatsapp']
E -- other --> Stub[Return success=false stub<br/>'Platform X is not yet implemented']
W --> F{result.success?}
Stub --> F
@@ -90,9 +117,11 @@ flowchart TD
## Decision Logic
-- **Recipient resolution**: Hard-coded mapping from `recipientType` to a
- specific parameter key. An unknown `recipientType` leaves `recipient = None`
- and the handler raises `ValueError("Recipient (<type>) is required")`.
+- **Recipient resolution**: Hard-coded mapping from `recipient_type` to a
+ specific parameter key (`phone`->`phone`, `group`->`group_id`,
+ `channel`->`channel_id`, `user`->`user_id`, `chat`->`chat_id`). An unknown
+ `recipient_type` leaves `recipient = None` and the handler raises
+ `ValueError("Recipient (<type>) is required")`.
- **Channel dispatch**: Python `if/elif` on `channel`. Only `"whatsapp"` is
wired. Every other value returns
`{"success": False, "error": "Platform '<x>' is not yet implemented"}`
@@ -112,8 +141,9 @@ flowchart TD
to may record usage.
- **Broadcasts**: none from `handle_social_send` itself; broadcasts originate
in the underlying platform handler.
-- **External API calls**: delegated to `routers.whatsapp.handle_whatsapp_send`
- for the `whatsapp` channel, which hits the WhatsApp RPC service
+- **External API calls**: delegated to the WhatsApp send handler resolved via
+ `get_social_send_handler("whatsapp")` (the whatsapp plugin self-registers it
+ from `nodes/whatsapp/__init__.py`), which hits the WhatsApp RPC service
(default `http://localhost:9400`).
- **File I/O**: none from this handler. `mediaSource=file` reads are delegated.
- **Subprocess**: none.
@@ -122,8 +152,8 @@ flowchart TD
- **Credentials**: none at this layer - the WhatsApp handler manages its own
RPC connection.
-- **Services**: `routers.whatsapp.handle_whatsapp_send` (imported lazily
- inside `_send_via_whatsapp`).
+- **Services**: the registered `"whatsapp"` social-send handler resolved via
+ `get_social_send_handler("whatsapp")` inside `_send_via_whatsapp`.
- **Python packages**: stdlib only (`time`, `datetime`, `typing`).
- **Environment variables**: none.
@@ -139,17 +169,18 @@ flowchart TD
handle, `recipient` stays `None` and the ValueError message reports the
received `recipient_type` string back to the user. Good for debugging, bad
for LLM tool use (the LLM then retries with a different type).
-- **`groupId` used for non-phone `recipient_type`**: Inside `_send_via_whatsapp`,
+- **`group_id` used for non-phone `recipient_type`**: Inside `_send_via_whatsapp`,
anything that is not `phone` maps onto `group_id` in the WhatsApp params,
including `channel`/`user`/`chat`. This means channels/users/chats end up
being sent as WhatsApp groups, which the RPC will likely reject.
-- **camelCase vs snake_case drift**: Node parameters are mostly camelCase
- (`recipientType`, `messageType`, `mediaUrl`) while the WhatsApp handler
- expects snake_case; the conversion in `_send_via_whatsapp` is manual and
- easy to drift from new parameters added on either side.
+- **Manual param mapping**: `_send_via_whatsapp` rebuilds the whatsapp-send param
+ dict field-by-field from `SocialSendParams`. Both schemas are snake_case now,
+ but the mapping is still hand-written and can drift when new fields are added
+ on either side.
## Related
- **Sibling nodes**: [`socialReceive`](./socialReceive.md), [`telegramSend`](./telegramSend.md)
-- **Downstream handler**: `routers.whatsapp.handle_whatsapp_send`
+- **Downstream handler**: registered `"whatsapp"` social-send handler (`nodes/whatsapp/__init__.py`)
+- **Shared handler body**: [`server/nodes/social/_base.py`](../../../server/nodes/social/_base.py)
- **Architecture docs**: WhatsApp Integration section in the root [`CLAUDE.md`](../../../CLAUDE.md)
diff --git a/docs-internal/node-logic-flows/telegram_social/telegramReceive.md b/docs-internal/node-logic-flows/telegram_social/telegramReceive.md
index 1eb1668a..291d6a5b 100644
--- a/docs-internal/node-logic-flows/telegram_social/telegramReceive.md
+++ b/docs-internal/node-logic-flows/telegram_social/telegramReceive.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | social / trigger |
-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) + [`server/services/event_waiter.py::build_telegram_filter`](../../../server/services/event_waiter.py) |
+| **Backend handler** | [`server/nodes/telegram/telegram_receive.py`](../../../server/nodes/telegram/telegram_receive.py) (`TelegramReceiveNode`, a `TriggerNode` with `event_type = "telegram_message_received"`); filter via `build_filter` -> [`build_telegram_filter`](../../../server/services/event_waiter.py) |
| **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no (trigger only) |
@@ -11,12 +11,14 @@
## Purpose
Wait for an incoming Telegram message and emit it as the workflow trigger
-event. Implementation is split between the generic `handle_trigger_node`
-(registers a waiter and blocks on `event_waiter.wait_for_event`) and the
-Telegram-specific filter builder that pre-compiles a `matches()` closure
-from the node parameters. Events are produced by the `TelegramService`
-long-polling loop inside `_on_message_received` and dispatched via
-`event_waiter.dispatch("telegram_message_received", <event_data>)`.
+event. `TelegramReceiveNode` subclasses `TriggerNode`; its `execute` override
+first refuses to register if the bot is not connected, then defers to
+`TriggerNode.execute`. The Telegram-specific `build_filter` pre-compiles a
+`matches()` closure (via the legacy `build_telegram_filter`) from the node
+parameters. Events are produced by the `TelegramService` long-polling loop
+inside `_on_message_received`. In deployment mode this trigger runs on the
+canary path (`TriggerListenerWorkflow` per deployment/trigger, fed via the
+event framework); the registered filter narrows the stream.
## Inputs (handles)
@@ -24,17 +26,19 @@ Trigger node - no inputs.
## Parameters
+All field names are snake_case (no camelCase aliases on the Pydantic model).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `contentTypeFilter` | options | `all` | no | - | One of `all`/`text`/`photo`/`video`/`audio`/`voice`/`document`/`sticker`/`location`/`contact`/`poll` |
-| `senderFilter` | options | `all` | no | - | `all`/`self`/`private`/`group`/`supergroup`/`channel`/`specific_chat`/`specific_user`/`keywords` |
-| `chat_id` | string | `""` | yes when `senderFilter=specific_chat` | `senderFilter: ['specific_chat']` | Numeric chat id or `@username` |
-| `from_user` | string | `""` | yes when `senderFilter=specific_user` | `senderFilter: ['specific_user']` | Numeric user id |
-| `keywords` | string | `""` | yes when `senderFilter=keywords` | `senderFilter: ['keywords']` | Comma-separated, matched case-insensitively |
-| `ignoreBots` | boolean | `true` | no | - | Skip messages where `is_bot=True` (ignored when `senderFilter=self`) |
+| `content_type_filter` | options | `all` | no | - | One of `all`/`text`/`photo`/`video`/`audio`/`voice`/`document`/`sticker`/`location`/`contact`/`poll` |
+| `sender_filter` | options | `all` | no | - | `all`/`self`/`private`/`group`/`supergroup`/`channel`/`specific_chat`/`specific_user`/`keywords` |
+| `chat_id` | string | `""` | yes when `sender_filter=specific_chat` | `sender_filter: ['specific_chat']` | Numeric chat id or `@username` |
+| `from_user` | string | `""` | yes when `sender_filter=specific_user` | `sender_filter: ['specific_user']` | Numeric user id |
+| `keywords` | string | `""` | yes when `sender_filter=keywords` | `sender_filter: ['keywords']` | Comma-separated, matched case-insensitively |
+| `ignore_bots` | boolean | `true` | no | shown for every `sender_filter` except `self` | Skip messages where `is_bot=True` (ignored when `sender_filter=self`) |
-Legacy alias: `chatTypeFilter` is still read when `senderFilter` is absent and
-reconstructs the new-style value.
+Legacy alias: `chatTypeFilter` is still read by `build_telegram_filter` when
+`sender_filter` is absent and reconstructs the new-style value.
## Outputs (handles)
@@ -73,11 +77,11 @@ Wrapped in the standard success envelope when an event is received.
```mermaid
flowchart TD
- A[handle_trigger_node] --> B{TelegramService.connected?}
- B -- no --> Eoff[Return success=false<br/>error: bot not connected]
- B -- yes --> C[event_waiter.register<br/>-> build_telegram_filter]
+ A[TelegramReceiveNode.execute] --> B{TelegramService.connected?}
+ B -- no --> Eoff[_wrap_error<br/>error: bot not connected]
+ B -- yes --> C[TriggerNode.execute<br/>register waiter via build_filter]
C --> D[broadcaster.update_node_status waiting]
- D --> E[await event_waiter.wait_for_event]
+ D --> E[await event delivery]
E -- CancelledError --> Ecx[Return success=false<br/>error: Cancelled by user]
E -- event resolved --> F[Return success=true<br/>result = event_data]
@@ -92,8 +96,9 @@ flowchart TD
## Decision Logic
-- **Pre-check**: Handler inspects `TelegramService.connected` before registering
- a waiter. Prevents hanging forever when the bot token was never added.
+- **Pre-check**: `execute` inspects `TelegramService.connected` before deferring
+ to `TriggerNode.execute`. Prevents hanging forever when the bot token was
+ never added.
- **Sender filter branches** (inside `build_telegram_filter.matches`):
- `self`: compares `m.from_id` to a lazily-resolved `owner_chat_id`. If no
owner is known, the message is rejected (but no error surfaces to the
@@ -104,9 +109,9 @@ flowchart TD
- `all`: accepts everything (subject to `ignore_bots`).
- **Content type filter**: Always applied first; if set and not `all`, rejects
anything with a non-matching `content_type`.
-- **`ignore_bots`**: Applied last; silently skipped when `senderFilter == 'self'`.
-- **Legacy fallback**: If `senderFilter` is unset the code reconstructs it
- from `chat_id`/`from_user`/`keywords`/`chatTypeFilter` so old workflows keep
+- **`ignore_bots`**: Applied last; silently skipped when `sender_filter == 'self'`.
+- **Legacy fallback**: If `sender_filter` is unset the filter builder reconstructs
+ it from `chat_id`/`from_user`/`keywords`/`chatTypeFilter` so old workflows keep
working.
## Side Effects
@@ -139,13 +144,10 @@ flowchart TD
matches `"chicken"`. Empty `keywords` accepts all messages.
- **String comparison for IDs**: `specific_chat`/`specific_user` compare via
`str(...)`. A leading `@` or whitespace difference will silently mismatch.
-- **`ignore_bots` override**: Hard-coded to be bypassed for `senderFilter=self`
+- **`ignore_bots` override**: Hard-coded to be bypassed for `sender_filter=self`
so bot-owners who are themselves bots still match.
- **Cancellation path**: If the trigger is cancelled mid-wait, the handler
returns `success=false, error="Cancelled by user"` rather than swallowing.
-- **Handler accepts unknown trigger types**: `handle_trigger_node` has a guard
- for missing `TriggerConfig`, but because the registry entry for
- `telegramReceive` is hard-coded this branch is unreachable in practice.
- **No timeout**: The node waits indefinitely. The only exit routes are an
event match, an explicit `cancel_event_wait` WebSocket call, or the server
restarting.
@@ -154,4 +156,4 @@ flowchart TD
- **Sibling nodes**: [`telegramSend`](./telegramSend.md), [`socialReceive`](./socialReceive.md)
- **Event waiter architecture**: [Event Waiter System](../../event_waiter_system.md)
-- **Service**: [`server/services/telegram_service.py`](../../../server/services/telegram_service.py)
+- **Service**: [`server/nodes/telegram/_service.py`](../../../server/nodes/telegram/_service.py)
diff --git a/docs-internal/node-logic-flows/telegram_social/telegramSend.md b/docs-internal/node-logic-flows/telegram_social/telegramSend.md
index 4fc434cc..dbb16314 100644
--- a/docs-internal/node-logic-flows/telegram_social/telegramSend.md
+++ b/docs-internal/node-logic-flows/telegram_social/telegramSend.md
@@ -2,19 +2,19 @@
| Field | Value |
|------|-------|
-| **Category** | social / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/telegram.py::handle_telegram_send`](../../../server/services/handlers/telegram.py) |
+| **Category** | social (workflow-only) |
+| **Backend handler** | [`server/nodes/telegram/telegram_send.py`](../../../server/nodes/telegram/telegram_send.py) (`TelegramSendNode`); dispatch via `BaseNode.execute()` -> `@Operation("send")` |
| **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) |
| **Skill (if any)** | none |
-| **Dual-purpose tool** | yes - group is `['social', 'tool']`; exposed to AI agents via the unified send path |
+| **Dual-purpose tool** | no - group is `("social",)` only; `usable_as_tool` not set (AI-tool exposure was dropped in Wave 11) |
## Purpose
Send text, photo, document, location, or contact messages through a connected
Telegram bot (python-telegram-bot v22.x). The node leans on the `TelegramService`
-singleton for the actual Bot API calls; the handler only does recipient
-resolution, parameter validation, and envelope packaging. When used as an AI
-tool, the LLM fills the same snake_case parameter schema documented below.
+singleton for the actual Bot API calls; the `send` operation only does recipient
+resolution, parameter validation, and envelope packaging. The operation body is
+inlined directly in the plugin file (no `handlers/telegram.py` shim).
## Inputs (handles)
@@ -32,8 +32,8 @@ tool, the LLM fills the same snake_case parameter schema documented below.
| `text` | string | `""` | yes when `message_type=text` | `message_type: ['text']` | Message text |
| `media_url` | string | `""` | yes when `message_type` in `photo`/`document` | `message_type: ['photo','document']` | Remote URL or `file_id` |
| `caption` | string | `""` | no | `message_type: ['photo','document']` | Optional caption |
-| `latitude` | number | `0` | yes when `message_type=location` | `message_type: ['location']` | Latitude |
-| `longitude` | number | `0` | yes when `message_type=location` | `message_type: ['location']` | Longitude |
+| `latitude` | number (Optional[float], default `None`) | `None` | yes when `message_type=location` | `message_type: ['location']` | Latitude. `None` (not `0`) so deliberate `0.0` (Null Island) is distinguishable from "unset" |
+| `longitude` | number (Optional[float], default `None`) | `None` | yes when `message_type=location` | `message_type: ['location']` | Longitude |
| `phone_number` | string | `""` | yes when `message_type=contact` | `message_type: ['contact']` | Contact phone |
| `first_name` | string | `""` | yes when `message_type=contact` | `message_type: ['contact']` | Contact first name |
| `last_name` | string | `""` | no | `message_type: ['contact']` | Contact last name |
@@ -46,10 +46,12 @@ tool, the LLM fills the same snake_case parameter schema documented below.
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Standard envelope containing Bot API message metadata |
-| `output-tool` | object | Same payload, emitted when wired into an AI agent's `input-tools` |
### Output payload
+The operation returns this dict (validated against `TelegramSendOutput`, which
+declares `message_id` / `chat_id` / `sent` and `extra="allow"`):
+
```ts
{
message_id: number;
@@ -59,14 +61,14 @@ tool, the LLM fills the same snake_case parameter schema documented below.
}
```
-Wrapped in `{ success: true, result: <payload>, execution_time, timestamp, node_id, node_type }`.
+`BaseNode.execute()` wraps it as `{ success: true, result: <payload>, execution_time, timestamp, node_id, node_type }`.
## Logic Flow
```mermaid
flowchart TD
- A[handle_telegram_send] --> B{TelegramService<br/>connected?}
- B -- no --> Efail[Return success=false<br/>error: bot not connected]
+ A[TelegramSendNode.send op] --> B{TelegramService<br/>connected?}
+ B -- no --> Efail[raise RuntimeError<br/>BaseNode wraps as success=false]
B -- yes --> C{recipient_type?}
C -- self --> D[chat_id = service.owner_chat_id]
D --> D2{owner set?}
@@ -86,7 +88,7 @@ flowchart TD
G -- document --> Doc{media_url set?}
Doc -- no --> Efail
Doc -- yes --> Dsend[service.send_document]
- G -- location --> L{lat and lng set?}
+ G -- location --> L{lat and lng<br/>not None?}
L -- no --> Efail
L -- yes --> Lsend[service.send_location]
G -- contact --> Ct{phone + first_name?}
@@ -103,32 +105,34 @@ flowchart TD
## Decision Logic
-- **Connection gate**: If `service.connected` is False the handler short-circuits with
- `"Telegram bot not connected. Add bot token in Credentials."` No retry is attempted.
+- **Connection gate**: If `service.connected` is False the op raises
+ `RuntimeError("Telegram bot not connected. Add bot token in Credentials.")`
+ (wrapped by `BaseNode.execute()` into a `success=false` envelope). No retry.
- **Self recipient**: Uses `service.owner_chat_id` (auto-captured on the first
- private message). If not set, the handler tries
- `auth_service.get_api_key("telegram_owner_chat_id")` and restores the owner by
- calling `service.set_owner(int(saved_owner))`. Any exception during restore is
- logged at WARNING and swallowed - the handler continues and ultimately returns
- the "Bot owner not detected" error envelope when the chat_id is still falsy.
+ private message). If not set, the op tries
+ `get_auth_service().get_api_key("telegram_owner_chat_id")` and restores the owner by
+ calling `service.set_owner(int(saved))`. Any exception during restore is
+ logged at WARNING and swallowed - the op continues and ultimately raises
+ the "Bot owner not detected" `RuntimeError` when the chat_id is still falsy.
- **Parse mode**: Passed through to the service. The service itself implements
`Auto` (GFM -> Telegram HTML via `markdown_formatter.to_telegram_html`) plus the
`BadRequest "can't parse entities"` fallback that retries with `parse_mode=None`
and the original unescaped text.
-- **Message type dispatch**: Uses a Python `match` statement. An unknown
- `message_type` falls into the `_` arm and returns
- `"Unsupported message type: <x>"`.
+- **Message type dispatch**: Uses an `if/elif/else` chain on `message_type`. An
+ unknown value falls into the `else` and raises
+ `RuntimeError("Unsupported message type: <x>")`.
- **Reply-to coercion**: `reply_to_message_id` is cast via `int(...)` only when
- truthy. A non-numeric string here raises `ValueError`, which is caught by the
- outer `except Exception` and returned as an error envelope.
+ truthy. A non-numeric string here raises `ValueError`, which `BaseNode.execute()`
+ catches and surfaces as an error envelope.
## Side Effects
-- **Database writes**: none from the handler directly. `handle_telegram_send`
- can call `auth_service.store_api_key("telegram_owner_chat_id", ...)` indirectly
- only through `TelegramService._on_message_received`, not from the send path.
-- **Broadcasts**: none from the send handler. The service broadcasts
- `update_telegram_status` on connect / disconnect only.
+- **Database writes**: none from the op directly. The owner is persisted via
+ `auth_service.store_api_key("telegram_owner_chat_id", ...)` only inside
+ `TelegramService._on_message_received`, not from the send path.
+- **Broadcasts**: none from the send op. The service broadcasts the typed
+ `telegram.status` CloudEvents envelope (wire key `telegram_status`) on
+ connect / disconnect only - see `server/nodes/telegram/_events.py`.
- **External API calls**: Telegram Bot API via `python-telegram-bot` (`bot.send_message`,
`bot.send_photo`, `bot.send_document`, `bot.send_location`, `bot.send_contact`).
- **File I/O**: none.
@@ -138,8 +142,9 @@ flowchart TD
- **Credentials**: `auth_service.get_api_key("telegram_owner_chat_id")` when
restoring the self-owner. Bot token comes from `auth_service.store_api_key("telegram", ...)`
- and is consumed by `TelegramService.connect()` elsewhere, not by this handler.
-- **Services**: `TelegramService` singleton (`services.telegram_service.get_telegram_service`).
+ and is consumed by `TelegramService.connect()` elsewhere, not by this op.
+ Declared on the plugin via `credentials = (TelegramCredential,)`.
+- **Services**: `TelegramService` singleton (`server/nodes/telegram/_service.py::get_telegram_service`).
- **Python packages**: `python-telegram-bot` (v22.x), `markdown-it-py` (via
`markdown_formatter`).
- **Environment variables**: `TELEGRAM_OWNER_CHAT_ID` (read by the service as a
@@ -151,11 +156,10 @@ flowchart TD
`telegram_owner_chat_id` is logged at WARNING and continues; the user only
sees the downstream "Bot owner not detected" error.
- **Silent ValueError on `reply_to_message_id`**: A non-numeric value raises
- and the outer `except Exception` returns `str(e)` as the error message.
-- **`recipient_type` spelling**: The handler treats anything that is not
- literally `"self"` as needing a `chat_id`, including unexpected values -
- this means a typo like `"slf"` falls through to the user/group branch and
- the user sees `"chat_id is required"` instead of a recipient-type error.
+ `ValueError`, surfaced by `BaseNode.execute()` as the error message.
+- **`recipient_type` is a `Literal["self","user","group"]`**: Pydantic rejects
+ any other value at param validation. For `user`/`group` an empty `chat_id`
+ raises `RuntimeError("chat_id is required")`.
- **Markdown fallback leaks unescaped text**: If `send_message` raises
`BadRequest("can't parse entities ...")`, the service retries with the
ORIGINAL (unescaped) text at `parse_mode=None`, not the escaped copy. If the
@@ -170,6 +174,6 @@ flowchart TD
## Related
- **Sibling nodes**: [`telegramReceive`](./telegramReceive.md), [`socialSend`](./socialSend.md)
-- **Service**: [`server/services/telegram_service.py`](../../../server/services/telegram_service.py)
+- **Service**: [`server/nodes/telegram/_service.py`](../../../server/nodes/telegram/_service.py)
- **Markdown formatter**: [`server/services/markdown_formatter.py`](../../../server/services/markdown_formatter.py)
- **Architecture docs**: [Credentials Encryption](../../credentials_encryption.md), [Status Broadcaster](../../status_broadcaster.md)
diff --git a/docs-internal/node-logic-flows/twitter/twitterReceive.md b/docs-internal/node-logic-flows/twitter/twitterReceive.md
index 8fdbc732..8edd30ec 100644
--- a/docs-internal/node-logic-flows/twitter/twitterReceive.md
+++ b/docs-internal/node-logic-flows/twitter/twitterReceive.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | social / trigger |
-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) + [`server/services/event_waiter.py::build_twitter_filter`](../../../server/services/event_waiter.py) |
+| **Backend handler** | Plugin [`server/nodes/twitter/twitter_receive/__init__.py`](../../../server/nodes/twitter/twitter_receive/__init__.py); `execute()` delegates to [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic). Filter: [`server/nodes/twitter/_filters.py::build_filter`](../../../server/nodes/twitter/_filters.py). Events: [`server/nodes/twitter/_events.py`](../../../server/nodes/twitter/_events.py). |
| **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no (trigger only) |
@@ -18,8 +18,17 @@ no push webhooks on the free tier); polling is driven by the deployment layer
register a waiter, block on `event_waiter.wait_for_event`, and surface the
matching event.
-Filtering is done by the closure returned from `build_twitter_filter` in
-`event_waiter.py`.
+**Canary status**: `twitterReceive` is the lone **deferred** trigger plugin —
+it is NOT in the canary registry (`register_canary_trigger_type` is not
+called). The Temporal-durable `TriggerListenerWorkflow` / `PollingTriggerWorkflow`
+path requires the node to subclass `PollingTriggerNode` and declare the four
+hooks (same shape as `GmailReceiveNode`); until that refactor lands it runs on
+the legacy `event_waiter` collector/processor path. `_events.py` carries the
+typed `WorkflowEvent` factory + dual-path dispatcher (`dispatch_twitter_event_received`)
+ready for canary opt-in.
+
+Filtering is done by the closure returned from `build_filter` in
+`nodes/twitter/_filters.py` (auto-registered into `event_waiter.FILTER_BUILDERS`).
## Inputs (handles)
@@ -40,13 +49,13 @@ Trigger node - no inputs.
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Raw event data returned by `event_waiter.wait_for_event` |
+| `output-main` | `TwitterReceiveOutput` | Raw event data returned by `event_waiter.wait_for_event`. The declared `Output` model (`tweet_id`, `text`, `author`, plus `extra="allow"`) passes through the full dispatched payload. |
### Output payload (contract)
The event shape is produced by whichever polling loop dispatches
-`twitter_event_received`. Documented fields the filter and downstream nodes
-rely on:
+`twitter_event_received` (legacy `event_type`, preserved in `_events.py`).
+Documented fields the filter and downstream nodes rely on:
```ts
{
@@ -72,7 +81,7 @@ The handler wraps this in `{ success: true, result: <event_data>, execution_time
flowchart TD
A[handle_trigger_node] --> B[event_waiter.get_trigger_config twitterReceive]
B -- None --> E1[Return success=false<br/>error: Unknown trigger type]
- B -- config --> C[event_waiter.register<br/>build_twitter_filter]
+ B -- config --> C[event_waiter.register<br/>_filters.build_filter]
C --> D[broadcaster.update_node_status waiting]
D --> E[await event_waiter.wait_for_event]
E -- CancelledError --> Ecx[Return success=false<br/>error: Cancelled by user]
@@ -82,15 +91,15 @@ flowchart TD
subgraph Producer
P1[Deployment polling task] --> P2[Fetch via XDK<br/>mentions/search/user_timeline]
P2 --> P3[event_waiter.dispatch<br/>twitter_event_received]
- P3 --> P4[build_twitter_filter.matches]
+ P3 --> P4[_filters.build_filter.matches]
P4 -- match --> P5[Waiter.future.set_result]
end
```
## Decision Logic (filter)
-`build_twitter_filter(params)` returns a `matches(data)` closure that rejects
-non-matching events:
+`build_filter(params)` (in `nodes/twitter/_filters.py`) returns a `matches(data)`
+closure that rejects non-matching events:
- `trigger_type != 'all'` and `data['trigger_type'] != trigger_type` -> reject.
(Frontend only exposes `mentions` / `search` / `timeline`; there is no way to
@@ -101,7 +110,7 @@ non-matching events:
- **user_timeline branch**: if `trigger_type == 'user_timeline'` and `user_id`
is set, require `data['user_id'] == user_id` (exact). Empty `user_id`
accepts any timeline event.
-- No `filter_retweets` / `filter_replies` logic exists in `build_twitter_filter`.
+- No `filter_retweets` / `filter_replies` logic exists in `build_filter`.
### Execution-side handler branches
diff --git a/docs-internal/node-logic-flows/twitter/twitterSearch.md b/docs-internal/node-logic-flows/twitter/twitterSearch.md
index 0711a005..72363026 100644
--- a/docs-internal/node-logic-flows/twitter/twitterSearch.md
+++ b/docs-internal/node-logic-flows/twitter/twitterSearch.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | social / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/twitter.py::handle_twitter_search`](../../../server/services/handlers/twitter.py) |
+| **Backend handler** | [`server/nodes/twitter/twitter_search/__init__.py`](../../../server/nodes/twitter/twitter_search/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("search")` -> `_do_search` (helpers in [`_base.py`](../../../server/nodes/twitter/_base.py)) |
| **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) |
| **Skill (if any)** | [`server/skills/social_agent/twitter-search-skill/SKILL.md`](../../../server/skills/social_agent/twitter-search-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `twitter_search` |
@@ -40,11 +40,12 @@ The first result page is returned - pagination is not surfaced to the user.
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when invoked via `input-tools` |
+| `output-main` | `TwitterSearchOutput` | Search results; also returned to the LLM when invoked via `input-tools` |
### Output payload
+`_do_search` returns `TwitterSearchOutput(tweets=..., count=..., query=...)` (`extra="allow"`). Validated + dumped by `BaseNode._serialize_result`:
+
```ts
{
tweets: Array<{
@@ -98,55 +99,56 @@ The first result page is returned - pagination is not surfaced to the user.
}
```
-Wrapped in `{ success, result, execution_time }`.
-
## Logic Flow
```mermaid
flowchart TD
- A[handle_twitter_search] --> B[_get_twitter_client]
- B -- no tokens --> Eauth[Return success=false<br/>error: Twitter not connected]
- B --> C{query non-empty?}
- C -- no --> Eq[Return success=false<br/>error: Search query is required]
+ A[BaseNode.execute -> @Operation search] --> C{query non-empty?}
+ C -- no --> Eq[raise RuntimeError<br/>Search query is required]
C -- yes --> D[max_results clamp<br/>max 10, min requested, 100]
- D --> E[to_thread _sync_search_recent<br/>tweet_fields + expansions + media + user]
- E -. 401/403 .-> R[_refresh_and_get_client] --> E
- E -. other Exception .-> EGEN[Return success=false<br/>error: str e]
- E --> F[Build users_by_id, media_by_key, tweets_by_id<br/>from includes]
- F --> G[Map raw tweets -> _format_tweet]
+ D --> CWR[call_with_retry _do_search]
+ CWR --> B[get_twitter_client]
+ B -- no access_token --> Eauth[raise RuntimeError<br/>Twitter not connected]
+ B --> E[to_thread sync_search_recent<br/>tweet_fields + expansions + media + user]
+ E -. 401/403 .-> R[refresh_and_get_client] --> E
+ E -. other Exception .-> EGEN[propagate -> BaseNode error envelope]
+ E --> F[includes_lookups -> users_by_id, media_by_key, tweets_by_id]
+ F --> G[Map raw tweets -> format_tweet]
G --> H{tweets non-empty?}
- H -- yes --> K[_track_twitter_usage search count=len tweets]
+ H -- yes --> K[track_twitter_usage search count=len tweets]
H -- no --> OK
- K --> OK[Return success=true<br/>result={tweets, count, query}]
+ K --> OK[Return TwitterSearchOutput<br/>tweets, count, query]
```
## Decision Logic
-- **Validation**: missing/empty `query` -> error envelope, no API call.
+- **Validation**: missing/empty `query` raises `RuntimeError` in `search()`
+ before any client is acquired or API call made.
- **max_results clamp**: `max(10, min(params.max_results, 100))`. Requests below
10 are silently raised; above 100 are silently capped.
-- **SDK call**: `_sync_search_recent` iterates `client.posts.search_recent(...)`
+- **SDK call**: `sync_search_recent` iterates `client.posts.search_recent(...)`
once and returns `{tweets, includes}` for the first page. Subsequent pages
are discarded.
- **Expansions assembled**:
- `users_by_id[author_id]` from `includes.users`
- `media_by_key[media_key]` from `includes.media`
- `tweets_by_id[referenced.id]` from `includes.tweets`
-- **`_format_tweet` enrichment**:
+- **`format_tweet` enrichment**:
- Replaces every `t.co` shortlink in `text` with `expanded_url` in `display_text`.
- If `note_tweet.text` exists, it replaces both `text` and `display_text`.
- Attaches `author`, `urls`, `media`, `referenced_tweets` as optional keys
(only set when data is non-empty).
- Coerces Pydantic objects -> dicts via `model_dump()` where needed.
-- **Auth refresh**: same lazy-refresh loop as `twitterSend` - a single retry
- via `_refresh_and_get_client` on `_is_auth_error(e)`.
+- **Auth refresh**: same lazy-refresh loop as `twitterSend` (`call_with_retry`) -
+ a single retry via `refresh_and_get_client` on `_is_auth_error(e)`.
- **Usage tracking**: `resource_count` equals the number of tweets returned,
not 1. Skipped entirely when results are empty.
## Side Effects
- **Database writes**: one `api_usage_metrics` row per non-empty search (op
- `search`, `resource_count=len(tweets)`) via `database.save_api_usage_metric`.
+ `search`, `resource_count=len(tweets)`) via `track_twitter_usage` ->
+ `database.save_api_usage_metric`.
- **Broadcasts**: none.
- **External API calls**:
- `GET https://api.twitter.com/2/tweets/search/recent` with `expansions=author_id,attachments.media_keys,referenced_tweets.id,referenced_tweets.id.author_id`, `tweet_fields=author_id,created_at,entities,public_metrics,possibly_sensitive,lang,source,conversation_id,in_reply_to_user_id,referenced_tweets,note_tweet`, `media_fields=url,preview_image_url,type,alt_text,variants`, `user_fields=username,name,profile_image_url`.
@@ -167,14 +169,14 @@ flowchart TD
- **Unused frontend params**: `sort_order`, `start_time`, `end_time`,
`include_metrics`, `include_author` are wired in the UI but the handler
does not forward any of them to the SDK.
-- **Only first page returned**: `_sync_search_recent` breaks out of the
+- **Only first page returned**: `sync_search_recent` breaks out of the
generator after the first page regardless of `next_token`.
- **Empty-includes tolerance**: when the API returns no `includes`, `author`,
`media`, `urls`, and `referenced_tweets` are simply omitted from each tweet -
consumers must treat them as optional.
- **Referenced tweet id not in includes**: `referenced_tweets[].text` is
`null` rather than raising.
-- **Object/dict duality in `includes`**: `_format_tweet` handles both Pydantic
+- **Object/dict duality in `includes`**: `format_tweet` handles both Pydantic
objects and raw dicts - changes to XDK's model shapes may silently drop
fields.
- **Long-form tweets**: `note_tweet.text` replaces the 280-char `text`; callers
diff --git a/docs-internal/node-logic-flows/twitter/twitterSend.md b/docs-internal/node-logic-flows/twitter/twitterSend.md
index 65275acf..93af92fd 100644
--- a/docs-internal/node-logic-flows/twitter/twitterSend.md
+++ b/docs-internal/node-logic-flows/twitter/twitterSend.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | social / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/twitter.py::handle_twitter_send`](../../../server/services/handlers/twitter.py) |
+| **Backend handler** | [`server/nodes/twitter/twitter_send/__init__.py`](../../../server/nodes/twitter/twitter_send/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("send")` -> `_do_send` (helpers in [`_base.py`](../../../server/nodes/twitter/_base.py)) |
| **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) |
| **Skill (if any)** | [`server/skills/social_agent/twitter-send-skill/SKILL.md`](../../../server/skills/social_agent/twitter-send-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `twitter_send` |
@@ -28,105 +28,101 @@ wraps every call in `asyncio.to_thread(...)` so the event loop is never blocked.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `action` | options | `tweet` | yes | - | One of `tweet` / `reply` / `retweet` / `quote` / `like` / `unlike` / `delete`. Backend only implements `tweet`, `reply`, `retweet`, `like`, `unlike`, `delete`; `quote` falls through to the `Unknown action` branch. |
-| `text` | string | `""` | yes (tweet/reply/quote) | `action: ['tweet','reply','quote']` | Tweet content. Handler truncates to 280 chars via slice. |
-| `tweet_id` | string | `""` | yes (reply/retweet/quote/like/unlike/delete) | `action: ['reply','retweet','quote','like','unlike','delete']` | Target tweet ID. |
-| `reply_to_id` | string | `""` | yes (reply) | - | ID of the tweet being replied to. Passed in `reply.in_reply_to_tweet_id`. Note the parameter name mismatch vs. frontend `tweet_id` for other actions. |
-| `include_media`, `media_urls` | boolean/string | - | no | tweet/reply/quote | **Accepted by frontend but ignored by backend** - the handler never attaches media. |
-| `include_poll`, `poll_options`, `poll_duration` | bool/string/number | - | no | tweet | **Accepted by frontend but ignored by backend.** |
+| `action` | options (Literal) | `tweet` | yes | - | One of `tweet` / `reply` / `retweet` / `quote` / `like` / `unlike` / `delete`. All implemented in `_do_send`. |
+| `text` | string (rows: 4) | `""` | yes (tweet/reply/quote) | `action: ['tweet','reply','quote']` | Tweet content. `_do_send` truncates to 280 chars via slice (`text[:280]`). |
+| `tweet_id` | string | `""` | yes (reply/retweet/quote/like/unlike/delete) | `action: ['reply','retweet','quote','like','unlike','delete']` | Target tweet ID. Used by reply (`reply.in_reply_to_tweet_id`), quote (`quote_tweet_id`), and the id-based actions. |
+| `include_media` | boolean | `false` | no | `action: ['tweet','reply','quote']` | **Accepted by Params but ignored by `_do_send`** — the operation never attaches media. |
+| `media_urls` | string | `""` | no | `action: ['tweet','reply','quote']`, `include_media: [true]` | **Accepted by Params but ignored by `_do_send`.** Comma-separated URLs; max 4 images or 1 video. |
+| `include_poll` | boolean | `false` | no | `action: ['tweet']` | **Accepted by Params but ignored by `_do_send`.** |
+| `poll_options` | string | `""` | no | `action: ['tweet']`, `include_poll: [true]` | **Accepted but ignored.** Comma-separated 2-4 options, 25 chars each. |
+| `poll_duration` | number (5-10080) | `1440` | no | `action: ['tweet']`, `include_poll: [true]` | **Accepted but ignored.** Poll duration in minutes. |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope with XDK response payload |
-| `output-tool` | object | Same payload when invoked via `input-tools` |
+| `output-main` | `TwitterSendOutput` | Result of the action; also returned to the LLM when invoked via `input-tools` |
### Output payload
-Handler wraps the XDK response via `_format_response` then via `_success`:
+`_do_send` returns a `TwitterSendOutput` (`ConfigDict(extra="allow")`); `BaseNode._serialize_result` validates and dumps it:
```ts
-// On success (tweet/reply):
{
- success: true,
- result: {
- id: string; // new tweet id
- text: string;
- // ...any additional fields returned by POST /2/tweets (edit_history_tweet_ids, etc.)
- },
- execution_time: number
+ action: string; // "tweet_sent" | "reply_sent" | "quoted" | "retweeted" | "liked" | "unliked" | "deleted"
+ data: object; // _format_response(...) of the XDK result (e.g. { id, text, ... } for create)
}
-// On success (retweet/like): { retweeted: boolean } or { liked: boolean }
-// On success (unlike): { liked: false }
-// On success (delete): { deleted: boolean }
-// On error: { success: false, error: string, execution_time: number }
```
+> Note: the runtime `Output` model above is the plugin's authoritative shape. The frontend drag-source schema in [`node_output_schemas.py`](../../../server/services/node_output_schemas.py) (`TwitterSendOutput`: `tweet_id`, `text`, `author_id`, `created_at`, `action`) is a separate, flatter advertised shape and does not match the live envelope field-for-field.
+
## Logic Flow
```mermaid
flowchart TD
- A[handle_twitter_send] --> B[_get_twitter_client<br/>auth_service.get_oauth_tokens twitter]
- B -- no tokens --> Eauth[Return success=false<br/>error: Twitter not connected]
- B -- ok --> C[_do_twitter_send]
+ A[BaseNode.execute -> @Operation send] --> CWR[call_with_retry _do_send]
+ CWR --> B[get_twitter_client<br/>auth_service.get_oauth_tokens twitter]
+ B -- no access_token --> Eauth[raise RuntimeError<br/>Twitter not connected]
+ B -- ok --> C[_do_send]
C --> Dswitch{action}
Dswitch -- tweet --> T1{text non-empty?}
- T1 -- no --> Eval[raise ValueError]
- T1 -- yes --> T2[asyncio.to_thread _sync_create_post<br/>body=text 0..280]
- Dswitch -- reply --> R1{text AND reply_to_id?}
+ T1 -- no --> Eval[raise RuntimeError]
+ T1 -- yes --> T2[to_thread client.posts.create<br/>body=text 0..280]
+ Dswitch -- reply --> R1{text AND tweet_id?}
R1 -- no --> Eval
- R1 -- yes --> R2[to_thread _sync_create_post<br/>reply.in_reply_to_tweet_id]
- Dswitch -- retweet/like --> L1[to_thread _get_my_user_id<br/>client.users.get_me]
- L1 --> L2[to_thread _sync_repost or _sync_like]
- Dswitch -- unlike --> U1[to_thread _get_my_user_id]
- U1 --> U2[to_thread _sync_unlike]
- Dswitch -- delete --> D1[to_thread _sync_delete_post]
- Dswitch -- _ --> Eunk[raise ValueError Unknown action]
- T2 --> K[_track_twitter_usage -> DB]
+ R1 -- yes --> R2[to_thread posts.create<br/>reply.in_reply_to_tweet_id]
+ Dswitch -- quote --> Q1{text AND tweet_id?}
+ Q1 -- yes --> Q2[to_thread posts.create<br/>quote_tweet_id]
+ Dswitch -- retweet/like --> L1[to_thread get_my_user_id<br/>client.users.get_me]
+ L1 --> L2[to_thread repost_post or like_post]
+ Dswitch -- unlike --> U1[to_thread get_my_user_id]
+ U1 --> U2[to_thread unlike_post]
+ Dswitch -- delete --> D1[to_thread posts.delete]
+ Dswitch -- _ --> Eunk[raise RuntimeError Unknown action]
+ T2 --> K[track_twitter_usage -> DB]
R2 --> K
+ Q2 --> K
L2 --> K
U2 --> K
D1 --> K
- K --> OK[Return success=true envelope]
+ K --> OK[Return TwitterSendOutput]
C -. raises .-> AuthErr{_is_auth_error?<br/>401/403/Unauthorized/Forbidden}
- AuthErr -- yes --> REF[_refresh_and_get_client<br/>TwitterOAuth.refresh_access_token]
+ AuthErr -- yes --> REF[refresh_and_get_client<br/>TwitterOAuth.refresh_access_token]
REF -. new tokens .-> STORE[auth_service.store_oauth_tokens twitter]
- STORE --> C
- AuthErr -- no --> EGEN[Return success=false<br/>error: str e]
+ STORE --> C2[retry _do_send once]
+ AuthErr -- no --> EGEN[propagate -> BaseNode error envelope]
```
## Decision Logic
-- **Client acquisition**: `_get_twitter_client()` reads OAuth tokens via
- `auth_service.get_oauth_tokens("twitter", customer_id="owner")`. Raises
- `ValueError("Twitter not connected. Please authenticate via Credentials.")`
- when tokens are missing.
-- **No eager validation**: Handler never calls `get_me()` up-front to verify the
+- **Client acquisition**: `get_twitter_client()` (in `_base.py`) reads OAuth
+ tokens via `auth_service.get_oauth_tokens("twitter", customer_id="owner")`.
+ Raises `RuntimeError("Twitter not connected. Please authenticate via Credentials.")`
+ when the access token is missing.
+- **No eager validation**: never calls `get_me()` up-front to verify the
token; it jumps straight into the action call to conserve rate limits.
-- **Lazy refresh**: Any exception bubbling out of `_do_twitter_send` is inspected
- by `_is_auth_error` (substring match on `401`, `403`, `Unauthorized`,
- `Forbidden`). On match, a new client is built via `_refresh_and_get_client()`
- which calls `TwitterOAuth.refresh_access_token`, re-stores tokens via
- `store_oauth_tokens`, and the action is retried once. Non-auth errors surface
- as an error envelope.
-- **Validation**:
+- **Lazy refresh (`call_with_retry`)**: any exception bubbling out of `_do_send`
+ is inspected by `_is_auth_error` (substring match on `401`, `403`,
+ `Unauthorized`, `Forbidden`). On match, a new client is built via
+ `refresh_and_get_client()` which calls `TwitterOAuth.refresh_access_token`,
+ re-stores tokens via `store_oauth_tokens`, and `_do_send` is retried once.
+ Non-auth errors propagate to `BaseNode.execute()`'s error envelope.
+- **Validation** (raise `RuntimeError` on failure):
- `tweet`: requires non-empty `text`, sliced to 280 chars.
- - `reply`: requires both `text` and `reply_to_id`.
+ - `reply`/`quote`: require both `text` and `tweet_id`.
- `retweet`/`like`/`unlike`/`delete`: require `tweet_id`.
- **User id fetch**: `retweet`/`like`/`unlike` need the authenticated user id;
- `_get_my_user_id` calls `client.users.get_me()` via `asyncio.to_thread`.
-- **Unknown action**: `quote` (present in frontend) and any other value fall
- into the default match arm and raise `ValueError(f"Unknown action: {action}")`
- which is converted to an error envelope.
+ `get_my_user_id` calls `client.users.get_me()` via `asyncio.to_thread`.
+- **Unknown action**: any value outside the `Literal` set raises
+ `RuntimeError(f"Unknown action: {action}")` (unreachable via the UI dropdown).
## Side Effects
- **Database writes**: one row per successful action in `api_usage_metrics` via
- `database.save_api_usage_metric(...)` with `service='twitter'` and
- `operation` from `PricingService.calculate_api_cost`. Action -> op name:
- `tweet`, `reply`, `retweet`, `like`, `unlike`, `delete`.
+ `track_twitter_usage` -> `database.save_api_usage_metric(...)` with
+ `service='twitter'` and `operation` from `PricingService.calculate_api_cost`.
+ Action -> op name: `tweet`, `reply`, `quote`, `retweet`, `like`, `unlike`, `delete`.
- **Broadcasts**: none from the handler.
- **External API calls**:
- `POST https://api.twitter.com/2/tweets` (tweet, reply)
@@ -149,7 +145,7 @@ flowchart TD
System 1 (`EncryptedOAuthToken` table); NOT via `get_api_key("twitter_access_token")`.
Client id + secret are stored via `get_api_key("twitter_client_id")` /
`get_api_key("twitter_client_secret")` and only consulted on refresh.
-- **Services**: `TwitterOAuth` (`server/services/twitter_oauth.py`) for token
+- **Services**: `TwitterOAuth` (`server/nodes/twitter/_oauth.py`) for token
refresh. `PricingService` for cost calculation. `Database` for metric row.
- **Python packages**: `xdk` (X SDK), `httpx` (used transitively inside
`TwitterOAuth`).
@@ -157,19 +153,17 @@ flowchart TD
## Edge cases & known limits
-- **`quote` action is broken**: frontend exposes `Quote Tweet` but backend has no
- handler branch; it falls through to the default arm and surfaces as
- `Unknown action: quote`.
+- **`quote` action is implemented**: builds `posts.create(body={"text":..., "quote_tweet_id": tweet_id})`.
- **Media and polls ignored**: the `include_media` / `include_poll` /
`media_urls` / `poll_options` / `poll_duration` parameters are never read by
- the handler - the node silently sends a text-only tweet.
+ `_do_send` - the node silently sends a text-only tweet.
- **280-char silent truncation**: `text[:280]` is applied without warning; the
emitted tweet may be shorter than the user typed.
- **Single retry on auth error**: refresh + retry happens at most once; a
- second 401 surfaces as an error envelope.
-- **Refresh failure message**: If `refresh_access_token` fails, the user sees
- `Twitter token expired and refresh failed. Please re-authenticate.` - the
- original upstream error message is lost.
+ second 401 propagates to the error envelope.
+- **Refresh failure message**: if `refresh_access_token` fails, the user sees
+ `Twitter token refresh failed. Please re-authenticate.`; the missing-refresh-token
+ case surfaces `Twitter token expired. Please re-authenticate.`
- **`_is_auth_error` uses substring matching**: any error whose `str(e)`
contains `401`, `403`, `Unauthorized`, or `Forbidden` triggers the refresh
path, even if the real cause is something else (e.g. a log message quoting
diff --git a/docs-internal/node-logic-flows/twitter/twitterUser.md b/docs-internal/node-logic-flows/twitter/twitterUser.md
index 83288119..3625d0ee 100644
--- a/docs-internal/node-logic-flows/twitter/twitterUser.md
+++ b/docs-internal/node-logic-flows/twitter/twitterUser.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | social / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/twitter.py::handle_twitter_user`](../../../server/services/handlers/twitter.py) |
+| **Backend handler** | [`server/nodes/twitter/twitter_user/__init__.py`](../../../server/nodes/twitter/twitter_user/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("lookup")` -> `_do_lookup` (helpers in [`_base.py`](../../../server/nodes/twitter/_base.py)) |
| **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) |
| **Skill (if any)** | [`server/skills/social_agent/twitter-user-skill/SKILL.md`](../../../server/skills/social_agent/twitter-user-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `twitter_user` |
@@ -28,104 +28,100 @@ workflow node and an AI agent tool. Every SDK call is wrapped in
|------|------|---------|----------|---------------------|-------------|
| `operation` | options | `me` | yes | - | `me` / `by_username` / `by_id` / `followers` / `following` |
| `username` | string | `""` | yes (by_username) | `operation: ['by_username']` | Handle without `@`. |
-| `user_id` | string | `""` | yes (by_id), optional (followers/following) | `operation: ['by_id','followers','following']` | If omitted for followers/following, the authenticated user is used via `_get_my_user_id`. |
+| `user_id` | string | `""` | yes (by_id), optional (followers/following) | `operation: ['by_id','followers','following']` | If omitted for followers/following, the authenticated user is used via `get_my_user_id`. |
| `max_results` | number | `100` | no | `operation: ['followers','following']` | Clamped `max(1, min(requested, 1000))`. |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when invoked via `input-tools` |
+| `output-main` | `TwitterUserOutput` | Lookup result; also returned to the LLM when invoked via `input-tools` |
### Output payload
-`me` / `by_username` / `by_id`:
+`_do_lookup` returns a `TwitterUserOutput(operation, user?, users?, count?)` (`extra="allow"`); validated + dumped by `BaseNode._serialize_result`.
+
+`me` / `by_username` / `by_id` -> `{ operation, user }` where `user` is `format_user(...)`:
```ts
{
- id: string;
- username: string;
- name: string;
- profile_image_url: string | null;
- verified: boolean; // defaults to false when API omits it
- description: string | null;
- created_at: string; // str(...) - empty string when missing
+ operation: string;
+ user: {
+ id: string;
+ username: string;
+ name: string;
+ profile_image_url: string | null;
+ verified: boolean; // defaults to false when API omits it
+ description: string | null;
+ created_at: string; // str(...) - empty string when missing
+ }
}
```
-`followers` / `following`:
+`followers` / `following` -> `{ operation, users, count }`:
```ts
{
- users: Array<UserData>; // same shape as above
+ operation: string;
+ users: Array<UserData>; // same shape as user above
count: number;
}
```
-All wrapped in `{ success, result, execution_time }`. `me` / `by_username` /
-`by_id` additionally take the `_success` path that wraps the dict as
-`result` directly (no `action` field leaks through when the payload is already
-a dict, thanks to `_success`'s dict-shortcut).
-
## Logic Flow
```mermaid
flowchart TD
- A[handle_twitter_user] --> B[_get_twitter_client]
- B -- no tokens --> Eauth[Return success=false<br/>error: Twitter not connected]
- B --> C[_do_twitter_user]
+ A[BaseNode.execute -> @Operation lookup] --> CWR[call_with_retry _do_lookup]
+ CWR --> B[get_twitter_client]
+ B -- no access_token --> Eauth[raise RuntimeError<br/>Twitter not connected]
+ B --> C[_do_lookup]
C --> Dswitch{operation}
- Dswitch -- me --> M1[to_thread _sync_get_me<br/>user_fields=created_at, description]
+ Dswitch -- me --> M1[to_thread users.get_me<br/>user_fields=created_at, description]
Dswitch -- by_username --> U1{username?}
- U1 -- no --> EV[raise ValueError Username is required]
- U1 -- yes --> U2[to_thread _sync_get_by_usernames]
+ U1 -- no --> EV[raise RuntimeError Username is required]
+ U1 -- yes --> U2[to_thread users.get_by_usernames]
U2 --> U3{data list empty?}
- U3 -- yes --> EVn[raise ValueError User @handle not found]
+ U3 -- yes --> EVn[raise RuntimeError User @handle not found]
Dswitch -- by_id --> I1{user_id?}
I1 -- no --> EV
- I1 -- yes --> I2[to_thread _sync_get_by_ids]
+ I1 -- yes --> I2[to_thread users.get_by_ids]
I2 --> I3{data list empty?}
I3 -- yes --> EVn
- Dswitch -- followers --> F1{user_id present?}
- F1 -- no --> F0[to_thread _get_my_user_id]
- F0 --> F2[to_thread _sync_get_followers]
+ Dswitch -- followers/following --> F1{user_id present?}
+ F1 -- no --> F0[to_thread get_my_user_id]
+ F0 --> F2[to_thread _sync_followers / _sync_following]
F1 -- yes --> F2
- Dswitch -- following --> G1{user_id present?}
- G1 -- no --> G0[to_thread _get_my_user_id]
- G0 --> G2[to_thread _sync_get_following]
- G1 -- yes --> G2
- Dswitch -- _ --> Eunk[raise ValueError Unknown operation]
- M1 --> K1[_format_user_data + _track_twitter_usage me 1]
- U2 --> K2[_format_user_data users 0 + _track_twitter_usage by_username 1]
- I2 --> K3[_format_user_data + _track_twitter_usage by_id 1]
- F2 --> K4[map _format_user_data + _track_twitter_usage followers len users]
- G2 --> K5[same for following]
- K1 --> OK[Return success envelope]
+ Dswitch -- _ --> Eunk[raise RuntimeError Unknown operation]
+ M1 --> K1[format_user + track_twitter_usage me 1]
+ U2 --> K2[format_user users 0 + track_twitter_usage by_username 1]
+ I2 --> K3[format_user + track_twitter_usage by_id 1]
+ F2 --> K4[map format_user + track_twitter_usage op len users]
+ K1 --> OK[Return TwitterUserOutput]
K2 --> OK
K3 --> OK
K4 --> OK
- K5 --> OK
- C -. raises 401/403 .-> REF[_refresh_and_get_client] --> C
- C -. other Exception .-> EGEN[Return success=false<br/>error: str e]
+ C -. raises 401/403 .-> REF[refresh_and_get_client] --> C2[retry _do_lookup once]
+ C -. other Exception .-> EGEN[propagate -> BaseNode error envelope]
```
## Decision Logic
-- **Operation dispatch** uses a Python 3.10+ `match` / `case` block. Any value
- other than the five documented operations raises `ValueError(f"Unknown operation: {operation}")`.
+- **Operation dispatch** uses an `if`/`elif` chain in `_do_lookup`. Any value
+ outside the five `Literal` operations raises `RuntimeError(f"Unknown operation: {op}")`
+ (unreachable via the UI dropdown).
- **Validation**:
- - `by_username`: empty `username` -> `ValueError("Username is required")`.
- - `by_id`: empty `user_id` -> `ValueError("User ID is required")`.
- - `followers` / `following`: missing `user_id` is **not** an error - handler
- falls back to `_get_my_user_id`.
-- **Not-found handling**: `by_username` / `by_id` raise `ValueError` when the
- SDK returns an empty `data` list; the string is propagated as the envelope
- `error`.
-- **Lazy auth refresh**: identical to `twitterSend` - any exception whose
- `str(e)` contains `401`/`403`/`Unauthorized`/`Forbidden` triggers one
- refresh-and-retry.
+ - `by_username`: empty `username` -> `RuntimeError("Username is required")`.
+ - `by_id`: empty `user_id` -> `RuntimeError("User ID is required")`.
+ - `followers` / `following`: missing `user_id` is **not** an error -
+ falls back to `get_my_user_id`.
+- **Not-found handling**: `by_username` / `by_id` raise `RuntimeError` when the
+ SDK returns an empty `data` list; it propagates to `BaseNode.execute()`'s
+ error envelope.
+- **Lazy auth refresh**: identical to `twitterSend` (`call_with_retry`) - any
+ exception whose `str(e)` contains `401`/`403`/`Unauthorized`/`Forbidden`
+ triggers one refresh-and-retry.
- **max_results clamping**: `max(1, min(requested, 1000))` for followers /
following. Anything else is ignored.
- **Usage tracking**:
@@ -161,12 +157,12 @@ flowchart TD
- **`max_results < 1`**: silently clamped to 1.
- **`max_results > 1000`**: silently clamped to 1000. The SDK may still paginate
internally; only the first page is returned regardless.
-- **Only first page for followers/following**: `_sync_get_followers` /
- `_sync_get_following` break after the first page - `next_token` is discarded.
+- **Only first page for followers/following**: `_sync_followers` /
+ `_sync_following` return after the first page - `next_token` is discarded.
- **Not-found vs. empty**: when the target user exists but has zero
followers/following, the handler returns `{users: [], count: 0}` - it does
NOT raise. `by_username` / `by_id` treat empty data as an error.
-- **`_get_my_user_id` cost not tracked**: the implicit `users/me` call made on
+- **`get_my_user_id` cost not tracked**: the implicit `users/me` call made on
followers/following with empty `user_id` is not reflected in
`api_usage_metrics`.
- **`verified` default is `False`**: even when the API omits the field the
diff --git a/docs-internal/node-logic-flows/web_automation/apifyActor.md b/docs-internal/node-logic-flows/web_automation/apifyActor.md
index 716b38cb..c938f447 100644
--- a/docs-internal/node-logic-flows/web_automation/apifyActor.md
+++ b/docs-internal/node-logic-flows/web_automation/apifyActor.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | web_automation / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/apify.py::handle_apify_actor`](../../../server/services/handlers/apify.py) |
+| **Backend handler** | [`server/nodes/scraper/apify_actor/__init__.py::ApifyActorNode`](../../../server/nodes/scraper/apify_actor/__init__.py) — dispatch via `BaseNode.execute()` -> `@Operation("run")` |
| **Tests** | [`server/tests/nodes/test_web_automation.py`](../../../server/tests/nodes/test_web_automation.py) |
| **Skill (if any)** | [`server/skills/web_agent/apify-skill/SKILL.md`](../../../server/skills/web_agent/apify-skill/SKILL.md) |
| **Dual-purpose tool** | yes |
@@ -28,30 +28,39 @@ curated actor ids (e.g. `instagramUrls` for `apify/instagram-scraper`).
## Parameters
+Params model is `ApifyActorParams` (field names are snake_case).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `actorId` | options/string | `""` | **yes** | - | Actor id (e.g. `apify/instagram-scraper`) or `custom` |
-| `customActorId` | string | `""` | yes if `actorId=custom` | `actorId=custom` | Used when `actorId=custom` |
-| `actorInput` | string (JSON) | `"{}"` | no | - | Raw JSON run-input; invalid JSON silently becomes `{}` |
-| `timeout` | number | `300` | no | - | Run timeout in seconds (SDK forwards as `timeout_secs`) |
-| `maxResults` | number | `100` | no | - | Forwarded to `dataset.list_items(limit=...)` |
-| `memory` | number | `1024` | no | - | Actor memory in MB (`memory_mbytes`) |
-| `instagramUrls` | string | `""` | no | `actorId=apify/instagram-scraper` | Comma-separated -> `directUrls[]` |
-| `tiktokProfiles` | string | `""` | no | `actorId=clockworks/tiktok-scraper` | Comma-separated -> `profiles[]` |
-| `tiktokHashtags` | string | `""` | no | `actorId=clockworks/tiktok-scraper` | Comma-separated -> `hashtags[]` |
-| `twitterSearchTerms` | string | `""` | no | `actorId=apidojo/tweet-scraper` | Comma-separated -> `searchTerms[]` |
-| `twitterHandles` | string | `""` | no | `actorId=apidojo/tweet-scraper` | Comma-separated -> `twitterHandles[]` |
-| `googleSearchQuery` | string | `""` | no | `actorId=apify/google-search-scraper` | -> `searchQuery` |
-| `googleSearchPages` | number | `1` | no | `actorId=apify/google-search-scraper` | -> `maxPagesPerQuery` |
-| `crawlerStartUrls` | string | `""` | no | `actorId=apify/website-content-crawler` | Comma-separated -> `startUrls[{url}]` |
-| `crawlerMaxDepth` | number | `2` | no | `actorId=apify/website-content-crawler` | -> `maxCrawlDepth` |
-| `crawlerMaxPages` | number | `50` | no | `actorId=apify/website-content-crawler` | -> `maxCrawlPages` |
+| `actor_id` | options (Literal) | `apify/instagram-scraper` | **yes** | - | One of 10 actor presets or `custom` |
+| `custom_actor_id` | string | `""` | yes if `actor_id=custom` | `actor_id=custom` | Used when `actor_id=custom` |
+| `actor_input` | Any (JSON) | `{}` (empty dict) | no | - | Raw JSON run-input; merged with quick helpers. String input that fails `json.loads` (or non-dict) silently becomes `{}` |
+| `instagram_urls` | string | `""` | no | `actor_id=apify/instagram-scraper` | Comma-separated -> `directUrls[]` |
+| `tiktok_profiles` | string | `""` | no | `actor_id=clockworks/tiktok-scraper` | Comma-separated -> `profiles[]` |
+| `tiktok_hashtags` | string | `""` | no | `actor_id=clockworks/tiktok-scraper` | Comma-separated -> `hashtags[]` |
+| `twitter_search_terms` | string | `""` | no | `actor_id=apidojo/tweet-scraper` | Comma-separated -> `searchTerms[]` |
+| `twitter_handles` | string | `""` | no | `actor_id=apidojo/tweet-scraper` | Comma-separated -> `twitterHandles[]` |
+| `google_search_query` | string | `""` | no | `actor_id=apify/google-search-scraper` | -> `searchQuery` |
+| `google_search_pages` | int (1-100) | `1` | no | `actor_id=apify/google-search-scraper` | -> `maxPagesPerQuery` |
+| `crawler_start_urls` | string | `""` | no | `actor_id=apify/website-content-crawler` | Comma-separated -> `startUrls[{url}]` |
+| `crawler_max_depth` | int (0-20) | `2` | no | `actor_id=apify/website-content-crawler` | -> `maxCrawlDepth` |
+| `crawler_max_pages` | int (1-10000) | `50` | no | `actor_id=apify/website-content-crawler` | -> `maxCrawlPages` |
+| `max_results` | int (1-10000) | `100` | no | - | Forwarded to `dataset.list_items(limit=...)` |
+| `timeout` | int (1-3600) | `300` | no | - | Run timeout in seconds (SDK forwards as `timeout_secs`) |
+| `memory` | options (Literal: 128/256/512/1024/2048/4096/8192) | `1024` | no | - | Actor memory in MB (`memory_mbytes`) |
+
+Actor presets (`actor_id`): `apify/instagram-scraper`, `clockworks/tiktok-scraper`,
+`apidojo/tweet-scraper`, `apify/linkedin-scraper`, `apify/facebook-pages-scraper`,
+`streamers/youtube-scraper`, `apify/google-search-scraper`,
+`compass/crawler-google-places`, `apify/website-content-crawler`,
+`curious_coder/web-scraper`, `custom`.
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Standard envelope payload |
+| `output-tool` | object | Same payload when wired to an AI agent's `input-tools` (`usable_as_tool=True`, tool name `apify_actor`) |
### Output payload (success)
@@ -69,59 +78,59 @@ curated actor ids (e.g. `instagramUrls` for `apify/instagram-scraper`).
}
```
-On `FAILED` / `TIMED-OUT` / `ABORTED` the envelope is `success: false` with a
-human error message AND a trimmed `result` dict carrying `run_id`, `actor_id`,
-and `status` so downstream nodes can still diagnose.
+On `FAILED` / `TIMED-OUT` / `ABORTED` (and missing token / empty actor id /
+`None` run result) the operation raises `NodeUserError`, which `BaseNode.execute()`
+catches into a `{ success: false, error_type: "NodeUserError", error: <message> }`
+envelope (single WARN log line, no traceback). There is no trimmed `result` dict.
## Logic Flow
```mermaid
flowchart TD
- A[handle_apify_actor] --> B[await _get_apify_client<br/>via container.auth_service.get_api_key 'apify']
- B -- token missing --> Enotk[Return error:<br/>Apify API token not configured]
- B -- token ok --> C[Read actorId, swap to customActorId if 'custom']
+ A[BaseNode.execute -> ApifyActorNode.run] --> B[await _get_apify_client<br/>via get_auth_service.get_api_key 'apify']
+ B -- token missing --> Enotk[raise NodeUserError:<br/>Apify API token not configured]
+ B -- token ok --> C[Read actor_id, swap to custom_actor_id if 'custom']
C --> C1{actor_id empty?}
- C1 -- yes --> Eaid[Return error: Actor ID is required]
- C1 -- no --> D[_build_actor_input<br/>parse actorInput JSON else {}<br/>merge per-actor quick helpers]
- D --> E[Read timeout, maxResults, memory_mbytes]
+ C1 -- yes --> Eaid[raise NodeUserError: Actor ID is required]
+ C1 -- no --> D[_build_actor_input params.model_dump<br/>parse actor_input JSON else {}<br/>merge per-actor quick helpers]
+ D --> E[Read timeout, max_results, memory_mbytes]
E --> F[client.actor id .call<br/>run_input, timeout_secs, memory_mbytes]
- F -- run_info None --> Enone[Return error:<br/>no result returned]
+ F -- run_info None --> Enone[raise NodeUserError:<br/>no result returned]
F -- ok --> G{run_status?}
- G -- FAILED --> Gf[Return success=false<br/>error=errorMessage, result={run_id, actor_id, status}]
- G -- TIMED-OUT --> Gt[Return success=false<br/>error=Actor timed out]
- G -- ABORTED --> Ga[Return success=false<br/>error=Actor run was aborted]
+ G -- FAILED --> Gf[raise NodeUserError errorMessage]
+ G -- TIMED-OUT --> Gt[raise NodeUserError: Actor timed out]
+ G -- ABORTED --> Ga[raise NodeUserError: Actor run was aborted]
G -- other --> H{dataset_id truthy?}
- H -- yes --> H1[client.dataset id .list_items limit=maxResults<br/>items = list_result.items]
+ H -- yes --> H1[client.dataset id .list_items limit=max_results<br/>items = listing.items]
H -- no --> H2[items = empty list]
- H1 & H2 --> I[Build result dict:<br/>run_id, actor_id, status, items, item_count,<br/>dataset_id, compute_units=usageTotalUsd,<br/>started_at, finished_at]
- I --> J[Return success envelope]
- F -- Exception --> K[Catch all:<br/>rewrite 401/Unauthorized -> 'Invalid API token'<br/>rewrite 404/not found -> 'Actor not found']
- K --> Eexc[Return error envelope]
+ H1 & H2 --> I[Build ApifyActorOutput:<br/>run_id, actor_id, status, items, item_count,<br/>dataset_id, compute_units=usageTotalUsd,<br/>started_at, finished_at]
+ I --> J[Return BaseNode success envelope]
```
## Decision Logic
-- **Auth lookup**: `auth_service.get_api_key("apify", "default")` - a None
- return short-circuits with `Apify API token not configured`.
+- **Auth lookup**: `get_auth_service().get_api_key("apify", "default")` - a None
+ return raises `NodeUserError("Apify API token not configured")`.
- **Actor id swap**: literal string `"custom"` triggers replacement with
- `customActorId` BEFORE the empty check. An empty `customActorId` still
- triggers the `Actor ID is required` error.
-- **Run-input merge**:
- - `actorInput` parsed as JSON; string input that fails `json.loads` (or is
+ `custom_actor_id` BEFORE the empty check. An empty `custom_actor_id` still
+ raises the `Actor ID is required` error.
+- **Run-input merge** (`_build_actor_input` on `params.model_dump()`):
+ - `actor_input` parsed as JSON; string input that fails `json.loads` (or is
empty/blank) silently becomes `{}`. Non-dict, non-string input becomes `{}`.
- - Quick helpers run even when `actorInput` already contains those keys -
+ - Quick helpers run even when `actor_input` already contains those keys -
they OVERWRITE any existing value.
- - Quick helpers only fire for the five hard-coded actor ids.
+ - Quick helpers only fire for five hard-coded actor ids
+ (instagram / tiktok / tweet-scraper / google-search / website-content-crawler).
- **Run statuses**: three status strings (`FAILED`, `TIMED-OUT`, `ABORTED`)
- produce `success=false` envelopes with specific error messages. Every other
- status (including `SUCCEEDED`, `RUNNING`, `READY`, etc.) proceeds to dataset
- fetch. Runs that stay `RUNNING` past the SDK's internal polling are expected
- to be handled by the SDK and should not reach here.
+ raise `NodeUserError` with specific messages. Every other status (including
+ `SUCCEEDED`, `RUNNING`, `READY`, etc.) proceeds to dataset fetch. Runs that
+ stay `RUNNING` past the SDK's internal polling are handled by the SDK and
+ should not reach here.
- **Empty dataset id**: `items = []`, no API call to the dataset endpoint.
-- **Error string rewriting**: `str(e)` is pattern-matched for `401`,
- `Unauthorized`, `404`, and `not found` (case-insensitive for the last one)
- to produce friendlier user errors. All other exceptions bubble up as their
- `str(e)` text.
+- **Error rewriting**: the `run` operation does NOT rewrite SDK exception
+ strings — those propagate to `BaseNode.execute()`'s generic handler. Only the
+ separate `validate_apify_token()` helper (used by the `validate_apify_key` WS
+ handler / Credentials modal) rewrites `401`/`Unauthorized` -> `Invalid API token`.
## Side Effects
@@ -153,9 +162,9 @@ flowchart TD
raises. `timeout` is the hard cap.
- **`compute_units` is misleading**: the value returned is `run.usageTotalUsd`
(USD), not compute units. The key name is preserved for backward compat.
-- **Error rewriting is substring-based**: an actor's genuine error message
- that contains the word "Unauthorized" or "not found" will be replaced with
- the generic MachinaOs error, masking the real cause.
+- **Error rewriting is substring-based (validation only)**: `validate_apify_token`
+ pattern-matches `401`/`Unauthorized` to a generic message; the runtime `run`
+ operation does not, so a genuine SDK error surfaces verbatim.
- **No usage tracking**: unlike search / HTTP nodes, there is no
`api_usage_metrics` row written for Apify calls.
- **Stale dataset on repeat runs**: `dataset.list_items(limit=maxResults)`
diff --git a/docs-internal/node-logic-flows/web_automation/browser.md b/docs-internal/node-logic-flows/web_automation/browser.md
index 9ddd2599..1dd9589a 100644
--- a/docs-internal/node-logic-flows/web_automation/browser.md
+++ b/docs-internal/node-logic-flows/web_automation/browser.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | web_automation / tool (dual-purpose) |
-| **Backend handler** | [`server/nodes/browser/browser/__init__.py::BrowserNode`](../../../server/nodes/browser/browser/__init__.py) |
+| **Backend handler** | [`server/nodes/browser/browser/__init__.py::BrowserNode`](../../../server/nodes/browser/browser/__init__.py) — dispatch via `BaseNode.execute()` -> `@Operation("dispatch")` |
| **Service** | [`server/nodes/browser/_service.py::BrowserService`](../../../server/nodes/browser/_service.py) |
| **Tests** | [`server/tests/nodes/test_web_automation.py`](../../../server/tests/nodes/test_web_automation.py) |
| **Skill (if any)** | [`server/skills/web_agent/browser-skill/SKILL.md`](../../../server/skills/web_agent/browser-skill/SKILL.md) |
@@ -11,8 +11,12 @@
## Purpose
-Interactive browser automation via the `agent-browser` CLI (a pinned npm dependency
-invoked through `npx --no-install`). Exposes 14 discrete operations (navigate,
+Interactive browser automation via the `agent-browser` CLI. The binary is a
+MachinaOs-managed local install resolved by
+[`nodes/browser/_install.py::agent_browser_binary_path`](../../../server/nodes/browser/_install.py)
+(installed via `npm install agent-browser --prefix <package_dir>` on first use,
+same pattern as Claude Code's project-local CLI — NOT a workspace `package.json`
+dependency and NOT invoked through `npx`). Exposes 14 discrete operations (navigate,
click, type, fill, screenshot, snapshot, get_text, get_html, eval, wait, scroll,
select, console, errors) plus a `batch` meta-op. The preferred workflow for an
AI agent is `navigate` -> `snapshot` -> `click`/`fill` using the stable `@eN`
@@ -45,34 +49,35 @@ for both knobs live in `.env.template`.
## Parameters
+Params model is `BrowserParams` (field names are snake_case). `tool_name` /
+`tool_description` are class attributes (not Params fields).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `toolName` | string | `browser` | no | - | Tool name exposed to the AI agent |
-| `toolDescription` | string | (see frontend) | no | - | Tool description shown to LLM |
-| `operation` | options | `navigate` | no | - | One of `navigate`, `click`, `type`, `fill`, `screenshot`, `snapshot`, `get_text`, `get_html`, `eval`, `wait`, `scroll`, `select`, `console`, `errors`, `batch` |
-| `session` | string | `""` | no | - | Session id; if empty, handler uses `machina_<execution_id>` |
-| `timeout` | number | `30` | no | - | Per-operation timeout in seconds |
-| `headed` | boolean | `true` | no | - | Passes `--headed` to agent-browser |
-| `autoConnect` | boolean | `false` | no | - | Passes `--auto-connect` |
-| `browser` | options | `chrome` | no | - | `chrome` / `edge` / `chromium` / `custom` / `bundled` / `bundled_explicit`. Legacy `bundled` silently upgraded to `chrome` |
-| `executablePath` | string | `""` | no | `browser=custom` | Explicit browser path when `browser=custom` |
-| `newWindow` | boolean | `true` | no | - | Appended as `--args --new-window`; only honoured when an explicit `executable_path` resolved |
-| `chromeProfile` | string | `""` | no | - | Chrome user-data-dir profile |
-| `userAgent` | string | `""` | no | - | Custom UA string |
-| `proxy` | string | `""` | no | - | `host:port` or `user:pass@host:port` |
-| `actionDelay` | number | `0` | no | - | Seconds to wait before the real op (a pre-command `wait` is issued first) |
+| `operation` | options (Literal) | `navigate` | no | - | One of `navigate`, `click`, `type`, `fill`, `screenshot`, `snapshot`, `get_text`, `get_html`, `eval`, `wait`, `scroll`, `select`, `console`, `errors`, `batch` |
| `url` | string | `""` | yes (op=navigate) | `operation=navigate` | Target URL |
-| `selector` | string | `""` | yes (ops that act on an element) | see below | CSS selector or `@eN` ref from snapshot |
-| `text` | string | `""` | no | `operation=type` | Text to type |
-| `value` | string | `""` | no | `operation in [fill, select]` | Value to fill / dropdown value |
+| `selector` | string | `""` | yes (ops that act on an element) | `operation in [click,type,fill,get_text,get_html,wait,select]` | CSS selector or `@eN` ref from snapshot |
+| `text` | string | `""` | no | `operation=type` | Text to type keystroke-by-keystroke |
+| `value` | string | `""` | no | `operation in [fill, select]` | Value to fill / dropdown option value |
| `expression` | string | `""` | yes (op=eval) | `operation=eval` | JS expression to evaluate |
-| `direction` | options | `down` | no | `operation=scroll` | `up` / `down` / `left` / `right` |
-| `amount` | number | `500` | no | `operation=scroll` | Pixels to scroll |
-| `fullPage` | boolean | `false` | no | `operation=screenshot` | Passes `--full` |
+| `direction` | options (Literal: up/down/left/right) | `down` | no | `operation=scroll` | Scroll direction |
+| `amount` | int (1-20000) | `500` | no | `operation=scroll` | Pixels to scroll |
+| `commands` | string (JSON array) | `"[]"` | yes (op=batch) | `operation=batch` | JSON array of batch command dicts piped to stdin |
+| `full_page` | boolean | `false` | no | `operation=screenshot` | Passes `--full` |
| `annotate` | boolean | `false` | no | `operation=screenshot` | Passes `--annotate` |
-| `screenshotFormat` | options | `png` | no | `operation=screenshot` | `png` / `jpeg` / `webp` |
-| `screenshotQuality` | number | - | no | `operation=screenshot, screenshotFormat=jpeg` | 1-100, only forwarded for jpeg |
-| `commands` | string (JSON array) | `"[]"` | yes (op=batch) | `operation=batch` | JSON array of command dicts piped to stdin |
+| `screenshot_format` | options (Literal: png/jpeg) | `png` | no | `operation=screenshot` | Forwarded as `--screenshot-format` when not png |
+| `screenshot_quality` | int (1-100) | `85` | no | `operation=screenshot, screenshot_format=jpeg` | Forwarded as `--screenshot-quality`, only for jpeg |
+| `session` | string | `""` | no | - | Session id; if empty, handler uses `machina_<execution_id>` |
+| `browser` | options (Literal) | `chrome` | no | - | `chrome` / `edge` / `chromium` / `bundled_explicit` / `custom`. Legacy/empty `bundled` silently upgraded to `chrome` at dispatch |
+| `executable_path` | string | `""` | no | `browser=custom` | Explicit browser path when `browser=custom` |
+| `headed` | boolean | `true` | no | - | Show browser window (false = headless `--headed` omitted) |
+| `new_window` | boolean | `true` | no | - | Appended as `--args --new-window`; only honoured when an explicit `executable_path` resolved |
+| `auto_connect` | boolean | `false` | no | - | Reuse an already-running CDP browser (`--auto-connect`) |
+| `chrome_profile` | string | `""` | no | - | Chrome user-profile name |
+| `user_agent` | string | `""` | no | - | Custom UA string |
+| `proxy` | string | `""` | no | - | Proxy URL (e.g. `http://user:pass@host:port`) |
+| `action_delay` | int (0-60000) | `0` | no | - | Milliseconds to wait before the real op (a pre-command `wait` is issued first) |
+| `timeout` | int (1-600) | `30` | no | - | Per-action timeout in seconds |
Operations that **require** `selector`: `click`, `type`, `fill`, `get_text`,
`get_html`, `wait`, `select`.
@@ -82,7 +87,7 @@ Operations that **require** `selector`: `click`, `type`, `fill`, `get_text`,
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Standard envelope payload |
-| `output-tool` | object | Same payload when wired to an AI agent |
+| `output-tool` | object | Same payload when wired to an AI agent's `input-tools` (`usable_as_tool=True`, tool name `browser`) |
### Output payload
@@ -101,7 +106,7 @@ On failure: `{ success: false, error: <string>, execution_time: number }`.
```mermaid
flowchart TD
- A[handle_browser] --> B{BrowserService<br/>available?}
+ A[BaseNode.execute -> BrowserNode.dispatch] --> B{BrowserService<br/>available?}
B -- no --> Einst[Return error:<br/>agent-browser not installed]
B -- yes --> C[Resolve session, timeout, headed, browser,<br/>executable_path, proxy, userAgent,<br/>newWindow, chromeProfile, actionDelay]
C --> C1[Upgrade legacy 'bundled' -> 'chrome'<br/>_resolve_browser looks up shutil.which or Win registry]
@@ -126,7 +131,7 @@ flowchart TD
Gsw -- unknown --> Gx[raise ValueError<br/>Unknown operation]
Gn & Gc & Gt & Gf & Gs & Gsn & Gg & Gev & Gsc & Gco --> H[svc.run args, session, timeout, run_kw]
F --> H
- H --> H1[Popen argv via npx --no-install agent-browser<br/>shell=False, reads first JSON line of stdout]
+ H --> H1[Popen argv via resolved agent-browser binary<br/>shell=False, reads first JSON line of stdout]
H1 --> H2{first line<br/>parseable JSON?}
H2 -- yes --> H3[data = parsed JSON]
H2 -- no --> H4[data = output: raw string]
@@ -134,15 +139,16 @@ flowchart TD
I -- yes --> I1[truncate + append '...truncated']
I -- no --> J[finally: psutil kill process tree]
I1 --> J
- J --> K[Return success envelope<br/>operation, data, session]
- Gx --> Eval[ValueError caught:<br/>return error envelope]
- H --> Eother[TimeoutError / RuntimeError caught<br/>return error envelope]
+ J --> K[Return BaseNode success envelope<br/>operation, data, session]
+ Gx --> Eval[ValueError -> BaseNode generic envelope]
+ H --> Eother[RuntimeError / NodeUserError -> BaseNode envelope]
```
## Decision Logic
-- **Service missing**: `get_browser_service()` returns `None` when `npx` is not on
- PATH -> immediate `agent-browser not installed` error envelope.
+- **Service missing**: `get_browser_service()` returns `None` when the
+ agent-browser binary cannot be resolved (Node toolchain / `npm` unavailable)
+ -> `RuntimeError("agent-browser not installed...")` -> error envelope.
- **Session resolution**: empty `session` -> `machina_<execution_id>` via the
typed `ctx.execution_id` accessor (handles present-but-None on the agent
tool-dispatch path). Only the stripped value is considered empty; whitespace
@@ -162,13 +168,14 @@ flowchart TD
- **`newWindow`**: gated on `executable_path` actually resolving - if a user
requests new-window while `browser=bundled_explicit`, the flag is silently
dropped.
-- **`actionDelay > 0`**: issues a `wait <seconds>` command to agent-browser
- BEFORE the real operation. Failures in the delay call propagate as errors.
+- **`action_delay > 0`**: issues a `wait <action_delay>` command (milliseconds)
+ to agent-browser BEFORE the real operation. Failures in the delay call
+ propagate as errors.
- **`batch`**: reads `commands` JSON string (defaults to `"[]"`) and pipes
`json.dumps(cmds).encode()` to the subprocess' stdin with flags `batch --json`.
- **Required params**: `_req(p, "url")` and `_req(p, "expression")` raise
- `ValueError` on empty strings; `_req_sel(s)` does the same for `selector`.
- These surface as the error envelope via the outer `except (ValueError, ...)`.
+ `NodeUserError` on empty strings; `_req_sel(s)` does the same for `selector`.
+ `BaseNode.execute()` catches `NodeUserError` into a single-WARN error envelope.
- **Unknown operation**: `_build_args` raises `ValueError("Unknown operation: <op>")`.
## Side Effects
@@ -178,11 +185,13 @@ flowchart TD
- **External API calls**: none (all work happens via subprocess).
- **File I/O**: agent-browser itself may write screenshots or a profile dir to
its own cache - not under this handler's control.
-- **Subprocess**: spawns `npx --no-install agent-browser ...` via
+- **Subprocess**: spawns the resolved `agent-browser` binary directly via
`subprocess.Popen` with `shell=False`. Reads only the FIRST stdout line
(daemon holds stdout open), caps at 100 KB, then force-kills the entire
- process tree via `psutil.Process.children(recursive=True) -> kill()`.
+ process tree via `kill_tree(proc.pid)` (`services._supervisor.util`, psutil-backed).
Always calls `proc.wait()` in a finally block.
+ `AGENT_BROWSER_IDLE_TIMEOUT_MS` is injected into the spawn env when the idle
+ timeout is > 0.
- **Long-lived daemon**: agent-browser spawns a persistent daemon between
calls. `shutdown_browser_service()` (registered in FastAPI lifespan) runs
`agent-browser close --all` to stop it.
@@ -190,9 +199,10 @@ flowchart TD
## External Dependencies
- **Credentials**: none.
-- **Services**: `BrowserService` singleton; `npx` on PATH; `agent-browser`
- pinned in the project's `package.json` and installed under `node_modules`.
-- **Python packages**: `psutil` (for process-tree kill).
+- **Services**: `BrowserService` singleton; `npm` on PATH (for first-use
+ install); `agent-browser` installed under `<package_dir("browser")>/npm/`
+ by `nodes/browser/_install.py` (MachinaOs-managed, not a workspace dep).
+- **Python packages**: `psutil` (via `kill_tree` for process-tree kill).
- **Environment variables**: `BROWSER_MAX_INSTANCES` (concurrent session cap,
default 3), `BROWSER_IDLE_TIMEOUT_MS` (daemon idle auto-shutdown in ms,
default 600000, 0 disables) — canonical values in `.env.template`, mirrored
@@ -211,15 +221,18 @@ flowchart TD
- **Silent `bundled` -> `chrome` upgrade**: a user who saved a workflow when
`bundled` was the default will now actually use system Chrome on execution.
- **`newWindow` silently disabled**: see Decision Logic.
-- **Only `ValueError`, `TimeoutError`, `RuntimeError` are caught**: other
- exceptions (e.g. `OSError` from subprocess, `psutil.Error`) propagate and
- the executor wraps them in its generic envelope.
+- **Exception handling is in `BaseNode.execute()`**: `NodeUserError` (empty
+ required field) -> single WARN + structured envelope; `ValueError` (unknown
+ op) / `RuntimeError` (service missing or empty stdout) / other exceptions ->
+ generic envelope with traceback. The plugin's `dispatch` itself does not
+ catch.
- **Process tree kill race**: `psutil.NoSuchProcess` is swallowed for both the
parent and every child, so a fast-exiting daemon never leaks an exception
but may leave orphaned Chromium processes if the tree changed shape between
the `children()` call and the `kill()` call.
-- **`actionDelay` is billable**: the preliminary `wait` call consumes part of
- `timeout` because each subprocess invocation resets the timer.
+- **`action_delay` (ms) is a separate call**: the preliminary `wait` runs as its
+ own subprocess invocation with its own `timeout` budget (each invocation
+ resets the timer).
## Related
diff --git a/docs-internal/node-logic-flows/web_automation/crawleeScraper.md b/docs-internal/node-logic-flows/web_automation/crawleeScraper.md
index db326b45..f4097bee 100644
--- a/docs-internal/node-logic-flows/web_automation/crawleeScraper.md
+++ b/docs-internal/node-logic-flows/web_automation/crawleeScraper.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | web_automation / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/crawlee.py::handle_crawlee_scraper`](../../../server/services/handlers/crawlee.py) |
+| **Backend handler** | [`server/nodes/scraper/crawlee_scraper/__init__.py::CrawleeScraperNode`](../../../server/nodes/scraper/crawlee_scraper/__init__.py) — dispatch via `BaseNode.execute()` -> `@Operation("scrape")` |
| **Tests** | [`server/tests/nodes/test_web_automation.py`](../../../server/tests/nodes/test_web_automation.py) |
| **Skill (if any)** | [`server/skills/web_agent/crawlee-scraper-skill/SKILL.md`](../../../server/skills/web_agent/crawlee-scraper-skill/SKILL.md) |
| **Dual-purpose tool** | yes |
@@ -33,31 +33,41 @@ config and collects a page list from the route handler closure.
## Parameters
+Params model is `CrawleeScraperParams` (field names are snake_case).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
+| `tool_name` | string | `web_scraper` | no | - | Name shown to the LLM when used as a tool (note: class `tool_name` is `web_reader`) |
+| `tool_description` | string | (see default) | no | - | Description shown to the LLM |
| `url` | string | `""` | **yes** | - | Seed URL |
-| `crawlerType` | options | `beautifulsoup` | no | - | `beautifulsoup` / `playwright` / `adaptive` |
-| `mode` | options | `single` | no | - | `single` or `crawl` |
-| `cssSelector` | string | `""` | no | - | CSS selector used for text/HTML extraction. Empty -> whole page |
-| `extractLinks` | boolean | `false` | no | - | Include `links[]` on each page |
-| `outputFormat` | options | `text` | no | - | `text` / `html` / `markdown`. `markdown` falls back to text if `html2text` not installed |
-| `maxPages` | number | `10` | no | `mode=crawl` | Ignored and clamped to 1 when `mode=single` |
-| `maxDepth` | number | `2` | no | `mode=crawl` | Ignored and set to 0 when `mode=single` |
-| `maxConcurrency` | number | `5` | no | - | Forwarded as `max_concurrency`; `desired_concurrency = min(maxConcurrency, 10)` |
-| `timeout` | number | `60` | no | - | Per-request handler timeout AND outer `asyncio.wait_for` timeout |
-| `linkSelector` | string | `""` | no | `mode=crawl` | Defaults to `a[href]` when empty |
-| `urlPattern` | string | `""` | no | `mode=crawl` | `fnmatch`-style pattern compiled with `fnmatch.translate` -> passed to `enqueue_links(include=[regex])` |
-| `browserType` | options | `chromium` | no | `crawlerType=playwright` | `chromium` / `firefox` / `webkit` |
-| `waitForSelector` | string | `""` | no | `crawlerType=playwright` | Awaits selector before extracting |
-| `waitTimeout` | number | `30000` | no | `crawlerType=playwright` | Milliseconds for `wait_for_selector` |
-| `screenshot` | boolean | `false` | no | `crawlerType=playwright` | Capture base64 PNG per page |
-| `useProxy` | boolean | `false` | no | - | Route through `ProxyService` (see Decision Logic) |
+| `crawler_type` | options (Literal) | `beautifulsoup` | no | - | `beautifulsoup` / `playwright` / `adaptive` |
+| `mode` | options (Literal) | `single` | no | - | `single` or `crawl` |
+| `css_selector` | string | `""` | no | - | CSS selector used for text/HTML extraction. Empty -> whole page |
+| `extract_links` | boolean | `false` | no | - | Include `links[]` on each page |
+| `output_format` | options (Literal) | `text` | no | - | `text` / `html` / `markdown`. `markdown` falls back to text if `html2text` not installed |
+| `max_pages` | int (1-1000) | `10` | no | - | Clamped to 1 when `mode=single` |
+| `max_concurrency` | int (1-50) | `5` | no | - | Forwarded as `max_concurrency`; `desired_concurrency = min(max_concurrency, 10)` |
+| `timeout` | int (1-3600) | `60` | no | - | Per-request handler timeout AND outer `asyncio.wait_for` timeout |
+| `link_selector` | string | `""` | no | `mode=crawl` | Defaults to `a[href]` when empty |
+| `url_pattern` | string | `""` | no | `mode=crawl` | `fnmatch`-style pattern compiled with `fnmatch.translate` -> passed to `enqueue_links(include=[regex])` |
+| `max_depth` | int (0-50) | `2` | no | `mode=crawl` | Set to 0 when `mode=single` |
+| `browser_type` | options (Literal) | `chromium` | no | `crawler_type=playwright` | `chromium` / `firefox` / `webkit` |
+| `wait_for_selector` | string | `""` | no | `crawler_type=playwright` | Awaits selector before extracting |
+| `wait_timeout` | int (0-600000) | `30000` | no | `crawler_type=playwright` | Milliseconds for `wait_for_selector` |
+| `take_screenshot` | boolean | `false` | no | `crawler_type=playwright` | Capture base64 PNG per page |
+| `block_resources` | boolean | `false` | no | `crawler_type=playwright` | Declared in Params; currently not consumed by the run helpers |
+| `use_proxy` | boolean | `false` | no | - | Route through `ProxyService` (see Decision Logic) |
+| `proxy_provider` | string | `auto` | no | `use_proxy=true` | Provider hint passed to `ProxyService.get_proxy_url` |
+| `proxy_country` | string | `""` | no | `use_proxy=true` | ISO country hint |
+| `session_type` | options (Literal: rotating/sticky) | `rotating` | no | `use_proxy=true` | Proxy session type |
+| `sticky_duration` | int (>=1) | `600` | no | `use_proxy=true, session_type=sticky` | Sticky session duration (s) |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Standard envelope payload |
+| `output-tool` | object | Same payload when wired to an AI agent's `input-tools` (`usable_as_tool=True`, class tool name `web_reader`) |
### Output payload
@@ -68,7 +78,7 @@ config and collects a page list from the route handler closure.
title: string;
content: string;
links?: string[]; // when extractLinks=true
- screenshot?: string; // base64 PNG, playwright + screenshot=true
+ screenshot?: string; // base64 PNG, playwright + take_screenshot=true
}>;
page_count: number;
crawler_type: 'beautifulsoup' | 'playwright' | 'adaptive';
@@ -81,8 +91,8 @@ config and collects a page list from the route handler closure.
```mermaid
flowchart TD
- A[handle_crawlee_scraper] --> B{url stripped<br/>non-empty?}
- B -- no --> Eurl[Return error: URL is required]
+ A[BaseNode.execute -> CrawleeScraperNode.scrape] --> B{url stripped<br/>non-empty?}
+ B -- no --> Eurl[raise NodeUserError: URL is required]
B -- yes --> C[Read crawlerType, mode, cssSelector,<br/>extractLinks, outputFormat, maxPages,<br/>maxDepth, maxConcurrency, timeout]
C --> C1{mode == 'single'?}
C1 -- yes --> C2[Force maxPages=1, maxDepth=0]
@@ -94,21 +104,20 @@ flowchart TD
D2 & D3 --> E{crawlerType?}
E -- beautifulsoup / adaptive --> F1[_run_beautifulsoup]
E -- playwright --> F2[_run_playwright]
- E -- other --> Ebad[Return error:<br/>Unknown crawler type]
+ E -- other --> Ebad[raise NodeUserError:<br/>Unknown crawler type]
F1 --> G[BeautifulSoupCrawler<br/>max_requests_per_crawl=maxPages<br/>max_crawl_depth=maxDepth if crawl else 0<br/>request_handler_timeout=timeout<br/>ConcurrencySettings<br/>MemoryStorageClient<br/>proxy_configuration=proxy_config]
F2 --> H[PlaywrightCrawler<br/>browser_type, headless=True, proxy_configuration]
G & H --> I[register default_handler closure]
I --> I1[handler: extract title/content<br/>via _extract_text+_extract_links<br/>append to pages list<br/>if mode=crawl: ctx.enqueue_links<br/>with optional fnmatch-compiled include regex]
I1 --> J[asyncio.wait_for crawler.run URL, timeout=timeout]
- J -- ImportError --> Eimp[Return error with install hint<br/>playwright vs crawlee]
- J -- other Exception --> Egen[Return error: str e]
- J -- ok --> K[Return success<br/>pages, page_count, crawler_type, mode, proxied]
+ J -- ImportError --> Eimp[raise NodeUserError install hint<br/>playwright vs crawlee, else re-raise]
+ J -- ok --> K[Return CrawleeScraperOutput<br/>pages, page_count, crawler_type, mode, proxied]
```
## Decision Logic
-- **Empty URL**: `url.strip() == ""` -> immediate failure envelope, no crawler
- is constructed.
+- **Empty URL**: `url.strip() == ""` -> `NodeUserError("URL is required")`, no
+ crawler is constructed.
- **`mode='single'`**: hard clamps `max_pages=1` and `max_depth=0`, regardless
of what the user set. In `crawl` mode the requested values are forwarded.
- **`crawlerType='adaptive'`**: routed to `_run_beautifulsoup` - there is no
@@ -123,13 +132,16 @@ flowchart TD
For markdown the handler falls back to plain text if `html2text` is missing
(`ImportError` swallowed). All output is capped at 100 000 chars
(`_MAX_CONTENT_LENGTH`).
-- **Error handling**:
- - `ImportError` whose message contains `playwright` -> rewritten to an
- install hint for `pip install 'crawlee[playwright]' && playwright install`.
- - `ImportError` whose message contains `crawlee` -> rewritten to an install
- hint for `pip install 'crawlee[beautifulsoup]'`.
- - All other exceptions are caught and the `str(e)` returned as-is; the
- handler never raises.
+- **Error handling**: only `ImportError` is caught inside `scrape`:
+ - message contains `playwright` -> `NodeUserError` install hint for
+ `pip install 'crawlee[playwright]' && playwright install chromium`.
+ - message contains `crawlee` -> `NodeUserError` install hint for
+ `pip install 'crawlee[beautifulsoup]'`.
+ - any other `ImportError` is re-raised.
+ - All non-`ImportError` exceptions propagate to `BaseNode.execute()`'s generic
+ handler (full-traceback envelope); `NodeUserError` (empty URL / unknown
+ crawler type) yields a single-WARN structured envelope. The plugin does not
+ swallow errors into `str(e)`.
## Side Effects
diff --git a/docs-internal/node-logic-flows/whatsapp/whatsappDb.md b/docs-internal/node-logic-flows/whatsapp/whatsappDb.md
index ff593562..435cb8b4 100644
--- a/docs-internal/node-logic-flows/whatsapp/whatsappDb.md
+++ b/docs-internal/node-logic-flows/whatsapp/whatsappDb.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | whatsapp / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/whatsapp.py::handle_whatsapp_db`](../../../server/services/handlers/whatsapp.py) |
+| **Backend handler** | [`server/nodes/whatsapp/whatsapp_db.py`](../../../server/nodes/whatsapp/whatsapp_db.py) (`WhatsAppDbNode`); dispatched via `BaseNode.execute()` -> `@Operation("query")`, which delegates to [`server/nodes/whatsapp/_base.py::handle_whatsapp_db`](../../../server/nodes/whatsapp/_base.py) |
| **Tests** | [`server/tests/nodes/test_whatsapp.py`](../../../server/tests/nodes/test_whatsapp.py) |
| **Skill (if any)** | [`server/skills/social_agent/whatsapp-db-skill/SKILL.md`](../../../server/skills/social_agent/whatsapp-db-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `whatsappDb` |
+| **Dual-purpose tool** | yes - tool name `whatsapp_db` |
## Purpose
@@ -55,8 +55,9 @@ frontend.
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Envelope payload with `operation` field + operation-specific fields |
-| `output-tool` | object | Same payload for AI tool wiring |
+| `output-main` | object | The handler `result` dict with `operation` field + operation-specific fields. Plugin `Output` model is `WhatsAppDbOutput` (`operation`, `messages`, `contacts`, `groups`, `channels`, `total`, `extra="allow"`). |
+
+When wired into an AI agent's `input-tools` handle (`usable_as_tool=True`, tool name `whatsapp_db`) the same payload is returned to the LLM.
All payloads include the read-back `operation` and `timestamp`. Collection
operations (`search_groups`, `list_contacts`, `list_channels`) apply a soft
@@ -121,8 +122,8 @@ flowchart TD
## Side Effects
- **Database writes**: none.
-- **Broadcasts**: none - standard `node_status` comes from `NodeExecutor`.
-- **External calls**: WebSocket RPC to the Go service (`whatsapp_rpc_call` and `handle_whatsapp_chat_history` in `routers/whatsapp.py`). For `channel_messages` and `chat_history` with `include_media_data=true`, one additional `media` RPC call per media message (up to 5 in parallel).
+- **Broadcasts**: none - standard `node_status` comes from `BaseNode.execute()`.
+- **External calls**: WebSocket RPC to the Go service (`whatsapp_rpc_call` and `handle_whatsapp_chat_history` in [`nodes/whatsapp/_service.py`](../../../server/nodes/whatsapp/_service.py)). For `channel_messages` and `chat_history` with `include_media_data=true`, one additional `media` RPC call per media message (up to 5 in parallel).
- **File I/O**: none.
- **Subprocess**: none.
diff --git a/docs-internal/node-logic-flows/whatsapp/whatsappReceive.md b/docs-internal/node-logic-flows/whatsapp/whatsappReceive.md
index 23cb61dc..480f81f0 100644
--- a/docs-internal/node-logic-flows/whatsapp/whatsappReceive.md
+++ b/docs-internal/node-logic-flows/whatsapp/whatsappReceive.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | whatsapp / trigger |
-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) + [`server/services/event_waiter.py::build_whatsapp_filter`](../../../server/services/event_waiter.py) |
+| **Backend handler** | [`server/nodes/whatsapp/whatsapp_receive.py`](../../../server/nodes/whatsapp/whatsapp_receive.py) (`WhatsAppReceiveNode`, a `TriggerNode`); the filter closure is built by `build_filter()` -> [`server/nodes/whatsapp/_filters.py::build_filter`](../../../server/nodes/whatsapp/_filters.py) (self-registered into `event_waiter.FILTER_BUILDERS` as `whatsappReceive`). Canary-registered with CloudEvents type `com.machinaos.whatsapp.message.received`; deployment fires via `TriggerListenerWorkflow`. |
| **Tests** | [`server/tests/nodes/test_whatsapp.py`](../../../server/tests/nodes/test_whatsapp.py) |
| **Skill (if any)** | n/a |
| **Dual-purpose tool** | no - pure trigger node |
@@ -11,11 +11,15 @@
## Purpose
Event-driven trigger that waits for incoming WhatsApp messages and starts a
-workflow run when one matches. Uses the generic event-waiter machinery: the
-handler registers a waiter with a filter closure, broadcasts `waiting`
-status, then awaits a `whatsapp_message_received` event dispatched by the
-Go RPC listener. Matching messages resolve the waiter and become the
-trigger's output.
+workflow run when one matches. `whatsappReceive` is canary-registered
+(CloudEvents type `com.machinaos.whatsapp.message.received`): on deployment the
+manager spawns a `TriggerListenerWorkflow` per (deployment, trigger) pair which
+receives the CloudEvents envelope via Temporal Signal and spawns a child
+`MachinaWorkflow` per matching event. Single-node "Run" (non-deployed test)
+still uses the `TriggerNode.execute` event-waiter path: register a waiter with
+the filter closure, broadcast `waiting`, then await a
+`whatsapp_message_received` event. In both paths `build_filter` decides which
+inbound messages match.
## Inputs (handles)
@@ -23,17 +27,20 @@ Triggers have no data inputs - they are workflow starting points.
## Parameters
+Field names are snake_case throughout (JSON Schema keys match the Pydantic field names exactly, no camelCase aliases).
+
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `messageTypeFilter` | options | `all` | no | - | `all`, `text`, `image`, `video`, `audio`, `document`, `location`, `contact`, `sticker` |
+| `message_type_filter` | options | `all` | no | - | `all`, `text`, `image`, `video`, `audio`, `document`, `location`, `contact` |
| `filter` | options | `all` | no | - | `all`, `self`, `any_contact`, `contact`, `group`, `channel`, `keywords` |
-| `contactPhone` | string | `""` | conditional | `filter == 'contact'` | Phone number substring matched against resolved `sender_phone` |
-| `group_id` | string | `""` | conditional | `filter == 'group'` | Group JID |
-| `senderNumber` | string | `""` | no | `filter == 'group'` | Optional phone substring to match a specific sender within the group |
-| `channel_jid` | string | `""` | no | `filter == 'channel'` | Optional exact channel JID; empty = any `@newsletter` chat |
+| `phone_number` | string | `""` | conditional | `filter == 'contact'` | Phone number substring matched against resolved `sender_phone` |
+| `group_id` | string | `""` | conditional | `filter == 'group'` | Group JID (component `GroupIdSelector`) |
+| `sender_number` | string | `""` | no | `filter == 'group'` | Optional phone substring to match a specific sender within the group (component `SenderNumberSelector`, `dependsOn: group_id`) |
+| `channel_jid` | string | `""` | conditional | `filter == 'channel'` | Optional exact channel JID; empty = any `@newsletter` chat (component `ChannelJidSelector`) |
| `keywords` | string | `""` | conditional | `filter == 'keywords'` | Comma-separated keywords, case-insensitive substring match against `text` |
-| `forwardedFilter` | options | `all` | no | - | `all`, `only_forwarded`, `ignore_forwarded` |
-| `ignoreOwnMessages` | boolean | `true` | no | - | Skip `is_from_me == true` (except when `filter == 'self'`) |
+| `forwarded_filter` | options | `all` | no | - | `all`, `only_forwarded`, `ignore_forwarded` |
+| `ignore_own_messages` | boolean | `true` | no | `filter in [all, any_contact, contact, group, channel, keywords]` | Skip `is_from_me == true` (except when `filter == 'self'`) |
+| `include_media_data` | boolean | `false` | no | - | Download base64 media data with the event |
## Outputs (handles)
@@ -77,8 +84,8 @@ CLAUDE.md "WhatsApp Group/Sender Name Persistence".
```mermaid
flowchart TD
- A[handle_trigger_node] --> B[event_waiter.get_trigger_config<br/>whatsappReceive -> whatsapp_message_received]
- B --> C[event_waiter.register<br/>builds filter via build_whatsapp_filter]
+ A[TriggerNode.execute - test Run path] --> B[event_type =<br/>whatsapp_message_received]
+ B --> C[event_waiter.register<br/>builds filter via _filters.build_filter]
C --> D[broadcaster.update_node_status<br/>status=waiting]
D --> E[await event_waiter.wait_for_event]
E -- CancelledError --> X[success=false<br/>error=Cancelled by user]
@@ -87,20 +94,20 @@ flowchart TD
F --> OK[success=true<br/>result=event_data]
```
-### Filter closure (build_whatsapp_filter)
+### Filter closure (`_filters.build_filter`)
```mermaid
flowchart LR
- M[message dict] --> R1{messageTypeFilter<br/>== 'all' or matches?}
+ M[message dict] --> R1{message_type_filter<br/>== 'all' or matches?}
R1 -- no --> REJ[reject]
R1 -- yes --> R2{filter mode?}
R2 -- self --> S1{is_from_me AND<br/>chat_id == sender?}
S1 -- no --> REJ
R2 -- any_contact --> S2{is_group?}
S2 -- yes --> REJ
- R2 -- contact --> S3{contactPhone in<br/>sender_phone?}
+ R2 -- contact --> S3{phone_number in<br/>sender_phone?}
S3 -- no --> REJ
- R2 -- group --> S4{is_group AND<br/>chat_id == group_id AND<br/>senderNumber match if set?}
+ R2 -- group --> S4{is_group AND<br/>chat_id == group_id AND<br/>sender_number match if set?}
S4 -- no --> REJ
R2 -- channel --> S5{chat_id ends @newsletter<br/>and matches channel_jid if set?}
S5 -- no --> REJ
@@ -113,10 +120,10 @@ flowchart LR
S4 --> OWN
S5 --> OWN
S6 --> OWN
- OWN{ignoreOwnMessages AND<br/>is_from_me AND filter != self?}
+ OWN{ignore_own_messages AND<br/>is_from_me AND filter != self?}
OWN -- yes --> REJ
OWN -- no --> FWD
- FWD{forwardedFilter check}
+ FWD{forwarded_filter check}
FWD -- fail --> REJ
FWD -- pass --> ACC[accept]
```
@@ -125,7 +132,7 @@ flowchart LR
- **Filter priority**: message_type -> sender filter -> own-message -> forwarded. Every check is a short-circuit reject.
- **Sender phone fallback**: uses `group_info.sender_phone` for group messages, falls back to root `sender_phone`, and if both empty splits the JID at `@` to extract the phone prefix.
-- **`filter == 'self'` is special**: disables the `ignoreOwnMessages` gate (otherwise every self-message would be filtered out).
+- **`filter == 'self'` is special**: disables the `ignore_own_messages` gate (otherwise every self-message would be filtered out).
- **Keyword match is substring**, not token match: `"hi"` matches `"this"`.
- **Channel filter** requires `chat_id` to end exactly with `@newsletter`; if `channel_jid` is provided it must match exactly, otherwise any channel passes.
- **Pre-check**: unlike `telegramReceive`, this trigger does not check WhatsApp pairing state before registering - it will wait forever if the Go RPC is disconnected.
@@ -134,14 +141,14 @@ flowchart LR
- **Database writes**: none from the handler directly (frontend may save `group_name`/`sender_name` back to node parameters via the WebSocket `save_node_parameters` handler - that is UI-driven, not part of trigger execution).
- **Broadcasts**: `update_node_status(node_id, "waiting", {...})` once before awaiting.
-- **External calls**: none outbound. Receives events dispatched by `routers/whatsapp.py` when the Go RPC emits `event.message_received`.
+- **External calls**: none outbound. Inbound `whatsapp_message_received` events are produced by the WhatsApp WS handlers / Go RPC bridge and routed via [`nodes/whatsapp/_events.py`](../../../server/nodes/whatsapp/_events.py) (`broadcast_whatsapp_message`), which both broadcasts the legacy raw frame and emits the typed CloudEvents envelope through `dispatch.emit`.
- **File I/O**: none.
- **Subprocess**: none.
## External Dependencies
- **Credentials**: none - pairing is held by the Go service.
-- **Services**: `whatsapp-rpc` (Go) for event dispatch via `event_waiter.dispatch('whatsapp_message_received', data)`.
+- **Services**: `whatsapp-rpc` (Go) for inbound message events, surfaced via `nodes/whatsapp/_events.py::broadcast_whatsapp_message`.
- **Python packages**: `asyncio`.
- **Environment variables**: none in the handler.
@@ -151,6 +158,7 @@ flowchart LR
- **LID-to-phone resolution is implicit**: the Go RPC already populates `sender_phone` with the resolved phone before dispatch, so the filter never needs to call back into the RPC. Missing `sender_phone` is still tolerated via the JID-prefix fallback.
- **`filter == 'self'` semantics are narrow**: requires `is_from_me` AND `chat_id == sender` (true only for notes-to-self chats). Self-messages sent to other chats are rejected.
- **`keywords` is trivially tokenized**: splits on `,` and `.strip().lower()` each; there is no escaping or whole-word matching.
+- **Parameter field names are snake_case** (`message_type_filter`, `phone_number`, `sender_number`, `forwarded_filter`, `ignore_own_messages`). `message_type_filter` has no `sticker` option (unlike `whatsappSend`'s `message_type`).
- **Cancellation path** returns `success=false` with `error="Cancelled by user"` - this is the documented path for the Cancel button, not a real error.
- **Filter closure is built once at register-time**: changes to node parameters during a waiting run are not reflected; user must cancel and re-run.
@@ -158,4 +166,4 @@ flowchart LR
- **Companion nodes**: [`whatsappSend`](./whatsappSend.md), [`whatsappDb`](./whatsappDb.md)
- **Architecture docs**: `CLAUDE.md` -> "Event-Driven Trigger Node System", "WhatsApp Integration"
-- **Trigger infrastructure**: [`event_waiter.py`](../../../server/services/event_waiter.py), [`triggers.py`](../../../server/services/handlers/triggers.py)
+- **Trigger infrastructure**: [`event_waiter.py`](../../../server/services/event_waiter.py), [`_filters.py`](../../../server/nodes/whatsapp/_filters.py), [`_events.py`](../../../server/nodes/whatsapp/_events.py), canary path in [`canary_registry.py`](../../../server/services/deployment/canary_registry.py)
diff --git a/docs-internal/node-logic-flows/whatsapp/whatsappSend.md b/docs-internal/node-logic-flows/whatsapp/whatsappSend.md
index 7acb5e64..faff3c1e 100644
--- a/docs-internal/node-logic-flows/whatsapp/whatsappSend.md
+++ b/docs-internal/node-logic-flows/whatsapp/whatsappSend.md
@@ -3,10 +3,10 @@
| Field | Value |
|------|-------|
| **Category** | whatsapp / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/whatsapp.py::handle_whatsapp_send`](../../../server/services/handlers/whatsapp.py) |
+| **Backend handler** | [`server/nodes/whatsapp/whatsapp_send.py`](../../../server/nodes/whatsapp/whatsapp_send.py) (`WhatsAppSendNode`); dispatched via `BaseNode.execute()` -> `@Operation("send")`, which delegates to [`server/nodes/whatsapp/_base.py::handle_whatsapp_send`](../../../server/nodes/whatsapp/_base.py) |
| **Tests** | [`server/tests/nodes/test_whatsapp.py`](../../../server/tests/nodes/test_whatsapp.py) |
| **Skill (if any)** | [`server/skills/social_agent/whatsapp-send-skill/SKILL.md`](../../../server/skills/social_agent/whatsapp-send-skill/SKILL.md) |
-| **Dual-purpose tool** | yes - tool name `whatsappSend` |
+| **Dual-purpose tool** | yes - tool name `whatsapp_send` |
## Purpose
@@ -35,7 +35,7 @@ Key parameters (see frontend definition for the full list and `displayOptions.sh
| `channel_jid` | string | `""` | conditional | Required when `recipient_type == 'channel'`, must end `@newsletter` |
| `message_type` | options | `text` | yes | `text`, `image`, `video`, `audio`, `document`, `sticker`, `location`, `contact` |
| `message` | string | `""` | conditional | Required when `message_type == 'text'` |
-| `format_markdown` | boolean | `false` | no | When true and text message, convert GFM to WhatsApp syntax via `markdown_formatter.to_whatsapp` |
+| `format_markdown` | boolean | `true` | no | When true and text message, convert GFM to WhatsApp syntax via `markdown_formatter.to_whatsapp` |
| `media_source` | options | `base64` | conditional | `base64`, `file`, or `url` for media types |
| `media_data` | string | `""` | conditional | Base64 data when `media_source == 'base64'` |
| `file_path` | string/object | `""` | conditional | Either a path string or upload dict `{type: 'upload', filename, mimeType}` |
@@ -53,8 +53,9 @@ Key parameters (see frontend definition for the full list and `displayOptions.sh
| Handle | Shape | Description |
|--------|-------|-------------|
-| `output-main` | object | Standard envelope result |
-| `output-tool` | object | Same payload when wired into an AI agent's `input-tools` |
+| `output-main` | object | The handler `result` dict (see below). The plugin `Output` model is `WhatsAppSendOutput` (`message_id`, `sent`, `extra="allow"`). |
+
+When wired into an AI agent's `input-tools` handle (`usable_as_tool=True`, tool name `whatsapp_send`) the same `result` payload is returned to the LLM.
### Output payload
@@ -101,7 +102,7 @@ flowchart TD
T2 -- no --> S
T3 --> S
T -- no --> S
- S[await routers.whatsapp.handle_whatsapp_send params] --> S2{data.success?}
+ S[await nodes.whatsapp._service.handle_whatsapp_send params] --> S2{data.success?}
S2 -- no --> Eret6[success=false<br/>error from RPC]
S2 -- yes --> F[Build result dict with<br/>status/recipient/type-specific fields]
F --> OK[success=true]
@@ -117,21 +118,21 @@ flowchart TD
- Empty `message` when `message_type == 'text'`
- **Markdown transform**: only runs when `message_type == 'text'` AND `format_markdown` truthy. Overwrites `parameters['message']` in place before RPC call.
- **Type-specific result fields**: `match message_type` adds `preview`, `media_source`, `caption`, `filename`, `mime_type`, `uploaded_file`, `location`, or `contact_name` based on message type.
-- **File upload detection**: if `file_path` is a dict with `type == 'upload'`, the router uploads bytes and the result includes `uploaded_file` and `mime_type` from the upload dict.
+- **File upload detection**: if `file_path` is a dict with `type == 'upload'`, the `_service.py` send handler uploads bytes and the result includes `uploaded_file` and `mime_type` from the upload dict.
## Side Effects
-- **Database writes**: none directly from this handler. (Tracking, if any, is owned by the router.)
-- **Broadcasts**: none directly - standard node_status transitions come from `NodeExecutor`.
-- **External calls**: WebSocket RPC to the Go service. Default URL `ws://localhost:9400/ws/rpc` (overridable via `WHATSAPP_RPC_URL`). For media URLs, the router additionally downloads via `httpx`.
-- **File I/O**: none in this handler - file uploads are handled by the router.
+- **Database writes**: none directly from this handler. (API-usage cost is recorded by the `@Operation` `cost` metadata via `BaseNode.execute()`.)
+- **Broadcasts**: none directly - standard node_status transitions come from `BaseNode.execute()`.
+- **External calls**: WebSocket RPC to the Go `whatsapp-rpc` service via [`nodes/whatsapp/_service.py::whatsapp_rpc_call`](../../../server/nodes/whatsapp/_service.py). Default URL `ws://localhost:9400/ws/rpc` (overridable via `WHATSAPP_RPC_URL`). For media URLs, `_service.py` additionally downloads via `httpx`.
+- **File I/O**: none in this handler - file uploads are handled in `_service.py`.
- **Subprocess**: none.
## External Dependencies
- **Credentials**: none at the node handler level. Pairing state is owned by the Go service; the handler is credential-free.
- **Services**: `whatsapp-rpc` (Go) on port 9400 (default). `services.markdown_formatter.to_whatsapp` when `format_markdown` is enabled.
-- **Python packages**: `asyncio`. (`httpx` is used inside the router for URL media fetches.)
+- **Python packages**: `asyncio`. (`httpx` is used inside `_service.py` for URL media fetches.)
- **Environment variables**: `WHATSAPP_RPC_URL` (optional, default `ws://localhost:9400/ws/rpc`).
## Edge cases & known limits
@@ -147,4 +148,4 @@ flowchart TD
- **Skills using this as a tool**: [`whatsapp-send-skill/SKILL.md`](../../../server/skills/social_agent/whatsapp-send-skill/SKILL.md)
- **Companion nodes**: [`whatsappDb`](./whatsappDb.md), [`whatsappReceive`](./whatsappReceive.md)
-- **Architecture docs**: [Markdown Formatter](../../../CLAUDE.md) (see "Markdown Formatter Service"), WhatsApp RPC via `routers/whatsapp.py`
+- **Architecture docs**: [Markdown Formatter](../../../CLAUDE.md) (see "Markdown Formatter Service"), WhatsApp RPC via `nodes/whatsapp/_service.py`
diff --git a/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.md b/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.md
index d1c2a755..8954574d 100644
--- a/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.md
+++ b/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | workflow / trigger / utility |
-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) |
+| **Backend handler** | Plugin [`server/nodes/trigger/chat_trigger/__init__.py`](../../../server/nodes/trigger/chat_trigger/__init__.py) (`ChatTriggerNode`); dispatch via `BaseNode.execute()`. The base `TriggerNode.execute` handles the event-waiter wait — the `@Operation("wait")` body is a stub. The generic legacy path is [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (reachable only for the lone deferred-canary trigger, not chatTrigger). |
| **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -11,12 +11,15 @@
## Purpose
Fires when the user sends a message from the Console Panel chat tab. The
-WebSocket handler `send_chat_message` calls
-`broadcaster.send_custom_event('chat_message_received', {...})` (or the
-event-waiter dispatch path) and any `chatTrigger` node whose `sessionId`
-matches (or is `'default'`) receives the event and emits it as output. This
-is the primary way a user feeds an interactive prompt into an `aiAgent` or
-`chatAgent`.
+producer [`server/nodes/trigger/chat_trigger/_events.py`](../../../server/nodes/trigger/chat_trigger/_events.py)
+emits a CloudEvents `WorkflowEvent` (`type: com.machinaos.chat.message.received`)
+via `dispatch.emit`. `chatTrigger` is canary-registered
+(`register_canary_trigger_type`), so `DeploymentManager` starts a
+`TriggerListenerWorkflow` for it; the listener receives the event via Temporal
+Signal and spawns a child `MachinaWorkflow` per matching event. Any `chatTrigger`
+node whose `session_id` matches (or is `'default'`) receives the event and emits
+it as output. This is the primary way a user feeds an interactive prompt into an
+`aiAgent` or `chatAgent`.
## Inputs (handles)
@@ -28,7 +31,7 @@ is the primary way a user feeds an interactive prompt into an `aiAgent` or
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `sessionId` | string | `default` | no | - | Matches the `session_id` on the incoming chat event. If set to `default`, the filter accepts every event. Otherwise it only accepts events with the same `session_id`. |
+| `session_id` | string | `default` | no | - | Matches the `session_id` on the incoming chat event. If set to `default`, the filter accepts every event. Otherwise it only accepts events with the same `session_id`. |
| `placeholder` | string | `Type a message...` | no | - | Frontend display only - not used by the handler. |
## Outputs (handles)
@@ -58,35 +61,34 @@ Wrapped in the standard envelope.
```mermaid
flowchart TD
- A[handle_trigger_node] --> B[event_waiter.get_trigger_config 'chatTrigger']
- B -- None --> Eunk[Return success=false<br/>error: Unknown trigger type]
- B -- ok --> C[await event_waiter.register<br/>params include sessionId]
- C --> D[build_chat_filter:<br/>if sessionId != 'default' require exact match]
- D --> E[broadcaster.update_node_status 'waiting'<br/>event_type=chat_message_received]
- E --> F[await event_waiter.wait_for_event waiter]
- F -- CancelledError --> G[Return success=false<br/>error: Cancelled by user]
- F -- Exception --> H[Return success=false<br/>error: str e]
- F -- ok --> I[Return success envelope<br/>result = event_data]
+ P[chat tab sends message] --> Q[_events.py dispatch.emit<br/>WorkflowEvent com.machinaos.chat.message.received]
+ Q --> R[TriggerListenerWorkflow receives via Temporal Signal]
+ R --> S[ChatTriggerNode.build_filter:<br/>if session_id != 'default' require exact match]
+ S -- match --> T[spawn child MachinaWorkflow<br/>trigger pre-executed with event payload]
+ S -- no match --> R
```
## Decision Logic
-- **Filter** (`build_chat_filter` in `triggers.py`):
+- **Filter** (`ChatTriggerNode.build_filter`):
```python
- if session_id != 'default' and event_session != session_id:
- return False
+ if session_id and session_id != 'default':
+ return event.get('session_id') == session_id
return True
```
- So `sessionId='default'` (the frontend default) is a wildcard - the
+ So `session_id='default'` (the frontend default) is a wildcard - the
trigger fires for every chat message regardless of which session the user
is in.
- **Cancellation**: yields `success=False, error="Cancelled by user"`.
## Side Effects
-- **Database writes**: none in the trigger handler itself. (The chat message
+- **Database writes**: none in the trigger plugin itself. (The chat message
is separately persisted by `send_chat_message` via `database.add_chat_message`.)
-- **Broadcasts**: `update_node_status(node_id, "waiting", {message, event_type, waiter_id}, workflow_id)`.
+- **Broadcasts**: the producer emits a CloudEvents `WorkflowEvent` via
+ `dispatch.emit` (Temporal Signal fan-out + in-process WS broadcast). The
+ `TriggerListenerWorkflow` emits firing-pulse status via
+ `broadcast_trigger_status_activity` before/after each child spawn.
- **External API calls**: none.
- **File I/O**: none.
- **Subprocess**: none.
@@ -100,7 +102,7 @@ flowchart TD
## Edge cases & known limits
-- `sessionId='default'` behaves as a wildcard; setting a unique session ID
+- `session_id='default'` behaves as a wildcard; setting a unique session ID
per trigger is the only way to scope messages to a specific node.
- When multiple `chatTrigger` nodes exist with the same session ID, all of
them fire for a matching message.
diff --git a/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.md b/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.md
index cd80cfe1..ff32095d 100644
--- a/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.md
+++ b/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | workflow / trigger / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_cron_scheduler`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | Plugin [`server/nodes/scheduler/cron_scheduler/__init__.py`](../../../server/nodes/scheduler/cron_scheduler/__init__.py) (`CronSchedulerNode`); dispatch via `BaseNode.execute()` + the `@Operation("trigger")` method. Deployed-workflow cron lifecycle lives in `DeploymentManager` (Temporal Schedules canary via `CronTriggerWorkflow`); this op only runs on a manual / AI-tool invocation. |
| **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) |
| **Skill (if any)** | [`server/skills/task_agent/cron-scheduler-skill/SKILL.md`](../../../server/skills/task_agent/cron-scheduler-skill/SKILL.md) |
| **Dual-purpose tool** | yes - exposed on the `tool` output handle |
@@ -34,11 +34,12 @@ call `event_waiter.register()`.
| `daily_time` | options | `09:00` | no | frequency == `days` | Display-only in the handler. |
| `weekday` | options | `'1'` (Mon) | no | frequency == `weeks` | Display-only. |
| `weekly_time` | options | `09:00` | no | frequency == `weeks` | Display-only. |
-| `month_day` | options | `'1'` | no | frequency == `months` | Display-only. |
-| `monthly_time` | options | `09:00` | no | frequency == `months` | Display-only. |
+| `month_day` | options | `'1'` | no | frequency == `months` | `1`..`28` or `L`. Display-only in this op. |
| `timezone` | options | `UTC` | no | - | Used in the description string only. |
+| `cron_expression` | string | `0 * * * *` | no | - | Vestigial; kept for backward compatibility, unused by the op. |
+| `monthly_time` | string | `09:00` | no | - | Vestigial; kept for backward compatibility, no displayOptions. |
-Note: for `days` / `weeks` / `months` the handler hard-codes the wait to 24h
+Note: for `days` / `weeks` / `months` the op hard-codes the wait to 24h
/ 7d / ~30d respectively; the `daily_time` / `weekday` / `weekly_time` /
`month_day` / `monthly_time` parameters only flow into the human-readable
`schedule` string, not the actual wait duration.
@@ -48,7 +49,8 @@ Note: for `days` / `weeks` / `months` the handler hard-codes the wait to 24h
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Schedule metadata (see below). |
-| `output-tool` | object | Same payload when invoked as an AI agent tool. |
+
+Dual-purpose: the plugin sets `usable_as_tool = True` with `tool_name = "cron_scheduler"`, so when wired to an agent's `input-tools` handle the LLM fills the same Params schema and receives the same payload. There is no separate `output-tool` handle declared.
### Output payload
@@ -73,7 +75,7 @@ Wrapped in the standard envelope.
```mermaid
flowchart TD
- A[handle_cron_scheduler] --> B[frequency = parameters.frequency default 'minutes']
+ A[BaseNode.execute -> trigger op] --> B[frequency = params.frequency default 'minutes']
B --> C[_get_schedule_description<br/>returns e.g. 'Every 5 minutes']
C --> D[_calculate_wait_seconds<br/>maps frequency to seconds]
D --> E[get_status_broadcaster.update_node_status 'waiting']
@@ -122,20 +124,20 @@ flowchart TD
## Edge cases & known limits
-- **Handler is not the real scheduler.** Real cron semantics
+- **This op is not the real scheduler.** Real cron semantics
(exact-time-of-day, weekday selection, timezones) live in
- `DeploymentManager` + APScheduler. The manual-run handler does not use any
- of that - it only sleeps for a fixed interval. `daily_time`, `weekday`,
- `weekly_time`, `month_day`, `monthly_time`, and `timezone` are
- **display-only** in this code path.
-- `iteration` is always `1`. The handler does not loop; repeated firings
- come from APScheduler calling the handler again each tick (deployment mode).
-- `interval`, `interval_minutes`, `interval_hours` are coerced with
- `int(...)`; non-numeric values raise `ValueError` which is caught and
- returned as an error envelope.
-- `weekday` is read via `parameters.get('weekday', '1')` and expected to be
- a string digit - if a number is passed it will raise on `.isdigit()` in
- `_get_schedule_description`, but that only affects the description string.
+ `DeploymentManager` + the Temporal Schedule canary (`CronTriggerWorkflow`).
+ The manual-run op does not use any of that - it only sleeps for a fixed
+ interval. `daily_time`, `weekday`, `weekly_time`, `month_day`,
+ `monthly_time`, and `timezone` are **display-only** in this code path.
+- `iteration` is always `1`. The op does not loop; repeated firings
+ come from the deployed Schedule firing per tick (deployment mode).
+- `interval`, `interval_minutes`, `interval_hours` are Pydantic-validated
+ ints with ranges (`ge`/`le`); out-of-range values fail validation before
+ the op runs.
+- `weekday` is a Pydantic `Literal["0".."6"]`; the op casts it via
+ `int(weekday)` inside `_get_schedule_description`, affecting only the
+ description string.
- Unknown `frequency` values silently use 300s wait and produce
`schedule: "Unknown schedule"`.
diff --git a/docs-internal/node-logic-flows/workflow_triggers/start.md b/docs-internal/node-logic-flows/workflow_triggers/start.md
index fb4b57b7..d0d47d01 100644
--- a/docs-internal/node-logic-flows/workflow_triggers/start.md
+++ b/docs-internal/node-logic-flows/workflow_triggers/start.md
@@ -2,8 +2,8 @@
| Field | Value |
|------|-------|
-| **Category** | workflow / trigger |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_start`](../../../server/services/handlers/utility.py) |
+| **Category** | workflow |
+| **Backend handler** | Plugin [`server/nodes/workflow/start/__init__.py`](../../../server/nodes/workflow/start/__init__.py) (`StartNode`); dispatch via `BaseNode.execute()` + the `@Operation("emit")` method. |
| **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -27,7 +27,7 @@ data.
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `initialData` | string (JSON) | `"{}"` | no | - | JSON document surfaced as the node output. Any parse failure silently becomes `{}`. |
+| `initial_data` | any (JSON string or value) | `None` | no | - | Surfaced as the node output. If a JSON string, it is parsed (parse failure -> `{}`); if `None` -> `{}`; otherwise returned as-is. |
## Outputs (handles)
@@ -38,9 +38,11 @@ data.
### Output payload
```ts
-// Whatever the user put in initialData, parsed as JSON.
-// Invalid JSON -> {}.
-Record<string, unknown>
+// initial_data resolved:
+// None -> {}
+// JSON str -> parsed value (invalid -> {})
+// any other -> returned as-is
+unknown
```
Wrapped in the standard envelope: `{ success: true, result: <payload>, node_id, node_type: "start" }`.
@@ -49,23 +51,25 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, node_id,
```mermaid
flowchart TD
- A[handle_start] --> B[Read parameters.initialData<br/>default: '{}']
- B --> C{json.loads?}
- C -- ok --> D[initial_data = parsed]
- C -- Exception --> E[initial_data = {}]
- D --> F[Return success envelope<br/>result = initial_data]
- E --> F
+ A[BaseNode.execute -> emit op] --> B[raw = params.initial_data]
+ B --> C{raw is None?}
+ C -- yes --> E[return {}]
+ C -- no --> D{raw is str?}
+ D -- yes --> F{json.loads?}
+ F -- ok --> G[return parsed]
+ F -- Exception --> E
+ D -- no --> H[return raw as-is]
```
## Decision Logic
-- **Validation**: none. Empty / missing `initialData` is treated as `"{}"`.
-- **Branches**: single path; either `initialData` parses or it is silently
- replaced with `{}`.
+- **Validation**: none. `None` `initial_data` returns `{}`.
+- **Branches**: `None` -> `{}`; string -> `json.loads` (failure -> `{}`);
+ any other value returned unchanged.
- **Fallbacks**: JSON decode errors are swallowed and produce `{}`.
-- **Error paths**: the handler does not raise; it always returns a
- `success=True` envelope. A malformed `initialData` is **not** reported as an
- error.
+- **Error paths**: the op does not raise; it always returns a
+ `success=True` envelope. A malformed `initial_data` string is **not** reported
+ as an error.
## Side Effects
@@ -85,16 +89,14 @@ flowchart TD
## Edge cases & known limits
-- `start` always returns `success=True` regardless of `initialData` content.
- Invalid JSON is silently coerced to `{}` - users get no warning their data
- was dropped.
-- Although `start` is listed in `TRIGGER_REGISTRY` (as `deploy_triggered`),
- the handler registry in `NodeExecutor._build_handler_registry()` maps
- `start` directly to `handle_start`. The event-waiter path is only exercised
- inside `DeploymentManager` when a workflow is deployed; ad-hoc runs go
- through `handle_start` without waiting for any event.
-- The payload is returned as-is; no validation, schema enforcement, or nested
- template resolution is performed by the handler.
+- `start` always returns `success=True` regardless of `initial_data` content.
+ Invalid JSON in a string is silently coerced to `{}` - users get no warning
+ their data was dropped.
+- `start` is a plain `ActionNode` (group `workflow`), not an event trigger -
+ it registers no event waiter and completes immediately. It carries
+ `ui_hints` `hideInputSection` / `hideOutputSection` / `hasInitialDataBlob`.
+- The payload is returned as-is for non-string values; no validation, schema
+ enforcement, or nested template resolution is performed by the op.
## Related
diff --git a/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.md b/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.md
index 905d1bb3..c371ebb6 100644
--- a/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.md
+++ b/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | workflow / trigger |
-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) |
+| **Backend handler** | Plugin [`server/nodes/trigger/task_trigger/__init__.py`](../../../server/nodes/trigger/task_trigger/__init__.py) (`TaskTriggerNode`); dispatch via `BaseNode.execute()`. The base `TriggerNode.execute` runs the event wait; the `@Operation("wait")` body is a stub. |
| **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -13,10 +13,14 @@
Fires when a child agent delegated via `delegate_to_*` tools completes or
errors. The delegation code path in
`server/services/handlers/tools.py::_execute_delegated_agent` calls
-`broadcaster.send_custom_event('task_completed', {...})`, which is
-dispatched to this trigger via `event_waiter`. Used to let a parent
-workflow react to child completion without blocking the parent on the
-child's result (fire-and-forget delegation).
+`nodes.agent._events.broadcast_agent_task_completed` for both the success and
+error paths, which emits a CloudEvents `WorkflowEvent`
+(`type: com.machinaos.agent.task.completed`, with the success/error
+discriminator carried in `data.status`) via `dispatch.emit`. `taskTrigger` is
+canary-registered, so `DeploymentManager` starts a `TriggerListenerWorkflow`
+that receives the event via Temporal Signal and spawns a child workflow per
+match. Used to let a parent workflow react to child completion without blocking
+the parent on the child's result (fire-and-forget delegation).
## Inputs (handles)
@@ -41,39 +45,38 @@ child's result (fire-and-forget delegation).
### Output payload
+Declared `TaskTriggerOutput` fields:
+
```ts
{
- task_id: string;
- status: 'completed' | 'error';
- agent_name: string;
- agent_node_id: string;
- parent_node_id: string;
+ task_id?: string;
+ status?: 'completed' | 'error';
+ agent_name?: string;
result?: string; // Present when status='completed'
error?: string; // Present when status='error'
- workflow_id: string;
+ workflow_id?: string;
}
```
-Wrapped in the standard envelope.
+`TaskTriggerOutput` is `model_config extra="allow"`, so the producer's extra
+fields (`agent_node_id`, `parent_node_id`) pass through to downstream nodes at
+runtime even though they are not declared on the model. Wrapped in the standard
+envelope.
## Logic Flow
```mermaid
flowchart TD
- A[handle_trigger_node] --> B[event_waiter.get_trigger_config 'taskTrigger']
- B -- None --> Eunk[Return success=false<br/>error: Unknown trigger type]
- B -- ok --> C[await event_waiter.register]
- C --> D[build_task_completed_filter]
- D --> E[broadcaster.update_node_status 'waiting'<br/>event_type=task_completed]
- E --> F[await event_waiter.wait_for_event waiter]
- F -- CancelledError --> G[Return success=false<br/>error: Cancelled by user]
- F -- Exception --> H[Return success=false<br/>error: str e]
- F -- ok --> I[Return success envelope<br/>result = event_data]
+ P[child agent completes/errors] --> Q[broadcast_agent_task_completed<br/>dispatch.emit com.machinaos.agent.task.completed]
+ Q --> R[TriggerListenerWorkflow receives via Temporal Signal]
+ R --> S[TaskTriggerNode.build_filter<br/>task_id / status / agent_name / parent_node_id]
+ S -- match --> T[spawn child MachinaWorkflow<br/>trigger pre-executed with event payload]
+ S -- no match --> R
```
## Decision Logic
-`build_task_completed_filter` applies all configured filters as AND:
+`TaskTriggerNode.build_filter` applies all configured filters as AND:
- **task_id**: exact match if non-empty.
- **agent_name**: `agent_name_filter.lower() in event_agent.lower()` -
@@ -89,7 +92,9 @@ If any filter rejects, the event is skipped and the waiter stays blocked.
## Side Effects
- **Database writes**: none.
-- **Broadcasts**: `update_node_status(node_id, "waiting", {message, event_type, waiter_id}, workflow_id)`.
+- **Broadcasts**: the producer emits a CloudEvents `WorkflowEvent` via
+ `dispatch.emit`. The `TriggerListenerWorkflow` emits firing-pulse status via
+ `broadcast_trigger_status_activity` around each child spawn.
- **External API calls**: none.
- **File I/O**: none.
- **Subprocess**: none.
@@ -97,9 +102,11 @@ If any filter rejects, the event is skipped and the waiter stays blocked.
## External Dependencies
- **Credentials**: none.
-- **Services**: `services.event_waiter`, `services.status_broadcaster`.
+- **Services**: `services.deployment` (canary listener), `services.events.dispatch`,
+ `services.status_broadcaster`.
- **Upstream dispatcher**: `services.handlers.tools._execute_delegated_agent`
- fires `task_completed` events for both success and error paths.
+ calls `nodes.agent._events.broadcast_agent_task_completed` for both success and
+ error paths.
- **Python packages**: stdlib only.
- **Environment variables**: none.
diff --git a/docs-internal/node-logic-flows/workflow_triggers/timer.md b/docs-internal/node-logic-flows/workflow_triggers/timer.md
index eda42c21..8d5b1827 100644
--- a/docs-internal/node-logic-flows/workflow_triggers/timer.md
+++ b/docs-internal/node-logic-flows/workflow_triggers/timer.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | workflow / utility / tool (dual-purpose) |
-| **Backend handler** | [`server/services/handlers/utility.py::handle_timer`](../../../server/services/handlers/utility.py) |
+| **Backend handler** | Plugin [`server/nodes/scheduler/timer/__init__.py`](../../../server/nodes/scheduler/timer/__init__.py) (`TimerNode`); dispatch via `BaseNode.execute()` + the `@Operation("wait")` method. |
| **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | yes - exposed on the `tool` output handle so AI agents can invoke it |
@@ -26,7 +26,7 @@ of next-run; it simply pauses the branch of the workflow it is on for
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `duration` | number | `5` | yes | - | How long to wait. Frontend clamps 1..3600. |
+| `duration` | number | `1` | yes | - | How long to wait. Pydantic-validated `ge=1, le=86400`. |
| `unit` | options | `seconds` | no | - | One of `seconds` / `minutes` / `hours`. Unknown values are treated as `seconds`. |
## Outputs (handles)
@@ -34,7 +34,8 @@ of next-run; it simply pauses the branch of the workflow it is on for
| Handle | Shape | Description |
|--------|-------|-------------|
| `output-main` | object | Timing metadata (see below). |
-| `output-tool` | object | Same payload when used as an AI agent tool. |
+
+Dual-purpose: the plugin sets `usable_as_tool = True` with `tool_name = "timer"`, so when wired to an agent's `input-tools` handle the LLM fills the same Params schema. There is no separate `output-tool` handle declared.
### Output payload
@@ -54,7 +55,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution
```mermaid
flowchart TD
- A[handle_timer] --> B[duration = int parameters.duration default 5]
+ A[BaseNode.execute -> wait op] --> B[duration = int params.duration]
B --> C{unit?}
C -- seconds --> S1[wait_seconds = duration]
C -- minutes --> S2[wait_seconds = duration * 60]
@@ -101,13 +102,12 @@ flowchart TD
- Only three unit strings are supported - anything else is silently treated
as raw seconds. There is no surfaced validation error.
-- The handler has no timeout ceiling of its own; the frontend clamps at
- 3600 (1h) for `seconds`, but if a workflow injects a larger value via the
- backend it will actually wait that long.
-- Cancellation returns a failed envelope (`success=false`). Downstream nodes
- that key off `success` will not execute when the timer is cancelled.
-- Duration is coerced with `int(...)`; a non-numeric string raises `ValueError`
- which is caught by the generic handler and turned into an error envelope.
+- `duration` is Pydantic-validated `ge=1, le=86400`; for `hours` that is up
+ to 86400 hours, so a large value with `unit=hours` can sleep effectively
+ indefinitely.
+- Cancellation (`asyncio.CancelledError`) is re-raised as `RuntimeError("Timer
+ cancelled")`, surfaced as a failed envelope. Downstream nodes that key off
+ `success` will not execute when the timer is cancelled.
## Related
diff --git a/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.md b/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.md
index 9336905b..f63c1269 100644
--- a/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.md
+++ b/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | workflow / utility |
-| **Backend handler** | [`server/services/handlers/http.py::handle_webhook_response`](../../../server/services/handlers/http.py) |
+| **Backend handler** | Plugin [`server/nodes/utility/webhook_response/__init__.py`](../../../server/nodes/utility/webhook_response/__init__.py) (`WebhookResponseNode`); dispatch via `BaseNode.execute()` + the `@Operation("respond")` method (body inlined from the deleted `handlers/http.py`). |
| **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -22,21 +22,22 @@ out.
| Handle | Connection type | Required | Purpose |
|--------|-----------------|----------|---------|
-| `input-main` | main | yes (semantically) | Upstream node outputs are collected via `_get_connected_outputs_with_info` and made available for template substitution / default body. |
+| `input-main` | main | yes (semantically) | Upstream node outputs are surfaced as `ctx.raw["connected_outputs"]` and made available for template substitution / default body. |
## Parameters
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `statusCode` | number | `200` | no | - | HTTP status code. Coerced via `int(...)`. |
-| `responseBody` | string | `""` | no | - | Response body. Supports `{{input.<field>}}` and `{{<nodeType>.<field>}}` template substitutions using connected upstream outputs. |
-| `contentType` | options | `application/json` | no | - | One of `application/json` / `text/plain` / `text/html`. |
+| `status_code` | number | `200` | no | - | HTTP status code. Validated `100..599` (Pydantic `ge=100, le=599`). |
+| `body` | any | `None` | no | - | Response body. When a string, supports `{{input.<field>}}` and `{{<nodeType>.<field>}}` template substitutions using connected upstream outputs. Non-string values are JSON-serialized. |
+| `headers` | object | `{}` | no | - | Header dict (currently not forwarded by `resolve_webhook_response`). |
+| `content_type` | string | `application/json` | no | - | Free-form content type string. |
## Outputs (handles)
| Handle | Shape | Description |
|--------|-------|-------------|
-| (none) | - | `webhookResponse` has no output handle - it terminates a response branch. The handler still returns a standard envelope (used by the executor for status tracking). |
+| `output-main` | object | The respond op's return payload (see below); typically terminates a response branch. |
### Handler return payload
@@ -55,29 +56,26 @@ Wrapped in the standard envelope.
```mermaid
flowchart TD
- A[NodeExecutor._dispatch] --> B[_get_connected_outputs_with_info<br/>returns connected_outputs + source_nodes]
- B --> C[handle_webhook_response<br/>node_id / params / context / connected_outputs]
- C --> D[status_code = int params.statusCode default 200]
- D --> E[response_body = params.responseBody]
- E --> F{response_body non-empty AND connected_outputs?}
- F -- yes --> G[For each node_type, output in connected_outputs:<br/>replace '{{input.key}}' and '{{node_type.key}}' with str value]
+ A[BaseNode.execute -> respond op] --> B[outputs = ctx.raw connected_outputs]
+ B --> D[body = params.body]
+ D --> F{body is str AND outputs?}
+ F -- yes --> G[For each node_type, output in outputs:<br/>replace '{{input.key}}' and '{{node_type.key}}' with str value]
F -- no --> H
- G --> H{response_body still empty?}
- H -- yes AND connected_outputs present --> I[response_body = json.dumps first output]
+ G --> H{body falsy AND outputs present?}
+ H -- yes --> I[body = json.dumps first output]
H -- no --> J
- I --> J[resolve_webhook_response node_id, statusCode, body, contentType]
- J --> K[Return success envelope]
- J -- Exception --> L[Return success=false<br/>error: str e]
+ I --> J[resolve_webhook_response node_id, status_code, body_text, content_type]
+ J --> K[Return sent / statusCode / contentType / bodyLength]
```
## Decision Logic
-- **Template resolution**: only applied when `response_body` is non-empty.
- Two template formats are supported:
+- **Template resolution**: only applied when `body` is a non-empty string AND
+ there are connected outputs. Two template formats are supported:
- `{{input.<key>}}` - pulls from any connected node's output dict
- `{{<nodeType>.<key>}}` - pulls from a specific connected node type
-- **Empty body fallback**: if `response_body` is empty AND at least one
- upstream node has output, the handler JSON-serialises the FIRST output
+- **Empty body fallback**: if `body` is falsy AND at least one
+ upstream node has output, the op JSON-serialises the FIRST output
(iteration order of a Python dict keyed by node type) and uses that as
the body. If there are no upstream outputs the body stays empty.
- **resolve_webhook_response**: imported lazily from `routers.webhook`;
@@ -105,8 +103,8 @@ flowchart TD
- `resolve_webhook_response` is looked up by `node_id`. If no matching
webhookTrigger is pending (e.g. this node runs in a workflow without an
- upstream `webhookTrigger`, or the trigger uses `responseMode=immediate`),
- `resolve_webhook_response` simply no-ops - the handler still returns
+ upstream `webhookTrigger`, or the trigger uses `response_mode=immediate`),
+ `resolve_webhook_response` simply no-ops - the op still returns
`success=True`. There is no warning that the response was dropped.
- Template substitution uses plain `str.replace`; there is no escaping.
Binary / non-stringifiable values will render as their Python `repr`.
@@ -114,12 +112,10 @@ flowchart TD
(insertion order in CPython 3.7+) - which in turn depends on edge order.
If multiple upstream nodes are connected, the output picked is not
explicitly controlled by the user.
-- `statusCode` coercion via `int(...)` will raise on non-numeric strings,
- which is then caught and returned as a failed envelope.
-- Unlike other handlers in the registry, `webhookResponse` is dispatched
- in `NodeExecutor._dispatch` (not the handler registry) because it needs
- the `connected_outputs` list - see
- [`node_executor.py:367-381`](../../../server/services/node_executor.py).
+- `status_code` is validated by Pydantic (`ge=100, le=599`); an out-of-range
+ value fails validation and produces an error envelope before the op runs.
+- `webhookResponse` reads `ctx.raw["connected_outputs"]` — the upstream
+ outputs are surfaced into the node context by the executor for this plugin.
## Related
diff --git a/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.md b/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.md
index bf586d5a..0c8065be 100644
--- a/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.md
+++ b/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.md
@@ -3,7 +3,7 @@
| Field | Value |
|------|-------|
| **Category** | workflow / trigger |
-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) |
+| **Backend handler** | Plugin [`server/nodes/trigger/webhook_trigger/__init__.py`](../../../server/nodes/trigger/webhook_trigger/__init__.py) (`WebhookTriggerNode`); dispatch via `BaseNode.execute()`. The base `TriggerNode.execute` runs the event wait; the `@Operation("wait")` body is a stub. |
| **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) |
| **Skill (if any)** | none |
| **Dual-purpose tool** | no |
@@ -11,11 +11,13 @@
## Purpose
Start a workflow when an HTTP request hits `/webhook/{path}`. The
-`webhook` router in `server/routers/webhook.py` receives the request and
-dispatches a `webhook_received` event via
-`event_waiter.dispatch()`. This node registers an asyncio.Future waiter
-(memory mode) or a Redis-Streams consumer group (Redis mode) via the shared
-`event_waiter` module and blocks until a matching event is delivered.
+`webhook` router in `server/routers/webhook.py` receives the request and the
+producer [`server/nodes/trigger/webhook_trigger/_events.py`](../../../server/nodes/trigger/webhook_trigger/_events.py)
+emits a CloudEvents `WorkflowEvent` (`type: com.machinaos.webhook.received`)
+via `dispatch.emit`. `webhookTrigger` is canary-registered, so
+`DeploymentManager` starts a `TriggerListenerWorkflow` that receives the event
+via Temporal Signal and spawns a child `MachinaWorkflow` per matching event
+(filtered by `path`).
## Inputs (handles)
@@ -27,12 +29,12 @@ dispatches a `webhook_received` event via
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
-| `path` | string | `""` | yes | - | URL path segment - full URL is `http://host:3010/webhook/{path}`. Must match the incoming request's `path` for the filter to accept it. |
-| `method` | options | `POST` | no | - | Filter for HTTP method at the router layer (not the handler's filter). Values: `GET` / `POST` / `PUT` / `DELETE` / `ALL`. |
-| `responseMode` | options | `immediate` | no | - | `immediate` returns 200 OK right away; `responseNode` waits for a downstream `webhookResponse` node (see [`webhookResponse`](./webhookResponse.md)). |
+| `path` | string | `""` | no | - | URL path segment - full URL is `http://host:3010/webhook/{path}`. Must match the incoming request's `path` for the filter to accept it. Empty path = wildcard (matches any). |
+| `method` | options | `POST` | no | - | Filter for HTTP method at the router layer (not the plugin filter). Values: `GET` / `POST` / `PUT` / `DELETE` / `ALL`. |
+| `response_mode` | options | `immediate` | no | - | `immediate` returns 200 OK right away; `responseNode` waits for a downstream `webhookResponse` node (see [`webhookResponse`](./webhookResponse.md)). |
| `authentication` | options | `none` | no | - | `none` or `header`. |
-| `headerName` | string | `X-API-Key` | no | authentication == `header` | Expected header name. |
-| `headerValue` | string | `""` | no | authentication == `header` | Expected header value. |
+| `header_name` | string | `X-API-Key` | no | authentication == `header` | Expected header name. |
+| `header_value` | string (secret) | `""` | no | authentication == `header` | Expected header value. |
## Outputs (handles)
@@ -61,21 +63,17 @@ Wrapped in the standard envelope.
```mermaid
flowchart TD
- A[handle_trigger_node] --> B[event_waiter.get_trigger_config 'webhookTrigger']
- B -- None --> Eunk[Return success=false<br/>error: Unknown trigger type]
- B -- ok --> C[await event_waiter.register<br/>node_type, node_id, params]
- C --> D[build_webhook_filter: checks path equality]
- D --> E[StatusBroadcaster.update_node_status 'waiting'<br/>with event_type=webhook_received, waiter_id]
- E --> F[await event_waiter.wait_for_event waiter]
- F -- CancelledError --> G[Return success=false<br/>error: Cancelled by user]
- F -- Exception --> H[Return success=false<br/>error: str e]
- F -- ok --> I[Return success envelope<br/>result = event_data]
+ P[HTTP request hits /webhook/path] --> Q[routers/webhook.py + _events.py<br/>dispatch.emit com.machinaos.webhook.received]
+ Q --> R[TriggerListenerWorkflow receives via Temporal Signal]
+ R --> S[WebhookTriggerNode.build_filter:<br/>event.path == params.path]
+ S -- match --> T[spawn child MachinaWorkflow<br/>trigger pre-executed with event payload]
+ S -- no match --> R
```
## Decision Logic
-- **Filter match** (`build_webhook_filter` in `triggers.py`): accepts any
- event whose `data.path` equals `params.path`. If `path` is empty the
+- **Filter match** (`WebhookTriggerNode.build_filter`): accepts any
+ event whose `path` equals `params.path`. If `path` is empty the
filter accepts anything.
- **Method / authentication** are NOT enforced inside the filter - they are
supposed to be enforced at the router layer when the HTTP request lands.
@@ -85,10 +83,10 @@ flowchart TD
## Side Effects
-- **Database writes**: none inside the handler. (Redis mode writes waiter
- metadata to a Redis key `waiters:<uuid>` with a 24h TTL via `CacheService`.)
-- **Broadcasts**: `update_node_status(node_id, "waiting", {message, event_type, waiter_id}, workflow_id)`
- exactly once when the waiter is registered.
+- **Database writes**: none inside the plugin.
+- **Broadcasts**: the producer emits a CloudEvents `WorkflowEvent` via
+ `dispatch.emit`. The `TriggerListenerWorkflow` emits firing-pulse status via
+ `broadcast_trigger_status_activity` around each child spawn.
- **External API calls**: none.
- **File I/O**: none.
- **Subprocess**: none.
@@ -96,9 +94,9 @@ flowchart TD
## External Dependencies
- **Credentials**: none.
-- **Services**: `services.event_waiter`, `services.status_broadcaster`,
- `routers.webhook` (dispatches events via `broadcaster.send_custom_event` /
- `event_waiter.dispatch`).
+- **Services**: `services.deployment` (canary listener), `services.events.dispatch`,
+ `services.status_broadcaster`, `routers.webhook` (receives the HTTP request and
+ drives `_events.py` to emit the event).
- **Python packages**: stdlib only.
- **Environment variables**: none.
diff --git a/docs-internal/node_creation.md b/docs-internal/node_creation.md
index 3ff26134..440e6fa1 100644
--- a/docs-internal/node_creation.md
+++ b/docs-internal/node_creation.md
@@ -36,68 +36,18 @@ stripe) layer additional bases from `services.events` underneath them.
## Five-minute recipe — single file
For nodes with no state, no daemon, no signed webhooks (the common
-case):
-
-```python
-# server/nodes/<group>/<name>.py
-from pydantic import BaseModel, Field
-from services.plugin import (
- ActionNode, ApiKeyCredential, NodeContext, Operation, TaskQueue,
-)
-
-
-class MyCredential(ApiKeyCredential):
- id = "my_api"
- display_name = "My Service"
- category = "Search"
- key_name = "X-API-Key"
-
-
-class MyParams(BaseModel):
- query: str = Field(..., min_length=1)
-
-
-class MyOutput(BaseModel):
- results: list
-
-
-class MyNode(ActionNode):
- type = "myNode"
- display_name = "My Node"
- # Icon + color: drop `icon.svg` into this plugin folder + create
- # `meta.json` with `{"color": "#abcdef"}`. For multi-node folders
- # (one folder, multiple node classes — telegram / whatsapp / stripe),
- # use `icon_<nodeType>.svg` for per-node-type icons; the resolver
- # picks per-node first, falling back to shared `icon.svg`. emoji /
- # lobehub:<brand> live in `visuals.json` for icon-only entries.
- group = ("search", "tool")
- description = "Brief description for palette + AI tool."
- component_kind = "square"
- handles = (
- {"name": "input-main", "kind": "input", "position": "left",
- "label": "Input", "role": "main"},
- {"name": "output-main", "kind": "output", "position": "right",
- "label": "Output", "role": "main"},
- )
- credentials = (MyCredential,)
- task_queue = TaskQueue.REST_API
- usable_as_tool = True
- Params = MyParams
- Output = MyOutput
-
- @Operation("search")
- async def search(self, ctx: NodeContext, params: MyParams) -> MyOutput:
- async with ctx.connection("my_api") as conn:
- resp = await conn.get(
- "https://api.example.com/search",
- params={"q": params.query},
- )
- return MyOutput(results=resp.json().get("hits", []))
-```
-
+case): the canonical single-file recipe code block lives in the
+cookbook — see
+[`server/nodes/README.md` → Five-minute recipe](../server/nodes/README.md#five-minute-recipe).
+
+It is a single `server/nodes/<group>/<name>.py` declaring a
+`Credential` subclass, a `Params` model, an `Output` model, and one
+`ActionNode` subclass with `type` / `display_name` / `group` /
+`component_kind` / `handles` / `credentials` / `task_queue` /
+`usable_as_tool` / `Params` / `Output` plus one `@Operation` method.
That's the entire node. On server restart it auto-registers, the
-NodeSpec is emitted at `/api/schemas/nodes/myNode/spec.json`, and it
-appears in the Component Palette under the `search` group.
+NodeSpec is emitted at `/api/schemas/nodes/<type>/spec.json`, and it
+appears in the Component Palette under its first `group` entry.
## Five-minute recipe — Wave 12 self-contained folder (signed webhook + CLI)
diff --git a/docs-internal/onboarding.md b/docs-internal/onboarding.md
index de9d46ac..2be60e17 100644
--- a/docs-internal/onboarding.md
+++ b/docs-internal/onboarding.md
@@ -4,6 +4,8 @@
The onboarding service provides a multi-step welcome wizard that appears after a user's first launch, guiding them through platform capabilities, key concepts, API key setup, UI layout, and getting started. It is database-backed, skippable, resumable, and replayable from Settings.
+The frontend is **fully shadcn/ui + Tailwind** — antd was removed from `client/src/`. The wizard composes the project's `Modal` primitive, shadcn `Button` / `ActionButton` / `Card` / `Badge` / `Alert`, and `lucide-react` icons. The step progress indicator is a hand-rolled `<ol>` driven by node-role tokens (no antd `Steps`).
+
## Architecture
```
@@ -12,24 +14,27 @@ The onboarding service provides a multi-step welcome wizard that appears after a
│ ┌────────────────────────────────────────────────────────┐ │
│ │ OnboardingWizard.tsx │ │
│ │ ┌────────────────────────────────────────────────┐ │ │
-│ │ │ useOnboarding(reopenTrigger) │ │ │
-│ │ │ - Checks onboarding_completed via WebSocket │ │ │
+│ │ │ useOnboarding(reopenTrigger, STEPS.length) │ │ │
+│ │ │ - Reads onboarding_completed/step via │ │ │
+│ │ │ useUserSettingsQuery (TanStack Query, WS) │ │ │
│ │ │ - Manages step navigation + persistence │ │ │
│ │ └───────────────┬────────────────────────────────┘ │ │
│ │ │ │ │
-│ │ ┌───────┬───────┼───────┬──────────┐ │ │
-│ │ │Step 0 │Step 1 │Step 2 │Step 3 │Step 4 │ │
-│ │ │Welcome│Concept│APIKey │Canvas │GetStarted │ │
-│ │ └───────┴───────┴───────┴──────────┘ │ │
+│ │ STEPS array (single source of truth in wizard): │ │
+│ │ ┌───────┬───────┬───────┬──────────┬──────────┐ │ │
+│ │ │Step 0 │Step 1 │Step 2 │Step 3 │Step 4 │ │ │
+│ │ │Welcome│Concept│APIKey │Canvas │GetStarted│ │ │
+│ │ └───────┴───────┴───────┴──────────┴──────────┘ │ │
│ │ │ │
-│ │ Modal (Radix UI) + Steps (Ant Design) │ │
+│ │ Modal (project primitive, Radix-backed) │ │
+│ │ + <ol> progress stepper (Tailwind + role tokens) │ │
│ └────────────────────────────────────────────────────────┘ │
│ │
│ SettingsPanel.tsx → "Replay Welcome Guide" button │
│ └── onReplayOnboarding → increments reopenTrigger │
└────────────────────────────────────────────────────────────────┘
│ │
- │ WebSocket │ WebSocket
+ │ WebSocket (via TanStack Q) │ WebSocket
▼ ▼
┌──────────────────────────────────────────────────────────────┐
│ server/routers/websocket.py │
@@ -82,16 +87,19 @@ if "onboarding_step" not in columns:
```
client/src/
├── components/
-│ └── onboarding/
-│ ├── OnboardingWizard.tsx # Main wizard modal orchestrator
-│ └── steps/
-│ ├── WelcomeStep.tsx # Step 0: Platform intro
-│ ├── ConceptsStep.tsx # Step 1: Nodes, Edges, Agents, Skills, Modes
-│ ├── ApiKeyStep.tsx # Step 2: AI provider key setup
-│ ├── CanvasStep.tsx # Step 3: UI layout tour
-│ └── GetStartedStep.tsx # Step 4: First workflow tips
+│ ├── onboarding/
+│ │ ├── OnboardingWizard.tsx # Main wizard modal orchestrator + STEPS SSOT + <ol> stepper
+│ │ ├── nodeRoleClasses.ts # Shared role→Tailwind-token map for step cards
+│ │ └── steps/
+│ │ ├── WelcomeStep.tsx # Step 0: Platform intro
+│ │ ├── ConceptsStep.tsx # Step 1: Nodes, Edges, Agents, Skills, Modes
+│ │ ├── ApiKeyStep.tsx # Step 2: AI provider key setup
+│ │ ├── CanvasStep.tsx # Step 3: UI layout tour
+│ │ └── GetStartedStep.tsx # Step 4: First workflow tips
+│ └── icons/
+│ └── AIProviderIcons.tsx # Provider SVG icons used by ApiKeyStep
└── hooks/
- └── useOnboarding.ts # State management + WebSocket persistence
+ └── useOnboarding.ts # State management + TanStack-Query persistence
```
## Components
@@ -100,29 +108,33 @@ client/src/
**Location**: `client/src/hooks/useOnboarding.ts`
-Custom hook managing the full onboarding lifecycle:
+Custom hook managing the full onboarding lifecycle. Persistence rides the **TanStack Query** server-state layer (`useUserSettingsQuery` / `useSaveUserSettingsMutation` from `useUserSettingsQuery.ts`), which are themselves WebSocket-backed (`get_user_settings` / `save_user_settings`). The hook does NOT call WebSocket handlers directly.
```typescript
-export const useOnboarding = (reopenTrigger?: number) => {
- // Returns:
+export const useOnboarding = (
+ reopenTrigger?: number,
+ totalSteps: number = 5, // DEFAULT_TOTAL_STEPS; caller passes STEPS.length
+) => {
+ // Returns (spread of OnboardingState + actions):
// - isVisible: boolean - Whether wizard should render
- // - currentStep: number - Active step index (0-4)
+ // - currentStep: number - Active step index (0..totalSteps-1)
// - isCompleted: boolean - Whether already completed/skipped
- // - isLoading: boolean - WebSocket check in progress
- // - hasChecked: boolean - Initial check done
- // - totalSteps: number - Always 5
- // - nextStep(): void - Advance to next step (or complete if last)
- // - prevStep(): void - Go back one step
- // - skip(): void - Skip onboarding entirely
- // - complete(): void - Mark as completed
+ // - isLoading: boolean - Settings query in progress
+ // - hasChecked: boolean - Initial hydration done
+ // - totalSteps: number - Echoed back from the param
+ // - nextStep(): void - Advance (completes + persists when next >= totalSteps)
+ // - prevStep(): void - Go back one step (clamped at 0)
+ // - skip(): void - Skip: persist current step, completed=true, hide
+ // - complete(): void - Persist totalSteps, completed=true, hide
};
```
**Key behaviors**:
-- On WebSocket connect, calls `get_user_settings` to check `onboarding_completed` (once, using `hasCheckedRef`)
-- Each step transition calls `save_user_settings` to persist progress
-- `reopenTrigger` prop change resets state and reopens wizard from step 0
-- StrictMode-safe via `hasCheckedRef` pattern (same as Dashboard.tsx)
+- `totalSteps` is a **parameter** (default 5). The wizard owns the step list and passes `STEPS.length`, so the hook never hardcodes the count — it uses `totalSteps` only to detect last-step completion in `nextStep`.
+- Hydrates UI state from `settingsQuery.data` on `isSuccess`: reads `onboarding_completed` / `onboarding_step`. Visibility flips only on first hydration (`prev.hasChecked ? prev.isVisible : !completed`) so a user-closed wizard does not re-open on later query refetches.
+- Each navigation (`nextStep` / `prevStep` / `skip` / `complete`) calls `saveSettings.mutate({ onboarding_step, onboarding_completed })` to persist progress.
+- Query errors surface as a non-blocking "checked" state (`isLoading=false, hasChecked=true`) so the app continues even if the round-trip failed.
+- `reopenTrigger` prop change (when `> 0`) resets state and reopens the wizard from step 0.
### OnboardingWizard
@@ -134,27 +146,47 @@ export const useOnboarding = (reopenTrigger?: number) => {
| `onOpenCredentials` | `() => void` | Opens CredentialsModal (passed from Dashboard) |
| `reopenTrigger` | `number?` | Incrementing counter triggers wizard reopen |
-**UI Structure**:
-- Uses existing `Modal` component (`maxWidth="580px"`, `maxHeight="70vh"`, `autoHeight`)
-- Ant Design `Steps` component for progress indicator
-- Step content rendered via switch on `currentStep`
-- Footer: "Skip for now" (left) | "Back" + "Next"/"Start Building" (right)
-- Button colors: Next = `dracula.purple`, final "Start Building" = `dracula.green`
-- Only renders when `isVisible && hasChecked && !isLoading`
+**`STEPS` is the single source of truth.** The wizard declares a module-scope `STEPS` array of `{ title, render }` entries. Its `.length` feeds the hook's `totalSteps`, the progress indicator renders one node per entry, and the active step's `render` is dispatched by index. Adding a step is a one-line edit to this array.
+
+```typescript
+const STEPS = [
+ { title: 'Welcome', render: () => <WelcomeStep /> },
+ { title: 'Concepts', render: () => <ConceptsStep /> },
+ { title: 'API Keys', render: ({ onOpenCredentials }) => <ApiKeyStep onOpenCredentials={onOpenCredentials} /> },
+ { title: 'Canvas', render: () => <CanvasStep /> },
+ { title: 'Get Started', render: () => <GetStartedStep /> },
+];
+```
+
+**UI Structure** (all shadcn/Tailwind, no antd):
+- Project `Modal` primitive with `maxWidth="95vw"`, `maxHeight="95vh"`, titled "Welcome Guide"; `onClose` is wired to `skip`.
+- Progress indicator: a hand-rolled `<ol>` of step pills. Each pill is a rounded number/`Check` (lucide) badge with one of three statuses — `completed` (filled `bg-primary text-primary-foreground`), `active` (`border-primary text-primary`), `upcoming` (`border-border text-muted-foreground`) — joined by a connector `<div>` (`bg-primary` once passed, else `bg-border`). No antd `Steps`.
+- Step content rendered via `STEPS[safeIndex].render({ onOpenCredentials })` inside a scrollable `max-h-[calc(95vh-200px)]` container.
+- Footer: shadcn `Button variant="ghost"` "Skip for now" (left) | `Button variant="outline"` "Back" (shown when `currentStep > 0`) + an `ActionButton` on the right.
+- The right-side primary button uses **`ActionButton` intents**, not raw colour hex: `<ActionButton intent="tools">` for "Next" (with `ArrowRight`), `<ActionButton intent="run">` for the final "Start Building" (with `Check`). The pre-antd-removal dracula-purple/green hardcoding is gone.
+- Only renders when `isVisible && hasChecked && !isLoading`.
+
+### Node-role token map
+
+**Location**: `client/src/components/onboarding/nodeRoleClasses.ts`
+
+`NODE_ROLE_CLASSES` maps a `NodeRole` (`model | skill | agent | workflow | trigger`) to the matching `--node-X` triplet (`{ card: 'bg-node-X-soft border-node-X-border', text: 'text-node-X' }`). `ConceptsStep` and `GetStartedStep` key their card surfaces off this so the cards track every theme with **no opacity arithmetic at the call site**.
### Step Components
-All steps use Ant Design components (`Typography`, `Card`, `Tag`, `Space`, `Button`, `Alert`, `Row`, `Col`) and `@ant-design/icons` for consistent theming.
+All steps are shadcn/Tailwind compositions using `lucide-react` icons (`Card` / `CardContent`, `Badge`, `Alert` / `AlertDescription`, `Button`, plus role tokens). No antd, no `@ant-design/icons`.
+
+| Step | Component | Title | Purpose | Primitives used |
+|------|-----------|-------|---------|-----------------|
+| 0 | `WelcomeStep` | Welcome to MachinaOs | Platform intro + 2×2 feature grid | `Card` / `CardContent`, lucide `Rocket` / `Plug` / `Move` / `Zap`, `bg-node-agent-soft` |
+| 1 | `ConceptsStep` | Key Concepts | Nodes, Edges, AI Agents, Skills & Tools, Normal vs Dev Mode | role-token cards via `NODE_ROLE_CLASSES`, lucide `LayoutGrid` / `GitBranch` / `Bot` / `Wrench` / `ArrowLeftRight` |
+| 2 | `ApiKeyStep` | API Key Setup | Provider list + "Open Credentials" button | shadcn `Button`, `Alert variant="info"`, `AIProviderIcons`, lucide `Key` / `ExternalLink` |
+| 3 | `CanvasStep` | Canvas Tour | Visual UI layout diagram + keyboard shortcuts | `Badge`, role-token region tints, lucide `Layout` / `Wrench` / `Terminal` |
+| 4 | `GetStartedStep` | Get Started | Example workflows, quick recipe, tips | `Card` / `CardContent`, `Badge`, role-token cards, lucide `Play` / `FlaskConical` / `BookOpen` / `Settings` |
-| Step | Component | Title | Purpose | Ant Design Components |
-|------|-----------|-------|---------|----------------------|
-| 0 | `WelcomeStep` | Welcome to MachinaOs | Platform intro + feature highlights | `Card`, `Row`, `Col`, `Typography` |
-| 1 | `ConceptsStep` | Key Concepts | Nodes, Edges, Agents, Skills, Normal/Dev Mode | `Space`, `Typography` |
-| 2 | `ApiKeyStep` | API Key Setup | Provider list + "Open Credentials" button | `Button`, `Alert`, `Space` |
-| 3 | `CanvasStep` | Canvas Tour | Visual UI layout diagram + shortcuts | `Tag`, `Space`, `Typography` |
-| 4 | `GetStartedStep` | Get Started | Example workflows, quick recipe, tips | `Card`, `Tag`, `Space` |
+**ApiKeyStep** accepts an `onOpenCredentials` prop to link to the existing CredentialsModal without duplicating key input logic. It lists six providers (OpenAI, Anthropic, Google, Groq, OpenRouter, Cerebras) with brand icons from `client/src/components/icons/AIProviderIcons.tsx` (imported as `../../icons/AIProviderIcons`), and closes with an `Alert variant="info"` noting keys can be changed later from the toolbar.
-**ApiKeyStep** accepts an `onOpenCredentials` prop to link to the existing CredentialsModal without duplicating key input logic.
+**CanvasStep** renders a miniature UI-region diagram whose tints map node groups to regions: toolbar→`workflow`, sidebar→`model`, canvas→`agent`, palette→`skill`, console→`trigger`, each via the `--node-X-soft` / `text-node-X` tokens. The shortcut list (`Ctrl+S` Save, `F2` Rename node, `Delete` Remove node, `Ctrl+C` Copy node) renders as `Badge variant="outline"` mono chips.
## Integration Points
@@ -181,18 +213,15 @@ const [onboardingReopenTrigger, setOnboardingReopenTrigger] = React.useState(0);
### SettingsPanel.tsx
-Added `onReplayOnboarding?: () => void` prop and a "Help" section with:
-- `QuestionCircleOutlined` icon header
-- "Replay Welcome Guide" button styled with `dracula.cyan`
-- Description text
+`SettingsPanel` takes an `onReplayOnboarding?: () => void` prop. The Help section renders a shadcn `Button variant="default"` "Replay Welcome Guide" (lucide `HelpCircle` icon, `disabled` when the callback is absent) that fires `onReplayOnboarding`.
## WebSocket Handlers
-No new handlers were needed. The onboarding system reuses existing generic handlers:
+No new handlers were needed. The onboarding system reuses existing generic handlers, accessed through the TanStack Query user-settings layer:
| Handler | Usage |
|---------|-------|
-| `get_user_settings` | Check `onboarding_completed` and `onboarding_step` on connect |
+| `get_user_settings` | Check `onboarding_completed` and `onboarding_step` on hydration |
| `save_user_settings` | Persist step progress on each navigation, skip, or complete |
## Lifecycle
@@ -200,10 +229,10 @@ No new handlers were needed. The onboarding system reuses existing generic handl
### First Launch (New User)
1. User opens app, WebSocket connects
-2. `useOnboarding` calls `get_user_settings` -- no settings exist yet
+2. `useOnboarding` reads `useUserSettingsQuery` -- no settings exist yet
3. `onboarding_completed` defaults to `false`, `onboarding_step` defaults to `0`
4. Wizard opens at step 0
-5. User navigates steps -- each transition saves via `save_user_settings`
+5. User navigates steps -- each transition saves via the save mutation (`save_user_settings`)
6. On "Start Building" or "Skip for now", `onboarding_completed` set to `true`
7. Wizard closes, does not reappear on refresh
@@ -227,9 +256,8 @@ No new handlers were needed. The onboarding system reuses existing generic handl
2. `onReplayOnboarding()` callback fires:
- Closes SettingsPanel
- Increments `onboardingReopenTrigger`
-3. `useOnboarding` detects trigger change:
- - Resets `hasCheckedRef` to allow re-check
- - Sets `isVisible = true, currentStep = 0`
+3. `useOnboarding` detects the trigger change (`> 0`):
+ - Sets `isVisible = true, currentStep = 0, isCompleted = false`
4. Wizard opens from step 0
## Edge Cases
@@ -237,38 +265,42 @@ No new handlers were needed. The onboarding system reuses existing generic handl
| Scenario | Behavior |
|----------|----------|
| Auth disabled (`VITE_AUTH_ENABLED=false`) | Works unchanged -- reads from `user_id="default"` |
-| WebSocket not connected yet | Hook waits for `isConnected`, `isLoading=true` prevents render |
+| Settings query not resolved yet | `isLoading=true` prevents render until `hasChecked` |
+| Settings query errors | Non-blocking: `isLoading=false, hasChecked=true`, app continues |
| Browser closed mid-wizard | `onboarding_step` saved on each transition, resumes from last step |
-| Multiple tabs | Completing in one tab doesn't update others until refresh |
-| Replay from Settings | Resets backend state and reopens wizard from step 0 |
-| Fresh database (no machina.db) | Onboarding appears after first WebSocket connect |
+| Multiple tabs | Completing in one tab doesn't update others until query refetch |
+| Replay from Settings | Resets local state and reopens wizard from step 0 |
+| Fresh database (no machina.db) | Onboarding appears after first settings query resolves |
## Verification Checklist
-1. **Fresh database**: Delete `server/machina.db`, start server -- wizard appears
-2. **Step navigation**: Click through all 5 steps -- progress bar updates, Back/Next work
+1. **Fresh database**: Delete `server/machina.db` (or the configured DB), start server -- wizard appears
+2. **Step navigation**: Click through all 5 steps -- the `<ol>` stepper updates, Back/Next work
3. **Skip**: Click "Skip for now" -- wizard closes, doesn't reappear on refresh
4. **Resume**: Advance to step 3, close browser, reopen -- wizard resumes at step 3
5. **Complete**: Finish all steps via "Start Building" -- wizard doesn't reappear
6. **API Key step**: Click "Open Credentials" button -- CredentialsModal opens
7. **Existing user migration**: With existing `machina.db` where `examples_loaded=1` -- onboarding does NOT appear
-8. **Theme support**: Toggle dark/light mode -- all onboarding UI adapts correctly
+8. **Theme support**: Switch themes -- role-token cards and region tints adapt correctly
9. **Replay**: Open Settings, click "Replay Welcome Guide" -- wizard reopens from step 0
-10. **TypeScript**: `npx tsc --noEmit` passes clean
+10. **TypeScript**: `tsgo --noEmit` (or `npx tsc --noEmit`) passes clean
## Key Files
| File | Description |
|------|-------------|
-| `client/src/hooks/useOnboarding.ts` | Onboarding state hook with WebSocket persistence |
-| `client/src/components/onboarding/OnboardingWizard.tsx` | Main wizard modal with Ant Design Steps |
+| `client/src/hooks/useOnboarding.ts` | Onboarding state hook; persists via TanStack-Query user-settings layer |
+| `client/src/hooks/useUserSettingsQuery.ts` | `useUserSettingsQuery` / `useSaveUserSettingsMutation` (WS-backed) |
+| `client/src/components/onboarding/OnboardingWizard.tsx` | Main wizard modal: `STEPS` SSOT + `<ol>` stepper + ActionButton footer |
+| `client/src/components/onboarding/nodeRoleClasses.ts` | `NODE_ROLE_CLASSES` role→token map for step cards |
| `client/src/components/onboarding/steps/WelcomeStep.tsx` | Step 0: Platform introduction |
-| `client/src/components/onboarding/steps/ConceptsStep.tsx` | Step 1: Key concepts (Nodes, Edges, Agents) |
+| `client/src/components/onboarding/steps/ConceptsStep.tsx` | Step 1: Key concepts (Nodes, Edges, Agents, Skills, Modes) |
| `client/src/components/onboarding/steps/ApiKeyStep.tsx` | Step 2: API key setup with Credentials link |
| `client/src/components/onboarding/steps/CanvasStep.tsx` | Step 3: UI layout diagram + shortcuts |
| `client/src/components/onboarding/steps/GetStartedStep.tsx` | Step 4: Getting started tips |
+| `client/src/components/icons/AIProviderIcons.tsx` | Provider brand icons used by ApiKeyStep |
| `client/src/Dashboard.tsx` | Integration: renders wizard + passes replay trigger |
-| `client/src/components/ui/SettingsPanel.tsx` | Replay button in Help section |
+| `client/src/components/ui/SettingsPanel.tsx` | "Replay Welcome Guide" button in Help section |
| `server/models/database.py` | `UserSettings.onboarding_completed`, `onboarding_step` fields |
| `server/core/database.py` | Migration + CRUD for onboarding fields |
@@ -276,8 +308,6 @@ No new handlers were needed. The onboarding system reuses existing generic handl
To add a new onboarding step:
-1. Create `client/src/components/onboarding/steps/NewStep.tsx` using Ant Design components
-2. Import in `OnboardingWizard.tsx` and add to the `renderStep()` switch
-3. Add entry to `stepItems` array
-4. Update `TOTAL_STEPS` in `useOnboarding.ts`
-5. No backend changes needed (step index is just a number)
+1. Create `client/src/components/onboarding/steps/NewStep.tsx` composing shadcn primitives + Tailwind tokens + lucide icons (use `NODE_ROLE_CLASSES` for tinted cards). Do NOT introduce antd.
+2. Add a `{ title, render }` entry to the `STEPS` array in `OnboardingWizard.tsx`. Its `.length` automatically updates the hook's `totalSteps` and the progress stepper — no separate count to maintain.
+3. No backend changes needed (step index is just a number).
diff --git a/docs-internal/performance.md b/docs-internal/performance.md
index 9759f08d..9c0a865f 100644
--- a/docs-internal/performance.md
+++ b/docs-internal/performance.md
@@ -76,15 +76,15 @@ T+12.27 — Status broadcasters settled ◀ "everything green"
is idle in this window; the cost lives on the frontend. Likely
contributors:
-1. **TanStack Query auth-bootstrap retry budget** ([client/src/contexts/AuthContext.tsx](client/src/contexts/AuthContext.tsx) + [client/src/lib/connectionConfig.ts](client/src/lib/connectionConfig.ts)). The `AUTH_RETRY` envelope (BASE 50 ms, CAP 4000 ms, MAX_ATTEMPTS 7) covers the typical 4 s backend cold-start window in 4-5 attempts. If the backend finishes mid-retry-cycle, the next jittered draw can land 1-3 s after readiness.
-2. **React Strict Mode dual-mount** in dev. The 100 ms guard in [WebSocketContext.tsx:2554](client/src/contexts/WebSocketContext.tsx#L2554) absorbs the bulk; remaining cost is React reconciliation + babel-plugin-react-compiler overhead on first render.
+1. **TanStack Query auth-bootstrap retry budget** ([client/src/contexts/AuthContext.tsx](../client/src/contexts/AuthContext.tsx) + [client/src/lib/connectionConfig.ts](../client/src/lib/connectionConfig.ts)). The `AUTH_RETRY` envelope (BASE 50 ms, CAP 4000 ms, MAX_ATTEMPTS 7) covers the typical 4 s backend cold-start window in 4-5 attempts. If the backend finishes mid-retry-cycle, the next jittered draw can land 1-3 s after readiness.
+2. **React Strict Mode dual-mount** in dev. The 100 ms guard in [WebSocketContext.tsx:2554](../client/src/contexts/WebSocketContext.tsx#L2554) absorbs the bulk; remaining cost is React reconciliation + babel-plugin-react-compiler overhead on first render.
3. **PartySocket upgrade handshake**. Sub-100 ms in normal cases; would only matter on slow networks.
To attribute definitively: add `console.time('auth.queryFn')` / `console.timeLog('auth.queryFn')` markers and a corresponding pair around `connect()`. Not blocking — the contract this layer was meant to fix (the +12 s disconnect-reconnect cycle) is resolved.
## Frontend retry / reconnect envelope
-Single source of truth: [client/src/lib/connectionConfig.ts](client/src/lib/connectionConfig.ts).
+Single source of truth: [client/src/lib/connectionConfig.ts](../client/src/lib/connectionConfig.ts).
| Constant | Value | Notes |
|---|---|---|
@@ -101,7 +101,7 @@ Backoff formula (AWS Architecture Blog "full jitter" pattern):
sleep = random(0, min(CAP_MS, BASE_MS * 2^attempt))
-Lock-in tests: [client/src/lib/__tests__/connectionConfig.test.ts](client/src/lib/__tests__/connectionConfig.test.ts).
+Lock-in tests: [client/src/lib/__tests__/connectionConfig.test.ts](../client/src/lib/__tests__/connectionConfig.test.ts).
## Bottleneck inventory (cold-start, warm cache)
diff --git a/docs-internal/plugin_system.md b/docs-internal/plugin_system.md
index 8e294755..ebe98156 100644
--- a/docs-internal/plugin_system.md
+++ b/docs-internal/plugin_system.md
@@ -19,65 +19,12 @@ pitfalls).
## Quick start — adding a new node
-```python
-# server/nodes/search/example_search.py
-from pydantic import BaseModel, Field
-from services.plugin import (
- ActionNode, ApiKeyCredential, NodeContext, Operation, TaskQueue,
-)
-
-
-class ExampleCredential(ApiKeyCredential):
- id = "example_api"
- display_name = "Example Search"
- category = "Search"
- key_name = "X-API-Key"
- key_location = "header"
-
-
-class ExampleParams(BaseModel):
- query: str = Field(..., min_length=1)
- max_results: int = Field(default=10, ge=1, le=50)
-
-
-class ExampleOutput(BaseModel):
- results: list
- count: int
-
-
-class ExampleSearchNode(ActionNode):
- type = "exampleSearch"
- display_name = "Example Search"
- # Icon + color are NOT declared on the class. Drop `icon.svg` into
- # the plugin folder; create `meta.json` with `{"color": "#abcdef"}`.
- # `_metadata_dict` resolves both at registration time via the
- # central handler in server/nodes/_visuals.py.
- group = ("search", "tool")
- description = "Example search via external API"
- component_kind = "square"
- handles = (
- {"name": "input-main", "kind": "input", "position": "left",
- "label": "Input", "role": "main"},
- {"name": "output-main", "kind": "output", "position": "right",
- "label": "Output", "role": "main"},
- )
- credentials = (ExampleCredential,)
- task_queue = TaskQueue.REST_API
- usable_as_tool = True
- Params = ExampleParams
- Output = ExampleOutput
-
- @Operation("search")
- async def search(self, ctx: NodeContext, params: ExampleParams) -> ExampleOutput:
- async with ctx.connection("example_api") as conn:
- resp = await conn.get(
- "https://api.example.com/search",
- params={"q": params.query, "limit": params.max_results},
- )
- resp.raise_for_status()
- data = resp.json()
- return ExampleOutput(results=data.get("hits", []), count=len(data.get("hits", [])))
-```
+The full single-file recipe code block lives in the cookbook — see
+[`server/nodes/README.md` → Five-minute recipe](../server/nodes/README.md#five-minute-recipe).
+A self-contained folder (`AcmeSearchNode`) declares `type` /
+`display_name` / `group` / `component_kind` / `handles` /
+`credentials` / `task_queue` / `usable_as_tool` / `Params` / `Output`
+plus one `@Operation` method, and nothing else.
**That's it.** On server restart:
diff --git a/docs-internal/theme_system.md b/docs-internal/theme_system.md
index b5d10bff..9afd282d 100644
--- a/docs-internal/theme_system.md
+++ b/docs-internal/theme_system.md
@@ -353,6 +353,38 @@ Selection state: `selected` co-class on the outer wrapper, driven by React Flow'
Type-specific co-classes (Cyber-only today): `.node-agent` colors AIAgentNode neon magenta; `.sq-node.node-trigger .sq-node-box` colors TriggerNode neon green. Other themes ignore these co-classes — single colour by node type is fine when not differentiating.
+## Canvas-wide edge + node status animations (`canvasAnimations.ts`)
+
+[client/src/styles/canvasAnimations.ts](../client/src/styles/canvasAnimations.ts) owns the `@keyframes` + `.react-flow__edge.{status}` + `.react-flow__node.{status}` rules injected once into Dashboard's `<style>` tag. It is the single home for canvas-wide rules that need to match React Flow's wrapper classes (per-node inline animations live in their own components and read theme tokens directly — see the visual contract above).
+
+**Three named exports** — adding a new keyframe or status visual is a single-file change:
+
+| Symbol | Role |
+|---|---|
+| `KEYFRAMES` | `@keyframes` definitions for edges (`dashFlow` — the marching-ants stroke-dashoffset cycle on executing edges) |
+| `edgeStatusStyles(colors)` | `.react-flow__edge.{selected,executing,completed,error,pending,memory-active,tool-active}` stroke rules |
+| `nodeStatusStyles(_colors)` | `.react-flow__node.{...}` status-class colours (currently a no-op stub; the arg is kept on the signature so the `buildCanvasStyles` / `CanvasStatusColors` contract stays stable for downstream consumers) |
+| `buildCanvasStyles(colors)` | Composes the three into the final string for Dashboard |
+
+**Light/dark distinction lives entirely in `theme.ts`.** `buildCanvasStyles(colors)` is single-arg and the file ships **zero hardcoded hex colours** — the theme object provides different values per mode, so this module knows nothing about which theme is active.
+
+`CanvasStatusColors` is the contract interface, carrying exactly eight edge-stroke keys:
+
+```ts
+export interface CanvasStatusColors {
+ edgeDefault: string;
+ edgeSelected: string;
+ edgeExecuting: string;
+ edgeCompleted: string;
+ edgeError: string;
+ edgePending: string;
+ edgeMemoryActive: string;
+ edgeToolActive: string;
+}
+```
+
+**No `nodeGlow` keyframe here.** This module used to inject a `nodeGlow` keyframe targeting the React Flow wrapper, but it was dead code (only the inner `.node` child animated) and was removed. Node execution glow is owned solely by [base.css](../client/src/themes/base.css) — the `node-pulse` keyframe + `.react-flow__node.executing .node` / `.sq-node[data-executing] .sq-node-box` rules (see [`--node-pulse-color` contract](#done) above). Do not re-introduce a competing `nodeGlow` keyframe here.
+
## Sound system
Each theme declares `--sound-pack: "<pack>"` in its `:root[data-theme="..."]` block. Pack names map to event tables in [client/src/lib/sound.ts](../client/src/lib/sound.ts):
diff --git a/docs-internal/tool_building_pipeline.md b/docs-internal/tool_building_pipeline.md
index 18a3d643..7da6a056 100644
--- a/docs-internal/tool_building_pipeline.md
+++ b/docs-internal/tool_building_pipeline.md
@@ -11,7 +11,7 @@ Canonical home for how AI Agents discover connected tool nodes, build LangChain
## 1. Five stages
```
-1. DISCOVER handlers/ai.py:_collect_agent_connections
+1. DISCOVER plugin/edge_walker.py:collect_agent_connections
→ reads edges → tool_data list
│
▼
@@ -32,9 +32,9 @@ Canonical home for how AI Agents discover connected tool nodes, build LangChain
→ _dispatch_tool routes by node_type to a handler
```
-## 2. Stage 1 — Discovery (handlers/ai.py)
+## 2. Stage 1 — Discovery (plugin/edge_walker.py)
-[`_collect_agent_connections(node_id, context, database)`](../server/services/handlers/ai.py) walks the workflow edges and returns a 4-tuple per agent:
+[`collect_agent_connections(node_id, context, database)`](../server/services/plugin/edge_walker.py) walks the workflow edges and returns a 4-tuple per agent:
```
(memory_data, skill_data, tool_data, input_data, task_data)
diff --git a/server/nodes/README.md b/server/nodes/README.md
index a393ffa3..3455ca8d 100644
--- a/server/nodes/README.md
+++ b/server/nodes/README.md
@@ -265,6 +265,13 @@ package's `__init__.py` — the consumer never imports your folder.
All six are idempotent (same callable / class for the same key is a
no-op; conflicts raise `ValueError`).
+**Register, don't edit the consumer.** Never edit `event_waiter.py`,
+`status_broadcaster.py`, or `routers/websocket.py` to add a plugin's
+handler / filter / refresh — call the matching `register_*` from your
+plugin's `__init__.py` instead. Likewise never import a plugin folder
+from `routers/`, `services/`, or another `nodes/` subfolder; the
+consumer reaches your code through the registry, never by import path.
+
### Credential validation (Wave 11.I)
The credential-validator dispatch is a sibling concern, handled by the