Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs: restructure CLAUDE.md to lean index, dedupe canonical docs, audit node-flow cards
trohitg0b5ad7fc108f
diff --git a/CLAUDE.md b/CLAUDE.mdindex cff0e490..7e452204 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -46,7 +46,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire| **[Credentials Encryption](./docs-internal/credentials_encryption.md)** | Fernet + PBKDF2 encryption pipeline, separate credentials.db, two credential systems (OAuth vs API keys), multi-backend abstraction || **[Status Broadcaster](./docs-internal/status_broadcaster.md)** | WebSocket-first communication: StatusBroadcaster singleton, live count via `len(MESSAGE_HANDLERS) + len(get_ws_handlers())`, broadcast message types, Android two-state model || **[RLM Service](./docs-internal/rlm_service.md)** | Recursive Language Model agent with REPL-based execution (llm_query, rlm_query, FINAL) |-| **[Claude Code Agent](./docs-internal/claude_code_agent_architecture.md)** | Claude Code integration internals (LangGraph reference; companion to `cli_agent_framework.md`) |+| **[Claude Code Agent](./docs-internal/claude_code_agent.md)** | Hub for the Claude Code agent — routes to architecture, interactive mode, CLI framework, canonical-patterns RFC, and the 5 CLI snapshot references. || **[CLI Agent Framework](./docs-internal/cli_agent_framework.md)** | Multi-provider CLI runtime (Claude Code / Codex / Gemini): `AICliService.run_batch`, per-task worktree isolation, FastMCP bridge, **memory bridge** (`--continue` for first run + intra-process stream-json multi-turn + `--resume <UUID>` for crash recovery, all on a stable `cwd=repo_root`; `node_parameters_updated` broadcast on every successful turn). **Plugin-folder layout** (post-cutover): all claude-specific code lives in [`server/nodes/agent/claude_code_agent/`](./server/nodes/agent/claude_code_agent/) — `_provider.py`, `_pool.py`, `_skills.py`, `_oauth.py`, `_handlers.py` — and self-registers via three `factory.py` registries (`register_provider` / `register_session_pool` / `register_skill_materialiser`) plus `register_ws_handlers`. The generic framework at `services/cli_agent/` imports nothing from `nodes/`. || **[Claude Code Interactive Mode](./docs-internal/claude_code_interactive_mode.md)** | The interactive-mode cutover: MachinaOs no longer uses `claude -p` headless. `ClaudeSessionPool` (keyed by `simpleMemory.node_id`) spawns `claude` as a plain subprocess with stdio pipes — **no PTY** — and drives it over the VSCode-extension protocol: `--output-format stream-json --input-format stream-json --verbose --ide`. User prompts written as JSON to `proc.stdin` (`{"type":"user","message":{"role":"user","content":"..."}}\n`); `system/init` / `assistant` / `result` / `system/compact_boundary` events stream back on `proc.stdout` (parsed by a background `stdout_reader_task` — the on-disk JSONL is persistence-only, not the runtime contract, because `result` events are stdout-only in stream-json mode). Cross-platform via plain pipes (the earlier PTY pattern was broken on Windows because pywinpty/ConPTY's emulated stdin never reached claude's Ink TUI keystroke handler). Stays in interactive billing — entrypoint `claude-vscode`, NOT `sdk-cli`. Multi-turn within one warm subprocess preserves the session UUID (verified end-to-end); cross-batch continuity via `--continue` (first run) or `--resume <UUID>` (crash recovery). Four typed CloudEvents (`claude.session.{spawned,cleared,terminated,usage}`) fire from the pool. `/compact` events forward to `CompactionService`. The non-pooled `AICliSession` path (one-shot prompt-in-argv runs) still uses PTY (out of scope for this refactor; works on POSIX). || **[CLI Agent Canonical Patterns RFC](./docs-internal/cli_agent_canonical_patterns_rfc.md)** | Audit of MachinaOs's `services/cli_agent` against the official Claude Code spec — six invariants (skills-as-files, MCP-only transport, `list_changed`, deferral, visible-tool filtering, native-session-continuity-via-stable-cwd) with current implementation status. |@@ -57,6 +57,12 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire| **[Claude Code Skills (snapshot)](./docs-internal/claude_code_skills_reference.md)** | Verbatim [code.claude.com/docs/en/skills](https://code.claude.com/docs/en/skills) — `SKILL.md` frontmatter spec, discovery paths, content lifecycle, `context: fork`, dynamic context injection via `` !`<command>` ``. The spec MachinaOs materialises connected skills against in `_pre_spawn`. || **[Autonomous Agent Creation](./docs-internal/autonomous_agent_creation.md)** | Creating autonomous agents with Code Mode patterns and agentic loops || **[Polyglot Server](../polyglot-server/ARCHITECTURE.md)** | Plugin registry microservice with MCP gateway (optional integration) |+| **[Authentication](./docs-internal/authentication.md)** | JWT/cookie auth: toggle, single/multi mode, backend + frontend, middleware, startup retry. |+| **[Credentials Panel](./docs-internal/credentials_panel.md)** | CredentialsModal.tsx logic-flow reference (state taxonomy, per-provider panels, invariants). |+| **[Memory Lifecycle](./docs-internal/memory_lifecycle.md)** | Canonical conversation-memory lifecycle: load/append/trim/archive/clear/resume across all agents. |+| **[Node Parameter Panel](./docs-internal/node_panels.md)** | Three-section node config UI (Input / Parameters / Output) logic-flow reference. |+| **[Deployment (legacy reference)](./docs-internal/deployment_legacy.md)** | machina deploy CLI summary + the historical Docker Compose topology. |+| **[GCP VM Deploy Runbook](./docs-internal/gcp_vm_deploy_runbook.md)** | Manual gcloud + Cloudflare runbook for deploying the released npm package (non-Terraform path). |## Design Principles & Standards@@ -196,91 +202,7 @@ server/nodejs/ # Persistent Node.js server for JS/TS execution```### Polyglot Server Integration (Optional)-MachinaOs can optionally integrate with **polyglot-server** - a centralized plugin registry microservice that exposes integrations through REST API, MCP (Model Context Protocol), and WebSocket.--**Architecture:**-```-┌─────────────────────────────────────────────────────────────────┐-│ MachinaOs │-│ ┌────────────────┐ ┌────────────────┐ │-│ │ React Flow │───▶│ FastAPI Backend│ │-│ │ Frontend │ │ (port 3010) │ │-│ └────────────────┘ └───────┬────────┘ │-│ │ │-│ ┌─────────────────┴─────────────────┐ │-│ │ NodeExecutor │ │-│ │ (registry-based dispatch) │ │-│ └─────────────────┬─────────────────┘ │-└────────────────────────────────│────────────────────────────────┘- │ HTTP (aiohttp)- ▼-┌─────────────────────────────────────────────────────────────────┐-│ Polyglot Server (port 8080) │-│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │-│ │ REST API │ │ MCP │ │ WebSocket │ │-│ │ Gateway │ │ Server │ │ Handler │ │-│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │-│ └────────────────┼────────────────┘ │-│ ▼ │-│ ┌───────────────────────┐ │-│ │ Plugin Registry │ │-│ │ Discord, Telegram, │ │-│ │ Notion, GitHub, etc. │ │-│ └───────────────────────┘ │-└─────────────────────────────────────────────────────────────────┘-```--**Files:**-```-server/services/-├── polyglot_client.py # HTTP client for polyglot-server-└── handlers/- └── polyglot.py # Standalone handler (not wired to NodeExecutor)-```--**PolyglotClient** (`server/services/polyglot_client.py`):-```python-class PolyglotClient:- """HTTP client for polyglot-server plugin registry."""-- async def list_plugins(self) -> List[Dict[str, Any]]:- """List all available plugins."""-- async def get_schema(self, plugin_name: str) -> Optional[Dict[str, Any]]:- """Get plugin input/output schema for workflow node integration."""-- async def execute(self, plugin_name: str, action: str, params: Dict) -> Dict:- """Execute a plugin action."""-```--**Polyglot Handler** (`server/services/handlers/polyglot.py`):-```python-async def handle_polyglot_node(- node_id: str,- node_type: str,- parameters: Dict[str, Any],- context: Dict[str, Any],- polyglot_client, # Injected via functools.partial-) -> Dict[str, Any]:- """Execute a workflow node via polyglot-server plugin registry."""- plugin_name = node_type.replace("Node", "").lower()- result = await polyglot_client.execute(plugin_name, action, params)- return {"success": True, "result": result.get("result", {}), ...}--# Node types that can be routed to polyglot-server-POLYGLOT_NODE_TYPES = frozenset([- "discordNode", "telegramNode", "slackNode", "notionNode",- "todoistNode", "gmailNode", "twitterNode", "githubNode", ...-])-```--**Configuration** (when enabled):-```bash-# In server/.env-POLYGLOT_SERVER_URL=http://localhost:8080 # polyglot-server address-```--**Current Status**: Standalone integration files created. Not wired into NodeExecutor to avoid disturbing existing workflow execution flow. Future integration will add polyglot node types to handler registry via `functools.partial` pattern.+MachinaOs can optionally integrate with the sibling **polyglot-server** repo (a plugin-registry microservice exposing REST + MCP + WebSocket). NOTE: the MachinaOs-side client/handler (`polyglot_client.py`, `handlers/polyglot.py`) are not currently present in the tree — this is a possible future integration, not wired. See [Polyglot Server](../polyglot-server/ARCHITECTURE.md).### Node.js Code ExecutorPersistent Node.js server for JavaScript/TypeScript code execution, replacing subprocess spawning per execution.@@ -672,28 +594,7 @@ AI model nodes route through `SquareNode` via `Dashboard.tsx`'s `COMPONENT_BY_KI### Theme System-Token VALUES are **hex + `color-mix()`** (not HSL triplets) and live in the per-theme files: [light.css](./client/src/themes/light.css) / [dark.css](./client/src/themes/dark.css) own the shadcn + dracula + node + action colour hex; [base.css](./client/src/themes/base.css) owns the shared scale / chrome / shadow / type tokens **plus the `--tint-*` alpha scale — the single home for `color-mix` percentages** (consumers reference `var(--tint-X)`; skins keep only their bespoke decorative alphas inline). [index.css](./client/src/index.css) is **plumbing-only**: its `@theme inline` bridge maps `--color-X: var(--X)` (NO `hsl()` wrapper). Tailwind v4 still composes alpha (`bg-primary/50`, `text-foreground/80`) by wrapping any colour in `color-mix()`, so utilities work against hex. Action intents also expose an **`--action-X-ink`** readable-text variant (light: a darkened accent; dark: the raw accent) — `ActionButton` reads `text-action-X-ink`. The **animation system** — per-theme `--pulse-keyframe` / `--pulse-duration` / `--pulse-timing` tokens, the executing + trigger keyframes, and the `.machina-pulse` / `.machina-trigger-armed` / `.machina-bolt` / `.machina-crt` / `.machina-scanline` helpers — lives in [animations.css](./client/src/themes/animations.css); base.css's `.executing` rule reads `var(--pulse-keyframe)` so a theme switches its pulse by overriding one token. Trigger nodes show a distinct continuous `.machina-trigger-armed` "listening" heartbeat (waiting state) vs the one-shot execution pulse, with the ⚡ badge pulsing via `.machina-bolt`.--**Token tiers** (pick the most specific that fits):--1. **shadcn semantic tokens** — `background`, `foreground`, `card`, `popover`, `primary`, `secondary`, `muted`, `accent`, `destructive`, `success`, `warning`, `info`, `border`, `input`, `ring`. Each rotates per theme.-2. **Node-type role tokens** — `--node-agent`, `--node-model`, `--node-skill`, `--node-tool`, `--node-trigger`, `--node-workflow`. Each exposes three variants:- - `bg-node-X` / `text-node-X` — solid (icons, badges, accents)- - `bg-node-X-soft` — tinted surface (cards, tinted backgrounds)- - `border-node-X-border` — tinted outline- Themes redefine these in their own scope without touching call sites. **Never use opacity arithmetic at the call site** (`bg-node-agent/10`); add a new `-soft` / `-border` variant if you need a different opacity.-3. **Action role tokens** — `--action-run`, `--action-stop`, `--action-save`, `--action-config`, `--action-secret`, `--action-tools`. `base / -soft / -hover / -border / -ink` set (e.g. `bg-action-run-soft text-action-run-ink border-action-run-border`, plus `hover:bg-action-run-hover`). `-ink` is the readable label colour (light: darkened accent; dark: raw accent). Used for toolbar icon buttons, File menu items, and as the tokens behind `<ActionButton>`'s `intent` variants. Themes redefine these without touching call sites; ActionButton composes hover via `-hover` (no opacity arithmetic).-4. **Dracula raw accents** — `--dracula-green/purple/pink/cyan/red/orange/yellow`. Same value across light + dark themes. Used as the underlying palette that `--action-X` and `--node-X` reference; do not consume directly in components — go through the semantic role token instead.--**Action buttons** — use [ActionButton](./client/src/components/ui/action-button.tsx) (CVA primitive with semantic `intent` variants: `run | stop | save | config | secret | tools`). Each intent reads the matching `--action-X` set (`-soft` for resting bg, `-hover` for hover bg, `-border` for outline, `-ink` for the label text). Disabled state is the shadcn-idiomatic `disabled:opacity-50` on the base class (one rule, all intents) — never per-token opacity arithmetic. Replaces the old `actionButtonStyle()` helper. Credential-modal panels (`OAuthConnect`, `EmailPanel`, `QrPairingPanel`, `ActionBar`) consume `<ActionButton>` directly; their `ActionDef` carries an `intent` key, not a free-form colour.--**Canvas-wide animations** — [client/src/styles/canvasAnimations.ts](./client/src/styles/canvasAnimations.ts) owns the `@keyframes` + `.react-flow__edge.{status}` + `.react-flow__node.{status}` rules injected once into Dashboard's `<style>` tag. Three named groups (`KEYFRAMES`, `edgeStatusStyles`, `nodeStatusStyles`) -- adding a new keyframe or status visual is a single-file change. Per-node inline animations (border pulse on `isExecuting`, etc.) live in their components and read theme tokens directly. Light/dark distinction lives entirely in `theme.ts` — `buildCanvasStyles(colors)` is single-arg and the file ships zero hardcoded hex colours; `CanvasStatusColors` carries `edgeDefault | edgeSelected | edgeExecuting | edgeCompleted | edgeError | edgePending | edgeMemoryActive | edgeToolActive`. The `nodeGlow` keyframe consumes scoped `--node-glow` / `--node-glow-soft` vars so one keyframe serves both themes.--**shadcn primitives** — full set under `client/src/components/ui/` (Button, Badge, Alert, Dialog, DropdownMenu, Select, Popover, Tooltip, Tabs, Card, Input, Textarea, Switch, Checkbox, Slider, Label, Form, Sonner, Skeleton, Collapsible, Accordion, AlertDialog). New code composes these with Tailwind classes.--**`useAppTheme()`** ([src/styles/theme.ts](./client/src/styles/theme.ts), [src/hooks/useAppTheme.ts](./client/src/hooks/useAppTheme.ts)) is grandfathered for the canvas node components (`AIAgentNode`, `SquareNode`, `TriggerNode`, `StartNode`, `ToolkitNode`, `TeamMonitorNode`, `GenericNode`) and `EdgeConditionEditor` — they interpolate per-definition `nodeColor` into gradients, borders, and React Flow `<Handle>` styles. Credential-modal panels (`OAuthConnect`, `EmailPanel`, `QrPairingPanel`) and the skill/tool editors (`SkillEditorModal`, `ToolSchemaEditor`) no longer call it — they compose `<ActionButton intent="...">` plus shadcn semantic tokens (`bg-warning/10`, `bg-accent/10`, `<Alert variant="destructive">`). Every other surface uses Tailwind + the tokens above.--**Theme switching** — `[data-theme="dark"]` + Tailwind's `.dark` class, both set by `App.tsx`'s `useEffect` on `isDarkMode`. See [docs-internal/frontend_architecture.md](./docs-internal/frontend_architecture.md#tokens--theming) for the full token table and migration patterns.+10-way visual theme system (5 utopian: light, dark, renaissance, greek, edo, steampunk, atomic + 5 dystopian: cyber, wasteland, rot, plague, surveillance) driven by `<html data-theme>` (set by [ThemeContext.tsx](./client/src/contexts/ThemeContext.tsx), which also toggles `.dark` for DARK_FAMILY themes) + per-theme CSS in `client/src/themes/`. Token VALUES are hex + `color-mix()` (never HSL): per-theme files own shadcn/dracula/node/action hex, `base.css` owns the shared `--tint-*` scale, `index.css` is plumbing (`@theme inline` maps `--color-X: var(--X)`). Six token tiers, the per-theme `--pulse-keyframe` animation system, decorative-layer wrappers, the 10-pack WebAudio sound system, and the canvas-wide edge/node status rules (`canvasAnimations.ts`) are all documented in **[Theme System](./docs-internal/theme_system.md)** — read it before adding a theme or a canvas-node component. The strict frontend theme RULES remain normative under "Frontend Design + Theme System (strict)" above.### WebSocket-First ArchitectureThe project uses WebSocket as the primary communication method between frontend and backend, replacing most REST API calls:@@ -707,884 +608,11 @@ The project uses WebSocket as the primary communication method between frontend> **Authoritative source: backend plugin registry.** Glob [`server/nodes/**/*.py`](./server/nodes/) (excluding `_*.py` helpers and `__init__.py`) for the live count. The per-node descriptions below are reference material — they drift on every plugin add and should be cross-checked against [`server/nodes/README.md`](./server/nodes/README.md), the per-domain docs in `docs-internal/`, and the actual plugin classes before relying on any specific detail.-### AI Chat Models (9 nodes)-- **openaiChatModel**: OpenAI GPT models with response format options. O-series models (o1, o3, o4) support reasoning effort parameter.-- **anthropicChatModel**: Claude models (Fable 5, Opus 4.8/4.7, Sonnet 4.6, Haiku 4.5) with extended thinking support-- **geminiChatModel**: Google Gemini models with multimodal capabilities, safety settings, and thinking support for 2.5/Flash Thinking models-- **openrouterChatModel**: OpenRouter unified API - access 200+ models from OpenAI, Anthropic, Google, Meta, Mistral, and more through a single API. Features free/paid model grouping in dropdown.-- **groqChatModel**: Groq ultra-fast inference with Llama, Qwen3, and GPT-OSS models. Qwen3-32b supports reasoning_format.-- **cerebrasChatModel**: Cerebras ultra-fast inference on custom AI hardware with Llama and Qwen models-- **deepseekChatModel**: DeepSeek V4 models (deepseek-v4-flash, deepseek-v4-pro); `deepseek-chat` / `deepseek-reasoner` remain as legacy aliases (deprecate 2026-07-24). Up to 1M context (V4), up to 64K output.-- **kimiChatModel**: Kimi models by Moonshot AI (kimi-k2.6, kimi-k2.5, kimi-k2.7-code). 256K context, 96K output. Thinking on by default (explicitly disabled for tool-calling agent compatibility). Fixed temperature 0.6.-- **mistralChatModel**: Mistral AI models (mistral-large-latest, mistral-medium-latest, mistral-small-latest, codestral-latest). Up to 256K context. No thinking support. Temperature 0-1.5.--### AI Agents & Memory (3 nodes)-- **aiAgent**: Advanced AI agent with tool calling, memory input handle, and iterative reasoning. Uses the plain-async `_run_agent_loop` for structured execution. Parameters: Provider, Model, Prompt, System Message, Options.-- **chatAgent**: Conversational AI agent with memory and skill support for multi-turn chat interactions. Parameters: Provider, Model, Prompt (supports `{{chatTrigger.message}}` template or auto-fallback from connected input), System Message. Behavior extended by connected skills.-### AI Agent Tool Nodes (6 dedicated + 10 dual-purpose)-Tool nodes connect to AI Agent's `input-tools` handle to provide capabilities the agent can invoke during reasoning. Both `masterSkill` and `simpleMemory` are in the AI Tools category.--#### Dedicated Tool Nodes (passive, tool-only)-- **masterSkill**: Master Skill (icon: target) - Aggregates multiple skills with enable/disable toggles. Split-panel UI: left panel shows skill list with checkboxes, right panel shows selected skill's markdown editor. Supports both built-in skills (from `server/skills/` folders) and user-created skills (stored in database). User skills can be created/edited/deleted inline via the "+" button.-- **simpleMemory**: Markdown-based conversation memory with editable UI, window-based trimming, and optional vector DB for long-term semantic retrieval--#### Skill Node Architecture-Skills are organized in subfolders under `server/skills/`. Each top-level folder appears as an option in the Master Skill node's folder dropdown. See **[Skill Creation Guide](./server/skills/GUIDE.md)** for full documentation.--```-server/skills/-├── GUIDE.md # Skill creation guide-├── assistant/ # General-purpose assistant skills-│ ├── agent-builder-skill/SKILL.md # 5 canvas-mutation ops for live tool growth-│ ├── assistant-personality/SKILL.md-│ ├── compaction-skill/SKILL.md-│ ├── humanify-skill/SKILL.md-│ ├── memory-skill/SKILL.md-│ ├── subagent-skill/SKILL.md-│ └── write-todos-skill/SKILL.md # Task planning with plan-work-update loop-├── android_agent/ # Android device control skills-│ ├── personality/SKILL.md-│ ├── battery-skill/SKILL.md-│ ├── wifi-skill/SKILL.md-│ └── ... (12 skills total)-├── autonomous/ # Autonomous agent patterns-│ ├── code-mode-skill/SKILL.md-│ ├── agentic-loop-skill/SKILL.md-│ ├── progressive-discovery-skill/SKILL.md-│ ├── error-recovery-skill/SKILL.md-│ └── multi-tool-orchestration-skill/SKILL.md-├── coding_agent/ # Code execution & filesystem skills-│ ├── python-skill/SKILL.md-│ ├── javascript-skill/SKILL.md-│ ├── file-read-skill/SKILL.md-│ ├── file-modify-skill/SKILL.md-│ └── fs-search-skill/SKILL.md-├── productivity_agent/ # Google Workspace skills-│ ├── gmail-skill/SKILL.md # Send, search, read emails-│ ├── calendar-skill/SKILL.md # Create, list, update, delete events-│ ├── drive-skill/SKILL.md # Upload, download, list, share files-│ ├── sheets-skill/SKILL.md # Read, write, append spreadsheet data-│ ├── tasks-skill/SKILL.md # Create, list, complete tasks-│ └── contacts-skill/SKILL.md # Create, list, search contacts-├── social_agent/ # Social media platform skills-│ ├── twitter-send-skill/SKILL.md-│ ├── twitter-search-skill/SKILL.md-│ ├── twitter-user-skill/SKILL.md-│ ├── whatsapp-send-skill/SKILL.md-│ └── whatsapp-db-skill/SKILL.md-├── task_agent/ # Task management skills-│ ├── timer-skill/SKILL.md-│ ├── cron-scheduler-skill/SKILL.md-│ └── task-manager-skill/SKILL.md-├── travel_agent/ # Location and maps skills-│ ├── geocoding-skill/SKILL.md-│ └── nearby-places-skill/SKILL.md-├── rlm_agent/ # Recursive Language Model agent skills-│ └── rlm-reasoning-skill/SKILL.md-├── terminal/ # Shell, process management, OS-specific terminals-│ ├── shell-skill/SKILL.md # Sandboxed shell (no system PATH)-│ ├── process-manager-skill/SKILL.md # Long-running processes (full PATH, streams output)-│ ├── bash-skill/SKILL.md # Linux/macOS terminal patterns-│ ├── powershell-skill/SKILL.md # Windows terminal patterns-│ └── wsl-skill/SKILL.md # WSL bridge for Linux tools on Windows-└── web_agent/ # Web automation skills- ├── http-request-skill/SKILL.md- ├── proxy-config-skill/SKILL.md- ├── apify-skill/SKILL.md- ├── crawlee-scraper-skill/SKILL.md- ├── browser-skill/SKILL.md- ├── duckduckgo-search-skill/SKILL.md- ├── brave-search-skill/SKILL.md- ├── serper-search-skill/SKILL.md- └── perplexity-search-skill/SKILL.md-```--**SKILL.md Format:**-```yaml-----name: skill-name-description: Brief description for LLM visibility-allowed-tools: tool1 tool2-metadata:- author: machina- version: "1.0"- category: general- icon: "🔧"- color: "#6366F1"------# Skill Instructions (Markdown)-Detailed instructions loaded when skill is activated.-```--**Skill Content Lifecycle:**-1. First load reads from SKILL.md on disk, seeds to database-2. Database is source of truth after first activation-3. Users edit instructions in UI, edits saved to DB only-4. "Reset to Default" reloads from original SKILL.md file--#### Skill Content Editor-Skill nodes display an Ant Design `Input.TextArea` (markdown content) to view and edit instructions:-- Instructions loaded automatically when skill node is selected-- Save button writes changes back to database-- Uses `get_skill_content` / `save_skill_content` WebSocket handlers-- **No Input/Output panels**: Skill nodes only show the middle section (parameters + editor) in the parameter panel--**Key Files:**-| File | Description |-|------|-------------|-| `client/src/hooks/useParameterPanel.ts` | Loads/saves skill content for skill nodes |-| `server/services/skill_loader.py` | SkillLoader for filesystem and database skills (DB-wired via `get_skill_loader()`) |-| `server/services/skills/handlers.py` | `get_skill_content`, `save_skill_content`, `list_skill_folders`, `scan_skill_folder` handlers |--#### Master Skill Editor-The Master Skill node uses a custom split-panel editor (`MasterSkillEditor.tsx`) instead of standard parameters:--**UI Layout:**-```-+----------------------------------------------------------+-| [Folder Dropdown: assistant v] |-+----------------------------------------------------------+-| +--------------------+ +--------------------------------+ |-| | SKILLS LIST | | SKILL INSTRUCTIONS | |-| | [Badge: 3 enabled] | | | |-| | [Search skills...] | | # WhatsApp Skill | |-| | | | | |-| | [x] WhatsApp | | This skill provides WhatsApp | |-| | [x] Memory | | messaging capabilities... | |-| | [ ] Android | | | |-| | [x] Maps | | [Reset to Default] | |-| | [ ] HTTP | | | |-| +--------------------+ +--------------------------------+ |-+----------------------------------------------------------+-```--**Folder Dropdown:**-- Ant Design `Select` listing available skill folders from backend via `list_skill_folders` WebSocket handler-- Shows loading/disabled state while folders are being fetched-- Uses `getPopupContainer` to ensure dropdown renders correctly regardless of parent overflow-- Default folder: `assistant`--**Icon Resolution Priority:**-Skill icons resolve through the same chain as the teaching node (mirrors `BaseNode._metadata_dict`): per-plugin `<plugin>/icon.svg` (or `<plugin>/icon_<type>.svg`) served as `/api/schemas/nodes/<type>/icon` → [`server/nodes/visuals.json`](./server/nodes/visuals.json) entry (emoji or `lobehub:<brand>`). Color falls through per-plugin `<plugin>/meta.json` then `visuals.json`. Resolution lives in [`server/services/skill_loader.py::_parse_skill_metadata`](./server/services/skill_loader.py) which calls `nodes._visuals.get_plugin_icon_path` / `get_plugin_meta` / `get_icon` / `get_color`. The pre-Wave-10 `skillNodes.ts` + `getNodeDefaults()` helper that did per-skill icon overrides on the frontend was retired; icons + colours now live exclusively on disk (per-plugin folder) with `visuals.json` as the fallback registry, and SKILL.md frontmatter is read only for orphan skills (no node target).--**Keyboard Handling:**-The editor uses a native DOM `addEventListener('keydown')` on a wrapper div (via `useRef`) to `stopPropagation()` for Ctrl/Meta key events. This prevents React Flow's document-level `useKeyPress` hook from intercepting Ctrl+A (select all) and other Ctrl shortcuts inside the textarea. React synthetic `stopPropagation()` is insufficient because React Flow uses native `document.addEventListener`.--**Data Structure:**-```typescript-// skillsConfig parameter stored in node parameters-interface MasterSkillConfig {- [skillName: string]: {- enabled: boolean; // Whether skill is active- instructions: string; // Custom or default SKILL.md content- isCustomized: boolean; // True if user modified instructions- };-}-```--**skillsConfig Persistence:**-- `skillsConfig` persists skills from **all folders**, not just the currently selected one. Switching folders does not remove previously enabled skills from config.-- The backend (`handlers/ai.py`) handles stale config entries gracefully at execution time -- it checks `enabled`, tries to load instructions, and logs a warning if a skill file is missing. Stale disabled entries are simply skipped.-- No frontend cleanup of stale skills is performed to avoid race conditions with async skill loading.--**Custom Skill Creation:**-- The `create_user_skill` WebSocket handler requires `name`, `display_name`, and `instructions`. The `description` field is optional (defaults to empty string). Skill handlers live in `server/services/skills/handlers.py` (self-registered via `services.ws_handler_registry`).-- On create, `MasterSkillEditor` persists the updated `skills_config` to the node via `save_node_parameters` (mirroring the delete path) so the new skill's enabled state survives panel close / reload, then `invalidateUserSkills()` + the `skill_lifecycle` broadcast refresh the list.-- User skills are stored in the database and resolve through the database-wired `get_skill_loader().load_skill_async()` (so `get_skill_content` returns their instructions the same way it does for built-in skills).--**Backend Expansion:**-When AI Agent executes with a connected Master Skill node, `_collect_agent_connections()` in `handlers/ai.py` expands the skillsConfig into individual skill entries:-```python-if skill_type == 'masterSkill':- skills_config = skill_params.get('skillsConfig', {})- for skill_key, skill_cfg in skills_config.items():- if skill_cfg.get('enabled', False):- # Load from customized or skill folder- instructions = skill_cfg.get('instructions') if skill_cfg.get('isCustomized') else skill_loader.load_skill(skill_key).instructions- skill_data.append({- 'node_id': f"{source_node_id}_{skill_key}",- 'node_type': 'masterSkill',- 'skill_name': skill_key,- 'parameters': {'instructions': instructions},- 'label': skill_key- })-```--**Key Files:**-| File | Description |-|------|-------------|-| `client/src/components/parameterPanel/MasterSkillEditor.tsx` | Split-panel skill aggregator UI with inline user skill CRUD |-| `server/services/skills/handlers.py` | User skill CRUD: `get_user_skills`, `create_user_skill`, `update_user_skill`, `delete_user_skill` |-| `server/core/database.py` | UserSkill model and database CRUD methods |-| `server/services/handlers/ai.py` | Expands masterSkill into individual skills at execution |-| `server/skills/GUIDE.md` | Skill creation guide for built-in skills |--#### Other Dedicated Tool Nodes-- **calculatorTool**: Mathematical operations (add, subtract, multiply, divide, power, sqrt, mod, abs)-- **currentTimeTool**: Get current date/time with timezone support-- **duckduckgoSearch**: DuckDuckGo web search (free, uses `ddgs` library, no API key required)-- **taskManager**: Task management tool for AI agents to create, track, and manage tasks-- **writeTodos**: Structured task list planning for complex multi-step operations. Connects to any agent's `input-tools` handle. Dual output (`tool` + `main`), dracula purple `#bd93f9`. Backed by `TodoService` singleton (`server/services/todo_service.py`) with JSON-based per-session state keyed by workflow_id. Schema: `WriteTodosSchema` with `TodoItem`/`TodoStatus` Pydantic enum (`pending` | `in_progress` | `completed`). Handler broadcasts `phase: "todo_update"` via WebSocket for real-time UI; `formatTodoOutput()` in `OutputDisplayPanel.tsx` renders as a checklist. Skill: `server/skills/assistant/write-todos-skill/SKILL.md` teaches the plan-work-update loop. **Parameter panel = editable Current Todos manager** — `uiHints.isTodoEditor` routes `MiddleSection` to [`client/src/components/parameterPanel/TodoEditor.tsx`](./client/src/components/parameterPanel/TodoEditor.tsx) instead of the empty generic-params card. It reads/writes the LIVE TodoService list (keyed `workflow_id` → `node_id`, so all writeTodos nodes in one workflow share one list) via the `get_todos` / `set_todos` WS handlers, with add / remove / edit-text / click-to-cycle-status. Both the tool's `write` op and manual panel edits emit the typed `todos_updated` CloudEvent through the centralized `services.events.dispatch.emit`; the FE handler refreshes the `['todos', session_key]` query live. Self-contained plugin folder `server/nodes/tool/write_todos/` (`__init__.py` + `_handlers.py` + `_events.py`).--#### Dual-Purpose Search Nodes (workflow node + AI tool)-Search API nodes that work BOTH as standalone workflow nodes AND as AI Agent tools. When connected to `input-tools`, the LLM fills the node's parameter schema.-- **braveSearch**: **Dual-purpose node** - Search the web using Brave Search API. Returns web results with titles, snippets, and URLs. Group: `['search', 'tool']`. Parameters: query, maxResults, country, searchLang, safeSearch.-- **serperSearch**: **Dual-purpose node** - Search the web using Google via Serper API. Supports web, news, images, and places search types with knowledge graph. Group: `['search', 'tool']`. Parameters: query, searchType, maxResults, country, language.-- **perplexitySearch**: **Dual-purpose node** - AI-powered search using Perplexity Sonar. Returns a markdown-formatted AI answer with inline citation references and source URLs. Group: `['search', 'tool']`. Parameters: query, model (sonar/sonar-pro/sonar-reasoning/sonar-reasoning-pro), searchRecencyFilter, returnImages, returnRelatedQuestions.--**Key Files:**-| File | Description |-|------|-------------|-| `server/services/handlers/search.py` | 3 handler functions with API key fetch + usage tracking |-| `server/services/handlers/tools.py` | Tool dispatch wrappers for AI Agent tool calling |-| `server/services/ai.py` | Tool names, descriptions, and Pydantic schemas |-| `server/constants.py` | `SEARCH_NODE_TYPES` and `SEARCH_TOOL_TYPES` constants |-| `client/src/assets/icons/search/` | SVG icons for Brave, Serper, Perplexity |--**Search API Authentication:**-| Provider | Credential Key | Header | API Endpoint |-|----------|---------------|--------|-------------|-| Brave Search | `brave_search` | `X-Subscription-Token` | `GET https://api.search.brave.com/res/v1/web/search` |-| Serper | `serper` | `X-API-KEY` | `POST https://google.serper.dev/search` |-| Perplexity | `perplexity` | `Authorization: Bearer` | `POST https://api.perplexity.ai/chat/completions` |--**Credentials Modal Layout:**-- Brave Search and Perplexity in **Search** category-- Serper in **Scrapers** category (Google SERP scraping)--**Perplexity validation note:** `PerplexityCredential._probe` short-circuits to `ProbeResult(valid=True)` without an HTTP call — Perplexity exposes no cheap auth-gated endpoint (`/v1/models` returns 200 for any key including garbage; every other endpoint charges tokens, and `chat/completions` rejects `max_tokens<16` for `sonar` with HTTP 400 before checking auth). Runtime calls will surface 401 if the key is wrong. `ProbeResult` is exported from `services.plugin` for `_probe` overrides like this.--### Specialized AI Agents (15 nodes)-Specialized agents are AI Agents pre-configured for specific domains. They inherit full AI Agent functionality (provider, model, prompt, system message, thinking/reasoning) while being tailored for specific capabilities. All specialized agents dispatch through `BaseNode.execute()` via the node registry (Wave 11) and support the same input handles. Node colors use centralized dracula theme constants imported from `client/src/styles/theme.ts`.--**Input Handles:**-- `input-main` - Main data input (auto-prompting fallback)-- `input-skill` - Skill nodes (including Master Skill for aggregated skills)-- `input-memory` - Memory node for conversation history-- `input-tools` - Tool nodes for LLM tool calling-- `input-task` - Task completion events from taskTrigger nodes--**Specialized Agent Types:**-- **android_agent**: Android Control Agent - AI agent for Android device control. Connect Android service nodes (battery, wifi, bluetooth, apps, location, camera, sensors) as tools.-- **coding_agent**: Coding Agent - AI agent for code execution. Connect code executor nodes (Python, JavaScript) as tools.-- **web_agent**: Web Control Agent - AI agent for web automation. Connect web/browser nodes (scraper, HTTP, browser) as tools.-- **task_agent**: Task Management Agent - AI agent for task automation. Connect scheduling nodes (scheduler, reminders) as tools.-- **social_agent**: Social Media Agent - AI agent for social messaging. Connect messaging nodes (WhatsApp, Telegram) as tools.-- **travel_agent**: Travel Agent - AI agent for travel planning. Connect location, maps, and scheduling nodes as tools.-- **tool_agent**: Tool Agent - AI agent for tool orchestration. Connect any combination of tool nodes for flexible automation.-- **productivity_agent**: Productivity Agent - AI agent for productivity workflows. Connect scheduling, task, and utility nodes as tools.-- **payments_agent**: Payments Agent - AI agent for payment processing. Connect payment, invoice, and financial tool nodes.-- **consumer_agent**: Consumer Agent - AI agent for consumer interactions. Connect customer support, product, and order management tools.-- **autonomous_agent**: Autonomous Agent - AI agent for autonomous operations using Code Mode patterns. Uses agentic loops, progressive discovery, error recovery, and multi-tool orchestration for 81-98% token savings. Connect autonomous skills via Master Skill.-- **orchestrator_agent**: Orchestrator Agent - Team lead agent for coordinating multiple agents. Connect specialized agents via `input-teammates` handle; they become `delegate_to_*` tools the AI can invoke.-- **ai_employee**: AI Employee - Team lead agent similar to orchestrator_agent. Connect specialized agents via `input-teammates` handle for intelligent task delegation.-- **rlm_agent**: RLM Agent - Recursive Language Model agent using REPL-based code execution with recursive LM calls. Replaces the standard tool-calling loop with RLM's `exec()` REPL loop (`llm_query()`, `rlm_query()`, `FINAL()`). Routes to dedicated `handle_rlm_agent` handler and `RLMService` (not `handle_chat_agent`). Connect AI chat model nodes as small LMs for depth>=1 calls. See `docs-internal/rlm_service.md`.--**Backend Routing:**-Specialized agents are detected by `SPECIALIZED_AGENT_TYPES` and dispatched through `BaseNode.execute()` via the node registry (Wave 11). Dedicated engine: `rlm_agent` -> `RLMService`.-```python-SPECIALIZED_AGENT_TYPES = {- 'android_agent', 'coding_agent', 'web_agent', 'task_agent', 'social_agent',- 'travel_agent', 'tool_agent', 'productivity_agent', 'payments_agent', 'consumer_agent',- 'autonomous_agent', 'orchestrator_agent', 'ai_employee',-}-```--**Team Lead Types (Agent Teams Pattern):**-Team leads (`orchestrator_agent`, `ai_employee`) have a special `input-teammates` handle. Agents connected to this handle become delegation tools:-```python-# In handlers/ai.py-TEAM_LEAD_TYPES = {'orchestrator_agent', 'ai_employee'}--# Teammates become delegate_to_* tools automatically-if node_type in TEAM_LEAD_TYPES:- teammates = await _collect_teammate_connections(node_id, context, database)- if teammates:- for tm in teammates:- tool_data.append({- 'node_id': tm['node_id'],- 'node_type': tm['node_type'], # e.g., 'coding_agent'- 'label': tm['label'],- })- # AI now has delegate_to_coding_agent, delegate_to_web_agent, etc.-```--**Direct Android Service Tools:**-Android service nodes (batteryMonitor, wifiAutomation, etc.) can be connected directly to any agent's `input-tools` handle. The backend maps camelCase node types to snake_case service IDs:-```python-# In handlers/tools.py-service_id_map = {- 'batteryMonitor': 'battery',- 'wifiAutomation': 'wifi_automation',- 'bluetoothAutomation': 'bluetooth_automation',- # ... etc-}-```--#### Dual-Purpose Tool Nodes (workflow node + AI tool)-Nodes that work BOTH as standalone workflow nodes AND as AI Agent tools. When connected to `input-tools`, the LLM fills the node's parameter schema.-- **whatsappSend**: Send WhatsApp messages (text, media, location, contact) to contacts, groups, or newsletter channels. Full schema for all message types.-- **whatsappDb**: Query WhatsApp database - chat history, contacts, groups, and newsletter channels with filtering and pagination.-- **pythonExecutor**: Execute Python code for calculations, data processing, and automation. Tool name: `python_code`. Available: math, json, datetime, Counter, defaultdict, random.-- **montyExecutor**: Sandboxed Python via Pydantic Monty (`pydantic-monty`). Tool name: `sandboxed_python`. Deny-by-default with ENFORCED time/memory limits + dynamic `capabilities` (`http_get`/`workspace_read`/`workspace_write`). Supported subset only (no class/yield/with/match/random/collections).-- **javascriptExecutor**: Execute JavaScript code via persistent Node.js server for calculations, data processing, and JSON manipulation. Tool name: `javascript_code`.-- **typescriptExecutor**: Execute TypeScript code via persistent Node.js server with type safety. Tool name: `typescript_code`.-- **gmaps_locations**: Google Maps Geocoding service for address-to-coordinates conversion. Tool name: `geocode`.-- **gmaps_nearby_places**: Google Places API nearbySearch. Tool name: `nearby_places`.--See [plugin_system.md](./docs-internal/plugin_system.md) for the dual-purpose pattern (mark `group: ['category', 'tool']` and set `usable_as_tool = True` on the plugin class); whatsapp / twitter / google plugin folders are the live examples.--### Location Services (3 nodes)-- **gmaps_create**: Google Maps creation with customizable center, zoom, and map type (display only, not a tool)-- **gmaps_locations**: **Dual-purpose** - Google Maps Geocoding service for address-to-coordinates conversion-- **gmaps_nearby_places**: **Dual-purpose** - Google Places API nearbySearch with detailed place information--#### Google Maps API Key Resolution-The MapsSection component fetches the Google Maps API key from backend credentials:-```typescript-// In MapsSection.tsx-const { getStoredApiKey, isConnected } = useApiKeys();-const [apiKey, setApiKey] = useState<string | undefined>(() => getGoogleMapsApiKey()); // Env fallback-const hasFetchedRef = useRef(false);--useEffect(() => {- if (!isConnected || hasFetchedRef.current) return;- const fetchApiKey = async () => {- hasFetchedRef.current = true;- const storedKey = await getStoredApiKey('google_maps');- if (storedKey) setApiKey(storedKey);- };- fetchApiKey();-}, [isConnected, getStoredApiKey]);-```-- Falls back to environment variable if no stored key-- Uses `hasFetchedRef` to prevent multiple fetches-- Only fetches when WebSocket is connected--### Android Services (16 nodes)-Android device connection is configured via the Credentials Modal (Android panel), not via workflow nodes.--#### System Monitoring (4 nodes)-- **batteryMonitor**: Monitor battery status, level, charging state, temperature, and health-- **networkMonitor**: Monitor network connectivity, type, and internet availability-- **systemInfo**: Get device and OS information including Android version, API level, memory, and hardware details-- **location**: GPS location tracking with latitude, longitude, accuracy, and provider information--#### App Management (2 nodes)-- **appLauncher**: Launch applications by package name-- **appList**: Get list of installed applications with package names, versions, and metadata--#### Device Automation (6 nodes)-- **wifiAutomation**: WiFi control and scanning - enable, disable, get status, scan for networks-- **bluetoothAutomation**: Bluetooth control - enable, disable, get status, and paired devices-- **audioAutomation**: Volume and audio control - get/set volume, mute, unmute-- **deviceStateAutomation**: Device state control - airplane mode, screen on/off, power save mode, brightness-- **screenControlAutomation**: Screen control - brightness adjustment, wake screen, auto-brightness, screen timeout-- **airplaneModeControl**: Airplane mode status monitoring and control--#### Sensors (2 nodes)-- **motionDetection**: Accelerometer and gyroscope data - detect motion, shake gestures, device orientation-- **environmentalSensors**: Environmental sensors - temperature, humidity, pressure, light level--#### Media (2 nodes)-- **cameraControl**: Camera control - get camera info, take photos, camera capabilities-- **mediaControl**: Media playback control - volume control, playback control, play media files--### WhatsApp Nodes (3 nodes)-- **whatsappSend**: **Dual-purpose node** - Send WhatsApp messages (text, image, video, audio, document, sticker, location, contact) to contacts, groups, or newsletter channels. Works as workflow node OR AI Agent tool. Group: `['whatsapp', 'tool']`. Recipient types: Self (connected phone), Phone Number, Group, Channel (newsletter). Channel messages only support text, image, video, audio, document (NOT sticker, location, contact). **Format Markdown** toggle (default: true): converts GFM markdown to WhatsApp-native formatting via `markdown_formatter.to_whatsapp()`. Full parameter schema for message type, media URL, location coordinates, contact vCard, channel JID.-- **whatsappDb**: **Dual-purpose node** - Comprehensive WhatsApp database query node with 18 operations. Works as workflow node OR AI Agent tool. Group: `['whatsapp', 'tool']`. Operations:- - `chat_history`: Retrieve messages from individual or group chats with filtering, pagination, and optional media download- - `search_groups`: Search groups by name- - `get_group_info`: Get group details with participant names and phone numbers- - `get_contact_info`: Get full contact info (name, phone, profile picture) for sending/replying- - `list_contacts`: List all contacts with saved names- - `check_contacts`: Check WhatsApp registration status for phone numbers- - `list_channels`: List subscribed newsletter channels with optional server refresh- - `get_channel_info`: Get channel details (name, description, subscribers)- - `channel_messages`: Get channel message history with pagination, date range, media type filter, text search, and optional media download- - `channel_stats`: Get channel subscriber/view statistics- - `channel_follow`: Follow/subscribe to a newsletter channel- - `channel_unfollow`: Unfollow/unsubscribe from a newsletter channel- - `channel_create`: Create a new newsletter channel (with optional picture)- - `channel_mute`: Mute/unmute a newsletter channel- - `channel_mark_viewed`: Mark channel messages as viewed- - `newsletter_react`: React to a channel message with an emoji- - `newsletter_live_updates`: Subscribe to live view/reaction counts for channel messages- - `contact_profile_pic`: Get profile picture for a contact or group-- **whatsappReceive**: Event-driven trigger that waits for incoming WhatsApp messages with filters (message type, sender, group, channel, keywords, forwarded status). Marked with `['whatsapp', 'trigger']` group for n8n-style trigger identification. Stores group/sender names alongside JID/phone for display persistence. The Go RPC resolves LIDs to phone numbers before sending events - `sender_phone` field is already resolved. Filter options: All Messages, From Self (notes to self chat only), From Any Contact (Non-Group), From Specific Contact, From Specific Group, From Channel (Newsletter), Contains Keywords--### Social Nodes (2 nodes)-Unified social messaging nodes for multi-platform communication. Supports WhatsApp, Telegram, Discord, Slack, Signal, SMS, Webchat, Email, Matrix, Teams.--- **socialReceive**: Normalizes messages from platform triggers into unified format. Multiple outputs: Message, Media, Contact, Metadata. Filters by channel, message type, sender.-- **socialSend**: **Dual-purpose node** - Send messages to any supported platform. Works as workflow node OR AI Agent tool. Supports text, image, video, audio, document, sticker, location, contact, poll, buttons, list message types.--### Twitter/X Nodes (4 nodes)-Twitter/X integration using the official XDK Python SDK with OAuth 2.0 PKCE authentication. All sync XDK calls wrapped in `asyncio.to_thread()` to avoid blocking the event loop. Lazy token refresh on 401/403 errors instead of validating on every call.--- **twitterSend**: **Dual-purpose node** - Post tweets, reply, retweet, like/unlike, and delete tweets. Works as workflow node OR AI Agent tool. Group: `['social', 'tool']`. Actions: `tweet`, `reply`, `retweet`, `like`, `unlike`, `delete`. Parameters: action, text (280 char max), tweet_id, reply_to_id.-- **twitterSearch**: **Dual-purpose node** - Search recent tweets with rich data via X API v2 expansions. Returns enriched tweets with `display_text` (expanded URLs), `author` profile, `public_metrics` (likes/retweets/replies/quotes/bookmarks/impressions), `media` attachments, `referenced_tweets` (quoted/replied), `note_tweet` for long-form content. Works as workflow node OR AI Agent tool. Group: `['social', 'tool']`. `max_results` clamped to 10-100 (X API v2 minimum is 10). Supports query operators: keywords, hashtags (#), mentions (@), from:user, to:user, -exclude, OR, lang:, has:links, has:media, has:images, has:videos, is:retweet, -is:retweet, is:reply, is:quote, url:.-- **twitterUser**: **Dual-purpose node** - Look up user profiles and social connections with description and created_at. Works as workflow node OR AI Agent tool. Group: `['social', 'tool']`. Operations: `me` (get authenticated user), `by_username`, `by_id`, `followers` (max_results 1-1000), `following` (max_results 1-1000).-- **twitterReceive**: Event-driven trigger that waits for incoming Twitter events (mentions, DMs, timeline updates). Group: `['social', 'trigger']`. Polling-based since X API free tier lacks webhooks.--#### Twitter OAuth 2.0 Authentication-Authentication is handled via OAuth 2.0 PKCE flow in the Credentials Modal:-1. User clicks "Login with Twitter" button-2. Backend generates PKCE code challenge and authorization URL-3. Browser opens Twitter authorization page-4. User grants permission, Twitter redirects with auth code-5. Backend exchanges code for access_token + refresh_token-6. Tokens stored in database via auth_service--**Key Files:**-| File | Description |-|------|-------------|-| `server/services/twitter_oauth.py` | OAuth 2.0 PKCE flow implementation |-| `server/services/oauth_utils.py` | Runtime OAuth redirect URI derivation from request context |-| `server/routers/twitter.py` | OAuth callback endpoint, token exchange |-| `server/services/handlers/twitter.py` | Node handlers using XDK SDK |-| `client/src/components/CredentialsModal.tsx` | Twitter panel with OAuth button |-| `server/skills/social_agent/twitter-*-skill/` | 3 Twitter skills for AI agents |--**Handler Architecture:**-- All sync XDK calls wrapped in `asyncio.to_thread()` (XDK uses sync `requests` internally)-- No `get_me()` validation on every call -- lazy token refresh on 401/403 via `_refresh_and_get_client()`-- Search uses full X API v2 expansions and returns enriched data with expanded URLs, author profiles, media, metrics, referenced tweets-- `max_results` clamped: search 10-100, followers/following 1-1000--**XDK SDK API Patterns:**-```python-from xdk import Client-client = Client(access_token=access_token)--client.posts.create(body={"text": "Hello world!"})-client.posts.create(body={"text": "Reply", "reply": {"in_reply_to_tweet_id": "123"}})-client.users.repost_post(user_id, body={"tweet_id": "123"})-client.users.like_post(user_id, body={"tweet_id": "123"})-client.users.unlike_post(user_id, tweet_id="123")-client.posts.delete(tweet_id)-client.posts.search_recent(query="...", max_results=100,- tweet_fields=[...], expansions=[...], media_fields=[...], user_fields=[...])-client.users.get_me(user_fields=["created_at", "description"])-client.users.get_by_usernames(usernames=["user1"], user_fields=["description"])-client.users.get_followers(user_id, max_results=100, user_fields=["created_at"])-```--**Environment Variables:**-```bash-TWITTER_CLIENT_ID=your_client_id-TWITTER_CLIENT_SECRET=your_client_secret-# TWITTER_REDIRECT_URI is derived at runtime from request context (no env var needed)-```--### Telegram Nodes (2 nodes) — reference for self-contained plugin folders-Telegram bot integration using `python-telegram-bot` SDK with long-polling for incoming messages. **All telegram code lives in `server/nodes/telegram/`** — the plugin folder owns its service, WebSocket handlers, event filter, lifecycle hooks, and credentials. Read this folder first before adding any plugin that needs more than a single `BaseNode` subclass; see [docs-internal/plugin_system.md → "Self-contained plugin folders"](./docs-internal/plugin_system.md#self-contained-plugin-folders).--- **telegramSend**: Workflow-only ActionNode (dropped dual-tool support in `190dbb9` — `usable_as_tool=True` removed, group reduced from `('social', 'tool')` to `('social',)`). Send text, photo, document, location, or contact messages via Telegram bot. Recipient types: Self (bot owner), User/Chat ID, Group. Parameters: recipient_type, chat_id, message_type, text, media_url, caption, parse_mode (Auto/HTML/Markdown/MarkdownV2/None), silent, reply_to_message_id. **Auto parse_mode** (default, recommended): converts GFM markdown to Telegram HTML via `markdown_formatter.to_telegram_html()`. MarkdownV2/Markdown text auto-escaped via `_escape_text()`; falls back to plain text on `BadRequest`. "Self" restores `owner_chat_id` from credentials DB if not in memory.-- **telegramReceive**: Event-driven trigger that waits for incoming Telegram messages. Group: `['social', 'trigger']`. Uses `senderFilter` dropdown: All Messages, From Self (Bot Owner) (zero-config), Private/Group/Supergroup/Channel Only, Specific Chat, Specific User, Keywords. Content type filter and ignore bots option. "From Self" uses lazy `_get_owner_chat_id()` lookup at match time. Auto-reconnects on server restart via the registered service-refresh callback.--**Key Files** (all under `server/nodes/telegram/`):-| File | Description |-|------|-------------|-| `__init__.py` | Imports + 6 self-registration calls — the only wiring needed (zero logic). |-| `_credentials.py` | `TelegramCredential(ApiKeyCredential)` — bot token + `telegram_owner_chat_id` extra field. |-| `_service.py` | `TelegramService` singleton: connect/disconnect/send/poll lifecycle, idempotent connect, parse-mode fallback helper. |-| `_handlers.py` | 7 WebSocket handlers (`telegram_connect` / `_disconnect` / `_status` / `_send` / `_reconnect` / `_get_me` / `_get_chat`) + `WS_HANDLERS` dict registered into `services.ws_handler_registry`. |-| `_filters.py` | `build_telegram_filter` — registered into `event_waiter.FILTER_BUILDERS`. |-| `_refresh.py` | `refresh_telegram_status` (registered into `status_broadcaster._SERVICE_REFRESH_CALLBACKS`) + `precheck_telegram_trigger` (registered into `event_waiter._TRIGGER_PRECHECKS`). |-| `telegram_send.py` | `TelegramSendNode(ActionNode)` + `TelegramSendOutput` — the workflow + AI-tool plugin. |-| `telegram_receive.py` | `TelegramReceiveNode(TriggerNode)` + `TelegramReceiveOutput` — the trigger plugin (declares `event_type = "telegram_message_received"`). |-| `client/src/components/CredentialsModal.tsx` | Telegram bot token panel (frontend; identifies commands by WebSocket message-type strings, not Python paths). |--**Authentication:** Bot token from @BotFather on Telegram. Stored via `auth_service.store_api_key('telegram', token, models=[])` (renamed from `telegram_bot_token` — class id, catalogue key, and co-located SVG basename now all align with the brand name). **Owner detection has three independent layers** (commit f746ddf): (1) explicit `Your Chat ID (optional)` text input in the Telegram credentials modal — declared as a secondary `FieldDef` in `credential_providers.json` and rendered by `ApiKeyPanel`'s `SecondaryFieldRow`; saves directly to `telegram_owner_chat_id` via `auth_service.store_api_key`; (2) pre-poll peek in `connect()` via `_capture_owner_from_pending_updates()` — `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue; (3) atomic write-through in `_on_message_received` — persists FIRST, sets in-memory only on success, ERROR-level logging on failure. Lazy fallback in `telegram_send.py` reads DB and calls `service.set_owner` if in-memory is empty. Credentials panel: Save (store only), Connect (store + connect), Reconnect. All credential access uses `from core.container import container; container.auth_service()` (NOT `from services.auth import get_auth_service`).--**Environment Variables:**-```bash-# Optional: Set owner chat ID for "Self" recipient type-TELEGRAM_OWNER_CHAT_ID=your_chat_id-```--### Markdown Formatter Service-Platform-specific markdown formatting using `markdown-it-py` (Python port of `markdown-it`, 18K+ GitHub stars). Converts GFM markdown (as produced by LLMs) to platform-native formats.--**Key File:** `server/services/markdown_formatter.py`--**Public Functions:**-| Function | Target | Description |-|----------|--------|-------------|-| `to_telegram_html(text)` | Telegram | Renders GFM to HTML, converts unsupported tags (`<h1>`-`<h6>` -> `<b>`, `<ul>/<li>` -> bullets, strips `<p>`) |-| `to_whatsapp(text)` | WhatsApp | Walks markdown-it token stream, maps to WhatsApp syntax (`*bold*`, `_italic_`, `~strike~`, `` ```code``` ``, `> quote`) |-| `to_plain(text)` | Any | Renders to HTML, strips all tags |--**Usage:**-- Telegram Send node: Auto parse_mode (default) calls `to_telegram_html()` via `TelegramService._format_auto()`-- WhatsApp Send node: `format_markdown` toggle (default: true) calls `to_whatsapp()` in handler-- No new dependencies -- `markdown-it-py` v4.0.0 already installed as transitive dependency--### Google Workspace Nodes (7 nodes)-Consolidated Google Workspace integration with 6 unified operation-based nodes + 1 polling trigger. Each service node uses an `operation` parameter to select the action (e.g., gmail with operation: send/search/read). All services share a single OAuth connection with combined scopes.--#### Consolidated Service Nodes (6 nodes)-- **gmail**: **Dual-purpose node** - Operations: `send`, `search`, `read`. Handler: `handle_google_gmail()` dispatcher.-- **calendar**: **Dual-purpose node** - Operations: `create`, `list`, `update`, `delete`.-- **drive**: **Dual-purpose node** - Operations: `upload`, `download`, `list`, `share`.-- **sheets**: **Dual-purpose node** - Operations: `read`, `write`, `append`.-- **tasks**: **Dual-purpose node** - Operations: `create`, `list`, `complete`, `update`, `delete`.-- **contacts**: **Dual-purpose node** - Operations: `create`, `list`, `search`, `get`, `update`, `delete`.--#### Trigger Nodes (1 node)-- **gmailReceive**: Polling-based trigger for incoming emails. Polls Gmail API at configurable interval (10-3600s). Parameters: `filter_query` (default: `is:unread`), `label_filter` (default: `INBOX`), `mark_as_read`, `poll_interval` (default: 60s). In deployment mode, uses `setup_polling_trigger` with baseline detection to avoid triggering on existing emails.--#### Google Workspace OAuth 2.0 Authentication-Authentication is handled via OAuth 2.0 flow in the Credentials Modal:-1. User enters Google Cloud Client ID and Secret (OAuth 2.0 credentials)-2. User clicks "Login with Google"-3. Consent screen shows all requested scopes (Gmail, Calendar, Drive, etc.)-4. User grants permission, Google redirects with auth code-5. Backend exchanges code for access_token + refresh_token-6. Tokens stored via auth_service (owner mode) or google_connections table (customer mode)--**Combined OAuth Scopes:**-```python-GOOGLE_WORKSPACE_SCOPES = [- # User Info- "openid",- "https://www.googleapis.com/auth/userinfo.email",- "https://www.googleapis.com/auth/userinfo.profile",- # Gmail- "https://www.googleapis.com/auth/gmail.send",- "https://www.googleapis.com/auth/gmail.readonly",- "https://www.googleapis.com/auth/gmail.modify",- # Calendar- "https://www.googleapis.com/auth/calendar",- "https://www.googleapis.com/auth/calendar.events",- # Drive- "https://www.googleapis.com/auth/drive",- "https://www.googleapis.com/auth/drive.file",- # Sheets- "https://www.googleapis.com/auth/spreadsheets",- # Tasks- "https://www.googleapis.com/auth/tasks",- # Contacts- "https://www.googleapis.com/auth/contacts",- "https://www.googleapis.com/auth/contacts.readonly",-]-```--**Key Files:**-| File | Description |-|------|-------------|-| `server/services/google_oauth.py` | OAuth 2.0 flow (authorization URL, token exchange, `get_callback_paths()`) |-| `server/services/oauth_utils.py` | Runtime OAuth redirect URI derivation from request context |-| `server/services/handlers/google_auth.py` | Shared credential helper (`get_google_credentials()`) used by all 6 handlers |-| `server/config/google_apis.json` | API endpoints, scopes, and OAuth callback paths |-| `server/routers/google.py` | OAuth callback and status endpoints |-| `server/services/handlers/gmail.py` | Gmail handlers + `handle_gmail_receive` polling trigger |-| `server/services/handlers/calendar.py` | Calendar handlers |-| `server/services/handlers/drive.py` | Drive handlers |-| `server/services/handlers/sheets.py` | Sheets handlers |-| `server/services/handlers/tasks.py` | Tasks handlers |-| `server/services/handlers/contacts.py` | Contacts handlers |-| `server/models/database.py` | `GoogleConnection` model |-| `server/skills/productivity_agent/` | Google Workspace skills for AI agents |-| `client/src/assets/icons/google/` | Official Google service SVG icons (n8n pattern) |-| `client/src/components/CredentialsModal.tsx` | Google Workspace panel |--**Token Storage (Two Separate Systems):**--Client ID and Secret are stored as API keys (user enters manually):-| Key | Storage Method | Description |-|-----|---------------|-------------|-| `google_client_id` | `auth_service.get_api_key()` / `EncryptedAPIKey` table | OAuth 2.0 Client ID |-| `google_client_secret` | `auth_service.get_api_key()` / `EncryptedAPIKey` table | OAuth 2.0 Client Secret |--Access and refresh tokens are stored via OAuth system (after Google login):-| Field | Storage Method | Description |-|-------|---------------|-------------|-| `access_token` | `auth_service.get_oauth_tokens("google")` / `EncryptedOAuthToken` table | Access token for API calls |-| `refresh_token` | `auth_service.get_oauth_tokens("google")` / `EncryptedOAuthToken` table | Refresh token for renewal |-| `email`, `name` | `auth_service.get_oauth_tokens("google")` / `EncryptedOAuthToken` table | Connected user info |--**IMPORTANT**: All handlers use `get_google_credentials()` from `google_auth.py`, which reads tokens via `get_oauth_tokens()`. Never use `get_api_key("google_access_token")` — that reads from the wrong table.--**AI Agent Skills (productivity_agent folder):**-| Skill | Tools | Description |-|-------|-------|-------------|-| `gmail-skill` | `gmail_send`, `gmail_search`, `gmail_read` | Send, search, read emails |-| `calendar-skill` | `calendar_create`, `calendar_list`, `calendar_update`, `calendar_delete` | Manage calendar events |-| `drive-skill` | `drive_upload`, `drive_download`, `drive_list`, `drive_share` | Manage Drive files |-| `sheets-skill` | `sheets_read`, `sheets_write`, `sheets_append` | Read/write spreadsheet data |-| `tasks-skill` | `tasks_create`, `tasks_list`, `tasks_complete` | Manage Google Tasks |-| `contacts-skill` | `contacts_create`, `contacts_list`, `contacts_search` | Manage contacts |--**OAuth Redirect URIs:**-OAuth redirect URIs are derived at runtime from the request/WebSocket context via `server/services/oauth_utils.py`. No environment variable needed -- works automatically in dev (`http://localhost:3010`) and production (`https://domain.com`).--**Google API Pricing:** All Google Workspace APIs are free with rate limits. See `server/config/pricing.json` for configured limits.--### Email Nodes (3 nodes)-IMAP/SMTP email integration via the [Himalaya CLI](https://github.com/pimalaya/himalaya). Supports any IMAP/SMTP provider -- Gmail, Outlook/Office 365, Yahoo, iCloud, ProtonMail (Bridge), Fastmail, and custom/self-hosted. Credentials are stored via `auth_service.store_api_key()` and TOML config files are generated on the fly for each operation.--- **emailSend**: **Dual-purpose node** - Send emails via SMTP. Works as workflow node OR AI Agent tool. Group: `['email', 'tool']`. Parameters: provider, to, subject, body, cc, bcc, body_type (text/html).-- **emailRead**: **Dual-purpose node** - Read and manage emails via IMAP. Works as workflow node OR AI Agent tool. Group: `['email', 'tool']`. Operations:- - `list`: List envelopes in a folder with pagination- - `search`: Search emails by query string (e.g., `from:john subject:meeting`)- - `read`: Read full message content by ID- - `folders`: List all mailbox folders- - `move`: Move message to target folder- - `delete`: Delete message- - `flag`: Add/remove flag (Seen, Answered, Flagged, Draft, Deleted)-- **emailReceive**: Polling-based trigger that fires on new emails. Group: `['email', 'trigger']`. Parameters: provider, folder (default `INBOX`), poll_interval (30-3600s), filter_query, mark_as_read. Polls IMAP via Himalaya at the configured interval.--**Architecture:**-```-emailSend/Read/Receive node- |- v-handle_email_send/read/receive (handlers/email.py)- |- v-EmailService (email_service.py) <-- credential resolution + provider defaults- |- v-HimalayaService (himalaya_service.py) <-- CLI wrapper- |- v-himalaya CLI binary <-- generates TOML config, calls IMAP/SMTP-```--**Key Files:**-| File | Description |-|------|-------------|-| `client/src/assets/icons/email/` | Email icons (send, read, receive) loaded via `?raw` + `svgToDataUri` |-| `client/src/components/CredentialsModal.tsx` | Email credentials panel (provider dropdown, email/password, conditional custom IMAP/SMTP) |-| `server/services/himalaya_service.py` | Himalaya CLI wrapper -- send, list_envelopes, search, read, move, delete, flag, list_folders |-| `server/services/email_service.py` | EmailService orchestrator (credential resolution, operation dispatch, polling helpers) |-| `server/services/handlers/email.py` | handle_email_send, handle_email_read, handle_email_receive (thin handlers) |-| `server/services/deployment/manager.py` | `_create_email_poll_coroutine` - deployment-mode continuous polling |-| `server/services/event_waiter.py` | `TRIGGER_REGISTRY['emailReceive']` + `build_email_filter` |-| `server/config/email_providers.json` | IMAP/SMTP provider presets, defaults, polling config (zero magic numbers) |--**Credential Storage** (via `auth_service.store_api_key()`):--| Key | Scope | Description |-|-----|-------|-------------|-| `email_provider` | Always | One of `gmail`, `outlook`, `yahoo`, `icloud`, `protonmail`, `fastmail`, `custom` |-| `email_address` | Always | Account email (IMAP/SMTP login) |-| `email_password` | Always | Password or App Password (stored as secret) |-| `email_imap_host` | Custom only | Fallback when preset empty -- required for `custom` provider |-| `email_imap_port` | Custom only | Stored as string, coerced to int via `_coerce_port` |-| `email_imap_encryption` | Custom only | `tls` / `start-tls` / `none` |-| `email_smtp_host` | Custom only | Fallback SMTP hostname |-| `email_smtp_port` | Custom only | |-| `email_smtp_encryption` | Custom only | |--**Credential Resolution Precedence** (per field, in `EmailService.resolve_credentials`):--1. **Node parameter** (per-node override on the workflow node)-2. **Provider preset** from `email_providers.json` (e.g., `gmail` preset populates `imap_host = "imap.gmail.com"`)-3. **Stored custom API key** (`email_imap_host`, etc.) -- only reached when the preset is empty, i.e. `provider == 'custom'`--For named providers the preset always wins before the custom-key fallback. Credentials panel saves the custom IMAP/SMTP keys only when the user selects "Custom / Self-hosted".--**Installation Requirement:** The `himalaya` CLI binary must be installed and on PATH. Install via `cargo install himalaya`, `brew install himalaya`, or download from https://github.com/pimalaya/himalaya/releases. `HimalayaService.ensure_binary()` caches the resolved path on the singleton after first detection; missing binary raises `RuntimeError` with install instructions.--**Authentication:** Email/password (or App Password for Gmail/Outlook/Yahoo) stored per provider. ProtonMail requires running the ProtonMail Bridge locally (IMAP: `localhost:1143`, SMTP: `localhost:1025`, encryption: `none`).--See **[Email Service](./docs-internal/email_service.md)** for the full architecture, API reference, and operational details.-------### Browser Nodes (1 node)-Interactive browser automation via the [agent-browser](https://www.npmjs.com/package/agent-browser) CLI binary. Wraps a persistent headless Chromium browser with an accessibility-tree-based interaction model designed for AI agents.--- **browser**: **Dual-purpose node** - Interactive browser automation. Works as workflow node OR AI Agent tool. Group: `['browser', 'tool']`. Session persistence across chained operations via `session` parameter (auto-derived as `machina_{execution_id}` if not specified; execution_id is stable per workflow/agent run — threaded through MachinaWorkflow node contexts, AgentWorkflow tool payloads, the legacy tool dispatch, and delegated child agents — so every browser call in one run, including delegated sub-agents, reuses ONE browser instance while separate runs stay isolated). **Instance cap**: each distinct session = one browser instance; `BrowserService` gates new sessions against agent-browser's own registry (`session list --json`) and closes the oldest via per-session `close` when `BROWSER_MAX_INSTANCES` (default 3) would be exceeded; idle browsers self-reap via `AGENT_BROWSER_IDLE_TIMEOUT_MS` injected from `BROWSER_IDLE_TIMEOUT_MS` (default 10 min). Canonical values for both knobs live in `.env.template`. Operations:- - `navigate`: Open a URL- - `click`: Click an element (CSS selector or `@eN` ref from snapshot)- - `type`: Type text keystroke-by-keystroke- - `fill`: Clear and fill an input field- - `screenshot`: Take visible/full-page screenshot- - `snapshot`: Get accessibility tree with `@eN` element refs (AI-optimized, stable within session)- - `get_text`: Extract text from element- - `get_html`: Extract innerHTML from element- - `eval`: Execute JavaScript in page context- - `wait`: Wait for element to appear- - `scroll`: Scroll the page (up/down/left/right)- - `select`: Select dropdown option- - `batch`: Execute multiple commands at once--**Recommended Workflow (AI agents):** `navigate` -> `snapshot` -> `click`/`type`/`fill` using `@eN` refs -> `snapshot` again to verify. Prefer `@eN` refs over CSS selectors (more stable), and prefer `snapshot` + `click`/`fill` over raw `eval`.--**Key Files:**-| File | Description |-|------|-------------|-| `client/src/assets/icons/browser/` | Chrome browser icon (from @ant-design/icons-svg) |-| `server/nodes/browser/_service.py` | BrowserService singleton -- thin subprocess wrapper, reads first JSON line from stdout; instance-cap gate via agent-browser's session registry |-| `server/nodes/browser/browser/__init__.py` | BrowserNode plugin mapping operation+params to agent-browser CLI args; derives the per-run session |-| `server/skills/web_agent/browser-skill/SKILL.md` | AI agent skill documenting snapshot-act-snapshot loop + session semantics |--**Installation Requirement:** `agent-browser` is MachinaOs-managed (NOT a `package.json` dep). On first browser-node invocation, [`server/nodes/browser/_install.py::agent_browser_binary_path`](../server/nodes/browser/_install.py) runs `npm install agent-browser@latest --prefix <packages_dir()>` then `<bin> install` for the Chromium runtime. Same shape as [`claude_binary_path`](../server/nodes/agent/claude_code_agent/_oauth.py) (the precedent) — both land in the single shared npm tree at [`core.paths.packages_dir()`](../server/core/paths.py) = `<DATA_DIR>/packages/` (= `~/.machina/packages/` by default), resolving to `~/.machina/packages/node_modules/.bin/agent-browser[.cmd]`. Pre-fix `packages_dir()` routed through `platformdirs.user_cache_path` (`~/.cache/MachinaOs/`, `~/Library/Caches/MachinaOs/`, `%LOCALAPPDATA%\MachinaOs\Cache\`); it now sits under DATA_DIR with everything else MachinaOs owns.+### Node Catalogue (collapsed)-**Implementation Detail:** `nodes/browser/_service.py` invokes the resolved binary directly via `subprocess.Popen([binary, ...args], shell=False)`. Reads only the first JSON output line from agent-browser (the daemon holds stdout open), truncates at 100KB, and kills the process tree via `psutil` (the Rust daemon detaches and persists between commands). `shell=False` with list argv avoids BatBadBut (CVE-2024-1874) on Windows; `npm` itself is located via `shutil.which("npm")` which honours `PATHEXT` for the `.cmd` shim. The `[Browser]` info log line carries `session=` so per-run session reuse is visible in the operator log.+> The authoritative, per-node reference is the backend plugin registry plus the per-node "logic-flow" cards under [docs-internal/node-logic-flows/](./docs-internal/node-logic-flows/) (one card per node, grouped by category, with handles / params / outputs / side-effects / edge-cases). Live node list = glob `server/nodes/**/__init__.py` (~138 today); live total via `pytest --collect-only`. Do NOT maintain a per-node catalogue here — it drifts on every plugin add.------### Crawlee Nodes (1 node)-Python-based web scraping via the [crawlee](https://github.com/apify/crawlee-python) library. Supports static HTML scraping (BeautifulSoup) and JS-rendered content (Playwright) with built-in concurrency, retries, and anti-bot handling.--- **crawleeScraper**: **Dual-purpose node** - Web scraper. Works as workflow node OR AI Agent tool. Group: `['scraper', 'tool']`. Configuration:- - **Crawler Type**: `beautifulsoup` (static HTML), `playwright` (full browser), `adaptive` (auto-detect)- - **Mode**: `single` (scrape one URL) or `crawl` (follow links with pattern matching)- - **Parameters**: url, cssSelector, extractLinks, linkSelector, urlPattern, maxPages, maxDepth, waitForSelector, waitTimeout, takeScreenshot-- Useful for JavaScript-rendered SPAs where a simple HTTP scraper fails, and for multi-page crawls where concurrency matters.--**Key Files:**-| File | Description |-|------|-------------|-| `server/services/handlers/crawlee.py` | Handler using `BeautifulSoupCrawler` + `PlaywrightCrawler` from crawlee library |-| `server/skills/web_agent/crawlee-scraper-skill/SKILL.md` | AI agent skill for web scraping |--**Installation Requirement:** The `crawleeScraper` node requires `playwright` chromium for JS-rendered content. The Docker image pre-installs Playwright chromium; for local dev run `playwright install chromium`.-------### Apify Nodes (1 node)-Web scraping service for social media, search engines, and websites using pre-built actors.--- **apifyActor**: **Dual-purpose node** - Run Apify actors (web scrapers) for Instagram, TikTok, Twitter/X, LinkedIn, Facebook, YouTube, Google Search, Google Maps, and website crawling. Works as workflow node OR AI Agent tool. Group: `['api', 'scraper', 'tool']`. Pre-built actor dropdown with quick input helpers per actor type. Parameters: actorId, actorInput (JSON), maxResults, timeout, memory.--**Key Files:**-| File | Description |-|------|-------------|-| `server/services/handlers/apify.py` | Actor execution via apify-client SDK |-| `server/skills/web_agent/apify-skill/SKILL.md` | AI agent skill for web scraping |-| `client/src/components/CredentialsModal.tsx` | Apify API token panel |--**Authentication:** Single API token (Personal or Organization) from Apify Console -> Settings -> Integrations.--### Proxy Nodes (3 nodes)-Residential proxy provider management with geo-targeting, session control, and automatic failover.--- **proxyRequest**: Make HTTP requests through residential proxy providers with geo-targeting and failover. Parameters: method, url, headers, body, timeout, proxyProvider (auto-select or specific), proxyCountry (ISO code), sessionType (rotating/sticky), stickyDuration, maxRetries, followRedirects.-- **proxyConfig**: **Dual-purpose node** - Configure proxy providers and routing rules. Works as workflow node OR AI Agent tool. Operations: list_providers, add_provider, update_provider, remove_provider, set_credentials, test_provider, get_stats, add_routing_rule, list_routing_rules, remove_routing_rule.-- **proxyStatus**: View proxy provider health, scores, and usage statistics. Filter by specific provider name or view all.--**Key Architecture:**-- **TemplateProxyProvider**: Single class using JSON `url_template` to format proxy URLs for any provider (username-based, password-based, or no encoding)-- **Auto-selection**: ProxyService ranks providers by health score (success rate, latency) and selects the best one-- **Transparent proxy on HTTP nodes**: `httpRequest` and `httpScraper` nodes support `useProxy: true` flag. The proxy service handles provider selection, geo-targeting, and session type automatically.--**Key Files:**-| File | Description |-|------|-------------|-| `server/services/proxy/service.py` | ProxyService singleton with provider selection and URL generation |-| `server/services/proxy/providers.py` | TemplateProxyProvider for JSON url_template formatting |-| `server/services/proxy/models.py` | ProxyProvider, RoutingRule, SessionType dataclasses |-| `server/nodes/proxy/proxy_config.py`, `proxy_request.py`, `proxy_status.py` | Plugin nodes (Wave 11); ProxyConfig 10-op matrix lives in `proxy_config.py` |-| `server/skills/web_agent/proxy-config-skill/SKILL.md` | AI agent skill for proxy configuration |-| `server/skills/web_agent/http-request-skill/SKILL.md` | HTTP request skill with proxy usage docs |--### Workflow Nodes (2 nodes)-- **start**: Manual workflow trigger to start workflow execution-- **taskTrigger**: Event-driven trigger that fires when a delegated child agent completes its task (success or error). Filters by task_id, agent_name, status (all/completed/error), and parent_node_id. Output includes task_id, status, agent_name, result/error, workflow_id.--### Code Nodes (4 nodes)-- **pythonExecutor**: **Dual-purpose node** - Execute Python code with syntax-highlighted editor, input_data access, and console output. Works as workflow node OR AI Agent tool (`python_code`). Available libraries: math, json, datetime, timedelta, re, random, Counter, defaultdict. CPython `exec()` with a restricted-builtins namespace; its `timeout` param is NOT enforced (only Temporal's CODE_EXEC activity timeout bounds it).-- **montyExecutor**: **Dual-purpose node** - Execute Python in a hard sandbox via Pydantic's [Monty](https://github.com/pydantic/monty) interpreter (`pydantic-monty==0.0.18`, pinned alpha). Works as workflow node OR AI Agent tool (`sandboxed_python`). Deny-by-default: no filesystem/network/env access unless the LLM requests it via the dynamic `capabilities` param (`http_get` → SSRF-guarded host fetch; `workspace_read`/`workspace_write` → `MountDir` over the per-workflow workspace at `/workspace`). **ENFORCED** `timeout` (→ `ResourceLimits.max_duration_secs`) and `max_memory_mb` (→ `max_memory` bytes) — the headline win over pythonExecutor. Supported subset: def/closures/lambda, comprehensions, f-strings, try/except, async def/await, and `import math|json|re`; NOT supported: class, yield/generators, with, match, `import random|collections|os`. Reads `input_data`; returns the last expression as `output` + captured `print()` as `console_output`. Lazy-imports `pydantic_monty` inside the op (graceful `NodeUserError` if the wheel is missing); plugin at [server/nodes/code/monty_executor/](./server/nodes/code/monty_executor/). Skill: `server/skills/coding_agent/monty-skill/SKILL.md`.-- **javascriptExecutor**: **Dual-purpose node** - Execute JavaScript code via persistent Node.js server with syntax-highlighted editor and console output. Works as workflow node OR AI Agent tool (`javascript_code`).-- **typescriptExecutor**: **Dual-purpose node** - Execute TypeScript code via persistent Node.js server with type safety, syntax-highlighted editor and console output. Works as workflow node OR AI Agent tool (`typescript_code`).--### Filesystem & Shell Nodes (4 nodes)-Dual-purpose tool nodes wrapping `deepagents.backends.LocalShellBackend`. Per-workflow workspace from execution context (`context["workspace_dir"]` = `~/.machina/workspaces/<workflow_slug>/` post-Wave-14; the slug is the human-readable form `AI_Assistant_1`, see "Workflow Naming" section). Fallback uses `Settings().workspace_base_dir` (never `os.getcwd()`).--**Path safety**: `nodes/filesystem/_backend.py` exposes `normalize_virtual_path()` which uses `pathlib.PureWindowsPath` (host-OS independent) to strip Windows drives, POSIX root, and UNC anchors uniformly, then delegates to `deepagents.backends.utils.validate_path` for `..`/`~` rejection. Wired into `file_read`, `file_modify`, and `fs_search` so LLM-emitted paths in any flavour (`C:\foo`, `/tmp/foo`, `\\server\share\x`, `foo\bar`) all map to a virtual path under the workspace. `virtual_mode=True` only sandboxes filesystem ops — `execute()` itself is never path-restricted (deepagents documents this).--**Shell backend is Nushell** (`NushellBackend` subclassing `LocalShellBackend`). Cross-platform parity — same grammar on Windows/macOS/Linux. `inherit_env=True` so `npm`, `node`, `python`, `git`, etc. are reachable on PATH. Nu stdlib is loaded (no `--no-std-lib`). Falls back to upstream `LocalShellBackend.execute()` (POSIX `sh` / cmd.exe) when `nu` isn't on PATH. **Bash idioms (`&&`, `||`, `$VAR`, backticks, `>`) do not work** — see `server/skills/terminal/shell-skill/SKILL.md` for the Nu equivalents.--- **fileRead**: **Dual-purpose node** - Read file contents with line numbers and pagination. Works as workflow node OR AI Agent tool (`file_read`). Parameters: file_path, offset, limit.-- **fileModify**: **Dual-purpose node** - Write new files or edit existing files with string replacement. Works as workflow node OR AI Agent tool (`file_modify`). Operations: write (wholesale create-or-replace, no overwrite flag), edit (find and replace with old_string/new_string/replace_all).-- **shell**: **Dual-purpose node** - Execute Nushell commands with timeout. Works as workflow node OR AI Agent tool (`shell_execute`). PATH inherited; external tools (npm, node, python, git, ...) work. All backend calls wrapped in `asyncio.to_thread()` to avoid blocking the event loop. Returns stdout, exit_code, truncated flag.-- **fsSearch**: **Dual-purpose node** - Search the filesystem with three modes. Works as workflow node OR AI Agent tool (`fs_search`). Modes: ls (list directory), glob (pattern match), grep (search file contents).--**Key Files:**-| File | Description |-|------|-------------|-| `server/nodes/filesystem/_backend.py` | `NushellBackend`, `get_backend()`, `normalize_virtual_path()` |-| `server/skills/coding_agent/` | Skills: file-read-skill, file-modify-skill, fs-search-skill |-| `server/skills/terminal/` | Skills: shell-skill (Nushell), process-manager-skill, bash-skill, powershell-skill, wsl-skill |--### Process Manager Node (1 node)-Cross-platform process manager for long-running subprocesses (dev servers, watchers, build tools). Uses `asyncio.create_subprocess_exec` with full system PATH (`env={**os.environ}`). Output streams to Terminal tab via `broadcast_terminal_log()` and persists to log files in the workspace. **Captured lines are ANSI-stripped at the source** — `process_service._read_stream` runs each decoded line through `core.ansi.strip_ansi` (a thin wrapper over **`click.unstyle`**, a declared dep) before the log-file write / Terminal broadcast / `get_output` / `line_handler`, so colour + cursor/erase codes from `vite`/`npm`/`git`/etc. don't render as garbage (`[36m…[39m`). The `shell` node strips its `stdout` the same way. `click.unstyle` was chosen over `rich.Text.from_ansi` because rich line-normalizes and drops trailing newlines; `click.unstyle` is byte-faithful except for the stripped escapes.--- **processManager**: **Dual-purpose node** - Start, stop, restart, and manage long-running processes. Works as workflow node OR AI Agent tool (`process_manager`). Operations: start, stop, restart, list, send_input, get_output. Max concurrent processes configurable in Settings (default: 10).--**Key Architecture:**-- **ProcessService** singleton (`server/services/process_service.py`) tracks running processes per workflow-- Each process writes stdout/stderr to `{workspace}/{agent_node_id}/.processes/{name}/stdout.log` and `stderr.log`-- AI agents read output selectively via `get_output(name, stream, tail)` with pagination-- Process tree cleanup via `psutil.children(recursive=True)` on stop and server shutdown-- `PYTHONUNBUFFERED=1` injected for line-buffered Python output--**Key Files:**-| File | Description |-|------|-------------|-| `server/services/process_service.py` | ProcessService singleton with start/stop/restart/list/send_input/get_output/shutdown |-| `server/services/handlers/process.py` | Dual-purpose handler (workflow + AI tool) |-| `server/skills/terminal/process-manager-skill/` | Skill for AI agents |--### Utility Nodes (6 nodes)-- **httpRequest**: Make HTTP requests to external APIs (GET, POST, PUT, DELETE, PATCH) with configurable headers, body, timeout, and optional proxy support (`useProxy: true` routes through configured residential proxy)-- **webhookTrigger**: Event-driven trigger that waits for incoming HTTP requests at `/webhook/{path}` with method filtering and authentication options-- **webhookResponse**: Send custom response back to webhook caller with configurable status code, body, and content type-- **chatTrigger**: Trigger node that receives messages from the Console Panel chat interface-- **console**: Console output node for logging workflow execution data-- **teamMonitor**: Real-time monitoring of Agent Team operations. Connect to AI Employee or Orchestrator to display team status, active tasks, and event stream--### Document Processing Nodes (6 nodes)-RAG pipeline nodes for document ingestion, processing, and vector storage. Supports multiple providers and backends.--- **httpScraper**: Scrape links from web pages with date/page pagination support. Modes: single request, date range iteration, page pagination. Outputs: items array with URLs. Supports `useProxy: true` for proxy routing.-- **fileDownloader**: Download files from URLs in parallel using semaphore-based concurrency. Parameters: output directory, max workers (1-32), skip existing, timeout.-- **documentParser**: Parse documents to text using configurable parsers. Parsers: PyPDF (fast), Marker (GPU OCR), Unstructured (multi-format), BeautifulSoup (HTML).-- **textChunker**: Split text into overlapping chunks for embedding. Strategies: recursive (recommended), markdown, token. Parameters: chunk size (100-8000), overlap (0-1000).-- **embeddingGenerator**: Generate vector embeddings from text chunks. Providers: HuggingFace (local), OpenAI, Ollama. Default model: BAAI/bge-small-en-v1.5.-- **vectorStore**: Store and query vector embeddings. Operations: store, query, delete. Backends: ChromaDB (local), Qdrant (production), Pinecone (cloud).--### Chat Nodes (2 nodes)-- **chatSend**: Send messages to chat conversations-- **chatHistory**: Retrieve chat conversation history--### Scheduler Nodes (2 nodes)-- **timer**: Timer-based trigger with configurable delay-- **cronScheduler**: Cron expression-based scheduling trigger--#### Document Processing Dependencies-```-# Required (in server/requirements.txt)-beautifulsoup4>=4.12.0-langchain-text-splitters>=0.3.0-langchain-huggingface>=0.1.0-chromadb>=0.5.0-qdrant-client>=1.12.0-sentence-transformers>=3.0.0-pypdf>=4.0.0--# Optional (GPU OCR and multi-format parsing)-# marker-pdf>=1.0.0 # Requires CUDA-# unstructured>=0.16.0 # Multi-format document parsing-```--#### Document Node Architecture-```-server/nodes/document/-├── http_scraper.py # each plugin emits its own NodeSpec-├── file_downloader.py-├── document_parser.py-├── text_chunker.py-├── embedding_generator.py-└── vector_store.py-```+Node groups (palette categories): agent, model, skill, tool, trigger, workflow, search, google, android, whatsapp, telegram, twitter, social, email, proxy, chat, scheduler, text, code, document, location, utility, browser, scraper, filesystem, stripe. See [docs-internal/node-logic-flows/](./docs-internal/node-logic-flows/) for the card index.## Backend Services@@ -1618,7 +646,7 @@ server/nodes/document/- `GET /webhook/` - Webhook endpoint info and usage documentation#### Workflow Services (`server/services/workflow.py`)-- Node execution handlers for all 27 node types (including httpRequest, webhookResponse)+- Node execution dispatches every registered plugin via the `BaseNode` registry (authoritative count = glob `server/nodes/**/__init__.py`, ~138 node files today; live total via `pytest --collect-only`)- Parameter resolution and template variable substitution- Result formatting and error handling@@ -1653,89 +681,7 @@ server/nodes/document/### Temporal Distributed Execution-Workflows execute via Temporal for durability and horizontal scaling. Three dispatch paths (legacy `execute_node_activity` / per-type `node.{type}.v{version}` (F4.A) / Agent-as-child-workflow (F4.B)) gated by `TEMPORAL_PER_TYPE_DISPATCH` and `TEMPORAL_AGENT_WORKFLOW_ENABLED` settings flags.--Full architecture, dispatch matrix, per-node lifecycle, heartbeat semantics, the 14 migrating agent types + 2 bypass agents (rlm_agent / claude_code_agent), the 6 F4.B agent activities (`prepare_payload.v1` / `execute_llm_step.v1` / `persist_turn.v1` / `compact_memory.v1` / `store_output.v1` / `broadcast_progress.v1`), the **`delegate_to_<x>` Temporal-native dispatch** (child agent type in `AGENT_WORKFLOW_TYPES` → `workflow.execute_child_workflow` with `parent_node_id` propagation + parent-mirror in `_emit_phase` so the parent's canvas badge advances live; falls through to `execute_activity` otherwise), and the future `TemporalWorkerPool` per-queue routing live in [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md). Tool-call dispatch under F4.A documented at [docs-internal/tool_building_pipeline.md §9](./docs-internal/tool_building_pipeline.md). **Delegation input contract**: the LLM's `{task, context}` args travel as the child workflow input's `invocation` field (per-invocation input vs node config); `prepare_agent_payload` applies it AFTER config resolution so stored node parameters never override the delegated task — empty-task calls are rejected at the parent's call boundary, and `execute_llm_step` raises `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)` on system-only message lists (providers require ≥1 non-system message).--**Configuration** (`.env.template` — canonical defaults; `.env` layers user overrides):-```env-TEMPORAL_ENABLED=true-TEMPORAL_SERVER_ADDRESS=localhost:7233-TEMPORAL_NAMESPACE=default-TEMPORAL_TASK_QUEUE=machina-tasks-TEMPORAL_PER_TYPE_DISPATCH=true # F4.A flag — per-type activity dispatch-TEMPORAL_AGENT_WORKFLOW_ENABLED=true # F4.B flag — agent-as-child-workflow-TEMPORAL_FRONTEND_GRPC_PORT=7233-TEMPORAL_UI_PORT=8080 # Web UI (override of CLI default 8233)-TEMPORAL_SQLITE_PATH=temporal.db # Flat under DATA_DIR (~/.machina/temporal.db)-TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS=30-TEMPORAL_SERVER_READY_TIMEOUT_SECONDS=120-TEMPORAL_TERMINATE_RUNNING_ON_STARTUP=true # see "Resumption" below-```--**Temporal binary + persistence** are managed in-process by the plugin-folder pattern under [`server/services/temporal/`](./server/services/temporal/). Single supervised process — the official `temporal` CLI's `server start-dev` mode (per [docs.temporal.io/develop/python/set-up-your-local-python](https://docs.temporal.io/develop/python/set-up-your-local-python)):--- **`_install.py`** — [`pooch`](https://pypi.org/project/pooch/) downloads the official `temporal` CLI archive from `https://temporal.download/cli/archive/latest?platform=<os>&arch=<arch>`. Cross-platform (Windows zip / macOS+Linux tar.gz), cached at `<DATA_DIR>/packages/temporal/`. Download uses an explicit `HTTPDownloader(timeout=300)` — the timeout is per-socket-read, not total transfer, so slow links can finish a ~114 MB fetch (pooch's 30 s default aborted them). Standalone entry: `python -m services.temporal._install` — invoked by `machina build` step [6/6] (fatal there, locked by `test_temporal_install_is_fatal_on_failure`) and by npm postinstall (**non-fatal** there: `TemporalServerRuntime._pre_spawn()` re-downloads lazily on first `machina start`, so a failed eager fetch must never fail `npm install -g`). Note dev mode resolves `DATA_DIR` to repo-local `.machina/`, so dev and `~/.machina` installs keep separate binary caches.-- **`_runtime.py`** — `TemporalServerRuntime(BaseProcessSupervisor)` spawns `temporal server start-dev` with `--port`, `--ui-port`, `--db-filename`, `--namespace`. Same singleton + lifecycle surface every other supervised service uses ([`nodes/whatsapp/_runtime.py`](./server/nodes/whatsapp/_runtime.py) is the template).-- **`_handlers.py` + `_refresh.py`** — `temporal_status` / `_start` / `_stop` WS commands and the WS-connect status snapshot, registered through `services.ws_handler_registry` and `services.status_broadcaster.register_service_refresh`.-- **`client.py`** — `TemporalClientWrapper.connect()` + `terminate_running_workflows()` (resumption toggle, see below).--**Ports:**-| Service | Port | URL |-|---------|------|------------------------|-| gRPC | 7233 | localhost:7233 |-| Web UI | 8080 | http://localhost:8080 |--Both ports are bound by the same `temporal.exe` process (per docs.temporal.io/cli/server — "all running in a single process").--**Resumption** — controlled by `TEMPORAL_TERMINATE_RUNNING_ON_STARTUP` (default `true`):-- Temporal SQLite is always persisted (history preserved across restarts; UI keeps showing every workflow that ever ran).-- On server boot, after the Temporal client connects but BEFORE the worker starts, `TemporalClientWrapper.terminate_running_workflows()` queries Visibility for every `Running` workflow and calls `handle.terminate(reason="MachinaOS startup: auto-resumption disabled")`. Workflows show as `Terminated` (not deleted) in the Temporal UI.-- Why: `DeploymentManager` has no boot-time reconcile against Temporal Visibility yet — resumed workflows would otherwise keep executing but be invisible to the MachinaOS UI. Flip to `false` once the reconcile lands.--**Start script integration** ([`cli/commands/start.py`](./cli/commands/start.py) + [`cli/commands/dev.py`](./cli/commands/dev.py)): both call `temporal_specs(root, cfg)` ([`cli/commands/_temporal_specs.py`](./cli/commands/_temporal_specs.py)) which returns the single ServiceSpec for the supervised Temporal dev server. The supervised-runtime shim lives at [`server/services/temporal/_supervised_runtime.py`](./server/services/temporal/_supervised_runtime.py) so the spawned `uv run python -m services.temporal._supervised_runtime` resolves out of the workspace `.venv` without any cross-tree path plumbing. The supervisor's `_temporal_running()` probe short-circuits if Temporal is already running on 7233 (lets you run `temporal server start-dev` standalone for debugging).--**Docker**: Not used. The Python supervisor handles cross-platform lifecycle via the pooch-downloaded CLI — single binary, single process, single SQLite file.--**Execution Routing** (`workflow.py`):-1. If `TEMPORAL_ENABLED=true` and Temporal configured → `_execute_temporal()`-2. Else if Redis available → `_execute_parallel()` (local parallel)-3. Else → `_execute_sequential()` (fallback)--**Running with Temporal**:-Temporal server and embedded worker start automatically with all launch scripts:-```bash-npm run start # Starts Temporal server + all services-npm run dev # Starts Temporal server + all services (dev mode)-npm run stop # Stops all services including Temporal-```-The embedded Temporal worker runs inside the Python backend (`main.py` lifespan via `TemporalWorkerManager`), not as a separate process.--**Standalone Worker** (horizontal scaling):-```bash-cd server-python -m services.temporal.worker-```--**Node Filtering** (`workflow.py`):-- Config nodes filtered by handle: `CONFIG_HANDLES = {input-tools, input-memory, input-model, input-skill, input-task, input-teammates}`-- Trigger nodes (`TRIGGER_NODE_TYPES`) auto-completed if not pre-executed (prevents blocking on event waiter)-- Android service nodes and skill nodes filtered by type--**Key Files**:-- `services/temporal/workflow.py` - MachinaWorkflow orchestrator (FIRST_COMPLETED pattern, `_resolve_activity` dispatch)-- `services/temporal/activities.py` - Legacy `execute_node_activity` (WebSocket round-trip path)-- `services/temporal/plugin_activities.py` - `collect_plugin_activities` → per-type activities for F4.A-- `services/temporal/agent_workflow.py` - `AgentWorkflow` child workflow for F4.B agent loops; its first step calls `agent.prepare_payload.v1` to resolve the DB-backed payload, then loops LLM steps + tool dispatch + per-turn persist + compaction-- `services/temporal/agent_activities.py` - F4.B activities (6 total): `agent.prepare_payload.v1` (resolves DB params + runs `_param_resolver.resolve` so `{{templates}}` work inside the agent's own prompt), `agent.execute_llm_step.v1` (rebuilds tools via `ai_service._build_tool_from_node`; uses LangChain `messages_to_dict`/`messages_from_dict` so Gemini `thought_signature` survives), `agent.persist_turn.v1`, `agent.compact_memory.v1`, `agent.store_output.v1` (wraps `workflow_service.store_node_output` so downstream nodes can resolve `{{aiAgent.response}}`), `agent.broadcast_progress.v1` (emits CloudEvents-shaped `agent_progress` for every phase: `starting` / `llm_step` / `executing_tool` / `tool_completed` / `completed`, optionally driving raw-dict `update_node_status` for canvas-glow color)-- `services/temporal/worker.py` - TemporalWorkerManager (registers MachinaWorkflow + AgentWorkflow + activities) + `run_standalone_worker()` + `TemporalWorkerPool` (multi-queue, future enhancement)-- `services/temporal/executor.py` - TemporalExecutor interface matching WorkflowExecutor-- `services/temporal/client.py` - Client wrapper with runtime heartbeat disabled-- `services/plugin/base.py::BaseNode.as_activity()` - Per-type activity body for F4.A (mirrors the legacy pipeline)--**Runtime Configuration**: Worker heartbeating is disabled via `Runtime(worker_heartbeat_interval=None)` to avoid warnings on older Temporal server versions.--**Temporal UI**: http://localhost:8080 (Web UI), http://localhost:8233 (HTTP API)+Workflows execute via Temporal for durability and horizontal scaling, gated by `TEMPORAL_ENABLED`. Three dispatch paths exist — legacy single `execute_node_activity` (default), per-type `node.{type}.v{version}` activities (F4.A, `TEMPORAL_PER_TYPE_DISPATCH=true`), and Agent-as-child-workflow (F4.B, `TEMPORAL_AGENT_WORKFLOW_ENABLED=true`) — with `rlm_agent`/`claude_code_agent` always bypassing AgentWorkflow. Execution routing falls back Temporal → Redis-parallel → sequential; the Temporal dev server + embedded worker are managed in-process under `server/services/temporal/` (pooch-downloaded CLI, single `temporal server start-dev` process, gRPC 7233 + Web UI 8080, SQLite at `~/.machina/temporal.db`). Full architecture, dispatch matrix, per-node + agent-loop lifecycle, the 7 F4.B `agent.*.v1` activities, heartbeat semantics, delegation input contract, resumption toggle (`TEMPORAL_TERMINATE_RUNNING_ON_STARTUP`), and all `.env` tunables live in [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md).## Development Commands@@ -1811,7 +757,7 @@ See **[Scripts Reference](./docs-internal/SCRIPTS.md)** for full documentation.✅ **Authentication System**: n8n-style JWT authentication with HttpOnly cookies, single-owner and multi-user modes✅ **Cache System**: n8n-pattern cache with Redis (production) / SQLite (local dev) / Memory fallback hierarchy✅ **AI Thinking/Reasoning**: Extended thinking for Claude, Gemini 2.5/3, OpenAI GPT-5/o-series, Groq Qwen3 with output available in Input Data & Variables for downstream nodes-✅ **Onboarding Service**: 5-step welcome wizard with Ant Design UI, database persistence, skip/resume/replay support+✅ **Onboarding Service**: 5-step welcome wizard with shadcn UI, database persistence, skip/resume/replay support✅ **Proxy System**: Residential proxy provider management with template-based URL formatting, auto-selection by health score, transparent proxy injection on httpRequest/httpScraper nodes via `useProxy: true`✅ **Markdown Formatter**: GFM markdown to platform-native formatting (Telegram HTML, WhatsApp syntax, plain text) using markdown-it-py@@ -2148,495 +1094,77 @@ deploy_workflow() -> Sets up triggers, returns immediately- `server/services/execution/executor.py`: Continuous scheduling with `asyncio.wait(FIRST_COMPLETED)`### AI Chat Model System (5-Layer Architecture)-- **Visual Components**: Circular node design with real-time status indicators (green/yellow/red)-- **Node Definitions**: Backend NodeSpec (`server/nodes/models/<provider>.py`) is the single source of truth; the frontend renders via `lib/nodeSpec.ts` + `adapters/nodeSpecToDescription.ts`-- **Parameter System**: Universal renderer with drag-and-drop template variables (`{{variable}}`)-- **API Key Management**: Secure localStorage with base64 encryption and LangChain validation-- **Execution Engine**: Routes AI nodes to Python Flask backend with auto-injection of API keys--#### Supported AI Providers & Models-9 providers are available for aiAgent, chatAgent (Zeenie), and all specialized agents: OpenAI, Anthropic, Google (Gemini), DeepSeek, Kimi (Moonshot), Mistral, OpenRouter, **Ollama**, **LM Studio**. Groq and Cerebras are available as standalone chat model nodes only. Model parameters (max output, context length, thinking type, temperature range) are managed by `ModelRegistryService` (`server/services/model_registry.py`) which fetches from OpenRouter for cloud models, and from the user's running local server (via `ollama.AsyncClient.ps()` / `lmstudio.AsyncClient.llm.list_loaded()`) for Ollama / LM Studio. Falls back to `server/config/llm_defaults.json` only when neither source is available.--**Dual-path execution architecture:**-- **Native SDK path** (`execute_chat()`, `fetch_models()`): OpenAI, Anthropic, Gemini, OpenRouter, xAI, DeepSeek, Kimi, Mistral, Ollama, LM Studio use the `openai` Python SDK via `services/llm/` layer with config-driven `base_url`. Local providers (Ollama, LM Studio) ride the same OpenAI-compat path — `OpenAIProvider` is constructed with `base_url={user-stored URL}` so requests stay on `http://localhost:…` and never hit api.openai.com. Factory: `create_provider(name, api_key)` with lazy imports.-- **LangChain agent path** (`execute_agent()`, `execute_chat_agent()`): All providers use `ChatOpenAI` with `base_url` from `llm_defaults.json` + `chat_model.bind_tools` for tool calling, driven by the plain-async `_run_agent_loop`. OpenAI-compatible providers (DeepSeek, Kimi, Mistral) pass `max_tokens` via `extra_body` to bypass LangChain's `max_completion_tokens` conversion.-- **LangChain fallback** (`execute_chat()`): Groq, Cerebras still use LangChain `create_model()` + `chat_model.invoke()`.-- Native types aliased in ai.py: `NativeMessage`, `NativeThinkingConfig`, `LLMResponse` to avoid naming conflicts with LangChain's `ThinkingConfig`.-- Provider configs (base_url, models_endpoint, supported_params, temperature constraints) are driven by `server/config/llm_defaults.json` -- no hardcoded URLs in Python code.--**Local LLM SDKs (Ollama, LM Studio):** the validator at [`server/nodes/model/_local_validator.py`](./server/nodes/model/_local_validator.py) probes the user's running server through the official Python SDKs (`ollama>=0.6.0`, `lmstudio>=1.5.0`) and reads only typed fields — no Modelfile-parameters parsing, no `/api/show` modelinfo dict-key hunting. `ollama.AsyncClient.ps()` returns a typed `ProcessResponse.Model` per loaded model with typed `context_length` + typed `ModelDetails` (family, parameter_size, quantization_level, format). `lmstudio.AsyncClient.llm.list_loaded()` returns `AsyncModelHandle` objects whose `get_info()` is a typed `LlmInstanceInfo` (`context_length`, `max_context_length`, `vision`, `trained_for_tool_use`, `architecture`, `params_string`, `format`). The validator persists every probed field into `EncryptedAPIKey.models["model_params"]` (the same JSON column as the model list) and calls `model_registry.register_local_model()` so the sync `get_context_length()` / `get_max_output_tokens()` lookups return real values. The registry also writes through to `model_registry.json` so per-model params survive process restart — the user doesn't have to re-click "Fetch" every time the server bounces.--**Open-world providers** (`openrouter`, `ollama`, `lmstudio`) skip the model-name pattern check in `is_model_valid_for_provider` — local model names like `qwen/qwen3.6-27b` don't contain provider substrings, and the cloud-style filter would have produced a "invalid for provider … using default: <same name>" log line on every chat call. The upstream server still rejects genuinely missing models with a clear 404.--**Provider detection** (`detect_ai_provider` in `server/constants.py`) substring-matches the node type's lowercase form. Local-server providers MUST be listed there — without `ollama` / `lmstudio` branches, `lmstudioChatModel` falls through to `'openai'` and `execute_chat` ends up calling api.openai.com with the local-server placeholder key. Same fix is required in the agent `provider` Literal in `ai_agent.py` / `chat_agent.py` / `_specialized.py`: the dropdown only offers the values listed there, and any local-LLM user picking a missing entry silently falls back to OpenAI cloud.--| Provider | Key Models | Context | Max Output | Thinking | Temp Range |-|----------|-----------|---------|-----------|----------|------------|-| **OpenAI** | GPT-5.5/5.4/5.2 | 400K–1.05M | 128K | effort (hybrid) | 0-2 |-| **OpenAI** | GPT-4.1/4.1-mini/4.1-nano | 1M | 32K | none | 0-2 |-| **OpenAI** | o3, o4-mini | 200K | 100K | effort (reasoning) | fixed 1.0 |-| **OpenAI** | GPT-4o/4o-mini | 128K | 16K | none | 0-2 |-| **Anthropic** | Claude Fable 5 | 1M | 128K | budget | 0-1 |-| **Anthropic** | Claude Opus 4.8/4.7 | 1M | 128K | budget | 0-1 |-| **Anthropic** | Claude Sonnet 4.6 | 1M | 64K | budget | 0-1 |-| **Anthropic** | Claude Haiku 4.5 | 200K | 64K | budget | 0-1 |-| **Google** | Gemini 3.5-flash, 3.1-pro/flash-lite, 3-flash, 2.5-pro/flash/flash-lite | 1M | 64K | budget | 0-2 |-| **DeepSeek** | deepseek-v4-flash, deepseek-v4-pro (deepseek-chat/reasoner legacy) | 1M | 64K | thinking modes | 0-2 |-| **Kimi** | kimi-k2.6, kimi-k2.5, kimi-k2.7-code | 256K | 96K | on by default (disabled for agents) | fixed 0.6 |-| **Mistral** | mistral-large/medium/small-latest, codestral-latest | 256K | 131K | none | 0-1.5 |-| **Groq** | Llama 3.3-70b, Llama 3.1-8b, GPT-OSS-120b/20b, Qwen3-32b | 131K | 8-131K | format (Qwen3) | 0-2 |-| **OpenRouter** | 200+ models from multiple providers | varies | varies | varies | 0-2 |-| **Cerebras** | GPT-OSS-120b, Qwen-3-235b, GLM-4.7 | 131K | 65K | budget (Qwen) | 0-1.5 |-| **Ollama** | Whatever the user has pulled (qwen2.5, llama3.x, phi-3, deepseek-r1, ...) | per-loaded-model (typed via `ps()`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |-| **LM Studio** | Whatever the user has loaded in the LM Studio UI | per-loaded-model (typed via `LlmInstanceInfo.context_length`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |--`_resolve_max_tokens()` in `ai.py` clamps user-requested max_tokens to the model's actual limit.--#### Key Features-- Visual configuration with status indicators and parameter buttons-- Template variable system for dynamic parameter binding from connected nodes-- Provider-specific parameter sets (temperature, max tokens, penalties, safety settings)-- Secure API key validation with automatic model discovery and 30-day expiration-- Execution routing to Python Flask backend for AI model processing-- **Proxy-based authentication** for routing through local servers (Ollama pattern)-- **Provider default parameters** configurable in Credentials Modal (temperature, max_tokens, thinking settings)--#### Provider Default Parameters-Users can configure default parameter values per LLM provider in the Credentials Modal. These defaults are applied to new AI nodes using that provider.--**Configurable Parameters:**-- `temperature`: Controls randomness (range varies: Anthropic 0-1, Cerebras 0-1.5, others 0-2; o-series fixed at 1.0)-- `max_tokens` (1-200000): Maximum response length (clamped to model's actual limit by `_resolve_max_tokens()`)-- `thinking_enabled`: Enable extended thinking for supported models-- `thinking_budget` (1024-16000): Token budget for thinking (Claude, Gemini)-- `reasoning_effort` (low/medium/high): For OpenAI o-series and GPT-5 hybrid reasoning-- `reasoning_format` (parsed/hidden): For Groq Qwen3 models--**Database Model** (`server/models/database.py`):-```python-class ProviderDefaults(SQLModel, table=True):- provider: str # openai, anthropic, gemini, groq, openrouter, cerebras- temperature: float- max_tokens: int- thinking_enabled: bool- thinking_budget: int- reasoning_effort: str # low, medium, high- reasoning_format: str # parsed, hidden-```--**Key Files:**-| File | Description |-|------|-------------|-| `server/models/database.py` | `ProviderDefaults` SQLModel |-| `server/core/database.py` | `get_provider_defaults()`, `save_provider_defaults()` CRUD |-| `server/routers/websocket.py` | `get_provider_defaults`, `save_provider_defaults` handlers |-| `client/src/hooks/useApiKeys.ts` | `getProviderDefaults()`, `saveProviderDefaults()` methods |-| `client/src/components/CredentialsModal.tsx` | Default Parameters UI section |-#### Proxy-Based Authentication (Ollama Pattern)-AI providers support optional proxy-based authentication, allowing requests to route through a local proxy server that handles authentication. This follows the [Ollama Claude Code integration](https://docs.ollama.com/integrations/claude-code) pattern.+Chat-model nodes (`openaiChatModel`, `anthropicChatModel`, ...) render through `SquareNode` from the backend NodeSpec. Dual-path execution: native SDK (`ChatUnifier` in `services/llm/`) for chat, LangChain (`_run_agent_loop`) for agents. 11 providers are wired for agents (see the provider list above); model params (max output, context length, thinking type, temperature range) come from `ModelRegistryService`. The 5-layer architecture, the native-vs-LangChain split, proxy/local-LLM (Ollama, LM Studio) routing, the per-provider model + thinking/reasoning matrix (budget / effort / format), and `extract_thinking_from_response` all live in **[Native LLM SDK](./docs-internal/native_llm_sdk.md)**. The runtime output schema (`thinking` field for downstream nodes) is backend-served per [Schema Source of Truth RFC](./docs-internal/schema_source_of_truth_rfc.md).-**How it works:**-1. User configures a proxy URL in the Credentials Modal (e.g., `http://localhost:11434`)-2. Requests route through the proxy instead of directly to the provider API-3. Proxy handles authentication (token set to "ollama" automatically)-4. No API key storage needed in MachinaOs - auth delegated to proxy+## AI Agent Node Architecture-**Configuration:**-- Proxy URLs stored in database via `{provider}_proxy` pattern (e.g., `anthropic_proxy`, `openai_proxy`)-- Configured in Credentials Modal under each AI provider-- Falls back to direct API key if no proxy configured+AI Agent (`aiAgent`) and Chat Agent / Zeenie (`chatAgent`) run the same plain-async `_run_agent_loop` (`server/services/ai.py`) and support memory / skills / tools / task input; differences are the backend method (`execute_agent()` vs `execute_chat_agent()`) and error-handling softness. The loop routes purely on `response.tool_calls`, hot-rebinds the tool surface mid-run after canvas mutations, and treats compaction (token-based, post-turn) as the real termination signal. Connection collection is `collect_agent_connections` in `server/services/plugin/edge_walker.py` (5-tuple: memory, skill, tool, input, task); the pre-Wave-11 `handle_ai_agent`/`handle_chat_agent` handlers are gone (dispatch is per-plugin `execute_op` under `server/nodes/agent/<plugin>/__init__.py`).-**Key Files:**-| File | Description |-|------|-------------|-| `server/services/ai.py` | Native path: `create_provider(name, api_key, proxy_url=url)`. LangChain path: `create_model()` with `base_url` kwarg. |-| `client/src/components/CredentialsModal.tsx` | Proxy URL input for AI providers |+**`max_iterations` precedence (highest->lowest): per-node `parameters.max_iterations` > `UserSettings.agent_recursion_limit` > env `AGENT_RECURSION_LIMIT` (default 200) > `llm_defaults.json:agent.recursion_limit`.**-**Backend Implementation** (`server/services/ai.py`):-```python-def create_model(self, provider: str, api_key: str, model: str,- temperature: float, max_tokens: int,- thinking: Optional[ThinkingConfig] = None,- proxy_url: Optional[str] = None):- # ...- if proxy_url:- kwargs['base_url'] = proxy_url- kwargs[config.api_key_param] = "ollama" # Ollama-style token-```+Full reference — agent loop, skill injection, tool building, input/auto-prompt fallback (message>text>content>str), handle topology, spec-driven `AIAgentNode`, async delegation (F4.B child-workflow + legacy fire-and-forget), specialized-agent routing — in [docs-internal/agent_architecture.md](./docs-internal/agent_architecture.md); delegation deep-dive in [agent_delegation.md](./docs-internal/agent_delegation.md), team leads in [agent_teams.md](./docs-internal/agent_teams.md).-**Use Cases:**-- Claude Code CLI proxy for Anthropic models-- **Native Ollama / LM Studio support** — these are first-class providers (see "Local LLM SDKs" above). The user enters their server URL (e.g. `http://localhost:11434/v1`) in the Credentials Modal, the validator probes via the official SDK, and `{provider}_proxy` carries the URL into `OpenAIProvider`'s `base_url` at runtime. Identical mechanism, but the `base_url` resolves to the user's machine, not OpenAI cloud.-- Custom authentication proxies-- Development/testing with mock servers+## Architecture Patterns+- **Plugin-first (Wave 11).** One folder per plugin under [`server/nodes/<group>/<plugin>/`](./server/nodes/) rooted at `__init__.py`, subclassing `BaseNode` / `ActionNode` / `TriggerNode` / `ToolNode`. Auto-registers via `__init_subclass__`. See [`docs-internal/plugin_system.md`](./docs-internal/plugin_system.md).+- **Backend NodeSpec is the SSOT** for icon, colour, handles, params, output schema, uiHints, palette group. Frontend consumes via `useNodeSpec(type)` and adapts the JSON Schema → `INodeTypeDescription` shape through [`adapters/nodeSpecToDescription.ts`](./client/src/adapters/nodeSpecToDescription.ts) (legacy interface kept as a render contract; not a parallel schema system).+- **Component-driven frontend.** shadcn/ui primitives + Tailwind tokens; canvas nodes are spec-driven (see "Spec-driven component design" above).+- **State management.** TanStack Query owns server-backed data; Zustand (`useAppStore`, `nodeStatusStore`) owns UI state and slice-subscribed high-frequency push state. Slice-selector reads only — never whole-store destructure (see "Frontend Performance Architecture").+- **Execution pipeline.** Temporal-distributed activities for plugin execution; `WorkflowExecutor` for parallel orchestration; per-node retry / timeout / heartbeat declared on the plugin class.-## AI Chat Model Implementation Details+## File Structure Cleanup+**Removed Files:**+- `src/nodeDefinitions.ts` + `src/nodeDefinitions/` (27 files) — superseded by backend NodeSpec SSOT; frontend resolves specs via `lib/nodeSpec.ts` + `adapters/nodeSpecToDescription.ts`+- `src/nodeDefinitions.backup.ts` (backup file)+- `src/schemas/` directory (unused schema system)+- `src/utils/schemaParser.ts` (legacy parser)+- `src/utils/nodeSchemaParser.ts` (unused modern parser)+- `src/types/NodeSchema.ts` (legacy schema types)-### Component Architecture-AI chat model nodes (`openaiChatModel`, `anthropicChatModel`, `geminiChatModel`, etc.) render through `SquareNode` via the spec-driven `COMPONENT_BY_KIND['model']` dispatch in `Dashboard.tsx`. Per-provider visual config (icon, color, displayName, parameter schema) comes from the backend `NodeSpec` declared in `server/nodes/model/<provider>_chat_model.py`. Provider icons use `lobehub:<brand>` strings resolved by `resolveLibraryIcon` against `@lobehub/icons`; brand-specific SVGs (deepseek/kimi/mistral) live under `client/src/assets/icons/llm/` and are addressed via `asset:<key>`. There are no per-provider wrapper components anymore — adding a new provider takes a single Python file under `server/nodes/model/`.+**Cleaned Code:**+- Removed unused imports and dead functions+- Eliminated legacy NodeDefinition interface+- Streamlined parameter handling logic+- Maintained backward compatibility only where actively used-### API Key Management (`src/services/apiKeyManager.ts`)-- **Validation**: Uses LangChain for real API testing with provider-specific chat models-- **Storage**: localStorage with base64 encryption and key hashing for security-- **Models**: Automatic discovery and caching of available models per provider-- **Expiration**: 30-day validation period with automatic cleanup+## Testing & Validation+```bash+# Development server test+curl -I http://localhost:3001-### Execution Flow (`src/services/executionService.ts`)-- **Detection**: `isAIModelNode()` identifies AI chat models for Python routing-- **Enhancement**: `injectStoredApiKeys()` auto-injects stored credentials and models-- **Routing**: AI nodes → Python Flask backend, other nodes → Node.js backend-- **Logging**: Comprehensive debug output for API key injection and model selection-- **Supported Types**: `isNodeTypeSupported()` controls which nodes show Run button - includes AI models, agents, Android, WhatsApp, Twitter, Google Workspace, code executors, schedulers, utilities, and document processing nodes+# TypeScript validation+npx tsc --noEmit-### Parameter System Integration-- **Template Variables**: Support for `{{nodeId.output}}` syntax in all text parameters-- **Drag-and-Drop**: Visual parameter mapping from connected node outputs-- **Type-Specific**: Provider-specific parameters (OpenAI response format, Gemini safety settings)-- **Validation**: Real-time parameter validation with visual feedback+# Build verification+npm run build-### AI Thinking/Reasoning System-Extended thinking and reasoning capabilities for supported AI models. When enabled, the model's internal reasoning process is captured and available for downstream nodes.+# Test WebSocket connection for remote Android devices+python test_websocket.py+```-#### Supported Providers & Configuration+### WebSocket Testing (`test_websocket.py`)+- Tests health endpoint configured via `WEBSOCKET_URL` environment variable+- Tests stats endpoint+- Tests WebSocket connection+- Validates connection establishment and ping/pong messaging+- Comprehensive test results with pass/fail status-| Provider | Models | Parameter | Thinking Type | Notes |-|----------|--------|-----------|---------------|-------|-| **Claude** | All Claude 4.x/3.5 | `thinkingBudget` (1024-16000 tokens) | budget | Requires `max_tokens > budget_tokens`. Temperature auto-set to 1. |-| **Gemini** | gemini-3.x, gemini-2.5-pro/flash | `thinkingBudget` (token count) | budget | Uses `thinking_budget` API parameter |-| **OpenAI** | o1, o3, o3-mini, o4-mini | `reasoningEffort` (low/medium/high) | effort | Reasoning-only models. Temperature fixed at 1.0. |-| **OpenAI** | GPT-5.2/5.1/5/5-mini/5-nano | `reasoningEffort` (low/medium/high/xhigh) | effort | Hybrid reasoning: can operate with or without thinking. |-| **Groq** | qwen3-32b | `reasoningFormat` ('parsed' or 'hidden') | format | 'parsed' returns reasoning, 'hidden' returns only final answer |-| **Cerebras** | qwen-3-235b | `reasoningFormat` ('parsed' or 'hidden') | format | Same format-based reasoning as Groq Qwen |+## Production Deployment-#### Thinking/Reasoning Parameters-The thinking/reasoning fields (`thinkingEnabled`, `thinkingBudget`, `reasoningEffort`, `reasoningFormat`) live in the backend NodeSpec for each chat model (`server/nodes/models/<provider>.py`) and are surfaced through `AIChatModelParams` in `server/models/nodes.py`. The frontend renders them automatically via the universal parameter panel.+### Self-Deploy CLI (`machina deploy`) — current path+One command provisions a login-gated MachinaOs VM on a cloud provider. Two stages: the+operator's **cloud CLI** (gcloud; aws planned) handles auth + project/region/zone resolution ++ADC verification + API enablement, then **Terraform** (`cli/terraform/gcp/`) owns all resources —+VM (always named `machinaos`), firewall, artifact bucket (local `npm pack` source), service+account, and a cloud-init startup script that installs Node 22 + uv + the package and runs+`machina serve` under systemd. Login gate = built-in auth (`VITE_AUTH_ENABLED=true`,+`AUTH_MODE=single`) with the owner credential generated at deploy time and seeded on first boot.-#### Backend Implementation (`server/services/ai.py`)-The AI service extracts thinking content from LangChain response objects:--```python-def extract_thinking_from_response(response, provider: str) -> Optional[str]:- """Extract thinking/reasoning from AI response based on provider."""- # Claude: content_blocks with type='thinking'- # Gemini: response_metadata.candidates[0].content.parts with thought=True- # Groq: additional_kwargs.reasoning or response_metadata.reasoning- # OpenAI o-series: requires organization verification-```--**Response Structure:**-```python-{- "success": True,- "result": {- "response": "The final answer text",- "thinking": "The model's internal reasoning (if available)",- "model": "claude-3-5-sonnet-20241022",- "provider": "anthropic",- "finish_reason": "stop",- "timestamp": "2025-01-23T..."- }-}-```--#### Output Schema for Connected Nodes-The `thinking` field is available in Input Data & Variables for downstream nodes. This schema applies to all AI nodes including chat models and specialized agents.--**Source of truth (Wave 3, April 2026):** runtime output shapes live on the **backend**, not the frontend. Declared in Pydantic models at `server/services/node_output_schemas.py` and served lazy via `GET /api/schemas/nodes/{node_type}.json` / the `get_node_output_schema` WebSocket handler. The frontend's InputSection prefers real execution data, falls back to the backend schema, then an empty state. See [docs-internal/schema_source_of_truth_rfc.md](./docs-internal/schema_source_of_truth_rfc.md).--```python-# server/services/node_output_schemas.py-class AIAgentOutput(_OutputBase):- response: Optional[str] = None- thinking: Optional[str] = None # Available for drag-and-drop mapping- model: Optional[str] = None- provider: Optional[str] = None- finish_reason: Optional[str] = None- timestamp: Optional[str] = None--# Shared across every LLM-backed agent + chat model:-_AGENT_TYPES = ['aiAgent', 'chatAgent', 'android_agent', 'coding_agent',- 'web_agent', 'task_agent', 'social_agent', 'travel_agent',- 'tool_agent', 'productivity_agent', 'payments_agent',- 'consumer_agent', 'autonomous_agent', 'orchestrator_agent',- 'ai_employee', 'rlm_agent', 'claude_code_agent']-_CHAT_MODEL_TYPES = ['openaiChatModel', 'anthropicChatModel', 'geminiChatModel', ...]-NODE_OUTPUT_SCHEMAS = {- **{t: AIAgentOutput for t in _AGENT_TYPES},- **{t: AIAgentOutput for t in _CHAT_MODEL_TYPES},- # ...-}-```--Adding a new node type's output shape: define one Pydantic model, register it in `NODE_OUTPUT_SCHEMAS`. Zero frontend change.--#### UI Display (`client/src/components/ui/NodeOutputPanel.tsx`)-- **ThinkingBlock Component**: Collapsible display for thinking content-- **Default Expanded**: Thinking block is expanded by default when present-- **Provider-Aware**: Shows appropriate label based on provider (e.g., "Claude Extended Thinking")--#### Limitations-- **OpenAI o-series**: Reasoning summaries are only available to organizations that have completed verification at platform.openai.com. Without verification, `thinking` will be `null`.-- **Claude**: `max_tokens` must be greater than `thinkingBudget`. Temperature is automatically set to 1 when thinking is enabled.-- **Groq**: Only Qwen3-32b supports reasoning (QwQ removed from Groq). Format 'hidden' suppresses reasoning output.-- **Cerebras**: Qwen-3-235b supports format-based reasoning (same as Groq Qwen).--## AI Agent Node Architecture--### Agent Loop Termination-The standard agent path (`_run_agent_loop` in `services/ai.py`) routes purely on `response.tool_calls` — no custom iteration counter beyond `max_iterations`. Resolution precedence (highest → lowest):--1. Per-agent-node `parameters.max_iterations` (set on the agent node itself).-2. Per-user `UserSettings.agent_recursion_limit` (Settings tab, DB-backed).-3. Env `Settings.agent_recursion_limit` from `AGENT_RECURSION_LIMIT` (default **200**).-4. JSON fallback `llm_defaults.json:agent.recursion_limit` (legacy SSOT, kept as last-resort).--The token-based compaction threshold (`compaction_ratio` × model context_length, default 80% × ~200K for claude-sonnet-4-6 ≈ 160K) is the real termination signal: `_track_token_usage` runs after each agent turn, and when cumulative tokens cross the threshold it invokes `CompactionService.compact_context` to summarise the transcript before the next turn. Ratio precedence mirrors the recursion limit chain — per-session `custom_threshold` > `UserSettings.compaction_ratio` > `Settings.compaction_ratio` (env `COMPACTION_RATIO`, default 0.8) > JSON fallback.--If the iteration cap is ever hit anyway, `_run_agent_loop` appends a terminal `AIMessage` carrying a truncation note and returns `truncated=True` so `_extract_text_content` returns a usable partial response and the workflow continues — `_track_token_usage` and any post-loop persistence still run.--### Hot rebind after canvas mutation--`_run_agent_loop` accepts a `rebind_from_operations: Callable[[List[Dict]], Awaitable[List[StructuredTool]]]` callback. After each iteration's tool calls return, the loop inspects every result for an `operations` field (workflow_ops batch, today only `agentBuilder` emits this), invokes the callback, extends the local `current_tools` list, and re-calls `chat_model.bind_tools(current_tools)` so the LLM sees the new tools in the NEXT iteration — no Run-stop-Run cycle. The callback is wired by `execute_agent` / `execute_chat_agent` as a closure that calls `_build_tool_from_node` for each `add_node` op with `component_kind == "tool"` OR `usable_as_tool=True` (excluding chat models). Gated by `UserSettings.auto_rebind_tools_after_canvas_change` (default True); when off the closure is `None` and the LLM is told "Available on your next turn" in the operation summary.--For the F4.B Temporal path, the same rebind happens via the `agent.refresh_tools.v1` activity called from `AgentWorkflow.run` between iterations. See [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md).--### Spec-driven component design (Wave 10.D)--`AIAgentNode.tsx` knows nothing about specific agent types. It calls `useNodeSpec(type)` once and reads `handles`, `color`, `displayName`, `subtitle`, and `uiHints` from the cached `NodeSpec` ([`AIAgentNode.tsx:55-68`](./client/src/components/AIAgentNode.tsx#L55-L68)). The earlier 60-line `AGENT_CONFIGS` map (one entry per agent type, each with hardcoded `themeColorKey` + handle topology) was retired in Wave 10.D.--Component dispatch lives in [`Dashboard.tsx:74-105`](./client/src/Dashboard.tsx#L74-L105):--```typescript-const COMPONENT_BY_KIND: Record<string, React.ComponentType<any>> = {- start: StartNode,- trigger: TriggerNode,- agent: AIAgentNode, // every plugin with component_kind="agent" routes here- chat: AIAgentNode,- model: SquareNode,- square: SquareNode,- tool: SquareNode,- generic: SquareNode,-};--const createNodeTypes = () => {- const types: Record<string, React.ComponentType<any>> = {};- listCachedNodeSpecs().forEach(spec => {- const kind = spec.componentKind;- if (kind && COMPONENT_BY_KIND[kind]) {- types[spec.type] = COMPONENT_BY_KIND[kind];- } else if (spec.type === 'teamMonitor') {- types[spec.type] = TeamMonitorNode;- } else if ((spec.uiHints as any)?.isMasterSkillEditor === true) {- types[spec.type] = ToolkitNode;- } else {- types[spec.type] = SquareNode;- }- });- return types;-};-```--The dispatch keys off `spec.componentKind` (a backend-declared string), never `spec.type`. Specialized agent visuals (icon, color, handles, subtitle, width, height) all come from the spec. Icons live as `icon.svg` in the plugin folder (served via `/api/schemas/nodes/<type>/icon`); colors live as `meta.json` in the plugin folder (post-F2). [`server/nodes/visuals.json`](./server/nodes/visuals.json) is the fallback registry for emoji / `lobehub:<brand>` icons + the skill reverse-map.--### AI Agent vs Zeenie--| Feature | AI Agent | Zeenie |-|---------|----------|------------|-| Tool Calling | Yes (agent loop) | Yes (agent loop) |-| Memory Support | Yes | Yes |-| Skill Support | Yes | Yes |-| Task Input | Yes (input-task) | Yes (input-task) |-| Bottom Handles | Skill, Tools | Skill, Tools |-| Left Handles | Input, Memory, Task | Input, Memory, Task |-| Backend Method | `execute_agent()` | `execute_chat_agent()` |-| Async Delegation | Yes (fire-and-forget) | Yes (fire-and-forget) |--### Unified Tool Calling Architecture--Both AI Agent and Zeenie use the **same tool calling pattern**:--1. **Tool Building**: Both use `_build_tool_from_node()` to create schema-only tools-2. **Tool Execution**: Both use `execute_tool()` from `handlers/tools.py`-3. **Supported Tools**: `calculatorTool`, `currentTimeTool`, `duckduckgoSearch`, `androidTool`, `httpRequest`, `braveSearch`, `serperSearch`, `perplexitySearch`--**Tool Execution Flow:**-```-Tool Node (connected to input-tools)- ↓-Handler collects tool_data: {node_id, node_type, parameters, label, connected_services}- ↓-AIService._build_tool_from_node() → creates schema-only StructuredTool- ↓-_run_agent_loop binds tools via chat_model.bind_tools(tools)- ↓-LLM decides to call tool with arguments- ↓-tool_executor callback → execute_tool() from handlers/tools.py- ↓-Dispatch to handler: _execute_http_request(), _execute_calculator(), etc.-```--**Key Files:**-| File | Purpose |-|------|---------|-| `server/services/ai.py` | `_build_tool_from_node()` - builds schema-only tools for both agents |-| `server/services/handlers/tools.py` | `execute_tool()` - dispatches to specific handlers |-| `server/services/handlers/ai.py` | Collects `tool_data` from `input-tools` handle |--### Zeenie Input Methods--Zeenie accepts input in two ways:--1. **Template Variable (Explicit)**: Set the Prompt field to `{{chatTrigger.message}}` or `{{whatsappReceive.text}}`- - Templates are resolved by `ParameterResolver` before handler execution- - Supports nested paths: `{{nodeName.nested.field}}`--2. **Auto-Fallback (Implicit)**: Leave the Prompt field empty- - Handler detects nodes connected to `input-main` handle- - Reads output from `context.get('outputs', {}).get(source_node_id)`- - Extracts text from `message`, `text`, `content` fields (in order)- - Falls back to string representation of entire output--**Example Workflow:**-```-Chat Trigger → Zeenie ← HTTP Skill (SKILL.md context)- ← HTTP Request (tool node)-```--The Zeenie will:-- Load SKILL.md instructions from connected skill nodes-- Build tools from connected tool nodes (httpRequest, calculatorTool, etc.)-- Use `_run_agent_loop` for tool execution when tools are connected--### Backend Handlers-Wave 11: `handle_ai_agent` / `handle_chat_agent` were deleted. Agent execution now flows through `BaseNode.execute()` + `NodeContext.from_legacy()` via the node registry (`server/nodes/agent/`). Connection collection is handled by the agent node classes, which internally call `_collect_agent_connections()` to:-- Scans edges for nodes connected to `input-memory`, `input-skill`, `input-tools`, `input-main`/`input-chat` handles-- Returns a 4-tuple: `(memory_data, skill_data, tool_data, input_data)`-- Handles MasterSkill expansion into individual skill entries--Both call corresponding AI service methods:-- `AIService.execute_agent()` - Runs `_run_agent_loop` with tool binding-- `AIService.execute_chat_agent()` - Runs `_run_agent_loop` (when tools) or single `ainvoke` (no tools), with skills providing context + tools--### Async Agent Delegation (Nested Agents)--AI Agents can delegate tasks to other agents connected to their `input-tools` handle. This enables hierarchical agent architectures where a parent agent can spawn child agents that work independently.--**Two execution paths** depending on whether the parent runs under Temporal F4.B:--- **F4.B (default)**: parent `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow` when the child type is in `AGENT_WORKFLOW_TYPES`. Parent's `node_id` is passed in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent's canvas badge (parent badge advances live alongside child iteration counter instead of freezing). The LLM's `{task, context}` args travel as `child_context["invocation"]` — per-invocation input applied AFTER config resolution in `prepare_agent_payload` (`task` → system_message, `context`-or-`task` → prompt, same semantics as the legacy path), so stored node parameters never override the delegated task; empty-task calls are rejected at the call boundary without spawning. Deterministic child workflow id `f"{parent_workflow_id}-delegate-{child_node_id}-{iteration}"`. Parent waits for child result through the child-workflow handle — synchronous within the parent's loop.-- **Legacy / F4.A-only**: the fire-and-forget pattern below. Still the path for `rlm_agent` / `claude_code_agent` (excluded from `AGENT_WORKFLOW_TYPES`) and any deployment with `TEMPORAL_AGENT_WORKFLOW_ENABLED=false`.--**Architecture: Fire-and-Forget Pattern (legacy / F4.A)**-```-Parent Agent calls "delegate_to_ai_agent" tool- |-Tool handler spawns asyncio.Task for Child Agent- |-Returns immediately: {"status": "delegated", "task_id": "..."}- |-Parent Agent continues working- |-Child Agent executes independently in background- |-Child broadcasts its own status updates (executing, success, error)-```--**How It Works (legacy / F4.A):**-1. Connect a Child Agent (aiAgent/chatAgent) to Parent Agent's `input-tools` handle-2. Parent sees a tool like `delegate_to_ai_agent` with schema `{task: string, context?: string}`-3. When Parent calls the tool, handler spawns Child as `asyncio.create_task()`-4. Tool returns immediately with `{"status": "delegated", "task_id": "..."}`-5. Parent continues without waiting-6. Child executes with its own connected tools, skills, and memory-7. Both agents can execute simultaneously with independent status indicators--**Key Files:**-| File | Purpose |-|------|---------|-| `server/services/ai.py` | `DelegateToAgentSchema` in `_get_tool_schema()`, injects `ai_service`, `database`, `nodes`, `edges` into tool config |-| `server/services/handlers/tools.py` | `_execute_delegated_agent()` - spawns child as background task, `get_delegated_task_status()` utility |-| `server/services/handlers/ai.py` | Passes `context` to `execute_agent()`/`execute_chat_agent()` for nested delegation |--**Design Decisions:**-- **Memory Isolation**: Child uses its own connected memory, not shared with Parent-- **Error Isolation**: Child errors don't propagate to Parent - logged and broadcast independently-- **Task Tracking**: Background tasks tracked in `_delegated_tasks` dict, cleaned up on completion--### Specialized AI Agents--The system ships specialized agent variants — each is a folder under [`server/nodes/agent/`](./server/nodes/agent/) with `__init__.py` declaring a `SpecializedAgentBase` subclass. Authoritative list: glob that folder. Per-type display name / subtitle / description live on the plugin class; icon comes from `<plugin>/icon.svg` (or visuals.json emoji fallback); color comes from `<plugin>/meta.json` (post-F2). Do not maintain a hand-list of agent types in this doc — it drifts on every plugin add.--Standard handle topology (declared on `SpecializedAgentBase` via `std_agent_handles()`):-- **Left**: `input-main` (Input, 30%), `input-memory` (Memory, 55%), `input-task` (Task, 85%)-- **Bottom**: `input-skill` (Skill, 25%), `input-tools` (Tool, 75%)-- **Top**: `output-top` (Output)--**Team Lead Agents** (`orchestrator_agent`, `ai_employee`) declare an extra `input-teammates` handle on the bottom (50%) for delegation. They are listed in `TEAM_LEAD_TYPES` in [`client/src/components/TeamMonitorNode.tsx`](./client/src/components/TeamMonitorNode.tsx) (frontend tribal array — flagged as tech debt; see "remaining tribal arrays" note).--`AIAgentNode.tsx` is type-agnostic: it calls `useNodeSpec(type)` and renders whatever handles/icon/color the spec returns — no `AGENT_CONFIGS` map.--## Architecture Patterns-- **Plugin-first (Wave 11).** One folder per plugin under [`server/nodes/<group>/<plugin>/`](./server/nodes/) rooted at `__init__.py`, subclassing `BaseNode` / `ActionNode` / `TriggerNode` / `ToolNode`. Auto-registers via `__init_subclass__`. See [`docs-internal/plugin_system.md`](./docs-internal/plugin_system.md).-- **Backend NodeSpec is the SSOT** for icon, colour, handles, params, output schema, uiHints, palette group. Frontend consumes via `useNodeSpec(type)` and adapts the JSON Schema → `INodeTypeDescription` shape through [`adapters/nodeSpecToDescription.ts`](./client/src/adapters/nodeSpecToDescription.ts) (legacy interface kept as a render contract; not a parallel schema system).-- **Component-driven frontend.** shadcn/ui primitives + Tailwind tokens; canvas nodes are spec-driven (see "Spec-driven component design" above).-- **State management.** TanStack Query owns server-backed data; Zustand (`useAppStore`, `nodeStatusStore`) owns UI state and slice-subscribed high-frequency push state. Slice-selector reads only — never whole-store destructure (see "Frontend Performance Architecture").-- **Execution pipeline.** Temporal-distributed activities for plugin execution; `WorkflowExecutor` for parallel orchestration; per-node retry / timeout / heartbeat declared on the plugin class.--## File Structure Cleanup-**Removed Files:**-- `src/nodeDefinitions.ts` + `src/nodeDefinitions/` (27 files) — superseded by backend NodeSpec SSOT; frontend resolves specs via `lib/nodeSpec.ts` + `adapters/nodeSpecToDescription.ts`-- `src/nodeDefinitions.backup.ts` (backup file)-- `src/schemas/` directory (unused schema system)-- `src/utils/schemaParser.ts` (legacy parser)-- `src/utils/nodeSchemaParser.ts` (unused modern parser)-- `src/types/NodeSchema.ts` (legacy schema types)--**Cleaned Code:**-- Removed unused imports and dead functions-- Eliminated legacy NodeDefinition interface-- Streamlined parameter handling logic-- Maintained backward compatibility only where actively used--## Testing & Validation-```bash-# Development server test-curl -I http://localhost:3001--# TypeScript validation-npx tsc --noEmit--# Build verification-npm run build--# Test WebSocket connection for remote Android devices-python test_websocket.py-```--### WebSocket Testing (`test_websocket.py`)-- Tests health endpoint configured via `WEBSOCKET_URL` environment variable-- Tests stats endpoint-- Tests WebSocket connection-- Validates connection establishment and ping/pong messaging-- Comprehensive test results with pass/fail status--## Production Deployment--### Self-Deploy CLI (`machina deploy`) — current path-One command provisions a login-gated MachinaOs VM on a cloud provider. Two stages: the-operator's **cloud CLI** (gcloud; aws planned) handles auth + project/region/zone resolution +-ADC verification + API enablement, then **Terraform** (`cli/terraform/gcp/`) owns all resources —-VM (always named `machinaos`), firewall, artifact bucket (local `npm pack` source), service-account, and a cloud-init startup script that installs Node 22 + uv + the package and runs-`machina serve` under systemd. Login gate = built-in auth (`VITE_AUTH_ENABLED=true`,-`AUTH_MODE=single`) with the owner credential generated at deploy time and seeded on first boot.--```bash-machina deploy up --provider gcp --owner-email [email protected] # provision + install + print URL/creds-machina deploy status # URL + /health-machina deploy destroy # terraform destroy + clear state-```+```bash+machina deploy up --provider gcp --owner-email [email protected] # provision + install + print URL/creds+machina deploy status # URL + /health+machina deploy destroy # terraform destroy + clear state+```Key files: `cli/commands/serve.py` (single-port runtime: uvicorn fronts API + WS + built SPA, plusthe node sidecar), `cli/commands/deploy/` (verbs, secrets, Terraform driver, provider CLI adapters),@@ -2649,644 +1177,20 @@ The legacy `deploy.sh` (docker-compose images over SCP to a GCE box) was removedCompose notes below are retained for reference for the historical container topology.### Docker Deployment (legacy reference)-The project previously deployed using Docker Compose with nginx reverse proxy.--#### Docker Configuration (4-Container Stack)--**Services:**-| Container | Image | Port | Description |-|-----------|-------|------|-------------|-| redis | redis:7-alpine | 6379 | Cache and pub/sub for workflows |-| backend | machinaos-backend | 3010 | FastAPI Python backend |-| frontend | machinaos-frontend | 3000 | React app via nginx |-| whatsapp | machinaos-whatsapp | 5000 | Go WhatsApp bridge service |--**Frontend (`client/Dockerfile`):**-- Multi-stage build: Node.js builder → nginx:alpine production-- Serves static files via nginx on port 80 (mapped to 3000)-- Size: ~54 MB--**Backend (`server/Dockerfile`):**-- Python 3.12-slim base with Node.js 22.x for JS/TS execution-- Includes Playwright chromium for JS-rendered web scraping (crawleeScraper node)-- Includes persistent Node.js server (Express + tsx) on port 3020-- Optimized bytecode compilation (`python -O -m compileall`)-- Health check endpoint on port 3010-- Startup script (`start.sh`) runs both Python and Node.js servers (must have LF line endings, not CRLF)-- Depends on: redis, whatsapp-- Size: ~800 MB (includes Playwright chromium)--**WhatsApp (`docker/Dockerfile.whatsapp`):**-- Uses npm package `whatsapp-rpc` with pre-built binaries-- Node.js 20-alpine base with `npx whatsapp-rpc api --foreground`-- Binary downloaded from GitHub releases during npm postinstall-- Exposed on port 9400 (configurable via `PORT`, `WHATSAPP_RPC_PORT` env vars, or `--port` CLI flag)-- QR codes generated as base64 PNG in memory (no file I/O)-- Also published to PyPI as `whatsapp-rpc` (async Python client)-- Size: ~150 MB (includes Node.js runtime)--**Redis:**-- Official redis:7-alpine image-- Healthcheck: `redis-cli ping`-- Persistent volume: `redis_data`-- No authentication (internal network only)--**Development Compose (`docker-compose.yml`):**-```yaml-services:- # Redis uses profiles - only starts when REDIS_ENABLED=true- redis:- image: redis:7-alpine- profiles:- - redis # Only starts with --profile redis flag- ports: ["${REDIS_PORT:-6379}:6379"]- healthcheck:- test: ["CMD", "redis-cli", "ping"]-- backend:- build: ./server- ports: ["${PYTHON_BACKEND_PORT:-3010}:${PYTHON_BACKEND_PORT:-3010}"]- volumes:- - ./server:/app- - /app/nodejs/node_modules # Preserve Linux binaries (prevents Windows esbuild conflict)- depends_on:- whatsapp: { condition: service_healthy } # No Redis dependency- environment:- - REDIS_ENABLED=${REDIS_ENABLED:-false}- - REDIS_URL=redis://redis:6379-- frontend:- build: ./client- ports: ["${VITE_CLIENT_PORT:-3000}:${VITE_CLIENT_PORT:-3000}"]-- whatsapp:- build:- context: .- dockerfile: docker/Dockerfile.whatsapp- ports: ["${WHATSAPP_RPC_PORT:-9400}:${WHATSAPP_RPC_PORT:-9400}"]-```--**Docker Scripts Wrapper (`scripts/docker.js`):**-Auto-detects `REDIS_ENABLED` in `.env` and adds `--profile redis` flag when enabled:-```javascript-// Reads .env and checks REDIS_ENABLED value-function isRedisEnabled() {- const content = readFileSync(resolve(ROOT, '.env'), 'utf8');- const match = content.match(/^REDIS_ENABLED\s*=\s*(.+)$/m);- const value = match?.[1].trim().toLowerCase();- return value === 'true' || value === '1' || value === 'yes';-}--// Adds --profile redis when enabled-if (isRedisEnabled()) {- composeArgs.push('--profile', 'redis');-}-```--**Production Compose (`docker-compose.prod.yml`):**-```yaml-services:- redis:- image: redis:7-alpine- ports: ["6379:6379"]- volumes: ["redis_data:/data"]- healthcheck:- test: ["CMD", "redis-cli", "ping"]-- whatsapp:- build:- context: .- dockerfile: docker/Dockerfile.whatsapp- ports: ["9400:9400"]- healthcheck:- test: ["CMD", "wget", "-q", "--spider", "http://localhost:9400/health"]-- backend:- build: ./server- ports: ["3010:3010"]- depends_on:- redis: { condition: service_healthy }- whatsapp: { condition: service_healthy }- environment:- - REDIS_ENABLED=true- - REDIS_URL=redis://redis:6379-- frontend:- build: ./client- ports: ["3000:80"]-```--#### Nginx Configuration-Located at `/etc/nginx/sites-available/flow.zeenie.xyz`:-- Frontend: `/` → `http://127.0.0.1:3000`-- Backend API: `/api/` → `http://127.0.0.1:3010/api/`-- WebSocket: `/ws/` → `http://127.0.0.1:3010/ws/` (with upgrade headers)-- Webhook: `/webhook/` → `http://127.0.0.1:3010/webhook/`-- Health: `/health` → `http://127.0.0.1:3010/health`-- SSL via Let's Encrypt certbot--#### Environment Configuration--**Development** (`server/.env`):-- `DEBUG=true`-- `CORS_ORIGINS` includes localhost ports-- `REDIS_ENABLED=false` (uses SQLite cache for local dev)--**Production** (Docker environment variables):-- `DEBUG=false`-- `CORS_ORIGINS=["https://your-domain.com"]`-- `REDIS_ENABLED=true` (Docker Redis container)-- `REDIS_URL=redis://redis:6379`-- Environment set in `docker-compose.prod.yml`, not `.env` file--#### Frontend API URL Resolution-The frontend automatically detects production vs development:--```typescript-// client/src/config/api.ts-const isProduction = typeof window !== 'undefined' &&- !window.location.hostname.includes('localhost') &&- !window.location.hostname.includes('127.0.0.1');--return {- PYTHON_BASE_URL: isProduction ? '' : 'http://localhost:3010',-};-```--- **Production**: Empty base URL = relative URLs (same origin)-- **Development**: Explicit `http://localhost:3010`--WebSocket URL derived from base URL:-```typescript-// client/src/contexts/WebSocketContext.tsx-if (!baseUrl) {- // Production: use current origin- const wsProtocol = window.location.protocol === 'https:' ? 'wss' : 'ws';- return `${wsProtocol}://${window.location.host}/ws/status`;-}-```--#### Resource Usage (GCP e2-micro)-| Resource | Value |-|----------|-------|-| CPU | 2 cores (Intel Xeon @ 2.20GHz) |-| RAM | 1.9 GB total, ~820 MB used |-| Disk | 14 GB total, ~9.2 GB used |-| Backend Memory | ~144 MB |-| Frontend Memory | ~3.4 MB |--#### Useful Commands-```bash-# View logs (all containers)-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f'--# View specific service logs-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f backend'-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f whatsapp'--# Restart all services-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart'-# Restart specific service-ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart backend'--# Check container status-ssh $DEPLOY_HOST 'docker ps'--# Check resource usage-ssh $DEPLOY_HOST 'docker stats --no-stream'--# Check Redis connection-ssh $DEPLOY_HOST 'docker exec machinaos-redis-1 redis-cli ping'--# Check backend health (shows redis_enabled status)-curl -s https://$DEPLOY_DOMAIN/health | jq--# Clean up Docker resources (if disk full)-ssh $DEPLOY_HOST 'docker system prune -af && docker builder prune -af'-```--### Local Docker Development-For testing the full production stack locally:--```bash-# Build and start all containers-docker-compose -f docker-compose.prod.yml up --build--# Access locally-# Frontend: http://localhost:3000-# Backend API: http://localhost:3010-# WhatsApp RPC: http://localhost:9400-# Redis: localhost:6379--# Stop all containers-docker-compose -f docker-compose.prod.yml down--# Remove volumes (clean slate)-docker-compose -f docker-compose.prod.yml down -v-```--### Local Development Build-```bash-# Create optimized build-npm run build--# Serve built files locally-npm run preview-```+The historical Docker Compose topology (4-container stack: redis / backend / frontend / whatsapp, nginx reverse proxy, dev + prod compose files, env config, resource usage, useful commands) and the `npm run build` / `npm run preview` local-build commands are preserved in **[Deployment (legacy reference)](./docs-internal/deployment_legacy.md)**.## Authentication System-### Overview-n8n-inspired authentication system with JWT tokens stored in HttpOnly cookies. Authentication can be completely disabled for development or supports two deployment modes for different use cases.--### Authentication Toggle-| Setting | Environment Variable | Description |-|---------|---------------------|-------------|-| **Enabled** | `VITE_AUTH_ENABLED=true` | Require login (default) |-| **Disabled** | `VITE_AUTH_ENABLED=false` | Bypass authentication, anonymous access |--When `VITE_AUTH_ENABLED=false`:-- Frontend skips login page entirely-- User is set to anonymous with owner privileges-- No backend auth API calls are made-- Useful for local development and testing--### Deployment Modes (when auth enabled)-| Mode | Environment Variable | Description |-|------|---------------------|-------------|-| **Single Owner** | `AUTH_MODE=single` | First user becomes owner, registration disabled after |-| **Multi User** | `AUTH_MODE=multi` | Open registration for cloud deployments |--### Architecture-```-Frontend (LoginPage.tsx) → AuthContext → Backend (/api/auth/*) → JWT Cookie- ↓- AuthMiddleware- ↓- Protected Routes-```--### Backend Implementation--#### User Model (`server/models/auth.py`)-```python-class User(SQLModel, table=True):- __tablename__ = "users"- id: Optional[int] = Field(default=None, primary_key=True)- email: str = Field(unique=True, index=True)- password_hash: str- display_name: str- is_owner: bool = Field(default=False)- is_active: bool = Field(default=True)- created_at: datetime- last_login: Optional[datetime]-- def set_password(self, password: str) -> None:- # Uses bcrypt for secure hashing-- def verify_password(self, password: str) -> bool:- # Verifies against bcrypt hash-```--#### Auth Service (`server/services/user_auth.py`)-- `register_user()` - Creates new user, sets as owner if first user in single mode-- `authenticate_user()` - Validates credentials, returns user-- `create_token()` - Generates JWT token-- `verify_token()` - Validates JWT token-- `get_auth_status()` - Returns mode, registration availability, user count--#### Auth Router (`server/routers/auth.py`)-| Endpoint | Method | Description |-|----------|--------|-------------|-| `/api/auth/status` | GET | Get auth mode and registration status |-| `/api/auth/register` | POST | Register new user |-| `/api/auth/login` | POST | Login and set cookie |-| `/api/auth/logout` | POST | Clear auth cookie |-| `/api/auth/me` | GET | Get current user info |--#### Auth Middleware (`server/middleware/auth.py`)-Protects all routes except public paths:-```python-PUBLIC_PATHS = frozenset([- "/health", "/docs", "/openapi.json", "/redoc",- "/api/auth/status", "/api/auth/login", "/api/auth/register", "/api/auth/logout",-])-PUBLIC_PREFIXES = ("/webhook/",)-```--### Frontend Implementation--#### Auth Context (`client/src/contexts/AuthContext.tsx`)-```typescript-interface AuthContextValue {- user: User | null;- isAuthenticated: boolean;- isLoading: boolean;- error: string | null;- authMode: 'single' | 'multi';- canRegister: boolean;- login: (email: string, password: string) => Promise<boolean>;- register: (email: string, password: string, displayName: string) => Promise<boolean>;- logout: () => Promise<void>;- checkAuth: () => Promise<void>;-}-```--#### Protected Route (`client/src/components/auth/ProtectedRoute.tsx`)-Wraps protected content, shows LoginPage if not authenticated:-```typescript-const ProtectedRoute: React.FC<{ children: React.ReactNode }> = ({ children }) => {- const { isAuthenticated, isLoading } = useAuth();- if (isLoading) return <LoadingSpinner />;- if (!isAuthenticated) return <LoginPage />;- return <>{children}</>;-};-```+n8n-style JWT auth in HttpOnly cookies. `VITE_AUTH_ENABLED=false` bypasses login (anonymous owner, dev); when enabled, `AUTH_MODE=single` (first user = owner, registration then closed) or `multi` (open registration). Backend: `User` (bcrypt) + `UserAuthService` + `/api/auth/*` router + `AuthMiddleware` (public-path allowlist + `/webhook/` prefix). Frontend: `AuthContext` (TanStack-Query bootstrap with full-jitter backoff) + `ProtectedRoute` + `LoginPage`; all API calls send `credentials: 'include'`, and the WebSocket refuses to connect without the cookie.-#### Login Page (`client/src/components/auth/LoginPage.tsx`)-- Dracula-themed login/register form-- Switches between login and register based on `canRegister`-- Displays errors from auth context--### Configuration-Environment variables in `.env`:-```bash-# Authentication Toggle (frontend - Vite)-VITE_AUTH_ENABLED=true # 'true' or 'false' - disable to bypass login--# Authentication Mode (backend)-AUTH_MODE=single # 'single' or 'multi'-JWT_SECRET_KEY=your-secret-key-32 # Min 32 chars-JWT_EXPIRE_MINUTES=10080 # 7 days-JWT_COOKIE_NAME=machina_token-JWT_COOKIE_SECURE=false # true for HTTPS-JWT_COOKIE_SAMESITE=lax-```--### Race Condition Handling-The AuthContext includes retry logic with exponential backoff to handle the case where frontend starts before backend is ready:-- 5 retries with exponential backoff (1s, 2s, 4s, 8s, 16s)-- Shows "Failed to connect to server" only after all retries exhausted-- Prevents false "not authenticated" errors during startup--### Cookie-Based Auth for API Calls-All API calls must include `credentials: 'include'` for HttpOnly cookie:-```typescript-// In workflowApi.ts, all fetch calls include:-fetch(url, { credentials: 'include' })-```--### WebSocket Authentication-WebSocket checks cookie before accepting connection:-```python-# In websocket.py-token = websocket.cookies.get(settings.jwt_cookie_name)-if not token:- await websocket.close(code=4001, reason="Not authenticated")- return-```--WebSocketProvider only connects when authenticated:-```typescript-// In WebSocketContext.tsx-const { isAuthenticated, isLoading: authLoading } = useAuth();--useEffect(() => {- if (authLoading || !isAuthenticated) {- // Disconnect if logged out- return;- }- connect();-}, [isAuthenticated, authLoading]);-```--### Key Files-| File | Description |-|------|-------------|-| `client/src/config/api.ts` | API config with AUTH_ENABLED toggle |-| `client/src/contexts/AuthContext.tsx` | React auth state with retry logic |-| `client/src/components/auth/LoginPage.tsx` | Login UI |-| `client/src/components/auth/ProtectedRoute.tsx` | Route guard |-| `server/models/auth.py` | User SQLModel with bcrypt |-| `server/services/user_auth.py` | JWT creation/verification |-| `server/routers/auth.py` | REST endpoints |-| `server/middleware/auth.py` | Route protection |-| `server/core/config.py` | Settings with vite_auth_enabled field |--### Dependencies-```-# server/pyproject.toml-bcrypt>=4.1.0-pyjwt>=2.13.0-email-validator>=2.0.0-```--JWT handling uses **PyJWT** (`import jwt`, `jwt.encode` / `jwt.decode`, catch `jwt.PyJWTError`) — HS256 with `Settings.jwt_secret_key`. Do not reintroduce `python-jose`: it drags in pure-Python `ecdsa`, which carries an unpatchable Minerva timing-attack advisory (GHSA-wj6h-64fc-37mp).+**Load-bearing rule:** JWT handling uses **PyJWT** (`import jwt`, HS256 with `Settings.jwt_secret_key`). Do NOT reintroduce `python-jose` — it drags in pure-Python `ecdsa` with the unpatchable Minerva timing-attack advisory (GHSA-wj6h-64fc-37mp). Full reference (models, router, middleware, config, startup retry, key files, deps) in **[Authentication](./docs-internal/authentication.md)**.## Encrypted Credentials System-### Overview-API keys and OAuth tokens are stored in a separate encrypted database (`credentials.db`) using Fernet encryption (AES-128-CBC + HMAC-SHA256). Following the n8n pattern, the encryption key is derived from a server-scoped config key (`API_KEY_ENCRYPTION_KEY` in `.env`) using PBKDF2, initialized at startup and persisting across restarts.--### Security Architecture-```-Server Startup- ↓-API_KEY_ENCRYPTION_KEY (from .env) + Salt (from credentials.db)- ↓-PBKDF2HMAC (SHA256, 600K iterations)- ↓-Fernet Key (in-memory for application lifetime)- ↓-EncryptionService.encrypt()/decrypt()- ↓-credentials.db (encrypted ciphertext)-```--**Key Security Properties:**-- Server-scoped encryption key from `API_KEY_ENCRYPTION_KEY` in `.env` (n8n pattern)-- Key initialized at startup, persists across application lifetime-- Not tied to user sessions -- survives server restarts with valid JWT-- Salt stored in credentials database (not the main database)-- OWASP 2024 compliant: 600,000 PBKDF2 iterations--### Single Point of Access Pattern-**IMPORTANT**: All credential operations MUST go through `AuthService`. Routers should NEVER access `CredentialsDatabase` directly.--```python-# Correct: Use auth_service-auth_service = get_auth_service()-await auth_service.store_oauth_tokens(provider="google", ...)-tokens = await auth_service.get_oauth_tokens("google", customer_id="owner")--# Wrong: Direct database access-credentials_db = get_credentials_db() # Don't do this in routers-await credentials_db.save_oauth_tokens(...) # Don't do this-```--### Backend Implementation--#### EncryptionService (`server/core/encryption.py`)-```python-class EncryptionService:- """Fernet encryption with PBKDF2 key derivation."""-- def initialize(self, password: str, salt: bytes) -> None:- """Derive key from password using PBKDF2HMAC."""- kdf = PBKDF2HMAC(- algorithm=hashes.SHA256(),- length=32,- salt=salt,- iterations=600_000, # OWASP 2024 recommendation- )- key = base64.urlsafe_b64encode(kdf.derive(password.encode()))- self._fernet = Fernet(key)-- def encrypt(self, plaintext: str) -> str:- """Encrypt and return base64 ciphertext."""-- def decrypt(self, ciphertext: str) -> str:- """Decrypt base64 ciphertext."""-- def clear(self) -> None:- """Clear encryption key from memory."""-- def is_initialized(self) -> bool:- """Check if encryption is ready."""-```--#### CredentialsDatabase (`server/core/credentials_database.py`)-```python-class CredentialsDatabase:- """Async SQLite database for encrypted credentials."""-- async def initialize(self) -> bytes:- """Initialize database, create tables, return salt."""-- async def save_api_key(self, provider: str, key: str, metadata: Dict = None) -> bool- async def get_api_key(self, provider: str) -> Optional[str]- async def delete_api_key(self, provider: str) -> bool-- async def save_oauth_tokens(self, provider, access_token, refresh_token, ...) -> bool- async def get_oauth_tokens(self, provider, customer_id="owner") -> Optional[Dict]- async def delete_oauth_tokens(self, provider, customer_id="owner") -> bool-```--#### AuthService OAuth Methods (`server/services/auth.py`)-```python-class AuthService:- """Single point of access for all credentials."""-- # Memory-only cache for decrypted credentials- _api_key_cache: Dict[str, str] = {}- _oauth_cache: Dict[str, Dict[str, Any]] = {}+API keys and OAuth tokens live in a separate encrypted database (`credentials.db`): Fernet (AES-128-CBC + HMAC-SHA256) with the key derived via PBKDF2HMAC-SHA256 (600K iterations, OWASP-2024) from the server-scoped `API_KEY_ENCRYPTION_KEY` (`.env`) + a salt stored in `credentials.db`, initialized at startup and held for the process lifetime. Two distinct systems that never cross: API keys (`store_api_key`/`get_api_key` → `EncryptedAPIKey`) and OAuth tokens (`store_oauth_tokens`/`get_oauth_tokens` → `EncryptedOAuthToken`). Multi-backend (Fernet default / Keyring / AWS Secrets Manager) via `CREDENTIAL_BACKEND`.- async def store_api_key(self, provider: str, key: str) -> bool- async def get_api_key(self, provider: str) -> Optional[str]- async def delete_api_key(self, provider: str) -> bool-- async def store_oauth_tokens(self, provider, access_token, refresh_token, ...) -> bool- async def get_oauth_tokens(self, provider, customer_id="owner") -> Optional[Dict]- async def remove_oauth_tokens(self, provider, customer_id="owner") -> bool-- def clear_cache(self) -> None:- """Clear all cached credentials."""-```--#### UserAuthService Integration (`server/services/user_auth.py`)-```python-async def login(self, email: str, password: str):- # ... verify credentials ...- # Initialize encryption with user's password- await self._initialize_encryption(password)- return user, None--async def _initialize_encryption(self, password: str) -> None:- salt = await self.credentials_db.initialize()- self.encryption.initialize(password, salt)--def logout(self) -> None:- self.encryption.clear() # Clear key from memory-```--### Multi-Backend Support--For deployment flexibility, credentials can be stored in different backends:--#### Backend Options-| Backend | Use Case | Configuration |-|---------|----------|---------------|-| **Fernet** (default) | Local development, single-server | `CREDENTIAL_BACKEND=fernet` |-| **Keyring** | Desktop apps (OS-native storage) | `CREDENTIAL_BACKEND=keyring` |-| **AWS Secrets Manager** | Cloud deployments | `CREDENTIAL_BACKEND=aws` |--#### credential_backends.py-```python-class CredentialBackend(ABC):- async def store(self, key: str, value: str, metadata: Dict = None) -> bool- async def retrieve(self, key: str) -> Optional[str]- async def delete(self, key: str) -> bool- def is_available(self) -> bool--class FernetBackend(CredentialBackend):- """Default: Fernet-encrypted SQLite."""--class KeyringBackend(CredentialBackend):- """OS-native: Windows Credential Locker, macOS Keychain, Linux Secret Service."""- SERVICE_NAME = "MachinaOS"--class AWSSecretsBackend(CredentialBackend):- """AWS Secrets Manager for cloud deployments."""--def create_backend(settings, credentials_db=None) -> CredentialBackend:- """Factory with automatic fallback to Fernet."""-```--### Configuration--Environment variables in `.env`:-```bash-# Credentials Database-CREDENTIALS_DB_PATH=credentials.db--# Backend Selection-CREDENTIAL_BACKEND=fernet # fernet, keyring, or aws--# AWS Secrets Manager (when CREDENTIAL_BACKEND=aws)-AWS_SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789:secret:machinaos-creds-AWS_REGION=us-east-1-```--### Key Files-| File | Description |-|------|-------------|-| `server/core/encryption.py` | Fernet encryption with PBKDF2 key derivation |-| `server/core/credentials_database.py` | Async SQLite for encrypted credentials |-| `server/core/credential_backends.py` | Multi-backend abstraction (Fernet, Keyring, AWS) |-| `server/services/auth.py` | AuthService with OAuth methods (single point of access) |-| `server/services/user_auth.py` | Encryption initialization on login/logout |-| `server/core/config.py` | credential_backend, aws_secret_arn settings |--### Dependencies-```toml-# server/pyproject.toml-[project]-dependencies = [- "cryptography>=44.0.0", # Fernet encryption-]--[project.optional-dependencies]-keyring = ["keyring>=25.0.0"] # OS-native credential storage-aws = ["boto3>=1.34.0"] # AWS Secrets Manager-```--### Design Decisions-- **No Migration**: Users re-enter API keys after upgrade (simpler, more secure)-- **Memory-Only Cache**: Decrypted credentials never written to disk/Redis-- **Separate Database**: `credentials.db` isolated from main `machina.db`-- **Password-Derived Key**: Encryption key not stored anywhere-- **Single Point of Access**: AuthService prevents direct database access from routers+**Load-bearing rule:** every credential operation MUST go through `AuthService` (`from core.container import container; container.auth_service()`); routers must NEVER touch `CredentialsDatabase` directly. Full pipeline, cache contract, backends, config, key files, and design decisions are in **[Credentials Encryption](./docs-internal/credentials_encryption.md)**.## Example Workflows@@ -3407,507 +1311,23 @@ if "examples_loaded" not in columns:## Onboarding Service-### Overview-Multi-step welcome wizard that appears after first launch, guiding users through platform capabilities. Database-backed, skippable, resumable, and replayable from Settings.--See **[Onboarding Service](./docs-internal/onboarding.md)** for full documentation.--### Architecture-- **5-step wizard** using existing `Modal` component (autoHeight, maxWidth 580px, maxHeight 70vh) + Ant Design `Steps`, `Card`, `Button`, `Typography`, `Tag`-- **Database persistence** via `UserSettings.onboarding_completed` + `UserSettings.onboarding_step`-- **No new WebSocket handlers** -- reuses `get_user_settings` / `save_user_settings`-- **Existing users** auto-skip via migration (`examples_loaded=1` -> `onboarding_completed=1`)--### Steps--| Step | Component | Title | Purpose |-|------|-----------|-------|---------|-| 0 | `WelcomeStep` | Welcome to MachinaOs | Platform intro + feature highlights |-| 1 | `ConceptsStep` | Key Concepts | Nodes, Edges, Agents, Skills, Normal/Dev Mode |-| 2 | `ApiKeyStep` | API Key Setup | Provider list + "Open Credentials" button |-| 3 | `CanvasStep` | Canvas Tour | Visual UI layout diagram + keyboard shortcuts |-| 4 | `GetStartedStep` | Get Started | Example workflows, quick recipe, tips |--### Key Files-| File | Description |-|------|-------------|-| `client/src/hooks/useOnboarding.ts` | State hook with WebSocket persistence |-| `client/src/components/onboarding/OnboardingWizard.tsx` | Main wizard with Ant Design Steps |-| `client/src/components/onboarding/steps/*.tsx` | 5 step components using antd + @ant-design/icons |-| `client/src/Dashboard.tsx` | Renders wizard + passes `reopenTrigger` |-| `client/src/components/ui/SettingsPanel.tsx` | "Replay Welcome Guide" button in Help section |-| `server/models/database.py` | `UserSettings.onboarding_completed`, `onboarding_step` |-| `server/core/database.py` | Migration + CRUD for onboarding fields |--### Replay from Settings-- SettingsPanel has a "Replay Welcome Guide" button in the Help section-- Clicking it: closes Settings, increments `onboardingReopenTrigger` in Dashboard-- `useOnboarding` detects trigger change, resets state, reopens wizard from step 0--### Adding New Steps-1. Create `client/src/components/onboarding/steps/NewStep.tsx` using Ant Design components-2. Import in `OnboardingWizard.tsx`, add to `renderStep()` switch and `stepItems` array-3. Update `TOTAL_STEPS` in `useOnboarding.ts`+Multi-step welcome wizard shown on first launch — database-backed (`UserSettings.onboarding_completed` + `onboarding_step`), skippable, resumable, replayable from Settings, and auto-skipped for existing users (migration sets `onboarding_completed=1` where `examples_loaded=1`). Built with shadcn primitives (no antd). Full step list, key files, replay flow, and how to add a step are in **[Onboarding Service](./docs-internal/onboarding.md)**.## AI Chat Model Development Guide-### Adding a new AI provider (post-Wave-11)--A new chat-model provider is **a self-contained folder** under `server/nodes/model/<provider>_chat_model/` with `__init__.py` declaring a `ChatModelBase` subclass. The plugin auto-registers via `BaseNode.__init_subclass__`; the frontend renders it through `SquareNode` from the emitted NodeSpec without a single TS change.--```python-# server/nodes/model/openrouter_chat_model/__init__.py-class OpenRouterChatModel(ChatModelBase):- type = "openrouterChatModel"- metadata = NodeMetadata(- display_name="OpenRouter",- icon="lobehub:openrouter", # asset:<key>, lobehub:<brand>, or emoji- color="#6366F1",- component_kind="model", # routes to SquareNode in Dashboard.tsx- )-- class Params(ChatModelBase.Params):- # provider-specific overrides; everything else inherits from ChatModelBase- ...-```--The native chat path lives in `server/services/llm/providers/<provider>.py` (Protocol-based, see [Native LLM SDK](./docs-internal/native_llm_sdk.md)). The LangChain agent path uses `ChatOpenAI` with a custom `base_url` from `server/config/llm_defaults.json` -- no Python branching needed for OpenAI-compatible APIs.--Credentials live in `server/nodes/model/_credentials.py` (one `Credential` subclass per provider) and surface in the Credentials Modal automatically. There is no `nodeDefinitions/` to edit, no `ModelNode.tsx` to update, no `Dashboard.tsx` switch to extend.--### Key implementation files--| File | Purpose |-|---|---|-| `server/nodes/model/<provider>_chat_model.py` | Plugin entry — metadata + Params + auto-registers |-| `server/nodes/model/_credentials.py` | `Credential` subclass per provider |-| `server/config/llm_defaults.json` | base_url + supported_params + temperature constraints (no hardcoded URLs in Python) |-| `server/services/llm/providers/<provider>.py` | Native SDK provider (Protocol-based) |-| `server/services/ai.py` | Routing: native chat path + LangChain agent path |-| `client/src/Dashboard.tsx` | Generic `COMPONENT_BY_KIND` dispatch — no per-provider entry needed |+A new chat-model provider is one self-contained folder under `server/nodes/model/<provider>_chat_model/` with `__init__.py` declaring a `ChatModelBase` subclass (auto-registers via `BaseNode.__init_subclass__`; the frontend renders it through `SquareNode` from the NodeSpec with zero TS changes). Credentials live in `server/nodes/model/_credentials.py`. The full recipe — native chat path (`services/llm/providers/`), LangChain agent path, `llm_defaults.json` config, the dual-path routing, and the per-provider implementation-file map — is in **[Native LLM SDK](./docs-internal/native_llm_sdk.md)**.## Simple Memory System-### Overview-The Simple Memory node provides markdown-based conversation history storage for AI agents. It connects to the AI Agent's `input-memory` handle to provide context from previous conversations. Memory is stored in markdown format, visible and editable directly in the parameter panel, with optional long-term vector storage for semantic retrieval.--### Architecture-```-Simple Memory Node → (memory output) → AI Agent (input-memory handle)- ↓ ↓- Markdown editor Parses markdown, saves new exchanges- (visible in UI) Trims to window, archives to vector DB-```--### Memory Format-Conversation history is stored in markdown format with timestamps:-```markdown-# Conversation History--### **Human** (2025-01-30 14:23:45)-What is the weather like today?--### **Assistant** (2025-01-30 14:23:48)-I don't have access to real-time weather data...-```--### Key Features-- **Editable UI**: Conversation history visible in markdown editor in parameter panel-- **Window-Based Trimming**: Keeps last N message pairs, archives old messages-- **Long-Term Memory**: Optional vector DB storage for semantic retrieval of archived messages-- **Uses LangChain's InMemoryVectorStore**: Per-session vector stores with HuggingFaceEmbeddings (BAAI/bge-small-en-v1.5)--### Key Files-- **Node Definition**: `server/nodes/skill/simple_memory.py` - simpleMemory plugin (NodeSpec + execute). Carries an internal `last_session_id` field (hidden in UI) used by the claude_code_agent bridge for native session resume.-- **Memory Package**: `server/services/memory/` — promoted from the old `services/memory.py` so each concern owns its own module:- - `services/memory/markdown.py` — `parse_memory_markdown`, `append_to_memory_markdown`, `trim_markdown_window` (the canonical helpers every agent uses; **previously inlined into `services/ai.py`**, now imported from here)- - `services/memory/jsonl.py` — standalone JSONL `parse_jsonl` / `append_message` / `trim_window` (Anthropic Messages API shape, kept for future SDK migration; not used by any agent bridge today)- - `services/memory/vector_store.py` — `get_memory_vector_store` + per-session `InMemoryVectorStore` cache- - `services/memory/state.py` — `clear_agent_session_state(session_id, workflow_id, clear_long_term, memory_node_id)`; when `memory_node_id` is supplied it resets `memory_content` to the default placeholder AND wipes `last_session_id` server-side (single DB write)- - `services/memory/__init__.py` — re-exports the full surface so existing `from services.memory import …` callers keep working-- **AI Integration (aiAgent / chatAgent / rlm_agent)**: `server/services/ai.py` — `execute_agent` / `execute_chat_agent` consume `memory_data`, call the markdown helpers via the package re-exports-- **CLI bridge (`claude_code_agent`)**: `server/services/cli_agent/service.py:_persist_memory` — appends turns to `memory_content` via the same markdown helpers AND saves `simpleMemory.last_session_id` from the most recent successful run's `r.session_id`. Always broadcasts `node_parameters_updated` so the simpleMemory parameter panel refetches live (no page reload needed).-- **Edge walker**: `server/services/plugin/edge_walker.py:_build_memory_entry` — surfaces `memory_content` + `window_size` + `long_term_enabled` + `retrieval_count` + `last_session_id` to every consuming agent--### Node Properties-| Property | Type | Default | Description |-|----------|------|---------|-------------|-| sessionId | string | 'default' | Unique identifier for conversation session |-| windowSize | number | 10 | Number of message pairs to keep in short-term memory |-| memoryContent | string (markdown) | Initial template | Editable conversation history in markdown format |-| longTermEnabled | boolean | false | Archive old messages to vector DB for semantic retrieval |-| retrievalCount | number | 3 | Number of relevant memories to retrieve from long-term storage (shown when longTermEnabled=true) |--### Memory Flow-1. **Load**: AI Agent reads `memoryContent` markdown from connected Simple Memory node-2. **Parse**: `_parse_memory_markdown()` converts markdown to LangChain messages-3. **Retrieve** (if enabled): Semantic search of vector store for relevant context-4. **Execute**: AI processes prompt with conversation history-5. **Append**: New human/AI messages appended to markdown-6. **Trim**: `_trim_markdown_window()` keeps last N pairs, returns removed texts-7. **Archive**: Removed messages stored in InMemoryVectorStore (if longTermEnabled)-8. **Save**: Updated markdown saved back to node parameters--### Usage Flow-1. Add Simple Memory node to workflow-2. Connect its output to AI Agent's `input-memory` handle (bottom-left diamond)-3. Configure session ID (use different IDs for separate conversations)-4. Optionally enable long-term memory for semantic retrieval-5. Run AI Agent - it automatically reads/updates memory content-6. View/edit conversation history in the markdown editor--### Design Decisions-- **Passive Node**: Memory node has no Run button - AI Agent reads its configuration directly when executed-- **Markdown Storage**: Human-readable format, editable in UI, easy to debug-- **Per-Session Vector Stores**: `_memory_vector_stores` dict keyed by session_id-- **Window + Archive**: Short-term (recent messages) + long-term (semantic retrieval) memory pattern-- **Auto-Save**: AI Agent automatically saves updated markdown content to node parameters--### claude_code_agent bridge — `--continue` + warm subprocess pool (NOT markdown injection)--When `simpleMemory` is connected to `claude_code_agent`, memory continuity is handled by claude itself — the agent does NOT re-inject markdown as a system prompt. Claude stores per-session transcripts under `<CLAUDE_CONFIG_DIR>/projects/<project_key>/<session_id>.jsonl` where `project_key = re.sub(r"[^a-zA-Z0-9.-]", "-", str(cwd))` (verified against the on-disk `<DATA_DIR>/claude/projects/` listing; `CLAUDE_CONFIG_DIR = data_path("claude")`).--**The three coupled mechanisms** (see [docs-internal/cli_agent_framework.md → Memory bridge](./docs-internal/cli_agent_framework.md#memory-bridge--simplememory--claude_code_agent) for full plumbing):--1. **Stable cwd.** The pool keeps `cwd = repo_root` for every memory-bound spawn (no per-task worktree carving). Same cwd every run → claude's `project_key` is constant → `--continue` / `--resume <UUID>` resolve against the same JSONL across batches.-2. **`--continue` first, intra-process thereafter.** [`claude_code_agent.execute_op`](./server/nodes/agent/claude_code_agent/__init__.py) sets `spec.continue_session = bool(memory_data)` so the first cold spawn for a memory-wired run emits `--continue` (claude auto-loads the most recent conversation under the cwd's `project_key`, per [code.claude.com/docs/en/cli-reference](https://code.claude.com/docs/en/cli-reference)). Subsequent turns are written as stream-json to the SAME warm subprocess's `proc.stdin` — claude keeps the conversation in-process with the same `session_id` (verified: turn 1 + turn 2 share UUID within one process).-3. **`--resume <UUID>` crash recovery.** If the warm subprocess dies between batches, [`ClaudeSessionPool.acquire`](./server/nodes/agent/claude_code_agent/_pool.py) detects `process.returncode is not None`, captures the dead session's `current_session_uuid` (last seen on a `result` event), and respawns with `--resume <captured_uuid>` so the same on-disk JSONL keeps growing. Mutually exclusive with `--continue` in `nodes/agent/claude_code_agent/_provider.py:interactive_argv`.--**`memory_content` is the UI mirror, not the resume channel.** Claude reads its own JSONL via `--continue` / `--resume`. The markdown in `memory_content` is appended every successful run (same `append_to_memory_markdown` + `trim_markdown_window` helpers aiAgent uses) so the human reader sees the conversation. User edits to `memory_content` do not influence claude's next response.--**`simpleMemory.last_session_id` is display-only.** Pre-cutover code persisted the session UUID back to the memory node and emitted `--session-id <UUID5>` / `--resume <UUID>` on subsequent runs. That dance is gone — `claude_code_agent` no longer reads `last_session_id` from the memory data. The field is kept on the Pydantic params for back-compat and diagnostic display; clearing the memory wipes it.--**Parallel-batch guard.** When memory is wired AND `len(tasks) > 1`, `claude_code_agent` raises `NodeUserError` at handler entry — concurrent `--continue` spawns against one project_key would race claude's session-resolution.--**Live UI refresh — CloudEvents v1.0 envelope.** `_persist_memory` calls `broadcaster.broadcast_node_parameters_updated(...)` after `save_node_parameters`. The broadcast carries a `WorkflowEvent` envelope (`type: "com.machinaos.node.parameters.updated"`, `subject: <memory_node_id>`, `data: {node_id, parameters, version, source: "cli"}`) wrapped in the wire-key `{type: "node_parameters_updated", data: <envelope>}`. The FE handler in `client/src/contexts/WebSocketContext.tsx` casts to `WorkflowEvent<{node_id, parameters, version, source}>`, reads `inner.node_id || envelope.subject`, and routes the inner payload to `setNodeParameters` + `queryClient.setQueryData` so the simpleMemory parameter panel auto-refreshes the moment the turn completes. Pre-cutover the FE read the old flat top-level keys (`message.parameters`, `message.node_id`) and silently dropped every broadcast — symptom was "memory only updates after a page reload."--**Workspace dir routed via `--add-dir`.** The per-workflow workspace (`~/.machina/workspaces/<workflow_slug>/`, injected into `ctx.raw` by `workflow.py:_get_workspace_dir`) is spliced into each task's `add_dir` list in `AICliService.run_batch` so the spawned claude can read files dropped by upstream nodes (`fileDownloader`, `documentParser`, code executors). Without this the workspace is invisible: memory-bound runs spawn with `cwd=repo_root` (stable for `--continue`'s `project_key`) and non-memory runs with `cwd=worktree`, neither of which sees the workspace files. Mirrors the ai_agent pattern (`services/ai.py:1899` — `config['workspace_dir'] = context.get('workspace_dir', '')`) but uses claude's native `--add-dir` instead of MCP-tool-config injection because claude has its own filesystem tools.--**Stale-model coercion.** `ClaudeCodeAgentParams.model` is a `Literal` of supported model IDs + aliases — strict validation would reject legacy saved values like `"claude-sonnet-4.6"` (dot-spelled), `"claude-3-5-sonnet-20241022"` (date-suffixed), `""`. A `field_validator("model", "fallback_model", mode="before")` coerces unknown values to the default (`"claude-sonnet-4-6"` for `model`, `None` for `fallback_model`) so old workflows keep loading; the UI dropdown still constrains new edits.--**Strict tool allowlist — no claude built-ins (except conditional `Skill`), gated by `--permission-mode dontAsk`.** The spawn argv emits `--permission-mode dontAsk` (documented as *"only pre-approved tools, no prompts"* per [code.claude.com/docs/en/permission-modes](https://code.claude.com/docs/en/permission-modes)) so `--allowedTools` is actually enforced. The previous default `bypassPermissions` *"skips the permission layer entirely"* — same doc — which turned the allowlist into documentation-only and let claude's built-in `Read` / `Edit` / `Bash` / `Glob` / `Grep` / `Write` / `Skill` / `WebSearch` / `WebFetch` fire regardless of wiring. The new `--allowedTools` list contains exactly: (a) one `mcp__machinaos__<node_type>` per node wired through the agent's `input-tools` handle, (b) the claude built-in `Skill` **conditionally** — only when at least one skill is wired through `input-skill` (paired with the materialisation helper below; never enabled when no skill is connected), (c) the five MachinaOs MCP infrastructure tools (`getWorkspaceFiles`, `listSkills`, `getSkill`, `getCredential`, `broadcastLog`). Every filesystem / shell / web operation flows through wired MCP tools (`fileRead` / `fileModify` / `fsSearch` / `shell` / `browser` / `perplexitySearch`) — explicit wiring, no built-in escape hatches. Defaults live in [`server/config/ai_cli_providers.json`](./server/config/ai_cli_providers.json) (`default_allowed_tools: ""`, `default_permission_mode: "dontAsk"`); per-task override via `ClaudeTaskSpec.allowed_tools` is honored verbatim (no auto-merge).--**Skill materialisation — per-workflow workspace, live-watched, diff-based.** [`nodes/agent/claude_code_agent/_skills.py::materialise_skills`](./server/nodes/agent/claude_code_agent/_skills.py) writes connected-AND-enabled SKILL.md trees under `<workspace_dir>/.claude/skills/<name>/` where `workspace_dir = <DATA_DIR>/workspaces/<workflow_slug>/`. The workspace is passed via `--add-dir`, and claude scans `.claude/skills/` inside every `--add-dir` path per [the skills spec's "Automatic discovery from parent and nested directories" rule](https://code.claude.com/docs/en/skills#automatic-discovery-from-parent-and-nested-directories) — gives us per-workflow isolation (workflow A's skills can't bleed into workflow B's subprocess even when both spawn with `cwd=repo_root`). Pool tracks the materialised set in `PooledClaudeSession.materialised_skills: frozenset[str]`; on warm reuse, `acquire` calls `materialise_skills(workspace_dir, new_set, previous_skill_names=session.materialised_skills)` and the helper applies only the diff — `rmtree`'d skills disappear from claude's registry, new ones become invocable, all without respawning. Claude's [live filesystem watcher](https://code.claude.com/docs/en/skills#live-change-detection) picks up both edge events. Same helper drives both pool path and `AICliSession._pre_spawn` (non-pool) — uniform behaviour across transports. **Migration note**: pre-cutover the pool path wrote into `<repo_root>/.claude/skills/` so SKILL.md trees accumulated in the user's actual repo (gitignored but visible). Run `rm -rf .claude/skills/` once from the repo root to clean accumulated trees; future runs only write into per-workflow workspaces. We deliberately do NOT auto-prune because the repo's `.claude/` may contain user-authored skills outside the MachinaOs registry. **Architecture rationale**: surveyed alternatives — MCP `resources`/`prompts` require `@mention`/`/command` (not auto-loaded), hooks fire after skill discovery, `settings.json` doesn't expose per-session paths. Anthropic's own [Agent SDK Skills](https://code.claude.com/docs/en/agent-sdk/skills) uses the same filesystem-first pattern. Workspace-dir + live-watch is canonical.--**Warm-subprocess tool-surface rebind — no respawn cost.** Claude bakes the MCP bearer token into argv (`--mcp-config`) at spawn time and can't rotate without respawning. The pool stashes the spawn-time token on `PooledClaudeSession.batch_token` so subsequent batches can rebind the same persistent `BatchContext` in place. On warm reuse, `ClaudeSessionPool.acquire` calls [`rebind_batch(existing.batch_token, connected_tools=new_ctx.connected_tools, ...)`](./server/services/cli_agent/mcp_server.py) — diffs old vs new tool lists, calls `unexpose_workflow_tools(removed)` to decrement FastMCP refcounts (so a disconnected `duckduckgoSearch` falls off the tool list when its refcount hits zero), `expose_workflow_tools(added)` for newly-wired ones, then unregisters the redundant new token. `_terminate_locked` calls `unregister_batch(session.batch_token)` so refcounts drain on pool eviction / `clear` / `shutdown_all` (closes the long-standing leak where every pooled run permanently incremented every wired tool's refcount). The per-handler scope check in `workflow_tools._build_handler` reads `ctx.connected_tools` at call time, so even if claude's frozen `--allowedTools` argv still references a now-disconnected tool, the handler returns `{"error": "...not connected to this batch", "status": 403}` — double-lock against tool leakage across warm-reuse turns.--**Logs to grep on a live run:**-- `[Claude Code memory] memory_node=<id> -> --continue (claude auto-finds latest session under cwd)`-- `[ClaudeSessionPool] spawned new session memory_node=<id> pid=<N>` (cold spawn) or `warm reuse memory_node=<id> pid=<N> uuid=<UUID>` (intra-process turn)-- `[CC-Agent _persist_memory] saved memory_node=<id> last_session_id=<UUID> appended_turns=1 ... content_length=<N>`-- On tool-surface change between batches: `[CC-Agent MCP rebind_batch] node=<id> wf=<id> token=<8hex>... +<added> -<removed> kept=<n> (now <total> tools)` — confirms the in-place rebind ran instead of a respawn.-- On crash recovery: `[ClaudeSessionPool] dropping dead session memory_node=<id> ... — will respawn` followed by spawn with `--resume <UUID>` argv.+The Simple Memory node (`simpleMemory`, plugin `server/nodes/skill/simple_memory.py`) provides markdown-based conversation history on the agent's `input-memory` handle: window-based trimming, an editable markdown view, and optional long-term vector retrieval. Memory is loaded/parsed/appended/trimmed via the helpers in `server/services/memory/`. The full lifecycle (two storage formats, the markdown helper API, vector store, the `claude_code_agent` native-session bridge, clearing/resume) is the canonical **[Memory Lifecycle](./docs-internal/memory_lifecycle.md)**; token tracking + compaction thresholds are in **[Memory Compaction](./docs-internal/memory_compaction.md)**.## Memory Compaction, Token Tracking, and Cost Calculation-### Overview-The compaction service enables automatic memory compaction, token tracking, and **cost calculation** for all LLM providers. Cost is calculated using official pricing (per 1M tokens) and stored alongside token metrics. The Credentials Modal displays per-provider usage and costs.--### Architecture-```-AI Agent Execution- ↓-CompactionService.track() → PricingService.calculate_cost()- ↓ → Save token metrics + cost to DB- ↓ → Update cumulative state + cost- ↓ → Check if threshold exceeded- ↓-If needs_compaction:- - Anthropic: Native compaction via context_management API- - OpenAI: Native compaction via compact_threshold- - Others: Client-side summarization fallback- ↓-CompactionService.record() → Save compaction event to DB- → Reset cumulative token count-```--### Native Provider APIs--**Threshold strategy:** per-session `custom_threshold` > per-user `UserSettings.compaction_ratio` × context_length > env `Settings.compaction_ratio` (`COMPACTION_RATIO`, default 0.8) × context_length > JSON `llm_defaults.json` ratio × context_length.--**Anthropic SDK (tool_runner):**-```python-# Model-aware: threshold = compaction_ratio × model's context window-compaction_control = svc.anthropic_config(model="claude-opus-4.6", provider="anthropic")-# Returns: {"enabled": True, "context_token_threshold": 800000} (0.8 × 1M with default ratio)-```--**Anthropic Messages API:**-```python-api_config = svc.anthropic_api_config(model="claude-sonnet-4.5", provider="anthropic")-# Returns: {"betas": ["compact-2026-01-12"], "context_management": {"edits": [...]}}-# Threshold auto-computed from model's context window × compaction_ratio.-```--**OpenAI:**-```python-openai_config = svc.openai_config(model="gpt-5.2", provider="openai")-# Returns: {"context_management": {"compact_threshold": 320000}} (0.8 × 400K with default ratio)-```--### Key Files-| File | Description |-|------|-------------|-| `server/services/compaction.py` | CompactionService with model-aware thresholds, track(), record(), stats(), configure(), compact_context() methods |-| `server/services/model_registry.py` | ModelRegistryService providing context_length for model-aware threshold computation |-| `server/services/pricing.py` | PricingService with official pricing registry for all 6 providers |-| `server/services/ai.py` | `_track_token_usage()` with automatic compaction triggering |-| `server/models/database.py` | TokenUsageMetric, CompactionEvent, SessionTokenState tables (with cost fields) |-| `server/core/database.py` | CRUD methods for token metrics, compaction events, and `get_provider_usage_summary()` |-| `server/core/config.py` | compaction_enabled, compaction_threshold settings |-| `server/core/container.py` | Dependency injection for compaction_service |-| `server/main.py` | Wires AI service to compaction service at startup |-| `server/routers/websocket.py` | get_compaction_stats, configure_compaction, get_provider_usage_summary handlers |-| `client/src/hooks/useApiKeys.ts` | `getProviderUsageSummary()` hook method |-| `client/src/components/CredentialsModal.tsx` | Usage & Costs collapsible section per provider |--### Database Models--**TokenUsageMetric** - Per-execution token usage and cost:-```python-class TokenUsageMetric(SQLModel, table=True):- session_id: str # Memory session identifier- node_id: str # Agent node ID- provider: str # openai, anthropic, gemini, groq, cerebras, openrouter- model: str # Model name- input_tokens: int # Input token count- output_tokens: int # Output token count- total_tokens: int # Total tokens- cache_creation_tokens: int # Anthropic cache creation- cache_read_tokens: int # Anthropic cache read- reasoning_tokens: int # OpenAI o-series reasoning- # Cost fields (USD)- input_cost: float # Cost for input tokens- output_cost: float # Cost for output tokens- cache_cost: float # Cost for cache tokens- total_cost: float # Total cost-```--**SessionTokenState** - Cumulative state per session:-```python-class SessionTokenState(SQLModel, table=True):- session_id: str # Unique session identifier- cumulative_total: int # Running total tokens- custom_threshold: int # Per-session threshold override- compaction_count: int # Number of compactions- last_compaction_at: datetime # Last compaction timestamp- # Cumulative cost fields (USD)- cumulative_input_cost: float- cumulative_output_cost: float- cumulative_total_cost: float-```--**CompactionEvent** - Compaction history:-```python-class CompactionEvent(SQLModel, table=True):- session_id: str- trigger_reason: str # "native" or "threshold"- tokens_before: int- tokens_after: int- summary_content: str # Compacted summary (if available)-```--### CompactionService API--```python-from services.compaction import get_compaction_service--svc = get_compaction_service()--# Get model-aware provider config (threshold = compaction_ratio × context window)-anthropic_cfg = svc.anthropic_config(model="claude-opus-4.6", provider="anthropic")-# threshold: 800000 (0.8 × 1M context with default ratio)-openai_cfg = svc.openai_config(model="gpt-5.2", provider="openai")-# threshold: 320000 (0.8 × 400K context with default ratio)--# Or override with explicit threshold-anthropic_cfg = svc.anthropic_config(threshold=100000)--# Track token usage after AI execution (threshold auto-computed from model)-result = await svc.track(- session_id="user-123",- node_id="agent-1",- provider="anthropic",- model="claude-opus-4.6",- usage={"input_tokens": 5000, "output_tokens": 1000, "total_tokens": 6000}-)-# result: {"total": 6000, "total_cost": 0.021, "threshold": 800000, "needs_compaction": False} (80% of 1M)--# Record compaction event after native API handles it-await svc.record(- session_id="user-123",- node_id="agent-1",- provider="anthropic",- model="claude-opus-4.6",- tokens_before=805000,- tokens_after=15000,- summary="Compacted conversation summary..."-)--# Get session statistics (model-aware threshold when model/provider given)-stats = await svc.stats("user-123", model="claude-opus-4.6", provider="anthropic")-# {"session_id": "user-123", "total": 15000, "threshold": 800000, "count": 1} (80% of 1M)--# Configure per-session custom threshold (overrides model-aware threshold)-await svc.configure("user-123", threshold=50000, enabled=True)-```--### WebSocket Handlers--| Handler | Description |-|---------|-------------|-| `get_compaction_stats` | Get token statistics for a session |-| `configure_compaction` | Update threshold/enabled settings for a session |-| `get_provider_usage_summary` | Get aggregated usage and cost by provider for Credentials Modal |--### PricingService--Calculates cost based on official pricing (USD per 1M tokens):--```python-from services.pricing import get_pricing_service--pricing = get_pricing_service()--# Calculate cost for token usage-cost = pricing.calculate_cost(- provider="anthropic",- model="claude-3-5-sonnet",- input_tokens=5000,- output_tokens=1000,- cache_read_tokens=500-)-# cost: {"input_cost": 0.015, "output_cost": 0.015, "cache_cost": 0.00015, "total_cost": 0.03015}--# Get pricing for a model (supports partial matching)-pricing_info = pricing.get_pricing("anthropic", "claude-3-5-sonnet-20241022")-# pricing_info: ModelPricing(input_per_mtok=3.0, output_per_mtok=15.0, cache_read_per_mtok=0.30)-```--**Supported Providers & Pricing** (February 2026):-| Provider | Example Models | Input $/MTok | Output $/MTok |-|----------|---------------|--------------|---------------|-| OpenAI | gpt-5, gpt-4o, o3 | 1.25-15.00 | 10.00-60.00 |-| Anthropic | claude-opus-4.6, claude-sonnet-4 | 3.00-5.00 | 15.00-25.00 |-| Gemini | gemini-2.5-pro, gemini-2.0-flash | 0.10-1.25 | 0.40-10.00 |-| Groq | llama-3.3-70b, gpt-oss-120b | 0.05-0.59 | 0.08-0.79 |-| Cerebras | llama-3.1-70b | 0.10-0.60 | 0.10-0.60 |-| OpenRouter | Pass-through | Varies | Varies |--### Configuration--The compaction threshold + agent recursion limit are env-driven via `.env` (read by `core.config.Settings`); `server/config/llm_defaults.json` is the legacy SSOT and acts as last-resort fallback when Settings can't load:--```bash-# .env (mirrors .env.template)-COMPACTION_ENABLED=true # Enable/disable compaction globally-COMPACTION_RATIO=0.8 # Fraction of context window that triggers compaction (0.05-0.99)-AGENT_RECURSION_LIMIT=200 # Agent loop hard step cap (>=1)-```--```json-// server/config/llm_defaults.json — last-resort fallback only-"agent": {- "recursion_limit": 200,- "default_temperature": 0.7,- "compaction": { "ratio": 0.8 }-}-```--Effective threshold = `providers.<provider>.context_length.<model>` × `compaction_ratio` (e.g. claude-sonnet-4-6 → 1M × 0.8 ≈ 838K input tokens). When model/provider are unknown, `model_registry.get_context_length` falls through to the provider's `_default` entry in the same JSON.--**Threshold priority** (highest → lowest):-1. Per-session `SessionTokenState.custom_threshold` (WebSocket override).-2. Per-user `UserSettings.compaction_ratio` × context_length (Settings tab override).-3. Env `Settings.compaction_ratio` × context_length.-4. JSON `llm_defaults.json:agent.compaction.ratio` × context_length.+Automatic memory compaction, token tracking, and cost calculation for all LLM providers. After each agent turn, `CompactionService.track()` records token metrics + cost (via `PricingService`) and, when cumulative tokens cross the threshold, summarises the transcript (native Anthropic/OpenAI context-management APIs where available, client-side summarisation otherwise). Effective threshold = `compaction_ratio` x the model's context window.-Same precedence chain applies to `recursion_limit`: per-node `parameters.max_iterations` > `UserSettings.agent_recursion_limit` > `Settings.agent_recursion_limit` > JSON.--### Client-Side Compaction--When the token threshold is exceeded, the service automatically triggers compaction using the AI service to generate a structured summary:--```python-# Automatic compaction in _track_token_usage()-if tracking.get('needs_compaction') and memory_content and api_key:- result = await svc.compact_context(- session_id=session_id,- node_id=node_id,- memory_content=memory_content, # Current conversation markdown- provider=provider,- api_key=api_key,- model=model- )- # result: {"success": True, "summary": "# Conversation Summary (Compacted)...", "tokens_before": 105000, "tokens_after": 0}-```--**Summary Structure** (Claude Code pattern):-```markdown-# Conversation Summary (Compacted)-*Generated: 2025-02-12T10:30:00Z*--## Task Overview-What the user is trying to accomplish.--## Current State-What's been completed and what's in progress.--## Important Discoveries-Key findings, decisions, or problems encountered.--## Next Steps-What needs to happen next.--## Context to Preserve-Details that must be retained for continuity.-```--### WebSocket Broadcasts--Real-time updates broadcast to frontend:--| Event | Description |-|-------|-------------|-| `token_usage_update` | After each AI execution: `{session_id, data: {total, threshold, needs_compaction}}` |-| `compaction_starting` | Before compaction begins: `{session_id, node_id}` |-| `compaction_completed` | After compaction: `{session_id, success, tokens_before, tokens_after, error}` |--### Token Usage UI--The MiddleSection displays a Token Usage panel for memory nodes with:-- **Progress bar**: Visual tokens used vs threshold-- **Statistics**: Current token count, threshold, compaction count-- **Editable threshold**: Click edit icon to change per-session threshold--```typescript-// In client/src/components/parameterPanel/MiddleSection.tsx-<Collapse.Panel header="Token Usage" key="tokenUsage">- <Progress percent={Math.round((total / threshold) * 100)} />- <Statistic title="Tokens Used" value={`${total} / ${threshold}`} />- <Statistic title="Compactions" value={count} />- <InputNumber onChange={updateThreshold} /> {/* Edit threshold */}-</Collapse.Panel>-```--### Service Wiring--The compaction service requires AI service for summarization, wired at startup:--```python-# In server/main.py-from services.compaction import get_compaction_service-compaction_svc = container.compaction_service()-compaction_svc.set_ai_service(container.ai_service())-```+**Compaction-ratio precedence (highest->lowest): per-session `SessionTokenState.custom_threshold` > per-user `UserSettings.compaction_ratio` > env `COMPACTION_RATIO` (default 0.8) > `llm_defaults.json:agent.compaction.ratio`.**-### Design Decisions-- **Hybrid Approach**: Native APIs for Anthropic/OpenAI configs, client-side summarization for actual compaction-- **5-Section Summary**: Follows Claude Code's structured summary format for continuity-- **Automatic Triggering**: Compaction triggered in `_track_token_usage()` when threshold exceeded-- **Per-Session State**: Each memory session has independent token tracking and thresholds-- **Model-Aware Threshold**: `compaction_ratio` × model's context window. Default ratio 0.8 (env `COMPACTION_RATIO`, was 0.5 pre-2026.06). E.g. 800K for Claude Opus 4.8 with 1M context, 320K for GPT-5.2 with 400K context. Falls back to JSON `llm_defaults.json:agent.compaction.ratio` when Settings can't load. Per-session `custom_threshold` > per-user `UserSettings.compaction_ratio` > env Settings.-- **Singleton Pattern**: Service accessible via `get_compaction_service()`+Full reference — the `CompactionService` API, native vs client-side compaction, the 5-section summary format, DB models (`TokenUsageMetric` / `SessionTokenState` / `CompactionEvent`), WS handlers, and the broadcast events — in **[Memory Compaction](./docs-internal/memory_compaction.md)**; per-service API cost tracking is in **[Pricing Service](./docs-internal/pricing_service.md)**.## API Cost Tracking@@ -5306,7 +2726,7 @@ This function:- **Workflow Export/Import with Parameters**: Exported workflow JSON includes a `nodeParameters` field containing all node configuration (provider, model, prompt, skillsConfig, etc.) fetched from the database at export time. On import, embedded `nodeParameters` are saved back to the database. `sanitizeNodes()` in `workflowExport.ts` still strips `node.data` to UI-only fields (`label`, `disabled`, `condition`). A `parameterSanitizer.ts` utility exists for credential stripping but is currently disabled (pass-through). Old exports without `nodeParameters` import cleanly (backward compatible). Key files: `client/src/utils/workflowExport.ts`, `client/src/utils/parameterSanitizer.ts`, `client/src/Dashboard.tsx` (export/import handlers), `server/services/example_loader.py`.- **Skill System Architecture**: Skills organized in `server/skills/<folder>/` subfolders. Each folder appears in Master Skill dropdown. DB is source of truth for skill instructions (seeded from SKILL.md on first load). Icon resolution mirrors `BaseNode._metadata_dict`: per-plugin `<plugin>/icon.svg` (served as `/api/schemas/nodes/<type>/icon`) → `visuals.json` (emoji / `lobehub:<brand>`); color: per-plugin `<plugin>/meta.json` → `visuals.json`. Lives in [`skill_loader.py::_parse_skill_metadata`](./server/services/skill_loader.py). Native DOM keydown handler prevents React Flow from intercepting Ctrl shortcuts in skill editor. **Mutation broadcasts use the CloudEvents-typed `skill_lifecycle` wire key** with stages `created` / `updated` / `deleted` / `content_saved` — see "Plugin-folder location for plugin-specific CloudEvents factories" below.- **Example Workflows**: Auto-load example workflow seeds from `<repo>/.machina/workflows/` on first use (git-tracked; the only non-ignored content under `<repo>/.machina/`). Path resolved by `core.paths.example_workflows_dir()` — fixed, NOT under `DATA_DIR`, preserved by `machina clean` via `_MACHINA_KEEP`. Uses `UserSettings.examples_loaded` flag; supports anonymous users (`user_id="default"`); embedded `nodeParameters` saved to DB on import.-- **Onboarding Service**: 5-step welcome wizard (Welcome, Concepts, API Keys, Canvas Tour, Get Started) using Ant Design Steps/Card/Button/Typography. Database-backed via `UserSettings.onboarding_completed` + `onboarding_step`. Existing users auto-skip (migration marks `examples_loaded=1` as completed). Replayable from Settings "Help" section. No new WebSocket handlers needed. See [Onboarding Service](./docs-internal/onboarding.md) for details.+- **Onboarding Service**: 5-step welcome wizard (Welcome, Concepts, API Keys, Canvas Tour, Get Started) using shadcn primitives + lucide icons. Database-backed via `UserSettings.onboarding_completed` + `onboarding_step`. Existing users auto-skip (migration marks `examples_loaded=1` as completed). Replayable from Settings "Help" section. No new WebSocket handlers needed. See [Onboarding Service](./docs-internal/onboarding.md) for details.- **Node.js Code Executor**: Persistent Node.js server (Express + tsx) at port 3020 for JavaScript/TypeScript execution, replacing subprocess spawning per execution. Handlers in `server/services/handlers/code.py` call `NodeJSClient` which makes HTTP requests to the Node.js server. All config via environment variables (`NODEJS_EXECUTOR_URL`, `NODEJS_EXECUTOR_PORT`, etc.).- **writeTodos Tool Node**: Dedicated AI tool for task planning connecting to any agent's `input-tools` handle. `TodoService` singleton (`server/services/todo_service.py`) stores JSON-based per-session todo state keyed by workflow_id. Handler broadcasts `phase: "todo_update"` via WebSocket on each update for real-time UI; `formatTodoOutput()` in `OutputDisplayPanel.tsx` renders the result as a checklist with `[ ]` / `[~]` / `[x]` icons. Schema uses `TodoItem`/`TodoStatus` Pydantic enum. Skill at `server/skills/assistant/write-todos-skill/SKILL.md` teaches the plan-work-update loop.- **Temporal Activity Heartbeats**: Activities send `activity.heartbeat()` on every non-matching WebSocket broadcast inside the read loop in `services/temporal/activities.py`. This keeps long-running browser and claude_code_agent operations alive past the 2-minute `heartbeat_timeout`. Start/end heartbeats alone were causing `TIMEOUT_TYPE_HEARTBEAT` failures on ops taking 5-10 minutes. Connection config: `heartbeat=30`, `receive_timeout=540` (fits within 10-min `start_to_close_timeout`).diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.mdindex 6ba9a57f..2f19017a 100644--- a/docs-internal/TEMPORAL_ARCHITECTURE.md+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md@@ -12,6 +12,33 @@ Each workflow node executes as a **Temporal activity** with its own isolated con`rlm_agent`, `claude_code_agent` are intentionally excluded from AgentWorkflow — their externalised loops (RLM REPL / Claude CLI `--resume`) require single-process state continuity.+## Execution Routing & Running++`WorkflowService.execute_workflow` picks one of three executors in priority order (`server/services/workflow.py`):++1. If `TEMPORAL_ENABLED=true` and Temporal is configured → `_execute_temporal()` (the distributed path documented in this file).+2. Else if Redis is available → `_execute_parallel()` (local parallel orchestration via `WorkflowExecutor`).+3. Else → `_execute_sequential()` (single-threaded fallback).++**Running with Temporal** — the Temporal server and the embedded worker start automatically with every launch script:++```bash+npm run start # Starts Temporal server + all services+npm run dev # Starts Temporal server + all services (dev mode)+npm run stop # Stops all services including Temporal+```++The embedded Temporal worker runs **inside the Python backend** — registered in the `main.py` lifespan via `TemporalWorkerManager`, not as a separate process.++**Standalone worker** (for horizontal scaling — add more pollers against the same task queue):++```bash+cd server+python -m services.temporal.worker+```++This invokes `run_standalone_worker()` from `services/temporal/worker.py`.+## System Architecture```@@ -426,9 +453,12 @@ Trigger nodes that aren't the firing trigger are:```server/services/temporal/├── __init__.py # Exports TemporalExecutor, TemporalClientWrapper-├── activities.py # NodeExecutionActivities class+├── activities.py # NodeExecutionActivities class (legacy WebSocket round-trip path)│ ├── execute_node_activity() # Main activity method│ └── _execute_via_websocket() # WebSocket execution+├── plugin_activities.py # collect_plugin_activities() -> per-type node.{type}.v{ver} activities (F4.A)+├── agent_workflow.py # AgentWorkflow child workflow (F4.B agent loop)+├── agent_activities.py # collect_agent_activities() -> the 7 agent.*.v1 activities (F4.B)├── workflow.py # MachinaWorkflow class│ ├── run() # Main orchestrator│ ├── _filter_executable_graph() # Config node filtering@@ -525,6 +555,8 @@ One supervisor-build-time env var (read inside `_temporal_specs.py`, not in `Set## Debugging+The Web UI is at http://localhost:8080; the UI's HTTP API is served at http://localhost:8233.+```bash# Temporal Web UIopen http://localhost:8080diff --git a/docs-internal/agent_architecture.md b/docs-internal/agent_architecture.mdindex e413f747..1364a5d1 100644--- a/docs-internal/agent_architecture.md+++ b/docs-internal/agent_architecture.md@@ -41,19 +41,24 @@ WorkflowService.execute_node() server/services/workflow.pyvNodeExecutor._dispatch() server/services/node_executor.pyHandler registry lookup via functools.partial- Dispatches to handle_ai_agent() or handle_chat_agent()+ Dispatches via BaseNode.execute() to the agent plugin's execute_op()+ (server/nodes/agent/<plugin>/__init__.py — Wave 11 deleted handle_ai_agent /+ handle_chat_agent; agent logic moved into the plugin folder)|v-_collect_agent_connections() server/services/handlers/ai.py+collect_agent_connections() server/services/plugin/edge_walker.py+ (called by nodes/agent/_inline.prepare_agent_call)Scans edges where target == node_id- Groups by targetHandle into 4 buckets:- input-memory -> memory_data- input-skill -> skill_data[]- input-tools -> tool_data[]- input-main -> input_data+ Groups by targetHandle into 5 buckets (returns a 5-tuple+ memory_data, skill_data, tool_data, input_data, task_data):+ input-memory -> memory_data+ input-skill -> skill_data[]+ input-tools -> tool_data[]+ input-main / input-chat -> input_data+ input-task -> task_data|v-AIService.execute_agent() server/services/ai.py+AIService.execute_agent() / execute_chat_agent() server/services/ai.py1. Inject skills into system message2. Build LangChain StructuredTools from tool_data3. Call _run_agent_loop() (plain async while loop)@@ -76,7 +81,9 @@ Result broadcast via WebSocket| `server/routers/websocket.py` | WebSocket handler `handle_execute_node()` || `server/services/workflow.py` | Facade, builds context, delegates to NodeExecutor || `server/services/node_executor.py` | Handler registry, dispatches via `functools.partial` |-| `server/services/handlers/ai.py` | `_collect_agent_connections()`, `handle_ai_agent()`, `handle_chat_agent()` |+| `server/services/plugin/edge_walker.py` | `collect_agent_connections()` (the renamed `_collect_agent_connections`), `format_task_context()` |+| `server/nodes/agent/_inline.py` | `prepare_agent_call()` — task-context injection + auto-prompt fallback + teammate collection; calls `collect_agent_connections` then `ai_service.execute_[chat_]agent` |+| `server/nodes/agent/<plugin>/__init__.py` | Per-plugin `execute_op()` (Wave 11 replaced `handle_ai_agent` / `handle_chat_agent`) || `server/services/ai.py` | `AIService` -- `_run_agent_loop`, skill injection, tool building || `server/services/handlers/tools.py` | `execute_tool()` -- dispatch router for all tool types || `server/services/skill_loader.py` | `SkillLoader` -- filesystem/DB skill discovery and loading |@@ -159,7 +166,7 @@ Both build `initial_messages` (system + memory history + current prompt + skill### 1. Edge Scanning-In `_collect_agent_connections()` (`server/services/handlers/ai.py:14-49`), all edges targeting the agent node are scanned:+In `collect_agent_connections()` (`server/services/plugin/edge_walker.py`; the renamed Wave-11 successor to `_collect_agent_connections` — the old `handlers/ai.py` was deleted), all edges targeting the agent node are scanned:```pythonfor edge in edges:@@ -435,3 +442,82 @@ else:```This optimisation means tool-less Chat Agent conversations skip `_run_agent_loop` entirely for faster response times.++---++## AI Agent vs Chat Agent (Zeenie)++`aiAgent` and `chatAgent` (display name "Zeenie") are near-identical in capability — both run the same `_run_agent_loop`, support memory / skills / tools / task input, and can delegate asynchronously. The differences are which backend method they call and the softness of error handling.++| Feature | AI Agent (`aiAgent`) | Zeenie (`chatAgent`) |+|---------|----------------------|----------------------|+| Tool Calling | Yes (agent loop) | Yes (agent loop) |+| Memory Support | Yes | Yes |+| Skill Support | Yes | Yes |+| Task Input | Yes (`input-task`) | Yes (`input-task`) |+| Bottom Handles | Skill, Tools | Skill, Tools |+| Left Handles | Input, Memory, Task | Input, Memory, Task |+| Backend Method | `execute_agent()` | `execute_chat_agent()` |+| No-tool path | Always uses `_run_agent_loop` | Single `await chat_model.ainvoke(messages)` (skips loop overhead) |+| Tool-failure handling | Re-raises (loop wraps as `{"error": ...}` ToolMessages) | Returns `{"error": str(e)}` (softer) |+| Async Delegation | Yes (fire-and-forget) | Yes (fire-and-forget) |++See the full method comparison in [execute_agent vs execute_chat_agent](#execute_agent-vs-execute_chat_agent) above.++---++## Agent Input Methods++Both agents resolve their prompt the same way (logic in `server/nodes/agent/_inline.py:prepare_agent_call`):++1. **Template Variable (Explicit)** — set the Prompt field to a template like `{{chatTrigger.message}}` or `{{whatsappReceive.text}}`.+ - Templates are resolved by `ParameterResolver` before the plugin's `execute_op` runs.+ - Supports nested paths: `{{nodeName.nested.field}}`.++2. **Auto-Fallback (Implicit)** — leave the Prompt field empty (`prepare_agent_call` Step 2, lines ~85-93):+ - When `not parameters.get("prompt") and input_data`, the agent reads the output of the node wired to its `input-main` / `input-chat` handle (`input_data`, surfaced by `collect_agent_connections`).+ - Extraction order: `input_data["message"]` → `input_data["text"]` → `input_data["content"]` → `str(input_data)` (whole-output fallback).++**Task-context injection (Step 1)** runs before the prompt fallback: when an `input-task` edge supplies `task_data`, `format_task_context(task_data)` is prepended to the prompt and tools may be stripped (the agent is reacting to a completed delegation, not starting fresh).++**Example workflow:**+```+Chat Trigger → Zeenie ← HTTP Skill (SKILL.md context)+ ← HTTP Request (tool node)+```+Zeenie loads SKILL.md instructions from connected skill nodes, builds tools from connected tool nodes (httpRequest, calculatorTool, …), and uses `_run_agent_loop` for tool execution when tools are connected.++---++## Handle Topology++Handle layouts are declared in `server/nodes/agent/_handles.py` (local to `nodes/agent/`, not a global lookup). Two layouts cover the 20 agent variants:++**`std_agent_handles()`** — shared by 18 of 20 agent variants:++| Handle | Kind | Position | Offset | Role |+|---|---|---|---|---|+| `input-skill` | input | bottom | 25% | skill |+| `input-tools` | input | bottom | 75% | tools |+| `input-main` | input | left | 25% | main |+| `input-memory` | input | left | 50% | memory |+| `input-task` | input | left | 75% | task |+| `output-main` | output | right | 50% | main |+| `output-top` | output | top | — | main |++**`team_lead_agent_handles()`** — `orchestrator_agent` and `ai_employee` only; adds an `input-teammates` handle (bottom, 80%) and shifts the bottom skill/tool offsets to 20%/50%. Connected agents become `delegate_to_<type>` tools via `_collect_teammate_connections()`. Team leads are also listed in `TEAM_LEAD_TYPES` in [`client/src/components/TeamMonitorNode.tsx`](../client/src/components/TeamMonitorNode.tsx) (frontend tribal array, flagged as tech debt). See [agent_teams.md](agent_teams.md).++`AIAgentNode.tsx` is type-agnostic: it calls `useNodeSpec(type)` and renders whatever handles / icon / color the spec returns — there is no `AGENT_CONFIGS` map (retired Wave 10.D). Specialized-agent visuals all come from the NodeSpec; per-type display name / subtitle / description live on the plugin class, icon from `<plugin>/icon.svg` (visuals.json emoji fallback), color from `<plugin>/meta.json`. Glob `server/nodes/agent/` for the authoritative agent list — do not hand-maintain one.++---++## Async Agent Delegation (overview)++AI Agents delegate to other agents wired to their `input-tools` handle, enabling hierarchical agent trees. There are two execution paths depending on whether the parent runs under Temporal F4.B:++- **F4.B (default, `TEMPORAL_AGENT_WORKFLOW_ENABLED=true`)**: the parent `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow` when the child type is in `AGENT_WORKFLOW_TYPES`. The parent's `node_id` rides in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent's canvas badge. The LLM's `{task, context}` args travel as `child_context["invocation"]`, applied AFTER config resolution in `prepare_agent_payload` (so stored node parameters never override the delegated task; empty-task calls are rejected at the boundary without spawning). Deterministic child id `f"{parent_workflow_id}-delegate-{child_node_id}-{iteration}"`. The parent waits for the child result synchronously through the child-workflow handle.+- **Legacy / F4.A-only**: fire-and-forget via `asyncio.create_task`. Still the path for `rlm_agent` / `claude_code_agent` (excluded from `AGENT_WORKFLOW_TYPES`) and any deployment with `TEMPORAL_AGENT_WORKFLOW_ENABLED=false`. The `delegate_to_*` tool spawns the child as a background task and returns immediately with `{"status": "delegated", "task_id": "..."}`; the parent keeps working while the child executes independently and broadcasts its own status (executing / success / error).++Design decisions (legacy path): **memory isolation** (child uses its own connected memory, not the parent's), **error isolation** (child errors are logged + broadcast, never propagated to the parent), **task tracking** (background tasks live in `_delegated_tasks`, cleaned up on completion).++Key files: `server/services/ai.py` (`DelegateToAgentSchema` in `_get_tool_schema()`, injects `ai_service` / `database` / `nodes` / `edges` into tool config), `server/services/handlers/tools.py` (`_execute_delegated_agent()`, `get_delegated_task_status()`). Full plumbing — memory / parameter / execution-context flow — is in [agent_delegation.md](agent_delegation.md); the multi-agent team-lead variant is in [agent_teams.md](agent_teams.md).diff --git a/docs-internal/authentication.md b/docs-internal/authentication.mdnew file mode 100644index 00000000..8a4ab2e9--- /dev/null+++ b/docs-internal/authentication.md@@ -0,0 +1,239 @@+# Authentication System++## Overview+n8n-inspired authentication system with JWT tokens stored in HttpOnly cookies. Authentication can be completely disabled for development or supports two deployment modes for different use cases.++## Authentication Toggle+| Setting | Environment Variable | Description |+|---------|---------------------|-------------|+| **Enabled** | `VITE_AUTH_ENABLED=true` | Require login (default) |+| **Disabled** | `VITE_AUTH_ENABLED=false` | Bypass authentication, anonymous access |++When `VITE_AUTH_ENABLED=false`:+- Frontend skips login page entirely+- User is set to anonymous with owner privileges+- No backend auth API calls are made+- Useful for local development and testing++## Deployment Modes (when auth enabled)+| Mode | Environment Variable | Description |+|------|---------------------|-------------|+| **Single Owner** | `AUTH_MODE=single` | First user becomes owner, registration disabled after |+| **Multi User** | `AUTH_MODE=multi` | Open registration for cloud deployments |++## Architecture+```+Frontend (LoginPage.tsx) → AuthContext → Backend (/api/auth/*) → JWT Cookie+ ↓+ AuthMiddleware+ ↓+ Protected Routes+```++## Backend Implementation++### User Model (`server/models/auth.py`)+```python+class User(SQLModel, table=True):+ __tablename__ = "users"+ id: Optional[int] = Field(default=None, primary_key=True)+ email: str = Field(unique=True, index=True)+ password_hash: str+ display_name: str+ is_owner: bool = Field(default=False)+ is_active: bool = Field(default=True)+ created_at: datetime+ last_login: Optional[datetime]++ def set_password(self, password: str) -> None:+ # Uses bcrypt for secure hashing++ def verify_password(self, password: str) -> bool:+ # Verifies against bcrypt hash+```++### Auth Service (`server/services/user_auth.py`)+- `register_user()` - Creates new user, sets as owner if first user in single mode+- `authenticate_user()` - Validates credentials, returns user+- `create_token()` - Generates JWT token+- `verify_token()` - Validates JWT token+- `get_auth_status()` - Returns mode, registration availability, user count++`UserAuthService` is also the integration point for the encrypted-credentials+system: `login()` calls `_initialize_encryption(password)` (derives the Fernet+key via PBKDF2 from the user's password + the credentials DB salt) and+`logout()` calls `self.encryption.clear()` to wipe the key from memory. See+[Credentials Encryption](./credentials_encryption.md) for the full pipeline.++### Auth Router (`server/routers/auth.py`)+| Endpoint | Method | Description |+|----------|--------|-------------|+| `/api/auth/status` | GET | Get auth mode and registration status |+| `/api/auth/register` | POST | Register new user |+| `/api/auth/login` | POST | Login and set cookie |+| `/api/auth/logout` | POST | Clear auth cookie |+| `/api/auth/me` | GET | Get current user info |++`routers/auth.py` defines its own local `get_auth_service()` helper+(`return container.auth_service()`) — `get_auth_service` is NOT exported from+`services/auth.py`. The same local-helper pattern is repeated in+`routers/twitter.py`, `routers/google.py`, and `routers/websocket.py`. In+handlers/services use `from core.container import container; auth = container.auth_service()`.++### Auth Middleware (`server/middleware/auth.py`)+Protects all routes except public paths:+```python+PUBLIC_PATHS = frozenset([+ "/health", "/docs", "/openapi.json", "/redoc",+ "/api/auth/status", "/api/auth/login", "/api/auth/register", "/api/auth/logout",+ "/ws/internal", # Internal WebSocket for Temporal workers+])++# Path prefixes that are public. ``/mcp/`` is the CLI-agent MCP server, which+# enforces its own per-batch bearer-token auth (cookies don't apply to it), so+# it bypasses this cookie gate.+PUBLIC_PREFIXES = ("/webhook/", "/mcp/")+```++The middleware also lets the static SPA shell, built assets, and client-side+routes load BEFORE login (the SPA renders the login page itself) when the+container serves the client on a single port.++## Frontend Implementation++### Auth Context (`client/src/contexts/AuthContext.tsx`)+```typescript+interface AuthContextValue {+ user: User | null;+ isAuthenticated: boolean;+ isLoading: boolean;+ error: string | null;+ authMode: 'single' | 'multi';+ canRegister: boolean;+ login: (email: string, password: string) => Promise<boolean>;+ register: (email: string, password: string, displayName: string) => Promise<boolean>;+ logout: () => Promise<void>;+ checkAuth: () => Promise<void>;+}+```++### Protected Route (`client/src/components/auth/ProtectedRoute.tsx`)+Wraps protected content, shows LoginPage if not authenticated:+```typescript+const ProtectedRoute: React.FC<{ children: React.ReactNode }> = ({ children }) => {+ const { isAuthenticated, isLoading } = useAuth();+ if (isLoading) return <LoadingSpinner />;+ if (!isAuthenticated) return <LoginPage />;+ return <>{children}</>;+};+```++### Login Page (`client/src/components/auth/LoginPage.tsx`)+- Dracula-themed login/register form+- Switches between login and register based on `canRegister`+- Displays errors from auth context++## Configuration+Environment variables in `.env`:+```bash+# Authentication Toggle (frontend - Vite)+VITE_AUTH_ENABLED=true # 'true' or 'false' - disable to bypass login++# Authentication Mode (backend)+AUTH_MODE=single # 'single' or 'multi'+JWT_SECRET_KEY=your-secret-key-32 # Min 32 chars+JWT_EXPIRE_MINUTES=10080 # 7 days+JWT_COOKIE_NAME=machina_token+JWT_COOKIE_SECURE=false # true for HTTPS+JWT_COOKIE_SAMESITE=lax+```++`core/config.py` carries the `vite_auth_enabled` field (required because+Pydantic Settings uses `extra="forbid"`).++## Race Condition Handling (TanStack Query bootstrap)+The frontend starts before the backend is ready during cold launch, so the+auth-status check must tolerate transient failures.++The `AuthContext` bootstraps the auth-status check through TanStack Query+(`useQuery({ queryKey: AUTH_STATUS_QUERY_KEY, queryFn: fetchAuthStatus, retry, retryDelay, signal })`),+which replaced the previous recursive `setTimeout` retry chain. Behaviour:++- **Full-jitter exponential backoff** per the AWS Architecture Blog formula:+ `random(0, min(CAP_MS, BASE_MS * 2^attempt))`. Constants live in+ [`client/src/lib/connectionConfig.ts`](../client/src/lib/connectionConfig.ts)+ under `AUTH_RETRY`: `BASE_MS = 50`, `CAP_MS = 4000`, `MAX_ATTEMPTS = 7`.+ Cumulative budget is ~10 s (vs. the old ~31 s); sub-second granularity early+ covers the typical 4 s backend cold-start window in 4–5 attempts.+- **401/403 short-circuit the retry chain** — those are valid responses meaning+ "auth disabled / not logged in", not "backend unavailable", so no retry+ budget is burned (`authShouldRetry` returns `false` when the wrapped error+ message contains `HTTP 401` / `HTTP 403`).+- **AbortController `signal`** is plumbed through `queryFn` so unmount + React+ Strict Mode cleanup cancel in-flight requests automatically.+- `login` / `register` / `logout` invalidate the cache via+ `queryClient.invalidateQueries({ queryKey: AUTH_STATUS_QUERY_KEY })`+ (`AUTH_STATUS_QUERY_KEY = ['auth', 'status']`).++> Historical note: the original implementation used 5 fixed retries with+> exponential backoff (1 s, 2 s, 4 s, 8 s, 16 s) and a recursive `setTimeout`+> chain, surfacing "Failed to connect to server" only after all retries were+> exhausted. This was superseded by the TanStack Query bootstrap above.++## Cookie-Based Auth for API Calls+All API calls must include `credentials: 'include'` for the HttpOnly cookie:+```typescript+// In workflowApi.ts, all fetch calls include:+fetch(url, { credentials: 'include' })+```++## WebSocket Authentication+WebSocket checks the cookie before accepting the connection:+```python+# In websocket.py+token = websocket.cookies.get(settings.jwt_cookie_name)+if not token:+ await websocket.close(code=4001, reason="Not authenticated")+ return+```++`WebSocketProvider` only connects when authenticated:+```typescript+// In WebSocketContext.tsx+const { isAuthenticated, isLoading: authLoading } = useAuth();++useEffect(() => {+ if (authLoading || !isAuthenticated) {+ // Disconnect if logged out+ return;+ }+ connect();+}, [isAuthenticated, authLoading]);+```++## Key Files+| File | Description |+|------|-------------|+| `client/src/config/api.ts` | API config with AUTH_ENABLED toggle |+| `client/src/contexts/AuthContext.tsx` | React auth state with TanStack Query bootstrap + retry logic |+| `client/src/lib/connectionConfig.ts` | `AUTH_RETRY` backoff constants (`BASE_MS` / `CAP_MS` / `MAX_ATTEMPTS`) |+| `client/src/components/auth/LoginPage.tsx` | Login UI |+| `client/src/components/auth/ProtectedRoute.tsx` | Route guard |+| `server/models/auth.py` | User SQLModel with bcrypt |+| `server/services/user_auth.py` | JWT creation/verification + encryption init on login/logout |+| `server/routers/auth.py` | REST endpoints |+| `server/middleware/auth.py` | Route protection (`PUBLIC_PATHS` / `PUBLIC_PREFIXES`) |+| `server/core/config.py` | Settings with `vite_auth_enabled` field |++## Dependencies+```+# server/pyproject.toml+bcrypt>=4.1.0+pyjwt>=2.13.0+email-validator>=2.0.0+```++JWT handling uses **PyJWT** (`import jwt`, `jwt.encode` / `jwt.decode`, catch+`jwt.PyJWTError`) — HS256 with `Settings.jwt_secret_key`. Do **not** reintroduce+`python-jose`: it drags in pure-Python `ecdsa`, which carries an unpatchable+Minerva timing-attack advisory (GHSA-wj6h-64fc-37mp).diff --git a/docs-internal/claude_code_agent.md b/docs-internal/claude_code_agent.mdnew file mode 100644index 00000000..48229772--- /dev/null+++ b/docs-internal/claude_code_agent.md@@ -0,0 +1,49 @@+# Claude Code Agent — Documentation Hub++`claude_code_agent` is a MachinaOs agent node that drives the real `claude` CLI as a+long-lived **interactive** subprocess (stdio pipes, no PTY) over the VSCode-extension+stream-json protocol — not the `claude -p` headless path. A warm `ClaudeSessionPool`+(keyed by the connected `simpleMemory.node_id`) preserves the session across turns;+memory continuity is claude-native via `--continue` / `--resume <UUID>` on a stable+`cwd=repo_root`. Connected tools and skills reach claude through an MCP bridge; the spawn+runs under `--permission-mode dontAsk` with a strict `--allowedTools` allowlist and stays+in interactive billing (entrypoint `claude-vscode`, not `sdk-cli`).++This page is a **router**. The detail lives in the documents below — start here, then jump.++## Where to read what++| Your question | Read |+|---|---|+| How does the whole integration work end-to-end? (architecture, memory & context compaction, sub-agent creation, data structures) | [claude_code_agent_architecture.md](./claude_code_agent_architecture.md) |+| How does interactive mode work? (stdio pipes vs PTY, stream-json events, `system/init` / `assistant` / `result`, session continuity, the four `claude.session.*` CloudEvents) | [claude_code_interactive_mode.md](./claude_code_interactive_mode.md) |+| The generic multi-provider runtime (`AICliService.run_batch`, worktree isolation, FastMCP bridge, the memory bridge `--continue`/`--resume`, plugin-folder layout) | [cli_agent_framework.md](./cli_agent_framework.md) |+| Does our implementation match the official Claude Code spec? (six invariants + status) | [cli_agent_canonical_patterns_rfc.md](./cli_agent_canonical_patterns_rfc.md) |+| Which CLI subcommands/flags exist, and which subset we emit | [claude_code_cli_reference.md](./claude_code_cli_reference.md) (snapshot) |+| Environment variables (`CLAUDE_CONFIG_DIR`, `MAX_MCP_OUTPUT_TOKENS`, `ENABLE_TOOL_SEARCH`, Bedrock/Vertex, telemetry) | [claude_code_env_vars_reference.md](./claude_code_env_vars_reference.md) (snapshot) |+| Permission-mode semantics (`default` / `acceptEdits` / `plan` / `dontAsk` / `bypassPermissions`); why MachinaOs uses `dontAsk` | [claude_code_permission_modes_reference.md](./claude_code_permission_modes_reference.md) (snapshot) |+| Headless / print mode (`claude -p`, `--output-format`, stream-json event schema) — the schema the pool parses off stdout | [claude_code_headless_reference.md](./claude_code_headless_reference.md) (snapshot) |+| SKILL.md frontmatter spec, discovery paths, `context: fork` — what we materialise skills against | [claude_code_skills_reference.md](./claude_code_skills_reference.md) (snapshot) |++The four `cli-reference` / `env-vars` / `permission-modes` / `headless` / `skills`+documents are **verbatim snapshots** of code.claude.com docs (fetched 2026-05-11), kept so+the contract the pool parses is pinned even if upstream changes.++## Plugin folder — [server/nodes/agent/claude_code_agent/](../server/nodes/agent/claude_code_agent/)++All claude-specific code is self-contained here; the generic framework at+`server/services/cli_agent/` imports nothing from `nodes/`.++| File | Responsibility |+|---|---|+| `__init__.py` | `ClaudeCodeAgentNode(SpecializedAgentBase)` + `Params`/`Output`; self-registers via `factory.py` (`register_provider` / `register_session_pool` / `register_skill_materialiser`) + `register_ws_handlers`. |+| `_provider.py` | Builds the spawn argv (`--output-format stream-json --input-format stream-json --verbose --ide`, `--permission-mode dontAsk`, `--allowedTools`, `--continue`/`--resume`). |+| `_pool.py` | `ClaudeSessionPool` — warm subprocess keyed by `simpleMemory.node_id`; stdout reader, session-UUID capture, crash-recovery respawn. |+| `_skills.py` | `materialise_skills` — writes connected SKILL.md trees under `<workspace>/.claude/skills/`, diff-based on warm reuse. |+| `_oauth.py` | Isolated `CLAUDE_CONFIG_DIR` (`<DATA_DIR>/claude/`), browser-OAuth bridge, binary install. |+| `_handlers.py` | WebSocket handlers (`cli_login` / `cli_auth_status`). |++## See also++- Memory bridge specifics (how `simpleMemory` feeds `--continue`, why `last_session_id` is display-only): [cli_agent_framework.md → Memory bridge](./cli_agent_framework.md).+- Generic CLI agent runtime shared with Codex / Gemini providers: `server/services/cli_agent/`.diff --git a/docs-internal/credentials_encryption.md b/docs-internal/credentials_encryption.mdindex 376a4332..35d7ca3f 100644--- a/docs-internal/credentials_encryption.md+++ b/docs-internal/credentials_encryption.md@@ -54,6 +54,8 @@ API_KEY_ENCRYPTION_KEY (from .env)The derived Fernet key lives only in `EncryptionService._fernet` in process memory. It is never written to disk or to Redis.+`EncryptionService` (`server/core/encryption.py`) exposes exactly five methods: `initialize(password, salt)` (derive + store the Fernet cipher), `encrypt(plaintext) -> str` (base64 ciphertext), `decrypt(ciphertext) -> str`, `clear()` (drop the in-memory key), and `is_initialized() -> bool`. `CredentialsDatabase` (`server/core/credentials_database.py`) backs both systems with `initialize() -> bytes` (creates tables, returns the salt), `save_api_key` / `get_api_key` / `delete_api_key`, and `save_oauth_tokens` / `get_oauth_tokens` / `delete_oauth_tokens(provider, customer_id="owner")`.+## Lifecycle```@@ -147,12 +149,17 @@ class CredentialBackend(ABC):The factory `create_backend(settings, credentials_db)` returns the selected backend with automatic fallback to Fernet if the requested backend is unavailable (e.g. `boto3` not installed for AWS).-Optional dependencies for alternate backends:+Dependencies (`server/pyproject.toml`) — the core `cryptography` package is always required for the default Fernet backend; the keyring / AWS packages are optional extras:```toml+[project]+dependencies = [+ "cryptography>=44.0.0", # Fernet encryption (always required)+]+[project.optional-dependencies]-keyring = ["keyring>=25.0.0"]-aws = ["boto3>=1.34.0"]+keyring = ["keyring>=25.0.0"] # OS-native credential storage+aws = ["boto3>=1.34.0"] # AWS Secrets Manager```## Configurationdiff --git a/docs-internal/credentials_panel.md b/docs-internal/credentials_panel.mdindex 4c458526..5cca188f 100644--- a/docs-internal/credentials_panel.md+++ b/docs-internal/credentials_panel.md@@ -376,8 +376,8 @@ These are the non-negotiable behaviours the test suite enforces. The refactor is| Credential service | [server/services/auth.py](../server/services/auth.py) | Single point of access; memory caches || Encryption | [server/core/encryption.py](../server/core/encryption.py) | Fernet + PBKDF2 || DB | [server/core/credentials_database.py](../server/core/credentials_database.py) | Two encrypted tables + metadata |-| OAuth (Google) | [server/services/google_oauth.py](../server/services/google_oauth.py) | Flow, PKCE, refresh, service builders |-| OAuth (Twitter) | [server/services/twitter_oauth.py](../server/services/twitter_oauth.py) | Manual PKCE, token exchange/refresh/revoke |+| OAuth (Google) | [server/nodes/google/_oauth.py](../server/nodes/google/_oauth.py) | Flow, PKCE, refresh, service builders |+| OAuth (Twitter) | [server/nodes/twitter/_oauth.py](../server/nodes/twitter/_oauth.py) | Manual PKCE, token exchange/refresh/revoke || OAuth (Claude) | [server/nodes/agent/claude_code_agent/_oauth.py](../server/nodes/agent/claude_code_agent/_oauth.py) + [_handlers.py](../server/nodes/agent/claude_code_agent/_handlers.py) | CLI-managed `claude auth login` (subprocess, `stdin=PIPE`), `CLAUDE_CONFIG_DIR=<DATA_DIR>/claude/` || OAuth utils | [server/services/oauth_utils.py](../server/services/oauth_utils.py) | Runtime redirect URI derivation || Container | [server/core/container.py](../server/core/container.py) | DI wiring for `auth_service` etc. |diff --git a/docs-internal/deployment_legacy.md b/docs-internal/deployment_legacy.mdnew file mode 100644index 00000000..5f4d47d8--- /dev/null+++ b/docs-internal/deployment_legacy.md@@ -0,0 +1,311 @@+# Deployment — Current Self-Deploy CLI + Legacy Docker Reference++This doc is the single home for MachinaOS deployment topology. The **current** path is the+`machina deploy` self-deploy CLI (top section). The **Docker Compose** path below it is a+**legacy reference only** — the compose files, `docker/` directory, and `scripts/docker.js`+wrapper have all been removed from the repo. It is retained solely as documentation of the+historical container topology.++---++## Self-Deploy CLI (`machina deploy`) — current path++One command provisions a login-gated MachinaOS VM on a cloud provider. Two stages:++1. **Operator's cloud CLI** (gcloud; aws planned) handles auth + project/region/zone resolution ++ ADC verification + API enablement.+2. **Terraform** (`cli/terraform/gcp/`) owns all resources — VM (always named `machinaos`),+ firewall, artifact bucket (local `npm pack` source), service account, and a cloud-init startup+ script that installs Node 22 + uv + the package and runs `machina serve` under systemd.++Login gate = built-in auth (`VITE_AUTH_ENABLED=true`, `AUTH_MODE=single`) with the owner+credential generated at deploy time and seeded on first boot.++```bash+machina deploy up --provider gcp --owner-email [email protected] # provision + install + print URL/creds+machina deploy status # URL + /health+machina deploy destroy # terraform destroy + clear state+```++### Key files++| Path | Responsibility |+|------|----------------|+| `cli/commands/serve.py` | Single-port runtime: uvicorn fronts API + WS + built SPA, plus the node sidecar |+| `cli/commands/deploy/` | Verbs (`up.py` / `status.py` / `destroy.py`), `_secrets.py`, `_state.py`, `_terraform.py` (Terraform driver), `providers/` (`gcp.py` / `aws.py` provider CLI adapters) |+| `cli/terraform/gcp/` | HCL module (`main.tf` / `variables.tf` / `outputs.tf`) + `startup.sh.tftpl` cloud-init template |++### State + delinking++- Deployment state lives at `<user-data>/deploy/machinaos/` — preserved by `machina clean`+ (see `_MACHINA_KEEP` in `cli/commands/clean.py`); only `machina deploy destroy` removes it.+- The deploy code is fully delinked from `machina build` / `machina clean`: lazy verb stubs in+ `cli/cli.py` mean nothing in the build pipeline imports it.++The legacy `deploy.sh` (docker-compose images over SCP to a GCE box) was removed.++---++## Docker Deployment (LEGACY REFERENCE — removed from repo)++> **Legacy reference only.** The compose files (`docker-compose.yml`, `docker-compose.prod.yml`),+> the `docker/` directory (incl. `docker/Dockerfile.whatsapp`), `client/Dockerfile`,+> `server/Dockerfile`, and `scripts/docker.js` are **no longer in the repository**. This section+> documents the historical 4-container topology for reference; it is not a runnable path today.++The project previously deployed using Docker Compose with an nginx reverse proxy.++### Docker Configuration (4-Container Stack)++**Services:**++| Container | Image | Port | Description |+|-----------|-------|------|-------------|+| redis | redis:7-alpine | 6379 | Cache and pub/sub for workflows |+| backend | machinaos-backend | 3010 | FastAPI Python backend |+| frontend | machinaos-frontend | 3000 | React app via nginx |+| whatsapp | machinaos-whatsapp | 5000 | Go WhatsApp bridge service |++**Frontend (`client/Dockerfile`):**+- Multi-stage build: Node.js builder -> nginx:alpine production+- Serves static files via nginx on port 80 (mapped to 3000)+- Size: ~54 MB++**Backend (`server/Dockerfile`):**+- Python 3.12-slim base with Node.js 22.x for JS/TS execution+- Includes Playwright chromium for JS-rendered web scraping (crawleeScraper node)+- Includes persistent Node.js server (Express + tsx) on port 3020+- Optimized bytecode compilation (`python -O -m compileall`)+- Health check endpoint on port 3010+- Startup script (`start.sh`) runs both Python and Node.js servers (must have LF line endings, not CRLF)+- Depends on: redis, whatsapp+- Size: ~800 MB (includes Playwright chromium)++**WhatsApp (`docker/Dockerfile.whatsapp`):**+- Uses npm package `whatsapp-rpc` with pre-built binaries+- Node.js 20-alpine base with `npx whatsapp-rpc api --foreground`+- Binary downloaded from GitHub releases during npm postinstall+- Exposed on port 9400 (configurable via `PORT`, `WHATSAPP_RPC_PORT` env vars, or `--port` CLI flag)+- QR codes generated as base64 PNG in memory (no file I/O)+- Also published to PyPI as `whatsapp-rpc` (async Python client)+- Size: ~150 MB (includes Node.js runtime)++**Redis:**+- Official redis:7-alpine image+- Healthcheck: `redis-cli ping`+- Persistent volume: `redis_data`+- No authentication (internal network only)++**Development Compose (`docker-compose.yml`):**+```yaml+services:+ # Redis uses profiles - only starts when REDIS_ENABLED=true+ redis:+ image: redis:7-alpine+ profiles:+ - redis # Only starts with --profile redis flag+ ports: ["${REDIS_PORT:-6379}:6379"]+ healthcheck:+ test: ["CMD", "redis-cli", "ping"]++ backend:+ build: ./server+ ports: ["${PYTHON_BACKEND_PORT:-3010}:${PYTHON_BACKEND_PORT:-3010}"]+ volumes:+ - ./server:/app+ - /app/nodejs/node_modules # Preserve Linux binaries (prevents Windows esbuild conflict)+ depends_on:+ whatsapp: { condition: service_healthy } # No Redis dependency+ environment:+ - REDIS_ENABLED=${REDIS_ENABLED:-false}+ - REDIS_URL=redis://redis:6379++ frontend:+ build: ./client+ ports: ["${VITE_CLIENT_PORT:-3000}:${VITE_CLIENT_PORT:-3000}"]++ whatsapp:+ build:+ context: .+ dockerfile: docker/Dockerfile.whatsapp+ ports: ["${WHATSAPP_RPC_PORT:-9400}:${WHATSAPP_RPC_PORT:-9400}"]+```++**Docker Scripts Wrapper (`scripts/docker.js`):**+Auto-detected `REDIS_ENABLED` in `.env` and added the `--profile redis` flag when enabled:+```javascript+// Reads .env and checks REDIS_ENABLED value+function isRedisEnabled() {+ const content = readFileSync(resolve(ROOT, '.env'), 'utf8');+ const match = content.match(/^REDIS_ENABLED\s*=\s*(.+)$/m);+ const value = match?.[1].trim().toLowerCase();+ return value === 'true' || value === '1' || value === 'yes';+}++// Adds --profile redis when enabled+if (isRedisEnabled()) {+ composeArgs.push('--profile', 'redis');+}+```++**Production Compose (`docker-compose.prod.yml`):**+```yaml+services:+ redis:+ image: redis:7-alpine+ ports: ["6379:6379"]+ volumes: ["redis_data:/data"]+ healthcheck:+ test: ["CMD", "redis-cli", "ping"]++ whatsapp:+ build:+ context: .+ dockerfile: docker/Dockerfile.whatsapp+ ports: ["9400:9400"]+ healthcheck:+ test: ["CMD", "wget", "-q", "--spider", "http://localhost:9400/health"]++ backend:+ build: ./server+ ports: ["3010:3010"]+ depends_on:+ redis: { condition: service_healthy }+ whatsapp: { condition: service_healthy }+ environment:+ - REDIS_ENABLED=true+ - REDIS_URL=redis://redis:6379++ frontend:+ build: ./client+ ports: ["3000:80"]+```++### Nginx Configuration++Located at `/etc/nginx/sites-available/flow.zeenie.xyz`:+- Frontend: `/` -> `http://127.0.0.1:3000`+- Backend API: `/api/` -> `http://127.0.0.1:3010/api/`+- WebSocket: `/ws/` -> `http://127.0.0.1:3010/ws/` (with upgrade headers)+- Webhook: `/webhook/` -> `http://127.0.0.1:3010/webhook/`+- Health: `/health` -> `http://127.0.0.1:3010/health`+- SSL via Let's Encrypt certbot++### Environment Configuration++**Development** (`server/.env`):+- `DEBUG=true`+- `CORS_ORIGINS` includes localhost ports+- `REDIS_ENABLED=false` (uses SQLite cache for local dev)++**Production** (Docker environment variables):+- `DEBUG=false`+- `CORS_ORIGINS=["https://your-domain.com"]`+- `REDIS_ENABLED=true` (Docker Redis container)+- `REDIS_URL=redis://redis:6379`+- Environment set in `docker-compose.prod.yml`, not the `.env` file++### Frontend API URL Resolution++The frontend automatically detects production vs development. This logic still ships in+`client/src/config/api.ts` (it is independent of the deployment mechanism):++```typescript+// client/src/config/api.ts+const isProduction = typeof window !== 'undefined' &&+ !window.location.hostname.includes('localhost') &&+ !window.location.hostname.includes('127.0.0.1');++return {+ // Python FastAPI backend (port 3010 in dev, same origin in prod)+ PYTHON_BASE_URL: viteEnv.VITE_PYTHON_SERVICE_URL || (isProduction ? '' : 'http://localhost:3010'),+};+```++- **Production**: Empty base URL = relative URLs (same origin)+- **Development**: Explicit `http://localhost:3010`++WebSocket URL derived from base URL:+```typescript+// client/src/contexts/WebSocketContext.tsx+if (!baseUrl) {+ // Production: use current origin+ const wsProtocol = window.location.protocol === 'https:' ? 'wss' : 'ws';+ return `${wsProtocol}://${window.location.host}/ws/status`;+}+```++### Resource Usage (GCP e2-micro, legacy Docker deployment)++| Resource | Value |+|----------|-------|+| CPU | 2 cores (Intel Xeon @ 2.20GHz) |+| RAM | 1.9 GB total, ~820 MB used |+| Disk | 14 GB total, ~9.2 GB used |+| Backend Memory | ~144 MB |+| Frontend Memory | ~3.4 MB |++### Useful Commands (legacy Docker deployment)++```bash+# View logs (all containers)+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f'++# View specific service logs+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f backend'+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose logs -f whatsapp'++# Restart all services+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart'++# Restart specific service+ssh $DEPLOY_HOST 'cd /opt/machinaos && docker-compose restart backend'++# Check container status+ssh $DEPLOY_HOST 'docker ps'++# Check resource usage+ssh $DEPLOY_HOST 'docker stats --no-stream'++# Check Redis connection+ssh $DEPLOY_HOST 'docker exec machinaos-redis-1 redis-cli ping'++# Check backend health (shows redis_enabled status)+curl -s https://$DEPLOY_DOMAIN/health | jq++# Clean up Docker resources (if disk full)+ssh $DEPLOY_HOST 'docker system prune -af && docker builder prune -af'+```++### Local Docker Development (legacy)++For testing the full production stack locally (when the compose files existed):++```bash+# Build and start all containers+docker-compose -f docker-compose.prod.yml up --build++# Access locally+# Frontend: http://localhost:3000+# Backend API: http://localhost:3010+# WhatsApp RPC: http://localhost:9400+# Redis: localhost:6379++# Stop all containers+docker-compose -f docker-compose.prod.yml down++# Remove volumes (clean slate)+docker-compose -f docker-compose.prod.yml down -v+```++---++## Local Development Build (current, non-Docker)++```bash+# Create optimized build+npm run build++# Serve built files locally+npm run preview+```diff --git a/docs-internal/event_framework.md b/docs-internal/event_framework.mdindex 9b790566..6c72d01e 100644--- a/docs-internal/event_framework.md+++ b/docs-internal/event_framework.md@@ -2,7 +2,7 @@Temporal-native event-routing layer for MachinaOs. Implements RFC sections6.3 (Temporal worker contract) + 6.4 (CloudEvents broadcast contract) from-[plugin_authoring_rfc.md](./plugin_authoring_rfc.md).+[plugin_authoring_rfc.md](./ARCHIVE/plugin_authoring_rfc.md).This doc is the operator + plugin-author reference. The design rationale +phase plan lives in `~/.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`.@@ -267,6 +267,6 @@ Locked by `TestCancelSweepsStuckNodeStatuses` in [`test_deployment_canary_listen## References- Plan: `~/.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`-- RFC: [`plugin_authoring_rfc.md`](./plugin_authoring_rfc.md)+- RFC: [`plugin_authoring_rfc.md`](./ARCHIVE/plugin_authoring_rfc.md)- Temporal: [Search Attributes](https://docs.temporal.io/search-attribute) · [Signals](https://docs.temporal.io/develop/python/message-passing) · [Schedules](https://docs.temporal.io/develop/python/schedules) · [Retry Policies](https://docs.temporal.io/encyclopedia/retry-policies)- CloudEvents: [v1.0 spec](https://github.com/cloudevents/spec/blob/main/cloudevents/spec.md)diff --git a/docs-internal/memory_lifecycle.md b/docs-internal/memory_lifecycle.mdindex 377a4cd7..66aa5528 100644--- a/docs-internal/memory_lifecycle.md+++ b/docs-internal/memory_lifecycle.md@@ -91,7 +91,7 @@ await clear_agent_session_state(Three stores are cleared in one call:1. **Vector store** (when `clear_long_term=True`) — `del services.ai._memory_vector_stores[session_id]`.-2. **TodoService** — every candidate key (`workflow_id`, `session_id`, `"default"`) is cleared because [`server/nodes/tool/write_todos.py`](../server/nodes/tool/write_todos.py) uses `ctx.workflow_id or ctx.node_id or "default"` as the storage key and we want to clear whichever fallback the write path actually used.+2. **TodoService** — every candidate key (`workflow_id`, `session_id`, `"default"`) is cleared because [`server/nodes/tool/write_todos/__init__.py`](../server/nodes/tool/write_todos/__init__.py) uses `ctx.workflow_id or ctx.node_id or "default"` as the storage key and we want to clear whichever fallback the write path actually used.3. **simpleMemory node fields** (when `memory_node_id` provided) — `memory_content` → `DEFAULT_MEMORY_CONTENT`; `last_session_id` → `None` (so claude_code_agent starts fresh next run instead of `--resume`-ing into a wiped transcript); orphan `memory_jsonl` field popped if present.Frontend `clear_memory` WS handler ([`routers/websocket.py:2167`](../server/routers/websocket.py)) calls this; UI presents a Reset Memory button on the simpleMemory parameter panel.diff --git a/docs-internal/native_llm_sdk.md b/docs-internal/native_llm_sdk.mdindex d503676b..cfc61f91 100644--- a/docs-internal/native_llm_sdk.md+++ b/docs-internal/native_llm_sdk.md@@ -1,7 +1,6 @@# Native LLM SDK Architecture-> **⚠️ Pre-Wave-11 — historical reference only.**-> Node authoring now happens on the backend: each node is a Python plugin under `server/nodes/<category>/<node>.py` that emits a `NodeSpec`. The frontend reads specs via [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) + [adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts). See [plugin_system.md](./plugin_system.md) and [server/nodes/README.md](../server/nodes/README.md) for the current model. The snippets below that reference `client/src/nodeDefinitions/*` are kept for historical context.+> **Authoring model (post-Wave-11):** each chat-model node is a self-contained Python plugin folder under `server/nodes/model/<provider>_chat_model/` that emits a `NodeSpec`. The frontend reads specs via [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) + [adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts) and renders through `SquareNode` with zero TS edits. See [plugin_system.md](./plugin_system.md) and [server/nodes/README.md](../server/nodes/README.md) for the plugin model, and "Adding a New Provider" below for the chat-model recipe.MachinaOS uses a hybrid LLM architecture: a native SDK layer in `server/services/llm/` for direct chat completions and model fetching, and a LangChain-backed agent loop in `server/services/ai.py` (the plain `_run_agent_loop` async function) for agent tool-calling. This document describes the native layer, its design, and how both paths coexist.@@ -53,6 +52,38 @@ The native layer currently supports **12 providers**, grouped by implementation:Source of truth for this list: `server/config/llm_defaults.json` (the `providers` dict) and `server/services/llm/factory.py` (`NATIVE_PROVIDERS` constant).+### Native chat path vs agent dropdown — two different counts++- **Native chat path (`execute_chat` / `fetch_models`) supports 12 providers** (the 10 OpenAI-compat + Anthropic/Gemini natives above; Groq/Cerebras fall back to LangChain inside `execute_chat`). `xai` lives here.+- **The agent dropdown exposes 11 providers** for `aiAgent`, `chatAgent` (Zeenie), and all specialized agents — the `provider` Literal in [`nodes/agent/ai_agent/__init__.py`](../server/nodes/agent/ai_agent/__init__.py), [`chat_agent.py`](../server/nodes/agent/chat_agent/__init__.py), and [`_specialized.py`](../server/nodes/agent/_specialized.py): `openai`, `anthropic`, `gemini`, `openrouter`, `groq`, `cerebras`, `deepseek`, `kimi`, `mistral`, `ollama`, `lmstudio`. **`xai` is native-chat-only and is NOT in the agent Literal** (no agent dropdown entry).+- Groq and Cerebras are available as standalone chat-model nodes AND in the agent dropdown, but use the LangChain path for both chat and agent execution.++### Provider / model reference table++`ModelRegistryService` (`server/services/model_registry.py`) manages the per-model constraints below — fetching from OpenRouter for cloud models and from the user's running local server (`ollama.AsyncClient.ps()` / `lmstudio.AsyncClient.llm.list_loaded()`) for Ollama / LM Studio. Falls back to `llm_defaults.json` only when neither source is available.++| Provider | Key Models | Context | Max Output | Thinking | Temp Range |+|----------|-----------|---------|-----------|----------|------------|+| **OpenAI** | GPT-5.5/5.4/5.2 | 400K–1.05M | 128K | effort (hybrid) | 0-2 |+| **OpenAI** | GPT-4.1/4.1-mini/4.1-nano | 1M | 32K | none | 0-2 |+| **OpenAI** | o3, o4-mini | 200K | 100K | effort (reasoning) | fixed 1.0 |+| **OpenAI** | GPT-4o/4o-mini | 128K | 16K | none | 0-2 |+| **Anthropic** | Claude Fable 5 | 1M | 128K | budget | 0-1 |+| **Anthropic** | Claude Opus 4.8/4.7 | 1M | 128K | budget | 0-1 |+| **Anthropic** | Claude Sonnet 4.6 | 1M | 64K | budget | 0-1 |+| **Anthropic** | Claude Haiku 4.5 | 200K | 64K | budget | 0-1 |+| **Google** | Gemini 3.5-flash, 3.1-pro/flash-lite, 3-flash, 2.5-pro/flash/flash-lite | 1M | 64K | budget | 0-2 |+| **DeepSeek** | deepseek-v4-flash, deepseek-v4-pro (deepseek-chat/reasoner legacy) | 1M | 64K | thinking modes | 0-2 |+| **Kimi** | kimi-k2.6, kimi-k2.5, kimi-k2.7-code | 256K | 96K | on by default (disabled for agents) | fixed 0.6 |+| **Mistral** | mistral-large/medium/small-latest, codestral-latest | 256K | 131K | none | 0-1.5 |+| **Groq** | Llama 3.3-70b, Llama 3.1-8b, GPT-OSS-120b/20b, Qwen3-32b | 131K | 8-131K | format (Qwen3) | 0-2 |+| **OpenRouter** | 200+ models from multiple providers | varies | varies | varies | 0-2 |+| **Cerebras** | GPT-OSS-120b, Qwen-3-235b, GLM-4.7 | 131K | 65K | budget (Qwen) | 0-1.5 |+| **Ollama** | Whatever the user has pulled (qwen2.5, llama3.x, phi-3, deepseek-r1, ...) | per-loaded-model (typed via `ps()`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |+| **LM Studio** | Whatever the user has loaded in the LM Studio UI | per-loaded-model (typed via `LlmInstanceInfo.context_length`) | ctx ÷ 4 (capped 4096) | none (per-model) | 0-2 |++`_resolve_max_tokens()` in `server/services/ai.py` (a thin wrapper over `services/llm/config.py::resolve_max_tokens`) clamps user-requested `max_tokens` to the model's actual limit.+## Local LLM Providers (Ollama, LM Studio)Ollama and LM Studio expose an OpenAI-compatible `/v1` HTTP API, so they ride the same `OpenAIProvider` runtime path used by every other OpenAI-compat backend (DeepSeek, Kimi, Mistral). The differences are the **base URL** (the user enters their server's address, e.g. `http://localhost:11434/v1`) and the **per-model parameters** (which depend on what the user has loaded in the local server's UI, not a JSON default).@@ -71,7 +102,7 @@ Ollama and LM Studio expose an OpenAI-compatible `/v1` HTTP API, so they ride th**Runtime path** — `execute_chat()` and `execute_agent()` read `{provider}_proxy` via `auth.get_api_key()` and pass it as `proxy_url` to `create_provider()`. `OpenAIProvider.__init__` overrides `kwargs["base_url"]` with the user's URL and forces `api_key="ollama"` (the documented placeholder for unauthenticated local servers). The openai SDK then sends `POST {user_url}/chat/completions` — **traffic stays on the user's machine, never reaches api.openai.com**.-**Provider detection** ([`server/constants.py:detect_ai_provider`](../server/constants.py)) MUST list `ollama` / `lmstudio` substrings, and the agent dropdown's `provider` Literal in [`ai_agent.py`](../server/nodes/agent/ai_agent.py) / [`chat_agent.py`](../server/nodes/agent/chat_agent.py) / [`_specialized.py`](../server/nodes/agent/_specialized.py) MUST include `"ollama"` / `"lmstudio"` — otherwise the chat-model node silently falls through to `'openai'` and the runtime calls the OpenAI cloud with the local-server placeholder key.+**Provider detection** ([`server/constants.py:detect_ai_provider`](../server/constants.py)) MUST list `ollama` / `lmstudio` substrings, and the agent dropdown's `provider` Literal in [`ai_agent.py`](../server/nodes/agent/ai_agent/__init__.py) / [`chat_agent.py`](../server/nodes/agent/chat_agent/__init__.py) / [`_specialized.py`](../server/nodes/agent/_specialized.py) MUST include `"ollama"` / `"lmstudio"` — otherwise the chat-model node silently falls through to `'openai'` and the runtime calls the OpenAI cloud with the local-server placeholder key.**Open-world skip in `is_model_valid_for_provider`** — local model names like `qwen/qwen3.6-27b` don't contain provider substrings, so the cloud-style pattern check would always reject them. The function returns `True` for `openrouter` / `ollama` / `lmstudio` without consulting `detection_patterns`. The upstream server still rejects genuinely missing models with a clear 404.@@ -227,6 +258,57 @@ class ThinkingConfig:Each provider's `chat()` method reads only the fields it supports. The extracted thinking text is returned in `LLMResponse.thinking` so downstream nodes can display or drag it into parameters.+### Supported thinking/reasoning providers++| Provider | Models | Parameter | Thinking Type | Notes |+|----------|--------|-----------|---------------|-------|+| **Claude** | All Claude 4.x/3.5 | `thinkingBudget` (1024-16000 tokens) | budget | Requires `max_tokens > budget_tokens`. Temperature auto-set to 1. |+| **Gemini** | gemini-3.x, gemini-2.5-pro/flash | `thinkingBudget` (token count) | budget | Uses `thinking_budget` API parameter |+| **OpenAI** | o1, o3, o3-mini, o4-mini | `reasoningEffort` (low/medium/high) | effort | Reasoning-only models. Temperature fixed at 1.0. |+| **OpenAI** | GPT-5.2/5.1/5/5-mini/5-nano | `reasoningEffort` (low/medium/high/xhigh) | effort | Hybrid reasoning: can operate with or without thinking. |+| **Groq** | qwen3-32b | `reasoningFormat` ('parsed' or 'hidden') | format | 'parsed' returns reasoning, 'hidden' returns only final answer |+| **Cerebras** | qwen-3-235b | `reasoningFormat` ('parsed' or 'hidden') | format | Same format-based reasoning as Groq Qwen |++The thinking/reasoning fields (`thinkingEnabled`, `thinkingBudget`, `reasoningEffort`, `reasoningFormat`) live in the backend NodeSpec for each chat model (`server/nodes/model/<provider>_chat_model/`) and are surfaced through `AIChatModelParams` in `server/models/nodes.py`. The frontend renders them automatically via the universal parameter panel.++### Thinking extraction (LangChain path)++On the LangChain path, `extract_thinking_from_response(response, provider)` in `server/services/ai.py` pulls reasoning text out of provider-specific response shapes:++```python+def extract_thinking_from_response(response, provider: str) -> Optional[str]:+ """Extract thinking/reasoning from AI response based on provider."""+ # Claude: content_blocks with type='thinking'+ # Gemini: response_metadata.candidates[0].content.parts with thought=True+ # Groq: additional_kwargs.reasoning or response_metadata.reasoning+ # OpenAI o-series: requires organization verification+```++The agent/chat result envelope carries `thinking` alongside the answer:++```python+{+ "success": True,+ "result": {+ "response": "The final answer text",+ "thinking": "The model's internal reasoning (if available)",+ "model": "claude-3-5-sonnet-20241022",+ "provider": "anthropic",+ "finish_reason": "stop",+ "timestamp": "2025-01-23T...",+ }+}+```++The `thinking` field is exposed to downstream nodes via the backend output schema (`AIAgentOutput` in `server/services/node_output_schemas.py`, shared across every LLM-backed agent + chat model) and rendered in a collapsible `ThinkingBlock` (`client/src/components/ui/NodeOutputPanel.tsx`, default-expanded, provider-aware label).++### Thinking limitations++- **OpenAI o-series**: Reasoning summaries are only available to organizations that have completed verification at platform.openai.com. Without verification, `thinking` is `null`.+- **Claude**: `max_tokens` must be greater than `thinkingBudget`. Temperature is automatically set to 1 when thinking is enabled.+- **Groq**: Only Qwen3-32b supports reasoning (QwQ removed from Groq). Format `hidden` suppresses reasoning output.+- **Cerebras**: Qwen-3-235b supports format-based reasoning (same as Groq Qwen).+See [memory_compaction.md](memory_compaction.md) for how thinking token counts are tracked separately from output tokens.## Model Max Tokens Resolution@@ -255,23 +337,112 @@ Paired with `ModelRegistryService` (`server/services/model_registry.py`), which- `supports_thinking`- `thinking_type`+## Proxy-Based Authentication (Ollama Pattern)++AI providers support optional proxy-based authentication — requests route through a local proxy server that handles auth, following the [Ollama Claude Code integration](https://docs.ollama.com/integrations/claude-code) pattern.++**How it works:**+1. User configures a proxy URL in the Credentials Modal (e.g., `http://localhost:11434`).+2. Requests route through the proxy instead of directly to the provider API.+3. Proxy handles authentication (token set to `"ollama"` automatically).+4. No API key storage needed in MachinaOs — auth delegated to proxy.++**Configuration:** proxy URLs are stored in the credentials DB under the `{provider}_proxy` pattern (e.g., `anthropic_proxy`, `openai_proxy`). Falls back to direct API key if no proxy configured. This is the SAME mechanism the native Ollama / LM Studio path uses (see "Local LLM Providers" above) — the validator persists the user's server URL under `{provider}_proxy`, and at runtime it carries into `OpenAIProvider`'s `base_url`.++**Native path:** `create_provider(name, api_key, proxy_url=url)`. **LangChain path:** `create_model()` sets `base_url`:++```python+def create_model(self, provider: str, api_key: str, model: str,+ temperature: float, max_tokens: int,+ thinking: Optional[ThinkingConfig] = None,+ proxy_url: Optional[str] = None):+ # ...+ if proxy_url:+ kwargs['base_url'] = proxy_url+ kwargs[config.api_key_param] = "ollama" # Ollama-style placeholder token+```++**Use cases:** Claude Code CLI proxy for Anthropic models; native Ollama / LM Studio support; custom auth proxies; dev/testing with mock servers.++## Provider Default Parameters++Users configure default parameter values per LLM provider in the Credentials Modal; defaults apply to new AI nodes using that provider.++**Configurable parameters:**+- `temperature` — range varies by provider (Anthropic 0-1, Cerebras 0-1.5, others 0-2; o-series fixed 1.0)+- `max_tokens` (1-200000) — clamped to the model's actual limit by `_resolve_max_tokens()`+- `thinking_enabled` — extended thinking toggle+- `thinking_budget` (1024-16000) — token budget for thinking (Claude, Gemini)+- `reasoning_effort` (low/medium/high) — OpenAI o-series and GPT-5 hybrid reasoning+- `reasoning_format` (parsed/hidden) — Groq Qwen3 models++```python+# server/models/database.py+class ProviderDefaults(SQLModel, table=True):+ provider: str # openai, anthropic, gemini, groq, openrouter, cerebras+ temperature: float+ max_tokens: int+ thinking_enabled: bool+ thinking_budget: int+ reasoning_effort: str # low, medium, high+ reasoning_format: str # parsed, hidden+```++| File | Description |+|------|-------------|+| `server/models/database.py` | `ProviderDefaults` SQLModel |+| `server/core/database.py` | `get_provider_defaults()`, `save_provider_defaults()` CRUD |+| `server/routers/websocket.py` | `get_provider_defaults`, `save_provider_defaults` handlers |+| `client/src/hooks/useApiKeys.ts` | `getProviderDefaults()`, `saveProviderDefaults()` methods |+| `client/src/components/CredentialsModal.tsx` | Default Parameters UI section |+## Adding a New Provider+> **Post-Wave-11 authoring.** A chat-model provider is a self-contained folder under `server/nodes/model/<provider>_chat_model/` with `__init__.py` declaring a `ChatModelBase` subclass. It auto-registers via `BaseNode.__init_subclass__`; the frontend renders it through `SquareNode` from the emitted NodeSpec with **zero TypeScript changes**. There is no `client/src/nodeDefinitions/`, no `ModelNode.tsx`, no `Dashboard.tsx` switch to edit.++```python+# server/nodes/model/openrouter_chat_model/__init__.py+class OpenRouterChatModel(ChatModelBase):+ type = "openrouterChatModel"+ metadata = NodeMetadata(+ display_name="OpenRouter",+ icon="lobehub:openrouter", # asset:<key>, lobehub:<brand>, or emoji+ color="#6366F1",+ component_kind="model", # routes to SquareNode in Dashboard.tsx+ )++ class Params(ChatModelBase.Params):+ # provider-specific overrides; everything else inherits from ChatModelBase+ ...+```++**Backend steps:**+1. **OpenAI-compatible provider** (DeepSeek, Kimi, Mistral pattern):- Add an entry to `llm_defaults.json` with `base_url`, `default_model`, `detection_patterns`, `max_output_tokens._default`, `context_length._default`, `temperature_range`.- Add the provider name to `NATIVE_PROVIDERS` in `factory.py` if you want `execute_chat()` to route natively.- - Add to `PROVIDER_CONFIGS` in `services/ai.py` for LangChain agent path (reuse `ChatOpenAI`, set `base_url` in `create_model()`).+ - The LangChain agent path needs no Python branching — `create_model()` reuses `ChatOpenAI` with `base_url` from `llm_defaults.json`.2. **Custom-SDK provider** (Anthropic, Gemini pattern):- - Create `services/llm/providers/<name>.py` implementing `LLMProvider` protocol.+ - Create `services/llm/providers/<name>.py` implementing the `LLMProvider` protocol.- Add a branch in `create_provider()` in `factory.py`.- - Add the provider name to `_DEDICATED_PROVIDERS` and `NATIVE_PROVIDERS`.- - Add config entry in `llm_defaults.json` (no `base_url` needed if the SDK handles URLs itself).--3. **Frontend wiring** (in either case):- - Add node definition in `client/src/nodeDefinitions/aiModelNodes.ts` using `createBaseChatModel()`.- - Add to `CREDENTIAL_TO_PROVIDER` map in `ModelNode.tsx`.- - Add credential entry in `CredentialsModal.tsx`.+ - Add the provider name to the dedicated-provider set and `NATIVE_PROVIDERS`.+ - Add a config entry in `llm_defaults.json` (no `base_url` needed if the SDK handles URLs itself).++3. **Credentials + agent exposure:**+ - Add a `Credential` subclass in `server/nodes/model/_credentials.py` — surfaces in the Credentials Modal automatically.+ - To expose the provider in the **agent dropdown**, add its name to the `provider` Literal in `nodes/agent/ai_agent/__init__.py`, `chat_agent.py`, AND `_specialized.py`, and add the substring to `detect_ai_provider` in `server/constants.py` — otherwise an agent using it silently falls back to `'openai'`.++### Key implementation files++| File | Purpose |+|---|---|+| `server/nodes/model/<provider>_chat_model/__init__.py` | Plugin entry — metadata + Params + auto-registers |+| `server/nodes/model/_credentials.py` | `Credential` subclass per provider |+| `server/config/llm_defaults.json` | base_url + supported_params + temperature constraints (no hardcoded URLs in Python) |+| `server/services/llm/providers/<provider>.py` | Native SDK provider (Protocol-based) |+| `server/services/ai.py` | Routing: native chat path + LangChain agent path |+| `client/src/Dashboard.tsx` | Generic `COMPONENT_BY_KIND` dispatch — no per-provider entry needed |## Related Docsdiff --git a/docs-internal/node-logic-flows/README.md b/docs-internal/node-logic-flows/README.mdindex be3cac92..a4f4dd82 100644--- a/docs-internal/node-logic-flows/README.md+++ b/docs-internal/node-logic-flows/README.md@@ -12,8 +12,8 @@ match its doc here, and each doc must still describe what the code does.- **Refactoring a node**: update the matching contract test in`server/tests/nodes/test_<category>.py` *first*, then change the handler,then update this doc.-- **Rebuilding the index below**: run `node scripts/build-node-docs-index.js`- (also runs in CI to fail builds when a node is missing a doc).+- **The index below is maintained by hand** (there is no generator script+ currently). When you add, rename, or remove a card, update its entry below.## Conventions@@ -26,7 +26,7 @@ match its doc here, and each doc must still describe what the code does.## Index-<!-- AUTO-GENERATED-INDEX-START -->+<!-- INDEX-START (manually maintained; filenames are camelCase of the plugin `type`) -->### ai_agents@@ -42,12 +42,15 @@ match its doc here, and each doc must still describe what the code does.- [Gemini Chat Model (`geminiChatModel`)](./ai_chat_models/geminiChatModel.md)- [Groq Chat Model (`groqChatModel`)](./ai_chat_models/groqChatModel.md)- [Kimi Chat Model (`kimiChatModel`)](./ai_chat_models/kimiChatModel.md)+- [LM Studio Chat Model (`lmstudioChatModel`)](./ai_chat_models/lmstudioChatModel.md)- [Mistral Chat Model (`mistralChatModel`)](./ai_chat_models/mistralChatModel.md)+- [Ollama Chat Model (`ollamaChatModel`)](./ai_chat_models/ollamaChatModel.md)- [OpenAI Chat Model (`openaiChatModel`)](./ai_chat_models/openaiChatModel.md)- [OpenRouter Chat Model (`openrouterChatModel`)](./ai_chat_models/openrouterChatModel.md)### ai_tools+- [Agent Builder (`agentBuilder`)](./ai_tools/agentBuilder.md)- [Calculator Tool (`calculatorTool`)](./ai_tools/calculatorTool.md)- [Current Time Tool (`currentTimeTool`)](./ai_tools/currentTimeTool.md)- [DuckDuckGo Search (`duckduckgoSearch`)](./ai_tools/duckduckgoSearch.md)@@ -73,12 +76,53 @@ match its doc here, and each doc must still describe what the code does.- [System Info (`systemInfo`)](./android/systemInfo.md)- [WiFi Automation (`wifiAutomation`)](./android/wifiAutomation.md)+### chat_utility++- [Chat History (`chatHistory`)](./chat_utility/chatHistory.md)+- [Chat Send (`chatSend`)](./chat_utility/chatSend.md)+- [Console (`console`)](./chat_utility/console.md)+- [Create Map (`gmaps_create`)](./chat_utility/gmaps_create.md)+- [File Handler (`fileHandler`)](./chat_utility/fileHandler.md)+- [Team Monitor (`teamMonitor`)](./chat_utility/teamMonitor.md)+- [Text Generator (`textGenerator`)](./chat_utility/textGenerator.md)++### code_fs_process++- [File Modify (`fileModify`)](./code_fs_process/fileModify.md)+- [File Read (`fileRead`)](./code_fs_process/fileRead.md)+- [FS Search (`fsSearch`)](./code_fs_process/fsSearch.md)+- [JavaScript Executor (`javascriptExecutor`)](./code_fs_process/javascriptExecutor.md)+- [Monty Executor (`montyExecutor`)](./code_fs_process/montyExecutor.md)+- [Process Manager (`processManager`)](./code_fs_process/processManager.md)+- [Python Executor (`pythonExecutor`)](./code_fs_process/pythonExecutor.md)+- [Shell (`shell`)](./code_fs_process/shell.md)+- [TypeScript Executor (`typescriptExecutor`)](./code_fs_process/typescriptExecutor.md)++### document++- [Document Parser (`documentParser`)](./document/documentParser.md)+- [Embedding Generator (`embeddingGenerator`)](./document/embeddingGenerator.md)+- [File Downloader (`fileDownloader`)](./document/fileDownloader.md)+- [HTTP Scraper (`httpScraper`)](./document/httpScraper.md)+- [Text Chunker (`textChunker`)](./document/textChunker.md)+- [Vector Store (`vectorStore`)](./document/vectorStore.md)+- [Email Read (`emailRead`)](./email/emailRead.md)- [Email Receive (`emailReceive`)](./email/emailReceive.md)- [Email Send (`emailSend`)](./email/emailSend.md)+### google_workspace++- [Calendar (`googleCalendar`)](./google_workspace/googleCalendar.md)+- [Contacts (`googleContacts`)](./google_workspace/googleContacts.md)+- [Drive (`googleDrive`)](./google_workspace/googleDrive.md)+- [Gmail (`googleGmail`)](./google_workspace/googleGmail.md)+- [Gmail Receive (`googleGmailReceive`)](./google_workspace/googleGmailReceive.md)+- [Sheets (`googleSheets`)](./google_workspace/googleSheets.md)+- [Tasks (`googleTasks`)](./google_workspace/googleTasks.md)+### http_proxy- [HTTP Request (`httpRequest`)](./http_proxy/httpRequest.md)@@ -98,6 +142,7 @@ match its doc here, and each doc must still describe what the code does.- [Android Control Agent (`android_agent`)](./specialized_agents/androidAgent.md)- [Autonomous Agent (`autonomous_agent`)](./specialized_agents/autonomousAgent.md)- [Claude Code Agent (`claude_code_agent`)](./specialized_agents/claudeCodeAgent.md)+- [Codex (`codex_agent`)](./specialized_agents/codexAgent.md)- [Coding Agent (`coding_agent`)](./specialized_agents/codingAgent.md)- [Consumer Agent (`consumer_agent`)](./specialized_agents/consumerAgent.md)- [Orchestrator Agent (`orchestrator_agent`)](./specialized_agents/orchestratorAgent.md)@@ -146,4 +191,4 @@ match its doc here, and each doc must still describe what the code does.- [Webhook Response (`webhookResponse`)](./workflow_triggers/webhookResponse.md)- [Webhook Trigger (`webhookTrigger`)](./workflow_triggers/webhookTrigger.md)-<!-- AUTO-GENERATED-INDEX-END -->+<!-- INDEX-END -->diff --git a/docs-internal/node-logic-flows/ai_agents/aiAgent.md b/docs-internal/node-logic-flows/ai_agents/aiAgent.mdindex 4e912f5b..ebe4d230 100644--- a/docs-internal/node-logic-flows/ai_agents/aiAgent.md+++ b/docs-internal/node-logic-flows/ai_agents/aiAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | ai_agents / agent |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_agent`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/agent/ai_agent/__init__.py`](../../../server/nodes/agent/ai_agent/__init__.py) — dispatched via `BaseNode.execute()` + the `@Operation("execute")` method (`execute_op`). Pre-dispatch (edge walk + task inject + prompt fallback) in [`server/nodes/agent/_inline.py::prepare_agent_call`](../../../server/nodes/agent/_inline.py); LLM loop in `AIService.execute_agent`. || **Tests** | [`server/tests/nodes/test_ai_agents.py`](../../../server/tests/nodes/test_ai_agents.py) || **Skill (if any)** | n/a (the agent consumes skills via `input-skill`) || **Dual-purpose tool** | no |@@ -16,8 +16,9 @@ and system message, optionally merges conversation history from a connectedvia `chat_model.bind_tools`, and runs the tool-calling loop until the LLM produces afinal answer. All of the heavy lifting (LLM invocation, tool execution, memorypersistence) lives in `AIService.execute_agent`; the handler's job is purely-to gather the connected-node payloads via `_collect_agent_connections` and-forward them.+to gather the connected-node payloads via+`edge_walker.collect_agent_connections` (a 5-tuple: memory, skill, tool,+input, task) and forward them.## Inputs (handles)@@ -31,17 +32,18 @@ forward them.## Parameters-| Name | Type | Default | Required | displayOptions.show | Description |-|------|------|---------|----------|---------------------|-------------|-| `provider` | options | `openai` | no | - | AI provider (openai, anthropic, gemini, groq, cerebras, openrouter, deepseek, kimi, mistral, xai). |-| `model` | string | `""` | **yes** | - | Model id; loaded dynamically from the provider. |-| `prompt` | string | `{{ $json.chatInput }}` | **yes** | - | User prompt; may reference upstream node outputs via templates. |-| `systemMessage` | string | `You are a helpful assistant` | no | - | System prompt prepended to the conversation. |-| `options.temperature` | number | `0.7` | no | - | 0-2. |-| `options.maxTokens` | number | `4096` | no | - | 1-200000; clamped to model context in `AIService._resolve_max_tokens`. |-| `options.thinkingEnabled` | boolean | `false` | no | - | Enables extended thinking / reasoning. |-| `options.thinkingBudget` | number | `2048` | no | `thinkingEnabled=true` | Token budget (Claude, Gemini, Cerebras). |-| `options.reasoningEffort` | options | `medium` | no | `thinkingEnabled=true` | `low` / `medium` / `high` (OpenAI o-series, Groq). |+Source: `AIAgentParams` in [`ai_agent/__init__.py`](../../../server/nodes/agent/ai_agent/__init__.py).++| Name | Type | Default | Required | Group | Description |+|------|------|---------|----------|-------|-------------|+| `prompt` | string (textarea, rows 4) | `""` | no | - | User prompt; may reference upstream node outputs via templates. Empty -> auto-prompt fallback from `input-main`. |+| `provider` | Literal | `openai` | no | - | One of openai, anthropic, gemini, openrouter, groq, cerebras, deepseek, kimi, mistral, ollama, lmstudio. |+| `model` | string | `""` | no | - | Model id; loaded dynamically from the provider. |+| `system_message` | string (rows 3) | `You are a helpful assistant` | no | - | System prompt prepended to the conversation. |+| `temperature` | number (optional) | `None` | no | options | 0-2, step 0.1. Unset falls through to `agent.default_temperature` in `llm_defaults.json`. |+| `max_tokens` | int (optional) | `None` | no | options | 1-200000. Unset falls through to the per-model default; clamped in `AIService._resolve_max_tokens`. |++> Note: `api_key` is **not** a declared field — credentials are auto-injected at execution time by `node_executor._inject_api_keys`. The thinking/reasoning controls are resolved by the LLM service layer from provider/model metadata, not exposed as `AIAgentParams` fields.## Outputs (handles)@@ -68,7 +70,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution```mermaidflowchart TD- A[handle_ai_agent] --> B[_collect_agent_connections]+ A[execute_op -> prepare_agent_call] --> B[edge_walker.collect_agent_connections]B --> C{task_data?}C -- yes --> D[Format task_context<br/>prepend to prompt]D --> E{task status in<br/>completed/error AND<br/>tool_data?}@@ -85,7 +87,7 @@ flowchart TD## Decision Logic-- **`_collect_agent_connections`** scans `context.edges` for edges whose target+- **`edge_walker.collect_agent_connections`** scans `context.edges` for edges whose targetis this node, then routes each by `targetHandle`:- `input-memory` + source type `simpleMemory` -> build `memory_data` dict.`session_id` is auto-set to the agent's `node_id` unless the memory node@@ -106,30 +108,36 @@ flowchart TD- **Task completion short-circuit**: if `task_data` exists, `_format_task_context`produces a plain-English wrapper which is prepended to the prompt. If thetask status is `completed` or `error` and any tools are present, **all tools- are stripped** to prevent the LLM from delegating again.+ are stripped** to prevent the LLM from delegating again. (See+ `prepare_agent_call` in `_inline.py`.)- **Auto-prompt fallback**: if `parameters.prompt` is empty after the task- block, the handler extracts `input_data.message`, `input_data.text`, or- `input_data.content` (first truthy wins) and assigns it to `prompt`. If the+ block, `prepare_agent_call` extracts `input_data.message`, `input_data.text`,+ or `input_data.content` (first truthy wins) and assigns it to `prompt`. If theinput is not a dict the whole value is stringified.-- **Delegation to AIService**: the handler always returns whatever- `ai_service.execute_agent` returns; success/error envelope shape is- owned by the service, not the handler.+- **Team-lead delegation**: for `orchestrator_agent` / `ai_employee`,+ teammates connected via `input-teammates` are appended to `tool_data` as+ `delegate_to_*` tools (not applicable to `aiAgent` itself).+- **Delegation to AIService**: `execute_op` returns `response.get("result")`+ on success; a `success=False` envelope raises `RuntimeError`. Typed openai+ SDK failures surface as `NodeUserError` from inside `execute_agent`.## Side Effects-- **Database writes**: none directly in the handler. `AIService.execute_agent`+- **Database writes**: none directly in `execute_op`. `AIService.execute_agent`writes `token_usage_metrics` rows (via `CompactionService.track`), and`memory_content` markdown changes persist back through the`simpleMemory` node's `save_node_parameters` path owned by the service.-- **Broadcasts**: `handle_ai_agent` acquires `StatusBroadcaster` and passes+- **Broadcasts**: `prepare_agent_call` resolves `StatusBroadcaster` and passesit into `execute_agent`. The service then emits `update_node_status`(`thinking`, `executing_tool`, `success`, ...) plus `token_usage_update`and potentially `compaction_starting` / `compaction_completed` events.+ `BaseNode.execute()` additionally wraps the body in a `node.aiAgent.execute`+ OpenTelemetry span + `log_context(node_id, node_type, workflow_id)`.- **External API calls**: delegated to the native LLM provider SDKs (Anthropic,OpenAI, Gemini, OpenRouter, xAI, DeepSeek, Kimi, Mistral) or the LangChainfallback (Groq, Cerebras). Tool nodes may spawn their own HTTP or subprocesscalls via `execute_tool`.-- **File I/O**: none in the handler. Filesystem tool nodes may read/write the+- **File I/O**: none in `execute_op`. Filesystem tool nodes may read/write theper-workflow workspace (`context.workspace_dir`).- **Subprocess**: none directly. Tool executors (shell, process manager,browser, code executors) spawn subprocesses.@@ -143,7 +151,7 @@ flowchart TD`PricingService`, `SkillLoader` (for `masterSkill` expansion).- **Python packages**: `langchain-core`, `langchain-openai`,`langchain-anthropic`, provider SDKs (`anthropic`, `openai`, `google-genai`).-- **Environment variables**: none read directly by the handler.+- **Environment variables**: none read directly by `execute_op`.## Edge cases & known limits@@ -157,10 +165,11 @@ flowchart TD`writeTodos`) are removed when any task completes. This was introduced as a"CRITICAL FIX" because binding tools while instructing the LLM not to usethem confused Gemini into returning empty tool call arrays.-- **No pre-flight token budget check**: `_collect_agent_connections` can- produce an arbitrarily long prompt (task context + memory + skills). The- handler does **not** trim before calling `execute_agent`; the provider may- reject with `prompt_too_long`. See `docs-internal/memory_compaction.md`.+- **No pre-flight token budget check**: `collect_agent_connections` can+ produce an arbitrarily long prompt (task context + memory + skills).+ `prepare_agent_call` does **not** trim before calling `execute_agent`; the+ provider may reject with `prompt_too_long`. See+ [`../../memory_compaction.md`](../../memory_compaction.md).- **Input fallback is field-ordered**: `message > text > content > str(dict)`.An upstream node that outputs both `message` and `text` will never reach`text`.@@ -169,8 +178,9 @@ flowchart TD## Related-- **Shared collection helper**: `_collect_agent_connections` is used by every- specialized-agent type (android_agent, coding_agent, web_agent, ...). See+- **Shared collection helper**: `edge_walker.collect_agent_connections` ++ `_inline.prepare_agent_call` are used by every specialized-agent type+ (android_agent, coding_agent, web_agent, ...). See[`chatAgent.md`](./chatAgent.md) for the same contract with a differentsystem prompt.- **Memory node**: [`simpleMemory.md`](./simpleMemory.md)diff --git a/docs-internal/node-logic-flows/ai_agents/chatAgent.md b/docs-internal/node-logic-flows/ai_agents/chatAgent.mdindex 73157d65..99e8a85b 100644--- a/docs-internal/node-logic-flows/ai_agents/chatAgent.md+++ b/docs-internal/node-logic-flows/ai_agents/chatAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | ai_agents / agent |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/agent/chat_agent/__init__.py`](../../../server/nodes/agent/chat_agent/__init__.py) — dispatched via `BaseNode.execute()` + `@Operation("execute")` (`execute_op`). Pre-dispatch in [`_inline.py::prepare_agent_call`](../../../server/nodes/agent/_inline.py); LLM loop in `AIService.execute_chat_agent`. || **Tests** | [`server/tests/nodes/test_ai_agents.py`](../../../server/tests/nodes/test_ai_agents.py) || **Skill (if any)** | n/a (consumes skills via `input-skill`) || **Dual-purpose tool** | no |@@ -11,16 +11,18 @@## Purpose`chatAgent` (display name **Zeenie**) is the conversational variant of-`aiAgent`. Same connection model, same `_collect_agent_connections` helper,-same tool-calling stack - it differs from `aiAgent` in the frontend icon /-system prompt and in the service method it calls-(`AIService.execute_chat_agent` instead of `AIService.execute_agent`). It is-also the handler used by every **specialized agent** node+`aiAgent`. Same connection model, same `edge_walker.collect_agent_connections`+helper + `_inline.prepare_agent_call` pre-dispatch, same tool-calling stack -+it differs from `aiAgent` in the frontend icon / default system message and in+the service method it calls (`AIService.execute_chat_agent` instead of+`AIService.execute_agent`). Every **specialized agent** node(`android_agent`, `coding_agent`, `web_agent`, `task_agent`, `social_agent`,`travel_agent`, `tool_agent`, `productivity_agent`, `payments_agent`,-`consumer_agent`, `autonomous_agent`, `orchestrator_agent`, `ai_employee`) --the node registry wires all of those to `handle_chat_agent` via-`functools.partial`.+`consumer_agent`, `autonomous_agent`, `orchestrator_agent`, `ai_employee`)+takes the **same execution path** — they subclass+[`SpecializedAgentBase`](../../../server/nodes/agent/_specialized.py) whose+`execute_op` also runs `prepare_agent_call` + `execute_chat_agent`. Each is its+own plugin folder; there is no `functools.partial` wiring anymore.## Inputs (handles)@@ -35,13 +37,15 @@ the node registry wires all of those to `handle_chat_agent` via## Parameters-Identical to [`aiAgent`](./aiAgent.md) (`provider`, `model`, `prompt`,-`systemMessage`, `options.*`). The only frontend differences:+`ChatAgentParams` mirrors [`aiAgent`](./aiAgent.md)'s `AIAgentParams`+(`provider` Literal, `model`, `prompt`, `system_message`, `temperature` /+`max_tokens` in the `options` group). Differences:-- `prompt` default is `""` (empty) rather than `{{ $json.chatInput }}` - so- Zeenie relies on the auto-prompt fallback when wired to a chat trigger.-- `systemMessage` default is `"You are a helpful assistant."` (Zeenie persona- is layered on top by connected skills).+- `prompt` default `""` (empty) — Zeenie relies on the auto-prompt fallback+ when wired to a chat trigger (placeholder: "Optional: leave empty to use+ connected input").+- `system_message` default `""` (vs `aiAgent`'s "You are a helpful assistant")+ — the Zeenie persona is layered on top by connected skills.## Outputs (handles)@@ -53,7 +57,7 @@ Identical to [`aiAgent`](./aiAgent.md) (`provider`, `model`, `prompt`,```mermaidflowchart TD- A[handle_chat_agent] --> B[_collect_agent_connections]+ A[execute_op -> prepare_agent_call] --> B[edge_walker.collect_agent_connections]B --> C{task_data?}C -- yes --> D[Prepend task_context to prompt]D --> E{status completed/error<br/>AND tool_data?}@@ -64,7 +68,7 @@ flowchart TDG -- yes --> H[prompt := input_data.message<br/>or .text or .content or str]G -- no --> IH --> I{node_type in<br/>TEAM_LEAD_TYPES?}- I -- yes --> J[_collect_teammate_connections<br/>-> append each teammate to tool_data]+ I -- yes --> J[collect_teammate_connections<br/>-> append each teammate to tool_data]I -- no --> KJ --> K[await ai_service.execute_chat_agent<br/>memory/skill/tool/broadcaster/context/db]K --> L[Return envelope]@@ -72,21 +76,22 @@ flowchart TD## Decision Logic-- **Connection collection** is delegated to `_collect_agent_connections`; same- rules as `aiAgent` (see that doc for memory session derivation,- `masterSkill` expansion, Android toolkit, child-agent tool discovery).-- **Task context injection** mirrors `aiAgent`: `_format_task_context` wraps+- **Connection collection** is delegated to+ `edge_walker.collect_agent_connections`; same rules as `aiAgent` (see that doc+ for memory session derivation, `masterSkill` expansion, Android toolkit,+ child-agent tool discovery).+- **Task context injection** mirrors `aiAgent`: `format_task_context` wrapsthe task result as a plain-English instruction that the LLM must "reportnaturally", then all tools are stripped if the task has already completed- or errored.+ or errored. (In `_inline.prepare_agent_call`.)- **Auto-prompt fallback** is identical to `aiAgent` - `message` wins over`text` which wins over `content`, falling back to `str(input_data)`.-- **Team mode** (`orchestrator_agent`, `ai_employee`): after collection, the- handler calls `_collect_teammate_connections(node_id, context, database)`- to find nodes wired to `input-teammates`. Each teammate whose `node_type`- is in `AI_AGENT_TYPES` is appended to `tool_data` as a synthetic tool- entry. The AIService then exposes them to the LLM as `delegate_to_<type>`- tools (see [Agent Teams](../../agent_teams.md)).+- **Team mode** (`orchestrator_agent`, `ai_employee`): after collection,+ `prepare_agent_call` calls `collect_teammate_connections(node_id, context,+ database)` to find nodes wired to `input-teammates`. Each teammate is+ appended to `tool_data` as a synthetic entry (with `child_tools` describing+ its own `input-tools` neighbours). The AIService then exposes them to the LLM+ as `delegate_to_<type>` tools (see [Agent Teams](../../agent_teams.md)).## Side Effects@@ -111,8 +116,9 @@ flowchart TD## Edge cases & known limits-- **Specialized agents inherit every quirk**: any bug in `handle_chat_agent`- (e.g. the blanket tool-strip on task completion) affects 13 node types.+- **Specialized agents inherit every quirk**: any bug in `prepare_agent_call`+ or `execute_chat_agent` (e.g. the blanket tool-strip on task completion)+ affects all 13 specialized node types plus the two team leads.- **Team-lead teammates are appended after existing tools**: if a parentwires both regular tool nodes and teammates, tool order depends on edgescan order (not stable across clients).@@ -122,11 +128,12 @@ flowchart TDat delegate-time, not at collection-time.- **Input fallback field order is identical** to `aiAgent`: `message` >`text` > `content` > `str(dict)`.-- **Empty `teammates` on a team-lead is benign** - the handler simply runs- without delegation tools, so an `orchestrator_agent` with nothing wired to- `input-teammates` behaves exactly like `chatAgent`.-- **`rlm_agent` and `claude_code_agent` do NOT route here** - they have dedicated- handlers. All other specialized agents do.+- **Empty `teammates` on a team-lead is benign** - `prepare_agent_call` simply+ runs without delegation tools, so an `orchestrator_agent` with nothing wired+ to `input-teammates` behaves exactly like `chatAgent`.+- **`rlm_agent`, `claude_code_agent`, `codex_agent` do NOT take this path** -+ they have their own plugin execute methods (RLMService / CLI agent runtime).+ The other 13 specialized agents share `execute_chat_agent`.## Relateddiff --git a/docs-internal/node-logic-flows/ai_agents/simpleMemory.md b/docs-internal/node-logic-flows/ai_agents/simpleMemory.mdindex 0c821ac3..0c440a68 100644--- a/docs-internal/node-logic-flows/ai_agents/simpleMemory.md+++ b/docs-internal/node-logic-flows/ai_agents/simpleMemory.md@@ -2,8 +2,8 @@| Field | Value ||------|-------|-| **Category** | ai_agents / memory (UI group: `['tool', 'memory']`) |-| **Backend handler** | [`server/services/handlers/ai.py::handle_simple_memory`](../../../server/services/handlers/ai.py) |+| **Category** | ai_agents / memory (group: `('tool', 'memory')`, `component_kind = "model"`) |+| **Backend handler** | [`server/nodes/skill/simple_memory/__init__.py`](../../../server/nodes/skill/simple_memory/__init__.py) — dispatched via `BaseNode.execute()` + the `@Operation("read")` method (`read`). Agent-side consumption via [`edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) / `_build_memory_entry`. || **Tests** | [`server/tests/nodes/test_ai_agents.py`](../../../server/tests/nodes/test_ai_agents.py) || **Skill (if any)** | n/a || **Dual-purpose tool** | no |@@ -12,19 +12,20 @@`simpleMemory` is a **passive configuration node** that holds conversationhistory for connected agents. It has no inputs and is not meant to be run on-its own - it has no Run button in the UI. When an `aiAgent` / `chatAgent`-(or any specialized agent) executes, `_collect_agent_connections` reads this-node's saved parameters (`memoryContent`, `windowSize`, `longTermEnabled`,-etc.) and forwards them to `AIService.execute_agent` /-`execute_chat_agent`. The service parses the markdown, feeds the messages to-the LLM, appends the new exchange, trims to window, and saves the updated-markdown back to the memory node via `database.save_node_parameters`.--The actual handler (`handle_simple_memory`) is only invoked when a user-explicitly hits Run on the node; it returns an inspector-style snapshot of-the `services.memory_store` in-memory session, **which is a separate legacy-store from the markdown pipeline used by the agents**. See "Edge cases"-below.+its own — `ui_hints.hideRunButton` is set. When an `aiAgent` / `chatAgent`+(or any specialized agent) executes, `edge_walker.collect_agent_connections`+reads this node's saved parameters (`memory_content`, `window_size`,+`long_term_enabled`, etc.) via `_build_memory_entry` and forwards them to+`AIService.execute_agent` / `execute_chat_agent`. The service parses the+markdown, feeds the messages to the LLM, appends the new exchange, trims to+window, and saves the updated markdown back to the memory node via+`database.save_node_parameters`.++The plugin's own `@Operation("read")` method is only invoked when a user+explicitly hits Run (or via direct dispatch); it returns an inspector-style+snapshot of the `services.memory_store` in-memory session, **which is a+separate store from the markdown `memory_content` pipeline used by the+agents**. See "Edge cases" below.## Inputs (handles)@@ -32,103 +33,108 @@ _None._ `simpleMemory` has no incoming handles.## Parameters+Source: `SimpleMemoryParams` in+[`simple_memory/__init__.py`](../../../server/nodes/skill/simple_memory/__init__.py).+All field names are snake_case (Pydantic params are the post-Wave-11 source of+truth; the old `memoryType` / `clearOnRun` / buffer-vs-window split was removed+— trimming is always windowed and the UI exposes a manual "Clear Memory"+button).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `sessionId` | string | `""` | no | - | Session override. Empty or `default` -> agent node_id is used instead (see Decision Logic). |-| `windowSize` | number | `100` | no | - | Message *pairs* to keep in short-term memory; 1-100. |-| `memoryContent` | string | `# Conversation History\n\n*No messages yet.*\n` | no | - | Markdown body (`### **Human** (ts)` / `### **Assistant** (ts)` blocks). Edited in-place in the UI. |-| `longTermEnabled` | boolean | `false` | no | - | When true, trimmed messages are archived to an `InMemoryVectorStore`. |-| `retrievalCount` | number | `3` | no | `longTermEnabled=true` | Relevant messages to retrieve at query time (1-10). |+| `session_id` | string | `""` | no | - | Session override. Empty -> agent node_id is used by the agent-side collection; the `read` op resolves empty to `"default"`. |+| `window_size` | int | `100` | no | - | Message *pairs* to keep in short-term memory; 1-100. |+| `memory_content` | string (code editor, markdown, rows 15) | `# Conversation History\n\n*No messages yet.*\n` | no | - | Markdown body (`### **Human** (ts)` / `### **Assistant** (ts)` blocks). Edited in-place in the UI. |+| `long_term_enabled` | boolean | `false` | no | - | When true, trimmed messages are archived to an `InMemoryVectorStore`. |+| `retrieval_count` | int | `3` | no | `long_term_enabled=[true]` | Relevant messages to retrieve at query time (1-10). |+| `last_session_id` | string (optional, **hidden**) | `None` | no | - | Internal/display-only: last Claude session UUID. `claude_code_agent` no longer reads it; cleared when memory is wiped. |Parameters consumed when an agent pulls the memory:| Name | Where it comes from | Where it is read ||------|---------------------|------------------|-| `sessionId`, `windowSize`, `memoryContent`, `longTermEnabled`, `retrievalCount` | `database.get_node_parameters(source_node_id)` inside `_collect_agent_connections` | Packed into `memory_data` dict and forwarded to `AIService`. |+| `memory_content`, `window_size`, `long_term_enabled`, `retrieval_count`, `last_session_id` | `database.get_node_parameters(source_node_id)` inside `edge_walker._build_memory_entry` | Packed into the `memory_data` dict and forwarded to `AIService`. |-Parameters consumed when the node is run directly:+Parameters consumed when the node is run directly (`@Operation("read")`):-| Name | Type | Default | Description |-|------|------|---------|-------------|-| `sessionId` | string | `default` | Session key used for `services.memory_store`. |-| `memoryType` | string | `buffer` | Either `buffer` (all messages) or `window`. |-| `windowSize` | number | `10` | Window size; only applied when `memoryType == 'window'`. |-| `clearOnRun` | boolean | `false` | When true, clears the session before returning. |+| Name | Default | Description |+|------|---------|-------------|+| `session_id` | resolves to `"default"` when empty | Session key used for `services.memory_store.get_messages`. |+| `window_size` | `100` | Windowing applied to the message log. |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-memory` (`main` type) | object | Snapshot of the in-memory session (see below). |+| `output-memory` (output, top, role `memory`) | object | Snapshot of the in-memory session (see below). |-### Output payload (from `handle_simple_memory`)+### Output payload (`SimpleMemoryOutput` from the `read` op)```ts{- session_id: string;- messages: Array<{ role: 'human' | 'ai'; content: string; timestamp: string }>;- message_count: number;- memory_type: 'buffer' | 'window';- window_size: number | null;+ memory_content?: string; // echoes the editable markdown param+ message_count?: number;+ session_id?: string;+ messages?: any[]; // from services.memory_store.get_messages+ window_size?: number;+ // model_config extra="allow"}```-Wrapped in the standard envelope: `{ success: true, result: <payload> }`.+Serialized through `BaseNode._serialize_result` and wrapped in the standard+envelope: `{ success: true, result: <payload>, execution_time }`.## Logic Flow```mermaidflowchart TDsubgraph passiveUse["Passive use (agent collection)"]- P1[Agent handler scans edges] --> P2{target=input-memory<br/>source type=simpleMemory?}+ P1[edge_walker scans edges] --> P2{target=input-memory<br/>source type=simpleMemory?}P2 -- yes --> P3[database.get_node_parameters source_id]- P3 --> P4[memory_session_id = node_id<br/>unless sessionId explicit]- P4 --> P5[Build memory_data dict<br/>memoryContent/windowSize/...]+ P3 --> P4[memory_session_id = node_id<br/>unless session_id explicit]+ P4 --> P5[_build_memory_entry: memory_data dict<br/>memory_content/window_size/...]P5 --> P6[Forward to AIService]end- subgraph directRun["Direct run (handle_simple_memory)"]- D1[handle_simple_memory] --> D2[Read sessionId/memoryType/windowSize/clearOnRun]- D2 --> D3{clearOnRun?}- D3 -- yes --> D4[memory_store.clear_session]- D3 -- no --> D5- D4 --> D5[memory_store.get_messages<br/>session_id window_size]- D5 --> D6[Return success envelope]+ subgraph directRun["Direct run (Operation read)"]+ D1[read op] --> D2[session_id = session_id or default<br/>window_size]+ D2 --> D5[memory_store.get_messages<br/>session_id window_size]+ D5 --> D6[Return SimpleMemoryOutput envelope]end```## Decision Logic-- **Passive collection session key**: if the memory node's `sessionId` is- empty or literally `"default"`, the agent's own `node_id` is used as the- session key. This is what makes two agents wired to the same memory node- keep **separate** histories unless the user explicitly overrides- `sessionId`.-- **Instruction source of truth**: `memoryContent` in the DB is authoritative.+- **Passive collection session key**: if the memory node's `session_id` is+ empty (or `"default"`), the agent's own `node_id` is used as the session key.+ This is what makes two agents wired to the same memory node keep **separate**+ histories unless the user explicitly overrides `session_id`.+- **Instruction source of truth**: `memory_content` in the DB is authoritative.UI edits persist there via `save_node_parameters`. The agent side- re-serialises after each turn through `AIService._append_to_memory_markdown`- + `_trim_markdown_window`.-- **Window semantics**: `windowSize` counts *pairs* (Human + Assistant), so- `_trim_markdown_window` keeps `window_size * 2` blocks. Trimmed blocks are+ re-serialises after each turn through+ `services.memory.append_to_memory_markdown` + `trim_markdown_window`.+- **Window semantics**: `window_size` counts *pairs* (Human + Assistant), so+ `trim_markdown_window` keeps `window_size * 2` blocks. Trimmed blocks arereturned as raw text for optional vector-store archival.-- **Long-term retrieval**: `_get_memory_vector_store(session_id)` lazily- creates an `InMemoryVectorStore` with `HuggingFaceEmbeddings(BAAI/bge-small-en-v1.5)`- the first time it is requested for that session. On `ImportError` (e.g.- `langchain_huggingface` missing) the function returns `None` and the agent- silently skips archival - **no error is surfaced**.-- **Direct-run memory store is different**: `handle_simple_memory` pulls- from `services.memory_store._sessions`, **not** from the markdown- `memoryContent`. A user who clicks Run on a `simpleMemory` node that is- actively used by an agent will see the `memory_store` view, which may be- empty even though `memoryContent` contains history.+- **Long-term retrieval**: `services.memory.get_memory_vector_store(session_id)`+ lazily creates an `InMemoryVectorStore` with+ `HuggingFaceEmbeddings(BAAI/bge-small-en-v1.5)` the first time it is requested+ for that session. On `ImportError` (e.g. `langchain_huggingface` missing) the+ function returns `None` and the agent silently skips archival - **no error is+ surfaced**.+- **Direct-run memory store is different**: the `read` op pulls from+ `services.memory_store` (via `get_messages`), **not** from the markdown+ `memory_content`. A user who clicks Run on a `simpleMemory` node actively+ used by an agent sees the `memory_store` view, which may be empty even though+ `memory_content` contains history.## Side Effects-- **Database writes**: none in the handler. The agent pipeline writes the+- **Database writes**: none in the `read` op. The agent pipeline writes theupdated markdown back to this node's `node_parameters` row via`database.save_node_parameters` after each turn.- **Broadcasts**: none. The node is passive.-- **External API calls**: none in the handler. When `longTermEnabled` is set+- **External API calls**: none in the `read` op. When `long_term_enabled` is setand the agent archives, `HuggingFaceEmbeddings` loads the`BAAI/bge-small-en-v1.5` model locally (no network once cached).- **File I/O**: none. The vector store is `InMemoryVectorStore` - not@@ -146,33 +152,32 @@ flowchart TD## Edge cases & known limits-- **Two storage systems for one node**. The markdown `memoryContent` (used- by agents) and the `services.memory_store._sessions` dict (used by the- direct-run handler) are **independent**. Clicking Run on the node does not+- **Two storage systems for one node**. The markdown `memory_content` (used+ by agents) and the `services.memory_store` session log (used by the+ `read` op) are **independent**. Clicking Run on the node does notreflect what the agents see, and vice-versa. This is a known architecturalseam noted in the CLAUDE.md overview.-- **Vector store is in-memory only**. `_memory_vector_stores` is a module-- global dict keyed by session id. A server restart empties it; long-term- memory is not actually durable without external persistence.+- **Vector store is in-memory only**. The per-session cache in+ `services.memory.vector_store` is a module-global dict keyed by session id.+ A server restart empties it; long-term memory is not actually durable+ without external persistence.- **`HuggingFaceEmbeddings` import is silent-catch**. If the dependency is- missing at runtime, `_get_memory_vector_store` logs a warning and returns+ missing at runtime, `get_memory_vector_store` logs a warning and returns`None`; the agent continues without archival and the user never learns.-- **No token budgeting**. `windowSize` is a message-pair count, not a token+- **No token budgeting**. `window_size` is a message-pair count, not a tokencount. A long history below the window threshold can still exceed themodel's context window; see the "Prompt Too Long" note in CLAUDE.md.-- **`clearOnRun=true` only affects direct-run**, not the markdown used by- connected agents.-- **Markdown parser is regex-based**. `_parse_memory_markdown` matches+- **Markdown parser is regex-based**. `parse_memory_markdown` matches`### **Human**` / `### **Assistant**` exactly. If a user edits the UIcontent to a non-conforming header the messages will silently be dropped.-- **`windowSize` UI cap is 100**, but the handler accepts any int. The- agent-side `_trim_markdown_window` trusts whatever the parameter contains.+- **`window_size` is Pydantic-clamped 1-100** by the field constraint; the+ agent-side `trim_markdown_window` trusts whatever value is passed.- **Passive node, no envelope on read paths**. When consumed by an agent,this node never produces its own envelope - it only supplies parameters.## Related- **Consumer nodes**: [`aiAgent`](./aiAgent.md), [`chatAgent`](./chatAgent.md),- every specialized agent that routes to `handle_chat_agent`.+ every specialized agent that routes through `execute_chat_agent`.- **Architecture docs**: [Memory Compaction](../../memory_compaction.md),[Agent Architecture](../../agent_architecture.md)diff --git a/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.mdindex 0382ce65..47c410b1 100644--- a/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/anthropicChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/anthropic_chat_model/__init__.py`](../../../server/nodes/model/anthropic_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | yes - tool name `anthropic_chat_model` (advisor; `usable_as_tool = True`, group `('model', 'tool')`) |## Purpose-Single-turn chat completion against the Anthropic Messages API (Claude family). Shares the same handler as all other chat-model nodes (`handle_ai_chat_model`); provider-specific branching lives inside `AIService.execute_chat` and the native `services/llm/providers/anthropic.py`.+Single-turn chat completion against the Anthropic Messages API (Claude family). The `ChatModelBase.chat` operation (in `_base.py`) calls `AIService.execute_chat` for every chat-model node; provider-specific branching lives inside `execute_chat` and the native `services/llm/providers/anthropic.py`. Also wired as an AI-agent advisor tool (`usable_as_tool = True`) so an agent can consult a stronger model mid-task.## Inputs (handles)@@ -24,21 +24,23 @@ Single-turn chat completion against the Anthropic Messages API (Claude family).| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes (non-empty) | - | User message |-| `systemMessage` / `systemPrompt` / `system_prompt` | string | `""` | no | - | System prompt |-| `model` | string | injected from stored models, fallback `gpt-3.5-turbo` (Anthropic override via stored models list) | no | - | Anthropic model ID - **hyphenated** form (e.g. `claude-sonnet-4-6`, NOT `claude-sonnet-4.6`) |-| `temperature` | number | 0-1 range | no | - | Clamped to 0-1 for Claude; forced to 1 when thinking is enabled |-| `maxTokens` | number | model default | no | - | **Must be greater than `thinkingBudget`** when thinking is enabled |-| `topP` | number | - | no | - | Nucleus sampling |-| `thinkingEnabled` | boolean | false | no | - | Enable extended thinking |-| `thinkingBudget` | number | 2048 | no | `thinkingEnabled=[true]` | 1024-16000 tokens allocated to internal reasoning |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('anthropic', 'default')` |-| `options` | object | `{}` | no | - | Flattened into top-level params |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected from stored models) | no | - | Anthropic model ID - **hyphenated** form (e.g. `claude-sonnet-4-6`, NOT `claude-sonnet-4.6`) |+| `temperature` | number\|null | `null` -> falls through to `agent.default_temperature` | no | - | Clamped to 0-1 for Claude; forced to 1 when thinking is enabled |+| `max_tokens` | number\|null | `null` -> per-model default | no | - | 1-200000. **Must be greater than `thinking_budget`** when thinking is enabled |+| `top_p` | number\|null | `1.0` | no | - | Nucleus sampling (0-1) |+| `top_k` | number\|null | `40` | no | - | 1-100; Anthropic-specific |+| `thinking_enabled` | boolean | `false` | no | - | Enable extended thinking |+| `thinking_budget` | number\|null | `2048` | no | `thinking_enabled=[true]` | 1024-16000 tokens allocated to internal reasoning |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('anthropic', 'default')` |++(Field names are snake_case on the `AnthropicChatModelParams` Pydantic model; `model_config = ConfigDict(extra="ignore")` drops unknown keys.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload |### Output payload@@ -61,8 +63,8 @@ Wrapped in `{ success, node_id, node_type, result, execution_time }`.```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[_prepare_parameters]- B --> C[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]+ B --> C[get_ai_service]C --> D[AIService.execute_chat]D --> E{api_key + non-empty prompt?}E -- no --> X[ValueError -> error envelope]diff --git a/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.mdindex 99bc2bac..385cff46 100644--- a/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/cerebras_chat_model/__init__.py`](../../../server/nodes/model/cerebras_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | no (group `('model',)`) |## Purpose-Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1, GPT-OSS-120b, Qwen-3-235b. Shares `handle_ai_chat_model`. Like Groq, routes through the **LangChain fallback** (`is_native_provider('cerebras')` is False).+Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1, GPT-OSS-120b, Qwen-3-235b. The `ChatModelBase.chat` operation calls `AIService.execute_chat`. Like Groq, routes through the **LangChain fallback** (`is_native_provider('cerebras')` is False).## Inputs (handles)@@ -24,20 +24,23 @@ Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1,| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes | - | User message |-| `systemMessage` | string | `""` | no | - | System prompt |-| `model` | string | injected | no | - | e.g. `llama3.1-8b`, `gpt-oss-120b`, `qwen-3-235b-a22b` |-| `temperature` | number | 0-1.5 | no | - | Narrower range than OpenAI (0-1.5 rather than 0-2) |-| `maxTokens` | number | up to 8K | no | - | |-| `topP` | number | - | no | - | |-| `thinkingEnabled` | boolean | false | no | - | Only Qwen-3-235b supports format-based reasoning |-| `reasoningFormat` | options | `parsed` | no | `thinkingEnabled=[true]` | `parsed` / `hidden` - same semantics as Groq Qwen |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('cerebras', 'default')` |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | e.g. `llama3.1-8b`, `gpt-oss-120b`, `qwen-3-235b-a22b` |+| `temperature` | number\|null | `null` | no | - | Narrower range than OpenAI (0-1.5 rather than 0-2) |+| `max_tokens` | number\|null | `null` (up to 8K) | no | - | 1-200000 |+| `top_p` | number\|null | `1.0` | no | - | |+| `thinking_enabled` | boolean | `false` | no | - | Only Qwen-3-235b supports format-based reasoning |+| `thinking_budget` | number\|null | `2048` | no | `thinking_enabled=[true]` | 1024-16000 (Cerebras Qwen budget) |+| `reasoning_format` | enum | `parsed` | no | - | `parsed` / `hidden` - same semantics as Groq Qwen (inherited base field) |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('cerebras', 'default')` |++(Field names are snake_case on `CerebrasChatModelParams`; unknown keys ignored.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output; standard envelope payload |### Output payload@@ -58,7 +61,7 @@ Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1,```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]B --> C[AIService.execute_chat]C --> D{valid key + prompt?}D -- no --> X[error envelope]diff --git a/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.mdindex f381b595..4c5ae5c7 100644--- a/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/deepseekChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/deepseek_chat_model/__init__.py`](../../../server/nodes/model/deepseek_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | no (group `('model',)`) |## Purpose-DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / `deepseek-reasoner` remain as legacy aliases (deprecate 2026-07-24). Uses the OpenAI-compatible DeepSeek endpoint via the `services/llm/providers` layer (native path). Shares `handle_ai_chat_model`.+DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / `deepseek-reasoner` remain as legacy aliases (deprecate 2026-07-24). Uses the OpenAI-compatible DeepSeek endpoint via the `services/llm/providers` layer (native path). The `ChatModelBase.chat` operation calls `AIService.execute_chat`. The plugin docstring describes `deepseek-chat`/`deepseek-reasoner` (V3) but the registry description in `__init__.py` predates the V4 rename - the card's V4 listing reflects current `llm_defaults.json`.## Inputs (handles)@@ -24,18 +24,22 @@ DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / `| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes | - | User message |-| `systemMessage` | string | `""` | no | - | System prompt |-| `model` | string | injected | no | - | `deepseek-v4-flash` / `deepseek-v4-pro`; `deepseek-chat` / `deepseek-reasoner` legacy aliases (reasoner = always-on CoT) |-| `temperature` | number | 0-2 | no | - | |-| `maxTokens` | number | 8-64K | no | - | |-| `topP` | number | - | no | - | |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('deepseek', 'default')` |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | `deepseek-v4-flash` / `deepseek-v4-pro`; `deepseek-chat` / `deepseek-reasoner` legacy aliases (reasoner = always-on CoT) |+| `temperature` | number\|null | `null` | no | - | 0-2 |+| `max_tokens` | number\|null | `null` (8-64K) | no | - | 1-200000 |+| `top_p` | number\|null | `1.0` | no | - | |+| `frequency_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 (DeepSeek-specific) |+| `presence_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 (DeepSeek-specific) |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('deepseek', 'default')` |++(Field names are snake_case on `DeepseekChatModelParams`; unknown keys ignored.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output; standard envelope payload |### Output payload@@ -56,7 +60,7 @@ DeepSeek V4 models (`deepseek-v4-flash`, `deepseek-v4-pro`); `deepseek-chat` / ````mermaidflowchart TD- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]B --> C[AIService.execute_chat]C --> D{valid key + prompt?}D -- no --> X[error envelope]diff --git a/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.mdindex 65366c6e..9df2c6c1 100644--- a/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/gemini_chat_model/__init__.py`](../../../server/nodes/model/gemini_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | yes - tool name `gemini_chat_model` (advisor; `usable_as_tool = True`, group `('model', 'tool')`) |## Purpose-Single-turn chat completion against Google's Gemini API (google-genai native SDK). Shares the common `handle_ai_chat_model` handler.+Single-turn chat completion against Google's Gemini API (google-genai native SDK). The `ChatModelBase.chat` operation calls `AIService.execute_chat`. Also wired as an AI-agent advisor tool (`usable_as_tool = True`) so an agent can consult Gemini mid-task.## Inputs (handles)@@ -24,22 +24,24 @@ Single-turn chat completion against Google's Gemini API (google-genai native SDK| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes (non-empty) | - | User message |-| `systemMessage` | string | `""` | no | - | System prompt |-| `model` | string | injected | no | - | e.g. `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-3-pro` |-| `temperature` | number | 0-2 | no | - | 0-2 range |-| `maxTokens` | number | clamped to model ceiling | no | - | Up to 65K output |-| `topP` | number | - | no | - | |-| `thinkingEnabled` | boolean | false | no | - | Enable thinking (2.5 Pro/Flash, 3.x) |-| `thinkingBudget` | number | 2048 | no | `thinkingEnabled=[true]` | Token budget for internal reasoning |-| `safetySettings` | object | provider defaults | no | - | Gemini safety category thresholds |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('gemini', 'default')` |-| `options` | object | `{}` | no | - | Flattened |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | e.g. `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-3-pro` |+| `temperature` | number\|null | `null` | no | - | 0-2 range |+| `max_tokens` | number\|null | `null` (clamped to ceiling) | no | - | 1-200000; up to 65K output |+| `top_p` | number\|null | `1.0` | no | - | |+| `top_k` | number\|null | `40` | no | - | 1-100; Gemini-specific |+| `thinking_enabled` | boolean | `false` | no | - | Enable thinking (2.5 Pro/Flash, 3.x) |+| `thinking_budget` | number\|null | `2048` | no | `thinking_enabled=[true]` | 1024-16000 token budget for internal reasoning |+| `safety_settings` | enum | `default` | no | - | `default` / `strict` / `permissive` |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('gemini', 'default')` |++(Field names are snake_case on `GeminiChatModelParams`; unknown keys ignored.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output; standard envelope payload |### Output payload@@ -60,8 +62,8 @@ Single-turn chat completion against Google's Gemini API (google-genai native SDK```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[_prepare_parameters]- B --> C[handle_ai_chat_model -> AIService.execute_chat]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]+ B --> C[AIService.execute_chat]C --> D{valid key + prompt?}D -- no --> X[error envelope]D -- yes --> E[detect_ai_provider -> 'gemini']diff --git a/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.mdindex 4e46edf2..70268fbb 100644--- a/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/groq_chat_model/__init__.py`](../../../server/nodes/model/groq_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | no (group `('model',)`) |## Purpose-Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen3-32b, GPT-OSS. Shares `handle_ai_chat_model`; unlike OpenAI/Anthropic/Gemini, Groq routes through the **LangChain fallback** (`create_model` + `chat_model.invoke`) because `is_native_provider('groq')` returns False.+Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen3-32b, GPT-OSS. The `ChatModelBase.chat` operation calls `AIService.execute_chat`; unlike OpenAI/Anthropic/Gemini, Groq routes through the **LangChain fallback** (`create_model` + `chat_model.invoke`) because `is_native_provider('groq')` returns False.## Inputs (handles)@@ -24,20 +24,22 @@ Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes | - | User message |-| `systemMessage` | string | `""` | no | - | System prompt |-| `model` | string | injected | no | - | e.g. `llama-3.1-70b-versatile`, `qwen/qwen3-32b`, `groq/compound-beta` |-| `temperature` | number | 0-2 | no | - | |-| `maxTokens` | number | 8-131K per model | no | - | |-| `topP` | number | - | no | - | |-| `thinkingEnabled` | boolean | false | no | - | Only Qwen3-32b supports reasoning |-| `reasoningFormat` | options | `parsed` | no | `thinkingEnabled=[true]` | `parsed` (returns reasoning) or `hidden` (suppresses it) |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('groq', 'default')` |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | e.g. `llama-3.1-70b-versatile`, `qwen/qwen3-32b`, `groq/compound-beta` |+| `temperature` | number\|null | `null` | no | - | 0-2 |+| `max_tokens` | number\|null | `null` (8-131K per model) | no | - | 1-200000 |+| `top_p` | number\|null | `1.0` | no | - | |+| `thinking_enabled` | boolean | `false` | no | - | Only Qwen3-32b supports reasoning |+| `reasoning_format` | enum | `parsed` | no | `thinking_enabled=[true]` | `parsed` (returns reasoning) or `hidden` (suppresses it) |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('groq', 'default')` |++(Field names are snake_case on `GroqChatModelParams`; unknown keys ignored.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output; standard envelope payload |### Output payload@@ -58,7 +60,7 @@ Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]B --> C[AIService.execute_chat]C --> D{valid key + prompt?}D -- no --> X[error envelope]diff --git a/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.mdindex 7c6038dd..2e62dea1 100644--- a/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/kimiChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/kimi_chat_model/__init__.py`](../../../server/nodes/model/kimi_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | no (group `('model',)`) |## Purpose-Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K context, 96K output. Uses OpenAI-compatible Moonshot endpoint via the native path. Shares `handle_ai_chat_model`.+Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K context, 96K output. Uses OpenAI-compatible Moonshot endpoint via the native path. `KimiChatModelNode` uses the shared `ChatModelParams` unchanged (no provider override). The `ChatModelBase.chat` operation calls `AIService.execute_chat`.## Inputs (handles)@@ -24,18 +24,21 @@ Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K co| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes | - | User message |-| `systemMessage` | string | `""` | no | - | System prompt |-| `model` | string | injected | no | - | `kimi-k2.6`, `kimi-k2.5`, or `kimi-k2.7-code` |-| `temperature` | number | **fixed** | no | - | Fixed at 0.6 (instant) or 1.0 (thinking). Input is ignored. |-| `maxTokens` | number | up to 96K | no | - | |-| `thinkingEnabled` | boolean | true for k2.5 | no | - | ON by default for k2.5; explicitly disabled for tool-calling agents |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('kimi', 'default')` |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | `kimi-k2.6`, `kimi-k2.5`, or `kimi-k2.7-code` |+| `temperature` | number\|null | `null` (**ignored**) | no | - | Provider forces 0.6 (instant) or 1.0 (thinking); user input ignored |+| `max_tokens` | number\|null | `null` (up to 96K) | no | - | 1-200000 |+| `top_p` | number\|null | `1.0` | no | - | |+| `thinking_enabled` | boolean | `false` (Params default) | no | - | Base default is `false`; the provider treats k2.5 as thinking-on unless explicitly disabled |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('kimi', 'default')` |++(Kimi uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output; standard envelope payload |### Output payload@@ -56,7 +59,7 @@ Kimi models by Moonshot AI (`kimi-k2.6`, `kimi-k2.5`, `kimi-k2.7-code`). 256K co```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]B --> C[AIService.execute_chat]C --> D{valid key + prompt?}D -- no --> X[error envelope]diff --git a/docs-internal/node-logic-flows/ai_chat_models/lmstudioChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/lmstudioChatModel.mdnew file mode 100644index 00000000..438a089f--- /dev/null+++ b/docs-internal/node-logic-flows/ai_chat_models/lmstudioChatModel.md@@ -0,0 +1,107 @@+# LM Studio Chat Model (`lmstudioChatModel`)++| Field | Value |+|------|-------|+| **Category** | ai_chat_models |+| **Backend handler** | [`server/nodes/model/lmstudio_chat_model/__init__.py`](../../../server/nodes/model/lmstudio_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |+| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |+| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |+| **Skill (if any)** | n/a |+| **Dual-purpose tool** | no (group `('model',)`) |++## Purpose++Run local LLMs through LM Studio's OpenAI-compatible server (default `http://localhost:1234/v1`). The existing OpenAI-compatible fallback in `services/llm/factory.py` routes it through `OpenAIProvider` with `base_url` from `llm_defaults.json` — same path as deepseek/kimi/mistral/ollama. The user's custom server URL is stored as the `lmstudio_proxy` credential. `LMStudioChatModelNode` uses the shared `ChatModelParams` unchanged. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.++## Inputs (handles)++| Handle | Connection type | Required | Purpose |+|--------|-----------------|----------|---------|+| `input-main` | main | no | Upstream data; not consumed directly |++## Parameters++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `prompt` | string | `""` | yes | - | User message |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | Whatever the user has loaded in the LM Studio UI. Open-world: name not pattern-checked by `is_model_valid_for_provider` |+| `temperature` | number\|null | `null` | no | - | 0-2 |+| `max_tokens` | number\|null | `null` | no | - | 1-200000; default per-loaded-model ctx ÷ 4 (capped 4096) |+| `top_p` | number\|null | `1.0` | no | - | |+| `api_key` | string\|null | `null` -> placeholder `"ollama"` | no | - | Optional; local servers usually run with no auth. `LMStudioCredential.resolve()` returns `"ollama"` when none stored |++(LM Studio uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)++## Outputs (handles)++| Handle | Shape | Description |+|--------|-------|-------------|+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload |++### Output payload++```ts+{+ response: string;+ thinking: string | null; // per-model+ thinking_enabled: boolean;+ model: string;+ provider: 'lmstudio';+ finish_reason: string;+ timestamp: string;+ input: { prompt: string; system_prompt: string };+}+```++Wrapped in `{ success, node_id, node_type, result, execution_time }`.++## Logic Flow++```mermaid+flowchart TD+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]+ B --> C[AIService.execute_chat]+ C --> D{valid key + prompt?}+ D -- no --> X[error envelope]+ D -- yes --> E[detect_ai_provider -> 'lmstudio']+ E --> F[Lookup lmstudio_proxy credential -> base_url override]+ F --> G[create_provider lmstudio -> OpenAIProvider<br/>base_url=user server, api_key='ollama']+ G --> H[provider.chat]+ H --> I[success envelope]+ G -- Exception --> X+```++## Decision Logic++- **Validation**: empty prompt -> error envelope. `api_key` is never the blocker — `LMStudioCredential.resolve()` returns the placeholder `"ollama"` when no key is stored, so the central "API key required" check in `execute_chat` passes.+- **Provider routing**: `detect_ai_provider` MUST list `lmstudio` (in `server/constants.py`) or the node falls through to `'openai'` and `execute_chat` hits api.openai.com with the placeholder key.+- **Open-world model name**: `is_model_valid_for_provider` returns `True` for `lmstudio` so local model names are not rejected by the cloud-style pattern check.+- **Base URL routing**: `lmstudio_proxy` credential carries the user's server URL into `OpenAIProvider.base_url`; traffic stays on `localhost`.++## Side Effects++- **Database writes**: per-model context params persist in `EncryptedAPIKey.models["model_params"]` and `model_registry.json` at credential-validation time (via `_local_validator.py` + `model_registry.register_local_model()`), not on the bare chat path.+- **Broadcasts**: none on the bare chat path.+- **External API calls**: `POST {user_server}/v1/chat/completions` via the `openai` SDK with overridden `base_url` (default `http://localhost:1234/v1`).+- **File I/O**: none.+- **Subprocess**: none.++## External Dependencies++- **Credentials**: optional `auth_service.get_api_key('lmstudio')`; user server URL stored as `lmstudio_proxy`.+- **Services**: `services/llm/providers/openai.py` (reused with LM Studio base_url); `nodes/model/_local_validator.py` (SDK probe via `lmstudio.AsyncClient.llm.list_loaded()`).+- **Python packages**: `openai`, `lmstudio>=1.5.0` (validation only).+- **Environment variables**: none.++## Edge cases & known limits++- **Server must be running with a model loaded**: LM Studio reports loaded models via `list_loaded()`; if nothing is loaded the probe returns no models.+- **Provider routing dependency**: `lmstudio` must be present in `detect_ai_provider` AND in each agent's `provider` Literal, or the node silently falls back to OpenAI cloud.+- **Max output default**: ctx ÷ 4, capped at 4096, unless the user overrides `max_tokens`. Typed `LlmInstanceInfo.context_length` drives this.+- **Errors swallowed into envelope** on the chat path.++## Related++- **Peer nodes**: [`ollamaChatModel`](./ollamaChatModel.md) (other local-server provider), and the cloud chat-model docs in this folder.+- **Architecture docs**: [Native LLM SDK](../../native_llm_sdk.md).diff --git a/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.mdindex 7db34193..ab423a24 100644--- a/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/mistralChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/mistral_chat_model/__init__.py`](../../../server/nodes/model/mistral_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | no (group `('model',)`) |## Purpose-Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest`). Up to 256K context, 131K output. No thinking/reasoning support. Uses OpenAI-compatible Mistral endpoint via native path. Shares `handle_ai_chat_model`.+Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest`). Up to 256K context, 131K output. No thinking/reasoning support. Uses OpenAI-compatible Mistral endpoint via native path. `MistralChatModelNode` uses the shared `ChatModelParams` unchanged. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.## Inputs (handles)@@ -24,18 +24,20 @@ Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-sma| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes | - | User message |-| `systemMessage` | string | `""` | no | - | System prompt |-| `model` | string | injected | no | - | `mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest` |-| `temperature` | number | 0-1.5 | no | - | Narrower range than OpenAI (0-1.5) |-| `maxTokens` | number | up to 131K | no | - | |-| `topP` | number | - | no | - | |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('mistral', 'default')` |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | `mistral-large-latest`, `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest` |+| `temperature` | number\|null | `null` | no | - | Narrower range than OpenAI (0-1.5) |+| `max_tokens` | number\|null | `null` (up to 131K) | no | - | 1-200000 |+| `top_p` | number\|null | `1.0` | no | - | |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('mistral', 'default')` |++(Mistral uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output; standard envelope payload |### Output payload@@ -56,7 +58,7 @@ Mistral AI models (`mistral-large-latest`, `mistral-medium-latest`, `mistral-sma```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]B --> C[AIService.execute_chat]C --> D{valid key + prompt?}D -- no --> X[error envelope]diff --git a/docs-internal/node-logic-flows/ai_chat_models/ollamaChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/ollamaChatModel.mdnew file mode 100644index 00000000..54c22253--- /dev/null+++ b/docs-internal/node-logic-flows/ai_chat_models/ollamaChatModel.md@@ -0,0 +1,109 @@+# Ollama Chat Model (`ollamaChatModel`)++| Field | Value |+|------|-------|+| **Category** | ai_chat_models |+| **Backend handler** | [`server/nodes/model/ollama_chat_model/__init__.py`](../../../server/nodes/model/ollama_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) |+| **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) |+| **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |+| **Skill (if any)** | n/a |+| **Dual-purpose tool** | no (group `('model',)`) |++## Purpose++Run local LLMs (llama, mistral, qwen, deepseek-r1, ...) through a locally-running Ollama server. Ollama exposes an OpenAI-shaped `/v1` endpoint, so the existing OpenAI-compatible fallback in `services/llm/factory.py` hands it to `OpenAIProvider` with `base_url` from `llm_defaults.json` — same path as deepseek/kimi/mistral. The user's custom server URL (if not localhost) is stored as the `ollama_proxy` credential and flows through the same `proxy_url` parameter cloud providers already use for Ollama-style auth delegation. `OllamaChatModelNode` uses the shared `ChatModelParams` unchanged. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.++## Inputs (handles)++| Handle | Connection type | Required | Purpose |+|--------|-----------------|----------|---------|+| `input-main` | main | no | Upstream data; not consumed directly |++## Parameters++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `prompt` | string | `""` | yes | - | User message |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | Whatever the user has pulled, e.g. `qwen2.5`, `llama3.x`, `deepseek-r1`. Open-world: name not pattern-checked by `is_model_valid_for_provider` |+| `temperature` | number\|null | `null` | no | - | 0-2 |+| `max_tokens` | number\|null | `null` | no | - | 1-200000; default per-loaded-model ctx ÷ 4 (capped 4096) |+| `top_p` | number\|null | `1.0` | no | - | |+| `api_key` | string\|null | `null` -> placeholder `"ollama"` | no | - | Optional; local servers usually run with no auth. `OllamaCredential.resolve()` returns `"ollama"` when none stored |++(Ollama uses the shared `ChatModelParams` unchanged; field names are snake_case, unknown keys ignored.)++## Outputs (handles)++| Handle | Shape | Description |+|--------|-------|-------------|+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload |++### Output payload++```ts+{+ response: string;+ thinking: string | null; // per-model; Ollama has no generic thinking knob+ thinking_enabled: boolean;+ model: string;+ provider: 'ollama';+ finish_reason: string;+ timestamp: string;+ input: { prompt: string; system_prompt: string };+}+```++Wrapped in `{ success, node_id, node_type, result, execution_time }`.++## Logic Flow++```mermaid+flowchart TD+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]+ B --> C[AIService.execute_chat]+ C --> D{valid key + prompt?}+ D -- no --> X[error envelope]+ D -- yes --> E[detect_ai_provider -> 'ollama']+ E --> F[Lookup ollama_proxy credential -> base_url override]+ F --> G[create_provider ollama -> OpenAIProvider<br/>base_url=user server, api_key='ollama']+ G --> H[provider.chat]+ H --> I[success envelope]+ G -- Exception --> X+```++## Decision Logic++- **Validation**: empty prompt -> error envelope. `api_key` is never the blocker — `OllamaCredential.resolve()` returns the placeholder `"ollama"` when no key is stored, so the central "API key required" check in `execute_chat` passes.+- **Provider routing**: `detect_ai_provider` MUST list `ollama` (in `server/constants.py`) or the node falls through to `'openai'` and `execute_chat` hits api.openai.com with the placeholder key.+- **Open-world model name**: `is_model_valid_for_provider` returns `True` for `ollama` so local model names like `qwen/qwen3.6-27b` are not rejected by the cloud-style pattern check.+- **Base URL routing**: `ollama_proxy` credential carries the user's server URL into `OpenAIProvider.base_url`; traffic stays on `localhost`.+- **No thinking knob**: the shared `thinking_enabled` field is present but generic Ollama has no per-call thinking parameter; reasoning is per-model.++## Side Effects++- **Database writes**: per-model context params persist in `EncryptedAPIKey.models["model_params"]` and `model_registry.json` at credential-validation time (via `_local_validator.py` + `model_registry.register_local_model()`), not on the bare chat path.+- **Broadcasts**: none on the bare chat path.+- **External API calls**: `POST {user_server}/v1/chat/completions` via the `openai` SDK with overridden `base_url` (default `http://localhost:11434/v1`).+- **File I/O**: none.+- **Subprocess**: none.++## External Dependencies++- **Credentials**: optional `auth_service.get_api_key('ollama')`; user server URL stored as `ollama_proxy`.+- **Services**: `services/llm/providers/openai.py` (reused with Ollama base_url); `nodes/model/_local_validator.py` (SDK probe via `ollama.AsyncClient.ps()`).+- **Python packages**: `openai`, `ollama>=0.6.0` (validation only).+- **Environment variables**: none.++## Edge cases & known limits++- **Server must be running**: requests go to the user's local Ollama server; if it is down the request surfaces as a connection error in the envelope.+- **Provider routing dependency**: `ollama` must be present in `detect_ai_provider` AND in each agent's `provider` Literal, or the node silently falls back to OpenAI cloud.+- **Max output default**: ctx ÷ 4, capped at 4096, unless the user overrides `max_tokens`.+- **Per-model params survive restart**: written through to `model_registry.json` so context length is known without re-clicking Fetch.+- **Errors swallowed into envelope** on the chat path.++## Related++- **Peer nodes**: [`lmstudioChatModel`](./lmstudioChatModel.md) (other local-server provider), and the cloud chat-model docs in this folder.+- **Architecture docs**: [Native LLM SDK](../../native_llm_sdk.md).diff --git a/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.mdindex fe2cc4fd..5248b23d 100644--- a/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/openaiChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/openai_chat_model/__init__.py`](../../../server/nodes/model/openai_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) |-| **Skill (if any)** | n/a - chat models are model config nodes, not tools |-| **Dual-purpose tool** | no - pure chat completion node |+| **Skill (if any)** | n/a |+| **Dual-purpose tool** | yes - tool name `openai_chat_model` (advisor; `usable_as_tool = True`, group `('model', 'tool')`) |## Purpose-Single-turn chat completion against the OpenAI REST API. Used either as a standalone workflow node (user enters a prompt, the node produces a response) or as a model config node wired into an AI Agent via the `input-model` handle. All 9 chat-model node types share the same handler (`handle_ai_chat_model`) and the same dispatch path; per-provider differences live inside `AIService.execute_chat` (native SDK for OpenAI) and in the clamp / temperature helpers.+Single-turn chat completion against the OpenAI REST API. Used either as a standalone workflow node (user enters a prompt, the node produces a response), wired into an AI Agent via the `input-model` handle, or connected as an advisor tool (`usable_as_tool = True`). Every chat-model node runs through the shared `ChatModelBase.chat` operation (in `_base.py`), which calls `AIService.execute_chat`; per-provider differences live inside `execute_chat` (native SDK for OpenAI) and in the clamp / temperature helpers.## Inputs (handles)@@ -24,24 +24,25 @@ Single-turn chat completion against the OpenAI REST API. Used either as a standa| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes (non-empty) | - | User message content sent to the model |-| `systemMessage` / `systemPrompt` / `system_prompt` | string | `""` | no | - | System prompt; any of the three naming conventions accepted |-| `model` | string | auto-populated from stored models, fallback `gpt-3.5-turbo` | no | - | Model ID. `[FREE] ` prefix is stripped before the API call |-| `temperature` | number | provider default | no | - | 0-2 for most OpenAI models; o-series (o1/o3/o4) force temperature=1 |-| `maxTokens` / `max_tokens` | number | provider default | no | - | Clamped via `_resolve_max_tokens` to the model's actual context limit |-| `topP` | number | 1.0 | no | - | Nucleus sampling |-| `frequencyPenalty` | number | 0 | no | - | OpenAI penalty |-| `presencePenalty` | number | 0 | no | - | OpenAI penalty |-| `responseFormat` | options | `text` | no | - | `text` or `json_object` |-| `thinkingEnabled` | boolean | false | no | - | Turn on extended reasoning (GPT-5 hybrid / o-series) |-| `reasoningEffort` | options | `medium` | no | `thinkingEnabled=[true]` | `minimal` / `low` / `medium` / `high` |-| `apiKey` / `api_key` | string | injected by executor | no | - | Auto-injected by `NodeExecutor._inject_api_keys` via `auth_service.get_api_key('openai', 'default')` |-| `options` | object | `{}` | no | - | Collection bag; flattened into the top-level params dict before use |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected from stored models) | no | - | Model ID. `[FREE] ` prefix is stripped before the API call |+| `temperature` | number\|null | `null` -> `agent.default_temperature` | no | - | 0-2 for most OpenAI models; o-series (o1/o3/o4) force temperature=1 |+| `max_tokens` | number\|null | `null` -> per-model default | no | - | 1-200000; clamped via `_resolve_max_tokens` to the model's actual ceiling |+| `top_p` | number\|null | `1.0` | no | - | Nucleus sampling |+| `frequency_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 |+| `presence_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0 |+| `response_format` | enum | `text` | no | - | `text` or `json_object` |+| `thinking_enabled` | boolean | `false` | no | - | Turn on extended reasoning (GPT-5 hybrid / o-series) |+| `reasoning_effort` | enum | `medium` | no | `thinking_enabled=[true]` | `minimal` / `low` / `medium` / `high` (OpenAI override adds `minimal`) |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('openai', 'default')` |++(Field names are snake_case on `OpenAIChatModelParams`; `model_config = ConfigDict(extra="ignore")` drops unknown keys.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload (see below) |+| `output-model` | object | Model output (also feeds an agent's `input-model` handle); standard envelope payload (see below) |### Output payload@@ -64,8 +65,8 @@ Wrapped in the standard envelope: `{ success: true, node_id, node_type, result:```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[_prepare_parameters:<br/>merge DB params, validate, inject api_key + model]- B --> C[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation<br/>params validated by Pydantic]+ B --> C[get_ai_service]C --> D[AIService.execute_chat]D --> E[Flatten options into flat params dict]E --> F{api_key present?}diff --git a/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.mdindex e9c9bf8c..6eb02bef 100644--- a/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.md+++ b/docs-internal/node-logic-flows/ai_chat_models/openrouterChatModel.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | ai_chat_models |-| **Backend handler** | [`server/services/handlers/ai.py::handle_ai_chat_model`](../../../server/services/handlers/ai.py) |+| **Backend handler** | [`server/nodes/model/openrouter_chat_model/__init__.py`](../../../server/nodes/model/openrouter_chat_model/__init__.py) (dispatch via `BaseNode.execute()` -> `@Operation("chat")` in [`server/nodes/model/_base.py`](../../../server/nodes/model/_base.py)) || **AI service** | [`server/services/ai.py::AIService.execute_chat`](../../../server/services/ai.py) || **Tests** | [`server/tests/nodes/test_ai_chat_models.py`](../../../server/tests/nodes/test_ai_chat_models.py) || **Skill (if any)** | n/a |-| **Dual-purpose tool** | no |+| **Dual-purpose tool** | no (group `('model',)`) |## Purpose-Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google, Meta, Mistral, etc.) through a single OpenAI-compatible API. Shares the common `handle_ai_chat_model` handler.+Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google, Meta, Mistral, etc.) through a single OpenAI-compatible API. The `ChatModelBase.chat` operation calls `AIService.execute_chat`.## Inputs (handles)@@ -24,23 +24,25 @@ Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `prompt` | string | `""` | yes (non-empty) | - | User message |-| `systemMessage` | string | `""` | no | - | System prompt |-| `model` | string | injected | no | - | `provider/model-id` format, e.g. `anthropic/claude-opus-4.6`, `openai/gpt-4o`. `[FREE] ` prefix stripped before the API call but the prefix IS preserved for the dropdown grouping |-| `temperature` | number | 0-2 | no | - | |-| `maxTokens` | number | varies per model | no | - | |-| `topP` | number | - | no | - | |-| `frequencyPenalty` | number | 0 | no | - | Forwarded to downstream provider |-| `presencePenalty` | number | 0 | no | - | Forwarded |-| `thinkingEnabled` | boolean | false | no | - | Only honored if the routed model supports it |-| `thinkingBudget` | number | 2048 | no | `thinkingEnabled=[true]` | |-| `reasoningEffort` | options | `medium` | no | `thinkingEnabled=[true]` | |-| `apiKey` | string | injected | no | - | `auth_service.get_api_key('openrouter', 'default')` |+| `system_prompt` | string | `""` | no | - | System prompt |+| `model` | string | `""` (injected) | no | - | `provider/model-id` format, e.g. `anthropic/claude-opus-4.6`, `openai/gpt-4o`. `[FREE] ` prefix stripped before the API call but preserved for dropdown grouping |+| `temperature` | number\|null | `null` | no | - | 0-2 |+| `max_tokens` | number\|null | `null` (varies per model) | no | - | 1-200000 |+| `top_p` | number\|null | `1.0` | no | - | |+| `frequency_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0; forwarded to downstream provider |+| `presence_penalty` | number\|null | `0.0` | no | - | -2.0 to 2.0; forwarded |+| `thinking_enabled` | boolean | `false` | no | - | Only honored if the routed model supports it |+| `thinking_budget` | number\|null | `2048` | no | - | Inherited base field (no displayOptions); 1024-16000 |+| `reasoning_effort` | enum\|null | `null` | no | - | Inherited base field (no displayOptions); `low`/`medium`/`high` |+| `api_key` | string\|null | `null` (injected) | no | - | `auth_service.get_api_key('openrouter', 'default')` |++(Only `frequency_penalty` / `presence_penalty` are OpenRouter overrides; the thinking/reasoning fields come from the shared base `ChatModelParams` with no displayOptions. Field names are snake_case, unknown keys ignored.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |+| `output-model` | object | Model output; standard envelope payload |### Output payload@@ -61,7 +63,7 @@ Unified access to 200+ models from multiple providers (OpenAI, Anthropic, Google```mermaidflowchart TD- A[NodeExecutor dispatch] --> B[handle_ai_chat_model]+ A[NodeExecutor dispatch -> BaseNode.execute] --> B[ChatModelBase.chat Operation]B --> C[AIService.execute_chat]C --> D[Strip '[FREE] ' prefix from model]D --> E{valid key + prompt?}diff --git a/docs-internal/node-logic-flows/ai_tools/agentBuilder.md b/docs-internal/node-logic-flows/ai_tools/agentBuilder.mdnew file mode 100644index 00000000..65b15e2a--- /dev/null+++ b/docs-internal/node-logic-flows/ai_tools/agentBuilder.md@@ -0,0 +1,132 @@+# Agent Builder (`agentBuilder`)++| Field | Value |+|------|-------|+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |+| **Backend handler** | [`server/nodes/tool/agent_builder/__init__.py`](../../../server/nodes/tool/agent_builder/__init__.py) — `AgentBuilderNode`, dispatched via `BaseNode.execute()` + one `@Operation` per canvas mutation (`inspect_canvas` / `add_tool` / `add_skill` / `add_subagent` / `create_workflow`) |+| **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) |+| **Skill (if any)** | [`server/skills/assistant/agent-builder-skill/SKILL.md`](../../../server/skills/assistant/agent-builder-skill/SKILL.md) |+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `agent_builder` (`tool_name` class attr) |++## Purpose++Runtime canvas-mutation tool. Wired to an AI agent's `input-tools` handle, it+lets the LLM inspect the live workflow canvas and grow its own toolset mid-run:+spawn tool nodes, enable skills on a Master Skill, add delegate agents (team+leads only), or create a fresh workflow. One node, one `Params` model with an+`operation: Literal[...]` discriminator, five `@Operation` methods — the LLM+sees ONE tool with a select-style `operation` field. Each mutation persists to+the DB and pushes a `workflow_ops_apply` event so the React Flow canvas updates+live.++## Inputs (handles)++| Handle | Connection type | Required | Purpose |+|--------|-----------------|----------|---------|+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools`. Operations walk `ctx.edges` (via `needs_canvas = True`) to resolve the calling agent and mutate the canvas. |++## Parameters++The `AgentBuilderParams` model fields ARE the LLM-provided tool args (no+separate `toolName` / `toolDescription` node params — those live on the class+as `tool_name` / `tool_description`). `model_config = ConfigDict(extra="ignore")`.++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `operation` | enum | `inspect_canvas` | no | - | One of `inspect_canvas`, `add_tool`, `add_skill`, `add_subagent`, `create_workflow` |+| `node_type` | string | `""` | no | `operation == add_tool` | Tool node type to spawn (must be a registered `component_kind="tool"` or `usable_as_tool=True` node, minus the allowlist blocklist) |+| `skill_folder` | string | `""` | no | `operation == add_skill` | Skill folder name under `server/skills/**` |+| `agent_type` | string | `""` | no | `operation == add_subagent` | Agent node type to spawn as a teammate (caller must be a team lead) |+| `workflow_name` | string | `""` | no | `operation == create_workflow` | Display name (create_workflow is currently disabled) |+| `workflow_description` | string | `""` | no | `operation == create_workflow` | Optional one-line description |++## Outputs (handles)++| Handle | Shape | Description |+|--------|-------|-------------|+| `output-tool` | object | `AgentBuilderOutput` model (`extra="allow"`), serialized per `BaseNode._serialize_result` |++### Output payload (TypeScript shape)++```ts+{+ operation?: string;+ summary?: string;+ operations?: Array<Record<string, unknown>>; // workflow-ops batch applied+ // inspect_canvas extras:+ nodes?: Array<{ id; type; label; key_params }>;+ edges?: Array<{ source; target; source_handle; target_handle }>;+ you?: { node_id: string; incoming: unknown[]; outgoing: unknown[] };+ available_tools?: Array<{ type; display_name; description }>;+ available_agents?: Array<{ type; display_name; description }>;+ available_skills?: Array<{ folder; name; description }>;+ // create_workflow extras:+ workflow_id?: string;+}+```++## Logic Flow++```mermaid+flowchart TD+ A[BaseNode.execute -> operation method] --> R{operation}+ R -- inspect_canvas --> I[reload canvas from DB<br/>resolve caller via input-tools edge<br/>build node/edge summaries + tool/agent/skill catalogues]+ R -- add_tool --> T{node_type allowed + not already wired?}+ T -- no --> Tn[Return summary, operations=empty]+ T -- yes --> Top[mint node id + add_node + add_edge ops]+ R -- add_skill --> S{masterSkill wired?}+ S -- yes --> Sy[toggle skill in skills_config via set_node_parameters]+ S -- no --> Sn[spawn masterSkill node + edge with skill enabled]+ R -- add_subagent --> G{caller is team lead + agent_type allowed + not dup?}+ G -- no --> Gn[Return summary, operations=empty]+ G -- yes --> Gop[mint node id + add_node + add_edge to input-teammates]+ R -- create_workflow --> C[disabled: return summary, no-op]+ Top --> B[_broadcast: persist to DB + broadcast_workflow_ops]+ Sy --> B+ Sn --> B+ Gop --> B+ B --> Z[Return AgentBuilderOutput summary + operations]+ I --> Z+ Tn --> Z+ Gn --> Z+ C --> Z+```++## Decision Logic++- **Caller resolution**: walks edges for `source == self_id && targetHandle == input-tools`; falls back to self as caller (standalone Run).+- **Live canvas reload**: `_load_live_canvas` re-reads `workflow.data` from the DB so successive ops in the same run see each other's mutations (the per-tool `ctx` snapshot is frozen at MachinaWorkflow start).+- **Allowlist gating**: `_allowed_tool_types` / `_allowed_subagent_types` honor `node_allowlist.json` (`disabled_nodes` / `disabled_groups`); `_DENIED_TOOL_TYPES` (`agentBuilder`, `masterSkill`) are never spawnable (recursion guard).+- **Idempotency**: `add_tool` / `add_subagent` / `add_skill` are no-ops (empty `operations`) when the target is already wired/enabled, surfacing the existing node id in the summary.+- **add_subagent gate**: only team leads (`orchestrator_agent`, `ai_employee`) may spawn delegates; cannot spawn another team lead.+- **create_workflow disabled**: `_CREATE_WORKFLOW_ENABLED = False` short-circuits with a "temporarily disabled" summary; the implementation body is intact for one-line re-enable.+- **Summary suffix**: depends on `ctx.raw["auto_rebind_tools"]` (UserSettings) — "Available immediately" when auto-rebind is on, else "Available on your next turn" (the agent loop binds tools at turn start, so a mid-run mutation is callable next turn unless auto-rebind rebinds it in place).++## Side Effects++- **Database writes**: `_persist_canvas_mutation` applies `add_node` / `add_edge` / `set_node_parameters` ops to `workflow.data` via `database.get_workflow` + `database.save_workflow` (adopts the BE-minted `minted_id` so FE/BE node ids align). `create_workflow` (when enabled) persists a fresh workflow row.+- **Broadcasts**: `broadcast_workflow_ops` (in `_events.py`) emits a flat `workflow_ops_apply` WS frame consumed by the frontend `useWorkflowOpsListener` hook. NOT a `dispatch.emit` event (no Temporal/event_waiter consumer for canvas mutations). A `WorkflowEvent` envelope (`com.machinaos.workflow.ops.applied`) is constructed for parity but not put on the wire today.+- **External API calls**: none.+- **File I/O**: reads `server/skills/**` to validate skill folders (`_skill_folder_exists`) and catalogue skills.+- **Subprocess**: none.++## External Dependencies++- **Credentials**: none.+- **Services**: `database` (via `services.plugin.deps.get_database`), `NodeAllowlistService`, `SkillLoader`, `services.node_registry.registered_node_classes`, `services.workflow_ops`, `services.workflow_naming`.+- **Python packages**: stdlib only.+- **Environment variables**: none.++## Edge cases & known limits++- `create_workflow` is disabled behind `_CREATE_WORKFLOW_ENABLED`; the operation returns a guidance summary and performs no mutation.+- Mid-run mutations are not callable in the SAME turn unless the user's "Auto-Rebind Tools After Canvas Changes" toggle is on (default) — otherwise the LLM must wait for its next turn.+- `inspect_canvas` is read-only and emits no `operations`.+- Persist failures are logged at WARN (`exc_info=True`) and do not abort the broadcast — the canvas may diverge from the DB if persistence fails mid-mutation.+- The `workflow_ops_apply` wire format is flat (`{workflow_id, caller_node_id, operations}`) for FE back-compat, not the CloudEvents envelope.++## Related++- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)+- **Skill using this tool**: [`agent-builder-skill/SKILL.md`](../../../server/skills/assistant/agent-builder-skill/SKILL.md)+- **Architecture docs**: [Workflow Ops Protocol](../../workflow_ops_protocol.md), [Agent Architecture](../../agent_architecture.md), [Node Creation Guide](../../node_creation.md)diff --git a/docs-internal/node-logic-flows/ai_tools/calculatorTool.md b/docs-internal/node-logic-flows/ai_tools/calculatorTool.mdindex e53ccdb1..1d66c473 100644--- a/docs-internal/node-logic-flows/ai_tools/calculatorTool.md+++ b/docs-internal/node-logic-flows/ai_tools/calculatorTool.md@@ -2,85 +2,71 @@| Field | Value ||------|-------|-| **Category** | ai_tools (dedicated AI tool) |-| **Backend handler** | [`server/services/handlers/tools.py::_execute_calculator`](../../../server/services/handlers/tools.py) |+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |+| **Backend handler** | [`server/nodes/tool/calculator_tool/__init__.py`](../../../server/nodes/tool/calculator_tool/__init__.py) — `CalculatorToolNode`, dispatched via `BaseNode.execute()` + the `@Operation("calculate")` method || **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) || **Skill (if any)** | None (referenced directly by AI agents via tool-calling) |-| **Dual-purpose tool** | tool-only - LLM invokes as `calculator` (configurable via `toolName` param) |+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `calculator` (`tool_name` class attr) |## PurposeExposes basic arithmetic as a structured tool that an AI Agent can call during-reasoning. The LLM fills the `operation` + operand arguments and the handler+reasoning. The LLM fills the `operation` + operand arguments and the operationreturns the numeric result. Supports the eight operations listed below; no-external calls, no state.+external calls, no state. The `CalculatorParams` Pydantic model is the+LLM-visible tool schema (via `ToolNode.as_tool_schema()`).## Inputs (handles)| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| (none) | - | - | Passive node - connect `output-tool` to an AI Agent's `input-tools` |+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |## Parameters+The `CalculatorParams` model fields ARE the LLM-provided tool args (no separate+`toolName` / `toolDescription` node params — those live on the class as+`tool_name` / `tool_description`).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `calculator` | yes | - | Name the LLM uses to invoke the tool |-| `toolDescription` | string | (see frontend) | no | - | LLM-visible tool description |--### LLM-provided tool args (at invocation time)--| Arg | Type | Description |-|-----|------|-------------|-| `operation` | string | One of `add`, `subtract`, `multiply`, `divide`, `power`, `sqrt`, `mod`, `abs` |-| `a` | number | First operand (defaults to `0` if missing) |-| `b` | number | Second operand (defaults to `0`; unused for `sqrt` / `abs`) |+| `operation` | enum | (required) | yes | - | One of `add`, `subtract`, `multiply`, `divide`, `power`, `sqrt`, `mod`, `abs` |+| `a` | float | (required) | yes | - | First operand (or the sole input for `sqrt` / `abs`) |+| `b` | float? | `None` | no | - | Second operand; required for add/subtract/multiply/divide/power/mod, unused for `sqrt` / `abs` |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-tool` | object | Raw dict returned to the LLM (no envelope wrapper at the tool level) |+| `output-tool` | object | `CalculatorOutput` model, serialized per `BaseNode._serialize_result` |### Output payload (TypeScript shape)-On success:+On success (matches the `CalculatorOutput` model):```ts{operation: string;a: number;- b: number;+ b: number | null;result: number; // float('inf') for x/0, 0 for x mod 0}```-On unsupported operation:-```ts-{- error: string; // "Unknown operation: <op>"- supported_operations: string[];-}-```--On any raised exception inside the op lambda:-```ts-{ error: string } // str(exception)-```+Errors are not produced inside the op (the `Literal` schema rejects unknown+operations at Pydantic validation before the op runs; a validation failure+surfaces as the framework's standard error envelope from `BaseNode.execute()`).## Logic Flow```mermaidflowchart TD- A[_execute_calculator args] --> B[operation = args.operation.lower<br/>a = float args.a default 0<br/>b = float args.b default 0]- B --> C{operation in<br/>supported map?}- C -- no --> Eunknown[Return error + supported_operations list]- C -- yes --> D{Invoke op lambda}+ A[BaseNode.execute -> calculate ctx params] --> B[fn = _OPERATIONS params.operation<br/>b_value = params.b if not None else 0.0]+ B --> D{Invoke op lambda}D -- divide with b==0 --> Finf[result = float inf]D -- mod with b==0 --> Fmod[result = 0]D -- sqrt --> Fsqrt[result = sqrt of abs a]- D -- other --> Fok[result = a op b]- D -- exception --> Eexc[Return error: str exception]- Finf --> R[Return operation / a / b / result]+ D -- other --> Fok[result = a op b_value]+ Finf --> R[Return CalculatorOutput operation/a/b/result]Fmod --> RFsqrt --> RFok --> R@@ -88,18 +74,16 @@ flowchart TD## Decision Logic-- **Unknown operation**: returns `{error, supported_operations}` and does not- raise. `operations.keys()` is the authoritative supported list.+- **Unknown operation**: cannot occur — `operation` is a `Literal[...]` so an+ out-of-set value fails Pydantic validation before the op runs.- **Division by zero** (`divide`): returns `float('inf')` instead of erroring.- **Mod by zero** (`mod`): returns `0` instead of erroring.-- **Negative sqrt**: handler silently calls `math.sqrt(abs(a))`, so `sqrt(-9)`- returns `3.0` (not an error).-- **`power` overflow**: raised as `OverflowError` by `math.pow`, caught by the- outer `except Exception` and returned as `{error: str}`.-- **Tool mode only**: when dropped on the canvas and run directly, the node- type is not registered in `NodeExecutor._handlers`, so the executor falls- through to the generic success fallback. The real contract is the tool- invocation path via `execute_tool`.+- **Negative sqrt**: silently calls `math.sqrt(abs(a))`, so `sqrt(-9)` returns+ `3.0` (not an error).+- **Missing `b`**: when `params.b is None`, `b_value` defaults to `0.0` before+ the lambda runs (so `add`/`multiply`/etc. with no `b` operate against 0).+- **`power` overflow**: raised as `OverflowError` by `math.pow`; bubbles up to+ `BaseNode.execute()`'s generic exception path (full traceback envelope).## Side Effects@@ -121,14 +105,14 @@ flowchart TD- `divide` returns `float('inf')` for division by zero - downstream JSONserializers may choke on `Infinity` (not strict JSON).- `mod` returns `0` for modulo-by-zero, which is mathematically wrong but- matches the handler. Document this for agents that might reason about it.+ matches the op lambda. Document this for agents that might reason about it.- `sqrt` of a negative number silently returns the sqrt of the absolute valuerather than erroring or returning NaN.-- `float(args.get('a', 0))` will raise `ValueError` if the LLM returns- non-numeric strings; that error is caught and returned as `{error: ...}`.-- Operation names are case-insensitive (`.lower()` applied).+- `a` is typed `float` (required) — a non-numeric LLM argument fails Pydantic+ coercion and produces a validation error the LLM can correct.+- Operation names are case-sensitive (`Literal` exact match; no `.lower()`).## Related-- **Sibling tools**: [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)+- **Sibling tools**: [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)- **Architecture docs**: [Agent Architecture](../../agent_architecture.md), [Tool Building Pipeline](../../tool_building_pipeline.md), [Node Creation Guide](../../node_creation.md)diff --git a/docs-internal/node-logic-flows/ai_tools/currentTimeTool.md b/docs-internal/node-logic-flows/ai_tools/currentTimeTool.mdindex 797d5da1..bdc06cbf 100644--- a/docs-internal/node-logic-flows/ai_tools/currentTimeTool.md+++ b/docs-internal/node-logic-flows/ai_tools/currentTimeTool.md@@ -2,84 +2,71 @@| Field | Value ||------|-------|-| **Category** | ai_tools (dedicated AI tool) |-| **Backend handler** | [`server/services/handlers/tools.py::_execute_current_time`](../../../server/services/handlers/tools.py) |+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |+| **Backend handler** | [`server/nodes/tool/current_time_tool/__init__.py`](../../../server/nodes/tool/current_time_tool/__init__.py) — `CurrentTimeToolNode`, dispatched via `BaseNode.execute()` + the `@Operation("now")` method || **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) || **Skill (if any)** | None |-| **Dual-purpose tool** | tool-only - LLM invokes as `get_current_time` (configurable via `toolName` param) |+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `get_current_time` (`tool_name` class attr) |## Purpose-Provides the current date, time, day-of-week, and unix timestamp in a-caller-specified timezone. Intended for AI Agents that need temporal grounding-(e.g., "what day is it today?" before scheduling something). Pure wrapper-around `datetime.now(tz)` with `pytz` for tz resolution.+Provides the current ISO timestamp and unix epoch in a caller-specified+timezone. Intended for AI Agents that need temporal grounding (e.g., "what day+is it today?" before scheduling something). Pure wrapper around+`datetime.now(tz)` with `zoneinfo.ZoneInfo` for tz resolution.## Inputs (handles)| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| (none) | - | - | Passive node |+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |## Parameters+The `CurrentTimeParams` model field IS the LLM-provided tool arg (no separate+`toolName` / `toolDescription` node params — those live on the class as+`tool_name` / `tool_description`).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `get_current_time` | yes | - | LLM-visible tool name |-| `toolDescription` | string | (see frontend) | no | - | LLM-visible description |-| `timezone` | string | `UTC` | no | - | Default timezone used if the LLM does not pass one |--### LLM-provided tool args (at invocation time)--| Arg | Type | Description |-|-----|------|-------------|-| `timezone` | string? | IANA name (`UTC`, `America/New_York`, `Europe/London`, ...). When omitted or empty, handler falls back to the node's `timezone` parameter. |+| `timezone` | string | `UTC` | no | - | IANA name (`UTC`, `America/New_York`, `Europe/London`, ...). Invalid values silently fall back to `UTC`. |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-tool` | object | Raw dict returned to the LLM |+| `output-tool` | object | `CurrentTimeOutput` model, serialized per `BaseNode._serialize_result` |### Output payload (TypeScript shape)-On success:+On success (matches the `CurrentTimeOutput` model):```ts{- datetime: string; // ISO 8601 with offset, e.g. "2026-04-15T12:00:00+00:00"- date: string; // "YYYY-MM-DD"- time: string; // "HH:MM:SS"- timezone: string; // echoed back (the resolved string, not the tz object)- day_of_week: string; // "Monday" ... "Sunday"- timestamp: number; // unix seconds, integer+ iso: string; // ISO 8601 with offset, e.g. "2026-04-15T12:00:00+00:00"+ timezone: string; // echoed back (the requested input string)+ unix: number; // unix seconds, integer}```-On invalid timezone:-```ts-{ error: string } // "Invalid timezone: <tz>. Error: <details>"-```-## Logic Flow```mermaidflowchart TD- A[_execute_current_time args node_params] --> B[tz_str = args.timezone OR node_params.timezone OR UTC]- B --> C{pytz.timezone tz_str}- C -- raises UnknownTimeZoneError --> E[Return error: Invalid timezone ...]- C -- ok --> D[now = datetime.now tz]- D --> F[Format ISO / date / time / day_of_week / timestamp]- F --> R[Return dict]+ A[BaseNode.execute -> now ctx params] --> B{ZoneInfo params.timezone}+ B -- raises any exception --> C[tz = ZoneInfo UTC]+ B -- ok --> D[tz resolved]+ C --> E[now = datetime.now tz]+ D --> E+ E --> R[Return CurrentTimeOutput iso/timezone/unix]```## Decision Logic-- **Timezone resolution**: `args.timezone` wins if truthy; otherwise- `node_params.timezone`; otherwise the string literal `'UTC'`.-- **Invalid timezone**: caught by the outer `except Exception` and returned- as `{error: "Invalid timezone: <tz>. Error: <str>"}` (no raise).-- **Empty string**: an empty `timezone` arg is falsy so falls back to- `node_params.timezone`; an empty node param then falls back to `'UTC'`.+- **Timezone resolution**: `ZoneInfo(params.timezone)`; `timezone` defaults to+ `"UTC"` when the LLM omits it.+- **Invalid timezone**: any exception constructing `ZoneInfo` is caught and+ silently falls back to `ZoneInfo("UTC")` (the result still echoes the+ requested `params.timezone` string in the `timezone` field).## Side Effects@@ -93,20 +80,20 @@ flowchart TD- **Credentials**: none.- **Services**: none.-- **Python packages**: `pytz`, `datetime` (stdlib).+- **Python packages**: `zoneinfo`, `datetime` (stdlib).- **Environment variables**: none.## Edge cases & known limits-- `timestamp` is truncated to integer seconds via `int(now.timestamp())` -+- `unix` is truncated to integer seconds via `int(now.timestamp())` -millisecond precision is lost.-- `timezone` field in the output is the input string, not the canonical- zone name `pytz` may have normalised to (e.g. aliases are not rewritten).+- `timezone` field in the output is the requested input string, not the+ canonical zone name (and stays the input even when the lookup falls back to+ UTC, so an invalid tz is silently masked).- If the server system clock is wrong, output is wrong. No NTP sync.-- `pytz.UnknownTimeZoneError` is the typical failure mode; any other- unexpected exception (e.g. strftime locale issues) is also swallowed.+- A bad/unknown IANA name does NOT error — it falls back to UTC silently.## Related-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)- **Architecture docs**: [Agent Architecture](../../agent_architecture.md)diff --git a/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.md b/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.mdindex b6eca692..2a71cca5 100644--- a/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.md+++ b/docs-internal/node-logic-flows/ai_tools/duckduckgoSearch.md@@ -2,11 +2,11 @@| Field | Value ||------|-------|-| **Category** | ai_tools (dedicated AI tool) |-| **Backend handler** | [`server/services/handlers/tools.py::_execute_duckduckgo_search`](../../../server/services/handlers/tools.py) |+| **Category** | ai_tools (dedicated AI tool; plugin lives under `server/nodes/search/`, group `("tool", "ai", "search")`) |+| **Backend handler** | [`server/nodes/search/duckduckgo_search/__init__.py`](../../../server/nodes/search/duckduckgo_search/__init__.py) — `DuckDuckGoSearchNode`, dispatched via `BaseNode.execute()` + the `@Operation("search")` method || **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) || **Skill (if any)** | [`server/skills/web_agent/duckduckgo-search-skill/SKILL.md`](../../../server/skills/web_agent/duckduckgo-search-skill/SKILL.md) |-| **Dual-purpose tool** | tool-only - LLM invokes as `web_search` (configurable via `toolName` param) |+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `web_search` (`tool_name` class attr) |## Purpose@@ -19,75 +19,62 @@ default web search tool for AI Agents, complementing the keyed alternatives| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| (none) | - | - | Passive node |+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |## Parameters+The `DuckDuckGoSearchParams` model fields ARE the LLM-provided tool args (no+separate `toolName` / `toolDescription` node params — those live on the class+as `tool_name` / `tool_description`).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `web_search` | yes | - | LLM-visible tool name |-| `toolDescription` | string | (see frontend) | no | - | LLM-visible description |-| `maxResults` | number | `5` | no | - | Result cap (frontend clamps `1..10`); cast to `int` in handler |--### LLM-provided tool args (at invocation time)--| Arg | Type | Description |-|-----|------|-------------|-| `query` | string | Search query; empty string short-circuits with an error |+| `query` | string | (required) | yes | - | Search query; `min_length=1` (empty fails Pydantic validation) |+| `max_results` | int | `5` | no | - | Result cap; Pydantic-clamped `1..20` (`ge=1, le=20`) |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-tool` | object | Raw dict returned to the LLM |+| `output-tool` | object | `DuckDuckGoSearchOutput` model, serialized per `BaseNode._serialize_result` |### Output payload (TypeScript shape)-On success:+On success (matches the `DuckDuckGoSearchOutput` model):```ts{query: string;results: Array<{ title: string; snippet: string; url: string }>;- provider: 'duckduckgo';+ provider: 'duckduckgo'; // fixed default}```-On empty query:-```ts-{ error: 'No search query provided' }-```+On a `DDGSException` (rate-limit / network / bot detection) the op raises+`NodeUserError("DuckDuckGo search failed: <e>")`, which `BaseNode.execute()`+catches into a single WARN line + `{success: false, error_type: "NodeUserError",+error: ...}` envelope (no traceback).## Logic Flow```mermaidflowchart TD- A[_execute_duckduckgo_search args node_params] --> B{query non-empty?}- B -- no --> Eq[Return error: No search query provided]- B -- yes --> C[provider = node_params.provider OR duckduckgo<br/>max_results = int node_params.maxResults OR 5]- C --> D{provider == duckduckgo}- D -- yes --> E{Try import ddgs}- E -- ok --> F[run_in_executor DDGS.text query max_results]- F --> G[Map title / body->snippet / href->url]- G --> R[Return query / results / provider]- E -- ImportError --> H[Fallback: httpx GET api.duckduckgo.com Instant Answer API]- H --> I[Map AbstractText + RelatedTopics into results]- I --> R- F -- Exception --> Eexc[Logged; propagates to outer generic handler]+ A[BaseNode.execute -> search ctx params] --> B[import ddgs DDGS + DDGSException]+ B --> F[run_in_executor: DDGS.text query max_results]+ F -- DDGSException --> Eq[raise NodeUserError: DuckDuckGo search failed]+ F -- ok --> G[Map title / body->snippet / href->url]+ G --> R[Return DuckDuckGoSearchOutput query/results/provider]```## Decision Logic-- **Empty query**: `if not query: return {error: ...}` - no library call.-- **Provider switch**: `node_params.provider` defaults to `'duckduckgo'` and- the handler only implements that branch. Any other value falls through- without raising (returns whatever the outer try returns, typically nothing- useful). The frontend node does not expose `provider`, so in practice this- branch is always taken.-- **`ddgs` ImportError fallback**: the handler quietly falls back to the- DuckDuckGo Instant Answer API (`api.duckduckgo.com`) via `httpx`, which- returns a very different shape (abstracts + related topics, not ranked web- results). Shown logs: `"ddgs not installed, falling back to Instant Answer API"`.-- **Sync-in-async**: `DDGS().text(...)` is synchronous; handler wraps it in+- **Empty query**: rejected at Pydantic validation (`query` has `min_length=1`)+ before the op runs — no library call.+- **No provider switch / no fallback**: the plugin always uses the `ddgs`+ library `DDGS().text(...)`. The old Instant Answer API (`api.duckduckgo.com`)+ httpx fallback was removed; `ddgs` is a hard dependency now.+- **`DDGSException`**: caught and re-raised as `NodeUserError` so the LLM can+ retry with a different query (one WARN line, no traceback).+- **Sync-in-async**: `DDGS().text(...)` is synchronous; the op wraps it in`loop.run_in_executor(None, ...)` to avoid blocking the event loop.## Side Effects@@ -96,11 +83,8 @@ flowchart TDthis handler does **not** write to `api_usage_metrics` - it is free andtracking cost is skipped.)- **Broadcasts**: none.-- **External API calls**:- - Primary: `ddgs` library (HTTPS to `duckduckgo.com`; the library manages- its own networking, timeout, and rate limits).- - Fallback: `GET https://api.duckduckgo.com/?q=<query>&format=json&no_html=1`- (timeout 10s) via `httpx.AsyncClient`.+- **External API calls**: `ddgs` library (HTTPS to `duckduckgo.com`; the+ library manages its own networking, timeout, and rate limits).- **File I/O**: none.- **Subprocess**: none.@@ -108,27 +92,22 @@ flowchart TD- **Credentials**: none.- **Services**: none.-- **Python packages**: `ddgs` (preferred), `httpx` (fallback).+- **Python packages**: `ddgs` (hard dependency; lazy-imported inside the op).- **Environment variables**: none.## Edge cases & known limits-- Handler does **not** wrap network errors from `ddgs` - they propagate up to- `execute_tool`, which will bubble them into the tool-call result as an- uncaught exception. (Mocking the library in tests is therefore required.)-- Missing `ddgs` silently switches to Instant Answer API, which returns a- noticeably smaller and differently-structured result set (abstract +- related topics, not ranked web results). The `provider` field still says- `"duckduckgo"`, masking the fallback.-- `maxResults` clamp (`1..10`) is enforced only by the frontend; an LLM could- in theory pass `node_params.maxResults = 999` via stored defaults and the- handler would honour it.-- `results[].url` uses the `href` field from `ddgs`; older library versions- used a different key - the handler fills missing keys with `""` to stay- defensive.+- A `DDGSException` (the only caught error) becomes a `NodeUserError`; other+ unexpected exceptions bubble up to `BaseNode.execute()`'s generic handler+ with a full traceback.+- `max_results` clamp (`1..20`) is enforced by Pydantic on the model+ (`ge=1, le=20`), so an out-of-range LLM value fails validation rather than+ being silently honoured.+- `results[].url` uses the `href` field from `ddgs`; missing keys default to+ `""` to stay defensive.## Related-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md)+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`taskManager`](./taskManager.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)- **Companion search nodes** (API-keyed, track cost): [`braveSearch`](../search/braveSearch.md), [`serperSearch`](../search/serperSearch.md), [`perplexitySearch`](../search/perplexitySearch.md)- **Skill using this tool**: [`duckduckgo-search-skill/SKILL.md`](../../../server/skills/web_agent/duckduckgo-search-skill/SKILL.md)diff --git a/docs-internal/node-logic-flows/ai_tools/taskManager.md b/docs-internal/node-logic-flows/ai_tools/taskManager.mdindex 54da3352..69df9b55 100644--- a/docs-internal/node-logic-flows/ai_tools/taskManager.md+++ b/docs-internal/node-logic-flows/ai_tools/taskManager.md@@ -2,42 +2,53 @@| Field | Value ||------|-------|-| **Category** | ai_tools (dual-purpose) |-| **Backend handler** | [`server/services/handlers/tools.py::handle_task_manager`](../../../server/services/handlers/tools.py) (workflow-node entry) -> `_execute_task_manager` |+| **Category** | ai_tools (dedicated AI tool, group `("tool", "ai")`) |+| **Backend handler** | [`server/nodes/tool/task_manager/__init__.py`](../../../server/nodes/tool/task_manager/__init__.py) — `TaskManagerNode`, dispatched via `BaseNode.execute()` + the `@Operation("manage")` method, which calls the inlined `_execute_task_manager` in the same file || **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) || **Skill (if any)** | [`server/skills/task_agent/task-manager-skill/SKILL.md`](../../../server/skills/task_agent/task-manager-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `task_manager`; handler registered under both registry key `taskManager` and `execute_tool` dispatch |+| **Dual-purpose tool** | tool-only — `ToolNode` exposed to the LLM as `task_manager` (`tool_name` class attr) |## Purpose-Inspects the in-process registry of delegated agent tasks (`_delegated_tasks`-and `_delegation_results` dicts in `tools.py`). Lets a parent agent (or a-workflow node run) list currently-running / completed delegations, fetch-details for a specific `task_id`, or clear a finished task from tracking.-Does **not** spawn work itself; it only reads/mutates the delegation state-populated by `_execute_delegated_agent`.+Inspects the in-process registry of delegated agent tasks. The+`_execute_task_manager` body lives in the plugin file but reads **through** to+the delegation registry owned by `services/handlers/tools.py` (the+`_delegated_tasks` + `_delegation_results` dicts + `get_delegated_task_status`+3-layer lookup) — that lifecycle state stays in `tools.py` intentionally. Lets a+parent agent list currently-running / completed delegations, fetch details for a+specific `task_id`, or clear a finished task from tracking. Does **not** spawn+work itself; it only reads/mutates the delegation state populated by+`_execute_delegated_agent`.## Inputs (handles)| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| (none) | - | - | Passive node |+| `input-main` | main | no | Passive node - connect `output-tool` to an AI Agent's `input-tools` |## Parameters+The declared `TaskManagerParams` model fields are below. The model carries+`model_config = ConfigDict(extra="allow")`, so the LLM-driven `operation` /+`status_filter` args (which `_execute_task_manager` reads) pass through even+though they are not declared fields.+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `task_manager` | no | - | LLM-visible tool name |-| `toolDescription` | string | (see frontend) | no | - | LLM-visible description |-| `operation` | options | `list_tasks` | no | - | One of `list_tasks`, `get_task`, `mark_done` (workflow-node mode) |-| `task_id` | string | `""` | no | `operation in [get_task, mark_done]` | Target task id |-| `status_filter` | options | `""` | no | `operation == list_tasks` | One of `""`, `running`, `completed`, `error` |+| `action` | enum | `create` | no | - | Declared field: one of `create`, `list`, `complete`, `delete`, `update` (not the dispatch discriminator — see below) |+| `task_id` | string? | `None` | no | - | Target task id |+| `title` | string? | `None` | no | - | Declared but unused by the inlined op |+| `description` | string? | `None` | no | - | Declared but unused by the inlined op |+| `status` | string? | `None` | no | - | Declared but unused by the inlined op |++### LLM-provided tool args (the operations the op actually dispatches on)-### LLM-provided tool args (when used as a tool)+The `manage` op passes `params.model_dump()` (with extras) into+`_execute_task_manager`, which reads `operation` / `task_id` / `status_filter`:| Arg | Type | Description ||-----|------|-------------|-| `operation` | string | Same three values; falls back to node param, then to `'list_tasks'` |+| `operation` | string | One of `list_tasks`, `get_task`, `mark_done`; defaults to `'list_tasks'` || `task_id` | string | Required for `get_task` / `mark_done`; otherwise ignored || `status_filter` | string | Optional filter applied to `list_tasks` |@@ -45,8 +56,7 @@ populated by `_execute_delegated_agent`.| Handle | Shape | Description ||--------|-------|-------------|-| `output-tool` | object | Raw dict returned to the LLM |-| `output-main` | object | Same payload, available when run as a workflow node |+| `output-tool` | object | Raw dict returned to the LLM (the `TaskManagerOutput` model has `extra="allow"`, so the full operation dict passes through) |### Output payload (TypeScript shape)@@ -99,7 +109,8 @@ Unknown operation: `{success: false, error: "Unknown operation: <op>"}`.```mermaidflowchart TD- A[_execute_task_manager args config] --> B[Merge operation / task_id / status_filter<br/>args wins, then node params, then defaults]+ A[BaseNode.execute -> manage ctx params] --> A2[_execute_task_manager params.model_dump config]+ A2 --> B[Read operation / task_id / status_filter<br/>tool_args wins, then params, then defaults]B --> C{operation}C -- list_tasks --> D[Iterate _delegated_tasks -> derive running/completed/cancelled/error]D --> E[Merge _delegation_results entries not already in list]@@ -171,11 +182,12 @@ flowchart TDthree aggregate counters.- Unknown `operation` returns a failed envelope rather than raising; theagent might keep retrying with the same bad operation string.-- `handle_task_manager` (workflow-node entry) passes an empty `tool_args`- dict and relies entirely on node parameters.+- The dispatch discriminator is `operation` (read from the extra-allowed+ tool args), NOT the declared `action` field — the `action` / `title` /+ `description` / `status` model fields are vestigial and unused by the op.## Related-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`writeTodos`](./writeTodos.md)+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`writeTodos`](./writeTodos.md), [`agentBuilder`](./agentBuilder.md)- **Skill using this tool**: [`task-manager-skill/SKILL.md`](../../../server/skills/task_agent/task-manager-skill/SKILL.md)- **Architecture docs**: [Agent Delegation](../../agent_delegation.md)diff --git a/docs-internal/node-logic-flows/ai_tools/writeTodos.md b/docs-internal/node-logic-flows/ai_tools/writeTodos.mdindex a092c7e3..da1ec5a1 100644--- a/docs-internal/node-logic-flows/ai_tools/writeTodos.md+++ b/docs-internal/node-logic-flows/ai_tools/writeTodos.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | ai_tools (dual-purpose) |-| **Backend handler** | [`server/services/handlers/todo.py::handle_write_todos`](../../../server/services/handlers/todo.py) (workflow) / `execute_write_todos` (tool) |+| **Backend handler** | [`server/nodes/tool/write_todos/__init__.py`](../../../server/nodes/tool/write_todos/__init__.py) — `WriteTodosNode`, dispatched via `BaseNode.execute()` + the `@Operation("write")` method (logic inlined; the standalone `services/handlers/todo.py::handle_write_todos` / `execute_write_todos` still exist but are no longer the dispatch path) || **Service** | [`server/services/todo_service.py::TodoService`](../../../server/services/todo_service.py) || **Tests** | [`server/tests/nodes/test_ai_tools.py`](../../../server/tests/nodes/test_ai_tools.py) || **Skill (if any)** | [`server/skills/assistant/write-todos-skill/SKILL.md`](../../../server/skills/assistant/write-todos-skill/SKILL.md) |@@ -51,51 +51,49 @@ generic-params card. `uiHints.isTodoEditor` (declared on the plugin) routes## Parameters+The `WriteTodosParams` model field IS the LLM-provided tool arg (no separate+`toolName` / `toolDescription` node params — those live on the class as+`tool_name` / `tool_description`).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `write_todos` | no | - | LLM-visible tool name |-| `toolDescription` | string | (see frontend, 3-row hint) | no | - | LLM-visible description |--### LLM-provided tool args (at invocation time)--| Arg | Type | Description |-|-----|------|-------------|-| `todos` | `Array<{content: string, status: 'pending' \| 'in_progress' \| 'completed'}>` | Full replacement snapshot of the todo list |+| `todos` | `Array<TodoItem>` | `[]` | no | - | Full replacement snapshot. `TodoItem = {id?: string, content: string, status: 'pending' \| 'in_progress' \| 'completed'}`. A JSON-encoded string is coerced to a list by the `_coerce_todos` `field_validator(mode="before")` (handles Gemini stringified-array args). |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-tool` | object | Tool result returned to the LLM |-| `output-main` | object | Same payload, available when run as a workflow node |+| `output-tool` | object | Tool result returned to the LLM (`WriteTodosOutput` has `extra="allow"`, so the dict below passes through) |### Output payload (TypeScript shape)```ts{- success: true;message: string; // "Updated todo list (<N> items)"- todos: string; // JSON string of the validated list+ todos: Array<{ id?: string; content: string; status: string }>; // the validated list (NOT a JSON string)count: number; // len(stored)}```+Note: the op returns the validated `todos` LIST, not a JSON string. The+pre-fix `format_for_llm()` call that stringified this leaked a raw JSON string+into the key, which the `WriteTodosOutput` contract now rejects — the+LLM-facing serialization happens downstream.+## Logic Flow```mermaidflowchart TD- A[handle_write_todos node_id / parameters / context] --> B[Build config: node_id + workflow_id + broadcaster]- B --> C[execute_write_todos tool_args=parameters config]- C --> D[session_key = workflow_id OR node_id OR default]+ A[BaseNode.execute -> write ctx params] --> D[session_key = ctx.workflow_id OR ctx.node_id OR default]D --> E[service = get_todo_service singleton]- E --> F[service.write session_key todos_input]- F --> G[Per-item validation: dict? content stripped non-empty? status in VALID_STATUSES?]- G --> H[Serialize validated list to JSON, store in _store session_key]- H --> I{broadcaster + node_id?}- I -- yes --> J[await broadcaster.update_node_status node_id executing<br/>data.phase = todo_update<br/>data.todos = stored<br/>workflow_id = workflow_id]+ E --> F[service.write session_key todos dumped from params.todos]+ F --> G[Per-item validation inside TodoService]+ G --> I{broadcaster on ctx.raw?}+ I -- yes --> J[await broadcaster.update_node_status node_id executing<br/>data.phase = todo_update<br/>data.todos = stored<br/>workflow_id]I -- no --> K[skip]- J --> L[Return success + message + todos JSON + count]- K --> L+ J --> M[dispatch_todos_updated CloudEvent via services.events.dispatch.emit]+ K --> M+ M --> L[Return message + todos list + count]```## Decision Logic@@ -123,9 +121,14 @@ flowchart TD- **In-memory state writes**: `TodoService._store[session_key] = json.dumps(validated)`.State persists for the lifetime of the Python process only.- **Database writes**: none.-- **Broadcasts**: `StatusBroadcaster.update_node_status(node_id, "executing",- {"phase": "todo_update", "todos": [...]}, workflow_id=workflow_id)` - one- per successful write, when a broadcaster is present in context.+- **Broadcasts** (two per successful write):+ - `StatusBroadcaster.update_node_status(node_id, "executing",+ {"phase": "todo_update", "todos": [...]}, workflow_id=...)` — drives the+ canvas glow; only when a broadcaster is present on `ctx.raw`.+ - `dispatch_todos_updated(...)` — typed `todos_updated` CloudEvent+ (`com.machinaos.todos.updated`) via `services.events.dispatch.emit`+ (`server/nodes/tool/write_todos/_events.py`), refreshing the open Current+ Todos panel's `['todos', session_key]` query. Fires unconditionally.- **External API calls**: none.- **File I/O**: none.- **Subprocess**: none.@@ -148,16 +151,17 @@ flowchart TD- **All validation silent**: dropping an item with empty content or coercingan invalid status never produces a warning in the return payload - only aDEBUG log line.-- **Broadcast failures swallowed by caller**: `handle_write_todos` awaits+- **Broadcast failures bubble up**: the `write` op awaits`update_node_status`; if the broadcaster raises, the exception bubbles up- to the executor (no `try/except` inside the handler).-- **`todos` returned as a JSON string, not an object**: the LLM must parse- it again. This matches Claude Code's `TodoWrite` tool convention.+ to `BaseNode.execute()` (no `try/except` inside the op).+- **`todos` returned as a list, not a JSON string**: the Output contract+ declares `todos: Optional[list]`; returning a stringified list is rejected+ by `BaseNode._serialize_result`.- **Content is trimmed** (`.strip()`) but otherwise not sanitised - markdownor HTML in content survives untouched.## Related-- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md)+- **Sibling tools**: [`calculatorTool`](./calculatorTool.md), [`currentTimeTool`](./currentTimeTool.md), [`duckduckgoSearch`](./duckduckgoSearch.md), [`taskManager`](./taskManager.md), [`agentBuilder`](./agentBuilder.md)- **Skill using this tool**: [`write-todos-skill/SKILL.md`](../../../server/skills/assistant/write-todos-skill/SKILL.md)- **Architecture docs**: [Status Broadcaster](../../status_broadcaster.md), [Agent Architecture](../../agent_architecture.md)diff --git a/docs-internal/node-logic-flows/android/_pattern.md b/docs-internal/node-logic-flows/android/_pattern.mdindex b7273cff..39f9978e 100644--- a/docs-internal/node-logic-flows/android/_pattern.md+++ b/docs-internal/node-logic-flows/android/_pattern.md@@ -5,25 +5,26 @@ factory, and a single execution path. This document is the authoritativedescription of that shared behaviour. Individual per-node docs link back herefor everything that is not node-specific.-## Single handler, registry-driven dispatch+## Single plugin base, registry-driven dispatch++Wave 11.C migrated all 16 Android nodes to self-contained plugin folders+under `server/nodes/android/<service>/__init__.py`. Each is a trivial+subclass of the shared `AndroidServiceBase` (`server/nodes/android/_base.py`)+that sets `type` / `display_name` / `description` / `tool_name` only. The old+`server/services/handlers/android.py::handle_android_service` and+`server/services/android_service.py` files are **deleted**. Dispatch flows+through `BaseNode.execute()` -> the `@Operation("invoke")` method on+`AndroidServiceBase`.| Concern | Location ||---------|----------|-| Backend handler | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) |-| Underlying service | [`server/services/android_service.py::AndroidService.execute_service`](../../../server/services/android_service.py) |-| Registry binding | [`server/services/node_executor.py`](../../../server/services/node_executor.py) lines 228-230 |-| Output flatten | [`server/services/node_executor.py`](../../../server/services/node_executor.py) lines 277-285 |--The `NodeExecutor._build_handler_registry()` method binds every entry in-`ANDROID_SERVICE_NODE_TYPES` to the same partial:--```python-for node_type in ANDROID_SERVICE_NODE_TYPES:- registry[node_type] = partial(handle_android_service, android_service=self.android_service)-```+| Plugin base | [`server/nodes/android/_base.py::AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) |+| Underlying service | [`server/nodes/android/_dispatcher.py::AndroidService.execute_service`](../../../server/nodes/android/_dispatcher.py) |+| Registry binding | plugins self-register via `BaseNode.__init_subclass__`; merged into `NodeExecutor` via `_PLUGIN_HANDLERS` ([`server/services/node_executor.py`](../../../server/services/node_executor.py) line 124) |+| Output flatten | [`server/services/node_executor.py`](../../../server/services/node_executor.py) lines 173-181 |-The handler then maps the incoming `node_type` (camelCase) to a backend-`service_id` (snake_case) via a hard-coded dict **inside the handler**:+The `@Operation("invoke")` method maps the incoming `node_type` (camelCase) to a+backend `service_id` (snake_case) via the `SERVICE_ID_MAP` dict in `_base.py`:```pythonSERVICE_ID_MAP = {@@ -47,7 +48,8 @@ SERVICE_ID_MAP = {```The frontend hidden `service_id` param is **ignored** for known node types;-the handler's own map wins. See "Known inconsistencies" below.+the plugin's own map wins (`SERVICE_ID_MAP.get(self.type, ...)`). See "Known+inconsistencies" below.## Shared parameter set@@ -61,12 +63,13 @@ the handler's own map wins. See "Known inconsistencies" below.| `parameters` | dict or JSON string | `{}` | no | Per-action nested parameters || `package_name` | string | `""` | conditional | Only surfaced by `appLauncher`; handler promotes it into `parameters` automatically |-Any extra root-level key in the `additional_param_keys` list is hoisted into-`parameters` before dispatch. Currently the only key in that list is-`package_name`.+Root-level `package_name` is hoisted into `parameters` before dispatch by the+`invoke` method (the only key promoted today).-If `parameters` is received as a string the handler calls `json.loads()` on it;-on `JSONDecodeError` the value is silently replaced with `{}`.+If `parameters` is received as a string it is coerced via `json.loads()` —+first in `AndroidServiceParams._coerce_parameters` (a `field_validator(mode="before")`)+and again defensively inside `invoke`; on `JSONDecodeError` the value is+silently replaced with `{}`.## Execution paths@@ -82,7 +85,7 @@ on `JSONDecodeError` the value is silently replaced with `{}`.```mermaidflowchart TD- A[handle_android_service] --> B[Look up service_id via SERVICE_ID_MAP<br/>fallback to parameters.service_id then 'battery']+ A[AndroidServiceBase.invoke @Operation] --> B[Look up service_id via SERVICE_ID_MAP self.type<br/>fallback to parameters.service_id then 'battery']B --> C[Parse parameters<br/>coerce JSON string, promote package_name]C --> D[android_service.execute_service<br/>node_id, service_id, action, parameters, host, port]D --> E{relay client paired?}@@ -163,13 +166,14 @@ preserved.- `screenControlAutomation` -> `screen_control`- `airplaneModeControl` -> `airplane_mode`- The handler map wins because it is looked up first, but any consumer that+ The plugin map wins because it is looked up first, but any consumer thatreads the hidden param directly would see a different value. The same- discrepancy exists in `handlers/tools.py::_execute_android_service`.+ discrepancy exists in the AI-tool path+ `server/nodes/android/_base.py::execute_android_service_tool`.2. **JSON parse failure is silent**. A `parameters` string that fails to parseis replaced with `{}` and logged only at debug level.3. **No validation of required action params**. `appLauncher` needs- `package_name`, but the handler never checks for it - an empty value is+ `package_name`, but the plugin never checks for it - an empty value ispassed through and the Android side is expected to error.4. **`default_timeout` is service-instance-wide**. All sensor / camera / mediacalls share the same timeout; there is no per-action override.@@ -177,15 +181,20 @@ preserved.dict whose keys collide with `result` metadata keys (`action`,`service_id`, `timestamp`, ...), the flatten merges `data` **after**metadata, so the data wins and the metadata is lost from the promoted view.-6. **`execute_service` never raises**. Every exception path returns an envelope- with `success=false` and the error string, so the handler itself never- raises past `NodeExecutor`.+6. **`execute_service` never raises, but `invoke` does on failure**.+ `AndroidService.execute_service` returns a `success=false` envelope for+ every exception path. The `@Operation("invoke")` wrapper then re-raises a+ `RuntimeError` when `result["success"] is False` (using `result["error"]`),+ so `BaseNode.execute()` produces the standard error envelope. On success+ `invoke` returns `result["result"]` (or the whole dict if no `result` key).## Related- Android service infrastructure: [`server/services/android/`](../../../server/services/android/)- Status broadcaster (two-state model): [Status Broadcaster](../../status_broadcaster.md)- Android-agent skills: [`server/skills/android_agent/`](../../../server/skills/android_agent/)-- Toolkit sub-node execution: see `TOOLKIT_NODE_TYPES` and- `_execute_android_service` in- [`server/services/handlers/tools.py`](../../../server/services/handlers/tools.py)+- AI-tool dispatch (`androidTool` aggregator + direct service tools):+ `execute_android_toolkit` and `execute_android_service_tool` in+ [`server/nodes/android/_base.py`](../../../server/nodes/android/_base.py),+ called from+ [`server/services/handlers/tools.py::execute_tool`](../../../server/services/handlers/tools.py)diff --git a/docs-internal/node-logic-flows/android/airplaneModeControl.md b/docs-internal/node-logic-flows/android/airplaneModeControl.mdindex 49cde069..2a34ea56 100644--- a/docs-internal/node-logic-flows/android/airplaneModeControl.md+++ b/docs-internal/node-logic-flows/android/airplaneModeControl.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / automation |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/airplane_mode_control/__init__.py`](../../../server/nodes/android/airplane_mode_control/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | none || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/appLauncher.md b/docs-internal/node-logic-flows/android/appLauncher.mdindex c3b27f1a..0c13aa61 100644--- a/docs-internal/node-logic-flows/android/appLauncher.md+++ b/docs-internal/node-logic-flows/android/appLauncher.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / apps |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/app_launcher/__init__.py`](../../../server/nodes/android/app_launcher/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/app-launcher-skill/SKILL.md`](../../../server/skills/android_agent/app-launcher-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/appList.md b/docs-internal/node-logic-flows/android/appList.mdindex 095610ab..5bf97111 100644--- a/docs-internal/node-logic-flows/android/appList.md+++ b/docs-internal/node-logic-flows/android/appList.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / apps |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/app_list/__init__.py`](../../../server/nodes/android/app_list/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/app-list-skill/SKILL.md`](../../../server/skills/android_agent/app-list-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/audioAutomation.md b/docs-internal/node-logic-flows/android/audioAutomation.mdindex 087be70f..a6fcd004 100644--- a/docs-internal/node-logic-flows/android/audioAutomation.md+++ b/docs-internal/node-logic-flows/android/audioAutomation.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / automation |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/audio_automation/__init__.py`](../../../server/nodes/android/audio_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/audio-skill/SKILL.md`](../../../server/skills/android_agent/audio-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/batteryMonitor.md b/docs-internal/node-logic-flows/android/batteryMonitor.mdindex fc08e420..5026224b 100644--- a/docs-internal/node-logic-flows/android/batteryMonitor.md+++ b/docs-internal/node-logic-flows/android/batteryMonitor.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / monitoring |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/battery_monitor/__init__.py`](../../../server/nodes/android/battery_monitor/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/battery-skill/SKILL.md`](../../../server/skills/android_agent/battery-skill/SKILL.md) || **Dual-purpose tool** | direct sub-node of `androidTool` - can also connect straight to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/bluetoothAutomation.md b/docs-internal/node-logic-flows/android/bluetoothAutomation.mdindex f8304da5..3bd4f8ce 100644--- a/docs-internal/node-logic-flows/android/bluetoothAutomation.md+++ b/docs-internal/node-logic-flows/android/bluetoothAutomation.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / automation |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/bluetooth_automation/__init__.py`](../../../server/nodes/android/bluetooth_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/bluetooth-skill/SKILL.md`](../../../server/skills/android_agent/bluetooth-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/cameraControl.md b/docs-internal/node-logic-flows/android/cameraControl.mdindex a5b04d9c..e54f2286 100644--- a/docs-internal/node-logic-flows/android/cameraControl.md+++ b/docs-internal/node-logic-flows/android/cameraControl.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / media |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/camera_control/__init__.py`](../../../server/nodes/android/camera_control/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/camera-skill/SKILL.md`](../../../server/skills/android_agent/camera-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/deviceStateAutomation.md b/docs-internal/node-logic-flows/android/deviceStateAutomation.mdindex 00d072ad..1d221095 100644--- a/docs-internal/node-logic-flows/android/deviceStateAutomation.md+++ b/docs-internal/node-logic-flows/android/deviceStateAutomation.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / automation |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/device_state_automation/__init__.py`](../../../server/nodes/android/device_state_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | none || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/environmentalSensors.md b/docs-internal/node-logic-flows/android/environmentalSensors.mdindex b8fb7807..44634ce6 100644--- a/docs-internal/node-logic-flows/android/environmentalSensors.md+++ b/docs-internal/node-logic-flows/android/environmentalSensors.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / sensors |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/environmental_sensors/__init__.py`](../../../server/nodes/android/environmental_sensors/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/environmental-skill/SKILL.md`](../../../server/skills/android_agent/environmental-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/location.md b/docs-internal/node-logic-flows/android/location.mdindex d2c49391..145dd4b6 100644--- a/docs-internal/node-logic-flows/android/location.md+++ b/docs-internal/node-logic-flows/android/location.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / monitoring |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/location/__init__.py`](../../../server/nodes/android/location/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/location-skill/SKILL.md`](../../../server/skills/android_agent/location-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/mediaControl.md b/docs-internal/node-logic-flows/android/mediaControl.mdindex cfa877e5..acd120bb 100644--- a/docs-internal/node-logic-flows/android/mediaControl.md+++ b/docs-internal/node-logic-flows/android/mediaControl.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / media |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/media_control/__init__.py`](../../../server/nodes/android/media_control/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | none || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/motionDetection.md b/docs-internal/node-logic-flows/android/motionDetection.mdindex 1508a207..63e9ebef 100644--- a/docs-internal/node-logic-flows/android/motionDetection.md+++ b/docs-internal/node-logic-flows/android/motionDetection.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / sensors |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/motion_detection/__init__.py`](../../../server/nodes/android/motion_detection/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/motion-skill/SKILL.md`](../../../server/skills/android_agent/motion-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/networkMonitor.md b/docs-internal/node-logic-flows/android/networkMonitor.mdindex 794e743d..34ba5a6f 100644--- a/docs-internal/node-logic-flows/android/networkMonitor.md+++ b/docs-internal/node-logic-flows/android/networkMonitor.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / monitoring |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/network_monitor/__init__.py`](../../../server/nodes/android/network_monitor/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | none || **Dual-purpose tool** | sub-node of `androidTool`; also connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/screenControlAutomation.md b/docs-internal/node-logic-flows/android/screenControlAutomation.mdindex 37056c8e..e8d150c2 100644--- a/docs-internal/node-logic-flows/android/screenControlAutomation.md+++ b/docs-internal/node-logic-flows/android/screenControlAutomation.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / automation |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/screen_control_automation/__init__.py`](../../../server/nodes/android/screen_control_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/screen-control-skill/SKILL.md`](../../../server/skills/android_agent/screen-control-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/systemInfo.md b/docs-internal/node-logic-flows/android/systemInfo.mdindex 2b38ed0a..fdf23370 100644--- a/docs-internal/node-logic-flows/android/systemInfo.md+++ b/docs-internal/node-logic-flows/android/systemInfo.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / monitoring |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/system_info/__init__.py`](../../../server/nodes/android/system_info/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | none || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/android/wifiAutomation.md b/docs-internal/node-logic-flows/android/wifiAutomation.mdindex 458c2820..1dccb5f5 100644--- a/docs-internal/node-logic-flows/android/wifiAutomation.md+++ b/docs-internal/node-logic-flows/android/wifiAutomation.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | android / automation |-| **Backend handler** | [`server/services/handlers/android.py::handle_android_service`](../../../server/services/handlers/android.py) (shared) |+| **Backend handler** | plugin [`server/nodes/android/wifi_automation/__init__.py`](../../../server/nodes/android/wifi_automation/__init__.py); dispatch via `BaseNode.execute()` -> shared [`AndroidServiceBase.invoke`](../../../server/nodes/android/_base.py) (`@Operation("invoke")`) || **Tests** | [`server/tests/nodes/test_android.py`](../../../server/tests/nodes/test_android.py) || **Skill (if any)** | [`server/skills/android_agent/wifi-skill/SKILL.md`](../../../server/skills/android_agent/wifi-skill/SKILL.md) || **Dual-purpose tool** | sub-node of `androidTool`; connectable directly to any agent's `input-tools` |diff --git a/docs-internal/node-logic-flows/chat_utility/chatHistory.md b/docs-internal/node-logic-flows/chat_utility/chatHistory.mdindex bd7ebdda..c1bacdd3 100644--- a/docs-internal/node-logic-flows/chat_utility/chatHistory.md+++ b/docs-internal/node-logic-flows/chat_utility/chatHistory.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | chat_utility |-| **Backend handler** | [`server/services/handlers/utility.py::handle_chat_history`](../../../server/services/handlers/utility.py) |+| **Backend handler** | [`server/nodes/chat/chat_history/__init__.py`](../../../server/nodes/chat/chat_history/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("read")` || **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) || **Skill (if any)** | - || **Dual-purpose tool** | no |@@ -25,20 +25,21 @@ workflows that need to read a conversation from a remote chat host.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `host` | string | `localhost` | no | - | Chat backend host |-| `port` | number | `8080` | no | - | Chat backend port |+| `port` | number | `8080` | no | - | Chat backend port (1..65535) || `session_id` | string | `default` | no | - | Chat session identifier |-| `api_key` | string | `""` | no | - | Auth token forwarded to the chat backend |-| `limit` | number | `50` | no | - | Maximum messages to return |+| `api_key` | string (password) | `""` | no | - | Auth token forwarded to the chat backend |+| `limit` | number | `50` | no | - | Maximum messages to return (1..500) |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | `{ messages: Array<ChatMessage> }` |+| `output-main` | object | `{ messages: Array<ChatMessage>, count: number }` |### Output payload (TypeScript shape)```ts+// ChatHistoryOutput (model_config extra="allow"){messages: Array<{role: string; // e.g. "user" | "assistant"@@ -46,6 +47,7 @@ workflows that need to read a conversation from a remote chat host.created_at: string;[key: string]: unknown;}>;+ count: number; // len(messages)}```@@ -57,7 +59,7 @@ flowchart TDB -- JSON-RPC error --> E[Return error envelope]B -- RPC ok --> F{result.success?}F -- false --> E- F -- true --> G[Wrap result.messages into envelope]+ F -- true --> G[Wrap result.messages + count into ChatHistoryOutput]```## Decision Logic@@ -65,8 +67,9 @@ flowchart TD- **Validation**: none; no parameter is required.- **Branches**: success vs error branch on RPC response `success` flag.- **Fallbacks**: `host`/`port`/`session_id`/`api_key`/`limit` all have defaults.-- **Error paths**: every exception logged via `logger.error` and returned as- `{success: false, error: str(e)}`.+- **Error paths**: `not result.success` raises `RuntimeError(result.error or "chatHistory fetch failed")`;+ `BaseNode.execute()` wraps the raised exception into the standard error envelope+ (`success=false`, `error`, plus full traceback at `logger.exception` for non-`NodeUserError`).## Side Effectsdiff --git a/docs-internal/node-logic-flows/chat_utility/chatSend.md b/docs-internal/node-logic-flows/chat_utility/chatSend.mdindex 7da8d7d9..d4447582 100644--- a/docs-internal/node-logic-flows/chat_utility/chatSend.md+++ b/docs-internal/node-logic-flows/chat_utility/chatSend.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | chat_utility |-| **Backend handler** | [`server/services/handlers/utility.py::handle_chat_send`](../../../server/services/handlers/utility.py) |+| **Backend handler** | [`server/nodes/chat/chat_send/__init__.py`](../../../server/nodes/chat/chat_send/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("send")` || **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) || **Skill (if any)** | - || **Dual-purpose tool** | no |@@ -28,23 +28,27 @@ chat), `chatSend` pushes into a remote chat host.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `host` | string | `localhost` | no | - | Chat backend host |-| `port` | number | `8080` | no | - | Chat backend port |+| `port` | number | `8080` | no | - | Chat backend port (1..65535) || `session_id` | string | `default` | no | - | Chat session identifier |-| `api_key` | string | `""` | no | - | Auth token forwarded to the chat backend |-| `content` | string | `""` | yes | - | Message body |+| `api_key` | string (password) | `""` | no | - | Auth token forwarded to the chat backend |+| `content` | string | `""` | conditional | - | Message body. Defaults to `""` at the Param level; the op raises if both `content` and `message` are empty |+| `message` | string | `""` | no | - | Legacy alias for `content` (used as fallback when `content` is empty) |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | The `result` payload returned by the chat backend |+| `output-main` | object | `{ sent: true, message_id, ...rest }` from the chat backend |### Output payload (TypeScript shape)```ts+// ChatSendOutput (model_config extra="allow"){- // Pass-through of whatever the chat backend's send_message RPC returned,- // e.g. { message_id: string, timestamp: string, ... }+ sent: true; // hard-set on success+ message_id: string | null; // from RPC result.message_id+ // plus every other key from the RPC result payload (extra="allow"),+ // e.g. timestamp[key: string]: unknown;}```@@ -53,24 +57,26 @@ chat), `chatSend` pushes into a remote chat host.```mermaidflowchart TD- A[Receive params] --> B{content non-empty?}- B -- no --> E[Return error envelope: content required]+ A[Receive params] --> B{content or message non-empty?}+ B -- no --> E[Raise RuntimeError: Message content is required]B -- yes --> C[Call services.chat_client.send_chat_message]C -- JSON-RPC error --> EC -- RPC ok --> F{result.success?}F -- false --> E- F -- true --> G[Return success envelope with result.result]+ F -- true --> G[Return ChatSendOutput sent=true + message_id + rest]```## Decision Logic-- **Validation**: empty `content` raises `ValueError("Message content is required")`- which is caught and returned as `success=false`.-- **Branches**: success vs error branch on RPC response `success` flag.+- **Validation**: empty `content` AND empty `message` raises+ `RuntimeError("Message content is required")`.+- **Branches**: success vs error branch on RPC response `success` flag;+ `content = params.content or params.message` (legacy alias fallback).- **Fallbacks**: `host`/`port`/`session_id` default to `localhost:8080/default`,`api_key` defaults to empty string.-- **Error paths**: every exception is logged via `logger.error` and returned as- `{success: false, error: str(e)}` with `node_id`, `node_type`, timing fields.+- **Error paths**: `not result.success` raises `RuntimeError(result.error or "chatSend failed")`;+ `BaseNode.execute()` wraps any raised exception into the standard error envelope+ (`success=false`, `error`, `node_id`, `node_type`, timing fields).## Side Effects@@ -96,8 +102,8 @@ flowchart TD- Any exception from `send_chat_message` (connection refused, timeout, malformedRPC response) is swallowed and surfaced as `success=false` with the stringifiederror. No retry is performed.-- `port` is coerced via `int(parameters.get('port', 8080))`; a non-numeric port- raises `ValueError` and is returned as `success=false`.+- `port` is validated by the Pydantic Param (`int`, `ge=1, le=65535`); a non-coercible+ or out-of-range value fails Param validation before the op runs.- Templates in `content` (e.g. `{{aiAgent.response}}`) are resolved upstream by`ParameterResolver`; this handler never sees unresolved `{{...}}`.- `session_id` here is the chat-backend session, not the MachinaOs workflowdiff --git a/docs-internal/node-logic-flows/chat_utility/console.md b/docs-internal/node-logic-flows/chat_utility/console.mdindex 72037213..32c3d9d9 100644--- a/docs-internal/node-logic-flows/chat_utility/console.md+++ b/docs-internal/node-logic-flows/chat_utility/console.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | chat_utility |-| **Backend handler** | [`server/services/handlers/utility.py::handle_console`](../../../server/services/handlers/utility.py) |+| **Backend handler** | [`server/nodes/utility/console/__init__.py`](../../../server/nodes/utility/console/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("log")` (logic fully inlined in the plugin) || **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) || **Skill (if any)** | - || **Dual-purpose tool** | no |@@ -23,35 +23,39 @@ spliced into a pipeline without breaking data flow.|--------|-----------------|----------|---------|| `input-main` | main | no | Any number of upstream outputs merged into `input_data` |-`console` is dispatched via `NodeExecutor._dispatch`'s special-handlers branch,-which resolves upstream outputs via `_get_connected_outputs_with_info` and-passes `connected_outputs` (keyed by source node type) and `source_nodes`-(ids/types/labels of source nodes) to the handler.+`console` declares its type in `NodeExecutor._NEEDS_CONNECTED_OUTPUTS`, so the+executor injects `connected_outputs` (keyed by source node type) and+`source_nodes` (ids/types/labels of source nodes) into `ctx.raw` before+dispatching. The op reads `ctx.raw.get("connected_outputs")` /+`ctx.raw.get("source_nodes")`. The node has `hide_output_handle = True` and+`ui_hints = {"isConsoleSink": True}`, so it renders as a sink with only an+`input-main` handle on the canvas.## Parameters| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `label` | string | `""` | no | - | Display label for this console entry |-| `logMode` | string | `all` | no | - | `all`, `field`, or `expression` |+| `log_mode` | string | `all` | no | - | `all`, `field`, or `expression` |+| `field_path` | string | `""` | no | `log_mode=field` | Dot path like `data.items[0].name` |+| `expression` | string | `""` | no | `log_mode=expression` | Pre-resolved template output || `format` | string | `json` | no | - | `json`, `json_compact`, `text`, or `table` |-| `fieldPath` | string | `""` | no | `logMode=field` | Dot path like `data.items[0].name` |-| `expression` | string | `""` | no | `logMode=expression` | Pre-resolved template output |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | `{label, logged_at, format, data, formatted, ...input_data}` |+| (no output handle) | object | `hide_output_handle = True` — renders as a sink, but the op still RETURNS the pass-through dict below so the output store can serve `{{console.*}}` downstream |### Output payload (TypeScript shape)```ts+// ConsoleOutput (model_config extra="allow"){label: string; // user label or "Console (<nodeid[:8]>)"logged_at: string; // ISO timestampformat: string; // format parameter (json/text/table/json_compact)- data: unknown; // the logged value (respects logMode)+ data: unknown; // the logged value (respects log_mode)formatted: string; // stringified representation for display// plus every key from the merged upstream input so downstream nodes// can keep templating against the upstream shape@@ -63,12 +67,12 @@ passes `connected_outputs` (keyed by source node type) and `source_nodes````mermaidflowchart TD- A[Receive params + connected_outputs + source_nodes] --> B[Merge upstream dicts into input_data]- B --> C{logMode}+ A[Receive params + ctx.raw connected_outputs + source_nodes] --> B[Merge upstream dicts into input_data]+ B --> C{log_mode}C -- all --> D[log_value = input_data]- C -- field --> E{fieldPath has curly braces OR key exists?}- E -- resolved template --> F[log_value = fieldPath as-is]- E -- literal path --> G[log_value = _navigate_field_path]+ C -- field --> E{first segment of field_path is a key in input_data?}+ E -- no --> F[log_value = field_path as-is resolved scalar]+ E -- yes --> G[log_value = _navigate_field_path]C -- expression --> H[log_value = expression or input_data]C -- unknown --> DD --> I[_format_console_output]@@ -83,25 +87,25 @@ flowchart TD- **Validation**: none; all parameters optional.- **Branches**:- - `logMode=all` - logs the full merged `input_data`.- - `logMode=field` + `fieldPath` without `{{` and not a key in `input_data` ->- treat `fieldPath` as a pre-resolved template value.- - `logMode=field` + valid literal path -> `_navigate_field_path` returns- nested value or `None` on mismatch.- - `logMode=expression` -> use resolved `expression` or fall back to+ - `log_mode=all` - logs the full merged `input_data`.+ - `log_mode=field`: takes the first segment of `field_path` (before `.` / `[`).+ If that segment matches a top-level key in `input_data` -> `_navigate_field_path`.+ Otherwise the resolver already replaced `{{...}}` with a scalar, so log+ `field_path` as-is. (Empty `field_path` -> `input_data`.)+ - `log_mode=expression` -> use resolved `expression` or fall back to`input_data` when empty.- - Any other `logMode` falls through to `input_data` (via `match _`).+ - Any other `log_mode` falls through to `input_data` (via `match _`).- **Fallbacks**: non-dict upstream output is wrapped as `{"value": output}` in`input_data`; label defaults to `Console (<first 8 chars of node_id>)`.-- **Error paths**: top-level try/except returns `success=false` with the- stringified exception.+- **Error paths**: no in-op try/except; any raised exception is wrapped by+ `BaseNode.execute()` into the standard error envelope.## Side Effects- **Database writes**: none (console logs are transient; they are NOT persistedvia `database.add_console_log` in this handler - persistence happens via theWebSocket broadcast consumer in the frontend/DB layer).-- **Broadcasts**: `StatusBroadcaster.broadcast_console_log` with+- **Broadcasts**: `get_status_broadcaster().broadcast_console_log(...)` with`{node_id, label, timestamp, data, formatted, format, workflow_id,source_node_id, source_node_type, source_node_label}`.- **External API calls**: none.@@ -119,10 +123,10 @@ flowchart TD- `_navigate_field_path` returns `None` for any missing key or out-of-rangeindex, so `log_value` silently becomes `null` rather than raising.-- `fieldPath` heuristic: the handler treats a field path as "already a resolved- template value" when it has no `{{` AND is not a key in `input_data`. This- can misfire if the user's literal path happens not to match a top-level key- even though the intent was to navigate.+- `field_path` heuristic: the op treats a field path as "already a resolved+ template scalar" when its first segment is NOT a top-level key in `input_data`.+ This can misfire if the user's literal path happens not to match a top-level+ key even though the intent was to navigate.- `format=table` falls back to indented JSON for non-tabular data (not a listof dicts).- JSON formatting uses `default=str` so non-JSON-serialisable types becomediff --git a/docs-internal/node-logic-flows/chat_utility/fileHandler.md b/docs-internal/node-logic-flows/chat_utility/fileHandler.mdindex 4aa4f205..5f70f78a 100644--- a/docs-internal/node-logic-flows/chat_utility/fileHandler.md+++ b/docs-internal/node-logic-flows/chat_utility/fileHandler.md@@ -3,18 +3,19 @@| Field | Value ||------|-------|| **Category** | chat_utility |-| **Backend handler** | [`server/services/handlers/utility.py::handle_file_handler`](../../../server/services/handlers/utility.py) |+| **Backend handler** | [`server/nodes/text/file_handler/__init__.py`](../../../server/nodes/text/file_handler/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("wrap")` → [`server/services/text.py::TextService.execute_file_handler`](../../../server/services/text.py) || **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) || **Skill (if any)** | - || **Dual-purpose tool** | no |## Purpose-Legacy file-shape wrapper retained from the Node.js-era engine. Does NOT touch-the filesystem - it only composes a descriptive dict (`fileName`, `fileType`,-`content`, `size`) around an already-in-memory string. For real filesystem-access use the `fileRead`, `fileModify`, `fsSearch`, and `fileDownloader`-nodes in the `code_fs_process` and `document` categories.+Content-metadata wrapper (NOT file I/O), retained from the Node.js-era engine.+It only composes a descriptive dict (`fileName`, `fileType`, `content`, `size`)+around an already-in-memory string and returns a `{type: "file", data: {...}}`+envelope for downstream nodes. For real filesystem access use the `fileRead`,+`fileModify`, `fsSearch`, and `fileDownloader` nodes in the `code_fs_process`+and `document` categories.## Inputs (handles)@@ -26,9 +27,13 @@ nodes in the `code_fs_process` and `document` categories.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `fileType` | string | `generic` | no | - | Arbitrary tag stored in output |-| `content` | string | `""` | no | - | In-memory file contents |-| `fileName` | string | `untitled.txt` | no | - | Display file name |+| `file_type` | enum | `generic` | no | - | One of `generic`, `markdown`, `text`, `json`, `csv`, `html`, `xml` — content-type hint stored in output |+| `file_name` | string | `untitled.txt` | no | - | Filename label attached to the wrapped content |+| `content` | string | `""` | no | - | In-memory text content to wrap (8 rows) |++Note: parameters in are snake_case (`file_type` / `file_name` / `content`), but+the service's output dict keeps camelCase keys (`fileName` / `fileType`) by+historical wire contract — the frontend reads them that way.## Outputs (handles)@@ -39,13 +44,15 @@ nodes in the `code_fs_process` and `document` categories.### Output payload (TypeScript shape)```ts+// FileHandlerOutput (model_config extra="allow"); the service returns+// result = { type, data, nodeId, timestamp }, which the op returns verbatim.{type: "file";data: {fileName: string;fileType: string;content: string;- size: number; // len(content), bytes-of-str length+ size: number; // len(content) — character count, not byte countprocessed: true;processingType: string; // === fileTypenodeId: string;@@ -84,8 +91,7 @@ flowchart TD## External Dependencies- **Credentials**: none.-- **Services**: `TextService` injected via `functools.partial` in- `NodeExecutor._build_handler_registry`.+- **Services**: `TextService` obtained via `services.plugin.deps.get_text_service()`.- **Python packages**: stdlib only.- **Environment variables**: none.diff --git a/docs-internal/node-logic-flows/chat_utility/gmaps_create.md b/docs-internal/node-logic-flows/chat_utility/gmaps_create.mdindex cc5f597c..dcec5aa0 100644--- a/docs-internal/node-logic-flows/chat_utility/gmaps_create.md+++ b/docs-internal/node-logic-flows/chat_utility/gmaps_create.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | chat_utility (grouping) / location (functional domain) |-| **Backend handler** | [`server/services/handlers/utility.py::handle_create_map`](../../../server/services/handlers/utility.py) -> [`server/services/maps.py::MapsService.create_map`](../../../server/services/maps.py) |+| **Backend handler** | [`server/nodes/location/gmaps_create/__init__.py`](../../../server/nodes/location/gmaps_create/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("create")` → [`server/nodes/location/_service.py::MapsService.create_map`](../../../server/nodes/location/_service.py) || **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) || **Skill (if any)** | - || **Dual-purpose tool** | no (display-only; `gmaps_locations` and `gmaps_nearby_places` are the dual-purpose pair) |@@ -23,13 +23,20 @@ the Maps API - it assembles the URL from parameters and tracks a## Parameters+Params model (`GmapsCreateParams`):+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `api_key` | string | - | yes (auto-injected) | - | Google Maps API key; auto-injected from `auth_service.get_api_key('google_maps')` or `settings.google_maps_api_key` |-| `lat` | number | `40.7128` | no | - | Latitude, -90 .. 90 |-| `lng` | number | `-74.0060` | no | - | Longitude, -180 .. 180 |-| `zoom` | number | `13` | no | - | Zoom level, 0 .. 21 |-| `map_type_id` | string | `ROADMAP` | no | - | One of `ROADMAP`, `SATELLITE`, `HYBRID`, `TERRAIN` |+| `center_lat` | number | `0.0` | no | - | Map center latitude |+| `center_lng` | number | `0.0` | no | - | Map center longitude |+| `zoom` | number | `10` | no | - | Zoom level, `ge=1, le=20` (1=world, 20=street) |+| `map_type` | enum | `roadmap` | no | - | One of `roadmap`, `satellite`, `hybrid`, `terrain` |+| `options` | object | `{}` | no | - | Map customization (`disable_default_ui`, `zoom_control`, `street_view_control`, `map_type_control`, `fullscreen_control`); 4 rows |++The Google Maps API key is NOT a node Param — `MapsService.create_map` reads it+from `parameters.get("api_key")` (absent here) falling back to+`settings.google_maps_api_key`. The `GoogleMapsCredential` (`google_maps`) is+declared on the node for credential-modal visibility.## Outputs (handles)@@ -66,15 +73,18 @@ flowchart TD## Decision Logic-- **Validation**:- - `api_key` required (from params, else `settings.google_maps_api_key`).+- **Validation** (inside `MapsService.create_map`):+ - `api_key` required (from `parameters.get("api_key")`, else `settings.google_maps_api_key`).- `lat`/`lng` validated via `validate_coordinates`.- `zoom` validated via `validate_zoom_level` (0-21).- - `map_type` restricted to the four Google Maps MapTypeId values.+ - `map_type` restricted to `["ROADMAP", "SATELLITE", "HYBRID", "TERRAIN"]` (uppercase).- **Branches**: success envelope vs error envelope.-- **Fallbacks**: defaults centre the map on New York City.-- **Error paths**: any raised `ValueError` bubbles into the generic- `except Exception` and returns `success=false`.+- **Fallbacks**: when `lat`/`lng`/`zoom`/`map_type_id` keys are absent in the+ dumped params, the service centres on New York City (40.7128, -74.0060), zoom 13,+ `ROADMAP` (see the key-mismatch note under Edge cases).+- **Error paths**: on `not response.success` the op raises+ `NodeUserError(response.error or "Map create failed")` (single WARN line,+ no traceback); `BaseNode.execute()` builds the structured error envelope.## Side Effects@@ -88,21 +98,26 @@ flowchart TD## External Dependencies-- **Credentials**: Google Maps API key via `auth_service.get_api_key('google_maps')`- or `settings.google_maps_api_key` (env `GOOGLE_MAPS_API_KEY`).-- **Services**: `MapsService` injected via `functools.partial` in- `NodeExecutor._build_handler_registry`.+- **Credentials**: `GoogleMapsCredential` (`google_maps`); the service resolves+ the key from `parameters.get("api_key")` else `settings.google_maps_api_key`+ (env `GOOGLE_MAPS_API_KEY`).+- **Services**: `MapsService` obtained via `services.plugin.deps.get_maps_service()`.- **Python packages**: stdlib only for this path (googlemaps is used by othermap handlers, not this one).- **Environment variables**: `GOOGLE_MAPS_API_KEY` (fallback).## Edge cases & known limits+- Param/service key mismatch: the plugin Params are `center_lat` / `center_lng`+ / `map_type`, but `MapsService.create_map` reads `lat` / `lng` / `map_type_id`.+ With the current `params.model_dump()` wiring the service falls back to its+ NYC / ROADMAP defaults regardless of the configured center / map type.+ (Documented, not fixed here.)- The static map URL embeds the API key in plain text - any downstream node orlog that captures the output leaks the key. Treat the output as sensitive.- Size is hard-coded to `600x400`; there is no parameter to override.-- `map_type` comparison is case-sensitive; `roadmap` (lowercase) is rejected as- invalid even though Google Maps accepts it.+- `map_type` validation is case-sensitive (uppercase `ROADMAP`...); the lowercase+ enum values the Params expose would be rejected if they reached the service.- Cost tracking is optimistic: the $0.002 is charged even if the frontendnever actually fetches the static map URL.diff --git a/docs-internal/node-logic-flows/chat_utility/teamMonitor.md b/docs-internal/node-logic-flows/chat_utility/teamMonitor.mdindex e2f5667a..e1f0b095 100644--- a/docs-internal/node-logic-flows/chat_utility/teamMonitor.md+++ b/docs-internal/node-logic-flows/chat_utility/teamMonitor.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | chat_utility |-| **Backend handler** | [`server/services/handlers/utility.py::handle_team_monitor`](../../../server/services/handlers/utility.py) |+| **Backend handler** | [`server/nodes/utility/team_monitor/__init__.py`](../../../server/nodes/utility/team_monitor/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("monitor")` || **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) || **Skill (if any)** | - || **Dual-purpose tool** | no |@@ -26,7 +26,12 @@ architecture.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `maxHistoryItems` | number | `50` | no | - | Cap on `recent_events` slice length |+| `team_id` | string | `""` | no | - | Explicit team id (usually resolved from upstream output instead) |+| `auto_refresh` | boolean | `true` | no | - | UI flag for periodic refresh of the monitor panel |+| `max_history_items` | number | `50` | no | - | Cap on `recent_events` slice length (`ge=1`) |++The node carries `ui_hints = {"isMonitorPanel": True, "hideInputSection": True,+"hideOutputSection": True}` — it renders as a dedicated monitor panel.## Outputs (handles)@@ -58,12 +63,12 @@ architecture.```mermaidflowchart TD- A[Receive params + context] --> B{team_id in context?}+ A[Receive params + ctx.raw] --> B{ctx.raw.team_id set?}B -- yes --> E[team_service.get_team_status]- B -- no --> C[Scan context.outputs for dict with team_id]+ B -- no --> C[Scan ctx.raw.outputs for dict with team_id]C -- found --> EC -- not found --> D[Return empty envelope with message=No team connected]- E --> F[Slice recent_events by maxHistoryItems]+ E --> F[Slice recent_events by max_history_items]F --> G[Return envelope with counters + lists]```@@ -71,14 +76,14 @@ flowchart TD- **Validation**: none.- **Branches**:- - `team_id` resolved via `context['team_id']` first, else the first connected- output that is a dict containing `team_id`.+ - `team_id` resolved via `ctx.raw["team_id"]` first, else the first value in+ `ctx.raw["outputs"]` that is a dict containing a truthy `team_id`.- When no `team_id` is found, returns an intentional success envelope withzero counters - the node does not error.- **Fallbacks**: counters default to 0 for any missing key; `members`,`active_tasks`, `recent_events` default to empty lists.-- **Error paths**: any exception from `team_service.get_team_status` is caught- and returned as `success=false` with the stringified error.+- **Error paths**: no in-op try/except; any exception from+ `get_team_status` is wrapped by `BaseNode.execute()` into the error envelope.## Side Effects@@ -100,8 +105,7 @@ flowchart TD## Edge cases & known limits-- `maxHistoryItems` is used as a negative slice (`[-max:]`); passing a- non-integer triggers a `TypeError` which surfaces as `success=false`.+- `max_history_items` is used as a negative slice (`[-max:]`) on `recent_events`.- The "find team_id in outputs" loop stops at the first dict it sees containinga truthy `team_id`; if two upstream outputs carry different team ids, behaviouris order-dependent.diff --git a/docs-internal/node-logic-flows/chat_utility/textGenerator.md b/docs-internal/node-logic-flows/chat_utility/textGenerator.mdindex 72308e90..09d735c4 100644--- a/docs-internal/node-logic-flows/chat_utility/textGenerator.md+++ b/docs-internal/node-logic-flows/chat_utility/textGenerator.md@@ -3,18 +3,18 @@| Field | Value ||------|-------|| **Category** | chat_utility |-| **Backend handler** | [`server/services/handlers/utility.py::handle_text_generator`](../../../server/services/handlers/utility.py) |+| **Backend handler** | [`server/nodes/text/text_generator/__init__.py`](../../../server/nodes/text/text_generator/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("generate")` → [`server/services/text.py::TextService.execute_text_generator`](../../../server/services/text.py) || **Tests** | [`server/tests/nodes/test_chat_utility.py`](../../../server/tests/nodes/test_chat_utility.py) || **Skill (if any)** | - || **Dual-purpose tool** | no |## Purpose-Trivial text factory node retained from the original Node.js-era workflow-engine. Emits a text string and its length (optionally with a timestamp). Used-by legacy workflows and as a scaffolding node during tests. Not surfaced in the-current React component palette but still reachable through the backend-registry.+Text factory node retained from the original Node.js-era workflow engine.+Emits a text string plus its length and an ISO timestamp. The Params expose a+`source` selector (`static` / `ai` / `file` / `api`) for future expansion, but+the current `TextService.execute_text_generator` only consumes `text` and+returns the static-text envelope regardless of `source`.## Inputs (handles)@@ -24,10 +24,18 @@ registry.## Parameters+Params model (`TextGeneratorParams`, `model_config extra="allow"`):+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `text` | string | `Hello World` | no | - | Text payload to emit |-| `includeTimestamp` | boolean | `true` | no | - | Include ISO timestamp in `data` |+| `source` | enum | `static` | no | - | One of `static`, `ai`, `file`, `api` (only `static` behaviour is implemented today) |+| `text` | string | `""` | no | - | Text payload to emit (4 rows) |+| `file_path` | string | `""` | no | - | Reserved for `source=file` (unused by the current service) |+| `api_url` | string | `""` | no | - | Reserved for `source=api` (unused by the current service) |++Note: the service reads `include_timestamp` (default `True`) but no Param exposes+it, so the ISO `timestamp` is always attached to `data`. If `text` is omitted the+service substitutes `"Hello World"`.## Outputs (handles)@@ -38,13 +46,16 @@ registry.### Output payload (TypeScript shape)```ts+// Service returns result = { type, data, nodeId, timestamp }, returned verbatim.+// The TextGeneratorOutput model (extra="allow") declares text/source but the+// op returns the service result dict, so the runtime shape is:{type: "text";data: {text: string;- length: number;+ length: number; // len(text) — character countnodeId: string;- timestamp?: string; // only when includeTimestamp=true+ timestamp: string; // always present (include_timestamp defaults True)};nodeId: string;timestamp: string;@@ -57,20 +68,18 @@ registry.flowchart TDA[Receive params] --> B[TextService.execute_text_generator]B --> C[Compose data with text/length/nodeId]- C --> D{includeTimestamp?}+ C --> D{include_timestamp default True}D -- yes --> E[Attach isoformat timestamp]- D -- no --> F[Skip timestamp]- E --> G[Return success envelope]- F --> G+ E --> G[Return service result envelope]```## Decision Logic-- **Validation**: none; both params have defaults.-- **Branches**: `includeTimestamp` toggles the nested timestamp.-- **Fallbacks**: `text` defaults to `Hello World`.-- **Error paths**: any exception inside `TextService.execute_text_generator`- is caught there and returned as `success=false`.+- **Validation**: none; all params have defaults.+- **Branches**: `source` is accepted but ignored; only the static-text path runs.+- **Fallbacks**: empty `text` falls back to `"Hello World"` inside the service.+- **Error paths**: the service catches exceptions and returns `success=false`;+ on `not response.success` the op raises `RuntimeError(response.error or "Text generator failed")`.## Side Effects@@ -83,8 +92,7 @@ flowchart TD## External Dependencies- **Credentials**: none.-- **Services**: `TextService` injected via `functools.partial` in- `NodeExecutor._build_handler_registry`.+- **Services**: `TextService` obtained via `services.plugin.deps.get_text_service()`.- **Python packages**: stdlib only.- **Environment variables**: none.diff --git a/docs-internal/node-logic-flows/code_fs_process/fileModify.md b/docs-internal/node-logic-flows/code_fs_process/fileModify.mdindex 1bd45aba..c70dc0e4 100644--- a/docs-internal/node-logic-flows/code_fs_process/fileModify.md+++ b/docs-internal/node-logic-flows/code_fs_process/fileModify.md@@ -3,8 +3,8 @@| Field | Value ||------|-------|| **Category** | code_fs_process / filesystem |-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_file_modify`](../../../server/services/handlers/filesystem.py) |-| **Backend** | [`deepagents.backends.LocalShellBackend`](https://github.com/langchain-ai/deepagents) |+| **Backend handler** | [`server/nodes/filesystem/file_modify/__init__.py::FileModifyNode.modify`](../../../server/nodes/filesystem/file_modify/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("modify")`) |+| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | [`server/skills/coding_agent/file-modify-skill/SKILL.md`](../../../server/skills/coding_agent/file-modify-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `file_modify` |@@ -13,8 +13,12 @@Writes a new file or edits an existing one inside the per-workflow workspace.Two operations: `write` (create/overwrite) and `edit` (string find-and-replace).-Delegates to `LocalShellBackend.write()` / `backend.edit()` with-`virtual_mode=True` sandboxing, invoked through `asyncio.to_thread()`.+Delegates to `backend.write()` / `backend.edit()` on `NushellBackend`+(`virtual_mode=True`, `inherit_env=True`) via `get_backend()`, invoked through+`asyncio.to_thread()`. Paths are normalised with `normalize_virtual_path()`+first. Because deepagents' `backend.write()` refuses to overwrite, the `write`+op first unlinks any pre-existing file at the resolved path (wholesale+create-or-replace), guarding against `IsADirectoryError`.## Inputs (handles)@@ -26,20 +30,21 @@ Delegates to `LocalShellBackend.write()` / `backend.edit()` with| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `operation` | options | `write` | no | - | `write` or `edit` |+| `operation` | `write` \| `edit` (Literal) | `write` | no | - | `write` or `edit` || `file_path` | string | `""` | yes | - | Target file path || `content` | string | `""` | yes (when `operation=write`) | `operation=write` | File content || `old_string` | string | `""` | yes (when `operation=edit`) | `operation=edit` | Text to find || `new_string` | string | `""` | no | `operation=edit` | Replacement text || `replace_all` | boolean | `false` | no | `operation=edit` | Replace every occurrence; when `false` the backend insists `old_string` is unique |-| `working_directory` | string | `""` | no | - | Overrides context workspace |++`FileModifyParams` uses `extra="ignore"` — `working_directory` is NOT exposed by+this node (the model drops unknown keys before `model_dump()`).## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `file_modify` tool result) |### Output payload@@ -53,46 +58,52 @@ Edit:{ operation: "edit"; file_path: string; occurrences: number }```+`node_output_schemas.FileModifyOutput` declares `operation` / `file_path` /+`occurrences` (the model's declared `Output` has `written` / `replacements`,+but the operation returns the dicts above; `_OutputBase`/`extra="allow"` keeps+them).+## Logic Flow```mermaidflowchart TD- A[handle_file_modify] --> B{file_path empty?}- B -- yes --> E[Return error:<br/>file_path is required]- B -- no --> C[_get_backend root=workspace]+ A[modify] --> B{file_path empty?}+ B -- yes --> E[raise NodeUserError:<br/>file_path is required]+ B -- no --> C[get_backend + normalize_virtual_path]C --> D{operation?}- D -- write --> W[to_thread backend.write file_path, content]- W --> W1{result.error?}- W1 -- yes --> Eenv[Return error result.error]- W1 -- no --> Wok[Return success<br/>operation=write, file_path=result.path]+ D -- write --> W[to_thread _do_write:<br/>unlink existing if file,<br/>raise IsADir if dir, backend.write]+ W -- OSError/ValueError --> Eenv[raise NodeUserError str e]+ W -- result.error --> Eenv+ W -- ok --> Wok[Return dict<br/>operation=write, file_path=result.path]D -- edit --> Ei{old_string empty?}- Ei -- yes --> Ereq[Return error:<br/>old_string is required for edit]+ Ei -- yes --> Ereq[raise NodeUserError:<br/>old_string is required for edit]Ei -- no --> Ed[to_thread backend.edit file_path,<br/>old_string, new_string, replace_all]- Ed --> Ed1{result.error?}- Ed1 -- yes --> Eenv- Ed1 -- no --> Edok[Return success<br/>occurrences=result.occurrences]- D -- other --> Unk[Return error:<br/>Unknown operation: <op>]+ Ed -- result.error --> Eenv+ Ed -- ok --> Edok[Return dict<br/>occurrences=result.occurrences]+ D -- other --> Unk[raise NodeUserError:<br/>Unknown operation: <op>]```## Decision Logic- **Validation**:- - empty `file_path` -> `"file_path is required"`.- - `operation=edit` with empty `old_string` -> `"old_string is required for- edit"`.- - Anything other than `write` / `edit` -> `"Unknown operation: <op>"`.-- **Backend-level errors**: both `backend.write()` and `backend.edit()`- return a result object with an `error` field. A non-empty `error` short-- circuits before the success envelope is built. Common backend errors:- non-unique `old_string` when `replace_all=False`, path escape with- `virtual_mode=True`.-- **`operations.edit` uniqueness constraint**: when `replace_all=False`, the- backend REQUIRES `old_string` to appear exactly once. Zero or multiple- matches trigger a backend-level error.+ - empty `file_path` -> `raise NodeUserError("file_path is required")`.+ - `operation=edit` with empty `old_string` -> `raise NodeUserError("old_string+ is required for edit")`.+ - Anything other than `write` / `edit` -> `raise NodeUserError("Unknown+ operation: <op>")` (Pydantic Literal already constrains it).+- **`write` overwrite handling**: `_do_write` resolves the path, raises+ `IsADirectoryError` if it's a directory, `unlink()`s an existing file, then+ calls `backend.write()` (which itself refuses to overwrite). `OSError` /+ `ValueError` are caught and re-raised as `NodeUserError`.+- **Backend-level errors**: a non-empty `result.error` from `write`/`edit`+ short-circuits with `raise NodeUserError(result.error)`. Common: non-unique+ `old_string` when `replace_all=False`, path escape with `virtual_mode=True`.+- **`edit` uniqueness constraint**: when `replace_all=False`, the backend+ REQUIRES `old_string` to appear exactly once. Zero or multiple matches+ trigger a backend-level error.- **`file_path` echoed from `result.path`**: on success the returned`file_path` comes from the backend's normalised path, falling back to the- user-supplied path only if `result.path` is empty.-- **Broad `except Exception`**: catches unexpected backend or I/O errors.+ normalised `file_path` only if `result.path` is empty.## Side Effects@@ -100,26 +111,27 @@ flowchart TD- **Broadcasts**: none.- **External API calls**: none.- **File I/O**:- - `os.makedirs(root, exist_ok=True)` on the workspace root.- - Writes/edits `<root>/<file_path>`.+ - `get_backend` ensures the workspace root exists.+ - For `write`: `unlink()` an existing file then writes `<root>/<file_path>`.+ - For `edit`: rewrites `<root>/<file_path>` in place.- **Subprocess**: none.## External Dependencies-- **Python packages**: `deepagents`.+- **Python packages**: `deepagents` (via `NushellBackend`).- **Environment variables**: `WORKSPACE_BASE_DIR`.## Edge cases & known limits- **Silent overwrite on `write`**: there is no "fail if exists" flag - every- `write` call replaces the file.-- **`edit` without a match**: returns an error envelope (backend surfaces the+ `write` call unlinks then replaces the file.+- **`edit` without a match**: raises `NodeUserError` (backend surfaces themessage). The handler never distinguishes "not found" from "multiple- matches" - both are string errors.+ matches" - both are `NodeUserError` strings.- **No encoding option**: the backend opens files in text mode with theplatform default encoding.-- **`working_directory` escape**: as in `fileRead`, the node trusts the- parameter which can bypass the workspace sandbox.+- **`working_directory` not exposed**: `extra="ignore"` means the sandbox+ cannot be widened via a node param on this node.- **`replace_all=true` with empty `new_string`**: effectively deletes everyoccurrence of `old_string`. No safeguard.- **Atomicity**: the backend does not do atomic writes - a crash mid-writediff --git a/docs-internal/node-logic-flows/code_fs_process/fileRead.md b/docs-internal/node-logic-flows/code_fs_process/fileRead.mdindex 871d1c2c..1812d8cb 100644--- a/docs-internal/node-logic-flows/code_fs_process/fileRead.md+++ b/docs-internal/node-logic-flows/code_fs_process/fileRead.md@@ -3,20 +3,24 @@| Field | Value ||------|-------|| **Category** | code_fs_process / filesystem |-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_file_read`](../../../server/services/handlers/filesystem.py) |-| **Backend** | [`deepagents.backends.LocalShellBackend`](https://github.com/langchain-ai/deepagents) (third-party) |+| **Backend handler** | [`server/nodes/filesystem/file_read/__init__.py::FileReadNode.read`](../../../server/nodes/filesystem/file_read/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("read")`) |+| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | [`server/skills/coding_agent/file-read-skill/SKILL.md`](../../../server/skills/coding_agent/file-read-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `file_read` |## Purpose-Reads a file within the per-workflow workspace. Delegates to-`LocalShellBackend.read()` from the `deepagents` package. The backend is-instantiated with `virtual_mode=True` and a `root_dir` pinned to-`<DATA_DIR>/workspaces/<workflow_slug>/` (or the node-level `working_directory`-override), so path traversal outside the workspace is rejected by the-backend itself.+Reads a file within the per-workflow workspace. Delegates to `backend.read()`+on `NushellBackend` (a `LocalShellBackend` subclass) via `get_backend()` in+[`_backend.py`](../../../server/nodes/filesystem/_backend.py). The backend is+instantiated with `virtual_mode=True`, `inherit_env=True`, and a `root_dir`+pinned to `<DATA_DIR>/workspaces/<workflow_slug>/` (resolution:+`working_directory` param > `ctx.workspace_dir` >+`Settings().workspace_base_resolved/default`), so path traversal outside the+workspace is rejected by the backend. The plugin first runs the requested+`file_path` through `normalize_virtual_path()`, which strips Windows drives /+POSIX root / UNC anchors before the backend's `..`/`~` rejection.All calls run inside `asyncio.to_thread()` because the underlying backendAPIs are synchronous file I/O.@@ -32,53 +36,64 @@ APIs are synchronous file I/O.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `file_path` | string | `""` | yes | - | File path relative to the workspace root |-| `offset` | number | `0` | no | - | 0-indexed starting line |-| `limit` | number | `100` | no | - | Max lines to read (1-10000 in UI) |-| `working_directory` | string | `""` | no | - | Overrides the context workspace; resolved before backend creation |+| `offset` | number | `0` (ge=0) | no | - | 0-indexed starting line |+| `limit` | number | `2000` (ge=1, le=10000) | no | - | Max lines to read |++`FileReadParams` uses `extra="ignore"`, so `working_directory` is NOT exposed by+this node — `get_backend()` reads it from the params dict, but the model drops+unknown keys before `model_dump()`. Workspace resolution effectively uses+`ctx.workspace_dir`.## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `file_read` tool result) |### Output payload```ts{- content: string; // File content returned by LocalShellBackend.read()- file_path: string; // Echo of the requested path+ content: string; // File content from backend.read()+ line_count: number; // len(content.splitlines())+ file_path: string; // Normalised path}```+The plugin returns a `FileReadOutput(content, line_count, file_path)`.+`node_output_schemas.FileReadOutput` declares `content` / `file_path` /+`encoding` (extra fields like `line_count` allowed by `_OutputBase`).+## Logic Flow```mermaidflowchart TD- A[handle_file_read] --> B{file_path empty?}- B -- yes --> E[Return error:<br/>file_path is required]- B -- no --> C[_get_backend:<br/>root = param.working_directory OR<br/>context.workspace_dir OR<br/>Settings().workspace_base_resolved/default]- C --> D[os.makedirs root, exist_ok=True]- D --> F[LocalShellBackend root_dir=root, virtual_mode=True]- F --> G[to_thread backend.read file_path, offset, limit]- G -- exception --> H[Return error envelope str e]- G -- ok --> I[Return success envelope<br/>content, file_path]+ A[read] --> B{file_path empty?}+ B -- yes --> E[raise NodeUserError:<br/>file_path is required]+ B -- no --> C[get_backend params, ctx.raw:<br/>root = ctx.workspace_dir OR<br/>Settings().workspace_base_resolved/default]+ C --> D[NushellBackend root_dir=root,<br/>virtual_mode=True, inherit_env=True]+ D --> N[file_path = normalize_virtual_path]+ N --> G[to_thread backend.read file_path, offset, limit]+ G -- FileNotFound/IsADir/ValueError --> H[raise NodeUserError str e]+ G -- result.error --> H2[raise NodeUserError result.error]+ G -- ok --> I[Return FileReadOutput<br/>content, line_count, file_path]```## Decision Logic-- **Validation**: empty `file_path` -> `"file_path is required"` error.-- **Workspace resolution precedence**: node param `working_directory` >- `context["workspace_dir"]` > `Settings().workspace_base_resolved/default`.-- **Directory creation**: the resolved root is `os.makedirs(... exist_ok=True)`- BEFORE the backend is built, so read against a fresh workflow works.-- **`virtual_mode=True`**: the backend rejects absolute paths and `..`- traversal that escape the root. Violations raise an exception which is- caught by the broad `except` and returned as a string.-- **Broad `except Exception`**: missing files, permission errors, decoding- errors all collapse to the same shape - users must parse the `error` string- to distinguish them.+- **Validation**: empty `file_path` -> `raise NodeUserError("file_path is required")`.+- **Workspace resolution precedence** (in `get_backend`): `working_directory`+ param (not exposed here, `extra="ignore"`) > `ctx.workspace_dir` >+ `Settings().workspace_base_resolved/default`.+- **Directory creation**: `get_backend` ensures the resolved root exists before+ building the backend, so a read against a fresh workflow works.+- **Path normalization**: `normalize_virtual_path()` strips drive/root/UNC+ anchors; `virtual_mode=True` then rejects `..`/`~` traversal that escapes the+ root.+- **Error paths**: `FileNotFoundError` / `IsADirectoryError` / `ValueError`+ (bad offset, path escape) are caught and re-raised as `NodeUserError`; a+ non-empty `result.error` from the backend also raises `NodeUserError`. Both+ produce a single WARN line + structured envelope from `BaseNode.execute()`.## Side Effects@@ -86,7 +101,7 @@ flowchart TD- **Broadcasts**: none.- **External API calls**: none.- **File I/O**:- - `os.makedirs(root, exist_ok=True)` on the resolved workspace root.+ - `get_backend` ensures the resolved workspace root exists.- Reads `<root>/<file_path>` via the backend.- **Subprocess**: none.@@ -94,25 +109,24 @@ flowchart TD- **Credentials**: none.- **Services**: none.-- **Python packages**: `deepagents` (imported lazily inside `_get_backend`),+- **Python packages**: `deepagents` (via `NushellBackend` in `_backend.py`),`asyncio`.- **Environment variables**: `WORKSPACE_BASE_DIR` (read by `core.config.Settings`).## Edge cases & known limits-- **Errors are strings, not structured**: "file not found", "permission- denied", "path escapes workspace" all return `{success: false, error: <str>}`- with no error code.-- **Binary files**: `LocalShellBackend.read()` decodes as text; a binary file- raises a `UnicodeDecodeError` that surfaces as a vague error string.-- **`offset`/`limit` typed as numbers**: negative offsets are NOT validated by- the handler - they are forwarded to the backend which may return an empty- string or raise.-- **`working_directory` can point outside the workspace root**: the handler- trusts it, so setting `working_directory="/"` effectively disables the- `virtual_mode` sandbox. Dangerous when the node is exposed as an AI tool.-- **No line-count cap**: `limit` default is 100 but the UI allows up to- 10000; very large files with `limit=10000` block the thread longer.+- **Errors are `NodeUserError`**: "file not found", "is a directory", "path+ escapes workspace", "bad offset" all become `error_type="NodeUserError"`+ envelopes with a string message (no distinct error code per case).+- **Binary files**: `backend.read()` decodes as text; a binary file raises a+ `UnicodeDecodeError` (uncaught here, so it surfaces with a full traceback as a+ generic error envelope, not `NodeUserError`).+- **`offset`/`limit` Pydantic-validated**: `offset>=0`, `limit` clamped 1-10000.+ Negative inputs are rejected at validation, not forwarded.+- **`working_directory` not exposed**: `FileReadParams` uses `extra="ignore"`,+ so the sandbox cannot be widened via a node param on this node.+- **`limit` default 2000**: very large files with `limit=10000` block the+ thread longer.## Relateddiff --git a/docs-internal/node-logic-flows/code_fs_process/fsSearch.md b/docs-internal/node-logic-flows/code_fs_process/fsSearch.mdindex 25565156..a477007e 100644--- a/docs-internal/node-logic-flows/code_fs_process/fsSearch.md+++ b/docs-internal/node-logic-flows/code_fs_process/fsSearch.md@@ -3,8 +3,8 @@| Field | Value ||------|-------|| **Category** | code_fs_process / filesystem |-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_fs_search`](../../../server/services/handlers/filesystem.py) |-| **Backend** | [`deepagents.backends.LocalShellBackend`](https://github.com/langchain-ai/deepagents) (`ls_info`, `glob_info`, `grep_raw`) |+| **Backend handler** | [`server/nodes/filesystem/fs_search/__init__.py::FsSearchNode.search`](../../../server/nodes/filesystem/fs_search/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("search")`) |+| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`; `ls_info`, `glob_info`, `grep_raw`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | [`server/skills/coding_agent/fs-search-skill/SKILL.md`](../../../server/skills/coding_agent/fs-search-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `fs_search` |@@ -12,8 +12,9 @@## PurposeThree-mode filesystem query node confined to the per-workflow workspace.-Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on-`LocalShellBackend` (`virtual_mode=True`) depending on `mode`.+Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on `NushellBackend`+(`virtual_mode=True`, `inherit_env=True`) via `get_backend()` depending on+`mode`. The `path` is run through `normalize_virtual_path()` first.## Inputs (handles)@@ -25,18 +26,20 @@ Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `mode` | options | `ls` | no | - | `ls`, `glob`, or `grep` |+| `mode` | `ls` \| `glob` \| `grep` (Literal) | `ls` | no | - | `ls`, `glob`, or `grep` || `path` | string | `.` | no | - | Directory path to search in (workspace-relative) |-| `pattern` | string | `""` | yes (when `mode != ls`) | `mode=glob`, `mode=grep` | Glob pattern or grep regex |-| `file_filter` | string | `""` | no | `mode=grep` | Glob pattern restricting which files are searched |-| `working_directory` | string | `""` | no | - | Overrides context workspace |+| `pattern` | string | `""` | yes (when `mode != ls`) | - | Glob pattern or grep regex |++`FsSearchParams` uses `extra="ignore"` — there are NO `file_filter` or+`working_directory` params; the model drops unknown keys. The grep mode passes+only `pattern` + `path` to `backend.grep_raw()`. (displayOptions are not+declared on these fields in the current Params model.)## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `fs_search` tool result) |### Output payload@@ -51,43 +54,43 @@ Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on`FileInfo` and `GrepMatch` are the dataclass shapes returned by the`deepagents` backend, converted via `dict(entry)`.+`node_output_schemas.FsSearchOutput` declares `path` / `entries` / `matches` /+`pattern` / `count` (extra fields allowed by `_OutputBase`).## Logic Flow```mermaidflowchart TD- A[handle_fs_search] --> C[_get_backend virtual_mode=True]+ A[search] --> C[get_backend + normalize_virtual_path]C --> D{mode?}D -- ls --> L[to_thread backend.ls_info path]- L --> Lok[Return success entries, count]+ L --> Lok[Return dict path, entries, count]D -- glob --> G{pattern empty?}- G -- yes --> Gp[Return error:<br/>pattern is required for glob mode]+ G -- yes --> Gp[raise NodeUserError:<br/>pattern is required for glob mode]G -- no --> Gr[to_thread backend.glob_info pattern, path]- Gr --> Grok[Return success matches, count]+ Gr --> Grok[Return dict path, pattern, matches, count]D -- grep --> Gg{pattern empty?}- Gg -- yes --> Ggp[Return error:<br/>pattern is required for grep mode]- Gg -- no --> Gx[to_thread backend.grep_raw pattern, path, glob=file_filter]+ Gg -- yes --> Ggp[raise NodeUserError:<br/>pattern is required for grep mode]+ Gg -- no --> Gx[to_thread backend.grep_raw pattern, path]Gx --> Gxt{result type?}- Gxt -- str error --> Gxerr[Return error<br/>error=result]- Gxt -- list --> Gxok[Return success matches, count]- D -- other --> U[Return error:<br/>Unknown mode: <mode>]+ Gxt -- str error --> Gxerr[raise NodeUserError result]+ Gxt -- list --> Gxok[Return dict path, pattern, matches, count]+ D -- other --> U[raise NodeUserError:<br/>Unknown mode: <mode>]```## Decision Logic- **`mode=ls`** requires nothing; defaults `path='.'`.-- **`mode=glob`** requires `pattern`. Missing pattern -> error envelope.-- **`mode=grep`** requires `pattern`. Optional `file_filter` is forwarded as- the backend's `glob` kwarg.+- **`mode=glob`** requires `pattern`. Missing pattern -> `raise NodeUserError`.+- **`mode=grep`** requires `pattern`. No file-filter kwarg is passed (only+ `pattern` + `path`).- **`grep_raw` polymorphic return**: the backend returns either a list of`GrepMatch` objects (success) OR a plain string (error description). The- handler checks `isinstance(result, str)` and turns a string into an error- envelope verbatim. This is the only place error details flow through as a- direct backend string rather than the generic `str(Exception)` path.-- **`file_filter` empty-string handling**: `file_filter or None` -> empty- string is coerced to `None` before forwarding.-- **Unknown modes**: returns error envelope `"Unknown mode: <mode>"`.-- **Broad `except Exception`**: catches backend-level surprises.+ plugin checks `isinstance(result, str)` and `raise NodeUserError(result)`.+- **Unknown modes**: `raise NodeUserError("Unknown mode: <mode>")` (Pydantic+ Literal already constrains it).+- All raises become single-WARN-line `error_type="NodeUserError"` envelopes via+ `BaseNode.execute()`.## Side Effects@@ -95,29 +98,29 @@ flowchart TD- **Broadcasts**: none.- **External API calls**: none.- **File I/O**:- - `os.makedirs(root, exist_ok=True)` on the workspace root.+ - `get_backend` ensures the workspace root exists.- Reads directory listings and file contents under `<root>/<path>`.- **Subprocess**: none (`grep_raw` is a Python regex walk in recent`deepagents`; not a `grep(1)` exec).## External Dependencies-- **Python packages**: `deepagents`.+- **Python packages**: `deepagents` (via `NushellBackend`).- **Environment variables**: `WORKSPACE_BASE_DIR`.## Edge cases & known limits- **`grep_raw` string-error path is unique**: unlike ls/glob which raise,- grep can "succeed" at the Python level but return a string. Frontends- expecting a consistent error shape across modes may miss this.+ grep can "succeed" at the Python level but return a string; the plugin+ converts that to a `NodeUserError` so the final envelope shape is consistent.- **Glob patterns match the backend's semantics**: `**` globs work, butthe backend's behaviour for symlinks, hidden files, and case sensitivityis inherited from `pathlib`.- **No maxResults cap**: a `**/*` glob over a huge workspace returns everymatch; the handler does not paginate. The caller must truncate.- **`path='.'` resolves against the workspace root, not the server CWD**.-- **`working_directory` escape**: same caveat as other filesystem nodes -- a caller can bypass the sandbox.+- **`working_directory` not exposed**: `extra="ignore"` means the sandbox+ cannot be widened via a node param on this node.- **Dataclass `dict()` conversion**: relies on `FileInfo` / `GrepMatch`being dataclasses (they implement `__iter__` via dataclass). A futurebackend upgrade that returns `TypedDict`s could break this.diff --git a/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.md b/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.mdindex 55f5b37f..4478d163 100644--- a/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.md+++ b/docs-internal/node-logic-flows/code_fs_process/javascriptExecutor.md@@ -3,8 +3,8 @@| Field | Value ||------|-------|| **Category** | code_fs_process / code |-| **Backend handler** | [`server/services/handlers/code.py::handle_javascript_executor`](../../../server/services/handlers/code.py) |-| **Node.js client** | [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py) |+| **Backend handler** | [`server/nodes/code/javascript_executor/__init__.py::JavaScriptExecutorNode.execute_op`](../../../server/nodes/code/javascript_executor/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`; base in [`_base.py`](../../../server/nodes/code/_base.py)) |+| **Node.js client** | [`server/nodes/code/_nodejs.py::get_nodejs_client`](../../../server/nodes/code/_nodejs.py) (singleton over [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py)) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | [`server/skills/coding_agent/javascript-skill/SKILL.md`](../../../server/skills/coding_agent/javascript-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `javascript_code` |@@ -12,17 +12,15 @@## PurposeRuns user JavaScript through the persistent Node.js executor server (Express +-tsx) that the Python backend spawns on startup. The handler does **not** spawn-`node` per call - it POSTs the code to `http://localhost:3020/execute` via an-async `aiohttp` client. The Node.js server evaluates the script inside a VM-sandbox and returns `{success, output, console_output, execution_time_ms}`.+tsx) that the Python backend spawns on startup. The plugin does **not** spawn+`node` per call - it POSTs the code to `http://localhost:3020/execute` via the+shared `get_nodejs_client()` singleton (an async `aiohttp` client). The Node.js+server evaluates the script and returns `{success, output, console_output, ...}`.-The handler merges the caller's upstream outputs into an `input_data` object,-injects the workflow's `workspace_dir` as a key, then forwards the payload.--Dispatched through the special-handlers branch at-[`node_executor.py:367-381`](../../../server/services/node_executor.py) so it-receives `connected_outputs`.+The plugin merges the caller's upstream outputs (`ctx.raw["connected_outputs"]`)+into an `input_data` object, injects the workflow's `workspace_dir`+(`ctx.workspace_dir`) as a key, then forwards the payload with `language="javascript"`.+`connected_outputs` is injected by the executor for code-executor node types.## Inputs (handles)@@ -34,22 +32,23 @@ receives `connected_outputs`.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `code` | string (code editor) | (boilerplate stub) | yes | - | JavaScript source. User must assign to `output` |-| `timeout` | number | `30` | no | - | Seconds - multiplied by 1000 and forwarded as millisecond timeout to the Node server |+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | JavaScript source. User must assign to `output` |+| `timeout` | number | `30` (ge=1, le=600) | no | - | Seconds - multiplied by 1000 and forwarded as millisecond timeout to the Node server |++`CodeExecutorParams` uses `extra="allow"` (extra params persist but are unread).-## Handler-level kwargs (bound via partial at registry setup)+## Node.js client singleton-| Name | Default | Description |-|------|---------|-------------|-| `nodejs_url` | `http://localhost:3020` | Overridable via `NODEJS_EXECUTOR_URL` at container wiring |-| `nodejs_timeout` | `30` | Seconds; overridable via `NODEJS_EXECUTOR_TIMEOUT` |+`get_nodejs_client()` in [`_nodejs.py`](../../../server/nodes/code/_nodejs.py) lazily+constructs one `NodeJSClient(base_url="http://localhost:3020", timeout=30)` shared+across the JS + TS plugins. The defaults are hard-coded in the helper signature+(no per-call kwargs from the plugin).## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `javascript_code` tool result) |### Output payload@@ -57,44 +56,42 @@ receives `connected_outputs`.{output: any; // Value the Node.js server parsed from `output` in the scriptconsole_output: string; // Captured console.log/.error etc.- timestamp: string; // ISO8601}```+`node_output_schemas.CodeExecutorOutput` declares only `output` (the+`_OutputBase` base allows extra fields like `console_output`).+## Logic Flow```mermaidflowchart TD- A[handle_javascript_executor] --> B{code strip empty?}- B -- yes --> E[Return error envelope:<br/>No code provided]- B -- no --> C[timeout_ms = int timeout * 1000]- C --> D[input_data = connected_outputs or {}<br/>inject workspace_dir]+ A[execute_op] --> B{code strip empty?}+ B -- yes --> E[raise NodeUserError:<br/>No code provided]+ B -- no --> C[timeout_ms = timeout * 1000]+ C --> D[input_data = ctx.raw connected_outputs or {}<br/>inject workspace_dir]D --> F[get_nodejs_client<br/>lazy singleton]F --> G[client.execute<br/>POST /execute<br/>language=javascript]- G -- exception --> H[Return error envelope<br/>error=str e]- G -- ok + success=true --> I[Return success envelope<br/>output + console_output]- G -- ok + success=false --> J[Return error envelope<br/>preserve console_output]+ G -- ClientConnectorError --> H1[raise NodeUserError<br/>JS executor not running on :3020]+ G -- success=false --> H2[raise NodeUserError<br/>error=result.error]+ G -- success=true --> I[Return dict<br/>output, console_output]```## Decision Logic-- **Validation**: `code.strip() == ""` -> immediate error envelope- `"No code provided"`.-- **Timeout unit mismatch**: UI accepts seconds, handler multiplies by 1000- before forwarding. Default 30 -> 30000 ms. No upper cap enforced in Python.+- **Validation**: `code.strip() == ""` -> `raise NodeUserError("No code provided")`.+- **Timeout unit mismatch**: UI accepts seconds, plugin multiplies by 1000+ before forwarding. Default 30 -> 30000 ms. Pydantic clamps the input to 1-600 s.- **`workspace_dir` injection**: unconditionally sets`input_data["workspace_dir"]`, shadowing any upstream node that happened toproduce a key with that name.-- **Client reuse**: `_nodejs_client` is a **module-global** singleton. First- call fixes the `base_url` and `timeout`; subsequent calls with different- kwargs silently reuse the first one. See Edge cases.-- **Node server error propagation**: if the Node.js server returns- `{success: false, error, console_output}`, the handler forwards the `error`- string and **keeps** `console_output` so users still see logs from the failed- run.-- **Broad `except Exception`**: any networking error (aiohttp.ClientError,- TimeoutError, ConnectionRefusedError) is caught, logged at ERROR, and- surfaced as an error envelope with `console_output=""`.+- **Client reuse**: `get_nodejs_client()`'s `_client` is a **module-global**+ singleton in `_nodejs.py`, fixed at the hard-coded `base_url`/`timeout`.+- **Node server error propagation**: if `result["success"]` is falsey, the+ plugin raises `NodeUserError(result["error"] or "JavaScript executor failed")`.+- **Sidecar down**: `aiohttp.ClientConnectorError` is caught and re-raised as a+ `NodeUserError` telling the LLM the Node executor is unreachable on+ localhost:3020 and to fall back to `python_executor`.## Side Effects@@ -107,8 +104,8 @@ flowchart TDpackages at `server/nodejs/user-packages/`.- **Subprocess**: none directly. The Node.js server itself is a long-livedsubprocess started by `main.py` lifespan.-- **Module-level state**: the `_nodejs_client` module global is created on- first use and never reset.+- **Module-level state**: the `_client` module global in `_nodejs.py` is created+ on first use and never reset.## External Dependencies@@ -122,10 +119,9 @@ flowchart TD## Edge cases & known limits-- **Module-level client singleton**: `_nodejs_client` is cached on first call.- If the injected `nodejs_url` / `nodejs_timeout` kwargs change mid-process- (e.g., tests re-wiring the container), the cached instance keeps the- original values. Reset by `services.handlers.code._nodejs_client = None`.+- **Module-level client singleton**: `_client` in `_nodejs.py` is cached on+ first call at the hard-coded `base_url`/`timeout`. Reset by setting+ `services.code._nodejs._client = None` (or `nodes.code._nodejs._client`).- **Node server down**: connection refused surfaces as`error="Cannot connect to host localhost:3020 ssl:default [Connect callfailed]"` or similar aiohttp message. No automatic retry.@@ -146,5 +142,5 @@ flowchart TD## Related- **Skills using this as a tool**: [`javascript-skill/SKILL.md`](../../../server/skills/coding_agent/javascript-skill/SKILL.md)-- **Sibling nodes**: [`typescriptExecutor`](./typescriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md)-- **Architecture docs**: [DESIGN.md](../../DESIGN.md)+- **Sibling nodes**: [`typescriptExecutor`](./typescriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md), [`montyExecutor`](./montyExecutor.md)+- **Architecture docs**: [DESIGN.md](../../DESIGN.md), [Plugin System](../../plugin_system.md)diff --git a/docs-internal/node-logic-flows/code_fs_process/montyExecutor.md b/docs-internal/node-logic-flows/code_fs_process/montyExecutor.mdnew file mode 100644index 00000000..6c051b32--- /dev/null+++ b/docs-internal/node-logic-flows/code_fs_process/montyExecutor.md@@ -0,0 +1,148 @@+# Monty Executor (`montyExecutor`)++| Field | Value |+|------|-------|+| **Category** | code_fs_process / code |+| **Backend handler** | [`server/nodes/code/monty_executor/__init__.py::MontyExecutorNode.execute_op`](../../../server/nodes/code/monty_executor/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`; base in [`_base.py`](../../../server/nodes/code/_base.py)) |+| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |+| **Skill (if any)** | [`server/skills/coding_agent/monty-skill/SKILL.md`](../../../server/skills/coding_agent/monty-skill/SKILL.md) |+| **Dual-purpose tool** | yes - tool name `sandboxed_python` |++## Purpose++A deny-by-default, hard-sandboxed alternative to [`pythonExecutor`](./pythonExecutor.md).+Runs AI-generated/untrusted Python through Pydantic's+[Monty](https://github.com/pydantic/monty) interpreter (`pydantic-monty==0.0.18`,+a Python subset implemented in Rust) with **ENFORCED** wall-clock + memory limits+and zero host access unless explicitly granted. The headline win over+`pythonExecutor` is that `timeout` and `max_memory_mb` are actually enforced.++Capabilities are dynamic: the caller (an LLM, when wired as the+`sandboxed_python` tool) selects from a fixed menu via the `capabilities` param.+Each selection wires a real Monty grant — `http_get` (SSRF-guarded host fetch+via an `external_functions` entry) and/or `workspace_read` / `workspace_write`+(a `MountDir` over the per-workflow workspace at `/workspace`). The default+(empty list) is pure deny-by-default. The program's last expression becomes+`output`; captured `print()` becomes `console_output`.++`pydantic_monty` is lazy-imported inside the op so the server still boots and+registers every other node if no wheel exists for the platform.++## Inputs (handles)++| Handle | Connection type | Required | Purpose |+|--------|-----------------|----------|---------|+| `input-main` | main | no | Upstream outputs exposed as the `input_data` dict inside the sandbox (from `ctx.raw["connected_outputs"]`, injected by the executor for code-executor node types) |++## Parameters++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | Python (Monty subset) source. Last expression is returned as `output` |+| `timeout` | number | `30` (ge=1, le=600) | no | - | Wall-clock limit in seconds, ENFORCED via `ResourceLimits.max_duration_secs` |+| `max_memory_mb` | number | `256` (ge=16, le=2048) | no | - | Memory limit in MB, ENFORCED via `ResourceLimits.max_memory` (converted to bytes) |+| `capabilities` | string[] (enum: `http_get`, `workspace_read`, `workspace_write`) | `[]` | no | - | Opt-in grants; empty = deny-by-default |++`MontyExecutorParams` uses `extra="allow"` (extra params persist but are unread).++## Outputs (handles)++| Handle | Shape | Description |+|--------|-------|-------------|+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `sandboxed_python` tool result) |++### Output payload++```ts+{+ output: any; // The program's last expression value+ console_output: string; // Captured print() output (Monty CollectString)+}+```++`montyExecutor` is mapped to `node_output_schemas.CodeExecutorOutput` (declares+only `output`; the `_OutputBase` base allows extra fields like `console_output`).++## Logic Flow++```mermaid+flowchart TD+ A[execute_op] --> B{code strip empty?}+ B -- yes --> E[raise NodeUserError:<br/>No code provided]+ B -- no --> I{import pydantic_monty ok?}+ I -- no --> Ie[raise NodeUserError:<br/>Monty unavailable, use python_code]+ I -- yes --> C[input_data = ctx.raw connected_outputs or {}]+ C --> Cap[_build_capabilities:<br/>http_get -> external_functions,<br/>workspace_* -> MountDir /workspace]+ Cap -- unknown cap --> Ee[raise NodeUserError:<br/>Unknown capability]+ Cap -- workspace cap & no workspace_dir --> Ew[raise NodeUserError]+ Cap -- ok --> L[ResourceLimits max_duration_secs, max_memory]+ L --> R[Monty code parse + to_thread m.run<br/>inputs, limits, external_functions,<br/>print_callback, mount]+ R -- MontyError --> Cl[classify by message ->raise NodeUserError]+ R -- ok --> O[output = result.output<br/>console_output = collector.output]+```++## Decision Logic++- **Validation**: `code.strip() == ""` -> `raise NodeUserError("No code provided")`.+- **Missing wheel**: `ImportError` on `import pydantic_monty` -> `NodeUserError`+ telling the caller Monty is unavailable and to use `python_code` instead.+- **Capability resolution** (`_build_capabilities`): unknown capability ->+ `NodeUserError`. `http_get` wires an SSRF-guarded `http_get(url)` host+ function. `workspace_read` / `workspace_write` require `ctx.workspace_dir`+ (else `NodeUserError`) and mount it at `/workspace` (`read-only` /+ `read-write`).+- **Enforced limits**: `timeout` -> `max_duration_secs`, `max_memory_mb * 1024+ * 1024` -> `max_memory`. Both are enforced by the Rust interpreter.+- **Error classification**: all Monty failures subclass `MontyError`; the op+ classifies by message into: unsupported-feature (`does not yet support` /+ `notimplementederror`), time-limit, memory-limit, `MontySyntaxError`, and+ plain runtime errors (incl. exceptions raised by a capability host function+ like a blocked `http_get`). Each becomes a `NodeUserError` with an actionable+ hint. `class` surfaces a `MontyRuntimeError` at construct time and is caught+ by the same handler.+- **Result unwrap**: `result.output` if the result is a `MontyComplete`, else+ the raw result.++## Side Effects++- **Database writes**: none.+- **Broadcasts**: none from this op.+- **External API calls**: only when `http_get` is granted — an outbound+ `httpx.get(url)` (follow-redirects, 15 s timeout) to a publicly-routable host.+- **File I/O**: only when `workspace_read` / `workspace_write` is granted — the+ sandbox sees `/workspace` mapped to `ctx.workspace_dir` (read-only or+ read-write).+- **Subprocess**: none. Monty runs in-process; `m.run` is offloaded via+ `asyncio.to_thread` so the CPU-bound Rust execution doesn't block the loop.++## External Dependencies++- **Credentials**: none.+- **Services**: none.+- **Python packages**: `pydantic-monty==0.0.18` (lazy-imported), `httpx` (only+ for the `http_get` capability), `ipaddress` / `socket` (SSRF guard).+- **Environment variables**: none.++## Edge cases & known limits++- **Python SUBSET only**: supports def/closures/lambda, if/for/while,+ try/except, comprehensions, f-strings, async def/await, and `import math|json|re`.+ NOT supported: `class`, yield/generators, `with`, `match`, `import+ random|collections|os`. Unsupported features raise `NodeUserError` pointing the+ caller to `python_code`.+- **Enforced limits** (unlike `pythonExecutor`): time + memory breaches are+ caught and surfaced as `NodeUserError`.+- **SSRF guard is best-effort**: `_is_public_url` re-resolves DNS and rejects+ loopback / private / link-local / reserved / multicast / unspecified+ addresses, but httpx re-resolves at request time so it is not TOCTOU-proof.+- **Workspace capability requires a workspace**: requesting `workspace_read` /+ `workspace_write` without a `ctx.workspace_dir` raises `NodeUserError`.+- **Missing wheel degrades gracefully**: a platform without a `pydantic-monty`+ wheel returns a `NodeUserError`, not a server crash — every other node still+ registers.++## Related++- **Skills using this as a tool**: [`monty-skill/SKILL.md`](../../../server/skills/coding_agent/monty-skill/SKILL.md)+- **Sibling nodes**: [`pythonExecutor`](./pythonExecutor.md), [`javascriptExecutor`](./javascriptExecutor.md), [`typescriptExecutor`](./typescriptExecutor.md), [`shell`](./shell.md), [`processManager`](./processManager.md)+- **Architecture docs**: [DESIGN.md](../../DESIGN.md), [Plugin System](../../plugin_system.md)diff --git a/docs-internal/node-logic-flows/code_fs_process/processManager.md b/docs-internal/node-logic-flows/code_fs_process/processManager.mdindex 1c6b65c3..cff52fef 100644--- a/docs-internal/node-logic-flows/code_fs_process/processManager.md+++ b/docs-internal/node-logic-flows/code_fs_process/processManager.md@@ -2,8 +2,8 @@| Field | Value ||------|-------|-| **Category** | code_fs_process / process |-| **Backend handler** | [`server/services/handlers/process.py::handle_process_manager`](../../../server/services/handlers/process.py) |+| **Category** | code_fs_process / process (plugin lives under `server/nodes/utility/`) |+| **Backend handler** | [`server/nodes/utility/process_manager/__init__.py::ProcessManagerNode.dispatch`](../../../server/nodes/utility/process_manager/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("dispatch")`) || **Service** | [`server/services/process_service.py::ProcessService`](../../../server/services/process_service.py) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | [`server/skills/terminal/process-manager-skill/SKILL.md`](../../../server/skills/terminal/process-manager-skill/SKILL.md) |@@ -17,10 +17,12 @@ workflow. Unlike [`shell`](./shell.md), processes spawned here inherit thethe Terminal tab via `broadcast_terminal_log()` and persists to per-process logfiles at `<workspace>/<agent_node_id>/.processes/<name>/{stdout,stderr}.log`.-Six operations: `start`, `stop`, `restart`, `send_input`, `list`, `get_output`.-The handler is a thin dispatcher over the `ProcessService` singleton; the-singleton owns a `{(workflow_id, name): ManagedProcess}` dict, the streaming-tasks, and the cleanup scheduler.+Six operations: `start`, `stop`, `restart`, `send_input`, `list`, `get_output`+(default `list`). The plugin's `dispatch` op is a thin wrapper over the+`ProcessService` singleton; the singleton owns a `{(workflow_id, name):+ManagedProcess}` dict, the streaming tasks, and the cleanup scheduler. The+wrapper's `_unwrap` raises `NodeUserError` on any service `{"success": False}`+envelope so the framework returns a clean failure.## Inputs (handles)@@ -32,38 +34,45 @@ tasks, and the cleanup scheduler.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `process_manager` | no | - | Tool name exposed to AI agents |-| `toolDescription` | string | (see frontend) | no | - | Description shown to LLM |-| `operation` | options | `start` | no | - | One of `start`, `stop`, `restart`, `send_input`, `list`, `get_output` |-| `name` | string | `""` | yes (all except `list`) | `operation in [start, stop, restart, send_input, get_output]` | Unique process name within the workflow |-| `command` | string | `""` | yes (`start` only) | `operation=start` | Shell command (parsed with `shlex.split`) |-| `working_directory` | string | `""` | no | `operation=start` | Defaults to `<workspace>/<node_id>` |-| `text` | string | `""` | yes (`send_input`) | `operation=send_input` | Text to write to stdin (newline auto-appended) |-| `stream` | string | `"stdout"` | no | - (tool-only) | `stdout` or `stderr` for `get_output` |-| `tail` | number | `50` | no | - (tool-only) | Tail lines; 0 means all lines from `offset` |-| `offset` | number | `0` | no | - (tool-only) | Skip N lines (only when `tail=0`) |+| `operation` | `start`/`stop`/`restart`/`list`/`send_input`/`get_output` (Literal) | `list` | no | - | Operation to run |+| `name` | string | `""` | yes (all except `list`) | - | Unique process name within the workflow (LLM `"None"` coerced to empty by `_clean`) |+| `command` | string | `""` | yes (`start` only) | `operation=start` | Command (`ProcessService` parses with `shlex.split`) |+| `cwd` | string | `""` | no | `operation=start` | Working directory; defaults to `<workspace>/<node_id>` |+| `env` | object (Dict[str,str]) | `{}` | no | `operation=start` | Declared param, but the `dispatch` op does NOT forward it to `svc.start` — currently inert |+| `input_text` | string | `""` | yes (`send_input`) | `operation=send_input` | Text to write to stdin (newline auto-appended by the service) |+| `stream` | `stdout`/`stderr` (Literal) | `stdout` | no | `operation=get_output` | Stream to read for `get_output` |+| `tail` | number | `100` (ge=1, le=10000) | no | `operation=get_output` | Tail N lines |++`ProcessManagerParams` uses `extra="ignore"`. There are NO `toolName` /+`toolDescription` / `working_directory` / `text` / `offset` params (the old+card listed pre-Wave-11 frontend fields). `get_output` is always called with+`offset=0`.## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `process_manager` tool result) |### Output payload per operation-- `start` / `stop` / `restart`: `{name, command, pid, status, started_at,- exit_code, working_directory, stdout_lines, stderr_lines, log_dir}`.-- `send_input`: `{sent: <text>}`.+The shapes below are whatever `ProcessService` returns (passed through `_unwrap`).+`node_output_schemas.ProcessManagerOutput` declares `operation` / `pid` /+`status` / `output` / `processes` (extra fields allowed by `_OutputBase`).++- `start` / `stop` / `restart`: the `ProcessInfo` dict from the service+ (`{name, command, pid, status, started_at, exit_code, working_directory,+ log_dir, ...}`).+- `send_input`: the service's send-input result dict.- `list`: `{processes: Array<ProcessInfo>}` for the current workflow.-- `get_output`: `{lines: string[], total: number, file: string}`.+- `get_output`: the service's `{lines, total, file, ...}` dict.## Logic Flow```mermaidflowchart TD- A[handle_process_manager] --> B[Build tool_args:<br/>clean 'None' strings,<br/>default working_directory to workspace/node_id]- B --> C[execute_process_manager]+ A[dispatch] --> B[name = _clean params.name<br/>agent_dir = workspace/node_id]+ B --> C[route on params.operation]C --> D{operation}D -- start --> S1{command blocked?<br/>rm, format, mkfs...}S1 -- yes --> Sblk[Return error:<br/>Destructive commands blocked]@@ -83,18 +92,22 @@ flowchart TDSt2 --> St3[wait exit, schedule 60s cleanup]St3 --> St4[Return success]D -- restart --> R[stop then start with same cmd+cwd]- D -- send_input --> I1[write text+newline to stdin.drain]- D -- list --> L[return info for workflow_id processes]- D -- get_output --> O[read log file, tail or offset slice]- D -- unknown --> U[Return error:<br/>Unknown operation]+ D -- send_input --> I1[svc.send_input name, wf, input_text]+ D -- list --> L[svc.list_processes workflow_id]+ D -- get_output --> O[svc.get_output name, wf, stream, tail, 0]+ D -- unknown --> U[raise NodeUserError:<br/>Unknown operation]```+Each service call (except `list`) is wrapped in `_unwrap`, which raises+`NodeUserError` if the service returned `{"success": False}`.+## Decision Logic-- **`_clean_arg()`**: LLMs sometimes pass the literal string `"None"` for- missing fields; the handler coerces `""` and `"None"` to empty.-- **`working_directory` fallback**: if the param is empty the handler uses- `<workspace>/<node_id>` (each agent node gets its own subfolder). If the+- **`_clean()`**: LLMs sometimes pass the literal string `"None"` for missing+ fields; the wrapper coerces `""` and `"None"` to empty (applied to `name`,+ `command`, `cwd`, `input_text`).+- **`cwd` fallback**: if `cwd` is empty the wrapper uses `agent_dir =+ <workspace>/<node_id>` (each agent node gets its own subfolder). If theworkspace is also empty, `ProcessService.start` falls back to`<workspace_base>/default` and mkdir's it.- **Workspace guardrail**: `cwd` must resolve inside `workspace_base_resolved`diff --git a/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.md b/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.mdindex 65288e66..3269247e 100644--- a/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.md+++ b/docs-internal/node-logic-flows/code_fs_process/pythonExecutor.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | code_fs_process / code |-| **Backend handler** | [`server/services/handlers/code.py::handle_python_executor`](../../../server/services/handlers/code.py) |+| **Backend handler** | [`server/nodes/code/python_executor/__init__.py::PythonExecutorNode.execute_op`](../../../server/nodes/code/python_executor/__init__.py) (dispatched via `BaseNode.execute()` + the `@Operation("execute")` method; base in [`_base.py`](../../../server/nodes/code/_base.py)) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | [`server/skills/coding_agent/python-skill/SKILL.md`](../../../server/skills/coding_agent/python-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `python_code` |@@ -19,9 +19,10 @@ are intentionally excluded - only `math`, `json`, `datetime`, `re`, `random`,redirected to an `io.StringIO` buffer so stdout becomes the `console_output`return field.-Dispatched through the special-handlers branch at-[`node_executor.py:367-381`](../../../server/services/node_executor.py) so it-receives `connected_outputs` (dict of upstream outputs keyed by source node id).+The plugin reads upstream outputs from `ctx.raw["connected_outputs"]` (dict of+upstream outputs keyed by source node id), which the executor injects for+code-executor node types, and `ctx.workspace_dir` for the per-workflow scratch+directory.## Inputs (handles)@@ -33,15 +34,17 @@ receives `connected_outputs` (dict of upstream outputs keyed by source node id).| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `code` | string (code editor) | (boilerplate stub) | yes | - | Python source. Must assign to the free variable `output` to emit a value |-| `timeout` | number | `30` | no | - | Read and coerced to int but **not enforced** - `exec()` is synchronous with no wall-clock guard |+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | Python source. Must assign to the free variable `output` to emit a value |+| `timeout` | number | `30` (ge=1, le=600) | no | - | Validated/clamped but **not enforced** - `exec()` is synchronous with no wall-clock guard. Use `montyExecutor` for an enforced limit |++`CodeExecutorParams` uses `extra="allow"`, so any extra params persist but are+not read by the operation.## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (the node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `python_code` tool result) |### Output payload@@ -49,48 +52,52 @@ receives `connected_outputs` (dict of upstream outputs keyed by source node id).{output: any; // Value assigned to `output` in the user code (None by default)console_output: string; // Captured stdout from redirected print()- timestamp: string; // ISO8601 local time}```-On failure the envelope is `{success: false, error, console_output: "", ...}`.-Note that `console_output` is **only** preserved on the success path; when an-exception fires mid-execution the already-captured buffer is discarded because-the `except` branch hard-codes `"console_output": console_output` but-`console_output` is only assigned after `exec()` finishes. See Edge cases.+On failure the operation raises `NodeUserError` (not an error envelope). The+framework's `BaseNode.execute()` catches it and returns+`{success: false, error_type: "NodeUserError", error, ...}` with a single WARN+line (no traceback). The error message preserves any captured stdout via a+`stdout before error:` suffix, so prints before the exception are NOT lost.+`node_output_schemas.CodeExecutorOutput` declares only `output` (the+`_OutputBase` base allows extra fields like `console_output`).## Logic Flow```mermaidflowchart TD- A[handle_python_executor] --> B{code strip empty?}- B -- yes --> E[Return error envelope:<br/>No code provided]- B -- no --> C[Read timeout<br/>int-coerce only, unused]+ A[execute_op] --> B{code strip empty?}+ B -- yes --> E[raise NodeUserError:<br/>No code provided]+ B -- no --> C[timeout validated by Pydantic, unused at runtime]C --> D[Build safe_builtins dict:<br/>abs..zip, math, json, captured_print]- D --> F[Build namespace:<br/>input_data=connected_outputs or {},<br/>workspace_dir=context,<br/>output=None]+ D --> F[Build namespace:<br/>input_data=ctx.raw connected_outputs or {},<br/>workspace_dir=ctx.workspace_dir,<br/>output=None]F --> G[exec code in namespace]- G -- exception --> H[Return error envelope<br/>error=str e, console_output=empty]+ G -- ImportError __import__ --> H1[raise NodeUserError<br/>import not allowed, list sandbox names]+ G -- other exception --> H2[raise NodeUserError<br/>ErrType at line N: msg + stdout before error]G -- ok --> I[output = namespace.get output<br/>console_output = StringIO.getvalue]- I --> J[Return success envelope<br/>output, console_output, timestamp]+ I --> J[Return dict<br/>output, console_output]```## Decision Logic-- **Validation**: `code.strip() == ""` -> immediate error envelope with message- `"No code provided"`.+- **Validation**: `code.strip() == ""` -> `raise NodeUserError("No code provided")`.- **Namespace seeding**: `input_data` defaults to `{}` if `connected_outputs` is- None. `workspace_dir` is pulled from `context["workspace_dir"]` but never+ None. `workspace_dir` is pulled from `ctx.workspace_dir` but nevervalidated; an empty string is fine.- **Print redirection**: `safe_builtins["print"]` is a wrapper that forces`file=stdout_capture`, so any user-supplied `file=` kwarg is **overwritten**.- **Output extraction**: `namespace.get("output", None)` - the user writing`output = <value>` is the only way to emit data. A top-level expression isevaluated but not captured.-- **No timeout enforcement**: the `timeout` parameter is int-coerced only as a- sanity check (raises on non-numeric strings) but never passed to a watchdog.- Long loops will block the async event loop for the whole backend.-- **Error path**: broad `except Exception` catches everything, logs via- `logger.error`, and returns `{success: false, error: str(e)}`.+- **No timeout enforcement**: the `timeout` parameter is Pydantic-validated+ (1-600) but never passed to a watchdog. Long loops will block the async event+ loop for the whole backend.+- **Error path**: `except Exception` re-raises as `NodeUserError`. `import X`+ (which hits the sandboxed builtins' missing `__import__`) gets a dedicated+ message listing the pre-injected names; other exceptions are formatted as+ `<ErrType> at line N: <msg>` (line walked from the `<string>` traceback frame)+ with any captured stdout appended.## Side Effects@@ -121,10 +128,9 @@ flowchart TD`math` and `json` are module objects, and user code can re-import anything via`().__class__.__base__.__subclasses__()` or similar well-known escapes. Treatthis node as trusted-input only.-- **`console_output` swallowed on error**: the `except` path hard-codes- `"console_output": console_output`, but `console_output` is initialised to- `""` and only overwritten **after** `exec()` succeeds. Any print()s before an- exception are therefore lost.+- **`console_output` preserved on error**: the `NodeUserError` message appends a+ `stdout before error:` block with whatever the StringIO captured before the+ exception, so prints before a failure are surfaced (not lost) in the error.- **`output=None` is indistinguishable from a user explicitly returning`None`**: both surface as `{"output": None}` in the envelope.- **`input_data` mutation is visible to later siblings**: the handler passes@@ -139,5 +145,5 @@ flowchart TD## Related- **Skills using this as a tool**: [`python-skill/SKILL.md`](../../../server/skills/coding_agent/python-skill/SKILL.md)-- **Sibling nodes**: [`javascriptExecutor`](./javascriptExecutor.md), [`typescriptExecutor`](./typescriptExecutor.md), [`shell`](./shell.md), [`processManager`](./processManager.md)-- **Architecture docs**: [DESIGN.md](../../DESIGN.md)+- **Sibling nodes**: [`montyExecutor`](./montyExecutor.md), [`javascriptExecutor`](./javascriptExecutor.md), [`typescriptExecutor`](./typescriptExecutor.md), [`shell`](./shell.md), [`processManager`](./processManager.md)+- **Architecture docs**: [DESIGN.md](../../DESIGN.md), [Plugin System](../../plugin_system.md)diff --git a/docs-internal/node-logic-flows/code_fs_process/shell.md b/docs-internal/node-logic-flows/code_fs_process/shell.mdindex 31f92ffc..5e92026d 100644--- a/docs-internal/node-logic-flows/code_fs_process/shell.md+++ b/docs-internal/node-logic-flows/code_fs_process/shell.md@@ -3,20 +3,31 @@| Field | Value ||------|-------|| **Category** | code_fs_process / filesystem |-| **Backend handler** | [`server/services/handlers/filesystem.py::handle_shell`](../../../server/services/handlers/filesystem.py) |-| **Backend** | [`deepagents.backends.LocalShellBackend.execute`](https://github.com/langchain-ai/deepagents) |+| **Backend handler** | [`server/nodes/filesystem/shell/__init__.py::ShellNode.execute_op`](../../../server/nodes/filesystem/shell/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`) |+| **Backend** | `NushellBackend.execute` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | [`server/skills/terminal/shell-skill/SKILL.md`](../../../server/skills/terminal/shell-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `shell_execute` |## Purpose-Runs a short-lived shell command inside the per-workflow workspace. Delegates-to `LocalShellBackend.execute()`, which invokes the command via `subprocess`-but intentionally **scrubs the system PATH**, so tools like `npm`, `node`,-`python`, and `git` are NOT resolvable unless supplied with an absolute path.-For long-running processes or tools that need the real PATH, users are-directed to [`processManager`](./processManager.md).+Runs a short-lived shell command inside the per-workflow workspace. The backend+is **Nushell** (`NushellBackend`, a `LocalShellBackend` subclass) constructed+with `inherit_env=True`, so external tools like `npm`, `node`, `python`, and+`git` ARE reachable on PATH (it falls back to upstream `LocalShellBackend.execute()`+/ POSIX `sh` / cmd.exe only when `nu` isn't installed). The command grammar is+Nushell, not bash — `&&` / `||` / `$VAR` / backticks / `>` do not work; see+`shell-skill/SKILL.md` for the Nu equivalents. For long-running processes+(dev servers, watchers) users are directed to [`processManager`](./processManager.md);+this node kills the command at `timeout`.++NOTE: the node's `description` metadata still reads "sandboxed; no system PATH",+which no longer matches the `inherit_env=True` backend — a code-side copy+mismatch (out of scope for this card).++A pre-flight regex (`_BASH_CHAIN_RE`) rejects bash chain operators (` && ` / ` || `)+up-front with a corrective message before Nushell's parser would emit a cryptic+`shell_andand` / `shell_oror` error.Runs synchronously under `asyncio.to_thread()` so the event loop keepsservicing other requests while the command executes.@@ -31,16 +42,18 @@ servicing other requests while the command executes.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `command` | string | `""` | yes | - | Shell command |-| `timeout` | number | `30` | no | - | Max seconds (UI clamps 1-300) |-| `working_directory` | string | `""` | no | - | Overrides context workspace |+| `command` | string | (required, `min_length=1`) | yes | - | Nushell command |+| `cwd` | string | `""` | no | - | Declared param, but `get_backend` resolves the working dir from `working_directory` (not exposed) / `ctx.workspace_dir`, so `cwd` does NOT change the backend root |+| `timeout` | number | `30` (ge=1, le=600) | no | - | Max seconds |++`ShellParams` uses `extra="ignore"` — `working_directory` is NOT exposed; the+backend root is `ctx.workspace_dir`.## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `shell_execute` tool result) |### Output payload@@ -53,29 +66,34 @@ servicing other requests while the command executes.}```+`node_output_schemas.ShellOutput` declares `stdout` / `exit_code` / `truncated`+/ `command`.+## Logic Flow```mermaidflowchart TD- A[handle_shell] --> B{command empty?}- B -- yes --> E[Return error:<br/>command is required]- B -- no --> C[_get_backend workspace root<br/>virtual_mode=True]+ A[execute_op] --> P{_BASH_CHAIN_RE matches<br/> && / || ?}+ P -- yes --> Pe[raise NodeUserError:<br/>use ; or try{}catch{}]+ P -- no --> C[get_backend workspace root<br/>NushellBackend virtual_mode, inherit_env]C --> D[to_thread backend.execute command, timeout]- D -- exception --> H[Return error envelope]- D -- ok --> I{exit_code?}+ D --> S[strip_ansi result.output]+ S --> I{exit_code?}I -- 124 --> I1[Log warning:<br/>Timed out]I -- nonzero --> I2[Log warning:<br/>Non-zero exit]I -- 0 --> I3[Log info: Completed]- I1 & I2 & I3 --> J[Return success envelope<br/>stdout, exit_code, truncated, command]+ I1 & I2 & I3 --> J[Return dict<br/>stdout, exit_code, truncated, command]```## Decision Logic-- **Validation**: empty `command` -> `"command is required"` error.-- **Sandbox PATH**: `LocalShellBackend` strips the system PATH before spawn,- so by default only built-ins and absolute paths work. This is a deliberate- safety choice; AI agents are steered toward `process_manager` for any- install/build tool invocation.+- **Validation**: empty `command` is rejected by Pydantic `min_length=1` (no+ manual check). A ` && ` / ` || ` in the command -> `raise NodeUserError` with+ a Nushell-equivalent hint (`;` or `try { … } catch { … }`).+- **Inherited PATH**: `NushellBackend(inherit_env=True)` — external tools+ (`npm`, `node`, `python`, `git`, …) ARE reachable. The previous "scrubbed+ PATH" framing no longer holds. AI agents are still steered toward+ `process_manager` for long-running daemons (the shell kills at `timeout`).- **Timeout handling**: `timeout` is forwarded to`backend.execute(command, timeout=timeout)`. On timeout the backend killsthe process and sets `exit_code=124` (POSIX convention). Logged at WARNING.@@ -90,8 +108,10 @@ flowchart TDbefore the operator-log line), so colour/cursor codes from tools like`vite`/`npm` don't render as garbage in the Output panel. `click.unstyle` isbyte-faithful apart from the stripped escapes.-- **Broad `except Exception`**: backend failures (missing binary,- `virtual_mode` violations) become an error envelope.+- **No catch-all error wrapper**: the op does not wrap `backend.execute` in a+ try/except — non-zero exit codes are returned as a normal success envelope+ with `exit_code != 0`. An unexpected backend exception surfaces with a full+ traceback via `BaseNode.execute()`'s generic path.## Side Effects@@ -99,23 +119,22 @@ flowchart TD- **Broadcasts**: none (but the process stdout/stderr is NOT streamed toTerminal - only `processManager` does that).- **External API calls**: none.-- **File I/O**: `os.makedirs(root, exist_ok=True)` on the workspace root. The- command itself may read/write under that root.-- **Subprocess**: one per call, via the backend's `subprocess.run`- (synchronous, scrubbed PATH). Finished before the handler returns.+- **File I/O**: `get_backend` ensures the workspace root exists. The command+ itself may read/write under that root.+- **Subprocess**: one per call, via the backend's subprocess (synchronous,+ inherited PATH). Finished before the operation returns.## External Dependencies-- **Python packages**: `deepagents`.+- **Python packages**: `deepagents` (via `NushellBackend`), `core.ansi`.- **Environment variables**: `WORKSPACE_BASE_DIR`.-- **OS utilities**: whatever the command references - but PATH is empty, so- absolute paths only.+- **OS utilities**: `nu` (Nushell) on PATH for the primary path; whatever the+ command references is reachable because PATH is inherited.## Edge cases & known limits-- **Missing binaries masquerade as command failures**: because PATH is- scrubbed, a plain `npm test` gives "command not found" rather than a- permissions error. Users often confuse this for a broken node.+- **Nushell grammar, not bash**: `&&` / `||` are pre-flight-rejected; `$VAR`,+ backticks, and `>` redirection differ from bash. See `shell-skill/SKILL.md`.- **Truncation corrupts JSON output**: if a command prints JSON and thebackend truncates, the `stdout` string becomes invalid JSON with notrailing marker beyond `truncated=true`.@@ -124,10 +143,10 @@ flowchart TDSIGTERM-on-timeout.- **`timeout` is best-effort**: the backend uses `subprocess.run(..., timeout=)`.A child process that ignores SIGTERM may keep running until SIGKILL.-- **Stdin is empty**: the handler exposes no way to pipe data into the+- **Stdin is empty**: the node exposes no way to pipe data into thecommand - `input_data` from upstream nodes does not reach it.- **No environment variable override**: users cannot inject env vars via- parameters; the backend sets its own minimal env.+ parameters; the backend inherits the server's env (`inherit_env=True`).- **Windows path traversal check is string-based**: `virtual_mode=True`normalises paths but trust-boundary violations depend on the backendimplementation.diff --git a/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.md b/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.mdindex 511d83d9..79c6c5fb 100644--- a/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.md+++ b/docs-internal/node-logic-flows/code_fs_process/typescriptExecutor.md@@ -3,8 +3,8 @@| Field | Value ||------|-------|| **Category** | code_fs_process / code |-| **Backend handler** | [`server/services/handlers/code.py::handle_typescript_executor`](../../../server/services/handlers/code.py) |-| **Node.js client** | [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py) |+| **Backend handler** | [`server/nodes/code/typescript_executor/__init__.py::TypeScriptExecutorNode.execute_op`](../../../server/nodes/code/typescript_executor/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`; base in [`_base.py`](../../../server/nodes/code/_base.py)) |+| **Node.js client** | [`server/nodes/code/_nodejs.py::get_nodejs_client`](../../../server/nodes/code/_nodejs.py) (singleton over [`server/services/nodejs_client.py::NodeJSClient`](../../../server/services/nodejs_client.py)) || **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) || **Skill (if any)** | - (shared with `javascript-skill`) || **Dual-purpose tool** | yes - tool name `typescript_code` |@@ -18,9 +18,9 @@ All inputs, outputs, and error paths match the JS variant - the onlyuser-facing difference is that TypeScript type annotations parse withouterror.-Dispatched through the special-handlers branch at-[`node_executor.py:367-381`](../../../server/services/node_executor.py) so it-receives `connected_outputs`.+The plugin reads upstream outputs from `ctx.raw["connected_outputs"]` (injected+by the executor for code-executor node types) and the workflow's+`ctx.workspace_dir`, then POSTs with `language="typescript"`.## Inputs (handles)@@ -32,22 +32,22 @@ receives `connected_outputs`.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `code` | string (code editor) | (boilerplate stub with `interface Result { ... }`) | yes | - | TypeScript source. User must assign to `output` |-| `timeout` | number | `30` | no | - | Seconds - multiplied by 1000 before forwarding to the Node server |+| `code` | string (code editor) | (required, `min_length=1`) | yes | - | TypeScript source. User must assign to `output` |+| `timeout` | number | `30` (ge=1, le=600) | no | - | Seconds - multiplied by 1000 before forwarding to the Node server |-## Handler-level kwargs+`CodeExecutorParams` uses `extra="allow"` (extra params persist but are unread).-| Name | Default | Description |-|------|---------|-------------|-| `nodejs_url` | `http://localhost:3020` | |-| `nodejs_timeout` | `30` | |+## Node.js client singleton++Shares `get_nodejs_client()` in [`_nodejs.py`](../../../server/nodes/code/_nodejs.py)+with the JS plugin — one `NodeJSClient(base_url="http://localhost:3020",+timeout=30)` instance, hard-coded defaults.## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-main` | object | Standard envelope payload (node declares only `input-main` / `output-main`; `usable_as_tool=True` exposes the same payload as the `typescript_code` tool result) |### Output payload@@ -55,23 +55,25 @@ receives `connected_outputs`.{output: any;console_output: string;- timestamp: string;}```+`node_output_schemas.CodeExecutorOutput` declares only `output` (extra fields+like `console_output` allowed by `_OutputBase`).+## Logic Flow```mermaidflowchart TD- A[handle_typescript_executor] --> B{code strip empty?}- B -- yes --> E[Return error envelope:<br/>No code provided]- B -- no --> C[timeout_ms = int timeout * 1000]- C --> D[input_data = connected_outputs or {}<br/>inject workspace_dir]+ A[execute_op] --> B{code strip empty?}+ B -- yes --> E[raise NodeUserError:<br/>No code provided]+ B -- no --> C[timeout_ms = timeout * 1000]+ C --> D[input_data = ctx.raw connected_outputs or {}<br/>inject workspace_dir]D --> F[get_nodejs_client<br/>lazy singleton]F --> G[client.execute<br/>POST /execute<br/>language=typescript]- G -- exception --> H[Return error envelope]- G -- ok + success=true --> I[Return success envelope]- G -- ok + success=false --> J[Return error envelope<br/>preserve console_output]+ G -- ClientConnectorError --> H1[raise NodeUserError<br/>TS executor not running on :3020]+ G -- success=false --> H2[raise NodeUserError<br/>error=result.error]+ G -- success=true --> I[Return dict<br/>output, console_output]```## Decision Logic@@ -113,4 +115,4 @@ wire-level difference is the POST body's `language` field.## Related- **Shared skill**: [`javascript-skill/SKILL.md`](../../../server/skills/coding_agent/javascript-skill/SKILL.md)-- **Sibling nodes**: [`javascriptExecutor`](./javascriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md)+- **Sibling nodes**: [`javascriptExecutor`](./javascriptExecutor.md), [`pythonExecutor`](./pythonExecutor.md), [`montyExecutor`](./montyExecutor.md)diff --git a/docs-internal/node-logic-flows/document/documentParser.md b/docs-internal/node-logic-flows/document/documentParser.mdindex e7998946..36d70da4 100644--- a/docs-internal/node-logic-flows/document/documentParser.md+++ b/docs-internal/node-logic-flows/document/documentParser.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | document |-| **Backend handler** | [`server/services/handlers/document.py::handle_document_parser`](../../../server/services/handlers/document.py) |+| **Backend handler** | [`server/nodes/document/document_parser/__init__.py`](../../../server/nodes/document/document_parser/__init__.py) — `DocumentParserNode`; dispatch via `BaseNode.execute()` + `@Operation("parse")` || **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -21,16 +21,18 @@ glob pattern.| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| `input-main` | main | no | Provides `files[]` upstream (e.g. from `fileDownloader`) |+| `input-main` | main | no | Upstream data resolved into params (e.g. `input_dir` pointing at `fileDownloader`'s `output_dir`) |## Parameters| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `files` | array | `[]` | no | - | List of `{path: string}` or raw path strings |-| `inputDir` | string | `""` | no | - | Directory to glob for files |-| `filePattern` | string | `*.pdf` | no | - | Glob pattern applied inside `inputDir` || `parser` | options | `pypdf` | no | - | `pypdf` / `marker` / `unstructured` / `beautifulsoup` |+| `file_path` | string | `""` | no | - | Single file path (takes precedence over `input_dir`) |+| `input_dir` | string | `""` | no | - | Directory to glob for files |+| `file_pattern` | string | `*.pdf` | no | - | Glob pattern applied inside `input_dir` |++Params are snake_case (`Params = DocumentParserParams`, `extra="ignore"`). There is no `files` param — paths come from `file_path` and/or the `input_dir` glob.## Outputs (handles)@@ -60,9 +62,9 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_```mermaidflowchart TD- A[handle_document_parser] --> B[Normalize files param<br/>dict -> path, str -> path]- B --> C{inputDir exists?}- C -- yes --> C1[extend paths with inputDir.glob(filePattern)]+ A[DocumentParserNode.parse] --> B[Collect paths from file_path]+ B --> C{input_dir exists?}+ C -- yes --> C1[extend paths with input_dir.glob(file_pattern)]C -- no --> D{paths empty?}C1 --> DD -- yes --> Ret0[Return success=true<br/>documents=[], parsed_count=0]@@ -86,7 +88,7 @@ flowchart TD- **No paths**: returns success with empty `documents[]`, no error.- **Per-file failure**: any exception inside `_parse_file_sync` is captured into `failed` with `{file, error}`; other files continue.-- **Unknown parser**: raised by `_parse_file_sync`, recorded per-file in `failed` (NOT a top-level failure - every file fails identically).+- **Unknown parser**: `_parse_file_sync` raises `ValueError`, but `parser` is a `Literal` so Pydantic rejects unknown values before dispatch; if it ever reaches the loop it is recorded per-file in `failed` (every file fails identically).- **Lazy imports**: each parser imports its backend only when selected, so unused heavy deps (marker CUDA stack, unstructured) are not loaded.- **BeautifulSoup special handling**: strips `<script>` and `<style>` tags before extracting text.@@ -114,9 +116,9 @@ flowchart TD- `marker`: downloads models on first call and requires CUDA; will fail on CPU-only hosts.- `unstructured`: classifies file type from extension/content; no explicit type param.- BeautifulSoup branch is the only one that handles HTML via `read_text`, so passing a binary `.pdf` there returns garbled `get_text`.-- `files` and `inputDir` can be combined - directory glob is appended to the explicit list.-- `filePattern` is applied via `Path.glob` (non-recursive). Use `**/*.pdf` for recursive.-- The handler returns `success=true` even when every file failed - consumers must inspect `failed` and `parsed_count`.+- `file_path` and `input_dir` can be combined - directory glob is appended to the explicit single path.+- `file_pattern` is applied via `Path.glob` (non-recursive). Use `**/*.pdf` for recursive.+- Returns `success=true` even when every file failed - consumers must inspect `failed` and `parsed_count`.## Relateddiff --git a/docs-internal/node-logic-flows/document/embeddingGenerator.md b/docs-internal/node-logic-flows/document/embeddingGenerator.mdindex 9382daa7..4458d63b 100644--- a/docs-internal/node-logic-flows/document/embeddingGenerator.md+++ b/docs-internal/node-logic-flows/document/embeddingGenerator.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | document |-| **Backend handler** | [`server/services/handlers/document.py::handle_embedding_generator`](../../../server/services/handlers/document.py) |+| **Backend handler** | [`server/nodes/document/embedding_generator/__init__.py`](../../../server/nodes/document/embedding_generator/__init__.py) — `EmbeddingGeneratorNode`; dispatch via `BaseNode.execute()` + `@Operation("embed")` || **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -28,7 +28,10 @@ thread pool via `asyncio.to_thread` so the event loop is not blocked.| `chunks` | array | `[]` | yes | - | List of `{content, source, chunk_index}` dicts || `provider` | options | `huggingface` | no | - | `huggingface` / `openai` / `ollama` || `model` | string | `BAAI/bge-small-en-v1.5` | no | - | Model name for the chosen provider |-| `apiKey` | string | `""` | no (required for openai) | - | OpenAI API key |+| `batch_size` | number | `32` | no | - | Texts per embedding batch (1-256). Declared but not currently used by the embed op. |+| `api_key` | string (password) | `""` | no (required for openai) | `provider=openai` | OpenAI API key |++Params are snake_case (`Params = EmbeddingGeneratorParams`).## Outputs (handles)@@ -61,16 +64,16 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_```mermaidflowchart TD- A[handle_embedding_generator] --> B{chunks empty?}+ A[EmbeddingGeneratorNode.embed] --> B{chunks empty?}B -- yes --> Ret0[Return success=true<br/>zero-valued payload]B -- no --> C[Extract text list: dict.content or str(chunk)]C --> D{provider}- D -- huggingface --> Dh[Import langchain_huggingface<br/>raise ImportError with install hint if missing]+ D -- huggingface --> Dh[Import langchain_huggingface<br/>raise NodeUserError with install hint if missing]D -- openai --> Do[Import langchain_openai.OpenAIEmbeddings]D -- ollama --> Dl[Import langchain_ollama.OllamaEmbeddings]- D -- other --> Dx[raise ValueError Unknown provider]+ D -- other --> Dx[raise NodeUserError Unknown provider]Dh --> E[HuggingFaceEmbeddings(model_name=model)]- Do --> F[OpenAIEmbeddings(model=model, api_key=apiKey)]+ Do --> F[OpenAIEmbeddings(model=model, api_key=api_key)]Dl --> G[OllamaEmbeddings(model=model)]E --> H[asyncio.to_thread embedder.embed_documents(texts)]F --> H@@ -82,11 +85,10 @@ flowchart TD## Decision Logic- **Empty chunks**: short-circuits to the empty-payload success envelope.-- **Provider dispatch**: string match on `provider`, else `ValueError`.+- **Provider dispatch**: string match on `provider`; `provider` is a `Literal` so Pydantic rejects unknown values before dispatch (the `else: NodeUserError` is a dead path).- **Embedding call**: always runs in a worker thread via `asyncio.to_thread`.- **Dimensions**: inferred from first embedding; `0` if the call returned `[]`.-- **Unknown provider**: raised inside the try block -> caught by outer `except` -> `success=false`.-- **Missing huggingface package**: re-raised with an install hint (`pip install langchain-huggingface sentence-transformers`).+- **Missing huggingface package**: raises `NodeUserError` with an install hint (`pip install langchain-huggingface sentence-transformers`); `BaseNode.execute()` logs one WARN line (no traceback) and returns `{success: false, error_type: "NodeUserError"}`.## Side Effects@@ -101,14 +103,14 @@ flowchart TD## External Dependencies-- **Credentials**: `apiKey` param (OpenAI only). Not fetched from `auth_service` - must be passed as a node parameter.+- **Credentials**: `api_key` param (OpenAI only). Not fetched from `auth_service` - must be passed as a node parameter.- **Python packages**: `langchain-huggingface` + `sentence-transformers` (HF), `langchain-openai` (OpenAI), `langchain-ollama` (Ollama).- **Environment variables**: `OLLAMA_HOST` respected by `langchain_ollama`.## Edge cases & known limits-- OpenAI `apiKey` is read from the node parameter, NOT from `auth_service.get_api_key('openai')`. This is inconsistent with other OpenAI-using nodes (documented gotcha).-- No batching controls - everything is passed in one `embed_documents` call; provider-side batch limits apply.+- OpenAI `api_key` is read from the node parameter, NOT from `auth_service.get_api_key('openai')`. This is inconsistent with other OpenAI-using nodes (documented gotcha).+- `batch_size` is a declared param but the embed op passes everything in one `embed_documents` call; provider-side batch limits apply.- HF first-run model download can take minutes; the node will appear to hang.- Model name default (`BAAI/bge-small-en-v1.5`) is HF-specific; you must override `model` when switching provider.- No cost tracking (unlike LLM chat nodes).diff --git a/docs-internal/node-logic-flows/document/fileDownloader.md b/docs-internal/node-logic-flows/document/fileDownloader.mdindex fd7abe76..6a07282c 100644--- a/docs-internal/node-logic-flows/document/fileDownloader.md+++ b/docs-internal/node-logic-flows/document/fileDownloader.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | document |-| **Backend handler** | [`server/services/handlers/document.py::handle_file_downloader`](../../../server/services/handlers/document.py) |+| **Backend handler** | [`server/nodes/document/file_downloader/__init__.py`](../../../server/nodes/document/file_downloader/__init__.py) — `FileDownloaderNode`; dispatch via `BaseNode.execute()` + `@Operation("download")` || **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -26,11 +26,13 @@ downloaded files land in the per-workflow workspace and can be picked up by| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `items` | array | `[]` | yes | - | List of `{url: string, ...}` dicts or raw URL strings |-| `outputDir` | string | `""` | no | - | Falls back to `<workspace_dir>/downloads` or `downloads` |-| `maxWorkers` | number | `8` | no | - | Semaphore width, 1-32 |-| `skipExisting` | boolean | `true` | no | - | Skip when destination file already exists |-| `timeout` | number | `60` | no | - | Per-request timeout (seconds) |+| `urls` | array | `[]` | no | - | Declared Params field; wrapped into `items` as `[{url}]` if `items` not present |+| `output_dir` | string | `downloads` | no | - | Declared Params field |+| `max_workers` | number | `4` | no | - | Declared Params field (1-32) |+| `skip_existing` | boolean | `true` | no | - | Declared Params field |+| `timeout` | number | `60` | no | - | Declared Params field (1-600) |++**Quirk**: `Params = FileDownloaderParams` declares snake_case fields (above), but `extra="allow"` lets camelCase keys pass through, and the `download` op reads camelCase off `model_dump()`: `items`, `outputDir`, `maxWorkers` (default `8` here, not `4`), `skipExisting`, plus `urls`/`timeout`. So the operative keys consumed at runtime are `items`/`urls`, `outputDir`, `maxWorkers`, `skipExisting`, `timeout`.## Outputs (handles)@@ -62,7 +64,7 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_```mermaidflowchart TD- A[handle_file_downloader] --> B{items empty?}+ A[FileDownloaderNode.download] --> B{items empty?}B -- yes --> Ret0[Return success=true<br/>counts all zero]B -- no --> C[Resolve output_dir from outputDir or context.workspace_dir or ./downloads]C --> D[mkdir parents exist_ok]diff --git a/docs-internal/node-logic-flows/document/httpScraper.md b/docs-internal/node-logic-flows/document/httpScraper.mdindex 54291e2d..babe1077 100644--- a/docs-internal/node-logic-flows/document/httpScraper.md+++ b/docs-internal/node-logic-flows/document/httpScraper.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | document |-| **Backend handler** | [`server/services/handlers/document.py::handle_http_scraper`](../../../server/services/handlers/document.py) |+| **Backend handler** | [`server/nodes/document/http_scraper/__init__.py`](../../../server/nodes/document/http_scraper/__init__.py) — `HttpScraperNode`; dispatch via `BaseNode.execute()` + `@Operation("scrape")` || **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -28,15 +28,22 @@ elements become items. Typically the first stage of an ingestion pipeline:| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `url` | string | `""` | **yes** | - | URL with optional `{date}` / `{page}` placeholder |-| `iterationMode` | options | `single` | no | - | `single` / `date` / `page` |-| `startDate` | string | `""` | yes (date mode) | `iterationMode=date` | `YYYY-MM-DD` |-| `endDate` | string | `""` | yes (date mode) | `iterationMode=date` | `YYYY-MM-DD`, inclusive |-| `datePlaceholder` | string | `{date}` | no | `iterationMode=date` | Substring replaced with formatted date |-| `startPage` | number | `1` | no | `iterationMode=page` | Inclusive |-| `endPage` | number | `10` | no | `iterationMode=page` | Inclusive; `{page}` literal replaced |-| `linkSelector` | string | `a[href$=".pdf"]` | no | - | BeautifulSoup CSS selector |+| `iteration_mode` | options | `single` | no | - | `single` / `date` / `page` |+| `link_selector` | string | `a[href$=".pdf"]` | no | - | BeautifulSoup CSS selector || `headers` | string (JSON) | `{}` | no | - | Extra HTTP headers as JSON object |-| `useProxy` | boolean | `false` | no | - | Route through proxy service if enabled |+| `start_date` | string | `""` | yes (date mode) | `iteration_mode=date` | `YYYY-MM-DD` |+| `end_date` | string | `""` | yes (date mode) | `iteration_mode=date` | `YYYY-MM-DD`, inclusive |+| `date_placeholder` | string | `{date}` | no | `iteration_mode=date` | Substring replaced with formatted date |+| `start_page` | number | `1` | no | `iteration_mode=page` | Inclusive |+| `end_page` | number | `10` | no | `iteration_mode=page` | Inclusive; `{page}` literal replaced |+| `max_pages` | number | `10` | no | - | Safety cap on pages fetched (1-1000) |+| `use_proxy` | boolean | `false` | no | - | Route through proxy service if enabled |+| `proxy_provider` | string | `auto` | no | `use_proxy=true` | Provider name |+| `proxy_country` | string | `""` | no | `use_proxy=true` | ISO country code |+| `session_type` | options | `rotating` | no | `use_proxy=true` | `rotating` / `sticky` |+| `sticky_duration` | number | `600` | no | `use_proxy=true, session_type=sticky` | Sticky session seconds |++Params are snake_case (`Params = HttpScraperParams`, `extra="ignore"`).## Outputs (handles)@@ -66,12 +73,12 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_```mermaidflowchart TD- A[handle_http_scraper] --> B{url provided?}- B -- no --> Eret[Return success=false<br/>error: URL is required]- B -- yes --> C[Parse headers JSON]- C --> D{iterationMode}- D -- date --> D1[Iterate start_date..end_date days<br/>replace datePlaceholder per URL]- D -- page --> D2[Iterate startPage..endPage<br/>replace `{page}` literal per URL]+ A[HttpScraperNode.scrape] --> B{url provided?}+ B -- no --> Eret[raise NodeUserError URL is required<br/>-> success=false envelope]+ B -- yes --> C[json.loads headers]+ C --> D{iteration_mode}+ D -- date --> D1[Iterate start_date..end_date days<br/>replace date_placeholder per URL]+ D -- page --> D2[Iterate start_page..end_page<br/>replace `{page}` literal per URL]D -- single --> D3[Single URL, meta=`{}`]D1 --> E[Build urls_to_fetch list]D2 --> E@@ -89,10 +96,10 @@ flowchart TD## Decision Logic-- **Validation**: missing `url` raises `ValueError("URL is required")`; caught by outer `except` -> `success=false`.-- **Date mode**: missing `startDate` or `endDate` raises `ValueError` -> `success=false`.-- **Iteration mode fallback**: any value other than `date` or `page` takes the single-URL branch (no validation of unknown modes).-- **Per-URL errors**: collected into `errors` list; the handler still returns `success=true` even if every URL failed (only top-level setup errors fail the envelope).+- **Validation**: missing `url` raises `NodeUserError("URL is required")` (single WARN line, no traceback) -> `success=false` envelope.+- **Date mode**: missing `start_date` or `end_date` raises `NodeUserError` -> `success=false`.+- **Iteration mode**: `iteration_mode` is a `Literal`, so unknown values are rejected by Pydantic; `single` is the only non-date/page branch.+- **Per-URL errors**: collected into `errors` list; the op still returns `success=true` even if every URL failed (only top-level setup errors fail the envelope).- **Proxy failure**: logged at warning, request proceeds without proxy.## Side Effects@@ -112,12 +119,13 @@ flowchart TD## Edge cases & known limits-- `datePlaceholder` is a plain `str.replace`, not a token boundary - a URL containing the placeholder substring twice is replaced twice.+- `date_placeholder` is a plain `str.replace`, not a token boundary - a URL containing the placeholder substring twice is replaced twice.- Date mode formats dates as `YYYY-MM-DD`; placeholder in URL is replaced with this exact format regardless of what the user wrote.- `{page}` in page mode is hard-coded, not configurable (unlike the date placeholder).-- `headers` is parsed with `json.loads` with no try/except - invalid JSON raises at the top level and fails the envelope.+- `headers` is parsed with `json.loads` with no try/except - invalid JSON raises (caught as a bare `Exception` -> full traceback) and fails the envelope.+- `max_pages` is declared but not enforced in the page-iteration loop (the loop runs `start_page..end_page`).- Per-URL failures are silently aggregated into `errors` but the envelope is still `success=true`; downstream nodes must check `errors`.-- `endPage`/`endDate` are inclusive.+- `end_page`/`end_date` are inclusive.## Relateddiff --git a/docs-internal/node-logic-flows/document/textChunker.md b/docs-internal/node-logic-flows/document/textChunker.mdindex 0a39dcf9..3deb6a56 100644--- a/docs-internal/node-logic-flows/document/textChunker.md+++ b/docs-internal/node-logic-flows/document/textChunker.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | document |-| **Backend handler** | [`server/services/handlers/document.py::handle_text_chunker`](../../../server/services/handlers/document.py) |+| **Backend handler** | [`server/nodes/document/text_chunker/__init__.py`](../../../server/nodes/document/text_chunker/__init__.py) — `TextChunkerNode`; dispatch via `BaseNode.execute()` + `@Operation("chunk")` || **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -26,10 +26,12 @@ pipeline.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `documents` | array | `[]` | yes | - | List of `{content, source}` dicts (or raw strings) |-| `chunkSize` | number | `1024` | no | - | Max chars per chunk (100-8000) |-| `chunkOverlap` | number | `200` | no | - | Char overlap between adjacent chunks (0-1000) |-| `strategy` | options | `recursive` | no | - | `recursive` / `markdown` (token strategy is documented but falls through to recursive) |+| `text` | string | `""` | no | - | Declared Params field; wrapped into `documents` as `[{content: text}]` if `documents` absent |+| `strategy` | options | `recursive` | no | - | `recursive` / `markdown` / `token` (token falls through to recursive) |+| `chunk_size` | number | `1000` | no | - | Declared Params field (100-8000) |+| `overlap` | number | `200` | no | - | Declared Params field (0-1000) |++**Quirk**: `Params = TextChunkerParams` declares `text` / `strategy` / `chunk_size` / `overlap` (snake_case), but `extra="allow"` passes camelCase through and the `chunk` op reads `documents`, `chunkSize` (default `1000`), `chunkOverlap` or `overlap`, and `strategy` off `model_dump()`. So the operative inputs at runtime are `documents`/`text`, `chunkSize`, `chunkOverlap`/`overlap`, `strategy`.## Outputs (handles)@@ -57,7 +59,7 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_```mermaidflowchart TD- A[handle_text_chunker] --> B{documents empty?}+ A[TextChunkerNode.chunk] --> B{documents empty?}B -- yes --> Ret0[Return success=true<br/>chunks=[], chunk_count=0]B -- no --> C{strategy}C -- markdown --> Cm[MarkdownTextSplitter]diff --git a/docs-internal/node-logic-flows/document/vectorStore.md b/docs-internal/node-logic-flows/document/vectorStore.mdindex 59067e5e..89ff2fe2 100644--- a/docs-internal/node-logic-flows/document/vectorStore.md+++ b/docs-internal/node-logic-flows/document/vectorStore.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | document |-| **Backend handler** | [`server/services/handlers/document.py::handle_vector_store`](../../../server/services/handlers/document.py) |+| **Backend handler** | [`server/nodes/document/vector_store/__init__.py`](../../../server/nodes/document/vector_store/__init__.py) — `VectorStoreNode`; dispatch via `BaseNode.execute()` + `@Operation("dispatch")` || **Tests** | [`server/tests/nodes/test_document.py`](../../../server/tests/nodes/test_document.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -26,16 +26,18 @@ pipeline and/or the retrieval stage at query time.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `operation` | options | `store` | no | - | `store` / `query` / `delete` |-| `backend` | options | `chroma` | no | - | `chroma` / `qdrant` / `pinecone` |-| `collectionName` | string | `documents` | no | - | Collection / index name |-| `embeddings` | array | `[]` | yes (store) | `operation=store` | Vectors from `embeddingGenerator` |-| `chunks` | array | `[]` | no (store) | `operation=store` | Metadata pairs for the vectors |-| `queryEmbedding` | array | `[]` | yes (query) | `operation=query` | Single vector to query |-| `topK` | number | `5` | no | `operation=query` | Max matches to return |-| `ids` | array | `[]` | yes (delete) | `operation=delete` | Vector IDs to delete |-| `persistDir` | string | `./data/vectors` | no | `backend=chroma` | ChromaDB persistence dir |-| `qdrantUrl` | string | `http://localhost:6333` | no | `backend=qdrant` | Qdrant URL |-| `pineconeApiKey` | string | `""` | yes | `backend=pinecone` | Pinecone API key |+| `backend` | options | `chroma` | no | - | `chroma` / `chromadb` / `qdrant` / `pinecone` (`chromadb` aliased to `chroma`) |+| `collection_name` | string | `documents` | no | - | Collection / index name |+| `embeddings` | array | `null` | yes (store) | - | Vectors from `embeddingGenerator` |+| `chunks` | array | `null` | no (store) | `operation=store` | Metadata pairs for the vectors |+| `query_embedding` | array | `null` | yes (query) | `operation=query` | Single vector to query |+| `top_k` | number | `5` | no | `operation=query` | Max matches to return (1-100) |+| `ids` | array | `null` | yes (delete) | `operation=delete` | Vector IDs to delete |+| `persist_dir` | string | `./data/vectors` | no | `backend=chroma,chromadb` | ChromaDB persistence dir |+| `qdrant_url` | string | `http://localhost:6333` | no | `backend=qdrant` | Qdrant URL |+| `pinecone_api_key` | string | `""` | yes | `backend=pinecone` | Pinecone API key |++Params are snake_case (`Params = VectorStoreParams`, `extra="ignore"`).## Outputs (handles)@@ -76,16 +78,16 @@ Wrapped in standard envelope: `{ success, result, execution_time, node_id, node_```mermaidflowchart TD- A[handle_vector_store] --> B{backend}- B -- chroma --> C1[_chroma_op]+ A[VectorStoreNode.dispatch] --> B{backend}+ B -- chroma/chromadb --> C1[_chroma_op]B -- qdrant --> C2[_qdrant_op]B -- pinecone --> C3[_pinecone_op]- B -- other --> Ex[raise ValueError Unknown backend]+ B -- other --> Ex[raise NodeUserError Unknown backend]C1 --> D{operation}C2 --> DC3 --> DD -- store --> Ds[Auto-create collection if needed<br/>generate uuid ids + metas<br/>upsert via asyncio.to_thread]- D -- query --> Dq[query / search with queryEmbedding + topK<br/>normalize matches across backends]+ D -- query --> Dq[query / search with query_embedding + top_k<br/>normalize matches across backends]D -- delete --> Dd[delete by ids, count = len(ids)]Ds --> E[Return partial result]Dq --> E@@ -96,9 +98,10 @@ flowchart TD## Decision Logic-- **Backend dispatch**: `chroma` / `qdrant` / `pinecone` - anything else raises `ValueError` caught as `success=false`.+- **Backend dispatch**: `backend` is a `Literal` (`chroma` / `chromadb` / `qdrant` / `pinecone`); `chromadb` is normalized to `chroma`. Unknown values are rejected by Pydantic before dispatch (the `else: NodeUserError` is a dead path).+- **Pinecone without key**: `_pinecone_op` raises `NodeUserError("Pinecone API key required")` when `pinecone_api_key` is empty.- **Empty embeddings (store)**: returns `stored_count=0` without raising; for Chroma also returns current `collection_count`.-- **Empty queryEmbedding (query)**: returns `matches=[]` without hitting the backend.+- **Empty query_embedding (query)**: returns `matches=[]` without hitting the backend.- **Empty ids (delete)**: returns `deleted=true, count=0` without hitting the backend.- **Collection auto-creation**:- Chroma: `get_or_create_collection` always.@@ -120,7 +123,7 @@ flowchart TD## External Dependencies-- **Credentials**: `pineconeApiKey` node parameter (Pinecone only). Not fetched from `auth_service`.+- **Credentials**: `pinecone_api_key` node parameter (Pinecone only). Not fetched from `auth_service`.- **Python packages**: `chromadb>=0.5.0`, `qdrant-client>=1.12.0`, `pinecone` (all lazy-imported per backend).- **Environment variables**: none (Qdrant URL is a param, Pinecone key is a param).@@ -130,7 +133,7 @@ flowchart TD- Chroma is the only backend that returns `collection_count` after a store.- Qdrant collection creation infers the vector size from the first embedding; mixed-size batches will be rejected by Qdrant on upsert.- Pinecone `pc.Index(collection)` does not create the index - it must exist, or the op fails with Pinecone's own error.-- Pinecone `apiKey` is a node parameter, inconsistent with other cloud services that pull from `auth_service`.+- Pinecone `pinecone_api_key` is a node parameter, inconsistent with other cloud services that pull from `auth_service`.- Delete is by ID only - no metadata-filter delete.- Query match shape differs per backend: Chroma returns `distance`, Qdrant and Pinecone return `score`. Downstream consumers must handle both.diff --git a/docs-internal/node-logic-flows/email/emailRead.md b/docs-internal/node-logic-flows/email/emailRead.mdindex 408e0cb8..e618a738 100644--- a/docs-internal/node-logic-flows/email/emailRead.md+++ b/docs-internal/node-logic-flows/email/emailRead.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | email / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/email.py::handle_email_read`](../../../server/services/handlers/email.py) |+| **Backend handler** | [`server/nodes/email/email_read/__init__.py`](../../../server/nodes/email/email_read/__init__.py) — dispatched via `BaseNode.execute()` -> `@Operation("query")` -> `EmailService.read` ([`_service.py`](../../../server/nodes/email/_service.py)) || **Tests** | [`server/tests/nodes/test_email.py`](../../../server/tests/nodes/test_email.py) || **Skill (if any)** | none shipped || **Dual-purpose tool** | yes - connect to `input-tools` as the node's own name |@@ -25,15 +25,22 @@ node exposes seven operations selected by the `operation` parameter.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `provider` | options | `gmail` | no | - | Preset key in `email_providers.json` |-| `operation` | options | `list` | **yes** | - | `list` / `search` / `read` / `folders` / `move` / `delete` / `flag` |-| `folder` | string | `INBOX` | no | show for all except `folders` | IMAP folder name |-| `query` | string | `""` | **yes** (`search`) | `operation: ['search']` | Himalaya search expression, e.g. `from:x subject:y` |-| `message_id` | string | `""` | **yes** for `read`/`move`/`delete`/`flag` | `operation: ['read','move','delete','flag']` | IMAP message UID |-| `target_folder` | string | `""` | **yes** (`move`) | `operation: ['move']` | Destination folder for `move` |-| `flag` | options | `Seen` | no | `operation: ['flag']` | One of `Seen` / `Answered` / `Flagged` / `Draft` / `Deleted` |+| `operation` | options | `list` | no | - | `list` / `search` / `read` / `folders` / `move` / `delete` / `flag` |+| `folder` | string | `INBOX` | no | `operation: ['list','search']` | IMAP folder name |+| `query` | string | `""` | no (`search`) | `operation: ['search']` | Himalaya search expression, e.g. `from:x subject:y` |+| `message_id` | string | `""` | no (used by `read`/`move`/`delete`/`flag`) | `operation: ['read','move','delete','flag']` | IMAP message UID |+| `target_folder` | string | `""` | no (`move`) | `operation: ['move']` | Destination folder for `move` |+| `flag` | options | `""` | no | `operation: ['flag']` | One of `Seen` / `Answered` / `Flagged` / `Draft` / `Deleted` (empty default) || `flag_action` | options | `add` | no | `operation: ['flag']` | `add` or `remove` |-| `page` | number | `1` | no | `operation: ['list']` | 1-based page number |-| `page_size` | number | `20` | no | `operation: ['list']` | 1-100 envelopes per page |+| `limit` | number | `20` | no | `operation: ['list','search']` | Max envelopes per page (1-500) |+| `page` | number | `1` | no | `operation: ['list','search']` | 1-based page number |+| `page_size` | number | `20` | no | `operation: ['list','search']` | Items per page (1-500); overrides `limit` when paginating |+| `offset` | number | `0` | no | `operation: ['list','search']` | Skip this many messages (alternative to page-based pagination) |++> Note: no parameter is Pydantic-required (`operation` defaults to `list`); the+> operation router in `EmailService.read` passes empty strings straight to+> Himalaya, which fails with a non-zero exit for ops that genuinely need+> `message_id` / `query` / `target_folder`.Credential params (`email`, `password`, `imap_host`, etc.) follow the sameresolution rules as [`emailSend`](./emailSend.md#decision-logic).@@ -43,7 +50,8 @@ resolution rules as [`emailSend`](./emailSend.md#decision-logic).| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Operation result |-| `output-tool` | object | Same payload, used when wired as an AI tool |++When wired to an AI agent's `input-tools` handle (`usable_as_tool = True`, tool name `email_read`), the same payload is returned to the LLM via the tool-dispatch path — there is no separate `output-tool` handle.### Output payload@@ -66,7 +74,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution```mermaidflowchart TD- A[handle_email_read] --> B[EmailService.read params]+ A[BaseNode.execute -> Operation query] --> B[EmailService.read params]B --> C[resolve_credentials]C -->|missing email/password| Ec[ValueError -> error envelope]C --> D{operation in router?}@@ -127,9 +135,10 @@ flowchart TD## Edge cases & known limits-- **Page-size clamp is frontend-only**: the node def caps `page_size` at 100,- but the backend passes whatever value arrives. Requesting 10000 will hit- the IMAP server's own limits.+- **Pagination uses `page`/`page_size` only**: the router in `EmailService.read`+ reads `page` + `page_size`; the `limit` and `offset` params exist on the model+ but are not forwarded to Himalaya for `list`/`search`. `page_size` is Pydantic-+ validated to `1-500`; the IMAP server's own limits still apply.- **No pagination metadata**: `list` returns whatever Himalaya gives; thereis no `has_more` / `total` helper.- **Deletion is permanent** (no undo), and the handler does not prompt ordiff --git a/docs-internal/node-logic-flows/email/emailReceive.md b/docs-internal/node-logic-flows/email/emailReceive.mdindex 8cf00aaa..95b98ba8 100644--- a/docs-internal/node-logic-flows/email/emailReceive.md+++ b/docs-internal/node-logic-flows/email/emailReceive.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | email / trigger |-| **Backend handler** | [`server/services/handlers/email.py::handle_email_receive`](../../../server/services/handlers/email.py) |+| **Backend handler** | [`server/nodes/email/email_receive/__init__.py`](../../../server/nodes/email/email_receive/__init__.py) — Run-button path is the `execute()` override; deployment-mode polling uses the `PollingTriggerNode` hooks (`setup_service` / `fetch_ids` / `fetch_detail` / `post_emit`) draining via the canary `TriggerListenerWorkflow` path || **Tests** | [`server/tests/nodes/test_email.py`](../../../server/tests/nodes/test_email.py) || **Skill (if any)** | none shipped || **Dual-purpose tool** | no (trigger only) |@@ -24,9 +24,9 @@ None. Triggers have no inputs.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `provider` | options | `googleGmail` | no | - | Preset key in `email_providers.json` |+| `provider` | options | `gmail` | no | - | Preset key in `email_providers.json` || `folder` | string | `INBOX` | no | - | IMAP folder to watch |-| `poll_interval` | number | `60` | no | - | Seconds between polls. Clamped to `[min_interval, max_interval]` from config (30-3600) |+| `poll_interval` | number | `60` | no | - | Seconds between polls. Pydantic-validated `30-3600` (`poll_interval_clamp = (30, 3600)`) and re-clamped against config `min_interval`/`max_interval` || `filter_query` | string | `""` | no | - | Reserved, currently not applied during polling || `mark_as_read` | boolean | `false` | no | - | When true, adds the `Seen` flag to the new message after fetch |@@ -57,7 +57,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution```mermaidflowchart TD- A[handle_email_receive] --> B[resolve_credentials]+ A[EmailReceiveNode.execute] --> B[resolve_credentials]B -->|ValueError| Ec[error envelope]B --> C[resolve_poll_params]C --> D[broadcaster.update_node_status waiting]@@ -70,7 +70,7 @@ flowchart TDI --> J{mark_as_read?}J -- yes --> K[himalaya flag add Seen id]J -- no --> L[skip]- K --> M[event_waiter.dispatch email_received]+ K --> M[dispatch_email_received -> dispatch.emit<br/>CloudEvents WorkflowEvent]L --> MM --> N[Return success envelope]```@@ -83,9 +83,10 @@ flowchart TD- **Interval clamping** in `EmailService.resolve_poll_params`:`max(min_interval, min(max_interval, requested))`. Defaults come from`email_providers.json -> polling` block.-- **One event per handler run**: as soon as any new IDs are seen, the handler- fetches the *first* of them, dispatches a single `email_received` event,- and returns. Remaining new IDs are added to `seen` but not processed;+- **One event per Run-button `execute()` run**: as soon as any new IDs are seen,+ it fetches the *first* of them, dispatches a single `email_received` event via+ `dispatch_email_received`, and returns. Remaining new IDs are added to `seen`+ but not processed;they will be skipped on the next run because they are now consideredbaseline.- **`filter_query` is unused in polling**: the parameter appears in the UI@@ -99,13 +100,20 @@ flowchart TD## Side Effects-- **Database writes**: none directly. (The `event_waiter.dispatch` side is- in-memory only - no DB row is written for the event.)+- **Database writes**: none directly.- **Broadcasts**: one `update_node_status(..., "waiting", ...)` when pollingstarts, with `workflow_id` from context.-- **Event bus**: `event_waiter.dispatch("email_received", email_data)` fires- once per handler run. Downstream trigger consumers (e.g. another- `emailReceive` listener) receive this payload.+- **Event bus (canary path)**: `dispatch_email_received(email_data)` ->+ `services.events.dispatch.emit` with a CloudEvents `WorkflowEvent`+ (`type = "com.machinaos.email.message.received"`, `subject = message_id`,+ outer wire-routing key `email_received`). `emailReceive` is canary-registered+ via `register_canary_trigger_type("emailReceive", "com.machinaos.email.message.received")`+ in [`nodes/email/__init__.py`](../../../server/nodes/email/__init__.py), so the+ deployment manager skips the legacy `setup_event_trigger` and a Temporal-durable+ `TriggerListenerWorkflow` consumes the event; the same call also broadcasts the+ envelope to the FE on the `email_received` wire key. The legacy+ `event_waiter.dispatch` path has zero consumers in canary mode and is no longer+ called (`_events.py`).- **External API calls**: none direct - IMAP traffic flows through Himalaya.- **File I/O**: one `himalaya_*.toml` tempfile per subprocess call (baseline,each poll, fetch, flag). Each is deleted in `finally`.@@ -117,7 +125,8 @@ flowchart TD- **Binary**: `himalaya` on `PATH`.- **Credentials**: same as [`emailSend`](./emailSend.md#external-dependencies).- **Services**: `StatusBroadcaster` (for the waiting status),- `event_waiter` (for dispatch).+ `services.events.dispatch` (canary CloudEvents emit via `_events.py`);+ `event_waiter.register_filter_builder` only for the `emailReceive` filter.- **Config**: `server/config/email_providers.json -> polling`.## Edge cases & known limitsdiff --git a/docs-internal/node-logic-flows/email/emailSend.md b/docs-internal/node-logic-flows/email/emailSend.mdindex debb2d51..6a1c5c8a 100644--- a/docs-internal/node-logic-flows/email/emailSend.md+++ b/docs-internal/node-logic-flows/email/emailSend.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | email / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/email.py::handle_email_send`](../../../server/services/handlers/email.py) |+| **Backend handler** | [`server/nodes/email/email_send/__init__.py`](../../../server/nodes/email/email_send/__init__.py) — dispatched via `BaseNode.execute()` -> `@Operation("send")` -> `EmailService.send` ([`_service.py`](../../../server/nodes/email/_service.py)) || **Tests** | [`server/tests/nodes/test_email.py`](../../../server/tests/nodes/test_email.py) || **Skill (if any)** | none shipped || **Dual-purpose tool** | yes - connect to `input-tools` as the node's own name |@@ -29,7 +29,7 @@ presets (see `server/config/email_providers.json`). When wired to an AI agent's| `provider` | options | `gmail` | no | - | One of `gmail` / `outlook` / `yahoo` / `icloud` / `protonmail` / `fastmail` / `custom`. Picks the preset in `email_providers.json` || `to` | string | `""` | **yes** | - | Comma-separated recipient list || `subject` | string | `""` | **yes** | - | Email subject line |-| `body` | string | `""` | **yes** | - | Email body (plain text or HTML, see `body_type`) |+| `body` | string | `""` | no | - | Email body (plain text or HTML, see `body_type`); `rows: 6` textarea hint || `cc` | string | `""` | no | - | CC recipients, only added if truthy || `bcc` | string | `""` | no | - | BCC recipients, only added if truthy || `body_type` | options | `text` | no | - | `text` or `html`. HTML sends a `multipart/alternative` MIME message |@@ -43,7 +43,8 @@ Also resolved at runtime (not node-exposed):| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Himalaya send result merged with `from` (sender address) |-| `output-tool` | object | Same payload, used when wired as an AI tool |++When wired to an AI agent's `input-tools` handle (`usable_as_tool = True`, tool name `email_send`), the same payload is returned to the LLM via the tool-dispatch path — there is no separate `output-tool` handle.### Output payload@@ -61,7 +62,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution```mermaidflowchart TD- A[handle_email_send] --> B[EmailService.send params]+ A[BaseNode.execute -> Operation send] --> B[EmailService.send params]B --> C[resolve_credentials]C -->|no email/password| Ee[ValueError -> error envelope]C --> D[Merge with provider preset from email_providers.json]@@ -88,8 +89,9 @@ flowchart TD`provider == 'custom'`.- **Port coercion**: stored ports are strings in the API-key store; `_coerce_port`casts them to `int` and silently returns `None` on invalid values.-- **Missing email/password**: `resolve_credentials` raises `ValueError` before- any subprocess is spawned -> `handle_email_send` catches and returns+- **Missing email/password**: `resolve_credentials` raises `ValueError`+ (`"Email address not configured"` / `"Email password not configured"`) before+ any subprocess is spawned -> `BaseNode.execute()` catches and returns`success=false`.- **HTML vs plain**: `body_type == 'html'` -> `multipart/alternative` with anHTML part (no plain-text fallback part is attached). Anything else -> a@@ -101,8 +103,8 @@ flowchart TD`HimalayaService._account_name` (`[email protected]` -> `john_doe`). Dots and`+` are replaced with `_`.- **Subprocess errors**: non-zero returncode raises `RuntimeError("himalaya- error: <stderr>")`; the handler's `except Exception` catches and emits an- error envelope.+ error: <stderr>")`; `BaseNode.execute()`'s `except Exception` catches and+ emits the error envelope (with full traceback logging).## Side Effectsdiff --git a/docs-internal/node-logic-flows/google_workspace/googleCalendar.md b/docs-internal/node-logic-flows/google_workspace/googleCalendar.mdindex 6e82bfb8..633bcf07 100644--- a/docs-internal/node-logic-flows/google_workspace/googleCalendar.md+++ b/docs-internal/node-logic-flows/google_workspace/googleCalendar.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | google_workspace / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/calendar.py::handle_google_calendar`](../../../server/services/handlers/calendar.py) |+| **Backend handler** | [`server/nodes/google/calendar/__init__.py`](../../../server/nodes/google/calendar/__init__.py) (`CalendarNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) || **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) || **Skill (if any)** | [`server/skills/productivity_agent/google-calendar-skill/SKILL.md`](../../../server/skills/productivity_agent/google-calendar-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `googleCalendar` |+| **Dual-purpose tool** | yes - tool name `google_calendar` |## Purpose@@ -57,8 +57,11 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `update`, `delete`).| `title` / `start_time` / `end_time` / `description` / `location` | string | `""` | no | Patch fields || `calendar_id` | string | `primary` | no | - |-Also accepts `update_title`, `update_start_time`, etc. which the dispatcher-aliases onto the standard names.+Also accepts the optional `update_title`, `update_start_time`,+`update_end_time`, `update_description`, `update_location` fields. The dispatch+method does NOT mutate the params dict — it reads `params.update_title or+params.title` (etc.) as fallback chains so either the `update_*` field or the+plain field works.### `operation = delete`@@ -70,10 +73,13 @@ aliases onto the standard names.## Outputs (handles)+The node declares only `input-main` and `output-main`. Tool mode+(`usable_as_tool = True`, tool name `google_calendar`) returns the same+`output-main` payload — there is no separate `output-tool` handle.+| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Operation-specific payload |-| `output-tool` | object | Same, for AI tool wiring |+| `output-main` | object | Operation-specific `CalendarOutput` payload |- `create` / `update`: `{event_id, title, start, end, html_link, status, created?/updated?}`- `list`: `{events: [{event_id, title, start, end, description, location, status, html_link, attendees}], count, time_range: {start, end}}`@@ -83,58 +89,48 @@ aliases onto the standard names.```mermaidflowchart TD- A[handle_google_calendar] --> AL{operation == update?}- AL -- yes --> AM[Alias update_* params<br/>to standard names]- AL -- no --> B- AM --> B{operation?}- B -- create --> C1[handle_calendar_create]- B -- list --> C2[handle_calendar_list]- B -- update --> C3[handle_calendar_update]- B -- delete --> C4[handle_calendar_delete]- B -- unknown --> Eret[success=false<br/>Unknown Calendar operation]- C1 --> G[get_google_credentials + build v3]- C2 --> G- C3 --> G- C4 --> G- G -- ValueError --> Eret- C1 --> V1{title & start & end present?}- V1 -- no --> Eret- V1 -- yes --> R1[events.insert sendUpdates=all]- C2 --> DP[Parse date shortcuts:<br/>today / today+Nd]+ A[BaseNode.execute -> CalendarNode.dispatch] --> G[build_google_service calendar v3]+ G --> B{operation?}+ B -- create --> V1{title & start & end present?}+ V1 -- no --> Eret[raise RuntimeError]+ V1 -- yes --> R1[events.insert sendUpdates]+ B -- list --> DP[Parse date shortcuts:<br/>today / today+Nd]DP --> R2[events.list timeMin/timeMax]- C3 --> V3{event_id?}+ B -- update --> V3{event_id?}V3 -- no --> Eret- V3 -- yes --> G3[events.get -> merge updates -> events.update]- C4 --> V4{event_id?}+ V3 -- yes --> G3[events.get -> merge update_* fallbacks -> events.update]+ B -- delete --> V4{event_id?}V4 -- no --> EretV4 -- yes --> R4[events.delete sendUpdates]- R1 --> T[_track_calendar_usage]+ B -- unknown --> Eret+ R1 --> T[track_google_usage google_calendar]R2 --> TG3 --> TR4 --> T- T --> OUT[Return success envelope]+ T --> OUT[Return CalendarOutput; BaseNode serializes envelope]```## Decision Logic+- **Operation branch**: one `@Operation("dispatch")` method branches on `params.operation` (`create` / `list` / `update` / `delete`); unknown raises `RuntimeError`.- **Date shortcut parsing** (list): `start_date` of `today` or empty -> today 00:00 UTC; `end_date` starting with `today+` is `today + Nd`; otherwise the raw string is used and `Z` is appended if missing.- **Timezone fallback** (update): when updating start/end, the existing event's `timeZone` is reused; if absent falls back to `UTC`.-- **Update parameter aliasing**: the dispatcher copies `update_title`, `update_start_time`, `update_end_time`, `update_description`, `update_location` onto their plain names before delegating - this is irreversible mutation of the incoming `parameters` dict.+- **Update fallback chains**: `params.update_title or params.title` (etc.) — no mutation of the params dict; either field works.- **Description / location**: treated as `is not None` rather than truthy - empty string is a valid clear.- **Attendees**: empty strings after `split(',')` are filtered; the whole `attendees` field is omitted if the list is empty.-- **`order_by`**: only sent when `single_events=True`; otherwise `None`, which the Google client library strips.+- **`order_by`**: only sent when `single_events=True`.## Side Effects-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_calendar'`.-- **Broadcasts**: none.-- **External API calls**: Calendar API v3 - `events().insert/list/update/delete/get`. `sendUpdates='all'` on create/update sends email invitations to attendees.+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_calendar'`.+- **Broadcasts**: none from the operation; executor emits standard `node_status`.+- **External API calls**: Calendar API v3 - `events().insert/list/update/delete/get`. `sendUpdates` on create/update/delete drives email invitations/cancellations.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.- **Services**: Google Calendar API, `PricingService`, `Database`.- **Python packages**: `google-api-python-client`.- **Environment variables**: none.@@ -144,7 +140,6 @@ flowchart TD- `max_results` clamped to 250 silently.- `update` performs a read-modify-write cycle: two API calls per update. A concurrent modification between the `get` and `update` is overwritten without ETag checking.- `delete` with an invalid event_id will surface the `HttpError 404` message into the envelope as a string.-- The dispatcher mutates the incoming `parameters` dict for update aliasing; callers that reuse the dict will see the aliased fields.- `description=""` is treated as a real clear; `description=None` or missing leaves the field untouched. Subtle but load-bearing for update callers.## Relateddiff --git a/docs-internal/node-logic-flows/google_workspace/googleContacts.md b/docs-internal/node-logic-flows/google_workspace/googleContacts.mdindex f845edc8..92a2841f 100644--- a/docs-internal/node-logic-flows/google_workspace/googleContacts.md+++ b/docs-internal/node-logic-flows/google_workspace/googleContacts.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | google_workspace / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/contacts.py::handle_google_contacts`](../../../server/services/handlers/contacts.py) |+| **Backend handler** | [`server/nodes/google/contacts/__init__.py`](../../../server/nodes/google/contacts/__init__.py) (`ContactsNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) || **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) || **Skill (if any)** | [`server/skills/productivity_agent/google-contacts-skill/SKILL.md`](../../../server/skills/productivity_agent/google-contacts-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `googleContacts` |+| **Dual-purpose tool** | yes - tool name `google_contacts` |## Purpose@@ -64,8 +64,10 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `search`, `get`, `up| `resource_name` | string | `""` | **yes** | - || `first_name` / `last_name` / `email` / `phone` / `company` / `job_title` | string | `""` | no | Any provided field triggers its person field group to be replaced. **At least one patch field required.** |-Also accepts `update_first_name`, `update_last_name`, `update_email`,-`update_phone`, `update_company`, `update_job_title` aliases.+Also accepts the optional `update_first_name`, `update_last_name`,+`update_email`, `update_phone`, `update_company`, `update_job_title` fields. The+dispatch method reads `params.update_first_name or params.first_name` (etc.) as+fallback chains — it does NOT mutate the params dict.### `operation = delete`@@ -75,10 +77,13 @@ Also accepts `update_first_name`, `update_last_name`, `update_email`,## Outputs (handles)+The node declares only `input-main` and `output-main`. Tool mode+(`usable_as_tool = True`, tool name `google_contacts`) returns the same+`output-main` payload — there is no separate `output-tool` handle.+| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Formatted contact(s) payload |-| `output-tool` | object | Same, for AI tool wiring |+| `output-main` | object | Formatted contact(s) `ContactsOutput` payload |### Formatted contact shape (`_format_contact`)@@ -106,72 +111,57 @@ Also accepts `update_first_name`, `update_last_name`, `update_email`,```mermaidflowchart TD- A[handle_google_contacts] --> AL{operation == update?}- AL -- yes --> AM[Alias update_* params<br/>to plain names]- AL -- no --> B- AM --> B{operation?}- B -- create --> C1[handle_contacts_create]- B -- list --> C2[handle_contacts_list]- B -- search --> C3[handle_contacts_search]- B -- get --> C4[handle_contacts_get]- B -- update --> C5[handle_contacts_update]- B -- delete --> C6[handle_contacts_delete]- B -- unknown --> Eret[success=false<br/>Unknown Contacts operation]- C1 --> G[get_google_credentials + build people v1]- C2 --> G- C3 --> G- C4 --> G- C5 --> G- C6 --> G- G -- ValueError --> Eret- C1 --> V1{first_name?}- V1 -- no --> Eret+ A[BaseNode.execute -> ContactsNode.dispatch] --> G[build_google_service people v1]+ G --> B{operation?}+ B -- create --> V1{first_name?}+ V1 -- no --> Eret[raise RuntimeError]V1 -- yes --> BB[Build names/emails/phones<br/>organizations/biographies]BB --> R1[people.createContact]- C2 --> R2[people.connections.list<br/>personFields names emails phones<br/>organizations photos biographies]- C3 --> V3{query?}+ B -- list --> R2[people.connections.list<br/>personFields names emails phones<br/>organizations photos biographies]+ B -- search --> V3{query?}V3 -- no --> EretV3 -- yes --> R3[people.searchContacts]- C4 --> V4{resource_name?}+ B -- get --> V4{resource_name?}V4 -- no --> EretV4 -- yes --> R4[people.get]- C5 --> V5{resource_name?}+ B -- update --> V5{resource_name?}V5 -- no --> Eret- V5 -- yes --> G5[people.get etag -> build update body -> check updatePersonFields non-empty]- G5 -- no fields --> Eret2[raise<br/>At least one field to update required]+ V5 -- yes --> G5[people.get etag -> build update body via update_* fallbacks -> check fields]+ G5 -- no fields --> Eret2[raise RuntimeError<br/>At least one field to update required]G5 -- ok --> R5[people.updateContact updatePersonFields=csv]- C6 --> V6{resource_name?}+ B -- delete --> V6{resource_name?}V6 -- no --> EretV6 -- yes --> R6[people.deleteContact]- R1 --> T[_track_contacts_usage]+ B -- unknown --> Eret+ R1 --> T[track_google_usage google_contacts]R2 --> TR3 --> TR4 --> TR5 --> TR6 --> T- T --> OUT[Return success envelope]+ T --> OUT[Return ContactsOutput; BaseNode serializes envelope]```## Decision Logic-- **Dispatcher aliasing**: `update_*` params are copied over their plain counterparts (mutating input) before delegation.-- **Update requires etag**: `update` does a `people.get` first to fetch the current etag, then submits `updateContact` with `updatePersonFields` set to the comma-joined list of person field groups touched. If no field group was modified, raises `ValueError("At least one field to update is required")`.-- **Search requires query**: empty query -> `ValueError` captured by outer except.+- **Update fallback chains**: `params.update_first_name or params.first_name` (etc.) — no mutation of the params dict.+- **Update requires etag**: `update` does a `people.get` first to fetch the current etag, then submits `updateContact` with `updatePersonFields` set to the comma-joined list of person field groups touched. If no field group was modified, raises `RuntimeError("At least one field to update is required")`.+- **Search requires query**: empty query -> `RuntimeError` surfaced by `BaseNode.execute`.- **Notes field on create**: non-empty `notes` writes a `biographies[0]` entry; empty notes skip the field. Cannot be cleared via update (not wired).- **Format fallbacks**: `_format_contact` returns `""` for missing headers rather than `None`, so callers should not distinguish absence from empty.- **`list` resource_name** is hard-coded to `people/me` - this node cannot list another user's contacts.## Side Effects-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_contacts'`. `list` DOES call `_track_contacts_usage`.-- **Broadcasts**: none.+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_contacts'`. `list` DOES call `track_google_usage`.+- **Broadcasts**: none from the operation; executor emits standard `node_status`.- **External API calls**: People API v1 - `people().createContact/get/updateContact/deleteContact/searchContacts`, `people().connections().list`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.- **Services**: Google People API, `PricingService`, `Database`.- **Python packages**: `google-api-python-client`.- **Environment variables**: none.@@ -182,7 +172,6 @@ flowchart TD- Update replaces entire field groups: passing `email` wipes any additional emails on the contact. The handler does not merge.- `search` returns at most `page_size` results; there is no pagination wrapper.- `delete` does not prompt or verify - the deletion is immediate and final on Google's side.-- An unused local statement `person.get('addresses', [])` exists in `_format_contact` without assignment - noted as dead code, not a bug.- `update` requires at least one patch field; however the handler only treats first_name/last_name/email/phone/company/job_title. Notes and other fields cannot be updated here.## Relateddiff --git a/docs-internal/node-logic-flows/google_workspace/googleDrive.md b/docs-internal/node-logic-flows/google_workspace/googleDrive.mdindex a2a4300d..cc8e9e75 100644--- a/docs-internal/node-logic-flows/google_workspace/googleDrive.md+++ b/docs-internal/node-logic-flows/google_workspace/googleDrive.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | google_workspace / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/drive.py::handle_google_drive`](../../../server/services/handlers/drive.py) |+| **Backend handler** | [`server/nodes/google/drive/__init__.py`](../../../server/nodes/google/drive/__init__.py) (`DriveNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) || **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) || **Skill (if any)** | [`server/skills/productivity_agent/google-drive-skill/SKILL.md`](../../../server/skills/productivity_agent/google-drive-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `googleDrive` |+| **Dual-purpose tool** | yes - tool name `google_drive` |## Purpose@@ -65,10 +65,13 @@ Top-level dispatcher: `operation` (one of `upload`, `download`, `list`, `share`)## Outputs (handles)+The node declares only `input-main` and `output-main`. Tool mode+(`usable_as_tool = True`, tool name `google_drive`) returns the same+`output-main` payload — there is no separate `output-tool` handle.+| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Operation-specific payload |-| `output-tool` | object | Same, for AI tool wiring |+| `output-main` | object | Operation-specific `DriveOutput` payload |- `upload`: `{file_id, name, mime_type, size, web_link, download_link, created_time}`- `download` (base64): `{file_id, name, mime_type, size, content_base64}`@@ -80,17 +83,13 @@ Top-level dispatcher: `operation` (one of `upload`, `download`, `list`, `share`)```mermaidflowchart TD- A[handle_google_drive] --> B{operation?}- B -- upload --> U[handle_drive_upload]- B -- download --> D[handle_drive_download]- B -- list --> L[handle_drive_list]- B -- share --> S[handle_drive_share]- B -- unknown --> Eret[success=false<br/>Unknown Drive operation]- U --> G[get_google_credentials + build v3]- D --> G- L --> G- S --> G- G -- ValueError --> Eret+ A[BaseNode.execute -> DriveNode.dispatch] --> G[build_google_service drive v3]+ G --> B{operation?}+ B -- upload --> U[upload branch]+ B -- download --> D[download branch]+ B -- list --> L[list branch]+ B -- share --> S[share branch]+ B -- unknown --> Eret[raise RuntimeError]U --> V1{filename &<br/>file_url OR file_content?}V1 -- no --> EretV1 -- yes --> SRC{file_url set?}@@ -110,12 +109,12 @@ flowchart TDS --> V3{file_id & email?}V3 -- no --> EretV3 -- yes --> R4[permissions.create +<br/>files.get for name/link]- R1 --> T[_track_drive_usage]+ R1 --> T[track_google_usage google_drive]DL --> TOUT --> TR3 --> TR4 --> T- T --> OE[Return success envelope]+ T --> OE[Return DriveOutput; BaseNode serializes envelope]```## Decision Logic@@ -128,15 +127,15 @@ flowchart TD## Side Effects-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_drive'`.-- **Broadcasts**: none.+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_drive'`.+- **Broadcasts**: none from the operation; executor emits standard `node_status`.- **External API calls**: Drive API v3 - `files().create/get/list`, `permissions().create`, `files().get_media`. Plus `httpx.AsyncClient.get(file_url)` on upload-from-URL.- **File I/O**: in-memory `io.BytesIO` buffers only; no disk writes.- **Subprocess**: none.## External Dependencies-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.- **Services**: Google Drive API, `PricingService`, `Database`.- **Python packages**: `google-api-python-client`, `httpx`.- **Environment variables**: none.diff --git a/docs-internal/node-logic-flows/google_workspace/googleGmail.md b/docs-internal/node-logic-flows/google_workspace/googleGmail.mdindex bb5b6e2d..fa275f3a 100644--- a/docs-internal/node-logic-flows/google_workspace/googleGmail.md+++ b/docs-internal/node-logic-flows/google_workspace/googleGmail.md@@ -3,18 +3,18 @@| Field | Value ||------|-------|| **Category** | google_workspace / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/gmail.py::handle_google_gmail`](../../../server/services/handlers/gmail.py) |+| **Backend handler** | [`server/nodes/google/gmail/__init__.py`](../../../server/nodes/google/gmail/__init__.py) (`GmailNode`; dispatched via `BaseNode.execute()` -> per-operation `@Operation` methods `send` / `search` / `read`) || **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) || **Skill (if any)** | [`server/skills/productivity_agent/google-gmail-skill/SKILL.md`](../../../server/skills/productivity_agent/google-gmail-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `googleGmail` |+| **Dual-purpose tool** | yes - tool name `google_gmail` |## PurposeConsolidated Gmail node covering send, search, and read operations. Uses theGoogle Gmail API v1 via the official `google-api-python-client`. One node, three-operations switched via the `operation` parameter. Authenticates through the-shared `get_google_credentials()` helper that reads OAuth tokens stored via-`auth_service.store_oauth_tokens("google", ...)`.+operations, each its own `@Operation` method selected by the `operation` Param.+Authenticates through `GoogleCredential.build_credentials()` (via+`build_google_service`) which reads OAuth tokens stored under provider `google`.## Inputs (handles)@@ -52,76 +52,75 @@ Top-level dispatcher: `operation` (one of `send`, `search`, `read`).| `message_id` | string | `""` | **yes** | Gmail message ID || `format` | options | `full` | no | `full` / `minimal` / `raw` / `metadata` |-Shared: `account_mode` (default `owner`), `customer_id` (required when-`account_mode == 'customer'`).+Params model: `GmailParams` (`model_config extra="ignore"`). All non-`operation`+fields carry `displayOptions.show` keyed on the active operation. There is no+`account_mode` / `customer_id` field — credential scoping is resolved inside+`GoogleCredential.build_credentials`.## Outputs (handles)+The node declares only `input-main` and `output-main`. When wired to an AI+agent's `input-tools` handle (`usable_as_tool = True`, tool name `google_gmail`)+the same `output-main` payload is returned to the LLM — there is no separate+`output-tool` handle.+| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Operation-specific payload (see below) |-| `output-tool` | object | Same payload, used when wired to an AI agent `input-tools` |+| `output-main` | object | Operation-specific `GmailOutput` payload (see below) |-### Output payloads+### Output payloads (`GmailOutput`, `model_config extra="allow"`)-- `send`: `{message_id, thread_id, label_ids, to, subject}`-- `search`: `{messages: [...], count, query, result_size_estimate}` where each- message is `{message_id, thread_id, from, to, cc, subject, date, snippet,- labels, size_estimate, body?, attachments?}`-- `read`: formatted message (same shape as a single entry in `search`)+- `send`: `{operation: "send", message_id, thread_id, label_ids, to, subject}`+- `search`: `{operation: "search", messages: [...], count, query, result_size_estimate}`+ where each message comes from `format_message`+- `read`: `{operation: "read", ...format_message(result)}` (the `size_estimate`+ key is stripped out)## Logic Flow```mermaidflowchart TD- A[handle_google_gmail] --> B{operation?}- B -- send --> S1[handle_gmail_send]- B -- search --> S2[handle_gmail_search]- B -- read --> S3[handle_gmail_read]- B -- unknown --> Eret[Return success=false<br/>Unknown Gmail operation: X]- S1 --> C1[get_google_credentials]- S2 --> C2[get_google_credentials]- S3 --> C3[get_google_credentials]- C1 -- ValueError --> Eret- C2 -- ValueError --> Eret- C3 -- ValueError --> Eret- C1 --> V1{to/subject/body<br/>all present?}- V1 -- no --> Eret+ A[BaseNode.execute -> @Operation method] --> B{operation?}+ B -- send --> S1[GmailNode.send]+ B -- search --> S2[GmailNode.search]+ B -- read --> S3[GmailNode.read]+ S1 --> V1{to/subject/body<br/>all present?}+ V1 -- no --> Eret[raise RuntimeError]V1 -- yes --> B1[Build MIMEText/MIMEMultipart<br/>base64 urlsafe encode]- B1 --> R1[service.users.messages.send<br/>run_in_executor]- C2 --> V2{query non-empty?}+ B1 --> SVC1[_service -> build_google_service gmail v1]+ SVC1 --> R1[users.messages.send via run_sync]+ S2 --> V2{query non-empty?}V2 -- no --> Eret- V2 -- yes --> R2[service.users.messages.list q=query]+ V2 -- yes --> R2[users.messages.list q=query]R2 --> L2[Loop: users.messages.get per id<br/>format=full if include_body else metadata]- C3 --> V3{message_id?}+ S3 --> V3{message_id?}V3 -- no --> Eret- V3 -- yes --> R3[service.users.messages.get id=message_id]- R1 --> T[_track_gmail_usage -> save_api_usage_metric]+ V3 -- yes --> R3[users.messages.get id=message_id]+ R1 --> T[track_google_usage gmail -> save_api_usage_metric]L2 --> TR3 --> T- T --> OUT[Return success envelope]+ T --> OUT[Return GmailOutput; BaseNode.execute serializes envelope]```## Decision Logic-- **Operation dispatch**: `if/elif` chain in `handle_google_gmail`. Unknown operation returns an error envelope (does NOT raise).-- **Credential resolution** (`get_google_credentials`): owner mode reads from `auth_service.get_oauth_tokens("google", customer_id="owner")`; customer mode reads from `google_connections` table. Raises `ValueError` if no tokens found.-- **Proactive refresh**: if the Credentials object reports `expired and not valid` and a `refresh_token` is present, a refresh is attempted in a thread executor; result is persisted via `auth_service.store_oauth_tokens`. Refresh failures are swallowed to DEBUG.-- **Search body fetching**: `include_body=False` uses `format=metadata` with a fixed header list (`From`, `To`, `Subject`, `Date`); `True` uses `format=full`. Body extraction recurses into multipart payloads; prefers `text/plain` over `text/html`.+- **Operation dispatch**: the `@Operation`-decorated method (`send` / `search` / `read`) is selected by the framework from `params.operation`. Missing-field validation in each method raises `RuntimeError` (surfaced by `BaseNode.execute` as an error envelope).+- **Credential resolution**: `build_google_service` -> `GoogleCredential.build_credentials(params, ctx.raw)` reads stored OAuth tokens for provider `google` (owner / customer scoping handled inside the credential class). The blocking `build(...)` runs in an executor.+- **Search body fetching**: `include_body=False` uses `format=metadata` with a fixed header list (`From`, `To`, `Subject`, `Date`); `True` uses `format=full`. `format_message` (in `_gmail.py`) extracts the body.- **Send body type**: `body_type=='html'` -> `MIMEMultipart('alternative')` with an HTML part; otherwise `MIMEText(body, 'plain')`. Message is `urlsafe_b64encode`d to base64.-- **Error paths**: every sub-handler wraps the whole body in `try/except Exception` and returns `{success: false, error: str(e), execution_time: ...}`.+- **Error paths**: each method raises `RuntimeError` for missing required fields; `BaseNode.execute()` wraps the body in `log_context` + a span and converts exceptions into the standard error envelope.## Side Effects-- **Database writes**: one row per API call in `api_usage_metrics` via `database.save_api_usage_metric` with `service='googleGmail'`, `operation=<action>`, `endpoint=<action>`, `resource_count`, `cost` (always 0 for Google APIs).-- **Broadcasts**: none from the handler itself; `NodeExecutor` emits standard `node_status`.-- **External API calls**: Gmail API v1 via `googleapiclient.discovery.build("googleGmail", "v1", creds)` - `users().messages().send/list/get`. Proactive token refresh hits `https://oauth2.googleapis.com/token`.+- **Database writes**: one row per API call in `api_usage_metrics` via `track_google_usage` (`_base.py`) -> `database.save_api_usage_metric` with `service='gmail'`, `endpoint=<action>`, `resource_count`, `cost` (always 0 for Google APIs).+- **Broadcasts**: none from the operation itself; `BaseNode.execute()` / the executor emit standard `node_status`.+- **External API calls**: Gmail API v1 via `googleapiclient.discovery.build("gmail", "v1", creds)` - `users().messages().send/list/get`. Token refresh hits `https://oauth2.googleapis.com/token`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: OAuth tokens via `auth_service.get_oauth_tokens("google", customer_id="owner")`; `google_client_id` / `google_client_secret` via `auth_service.get_api_key(...)`.+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google` (`access_token` / `refresh_token`); `google_client_id` / `google_client_secret` via the API-key store.- **Services**: Google Gmail API, `PricingService`, `Database`.- **Python packages**: `google-auth`, `google-auth-oauthlib`, `google-api-python-client`.- **Environment variables**: none (OAuth redirect URI derived from request at auth time).@@ -130,8 +129,7 @@ flowchart TD- `max_results` is silently clamped to 100 (hard ceiling).- Search fetches every matching message individually in sequence - with `max_results=100` this is 100 round trips. No batching.-- The proactive refresh path swallows exceptions - if the refresh token has been revoked the handler continues and subsequently fails with a 401-shaped error surfaced as `"<status>"` in the envelope.-- `_format_message` returns empty strings for missing headers rather than omitting them; callers should not treat `""` as "header was present and empty".+- `format_message` (in `_gmail.py`) returns empty strings for missing headers rather than omitting them; callers should not treat `""` as "header was present and empty".- `body` is returned as a UTF-8 decoded string with `errors='ignore'` - binary parts may produce partial output without raising.## Relateddiff --git a/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.md b/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.mdindex 000341b1..7b40e72e 100644--- a/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.md+++ b/docs-internal/node-logic-flows/google_workspace/googleGmailReceive.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | google_workspace / trigger (polling) |-| **Backend handler** | [`server/services/handlers/gmail.py::handle_gmail_receive`](../../../server/services/handlers/gmail.py) |+| **Backend handler** | [`server/nodes/google/gmail_receive/__init__.py`](../../../server/nodes/google/gmail_receive/__init__.py) (`GmailReceiveNode`, a `PollingTriggerNode`; inline-Run path uses the `execute()` override, deployment path uses the `setup_service` / `fetch_ids` / `fetch_detail` / `post_emit` hooks driven by `PollingTriggerWorkflow`) || **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -12,13 +12,18 @@Polling-based trigger that fires when a new Gmail message matching a filterquery arrives. Unlike push-based triggers (webhook, WhatsApp, Telegram) Gmail-has no official push without extra Pub/Sub setup, so this handler polls+has no official push without extra Pub/Sub setup, so this node polls`users.messages.list` at a configurable interval, diffs IDs against abaseline, fetches full details for new messages, and optionally marks themas read.-Dispatches a `gmail_email_received` event via `event_waiter.dispatch` so-deployment-mode listeners can react in parallel.+`event_type = "gmail_email_received"`. In deployment (canary) mode the+`PollingTriggerWorkflow` owns event fan-out directly: it calls the per-cycle+activity built from `PollingTriggerNode.as_poll_activity()` and spawns the+child `MachinaWorkflow` per new email — there is no `event_waiter.dispatch`+step. The `_events.py` `gmail_message_received` CloudEvents factory is kept for+parity / future ad-hoc emits; the legacy `dispatch_gmail_received` shim was+removed in Wave 13.## Inputs (handles)diff --git a/docs-internal/node-logic-flows/google_workspace/googleSheets.md b/docs-internal/node-logic-flows/google_workspace/googleSheets.mdindex 419e8e0d..28275448 100644--- a/docs-internal/node-logic-flows/google_workspace/googleSheets.md+++ b/docs-internal/node-logic-flows/google_workspace/googleSheets.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | google_workspace / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/sheets.py::handle_google_sheets`](../../../server/services/handlers/sheets.py) |+| **Backend handler** | [`server/nodes/google/sheets/__init__.py`](../../../server/nodes/google/sheets/__init__.py) (`SheetsNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) || **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) || **Skill (if any)** | [`server/skills/productivity_agent/google-sheets-skill/SKILL.md`](../../../server/skills/productivity_agent/google-sheets-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `googleSheets` |+| **Dual-purpose tool** | yes - tool name `google_sheets` |## Purpose@@ -29,7 +29,7 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).| Name | Type | Default | Required | Description ||------|------|---------|----------|-------------|| `spreadsheet_id` | string | `""` | **yes** | Sheet ID (from URL) |-| `range` | string | `""` | **yes** | A1 notation, e.g. `Sheet1!A1:D10` |+| `range` | string | `A1:Z1000` | **yes** | A1 notation, e.g. `Sheet1!A1:D10` || `value_render_option` | options | `FORMATTED_VALUE` | no | `FORMATTED_VALUE` / `UNFORMATTED_VALUE` / `FORMULA` || `major_dimension` | options | `ROWS` | no | `ROWS` / `COLUMNS` |@@ -38,7 +38,7 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).| Name | Type | Default | Required | Description ||------|------|---------|----------|-------------|| `spreadsheet_id` | string | `""` | **yes** | - |-| `range` | string | `""` | **yes** | e.g. `Sheet1!A1` |+| `range` | string | `A1:Z1000` | **yes** | e.g. `Sheet1!A1` || `values` | array/string | `[]` | **yes** | 2D array, or JSON string that parses to 2D array, or 1D auto-wrapped to 2D || `value_input_option` | options | `USER_ENTERED` | no | `RAW` or `USER_ENTERED` |@@ -47,17 +47,20 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).| Name | Type | Default | Required | Description ||------|------|---------|----------|-------------|| `spreadsheet_id` | string | `""` | **yes** | - |-| `range` | string | `""` | **yes** | e.g. `Sheet1!A:D` |+| `range` | string | `A1:Z1000` | **yes** | e.g. `Sheet1!A:D` || `values` | array/string | `[]` | **yes** | Same coercion as write || `value_input_option` | options | `USER_ENTERED` | no | - || `insert_data_option` | options | `INSERT_ROWS` | no | `INSERT_ROWS` / `OVERWRITE` |## Outputs (handles)+The node declares only `input-main` and `output-main`. Tool mode+(`usable_as_tool = True`, tool name `google_sheets`) returns the same+`output-main` payload — there is no separate `output-tool` handle.+| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Operation-specific payload |-| `output-tool` | object | Same, for AI tool wiring |+| `output-main` | object | Operation-specific `SheetsOutput` payload |- `read`: `{values: [[...],...], range, rows, columns, major_dimension}`- `write` / `append`: `{updated_range, updated_rows, updated_columns, updated_cells, table_range?}`@@ -66,50 +69,44 @@ Top-level dispatcher: `operation` (one of `read`, `write`, `append`).```mermaidflowchart TD- A[handle_google_sheets] --> B{operation?}- B -- read --> R[handle_sheets_read]- B -- write --> W[handle_sheets_write]- B -- append --> AP[handle_sheets_append]- B -- unknown --> Eret[success=false<br/>Unknown Sheets operation]- R --> G[get_google_credentials + build v4]- W --> G- AP --> G- G -- ValueError --> Eret- R --> V1{spreadsheet_id & range?}- V1 -- no --> Eret- V1 -- yes --> R1[spreadsheets.values.get<br/>valueRenderOption majorDimension]- W --> V2{spreadsheet_id & range & values?}- V2 -- no --> Eret- V2 -- yes --> C2[Coerce values:<br/>str -> json.loads<br/>1D -> wrap to 2D]- C2 --> R2[spreadsheets.values.update<br/>valueInputOption body]- AP --> V3{spreadsheet_id & range & values?}- V3 -- no --> Eret- V3 -- yes --> C3[Same coercion as write]- C3 --> R3[spreadsheets.values.append<br/>valueInputOption insertDataOption]- R1 --> T[_track_sheets_usage]+ A[BaseNode.execute -> SheetsNode.dispatch] --> V0{spreadsheet_id & range present?}+ V0 -- no --> Eret[raise RuntimeError]+ V0 -- yes --> G[build_google_service sheets v4]+ G --> B{operation?}+ B -- read --> R1[spreadsheets.values.get<br/>valueRenderOption majorDimension]+ B -- write --> CW[Coerce values:<br/>str -> json.loads, 1D -> wrap to 2D]+ CW --> VW{values non-empty?}+ VW -- no --> Eret+ VW -- yes --> R2[spreadsheets.values.update<br/>valueInputOption body]+ B -- append --> CA[Same coercion as write]+ CA --> VA{values non-empty?}+ VA -- no --> Eret+ VA -- yes --> R3[spreadsheets.values.append<br/>valueInputOption insertDataOption]+ B -- unknown --> Eret+ R1 --> T[track_google_usage google_sheets]R2 --> TR3 --> T- T --> OUT[Return success envelope]+ T --> OUT[Return SheetsOutput; BaseNode serializes envelope]```## Decision Logic- **Values coercion**: if `values` is a `str`, parsed via `json.loads`. If the (possibly parsed) sequence's first element is not a list, the whole thing is wrapped into `[values]` to guarantee 2D.-- **Required fields**: each operation has its own validation block; all three require `spreadsheet_id` and `range`, write/append additionally require a non-empty `values`.+- **Required fields**: `spreadsheet_id` and `range` are validated once at the top of `dispatch` (before the operation branch); write/append additionally require a non-empty `values`.- **Usage tracking count**: read reports `len(values)` rows; write reports `updatedCells`; append reports `updates.updatedCells`. If any field is missing in the response, usage count is 0 (no exception).- **JSON parse errors** on `values` propagate into the outer `except Exception` and become `error: "<ValueError str>"`.## Side Effects-- **Database writes**: `api_usage_metrics` row per call via `save_api_usage_metric` with `service='google_sheets'`.-- **Broadcasts**: none.+- **Database writes**: `api_usage_metrics` row per call via `track_google_usage` -> `save_api_usage_metric` with `service='google_sheets'`.+- **Broadcasts**: none from the operation; executor emits standard `node_status`.- **External API calls**: Sheets API v4 - `spreadsheets().values().get/update/append`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.- **Services**: Google Sheets API, `PricingService`, `Database`.- **Python packages**: `google-api-python-client`.- **Environment variables**: none.diff --git a/docs-internal/node-logic-flows/google_workspace/googleTasks.md b/docs-internal/node-logic-flows/google_workspace/googleTasks.mdindex 08292cb5..6e34a58f 100644--- a/docs-internal/node-logic-flows/google_workspace/googleTasks.md+++ b/docs-internal/node-logic-flows/google_workspace/googleTasks.md@@ -3,15 +3,15 @@| Field | Value ||------|-------|| **Category** | google_workspace / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/tasks.py::handle_google_tasks`](../../../server/services/handlers/tasks.py) |+| **Backend handler** | [`server/nodes/google/tasks/__init__.py`](../../../server/nodes/google/tasks/__init__.py) (`TasksNode`; dispatched via `BaseNode.execute()` -> single `@Operation("dispatch")` method that branches on `params.operation`) || **Tests** | [`server/tests/nodes/test_google_workspace.py`](../../../server/tests/nodes/test_google_workspace.py) || **Skill (if any)** | [`server/skills/productivity_agent/google-tasks-skill/SKILL.md`](../../../server/skills/productivity_agent/google-tasks-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `googleTasks` |+| **Dual-purpose tool** | yes - tool name `google_tasks` |## PurposeConsolidated Google Tasks node for managing personal to-do items. Uses Google-Tasks API v1 (`tasklists` + `googleTasks` collections). One node, five operations+Tasks API v1 (`tasklists` + `tasks` collections). One node, five operationsswitched via the `operation` parameter.## Inputs (handles)@@ -28,10 +28,11 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `complete`, `update`| Name | Type | Default | Required | Description ||------|------|---------|----------|-------------|-| `title` | string | `""` | **yes** | Task title |-| `notes` | string | `""` | no | Task description |-| `due_date` | string | `""` | no | RFC 3339; date-only is upgraded to `T00:00:00.000Z` |-| `tasklist_id` | string | `@default` | no | Which tasklist |+| `title` | string | `null` | **yes** | Task title |+| `notes` | string | `null` | no | Task description |+| `due_date` | string | `null` | no | RFC 3339; date-only is upgraded to `T00:00:00.000Z` |+| `status` | string | `null` | no | Task status (Param exists; not written by the `create` branch) |+| `tasklist_id` | string | `@default` | no | Which tasklist (`loadOptionsMethod: googleTasklists`) |### `operation = list`@@ -57,8 +58,9 @@ Top-level dispatcher: `operation` (one of `create`, `list`, `complete`, `update`| `tasklist_id` | string | `@default` | no | - || `title` / `notes` / `due_date` / `status` | string | `""` | no | Patch fields |-Also accepts `update_title`, `update_notes`, `update_due_date`, `update_status`-aliases (the dispatcher rewrites them before delegating).+Also accepts the optional `update_title`, `update_notes`, `update_due_date`,+`update_status` fields. The dispatch method reads `params.update_title or+params.title` (etc.) as fallback chains — it does NOT mutate the params dict.### `operation = delete`@@ -69,10 +71,13 @@ aliases (the dispatcher rewrites them before delegating).## Outputs (handles)+The node declares only `input-main` and `output-main`. Tool mode+(`usable_as_tool = True`, tool name `google_tasks`) returns the same+`output-main` payload — there is no separate `output-tool` handle.+| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Operation-specific payload |-| `output-tool` | object | Same, for AI tool wiring |+| `output-main` | object | Operation-specific `TasksOutput` payload |- `create` / `complete` / `update`: `{task_id, title, notes?, due, status, completed?, self_link?}`- `list`: `{tasks: [{task_id, title, notes, due, status, completed, position}], count}`@@ -82,73 +87,59 @@ aliases (the dispatcher rewrites them before delegating).```mermaidflowchart TD- A[handle_google_tasks] --> AL{operation == update?}- AL -- yes --> AM[Alias update_* params<br/>to plain names]- AL -- no --> B- AM --> B{operation?}- B -- create --> C1[handle_tasks_create]- B -- list --> C2[handle_tasks_list]- B -- complete --> C3[handle_tasks_complete]- B -- update --> C4[handle_tasks_update]- B -- delete --> C5[handle_tasks_delete]- B -- unknown --> Eret[success=false<br/>Unknown Tasks operation]- C1 --> G[get_google_credentials + build v1]- C2 --> G- C3 --> G- C4 --> G- C5 --> G- G -- ValueError --> Eret- C1 --> V1{title?}- V1 -- no --> Eret+ A[BaseNode.execute -> TasksNode.dispatch] --> G[build_google_service tasks v1]+ G --> B{operation?}+ B -- create --> V1{title?}+ V1 -- no --> Eret[raise RuntimeError]V1 -- yes --> DF[Normalize due_date<br/>append T00:00:00.000Z if missing]DF --> R1[tasks.insert]- C2 --> R2[tasks.list]- C3 --> V3{task_id?}+ B -- list --> R2[tasks.list]+ B -- complete --> V3{task_id?}V3 -- no --> EretV3 -- yes --> G3[tasks.get -> set status=completed -> tasks.update]- C4 --> V4{task_id?}+ B -- update --> V4{task_id?}V4 -- no --> Eret- V4 -- yes --> G4[tasks.get -> merge patch -> tasks.update]- C5 --> V5{task_id?}+ V4 -- yes --> G4[tasks.get -> merge update_* fallbacks -> tasks.update]+ B -- delete --> V5{task_id?}V5 -- no --> EretV5 -- yes --> R5[tasks.delete]- R1 --> T[_track_tasks_usage]+ B -- unknown --> Eret+ R1 --> T[track_google_usage google_tasks]R2 --> TG3 --> TG4 --> TR5 --> T- T --> OUT[Return success envelope]+ T --> OUT[Return TasksOutput; BaseNode serializes envelope]```## Decision Logic-- **Update parameter aliasing** (dispatcher): the dispatcher mutates the incoming `parameters` dict to promote `update_*` keys onto their plain counterparts BEFORE delegating. Irreversible.+- **Operation branch**: one `@Operation("dispatch")` method branches on `params.operation`; `complete`/`update`/`delete` share a `task_id`-required block.+- **Update fallback chains**: `params.update_title or params.title` (etc.) — no mutation of the params dict; either field works.- **Due-date upgrade**: any `due_date` that lacks a `T` gets `T00:00:00.000Z` appended. There is no strict ISO validation otherwise.- **Complete** is a convenience op - it does a read-modify-write (`tasks.get` then `tasks.update`) with `status='completed'`. Two API calls per completion.-- **Update** merges only truthy fields (uses `parameters.get('title')` etc. which skips empty strings). Callers cannot clear `notes` via the update path - use an explicit `status='needsAction'` or direct API.+- **Update** merges only truthy fields (skips empty strings). Callers cannot clear `notes` via the update path.- **`tasks.list`** does NOT reject `max_results > 100` - the API itself caps silently.## Side Effects-- **Database writes**: `api_usage_metrics` row per API call via `save_api_usage_metric` with `service='google_tasks'`. `list` records one row with `resource_count = len(tasks)`, but `_track_tasks_usage` is NOT called on the list path (see bug below).-- **Broadcasts**: none.+- **Database writes**: `api_usage_metrics` row per API call via `track_google_usage` -> `save_api_usage_metric` with `service='google_tasks'`. The `list` branch DOES call `track_google_usage` (`resource_count = len(tasks)`).+- **Broadcasts**: none from the operation; executor emits standard `node_status`.- **External API calls**: Tasks API v1 - `tasks().insert/list/get/update/delete`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: OAuth via `auth_service.get_oauth_tokens("google")`.+- **Credentials**: `GoogleCredential` -> OAuth tokens for provider `google`.- **Services**: Google Tasks API, `PricingService`, `Database`.- **Python packages**: `google-api-python-client`.- **Environment variables**: none.## Edge cases & known limits-- **`handle_tasks_list` does not call `_track_tasks_usage`** (all other operations do). The list operation therefore skips the `api_usage_metrics` row. Handler bug noted, not fixed.- Update cannot clear a field - empty strings are filtered out by truthiness. To clear notes or title you must bypass this node.- Complete is non-idempotent on the server side: re-completing a completed task overwrites the `completed` timestamp.-- Dispatcher mutates `parameters` in place - callers that reuse the dict will see `title`/`notes`/`due_date`/`status` filled in from `update_*` aliases.- `tasklist_id='@default'` only resolves for authenticated users who have created at least one list; freshly authenticated users may get a 404 until Google Tasks provisions their default list.## Relateddiff --git a/docs-internal/node-logic-flows/http_proxy/httpRequest.md b/docs-internal/node-logic-flows/http_proxy/httpRequest.mdindex e49caf72..6af9872c 100644--- a/docs-internal/node-logic-flows/http_proxy/httpRequest.md+++ b/docs-internal/node-logic-flows/http_proxy/httpRequest.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | utility / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/http.py::handle_http_request`](../../../server/services/handlers/http.py) |+| **Backend handler** | [`server/nodes/utility/http_request/__init__.py`](../../../server/nodes/utility/http_request/__init__.py) — `HttpRequestNode.request` (dispatched via `BaseNode.execute()` + `@Operation("request")`; the legacy `handlers/http.py` was deleted in Wave 11.D.3) || **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) || **Skill (if any)** | [`server/skills/web_agent/http-request-skill/SKILL.md`](../../../server/skills/web_agent/http-request-skill/SKILL.md) |-| **Dual-purpose tool** | yes - exposed to agents as a generic HTTP tool |+| **Dual-purpose tool** | yes - tool name `http_request` (`usable_as_tool = True`) |## Purpose@@ -26,14 +26,15 @@ configured residential proxy provider via `ProxyService`.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|| `method` | options | `GET` | no | - | One of `GET` / `POST` / `PUT` / `DELETE` / `PATCH` |-| `url` | string | `""` | **yes** | - | Target URL; empty value short-circuits with `URL is required` |-| `headers` | string (JSON) | `"{}"` | no | - | JSON object; invalid JSON silently falls back to `{}` |-| `body` | string | `""` | no | `method in [POST, PUT, PATCH]` | Sent as JSON when parseable, else as raw content |-| `timeout` | number | `30` | no | - | Seconds; coerced via `float(...)` |-| `useProxy` | boolean | `false` | no | - | Route through `ProxyService.get_proxy_url` |-| `proxyProvider` | string | `""` | no | `useProxy=true` | Specific provider name (empty -> auto-select) |-| `proxyCountry` | string | `""` | no | `useProxy=true` | ISO country code for geo-targeting |-| `sessionType` | options | `rotating` | no | `useProxy=true` | `rotating` or `sticky` |+| `url` | string | `""` | **yes** | - | Target URL; empty value raises `RuntimeError('URL is required')` |+| `headers` | object (`Dict[str,str]`) | `{}` | no | - | Request headers passed straight to httpx |+| `body` | `Optional[Any]` | `null` | no | `method in [POST, PUT, PATCH]` | JSON object or raw string; ignored for GET/DELETE. String parsed as JSON when possible, else sent as raw content |+| `timeout` | number | `30` | no | - | Seconds (1-600); coerced via `float(...)` |+| `use_proxy` | boolean | `false` | no | - | Route through `ProxyService.get_proxy_url` |+| `proxy_provider` | string | `auto` | no | `use_proxy=true` | Specific provider name (`auto` selects by health score) |+| `proxy_country` | string | `""` | no | `use_proxy=true` | ISO country code for geo-targeting |+| `session_type` | options | `rotating` | no | `use_proxy=true` | `rotating` or `sticky` |+| `sticky_duration` | number | `600` | no | `use_proxy=true` and `session_type=sticky` | Sticky session duration in seconds |## Outputs (handles)@@ -54,50 +55,48 @@ configured residential proxy provider via `ProxyService`.}```-Wrapped in the standard envelope: `{ success: <status<400>, node_id, node_type: 'httpRequest', result, execution_time, timestamp }`.+The `request` op returns this dict directly; `BaseNode.execute()` wraps it in the standard success envelope. A status `>= 400` raises `RuntimeError(f"HTTP {status}: ...")` instead of returning, so it lands in the error envelope. The declared `HttpRequestOutput` model only pins `status` / `headers` / `body` (`extra="allow"`), so the extra `data` / `url` / `method` / `proxied` keys pass through unvalidated.## Logic Flow```mermaidflowchart TD- A[handle_http_request] --> B[Read method/url/headers/body/timeout]+ A[HttpRequestNode.request] --> B[Read params method/url/headers/body/timeout]B --> C{url truthy?}- C -- no --> Eerr[Return success=false<br/>error: URL is required]- C -- yes --> D[Parse headers JSON<br/>fallback {} on JSONDecodeError]- D --> E{useProxy?}- E -- no --> G[httpx.AsyncClient timeout=timeout]- E -- yes --> F[_get_proxy_url_if_enabled<br/>-> proxy_svc.get_proxy_url]- F -- returns None --> G- F -- returns url --> G2[httpx.AsyncClient proxy=proxy_url timeout=timeout]- F -- raises --> G3[Log warning, proceed without proxy -> G]- G --> H{method in POST/PUT/PATCH<br/>and body truthy?}+ C -- no --> Eerr[raise RuntimeError<br/>URL is required]+ C -- yes --> D[_resolve_proxy_url use_proxy params]+ D -- use_proxy false --> G[httpx.AsyncClient timeout=timeout]+ D -- returns url --> G2[httpx.AsyncClient proxy=proxy_url timeout=timeout]+ D -- service off / raises --> G3[Log warning, proceed without proxy -> G]+ G --> H{method in POST/PUT/PATCH<br/>and body is not None?}G2 --> HG3 --> H- H -- yes JSON parseable --> I[kwargs.json = parsed]- H -- yes non-JSON --> I2[kwargs.content = body]+ H -- yes str JSON parseable --> I[kwargs.json = parsed]+ H -- yes str non-JSON --> I2[kwargs.content = body]+ H -- yes non-str --> I4[kwargs.json = body]H -- no --> I3[no body]I --> J[await client.request]I2 --> JI3 --> J- J -- response --> K[response.json fallback text]- K --> L[Return success=(status<400)<br/>with status/data/headers/url/method/proxied]- J -- TimeoutException --> T[Return success=false<br/>error: Request timed out]- J -- Exception --> Z[Return success=false<br/>error: str(e)]+ I4 --> J+ J --> K[response.json fallback text]+ K --> L{status >= 400?}+ L -- yes --> M[raise RuntimeError HTTP status]+ L -- no --> N[return dict status/data/headers/url/method/proxied]```## Decision Logic-- **Validation**: `url` empty -> raises `ValueError('URL is required')` caught by outer `try`.+- **Validation**: `url` empty -> raises `RuntimeError('URL is required')` (surfaced via the error envelope by `BaseNode.execute()`).- **Branches**:- - `useProxy=true` -> attempt proxy lookup; swallow proxy errors and fall back to direct call.- - Body handling depends on `method` and JSON-parseability of `body`.+ - `use_proxy=true` -> `_resolve_proxy_url` attempts a lookup; proxy errors are swallowed and the request falls back to a direct call.+ - Body handling depends on `method` and the type / JSON-parseability of `body` (str-JSON -> `json`, str-non-JSON -> `content`, non-str -> `json`).- **Fallbacks**:- - Invalid headers JSON -> `{}`.- - Proxy service disabled / `get_proxy_url` raises -> proceed without proxy.+ - Proxy service disabled / not enabled / `get_proxy_url` raises -> proceed without proxy (returns `None`).- Non-JSON response body -> plain `text` string in `data`.- **Error paths**:- - `TimeoutException` (class name match) -> user-friendly `Request timed out after N seconds`.- - Any other exception -> `str(e)` verbatim in envelope.+ - Status `>= 400` -> `raise RuntimeError(f"HTTP {status}: {body!r}")`.+ - Any httpx exception (timeout, connection error, etc.) propagates uncaught; `BaseNode.execute()` produces the error envelope with the traceback. There is no timeout special-casing.## Side Effects@@ -116,10 +115,9 @@ flowchart TD## Edge cases & known limits-- Proxy lookup errors are **swallowed** (`_get_proxy_url_if_enabled` logs a warning and returns `None`); the request then proceeds directly, silently losing the proxy. Downstream nodes can detect this by checking `result.proxied`.-- Timeout detection is by `type(e).__name__ == 'TimeoutException'` (string match) rather than `isinstance`; subclass hierarchy changes in httpx could bypass this branch.-- `success` in the envelope reflects `status < 400`, so non-2xx 3xx responses (e.g. unfollowed 302) count as success.-- The handler always wraps with `execution_time = time.time() - start_time`; no caching, no retry (retry is only implemented in `proxyRequest`).+- Proxy lookup errors are **swallowed** (`_resolve_proxy_url` logs a warning and returns `None`); the request then proceeds directly, silently losing the proxy. Downstream nodes can detect this by checking `result.proxied`.+- A `4xx`/`5xx` response raises `RuntimeError` rather than returning the body, so the error envelope carries the status; `3xx` are followed by httpx default and only surface if final status is `>= 400`.+- No caching, no retry (retry / failover is only implemented in `proxyRequest`).- `response.url` (final URL) is stringified; for non-proxied requests with redirects disabled this equals the request URL.## Relateddiff --git a/docs-internal/node-logic-flows/http_proxy/proxyConfig.md b/docs-internal/node-logic-flows/http_proxy/proxyConfig.mdindex 48b988bf..99cc7ead 100644--- a/docs-internal/node-logic-flows/http_proxy/proxyConfig.md+++ b/docs-internal/node-logic-flows/http_proxy/proxyConfig.md@@ -3,17 +3,18 @@| Field | Value ||------|-------|| **Category** | proxy / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/proxy.py::handle_proxy_config`](../../../server/services/handlers/proxy.py) (delegates to [`services/handlers/tools.py::_execute_proxy_config`](../../../server/services/handlers/tools.py)) |+| **Backend handler** | [`server/nodes/proxy/proxy_config/__init__.py`](../../../server/nodes/proxy/proxy_config/__init__.py) — `ProxyConfigNode.dispatch` (via `BaseNode.execute()` + `@Operation("dispatch")`) calls the module-level `execute_proxy_config(...)` 10-op dispatcher. The AI-tool fast path in [`server/services/handlers/tools.py`](../../../server/services/handlers/tools.py) imports the same `execute_proxy_config`. The legacy `handlers/proxy.py` was deleted in Wave 11. || **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) || **Skill (if any)** | [`server/skills/web_agent/proxy-config-skill/SKILL.md`](../../../server/skills/web_agent/proxy-config-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `proxy_config` |+| **Dual-purpose tool** | yes - tool name `proxy_config` (`destructive = True`, `usable_as_tool = True`) |## PurposeAdmin-style CRUD node for managing proxy providers, credentials, and routingrules. Dispatches on the `operation` param. Works the same as both a workflow-node and an AI-agent tool: the node handler simply forwards its parameters as-both `args` and `node_params` to the tool handler `_execute_proxy_config`.+node and an AI-agent tool: `ProxyConfigNode.dispatch` calls the module-level+`execute_proxy_config(params.model_dump())`, and the AI-tool fast path in+`tools.py` imports the same function — there is no separate handler shim.## Inputs (handles)@@ -31,15 +32,15 @@ both `args` and `node_params` to the tool handler `_execute_proxy_config`.| `gateway_port` | number | `0` | no | `operation in [add/update]` | Proxy port || `url_template` | string (JSON) | `"{}"` | no | `operation in [add/update]` | JSON template config || `cost_per_gb` | number | `0` | no | `operation in [add/update]` | USD per GB |-| `priority` | number | `50` | no | `operation in [add/update]` | Lower = preferred |-| `enabled` | boolean | `true` | no | `operation in [add/update]` | Provider enabled |+| `priority` | number | `50` | no | `operation in [add/update]` | Selection priority 0-100, higher ranks first |+| `enabled` | `Optional[bool]` | `null` | no | `operation in [add/update]` | Whether the provider is active (None = unchanged on update) || `username` | string | `""` | yes (op=set_credentials) | `operation=set_credentials` | Proxy username |-| `password` | string | `""` | yes (op=set_credentials) | `operation=set_credentials` | Proxy password |-| `domain_pattern` | string | `""` | yes (op=add_routing_rule) | `operation=add_routing_rule` | Glob like `*.linkedin.com` |-| `preferred_providers` | string (JSON array) | `"[]"` | no | `operation=add_routing_rule` | `["provider1"]` |+| `password` | string (password widget) | `""` | yes (op=set_credentials) | `operation=set_credentials` | Proxy password |+| `domain_pattern` | string | `""` | yes (op=add_routing_rule) | `operation=add_routing_rule` | Glob like `*.example.com` |+| `preferred_providers` | `Any` (JSON array string) | `"[]"` | no | `operation=add_routing_rule` | `["provider1"]` || `required_country` | string | `""` | no | `operation=add_routing_rule` | ISO country || `session_type` | options | `rotating` | no | `operation=add_routing_rule` | `rotating` or `sticky` |-| `rule_id` | number | `0` | yes (op=remove_routing_rule) | `operation=remove_routing_rule` | Rule PK |+| `rule_id` | `Optional[int]` | `null` | yes (op=remove_routing_rule) | `operation=remove_routing_rule` | Routing rule ID to remove |### Operation index@@ -65,19 +66,22 @@ both `args` and `node_params` to the tool handler `_execute_proxy_config`.### Output payload (success)```ts-// The node handler wraps the tool result under result = {...}+// execute_proxy_config returns this dict; dispatch returns it directly,+// then BaseNode.execute() wraps it once in the standard success envelope.{success: true,operation: string,- // op-specific fields: providers / rules / stats / name / rule_id / ip+latency_ms / updated_fields+ // op-specific fields: providers / rules / stats / name / rule_id / ip+latency_ms / status_code / updated_fields / domain_pattern}```+On `success=false`, `dispatch` raises `NodeUserError(result.error)` so the failure surfaces as an error envelope (a single WARN line), NOT a nested `result.result.success=false` payload.+## Logic Flow```mermaidflowchart TD- A[handle_proxy_config] --> B[_execute_proxy_config args=params node_params=params]+ A[ProxyConfigNode.dispatch] --> B[execute_proxy_config params.model_dump]B --> C{operation}C -- list_providers --> LP[proxy_svc.get_providers -> model_dump list]C -- get_stats --> GS[proxy_svc.get_stats]@@ -108,7 +112,7 @@ flowchart TDRR -- no --> Erid[success=false]RR -- yes --> RR2[db.delete_proxy_routing_rule int(rule_id) + reload]C -- unknown --> Eunk[success=false error Unknown operation]- LP --> W[handle_proxy_config wraps tool_result<br/>under result=]+ LP --> W[dispatch: success? return dict<br/>else raise NodeUserError]GS --> WLR --> WAP3 --> W@@ -133,7 +137,7 @@ flowchart TD- **Validation**: each op checks its own required fields before any service call.- **Branches**: 10 operations dispatched by string match; unknown op returns `Unknown operation: <op>`.- **Fallbacks**: `preferred_providers` JSON parse error -> `[]`; `url_template` JSON parse error -> early return with error (no fallback).-- **Error paths**: validation failures return `{success: false, error: "..."}` *without* setting `operation`. The node handler wraps the tool result in its own envelope and copies the tool's `success` into the outer envelope, so validation errors look like `{success: false, result: {success: false, error: "..."}}`.+- **Error paths**: per-op validation failures return `{success: false, error: "..."}` from `execute_proxy_config`. `dispatch` checks `result.get("success")` and, when false, `raise NodeUserError(result.error or "Proxy config failed")`, so the failure surfaces as a single-WARN error envelope rather than a nested success-false payload.## Side Effects@@ -154,9 +158,9 @@ flowchart TD## Edge cases & known limits-- `handle_proxy_config` passes the same dict for both `args` and `node_params`, so the `args.get(..., node_params.get(...))` fallback resolves to the same source. This is harmless but tautological.-- The outer node handler copies the tool's `success` into the envelope, yet still wraps the tool payload under `result`; tests must inspect both `result["success"]` AND `result["result"]["success"]`.-- `test_provider` does a **real** HTTP call to `httpbin.org/ip` when the handler is invoked - there is no offline / dry-run path; this call is un-mocked in production.+- The workflow-node path and the AI-tool path call the same `execute_proxy_config` function, so behaviour is identical between them.+- A failed op no longer double-wraps: `dispatch` raises `NodeUserError` on `success=false`, so tests inspect the error envelope rather than `result["result"]["success"]`.+- `test_provider` does a **real** HTTP call to `httpbin.org/ip` when invoked - there is no offline / dry-run path; this call is un-mocked in production.- `update_provider` silently returns `success=true` with `updated_fields=[]` if no fields were changed.- `remove_provider` / `remove_routing_rule` do not verify the row existed before issuing the delete; they always return `success=true` unless the DB raises.- `url_template` is re-serialized via `json.dumps(...)` before being saved, so whitespace in the caller's JSON is normalised.diff --git a/docs-internal/node-logic-flows/http_proxy/proxyRequest.md b/docs-internal/node-logic-flows/http_proxy/proxyRequest.mdindex 66bc5613..3c52e7fc 100644--- a/docs-internal/node-logic-flows/http_proxy/proxyRequest.md+++ b/docs-internal/node-logic-flows/http_proxy/proxyRequest.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | proxy / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/proxy.py::handle_proxy_request`](../../../server/services/handlers/proxy.py) |+| **Backend handler** | [`server/nodes/proxy/proxy_request/__init__.py`](../../../server/nodes/proxy/proxy_request/__init__.py) — `ProxyRequestNode.request` (dispatched via `BaseNode.execute()` + `@Operation("request")`; the legacy `handlers/proxy.py` was deleted in Wave 11.D.3) || **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) |-| **Skill (if any)** | [`server/skills/web_agent/http-request-skill/SKILL.md`](../../../server/skills/web_agent/http-request-skill/SKILL.md) (covers useProxy + proxyRequest) |-| **Dual-purpose tool** | yes - workflow node and AI tool |+| **Skill (if any)** | [`server/skills/web_agent/http-request-skill/SKILL.md`](../../../server/skills/web_agent/http-request-skill/SKILL.md) (covers use_proxy + proxyRequest) |+| **Dual-purpose tool** | yes - tool name `proxy_request` (`usable_as_tool = True`) |## Purpose@@ -28,16 +28,17 @@ latency per attempt and feeds them back into the provider's health score.|------|------|---------|----------|---------------------|-------------|| `method` | options | `GET` | no | - | `GET` / `POST` / `PUT` / `DELETE` / `PATCH` || `url` | string | `""` | **yes** | - | Target URL |-| `headers` | string (JSON) | `"{}"` | no | - | JSON object, falls back to `{}` on parse error |-| `body` | string | `""` | no | `method in [POST, PUT, PATCH]` | JSON if parseable else raw content |-| `timeout` | number | `30` | no | - | Seconds |-| `proxyProvider` | string | `""` | no | - | Specific provider name, empty for auto-select |-| `proxyCountry` | string | `""` | no | - | ISO country code |-| `sessionType` | options | `rotating` | no | - | `rotating` or `sticky` |-| `stickyDuration` | number | `300` | no | `sessionType=sticky` | Seconds (30-3600) |-| `maxRetries` | number | `3` | no | - | Number of retry attempts (`proxyMaxRetries` is also read) |-| `proxyFailover` | boolean | `true` | no | - | On failure, request a new proxy URL and retry |-| `followRedirects` | boolean | `true` | no | - | Declared in frontend; handler does not currently forward to httpx |+| `headers` | object (`Dict[str,str]`) | `{}` | no | - | Request headers passed straight to httpx |+| `body` | `Optional[Any]` | `null` | no | `method in [POST, PUT, PATCH]` | JSON object or raw string; string parsed as JSON if possible else raw content |+| `timeout` | number | `30` | no | - | Seconds (1-600) |+| `proxy_provider` | string | `auto` | no | - | Specific provider name; `auto` for health-based auto-select |+| `proxy_country` | string | `""` | no | - | ISO country code |+| `session_type` | options | `rotating` | no | - | `rotating` or `sticky` |+| `sticky_duration` | number | `600` | no | - | Sticky session duration in seconds (`ge=1`) |+| `max_retries` | number | `3` | no | - | Retry attempts (0-10); loop runs `max_retries + 1` times |+| `follow_redirects` | boolean | `true` | no | - | Declared field; handler does not currently forward it to httpx |++Params uses `extra="allow"`, so the retry loop also reads an undeclared `proxy_failover` key from the raw params (default `true`) — when `false` it breaks after the first failure.## Outputs (handles)@@ -63,57 +64,55 @@ latency per attempt and feeds them back into the provider's health score.### Output payload (failure)-`{ success: false, error: "All N attempts failed. Last error: <str>", ... }` or a short-circuit error if proxy service is unavailable.+All retries exhausted -> `raise RuntimeError("All N attempts failed. Last error: <str>")`. Short-circuit raises are `NodeUserError` ("Proxy service not initialized..." when the service is disabled; "No proxy provider available" when `get_proxy_url` returns `None`). A status `>= 400` on an attempt raises `NodeUserError(f"HTTP {status}: ...")`. `BaseNode.execute()` maps `NodeUserError` to a single WARN line + structured envelope, and the bare `RuntimeError` to a full-traceback error envelope.## Logic Flow```mermaidflowchart TD- A[handle_proxy_request] --> B{proxy_svc.is_enabled?}- B -- no --> Edis[Return success=false<br/>error: Proxy service not initialized]- B -- yes --> C[Read method/url/headers/body/timeout/maxRetries/failover]- C --> D{url truthy?}- D -- no --> Eval[Return success=false<br/>error: URL is required]- D -- yes --> E[Parse headers JSON; fallback {}]- E --> F[await proxy_svc.get_proxy_url]- F -- None --> Enop[Return success=false<br/>error: No proxy provider available]+ A[ProxyRequestNode.request] --> B{proxy_svc enabled?}+ B -- no --> Edis[raise NodeUserError<br/>Proxy service not initialized]+ B -- yes --> F[await proxy_svc.get_proxy_url url raw]+ F -- None --> Enop[raise NodeUserError<br/>No proxy provider available]F -- url --> G[attempt = 0]G --> H[httpx.AsyncClient proxy=proxy_url timeout=timeout]- H --> I{method in POST/PUT/PATCH and body?}+ H --> I{method in POST/PUT/PATCH and body is not None?}I -- yes JSON --> J[kwargs.json]I -- yes non-JSON --> J2[kwargs.content]I -- no --> J3[no body]J --> K[await client.request]J2 --> KJ3 --> K- K -- response --> L[proxy_svc.report_result success=status<400]- L --> M[_track_proxy_usage -> db.save_api_usage_metric]- M --> N[Return success=(status<400) with status/data/headers/url/method/provider/latency_ms/bytes/attempt]- K -- Exception --> R[proxy_svc.report_result success=False<br/>log warning]- R --> S{failover and attempt<maxRetries?}+ K --> L[proxy_svc.report_result success=status<400]+ L --> M[track_proxy_usage -> db.save_api_usage_metric]+ M --> Lq{status >= 400?}+ Lq -- yes --> Lerr[raise NodeUserError HTTP status]+ Lq -- no --> N[return dict status/data/headers/url/method/provider/latency_ms/bytes/attempt]+ H -- Exception --> R[proxy_svc.report_result success=False<br/>log warning]+ Lerr -. caught by attempt loop .-> R+ R --> S{failover and attempt<max_retries?}S -- yes --> T[await get_proxy_url again<br/>on exception break]T --> H- S -- no --> U[Return success=false<br/>error: All N attempts failed]+ S -- no --> U[raise RuntimeError<br/>All N attempts failed]```## Decision Logic- **Validation**:- - Service disabled -> short-circuit error envelope.- - Empty `url` -> `ValueError` caught -> error envelope.- - `get_proxy_url` returns `None` -> `No proxy provider available` error.+ - Service disabled / not enabled -> `raise NodeUserError("Proxy service not initialized...")`.+ - `get_proxy_url` returns `None` -> `raise NodeUserError("No proxy provider available")`.+ - (`url` is a required Pydantic field; an empty value is rejected at param validation, not in the op body.)- **Branches**:- - Retry loop runs up to `maxRetries + 1` times; `failover=false` breaks after first failure.- - Body JSON-parseable -> `kwargs.json`, else `kwargs.content`.-- **Fallbacks**: invalid headers JSON -> `{}`.+ - Retry loop runs up to `max_retries + 1` times; `proxy_failover=false` breaks after the first failure.+ - Body str-JSON-parseable -> `kwargs.json`, str-non-JSON -> `kwargs.content`, non-str -> `kwargs.json`.- **Error paths**:+ - A `>= 400` status raises `NodeUserError(HTTP ...)` *inside* the try block, so it is caught by the attempt loop and triggers retry/failover.- Per-attempt exception logged at `warning` and reported via `ProxyResult(success=False, error=...)`.- - Final error string: `All <maxRetries+1> attempts failed. Last error: <last_error>`.- - Outer unexpected exception -> `error: str(e)`.+ - After all attempts: `raise RuntimeError("All <max_retries+1> attempts failed. Last error: <last_error>")`.## Side Effects-- **Database writes**: one row per successful attempt in `api_usage_metrics` via `database.save_api_usage_metric`, with `service=proxy_<provider_name>`, `operation=proxy_request`, `cost` computed as `bytes / 1GB * cost_per_gb` from `config/pricing.json`.+- **Database writes**: one row per attempt that returned a response in `api_usage_metrics` via `database.save_api_usage_metric` (called from `track_proxy_usage` in [`server/nodes/proxy/_usage.py`](../../../server/nodes/proxy/_usage.py)), with `service=proxy_<provider_name>` (or `proxy` when empty), `operation=proxy_request`, `cost` computed as `bytes / 1GB * cost_per_gb` from `config/pricing.json`.- **In-memory writes**: `proxy_svc.report_result(...)` mutates the provider's rolling history deque (length 100), which feeds `compute_score()`. The service also increments `_daily_spend_usd`.- **Broadcasts**: none.- **External API calls**: the target URL, tunneled through the selected proxy URL.@@ -123,15 +122,15 @@ flowchart TD## External Dependencies- **Credentials**: `auth_service` stores proxy username/password under keys `proxy_<name>_username` / `proxy_<name>_password` (read by ProxyService when loading providers).-- **Services**: `ProxyService` (must be enabled), `PricingService` (read through `_track_proxy_usage` -> `pricing._config["proxy"]`), `Database`.+- **Services**: `ProxyService` (must be enabled), `PricingService` (read through `track_proxy_usage` -> `pricing._config["proxy"]`), `Database`.- **Python packages**: `httpx`.- **Environment variables**: none directly; ProxyService reads daily budget from `Settings.proxy_budget_daily_usd`.## Edge cases & known limits- Failed attempts are **not** tracked in `api_usage_metrics` - only successful requests that returned a response. If every attempt raises, the DB sees nothing for that call.-- `followRedirects` is defined in the frontend but never forwarded to `httpx`; httpx default follows redirects.-- `provider_name` in the result echoes the `proxyProvider` param verbatim; when left empty (auto-select), the envelope shows `""` even though a specific provider was used.+- `follow_redirects` is a declared field but never forwarded to `httpx`; httpx default follows redirects.+- `provider_name` in the result echoes `params.proxy_provider or ""` verbatim; with the default `auto` it shows `"auto"` even though `ProxyService` picked a concrete provider.- Retry loop re-requests a new proxy URL via `get_proxy_url` on each retry; if that call raises (e.g. `BudgetExceededError`), the retry loop silently `break`s and the envelope reports the previous network error rather than the budget exception.- Cost tracking reads `pricing._config["proxy"]` directly - touching a private attribute. If the pricing config is missing a `proxy` section, `cost_per_gb` falls back to `0.0` and cost rows are still written with `cost=0`.diff --git a/docs-internal/node-logic-flows/http_proxy/proxyStatus.md b/docs-internal/node-logic-flows/http_proxy/proxyStatus.mdindex c04fa10e..6243611f 100644--- a/docs-internal/node-logic-flows/http_proxy/proxyStatus.md+++ b/docs-internal/node-logic-flows/http_proxy/proxyStatus.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | proxy / tool |-| **Backend handler** | [`server/services/handlers/proxy.py::handle_proxy_status`](../../../server/services/handlers/proxy.py) |+| **Backend handler** | [`server/nodes/proxy/proxy_status/__init__.py`](../../../server/nodes/proxy/proxy_status/__init__.py) — `ProxyStatusNode.status` (dispatched via `BaseNode.execute()` + `@Operation("status")`; the legacy `handlers/proxy.py` was deleted in Wave 11.D.3) || **Tests** | [`server/tests/nodes/test_http_proxy.py`](../../../server/tests/nodes/test_http_proxy.py) || **Skill (if any)** | [`server/skills/web_agent/proxy-config-skill/SKILL.md`](../../../server/skills/web_agent/proxy-config-skill/SKILL.md) |-| **Dual-purpose tool** | yes |+| **Dual-purpose tool** | yes - tool name `proxy_status` (`readonly = True`, `usable_as_tool = True`) |## Purpose@@ -25,7 +25,7 @@ provider to target.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `providerFilter` | string | `""` | no | - | Declared in frontend; handler currently **ignores** it and returns all providers |+| `provider_name` | string | `""` | no | - | Declared field; the op currently **ignores** it and always returns all providers |## Outputs (handles)@@ -51,20 +51,19 @@ provider to target.```mermaidflowchart TD- A[handle_proxy_status] --> B{proxy_svc and<br/>proxy_svc.is_enabled?}- B -- no --> C[Return success=true<br/>result = enabled:false providers:[] stats:{}]- B -- yes --> D[stats = proxy_svc.get_stats]- D --> E[providers = proxy_svc.get_providers -> model_dump each]- E --> F[Return success=true<br/>result = enabled:true providers stats]- A -- Exception --> Eerr[Return success=false<br/>error: str(e)]+ A[ProxyStatusNode.status] --> B{proxy_svc and<br/>proxy_svc.is_enabled?}+ B -- no --> C[return enabled:false providers:[] stats:{}]+ B -- yes --> D[providers = proxy_svc.get_providers -> model_dump each]+ D --> E[stats = proxy_svc.get_stats]+ E --> F[return enabled:true providers stats]```## Decision Logic- **Validation**: none - no required params.-- **Branches**: service-disabled short-circuit returns `enabled=false` with empty collections; still `success=true`.-- **Fallbacks**: `providerFilter` is ignored - the handler always returns the full list.-- **Error paths**: only the outer `Exception` catch; returns `{success: false, error: str(e)}`.+- **Branches**: service-disabled short-circuit returns `enabled=false` with empty collections (still a success envelope from `BaseNode.execute()`).+- **Fallbacks**: `provider_name` is ignored - the op always returns the full list.+- **Error paths**: the op has no try/except; any exception propagates to `BaseNode.execute()`, which produces the error envelope.## Side Effects@@ -84,8 +83,8 @@ flowchart TD## Edge cases & known limits-- Even when proxy is disabled, the handler returns `success=true` with empty collections - callers need to inspect `result.enabled` rather than `success`.-- `providerFilter` is defined in the frontend UI but never applied in the backend; filtering must happen in the consumer.+- Even when proxy is disabled, the op returns a success envelope with `enabled=false` and empty collections - callers need to inspect `result.enabled` rather than `success`.+- `provider_name` is a declared field but never applied in the op; filtering must happen in the consumer.- `ProviderStats.model_dump()` exposes every field including computed `score`; the shape is governed by `services/proxy/models.py::ProviderStats`.- No locking around the read: if another request is mutating `ProxyService._providers` (e.g. `reload_providers` mid-flight), the returned list may reflect a partial state.diff --git a/docs-internal/node-logic-flows/search/braveSearch.md b/docs-internal/node-logic-flows/search/braveSearch.mdindex 9ac04cd2..04419c16 100644--- a/docs-internal/node-logic-flows/search/braveSearch.md+++ b/docs-internal/node-logic-flows/search/braveSearch.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | search / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/search.py::handle_brave_search`](../../../server/services/handlers/search.py) |+| **Backend handler** | [`server/nodes/search/brave_search/__init__.py`](../../../server/nodes/search/brave_search/__init__.py) - `BraveSearchNode`; dispatched via `BaseNode.execute()` + the `@Operation("search")` method (Wave 11; the old `server/services/handlers/search.py::handle_brave_search` is deleted) || **Tests** | [`server/tests/nodes/test_search.py`](../../../server/tests/nodes/test_search.py) || **Skill (if any)** | [`server/skills/web_agent/brave-search-skill/SKILL.md`](../../../server/skills/web_agent/brave-search-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `brave_search` |+| **Dual-purpose tool** | yes - tool name `brave_search` (`usable_as_tool = True`) |## Purpose@@ -23,25 +23,28 @@ the LLM fills the same parameter schema documented below.## Parameters+(Pydantic `BraveSearchParams`, `model_config = {"extra": "ignore"}`.)+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `brave_search` | no | - | Name of this node when exposed as an AI tool |-| `toolDescription` | string | (see frontend) | no | - | Description shown to LLM in tool schema |-| `query` | string | `""` | **yes** | - | Search query |-| `maxResults` | number | `10` | no | - | 1-100; clamped via `min(maxResults, 100)` before API call |+| `tool_name` | string | `brave_search` | no | - | Name of this node when exposed as an AI tool |+| `tool_description` | string | (see Params) | no | - | Description shown to LLM in tool schema |+| `query` | string | (none) | **yes** | - | Search query (`min_length=1`) |+| `max_results` | int | `10` | no | - | `ge=1, le=20`; clamped via `min(max_results, 20)` before API call || `country` | string | `""` | no | - | ISO country code, e.g. `US` - only sent when truthy |-| `searchLang` | string | `""` | no | - | ISO language code, e.g. `en` - only sent when truthy |-| `safeSearch` | options | `moderate` | no | - | One of `off` / `moderate` / `strict` |+| `search_lang` | string | `"en"` | no | - | ISO language code - only sent when truthy |+| `safe_search` | enum | `moderate` | no | - | One of `off` / `moderate` / `strict` |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard search payload (see below) |-| `output-tool` | object | Same payload, used when this node is wired to an AI agent's `input-tools` |+| `output-main` | object | Standard search payload (see below). When wired to an AI agent's `input-tools`, the same payload is returned to the LLM. |### Output payload+`BraveSearchOutput`:+```ts{query: string;@@ -51,56 +54,53 @@ the LLM fills the same parameter schema documented below.}```-Wrapped in the standard envelope: `{ success: true, result: <payload>, execution_time: number }`.+Wrapped by `BaseNode._serialize_result` in the standard envelope: `{ success: true, result: <payload>, ... }`. Shared runtime schema: `SearchOutput` in [`server/services/node_output_schemas.py`](../../../server/services/node_output_schemas.py).## Logic Flow```mermaidflowchart TD- A[handle_brave_search] --> B{query stripped<br/>non-empty?}- B -- no --> Eret[Return success=false<br/>error: Search query is required]- B -- yes --> C[Read maxResults / country / searchLang / safeSearch from params]- C --> D[await _get_api_key brave_search]- D -- ValueError --> Eapi[Return success=false<br/>error: API key not configured]- D -- ok --> E[Build params dict; only include country/searchLang/safeSearch if truthy]- E --> F[GET api.search.brave.com/res/v1/web/search<br/>headers X-Subscription-Token + Accept: application/json<br/>timeout 30s]- F -- HTTPStatusError --> Estat[Return success=false<br/>error: Brave Search API error STATUS]- F -- other Exception --> Egen[Return success=false<br/>error: Brave Search failed STR]- F -- 2xx --> G[data.web.results -> map first maxResults to title/snippet/url]- G --> H[await _track_search_usage<br/>service=brave_search action=web_search count=1]- H --> I[Return success=true with query / results / result_count / provider]+ A[BaseNode.execute -> BraveSearchNode.search] --> B{Pydantic validates params}+ B -- query missing / max_results out of range --> Eret[NodeUserError -> success=false<br/>error: invalid parameters]+ B -- ok --> C[Build qs dict: q + count=min max_results,20<br/>add country/search_lang/safesearch when truthy]+ C --> D[ctx.connection 'brave_search'<br/>injects X-Subscription-Token header]+ D --> F[GET api.search.brave.com/res/v1/web/search]+ F -- raise_for_status fails --> Estat[Exception -> success=false envelope]+ F -- 2xx --> G[data.web.results -> map first max_results to title/snippet/url]+ G --> H[Operation cost spec: service=brave_search action=web_search count=1]+ H --> I[Return BraveSearchOutput]```## Decision Logic-- **Empty query**: `query.strip() == ""` -> immediate failure envelope, no API call.-- **Param trimming**: `country`, `searchLang`, `safeSearch` only added to the request when truthy. Note: `safeSearch` defaults to `'moderate'` in the frontend so it is effectively always sent.-- **maxResults clamp**: API receives `min(maxResults, 100)` regardless of the requested value; the response is then sliced to the original `maxResults` again before mapping.+- **Validation**: handled by Pydantic on `BraveSearchParams`. Empty / missing `query` (`min_length=1`) and out-of-range `max_results` (`le=20`) are rejected before the operation body runs; `BaseNode.execute` surfaces an `invalid parameters` failure envelope.+- **Param trimming**: `country`, `search_lang`, `safe_search` only added to the request querystring when truthy. `search_lang` defaults to `'en'` and `safe_search` defaults to `'moderate'`, so both are effectively always sent.+- **max_results clamp**: API receives `min(max_results, 20)`; the response list is then sliced to `max_results` before mapping.- **Result mapping**: missing fields fall back to empty string (`title`, `snippet`, `url`).## Side Effects-- **Database writes**: one row in `api_usage_metrics` (via `database.save_api_usage_metric`) with `service='brave_search'`, `operation` from PricingService, `cost` from PricingService, plus `session_id` / `node_id` / `workflow_id`.-- **Broadcasts**: none.-- **External API calls**: `GET https://api.search.brave.com/res/v1/web/search` (timeout 30s).+- **Database writes**: one row in `api_usage_metrics` via the framework's cost-tracking on the `@Operation("search", cost={...})` spec (`service='brave_search'`, `action='web_search'`, `count=1`).+- **Broadcasts**: per-node status via `BaseNode.execute` (`update_node_status` executing/success/error); no plugin-specific broadcasts.+- **External API calls**: `GET https://api.search.brave.com/res/v1/web/search` via `ctx.connection`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: `auth_service.get_api_key('brave_search')` - looked up at execution time; stored under `EncryptedAPIKey` table.-- **Services**: `PricingService` for cost calc, `Database` for usage row.-- **Python packages**: `httpx`.+- **Credentials**: `BraveSearchCredential` (`ApiKeyCredential`, id `brave_search`, header `X-Subscription-Token`) - resolved by the Connection facade at execution time; stored in `EncryptedAPIKey`.+- **Services**: `ctx.connection` (Connection facade), framework cost tracking.+- **Python packages**: `httpx` (via Connection facade).- **Environment variables**: none.## Edge cases & known limits-- API hard-caps `count` at 100; values above 100 are silently capped.-- Network errors are caught by the generic `Exception` handler and returned as `Brave Search failed: <str>` - i.e. the handler never raises; downstream nodes always see an envelope.-- The handler does *not* validate `safeSearch` values - the API rejects unknown values with 4xx, which becomes the `Brave Search API error: 400` envelope.+- `max_results` is hard-bounded at 20 by Pydantic (`le=20`); values above 20 fail validation rather than being silently capped (test `test_clamps_max_results_to_100` asserts `success=false`).+- `response.raise_for_status()` propagates non-2xx as an exception; `BaseNode.execute` converts it into a failure envelope, so downstream nodes always see an envelope.+- The handler does not validate `safe_search` enum at runtime beyond Pydantic - the API rejects unknown values with 4xx.## Related- **Skills using this as a tool**: [`brave-search-skill/SKILL.md`](../../../server/skills/web_agent/brave-search-skill/SKILL.md)-- **Companion nodes**: [`serperSearch`](./serperSearch.md), [`perplexitySearch`](./perplexitySearch.md), [`duckduckgoSearch`](../tools/duckduckgoSearch.md)-- **Architecture docs**: [Pricing Service](../../pricing_service.md), [Credentials Encryption](../../credentials_encryption.md)+- **Companion nodes**: [`serperSearch`](./serperSearch.md), [`perplexitySearch`](./perplexitySearch.md), `duckduckgoSearch` (tool category)+- **Architecture docs**: [Plugin System](../../plugin_system.md), [Pricing Service](../../pricing_service.md), [Credentials Encryption](../../credentials_encryption.md)diff --git a/docs-internal/node-logic-flows/search/perplexitySearch.md b/docs-internal/node-logic-flows/search/perplexitySearch.mdindex 986319ab..2e467ad9 100644--- a/docs-internal/node-logic-flows/search/perplexitySearch.md+++ b/docs-internal/node-logic-flows/search/perplexitySearch.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | search / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/search.py::handle_perplexity_search`](../../../server/services/handlers/search.py) |+| **Backend handler** | [`server/nodes/search/perplexity_search/__init__.py`](../../../server/nodes/search/perplexity_search/__init__.py) - `PerplexitySearchNode`; dispatched via `BaseNode.execute()` + the `@Operation("search")` method (Wave 11; the old `server/services/handlers/search.py::handle_perplexity_search` is deleted) || **Tests** | [`server/tests/nodes/test_search.py`](../../../server/tests/nodes/test_search.py) || **Skill (if any)** | [`server/skills/web_agent/perplexity-search-skill/SKILL.md`](../../../server/skills/web_agent/perplexity-search-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `perplexity_search` |+| **Dual-purpose tool** | yes - tool name `perplexity_search` (`usable_as_tool = True`) |## Purpose@@ -22,98 +22,99 @@ downstream nodes can render references or follow the sources.## Parameters+(Pydantic `PerplexitySearchParams`, `model_config = {"extra": "ignore"}`.)+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `perplexity_search` | no | - | Tool name when exposed to AI |-| `toolDescription` | string | (see frontend) | no | - | Tool description for AI |-| `query` | string | `""` | **yes** | - | Question for the model |-| `model` | options | `sonar` | no | - | One of `sonar` / `sonar-pro` / `sonar-reasoning` / `sonar-reasoning-pro` |-| `searchRecencyFilter` | options | `""` | no | - | One of `""` / `month` / `week` / `day` / `hour` |-| `returnImages` | boolean | `false` | no | - | When true, include images array in payload |-| `returnRelatedQuestions` | boolean | `false` | no | - | When true, include related_questions array |+| `tool_name` | string | `perplexity_search` | no | - | Tool name when exposed to AI |+| `tool_description` | string | (see Params) | no | - | Tool description for AI |+| `query` | string | (none) | **yes** | - | Question for the model (`min_length=1`) |+| `model` | enum | `sonar` | no | - | One of `sonar` / `sonar-pro` / `sonar-reasoning` / `sonar-reasoning-pro` |+| `search_recency_filter` | enum | `all` | no | - | One of `all` / `month` / `week` / `day` / `hour` (`all` = off, not sent) |+| `return_images` | boolean | `false` | no | - | When true, request images from the API |+| `return_related_questions` | boolean | `false` | no | - | When true, request related questions from the API |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Answer + citations payload |-| `output-tool` | object | Same payload, AI-tool wiring |### Output payload+`PerplexitySearchOutput`:+```ts{query: string;answer: string; // markdown content from choices[0].message.contentcitations: string[]; // URLs the model citedresults: Array<{ url: string }>; // citations remapped as result objects+ images: dict[] | null; // data.images verbatim (null when API omits it)+ related_questions: string[] | null; // data.related_questions verbatim (null when API omits it)model: string; // echoes the model paramprovider: 'perplexity';- images?: string[]; // present only if requested AND returned- related_questions?: string[]; // present only if requested AND returned}```-Wrapped in the standard envelope.+Wrapped by `BaseNode._serialize_result` in the standard envelope. Shared runtime schema: `SearchOutput` in [`server/services/node_output_schemas.py`](../../../server/services/node_output_schemas.py).## Logic Flow```mermaidflowchart TD- A[handle_perplexity_search] --> B{query stripped<br/>non-empty?}- B -- no --> Eret[Return error: Search query is required]- B -- yes --> C[Read model, searchRecencyFilter, returnImages,<br/>returnRelatedQuestions]- C --> D[await _get_api_key 'perplexity']- D -- ValueError --> Eapi[Return error: API key not configured]- D -- ok --> E[Build body:<br/>model + messages role=user content=query]- E --> F{searchRecencyFilter truthy?}- F -- yes --> G[Add search_recency_filter to body]- F -- no --> H{returnImages?}+ A[BaseNode.execute -> PerplexitySearchNode.search] --> B{Pydantic validates params}+ B -- query missing --> Eret[NodeUserError -> success=false<br/>error: invalid parameters]+ B -- ok --> C[Build body: model + messages role=user content=query]+ C --> D{search_recency_filter != 'all'?}+ D -- yes --> E[Add search_recency_filter to body]+ D -- no --> F{return_images?}+ E --> F+ F -- yes --> G[body.return_images=true]+ F --> H{return_related_questions?}G --> H- H -- yes --> I[body.return_images=true]- H -- no --> J{returnRelatedQuestions?}+ H -- yes --> I[body.return_related_questions=true]+ H --> J[ctx.connection 'perplexity'<br/>injects Authorization: Bearer]I --> J- J -- yes --> K[body.return_related_questions=true]- J -- no --> L[POST api.perplexity.ai/chat/completions<br/>headers Authorization Bearer + JSON<br/>timeout 60s]- K --> L- L -- HTTPStatusError --> Estat[Return error: Perplexity API error STATUS]- L -- other Exception --> Egen[Return error: Perplexity search failed STR]- L -- 2xx --> M[answer = data.choices 0 .message.content or empty]- M --> N[citations = data.citations or empty list]- N --> O[results = list of url-only objects]- O --> P[await _track_search_usage 'perplexity' 'sonar_search' 1]- P --> Q[Return success with query / answer / citations / results / model / provider<br/>plus images and related_questions when present]+ J --> K[POST api.perplexity.ai/chat/completions]+ K -- raise_for_status fails --> Estat[Exception -> success=false envelope]+ K -- 2xx --> L[answer = data.choices 0 .message.content or empty]+ L --> M[citations = data.citations or empty list]+ M --> N[results = list of url-only objects]+ N --> O[images/related_questions = data.get verbatim]+ O --> P[Operation cost spec: service=perplexity action=sonar_search count=1]+ P --> Q[Return PerplexitySearchOutput]```## Decision Logic-- **Empty query**: short-circuit failure envelope.-- **Optional body fields** (`search_recency_filter`, `return_images`, `return_related_questions`) only added when the corresponding param is truthy. `searchRecencyFilter=''` is the explicit "off" value.+- **Validation**: Pydantic on `PerplexitySearchParams`. Empty / missing `query` (`min_length=1`) is rejected before the operation body (`invalid parameters` failure envelope).+- **Optional body fields**: `search_recency_filter` is added only when `!= 'all'`; `return_images` / `return_related_questions` are added only when truthy. `search_recency_filter='all'` is the explicit "off" value.- **Empty `choices`**: `answer` falls back to `''` rather than raising.-- **Optional response fields** (`images`, `related_questions`) only added to the output payload when present *and non-empty* (`if images:` / `if related_questions:`).+- **Optional response fields**: `images` / `related_questions` are set verbatim from `data.get(...)` - they are `None` when the API omits them (no `if x:` gating; this differs from the deleted pre-Wave-11 handler which only included them when non-empty).## Side Effects-- **Database writes**: one `api_usage_metrics` row per call (`service='perplexity'`, `operation='sonar_search'`).-- **Broadcasts**: none.-- **External API calls**: `POST https://api.perplexity.ai/chat/completions` (timeout 60s - longer than the other search nodes because Sonar Reasoning models can be slow).+- **Database writes**: one `api_usage_metrics` row per call via the `@Operation` cost spec (`service='perplexity'`, `action='sonar_search'`, `count=1`).+- **Broadcasts**: per-node status via `BaseNode.execute`; no plugin-specific broadcasts.+- **External API calls**: `POST https://api.perplexity.ai/chat/completions` via `ctx.connection`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: `auth_service.get_api_key('perplexity')`.-- **Services**: `PricingService`, `Database`.-- **Python packages**: `httpx`.+- **Credentials**: `PerplexityCredential` (`ApiKeyCredential`, id `perplexity`, bearer auth). Its `_probe` short-circuits to `ProbeResult(valid=True)` - Perplexity has no cheap auth-gated validation endpoint, so a bad key only surfaces as a runtime 401.+- **Services**: `ctx.connection`, framework cost tracking.+- **Python packages**: `httpx` (via Connection facade).## Edge cases & known limits-- 60s timeout - reasoning-pro models can occasionally exceed this and surface as `Perplexity search failed: ReadTimeout`.-- Citations field shape can vary across Perplexity response versions (`list[str]` historically). The handler assumes `list[str]`; a list-of-objects response will pass through to the output unchanged but break the `results` mapping (`{'url': <obj>}`).+- Sonar Reasoning models can be slow; the Connection facade's default timeout governs (no per-call override in the plugin body, unlike the old handler's explicit 60s).+- Citations field shape can vary across Perplexity response versions (`list[str]` historically). The plugin assumes `list[str]`; a list-of-objects response passes through to the output unchanged but breaks the `results` mapping (`{'url': <obj>}`).- The `results` list is purely a citations remap; downstream nodes expecting `title` / `snippet` will not find them - use `braveSearch` or `serperSearch` for those.## Related- **Skills using this as a tool**: [`perplexity-search-skill/SKILL.md`](../../../server/skills/web_agent/perplexity-search-skill/SKILL.md)- **Companion nodes**: [`braveSearch`](./braveSearch.md), [`serperSearch`](./serperSearch.md)-- **Architecture docs**: [Pricing Service](../../pricing_service.md)+- **Architecture docs**: [Plugin System](../../plugin_system.md), [Pricing Service](../../pricing_service.md)diff --git a/docs-internal/node-logic-flows/search/serperSearch.md b/docs-internal/node-logic-flows/search/serperSearch.mdindex 93dac436..fc2e399c 100644--- a/docs-internal/node-logic-flows/search/serperSearch.md+++ b/docs-internal/node-logic-flows/search/serperSearch.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | search / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/search.py::handle_serper_search`](../../../server/services/handlers/search.py) |+| **Backend handler** | [`server/nodes/search/serper_search/__init__.py`](../../../server/nodes/search/serper_search/__init__.py) - `SerperSearchNode`; dispatched via `BaseNode.execute()` + the `@Operation("search")` method (Wave 11; the old `server/services/handlers/search.py::handle_serper_search` is deleted) || **Tests** | [`server/tests/nodes/test_search.py`](../../../server/tests/nodes/test_search.py) || **Skill (if any)** | [`server/skills/web_agent/serper-search-skill/SKILL.md`](../../../server/skills/web_agent/serper-search-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `serper_search` |+| **Dual-purpose tool** | yes - tool name `serper_search` (`usable_as_tool = True`) |## Purpose@@ -22,100 +22,104 @@ default web search. Used as a workflow node and as an AI agent tool.## Parameters+(Pydantic `SerperSearchParams`, `model_config = {"extra": "ignore"}`.)+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `serper_search` | no | - | Tool name when exposed to AI |-| `toolDescription` | string | (see frontend) | no | - | Tool description for AI |-| `query` | string | `""` | **yes** | - | Search query |-| `searchType` | options | `search` | no | - | One of `search` / `news` / `images` / `places` |-| `maxResults` | number | `10` | no | - | 1-100; clamped via `min(maxResults, 100)` before API call |+| `tool_name` | string | `serper_search` | no | - | Tool name when exposed to AI |+| `tool_description` | string | (see Params) | no | - | Tool description for AI |+| `query` | string | (none) | **yes** | - | Search query (`min_length=1`) |+| `search_type` | enum | `search` | no | - | One of `search` / `news` / `images` / `places` |+| `max_results` | int | `10` | no | - | `ge=1, le=100`; clamped via `min(max_results, 100)` before API call || `country` | string | `""` | no | - | Sent as `gl` only when truthy |-| `language` | string | `""` | no | - | Sent as `hl` only when truthy |+| `language` | string | `"en"` | no | - | Sent as `hl` only when truthy |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Search payload (shape varies by `searchType`) |-| `output-tool` | object | Same payload, AI-tool wiring |+| `output-main` | object | Search payload (`results` field shape varies by `search_type`). |### Output payload+`SerperSearchOutput`. Every result is the unified `SerperSearchResult` model+(`title` / `snippet` / `url` / `position?`); fields not populated for a given+vertical are left at their defaults (empty string / null).+```ts{query: string;- results: Array<SearchResult | NewsResult | ImageResult | PlaceResult>;+ results: Array<{ title: string; snippet: string; url: string; position: number | null }>;result_count: number;search_type: 'search' | 'news' | 'images' | 'places';+ knowledge_graph: object | null; // only set when API returns knowledgeGraphprovider: 'serper';- knowledge_graph?: object; // only when API returns it}--type SearchResult = { title: string; snippet: string; url: string; position?: number };-type NewsResult = { title: string; snippet: string; url: string; date: string; source: string };-type ImageResult = { title: string; imageUrl: string; url: string };-type PlaceResult = { title: string; address: string; rating?: number; url: string };```-Wrapped in the standard envelope.+Per vertical (mapping in the operation body):+- `search`: `title`, `snippet`, `url` (from `organic[].link`), `position`.+- `news`: `title`, `snippet`, `url` (from `news[].link`). No `position`.+- `images`: `title`, `url` (from `images[].imageUrl` or `link`). No `snippet`.+- `places`: `title`, `url` (from `places[].website`). No `snippet`.++Wrapped by `BaseNode._serialize_result` in the standard envelope. Shared runtime schema: `SearchOutput` in [`server/services/node_output_schemas.py`](../../../server/services/node_output_schemas.py).## Logic Flow```mermaidflowchart TD- A[handle_serper_search] --> B{query stripped non-empty?}- B -- no --> Eret[Return error: Search query is required]- B -- yes --> C[Read maxResults, searchType, country, language]- C --> D[await _get_api_key 'serper']- D -- ValueError --> Eapi[Return error: API key not configured]- D -- ok --> E[Build body q + num; add gl/hl when truthy]- E --> F[Pick endpoint by searchType<br/>fallback to /search]- F --> G[POST endpoint headers X-API-KEY + Content-Type: application/json<br/>timeout 30s]- G -- HTTPStatusError --> Estat[Return error: Serper API error STATUS]- G -- other Exception --> Egen[Return error: Serper search failed STR]- G -- 2xx --> H{searchType branch}+ A[BaseNode.execute -> SerperSearchNode.search] --> B{Pydantic validates params}+ B -- query missing / max_results out of range --> Eret[NodeUserError -> success=false<br/>error: invalid parameters]+ B -- ok --> C[Build body: q + num=min max_results,100<br/>add gl/hl when truthy]+ C --> F[Pick endpoint by search_type<br/>fallback to /search]+ F --> D[ctx.connection 'serper'<br/>injects X-API-KEY header]+ D --> G[POST endpoint]+ G -- raise_for_status fails --> Estat[Exception -> success=false envelope]+ G -- 2xx --> H{search_type branch}H -- search --> J1[Map data.organic -> title/snippet/url/position]- H -- news --> J2[Map data.news -> title/snippet/url/date/source]- H -- images --> J3[Map data.images -> title/imageUrl/url]- H -- places --> J4[Map data.places -> title/address/rating/url]+ H -- news --> J2[Map data.news -> title/snippet/url]+ H -- images --> J3[Map data.images -> title/url]+ H -- places --> J4[Map data.places -> title/url]J1 --> K[knowledge_graph = data.knowledgeGraph or None]J2 --> KJ3 --> KJ4 --> K- K --> L[await _track_search_usage 'serper' 'web_search' 1]- L --> M[Return success with query results result_count search_type provider plus knowledge_graph if present]+ K --> L[Operation cost spec: service=serper action=web_search count=1]+ L --> M[Return SerperSearchOutput]```## Decision Logic-- **Empty query**: short-circuit failure envelope.-- **searchType dispatch**: hard-coded `endpoint_map`; unknown values fall through to `https://google.serper.dev/search` *and* take the `if search_type == 'search'` branch... wait no - they hit the default endpoint but skip every branch in the result loop, returning `results: []`. Document this as a known gotcha.-- **knowledge_graph**: only added to the payload when the API includes a non-falsy `knowledgeGraph` field.-- **maxResults clamp**: API receives `min(maxResults, 100)`; results list is then sliced again to `maxResults`.-- **gl/hl trimming**: `country` -> `gl`, `language` -> `hl`. Only sent when truthy.+- **Validation**: Pydantic on `SerperSearchParams`. Empty / missing `query` and out-of-range `max_results` are rejected before the operation body (`invalid parameters` failure envelope).+- **search_type dispatch**: `_ENDPOINTS` map keyed by `search_type`; unknown values fall back to `https://google.serper.dev/search` for the request BUT match no result-mapping branch, so they return `results: []` (known gotcha; locked by `test_unknown_search_type_falls_back_to_web_endpoint_but_returns_empty_results`). Note `search_type` is a `Literal`, so frontend/agent inputs are constrained; an out-of-Literal value only reaches the body via the AI-tool path if validation is bypassed.+- **knowledge_graph**: only set when the API includes a non-falsy `knowledgeGraph` field.+- **max_results clamp**: API receives `min(max_results, 100)`; each branch slices its source list to `max_results`.+- **gl/hl trimming**: `country` -> `gl`, `language` -> `hl`. Only sent when truthy (`language` defaults to `'en'`).## Side Effects-- **Database writes**: one `api_usage_metrics` row per call (`service='serper'`, `operation` from PricingService).-- **Broadcasts**: none.-- **External API calls**: `POST` to one of `https://google.serper.dev/{search,news,images,places}` (timeout 30s).+- **Database writes**: one `api_usage_metrics` row per call via the `@Operation` cost spec (`service='serper'`, `action='web_search'`, `count=1`).+- **Broadcasts**: per-node status via `BaseNode.execute`; no plugin-specific broadcasts.+- **External API calls**: `POST` to one of `https://google.serper.dev/{search,news,images,places}` via `ctx.connection`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies-- **Credentials**: `auth_service.get_api_key('serper')`.-- **Services**: `PricingService`, `Database`.-- **Python packages**: `httpx`.+- **Credentials**: `SerperCredential` (`ApiKeyCredential`, id `serper`, header `X-API-KEY`) - resolved by the Connection facade.+- **Services**: `ctx.connection`, framework cost tracking.+- **Python packages**: `httpx` (via Connection facade).## Edge cases & known limits-- An unrecognised `searchType` value silently returns `results: []` (no error). Refactors must preserve this or update the test to assert the new behaviour.-- 4xx/5xx API errors are translated into `Serper API error: <status>` - the response body is logged but not exposed.+- An unrecognised `search_type` value returns `results: []` (no error). Refactors must preserve this or update the contract test.+- `response.raise_for_status()` propagates non-2xx as an exception; `BaseNode.execute` converts it into a failure envelope.- The handler trusts the API to return the expected nested keys (`organic`, `news`, etc.); a malformed response yields an empty results list rather than an exception.+- Result models are unified - `news`/`images`/`places` do NOT carry `date` / `source` / `address` / `rating` fields; only `title` / `snippet` / `url` / `position` exist.## Related- **Skills using this as a tool**: [`serper-search-skill/SKILL.md`](../../../server/skills/web_agent/serper-search-skill/SKILL.md)- **Companion nodes**: [`braveSearch`](./braveSearch.md), [`perplexitySearch`](./perplexitySearch.md)-- **Architecture docs**: [Pricing Service](../../pricing_service.md)+- **Architecture docs**: [Plugin System](../../plugin_system.md), [Pricing Service](../../pricing_service.md)diff --git a/docs-internal/node-logic-flows/specialized_agents/_pattern.md b/docs-internal/node-logic-flows/specialized_agents/_pattern.mdindex b077c7e4..0547ad16 100644--- a/docs-internal/node-logic-flows/specialized_agents/_pattern.md+++ b/docs-internal/node-logic-flows/specialized_agents/_pattern.md@@ -1,23 +1,30 @@-# Generic Specialized Agent Pattern (`handle_chat_agent`)+# Generic Specialized Agent Pattern (`SpecializedAgentBase`)This document describes the shared behavioural contract for the 13 specialized-agent nodes that route to the same backend handler:+agent nodes that share one plugin base class:`android_agent`, `coding_agent`, `web_agent`, `task_agent`, `social_agent`,`travel_agent`, `tool_agent`, `productivity_agent`, `payments_agent`,`consumer_agent`, `autonomous_agent`, `orchestrator_agent`, `ai_employee`.-All specialized nodes use the `handle_chat_agent` handler wired in-`server/services/node_executor.py`. They differ only in frontend-presentation (icon, theme color, default skills) and intended use case.-The two agents with **dedicated** handlers (`rlm_agent`,+Post-Wave-11 there is no `handle_chat_agent` handler. Each agent is a+self-contained plugin folder under `server/nodes/agent/<agent>/__init__.py`+declaring a `SpecializedAgentBase` subclass; the body lives in+[`server/nodes/agent/_specialized.py`](../../../server/nodes/agent/_specialized.py).+Dispatch is via `BaseNode.execute()` -> the `@Operation("execute")` method+(`SpecializedAgentBase.execute_op`), which calls+[`prepare_agent_call`](../../../server/nodes/agent/_inline.py) for the shared+pre-dispatch flow then `AIService.execute_chat_agent`. The agents differ only+in frontend presentation (icon, color, subtitle) and intended use case. The+two agents that DON'T use `execute_chat_agent` (`rlm_agent`,`claude_code_agent`) are documented separately.| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |-| **Registry binding** | `partial(handle_chat_agent, ai_service=..., database=...)` |+| **Plugin** | [`server/nodes/agent/<agent>/__init__.py`](../../../server/nodes/agent/) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) |+| **Dispatch** | `BaseNode.execute()` -> `@Operation("execute")` -> `prepare_agent_call` -> `AIService.execute_chat_agent` |+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) (5-tuple: memory, skill, tool, input, task) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) || **Architecture docs** | [Agent Architecture](../../agent_architecture.md), [Agent Delegation](../../agent_delegation.md), [Agent Teams](../../agent_teams.md) |@@ -25,15 +32,17 @@ The two agents with **dedicated** handlers (`rlm_agent`,A specialized agent is an `aiAgent` pre-configured for a specific domain(Android control, coding, web automation, etc.). From the backend's-perspective it is identical to `chatAgent`: same handler, same inputs, same-output envelope, same delegation semantics. The specialization is conveyed-only through:+perspective it is identical to `chatAgent`: same `execute_chat_agent` path,+same inputs, same output envelope, same delegation semantics. The+specialization is conveyed only through:1. Connected skills (usually via a Master Skill node seeded with thedomain-specific folder in `server/skills/`).2. Connected tools (e.g. `coding_agent` typically has `pythonExecutor` wiredto `input-tools`).-3. Frontend cosmetics (icon, theme color, AGENT_CONFIGS label).+3. Frontend cosmetics (icon + color from the plugin folder / `visuals.json`,+ plus `display_name` / `subtitle` on the plugin class). The old+ `AGENT_CONFIGS` map was retired -- `AIAgentNode.tsx` reads the NodeSpec.## Inputs (handles)@@ -41,30 +50,33 @@ All 13 agents share these input handles.| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| `input-main` | main | no | Upstream data; auto-prompt fallback when `prompt` param is empty |-| `input-skill` | main | no | Skill node(s) providing SKILL.md context |-| `input-memory` | main | no | `simpleMemory` node for conversation history |-| `input-tools` | main | no | Tool nodes exposed to the LLM via `chat_model.bind_tools` |-| `input-task` | main | no | `taskTrigger` events from delegated child agents |-| `input-teammates` | main | no | **Only on `orchestrator_agent` and `ai_employee`** -- connected agents become `delegate_to_*` tools |+| `input-main` | input/main | no | Upstream data; auto-prompt fallback when `prompt` param is empty |+| `input-skill` | input/skill | no | Skill node(s) providing SKILL.md context |+| `input-memory` | input/memory | no | `simpleMemory` node for conversation history |+| `input-tools` | input/tools | no | Tool nodes exposed to the LLM via `chat_model.bind_tools` |+| `input-task` | input/task | no | `taskTrigger` events from delegated child agents |+| `input-teammates` | input/teammates | no | **Only on `orchestrator_agent` and `ai_employee`** (via `team_lead_agent_handles()`) -- connected agents become `delegate_to_*` tools |++Handle topology comes from `std_agent_handles()` (or `team_lead_agent_handles()`+for the two team leads) in+[`server/nodes/agent/_handles.py`](../../../server/nodes/agent/_handles.py).## Parameters-All specialized agents share the `AI_AGENT_PROPERTIES` schema from-`specializedAgentNodes.ts` plus node-specific extras (e.g. orchestrator has-`teamMode`, `ai_employee` has `teamMode` + `maxConcurrent`).--| Name | Type | Default | Description |-|------|------|---------|-------------|-| `provider` | options | `openai` | AI provider (openai / anthropic / gemini / ...) |-| `model` | string | `""` | Model ID; required |-| `prompt` | string | `""` | User prompt; falls back to upstream output when empty |-| `systemMessage` | string | `""` | Additional system instructions (prepended to skill content) |-| `options.temperature` | number | `0.7` | Sampling temperature |-| `options.maxTokens` | number | `4096` | Max response tokens |-| `options.thinkingEnabled` | boolean | `false` | Extended thinking / reasoning |-| `options.thinkingBudget` | number | `2048` | Thinking budget tokens (Claude/Gemini) |-| `options.reasoningEffort` | options | `medium` | For OpenAI o-series and GPT-5 |+All 13 specialized agents share the `SpecializedAgentParams` Pydantic model in+[`server/nodes/agent/_specialized.py`](../../../server/nodes/agent/_specialized.py).+There are **no** per-agent parameter extras: `orchestrator_agent` and+`ai_employee` reuse the same `Params` (they override only `handles` ++`tool_description`), so there is no `teamMode` / `maxConcurrent` field.++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `prompt` | string | `""` | no | - | User prompt; falls back to upstream output when empty (4 rows) |+| `provider` | enum | `openai` | no | - | `openai` / `anthropic` / `gemini` / `openrouter` / `groq` / `cerebras` / `deepseek` / `kimi` / `mistral` / `ollama` / `lmstudio` |+| `model` | string | `""` | no | - | Model ID (resolved against the provider; empty falls through to default) |+| `system_message` | string\|null | `"You are a helpful assistant"` | no | - | System instructions (3 rows) |+| `temperature` | float\|null | `None` | no | group `options` | 0.0-2.0; `None` -> `agent.default_temperature` in `llm_defaults.json` |+| `max_tokens` | int\|null | `None` | no | group `options` | 1-200000; `None` -> per-model default (not silently capped) |## Outputs (handles)@@ -92,7 +104,8 @@ Wrapped in the standard envelope:```mermaidflowchart TD- A[handle_chat_agent] --> B[_collect_agent_connections]+ A[BaseNode.execute -> execute_op] --> P[prepare_agent_call]+ P --> B[collect_agent_connections]B --> B1[memory_data from input-memory]B --> B2[skill_data from input-skill<br/>masterSkill expanded to N skills]B --> B3[tool_data from input-tools<br/>androidTool sub-nodes discovered]@@ -103,16 +116,16 @@ flowchart TDB3 --> CB4 --> CB5 --> C- C -- yes + completed/error --> D[_format_task_context<br/>prepend to prompt<br/>STRIP ALL tools]+ C -- yes + completed/error --> D[format_task_context<br/>prepend to prompt<br/>STRIP ALL tools]C -- no --> E{prompt empty<br/>and input_data?}D --> EE -- yes --> F[prompt = input_data.message/text/content/str]E -- no --> G{team lead?<br/>orchestrator / ai_employee}F --> G- G -- yes --> H[_collect_teammate_connections<br/>append as delegate_to_* tools]+ G -- yes --> H[collect_teammate_connections<br/>append as delegate_to_* tools]G -- no --> I[ai_service.execute_chat_agent]H --> I- I --> J[Return envelope]+ I --> J[Return result payload<br/>or raise RuntimeError]```## Decision Logic@@ -126,11 +139,18 @@ flowchart TD`input_data.message` -> `text` -> `content` -> `str(input_data)` in thatorder.- **Team-lead detection**: `TEAM_LEAD_TYPES = {'orchestrator_agent',- 'ai_employee'}`. Teammates are collected only for these two node types- and appended to `tool_data` as delegation targets.+ 'ai_employee'}` (in `_inline.py`). Teammates are collected only for these+ two node types and appended to `tool_data` as delegation targets; each+ teammate's own `input-tools` edges are walked to populate `child_tools` so+ the delegation tool description lists what the teammate can do.- **Master Skill expansion**: `masterSkill` source nodes are expanded in- `_collect_agent_connections` - each enabled entry in `skillsConfig`+ `collect_agent_connections` - each enabled entry in `skillsConfig`becomes a separate skill entry with `node_id = f"{source_id}_{skill_key}"`.+- **api_key recovery**: `prepare_agent_call` reads the resolved key from+ `context["_raw_parameters"]["api_key"]` (injected before Pydantic strips+ it) and re-attaches it to `parameters` so `execute_chat_agent` sees it.+- **Failure**: `execute_op` raises `RuntimeError(response["error"])` when+ `execute_chat_agent` returns `success=False`.## Side Effects@@ -144,7 +164,7 @@ flowchart TDinvokes them.- **External API calls**: one or more calls to the configured LLM provider(see `services/ai.py::execute_chat_agent` -> LangChain chat model + `_run_agent_loop`).-- **Subprocess / file I/O**: none directly in the handler; tools invoked by+- **Subprocess / file I/O**: none directly in the plugin; tools invoked bythe LLM may trigger those themselves.## External Dependencies@@ -159,7 +179,7 @@ flowchart TD- Empty `prompt` with no `input-main` connection still calls`execute_chat_agent` -- the LLM may produce a generic response. There is- no early-return error envelope for missing prompts in this handler.+ no early-return error envelope for missing prompts in this path.- When a `masterSkill` has zero enabled skills, no skill context isinjected and the agent runs without domain-specific instructions.- `input-teammates` is silently ignored for all non-team-lead node types@@ -169,7 +189,7 @@ flowchart TD## Related-- **Dedicated-handler siblings**: [`deepAgent`](./deepAgent.md), [`rlmAgent`](./rlmAgent.md), [`claudeCodeAgent`](./claudeCodeAgent.md)+- **Dedicated-path siblings**: [`rlmAgent`](./rlmAgent.md), [`claudeCodeAgent`](./claudeCodeAgent.md)- **Per-node variants** (short link-style docs):[`androidAgent`](./androidAgent.md), [`codingAgent`](./codingAgent.md),[`webAgent`](./webAgent.md), [`taskAgent`](./taskAgent.md),diff --git a/docs-internal/node-logic-flows/specialized_agents/aiEmployee.md b/docs-internal/node-logic-flows/specialized_agents/aiEmployee.mdindex 4dd05e02..c131c587 100644--- a/docs-internal/node-logic-flows/specialized_agents/aiEmployee.md+++ b/docs-internal/node-logic-flows/specialized_agents/aiEmployee.md@@ -3,28 +3,27 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |-| **Theme color** | `dracula.purple` |-| **Icon** | people / briefcase (U+1F465) |-| **Team lead** | **yes** -- `input-teammates` handle enabled |+| **Plugin** | [`server/nodes/agent/ai_employee/__init__.py`](../../../server/nodes/agent/ai_employee/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |+| **Team lead** | **yes** -- `input-teammates` handle enabled (`team_lead_agent_handles()`) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |## PurposeTeam-lead agent identical in behaviour to `orchestrator_agent`. Distinct-only in frontend presentation (different icon, theme color, default-labels) and an extra `maxConcurrent` parameter. Backend routing is the-same: `handle_chat_agent` -> `_collect_teammate_connections` ->-teammates appended to `tool_data` as `delegate_to_*` tools.+only in frontend presentation (display name "AI Employee", subtitle+"Team Orchestration"). Backend routing is the same:+`SpecializedAgentBase.execute_op` -> `prepare_agent_call`+(`collect_teammate_connections`) -> teammates appended to `tool_data` as+`delegate_to_*` tools -> `execute_chat_agent`.## What is unique to this node-- **`input-teammates` handle** (same as `orchestrator_agent`).-- **`teamMode` parameter** (`parallel` / `sequential`, default `parallel`;- no `""` option).-- **`maxConcurrent` parameter** (number, default `5`, range 1-20)- controlling max concurrent task executions.-- **Frontend theming**: purple dracula accent.+- **`input-teammates` handle** (same as `orchestrator_agent`, via+ `team_lead_agent_handles()`).+- **No extra parameters**: reuses `SpecializedAgentParams` verbatim. There+ is **no** `teamMode` or `maxConcurrent` field on the current Pydantic+ model — those claims were stale.+- **`tool_description`** declared for when a parent team lead delegates to it.## Behaviour@@ -34,10 +33,9 @@ for the shared contract.## Edge cases-- Same as `orchestrator_agent`: non-agent teammates are silently skipped,- and `maxConcurrent` / `teamMode` are passed through as parameters but- not enforced by `handle_chat_agent` itself -- the `delegate_to_*`- implementation owns concurrency.+- Same as `orchestrator_agent`: non-agent teammates are silently skipped;+ concurrency of `delegate_to_*` calls is owned by the downstream tool /+ Temporal child-workflow dispatch, not by any node parameter.## Relateddiff --git a/docs-internal/node-logic-flows/specialized_agents/androidAgent.md b/docs-internal/node-logic-flows/specialized_agents/androidAgent.mdindex 751dd20d..85a3a631 100644--- a/docs-internal/node-logic-flows/specialized_agents/androidAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/androidAgent.md@@ -3,10 +3,8 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) (routed via registry) |-| **Theme color** | `dracula.green` |-| **Icon** | phone (U+1F4F1) |-| **AGENT_CONFIGS entry** | `AIAgentNode.tsx` key `android_agent` |+| **Plugin** | [`server/nodes/agent/android_agent/__init__.py`](../../../server/nodes/agent/android_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |+| **Subtitle** | Device Control || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |## Purpose@@ -29,8 +27,9 @@ the `android_agent` folder via a Master Skill.See **[Generic Specialized Agent Pattern](./_pattern.md)** for the fullcontract: inputs, parameters, outputs, logic flow, decision logic, side-effects, edge cases. This node routes to `handle_chat_agent` with no-behavioural differences from the other 12 variants.+effects, edge cases. This node routes through `SpecializedAgentBase.execute_op`+-> `execute_chat_agent` with no behavioural differences from the other 12+variants.## Relateddiff --git a/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.md b/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.mdindex 520e69ac..cb72c117 100644--- a/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/autonomousAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/autonomous_agent/__init__.py`](../../../server/nodes/agent/autonomous_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.purple` || **Icon** | target (U+1F3AF) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |@@ -26,9 +26,11 @@ the autonomous skill pack.## Behaviour-See **[Generic Specialized Agent Pattern](./_pattern.md)**. Routes to the-same `handle_chat_agent` -- the "autonomous" behaviour comes entirely from-the attached skill content, not a different execution engine.+See **[Generic Specialized Agent Pattern](./_pattern.md)**. Routes through the+same `SpecializedAgentBase.execute_op` -> `execute_chat_agent` -- the+"autonomous" behaviour comes entirely from the attached skill content, not a+different execution engine. (The plugin declares a `tool_description` for when+it is delegated to by a team lead.)## Relateddiff --git a/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.md b/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.mdindex a4fbb8c7..7547e0f5 100644--- a/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/claudeCodeAgent.md@@ -3,104 +3,154 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/claude_code.py::handle_claude_code_agent`](../../../server/services/handlers/claude_code.py) |-| **Backend service** | [`server/services/claude_code_service.py::ClaudeCodeService.execute`](../../../server/services/claude_code_service.py) |+| **Plugin** | [`server/nodes/agent/claude_code_agent/__init__.py::ClaudeCodeAgentNode.execute_op`](../../../server/nodes/agent/claude_code_agent/__init__.py) (dispatch via `BaseNode.execute()`) |+| **Backend service** | [`server/services/cli_agent/service.py::AICliService.run_batch`](../../../server/services/cli_agent/service.py) (provider `"claude"`); pool [`_pool.py`](../../../server/nodes/agent/claude_code_agent/_pool.py), argv [`_provider.py`](../../../server/nodes/agent/claude_code_agent/_provider.py), skill materialiser [`_skills.py`](../../../server/nodes/agent/claude_code_agent/_skills.py) |+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) || **Tests** | [`server/tests/nodes/test_specialized_agents.py::TestClaudeCodeAgent`](../../../server/tests/nodes/test_specialized_agents.py) |## Purpose-Runs a Claude Code CLI subprocess (`claude -p <prompt> --output-format json-...`) and returns the parsed JSON. Keeps a per-`node_id` session map so-follow-up invocations resume with `--resume <session_id>`. This is the-only specialized-agent node that shells out; the others stay in-process.+Runs N parallel Claude Code CLI sessions over a list of `ClaudeTaskSpec`+tasks via `AICliService.run_batch("claude", ...)`. The pool spawns `claude`+as a plain subprocess driven over the VSCode-extension stream-json protocol+(`--output-format stream-json --input-format stream-json --verbose --ide`) —+NOT the old headless `claude -p`. The common case is a single visible+`prompt` that synthesises a one-task batch. Memory continuity is handled by+claude's own on-disk session JSONL via `--continue` / `--resume` over a+stable `cwd`, not by re-injecting markdown. This is the only+specialized-agent node that shells out; the others stay in-process.## Inputs (handles)+Standard `std_agent_handles()` topology (same as the generic agents).+| Handle | Purpose ||--------|---------|-| `input-main` | Auto-prompt fallback (reads `source.message / text / content`) |-| `input-skill` | Skill instructions appended to `--append-system-prompt` |--All other handles (`input-memory`, `input-tools`, `input-task`,-`input-teammates`) are **ignored** by this handler.+| `input-main` | Auto-prompt fallback (reads `source.message / text / content / str`) |+| `input-skill` | Connected skill names collected; SKILL.md trees materialised under `<workspace>/.claude/skills/` and exposed to the CLI (not injected into a system prompt) |+| `input-memory` | When wired, sets `continue_session=True` so claude emits `--continue` and resumes its on-disk session under the stable cwd |+| `input-tools` | Connected tool nodes exposed to the CLI as `mcp__machinaos__<type>` MCP tools |+| `input-task` | Collected by the edge-walker; `task_data` is unpacked but the 5th element is ignored (`_`) here |## Parameters-| Name | Type | Default | Description |-|------|------|---------|-------------|-| `prompt` | string | `""` | User prompt passed via `-p` |-| `systemPrompt` | string | `""` | Prepended to any skill content for `--append-system-prompt` |-| `model` | string | `claude-sonnet-4-6` | Passed via `--model` |-| `workingDirectory` | string | (default workspace) | `cwd` for the subprocess |-| `allowedTools` | string | `Read,Edit,Bash,Glob,Grep,Write` | Passed via `--allowedTools` |-| `maxTurns` | number | `10` | `--max-turns` |-| `maxBudgetUsd` | number | `5.0` | **`--max-budget-usd`** (note: historical bug fixed to use this exact flag name) |+`ClaudeCodeAgentParams` — only `prompt` is visible; every other field carries+`json_schema_extra.hidden=True` and feeds defaults into the per-task+`ClaudeTaskSpec`.++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `prompt` | string | `""` | no | - | The prompt sent to Claude Code (only visible field, 4 rows) |+| `tasks` | `ClaudeTaskSpec[]` | `[]` | no | hidden | Advanced multi-task batch (max 5 concurrent); `null` coerced to `[]` |+| `model` | enum (`ClaudeCodeModel`) | `claude-sonnet-4-6` | no | hidden | `--model`; aliases `sonnet`/`opus`/`haiku` or pinned IDs; unknown values coerced to default |+| `system_prompt` | string\|null | `None` | no | hidden | System prompt (3 rows) |+| `working_directory` | string\|null | `None` | no | hidden | Git repo root; defaults to workflow workspace dir |+| `max_parallel` | int | `5` | no | hidden | 1-20 concurrency cap |+| `allowed_credentials` | string[] | `[]` | no | hidden | Credential names the CLI may fetch via MCP; `null` coerced to `[]` |+| `effort` | enum\|null (`ClaudeCodeEffort`) | `None` | no | hidden | `low`/`medium`/`high`/`xhigh`/`max` via `--effort` (thinking models) |+| `fallback_model` | enum\|null | `None` | no | hidden | `--fallback-model`; unknown values coerced to `None` |## Outputs (handles)+`Output = ClaudeCodeAgentOutput` (`extra="forbid"`) — aggregated batch shape:+| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | `{ response, model, provider: 'anthropic', session_id, usage, timestamp }` |+| `output-main` / `output-top` | object | Batch result (see payload below) |++### Output payload++```ts+{+ success: boolean; // result.n_failed == 0+ n_tasks: number;+ n_succeeded: number;+ n_failed: number;+ total_cost_usd: number | null;+ wall_clock_ms: number;+ tasks: SessionResultModel[]; // per-task result models+ provider: "claude";+ timestamp: string;+ // Legacy single-task convenience (populated when len(tasks) == 1):+ response: string | null;+ session_id: string | null;+ cost_usd: number | null;+}+```++`node_output_schemas.py` registers `claude_code_agent` as `AIAgentOutput`+(`response`/`thinking`/`model`/`provider`/`finish_reason`/`timestamp`) for the+frontend Input Data panel — a separate, looser schema from the plugin's strict+`ClaudeCodeAgentOutput`.## Logic Flow```mermaidflowchart TD- A[handle_claude_code_agent] --> B[broadcast executing]- B --> C{prompt param set?}- C -- no --> D[Scan input-main edges<br/>pull source.message/text/content]- C -- yes --> E[_collect_agent_connections for skills]- D --> F{prompt still empty?}- F -- yes --> G[Return envelope success=false<br/>error: No prompt provided]- F -- no --> E- E --> H[Build system_parts<br/>systemPrompt + skill.instructions]- H --> I[get_claude_code_service]- I --> J[service.execute<br/>spawns claude subprocess<br/>300s timeout]- J -- returncode != 0 --> K[RuntimeError<br/>caught by outer try/except<br/>returns success=false envelope]- J -- returncode == 0 --> L[Parse JSON stdout<br/>or fallback to raw text]- L --> M[Store session_id in service._session_map]- M --> N[broadcast success]- N --> O[Return envelope success=true]+ A[ClaudeCodeAgentNode.execute_op] --> B[broadcast executing]+ B --> C[collect_agent_connections<br/>memory/skill/tool/input]+ C --> D{memory_data wired?}+ D -- yes --> D1[continue_session = True]+ D -- no --> E{params.tasks empty?}+ D1 --> E+ E -- yes --> F[Resolve prompt:<br/>params.prompt else input_data<br/>message/text/content/str]+ F --> G{prompt empty?}+ G -- yes --> H[raise NodeUserError<br/>no prompt]+ G -- no --> I[Build single ClaudeTaskSpec]+ E -- no --> J[Apply node defaults to each task<br/>model/system_prompt/effort/fallback/continue]+ I --> K{memory + len tasks > 1?}+ J --> K+ K -- yes --> L[raise NodeUserError<br/>memory batch must be serial]+ K -- no --> M[Resolve workspace_dir + repo_root]+ M --> N[AICliService.run_batch claude]+ N --> O[Aggregate result -> dict payload]+ O --> P[broadcast success/warning]```## Decision Logic- **Prompt resolution**: explicit `prompt` param wins; otherwise scans- `input-main` edges and reads `source.message / text / content / str`.-- **Skill collection**: uses `_collect_agent_connections` only to gather- skills. Memory, tools, teammates, task triggers are all discarded.-- **System prompt assembly**: `systemPrompt` param + every enabled skill's- `instructions`, joined by `\n\n`, passed as- `--append-system-prompt`.-- **Session resume**: `ClaudeCodeService._session_map[node_id]` is- consulted; when set, `--resume <session_id>` is added to argv.-- **Working directory**: if `workingDirectory` is empty, the service- defaults to `{workspace_base_resolved}/default` and creates it if- missing.+ `input_data` and reads `message / text / content / str` (same chain as+ `_inline.py`). Empty -> `NodeUserError`.+- **Single-task synthesis**: when `params.tasks` is empty, one+ `ClaudeTaskSpec` is built from `prompt` + `model` + `system_prompt` ++ `effort` + `fallback_model` + `continue_session`.+- **Batch defaulting**: when `params.tasks` is non-empty, node-level+ `model`/`system_prompt`/`effort`/`fallback_model` fill any task that+ didn't override; `continue_session=True` is auto-set only when memory is+ wired AND the task didn't explicitly opt in or pick a resume UUID.+- **Memory bridge**: `continue_session = bool(memory_data)`; the argv builder+ emits `--continue` (first run is a benign no-op, later runs resume the+ on-disk JSONL). NO markdown re-injection.+- **Serial-memory guard**: memory wired AND `len(tasks) > 1` -> `NodeUserError`+ (parallel `--resume` against one JSONL would corrupt it).+- **Workspace**: `ctx.raw["workspace_dir"]` (injected by workflow.py) or+ `working_directory`; falls back to `{workspace_base_resolved}/{workflow_id|default}`.## Side Effects-- **Subprocess spawn**: `asyncio.create_subprocess_exec('claude', '-p',- <prompt>, '--output-format', 'json', '--model', <model>, '--max-turns',- N, '--allowedTools', <list>, '--max-budget-usd', <usd>, [--resume- <session_id>,] [--append-system-prompt <system>,] ...)` with a 300s- `asyncio.wait_for` timeout. Falls back to `npx -y @anthropic-ai/claude-code`- when the `claude` binary is not on PATH.+- **Subprocess spawn**: `AICliService.run_batch` -> `ClaudeSessionPool` spawns+ `claude` as a plain-pipe subprocess (no PTY) with `--output-format+ stream-json --input-format stream-json --verbose --ide+ --permission-mode dontAsk` + `--allowedTools <mcp tools + conditional Skill ++ 5 infra tools>` + `--mcp-config <bearer>` + `--add-dir <workspace>`. Warm+ subprocesses are reused across turns/batches; crash recovery via+ `--resume <UUID>`.- **Broadcasts**: `StatusBroadcaster.update_node_status` -- executing- ("Starting Claude Code..." and "Running Claude Code (<model>)..."),- then success on completion.-- **Database**: none directly; usage tracking happens only if the caller- wires it in (handler signature accepts `database=None` and never uses- it).-- **File I/O**: creates `{workspace_base_resolved}/default` when the- default cwd does not exist.+ ("Starting Claude Code batch..."), then `success`/`warning` on completion+ (`n_succeeded`/`n_tasks`). The pool emits 4 typed CloudEvents+ (`claude.session.{spawned,cleared,terminated,usage}`); memory turns emit+ `node_parameters_updated` so the simpleMemory panel refreshes live.+- **File I/O**: materialises connected SKILL.md trees under+ `<workspace>/.claude/skills/<name>/` (diff-based on warm reuse). Claude+ keeps its own session JSONL under `<CLAUDE_CONFIG_DIR>/projects/<cwd-key>/`.- **External API**: indirect -- the CLI calls Anthropic internally.## External Dependencies- **Binaries**: `@anthropic-ai/claude-code`, MachinaOs-managed — auto-installed on first use into the shared npm tree at `<DATA_DIR>/packages/` (binary at `<DATA_DIR>/packages/node_modules/.bin/claude[.cmd]`); requires `npm` on PATH.-- **Python packages**: standard library only- (`asyncio.create_subprocess_exec`).+- **Python packages**: standard library + the `services/cli_agent/` framework+ (FastMCP bridge, session pool).- **Credentials**: Claude Code handles its own authentication via`CLAUDE_CONFIG_DIR=<DATA_DIR>/claude/` (= `~/.machina/claude/` bydefault; see [Claude OAuth](../../claude_code_agent_architecture.md));@@ -108,26 +158,29 @@ flowchart TD## Edge cases & known limits-- **No prompt -> early failure**: unlike `handle_chat_agent`, this- handler explicitly returns an error envelope when neither `prompt`- nor a connected `input-main` output yields text.-- **`max_budget_usd <= 0`**: the `--max-budget-usd` flag is omitted- entirely (see `claude_code_service.py:50-51`). This silently disables- budget enforcement rather than passing `0`.-- **Non-zero exit**: `ClaudeCodeService.execute` raises `RuntimeError`,- the handler's outer `except Exception` converts it to- `success=false` with the error message as the envelope's `error` field.-- **300-second timeout**: `asyncio.wait_for(proc.communicate(),- timeout=300)` is hard-coded. Long CLI runs beyond 5 minutes abort and- surface as `TimeoutError` in the envelope.-- **Tools/memory/teammates ignored**: connecting memory, tools, or- teammates has no effect. The CLI's own tool set is controlled by- `allowedTools`.-- **JSON parsing fallback**: if stdout is not valid JSON, the service- returns `{"result": <raw text>}` -- the handler still reports- `success=true` and `response` contains the raw text.+- **No prompt -> `NodeUserError`**: when `tasks` is empty AND neither+ `prompt` nor a connected `input-main` output yields text, `execute_op`+ raises `NodeUserError` (single WARN line, structured envelope).+- **Memory-bound batches must be serial**: memory wired AND `len(tasks) > 1`+ raises `NodeUserError` (parallel `--resume` would corrupt one JSONL).+- **Stale-model coercion**: legacy saved `model` / `fallback_model` strings+ (dot-spelled, date-suffixed, empty) coerce to the default / `None` via a+ `mode="before"` field validator so old workflows still load.+- **Output is strict** (`extra="forbid"`): a malformed CLI envelope raises a+ `ValidationError` at the Output boundary instead of emitting an+ unparseable shape downstream.+- **Strict tool allowlist**: `--permission-mode dontAsk` enforces+ `--allowedTools` (only wired `mcp__machinaos__*` tools, conditional `Skill`,+ 5 infra tools); claude's own built-in Read/Edit/Bash/etc. do NOT fire+ unless wired.+- **Per-workflow skill isolation**: SKILL.md trees materialise under the+ workspace's `.claude/skills/`, not the repo, so workflow A's skills can't+ leak into workflow B's subprocess.## Related-- **Pattern siblings**: [`deepAgent`](./deepAgent.md), [`rlmAgent`](./rlmAgent.md)-- **Architecture**: [Claude Code Agent](../../claude_code_agent_architecture.md)+- **Dedicated-path siblings**: [`rlmAgent`](./rlmAgent.md)+- **Generic pattern**: [`_pattern.md`](./_pattern.md)+- **Architecture**: [CLI Agent Framework](../../cli_agent_framework.md),+ [Claude Code Interactive Mode](../../claude_code_interactive_mode.md),+ [Claude Code Agent](../../claude_code_agent_architecture.md)diff --git a/docs-internal/node-logic-flows/specialized_agents/codexAgent.md b/docs-internal/node-logic-flows/specialized_agents/codexAgent.mdnew file mode 100644index 00000000..d45a6276--- /dev/null+++ b/docs-internal/node-logic-flows/specialized_agents/codexAgent.md@@ -0,0 +1,142 @@+# Codex (`codex_agent`)++| Field | Value |+|------|-------|+| **Category** | specialized_agents |+| **Plugin** | [`server/nodes/agent/codex_agent/__init__.py::CodexAgentNode.execute_op`](../../../server/nodes/agent/codex_agent/__init__.py) (dispatch via `BaseNode.execute()`) |+| **Backend service** | [`server/services/cli_agent/service.py::AICliService.run_batch`](../../../server/services/cli_agent/service.py) (provider `"codex"`) |+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) |+| **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |+| **Dual-purpose tool** | no |++## Purpose++Runs N parallel OpenAI Codex CLI sessions over a list of `CodexTaskSpec`+tasks via `AICliService.run_batch("codex", ...)`. Sandbox-first companion to+`claude_code_agent`: the sandbox is enforced by Codex itself (not MachinaOs),+and each task runs in its own git worktree. Codex exposes no+session/resume/budget/turns surface — only `sandbox` + `ask_for_approval` are+task-level overrides. The common case is a single `prompt` that synthesises a+one-task batch.++## Inputs (handles)++Standard `std_agent_handles()` topology.++| Handle | Connection type | Required | Purpose |+|--------|-----------------|----------|---------|+| `input-main` | input/main | no | Prompt fallback: `_infer_prompt_from_inputs` reads `outputs[source].message/text/content/prompt/str` |+| `input-skill` | input/skill | no | Connected skill names collected and passed to `run_batch` |+| `input-memory` | input/memory | no | Collected by edge-walker but unused (Codex has no resume surface) |+| `input-tools` | input/tools | no | Collected by edge-walker (5-tuple) but discarded here |+| `input-task` | input/task | no | Collected but discarded |++## Parameters++`CodexAgentParams` (`extra="ignore"`):++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `tasks` | `CodexTaskSpec[]` | `[]` | no | - | List of Codex tasks (max 5 concurrent) |+| `prompt` | string | `""` | no | - | Single-prompt fallback used only when `tasks` is empty (4 rows) |+| `model` | string | `gpt-5.2-codex` | no | - | Default model for tasks that don't override |+| `sandbox` | string | `workspace-write` | no | - | `read-only` / `workspace-write` / `danger-full-access` |+| `ask_for_approval` | string | `never` | no | - | `untrusted` / `on-request` / `never` |+| `system_prompt` | string\|null | `None` | no | - | System prompt (3 rows) |+| `working_directory` | string\|null | `None` | no | - | Git repo root; defaults to workflow workspace dir |+| `max_parallel` | int | `5` | no | - | 1-20 concurrency cap |+| `allowed_credentials` | string[] | `[]` | no | - | Credential names the CLI may fetch via MCP |++## Outputs (handles)++`Output = CodexAgentOutput` (`extra="allow"`) — aggregated batch shape.++| Handle | Shape | Description |+|--------|-------|-------------|+| `output-main` / `output-top` | object | Batch result (see payload below) |++### Output payload++```ts+{+ success: boolean; // result.n_failed == 0+ n_tasks: number;+ n_succeeded: number;+ n_failed: number;+ total_cost_usd: null; // always None for Codex+ wall_clock_ms: number;+ tasks: SessionResultModel[];+ provider: "codex";+ timestamp: string;+ response: string | null; // populated when len(tasks) == 1+}+```++Note: `codex_agent` is **not** registered in+`server/services/node_output_schemas.py::_AGENT_TYPES`, so the frontend Input+Data panel has no declared output schema for it (falls back to live execution+data or empty state).++## Logic Flow++```mermaid+flowchart TD+ A[CodexAgentNode.execute_op] --> B[broadcast executing]+ B --> C{params.tasks empty?}+ C -- yes --> D[prompt = params.prompt<br/>else _infer_prompt_from_inputs]+ D --> E{prompt empty?}+ E -- yes --> F[raise RuntimeError<br/>provide tasks or prompt]+ E -- no --> G[Build single CodexTaskSpec]+ C -- no --> H[Apply node defaults<br/>model/system_prompt per task]+ G --> I[collect_agent_connections<br/>skill names]+ H --> I+ I --> J[Resolve workspace_dir + repo_root]+ J --> K[AICliService.run_batch codex]+ K --> L[Aggregate result -> dict payload]+ L --> M[broadcast success/warning]+```++## Decision Logic++- **Prompt resolution**: `params.prompt` wins; otherwise+ `_infer_prompt_from_inputs` scans `input-main` (or untagged) edges and reads+ `outputs[source].message/text/content/prompt/str`. Empty -> `RuntimeError`.+- **Single-task synthesis**: empty `tasks` builds one `CodexTaskSpec` from+ `prompt` + `model` + `sandbox` + `ask_for_approval` + `system_prompt`.+- **Batch defaulting**: non-empty `tasks` get node-level `model` /+ `system_prompt` filled where the task didn't override.+- **No memory/resume/budget/turns**: Codex has no session surface; memory /+ tools / task data are collected by the edge-walker but discarded.++## Side Effects++- **Subprocess spawn**: `AICliService.run_batch("codex", ...)` spawns the+ Codex CLI per task (per-task git worktree isolation; sandbox enforced by+ Codex).+- **Broadcasts**: `StatusBroadcaster.update_node_status` -- executing+ ("Starting Codex batch..."), then `success`/`warning` on completion.+- **File I/O**: per-task git worktrees under the workflow workspace dir.+- **External API**: indirect -- the Codex CLI calls OpenAI internally.++## External Dependencies++- **Binaries**: the OpenAI Codex CLI (provider `"codex"` in the cli_agent+ framework).+- **Python packages**: the `services/cli_agent/` framework.+- **Credentials**: Codex handles its own authentication; MachinaOs does not+ inject an API key here. `allowed_credentials` gates what the CLI may fetch+ via the MachinaOs MCP bridge.++## Edge cases & known limits++- **No prompt / tasks -> `RuntimeError`**: when `tasks` is empty AND no input+ yields a prompt, `execute_op` raises `RuntimeError` (full traceback path —+ not `NodeUserError` like `claude_code_agent`).+- **`total_cost_usd` is always `None`**: Codex does not surface per-run cost.+- **No output schema registration**: see Outputs note above.++## Related++- **Dedicated-path siblings**: [`claudeCodeAgent`](./claudeCodeAgent.md), [`rlmAgent`](./rlmAgent.md)+- **Generic pattern**: [`_pattern.md`](./_pattern.md)+- **Architecture**: [CLI Agent Framework](../../cli_agent_framework.md)diff --git a/docs-internal/node-logic-flows/specialized_agents/codingAgent.md b/docs-internal/node-logic-flows/specialized_agents/codingAgent.mdindex b5e89f9e..3f07c6ed 100644--- a/docs-internal/node-logic-flows/specialized_agents/codingAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/codingAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/coding_agent/__init__.py`](../../../server/nodes/agent/coding_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.cyan` || **Icon** | laptop (U+1F4BB) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |diff --git a/docs-internal/node-logic-flows/specialized_agents/consumerAgent.md b/docs-internal/node-logic-flows/specialized_agents/consumerAgent.mdindex 4f55140d..9a6f575c 100644--- a/docs-internal/node-logic-flows/specialized_agents/consumerAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/consumerAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/consumer_agent/__init__.py`](../../../server/nodes/agent/consumer_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.purple` || **Icon** | shopping cart (U+1F6D2) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |@@ -12,7 +12,7 @@AI agent pre-configured for customer support and consumer interactions(order status, product recommendations). Like `payments_agent`, this is a-naming / theming shell around `handle_chat_agent`.+naming / theming shell around `SpecializedAgentBase` -> `execute_chat_agent`.## What is unique to this nodediff --git a/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.md b/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.mdindex 4a557388..f3ab3810 100644--- a/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/orchestratorAgent.md@@ -3,10 +3,8 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |-| **Theme color** | `dracula.cyan` |-| **Icon** | musical score (U+1F3BC) |-| **Team lead** | **yes** -- `input-teammates` handle enabled |+| **Plugin** | [`server/nodes/agent/orchestrator_agent/__init__.py`](../../../server/nodes/agent/orchestrator_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) |+| **Team lead** | **yes** -- `input-teammates` handle enabled (`team_lead_agent_handles()`) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |## Purpose@@ -17,18 +15,21 @@ are wired via the extra `input-teammates` handle and become## What is unique to this node-- **`input-teammates` handle** (extra input beyond the shared 5).-- **`teamMode` parameter** (`""` / `parallel` / `sequential`) controlling- how delegate_to_* calls are executed.-- **Team lead detection**: `handle_chat_agent` checks `node_type in- TEAM_LEAD_TYPES = {'orchestrator_agent', 'ai_employee'}` and calls- `_collect_teammate_connections` to expand teammates into tools.-- **Frontend theming**: cyan dracula accent.+- **`input-teammates` handle** (extra input beyond the shared 5; declared+ via `team_lead_agent_handles()` in `_handles.py`).+- **No extra parameters**: uses the same `SpecializedAgentParams` as every+ other agent. There is **no** `teamMode` / `maxConcurrent` field — those+ were removed (or never existed on the current Pydantic model).+- **`tool_description`** declared so a parent team lead delegating to this+ node gets a meaningful tool string ("ONE-SHOT delegation ...").+- **Team lead detection**: `prepare_agent_call` (in `_inline.py`) checks+ `node_type in TEAM_LEAD_TYPES = {'orchestrator_agent', 'ai_employee'}` and+ calls `collect_teammate_connections` to expand teammates into tools.## Teammate collection-`_collect_teammate_connections(node_id, context, database)` in-`handlers/ai.py`:+`collect_teammate_connections(node_id, context, database)` in+[`server/services/plugin/edge_walker.py`](../../../server/services/plugin/edge_walker.py):1. Scans `context.edges` for `edge.target == node_id` and`edge.targetHandle == 'input-teammates'`.@@ -37,8 +38,11 @@ are wired via the extra `input-teammates` handle and become4. Loads `database.get_node_parameters(source_id)` for each teammate.5. Returns a list of `{node_id, node_type, label, parameters}` dicts.-The teammates are then appended to `tool_data` before-`execute_chat_agent` is called, so the LLM sees them as ordinary tools.+`prepare_agent_call` then walks each teammate's own `input-tools` edges to+populate `child_tools` (so the delegation tool description lists what each+teammate can do) and appends the teammates to `tool_data` before+`execute_chat_agent` is called, so the LLM sees them as ordinary+`delegate_to_*` tools.## Behaviour@@ -50,10 +54,10 @@ expansion above.- Non-`AI_AGENT_TYPES` nodes wired to `input-teammates` are silentlyskipped.-- `teamMode` is collected by `execute_chat_agent` but the actual- parallel / sequential scheduling is not enforced at this layer -- it- depends on how the downstream `delegate_to_*` tool implementation handles- concurrent invocations.+- Concurrency of `delegate_to_*` calls is not controlled by any node+ parameter (no `teamMode`); it depends on how the downstream+ `delegate_to_*` tool implementation / Temporal child-workflow dispatch+ handles concurrent invocations.- When zero teammates are connected, the node behaves identically to anyother specialized agent.diff --git a/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.md b/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.mdindex e9f53867..78c44b0b 100644--- a/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/paymentsAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/payments_agent/__init__.py`](../../../server/nodes/agent/payments_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.green` || **Icon** | credit card (U+1F4B3) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |@@ -19,7 +19,7 @@ wire `httpRequest` against a payment provider (Stripe, Razorpay) as a tool.- **Intended tool set**: `httpRequest` against payment APIs.- **Intended skills**: no dedicated `server/skills/payments_agent/` folderships today -- the node is a naming / theming shell around- `handle_chat_agent`.+ `SpecializedAgentBase` -> `execute_chat_agent`.- **Frontend theming**: green dracula accent.## Behaviourdiff --git a/docs-internal/node-logic-flows/specialized_agents/productivityAgent.md b/docs-internal/node-logic-flows/specialized_agents/productivityAgent.mdindex ba720152..9e759cd2 100644--- a/docs-internal/node-logic-flows/specialized_agents/productivityAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/productivityAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/productivity_agent/__init__.py`](../../../server/nodes/agent/productivity_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.cyan` || **Icon** | clock (U+23F0) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |diff --git a/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md b/docs-internal/node-logic-flows/specialized_agents/rlmAgent.mdindex a85ddf0a..be5dc4ac 100644--- a/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md@@ -3,10 +3,9 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/rlm.py::handle_rlm_agent`](../../../server/services/handlers/rlm.py) |+| **Plugin** | [`server/nodes/agent/rlm_agent/__init__.py::RLMAgentNode.execute_op`](../../../server/nodes/agent/rlm_agent/__init__.py) (dispatch via `BaseNode.execute()`) || **Backend service** | `AIService.rlm_service` -> `RLMService.execute` |-| **Theme color** | `dracula.orange` |-| **Icon** | brain (U+1F9E0) |+| **Connection collection** | [`server/services/plugin/edge_walker.py::collect_agent_connections`](../../../server/services/plugin/edge_walker.py) || **Tests** | [`server/tests/nodes/test_specialized_agents.py::TestRLMAgent`](../../../server/tests/nodes/test_specialized_agents.py) |## Purpose@@ -23,23 +22,35 @@ Same 5 shared handles as the generic specialized agents. **No## Parameters-Standard `AI_AGENT_PROPERTIES` plus RLM extras:--| Name | Type | Default | Description |-|------|------|---------|-------------|-| `maxIterations` | number | `30` | Hard cap on REPL iterations before forcing `FINAL(...)` |+`RLMAgentNode.Params = SpecializedAgentParams` — the **same** model as the+generic specialized agents (`prompt`, `provider`, `model`, `system_message`,+`temperature`, `max_tokens`). There is **no** `maxIterations` field on the node+params; REPL iteration bounding lives inside `RLMService` /+`server/config/llm_defaults.json`, not on this plugin.++| Name | Type | Default | Required | displayOptions.show | Description |+|------|------|---------|----------|---------------------|-------------|+| `prompt` | string | `""` | no | - | Reasoning task; falls back to upstream input |+| `provider` | enum | `openai` | no | - | Big-LM provider |+| `model` | string | `""` | no | - | Big-LM model ID |+| `system_message` | string\|null | `"You are a helpful assistant"` | no | - | System instructions |+| `temperature` | float\|null | `None` | no | group `options` | 0.0-2.0 |+| `max_tokens` | int\|null | `None` | no | group `options` | 1-200000 |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | `{ response, model, provider, iterations, timestamp }` |+| `output-main` / `output-top` | object | `SpecializedAgentOutput` (`extra="allow"`): `{ response, thinking?, model, provider, finish_reason?, timestamp }` plus any RLM extras the service emits |++`node_output_schemas.py` registers `rlm_agent` as `AIAgentOutput` (same field+set) for the frontend Input Data panel.## Logic Flow```mermaidflowchart TD- A[handle_rlm_agent] --> B[_collect_agent_connections]+ A[RLMAgentNode.execute_op] --> B[collect_agent_connections]B --> C{task_data present?}C -- completed/error --> D[prepend task context<br/>strip tools]C -- no --> E{prompt empty<br/>and input_data?}@@ -48,18 +59,22 @@ flowchart TDE -- no --> G[ai_service.rlm_service.execute]F --> GG --> H[RLMService REPL loop<br/>llm_query / rlm_query / FINAL]- H --> I[Return envelope]+ H --> I[Return result payload<br/>or raise RuntimeError]```## Decision Logic-- **Same preprocessing as `handle_chat_agent`**: task-completion tool- strip, auto-prompt fallback.-- **No teammate expansion**: `_collect_teammate_connections` is not- called.+- **Same preprocessing as the generic agent path**: task-completion tool+ strip (when `task_data.status in {completed, error}`), auto-prompt+ fallback. Implemented inline in `RLMAgentNode.execute_op` (NOT via+ `prepare_agent_call` — RLM has its own copy of the 3-step preamble).+- **No teammate expansion**: `collect_teammate_connections` is not called;+ `rlm_agent` is not in `TEAM_LEAD_TYPES`.- **No workspace injection**: RLM executes code in an in-memory REPL- (`exec()`), not on a real filesystem; the handler does not pass+ (`exec()`), not on a real filesystem; the plugin does not pass`workspace_dir`.+- **Failure**: `execute_op` raises `RuntimeError` when the service returns+ `success=False`; `BaseNode.execute()` wraps it into the error envelope.## Side Effects@@ -89,8 +104,9 @@ flowchart TDprocess with no sandboxing. The skill prompt is expected to constrainthe LLM, but nothing prevents a misbehaving LLM from importing`subprocess` or `os` and doing I/O.-- **No per-step timeout**: `maxIterations` is the only bound; a single- REPL step can hang the handler if user code blocks.+- **No per-step timeout**: the RLM iteration cap (configured in+ `RLMService` / `llm_defaults.json`, not on the node) is the only bound;+ a single REPL step can hang the operation if user code blocks.- **No team-lead support**: `input-teammates` is not read. Connectingagents there has no effect.- **LLM output parsing**: the REPL parser expects a specific fenced-code@@ -99,5 +115,6 @@ flowchart TD## Related-- **Pattern siblings**: [`deepAgent`](./deepAgent.md), [`claudeCodeAgent`](./claudeCodeAgent.md)+- **Dedicated-path siblings**: [`claudeCodeAgent`](./claudeCodeAgent.md)+- **Generic pattern**: [`_pattern.md`](./_pattern.md)- **Architecture**: [RLM Service](../../rlm_service.md)diff --git a/docs-internal/node-logic-flows/specialized_agents/socialAgent.md b/docs-internal/node-logic-flows/specialized_agents/socialAgent.mdindex 0d1f001c..c5542f47 100644--- a/docs-internal/node-logic-flows/specialized_agents/socialAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/socialAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/social_agent/__init__.py`](../../../server/nodes/agent/social_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.green` || **Icon** | phone (U+1F4F1) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |diff --git a/docs-internal/node-logic-flows/specialized_agents/taskAgent.md b/docs-internal/node-logic-flows/specialized_agents/taskAgent.mdindex 8de427ef..61251c60 100644--- a/docs-internal/node-logic-flows/specialized_agents/taskAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/taskAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/task_agent/__init__.py`](../../../server/nodes/agent/task_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.purple` || **Icon** | clipboard (U+1F4CB) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |diff --git a/docs-internal/node-logic-flows/specialized_agents/toolAgent.md b/docs-internal/node-logic-flows/specialized_agents/toolAgent.mdindex 26016bc1..bbf93a15 100644--- a/docs-internal/node-logic-flows/specialized_agents/toolAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/toolAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/tool_agent/__init__.py`](../../../server/nodes/agent/tool_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.yellow` || **Icon** | wrench (U+1F527) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |diff --git a/docs-internal/node-logic-flows/specialized_agents/travelAgent.md b/docs-internal/node-logic-flows/specialized_agents/travelAgent.mdindex 0a21d5e3..ddbf442d 100644--- a/docs-internal/node-logic-flows/specialized_agents/travelAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/travelAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/travel_agent/__init__.py`](../../../server/nodes/agent/travel_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.orange` || **Icon** | airplane (U+2708) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |diff --git a/docs-internal/node-logic-flows/specialized_agents/webAgent.md b/docs-internal/node-logic-flows/specialized_agents/webAgent.mdindex 23e1472c..96c75762 100644--- a/docs-internal/node-logic-flows/specialized_agents/webAgent.md+++ b/docs-internal/node-logic-flows/specialized_agents/webAgent.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | specialized_agents |-| **Backend handler** | [`server/services/handlers/ai.py::handle_chat_agent`](../../../server/services/handlers/ai.py) |+| **Plugin** | [`server/nodes/agent/web_agent/__init__.py`](../../../server/nodes/agent/web_agent/__init__.py) -> [`_specialized.py::SpecializedAgentBase.execute_op`](../../../server/nodes/agent/_specialized.py) (dispatch via `BaseNode.execute()`) || **Theme color** | `dracula.pink` || **Icon** | globe (U+1F310) || **Tests** | [`server/tests/nodes/test_specialized_agents.py`](../../../server/tests/nodes/test_specialized_agents.py) |diff --git a/docs-internal/node-logic-flows/telegram_social/socialReceive.md b/docs-internal/node-logic-flows/telegram_social/socialReceive.mdindex 243abebf..c750bee9 100644--- a/docs-internal/node-logic-flows/telegram_social/socialReceive.md+++ b/docs-internal/node-logic-flows/telegram_social/socialReceive.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | social |-| **Backend handler** | [`server/services/handlers/social.py::handle_social_receive`](../../../server/services/handlers/social.py) |+| **Backend handler** | plugin [`server/nodes/social/social_receive/__init__.py`](../../../server/nodes/social/social_receive/__init__.py) (`SocialReceiveNode`, custom `execute` override) delegating to [`server/nodes/social/_base.py::handle_social_receive`](../../../server/nodes/social/_base.py) || **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -16,37 +16,44 @@ Applies channel / message-type / sender / keyword filters, then fans theresult out across four dedicated handles (`message`, `media`, `contact`,`metadata`) so downstream agents can grab just the slice they need.-The dispatcher in `NodeExecutor._dispatch` special-cases `socialReceive`-(node_executor.py:367-374) to collect upstream outputs via-`_get_connected_outputs_with_info` and inject them as `outputs` / `source_nodes`.+`socialReceive` is in the executor's `_NEEDS_CONNECTED_OUTPUTS` set, so the+executor injects upstream outputs as `connected_outputs` / `source_nodes` into+the execution context. `SocialReceiveNode.execute` overrides the default+envelope wrap (reading `context.raw["connected_outputs"]` and+`context.raw["source_nodes"]`) so the `filtered` / `reason` keys the handler+emits survive to the top-level result dict.## Inputs (handles)-| Handle | Connection type | Required | Purpose |-|--------|-----------------|----------|---------|-| `input-main` | main | yes | Upstream trigger output (`whatsappReceive`, `telegramReceive`, `chatTrigger`, ...) - this is what gets normalized |--The executor resolves upstream outputs keyed by source node **type**, so the-handler walks `source_nodes` and looks up `outputs[source_type]`.+The plugin declares **no explicit input handle** (`component_kind = "agent"`,+the `handles` tuple lists only the four right-side output handles). Upstream+trigger outputs reach the node via the executor's `connected_outputs` injection+(`socialReceive` is in `_NEEDS_CONNECTED_OUTPUTS`), keyed by source node+**type**, so the handler walks `source_nodes` and looks up+`connected_outputs[source_type]`. Typical sources: `whatsappReceive`,+`telegramReceive`, `chatTrigger`.## Parameters+All field names are snake_case (no camelCase aliases on `SocialReceiveParams`).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `channelFilter` | options | `all` | no | - | Only allow messages whose detected `channel` matches |-| `messageTypeFilter` | options | `all` | no | - | Filter by `message_type` |-| `senderFilter` | options | `all` | no | - | `all` / `any_contact` / `contact` / `group` / `keywords` |-| `contactPhone` | string | `""` | yes when `senderFilter=contact` | `senderFilter: ['contact']` | Exact phone compare |-| `groupId` | string | `""` | yes when `senderFilter=group` | `senderFilter: ['group']` | Exact chat_id compare |-| `keywords` | string | `""` | no | `senderFilter: ['keywords']` | Comma-separated, lower-case substring match |-| `ignoreOwnMessages` | boolean | `true` | no | - | Drop messages where `is_from_me=True` |-| `ignoreBots` | boolean | `false` | no | - | Drop messages where `is_bot=True` |+| `channel_filter` | options | `all` | no | - | `all`/`whatsapp`/`telegram`/`discord`/`slack`/`signal`/`sms`/`webchat`/`email`/`matrix`/`teams` - exact match against detected `channel` |+| `message_type_filter` | options | `all` | no | - | `all`/`text`/`image`/`video`/`audio`/`document`/`sticker`/`location`/`contact`/`poll`/`reaction` |+| `sender_filter` | options | `all` | no | - | `all`/`any_contact`/`contact`/`group`/`keywords` |+| `contact_phone` | string | `""` | yes when `sender_filter=contact` | `sender_filter: ['contact']` | Exact phone compare |+| `group_id` | string | `""` | yes when `sender_filter=group` | `sender_filter: ['group']` | Exact chat_id compare |+| `keywords` | string | `""` | no | `sender_filter: ['keywords']` | Comma-separated, lower-case substring match |+| `ignore_own_messages` | boolean | `true` | no | - | Drop messages where `is_from_me=True` |+| `ignore_bots` | boolean | `false` | no | - | Drop messages where `is_bot=True` |+| `include_media_data` | boolean | `false` | no | - | Include base64 media payload (memory-intensive) |## Outputs (handles)`NodeExecutor` stores the handler's `result.result` under `output_main`**and** splits it into four additional output-store keys so different-downstream handles can consume different slices (node_executor.py:292-296):+downstream handles can consume different slices (node_executor.py:188-192):| Handle / key | Shape | Description ||--------------|-------|-------------|@@ -105,7 +112,7 @@ When a message is filtered out the handler returns```mermaidflowchart TD- A[_dispatch specialcase] --> A1[_get_connected_outputs_with_info]+ A[SocialReceiveNode.execute] --> A1[read context.raw connected_outputs<br/>+ source_nodes]A1 --> B[handle_social_receive]B --> C[Scan source_nodes,<br/>pick first output with<br/>message_id / text / message]C -- none --> C2[Fallback: iterate outputs dict]@@ -141,16 +148,16 @@ flowchart TD- **Chat type**: `_determine_chat_type` returns `data.chat_type` if set, else`"group"` if `is_group` else `"dm"`.- **Filters** (`_apply_filters`):- - `channelFilter`: exact match against normalized `channel`.- - `messageTypeFilter`: exact match against `message_type`.- - `senderFilter=any_contact`: rejects group messages only.- - `senderFilter=contact`: rejects when `sender_phone != contactPhone` (only- if `contactPhone` is truthy).- - `senderFilter=group`: rejects when `chat_id != groupId` (only if `groupId` is truthy).- - `senderFilter=keywords`: lower-cased substring match on the `text`; empty+ - `channel_filter`: exact match against normalized `channel`.+ - `message_type_filter`: exact match against `message_type`.+ - `sender_filter=any_contact`: rejects group messages only.+ - `sender_filter=contact`: rejects when `sender_phone != contact_phone` (only+ if `contact_phone` is truthy).+ - `sender_filter=group`: rejects when `chat_id != group_id` (only if `group_id` is truthy).+ - `sender_filter=keywords`: lower-cased substring match on the `text`; empty`keywords` accepts all.- - `ignoreOwnMessages` (default `true`): drops `is_from_me`.- - `ignoreBots`: drops `is_bot`.+ - `ignore_own_messages` (default `true`): drops `is_from_me`.+ - `ignore_bots`: drops `is_bot`.- **Message id synthesis**: If upstream omits `message_id`, a random 8-charhex (`uuid4().hex[:8]`) is assigned.- **Webchat defaults**: When `source_channel == 'webchat'`, missing `sender` is@@ -187,7 +194,7 @@ flowchart TDenvelope is `success=true, result=null, filtered=true`. Downstream nodesreceive `null` via the output store - anything that expects a dict musthandle this.-- **`_apply_filters` default for `ignoreOwnMessages` is `True`**: Loopback+- **`_apply_filters` default for `ignore_own_messages` is `True`**: Loopbackmessages (echo bots, self-tests) are silently dropped unless explicitlyturned off in the UI.- **Top-level spread**: `{ ...unified_message }` overwrites the dedicated@@ -199,13 +206,15 @@ flowchart TD- **UUID `message_id` on fallback**: Not cryptographically unique acrosssessions; two workflows receiving chatTrigger messages at the same timecan generate colliding ids.-- **`source_nodes` parameter typo risk**: Handler accepts `outputs=None` and- `source_nodes=None` and falls back to `context['outputs']` / `[]` - useful- for direct unit tests but also means a broken executor path would silently+- **Empty connected outputs**: `execute` reads `context.raw["connected_outputs"]`+ and `["source_nodes"]`, defaulting to `{}` / `[]`. The handler still accepts+ `outputs=None` / `source_nodes=None` and falls back to `context["outputs"]` -+ useful for direct unit tests, but a broken executor injection would silentlyreturn "No message data received" instead of failing fast.## Related- **Sibling nodes**: [`socialSend`](./socialSend.md), [`telegramReceive`](./telegramReceive.md)- **Upstream triggers**: [`telegramReceive`](./telegramReceive.md); WhatsApp/Chat triggers live in other categories-- **Dispatcher special-case**: `server/services/node_executor.py:287-296, 367-374`+- **Shared handler body**: [`server/nodes/social/_base.py`](../../../server/nodes/social/_base.py)+- **Output handle split**: `server/services/node_executor.py:188-192` (`_NEEDS_CONNECTED_OUTPUTS` injection + 4-key split)diff --git a/docs-internal/node-logic-flows/telegram_social/socialSend.md b/docs-internal/node-logic-flows/telegram_social/socialSend.mdindex da96a8c3..ee786b82 100644--- a/docs-internal/node-logic-flows/telegram_social/socialSend.md+++ b/docs-internal/node-logic-flows/telegram_social/socialSend.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | social / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/social.py::handle_social_send`](../../../server/services/handlers/social.py) |+| **Backend handler** | plugin [`server/nodes/social/social_send/__init__.py`](../../../server/nodes/social/social_send/__init__.py) (`SocialSendNode`, dispatch via `BaseNode.execute()` -> `@Operation("send")`) delegating to [`server/nodes/social/_base.py::handle_social_send`](../../../server/nodes/social/_base.py) || **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) || **Skill (if any)** | none |-| **Dual-purpose tool** | yes - works as a workflow node and as an AI Agent tool |+| **Dual-purpose tool** | yes - `group = ("social", "tool")`, `usable_as_tool = True`; works as a workflow node and as an AI Agent tool |## Purpose@@ -18,69 +18,96 @@ and surfaces as a failed envelope.## Inputs (handles)+`component_kind = "agent"` (multi-handle layout). Four left-side input handles+(all `role: main`):+| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| `input-main` | main | no | Upstream data; consumed only via template resolution on individual params (e.g. `message`) |+| `input-message` | main | no | Upstream message text (consumed via template resolution on `message`) |+| `input-media` | main | no | Upstream media payload |+| `input-contact` | main | no | Upstream contact payload |+| `input-metadata` | main | no | Upstream metadata payload |## Parameters-| Name | Type | Default | Required | displayOptions.show | Description |-|------|------|---------|----------|---------------------|-------------|-| `channel` | options | `whatsapp` | no | - | Destination platform - only `whatsapp` is wired today |-| `recipientType` | options | `phone` | no | - | `phone` / `group` / `channel` / `user` / `chat` |-| `phone` | string | `""` | yes when `recipientType=phone` | `recipientType: ['phone']` | Destination phone (WhatsApp style incl. country code) |-| `groupId` | string | `""` | yes when `recipientType=group` | `recipientType: ['group']` | Group JID |-| `channelId` | string | `""` | yes when `recipientType=channel` | `recipientType: ['channel']` | Newsletter/channel JID |-| `userId` | string | `""` | yes when `recipientType=user` | `recipientType: ['user']` | Platform user id |-| `chatId` | string | `""` | yes when `recipientType=chat` | `recipientType: ['chat']` | Generic chat id |-| `messageType` | options | `text` | no | - | `text`/`image`/`video`/`audio`/`document`/`sticker`/`location`/`contact`/`poll`/`buttons`/`list` |-| `message` | string | `""` | yes when `messageType=text` | `messageType: ['text']` | Message body |-| `mediaSource` | options | `url` | no | `messageType: ['image','video','audio','document','sticker']` | One of `url`/`base64`/`file` |-| `mediaUrl` / `mediaData` / `filePath` | string | `""` | depends on `mediaSource` | (see above) | Media payload |-| `mimeType`, `caption`, `filename` | string | `""` | no | media types | Extra media fields |-| `latitude`, `longitude`, `locationName`, `address` | mixed | - | yes for location | `messageType: ['location']` | Location fields |-| `contactName`, `vcard` | string | `""` | yes for contact | `messageType: ['contact']` | vCard fields |-| `replyToMessage` | boolean | `false` | no | - | Send as reply |-| `replyMessageId` | string | `""` | yes when `replyToMessage=true` | - | Message id to reply to |+All field names are snake_case (the `SocialSendParams` model uses no camelCase+aliases). 40 fields total; abbreviated below by group.++| Name | Type | Default | displayOptions.show | Description |+|------|------|---------|---------------------|-------------|+| `channel` | options | `whatsapp` | - | `whatsapp`/`telegram`/`discord`/`slack`/`signal`/`sms`/`webchat`/`email`/`matrix`/`teams` - only `whatsapp` is wired |+| `recipient_type` | options | `phone` | - | `phone`/`group`/`channel`/`user`/`chat` |+| `phone` | string | `""` | `recipient_type: ['phone']` | Destination phone (no `+` prefix) |+| `group_id` | string | `""` | `recipient_type: ['group']` | Group identifier |+| `channel_id` | string | `""` | `recipient_type: ['channel']` | Channel identifier |+| `user_id` | string | `""` | `recipient_type: ['user']` | User identifier |+| `chat_id` | string | `""` | `recipient_type: ['chat']` | Generic chat identifier |+| `thread_id` | string | `""` | - | Thread to reply in (optional) |+| `message_type` | options | `text` | - | `text`/`image`/`video`/`audio`/`document`/`sticker`/`location`/`contact`/`poll`/`buttons`/`list` |+| `message` | string | `""` | `message_type: ['text']` | Message body |+| `format` | options | `plain` | `message_type: ['text']` | `plain`/`markdown`/`html` |+| `media_source` | options | `url` | media types | `url`/`base64`/`file` |+| `media_url` / `media_data` / `file_path` | string | `""` | media types + matching `media_source` | Media payload |+| `mime_type`, `caption`, `filename` | string | `""` | media types | Extra media fields |+| `latitude`, `longitude` | float | `0.0` | `message_type: ['location']` | Location coords |+| `location_name`, `address` | string | `""` | `message_type: ['location']` | Location text |+| `contact_name`, `contact_phone`, `vcard` | string | `""` | `message_type: ['contact']` | Contact fields |+| `poll_question`, `poll_options` | string | `""` | `message_type: ['poll']` | Poll text |+| `poll_allow_multiple` | boolean | `false` | `message_type: ['poll']` | Multi-select |+| `button_text`, `buttons` | string | `""` / `"[]"` | `message_type: ['buttons']` | Buttons (JSON array) |+| `list_title`, `list_button_text`, `list_sections` | string | `""` / `"View Options"` / `"[]"` | `message_type: ['list']` | List message |+| `reply_to_message` | boolean | `false` | - | Quote an existing message |+| `reply_message_id` | string | `""` | `reply_to_message: [true]` | Message id to reply to |+| `reply_to_current` | boolean | `false` | `reply_to_message: [true]` | Reply to the triggering message |+| `audio_as_voice` | boolean | `false` | `message_type: ['audio']` | Send audio as voice |+| `disable_preview` | boolean | `false` | `message_type: ['text']` | Disable link preview |+| `silent` | boolean | `false` | - | Send without notification |+| `protect_content` | boolean | `false` | - | Prevent forwarding/saving |## Outputs (handles)-| Handle | Shape | Description |-|--------|-------|-------------|-| `output-main` | object | Send result envelope |-| `output-tool` | object | Same payload, emitted when wired to an AI agent's `input-tools` |+The plugin declares no explicit output handle (the `handles` tuple lists only+the four input handles); the operation result flows out through the default+`output_main` store key. When wired into an agent's `input-tools` the same+payload is returned to the LLM (`usable_as_tool = True`).++| Handle / key | Shape | Description |+|--------------|-------|-------------|+| `output_main` | object | Send result envelope (validated against `SocialSendOutput`: `sent`/`message_id` + `extra="allow"`) |-### Output payload+### Output payload (`result`)```ts{success: true;message_id: string | null;channel: string; // e.g. 'whatsapp'- recipient: string; // resolved phone / groupId / channelId / userId / chatId+ recipient: string; // resolved phone / group_id / channel_id / user_id / chat_idrecipient_type: string;message_type: string;timestamp: string; // ISO}```-Wrapped in the standard envelope `{ success, node_id, node_type, result, execution_time, timestamp }`.+`handle_social_send` already returns the full `{ success, node_id, node_type,+result, execution_time, timestamp }` envelope; the plugin's `send` op returns+`response["result"]` on success and raises `RuntimeError(error)` otherwise.## Logic Flow```mermaidflowchart TD- A[handle_social_send] --> B[Read channel / recipientType / messageType]- B --> C{recipientType}+ A[SocialSendNode.send op<br/>-> handle_social_send] --> B[Read channel / recipient_type / message_type]+ B --> C{recipient_type}C -- phone --> C1[recipient = params.phone]- C -- group --> C2[recipient = params.groupId]- C -- channel --> C3[recipient = params.channelId]- C -- user --> C4[recipient = params.userId]- C -- chat --> C5[recipient = params.chatId]+ C -- group --> C2[recipient = params.group_id]+ C -- channel --> C3[recipient = params.channel_id]+ C -- user --> C4[recipient = params.user_id]+ C -- chat --> C5[recipient = params.chat_id]C1 & C2 & C3 & C4 & C5 --> D{recipient truthy?}D -- no --> Eval[raise ValueError<br/>caught -> success=false]D -- yes --> E{channel}- E -- whatsapp --> W[_send_via_whatsapp<br/>-> routers.whatsapp.handle_whatsapp_send]+ E -- whatsapp --> W[_send_via_whatsapp<br/>-> get_social_send_handler 'whatsapp']E -- other --> Stub[Return success=false stub<br/>'Platform X is not yet implemented']W --> F{result.success?}Stub --> F@@ -90,9 +117,11 @@ flowchart TD## Decision Logic-- **Recipient resolution**: Hard-coded mapping from `recipientType` to a- specific parameter key. An unknown `recipientType` leaves `recipient = None`- and the handler raises `ValueError("Recipient (<type>) is required")`.+- **Recipient resolution**: Hard-coded mapping from `recipient_type` to a+ specific parameter key (`phone`->`phone`, `group`->`group_id`,+ `channel`->`channel_id`, `user`->`user_id`, `chat`->`chat_id`). An unknown+ `recipient_type` leaves `recipient = None` and the handler raises+ `ValueError("Recipient (<type>) is required")`.- **Channel dispatch**: Python `if/elif` on `channel`. Only `"whatsapp"` iswired. Every other value returns`{"success": False, "error": "Platform '<x>' is not yet implemented"}`@@ -112,8 +141,9 @@ flowchart TDto may record usage.- **Broadcasts**: none from `handle_social_send` itself; broadcasts originatein the underlying platform handler.-- **External API calls**: delegated to `routers.whatsapp.handle_whatsapp_send`- for the `whatsapp` channel, which hits the WhatsApp RPC service+- **External API calls**: delegated to the WhatsApp send handler resolved via+ `get_social_send_handler("whatsapp")` (the whatsapp plugin self-registers it+ from `nodes/whatsapp/__init__.py`), which hits the WhatsApp RPC service(default `http://localhost:9400`).- **File I/O**: none from this handler. `mediaSource=file` reads are delegated.- **Subprocess**: none.@@ -122,8 +152,8 @@ flowchart TD- **Credentials**: none at this layer - the WhatsApp handler manages its ownRPC connection.-- **Services**: `routers.whatsapp.handle_whatsapp_send` (imported lazily- inside `_send_via_whatsapp`).+- **Services**: the registered `"whatsapp"` social-send handler resolved via+ `get_social_send_handler("whatsapp")` inside `_send_via_whatsapp`.- **Python packages**: stdlib only (`time`, `datetime`, `typing`).- **Environment variables**: none.@@ -139,17 +169,18 @@ flowchart TDhandle, `recipient` stays `None` and the ValueError message reports thereceived `recipient_type` string back to the user. Good for debugging, badfor LLM tool use (the LLM then retries with a different type).-- **`groupId` used for non-phone `recipient_type`**: Inside `_send_via_whatsapp`,+- **`group_id` used for non-phone `recipient_type`**: Inside `_send_via_whatsapp`,anything that is not `phone` maps onto `group_id` in the WhatsApp params,including `channel`/`user`/`chat`. This means channels/users/chats end upbeing sent as WhatsApp groups, which the RPC will likely reject.-- **camelCase vs snake_case drift**: Node parameters are mostly camelCase- (`recipientType`, `messageType`, `mediaUrl`) while the WhatsApp handler- expects snake_case; the conversion in `_send_via_whatsapp` is manual and- easy to drift from new parameters added on either side.+- **Manual param mapping**: `_send_via_whatsapp` rebuilds the whatsapp-send param+ dict field-by-field from `SocialSendParams`. Both schemas are snake_case now,+ but the mapping is still hand-written and can drift when new fields are added+ on either side.## Related- **Sibling nodes**: [`socialReceive`](./socialReceive.md), [`telegramSend`](./telegramSend.md)-- **Downstream handler**: `routers.whatsapp.handle_whatsapp_send`+- **Downstream handler**: registered `"whatsapp"` social-send handler (`nodes/whatsapp/__init__.py`)+- **Shared handler body**: [`server/nodes/social/_base.py`](../../../server/nodes/social/_base.py)- **Architecture docs**: WhatsApp Integration section in the root [`CLAUDE.md`](../../../CLAUDE.md)diff --git a/docs-internal/node-logic-flows/telegram_social/telegramReceive.md b/docs-internal/node-logic-flows/telegram_social/telegramReceive.mdindex 1eb1668a..291d6a5b 100644--- a/docs-internal/node-logic-flows/telegram_social/telegramReceive.md+++ b/docs-internal/node-logic-flows/telegram_social/telegramReceive.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | social / trigger |-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) + [`server/services/event_waiter.py::build_telegram_filter`](../../../server/services/event_waiter.py) |+| **Backend handler** | [`server/nodes/telegram/telegram_receive.py`](../../../server/nodes/telegram/telegram_receive.py) (`TelegramReceiveNode`, a `TriggerNode` with `event_type = "telegram_message_received"`); filter via `build_filter` -> [`build_telegram_filter`](../../../server/services/event_waiter.py) || **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) || **Skill (if any)** | none || **Dual-purpose tool** | no (trigger only) |@@ -11,12 +11,14 @@## PurposeWait for an incoming Telegram message and emit it as the workflow trigger-event. Implementation is split between the generic `handle_trigger_node`-(registers a waiter and blocks on `event_waiter.wait_for_event`) and the-Telegram-specific filter builder that pre-compiles a `matches()` closure-from the node parameters. Events are produced by the `TelegramService`-long-polling loop inside `_on_message_received` and dispatched via-`event_waiter.dispatch("telegram_message_received", <event_data>)`.+event. `TelegramReceiveNode` subclasses `TriggerNode`; its `execute` override+first refuses to register if the bot is not connected, then defers to+`TriggerNode.execute`. The Telegram-specific `build_filter` pre-compiles a+`matches()` closure (via the legacy `build_telegram_filter`) from the node+parameters. Events are produced by the `TelegramService` long-polling loop+inside `_on_message_received`. In deployment mode this trigger runs on the+canary path (`TriggerListenerWorkflow` per deployment/trigger, fed via the+event framework); the registered filter narrows the stream.## Inputs (handles)@@ -24,17 +26,19 @@ Trigger node - no inputs.## Parameters+All field names are snake_case (no camelCase aliases on the Pydantic model).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `contentTypeFilter` | options | `all` | no | - | One of `all`/`text`/`photo`/`video`/`audio`/`voice`/`document`/`sticker`/`location`/`contact`/`poll` |-| `senderFilter` | options | `all` | no | - | `all`/`self`/`private`/`group`/`supergroup`/`channel`/`specific_chat`/`specific_user`/`keywords` |-| `chat_id` | string | `""` | yes when `senderFilter=specific_chat` | `senderFilter: ['specific_chat']` | Numeric chat id or `@username` |-| `from_user` | string | `""` | yes when `senderFilter=specific_user` | `senderFilter: ['specific_user']` | Numeric user id |-| `keywords` | string | `""` | yes when `senderFilter=keywords` | `senderFilter: ['keywords']` | Comma-separated, matched case-insensitively |-| `ignoreBots` | boolean | `true` | no | - | Skip messages where `is_bot=True` (ignored when `senderFilter=self`) |+| `content_type_filter` | options | `all` | no | - | One of `all`/`text`/`photo`/`video`/`audio`/`voice`/`document`/`sticker`/`location`/`contact`/`poll` |+| `sender_filter` | options | `all` | no | - | `all`/`self`/`private`/`group`/`supergroup`/`channel`/`specific_chat`/`specific_user`/`keywords` |+| `chat_id` | string | `""` | yes when `sender_filter=specific_chat` | `sender_filter: ['specific_chat']` | Numeric chat id or `@username` |+| `from_user` | string | `""` | yes when `sender_filter=specific_user` | `sender_filter: ['specific_user']` | Numeric user id |+| `keywords` | string | `""` | yes when `sender_filter=keywords` | `sender_filter: ['keywords']` | Comma-separated, matched case-insensitively |+| `ignore_bots` | boolean | `true` | no | shown for every `sender_filter` except `self` | Skip messages where `is_bot=True` (ignored when `sender_filter=self`) |-Legacy alias: `chatTypeFilter` is still read when `senderFilter` is absent and-reconstructs the new-style value.+Legacy alias: `chatTypeFilter` is still read by `build_telegram_filter` when+`sender_filter` is absent and reconstructs the new-style value.## Outputs (handles)@@ -73,11 +77,11 @@ Wrapped in the standard success envelope when an event is received.```mermaidflowchart TD- A[handle_trigger_node] --> B{TelegramService.connected?}- B -- no --> Eoff[Return success=false<br/>error: bot not connected]- B -- yes --> C[event_waiter.register<br/>-> build_telegram_filter]+ A[TelegramReceiveNode.execute] --> B{TelegramService.connected?}+ B -- no --> Eoff[_wrap_error<br/>error: bot not connected]+ B -- yes --> C[TriggerNode.execute<br/>register waiter via build_filter]C --> D[broadcaster.update_node_status waiting]- D --> E[await event_waiter.wait_for_event]+ D --> E[await event delivery]E -- CancelledError --> Ecx[Return success=false<br/>error: Cancelled by user]E -- event resolved --> F[Return success=true<br/>result = event_data]@@ -92,8 +96,9 @@ flowchart TD## Decision Logic-- **Pre-check**: Handler inspects `TelegramService.connected` before registering- a waiter. Prevents hanging forever when the bot token was never added.+- **Pre-check**: `execute` inspects `TelegramService.connected` before deferring+ to `TriggerNode.execute`. Prevents hanging forever when the bot token was+ never added.- **Sender filter branches** (inside `build_telegram_filter.matches`):- `self`: compares `m.from_id` to a lazily-resolved `owner_chat_id`. If noowner is known, the message is rejected (but no error surfaces to the@@ -104,9 +109,9 @@ flowchart TD- `all`: accepts everything (subject to `ignore_bots`).- **Content type filter**: Always applied first; if set and not `all`, rejectsanything with a non-matching `content_type`.-- **`ignore_bots`**: Applied last; silently skipped when `senderFilter == 'self'`.-- **Legacy fallback**: If `senderFilter` is unset the code reconstructs it- from `chat_id`/`from_user`/`keywords`/`chatTypeFilter` so old workflows keep+- **`ignore_bots`**: Applied last; silently skipped when `sender_filter == 'self'`.+- **Legacy fallback**: If `sender_filter` is unset the filter builder reconstructs+ it from `chat_id`/`from_user`/`keywords`/`chatTypeFilter` so old workflows keepworking.## Side Effects@@ -139,13 +144,10 @@ flowchart TDmatches `"chicken"`. Empty `keywords` accepts all messages.- **String comparison for IDs**: `specific_chat`/`specific_user` compare via`str(...)`. A leading `@` or whitespace difference will silently mismatch.-- **`ignore_bots` override**: Hard-coded to be bypassed for `senderFilter=self`+- **`ignore_bots` override**: Hard-coded to be bypassed for `sender_filter=self`so bot-owners who are themselves bots still match.- **Cancellation path**: If the trigger is cancelled mid-wait, the handlerreturns `success=false, error="Cancelled by user"` rather than swallowing.-- **Handler accepts unknown trigger types**: `handle_trigger_node` has a guard- for missing `TriggerConfig`, but because the registry entry for- `telegramReceive` is hard-coded this branch is unreachable in practice.- **No timeout**: The node waits indefinitely. The only exit routes are anevent match, an explicit `cancel_event_wait` WebSocket call, or the serverrestarting.@@ -154,4 +156,4 @@ flowchart TD- **Sibling nodes**: [`telegramSend`](./telegramSend.md), [`socialReceive`](./socialReceive.md)- **Event waiter architecture**: [Event Waiter System](../../event_waiter_system.md)-- **Service**: [`server/services/telegram_service.py`](../../../server/services/telegram_service.py)+- **Service**: [`server/nodes/telegram/_service.py`](../../../server/nodes/telegram/_service.py)diff --git a/docs-internal/node-logic-flows/telegram_social/telegramSend.md b/docs-internal/node-logic-flows/telegram_social/telegramSend.mdindex 4fc434cc..dbb16314 100644--- a/docs-internal/node-logic-flows/telegram_social/telegramSend.md+++ b/docs-internal/node-logic-flows/telegram_social/telegramSend.md@@ -2,19 +2,19 @@| Field | Value ||------|-------|-| **Category** | social / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/telegram.py::handle_telegram_send`](../../../server/services/handlers/telegram.py) |+| **Category** | social (workflow-only) |+| **Backend handler** | [`server/nodes/telegram/telegram_send.py`](../../../server/nodes/telegram/telegram_send.py) (`TelegramSendNode`); dispatch via `BaseNode.execute()` -> `@Operation("send")` || **Tests** | [`server/tests/nodes/test_telegram_social.py`](../../../server/tests/nodes/test_telegram_social.py) || **Skill (if any)** | none |-| **Dual-purpose tool** | yes - group is `['social', 'tool']`; exposed to AI agents via the unified send path |+| **Dual-purpose tool** | no - group is `("social",)` only; `usable_as_tool` not set (AI-tool exposure was dropped in Wave 11) |## PurposeSend text, photo, document, location, or contact messages through a connectedTelegram bot (python-telegram-bot v22.x). The node leans on the `TelegramService`-singleton for the actual Bot API calls; the handler only does recipient-resolution, parameter validation, and envelope packaging. When used as an AI-tool, the LLM fills the same snake_case parameter schema documented below.+singleton for the actual Bot API calls; the `send` operation only does recipient+resolution, parameter validation, and envelope packaging. The operation body is+inlined directly in the plugin file (no `handlers/telegram.py` shim).## Inputs (handles)@@ -32,8 +32,8 @@ tool, the LLM fills the same snake_case parameter schema documented below.| `text` | string | `""` | yes when `message_type=text` | `message_type: ['text']` | Message text || `media_url` | string | `""` | yes when `message_type` in `photo`/`document` | `message_type: ['photo','document']` | Remote URL or `file_id` || `caption` | string | `""` | no | `message_type: ['photo','document']` | Optional caption |-| `latitude` | number | `0` | yes when `message_type=location` | `message_type: ['location']` | Latitude |-| `longitude` | number | `0` | yes when `message_type=location` | `message_type: ['location']` | Longitude |+| `latitude` | number (Optional[float], default `None`) | `None` | yes when `message_type=location` | `message_type: ['location']` | Latitude. `None` (not `0`) so deliberate `0.0` (Null Island) is distinguishable from "unset" |+| `longitude` | number (Optional[float], default `None`) | `None` | yes when `message_type=location` | `message_type: ['location']` | Longitude || `phone_number` | string | `""` | yes when `message_type=contact` | `message_type: ['contact']` | Contact phone || `first_name` | string | `""` | yes when `message_type=contact` | `message_type: ['contact']` | Contact first name || `last_name` | string | `""` | no | `message_type: ['contact']` | Contact last name |@@ -46,10 +46,12 @@ tool, the LLM fills the same snake_case parameter schema documented below.| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Standard envelope containing Bot API message metadata |-| `output-tool` | object | Same payload, emitted when wired into an AI agent's `input-tools` |### Output payload+The operation returns this dict (validated against `TelegramSendOutput`, which+declares `message_id` / `chat_id` / `sent` and `extra="allow"`):+```ts{message_id: number;@@ -59,14 +61,14 @@ tool, the LLM fills the same snake_case parameter schema documented below.}```-Wrapped in `{ success: true, result: <payload>, execution_time, timestamp, node_id, node_type }`.+`BaseNode.execute()` wraps it as `{ success: true, result: <payload>, execution_time, timestamp, node_id, node_type }`.## Logic Flow```mermaidflowchart TD- A[handle_telegram_send] --> B{TelegramService<br/>connected?}- B -- no --> Efail[Return success=false<br/>error: bot not connected]+ A[TelegramSendNode.send op] --> B{TelegramService<br/>connected?}+ B -- no --> Efail[raise RuntimeError<br/>BaseNode wraps as success=false]B -- yes --> C{recipient_type?}C -- self --> D[chat_id = service.owner_chat_id]D --> D2{owner set?}@@ -86,7 +88,7 @@ flowchart TDG -- document --> Doc{media_url set?}Doc -- no --> EfailDoc -- yes --> Dsend[service.send_document]- G -- location --> L{lat and lng set?}+ G -- location --> L{lat and lng<br/>not None?}L -- no --> EfailL -- yes --> Lsend[service.send_location]G -- contact --> Ct{phone + first_name?}@@ -103,32 +105,34 @@ flowchart TD## Decision Logic-- **Connection gate**: If `service.connected` is False the handler short-circuits with- `"Telegram bot not connected. Add bot token in Credentials."` No retry is attempted.+- **Connection gate**: If `service.connected` is False the op raises+ `RuntimeError("Telegram bot not connected. Add bot token in Credentials.")`+ (wrapped by `BaseNode.execute()` into a `success=false` envelope). No retry.- **Self recipient**: Uses `service.owner_chat_id` (auto-captured on the first- private message). If not set, the handler tries- `auth_service.get_api_key("telegram_owner_chat_id")` and restores the owner by- calling `service.set_owner(int(saved_owner))`. Any exception during restore is- logged at WARNING and swallowed - the handler continues and ultimately returns- the "Bot owner not detected" error envelope when the chat_id is still falsy.+ private message). If not set, the op tries+ `get_auth_service().get_api_key("telegram_owner_chat_id")` and restores the owner by+ calling `service.set_owner(int(saved))`. Any exception during restore is+ logged at WARNING and swallowed - the op continues and ultimately raises+ the "Bot owner not detected" `RuntimeError` when the chat_id is still falsy.- **Parse mode**: Passed through to the service. The service itself implements`Auto` (GFM -> Telegram HTML via `markdown_formatter.to_telegram_html`) plus the`BadRequest "can't parse entities"` fallback that retries with `parse_mode=None`and the original unescaped text.-- **Message type dispatch**: Uses a Python `match` statement. An unknown- `message_type` falls into the `_` arm and returns- `"Unsupported message type: <x>"`.+- **Message type dispatch**: Uses an `if/elif/else` chain on `message_type`. An+ unknown value falls into the `else` and raises+ `RuntimeError("Unsupported message type: <x>")`.- **Reply-to coercion**: `reply_to_message_id` is cast via `int(...)` only when- truthy. A non-numeric string here raises `ValueError`, which is caught by the- outer `except Exception` and returned as an error envelope.+ truthy. A non-numeric string here raises `ValueError`, which `BaseNode.execute()`+ catches and surfaces as an error envelope.## Side Effects-- **Database writes**: none from the handler directly. `handle_telegram_send`- can call `auth_service.store_api_key("telegram_owner_chat_id", ...)` indirectly- only through `TelegramService._on_message_received`, not from the send path.-- **Broadcasts**: none from the send handler. The service broadcasts- `update_telegram_status` on connect / disconnect only.+- **Database writes**: none from the op directly. The owner is persisted via+ `auth_service.store_api_key("telegram_owner_chat_id", ...)` only inside+ `TelegramService._on_message_received`, not from the send path.+- **Broadcasts**: none from the send op. The service broadcasts the typed+ `telegram.status` CloudEvents envelope (wire key `telegram_status`) on+ connect / disconnect only - see `server/nodes/telegram/_events.py`.- **External API calls**: Telegram Bot API via `python-telegram-bot` (`bot.send_message`,`bot.send_photo`, `bot.send_document`, `bot.send_location`, `bot.send_contact`).- **File I/O**: none.@@ -138,8 +142,9 @@ flowchart TD- **Credentials**: `auth_service.get_api_key("telegram_owner_chat_id")` whenrestoring the self-owner. Bot token comes from `auth_service.store_api_key("telegram", ...)`- and is consumed by `TelegramService.connect()` elsewhere, not by this handler.-- **Services**: `TelegramService` singleton (`services.telegram_service.get_telegram_service`).+ and is consumed by `TelegramService.connect()` elsewhere, not by this op.+ Declared on the plugin via `credentials = (TelegramCredential,)`.+- **Services**: `TelegramService` singleton (`server/nodes/telegram/_service.py::get_telegram_service`).- **Python packages**: `python-telegram-bot` (v22.x), `markdown-it-py` (via`markdown_formatter`).- **Environment variables**: `TELEGRAM_OWNER_CHAT_ID` (read by the service as a@@ -151,11 +156,10 @@ flowchart TD`telegram_owner_chat_id` is logged at WARNING and continues; the user onlysees the downstream "Bot owner not detected" error.- **Silent ValueError on `reply_to_message_id`**: A non-numeric value raises- and the outer `except Exception` returns `str(e)` as the error message.-- **`recipient_type` spelling**: The handler treats anything that is not- literally `"self"` as needing a `chat_id`, including unexpected values -- this means a typo like `"slf"` falls through to the user/group branch and- the user sees `"chat_id is required"` instead of a recipient-type error.+ `ValueError`, surfaced by `BaseNode.execute()` as the error message.+- **`recipient_type` is a `Literal["self","user","group"]`**: Pydantic rejects+ any other value at param validation. For `user`/`group` an empty `chat_id`+ raises `RuntimeError("chat_id is required")`.- **Markdown fallback leaks unescaped text**: If `send_message` raises`BadRequest("can't parse entities ...")`, the service retries with theORIGINAL (unescaped) text at `parse_mode=None`, not the escaped copy. If the@@ -170,6 +174,6 @@ flowchart TD## Related- **Sibling nodes**: [`telegramReceive`](./telegramReceive.md), [`socialSend`](./socialSend.md)-- **Service**: [`server/services/telegram_service.py`](../../../server/services/telegram_service.py)+- **Service**: [`server/nodes/telegram/_service.py`](../../../server/nodes/telegram/_service.py)- **Markdown formatter**: [`server/services/markdown_formatter.py`](../../../server/services/markdown_formatter.py)- **Architecture docs**: [Credentials Encryption](../../credentials_encryption.md), [Status Broadcaster](../../status_broadcaster.md)diff --git a/docs-internal/node-logic-flows/twitter/twitterReceive.md b/docs-internal/node-logic-flows/twitter/twitterReceive.mdindex 8fdbc732..8edd30ec 100644--- a/docs-internal/node-logic-flows/twitter/twitterReceive.md+++ b/docs-internal/node-logic-flows/twitter/twitterReceive.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | social / trigger |-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) + [`server/services/event_waiter.py::build_twitter_filter`](../../../server/services/event_waiter.py) |+| **Backend handler** | Plugin [`server/nodes/twitter/twitter_receive/__init__.py`](../../../server/nodes/twitter/twitter_receive/__init__.py); `execute()` delegates to [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic). Filter: [`server/nodes/twitter/_filters.py::build_filter`](../../../server/nodes/twitter/_filters.py). Events: [`server/nodes/twitter/_events.py`](../../../server/nodes/twitter/_events.py). || **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) || **Skill (if any)** | none || **Dual-purpose tool** | no (trigger only) |@@ -18,8 +18,17 @@ no push webhooks on the free tier); polling is driven by the deployment layerregister a waiter, block on `event_waiter.wait_for_event`, and surface thematching event.-Filtering is done by the closure returned from `build_twitter_filter` in-`event_waiter.py`.+**Canary status**: `twitterReceive` is the lone **deferred** trigger plugin —+it is NOT in the canary registry (`register_canary_trigger_type` is not+called). The Temporal-durable `TriggerListenerWorkflow` / `PollingTriggerWorkflow`+path requires the node to subclass `PollingTriggerNode` and declare the four+hooks (same shape as `GmailReceiveNode`); until that refactor lands it runs on+the legacy `event_waiter` collector/processor path. `_events.py` carries the+typed `WorkflowEvent` factory + dual-path dispatcher (`dispatch_twitter_event_received`)+ready for canary opt-in.++Filtering is done by the closure returned from `build_filter` in+`nodes/twitter/_filters.py` (auto-registered into `event_waiter.FILTER_BUILDERS`).## Inputs (handles)@@ -40,13 +49,13 @@ Trigger node - no inputs.| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Raw event data returned by `event_waiter.wait_for_event` |+| `output-main` | `TwitterReceiveOutput` | Raw event data returned by `event_waiter.wait_for_event`. The declared `Output` model (`tweet_id`, `text`, `author`, plus `extra="allow"`) passes through the full dispatched payload. |### Output payload (contract)The event shape is produced by whichever polling loop dispatches-`twitter_event_received`. Documented fields the filter and downstream nodes-rely on:+`twitter_event_received` (legacy `event_type`, preserved in `_events.py`).+Documented fields the filter and downstream nodes rely on:```ts{@@ -72,7 +81,7 @@ The handler wraps this in `{ success: true, result: <event_data>, execution_timeflowchart TDA[handle_trigger_node] --> B[event_waiter.get_trigger_config twitterReceive]B -- None --> E1[Return success=false<br/>error: Unknown trigger type]- B -- config --> C[event_waiter.register<br/>build_twitter_filter]+ B -- config --> C[event_waiter.register<br/>_filters.build_filter]C --> D[broadcaster.update_node_status waiting]D --> E[await event_waiter.wait_for_event]E -- CancelledError --> Ecx[Return success=false<br/>error: Cancelled by user]@@ -82,15 +91,15 @@ flowchart TDsubgraph ProducerP1[Deployment polling task] --> P2[Fetch via XDK<br/>mentions/search/user_timeline]P2 --> P3[event_waiter.dispatch<br/>twitter_event_received]- P3 --> P4[build_twitter_filter.matches]+ P3 --> P4[_filters.build_filter.matches]P4 -- match --> P5[Waiter.future.set_result]end```## Decision Logic (filter)-`build_twitter_filter(params)` returns a `matches(data)` closure that rejects-non-matching events:+`build_filter(params)` (in `nodes/twitter/_filters.py`) returns a `matches(data)`+closure that rejects non-matching events:- `trigger_type != 'all'` and `data['trigger_type'] != trigger_type` -> reject.(Frontend only exposes `mentions` / `search` / `timeline`; there is no way to@@ -101,7 +110,7 @@ non-matching events:- **user_timeline branch**: if `trigger_type == 'user_timeline'` and `user_id`is set, require `data['user_id'] == user_id` (exact). Empty `user_id`accepts any timeline event.-- No `filter_retweets` / `filter_replies` logic exists in `build_twitter_filter`.+- No `filter_retweets` / `filter_replies` logic exists in `build_filter`.### Execution-side handler branchesdiff --git a/docs-internal/node-logic-flows/twitter/twitterSearch.md b/docs-internal/node-logic-flows/twitter/twitterSearch.mdindex 0711a005..72363026 100644--- a/docs-internal/node-logic-flows/twitter/twitterSearch.md+++ b/docs-internal/node-logic-flows/twitter/twitterSearch.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | social / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/twitter.py::handle_twitter_search`](../../../server/services/handlers/twitter.py) |+| **Backend handler** | [`server/nodes/twitter/twitter_search/__init__.py`](../../../server/nodes/twitter/twitter_search/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("search")` -> `_do_search` (helpers in [`_base.py`](../../../server/nodes/twitter/_base.py)) || **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) || **Skill (if any)** | [`server/skills/social_agent/twitter-search-skill/SKILL.md`](../../../server/skills/social_agent/twitter-search-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `twitter_search` |@@ -40,11 +40,12 @@ The first result page is returned - pagination is not surfaced to the user.| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when invoked via `input-tools` |+| `output-main` | `TwitterSearchOutput` | Search results; also returned to the LLM when invoked via `input-tools` |### Output payload+`_do_search` returns `TwitterSearchOutput(tweets=..., count=..., query=...)` (`extra="allow"`). Validated + dumped by `BaseNode._serialize_result`:+```ts{tweets: Array<{@@ -98,55 +99,56 @@ The first result page is returned - pagination is not surfaced to the user.}```-Wrapped in `{ success, result, execution_time }`.-## Logic Flow```mermaidflowchart TD- A[handle_twitter_search] --> B[_get_twitter_client]- B -- no tokens --> Eauth[Return success=false<br/>error: Twitter not connected]- B --> C{query non-empty?}- C -- no --> Eq[Return success=false<br/>error: Search query is required]+ A[BaseNode.execute -> @Operation search] --> C{query non-empty?}+ C -- no --> Eq[raise RuntimeError<br/>Search query is required]C -- yes --> D[max_results clamp<br/>max 10, min requested, 100]- D --> E[to_thread _sync_search_recent<br/>tweet_fields + expansions + media + user]- E -. 401/403 .-> R[_refresh_and_get_client] --> E- E -. other Exception .-> EGEN[Return success=false<br/>error: str e]- E --> F[Build users_by_id, media_by_key, tweets_by_id<br/>from includes]- F --> G[Map raw tweets -> _format_tweet]+ D --> CWR[call_with_retry _do_search]+ CWR --> B[get_twitter_client]+ B -- no access_token --> Eauth[raise RuntimeError<br/>Twitter not connected]+ B --> E[to_thread sync_search_recent<br/>tweet_fields + expansions + media + user]+ E -. 401/403 .-> R[refresh_and_get_client] --> E+ E -. other Exception .-> EGEN[propagate -> BaseNode error envelope]+ E --> F[includes_lookups -> users_by_id, media_by_key, tweets_by_id]+ F --> G[Map raw tweets -> format_tweet]G --> H{tweets non-empty?}- H -- yes --> K[_track_twitter_usage search count=len tweets]+ H -- yes --> K[track_twitter_usage search count=len tweets]H -- no --> OK- K --> OK[Return success=true<br/>result={tweets, count, query}]+ K --> OK[Return TwitterSearchOutput<br/>tweets, count, query]```## Decision Logic-- **Validation**: missing/empty `query` -> error envelope, no API call.+- **Validation**: missing/empty `query` raises `RuntimeError` in `search()`+ before any client is acquired or API call made.- **max_results clamp**: `max(10, min(params.max_results, 100))`. Requests below10 are silently raised; above 100 are silently capped.-- **SDK call**: `_sync_search_recent` iterates `client.posts.search_recent(...)`+- **SDK call**: `sync_search_recent` iterates `client.posts.search_recent(...)`once and returns `{tweets, includes}` for the first page. Subsequent pagesare discarded.- **Expansions assembled**:- `users_by_id[author_id]` from `includes.users`- `media_by_key[media_key]` from `includes.media`- `tweets_by_id[referenced.id]` from `includes.tweets`-- **`_format_tweet` enrichment**:+- **`format_tweet` enrichment**:- Replaces every `t.co` shortlink in `text` with `expanded_url` in `display_text`.- If `note_tweet.text` exists, it replaces both `text` and `display_text`.- Attaches `author`, `urls`, `media`, `referenced_tweets` as optional keys(only set when data is non-empty).- Coerces Pydantic objects -> dicts via `model_dump()` where needed.-- **Auth refresh**: same lazy-refresh loop as `twitterSend` - a single retry- via `_refresh_and_get_client` on `_is_auth_error(e)`.+- **Auth refresh**: same lazy-refresh loop as `twitterSend` (`call_with_retry`) -+ a single retry via `refresh_and_get_client` on `_is_auth_error(e)`.- **Usage tracking**: `resource_count` equals the number of tweets returned,not 1. Skipped entirely when results are empty.## Side Effects- **Database writes**: one `api_usage_metrics` row per non-empty search (op- `search`, `resource_count=len(tweets)`) via `database.save_api_usage_metric`.+ `search`, `resource_count=len(tweets)`) via `track_twitter_usage` ->+ `database.save_api_usage_metric`.- **Broadcasts**: none.- **External API calls**:- `GET https://api.twitter.com/2/tweets/search/recent` with `expansions=author_id,attachments.media_keys,referenced_tweets.id,referenced_tweets.id.author_id`, `tweet_fields=author_id,created_at,entities,public_metrics,possibly_sensitive,lang,source,conversation_id,in_reply_to_user_id,referenced_tweets,note_tweet`, `media_fields=url,preview_image_url,type,alt_text,variants`, `user_fields=username,name,profile_image_url`.@@ -167,14 +169,14 @@ flowchart TD- **Unused frontend params**: `sort_order`, `start_time`, `end_time`,`include_metrics`, `include_author` are wired in the UI but the handlerdoes not forward any of them to the SDK.-- **Only first page returned**: `_sync_search_recent` breaks out of the+- **Only first page returned**: `sync_search_recent` breaks out of thegenerator after the first page regardless of `next_token`.- **Empty-includes tolerance**: when the API returns no `includes`, `author`,`media`, `urls`, and `referenced_tweets` are simply omitted from each tweet -consumers must treat them as optional.- **Referenced tweet id not in includes**: `referenced_tweets[].text` is`null` rather than raising.-- **Object/dict duality in `includes`**: `_format_tweet` handles both Pydantic+- **Object/dict duality in `includes`**: `format_tweet` handles both Pydanticobjects and raw dicts - changes to XDK's model shapes may silently dropfields.- **Long-form tweets**: `note_tweet.text` replaces the 280-char `text`; callersdiff --git a/docs-internal/node-logic-flows/twitter/twitterSend.md b/docs-internal/node-logic-flows/twitter/twitterSend.mdindex 65275acf..93af92fd 100644--- a/docs-internal/node-logic-flows/twitter/twitterSend.md+++ b/docs-internal/node-logic-flows/twitter/twitterSend.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | social / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/twitter.py::handle_twitter_send`](../../../server/services/handlers/twitter.py) |+| **Backend handler** | [`server/nodes/twitter/twitter_send/__init__.py`](../../../server/nodes/twitter/twitter_send/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("send")` -> `_do_send` (helpers in [`_base.py`](../../../server/nodes/twitter/_base.py)) || **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) || **Skill (if any)** | [`server/skills/social_agent/twitter-send-skill/SKILL.md`](../../../server/skills/social_agent/twitter-send-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `twitter_send` |@@ -28,105 +28,101 @@ wraps every call in `asyncio.to_thread(...)` so the event loop is never blocked.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `action` | options | `tweet` | yes | - | One of `tweet` / `reply` / `retweet` / `quote` / `like` / `unlike` / `delete`. Backend only implements `tweet`, `reply`, `retweet`, `like`, `unlike`, `delete`; `quote` falls through to the `Unknown action` branch. |-| `text` | string | `""` | yes (tweet/reply/quote) | `action: ['tweet','reply','quote']` | Tweet content. Handler truncates to 280 chars via slice. |-| `tweet_id` | string | `""` | yes (reply/retweet/quote/like/unlike/delete) | `action: ['reply','retweet','quote','like','unlike','delete']` | Target tweet ID. |-| `reply_to_id` | string | `""` | yes (reply) | - | ID of the tweet being replied to. Passed in `reply.in_reply_to_tweet_id`. Note the parameter name mismatch vs. frontend `tweet_id` for other actions. |-| `include_media`, `media_urls` | boolean/string | - | no | tweet/reply/quote | **Accepted by frontend but ignored by backend** - the handler never attaches media. |-| `include_poll`, `poll_options`, `poll_duration` | bool/string/number | - | no | tweet | **Accepted by frontend but ignored by backend.** |+| `action` | options (Literal) | `tweet` | yes | - | One of `tweet` / `reply` / `retweet` / `quote` / `like` / `unlike` / `delete`. All implemented in `_do_send`. |+| `text` | string (rows: 4) | `""` | yes (tweet/reply/quote) | `action: ['tweet','reply','quote']` | Tweet content. `_do_send` truncates to 280 chars via slice (`text[:280]`). |+| `tweet_id` | string | `""` | yes (reply/retweet/quote/like/unlike/delete) | `action: ['reply','retweet','quote','like','unlike','delete']` | Target tweet ID. Used by reply (`reply.in_reply_to_tweet_id`), quote (`quote_tweet_id`), and the id-based actions. |+| `include_media` | boolean | `false` | no | `action: ['tweet','reply','quote']` | **Accepted by Params but ignored by `_do_send`** — the operation never attaches media. |+| `media_urls` | string | `""` | no | `action: ['tweet','reply','quote']`, `include_media: [true]` | **Accepted by Params but ignored by `_do_send`.** Comma-separated URLs; max 4 images or 1 video. |+| `include_poll` | boolean | `false` | no | `action: ['tweet']` | **Accepted by Params but ignored by `_do_send`.** |+| `poll_options` | string | `""` | no | `action: ['tweet']`, `include_poll: [true]` | **Accepted but ignored.** Comma-separated 2-4 options, 25 chars each. |+| `poll_duration` | number (5-10080) | `1440` | no | `action: ['tweet']`, `include_poll: [true]` | **Accepted but ignored.** Poll duration in minutes. |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope with XDK response payload |-| `output-tool` | object | Same payload when invoked via `input-tools` |+| `output-main` | `TwitterSendOutput` | Result of the action; also returned to the LLM when invoked via `input-tools` |### Output payload-Handler wraps the XDK response via `_format_response` then via `_success`:+`_do_send` returns a `TwitterSendOutput` (`ConfigDict(extra="allow")`); `BaseNode._serialize_result` validates and dumps it:```ts-// On success (tweet/reply):{- success: true,- result: {- id: string; // new tweet id- text: string;- // ...any additional fields returned by POST /2/tweets (edit_history_tweet_ids, etc.)- },- execution_time: number+ action: string; // "tweet_sent" | "reply_sent" | "quoted" | "retweeted" | "liked" | "unliked" | "deleted"+ data: object; // _format_response(...) of the XDK result (e.g. { id, text, ... } for create)}-// On success (retweet/like): { retweeted: boolean } or { liked: boolean }-// On success (unlike): { liked: false }-// On success (delete): { deleted: boolean }-// On error: { success: false, error: string, execution_time: number }```+> Note: the runtime `Output` model above is the plugin's authoritative shape. The frontend drag-source schema in [`node_output_schemas.py`](../../../server/services/node_output_schemas.py) (`TwitterSendOutput`: `tweet_id`, `text`, `author_id`, `created_at`, `action`) is a separate, flatter advertised shape and does not match the live envelope field-for-field.+## Logic Flow```mermaidflowchart TD- A[handle_twitter_send] --> B[_get_twitter_client<br/>auth_service.get_oauth_tokens twitter]- B -- no tokens --> Eauth[Return success=false<br/>error: Twitter not connected]- B -- ok --> C[_do_twitter_send]+ A[BaseNode.execute -> @Operation send] --> CWR[call_with_retry _do_send]+ CWR --> B[get_twitter_client<br/>auth_service.get_oauth_tokens twitter]+ B -- no access_token --> Eauth[raise RuntimeError<br/>Twitter not connected]+ B -- ok --> C[_do_send]C --> Dswitch{action}Dswitch -- tweet --> T1{text non-empty?}- T1 -- no --> Eval[raise ValueError]- T1 -- yes --> T2[asyncio.to_thread _sync_create_post<br/>body=text 0..280]- Dswitch -- reply --> R1{text AND reply_to_id?}+ T1 -- no --> Eval[raise RuntimeError]+ T1 -- yes --> T2[to_thread client.posts.create<br/>body=text 0..280]+ Dswitch -- reply --> R1{text AND tweet_id?}R1 -- no --> Eval- R1 -- yes --> R2[to_thread _sync_create_post<br/>reply.in_reply_to_tweet_id]- Dswitch -- retweet/like --> L1[to_thread _get_my_user_id<br/>client.users.get_me]- L1 --> L2[to_thread _sync_repost or _sync_like]- Dswitch -- unlike --> U1[to_thread _get_my_user_id]- U1 --> U2[to_thread _sync_unlike]- Dswitch -- delete --> D1[to_thread _sync_delete_post]- Dswitch -- _ --> Eunk[raise ValueError Unknown action]- T2 --> K[_track_twitter_usage -> DB]+ R1 -- yes --> R2[to_thread posts.create<br/>reply.in_reply_to_tweet_id]+ Dswitch -- quote --> Q1{text AND tweet_id?}+ Q1 -- yes --> Q2[to_thread posts.create<br/>quote_tweet_id]+ Dswitch -- retweet/like --> L1[to_thread get_my_user_id<br/>client.users.get_me]+ L1 --> L2[to_thread repost_post or like_post]+ Dswitch -- unlike --> U1[to_thread get_my_user_id]+ U1 --> U2[to_thread unlike_post]+ Dswitch -- delete --> D1[to_thread posts.delete]+ Dswitch -- _ --> Eunk[raise RuntimeError Unknown action]+ T2 --> K[track_twitter_usage -> DB]R2 --> K+ Q2 --> KL2 --> KU2 --> KD1 --> K- K --> OK[Return success=true envelope]+ K --> OK[Return TwitterSendOutput]C -. raises .-> AuthErr{_is_auth_error?<br/>401/403/Unauthorized/Forbidden}- AuthErr -- yes --> REF[_refresh_and_get_client<br/>TwitterOAuth.refresh_access_token]+ AuthErr -- yes --> REF[refresh_and_get_client<br/>TwitterOAuth.refresh_access_token]REF -. new tokens .-> STORE[auth_service.store_oauth_tokens twitter]- STORE --> C- AuthErr -- no --> EGEN[Return success=false<br/>error: str e]+ STORE --> C2[retry _do_send once]+ AuthErr -- no --> EGEN[propagate -> BaseNode error envelope]```## Decision Logic-- **Client acquisition**: `_get_twitter_client()` reads OAuth tokens via- `auth_service.get_oauth_tokens("twitter", customer_id="owner")`. Raises- `ValueError("Twitter not connected. Please authenticate via Credentials.")`- when tokens are missing.-- **No eager validation**: Handler never calls `get_me()` up-front to verify the+- **Client acquisition**: `get_twitter_client()` (in `_base.py`) reads OAuth+ tokens via `auth_service.get_oauth_tokens("twitter", customer_id="owner")`.+ Raises `RuntimeError("Twitter not connected. Please authenticate via Credentials.")`+ when the access token is missing.+- **No eager validation**: never calls `get_me()` up-front to verify thetoken; it jumps straight into the action call to conserve rate limits.-- **Lazy refresh**: Any exception bubbling out of `_do_twitter_send` is inspected- by `_is_auth_error` (substring match on `401`, `403`, `Unauthorized`,- `Forbidden`). On match, a new client is built via `_refresh_and_get_client()`- which calls `TwitterOAuth.refresh_access_token`, re-stores tokens via- `store_oauth_tokens`, and the action is retried once. Non-auth errors surface- as an error envelope.-- **Validation**:+- **Lazy refresh (`call_with_retry`)**: any exception bubbling out of `_do_send`+ is inspected by `_is_auth_error` (substring match on `401`, `403`,+ `Unauthorized`, `Forbidden`). On match, a new client is built via+ `refresh_and_get_client()` which calls `TwitterOAuth.refresh_access_token`,+ re-stores tokens via `store_oauth_tokens`, and `_do_send` is retried once.+ Non-auth errors propagate to `BaseNode.execute()`'s error envelope.+- **Validation** (raise `RuntimeError` on failure):- `tweet`: requires non-empty `text`, sliced to 280 chars.- - `reply`: requires both `text` and `reply_to_id`.+ - `reply`/`quote`: require both `text` and `tweet_id`.- `retweet`/`like`/`unlike`/`delete`: require `tweet_id`.- **User id fetch**: `retweet`/`like`/`unlike` need the authenticated user id;- `_get_my_user_id` calls `client.users.get_me()` via `asyncio.to_thread`.-- **Unknown action**: `quote` (present in frontend) and any other value fall- into the default match arm and raise `ValueError(f"Unknown action: {action}")`- which is converted to an error envelope.+ `get_my_user_id` calls `client.users.get_me()` via `asyncio.to_thread`.+- **Unknown action**: any value outside the `Literal` set raises+ `RuntimeError(f"Unknown action: {action}")` (unreachable via the UI dropdown).## Side Effects- **Database writes**: one row per successful action in `api_usage_metrics` via- `database.save_api_usage_metric(...)` with `service='twitter'` and- `operation` from `PricingService.calculate_api_cost`. Action -> op name:- `tweet`, `reply`, `retweet`, `like`, `unlike`, `delete`.+ `track_twitter_usage` -> `database.save_api_usage_metric(...)` with+ `service='twitter'` and `operation` from `PricingService.calculate_api_cost`.+ Action -> op name: `tweet`, `reply`, `quote`, `retweet`, `like`, `unlike`, `delete`.- **Broadcasts**: none from the handler.- **External API calls**:- `POST https://api.twitter.com/2/tweets` (tweet, reply)@@ -149,7 +145,7 @@ flowchart TDSystem 1 (`EncryptedOAuthToken` table); NOT via `get_api_key("twitter_access_token")`.Client id + secret are stored via `get_api_key("twitter_client_id")` /`get_api_key("twitter_client_secret")` and only consulted on refresh.-- **Services**: `TwitterOAuth` (`server/services/twitter_oauth.py`) for token+- **Services**: `TwitterOAuth` (`server/nodes/twitter/_oauth.py`) for tokenrefresh. `PricingService` for cost calculation. `Database` for metric row.- **Python packages**: `xdk` (X SDK), `httpx` (used transitively inside`TwitterOAuth`).@@ -157,19 +153,17 @@ flowchart TD## Edge cases & known limits-- **`quote` action is broken**: frontend exposes `Quote Tweet` but backend has no- handler branch; it falls through to the default arm and surfaces as- `Unknown action: quote`.+- **`quote` action is implemented**: builds `posts.create(body={"text":..., "quote_tweet_id": tweet_id})`.- **Media and polls ignored**: the `include_media` / `include_poll` /`media_urls` / `poll_options` / `poll_duration` parameters are never read by- the handler - the node silently sends a text-only tweet.+ `_do_send` - the node silently sends a text-only tweet.- **280-char silent truncation**: `text[:280]` is applied without warning; theemitted tweet may be shorter than the user typed.- **Single retry on auth error**: refresh + retry happens at most once; a- second 401 surfaces as an error envelope.-- **Refresh failure message**: If `refresh_access_token` fails, the user sees- `Twitter token expired and refresh failed. Please re-authenticate.` - the- original upstream error message is lost.+ second 401 propagates to the error envelope.+- **Refresh failure message**: if `refresh_access_token` fails, the user sees+ `Twitter token refresh failed. Please re-authenticate.`; the missing-refresh-token+ case surfaces `Twitter token expired. Please re-authenticate.`- **`_is_auth_error` uses substring matching**: any error whose `str(e)`contains `401`, `403`, `Unauthorized`, or `Forbidden` triggers the refreshpath, even if the real cause is something else (e.g. a log message quotingdiff --git a/docs-internal/node-logic-flows/twitter/twitterUser.md b/docs-internal/node-logic-flows/twitter/twitterUser.mdindex 83288119..3625d0ee 100644--- a/docs-internal/node-logic-flows/twitter/twitterUser.md+++ b/docs-internal/node-logic-flows/twitter/twitterUser.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | social / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/twitter.py::handle_twitter_user`](../../../server/services/handlers/twitter.py) |+| **Backend handler** | [`server/nodes/twitter/twitter_user/__init__.py`](../../../server/nodes/twitter/twitter_user/__init__.py) — dispatch via `BaseNode.execute()` + `@Operation("lookup")` -> `_do_lookup` (helpers in [`_base.py`](../../../server/nodes/twitter/_base.py)) || **Tests** | [`server/tests/nodes/test_twitter.py`](../../../server/tests/nodes/test_twitter.py) || **Skill (if any)** | [`server/skills/social_agent/twitter-user-skill/SKILL.md`](../../../server/skills/social_agent/twitter-user-skill/SKILL.md) || **Dual-purpose tool** | yes - tool name `twitter_user` |@@ -28,104 +28,100 @@ workflow node and an AI agent tool. Every SDK call is wrapped in|------|------|---------|----------|---------------------|-------------|| `operation` | options | `me` | yes | - | `me` / `by_username` / `by_id` / `followers` / `following` || `username` | string | `""` | yes (by_username) | `operation: ['by_username']` | Handle without `@`. |-| `user_id` | string | `""` | yes (by_id), optional (followers/following) | `operation: ['by_id','followers','following']` | If omitted for followers/following, the authenticated user is used via `_get_my_user_id`. |+| `user_id` | string | `""` | yes (by_id), optional (followers/following) | `operation: ['by_id','followers','following']` | If omitted for followers/following, the authenticated user is used via `get_my_user_id`. || `max_results` | number | `100` | no | `operation: ['followers','following']` | Clamped `max(1, min(requested, 1000))`. |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when invoked via `input-tools` |+| `output-main` | `TwitterUserOutput` | Lookup result; also returned to the LLM when invoked via `input-tools` |### Output payload-`me` / `by_username` / `by_id`:+`_do_lookup` returns a `TwitterUserOutput(operation, user?, users?, count?)` (`extra="allow"`); validated + dumped by `BaseNode._serialize_result`.++`me` / `by_username` / `by_id` -> `{ operation, user }` where `user` is `format_user(...)`:```ts{- id: string;- username: string;- name: string;- profile_image_url: string | null;- verified: boolean; // defaults to false when API omits it- description: string | null;- created_at: string; // str(...) - empty string when missing+ operation: string;+ user: {+ id: string;+ username: string;+ name: string;+ profile_image_url: string | null;+ verified: boolean; // defaults to false when API omits it+ description: string | null;+ created_at: string; // str(...) - empty string when missing+ }}```-`followers` / `following`:+`followers` / `following` -> `{ operation, users, count }`:```ts{- users: Array<UserData>; // same shape as above+ operation: string;+ users: Array<UserData>; // same shape as user abovecount: number;}```-All wrapped in `{ success, result, execution_time }`. `me` / `by_username` /-`by_id` additionally take the `_success` path that wraps the dict as-`result` directly (no `action` field leaks through when the payload is already-a dict, thanks to `_success`'s dict-shortcut).-## Logic Flow```mermaidflowchart TD- A[handle_twitter_user] --> B[_get_twitter_client]- B -- no tokens --> Eauth[Return success=false<br/>error: Twitter not connected]- B --> C[_do_twitter_user]+ A[BaseNode.execute -> @Operation lookup] --> CWR[call_with_retry _do_lookup]+ CWR --> B[get_twitter_client]+ B -- no access_token --> Eauth[raise RuntimeError<br/>Twitter not connected]+ B --> C[_do_lookup]C --> Dswitch{operation}- Dswitch -- me --> M1[to_thread _sync_get_me<br/>user_fields=created_at, description]+ Dswitch -- me --> M1[to_thread users.get_me<br/>user_fields=created_at, description]Dswitch -- by_username --> U1{username?}- U1 -- no --> EV[raise ValueError Username is required]- U1 -- yes --> U2[to_thread _sync_get_by_usernames]+ U1 -- no --> EV[raise RuntimeError Username is required]+ U1 -- yes --> U2[to_thread users.get_by_usernames]U2 --> U3{data list empty?}- U3 -- yes --> EVn[raise ValueError User @handle not found]+ U3 -- yes --> EVn[raise RuntimeError User @handle not found]Dswitch -- by_id --> I1{user_id?}I1 -- no --> EV- I1 -- yes --> I2[to_thread _sync_get_by_ids]+ I1 -- yes --> I2[to_thread users.get_by_ids]I2 --> I3{data list empty?}I3 -- yes --> EVn- Dswitch -- followers --> F1{user_id present?}- F1 -- no --> F0[to_thread _get_my_user_id]- F0 --> F2[to_thread _sync_get_followers]+ Dswitch -- followers/following --> F1{user_id present?}+ F1 -- no --> F0[to_thread get_my_user_id]+ F0 --> F2[to_thread _sync_followers / _sync_following]F1 -- yes --> F2- Dswitch -- following --> G1{user_id present?}- G1 -- no --> G0[to_thread _get_my_user_id]- G0 --> G2[to_thread _sync_get_following]- G1 -- yes --> G2- Dswitch -- _ --> Eunk[raise ValueError Unknown operation]- M1 --> K1[_format_user_data + _track_twitter_usage me 1]- U2 --> K2[_format_user_data users 0 + _track_twitter_usage by_username 1]- I2 --> K3[_format_user_data + _track_twitter_usage by_id 1]- F2 --> K4[map _format_user_data + _track_twitter_usage followers len users]- G2 --> K5[same for following]- K1 --> OK[Return success envelope]+ Dswitch -- _ --> Eunk[raise RuntimeError Unknown operation]+ M1 --> K1[format_user + track_twitter_usage me 1]+ U2 --> K2[format_user users 0 + track_twitter_usage by_username 1]+ I2 --> K3[format_user + track_twitter_usage by_id 1]+ F2 --> K4[map format_user + track_twitter_usage op len users]+ K1 --> OK[Return TwitterUserOutput]K2 --> OKK3 --> OKK4 --> OK- K5 --> OK- C -. raises 401/403 .-> REF[_refresh_and_get_client] --> C- C -. other Exception .-> EGEN[Return success=false<br/>error: str e]+ C -. raises 401/403 .-> REF[refresh_and_get_client] --> C2[retry _do_lookup once]+ C -. other Exception .-> EGEN[propagate -> BaseNode error envelope]```## Decision Logic-- **Operation dispatch** uses a Python 3.10+ `match` / `case` block. Any value- other than the five documented operations raises `ValueError(f"Unknown operation: {operation}")`.+- **Operation dispatch** uses an `if`/`elif` chain in `_do_lookup`. Any value+ outside the five `Literal` operations raises `RuntimeError(f"Unknown operation: {op}")`+ (unreachable via the UI dropdown).- **Validation**:- - `by_username`: empty `username` -> `ValueError("Username is required")`.- - `by_id`: empty `user_id` -> `ValueError("User ID is required")`.- - `followers` / `following`: missing `user_id` is **not** an error - handler- falls back to `_get_my_user_id`.-- **Not-found handling**: `by_username` / `by_id` raise `ValueError` when the- SDK returns an empty `data` list; the string is propagated as the envelope- `error`.-- **Lazy auth refresh**: identical to `twitterSend` - any exception whose- `str(e)` contains `401`/`403`/`Unauthorized`/`Forbidden` triggers one- refresh-and-retry.+ - `by_username`: empty `username` -> `RuntimeError("Username is required")`.+ - `by_id`: empty `user_id` -> `RuntimeError("User ID is required")`.+ - `followers` / `following`: missing `user_id` is **not** an error -+ falls back to `get_my_user_id`.+- **Not-found handling**: `by_username` / `by_id` raise `RuntimeError` when the+ SDK returns an empty `data` list; it propagates to `BaseNode.execute()`'s+ error envelope.+- **Lazy auth refresh**: identical to `twitterSend` (`call_with_retry`) - any+ exception whose `str(e)` contains `401`/`403`/`Unauthorized`/`Forbidden`+ triggers one refresh-and-retry.- **max_results clamping**: `max(1, min(requested, 1000))` for followers /following. Anything else is ignored.- **Usage tracking**:@@ -161,12 +157,12 @@ flowchart TD- **`max_results < 1`**: silently clamped to 1.- **`max_results > 1000`**: silently clamped to 1000. The SDK may still paginateinternally; only the first page is returned regardless.-- **Only first page for followers/following**: `_sync_get_followers` /- `_sync_get_following` break after the first page - `next_token` is discarded.+- **Only first page for followers/following**: `_sync_followers` /+ `_sync_following` return after the first page - `next_token` is discarded.- **Not-found vs. empty**: when the target user exists but has zerofollowers/following, the handler returns `{users: [], count: 0}` - it doesNOT raise. `by_username` / `by_id` treat empty data as an error.-- **`_get_my_user_id` cost not tracked**: the implicit `users/me` call made on+- **`get_my_user_id` cost not tracked**: the implicit `users/me` call made onfollowers/following with empty `user_id` is not reflected in`api_usage_metrics`.- **`verified` default is `False`**: even when the API omits the field thediff --git a/docs-internal/node-logic-flows/web_automation/apifyActor.md b/docs-internal/node-logic-flows/web_automation/apifyActor.mdindex 716b38cb..c938f447 100644--- a/docs-internal/node-logic-flows/web_automation/apifyActor.md+++ b/docs-internal/node-logic-flows/web_automation/apifyActor.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | web_automation / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/apify.py::handle_apify_actor`](../../../server/services/handlers/apify.py) |+| **Backend handler** | [`server/nodes/scraper/apify_actor/__init__.py::ApifyActorNode`](../../../server/nodes/scraper/apify_actor/__init__.py) — dispatch via `BaseNode.execute()` -> `@Operation("run")` || **Tests** | [`server/tests/nodes/test_web_automation.py`](../../../server/tests/nodes/test_web_automation.py) || **Skill (if any)** | [`server/skills/web_agent/apify-skill/SKILL.md`](../../../server/skills/web_agent/apify-skill/SKILL.md) || **Dual-purpose tool** | yes |@@ -28,30 +28,39 @@ curated actor ids (e.g. `instagramUrls` for `apify/instagram-scraper`).## Parameters+Params model is `ApifyActorParams` (field names are snake_case).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `actorId` | options/string | `""` | **yes** | - | Actor id (e.g. `apify/instagram-scraper`) or `custom` |-| `customActorId` | string | `""` | yes if `actorId=custom` | `actorId=custom` | Used when `actorId=custom` |-| `actorInput` | string (JSON) | `"{}"` | no | - | Raw JSON run-input; invalid JSON silently becomes `{}` |-| `timeout` | number | `300` | no | - | Run timeout in seconds (SDK forwards as `timeout_secs`) |-| `maxResults` | number | `100` | no | - | Forwarded to `dataset.list_items(limit=...)` |-| `memory` | number | `1024` | no | - | Actor memory in MB (`memory_mbytes`) |-| `instagramUrls` | string | `""` | no | `actorId=apify/instagram-scraper` | Comma-separated -> `directUrls[]` |-| `tiktokProfiles` | string | `""` | no | `actorId=clockworks/tiktok-scraper` | Comma-separated -> `profiles[]` |-| `tiktokHashtags` | string | `""` | no | `actorId=clockworks/tiktok-scraper` | Comma-separated -> `hashtags[]` |-| `twitterSearchTerms` | string | `""` | no | `actorId=apidojo/tweet-scraper` | Comma-separated -> `searchTerms[]` |-| `twitterHandles` | string | `""` | no | `actorId=apidojo/tweet-scraper` | Comma-separated -> `twitterHandles[]` |-| `googleSearchQuery` | string | `""` | no | `actorId=apify/google-search-scraper` | -> `searchQuery` |-| `googleSearchPages` | number | `1` | no | `actorId=apify/google-search-scraper` | -> `maxPagesPerQuery` |-| `crawlerStartUrls` | string | `""` | no | `actorId=apify/website-content-crawler` | Comma-separated -> `startUrls[{url}]` |-| `crawlerMaxDepth` | number | `2` | no | `actorId=apify/website-content-crawler` | -> `maxCrawlDepth` |-| `crawlerMaxPages` | number | `50` | no | `actorId=apify/website-content-crawler` | -> `maxCrawlPages` |+| `actor_id` | options (Literal) | `apify/instagram-scraper` | **yes** | - | One of 10 actor presets or `custom` |+| `custom_actor_id` | string | `""` | yes if `actor_id=custom` | `actor_id=custom` | Used when `actor_id=custom` |+| `actor_input` | Any (JSON) | `{}` (empty dict) | no | - | Raw JSON run-input; merged with quick helpers. String input that fails `json.loads` (or non-dict) silently becomes `{}` |+| `instagram_urls` | string | `""` | no | `actor_id=apify/instagram-scraper` | Comma-separated -> `directUrls[]` |+| `tiktok_profiles` | string | `""` | no | `actor_id=clockworks/tiktok-scraper` | Comma-separated -> `profiles[]` |+| `tiktok_hashtags` | string | `""` | no | `actor_id=clockworks/tiktok-scraper` | Comma-separated -> `hashtags[]` |+| `twitter_search_terms` | string | `""` | no | `actor_id=apidojo/tweet-scraper` | Comma-separated -> `searchTerms[]` |+| `twitter_handles` | string | `""` | no | `actor_id=apidojo/tweet-scraper` | Comma-separated -> `twitterHandles[]` |+| `google_search_query` | string | `""` | no | `actor_id=apify/google-search-scraper` | -> `searchQuery` |+| `google_search_pages` | int (1-100) | `1` | no | `actor_id=apify/google-search-scraper` | -> `maxPagesPerQuery` |+| `crawler_start_urls` | string | `""` | no | `actor_id=apify/website-content-crawler` | Comma-separated -> `startUrls[{url}]` |+| `crawler_max_depth` | int (0-20) | `2` | no | `actor_id=apify/website-content-crawler` | -> `maxCrawlDepth` |+| `crawler_max_pages` | int (1-10000) | `50` | no | `actor_id=apify/website-content-crawler` | -> `maxCrawlPages` |+| `max_results` | int (1-10000) | `100` | no | - | Forwarded to `dataset.list_items(limit=...)` |+| `timeout` | int (1-3600) | `300` | no | - | Run timeout in seconds (SDK forwards as `timeout_secs`) |+| `memory` | options (Literal: 128/256/512/1024/2048/4096/8192) | `1024` | no | - | Actor memory in MB (`memory_mbytes`) |++Actor presets (`actor_id`): `apify/instagram-scraper`, `clockworks/tiktok-scraper`,+`apidojo/tweet-scraper`, `apify/linkedin-scraper`, `apify/facebook-pages-scraper`,+`streamers/youtube-scraper`, `apify/google-search-scraper`,+`compass/crawler-google-places`, `apify/website-content-crawler`,+`curious_coder/web-scraper`, `custom`.## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Standard envelope payload |+| `output-tool` | object | Same payload when wired to an AI agent's `input-tools` (`usable_as_tool=True`, tool name `apify_actor`) |### Output payload (success)@@ -69,59 +78,59 @@ curated actor ids (e.g. `instagramUrls` for `apify/instagram-scraper`).}```-On `FAILED` / `TIMED-OUT` / `ABORTED` the envelope is `success: false` with a-human error message AND a trimmed `result` dict carrying `run_id`, `actor_id`,-and `status` so downstream nodes can still diagnose.+On `FAILED` / `TIMED-OUT` / `ABORTED` (and missing token / empty actor id /+`None` run result) the operation raises `NodeUserError`, which `BaseNode.execute()`+catches into a `{ success: false, error_type: "NodeUserError", error: <message> }`+envelope (single WARN log line, no traceback). There is no trimmed `result` dict.## Logic Flow```mermaidflowchart TD- A[handle_apify_actor] --> B[await _get_apify_client<br/>via container.auth_service.get_api_key 'apify']- B -- token missing --> Enotk[Return error:<br/>Apify API token not configured]- B -- token ok --> C[Read actorId, swap to customActorId if 'custom']+ A[BaseNode.execute -> ApifyActorNode.run] --> B[await _get_apify_client<br/>via get_auth_service.get_api_key 'apify']+ B -- token missing --> Enotk[raise NodeUserError:<br/>Apify API token not configured]+ B -- token ok --> C[Read actor_id, swap to custom_actor_id if 'custom']C --> C1{actor_id empty?}- C1 -- yes --> Eaid[Return error: Actor ID is required]- C1 -- no --> D[_build_actor_input<br/>parse actorInput JSON else {}<br/>merge per-actor quick helpers]- D --> E[Read timeout, maxResults, memory_mbytes]+ C1 -- yes --> Eaid[raise NodeUserError: Actor ID is required]+ C1 -- no --> D[_build_actor_input params.model_dump<br/>parse actor_input JSON else {}<br/>merge per-actor quick helpers]+ D --> E[Read timeout, max_results, memory_mbytes]E --> F[client.actor id .call<br/>run_input, timeout_secs, memory_mbytes]- F -- run_info None --> Enone[Return error:<br/>no result returned]+ F -- run_info None --> Enone[raise NodeUserError:<br/>no result returned]F -- ok --> G{run_status?}- G -- FAILED --> Gf[Return success=false<br/>error=errorMessage, result={run_id, actor_id, status}]- G -- TIMED-OUT --> Gt[Return success=false<br/>error=Actor timed out]- G -- ABORTED --> Ga[Return success=false<br/>error=Actor run was aborted]+ G -- FAILED --> Gf[raise NodeUserError errorMessage]+ G -- TIMED-OUT --> Gt[raise NodeUserError: Actor timed out]+ G -- ABORTED --> Ga[raise NodeUserError: Actor run was aborted]G -- other --> H{dataset_id truthy?}- H -- yes --> H1[client.dataset id .list_items limit=maxResults<br/>items = list_result.items]+ H -- yes --> H1[client.dataset id .list_items limit=max_results<br/>items = listing.items]H -- no --> H2[items = empty list]- H1 & H2 --> I[Build result dict:<br/>run_id, actor_id, status, items, item_count,<br/>dataset_id, compute_units=usageTotalUsd,<br/>started_at, finished_at]- I --> J[Return success envelope]- F -- Exception --> K[Catch all:<br/>rewrite 401/Unauthorized -> 'Invalid API token'<br/>rewrite 404/not found -> 'Actor not found']- K --> Eexc[Return error envelope]+ H1 & H2 --> I[Build ApifyActorOutput:<br/>run_id, actor_id, status, items, item_count,<br/>dataset_id, compute_units=usageTotalUsd,<br/>started_at, finished_at]+ I --> J[Return BaseNode success envelope]```## Decision Logic-- **Auth lookup**: `auth_service.get_api_key("apify", "default")` - a None- return short-circuits with `Apify API token not configured`.+- **Auth lookup**: `get_auth_service().get_api_key("apify", "default")` - a None+ return raises `NodeUserError("Apify API token not configured")`.- **Actor id swap**: literal string `"custom"` triggers replacement with- `customActorId` BEFORE the empty check. An empty `customActorId` still- triggers the `Actor ID is required` error.-- **Run-input merge**:- - `actorInput` parsed as JSON; string input that fails `json.loads` (or is+ `custom_actor_id` BEFORE the empty check. An empty `custom_actor_id` still+ raises the `Actor ID is required` error.+- **Run-input merge** (`_build_actor_input` on `params.model_dump()`):+ - `actor_input` parsed as JSON; string input that fails `json.loads` (or isempty/blank) silently becomes `{}`. Non-dict, non-string input becomes `{}`.- - Quick helpers run even when `actorInput` already contains those keys -+ - Quick helpers run even when `actor_input` already contains those keys -they OVERWRITE any existing value.- - Quick helpers only fire for the five hard-coded actor ids.+ - Quick helpers only fire for five hard-coded actor ids+ (instagram / tiktok / tweet-scraper / google-search / website-content-crawler).- **Run statuses**: three status strings (`FAILED`, `TIMED-OUT`, `ABORTED`)- produce `success=false` envelopes with specific error messages. Every other- status (including `SUCCEEDED`, `RUNNING`, `READY`, etc.) proceeds to dataset- fetch. Runs that stay `RUNNING` past the SDK's internal polling are expected- to be handled by the SDK and should not reach here.+ raise `NodeUserError` with specific messages. Every other status (including+ `SUCCEEDED`, `RUNNING`, `READY`, etc.) proceeds to dataset fetch. Runs that+ stay `RUNNING` past the SDK's internal polling are handled by the SDK and+ should not reach here.- **Empty dataset id**: `items = []`, no API call to the dataset endpoint.-- **Error string rewriting**: `str(e)` is pattern-matched for `401`,- `Unauthorized`, `404`, and `not found` (case-insensitive for the last one)- to produce friendlier user errors. All other exceptions bubble up as their- `str(e)` text.+- **Error rewriting**: the `run` operation does NOT rewrite SDK exception+ strings — those propagate to `BaseNode.execute()`'s generic handler. Only the+ separate `validate_apify_token()` helper (used by the `validate_apify_key` WS+ handler / Credentials modal) rewrites `401`/`Unauthorized` -> `Invalid API token`.## Side Effects@@ -153,9 +162,9 @@ flowchart TDraises. `timeout` is the hard cap.- **`compute_units` is misleading**: the value returned is `run.usageTotalUsd`(USD), not compute units. The key name is preserved for backward compat.-- **Error rewriting is substring-based**: an actor's genuine error message- that contains the word "Unauthorized" or "not found" will be replaced with- the generic MachinaOs error, masking the real cause.+- **Error rewriting is substring-based (validation only)**: `validate_apify_token`+ pattern-matches `401`/`Unauthorized` to a generic message; the runtime `run`+ operation does not, so a genuine SDK error surfaces verbatim.- **No usage tracking**: unlike search / HTTP nodes, there is no`api_usage_metrics` row written for Apify calls.- **Stale dataset on repeat runs**: `dataset.list_items(limit=maxResults)`diff --git a/docs-internal/node-logic-flows/web_automation/browser.md b/docs-internal/node-logic-flows/web_automation/browser.mdindex 9ddd2599..1dd9589a 100644--- a/docs-internal/node-logic-flows/web_automation/browser.md+++ b/docs-internal/node-logic-flows/web_automation/browser.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | web_automation / tool (dual-purpose) |-| **Backend handler** | [`server/nodes/browser/browser/__init__.py::BrowserNode`](../../../server/nodes/browser/browser/__init__.py) |+| **Backend handler** | [`server/nodes/browser/browser/__init__.py::BrowserNode`](../../../server/nodes/browser/browser/__init__.py) — dispatch via `BaseNode.execute()` -> `@Operation("dispatch")` || **Service** | [`server/nodes/browser/_service.py::BrowserService`](../../../server/nodes/browser/_service.py) || **Tests** | [`server/tests/nodes/test_web_automation.py`](../../../server/tests/nodes/test_web_automation.py) || **Skill (if any)** | [`server/skills/web_agent/browser-skill/SKILL.md`](../../../server/skills/web_agent/browser-skill/SKILL.md) |@@ -11,8 +11,12 @@## Purpose-Interactive browser automation via the `agent-browser` CLI (a pinned npm dependency-invoked through `npx --no-install`). Exposes 14 discrete operations (navigate,+Interactive browser automation via the `agent-browser` CLI. The binary is a+MachinaOs-managed local install resolved by+[`nodes/browser/_install.py::agent_browser_binary_path`](../../../server/nodes/browser/_install.py)+(installed via `npm install agent-browser --prefix <package_dir>` on first use,+same pattern as Claude Code's project-local CLI — NOT a workspace `package.json`+dependency and NOT invoked through `npx`). Exposes 14 discrete operations (navigate,click, type, fill, screenshot, snapshot, get_text, get_html, eval, wait, scroll,select, console, errors) plus a `batch` meta-op. The preferred workflow for anAI agent is `navigate` -> `snapshot` -> `click`/`fill` using the stable `@eN`@@ -45,34 +49,35 @@ for both knobs live in `.env.template`.## Parameters+Params model is `BrowserParams` (field names are snake_case). `tool_name` /+`tool_description` are class attributes (not Params fields).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `toolName` | string | `browser` | no | - | Tool name exposed to the AI agent |-| `toolDescription` | string | (see frontend) | no | - | Tool description shown to LLM |-| `operation` | options | `navigate` | no | - | One of `navigate`, `click`, `type`, `fill`, `screenshot`, `snapshot`, `get_text`, `get_html`, `eval`, `wait`, `scroll`, `select`, `console`, `errors`, `batch` |-| `session` | string | `""` | no | - | Session id; if empty, handler uses `machina_<execution_id>` |-| `timeout` | number | `30` | no | - | Per-operation timeout in seconds |-| `headed` | boolean | `true` | no | - | Passes `--headed` to agent-browser |-| `autoConnect` | boolean | `false` | no | - | Passes `--auto-connect` |-| `browser` | options | `chrome` | no | - | `chrome` / `edge` / `chromium` / `custom` / `bundled` / `bundled_explicit`. Legacy `bundled` silently upgraded to `chrome` |-| `executablePath` | string | `""` | no | `browser=custom` | Explicit browser path when `browser=custom` |-| `newWindow` | boolean | `true` | no | - | Appended as `--args --new-window`; only honoured when an explicit `executable_path` resolved |-| `chromeProfile` | string | `""` | no | - | Chrome user-data-dir profile |-| `userAgent` | string | `""` | no | - | Custom UA string |-| `proxy` | string | `""` | no | - | `host:port` or `user:pass@host:port` |-| `actionDelay` | number | `0` | no | - | Seconds to wait before the real op (a pre-command `wait` is issued first) |+| `operation` | options (Literal) | `navigate` | no | - | One of `navigate`, `click`, `type`, `fill`, `screenshot`, `snapshot`, `get_text`, `get_html`, `eval`, `wait`, `scroll`, `select`, `console`, `errors`, `batch` || `url` | string | `""` | yes (op=navigate) | `operation=navigate` | Target URL |-| `selector` | string | `""` | yes (ops that act on an element) | see below | CSS selector or `@eN` ref from snapshot |-| `text` | string | `""` | no | `operation=type` | Text to type |-| `value` | string | `""` | no | `operation in [fill, select]` | Value to fill / dropdown value |+| `selector` | string | `""` | yes (ops that act on an element) | `operation in [click,type,fill,get_text,get_html,wait,select]` | CSS selector or `@eN` ref from snapshot |+| `text` | string | `""` | no | `operation=type` | Text to type keystroke-by-keystroke |+| `value` | string | `""` | no | `operation in [fill, select]` | Value to fill / dropdown option value || `expression` | string | `""` | yes (op=eval) | `operation=eval` | JS expression to evaluate |-| `direction` | options | `down` | no | `operation=scroll` | `up` / `down` / `left` / `right` |-| `amount` | number | `500` | no | `operation=scroll` | Pixels to scroll |-| `fullPage` | boolean | `false` | no | `operation=screenshot` | Passes `--full` |+| `direction` | options (Literal: up/down/left/right) | `down` | no | `operation=scroll` | Scroll direction |+| `amount` | int (1-20000) | `500` | no | `operation=scroll` | Pixels to scroll |+| `commands` | string (JSON array) | `"[]"` | yes (op=batch) | `operation=batch` | JSON array of batch command dicts piped to stdin |+| `full_page` | boolean | `false` | no | `operation=screenshot` | Passes `--full` || `annotate` | boolean | `false` | no | `operation=screenshot` | Passes `--annotate` |-| `screenshotFormat` | options | `png` | no | `operation=screenshot` | `png` / `jpeg` / `webp` |-| `screenshotQuality` | number | - | no | `operation=screenshot, screenshotFormat=jpeg` | 1-100, only forwarded for jpeg |-| `commands` | string (JSON array) | `"[]"` | yes (op=batch) | `operation=batch` | JSON array of command dicts piped to stdin |+| `screenshot_format` | options (Literal: png/jpeg) | `png` | no | `operation=screenshot` | Forwarded as `--screenshot-format` when not png |+| `screenshot_quality` | int (1-100) | `85` | no | `operation=screenshot, screenshot_format=jpeg` | Forwarded as `--screenshot-quality`, only for jpeg |+| `session` | string | `""` | no | - | Session id; if empty, handler uses `machina_<execution_id>` |+| `browser` | options (Literal) | `chrome` | no | - | `chrome` / `edge` / `chromium` / `bundled_explicit` / `custom`. Legacy/empty `bundled` silently upgraded to `chrome` at dispatch |+| `executable_path` | string | `""` | no | `browser=custom` | Explicit browser path when `browser=custom` |+| `headed` | boolean | `true` | no | - | Show browser window (false = headless `--headed` omitted) |+| `new_window` | boolean | `true` | no | - | Appended as `--args --new-window`; only honoured when an explicit `executable_path` resolved |+| `auto_connect` | boolean | `false` | no | - | Reuse an already-running CDP browser (`--auto-connect`) |+| `chrome_profile` | string | `""` | no | - | Chrome user-profile name |+| `user_agent` | string | `""` | no | - | Custom UA string |+| `proxy` | string | `""` | no | - | Proxy URL (e.g. `http://user:pass@host:port`) |+| `action_delay` | int (0-60000) | `0` | no | - | Milliseconds to wait before the real op (a pre-command `wait` is issued first) |+| `timeout` | int (1-600) | `30` | no | - | Per-action timeout in seconds |Operations that **require** `selector`: `click`, `type`, `fill`, `get_text`,`get_html`, `wait`, `select`.@@ -82,7 +87,7 @@ Operations that **require** `selector`: `click`, `type`, `fill`, `get_text`,| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Standard envelope payload |-| `output-tool` | object | Same payload when wired to an AI agent |+| `output-tool` | object | Same payload when wired to an AI agent's `input-tools` (`usable_as_tool=True`, tool name `browser`) |### Output payload@@ -101,7 +106,7 @@ On failure: `{ success: false, error: <string>, execution_time: number }`.```mermaidflowchart TD- A[handle_browser] --> B{BrowserService<br/>available?}+ A[BaseNode.execute -> BrowserNode.dispatch] --> B{BrowserService<br/>available?}B -- no --> Einst[Return error:<br/>agent-browser not installed]B -- yes --> C[Resolve session, timeout, headed, browser,<br/>executable_path, proxy, userAgent,<br/>newWindow, chromeProfile, actionDelay]C --> C1[Upgrade legacy 'bundled' -> 'chrome'<br/>_resolve_browser looks up shutil.which or Win registry]@@ -126,7 +131,7 @@ flowchart TDGsw -- unknown --> Gx[raise ValueError<br/>Unknown operation]Gn & Gc & Gt & Gf & Gs & Gsn & Gg & Gev & Gsc & Gco --> H[svc.run args, session, timeout, run_kw]F --> H- H --> H1[Popen argv via npx --no-install agent-browser<br/>shell=False, reads first JSON line of stdout]+ H --> H1[Popen argv via resolved agent-browser binary<br/>shell=False, reads first JSON line of stdout]H1 --> H2{first line<br/>parseable JSON?}H2 -- yes --> H3[data = parsed JSON]H2 -- no --> H4[data = output: raw string]@@ -134,15 +139,16 @@ flowchart TDI -- yes --> I1[truncate + append '...truncated']I -- no --> J[finally: psutil kill process tree]I1 --> J- J --> K[Return success envelope<br/>operation, data, session]- Gx --> Eval[ValueError caught:<br/>return error envelope]- H --> Eother[TimeoutError / RuntimeError caught<br/>return error envelope]+ J --> K[Return BaseNode success envelope<br/>operation, data, session]+ Gx --> Eval[ValueError -> BaseNode generic envelope]+ H --> Eother[RuntimeError / NodeUserError -> BaseNode envelope]```## Decision Logic-- **Service missing**: `get_browser_service()` returns `None` when `npx` is not on- PATH -> immediate `agent-browser not installed` error envelope.+- **Service missing**: `get_browser_service()` returns `None` when the+ agent-browser binary cannot be resolved (Node toolchain / `npm` unavailable)+ -> `RuntimeError("agent-browser not installed...")` -> error envelope.- **Session resolution**: empty `session` -> `machina_<execution_id>` via thetyped `ctx.execution_id` accessor (handles present-but-None on the agenttool-dispatch path). Only the stripped value is considered empty; whitespace@@ -162,13 +168,14 @@ flowchart TD- **`newWindow`**: gated on `executable_path` actually resolving - if a userrequests new-window while `browser=bundled_explicit`, the flag is silentlydropped.-- **`actionDelay > 0`**: issues a `wait <seconds>` command to agent-browser- BEFORE the real operation. Failures in the delay call propagate as errors.+- **`action_delay > 0`**: issues a `wait <action_delay>` command (milliseconds)+ to agent-browser BEFORE the real operation. Failures in the delay call+ propagate as errors.- **`batch`**: reads `commands` JSON string (defaults to `"[]"`) and pipes`json.dumps(cmds).encode()` to the subprocess' stdin with flags `batch --json`.- **Required params**: `_req(p, "url")` and `_req(p, "expression")` raise- `ValueError` on empty strings; `_req_sel(s)` does the same for `selector`.- These surface as the error envelope via the outer `except (ValueError, ...)`.+ `NodeUserError` on empty strings; `_req_sel(s)` does the same for `selector`.+ `BaseNode.execute()` catches `NodeUserError` into a single-WARN error envelope.- **Unknown operation**: `_build_args` raises `ValueError("Unknown operation: <op>")`.## Side Effects@@ -178,11 +185,13 @@ flowchart TD- **External API calls**: none (all work happens via subprocess).- **File I/O**: agent-browser itself may write screenshots or a profile dir toits own cache - not under this handler's control.-- **Subprocess**: spawns `npx --no-install agent-browser ...` via+- **Subprocess**: spawns the resolved `agent-browser` binary directly via`subprocess.Popen` with `shell=False`. Reads only the FIRST stdout line(daemon holds stdout open), caps at 100 KB, then force-kills the entire- process tree via `psutil.Process.children(recursive=True) -> kill()`.+ process tree via `kill_tree(proc.pid)` (`services._supervisor.util`, psutil-backed).Always calls `proc.wait()` in a finally block.+ `AGENT_BROWSER_IDLE_TIMEOUT_MS` is injected into the spawn env when the idle+ timeout is > 0.- **Long-lived daemon**: agent-browser spawns a persistent daemon betweencalls. `shutdown_browser_service()` (registered in FastAPI lifespan) runs`agent-browser close --all` to stop it.@@ -190,9 +199,10 @@ flowchart TD## External Dependencies- **Credentials**: none.-- **Services**: `BrowserService` singleton; `npx` on PATH; `agent-browser`- pinned in the project's `package.json` and installed under `node_modules`.-- **Python packages**: `psutil` (for process-tree kill).+- **Services**: `BrowserService` singleton; `npm` on PATH (for first-use+ install); `agent-browser` installed under `<package_dir("browser")>/npm/`+ by `nodes/browser/_install.py` (MachinaOs-managed, not a workspace dep).+- **Python packages**: `psutil` (via `kill_tree` for process-tree kill).- **Environment variables**: `BROWSER_MAX_INSTANCES` (concurrent session cap,default 3), `BROWSER_IDLE_TIMEOUT_MS` (daemon idle auto-shutdown in ms,default 600000, 0 disables) — canonical values in `.env.template`, mirrored@@ -211,15 +221,18 @@ flowchart TD- **Silent `bundled` -> `chrome` upgrade**: a user who saved a workflow when`bundled` was the default will now actually use system Chrome on execution.- **`newWindow` silently disabled**: see Decision Logic.-- **Only `ValueError`, `TimeoutError`, `RuntimeError` are caught**: other- exceptions (e.g. `OSError` from subprocess, `psutil.Error`) propagate and- the executor wraps them in its generic envelope.+- **Exception handling is in `BaseNode.execute()`**: `NodeUserError` (empty+ required field) -> single WARN + structured envelope; `ValueError` (unknown+ op) / `RuntimeError` (service missing or empty stdout) / other exceptions ->+ generic envelope with traceback. The plugin's `dispatch` itself does not+ catch.- **Process tree kill race**: `psutil.NoSuchProcess` is swallowed for both theparent and every child, so a fast-exiting daemon never leaks an exceptionbut may leave orphaned Chromium processes if the tree changed shape betweenthe `children()` call and the `kill()` call.-- **`actionDelay` is billable**: the preliminary `wait` call consumes part of- `timeout` because each subprocess invocation resets the timer.+- **`action_delay` (ms) is a separate call**: the preliminary `wait` runs as its+ own subprocess invocation with its own `timeout` budget (each invocation+ resets the timer).## Relateddiff --git a/docs-internal/node-logic-flows/web_automation/crawleeScraper.md b/docs-internal/node-logic-flows/web_automation/crawleeScraper.mdindex db326b45..f4097bee 100644--- a/docs-internal/node-logic-flows/web_automation/crawleeScraper.md+++ b/docs-internal/node-logic-flows/web_automation/crawleeScraper.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | web_automation / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/crawlee.py::handle_crawlee_scraper`](../../../server/services/handlers/crawlee.py) |+| **Backend handler** | [`server/nodes/scraper/crawlee_scraper/__init__.py::CrawleeScraperNode`](../../../server/nodes/scraper/crawlee_scraper/__init__.py) — dispatch via `BaseNode.execute()` -> `@Operation("scrape")` || **Tests** | [`server/tests/nodes/test_web_automation.py`](../../../server/tests/nodes/test_web_automation.py) || **Skill (if any)** | [`server/skills/web_agent/crawlee-scraper-skill/SKILL.md`](../../../server/skills/web_agent/crawlee-scraper-skill/SKILL.md) || **Dual-purpose tool** | yes |@@ -33,31 +33,41 @@ config and collects a page list from the route handler closure.## Parameters+Params model is `CrawleeScraperParams` (field names are snake_case).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|+| `tool_name` | string | `web_scraper` | no | - | Name shown to the LLM when used as a tool (note: class `tool_name` is `web_reader`) |+| `tool_description` | string | (see default) | no | - | Description shown to the LLM || `url` | string | `""` | **yes** | - | Seed URL |-| `crawlerType` | options | `beautifulsoup` | no | - | `beautifulsoup` / `playwright` / `adaptive` |-| `mode` | options | `single` | no | - | `single` or `crawl` |-| `cssSelector` | string | `""` | no | - | CSS selector used for text/HTML extraction. Empty -> whole page |-| `extractLinks` | boolean | `false` | no | - | Include `links[]` on each page |-| `outputFormat` | options | `text` | no | - | `text` / `html` / `markdown`. `markdown` falls back to text if `html2text` not installed |-| `maxPages` | number | `10` | no | `mode=crawl` | Ignored and clamped to 1 when `mode=single` |-| `maxDepth` | number | `2` | no | `mode=crawl` | Ignored and set to 0 when `mode=single` |-| `maxConcurrency` | number | `5` | no | - | Forwarded as `max_concurrency`; `desired_concurrency = min(maxConcurrency, 10)` |-| `timeout` | number | `60` | no | - | Per-request handler timeout AND outer `asyncio.wait_for` timeout |-| `linkSelector` | string | `""` | no | `mode=crawl` | Defaults to `a[href]` when empty |-| `urlPattern` | string | `""` | no | `mode=crawl` | `fnmatch`-style pattern compiled with `fnmatch.translate` -> passed to `enqueue_links(include=[regex])` |-| `browserType` | options | `chromium` | no | `crawlerType=playwright` | `chromium` / `firefox` / `webkit` |-| `waitForSelector` | string | `""` | no | `crawlerType=playwright` | Awaits selector before extracting |-| `waitTimeout` | number | `30000` | no | `crawlerType=playwright` | Milliseconds for `wait_for_selector` |-| `screenshot` | boolean | `false` | no | `crawlerType=playwright` | Capture base64 PNG per page |-| `useProxy` | boolean | `false` | no | - | Route through `ProxyService` (see Decision Logic) |+| `crawler_type` | options (Literal) | `beautifulsoup` | no | - | `beautifulsoup` / `playwright` / `adaptive` |+| `mode` | options (Literal) | `single` | no | - | `single` or `crawl` |+| `css_selector` | string | `""` | no | - | CSS selector used for text/HTML extraction. Empty -> whole page |+| `extract_links` | boolean | `false` | no | - | Include `links[]` on each page |+| `output_format` | options (Literal) | `text` | no | - | `text` / `html` / `markdown`. `markdown` falls back to text if `html2text` not installed |+| `max_pages` | int (1-1000) | `10` | no | - | Clamped to 1 when `mode=single` |+| `max_concurrency` | int (1-50) | `5` | no | - | Forwarded as `max_concurrency`; `desired_concurrency = min(max_concurrency, 10)` |+| `timeout` | int (1-3600) | `60` | no | - | Per-request handler timeout AND outer `asyncio.wait_for` timeout |+| `link_selector` | string | `""` | no | `mode=crawl` | Defaults to `a[href]` when empty |+| `url_pattern` | string | `""` | no | `mode=crawl` | `fnmatch`-style pattern compiled with `fnmatch.translate` -> passed to `enqueue_links(include=[regex])` |+| `max_depth` | int (0-50) | `2` | no | `mode=crawl` | Set to 0 when `mode=single` |+| `browser_type` | options (Literal) | `chromium` | no | `crawler_type=playwright` | `chromium` / `firefox` / `webkit` |+| `wait_for_selector` | string | `""` | no | `crawler_type=playwright` | Awaits selector before extracting |+| `wait_timeout` | int (0-600000) | `30000` | no | `crawler_type=playwright` | Milliseconds for `wait_for_selector` |+| `take_screenshot` | boolean | `false` | no | `crawler_type=playwright` | Capture base64 PNG per page |+| `block_resources` | boolean | `false` | no | `crawler_type=playwright` | Declared in Params; currently not consumed by the run helpers |+| `use_proxy` | boolean | `false` | no | - | Route through `ProxyService` (see Decision Logic) |+| `proxy_provider` | string | `auto` | no | `use_proxy=true` | Provider hint passed to `ProxyService.get_proxy_url` |+| `proxy_country` | string | `""` | no | `use_proxy=true` | ISO country hint |+| `session_type` | options (Literal: rotating/sticky) | `rotating` | no | `use_proxy=true` | Proxy session type |+| `sticky_duration` | int (>=1) | `600` | no | `use_proxy=true, session_type=sticky` | Sticky session duration (s) |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Standard envelope payload |+| `output-tool` | object | Same payload when wired to an AI agent's `input-tools` (`usable_as_tool=True`, class tool name `web_reader`) |### Output payload@@ -68,7 +78,7 @@ config and collects a page list from the route handler closure.title: string;content: string;links?: string[]; // when extractLinks=true- screenshot?: string; // base64 PNG, playwright + screenshot=true+ screenshot?: string; // base64 PNG, playwright + take_screenshot=true}>;page_count: number;crawler_type: 'beautifulsoup' | 'playwright' | 'adaptive';@@ -81,8 +91,8 @@ config and collects a page list from the route handler closure.```mermaidflowchart TD- A[handle_crawlee_scraper] --> B{url stripped<br/>non-empty?}- B -- no --> Eurl[Return error: URL is required]+ A[BaseNode.execute -> CrawleeScraperNode.scrape] --> B{url stripped<br/>non-empty?}+ B -- no --> Eurl[raise NodeUserError: URL is required]B -- yes --> C[Read crawlerType, mode, cssSelector,<br/>extractLinks, outputFormat, maxPages,<br/>maxDepth, maxConcurrency, timeout]C --> C1{mode == 'single'?}C1 -- yes --> C2[Force maxPages=1, maxDepth=0]@@ -94,21 +104,20 @@ flowchart TDD2 & D3 --> E{crawlerType?}E -- beautifulsoup / adaptive --> F1[_run_beautifulsoup]E -- playwright --> F2[_run_playwright]- E -- other --> Ebad[Return error:<br/>Unknown crawler type]+ E -- other --> Ebad[raise NodeUserError:<br/>Unknown crawler type]F1 --> G[BeautifulSoupCrawler<br/>max_requests_per_crawl=maxPages<br/>max_crawl_depth=maxDepth if crawl else 0<br/>request_handler_timeout=timeout<br/>ConcurrencySettings<br/>MemoryStorageClient<br/>proxy_configuration=proxy_config]F2 --> H[PlaywrightCrawler<br/>browser_type, headless=True, proxy_configuration]G & H --> I[register default_handler closure]I --> I1[handler: extract title/content<br/>via _extract_text+_extract_links<br/>append to pages list<br/>if mode=crawl: ctx.enqueue_links<br/>with optional fnmatch-compiled include regex]I1 --> J[asyncio.wait_for crawler.run URL, timeout=timeout]- J -- ImportError --> Eimp[Return error with install hint<br/>playwright vs crawlee]- J -- other Exception --> Egen[Return error: str e]- J -- ok --> K[Return success<br/>pages, page_count, crawler_type, mode, proxied]+ J -- ImportError --> Eimp[raise NodeUserError install hint<br/>playwright vs crawlee, else re-raise]+ J -- ok --> K[Return CrawleeScraperOutput<br/>pages, page_count, crawler_type, mode, proxied]```## Decision Logic-- **Empty URL**: `url.strip() == ""` -> immediate failure envelope, no crawler- is constructed.+- **Empty URL**: `url.strip() == ""` -> `NodeUserError("URL is required")`, no+ crawler is constructed.- **`mode='single'`**: hard clamps `max_pages=1` and `max_depth=0`, regardlessof what the user set. In `crawl` mode the requested values are forwarded.- **`crawlerType='adaptive'`**: routed to `_run_beautifulsoup` - there is no@@ -123,13 +132,16 @@ flowchart TDFor markdown the handler falls back to plain text if `html2text` is missing(`ImportError` swallowed). All output is capped at 100 000 chars(`_MAX_CONTENT_LENGTH`).-- **Error handling**:- - `ImportError` whose message contains `playwright` -> rewritten to an- install hint for `pip install 'crawlee[playwright]' && playwright install`.- - `ImportError` whose message contains `crawlee` -> rewritten to an install- hint for `pip install 'crawlee[beautifulsoup]'`.- - All other exceptions are caught and the `str(e)` returned as-is; the- handler never raises.+- **Error handling**: only `ImportError` is caught inside `scrape`:+ - message contains `playwright` -> `NodeUserError` install hint for+ `pip install 'crawlee[playwright]' && playwright install chromium`.+ - message contains `crawlee` -> `NodeUserError` install hint for+ `pip install 'crawlee[beautifulsoup]'`.+ - any other `ImportError` is re-raised.+ - All non-`ImportError` exceptions propagate to `BaseNode.execute()`'s generic+ handler (full-traceback envelope); `NodeUserError` (empty URL / unknown+ crawler type) yields a single-WARN structured envelope. The plugin does not+ swallow errors into `str(e)`.## Side Effectsdiff --git a/docs-internal/node-logic-flows/whatsapp/whatsappDb.md b/docs-internal/node-logic-flows/whatsapp/whatsappDb.mdindex ff593562..435cb8b4 100644--- a/docs-internal/node-logic-flows/whatsapp/whatsappDb.md+++ b/docs-internal/node-logic-flows/whatsapp/whatsappDb.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | whatsapp / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/whatsapp.py::handle_whatsapp_db`](../../../server/services/handlers/whatsapp.py) |+| **Backend handler** | [`server/nodes/whatsapp/whatsapp_db.py`](../../../server/nodes/whatsapp/whatsapp_db.py) (`WhatsAppDbNode`); dispatched via `BaseNode.execute()` -> `@Operation("query")`, which delegates to [`server/nodes/whatsapp/_base.py::handle_whatsapp_db`](../../../server/nodes/whatsapp/_base.py) || **Tests** | [`server/tests/nodes/test_whatsapp.py`](../../../server/tests/nodes/test_whatsapp.py) || **Skill (if any)** | [`server/skills/social_agent/whatsapp-db-skill/SKILL.md`](../../../server/skills/social_agent/whatsapp-db-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `whatsappDb` |+| **Dual-purpose tool** | yes - tool name `whatsapp_db` |## Purpose@@ -55,8 +55,9 @@ frontend.| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Envelope payload with `operation` field + operation-specific fields |-| `output-tool` | object | Same payload for AI tool wiring |+| `output-main` | object | The handler `result` dict with `operation` field + operation-specific fields. Plugin `Output` model is `WhatsAppDbOutput` (`operation`, `messages`, `contacts`, `groups`, `channels`, `total`, `extra="allow"`). |++When wired into an AI agent's `input-tools` handle (`usable_as_tool=True`, tool name `whatsapp_db`) the same payload is returned to the LLM.All payloads include the read-back `operation` and `timestamp`. Collectionoperations (`search_groups`, `list_contacts`, `list_channels`) apply a soft@@ -121,8 +122,8 @@ flowchart TD## Side Effects- **Database writes**: none.-- **Broadcasts**: none - standard `node_status` comes from `NodeExecutor`.-- **External calls**: WebSocket RPC to the Go service (`whatsapp_rpc_call` and `handle_whatsapp_chat_history` in `routers/whatsapp.py`). For `channel_messages` and `chat_history` with `include_media_data=true`, one additional `media` RPC call per media message (up to 5 in parallel).+- **Broadcasts**: none - standard `node_status` comes from `BaseNode.execute()`.+- **External calls**: WebSocket RPC to the Go service (`whatsapp_rpc_call` and `handle_whatsapp_chat_history` in [`nodes/whatsapp/_service.py`](../../../server/nodes/whatsapp/_service.py)). For `channel_messages` and `chat_history` with `include_media_data=true`, one additional `media` RPC call per media message (up to 5 in parallel).- **File I/O**: none.- **Subprocess**: none.diff --git a/docs-internal/node-logic-flows/whatsapp/whatsappReceive.md b/docs-internal/node-logic-flows/whatsapp/whatsappReceive.mdindex 23cb61dc..480f81f0 100644--- a/docs-internal/node-logic-flows/whatsapp/whatsappReceive.md+++ b/docs-internal/node-logic-flows/whatsapp/whatsappReceive.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | whatsapp / trigger |-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) + [`server/services/event_waiter.py::build_whatsapp_filter`](../../../server/services/event_waiter.py) |+| **Backend handler** | [`server/nodes/whatsapp/whatsapp_receive.py`](../../../server/nodes/whatsapp/whatsapp_receive.py) (`WhatsAppReceiveNode`, a `TriggerNode`); the filter closure is built by `build_filter()` -> [`server/nodes/whatsapp/_filters.py::build_filter`](../../../server/nodes/whatsapp/_filters.py) (self-registered into `event_waiter.FILTER_BUILDERS` as `whatsappReceive`). Canary-registered with CloudEvents type `com.machinaos.whatsapp.message.received`; deployment fires via `TriggerListenerWorkflow`. || **Tests** | [`server/tests/nodes/test_whatsapp.py`](../../../server/tests/nodes/test_whatsapp.py) || **Skill (if any)** | n/a || **Dual-purpose tool** | no - pure trigger node |@@ -11,11 +11,15 @@## PurposeEvent-driven trigger that waits for incoming WhatsApp messages and starts a-workflow run when one matches. Uses the generic event-waiter machinery: the-handler registers a waiter with a filter closure, broadcasts `waiting`-status, then awaits a `whatsapp_message_received` event dispatched by the-Go RPC listener. Matching messages resolve the waiter and become the-trigger's output.+workflow run when one matches. `whatsappReceive` is canary-registered+(CloudEvents type `com.machinaos.whatsapp.message.received`): on deployment the+manager spawns a `TriggerListenerWorkflow` per (deployment, trigger) pair which+receives the CloudEvents envelope via Temporal Signal and spawns a child+`MachinaWorkflow` per matching event. Single-node "Run" (non-deployed test)+still uses the `TriggerNode.execute` event-waiter path: register a waiter with+the filter closure, broadcast `waiting`, then await a+`whatsapp_message_received` event. In both paths `build_filter` decides which+inbound messages match.## Inputs (handles)@@ -23,17 +27,20 @@ Triggers have no data inputs - they are workflow starting points.## Parameters+Field names are snake_case throughout (JSON Schema keys match the Pydantic field names exactly, no camelCase aliases).+| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `messageTypeFilter` | options | `all` | no | - | `all`, `text`, `image`, `video`, `audio`, `document`, `location`, `contact`, `sticker` |+| `message_type_filter` | options | `all` | no | - | `all`, `text`, `image`, `video`, `audio`, `document`, `location`, `contact` || `filter` | options | `all` | no | - | `all`, `self`, `any_contact`, `contact`, `group`, `channel`, `keywords` |-| `contactPhone` | string | `""` | conditional | `filter == 'contact'` | Phone number substring matched against resolved `sender_phone` |-| `group_id` | string | `""` | conditional | `filter == 'group'` | Group JID |-| `senderNumber` | string | `""` | no | `filter == 'group'` | Optional phone substring to match a specific sender within the group |-| `channel_jid` | string | `""` | no | `filter == 'channel'` | Optional exact channel JID; empty = any `@newsletter` chat |+| `phone_number` | string | `""` | conditional | `filter == 'contact'` | Phone number substring matched against resolved `sender_phone` |+| `group_id` | string | `""` | conditional | `filter == 'group'` | Group JID (component `GroupIdSelector`) |+| `sender_number` | string | `""` | no | `filter == 'group'` | Optional phone substring to match a specific sender within the group (component `SenderNumberSelector`, `dependsOn: group_id`) |+| `channel_jid` | string | `""` | conditional | `filter == 'channel'` | Optional exact channel JID; empty = any `@newsletter` chat (component `ChannelJidSelector`) || `keywords` | string | `""` | conditional | `filter == 'keywords'` | Comma-separated keywords, case-insensitive substring match against `text` |-| `forwardedFilter` | options | `all` | no | - | `all`, `only_forwarded`, `ignore_forwarded` |-| `ignoreOwnMessages` | boolean | `true` | no | - | Skip `is_from_me == true` (except when `filter == 'self'`) |+| `forwarded_filter` | options | `all` | no | - | `all`, `only_forwarded`, `ignore_forwarded` |+| `ignore_own_messages` | boolean | `true` | no | `filter in [all, any_contact, contact, group, channel, keywords]` | Skip `is_from_me == true` (except when `filter == 'self'`) |+| `include_media_data` | boolean | `false` | no | - | Download base64 media data with the event |## Outputs (handles)@@ -77,8 +84,8 @@ CLAUDE.md "WhatsApp Group/Sender Name Persistence".```mermaidflowchart TD- A[handle_trigger_node] --> B[event_waiter.get_trigger_config<br/>whatsappReceive -> whatsapp_message_received]- B --> C[event_waiter.register<br/>builds filter via build_whatsapp_filter]+ A[TriggerNode.execute - test Run path] --> B[event_type =<br/>whatsapp_message_received]+ B --> C[event_waiter.register<br/>builds filter via _filters.build_filter]C --> D[broadcaster.update_node_status<br/>status=waiting]D --> E[await event_waiter.wait_for_event]E -- CancelledError --> X[success=false<br/>error=Cancelled by user]@@ -87,20 +94,20 @@ flowchart TDF --> OK[success=true<br/>result=event_data]```-### Filter closure (build_whatsapp_filter)+### Filter closure (`_filters.build_filter`)```mermaidflowchart LR- M[message dict] --> R1{messageTypeFilter<br/>== 'all' or matches?}+ M[message dict] --> R1{message_type_filter<br/>== 'all' or matches?}R1 -- no --> REJ[reject]R1 -- yes --> R2{filter mode?}R2 -- self --> S1{is_from_me AND<br/>chat_id == sender?}S1 -- no --> REJR2 -- any_contact --> S2{is_group?}S2 -- yes --> REJ- R2 -- contact --> S3{contactPhone in<br/>sender_phone?}+ R2 -- contact --> S3{phone_number in<br/>sender_phone?}S3 -- no --> REJ- R2 -- group --> S4{is_group AND<br/>chat_id == group_id AND<br/>senderNumber match if set?}+ R2 -- group --> S4{is_group AND<br/>chat_id == group_id AND<br/>sender_number match if set?}S4 -- no --> REJR2 -- channel --> S5{chat_id ends @newsletter<br/>and matches channel_jid if set?}S5 -- no --> REJ@@ -113,10 +120,10 @@ flowchart LRS4 --> OWNS5 --> OWNS6 --> OWN- OWN{ignoreOwnMessages AND<br/>is_from_me AND filter != self?}+ OWN{ignore_own_messages AND<br/>is_from_me AND filter != self?}OWN -- yes --> REJOWN -- no --> FWD- FWD{forwardedFilter check}+ FWD{forwarded_filter check}FWD -- fail --> REJFWD -- pass --> ACC[accept]```@@ -125,7 +132,7 @@ flowchart LR- **Filter priority**: message_type -> sender filter -> own-message -> forwarded. Every check is a short-circuit reject.- **Sender phone fallback**: uses `group_info.sender_phone` for group messages, falls back to root `sender_phone`, and if both empty splits the JID at `@` to extract the phone prefix.-- **`filter == 'self'` is special**: disables the `ignoreOwnMessages` gate (otherwise every self-message would be filtered out).+- **`filter == 'self'` is special**: disables the `ignore_own_messages` gate (otherwise every self-message would be filtered out).- **Keyword match is substring**, not token match: `"hi"` matches `"this"`.- **Channel filter** requires `chat_id` to end exactly with `@newsletter`; if `channel_jid` is provided it must match exactly, otherwise any channel passes.- **Pre-check**: unlike `telegramReceive`, this trigger does not check WhatsApp pairing state before registering - it will wait forever if the Go RPC is disconnected.@@ -134,14 +141,14 @@ flowchart LR- **Database writes**: none from the handler directly (frontend may save `group_name`/`sender_name` back to node parameters via the WebSocket `save_node_parameters` handler - that is UI-driven, not part of trigger execution).- **Broadcasts**: `update_node_status(node_id, "waiting", {...})` once before awaiting.-- **External calls**: none outbound. Receives events dispatched by `routers/whatsapp.py` when the Go RPC emits `event.message_received`.+- **External calls**: none outbound. Inbound `whatsapp_message_received` events are produced by the WhatsApp WS handlers / Go RPC bridge and routed via [`nodes/whatsapp/_events.py`](../../../server/nodes/whatsapp/_events.py) (`broadcast_whatsapp_message`), which both broadcasts the legacy raw frame and emits the typed CloudEvents envelope through `dispatch.emit`.- **File I/O**: none.- **Subprocess**: none.## External Dependencies- **Credentials**: none - pairing is held by the Go service.-- **Services**: `whatsapp-rpc` (Go) for event dispatch via `event_waiter.dispatch('whatsapp_message_received', data)`.+- **Services**: `whatsapp-rpc` (Go) for inbound message events, surfaced via `nodes/whatsapp/_events.py::broadcast_whatsapp_message`.- **Python packages**: `asyncio`.- **Environment variables**: none in the handler.@@ -151,6 +158,7 @@ flowchart LR- **LID-to-phone resolution is implicit**: the Go RPC already populates `sender_phone` with the resolved phone before dispatch, so the filter never needs to call back into the RPC. Missing `sender_phone` is still tolerated via the JID-prefix fallback.- **`filter == 'self'` semantics are narrow**: requires `is_from_me` AND `chat_id == sender` (true only for notes-to-self chats). Self-messages sent to other chats are rejected.- **`keywords` is trivially tokenized**: splits on `,` and `.strip().lower()` each; there is no escaping or whole-word matching.+- **Parameter field names are snake_case** (`message_type_filter`, `phone_number`, `sender_number`, `forwarded_filter`, `ignore_own_messages`). `message_type_filter` has no `sticker` option (unlike `whatsappSend`'s `message_type`).- **Cancellation path** returns `success=false` with `error="Cancelled by user"` - this is the documented path for the Cancel button, not a real error.- **Filter closure is built once at register-time**: changes to node parameters during a waiting run are not reflected; user must cancel and re-run.@@ -158,4 +166,4 @@ flowchart LR- **Companion nodes**: [`whatsappSend`](./whatsappSend.md), [`whatsappDb`](./whatsappDb.md)- **Architecture docs**: `CLAUDE.md` -> "Event-Driven Trigger Node System", "WhatsApp Integration"-- **Trigger infrastructure**: [`event_waiter.py`](../../../server/services/event_waiter.py), [`triggers.py`](../../../server/services/handlers/triggers.py)+- **Trigger infrastructure**: [`event_waiter.py`](../../../server/services/event_waiter.py), [`_filters.py`](../../../server/nodes/whatsapp/_filters.py), [`_events.py`](../../../server/nodes/whatsapp/_events.py), canary path in [`canary_registry.py`](../../../server/services/deployment/canary_registry.py)diff --git a/docs-internal/node-logic-flows/whatsapp/whatsappSend.md b/docs-internal/node-logic-flows/whatsapp/whatsappSend.mdindex 7acb5e64..faff3c1e 100644--- a/docs-internal/node-logic-flows/whatsapp/whatsappSend.md+++ b/docs-internal/node-logic-flows/whatsapp/whatsappSend.md@@ -3,10 +3,10 @@| Field | Value ||------|-------|| **Category** | whatsapp / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/whatsapp.py::handle_whatsapp_send`](../../../server/services/handlers/whatsapp.py) |+| **Backend handler** | [`server/nodes/whatsapp/whatsapp_send.py`](../../../server/nodes/whatsapp/whatsapp_send.py) (`WhatsAppSendNode`); dispatched via `BaseNode.execute()` -> `@Operation("send")`, which delegates to [`server/nodes/whatsapp/_base.py::handle_whatsapp_send`](../../../server/nodes/whatsapp/_base.py) || **Tests** | [`server/tests/nodes/test_whatsapp.py`](../../../server/tests/nodes/test_whatsapp.py) || **Skill (if any)** | [`server/skills/social_agent/whatsapp-send-skill/SKILL.md`](../../../server/skills/social_agent/whatsapp-send-skill/SKILL.md) |-| **Dual-purpose tool** | yes - tool name `whatsappSend` |+| **Dual-purpose tool** | yes - tool name `whatsapp_send` |## Purpose@@ -35,7 +35,7 @@ Key parameters (see frontend definition for the full list and `displayOptions.sh| `channel_jid` | string | `""` | conditional | Required when `recipient_type == 'channel'`, must end `@newsletter` || `message_type` | options | `text` | yes | `text`, `image`, `video`, `audio`, `document`, `sticker`, `location`, `contact` || `message` | string | `""` | conditional | Required when `message_type == 'text'` |-| `format_markdown` | boolean | `false` | no | When true and text message, convert GFM to WhatsApp syntax via `markdown_formatter.to_whatsapp` |+| `format_markdown` | boolean | `true` | no | When true and text message, convert GFM to WhatsApp syntax via `markdown_formatter.to_whatsapp` || `media_source` | options | `base64` | conditional | `base64`, `file`, or `url` for media types || `media_data` | string | `""` | conditional | Base64 data when `media_source == 'base64'` || `file_path` | string/object | `""` | conditional | Either a path string or upload dict `{type: 'upload', filename, mimeType}` |@@ -53,8 +53,9 @@ Key parameters (see frontend definition for the full list and `displayOptions.sh| Handle | Shape | Description ||--------|-------|-------------|-| `output-main` | object | Standard envelope result |-| `output-tool` | object | Same payload when wired into an AI agent's `input-tools` |+| `output-main` | object | The handler `result` dict (see below). The plugin `Output` model is `WhatsAppSendOutput` (`message_id`, `sent`, `extra="allow"`). |++When wired into an AI agent's `input-tools` handle (`usable_as_tool=True`, tool name `whatsapp_send`) the same `result` payload is returned to the LLM.### Output payload@@ -101,7 +102,7 @@ flowchart TDT2 -- no --> ST3 --> ST -- no --> S- S[await routers.whatsapp.handle_whatsapp_send params] --> S2{data.success?}+ S[await nodes.whatsapp._service.handle_whatsapp_send params] --> S2{data.success?}S2 -- no --> Eret6[success=false<br/>error from RPC]S2 -- yes --> F[Build result dict with<br/>status/recipient/type-specific fields]F --> OK[success=true]@@ -117,21 +118,21 @@ flowchart TD- Empty `message` when `message_type == 'text'`- **Markdown transform**: only runs when `message_type == 'text'` AND `format_markdown` truthy. Overwrites `parameters['message']` in place before RPC call.- **Type-specific result fields**: `match message_type` adds `preview`, `media_source`, `caption`, `filename`, `mime_type`, `uploaded_file`, `location`, or `contact_name` based on message type.-- **File upload detection**: if `file_path` is a dict with `type == 'upload'`, the router uploads bytes and the result includes `uploaded_file` and `mime_type` from the upload dict.+- **File upload detection**: if `file_path` is a dict with `type == 'upload'`, the `_service.py` send handler uploads bytes and the result includes `uploaded_file` and `mime_type` from the upload dict.## Side Effects-- **Database writes**: none directly from this handler. (Tracking, if any, is owned by the router.)-- **Broadcasts**: none directly - standard node_status transitions come from `NodeExecutor`.-- **External calls**: WebSocket RPC to the Go service. Default URL `ws://localhost:9400/ws/rpc` (overridable via `WHATSAPP_RPC_URL`). For media URLs, the router additionally downloads via `httpx`.-- **File I/O**: none in this handler - file uploads are handled by the router.+- **Database writes**: none directly from this handler. (API-usage cost is recorded by the `@Operation` `cost` metadata via `BaseNode.execute()`.)+- **Broadcasts**: none directly - standard node_status transitions come from `BaseNode.execute()`.+- **External calls**: WebSocket RPC to the Go `whatsapp-rpc` service via [`nodes/whatsapp/_service.py::whatsapp_rpc_call`](../../../server/nodes/whatsapp/_service.py). Default URL `ws://localhost:9400/ws/rpc` (overridable via `WHATSAPP_RPC_URL`). For media URLs, `_service.py` additionally downloads via `httpx`.+- **File I/O**: none in this handler - file uploads are handled in `_service.py`.- **Subprocess**: none.## External Dependencies- **Credentials**: none at the node handler level. Pairing state is owned by the Go service; the handler is credential-free.- **Services**: `whatsapp-rpc` (Go) on port 9400 (default). `services.markdown_formatter.to_whatsapp` when `format_markdown` is enabled.-- **Python packages**: `asyncio`. (`httpx` is used inside the router for URL media fetches.)+- **Python packages**: `asyncio`. (`httpx` is used inside `_service.py` for URL media fetches.)- **Environment variables**: `WHATSAPP_RPC_URL` (optional, default `ws://localhost:9400/ws/rpc`).## Edge cases & known limits@@ -147,4 +148,4 @@ flowchart TD- **Skills using this as a tool**: [`whatsapp-send-skill/SKILL.md`](../../../server/skills/social_agent/whatsapp-send-skill/SKILL.md)- **Companion nodes**: [`whatsappDb`](./whatsappDb.md), [`whatsappReceive`](./whatsappReceive.md)-- **Architecture docs**: [Markdown Formatter](../../../CLAUDE.md) (see "Markdown Formatter Service"), WhatsApp RPC via `routers/whatsapp.py`+- **Architecture docs**: [Markdown Formatter](../../../CLAUDE.md) (see "Markdown Formatter Service"), WhatsApp RPC via `nodes/whatsapp/_service.py`diff --git a/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.md b/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.mdindex d1c2a755..8954574d 100644--- a/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.md+++ b/docs-internal/node-logic-flows/workflow_triggers/chatTrigger.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | workflow / trigger / utility |-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) |+| **Backend handler** | Plugin [`server/nodes/trigger/chat_trigger/__init__.py`](../../../server/nodes/trigger/chat_trigger/__init__.py) (`ChatTriggerNode`); dispatch via `BaseNode.execute()`. The base `TriggerNode.execute` handles the event-waiter wait — the `@Operation("wait")` body is a stub. The generic legacy path is [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (reachable only for the lone deferred-canary trigger, not chatTrigger). || **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -11,12 +11,15 @@## PurposeFires when the user sends a message from the Console Panel chat tab. The-WebSocket handler `send_chat_message` calls-`broadcaster.send_custom_event('chat_message_received', {...})` (or the-event-waiter dispatch path) and any `chatTrigger` node whose `sessionId`-matches (or is `'default'`) receives the event and emits it as output. This-is the primary way a user feeds an interactive prompt into an `aiAgent` or-`chatAgent`.+producer [`server/nodes/trigger/chat_trigger/_events.py`](../../../server/nodes/trigger/chat_trigger/_events.py)+emits a CloudEvents `WorkflowEvent` (`type: com.machinaos.chat.message.received`)+via `dispatch.emit`. `chatTrigger` is canary-registered+(`register_canary_trigger_type`), so `DeploymentManager` starts a+`TriggerListenerWorkflow` for it; the listener receives the event via Temporal+Signal and spawns a child `MachinaWorkflow` per matching event. Any `chatTrigger`+node whose `session_id` matches (or is `'default'`) receives the event and emits+it as output. This is the primary way a user feeds an interactive prompt into an+`aiAgent` or `chatAgent`.## Inputs (handles)@@ -28,7 +31,7 @@ is the primary way a user feeds an interactive prompt into an `aiAgent` or| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `sessionId` | string | `default` | no | - | Matches the `session_id` on the incoming chat event. If set to `default`, the filter accepts every event. Otherwise it only accepts events with the same `session_id`. |+| `session_id` | string | `default` | no | - | Matches the `session_id` on the incoming chat event. If set to `default`, the filter accepts every event. Otherwise it only accepts events with the same `session_id`. || `placeholder` | string | `Type a message...` | no | - | Frontend display only - not used by the handler. |## Outputs (handles)@@ -58,35 +61,34 @@ Wrapped in the standard envelope.```mermaidflowchart TD- A[handle_trigger_node] --> B[event_waiter.get_trigger_config 'chatTrigger']- B -- None --> Eunk[Return success=false<br/>error: Unknown trigger type]- B -- ok --> C[await event_waiter.register<br/>params include sessionId]- C --> D[build_chat_filter:<br/>if sessionId != 'default' require exact match]- D --> E[broadcaster.update_node_status 'waiting'<br/>event_type=chat_message_received]- E --> F[await event_waiter.wait_for_event waiter]- F -- CancelledError --> G[Return success=false<br/>error: Cancelled by user]- F -- Exception --> H[Return success=false<br/>error: str e]- F -- ok --> I[Return success envelope<br/>result = event_data]+ P[chat tab sends message] --> Q[_events.py dispatch.emit<br/>WorkflowEvent com.machinaos.chat.message.received]+ Q --> R[TriggerListenerWorkflow receives via Temporal Signal]+ R --> S[ChatTriggerNode.build_filter:<br/>if session_id != 'default' require exact match]+ S -- match --> T[spawn child MachinaWorkflow<br/>trigger pre-executed with event payload]+ S -- no match --> R```## Decision Logic-- **Filter** (`build_chat_filter` in `triggers.py`):+- **Filter** (`ChatTriggerNode.build_filter`):```python- if session_id != 'default' and event_session != session_id:- return False+ if session_id and session_id != 'default':+ return event.get('session_id') == session_idreturn True```- So `sessionId='default'` (the frontend default) is a wildcard - the+ So `session_id='default'` (the frontend default) is a wildcard - thetrigger fires for every chat message regardless of which session the useris in.- **Cancellation**: yields `success=False, error="Cancelled by user"`.## Side Effects-- **Database writes**: none in the trigger handler itself. (The chat message+- **Database writes**: none in the trigger plugin itself. (The chat messageis separately persisted by `send_chat_message` via `database.add_chat_message`.)-- **Broadcasts**: `update_node_status(node_id, "waiting", {message, event_type, waiter_id}, workflow_id)`.+- **Broadcasts**: the producer emits a CloudEvents `WorkflowEvent` via+ `dispatch.emit` (Temporal Signal fan-out + in-process WS broadcast). The+ `TriggerListenerWorkflow` emits firing-pulse status via+ `broadcast_trigger_status_activity` before/after each child spawn.- **External API calls**: none.- **File I/O**: none.- **Subprocess**: none.@@ -100,7 +102,7 @@ flowchart TD## Edge cases & known limits-- `sessionId='default'` behaves as a wildcard; setting a unique session ID+- `session_id='default'` behaves as a wildcard; setting a unique session IDper trigger is the only way to scope messages to a specific node.- When multiple `chatTrigger` nodes exist with the same session ID, all ofthem fire for a matching message.diff --git a/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.md b/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.mdindex cd80cfe1..ff32095d 100644--- a/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.md+++ b/docs-internal/node-logic-flows/workflow_triggers/cronScheduler.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | workflow / trigger / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/utility.py::handle_cron_scheduler`](../../../server/services/handlers/utility.py) |+| **Backend handler** | Plugin [`server/nodes/scheduler/cron_scheduler/__init__.py`](../../../server/nodes/scheduler/cron_scheduler/__init__.py) (`CronSchedulerNode`); dispatch via `BaseNode.execute()` + the `@Operation("trigger")` method. Deployed-workflow cron lifecycle lives in `DeploymentManager` (Temporal Schedules canary via `CronTriggerWorkflow`); this op only runs on a manual / AI-tool invocation. || **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) || **Skill (if any)** | [`server/skills/task_agent/cron-scheduler-skill/SKILL.md`](../../../server/skills/task_agent/cron-scheduler-skill/SKILL.md) || **Dual-purpose tool** | yes - exposed on the `tool` output handle |@@ -34,11 +34,12 @@ call `event_waiter.register()`.| `daily_time` | options | `09:00` | no | frequency == `days` | Display-only in the handler. || `weekday` | options | `'1'` (Mon) | no | frequency == `weeks` | Display-only. || `weekly_time` | options | `09:00` | no | frequency == `weeks` | Display-only. |-| `month_day` | options | `'1'` | no | frequency == `months` | Display-only. |-| `monthly_time` | options | `09:00` | no | frequency == `months` | Display-only. |+| `month_day` | options | `'1'` | no | frequency == `months` | `1`..`28` or `L`. Display-only in this op. || `timezone` | options | `UTC` | no | - | Used in the description string only. |+| `cron_expression` | string | `0 * * * *` | no | - | Vestigial; kept for backward compatibility, unused by the op. |+| `monthly_time` | string | `09:00` | no | - | Vestigial; kept for backward compatibility, no displayOptions. |-Note: for `days` / `weeks` / `months` the handler hard-codes the wait to 24h+Note: for `days` / `weeks` / `months` the op hard-codes the wait to 24h/ 7d / ~30d respectively; the `daily_time` / `weekday` / `weekly_time` /`month_day` / `monthly_time` parameters only flow into the human-readable`schedule` string, not the actual wait duration.@@ -48,7 +49,8 @@ Note: for `days` / `weeks` / `months` the handler hard-codes the wait to 24h| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Schedule metadata (see below). |-| `output-tool` | object | Same payload when invoked as an AI agent tool. |++Dual-purpose: the plugin sets `usable_as_tool = True` with `tool_name = "cron_scheduler"`, so when wired to an agent's `input-tools` handle the LLM fills the same Params schema and receives the same payload. There is no separate `output-tool` handle declared.### Output payload@@ -73,7 +75,7 @@ Wrapped in the standard envelope.```mermaidflowchart TD- A[handle_cron_scheduler] --> B[frequency = parameters.frequency default 'minutes']+ A[BaseNode.execute -> trigger op] --> B[frequency = params.frequency default 'minutes']B --> C[_get_schedule_description<br/>returns e.g. 'Every 5 minutes']C --> D[_calculate_wait_seconds<br/>maps frequency to seconds]D --> E[get_status_broadcaster.update_node_status 'waiting']@@ -122,20 +124,20 @@ flowchart TD## Edge cases & known limits-- **Handler is not the real scheduler.** Real cron semantics+- **This op is not the real scheduler.** Real cron semantics(exact-time-of-day, weekday selection, timezones) live in- `DeploymentManager` + APScheduler. The manual-run handler does not use any- of that - it only sleeps for a fixed interval. `daily_time`, `weekday`,- `weekly_time`, `month_day`, `monthly_time`, and `timezone` are- **display-only** in this code path.-- `iteration` is always `1`. The handler does not loop; repeated firings- come from APScheduler calling the handler again each tick (deployment mode).-- `interval`, `interval_minutes`, `interval_hours` are coerced with- `int(...)`; non-numeric values raise `ValueError` which is caught and- returned as an error envelope.-- `weekday` is read via `parameters.get('weekday', '1')` and expected to be- a string digit - if a number is passed it will raise on `.isdigit()` in- `_get_schedule_description`, but that only affects the description string.+ `DeploymentManager` + the Temporal Schedule canary (`CronTriggerWorkflow`).+ The manual-run op does not use any of that - it only sleeps for a fixed+ interval. `daily_time`, `weekday`, `weekly_time`, `month_day`,+ `monthly_time`, and `timezone` are **display-only** in this code path.+- `iteration` is always `1`. The op does not loop; repeated firings+ come from the deployed Schedule firing per tick (deployment mode).+- `interval`, `interval_minutes`, `interval_hours` are Pydantic-validated+ ints with ranges (`ge`/`le`); out-of-range values fail validation before+ the op runs.+- `weekday` is a Pydantic `Literal["0".."6"]`; the op casts it via+ `int(weekday)` inside `_get_schedule_description`, affecting only the+ description string.- Unknown `frequency` values silently use 300s wait and produce`schedule: "Unknown schedule"`.diff --git a/docs-internal/node-logic-flows/workflow_triggers/start.md b/docs-internal/node-logic-flows/workflow_triggers/start.mdindex fb4b57b7..d0d47d01 100644--- a/docs-internal/node-logic-flows/workflow_triggers/start.md+++ b/docs-internal/node-logic-flows/workflow_triggers/start.md@@ -2,8 +2,8 @@| Field | Value ||------|-------|-| **Category** | workflow / trigger |-| **Backend handler** | [`server/services/handlers/utility.py::handle_start`](../../../server/services/handlers/utility.py) |+| **Category** | workflow |+| **Backend handler** | Plugin [`server/nodes/workflow/start/__init__.py`](../../../server/nodes/workflow/start/__init__.py) (`StartNode`); dispatch via `BaseNode.execute()` + the `@Operation("emit")` method. || **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -27,7 +27,7 @@ data.| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `initialData` | string (JSON) | `"{}"` | no | - | JSON document surfaced as the node output. Any parse failure silently becomes `{}`. |+| `initial_data` | any (JSON string or value) | `None` | no | - | Surfaced as the node output. If a JSON string, it is parsed (parse failure -> `{}`); if `None` -> `{}`; otherwise returned as-is. |## Outputs (handles)@@ -38,9 +38,11 @@ data.### Output payload```ts-// Whatever the user put in initialData, parsed as JSON.-// Invalid JSON -> {}.-Record<string, unknown>+// initial_data resolved:+// None -> {}+// JSON str -> parsed value (invalid -> {})+// any other -> returned as-is+unknown```Wrapped in the standard envelope: `{ success: true, result: <payload>, node_id, node_type: "start" }`.@@ -49,23 +51,25 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, node_id,```mermaidflowchart TD- A[handle_start] --> B[Read parameters.initialData<br/>default: '{}']- B --> C{json.loads?}- C -- ok --> D[initial_data = parsed]- C -- Exception --> E[initial_data = {}]- D --> F[Return success envelope<br/>result = initial_data]- E --> F+ A[BaseNode.execute -> emit op] --> B[raw = params.initial_data]+ B --> C{raw is None?}+ C -- yes --> E[return {}]+ C -- no --> D{raw is str?}+ D -- yes --> F{json.loads?}+ F -- ok --> G[return parsed]+ F -- Exception --> E+ D -- no --> H[return raw as-is]```## Decision Logic-- **Validation**: none. Empty / missing `initialData` is treated as `"{}"`.-- **Branches**: single path; either `initialData` parses or it is silently- replaced with `{}`.+- **Validation**: none. `None` `initial_data` returns `{}`.+- **Branches**: `None` -> `{}`; string -> `json.loads` (failure -> `{}`);+ any other value returned unchanged.- **Fallbacks**: JSON decode errors are swallowed and produce `{}`.-- **Error paths**: the handler does not raise; it always returns a- `success=True` envelope. A malformed `initialData` is **not** reported as an- error.+- **Error paths**: the op does not raise; it always returns a+ `success=True` envelope. A malformed `initial_data` string is **not** reported+ as an error.## Side Effects@@ -85,16 +89,14 @@ flowchart TD## Edge cases & known limits-- `start` always returns `success=True` regardless of `initialData` content.- Invalid JSON is silently coerced to `{}` - users get no warning their data- was dropped.-- Although `start` is listed in `TRIGGER_REGISTRY` (as `deploy_triggered`),- the handler registry in `NodeExecutor._build_handler_registry()` maps- `start` directly to `handle_start`. The event-waiter path is only exercised- inside `DeploymentManager` when a workflow is deployed; ad-hoc runs go- through `handle_start` without waiting for any event.-- The payload is returned as-is; no validation, schema enforcement, or nested- template resolution is performed by the handler.+- `start` always returns `success=True` regardless of `initial_data` content.+ Invalid JSON in a string is silently coerced to `{}` - users get no warning+ their data was dropped.+- `start` is a plain `ActionNode` (group `workflow`), not an event trigger -+ it registers no event waiter and completes immediately. It carries+ `ui_hints` `hideInputSection` / `hideOutputSection` / `hasInitialDataBlob`.+- The payload is returned as-is for non-string values; no validation, schema+ enforcement, or nested template resolution is performed by the op.## Relateddiff --git a/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.md b/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.mdindex 905d1bb3..c371ebb6 100644--- a/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.md+++ b/docs-internal/node-logic-flows/workflow_triggers/taskTrigger.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | workflow / trigger |-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) |+| **Backend handler** | Plugin [`server/nodes/trigger/task_trigger/__init__.py`](../../../server/nodes/trigger/task_trigger/__init__.py) (`TaskTriggerNode`); dispatch via `BaseNode.execute()`. The base `TriggerNode.execute` runs the event wait; the `@Operation("wait")` body is a stub. || **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -13,10 +13,14 @@Fires when a child agent delegated via `delegate_to_*` tools completes orerrors. The delegation code path in`server/services/handlers/tools.py::_execute_delegated_agent` calls-`broadcaster.send_custom_event('task_completed', {...})`, which is-dispatched to this trigger via `event_waiter`. Used to let a parent-workflow react to child completion without blocking the parent on the-child's result (fire-and-forget delegation).+`nodes.agent._events.broadcast_agent_task_completed` for both the success and+error paths, which emits a CloudEvents `WorkflowEvent`+(`type: com.machinaos.agent.task.completed`, with the success/error+discriminator carried in `data.status`) via `dispatch.emit`. `taskTrigger` is+canary-registered, so `DeploymentManager` starts a `TriggerListenerWorkflow`+that receives the event via Temporal Signal and spawns a child workflow per+match. Used to let a parent workflow react to child completion without blocking+the parent on the child's result (fire-and-forget delegation).## Inputs (handles)@@ -41,39 +45,38 @@ child's result (fire-and-forget delegation).### Output payload+Declared `TaskTriggerOutput` fields:+```ts{- task_id: string;- status: 'completed' | 'error';- agent_name: string;- agent_node_id: string;- parent_node_id: string;+ task_id?: string;+ status?: 'completed' | 'error';+ agent_name?: string;result?: string; // Present when status='completed'error?: string; // Present when status='error'- workflow_id: string;+ workflow_id?: string;}```-Wrapped in the standard envelope.+`TaskTriggerOutput` is `model_config extra="allow"`, so the producer's extra+fields (`agent_node_id`, `parent_node_id`) pass through to downstream nodes at+runtime even though they are not declared on the model. Wrapped in the standard+envelope.## Logic Flow```mermaidflowchart TD- A[handle_trigger_node] --> B[event_waiter.get_trigger_config 'taskTrigger']- B -- None --> Eunk[Return success=false<br/>error: Unknown trigger type]- B -- ok --> C[await event_waiter.register]- C --> D[build_task_completed_filter]- D --> E[broadcaster.update_node_status 'waiting'<br/>event_type=task_completed]- E --> F[await event_waiter.wait_for_event waiter]- F -- CancelledError --> G[Return success=false<br/>error: Cancelled by user]- F -- Exception --> H[Return success=false<br/>error: str e]- F -- ok --> I[Return success envelope<br/>result = event_data]+ P[child agent completes/errors] --> Q[broadcast_agent_task_completed<br/>dispatch.emit com.machinaos.agent.task.completed]+ Q --> R[TriggerListenerWorkflow receives via Temporal Signal]+ R --> S[TaskTriggerNode.build_filter<br/>task_id / status / agent_name / parent_node_id]+ S -- match --> T[spawn child MachinaWorkflow<br/>trigger pre-executed with event payload]+ S -- no match --> R```## Decision Logic-`build_task_completed_filter` applies all configured filters as AND:+`TaskTriggerNode.build_filter` applies all configured filters as AND:- **task_id**: exact match if non-empty.- **agent_name**: `agent_name_filter.lower() in event_agent.lower()` -@@ -89,7 +92,9 @@ If any filter rejects, the event is skipped and the waiter stays blocked.## Side Effects- **Database writes**: none.-- **Broadcasts**: `update_node_status(node_id, "waiting", {message, event_type, waiter_id}, workflow_id)`.+- **Broadcasts**: the producer emits a CloudEvents `WorkflowEvent` via+ `dispatch.emit`. The `TriggerListenerWorkflow` emits firing-pulse status via+ `broadcast_trigger_status_activity` around each child spawn.- **External API calls**: none.- **File I/O**: none.- **Subprocess**: none.@@ -97,9 +102,11 @@ If any filter rejects, the event is skipped and the waiter stays blocked.## External Dependencies- **Credentials**: none.-- **Services**: `services.event_waiter`, `services.status_broadcaster`.+- **Services**: `services.deployment` (canary listener), `services.events.dispatch`,+ `services.status_broadcaster`.- **Upstream dispatcher**: `services.handlers.tools._execute_delegated_agent`- fires `task_completed` events for both success and error paths.+ calls `nodes.agent._events.broadcast_agent_task_completed` for both success and+ error paths.- **Python packages**: stdlib only.- **Environment variables**: none.diff --git a/docs-internal/node-logic-flows/workflow_triggers/timer.md b/docs-internal/node-logic-flows/workflow_triggers/timer.mdindex eda42c21..8d5b1827 100644--- a/docs-internal/node-logic-flows/workflow_triggers/timer.md+++ b/docs-internal/node-logic-flows/workflow_triggers/timer.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | workflow / utility / tool (dual-purpose) |-| **Backend handler** | [`server/services/handlers/utility.py::handle_timer`](../../../server/services/handlers/utility.py) |+| **Backend handler** | Plugin [`server/nodes/scheduler/timer/__init__.py`](../../../server/nodes/scheduler/timer/__init__.py) (`TimerNode`); dispatch via `BaseNode.execute()` + the `@Operation("wait")` method. || **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) || **Skill (if any)** | none || **Dual-purpose tool** | yes - exposed on the `tool` output handle so AI agents can invoke it |@@ -26,7 +26,7 @@ of next-run; it simply pauses the branch of the workflow it is on for| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `duration` | number | `5` | yes | - | How long to wait. Frontend clamps 1..3600. |+| `duration` | number | `1` | yes | - | How long to wait. Pydantic-validated `ge=1, le=86400`. || `unit` | options | `seconds` | no | - | One of `seconds` / `minutes` / `hours`. Unknown values are treated as `seconds`. |## Outputs (handles)@@ -34,7 +34,8 @@ of next-run; it simply pauses the branch of the workflow it is on for| Handle | Shape | Description ||--------|-------|-------------|| `output-main` | object | Timing metadata (see below). |-| `output-tool` | object | Same payload when used as an AI agent tool. |++Dual-purpose: the plugin sets `usable_as_tool = True` with `tool_name = "timer"`, so when wired to an agent's `input-tools` handle the LLM fills the same Params schema. There is no separate `output-tool` handle declared.### Output payload@@ -54,7 +55,7 @@ Wrapped in the standard envelope: `{ success: true, result: <payload>, execution```mermaidflowchart TD- A[handle_timer] --> B[duration = int parameters.duration default 5]+ A[BaseNode.execute -> wait op] --> B[duration = int params.duration]B --> C{unit?}C -- seconds --> S1[wait_seconds = duration]C -- minutes --> S2[wait_seconds = duration * 60]@@ -101,13 +102,12 @@ flowchart TD- Only three unit strings are supported - anything else is silently treatedas raw seconds. There is no surfaced validation error.-- The handler has no timeout ceiling of its own; the frontend clamps at- 3600 (1h) for `seconds`, but if a workflow injects a larger value via the- backend it will actually wait that long.-- Cancellation returns a failed envelope (`success=false`). Downstream nodes- that key off `success` will not execute when the timer is cancelled.-- Duration is coerced with `int(...)`; a non-numeric string raises `ValueError`- which is caught by the generic handler and turned into an error envelope.+- `duration` is Pydantic-validated `ge=1, le=86400`; for `hours` that is up+ to 86400 hours, so a large value with `unit=hours` can sleep effectively+ indefinitely.+- Cancellation (`asyncio.CancelledError`) is re-raised as `RuntimeError("Timer+ cancelled")`, surfaced as a failed envelope. Downstream nodes that key off+ `success` will not execute when the timer is cancelled.## Relateddiff --git a/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.md b/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.mdindex 9336905b..f63c1269 100644--- a/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.md+++ b/docs-internal/node-logic-flows/workflow_triggers/webhookResponse.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | workflow / utility |-| **Backend handler** | [`server/services/handlers/http.py::handle_webhook_response`](../../../server/services/handlers/http.py) |+| **Backend handler** | Plugin [`server/nodes/utility/webhook_response/__init__.py`](../../../server/nodes/utility/webhook_response/__init__.py) (`WebhookResponseNode`); dispatch via `BaseNode.execute()` + the `@Operation("respond")` method (body inlined from the deleted `handlers/http.py`). || **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -22,21 +22,22 @@ out.| Handle | Connection type | Required | Purpose ||--------|-----------------|----------|---------|-| `input-main` | main | yes (semantically) | Upstream node outputs are collected via `_get_connected_outputs_with_info` and made available for template substitution / default body. |+| `input-main` | main | yes (semantically) | Upstream node outputs are surfaced as `ctx.raw["connected_outputs"]` and made available for template substitution / default body. |## Parameters| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `statusCode` | number | `200` | no | - | HTTP status code. Coerced via `int(...)`. |-| `responseBody` | string | `""` | no | - | Response body. Supports `{{input.<field>}}` and `{{<nodeType>.<field>}}` template substitutions using connected upstream outputs. |-| `contentType` | options | `application/json` | no | - | One of `application/json` / `text/plain` / `text/html`. |+| `status_code` | number | `200` | no | - | HTTP status code. Validated `100..599` (Pydantic `ge=100, le=599`). |+| `body` | any | `None` | no | - | Response body. When a string, supports `{{input.<field>}}` and `{{<nodeType>.<field>}}` template substitutions using connected upstream outputs. Non-string values are JSON-serialized. |+| `headers` | object | `{}` | no | - | Header dict (currently not forwarded by `resolve_webhook_response`). |+| `content_type` | string | `application/json` | no | - | Free-form content type string. |## Outputs (handles)| Handle | Shape | Description ||--------|-------|-------------|-| (none) | - | `webhookResponse` has no output handle - it terminates a response branch. The handler still returns a standard envelope (used by the executor for status tracking). |+| `output-main` | object | The respond op's return payload (see below); typically terminates a response branch. |### Handler return payload@@ -55,29 +56,26 @@ Wrapped in the standard envelope.```mermaidflowchart TD- A[NodeExecutor._dispatch] --> B[_get_connected_outputs_with_info<br/>returns connected_outputs + source_nodes]- B --> C[handle_webhook_response<br/>node_id / params / context / connected_outputs]- C --> D[status_code = int params.statusCode default 200]- D --> E[response_body = params.responseBody]- E --> F{response_body non-empty AND connected_outputs?}- F -- yes --> G[For each node_type, output in connected_outputs:<br/>replace '{{input.key}}' and '{{node_type.key}}' with str value]+ A[BaseNode.execute -> respond op] --> B[outputs = ctx.raw connected_outputs]+ B --> D[body = params.body]+ D --> F{body is str AND outputs?}+ F -- yes --> G[For each node_type, output in outputs:<br/>replace '{{input.key}}' and '{{node_type.key}}' with str value]F -- no --> H- G --> H{response_body still empty?}- H -- yes AND connected_outputs present --> I[response_body = json.dumps first output]+ G --> H{body falsy AND outputs present?}+ H -- yes --> I[body = json.dumps first output]H -- no --> J- I --> J[resolve_webhook_response node_id, statusCode, body, contentType]- J --> K[Return success envelope]- J -- Exception --> L[Return success=false<br/>error: str e]+ I --> J[resolve_webhook_response node_id, status_code, body_text, content_type]+ J --> K[Return sent / statusCode / contentType / bodyLength]```## Decision Logic-- **Template resolution**: only applied when `response_body` is non-empty.- Two template formats are supported:+- **Template resolution**: only applied when `body` is a non-empty string AND+ there are connected outputs. Two template formats are supported:- `{{input.<key>}}` - pulls from any connected node's output dict- `{{<nodeType>.<key>}}` - pulls from a specific connected node type-- **Empty body fallback**: if `response_body` is empty AND at least one- upstream node has output, the handler JSON-serialises the FIRST output+- **Empty body fallback**: if `body` is falsy AND at least one+ upstream node has output, the op JSON-serialises the FIRST output(iteration order of a Python dict keyed by node type) and uses that asthe body. If there are no upstream outputs the body stays empty.- **resolve_webhook_response**: imported lazily from `routers.webhook`;@@ -105,8 +103,8 @@ flowchart TD- `resolve_webhook_response` is looked up by `node_id`. If no matchingwebhookTrigger is pending (e.g. this node runs in a workflow without an- upstream `webhookTrigger`, or the trigger uses `responseMode=immediate`),- `resolve_webhook_response` simply no-ops - the handler still returns+ upstream `webhookTrigger`, or the trigger uses `response_mode=immediate`),+ `resolve_webhook_response` simply no-ops - the op still returns`success=True`. There is no warning that the response was dropped.- Template substitution uses plain `str.replace`; there is no escaping.Binary / non-stringifiable values will render as their Python `repr`.@@ -114,12 +112,10 @@ flowchart TD(insertion order in CPython 3.7+) - which in turn depends on edge order.If multiple upstream nodes are connected, the output picked is notexplicitly controlled by the user.-- `statusCode` coercion via `int(...)` will raise on non-numeric strings,- which is then caught and returned as a failed envelope.-- Unlike other handlers in the registry, `webhookResponse` is dispatched- in `NodeExecutor._dispatch` (not the handler registry) because it needs- the `connected_outputs` list - see- [`node_executor.py:367-381`](../../../server/services/node_executor.py).+- `status_code` is validated by Pydantic (`ge=100, le=599`); an out-of-range+ value fails validation and produces an error envelope before the op runs.+- `webhookResponse` reads `ctx.raw["connected_outputs"]` — the upstream+ outputs are surfaced into the node context by the executor for this plugin.## Relateddiff --git a/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.md b/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.mdindex bf586d5a..0c8065be 100644--- a/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.md+++ b/docs-internal/node-logic-flows/workflow_triggers/webhookTrigger.md@@ -3,7 +3,7 @@| Field | Value ||------|-------|| **Category** | workflow / trigger |-| **Backend handler** | [`server/services/handlers/triggers.py::handle_trigger_node`](../../../server/services/handlers/triggers.py) (generic) |+| **Backend handler** | Plugin [`server/nodes/trigger/webhook_trigger/__init__.py`](../../../server/nodes/trigger/webhook_trigger/__init__.py) (`WebhookTriggerNode`); dispatch via `BaseNode.execute()`. The base `TriggerNode.execute` runs the event wait; the `@Operation("wait")` body is a stub. || **Tests** | [`server/tests/nodes/test_workflow_triggers.py`](../../../server/tests/nodes/test_workflow_triggers.py) || **Skill (if any)** | none || **Dual-purpose tool** | no |@@ -11,11 +11,13 @@## PurposeStart a workflow when an HTTP request hits `/webhook/{path}`. The-`webhook` router in `server/routers/webhook.py` receives the request and-dispatches a `webhook_received` event via-`event_waiter.dispatch()`. This node registers an asyncio.Future waiter-(memory mode) or a Redis-Streams consumer group (Redis mode) via the shared-`event_waiter` module and blocks until a matching event is delivered.+`webhook` router in `server/routers/webhook.py` receives the request and the+producer [`server/nodes/trigger/webhook_trigger/_events.py`](../../../server/nodes/trigger/webhook_trigger/_events.py)+emits a CloudEvents `WorkflowEvent` (`type: com.machinaos.webhook.received`)+via `dispatch.emit`. `webhookTrigger` is canary-registered, so+`DeploymentManager` starts a `TriggerListenerWorkflow` that receives the event+via Temporal Signal and spawns a child `MachinaWorkflow` per matching event+(filtered by `path`).## Inputs (handles)@@ -27,12 +29,12 @@ dispatches a `webhook_received` event via| Name | Type | Default | Required | displayOptions.show | Description ||------|------|---------|----------|---------------------|-------------|-| `path` | string | `""` | yes | - | URL path segment - full URL is `http://host:3010/webhook/{path}`. Must match the incoming request's `path` for the filter to accept it. |-| `method` | options | `POST` | no | - | Filter for HTTP method at the router layer (not the handler's filter). Values: `GET` / `POST` / `PUT` / `DELETE` / `ALL`. |-| `responseMode` | options | `immediate` | no | - | `immediate` returns 200 OK right away; `responseNode` waits for a downstream `webhookResponse` node (see [`webhookResponse`](./webhookResponse.md)). |+| `path` | string | `""` | no | - | URL path segment - full URL is `http://host:3010/webhook/{path}`. Must match the incoming request's `path` for the filter to accept it. Empty path = wildcard (matches any). |+| `method` | options | `POST` | no | - | Filter for HTTP method at the router layer (not the plugin filter). Values: `GET` / `POST` / `PUT` / `DELETE` / `ALL`. |+| `response_mode` | options | `immediate` | no | - | `immediate` returns 200 OK right away; `responseNode` waits for a downstream `webhookResponse` node (see [`webhookResponse`](./webhookResponse.md)). || `authentication` | options | `none` | no | - | `none` or `header`. |-| `headerName` | string | `X-API-Key` | no | authentication == `header` | Expected header name. |-| `headerValue` | string | `""` | no | authentication == `header` | Expected header value. |+| `header_name` | string | `X-API-Key` | no | authentication == `header` | Expected header name. |+| `header_value` | string (secret) | `""` | no | authentication == `header` | Expected header value. |## Outputs (handles)@@ -61,21 +63,17 @@ Wrapped in the standard envelope.```mermaidflowchart TD- A[handle_trigger_node] --> B[event_waiter.get_trigger_config 'webhookTrigger']- B -- None --> Eunk[Return success=false<br/>error: Unknown trigger type]- B -- ok --> C[await event_waiter.register<br/>node_type, node_id, params]- C --> D[build_webhook_filter: checks path equality]- D --> E[StatusBroadcaster.update_node_status 'waiting'<br/>with event_type=webhook_received, waiter_id]- E --> F[await event_waiter.wait_for_event waiter]- F -- CancelledError --> G[Return success=false<br/>error: Cancelled by user]- F -- Exception --> H[Return success=false<br/>error: str e]- F -- ok --> I[Return success envelope<br/>result = event_data]+ P[HTTP request hits /webhook/path] --> Q[routers/webhook.py + _events.py<br/>dispatch.emit com.machinaos.webhook.received]+ Q --> R[TriggerListenerWorkflow receives via Temporal Signal]+ R --> S[WebhookTriggerNode.build_filter:<br/>event.path == params.path]+ S -- match --> T[spawn child MachinaWorkflow<br/>trigger pre-executed with event payload]+ S -- no match --> R```## Decision Logic-- **Filter match** (`build_webhook_filter` in `triggers.py`): accepts any- event whose `data.path` equals `params.path`. If `path` is empty the+- **Filter match** (`WebhookTriggerNode.build_filter`): accepts any+ event whose `path` equals `params.path`. If `path` is empty thefilter accepts anything.- **Method / authentication** are NOT enforced inside the filter - they aresupposed to be enforced at the router layer when the HTTP request lands.@@ -85,10 +83,10 @@ flowchart TD## Side Effects-- **Database writes**: none inside the handler. (Redis mode writes waiter- metadata to a Redis key `waiters:<uuid>` with a 24h TTL via `CacheService`.)-- **Broadcasts**: `update_node_status(node_id, "waiting", {message, event_type, waiter_id}, workflow_id)`- exactly once when the waiter is registered.+- **Database writes**: none inside the plugin.+- **Broadcasts**: the producer emits a CloudEvents `WorkflowEvent` via+ `dispatch.emit`. The `TriggerListenerWorkflow` emits firing-pulse status via+ `broadcast_trigger_status_activity` around each child spawn.- **External API calls**: none.- **File I/O**: none.- **Subprocess**: none.@@ -96,9 +94,9 @@ flowchart TD## External Dependencies- **Credentials**: none.-- **Services**: `services.event_waiter`, `services.status_broadcaster`,- `routers.webhook` (dispatches events via `broadcaster.send_custom_event` /- `event_waiter.dispatch`).+- **Services**: `services.deployment` (canary listener), `services.events.dispatch`,+ `services.status_broadcaster`, `routers.webhook` (receives the HTTP request and+ drives `_events.py` to emit the event).- **Python packages**: stdlib only.- **Environment variables**: none.diff --git a/docs-internal/node_creation.md b/docs-internal/node_creation.mdindex 3ff26134..440e6fa1 100644--- a/docs-internal/node_creation.md+++ b/docs-internal/node_creation.md@@ -36,68 +36,18 @@ stripe) layer additional bases from `services.events` underneath them.## Five-minute recipe — single fileFor nodes with no state, no daemon, no signed webhooks (the common-case):--```python-# server/nodes/<group>/<name>.py-from pydantic import BaseModel, Field-from services.plugin import (- ActionNode, ApiKeyCredential, NodeContext, Operation, TaskQueue,-)---class MyCredential(ApiKeyCredential):- id = "my_api"- display_name = "My Service"- category = "Search"- key_name = "X-API-Key"---class MyParams(BaseModel):- query: str = Field(..., min_length=1)---class MyOutput(BaseModel):- results: list---class MyNode(ActionNode):- type = "myNode"- display_name = "My Node"- # Icon + color: drop `icon.svg` into this plugin folder + create- # `meta.json` with `{"color": "#abcdef"}`. For multi-node folders- # (one folder, multiple node classes — telegram / whatsapp / stripe),- # use `icon_<nodeType>.svg` for per-node-type icons; the resolver- # picks per-node first, falling back to shared `icon.svg`. emoji /- # lobehub:<brand> live in `visuals.json` for icon-only entries.- group = ("search", "tool")- description = "Brief description for palette + AI tool."- component_kind = "square"- handles = (- {"name": "input-main", "kind": "input", "position": "left",- "label": "Input", "role": "main"},- {"name": "output-main", "kind": "output", "position": "right",- "label": "Output", "role": "main"},- )- credentials = (MyCredential,)- task_queue = TaskQueue.REST_API- usable_as_tool = True- Params = MyParams- Output = MyOutput-- @Operation("search")- async def search(self, ctx: NodeContext, params: MyParams) -> MyOutput:- async with ctx.connection("my_api") as conn:- resp = await conn.get(- "https://api.example.com/search",- params={"q": params.query},- )- return MyOutput(results=resp.json().get("hits", []))-```-+case): the canonical single-file recipe code block lives in the+cookbook — see+[`server/nodes/README.md` → Five-minute recipe](../server/nodes/README.md#five-minute-recipe).++It is a single `server/nodes/<group>/<name>.py` declaring a+`Credential` subclass, a `Params` model, an `Output` model, and one+`ActionNode` subclass with `type` / `display_name` / `group` /+`component_kind` / `handles` / `credentials` / `task_queue` /+`usable_as_tool` / `Params` / `Output` plus one `@Operation` method.That's the entire node. On server restart it auto-registers, the-NodeSpec is emitted at `/api/schemas/nodes/myNode/spec.json`, and it-appears in the Component Palette under the `search` group.+NodeSpec is emitted at `/api/schemas/nodes/<type>/spec.json`, and it+appears in the Component Palette under its first `group` entry.## Five-minute recipe — Wave 12 self-contained folder (signed webhook + CLI)diff --git a/docs-internal/onboarding.md b/docs-internal/onboarding.mdindex de9d46ac..2be60e17 100644--- a/docs-internal/onboarding.md+++ b/docs-internal/onboarding.md@@ -4,6 +4,8 @@The onboarding service provides a multi-step welcome wizard that appears after a user's first launch, guiding them through platform capabilities, key concepts, API key setup, UI layout, and getting started. It is database-backed, skippable, resumable, and replayable from Settings.+The frontend is **fully shadcn/ui + Tailwind** — antd was removed from `client/src/`. The wizard composes the project's `Modal` primitive, shadcn `Button` / `ActionButton` / `Card` / `Badge` / `Alert`, and `lucide-react` icons. The step progress indicator is a hand-rolled `<ol>` driven by node-role tokens (no antd `Steps`).+## Architecture```@@ -12,24 +14,27 @@ The onboarding service provides a multi-step welcome wizard that appears after a│ ┌────────────────────────────────────────────────────────┐ ││ │ OnboardingWizard.tsx │ ││ │ ┌────────────────────────────────────────────────┐ │ │-│ │ │ useOnboarding(reopenTrigger) │ │ │-│ │ │ - Checks onboarding_completed via WebSocket │ │ │+│ │ │ useOnboarding(reopenTrigger, STEPS.length) │ │ │+│ │ │ - Reads onboarding_completed/step via │ │ │+│ │ │ useUserSettingsQuery (TanStack Query, WS) │ │ ││ │ │ - Manages step navigation + persistence │ │ ││ │ └───────────────┬────────────────────────────────┘ │ ││ │ │ │ │-│ │ ┌───────┬───────┼───────┬──────────┐ │ │-│ │ │Step 0 │Step 1 │Step 2 │Step 3 │Step 4 │ │-│ │ │Welcome│Concept│APIKey │Canvas │GetStarted │ │-│ │ └───────┴───────┴───────┴──────────┘ │ │+│ │ STEPS array (single source of truth in wizard): │ │+│ │ ┌───────┬───────┬───────┬──────────┬──────────┐ │ │+│ │ │Step 0 │Step 1 │Step 2 │Step 3 │Step 4 │ │ │+│ │ │Welcome│Concept│APIKey │Canvas │GetStarted│ │ │+│ │ └───────┴───────┴───────┴──────────┴──────────┘ │ ││ │ │ │-│ │ Modal (Radix UI) + Steps (Ant Design) │ │+│ │ Modal (project primitive, Radix-backed) │ │+│ │ + <ol> progress stepper (Tailwind + role tokens) │ ││ └────────────────────────────────────────────────────────┘ ││ ││ SettingsPanel.tsx → "Replay Welcome Guide" button ││ └── onReplayOnboarding → increments reopenTrigger │└────────────────────────────────────────────────────────────────┘│ │- │ WebSocket │ WebSocket+ │ WebSocket (via TanStack Q) │ WebSocket▼ ▼┌──────────────────────────────────────────────────────────────┐│ server/routers/websocket.py │@@ -82,16 +87,19 @@ if "onboarding_step" not in columns:```client/src/├── components/-│ └── onboarding/-│ ├── OnboardingWizard.tsx # Main wizard modal orchestrator-│ └── steps/-│ ├── WelcomeStep.tsx # Step 0: Platform intro-│ ├── ConceptsStep.tsx # Step 1: Nodes, Edges, Agents, Skills, Modes-│ ├── ApiKeyStep.tsx # Step 2: AI provider key setup-│ ├── CanvasStep.tsx # Step 3: UI layout tour-│ └── GetStartedStep.tsx # Step 4: First workflow tips+│ ├── onboarding/+│ │ ├── OnboardingWizard.tsx # Main wizard modal orchestrator + STEPS SSOT + <ol> stepper+│ │ ├── nodeRoleClasses.ts # Shared role→Tailwind-token map for step cards+│ │ └── steps/+│ │ ├── WelcomeStep.tsx # Step 0: Platform intro+│ │ ├── ConceptsStep.tsx # Step 1: Nodes, Edges, Agents, Skills, Modes+│ │ ├── ApiKeyStep.tsx # Step 2: AI provider key setup+│ │ ├── CanvasStep.tsx # Step 3: UI layout tour+│ │ └── GetStartedStep.tsx # Step 4: First workflow tips+│ └── icons/+│ └── AIProviderIcons.tsx # Provider SVG icons used by ApiKeyStep└── hooks/- └── useOnboarding.ts # State management + WebSocket persistence+ └── useOnboarding.ts # State management + TanStack-Query persistence```## Components@@ -100,29 +108,33 @@ client/src/**Location**: `client/src/hooks/useOnboarding.ts`-Custom hook managing the full onboarding lifecycle:+Custom hook managing the full onboarding lifecycle. Persistence rides the **TanStack Query** server-state layer (`useUserSettingsQuery` / `useSaveUserSettingsMutation` from `useUserSettingsQuery.ts`), which are themselves WebSocket-backed (`get_user_settings` / `save_user_settings`). The hook does NOT call WebSocket handlers directly.```typescript-export const useOnboarding = (reopenTrigger?: number) => {- // Returns:+export const useOnboarding = (+ reopenTrigger?: number,+ totalSteps: number = 5, // DEFAULT_TOTAL_STEPS; caller passes STEPS.length+) => {+ // Returns (spread of OnboardingState + actions):// - isVisible: boolean - Whether wizard should render- // - currentStep: number - Active step index (0-4)+ // - currentStep: number - Active step index (0..totalSteps-1)// - isCompleted: boolean - Whether already completed/skipped- // - isLoading: boolean - WebSocket check in progress- // - hasChecked: boolean - Initial check done- // - totalSteps: number - Always 5- // - nextStep(): void - Advance to next step (or complete if last)- // - prevStep(): void - Go back one step- // - skip(): void - Skip onboarding entirely- // - complete(): void - Mark as completed+ // - isLoading: boolean - Settings query in progress+ // - hasChecked: boolean - Initial hydration done+ // - totalSteps: number - Echoed back from the param+ // - nextStep(): void - Advance (completes + persists when next >= totalSteps)+ // - prevStep(): void - Go back one step (clamped at 0)+ // - skip(): void - Skip: persist current step, completed=true, hide+ // - complete(): void - Persist totalSteps, completed=true, hide};```**Key behaviors**:-- On WebSocket connect, calls `get_user_settings` to check `onboarding_completed` (once, using `hasCheckedRef`)-- Each step transition calls `save_user_settings` to persist progress-- `reopenTrigger` prop change resets state and reopens wizard from step 0-- StrictMode-safe via `hasCheckedRef` pattern (same as Dashboard.tsx)+- `totalSteps` is a **parameter** (default 5). The wizard owns the step list and passes `STEPS.length`, so the hook never hardcodes the count — it uses `totalSteps` only to detect last-step completion in `nextStep`.+- Hydrates UI state from `settingsQuery.data` on `isSuccess`: reads `onboarding_completed` / `onboarding_step`. Visibility flips only on first hydration (`prev.hasChecked ? prev.isVisible : !completed`) so a user-closed wizard does not re-open on later query refetches.+- Each navigation (`nextStep` / `prevStep` / `skip` / `complete`) calls `saveSettings.mutate({ onboarding_step, onboarding_completed })` to persist progress.+- Query errors surface as a non-blocking "checked" state (`isLoading=false, hasChecked=true`) so the app continues even if the round-trip failed.+- `reopenTrigger` prop change (when `> 0`) resets state and reopens the wizard from step 0.### OnboardingWizard@@ -134,27 +146,47 @@ export const useOnboarding = (reopenTrigger?: number) => {| `onOpenCredentials` | `() => void` | Opens CredentialsModal (passed from Dashboard) || `reopenTrigger` | `number?` | Incrementing counter triggers wizard reopen |-**UI Structure**:-- Uses existing `Modal` component (`maxWidth="580px"`, `maxHeight="70vh"`, `autoHeight`)-- Ant Design `Steps` component for progress indicator-- Step content rendered via switch on `currentStep`-- Footer: "Skip for now" (left) | "Back" + "Next"/"Start Building" (right)-- Button colors: Next = `dracula.purple`, final "Start Building" = `dracula.green`-- Only renders when `isVisible && hasChecked && !isLoading`+**`STEPS` is the single source of truth.** The wizard declares a module-scope `STEPS` array of `{ title, render }` entries. Its `.length` feeds the hook's `totalSteps`, the progress indicator renders one node per entry, and the active step's `render` is dispatched by index. Adding a step is a one-line edit to this array.++```typescript+const STEPS = [+ { title: 'Welcome', render: () => <WelcomeStep /> },+ { title: 'Concepts', render: () => <ConceptsStep /> },+ { title: 'API Keys', render: ({ onOpenCredentials }) => <ApiKeyStep onOpenCredentials={onOpenCredentials} /> },+ { title: 'Canvas', render: () => <CanvasStep /> },+ { title: 'Get Started', render: () => <GetStartedStep /> },+];+```++**UI Structure** (all shadcn/Tailwind, no antd):+- Project `Modal` primitive with `maxWidth="95vw"`, `maxHeight="95vh"`, titled "Welcome Guide"; `onClose` is wired to `skip`.+- Progress indicator: a hand-rolled `<ol>` of step pills. Each pill is a rounded number/`Check` (lucide) badge with one of three statuses — `completed` (filled `bg-primary text-primary-foreground`), `active` (`border-primary text-primary`), `upcoming` (`border-border text-muted-foreground`) — joined by a connector `<div>` (`bg-primary` once passed, else `bg-border`). No antd `Steps`.+- Step content rendered via `STEPS[safeIndex].render({ onOpenCredentials })` inside a scrollable `max-h-[calc(95vh-200px)]` container.+- Footer: shadcn `Button variant="ghost"` "Skip for now" (left) | `Button variant="outline"` "Back" (shown when `currentStep > 0`) + an `ActionButton` on the right.+- The right-side primary button uses **`ActionButton` intents**, not raw colour hex: `<ActionButton intent="tools">` for "Next" (with `ArrowRight`), `<ActionButton intent="run">` for the final "Start Building" (with `Check`). The pre-antd-removal dracula-purple/green hardcoding is gone.+- Only renders when `isVisible && hasChecked && !isLoading`.++### Node-role token map++**Location**: `client/src/components/onboarding/nodeRoleClasses.ts`++`NODE_ROLE_CLASSES` maps a `NodeRole` (`model | skill | agent | workflow | trigger`) to the matching `--node-X` triplet (`{ card: 'bg-node-X-soft border-node-X-border', text: 'text-node-X' }`). `ConceptsStep` and `GetStartedStep` key their card surfaces off this so the cards track every theme with **no opacity arithmetic at the call site**.### Step Components-All steps use Ant Design components (`Typography`, `Card`, `Tag`, `Space`, `Button`, `Alert`, `Row`, `Col`) and `@ant-design/icons` for consistent theming.+All steps are shadcn/Tailwind compositions using `lucide-react` icons (`Card` / `CardContent`, `Badge`, `Alert` / `AlertDescription`, `Button`, plus role tokens). No antd, no `@ant-design/icons`.++| Step | Component | Title | Purpose | Primitives used |+|------|-----------|-------|---------|-----------------|+| 0 | `WelcomeStep` | Welcome to MachinaOs | Platform intro + 2×2 feature grid | `Card` / `CardContent`, lucide `Rocket` / `Plug` / `Move` / `Zap`, `bg-node-agent-soft` |+| 1 | `ConceptsStep` | Key Concepts | Nodes, Edges, AI Agents, Skills & Tools, Normal vs Dev Mode | role-token cards via `NODE_ROLE_CLASSES`, lucide `LayoutGrid` / `GitBranch` / `Bot` / `Wrench` / `ArrowLeftRight` |+| 2 | `ApiKeyStep` | API Key Setup | Provider list + "Open Credentials" button | shadcn `Button`, `Alert variant="info"`, `AIProviderIcons`, lucide `Key` / `ExternalLink` |+| 3 | `CanvasStep` | Canvas Tour | Visual UI layout diagram + keyboard shortcuts | `Badge`, role-token region tints, lucide `Layout` / `Wrench` / `Terminal` |+| 4 | `GetStartedStep` | Get Started | Example workflows, quick recipe, tips | `Card` / `CardContent`, `Badge`, role-token cards, lucide `Play` / `FlaskConical` / `BookOpen` / `Settings` |-| Step | Component | Title | Purpose | Ant Design Components |-|------|-----------|-------|---------|----------------------|-| 0 | `WelcomeStep` | Welcome to MachinaOs | Platform intro + feature highlights | `Card`, `Row`, `Col`, `Typography` |-| 1 | `ConceptsStep` | Key Concepts | Nodes, Edges, Agents, Skills, Normal/Dev Mode | `Space`, `Typography` |-| 2 | `ApiKeyStep` | API Key Setup | Provider list + "Open Credentials" button | `Button`, `Alert`, `Space` |-| 3 | `CanvasStep` | Canvas Tour | Visual UI layout diagram + shortcuts | `Tag`, `Space`, `Typography` |-| 4 | `GetStartedStep` | Get Started | Example workflows, quick recipe, tips | `Card`, `Tag`, `Space` |+**ApiKeyStep** accepts an `onOpenCredentials` prop to link to the existing CredentialsModal without duplicating key input logic. It lists six providers (OpenAI, Anthropic, Google, Groq, OpenRouter, Cerebras) with brand icons from `client/src/components/icons/AIProviderIcons.tsx` (imported as `../../icons/AIProviderIcons`), and closes with an `Alert variant="info"` noting keys can be changed later from the toolbar.-**ApiKeyStep** accepts an `onOpenCredentials` prop to link to the existing CredentialsModal without duplicating key input logic.+**CanvasStep** renders a miniature UI-region diagram whose tints map node groups to regions: toolbar→`workflow`, sidebar→`model`, canvas→`agent`, palette→`skill`, console→`trigger`, each via the `--node-X-soft` / `text-node-X` tokens. The shortcut list (`Ctrl+S` Save, `F2` Rename node, `Delete` Remove node, `Ctrl+C` Copy node) renders as `Badge variant="outline"` mono chips.## Integration Points@@ -181,18 +213,15 @@ const [onboardingReopenTrigger, setOnboardingReopenTrigger] = React.useState(0);### SettingsPanel.tsx-Added `onReplayOnboarding?: () => void` prop and a "Help" section with:-- `QuestionCircleOutlined` icon header-- "Replay Welcome Guide" button styled with `dracula.cyan`-- Description text+`SettingsPanel` takes an `onReplayOnboarding?: () => void` prop. The Help section renders a shadcn `Button variant="default"` "Replay Welcome Guide" (lucide `HelpCircle` icon, `disabled` when the callback is absent) that fires `onReplayOnboarding`.## WebSocket Handlers-No new handlers were needed. The onboarding system reuses existing generic handlers:+No new handlers were needed. The onboarding system reuses existing generic handlers, accessed through the TanStack Query user-settings layer:| Handler | Usage ||---------|-------|-| `get_user_settings` | Check `onboarding_completed` and `onboarding_step` on connect |+| `get_user_settings` | Check `onboarding_completed` and `onboarding_step` on hydration || `save_user_settings` | Persist step progress on each navigation, skip, or complete |## Lifecycle@@ -200,10 +229,10 @@ No new handlers were needed. The onboarding system reuses existing generic handl### First Launch (New User)1. User opens app, WebSocket connects-2. `useOnboarding` calls `get_user_settings` -- no settings exist yet+2. `useOnboarding` reads `useUserSettingsQuery` -- no settings exist yet3. `onboarding_completed` defaults to `false`, `onboarding_step` defaults to `0`4. Wizard opens at step 0-5. User navigates steps -- each transition saves via `save_user_settings`+5. User navigates steps -- each transition saves via the save mutation (`save_user_settings`)6. On "Start Building" or "Skip for now", `onboarding_completed` set to `true`7. Wizard closes, does not reappear on refresh@@ -227,9 +256,8 @@ No new handlers were needed. The onboarding system reuses existing generic handl2. `onReplayOnboarding()` callback fires:- Closes SettingsPanel- Increments `onboardingReopenTrigger`-3. `useOnboarding` detects trigger change:- - Resets `hasCheckedRef` to allow re-check- - Sets `isVisible = true, currentStep = 0`+3. `useOnboarding` detects the trigger change (`> 0`):+ - Sets `isVisible = true, currentStep = 0, isCompleted = false`4. Wizard opens from step 0## Edge Cases@@ -237,38 +265,42 @@ No new handlers were needed. The onboarding system reuses existing generic handl| Scenario | Behavior ||----------|----------|| Auth disabled (`VITE_AUTH_ENABLED=false`) | Works unchanged -- reads from `user_id="default"` |-| WebSocket not connected yet | Hook waits for `isConnected`, `isLoading=true` prevents render |+| Settings query not resolved yet | `isLoading=true` prevents render until `hasChecked` |+| Settings query errors | Non-blocking: `isLoading=false, hasChecked=true`, app continues || Browser closed mid-wizard | `onboarding_step` saved on each transition, resumes from last step |-| Multiple tabs | Completing in one tab doesn't update others until refresh |-| Replay from Settings | Resets backend state and reopens wizard from step 0 |-| Fresh database (no machina.db) | Onboarding appears after first WebSocket connect |+| Multiple tabs | Completing in one tab doesn't update others until query refetch |+| Replay from Settings | Resets local state and reopens wizard from step 0 |+| Fresh database (no machina.db) | Onboarding appears after first settings query resolves |## Verification Checklist-1. **Fresh database**: Delete `server/machina.db`, start server -- wizard appears-2. **Step navigation**: Click through all 5 steps -- progress bar updates, Back/Next work+1. **Fresh database**: Delete `server/machina.db` (or the configured DB), start server -- wizard appears+2. **Step navigation**: Click through all 5 steps -- the `<ol>` stepper updates, Back/Next work3. **Skip**: Click "Skip for now" -- wizard closes, doesn't reappear on refresh4. **Resume**: Advance to step 3, close browser, reopen -- wizard resumes at step 35. **Complete**: Finish all steps via "Start Building" -- wizard doesn't reappear6. **API Key step**: Click "Open Credentials" button -- CredentialsModal opens7. **Existing user migration**: With existing `machina.db` where `examples_loaded=1` -- onboarding does NOT appear-8. **Theme support**: Toggle dark/light mode -- all onboarding UI adapts correctly+8. **Theme support**: Switch themes -- role-token cards and region tints adapt correctly9. **Replay**: Open Settings, click "Replay Welcome Guide" -- wizard reopens from step 0-10. **TypeScript**: `npx tsc --noEmit` passes clean+10. **TypeScript**: `tsgo --noEmit` (or `npx tsc --noEmit`) passes clean## Key Files| File | Description ||------|-------------|-| `client/src/hooks/useOnboarding.ts` | Onboarding state hook with WebSocket persistence |-| `client/src/components/onboarding/OnboardingWizard.tsx` | Main wizard modal with Ant Design Steps |+| `client/src/hooks/useOnboarding.ts` | Onboarding state hook; persists via TanStack-Query user-settings layer |+| `client/src/hooks/useUserSettingsQuery.ts` | `useUserSettingsQuery` / `useSaveUserSettingsMutation` (WS-backed) |+| `client/src/components/onboarding/OnboardingWizard.tsx` | Main wizard modal: `STEPS` SSOT + `<ol>` stepper + ActionButton footer |+| `client/src/components/onboarding/nodeRoleClasses.ts` | `NODE_ROLE_CLASSES` role→token map for step cards || `client/src/components/onboarding/steps/WelcomeStep.tsx` | Step 0: Platform introduction |-| `client/src/components/onboarding/steps/ConceptsStep.tsx` | Step 1: Key concepts (Nodes, Edges, Agents) |+| `client/src/components/onboarding/steps/ConceptsStep.tsx` | Step 1: Key concepts (Nodes, Edges, Agents, Skills, Modes) || `client/src/components/onboarding/steps/ApiKeyStep.tsx` | Step 2: API key setup with Credentials link || `client/src/components/onboarding/steps/CanvasStep.tsx` | Step 3: UI layout diagram + shortcuts || `client/src/components/onboarding/steps/GetStartedStep.tsx` | Step 4: Getting started tips |+| `client/src/components/icons/AIProviderIcons.tsx` | Provider brand icons used by ApiKeyStep || `client/src/Dashboard.tsx` | Integration: renders wizard + passes replay trigger |-| `client/src/components/ui/SettingsPanel.tsx` | Replay button in Help section |+| `client/src/components/ui/SettingsPanel.tsx` | "Replay Welcome Guide" button in Help section || `server/models/database.py` | `UserSettings.onboarding_completed`, `onboarding_step` fields || `server/core/database.py` | Migration + CRUD for onboarding fields |@@ -276,8 +308,6 @@ No new handlers were needed. The onboarding system reuses existing generic handlTo add a new onboarding step:-1. Create `client/src/components/onboarding/steps/NewStep.tsx` using Ant Design components-2. Import in `OnboardingWizard.tsx` and add to the `renderStep()` switch-3. Add entry to `stepItems` array-4. Update `TOTAL_STEPS` in `useOnboarding.ts`-5. No backend changes needed (step index is just a number)+1. Create `client/src/components/onboarding/steps/NewStep.tsx` composing shadcn primitives + Tailwind tokens + lucide icons (use `NODE_ROLE_CLASSES` for tinted cards). Do NOT introduce antd.+2. Add a `{ title, render }` entry to the `STEPS` array in `OnboardingWizard.tsx`. Its `.length` automatically updates the hook's `totalSteps` and the progress stepper — no separate count to maintain.+3. No backend changes needed (step index is just a number).diff --git a/docs-internal/performance.md b/docs-internal/performance.mdindex 9759f08d..9c0a865f 100644--- a/docs-internal/performance.md+++ b/docs-internal/performance.md@@ -76,15 +76,15 @@ T+12.27 — Status broadcasters settled ◀ "everything green"is idle in this window; the cost lives on the frontend. Likelycontributors:-1. **TanStack Query auth-bootstrap retry budget** ([client/src/contexts/AuthContext.tsx](client/src/contexts/AuthContext.tsx) + [client/src/lib/connectionConfig.ts](client/src/lib/connectionConfig.ts)). The `AUTH_RETRY` envelope (BASE 50 ms, CAP 4000 ms, MAX_ATTEMPTS 7) covers the typical 4 s backend cold-start window in 4-5 attempts. If the backend finishes mid-retry-cycle, the next jittered draw can land 1-3 s after readiness.-2. **React Strict Mode dual-mount** in dev. The 100 ms guard in [WebSocketContext.tsx:2554](client/src/contexts/WebSocketContext.tsx#L2554) absorbs the bulk; remaining cost is React reconciliation + babel-plugin-react-compiler overhead on first render.+1. **TanStack Query auth-bootstrap retry budget** ([client/src/contexts/AuthContext.tsx](../client/src/contexts/AuthContext.tsx) + [client/src/lib/connectionConfig.ts](../client/src/lib/connectionConfig.ts)). The `AUTH_RETRY` envelope (BASE 50 ms, CAP 4000 ms, MAX_ATTEMPTS 7) covers the typical 4 s backend cold-start window in 4-5 attempts. If the backend finishes mid-retry-cycle, the next jittered draw can land 1-3 s after readiness.+2. **React Strict Mode dual-mount** in dev. The 100 ms guard in [WebSocketContext.tsx:2554](../client/src/contexts/WebSocketContext.tsx#L2554) absorbs the bulk; remaining cost is React reconciliation + babel-plugin-react-compiler overhead on first render.3. **PartySocket upgrade handshake**. Sub-100 ms in normal cases; would only matter on slow networks.To attribute definitively: add `console.time('auth.queryFn')` / `console.timeLog('auth.queryFn')` markers and a corresponding pair around `connect()`. Not blocking — the contract this layer was meant to fix (the +12 s disconnect-reconnect cycle) is resolved.## Frontend retry / reconnect envelope-Single source of truth: [client/src/lib/connectionConfig.ts](client/src/lib/connectionConfig.ts).+Single source of truth: [client/src/lib/connectionConfig.ts](../client/src/lib/connectionConfig.ts).| Constant | Value | Notes ||---|---|---|@@ -101,7 +101,7 @@ Backoff formula (AWS Architecture Blog "full jitter" pattern):sleep = random(0, min(CAP_MS, BASE_MS * 2^attempt))-Lock-in tests: [client/src/lib/__tests__/connectionConfig.test.ts](client/src/lib/__tests__/connectionConfig.test.ts).+Lock-in tests: [client/src/lib/__tests__/connectionConfig.test.ts](../client/src/lib/__tests__/connectionConfig.test.ts).## Bottleneck inventory (cold-start, warm cache)diff --git a/docs-internal/plugin_system.md b/docs-internal/plugin_system.mdindex 8e294755..ebe98156 100644--- a/docs-internal/plugin_system.md+++ b/docs-internal/plugin_system.md@@ -19,65 +19,12 @@ pitfalls).## Quick start — adding a new node-```python-# server/nodes/search/example_search.py-from pydantic import BaseModel, Field-from services.plugin import (- ActionNode, ApiKeyCredential, NodeContext, Operation, TaskQueue,-)---class ExampleCredential(ApiKeyCredential):- id = "example_api"- display_name = "Example Search"- category = "Search"- key_name = "X-API-Key"- key_location = "header"---class ExampleParams(BaseModel):- query: str = Field(..., min_length=1)- max_results: int = Field(default=10, ge=1, le=50)---class ExampleOutput(BaseModel):- results: list- count: int---class ExampleSearchNode(ActionNode):- type = "exampleSearch"- display_name = "Example Search"- # Icon + color are NOT declared on the class. Drop `icon.svg` into- # the plugin folder; create `meta.json` with `{"color": "#abcdef"}`.- # `_metadata_dict` resolves both at registration time via the- # central handler in server/nodes/_visuals.py.- group = ("search", "tool")- description = "Example search via external API"- component_kind = "square"- handles = (- {"name": "input-main", "kind": "input", "position": "left",- "label": "Input", "role": "main"},- {"name": "output-main", "kind": "output", "position": "right",- "label": "Output", "role": "main"},- )- credentials = (ExampleCredential,)- task_queue = TaskQueue.REST_API- usable_as_tool = True- Params = ExampleParams- Output = ExampleOutput-- @Operation("search")- async def search(self, ctx: NodeContext, params: ExampleParams) -> ExampleOutput:- async with ctx.connection("example_api") as conn:- resp = await conn.get(- "https://api.example.com/search",- params={"q": params.query, "limit": params.max_results},- )- resp.raise_for_status()- data = resp.json()- return ExampleOutput(results=data.get("hits", []), count=len(data.get("hits", [])))-```+The full single-file recipe code block lives in the cookbook — see+[`server/nodes/README.md` → Five-minute recipe](../server/nodes/README.md#five-minute-recipe).+A self-contained folder (`AcmeSearchNode`) declares `type` /+`display_name` / `group` / `component_kind` / `handles` /+`credentials` / `task_queue` / `usable_as_tool` / `Params` / `Output`+plus one `@Operation` method, and nothing else.**That's it.** On server restart:diff --git a/docs-internal/theme_system.md b/docs-internal/theme_system.mdindex b5d10bff..9afd282d 100644--- a/docs-internal/theme_system.md+++ b/docs-internal/theme_system.md@@ -353,6 +353,38 @@ Selection state: `selected` co-class on the outer wrapper, driven by React Flow'Type-specific co-classes (Cyber-only today): `.node-agent` colors AIAgentNode neon magenta; `.sq-node.node-trigger .sq-node-box` colors TriggerNode neon green. Other themes ignore these co-classes — single colour by node type is fine when not differentiating.+## Canvas-wide edge + node status animations (`canvasAnimations.ts`)++[client/src/styles/canvasAnimations.ts](../client/src/styles/canvasAnimations.ts) owns the `@keyframes` + `.react-flow__edge.{status}` + `.react-flow__node.{status}` rules injected once into Dashboard's `<style>` tag. It is the single home for canvas-wide rules that need to match React Flow's wrapper classes (per-node inline animations live in their own components and read theme tokens directly — see the visual contract above).++**Three named exports** — adding a new keyframe or status visual is a single-file change:++| Symbol | Role |+|---|---|+| `KEYFRAMES` | `@keyframes` definitions for edges (`dashFlow` — the marching-ants stroke-dashoffset cycle on executing edges) |+| `edgeStatusStyles(colors)` | `.react-flow__edge.{selected,executing,completed,error,pending,memory-active,tool-active}` stroke rules |+| `nodeStatusStyles(_colors)` | `.react-flow__node.{...}` status-class colours (currently a no-op stub; the arg is kept on the signature so the `buildCanvasStyles` / `CanvasStatusColors` contract stays stable for downstream consumers) |+| `buildCanvasStyles(colors)` | Composes the three into the final string for Dashboard |++**Light/dark distinction lives entirely in `theme.ts`.** `buildCanvasStyles(colors)` is single-arg and the file ships **zero hardcoded hex colours** — the theme object provides different values per mode, so this module knows nothing about which theme is active.++`CanvasStatusColors` is the contract interface, carrying exactly eight edge-stroke keys:++```ts+export interface CanvasStatusColors {+ edgeDefault: string;+ edgeSelected: string;+ edgeExecuting: string;+ edgeCompleted: string;+ edgeError: string;+ edgePending: string;+ edgeMemoryActive: string;+ edgeToolActive: string;+}+```++**No `nodeGlow` keyframe here.** This module used to inject a `nodeGlow` keyframe targeting the React Flow wrapper, but it was dead code (only the inner `.node` child animated) and was removed. Node execution glow is owned solely by [base.css](../client/src/themes/base.css) — the `node-pulse` keyframe + `.react-flow__node.executing .node` / `.sq-node[data-executing] .sq-node-box` rules (see [`--node-pulse-color` contract](#done) above). Do not re-introduce a competing `nodeGlow` keyframe here.+## Sound systemEach theme declares `--sound-pack: "<pack>"` in its `:root[data-theme="..."]` block. Pack names map to event tables in [client/src/lib/sound.ts](../client/src/lib/sound.ts):diff --git a/docs-internal/tool_building_pipeline.md b/docs-internal/tool_building_pipeline.mdindex 18a3d643..7da6a056 100644--- a/docs-internal/tool_building_pipeline.md+++ b/docs-internal/tool_building_pipeline.md@@ -11,7 +11,7 @@ Canonical home for how AI Agents discover connected tool nodes, build LangChain## 1. Five stages```-1. DISCOVER handlers/ai.py:_collect_agent_connections+1. DISCOVER plugin/edge_walker.py:collect_agent_connections→ reads edges → tool_data list│▼@@ -32,9 +32,9 @@ Canonical home for how AI Agents discover connected tool nodes, build LangChain→ _dispatch_tool routes by node_type to a handler```-## 2. Stage 1 — Discovery (handlers/ai.py)+## 2. Stage 1 — Discovery (plugin/edge_walker.py)-[`_collect_agent_connections(node_id, context, database)`](../server/services/handlers/ai.py) walks the workflow edges and returns a 4-tuple per agent:+[`collect_agent_connections(node_id, context, database)`](../server/services/plugin/edge_walker.py) walks the workflow edges and returns a 4-tuple per agent:```(memory_data, skill_data, tool_data, input_data, task_data)diff --git a/server/nodes/README.md b/server/nodes/README.mdindex a393ffa3..3455ca8d 100644--- a/server/nodes/README.md+++ b/server/nodes/README.md@@ -265,6 +265,13 @@ package's `__init__.py` — the consumer never imports your folder.All six are idempotent (same callable / class for the same key is ano-op; conflicts raise `ValueError`).+**Register, don't edit the consumer.** Never edit `event_waiter.py`,+`status_broadcaster.py`, or `routers/websocket.py` to add a plugin's+handler / filter / refresh — call the matching `register_*` from your+plugin's `__init__.py` instead. Likewise never import a plugin folder+from `routers/`, `services/`, or another `nodes/` subfolder; the+consumer reaches your code through the registry, never by import path.+### Credential validation (Wave 11.I)The credential-validator dispatch is a sibling concern, handled by the