Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
Bump version to 0.0.10, track internal docs, clean up gitignore
trohitg453672e51b8b
diff --git a/.gitignore b/.gitignoreindex 43ee1e5d..9a231de3 100644--- a/.gitignore+++ b/.gitignore@@ -85,6 +85,7 @@ NUL# Build artifacts# =============================================================================build/+*.tgz*.exe*.dll*.so@@ -118,9 +119,9 @@ server/.python-version.claude/# =============================================================================-# Documentation (internal docs excluded, public docs tracked)+# Documentation (docs-MachinaOs is a separate GitHub project)# =============================================================================-docs-internal/+docs-MachinaOs/# =============================================================================# Experiments (work in progress)diff --git a/CLAUDE.md b/CLAUDE.mdindex 932faade..f695eb2d 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -13,6 +13,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire| **[AI Tool Node Guide](./docs-internal/ai_tool_node_creation.md)** | Detailed guide for creating dedicated AI Agent tool nodes (schemas, handlers, toolkits) || **[Specialized Agent Guide](./docs-internal/specialized_agent_node_creation.md)** | Guide for creating specialized AI agents (Android, Coding, Web, Task, Social, Travel, Tool, Productivity, Payments, Consumer) with full AI configuration || **[Dual-Purpose Tool Guide](./docs-internal/dual_purpose_tool_node_creation.md)** | Guide for nodes that work as both workflow nodes AND AI Agent tools (e.g., whatsappSend) |+| **[Agent Architecture](./docs-internal/agent_architecture.md)** | How AI Agent and Chat Agent discover skills/tools, inject them into LLM prompts, and execute via LangGraph || **[Workflow Schema](./docs-internal/workflow-schema.md)** | JSON schema for workflows, edge handle conventions, config node architecture || **[Execution Engine Design](./docs-internal/DESIGN.md)** | Architecture patterns, design standards, and implementation details for the workflow execution engine || **[Execution Roadmap](./docs-internal/ROADMAP.md)** | Implementation status, completed phases, and pending features |diff --git a/docs-internal/DESIGN.md b/docs-internal/DESIGN.mdnew file mode 100644index 00000000..5d66270b--- /dev/null+++ b/docs-internal/DESIGN.md@@ -0,0 +1,605 @@+# Execution Engine Design Document++## Overview++The MachinaOs execution engine implements a robust workflow orchestration system combining industry-standard patterns from **Netflix Conductor**, **Prefect 3.0**, and **Redis Streams**. This document details the architectural decisions, design patterns, and standards used.++---++## Architecture Principles++### 1. Isolated Execution Contexts+Each workflow execution has its own `ExecutionContext` - no shared global state between concurrent runs.++```+┌─────────────────────────────────────────────────────────────┐+│ WorkflowService │+│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │+│ │ Execution │ │ Execution │ │ Execution │ │+│ │ Context A │ │ Context B │ │ Context C │ │+│ │ (workflow 1) │ │ (workflow 2) │ │ (workflow 1) │ │+│ └──────────────┘ └──────────────┘ └──────────────┘ │+└─────────────────────────────────────────────────────────────┘+```++### 2. Separation of Concerns+- **Models** (`models.py`): Pure data structures, no business logic+- **Cache** (`cache.py`): Persistence layer abstraction+- **Executor** (`executor.py`): Orchestration logic+- **Recovery** (`recovery.py`): Fault tolerance++### 3. Dependency Injection+Services receive dependencies via constructor, enabling testing and flexibility:++```python+class WorkflowExecutor:+ def __init__(self, cache: ExecutionCache,+ node_executor: Callable,+ status_callback: Callable = None):+ self.cache = cache+ self.node_executor = node_executor+ self.status_callback = status_callback+```++---++## Design Patterns++### 1. Conductor's Decide Pattern++**Source**: Netflix Conductor OSS++The decide pattern is the core orchestration loop. It evaluates current state, finds ready tasks, executes them, and recurses until completion.++```python+async def _workflow_decide(self, ctx: ExecutionContext):+ """Core orchestration loop - Conductor's decide pattern."""+ async with distributed_lock(f"execution:{ctx.execution_id}:decide"):+ # 1. Find nodes ready to execute (dependencies satisfied)+ ready_nodes = self._find_ready_nodes(ctx)++ if not ready_nodes:+ return # All done or stuck++ # 2. Execute batch (parallel if multiple)+ if len(ready_nodes) > 1:+ await self._execute_parallel_nodes(ctx, ready_nodes)+ else:+ await self._execute_single_node(ctx, ready_nodes[0])++ # 3. Checkpoint and persist+ await self.cache.save_execution_state(ctx)++ # 4. Recurse for next batch+ await self._decide_iteration(ctx)+```++**Benefits**:+- Single point of orchestration logic+- Distributed lock prevents race conditions+- Natural support for parallel execution+- Easy to add features (retries, timeouts)++### 2. Fork/Join Parallel Execution++**Source**: Netflix Conductor + Java ForkJoinPool++Independent nodes (no dependencies on each other) execute in parallel using `asyncio.gather()`.++```python+async def _execute_parallel_nodes(self, ctx, nodes):+ """Fork/Join pattern with asyncio.gather."""+ tasks = [+ self._execute_node_with_caching(ctx, node)+ for node in nodes+ ]++ # True parallelism - all tasks run concurrently+ results = await asyncio.gather(*tasks, return_exceptions=True)++ # Join - process all results+ for node, result in zip(nodes, results):+ if isinstance(result, Exception):+ ctx.status = WorkflowStatus.FAILED+```++**DAG Layer Computation** (Kahn's Algorithm):++```+Layer 0: [Start] ← No dependencies+Layer 1: [A, B, C] ← Depend only on Start (parallel)+Layer 2: [D] ← Depends on A and B+Layer 3: [E, F] ← Depend on D (parallel)+Layer 4: [End] ← Depends on E and F+```++### 3. Prefect-Style Task Caching++**Source**: Prefect 3.0 Task Caching++Results are cached by hashing inputs, enabling idempotent execution.++```python+async def _execute_node_with_caching(self, ctx, node, enable_caching):+ # Generate cache key from inputs+ inputs = self._gather_node_inputs(ctx, node.node_id)++ if enable_caching:+ # Check cache first+ cached = await self.cache.get_cached_result(+ ctx.execution_id, node.node_id, inputs+ )+ if cached:+ node.status = TaskStatus.CACHED+ return cached++ # Execute and cache result+ result = await self.node_executor(...)++ if enable_caching and result.get("success"):+ await self.cache.set_cached_result(+ ctx.execution_id, node.node_id, inputs, result+ )++ return result+```++**Cache Key Generation**:++```python+def hash_inputs(inputs: Dict[str, Any]) -> str:+ """Create deterministic hash of inputs for cache key."""+ # Sort keys for determinism+ sorted_json = json.dumps(inputs, sort_keys=True, default=str)+ return hashlib.sha256(sorted_json.encode()).hexdigest()[:16]++def generate_cache_key(execution_id: str, node_id: str, inputs: Dict) -> str:+ """Generate full cache key."""+ input_hash = hash_inputs(inputs)+ return f"result:{execution_id}:{node_id}:{input_hash}"+```++### 4. State Machine Pattern++**Source**: Standard FSM + Conductor TaskStatus++Tasks and workflows follow explicit state machines:++```+TaskStatus State Machine:+ ┌─────────────────┐+ │ PENDING │+ └────────┬────────┘+ │ schedule+ ┌────────▼────────┐+ │ SCHEDULED │+ └────────┬────────┘+ │ start+ ┌──────────────────▼──────────────────┐+ │ RUNNING │+ └──┬──────────┬──────────┬──────────┬─┘+ │ │ │ │+ complete cache hit fail cancel+ │ │ │ │+ ┌────────▼───┐ ┌────▼────┐ ┌──▼───┐ ┌───▼─────┐+ │ COMPLETED │ │ CACHED │ │FAILED│ │CANCELLED│+ └────────────┘ └─────────┘ └──────┘ └─────────┘++WorkflowStatus State Machine:+ PENDING → RUNNING → COMPLETED+ ↓ ↓+ PAUSED FAILED+ ↓ ↓+ CANCELLED ←────┘+```++### 5. Distributed Locking++**Source**: Redis SETNX pattern++Prevents concurrent decide operations on the same execution:++```python+@asynccontextmanager+async def distributed_lock(self, lock_name: str, timeout: int = 60):+ """Acquire distributed lock using Redis SETNX."""+ lock_key = f"lock:{lock_name}"+ lock_token = str(uuid.uuid4())++ # Try to acquire lock+ acquired = await self.cache.redis.set(+ lock_key, lock_token,+ nx=True, # Only set if not exists+ ex=timeout # Auto-expire after timeout+ )++ if not acquired:+ raise TimeoutError(f"Could not acquire lock: {lock_name}")++ try:+ yield+ finally:+ # Release lock (only if we own it)+ current = await self.cache.redis.get(lock_key)+ if current == lock_token:+ await self.cache.redis.delete(lock_key)+```++### 6. Event Sourcing (Partial)++**Source**: Event Sourcing pattern + Redis Streams++All state changes are recorded as immutable events:++```python+# Event types+EVENT_TYPES = [+ "workflow_started",+ "node_started",+ "node_completed",+ "node_failed",+ "node_cached",+ "workflow_completed",+ "workflow_failed",+ "workflow_cancelled"+]++# Store event+await cache.add_event(execution_id, "node_completed", {+ "node_id": node.node_id,+ "execution_time": duration,+})++# Events stored in Redis Stream+# Key: execution:{id}:events+# Enables: replay, debugging, audit trail+```++### 7. Heartbeat Pattern++**Source**: Conductor + Distributed Systems++Running nodes send periodic heartbeats for crash detection:++```python+# During execution+await cache.update_heartbeat(execution_id, node_id)++# Recovery sweeper checks+for node_id, node_exec in ctx.node_executions.items():+ if node_exec.status == TaskStatus.RUNNING:+ last_heartbeat = await cache.get_heartbeat(execution_id, node_id)++ if time.time() - last_heartbeat > HEARTBEAT_TIMEOUT:+ # Node is stuck - trigger recovery+ needs_recovery = True+```++### 8. Adapter Pattern++**Source**: Gang of Four++Bridges new executor to existing node handlers without modification:++```python+async def _node_executor_adapter(self, node_id, node_type, parameters, context):+ """Adapter bridges WorkflowExecutor to existing execute_node()."""+ return await self.execute_node(+ node_id=node_id,+ node_type=node_type,+ parameters=parameters,+ nodes=context.get("nodes"),+ edges=context.get("edges"),+ session_id=context.get("session_id"),+ execution_id=context.get("execution_id"),+ )+```++---++## Redis Key Schema++```+# Execution State (HASH)+execution:{id}:state+ - status: WorkflowStatus+ - workflow_id: string+ - session_id: string+ - started_at: timestamp+ - completed_at: timestamp+ - current_layer: int+ - checkpoints: JSON array++# Node States (HASH)+execution:{id}:nodes+ - {node_id}: JSON(NodeExecution)++# Node Outputs (HASH)+execution:{id}:outputs+ - {node_id}: JSON(output data)++# Result Cache (STRING with TTL)+result:{execution_id}:{node_id}:{input_hash}+ - JSON(cached result)+ - TTL: 3600 seconds++# Event History (STREAM)+execution:{id}:events+ - Immutable append-only log+ - Fields: event_type, timestamp, data++# Heartbeats (STRING with TTL)+heartbeat:{execution_id}:{node_id}+ - timestamp+ - TTL: 60 seconds++# Distributed Locks (STRING with TTL)+lock:execution:{id}:decide+ - lock_token (UUID)+ - TTL: 60 seconds++# Active Executions (SET)+executions:active+ - Set of execution_ids currently running+```++---++## Data Models++### ExecutionContext++```python+@dataclass+class ExecutionContext:+ """Isolated state for a single workflow execution."""++ # Identity+ execution_id: str # Unique ID (auto-generated)+ workflow_id: str # Workflow definition ID+ session_id: str # User session++ # State+ status: WorkflowStatus # Current workflow status+ node_executions: Dict[str, NodeExecution] # Per-node state+ outputs: Dict[str, Any] # Node outputs for template resolution++ # Workflow Definition (immutable during execution)+ nodes: List[Dict] # Node definitions+ edges: List[Dict] # Edge definitions++ # Execution Progress+ execution_order: List[List[str]] # Computed layers+ current_layer: int # Current layer index+ checkpoints: List[str] # Completed node IDs++ # Timing+ started_at: float+ completed_at: float+ updated_at: float++ # Errors+ errors: List[Dict] # Error records+```++### NodeExecution++```python+@dataclass+class NodeExecution:+ """State for a single node within an execution."""++ node_id: str+ node_type: str+ status: TaskStatus++ # Execution details+ input_hash: str # For cache key+ output: Dict[str, Any]+ error: str++ # Timing+ started_at: float+ completed_at: float++ # Retry tracking (future)+ retry_count: int = 0+```++---++## Execution Flow++```+┌─────────────────────────────────────────────────────────────────────┐+│ execute_workflow() │+└────────────────────────────────┬────────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────────┐+│ 1. Create ExecutionContext │+│ - Generate unique execution_id │+│ - Initialize node_executions (all PENDING) │+│ - Compute execution layers (DAG analysis) │+└────────────────────────────────┬────────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────────┐+│ 2. Persist Initial State │+│ - Save to Redis: execution:{id}:state │+│ - Add to executions:active set │+│ - Emit workflow_started event │+└────────────────────────────────┬────────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────────┐+│ 3. workflow_decide() Loop │+│ ┌─────────────────────────────────────────────────────────────┐ │+│ │ a. Acquire distributed lock │ │+│ │ b. Find ready nodes (deps satisfied) │ │+│ │ c. Execute batch (parallel if multiple) │ │+│ │ d. Update node states │ │+│ │ e. Checkpoint to Redis │ │+│ │ f. Release lock │ │+│ │ g. Recurse until no ready nodes │ │+│ └─────────────────────────────────────────────────────────────┘ │+└────────────────────────────────┬────────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────────┐+│ 4. Finalize │+│ - Set final workflow status │+│ - Remove from executions:active │+│ - Emit workflow_completed/failed event │+│ - Return results │+└─────────────────────────────────────────────────────────────────────┘+```++---++## Recovery Mechanism++### Crash Detection++```+┌─────────────────────────────────────────────────────────────────────┐+│ RecoverySweeper (Background) │+│ │+│ Every 60 seconds: │+│ 1. Get all execution IDs from executions:active │+│ 2. For each execution: │+│ a. Load state from Redis │+│ b. Check if already complete → remove from active │+│ c. Find RUNNING nodes │+│ d. Check heartbeat age │+│ - If > 5 minutes → mark as stuck │+│ 3. Trigger recovery callback for stuck executions │+└─────────────────────────────────────────────────────────────────────┘+```++### Startup Recovery++```python+async def scan_on_startup(self) -> List[str]:+ """Find executions that need recovery on server restart."""+ needs_recovery = []++ for execution_id in await cache.get_active_executions():+ ctx = await cache.load_execution_state(execution_id)++ if ctx.status == WorkflowStatus.RUNNING:+ age = time.time() - ctx.updated_at+ if age > HEARTBEAT_TIMEOUT:+ needs_recovery.append(execution_id)++ return needs_recovery+```++---++## Thread Safety++### Cross-Thread Event Dispatch++APScheduler runs callbacks in thread pools. The event waiter handles this:++```python+def dispatch(event_type: str, data: Dict) -> int:+ """Thread-safe event dispatch."""+ try:+ # Check if we're in async context+ loop = asyncio.get_running_loop()+ asyncio.create_task(dispatch_async(event_type, data))+ except RuntimeError:+ # No event loop - we're in a thread+ if _main_loop and _main_loop.is_running():+ asyncio.run_coroutine_threadsafe(+ dispatch_async(event_type, data),+ _main_loop+ )+```++---++## Configuration++### Deployment Settings++```python+_deployment_settings = {+ "delay_between_runs": 1.0, # Seconds between iterations+ "stop_on_error": False, # Stop on first error+ "max_iterations": 0, # 0 = unlimited+ "use_parallel_executor": True # Use new parallel engine+}+```++### Execution Toggle++```python+# Automatic fallback+if use_parallel and settings.redis_enabled:+ return await self._execute_workflow_parallel(...)+else:+ return await self._execute_workflow_sequential(...)+```++---++## Comparison: Before vs After++| Aspect | Sequential (Before) | Parallel (After) |+|--------|---------------------|------------------|+| Execution | BFS traversal, one-by-one | DAG layers, concurrent |+| State | In-memory only | Redis-persisted |+| Recovery | None (lost on crash) | Automatic via sweeper |+| Idempotency | None | Input-hashed caching |+| Concurrency | Global flag blocks | Isolated contexts |+| Locking | None | Distributed Redis locks |+| Events | None | Redis Streams |++---++## Implemented Features++### Runtime Conditional Branching (conditions.py)+Edge conditions are evaluated at runtime using Prefect-style dynamic branching:+- 20+ operators: eq, neq, gt, lt, gte, lte, contains, exists, matches, in, starts_with, etc.+- Supports nested field access via dot notation (e.g., `result.status`)+- AND/OR condition groups with recursive evaluation++### Dead Letter Queue (dlq.py)+Failed nodes (after all retries exhausted) are stored for inspection and replay:+- `DLQHandler` - Active handler storing failed nodes with full context+- `NullDLQHandler` - No-op handler when DLQ disabled (Null Object pattern)+- WebSocket handlers: `get_dlq_entries`, `replay_dlq_entry`, `remove_dlq_entry`, `get_dlq_stats`++### Temporal Distributed Execution (Optional)+When `TEMPORAL_ENABLED=true`, each node executes as an independent Temporal activity:+- Per-node retry (3 attempts) and timeout (10 min default)+- Horizontal scaling across worker pool+- FIRST_COMPLETED pattern for dependency resolution+- Connection pooling for WebSocket activity execution++---++## Future Enhancements++### Planned+1. **Sub-Workflows** - Execute workflow as a node+2. **Metrics** - Prometheus integration+3. **Workflow Versioning** - Track definition versions++### Potential+1. **Distributed Workers** - Redis Streams consumer groups+2. **Priority Queues** - Task prioritization++---++## References++- **Netflix Conductor**: [Architecture](https://conductor-oss.github.io/conductor/devguide/architecture/index.html)+- **Prefect 3.0**: [Task Caching](https://docs.prefect.io/v3/develop/task-caching)+- **Redis Streams**: [Consumer Groups](https://redis.io/docs/data-types/streams/)+- **Kahn's Algorithm**: [Topological Sort](https://en.wikipedia.org/wiki/Topological_sorting#Kahn's_algorithm)+- **Event Sourcing**: [Martin Fowler](https://martinfowler.com/eaaDev/EventSourcing.html)diff --git a/docs-internal/ROADMAP.md b/docs-internal/ROADMAP.mdnew file mode 100644index 00000000..c6d8b4b6--- /dev/null+++ b/docs-internal/ROADMAP.md@@ -0,0 +1,545 @@+# Execution Engine Roadmap++## Overview++This document tracks the implementation status of the robust workflow execution engine based on industry-standard patterns from Netflix Conductor, Prefect 3.0, and Redis Streams.++---++## Completed Features++### Phase 1: State Machine + Caching+- [x] `TaskStatus` enum (PENDING, SCHEDULED, RUNNING, COMPLETED, FAILED, CACHED, CANCELLED, WAITING, SKIPPED)+- [x] `WorkflowStatus` enum (PENDING, RUNNING, PAUSED, COMPLETED, FAILED, CANCELLED)+- [x] `ExecutionContext` dataclass with isolated state per workflow run+- [x] `NodeExecution` dataclass for tracking individual node state+- [x] `execute_node_cached()` with input hashing for idempotency (Prefect pattern)+- [x] `hash_inputs()` and `generate_cache_key()` utilities+- [x] Redis HASH storage for execution state++### Phase 2: Decide Pattern + Parallel Execution+- [x] `workflow_decide()` with distributed locking (Conductor pattern)+- [x] `execute_parallel_nodes()` using `asyncio.gather()` for Fork/Join+- [x] `_compute_execution_layers()` for DAG topological sort+- [x] `_find_ready_nodes()` to detect nodes with satisfied dependencies+- [x] Redis distributed locks via SETNX with TTL++### Phase 4: Crash Recovery+- [x] `RecoverySweeper` background task+- [x] Heartbeat tracking for running nodes+- [x] `scan_on_startup()` to find interrupted executions+- [x] Automatic recovery trigger for stuck nodes+- [x] Integration in `main.py` lifespan++### Phase 5: Event History+- [x] Event storage in Redis Streams per execution+- [x] Event types: workflow_started, node_started, node_completed, node_failed, workflow_completed+- [x] `add_event()` and `get_events()` methods in ExecutionCache++### Phase 6: Integration with Existing Handlers+- [x] `_node_executor_adapter()` bridging WorkflowExecutor to existing handlers+- [x] `_get_workflow_executor()` for lazy initialization with status callback+- [x] `execute_workflow()` dispatches to parallel or sequential based on settings+- [x] `_execute_workflow_parallel()` using new WorkflowExecutor+- [x] `_execute_workflow_sequential()` as fallback when Redis unavailable+- [x] `use_parallel_executor` deployment setting (default: True)++---++## Completed Features (Continued)++### Phase 3: Runtime Conditional Branching [COMPLETED]++**Goal**: Enable dynamic workflow paths based on node output values at runtime.++**Status**: COMPLETED - Full implementation of Prefect-style runtime conditional branching.++**Implementation**:+- Created `server/services/execution/conditions.py` with 20+ condition operators+- Updated `executor.py` `_find_ready_nodes()` to evaluate edge conditions+- Added `TaskStatus.SKIPPED` handling for unmatched conditional branches+- Created `client/src/types/EdgeCondition.ts` with TypeScript types and utilities+- Created `client/src/components/EdgeConditionEditor.tsx` for condition UI+- Created `client/src/components/ConditionalEdge.tsx` for custom edge rendering+- Integrated ConditionalEdge into Dashboard with edgeTypes registration++#### Implementation Tasks++1. **Edge Condition Schema**+ ```typescript+ // Frontend: client/src/types/+ interface EdgeCondition {+ field: string; // Output field to check (e.g., "status", "result.success")+ operator: "eq" | "neq" | "gt" | "lt" | "gte" | "lte" | "contains" | "exists";+ value: any; // Value to compare against+ }++ interface ConditionalEdge extends Edge {+ data?: {+ condition?: EdgeCondition;+ label?: string; // Display label for the edge+ };+ }+ ```++2. **Condition Evaluator** (`server/services/execution/conditions.py`)+ ```python+ def evaluate_condition(condition: dict, output: dict) -> bool:+ """Evaluate edge condition against node output.++ Args:+ condition: {field, operator, value}+ output: Node execution output++ Returns:+ True if condition matches+ """+ field_value = get_nested_value(output, condition["field"])+ operator = condition["operator"]+ target_value = condition["value"]++ if operator == "eq":+ return field_value == target_value+ elif operator == "neq":+ return field_value != target_value+ elif operator == "gt":+ return field_value > target_value+ # ... etc+ ```++3. **Update `decide_next_node()` in executor.py**+ ```python+ def decide_next_node(self, ctx: ExecutionContext, current_node_id: str,+ output: dict) -> List[str]:+ """Determine next nodes based on edge conditions.++ Returns list of node IDs to execute next (may be multiple for fork).+ """+ next_nodes = []+ for edge in ctx.edges:+ if edge.get("source") != current_node_id:+ continue++ condition = edge.get("data", {}).get("condition")+ if condition:+ if evaluate_condition(condition, output):+ next_nodes.append(edge["target"])+ else:+ # No condition = always follow+ next_nodes.append(edge["target"])++ return next_nodes+ ```++4. **Frontend Edge Condition UI** (`client/src/components/EdgeConditionEditor.tsx`)+ - Modal to configure edge conditions+ - Field selector (from source node output schema)+ - Operator dropdown+ - Value input (type-aware based on field)+ - Visual indicator on conditional edges (different color/style)++5. **Edge Label Display** (`client/src/components/ConditionalEdge.tsx`)+ - Custom React Flow edge component+ - Shows condition summary as label+ - Different styling for conditional vs unconditional edges++#### Files Created/Modified++| File | Status | Description |+|------|--------|-------------|+| `server/services/execution/conditions.py` | DONE | Condition evaluation logic (20+ operators) |+| `server/services/execution/executor.py` | DONE | Updated `_find_ready_nodes()` with condition evaluation |+| `server/services/execution/models.py` | DONE | Added SKIPPED status handling |+| `client/src/types/EdgeCondition.ts` | DONE | Edge condition TypeScript types |+| `client/src/components/EdgeConditionEditor.tsx` | DONE | UI for editing conditions |+| `client/src/components/ConditionalEdge.tsx` | DONE | Custom edge rendering |+| `client/src/Dashboard.tsx` | DONE | Integrated edgeTypes for conditional edges |++---++### Phase 6: Integration - Connect WorkflowExecutor to Existing Handlers [COMPLETED]++**Goal**: Replace the sequential execution in `workflow.py` with the new parallel executor.++**Status**: COMPLETED - Parallel executor integrated with toggle for sequential fallback.++**Implementation**:+- Added `_node_executor_adapter()` to bridge WorkflowExecutor to existing handlers+- Added `_get_workflow_executor()` for lazy initialization with status callback+- Modified `execute_workflow()` to dispatch to parallel or sequential based on settings+- Added `_execute_workflow_parallel()` using new WorkflowExecutor+- Renamed original method to `_execute_workflow_sequential()` as fallback+- Added `use_parallel_executor` to deployment settings (default: True)+- Parallel execution automatically enabled when Redis is available++#### Implementation Tasks++1. **Create Adapter Function**+ ```python+ # server/services/workflow.py++ async def _node_executor_adapter(self, node_id: str, node_type: str,+ parameters: dict, context: dict) -> dict:+ """Adapter to bridge WorkflowExecutor to existing node handlers."""+ # Reuse existing execute_node logic+ return await self.execute_node(+ node_id=node_id,+ node_type=node_type,+ parameters=parameters,+ nodes=context.get("nodes"),+ edges=context.get("edges"),+ session_id=context.get("session_id"),+ execution_id=context.get("execution_id"),+ )+ ```++2. **Initialize WorkflowExecutor in WorkflowService**+ ```python+ def __init__(self, ...):+ # ... existing init ...++ # Initialize execution engine+ from services.execution import WorkflowExecutor, ExecutionCache+ self._execution_cache = ExecutionCache(cache)+ self._workflow_executor = WorkflowExecutor(+ cache=self._execution_cache,+ node_executor=self._node_executor_adapter,+ status_callback=self._broadcast_node_status,+ )+ ```++3. **Replace execute_workflow Implementation**+ ```python+ async def execute_workflow(self, nodes, edges, session_id, status_callback):+ """Execute workflow using parallel executor."""+ workflow_id = f"workflow_{session_id}"++ return await self._workflow_executor.execute_workflow(+ workflow_id=workflow_id,+ nodes=nodes,+ edges=edges,+ session_id=session_id,+ enable_caching=True,+ )+ ```++4. **Update deploy_workflow for Concurrent Executions**+ ```python+ async def deploy_workflow(self, nodes, edges, session_id, status_callback):+ """Deploy workflow with support for concurrent executions."""+ # Generate unique execution_id per deployment iteration+ # Track multiple active executions+ # Use WorkflowExecutor for each iteration+ ```++5. **Wire Status Callback to WebSocket Broadcaster**+ ```python+ async def _broadcast_node_status(self, node_id: str, status: str, data: dict):+ """Broadcast node status via WebSocket."""+ from services.status_broadcaster import get_status_broadcaster+ broadcaster = get_status_broadcaster()+ await broadcaster.update_node_status(node_id, status, data)+ ```++#### Files to Modify++| File | Changes |+|------|---------|+| `server/services/workflow.py` | Add WorkflowExecutor integration, adapter function |+| `server/routers/websocket.py` | Update handlers to use new executor |+| `server/core/container.py` | Add ExecutionCache to DI container |++---++### Phase 7: Retry Policies + Dead Letter Queue [COMPLETED]++**Goal**: Implement robust error handling with automatic retries and failed execution quarantine.++**Status**: COMPLETED - Full retry policies with exponential backoff and DLQ for failed executions.++**Implementation**:++#### RetryPolicy Dataclass (`server/services/execution/models.py`)+```python+@dataclass+class RetryPolicy:+ """Retry configuration for node execution."""+ max_attempts: int = 3+ initial_delay: float = 1.0 # seconds+ max_delay: float = 60.0 # seconds+ backoff_multiplier: float = 2.0+ retry_on_timeout: bool = True+ retry_on_connection_error: bool = True+ retry_on_server_error: bool = True # 5xx errors++ def calculate_delay(self, attempt: int) -> float:+ """Exponential backoff: min(initial * multiplier^attempt, max_delay)"""++ def should_retry(self, error: str, attempt: int) -> bool:+ """Check if error is retryable based on type and attempt count"""+```++#### Default Retry Policies+```python+DEFAULT_RETRY_POLICIES = {+ "httpRequest": RetryPolicy(max_attempts=3, initial_delay=2.0),+ "webhookTrigger": RetryPolicy(max_attempts=1), # Don't retry triggers+ "whatsappReceive": RetryPolicy(max_attempts=1), # Don't retry triggers+ "aiAgent": RetryPolicy(max_attempts=2, initial_delay=5.0, max_delay=30.0),+ "openaiChatModel": RetryPolicy(max_attempts=2, initial_delay=5.0),+ "anthropicChatModel": RetryPolicy(max_attempts=2, initial_delay=5.0),+ "googleChatModel": RetryPolicy(max_attempts=2, initial_delay=5.0),+}+```++#### DLQEntry Dataclass (`server/services/execution/models.py`)+```python+@dataclass+class DLQEntry:+ """Dead Letter Queue entry for failed node executions."""+ id: str+ execution_id: str+ workflow_id: str+ node_id: str+ node_type: str+ error: str+ inputs: Dict[str, Any]+ retry_count: int+ created_at: float+ last_error_at: float+```++#### ExecutionCache DLQ Methods (`server/services/execution/cache.py`)+- `add_to_dlq(entry)` - Add failed node to DLQ with multi-index storage+- `get_dlq_entry(entry_id)` - Get single DLQ entry+- `get_dlq_entries(workflow_id, node_type, limit)` - Query entries with filtering+- `remove_from_dlq(entry_id)` - Remove after successful replay+- `update_dlq_entry(entry_id, retry_count, error)` - Update after retry attempt+- `get_dlq_stats()` - Get DLQ statistics (total, by node type, by workflow)+- `purge_dlq(workflow_id, node_type, older_than)` - Bulk remove entries++#### WorkflowExecutor Retry Integration (`server/services/execution/executor.py`)+- `_execute_node_with_retry()` - Wraps node execution with retry loop+- `_add_to_dlq()` - Adds failed nodes to DLQ after exhausted retries+- `replay_dlq_entry()` - Re-execute failed node from DLQ+- `_execute_parallel_nodes()` - Updated to use retry wrapper+- `_execute_single_node()` - Updated to use retry wrapper++#### WebSocket API Handlers (`server/routers/websocket.py`)+| Handler | Description |+|---------|-------------|+| `get_dlq_entries` | Get DLQ entries with filtering |+| `get_dlq_entry` | Get single entry by ID |+| `get_dlq_stats` | Get DLQ statistics |+| `replay_dlq_entry` | Re-execute failed node |+| `remove_dlq_entry` | Remove entry without replay |+| `purge_dlq` | Bulk remove entries |++#### Redis Key Schema (DLQ)+```+dlq:entries:{id} -> HASH (entry data)+dlq:workflow:{workflow_id} -> LIST (entry IDs for workflow)+dlq:node_type:{node_type} -> LIST (entry IDs by type)+dlq:all -> SET (all entry IDs)+```++#### Files Modified+| File | Changes |+|------|---------|+| `server/services/execution/models.py` | Added RetryPolicy, DLQEntry, DEFAULT_RETRY_POLICIES |+| `server/services/execution/cache.py` | Added DLQ methods (add, get, remove, update, purge, stats) |+| `server/services/execution/executor.py` | Added retry wrapper, DLQ integration, replay method |+| `server/services/execution/__init__.py` | Export new classes and functions |+| `server/routers/websocket.py` | Added 6 DLQ WebSocket handlers |++---++### Phase 8: Temporal Distributed Execution [COMPLETED]++**Goal**: Enable horizontal scaling via Temporal for production deployments.++**Status**: COMPLETED - Optional Temporal integration with per-node activities.++**Implementation**:++#### Architecture+When `TEMPORAL_ENABLED=true`, each workflow node executes as an independent Temporal activity:+- Per-node retry policy (3 attempts with exponential backoff)+- Per-node timeout (10 min default)+- Horizontal scaling across worker pool+- FIRST_COMPLETED pattern for dependency resolution++#### Key Components+| File | Purpose |+|------|---------|+| `server/services/temporal/workflow.py` | MachinaWorkflow orchestrator (pure orchestration, no business logic) |+| `server/services/temporal/activities.py` | NodeExecutionActivities class with connection pooling |+| `server/services/temporal/executor.py` | TemporalExecutor interface matching WorkflowExecutor |+| `server/services/temporal/worker.py` | TemporalWorkerManager + `run_standalone_worker()` |+| `server/services/temporal/client.py` | Client wrapper with runtime heartbeat disabled |++#### Configuration+```env+TEMPORAL_ENABLED=true+TEMPORAL_SERVER_ADDRESS=localhost:7233+TEMPORAL_NAMESPACE=default+TEMPORAL_TASK_QUEUE=machina-tasks+```++#### Execution Routing (`workflow.py`)+1. If `TEMPORAL_ENABLED=true` and Temporal configured -> `_execute_temporal()`+2. Else if Redis available -> `_execute_parallel()` (local parallel)+3. Else -> `_execute_sequential()` (fallback)++---++### Phase 9: Expanded Node Types [COMPLETED]++**Goal**: Comprehensive workflow automation coverage.++**Status**: COMPLETED - 58 total nodes across 14 categories.++**Implementation**:++#### Node Categories (58 total)+| Category | Count | Nodes |+|----------|-------|-------|+| AI Chat Models | 6 | openai, anthropic, gemini, openrouter, groq, cerebras |+| AI Agents & Memory | 3 | aiAgent, chatAgent, simpleMemory |+| AI Skills | 9 | claude, whatsapp, memory, maps, http, scheduler, android, code, custom |+| AI Tools | 4 | calculator, currentTime, webSearch, androidTool |+| Android Services | 16 | battery, network, system, location, apps(2), automation(6), sensors(2), media(2) |+| Android Device | 1 | androidDeviceSetup |+| WhatsApp | 4 | send, connect, receive, chatHistory |+| Location/Maps | 3 | createMap, addLocations, showNearbyPlaces |+| Utility | 5 | httpRequest, webhookTrigger, webhookResponse, chatTrigger, console |+| Code | 2 | pythonExecutor, javascriptExecutor |+| Scheduler | 2 | timer, cronScheduler |+| Chat | 2 | chatSend, chatHistory |+| Workflow | 1 | start |++#### WebSocket Handlers (51 total)+- Execution: 4 handlers+- Parameters: 7 handlers+- Status & Variables: 6 handlers+- AI Services: 6 handlers+- Android Services: 7 handlers+- Deployment: 4 handlers+- Triggers & Events: 3 handlers+- Workflow Management: 4 handlers+- Dead Letter Queue: 5 handlers+- WhatsApp: 8 handlers+- Maps: 1 handler+- Skills & Chat: 6 handlers++---++## Pending Features++### Future Enhancements++#### Sub-Workflow Support+- Execute workflow as a node within another workflow+- Parent-child execution tracking+- Output propagation from sub-workflow++#### Workflow Versioning+- Track workflow definition versions+- Support running specific versions+- Migration between versions++#### Metrics & Observability+- Execution duration histograms+- Success/failure rates+- Node-level performance tracking+- Integration with Prometheus/Grafana++---++## Architecture Diagram++```+┌─────────────────────────────────────────────────────────────────┐+│ WebSocket Router │+│ (routers/websocket.py) │+└─────────────────────────────┬───────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────┐+│ WorkflowService │+│ (services/workflow.py) │+│ ┌─────────────────────────────────────────────────────────┐ │+│ │ Node Handler Registry │ │+│ │ - AI handlers, Android handlers, HTTP handlers, etc. │ │+│ └─────────────────────────────────────────────────────────┘ │+└─────────────────────────────┬───────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────┐+│ WorkflowExecutor │+│ (services/execution/executor.py) │+│ │+│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │+│ │ workflow_ │ │ execute_ │ │ _compute_execution_ │ │+│ │ decide() │ │ parallel_ │ │ layers() │ │+│ │ │ │ nodes() │ │ │ │+│ │ Conductor │ │ │ │ DAG topological │ │+│ │ pattern │ │ asyncio. │ │ sort │ │+│ │ │ │ gather() │ │ │ │+│ └──────────────┘ └──────────────┘ └──────────────────────┘ │+└─────────────────────────────┬───────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────┐+│ ExecutionCache │+│ (services/execution/cache.py) │+│ │+│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │+│ │ Result │ │ Distributed │ │ Event History │ │+│ │ Caching │ │ Locks │ │ (Redis Streams) │ │+│ │ (Prefect) │ │ (Conductor) │ │ │ │+│ └──────────────┘ └──────────────┘ └──────────────────────┘ │+└─────────────────────────────┬───────────────────────────────────┘+ │+ ▼+┌─────────────────────────────────────────────────────────────────┐+│ Redis │+│ │+│ execution:{id}:state execution:{id}:events │+│ result:{exec}:{node} lock:execution:{id} │+│ heartbeat:{exec}:{node} executions:active │+│ dlq:entries:{id} dlq:all │+└─────────────────────────────────────────────────────────────────┘+```++---++## Testing Checklist++### Unit Tests+- [ ] `test_models.py` - TaskStatus, WorkflowStatus, ExecutionContext serialization+- [ ] `test_cache.py` - Result caching, distributed locks, event storage+- [ ] `test_executor.py` - Parallel execution, decide pattern, layer computation+- [ ] `test_recovery.py` - Sweeper detection, startup scan+- [ ] `test_conditions.py` - Edge condition evaluation++### Integration Tests+- [ ] Execute simple linear workflow+- [ ] Execute workflow with parallel branches+- [ ] Execute workflow with conditional edges+- [ ] Crash recovery simulation+- [ ] Concurrent workflow execution++### Performance Tests+- [ ] Parallel execution speedup vs sequential+- [ ] Redis operation latency+- [ ] Large workflow (100+ nodes) execution time++---++## References++- **Netflix Conductor**: [Architecture](https://conductor-oss.github.io/conductor/devguide/architecture/index.html)+- **Prefect 3.0**: [Task Caching](https://docs.prefect.io/v3/develop/task-caching)+- **Redis Streams**: [Consumer Groups](https://redis.io/docs/data-types/streams/)+- **Plan Document**: `C:\Users\Tgroh\.claude\plans\buzzing-gathering-pie.md`diff --git a/docs-internal/SCRIPTS.md b/docs-internal/SCRIPTS.mdnew file mode 100644index 00000000..1275127a--- /dev/null+++ b/docs-internal/SCRIPTS.md@@ -0,0 +1,275 @@+# MachinaOS Scripts Reference++## Quick Start++```bash+npm install -g machinaos+machinaos start+```++Open http://localhost:3000++## CLI Commands++### Available Commands++| Command | Description |+|---------|-------------|+| `machinaos start` | Start the development server (checks dependencies first) |+| `machinaos stop` | Stop all running services |+| `machinaos build` | Build the project for production (checks dependencies first) |+| `machinaos clean` | Clean build artifacts and node_modules |+| `machinaos docker:up` | Start with Docker Compose (detached) |+| `machinaos docker:down` | Stop Docker Compose services |+| `machinaos docker:build` | Build Docker images |+| `machinaos docker:logs` | View Docker logs (follows) |+| `machinaos help` | Show help message |++### Dependency Checks++The `start` and `build` commands verify these dependencies before running:+- Node.js 18++- Python 3.11++- uv (Python package manager)++---++## npm Scripts++Run with `npm run <script>` from the project root.++### Core Scripts++| Script | Command | Description |+|--------|---------|-------------|+| `start` | `node scripts/start.js` | Start all services concurrently |+| `start:temporal` | `cross-env TEMPORAL_ENABLED=true node scripts/start.js` | Start with Temporal worker |+| `stop` | `node scripts/stop.js` | Stop all running services |+| `build` | `node scripts/build.js` | Build entire project |+| `clean` | `node scripts/clean.js` | Remove build artifacts |++### Service Scripts++| Script | Command | Description |+|--------|---------|-------------|+| `client:start` | `cd client && npm run start` | Start React frontend (Vite) |+| `python:start` | `cd server && uv run uvicorn main:app ...` | Start Python backend |+| `whatsapp:api` | `cd server/whatsapp-rpc && npm run api` | Start WhatsApp API server |+| `whatsapp:start` | `cd server/whatsapp-rpc && npm start` | Start WhatsApp connection |+| `whatsapp:stop` | `cd server/whatsapp-rpc && npm stop` | Stop WhatsApp service |+| `whatsapp:status` | `cd server/whatsapp-rpc && npm run status` | Check WhatsApp status |+| `whatsapp:build` | `cd server/whatsapp-rpc && npm run build` | Build WhatsApp Go binary |+| `temporal:worker` | `cd server && uv run python -m services.temporal.worker` | Start Temporal worker |++### Docker Scripts (Development)++| Script | Command | Description |+|--------|---------|-------------|+| `docker:up` | `node scripts/docker.js up` | Start dev stack (detached) |+| `docker:down` | `node scripts/docker.js down` | Stop dev stack |+| `docker:build` | `node scripts/docker.js build` | Build dev images |+| `docker:logs` | `node scripts/docker.js logs` | View logs (follows) |+| `docker:restart` | `node scripts/docker.js restart` | Restart dev stack |++### Docker Scripts (Production)++| Script | Command | Description |+|--------|---------|-------------|+| `docker:prod:up` | `docker-compose -f docker-compose.prod.yml up -d` | Start production stack |+| `docker:prod:down` | `docker-compose -f docker-compose.prod.yml down` | Stop production stack |+| `docker:prod:build` | `docker-compose -f docker-compose.prod.yml build` | Build production images |+| `docker:prod:logs` | `docker-compose -f docker-compose.prod.yml logs -f` | View production logs |++### Deployment Scripts++| Script | Command | Description |+|--------|---------|-------------|+| `deploy` | `bash deploy.sh` | Deploy to production server |+| `deploy:gcp` | `bash deploy.sh` | Deploy to GCP |++---++## Script Details++### start.js++**Location:** `scripts/start.js`++Cross-platform start script that runs all services concurrently.++**What it does:**+1. Creates `.env` from `.env.template` if missing+2. Installs client dependencies if `client/node_modules` doesn't exist+3. Frees configured ports by killing existing processes+4. Starts services via `concurrently`:+ - `npm:client:start` - React frontend (Vite dev server)+ - `npm:python:start` - Python backend (uvicorn with hot reload)+ - `npm:whatsapp:api` - WhatsApp API server+ - `npm:temporal:worker` - Temporal worker (if `TEMPORAL_ENABLED=true`)++**Ports (configurable in .env):**+- `VITE_CLIENT_PORT` - Frontend (default: 3000)+- `PYTHON_BACKEND_PORT` - Backend (default: 3010)+- `WHATSAPP_RPC_PORT` - WhatsApp (default: 9400)++**Platform support:** Windows, macOS, Linux, WSL, Git Bash++---++### build.js++**Location:** `scripts/build.js`++Production build script that compiles all components.++**Build steps:**+1. `[0/6]` Create `.env` from template if missing+2. `[1/6]` Install root dependencies (`npm ci` or `npm install`)+3. `[2/6]` Install client dependencies+4. `[3/6]` Build client (`vite build`)+5. `[4/6]` Install Python dependencies (`uv venv && uv sync`)+6. `[5/6]` Install WhatsApp dependencies+7. `[6/6]` Build WhatsApp Go binary++**Dependencies checked:**+- Node.js, npm, Python, uv, Go++---++### stop.js++**Location:** `scripts/stop.js`++Cross-platform stop script that kills all MachinaOS processes.++**What it does:**+1. Reads port configuration from `.env`+2. Finds processes on each port+3. Kills processes including child processes+4. Verifies processes are stopped+5. Retries stubborn processes with force kill+6. Kills Temporal workers if enabled++**Platform-specific commands:**+| Platform | Find processes | Kill process |+|----------|----------------|--------------|+| Unix/Mac | `lsof -ti:PORT` | `kill -15`, then `kill -9` |+| Linux | `ss -tlnp` (fallback) | `kill -15`, then `kill -9` |+| Windows | `netstat -ano \| findstr` | `taskkill /PID` |++---++### clean.js++**Location:** `scripts/clean.js`++Removes build artifacts and dependencies.++**Directories removed:**+- `node_modules/` - Root dependencies+- `client/node_modules/` - Frontend dependencies+- `client/dist/` - Built frontend+- `client/.vite/` - Vite cache+- `server/.venv/` - Python virtual environment+- `server/data/` - Server data+- `server/whatsapp-rpc/node_modules/` - WhatsApp dependencies+- `server/whatsapp-rpc/bin/` - WhatsApp Go binary+- `server/whatsapp-rpc/data/` - WhatsApp data++---++### docker.js++**Location:** `scripts/docker.js`++Docker Compose wrapper with automatic Redis profile detection.++**Usage:**+```bash+node scripts/docker.js <command> [args...]+```++**Commands:** `up`, `down`, `build`, `logs`, `restart`++**Features:**+- Creates `.env` from template if missing+- Reads `REDIS_ENABLED` from `.env`+- Adds `--profile redis` flag when Redis is enabled+- `up` runs detached (`-d`) by default+- `logs` follows (`-f`) by default++**Example output:**+```+[Docker] Redis profile enabled (REDIS_ENABLED=true in .env)+[Docker] Running: docker-compose --profile redis up -d+```++---++## Environment Variables++Key variables in `.env` (see `.env.template` for full list):++### Ports+| Variable | Default | Description |+|----------|---------|-------------|+| `VITE_CLIENT_PORT` | 3000 | Frontend port |+| `PYTHON_BACKEND_PORT` | 3010 | Backend port |+| `WHATSAPP_RPC_PORT` | 9400 | WhatsApp API port |++### Features+| Variable | Default | Description |+|----------|---------|-------------|+| `TEMPORAL_ENABLED` | false | Enable Temporal worker |+| `REDIS_ENABLED` | false | Enable Redis cache (uses SQLite if false) |++### Redis (when enabled)+| Variable | Default | Description |+|----------|---------|-------------|+| `REDIS_URL` | redis://redis:6379 | Redis connection URL |+| `REDIS_PORT` | 6379 | Redis port |++---++## Required Dependencies++### For `start` and `build` commands++| Dependency | Version | Install |+|------------|---------|---------|+| Node.js | 18+ | https://nodejs.org/ |+| Python | 3.11+ | https://python.org/ |+| uv | latest | `curl -LsSf https://astral.sh/uv/install.sh \| sh` |+| Go | latest | https://go.dev/dl/ (for WhatsApp binary) |++### For Docker commands++| Dependency | Install |+|------------|---------|+| Docker | https://docs.docker.com/get-docker/ |+| Docker Compose | Included with Docker Desktop |++---++## Quick Reference++```bash+# Development+machinaos start # Start all services+machinaos stop # Stop all services++# Build+machinaos build # Build for production+machinaos clean # Clean everything++# Docker (development)+machinaos docker:up # Start containers+machinaos docker:down # Stop containers+machinaos docker:logs # View logs+machinaos docker:build # Rebuild images++# Docker (production)+npm run docker:prod:up # Start production+npm run docker:prod:down # Stop production+npm run deploy # Deploy to server+```diff --git a/docs-internal/SETUP.md b/docs-internal/SETUP.mdnew file mode 100644index 00000000..a5510796--- /dev/null+++ b/docs-internal/SETUP.md@@ -0,0 +1,175 @@+# MachinaOs - Development Setup++## Project Structure++```+MachinaOs/+├── client/ # React frontend (port 3000)+│ ├── src/+│ └── package.json+├── server/ # Python FastAPI backend (port 3010)+│ ├── services/ # Business logic (workflow, AI, etc.)+│ ├── routers/ # API endpoints+│ ├── core/ # DI container, database, cache+│ ├── models/ # SQLModel definitions+│ ├── whatsapp-rpc/ # Go WhatsApp service (port 9400)+│ └── requirements.txt+├── scripts/ # Build and utility scripts+└── package.json # Workspace root with npm scripts+```++## Quick Start++```bash+npm install -g machinaos+machinaos start+```++Open http://localhost:3000++### Local Development (from source)++**Prerequisites:** Node.js 18+, Python 3.11+, uv, Go 1.21+++```bash+git clone https://github.com/trohitg/MachinaOS.git+cd MachinaOS+npm run build+npm run start+```++### Docker++```bash+git clone https://github.com/trohitg/MachinaOS.git+cd MachinaOS+npm run docker:up+```++Services will be available at:+- **Frontend**: http://localhost:3000+- **Backend API**: http://localhost:3010+- **WhatsApp Service**: http://localhost:9400++## Services Overview++### Frontend (React - Port 3000)+- React 19 with TypeScript+- React Flow for workflow canvas+- Zustand for state management+- WebSocket connection to backend++### Backend (Python FastAPI - Port 3010)+- FastAPI with async support+- SQLAlchemy + SQLite database+- LangChain for AI integrations+- WebSocket for real-time updates++**Key Endpoints:**+- `GET /health` - Health check+- `WS /ws/status` - Real-time status WebSocket+- `ANY /webhook/{path}` - Dynamic webhook endpoints+- `POST /api/ai/*` - AI model execution+- `POST /api/android/*` - Android device operations++### WhatsApp Service (Go - Default Port 9400)+- Go service using whatsmeow library+- QR code authentication (base64 PNG in memory, no file I/O)+- Message send/receive via JSON-RPC+- Port configurable via `--port` flag, `PORT` or `WHATSAPP_RPC_PORT` env vars++### Database (SQLite)+- **workflows** - Workflow definitions+- **node_parameters** - Node parameter storage+- **conversation_messages** - AI conversation history+- **cache_entries** - Execution cache (when Redis disabled)+- **users** - Authentication (single/multi-user modes)++## Environment Configuration++Copy the example environment file:++```bash+cp .env.example .env+```++### Key Settings++| Variable | Default | Description |+|----------|---------|-------------|+| `VITE_CLIENT_PORT` | 3000 | Frontend port |+| `PYTHON_BACKEND_PORT` | 3010 | Backend API port |+| `AUTH_MODE` | single | Authentication mode (single/multi) |+| `REDIS_ENABLED` | false | Enable Redis cache (production) |+| `DEBUG` | true | Debug mode |++### API Keys (Optional)+Add these to `.env` or configure via the Credentials UI:+- `OPENAI_API_KEY`+- `ANTHROPIC_API_KEY`+- `GOOGLE_AI_API_KEY`+- `GOOGLE_MAPS_API_KEY`++## Docker Commands++| Command | Description |+|---------|-------------|+| `npm run docker:up` | Start containers |+| `npm run docker:down` | Stop containers |+| `npm run docker:logs` | View logs |+| `npm run docker:build` | Rebuild images |++**Redis (optional):** Set `REDIS_ENABLED=true` in `.env`++## Local Commands++| Command | Description |+|---------|-------------|+| `npm run start` | Start all services |+| `npm run stop` | Stop all services |+| `npm run build` | Install dependencies |+| `npm run dev` | Start development server |++## Troubleshooting++### Port already in use+Change the port in `.env`:+```bash+VITE_CLIENT_PORT=3001+PYTHON_BACKEND_PORT=3011+```++### Python dependencies fail+Create a virtual environment:+```bash+cd server+python -m venv venv+source venv/bin/activate # or venv\Scripts\activate on Windows+pip install -r requirements.txt+```++### Database issues+SQLite database is created automatically at `server/workflow.db`.+Delete the database file to reset all data.++### Docker containers won't start+Check logs and rebuild:+```bash+docker-compose logs -f+docker-compose down+docker-compose up --build -d+```++## Development Workflow++1. **Make changes** in client/ or server/+2. **Hot reload** automatically updates running services+3. **WebSocket** provides real-time status updates+4. **Database** persists data between restarts++## Architecture Notes++- **WebSocket-First**: 25 message handlers replace most REST APIs+- **n8n-inspired**: Node definitions follow n8n INodeProperties pattern+- **Cache Fallback**: Redis (production) → SQLite (dev) → Memory+- **Event-Driven**: Trigger nodes use asyncio.Future for event waitingdiff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.mdnew file mode 100644index 00000000..6cf6d6ce--- /dev/null+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md@@ -0,0 +1,330 @@+# Temporal Distributed Node Execution Architecture++## Overview++Each workflow node executes as an **independent Temporal activity** with its own isolated context, enabling horizontal scaling across distributed workers.++## System Architecture++```+ TEMPORAL SERVER (port 7233)+ |+ Task Queue: machina-tasks+ +---------------------------------------------------------------++ | Workflow: MachinaWorkflow (orchestrator only) |+ | - Parses graph structure from React Flow |+ | - Filters config nodes (tools, memory, services) |+ | - Schedules node activities (FIRST_COMPLETED pattern) |+ | - Collects results and routes outputs to dependent nodes |+ +---------------------------------------------------------------++ |+ Activity Queue (distributed to workers)+ +--------+ +--------+ +--------+ +--------++ | Node A | | Node B | | Node C | | Node D |+ +--------+ +--------+ +--------+ +--------++ | | | |+ v v v v+ +----------+ +----------+ +----------+ +----------++ | Worker 1 | | Worker 2 | | Worker 3 | | Worker N |+ | aiAgent | | timer | | console | | http |+ +----------+ +----------+ +----------+ +----------++ | | | |+ +----------+-----------+-----------++ |+ v WebSocket+ +----------------++ | MachinaOs |+ | /ws/internal |+ +----------------++```++## Key Architecture Principles++### 1. Node = Independent Activity++Each node runs as a separate Temporal activity with:+- **Own context** - No shared mutable state between nodes+- **Own retry policy** - Failed nodes retry independently (up to 3 attempts)+- **Own timeout** - Long AI nodes don't block short nodes (10 min default)+- **Own worker** - Can execute on any available worker in the cluster++### 2. Workflow = Pure Orchestrator++The workflow ONLY orchestrates:+- Parses the graph structure from React Flow nodes/edges+- Filters out config nodes (tools, memory, model configs)+- Determines execution order based on dependencies+- Schedules activities using FIRST_COMPLETED pattern+- Collects results and routes outputs to dependent nodes++**NO business logic in workflow** - all execution happens in activities.++### 3. Context Passing (Immutable)++Each node receives an immutable context snapshot:++```python+context = {+ "node_id": "aiAgent-123",+ "node_type": "aiAgent",+ "node_data": {+ "model": "gpt-4",+ "prompt": "{{chattrigger.message}}",+ "systemMessage": "You are a helpful assistant"+ },+ "inputs": { # Outputs from upstream nodes+ "chatTrigger-456": {"message": "Hello", "timestamp": "..."},+ },+ "workflow_id": "workflow-789",+ "session_id": "session-xyz",+ "nodes": [...], # Full list for tool/memory detection+ "edges": [...], # Full list for tool/memory detection+}+```++## Execution Flow++### 1. Workflow Receives Request++```+MachinaOs Server+ |+ v execute_workflow()+TemporalExecutor+ |+ v client.execute_workflow()+Temporal Server+ |+ v schedules workflow task+MachinaWorkflow.run()+```++### 2. Workflow Orchestrates Nodes (FIRST_COMPLETED Pattern)++```python+# 1. Filter config nodes+exec_nodes, exec_edges = self._filter_executable_graph(nodes, edges)++# 2. Build dependency graph+deps, node_map = self._build_dependency_maps(exec_nodes, exec_edges)++# 3. Handle pre-executed triggers+for node in exec_nodes:+ if node.get("_pre_executed"):+ completed.add(node["id"])++# 4. Continuous scheduling loop+while True:+ ready = self._find_ready_nodes(deps, completed, running, node_map)++ for node_id in ready:+ handle = workflow.start_activity(+ "execute_node_activity",+ args=[context],+ start_to_close_timeout=timedelta(minutes=10),+ )+ running[node_id] = handle++ if not running:+ break++ done_id, result = await self._wait_any_complete(running)+ completed.add(done_id)+ outputs[done_id] = result+```++### 3. Activity Executes Node via WebSocket++```python+async def execute_node_activity(self, context: Dict) -> Dict:+ node_id = context["node_id"]+ node_type = context["node_type"]++ # Execute via WebSocket to MachinaOs+ async with self.session.ws_connect(self.ws_url) as ws:+ await ws.send_json({+ "type": "execute_node",+ "node_id": node_id,+ "node_type": node_type,+ "parameters": context["node_data"],+ ...+ })++ # Wait for response+ async for msg in ws:+ if msg.data["request_id"] == request_id:+ return msg.data++ return result+```++## Connection Pooling++Activities use a shared aiohttp.ClientSession for connection pooling:++```python+class NodeExecutionActivities:+ def __init__(self, session: aiohttp.ClientSession):+ self.session = session # Shared session with connection pool++ @activity.defn+ async def execute_node_activity(self, context: Dict) -> Dict:+ # Each activity gets its own WebSocket from the pool+ async with self.session.ws_connect(self.ws_url) as ws:+ await ws.send_json(message)+ async for msg in ws:+ return msg.data++# Session configuration+connector = aiohttp.TCPConnector(+ limit=100, # Max connections in pool+ limit_per_host=100, # Max connections per host+ enable_cleanup_closed=True,+)+session = aiohttp.ClientSession(connector=connector)+```++Benefits:+- **No race conditions** - Each activity has exclusive WebSocket+- **Connection reuse** - TCP connections are pooled and reused+- **Configurable limits** - Control max concurrent connections++## Scaling Patterns++### Horizontal Worker Scaling++```+ Temporal Server+ |+ +---------------+---------------++ v v v+ +---------+ +---------+ +---------++ |Worker 1 | |Worker 2 | |Worker 3 |+ | Node A | | Node B | | Node C |+ | Node D | | Node E | | Node F |+ +---------+ +---------+ +---------+++Add more workers = handle more concurrent nodes+```++### Specialized Worker Pools (Future)++```+Queue: machina-tasks-cpu Queue: machina-tasks-gpu+ | |+ +----+----+ +----+----++ v v v v++-------+ +-------+ +-------+ +-------++|CPU | |CPU | |GPU | |GPU |+|Worker | |Worker | |Worker | |Worker |+|timer | |http | |aiAgent| |aiAgent|++-------+ +-------+ +-------+ +-------+++Route AI nodes to GPU workers, light nodes to CPU workers+```++## Config Node Filtering++Certain nodes provide configuration rather than executing:++```python+# Config handles - nodes connecting via these are filtered out+CONFIG_HANDLES = {"input-tools", "input-memory", "input-model"}++# Android service types (connect to androidTool)+ANDROID_SERVICE_TYPES = {+ "batteryMonitor", "locationService", "deviceState",+ "systemInfo", "appList", "appLauncher",+}+```++Config nodes are:+- Filtered from the execution graph+- Their configuration is passed to target nodes via node_data+- Not scheduled as activities++## Retry & Fault Tolerance++| Scenario | Behavior |+|----------|----------|+| Node WebSocket call fails | Temporal retries (up to 3 attempts with backoff) |+| Worker crashes mid-execution | Temporal reschedules on another worker |+| Node times out (10 min) | Temporal retries with backoff |+| All retries exhausted | Workflow receives failure, stops execution |++## File Structure++```+server/services/temporal/+├── __init__.py # Exports TemporalExecutor, TemporalClientWrapper+├── activities.py # NodeExecutionActivities class+│ ├── execute_node_activity() # Main activity method+│ └── _execute_via_websocket() # WebSocket execution+├── workflow.py # MachinaWorkflow class+│ ├── run() # Main orchestrator+│ ├── _filter_executable_graph() # Config node filtering+│ ├── _build_dependency_maps() # Graph analysis+│ ├── _find_ready_nodes() # Dependency resolution+│ └── _wait_any_complete() # FIRST_COMPLETED wait+├── worker.py # TemporalWorkerManager+│ ├── start() # Start embedded worker+│ ├── stop() # Cleanup+│ └── run_standalone_worker() # For horizontal scaling+├── executor.py # TemporalExecutor entry point+└── client.py # TemporalClientWrapper (runtime heartbeat disabled)+```++## Implementation Notes++### Worker Registration (Critical)++For class-based activities, pass the **bound method**:++```python+# WRONG - causes "Activity <unknown> missing attributes"+activities=[self._activities]++# CORRECT - pass the bound method+activities=[self._activities.execute_node_activity]+```++### Activity Invocation (Critical)++When using class-based activities, invoke by **string name**:++```python+# WRONG - works only with standalone function activities+workflow.start_activity(execute_node_activity, args=[context])++# CORRECT - use string name for class-based activities+workflow.start_activity("execute_node_activity", args=[context])+```++### Runtime Configuration++Worker heartbeating is disabled to avoid warnings on older Temporal server versions:++```python+runtime = Runtime(+ telemetry=TelemetryConfig(),+ worker_heartbeat_interval=None, # Disable runtime heartbeating+)+client = await Client.connect(server_address, namespace=namespace, runtime=runtime)+```++## Debugging++```bash+# Check task queue pollers+curl http://localhost:8233/api/v1/namespaces/default/task-queues/machina-tasks++# List recent workflows+curl "http://localhost:8233/api/v1/namespaces/default/workflows"++# Get workflow history+curl "http://localhost:8233/api/v1/namespaces/default/workflows/{id}/history"++# Temporal UI+open http://localhost:8233+```diff --git a/docs-internal/agent_architecture.md b/docs-internal/agent_architecture.mdnew file mode 100644index 00000000..5262e17d--- /dev/null+++ b/docs-internal/agent_architecture.md@@ -0,0 +1,714 @@+# AI Agent Architecture: Skill Injection & Tool Execution++Detailed architecture reference for how AI Agent (`aiAgent`) and Chat Agent (`chatAgent`) discover skills and tools from connected nodes, inject them into the LLM prompt, and execute tools via LangGraph.++> **Related Documentation:**+> - [AI Tool Node Guide](./ai_tool_node_creation.md) - Creating dedicated AI Agent tool nodes+> - [Dual-Purpose Tool Guide](./dual_purpose_tool_node_creation.md) - Nodes that work as workflow nodes AND AI tools+> - [Specialized Agent Guide](./specialized_agent_node_creation.md) - Creating specialized AI agents+> - [CLAUDE.md](../CLAUDE.md) - Project overview and full node inventory++## Table of Contents++1. [End-to-End Data Flow](#end-to-end-data-flow)+2. [LangGraph Agent Graph](#langgraph-agent-graph)+3. [Skill Injection Pipeline](#skill-injection-pipeline)+4. [Tool Building Pipeline](#tool-building-pipeline)+5. [Tool Execution Flow](#tool-execution-flow)+6. [Memory Integration](#memory-integration)+7. [execute_agent vs execute_chat_agent](#execute_agent-vs-execute_chat_agent)++---++## End-to-End Data Flow++```+User clicks "Run" on AI Agent+ |+ v+useExecution.executeNode() client/src/hooks/useExecution.ts+ Sends ALL workflow nodes + edges+ |+ v+WebSocket: handle_execute_node() server/routers/websocket.py+ Passes nodes[], edges[] to WorkflowService+ |+ v+WorkflowService.execute_node() server/services/workflow.py+ Builds context = {nodes, edges, session_id, workflow_id}+ Calls NodeExecutor.execute()+ |+ v+NodeExecutor._dispatch() server/services/node_executor.py+ Handler registry lookup via functools.partial+ Dispatches to handle_ai_agent() or handle_chat_agent()+ |+ v+_collect_agent_connections() server/services/handlers/ai.py+ Scans edges where target == node_id+ Groups by targetHandle into 4 buckets:+ input-memory -> memory_data+ input-skill -> skill_data[]+ input-tools -> tool_data[]+ input-main -> input_data+ |+ v+AIService.execute_agent() server/services/ai.py+ 1. Inject skills into system message+ 2. Build LangChain StructuredTools from tool_data+ 3. Construct LangGraph with agent + tools nodes+ 4. Run graph with initial messages+ 5. Save memory, return result+ |+ v+LangGraph StateGraph execution+ agent node <-> tool node loop+ LLM decides when to call tools+ |+ v+Result broadcast via WebSocket+```++### Key Files++| File | Responsibility |+|------|---------------|+| `client/src/hooks/useExecution.ts` | Frontend execution trigger, sends nodes + edges |+| `server/routers/websocket.py` | WebSocket handler `handle_execute_node()` |+| `server/services/workflow.py` | Facade, builds context, delegates to NodeExecutor |+| `server/services/node_executor.py` | Handler registry, dispatches via `functools.partial` |+| `server/services/handlers/ai.py` | `_collect_agent_connections()`, `handle_ai_agent()`, `handle_chat_agent()` |+| `server/services/ai.py` | `AIService` -- LangGraph construction, skill injection, tool building |+| `server/services/handlers/tools.py` | `execute_tool()` -- dispatch router for all tool types |+| `server/services/skill_loader.py` | `SkillLoader` -- filesystem/DB skill discovery and loading |++---++## LangGraph Agent Graph++### AgentState Schema++Defined in `server/services/ai.py:386-402`:++```python+class AgentState(TypedDict):+ messages: Annotated[Sequence[BaseMessage], operator.add] # Accumulates via operator.add+ tool_outputs: Dict[str, Any]+ pending_tool_calls: List[Dict[str, Any]]+ iteration: int+ max_iterations: int+ should_continue: bool+ thinking_content: Optional[str] # Accumulated across iterations+```++Key design: `Annotated[..., operator.add]` on messages means new messages are **appended** to state, not replaced.++### Graph Topology++Built by `build_agent_graph()` in `server/services/ai.py:646-709`:++```+WITH TOOLS:+ START --> agent --> should_continue() --> tools --> agent --> ... --> END+ |+ +--> END (no tool calls or max iterations)++WITHOUT TOOLS:+ START --> agent --> END+```++```python+def build_agent_graph(chat_model, tools=None, tool_executor=None):+ graph = StateGraph(AgentState)++ # Bind tools to model (makes LLM aware of tool schemas)+ model_with_tools = chat_model+ if tools:+ model_with_tools = chat_model.bind_tools(tools)++ # Add nodes+ graph.add_node("agent", create_agent_node(model_with_tools))+ graph.set_entry_point("agent")++ if tools and tool_executor:+ graph.add_node("tools", create_tool_node(tool_executor))+ graph.add_conditional_edges("agent", should_continue, {"tools": "tools", "end": END})+ graph.add_edge("tools", "agent") # Loop back++ return graph.compile()+```++### Agent Node++`create_agent_node()` in `server/services/ai.py:509-584`:++1. Receives `state["messages"]` (accumulated conversation)+2. Filters empty messages (prevents API errors for Gemini/Claude)+3. Invokes LLM: `response = chat_model.invoke(filtered_messages)`+4. Extracts thinking content (Claude content_blocks, Gemini metadata, Groq additional_kwargs)+5. Checks `response.tool_calls` -- if present, sets `should_continue=True`+6. Returns updated state with `[response]` appended via `operator.add`++### Tool Node++`create_tool_node()` in `server/services/ai.py:587-631`:++```python+async def tool_node(state: AgentState) -> Dict[str, Any]:+ tool_messages = []+ for tool_call in state.get("pending_tool_calls", []):+ tool_name = tool_call.get("name")+ tool_args = tool_call.get("args", {})+ tool_id = tool_call.get("id", "")++ try:+ result = await tool_executor(tool_name, tool_args)+ except Exception as e:+ result = {"error": str(e)}++ tool_messages.append(ToolMessage(+ content=json.dumps(result, default=str),+ tool_call_id=tool_id,+ name=tool_name+ ))++ return {"messages": tool_messages, "pending_tool_calls": []}+```++- Iterates all pending tool calls from the LLM+- Calls the `tool_executor` callback (closure built in `execute_agent()`)+- Wraps results as `ToolMessage` objects with matching `tool_call_id`+- Errors are caught and returned as `{"error": ...}` -- the LLM sees the error and can retry or respond++### Routing Logic++`should_continue()` in `server/services/ai.py:634-643`:++```python+def should_continue(state: AgentState) -> str:+ if state.get("should_continue", False):+ if state.get("iteration", 0) < state.get("max_iterations", 10):+ return "tools"+ return "end"+```++Routes to `"tools"` when:+- LLM returned tool calls (`should_continue=True`)+- AND iteration count < `max_iterations` (default 10)++---++## Skill Injection Pipeline++### 1. Edge Scanning++In `_collect_agent_connections()` (`server/services/handlers/ai.py:14-49`), all edges targeting the agent node are scanned:++```python+for edge in edges:+ if edge.get('target') != node_id:+ continue++ target_handle = edge.get('targetHandle')+ source_node_id = edge.get('source')++ if target_handle == 'input-skill':+ # Collect skill data...+```++### 2. Regular Skill Nodes++For standard skill nodes (claudeSkill, whatsappSkill, etc.), a single entry is created:++```python+skill_entry = {+ 'node_id': source_node_id,+ 'node_type': skill_type, # e.g., 'whatsappSkill'+ 'skill_name': skill_params.get('skillName', skill_type),+ 'parameters': skill_params, # All node parameters from DB+ 'label': source_node.get('data', {}).get('label', skill_type)+}+skill_data.append(skill_entry)+```++### 3. Master Skill Expansion++When the connected skill is a `masterSkill`, its `skillsConfig` parameter is expanded into N individual entries:++```python+if skill_type == 'masterSkill':+ skills_config = skill_params.get('skillsConfig', {})+ # Structure: {'whatsapp-skill': {'enabled': True, 'instructions': '...'}, ...}++ for skill_key, skill_cfg in skills_config.items():+ if not skill_cfg.get('enabled', False):+ continue # Skip disabled skills++ # DB-first: use stored instructions+ instructions = skill_cfg.get('instructions', '')++ if not instructions:+ # Fallback: load from SKILL.md on disk+ skill = skill_loader.load_skill(skill_key)+ if skill:+ instructions = skill.instructions++ skill_data.append({+ 'node_id': f"{source_node_id}_{skill_key}", # Unique composite ID+ 'node_type': 'masterSkill',+ 'skill_name': skill_key,+ 'parameters': {'instructions': instructions, 'skillName': skill_key},+ 'label': skill_key+ })+```++One Master Skill node with 5 enabled skills produces 5 separate `skill_data` entries.++### 4. SkillLoader Architecture++Defined in `server/services/skill_loader.py:38+`:++```+SkillLoader+├── _skill_dirs: [server/skills/, .machina/skills/]+├── _registry: Dict[name -> SkillMetadata] # Metadata only (~100 tokens each)+├── _cache: Dict[name -> Skill] # Full content (lazy-loaded)+│+├── scan_skills() # rglob("SKILL.md") across all dirs, parses frontmatter+├── load_skill(name) # Loads full content: cache -> registry -> filesystem+├── get_registry_prompt() # Generates "## Available Skills" for system message+└── get_skill_instructions()# Shortcut for load_skill().instructions+```++**`scan_skills()`** (`skill_loader.py:63-88`):+- Iterates `_skill_dirs`, uses `rglob("SKILL.md")` for recursive discovery+- Parses YAML frontmatter for each file (`_parse_skill_metadata`)+- Populates `_registry` with `SkillMetadata` (name, description, allowed_tools, path)++**`load_skill(name)`** (`skill_loader.py:174-249`):+1. Check `_cache` -- return immediately if cached+2. Look up `_registry[name]` -- fail if not registered+3. Read `SKILL.md`, strip frontmatter, extract markdown body as `instructions`+4. Load optional `scripts/` and `references/` directories+5. Cache and return `Skill` dataclass++**SKILL.md frontmatter parsing** (`skill_loader.py:126-172`):+```yaml+---+name: http-skill # Lowercase with hyphens, validated by regex+description: Make HTTP requests... # Brief description for LLM visibility+allowed-tools: http-request # Space-delimited tool names+metadata:+ author: machina+ version: "2.0"+---+```++### 5. System Message Injection++In `execute_agent()` and `execute_chat_agent()` within `server/services/ai.py`:++```python+if skill_data:+ skill_loader = get_skill_loader()+ skill_loader.scan_skills()++ # Extract skill names from collected data+ skill_names = []+ for skill_info in skill_data:+ skill_name = skill_info.get('skill_name') or ...+ skill_names.append(skill_name)++ # Generate structured skill listing+ skill_prompt = skill_loader.get_registry_prompt(skill_names)+ if skill_prompt:+ system_message = f"{system_message}\n\n{skill_prompt}"+```++### 6. Registry Prompt Output++`get_registry_prompt()` in `skill_loader.py:311-341` generates:++```+## Available Skills++You have access to the following skills. When a user's request matches+a skill's purpose, activate it to help them.++- **http-skill**: Make HTTP requests to external APIs+ - Tools: http-request+- **whatsapp-skill**: Send and receive WhatsApp messages+ - Tools: whatsapp-send, whatsapp-db+- **maps-skill**: Location services via Google Maps++To use a skill, identify when the user's request matches its purpose+and apply the skill's instructions.+```++This text is appended to the system message. The full SKILL.md body (instructions) is available via individual skill entries but the registry prompt provides the high-level listing.++### 7. allowed-tools++- **Parsed** from SKILL.md frontmatter as space-delimited list+- **Included** in registry prompt as informational text for the LLM+- **NOT enforced** in code -- the LLM can call any tool connected to `input-tools`+- Purpose: guides the LLM on which tools are relevant to each skill++---++## Tool Building Pipeline++### 1. Tool Data Collection++In `_collect_agent_connections()`, edges targeting `input-tools` are collected:++```python+elif target_handle == 'input-tools':+ tool_params = await database.get_node_parameters(source_node_id)+ tool_entry = {+ 'node_id': source_node_id,+ 'node_type': tool_type, # 'calculatorTool', 'pythonExecutor', etc.+ 'parameters': tool_params,+ 'label': source_node.get('data', {}).get('label', tool_type),+ 'connected_services': [] # Populated for androidTool only+ }+```++For `androidTool` (toolkit pattern), edges are scanned a second time to find Android service nodes connected to the toolkit, populating `connected_services`.++### 2. Tool Name and Schema Mapping++`_build_tool_from_node()` in `server/services/ai.py` maps each node type:++| Node Type | Tool Name | Pydantic Schema |+|-----------|-----------|-----------------|+| `calculatorTool` | `calculator` | `CalculatorSchema(operation, a, b)` |+| `currentTimeTool` | `get_current_time` | `CurrentTimeSchema(timezone)` |+| `timer` | `timer` | `TimerSchema(delay, unit)` |+| `webSearchTool` | `web_search` | `WebSearchSchema(query, max_results)` |+| `pythonExecutor` | `python_code` | `PythonCodeSchema(code)` |+| `javascriptExecutor` | `javascript_code` | `JavaScriptCodeSchema(code)` |+| `httpRequest` | `http_request` | `HttpRequestSchema(url, method, body)` |+| `whatsappSend` | `whatsapp_send` | `WhatsAppSendSchema(recipient_type, phone, message_type, message, ...)` |+| `whatsappDb` | `whatsapp_db` | `WhatsAppDbSchema(operation, chat_id, query, limit)` |+| `gmaps_locations` | `geocode` | `GeocodeSchema(address)` |+| `gmaps_nearby_places` | `nearby_places` | `NearbyPlacesSchema(location, type, radius)` |+| `androidTool` | `android_device` | `AndroidToolSchema(service_id, action, parameters)` |+| Direct Android services | `android_<service>` | `AndroidServiceSchema(action, parameters)` |+| `aiAgent` | `delegate_to_ai_agent` | `DelegateToAgentSchema(task, context)` |+| Specialized agents | `delegate_to_<type>` | `DelegateToAgentSchema(task, context)` |++### 3. StructuredTool Construction++```python+# Lookup or generate schema+schema = _get_tool_schema(node_type, schema_params)++# Create LangChain tool with placeholder function+tool = StructuredTool.from_function(+ name=tool_name,+ description=tool_description,+ func=lambda **kwargs: kwargs, # Placeholder -- actual execution via callback+ args_schema=schema # Pydantic BaseModel subclass+)++# Store config for execution+tool_configs[tool.name] = {+ 'node_id': node_id,+ 'node_type': node_type,+ 'parameters': node_params,+ 'connected_services': connected_services+}+```++### 4. Database Schema Override++The Tool Schema Editor UI stores custom schemas in the `tool_schemas` table. If a custom schema exists for a node, it takes priority over the default:++```python+db_schema = await self.database.get_tool_schema(node_id)+if db_schema:+ tool_name = db_schema.tool_name+ tool_description = db_schema.tool_description+ schema = self._build_schema_from_config(db_schema.schema_config)+```++### 5. Tool Binding++After all tools are built, they are bound to the LLM model via LangChain:++```python+model_with_tools = chat_model.bind_tools(tools)+```++This makes the LLM aware of all available tool schemas during generation. The LLM decides when to call tools based on the user's request and tool descriptions.++---++## Tool Execution Flow++### 1. Tool Executor Callback++Built as a closure in `execute_agent()` / `execute_chat_agent()`:++```python+async def tool_executor(tool_name: str, tool_args: Dict) -> Any:+ config = tool_configs.get(tool_name, {})+ tool_node_id = config.get('node_id')++ # Broadcast "executing" status to tool node (UI glow animation)+ await broadcaster.update_node_status(tool_node_id, "executing", {...})++ # Inject services for nested delegation+ config['ai_service'] = self+ config['database'] = self.database+ config['nodes'] = context.get('nodes', [])+ config['edges'] = context.get('edges', [])++ # Dispatch to handler+ result = await execute_tool(tool_name, tool_args, config)++ # Broadcast "success" status+ await broadcaster.update_node_status(tool_node_id, "success", {...})++ return result+```++### 2. Dispatch Router++`execute_tool()` in `server/services/handlers/tools.py` routes by `node_type`:++```python+async def execute_tool(tool_name, tool_args, config):+ node_type = config.get('node_type', '')++ if node_type == 'calculatorTool': return await _execute_calculator(tool_args)+ elif node_type == 'pythonExecutor': return await _execute_python_code(tool_args, config['parameters'])+ elif node_type == 'javascriptExecutor': return await _execute_javascript_code(tool_args, config['parameters'])+ elif node_type == 'currentTimeTool': return await _execute_current_time(tool_args)+ elif node_type == 'timer': return await handle_timer(...)+ elif node_type == 'webSearchTool': return await _execute_web_search(tool_args)+ elif node_type == 'whatsappSend': return await _execute_whatsapp_send(tool_args)+ elif node_type == 'whatsappDb': return await _execute_whatsapp_db(tool_args)+ elif node_type == 'androidTool': return await _execute_android_toolkit(tool_args, config)+ elif node_type in ANDROID_SERVICE_TYPES: return await _execute_android_service(tool_args, config)+ elif node_type == 'gmaps_locations': return await _execute_geocoding(tool_args)+ elif node_type == 'gmaps_nearby_places': return await _execute_nearby_places(tool_args)+ elif node_type in DELEGATED_AGENT_TYPES: return await _execute_delegated_agent(tool_args, config)+ else: return await _execute_generic(tool_args, config)+```++### 3. Status Broadcasting++Three-level broadcasting ensures the UI shows tool execution state:++```+1. Agent node: "executing_tool" phase with tool_name+2. Tool node: "executing" status (cyan glow animation on SquareNode)+3. Completion: "success" or "error" on tool node+```++For the Android Toolkit, status is broadcast to the **connected service node** (e.g., Battery Monitor), not the toolkit itself -- so the user sees the actual service node glowing.++### 4. Dual-Purpose Tools++Nodes like `pythonExecutor`, `whatsappSend`, `gmaps_locations` work as both workflow nodes and AI tools. The handler prioritizes LLM-provided args, falling back to node parameters:++```python+async def _execute_python_code(args, node_params):+ # LLM args take priority, node params as fallback+ code = args.get('code', '') or node_params.get('code', '')+```++### 5. Android Toolkit (Sub-Node Pattern)++```+[Battery Monitor] --++[WiFi Automation] --+--> [Android Toolkit] --(tools)--> [AI Agent]+[Location] --------++```++The LLM sees a single `android_device` tool. It specifies `service_id` to choose which connected service to invoke:++```python+async def _execute_android_toolkit(args, config):+ service_id = args.get('service_id')+ connected_services = config.get('connected_services', [])++ # Find matching service by service_id+ target = next((s for s in connected_services if s['service_id'] == service_id), None)++ # Broadcast to connected service node (not toolkit)+ await broadcaster.update_node_status(target['node_id'], "executing", {...})++ # Execute via AndroidService+ result = await android_service.execute_service(service_id=service_id, action=action, ...)+```++### 6. Direct Android Service Tools++Android service nodes can also connect directly to `input-tools` (bypassing the toolkit). The handler maps camelCase node types to snake_case service IDs:++```python+SERVICE_ID_MAP = {+ 'batteryMonitor': 'battery',+ 'wifiAutomation': 'wifi_automation',+ 'bluetoothAutomation': 'bluetooth_automation',+ 'location': 'location',+ 'appLauncher': 'app_launcher',+ # ... 16 total mappings+}+```++### 7. Agent Delegation (Fire-and-Forget)++When an AI Agent or specialized agent is connected to `input-tools`, the parent can delegate tasks:++```python+async def _execute_delegated_agent(args, config):+ # Inject API key for child agent+ provider = detect_ai_provider(node_type, child_params)+ api_key = await ai_service.auth.get_api_key(provider)+ child_params['api_key'] = api_key++ # Build prompt from delegation args+ child_params['prompt'] = f"{args['task']}\n\nContext:\n{args.get('context', '')}"++ # Spawn as background task+ async def run_child_agent():+ await broadcaster.update_node_status(node_id, "executing", {...})+ result = await handle_chat_agent(node_id, node_type, child_params, child_context, ...)+ await broadcaster.update_node_status(node_id, "success", {...})+ return result++ task = asyncio.create_task(run_child_agent())+ _delegated_tasks[task_id] = task++ # Return immediately -- parent continues without waiting+ return {"status": "delegated", "task_id": task_id}+```++Design decisions:+- **Memory isolation**: Child uses its own connected memory, not shared with parent+- **Error isolation**: Child errors don't propagate to parent+- **Task tracking**: Background tasks in `_delegated_tasks` dict, cleaned up on completion++---++## Memory Integration++### Collection++Memory data collected from `input-memory` handle in `_collect_agent_connections()`:++```python+if target_handle == 'input-memory':+ memory_params = await database.get_node_parameters(source_node_id)+ memory_data = {+ 'node_id': source_node_id,+ 'session_id': memory_params.get('sessionId', 'default'),+ 'window_size': int(memory_params.get('windowSize', 10)),+ 'memory_content': memory_params.get('memoryContent', ''), # Markdown+ 'long_term_enabled': memory_params.get('longTermEnabled', False),+ 'retrieval_count': int(memory_params.get('retrievalCount', 3))+ }+```++### Message Construction++```python+# 1. System message (with skills already injected)+initial_messages = [SystemMessage(content=system_message)]++# 2. Long-term memory retrieval (if enabled)+if memory_data.get('long_term_enabled'):+ store = _get_memory_vector_store(session_id) # InMemoryVectorStore per session+ docs = store.similarity_search(prompt, k=retrieval_count)+ initial_messages.append(SystemMessage(content=f"Relevant context:\n{docs}"))++# 3. Short-term conversation history+history_messages = _parse_memory_markdown(memory_content) # Markdown -> [HumanMessage, AIMessage, ...]+initial_messages.extend(history_messages)++# 4. Current user prompt+initial_messages.append(HumanMessage(content=prompt))+```++### Post-Execution Save++After graph execution completes:++```python+# Append new exchange+updated = _append_to_memory_markdown(memory_content, 'human', prompt)+updated = _append_to_memory_markdown(updated, 'ai', response)++# Trim to window size, get removed texts+updated, removed = _trim_markdown_window(updated, window_size)++# Archive removed texts to vector store (if enabled)+if removed and long_term_enabled:+ store = _get_memory_vector_store(session_id)+ store.add_texts(removed)++# Save updated markdown back to node parameters+await database.save_node_parameters(memory_node_id, {'memoryContent': updated})+```++### Memory Format++Stored as human-readable markdown:++```markdown+# Conversation History++### **Human** (2025-01-30 14:23:45)+What is the weather like today?++### **Assistant** (2025-01-30 14:23:48)+I don't have access to real-time weather data...+```++---++## execute_agent vs execute_chat_agent++Both methods live in `server/services/ai.py` and follow the same general pattern. Key differences:++| Aspect | `execute_agent()` | `execute_chat_agent()` |+|--------|-------------------|----------------------|+| **Graph construction** | Always builds full LangGraph StateGraph | Conditional: LangGraph if tools, simple `ainvoke()` if no tools |+| **Tool failure** | Re-raises exceptions (LangGraph handles) | Returns `{"error": str(e)}` (softer handling) |+| **No-tool path** | N/A -- always uses graph | `response = await chat_model.ainvoke(messages)` (no graph overhead) |+| **Result metadata** | `agent_type: "agent"` | `agent_type: "chat" / "chat_with_skills" / "chat_with_tools" / "chat_with_skills_and_tools"` |++### Specialized Agent Routing++All 10 specialized agents route to `handle_chat_agent` via the NodeExecutor registry:++```python+# server/services/node_executor.py+SPECIALIZED_AGENT_TYPES = {+ 'android_agent', 'coding_agent', 'web_agent', 'task_agent', 'social_agent',+ 'travel_agent', 'tool_agent', 'productivity_agent', 'payments_agent', 'consumer_agent'+}++# All specialized agents map to handle_chat_agent+for agent_type in SPECIALIZED_AGENT_TYPES:+ registry[agent_type] = partial(handle_chat_agent, ai_service=self.ai_service, database=self.database)+```++### Chat Agent Conditional Graph++```python+# In execute_chat_agent():+if all_tools:+ # Full LangGraph with tool execution loop+ agent_graph = build_agent_graph(chat_model, tools=all_tools, tool_executor=executor)+ final_state = await agent_graph.ainvoke(initial_state)+else:+ # Simple invoke -- no graph, no tool overhead+ response = await chat_model.ainvoke(messages)+```++This optimization means tool-less Chat Agent conversations skip LangGraph entirely for faster response times.diff --git a/docs-internal/ai_tool_node_creation.md b/docs-internal/ai_tool_node_creation.mdnew file mode 100644index 00000000..dc29469a--- /dev/null+++ b/docs-internal/ai_tool_node_creation.md@@ -0,0 +1,663 @@+# AI Agent Tool Node Creation Guide++This guide provides a complete walkthrough for creating **dedicated AI Agent tool nodes** - nodes that ONLY work as AI tools (passive, no workflow execution). For nodes that work as BOTH workflow nodes AND AI tools, see the [Dual-Purpose Tool Node Guide](./dual_purpose_tool_node_creation.md).++> **Related Documentation:**+> - [Specialized Agent Guide](./specialized_agent_node_creation.md) - Create specialized AI agents (Android, Coding, Web, Social, Task)+> - [Dual-Purpose Tool Node Guide](./dual_purpose_tool_node_creation.md) - Nodes that work as workflow nodes AND AI tools+> - [Node Creation Guide](./node_creation.md) - General node creation patterns+> - [CLAUDE.md](../CLAUDE.md) - Project overview, toolkit sub-node execution pattern++## Table of Contents++1. [Architecture Overview](#architecture-overview)+2. [Tool Calling Architecture](#tool-calling-architecture)+3. [Tool Types](#tool-types)+4. [Creating a Simple Tool Node](#creating-a-simple-tool-node)+5. [Backend Implementation](#backend-implementation)+6. [Status Broadcasting](#status-broadcasting)+7. [Complete Example: Weather Tool](#complete-example-weather-tool)+8. [Checklist](#checklist)++> **Note:** For creating **Specialized Agents** (like Android Control Agent, Social Media Agent), see the [Specialized Agent Guide](./specialized_agent_node_creation.md).++---++## Architecture Overview++### Tool Execution Flow++```+User Query → AI Agent → LLM decides to use tool+ ↓+ LangGraph tool binding+ ↓+ Tool executor callback+ ↓+ Status broadcast (executing)+ ↓+ Handler executes tool logic+ ↓+ Status broadcast (success)+ ↓+ Result returned to LLM+ ↓+ LLM continues reasoning+```++### Key Components++| Component | Location | Purpose |+|-----------|----------|---------|+| Tool Node Definition | `client/src/nodeDefinitions/toolNodes.ts` | Frontend node structure |+| Tool Building | `server/services/ai.py` → `_build_tool_from_node()` | Build LangChain tools from nodes |+| Tool Schema | `server/services/ai.py` → `_get_tool_schema()` | Pydantic schema for LLM |+| Tool Handler | `server/services/handlers/tools.py` → `execute_tool()` | Execution dispatch |+| Constants | `server/constants.py` | Type definitions |+| Status Broadcaster | `server/services/status_broadcaster.py` | UI updates |++---++## Tool Calling Architecture++### Overview++Tool nodes connect to AI Agent's `input-tools` handle. The LLM uses the tool's Pydantic schema to understand parameters and fills them automatically during reasoning.++**Key Principle**: Skills (`input-skill`) provide SKILL.md context only. Actual tool execution happens via nodes connected to `input-tools`.++### Tool Building Pipeline++```+Tool Node (connected to input-tools)+ ↓+handle_ai_agent() / handle_chat_agent() collects tool_data+ ↓+_build_tool_from_node() creates StructuredTool from node config+ ↓+_get_tool_schema() returns Pydantic schema for LLM+ ↓+LangGraph binds tools to LLM+ ↓+LLM decides to call tool with auto-filled arguments+ ↓+tool_executor callback → execute_tool() in handlers/tools.py+ ↓+Dispatch to handler: _execute_calculator(), _execute_http_request(), etc.+```++### Key Functions++| Function | File | Purpose |+|----------|------|---------|+| `_build_tool_from_node()` | `ai.py` | Converts node config to LangChain StructuredTool |+| `_get_tool_schema()` | `ai.py` | Returns Pydantic schema for tool's LLM-visible parameters |+| `execute_tool()` | `handlers/tools.py` | Dispatches tool execution to appropriate handler |++### Supported Tool Types++All tools use the same schema-based approach:++| Tool Type | Schema | Handler |+|-----------|--------|---------|+| `calculatorTool` | `CalculatorSchema` | `_execute_calculator()` |+| `currentTimeTool` | `CurrentTimeSchema` | `_execute_current_time()` |+| `webSearchTool` | `WebSearchSchema` | `_execute_web_search()` |+| `httpRequest` | `HttpRequestSchema` | `_execute_http_request()` |+| `pythonExecutor` | `PythonCodeSchema` | `_execute_python_code()` |+| `whatsappSend` | `WhatsAppSendSchema` | `_execute_whatsapp_send()` |+| `whatsappDb` | `WhatsAppDbSchema` | `_execute_whatsapp_db()` |+| `androidTool` | `AndroidToolSchema` | `_execute_android_toolkit()` |+| `aiAgent` | `DelegateToAgentSchema` | `_execute_delegated_agent()` |+| `chatAgent` | `DelegateToAgentSchema` | `_execute_delegated_agent()` |+| Specialized Agents | `DelegateToAgentSchema` | `_execute_delegated_agent()` |++> **Note:** AI Agents and Specialized Agents can be used as tools by other agents. When connected to `input-tools`, they use the `DelegateToAgentSchema` with `task` and optional `context` fields. Execution is fire-and-forget (async) - the parent agent continues immediately while the child executes in background. See [Specialized Agent Guide](./specialized_agent_node_creation.md#async-agent-delegation) for details.++### Skills vs Tools++| Aspect | Skills (`input-skill`) | Tools (`input-tools`) |+|--------|------------------------|----------------------|+| Purpose | Provide context/instructions | Execute actions |+| Content | SKILL.md markdown loaded into system message | Node configuration with Pydantic schema |+| Execution | None - context only | LLM calls tool, handler executes |+| Connection | Connect to `input-skill` handle | Connect to `input-tools` handle |+| Example | `httpSkill` provides HTTP capability context | `httpRequest` tool actually makes requests |++---++## Tool Types++### Simple Tool Nodes++Self-contained tools with all logic in a single handler.++**Examples:** `calculatorTool`, `currentTimeTool`, `webSearchTool`++```+[Calculator Tool] ──────────────────────→ [AI Agent]+ │ │+ └─ Pydantic schema + handler └─ Uses tool during reasoning+```++> **For Specialized Agents** (like Android Control Agent, Social Media Agent), see the [Specialized Agent Guide](./specialized_agent_node_creation.md).++---++## Creating a Simple Tool Node++### Step 1: Frontend Node Definition++Create or update `client/src/nodeDefinitions/toolNodes.ts`:++```typescript+import { INodeTypeDescription, NodeConnectionType } from '../types/INodeProperties';++// Tool-specific icon (use Unicode symbols for consistency)+const WEATHER_TOOL_ICON = `<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">+ <circle cx="12" cy="12" r="5"/>+ <path d="M12 1v2M12 21v2M4.22 4.22l1.42 1.42M18.36 18.36l1.42 1.42M1 12h2M21 12h2M4.22 19.78l1.42-1.42M18.36 5.64l1.42-1.42"/>+</svg>`;++export const toolNodes: Record<string, INodeTypeDescription> = {+ // ... existing tools ...++ weatherTool: {+ displayName: 'Weather Tool',+ name: 'weatherTool',+ icon: WEATHER_TOOL_ICON,+ group: ['ai', 'tool'], // CRITICAL: 'tool' marks it as AI tool+ version: 1,+ subtitle: 'Weather Information',+ description: 'Allow AI Agent to get current weather and forecasts',+ defaults: {+ name: 'Weather',+ color: '#3b82f6' // Blue accent+ },++ // Tool nodes are passive - no main input+ inputs: [],++ // Single output connects to AI Agent's input-tools handle+ outputs: [{+ name: 'tool',+ displayName: 'Tool',+ type: 'main' as NodeConnectionType,+ description: 'Connect to AI Agent tool handle'+ }],++ // Tool parameters+ properties: [+ {+ displayName: 'Tool Name',+ name: 'toolName',+ type: 'string',+ default: 'get_weather',+ required: true,+ description: 'Name the AI will use to call this tool'+ },+ {+ displayName: 'Tool Description',+ name: 'toolDescription',+ type: 'string',+ default: 'Get current weather conditions and forecast for a location. Returns temperature, humidity, conditions, and forecast.',+ typeOptions: { rows: 3 },+ description: 'Description helps the AI understand when to use this tool'+ },+ {+ displayName: 'API Provider',+ name: 'apiProvider',+ type: 'options',+ default: 'openweathermap',+ options: [+ { name: 'OpenWeatherMap', value: 'openweathermap' },+ { name: 'WeatherAPI', value: 'weatherapi' }+ ],+ description: 'Weather data provider'+ },+ {+ displayName: 'Units',+ name: 'units',+ type: 'options',+ default: 'metric',+ options: [+ { name: 'Metric (Celsius)', value: 'metric' },+ { name: 'Imperial (Fahrenheit)', value: 'imperial' }+ ]+ }+ ]+ }+};++// Add to TOOL_NODE_TYPES array+export const TOOL_NODE_TYPES = [+ 'calculatorTool',+ 'currentTimeTool',+ 'webSearchTool',+ 'androidTool',+ 'weatherTool' // Add new tool here+];+```++### Step 2: Register in Node Definitions++Update `client/src/nodeDefinitions.ts`:++```typescript+import { toolNodes, TOOL_NODE_TYPES } from './nodeDefinitions/toolNodes';++export const nodeDefinitions: Record<string, INodeTypeDescription> = {+ // ... other nodes ...+ ...toolNodes,+};++export { TOOL_NODE_TYPES };+```++### Step 3: Add to Constants++Update `server/constants.py`:++```python+# Tool node types (connect to AI Agent's input-tools handle)+AI_TOOL_TYPES: FrozenSet[str] = frozenset([+ 'calculatorTool',+ 'currentTimeTool',+ 'webSearchTool',+ 'androidTool',+ 'weatherTool', # Add new tool here+])+```++---++## Backend Implementation++### Step 4: Add Pydantic Schema to `_get_tool_schema()`++In `server/services/ai.py`, add the tool schema inside the `_get_tool_schema()` method. Schemas are defined inline using Pydantic:++```python+def _get_tool_schema(self, node_type: str, params: Dict[str, Any]) -> Type[BaseModel]:+ """Get Pydantic schema for tool based on node type."""++ # ... existing schemas ...++ # Weather tool schema+ if node_type == 'weatherTool':+ class WeatherToolSchema(BaseModel):+ """Schema for weather tool - defines what parameters the LLM can pass."""+ location: str = Field(+ description="City name or coordinates (e.g., 'London', 'New York, US', '51.5,-0.1')"+ )+ include_forecast: bool = Field(+ default=False,+ description="Whether to include 5-day forecast"+ )++ return WeatherToolSchema++ # Fallback: generic schema+ class GenericSchema(BaseModel):+ input: str = Field(description="Input for the tool")++ return GenericSchema+```++**Important**: Schemas define what the LLM can pass to the tool. Use clear `Field(description=...)` to help the LLM understand each parameter.++### Step 5: Create Tool Handler++In `server/services/handlers/tools.py`, add a handler function and register it in the dispatcher:++```python+import httpx+from typing import Dict, Any+from core.logging import get_logger++logger = get_logger(__name__)+++async def _execute_weather_tool(+ llm_params: Dict[str, Any],+ tool_params: Dict[str, Any]+) -> str:+ """Execute weather tool.++ Args:+ llm_params: Parameters from LLM (auto-filled based on schema)+ tool_params: Node configuration from frontend (apiProvider, units, etc.)++ Returns:+ Weather information as string for LLM+ """+ # LLM-provided parameters (from schema)+ location = llm_params.get('location', '')+ include_forecast = llm_params.get('include_forecast', False)++ # Node configuration parameters (from frontend)+ api_provider = tool_params.get('apiProvider', 'openweathermap')+ units = tool_params.get('units', 'metric')+ api_key = tool_params.get('api_key', '')++ if not location:+ return "Error: No location provided"++ if not api_key:+ return "Error: Weather API key not configured"++ try:+ async with httpx.AsyncClient(timeout=10.0) as client:+ if api_provider == 'openweathermap':+ url = "https://api.openweathermap.org/data/2.5/weather"+ params = {'q': location, 'appid': api_key, 'units': units}+ response = await client.get(url, params=params)+ response.raise_for_status()+ data = response.json()++ temp_unit = "C" if units == "metric" else "F"+ result = f"""Weather in {data['name']}:+- Temperature: {data['main']['temp']}{temp_unit}+- Conditions: {data['weather'][0]['description']}+- Humidity: {data['main']['humidity']}%"""++ if include_forecast:+ forecast_url = "https://api.openweathermap.org/data/2.5/forecast"+ forecast_response = await client.get(forecast_url, params=params)+ if forecast_response.status_code == 200:+ forecast_data = forecast_response.json()+ result += "\n\n5-Day Forecast:"+ for item in forecast_data['list'][:5]:+ result += f"\n- {item['dt_txt']}: {item['main']['temp']}{temp_unit}"++ return result+ else:+ return f"Error: Unknown API provider: {api_provider}"++ except httpx.HTTPStatusError as e:+ logger.error(f"Weather API error: {e}")+ return f"Error: Weather API returned {e.response.status_code}"+ except Exception as e:+ logger.error(f"Weather tool error: {e}")+ return f"Error fetching weather: {str(e)}"+```++### Step 6: Register in `execute_tool()` Dispatcher++The `execute_tool()` function dispatches to the appropriate handler based on node type:++```python+async def execute_tool(+ tool_type: str,+ tool_params: Dict[str, Any],+ llm_params: Dict[str, Any]+) -> str:+ """Dispatch tool execution to appropriate handler.++ Args:+ tool_type: Node type (e.g., 'weatherTool', 'calculatorTool')+ tool_params: Node configuration from frontend+ llm_params: Parameters auto-filled by LLM based on schema++ Returns:+ Tool result as string for LLM consumption+ """+ if tool_type == 'calculatorTool':+ return await _execute_calculator(llm_params)+ elif tool_type == 'currentTimeTool':+ return await _execute_current_time(llm_params)+ elif tool_type == 'webSearchTool':+ return await _execute_web_search(llm_params, tool_params)+ elif tool_type == 'httpRequest':+ return await _execute_http_request(llm_params, tool_params)+ elif tool_type == 'pythonExecutor':+ return await _execute_python_code(llm_params, tool_params)+ elif tool_type == 'whatsappSend':+ return await _execute_whatsapp_send(llm_params, tool_params)+ elif tool_type == 'whatsappDb':+ return await _execute_whatsapp_db(llm_params, tool_params)+ elif tool_type == 'androidTool':+ return await _execute_android_toolkit(llm_params, tool_params)+ elif tool_type == 'weatherTool':+ return await _execute_weather_tool(llm_params, tool_params)+ else:+ return f"Unknown tool type: {tool_type}"+```++### Step 7: Add to Default Tool Names/Descriptions++In `server/services/ai.py`, add the tool to `_build_tool_from_node()`:++```python+DEFAULT_TOOL_NAMES = {+ 'calculatorTool': 'calculator',+ 'currentTimeTool': 'get_current_time',+ 'webSearchTool': 'web_search',+ 'androidTool': 'android_device',+ 'whatsappSend': 'whatsapp_send',+ 'whatsappDb': 'whatsapp_db',+ 'weatherTool': 'get_weather', # Add new tool+}++DEFAULT_TOOL_DESCRIPTIONS = {+ 'calculatorTool': 'Perform mathematical calculations.',+ 'currentTimeTool': 'Get the current date and time.',+ 'webSearchTool': 'Search the web for information.',+ 'androidTool': 'Control Android device.',+ 'whatsappSend': 'Send WhatsApp messages.',+ 'whatsappDb': 'Query WhatsApp database.',+ 'weatherTool': 'Get current weather and forecasts.', # Add description+}+```++---++## Status Broadcasting++Tool nodes should show execution status in the UI when the AI Agent calls them.++### Backend: Broadcast Status++In `server/services/ai.py`, the `execute_agent` function broadcasts status:++```python+async def execute_agent(+ self,+ node_id: str,+ parameters: Dict[str, Any],+ context: Dict[str, Any]+) -> Dict[str, Any]:+ """Execute AI Agent with tools."""+ workflow_id = context.get('workflow_id')+ broadcaster = get_status_broadcaster()++ # Build tool executor with status broadcasting+ async def tool_executor(tool_name: str, tool_args: Dict) -> Any:+ config = tool_configs.get(tool_name, {})+ tool_node_id = config.get('node_id')+ tool_type = config.get('tool_type')++ # Broadcast "executing" to tool node+ if tool_node_id and broadcaster:+ await broadcaster.update_node_status(+ tool_node_id,+ "executing",+ {"message": f"Executing {tool_name}", "tool_name": tool_name},+ workflow_id=workflow_id+ )++ # Execute tool+ result = await execute_tool(tool_type, config, tool_args)++ # Broadcast "success" to tool node+ if tool_node_id and broadcaster:+ await broadcaster.update_node_status(+ tool_node_id,+ "success",+ {"message": f"{tool_name} completed", "result": str(result)[:200]},+ workflow_id=workflow_id+ )++ return result++ # ... rest of agent execution+```++### Frontend: Status Display++Tool nodes use the same status mechanism as other nodes via `useNodeStatus`:++```typescript+// In SquareNode.tsx or custom component+const nodeStatus = useNodeStatus(id);+const isExecuting = nodeStatus?.status === 'executing';++// Apply visual effects+style={{+ animation: isExecuting ? 'pulse 1.5s ease-in-out infinite' : 'none',+ boxShadow: isExecuting ? `0 0 10px ${theme.dracula.cyan}` : 'none'+}}+```++---++## Complete Example: Weather Tool++Here's a complete implementation checklist:++### Files to Create/Modify++| File | Action | Purpose |+|------|--------|---------|+| `client/src/nodeDefinitions/toolNodes.ts` | Modify | Add node definition |+| `server/constants.py` | Modify | Add to AI_TOOL_TYPES |+| `server/services/ai.py` | Modify | Add schema + DEFAULT_TOOL_NAMES |+| `server/services/handlers/tools.py` | Modify | Add handler + dispatcher case |++### Frontend Code++```typescript+// toolNodes.ts+weatherTool: {+ displayName: 'Weather Tool',+ name: 'weatherTool',+ icon: '...',+ group: ['ai', 'tool'],+ outputs: [{+ name: 'tool',+ displayName: 'Tool',+ type: 'main' as NodeConnectionType,+ description: 'Connect to AI Agent tool handle'+ }],+ properties: [+ { name: 'toolName', type: 'string', default: 'get_weather' },+ { name: 'toolDescription', type: 'string', default: 'Get weather...' },+ { name: 'apiProvider', type: 'options', default: 'openweathermap', ... },+ { name: 'units', type: 'options', default: 'metric', ... }+ ]+}++// Add to TOOL_NODE_TYPES+export const TOOL_NODE_TYPES = [..., 'weatherTool'];+```++### Backend Code++```python+# constants.py+AI_TOOL_TYPES: FrozenSet[str] = frozenset([+ ...,+ 'weatherTool',+])++# ai.py - in _get_tool_schema()+if node_type == 'weatherTool':+ class WeatherToolSchema(BaseModel):+ location: str = Field(description="City name or coordinates")+ include_forecast: bool = Field(default=False, description="Include 5-day forecast")+ return WeatherToolSchema++# ai.py - in _build_tool_from_node()+DEFAULT_TOOL_NAMES = {+ ...,+ 'weatherTool': 'get_weather',+}+DEFAULT_TOOL_DESCRIPTIONS = {+ ...,+ 'weatherTool': 'Get current weather conditions and forecast for a location.',+}++# tools.py - handler function+async def _execute_weather_tool(llm_params: Dict, tool_params: Dict) -> str:+ location = llm_params.get('location', '')+ include_forecast = llm_params.get('include_forecast', False)+ api_key = tool_params.get('api_key', '')+ # ... implementation ...++# tools.py - in execute_tool() dispatcher+elif tool_type == 'weatherTool':+ return await _execute_weather_tool(llm_params, tool_params)+```++---++## Checklist++### Tool Node Creation++**Frontend (`client/src/nodeDefinitions/toolNodes.ts`):**+- [ ] Add node definition+ - [ ] Set `group: ['tool', 'ai']` (CRITICAL: 'tool' marks as AI tool)+ - [ ] Define `toolName` and `toolDescription` properties+ - [ ] Set output with `name: 'tool'` connecting to AI Agent's `input-tools`+- [ ] Add to `TOOL_NODE_TYPES` array+- [ ] Verify node appears in Component Palette under "AI Tools" category++**Backend - Constants (`server/constants.py`):**+- [ ] Add to `AI_TOOL_TYPES` frozenset++**Backend - Schema (`server/services/ai.py`):**+- [ ] Add Pydantic schema in `_get_tool_schema()` method+- [ ] Add tool name to `DEFAULT_TOOL_NAMES` dict in `_build_tool_from_node()`+- [ ] Add tool description to `DEFAULT_TOOL_DESCRIPTIONS` dict++**Backend - Handler (`server/services/handlers/tools.py`):**+- [ ] Create handler function `_execute_<tool_name>(llm_params, tool_params)`+- [ ] Add dispatcher case in `execute_tool()` function++**Testing:**+- [ ] Connect tool to AI Agent's `input-tools` handle+- [ ] Verify tool appears in agent's available tools (check logs)+- [ ] Test tool execution via agent query+- [ ] Verify status broadcast shows execution animation in UI++> **For Specialized Agents**, see the [Specialized Agent Guide](./specialized_agent_node_creation.md#testing).++---++## Best Practices++1. **Tool Descriptions**: Write clear, detailed descriptions that help the LLM understand when to use the tool. The `DEFAULT_TOOL_DESCRIPTIONS` in `ai.py` is what the LLM sees.++2. **Schema Fields**: Use descriptive `Field(description=...)` parameters. The LLM reads these to understand what values to provide.++3. **Parameter Separation**:+ - `llm_params`: Values the LLM provides based on schema (e.g., `location`, `query`)+ - `tool_params`: Node configuration from frontend (e.g., `api_key`, `apiProvider`)++4. **Error Handling**: Return clear error messages as strings. The LLM can understand and act on them.++5. **Status Broadcasting**: The AI Agent handles status broadcasting automatically when tools execute.++6. **Logging**: Use structured logging with tool name and parameters:+ ```python+ logger.info(f"[{tool_type}] Executing with params: {llm_params}")+ ```++7. **Timeouts**: Set reasonable timeouts for external API calls (default 10-30 seconds).++8. **API Keys**: Retrieve API keys from `tool_params` (node configuration), not `llm_params`.++9. **Return Strings**: Tool handlers should return strings that the LLM can process. Format results clearly.++10. **Skills Provide Context Only**: Remember that SKILL.md files only provide context to the system message. Actual tool execution happens via nodes connected to `input-tools`.diff --git a/docs-internal/dual_purpose_tool_node_creation.md b/docs-internal/dual_purpose_tool_node_creation.mdnew file mode 100644index 00000000..893b2d8d--- /dev/null+++ b/docs-internal/dual_purpose_tool_node_creation.md@@ -0,0 +1,757 @@+# Dual-Purpose Tool Node Creation Guide++This guide provides a complete walkthrough for creating **dual-purpose nodes** - nodes that work both as standalone workflow nodes AND as AI Agent/Zeenie tools. This pattern allows existing workflow nodes to be connected directly to an AI Agent's `input-tools` handle, where the LLM fills in the node's parameter schema.++> **Related Documentation:**+> - [AI Agent Tool Node Creation Guide](./ai_tool_node_creation.md) - Dedicated tool nodes (passive, tool-only)+> - [Node Creation Guide](./node_creation.md) - General node creation patterns+> - [CLAUDE.md](../CLAUDE.md) - Project overview and architecture++## Table of Contents++1. [Architecture Overview](#architecture-overview)+2. [Dual-Purpose vs Dedicated Tool Nodes](#dual-purpose-vs-dedicated-tool-nodes)+3. [Creating a Dual-Purpose Tool Node](#creating-a-dual-purpose-tool-node)+4. [Backend Implementation](#backend-implementation)+5. [Complete Example: WhatsApp Nodes](#complete-example-whatsapp-nodes)+6. [Checklist](#checklist)++---++## Architecture Overview++### Dual-Purpose Node Design++```+ ┌─────────────────────────────────────────┐+ │ Dual-Purpose Node │+ │ (e.g., whatsappSend) │+ │ │+ Workflow Input ──┤ input-main │+ │ │ │+ │ ▼ │+ │ ┌───────────┐ ┌───────────────────┐ │+ │ │ Parameters│ │ Tool Output │─┼──→ AI Agent input-tools+ │ └───────────┘ └───────────────────┘ │+ │ │ │+ │ ▼ │+ Workflow Output ◄┼── output-main │+ │ │+ └─────────────────────────────────────────┘+```++### Two Usage Modes++**Mode 1: Workflow Node**+```+[Trigger] → [WhatsApp Send] → [Next Node]+ │+ Executes with configured parameters+```++**Mode 2: AI Tool**+```+[WhatsApp Send] ──(tool output)──→ [AI Agent input-tools]+ │+ LLM fills parameter schema+ Tool handler executes+```++### Key Components++| Component | Location | Purpose |+|-----------|----------|---------|+| Node Definition | `client/src/nodeDefinitions/*.ts` | Frontend structure with both outputs |+| Tool Schema | `server/services/ai.py` | Pydantic schema for LLM |+| Tool Handler | `server/services/handlers/tools.py` | Tool execution logic |+| Node Handler | `server/services/handlers/*.py` | Workflow execution logic |+| Execution Logic | `server/services/execution/models.py` | Dynamic exclusion when used as tool |++---++## Dual-Purpose vs Dedicated Tool Nodes++### Dedicated Tool Nodes (Passive)++Nodes that ONLY work as AI tools, not as workflow nodes.++**Characteristics:**+- `inputs: []` - No main input+- `outputs: [{ name: 'tool' }]` - Single tool output+- Added to `AI_TOOL_TYPES` constant+- Excluded from workflow execution via `CONFIG_NODE_TYPES`++**Examples:** `calculatorTool`, `currentTimeTool`, `webSearchTool`++### Dual-Purpose Tool Nodes (Active)++Nodes that work BOTH as workflow nodes AND as AI tools.++**Characteristics:**+- Has main input AND main output for workflow data flow+- Has additional `tool` output for AI Agent connection+- `group` includes `'tool'` marker+- NOT in `AI_TOOL_TYPES` (would break workflow execution)+- Dynamically excluded when connected to `input-tools` handle++**Examples:** `whatsappSend`, `whatsappDb`, `httpRequest`, `pythonExecutor`, `javascriptExecutor`, `addLocations`, `showNearbyPlaces`++### Comparison Table++| Feature | Dedicated Tool | Dual-Purpose Tool |+|---------|----------------|-------------------|+| Workflow execution | No | Yes |+| AI tool usage | Yes | Yes |+| Main input | No | Yes |+| Main output | No | Yes |+| Tool output | Yes | Yes |+| In `AI_TOOL_TYPES` | Yes | **No** |+| Group array | `['ai', 'tool']` | `['category', 'tool']` |++---++## Creating a Dual-Purpose Tool Node++### Step 1: Frontend Node Definition++Add/modify the node definition with both workflow handles AND tool output:++```typescript+// client/src/nodeDefinitions/myNodes.ts+import { INodeTypeDescription, NodeConnectionType } from '../types/INodeProperties';++export const myNodes: Record<string, INodeTypeDescription> = {+ myDualPurposeNode: {+ displayName: 'My Node',+ name: 'myDualPurposeNode',+ icon: '...',+ // CRITICAL: Include 'tool' in group for dual-purpose behavior+ group: ['myCategory', 'tool'],+ version: 1,+ subtitle: 'Description',+ description: 'Works as workflow node and AI tool',+ defaults: { name: 'My Node', color: '#6366f1' },++ // Main input for workflow data flow+ inputs: [{+ name: 'main',+ displayName: 'Input',+ type: 'main' as NodeConnectionType,+ description: 'Trigger input'+ }],++ // BOTH outputs: main for workflow, tool for AI Agent+ outputs: [+ {+ name: 'main',+ displayName: 'Output',+ type: 'main' as NodeConnectionType,+ description: 'Result output'+ },+ {+ name: 'tool',+ displayName: 'Tool',+ type: 'main' as NodeConnectionType,+ description: 'Connect to AI Agent tool handle'+ }+ ],++ // Full parameter schema - LLM will fill these when used as tool+ properties: [+ {+ displayName: 'Action',+ name: 'action',+ type: 'options',+ default: 'default_action',+ options: [+ { name: 'Action 1', value: 'action1' },+ { name: 'Action 2', value: 'action2' }+ ],+ description: 'Action to perform'+ },+ {+ displayName: 'Parameter',+ name: 'param1',+ type: 'string',+ default: '',+ description: 'Required parameter'+ },+ // ... more parameters+ ]+ }+};+```++### Step 2: Define Tool Schema (Backend)++In `server/services/ai.py`, add the Pydantic schema in `_get_tool_schema()`:++```python+from pydantic import BaseModel, Field+from typing import Optional++def _get_tool_schema(node_type: str) -> Optional[type]:+ """Get Pydantic schema for tool type."""++ # ... existing schemas ...++ # My dual-purpose node schema+ if node_type == 'myDualPurposeNode':+ class MyNodeSchema(BaseModel):+ """Description for LLM to understand the tool."""+ action: str = Field(+ default="action1",+ description="Action to perform: 'action1' or 'action2'"+ )+ param1: str = Field(+ description="Required parameter description"+ )+ optional_param: Optional[str] = Field(+ default=None,+ description="Optional parameter"+ )+ return MyNodeSchema++ return None+```++**Schema Design Tips:**+- Use snake_case for field names (LLM convention)+- Provide clear descriptions with examples+- Use `Optional` for non-required fields+- Include validation in Field() where appropriate++### Step 3: Add Tool Handler (Backend)++In `server/services/handlers/tools.py`:++```python+from typing import Dict, Any+from core.logging import get_logger++logger = get_logger(__name__)+++async def _execute_my_dual_purpose_node(+ tool_args: Dict[str, Any],+ node_params: Dict[str, Any]+) -> Dict[str, Any]:+ """Execute my dual-purpose node as a tool.++ Args:+ tool_args: LLM-provided arguments matching the schema+ node_params: Node configuration parameters (from node definition)++ Returns:+ Dict with execution result+ """+ # Import the existing node handler+ from services.handlers.my_handlers import handle_my_node++ # Map LLM args (snake_case) to node params (camelCase)+ parameters = {+ 'action': tool_args.get('action', 'action1'),+ 'param1': tool_args.get('param1', ''),+ 'optionalParam': tool_args.get('optional_param', ''),+ }++ # Validate required fields+ if not parameters['param1']:+ return {"error": "param1 is required"}++ logger.info(f"[MyNode Tool] Executing {parameters['action']}...")++ try:+ # Call the existing node handler+ result = await handle_my_node(+ node_id="tool_my_node",+ node_type="myDualPurposeNode",+ parameters=parameters,+ context={}+ )++ if result.get('success'):+ return {+ "success": True,+ "action": parameters['action'],+ "result": result.get('result', {})+ }+ else:+ return {"error": result.get('error', 'Unknown error')}++ except Exception as e:+ logger.error(f"[MyNode Tool] Error: {e}")+ return {"error": f"Execution failed: {str(e)}"}+```++### Step 4: Register in Tool Dispatcher++In `server/services/handlers/tools.py`, add to `execute_tool()`:++```python+async def execute_tool(tool_name: str, tool_args: Dict[str, Any],+ config: Dict[str, Any]) -> Dict[str, Any]:+ """Execute a tool by name using the appropriate handler."""+ node_type = config.get('node_type', '')++ # ... existing tool handlers ...++ # My dual-purpose node (existing node used as tool)+ if node_type == 'myDualPurposeNode':+ return await _execute_my_dual_purpose_node(tool_args, config.get('parameters', {}))++ # Generic fallback+ logger.warning(f"[Tool] Unknown tool type: {node_type}")+ return await _execute_generic(tool_args, config)+```++### Step 5: DO NOT Add to AI_TOOL_TYPES++**Important:** Dual-purpose nodes should NOT be added to `AI_TOOL_TYPES` in `server/constants.py`.++Why? Because `AI_TOOL_TYPES` is included in `CONFIG_NODE_TYPES`, which excludes nodes from independent workflow execution. Adding a dual-purpose node there would break its workflow functionality.++The execution engine dynamically detects when a node is connected to `input-tools` handle and excludes it from parallel execution in that specific workflow only.++---++## Backend Execution Logic++### Dynamic Tool Detection++The execution engine in `server/services/execution/models.py` dynamically detects tools:++```python+# In ExecutionContext.create()+# Nodes connected to AI Agent/Zeenie config handles are sub-nodes+# These handles: input-memory, input-tools, input-skill+if target in ai_agent_node_ids and source and target_handle:+ if target_handle in ('input-memory', 'input-tools', 'input-skill'):+ subnode_ids.add(source)++# Skip toolkit sub-nodes - they execute only via toolkit tool calls+if node_id in subnode_ids:+ continue+```++This means:+1. When `whatsappSend` is used in a normal workflow → executes independently+2. When `whatsappSend` is connected to AI Agent's `input-tools` → excluded from independent execution, only executes via tool call++---++## Complete Example: WhatsApp Nodes++The `whatsappSend` and `whatsappDb` nodes are canonical examples of dual-purpose tools.++### Frontend Definition (whatsappNodes.ts)++```typescript+whatsappSend: {+ displayName: 'WhatsApp Send',+ name: 'whatsappSend',+ icon: WHATSAPP_SEND_ICON,+ group: ['whatsapp', 'tool'], // Include 'tool' for dual-purpose+ version: 1,+ subtitle: 'Send WhatsApp Message',+ description: 'Send text, media, location, or contact messages via WhatsApp',+ defaults: { name: 'WhatsApp Send', color: '#25D366' },++ // Main input for workflow+ inputs: [{+ name: 'main',+ displayName: 'Input',+ type: 'main' as NodeConnectionType,+ description: 'Message input'+ }],++ // Both outputs+ outputs: [+ {+ name: 'main',+ displayName: 'Output',+ type: 'main' as NodeConnectionType,+ description: 'Message output'+ },+ {+ name: 'tool',+ displayName: 'Tool',+ type: 'main' as NodeConnectionType,+ description: 'Connect to AI Agent tool handle'+ }+ ],++ // Full parameter schema+ properties: [+ {+ displayName: 'Send To',+ name: 'recipientType',+ type: 'options',+ options: [+ { name: 'Phone Number', value: 'phone' },+ { name: 'Group', value: 'group' }+ ],+ default: 'phone'+ },+ {+ displayName: 'Phone Number',+ name: 'phone',+ type: 'string',+ default: '',+ displayOptions: { show: { recipientType: ['phone'] } }+ },+ {+ displayName: 'Message Type',+ name: 'messageType',+ type: 'options',+ options: [+ { name: 'Text', value: 'text' },+ { name: 'Image', value: 'image' },+ { name: 'Location', value: 'location' },+ // ...+ ],+ default: 'text'+ },+ // ... more parameters+ ]+}+```++### Backend Schema (ai.py)++```python+if node_type == 'whatsappSend':+ class WhatsAppSendSchema(BaseModel):+ """Send WhatsApp messages to contacts or groups."""+ recipient_type: str = Field(+ default="phone",+ description="Send to: 'phone' for individual or 'group' for group chat"+ )+ phone: Optional[str] = Field(+ default=None,+ description="Phone number without + prefix (e.g., 1234567890). Required for recipient_type='phone'"+ )+ group_id: Optional[str] = Field(+ default=None,+ description="Group JID (e.g., [email protected]). Required for recipient_type='group'"+ )+ message_type: str = Field(+ default="text",+ description="Message type: 'text', 'image', 'video', 'audio', 'document', 'sticker', 'location', 'contact'"+ )+ message: Optional[str] = Field(+ default=None,+ description="Text message content. Required for message_type='text'"+ )+ media_url: Optional[str] = Field(+ default=None,+ description="URL for media (image/video/audio/document/sticker)"+ )+ caption: Optional[str] = Field(+ default=None,+ description="Caption for media messages"+ )+ latitude: Optional[float] = Field(default=None, description="Latitude for location")+ longitude: Optional[float] = Field(default=None, description="Longitude for location")+ location_name: Optional[str] = Field(default=None, description="Location name")+ address: Optional[str] = Field(default=None, description="Address")+ contact_name: Optional[str] = Field(default=None, description="Contact name")+ vcard: Optional[str] = Field(default=None, description="vCard 3.0 format")+ return WhatsAppSendSchema+```++### Backend Handler (tools.py)++```python+async def _execute_whatsapp_send(args: Dict[str, Any],+ node_params: Dict[str, Any]) -> Dict[str, Any]:+ """Send WhatsApp message with full message type support."""+ from services.handlers.whatsapp import handle_whatsapp_send++ # Map LLM args (snake_case) to node params (camelCase)+ parameters = {+ 'recipientType': args.get('recipient_type', 'phone'),+ 'phone': args.get('phone', ''),+ 'group_id': args.get('group_id', ''),+ 'messageType': args.get('message_type', 'text'),+ 'message': args.get('message', ''),+ 'mediaSource': 'url' if args.get('media_url') else 'none',+ 'mediaUrl': args.get('media_url', ''),+ 'caption': args.get('caption', ''),+ 'latitude': args.get('latitude'),+ 'longitude': args.get('longitude'),+ 'locationName': args.get('location_name', ''),+ 'address': args.get('address', ''),+ 'contactName': args.get('contact_name', ''),+ 'vcard': args.get('vcard', ''),+ }++ # Validate based on message type+ recipient_type = parameters['recipientType']+ message_type = parameters['messageType']++ if recipient_type == 'phone' and not parameters['phone']:+ return {"error": "Phone number is required for recipient_type='phone'"}+ if recipient_type == 'group' and not parameters['group_id']:+ return {"error": "Group ID is required for recipient_type='group'"}+ if message_type == 'text' and not parameters['message']:+ return {"error": "Message content is required for message_type='text'"}+ # ... more validation++ try:+ result = await handle_whatsapp_send(+ node_id="tool_whatsapp_send",+ node_type="whatsappSend",+ parameters=parameters,+ context={}+ )++ if result.get('success'):+ return {+ "success": True,+ "recipient": parameters['phone'] or parameters['group_id'],+ "message_type": message_type,+ "details": result.get('result', {})+ }+ else:+ return {"error": result.get('error', 'Unknown error')}++ except Exception as e:+ logger.error(f"[WhatsApp Tool] Error: {e}")+ return {"error": f"WhatsApp send failed: {str(e)}"}+```++### WhatsApp DB Node (Multi-Operation Pattern)++The `whatsappDb` node demonstrates a multi-operation pattern with 6 different operations in a single dual-purpose node.++**Frontend Definition:**+```typescript+whatsappDb: {+ displayName: 'WhatsApp DB',+ name: 'whatsappDb',+ icon: DATABASE_ICON,+ group: ['whatsapp', 'tool'],+ version: 1,+ description: 'Query WhatsApp database - contacts, groups, messages',++ inputs: [{ name: 'main', type: 'main' }],+ outputs: [+ { name: 'main', type: 'main', description: 'Query result' },+ { name: 'tool', type: 'main', description: 'Connect to AI Agent tool handle' }+ ],++ properties: [+ {+ displayName: 'Operation',+ name: 'operation',+ type: 'options',+ default: 'chat_history',+ options: [+ { name: 'Chat History', value: 'chat_history' },+ { name: 'Search Groups', value: 'search_groups' },+ { name: 'Get Group Info', value: 'get_group_info' },+ { name: 'Get Contact Info', value: 'get_contact_info' },+ { name: 'List Contacts', value: 'list_contacts' },+ { name: 'Check Contacts', value: 'check_contacts' },+ ],+ },+ // Conditional parameters per operation using displayOptions...+ ]+}+```++**Backend Schema (ai.py):**+```python+if node_type == 'whatsappDb':+ class WhatsAppDbSchema(BaseModel):+ """Query WhatsApp database - contacts, groups, messages."""+ operation: str = Field(+ default="chat_history",+ description="Operation: 'chat_history', 'search_groups', 'get_group_info', 'get_contact_info', 'list_contacts', 'check_contacts'"+ )+ # chat_history params+ chat_type: Optional[str] = Field(default="individual", description="'individual' or 'group'")+ phone: Optional[str] = Field(default=None, description="Phone number for individual chat")+ group_id: Optional[str] = Field(default=None, description="Group JID for group operations")+ limit: Optional[int] = Field(default=50, description="Max messages (1-500)")+ offset: Optional[int] = Field(default=0, description="Pagination offset")+ message_filter: Optional[str] = Field(default="all", description="'all' or 'text_only'")+ # contact params+ query: Optional[str] = Field(default=None, description="Search query for contacts/groups")+ phones: Optional[str] = Field(default=None, description="Comma-separated phones for check_contacts")+ return WhatsAppDbSchema+```++**Backend Handler (tools.py):**+```python+async def _execute_whatsapp_db(args: Dict[str, Any], node_params: Dict[str, Any]) -> Dict[str, Any]:+ """Execute WhatsApp DB query operations."""+ from services.handlers.whatsapp import handle_whatsapp_db++ operation = args.get('operation', 'chat_history')++ # Map LLM args to node parameters+ parameters = {+ 'operation': operation,+ 'chatType': args.get('chat_type', 'individual'),+ 'phone': args.get('phone', ''),+ 'group_id': args.get('group_id', ''),+ 'limit': args.get('limit', 50),+ 'offset': args.get('offset', 0),+ 'messageFilter': args.get('message_filter', 'all'),+ 'query': args.get('query', ''),+ 'phones': args.get('phones', ''),+ 'groupIdForInfo': args.get('group_id', ''),+ 'contactPhone': args.get('phone', ''),+ }++ result = await handle_whatsapp_db(+ node_id="tool_whatsapp_db",+ node_type="whatsappDb",+ parameters=parameters,+ context={}+ )++ if result.get('success'):+ return {"success": True, "operation": operation, **result.get('result', {})}+ return {"error": result.get('error', 'Query failed')}+```++### Skill Documentation (SKILL.md)++Update the skill documentation to reflect tool capabilities:++```markdown+---+name: whatsapp-skill+description: Send WhatsApp messages, query contacts/groups, retrieve chat history.+allowed-tools: whatsapp_send whatsapp_db+metadata:+ author: machina+ version: "3.0"+ category: messaging+---++# WhatsApp Messaging Skill++## whatsapp_send++Send messages to contacts or groups.++### Parameters++| Parameter | Type | Required | Description |+|-----------|------|----------|-------------|+| recipient_type | string | Yes | "phone" or "group" |+| phone | string | If phone | Phone number without + prefix |+| group_id | string | If group | Group JID |+| message_type | string | Yes | "text", "image", "video", etc. |+| message | string | If text | Text content |+| media_url | string | If media | URL for media |++## whatsapp_db++Query WhatsApp database - contacts, groups, messages.++### Operations++| Operation | Description |+|-----------|-------------|+| chat_history | Retrieve message history |+| search_groups | Search groups by name |+| get_group_info | Get group details with participant names |+| get_contact_info | Get contact info (name, phone, photo) |+| list_contacts | List all contacts with saved names |+| check_contacts | Check WhatsApp registration status |++### Examples++**Get chat history:**+```json+{+ "operation": "chat_history",+ "chat_type": "individual",+ "phone": "1234567890",+ "limit": 20+}+```++**Find contact by name:**+```json+{+ "operation": "list_contacts",+ "query": "mom"+}+```++**Get group participants:**+```json+{+ "operation": "get_group_info",+ "group_id": "[email protected]"+}+```+```++---++## Checklist++### Frontend++- [ ] Add node definition with both `main` input AND outputs (`main` + `tool`)+- [ ] Include `'tool'` in the `group` array+- [ ] Define comprehensive `properties` (LLM will fill these)+- [ ] Ensure node renders correctly (SquareNode handles multiple outputs)++### Backend Schema (ai.py)++- [ ] Create Pydantic schema matching node properties+- [ ] Use snake_case for field names+- [ ] Add clear descriptions with examples+- [ ] Use `Optional` for non-required fields+- [ ] Add schema to `_get_tool_schema()` function++### Backend Handler (tools.py)++- [ ] Create `_execute_*` handler function+- [ ] Map LLM args (snake_case) to node params (camelCase)+- [ ] Add validation for required fields based on options+- [ ] Call existing node handler (reuse workflow logic)+- [ ] Return structured result or error+- [ ] Add dispatcher case in `execute_tool()`++### DO NOT Do++- [ ] Do NOT add to `AI_TOOL_TYPES` in constants.py+- [ ] Do NOT add to `CONFIG_NODE_TYPES`++### Documentation++- [ ] Update skill SKILL.md with tool parameters and examples+- [ ] Update CLAUDE.md with new tool capabilities++### Testing++- [ ] Verify node works in normal workflow+- [ ] Connect node to AI Agent's `input-tools` handle+- [ ] Verify LLM can call the tool with correct parameters+- [ ] Verify node is excluded from parallel execution when used as tool+- [ ] Test all parameter combinations++---++## Best Practices++1. **Reuse Existing Handlers**: The tool handler should call the existing node handler, not duplicate logic++2. **Parameter Mapping**: Always map LLM snake_case to node camelCase consistently++3. **Validation**: Validate required parameters based on conditional logic (e.g., phone required if recipient_type='phone')++4. **Error Messages**: Return clear error messages the LLM can understand and act on++5. **Schema Descriptions**: Write detailed Field descriptions with examples for the LLM++6. **Testing Both Modes**: Always test the node in both workflow mode and tool modediff --git a/docs-internal/node_creation.md b/docs-internal/node_creation.mdnew file mode 100644index 00000000..3167a378--- /dev/null+++ b/docs-internal/node_creation.md@@ -0,0 +1,1574 @@+# Node Creation Guide++This guide explains how to create new nodes for MachinaOs, covering frontend definitions, backend handlers, and the complete integration flow.++> **Related Documentation:**+> - [Workflow Schema](./workflow-schema.md) - JSON schema, edge handle conventions, config node architecture+> - [CLAUDE.md](../CLAUDE.md) - Project overview, key files, architecture patterns++## Table of Contents++1. [Node Types Overview](#node-types-overview)+2. [Frontend: Node Definition](#frontend-node-definition)+3. [Frontend: Visual Component](#frontend-visual-component)+4. [Backend: Workflow Handler](#backend-workflow-handler)+5. [Template Resolution](#template-resolution)+6. [Workflow Context and Status Broadcasting](#workflow-context-and-status-broadcasting)+7. [Input & Output Panels](#input--output-panels)+8. [Nodes Without Output Handles](#nodes-without-output-handles)+9. [Config Nodes (Memory, Tools)](#config-nodes)+10. [Trigger Nodes](#trigger-nodes)+11. [AI Provider Nodes](#ai-provider-nodes)+12. [AI Tool Nodes](#ai-tool-nodes)+13. [Complete Examples](#complete-examples)++---++## Node Types Overview++MachinaOs supports 58 nodes across 15 categories:++| Category | Visual Component | Example Nodes | Count |+|----------|-----------------|---------------|-------|+| **Workflow** | StartNode | start | 1 |+| **Scheduler** | SquareNode | timer, cronScheduler | 2 |+| **AI Chat Models** | SquareNode | openaiChatModel, anthropicChatModel, geminiChatModel, openrouterChatModel, groqChatModel, cerebrasChatModel | 6 |+| **AI Agents** | AIAgentNode | aiAgent, chatAgent | 2 |+| **AI Memory** | SquareNode | simpleMemory | 1 |+| **AI Skills** | SquareNode | claudeSkill, whatsappSkill, memorySkill, mapsSkill, httpSkill, schedulerSkill, androidSkill, codeSkill, customSkill | 9 |+| **AI Tools** | SquareNode | calculatorTool, currentTimeTool, webSearchTool, androidTool | 4 |+| **Triggers** | TriggerNode | whatsappReceive, webhookTrigger, chatTrigger | 3 |+| **Android** | SquareNode | batteryMonitor, wifiAutomation, location, cameraControl (16 services + 1 setup) | 17 |+| **WhatsApp** | SquareNode | whatsappSend, whatsappConnect, whatsappReceive, whatsappDb | 4 |+| **Location/Maps** | SquareNode | createMap, addLocations, showNearbyPlaces | 3 |+| **Utility** | SquareNode | httpRequest, webhookResponse, console | 3 |+| **Code** | SquareNode | pythonExecutor, javascriptExecutor | 2 |+| **Chat** | SquareNode | chatSend, chatHistory | 2 |+| **Document** | SquareNode | httpScraper, fileDownloader, documentParser, textChunker, embeddingGenerator, vectorStore | 6 |++---++## Frontend: Node Definition++Node definitions are located in `client/src/nodeDefinitions/`. Create your definition in the appropriate file or create a new category file.++### Basic Structure++```typescript+// client/src/nodeDefinitions/yourNodes.ts+import { INodeTypeDescription, NodeConnectionType } from '../types/INodeProperties';++export const yourNodes: Record<string, INodeTypeDescription> = {+ yourNodeName: {+ // === Required Fields ===+ displayName: 'Your Node Name', // Shown in UI+ name: 'yourNodeName', // Internal identifier (camelCase)+ icon: '🔧', // Emoji or image URL+ group: ['category'], // For palette grouping+ version: 1,+ description: 'What this node does',++ // === Visual Defaults ===+ defaults: {+ name: 'Your Node',+ color: '#FF5722' // Hex color for node accent+ },++ // === Connections ===+ inputs: [{+ name: 'main',+ displayName: 'Input',+ type: 'main' as NodeConnectionType,+ description: 'Node input'+ }],+ outputs: [{+ name: 'main',+ displayName: 'Output',+ type: 'main' as NodeConnectionType,+ description: 'result, status, timestamp' // Document output fields+ }],++ // === Parameters ===+ properties: [+ // See Parameter Types section below+ ]+ }+};+```++### Parameter Types++```typescript+properties: [+ // String input+ {+ displayName: 'API Key',+ name: 'apiKey',+ type: 'string',+ default: '',+ required: true,+ description: 'Your API key',+ placeholder: 'Enter API key...',+ typeOptions: {+ password: true, // Masked input+ rows: 4 // Multiline textarea+ }+ },++ // Number input+ {+ displayName: 'Timeout',+ name: 'timeout',+ type: 'number',+ default: 30,+ typeOptions: {+ minValue: 1,+ maxValue: 300,+ numberStepSize: 1+ }+ },++ // Boolean toggle+ {+ displayName: 'Enable Feature',+ name: 'enableFeature',+ type: 'boolean',+ default: false+ },++ // Dropdown select+ {+ displayName: 'Method',+ name: 'method',+ type: 'options',+ default: 'GET',+ options: [+ { name: 'GET', value: 'GET' },+ { name: 'POST', value: 'POST' },+ { name: 'PUT', value: 'PUT' },+ { name: 'DELETE', value: 'DELETE' }+ ]+ },++ // Conditional display+ {+ displayName: 'Request Body',+ name: 'body',+ type: 'string',+ default: '',+ typeOptions: { rows: 6 },+ displayOptions: {+ show: {+ method: ['POST', 'PUT'] // Only show when method is POST or PUT+ }+ }+ },++ // Collection (grouped options)+ {+ displayName: 'Options',+ name: 'options',+ type: 'collection',+ placeholder: 'Add Option',+ default: {},+ options: [+ {+ displayName: 'Headers',+ name: 'headers',+ type: 'string',+ default: ''+ }+ ]+ },++ // Dynamic options (loaded from backend)+ {+ displayName: 'Model',+ name: 'model',+ type: 'string',+ default: '',+ typeOptions: {+ dynamicOptions: true,+ dependsOn: ['provider'] // Reload when provider changes+ }+ }+]+```++### Register the Node++Add to the main registry in `client/src/nodeDefinitions.ts`:++```typescript+import { yourNodes } from './nodeDefinitions/yourNodes';++export const nodeDefinitions: Record<string, INodeTypeDescription> = {+ ...existingNodes,+ ...yourNodes,+};+```++---++## Frontend: Visual Component++### Map to Component Type++In `client/src/Dashboard.tsx`, map your node to a visual component:++```typescript+const createNodeTypes = () => {+ const types: NodeTypes = {};++ Object.entries(nodeDefinitions).forEach(([type, def]) => {+ if (type === 'yourNodeName') {+ types[type] = SquareNode; // Standard square node+ } else if (type === 'yourModelNode') {+ types[type] = ModelNode; // Circular model node+ } else if (type === 'aiAgent') {+ types[type] = AIAgentNode; // Special AI agent layout+ }+ // ... etc+ });++ return types;+};+```++### Available Visual Components++| Component | Shape | Use Case |+|-----------|-------|----------|+| `SquareNode` | 80x80px square | Standard nodes, triggers |+| `ModelNode` | 80x80px circle | AI models, config nodes |+| `AIAgentNode` | Rectangle with handles | AI agents with tool/memory inputs |+| `WhatsAppNode` | Square with status | WhatsApp-specific features |++---++## Backend: Workflow Handler++### Add to Execution Service++First, add your node to supported types in `client/src/services/executionService.ts`:++```typescript+static isNodeTypeSupported(nodeType: string): boolean {+ const supportedTypes = [+ // ... existing types+ 'yourNodeName',+ ];+ return supportedTypes.includes(nodeType);+}+```++### Create Backend Handler++In `server/services/workflow.py`, add a handler in `_execute_single_node()`:++```python+elif node_type == 'yourNodeName':+ result = await self._execute_your_node(node_id, resolved_parameters)+```++Then implement the execution method:++```python+async def _execute_your_node(self, node_id: str, parameters: Dict[str, Any]) -> Dict[str, Any]:+ """Execute your custom node."""+ start_time = time.time()++ try:+ # Extract parameters+ api_key = parameters.get('apiKey', '')+ timeout = int(parameters.get('timeout', 30))+ method = parameters.get('method', 'GET')++ # Your node logic here+ result_data = {+ "status": "success",+ "data": "your result",+ "timestamp": datetime.now().isoformat()+ }++ return {+ "success": True,+ "node_id": node_id,+ "node_type": "yourNodeName",+ "result": result_data,+ "execution_time": time.time() - start_time+ }++ except Exception as e:+ logger.error(f"Your node execution failed: {e}")+ return {+ "success": False,+ "node_id": node_id,+ "node_type": "yourNodeName",+ "error": str(e),+ "execution_time": time.time() - start_time+ }+```++---++## Template Resolution++Template resolution allows nodes to reference outputs from other nodes using `{{nodeName.fieldPath}}` syntax.++### How It Works++1. **ParameterResolver** (`server/services/parameter_resolver.py`) gathers outputs from ALL nodes in the workflow+2. Template patterns `{{nodeName.field}}` are replaced with actual values+3. Resolution happens BEFORE the node handler executes++### Template Syntax++```+{{nodeName.field}} # Simple field access+{{nodeName.nested.field}} # Nested field access+{{nodeName.array[0]}} # Array indexing+{{nodeName.data.items[0].name}} # Complex path+```++### Node Name Resolution++Node names are normalized for template matching:+- `aiAgent-1234567890` → `aiagent`+- `WhatsApp Chat History` (custom label) → `whatsappchathistory`+- Spaces and special characters are removed, converted to lowercase++### Example Usage++```typescript+// In node definition - parameter that accepts templates+{+ displayName: 'Message',+ name: 'message',+ type: 'string',+ default: '',+ description: 'Supports templates: {{aiagent.response}}'+}+```++### Backend Resolution Flow++```+NodeExecutor.execute()+ ↓+ParameterResolver.resolve(params, node_id, nodes, edges, session_id)+ ↓+_gather_all_outputs() // Collects outputs from ALL nodes+ ↓+_resolve_value() // Replaces {{templates}} with actual values+ ↓+Handler receives resolved parameters+```++### Important Notes++- Templates are resolved by ParameterResolver, not by individual handlers+- If a template references a node that hasn't executed yet, it resolves to empty string+- Nodes that need raw template values (not resolved) should use a different parameter name++---++## Workflow Context and Status Broadcasting++When a node executes, it receives a context object and can broadcast status updates to the frontend. This is essential for real-time UI updates and per-workflow status scoping.++### Execution Context Structure++The backend passes a `context` dictionary to every node handler:++```python+context = {+ "nodes": List[Dict], # All nodes in the workflow+ "edges": List[Dict], # All edges in the workflow+ "session_id": str, # Execution session ID+ "execution_id": str, # Unique execution ID+ "workflow_id": str, # Current workflow ID (for status scoping)+ "get_output_fn": Callable, # Function to get node outputs+}+```++### Frontend: Passing Workflow ID++When executing a node from the frontend, include `workflow_id` in the request:++```typescript+// In WebSocketContext.tsx+const executeNodeAsync = useCallback(async (+ nodeId: string,+ nodeType: string,+ parameters: Record<string, any>,+ nodes?: any[],+ edges?: any[]+ ): Promise<any> => {+ const response = await sendRequest<any>('execute_node', {+ node_id: nodeId,+ node_type: nodeType,+ parameters,+ nodes,+ edges,+ workflow_id: currentWorkflowId // CRITICAL: Include for status scoping+ }, timeoutMs);+ return response;+ }, [sendRequest, currentWorkflowId]);+```++**Why This Matters:**+- Without `workflow_id`, status updates are stored under `'unknown'` key+- Frontend retrieves status from `allNodeStatuses[currentWorkflowId][nodeId]`+- Mismatched workflow IDs cause status updates to be "lost" (stored but not displayed)++### Backend: Status Broadcasting++Use the `StatusBroadcaster` to send real-time updates to the frontend:++```python+from services.status_broadcaster import get_status_broadcaster++async def handle_my_node(+ node_id: str,+ node_type: str,+ parameters: Dict[str, Any],+ context: Dict[str, Any]+) -> Dict[str, Any]:+ workflow_id = context.get('workflow_id') # Extract from context+ broadcaster = get_status_broadcaster()++ # Broadcast "executing" status+ await broadcaster.update_node_status(+ node_id,+ "executing",+ {"message": "Starting execution..."},+ workflow_id=workflow_id # CRITICAL: Include workflow_id+ )++ # Do your work...+ result = do_something()++ # Broadcast "success" status+ await broadcaster.update_node_status(+ node_id,+ "success",+ {"message": "Completed", "result": result},+ workflow_id=workflow_id+ )++ return {"success": True, "result": result, ...}+```++### Status Types++| Status | Description | Visual Effect |+|--------|-------------|---------------|+| `idle` | Node is not executing | Default appearance |+| `executing` | Node is currently running | Cyan glow, pulse animation |+| `waiting` | Trigger node waiting for event | Cyan glow, pulse animation |+| `success` | Node completed successfully | Green border (brief) |+| `error` | Node execution failed | Red border, error message |++### Frontend: Reading Status++Components read status via the `useNodeStatus` hook:++```typescript+import { useNodeStatus } from '../contexts/WebSocketContext';++const MyNodeComponent: React.FC<NodeProps> = ({ id }) => {+ const nodeStatus = useNodeStatus(id);+ const executionStatus = nodeStatus?.status || 'idle';+ const isExecuting = executionStatus === 'executing';++ return (+ <div style={{+ animation: isExecuting ? 'pulse 1.5s ease-in-out infinite' : 'none',+ boxShadow: isExecuting ? '0 0 10px cyan' : 'none'+ }}>+ {/* Node content */}+ </div>+ );+};+```++### Status Storage Architecture++```+Frontend Backend+──────── ───────+allNodeStatuses: { StatusBroadcaster.broadcast({+ "workflow-123": { type: "node_status",+ "node-abc": { node_id: "node-abc",+ status: "executing", workflow_id: "workflow-123",+ data: {...} data: {status: "executing", ...}+ } })+ }+}+ ↑ │+ │ WebSocket │+ └─────────────────────────────────────┘+```++### Example: Tool Node Status Broadcasting++Tool nodes (executed by AI Agent) need proper status flow:++```python+# In ai.py - tool_executor callback+async def tool_executor(tool_name: str, tool_args: Dict) -> Any:+ config = tool_configs.get(tool_name, {})+ tool_node_id = config.get('node_id')++ # Broadcast "executing" to tool node+ if tool_node_id and broadcaster:+ await broadcaster.update_node_status(+ tool_node_id,+ "executing",+ {"message": f"Executing {tool_name}"},+ workflow_id=workflow_id # From execute_agent closure+ )++ # Execute tool...+ result = await execute_tool(tool_name, tool_args, config)++ # Broadcast "success" to tool node+ if tool_node_id and broadcaster:+ await broadcaster.update_node_status(+ tool_node_id,+ "success",+ {"message": f"{tool_name} completed", "result": result},+ workflow_id=workflow_id+ )++ return result+```++### Common Issues++| Issue | Cause | Fix |+|-------|-------|-----|+| Status not updating | Missing `workflow_id` in frontend request | Add `workflow_id: currentWorkflowId` to execute request |+| Status stuck on "executing" | Success broadcast missing | Add success broadcast after execution |+| Wrong workflow shows status | `workflow_id` mismatch | Ensure frontend sends and backend uses same ID |+| Multiple nodes show same status | Broadcasting to wrong `node_id` | Verify correct node ID in broadcast call |++### Checklist for Status Broadcasting++- [ ] Frontend: Include `workflow_id` in execution request+- [ ] Backend: Extract `workflow_id` from context+- [ ] Backend: Pass `workflow_id` to all `update_node_status()` calls+- [ ] Backend: Broadcast "executing" at start+- [ ] Backend: Broadcast "success" or "error" at end+- [ ] Frontend: Use `useNodeStatus(id)` hook to read status++---++## Input & Output Panels++The parameter panel shows inputs from connected upstream nodes and allows drag-and-drop mapping.++### Input Section (`client/src/components/parameterPanel/InputSection.tsx`)++Shows outputs from nodes connected to the current node's input handles:+- Displays source node label and type+- Shows available output fields that can be dragged to parameters+- Config nodes inherit their parent's inputs (labeled "via Parent")++### Output Panel (`client/src/components/OutputPanel.tsx`)++Shows the schema of outputs the node produces:+- Used for drag-and-drop to downstream node parameters+- Schema defined in `outputs[].description` of node definition++### Documenting Output Schema++In your node definition, document output fields in the `description`:++```typescript+outputs: [{+ name: 'main',+ displayName: 'Output',+ type: 'main' as NodeConnectionType,+ description: 'response, model, provider, timestamp' // Comma-separated fields+}]+```++### Sample Schemas for Common Nodes++Add sample schemas in `InputSection.tsx` for better drag-and-drop UX:++```typescript+const sampleSchemas: Record<string, Record<string, string>> = {+ aiAgent: {+ response: 'string',+ model: 'string',+ provider: 'string',+ iterations: 'number',+ timestamp: 'string'+ },+ httpRequest: {+ status: 'number',+ data: 'object',+ headers: 'object'+ },+ yourNewNode: {+ field1: 'string',+ field2: 'number',+ // ... document your output fields+ }+};+```++---++## Nodes Without Output Handles++Some nodes are input-only (like Console) and should not have output handles.++### Frontend: Hide Output Handle++In `client/src/components/SquareNode.tsx`, add your node type to `NO_OUTPUT_NODE_TYPES`:++```typescript+// Nodes that should not have output handles (input-only nodes)+const NO_OUTPUT_NODE_TYPES = ['console', 'yourInputOnlyNode'];+```++The output handle is conditionally rendered:++```tsx+{/* Square Output Handle - hidden for input-only nodes */}+{!NO_OUTPUT_NODE_TYPES.includes(type || '') && (+ <Handle+ id="output-main"+ type="source"+ position={Position.Right}+ // ...+ />+)}+```++### Backend: Special Handler for Connected Outputs++Input-only nodes typically need access to connected upstream outputs. Register them in `NodeExecutor._dispatch()`:++```python+# server/services/node_executor.py++async def _dispatch(self, node_id: str, node_type: str, params: Dict, context: Dict) -> Dict:+ # Check registry first+ handler = self._handlers.get(node_type)+ if handler:+ return await handler(node_id, node_type, params, context)++ # Special handlers needing connected outputs+ if node_type in ('pythonExecutor', 'javascriptExecutor', 'webhookResponse', 'console', 'yourNode'):+ outputs, source_nodes = await self._get_connected_outputs_with_info(context, node_id)+ if node_type == 'yourNode':+ return await handle_your_node(node_id, node_type, params, context, outputs, source_nodes)+ # ... other handlers+```++### Handler Signature for Connected Outputs++```python+async def handle_your_node(+ node_id: str,+ node_type: str,+ parameters: Dict[str, Any],+ context: Dict[str, Any],+ connected_outputs: Dict[str, Any] = None, # Keyed by source node type+ source_nodes: list = None # List of {id, type, label}+) -> Dict[str, Any]:+ """Handle node with access to connected upstream outputs."""+ connected_outputs = connected_outputs or {}+ source_nodes = source_nodes or []++ # Merge all connected outputs into single dict+ input_data = {}+ for node_type, output in connected_outputs.items():+ if isinstance(output, dict):+ input_data.update(output)++ # Your logic here using input_data+ # ...+```++---++## Config Nodes++Config nodes (memory, tools) connect to parent nodes via special handles and inherit the parent's inputs.++### Define a Config Node++```typescript+// group must include 'memory' or 'tool'+myConfigNode: {+ displayName: 'My Config',+ name: 'myConfigNode',+ icon: '🔧',+ group: ['ai', 'memory'], // 'memory' or 'tool' = config node+ version: 1,+ description: 'Configuration node',+ defaults: { name: 'Config', color: '#8b5cf6' },++ // Config nodes typically have no main input+ inputs: [],++ // Output connects to parent's config handle+ outputs: [{+ name: 'config',+ displayName: 'Config Output',+ type: 'main' as NodeConnectionType,+ description: 'Configuration data'+ }],++ properties: [+ // Config-specific parameters+ ]+}+```++### Parent Node with Config Handles++Config handles follow the pattern `input-<type>`:++```tsx+// In your parent node component (e.g., AIAgentNode.tsx)+<Handle+ id="input-memory" // Config handle naming+ type="target"+ position={Position.Bottom}+ // ... styling+/>++<Handle+ id="input-tools" // Another config handle+ type="target"+ position={Position.Bottom}+ // ... styling+/>+```++### Input Inheritance++When a config node is connected to a parent's config handle:+1. The parent node does NOT see the config node as an input+2. The config node DOES see the parent's main inputs (labeled "via Parent")++This is handled automatically by `InputSection.tsx` and `OutputPanel.tsx` using:+- `isConfigHandle()` - Detects config handles by pattern+- `isConfigNode()` - Detects config nodes by group++---++## Trigger Nodes++Trigger nodes wait for external events before completing execution.++### Define a Trigger Node++```typescript+myTrigger: {+ displayName: 'My Trigger',+ name: 'myTrigger',+ icon: '⚡',+ group: ['trigger'],+ version: 1,+ description: 'Waits for an event',+ defaults: { name: 'Trigger', color: '#f59e0b' },++ // Triggers typically have no input (they start workflows)+ inputs: [],++ outputs: [{+ name: 'main',+ displayName: 'Event',+ type: 'main' as NodeConnectionType,+ description: 'event_type, data, timestamp'+ }],++ properties: [+ // Filter parameters+ {+ displayName: 'Event Filter',+ name: 'eventFilter',+ type: 'string',+ default: '*',+ description: 'Filter pattern for events'+ }+ ]+}+```++### Register Trigger in Backend++In `server/services/event_waiter.py`:++```python+TRIGGER_REGISTRY: Dict[str, TriggerConfig] = {+ 'myTrigger': TriggerConfig(+ node_type='myTrigger',+ event_type='my_event_received', # Event to listen for+ display_name='My Event'+ ),+}++# Add filter builder+def build_my_filter(params: Dict) -> Callable[[Dict], bool]:+ """Build filter for my events."""+ event_filter = params.get('eventFilter', '*')++ def filter_fn(event_data: Dict) -> bool:+ if event_filter == '*':+ return True+ return event_data.get('type') == event_filter++ return filter_fn++FILTER_BUILDERS['myTrigger'] = build_my_filter+```++### Dispatch Events++From your event source (webhook, WebSocket, etc.):++```python+from services import event_waiter++# When event occurs+event_waiter.dispatch('my_event_received', {+ 'type': 'some_type',+ 'data': {...},+ 'timestamp': datetime.now().isoformat()+})+```++---++## AI Provider Nodes++AI provider nodes use a factory pattern for consistent structure. They connect to the AI service backend and dynamically fetch available models.++### Factory Pattern++AI chat models use `createBaseChatModel()` from `src/factories/baseChatModelFactory.ts`:++```typescript+import { createBaseChatModel, ChatModelConfig, STANDARD_PARAMETERS } from '../factories/baseChatModelFactory';++const myProviderConfig: ChatModelConfig = {+ providerId: 'myprovider', // Internal identifier+ displayName: 'My Provider', // Shown in UI+ icon: '🤖', // Emoji icon+ color: '#6366F1', // Accent color+ description: 'Provider description',+ models: [], // Empty - fetched dynamically+ parameters: [+ STANDARD_PARAMETERS.temperature,+ STANDARD_PARAMETERS.maxTokens,+ STANDARD_PARAMETERS.topP,+ // Provider-specific parameters...+ ]+};++export const aiModelNodes = {+ myProviderChatModel: createBaseChatModel(myProviderConfig)+};+```++### Standard Parameters++Available in `STANDARD_PARAMETERS`:++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `temperature` | number | 0.7 | Randomness (0-2) |+| `maxTokens` | number | 1000 | Response length limit |+| `topP` | number | 1 | Nucleus sampling |+| `topK` | number | 40 | Top-K sampling |+| `frequencyPenalty` | number | 0 | Token frequency penalty |+| `presencePenalty` | number | 0 | Token presence penalty |++### Backend Provider Configuration++Add to `server/services/ai.py`:++```python+from dataclasses import dataclass+from langchain_openai import ChatOpenAI++@dataclass+class ProviderConfig:+ name: str+ model_class: type+ api_key_param: str+ max_tokens_param: str+ detection_patterns: tuple+ default_model: str+ models_endpoint: str+ models_header_fn: Callable++# Header function for API authentication+def _myprovider_headers(api_key: str) -> dict:+ return {'Authorization': f'Bearer {api_key}'}++PROVIDER_CONFIGS = {+ 'myprovider': ProviderConfig(+ name='myprovider',+ model_class=ChatOpenAI,+ api_key_param='api_key',+ max_tokens_param='max_tokens',+ detection_patterns=('myprovider',),+ default_model='default-model-id',+ models_endpoint='https://api.provider.com/models',+ models_header_fn=_myprovider_headers+ ),+}+```++### Model Fetching++Add model fetching logic in `AIService.fetch_models()`:++```python+elif provider == 'myprovider':+ response = await client.get(+ 'https://api.provider.com/models',+ headers=_myprovider_headers(api_key)+ )+ response.raise_for_status()+ data = response.json()++ models = []+ for model in data.get('data', []):+ model_id = model.get('id', '')+ if model_id:+ models.append(model_id)++ return sorted(models)+```++### Provider Detection++Add to `server/constants.py`:++```python+def detect_ai_provider(node_type: str, parameters: dict = None) -> str:+ if 'myprovider' in node_type.lower():+ return 'myprovider'+ # ... other providers+ return 'openai' # Default fallback+```++### Frontend Integration++1. **Dashboard.tsx** - Map to visual component:+ ```typescript+ } else if (type === 'myProviderChatModel') {+ types[type] = SquareNode;+ }+ ```++2. **ModelNode.tsx** - Credential mapping:+ ```typescript+ const CREDENTIAL_TO_PROVIDER = {+ 'myProviderApi': 'myprovider'+ };+ ```++3. **CredentialsModal.tsx** - API key entry:+ ```typescript+ {+ id: 'myprovider',+ name: 'My Provider',+ icon: '🤖',+ fields: [{ name: 'apiKey', label: 'API Key', type: 'password', required: true }]+ }+ ```++### Complete Example: Adding Groq Provider++Groq provides ultra-fast LLM inference. Here's the complete implementation:++#### 1. Frontend Node Definition (`client/src/nodeDefinitions/aiModelNodes.ts`)++```typescript+import { createBaseChatModel, ChatModelConfig, STANDARD_PARAMETERS } from '../factories/baseChatModelFactory';++const groqConfig: ChatModelConfig = {+ providerId: 'groq',+ displayName: 'Groq',+ icon: '⚡',+ color: '#F55036',+ description: 'Groq - Ultra-fast LLM inference with Llama, Mixtral, and Gemma models',+ models: [], // Fetched dynamically via API+ parameters: [+ STANDARD_PARAMETERS.temperature,+ STANDARD_PARAMETERS.maxTokens,+ STANDARD_PARAMETERS.topP,+ {+ displayName: 'Timeout',+ name: 'timeout',+ type: 'number',+ default: 60000,+ typeOptions: { minValue: 1000, maxValue: 180000 },+ description: 'Timeout for the request in milliseconds'+ },+ {+ displayName: 'Max Retries',+ name: 'maxRetries',+ type: 'number',+ default: 2,+ typeOptions: { minValue: 0, maxValue: 5 },+ description: 'Maximum number of retries'+ }+ ]+};++export const aiModelNodes = {+ // ... other providers+ groqChatModel: createBaseChatModel(groqConfig)+};++export { groqConfig };+```++#### 2. Backend Provider Config (`server/services/ai.py`)++```python+from langchain_groq import ChatGroq++def _groq_headers(api_key: str) -> dict:+ return {'Authorization': f'Bearer {api_key}'}++PROVIDER_CONFIGS = {+ # ... other providers+ 'groq': ProviderConfig(+ name='groq',+ model_class=ChatGroq,+ api_key_param='api_key',+ max_tokens_param='max_tokens',+ detection_patterns=('groq', 'llama', 'mixtral', 'gemma'),+ default_model='llama-3.3-70b-versatile',+ models_endpoint='https://api.groq.com/openai/v1/models',+ models_header_fn=_groq_headers+ ),+}+```++#### 3. Model Fetching (`server/services/ai.py` - in `fetch_models` method)++```python+elif provider == 'groq':+ response = await client.get(+ 'https://api.groq.com/openai/v1/models',+ headers={'Authorization': f'Bearer {api_key}'}+ )+ response.raise_for_status()+ data = response.json()++ models = []+ for model in data.get('data', []):+ model_id = model.get('id', '')+ if model_id:+ models.append(model_id)++ return sorted(models)+```++#### 4. Chat Execution (`server/services/ai.py` - in `execute_chat` method)++```python+# Determine provider from node_type+if node_type == 'groqChatModel':+ provider = 'groq'+else:+ provider = self.detect_provider(model)+```++#### 5. Backend Constants (`server/constants.py`)++```python+AI_CHAT_MODEL_TYPES: FrozenSet[str] = frozenset([+ 'openaiChatModel',+ 'anthropicChatModel',+ 'geminiChatModel',+ 'openrouterChatModel',+ 'groqChatModel', # Add here+])++def detect_ai_provider(node_type: str, parameters: dict = None) -> str:+ # ... existing checks+ elif 'groq' in node_type.lower():+ return 'groq'+ # ... rest of function+```++#### 6. Frontend Provider Mapping (`client/src/components/ParameterRenderer.tsx`)++```typescript+const NODE_TYPE_TO_PROVIDER: Record<string, string> = {+ 'openaiChatModel': 'openai',+ 'anthropicChatModel': 'anthropic',+ // ... other providers+ 'groqChatModel': 'groq' // Add here+};+```++#### 7. Execution Service (`client/src/services/executionService.ts`)++```typescript+static isNodeTypeSupported(nodeType: string): boolean {+ const supportedTypes = [+ // AI Nodes+ 'aiAgent',+ 'openaiChatModel',+ 'anthropicChatModel',+ 'geminiChatModel',+ 'openrouterChatModel',+ 'groqChatModel', // Add here+ // ... other nodes+ ];+ return supportedTypes.includes(nodeType);+}+```++#### 8. Credentials Modal (`client/src/components/CredentialsModal.tsx`)++```typescript+const PROVIDERS = [+ // ... other providers+ {+ id: 'groq',+ name: 'Groq',+ placeholder: 'gsk_...',+ color: '#F55036',+ desc: 'Ultra-fast LLM inference - Llama, Mixtral, Gemma'+ },+];+```++#### 9. Model Node Credential Mapping (`client/src/components/ModelNode.tsx`)++```typescript+const CREDENTIAL_TO_PROVIDER: Record<string, string> = {+ // ... other mappings+ 'groqApi': 'groq'+};++// In provider detection fallback:+if (type?.includes('groq')) return 'groq';+```++#### 10. AI Provider Icons (`client/src/components/icons/AIProviderIcons.tsx`)++```typescript+import { Groq } from '@lobehub/icons';++// Groq uses Avatar variant (no Color variant available)+export const GroqIcon: React.FC<{ size?: number }> = ({ size = ICON_SIZE }) => (+ <Groq.Avatar size={size} />+);++export const AI_PROVIDER_ICONS: Record<string, React.FC<{ size?: number }>> = {+ // ... other icons+ groq: GroqIcon,+};+```++#### 11. Python Dependency (`server/requirements.txt`)++```+langchain-groq>=0.1.0+```++### OpenRouter Example++OpenRouter demonstrates advanced patterns including:+- OpenAI-compatible API with custom base_url+- Free/paid model grouping with `[FREE]` prefix+- Multi-provider model IDs (e.g., `openai/gpt-4o`)++See the [AI Chat Model Development Guide](../CLAUDE.md#ai-chat-model-development-guide) in CLAUDE.md for the complete OpenRouter implementation.++---++## AI Tool Nodes++AI Tool nodes provide capabilities that AI Agents can invoke during reasoning. They connect to the AI Agent's `input-tools` handle and define schemas that tell the LLM how to call them.++### Architecture++```+Tool Node (calculatorTool) → (tool output) → AI Agent (input-tools handle)+ ↓+ LangGraph builds tools from schemas+ ↓+ LLM decides when to call tools+ ↓+ Tool executor runs handler+ ↓+ Result returned to LLM+```++### Existing Tool Nodes++| Tool | Icon | Description |+|------|------|-------------|+| `calculatorTool` | `🔢` | Mathematical operations (add, subtract, multiply, divide, power, sqrt, mod, abs) |+| `currentTimeTool` | `🕐` | Get current date/time with timezone support |+| `webSearchTool` | `🔍` | Web search via DuckDuckGo (free) or Serper API |+| `androidTool` | `📱` | Android device control toolkit - aggregates connected Android service nodes |++### Define a Tool Node++Tool nodes are defined in `client/src/nodeDefinitions/toolNodes.ts`:++```typescript+import { INodeTypeDescription, NodeConnectionType } from '../types/INodeProperties';++export const toolNodes: Record<string, INodeTypeDescription> = {+ myCustomTool: {+ displayName: 'My Custom Tool',+ name: 'myCustomTool',+ icon: '🔧',+ group: ['ai', 'tool'], // 'tool' = AI tool node+ version: 1,+ subtitle: 'Custom Operations',+ description: 'Allow AI Agent to perform custom operations',+ defaults: { name: 'My Tool', color: '#50fa7b' },++ // Tool nodes typically have no main input (they're passive)+ inputs: [],++ // Output connects to AI Agent's input-tools handle+ outputs: [{+ name: 'tool',+ displayName: 'Tool',+ type: 'main' as NodeConnectionType,+ description: 'Connect to AI Agent tool handle'+ }],++ properties: [+ {+ displayName: 'Tool Name',+ name: 'toolName',+ type: 'string',+ default: 'my_custom_tool',+ required: true,+ description: 'Name the AI will use to call this tool'+ },+ {+ displayName: 'Description',+ name: 'toolDescription',+ type: 'string',+ default: 'Describe what this tool does for the AI',+ typeOptions: { rows: 2 },+ description: 'Describe the tool capabilities for the AI'+ },+ // Tool-specific parameters...+ ]+ }+};++// Add to TOOL_NODE_TYPES for identification+export const TOOL_NODE_TYPES = ['calculatorTool', 'currentTimeTool', 'webSearchTool', 'androidTool', 'myCustomTool'];+```++### Backend: Tool Schema++Add a Pydantic schema for your tool in `server/services/ai.py`:++```python+from pydantic import BaseModel, Field++class MyCustomToolSchema(BaseModel):+ """Schema for my custom tool."""+ operation: str = Field(description="The operation to perform")+ value: str = Field(description="The value to operate on")++def _get_tool_schema(tool_type: str) -> type:+ """Get Pydantic schema for tool type."""+ schemas = {+ 'calculatorTool': CalculatorSchema,+ 'currentTimeTool': CurrentTimeSchema,+ 'webSearchTool': WebSearchSchema,+ 'myCustomTool': MyCustomToolSchema, # Add here+ }+ return schemas.get(tool_type)+```++### Backend: Tool Handler++Add a handler in `server/services/handlers/tools.py`:++```python+async def _execute_my_custom_tool(params: Dict[str, Any]) -> str:+ """Execute my custom tool."""+ operation = params.get('operation', '')+ value = params.get('value', '')++ # Your tool logic here+ result = f"Performed {operation} on {value}"++ return result++async def execute_tool(tool_type: str, tool_params: Dict, llm_params: Dict) -> str:+ """Dispatch tool execution to appropriate handler."""+ if tool_type == 'calculatorTool':+ return await _execute_calculator(llm_params)+ elif tool_type == 'currentTimeTool':+ return await _execute_current_time(llm_params)+ elif tool_type == 'webSearchTool':+ return await _execute_web_search(llm_params, tool_params)+ elif tool_type == 'myCustomTool':+ return await _execute_my_custom_tool(llm_params) # Add here+ else:+ return f"Unknown tool type: {tool_type}"+```++### Android Toolkit Pattern++The `androidTool` demonstrates a special pattern for aggregating multiple nodes:++```+[Battery Monitor] --++ +--> [Android Toolkit] --> [AI Agent]+[WiFi Automation] --++```++Key features:+- **Gateway Pattern**: Single tool node aggregates multiple service nodes+- **Dynamic Schema**: Schema built at runtime from connected services only+- **Service Routing**: Tool execution routes to appropriate connected node++The AI sees a single `android_device` tool with schema showing only connected services:+- `service_id`: Which service to use (e.g., "battery", "wifi_automation")+- `action`: Action to perform (e.g., "status", "enable", "disable")+- `parameters`: Action-specific parameters++### Tool Execution Flow++1. **Tool Discovery**: AI Agent scans edges for nodes connected to `input-tools` handle+2. **Schema Building**: `_get_tool_schema()` creates Pydantic schema for each tool type+3. **Tool Binding**: LangGraph binds tools to the LLM model+4. **LLM Decision**: LLM decides when to call tools based on user query+5. **Status Broadcast**: `executing_tool` status broadcast with tool_name for UI animation+6. **Tool Execution**: `execute_tool()` dispatches to appropriate handler+7. **Result Return**: Tool result returned to LLM for continued reasoning++### Checklist for New Tool Nodes++**Frontend:**+- [ ] Add tool definition in `client/src/nodeDefinitions/toolNodes.ts`+- [ ] Add to `TOOL_NODE_TYPES` array in same file+- [ ] Register in `client/src/nodeDefinitions.ts` (if not already importing toolNodes)++**Backend:**+- [ ] Add Pydantic schema in `server/services/ai.py` `_get_tool_schema()`+- [ ] Add handler function in `server/services/handlers/tools.py`+- [ ] Add dispatcher case in `execute_tool()` function+- [ ] Add to `AI_TOOL_TYPES` in `server/constants.py`++---++## Complete Examples++### Example 1: HTTP Request Node++**Frontend Definition:**+```typescript+httpRequest: {+ displayName: 'HTTP Request',+ name: 'httpRequest',+ icon: '🌐',+ group: ['utility'],+ version: 1,+ description: 'Make HTTP requests to external APIs',+ defaults: { name: 'HTTP', color: '#3b82f6' },+ inputs: [{ name: 'main', displayName: 'Input', type: 'main' }],+ outputs: [{+ name: 'main',+ displayName: 'Response',+ type: 'main',+ description: 'status, data, headers'+ }],+ properties: [+ { displayName: 'URL', name: 'url', type: 'string', required: true },+ { displayName: 'Method', name: 'method', type: 'options', default: 'GET',+ options: [{ name: 'GET', value: 'GET' }, { name: 'POST', value: 'POST' }] },+ { displayName: 'Body', name: 'body', type: 'string', typeOptions: { rows: 4 },+ displayOptions: { show: { method: ['POST'] } } }+ ]+}+```++**Backend Handler:**+```python+async def _execute_http_request(self, node_id: str, parameters: Dict) -> Dict:+ import httpx++ url = parameters.get('url', '')+ method = parameters.get('method', 'GET')+ body = parameters.get('body', '')++ async with httpx.AsyncClient(timeout=30) as client:+ if method == 'GET':+ response = await client.get(url)+ else:+ response = await client.post(url, content=body)++ return {+ "success": True,+ "node_id": node_id,+ "node_type": "httpRequest",+ "result": {+ "status": response.status_code,+ "data": response.text,+ "headers": dict(response.headers)+ }+ }+```++### Example 2: Memory Config Node++The Simple Memory node uses markdown-based storage with optional vector DB archival.++**Frontend Definition:**+```typescript+simpleMemory: {+ displayName: 'Simple Memory',+ name: 'simpleMemory',+ icon: '🧠',+ group: ['skill', 'memory'], // 'skill' = appears in AI Skills category+ version: 1,+ description: 'Markdown-based conversation memory with optional vector DB for long-term retrieval',+ defaults: { name: 'Memory', color: '#8b5cf6' },+ inputs: [], // No input - passive config node+ outputs: [{+ name: 'memory',+ displayName: 'Memory',+ type: 'main',+ description: 'session_id, messages, message_count'+ }],+ properties: [+ { displayName: 'Session ID', name: 'sessionId', type: 'string', default: 'default' },+ { displayName: 'Window Size', name: 'windowSize', type: 'number', default: 10,+ typeOptions: { minValue: 1, maxValue: 100 },+ description: 'Number of message pairs to keep in short-term memory' },+ { displayName: 'Conversation History', name: 'memoryContent', type: 'string',+ default: '# Conversation History\n\n*No messages yet.*\n',+ typeOptions: { rows: 15, editor: 'code', editorLanguage: 'markdown' },+ description: 'Recent conversation history (editable)' },+ { displayName: 'Enable Long-Term Memory', name: 'longTermEnabled', type: 'boolean',+ default: false, description: 'Archive old messages to vector DB for semantic retrieval' },+ { displayName: 'Retrieval Count', name: 'retrievalCount', type: 'number', default: 3,+ typeOptions: { minValue: 1, maxValue: 10 },+ displayOptions: { show: { longTermEnabled: [true] } },+ description: 'Number of relevant memories to retrieve from long-term storage' }+ ]+}+```++**Markdown Format:**+```markdown+# Conversation History++### **Human** (2025-01-30 14:23:45)+What is the weather like today?++### **Assistant** (2025-01-30 14:23:48)+I don't have access to real-time weather data...+```++**Backend Flow (in AI Agent handler):**+The AI Agent collects memory_data from connected Simple Memory node:+```python+# In handlers/ai.py - handle_ai_agent()+if target_handle == 'input-memory' and source_node.get('type') == 'simpleMemory':+ memory_params = await database.get_node_parameters(source_node_id) or {}+ memory_data = {+ 'node_id': source_node_id,+ 'session_id': memory_params.get('sessionId', 'default'),+ 'window_size': int(memory_params.get('windowSize', 10)),+ 'memory_content': memory_params.get('memoryContent', '# Conversation History\n\n*No messages yet.*\n'),+ 'long_term_enabled': memory_params.get('longTermEnabled', False),+ 'retrieval_count': int(memory_params.get('retrievalCount', 3))+ }+```++**Memory Processing (in ai.py):**+```python+# Parse markdown to LangChain messages+history_messages = _parse_memory_markdown(memory_content)++# After AI response, append and trim+updated_content = _append_to_memory_markdown(updated_content, 'human', prompt)+updated_content = _append_to_memory_markdown(updated_content, 'ai', response_content)+updated_content, removed_texts = _trim_markdown_window(updated_content, window_size)++# Archive to vector DB if enabled+if removed_texts and memory_data.get('long_term_enabled'):+ store = _get_memory_vector_store(session_id) # InMemoryVectorStore + HuggingFaceEmbeddings+ store.add_texts(removed_texts)++# Save updated markdown to node parameters+await database.save_node_parameters(memory_node_id, {'memoryContent': updated_content})+```++---++## Checklist for New Nodes++### Frontend+- [ ] Create node definition in `client/src/nodeDefinitions/`+- [ ] Register in `client/src/nodeDefinitions.ts`+- [ ] Map to visual component in `Dashboard.tsx`+- [ ] Add to supported types in `executionService.ts`+- [ ] Add output schema in `InputSection.tsx` (for drag-and-drop UX)+- [ ] For input-only nodes: Add to `NO_OUTPUT_NODE_TYPES` in `SquareNode.tsx`++### Backend+- [ ] Add handler in `server/services/handlers/` (appropriate file)+- [ ] Export handler in `server/services/handlers/__init__.py`+- [ ] Register in `NodeExecutor._build_handler_registry()` OR add to special handlers in `_dispatch()`+- [ ] For nodes needing connected outputs: Use `_get_connected_outputs_with_info()` pattern+- [ ] For triggers: Register in `event_waiter.py` TRIGGER_REGISTRY+- [ ] For triggers: Add filter builder function++### Template Resolution+- [ ] If node uses template parameters: Templates are auto-resolved by ParameterResolver+- [ ] If node needs raw template values: Use different parameter name to avoid resolution+- [ ] Document template-compatible parameters in node description++### AI Provider Nodes (additional steps)++**Frontend:**+- [ ] Add ChatModelConfig in `client/src/nodeDefinitions/aiModelNodes.ts`+- [ ] Export via `createBaseChatModel()` factory+- [ ] Add to `NODE_TYPE_TO_PROVIDER` in `client/src/components/ParameterRenderer.tsx`+- [ ] Add to supported types in `client/src/services/executionService.ts`+- [ ] Add credential mapping in `client/src/components/ModelNode.tsx` (CREDENTIAL_TO_PROVIDER)+- [ ] Add provider fallback detection in `ModelNode.tsx` (type?.includes check)+- [ ] Add credentials entry in `client/src/components/CredentialsModal.tsx`+- [ ] Add provider icon in `client/src/components/icons/AIProviderIcons.tsx` (if using @lobehub/icons)++**Backend:**+- [ ] Add LangChain dependency in `server/requirements.txt` (e.g., `langchain-groq>=0.1.0`)+- [ ] Import model class in `server/services/ai.py` (e.g., `from langchain_groq import ChatGroq`)+- [ ] Add header function in `server/services/ai.py` (e.g., `_groq_headers()`)+- [ ] Add ProviderConfig in `PROVIDER_CONFIGS` dict+- [ ] Add model fetching case in `AIService.fetch_models()`+- [ ] Add node type check in `AIService.execute_chat()` for provider detection+- [ ] Add to `AI_CHAT_MODEL_TYPES` in `server/constants.py`+- [ ] Add provider detection in `detect_ai_provider()` in `server/constants.py`++### Documentation+- [ ] Add to node count in `CLAUDE.md`+- [ ] Document in appropriate section of `CLAUDE.md`+- [ ] For AI providers: Update AI Chat Models section+- [ ] Add to workflow-schema.md supported node typesdiff --git a/docs-internal/server-readme.md b/docs-internal/server-readme.mdnew file mode 100644index 00000000..bf4e4ef2--- /dev/null+++ b/docs-internal/server-readme.md@@ -0,0 +1,73 @@+# Python Backend - Environment Setup++## Quick Start++1. Copy the template to create your .env file:+```bash+cp .env.template .env+```++2. Edit `.env` and add your API keys:+```bash+# Required for remote Android devices+WEBSOCKET_API_KEY=your-actual-key-here++# Optional: Default API keys (users can also add these through the UI)+GOOGLE_MAPS_API_KEY=your-google-maps-key+OPENAI_API_KEY=your-openai-key+```++3. Start the server:+```bash+python main.py+```++## Environment Variables++### Required++- `SECRET_KEY` - Server encryption key (32+ characters)+- `API_KEY_ENCRYPTION_KEY` - API key encryption (32+ characters)++### Optional but Recommended++- `WEBSOCKET_API_KEY` - Required for remote Android device connections+- `GOOGLE_MAPS_API_KEY` - Default Google Maps API key+- `OPENAI_API_KEY` - Default OpenAI API key+- `ANTHROPIC_API_KEY` - Default Anthropic API key+- `GOOGLE_AI_API_KEY` - Default Google AI API key++### Service URLs++- `WEBSOCKET_URL` - WebSocket server URL (default: ws://www.disutopia.xyz/ws)+- `WHATSAPP_SERVICE_URL` - WhatsApp service URL (default: http://localhost:3012)++### Development vs Production++The `.env.template` file contains development-safe defaults.++For production:+1. Change `DEBUG=false`+2. Generate new secure `SECRET_KEY` and `API_KEY_ENCRYPTION_KEY`+3. Set `RATE_LIMIT_ENABLED=true`+4. Configure proper CORS origins+5. Consider enabling Redis caching++## Security Notes++**NEVER commit `.env` files to git!**++The `.env.template` file is safe to commit because it contains no real credentials.++All API keys should be:+- Stored in `.env` files (ignored by git)+- Or set as environment variables+- Or managed through the UI (stored encrypted in database)++## Files++- `.env` - Your local configuration (git-ignored, created from template)+- `.env.template` - Template with safe defaults (committed to git)+- `.env.example` - Full example with all options (committed to git)+- `.env.development` - Development environment (git-ignored)+- `.env.production` - Production environment (git-ignored)diff --git a/docs-internal/specialized_agent_node_creation.md b/docs-internal/specialized_agent_node_creation.mdnew file mode 100644index 00000000..63ae49e2--- /dev/null+++ b/docs-internal/specialized_agent_node_creation.md@@ -0,0 +1,530 @@+# Specialized AI Agent Node Creation Guide++## Overview++Specialized Agent nodes are AI Agents pre-configured for specific domains and use cases. They inherit full AI Agent functionality (provider selection, model configuration, prompt templates, thinking/reasoning) while being tailored for specific capabilities like Android control, coding, web automation, task management, or social messaging.++## Architecture++```+Skill Nodes Specialized Agent Output+[Android Skill] ----++ +---> [Android Control Agent] ---> Workflow Output+Memory Node --------+ (AI Agent)+ ++Tool Nodes ---------++[Battery Monitor]+[WiFi Automation]+```++**Key Characteristics:**+- **Full AI Agent Inheritance**: Same AI configuration options as base AI Agent (provider, model, prompt, system message, temperature, thinking)+- **Left Input Handles**: Main input (30%) and Memory input (70%) on the left side+- **Bottom Handles**: Skill (25%) and Tool (75%) connections at the bottom+- **Right Output Handle**: Agent output for workflow continuation+- **Domain-Specific**: Pre-configured descriptions and use cases for specific capabilities++## Existing Specialized Agents++All agent colors use centralized dracula theme constants from `client/src/styles/theme.ts`.++| Node | Name | Description | Dracula Color |+|------|------|-------------|---------------|+| `android_agent` | Android Control Agent | Android device control agent | `dracula.green` |+| `coding_agent` | Coding Agent | Code execution agent | `dracula.cyan` |+| `web_agent` | Web Control Agent | Web automation agent | `dracula.pink` |+| `task_agent` | Task Management Agent | Task automation agent | `dracula.purple` |+| `social_agent` | Social Media Agent | Social messaging agent | `dracula.green` |+| `travel_agent` | Travel Agent | Travel planning agent | `dracula.orange` |+| `tool_agent` | Tool Agent | Tool orchestration agent | `dracula.yellow` |+| `productivity_agent` | Productivity Agent | Productivity workflows agent | `dracula.cyan` |+| `payments_agent` | Payments Agent | Payment processing agent | `dracula.green` |+| `consumer_agent` | Consumer Agent | Consumer interactions agent | `dracula.purple` |++## Shared AI Agent Properties++All specialized agents share the same AI configuration properties defined in `AI_AGENT_PROPERTIES`:++```typescript+const AI_AGENT_PROPERTIES: INodeProperties[] = [+ {+ displayName: 'AI Provider',+ name: 'provider',+ type: 'options',+ options: [+ { name: 'OpenAI', value: 'openai' },+ { name: 'Anthropic (Claude)', value: 'anthropic' },+ { name: 'Google (Gemini)', value: 'gemini' },+ { name: 'Groq', value: 'groq' },+ { name: 'OpenRouter', value: 'openrouter' }+ ],+ default: 'openai',+ description: 'The AI provider to use (configure API keys in Credentials)'+ },+ {+ displayName: 'Model',+ name: 'model',+ type: 'string',+ default: '',+ required: true,+ placeholder: 'Select a model...',+ description: 'AI model to use for the agent',+ typeOptions: {+ dynamicOptions: true,+ dependsOn: ['provider']+ }+ },+ {+ displayName: 'Prompt',+ name: 'prompt',+ type: 'string',+ default: '{{ $json.chatInput }}',+ required: true,+ typeOptions: { rows: 4 },+ description: 'The prompt template for the AI agent',+ placeholder: 'Enter your prompt or use template variables...'+ },+ {+ displayName: 'System Message',+ name: 'systemMessage',+ type: 'string',+ default: 'You are a helpful assistant',+ typeOptions: { rows: 3 },+ description: 'Define the behavior and personality of the AI agent'+ },+ {+ displayName: 'Options',+ name: 'options',+ type: 'collection',+ placeholder: 'Add Option',+ default: {},+ options: [+ {+ displayName: 'Temperature',+ name: 'temperature',+ type: 'number',+ default: 0.7,+ typeOptions: { minValue: 0, maxValue: 2, numberStepSize: 0.1 },+ description: 'Controls randomness in responses'+ },+ {+ displayName: 'Maximum Tokens',+ name: 'maxTokens',+ type: 'number',+ default: 1000,+ typeOptions: { minValue: 1, maxValue: 8192 },+ description: 'Maximum number of tokens to generate'+ },+ {+ displayName: 'Thinking/Reasoning',+ name: 'thinkingEnabled',+ type: 'boolean',+ default: false,+ description: 'Enable extended thinking for supported providers'+ },+ {+ displayName: 'Thinking Budget',+ name: 'thinkingBudget',+ type: 'number',+ default: 2048,+ typeOptions: { minValue: 1024, maxValue: 16000 },+ description: 'Token budget for thinking (Claude, Gemini, Cerebras)',+ displayOptions: { show: { thinkingEnabled: [true] } }+ },+ {+ displayName: 'Reasoning Effort',+ name: 'reasoningEffort',+ type: 'options',+ options: [+ { name: 'Low', value: 'low' },+ { name: 'Medium', value: 'medium' },+ { name: 'High', value: 'high' }+ ],+ default: 'medium',+ description: 'Reasoning effort level (OpenAI o-series, Groq)',+ displayOptions: { show: { thinkingEnabled: [true] } }+ }+ ]+ }+];+```++## Creating a New Specialized Agent++### Step 1: Add Node Definition++Add to `client/src/nodeDefinitions/specializedAgentNodes.ts`:++```typescript+import { dracula } from '../styles/theme';++// Database Agent - AI Agent with database capabilities+database_agent: {+ displayName: 'Database Agent',+ name: 'database_agent',+ icon: '...',+ group: ['agent', 'ai'], // CRITICAL: 'agent' first for category placement+ version: 1,+ subtitle: 'Data Operations',+ description: 'AI Agent specialized for database operations. Connect skills, memory, and database nodes.',+ defaults: { name: 'Database Agent', color: dracula.cyan }, // Use centralized dracula theme+ inputs: [+ {+ name: 'main',+ displayName: 'Input',+ type: 'main' as NodeConnectionType,+ description: 'Agent input'+ },+ {+ name: 'skill',+ displayName: 'Skill',+ type: 'main' as NodeConnectionType,+ description: 'Skill nodes that provide context and instructions'+ },+ {+ name: 'memory',+ displayName: 'Memory',+ type: 'main' as NodeConnectionType,+ description: 'Memory node for conversation history'+ },+ {+ name: 'tools',+ displayName: 'Tool',+ type: 'main' as NodeConnectionType,+ description: 'Database tool nodes (SQL, MongoDB, Redis, etc.)'+ }+ ],+ outputs: [{+ name: 'main',+ displayName: 'Output',+ type: 'main' as NodeConnectionType,+ description: 'Agent output'+ }],+ properties: AI_AGENT_PROPERTIES // CRITICAL: Use shared properties for full AI configuration+},+```++### Step 2: Add to SPECIALIZED_AGENT_TYPES++In the same file, add to the array:++```typescript+export const SPECIALIZED_AGENT_TYPES = [+ 'android_agent', 'coding_agent', 'web_agent', 'task_agent', 'social_agent',+ 'travel_agent', 'tool_agent', 'productivity_agent', 'payments_agent', 'consumer_agent',+ 'database_agent' // Add new agent here+];+```++### Step 3: Add AIAgentNode Config++Add to `client/src/components/AIAgentNode.tsx` in `AGENT_CONFIGS`:++```typescript+database_agent: {+ icon: <span style={{ fontSize: '28px' }}>...</span>,+ title: 'Database Agent',+ subtitle: 'Data Operations',+ themeColorKey: 'cyan', // References dracula.cyan - resolved at runtime via dracula[config.themeColorKey]+ bottomHandles: [+ { id: 'input-skill', label: 'Skill', position: '25%' },+ { id: 'input-tools', label: 'Tool', position: '75%' },+ ],+ leftHandles: [+ { id: 'input-memory', label: 'Memory', position: '70%' },+ ],+ topOutputHandle: { id: 'output-top', label: 'Output' },+ width: 260,+ height: 160,+},+```++### Step 4: Verify Dashboard Mapping++The `Dashboard.tsx` automatically maps specialized agents via `SPECIALIZED_AGENT_TYPES`:++```typescript+// Pre-register specialized agent nodes explicitly+SPECIALIZED_AGENT_TYPES.forEach(type => {+ types[type] = AIAgentNode;+});+```++No changes needed if you added to `SPECIALIZED_AGENT_TYPES`.++### Step 5: Add to Parameter Panel Arrays++In `client/src/components/parameterPanel/MiddleSection.tsx`, add to `AGENT_WITH_SKILLS_TYPES`:+```typescript+const AGENT_WITH_SKILLS_TYPES = [+ 'chatAgent', 'android_agent', 'coding_agent', 'web_agent', 'task_agent', 'social_agent',+ 'travel_agent', 'tool_agent', 'productivity_agent', 'payments_agent', 'consumer_agent',+ 'database_agent' // Add new agent+];+```++In `client/src/components/parameterPanel/InputSection.tsx`, add to both arrays:+- `AGENT_WITH_SKILLS_TYPES` (same as above)+- `aiAgentTypes` (includes `'aiAgent'` as well)++### Step 6: Backend - Add to Constants++In `server/constants.py`, add to `AI_AGENT_TYPES` frozenset:+```python+AI_AGENT_TYPES: FrozenSet[str] = frozenset([+ 'aiAgent', 'chatAgent',+ 'android_agent', 'coding_agent', 'web_agent', 'task_agent', 'social_agent',+ 'travel_agent', 'tool_agent', 'productivity_agent', 'payments_agent', 'consumer_agent',+ 'database_agent', # Add new agent+])+```++### Step 7: Backend - Add Handler Registry Entry++In `server/services/node_executor.py`, add to `_build_handler_registry()`:+```python+'database_agent': partial(handle_chat_agent, ai_service=self.ai_service, database=self.database),+```++### Step 8: Backend - Add to Delegation Check++In `server/services/handlers/tools.py`, add to the delegation check tuple in `execute_tool()`:+```python+if node_type in ('aiAgent', 'chatAgent', 'android_agent', ..., 'database_agent'):+ return await _execute_delegated_agent(tool_args, config)+```++### Step 9: Backend - Add Delegation Tool Definitions++In `server/services/ai.py`, add to three locations:+1. `DEFAULT_TOOL_NAMES`: `'database_agent': 'delegate_to_database_agent'`+2. `DEFAULT_TOOL_DESCRIPTIONS`: `'database_agent': 'Delegate database tasks to the Database Agent'`+3. `DelegateToAgentSchema` condition tuple: add `'database_agent'`++## AgentConfig Interface++```typescript+type ThemeColorKey = 'purple' | 'cyan' | 'green' | 'pink' | 'orange' | 'yellow' | 'red';++interface AgentConfig {+ icon: React.ReactNode; // Emoji with fontSize 28px+ title: string; // Display title+ subtitle: string; // Subtitle below title+ themeColorKey: ThemeColorKey; // Dracula theme color key - resolved via dracula[themeColorKey]+ bottomHandles: Array<{+ id: string; // Handle ID (e.g., 'input-skill')+ label: string; // Handle label+ position: string; // CSS position (e.g., '25%')+ }>;+ leftHandles?: Array<{...}>; // Optional left-side handles (below main input)+ rightHandles?: Array<{...}>; // Optional right-side handles+ topOutputHandle?: {...}; // Optional top output handle+ skipInputHandle?: boolean; // Skip left input handle+ skipRightOutput?: boolean; // Skip right output handle+ width?: number; // Custom width in pixels+ height?: number; // Custom height in pixels+}+```++## Visual Design++Specialized agents render as rectangular cards with AI Agent-style handles:++```++-------------------------------------------++| ... o | (params button, status indicator)+| |+| Input o--------- | (left handle at 30%)+| |+| ... | (icon - emoji)+| Database Agent | (title)+| Data Operations | (subtitle)+| |+| Memory <>--------- | (left handle at 70% - diamond)+| |+| Skill Tool | (bottom handle labels)++-----------<>--------------<>--------------+ (bottom handles - diamond)+ o--- (right output handle)+```++## Handle Layout++| Handle | Position | ID | Purpose |+|--------|----------|----|---------|+| Left Input | 30% from top | `input-main` | Main workflow data input |+| Left Memory | 70% from top | `input-memory` | Memory node for history |+| Bottom Skill | 25% from left | `input-skill` | Skill nodes for context |+| Bottom Tool | 75% from left | `input-tools` | Tool nodes for capabilities |+| Right Output | Center right | `output-main` | Agent output |++## Handle Styling++All handles use diamond shape (45-degree rotation) with consistent styling:++```typescript+// Left handles (Input, Memory)+style={{+ position: 'absolute',+ left: '-6px',+ top: position, // '30%' or '70%'+ width: theme.nodeSize.handle,+ height: theme.nodeSize.handle,+ backgroundColor: theme.colors.background,+ border: `2px solid ${theme.colors.textSecondary}`,+ borderRadius: '0',+ transform: 'translateY(-50%) rotate(45deg)'+}}++// Handle labels+style={{+ fontSize: '10px',+ color: theme.colors.text,+ fontWeight: theme.fontWeight.medium,+ opacity: 0.8+}}+```++## Component Palette Category++Specialized agents appear in the **AI Agents** category:+- Group: `['agent', 'ai']` - First element determines category+- Visible in Normal mode (SIMPLE_MODE_CATEGORIES includes 'agent')+- Purple color theme (theme.dracula.purple)++## Backend Handler++Specialized agents use the same backend handler as AI Agents since they inherit the same functionality. The handler should:++1. Collect skill data from `input-skill` handle+2. Collect memory data from `input-memory` handle+3. Collect tool data from `input-tools` handle+4. Execute LangGraph agent with collected context++See `server/services/handlers/ai.py` for the `handle_chat_agent()` implementation.++## Supported AI Features++All specialized agents support:++| Feature | Description |+|---------|-------------|+| **Multi-Provider** | OpenAI, Anthropic, Google, Groq, OpenRouter |+| **Dynamic Models** | Model dropdown populated based on selected provider |+| **Prompt Templates** | Support for `{{ $json.field }}` template variables |+| **System Message** | Customizable agent personality and behavior |+| **Temperature** | Response randomness control (0-2) |+| **Max Tokens** | Output length limit (1-8192) |+| **Thinking/Reasoning** | Extended thinking for Claude, Gemini, Groq models |+| **Thinking Budget** | Token budget for reasoning (1024-16000) |+| **Reasoning Effort** | Low/Medium/High for OpenAI o-series and Groq |+| **Async Delegation** | Can be used as tools by parent agents (fire-and-forget) |++## Async Agent Delegation++Specialized agents can be used as tools by other agents via the `input-tools` handle. This enables hierarchical agent architectures where a parent agent delegates tasks to specialized child agents.++### Architecture++```+Parent Agent decides to delegate+ |+ v+calls delegate_to_<agent_type>(task="...", context="...")+ |+ v+Tool handler spawns child agent via asyncio.create_task()+ |+ v+Returns immediately: {"status": "delegated", "task_id": "..."}+ |+Parent Agent continues working+ |+Child Agent executes independently in background+ |+Child broadcasts its own status updates+```++### How It Works++1. **Connect agent as tool**: Connect any specialized agent to a parent agent's `input-tools` handle+2. **Parent agent calls tool**: LLM decides to delegate using `delegate_to_<agent_type>` tool+3. **Fire-and-forget**: Child agent spawns as background task, parent continues immediately+4. **Independent execution**: Child agent uses its own connected skills, memory, and tools++### Tool Schema++When connected as a tool, the agent exposes this schema to the parent's LLM:++```python+class DelegateToAgentSchema(BaseModel):+ task: str = Field(description="The task to delegate to the child agent")+ context: Optional[str] = Field(+ default=None,+ description="Additional context for the task"+ )+```++### Example: Parent Delegates to Android Agent++```+[AI Agent] ←──tools── [Android Control Agent] ←──tools── [Battery Monitor]+ ←──tools── [WiFi Automation]+```++When the AI Agent needs to control Android devices:+1. LLM calls `delegate_to_android_agent(task="Check battery and enable WiFi if low")`+2. Android Agent spawns as background task+3. AI Agent receives `{"status": "delegated", "task_id": "..."}` immediately+4. Android Agent executes with its own battery and WiFi tools+5. Android Agent broadcasts its own status updates++### Key Design Decisions++- **Non-blocking**: Parent doesn't wait for child completion+- **Memory isolation**: Child uses its own connected memory, not parent's+- **Independent tools**: Child has access to its own connected tools only+- **Error isolation**: Child errors don't propagate to parent+- **Status broadcasting**: Both agents show their own execution status in UI++### Implementation Files++| File | Purpose |+|------|---------|+| `server/services/ai.py` | `DelegateToAgentSchema` in `_get_tool_schema()` |+| `server/services/handlers/tools.py` | `_execute_delegated_agent()` spawns child via `asyncio.create_task()` |+| `server/services/handlers/ai.py` | Passes `context` for nested agent delegation |++## Testing++1. Hard refresh browser (Ctrl+Shift+R) after changes+2. Check React DevTools for node type registration+3. Verify node appears in Component Palette under "AI Agents"+4. Test drag-and-drop to canvas+5. Open parameter panel - verify AI configuration options appear+6. Test left handle connections (Input, Memory)+7. Test bottom handle connections (Skill, Tool)+8. Test right output handle connections+9. Execute agent and verify output++## Files Reference++### Frontend+| File | Purpose |+|------|---------|+| `client/src/nodeDefinitions/specializedAgentNodes.ts` | Node definitions + AI_AGENT_PROPERTIES + SPECIALIZED_AGENT_TYPES |+| `client/src/styles/theme.ts` | Centralized dracula theme constants (colors used in node defaults) |+| `client/src/components/AIAgentNode.tsx` | AGENT_CONFIGS visual configuration with themeColorKey |+| `client/src/components/parameterPanel/MiddleSection.tsx` | AGENT_WITH_SKILLS_TYPES array |+| `client/src/components/parameterPanel/InputSection.tsx` | AGENT_WITH_SKILLS_TYPES + aiAgentTypes arrays |+| `client/src/Dashboard.tsx` | Node type to component mapping (automatic via SPECIALIZED_AGENT_TYPES) |+| `client/src/components/ui/ComponentPalette.tsx` | Category display (automatic via group) |+| `client/src/nodeDefinitions.ts` | Main registry (imports specializedAgentNodes) |++### Backend+| File | Purpose |+|------|---------|+| `server/constants.py` | AI_AGENT_TYPES frozenset (provider detection, config node filtering) |+| `server/services/node_executor.py` | Handler registry mapping agent types to handle_chat_agent |+| `server/services/handlers/ai.py` | Backend handler (shared with AI Agents) |+| `server/services/handlers/tools.py` | Delegation check in execute_tool() |+| `server/services/ai.py` | DEFAULT_TOOL_NAMES, DEFAULT_TOOL_DESCRIPTIONS, DelegateToAgentSchema |diff --git a/docs-internal/workflow-schema.md b/docs-internal/workflow-schema.mdnew file mode 100644index 00000000..c547c473--- /dev/null+++ b/docs-internal/workflow-schema.md@@ -0,0 +1,607 @@+# Workflow JSON Schema++> **Related Documentation:**+> - [Node Creation Guide](./node_creation.md) - How to create new nodes (frontend definitions, backend handlers)+> - [CLAUDE.md](../CLAUDE.md) - Project overview, key files, architecture patterns++## Overview++The workflow JSON schema defines the structure and validation rules for workflow automation data. This document describes the schema format, validation, and usage examples.++## Schema Structure++A workflow JSON document contains:++- **Metadata**: ID, name, timestamps, version+- **Nodes**: Array of workflow nodes with their types, positions, and parameters+- **Edges**: Array of connections between nodes++## Complete Schema++```json+{+ "$schema": "http://json-schema.org/draft-07/schema#",+ "title": "Workflow",+ "description": "A workflow automation definition containing nodes and connections",+ "type": "object",+ "required": ["id", "name", "nodes", "edges", "createdAt", "lastModified"],+ "properties": {+ "id": {+ "type": "string",+ "pattern": "^workflow_[0-9]+$"+ },+ "name": {+ "type": "string",+ "minLength": 1,+ "maxLength": 100+ },+ "nodes": {+ "type": "array",+ "items": { "$ref": "#/definitions/node" }+ },+ "edges": {+ "type": "array",+ "items": { "$ref": "#/definitions/edge" }+ },+ "createdAt": {+ "type": "string",+ "format": "date-time"+ },+ "lastModified": {+ "type": "string",+ "format": "date-time"+ },+ "version": {+ "type": "string",+ "default": "1.0.0"+ }+ }+}+```++## Supported Node Types (64 Total)++### Workflow Nodes (1 node)+- `start` - Workflow entry point with initial data++### Scheduler Nodes (2 nodes)+- `timer` - Delay/wait before continuing (seconds, minutes, hours)+- `cronScheduler` - Recurring scheduled execution (seconds to months, timezone support)++### AI Chat Model Nodes (6 nodes)+- `openaiChatModel` - OpenAI GPT models (GPT-4o, GPT-4, o1/o3/o4 series with reasoning)+- `anthropicChatModel` - Anthropic Claude models (Claude 3.5/3 with extended thinking)+- `geminiChatModel` - Google Gemini models (2.5/Flash with thinking support)+- `openrouterChatModel` - OpenRouter unified API (200+ models from multiple providers)+- `groqChatModel` - Groq ultra-fast inference (Llama, Mixtral, Qwen3/QwQ with reasoning)+- `cerebrasChatModel` - Cerebras custom AI hardware (Llama, Qwen)++### AI Agent Nodes (3 nodes)+- `aiAgent` - AI agent with tool calling, memory support (LangGraph)+- `chatAgent` - Conversational agent with skill support+- `simpleMemory` - Conversation history storage (buffer/window modes)++### AI Skill Nodes (9 nodes)+Connect to Chat Agent's `input-skill` handle:+- `claudeSkill` - Default Claude assistant personality+- `whatsappSkill` - WhatsApp messaging via Chat Agent+- `memorySkill` - Long-term memory management+- `mapsSkill` - Location services (geocoding, places, directions)+- `httpSkill` - HTTP requests to external APIs+- `schedulerSkill` - Task scheduling (timers, cron)+- `androidSkill` - Android device control+- `codeSkill` - Python/JavaScript code execution+- `customSkill` - User-created custom skills (database-stored)++### AI Tool Nodes (4 nodes)+Connect to AI Agent's `input-tools` handle:+- `calculatorTool` - Mathematical operations+- `currentTimeTool` - Current date/time with timezone+- `webSearchTool` - Web search (DuckDuckGo, Serper, Google)+- `androidTool` - Android toolkit aggregator (sub-node pattern)++### Location Nodes (3 nodes)+- `createMap` - Google Maps creation with center, zoom, map type+- `addLocations` - Google Maps Geocoding (address to coordinates)+- `showNearbyPlaces` - Google Places nearbySearch++### WhatsApp Nodes (4 nodes)+- `whatsappSend` - Send messages (text, media, location, contact)+- `whatsappConnect` - Connection status and QR code+- `whatsappReceive` - Event-driven trigger with filters (type, sender, group, keywords)+- `whatsappChatHistory` - Retrieve message history++### Android Nodes (17 nodes)++#### Device Setup (1 node)+- `androidDeviceSetup` - Connect via local ADB or remote WebSocket relay++#### System Monitoring (4 nodes)+- `batteryMonitor` - Battery status, level, charging, temperature+- `networkMonitor` - Network connectivity and type+- `systemInfo` - Device/OS info, memory, hardware+- `location` - GPS tracking with accuracy++#### App Management (2 nodes)+- `appLauncher` - Launch applications by package name+- `appList` - List installed applications++#### Device Automation (6 nodes)+- `wifiAutomation` - WiFi enable/disable, scan, status+- `bluetoothAutomation` - Bluetooth enable/disable, paired devices+- `audioAutomation` - Volume control, mute/unmute+- `deviceStateAutomation` - Airplane mode, screen, power save, brightness+- `screenControlAutomation` - Brightness, wake, timeout+- `airplaneModeControl` - Airplane mode status and control++#### Sensors (2 nodes)+- `motionDetection` - Accelerometer, gyroscope, shake detection+- `environmentalSensors` - Temperature, humidity, pressure, light++#### Media (2 nodes)+- `cameraControl` - Camera info, take photos+- `mediaControl` - Media playback, volume++### Utility Nodes (5 nodes)+- `httpRequest` - HTTP requests (GET, POST, PUT, DELETE, PATCH)+- `webhookTrigger` - Incoming HTTP webhook trigger at `/webhook/{path}`+- `webhookResponse` - Custom response to webhook caller+- `chatTrigger` - Console message input trigger+- `console` - Debug logging output++### Code Nodes (2 nodes)+- `pythonExecutor` - Python code execution with input_data access+- `javascriptExecutor` - JavaScript code execution++### Chat Nodes (2 nodes)+- `chatSend` - Send via JSON-RPC 2.0 WebSocket+- `chatHistory` - Retrieve chat message history++### Document Processing Nodes (6 nodes)+RAG pipeline nodes for document ingestion, processing, and vector storage:+- `httpScraper` - Scrape links from web pages (date/page pagination modes)+- `fileDownloader` - Parallel file downloads with semaphore concurrency+- `documentParser` - Parse documents to text (PyPDF, Marker OCR, Unstructured, BeautifulSoup)+- `textChunker` - Split text into overlapping chunks (recursive, markdown, token strategies)+- `embeddingGenerator` - Generate vector embeddings (HuggingFace, OpenAI, Ollama providers)+- `vectorStore` - Store/query vectors (ChromaDB, Qdrant, Pinecone backends)++## Example Workflow JSON++### Basic Workflow (Start -> AI Agent)++```json+{+ "id": "workflow_1234567890",+ "name": "AI Agent with Start Node",+ "version": "1.0.0",+ "createdAt": "2025-01-06T12:00:00.000Z",+ "lastModified": "2025-01-06T12:30:00.000Z",+ "nodes": [+ {+ "id": "start-1",+ "type": "start",+ "position": { "x": 100, "y": 100 },+ "data": {+ "label": "Start",+ "initialData": "{\"message\": \"Hello World\", \"value\": 123}"+ }+ },+ {+ "id": "aiAgent-1",+ "type": "aiAgent",+ "position": { "x": 400, "y": 100 },+ "data": {+ "label": "AI Agent",+ "provider": "anthropic",+ "model": "claude-haiku-4-5-20251001",+ "prompt": "{{start.message}}",+ "systemMessage": "You are a helpful assistant"+ }+ }+ ],+ "edges": [+ {+ "id": "edge-1",+ "source": "start-1",+ "target": "aiAgent-1",+ "sourceHandle": "output-main",+ "targetHandle": "input-main"+ }+ ]+}+```++### Workflow with Memory (Conversational AI)++```json+{+ "id": "workflow_1234567891",+ "name": "AI Agent with Memory",+ "version": "1.0.0",+ "createdAt": "2025-01-06T12:00:00.000Z",+ "lastModified": "2025-01-06T12:30:00.000Z",+ "nodes": [+ {+ "id": "start-1",+ "type": "start",+ "position": { "x": 100, "y": 100 },+ "data": {+ "label": "Start",+ "initialData": "{\"chatInput\": \"What did we discuss earlier?\"}"+ }+ },+ {+ "id": "simpleMemory-1",+ "type": "simpleMemory",+ "position": { "x": 400, "y": 250 },+ "data": {+ "label": "Memory",+ "sessionId": "user-session-123",+ "memoryType": "window",+ "windowSize": 20+ }+ },+ {+ "id": "aiAgent-1",+ "type": "aiAgent",+ "position": { "x": 400, "y": 100 },+ "data": {+ "label": "AI Agent",+ "provider": "anthropic",+ "model": "claude-haiku-4-5-20251001",+ "prompt": "{{start.chatInput}}",+ "systemMessage": "You are a helpful assistant with memory of our conversation"+ }+ }+ ],+ "edges": [+ {+ "id": "edge-main",+ "source": "start-1",+ "target": "aiAgent-1",+ "sourceHandle": "output-main",+ "targetHandle": "input-main"+ },+ {+ "id": "edge-memory",+ "source": "simpleMemory-1",+ "target": "aiAgent-1",+ "sourceHandle": "output-memory",+ "targetHandle": "input-memory"+ }+ ]+}+```++**Memory Workflow Behavior:**+1. Start node provides the user's chat input+2. Simple Memory connects to AI Agent's memory handle (config connection)+3. When AI Agent runs, it loads conversation history from the memory session+4. AI Agent's response is automatically saved to the memory session+5. Simple Memory node can see Start node's outputs (via AI Agent) for parameter mapping++## Usage Examples++### Export Workflow++```typescript+import { useAppStore } from './store/useAppStore';++// Export to JSON string+const workflow = useAppStore.getState().currentWorkflow;+const jsonString = useAppStore.getState().exportWorkflowToJSON();+console.log(jsonString);++// Export to file download+useAppStore.getState().exportWorkflowToFile();+```++### Import Workflow++```typescript+import { useAppStore } from './store/useAppStore';++// Import from JSON string+const jsonString = '{ ... }';+useAppStore.getState().importWorkflowFromJSON(jsonString);++// Import from file+import { importWorkflowFromFile } from './utils/workflowExport';++const fileInput = document.createElement('input');+fileInput.type = 'file';+fileInput.accept = '.json';+fileInput.onchange = async (e) => {+ const file = (e.target as HTMLInputElement).files?.[0];+ if (file) {+ const workflow = await importWorkflowFromFile(file);+ useAppStore.getState().setCurrentWorkflow(workflow);+ }+};+fileInput.click();+```++### Validate Workflow++```typescript+import { validateWorkflow } from './schemas/workflowSchema';++const workflow = {+ id: 'workflow_123',+ name: 'My Workflow',+ nodes: [...],+ edges: [...],+ createdAt: new Date().toISOString(),+ lastModified: new Date().toISOString()+};++const validation = validateWorkflow(workflow);++if (validation.valid) {+ console.log('Workflow is valid');+} else {+ console.error('Validation errors:', validation.errors);+}+```++## Config Node Architecture++Config nodes (memory, tools) are auxiliary nodes that connect to parent nodes via special handles instead of the main data flow.++### Characteristics of Config Nodes++| Property | Config Node | Standard Node |+|----------|-------------|---------------|+| Group | Contains 'memory' or 'tool' | Other groups |+| Input Handle | None (passive) | `input-main` |+| Output Handle | `output-memory`, `output-model` | `output-main` |+| Target Handle | Parent's `input-memory`, `input-tools` | Parent's `input-main` |+| Parent Visibility | NOT shown in parent's input panel | Shown in parent's input panel |+| Input Inheritance | Inherits parent's main inputs | Only direct connections |++### Config Node Detection++Nodes are identified as config nodes by their group membership:++```typescript+const isConfigNode = (nodeType: string): boolean => {+ const definition = nodeDefinitions[nodeType];+ const groups = definition?.group || [];+ return groups.includes('memory') || groups.includes('tool');+};+```++### Input Inheritance for Config Nodes++When a config node is connected to a parent's config handle:+1. The parent node does NOT see the config node as an input+2. The config node DOES see the parent's main inputs (labeled "via Parent")++This allows memory nodes to access the same input data as the AI Agent they're connected to, enabling template variable mapping like `{{start.chatInput}}` in the memory node's session ID.++```+┌─────────────┐ main ┌─────────────┐+│ Start │──────────────▶│ AI Agent │+└─────────────┘ └─────────────┘+ ▲+ │ input-memory+ ┌─────────────┐+ │ Memory │+ │ (sees Start │+ │ via Agent) │+ └─────────────┘+```++## Node Data Structure++Each node contains:++- `id`: Unique identifier (string)+- `type`: Node type (see supported types)+- `position`: { x: number, y: number }+- `data`: Node-specific parameters (object)+ - `label`: Display label+ - Additional parameters based on node type++### Start Node Data++```json+{+ "initialData": "{\"key1\": \"value1\", \"key2\": 123}"+}+```++### AI Agent Node Data++```json+{+ "provider": "anthropic" | "openai" | "gemini" | "openrouter",+ "model": "model-name",+ "apiKey": "api-key-string",+ "prompt": "{{start.message}}",+ "systemMessage": "System instructions",+ "temperature": 0.7,+ "maxTokens": 1000+}+```++### OpenRouter Chat Model Node Data++```json+{+ "model": "openai/gpt-4o-mini",+ "prompt": "{{start.message}}",+ "options": {+ "systemMessage": "You are a helpful assistant",+ "temperature": 0.7,+ "maxTokens": 1000,+ "topP": 1,+ "frequencyPenalty": 0,+ "presencePenalty": 0,+ "timeout": 60000,+ "maxRetries": 2+ }+}+```++OpenRouter model IDs use the format `provider/model-name`:+- `openai/gpt-4o`, `openai/gpt-4o-mini`+- `anthropic/claude-3.5-sonnet`, `anthropic/claude-3-haiku`+- `google/gemini-pro`, `google/gemini-flash-1.5`+- `meta-llama/llama-3.1-70b-instruct`+- Free models are prefixed with `[FREE]` in the dropdown++### Simple Memory Node Data++```json+{+ "sessionId": "conversation-session-id",+ "memoryType": "buffer" | "window",+ "windowSize": 10+}+```++Memory nodes are config nodes that:+- Connect to AI Agent via `input-memory` handle+- Store conversation history in database (persisted across restarts)+- Support buffer mode (all messages) or window mode (last N messages)+- Inherit parent node's main inputs for parameter mapping++### Trigger Node Data (WhatsApp Trigger)++```json+{+ "messageTypeFilter": "all" | "text" | "image" | "video" | "audio",+ "filter": "all" | "contact" | "group" | "keywords",+ "contactPhone": "+1234567890",+ "groupId": "group-id",+ "keywords": "word1, word2",+ "ignoreOwnMessages": true,+ "includeMediaData": false+}+```++Trigger nodes:+- Wait for external events using asyncio.Future+- Have no main input (they start workflows)+- Output event data when triggered++## Edge Data Structure++Each edge contains:++- `id`: Unique identifier+- `source`: Source node ID+- `target`: Target node ID+- `sourceHandle`: Output handle ID (default: "output-main")+- `targetHandle`: Input handle ID (default: "input-main")+- `type`: Rendering type ("default", "straight", "step", etc.)++### Handle Naming Conventions++#### Main Data Flow Handles+- `output-main` / `input-main` - Primary data flow between nodes++#### Config Handles (for auxiliary nodes)+Config handles connect memory/tool nodes to parent nodes without being part of main data flow:+- `input-memory` - Memory configuration input (AI Agent)+- `input-tools` - Tools configuration input (AI Agent)+- `input-model` - Model configuration input+- `output-model` - Model/config output (circular nodes)+- `output-memory` - Memory output (simpleMemory node)++#### Handle Detection Pattern+Config handles follow the pattern `input-<type>` where type is NOT 'main':+```typescript+const isConfigHandle = (handle: string): boolean => {+ return handle.startsWith('input-') && handle !== 'input-main';+};+```++### Config Node Connection Example++```json+{+ "edges": [+ {+ "id": "edge-memory",+ "source": "simpleMemory-1",+ "target": "aiAgent-1",+ "sourceHandle": "output-memory",+ "targetHandle": "input-memory"+ },+ {+ "id": "edge-main",+ "source": "start-1",+ "target": "aiAgent-1",+ "sourceHandle": "output-main",+ "targetHandle": "input-main"+ }+ ]+}+```++In this example:+- The simpleMemory node connects to AI Agent's `input-memory` handle (config connection)+- The start node connects to AI Agent's `input-main` handle (main data flow)+- AI Agent does NOT see simpleMemory as an input (config handles are filtered)+- simpleMemory DOES see Start node's outputs (inherits parent's main inputs)++## Dynamic Parameter Resolution++Template variables in node parameters are resolved using the format `{{nodeName.property}}`:++```json+{+ "prompt": "{{start.message}}"+}+```++This resolves to the `message` property from the Start node's output data.++## Validation Rules++1. Workflow must have required fields: id, name, nodes, edges, createdAt, lastModified+2. Workflow ID must match pattern: `workflow_[0-9]+`+3. Each node must have: id, type, position (with x and y)+4. Each edge must have: id, source, target+5. Edge source and target must reference existing nodes+6. Node types must be one of the supported types++## Version History++- **1.4.0** (2026-01-27): Comprehensive node type expansion to 58 nodes+ - Added Groq and Cerebras AI chat model nodes (6 total AI models)+ - Added 9 AI Skill nodes for Chat Agent (claude, whatsapp, memory, maps, http, scheduler, android, code, custom)+ - Added 4 AI Tool nodes for AI Agent (calculator, currentTime, webSearch, androidTool)+ - Added whatsappChatHistory node (4 WhatsApp nodes total)+ - Added scheduler nodes: timer, cronScheduler (with timezone support)+ - Added chat nodes: chatSend, chatHistory (JSON-RPC 2.0 WebSocket)+ - Added utility nodes: chatTrigger, console (5 utility nodes total)+ - Added javascriptExecutor code node (2 code nodes total)+ - Updated WebSocket handlers count to 51++- **1.3.0** (2026-01-19): OpenRouter AI provider integration+ - Added openrouterChatModel to AI Nodes list+ - Added OpenRouter node data structure with model ID format+ - Updated AI Agent provider options to include openrouter++- **1.2.0** (2025-12-19): Config node architecture and expanded node types+ - Added Config Node Architecture section with input inheritance+ - Added 31 supported node types (AI, Android, WhatsApp, Location, Code)+ - Added handle naming conventions (main vs config handles)+ - Added Simple Memory and Trigger node data structures+ - Added workflow with memory example+ - Updated edge documentation with config handle patterns++- **1.1.0** (2025-01-10): Android and WhatsApp integration+ - Added 17 Android service nodes+ - Added WhatsApp messaging nodes+ - Added trigger node support++- **1.0.0** (2025-01-06): Initial schema definition+ - Basic workflow structure+ - Node and edge definitions+ - Validation rules+ - Export/import functionalitydiff --git a/package.json b/package.jsonindex 2650b881..2b371992 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "machinaos",- "version": "0.0.9",+ "version": "0.0.10","description": "Open source workflow automation platform with AI agents, React Flow, and n8n-inspired architecture","type": "module","keywords": [