Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs: Update documentation and fix Apify integration
trohitg65779cbd429a
diff --git a/CLAUDE.md b/CLAUDE.mdindex 2b2e6b46..b7fc7564 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -364,7 +364,7 @@ class CacheEntry(SQLModel, table=True):## Codebase Summary- **Hybrid architecture**: Node.js + Python + React TypeScript-- **104 implemented workflow nodes** with clean service separation (6 AI models + 3 AI agents/memory + 13 specialized agents + 11 skills + 3 dedicated tools + 6 dual-purpose tools + 16 Android + 3 WhatsApp + 4 Twitter + 2 Social + 3 Location + 3 Code + 6 Utility + 6 Document + 2 Chat + 2 Scheduler + 1 Workflow + 26 Google Workspace)+- **105 implemented workflow nodes** with clean service separation (6 AI models + 3 AI agents/memory + 13 specialized agents + 11 skills + 3 dedicated tools + 6 dual-purpose tools + 16 Android + 3 WhatsApp + 4 Twitter + 2 Social + 3 Location + 3 Code + 6 Utility + 6 Document + 2 Chat + 2 Scheduler + 1 Workflow + 26 Google Workspace + 1 Apify)- **WebSocket-First Architecture**: WebSocket as primary frontend-backend communication (87 message handlers)- **Recent optimizations**: REST APIs replaced with WebSocket, AI endpoints migrated to Python, Android automation integrated@@ -525,7 +525,7 @@ The project uses WebSocket as the primary communication method between frontend- `server/services/status_broadcaster.py` - Connection management and broadcasting## Implemented Node Types-The following 104 nodes are currently implemented and functional:+The following 105 nodes are currently implemented and functional:### AI Chat Models (6 nodes)- **openaiChatModel**: OpenAI GPT models with response format options. O-series models (o1, o3, o4) support reasoning effort parameter.@@ -1064,6 +1064,21 @@ GOOGLE_REDIRECT_URI=http://localhost:3010/api/google/callback**Google API Pricing:** All Google Workspace APIs are free with rate limits. See `server/config/pricing.json` for configured limits.+### Apify Nodes (1 node)+Web scraping service for social media, search engines, and websites using pre-built actors.++- **apifyActor**: **Dual-purpose node** - Run Apify actors (web scrapers) for Instagram, TikTok, Twitter/X, LinkedIn, Facebook, YouTube, Google Search, Google Maps, and website crawling. Works as workflow node OR AI Agent tool. Group: `['api', 'scraper', 'tool']`. Pre-built actor dropdown with quick input helpers per actor type. Parameters: actorId, actorInput (JSON), maxResults, timeout, memory.++**Key Files:**+| File | Description |+|------|-------------|+| `client/src/nodeDefinitions/apifyNodes.ts` | Node definition with actor presets |+| `server/services/handlers/apify.py` | Actor execution via apify-client SDK |+| `server/skills/web_agent/apify-skill/SKILL.md` | AI agent skill for web scraping |+| `client/src/components/CredentialsModal.tsx` | Apify API token panel |++**Authentication:** Single API token (Personal or Organization) from Apify Console -> Settings -> Integrations.+### Workflow Nodes (2 nodes)- **start**: Manual workflow trigger to start workflow execution- **taskTrigger**: Event-driven trigger that fires when a delegated child agent completes its task (success or error). Filters by task_id, agent_name, status (all/completed/error), and parent_node_id. Output includes task_id, status, agent_name, result/error, workflow_id.diff --git a/client/src/assets/icons/apify/apify.svg b/client/src/assets/icons/apify/apify.svgindex fb575161..af055255 100644--- a/client/src/assets/icons/apify/apify.svg+++ b/client/src/assets/icons/apify/apify.svg@@ -1,4 +1,4 @@-<svg width="150" height="150" viewBox="0 0 150 150" fill="none" xmlns="http://www.w3.org/2000/svg">+<svg viewBox="0 0 150 150" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M86.0214 0H147.727C148.982 0 150 1.01753 150 2.27273V96.5744C150 98.8332 147.062 99.7089 145.826 97.8188L84.1196 3.51714C83.1305 2.00559 84.215 0 86.0214 0Z" fill="#246DFF"/><path d="M63.9786 0H2.27273C1.01753 0 0 1.01753 0 2.27273V96.5744C0 98.8332 2.93774 99.7089 4.1745 97.8188L65.8804 3.51714C66.8695 2.00559 65.785 0 63.9786 0Z" fill="#20A34E"/><path d="M73.9429 75.5012L3.84485 146.126C2.42137 147.56 3.43724 150 5.45792 150H144.6C146.612 150 147.632 147.578 146.225 146.139L77.1811 75.5135C76.2942 74.6063 74.8365 74.6008 73.9429 75.5012Z" fill="#F86606"/>diff --git a/client/src/components/CredentialsModal.tsx b/client/src/components/CredentialsModal.tsxindex 0ffb7990..8fc21e13 100644--- a/client/src/components/CredentialsModal.tsx+++ b/client/src/components/CredentialsModal.tsx@@ -57,9 +57,16 @@ const GoogleWorkspaceIcon = () => (<GoogleOutlined style={{ fontSize: 20, color: '#4285F4' }} />);-const ApifyIcon = () => (- <div dangerouslySetInnerHTML={{ __html: ApifyIcons.apify }} style={{ width: 20, height: 20 }} />-);+const ApifyIcon = () => {+ // Replace the raw SVG to ensure it scales to container+ const scaledSvg = ApifyIcons.apify.replace('<svg', '<svg width="100%" height="100%"');+ return (+ <div+ dangerouslySetInnerHTML={{ __html: scaledSvg }}+ style={{ width: 20, height: 20 }}+ />+ );+};// ============================================================================// TYPES & DATAdiff --git a/client/src/components/ParameterRenderer.tsx b/client/src/components/ParameterRenderer.tsxindex d4465fdc..84271fac 100644--- a/client/src/components/ParameterRenderer.tsx+++ b/client/src/components/ParameterRenderer.tsx@@ -1872,6 +1872,38 @@ const ParameterRenderer: React.FC<ParameterRendererProps> = ({</div>);+ case 'json':+ // JSON editor - textarea for JSON input+ const jsonRows = (parameter as any).typeOptions?.rows || 6;+ return (+ <textarea+ value={currentValue || parameter.default || '{}'}+ onChange={(e) => onChange(e.target.value)}+ placeholder={parameter.placeholder || '{}'}+ rows={jsonRows}+ onDragOver={handleDragOver}+ onDragLeave={handleDragLeave}+ onDrop={handleDrop}+ style={{+ width: '100%',+ padding: '10px 12px',+ border: isDragOver ? `2px solid ${theme.colors.focus}` : `1px solid ${theme.colors.border}`,+ borderRadius: '6px',+ fontSize: '13px',+ backgroundColor: isDragOver ? theme.colors.focusRing : theme.colors.background,+ color: theme.colors.text,+ outline: 'none',+ resize: 'vertical',+ minHeight: '100px',+ fontFamily: 'monospace',+ lineHeight: '1.5',+ transition: 'all 0.2s ease'+ }}+ onFocus={(e) => e.target.style.borderColor = theme.colors.focus}+ onBlur={(e) => e.target.style.borderColor = theme.colors.border}+ />+ );+default:return <div style={{ color: theme.colors.error, fontSize: '14px', padding: '8px 12px', backgroundColor: `${theme.colors.error}15`, border: `1px solid ${theme.colors.error}30`, borderRadius: '6px' }}>Unsupported parameter type: {parameter.type}</div>;}diff --git a/client/src/services/executionService.ts b/client/src/services/executionService.tsindex e9a1f71a..35fffc79 100644--- a/client/src/services/executionService.ts+++ b/client/src/services/executionService.ts@@ -12,6 +12,7 @@ import { DRIVE_NODE_TYPES } from '../nodeDefinitions/driveNodes';import { SHEETS_NODE_TYPES } from '../nodeDefinitions/sheetsNodes';import { TASKS_NODE_TYPES } from '../nodeDefinitions/tasksNodes';import { CONTACTS_NODE_TYPES } from '../nodeDefinitions/contactsNodes';+import { APIFY_NODE_TYPES } from '../nodeDefinitions/apifyNodes';import { Node, Edge } from 'reactflow';import { INodeExecutionData } from '../types/INodeProperties';import { API_CONFIG } from '../config/api';@@ -239,7 +240,9 @@ export class ExecutionService {...DRIVE_NODE_TYPES,...SHEETS_NODE_TYPES,...TASKS_NODE_TYPES,- ...CONTACTS_NODE_TYPES+ ...CONTACTS_NODE_TYPES,+ // Apify web scraping+ ...APIFY_NODE_TYPES];return supportedTypes.includes(nodeType);diff --git a/docs-internal/SETUP.md b/docs-internal/SETUP.mdindex 905483c4..b9e2f2b3 100644--- a/docs-internal/SETUP.md+++ b/docs-internal/SETUP.md@@ -110,6 +110,18 @@ Add these to `.env` or configure via the Credentials UI:- `GOOGLE_AI_API_KEY`- `GOOGLE_MAPS_API_KEY`+### Authentication Toggle+| Variable | Default | Description |+|----------|---------|-------------|+| `VITE_AUTH_ENABLED` | true | Set to `false` to bypass login entirely |++When `VITE_AUTH_ENABLED=false`:+- Login page is skipped entirely+- User is set as anonymous with owner privileges+- Encryption service auto-initializes with `API_KEY_ENCRYPTION_KEY` as the password+- API keys can be saved/retrieved without user authentication+- Useful for local development and testing+## Docker Commands| Command | Description |diff --git a/docs-internal/dual_purpose_tool_node_creation.md b/docs-internal/dual_purpose_tool_node_creation.mdindex 17454938..ffcb26a2 100644--- a/docs-internal/dual_purpose_tool_node_creation.md+++ b/docs-internal/dual_purpose_tool_node_creation.md@@ -999,6 +999,7 @@ Post and interact with tweets on Twitter/X.- [ ] Include `'tool'` in the `group` array- [ ] Define comprehensive `properties` (LLM will fill these)- [ ] Ensure node renders correctly (SquareNode handles multiple outputs)+- [ ] **Add to `executionService.ts` `isNodeTypeSupported()`** (CRITICAL - enables Run button)### Backend Schema (ai.py)diff --git a/docs-internal/new_service_integration.md b/docs-internal/new_service_integration.mdindex 49888d73..c7897c42 100644--- a/docs-internal/new_service_integration.md+++ b/docs-internal/new_service_integration.md@@ -601,7 +601,7 @@ export const nodeDefinitions: NodeDefinitionRegistry = {}```-### Add to executionService.ts (Enable Run Button)+### Add to executionService.ts (CRITICAL - Enables Run Button)```typescript// client/src/services/executionService.ts@@ -620,7 +620,7 @@ static isNodeTypeSupported(nodeType: string): boolean {}```-**Important:** Without this step, the Run button will not appear in the parameter panel for your new nodes.+**CRITICAL:** Without this step, the Run button will NOT appear in the parameter panel for your new nodes. This is the execution whitelist - only nodes in this list can be executed via the Run button.---@@ -939,7 +939,7 @@ Use this checklist when adding a new service:- [ ] Export node types array- [ ] Import in `nodeDefinitions.ts`- [ ] Add to Dashboard.tsx- - [ ] Add to `executionService.ts` `isNodeTypeSupported()` (enables Run button)+ - [ ] **Add to `executionService.ts` `isNodeTypeSupported()`** (CRITICAL - enables Run button)- [ ] **AI Tool Integration** (`server/services/ai.py`, `server/services/handlers/tools.py`)- [ ] Add Pydantic schemas in `_get_tool_schema()`diff --git a/docs-internal/node_creation.md b/docs-internal/node_creation.mdindex 8d89a889..71ae82f7 100644--- a/docs-internal/node_creation.md+++ b/docs-internal/node_creation.md@@ -26,7 +26,7 @@ This guide explains how to create new nodes for MachinaOs, covering frontend def## Node Types Overview-MachinaOs supports 79 nodes across 16 categories:+MachinaOs supports 105 nodes across 17 categories:| Category | Visual Component | Example Nodes | Count ||----------|-----------------|---------------|-------|@@ -272,11 +272,25 @@ const triggerTypes = ['start', 'cronScheduler', 'webhookTrigger', 'whatsappRecei## Backend: Workflow Handler-### Add to Execution Service+### Add to Execution Service (Enables Run Button)-First, add your node to supported types in `client/src/services/executionService.ts`:+**CRITICAL:** Add your node type to the supported types whitelist in `client/src/services/executionService.ts`. Without this, the "Run" button will NOT appear in the parameter panel for your node.```typescript+// client/src/services/executionService.ts++// Option 1: Import and spread node types array (recommended for modules)+import { YOUR_NODE_TYPES } from '../nodeDefinitions/yourNodes';++static isNodeTypeSupported(nodeType: string): boolean {+ const supportedTypes = [+ // ... existing types+ ...YOUR_NODE_TYPES, // Spread imported array+ ];+ return supportedTypes.includes(nodeType);+}++// Option 2: Add individual node types directlystatic isNodeTypeSupported(nodeType: string): boolean {const supportedTypes = [// ... existing types@@ -286,6 +300,11 @@ static isNodeTypeSupported(nodeType: string): boolean {}```+**Why this is required:**+- The `isNodeTypeSupported()` method acts as an execution whitelist+- Only nodes in this list show the "Run" button in the parameter panel+- This prevents accidental execution of incomplete/untested nodes+### Create Backend HandlerIn `server/services/workflow.py`, add a handler in `_execute_single_node()`:@@ -1645,7 +1664,7 @@ await database.save_node_parameters(memory_node_id, {'memoryContent': updated_co- [ ] Create node definition in `client/src/nodeDefinitions/`- [ ] Register in `client/src/nodeDefinitions.ts`- [ ] Map to visual component in `Dashboard.tsx`-- [ ] Add to supported types in `executionService.ts`+- [ ] **Add to supported types in `executionService.ts`** (CRITICAL - enables Run button)- [ ] Add output schema in `InputSection.tsx` (for drag-and-drop UX)- [ ] For input-only nodes: Add to `NO_OUTPUT_NODE_TYPES` in `SquareNode.tsx`diff --git a/docs-internal/server-readme.md b/docs-internal/server-readme.mdindex e65feed3..4f2b086c 100644--- a/docs-internal/server-readme.md+++ b/docs-internal/server-readme.md@@ -53,6 +53,34 @@ For production:4. Configure proper CORS origins5. Consider enabling Redis caching+## Authentication & Encryption++### Authentication Toggle++Set `VITE_AUTH_ENABLED=false` in `.env` to bypass the login page entirely.++When auth is disabled:+- Login page is skipped+- User is set as anonymous with owner privileges+- Encryption service auto-initializes using `API_KEY_ENCRYPTION_KEY` as the password+- API keys can be saved/retrieved without user login++### Encryption Auto-Initialization++The encryption service requires initialization before encrypting/decrypting API keys. This normally happens during user login via `_initialize_encryption()`.++When `VITE_AUTH_ENABLED=false`, the encryption is auto-initialized at startup in `main.py`:++```python+# In main.py lifespan()+if settings.vite_auth_enabled == "false":+ encryption = container.encryption_service()+ if not encryption.is_initialized():+ encryption.initialize(settings.api_key_encryption_key, salt)+```++This allows API keys to be stored without authentication, useful for local development.+## Security Notes**NEVER commit `.env` files to git!**diff --git a/server/main.py b/server/main.pyindex 8fc9a346..e38c6f44 100644--- a/server/main.py+++ b/server/main.py@@ -71,9 +71,19 @@ async def lifespan(app: FastAPI):await container.cache().startup()# Initialize credentials database (creates tables if not exist)- await container.credentials_database().initialize()+ credentials_db = container.credentials_database()+ salt = await credentials_db.initialize()logger.info("Credentials database initialized")+ # When auth is disabled, auto-initialize encryption with a default key+ # This allows saving API keys without user login+ if settings.vite_auth_enabled == "false":+ encryption = container.encryption_service()+ if not encryption.is_initialized():+ # Use api_key_encryption_key as the default password+ encryption.initialize(settings.api_key_encryption_key, salt)+ logger.info("Encryption service auto-initialized (auth disabled)")+# Initialize event waiter with cache service for Redis Streams supportfrom services import event_waiterevent_waiter.set_cache_service(container.cache())diff --git a/server/services/handlers/apify.py b/server/services/handlers/apify.pyindex ce9cc88f..eb353a1c 100644--- a/server/services/handlers/apify.py+++ b/server/services/handlers/apify.py@@ -15,7 +15,7 @@ async def _get_apify_client(context: Dict[str, Any]) -> Optional[ApifyClientAsyn"""Get authenticated Apify client from stored credentials."""from core.container import containerauth_service = container.auth_service()- api_token = await auth_service.get_api_key("apify_api_token")+ api_token = await auth_service.get_api_key("apify", "default")if not api_token:return Nonereturn ApifyClientAsync(api_token)