Skip to content

Self-improving AI Employees for everyone

feat: Transparent proxy on HTTP nodes, useProxy tool schema, docs update

trohitga85ab77ca8cb
diff --git a/CLAUDE.md b/CLAUDE.md
index 6a12e91f..3efae3b9 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -68,6 +68,11 @@ server/services/
├── pricing.py # LLM and API cost calculation (loads config/pricing.json)
├── telegram_service.py # TelegramService with python-telegram-bot long-polling
├── tracked_http.py # HTTPX event hooks for automatic API cost tracking
+├── proxy/ # Residential proxy provider management
+│ ├── __init__.py # Exports get_proxy_service, ProxyService
+│ ├── service.py # ProxyService singleton - provider selection, URL generation
+│ ├── providers.py # TemplateProxyProvider - JSON url_template formatting
+│ └── models.py # ProxyProvider, RoutingRule, SessionType enums
├── deployment/ # Event-driven deployment lifecycle
│ ├── __init__.py
│ ├── state.py # DeploymentState, TriggerInfo dataclasses
@@ -368,14 +373,14 @@ class CacheEntry(SQLModel, table=True):
## Codebase Summary
- **Hybrid architecture**: Node.js + Python + React TypeScript
-- **89 implemented workflow nodes** with clean service separation (6 AI models + 3 AI agents/memory + 13 specialized agents + 1 skill + 4 dedicated tools + 3 search + 16 Android + 3 WhatsApp + 4 Twitter + 2 Telegram + 2 Social + 3 Location + 3 Code + 6 Utility + 6 Document + 2 Chat + 2 Scheduler + 2 Workflow + 7 Google Workspace + 1 Apify)
+- **92 implemented workflow nodes** with clean service separation (6 AI models + 3 AI agents/memory + 13 specialized agents + 1 skill + 4 dedicated tools + 3 search + 16 Android + 3 WhatsApp + 4 Twitter + 2 Telegram + 2 Social + 3 Location + 3 Code + 6 Utility + 6 Document + 2 Chat + 2 Scheduler + 2 Workflow + 7 Google Workspace + 1 Apify + 3 Proxy)
- **WebSocket-First Architecture**: WebSocket as primary frontend-backend communication (119 message handlers)
- **Recent optimizations**: REST APIs replaced with WebSocket, AI endpoints migrated to Python, Android automation integrated
## Architecture Refactoring
The project was completely refactored from schema-based node definitions to explicit INodeProperties interface system inspired by n8n architecture. Key changes:
- **Pure INodeProperties System**: Removed all backward compatibility layers
-- **89 Implemented Node Components**: AI models, agents, skills, location services, Android automation, WhatsApp, Twitter/X, Telegram, social, code execution, HTTP/Webhook utilities, document processing, chat, schedulers, and search
+- **92 Implemented Node Components**: AI models, agents, skills, location services, Android automation, WhatsApp, Twitter/X, Telegram, social, code execution, HTTP/Webhook utilities, document processing, chat, schedulers, search, and proxy
- **WebSocket-First Communication**: 119 WebSocket message handlers replace most REST API calls
- **Resource-Operation Pattern**: Organized by functional categories (AI, Location, Android, WhatsApp)
- **TypeScript-First**: Full type safety with proper interface alignment
@@ -404,6 +409,7 @@ The project was completely refactored from schema-based node definitions to expl
- `src/nodeDefinitions/telegramNodes.ts` - Telegram bot integration (2 nodes: send, receive)
- `src/nodeDefinitions/socialNodes.ts` - Unified social messaging nodes (socialReceive, socialSend)
- `src/nodeDefinitions/codeNodes.ts` - Python, JavaScript, and TypeScript code execution nodes
+- `src/nodeDefinitions/proxyNodes.ts` - Proxy nodes (proxyRequest, proxyConfig, proxyStatus) and shared PROXY_PARAMETERS
- `src/nodeDefinitions/utilityNodes.ts` - HTTP, Webhook, chatTrigger, and console nodes
- `src/nodeDefinitions/documentNodes.ts` - Document processing nodes (httpScraper, fileDownloader, documentParser, textChunker, embeddingGenerator, vectorStore)
- `src/nodeDefinitions/chatNodes.ts` - Chat send and history nodes
@@ -526,7 +532,7 @@ The project uses WebSocket as the primary communication method between frontend
- `server/services/status_broadcaster.py` - Connection management and broadcasting
## Implemented Node Types
-The following 89 nodes are currently implemented and functional:
+The following 92 nodes are currently implemented and functional:
### AI Chat Models (6 nodes)
- **openaiChatModel**: OpenAI GPT models with response format options. O-series models (o1, o3, o4) support reasoning effort parameter.
@@ -594,7 +600,9 @@ server/skills/
│ └── nearby-places-skill/SKILL.md
└── web_agent/ # Web automation skills
├── web-search-skill/SKILL.md
- └── http-request-skill/SKILL.md
+ ├── http-request-skill/SKILL.md
+ ├── proxy-config-skill/SKILL.md
+ └── apify-skill/SKILL.md
```
**SKILL.md Format:**
@@ -1114,6 +1122,29 @@ Web scraping service for social media, search engines, and websites using pre-bu
**Authentication:** Single API token (Personal or Organization) from Apify Console -> Settings -> Integrations.
+### Proxy Nodes (3 nodes)
+Residential proxy provider management with geo-targeting, session control, and automatic failover.
+
+- **proxyRequest**: Make HTTP requests through residential proxy providers with geo-targeting and failover. Parameters: method, url, headers, body, timeout, proxyProvider (auto-select or specific), proxyCountry (ISO code), sessionType (rotating/sticky), stickyDuration, maxRetries, followRedirects.
+- **proxyConfig**: **Dual-purpose node** - Configure proxy providers and routing rules. Works as workflow node OR AI Agent tool. Operations: list_providers, add_provider, update_provider, remove_provider, set_credentials, test_provider, get_stats, add_routing_rule, list_routing_rules, remove_routing_rule.
+- **proxyStatus**: View proxy provider health, scores, and usage statistics. Filter by specific provider name or view all.
+
+**Key Architecture:**
+- **TemplateProxyProvider**: Single class using JSON `url_template` to format proxy URLs for any provider (username-based, password-based, or no encoding)
+- **Auto-selection**: ProxyService ranks providers by health score (success rate, latency) and selects the best one
+- **Transparent proxy on HTTP nodes**: `httpRequest` and `httpScraper` nodes support `useProxy: true` flag. The proxy service handles provider selection, geo-targeting, and session type automatically.
+
+**Key Files:**
+| File | Description |
+|------|-------------|
+| `client/src/nodeDefinitions/proxyNodes.ts` | 3 node definitions + shared PROXY_PARAMETERS |
+| `server/services/proxy/service.py` | ProxyService singleton with provider selection and URL generation |
+| `server/services/proxy/providers.py` | TemplateProxyProvider for JSON url_template formatting |
+| `server/services/proxy/models.py` | ProxyProvider, RoutingRule, SessionType dataclasses |
+| `server/services/handlers/proxy.py` | Handler functions for proxy_config, proxy_request, proxy_status |
+| `server/skills/web_agent/proxy-config-skill/SKILL.md` | AI agent skill for proxy configuration |
+| `server/skills/web_agent/http-request-skill/SKILL.md` | HTTP request skill with proxy usage docs |
+
### Workflow Nodes (2 nodes)
- **start**: Manual workflow trigger to start workflow execution
- **taskTrigger**: Event-driven trigger that fires when a delegated child agent completes its task (success or error). Filters by task_id, agent_name, status (all/completed/error), and parent_node_id. Output includes task_id, status, agent_name, result/error, workflow_id.
@@ -1124,7 +1155,7 @@ Web scraping service for social media, search engines, and websites using pre-bu
- **typescriptExecutor**: **Dual-purpose node** - Execute TypeScript code via persistent Node.js server with type safety, syntax-highlighted editor and console output. Works as workflow node OR AI Agent tool (`typescript_code`).
### Utility Nodes (6 nodes)
-- **httpRequest**: Make HTTP requests to external APIs (GET, POST, PUT, DELETE, PATCH) with configurable headers, body, and timeout
+- **httpRequest**: Make HTTP requests to external APIs (GET, POST, PUT, DELETE, PATCH) with configurable headers, body, timeout, and optional proxy support (`useProxy: true` routes through configured residential proxy)
- **webhookTrigger**: Event-driven trigger that waits for incoming HTTP requests at `/webhook/{path}` with method filtering and authentication options
- **webhookResponse**: Send custom response back to webhook caller with configurable status code, body, and content type
- **chatTrigger**: Trigger node that receives messages from the Console Panel chat interface
@@ -1134,7 +1165,7 @@ Web scraping service for social media, search engines, and websites using pre-bu
### Document Processing Nodes (6 nodes)
RAG pipeline nodes for document ingestion, processing, and vector storage. Supports multiple providers and backends.
-- **httpScraper**: Scrape links from web pages with date/page pagination support. Modes: single request, date range iteration, page pagination. Outputs: items array with URLs.
+- **httpScraper**: Scrape links from web pages with date/page pagination support. Modes: single request, date range iteration, page pagination. Outputs: items array with URLs. Supports `useProxy: true` for proxy routing.
- **fileDownloader**: Download files from URLs in parallel using semaphore-based concurrency. Parameters: output directory, max workers (1-32), skip existing, timeout.
- **documentParser**: Parse documents to text using configurable parsers. Parsers: PyPDF (fast), Marker (GPU OCR), Unstructured (multi-format), BeautifulSoup (HTML).
- **textChunker**: Split text into overlapping chunks for embedding. Strategies: recursive (recommended), markdown, token. Parameters: chunk size (100-8000), overlap (0-1000).
@@ -1340,7 +1371,7 @@ All scripts in `scripts/` are cross-platform Node.js (Windows, macOS, Linux, WSL
See **[Scripts Reference](./docs-internal/SCRIPTS.md)** for full documentation.
## Current Status
-✅ **INodeProperties System**: Fully implemented with 89 functional node components
+✅ **INodeProperties System**: Fully implemented with 92 functional node components
✅ **WebSocket-First Architecture**: 119 message handlers replacing REST APIs
✅ **Code Editor**: Python, JavaScript, and TypeScript executors with syntax-highlighted editor (react-simple-code-editor + prismjs) and console output
✅ **Node.js Executor**: Persistent Node.js server (Express + tsx) for fast JS/TS execution, replacing subprocess spawning
@@ -1372,6 +1403,7 @@ See **[Scripts Reference](./docs-internal/SCRIPTS.md)** for full documentation.
✅ **Cache System**: n8n-pattern cache with Redis (production) / SQLite (local dev) / Memory fallback hierarchy
✅ **AI Thinking/Reasoning**: Extended thinking for Claude, Gemini 2.5/3, OpenAI GPT-5/o-series, Groq Qwen3 with output available in Input Data & Variables for downstream nodes
✅ **Onboarding Service**: 5-step welcome wizard with Ant Design UI, database persistence, skip/resume/replay support
+✅ **Proxy System**: Residential proxy provider management with template-based URL formatting, auto-selection by health score, transparent proxy injection on httpRequest/httpScraper nodes via `useProxy: true`
## Key Features
diff --git a/client/src/Dashboard.tsx b/client/src/Dashboard.tsx
index 1697aabf..6d8a9992 100644
--- a/client/src/Dashboard.tsx
+++ b/client/src/Dashboard.tsx
@@ -35,6 +35,7 @@ import { SEARCH_NODE_TYPES } from './nodeDefinitions/searchNodes';
import { SOCIAL_NODE_TYPES } from './nodeDefinitions/socialNodes';
import { APIFY_NODE_TYPES } from './nodeDefinitions/apifyNodes';
import { GOOGLE_WORKSPACE_NODE_TYPES } from './nodeDefinitions/googleWorkspaceNodes';
+import { PROXY_NODE_TYPES } from './nodeDefinitions/proxyNodes';
import ParameterPanel from './ParameterPanel';
import LocationParameterPanel from './components/LocationParameterPanel';
import { useAppStore } from './store/useAppStore';
@@ -140,6 +141,9 @@ const createNodeTypes = (): Record<string, React.ComponentType<any>> => {
} else if (SEARCH_NODE_TYPES.includes(type)) {
// Search API nodes use SquareNode component
types[type] = SquareNode;
+ } else if (PROXY_NODE_TYPES.includes(type)) {
+ // Proxy nodes use SquareNode component
+ types[type] = SquareNode;
} else if (SOCIAL_NODE_TYPES.includes(type)) {
// Social nodes use AIAgentNode for multiple output handles support
types[type] = AIAgentNode;
diff --git a/client/src/components/parameterPanel/InputSection.tsx b/client/src/components/parameterPanel/InputSection.tsx
index eaac9700..89ff793c 100644
--- a/client/src/components/parameterPanel/InputSection.tsx
+++ b/client/src/components/parameterPanel/InputSection.tsx
@@ -637,6 +637,28 @@ const InputSection: React.FC<InputSectionProps> = ({ nodeId, visible = true }) =
compute_units: 'number',
started_at: 'string',
finished_at: 'string'
+ },
+ // Proxy node schemas
+ proxyRequest: {
+ status: 'number',
+ data: 'any',
+ headers: 'object',
+ url: 'string',
+ method: 'string',
+ proxy_provider: 'string',
+ latency_ms: 'number',
+ bytes_transferred: 'number',
+ attempt: 'number'
+ },
+ proxyStatus: {
+ enabled: 'boolean',
+ providers: 'array',
+ stats: 'object'
+ },
+ proxyConfig: {
+ operation: 'string',
+ success: 'boolean',
+ data: 'any'
}
};
@@ -687,6 +709,10 @@ const InputSection: React.FC<InputSectionProps> = ({ nodeId, visible = true }) =
// Apify web scraping node detection
const isApify = nodeType === 'apifyActor';
+ // Proxy node detection
+ const proxyNodeTypes = ['proxyRequest', 'proxyStatus', 'proxyConfig'];
+ const isProxyNode = proxyNodeTypes.includes(nodeType);
+
// Select appropriate schema
if (isAndroidLocation) {
outputSchema = sampleSchemas.androidLocation;
@@ -696,6 +722,8 @@ const InputSection: React.FC<InputSectionProps> = ({ nodeId, visible = true }) =
outputSchema = sampleSchemas[nodeType as keyof typeof sampleSchemas];
} else if (isGoogleWorkspaceNode && sampleSchemas[nodeType as keyof typeof sampleSchemas]) {
outputSchema = sampleSchemas[nodeType as keyof typeof sampleSchemas];
+ } else if (isProxyNode && sampleSchemas[nodeType as keyof typeof sampleSchemas]) {
+ outputSchema = sampleSchemas[nodeType as keyof typeof sampleSchemas];
} else {
outputSchema = isLocationNode || isGoogleMaps ? sampleSchemas.location :
isMemory ? sampleSchemas.memory :
diff --git a/client/src/components/ui/ComponentPalette.tsx b/client/src/components/ui/ComponentPalette.tsx
index 7697603a..8a9268c0 100644
--- a/client/src/components/ui/ComponentPalette.tsx
+++ b/client/src/components/ui/ComponentPalette.tsx
@@ -26,6 +26,7 @@ const CATEGORY_ICONS: Record<string, string | { svg: string }> = {
search: '🔍',
google: { svg: googleSvg },
scheduler: '📅',
+ proxy: '🛡',
};
// Categories that should be merged into 'social' (Social Media)
@@ -68,6 +69,7 @@ const ComponentPalette: React.FC<ComponentPaletteProps> = ({
search: colors.categoryModel || theme.dracula.cyan,
google: theme.accent.blue,
scheduler: colors.categoryTrigger || theme.dracula.pink,
+ proxy: colors.categoryUtil || theme.dracula.purple,
};
const labelMap: Record<string, string> = {
workflow: 'Workflows',
@@ -88,6 +90,7 @@ const ComponentPalette: React.FC<ComponentPaletteProps> = ({
search: 'Search',
google: 'Google Workspace',
scheduler: 'Schedulers',
+ proxy: 'Proxy',
};
return {
icon: CATEGORY_ICONS[key] || '📦',
diff --git a/client/src/nodeDefinitions.ts b/client/src/nodeDefinitions.ts
index b0630675..d307764f 100644
--- a/client/src/nodeDefinitions.ts
+++ b/client/src/nodeDefinitions.ts
@@ -23,6 +23,7 @@ import { apifyNodes, APIFY_NODE_TYPES } from './nodeDefinitions/apifyNodes';
import { searchNodes, SEARCH_NODE_TYPES } from './nodeDefinitions/searchNodes';
// Consolidated Google Workspace nodes (replaces gmailNodes, calendarNodes, driveNodes, sheetsNodes, tasksNodes, contactsNodes)
import { googleWorkspaceNodes, GOOGLE_WORKSPACE_NODE_TYPES } from './nodeDefinitions/googleWorkspaceNodes';
+import { proxyNodes, PROXY_NODE_TYPES } from './nodeDefinitions/proxyNodes';
// ============================================================================
// MAIN NODE REGISTRY - Combining all modular definitions
@@ -49,7 +50,8 @@ export const nodeDefinitions: Record<string, INodeTypeDescription> = {
...twitterNodes,
...apifyNodes,
...searchNodes,
- ...googleWorkspaceNodes
+ ...googleWorkspaceNodes,
+ ...proxyNodes
};
// ============================================================================
@@ -142,5 +144,9 @@ export const SEARCH_NODES = [
...SEARCH_NODE_TYPES
];
+export const PROXY_NODES = [
+ ...PROXY_NODE_TYPES
+];
+
// Re-export types and utilities from modular files for external access
export { createBaseChatModel } from './nodeDefinitions/aiModelNodes';
diff --git a/client/src/nodeDefinitions/documentNodes.ts b/client/src/nodeDefinitions/documentNodes.ts
index d5f249f7..d999c18a 100644
--- a/client/src/nodeDefinitions/documentNodes.ts
+++ b/client/src/nodeDefinitions/documentNodes.ts
@@ -92,6 +92,31 @@ export const documentNodes: Record<string, INodeTypeDescription> = {
default: '{}',
typeOptions: { rows: 2 },
placeholder: '{"Authorization": "Bearer token"}'
+ },
+ {
+ displayName: 'Use Proxy',
+ name: 'useProxy',
+ type: 'boolean',
+ default: false,
+ description: 'Route requests through a configured proxy provider'
+ },
+ {
+ displayName: 'Proxy Country',
+ name: 'proxyCountry',
+ type: 'string',
+ default: '',
+ placeholder: 'US',
+ description: 'ISO country code for geo-targeting (e.g. US, GB, DE)',
+ displayOptions: { show: { useProxy: [true] } }
+ },
+ {
+ displayName: 'Proxy Provider',
+ name: 'proxyProvider',
+ type: 'string',
+ default: '',
+ placeholder: 'Auto-select',
+ description: 'Specific provider name, or leave empty for auto-selection',
+ displayOptions: { show: { useProxy: [true] } }
}
]
},
diff --git a/client/src/nodeDefinitions/proxyNodes.ts b/client/src/nodeDefinitions/proxyNodes.ts
index 86ab8b24..16a43439 100644
--- a/client/src/nodeDefinitions/proxyNodes.ts
+++ b/client/src/nodeDefinitions/proxyNodes.ts
@@ -13,7 +13,7 @@ export const proxyNodes: Record<string, INodeTypeDescription> = {
displayName: 'Proxy Request',
name: 'proxyRequest',
icon: '🛡',
- group: ['proxy', 'api'],
+ group: ['proxy', 'tool'],
version: 1,
description: 'Make HTTP requests through residential proxy providers with geo-targeting and failover',
defaults: { name: 'Proxy Request', color: '#8B5CF6' },
@@ -295,7 +295,7 @@ export const proxyNodes: Record<string, INodeTypeDescription> = {
displayName: 'Proxy Status',
name: 'proxyStatus',
icon: '📊',
- group: ['proxy', 'utility'],
+ group: ['proxy', 'tool'],
version: 1,
description: 'View proxy provider health, scores, and usage statistics',
defaults: { name: 'Proxy Status', color: '#6366F1' },
diff --git a/client/src/nodeDefinitions/utilityNodes.ts b/client/src/nodeDefinitions/utilityNodes.ts
index 91befabb..f696708b 100644
--- a/client/src/nodeDefinitions/utilityNodes.ts
+++ b/client/src/nodeDefinitions/utilityNodes.ts
@@ -64,6 +64,42 @@ export const utilityNodes: Record<string, INodeTypeDescription> = {
name: 'timeout',
type: 'number',
default: 30
+ },
+ {
+ displayName: 'Use Proxy',
+ name: 'useProxy',
+ type: 'boolean',
+ default: false,
+ description: 'Route request through a configured proxy provider'
+ },
+ {
+ displayName: 'Proxy Country',
+ name: 'proxyCountry',
+ type: 'string',
+ default: '',
+ placeholder: 'US',
+ description: 'ISO country code for geo-targeting (e.g. US, GB, DE)',
+ displayOptions: { show: { useProxy: [true] } }
+ },
+ {
+ displayName: 'Proxy Provider',
+ name: 'proxyProvider',
+ type: 'string',
+ default: '',
+ placeholder: 'Auto-select',
+ description: 'Specific provider name, or leave empty for auto-selection',
+ displayOptions: { show: { useProxy: [true] } }
+ },
+ {
+ displayName: 'Session Type',
+ name: 'sessionType',
+ type: 'options',
+ default: 'rotating',
+ options: [
+ { name: 'Rotating (new IP per request)', value: 'rotating' },
+ { name: 'Sticky (same IP)', value: 'sticky' }
+ ],
+ displayOptions: { show: { useProxy: [true] } }
}
]
},
diff --git a/client/src/services/executionService.ts b/client/src/services/executionService.ts
index 76161919..ee33af92 100644
--- a/client/src/services/executionService.ts
+++ b/client/src/services/executionService.ts
@@ -238,7 +238,11 @@ export class ExecutionService {
// Search API nodes
...SEARCH_NODE_TYPES,
// Telegram Nodes
- ...TELEGRAM_NODE_TYPES
+ ...TELEGRAM_NODE_TYPES,
+ // Proxy Nodes
+ 'proxyRequest',
+ 'proxyConfig',
+ 'proxyStatus'
];
return supportedTypes.includes(nodeType);
diff --git a/docs-internal/node_creation.md b/docs-internal/node_creation.md
index 1a8f56d9..5ed6febb 100644
--- a/docs-internal/node_creation.md
+++ b/docs-internal/node_creation.md
@@ -26,7 +26,7 @@ This guide explains how to create new nodes for MachinaOs, covering frontend def
## Node Types Overview
-MachinaOs supports 108 nodes across 18 categories:
+MachinaOs supports 111 nodes across 19 categories:
| Category | Visual Component | Example Nodes | Count |
|----------|-----------------|---------------|-------|
@@ -46,6 +46,7 @@ MachinaOs supports 108 nodes across 18 categories:
| **Social** | SquareNode | socialReceive, socialSend | 2 |
| **Location/Maps** | SquareNode | createMap, addLocations, showNearbyPlaces | 3 |
| **Utility** | SquareNode | httpRequest, webhookTrigger, webhookResponse, chatTrigger, console | 5 |
+| **Proxy** | SquareNode | proxyRequest, proxyConfig, proxyStatus | 3 |
| **Code** | SquareNode | pythonExecutor, javascriptExecutor | 2 |
| **Chat** | SquareNode | chatSend, chatHistory | 2 |
| **Document** | SquareNode | httpScraper, fileDownloader, documentParser, textChunker, embeddingGenerator, vectorStore | 6 |
diff --git a/docs-internal/workflow-schema.md b/docs-internal/workflow-schema.md
index 9e46abb4..9eb82835 100644
--- a/docs-internal/workflow-schema.md
+++ b/docs-internal/workflow-schema.md
@@ -59,7 +59,7 @@ A workflow JSON document contains:
}
```
-## Supported Node Types (64 Total)
+## Supported Node Types (67 Total)
### Workflow Nodes (1 node)
- `start` - Workflow entry point with initial data
@@ -149,12 +149,17 @@ Dual-purpose nodes (workflow + AI tool). Defined in `searchNodes.ts`:
- `mediaControl` - Media playback, volume
### Utility Nodes (5 nodes)
-- `httpRequest` - HTTP requests (GET, POST, PUT, DELETE, PATCH)
+- `httpRequest` - HTTP requests (GET, POST, PUT, DELETE, PATCH) with optional proxy support (`useProxy: true`)
- `webhookTrigger` - Incoming HTTP webhook trigger at `/webhook/{path}`
- `webhookResponse` - Custom response to webhook caller
- `chatTrigger` - Console message input trigger
- `console` - Debug logging output
+### Proxy Nodes (3 nodes)
+- `proxyRequest` - HTTP requests through residential proxy providers with geo-targeting and failover
+- `proxyConfig` - Configure proxy providers, credentials, and routing rules
+- `proxyStatus` - View proxy provider health, scores, and usage statistics
+
### Code Nodes (2 nodes)
- `pythonExecutor` - Python code execution with input_data access
- `javascriptExecutor` - JavaScript code execution
diff --git a/scripts/dev.js b/scripts/dev.js
index 2049844e..76ace58b 100644
--- a/scripts/dev.js
+++ b/scripts/dev.js
@@ -71,14 +71,13 @@ async function main() {
}
// Start services
- log('Starting services...');
- log('Press Ctrl+C to stop\n');
+ log('Spawning services...');
const clientDist = resolve(ROOT, 'client', 'dist', 'index.html');
const hasVite = existsSync(resolve(ROOT, 'node_modules', 'vite'))
|| existsSync(resolve(ROOT, 'client', 'node_modules', 'vite'));
const isProduction = existsSync(clientDist) && !hasVite;
- log(`Mode: ${isProduction ? 'Production' : 'Development'}`);
+ log(`Client: ${isProduction ? 'Static server' : 'Vite dev server'}`);
const services = [];
if (isProduction) {
@@ -90,13 +89,41 @@ async function main() {
if (!skipWhatsApp) services.push('npm:whatsapp:api');
if (config.temporalEnabled) services.push('npm:temporal:worker');
+ // Ready-detection patterns for each service
+ const readyPatterns = [
+ { name: 'Client', pattern: /ready in|VITE.*ready|Client:\s*http/i },
+ { name: 'Backend', pattern: /Application startup complete|Uvicorn running/i },
+ ];
+ if (!skipWhatsApp) {
+ readyPatterns.push({ name: 'WhatsApp', pattern: /listening on|WhatsApp.*ready|API.*started|:9400/i });
+ }
+ const readySet = new Set();
+ const totalExpected = readyPatterns.length;
+
const proc = spawn('npx', ['concurrently', '--raw', '--kill-others', ...services], {
cwd: ROOT,
- stdio: 'inherit',
+ stdio: ['inherit', 'pipe', 'pipe'],
shell: true,
env: { ...process.env, FORCE_COLOR: '1' }
});
+ proc.stdout.on('data', (data) => {
+ process.stdout.write(data);
+ const text = data.toString();
+ for (const { name, pattern } of readyPatterns) {
+ if (!readySet.has(name) && pattern.test(text)) {
+ readySet.add(name);
+ log(`${name} ready`);
+ if (readySet.size === totalExpected) {
+ log(`All services ready -- http://localhost:${config.clientPort}`);
+ console.log('');
+ }
+ }
+ }
+ });
+
+ proc.stderr.on('data', (data) => process.stderr.write(data));
+
process.on('SIGINT', () => proc.kill('SIGINT'));
process.on('SIGTERM', () => proc.kill('SIGTERM'));
proc.on('close', (code) => {
diff --git a/scripts/start.js b/scripts/start.js
index f6d669cf..cdbe3b94 100755
--- a/scripts/start.js
+++ b/scripts/start.js
@@ -14,8 +14,12 @@ import {
loadEnvConfig,
ensureEnvFile,
killPort,
+ createLogger,
} from './utils.js';
+const START_TIME = Date.now();
+const log = createLogger(START_TIME);
+
const args = process.argv.slice(2);
const isVerbose = args.includes('--verbose') || args.includes('-v');
const skipWhatsApp = args.includes('--skip-whatsapp');
@@ -45,9 +49,11 @@ async function main() {
ensureEnvFile();
// Free ports
+ log('Freeing ports...');
for (const port of config.allPorts) {
await killPort(port);
}
+ log('Ports ready');
// Services: static client, backend (uvicorn), whatsapp, temporal
const services = [];
@@ -56,15 +62,44 @@ async function main() {
if (!skipWhatsApp) services.push('npm:whatsapp:api');
if (config.temporalEnabled) services.push('npm:temporal:worker');
+ // Ready-detection patterns for each service
+ const readyPatterns = [
+ { name: 'Client', pattern: /ready in|VITE.*ready|Client:\s*http/i },
+ { name: 'Backend', pattern: /Application startup complete|Uvicorn running/i },
+ ];
+ if (!skipWhatsApp) {
+ readyPatterns.push({ name: 'WhatsApp', pattern: /listening on|WhatsApp.*ready|API.*started|:9400/i });
+ }
+ const readySet = new Set();
+ const totalExpected = readyPatterns.length;
+
+ function checkReady(text) {
+ for (const { name, pattern } of readyPatterns) {
+ if (!readySet.has(name) && pattern.test(text)) {
+ readySet.add(name);
+ log(`${name} ready`);
+ if (readySet.size === totalExpected) {
+ log(`All services ready`);
+ }
+ }
+ }
+ }
+
if (isVerbose) {
console.log('\n=== MachinaOS Starting (verbose) ===\n');
const proc = spawn('npx', ['concurrently', '--raw', '--kill-others', ...services], {
cwd: ROOT,
- stdio: 'inherit',
+ stdio: ['inherit', 'pipe', 'pipe'],
shell: true,
env: { ...process.env, FORCE_COLOR: '1' },
});
+ proc.stdout.on('data', (data) => {
+ process.stdout.write(data);
+ checkReady(data.toString());
+ });
+ proc.stderr.on('data', (data) => process.stderr.write(data));
+
process.on('SIGINT', () => proc.kill('SIGINT'));
process.on('SIGTERM', () => proc.kill('SIGTERM'));
proc.on('close', (code) => process.exit(code || 0));
@@ -96,7 +131,9 @@ async function main() {
const essential = /error|fatal|exception|failed|started|ready|listening|running|address already in use|application startup complete/i;
proc.stdout.on('data', (data) => {
- for (const line of data.toString().split('\n')) {
+ const text = data.toString();
+ checkReady(text);
+ for (const line of text.split('\n')) {
const trimmed = line.trim();
if (trimmed && essential.test(trimmed)) {
process.stdout.write(line + '\n');
diff --git a/scripts/utils.js b/scripts/utils.js
index 5a71aed6..2bbe5653 100644
--- a/scripts/utils.js
+++ b/scripts/utils.js
@@ -27,7 +27,9 @@ export const isWSL = !isWindows && process.platform === 'linux' && (
);
// True when we should use Windows-style venv paths (Scripts/python.exe)
-export const useWindowsVenv = isWindows || isWSL;
+// WSL excluded: even with a Windows venv, WSL should prefer system python3
+// for port management (Windows python.exe via interop gives wrong process info)
+export const useWindowsVenv = isWindows && !isWSL;
// Cached Python path (undefined = not checked, null = unavailable, string = path)
let _pythonCmd;
diff --git a/server/constants.py b/server/constants.py
index 802d573b..abd1c097 100644
--- a/server/constants.py
+++ b/server/constants.py
@@ -80,6 +80,23 @@ CONFIG_NODE_TYPES: FrozenSet[str] = (
SKILL_NODE_TYPES # Skill nodes (connect to Zeenie's input-skill)
)
+# =============================================================================
+# PROXY NODE TYPES
+# =============================================================================
+
+PROXY_NODE_TYPES: FrozenSet[str] = frozenset([
+ 'proxyRequest',
+ 'proxyConfig',
+ 'proxyStatus',
+])
+
+# Dual-purpose proxy nodes (workflow node + AI tool)
+PROXY_TOOL_TYPES: FrozenSet[str] = frozenset([
+ 'proxyRequest',
+ 'proxyConfig',
+ 'proxyStatus',
+])
+
# =============================================================================
# SEARCH NODE TYPES
# =============================================================================
diff --git a/server/core/config.py b/server/core/config.py
index e5083602..adcd8c4b 100644
--- a/server/core/config.py
+++ b/server/core/config.py
@@ -67,6 +67,11 @@ class Settings(BaseSettings):
# AI Proxy Configuration (Ollama-style proxy)
ai_proxy_default_port: int = Field(default=11434, env="AI_PROXY_DEFAULT_PORT")
+ # Residential Proxy Configuration
+ proxy_enabled: bool = Field(default=False, env="PROXY_ENABLED")
+ proxy_budget_daily_usd: float = Field(default=50.0, env="PROXY_BUDGET_DAILY_USD")
+ proxy_default_country: str = Field(default="", env="PROXY_DEFAULT_COUNTRY")
+
# WebSocket Configuration
websocket_url: str = Field(default="", env="WEBSOCKET_URL")
websocket_api_key: Optional[str] = Field(default=None, env="WEBSOCKET_API_KEY")
diff --git a/server/core/container.py b/server/core/container.py
index 84661c81..90ba36b0 100644
--- a/server/core/container.py
+++ b/server/core/container.py
@@ -1,5 +1,15 @@
"""Dependency injection container for the application."""
+import time as _time
+
+_ct0 = _time.perf_counter()
+
+
+def _clog(msg):
+ elapsed = (_time.perf_counter() - _ct0) * 1000
+ print(f" container: {msg} ({elapsed:.0f}ms)", flush=True)
+
+
from dependency_injector import containers, providers
from core.config import Settings
@@ -7,14 +17,18 @@ from core.database import Database
from core.cache import CacheService
from core.encryption import EncryptionService
from core.credentials_database import CredentialsDatabase
+_clog("core imports done")
from services.ai import AIService
+_clog("AIService imported")
from services.maps import MapsService
from services.workflow import WorkflowService
+_clog("WorkflowService imported")
from services.auth import AuthService
from services.text import TextService
from services.android_service import AndroidService
from services.user_auth import UserAuthService
from services.compaction import init_compaction_service
+_clog("all service imports done")
# Temporal is optional - import with fallback
try:
diff --git a/server/core/database.py b/server/core/database.py
index 67d46111..442338ae 100644
--- a/server/core/database.py
+++ b/server/core/database.py
@@ -13,7 +13,8 @@ from models.database import (
NodeParameter, Workflow, Execution, APIKey, APIKeyValidation, NodeOutput,
ConversationMessage, ToolSchema, UserSkill, ChatMessage, UserSettings,
TokenUsageMetric, CompactionEvent, SessionTokenState, ProviderDefaults,
- AgentTeam, TeamMember, TeamTask, AgentMessage, GoogleConnection
+ AgentTeam, TeamMember, TeamTask, AgentMessage, GoogleConnection,
+ ProxyProviderConfig, ProxyRoutingRule
)
from models.cache import CacheEntry # SQLite-backed cache for Redis alternative
from core.logging import get_logger
@@ -2579,4 +2580,187 @@ class Database:
return False
except Exception as e:
logger.error(f"Failed to update Google last used: {e}")
+ return False
+
+ # ============================================================================
+ # Proxy Provider CRUD
+ # ============================================================================
+
+ async def save_proxy_provider(self, data: Dict[str, Any]) -> bool:
+ """Save or update a proxy provider configuration (upsert by name)."""
+ try:
+ name = data.get("name")
+ if not name:
+ logger.error("Cannot save proxy provider without name")
+ return False
+
+ async with self.get_session() as session:
+ stmt = select(ProxyProviderConfig).where(ProxyProviderConfig.name == name)
+ result = await session.execute(stmt)
+ existing = result.scalar_one_or_none()
+
+ if existing:
+ for field in ["enabled", "priority", "cost_per_gb", "gateway_host", "gateway_port", "url_template"]:
+ if field in data:
+ setattr(existing, field, data[field])
+ existing.updated_at = datetime.now(timezone.utc)
+ else:
+ existing = ProxyProviderConfig(
+ name=name,
+ enabled=data.get("enabled", True),
+ priority=data.get("priority", 50),
+ cost_per_gb=data.get("cost_per_gb", 0.0),
+ gateway_host=data.get("gateway_host", ""),
+ gateway_port=data.get("gateway_port", 0),
+ url_template=data.get("url_template", "{}"),
+ )
+ session.add(existing)
+
+ await session.commit()
+ logger.info(f"[DB] Proxy provider saved: {name}")
+ return True
+
+ except Exception as e:
+ logger.error(f"Failed to save proxy provider: {e}")
+ return False
+
+ async def get_proxy_provider(self, name: str) -> Optional[Dict[str, Any]]:
+ """Get a single proxy provider by name."""
+ try:
+ async with self.get_session() as session:
+ stmt = select(ProxyProviderConfig).where(ProxyProviderConfig.name == name)
+ result = await session.execute(stmt)
+ p = result.scalar_one_or_none()
+
+ if not p:
+ return None
+
+ return {
+ "id": p.id,
+ "name": p.name,
+ "enabled": p.enabled,
+ "priority": p.priority,
+ "cost_per_gb": p.cost_per_gb,
+ "gateway_host": p.gateway_host,
+ "gateway_port": p.gateway_port,
+ "url_template": p.url_template,
+ "created_at": p.created_at.isoformat() if p.created_at else None,
+ "updated_at": p.updated_at.isoformat() if p.updated_at else None,
+ }
+
+ except Exception as e:
+ logger.error(f"Failed to get proxy provider: {e}")
+ return None
+
+ async def get_proxy_providers(self) -> List[Dict[str, Any]]:
+ """Get all proxy provider configurations."""
+ try:
+ async with self.get_session() as session:
+ stmt = select(ProxyProviderConfig).order_by(ProxyProviderConfig.priority)
+ result = await session.execute(stmt)
+ providers = result.scalars().all()
+
+ return [
+ {
+ "id": p.id,
+ "name": p.name,
+ "enabled": p.enabled,
+ "priority": p.priority,
+ "cost_per_gb": p.cost_per_gb,
+ "gateway_host": p.gateway_host,
+ "gateway_port": p.gateway_port,
+ "url_template": p.url_template,
+ "created_at": p.created_at.isoformat() if p.created_at else None,
+ "updated_at": p.updated_at.isoformat() if p.updated_at else None,
+ }
+ for p in providers
+ ]
+
+ except Exception as e:
+ logger.error(f"Failed to get proxy providers: {e}")
+ return []
+
+ async def delete_proxy_provider(self, name: str) -> bool:
+ """Delete a proxy provider by name."""
+ try:
+ async with self.get_session() as session:
+ stmt = select(ProxyProviderConfig).where(ProxyProviderConfig.name == name)
+ result = await session.execute(stmt)
+ provider = result.scalar_one_or_none()
+
+ if provider:
+ await session.delete(provider)
+ await session.commit()
+ logger.info(f"[DB] Proxy provider deleted: {name}")
+
+ return True
+
+ except Exception as e:
+ logger.error(f"Failed to delete proxy provider: {e}")
+ return False
+
+ # ============================================================================
+ # Proxy Routing Rules CRUD
+ # ============================================================================
+
+ async def save_proxy_routing_rule(self, data: Dict[str, Any]) -> bool:
+ """Save a new proxy routing rule."""
+ try:
+ async with self.get_session() as session:
+ rule = ProxyRoutingRule(
+ domain_pattern=data.get("domain_pattern", ""),
+ preferred_providers=data.get("preferred_providers", "[]"),
+ required_country=data.get("required_country", ""),
+ session_type=data.get("session_type", "rotating"),
+ )
+ session.add(rule)
+ await session.commit()
+ logger.info(f"[DB] Proxy routing rule saved: {data.get('domain_pattern')}")
+ return True
+
+ except Exception as e:
+ logger.error(f"Failed to save proxy routing rule: {e}")
+ return False
+
+ async def get_proxy_routing_rules(self) -> List[Dict[str, Any]]:
+ """Get all proxy routing rules."""
+ try:
+ async with self.get_session() as session:
+ stmt = select(ProxyRoutingRule).order_by(ProxyRoutingRule.id)
+ result = await session.execute(stmt)
+ rules = result.scalars().all()
+
+ return [
+ {
+ "id": r.id,
+ "domain_pattern": r.domain_pattern,
+ "preferred_providers": r.preferred_providers,
+ "required_country": r.required_country,
+ "session_type": r.session_type,
+ "created_at": r.created_at.isoformat() if r.created_at else None,
+ }
+ for r in rules
+ ]
+
+ except Exception as e:
+ logger.error(f"Failed to get proxy routing rules: {e}")
+ return []
+
+ async def delete_proxy_routing_rule(self, rule_id: int) -> bool:
+ """Delete a proxy routing rule by ID."""
+ try:
+ async with self.get_session() as session:
+ stmt = select(ProxyRoutingRule).where(ProxyRoutingRule.id == rule_id)
+ result = await session.execute(stmt)
+ rule = result.scalar_one_or_none()
+
+ if rule:
+ await session.delete(rule)
+ await session.commit()
+ logger.info(f"[DB] Proxy routing rule deleted: {rule_id}")
+
+ return True
+
+ except Exception as e:
+ logger.error(f"Failed to delete proxy routing rule: {e}")
return False
\ No newline at end of file
diff --git a/server/main.py b/server/main.py
index 3c7c7f88..5fc48a75 100644
--- a/server/main.py
+++ b/server/main.py
@@ -4,6 +4,15 @@ Modern FastAPI backend for React Flow workflow automation platform.
Refactored with dependency injection, modular services, and clean architecture.
"""
+import time as _time
+_t0 = _time.perf_counter()
+
+
+def _startup_log(msg):
+ elapsed = _time.perf_counter() - _t0
+ print(f" [{elapsed:6.2f}s] {msg}", flush=True)
+
+
# Performance: Install uvloop if available (Linux/macOS only)
try:
import uvloop
@@ -19,14 +28,19 @@ from contextlib import asynccontextmanager
# Adding custom handlers that raise KeyboardInterrupt causes cascading errors
# during async operations (WebSocket handlers, logging, etc.).
+_startup_log("Importing FastAPI...")
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import ORJSONResponse
+_startup_log("Importing DI container + all services...")
from core.container import container
+_startup_log("Importing settings + logging...")
from core.config import Settings
from core.logging import configure_logging, get_logger, setup_websocket_logging, shutdown_websocket_logging
+_startup_log("Importing routers...")
from routers import workflow, database, maps, nodejs_compat, android, websocket, webhook, auth, twitter, google
+_startup_log("All imports complete")
# Initialize settings and logging
settings = Settings()
@@ -48,7 +62,7 @@ logging.getLogger("watchfiles").setLevel(logging.WARNING)
async def lifespan(app: FastAPI):
"""Application lifespan management."""
# Startup
- logger.info("Starting React Flow Python Services")
+ _startup_log("Lifespan startup begin")
# Wire dependency injection
container.wire(modules=[
@@ -68,6 +82,7 @@ async def lifespan(app: FastAPI):
# Start services
await container.database().startup()
await container.cache().startup()
+ _startup_log("Database + cache started")
# Initialize credentials database (creates tables if not exist)
credentials_db = container.credentials_database()
@@ -80,6 +95,7 @@ async def lifespan(app: FastAPI):
if not encryption.is_initialized():
encryption.initialize(settings.api_key_encryption_key, salt)
logger.info("Encryption service initialized")
+ _startup_log("Credentials + encryption initialized")
# Initialize event waiter with cache service for Redis Streams support
from services import event_waiter
@@ -133,6 +149,7 @@ async def lifespan(app: FastAPI):
compaction_svc = container.compaction_service() # Trigger singleton initialization
compaction_svc.set_ai_service(container.ai_service())
logger.info("Compaction service initialized")
+ _startup_log("Compaction service ready")
# Initialize model registry service
from services.model_registry import get_model_registry
@@ -156,6 +173,16 @@ async def lifespan(app: FastAPI):
init_agent_team_service(container.database(), get_status_broadcaster())
logger.info("Agent team service initialized")
+ # Initialize proxy service (loads providers from DB, reads credentials)
+ from services.proxy.service import init_proxy_service
+ proxy_svc = init_proxy_service(
+ auth_service=container.auth_service(),
+ database=container.database(),
+ settings=settings,
+ )
+ await proxy_svc.startup()
+ _startup_log("Proxy service initialized")
+
# Record startup time for health reporting
set_startup_time()
@@ -203,7 +230,8 @@ async def lifespan(app: FastAPI):
logger.warning(f"Temporal not available (optional): {str(e)}")
logger.info("Continuing without Temporal - using local parallel/sequential execution")
- logger.info("Services started successfully")
+ _startup_log("All services initialized")
+ print("Application startup complete", flush=True)
yield
# Shutdown
@@ -224,6 +252,12 @@ async def lifespan(app: FastAPI):
except Exception:
pass
+ # Shutdown proxy service
+ from services.proxy.service import get_proxy_service
+ _proxy_svc = get_proxy_service()
+ if _proxy_svc:
+ await _proxy_svc.shutdown()
+
# Close Android relay client (prevents "Unclosed client session" warning)
from services.android.manager import close_relay_client
await close_relay_client(clear_stored_session=False)
diff --git a/server/models/database.py b/server/models/database.py
index 7eb05244..3a0fdb7a 100644
--- a/server/models/database.py
+++ b/server/models/database.py
@@ -500,6 +500,53 @@ class AgentMessage(SQLModel, table=True):
)
+class ProxyProviderConfig(SQLModel, table=True):
+ """Proxy provider configuration stored in database.
+
+ Each row represents a configured residential proxy provider with
+ gateway endpoint, url_template for parameter encoding, and metadata.
+ """
+
+ __tablename__ = "proxy_providers"
+
+ id: Optional[int] = Field(default=None, primary_key=True)
+ name: str = Field(unique=True, index=True, max_length=255)
+ enabled: bool = Field(default=True)
+ priority: int = Field(default=50) # Lower = preferred
+ cost_per_gb: float = Field(default=0.0)
+ gateway_host: str = Field(default="", max_length=500)
+ gateway_port: int = Field(default=0)
+ url_template: str = Field(default="{}", max_length=5000) # JSON template config
+ created_at: datetime = Field(
+ default_factory=lambda: datetime.now(timezone.utc),
+ sa_column=Column(DateTime(timezone=True), server_default=func.now())
+ )
+ updated_at: datetime = Field(
+ default_factory=lambda: datetime.now(timezone.utc),
+ sa_column=Column(DateTime(timezone=True), onupdate=func.now())
+ )
+
+
+class ProxyRoutingRule(SQLModel, table=True):
+ """Domain-based routing rules for proxy providers.
+
+ Routes specific domain patterns through preferred providers
+ with optional geo-targeting and session type requirements.
+ """
+
+ __tablename__ = "proxy_routing_rules"
+
+ id: Optional[int] = Field(default=None, primary_key=True)
+ domain_pattern: str = Field(max_length=500) # e.g., *.linkedin.com
+ preferred_providers: str = Field(default="[]", max_length=2000) # JSON array
+ required_country: str = Field(default="", max_length=10)
+ session_type: str = Field(default="rotating", max_length=20)
+ created_at: datetime = Field(
+ default_factory=lambda: datetime.now(timezone.utc),
+ sa_column=Column(DateTime(timezone=True), server_default=func.now())
+ )
+
+
class GoogleConnection(SQLModel, table=True):
"""Google Workspace OAuth connections for customer access mode.
diff --git a/server/services/ai.py b/server/services/ai.py
index e3758a60..cf6f0b8e 100644
--- a/server/services/ai.py
+++ b/server/services/ai.py
@@ -8,9 +8,21 @@ from datetime import datetime
from typing import Dict, Any, List, Optional, Callable, Type, TypedDict, Annotated, Sequence
import operator
+_ai_t0 = time.perf_counter()
+
+
+def _ailog(msg):
+ elapsed = (time.perf_counter() - _ai_t0) * 1000
+ print(f" ai.py: {msg} ({elapsed:.0f}ms)", flush=True)
+
+
+_ailog("importing langchain_openai...")
from langchain_openai import ChatOpenAI
+_ailog("importing langchain_anthropic...")
from langchain_anthropic import ChatAnthropic
+_ailog("importing langchain_groq...")
from langchain_groq import ChatGroq
+_ailog("importing langchain_core.messages...")
from langchain_core.messages import HumanMessage, SystemMessage, AIMessage, BaseMessage, ToolMessage
# Lazy import for Google GenAI (google-generativeai init hangs on Windows/Python 3.13)
@@ -26,15 +38,18 @@ def _get_google_genai_class():
# Conditional import for Cerebras (requires Python <3.13)
CEREBRAS_IMPORT_ERROR = None
try:
+ _ailog("importing langchain_cerebras...")
from langchain_cerebras import ChatCerebras
CEREBRAS_AVAILABLE = True
except ImportError as e:
ChatCerebras = None
CEREBRAS_AVAILABLE = False
CEREBRAS_IMPORT_ERROR = str(e)
+_ailog("importing langgraph + pydantic...")
from langchain_core.tools import StructuredTool
from langgraph.graph import StateGraph, END
from pydantic import BaseModel, Field, create_model
+_ailog("all LangChain imports done")
import json
from core.config import Settings
@@ -815,6 +830,64 @@ def build_agent_graph(chat_model, tools: List = None, tool_executor: Callable =
return graph.compile()
+def _build_skill_system_prompt(skill_data: List[Dict[str, Any]], log_prefix: str = "[Agent]") -> tuple:
+ """Build skill injection text for the system message.
+
+ Personality skills (names ending in '-personality') get their FULL SKILL.md
+ instructions injected. All other skills get brief registry descriptions only.
+
+ Args:
+ skill_data: List of skill entries from _collect_agent_connections.
+ log_prefix: Log prefix for debug messages.
+
+ Returns:
+ Tuple of (prompt_text, has_personality). prompt_text is the string to
+ append to system_message. has_personality indicates whether any
+ personality skills were found (used to drop the default system message).
+ """
+ if not skill_data:
+ return "", False
+
+ from services.skill_loader import get_skill_loader
+
+ skill_loader = get_skill_loader()
+ skill_loader.scan_skills()
+
+ personality_blocks = []
+ non_personality_names = []
+
+ for skill_info in skill_data:
+ skill_name = skill_info.get('skill_name') or skill_info.get('node_type', '').replace('Skill', '-skill').lower()
+ if skill_name.endswith('skill') and '-' not in skill_name:
+ skill_name = skill_name[:-5] + '-skill'
+
+ if skill_name.endswith('-personality'):
+ instructions = skill_info.get('parameters', {}).get('instructions', '')
+ if instructions:
+ personality_blocks.append(instructions)
+ logger.debug(f"{log_prefix} Personality skill injected (full): {skill_name}")
+ else:
+ logger.warning(f"{log_prefix} Personality skill {skill_name} has no instructions")
+ else:
+ non_personality_names.append(skill_name)
+ logger.debug(f"{log_prefix} Skill detected: {skill_name}")
+
+ parts = []
+
+ for block in personality_blocks:
+ parts.append(block)
+
+ if non_personality_names:
+ registry_prompt = skill_loader.get_registry_prompt(non_personality_names)
+ if registry_prompt:
+ parts.append(registry_prompt)
+
+ if parts:
+ logger.debug(f"{log_prefix} Enhanced system message: {len(personality_blocks)} personality, {len(non_personality_names)} standard skills")
+
+ return "\n\n".join(parts), len(personality_blocks) > 0
+
+
class AIService:
"""AI model service for LangChain operations."""
@@ -1493,29 +1566,13 @@ class AIService:
prompt = parameters.get('prompt', 'Hello')
system_message = parameters.get('systemMessage', 'You are a helpful assistant')
- # Load skills and enhance system message with SKILL.md context
- # Skills provide instructions/context - actual tools come from tool nodes
- if skill_data:
- from services.skill_loader import get_skill_loader
-
- skill_loader = get_skill_loader()
- skill_loader.scan_skills()
-
- # Extract skill names from connected skill nodes
- skill_names = []
- for skill_info in skill_data:
- skill_name = skill_info.get('skill_name') or skill_info.get('node_type', '').replace('Skill', '-skill').lower()
- # Convert node type to skill name (e.g., 'whatsappSkill' -> 'whatsapp-skill')
- if skill_name.endswith('skill') and '-' not in skill_name:
- skill_name = skill_name[:-5] + '-skill' # whatsappskill -> whatsapp-skill
- skill_names.append(skill_name)
- logger.debug(f"[AIAgent] Skill detected: {skill_name}")
-
- # Add skill SKILL.md content to system message
- skill_prompt = skill_loader.get_registry_prompt(skill_names)
- if skill_prompt:
+ # Inject skills: personality skills get full instructions, others get brief descriptions
+ skill_prompt, has_personality = _build_skill_system_prompt(skill_data, log_prefix="[AIAgent]")
+ if skill_prompt:
+ if has_personality:
+ system_message = skill_prompt
+ else:
system_message = f"{system_message}\n\n{skill_prompt}"
- logger.debug(f"[AIAgent] Enhanced system message with {len(skill_names)} skill contexts")
# Flatten options collection from frontend
options = parameters.get('options', {})
@@ -1656,6 +1713,15 @@ class AIService:
tool_configs[check_tool.name] = check_config
logger.debug("[LangGraph] Auto-injected check_delegated_tasks tool")
+ # Add delegation guidance to system message
+ delegate_names = [n for n in tool_configs if n.startswith('delegate_to_')]
+ system_message += (
+ "\n\n## Agent Delegation\n"
+ "When delegating to sub-agents, use 'task' for the mission directive "
+ "(role and goal) and 'context' for input data the agent needs to work with.\n"
+ f"Available agents: {', '.join(delegate_names)}"
+ )
+
# Create tool executor callback
async def tool_executor(tool_name: str, tool_args: Dict) -> Any:
"""Execute a tool by name."""
@@ -1947,29 +2013,13 @@ class AIService:
prompt = parameters.get('prompt', 'Hello')
system_message = parameters.get('systemMessage', 'You are a helpful assistant')
- # Load skills and enhance system message with SKILL.md context
- # Skills only provide instructions/context - actual tools come from direct tool nodes
- if skill_data:
- from services.skill_loader import get_skill_loader
-
- skill_loader = get_skill_loader()
- skill_loader.scan_skills()
-
- # Extract skill names from connected skill nodes
- skill_names = []
- for skill_info in skill_data:
- skill_name = skill_info.get('skill_name') or skill_info.get('node_type', '').replace('Skill', '-skill').lower()
- # Convert node type to skill name (e.g., 'whatsappSkill' -> 'whatsapp-skill')
- if skill_name.endswith('skill') and '-' not in skill_name:
- skill_name = skill_name[:-5] + '-skill' # whatsappskill -> whatsapp-skill
- skill_names.append(skill_name)
- logger.debug(f"[ChatAgent] Skill detected: {skill_name}")
-
- # Add skill SKILL.md content to system message
- skill_prompt = skill_loader.get_registry_prompt(skill_names)
- if skill_prompt:
+ # Inject skills: personality skills get full instructions, others get brief descriptions
+ skill_prompt, has_personality = _build_skill_system_prompt(skill_data, log_prefix="[ChatAgent]")
+ if skill_prompt:
+ if has_personality:
+ system_message = skill_prompt
+ else:
system_message = f"{system_message}\n\n{skill_prompt}"
- logger.debug(f"[ChatAgent] Enhanced system message with {len(skill_names)} skill contexts")
# Build tools from tool_data using same method as AI Agent
# This supports ALL tool types: calculatorTool, currentTimeTool, duckduckgoSearch, androidTool, httpRequest
@@ -2005,6 +2055,15 @@ class AIService:
tool_node_configs[check_tool.name] = check_config
logger.debug("[ChatAgent] Auto-injected check_delegated_tasks tool")
+ # Add delegation guidance to system message
+ delegate_names = [n for n in tool_node_configs if n.startswith('delegate_to_')]
+ system_message += (
+ "\n\n## Agent Delegation\n"
+ "When delegating to sub-agents, use 'task' for the mission directive "
+ "(role and goal) and 'context' for input data the agent needs to work with.\n"
+ f"Available agents: {', '.join(delegate_names)}"
+ )
+
logger.debug(f"[ChatAgent] Total tools available: {len(all_tools)}")
# Debug: log all tool schemas to verify they're correct
for t in all_tools:
@@ -2436,6 +2495,10 @@ class AIService:
'sheets': 'google_sheets',
'tasks': 'google_tasks',
'contacts': 'google_contacts',
+ # Proxy (dual-purpose tools)
+ 'proxyRequest': 'proxy_request',
+ 'proxyConfig': 'proxy_config',
+ 'proxyStatus': 'proxy_status',
}
DEFAULT_TOOL_DESCRIPTIONS = {
'calculatorTool': 'Perform mathematical calculations. Operations: add, subtract, multiply, divide, power, sqrt, mod, abs',
@@ -2495,6 +2558,10 @@ class AIService:
'sheets': 'Read and write Google Sheets data. Operations: read (get range), write (set range), append (add rows).',
'tasks': 'Manage Google Tasks. Operations: create (new task), list (find tasks), complete (mark done), update (modify task), delete (remove task).',
'contacts': 'Manage Google Contacts. Operations: create (new contact), list (browse), search (find by name/email/phone), get (details by resource name), update (modify), delete (remove).',
+ # Proxy (dual-purpose tools)
+ 'proxyRequest': 'Make HTTP requests through residential proxy providers with geo-targeting, session control, and automatic failover. Supports GET, POST, PUT, DELETE, PATCH methods.',
+ 'proxyConfig': 'Manage proxy providers and routing rules. Operations: list_providers, add_provider, update_provider, remove_provider, set_credentials, test_provider, get_stats, add_routing_rule, list_routing_rules, remove_routing_rule.',
+ 'proxyStatus': 'Get proxy provider health stats, scores, and usage statistics. Returns enabled status, provider list with health scores, and aggregated stats.',
}
try:
@@ -2633,6 +2700,7 @@ class AIService:
url: str = Field(description="URL path or full URL to request")
method: str = Field(default="GET", description="HTTP method: GET, POST, PUT, DELETE")
body: Optional[Dict[str, Any]] = Field(default=None, description="Request body as JSON object")
+ useProxy: bool = Field(default=False, description="Route request through a configured residential proxy")
return HttpRequestSchema
@@ -3292,15 +3360,17 @@ class AIService:
"""Delegate a task to another AI Agent (non-blocking).
The child agent works independently in the background.
+ The 'task' becomes the agent's mission directive (system message).
+ The 'context' becomes the agent's input data (user prompt).
Returns a task_id immediately. Use 'check_delegated_tasks'
tool to check status and retrieve results when ready.
"""
task: str = Field(
- description=f"The task/instruction to delegate to '{agent_label}'. Be specific about what you want done."
+ description=f"The mission directive for '{agent_label}'. Describe the role and goal clearly, e.g. 'You are a coding assistant. Write a Python script that processes the given CSV data.'"
)
context: Optional[str] = Field(
default=None,
- description="Additional context or data the child agent needs to complete the task"
+ description="Input data or specific details the agent needs to work with, e.g. file contents, user requirements, or parameters"
)
return DelegateToAgentSchema
@@ -3333,6 +3403,129 @@ class AIService:
return ApifyActorSchema
+ # Proxy Request tool schema (dual-purpose: workflow node + AI tool)
+ if node_type == 'proxyRequest':
+ class ProxyRequestSchema(BaseModel):
+ """Make HTTP requests through residential proxy providers with geo-targeting and failover."""
+ url: str = Field(
+ description="Target URL to request through the proxy"
+ )
+ method: Optional[str] = Field(
+ default="GET",
+ description="HTTP method: GET, POST, PUT, DELETE, PATCH"
+ )
+ headers: Optional[str] = Field(
+ default=None,
+ description="JSON string of HTTP headers (e.g. '{\"Authorization\": \"Bearer token\"}')"
+ )
+ body: Optional[str] = Field(
+ default=None,
+ description="Request body (for POST/PUT/PATCH). JSON string or plain text."
+ )
+ proxyProvider: Optional[str] = Field(
+ default=None,
+ description="Provider name to use, or omit for auto-selection (best health score)"
+ )
+ proxyCountry: Optional[str] = Field(
+ default=None,
+ description="ISO 3166-1 alpha-2 country code for geo-targeting (e.g. US, GB, DE)"
+ )
+ sessionType: Optional[str] = Field(
+ default="rotating",
+ description="Session type: 'rotating' (new IP per request) or 'sticky' (same IP)"
+ )
+ timeout: Optional[int] = Field(
+ default=30,
+ description="Request timeout in seconds (5-300)"
+ )
+ maxRetries: Optional[int] = Field(
+ default=3,
+ description="Max retry attempts with failover (0-10)"
+ )
+
+ return ProxyRequestSchema
+
+ # Proxy Status tool schema (dual-purpose: workflow node + AI tool)
+ if node_type == 'proxyStatus':
+ class ProxyStatusSchema(BaseModel):
+ """Get proxy provider health stats, scores, and usage statistics."""
+ providerFilter: Optional[str] = Field(
+ default=None,
+ description="Filter by specific provider name, or omit for all providers"
+ )
+
+ return ProxyStatusSchema
+
+ # Proxy Config tool schema (dual-purpose: workflow node + AI tool)
+ if node_type == 'proxyConfig':
+ class ProxyConfigSchema(BaseModel):
+ """Manage proxy providers and routing rules.
+
+ Operations: list_providers, add_provider, update_provider, remove_provider,
+ set_credentials, test_provider, get_stats, add_routing_rule, list_routing_rules, remove_routing_rule.
+ """
+ operation: str = Field(
+ description="Operation: list_providers, add_provider, update_provider, remove_provider, set_credentials, test_provider, get_stats, add_routing_rule, list_routing_rules, remove_routing_rule"
+ )
+ name: Optional[str] = Field(
+ default=None,
+ description="Provider name (required for add/update/remove/set_credentials/test_provider)"
+ )
+ gateway_host: Optional[str] = Field(
+ default=None,
+ description="Gateway hostname (for add_provider/update_provider)"
+ )
+ gateway_port: Optional[int] = Field(
+ default=None,
+ description="Gateway port (for add_provider/update_provider)"
+ )
+ url_template: Optional[str] = Field(
+ default=None,
+ description="JSON template config for proxy URL encoding (for add_provider/update_provider)"
+ )
+ username: Optional[str] = Field(
+ default=None,
+ description="Proxy username (for set_credentials)"
+ )
+ password: Optional[str] = Field(
+ default=None,
+ description="Proxy password (for set_credentials)"
+ )
+ enabled: Optional[bool] = Field(
+ default=None,
+ description="Enable/disable provider (for add_provider/update_provider)"
+ )
+ priority: Optional[int] = Field(
+ default=None,
+ description="Provider priority, lower=preferred (for add_provider/update_provider)"
+ )
+ cost_per_gb: Optional[float] = Field(
+ default=None,
+ description="Cost per GB in USD (for add_provider/update_provider)"
+ )
+ domain_pattern: Optional[str] = Field(
+ default=None,
+ description="Domain glob pattern (for add_routing_rule, e.g. '*.linkedin.com')"
+ )
+ preferred_providers: Optional[str] = Field(
+ default=None,
+ description="JSON array of provider names (for add_routing_rule)"
+ )
+ required_country: Optional[str] = Field(
+ default=None,
+ description="ISO country code (for add_routing_rule)"
+ )
+ session_type: Optional[str] = Field(
+ default=None,
+ description="Session type: 'rotating' or 'sticky' (for add_routing_rule)"
+ )
+ rule_id: Optional[int] = Field(
+ default=None,
+ description="Routing rule ID (for remove_routing_rule)"
+ )
+
+ return ProxyConfigSchema
+
# Generic schema for other nodes
class GenericToolSchema(BaseModel):
"""Generic schema for tool arguments."""
diff --git a/server/services/handlers/document.py b/server/services/handlers/document.py
index 580d7a27..ff220742 100644
--- a/server/services/handlers/document.py
+++ b/server/services/handlers/document.py
@@ -76,9 +76,25 @@ async def handle_http_scraper(
else:
urls_to_fetch.append((url, {}))
- logger.info("[httpScraper] Starting", node_id=node_id, urls=len(urls_to_fetch))
+ # Transparent proxy injection
+ proxy_url = None
+ if parameters.get('useProxy', False):
+ try:
+ from services.proxy.service import get_proxy_service
+ proxy_svc = get_proxy_service()
+ if proxy_svc and proxy_svc.is_enabled():
+ proxy_url = await proxy_svc.get_proxy_url(url, parameters)
+ except Exception as e:
+ logger.warning("[httpScraper] Proxy lookup failed, proceeding without proxy", error=str(e))
+
+ logger.info("[httpScraper] Starting", node_id=node_id, urls=len(urls_to_fetch),
+ proxy=bool(proxy_url))
+
+ client_kwargs: dict = {"timeout": 30, "follow_redirects": True}
+ if proxy_url:
+ client_kwargs["proxy"] = proxy_url
- async with httpx.AsyncClient(timeout=30, follow_redirects=True) as client:
+ async with httpx.AsyncClient(**client_kwargs) as client:
for fetch_url, meta in urls_to_fetch:
try:
response = await client.get(fetch_url, headers=headers)
diff --git a/server/services/handlers/http.py b/server/services/handlers/http.py
index 181ab703..aac89edc 100644
--- a/server/services/handlers/http.py
+++ b/server/services/handlers/http.py
@@ -3,12 +3,33 @@
import json
import time
from datetime import datetime
-from typing import Dict, Any
+from typing import Any, Dict, Optional
from core.logging import get_logger
logger = get_logger(__name__)
+async def _get_proxy_url_if_enabled(url: str, parameters: Dict[str, Any]) -> Optional[str]:
+ """Return a proxy URL if useProxy is enabled and a provider is available.
+
+ Returns None when proxy should not be used (disabled, no providers, etc.).
+ Exceptions from the proxy service are logged and swallowed so that the
+ request proceeds without a proxy rather than failing.
+ """
+ if not parameters.get('useProxy', False):
+ return None
+
+ try:
+ from services.proxy.service import get_proxy_service
+ proxy_svc = get_proxy_service()
+ if not proxy_svc or not proxy_svc.is_enabled():
+ return None
+ return await proxy_svc.get_proxy_url(url, parameters)
+ except Exception as e:
+ logger.warning("Proxy URL lookup failed, proceeding without proxy", error=str(e))
+ return None
+
+
async def handle_http_request(
node_id: str,
node_type: str,
@@ -47,14 +68,20 @@ async def handle_http_request(
except json.JSONDecodeError:
headers = {}
- logger.info("[HTTP Request] Executing", node_id=node_id, method=method, url=url)
+ # Transparent proxy injection
+ proxy_url = await _get_proxy_url_if_enabled(url, parameters)
+ logger.info("[HTTP Request] Executing", node_id=node_id, method=method, url=url,
+ proxy=bool(proxy_url))
+
+ client_kwargs: Dict[str, Any] = {"timeout": timeout}
+ if proxy_url:
+ client_kwargs["proxy"] = proxy_url
- async with httpx.AsyncClient() as client:
+ async with httpx.AsyncClient(**client_kwargs) as client:
kwargs = {
'method': method,
'url': url,
'headers': headers,
- 'timeout': timeout
}
# Add body for POST/PUT/PATCH
@@ -77,7 +104,8 @@ async def handle_http_request(
"data": response_data,
"headers": dict(response.headers),
"url": str(response.url),
- "method": method
+ "method": method,
+ "proxied": proxy_url is not None,
}
return {
diff --git a/server/services/handlers/proxy.py b/server/services/handlers/proxy.py
index 710a67d9..fe60e244 100644
--- a/server/services/handlers/proxy.py
+++ b/server/services/handlers/proxy.py
@@ -91,7 +91,7 @@ async def handle_proxy_request(
"success": False,
"node_id": node_id,
"node_type": "proxyRequest",
- "error": "Proxy service is not enabled. Set PROXY_ENABLED=true in .env",
+ "error": "Proxy service not initialized. Use proxy_config tool to add a provider first.",
"execution_time": time.time() - start_time,
"timestamp": datetime.now().isoformat(),
}
diff --git a/server/services/handlers/tools.py b/server/services/handlers/tools.py
index d3512d7b..5e0ff7f6 100644
--- a/server/services/handlers/tools.py
+++ b/server/services/handlers/tools.py
@@ -171,6 +171,14 @@ async def execute_tool(tool_name: str, tool_args: Dict[str, Any],
if node_type == 'apifyActor':
return await _execute_apify_actor(tool_args, config.get('parameters', {}))
+ # Proxy nodes (dual-purpose: workflow node + AI tool)
+ if node_type == 'proxyRequest':
+ return await _execute_proxy_request(tool_args, config.get('parameters', {}), config)
+ if node_type == 'proxyStatus':
+ return await _execute_proxy_status(tool_args, config.get('parameters', {}))
+ if node_type == 'proxyConfig':
+ return await _execute_proxy_config(tool_args, config.get('parameters', {}))
+
# Built-in: Check delegated task results
# Auto-injected when parent has delegation tools
if node_type == '_builtin_check_delegated_tasks':
@@ -265,10 +273,27 @@ async def _execute_http_request(args: Dict[str, Any],
except Exception:
default_headers = {}
- logger.debug(f"[HTTP Tool] {method} {full_url}")
+ # Transparent proxy injection (useProxy from node params or LLM args)
+ proxy_url = None
+ use_proxy = args.get('useProxy', node_params.get('useProxy', False))
+ if use_proxy:
+ try:
+ from services.proxy.service import get_proxy_service
+ proxy_svc = get_proxy_service()
+ if proxy_svc and proxy_svc.is_enabled():
+ merged = {**node_params, **args}
+ proxy_url = await proxy_svc.get_proxy_url(full_url, merged)
+ except Exception as e:
+ logger.warning(f"[HTTP Tool] Proxy lookup failed, proceeding without proxy: {e}")
+
+ logger.debug(f"[HTTP Tool] {method} {full_url}", proxy=bool(proxy_url))
try:
- async with httpx.AsyncClient(timeout=30.0) as client:
+ client_kwargs: dict = {"timeout": 30.0}
+ if proxy_url:
+ client_kwargs["proxy"] = proxy_url
+
+ async with httpx.AsyncClient(**client_kwargs) as client:
response = await client.request(
method=method,
url=full_url,
@@ -286,7 +311,8 @@ async def _execute_http_request(args: Dict[str, Any],
"status": response.status_code,
"data": data,
"url": full_url,
- "method": method
+ "method": method,
+ "proxied": proxy_url is not None,
}
except httpx.TimeoutException:
@@ -1358,6 +1384,251 @@ async def _execute_apify_actor(args: Dict[str, Any],
)
+async def _execute_proxy_request(args: Dict[str, Any],
+ node_params: Dict[str, Any],
+ config: Dict[str, Any]) -> Dict[str, Any]:
+ """Execute proxy HTTP request as an AI tool.
+
+ Delegates to the standalone proxy request handler.
+
+ Args:
+ args: LLM-provided arguments (url, method, headers, body, etc.)
+ node_params: Node parameters
+ config: Full tool config with context
+
+ Returns:
+ Request result dict
+ """
+ from services.handlers.proxy import handle_proxy_request
+
+ # Merge LLM args over node params
+ merged = {**node_params, **{k: v for k, v in args.items() if v is not None}}
+
+ context = config.get('context', {})
+ node_id = config.get('node_id', 'tool_proxy_request')
+
+ result = await handle_proxy_request(node_id, 'proxyRequest', merged, context)
+ return result.get('result', result)
+
+
+async def _execute_proxy_status(args: Dict[str, Any],
+ node_params: Dict[str, Any]) -> Dict[str, Any]:
+ """Execute proxy status check as an AI tool.
+
+ Delegates to the standalone proxy status handler.
+
+ Args:
+ args: LLM-provided arguments (providerFilter)
+ node_params: Node parameters
+
+ Returns:
+ Status result dict
+ """
+ from services.handlers.proxy import handle_proxy_status
+
+ merged = {**node_params, **{k: v for k, v in args.items() if v is not None}}
+
+ result = await handle_proxy_status('tool_proxy_status', 'proxyStatus', merged, {})
+ return result.get('result', result)
+
+
+async def _execute_proxy_config(args: Dict[str, Any],
+ node_params: Dict[str, Any]) -> Dict[str, Any]:
+ """Execute proxy configuration operations.
+
+ Manages proxy providers, credentials, and routing rules.
+
+ Args:
+ args: LLM-provided arguments (operation, name, etc.)
+ node_params: Node parameters
+
+ Returns:
+ Operation result dict
+ """
+ import json as _json
+ from core.container import container
+ from services.proxy.service import get_proxy_service
+
+ operation = args.get('operation', node_params.get('operation', ''))
+ name = args.get('name', node_params.get('name', ''))
+ proxy_svc = get_proxy_service()
+
+ if operation == 'list_providers':
+ providers = proxy_svc.get_providers() if proxy_svc else []
+ return {"success": True, "operation": operation, "providers": [p.model_dump() for p in providers]}
+
+ if operation == 'get_stats':
+ stats = proxy_svc.get_stats() if proxy_svc else {}
+ return {"success": True, "operation": operation, "stats": stats}
+
+ if operation == 'list_routing_rules':
+ rules = proxy_svc.get_routing_rules() if proxy_svc else []
+ return {"success": True, "operation": operation, "rules": [r.model_dump() for r in rules]}
+
+ if operation == 'add_provider':
+ if not name:
+ return {"success": False, "error": "Provider name is required"}
+ gateway_host = args.get('gateway_host', node_params.get('gateway_host', ''))
+ gateway_port = int(args.get('gateway_port', node_params.get('gateway_port', 0)))
+ url_template_str = args.get('url_template', node_params.get('url_template', '{}'))
+ cost_per_gb = float(args.get('cost_per_gb', node_params.get('cost_per_gb', 0)))
+ enabled = args.get('enabled', node_params.get('enabled', True))
+ priority = int(args.get('priority', node_params.get('priority', 50)))
+
+ try:
+ url_template = _json.loads(url_template_str) if isinstance(url_template_str, str) else url_template_str
+ except _json.JSONDecodeError:
+ return {"success": False, "error": f"Invalid url_template JSON: {url_template_str}"}
+
+ db = container.database()
+ await db.save_proxy_provider({
+ "name": name,
+ "enabled": enabled,
+ "priority": priority,
+ "cost_per_gb": cost_per_gb,
+ "gateway_host": gateway_host,
+ "gateway_port": gateway_port,
+ "url_template": _json.dumps(url_template),
+ })
+
+ if proxy_svc:
+ await proxy_svc.reload_providers()
+
+ return {"success": True, "operation": operation, "name": name}
+
+ if operation == 'update_provider':
+ if not name:
+ return {"success": False, "error": "Provider name is required"}
+ db = container.database()
+ existing = await db.get_proxy_provider(name)
+ if not existing:
+ return {"success": False, "error": f"Provider '{name}' not found"}
+
+ updates = {}
+ for field in ['gateway_host', 'gateway_port', 'cost_per_gb', 'enabled', 'priority']:
+ val = args.get(field, node_params.get(field))
+ if val is not None:
+ updates[field] = val
+
+ url_template_str = args.get('url_template', node_params.get('url_template'))
+ if url_template_str:
+ try:
+ url_template = _json.loads(url_template_str) if isinstance(url_template_str, str) else url_template_str
+ updates['url_template'] = _json.dumps(url_template)
+ except _json.JSONDecodeError:
+ return {"success": False, "error": f"Invalid url_template JSON"}
+
+ if updates:
+ merged = {**existing, **updates}
+ await db.save_proxy_provider(merged)
+ if proxy_svc:
+ await proxy_svc.reload_providers()
+
+ return {"success": True, "operation": operation, "name": name, "updated_fields": list(updates.keys())}
+
+ if operation == 'remove_provider':
+ if not name:
+ return {"success": False, "error": "Provider name is required"}
+ db = container.database()
+ await db.delete_proxy_provider(name)
+ if proxy_svc:
+ await proxy_svc.reload_providers()
+ return {"success": True, "operation": operation, "name": name}
+
+ if operation == 'set_credentials':
+ if not name:
+ return {"success": False, "error": "Provider name is required"}
+ username = args.get('username', node_params.get('username', ''))
+ password = args.get('password', node_params.get('password', ''))
+ if not username or not password:
+ return {"success": False, "error": "Username and password are required"}
+
+ auth_svc = container.auth_service()
+ await auth_svc.store_api_key(f"proxy_{name}_username", username, [])
+ await auth_svc.store_api_key(f"proxy_{name}_password", password, [])
+
+ if proxy_svc:
+ await proxy_svc.reload_providers()
+
+ return {"success": True, "operation": operation, "name": name}
+
+ if operation == 'test_provider':
+ if not name:
+ return {"success": False, "error": "Provider name is required"}
+ if not proxy_svc:
+ return {"success": False, "error": "Proxy service not enabled"}
+
+ import time as _time
+ import httpx
+
+ try:
+ proxy_url = await proxy_svc.get_proxy_url("https://httpbin.org/ip", {"proxyProvider": name})
+ if not proxy_url:
+ return {"success": False, "error": f"Could not get proxy URL for '{name}'"}
+
+ start = _time.monotonic()
+ async with httpx.AsyncClient(proxy=proxy_url, timeout=30) as client:
+ resp = await client.get("https://httpbin.org/ip")
+ latency_ms = (_time.monotonic() - start) * 1000
+
+ try:
+ data = resp.json()
+ ip = data.get("origin", "unknown")
+ except Exception:
+ ip = "unknown"
+
+ return {
+ "success": resp.status_code == 200,
+ "operation": operation,
+ "name": name,
+ "ip": ip,
+ "latency_ms": round(latency_ms, 1),
+ "status_code": resp.status_code,
+ }
+ except Exception as e:
+ return {"success": False, "operation": operation, "name": name, "error": str(e)}
+
+ if operation == 'add_routing_rule':
+ domain_pattern = args.get('domain_pattern', node_params.get('domain_pattern', ''))
+ if not domain_pattern:
+ return {"success": False, "error": "domain_pattern is required"}
+
+ preferred_str = args.get('preferred_providers', node_params.get('preferred_providers', '[]'))
+ try:
+ preferred = _json.loads(preferred_str) if isinstance(preferred_str, str) else preferred_str
+ except _json.JSONDecodeError:
+ preferred = []
+
+ db = container.database()
+ rule_data = {
+ "domain_pattern": domain_pattern,
+ "preferred_providers": _json.dumps(preferred),
+ "required_country": args.get('required_country', node_params.get('required_country', '')),
+ "session_type": args.get('session_type', node_params.get('session_type', 'rotating')),
+ }
+ await db.save_proxy_routing_rule(rule_data)
+
+ if proxy_svc:
+ await proxy_svc.reload_providers()
+
+ return {"success": True, "operation": operation, "domain_pattern": domain_pattern}
+
+ if operation == 'remove_routing_rule':
+ rule_id = args.get('rule_id', node_params.get('rule_id'))
+ if not rule_id:
+ return {"success": False, "error": "rule_id is required"}
+
+ db = container.database()
+ await db.delete_proxy_routing_rule(int(rule_id))
+
+ if proxy_svc:
+ await proxy_svc.reload_providers()
+
+ return {"success": True, "operation": operation, "rule_id": rule_id}
+
+ return {"success": False, "error": f"Unknown operation: {operation}"}
+
+
async def _execute_generic(args: Dict[str, Any],
config: Dict[str, Any]) -> Dict[str, Any]:
"""Execute a generic tool (fallback handler).
@@ -1464,11 +1735,9 @@ async def _execute_delegated_agent(args: Dict[str, Any],
if models:
child_params['model'] = models[0]
- # Build prompt from task + context
- full_prompt = task_description
- if task_context:
- full_prompt = f"{task_description}\n\nContext:\n{task_context}"
- child_params['prompt'] = full_prompt
+ # Task goes into systemMessage (directive), context data goes into prompt
+ child_params['systemMessage'] = task_description
+ child_params['prompt'] = task_context if task_context else task_description
# Create execution context for child agent
child_context = {
diff --git a/server/services/node_executor.py b/server/services/node_executor.py
index ec201715..d8cd2ed3 100644
--- a/server/services/node_executor.py
+++ b/server/services/node_executor.py
@@ -41,6 +41,7 @@ from services.handlers import (
)
from services.handlers.telegram import handle_telegram_send, handle_telegram_receive
from services.handlers.apify import handle_apify_actor
+from services.handlers.proxy import handle_proxy_request, handle_proxy_status, handle_proxy_config
# Consolidated Google Workspace handlers (6 services with operation dispatchers)
from services.handlers.gmail import handle_google_gmail, handle_gmail_receive
from services.handlers.calendar import handle_google_calendar
@@ -185,6 +186,10 @@ class NodeExecutor:
'teamMonitor': handle_team_monitor,
# Apify web scraping
'apifyActor': handle_apify_actor,
+ # Proxy nodes
+ 'proxyRequest': handle_proxy_request,
+ 'proxyStatus': handle_proxy_status,
+ 'proxyConfig': handle_proxy_config,
# Note: 'console' handled in _dispatch with connected_outputs
}
diff --git a/server/services/proxy/service.py b/server/services/proxy/service.py
index 038fc881..6f753e98 100644
--- a/server/services/proxy/service.py
+++ b/server/services/proxy/service.py
@@ -139,11 +139,11 @@ class ProxyService:
self._initialized = False
async def startup(self) -> None:
- """Load provider configs from DB and credentials from AuthService."""
- if not self._settings.proxy_enabled:
- logger.info("Proxy service disabled (PROXY_ENABLED=false)")
- return
+ """Load provider configs from DB and credentials from AuthService.
+ Always initializes -- providers are managed dynamically by the LLM
+ via proxy_config tool. PROXY_ENABLED is not a gate.
+ """
try:
# Load provider configs from database
db_providers = await self._database.get_proxy_providers()
@@ -170,7 +170,7 @@ class ProxyService:
logger.info("Proxy service stopped")
def is_enabled(self) -> bool:
- return self._settings.proxy_enabled and self._initialized
+ return self._initialized
async def get_proxy_url(
self,
@@ -234,7 +234,7 @@ class ProxyService:
)
# Determine session type
- session_type_str = parameters.get("proxySessionType", "rotating")
+ session_type_str = parameters.get("sessionType") or parameters.get("proxySessionType", "rotating")
session_type = SessionType(session_type_str) if session_type_str else SessionType.ROTATING
if rule:
session_type = rule.session_type
diff --git a/server/skills/web_agent/http-request-skill/SKILL.md b/server/skills/web_agent/http-request-skill/SKILL.md
index a9876b44..cd32ccc1 100644
--- a/server/skills/web_agent/http-request-skill/SKILL.md
+++ b/server/skills/web_agent/http-request-skill/SKILL.md
@@ -29,6 +29,7 @@ Make an HTTP request to any URL.
| url | string | Yes | Full URL to request (e.g., `https://api.example.com/data`) |
| method | string | No | HTTP method: `GET`, `POST`, `PUT`, `DELETE`, `PATCH` (default: `GET`) |
| body | object | No | Request body as JSON object (for POST/PUT/PATCH) |
+| useProxy | boolean | No | Route through proxy (default: `false`). The proxy service handles provider selection, geo-targeting, and session type automatically. |
### Node Parameters
@@ -46,7 +47,8 @@ Additional options can be configured on the node:
"status": 200,
"data": { "key": "value" },
"url": "https://api.example.com/data",
- "method": "GET"
+ "method": "GET",
+ "proxied": false
}
```
@@ -188,6 +190,21 @@ Additional options can be configured on the node:
| Health check | GET | Verify service availability |
| Webhook trigger | POST | Send events to services |
+## Proxy Usage
+
+To route a request through a residential proxy, just set `useProxy: true`. The proxy service automatically selects the best provider, geo-target, and session type from its configuration. A proxy provider must be configured first via the `proxy_config` tool (see proxy-config-skill).
+
+**Proxied request:**
+```json
+{
+ "url": "https://example.com/data",
+ "method": "GET",
+ "useProxy": true
+}
+```
+
+If no proxy providers are configured or the proxy lookup fails, the request proceeds directly without a proxy and `"proxied": false` appears in the response.
+
## Guidelines
1. **URLs**: Must be fully qualified with protocol (https://)
@@ -195,6 +212,7 @@ Additional options can be configured on the node:
3. **Timeout**: Default 30 seconds
4. **JSON body**: Automatically serialized for POST/PUT/PATCH
5. **Response**: JSON responses are automatically parsed
+6. **Proxy**: Set `useProxy: true` to route through proxy -- the proxy service handles all routing details
## Security Notes
@@ -209,3 +227,4 @@ Additional options can be configured on the node:
1. Connect the **HTTP Request** node to Zeenie's `input-tools` handle
2. Configure authentication headers on the node if needed
3. Ensure network access to target APIs
+4. For proxy usage: configure a proxy provider via the proxy-config-skill first
diff --git a/server/skills/web_agent/proxy-config-skill/SKILL.md b/server/skills/web_agent/proxy-config-skill/SKILL.md
index 2e9d74c1..9fef1aec 100644
--- a/server/skills/web_agent/proxy-config-skill/SKILL.md
+++ b/server/skills/web_agent/proxy-config-skill/SKILL.md
@@ -1,400 +1,228 @@
---
name: proxy-config-skill
-description: Configure any residential proxy provider using template-driven URL encoding. Generate url_template JSON from provider docs, save providers, set credentials, manage routing rules.
-allowed-tools: proxy_config python_code
+description: Configure residential proxy providers and make proxied HTTP requests with geo-targeting.
+allowed-tools: proxy_config proxy_request proxy_status python_code
metadata:
author: machina
- version: "1.0"
+ version: "3.0"
category: integration
icon: "🛡"
color: "#8B5CF6"
---
-# Proxy Config Skill
+# Proxy Configuration Skill
-Configure any residential proxy provider in the world using the template-driven proxy system. No provider-specific code exists -- you generate a `url_template` JSON config that tells the system how to format proxy URLs for that provider.
+You manage residential proxy providers. Providers are configured with a JSON `url_template` that controls how geo/session parameters are encoded into the proxy URL.
-## How It Works
+## Setup Workflow
-1. User tells you which proxy provider they want to configure
-2. You use `python_code` to generate and validate the `url_template` JSON
-3. You use `proxy_config` to save the provider, set credentials, and optionally add routing rules
-4. You use `proxy_config(test_provider)` to verify the setup works
+To add a new proxy provider, make these tool calls in order:
-## Tools
+### Step 1: Add the provider
-- **proxy_config** -- Manage providers, credentials, and routing rules
-- **python_code** -- Generate and validate url_template JSON configs
+Call `proxy_config` with `operation: "add_provider"`. The `url_template` field is a **JSON string**.
----
-
-## url_template JSON Schema
+Most residential proxy providers use username-based parameter encoding with dash separators. Use this default template unless the provider docs show otherwise:
-The `url_template` is a JSON object that tells the proxy system how to encode geo-targeting, session, and other parameters into the proxy URL. Every residential proxy provider uses one of three encoding strategies.
-
-### Encoding Strategies (param_field)
+```
+proxy_config({
+ "operation": "add_provider",
+ "name": "smartproxy",
+ "gateway_host": "gate.smartproxy.com",
+ "gateway_port": 7777,
+ "url_template": "{\"param_field\":\"username\",\"username_prefix\":\"{username}\",\"username_param_separator\":\"-\",\"param_separator\":\"-\",\"param_keys\":{\"country\":\"country-{v}\",\"city\":\"city-{v}\",\"state\":\"state-{v}\",\"session_id\":\"session-{v}\",\"session_duration\":\"sessTime-{v}\"},\"country_case\":\"lower\",\"city_separator\":\"_\"}"
+})
+```
-| Strategy | param_field | How it works | Providers using this |
-|----------|-------------|--------------|---------------------|
-| Username encoding | `"username"` | Parameters appended to username string | ~80% of providers |
-| Password encoding | `"password"` | Parameters appended to password string | A few providers |
-| No encoding | `"none"` | Plain credentials, no parameter encoding | Generic/custom setups |
+### Step 2: Set credentials
-### Template Fields
+```
+proxy_config({
+ "operation": "set_credentials",
+ "name": "smartproxy",
+ "username": "the_username",
+ "password": "the_password"
+})
+```
-| Field | Type | Default | Description |
-|-------|------|---------|-------------|
-| `param_field` | string | `"username"` | Where to encode params: `"username"`, `"password"`, or `"none"` |
-| `username_prefix` | string | `"{username}"` | Template for the username base. `{username}` is replaced with the actual username |
-| `username_param_separator` | string | `"-"` | Separator between username prefix and first parameter |
-| `param_separator` | string | `"-"` | Separator between individual parameters |
-| `param_keys` | object | `{}` | Map of parameter names to format strings. `{v}` is replaced with the value |
-| `country_case` | string | `"lower"` | Case for country codes: `"lower"` or `"upper"` |
-| `city_separator` | string | `"_"` | Character replacing spaces in city/state names |
+### Step 3: Test
-### param_keys Reference
+```
+proxy_config({
+ "operation": "test_provider",
+ "name": "smartproxy"
+})
+```
-| Key | Purpose | Example format string |
-|-----|---------|----------------------|
-| `country` | ISO country code | `"country-{v}"`, `"_country-{v}"`, `"cc-{v}"` |
-| `state` | State/region | `"state-{v}"`, `"_state-{v}"` |
-| `city` | City name | `"city-{v}"`, `"_city-{v}"` |
-| `session_id` | Sticky session ID | `"session-{v}"`, `"sessid-{v}"` |
-| `session_duration` | Session duration | `"sessTime-{v}"`, `"_lifetime-{v}"` |
+If test returns `"success": true`, the provider is ready. If it fails, check credentials and gateway host/port.
-### How URL Formatting Works
+## Making Proxied Requests
-Given credentials `user123` / `pass456`, host `gate.example.com:7777`, country `US`:
+Once a provider is configured:
-**Username encoding** (`param_field: "username"`):
```
-http://user123-country-us:[email protected]:7777
- ^^^^^^^^^^^^^^^^^^^^^^^
- prefix + separator + params
+proxy_request({
+ "url": "https://example.com/page",
+ "proxyCountry": "US"
+})
```
-**Password encoding** (`param_field: "password"`):
-```
-http://user123:[email protected]:7777
- ^^^^^^^^^^^^^^^^^^^^^
- password + params (no separator usually)
-```
+The system auto-selects the best healthy provider. To force a specific one:
-**No encoding** (`param_field: "none"`):
```
-http://user123:[email protected]:7777
+proxy_request({
+ "url": "https://example.com/page",
+ "proxyProvider": "smartproxy",
+ "proxyCountry": "US",
+ "sessionType": "sticky"
+})
```
----
-
-## Workflow: Configure a New Provider
+## url_template Reference
-### Step 1: Generate url_template with python_code
+The `url_template` JSON controls how the system builds the proxy URL. It determines how username/password strings are constructed with geo and session parameters.
-Read the provider's documentation to identify:
-1. What the proxy gateway host and port are
-2. How parameters are encoded (username-based or password-based)
-3. What separator characters are used
-4. What the parameter format strings look like
-
-Then use `python_code` to generate and validate the template:
-
-```python
-import json
-
-# Build the url_template based on provider docs
-template = {
- "param_field": "username",
- "username_prefix": "{username}",
- "username_param_separator": "-",
- "param_separator": "-",
- "param_keys": {
- "country": "country-{v}",
- "city": "city-{v}",
- "session_id": "session-{v}",
- "session_duration": "sessTime-{v}"
- },
- "country_case": "lower",
- "city_separator": "_"
-}
+**Fields:**
+- `param_field`: Where params go. `"username"` (80% of providers), `"password"`, or `"none"`.
+- `username_prefix`: Template for username base. `{username}` is replaced with actual username.
+- `username_param_separator`: Separator between username and first param (e.g. `"-"`).
+- `param_separator`: Separator between params (e.g. `"-"`).
+- `param_keys`: Object mapping param names to format strings. `{v}` is replaced with the value.
+ - `country`: e.g. `"country-{v}"` produces `country-us`
+ - `city`: e.g. `"city-{v}"` produces `city-new_york`
+ - `state`: e.g. `"state-{v}"` produces `state-california`
+ - `session_id`: e.g. `"session-{v}"` produces `session-abc123`
+ - `session_duration`: e.g. `"sessTime-{v}"` produces `sessTime-10`
+- `country_case`: `"lower"` or `"upper"` for country codes.
+- `city_separator`: Replaces spaces in city/state names (e.g. `"_"` makes `new_york`).
-# Validate: simulate what the proxy system does
-username = "testuser"
-password = "testpass"
-country = "us"
-city = "new_york"
-session_id = "abc123"
+**Result for username encoding with country US:**
+`http://myuser-country-us:[email protected]:7777`
-params = []
-if "country" in template["param_keys"]:
- cc = country.lower() if template.get("country_case") == "lower" else country.upper()
- params.append(template["param_keys"]["country"].replace("{v}", cc))
-if "city" in template["param_keys"]:
- c = city.lower().replace(" ", template.get("city_separator", "_"))
- params.append(template["param_keys"]["city"].replace("{v}", c))
-if "session_id" in template["param_keys"]:
- params.append(template["param_keys"]["session_id"].replace("{v}", session_id))
-
-param_str = template.get("param_separator", "-").join(params)
-prefix = template.get("username_prefix", "{username}").replace("{username}", username)
-
-if template["param_field"] == "username" and param_str:
- sep = template.get("username_param_separator", "-")
- final_user = f"{prefix}{sep}{param_str}"
- final_pass = password
-elif template["param_field"] == "password" and param_str:
- final_user = prefix
- final_pass = f"{password}{param_str}"
-else:
- final_user = prefix
- final_pass = password
-
-url = f"http://{final_user}:{final_pass}@gate.example.com:7777"
-print(f"Generated URL: {url}")
-
-# Assert the URL matches expected format from provider docs
-assert "country-us" in url, "Country encoding failed"
-assert "city-new_york" in url, "City encoding failed"
-assert "session-abc123" in url, "Session encoding failed"
-
-# Output the validated template
-result = json.dumps(template, indent=2)
-print(f"\nValidated template:\n{result}")
-output = result
-```
+**Result for password encoding with country US:**
+`http://myuser:[email protected]:7777`
-### Step 2: Save the provider with proxy_config
+## Common Provider Templates
+### DataImpulse (username-based, dash separator)
+Gateway: `gw.dataimpulse.com:823`
```json
-{
- "operation": "add_provider",
- "name": "my_provider",
- "gateway_host": "gate.example.com",
- "gateway_port": 7777,
- "url_template": "{...the JSON template from step 1...}",
- "cost_per_gb": 2.50,
- "enabled": true,
- "priority": 50
-}
+{"param_field":"username","username_prefix":"{username}","username_param_separator":"__","param_separator":"__","param_keys":{"country":"country-{v}","city":"city-{v}","session_id":"session-{v}","session_duration":"sessionduration-{v}"},"country_case":"lower","city_separator":"_"}
```
-### Step 3: Set credentials
-
+### Decodo/Smartproxy (username-based, dash separator)
+Gateway: `gate.smartproxy.com:7777`
```json
-{
- "operation": "set_credentials",
- "name": "my_provider",
- "username": "actual_username",
- "password": "actual_password"
-}
+{"param_field":"username","username_prefix":"{username}","username_param_separator":"-","param_separator":"-","param_keys":{"country":"country-{v}","city":"city-{v}","state":"state-{v}","session_id":"session-{v}","session_duration":"sessTime-{v}"},"country_case":"lower","city_separator":"_"}
```
-### Step 4: Test the provider
-
+### Bright Data (username-based, dash separator)
+Gateway: `brd.superproxy.io:22225`
```json
-{
- "operation": "test_provider",
- "name": "my_provider"
-}
+{"param_field":"username","username_prefix":"{username}","username_param_separator":"-","param_separator":"-","param_keys":{"country":"country-{v}","city":"city-{v}","state":"state-{v}","session_id":"session-{v}"},"country_case":"lower","city_separator":"_"}
```
-Expected response:
+### IPRoyal (password-based)
+Gateway: `geo.iproyal.com:12321`
```json
-{
- "success": true,
- "ip": "203.0.113.42",
- "latency_ms": 1250.3,
- "status_code": 200
-}
+{"param_field":"password","username_prefix":"{username}","param_separator":"_","param_keys":{"country":"country-{v}","city":"city-{v}","session_id":"session-{v}","session_duration":"sessionduration-{v}"},"country_case":"lower","city_separator":"_"}
```
-### Step 5 (Optional): Add routing rules
-
-Route specific domains through specific providers:
-
+### Custom/Generic (no encoding)
+For providers that don't encode params in credentials:
```json
-{
- "operation": "add_routing_rule",
- "domain_pattern": "*.linkedin.com",
- "preferred_providers": "[\"my_provider\"]",
- "required_country": "US",
- "session_type": "sticky"
-}
+{"param_field":"none"}
```
----
-
-## proxy_config Tool Reference
-
-### Operations
-
-| Operation | Description | Required Fields |
-|-----------|-------------|-----------------|
-| `list_providers` | List all configured providers | (none) |
-| `add_provider` | Add a new provider | name, gateway_host, gateway_port, url_template |
-| `update_provider` | Update provider settings | name, + fields to change |
-| `remove_provider` | Remove a provider | name |
-| `set_credentials` | Set provider username/password | name, username, password |
-| `test_provider` | Health check via httpbin.org/ip | name |
-| `get_stats` | Get usage statistics | (none) |
-| `add_routing_rule` | Add domain routing rule | domain_pattern |
-| `list_routing_rules` | List all routing rules | (none) |
-| `remove_routing_rule` | Remove a routing rule | rule_id |
-
-### add_provider Fields
-
-| Field | Type | Required | Description |
-|-------|------|----------|-------------|
-| operation | string | Yes | `"add_provider"` |
-| name | string | Yes | Unique provider name (e.g., `"smartproxy"`, `"brightdata"`) |
-| gateway_host | string | Yes | Gateway hostname |
-| gateway_port | integer | Yes | Gateway port |
-| url_template | string | Yes | JSON string of template config |
-| enabled | boolean | No | Active status (default: true) |
-| priority | integer | No | Lower = preferred (default: 50) |
-| cost_per_gb | float | No | USD per GB (default: 0) |
-| geo_coverage | string | No | JSON array of ISO country codes |
-
-### set_credentials Fields
-
-| Field | Type | Required | Description |
-|-------|------|----------|-------------|
-| operation | string | Yes | `"set_credentials"` |
-| name | string | Yes | Provider name |
-| username | string | Yes | Proxy username from provider dashboard |
-| password | string | Yes | Proxy password from provider dashboard |
-
-### add_routing_rule Fields
-
-| Field | Type | Required | Description |
-|-------|------|----------|-------------|
-| operation | string | Yes | `"add_routing_rule"` |
-| domain_pattern | string | Yes | Glob pattern: `"*.example.com"`, `"*"` (catch-all) |
-| preferred_providers | string | No | JSON array of provider names |
-| required_country | string | No | ISO country code |
-| session_type | string | No | `"rotating"` or `"sticky"` (default: rotating) |
-
----
-
-## Reading Provider Documentation
-
-When the user gives you a provider name or documentation URL, look for these patterns:
+## If Provider Uses Unfamiliar Format
-### What to Extract
+Use `python_code` to reverse-engineer the template from the provider's example URL:
-1. **Gateway endpoint**: Usually formatted as `host:port` (e.g., `gate.smartproxy.com:7777`)
-2. **Username format**: How parameters are appended (e.g., `user-country-us-session-abc123`)
-3. **Separator characters**: Dash `-`, underscore `_`, dot `.`, or custom
-4. **Parameter names**: What the provider calls country, city, session, etc.
-5. **Country code format**: Lowercase `us` vs uppercase `US`
-
-### Common Patterns to Recognize
-
-**Pattern A: Dash-separated username params (most common)**
-```
-username: user-country-us-city-new_york-session-abc123
-password: pass
-```
-Template: `param_field: "username"`, separators: `"-"`
-
-**Pattern B: Underscore prefix params**
-```
-username: user_country-us_city-new_york_session-abc123
-password: pass
-```
-Template: `username_param_separator: "_"`, keys use `"_country-{v}"`
-
-**Pattern C: Password-based encoding**
-```
-username: user
-password: pass_country-us_session-abc123
-```
-Template: `param_field: "password"`
-
-**Pattern D: Custom prefix**
-```
-username: customer-user-zone-us
-password: pass
-```
-Template: `username_prefix: "customer-{username}"`, keys: `"zone-{v}"`
-
-### Validation Checklist
-
-After generating a template, always verify with `python_code`:
-
-1. Country code appears correctly in the URL
-2. City names have spaces replaced with the right separator
-3. Session IDs are properly formatted
-4. The overall URL matches what the provider docs show
-5. Both rotating (no session) and sticky (with session) modes work
-
----
-
-## Examples
-
-### Example: User says "Set up Smartproxy"
-
-1. Generate template with `python_code`:
```python
import json
+
+# Example from provider docs:
+# Username: user123-cc-US-city-NewYork-sess-abc123
+# Password: pass456
+# Host: proxy.example.com:8080
+
template = {
"param_field": "username",
"username_prefix": "{username}",
"username_param_separator": "-",
"param_separator": "-",
"param_keys": {
- "country": "country-{v}",
+ "country": "cc-{v}",
"city": "city-{v}",
- "state": "state-{v}",
- "session_id": "session-{v}",
- "session_duration": "sessTime-{v}"
+ "session_id": "sess-{v}"
},
- "country_case": "lower",
- "city_separator": "_"
+ "country_case": "upper",
+ "city_separator": ""
}
+
+# Verify it produces the right URL
+username = "user123"
+params = []
+params.append(template["param_keys"]["country"].replace("{v}", "US"))
+params.append(template["param_keys"]["city"].replace("{v}", "NewYork"))
+params.append(template["param_keys"]["session_id"].replace("{v}", "abc123"))
+param_str = template["param_separator"].join(params)
+result = f"{username}{template['username_param_separator']}{param_str}"
+print(f"Built username: {result}")
+assert result == "user123-cc-US-city-NewYork-sess-abc123", f"Mismatch: {result}"
+
output = json.dumps(template)
+print(f"Template: {output}")
```
-2. Save: `proxy_config(add_provider, name="smartproxy", gateway_host="gate.smartproxy.com", gateway_port=7777, url_template=<template>, cost_per_gb=4.00)`
-3. Credentials: `proxy_config(set_credentials, name="smartproxy", username="sp...", password="...")`
-4. Test: `proxy_config(test_provider, name="smartproxy")`
+Then pass the `output` as the `url_template` string to `proxy_config(add_provider)`.
-### Example: User provides custom proxy docs
+## All proxy_config Operations
-If the user pastes documentation showing:
-```
-Proxy: proxy.example.com:8080
-Username: your_key
-Password: your_secret_cc-US_city-NewYork_sess-12345
-```
+| Operation | Required Parameters | Description |
+|-----------|-------------------|-------------|
+| `list_providers` | none | List all configured providers |
+| `add_provider` | `name`, `gateway_host`, `gateway_port`, `url_template` | Add a new provider |
+| `update_provider` | `name` + any fields to change | Update existing provider |
+| `remove_provider` | `name` | Delete a provider |
+| `set_credentials` | `name`, `username`, `password` | Store proxy credentials |
+| `test_provider` | `name` | Test via httpbin.org/ip |
+| `get_stats` | none | Usage statistics |
+| `add_routing_rule` | `domain_pattern` | Route domains to specific providers |
+| `list_routing_rules` | none | List all routing rules |
+| `remove_routing_rule` | `rule_id` | Delete a routing rule |
-You identify: password-based encoding, underscore prefix, dash key-value separator.
+Optional fields for `add_provider`/`update_provider`: `enabled` (bool), `priority` (int, lower=preferred), `cost_per_gb` (float).
-```python
-import json
-template = {
- "param_field": "password",
- "username_prefix": "{username}",
- "param_separator": "_",
- "param_keys": {
- "country": "cc-{v}",
- "city": "city-{v}",
- "session_id": "sess-{v}"
- },
- "country_case": "upper",
- "city_separator": ""
-}
-output = json.dumps(template)
-```
+## proxy_request Parameters
----
+| Parameter | Required | Default | Description |
+|-----------|----------|---------|-------------|
+| `url` | yes | | Target URL |
+| `method` | no | GET | HTTP method |
+| `headers` | no | | JSON string of headers |
+| `body` | no | | Request body string |
+| `proxyProvider` | no | auto | Provider name |
+| `proxyCountry` | no | | ISO country code (US, GB, DE) |
+| `sessionType` | no | rotating | `rotating` or `sticky` |
+| `timeout` | no | 30 | Seconds |
+| `maxRetries` | no | 3 | Retry with failover |
+
+## Transparent Proxy on HTTP Nodes
+
+The `httpRequest` and `httpScraper` nodes have built-in proxy support. Set `useProxy: true` and the proxy service handles everything -- provider selection, geo-targeting, and session type are all managed from the proxy service configuration (provider priorities, routing rules, default country).
+
+When the AI Agent uses `http_request` as a tool, it can set `useProxy: true` in the tool arguments. No other proxy fields needed.
+
+**When to use which:**
+- Use **http_request with useProxy: true** for proxied requests (simple, proxy service handles routing)
+- Use **proxy_request** tool only when you need explicit control over retries and failover
-## Guidelines
+## Notes
-1. **Always validate** the template with `python_code` before saving
-2. **Always test** the provider after setting credentials
-3. **Never hardcode** provider-specific logic -- the template handles everything
-4. If the user does not know their credentials, tell them to find username/password in their proxy provider's dashboard
-5. If test fails, check: credentials correct? Gateway host/port correct? Template encoding matches docs?
-6. Cost per GB is optional but helps the system prefer cheaper providers
-7. Priority lower = preferred; use 10-30 for primary, 50 for default, 70-90 for fallback
+- Always run `test_provider` after setting up a new provider
+- If the user does not know their credentials, they need to get them from their proxy provider dashboard
+- If test fails: check credentials, gateway host/port, and that the url_template matches the provider's format
+- Use `proxy_status` to check provider health before making requests
+- `sessionType: "sticky"` keeps the same IP across requests (good for scraping sessions)
+- `sessionType: "rotating"` gets a new IP each request (good for anonymity)