Skip to content

ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.

Add workflow automation engine and triggers

Juan Denis13bd84eda1c8
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml
new file mode 100644
index 00000000..469ce162
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.yml
@@ -0,0 +1,58 @@
+name: πŸ› Bug Report
+description: Report a bug to help us improve ServerKit
+labels: ["bug"]
+body:
+ - type: markdown
+ attributes:
+ value: |
+ Thanks for taking the time to fill out this bug report!
+ - type: textarea
+ id: description
+ attributes:
+ label: Describe the bug
+ description: A clear and concise description of what the bug is.
+ placeholder: Bug description
+ validations:
+ required: true
+ - type: textarea
+ id: reproduction
+ attributes:
+ label: Steps to reproduce
+ description: How can we reproduce this issue?
+ placeholder: |
+ 1. Go to '...'
+ 2. Click on '....'
+ 3. Scroll down to '....'
+ 4. See error
+ validations:
+ required: true
+ - type: textarea
+ id: expected-behavior
+ attributes:
+ label: Expected behavior
+ description: A clear and concise description of what you expected to happen.
+ validations:
+ required: true
+ - type: input
+ id: environment
+ attributes:
+ label: Environment
+ description: OS version, browser, ServerKit version, etc.
+ placeholder: e.g. Ubuntu 22.04, Chrome 120, ServerKit v0.8.0
+ validations:
+ required: true
+ - type: textarea
+ id: logs
+ attributes:
+ label: Logs
+ description: Please provide any relevant logs from the backend or browser console.
+ render: shell
+ - type: checkboxes
+ id: checks
+ attributes:
+ label: Checks
+ options:
+ - label: I have searched the [existing issues](https://github.com/jhd3197/ServerKit/issues).
+ required: true
+ - label: I am using the latest version of ServerKit.
+ required: true
diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml
new file mode 100644
index 00000000..730cf24a
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.yml
@@ -0,0 +1,28 @@
+name: πŸš€ Feature Request
+description: Suggest an idea for ServerKit
+labels: ["enhancement"]
+body:
+ - type: textarea
+ id: feature-description
+ attributes:
+ label: Is your feature request related to a problem?
+ description: A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+ validations:
+ required: true
+ - type: textarea
+ id: solution
+ attributes:
+ label: Describe the solution you'd like
+ description: A clear and concise description of what you want to happen.
+ validations:
+ required: true
+ - type: textarea
+ id: alternatives
+ attributes:
+ label: Describe alternatives you've considered
+ description: A clear and concise description of any alternative solutions or features you've considered.
+ - type: textarea
+ id: additional-context
+ attributes:
+ label: Additional context
+ description: Add any other context or screenshots about the feature request here.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 00000000..e800fa5d
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,24 @@
+## Description
+<!-- Provide a brief summary of the changes and the motivation behind them. -->
+
+## Related Issues
+<!-- Link any related issues using keywords like Fixes #123 or Closes #456. -->
+
+## Type of Change
+- [ ] πŸ› Bug fix (non-breaking change which fixes an issue)
+- [ ] πŸš€ New feature (non-breaking change which adds functionality)
+- [ ] ⚠️ Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] πŸ“ Documentation update
+- [ ] 🎨 UI/UX improvement
+
+## How Has This Been Tested?
+<!-- Describe the tests that you ran to verify your changes. -->
+
+## Checklist
+- [ ] My code follows the style guidelines of this project
+- [ ] I have performed a self-review of my own code
+- [ ] I have commented my code, particularly in hard-to-understand areas
+- [ ] I have made corresponding changes to the documentation
+- [ ] My changes generate no new warnings
+- [ ] I have added tests that prove my fix is effective or that my feature works
+- [ ] New and existing unit tests pass locally with my changes
diff --git a/.gitignore b/.gitignore
index 26e52b30..ad077178 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,63 +1,95 @@
-# Dependencies
+# --- ServerKit Consolidated .gitignore ---
+
+# Node / Frontend
/frontend/node_modules/
+/frontend/dist/
+/frontend/npm-debug.log*
+/frontend/yarn-debug.log*
+/frontend/yarn-error.log*
+/frontend/.eslintcache
+/frontend/.stylelintcache
+/frontend/coverage/
+/frontend/.next/
+/frontend/out/
+/frontend/build/
+
+# Python / Backend
/backend/venv/
+/backend/instance/
/backend/__pycache__/
-**/__pycache__/
-*.pyc
+/backend/.pytest_cache/
+/backend/.coverage
+/backend/htmlcov/
+/backend/.mypy_cache/
+/backend/.tox/
+/backend/*.pyc
+/backend/*.pyo
+/backend/*.pyd
+/backend/.env
+/backend/.venv
+/backend/pip-log.txt
+/backend/pip-delete-this-directory.txt
+/backend/dev-data/
+/venv/
-# Build outputs
-/frontend/dist/
-/agent/dist/
+# Go / Agent
/agent/serverkit-agent
/agent/serverkit-agent.exe
-*.exe
-*.msi
-*.deb
-*.rpm
-
-# IDE and editors
-.idea/
-.vscode/
-*.swp
-*.swo
-*~
+/agent/dist/
+/agent/vendor/
+/agent/*.test
+/agent/*.out
+/agent/bin/
-# Environment and secrets
+# Environment / Secrets
.env
.env.local
-.env.*.local
+.env.development.local
+.env.test.local
+.env.production.local
*.key
*.pem
*.crt
*.csr
+/backend/.env
+/frontend/.env.local
-# OS files
+# Docker
+.docker-compose.untracked.yml
+docker-compose.override.yml
+
+# OS / IDE
.DS_Store
+.DS_Store?
+._*
+.Spotlight-V100
+.Trashes
+ehthumbs.db
Thumbs.db
-nul
-
-# Temporary files
-*.tmp
-*.temp
-*.log
-coverage.out
-coverage.html
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+*.sublime-project
+*.sublime-workspace
-# Project specific
+# Project Specific
.planning_old/
.mcp.json
new_templates/
/.marketing/
-
-# Test artifacts
-*.test
-*.out
-/template-icons
-
-# Dev data directory (local development path overrides)
-backend/dev-data/
-/.vscode
-/.vscode1
-/.pr
-/.reviews
+/template-icons/
+/.pr/
+/.reviews/
/.claude/settings.local.json
+/serverkit.db
+/backend/instance/serverkit.db
+
+# Logs & Temp
+*.log
+*.tmp
+*.temp
+/logs/
+/tmp/
+nul
diff --git a/ROADMAP.md b/ROADMAP.md
index ac278abe..55fd4786 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -304,18 +304,19 @@ The phases below are ordered by priority. Higher phases ship first.
---
-## Phase 23: Advanced Workflow & Automation Engine (NEW)
+## Phase 23: Workflow & Automation Engine β€” Core (Completed)
**Priority: Critical**
Moving beyond static design to dynamic, event-driven automation. This turns ServerKit into a powerful automation hub.
-- [x] **Event Triggers:** Run workflows on Git push, health check failure, high CPU usage, or Webhook receipt.
-- [x] **Cron Integration:** Schedule workflows to run on recurring intervals (e.g., "Every Sunday at 2 AM, backup all DBs and rotate logs").
-- [x] **Cross-Server Actions:** "When Server A's database is backed up, sync it to Server B and notify Discord."
-- [x] **Logic Nodes:** If/Else conditions, loops, and custom Python/Shell script execution nodes.
-- [x] **State Machine:** Track workflow execution history, retries on failure, and execution logs.
-- [x] **Global Variables:** Pass data between workflow steps (e.g., take the output of a build step and pass it to a deployment step).
+- [x] **Visual Workflow Builder:** Node-based canvas with drag-and-drop nodes, connection validation, and config panels
+- [x] **Cron Integration:** Schedule workflows to run on recurring intervals (e.g., "Every Sunday at 2 AM, backup all DBs and rotate logs")
+- [x] **Manual Execution:** Trigger workflows on demand with optional context data
+- [x] **Execution History:** Track workflow execution status, per-node results, and timestamped logs
+- [x] **Script Nodes:** Custom Shell script execution nodes with output capture
+- [x] **Notification Nodes:** Send alerts via configured notification channels
+- [x] **One-Click Stack Deployment:** Deploy full infrastructure (databases, apps, domains) from a workflow diagram
---
@@ -332,7 +333,26 @@ Let users personalize what they see. Not everyone runs email servers or manages
---
-## Phase 25: Agent Fleet Management (Planned)
+## Phase 25: Workflow Engine β€” Triggers & Completion (Completed)
+
+**Priority: High**
+
+Complete the workflow engine with proper execution logic, missing triggers, and production-grade reliability.
+
+- [x] **DAG Execution:** Full directed acyclic graph traversal with parallel branch support (replace current linear BFS)
+- [x] **Logic Node Evaluation:** If/Else condition evaluation with true/false branching
+- [x] **Variable Interpolation:** Pass data between steps using `${node_id.field}` and `{{placeholder}}` syntax in node configs
+- [x] **Webhook Triggers:** Register `/hooks/<webhook_id>` endpoint to fire workflows on incoming HTTP requests
+- [x] **Event Triggers:** Run workflows on system events (health check failure, high CPU/memory, git push, app stopped)
+- [x] **Notification Templating:** Message placeholder substitution (`${node_id.stdout}`, `{{workflow_name}}`) in notification nodes
+- [x] **Execution Timeouts:** Configurable timeout per node (1–3600s) to prevent hung workflows
+- [x] **Retry on Failure:** Configurable retry count (0–5) and delay per node
+- [x] **Circular Dependency Detection:** Kahn's algorithm validates graph on save and before execution
+- [x] **Script Sandboxing:** Timeout enforcement, output size limits, explicit `bash -c`/`python3 -c` execution
+
+---
+
+## Phase 26: Agent Fleet Management (Planned)
**Priority: High**
@@ -352,7 +372,7 @@ Level up agent management from "connect and monitor" to full fleet control.
---
-## Phase 26: Cross-Server Monitoring Dashboard (Planned)
+## Phase 27: Cross-Server Monitoring Dashboard (Planned)
**Priority: High**
@@ -371,7 +391,7 @@ Fleet-wide visibility β€” see everything at a glance and catch problems early.
---
-## Phase 27: Agent Plugin System (Planned)
+## Phase 28: Agent Plugin System (Planned)
**Priority: High**
@@ -393,7 +413,7 @@ Make the agent extensible β€” let users add custom capabilities without modifyin
---
-## Phase 28: Server Templates & Config Sync (Planned)
+## Phase 29: Server Templates & Config Sync (Planned)
**Priority: Medium**
@@ -412,7 +432,7 @@ Define what a server should look like, apply it, and detect when it drifts.
---
-## Phase 29: Multi-Tenancy & Workspaces (Planned)
+## Phase 30: Multi-Tenancy & Workspaces (Planned)
**Priority: Medium**
@@ -431,7 +451,7 @@ Isolate servers by team, client, or project. Essential for agencies, MSPs, and l
---
-## Phase 30: Advanced SSL Features (Planned)
+## Phase 31: Advanced SSL Features (Planned)
**Priority: Medium**
@@ -445,7 +465,7 @@ Isolate servers by team, client, or project. Essential for agencies, MSPs, and l
---
-## Phase 31: DNS Zone Management (Planned)
+## Phase 32: DNS Zone Management (Planned)
**Priority: Medium**
@@ -462,7 +482,7 @@ Full DNS record management with provider API integration.
---
-## Phase 32: Nginx Advanced Configuration (Planned)
+## Phase 33: Nginx Advanced Configuration (Planned)
**Priority: Medium**
@@ -480,7 +500,7 @@ Go beyond basic virtual hosts β€” full reverse proxy and performance configurati
---
-## Phase 33: Status Page & Health Checks (Planned)
+## Phase 34: Status Page & Health Checks (Planned)
**Priority: Medium**
@@ -499,7 +519,7 @@ Public-facing status page and automated health monitoring.
---
-## Phase 34: Server Provisioning APIs (Planned)
+## Phase 35: Server Provisioning APIs (Planned)
**Priority: Medium**
@@ -518,7 +538,7 @@ Spin up and manage cloud servers directly from the panel.
---
-## Phase 35: Performance Optimization (Planned)
+## Phase 36: Performance Optimization (Planned)
**Priority: Low**
@@ -531,7 +551,7 @@ Spin up and manage cloud servers directly from the panel.
---
-## Phase 36: Mobile App (Future)
+## Phase 37: Mobile App (Future)
**Priority: Low β€” v3.0+**
@@ -543,7 +563,7 @@ Spin up and manage cloud servers directly from the panel.
---
-## Phase 37: Marketplace & Extensions (Future)
+## Phase 38: Marketplace & Extensions (Future)
**Priority: Low β€” v3.0+**
@@ -566,7 +586,7 @@ Spin up and manage cloud servers directly from the panel.
| v1.3.0 | Email server, API enhancements | Completed |
| v1.4.0 | Team & permissions, SSO & OAuth login | Completed |
| v1.5.0 | New UI, Visual Designer, Services Page | Current |
-| v1.6.0 | Advanced Automation Engine, fleet management | Planned |
+| v1.6.0 | Workflow triggers & completion, fleet management | Planned |
| v1.7.0 | Cross-server monitoring, agent plugin system | Planned |
| v1.8.0 | Server templates, multi-tenancy | Planned |
| v1.9.0 | Advanced SSL, DNS management, Nginx config | Planned |
diff --git a/VERSION b/VERSION
index e05cb332..88c5fb89 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-1.3.8
+1.4.0
diff --git a/backend/app/api/workflows.py b/backend/app/api/workflows.py
index 6049430a..3ac8c2af 100644
--- a/backend/app/api/workflows.py
+++ b/backend/app/api/workflows.py
@@ -1,4 +1,6 @@
import json
+import secrets
+from datetime import datetime
from flask import Blueprint, request, jsonify
from flask_jwt_extended import jwt_required, get_jwt_identity
from app import db
@@ -7,6 +9,18 @@ from app.models import Workflow, User, WorkflowExecution, WorkflowLog
workflows_bp = Blueprint('workflows', __name__)
+def _validate_workflow_graph(data):
+ """Validate workflow graph for cycles if nodes/edges are present."""
+ nodes = data.get('nodes', [])
+ edges = data.get('edges', [])
+ if nodes and edges:
+ from app.services.workflow_engine import WorkflowEngine
+ err = WorkflowEngine.validate_graph(nodes, edges)
+ if err:
+ return err
+ return None
+
+
@workflows_bp.route('', methods=['GET'])
@jwt_required()
def get_workflows():
@@ -55,6 +69,16 @@ def create_workflow():
if not name:
return jsonify({'error': 'Name is required'}), 400
+ # Validate graph for cycles
+ cycle_err = _validate_workflow_graph(data)
+ if cycle_err:
+ return jsonify({'error': cycle_err}), 400
+
+ # Auto-generate webhook_id if trigger is webhook
+ trigger_config = data.get('trigger_config', {})
+ if data.get('trigger_type') == 'webhook' and not trigger_config.get('webhook_id'):
+ trigger_config['webhook_id'] = secrets.token_urlsafe(24)
+
workflow = Workflow(
name=name,
description=data.get('description', ''),
@@ -63,7 +87,7 @@ def create_workflow():
viewport=json.dumps(data.get('viewport')) if data.get('viewport') else None,
is_active=data.get('is_active', False),
trigger_type=data.get('trigger_type', 'manual'),
- trigger_config=json.dumps(data.get('trigger_config', {})),
+ trigger_config=json.dumps(trigger_config),
user_id=current_user_id
)
@@ -94,6 +118,11 @@ def update_workflow(workflow_id):
if not data:
return jsonify({'error': 'No data provided'}), 400
+ # Validate graph for cycles
+ cycle_err = _validate_workflow_graph(data)
+ if cycle_err:
+ return jsonify({'error': cycle_err}), 400
+
if 'name' in data:
workflow.name = data['name']
if 'description' in data:
@@ -104,14 +133,19 @@ def update_workflow(workflow_id):
workflow.edges = json.dumps(data['edges'])
if 'viewport' in data:
workflow.viewport = json.dumps(data['viewport']) if data['viewport'] else None
-
+
# Automation fields
if 'is_active' in data:
workflow.is_active = data['is_active']
if 'trigger_type' in data:
workflow.trigger_type = data['trigger_type']
if 'trigger_config' in data:
- workflow.trigger_config = json.dumps(data['trigger_config'])
+ trigger_config = data['trigger_config']
+ # Auto-generate webhook_id if switching to webhook trigger
+ if data.get('trigger_type') == 'webhook' and not trigger_config.get('webhook_id'):
+ existing = json.loads(workflow.trigger_config) if workflow.trigger_config else {}
+ trigger_config['webhook_id'] = existing.get('webhook_id') or secrets.token_urlsafe(24)
+ workflow.trigger_config = json.dumps(trigger_config)
db.session.commit()
@@ -254,3 +288,86 @@ def deploy_workflow(workflow_id):
else:
# Partial success or errors - return 200 with error details
return jsonify(result), 200
+
+
+@workflows_bp.route('/hooks/<webhook_id>', methods=['POST'])
+def webhook_trigger(webhook_id):
+ """
+ Public webhook endpoint to trigger a workflow.
+ No JWT required β€” the webhook_id acts as the authentication token.
+ """
+ from app.services.workflow_engine import WorkflowEngine, CycleDetectedError
+
+ # Find the workflow with this webhook_id
+ workflows = Workflow.query.filter_by(
+ is_active=True,
+ trigger_type='webhook'
+ ).all()
+
+ target_workflow = None
+ for wf in workflows:
+ try:
+ config = json.loads(wf.trigger_config) if wf.trigger_config else {}
+ if config.get('webhook_id') == webhook_id:
+ target_workflow = wf
+ break
+ except (json.JSONDecodeError, TypeError):
+ continue
+
+ if not target_workflow:
+ return jsonify({'error': 'Webhook not found'}), 404
+
+ # Build context from the incoming request
+ context = {
+ 'webhook_id': webhook_id,
+ 'method': request.method,
+ 'headers': dict(request.headers),
+ 'triggered_at': datetime.utcnow().isoformat()
+ }
+
+ # Include request body
+ if request.is_json:
+ context['body'] = request.get_json(silent=True) or {}
+ else:
+ body = request.get_data(as_text=True)
+ context['body'] = body[:8192] if body else ''
+
+ # Include query parameters
+ if request.args:
+ context['query'] = dict(request.args)
+
+ try:
+ execution_id = WorkflowEngine.execute_workflow(
+ workflow_id=target_workflow.id,
+ trigger_type='webhook',
+ context=context
+ )
+ return jsonify({
+ 'message': 'Workflow triggered',
+ 'execution_id': execution_id,
+ 'workflow': target_workflow.name
+ }), 200
+ except CycleDetectedError as e:
+ return jsonify({'error': str(e)}), 400
+ except Exception as e:
+ return jsonify({'error': f'Execution failed: {str(e)}'}), 500
+
+
+@workflows_bp.route('/validate', methods=['POST'])
+@jwt_required()
+def validate_workflow():
+ """Validate a workflow graph for cycles and other issues."""
+ from app.services.workflow_engine import WorkflowEngine
+
+ data = request.get_json()
+ if not data:
+ return jsonify({'error': 'No data provided'}), 400
+
+ nodes = data.get('nodes', [])
+ edges = data.get('edges', [])
+
+ err = WorkflowEngine.validate_graph(nodes, edges)
+ if err:
+ return jsonify({'valid': False, 'error': err}), 200
+
+ return jsonify({'valid': True}), 200
diff --git a/backend/app/services/docker_service.py b/backend/app/services/docker_service.py
index b478db0c..234c0492 100644
--- a/backend/app/services/docker_service.py
+++ b/backend/app/services/docker_service.py
@@ -217,6 +217,13 @@ class DockerService:
capture_output=True, text=True
)
if result.returncode == 0:
+ try:
+ from app.services.workflow_engine import WorkflowEventBus
+ WorkflowEventBus.emit('app_stopped', {
+ 'container_id': container_id
+ })
+ except Exception:
+ pass
return {'success': True}
return {'success': False, 'error': result.stderr}
except Exception as e:
diff --git a/backend/app/services/environment_health_service.py b/backend/app/services/environment_health_service.py
index 6398c403..8fcdc6d4 100644
--- a/backend/app/services/environment_health_service.py
+++ b/backend/app/services/environment_health_service.py
@@ -90,6 +90,18 @@ class EnvironmentHealthService:
site.last_health_check = datetime.utcnow()
db.session.commit()
+ # Emit event for workflow triggers on unhealthy status
+ if overall in ('unhealthy', 'degraded'):
+ try:
+ from app.services.workflow_engine import WorkflowEventBus
+ WorkflowEventBus.emit('health_check_failed', {
+ 'site_id': site_id,
+ 'overall_status': overall,
+ 'checks': checks
+ })
+ except Exception:
+ pass
+
return {
'success': True,
'site_id': site_id,
diff --git a/backend/app/services/git_service.py b/backend/app/services/git_service.py
index 01dc809c..03491df5 100644
--- a/backend/app/services/git_service.py
+++ b/backend/app/services/git_service.py
@@ -434,6 +434,17 @@ class GitService:
'message': f'Ignoring push to {ref}, configured branch is {branch}'
}
+ # Emit event for workflow triggers
+ try:
+ from app.services.workflow_engine import WorkflowEventBus
+ WorkflowEventBus.emit('git_push', {
+ 'app_id': app_id,
+ 'branch': branch,
+ 'ref': ref
+ })
+ except Exception:
+ pass
+
# Trigger deployment
return cls.deploy(app_id)
diff --git a/backend/app/services/monitoring_service.py b/backend/app/services/monitoring_service.py
index 57fd9a18..44beae86 100644
--- a/backend/app/services/monitoring_service.py
+++ b/backend/app/services/monitoring_service.py
@@ -280,6 +280,25 @@ class MonitoringService:
# Send to all configured notification channels (Discord, Slack, Telegram, etc.)
NotificationService.send_all(alerts_to_send)
+ # Emit events for workflow triggers
+ try:
+ from app.services.workflow_engine import WorkflowEventBus
+ for alert in alerts_to_send:
+ if alert['type'] == 'cpu':
+ WorkflowEventBus.emit('high_cpu', {
+ 'percent': alert.get('value'),
+ 'threshold': alert.get('threshold'),
+ 'severity': alert.get('severity')
+ })
+ elif alert['type'] == 'memory':
+ WorkflowEventBus.emit('high_memory', {
+ 'percent': alert.get('value'),
+ 'threshold': alert.get('threshold'),
+ 'severity': alert.get('severity')
+ })
+ except Exception:
+ pass
+
@classmethod
def log_alert(cls, alerts: List[Dict]) -> None:
"""Log alerts to file."""
diff --git a/backend/app/services/workflow_engine.py b/backend/app/services/workflow_engine.py
index c351151b..f5d75647 100644
--- a/backend/app/services/workflow_engine.py
+++ b/backend/app/services/workflow_engine.py
@@ -1,14 +1,21 @@
"""
Advanced Workflow & Automation Engine
-Executes event-driven workflows with logic nodes, triggers, and cross-server actions.
+Executes event-driven workflows with DAG-based execution, logic branching,
+variable interpolation, timeouts, retries, and script sandboxing.
"""
import json
import logging
+import os
+import re
+import signal
+import subprocess
+import threading
import traceback
+from collections import deque
from datetime import datetime
-from typing import Dict, List, Any, Optional
+from typing import Dict, List, Any, Optional, Set, Tuple
from app import db
from app.models import Workflow, WorkflowExecution, WorkflowLog, User
@@ -17,23 +24,100 @@ from app.services.docker_service import DockerService
from app.services.database_service import DatabaseService
from app.services.notification_service import NotificationService
+logger = logging.getLogger(__name__)
+
+# Defaults for node execution
+DEFAULT_TIMEOUT = 300 # 5 minutes
+MAX_TIMEOUT = 3600 # 1 hour
+DEFAULT_RETRY_COUNT = 0
+MAX_RETRY_COUNT = 5
+DEFAULT_RETRY_DELAY = 5 # seconds
+MAX_OUTPUT_SIZE = 1024 * 512 # 512 KB
+
+
+class CycleDetectedError(Exception):
+ """Raised when a cycle is detected in the workflow graph."""
+ pass
+
+
+class NodeTimeoutError(Exception):
+ """Raised when a node exceeds its timeout."""
+ pass
+
class WorkflowEngine:
- """Engine for executing advanced workflows."""
+ """Engine for executing advanced workflows with DAG support."""
+
+ # ------------------------------------------------------------------
+ # Public API
+ # ------------------------------------------------------------------
@staticmethod
- def execute_workflow(workflow_id: int, trigger_type: str = 'manual', context: Dict[str, Any] = None) -> int:
+ def validate_graph(nodes: List[Dict], edges: List[Dict]) -> Optional[str]:
+ """
+ Validate a workflow graph for cycles.
+
+ Returns None if valid, or an error message string if a cycle is found.
+ """
+ adj: Dict[str, List[str]] = {}
+ node_ids = {n['id'] for n in nodes}
+
+ for edge in edges:
+ src = edge['source']
+ if src not in adj:
+ adj[src] = []
+ adj[src].append(edge['target'])
+
+ # Kahn's algorithm for cycle detection
+ in_degree = {nid: 0 for nid in node_ids}
+ for src, targets in adj.items():
+ for t in targets:
+ if t in in_degree:
+ in_degree[t] += 1
+
+ queue = deque(nid for nid, deg in in_degree.items() if deg == 0)
+ visited_count = 0
+
+ while queue:
+ nid = queue.popleft()
+ visited_count += 1
+ for neighbor in adj.get(nid, []):
+ if neighbor in in_degree:
+ in_degree[neighbor] -= 1
+ if in_degree[neighbor] == 0:
+ queue.append(neighbor)
+
+ if visited_count < len(node_ids):
+ # Find nodes involved in the cycle for a helpful message
+ cycle_nodes = [nid for nid, deg in in_degree.items() if deg > 0]
+ node_labels = {}
+ for n in nodes:
+ node_labels[n['id']] = n.get('data', {}).get('label', n['id'])
+ cycle_labels = [node_labels.get(nid, nid) for nid in cycle_nodes[:5]]
+ return f"Cycle detected involving: {', '.join(cycle_labels)}"
+
+ return None
+
+ @staticmethod
+ def execute_workflow(workflow_id: int, trigger_type: str = 'manual',
+ context: Dict[str, Any] = None) -> int:
"""
Execute a workflow by ID.
- Returns:
- ID of the created WorkflowExecution
+ Returns the ID of the created WorkflowExecution.
"""
workflow = Workflow.query.get(workflow_id)
if not workflow:
raise ValueError(f"Workflow {workflow_id} not found")
- # Create execution record
+ nodes = json.loads(workflow.nodes) if workflow.nodes else []
+ edges = json.loads(workflow.edges) if workflow.edges else []
+
+ # Validate graph before executing
+ cycle_err = WorkflowEngine.validate_graph(nodes, edges)
+ if cycle_err:
+ raise CycleDetectedError(cycle_err)
+
execution = WorkflowExecution(
workflow_id=workflow_id,
trigger_type=trigger_type,
@@ -44,15 +128,13 @@ class WorkflowEngine:
db.session.add(execution)
db.session.commit()
- # Update workflow last run
workflow.last_run_at = execution.started_at
db.session.commit()
try:
- # Run the execution in the background or synchronously for now
- WorkflowEngine._run_execution(execution.id)
+ WorkflowEngine._run_execution(execution.id, nodes, edges)
except Exception as e:
- WorkflowEngine.log(execution.id, f"Engine Error: {str(e)}", level='ERROR')
+ WorkflowEngine._log(execution.id, f"Engine Error: {str(e)}", level='ERROR')
execution.status = 'failed'
execution.completed_at = datetime.utcnow()
workflow.last_status = 'failed'
@@ -60,169 +142,564 @@ class WorkflowEngine:
return execution.id
+ # ------------------------------------------------------------------
+ # DAG Execution
+ # ------------------------------------------------------------------
+
@staticmethod
- def _run_execution(execution_id: int):
- """Internal method to run the execution logic."""
+ def _run_execution(execution_id: int, nodes: List[Dict], edges: List[Dict]):
+ """Run the workflow using topological DAG execution with branch support."""
execution = WorkflowExecution.query.get(execution_id)
workflow = execution.workflow
-
- WorkflowEngine.log(execution_id, f"Starting workflow: {workflow.name}")
- try:
- nodes = json.loads(workflow.nodes) if workflow.nodes else []
- edges = json.loads(workflow.edges) if workflow.edges else []
-
- if not nodes:
- WorkflowEngine.log(execution_id, "Workflow has no nodes", level='WARNING')
- execution.status = 'success'
- execution.completed_at = datetime.utcnow()
- db.session.commit()
- return
-
- # Build adjacency list and find start nodes
- adj = {}
- for edge in edges:
- source = edge['source']
- target = edge['target']
- if source not in adj: adj[source] = []
- adj[source].append(target)
-
- # Find trigger nodes or start nodes (no incoming edges)
- incoming_count = {node['id']: 0 for node in nodes}
- for edge in edges:
- target = edge['target']
- if target in incoming_count:
- incoming_count[target] += 1
-
- # Start with nodes that have no incoming edges (usually triggers)
- queue = [node for node in nodes if incoming_count[node['id']] == 0]
-
- if not queue:
- # If everything is cyclic, just start with the first node
- queue = [nodes[0]]
-
- # Execution state
- results = {}
- context = json.loads(execution.context) if execution.context else {}
-
- # Simple linear/BFS execution for now
- # TODO: Implement full DAG execution with branch support
- processed = set()
-
- while queue:
- node = queue.pop(0)
- if node['id'] in processed:
- continue
-
- WorkflowEngine.log(execution_id, f"Executing node: {node.get('data', {}).get('label', node['id'])} ({node['type']})", node_id=node['id'])
-
- # Execute node
- try:
- node_result = WorkflowEngine._execute_node(node, edges, execution, context, results)
- results[node['id']] = node_result
-
- if not node_result.get('success', True):
- WorkflowEngine.log(execution_id, f"Node failed: {node_result.get('error')}", level='ERROR', node_id=node['id'])
- if node_result.get('critical', True):
- execution.status = 'failed'
- break
-
- # Add next nodes to queue
- if node['id'] in adj:
- for next_id in adj[node['id']]:
- next_node = next((n for n in nodes if n['id'] == next_id), None)
- if next_node:
- queue.append(next_node)
-
- except Exception as e:
- WorkflowEngine.log(execution_id, f"Node Execution Error: {str(e)}", level='ERROR', node_id=node['id'])
- WorkflowEngine.log(execution_id, traceback.format_exc(), level='DEBUG', node_id=node['id'])
- execution.status = 'failed'
- break
-
- processed.add(node['id'])
-
- if execution.status != 'failed':
- execution.status = 'success'
-
- execution.results = json.dumps(results)
- execution.completed_at = datetime.utcnow()
- workflow.last_status = execution.status
- db.session.commit()
-
- WorkflowEngine.log(execution_id, f"Workflow finished with status: {execution.status}")
+ WorkflowEngine._log(execution_id, f"Starting workflow: {workflow.name}")
- except Exception as e:
- WorkflowEngine.log(execution_id, f"Execution Loop Error: {str(e)}", level='ERROR')
- WorkflowEngine.log(execution_id, traceback.format_exc(), level='DEBUG')
- execution.status = 'failed'
+ if not nodes:
+ WorkflowEngine._log(execution_id, "Workflow has no nodes", level='WARNING')
+ execution.status = 'success'
execution.completed_at = datetime.utcnow()
db.session.commit()
+ return
+
+ # Build graph structures
+ node_map = {n['id']: n for n in nodes}
+ adj: Dict[str, List[Tuple[str, str]]] = {} # source -> [(target, sourceHandle)]
+ in_degree: Dict[str, int] = {n['id']: 0 for n in nodes}
+
+ for edge in edges:
+ src = edge['source']
+ tgt = edge['target']
+ src_handle = edge.get('sourceHandle', 'output')
+ if src not in adj:
+ adj[src] = []
+ adj[src].append((tgt, src_handle))
+ if tgt in in_degree:
+ in_degree[tgt] += 1
+
+ # Start with root nodes (no incoming edges)
+ ready = deque(nid for nid, deg in in_degree.items() if deg == 0)
+ if not ready:
+ ready.append(nodes[0]['id'])
+
+ context = json.loads(execution.context) if execution.context else {}
+ results: Dict[str, Dict] = {}
+ processed: Set[str] = set()
+ # Track which branches are active (for logic_if gating)
+ # Maps node_id -> set of sourceHandles that were activated
+ active_branches: Dict[str, Set[str]] = {}
+ failed = False
+
+ while ready and not failed:
+ node_id = ready.popleft()
+
+ if node_id in processed:
+ continue
+
+ node = node_map.get(node_id)
+ if not node:
+ continue
+
+ # Check if this node is gated by a logic_if branch
+ if not WorkflowEngine._is_node_reachable(node_id, edges, active_branches, processed):
+ processed.add(node_id)
+ # Still decrement successors so they can become ready
+ for tgt, _ in adj.get(node_id, []):
+ if tgt in in_degree:
+ in_degree[tgt] -= 1
+ if in_degree[tgt] == 0:
+ ready.append(tgt)
+ continue
+
+ node_label = node.get('data', {}).get('label', node_id)
+ WorkflowEngine._log(execution_id, f"Executing node: {node_label} ({node['type']})", node_id=node_id)
+
+ try:
+ node_result = WorkflowEngine._execute_node_with_retry(
+ node, edges, execution, context, results
+ )
+ results[node_id] = node_result
+
+ if not node_result.get('success', True):
+ WorkflowEngine._log(
+ execution_id,
+ f"Node failed: {node_result.get('error', 'unknown')}",
+ level='ERROR', node_id=node_id
+ )
+ if node_result.get('critical', True):
+ failed = True
+ break
+
+ # For logic_if nodes, record which branch was taken
+ if node['type'] == 'logic_if':
+ branch = node_result.get('branch', 'true')
+ if node_id not in active_branches:
+ active_branches[node_id] = set()
+ active_branches[node_id].add(branch)
+
+ # Enqueue successor nodes whose in-degree reaches 0
+ for tgt, src_handle in adj.get(node_id, []):
+ if tgt in in_degree:
+ in_degree[tgt] -= 1
+ if in_degree[tgt] == 0:
+ ready.append(tgt)
+
+ except Exception as e:
+ WorkflowEngine._log(execution_id, f"Node Execution Error: {str(e)}", level='ERROR', node_id=node_id)
+ WorkflowEngine._log(execution_id, traceback.format_exc(), level='DEBUG', node_id=node_id)
+ failed = True
+ break
+
+ processed.add(node_id)
+
+ execution.status = 'failed' if failed else 'success'
+ execution.results = json.dumps(results)
+ execution.completed_at = datetime.utcnow()
+ workflow.last_status = execution.status
+ db.session.commit()
+
+ WorkflowEngine._log(execution_id, f"Workflow finished with status: {execution.status}")
+
+ @staticmethod
+ def _is_node_reachable(node_id: str, edges: List[Dict],
+ active_branches: Dict[str, Set[str]],
+ processed: Set[str]) -> bool:
+ """
+ Check if a node should execute based on logic_if branching.
+
+ A node is unreachable if ALL its incoming edges from logic_if nodes
+ come through inactive branches.
+ """
+ incoming_from_logic = []
+
+ for edge in edges:
+ if edge['target'] != node_id:
+ continue
+ src = edge['source']
+ src_handle = edge.get('sourceHandle', 'output')
+
+ # Only gate on logic_if nodes that have already been processed
+ if src in active_branches:
+ incoming_from_logic.append((src, src_handle))
+
+ if not incoming_from_logic:
+ return True # No logic_if gating, always reachable
+
+ # Reachable if at least one logic_if branch leading here is active
+ for src, src_handle in incoming_from_logic:
+ if src_handle in active_branches[src]:
+ return True
+
+ return False
+
+ # ------------------------------------------------------------------
+ # Node Execution with Retry
+ # ------------------------------------------------------------------
@staticmethod
- def _execute_node(node: Dict, edges: List, execution: WorkflowExecution, context: Dict, results: Dict) -> Dict:
+ def _execute_node_with_retry(node: Dict, edges: List, execution: WorkflowExecution,
+ context: Dict, results: Dict) -> Dict:
+ """Execute a node with retry support."""
+ node_data = node.get('data', {})
+ retry_count = min(int(node_data.get('retryCount', DEFAULT_RETRY_COUNT)), MAX_RETRY_COUNT)
+ retry_delay = max(1, int(node_data.get('retryDelay', DEFAULT_RETRY_DELAY)))
+
+ last_result = None
+ for attempt in range(retry_count + 1):
+ if attempt > 0:
+ WorkflowEngine._log(
+ execution.id,
+ f"Retry {attempt}/{retry_count} after {retry_delay}s",
+ node_id=node['id']
+ )
+ import time
+ time.sleep(retry_delay)
+
+ last_result = WorkflowEngine._execute_node(node, edges, execution, context, results)
+
+ if last_result.get('success', True):
+ return last_result
+
+ return last_result
+
+ # ------------------------------------------------------------------
+ # Node Execution
+ # ------------------------------------------------------------------
+
+ @staticmethod
+ def _execute_node(node: Dict, edges: List, execution: WorkflowExecution,
+ context: Dict, results: Dict) -> Dict:
"""Execute a single node and return its results."""
node_type = node.get('type')
node_data = node.get('data', {})
-
+
if node_type == 'trigger':
return {'success': True, 'output': context}
-
+
elif node_type in ('database', 'dockerApp', 'service', 'domain'):
- # Delegate to WorkflowService
res = WorkflowService.deploy_node(node, edges, execution.workflow.user_id, results)
return res
-
+
elif node_type == 'notification':
- channel = node_data.get('channel', 'discord')
- message = node_data.get('message', 'Workflow notification')
- # Replace placeholders in message from context/results
- # TODO: Implementation
-
- NotificationService.send_notification(
- user_id=execution.workflow.user_id,
- title=f"Workflow: {execution.workflow.name}",
- message=message,
- level='info'
- )
- return {'success': True}
-
+ return WorkflowEngine._execute_notification(node, execution, context, results)
+
elif node_type == 'script':
- script_type = node_data.get('language', 'bash')
- content = node_data.get('content', '')
-
- if script_type == 'bash':
- import subprocess
- try:
- res = subprocess.run(content, shell=True, capture_output=True, text=True)
- return {
- 'success': res.returncode == 0,
- 'stdout': res.stdout,
- 'stderr': res.stderr,
- 'returncode': res.returncode
- }
- except Exception as e:
- return {'success': False, 'error': str(e)}
-
- return {'success': False, 'error': f"Unsupported script language: {script_type}"}
+ return WorkflowEngine._execute_script(node, execution, context, results)
elif node_type == 'logic_if':
- condition = node_data.get('condition', '')
- # TODO: Evaluate condition using context/results
- return {'success': True, 'branch': 'true'} # Dummy
-
- return {'success': True, 'message': f"Node type {node_type} not implemented yet"}
+ return WorkflowEngine._execute_logic_if(node, context, results)
+
+ return {'success': True, 'message': f"Node type {node_type} passed through"}
+
+ # ------------------------------------------------------------------
+ # Logic If Evaluation
+ # ------------------------------------------------------------------
@staticmethod
- def log(execution_id: int, message: str, level: str = 'INFO', node_id: str = None):
+ def _execute_logic_if(node: Dict, context: Dict, results: Dict) -> Dict:
+ """
+ Evaluate a logic_if condition.
+
+ The condition is a Python expression that has access to:
+ - results: dict of {node_id: node_result}
+ - context: the workflow execution context
+ """
+ node_data = node.get('data', {})
+ condition = node_data.get('condition', '').strip()
+
+ if not condition:
+ return {'success': True, 'branch': 'true'}
+
+ # Build a safe evaluation namespace
+ eval_globals = {"__builtins__": {}}
+ eval_locals = {
+ 'results': results,
+ 'context': context,
+ # Expose common helpers
+ 'len': len,
+ 'str': str,
+ 'int': int,
+ 'float': float,
+ 'bool': bool,
+ 'abs': abs,
+ 'min': min,
+ 'max': max,
+ 'any': any,
+ 'all': all,
+ 'isinstance': isinstance,
+ }
+
+ try:
+ result = eval(condition, eval_globals, eval_locals)
+ branch = 'true' if result else 'false'
+ return {
+ 'success': True,
+ 'branch': branch,
+ 'condition': condition,
+ 'evaluated': bool(result)
+ }
+ except Exception as e:
+ return {
+ 'success': False,
+ 'error': f"Condition evaluation failed: {str(e)}",
+ 'condition': condition,
+ 'branch': 'false',
+ 'critical': False # Don't kill the whole workflow for a bad condition
+ }
+
+ # ------------------------------------------------------------------
+ # Variable Interpolation
+ # ------------------------------------------------------------------
+
+ @staticmethod
+ def _interpolate(text: str, context: Dict, results: Dict,
+ execution: Optional[WorkflowExecution] = None) -> str:
+ """
+ Replace variable placeholders in text.
+
+ Supported syntax:
+ - ${node_id.field} β€” access a specific field from a node's result
+ - ${node_id.output} β€” shorthand for the output field
+ - {{workflow_name}} β€” built-in workflow variables
+ - {{execution_id}} β€” current execution ID
+ - {{started_at}} β€” execution start time
+ - {{context.field}} β€” access context fields
+ """
+ if not text or not isinstance(text, str):
+ return text
+
+ # Replace ${node_id.field} patterns
+ def replace_node_var(match):
+ node_id = match.group(1)
+ field = match.group(2)
+ node_result = results.get(node_id, {})
+ if field == 'output' and 'output' not in node_result:
+ # Try stdout as fallback for script nodes
+ return str(node_result.get('stdout', ''))
+ return str(node_result.get(field, ''))
+
+ text = re.sub(r'\$\{([^.}]+)\.([^}]+)\}', replace_node_var, text)
+
+ # Replace {{builtin}} patterns
+ builtins = {
+ 'workflow_name': execution.workflow.name if execution else '',
+ 'execution_id': str(execution.id) if execution else '',
+ 'started_at': execution.started_at.isoformat() if execution and execution.started_at else '',
+ 'trigger_type': execution.trigger_type if execution else '',
+ }
+
+ # Add context.* variables
+ for key, value in context.items():
+ builtins[f'context.{key}'] = str(value)
+
+ for key, value in builtins.items():
+ text = text.replace('{{' + key + '}}', value)
+
+ # Replace {{node_id.field}} as alternative syntax
+ def replace_node_var_braces(match):
+ node_id = match.group(1)
+ field = match.group(2)
+ node_result = results.get(node_id, {})
+ return str(node_result.get(field, ''))
+
+ text = re.sub(r'\{\{([^.}]+)\.([^}]+)\}\}', replace_node_var_braces, text)
+
+ return text
+
+ # ------------------------------------------------------------------
+ # Notification Node
+ # ------------------------------------------------------------------
+
+ @staticmethod
+ def _execute_notification(node: Dict, execution: WorkflowExecution,
+ context: Dict, results: Dict) -> Dict:
+ """Execute a notification node with variable interpolation."""
+ node_data = node.get('data', {})
+ channel = node_data.get('channel', 'system')
+ message = node_data.get('message', 'Workflow notification')
+
+ # Interpolate variables in the message
+ message = WorkflowEngine._interpolate(message, context, results, execution)
+
+ title = f"Workflow: {execution.workflow.name}"
+
+ # Build alert in the format NotificationService expects
+ alerts = [{
+ 'type': 'workflow',
+ 'severity': 'info',
+ 'message': message,
+ 'value': '',
+ 'threshold': ''
+ }]
+
+ try:
+ if channel == 'system' or channel == 'all':
+ result = NotificationService.send_all(alerts)
+ elif channel == 'discord':
+ config = NotificationService.get_config().get('discord', {})
+ result = NotificationService.send_discord(alerts, config)
+ elif channel == 'slack':
+ config = NotificationService.get_config().get('slack', {})
+ result = NotificationService.send_slack(alerts, config)
+ elif channel == 'email':
+ config = NotificationService.get_config().get('email', {})
+ result = NotificationService.send_email(alerts, config)
+ elif channel == 'telegram':
+ config = NotificationService.get_config().get('telegram', {})
+ result = NotificationService.send_telegram(alerts, config)
+ else:
+ result = NotificationService.send_all(alerts)
+
+ return {'success': result.get('success', True), 'channel': channel}
+ except Exception as e:
+ return {'success': False, 'error': str(e), 'critical': False}
+
+ # ------------------------------------------------------------------
+ # Script Node (Sandboxed)
+ # ------------------------------------------------------------------
+
+ @staticmethod
+ def _execute_script(node: Dict, execution: WorkflowExecution,
+ context: Dict, results: Dict) -> Dict:
+ """Execute a script node with timeout, output limits, and variable interpolation."""
+ node_data = node.get('data', {})
+ script_type = node_data.get('language', 'bash')
+ content = node_data.get('content', '')
+ timeout = min(int(node_data.get('timeout', DEFAULT_TIMEOUT)), MAX_TIMEOUT)
+
+ if not content.strip():
+ return {'success': True, 'stdout': '', 'stderr': '', 'returncode': 0}
+
+ # Interpolate variables in script content
+ content = WorkflowEngine._interpolate(content, context, results, execution)
+
+ # Build environment with node results available as env vars
+ env = os.environ.copy()
+ env['WORKFLOW_ID'] = str(execution.workflow_id)
+ env['EXECUTION_ID'] = str(execution.id)
+ env['TRIGGER_TYPE'] = execution.trigger_type or 'manual'
+
+ for nid, nresult in results.items():
+ safe_id = re.sub(r'[^a-zA-Z0-9_]', '_', nid).upper()
+ if isinstance(nresult, dict):
+ stdout = nresult.get('stdout', nresult.get('output', ''))
+ if isinstance(stdout, str):
+ env[f'NODE_{safe_id}_OUTPUT'] = stdout[:4096]
+ rc = nresult.get('returncode')
+ if rc is not None:
+ env[f'NODE_{safe_id}_RC'] = str(rc)
+
+ try:
+ if script_type == 'bash':
+ cmd = ['bash', '-c', content]
+ elif script_type == 'python':
+ cmd = ['python3', '-c', content]
+ else:
+ return {'success': False, 'error': f"Unsupported script language: {script_type}"}
+
+ proc = subprocess.run(
+ cmd,
+ capture_output=True,
+ text=True,
+ timeout=timeout,
+ env=env,
+ cwd='/tmp' if os.name != 'nt' else None
+ )
+
+ stdout = proc.stdout[:MAX_OUTPUT_SIZE] if proc.stdout else ''
+ stderr = proc.stderr[:MAX_OUTPUT_SIZE] if proc.stderr else ''
+
+ return {
+ 'success': proc.returncode == 0,
+ 'stdout': stdout,
+ 'stderr': stderr,
+ 'returncode': proc.returncode
+ }
+
+ except subprocess.TimeoutExpired:
+ return {
+ 'success': False,
+ 'error': f"Script timed out after {timeout}s",
+ 'stdout': '',
+ 'stderr': '',
+ 'returncode': -1
+ }
+ except FileNotFoundError:
+ fallback = 'python' if script_type == 'python' else script_type
+ try:
+ proc = subprocess.run(
+ [fallback, '-c', content] if script_type == 'python' else content,
+ capture_output=True, text=True, timeout=timeout,
+ env=env, shell=(script_type == 'bash'),
+ cwd='/tmp' if os.name != 'nt' else None
+ )
+ stdout = proc.stdout[:MAX_OUTPUT_SIZE] if proc.stdout else ''
+ stderr = proc.stderr[:MAX_OUTPUT_SIZE] if proc.stderr else ''
+ return {
+ 'success': proc.returncode == 0,
+ 'stdout': stdout, 'stderr': stderr,
+ 'returncode': proc.returncode
+ }
+ except Exception as e:
+ return {'success': False, 'error': str(e)}
+ except Exception as e:
+ return {'success': False, 'error': str(e)}
+
+ # ------------------------------------------------------------------
+ # Logging
+ # ------------------------------------------------------------------
+
+ @staticmethod
+ def _log(execution_id: int, message: str, level: str = 'INFO', node_id: str = None):
"""Add a log entry for an execution."""
- log = WorkflowLog(
+ log_entry = WorkflowLog(
execution_id=execution_id,
level=level,
message=message,
node_id=node_id
)
- db.session.add(log)
+ db.session.add(log_entry)
db.session.commit()
- # Also print to stdout for debugging
- print(f"[{level}] Workflow {execution_id}: {message}")
+ logger.info(f"[{level}] Workflow {execution_id}: {message}")
+
+ # Keep backward-compatible alias
+ log = _log
+
+
+# ------------------------------------------------------------------
+# Event Bus for workflow triggers
+# ------------------------------------------------------------------
+
+class WorkflowEventBus:
+ """
+ Simple in-process event bus for triggering workflows on system events.
+
+ Events are emitted by services (monitoring, health checks, git deploy)
+ and matched against workflows with trigger_type='event'.
+ """
+
+ _listeners_lock = threading.Lock()
+
+ @staticmethod
+ def emit(event_type: str, data: Dict[str, Any] = None):
+ """
+ Emit an event that may trigger workflows.
+
+ Args:
+ event_type: One of health_check_failed, high_cpu, high_memory,
+ git_push, app_stopped, or any custom string.
+ data: Event payload passed as workflow context.
+ """
+ from flask import current_app
+ try:
+ app = current_app._get_current_object()
+ except RuntimeError:
+ logger.warning(f"WorkflowEventBus.emit called outside app context for {event_type}")
+ return
+
+ threading.Thread(
+ target=WorkflowEventBus._process_event,
+ args=(app, event_type, data or {}),
+ daemon=True,
+ name=f'wf-event-{event_type}'
+ ).start()
+
+ @staticmethod
+ def _process_event(app, event_type: str, data: Dict):
+ """Find and execute workflows subscribed to this event type."""
+ with app.app_context():
+ try:
+ workflows = Workflow.query.filter_by(
+ is_active=True,
+ trigger_type='event'
+ ).all()
+
+ for workflow in workflows:
+ try:
+ config = json.loads(workflow.trigger_config) if workflow.trigger_config else {}
+ subscribed_event = config.get('eventType', '')
+
+ if subscribed_event != event_type:
+ continue
+
+ # Cooldown: don't re-trigger within 60 seconds
+ if workflow.last_run_at:
+ elapsed = (datetime.utcnow() - workflow.last_run_at).total_seconds()
+ if elapsed < 60:
+ continue
+
+ logger.info(f"Event '{event_type}' triggering workflow: {workflow.name}")
+ context = {
+ 'event_type': event_type,
+ 'event_data': data,
+ 'triggered_at': datetime.utcnow().isoformat()
+ }
+ WorkflowEngine.execute_workflow(
+ workflow_id=workflow.id,
+ trigger_type='event',
+ context=context
+ )
+ except Exception as e:
+ logger.error(f"Event trigger failed for workflow {workflow.id}: {e}")
+
+ except Exception as e:
+ logger.error(f"WorkflowEventBus._process_event error: {e}")
diff --git a/backend/migrations/versions/003_workflows_automation.py b/backend/migrations/versions/003_workflows_automation.py
new file mode 100644
index 00000000..6f59b3d9
--- /dev/null
+++ b/backend/migrations/versions/003_workflows_automation.py
@@ -0,0 +1,90 @@
+"""Add workflows and automation tables.
+
+Revision ID: 003_workflows_automation
+Revises: 002_permissions_invitations
+Create Date: 2026-03-23
+"""
+from alembic import op
+import sqlalchemy as sa
+from datetime import datetime
+
+revision = '003_workflows_automation'
+down_revision = '002_permissions_invitations'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+ conn = op.get_bind()
+ inspector = sa.inspect(conn)
+ existing_tables = inspector.get_table_names()
+
+ # Create workflows table if missing (old version might have it but without automation fields)
+ if 'workflows' not in existing_tables:
+ op.create_table('workflows',
+ sa.Column('id', sa.Integer(), primary_key=True),
+ sa.Column('name', sa.String(100), nullable=False),
+ sa.Column('description', sa.Text(), nullable=True),
+ sa.Column('nodes', sa.Text(), nullable=True),
+ sa.Column('edges', sa.Text(), nullable=True),
+ sa.Column('viewport', sa.Text(), nullable=True),
+ sa.Column('is_active', sa.Boolean(), default=False),
+ sa.Column('trigger_type', sa.String(50), default='manual'),
+ sa.Column('trigger_config', sa.Text(), nullable=True),
+ sa.Column('last_run_at', sa.DateTime(), nullable=True),
+ sa.Column('last_status', sa.String(20), nullable=True),
+ sa.Column('created_at', sa.DateTime(), default=datetime.utcnow),
+ sa.Column('updated_at', sa.DateTime(), default=datetime.utcnow),
+ sa.Column('user_id', sa.Integer(), sa.ForeignKey('users.id'), nullable=False)
+ )
+ else:
+ # Add automation columns to existing workflows table if they don't exist
+ existing_cols = {c['name'] for c in inspector.get_columns('workflows')}
+ with op.batch_alter_table('workflows') as batch_op:
+ if 'is_active' not in existing_cols:
+ batch_op.add_column(sa.Column('is_active', sa.Boolean(), nullable=True, server_default='0'))
+ if 'trigger_type' not in existing_cols:
+ batch_op.add_column(sa.Column('trigger_type', sa.String(50), nullable=True, server_default='manual'))
+ if 'trigger_config' not in existing_cols:
+ batch_op.add_column(sa.Column('trigger_config', sa.Text(), nullable=True))
+ if 'last_run_at' not in existing_cols:
+ batch_op.add_column(sa.Column('last_run_at', sa.DateTime(), nullable=True))
+ if 'last_status' not in existing_cols:
+ batch_op.add_column(sa.Column('last_status', sa.String(20), nullable=True))
+
+ # Create workflow_executions table
+ if 'workflow_executions' not in existing_tables:
+ op.create_table('workflow_executions',
+ sa.Column('id', sa.Integer(), primary_key=True),
+ sa.Column('workflow_id', sa.Integer(), sa.ForeignKey('workflows.id'), nullable=False),
+ sa.Column('status', sa.String(20), default='running'),
+ sa.Column('trigger_type', sa.String(50), nullable=True),
+ sa.Column('context', sa.Text(), nullable=True),
+ sa.Column('results', sa.Text(), nullable=True),
+ sa.Column('started_at', sa.DateTime(), default=datetime.utcnow),
+ sa.Column('completed_at', sa.DateTime(), nullable=True)
+ )
+
+ # Create workflow_logs table
+ if 'workflow_logs' not in existing_tables:
+ op.create_table('workflow_logs',
+ sa.Column('id', sa.Integer(), primary_key=True),
+ sa.Column('execution_id', sa.Integer(), sa.ForeignKey('workflow_executions.id'), nullable=False),
+ sa.Column('level', sa.String(10), default='INFO'),
+ sa.Column('message', sa.Text(), nullable=False),
+ sa.Column('node_id', sa.String(100), nullable=True),
+ sa.Column('timestamp', sa.DateTime(), default=datetime.utcnow)
+ )
+
+
+def downgrade():
+ op.drop_table('workflow_logs')
+ op.drop_table('workflow_executions')
+
+ # We don't drop 'workflows' as it existed before, but we can drop the new columns
+ with op.batch_alter_table('workflows') as batch_op:
+ batch_op.drop_column('last_status')
+ batch_op.drop_column('last_run_at')
+ batch_op.drop_column('trigger_config')
+ batch_op.drop_column('trigger_type')
+ batch_op.drop_column('is_active')
diff --git a/dev.sh b/dev.sh
index c157ffc6..8028c0cf 100644
--- a/dev.sh
+++ b/dev.sh
@@ -17,7 +17,7 @@ FRONTEND_DIR="$PROJECT_ROOT/frontend"
CYAN='\033[0;36m'
GREEN='\033[0;32m'
RED='\033[0;31m'
-YELLOW='\033[0;33m'
+YELLOW='\033[1;33m'
DIM='\033[2m'
NC='\033[0m'
diff --git a/frontend/src/components/workflow/nodes/TriggerNode.jsx b/frontend/src/components/workflow/nodes/TriggerNode.jsx
index 5247ca64..462ddd8b 100644
--- a/frontend/src/components/workflow/nodes/TriggerNode.jsx
+++ b/frontend/src/components/workflow/nodes/TriggerNode.jsx
@@ -3,12 +3,12 @@ import { Handle, Position } from '@xyflow/react';
import { Zap, Clock, Webhook, Activity, Play } from 'lucide-react';
const TriggerNode = ({ data, selected }) => {
- const {
- triggerType = 'manual',
+ const {
+ triggerType = 'manual',
label = 'Trigger',
isActive = false,
- lastRunAt = null,
- lastStatus = null
+ lastStatus = null,
+ triggerConfig = {}
} = data;
const getIcon = () => {
@@ -28,6 +28,30 @@ const TriggerNode = ({ data, selected }) => {
return 'border-blue-500 shadow-[0_0_10px_rgba(59,130,246,0.3)]';
};
+ const getSublabel = () => {
+ const typeName = triggerType.charAt(0).toUpperCase() + triggerType.slice(1);
+ const status = isActive ? 'Active' : 'Disabled';
+
+ if (triggerType === 'cron' && triggerConfig.cron) {
+ return `${triggerConfig.cron} \u2022 ${status}`;
+ }
+ if (triggerType === 'webhook' && triggerConfig.webhook_id) {
+ return `...${triggerConfig.webhook_id.slice(-8)} \u2022 ${status}`;
+ }
+ if (triggerType === 'event' && triggerConfig.eventType) {
+ const eventLabels = {
+ health_check_failed: 'Health Fail',
+ high_cpu: 'High CPU',
+ high_memory: 'High Memory',
+ git_push: 'Git Push',
+ app_stopped: 'App Stopped'
+ };
+ return `${eventLabels[triggerConfig.eventType] || triggerConfig.eventType} \u2022 ${status}`;
+ }
+
+ return `${typeName} \u2022 ${status}`;
+ };
+
return (
<div className={`workflow-node node-trigger ${selected ? 'node-selected' : ''} ${getStatusColor()}`}>
<div className="node-icon bg-gray-800">
@@ -35,12 +59,9 @@ const TriggerNode = ({ data, selected }) => {
</div>
<div className="node-content">
<div className="node-label">{label}</div>
- <div className="node-sublabel">
- {triggerType.charAt(0).toUpperCase() + triggerType.slice(1)}
- {isActive ? ' β€’ Active' : ' β€’ Disabled'}
- </div>
+ <div className="node-sublabel">{getSublabel()}</div>
</div>
-
+
<Handle
type="source"
position={Position.Bottom}
diff --git a/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsx b/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsx
index fe7525ed..1c1e3de5 100644
--- a/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsx
+++ b/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsx
@@ -1,5 +1,4 @@
-import React from 'react';
-import { X, GitBranch } from 'lucide-react';
+import { X } from 'lucide-react';
const LogicIfConfigPanel = ({ node, onChange, onClose }) => {
const { data } = node;
@@ -36,19 +35,23 @@ const LogicIfConfigPanel = ({ node, onChange, onClose }) => {
<div className="mt-4 p-3 bg-gray-800 rounded border border-gray-700">
<p className="text-[10px] text-gray-400">
<strong>Available Variables:</strong><br />
- <code>results</code>: Dictionary of previous node outputs<br />
- <code>context</code>: Trigger execution context
+ <code>results</code> β€” dict of previous node outputs, keyed by node ID<br />
+ <code>context</code> β€” trigger execution context<br /><br />
+ <strong>Examples:</strong><br />
+ <code className="text-gray-300">{"results['abc123']['returncode'] == 0"}</code><br />
+ <code className="text-gray-300">{"context.get('event_type') == 'high_cpu'"}</code><br />
+ <code className="text-gray-300">len(results) &gt; 0</code>
</p>
</div>
<div className="mt-4 space-y-2">
<div className="flex items-center gap-2 text-xs">
<div className="w-2 h-2 rounded-full bg-green-500"></div>
- <span className="text-gray-300">True branch connects to "TRUE" handle</span>
+ <span className="text-gray-300">True branch β€” condition evaluates to truthy</span>
</div>
<div className="flex items-center gap-2 text-xs">
<div className="w-2 h-2 rounded-full bg-red-500"></div>
- <span className="text-gray-300">False branch connects to "FALSE" handle</span>
+ <span className="text-gray-300">False branch β€” condition evaluates to falsy or errors</span>
</div>
</div>
</div>
diff --git a/frontend/src/components/workflow/panels/NotificationConfigPanel.jsx b/frontend/src/components/workflow/panels/NotificationConfigPanel.jsx
index 04e7ccfc..b325f8a7 100644
--- a/frontend/src/components/workflow/panels/NotificationConfigPanel.jsx
+++ b/frontend/src/components/workflow/panels/NotificationConfigPanel.jsx
@@ -51,7 +51,7 @@ const NotificationConfigPanel = ({ node, onChange, onClose }) => {
onClick={() => onChange({ ...data, channel: 'system' })}
>
<Bell size={14} />
- <span className="text-xs">System</span>
+ <span className="text-xs">All</span>
</button>
</div>
</div>
@@ -62,14 +62,17 @@ const NotificationConfigPanel = ({ node, onChange, onClose }) => {
className="text-xs bg-gray-900 border-gray-700 rounded p-2 h-48 focus:border-blue-500 focus:outline-none"
value={message}
onChange={(e) => onChange({ ...data, message: e.target.value })}
- placeholder="e.g. Workflow {{workflow_name}} failed on node {{failed_node_id}}"
+ placeholder="e.g. Build finished for {{workflow_name}} β€” exit code: ${build_node.returncode}"
/>
</div>
<div className="mt-4 p-3 bg-gray-800 rounded border border-gray-700">
<p className="text-[10px] text-gray-400">
- <strong>Available Placeholders:</strong><br />
- <code>{{workflow_name}}</code>, <code>{{execution_id}}</code>, <code>{{started_at}}</code>, <code>{{node_id.output}}</code>
+ <strong>Variables:</strong><br />
+ <code>{'{{workflow_name}}'}</code>, <code>{'{{execution_id}}'}</code>, <code>{'{{started_at}}'}</code><br />
+ <code>{'${node_id.stdout}'}</code> β€” output from a specific node<br />
+ <code>{'${node_id.returncode}'}</code> β€” exit code from a script node<br />
+ <code>{'{{context.field}}'}</code> β€” value from trigger context
</p>
</div>
</div>
diff --git a/frontend/src/components/workflow/panels/ScriptConfigPanel.jsx b/frontend/src/components/workflow/panels/ScriptConfigPanel.jsx
index 88111dab..7d1fa9c6 100644
--- a/frontend/src/components/workflow/panels/ScriptConfigPanel.jsx
+++ b/frontend/src/components/workflow/panels/ScriptConfigPanel.jsx
@@ -3,7 +3,14 @@ import { X, Terminal, Code } from 'lucide-react';
const ScriptConfigPanel = ({ node, onChange, onClose }) => {
const { data } = node;
- const { language = 'bash', label = 'Run Script', content = '' } = data;
+ const {
+ language = 'bash',
+ label = 'Run Script',
+ content = '',
+ timeout = 300,
+ retryCount = 0,
+ retryDelay = 5
+ } = data;
return (
<div className="config-panel">
@@ -52,9 +59,45 @@ const ScriptConfigPanel = ({ node, onChange, onClose }) => {
/>
</div>
+ <div className="grid grid-cols-3 gap-2">
+ <div className="form-group">
+ <label>Timeout (s)</label>
+ <input
+ type="number"
+ min="1"
+ max="3600"
+ value={timeout}
+ onChange={(e) => onChange({ ...data, timeout: parseInt(e.target.value) || 300 })}
+ />
+ </div>
+ <div className="form-group">
+ <label>Retries</label>
+ <input
+ type="number"
+ min="0"
+ max="5"
+ value={retryCount}
+ onChange={(e) => onChange({ ...data, retryCount: parseInt(e.target.value) || 0 })}
+ />
+ </div>
+ <div className="form-group">
+ <label>Delay (s)</label>
+ <input
+ type="number"
+ min="1"
+ max="300"
+ value={retryDelay}
+ onChange={(e) => onChange({ ...data, retryDelay: parseInt(e.target.value) || 5 })}
+ />
+ </div>
+ </div>
+
<div className="mt-4 p-3 bg-blue-900/20 rounded border border-blue-500/30">
<p className="text-[10px] text-blue-300">
- <strong>Environment Variables:</strong> You can use <code>$NODE_ID_OUTPUT</code> to access results from previous nodes.
+ <strong>Variables:</strong><br />
+ <code className="text-blue-200">{'${node_id.stdout}'}</code> β€” output from a previous node<br />
+ <code className="text-blue-200">$NODE_ID_OUTPUT</code> β€” same, as env var<br />
+ <code className="text-blue-200">$WORKFLOW_ID</code>, <code className="text-blue-200">$EXECUTION_ID</code> β€” workflow metadata
</p>
</div>
</div>
diff --git a/frontend/src/components/workflow/panels/TriggerConfigPanel.jsx b/frontend/src/components/workflow/panels/TriggerConfigPanel.jsx
index 24d4c06f..9d956231 100644
--- a/frontend/src/components/workflow/panels/TriggerConfigPanel.jsx
+++ b/frontend/src/components/workflow/panels/TriggerConfigPanel.jsx
@@ -1,9 +1,10 @@
-import React from 'react';
-import { X, Play, Clock, Webhook, Activity, Zap } from 'lucide-react';
+import React, { useState } from 'react';
+import { X, Play, Clock, Webhook, Zap, Copy, Check } from 'lucide-react';
const TriggerConfigPanel = ({ node, onChange, onClose }) => {
const { data } = node;
const { triggerType = 'manual', label = 'Trigger', triggerConfig = {} } = data;
+ const [copied, setCopied] = useState(false);
const handleTypeChange = (type) => {
onChange({
@@ -23,6 +24,18 @@ const TriggerConfigPanel = ({ node, onChange, onClose }) => {
});
};
+ const webhookUrl = triggerConfig.webhook_id
+ ? `${window.location.origin}/api/v1/workflows/hooks/${triggerConfig.webhook_id}`
+ : null;
+
+ const copyWebhookUrl = () => {
+ if (webhookUrl) {
+ navigator.clipboard.writeText(webhookUrl);
+ setCopied(true);
+ setTimeout(() => setCopied(false), 2000);
+ }
+ };
+
return (
<div className="config-panel">
<div className="panel-header">
@@ -92,17 +105,38 @@ const TriggerConfigPanel = ({ node, onChange, onClose }) => {
{triggerType === 'webhook' && (
<div className="form-group animate-in fade-in slide-in-from-top-1">
<label>Webhook URL</label>
- <div className="flex gap-2">
- <input
- type="text"
- value={triggerConfig.webhookPath || `/hooks/${node.id}`}
- readOnly
- className="bg-gray-900 text-gray-400 cursor-not-allowed"
- />
+ {webhookUrl ? (
+ <>
+ <div className="flex gap-2">
+ <input
+ type="text"
+ value={webhookUrl}
+ readOnly
+ className="bg-gray-900 text-gray-300 cursor-default font-mono text-xs flex-1"
+ />
+ <button
+ onClick={copyWebhookUrl}
+ className="px-3 py-1 rounded border border-gray-600 bg-gray-800 hover:bg-gray-700 transition-colors"
+ title="Copy URL"
+ >
+ {copied ? <Check size={14} className="text-green-400" /> : <Copy size={14} className="text-gray-400" />}
+ </button>
+ </div>
+ <p className="text-[10px] text-gray-500 mt-1">
+ Send POST requests to this URL to trigger the workflow.
+ </p>
+ </>
+ ) : (
+ <p className="text-[10px] text-yellow-400 mt-1">
+ Save the workflow to generate the webhook URL.
+ </p>
+ )}
+ <div className="mt-3 p-3 bg-gray-800 rounded border border-gray-700">
+ <p className="text-[10px] text-gray-400">
+ <strong>Request body</strong> is available in your workflow as <code>context.body</code>.
+ Query parameters are in <code>context.query</code>.
+ </p>
</div>
- <p className="text-[10px] text-gray-500 mt-1">
- Send POST requests to this endpoint to trigger the workflow.
- </p>
</div>
)}
@@ -114,11 +148,14 @@ const TriggerConfigPanel = ({ node, onChange, onClose }) => {
onChange={(e) => handleConfigChange('eventType', e.target.value)}
>
<option value="health_check_failed">Health Check Failed</option>
- <option value="high_cpu">High CPU Usage (>80%)</option>
- <option value="high_memory">High Memory Usage (>80%)</option>
+ <option value="high_cpu">High CPU Usage (&gt;80%)</option>
+ <option value="high_memory">High Memory Usage (&gt;80%)</option>
<option value="git_push">Git Push Received</option>
<option value="app_stopped">Application Stopped</option>
</select>
+ <p className="text-[10px] text-gray-500 mt-1">
+ Event data is available as <code>context.event_data</code>.
+ </p>
</div>
)}
</div>
diff --git a/frontend/src/services/api.js b/frontend/src/services/api.js
index 384651b4..bdce5b5b 100644
--- a/frontend/src/services/api.js
+++ b/frontend/src/services/api.js
@@ -2581,6 +2581,13 @@ class ApiService {
return this.request(`/workflows/executions/${executionId}/logs`);
}
+ async validateWorkflow(data) {
+ return this.request('/workflows/validate', {
+ method: 'POST',
+ body: JSON.stringify(data)
+ });
+ }
+
// ========================================
// Servers (Multi-Server Management) endpoints
// ========================================
diff --git a/quick-start.sh b/quick-start.sh
new file mode 100644
index 00000000..230df9d5
--- /dev/null
+++ b/quick-start.sh
@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+#
+# ServerKit - All-in-One Local Quick Start
+#
+# This script automates:
+# 1. System dependency checks
+# 2. Python venv & dependency setup
+# 3. Frontend npm installation
+# 4. Environment configuration (.env)
+# 5. Launching both Backend & Frontend
+#
+
+set -euo pipefail
+
+# Colors
+CYAN='\033[0;36m'
+GREEN='\033[0;32m'
+RED='\033[0;31m'
+YELLOW='\033[1;33m'
+NC='\033[0m'
+
+echo -e "${CYAN}πŸš€ Starting ServerKit Local Setup...${NC}"
+
+# 1. Check for required system tools
+check_tool() {
+ if ! command -v "$1" &>/dev/null; then
+ echo -e "${RED}βœ— $1 is not installed.${NC}"
+ echo -e "${YELLOW}Please run: sudo apt update && sudo apt install -y $2${NC}"
+ exit 1
+ fi
+}
+
+check_tool "python3" "python3 python3-venv python3-pip"
+check_tool "node" "nodejs"
+check_tool "npm" "npm"
+
+# 2. Setup Backend
+echo -e "\n${CYAN}πŸ“¦ Setting up Backend...${NC}"
+if [ ! -d "venv" ]; then
+ echo "Creating virtual environment..."
+ python3 -m venv venv
+fi
+
+source venv/bin/activate
+echo "Installing Python dependencies..."
+pip install -r backend/requirements.txt --quiet
+pip install gunicorn gevent gevent-websocket --quiet
+
+# 3. Setup Frontend
+echo -e "\n${CYAN}πŸ“¦ Setting up Frontend...${NC}"
+cd frontend
+if [ ! -d "node_modules" ]; then
+ echo "Installing Node modules (this may take a minute)..."
+ npm install --silent
+else
+ echo "Node modules already installed."
+fi
+cd ..
+
+# 4. Configuration
+echo -e "\n${CYAN}βš™οΈ Configuring Environment...${NC}"
+if [ ! -f ".env" ]; then
+ echo "Creating .env from .env.example..."
+ cp .env.example .env
+ # Generate random secrets for dev
+ sed -i "s/SECRET_KEY=.*/SECRET_KEY=$(openssl rand -hex 16)/" .env
+ sed -i "s/JWT_SECRET_KEY=.*/JWT_SECRET_KEY=$(openssl rand -hex 16)/" .env
+else
+ echo ".env file already exists."
+fi
+
+# 5. Launch
+echo -e "\n${GREEN}βœ… Setup Complete!${NC}"
+echo -e "Starting ServerKit..."
+echo -e " - ${CYAN}Frontend:${NC} http://localhost:5173"
+echo -e " - ${CYAN}Backend:${NC} http://localhost:5000"
+echo -e "\n${YELLOW}Press Ctrl+C to stop both servers.${NC}\n"
+
+# Run dev.sh to handle the dual-process execution
+chmod +x dev.sh
+./dev.sh