ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
Add workflow automation engine and triggers
Juan Denis13bd84eda1c8
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.ymlnew file mode 100644index 00000000..469ce162--- /dev/null+++ b/.github/ISSUE_TEMPLATE/bug_report.yml@@ -0,0 +1,58 @@+name: π Bug Report+description: Report a bug to help us improve ServerKit+labels: ["bug"]+body:+ - type: markdown+ attributes:+ value: |+ Thanks for taking the time to fill out this bug report!+ - type: textarea+ id: description+ attributes:+ label: Describe the bug+ description: A clear and concise description of what the bug is.+ placeholder: Bug description+ validations:+ required: true+ - type: textarea+ id: reproduction+ attributes:+ label: Steps to reproduce+ description: How can we reproduce this issue?+ placeholder: |+ 1. Go to '...'+ 2. Click on '....'+ 3. Scroll down to '....'+ 4. See error+ validations:+ required: true+ - type: textarea+ id: expected-behavior+ attributes:+ label: Expected behavior+ description: A clear and concise description of what you expected to happen.+ validations:+ required: true+ - type: input+ id: environment+ attributes:+ label: Environment+ description: OS version, browser, ServerKit version, etc.+ placeholder: e.g. Ubuntu 22.04, Chrome 120, ServerKit v0.8.0+ validations:+ required: true+ - type: textarea+ id: logs+ attributes:+ label: Logs+ description: Please provide any relevant logs from the backend or browser console.+ render: shell+ - type: checkboxes+ id: checks+ attributes:+ label: Checks+ options:+ - label: I have searched the [existing issues](https://github.com/jhd3197/ServerKit/issues).+ required: true+ - label: I am using the latest version of ServerKit.+ required: truediff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.ymlnew file mode 100644index 00000000..730cf24a--- /dev/null+++ b/.github/ISSUE_TEMPLATE/feature_request.yml@@ -0,0 +1,28 @@+name: π Feature Request+description: Suggest an idea for ServerKit+labels: ["enhancement"]+body:+ - type: textarea+ id: feature-description+ attributes:+ label: Is your feature request related to a problem?+ description: A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]+ validations:+ required: true+ - type: textarea+ id: solution+ attributes:+ label: Describe the solution you'd like+ description: A clear and concise description of what you want to happen.+ validations:+ required: true+ - type: textarea+ id: alternatives+ attributes:+ label: Describe alternatives you've considered+ description: A clear and concise description of any alternative solutions or features you've considered.+ - type: textarea+ id: additional-context+ attributes:+ label: Additional context+ description: Add any other context or screenshots about the feature request here.diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.mdnew file mode 100644index 00000000..e800fa5d--- /dev/null+++ b/.github/PULL_REQUEST_TEMPLATE.md@@ -0,0 +1,24 @@+## Description+<!-- Provide a brief summary of the changes and the motivation behind them. -->++## Related Issues+<!-- Link any related issues using keywords like Fixes #123 or Closes #456. -->++## Type of Change+- [ ] π Bug fix (non-breaking change which fixes an issue)+- [ ] π New feature (non-breaking change which adds functionality)+- [ ] β οΈ Breaking change (fix or feature that would cause existing functionality to not work as expected)+- [ ] π Documentation update+- [ ] π¨ UI/UX improvement++## How Has This Been Tested?+<!-- Describe the tests that you ran to verify your changes. -->++## Checklist+- [ ] My code follows the style guidelines of this project+- [ ] I have performed a self-review of my own code+- [ ] I have commented my code, particularly in hard-to-understand areas+- [ ] I have made corresponding changes to the documentation+- [ ] My changes generate no new warnings+- [ ] I have added tests that prove my fix is effective or that my feature works+- [ ] New and existing unit tests pass locally with my changesdiff --git a/.gitignore b/.gitignoreindex 26e52b30..ad077178 100644--- a/.gitignore+++ b/.gitignore@@ -1,63 +1,95 @@-# Dependencies+# --- ServerKit Consolidated .gitignore ---++# Node / Frontend/frontend/node_modules/+/frontend/dist/+/frontend/npm-debug.log*+/frontend/yarn-debug.log*+/frontend/yarn-error.log*+/frontend/.eslintcache+/frontend/.stylelintcache+/frontend/coverage/+/frontend/.next/+/frontend/out/+/frontend/build/++# Python / Backend/backend/venv/+/backend/instance//backend/__pycache__/-**/__pycache__/-*.pyc+/backend/.pytest_cache/+/backend/.coverage+/backend/htmlcov/+/backend/.mypy_cache/+/backend/.tox/+/backend/*.pyc+/backend/*.pyo+/backend/*.pyd+/backend/.env+/backend/.venv+/backend/pip-log.txt+/backend/pip-delete-this-directory.txt+/backend/dev-data/+/venv/-# Build outputs-/frontend/dist/-/agent/dist/+# Go / Agent/agent/serverkit-agent/agent/serverkit-agent.exe-*.exe-*.msi-*.deb-*.rpm--# IDE and editors-.idea/-.vscode/-*.swp-*.swo-*~+/agent/dist/+/agent/vendor/+/agent/*.test+/agent/*.out+/agent/bin/-# Environment and secrets+# Environment / Secrets.env.env.local-.env.*.local+.env.development.local+.env.test.local+.env.production.local*.key*.pem*.crt*.csr+/backend/.env+/frontend/.env.local-# OS files+# Docker+.docker-compose.untracked.yml+docker-compose.override.yml++# OS / IDE.DS_Store+.DS_Store?+._*+.Spotlight-V100+.Trashes+ehthumbs.dbThumbs.db-nul--# Temporary files-*.tmp-*.temp-*.log-coverage.out-coverage.html+.idea/+.vscode/+*.swp+*.swo+*~+*.sublime-project+*.sublime-workspace-# Project specific+# Project Specific.planning_old/.mcp.jsonnew_templates//.marketing/--# Test artifacts-*.test-*.out-/template-icons--# Dev data directory (local development path overrides)-backend/dev-data/-/.vscode-/.vscode1-/.pr-/.reviews+/template-icons/+/.pr/+/.reviews//.claude/settings.local.json+/serverkit.db+/backend/instance/serverkit.db++# Logs & Temp+*.log+*.tmp+*.temp+/logs/+/tmp/+nuldiff --git a/ROADMAP.md b/ROADMAP.mdindex ac278abe..55fd4786 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -304,18 +304,19 @@ The phases below are ordered by priority. Higher phases ship first.----## Phase 23: Advanced Workflow & Automation Engine (NEW)+## Phase 23: Workflow & Automation Engine β Core (Completed)**Priority: Critical**Moving beyond static design to dynamic, event-driven automation. This turns ServerKit into a powerful automation hub.-- [x] **Event Triggers:** Run workflows on Git push, health check failure, high CPU usage, or Webhook receipt.-- [x] **Cron Integration:** Schedule workflows to run on recurring intervals (e.g., "Every Sunday at 2 AM, backup all DBs and rotate logs").-- [x] **Cross-Server Actions:** "When Server A's database is backed up, sync it to Server B and notify Discord."-- [x] **Logic Nodes:** If/Else conditions, loops, and custom Python/Shell script execution nodes.-- [x] **State Machine:** Track workflow execution history, retries on failure, and execution logs.-- [x] **Global Variables:** Pass data between workflow steps (e.g., take the output of a build step and pass it to a deployment step).+- [x] **Visual Workflow Builder:** Node-based canvas with drag-and-drop nodes, connection validation, and config panels+- [x] **Cron Integration:** Schedule workflows to run on recurring intervals (e.g., "Every Sunday at 2 AM, backup all DBs and rotate logs")+- [x] **Manual Execution:** Trigger workflows on demand with optional context data+- [x] **Execution History:** Track workflow execution status, per-node results, and timestamped logs+- [x] **Script Nodes:** Custom Shell script execution nodes with output capture+- [x] **Notification Nodes:** Send alerts via configured notification channels+- [x] **One-Click Stack Deployment:** Deploy full infrastructure (databases, apps, domains) from a workflow diagram---@@ -332,7 +333,26 @@ Let users personalize what they see. Not everyone runs email servers or manages----## Phase 25: Agent Fleet Management (Planned)+## Phase 25: Workflow Engine β Triggers & Completion (Completed)++**Priority: High**++Complete the workflow engine with proper execution logic, missing triggers, and production-grade reliability.++- [x] **DAG Execution:** Full directed acyclic graph traversal with parallel branch support (replace current linear BFS)+- [x] **Logic Node Evaluation:** If/Else condition evaluation with true/false branching+- [x] **Variable Interpolation:** Pass data between steps using `${node_id.field}` and `{{placeholder}}` syntax in node configs+- [x] **Webhook Triggers:** Register `/hooks/<webhook_id>` endpoint to fire workflows on incoming HTTP requests+- [x] **Event Triggers:** Run workflows on system events (health check failure, high CPU/memory, git push, app stopped)+- [x] **Notification Templating:** Message placeholder substitution (`${node_id.stdout}`, `{{workflow_name}}`) in notification nodes+- [x] **Execution Timeouts:** Configurable timeout per node (1β3600s) to prevent hung workflows+- [x] **Retry on Failure:** Configurable retry count (0β5) and delay per node+- [x] **Circular Dependency Detection:** Kahn's algorithm validates graph on save and before execution+- [x] **Script Sandboxing:** Timeout enforcement, output size limits, explicit `bash -c`/`python3 -c` execution++---++## Phase 26: Agent Fleet Management (Planned)**Priority: High**@@ -352,7 +372,7 @@ Level up agent management from "connect and monitor" to full fleet control.----## Phase 26: Cross-Server Monitoring Dashboard (Planned)+## Phase 27: Cross-Server Monitoring Dashboard (Planned)**Priority: High**@@ -371,7 +391,7 @@ Fleet-wide visibility β see everything at a glance and catch problems early.----## Phase 27: Agent Plugin System (Planned)+## Phase 28: Agent Plugin System (Planned)**Priority: High**@@ -393,7 +413,7 @@ Make the agent extensible β let users add custom capabilities without modifyin----## Phase 28: Server Templates & Config Sync (Planned)+## Phase 29: Server Templates & Config Sync (Planned)**Priority: Medium**@@ -412,7 +432,7 @@ Define what a server should look like, apply it, and detect when it drifts.----## Phase 29: Multi-Tenancy & Workspaces (Planned)+## Phase 30: Multi-Tenancy & Workspaces (Planned)**Priority: Medium**@@ -431,7 +451,7 @@ Isolate servers by team, client, or project. Essential for agencies, MSPs, and l----## Phase 30: Advanced SSL Features (Planned)+## Phase 31: Advanced SSL Features (Planned)**Priority: Medium**@@ -445,7 +465,7 @@ Isolate servers by team, client, or project. Essential for agencies, MSPs, and l----## Phase 31: DNS Zone Management (Planned)+## Phase 32: DNS Zone Management (Planned)**Priority: Medium**@@ -462,7 +482,7 @@ Full DNS record management with provider API integration.----## Phase 32: Nginx Advanced Configuration (Planned)+## Phase 33: Nginx Advanced Configuration (Planned)**Priority: Medium**@@ -480,7 +500,7 @@ Go beyond basic virtual hosts β full reverse proxy and performance configurati----## Phase 33: Status Page & Health Checks (Planned)+## Phase 34: Status Page & Health Checks (Planned)**Priority: Medium**@@ -499,7 +519,7 @@ Public-facing status page and automated health monitoring.----## Phase 34: Server Provisioning APIs (Planned)+## Phase 35: Server Provisioning APIs (Planned)**Priority: Medium**@@ -518,7 +538,7 @@ Spin up and manage cloud servers directly from the panel.----## Phase 35: Performance Optimization (Planned)+## Phase 36: Performance Optimization (Planned)**Priority: Low**@@ -531,7 +551,7 @@ Spin up and manage cloud servers directly from the panel.----## Phase 36: Mobile App (Future)+## Phase 37: Mobile App (Future)**Priority: Low β v3.0+**@@ -543,7 +563,7 @@ Spin up and manage cloud servers directly from the panel.----## Phase 37: Marketplace & Extensions (Future)+## Phase 38: Marketplace & Extensions (Future)**Priority: Low β v3.0+**@@ -566,7 +586,7 @@ Spin up and manage cloud servers directly from the panel.| v1.3.0 | Email server, API enhancements | Completed || v1.4.0 | Team & permissions, SSO & OAuth login | Completed || v1.5.0 | New UI, Visual Designer, Services Page | Current |-| v1.6.0 | Advanced Automation Engine, fleet management | Planned |+| v1.6.0 | Workflow triggers & completion, fleet management | Planned || v1.7.0 | Cross-server monitoring, agent plugin system | Planned || v1.8.0 | Server templates, multi-tenancy | Planned || v1.9.0 | Advanced SSL, DNS management, Nginx config | Planned |diff --git a/VERSION b/VERSIONindex e05cb332..88c5fb89 100644--- a/VERSION+++ b/VERSION@@ -1 +1 @@-1.3.8+1.4.0diff --git a/backend/app/api/workflows.py b/backend/app/api/workflows.pyindex 6049430a..3ac8c2af 100644--- a/backend/app/api/workflows.py+++ b/backend/app/api/workflows.py@@ -1,4 +1,6 @@import json+import secrets+from datetime import datetimefrom flask import Blueprint, request, jsonifyfrom flask_jwt_extended import jwt_required, get_jwt_identityfrom app import db@@ -7,6 +9,18 @@ from app.models import Workflow, User, WorkflowExecution, WorkflowLogworkflows_bp = Blueprint('workflows', __name__)+def _validate_workflow_graph(data):+ """Validate workflow graph for cycles if nodes/edges are present."""+ nodes = data.get('nodes', [])+ edges = data.get('edges', [])+ if nodes and edges:+ from app.services.workflow_engine import WorkflowEngine+ err = WorkflowEngine.validate_graph(nodes, edges)+ if err:+ return err+ return None++@workflows_bp.route('', methods=['GET'])@jwt_required()def get_workflows():@@ -55,6 +69,16 @@ def create_workflow():if not name:return jsonify({'error': 'Name is required'}), 400+ # Validate graph for cycles+ cycle_err = _validate_workflow_graph(data)+ if cycle_err:+ return jsonify({'error': cycle_err}), 400++ # Auto-generate webhook_id if trigger is webhook+ trigger_config = data.get('trigger_config', {})+ if data.get('trigger_type') == 'webhook' and not trigger_config.get('webhook_id'):+ trigger_config['webhook_id'] = secrets.token_urlsafe(24)+workflow = Workflow(name=name,description=data.get('description', ''),@@ -63,7 +87,7 @@ def create_workflow():viewport=json.dumps(data.get('viewport')) if data.get('viewport') else None,is_active=data.get('is_active', False),trigger_type=data.get('trigger_type', 'manual'),- trigger_config=json.dumps(data.get('trigger_config', {})),+ trigger_config=json.dumps(trigger_config),user_id=current_user_id)@@ -94,6 +118,11 @@ def update_workflow(workflow_id):if not data:return jsonify({'error': 'No data provided'}), 400+ # Validate graph for cycles+ cycle_err = _validate_workflow_graph(data)+ if cycle_err:+ return jsonify({'error': cycle_err}), 400+if 'name' in data:workflow.name = data['name']if 'description' in data:@@ -104,14 +133,19 @@ def update_workflow(workflow_id):workflow.edges = json.dumps(data['edges'])if 'viewport' in data:workflow.viewport = json.dumps(data['viewport']) if data['viewport'] else None-+# Automation fieldsif 'is_active' in data:workflow.is_active = data['is_active']if 'trigger_type' in data:workflow.trigger_type = data['trigger_type']if 'trigger_config' in data:- workflow.trigger_config = json.dumps(data['trigger_config'])+ trigger_config = data['trigger_config']+ # Auto-generate webhook_id if switching to webhook trigger+ if data.get('trigger_type') == 'webhook' and not trigger_config.get('webhook_id'):+ existing = json.loads(workflow.trigger_config) if workflow.trigger_config else {}+ trigger_config['webhook_id'] = existing.get('webhook_id') or secrets.token_urlsafe(24)+ workflow.trigger_config = json.dumps(trigger_config)db.session.commit()@@ -254,3 +288,86 @@ def deploy_workflow(workflow_id):else:# Partial success or errors - return 200 with error detailsreturn jsonify(result), 200+++@workflows_bp.route('/hooks/<webhook_id>', methods=['POST'])+def webhook_trigger(webhook_id):+ """+ Public webhook endpoint to trigger a workflow.+ No JWT required β the webhook_id acts as the authentication token.+ """+ from app.services.workflow_engine import WorkflowEngine, CycleDetectedError++ # Find the workflow with this webhook_id+ workflows = Workflow.query.filter_by(+ is_active=True,+ trigger_type='webhook'+ ).all()++ target_workflow = None+ for wf in workflows:+ try:+ config = json.loads(wf.trigger_config) if wf.trigger_config else {}+ if config.get('webhook_id') == webhook_id:+ target_workflow = wf+ break+ except (json.JSONDecodeError, TypeError):+ continue++ if not target_workflow:+ return jsonify({'error': 'Webhook not found'}), 404++ # Build context from the incoming request+ context = {+ 'webhook_id': webhook_id,+ 'method': request.method,+ 'headers': dict(request.headers),+ 'triggered_at': datetime.utcnow().isoformat()+ }++ # Include request body+ if request.is_json:+ context['body'] = request.get_json(silent=True) or {}+ else:+ body = request.get_data(as_text=True)+ context['body'] = body[:8192] if body else ''++ # Include query parameters+ if request.args:+ context['query'] = dict(request.args)++ try:+ execution_id = WorkflowEngine.execute_workflow(+ workflow_id=target_workflow.id,+ trigger_type='webhook',+ context=context+ )+ return jsonify({+ 'message': 'Workflow triggered',+ 'execution_id': execution_id,+ 'workflow': target_workflow.name+ }), 200+ except CycleDetectedError as e:+ return jsonify({'error': str(e)}), 400+ except Exception as e:+ return jsonify({'error': f'Execution failed: {str(e)}'}), 500+++@workflows_bp.route('/validate', methods=['POST'])+@jwt_required()+def validate_workflow():+ """Validate a workflow graph for cycles and other issues."""+ from app.services.workflow_engine import WorkflowEngine++ data = request.get_json()+ if not data:+ return jsonify({'error': 'No data provided'}), 400++ nodes = data.get('nodes', [])+ edges = data.get('edges', [])++ err = WorkflowEngine.validate_graph(nodes, edges)+ if err:+ return jsonify({'valid': False, 'error': err}), 200++ return jsonify({'valid': True}), 200diff --git a/backend/app/services/docker_service.py b/backend/app/services/docker_service.pyindex b478db0c..234c0492 100644--- a/backend/app/services/docker_service.py+++ b/backend/app/services/docker_service.py@@ -217,6 +217,13 @@ class DockerService:capture_output=True, text=True)if result.returncode == 0:+ try:+ from app.services.workflow_engine import WorkflowEventBus+ WorkflowEventBus.emit('app_stopped', {+ 'container_id': container_id+ })+ except Exception:+ passreturn {'success': True}return {'success': False, 'error': result.stderr}except Exception as e:diff --git a/backend/app/services/environment_health_service.py b/backend/app/services/environment_health_service.pyindex 6398c403..8fcdc6d4 100644--- a/backend/app/services/environment_health_service.py+++ b/backend/app/services/environment_health_service.py@@ -90,6 +90,18 @@ class EnvironmentHealthService:site.last_health_check = datetime.utcnow()db.session.commit()+ # Emit event for workflow triggers on unhealthy status+ if overall in ('unhealthy', 'degraded'):+ try:+ from app.services.workflow_engine import WorkflowEventBus+ WorkflowEventBus.emit('health_check_failed', {+ 'site_id': site_id,+ 'overall_status': overall,+ 'checks': checks+ })+ except Exception:+ pass+return {'success': True,'site_id': site_id,diff --git a/backend/app/services/git_service.py b/backend/app/services/git_service.pyindex 01dc809c..03491df5 100644--- a/backend/app/services/git_service.py+++ b/backend/app/services/git_service.py@@ -434,6 +434,17 @@ class GitService:'message': f'Ignoring push to {ref}, configured branch is {branch}'}+ # Emit event for workflow triggers+ try:+ from app.services.workflow_engine import WorkflowEventBus+ WorkflowEventBus.emit('git_push', {+ 'app_id': app_id,+ 'branch': branch,+ 'ref': ref+ })+ except Exception:+ pass+# Trigger deploymentreturn cls.deploy(app_id)diff --git a/backend/app/services/monitoring_service.py b/backend/app/services/monitoring_service.pyindex 57fd9a18..44beae86 100644--- a/backend/app/services/monitoring_service.py+++ b/backend/app/services/monitoring_service.py@@ -280,6 +280,25 @@ class MonitoringService:# Send to all configured notification channels (Discord, Slack, Telegram, etc.)NotificationService.send_all(alerts_to_send)+ # Emit events for workflow triggers+ try:+ from app.services.workflow_engine import WorkflowEventBus+ for alert in alerts_to_send:+ if alert['type'] == 'cpu':+ WorkflowEventBus.emit('high_cpu', {+ 'percent': alert.get('value'),+ 'threshold': alert.get('threshold'),+ 'severity': alert.get('severity')+ })+ elif alert['type'] == 'memory':+ WorkflowEventBus.emit('high_memory', {+ 'percent': alert.get('value'),+ 'threshold': alert.get('threshold'),+ 'severity': alert.get('severity')+ })+ except Exception:+ pass+@classmethoddef log_alert(cls, alerts: List[Dict]) -> None:"""Log alerts to file."""diff --git a/backend/app/services/workflow_engine.py b/backend/app/services/workflow_engine.pyindex c351151b..f5d75647 100644--- a/backend/app/services/workflow_engine.py+++ b/backend/app/services/workflow_engine.py@@ -1,14 +1,21 @@"""Advanced Workflow & Automation Engine-Executes event-driven workflows with logic nodes, triggers, and cross-server actions.+Executes event-driven workflows with DAG-based execution, logic branching,+variable interpolation, timeouts, retries, and script sandboxing."""import jsonimport logging+import os+import re+import signal+import subprocess+import threadingimport traceback+from collections import dequefrom datetime import datetime-from typing import Dict, List, Any, Optional+from typing import Dict, List, Any, Optional, Set, Tuplefrom app import dbfrom app.models import Workflow, WorkflowExecution, WorkflowLog, User@@ -17,23 +24,100 @@ from app.services.docker_service import DockerServicefrom app.services.database_service import DatabaseServicefrom app.services.notification_service import NotificationService+logger = logging.getLogger(__name__)++# Defaults for node execution+DEFAULT_TIMEOUT = 300 # 5 minutes+MAX_TIMEOUT = 3600 # 1 hour+DEFAULT_RETRY_COUNT = 0+MAX_RETRY_COUNT = 5+DEFAULT_RETRY_DELAY = 5 # seconds+MAX_OUTPUT_SIZE = 1024 * 512 # 512 KB+++class CycleDetectedError(Exception):+ """Raised when a cycle is detected in the workflow graph."""+ pass+++class NodeTimeoutError(Exception):+ """Raised when a node exceeds its timeout."""+ pass+class WorkflowEngine:- """Engine for executing advanced workflows."""+ """Engine for executing advanced workflows with DAG support."""++ # ------------------------------------------------------------------+ # Public API+ # ------------------------------------------------------------------@staticmethod- def execute_workflow(workflow_id: int, trigger_type: str = 'manual', context: Dict[str, Any] = None) -> int:+ def validate_graph(nodes: List[Dict], edges: List[Dict]) -> Optional[str]:+ """+ Validate a workflow graph for cycles.++ Returns None if valid, or an error message string if a cycle is found.+ """+ adj: Dict[str, List[str]] = {}+ node_ids = {n['id'] for n in nodes}++ for edge in edges:+ src = edge['source']+ if src not in adj:+ adj[src] = []+ adj[src].append(edge['target'])++ # Kahn's algorithm for cycle detection+ in_degree = {nid: 0 for nid in node_ids}+ for src, targets in adj.items():+ for t in targets:+ if t in in_degree:+ in_degree[t] += 1++ queue = deque(nid for nid, deg in in_degree.items() if deg == 0)+ visited_count = 0++ while queue:+ nid = queue.popleft()+ visited_count += 1+ for neighbor in adj.get(nid, []):+ if neighbor in in_degree:+ in_degree[neighbor] -= 1+ if in_degree[neighbor] == 0:+ queue.append(neighbor)++ if visited_count < len(node_ids):+ # Find nodes involved in the cycle for a helpful message+ cycle_nodes = [nid for nid, deg in in_degree.items() if deg > 0]+ node_labels = {}+ for n in nodes:+ node_labels[n['id']] = n.get('data', {}).get('label', n['id'])+ cycle_labels = [node_labels.get(nid, nid) for nid in cycle_nodes[:5]]+ return f"Cycle detected involving: {', '.join(cycle_labels)}"++ return None++ @staticmethod+ def execute_workflow(workflow_id: int, trigger_type: str = 'manual',+ context: Dict[str, Any] = None) -> int:"""Execute a workflow by ID.- Returns:- ID of the created WorkflowExecution+ Returns the ID of the created WorkflowExecution."""workflow = Workflow.query.get(workflow_id)if not workflow:raise ValueError(f"Workflow {workflow_id} not found")- # Create execution record+ nodes = json.loads(workflow.nodes) if workflow.nodes else []+ edges = json.loads(workflow.edges) if workflow.edges else []++ # Validate graph before executing+ cycle_err = WorkflowEngine.validate_graph(nodes, edges)+ if cycle_err:+ raise CycleDetectedError(cycle_err)+execution = WorkflowExecution(workflow_id=workflow_id,trigger_type=trigger_type,@@ -44,15 +128,13 @@ class WorkflowEngine:db.session.add(execution)db.session.commit()- # Update workflow last runworkflow.last_run_at = execution.started_atdb.session.commit()try:- # Run the execution in the background or synchronously for now- WorkflowEngine._run_execution(execution.id)+ WorkflowEngine._run_execution(execution.id, nodes, edges)except Exception as e:- WorkflowEngine.log(execution.id, f"Engine Error: {str(e)}", level='ERROR')+ WorkflowEngine._log(execution.id, f"Engine Error: {str(e)}", level='ERROR')execution.status = 'failed'execution.completed_at = datetime.utcnow()workflow.last_status = 'failed'@@ -60,169 +142,564 @@ class WorkflowEngine:return execution.id+ # ------------------------------------------------------------------+ # DAG Execution+ # ------------------------------------------------------------------+@staticmethod- def _run_execution(execution_id: int):- """Internal method to run the execution logic."""+ def _run_execution(execution_id: int, nodes: List[Dict], edges: List[Dict]):+ """Run the workflow using topological DAG execution with branch support."""execution = WorkflowExecution.query.get(execution_id)workflow = execution.workflow-- WorkflowEngine.log(execution_id, f"Starting workflow: {workflow.name}")- try:- nodes = json.loads(workflow.nodes) if workflow.nodes else []- edges = json.loads(workflow.edges) if workflow.edges else []-- if not nodes:- WorkflowEngine.log(execution_id, "Workflow has no nodes", level='WARNING')- execution.status = 'success'- execution.completed_at = datetime.utcnow()- db.session.commit()- return-- # Build adjacency list and find start nodes- adj = {}- for edge in edges:- source = edge['source']- target = edge['target']- if source not in adj: adj[source] = []- adj[source].append(target)-- # Find trigger nodes or start nodes (no incoming edges)- incoming_count = {node['id']: 0 for node in nodes}- for edge in edges:- target = edge['target']- if target in incoming_count:- incoming_count[target] += 1-- # Start with nodes that have no incoming edges (usually triggers)- queue = [node for node in nodes if incoming_count[node['id']] == 0]-- if not queue:- # If everything is cyclic, just start with the first node- queue = [nodes[0]]-- # Execution state- results = {}- context = json.loads(execution.context) if execution.context else {}-- # Simple linear/BFS execution for now- # TODO: Implement full DAG execution with branch support- processed = set()-- while queue:- node = queue.pop(0)- if node['id'] in processed:- continue-- WorkflowEngine.log(execution_id, f"Executing node: {node.get('data', {}).get('label', node['id'])} ({node['type']})", node_id=node['id'])-- # Execute node- try:- node_result = WorkflowEngine._execute_node(node, edges, execution, context, results)- results[node['id']] = node_result-- if not node_result.get('success', True):- WorkflowEngine.log(execution_id, f"Node failed: {node_result.get('error')}", level='ERROR', node_id=node['id'])- if node_result.get('critical', True):- execution.status = 'failed'- break-- # Add next nodes to queue- if node['id'] in adj:- for next_id in adj[node['id']]:- next_node = next((n for n in nodes if n['id'] == next_id), None)- if next_node:- queue.append(next_node)-- except Exception as e:- WorkflowEngine.log(execution_id, f"Node Execution Error: {str(e)}", level='ERROR', node_id=node['id'])- WorkflowEngine.log(execution_id, traceback.format_exc(), level='DEBUG', node_id=node['id'])- execution.status = 'failed'- break-- processed.add(node['id'])-- if execution.status != 'failed':- execution.status = 'success'-- execution.results = json.dumps(results)- execution.completed_at = datetime.utcnow()- workflow.last_status = execution.status- db.session.commit()-- WorkflowEngine.log(execution_id, f"Workflow finished with status: {execution.status}")+ WorkflowEngine._log(execution_id, f"Starting workflow: {workflow.name}")- except Exception as e:- WorkflowEngine.log(execution_id, f"Execution Loop Error: {str(e)}", level='ERROR')- WorkflowEngine.log(execution_id, traceback.format_exc(), level='DEBUG')- execution.status = 'failed'+ if not nodes:+ WorkflowEngine._log(execution_id, "Workflow has no nodes", level='WARNING')+ execution.status = 'success'execution.completed_at = datetime.utcnow()db.session.commit()+ return++ # Build graph structures+ node_map = {n['id']: n for n in nodes}+ adj: Dict[str, List[Tuple[str, str]]] = {} # source -> [(target, sourceHandle)]+ in_degree: Dict[str, int] = {n['id']: 0 for n in nodes}++ for edge in edges:+ src = edge['source']+ tgt = edge['target']+ src_handle = edge.get('sourceHandle', 'output')+ if src not in adj:+ adj[src] = []+ adj[src].append((tgt, src_handle))+ if tgt in in_degree:+ in_degree[tgt] += 1++ # Start with root nodes (no incoming edges)+ ready = deque(nid for nid, deg in in_degree.items() if deg == 0)+ if not ready:+ ready.append(nodes[0]['id'])++ context = json.loads(execution.context) if execution.context else {}+ results: Dict[str, Dict] = {}+ processed: Set[str] = set()+ # Track which branches are active (for logic_if gating)+ # Maps node_id -> set of sourceHandles that were activated+ active_branches: Dict[str, Set[str]] = {}+ failed = False++ while ready and not failed:+ node_id = ready.popleft()++ if node_id in processed:+ continue++ node = node_map.get(node_id)+ if not node:+ continue++ # Check if this node is gated by a logic_if branch+ if not WorkflowEngine._is_node_reachable(node_id, edges, active_branches, processed):+ processed.add(node_id)+ # Still decrement successors so they can become ready+ for tgt, _ in adj.get(node_id, []):+ if tgt in in_degree:+ in_degree[tgt] -= 1+ if in_degree[tgt] == 0:+ ready.append(tgt)+ continue++ node_label = node.get('data', {}).get('label', node_id)+ WorkflowEngine._log(execution_id, f"Executing node: {node_label} ({node['type']})", node_id=node_id)++ try:+ node_result = WorkflowEngine._execute_node_with_retry(+ node, edges, execution, context, results+ )+ results[node_id] = node_result++ if not node_result.get('success', True):+ WorkflowEngine._log(+ execution_id,+ f"Node failed: {node_result.get('error', 'unknown')}",+ level='ERROR', node_id=node_id+ )+ if node_result.get('critical', True):+ failed = True+ break++ # For logic_if nodes, record which branch was taken+ if node['type'] == 'logic_if':+ branch = node_result.get('branch', 'true')+ if node_id not in active_branches:+ active_branches[node_id] = set()+ active_branches[node_id].add(branch)++ # Enqueue successor nodes whose in-degree reaches 0+ for tgt, src_handle in adj.get(node_id, []):+ if tgt in in_degree:+ in_degree[tgt] -= 1+ if in_degree[tgt] == 0:+ ready.append(tgt)++ except Exception as e:+ WorkflowEngine._log(execution_id, f"Node Execution Error: {str(e)}", level='ERROR', node_id=node_id)+ WorkflowEngine._log(execution_id, traceback.format_exc(), level='DEBUG', node_id=node_id)+ failed = True+ break++ processed.add(node_id)++ execution.status = 'failed' if failed else 'success'+ execution.results = json.dumps(results)+ execution.completed_at = datetime.utcnow()+ workflow.last_status = execution.status+ db.session.commit()++ WorkflowEngine._log(execution_id, f"Workflow finished with status: {execution.status}")++ @staticmethod+ def _is_node_reachable(node_id: str, edges: List[Dict],+ active_branches: Dict[str, Set[str]],+ processed: Set[str]) -> bool:+ """+ Check if a node should execute based on logic_if branching.++ A node is unreachable if ALL its incoming edges from logic_if nodes+ come through inactive branches.+ """+ incoming_from_logic = []++ for edge in edges:+ if edge['target'] != node_id:+ continue+ src = edge['source']+ src_handle = edge.get('sourceHandle', 'output')++ # Only gate on logic_if nodes that have already been processed+ if src in active_branches:+ incoming_from_logic.append((src, src_handle))++ if not incoming_from_logic:+ return True # No logic_if gating, always reachable++ # Reachable if at least one logic_if branch leading here is active+ for src, src_handle in incoming_from_logic:+ if src_handle in active_branches[src]:+ return True++ return False++ # ------------------------------------------------------------------+ # Node Execution with Retry+ # ------------------------------------------------------------------@staticmethod- def _execute_node(node: Dict, edges: List, execution: WorkflowExecution, context: Dict, results: Dict) -> Dict:+ def _execute_node_with_retry(node: Dict, edges: List, execution: WorkflowExecution,+ context: Dict, results: Dict) -> Dict:+ """Execute a node with retry support."""+ node_data = node.get('data', {})+ retry_count = min(int(node_data.get('retryCount', DEFAULT_RETRY_COUNT)), MAX_RETRY_COUNT)+ retry_delay = max(1, int(node_data.get('retryDelay', DEFAULT_RETRY_DELAY)))++ last_result = None+ for attempt in range(retry_count + 1):+ if attempt > 0:+ WorkflowEngine._log(+ execution.id,+ f"Retry {attempt}/{retry_count} after {retry_delay}s",+ node_id=node['id']+ )+ import time+ time.sleep(retry_delay)++ last_result = WorkflowEngine._execute_node(node, edges, execution, context, results)++ if last_result.get('success', True):+ return last_result++ return last_result++ # ------------------------------------------------------------------+ # Node Execution+ # ------------------------------------------------------------------++ @staticmethod+ def _execute_node(node: Dict, edges: List, execution: WorkflowExecution,+ context: Dict, results: Dict) -> Dict:"""Execute a single node and return its results."""node_type = node.get('type')node_data = node.get('data', {})-+if node_type == 'trigger':return {'success': True, 'output': context}-+elif node_type in ('database', 'dockerApp', 'service', 'domain'):- # Delegate to WorkflowServiceres = WorkflowService.deploy_node(node, edges, execution.workflow.user_id, results)return res-+elif node_type == 'notification':- channel = node_data.get('channel', 'discord')- message = node_data.get('message', 'Workflow notification')- # Replace placeholders in message from context/results- # TODO: Implementation-- NotificationService.send_notification(- user_id=execution.workflow.user_id,- title=f"Workflow: {execution.workflow.name}",- message=message,- level='info'- )- return {'success': True}-+ return WorkflowEngine._execute_notification(node, execution, context, results)+elif node_type == 'script':- script_type = node_data.get('language', 'bash')- content = node_data.get('content', '')-- if script_type == 'bash':- import subprocess- try:- res = subprocess.run(content, shell=True, capture_output=True, text=True)- return {- 'success': res.returncode == 0,- 'stdout': res.stdout,- 'stderr': res.stderr,- 'returncode': res.returncode- }- except Exception as e:- return {'success': False, 'error': str(e)}-- return {'success': False, 'error': f"Unsupported script language: {script_type}"}+ return WorkflowEngine._execute_script(node, execution, context, results)elif node_type == 'logic_if':- condition = node_data.get('condition', '')- # TODO: Evaluate condition using context/results- return {'success': True, 'branch': 'true'} # Dummy-- return {'success': True, 'message': f"Node type {node_type} not implemented yet"}+ return WorkflowEngine._execute_logic_if(node, context, results)++ return {'success': True, 'message': f"Node type {node_type} passed through"}++ # ------------------------------------------------------------------+ # Logic If Evaluation+ # ------------------------------------------------------------------@staticmethod- def log(execution_id: int, message: str, level: str = 'INFO', node_id: str = None):+ def _execute_logic_if(node: Dict, context: Dict, results: Dict) -> Dict:+ """+ Evaluate a logic_if condition.++ The condition is a Python expression that has access to:+ - results: dict of {node_id: node_result}+ - context: the workflow execution context+ """+ node_data = node.get('data', {})+ condition = node_data.get('condition', '').strip()++ if not condition:+ return {'success': True, 'branch': 'true'}++ # Build a safe evaluation namespace+ eval_globals = {"__builtins__": {}}+ eval_locals = {+ 'results': results,+ 'context': context,+ # Expose common helpers+ 'len': len,+ 'str': str,+ 'int': int,+ 'float': float,+ 'bool': bool,+ 'abs': abs,+ 'min': min,+ 'max': max,+ 'any': any,+ 'all': all,+ 'isinstance': isinstance,+ }++ try:+ result = eval(condition, eval_globals, eval_locals)+ branch = 'true' if result else 'false'+ return {+ 'success': True,+ 'branch': branch,+ 'condition': condition,+ 'evaluated': bool(result)+ }+ except Exception as e:+ return {+ 'success': False,+ 'error': f"Condition evaluation failed: {str(e)}",+ 'condition': condition,+ 'branch': 'false',+ 'critical': False # Don't kill the whole workflow for a bad condition+ }++ # ------------------------------------------------------------------+ # Variable Interpolation+ # ------------------------------------------------------------------++ @staticmethod+ def _interpolate(text: str, context: Dict, results: Dict,+ execution: Optional[WorkflowExecution] = None) -> str:+ """+ Replace variable placeholders in text.++ Supported syntax:+ - ${node_id.field} β access a specific field from a node's result+ - ${node_id.output} β shorthand for the output field+ - {{workflow_name}} β built-in workflow variables+ - {{execution_id}} β current execution ID+ - {{started_at}} β execution start time+ - {{context.field}} β access context fields+ """+ if not text or not isinstance(text, str):+ return text++ # Replace ${node_id.field} patterns+ def replace_node_var(match):+ node_id = match.group(1)+ field = match.group(2)+ node_result = results.get(node_id, {})+ if field == 'output' and 'output' not in node_result:+ # Try stdout as fallback for script nodes+ return str(node_result.get('stdout', ''))+ return str(node_result.get(field, ''))++ text = re.sub(r'\$\{([^.}]+)\.([^}]+)\}', replace_node_var, text)++ # Replace {{builtin}} patterns+ builtins = {+ 'workflow_name': execution.workflow.name if execution else '',+ 'execution_id': str(execution.id) if execution else '',+ 'started_at': execution.started_at.isoformat() if execution and execution.started_at else '',+ 'trigger_type': execution.trigger_type if execution else '',+ }++ # Add context.* variables+ for key, value in context.items():+ builtins[f'context.{key}'] = str(value)++ for key, value in builtins.items():+ text = text.replace('{{' + key + '}}', value)++ # Replace {{node_id.field}} as alternative syntax+ def replace_node_var_braces(match):+ node_id = match.group(1)+ field = match.group(2)+ node_result = results.get(node_id, {})+ return str(node_result.get(field, ''))++ text = re.sub(r'\{\{([^.}]+)\.([^}]+)\}\}', replace_node_var_braces, text)++ return text++ # ------------------------------------------------------------------+ # Notification Node+ # ------------------------------------------------------------------++ @staticmethod+ def _execute_notification(node: Dict, execution: WorkflowExecution,+ context: Dict, results: Dict) -> Dict:+ """Execute a notification node with variable interpolation."""+ node_data = node.get('data', {})+ channel = node_data.get('channel', 'system')+ message = node_data.get('message', 'Workflow notification')++ # Interpolate variables in the message+ message = WorkflowEngine._interpolate(message, context, results, execution)++ title = f"Workflow: {execution.workflow.name}"++ # Build alert in the format NotificationService expects+ alerts = [{+ 'type': 'workflow',+ 'severity': 'info',+ 'message': message,+ 'value': '',+ 'threshold': ''+ }]++ try:+ if channel == 'system' or channel == 'all':+ result = NotificationService.send_all(alerts)+ elif channel == 'discord':+ config = NotificationService.get_config().get('discord', {})+ result = NotificationService.send_discord(alerts, config)+ elif channel == 'slack':+ config = NotificationService.get_config().get('slack', {})+ result = NotificationService.send_slack(alerts, config)+ elif channel == 'email':+ config = NotificationService.get_config().get('email', {})+ result = NotificationService.send_email(alerts, config)+ elif channel == 'telegram':+ config = NotificationService.get_config().get('telegram', {})+ result = NotificationService.send_telegram(alerts, config)+ else:+ result = NotificationService.send_all(alerts)++ return {'success': result.get('success', True), 'channel': channel}+ except Exception as e:+ return {'success': False, 'error': str(e), 'critical': False}++ # ------------------------------------------------------------------+ # Script Node (Sandboxed)+ # ------------------------------------------------------------------++ @staticmethod+ def _execute_script(node: Dict, execution: WorkflowExecution,+ context: Dict, results: Dict) -> Dict:+ """Execute a script node with timeout, output limits, and variable interpolation."""+ node_data = node.get('data', {})+ script_type = node_data.get('language', 'bash')+ content = node_data.get('content', '')+ timeout = min(int(node_data.get('timeout', DEFAULT_TIMEOUT)), MAX_TIMEOUT)++ if not content.strip():+ return {'success': True, 'stdout': '', 'stderr': '', 'returncode': 0}++ # Interpolate variables in script content+ content = WorkflowEngine._interpolate(content, context, results, execution)++ # Build environment with node results available as env vars+ env = os.environ.copy()+ env['WORKFLOW_ID'] = str(execution.workflow_id)+ env['EXECUTION_ID'] = str(execution.id)+ env['TRIGGER_TYPE'] = execution.trigger_type or 'manual'++ for nid, nresult in results.items():+ safe_id = re.sub(r'[^a-zA-Z0-9_]', '_', nid).upper()+ if isinstance(nresult, dict):+ stdout = nresult.get('stdout', nresult.get('output', ''))+ if isinstance(stdout, str):+ env[f'NODE_{safe_id}_OUTPUT'] = stdout[:4096]+ rc = nresult.get('returncode')+ if rc is not None:+ env[f'NODE_{safe_id}_RC'] = str(rc)++ try:+ if script_type == 'bash':+ cmd = ['bash', '-c', content]+ elif script_type == 'python':+ cmd = ['python3', '-c', content]+ else:+ return {'success': False, 'error': f"Unsupported script language: {script_type}"}++ proc = subprocess.run(+ cmd,+ capture_output=True,+ text=True,+ timeout=timeout,+ env=env,+ cwd='/tmp' if os.name != 'nt' else None+ )++ stdout = proc.stdout[:MAX_OUTPUT_SIZE] if proc.stdout else ''+ stderr = proc.stderr[:MAX_OUTPUT_SIZE] if proc.stderr else ''++ return {+ 'success': proc.returncode == 0,+ 'stdout': stdout,+ 'stderr': stderr,+ 'returncode': proc.returncode+ }++ except subprocess.TimeoutExpired:+ return {+ 'success': False,+ 'error': f"Script timed out after {timeout}s",+ 'stdout': '',+ 'stderr': '',+ 'returncode': -1+ }+ except FileNotFoundError:+ fallback = 'python' if script_type == 'python' else script_type+ try:+ proc = subprocess.run(+ [fallback, '-c', content] if script_type == 'python' else content,+ capture_output=True, text=True, timeout=timeout,+ env=env, shell=(script_type == 'bash'),+ cwd='/tmp' if os.name != 'nt' else None+ )+ stdout = proc.stdout[:MAX_OUTPUT_SIZE] if proc.stdout else ''+ stderr = proc.stderr[:MAX_OUTPUT_SIZE] if proc.stderr else ''+ return {+ 'success': proc.returncode == 0,+ 'stdout': stdout, 'stderr': stderr,+ 'returncode': proc.returncode+ }+ except Exception as e:+ return {'success': False, 'error': str(e)}+ except Exception as e:+ return {'success': False, 'error': str(e)}++ # ------------------------------------------------------------------+ # Logging+ # ------------------------------------------------------------------++ @staticmethod+ def _log(execution_id: int, message: str, level: str = 'INFO', node_id: str = None):"""Add a log entry for an execution."""- log = WorkflowLog(+ log_entry = WorkflowLog(execution_id=execution_id,level=level,message=message,node_id=node_id)- db.session.add(log)+ db.session.add(log_entry)db.session.commit()- # Also print to stdout for debugging- print(f"[{level}] Workflow {execution_id}: {message}")+ logger.info(f"[{level}] Workflow {execution_id}: {message}")++ # Keep backward-compatible alias+ log = _log+++# ------------------------------------------------------------------+# Event Bus for workflow triggers+# ------------------------------------------------------------------++class WorkflowEventBus:+ """+ Simple in-process event bus for triggering workflows on system events.++ Events are emitted by services (monitoring, health checks, git deploy)+ and matched against workflows with trigger_type='event'.+ """++ _listeners_lock = threading.Lock()++ @staticmethod+ def emit(event_type: str, data: Dict[str, Any] = None):+ """+ Emit an event that may trigger workflows.++ Args:+ event_type: One of health_check_failed, high_cpu, high_memory,+ git_push, app_stopped, or any custom string.+ data: Event payload passed as workflow context.+ """+ from flask import current_app+ try:+ app = current_app._get_current_object()+ except RuntimeError:+ logger.warning(f"WorkflowEventBus.emit called outside app context for {event_type}")+ return++ threading.Thread(+ target=WorkflowEventBus._process_event,+ args=(app, event_type, data or {}),+ daemon=True,+ name=f'wf-event-{event_type}'+ ).start()++ @staticmethod+ def _process_event(app, event_type: str, data: Dict):+ """Find and execute workflows subscribed to this event type."""+ with app.app_context():+ try:+ workflows = Workflow.query.filter_by(+ is_active=True,+ trigger_type='event'+ ).all()++ for workflow in workflows:+ try:+ config = json.loads(workflow.trigger_config) if workflow.trigger_config else {}+ subscribed_event = config.get('eventType', '')++ if subscribed_event != event_type:+ continue++ # Cooldown: don't re-trigger within 60 seconds+ if workflow.last_run_at:+ elapsed = (datetime.utcnow() - workflow.last_run_at).total_seconds()+ if elapsed < 60:+ continue++ logger.info(f"Event '{event_type}' triggering workflow: {workflow.name}")+ context = {+ 'event_type': event_type,+ 'event_data': data,+ 'triggered_at': datetime.utcnow().isoformat()+ }+ WorkflowEngine.execute_workflow(+ workflow_id=workflow.id,+ trigger_type='event',+ context=context+ )+ except Exception as e:+ logger.error(f"Event trigger failed for workflow {workflow.id}: {e}")++ except Exception as e:+ logger.error(f"WorkflowEventBus._process_event error: {e}")diff --git a/backend/migrations/versions/003_workflows_automation.py b/backend/migrations/versions/003_workflows_automation.pynew file mode 100644index 00000000..6f59b3d9--- /dev/null+++ b/backend/migrations/versions/003_workflows_automation.py@@ -0,0 +1,90 @@+"""Add workflows and automation tables.++Revision ID: 003_workflows_automation+Revises: 002_permissions_invitations+Create Date: 2026-03-23+"""+from alembic import op+import sqlalchemy as sa+from datetime import datetime++revision = '003_workflows_automation'+down_revision = '002_permissions_invitations'+branch_labels = None+depends_on = None+++def upgrade():+ conn = op.get_bind()+ inspector = sa.inspect(conn)+ existing_tables = inspector.get_table_names()++ # Create workflows table if missing (old version might have it but without automation fields)+ if 'workflows' not in existing_tables:+ op.create_table('workflows',+ sa.Column('id', sa.Integer(), primary_key=True),+ sa.Column('name', sa.String(100), nullable=False),+ sa.Column('description', sa.Text(), nullable=True),+ sa.Column('nodes', sa.Text(), nullable=True),+ sa.Column('edges', sa.Text(), nullable=True),+ sa.Column('viewport', sa.Text(), nullable=True),+ sa.Column('is_active', sa.Boolean(), default=False),+ sa.Column('trigger_type', sa.String(50), default='manual'),+ sa.Column('trigger_config', sa.Text(), nullable=True),+ sa.Column('last_run_at', sa.DateTime(), nullable=True),+ sa.Column('last_status', sa.String(20), nullable=True),+ sa.Column('created_at', sa.DateTime(), default=datetime.utcnow),+ sa.Column('updated_at', sa.DateTime(), default=datetime.utcnow),+ sa.Column('user_id', sa.Integer(), sa.ForeignKey('users.id'), nullable=False)+ )+ else:+ # Add automation columns to existing workflows table if they don't exist+ existing_cols = {c['name'] for c in inspector.get_columns('workflows')}+ with op.batch_alter_table('workflows') as batch_op:+ if 'is_active' not in existing_cols:+ batch_op.add_column(sa.Column('is_active', sa.Boolean(), nullable=True, server_default='0'))+ if 'trigger_type' not in existing_cols:+ batch_op.add_column(sa.Column('trigger_type', sa.String(50), nullable=True, server_default='manual'))+ if 'trigger_config' not in existing_cols:+ batch_op.add_column(sa.Column('trigger_config', sa.Text(), nullable=True))+ if 'last_run_at' not in existing_cols:+ batch_op.add_column(sa.Column('last_run_at', sa.DateTime(), nullable=True))+ if 'last_status' not in existing_cols:+ batch_op.add_column(sa.Column('last_status', sa.String(20), nullable=True))++ # Create workflow_executions table+ if 'workflow_executions' not in existing_tables:+ op.create_table('workflow_executions',+ sa.Column('id', sa.Integer(), primary_key=True),+ sa.Column('workflow_id', sa.Integer(), sa.ForeignKey('workflows.id'), nullable=False),+ sa.Column('status', sa.String(20), default='running'),+ sa.Column('trigger_type', sa.String(50), nullable=True),+ sa.Column('context', sa.Text(), nullable=True),+ sa.Column('results', sa.Text(), nullable=True),+ sa.Column('started_at', sa.DateTime(), default=datetime.utcnow),+ sa.Column('completed_at', sa.DateTime(), nullable=True)+ )++ # Create workflow_logs table+ if 'workflow_logs' not in existing_tables:+ op.create_table('workflow_logs',+ sa.Column('id', sa.Integer(), primary_key=True),+ sa.Column('execution_id', sa.Integer(), sa.ForeignKey('workflow_executions.id'), nullable=False),+ sa.Column('level', sa.String(10), default='INFO'),+ sa.Column('message', sa.Text(), nullable=False),+ sa.Column('node_id', sa.String(100), nullable=True),+ sa.Column('timestamp', sa.DateTime(), default=datetime.utcnow)+ )+++def downgrade():+ op.drop_table('workflow_logs')+ op.drop_table('workflow_executions')++ # We don't drop 'workflows' as it existed before, but we can drop the new columns+ with op.batch_alter_table('workflows') as batch_op:+ batch_op.drop_column('last_status')+ batch_op.drop_column('last_run_at')+ batch_op.drop_column('trigger_config')+ batch_op.drop_column('trigger_type')+ batch_op.drop_column('is_active')diff --git a/dev.sh b/dev.shindex c157ffc6..8028c0cf 100644--- a/dev.sh+++ b/dev.sh@@ -17,7 +17,7 @@ FRONTEND_DIR="$PROJECT_ROOT/frontend"CYAN='\033[0;36m'GREEN='\033[0;32m'RED='\033[0;31m'-YELLOW='\033[0;33m'+YELLOW='\033[1;33m'DIM='\033[2m'NC='\033[0m'diff --git a/frontend/src/components/workflow/nodes/TriggerNode.jsx b/frontend/src/components/workflow/nodes/TriggerNode.jsxindex 5247ca64..462ddd8b 100644--- a/frontend/src/components/workflow/nodes/TriggerNode.jsx+++ b/frontend/src/components/workflow/nodes/TriggerNode.jsx@@ -3,12 +3,12 @@ import { Handle, Position } from '@xyflow/react';import { Zap, Clock, Webhook, Activity, Play } from 'lucide-react';const TriggerNode = ({ data, selected }) => {- const {- triggerType = 'manual',+ const {+ triggerType = 'manual',label = 'Trigger',isActive = false,- lastRunAt = null,- lastStatus = null+ lastStatus = null,+ triggerConfig = {}} = data;const getIcon = () => {@@ -28,6 +28,30 @@ const TriggerNode = ({ data, selected }) => {return 'border-blue-500 shadow-[0_0_10px_rgba(59,130,246,0.3)]';};+ const getSublabel = () => {+ const typeName = triggerType.charAt(0).toUpperCase() + triggerType.slice(1);+ const status = isActive ? 'Active' : 'Disabled';++ if (triggerType === 'cron' && triggerConfig.cron) {+ return `${triggerConfig.cron} \u2022 ${status}`;+ }+ if (triggerType === 'webhook' && triggerConfig.webhook_id) {+ return `...${triggerConfig.webhook_id.slice(-8)} \u2022 ${status}`;+ }+ if (triggerType === 'event' && triggerConfig.eventType) {+ const eventLabels = {+ health_check_failed: 'Health Fail',+ high_cpu: 'High CPU',+ high_memory: 'High Memory',+ git_push: 'Git Push',+ app_stopped: 'App Stopped'+ };+ return `${eventLabels[triggerConfig.eventType] || triggerConfig.eventType} \u2022 ${status}`;+ }++ return `${typeName} \u2022 ${status}`;+ };+return (<div className={`workflow-node node-trigger ${selected ? 'node-selected' : ''} ${getStatusColor()}`}><div className="node-icon bg-gray-800">@@ -35,12 +59,9 @@ const TriggerNode = ({ data, selected }) => {</div><div className="node-content"><div className="node-label">{label}</div>- <div className="node-sublabel">- {triggerType.charAt(0).toUpperCase() + triggerType.slice(1)}- {isActive ? ' β’ Active' : ' β’ Disabled'}- </div>+ <div className="node-sublabel">{getSublabel()}</div></div>-+<Handletype="source"position={Position.Bottom}diff --git a/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsx b/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsxindex fe7525ed..1c1e3de5 100644--- a/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsx+++ b/frontend/src/components/workflow/panels/LogicIfConfigPanel.jsx@@ -1,5 +1,4 @@-import React from 'react';-import { X, GitBranch } from 'lucide-react';+import { X } from 'lucide-react';const LogicIfConfigPanel = ({ node, onChange, onClose }) => {const { data } = node;@@ -36,19 +35,23 @@ const LogicIfConfigPanel = ({ node, onChange, onClose }) => {<div className="mt-4 p-3 bg-gray-800 rounded border border-gray-700"><p className="text-[10px] text-gray-400"><strong>Available Variables:</strong><br />- <code>results</code>: Dictionary of previous node outputs<br />- <code>context</code>: Trigger execution context+ <code>results</code> β dict of previous node outputs, keyed by node ID<br />+ <code>context</code> β trigger execution context<br /><br />+ <strong>Examples:</strong><br />+ <code className="text-gray-300">{"results['abc123']['returncode'] == 0"}</code><br />+ <code className="text-gray-300">{"context.get('event_type') == 'high_cpu'"}</code><br />+ <code className="text-gray-300">len(results) > 0</code></p></div><div className="mt-4 space-y-2"><div className="flex items-center gap-2 text-xs"><div className="w-2 h-2 rounded-full bg-green-500"></div>- <span className="text-gray-300">True branch connects to "TRUE" handle</span>+ <span className="text-gray-300">True branch β condition evaluates to truthy</span></div><div className="flex items-center gap-2 text-xs"><div className="w-2 h-2 rounded-full bg-red-500"></div>- <span className="text-gray-300">False branch connects to "FALSE" handle</span>+ <span className="text-gray-300">False branch β condition evaluates to falsy or errors</span></div></div></div>diff --git a/frontend/src/components/workflow/panels/NotificationConfigPanel.jsx b/frontend/src/components/workflow/panels/NotificationConfigPanel.jsxindex 04e7ccfc..b325f8a7 100644--- a/frontend/src/components/workflow/panels/NotificationConfigPanel.jsx+++ b/frontend/src/components/workflow/panels/NotificationConfigPanel.jsx@@ -51,7 +51,7 @@ const NotificationConfigPanel = ({ node, onChange, onClose }) => {onClick={() => onChange({ ...data, channel: 'system' })}><Bell size={14} />- <span className="text-xs">System</span>+ <span className="text-xs">All</span></button></div></div>@@ -62,14 +62,17 @@ const NotificationConfigPanel = ({ node, onChange, onClose }) => {className="text-xs bg-gray-900 border-gray-700 rounded p-2 h-48 focus:border-blue-500 focus:outline-none"value={message}onChange={(e) => onChange({ ...data, message: e.target.value })}- placeholder="e.g. Workflow {{workflow_name}} failed on node {{failed_node_id}}"+ placeholder="e.g. Build finished for {{workflow_name}} β exit code: ${build_node.returncode}"/></div><div className="mt-4 p-3 bg-gray-800 rounded border border-gray-700"><p className="text-[10px] text-gray-400">- <strong>Available Placeholders:</strong><br />- <code>{{workflow_name}}</code>, <code>{{execution_id}}</code>, <code>{{started_at}}</code>, <code>{{node_id.output}}</code>+ <strong>Variables:</strong><br />+ <code>{'{{workflow_name}}'}</code>, <code>{'{{execution_id}}'}</code>, <code>{'{{started_at}}'}</code><br />+ <code>{'${node_id.stdout}'}</code> β output from a specific node<br />+ <code>{'${node_id.returncode}'}</code> β exit code from a script node<br />+ <code>{'{{context.field}}'}</code> β value from trigger context</p></div></div>diff --git a/frontend/src/components/workflow/panels/ScriptConfigPanel.jsx b/frontend/src/components/workflow/panels/ScriptConfigPanel.jsxindex 88111dab..7d1fa9c6 100644--- a/frontend/src/components/workflow/panels/ScriptConfigPanel.jsx+++ b/frontend/src/components/workflow/panels/ScriptConfigPanel.jsx@@ -3,7 +3,14 @@ import { X, Terminal, Code } from 'lucide-react';const ScriptConfigPanel = ({ node, onChange, onClose }) => {const { data } = node;- const { language = 'bash', label = 'Run Script', content = '' } = data;+ const {+ language = 'bash',+ label = 'Run Script',+ content = '',+ timeout = 300,+ retryCount = 0,+ retryDelay = 5+ } = data;return (<div className="config-panel">@@ -52,9 +59,45 @@ const ScriptConfigPanel = ({ node, onChange, onClose }) => {/></div>+ <div className="grid grid-cols-3 gap-2">+ <div className="form-group">+ <label>Timeout (s)</label>+ <input+ type="number"+ min="1"+ max="3600"+ value={timeout}+ onChange={(e) => onChange({ ...data, timeout: parseInt(e.target.value) || 300 })}+ />+ </div>+ <div className="form-group">+ <label>Retries</label>+ <input+ type="number"+ min="0"+ max="5"+ value={retryCount}+ onChange={(e) => onChange({ ...data, retryCount: parseInt(e.target.value) || 0 })}+ />+ </div>+ <div className="form-group">+ <label>Delay (s)</label>+ <input+ type="number"+ min="1"+ max="300"+ value={retryDelay}+ onChange={(e) => onChange({ ...data, retryDelay: parseInt(e.target.value) || 5 })}+ />+ </div>+ </div>+<div className="mt-4 p-3 bg-blue-900/20 rounded border border-blue-500/30"><p className="text-[10px] text-blue-300">- <strong>Environment Variables:</strong> You can use <code>$NODE_ID_OUTPUT</code> to access results from previous nodes.+ <strong>Variables:</strong><br />+ <code className="text-blue-200">{'${node_id.stdout}'}</code> β output from a previous node<br />+ <code className="text-blue-200">$NODE_ID_OUTPUT</code> β same, as env var<br />+ <code className="text-blue-200">$WORKFLOW_ID</code>, <code className="text-blue-200">$EXECUTION_ID</code> β workflow metadata</p></div></div>diff --git a/frontend/src/components/workflow/panels/TriggerConfigPanel.jsx b/frontend/src/components/workflow/panels/TriggerConfigPanel.jsxindex 24d4c06f..9d956231 100644--- a/frontend/src/components/workflow/panels/TriggerConfigPanel.jsx+++ b/frontend/src/components/workflow/panels/TriggerConfigPanel.jsx@@ -1,9 +1,10 @@-import React from 'react';-import { X, Play, Clock, Webhook, Activity, Zap } from 'lucide-react';+import React, { useState } from 'react';+import { X, Play, Clock, Webhook, Zap, Copy, Check } from 'lucide-react';const TriggerConfigPanel = ({ node, onChange, onClose }) => {const { data } = node;const { triggerType = 'manual', label = 'Trigger', triggerConfig = {} } = data;+ const [copied, setCopied] = useState(false);const handleTypeChange = (type) => {onChange({@@ -23,6 +24,18 @@ const TriggerConfigPanel = ({ node, onChange, onClose }) => {});};+ const webhookUrl = triggerConfig.webhook_id+ ? `${window.location.origin}/api/v1/workflows/hooks/${triggerConfig.webhook_id}`+ : null;++ const copyWebhookUrl = () => {+ if (webhookUrl) {+ navigator.clipboard.writeText(webhookUrl);+ setCopied(true);+ setTimeout(() => setCopied(false), 2000);+ }+ };+return (<div className="config-panel"><div className="panel-header">@@ -92,17 +105,38 @@ const TriggerConfigPanel = ({ node, onChange, onClose }) => {{triggerType === 'webhook' && (<div className="form-group animate-in fade-in slide-in-from-top-1"><label>Webhook URL</label>- <div className="flex gap-2">- <input- type="text"- value={triggerConfig.webhookPath || `/hooks/${node.id}`}- readOnly- className="bg-gray-900 text-gray-400 cursor-not-allowed"- />+ {webhookUrl ? (+ <>+ <div className="flex gap-2">+ <input+ type="text"+ value={webhookUrl}+ readOnly+ className="bg-gray-900 text-gray-300 cursor-default font-mono text-xs flex-1"+ />+ <button+ onClick={copyWebhookUrl}+ className="px-3 py-1 rounded border border-gray-600 bg-gray-800 hover:bg-gray-700 transition-colors"+ title="Copy URL"+ >+ {copied ? <Check size={14} className="text-green-400" /> : <Copy size={14} className="text-gray-400" />}+ </button>+ </div>+ <p className="text-[10px] text-gray-500 mt-1">+ Send POST requests to this URL to trigger the workflow.+ </p>+ </>+ ) : (+ <p className="text-[10px] text-yellow-400 mt-1">+ Save the workflow to generate the webhook URL.+ </p>+ )}+ <div className="mt-3 p-3 bg-gray-800 rounded border border-gray-700">+ <p className="text-[10px] text-gray-400">+ <strong>Request body</strong> is available in your workflow as <code>context.body</code>.+ Query parameters are in <code>context.query</code>.+ </p></div>- <p className="text-[10px] text-gray-500 mt-1">- Send POST requests to this endpoint to trigger the workflow.- </p></div>)}@@ -114,11 +148,14 @@ const TriggerConfigPanel = ({ node, onChange, onClose }) => {onChange={(e) => handleConfigChange('eventType', e.target.value)}><option value="health_check_failed">Health Check Failed</option>- <option value="high_cpu">High CPU Usage (>80%)</option>- <option value="high_memory">High Memory Usage (>80%)</option>+ <option value="high_cpu">High CPU Usage (>80%)</option>+ <option value="high_memory">High Memory Usage (>80%)</option><option value="git_push">Git Push Received</option><option value="app_stopped">Application Stopped</option></select>+ <p className="text-[10px] text-gray-500 mt-1">+ Event data is available as <code>context.event_data</code>.+ </p></div>)}</div>diff --git a/frontend/src/services/api.js b/frontend/src/services/api.jsindex 384651b4..bdce5b5b 100644--- a/frontend/src/services/api.js+++ b/frontend/src/services/api.js@@ -2581,6 +2581,13 @@ class ApiService {return this.request(`/workflows/executions/${executionId}/logs`);}+ async validateWorkflow(data) {+ return this.request('/workflows/validate', {+ method: 'POST',+ body: JSON.stringify(data)+ });+ }+// ========================================// Servers (Multi-Server Management) endpoints// ========================================diff --git a/quick-start.sh b/quick-start.shnew file mode 100644index 00000000..230df9d5--- /dev/null+++ b/quick-start.sh@@ -0,0 +1,81 @@+#!/usr/bin/env bash+#+# ServerKit - All-in-One Local Quick Start+#+# This script automates:+# 1. System dependency checks+# 2. Python venv & dependency setup+# 3. Frontend npm installation+# 4. Environment configuration (.env)+# 5. Launching both Backend & Frontend+#++set -euo pipefail++# Colors+CYAN='\033[0;36m'+GREEN='\033[0;32m'+RED='\033[0;31m'+YELLOW='\033[1;33m'+NC='\033[0m'++echo -e "${CYAN}π Starting ServerKit Local Setup...${NC}"++# 1. Check for required system tools+check_tool() {+ if ! command -v "$1" &>/dev/null; then+ echo -e "${RED}β $1 is not installed.${NC}"+ echo -e "${YELLOW}Please run: sudo apt update && sudo apt install -y $2${NC}"+ exit 1+ fi+}++check_tool "python3" "python3 python3-venv python3-pip"+check_tool "node" "nodejs"+check_tool "npm" "npm"++# 2. Setup Backend+echo -e "\n${CYAN}π¦ Setting up Backend...${NC}"+if [ ! -d "venv" ]; then+ echo "Creating virtual environment..."+ python3 -m venv venv+fi++source venv/bin/activate+echo "Installing Python dependencies..."+pip install -r backend/requirements.txt --quiet+pip install gunicorn gevent gevent-websocket --quiet++# 3. Setup Frontend+echo -e "\n${CYAN}π¦ Setting up Frontend...${NC}"+cd frontend+if [ ! -d "node_modules" ]; then+ echo "Installing Node modules (this may take a minute)..."+ npm install --silent+else+ echo "Node modules already installed."+fi+cd ..++# 4. Configuration+echo -e "\n${CYAN}βοΈ Configuring Environment...${NC}"+if [ ! -f ".env" ]; then+ echo "Creating .env from .env.example..."+ cp .env.example .env+ # Generate random secrets for dev+ sed -i "s/SECRET_KEY=.*/SECRET_KEY=$(openssl rand -hex 16)/" .env+ sed -i "s/JWT_SECRET_KEY=.*/JWT_SECRET_KEY=$(openssl rand -hex 16)/" .env+else+ echo ".env file already exists."+fi++# 5. Launch+echo -e "\n${GREEN}β Setup Complete!${NC}"+echo -e "Starting ServerKit..."+echo -e " - ${CYAN}Frontend:${NC} http://localhost:5173"+echo -e " - ${CYAN}Backend:${NC} http://localhost:5000"+echo -e "\n${YELLOW}Press Ctrl+C to stop both servers.${NC}\n"++# Run dev.sh to handle the dual-process execution+chmod +x dev.sh+./dev.sh