ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
feat: Create Milestone 4 - Container Logs & Monitoring (v1.3)
Juan Denisc25ea3851753
diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.mdindex 5527b29e..acfb58ea 100644--- a/.planning/ROADMAP.md+++ b/.planning/ROADMAP.md@@ -10,17 +10,19 @@ Completed 2026-01-19 | 9 phases | 109 commits | +14,667/-2,836 lines### [Milestone 2: Multi-Environment WordPress & Community Features](./milestones/v1.1-ROADMAP.md) (v1.1)Completed 2026-01-19 | 5 phases | Multi-environment app linking, shared DB support, GitHub integration+### [Milestone 3: Visual Workflow Builder](./milestones/v1.2-ROADMAP.md) (v1.2)+Completed 2026-01-19 | 6 phases | React Flow canvas, node types, deploy from workflow+---## Current Milestone-> **Milestone 3**: Visual Workflow Builder-> **Version**: v1.2+> **Milestone 4**: Container Logs & Monitoring+> **Version**: v1.3> **Started**: 2026-01-19-> **Completed**: 2026-01-19-> **Status**: Complete+> **Status**: In Progress-**Goal**: Create a visual, flow-based interface for orchestrating Docker apps, databases, and domain connections - an alternative "v2" way to manage infrastructure visually using a node-based editor.+**Goal**: Add comprehensive container logging and monitoring capabilities - real-time log streaming, search/filtering, per-app resource graphs, and configurable alert rules with notifications.---@@ -28,196 +30,169 @@ Completed 2026-01-19 | 5 phases | Multi-environment app linking, shared DB suppo| Phase | Name | Status | Goal ||-------|------|--------|------|-| 16 | Workflow Canvas Foundation | **completed** | React Flow canvas with pan/zoom, node rendering, connection system |-| 17 | Resource Node Types | **completed** | Create node types for Docker apps, databases, domains |-| 18 | Node Configuration Panels | **completed** | Side panels for configuring each node type |-| 19 | Connection Logic | **completed** | Define and validate connections between nodes |-| 20 | Workflow Save/Load | **completed** | Save workflows to backend, load existing infrastructure |-| 21 | Deploy from Workflow | **completed** | Execute workflow to create/update actual resources |+| 22 | Container Logs API | pending | Backend API to fetch/stream Docker container logs |+| 23 | Log Viewer UI | pending | Real-time log display with auto-scroll, ANSI color support |+| 24 | Log Search & Filtering | pending | Search within logs, filter by level/time, export |+| 25 | Per-App Resource Collection | pending | Collect per-container CPU/memory/network stats via Docker API |+| 26 | App Resource Graphs | pending | Per-app resource usage charts in application detail view |+| 27 | Alert Rules Engine | pending | Define thresholds, evaluate metrics, trigger alerts |+| 28 | Alert Notifications | pending | Send alerts via email, Discord, Telegram webhooks |----## Phase 16: Workflow Canvas Foundation ✓-**Goal**: Set up React Flow canvas with pan/zoom, basic node rendering, and connection system--**Status**: Completed 2026-01-19+## Phase 22: Container Logs API+**Goal**: Create backend API to fetch and stream Docker container logs-### What Was Built-- Installed @xyflow/react (React Flow) library-- Created WorkflowBuilder page with canvas at /workflow-- Added Background, Controls, and MiniMap components-- Created BaseNode component with connection handles-- Added node toolbar for adding Docker/Database/Domain nodes-- Comprehensive LESS styles for workflow UI+**Key Tasks**:+- Add Docker logs service to fetch container logs+- Create GET /apps/{id}/logs endpoint with tail/since parameters+- Add WebSocket endpoint for real-time log streaming+- Handle log rotation and buffer limits+- Support multiple output formats (raw, JSON)-### Success Criteria-- [x] Canvas renders with pan/zoom-- [x] Can add placeholder nodes-- [x] Can draw connections between nodes-- [x] Minimap shows overview--[Full Summary](./phases/16-workflow-canvas-foundation/SUMMARY.md)+**Success Criteria**:+- [ ] Can fetch last N lines of container logs+- [ ] Can stream logs in real-time via WebSocket+- [ ] Logs include timestamps+- [ ] Handles containers that aren't running----## Phase 17: Resource Node Types ✓-**Goal**: Create visual node types for Docker apps, databases, and domains--**Status**: Completed 2026-01-19+## Phase 23: Log Viewer UI+**Goal**: Create real-time log viewer component in application detail page-### What Was Built-- DockerAppNode with status, ports, memory display, multi-handles-- DatabaseNode with type badges (MySQL, PostgreSQL, MongoDB, Redis)-- DomainNode with SSL/DNS status badges-- ServiceNode for future extensibility (Redis, Memcached, RabbitMQ)-- Categorized palette (Compute, Storage, Network)-- Node-specific styles with animated status indicators+**Key Tasks**:+- Create LogViewer component with terminal-style display+- Implement auto-scroll with pause on user scroll+- Add ANSI color code parsing for colored output+- Connect to WebSocket for live streaming+- Add play/pause, clear, and download buttons-### Success Criteria-- [x] Each resource type has distinct visual node-- [x] Nodes display key info (name, status, type)-- [x] Node palette allows adding new nodes-- [x] Nodes are draggable on canvas--[Full Summary](./phases/17-resource-node-types/SUMMARY.md)+**Success Criteria**:+- [ ] Logs display in real-time+- [ ] Colors render correctly (ANSI codes)+- [ ] Auto-scroll works with manual override+- [ ] Can pause/resume streaming----## Phase 18: Node Configuration Panels ✓-**Goal**: Create side panels for configuring each node type--**Status**: Completed 2026-01-19--### What Was Built-- ConfigPanel base component with slide animation-- DockerAppConfigPanel with ports list and memory input-- DatabaseConfigPanel with type-aware default ports-- DomainConfigPanel with SSL/DNS status fields-- ServiceConfigPanel with service type selector-- Panel wiring in WorkflowBuilder with real-time updates+## Phase 24: Log Search & Filtering+**Goal**: Add search and filtering capabilities to log viewer-### Success Criteria-- [x] Clicking node opens config panel-- [x] Panel contains appropriate form for node type-- [x] Form changes reflect in node display-- [x] Panel can be closed/dismissed+**Key Tasks**:+- Add search input with highlight matches+- Filter by log level (INFO, WARN, ERROR, DEBUG)+- Filter by time range (last hour, today, custom)+- Add regex search option+- Export filtered logs to file-[Full Summary](./phases/18-node-configuration-panels/SUMMARY.md)+**Success Criteria**:+- [ ] Can search within logs+- [ ] Search matches highlighted+- [ ] Level filtering works+- [ ] Can export logs----## Phase 19: Connection Logic ✓-**Goal**: Define and validate connections between nodes+## Phase 25: Per-App Resource Collection+**Goal**: Collect and store per-container resource metrics-**Status**: Completed 2026-01-19+**Key Tasks**:+- Use Docker stats API to get container metrics+- Create ContainerMetrics model for storage+- Add background collection job (every 30s)+- Store CPU %, memory usage, network I/O per container+- Create API endpoint GET /apps/{id}/metrics/history-### What Was Built-- Connection validation utility with rules matrix-- isValidConnection callback preventing invalid connections-- Error toast for invalid connection attempts-- Custom ConnectionEdge with labels and delete button-- Edge selection and deletion (Delete/Backspace keys)-- Connection styles with hover/selected states--### Success Criteria-- [x] Can only create valid connections-- [x] Invalid connections show error feedback-- [x] Connections display relevant metadata-- [x] Can edit and delete connections--[Full Summary](./phases/19-connection-logic/SUMMARY.md)+**Success Criteria**:+- [ ] Per-container metrics collected+- [ ] Historical data stored (7 days retention)+- [ ] API returns time-series data+- [ ] Collection doesn't impact performance----## Phase 20: Workflow Save/Load ✓-**Goal**: Persist workflows and load existing infrastructure as workflows--**Status**: Completed 2026-01-19--### What Was Built-- Created Workflow SQLAlchemy model with nodes/edges/viewport as JSON-- Created CRUD API endpoints (GET/POST/PUT/DELETE /workflows)-- Registered workflows blueprint at /api/v1/workflows-- Added frontend API methods-- Added workflow toolbar with name input, New/Load/Import/Save buttons-- Created WorkflowListModal for loading saved workflows-- Added import existing infrastructure feature+## Phase 26: App Resource Graphs+**Goal**: Display per-app resource usage charts in application detail-### Success Criteria-- [x] Workflows save to database-- [x] Workflows load correctly-- [x] Existing apps/DBs appear as nodes-- [x] Can manage multiple workflows+**Key Tasks**:+- Create AppMetricsGraph component using Recharts+- Add to application detail page (new Metrics tab or Overview)+- Show CPU, memory, network charts+- Add time range selector (1h, 6h, 24h, 7d)+- Show current vs average stats-[Full Summary](./phases/20-workflow-save-load/SUMMARY.md)+**Success Criteria**:+- [ ] CPU/memory graphs display correctly+- [ ] Time range selection works+- [ ] Charts update with new data+- [ ] Mobile responsive----## Phase 21: Deploy from Workflow ✓-**Goal**: Execute workflow to create/update actual resources+## Phase 27: Alert Rules Engine+**Goal**: Create system for defining and evaluating alert rules-**Status**: Completed 2026-01-19+**Key Tasks**:+- Create AlertRule model (metric, operator, threshold, duration)+- Create AlertHistory model for triggered alerts+- Add alert evaluation service (runs with metrics collection)+- Create CRUD API for alert rules+- Support app-specific and system-wide rules-### What Was Built-- Created WorkflowService with deployment orchestration-- Deployment order: databases → apps → domains-- Database deployment creates MySQL/PostgreSQL with auto-generated credentials-- Docker app deployment creates Application + docker-compose files-- Domain deployment creates Domain linked to app with nginx config-- Added POST /workflows/{id}/deploy API endpoint-- Added deployWorkflow() frontend API method-- Added Deploy button to toolbar (green, requires saved workflow)-- Created DeploymentProgressModal with per-node status display--### Success Criteria-- [x] Can deploy new infrastructure from workflow-- [x] Existing resources update correctly-- [x] Progress visible during deployment-- [x] Errors handled with clear feedback--[Full Summary](./phases/21-workflow-deployment/SUMMARY.md)+**Success Criteria**:+- [ ] Can create/edit/delete alert rules+- [ ] Rules evaluate against metrics+- [ ] Alerts trigger when thresholds exceeded+- [ ] Alert history tracked----## Milestone 3 Complete+## Phase 28: Alert Notifications+**Goal**: Send alert notifications through multiple channels++**Key Tasks**:+- Integrate with existing notification preferences+- Send email alerts (if configured)+- Send Discord webhook alerts+- Send Telegram bot alerts+- Add alert cooldown to prevent spam+- Create notification templates-All 6 phases completed:-- Phase 16: Workflow Canvas Foundation-- Phase 17: Resource Node Types-- Phase 18: Node Configuration Panels-- Phase 19: Connection Logic-- Phase 20: Workflow Save/Load-- Phase 21: Deploy from Workflow+**Success Criteria**:+- [ ] Email notifications work+- [ ] Discord webhooks work+- [ ] Telegram notifications work+- [ ] Cooldown prevents duplicate alerts---## Dependencies```-Phase 16 (Canvas) ────────────► Phase 17 (Nodes)- │- ▼-Phase 17 ─────────────────────► Phase 18 (Panels)- │- ▼-Phase 18 ─────────────────────► Phase 19 (Connections)- │- ▼-Phase 19 ─────────────────────► Phase 20 (Save/Load)- │- ▼-Phase 20 ─────────────────────► Phase 21 (Deploy)+Phase 22 (Logs API) ──────► Phase 23 (Log Viewer)+ │+ ▼+Phase 23 ─────────────────► Phase 24 (Search/Filter)++Phase 25 (Metrics Collection) ──► Phase 26 (Graphs)+ │+ ▼+Phase 26 ─────────────────────► Phase 27 (Alert Rules)+ │+ ▼+Phase 27 ─────────────────────► Phase 28 (Notifications)```----## Notes+## Technical Notes-- This is a UI-layer feature - reuses all existing backend APIs-- React Flow is the recommended library for node-based editors-- Consider localStorage for autosave during development-- Start simple - MVP is canvas + nodes + basic connections-- Full deployment can be phased (start with import existing)+- Docker SDK for Python: `docker.containers.get(id).logs()`+- Docker stats API: `docker.containers.get(id).stats(stream=True)`+- WebSocket: Use existing Flask-SocketIO setup+- Charts: Recharts (already installed)+- Notifications: Extend existing NotificationService---@@ -225,9 +200,10 @@ Phase 20 ─────────────────────► Phas| Phase | Complexity | Files Changed ||-------|------------|---------------|-| 16 | Medium | 4-6 (new page, canvas setup) |-| 17 | Medium | 5-8 (node components) |-| 18 | Medium | 4-6 (panel components) |-| 19 | Medium | 3-5 (validation logic) |-| 20 | High | 6-8 (backend + frontend) |-| 21 | High | 5-7 (execution engine) |+| 22 | Medium | 4-6 (service, API, socket) |+| 23 | Medium | 3-5 (component, styles) |+| 24 | Medium | 2-4 (component updates) |+| 25 | High | 5-7 (model, service, API, job) |+| 26 | Medium | 3-5 (component, styles) |+| 27 | High | 5-7 (models, service, API) |+| 28 | Medium | 3-5 (service updates) |diff --git a/.planning/STATE.md b/.planning/STATE.mdindex a2a499b6..1ab7bc9c 100644--- a/.planning/STATE.md+++ b/.planning/STATE.md@@ -1,164 +1,88 @@# ServerKit Enhancement - State## Current Status-- **Milestone**: 3 - Visual Workflow Builder (COMPLETED)-- **Current Phase**: 21 - Workflow Deployment (Final)-- **Phase Status**: Completed+- **Milestone**: 4 - Container Logs & Monitoring+- **Current Phase**: 22 - Container Logs API+- **Phase Status**: Not Started- **Last Updated**: 2026-01-19## Phase Progress| Phase | Status | Started | Completed ||-------|--------|---------|-----------|-| 16 | completed | 2026-01-19 | 2026-01-19 |-| 17 | completed | 2026-01-19 | 2026-01-19 |-| 18 | completed | 2026-01-19 | 2026-01-19 |-| 19 | completed | 2026-01-19 | 2026-01-19 |-| 20 | completed | 2026-01-19 | 2026-01-19 |-| 21 | completed | 2026-01-19 | 2026-01-19 |--## Phase 21 Summary-Workflow Deployment completed:-- Created WorkflowService with deploy_workflow(), get_deployment_order()-- Deployment order: databases → apps → domains-- deploy_database() creates MySQL/PostgreSQL with users-- deploy_docker_app() creates Application + docker-compose files-- deploy_domain() creates Domain linked to app with nginx config-- Added POST /workflows/{id}/deploy API endpoint-- Added deployWorkflow() frontend API method-- Added Deploy button to toolbar (green, requires saved workflow)-- Created DeploymentProgressModal with per-node status-- Updates nodes with created resource IDs after deployment--## Phase 20 Summary-Workflow Save/Load completed:-- Created Workflow SQLAlchemy model with nodes/edges/viewport as JSON-- Created CRUD API endpoints (GET/POST/PUT/DELETE /workflows)-- Registered workflows blueprint at /api/v1/workflows-- Added frontend API methods (getWorkflows, createWorkflow, updateWorkflow, deleteWorkflow)-- Added workflow toolbar with name input, New/Load/Import/Save buttons-- Created WorkflowListModal for loading saved workflows with delete capability-- Added import existing infrastructure feature (apps + domains as nodes)--## Phase 19 Summary-Connection Logic completed:-- Created connection validation utility with rules matrix-- Wired isValidConnection callback preventing invalid connections-- Added error toast for invalid connection attempts-- Created custom ConnectionEdge with labels and delete button-- Added edge selection and deletion (Delete/Backspace keys)-- Comprehensive connection styles--## Phase 18 Summary-Node Configuration Panels completed:-- Created ConfigPanel base component with slide animation-- Created DockerAppConfigPanel with ports list-- Created DatabaseConfigPanel with type-aware defaults-- Created DomainConfigPanel with SSL/DNS fields-- Created ServiceConfigPanel with service type selector-- Wired panels to node selection in WorkflowBuilder-- Added comprehensive panel styles--## Phase 17 Summary-Resource Node Types completed:-- Created DockerAppNode with status, ports, memory display-- Created DatabaseNode with type badges (MySQL, PostgreSQL, MongoDB, Redis)-- Created DomainNode with SSL/DNS status badges-- Created ServiceNode for future extensibility-- Enhanced palette with Compute/Storage/Network categories-- Added comprehensive node-specific styles with animations--## Phase 16 Summary-Workflow Canvas Foundation completed:-- Installed @xyflow/react (React Flow) library-- Created WorkflowBuilder page with canvas, controls, minimap-- Added /workflow route and sidebar navigation-- Created BaseNode component with handles and styling-- Added node toolbar for adding Docker/Database/Domain nodes-- Comprehensive LESS styles for workflow UI--## Milestone 3 Overview-Visual Workflow Builder (v1.2)--**Goal**: Create a visual, flow-based interface for orchestrating Docker apps, databases, and domain connections using a node-based editor (similar to n8n, Node-RED).+| 22 | pending | - | - |+| 23 | pending | - | - |+| 24 | pending | - | - |+| 25 | pending | - | - |+| 26 | pending | - | - |+| 27 | pending | - | - |+| 28 | pending | - | - |++## Milestone 4 Overview+Container Logs & Monitoring (v1.3)++**Goal**: Add comprehensive container logging and monitoring capabilities - real-time log streaming, search/filtering, per-app resource graphs, and configurable alert rules with notifications.**Key Features**:-- Canvas with pan/zoom for visual orchestration-- Node types for Docker apps, databases, domains-- Connection system showing relationships-- Save/load workflows-- Deploy infrastructure from visual workflow+- Real-time container log streaming via WebSocket+- Log search with filtering by level, time, regex+- Per-app CPU/memory/network metrics collection+- Resource usage graphs in application detail+- Configurable alert rules with thresholds+- Multi-channel notifications (email, Discord, Telegram)++## Phase Descriptions++### Phase 22: Container Logs API+Backend API to fetch and stream Docker container logs++### Phase 23: Log Viewer UI+Real-time log display with auto-scroll, ANSI color support++### Phase 24: Log Search & Filtering+Search within logs, filter by level/time, export++### Phase 25: Per-App Resource Collection+Collect per-container CPU/memory/network stats via Docker API++### Phase 26: App Resource Graphs+Per-app resource usage charts in application detail view++### Phase 27: Alert Rules Engine+Define thresholds, evaluate metrics, trigger alerts++### Phase 28: Alert Notifications+Send alerts via email, Discord, Telegram webhooks## Previous Milestone Summary-### Milestone 2 (v1.1) - Completed-- Phase 11: WordPress External DB template-- Phase 12: Multi-environment app linking API-- Phase 13: Environment switching UI-- Phase 14: GitHub & community links-- Phase 15: Documentation & testing--## Phase 15 Summary-WordPress Dev Workflow Testing (documentation phase):-- Created TEST-PLAN.md with 8-step workflow validation procedure-- Created docs/MULTI_ENVIRONMENT.md user guide for the feature-- Updated docs/README.md with multi-environment section-- All documentation and testing artifacts in place--## Phase 13 Summary-Environment Switching UI implemented:-- Added environment badges (PROD/DEV/STAGING) to app list with color coding-- Added environment filter dropdown to apps toolbar with URL param persistence-- Created LinkedAppsSection component showing linked apps with navigation/unlink-- Created LinkAppModal for linking apps with environment selection and DB credential options-- Integrated linked apps into ApplicationDetail Overview tab-- Added environment settings to Settings tab with change/unlink functionality--## Phase 12 Summary-Multi-Environment App Linking implemented:-- Added `environment_type`, `linked_app_id`, `shared_config` fields to Application model-- Added app linking API endpoints (POST/GET/DELETE `/apps/{id}/link`, PUT `/apps/{id}/environment`)-- Added `propagate_db_credentials()` for sharing DB credentials with different table prefixes-- Added frontend API methods (`linkApp`, `getLinkedApps`, `unlinkApp`, `updateAppEnvironment`)-- Updated GET /apps with environment filtering and include_linked option--## Phase 11 Summary-Shared Database WordPress Template implemented:-- Created `wordpress-external-db.yaml` template for external MySQL connections-- Added `validate_mysql_connection()` with socket and pymysql validation-- Added `/templates/test-db-connection` API endpoint-- Added `testDatabaseConnection()` frontend API method-- Enables dev/prod WordPress instances sharing same database with different table prefixes--## Phase 14 Summary-GitHub & Community Links fully implemented:-- Updated About section with correct GitHub repository links-- Added version update check via GitHub releases API (1-hour cache)-- Added GitHub link to sidebar-- Added dismissible "Star on GitHub" prompt for new users-- All icons now use Lucide React components--## Milestone 2 Overview-Multi-Environment WordPress & Community Features (v1.1)--**Goals:**-- Create WordPress template with external database support-- Enable prod/dev environment linking with shared database-- Add environment switching UI with theme sync-- Add GitHub and community resource links+### Milestone 3 (v1.2) - Completed 2026-01-19+Visual Workflow Builder:+- Phase 16: Workflow Canvas Foundation+- Phase 17: Resource Node Types+- Phase 18: Node Configuration Panels+- Phase 19: Connection Logic+- Phase 20: Workflow Save/Load+- Phase 21: Deploy from Workflow++### Milestone 2 (v1.1) - Completed 2026-01-19+Multi-Environment WordPress & Community Features:+- Phase 11-15: WordPress external DB, app linking, environment UI, GitHub links++### Milestone 1 (v1.0) - Completed 2026-01-19+Infrastructure Fixes, Private URLs & UI Polish:+- Phases 1-10: Docker/domain debugging, templates, UI polish## BlockersNone currently identified.## Notes-- Milestone 1 (v1.0) completed 2026-01-19-- Milestone 2 (v1.1) completed 2026-01-19-- Milestone 3 (v1.2) started 2026-01-19-- React Flow recommended for canvas implementation-- This milestone is UI-focused - reuses existing backend APIs+- Docker SDK for Python will be used for container logs/stats+- Existing WebSocket infrastructure (Flask-SocketIO) will be extended+- Recharts already installed for graphs+- Existing NotificationService will be extended for alerts## Quick Links- [ROADMAP.md](./ROADMAP.md)- [PROJECT.md](./PROJECT.md)-- [Milestone 1 Archive](./milestones/v1.0-ROADMAP.md)-- [Milestone 2 Phases](./phases/) (11-15)+- [Milestone 3 Archive](./milestones/v1.2-ROADMAP.md)