Skip to content

ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.

docs(22): Complete Container Logs API phase

Juan Denis33fd67579f60
diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md
index acfb58ea..aa597c54 100644
--- a/.planning/ROADMAP.md
+++ b/.planning/ROADMAP.md
@@ -30,7 +30,7 @@ Completed 2026-01-19 | 6 phases | React Flow canvas, node types, deploy from wor
| Phase | Name | Status | Goal |
|-------|------|--------|------|
-| 22 | Container Logs API | pending | Backend API to fetch/stream Docker container logs |
+| 22 | Container Logs API | **completed** | Backend API to fetch/stream Docker container logs |
| 23 | Log Viewer UI | pending | Real-time log display with auto-scroll, ANSI color support |
| 24 | Log Search & Filtering | pending | Search within logs, filter by level/time, export |
| 25 | Per-App Resource Collection | pending | Collect per-container CPU/memory/network stats via Docker API |
@@ -40,7 +40,7 @@ Completed 2026-01-19 | 6 phases | React Flow canvas, node types, deploy from wor
---
-## Phase 22: Container Logs API
+## Phase 22: Container Logs API (Completed)
**Goal**: Create backend API to fetch and stream Docker container logs
**Key Tasks**:
@@ -51,10 +51,10 @@ Completed 2026-01-19 | 6 phases | React Flow canvas, node types, deploy from wor
- Support multiple output formats (raw, JSON)
**Success Criteria**:
-- [ ] Can fetch last N lines of container logs
-- [ ] Can stream logs in real-time via WebSocket
-- [ ] Logs include timestamps
-- [ ] Handles containers that aren't running
+- [x] Can fetch last N lines of container logs
+- [x] Can stream logs in real-time via WebSocket
+- [x] Logs include timestamps
+- [x] Handles containers that aren't running
---
diff --git a/.planning/STATE.md b/.planning/STATE.md
index 1ab7bc9c..a6773995 100644
--- a/.planning/STATE.md
+++ b/.planning/STATE.md
@@ -2,7 +2,7 @@
## Current Status
- **Milestone**: 4 - Container Logs & Monitoring
-- **Current Phase**: 22 - Container Logs API
+- **Current Phase**: 23 - Log Viewer UI
- **Phase Status**: Not Started
- **Last Updated**: 2026-01-19
@@ -10,7 +10,7 @@
| Phase | Status | Started | Completed |
|-------|--------|---------|-----------|
-| 22 | pending | - | - |
+| 22 | completed | 2026-01-19 | 2026-01-19 |
| 23 | pending | - | - |
| 24 | pending | - | - |
| 25 | pending | - | - |
@@ -77,11 +77,14 @@ Infrastructure Fixes, Private URLs & UI Polish:
None currently identified.
## Notes
-- Docker SDK for Python will be used for container logs/stats
-- Existing WebSocket infrastructure (Flask-SocketIO) will be extended
+- Using subprocess for Docker CLI commands (not Docker SDK)
+- WebSocket infrastructure extended with container log streaming
- Recharts already installed for graphs
- Existing NotificationService will be extended for alerts
+## Session Log
+- 2026-01-19: Phase 22 completed - Container Logs API (3 commits)
+
## Quick Links
- [ROADMAP.md](./ROADMAP.md)
- [PROJECT.md](./PROJECT.md)
diff --git a/.planning/phases/22-container-logs-api/PLAN.md b/.planning/phases/22-container-logs-api/PLAN.md
new file mode 100644
index 00000000..4f45d192
--- /dev/null
+++ b/.planning/phases/22-container-logs-api/PLAN.md
@@ -0,0 +1,318 @@
+# Phase 22: Container Logs API - Execution Plan
+
+## Goal
+Create backend API to fetch and stream Docker container logs for applications, enabling real-time log viewing in the UI (Phase 23).
+
+## Prerequisites
+- Docker apps exist with `container_id` or can be resolved via `root_path`
+- WebSocket infrastructure exists (Flask-SocketIO with threading async_mode)
+- Existing `DockerService.get_container_logs()` method available
+
+## Architecture
+
+### Existing Infrastructure
+
+**DockerService** (`backend/app/services/docker_service.py:237-254`):
+```python
+@staticmethod
+def get_container_logs(container_id, tail=100, since=None, timestamps=True):
+ """Get container logs."""
+ cmd = ['docker', 'logs']
+ if tail:
+ cmd.extend(['--tail', str(tail)])
+ if since:
+ cmd.extend(['--since', since])
+ if timestamps:
+ cmd.append('-t')
+ cmd.append(container_id)
+ # Returns {'success': True, 'logs': logs} or {'success': False, 'error': str}
+```
+
+**WebSocket Pattern** (`backend/app/sockets.py`):
+- Uses room-based streaming: `join_room(f'build_{app_id}')`
+- Build logs emit via `emit_build_log(app_id, message, level)`
+- Clients subscribe via `subscribe_build` event with `app_id`
+
+**Application Model** (`backend/app/models/application.py`):
+- `container_id`: String(100), nullable - stores Docker container ID
+- `root_path`: String(500) - path to docker-compose directory
+- For compose apps: container names follow `serverkit-app-{app.id}` pattern
+
+### Log Streaming Flow
+```
+1. Client connects to WebSocket with JWT token
+2. Client emits 'subscribe_container_logs' with app_id
+3. Server joins room 'logs_{app_id}'
+4. Server starts background thread to stream logs via `docker logs -f`
+5. Server emits 'container_log' events to room
+6. Client emits 'unsubscribe_container_logs' or disconnects
+7. Server stops background thread and leaves room
+```
+
+## Tasks
+
+### Task 1: Extend DockerService with Streaming Logs
+**File**: `backend/app/services/docker_service.py`
+
+Add method for streaming container logs:
+```python
+@staticmethod
+def stream_container_logs(container_id, callback, tail=100, since=None, timestamps=True):
+ """Stream container logs in real-time.
+
+ Args:
+ container_id: Docker container ID or name
+ callback: Function to call with each log line
+ tail: Number of existing lines to fetch first
+ since: Only logs since this timestamp
+ timestamps: Include timestamps in output
+
+ Returns:
+ subprocess.Popen object for the streaming process
+ """
+ cmd = ['docker', 'logs', '--follow']
+ if tail:
+ cmd.extend(['--tail', str(tail)])
+ if since:
+ cmd.extend(['--since', since])
+ if timestamps:
+ cmd.append('-t')
+ cmd.append(container_id)
+
+ process = subprocess.Popen(
+ cmd,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ text=True,
+ bufsize=1
+ )
+ return process
+```
+
+### Task 2: Create Container Logs REST API Endpoint
+**File**: `backend/app/api/apps.py`
+
+Extend existing `/apps/<id>/logs` endpoint to support container logs with more options:
+```python
+@apps_bp.route('/<int:app_id>/container-logs', methods=['GET'])
+@jwt_required()
+def get_container_logs(app_id):
+ """Get container logs for a Docker application.
+
+ Query params:
+ - tail: Number of lines from end (default: 100, max: 10000)
+ - since: ISO timestamp or duration (e.g., '10m', '1h', '2024-01-01T00:00:00Z')
+ - timestamps: Include timestamps (default: true)
+ - format: Output format - 'raw' or 'json' (default: 'raw')
+
+ Returns:
+ {
+ "success": true,
+ "logs": "...", // Raw format
+ "lines": [...] // JSON format with parsed lines
+ "container_id": "...",
+ "container_name": "...",
+ "app_id": 1
+ }
+ """
+```
+
+### Task 3: Add Container Logs WebSocket Events
+**File**: `backend/app/sockets.py`
+
+Add WebSocket handlers for real-time log streaming:
+```python
+# Store active log streams
+container_log_streams = {} # sid -> {'process': Popen, 'app_id': int, 'thread': Thread}
+
[email protected]('subscribe_container_logs')
+def handle_subscribe_container_logs(data):
+ """Subscribe to real-time container log streaming.
+
+ data: {
+ 'app_id': int,
+ 'tail': int (optional, default 100),
+ 'since': str (optional)
+ }
+ """
+ sid = request.sid
+ app_id = data.get('app_id')
+ tail = data.get('tail', 100)
+ since = data.get('since')
+
+ # Validate token and app access (already done in connect)
+ # Get container ID from app
+ # Start streaming thread
+ # Join room for this app's logs
+ join_room(f'logs_{app_id}')
+
[email protected]('unsubscribe_container_logs')
+def handle_unsubscribe_container_logs():
+ """Unsubscribe from container log streaming."""
+ sid = request.sid
+ # Stop streaming process
+ # Leave room
+ # Clean up
+
+def emit_container_log(app_id: int, line: str, timestamp: str = None):
+ """Emit a container log line to all subscribers."""
+ socketio.emit('container_log', {
+ 'app_id': app_id,
+ 'line': line,
+ 'timestamp': timestamp or time.time()
+ }, room=f'logs_{app_id}')
+```
+
+### Task 4: Handle Container Resolution for Compose Apps
+**File**: `backend/app/services/docker_service.py`
+
+Add method to get container name/ID for an app:
+```python
+@staticmethod
+def get_app_container_id(app):
+ """Get the main container ID for an application.
+
+ For apps with container_id set, use that directly.
+ For compose apps, query docker compose ps to find container.
+
+ Args:
+ app: Application model instance
+
+ Returns:
+ str: Container ID or name, or None if not found
+ """
+ if app.container_id:
+ return app.container_id
+
+ if app.root_path:
+ # Get containers from docker compose
+ containers = DockerService.compose_ps(app.root_path)
+ if containers:
+ # Return first container (main service)
+ # For compose apps, container name is typically service name
+ return containers[0].get('ID') or containers[0].get('Name')
+
+ return None
+
+@staticmethod
+def get_app_container_name(app_id):
+ """Get container name by convention: serverkit-app-{app_id}"""
+ return f"serverkit-app-{app_id}"
+```
+
+### Task 5: Add Log Line Parsing Utility
+**File**: `backend/app/services/docker_service.py`
+
+Add utility to parse Docker log lines:
+```python
+@staticmethod
+def parse_log_line(line):
+ """Parse a Docker log line into structured format.
+
+ Docker logs with timestamps look like:
+ 2024-01-15T10:30:45.123456789Z Log message here
+
+ Returns:
+ {
+ 'timestamp': '2024-01-15T10:30:45.123456789Z',
+ 'message': 'Log message here',
+ 'level': 'info' # Detected from content
+ }
+ """
+ # Parse timestamp if present
+ # Detect log level from message content (ERROR, WARN, INFO, DEBUG)
+ # Return structured object
+```
+
+### Task 6: Handle Non-Running Container Edge Cases
+**File**: `backend/app/api/apps.py` and `backend/app/sockets.py`
+
+Handle edge cases gracefully:
+- Container doesn't exist: Return appropriate error message
+- Container is stopped: Return existing logs (no streaming)
+- Container is restarting: Handle reconnection
+- Multiple containers (compose): Allow selecting specific service
+
+```python
+# In REST endpoint
+if not container_id:
+ return jsonify({
+ 'success': False,
+ 'error': 'No container found for this application',
+ 'hint': 'The application may not have been started yet'
+ }), 404
+
+# Check if container is running for streaming
+container_info = DockerService.get_container(container_id)
+if not container_info:
+ return jsonify({
+ 'success': False,
+ 'error': 'Container not found'
+ }), 404
+
+state = container_info.get('State', {}).get('Status', 'unknown')
+```
+
+### Task 7: Add API Documentation
+**File**: Update API with docstrings and OpenAPI-style docs
+
+Document the new endpoints:
+- `GET /apps/{id}/container-logs` - Fetch historical logs
+- WebSocket `subscribe_container_logs` - Stream real-time logs
+- WebSocket `unsubscribe_container_logs` - Stop streaming
+- WebSocket `container_log` event - Log line event
+
+## Success Criteria
+- [ ] Can fetch last N lines of container logs via REST API
+- [ ] Can stream logs in real-time via WebSocket
+- [ ] Logs include timestamps
+- [ ] Handles containers that aren't running (returns stored logs or error)
+- [ ] Handles compose apps (multiple containers)
+- [ ] JSON format option for structured log data
+- [ ] Proper cleanup when client disconnects
+
+## Files to Create/Modify
+
+| File | Action | Purpose |
+|------|--------|---------|
+| `backend/app/services/docker_service.py` | Modify | Add streaming logs, container resolution, log parsing |
+| `backend/app/api/apps.py` | Modify | Add `/container-logs` endpoint |
+| `backend/app/sockets.py` | Modify | Add container log streaming WebSocket events |
+
+## Technical Notes
+
+### Docker Logs Command Reference
+```bash
+# Basic logs
+docker logs <container>
+
+# With options
+docker logs --tail 100 --since "10m" -t --follow <container>
+
+# Options:
+# --tail N Show last N lines
+# --since TIME Show logs since timestamp (RFC3339) or duration (10m, 1h)
+# -t, --timestamps Show timestamps
+# -f, --follow Stream new logs
+```
+
+### WebSocket Room Naming Convention
+- Build logs: `build_{app_id}`
+- Container logs: `logs_{app_id}`
+- Metrics: broadcast to all subscribers
+
+### Log Buffer Limits
+- REST API: Max 10,000 lines per request
+- WebSocket: No buffer limit (streaming)
+- Frontend (Phase 23): Should implement virtual scrolling for large log volumes
+
+### ANSI Color Codes
+Docker logs may contain ANSI color codes. These will be preserved in the output and handled by the frontend (Phase 23) using an ANSI parser.
+
+## Estimated Commits
+- feat(22-1): Add streaming logs method to DockerService
+- feat(22-2): Add container-logs REST endpoint
+- feat(22-3): Add WebSocket container log streaming
+- feat(22-4): Add container resolution for compose apps
+- feat(22-5): Add log parsing utility
+- feat(22-6): Handle edge cases for non-running containers
diff --git a/.planning/phases/22-container-logs-api/SUMMARY.md b/.planning/phases/22-container-logs-api/SUMMARY.md
new file mode 100644
index 00000000..de907fe2
--- /dev/null
+++ b/.planning/phases/22-container-logs-api/SUMMARY.md
@@ -0,0 +1,133 @@
+# Phase 22: Container Logs API - Summary
+
+## Overview
+Created backend API and WebSocket infrastructure to fetch and stream Docker container logs for applications, enabling real-time log viewing in the UI.
+
+## Completed Tasks
+
+### Task 1: DockerService Streaming Extensions
+**Commit**: `f3010d9`
+
+Added methods to DockerService for container log operations:
+- `stream_container_logs()` - Start streaming logs via subprocess with `docker logs --follow`
+- `get_app_container_id()` - Resolve container ID from Application model (handles both direct container_id and compose apps)
+- `get_all_app_containers()` - List all containers for compose apps
+- `parse_log_line()` - Parse Docker log line to extract timestamp and detect level (error/warn/info/debug)
+- `parse_logs_to_lines()` - Batch parse logs text into structured lines
+- `get_container_state()` - Check if container is running, get status info
+
+### Task 2: REST API Endpoints
+**Commit**: `744c626`
+
+Added endpoints to `backend/app/api/apps.py`:
+
+**GET /apps/{id}/container-logs**
+- Query params: `tail`, `since`, `timestamps`, `format`, `service`
+- Returns logs in 'raw' or 'json' format
+- Includes container state and available services
+- Caps at 10,000 lines for performance
+
+**GET /apps/{id}/containers**
+- Lists all containers for a compose application
+- Returns container id, name, service, state
+
+### Task 3: WebSocket Real-Time Streaming
+**Commit**: `f55e399`
+
+Added WebSocket events to `backend/app/sockets.py`:
+
+**subscribe_container_logs** event
+- Input: `{app_id, tail?, since?, service?}`
+- Starts background thread streaming via `docker logs --follow`
+- Joins room `logs_{app_id}`
+- Emits `container_log` events with parsed log lines
+
+**unsubscribe_container_logs** event
+- Stops streaming subprocess
+- Leaves room
+- Cleans up resources
+
+**Events emitted:**
+- `container_log` - Log line with parsed timestamp/level
+- `container_log_ended` - When container stops
+- `container_log_error` - On stream failure
+
+### Edge Cases Handled
+- Container not found: Returns helpful error with hint
+- Container stopped: REST returns existing logs, WebSocket emits ended event
+- Compose apps: Service selection via `service` parameter
+- Process cleanup: Terminates subprocess on disconnect/unsubscribe
+- Large logs: Capped at 10,000 lines for REST API
+
+## Files Modified
+
+| File | Changes |
+|------|---------|
+| `backend/app/services/docker_service.py` | +185 lines: streaming, parsing, container resolution |
+| `backend/app/api/apps.py` | +151 lines: container-logs and containers endpoints |
+| `backend/app/sockets.py` | +227 lines: WebSocket container log streaming |
+
+## API Reference
+
+### REST Endpoints
+
+```
+GET /api/v1/apps/{id}/container-logs
+ ?tail=100 # Lines from end (max 10000)
+ ?since=10m # Duration or ISO timestamp
+ ?timestamps=true # Include timestamps
+ ?format=raw # 'raw' or 'json'
+ ?service=web # Specific service for compose apps
+
+GET /api/v1/apps/{id}/containers
+ # List containers for compose apps
+```
+
+### WebSocket Events
+
+```javascript
+// Subscribe to logs
+socket.emit('subscribe_container_logs', {
+ app_id: 1,
+ tail: 100,
+ since: '10m',
+ service: 'web' // optional
+});
+
+// Receive logs
+socket.on('container_log', (data) => {
+ // data: {app_id, line, parsed: {timestamp, message, level}, timestamp}
+});
+
+socket.on('container_log_ended', (data) => {
+ // Container stopped
+});
+
+socket.on('container_log_error', (data) => {
+ // Stream error
+});
+
+// Unsubscribe
+socket.emit('unsubscribe_container_logs');
+```
+
+## Testing Notes
+
+- Tested against Docker compose applications
+- Log streaming continues until container stops or client disconnects
+- ANSI color codes preserved in output (frontend will parse)
+- Timestamps parsed from Docker RFC3339 format
+
+## Next Phase
+
+Phase 23: Log Viewer UI
+- Create LogViewer component with terminal-style display
+- Implement auto-scroll with pause on user scroll
+- Add ANSI color code parsing
+- Connect to WebSocket for live streaming
+
+## Commits
+
+1. `f3010d9` - feat(22-1): Add streaming logs and container resolution to DockerService
+2. `744c626` - feat(22-2): Add container-logs REST API endpoint
+3. `f55e399` - feat(22-3): Add WebSocket container log streaming