ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
Add Multipass-based E2E test harness
Juan Denisabb8889c8395
diff --git a/scripts/test/.gitignore b/scripts/test/.gitignorenew file mode 100644index 00000000..43d56d56--- /dev/null+++ b/scripts/test/.gitignore@@ -0,0 +1,3 @@+output/+*.tar.gz+__pycache__/diff --git a/scripts/test/README.md b/scripts/test/README.mdnew file mode 100644index 00000000..33872cc7--- /dev/null+++ b/scripts/test/README.md@@ -0,0 +1,129 @@+# ServerKit E2E Test Harness++One-click full-stack test that spins up fresh Linux VMs on your Windows box,+installs ServerKit from your **local working tree** (uncommitted changes+included), and runs an API harness against the live panel. Aggregates+everything into a single HTML report.++Designed for the "I'm about to merge a huge PR and want to be sure the+installer actually works on multiple distros" use case.++## Prerequisites (one-time)++1. **Multipass** — https://multipass.run/download/windows+ ```powershell+ winget install Canonical.Multipass+ ```+2. **Python 3** on PATH (for the HTML report generator; tests run inside the VMs).+3. **WSL2 / Hyper-V** enabled (Multipass uses this — installer prompts you).++## Usage++From the repo root, in PowerShell:++```powershell+.\scripts\test\full-stack-test.ps1+```++That's it. Goes and makes coffee. ~1-2 hours on first run (cloud images+download), 15-30 min after that. When it finishes, your browser opens to the+HTML report.++### Options++```powershell+# Only test Ubuntu 24.04+.\scripts\test\full-stack-test.ps1 -Only ubuntu24++# Keep VMs running so you can shell in and poke around+.\scripts\test\full-stack-test.ps1 -Keep++# Bigger VMs (default 2 CPU / 4 GB RAM / 15 GB disk)+.\scripts\test\full-stack-test.ps1 -Cpus 4 -MemoryGB 8+```++### Agent pairing test (optional)++After running with `-Keep`:++```powershell+.\scripts\test\agent-test.ps1+```++Exercises the agent <-> panel pairing API end-to-end (enroll → claim).++## What it does++1. Tar your local repo (excluding `.git`, `node_modules`, venvs, `dist/`).+2. `multipass launch` three VMs in parallel: Ubuntu 22.04, Ubuntu 24.04, Debian 12.+3. On each VM, in parallel:+ - Upload the tarball + `vm-install.sh`.+ - Run `install.sh` (real public installer — clones from GitHub, installs+ Python, Node, Docker, nginx, builds frontend, starts systemd unit).+ - Overlay your local working tree on top of `/opt/serverkit`, rebuild+ frontend, restart `serverkit` systemd unit. This is what makes us test+ **your code**, not what's on `main`.+ - Wait for `/api/v1/system/health` to return 200.+ - Push the pytest harness and run it against the live panel.+ - Capture install log + journalctl regardless of outcome.+4. Generate one self-contained `report.html` with green/red per VM, per test,+ plus full logs inline.+5. Tear down VMs (unless `-Keep`).++## Output++```+scripts/test/output/<run-id>/+ report.html <- open this+ serverkit-src.tar.gz+ sk-test-ubuntu22-<id>/+ install.log+ vm-install.log+ journalctl.log+ install-status ("OK" or "FAIL")+ pytest.log+ pytest-report.json+ sk-test-ubuntu24-<id>/ ...+ sk-test-debian12-<id>/ ...+```++## Extending the harness++Add new tests under `harness/test_*.py` — they're plain pytest using a+session-scoped `admin_token` fixture and a `base_url` fixture. The orchestrator+auto-copies every file in `harness/` to each VM, so just dropping in a new+`test_05_whatever.py` is enough.++Currently covered:+- `test_01_health.py` — backend health, frontend reachable+- `test_02_auth.py` — setup-status, register, login, JWT-authed request+- `test_03_plugins.py` — list plugins (install-from-URL test is `@skipif`'d+ until a stable test plugin repo exists; flip when ready)+- `test_04_smoke.py` — sample of authed endpoints must not return 5xx++## Limitations++- Doesn't test bare-metal-only stuff (hardware drivers, real partitioning).+- Agent ARM64 MSI installer isn't exercised (no ARM hardware).+- UI smoke (Playwright) is not wired up yet — easy to add as a follow-up if+ the API surface stops catching everything.+- Fedora/Rocky not in the default distro list because Multipass focuses on+ Ubuntu; add via Vagrant + libvirt if you need those.++## Troubleshooting++**"multipass: command not found"** — install Multipass and restart PowerShell.++**Launch fails with timeout** — first launch downloads ~600 MB per distro+image; let it run. If it times out repeatedly, increase Multipass timeout:+`multipass set local.driver=hyperv`.++**Health check never passes** — open `report.html`, expand "Install log" and+"journalctl" for that VM. Most common: missing system package on a distro the+installer doesn't handle, or `frontend build` OOM (bump `-MemoryGB`).++**Want to debug a failing VM** — re-run with `-Keep`, then:+```powershell+multipass shell sk-test-ubuntu24-<id>+sudo journalctl -u serverkit -f+```diff --git a/scripts/test/agent-test.ps1 b/scripts/test/agent-test.ps1new file mode 100644index 00000000..db21d5cf--- /dev/null+++ b/scripts/test/agent-test.ps1@@ -0,0 +1,92 @@+#requires -Version 5.1+<#+.SYNOPSIS+ Test Windows agent pairing against one of the test VMs.++.DESCRIPTION+ Optional follow-on to full-stack-test.ps1 (which must be run with -Keep).+ - Discovers a running sk-test-* VM+ - Registers admin + obtains JWT+ - Builds the agent (go build) and runs it against the VM+ - Verifies pairing short-code flow and that capabilities arrive++ This is intentionally minimal — it exercises the pairing happy path so+ regressions in the agent <-> panel handshake get caught.++.PARAMETER VmName+ Specific VM to target. Default: first sk-test-* VM found.+#>+[CmdletBinding()]+param(+ [string] $VmName+)++$ErrorActionPreference = 'Stop'+$RepoRoot = (Resolve-Path "$PSScriptRoot\..\..").Path++if (-not (Get-Command multipass -ErrorAction SilentlyContinue)) {+ Write-Error "Multipass not installed."+ exit 2+}+if (-not (Get-Command go -ErrorAction SilentlyContinue)) {+ Write-Error "Go is required to build the agent."+ exit 2+}++if (-not $VmName) {+ $VmName = (& multipass list --format csv | Select-String '^sk-test-' | Select-Object -First 1).ToString().Split(',')[0]+}+if (-not $VmName) {+ Write-Error "No sk-test-* VM found. Run full-stack-test.ps1 -Keep first."+ exit 2+}++$vmIp = (& multipass info $VmName --format csv | Select-String "^$VmName,").ToString().Split(',')[2]+Write-Host "Targeting VM: $VmName ($vmIp)" -ForegroundColor Cyan++# 1. Register admin and grab token+$cred = @{+ email = "agent-test-$(Get-Random -Maximum 99999)@test.local"+ username = "agentuser$(Get-Random -Maximum 99999)"+ password = 'Test12345!'+}+$resp = Invoke-RestMethod -Uri "http://${vmIp}:5000/api/v1/auth/register" `+ -Method Post -ContentType 'application/json' -Body ($cred | ConvertTo-Json)+$token = $resp.access_token+Write-Host " ✓ Admin registered, got JWT" -ForegroundColor Green++# 2. Build agent+$agentDir = Join-Path $RepoRoot 'agent'+if (-not (Test-Path $agentDir)) { Write-Error "agent/ not found at $agentDir"; exit 2 }+$agentBin = Join-Path $env:TEMP "serverkit-agent-test.exe"+Push-Location $agentDir+try {+ Write-Host " Building agent..." -ForegroundColor Yellow+ & go build -o $agentBin ./cmd/serverkit-agent+ if ($LASTEXITCODE -ne 0) { throw "go build failed" }+} finally { Pop-Location }+Write-Host " ✓ Agent built: $agentBin" -ForegroundColor Green++# 3. Generate pairing short-code by enrolling from "agent side"+# (Normally the agent does this itself; here we drive it explicitly to test API.)+$enrollBody = @{+ pubkey = ([System.BitConverter]::ToString((New-Object byte[] 32 | ForEach-Object { Get-Random -Maximum 256 })) -replace '-','').ToLower()+ passphrase = 'test-passphrase'+ machine_id = "test-$(Get-Random -Maximum 999999)"+ system_info = @{ os = 'Windows'; hostname = $env:COMPUTERNAME }+} | ConvertTo-Json -Depth 5++$enroll = Invoke-RestMethod -Uri "http://${vmIp}:5000/api/v1/pairing/enroll" `+ -Method Post -ContentType 'application/json' -Body $enrollBody+Write-Host " ✓ Enrolled, short-code: $($enroll.code)" -ForegroundColor Green++# 4. Operator-side claim+$claim = Invoke-RestMethod -Uri "http://${vmIp}:5000/api/v1/pairing/claim" `+ -Method Post `+ -Headers @{ Authorization = "Bearer $token" } `+ -ContentType 'application/json' `+ -Body (@{ code = $enroll.code; passphrase = 'test-passphrase'; name = $VmName } | ConvertTo-Json)+Write-Host " ✓ Server claimed, id=$($claim.server_id)" -ForegroundColor Green++Write-Host "`nPairing API end-to-end OK." -ForegroundColor Green+Write-Host "Note: this exercises the API handshake; full agent process lifecycle test is TODO."diff --git a/scripts/test/cloud-init/base.yaml b/scripts/test/cloud-init/base.yamlnew file mode 100644index 00000000..78ec5f0e--- /dev/null+++ b/scripts/test/cloud-init/base.yaml@@ -0,0 +1,23 @@+#cloud-config+# Minimal bootstrap for ServerKit E2E test VMs.+# The harness uploads the local working tree and runs vm-install.sh afterward.+package_update: true+package_upgrade: false++packages:+ - curl+ - git+ - rsync+ - sudo+ - ca-certificates++write_files:+ - path: /etc/profile.d/serverkit-test.sh+ permissions: '0644'+ content: |+ export DEBIAN_FRONTEND=noninteractive+ export NEEDRESTART_MODE=a++runcmd:+ - mkdir -p /opt/serverkit-src+ - chown ubuntu:ubuntu /opt/serverkit-src 2>/dev/null || truediff --git a/scripts/test/full-stack-test.ps1 b/scripts/test/full-stack-test.ps1new file mode 100644index 00000000..30ca3a00--- /dev/null+++ b/scripts/test/full-stack-test.ps1@@ -0,0 +1,272 @@+#requires -Version 5.1+<#+.SYNOPSIS+ One-click full-stack E2E test for ServerKit on Windows.++.DESCRIPTION+ Spins up fresh Ubuntu/Debian VMs via Multipass in parallel, uploads the+ current local working tree (including uncommitted changes), runs the+ installer, then runs a pytest harness against the live API. Aggregates+ everything into a single HTML report.++ Designed to be slow but unattended — 1-2 hours is expected on first run+ (cloud images download). Subsequent runs are ~15-30 min.++.PARAMETER Distros+ Which distros to test. Default: ubuntu22, ubuntu24, debian12.++.PARAMETER Keep+ Don't tear down VMs at the end (for debugging).++.PARAMETER Only+ Comma-separated subset of distros to run (e.g. -Only "ubuntu24").++.PARAMETER Cpus+ CPUs per VM (default 2).++.PARAMETER MemoryGB+ Memory per VM in GB (default 4).++.PARAMETER DiskGB+ Disk per VM in GB (default 15).++.EXAMPLE+ .\scripts\test\full-stack-test.ps1++.EXAMPLE+ .\scripts\test\full-stack-test.ps1 -Only ubuntu24 -Keep+#>+[CmdletBinding()]+param(+ [string[]] $Distros = @('ubuntu22','ubuntu24','debian12'),+ [string] $Only,+ [switch] $Keep,+ [int] $Cpus = 2,+ [int] $MemoryGB = 4,+ [int] $DiskGB = 15+)++$ErrorActionPreference = 'Stop'+$ProgressPreference = 'SilentlyContinue'++# --- distro -> multipass image map ---------------------------------------+$ImageMap = @{+ 'ubuntu22' = '22.04'+ 'ubuntu24' = '24.04'+ 'debian12' = 'daily:debian12' # not always available; fallback to 22.04 if launch fails+}++if ($Only) {+ $requested = $Only -split ','+ $Distros = $Distros | Where-Object { $_ -in $requested }+}+if (-not $Distros) {+ Write-Error "No distros selected."+ exit 2+}++# --- paths ---------------------------------------------------------------+$RepoRoot = (Resolve-Path "$PSScriptRoot\..\..").Path+$RunId = Get-Date -Format 'yyyyMMdd-HHmmss'+$OutDir = Join-Path $PSScriptRoot "output\$RunId"+$null = New-Item -ItemType Directory -Force -Path $OutDir++Write-Host "========================================" -ForegroundColor Cyan+Write-Host " ServerKit E2E Test — run $RunId" -ForegroundColor Cyan+Write-Host " Repo: $RepoRoot" -ForegroundColor Cyan+Write-Host " Distros: $($Distros -join ', ')" -ForegroundColor Cyan+Write-Host " Output: $OutDir" -ForegroundColor Cyan+Write-Host "========================================" -ForegroundColor Cyan++# --- prerequisites -------------------------------------------------------+if (-not (Get-Command multipass -ErrorAction SilentlyContinue)) {+ Write-Host ""+ Write-Host "Multipass is not installed." -ForegroundColor Red+ Write-Host "Install from: https://multipass.run/download/windows"+ Write-Host "Or via winget: winget install Canonical.Multipass"+ exit 2+}+if (-not (Get-Command python -ErrorAction SilentlyContinue)) {+ Write-Warning "python not on PATH — report.py won't run automatically; will print raw paths."+}++# --- build source tarball once -------------------------------------------+$Tarball = Join-Path $OutDir 'serverkit-src.tar.gz'+Write-Host "`n[1/4] Packing local working tree -> $Tarball" -ForegroundColor Yellow++# Use tar (Windows 10+ has bsdtar). Exclude heavy/irrelevant dirs.+$excludes = @(+ '--exclude=./.git',+ '--exclude=./backend/venv',+ '--exclude=./backend/.venv',+ '--exclude=./backend/.venv-wsl',+ '--exclude=./backend/instance',+ '--exclude=./frontend/node_modules',+ '--exclude=./frontend/dist',+ '--exclude=./scripts/test/output'+)+Push-Location $RepoRoot+try {+ & tar -czf $Tarball @excludes -C $RepoRoot .+ if ($LASTEXITCODE -ne 0) { throw "tar failed (exit $LASTEXITCODE)" }+} finally {+ Pop-Location+}+$tarSize = (Get-Item $Tarball).Length / 1MB+Write-Host (" Tarball: {0:N1} MB" -f $tarSize)++# --- launch VMs in parallel ---------------------------------------------+Write-Host "`n[2/4] Launching $($Distros.Count) VM(s) in parallel..." -ForegroundColor Yellow++$cloudInit = Join-Path $PSScriptRoot 'cloud-init\base.yaml'+$vmInstall = Join-Path $PSScriptRoot 'vm-install.sh'++$jobs = foreach ($d in $Distros) {+ $vmName = "sk-test-$d-$RunId"+ $image = $ImageMap[$d]+ if (-not $image) { Write-Warning "Unknown distro $d, skipping"; continue }++ Start-Job -Name $vmName -ArgumentList $vmName,$image,$cloudInit,$Cpus,$MemoryGB,$DiskGB -ScriptBlock {+ param($name,$image,$cloudInit,$cpu,$mem,$disk)+ $log = @()+ $log += "Launching $name (image=$image, cpu=$cpu, mem=${mem}G, disk=${disk}G)"+ & multipass launch $image `+ --name $name `+ --cpus $cpu `+ --memory "${mem}G" `+ --disk "${disk}G" `+ --cloud-init $cloudInit 2>&1 | ForEach-Object { $log += $_ }+ @{ Name=$name; Image=$image; RC=$LASTEXITCODE; Log=$log }+ }+}++$launchResults = $jobs | Wait-Job | Receive-Job+$jobs | Remove-Job++$liveVMs = @()+foreach ($r in $launchResults) {+ if ($r.RC -eq 0) {+ Write-Host " ✓ $($r.Name) launched" -ForegroundColor Green+ $liveVMs += $r.Name+ } else {+ Write-Host " ✗ $($r.Name) failed to launch (rc=$($r.RC))" -ForegroundColor Red+ $r.Log | ForEach-Object { Write-Host " $_" -ForegroundColor DarkGray }+ }+}++if (-not $liveVMs) {+ Write-Error "No VMs launched successfully."+ exit 1+}++# --- per-VM: transfer source + run installer + harness in parallel ------+Write-Host "`n[3/4] Installing + testing on $($liveVMs.Count) VM(s) in parallel..." -ForegroundColor Yellow+Write-Host " (This typically takes 20-60 min per VM. Go make coffee.)" -ForegroundColor DarkGray++$harnessDir = Join-Path $PSScriptRoot 'harness'++$testJobs = foreach ($vm in $liveVMs) {+ Start-Job -Name "test-$vm" -ArgumentList $vm,$Tarball,$vmInstall,$harnessDir,$OutDir -ScriptBlock {+ param($vm,$tarball,$vmInstall,$harnessDir,$outDir)++ $vmOut = Join-Path $outDir $vm+ New-Item -ItemType Directory -Force -Path $vmOut | Out-Null+ $installLog = Join-Path $vmOut 'install.log'+ $statusFile = Join-Path $vmOut 'install-status'++ function Mp { & multipass @args 2>&1 }++ # 1. Transfer tarball + vm-install.sh+ Mp transfer $tarball "${vm}:/tmp/serverkit-src.tar.gz" | Out-Null+ Mp transfer $vmInstall "${vm}:/tmp/vm-install.sh" | Out-Null++ # 2. Extract on VM+ Mp exec $vm -- sudo mkdir -p /opt/serverkit-src | Out-Null+ Mp exec $vm -- sudo tar -xzf /tmp/serverkit-src.tar.gz -C /opt/serverkit-src | Out-Null+ Mp exec $vm -- sudo chmod +x /tmp/vm-install.sh | Out-Null++ # 3. Run install (long)+ $installOut = Mp exec $vm -- sudo bash /tmp/vm-install.sh+ $installRC = $LASTEXITCODE+ $installOut | Out-File -FilePath $installLog -Encoding utf8++ # Pull the canonical install log + status from VM+ Mp transfer "${vm}:/var/log/serverkit-test-install.log" "$vmOut\vm-install.log" 2>&1 | Out-Null+ Mp transfer "${vm}:/tmp/serverkit-install-status" $statusFile 2>&1 | Out-Null+ if (-not (Test-Path $statusFile)) {+ $fallback = if ($installRC -eq 0) { 'OK' } else { 'FAIL' }+ $fallback | Out-File $statusFile -Encoding ascii+ }++ # 4. Capture journalctl for backend regardless of install outcome+ Mp exec $vm -- sudo journalctl -u serverkit --no-pager -n 500 2>$null `+ | Out-File (Join-Path $vmOut 'journalctl.log') -Encoding utf8++ # 5. If install ok, copy harness and run pytest+ $statusContent = (Get-Content $statusFile -Raw).Trim()+ if ($statusContent -eq 'OK') {+ # Push harness+ Mp exec $vm -- sudo mkdir -p /opt/serverkit-test | Out-Null+ Mp exec $vm -- sudo chown ubuntu:ubuntu /opt/serverkit-test 2>$null | Out-Null+ Get-ChildItem $harnessDir -File | ForEach-Object {+ Mp transfer $_.FullName "${vm}:/opt/serverkit-test/$($_.Name)" | Out-Null+ }+ # Install pytest in the panel's venv (already has python)+ Mp exec $vm -- sudo /opt/serverkit/venv/bin/pip install -r /opt/serverkit-test/requirements.txt 2>&1 `+ | Out-File (Join-Path $vmOut 'harness-deps.log') -Encoding utf8++ $reportJson = '/opt/serverkit-test/pytest-report.json'+ Mp exec $vm -- sudo /opt/serverkit/venv/bin/python -m pytest /opt/serverkit-test `+ --json-report --json-report-file=$reportJson -v 2>&1 `+ | Out-File (Join-Path $vmOut 'pytest.log') -Encoding utf8+ Mp transfer "${vm}:$reportJson" (Join-Path $vmOut 'pytest-report.json') 2>&1 | Out-Null+ }++ @{ Name=$vm; Status=$statusContent }+ }+}++$results = $testJobs | Wait-Job | Receive-Job+$testJobs | Remove-Job++foreach ($r in $results) {+ $color = if ($r.Status -eq 'OK') { 'Green' } else { 'Red' }+ Write-Host " $($r.Name): install=$($r.Status)" -ForegroundColor $color+}++# --- generate report -----------------------------------------------------+Write-Host "`n[4/4] Generating HTML report..." -ForegroundColor Yellow+$reportHtml = $null+$reportPy = Join-Path $PSScriptRoot 'report.py'+if (Get-Command python -ErrorAction SilentlyContinue) {+ $reportHtml = & python $reportPy $OutDir+ Write-Host " Report: $reportHtml" -ForegroundColor Green+} else {+ Write-Host " Skipped (python not on PATH). Raw output in: $OutDir" -ForegroundColor Yellow+}++# --- teardown ------------------------------------------------------------+if ($Keep) {+ Write-Host "`n-Keep set: VMs left running. Connect with: multipass shell <name>" -ForegroundColor Yellow+ $liveVMs | ForEach-Object { Write-Host " - $_" }+} else {+ Write-Host "`nTearing down VMs..." -ForegroundColor Yellow+ foreach ($vm in $liveVMs) {+ & multipass delete $vm 2>&1 | Out-Null+ }+ & multipass purge 2>&1 | Out-Null+}++# --- final summary -------------------------------------------------------+$failed = @($results | Where-Object { $_.Status -ne 'OK' })+Write-Host ""+if ($failed.Count -eq 0) {+ Write-Host "ALL GREEN — $($liveVMs.Count) VM(s) installed and passed tests." -ForegroundColor Green+ if ($reportHtml) { Start-Process $reportHtml }+ exit 0+} else {+ Write-Host "FAILURES on $($failed.Count) VM(s):" -ForegroundColor Red+ $failed | ForEach-Object { Write-Host " - $($_.Name)" -ForegroundColor Red }+ if ($reportHtml) { Start-Process $reportHtml }+ exit 1+}diff --git a/scripts/test/harness/conftest.py b/scripts/test/harness/conftest.pynew file mode 100644index 00000000..df67bb8b--- /dev/null+++ b/scripts/test/harness/conftest.py@@ -0,0 +1,68 @@+import os+import time+import uuid+import requests+import pytest++BASE_URL = os.environ.get("SERVERKIT_URL", "http://127.0.0.1:5000")+TIMEOUT = 15++[email protected](scope="session")+def base_url():+ return BASE_URL++[email protected](scope="session")+def http():+ s = requests.Session()+ s.headers.update({"Accept": "application/json"})+ return s++[email protected](scope="session")+def admin_credentials():+ suffix = uuid.uuid4().hex[:8]+ return {+ "email": f"e2e-{suffix}@test.local",+ "username": f"e2e_{suffix}",+ "password": "Test12345!",+ }++[email protected](scope="session")+def admin_token(http, base_url, admin_credentials):+ """Register the first user (becomes admin) or log in if already exists."""+ # Wait briefly for backend in case test runs immediately after install+ for _ in range(30):+ try:+ r = http.get(f"{base_url}/api/v1/system/health", timeout=2)+ if r.status_code == 200:+ break+ except requests.RequestException:+ time.sleep(1)++ r = http.post(+ f"{base_url}/api/v1/auth/register",+ json=admin_credentials,+ timeout=TIMEOUT,+ )+ if r.status_code in (200, 201):+ return r.json()["access_token"]++ # Fallback: maybe already registered (re-run on same VM)+ r = http.post(+ f"{base_url}/api/v1/auth/login",+ json={+ "email": admin_credentials["email"],+ "password": admin_credentials["password"],+ },+ timeout=TIMEOUT,+ )+ r.raise_for_status()+ return r.json()["access_token"]+++def auth_headers(admin_token):+ return {"Authorization": f"Bearer {admin_token}"}diff --git a/scripts/test/harness/requirements.txt b/scripts/test/harness/requirements.txtnew file mode 100644index 00000000..daf0f940--- /dev/null+++ b/scripts/test/harness/requirements.txt@@ -0,0 +1,3 @@+pytest>=8.0+pytest-json-report>=1.5+requests>=2.31diff --git a/scripts/test/harness/test_01_health.py b/scripts/test/harness/test_01_health.pynew file mode 100644index 00000000..1e03a335--- /dev/null+++ b/scripts/test/harness/test_01_health.py@@ -0,0 +1,15 @@+import requests+++def test_backend_health(http, base_url):+ r = http.get(f"{base_url}/api/v1/system/health", timeout=10)+ assert r.status_code == 200, r.text+ body = r.json()+ assert body.get("status") == "healthy", body+++def test_frontend_reachable(base_url):+ # Frontend served by nginx on :80 (not the same as backend BASE_URL)+ url = base_url.replace(":5000", "").rstrip("/") or "http://127.0.0.1"+ r = requests.get(url, timeout=10, allow_redirects=True)+ assert r.status_code < 500, f"frontend returned {r.status_code}"diff --git a/scripts/test/harness/test_02_auth.py b/scripts/test/harness/test_02_auth.pynew file mode 100644index 00000000..2a0a314c--- /dev/null+++ b/scripts/test/harness/test_02_auth.py@@ -0,0 +1,29 @@+def test_setup_status_endpoint(http, base_url):+ r = http.get(f"{base_url}/api/v1/auth/setup-status", timeout=10)+ assert r.status_code == 200, r.text+ body = r.json()+ assert "needs_setup" in body, body+++def test_admin_register_and_login(admin_token, http, base_url, admin_credentials):+ # admin_token fixture already did the register; verify login works too+ assert admin_token, "no token from register"+ r = http.post(+ f"{base_url}/api/v1/auth/login",+ json={+ "email": admin_credentials["email"],+ "password": admin_credentials["password"],+ },+ timeout=10,+ )+ assert r.status_code == 200, r.text+ assert "access_token" in r.json()+++def test_authed_request(http, base_url, auth_headers):+ r = http.get(+ f"{base_url}/api/v1/auth/setup-status",+ headers=auth_headers,+ timeout=10,+ )+ assert r.status_code == 200, r.textdiff --git a/scripts/test/harness/test_03_plugins.py b/scripts/test/harness/test_03_plugins.pynew file mode 100644index 00000000..bdad3c17--- /dev/null+++ b/scripts/test/harness/test_03_plugins.py@@ -0,0 +1,47 @@+import pytest+++def test_list_plugins_empty(http, base_url, auth_headers):+ r = http.get(f"{base_url}/api/v1/plugins", headers=auth_headers, timeout=15)+ assert r.status_code == 200, r.text+ body = r.json()+ # accept either {"plugins": [...]} or a bare list+ if isinstance(body, dict):+ assert "plugins" in body+ else:+ assert isinstance(body, list)+++ True,+ reason="install-from-URL hits GitHub; enable when a stable test plugin repo exists",+)+def test_install_plugin_from_url(http, base_url, auth_headers):+ # Placeholder — wire to a known-good test plugin repo once one exists.+ r = http.post(+ f"{base_url}/api/v1/plugins/install",+ headers=auth_headers,+ json={"url": "https://github.com/jhd3197/serverkit-gui"},+ timeout=120,+ )+ assert r.status_code in (200, 201), r.text+ plugin = r.json()+ pid = plugin["id"]++ # disable+ r = http.post(+ f"{base_url}/api/v1/plugins/{pid}/disable", headers=auth_headers, timeout=15+ )+ assert r.status_code == 200, r.text++ # enable+ r = http.post(+ f"{base_url}/api/v1/plugins/{pid}/enable", headers=auth_headers, timeout=15+ )+ assert r.status_code == 200, r.text++ # uninstall+ r = http.delete(+ f"{base_url}/api/v1/plugins/{pid}", headers=auth_headers, timeout=30+ )+ assert r.status_code in (200, 204), r.textdiff --git a/scripts/test/harness/test_04_smoke.py b/scripts/test/harness/test_04_smoke.pynew file mode 100644index 00000000..fd009800--- /dev/null+++ b/scripts/test/harness/test_04_smoke.py@@ -0,0 +1,19 @@+"""Lightweight smoke tests on a sampling of authed endpoints.+Goal: catch import-time failures and 500s from blueprint registration."""++import pytest++# Endpoints we expect to exist and return something sane (200 or 4xx, not 5xx).+# Don't expand wildly — keep this focused on "did the blueprint load."+ENDPOINTS = [+ "/api/v1/servers",+ "/api/v1/plugins",+ "/api/v1/system/health",+ "/api/v1/auth/setup-status",+]++[email protected]("path", ENDPOINTS)+def test_endpoint_no_5xx(http, base_url, auth_headers, path):+ r = http.get(f"{base_url}{path}", headers=auth_headers, timeout=15)+ assert r.status_code < 500, f"{path} returned {r.status_code}: {r.text[:300]}"diff --git a/scripts/test/report.py b/scripts/test/report.pynew file mode 100644index 00000000..fc5dc65e--- /dev/null+++ b/scripts/test/report.py@@ -0,0 +1,172 @@+#!/usr/bin/env python3+"""Aggregate per-VM results into a single self-contained HTML report.++Inputs (from scripts/test/output/<run-id>/):+ <vm-name>/install.log — captured cloud-init / install output+ <vm-name>/install-status — "OK" or "FAIL"+ <vm-name>/pytest-report.json — pytest-json-report output (may be missing if install failed)+ <vm-name>/journalctl.log — backend logs (best-effort)++Output:+ report.html in the run directory.+"""+import json+import sys+import html+from pathlib import Path+from datetime import datetime+++def esc(s):+ return html.escape(str(s) if s is not None else "")+++def load_json(path):+ try:+ return json.loads(path.read_text(encoding="utf-8"))+ except Exception:+ return None+++def read_text(path, limit=200_000):+ try:+ data = path.read_text(encoding="utf-8", errors="replace")+ if len(data) > limit:+ data = "...[truncated]...\n" + data[-limit:]+ return data+ except Exception:+ return "(file missing)"+++def vm_summary(vm_dir: Path):+ name = vm_dir.name+ status_file = vm_dir / "install-status"+ install_ok = status_file.exists() and status_file.read_text().strip() == "OK"+ pytest_json = load_json(vm_dir / "pytest-report.json")++ tests = []+ pass_count = fail_count = skip_count = 0+ if pytest_json and "tests" in pytest_json:+ for t in pytest_json["tests"]:+ outcome = t.get("outcome", "unknown")+ tests.append((t.get("nodeid", "?"), outcome, t.get("call", {}).get("longrepr", "")))+ if outcome == "passed":+ pass_count += 1+ elif outcome == "failed":+ fail_count += 1+ elif outcome == "skipped":+ skip_count += 1++ overall = "PASS" if (install_ok and fail_count == 0 and (pass_count > 0 or pytest_json is None)) else "FAIL"+ if not install_ok:+ overall = "FAIL"++ return {+ "name": name,+ "install_ok": install_ok,+ "overall": overall,+ "tests": tests,+ "pass": pass_count,+ "fail": fail_count,+ "skip": skip_count,+ "install_log": read_text(vm_dir / "install.log"),+ "journal": read_text(vm_dir / "journalctl.log", limit=50_000),+ }+++HTML_TEMPLATE = """<!doctype html>+<html><head><meta charset="utf-8"><title>ServerKit E2E Report — {ts}</title>+<style>+ body{{font-family:-apple-system,Segoe UI,sans-serif;margin:24px;background:#0f1115;color:#e6e6e6}}+ h1{{margin-top:0}}+ .vm{{background:#1a1d24;border-radius:8px;padding:16px;margin:16px 0;border-left:6px solid #555}}+ .vm.pass{{border-left-color:#3fb950}}+ .vm.fail{{border-left-color:#f85149}}+ .badge{{display:inline-block;padding:2px 10px;border-radius:12px;font-size:12px;font-weight:600}}+ .badge.pass{{background:#1f6f33;color:#d2f5d8}}+ .badge.fail{{background:#7a2a26;color:#fdd}}+ .badge.skip{{background:#3a3a3a;color:#bbb}}+ details{{margin-top:8px;background:#0c0e13;border-radius:6px;padding:8px}}+ summary{{cursor:pointer;font-weight:600}}+ pre{{white-space:pre-wrap;word-break:break-word;font-size:12px;background:#000;padding:8px;border-radius:4px;max-height:400px;overflow:auto}}+ table{{border-collapse:collapse;width:100%;margin-top:8px;font-size:13px}}+ td,th{{padding:4px 8px;border-bottom:1px solid #2a2d34;text-align:left}}+ .counts span{{margin-right:12px}}+</style></head><body>+<h1>ServerKit E2E Test Report</h1>+<p>Run: <code>{run_id}</code> · {ts}</p>+<p>Overall: <span class="badge {overall_cls}">{overall}</span>+ VMs: {n_vms} Passed: {n_pass} Failed: {n_fail}</p>+{vm_sections}+</body></html>"""++VM_TEMPLATE = """<div class="vm {cls}">+ <h2>{name} <span class="badge {cls}">{overall}</span></h2>+ <div class="counts">+ <span>Install: <b>{install}</b></span>+ <span>Passed: <b>{p}</b></span>+ <span>Failed: <b>{f}</b></span>+ <span>Skipped: <b>{s}</b></span>+ </div>+ {tests_table}+ <details><summary>Install log</summary><pre>{install_log}</pre></details>+ <details><summary>journalctl -u serverkit</summary><pre>{journal}</pre></details>+</div>"""+++def render(run_dir: Path) -> Path:+ vm_dirs = sorted([p for p in run_dir.iterdir() if p.is_dir()])+ summaries = [vm_summary(d) for d in vm_dirs]++ n_pass = sum(1 for s in summaries if s["overall"] == "PASS")+ n_fail = sum(1 for s in summaries if s["overall"] != "PASS")+ overall = "PASS" if n_fail == 0 and n_pass > 0 else "FAIL"++ vm_sections = []+ for s in summaries:+ cls = "pass" if s["overall"] == "PASS" else "fail"+ if s["tests"]:+ rows = "".join(+ f"<tr><td>{esc(nid)}</td><td><span class='badge {oc}'>{esc(oc)}</span></td>"+ f"<td><pre style='max-height:120px'>{esc(repr_)}</pre></td></tr>"+ for nid, oc, repr_ in s["tests"]+ )+ tests_table = f"<table><tr><th>Test</th><th>Result</th><th>Details</th></tr>{rows}</table>"+ else:+ tests_table = "<p><i>No pytest results (install likely failed before harness ran).</i></p>"++ vm_sections.append(VM_TEMPLATE.format(+ cls=cls,+ name=esc(s["name"]),+ overall=esc(s["overall"]),+ install="OK" if s["install_ok"] else "FAIL",+ p=s["pass"], f=s["fail"], s=s["skip"],+ tests_table=tests_table,+ install_log=esc(s["install_log"]),+ journal=esc(s["journal"]),+ ))++ out = run_dir / "report.html"+ out.write_text(HTML_TEMPLATE.format(+ ts=datetime.now().strftime("%Y-%m-%d %H:%M:%S"),+ run_id=esc(run_dir.name),+ overall=overall,+ overall_cls="pass" if overall == "PASS" else "fail",+ n_vms=len(summaries),+ n_pass=n_pass,+ n_fail=n_fail,+ vm_sections="\n".join(vm_sections),+ ), encoding="utf-8")+ return out+++if __name__ == "__main__":+ if len(sys.argv) != 2:+ print("usage: report.py <run-output-dir>", file=sys.stderr)+ sys.exit(2)+ run_dir = Path(sys.argv[1]).resolve()+ if not run_dir.is_dir():+ print(f"not a directory: {run_dir}", file=sys.stderr)+ sys.exit(2)+ out = render(run_dir)+ print(str(out))diff --git a/scripts/test/vm-install.sh b/scripts/test/vm-install.shnew file mode 100644index 00000000..68a783b4--- /dev/null+++ b/scripts/test/vm-install.sh@@ -0,0 +1,85 @@+#!/bin/bash+# Runs inside the test VM.+# Strategy:+# 1. Run install.sh from the uploaded local source (so it bootstraps deps).+# 2. After install, replace /opt/serverkit with the uploaded local working+# tree (overlay), rebuild frontend, reinstall Python deps, restart.+# 3. Probe health endpoint and exit non-zero if anything failed.+#+# This way we test the EXACT local code, including uncommitted changes,+# instead of whatever is on origin/main.+set -u # NB: not -e — we want to capture failures and still emit a report++SRC=/opt/serverkit-src+INSTALL_DIR=/opt/serverkit+LOG=/var/log/serverkit-test-install.log++log() { echo "[vm-install] $*" | tee -a "$LOG"; }+fail() { echo "[vm-install] FAIL: $*" | tee -a "$LOG"; exit 1; }++mkdir -p "$(dirname "$LOG")"+: > "$LOG"++[ -d "$SRC" ] || fail "source dir $SRC missing — multipass transfer broken"+[ -f "$SRC/install.sh" ] || fail "install.sh not found in source"++log "Step 1/4: running install.sh (this clones origin/main, installs deps)"+# install.sh clones from GitHub — we let it, then overlay our local code.+bash "$SRC/install.sh" >> "$LOG" 2>&1+INSTALL_RC=$?+log "install.sh exit=$INSTALL_RC"++if [ ! -d "$INSTALL_DIR" ]; then+ fail "install.sh did not create $INSTALL_DIR (rc=$INSTALL_RC)"+fi++log "Step 2/4: overlaying local working tree onto $INSTALL_DIR"+# Preserve .env / instance / nginx ssl from the install+rsync -a \+ --exclude='.env' \+ --exclude='backend/instance/' \+ --exclude='nginx/ssl/' \+ --exclude='backend/venv/' \+ --exclude='backend/.venv/' \+ --exclude='backend/.venv-wsl/' \+ --exclude='frontend/node_modules/' \+ --exclude='frontend/dist/' \+ --exclude='.git/' \+ "$SRC/" "$INSTALL_DIR/" >> "$LOG" 2>&1 || fail "rsync overlay failed"++log "Step 3/4: rebuild + restart with local code"+cd "$INSTALL_DIR" || fail "cd $INSTALL_DIR"++# Reinstall Python deps in case requirements.txt changed+if [ -d "$INSTALL_DIR/venv" ]; then+ "$INSTALL_DIR/venv/bin/pip" install -r "$INSTALL_DIR/backend/requirements.txt" >> "$LOG" 2>&1 \+ || log "WARN: pip install had non-zero exit"+fi++# Rebuild frontend+cd "$INSTALL_DIR/frontend" || fail "cd frontend"+NODE_OPTIONS="--max-old-space-size=1024" npm run build >> "$LOG" 2>&1 \+ || fail "frontend build failed"++cd "$INSTALL_DIR" || true+docker compose build >> "$LOG" 2>&1 || log "WARN: docker compose build had non-zero exit"+docker compose up -d >> "$LOG" 2>&1 || log "WARN: docker compose up had non-zero exit"++systemctl restart serverkit >> "$LOG" 2>&1 || fail "systemctl restart serverkit failed"+systemctl restart nginx >> "$LOG" 2>&1 || log "WARN: nginx restart had non-zero exit"++log "Step 4/4: waiting for health endpoint"+for i in $(seq 1 30); do+ if curl -fsS http://127.0.0.1:5000/api/v1/system/health > /dev/null 2>&1; then+ log "Backend healthy after ${i}s"+ echo "OK" > /tmp/serverkit-install-status+ exit 0+ fi+ sleep 1+done++log "Backend never became healthy"+systemctl status serverkit --no-pager >> "$LOG" 2>&1 || true+journalctl -u serverkit --no-pager -n 100 >> "$LOG" 2>&1 || true+echo "FAIL" > /tmp/serverkit-install-status+exit 1