Skip to content

ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.

Add Vagrant + Hyper-V runner for Debian/Fedora/Rocky coverage

Juan Denis3b2974177a96
diff --git a/scripts/test/README.md b/scripts/test/README.md
index 33872cc7..6a3b84e2 100644
--- a/scripts/test/README.md
+++ b/scripts/test/README.md
@@ -10,12 +10,19 @@ installer actually works on multiple distros" use case.
## Prerequisites (one-time)
-1. **Multipass** — https://multipass.run/download/windows
+1. **Multipass** — Ubuntu 22.04 + 24.04 VMs
```powershell
winget install Canonical.Multipass
```
-2. **Python 3** on PATH (for the HTML report generator; tests run inside the VMs).
-3. **WSL2 / Hyper-V** enabled (Multipass uses this — installer prompts you).
+2. **Vagrant** — Debian 12, Fedora, Rocky 9 VMs (uses Hyper-V provider, no VirtualBox)
+ ```powershell
+ winget install Hashicorp.Vagrant
+ ```
+3. **Python 3** on PATH (for the HTML report generator; tests run inside the VMs).
+4. **Hyper-V** enabled (both Multipass and Vagrant use it on Windows).
+
+You can install only Multipass if Ubuntu coverage is enough — the harness
+auto-detects which backends are available and skips distros it can't run.
## Usage
@@ -32,16 +39,32 @@ HTML report.
### Options
```powershell
-# Only test Ubuntu 24.04
-.\scripts\test\full-stack-test.ps1 -Only ubuntu24
+# Full default suite: ubuntu22, ubuntu24, debian12, fedora, rocky9
+.\scripts\test\full-stack-test.ps1
+
+# Subset
+.\scripts\test\full-stack-test.ps1 -Only "ubuntu24,debian12"
# Keep VMs running so you can shell in and poke around
.\scripts\test\full-stack-test.ps1 -Keep
# Bigger VMs (default 2 CPU / 4 GB RAM / 15 GB disk)
.\scripts\test\full-stack-test.ps1 -Cpus 4 -MemoryGB 8
+
+# Re-run harness against an already-installed Multipass VM (fast iteration)
+.\scripts\test\full-stack-test.ps1 -ReuseVm sk-test-ubuntu24-<id>
```
+### Distro coverage
+
+| Distro | Backend | Box / Image |
+|---|---|---|
+| Ubuntu 22.04 | Multipass | `22.04` |
+| Ubuntu 24.04 | Multipass | `24.04` |
+| Debian 12 | Vagrant | `generic/debian12` |
+| Fedora | Vagrant | `generic/fedora39` |
+| Rocky 9 | Vagrant | `generic/rocky9` |
+
### Agent pairing test (optional)
After running with `-Keep`:
diff --git a/scripts/test/full-stack-test.ps1 b/scripts/test/full-stack-test.ps1
index bab6b607..e7be51f7 100644
--- a/scripts/test/full-stack-test.ps1
+++ b/scripts/test/full-stack-test.ps1
@@ -27,7 +27,10 @@
#>
[CmdletBinding()]
param(
- [string[]] $Distros = @('ubuntu22','ubuntu24','debian12'),
+ # Default suite: Ubuntu LTSes via Multipass + non-Ubuntu via Vagrant
+ # + Hyper-V provider. Vagrant boxes are heavier (first launch downloads
+ # ~600 MB each), so first full run is long. -Only to subset.
+ [string[]] $Distros = @('ubuntu22','ubuntu24','debian12','fedora','rocky9'),
[string] $Only,
[switch] $Keep,
[string] $ReuseVm,
@@ -42,10 +45,16 @@ param(
$ErrorActionPreference = 'Continue'
$ProgressPreference = 'SilentlyContinue'
-$ImageMap = @{
- 'ubuntu22' = '22.04'
- 'ubuntu24' = '24.04'
- 'debian12' = 'daily:debian12'
+# DistroMap drives both dispatch and image/box selection.
+# backend: 'multipass' uses the Canonical Ubuntu images; 'vagrant' uses
+# the Hyper-V provider with generic/* boxes. The login user differs by
+# backend (ubuntu vs vagrant), which matters for chown on the harness dir.
+$DistroMap = @{
+ 'ubuntu22' = @{ backend = 'multipass'; image = '22.04'; user = 'ubuntu' }
+ 'ubuntu24' = @{ backend = 'multipass'; image = '24.04'; user = 'ubuntu' }
+ 'debian12' = @{ backend = 'vagrant'; image = 'generic/debian12'; user = 'vagrant' }
+ 'fedora' = @{ backend = 'vagrant'; image = 'generic/fedora39'; user = 'vagrant' }
+ 'rocky9' = @{ backend = 'vagrant'; image = 'generic/rocky9'; user = 'vagrant' }
}
if ($Only) {
@@ -62,25 +71,44 @@ $null = New-Item -ItemType Directory -Force -Path $OutDir
# "running" rather than treating an empty dir as a failed run.
Set-Content -Path (Join-Path $OutDir 'state.json') -Value '{"running":true,"vms":{}}' -Encoding utf8
-# --- locate multipass ----------------------------------------------------
-$MpExe = $null
-$candidates = @(
+# --- locate backends -----------------------------------------------------
+function Find-Exe {
+ param([string[]] $Paths, [string] $Cmd)
+ foreach ($p in $Paths) { if (Test-Path $p) { return $p } }
+ return (Get-Command $Cmd -ErrorAction SilentlyContinue).Source
+}
+
+$MpExe = Find-Exe -Cmd 'multipass' -Paths @(
'C:\Program Files\Multipass\bin\multipass.exe',
"$env:LOCALAPPDATA\Programs\Multipass\bin\multipass.exe"
)
-foreach ($c in $candidates) { if (Test-Path $c) { $MpExe = $c; break } }
-if (-not $MpExe) {
- $MpExe = (Get-Command multipass -ErrorAction SilentlyContinue).Source
+$VgExe = Find-Exe -Cmd 'vagrant' -Paths @(
+ 'C:\Program Files\Vagrant\bin\vagrant.exe',
+ "$env:LOCALAPPDATA\Programs\Vagrant\bin\vagrant.exe"
+)
+
+# Filter distros down to whichever backends are actually installed.
+$missing = @()
+$Distros = $Distros | Where-Object {
+ $entry = $DistroMap[$_]
+ if (-not $entry) { return $false }
+ if ($entry.backend -eq 'multipass' -and -not $MpExe) { $missing += "$_ (needs Multipass)"; return $false }
+ if ($entry.backend -eq 'vagrant' -and -not $VgExe) { $missing += "$_ (needs Vagrant)"; return $false }
+ $true
}
-if (-not $MpExe) {
- Write-Host "Multipass not found. Install via: winget install Canonical.Multipass" -ForegroundColor Red
+
+if (-not $Distros -or $Distros.Count -eq 0) {
+ Write-Host "No runnable distros. Install Multipass and/or Vagrant first." -ForegroundColor Red
+ if ($missing) { $missing | ForEach-Object { Write-Host " skipped: $_" -ForegroundColor Yellow } }
exit 2
}
Write-Host "========================================" -ForegroundColor Cyan
Write-Host " ServerKit E2E Test - run $RunId" -ForegroundColor Cyan
-Write-Host " Multipass: $MpExe" -ForegroundColor Cyan
+if ($MpExe) { Write-Host " Multipass: $MpExe" -ForegroundColor Cyan }
+if ($VgExe) { Write-Host " Vagrant: $VgExe" -ForegroundColor Cyan }
Write-Host " Distros: $($Distros -join ', ')" -ForegroundColor Cyan
+if ($missing) { $missing | ForEach-Object { Write-Host " skipped: $_" -ForegroundColor Yellow } }
Write-Host " Output: $OutDir" -ForegroundColor Cyan
Write-Host "========================================" -ForegroundColor Cyan
@@ -159,6 +187,95 @@ function Test-VmInstall {
return $status
}
+function Test-VagrantInstall {
+ # Vagrant equivalent of Test-VmInstall. Uses 'vagrant upload' for file
+ # transfer (added in 2.2.7) and 'vagrant ssh -c' for exec. Each VM
+ # gets its own VAGRANT_CWD so multiple distros can coexist.
+ param([string]$Vm, [string]$VmOut, [string]$Tarball, [string]$VmInstallScript, [string]$HarnessDir, [string]$Box, [string]$User)
+
+ New-Item -ItemType Directory -Force -Path $VmOut | Out-Null
+ $statusFile = Join-Path $VmOut 'install-status'
+
+ # Per-VM Vagrant working dir holding the Vagrantfile + .vagrant state.
+ $vagrantSrc = Join-Path $PSScriptRoot 'vagrant\Vagrantfile'
+ $vagrantCwd = Join-Path $OutDir "vagrant-state\$Vm"
+ New-Item -ItemType Directory -Force -Path $vagrantCwd | Out-Null
+ Copy-Item $vagrantSrc (Join-Path $vagrantCwd 'Vagrantfile') -Force
+
+ $env:VAGRANT_CWD = $vagrantCwd
+ $env:SK_BOX = $Box
+ $env:SK_VM_NAME = $Vm
+ $env:SK_CPUS = "$Cpus"
+ $env:SK_MEMORY_MB = "$($MemoryGB * 1024)"
+
+ Set-VmStage -Vm $Vm -Stage 'launching VM (vagrant up, ~5-10 min first time)'
+ Write-Host " [$Vm] vagrant up..." -ForegroundColor DarkCyan
+ & $VgExe up --provider=hyperv 2>&1 | Out-File (Join-Path $VmOut 'vagrant-up.log') -Encoding utf8
+ if ($LASTEXITCODE -ne 0) {
+ Set-Content -Path $statusFile -Value 'LAUNCH_FAILED' -Encoding ascii
+ Set-VmStage -Vm $Vm -Stage 'launch failed'
+ return 'LAUNCH_FAILED'
+ }
+
+ Set-VmStage -Vm $Vm -Stage 'transferring source'
+ Write-Host " [$Vm] transferring source..." -ForegroundColor DarkCyan
+ & $VgExe upload $Tarball /tmp/serverkit-src.tar.gz 2>&1 | Out-Null
+ & $VgExe upload $VmInstallScript /tmp/vm-install.sh 2>&1 | Out-Null
+ & $VgExe ssh -c "sudo mkdir -p /opt/serverkit-src && sudo tar -xzf /tmp/serverkit-src.tar.gz -C /opt/serverkit-src && sudo chmod +x /tmp/vm-install.sh" 2>&1 | Out-Null
+
+ Set-VmStage -Vm $Vm -Stage 'installing (npm ci + build, ~10 min)'
+ Write-Host " [$Vm] running install..." -ForegroundColor DarkCyan
+ & $VgExe ssh -c "sudo bash /tmp/vm-install.sh > /tmp/vm-install-stdout.log 2>&1; rc=`$?; echo EXIT_RC=`$rc; exit `$rc" `
+ 2>&1 | Out-File (Join-Path $VmOut 'install.log') -Encoding utf8
+ $installRC = $LASTEXITCODE
+
+ # Pull canonical logs back. vagrant scp would need a plugin; we
+ # cat-then-redirect via vagrant ssh, which is universal.
+ & $VgExe ssh -c "sudo cat /var/log/serverkit-test-install.log" 2>$null `
+ | Out-File (Join-Path $VmOut 'vm-install.log') -Encoding utf8
+ & $VgExe ssh -c "sudo cat /tmp/serverkit-install-status" 2>$null `
+ | Out-File $statusFile -Encoding ascii
+ if (-not (Test-Path $statusFile) -or (Get-Content $statusFile -Raw -ErrorAction SilentlyContinue).Trim() -eq '') {
+ $fallback = if ($installRC -eq 0) { 'OK' } else { 'FAIL' }
+ Set-Content -Path $statusFile -Value $fallback -Encoding ascii
+ }
+ & $VgExe ssh -c "sudo journalctl -u serverkit --no-pager -n 500" 2>$null `
+ | Out-File (Join-Path $VmOut 'journalctl.log') -Encoding utf8
+
+ $status = (Get-Content $statusFile -Raw).Trim()
+ Write-Host " [$Vm] install status: $status" -ForegroundColor $(if ($status -eq 'OK') { 'Green' } else { 'Red' })
+ Save-State
+
+ if ($status -ne 'OK') {
+ Set-VmStage -Vm $Vm -Stage 'install failed'
+ return $status
+ }
+
+ Set-VmStage -Vm $Vm -Stage 'running pytest'
+ Write-Host " [$Vm] running pytest harness..." -ForegroundColor DarkCyan
+ & $VgExe ssh -c "sudo mkdir -p /opt/serverkit-test && sudo chown ${User}:${User} /opt/serverkit-test" 2>&1 | Out-Null
+ Get-ChildItem $HarnessDir -File | ForEach-Object {
+ & $VgExe upload $_.FullName "/opt/serverkit-test/$($_.Name)" 2>&1 | Out-Null
+ }
+ & $VgExe ssh -c "sudo /opt/serverkit/venv/bin/pip install -r /opt/serverkit-test/requirements.txt" 2>&1 `
+ | Out-File (Join-Path $VmOut 'harness-deps.log') -Encoding utf8
+ & $VgExe ssh -c "sudo /opt/serverkit/venv/bin/python -m pytest /opt/serverkit-test --json-report --json-report-file=/opt/serverkit-test/pytest-report.json -v > /tmp/pytest-stdout.log 2>&1; rc=`$?; echo EXIT_RC=`$rc; exit `$rc" `
+ 2>&1 | Out-File (Join-Path $VmOut 'pytest.log') -Encoding utf8
+ & $VgExe ssh -c "sudo cat /tmp/pytest-stdout.log" 2>$null | Out-File (Join-Path $VmOut 'pytest-stdout.log') -Encoding utf8
+ & $VgExe ssh -c "sudo cat /opt/serverkit-test/pytest-report.json" 2>$null | Out-File (Join-Path $VmOut 'pytest-report.json') -Encoding utf8
+
+ Set-VmStage -Vm $Vm -Stage 'done'
+ return $status
+}
+
+function Remove-VagrantVm {
+ param([string]$Vm)
+ $vagrantCwd = Join-Path $OutDir "vagrant-state\$Vm"
+ if (-not (Test-Path $vagrantCwd)) { return }
+ $env:VAGRANT_CWD = $vagrantCwd
+ & $VgExe destroy -f 2>&1 | Out-Null
+}
+
# --- pack source --------------------------------------------------------
$Tarball = Join-Path $OutDir 'serverkit-src.tar.gz'
Write-Host "`n[1/4] Packing local working tree -> $Tarball" -ForegroundColor Yellow
@@ -207,32 +324,43 @@ if (Test-Path $reportPath) {
if ($ReuseVm) {
Write-Host "`n[2-3/4] Reusing VM: $ReuseVm" -ForegroundColor Yellow
$vmOut = Join-Path $OutDir $ReuseVm
+ # Reuse only supports Multipass for now (Vagrant VMs are tied to a
+ # specific VAGRANT_CWD that doesn't exist outside the original run).
$status = Test-VmInstall -Vm $ReuseVm -VmOut $vmOut -Tarball $Tarball `
-VmInstallScript $vmInstallScript -HarnessDir $harnessDir
$results += @{ Name = $ReuseVm; Status = $status }
- $liveVMs += $ReuseVm
+ $liveVMs += @{ Name = $ReuseVm; Backend = 'multipass' }
} else {
Write-Host "`n[2/4] Launching + testing $($Distros.Count) VM(s) sequentially..." -ForegroundColor Yellow
foreach ($d in $Distros) {
$vmName = "sk-test-$d-$RunId"
- $image = $ImageMap[$d]
- if (-not $image) { Write-Warning "Unknown distro $d, skipping"; continue }
-
- Write-Host "`n--- $d ---" -ForegroundColor Yellow
- Write-Host " Launching $vmName from image $image..." -ForegroundColor DarkCyan
- & $MpExe launch $image --name $vmName --cpus $Cpus --memory "${MemoryGB}G" --disk "${DiskGB}G" --cloud-init $cloudInit
- if ($LASTEXITCODE -ne 0) {
- Write-Host " [FAIL] $vmName launch failed" -ForegroundColor Red
- $results += @{ Name = $vmName; Status = 'LAUNCH_FAILED' }
- continue
- }
- Write-Host " [OK] $vmName launched" -ForegroundColor Green
- $liveVMs += $vmName
-
+ $entry = $DistroMap[$d]
$vmOut = Join-Path $OutDir $vmName
- $status = Test-VmInstall -Vm $vmName -VmOut $vmOut -Tarball $Tarball `
- -VmInstallScript $vmInstallScript -HarnessDir $harnessDir
- $results += @{ Name = $vmName; Status = $status }
+
+ Write-Host "`n--- $d ($($entry.backend)) ---" -ForegroundColor Yellow
+
+ if ($entry.backend -eq 'multipass') {
+ Write-Host " Launching $vmName from image $($entry.image)..." -ForegroundColor DarkCyan
+ & $MpExe launch $entry.image --name $vmName --cpus $Cpus --memory "${MemoryGB}G" --disk "${DiskGB}G" --cloud-init $cloudInit
+ if ($LASTEXITCODE -ne 0) {
+ Write-Host " [FAIL] $vmName launch failed" -ForegroundColor Red
+ $results += @{ Name = $vmName; Status = 'LAUNCH_FAILED' }
+ continue
+ }
+ Write-Host " [OK] $vmName launched" -ForegroundColor Green
+ $liveVMs += @{ Name = $vmName; Backend = 'multipass' }
+ $status = Test-VmInstall -Vm $vmName -VmOut $vmOut -Tarball $Tarball `
+ -VmInstallScript $vmInstallScript -HarnessDir $harnessDir
+ $results += @{ Name = $vmName; Status = $status }
+ }
+ elseif ($entry.backend -eq 'vagrant') {
+ Write-Host " Bringing up $vmName via Vagrant (box: $($entry.image))..." -ForegroundColor DarkCyan
+ $liveVMs += @{ Name = $vmName; Backend = 'vagrant' }
+ $status = Test-VagrantInstall -Vm $vmName -VmOut $vmOut -Tarball $Tarball `
+ -VmInstallScript $vmInstallScript -HarnessDir $harnessDir `
+ -Box $entry.image -User $entry.user
+ $results += @{ Name = $vmName; Status = $status }
+ }
}
}
@@ -251,11 +379,17 @@ if (Test-Path $reportHtml) {
# --- teardown -----------------------------------------------------------
if ($Keep -or $ReuseVm) {
Write-Host "`nVMs left running:" -ForegroundColor Yellow
- $liveVMs | ForEach-Object { Write-Host " - $_" }
+ $liveVMs | ForEach-Object { Write-Host " - $($_.Name) ($($_.Backend))" }
} else {
Write-Host "`nTearing down VMs..." -ForegroundColor Yellow
- foreach ($vm in $liveVMs) { & $MpExe delete $vm 2>&1 | Out-Null }
- & $MpExe purge 2>&1 | Out-Null
+ foreach ($vm in $liveVMs) {
+ if ($vm.Backend -eq 'multipass') {
+ & $MpExe delete $vm.Name 2>&1 | Out-Null
+ } elseif ($vm.Backend -eq 'vagrant') {
+ Remove-VagrantVm -Vm $vm.Name
+ }
+ }
+ if ($MpExe) { & $MpExe purge 2>&1 | Out-Null }
}
# --- summary ------------------------------------------------------------
diff --git a/scripts/test/vagrant/Vagrantfile b/scripts/test/vagrant/Vagrantfile
new file mode 100644
index 00000000..a3d6ede7
--- /dev/null
+++ b/scripts/test/vagrant/Vagrantfile
@@ -0,0 +1,50 @@
+# Parameterized Vagrantfile for ServerKit E2E tests.
+# The orchestrator passes config via env vars and uses one working dir
+# per VM so multiple distros can coexist without colliding.
+#
+# Required env vars:
+# SK_BOX e.g. "generic/debian12"
+# SK_VM_NAME e.g. "sk-test-debian12-20260517-120000"
+#
+# Optional:
+# SK_CPUS default 2
+# SK_MEMORY_MB default 4096
+
+box_name = ENV['SK_BOX'] || abort("SK_BOX env var is required")
+vm_name = ENV['SK_VM_NAME'] || abort("SK_VM_NAME env var is required")
+cpus = (ENV['SK_CPUS'] || '2').to_i
+memory_mb = (ENV['SK_MEMORY_MB'] || '4096').to_i
+
+Vagrant.configure("2") do |config|
+ config.vm.box = box_name
+ config.vm.hostname = vm_name
+ config.vm.define vm_name
+
+ # No synced folders — we transfer files via vagrant scp/upload instead.
+ # Default rsync sync slows boot and can fail on Windows hosts.
+ config.vm.synced_folder ".", "/vagrant", disabled: true
+
+ config.vm.provider "hyperv" do |hv|
+ hv.vmname = vm_name
+ hv.cpus = cpus
+ hv.memory = memory_mb
+ hv.maxmemory = memory_mb
+ hv.enable_virtualization_extensions = false
+ hv.linked_clone = true
+ end
+
+ # Bootstrap: ensure rsync + curl are present (mirrors the multipass
+ # cloud-init). Best effort — different distros use different package
+ # managers, so try them in order.
+ config.vm.provision "shell", inline: <<-SHELL
+ set -e
+ if command -v apt-get >/dev/null 2>&1; then
+ export DEBIAN_FRONTEND=noninteractive
+ apt-get update -qq && apt-get install -y -qq rsync curl
+ elif command -v dnf >/dev/null 2>&1; then
+ dnf install -y -q rsync curl
+ elif command -v yum >/dev/null 2>&1; then
+ yum install -y -q rsync curl
+ fi
+ SHELL
+end