Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Refuse Windows agent mutation until service removal is confirmed

pulse-triage[bot]a6a81d35293f
diff --git a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
index 9ee7744b7e..58c044ce63 100644
--- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
+++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
@@ -15,6 +15,31 @@
## Purpose
+### Windows service removal is a prerequisite for destructive continuation
+
+The Windows installer shares one service-removal boundary between replacement
+and uninstall. An SCM read error is unknown, not absence: only the cmdlet's
+explicit service-not-found result authorises the already-removed path. An
+existing service must reach `Stopped` within 30 seconds, with a refreshed state
+read; a pending stop is awaited without requesting it again. The installer
+requests no forced dependent-service stop and kills no process.
+
+Controller handles are closed before `sc.exe delete`. Its exit must succeed,
+and fresh SCM reads must then confirm actual absence within 30 seconds;
+"marked for deletion" alone is insufficient. A failed read, stop, delete or
+absence observation exits nonzero before binary replacement, token/connection
+mutation, server deregistration or local erasure. Uninstall also reports local
+cleanup failure instead of declaring completion. Successful fresh install,
+replacement and repeated uninstall retain their existing enrolment semantics.
+
+`TestInstallPS1ServiceRemovalRefusesUnknownRuntimeBeforeMutation` pins both
+callers and adverse structural controls. `TestInstallPS1ServiceRemovalRuntime`
+executes the actual production functions with SCM failure/delay controls, using
+Windows PowerShell 5.1 in the existing native Windows job. A missing PowerShell
+runtime is an explicit skip, not native or mocked execution. The real Windows
+service lifecycle job remains required evidence for installed behaviour; these
+controls do not establish published-agent recovery or auto-update acceptance.
+
### Installer identity recovery accounts for the whole legacy file
The descriptor-safe collector command remains the primary agent-ID reader. Its
diff --git a/docs/release-control/v6/internal/subsystems/deployment-installability.md b/docs/release-control/v6/internal/subsystems/deployment-installability.md
index 00bdbfd0ae..3ca57ac031 100644
--- a/docs/release-control/v6/internal/subsystems/deployment-installability.md
+++ b/docs/release-control/v6/internal/subsystems/deployment-installability.md
@@ -15,6 +15,24 @@
## Purpose
+### Windows replacement and removal refuse uncertain service state
+
+Before replacing an agent binary or changing its token/connection state, and
+before uninstall deregistration or erasure, `scripts/install.ps1` must confirm
+the existing PulseAgent service stopped and actually removed. Only explicit
+SCM not-found establishes absence. Stop completion and post-delete absence
+each have a 30-second bound; stop/read/delete failures retain the old binary
+and agent state and return nonzero. Deletion success while SCM still lists a
+service is not completion. Local uninstall erasure errors also return nonzero,
+without a success message. No forced dependent-service stop, PID kill or trust
+relaxation is introduced.
+
+The registered Windows installer tests pin both mutation boundaries, their
+refusal exits and mocked production-function controls. The existing native
+Windows job runs those controls under PowerShell 5.1 and then the real
+install/update/uninstall lifecycle. Linux source tests cannot establish native
+SCM acceptance and retain a missing-runtime skip explicitly.
+
### Server removal requires every affected unit to be quiescent
Both the existing-installation menu and `--uninstall` must stop and disable the
diff --git a/scripts/install.ps1 b/scripts/install.ps1
index 194d905ec9..4ffcc71d22 100644
--- a/scripts/install.ps1
+++ b/scripts/install.ps1
@@ -514,10 +514,82 @@ function Get-ConnectionStateValue {
return $value.Trim("'")
}
+function Get-PulseService {
+ try {
+ return Get-Service -Name $AgentName -ErrorAction Stop
+ } catch {
+ # Only SCM's explicit not-found result establishes absence. Access or
+ # enumeration failures must not authorise replacement or data removal.
+ if ($_.FullyQualifiedErrorId.Split(',')[0] -eq 'NoServiceFoundForGivenName' -and
+ $_.CategoryInfo.Category -eq [System.Management.Automation.ErrorCategory]::ObjectNotFound) {
+ return $null
+ }
+ throw
+ }
+}
+
+function Remove-PulseService {
+ param([int]$TimeoutSeconds = 30)
+
+ $service = Get-PulseService
+ if ($null -eq $service) {
+ return
+ }
+
+ try {
+ $service.Refresh()
+ if ($service.Status -ne 'Stopped') {
+ # ServiceController.Stop requests a stop without Stop-Service's
+ # unbounded wait. Do not force-stop other services or kill a PID.
+ if ($service.Status -ne 'StopPending') {
+ $service.Stop()
+ }
+ $service.WaitForStatus([System.ServiceProcess.ServiceControllerStatus]::Stopped,
+ [TimeSpan]::FromSeconds($TimeoutSeconds))
+ }
+ $service.Refresh()
+ if ($service.Status -ne 'Stopped') {
+ throw "Service '$AgentName' is not confirmed stopped."
+ }
+ } finally {
+ # An open controller handle can itself keep an SCM deletion pending.
+ $service.Dispose()
+ }
+
+ $scOutput = sc.exe delete $AgentName 2>&1
+ if ($LASTEXITCODE -ne 0) {
+ throw "Failed to delete service '$AgentName': $scOutput"
+ }
+
+ # sc.exe success can mean 'marked for deletion', not actually absent.
+ # Keep the old binary and state until an independent SCM read confirms it.
+ $elapsed = [System.Diagnostics.Stopwatch]::StartNew()
+ do {
+ $remaining = Get-PulseService
+ if ($null -eq $remaining) {
+ return
+ }
+ $remaining.Dispose()
+ if ($elapsed.Elapsed.TotalSeconds -ge $TimeoutSeconds) {
+ throw "Service '$AgentName' is still present after deletion. Close service-management windows and retry after confirming it is stopped."
+ }
+ Start-Sleep -Milliseconds 200
+ } while ($true)
+}
+
# --- Uninstall Logic ---
if ($Uninstall) {
Write-Host "Uninstalling $AgentName..." -ForegroundColor Cyan
+ # Refuse remote deregistration and local erasure while runtime state is
+ # unknown. On failure keep the identity, token, logs and executable intact.
+ try {
+ Remove-PulseService
+ } catch {
+ Show-Error "Cannot uninstall '$AgentName'; binary and agent state have been retained. $_"
+ Exit 1
+ }
+
# Try to notify the Pulse server about uninstallation if we have connection details
if ([string]::IsNullOrWhiteSpace($Url)) {
$Url = Get-ConnectionStateValue "PULSE_URL"
@@ -619,23 +691,18 @@ if ($Uninstall) {
}
}
- if (Get-Service $AgentName -ErrorAction SilentlyContinue) {
- Stop-Service $AgentName -Force -ErrorAction SilentlyContinue
- $scOutput = sc.exe delete $AgentName 2>&1
- if ($LASTEXITCODE -ne 0) {
- Write-Host "Warning: Failed to delete service: $scOutput" -ForegroundColor Yellow
- } else {
- Write-Host "Service deleted successfully" -ForegroundColor Green
+ try {
+ if (Test-Path "$InstallDir\$BinaryName") {
+ Remove-Item "$InstallDir\$BinaryName" -Force -ErrorAction Stop
}
- } else {
- Write-Host "Service '$AgentName' not found (already removed)" -ForegroundColor Yellow
- }
-
- if (Test-Path $StateDir) {
- Remove-Item $StateDir -Recurse -Force -ErrorAction SilentlyContinue
+ if (Test-Path $StateDir) {
+ Remove-Item $StateDir -Recurse -Force -ErrorAction Stop
+ }
+ } catch {
+ Show-Error "Agent service was removed, but local cleanup did not complete. $_"
+ Exit 1
}
- Remove-Item "$InstallDir\$BinaryName" -Force -ErrorAction SilentlyContinue
Write-Host "Uninstallation complete." -ForegroundColor Green
Exit 0
}
@@ -974,15 +1041,14 @@ if ($CommandAuthority -eq 'monitoring-only' -and $EnableCommands) {
Exit 1
}
-# Stop existing service if running
-if (Get-Service $AgentName -ErrorAction SilentlyContinue) {
- Write-Host "Removing existing $AgentName service..." -ForegroundColor Yellow
- Stop-Service $AgentName -Force -ErrorAction SilentlyContinue
- $scOutput = sc.exe delete $AgentName 2>&1
- if ($LASTEXITCODE -ne 0) {
- Write-Host "Warning: Failed to delete existing service: $scOutput" -ForegroundColor Yellow
- }
- Start-Sleep -Seconds 2
+# Confirm existing service stopped and absent before changing the executable,
+# token or persisted connection. A failed read/stop/delete is not an upgrade.
+try {
+ Remove-PulseService
+} catch {
+ Cleanup
+ Show-Error "Cannot replace '$AgentName'; existing binary and agent state have been retained. $_"
+ Exit 1
}
# Move temp file to final location
diff --git a/scripts/installtests/install_ps1_test.go b/scripts/installtests/install_ps1_test.go
index 1c3790e754..0d90ee9e8e 100644
--- a/scripts/installtests/install_ps1_test.go
+++ b/scripts/installtests/install_ps1_test.go
@@ -1,8 +1,10 @@
package installtests
import (
+ "fmt"
"os"
"os/exec"
+ "runtime"
"strings"
"testing"
)
@@ -515,7 +517,7 @@ func TestInstallPS1ClearsPersistedStateAfterUninstall(t *testing.T) {
script := string(content)
required := []string{
`if (Test-Path $StateDir) {`,
- `Remove-Item $StateDir -Recurse -Force -ErrorAction SilentlyContinue`,
+ `Remove-Item $StateDir -Recurse -Force -ErrorAction Stop`,
`Write-Host "Uninstallation complete." -ForegroundColor Green`,
}
for _, needle := range required {
@@ -604,3 +606,112 @@ func TestWindowsAgentLifecycleHarnessChecksEveryNativeExit(t *testing.T) {
})
}
}
+
+// Native PowerShell controls execute the production functions, not a separate
+// model. The Linux proof's missing runtime remains an explicit skip; the
+// existing Windows native workflow selects these tests and uses PowerShell 5.1.
+func TestInstallPS1ServiceRemovalRuntime(t *testing.T) {
+ binary := "pwsh"
+ if runtime.GOOS == "windows" {
+ binary = "powershell.exe"
+ }
+ powerShell, err := exec.LookPath(binary)
+ if err != nil {
+ t.Skipf("%s not installed; native service-removal controls remain unexecuted", binary)
+ }
+ cmd := exec.Command(powerShell, "-NoLogo", "-NoProfile", "-NonInteractive", "-File",
+ repoFile("scripts", "installtests", "windows_service_removal_controls.ps1"),
+ "-InstallerPath", repoFile("scripts", "install.ps1"))
+ output, err := cmd.CombinedOutput()
+ if err != nil {
+ t.Fatalf("production service-removal controls failed: %v\n%s", err, output)
+ }
+ t.Logf("%s", output)
+}
+
+func windowsRemovalContract(script string) error {
+ for _, needle := range []string{
+ `return Get-Service -Name $AgentName -ErrorAction Stop`,
+ `$_.FullyQualifiedErrorId.Split(',')[0] -eq 'NoServiceFoundForGivenName'`,
+ `$_.CategoryInfo.Category -eq [System.Management.Automation.ErrorCategory]::ObjectNotFound`,
+ `param([int]$TimeoutSeconds = 30)`,
+ `$service.Stop()`,
+ `$service.WaitForStatus([System.ServiceProcess.ServiceControllerStatus]::Stopped,`,
+ `[TimeSpan]::FromSeconds($TimeoutSeconds))`,
+ `$service.Refresh()`,
+ `throw "Service '$AgentName' is not confirmed stopped."`,
+ `} finally {`,
+ `$service.Dispose()`,
+ `throw "Failed to delete service '$AgentName': $scOutput"`,
+ `$remaining = Get-PulseService`,
+ `$remaining.Dispose()`,
+ `$elapsed.Elapsed.TotalSeconds -ge $TimeoutSeconds`,
+ `throw "Service '$AgentName' is still present after deletion.`,
+ } {
+ if !strings.Contains(script, needle) {
+ return fmt.Errorf("missing removal boundary %q", needle)
+ }
+ }
+ if strings.Contains(script, "Stop-Service") && strings.Contains(script, "Stop-Service $AgentName") {
+ return fmt.Errorf("unbounded or suppressed stop bypasses the shared boundary")
+ }
+ // Pin each real caller's refusal before any remote deregistration, binary
+ // mutation or state mutation. A helper test alone cannot prove its wiring.
+ for _, boundary := range []struct{ start, end, refusal string }{
+ {"# --- Uninstall Logic ---", "# Try to notify the Pulse server", "Cannot uninstall"},
+ {"# Confirm existing service stopped and absent", "# Move temp file to final location", "Cannot replace"},
+ } {
+ start := strings.Index(script, boundary.start)
+ end := strings.Index(script, boundary.end)
+ if start < 0 || end <= start {
+ return fmt.Errorf("missing caller mutation boundary %s", boundary.start)
+ }
+ gate := script[start:end]
+ call := strings.Index(gate, "Remove-PulseService")
+ refusal := strings.Index(gate, boundary.refusal)
+ if call < 0 || refusal <= call || !strings.Contains(gate[call:refusal], "} catch {") ||
+ !strings.Contains(gate[refusal:], "Exit 1") {
+ return fmt.Errorf("caller can continue after removal failure: %s", boundary.start)
+ }
+ }
+ return nil
+}
+
+func TestInstallPS1ServiceRemovalRefusesUnknownRuntimeBeforeMutation(t *testing.T) {
+ content, err := os.ReadFile(repoFile("scripts", "install.ps1"))
+ if err != nil {
+ t.Fatal(err)
+ }
+ script := string(content)
+ if err := windowsRemovalContract(script); err != nil {
+ t.Fatal(err)
+ }
+ for name, needle := range map[string]string{
+ "unknown is not absent": `return Get-Service -Name $AgentName -ErrorAction Stop`,
+ "stop is observed": `$service.WaitForStatus([System.ServiceProcess.ServiceControllerStatus]::Stopped,`,
+ "stopped is rechecked": `throw "Service '$AgentName' is not confirmed stopped."`,
+ "delete exit is retained": `throw "Failed to delete service '$AgentName': $scOutput"`,
+ "delete is not absence": `$remaining = Get-PulseService`,
+ "pending delete is bounded": `$elapsed.Elapsed.TotalSeconds -ge $TimeoutSeconds`,
+ "uninstall refuses": `Cannot uninstall`,
+ "replacement refuses": `Cannot replace`,
+ } {
+ t.Run(name, func(t *testing.T) {
+ if err := windowsRemovalContract(strings.Replace(script, needle, "", 1)); err == nil {
+ t.Fatal("accepted a removed safety boundary")
+ }
+ })
+ }
+ // An exact-source proof can supply the complete actual parent as an
+ // independently hash-bound input. Do not make future CI depend on HEAD^.
+ if parentPath := os.Getenv("PULSE_WINDOWS_REMOVAL_PARENT"); parentPath != "" {
+ parent, err := os.ReadFile(parentPath)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := windowsRemovalContract(string(parent)); err == nil {
+ t.Fatal("accepted the parent's suppressed stop/delete failures")
+ }
+ t.Log("rejected the complete supplied parent installer")
+ }
+}
diff --git a/scripts/installtests/windows_service_removal_controls.ps1 b/scripts/installtests/windows_service_removal_controls.ps1
new file mode 100644
index 0000000000..7078e95762
--- /dev/null
+++ b/scripts/installtests/windows_service_removal_controls.ps1
@@ -0,0 +1,138 @@
+param([Parameter(Mandatory = $true)][string]$InstallerPath)
+
+$ErrorActionPreference = 'Stop'
+$AgentName = 'PulseAgent'
+
+# Load only the production boundary. No administrator check, download, real
+# SCM call or installer mutation runs in this failure-control harness.
+$errors = $null
+$ast = [System.Management.Automation.Language.Parser]::ParseFile(
+ $InstallerPath, [ref]$null, [ref]$errors)
+if ($errors.Count) { throw "Installer parse failed: $errors" }
+foreach ($name in @('Get-PulseService', 'Remove-PulseService')) {
+ $functions = @($ast.FindAll({ param($node)
+ $node -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $node.Name -eq $name
+ }, $true))
+ if ($functions.Count -ne 1) { throw "Expected one production function: $name" }
+ Invoke-Expression $functions[0].Extent.Text
+}
+
+function Assert-Control {
+ param([bool]$Condition, [string]$Message)
+ if (-not $Condition) { throw "$script:scenario : $Message; events=$($script:events -join ',')" }
+}
+
+function Get-Service {
+ [CmdletBinding()]
+ param([string]$Name)
+ Assert-Control ($Name -eq 'PulseAgent') 'queried a different service'
+ $script:events.Add('read')
+ if ($script:scenario -eq 'read-denied' -or
+ ($script:deleted -and $script:scenario -eq 'read-denied-after-delete')) {
+ throw 'SCM access denied'
+ }
+ $absent = $script:scenario -eq 'absent' -or
+ ($script:deleted -and $script:scenario -ne 'delete-pending' -and
+ ($script:scenario -ne 'delete-delayed' -or $script:afterDeleteReads -ge 1))
+ if ($absent -or $script:scenario -eq 'wrong-not-found-category') {
+ $category = [System.Management.Automation.ErrorCategory]::ObjectNotFound
+ if ($script:scenario -eq 'wrong-not-found-category') {
+ $category = [System.Management.Automation.ErrorCategory]::PermissionDenied
+ }
+ $record = [System.Management.Automation.ErrorRecord]::new(
+ [System.InvalidOperationException]::new('test-only SCM result'),
+ 'NoServiceFoundForGivenName', $category, $Name)
+ $PSCmdlet.ThrowTerminatingError($record)
+ }
+ if ($script:deleted) { $script:afterDeleteReads++ }
+ return $script:controller
+}
+
+function sc.exe {
+ param([string]$Verb, [string]$Name)
+ Assert-Control ($Verb -eq 'delete' -and $Name -eq 'PulseAgent') 'unexpected native operation'
+ Assert-Control ($script:controller.Status -eq 'Stopped') 'delete before confirmed stop'
+ Assert-Control ($script:events.Contains('dispose')) 'retained controller handle before delete'
+ $script:events.Add('delete')
+ $global:LASTEXITCODE = 0
+ if ($script:scenario -eq 'delete-refused') { $global:LASTEXITCODE = 5 }
+ $script:deleted = $global:LASTEXITCODE -eq 0
+ return 'test-only sc.exe output'
+}
+
+function Start-Sleep {
+ param([int]$Milliseconds)
+ Assert-Control ($Milliseconds -eq 200) 'unexpected poll interval'
+ $script:events.Add('sleep')
+}
+
+$scenarios = @('absent', 'stopped', 'running', 'stop-pending', 'read-denied',
+ 'wrong-not-found-category', 'stop-refused', 'stop-timeout', 'stop-not-observed',
+ 'delete-refused', 'read-denied-after-delete', 'delete-delayed', 'delete-pending')
+foreach ($script:scenario in $scenarios) {
+ $script:events = [System.Collections.Generic.List[string]]::new()
+ $script:deleted = $false
+ $script:afterDeleteReads = 0
+ $global:LASTEXITCODE = 0
+ $status = 'Running'
+ if ($script:scenario -eq 'stopped') { $status = 'Stopped' }
+ if ($script:scenario -eq 'stop-pending') { $status = 'StopPending' }
+ $script:controller = [PSCustomObject]@{ Status = $status }
+ $script:controller | Add-Member ScriptMethod Refresh {
+ $script:events.Add('refresh')
+ }
+ $script:controller | Add-Member ScriptMethod Stop {
+ $script:events.Add('stop')
+ if ($script:scenario -eq 'stop-refused') { throw 'test-only stop refusal' }
+ $this.Status = 'StopPending'
+ }
+ $script:controller | Add-Member ScriptMethod WaitForStatus {
+ param($ExpectedStatus, [TimeSpan]$Timeout)
+ $script:events.Add('wait')
+ Assert-Control ($ExpectedStatus.ToString() -eq 'Stopped') 'waited for a non-stopped state'
+ $expectedTimeout = 30
+ if ($script:scenario -eq 'delete-pending') { $expectedTimeout = 0 }
+ Assert-Control ($Timeout.TotalSeconds -eq $expectedTimeout) 'stop wait is not bounded'
+ if ($script:scenario -eq 'stop-timeout') { throw 'test-only stop timeout' }
+ if ($script:scenario -ne 'stop-not-observed') { $this.Status = 'Stopped' }
+ }
+ $script:controller | Add-Member ScriptMethod Dispose {
+ $script:events.Add('dispose')
+ }
+
+ $failure = $null
+ try {
+ if ($script:scenario -eq 'delete-pending') {
+ Remove-PulseService -TimeoutSeconds 0
+ } else {
+ Remove-PulseService
+ }
+ # Model the caller's destructive continuation, not SCM implementation.
+ $script:events.Add('binary-mutation')
+ $script:events.Add('state-mutation')
+ $script:events.Add('remote-deregistration')
+ } catch {
+ $failure = $_
+ }
+ $success = $script:scenario -in @('absent', 'stopped', 'running', 'stop-pending', 'delete-delayed')
+ Assert-Control (($null -eq $failure) -eq $success) "unexpected outcome: $failure"
+ foreach ($mutation in @('binary-mutation', 'state-mutation', 'remote-deregistration')) {
+ Assert-Control ($script:events.Contains($mutation) -eq $success) "incorrect continuation: $mutation"
+ }
+ if ($script:scenario -eq 'absent') {
+ Assert-Control ($script:events.Count -eq 4) 'absent service caused SCM mutation'
+ } elseif ($script:scenario -notin @('read-denied', 'wrong-not-found-category')) {
+ Assert-Control ($script:events.Contains('dispose')) 'controller not disposed on exit'
+ }
+ if ($script:scenario -in @('stopped', 'stop-pending')) {
+ Assert-Control (-not $script:events.Contains('stop')) 'stop was unnecessarily re-requested'
+ }
+ if ($script:scenario -in @('stop-refused', 'stop-timeout', 'stop-not-observed', 'read-denied', 'wrong-not-found-category')) {
+ Assert-Control (-not $script:events.Contains('delete')) 'failed stop/read still deleted service'
+ }
+ if ($script:scenario -eq 'delete-delayed') {
+ Assert-Control ($script:events.Contains('sleep')) 'did not observe delayed SCM deletion'
+ }
+ Write-Output "PASS $script:scenario : $($script:events -join ',')"
+}
+Write-Output "PASS all $($scenarios.Count) production service-removal controls; no real SCM or file mutation"