Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
fix(installer): contain destructive reset and bind lifecycle proofs
pulse-triage[bot]af32cbdcbfdd
diff --git a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md b/docs/release-control/v6/internal/subsystems/agent-lifecycle.mdindex 64328bd8fe..df3af36c2d 100644--- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md+++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md@@ -3773,6 +3773,19 @@ Agent` secondary handoff against the live setup wizard instead of relying## Current State+### Registered Windows native-exit completion proof (8 October 2026)++The workflow's seven immediate native exit checks and the lifecycle harness's+five checks are verified from `scripts/installtests/install_ps1_test.go`, the+registry-named completion proof for this boundary. The existing assertion+methods have moved there unchanged from `native_windows_exit_test.go`; the+shared parsers and native PowerShell execution controls remain in that file.+Each removed guard and either `continue-on-error` mutation must still be+rejected. No registry relaxation or runtime command change is made. Offline+Linux source proof must retain PowerShell-unavailable skips; actual native+Windows service lifecycle and exit propagation remain hosted CI obligations.++Notification destination edits in the shared `internal/api/` subtree preserveonly submitted masked values, honour neighbouring explicit map edits/removalsand reject unrecoverable masks before publication. Restored custom-fielddiff --git a/docs/release-control/v6/internal/subsystems/deployment-installability.md b/docs/release-control/v6/internal/subsystems/deployment-installability.mdindex 85eea5f7eb..ed43c42e6d 100644--- a/docs/release-control/v6/internal/subsystems/deployment-installability.md+++ b/docs/release-control/v6/internal/subsystems/deployment-installability.md@@ -3009,6 +3009,59 @@ artifact-selection behaviour.## Current State+### Registered Windows native-exit completion proof (8 October 2026)++The workflow's seven immediate native exit checks and the lifecycle harness's+five checks are verified from `scripts/installtests/install_ps1_test.go`, the+registry-named completion proof for this boundary. The existing assertion+methods have moved there unchanged from `native_windows_exit_test.go`; the+shared parsers and native PowerShell execution controls remain in that file.+Each removed guard and either `continue-on-error` mutation must still be+rejected. No registry relaxation or runtime command change is made. Offline+Linux source proof must retain PowerShell-unavailable skips; actual native+Windows service lifecycle and exit propagation remain hosted CI obligations.+++### Server reset observes every writer before deleting configuration (8 October 2026)++The root installer's existing `--reset` intent retains the binary and unit+files. Before changing services or data it records bounded, usable load,+active and enablement states for its timers, in-flight updaters and servers.+Default historical `pulse`/`pulse-backend` writers sharing configuration are+included; an explicit custom instance affects only its selected units.+Missing optional units are inert; a missing selected server, failed/empty+query, unknown or transitioning state refuses reset before any mutation.++Timers stop before updaters, which stop before servers. Only enabled timers+are temporarily disabled, preserving persistent versus runtime enablement;+server/updater enablement is never changed. Roles are recorded explicitly, not+inferred from service-name suffixes: a custom server named `*-update` is still+a server and must regain prior liveness. Stop/disable results require+readback and every writer is reconciled again immediately before deletion.+A failed stop, changed unit identity, reactivation or enablement drift preserves+configuration, reports incomplete reset and never attempts automatic restart.+A failed deletion leaves services/timers stopped for manual reconciliation.++After deletion only prior-active servers are restarted, with observed active+state and unchanged load/enablement required before timers are restored.+Prior-inactive/failed servers stay stopped. Timers regain their recorded+active and enabled states, with final readback required for completion. An+interrupted updater is deliberately **not replayed**, and this is reported.+Any restoration failure is an incomplete action, not success. This is bounded+quiescence, not an exclusive lock against an independent administrator starting+a unit after observation; concurrent manual lifecycle operations remain unsafe.++`server_reset_safety_test.go` executes the actual sourced reset path against+stateful on-disk systemctl doubles and fixture-only deletion. It covers all+writer observations, ignored stops/disablement, readback failure, reactivation,+prior state/runtime enablement, updater non-replay, restoration/deletion failures,+optional-unit absence, mixed idle-server/active-timer restoration and custom-instance+isolation. Stalled observation and stop controls exercise the real timeout deadline.+The root installer test+also pins bounds on every systemd call. These are offline source controls, not+native systemd, signed-installer or installed reset acceptance.++The helper remains network-isolated on all targets, and restart activates the installer-owned replacement rather than retaining old code after reload. Published security guidance discloses the PVE IPC limitation and partial-result semantics. This is not a published release or installed PVE acceptance.### Demo runtime footprint qualification@@ -7210,3 +7263,4 @@ contract checks the helper staging, unchanged cache/graph inputs and boundedcommand. These are offline command proofs, not a hosted Docker build, publicproxy recovery, E2E acceptance or installed customer result. The hosted checksstill have to build the image and execute the unchanged suites.+diff --git a/install.sh b/install.shindex 622fa446ba..b2eb5300f2 100755--- a/install.sh+++ b/install.sh@@ -5560,34 +5560,171 @@ uninstall_pulse() {exit 0}-# Reset function+# Reset retains the install and unit files. Observe every possible writer before+# mutation; only timers lose enablement temporarily. Never replay an interrupted+# updater, or restart services after an uncertain stop/deletion.+read_pulse_reset_unit_state() {+ local unit="$1"+ PULSE_RESET_LOAD="" PULSE_RESET_ACTIVE="" PULSE_RESET_ENABLED=""+ if ! PULSE_RESET_LOAD=$(timeout -k 1 5 systemctl show "$unit" --property=LoadState --value 2>/dev/null) ||+ ! PULSE_RESET_ACTIVE=$(timeout -k 1 5 systemctl show "$unit" --property=ActiveState --value 2>/dev/null); then+ print_error "Cannot observe unit ($unit); reset incomplete. No automatic recovery will be attempted; reconcile service state before retrying."+ return 1+ fi+ if [[ "$PULSE_RESET_LOAD" == "not-found" && "$PULSE_RESET_ACTIVE" == "inactive" ]]; then+ return 0+ fi+ case "$PULSE_RESET_LOAD" in+ loaded|masked) ;;+ *) print_error "Unknown unit load state ($unit); reset incomplete"; return 1 ;;+ esac+ case "$PULSE_RESET_ACTIVE" in+ active|inactive|failed) ;;+ *) print_error "Unsettled unit state ($unit); reset incomplete"; return 1 ;;+ esac+ if ! PULSE_RESET_ENABLED=$(timeout -k 1 5 systemctl show "$unit" --property=UnitFileState --value 2>/dev/null); then+ print_error "Cannot observe unit enablement ($unit); reset incomplete"+ return 1+ fi+ case "$PULSE_RESET_ENABLED" in+ enabled|enabled-runtime|disabled|static|indirect|masked|masked-runtime|linked|linked-runtime) ;;+ *) print_error "Unknown unit enablement ($unit); reset incomplete"; return 1 ;;+ esac+}++confirm_pulse_reset_unit_stopped() {+ local unit="$1" expected_load="$2" expected_enabled="$3"+ read_pulse_reset_unit_state "$unit" || return 1+ if [[ "$PULSE_RESET_ACTIVE" != "inactive" || "$PULSE_RESET_LOAD" != "$expected_load" || "$PULSE_RESET_ENABLED" != "$expected_enabled" ]]; then+ print_error "Unit quiescence changed or is unconfirmed ($unit); reset incomplete. Files are preserved; no services will be restarted. Reconcile service state before retrying."+ return 1+ fi+}++# Reset function: the caller's existing --reset intent is unchanged.reset_pulse() {check_rootprint_headerecho -e "\033[0;33mResetting Pulse configuration...\033[0m"echo-- # Detect service name- local service_name++ local service_name unit i prior_enabledservice_name=$(detect_service_name)-- # Stop service- if systemctl is-active --quiet "$service_name" 2>/dev/null; then- echo "Stopping $service_name..."- systemctl stop "$service_name"+ service_name="${service_name%.service}.service"+ local units=("$UPDATE_TIMER_UNIT") roles=(timer)+ # Historical default aliases share the same configuration. Custom instances+ # must not stop, enable or restart another installation's units.+ if [[ "$SERVICE_NAME_EXPLICIT" != "true" && "$UPDATE_TIMER_UNIT" != "pulse-backend-update.timer" ]]; then+ units+=(pulse-backend-update.timer); roles+=(timer)+ fi+ units+=("$UPDATE_SERVICE_UNIT"); roles+=(updater)+ if [[ "$SERVICE_NAME_EXPLICIT" != "true" && "$UPDATE_SERVICE_UNIT" != "pulse-backend-update.service" ]]; then+ units+=(pulse-backend-update.service); roles+=(updater)+ fi+ units+=("$service_name"); roles+=(server)+ if [[ "$SERVICE_NAME_EXPLICIT" != "true" ]]; then+ if [[ "$service_name" != "pulse.service" ]]; then units+=(pulse.service); roles+=(server); fi+ if [[ "$service_name" != "pulse-backend.service" ]]; then units+=(pulse-backend.service); roles+=(server); fifi-- # Remove config but keep binary+ local loads=() active_states=() enabled_states=() stopped_enabled_states=()+ # A failed observation anywhere must precede every stop and deletion.+ for unit in "${units[@]}"; do+ read_pulse_reset_unit_state "$unit" || return 1+ if [[ "$unit" == "$service_name" && "$PULSE_RESET_LOAD" == "not-found" ]]; then+ print_error "Pulse service is missing ($unit); refusing configuration reset"+ return 1+ fi+ loads+=("$PULSE_RESET_LOAD")+ active_states+=("$PULSE_RESET_ACTIVE")+ enabled_states+=("$PULSE_RESET_ENABLED")+ stopped_enabled_states+=("$PULSE_RESET_ENABLED")+ done+ for i in "${!units[@]}"; do+ unit="${units[$i]}"+ if [[ "${active_states[$i]}" != "inactive" ]]; then+ if ! timeout -k 1 5 systemctl stop "$unit"; then+ print_error "Cannot stop unit ($unit); reset incomplete. Configuration is preserved; no services will be restarted. Reconcile service state before retrying."+ return 1+ fi+ fi+ prior_enabled="${enabled_states[$i]}"+ if [[ "${roles[$i]}" == timer ]]; then+ case "$prior_enabled" in+ enabled)+ if ! timeout -k 1 5 systemctl disable "$unit"; then+ print_error "Cannot disable timer ($unit); reset incomplete; configuration is preserved"+ return 1+ fi+ stopped_enabled_states[$i]=disabled+ ;;+ enabled-runtime)+ if ! timeout -k 1 5 systemctl disable --runtime "$unit"; then+ print_error "Cannot disable runtime timer ($unit); reset incomplete; configuration is preserved"+ return 1+ fi+ stopped_enabled_states[$i]=disabled+ ;;+ esac+ fi+ confirm_pulse_reset_unit_stopped "$unit" "${loads[$i]}" "${stopped_enabled_states[$i]}" || return 1+ done+ # Recheck ALL writers immediately before deletion, including an earlier timer+ # that reactivated or an updater that appeared while the server was stopping.+ for i in "${!units[@]}"; do+ confirm_pulse_reset_unit_stopped "${units[$i]}" "${loads[$i]}" "${stopped_enabled_states[$i]}" || return 1+ done+echo "Removing configuration and data..."- rm -rf "$CONFIG_DIR"/*-- # Restart service- echo "Starting $service_name with fresh configuration..."- systemctl start "$service_name"-+ if ! rm -rf -- "$CONFIG_DIR"/*; then+ print_error "Configuration deletion failed; reset incomplete. Services and timers remain stopped; inspect retained data before recovery."+ return 1+ fi++ # Restore only previously active servers. A prior inactive/failed service is+ # not consent to start it. Interrupted updater jobs are deliberately not replayed.+ for i in "${!units[@]}"; do+ unit="${units[$i]}"+ [[ "${roles[$i]}" == server ]] || continue+ if [[ "${active_states[$i]}" == "active" ]]; then+ if ! timeout -k 1 5 systemctl start "$unit"; then+ print_error "Cannot restart prior-active Pulse ($unit); reset incomplete. Timers remain stopped; reconcile service state before recovery."+ return 1+ fi+ read_pulse_reset_unit_state "$unit" || return 1+ if [[ "$PULSE_RESET_ACTIVE" != "active" || "$PULSE_RESET_LOAD" != "${loads[$i]}" || "$PULSE_RESET_ENABLED" != "${enabled_states[$i]}" ]]; then+ print_error "Prior-active Pulse restoration is unconfirmed ($unit); reset incomplete. Timers remain stopped."+ return 1+ fi+ fi+ done+ for i in "${!units[@]}"; do+ unit="${units[$i]}"+ [[ "${roles[$i]}" == timer ]] || continue+ case "${enabled_states[$i]}" in+ enabled) timeout -k 1 5 systemctl enable "$unit" || { print_error "Timer enablement restoration failed ($unit); reset incomplete"; return 1; } ;;+ enabled-runtime) timeout -k 1 5 systemctl enable --runtime "$unit" || { print_error "Runtime timer enablement restoration failed ($unit); reset incomplete"; return 1; } ;;+ esac+ confirm_pulse_reset_unit_stopped "$unit" "${loads[$i]}" "${enabled_states[$i]}" || return 1+ if [[ "${active_states[$i]}" == "active" ]]; then+ timeout -k 1 5 systemctl start "$unit" || { print_error "Timer restart failed ($unit); reset incomplete"; return 1; }+ fi+ done+ # Completion requires actual readback, not successful start/enable commands.+ for i in "${!units[@]}"; do+ unit="${units[$i]}"+ read_pulse_reset_unit_state "$unit" || return 1+ local expected_active="${active_states[$i]}"+ [[ "${roles[$i]}" != updater && "$expected_active" != "failed" ]] || expected_active=inactive+ if [[ "$PULSE_RESET_ACTIVE" != "$expected_active" || "$PULSE_RESET_LOAD" != "${loads[$i]}" || "$PULSE_RESET_ENABLED" != "${enabled_states[$i]}" ]]; then+ print_error "Final unit restoration is unconfirmed ($unit); reset incomplete. Reconcile service state before recovery."+ return 1+ fi+ if [[ "${roles[$i]}" == updater && "${active_states[$i]}" == "active" ]]; then+ print_info "Interrupted updater ($unit) remains stopped and was not replayed."+ fi+ doneecho- echo -e "\033[0;32m✓ Pulse has been reset to fresh configuration\033[0m"- echo "Access Pulse at: http://$(hostname -I | awk '{print $1}'):$(current_frontend_port)"+ echo -e "\033[0;32m✓ Pulse configuration has been reset; prior-active server and timer states are restored\033[0m"exit 0}diff --git a/scripts/installtests/install_ps1_test.go b/scripts/installtests/install_ps1_test.goindex 9931fab477..1c3790e754 100644--- a/scripts/installtests/install_ps1_test.go+++ b/scripts/installtests/install_ps1_test.go@@ -549,3 +549,58 @@ func TestInstallPS1RequiresPinnedSignatureVerificationForReleaseDownloads(t *tes}}}++// These native exit checks live with the registry-named Windows lifecycle+// completion proof. Their shared parsers and native execution controls remain+// in native_windows_exit_test.go; no check or adverse control is removed.+func TestWindowsAgentLifecycleWorkflowChecksEveryNativeExit(t *testing.T) {+ steps := nativeWindowsExitSteps(t)+ guards, err := nativeWindowsExitGuards(steps)+ if err != nil {+ t.Fatal(err)+ }+ for _, guard := range guards {+ t.Run(guard, func(t *testing.T) {+ changed := append([]nativeWindowsExitStep(nil), steps...)+ for i := range changed {+ // Reproduce the parent's unchecked command at each boundary.+ changed[i].Run = strings.Replace(changed[i].Run, guard, "", 1)+ }+ if _, err := nativeWindowsExitGuards(changed); err == nil {+ t.Fatal("accepted a native failure that a later successful command could hide")+ }+ })+ }+ for _, stepName := range []string{"Build and execute native Windows agent", "Exercise native Windows service lifecycle"} {+ t.Run(stepName+" cannot tolerate failure", func(t *testing.T) {+ changed := append([]nativeWindowsExitStep(nil), steps...)+ for i := range changed {+ if changed[i].Name == stepName {+ changed[i].ContinueOnError = true+ }+ }+ if _, err := nativeWindowsExitGuards(changed); err == nil {+ t.Fatal("accepted a continue-on-error native Windows proof")+ }+ })+ }+}++func TestWindowsAgentLifecycleHarnessChecksEveryNativeExit(t *testing.T) {+ content, err := os.ReadFile(repoFile("scripts", "installtests", "windows_agent_lifecycle.ps1"))+ if err != nil {+ t.Fatal(err)+ }+ guards, err := nativeWindowsHarnessExitGuards(string(content))+ if err != nil {+ t.Fatal(err)+ }+ for _, guard := range guards {+ t.Run(guard, func(t *testing.T) {+ changed := strings.Replace(string(content), guard, "", 1)+ if _, err := nativeWindowsHarnessExitGuards(changed); err == nil {+ t.Fatal("accepted failed version/installer/service query evidence")+ }+ })+ }+}diff --git a/scripts/installtests/native_windows_exit_test.go b/scripts/installtests/native_windows_exit_test.goindex 33ddfcaadd..a21f193e4e 100644--- a/scripts/installtests/native_windows_exit_test.go+++ b/scripts/installtests/native_windows_exit_test.go@@ -88,39 +88,6 @@ func nativeWindowsExitGuards(steps []nativeWindowsExitStep) ([]string, error) {return guards, nil}-func TestWindowsAgentLifecycleWorkflowChecksEveryNativeExit(t *testing.T) {- steps := nativeWindowsExitSteps(t)- guards, err := nativeWindowsExitGuards(steps)- if err != nil {- t.Fatal(err)- }- for _, guard := range guards {- t.Run(guard, func(t *testing.T) {- changed := append([]nativeWindowsExitStep(nil), steps...)- for i := range changed {- // Reproduce the parent's unchecked command at each boundary.- changed[i].Run = strings.Replace(changed[i].Run, guard, "", 1)- }- if _, err := nativeWindowsExitGuards(changed); err == nil {- t.Fatal("accepted a native failure that a later successful command could hide")- }- })- }- for _, stepName := range []string{"Build and execute native Windows agent", "Exercise native Windows service lifecycle"} {- t.Run(stepName+" cannot tolerate failure", func(t *testing.T) {- changed := append([]nativeWindowsExitStep(nil), steps...)- for i := range changed {- if changed[i].Name == stepName {- changed[i].ContinueOnError = true- }- }- if _, err := nativeWindowsExitGuards(changed); err == nil {- t.Fatal("accepted a continue-on-error native Windows proof")- }- })- }-}-func nativeWindowsHarnessExitGuards(script string) ([]string, error) {commands := []string{`$version = ((& $AgentPath --version`,@@ -149,25 +116,6 @@ func nativeWindowsHarnessExitGuards(script string) ([]string, error) {return guards, nil}-func TestWindowsAgentLifecycleHarnessChecksEveryNativeExit(t *testing.T) {- content, err := os.ReadFile(repoFile("scripts", "installtests", "windows_agent_lifecycle.ps1"))- if err != nil {- t.Fatal(err)- }- guards, err := nativeWindowsHarnessExitGuards(string(content))- if err != nil {- t.Fatal(err)- }- for _, guard := range guards {- t.Run(guard, func(t *testing.T) {- changed := strings.Replace(string(content), guard, "", 1)- if _, err := nativeWindowsHarnessExitGuards(changed); err == nil {- t.Fatal("accepted failed version/installer/service query evidence")- }- })- }-}-func TestWindowsAgentLifecycleNativeExitChecksStopFollowingCommands(t *testing.T) {shell := "pwsh"if runtime.GOOS == "windows" {diff --git a/scripts/installtests/root_install_sh_test.go b/scripts/installtests/root_install_sh_test.goindex f80d106231..6948a0e0c2 100644--- a/scripts/installtests/root_install_sh_test.go+++ b/scripts/installtests/root_install_sh_test.go@@ -2549,3 +2549,14 @@ fit.Fatalf("key replacement failure changed original: %v", err)}}++func TestRootInstallResetBoundsEverySystemdObservationAndMutation(t *testing.T) {+ for _, name := range []string{"reset_pulse", "read_pulse_reset_unit_state", "confirm_pulse_reset_unit_stopped"} {+ body := extractRootInstallShellFunction(t, name)+ for _, line := range strings.Split(body, "\n") {+ if strings.Contains(line, "systemctl ") && !strings.Contains(line, "timeout -k 1 5 systemctl ") {+ t.Fatalf("%s has unbounded systemctl call: %s", name, line)+ }+ }+ }+}diff --git a/scripts/installtests/server_reset_safety_test.go b/scripts/installtests/server_reset_safety_test.gonew file mode 100644index 0000000000..5c7ea785d9--- /dev/null+++ b/scripts/installtests/server_reset_safety_test.go@@ -0,0 +1,364 @@+package installtests++import (+ "context"+ "os"+ "os/exec"+ "path/filepath"+ "strings"+ "testing"+ "time"+)++// This exercises reset_pulse from the actual sourced installer. All systemctl+// calls pass through the real timeout binary to a stateful executable double.+// Deletion is confined to fixture configuration; no host installation is used.+func TestRootInstallResetRejectsUnusableObservationsBeforeMutation(t *testing.T) {+ for _, unit := range []string{"pulse.service", "pulse-update.timer", "pulse-update.service", "pulse-backend.service", "pulse-backend-update.timer", "pulse-backend-update.service"} {+ for _, mode := range []string{"query-fails", "empty-load", "unknown-load", "empty-active", "unsettled", "empty-enabled", "unknown-enabled", "enabled-query-fails"} {+ t.Run(unit+"/"+mode, func(t *testing.T) {+ r := runServerResetFixture(t, mode, unit, "active", "enabled", false)+ assertResetIncomplete(t, r, true)+ for _, action := range []string{"stop ", "disable ", "start ", "enable "} {+ if strings.Contains(r.calls, action) {+ t.Fatalf("bad initial observation allowed mutation %s:\n%s", action, r.calls)+ }+ }+ })+ }+ }+}++func TestRootInstallResetPreservesDataOnUnconfirmedQuiescence(t *testing.T) {+ for _, unit := range []string{"pulse.service", "pulse-update.timer", "pulse-update.service", "pulse-backend.service", "pulse-backend-update.timer", "pulse-backend-update.service"} {+ for _, mode := range []string{"stop-fails", "stop-ignored", "stop-readback-fails", "still-failed", "reactivated"} {+ t.Run(unit+"/"+mode, func(t *testing.T) {+ assertResetIncomplete(t, runServerResetFixture(t, mode, unit, "active", "enabled", false), true)+ })+ }+ }+ for _, unit := range []string{"pulse-update.timer", "pulse-backend-update.timer"} {+ for _, mode := range []string{"disable-fails", "disable-ignored", "reenabled"} {+ t.Run(unit+"/"+mode, func(t *testing.T) {+ assertResetIncomplete(t, runServerResetFixture(t, mode, unit, "active", "enabled", false), true)+ })+ }+ }+}++func TestRootInstallResetRestoresOnlyPriorActiveStates(t *testing.T) {+ for _, active := range []string{"active", "inactive", "failed"} {+ for _, enabled := range []string{"enabled", "enabled-runtime", "disabled", "static", "indirect", "masked", "masked-runtime", "linked", "linked-runtime"} {+ t.Run(active+"/"+enabled, func(t *testing.T) {+ r := runServerResetFixture(t, "normal", "", active, enabled, false)+ if r.err != nil || !strings.Contains(r.output, "Pulse configuration has been reset") || r.configPresent {+ t.Fatalf("reset failed: %v\n%s\n%s", r.err, r.output, r.calls)+ }+ deleteAt := strings.Index(r.calls, "DELETE ")+ if deleteAt < 0 {+ t.Fatal("reset did not delete configuration")+ }+ for _, unit := range resetFixtureUnits(false) {+ if strings.Count(r.calls[:deleteAt], "show "+unit+" --property=ActiveState") < 3 {+ t.Fatalf("%s lacked snapshot, stop readback and final pre-delete reconciliation:\n%s", unit, r.calls)+ }+ expected := active+ if active == "failed" || strings.HasSuffix(unit, "-update.service") {+ expected = "inactive"+ }+ if r.states[unit+".active"] != expected || r.states[unit+".enabled"] != enabled {+ t.Fatalf("%s not restored: %v", unit, r.states)+ }+ if strings.HasSuffix(unit, "-update.service") && strings.Contains(r.calls, "start "+unit) {+ t.Fatalf("interrupted updater was replayed:\n%s", r.calls)+ }+ }+ if active != "active" && strings.Contains(r.calls, "start ") {+ t.Fatalf("originally inactive/failed unit was started:\n%s", r.calls)+ }+ if active == "active" {+ firstTimer := strings.Index(r.calls, "stop pulse-update.timer")+ lastTimer := strings.Index(r.calls, "stop pulse-backend-update.timer")+ firstUpdate := strings.Index(r.calls, "stop pulse-update.service")+ lastUpdate := strings.Index(r.calls, "stop pulse-backend-update.service")+ server := strings.Index(r.calls, "stop pulse.service")+ if !(firstTimer < lastTimer && lastTimer < firstUpdate && firstUpdate < lastUpdate && lastUpdate < server && server < deleteAt) {+ t.Fatalf("unsafe writer stop ordering:\n%s", r.calls)+ }+ if !strings.Contains(r.output, "remains stopped and was not replayed") {+ t.Fatalf("interrupted updater disposition is hidden:\n%s", r.output)+ }+ }+ if enabled == "enabled-runtime" && (!strings.Contains(r.calls, "disable --runtime pulse-update.timer") || !strings.Contains(r.calls, "enable --runtime pulse-update.timer")) {+ t.Fatalf("runtime enablement was not preserved:\n%s", r.calls)+ }+ for _, unit := range []string{"pulse.service", "pulse-backend.service", "pulse-update.service", "pulse-backend-update.service"} {+ if strings.Contains(r.calls, "enable "+unit) || strings.Contains(r.calls, "disable "+unit) {+ t.Fatalf("non-timer enablement was changed:\n%s", r.calls)+ }+ }+ })+ }+ }+}++func TestRootInstallResetReportsIncompleteDeletionOrRestoration(t *testing.T) {+ for _, tc := range []struct{ mode, target string }{+ {"delete-fails", ""},+ {"start-fails", "pulse.service"},+ {"start-ignored", "pulse.service"},+ {"start-readback-fails", "pulse.service"},+ {"enable-fails", "pulse-update.timer"},+ {"enable-ignored", "pulse-update.timer"},+ {"start-fails", "pulse-update.timer"},+ {"start-ignored", "pulse-update.timer"},+ {"start-readback-fails", "pulse-update.timer"},+ {"final-updater-active", "pulse-update.service"},+ } {+ t.Run(tc.mode+"/"+tc.target, func(t *testing.T) {+ r := runServerResetFixture(t, tc.mode, tc.target, "active", "enabled", false)+ assertResetIncomplete(t, r, tc.mode == "delete-fails")+ if tc.mode == "delete-fails" || tc.target == "pulse.service" {+ if strings.Contains(r.calls, "start pulse-update.timer") || strings.Contains(r.calls, "enable pulse-update.timer") {+ t.Fatalf("timer restored despite uncertain deletion/server recovery:\n%s", r.calls)+ }+ }+ })+ }+}++func TestRootInstallResetEnforcesSystemdDeadlines(t *testing.T) {+ for _, mode := range []string{"query-timeout", "stop-timeout"} {+ t.Run(mode, func(t *testing.T) {+ start := time.Now()+ r := runServerResetFixture(t, mode, "pulse-update.service", "active", "enabled", false)+ assertResetIncomplete(t, r, true)+ if elapsed := time.Since(start); elapsed >= 15*time.Second {+ t.Fatalf("bounded systemctl action took %s", elapsed)+ }+ })+ }+}++func TestRootInstallResetDoesNotStartAnIdleServerWhenTimerWasActive(t *testing.T) {+ r := runServerResetFixture(t, "mixed", "", "active", "enabled", false)+ if r.err != nil || r.configPresent || strings.Contains(r.calls, "start pulse.service") || strings.Contains(r.calls, "start pulse-backend.service") {+ t.Fatalf("idle/failed server was restarted: %v\n%s\n%s", r.err, r.output, r.calls)+ }+ if !strings.Contains(r.calls, "start pulse-update.timer") || r.states["pulse.service.active"] != "inactive" {+ t.Fatalf("timer/server states not independently restored: %v\n%s", r.states, r.calls)+ }+}++func TestRootInstallResetKeepsCustomInstanceIsolated(t *testing.T) {+ r := runServerResetFixture(t, "normal", "", "active", "enabled", true)+ if r.err != nil || r.configPresent {+ t.Fatalf("custom reset failed: %v\n%s\n%s", r.err, r.output, r.calls)+ }+ for _, unit := range resetFixtureUnits(false) {+ if strings.Contains(r.calls, unit) {+ t.Fatalf("custom reset touched other installation %s:\n%s", unit, r.calls)+ }+ }+}++func TestRootInstallResetKeepsUpdateSuffixedServerRole(t *testing.T) {+ r := runServerResetFixture(t, "update-named-server", "", "active", "enabled", true)+ if r.err != nil || r.states["pulse-other-update.service.active"] != "active" || !strings.Contains(r.calls, "start pulse-other-update.service") || strings.Contains(r.calls, "start pulse-other-update-update.service") {+ t.Fatalf("server name was confused with updater role: %v\n%s\n%s", r.err, r.output, r.calls)+ }+}++func TestRootInstallResetHandlesAbsentOptionalUnits(t *testing.T) {+ r := runServerResetFixture(t, "optional-absent", "", "inactive", "disabled", false)+ if r.err != nil || r.configPresent || strings.Contains(r.calls, "stop ") || strings.Contains(r.calls, "start ") || strings.Contains(r.calls, "disable ") {+ t.Fatalf("absent optional units should not cause mutation: %v\n%s\n%s", r.err, r.output, r.calls)+ }+ assertResetIncomplete(t, runServerResetFixture(t, "server-absent", "", "inactive", "disabled", false), true)+}++type serverResetResult struct {+ output, calls string+ err error+ configPresent bool+ states map[string]string+}++func assertResetIncomplete(t *testing.T, r serverResetResult, preserve bool) {+ t.Helper()+ if r.err == nil || strings.Contains(r.output, "Pulse configuration has been reset") || strings.Contains(r.output, "Pulse has been reset") {+ t.Fatalf("incomplete reset claimed completion: %v\n%s\n%s", r.err, r.output, r.calls)+ }+ if preserve && (!r.configPresent || strings.Contains(r.calls, "start ") || (strings.Contains(r.calls, "DELETE ") && !strings.Contains(r.output, "Configuration deletion failed"))) {+ t.Fatalf("uncertain pre-deletion state allowed mutation/restart:\n%s\n%s", r.output, r.calls)+ }+}++func resetFixtureUnits(custom bool) []string {+ if custom {+ return []string{"pulse-other-update.timer", "pulse-other-update.service", "pulse-other.service"}+ }+ return []string{"pulse-update.timer", "pulse-backend-update.timer", "pulse-update.service", "pulse-backend-update.service", "pulse.service", "pulse-backend.service"}+}++func runServerResetFixture(t *testing.T, mode, target, active, enabled string, custom bool) serverResetResult {+ t.Helper()+ dir := t.TempDir()+ binDir := filepath.Join(dir, "tools")+ for _, d := range []string{binDir, filepath.Join(dir, "config"), filepath.Join(dir, "install"), filepath.Join(dir, "other-config")} {+ if err := os.Mkdir(d, 0700); err != nil {+ t.Fatal(err)+ }+ if err := os.WriteFile(filepath.Join(d, "preserve"), []byte("original"), 0600); err != nil {+ t.Fatal(err)+ }+ }+ service := "pulse"+ if custom {+ service = "pulse-other"+ }+ if mode == "update-named-server" {+ service = "pulse-other-update"+ }+ units := resetFixtureUnits(custom)+ if mode == "update-named-server" {+ for i := range units {+ units[i] = strings.Replace(units[i], "pulse-other", service, 1)+ }+ }+ for _, unit := range units {+ load, a, e := "loaded", active, enabled+ if mode == "mixed" {+ if unit == "pulse.service" {+ a = "inactive"+ }+ if unit == "pulse-backend.service" {+ a = "failed"+ }+ }+ if mode == "server-absent" || (mode == "optional-absent" && unit != "pulse.service") {+ load, a, e = "not-found", "inactive", ""+ }+ for suffix, value := range map[string]string{"load": load, "active": a, "enabled": e} {+ if err := os.WriteFile(filepath.Join(dir, unit+"."+suffix), []byte(value+"\n"), 0600); err != nil {+ t.Fatal(err)+ }+ }+ }+ mock := `#!/bin/bash+set -eu+printf '%s\n' "$*" >> "$FIXTURE_DIR/calls"+action=$1; shift+runtime=false+if [[ "${1:-}" == --runtime ]]; then runtime=true; shift; fi+if [[ "${1:-}" == --quiet ]]; then shift; fi+unit=${1:-}; shift || true+[[ "$unit" == *.service || "$unit" == *.timer ]] || unit="$unit.service"+if [[ "$action" == show ]]; then+ prop=$1+ if [[ "$unit" == "$TARGET" ]]; then+ case "$MODE:$prop" in+ query-fails:*) exit 1 ;;+ query-timeout:*) sleep 7; exit 1 ;;+ empty-load:--property=LoadState|empty-active:--property=ActiveState|empty-enabled:--property=UnitFileState) exit 0 ;;+ unknown-load:--property=LoadState|unknown-enabled:--property=UnitFileState) echo unexpected; exit 0 ;;+ unsettled:--property=ActiveState) echo activating; exit 0 ;;+ enabled-query-fails:--property=UnitFileState) exit 1 ;;+ stop-readback-fails:--property=ActiveState) [[ ! -e "$FIXTURE_DIR/$unit.stopped" ]] || exit 1 ;;+ start-readback-fails:--property=ActiveState) [[ ! -e "$FIXTURE_DIR/$unit.started" ]] || exit 1 ;;+ esac+ if [[ -e "$FIXTURE_DIR/pulse-backend.service.stopped" && "$prop" == --property=ActiveState && "$MODE" == reactivated ]]; then echo active; exit 0; fi+ if [[ -e "$FIXTURE_DIR/pulse-backend.service.stopped" && "$prop" == --property=UnitFileState && "$MODE" == reenabled ]]; then echo enabled; exit 0; fi+ if [[ -e "$FIXTURE_DIR/pulse-update.timer.started" && "$prop" == --property=ActiveState && "$MODE" == final-updater-active ]]; then echo active; exit 0; fi+ fi+ case "$prop" in+ --property=LoadState) cat "$FIXTURE_DIR/$unit.load" ;;+ --property=ActiveState) cat "$FIXTURE_DIR/$unit.active" ;;+ --property=UnitFileState) cat "$FIXTURE_DIR/$unit.enabled" ;;+ *) exit 2 ;;+ esac+elif [[ "$action" == is-active ]]; then+ [[ "$(cat "$FIXTURE_DIR/$unit.active")" == active ]]+elif [[ "$action" == stop ]]; then+ [[ "$unit:$MODE" != "$TARGET:stop-timeout" ]] || { sleep 7; exit 1; }+ [[ "$unit:$MODE" != "$TARGET:stop-fails" ]] || exit 1+ touch "$FIXTURE_DIR/$unit.stopped"+ case "$unit:$MODE" in+ "$TARGET:stop-ignored") ;;+ "$TARGET:still-failed") echo failed > "$FIXTURE_DIR/$unit.active" ;;+ *) echo inactive > "$FIXTURE_DIR/$unit.active" ;;+ esac+elif [[ "$action" == disable ]]; then+ [[ "$unit:$MODE" != "$TARGET:disable-fails" ]] || exit 1+ [[ "$unit:$MODE" == "$TARGET:disable-ignored" ]] || echo disabled > "$FIXTURE_DIR/$unit.enabled"+elif [[ "$action" == enable ]]; then+ [[ "$unit:$MODE" != "$TARGET:enable-fails" ]] || exit 1+ if [[ "$unit:$MODE" != "$TARGET:enable-ignored" ]]; then+ if [[ "$runtime" == true ]]; then echo enabled-runtime; else echo enabled; fi > "$FIXTURE_DIR/$unit.enabled"+ fi+elif [[ "$action" == start ]]; then+ [[ "$unit:$MODE" != "$TARGET:start-fails" ]] || exit 1+ touch "$FIXTURE_DIR/$unit.started"+ [[ "$unit:$MODE" == "$TARGET:start-ignored" ]] || echo active > "$FIXTURE_DIR/$unit.active"+else exit 2+fi+`+ if err := os.WriteFile(filepath.Join(binDir, "systemctl"), []byte(mock), 0700); err != nil {+ t.Fatal(err)+ }+ installer := os.Getenv("PULSE_TEST_ROOT_INSTALLER")+ if installer == "" {+ installer = filepath.Join("..", "..", "install.sh")+ }+ installer, err := filepath.Abs(installer)+ if err != nil {+ t.Fatal(err)+ }+ script := `+source "$INSTALLER_UNDER_TEST"+CONFIG_DIR="$FIXTURE_DIR/config"+SERVICE_NAME="$SERVICE"+SERVICE_NAME_EXPLICIT="$CUSTOM"+UPDATE_SERVICE_UNIT="$SERVICE-update.service"+UPDATE_TIMER_UNIT="$SERVICE-update.timer"+check_root() { :; }+print_header() { :; }+detect_service_name() { echo "$SERVICE"; }+rm() {+ echo "DELETE $*" >> "$FIXTURE_DIR/calls"+ [[ "$MODE" != delete-fails ]] || return 1+ local arg+ for arg in "$@"; do+ case "$arg" in -rf|--) ;; "$FIXTURE_DIR/config/"*) /bin/rm -rf -- "$arg" ;; *) return 91 ;; esac+ done+}+reset_pulse+`+ ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)+ defer cancel()+ cmd := exec.CommandContext(ctx, "bash", "-c", script)+ cmd.Env = append(os.Environ(), "PATH="+binDir+":"+os.Getenv("PATH"), "INSTALLER_UNDER_TEST="+installer, "FIXTURE_DIR="+dir, "MODE="+mode, "TARGET="+target, "SERVICE="+service, "CUSTOM="+boolString(custom))+ out, runErr := cmd.CombinedOutput()+ if ctx.Err() != nil {+ t.Fatalf("fixture deadline: %v\n%s", ctx.Err(), out)+ }+ calls, _ := os.ReadFile(filepath.Join(dir, "calls"))+ config, configErr := os.ReadFile(filepath.Join(dir, "config", "preserve"))+ r := serverResetResult{output: string(out), calls: string(calls), err: runErr, configPresent: configErr == nil && string(config) == "original", states: map[string]string{}}+ for _, d := range []string{"install", "other-config"} {+ b, e := os.ReadFile(filepath.Join(dir, d, "preserve"))+ if e != nil || string(b) != "original" {+ t.Fatalf("reset changed %s: %v\n%s", d, e, out)+ }+ }+ for _, unit := range units {+ for _, suffix := range []string{"active", "enabled"} {+ b, e := os.ReadFile(filepath.Join(dir, unit+"."+suffix))+ if e != nil {+ t.Fatal(e)+ }+ r.states[unit+"."+suffix] = strings.TrimSpace(string(b))+ }+ }+ return r+}