Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Report incomplete configuration recovery without false success
pulse-triage[bot]a6336ca57fc7
diff --git a/docs/release-control/v6/internal/subsystems/api-contracts.md b/docs/release-control/v6/internal/subsystems/api-contracts.mdindex 21cd9989fe..1d21f793d1 100644--- a/docs/release-control/v6/internal/subsystems/api-contracts.md+++ b/docs/release-control/v6/internal/subsystems/api-contracts.md@@ -21,6 +21,34 @@## Purpose+### Configuration imports distinguish saved files from live recovery++After an archive commits, every attempted live alert, webhook, email and guest+metadata reload must succeed before the import acknowledges success. A failed+read or unavailable component of a present monitor returns HTTP 500 with a+bounded, fixed component list and a failed audit outcome. Independent reloads+still run; raw errors, private paths, destinations and archive inputs never+enter that response or audit detail. The complete plain-text error stays below+the API client's 200-character boundary, including its newline, so the existing+recovery dialog retains the explanation instead of reporting success and+reloading the page.++This is a **saved, incompletely applied** outcome, not rollback or permission to+repeat a replacement. Committed replay admission remains retained, including+after this failure. Operators check the destination's live settings before+relying on them. Initialization contexts without a monitor remain supported;+a present monitor with a missing alert/notification manager fails gracefully,+not by panic or a successful live-apply claim.++`TestHandleImportConfigRuntimeOutcome` exercises each read failure, combined+failure, missing managers, successful independent application, bounded private+errors, audit outcomes and newer-edit/replay preservation using real imports.+`TestContract_ConfigImportIncompleteRuntimeKeepsCommittedOutcome` binds the+real router, authentication, metadata reader, no-store and replay response.+The registered frontend transport test and browser proof consume these+post-commit responses through the unchanged recovery flow. This establishes+neither notification delivery nor installed/native recovery.+### Configuration-import transport replays are bounded, not fresh recoveryThe existing import POST keeps authentication, write scope, CSRF and tenant@@ -10244,8 +10272,10 @@ archive and rebinds shared runtime state, the reload path must tolerate absentnotification or monitoring managers and degrade gracefully instead ofpanicking on optional side effects. `/api/config/import` may be exercised fromproof or setup contexts that do not yet have every long-lived runtime manager-wired, but the contract must still leave the imported configuration readable-through the canonical API surface.+wired. A missing monitor remains supported; a missing component of a present+monitor or a failed attempted reload returns an explicit saved-but-incomplete+error. The imported configuration remains readable through the canonical API+surface, and a committed replacement is not automatically retried.`/api/config/export` and `/api/config/import` are therefore router auth-bypassentries only so their handlers can own stricter route-local auth andpublic-network checks; global middleware must not preempt those handlers ordiff --git a/docs/release-control/v6/internal/subsystems/storage-recovery.md b/docs/release-control/v6/internal/subsystems/storage-recovery.mdindex 40db273b47..af5e210ca6 100644--- a/docs/release-control/v6/internal/subsystems/storage-recovery.md+++ b/docs/release-control/v6/internal/subsystems/storage-recovery.md@@ -21,6 +21,22 @@## Purpose+### Saved configuration is not successful live recovery++Configuration import returns an explicit saved-but-incomplete error and failed+audit outcome when an attempted live alert, notification or guest metadata+reload fails. Independent reloads continue, but the recovery dialog must not+close and claim success or schedule its normal success reload. The response+names only fixed components, never private errors, destinations or archive+inputs, and fits the consuming API client's plain-text bound. Persistence is+already committed: this is not rollback, nor a reason to repeat the import.+The existing bounded replay admission protects newer edits after partial apply.+Initialization without a monitor remains supported; absent sub-managers in a+present monitor fail without panic. The real-import runtime-outcome and router+contract tests, frontend transport suite and source-bound consuming browser+proof cover the distinction. No workload, database schema, guest-agent request+or native recovery operation changes.+### Future backup dates cannot establish current protectionProtection freshness and verification must fall at or before their evaluationdiff --git a/frontend-modern/src/api/__tests__/configImportRuntimeOutcome.test.ts b/frontend-modern/src/api/__tests__/configImportRuntimeOutcome.test.tsnew file mode 100644index 0000000000..b436ab8b99--- /dev/null+++ b/frontend-modern/src/api/__tests__/configImportRuntimeOutcome.test.ts@@ -0,0 +1,45 @@+import { afterEach, describe, expect, it, vi } from 'vitest';+import { apiFetchJSON, clearApiToken, setOrgID } from '@/utils/apiClient';++afterEach(() => {+ vi.unstubAllGlobals();+ clearApiToken();+ setOrgID(null);+});++describe('configuration import post-commit failure transport', () => {+ it.each([+ 'alert settings',+ 'webhook settings',+ 'email settings',+ 'guest metadata',+ 'webhook settings, email settings',+ 'alert settings, webhook settings, email settings, guest metadata',+ ])('preserves the bounded %s error without repeating the import', async (components) => {+ const message = `Configuration saved, but live reload failed: ${components}. Check the destination before relying on it. Do not repeat the import.`;+ // Plain-text API errors are intentionally bounded. Even the combined+ // producer response, including http.Error's newline, must fit that limit.+ expect(`${message}\n`.length).toBeLessThan(200);+ const fetch = vi.fn().mockResolvedValue(+ new Response(`${message}\n`, {+ status: 500,+ headers: { 'Content-Type': 'text/plain; charset=utf-8' },+ }),+ );+ vi.stubGlobal('fetch', fetch);+ document.cookie = 'pulse_csrf=synthetic-import-csrf; Path=/';+ setOrgID('default');++ await expect(+ apiFetchJSON('/api/config/import', {+ method: 'POST',+ body: JSON.stringify({ data: 'synthetic-archive', passphrase: 'synthetic-passphrase' }),+ expectedOrgID: 'default',+ retry: false,+ }),+ ).rejects.toMatchObject({ message, status: 500 });+ expect(fetch).toHaveBeenCalledTimes(1);+ expect(fetch.mock.calls[0][0]).toBe('/api/config/import');+ expect(fetch.mock.calls[0][1].method).toBe('POST');+ });+});diff --git a/internal/api/configapi/config_export_import_compat_test.go b/internal/api/configapi/config_export_import_compat_test.goindex e5ed4eff7b..6d079d730a 100644--- a/internal/api/configapi/config_export_import_compat_test.go+++ b/internal/api/configapi/config_export_import_compat_test.go@@ -17,6 +17,7 @@ import ("time""github.com/rcourtman/pulse-go-rewrite/internal/config"+ "github.com/rcourtman/pulse-go-rewrite/internal/notifications""golang.org/x/crypto/pbkdf2")@@ -34,6 +35,7 @@ func TestHandleImportConfigAcceptsLegacyVersion40Bundle(t *testing.T) {ConfigPath: targetDir,}handler := newTestConfigHandlers(t, cfg)+ setUnexportedField(t, handler.defaultMonitor, "notificationMgr", ¬ifications.NotificationManager{})handler.reloadFunc = func() error { return nil }// Build source export payload with distinct node data.diff --git a/internal/api/configapi/config_export_import_handlers.go b/internal/api/configapi/config_export_import_handlers.goindex 8379b48d38..8e57c777a0 100644--- a/internal/api/configapi/config_export_import_handlers.go+++ b/internal/api/configapi/config_export_import_handlers.go@@ -3,6 +3,7 @@ package configapiimport ("encoding/json""net/http"+ "strings""time""github.com/rcourtman/pulse-go-rewrite/internal/api/apicontext"@@ -186,24 +187,37 @@ func (h *ConfigHandlers) handleImportConfig(w http.ResponseWriter, r *http.Reque// Also reload alert and notification configs explicitly// (the monitor reload only reloads nodes unless it's a full reload).- if h.getMonitor(r.Context()) != nil {+ // Persistence has already committed. Continue independent reloads, but do+ // not turn a partial live apply into a successful import acknowledgement.+ // Only fixed component names may reach the response or audit detail.+ var reloadFailures []string+ if monitor := h.getMonitor(r.Context()); monitor != nil {// Reload alert configuration.if alertConfig, err := h.getPersistence(r.Context()).LoadAlertConfig(); err == nil {- h.getMonitor(r.Context()).GetAlertManager().UpdateConfig(*alertConfig)- log.Info().Msg("Reloaded alert configuration after import")+ if alertManager := monitor.GetAlertManager(); alertManager == nil {+ reloadFailures = append(reloadFailures, "alert settings")+ log.Warn().Msg("Skipped alert reload after import because alert manager is unavailable")+ } else {+ alertManager.UpdateConfig(*alertConfig)+ log.Info().Msg("Reloaded alert configuration after import")+ }} else {+ reloadFailures = append(reloadFailures, "alert settings")log.Warn().Err(err).Msg("Failed to reload alert configuration after import")}+ notificationMgr := monitor.GetNotificationManager()// Reload webhook configuration.if webhooks, err := h.getPersistence(r.Context()).LoadWebhooks(); err == nil {- notificationMgr := h.getMonitor(r.Context()).GetNotificationManager()if notificationMgr == nil {+ reloadFailures = append(reloadFailures, "webhook settings")log.Warn().Msg("Skipped webhook reload after import because notification manager is unavailable")} else {// Clear existing webhooks and add new ones.+ webhooksApplied := truefor _, webhook := range notificationMgr.GetWebhooks() {if err := notificationMgr.DeleteWebhook(webhook.ID); err != nil {+ webhooksApplied = falselog.Warn().Err(err).Str("webhook", webhook.ID).Msg("Failed to delete existing webhook during reload")}}@@ -211,21 +225,28 @@ func (h *ConfigHandlers) handleImportConfig(w http.ResponseWriter, r *http.Requefor _, webhook := range webhooks {notificationMgr.AddWebhook(webhook)}- log.Info().Int("count", len(webhooks)).Msg("Reloaded webhook configuration after import")+ if webhooksApplied {+ log.Info().Int("count", len(webhooks)).Msg("Reloaded webhook configuration after import")+ } else {+ reloadFailures = append(reloadFailures, "webhook settings")+ }}} else {+ reloadFailures = append(reloadFailures, "webhook settings")log.Warn().Err(err).Msg("Failed to reload webhook configuration after import")}// Reload email configuration.if emailConfig, err := h.getPersistence(r.Context()).LoadEmailConfig(); err == nil {- if notificationMgr := h.getMonitor(r.Context()).GetNotificationManager(); notificationMgr == nil {+ if notificationMgr == nil {+ reloadFailures = append(reloadFailures, "email settings")log.Warn().Msg("Skipped email reload after import because notification manager is unavailable")} else {notificationMgr.SetEmailConfig(*emailConfig)log.Info().Msg("Reloaded email configuration after import")}} else {+ reloadFailures = append(reloadFailures, "email settings")log.Warn().Err(err).Msg("Failed to reload email configuration after import")}}@@ -233,12 +254,21 @@ func (h *ConfigHandlers) handleImportConfig(w http.ResponseWriter, r *http.Reque// Reload guest metadata from disk.if h.guestMetadataHandler != nil {if err := h.guestMetadataHandler.Reload(r.Context()); err != nil {+ reloadFailures = append(reloadFailures, "guest metadata")log.Warn().Err(err).Msg("Failed to reload guest metadata after import")} else {log.Info().Msg("Reloaded guest metadata after import")}}+ if len(reloadFailures) != 0 {+ failed := strings.Join(reloadFailures, ", ")+ deps.AuditEvent(orgID, "config_imported", importUser, deps.ClientIP(r), r.URL.Path, false,+ "Configuration saved but live reload incomplete: "+failed)+ http.Error(w, "Configuration saved, but live reload failed: "+failed+". Check the destination before relying on it. Do not repeat the import.", http.StatusInternalServerError)+ return+ }+deps.AuditEvent(apicontext.OrgID(r.Context()), "config_imported", importUser, deps.ClientIP(r), r.URL.Path, true,"Configuration imported")diff --git a/internal/api/configapi/config_handlers_admin_test.go b/internal/api/configapi/config_handlers_admin_test.goindex 3bb559a239..d860e1389b 100644--- a/internal/api/configapi/config_handlers_admin_test.go+++ b/internal/api/configapi/config_handlers_admin_test.go@@ -10,6 +10,7 @@ import ("testing""github.com/rcourtman/pulse-go-rewrite/internal/config"+ "github.com/rcourtman/pulse-go-rewrite/internal/notifications")func TestHandleExportConfig(t *testing.T) {@@ -106,6 +107,9 @@ func TestHandleImportConfig(t *testing.T) {}handler := newTestConfigHandlers(t, cfg)+ // A successful live apply requires its notification settings reader, but+ // these configuration-only tests must never start delivery workers.+ setUnexportedField(t, handler.defaultMonitor, "notificationMgr", ¬ifications.NotificationManager{})// Mock reload function to avoid actual reload logic failurehandler.reloadFunc = func() error { return nil }// Force usage of the persistence instance we prepared (dummyPersistence)diff --git a/internal/api/configapi/config_import_runtime_outcome_test.go b/internal/api/configapi/config_import_runtime_outcome_test.gonew file mode 100644index 0000000000..bb726c0355--- /dev/null+++ b/internal/api/configapi/config_import_runtime_outcome_test.go@@ -0,0 +1,172 @@+package configapi++import (+ "context"+ "encoding/json"+ "errors"+ "net/http"+ "os"+ "path/filepath"+ "reflect"+ "strings"+ "testing"++ "github.com/rcourtman/pulse-go-rewrite/internal/alerts"+ "github.com/rcourtman/pulse-go-rewrite/internal/config"+ "github.com/rcourtman/pulse-go-rewrite/internal/monitoring"+ "github.com/rcourtman/pulse-go-rewrite/internal/notifications"+)++type importMetadataReloadFunc func(context.Context) error++func (f importMetadataReloadFunc) Reload(ctx context.Context) error { return f(ctx) }++// These are real persistence imports followed by bounded, local read faults.+// No monitor, notification queue or external destination is started.+func TestHandleImportConfigRuntimeOutcome(t *testing.T) {+ for _, tc := range []struct {+ name string+ brokenFiles []string+ metadataFailure bool+ missingAlerts bool+ missingNotifications bool+ withoutMonitor bool+ wantFailures []string+ }{+ {name: "complete"},+ {name: "alerts", brokenFiles: []string{"alerts.json"}, wantFailures: []string{"alert settings"}},+ {name: "webhooks", brokenFiles: []string{"webhooks.enc"}, wantFailures: []string{"webhook settings"}},+ {name: "email", brokenFiles: []string{"email.enc"}, wantFailures: []string{"email settings"}},+ {name: "metadata", metadataFailure: true, wantFailures: []string{"guest metadata"}},+ {name: "missing-alert-manager", missingAlerts: true, wantFailures: []string{"alert settings"}},+ {name: "missing-notification-manager", missingNotifications: true, wantFailures: []string{"webhook settings", "email settings"}},+ {name: "combined", brokenFiles: []string{"alerts.json", "webhooks.enc", "email.enc"}, metadataFailure: true, wantFailures: []string{"alert settings", "webhook settings", "email settings", "guest metadata"}},+ {name: "setup-without-monitor", withoutMonitor: true},+ } {+ t.Run(tc.name, func(t *testing.T) {+ h, calls := replayTestHandler(t)+ dir := h.defaultPersistence.GetConfigDir()+ manager := alerts.NewManager()+ t.Cleanup(manager.Stop)+ // Configuration setters need no running delivery worker.+ notifier := ¬ifications.NotificationManager{}+ notifier.SetEmailConfig(notifications.EmailConfig{SMTPHost: "before.invalid"})+ notifier.AddWebhook(notifications.WebhookConfig{ID: "before", URL: "https://before.invalid", Enabled: false})+ monitor := &monitoring.Monitor{}+ if !tc.missingAlerts {+ setUnexportedField(t, monitor, "alertManager", manager)+ }+ if !tc.missingNotifications {+ setUnexportedField(t, monitor, "notificationMgr", notifier)+ }+ if !tc.withoutMonitor {+ h.defaultMonitor = monitor+ }++ const privateFailure = "synthetic-private-path-and-secret-must-not-reach-client"+ metadataCalls := 0+ h.guestMetadataHandler = importMetadataReloadFunc(func(context.Context) error {+ metadataCalls+++ if tc.metadataFailure {+ return errors.New(privateFailure)+ }+ return nil+ })+ h.reloadFunc = func() error {+ calls.Add(1)+ for _, name := range tc.brokenFiles {+ if err := os.WriteFile(filepath.Join(dir, name), []byte(privateFailure), 0600); err != nil {+ return err+ }+ }+ return nil+ }+ var audits []bool+ var auditDetails []string+ h.SetRuntimeDependencies(RuntimeDependencies{AuditEvent: func(_, event, _, _, _ string, success bool, detail string) {+ if event != "config_imported" {+ t.Errorf("unexpected audit event %q", event)+ }+ audits = append(audits, success)+ auditDetails = append(auditDetails, detail)+ }})++ importedAlerts := alerts.DefaultAlertConfig()+ importedAlerts.GuestDefaults.CPU = &alerts.HysteresisThreshold{Trigger: 73, Clear: 68}+ archive := config.ExportData{+ Version: "4.4", System: *config.DefaultSystemSettings(), Alerts: importedAlerts,+ Nodes: config.NodesConfig{PVEInstances: []config.PVEInstance{{Name: "runtime-fixture", Host: "https://runtime.invalid:8006", Disabled: true, VerifySSL: true}}},+ Email: notifications.EmailConfig{Enabled: false, SMTPHost: "after.invalid", Password: "synthetic-email-secret"},+ Webhooks: []notifications.WebhookConfig{{ID: "after", URL: "https://after.invalid", Enabled: false}},+ GuestMetadata: map[string]*config.GuestMetadata{"synthetic:101": {Description: "imported metadata"}},+ }+ archive.System.PublicURL = "https://imported.invalid"+ plain, err := json.Marshal(archive)+ if err != nil {+ t.Fatal(err)+ }+ request := ImportConfigRequest{Data: encryptedImportValidationFixture(t, plain, replayTestPassphrase), Passphrase: replayTestPassphrase}+ body := replayTestBody(t, request)+ rec := replayTestSubmit(h, body)+ failed := len(tc.wantFailures) != 0+ wantStatus := http.StatusOK+ if failed {+ wantStatus = http.StatusInternalServerError+ want := "Configuration saved, but live reload failed: " + strings.Join(tc.wantFailures, ", ") + ". Check the destination before relying on it. Do not repeat the import.\n"+ if rec.Body.String() != want {+ t.Errorf("partial apply response = %q, want %q", rec.Body.String(), want)+ }+ }+ if rec.Code != wantStatus || calls.Load() != 1 || metadataCalls != 1 || !reflect.DeepEqual(audits, []bool{!failed}) {+ t.Errorf("status=%d body=%s reloads=%d metadata=%d audits=%v", rec.Code, rec.Body.String(), calls.Load(), metadataCalls, audits)+ }+ for _, secret := range []string{privateFailure, dir, request.Data, request.Passphrase, archive.Email.Password} {+ if strings.Contains(rec.Body.String(), secret) || strings.Contains(strings.Join(auditDetails, " "), secret) {+ t.Error("client response or audit leaked private failure/input detail")+ }+ }+ if !tc.withoutMonitor && !tc.missingAlerts && !containsImportFailure(tc.wantFailures, "alert settings") && manager.GetConfig().GuestDefaults.CPU.Trigger != 73 {+ t.Error("an independent successful alert reload was abandoned")+ }+ if !tc.withoutMonitor && !tc.missingNotifications {+ if !containsImportFailure(tc.wantFailures, "email settings") && notifier.GetEmailConfig().SMTPHost != "after.invalid" {+ t.Error("an independent successful email reload was abandoned")+ }+ if !containsImportFailure(tc.wantFailures, "webhook settings") && (len(notifier.GetWebhooks()) != 1 || notifier.GetWebhooks()[0].ID != "after") {+ t.Error("an independent successful webhook reload was abandoned")+ }+ }+ settings, err := h.defaultPersistence.LoadSystemSettings()+ if err != nil || settings.PublicURL != "https://imported.invalid" {+ t.Fatalf("committed settings lost after live apply: %#v, %v", settings, err)+ }+ readback := map[string]any{"case": tc.name, "request": request, "status": rec.Code, "contentType": rec.Header().Get("Content-Type"), "body": rec.Body.String()}++ // The failure is post-commit, not permission to overwrite newer work.+ settings.PublicURL = "https://newer-edit.invalid"+ if err := h.defaultPersistence.SaveSystemSettings(*settings); err != nil {+ t.Fatal(err)+ }+ before := metadataImportFiles(t, dir)+ replay := replayTestSubmit(h, body)+ if replay.Code != http.StatusConflict || calls.Load() != 1 || metadataCalls != 1 || len(audits) != 1 || !reflect.DeepEqual(before, metadataImportFiles(t, dir)) {+ t.Error("post-commit replay changed newer settings, metadata, backups or live state")+ }+ readback["replay"] = map[string]any{"status": replay.Code, "contentType": replay.Header().Get("Content-Type"), "body": replay.Body.String()}+ encoded, err := json.Marshal(readback)+ if err != nil {+ t.Fatal(err)+ }+ t.Logf("SYNTHETIC_IMPORT_RUNTIME_READBACK %s", encoded)+ })+ }+}++func containsImportFailure(failures []string, component string) bool {+ for _, failure := range failures {+ if failure == component {+ return true+ }+ }+ return false+}diff --git a/internal/api/contract_test.go b/internal/api/contract_test.goindex 731c7d36ea..a6a7aa332c 100644--- a/internal/api/contract_test.go+++ b/internal/api/contract_test.go@@ -28046,3 +28046,95 @@ func TestContract_ConfigImportReplayRemainsAuthenticatedAndScoped(t *testing.T)}}}++// A post-commit metadata read failure must reach the browser as an incomplete+// apply, not a 200 success that closes the recovery dialog and reloads the page.+func TestContract_ConfigImportIncompleteRuntimeKeepsCommittedOutcome(t *testing.T) {+ t.Setenv("PULSE_DEV", "true")+ t.Setenv("PULSE_HOSTED_MODE", "false")+ dir := t.TempDir()+ t.Setenv("PULSE_DATA_DIR", dir)+ const rawToken = "synthetic-import-runtime-token.12345678"+ record := newTokenRecord(t, rawToken, []string{config.ScopeSettingsRead, config.ScopeSettingsWrite}, nil)+ cfg := newTestConfigWithTokens(t, record)+ cfg.DataPath, cfg.ConfigPath = dir, dir+ if err := os.WriteFile(filepath.Join(dir, "config.json"), []byte("{}"), 0600); err != nil {+ t.Fatal(err)+ }+ reloads := 0+ router := NewRouter(cfg, nil, nil, nil, func() error {+ reloads+++ // Fault after persistence has committed, before the real metadata reload.+ path := filepath.Join(dir, "guest_metadata.json")+ if err := os.Remove(path); err != nil {+ return err+ }+ return os.Mkdir(path, 0700)+ }, "test")+ cleanupTestRouter(t, router)++ source := config.NewConfigPersistence(t.TempDir())+ if err := source.SaveNodesConfig([]config.PVEInstance{{Name: "runtime-fixture", Host: "https://runtime.invalid:8006", Disabled: true, VerifySSL: true}}, nil, nil); err != nil {+ t.Fatal(err)+ }+ if err := source.SaveAPITokens([]config.APITokenRecord{record}); err != nil {+ t.Fatal(err)+ }+ settings := config.DefaultSystemSettings()+ settings.PublicURL = "https://imported.invalid"+ if err := source.SaveSystemSettings(*settings); err != nil {+ t.Fatal(err)+ }+ if err := source.GetGuestMetadataStore().Set("synthetic:101", &config.GuestMetadata{Description: "imported metadata"}); err != nil {+ t.Fatal(err)+ }+ t.Cleanup(func() {+ if err := source.GetGuestMetadataStore().Close(time.Second); err != nil {+ t.Error(err)+ }+ })+ const passphrase = "synthetic-runtime-contract-passphrase"+ archive, err := source.ExportConfig(passphrase)+ if err != nil {+ t.Fatal(err)+ }+ body, err := json.Marshal(ImportConfigRequest{Data: archive, Passphrase: passphrase})+ if err != nil {+ t.Fatal(err)+ }+ submit := func() *httptest.ResponseRecorder {+ req := httptest.NewRequest(http.MethodPost, "/api/config/import", bytes.NewReader(body))+ req.RemoteAddr = "127.0.0.1:1234"+ req.Header.Set("X-API-Token", rawToken)+ rec := httptest.NewRecorder()+ router.Handler().ServeHTTP(rec, req)+ return rec+ }+ rec := submit()+ if rec.Code != http.StatusInternalServerError || !strings.Contains(rec.Body.String(), "Configuration saved, but live reload failed: guest metadata.") || strings.Contains(rec.Body.String(), "success") {+ t.Fatalf("incomplete apply = %d %s", rec.Code, rec.Body.String())+ }+ for _, private := range []string{dir, archive, passphrase, rawToken, "guest_metadata.json"} {+ if strings.Contains(rec.Body.String(), private) {+ t.Error("runtime failure exposed private input or storage detail")+ }+ }+ if rec.Header().Get("Cache-Control") != "no-store" {+ t.Error("recovery error lost the router's no-store boundary")+ }+ committed, err := router.persistence.LoadSystemSettings()+ if err != nil || committed.PublicURL != settings.PublicURL {+ t.Fatalf("partial apply must not pretend to roll back the commit: %#v %v", committed, err)+ }+ committed.PublicURL = "https://newer.invalid"+ if err := router.persistence.SaveSystemSettings(*committed); err != nil {+ t.Fatal(err)+ }+ if replay := submit(); replay.Code != http.StatusConflict || reloads != 1 {+ t.Fatalf("replay after partial apply = %d %s; reloads=%d", replay.Code, replay.Body.String(), reloads)+ }+ newer, err := router.persistence.LoadSystemSettings()+ if err != nil || newer.PublicURL != committed.PublicURL {+ t.Fatalf("replay replaced a newer legitimate edit: %#v %v", newer, err)+ }+}