Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Keep Operational Trust upgrade checks read-only

pulse-triage[bot]e4a96c9302a4
diff --git a/docs/OPERATIONAL_TRUST.md b/docs/OPERATIONAL_TRUST.md
index 6fc9e08d58..a80a775d20 100644
--- a/docs/OPERATIONAL_TRUST.md
+++ b/docs/OPERATIONAL_TRUST.md
@@ -181,25 +181,67 @@ shape is now a local display adapter, not a writable source of truth.
Before upgrading:
-1. back up the Pulse data directory;
-2. confirm that the v6 process can write its alert, notification, recovery, and
- action database directories;
-3. confirm supported clients can read additive JSON fields;
-4. expose the metrics listener to a protected scraper if rollout telemetry is
- required;
-5. verify Pulse Pro entitlement connectivity before relying on action offers.
-
-After upgrading:
-
-1. confirm the Patrol navigation count matches the active queue;
-2. inspect an active item through its deepest evidence and protection detail;
-3. acknowledge and unacknowledge a test item;
-4. verify a bounded suppression returns to active attention;
-5. confirm stale collection remains visible rather than resolving;
-6. exercise notification retry/dead-letter monitoring;
-7. if using Pulse Pro actions, complete a review/approve/run/verify journey.
+1. Record the running version, edition and deployment configuration. Keep a
+ consistent private backup of the actual data directory, matching encryption
+ and audit signing keys, and any external stores; use the
+ [full-state recovery guidance](MIGRATION.md#full-state-recovery), not a copy
+ of a live database file or a configuration-only export.
+2. Check the existing service account and persistent mounts through the normal
+ deployment configuration. Do not broaden permissions or replace keys to
+ make an upgrade check pass.
+3. Keep independent monitoring and alert coverage available during the update.
+ For affected freeze-enabled Proxmox backups, preserve the separate
+ [guest-safety precaution](VM_DISK_MONITORING.md#backup-safety).
+4. Keep the previous binary or image and its matching data backup until
+ recovery is checked. Reverting a binary does not reverse data migrations;
+ follow the deployment's [update and rollback guidance](AUTO_UPDATE.md#rollback).
+
+### Read-only checks after upgrading
+
+Use an existing authenticated browser session and ordinary collection. These
+checks inspect existing evidence; they do not require a new incident or a
+change to a workload, notification queue or alert lifecycle.
+
+1. Check the running server version and edition, then the normal connection
+ health and observation times. Compare the same connection, resource and time
+ range before and after the upgrade; reused names or VMIDs are not identity.
+2. Compare the Patrol navigation count with the active queue using the same
+ organisation, access scope and filters. Open an existing item, if present,
+ and inspect its resource, lifecycle, evidence and protection detail. An
+ empty queue does not establish complete or healthy collection.
+3. Confirm that retained evidence still shows its observation time and any
+ stale, denied, partial or unknown state. Do not disconnect a collector or
+ induce a fault to obtain a test case. A successful task or action is not
+ fresh detector evidence of recovery, guest thaw or a tested restore.
+4. Inspect **Recent delivery activity** in Alert History and existing failed
+ delivery details, if any. Compare the original alert, queued attempt and
+ recipient's actual message separately; Test success does not establish
+ ordinary, grouped or resolved delivery. Use the
+ [notification checks](TROUBLESHOOTING.md#test-succeeds-but-real-alerts-are-missing)
+ for the actual symptom.
+5. If using Pro actions, inspect existing execution and verification records
+ separately. A successful execution does not establish verified recovery;
+ an unavailable action offer is not by itself a monitoring failure.
+
+**Do not acknowledge, suppress, retry, dismiss or clear records, send a Test,
+or approve or run an action merely to validate an upgrade.** Retry can send a
+real notification, suppression changes active attention, and an offered Docker
+restart changes a workload. These controls remain available for intentional
+operator work, not passive diagnostics. If a check needs an event that has not
+occurred, record it as unverified rather than manufacture one. Exercise those
+paths only in an isolated non-production fixture with synthetic destinations
+and disposable workloads, never against live resources.
+
+Keep any existing metrics listener private; reading through an already
+protected scraper is optional, not a reason to expose a new listener or widen
+network access. Keep authentication, TLS verification and least-privilege
+collection unchanged. Preserve errors and existing delivery/action records;
+share only the affected view, version, time and locally redacted explanation,
+not credentials, full inventories, notification payloads or Debug exports. See
+[safe issue reporting](TROUBLESHOOTING.md#-getting-help).
If a migration fails, stop the upgraded process, preserve the data directory
-and logs, and restore the prior release with the pre-upgrade data backup. Do
-not delete lifecycle, evidence, notification, recovery, or action records to
-force startup.
+and private logs, and use the deployment's recovery procedure with the prior
+release and its matching consistent pre-upgrade data backup. A failed read or
+an empty queue alone is not a reason to restore data. Do not delete lifecycle,
+evidence, notification, recovery or action records to force startup.
diff --git a/frontend-modern/public/docs/OPERATIONAL_TRUST.md b/frontend-modern/public/docs/OPERATIONAL_TRUST.md
index 6fc9e08d58..a80a775d20 100644
--- a/frontend-modern/public/docs/OPERATIONAL_TRUST.md
+++ b/frontend-modern/public/docs/OPERATIONAL_TRUST.md
@@ -181,25 +181,67 @@ shape is now a local display adapter, not a writable source of truth.
Before upgrading:
-1. back up the Pulse data directory;
-2. confirm that the v6 process can write its alert, notification, recovery, and
- action database directories;
-3. confirm supported clients can read additive JSON fields;
-4. expose the metrics listener to a protected scraper if rollout telemetry is
- required;
-5. verify Pulse Pro entitlement connectivity before relying on action offers.
-
-After upgrading:
-
-1. confirm the Patrol navigation count matches the active queue;
-2. inspect an active item through its deepest evidence and protection detail;
-3. acknowledge and unacknowledge a test item;
-4. verify a bounded suppression returns to active attention;
-5. confirm stale collection remains visible rather than resolving;
-6. exercise notification retry/dead-letter monitoring;
-7. if using Pulse Pro actions, complete a review/approve/run/verify journey.
+1. Record the running version, edition and deployment configuration. Keep a
+ consistent private backup of the actual data directory, matching encryption
+ and audit signing keys, and any external stores; use the
+ [full-state recovery guidance](MIGRATION.md#full-state-recovery), not a copy
+ of a live database file or a configuration-only export.
+2. Check the existing service account and persistent mounts through the normal
+ deployment configuration. Do not broaden permissions or replace keys to
+ make an upgrade check pass.
+3. Keep independent monitoring and alert coverage available during the update.
+ For affected freeze-enabled Proxmox backups, preserve the separate
+ [guest-safety precaution](VM_DISK_MONITORING.md#backup-safety).
+4. Keep the previous binary or image and its matching data backup until
+ recovery is checked. Reverting a binary does not reverse data migrations;
+ follow the deployment's [update and rollback guidance](AUTO_UPDATE.md#rollback).
+
+### Read-only checks after upgrading
+
+Use an existing authenticated browser session and ordinary collection. These
+checks inspect existing evidence; they do not require a new incident or a
+change to a workload, notification queue or alert lifecycle.
+
+1. Check the running server version and edition, then the normal connection
+ health and observation times. Compare the same connection, resource and time
+ range before and after the upgrade; reused names or VMIDs are not identity.
+2. Compare the Patrol navigation count with the active queue using the same
+ organisation, access scope and filters. Open an existing item, if present,
+ and inspect its resource, lifecycle, evidence and protection detail. An
+ empty queue does not establish complete or healthy collection.
+3. Confirm that retained evidence still shows its observation time and any
+ stale, denied, partial or unknown state. Do not disconnect a collector or
+ induce a fault to obtain a test case. A successful task or action is not
+ fresh detector evidence of recovery, guest thaw or a tested restore.
+4. Inspect **Recent delivery activity** in Alert History and existing failed
+ delivery details, if any. Compare the original alert, queued attempt and
+ recipient's actual message separately; Test success does not establish
+ ordinary, grouped or resolved delivery. Use the
+ [notification checks](TROUBLESHOOTING.md#test-succeeds-but-real-alerts-are-missing)
+ for the actual symptom.
+5. If using Pro actions, inspect existing execution and verification records
+ separately. A successful execution does not establish verified recovery;
+ an unavailable action offer is not by itself a monitoring failure.
+
+**Do not acknowledge, suppress, retry, dismiss or clear records, send a Test,
+or approve or run an action merely to validate an upgrade.** Retry can send a
+real notification, suppression changes active attention, and an offered Docker
+restart changes a workload. These controls remain available for intentional
+operator work, not passive diagnostics. If a check needs an event that has not
+occurred, record it as unverified rather than manufacture one. Exercise those
+paths only in an isolated non-production fixture with synthetic destinations
+and disposable workloads, never against live resources.
+
+Keep any existing metrics listener private; reading through an already
+protected scraper is optional, not a reason to expose a new listener or widen
+network access. Keep authentication, TLS verification and least-privilege
+collection unchanged. Preserve errors and existing delivery/action records;
+share only the affected view, version, time and locally redacted explanation,
+not credentials, full inventories, notification payloads or Debug exports. See
+[safe issue reporting](TROUBLESHOOTING.md#-getting-help).
If a migration fails, stop the upgraded process, preserve the data directory
-and logs, and restore the prior release with the pre-upgrade data backup. Do
-not delete lifecycle, evidence, notification, recovery, or action records to
-force startup.
+and private logs, and use the deployment's recovery procedure with the prior
+release and its matching consistent pre-upgrade data backup. A failed read or
+an empty queue alone is not a reason to restore data. Do not delete lifecycle,
+evidence, notification, recovery or action records to force startup.
diff --git a/frontend-modern/src/utils/__tests__/operationalUpgradeHelp.test.ts b/frontend-modern/src/utils/__tests__/operationalUpgradeHelp.test.ts
new file mode 100644
index 0000000000..288c4663f0
--- /dev/null
+++ b/frontend-modern/src/utils/__tests__/operationalUpgradeHelp.test.ts
@@ -0,0 +1,58 @@
+import { readFileSync } from 'node:fs';
+import { resolve } from 'node:path';
+import { describe, expect, it } from 'vitest';
+import { renderDocMarkdown } from '@/features/docs/docMarkdown';
+
+const read = (path: string) => readFileSync(resolve(process.cwd(), '..', path), 'utf8');
+
+const article = (name: string) => {
+ const element = document.createElement('article');
+ element.innerHTML = renderDocMarkdown(read(`docs/${name}.md`), name);
+ return element;
+};
+
+describe('Operational Trust upgrade help', () => {
+ it('ships the same recovery and passive-check guidance in the in-app document', () => {
+ expect(read('frontend-modern/public/docs/OPERATIONAL_TRUST.md')).toBe(
+ read('docs/OPERATIONAL_TRUST.md'),
+ );
+ const guide = article('OPERATIONAL_TRUST');
+ expect(guide.querySelector('#read-only-checks-after-upgrading')?.textContent).toBe(
+ 'Read-only checks after upgrading',
+ );
+ const warning = [...guide.querySelectorAll('strong')].find((element) =>
+ element.textContent?.includes('merely to validate an upgrade'),
+ );
+ expect(warning?.textContent?.replace(/\s+/g, ' ')).toBe(
+ 'Do not acknowledge, suppress, retry, dismiss or clear records, send a Test, or approve or run an action merely to validate an upgrade.',
+ );
+ expect(guide.textContent?.replace(/\s+/g, ' ')).toContain(
+ 'Retry can send a real notification, suppression changes active attention, and an offered Docker restart changes a workload.',
+ );
+ });
+
+ it('keeps every recovery, safety and reporting link on an existing in-app guide and anchor', () => {
+ const guide = article('OPERATIONAL_TRUST');
+ for (const [name, fragment, label] of [
+ ['MIGRATION', 'full-state-recovery', 'full-state recovery guidance'],
+ ['VM_DISK_MONITORING', 'backup-safety', 'guest-safety precaution'],
+ ['AUTO_UPDATE', 'rollback', 'update and rollback guidance'],
+ ['TROUBLESHOOTING', 'test-succeeds-but-real-alerts-are-missing', 'notification checks'],
+ ['TROUBLESHOOTING', '-getting-help', 'safe issue reporting'],
+ ]) {
+ const link = guide.querySelector(`a[href="/docs/${name}#${fragment}"]`);
+ expect(link?.textContent).toBe(label);
+ expect(link?.hasAttribute('data-doc-link')).toBe(true);
+ expect(article(name).querySelector(`[id="${fragment}"]`)).not.toBeNull();
+ }
+ });
+
+ it('does not turn an unobserved event or a failed read into an upgrade-success claim', () => {
+ const text = article('OPERATIONAL_TRUST').textContent!.replace(/\s+/g, ' ');
+ expect(text).toContain('An empty queue does not establish complete or healthy collection');
+ expect(text).toContain('record it as unverified rather than manufacture one');
+ expect(text).toContain('A failed read or an empty queue alone is not a reason to restore data');
+ expect(text).not.toContain('exercise notification retry/dead-letter monitoring');
+ expect(text).not.toContain('complete a review/approve/run/verify journey');
+ });
+});
diff --git a/scripts/tests/test_production_rollout_docs.py b/scripts/tests/test_production_rollout_docs.py
index 367f807e7a..dcb77c546c 100644
--- a/scripts/tests/test_production_rollout_docs.py
+++ b/scripts/tests/test_production_rollout_docs.py
@@ -11,6 +11,7 @@ import unittest
ROOT = Path(__file__).resolve().parents[2]
DOC = ROOT / "docs/PRODUCTION_SECURITY.md"
+TRUST_DOC = ROOT / "docs/OPERATIONAL_TRUST.md"
def section(heading):
@@ -112,6 +113,55 @@ class ProductionRolloutDocsTest(unittest.TestCase):
self.assertIn(boundary, text)
self.assertNotRegex(text, r"(?:--insecure|curl[^\n]*\||--token\s)")
+ def test_operational_upgrade_checks_are_observational_not_action_probes(self):
+ text = " ".join(TRUST_DOC.read_text().split("### Read-only checks after upgrading\n", 1)[1].split())
+ for boundary in ("existing authenticated browser session", "ordinary collection",
+ "same organisation, access scope and filters",
+ "An empty queue does not establish complete or healthy collection",
+ "Do not disconnect a collector or induce a fault",
+ "Recent delivery activity", "recipient's actual message separately",
+ "Test success does not establish ordinary, grouped or resolved delivery",
+ "execution and verification records separately",
+ "not by itself a monitoring failure", "Retry can send a real notification",
+ "suppression changes active attention", "restart changes a workload",
+ "Do not acknowledge, suppress, retry, dismiss or clear records, send a Test",
+ "or approve or run an action merely to validate an upgrade",
+ "unverified rather than manufacture one",
+ "isolated non-production fixture", "synthetic destinations",
+ "disposable workloads", "never against live resources"):
+ with self.subTest(boundary=boundary):
+ self.assertIn(boundary, text)
+ for old_instruction in ("acknowledge and unacknowledge a test item",
+ "verify a bounded suppression returns to active attention",
+ "exercise notification retry/dead-letter monitoring",
+ "complete a review/approve/run/verify journey"):
+ self.assertNotIn(old_instruction, text)
+ # The original advice crossed real mutation boundaries, not just UI
+ # demonstrations. Keep that reason grounded in the current owners.
+ queue = (ROOT / "internal/api/alerting/notification_queue.go").read_text()
+ self.assertIn("queue.ScheduleRetry(request.ID, 0)", queue)
+ actions = (ROOT / "internal/api/actions.go").read_text()
+ self.assertIn("h.ActionLifecycle().Execute(", actions)
+
+ def test_operational_upgrade_preserves_recovery_and_access_boundaries(self):
+ text = " ".join(TRUST_DOC.read_text().split("## Upgrade and compatibility\n", 1)[1].split())
+ for boundary in ("version, edition and deployment configuration", "consistent private backup",
+ "matching encryption and audit signing keys", "external stores",
+ "not a copy of a live database file or a configuration-only export",
+ "Do not broaden permissions or replace keys", "independent monitoring and alert coverage",
+ "MIGRATION.md#full-state-recovery", "VM_DISK_MONITORING.md#backup-safety",
+ "Reverting a binary does not reverse data migrations", "AUTO_UPDATE.md#rollback",
+ "metrics listener private", "not a reason to expose a new listener",
+ "authentication, TLS verification and least-privilege collection unchanged",
+ "Preserve errors and existing delivery/action records", "locally redacted explanation",
+ "not credentials, full inventories, notification payloads or Debug exports",
+ "A failed read or an empty queue alone is not a reason to restore data",
+ "Do not delete lifecycle, evidence, notification, recovery or action records"):
+ with self.subTest(boundary=boundary):
+ self.assertIn(boundary, text)
+ self.assertEqual(TRUST_DOC.read_bytes(),
+ (ROOT / "frontend-modern/public/docs/OPERATIONAL_TRUST.md").read_bytes())
+
if __name__ == "__main__":
unittest.main()