Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
docs: keep production rollout safe and edition-aware
pulse-triage[bot]795cd3fc1181
diff --git a/docs/PRODUCTION_SECURITY.md b/docs/PRODUCTION_SECURITY.mdindex 6d2fc393fc..c6d47cef24 100644--- a/docs/PRODUCTION_SECURITY.md+++ b/docs/PRODUCTION_SECURITY.md@@ -23,6 +23,15 @@ Platform connections**. Add a host agent only after identifying a metric thatthe Proxmox API cannot provide. The complete capability matrix is in[Agent Security](AGENT_SECURITY.md#proxmox-deployment-choices).+**API-only does not mean no guest-agent commands.** Pulse can request VM+filesystem and memory readings through Proxmox's QEMU Guest Agent API without a+Pulse agent installed. Those reads share the channel used by freeze-enabled+backups. For an affected installation, follow the+[backup safety precaution](VM_DISK_MONITORING.md#backup-safety), including the+actual Pulse server and automatic updaters; stopping only a Pulse agent is not+enough. An OK backup task does not prove guest thaw or application recovery.+Do not run a backup or guest-agent probe just to test the monitoring setup.+Generated Proxmox API setup uses a separate `pulse-monitor` account and tokenwith monitoring ACLs. Review the generated script before running it, especiallyif your organization maintains its own Proxmox roles. Do not reuse an@@ -116,12 +125,22 @@ covered in [Privacy](PRIVACY.md).For a controlled production installation:-1. Choose an exact release tag rather than `latest`.-2. Download `install.sh` and `install.sh.sshsig` from that release.-3. Verify the installer with the published Ed25519 installer key and the- `pulse-install` namespace.-4. Run the verified installer with the same pinned version.-5. Install or upgrade agents with the per-host command shown by your own Pulse+1. Record the current server version, edition and deployment configuration.+ Choose an exact target release rather than `latest`, and retain a consistent+ private backup of the actual data, matching keys and any external stores.+ See [update and rollback boundaries](DEPLOYMENT_MODELS.md#updates-by-model):+ replacing a binary or image does not reverse data migrations.+2. Obtain the correct server edition. **Public GitHub release archives and the+ public Docker image are Community builds**, even with an activation key.+ Paid Pro, continuing Relay and legacy customers must use their private Pro+ image or archive from the [existing download route](INSTALL.md). Do not+ replace an installed paid runtime with a public build during an update.+3. For a Community server installation, download `install.sh` and+ `install.sh.sshsig` from the exact release. Verify the installer with the+ published Ed25519 installer key and the `pulse-install` namespace before+ running it with that same pinned version. Follow the edition-appropriate+ [installation instructions](INSTALL.md), not a substitute installer.+4. Install or upgrade agents with the per-host command shown by your own Pulseserver under **Settings → Infrastructure → Install on a host**.The agent update path requires a server-provided SHA-256 checksum. Release@@ -135,13 +154,23 @@ can verify the installer before execution. See [Installation](INSTALL.md) and[Agent Security](AGENT_SECURITY.md#supply-chain-boundary) for the commands andthe documented v5-to-v6 first-update limitation.-## Community, Relay, and Pro boundaries+## Community and Pro boundariesCurrent self-hosted plans do not sell monitoring capacity by node, VM,container, or other child-resource volume. Community includes core self-hosted-monitoring and seven days of metric history. Relay and Pro add capabilities-such as remote access, longer history, Pulse Mobile, Patrol investigation and-governed fixes, RBAC, audit logging, reporting, and centralized agent profiles.+monitoring and seven days of metric history. Pro adds 90-day history, Patrol+investigation and governed fixes, RBAC, audit logging, reporting and centralized+agent profiles. SSO is included with Community and higher tiers.++**Relay is no longer sold. Pulse Mobile and Relay retire on 31 March 2027.**+Existing paired phones keep working until then; continuing Relay subscribers+receive Pro features at their current price for as long as their subscription+continues, including after the app retires. Relay connects the app, **not the+web UI**. Use your own VPN or tunnel for remote web access. For alerts on your+phone afterwards, add an ntfy, Gotify or Pushover destination under **Alerts**+and open Pulse in your phone's browser. See [Relay / Pulse Mobile](RELAY.md)+for existing pairing and security details; do not plan a new deployment around+the retiring app.That means an estate with 50 Proxmox hosts does not need Pro merely to add thefiftieth host. Review the current [Plans and Entitlements](PULSE_PRO.md) for the@@ -167,6 +196,8 @@ and measure their own workload.- [ ] Put Pulse on a dedicated VM, container, Kubernetes deployment, or hostwith durable storage and tested backups.- [ ] Pin and verify the server release before installation.+- [ ] Preserve the server edition, matching keys and deployment configuration+ when updating; a public Community build does not supply the private Pro runtime.- [ ] Enable HTTPS and restrict web/API access at the firewall or reverseproxy.- [ ] Start with a dedicated Proxmox API token and no host agents.@@ -176,10 +207,23 @@ and measure their own workload.one cluster before adding the rest.- [ ] Add root agents only to hosts that need specific local telemetry; keepcommands disabled unless there is a reviewed operational need.-- [ ] Test loss-of-node, failed-backup, high-capacity, notification, credential- rotation, upgrade, backup, and restore paths.+- [ ] Check existing inventory and provider identity before adding another+ cluster; matching node names or VMIDs do not establish the same resource.+ See [multiple installations](CONFIGURATION.md#multiple-proxmox-installations).+- [ ] Exercise disruptive failure, upgrade and restore scenarios only on an+ isolated non-production fixture with independent monitoring and a recovery+ path. Do not induce node loss, a failed backup, a full disk or an alert storm+ on live workloads as a rollout check. Normal Test notification success does+ not establish ordinary, grouped or resolved alert delivery.+- [ ] For affected freeze-enabled Proxmox backups, follow the backup safety+ precaution above. Keep monitoring paused until independent checks confirm+ thaw, fresh successful writes to every covered filesystem and workload+ liveness; restore only services and timers active before the pause. Arrange+ independent outage coverage while Pulse monitoring and alerts are unavailable.- [ ] Observe CPU, memory, database size, write rate, and dashboard latency- during the staged rollout, then set retention and polling to fit the estate.+ during normal staged operation. Keep measurement windows and resource limits+ comparable; database size is not a write rate. Do not prune history or change+ polling to hide a fault; use [performance guidance](TROUBLESHOOTING.md#excessive-cpu-writes-or-database-growth).- [ ] Record the Pulse version, enabled integrations, privileges, exposedports, backup location, and rollback procedure in the site's runbook.diff --git a/frontend-modern/public/docs/PRODUCTION_SECURITY.md b/frontend-modern/public/docs/PRODUCTION_SECURITY.mdindex 6d2fc393fc..c6d47cef24 100644--- a/frontend-modern/public/docs/PRODUCTION_SECURITY.md+++ b/frontend-modern/public/docs/PRODUCTION_SECURITY.md@@ -23,6 +23,15 @@ Platform connections**. Add a host agent only after identifying a metric thatthe Proxmox API cannot provide. The complete capability matrix is in[Agent Security](AGENT_SECURITY.md#proxmox-deployment-choices).+**API-only does not mean no guest-agent commands.** Pulse can request VM+filesystem and memory readings through Proxmox's QEMU Guest Agent API without a+Pulse agent installed. Those reads share the channel used by freeze-enabled+backups. For an affected installation, follow the+[backup safety precaution](VM_DISK_MONITORING.md#backup-safety), including the+actual Pulse server and automatic updaters; stopping only a Pulse agent is not+enough. An OK backup task does not prove guest thaw or application recovery.+Do not run a backup or guest-agent probe just to test the monitoring setup.+Generated Proxmox API setup uses a separate `pulse-monitor` account and tokenwith monitoring ACLs. Review the generated script before running it, especiallyif your organization maintains its own Proxmox roles. Do not reuse an@@ -116,12 +125,22 @@ covered in [Privacy](PRIVACY.md).For a controlled production installation:-1. Choose an exact release tag rather than `latest`.-2. Download `install.sh` and `install.sh.sshsig` from that release.-3. Verify the installer with the published Ed25519 installer key and the- `pulse-install` namespace.-4. Run the verified installer with the same pinned version.-5. Install or upgrade agents with the per-host command shown by your own Pulse+1. Record the current server version, edition and deployment configuration.+ Choose an exact target release rather than `latest`, and retain a consistent+ private backup of the actual data, matching keys and any external stores.+ See [update and rollback boundaries](DEPLOYMENT_MODELS.md#updates-by-model):+ replacing a binary or image does not reverse data migrations.+2. Obtain the correct server edition. **Public GitHub release archives and the+ public Docker image are Community builds**, even with an activation key.+ Paid Pro, continuing Relay and legacy customers must use their private Pro+ image or archive from the [existing download route](INSTALL.md). Do not+ replace an installed paid runtime with a public build during an update.+3. For a Community server installation, download `install.sh` and+ `install.sh.sshsig` from the exact release. Verify the installer with the+ published Ed25519 installer key and the `pulse-install` namespace before+ running it with that same pinned version. Follow the edition-appropriate+ [installation instructions](INSTALL.md), not a substitute installer.+4. Install or upgrade agents with the per-host command shown by your own Pulseserver under **Settings → Infrastructure → Install on a host**.The agent update path requires a server-provided SHA-256 checksum. Release@@ -135,13 +154,23 @@ can verify the installer before execution. See [Installation](INSTALL.md) and[Agent Security](AGENT_SECURITY.md#supply-chain-boundary) for the commands andthe documented v5-to-v6 first-update limitation.-## Community, Relay, and Pro boundaries+## Community and Pro boundariesCurrent self-hosted plans do not sell monitoring capacity by node, VM,container, or other child-resource volume. Community includes core self-hosted-monitoring and seven days of metric history. Relay and Pro add capabilities-such as remote access, longer history, Pulse Mobile, Patrol investigation and-governed fixes, RBAC, audit logging, reporting, and centralized agent profiles.+monitoring and seven days of metric history. Pro adds 90-day history, Patrol+investigation and governed fixes, RBAC, audit logging, reporting and centralized+agent profiles. SSO is included with Community and higher tiers.++**Relay is no longer sold. Pulse Mobile and Relay retire on 31 March 2027.**+Existing paired phones keep working until then; continuing Relay subscribers+receive Pro features at their current price for as long as their subscription+continues, including after the app retires. Relay connects the app, **not the+web UI**. Use your own VPN or tunnel for remote web access. For alerts on your+phone afterwards, add an ntfy, Gotify or Pushover destination under **Alerts**+and open Pulse in your phone's browser. See [Relay / Pulse Mobile](RELAY.md)+for existing pairing and security details; do not plan a new deployment around+the retiring app.That means an estate with 50 Proxmox hosts does not need Pro merely to add thefiftieth host. Review the current [Plans and Entitlements](PULSE_PRO.md) for the@@ -167,6 +196,8 @@ and measure their own workload.- [ ] Put Pulse on a dedicated VM, container, Kubernetes deployment, or hostwith durable storage and tested backups.- [ ] Pin and verify the server release before installation.+- [ ] Preserve the server edition, matching keys and deployment configuration+ when updating; a public Community build does not supply the private Pro runtime.- [ ] Enable HTTPS and restrict web/API access at the firewall or reverseproxy.- [ ] Start with a dedicated Proxmox API token and no host agents.@@ -176,10 +207,23 @@ and measure their own workload.one cluster before adding the rest.- [ ] Add root agents only to hosts that need specific local telemetry; keepcommands disabled unless there is a reviewed operational need.-- [ ] Test loss-of-node, failed-backup, high-capacity, notification, credential- rotation, upgrade, backup, and restore paths.+- [ ] Check existing inventory and provider identity before adding another+ cluster; matching node names or VMIDs do not establish the same resource.+ See [multiple installations](CONFIGURATION.md#multiple-proxmox-installations).+- [ ] Exercise disruptive failure, upgrade and restore scenarios only on an+ isolated non-production fixture with independent monitoring and a recovery+ path. Do not induce node loss, a failed backup, a full disk or an alert storm+ on live workloads as a rollout check. Normal Test notification success does+ not establish ordinary, grouped or resolved alert delivery.+- [ ] For affected freeze-enabled Proxmox backups, follow the backup safety+ precaution above. Keep monitoring paused until independent checks confirm+ thaw, fresh successful writes to every covered filesystem and workload+ liveness; restore only services and timers active before the pause. Arrange+ independent outage coverage while Pulse monitoring and alerts are unavailable.- [ ] Observe CPU, memory, database size, write rate, and dashboard latency- during the staged rollout, then set retention and polling to fit the estate.+ during normal staged operation. Keep measurement windows and resource limits+ comparable; database size is not a write rate. Do not prune history or change+ polling to hide a fault; use [performance guidance](TROUBLESHOOTING.md#excessive-cpu-writes-or-database-growth).- [ ] Record the Pulse version, enabled integrations, privileges, exposedports, backup location, and rollback procedure in the site's runbook.diff --git a/scripts/tests/test_production_rollout_docs.py b/scripts/tests/test_production_rollout_docs.pynew file mode 100644index 0000000000..367f807e7a--- /dev/null+++ b/scripts/tests/test_production_rollout_docs.py@@ -0,0 +1,117 @@+#!/usr/bin/env python3+"""Keep production decisions consistent with existing safety and edition help.++These are source/guidance checks, not native backup, installation, licence or+notification acceptance. No provider calls or disruptive operations are run.+"""++from pathlib import Path+import unittest+++ROOT = Path(__file__).resolve().parents[2]+DOC = ROOT / "docs/PRODUCTION_SECURITY.md"+++def section(heading):+ text = DOC.read_text(encoding="utf-8")+ return " ".join(text.split(heading + "\n", 1)[1].split("\n## ", 1)[0].split())+++class ProductionRolloutDocsTest(unittest.TestCase):+ def test_shipped_guide_matches_canonical(self):+ self.assertEqual(DOC.read_bytes(),+ (ROOT / "frontend-modern/public/docs/PRODUCTION_SECURITY.md").read_bytes())++ def test_api_only_keeps_the_guest_agent_backup_boundary(self):+ text = section("## Choose the least-privileged collection path")+ for boundary in ("API-only does not mean no guest-agent commands",+ "without a Pulse agent installed", "freeze-enabled backups",+ "VM_DISK_MONITORING.md#backup-safety", "automatic updaters",+ "stopping only a Pulse agent is not enough",+ "An OK backup task does not prove guest thaw",+ "Do not run a backup or guest-agent probe"):+ self.assertIn(boundary, text)+ # The actual server collector calls the Proxmox QGA endpoint. API-only+ # does not imply a passive or guest-agent-free transport.+ collector = (ROOT / "internal/monitoring/monitor_pve_guest_builders.go").read_text()+ client = (ROOT / "pkg/proxmox/client.go").read_text()+ self.assertIn("client.GetVMFSInfo(ctx, res.Node, res.VMID)", collector)+ self.assertIn('"/nodes/%s/qemu/%d/agent/get-fsinfo"', client)++ def test_installation_preserves_paid_edition_and_recoverable_data(self):+ text = section("## Installation and update integrity")+ for boundary in ("version, edition and deployment configuration",+ "consistent private backup", "matching keys", "external stores",+ "does not reverse data migrations", "Community builds",+ "even with an activation key", "continuing Relay",+ "private Pro image or archive", "Do not replace an installed paid runtime",+ "For a Community server installation", "pulse-install",+ "before running it", "same pinned version"):+ self.assertIn(boundary, text)+ install = " ".join((ROOT / "docs/INSTALL.md").read_text().split())+ self.assertIn("Community builds", install)+ self.assertIn("do not include the private Pulse Pro runtime hooks", install)+ self.assertIn("private Pulse Pro Docker image or Linux archive", install)+ deployment = " ".join((ROOT / "docs/DEPLOYMENT_MODELS.md").read_text().split())+ self.assertIn("does not undo data migrations", deployment)+ self.assertIn("Paid Pro installs must keep their private image", deployment)++ def test_retirement_is_not_a_current_remote_web_or_purchase_offer(self):+ text = section("## Community and Pro boundaries")+ for boundary in ("Relay is no longer sold", "31 March 2027",+ "Existing paired phones keep working until then",+ "current price", "for as long as their subscription continues",+ "including after the app retires", "**not the web UI**",+ "own VPN or tunnel", "ntfy, Gotify or Pushover",+ "phone's browser", "do not plan a new deployment"):+ self.assertIn(boundary, text)+ self.assertNotIn("Relay and Pro add", text)+ plans = " ".join((ROOT / "docs/PULSE_PRO.md").read_text().split())+ self.assertIn("Relay is no longer sold", plans)+ self.assertIn("Relay connects the app, not the web UI", plans)+ self.assertIn("Those Pro features do not end with the app's retirement", plans)++ def test_current_plan_copy_keeps_free_sso_and_history_distinct(self):+ text = section("## Community and Pro boundaries")+ self.assertIn("seven days of metric history", text)+ self.assertIn("Pro adds 90-day history", text)+ self.assertIn("SSO is included with Community and higher tiers", text)+ features = (ROOT / "pkg/licensing/features.go").read_text()+ free = features.split("var freeFeatures = []string{", 1)[1].split("}", 1)[0]+ self.assertRegex(free, r"\bFeatureSSO\b")+ history = features.split("var TierHistoryDays = map[Tier]int{", 1)[1].split("}", 1)[0]+ self.assertRegex(history, r"TierFree:\s+7,")+ self.assertRegex(history, r"TierPro:\s+90,")++ def test_rollout_does_not_inject_failures_or_hide_faults_on_live_workloads(self):+ text = section("## Production rollout checklist")+ for boundary in ("isolated non-production fixture", "independent monitoring",+ "recovery path", "Do not induce node loss", "failed backup",+ "full disk", "alert storm", "on live workloads",+ "Test notification success does not establish ordinary, grouped or resolved",+ "every covered filesystem", "workload liveness",+ "restore only services and timers active before the pause",+ "monitoring and alerts are unavailable", "independent outage coverage",+ "matching node names or VMIDs do not establish the same resource",+ "normal staged operation", "comparable",+ "database size is not a write rate", "Do not prune history or change polling"):+ self.assertIn(boundary, text)+ self.assertNotIn("Test loss-of-node", text)+ self.assertNotIn("```", text)+ self.assertIn("CONFIGURATION.md#multiple-proxmox-installations", text)+ self.assertIn("TROUBLESHOOTING.md#excessive-cpu-writes-or-database-growth", text)++ def test_scale_and_privilege_limits_remain_explicit(self):+ text = " ".join(DOC.read_text().split())+ for boundary in ("dedicated read-only or narrowly scoped API token",+ "root` by default", "monitoring-only", "agent:exec",+ "DISCOVERY_ENABLED=false", "127.0.0.1:9191", "AES-256-GCM",+ "not a blanket certification", "simulated 500-node",+ "stage the rollout", "independent security certification"):+ self.assertIn(boundary, text)+ self.assertNotRegex(text, r"(?:--insecure|curl[^\n]*\||--token\s)")+++if __name__ == "__main__":+ unittest.main()