Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Keep missing PVE backup diagnostics non-destructive

pulse-triage[bot]138901e422c1
diff --git a/docs/UNIFIED_AGENT.md b/docs/UNIFIED_AGENT.md
index 5bd325603e..0b891d5ead 100644
--- a/docs/UNIFIED_AGENT.md
+++ b/docs/UNIFIED_AGENT.md
@@ -1165,20 +1165,47 @@ If your Docker Swarm cluster isn't being detected:
### PVE Backups Not Showing (Recovery)
-If local PVE backups aren't appearing in Pulse after setting up via `--enable-proxmox`:
-
-1. **Check permissions**: The API token needs `PVEDatastoreAdmin` on `/storage`:
- ```bash
- pveum aclmod /storage -user pulse-monitor@pve -role PVEDatastoreAdmin
- pveum aclmod /storage -token 'pulse-monitor@pve!<token-name>' -role PVEDatastoreAdmin
- ```
- Replace `pulse-monitor@pve!<token-name>` with the full token ID shown in Pulse.
- Privilege-separated PVE tokens need the storage ACL on the token as well as the service user.
-
-2. **Re-run setup**: Delete the node in Pulse Settings and re-run the agent with `--enable-proxmox`. Recent versions grant this permission automatically.
-
-3. **Check state file**: If re-running doesn't trigger setup, remove the state file:
- ```bash
- rm /var/lib/pulse-agent/proxmox-pve-registered
- ```
- Then restart the agent.
+Pulse's **server collects PVE backup inventory through the saved Proxmox API
+connection**, including storage and storage-content requests. A working host
+agent or successful `--enable-proxmox` setup does not prove that this separate
+API collection is current. Guest-agent disk readings are not backup inventory.
+
+1. **Check existing evidence in Proxmox first.** In the native PVE interface,
+ identify the existing archive's node, storage, guest type/ID and time. Do not
+ run another backup or restore to create evidence. A missing Pulse row is not
+ proof that the archive is absent, and an OK task does not prove guest thaw.
+2. **Compare the same source in Pulse.** Open **Proxmox → Backups → By date**
+ and check the selected connection and filters against that archive. Coverage
+ posture is a different reading: expand its explanation rather than treating
+ an unprotected badge as a missing backup. Direct PBS collection and backups
+ seen through PVE are distinct sources; see [PBS data sources](PBS.md#data-source-indicator)
+ and [Recovery](RECOVERY.md#missing-or-inconsistent-evidence).
+3. **Inspect ordinary collection, not a new probe.** In **Settings →
+ Infrastructure**, check the affected PVE connection's existing status and
+ last successful collection, together with the original redacted error and
+ time. The server checks online nodes and queryable storage configured for
+ backup content. A failed, unavailable or partial read is not an empty
+ inventory; a successful connection test is not proof of sustained collection.
+4. **Repair access only when the evidence supports it.** Ask the Proxmox
+ administrator to compare the actual rejected endpoint and installed PVE
+ version with the configured service user and token's effective storage
+ permissions. With privilege separation, both user and token ACLs must allow
+ the request, including the relevant scope and inheritance. Audit-only
+ inventory access does not establish access to every backup-content request.
+ Do not apply blanket storage-administrator ACLs merely because a table is
+ empty, disable privilege separation or use an administrator token as a test.
+ See [Proxmox permission checks](TROUBLESHOOTING.md#check-permissions-proxmox).
+
+**Do not delete the monitored node, remove registration state or rerun setup as
+a diagnostic.** Setup can rotate an existing API token and change permissions;
+it is not a harmless refresh. Preserve the connection, credentials, registration
+state and backup/history data. Make any evidenced configuration repair through
+normal maintenance outside backups, then observe ordinary polling without
+manual guest-agent probes, restarts or repeated **Test Connection** requests.
+
+For freeze-enabled backups, follow the [backup safety precaution](VM_DISK_MONITORING.md#backup-safety).
+It is not an incident recovery procedure or proof of a repaired thaw failure.
+If reporting a continuing mismatch, retain the running server version, source,
+original time, relevant missing privilege or redacted error and the differing
+native/Pulse observations. Keep token secrets, full ACL listings and private
+infrastructure details out of public reports; follow [Getting Help](TROUBLESHOOTING.md#-getting-help).
diff --git a/frontend-modern/public/docs/UNIFIED_AGENT.md b/frontend-modern/public/docs/UNIFIED_AGENT.md
index 5bd325603e..0b891d5ead 100644
--- a/frontend-modern/public/docs/UNIFIED_AGENT.md
+++ b/frontend-modern/public/docs/UNIFIED_AGENT.md
@@ -1165,20 +1165,47 @@ If your Docker Swarm cluster isn't being detected:
### PVE Backups Not Showing (Recovery)
-If local PVE backups aren't appearing in Pulse after setting up via `--enable-proxmox`:
-
-1. **Check permissions**: The API token needs `PVEDatastoreAdmin` on `/storage`:
- ```bash
- pveum aclmod /storage -user pulse-monitor@pve -role PVEDatastoreAdmin
- pveum aclmod /storage -token 'pulse-monitor@pve!<token-name>' -role PVEDatastoreAdmin
- ```
- Replace `pulse-monitor@pve!<token-name>` with the full token ID shown in Pulse.
- Privilege-separated PVE tokens need the storage ACL on the token as well as the service user.
-
-2. **Re-run setup**: Delete the node in Pulse Settings and re-run the agent with `--enable-proxmox`. Recent versions grant this permission automatically.
-
-3. **Check state file**: If re-running doesn't trigger setup, remove the state file:
- ```bash
- rm /var/lib/pulse-agent/proxmox-pve-registered
- ```
- Then restart the agent.
+Pulse's **server collects PVE backup inventory through the saved Proxmox API
+connection**, including storage and storage-content requests. A working host
+agent or successful `--enable-proxmox` setup does not prove that this separate
+API collection is current. Guest-agent disk readings are not backup inventory.
+
+1. **Check existing evidence in Proxmox first.** In the native PVE interface,
+ identify the existing archive's node, storage, guest type/ID and time. Do not
+ run another backup or restore to create evidence. A missing Pulse row is not
+ proof that the archive is absent, and an OK task does not prove guest thaw.
+2. **Compare the same source in Pulse.** Open **Proxmox → Backups → By date**
+ and check the selected connection and filters against that archive. Coverage
+ posture is a different reading: expand its explanation rather than treating
+ an unprotected badge as a missing backup. Direct PBS collection and backups
+ seen through PVE are distinct sources; see [PBS data sources](PBS.md#data-source-indicator)
+ and [Recovery](RECOVERY.md#missing-or-inconsistent-evidence).
+3. **Inspect ordinary collection, not a new probe.** In **Settings →
+ Infrastructure**, check the affected PVE connection's existing status and
+ last successful collection, together with the original redacted error and
+ time. The server checks online nodes and queryable storage configured for
+ backup content. A failed, unavailable or partial read is not an empty
+ inventory; a successful connection test is not proof of sustained collection.
+4. **Repair access only when the evidence supports it.** Ask the Proxmox
+ administrator to compare the actual rejected endpoint and installed PVE
+ version with the configured service user and token's effective storage
+ permissions. With privilege separation, both user and token ACLs must allow
+ the request, including the relevant scope and inheritance. Audit-only
+ inventory access does not establish access to every backup-content request.
+ Do not apply blanket storage-administrator ACLs merely because a table is
+ empty, disable privilege separation or use an administrator token as a test.
+ See [Proxmox permission checks](TROUBLESHOOTING.md#check-permissions-proxmox).
+
+**Do not delete the monitored node, remove registration state or rerun setup as
+a diagnostic.** Setup can rotate an existing API token and change permissions;
+it is not a harmless refresh. Preserve the connection, credentials, registration
+state and backup/history data. Make any evidenced configuration repair through
+normal maintenance outside backups, then observe ordinary polling without
+manual guest-agent probes, restarts or repeated **Test Connection** requests.
+
+For freeze-enabled backups, follow the [backup safety precaution](VM_DISK_MONITORING.md#backup-safety).
+It is not an incident recovery procedure or proof of a repaired thaw failure.
+If reporting a continuing mismatch, retain the running server version, source,
+original time, relevant missing privilege or redacted error and the differing
+native/Pulse observations. Keep token secrets, full ACL listings and private
+infrastructure details out of public reports; follow [Getting Help](TROUBLESHOOTING.md#-getting-help).
diff --git a/scripts/tests/test_pve_backup_troubleshooting_docs.py b/scripts/tests/test_pve_backup_troubleshooting_docs.py
new file mode 100644
index 0000000000..44215d7f0a
--- /dev/null
+++ b/scripts/tests/test_pve_backup_troubleshooting_docs.py
@@ -0,0 +1,82 @@
+#!/usr/bin/env python3
+"""Keep missing-backup guidance on the real collector and non-destructive path.
+
+No provider calls, setup commands, backups or guest-agent probes are executed.
+"""
+
+from pathlib import Path
+import re
+import unittest
+
+
+ROOT = Path(__file__).resolve().parents[2]
+GUIDE = "UNIFIED_AGENT.md"
+HEADING = "### PVE Backups Not Showing (Recovery)"
+
+
+def backup_section():
+ text = (ROOT / "docs" / GUIDE).read_text(encoding="utf-8")
+ return text.split(HEADING, 1)[1].split("\n### ", 1)[0]
+
+
+class PVEBackupTroubleshootingDocsTest(unittest.TestCase):
+ def test_shipped_help_matches_source(self):
+ self.assertEqual((ROOT / "docs" / GUIDE).read_bytes(),
+ (ROOT / "frontend-modern/public/docs" / GUIDE).read_bytes())
+
+ def test_explains_actual_server_collection_not_guest_agent_readings(self):
+ section = backup_section()
+ for phrase in ("server collects PVE backup inventory", "saved Proxmox API",
+ "storage-content requests", "Guest-agent disk readings are not backup inventory",
+ "online nodes", "queryable storage", "backup content"):
+ self.assertIn(phrase, section)
+ collector = (ROOT / "internal/monitoring/monitor_backups.go").read_text()
+ for operation in ("client.GetStorage(ctx, node.Node)",
+ "client.GetStorageContent(ctx, node.Node, storage.Storage)",
+ 'nodeEffectiveStatus[node.Node] != "online"',
+ 'strings.Contains(storage.Content, "backup")',
+ "storageContentQueryable(storage)"):
+ self.assertIn(operation, collector)
+
+ def test_no_destructive_reenrolment_or_blanket_acl_recipe(self):
+ section = backup_section()
+ shell = "\n".join(re.findall(r"```(?:bash|sh)\n(.*?)```", section, re.DOTALL))
+ self.assertNotRegex(shell, r"\brm\b|aclmod|--enable-proxmox|systemctl\s+restart")
+ normalised = " ".join(section.split())
+ self.assertIn("Do not delete the monitored node, remove registration state or rerun setup", normalised)
+ self.assertIn("Setup can rotate an existing API token", normalised)
+ for item in ("connection", "credentials", "registration state", "backup/history data"):
+ self.assertIn(item, normalised)
+ setup = (ROOT / "internal/hostagent/proxmox_setup.go").read_text()
+ self.assertIn("rotating token in place", setup)
+ self.assertIn('"token", "remove", proxmoxUserPVE, tokenName', setup)
+
+ def test_permissions_are_endpoint_specific_and_keep_privilege_separation(self):
+ section = " ".join(backup_section().split())
+ for phrase in ("actual rejected endpoint", "installed PVE version",
+ "both user and token ACLs", "scope and inheritance",
+ "Audit-only inventory access does not establish access",
+ "Do not apply blanket storage-administrator ACLs",
+ "disable privilege separation", "administrator token as a test"):
+ self.assertIn(phrase, section)
+ self.assertIn("TROUBLESHOOTING.md#check-permissions-proxmox", section)
+
+ def test_preserves_source_freshness_and_backup_safety_distinctions(self):
+ section = " ".join(backup_section().split())
+ for phrase in ("node, storage, guest type/ID and time", "Proxmox → Backups → By date",
+ "Coverage posture is a different reading", "distinct sources",
+ "failed, unavailable or partial read is not an empty inventory",
+ "OK task does not prove guest thaw", "Do not run another backup or restore",
+ "outside backups", "without manual guest-agent probes",
+ "Keep token secrets, full ACL listings and private infrastructure details"):
+ self.assertIn(phrase, section)
+ for target in ("PBS.md#data-source-indicator", "RECOVERY.md#missing-or-inconsistent-evidence",
+ "VM_DISK_MONITORING.md#backup-safety", "TROUBLESHOOTING.md#-getting-help"):
+ self.assertIn(target, section)
+ safety = (ROOT / "docs/VM_DISK_MONITORING.md").read_text()
+ self.assertIn("every filesystem covered by the backup", safety)
+ self.assertIn("Pulse monitoring and alerts are unavailable", safety)
+
+
+if __name__ == "__main__":
+ unittest.main()