Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Explain conflicting PBS backup health without unsafe diagnostics
pulse-triage[bot]42734802424d
diff --git a/docs/PBS.md b/docs/PBS.mdindex 8f31f3cda4..d73a6b2925 100644--- a/docs/PBS.md+++ b/docs/PBS.md@@ -362,6 +362,54 @@ Pulse Agent's stored History request. Restarting that guest is not a diagnosisof an empty PBS chart. See [VM Disk Monitoring](VM_DISK_MONITORING.md) if theseparate symptom is missing filesystem information in a PVE VM row.+### Backup health disagrees with visible PBS backups++The backup health strip rates monitored workloads, not individual artifacts.+A count of **with PBS snapshots**, a visible restore point or a **Verified**+label does not by itself establish the workload's protection posture:++| Reading | What it tells you | What it does not prove |+|---|---|---|+| With PBS snapshots / restore points | Pulse lists backup artifacts matched in that view | The protection calculation has the same linked evidence and readable history |+| Verified | PBS reported verification for that artifact | Every workload is protected, guest thaw succeeded or an application restore works |+| Coverage posture | Pulse's assessment of linked backup and provider evidence | A recovery guarantee or a reason to delete other backups |++**Attention** can mean an old qualifying backup, a newer failed job, partial+history or missing/overdue verification when expected. **Unprotected** means+Pulse's protection calculation sees complete history but no qualifying backup;+it is not the label for an old successful backup alone. **Unknown** means the+available evidence cannot support a protection claim. A guest-local snapshot+alone is not an independent backup; a PBS backup snapshot is different. See+[protection posture](RECOVERY.md#protection-posture) for the policy distinctions.++If the strip disagrees with recent PBS records, keep the disagreement visible+and use PBS's own backup and verification records to judge coverage meanwhile:++1. In **Proxmox → Backups → Coverage**, expand one affected workload. Note its+ posture explanation and the PBS provider's **Job**, **History** and **Access**+ values (Access describes permissions). If the provider evidence is absent,+ record that rather than assuming access is complete. Use the existing page;+ do not run **Run Diagnostics** or a guest-agent probe to fill the gap.+2. Compare the corresponding **By date** artifact with PBS's own inventory in+ an existing authorised session: PBS connection, datastore, namespace, guest+ type/ID, backup time and verification result. Keep its PVE connection/node+ context too; independent PVE installations can reuse a VMID. A recent backup+ for a same-named or same-numbered guest is not necessarily this workload's+ backup. A successful connection test is not proof of readable backup history.+3. If they still disagree, report just that row's explanation, Job/History/Access+ values, artifact source/time/verification and the running Pulse version.+ Use consistent placeholders for private identities; retain full inventories+ and credentials locally. An absent or unavailable value is not zero or+ evidence that no backup exists.++Do not restart, downgrade, recreate connections, re-enrol agents, delete history,+change retention or run another backup just to make the strip green. Do not+relax TLS or grant write/admin permissions for this check. **Verified** or an+OK backup task does not prove guest thaw or a tested restore. If a workload+stopped responding during a backup, stop this display check and follow the+[backup safety precaution](VM_DISK_MONITORING.md#backup-safety); do not reproduce+freeze/thaw to diagnose a status disagreement.+### Slow Backup LoadingIf you notice slow loading for PBS storage accessed via PVE:diff --git a/docs/RECOVERY.md b/docs/RECOVERY.mdindex 2fc26450f2..6c77d7c4ff 100644--- a/docs/RECOVERY.md+++ b/docs/RECOVERY.md@@ -99,6 +99,11 @@ A retained artifact can remain visible while posture is unknown. A runningbackup is not a new completed recovery point; inspect any earlier completedpoint separately rather than treating current activity as protection.+For PBS artifacts that disagree with the health strip, use the+[backup health checks](PBS.md#backup-health-disagrees-with-visible-pbs-backups)+to compare one workload's explanation and provider evidence without running a+new backup or changing stored history.+## Missing or inconsistent evidence1. Open the relevant platform view, clear its search and filters, and check thediff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdindex 8c5cb2c99e..9805e32e94 100644--- a/docs/TROUBLESHOOTING.md+++ b/docs/TROUBLESHOOTING.md@@ -335,6 +335,14 @@ repair an older generated unit rather than adding a JSON-parsing wrapper.It checks the server service and update timer before the backup, then requiresindependent guest thaw confirmation before starting Pulse again.+#### Backup health disagrees with PBS++A visible or Verified PBS backup is not the same reading as a workload's+Coverage posture. Use the [backup health checks](PBS.md#backup-health-disagrees-with-visible-pbs-backups)+to compare one affected row's explanation, Job, History and Access with the+matching native PBS record. Do not run a new backup, restart or clear history+just to diagnose the disagreement.+#### Temperature data missing- Compare the affected host's active agent version, last report, sensor andobservation time. A current server or another sensor's value is not evidencediff --git a/frontend-modern/public/docs/PBS.md b/frontend-modern/public/docs/PBS.mdindex 8f31f3cda4..d73a6b2925 100644--- a/frontend-modern/public/docs/PBS.md+++ b/frontend-modern/public/docs/PBS.md@@ -362,6 +362,54 @@ Pulse Agent's stored History request. Restarting that guest is not a diagnosisof an empty PBS chart. See [VM Disk Monitoring](VM_DISK_MONITORING.md) if theseparate symptom is missing filesystem information in a PVE VM row.+### Backup health disagrees with visible PBS backups++The backup health strip rates monitored workloads, not individual artifacts.+A count of **with PBS snapshots**, a visible restore point or a **Verified**+label does not by itself establish the workload's protection posture:++| Reading | What it tells you | What it does not prove |+|---|---|---|+| With PBS snapshots / restore points | Pulse lists backup artifacts matched in that view | The protection calculation has the same linked evidence and readable history |+| Verified | PBS reported verification for that artifact | Every workload is protected, guest thaw succeeded or an application restore works |+| Coverage posture | Pulse's assessment of linked backup and provider evidence | A recovery guarantee or a reason to delete other backups |++**Attention** can mean an old qualifying backup, a newer failed job, partial+history or missing/overdue verification when expected. **Unprotected** means+Pulse's protection calculation sees complete history but no qualifying backup;+it is not the label for an old successful backup alone. **Unknown** means the+available evidence cannot support a protection claim. A guest-local snapshot+alone is not an independent backup; a PBS backup snapshot is different. See+[protection posture](RECOVERY.md#protection-posture) for the policy distinctions.++If the strip disagrees with recent PBS records, keep the disagreement visible+and use PBS's own backup and verification records to judge coverage meanwhile:++1. In **Proxmox → Backups → Coverage**, expand one affected workload. Note its+ posture explanation and the PBS provider's **Job**, **History** and **Access**+ values (Access describes permissions). If the provider evidence is absent,+ record that rather than assuming access is complete. Use the existing page;+ do not run **Run Diagnostics** or a guest-agent probe to fill the gap.+2. Compare the corresponding **By date** artifact with PBS's own inventory in+ an existing authorised session: PBS connection, datastore, namespace, guest+ type/ID, backup time and verification result. Keep its PVE connection/node+ context too; independent PVE installations can reuse a VMID. A recent backup+ for a same-named or same-numbered guest is not necessarily this workload's+ backup. A successful connection test is not proof of readable backup history.+3. If they still disagree, report just that row's explanation, Job/History/Access+ values, artifact source/time/verification and the running Pulse version.+ Use consistent placeholders for private identities; retain full inventories+ and credentials locally. An absent or unavailable value is not zero or+ evidence that no backup exists.++Do not restart, downgrade, recreate connections, re-enrol agents, delete history,+change retention or run another backup just to make the strip green. Do not+relax TLS or grant write/admin permissions for this check. **Verified** or an+OK backup task does not prove guest thaw or a tested restore. If a workload+stopped responding during a backup, stop this display check and follow the+[backup safety precaution](VM_DISK_MONITORING.md#backup-safety); do not reproduce+freeze/thaw to diagnose a status disagreement.+### Slow Backup LoadingIf you notice slow loading for PBS storage accessed via PVE:diff --git a/frontend-modern/public/docs/RECOVERY.md b/frontend-modern/public/docs/RECOVERY.mdindex 2fc26450f2..6c77d7c4ff 100644--- a/frontend-modern/public/docs/RECOVERY.md+++ b/frontend-modern/public/docs/RECOVERY.md@@ -99,6 +99,11 @@ A retained artifact can remain visible while posture is unknown. A runningbackup is not a new completed recovery point; inspect any earlier completedpoint separately rather than treating current activity as protection.+For PBS artifacts that disagree with the health strip, use the+[backup health checks](PBS.md#backup-health-disagrees-with-visible-pbs-backups)+to compare one workload's explanation and provider evidence without running a+new backup or changing stored history.+## Missing or inconsistent evidence1. Open the relevant platform view, clear its search and filters, and check thediff --git a/frontend-modern/public/docs/TROUBLESHOOTING.md b/frontend-modern/public/docs/TROUBLESHOOTING.mdindex 8c5cb2c99e..9805e32e94 100644--- a/frontend-modern/public/docs/TROUBLESHOOTING.md+++ b/frontend-modern/public/docs/TROUBLESHOOTING.md@@ -335,6 +335,14 @@ repair an older generated unit rather than adding a JSON-parsing wrapper.It checks the server service and update timer before the backup, then requiresindependent guest thaw confirmation before starting Pulse again.+#### Backup health disagrees with PBS++A visible or Verified PBS backup is not the same reading as a workload's+Coverage posture. Use the [backup health checks](PBS.md#backup-health-disagrees-with-visible-pbs-backups)+to compare one affected row's explanation, Job, History and Access with the+matching native PBS record. Do not run a new backup, restart or clear history+just to diagnose the disagreement.+#### Temperature data missing- Compare the affected host's active agent version, last report, sensor andobservation time. A current server or another sensor's value is not evidencediff --git a/frontend-modern/src/utils/__tests__/backupHealthHelp.test.ts b/frontend-modern/src/utils/__tests__/backupHealthHelp.test.tsnew file mode 100644index 0000000000..baeee83ce5--- /dev/null+++ b/frontend-modern/src/utils/__tests__/backupHealthHelp.test.ts@@ -0,0 +1,102 @@+import { readFileSync } from 'node:fs';+import path from 'node:path';+import { fileURLToPath } from 'node:url';+import { describe, expect, it } from 'vitest';++import { renderDocMarkdown } from '@/features/docs/docMarkdown';++const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../..');+const read = (relative: string): string => readFileSync(path.join(repoRoot, relative), 'utf8');+const guide = read('docs/PBS.md');+const heading = 'Backup health disagrees with visible PBS backups';+const fragment = 'backup-health-disagrees-with-visible-pbs-backups';+const section = guide.split(`### ${heading}\n`)[1]?.split('\n### ')[0] ?? '';+const render = (markdown: string, slug: string) => {+ const article = document.createElement('article');+ article.innerHTML = renderDocMarkdown(markdown, slug);+ return article;+};+const text = () => render(section, 'PBS').textContent?.replace(/\s+/g, ' ');++// These checks exercise the shipped renderer and navigation, not a new+// classification oracle. The owning posture tests establish actual states.+describe('backup health help', () => {+ it('ships the same guide and makes the specific check reachable from both help entry points', () => {+ const target = render(guide, 'PBS');+ expect(target.querySelector(`#${fragment}`)?.textContent).toBe(heading);+ for (const slug of ['PBS', 'RECOVERY', 'TROUBLESHOOTING']) {+ expect(read(`frontend-modern/public/docs/${slug}.md`)).toBe(read(`docs/${slug}.md`));+ }+ for (const slug of ['RECOVERY', 'TROUBLESHOOTING']) {+ const link = render(read(`docs/${slug}.md`), slug).querySelector(+ `a[href="/docs/PBS#${fragment}"]`,+ );+ expect(link?.textContent).toBe('backup health checks');+ expect(link?.hasAttribute('data-doc-link')).toBe(true);+ }+ });++ it('separates listed artifacts, individual verification and workload-level protection', () => {+ const rows = [...render(section, 'PBS').querySelectorAll('tbody tr')];+ expect(rows).toHaveLength(3);+ expect(text()).toContain('rates monitored workloads, not individual artifacts');+ expect(text()).toContain('protection calculation has the same linked evidence');+ expect(text()).toContain(+ 'Every workload is protected, guest thaw succeeded or an application restore works',+ );+ expect(text()).toContain('it is not the label for an old successful backup alone');+ expect(text()).toContain('a PBS backup snapshot is different');+ expect(text()).not.toMatch(/(?:7|seven)[ -]day/);+ expect(render(section, 'PBS').querySelector('pre')).toBeNull();+ });++ it('names the existing expansion and provider labels without inventing a diagnostic action', () => {+ const ui = read('frontend-modern/src/features/proxmox/ProxmoxCoverageTable.tsx');+ expect(ui).toContain('{postureExplanation(row)}');+ for (const [label, field] of [+ ['Job', 'jobState'],+ ['History', 'historyCompleteness'],+ ['Access', 'permissions'],+ ]) {+ expect(ui).toContain(`${label} {evidenceQualityLabel(provider.${field})}`);+ expect(text()).toContain(label);+ }+ expect(text()).toContain('Access describes permissions');+ expect(text()).toContain('If the provider evidence is absent, record that');+ expect(text()).toContain(+ 'Use the existing page; do not run Run Diagnostics or a guest-agent probe',+ );+ });++ it('retains native provenance and missing-value distinctions before requesting evidence', () => {+ expect(text()).toContain("PBS's own backup and verification records");+ expect(text()).toContain('existing authorised session');+ expect(text()).toContain(+ 'datastore, namespace, guest type/ID, backup time and verification result',+ );+ expect(text()).toContain('independent PVE installations can reuse a VMID');+ expect(text()).toContain('not proof of readable backup history');+ expect(text()).toContain('report just that row');+ expect(text()).toContain('retain full inventories and credentials locally');+ expect(text()).toContain('An absent or unavailable value is not zero');+ });++ it('keeps mutation and freeze/thaw out of a display check and resolves both safety links', () => {+ expect(text()).toContain(+ 'Do not restart, downgrade, recreate connections, re-enrol agents, delete history',+ );+ expect(text()).toContain('change retention or run another backup');+ expect(text()).toContain('Do not relax TLS or grant write/admin permissions');+ expect(text()).toContain('stop this display check');+ expect(text()).toContain('do not reproduce freeze/thaw');+ for (const [slug, anchor] of [+ ['RECOVERY', 'protection-posture'],+ ['VM_DISK_MONITORING', 'backup-safety'],+ ]) {+ expect(+ render(section, 'PBS').querySelector(`a[href="/docs/${slug}#${anchor}"]`),+ ).not.toBeNull();+ expect(render(read(`docs/${slug}.md`), slug).querySelector(`#${anchor}`)).not.toBeNull();+ }+ });+});