Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Explain the clocks behind backup protection assessments
pulse-triage[bot]bd42426cf010
diff --git a/docs/RECOVERY.md b/docs/RECOVERY.mdindex a788b57934..9391fda1e9 100644--- a/docs/RECOVERY.md+++ b/docs/RECOVERY.md@@ -238,6 +238,42 @@ new backup or changing stored history. For local `vzdump` archives, use the[PVE comparison below](#local-pve-backups-are-newer-than-coverage); PBS is notrequired.+### Protection assessment times++In **Proxmox → Backups → Coverage**, expand the affected workload and read+**Protection assessment**, where available. Its recorded times are shown in+**UTC**, not your browser's local timezone:++- **Evaluated**: when the server assessed protection, not when the backup+ completed or the provider last collected data. A recent evaluation can use+ older or incomplete evidence; read **Provider evidence** and the posture+ explanation as well.+- **Latest successful point**: the successful-point time recorded by that+ assessment. It is not the latest task's result or necessarily **Last backup**+ in the current filtered list.+- **Last verified**: the verification time recorded by the assessment, not+ proof of a tested restore, application health or guest thaw.++These clocks are workload-wide. Selecting a **Backup location** changes the+listed artifacts and their backup age, not the assessment or its times. Compare+[the matching archive](#local-pve-backups-are-newer-than-coverage), owning+installation and guest; repeated names or VMIDs do not establish identity.+Keep an archive's creation time separate from its task's completion time, and+compare the same instant with its timezone, not just the calendar date.++**Not reported** means that clock was not supplied; **Unknown** means its+supplied value could not be interpreted. Neither means zero, no backups or a+failed verification. Do not borrow another clock to fill it. If the section is+absent or the assessment cannot be loaded, a retained artifact or green+connection does not establish a current assessment. Older versions may not+show these details; use existing provider records without rerunning collection.++Use existing observations only; do not run Diagnostics, another backup or a+guest-agent probe, restart monitoring or clear History to reconcile the clocks.+If a guest is unresponsive or freeze/thaw is uncertain, follow+[Backup safety](VM_DISK_MONITORING.md#backup-safety) instead. A current timestamp+or corrected badge does not establish safe workload recovery.+### Backup task results and backup ageA recent backup and a failed task can both be genuine. In **Proxmox → Backups@@ -305,7 +341,9 @@ protected. Compare one existing archive with the same workload in Pulse:for a whole-workload comparison. Selecting a location limits the listedartifacts and backup age, but does not recalculate the workload's posture.A visible archive and a green connection do not establish that the posture- assessment received the same complete, correctly linked evidence.+ assessment received the same complete, correctly linked evidence. Read the+ [Protection assessment times](#protection-assessment-times) separately from+ the archive and task clocks; a recent evaluation is not a fresh collection.4. Compare **Last backup** (**Backup** or **Age** at narrower widths) with thearchive's timestamp. **Restore evidence** can show only a selection of theworkload's points; use **By date** to locate an omitted archive. **Unknown**diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdindex 5b52b1eb0e..ded3f56762 100644--- a/docs/TROUBLESHOOTING.md+++ b/docs/TROUBLESHOOTING.md@@ -41,6 +41,10 @@ establish safe recovery.Unknown is not zero, and a working Pulse Agent view does not prove API recovery.If only one disk is absent, use [missing-filesystem checks](VM_DISK_MONITORING.md#one-filesystem-is-missing-but-other-disks-have-readings);a container-data directory can hold a real disk, independently of backup selection.+- **Backups are visible but protection looks stale:** inspect the workload's+ [recorded assessment times](RECOVERY.md#protection-assessment-times) beside+ its backup and task evidence. A recent evaluation is not a fresh backup or+ proof of fresh collection; do not run another backup to reconcile a badge.- **Proxmox data belongs to the wrong host, or healthy jobs look Pending:** use[cross-installation identity checks](TROUBLESHOOTING.md#monitoring-is-mixed-between-proxmox-installations)or [replication status checks](TROUBLESHOOTING.md#replication-jobs-are-pending-stale-or-missing).diff --git a/frontend-modern/public/docs/RECOVERY.md b/frontend-modern/public/docs/RECOVERY.mdindex a788b57934..9391fda1e9 100644--- a/frontend-modern/public/docs/RECOVERY.md+++ b/frontend-modern/public/docs/RECOVERY.md@@ -238,6 +238,42 @@ new backup or changing stored history. For local `vzdump` archives, use the[PVE comparison below](#local-pve-backups-are-newer-than-coverage); PBS is notrequired.+### Protection assessment times++In **Proxmox → Backups → Coverage**, expand the affected workload and read+**Protection assessment**, where available. Its recorded times are shown in+**UTC**, not your browser's local timezone:++- **Evaluated**: when the server assessed protection, not when the backup+ completed or the provider last collected data. A recent evaluation can use+ older or incomplete evidence; read **Provider evidence** and the posture+ explanation as well.+- **Latest successful point**: the successful-point time recorded by that+ assessment. It is not the latest task's result or necessarily **Last backup**+ in the current filtered list.+- **Last verified**: the verification time recorded by the assessment, not+ proof of a tested restore, application health or guest thaw.++These clocks are workload-wide. Selecting a **Backup location** changes the+listed artifacts and their backup age, not the assessment or its times. Compare+[the matching archive](#local-pve-backups-are-newer-than-coverage), owning+installation and guest; repeated names or VMIDs do not establish identity.+Keep an archive's creation time separate from its task's completion time, and+compare the same instant with its timezone, not just the calendar date.++**Not reported** means that clock was not supplied; **Unknown** means its+supplied value could not be interpreted. Neither means zero, no backups or a+failed verification. Do not borrow another clock to fill it. If the section is+absent or the assessment cannot be loaded, a retained artifact or green+connection does not establish a current assessment. Older versions may not+show these details; use existing provider records without rerunning collection.++Use existing observations only; do not run Diagnostics, another backup or a+guest-agent probe, restart monitoring or clear History to reconcile the clocks.+If a guest is unresponsive or freeze/thaw is uncertain, follow+[Backup safety](VM_DISK_MONITORING.md#backup-safety) instead. A current timestamp+or corrected badge does not establish safe workload recovery.+### Backup task results and backup ageA recent backup and a failed task can both be genuine. In **Proxmox → Backups@@ -305,7 +341,9 @@ protected. Compare one existing archive with the same workload in Pulse:for a whole-workload comparison. Selecting a location limits the listedartifacts and backup age, but does not recalculate the workload's posture.A visible archive and a green connection do not establish that the posture- assessment received the same complete, correctly linked evidence.+ assessment received the same complete, correctly linked evidence. Read the+ [Protection assessment times](#protection-assessment-times) separately from+ the archive and task clocks; a recent evaluation is not a fresh collection.4. Compare **Last backup** (**Backup** or **Age** at narrower widths) with thearchive's timestamp. **Restore evidence** can show only a selection of theworkload's points; use **By date** to locate an omitted archive. **Unknown**diff --git a/frontend-modern/public/docs/TROUBLESHOOTING.md b/frontend-modern/public/docs/TROUBLESHOOTING.mdindex 5b52b1eb0e..ded3f56762 100644--- a/frontend-modern/public/docs/TROUBLESHOOTING.md+++ b/frontend-modern/public/docs/TROUBLESHOOTING.md@@ -41,6 +41,10 @@ establish safe recovery.Unknown is not zero, and a working Pulse Agent view does not prove API recovery.If only one disk is absent, use [missing-filesystem checks](VM_DISK_MONITORING.md#one-filesystem-is-missing-but-other-disks-have-readings);a container-data directory can hold a real disk, independently of backup selection.+- **Backups are visible but protection looks stale:** inspect the workload's+ [recorded assessment times](RECOVERY.md#protection-assessment-times) beside+ its backup and task evidence. A recent evaluation is not a fresh backup or+ proof of fresh collection; do not run another backup to reconcile a badge.- **Proxmox data belongs to the wrong host, or healthy jobs look Pending:** use[cross-installation identity checks](TROUBLESHOOTING.md#monitoring-is-mixed-between-proxmox-installations)or [replication status checks](TROUBLESHOOTING.md#replication-jobs-are-pending-stale-or-missing).diff --git a/frontend-modern/src/features/docs/__tests__/backupAssessmentTimesGuide.test.ts b/frontend-modern/src/features/docs/__tests__/backupAssessmentTimesGuide.test.tsnew file mode 100644index 0000000000..0e9ea9d3cf--- /dev/null+++ b/frontend-modern/src/features/docs/__tests__/backupAssessmentTimesGuide.test.ts@@ -0,0 +1,86 @@+import { readFileSync } from 'node:fs';+import path from 'node:path';+import { fileURLToPath } from 'node:url';+import { describe, expect, it } from 'vitest';+import { createGuideFixture } from './guideFixture';++const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../../..');+const read = (file: string) => readFileSync(path.join(root, file), 'utf8');+const guide = createGuideFixture((name) => read(`frontend-modern/public/docs/${name}.md`));+const fragment = '#protection-assessment-times';+const section = () => {+ const heading = guide('RECOVERY').querySelector(fragment);+ expect(heading?.textContent).toBe('Protection assessment times');+ const result = document.createElement('section');+ for (let next = heading!.nextElementSibling; next; next = next.nextElementSibling) {+ if (/^H[1-3]$/.test(next.tagName)) break;+ result.appendChild(next.cloneNode(true));+ }+ return result;+};+const prose = () => section().textContent!.replace(/\s+/g, ' ');++describe('backup assessment time guidance', () => {+ it('routes the stale-protection symptom and archive comparison to the shipped explanation', () => {+ const link = guide('TROUBLESHOOTING').querySelector(`a[href="/docs/RECOVERY${fragment}"]`);+ expect(link?.textContent).toBe('recorded assessment times');+ expect(link?.hasAttribute('data-doc-link')).toBe(true);+ expect(link?.hasAttribute('target')).toBe(false);+ expect(guide('RECOVERY').querySelector(`a[href="${fragment}"]`)).not.toBeNull();+ for (const name of ['RECOVERY', 'TROUBLESHOOTING']) {+ expect(read(`frontend-modern/public/docs/${name}.md`)).toBe(read(`docs/${name}.md`));+ }+ });++ it('explains all three assessment clocks without substituting task, collection or filter times', () => {+ const rows = [...section().querySelectorAll('ul > li')];+ expect(rows.map((row) => row.querySelector('strong')?.textContent)).toEqual([+ 'Evaluated',+ 'Latest successful point',+ 'Last verified',+ ]);+ const text = prose();+ for (const distinction of [+ "UTC, not your browser's local timezone",+ 'not when the backup completed or the provider last collected data',+ 'A recent evaluation can use older or incomplete evidence',+ "not the latest task's result or necessarily Last backup",+ 'These clocks are workload-wide',+ 'not the assessment or its times',+ 'compare the same instant with its timezone, not just the calendar date',+ ]) {+ expect(text).toContain(distinction);+ }+ });++ it('keeps missing, invalid and unavailable assessment evidence honest', () => {+ const text = prose();+ for (const distinction of [+ 'Not reported means that clock was not supplied',+ 'Unknown means its supplied value could not be interpreted',+ 'Neither means zero, no backups or a failed verification',+ 'Do not borrow another clock to fill it',+ 'green connection does not establish a current assessment',+ 'Older versions may not show these details',+ ]) {+ expect(text).toContain(distinction);+ }+ });++ it('uses passive comparison and preserves the independent guest-safety path', () => {+ const text = prose();+ expect(section().querySelector('pre')).toBeNull();+ expect(text).toContain('repeated names or VMIDs do not establish identity');+ expect(text).toContain('use existing provider records without rerunning collection');+ expect(text).toContain('do not run Diagnostics, another backup or a guest-agent probe');+ expect(text).toContain('restart monitoring or clear History');+ expect(text).toContain('not proof of a tested restore, application health or guest thaw');+ expect(text).toContain('does not establish safe workload recovery');+ const safety = section().querySelector('a[href="/docs/VM_DISK_MONITORING#backup-safety"]');+ expect(safety).not.toBeNull();+ expect(guide('VM_DISK_MONITORING').querySelector('#backup-safety')).not.toBeNull();+ const archive = section().querySelector('a[href="#local-pve-backups-are-newer-than-coverage"]');+ expect(archive).not.toBeNull();+ expect(guide('RECOVERY').querySelector(archive!.getAttribute('href')!)).not.toBeNull();+ });+});diff --git a/frontend-modern/src/features/docs/__tests__/troubleshootingEntryGuide.test.ts b/frontend-modern/src/features/docs/__tests__/troubleshootingEntryGuide.test.tsindex 5f4b28c6ad..6518667776 100644--- a/frontend-modern/src/features/docs/__tests__/troubleshootingEntryGuide.test.ts+++ b/frontend-modern/src/features/docs/__tests__/troubleshootingEntryGuide.test.ts@@ -16,6 +16,7 @@ const entryGuideNames = ['UNIFIED_AGENT','TRUENAS','PBS',+ 'RECOVERY',];function renderGuide(name: string): HTMLElement {@@ -113,6 +114,10 @@ describe('troubleshooting workflow entry', () => {'/docs/TROUBLESHOOTING#guest-memory-is-high-but-available-memory-is-plentiful',],],+ [+ 'Backups are visible but protection looks stale:',+ ['/docs/RECOVERY#protection-assessment-times'],+ ],['Proxmox data belongs to the wrong host, or healthy jobs look Pending:',[