Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Keep backup inventory failures source-local
pulse-triage[bot]d9292ef38cc7
diff --git a/docs/release-control/v6/internal/subsystems/frontend-primitives.md b/docs/release-control/v6/internal/subsystems/frontend-primitives.mdindex 9f64f9576c..b42308955a 100644--- a/docs/release-control/v6/internal/subsystems/frontend-primitives.md+++ b/docs/release-control/v6/internal/subsystems/frontend-primitives.md@@ -68,6 +68,35 @@ guidance and existing touch/keyboard collapse. Synthetic observations establishpresentation, not native task success, archive restorability, QGA/thaw or releaseavailability; those require their own containing native and delivery evidence.+### Independent backup inventory read ownership++PVE files/snapshots/tasks and PBS inventory have independent, non-suspending+read owners on the existing Backups page. A pending or failed source is not an+empty successful inventory. Keep the other source's readable artifacts, filters+and server health usable; name the missing source beside the evidence, mark+restore-point totals as incomplete and retry only that source. If neither source+can be read, the shared error action retries both. An initial pending read must+not show “No backups yet” or a measured zero PBS count. Only a fulfilled empty+PBS inventory establishes a zero count.++Pending retries retain independently readable evidence. A settled failed read+withdraws that source's artifacts and counts until it recovers, without clearing+the other source or inferring server-owned protection posture. A retry remains+disabled and its previous warning visible while pending. Organisation switches+withdraw prior inventory, replace/abort the read owners and refetch in the new+API context; late responses cannot repopulate old evidence. Disposal aborts+reads and removes subscriptions. This uses the existing query lifecycle, without+new inventory polling or a remount cache.++Verification: mounted `ProxmoxBackupsTable.test.tsx` covers source pending,+partial/both-source failure, isolated retry, fulfilled empty counts, 401/403,+organisation replacement, late response and disposal. Production component,+router, API adapter and stylesheet fixture `browser-tests/backup-inventory.cjs`+checks desktop Chromium and touch WebKit recovery, warning readability, keyboard+or touch retry, source isolation and actual new organisation headers. Synthetic+HTTP results establish frontend truthfulness/lifecycle, not native PBS/PVE+permission, archive restorability, guest thaw or released availability.+### Composed operator tables and conservative rollback consentdiff --git a/docs/release-control/v6/internal/subsystems/storage-recovery.md b/docs/release-control/v6/internal/subsystems/storage-recovery.mdindex ef5654ebc6..0d7f50f36c 100644--- a/docs/release-control/v6/internal/subsystems/storage-recovery.md+++ b/docs/release-control/v6/internal/subsystems/storage-recovery.md@@ -21,6 +21,26 @@## Purpose+### Partial backup inventory is not empty recovery evidence++The existing Proxmox Backups view treats PVE and PBS inventories as independent+reads. Fulfilled artifacts remain usable when the other source is loading or+unavailable. The page identifies that source, marks totals as incomplete and+does not claim no backups exist until both inventories have fulfilled. PBS server+backup counts show loading/unavailable rather than a fabricated zero; current+server health and server-owned protection posture remain independent facts.+Isolated retry preserves filters and the other source, while settled failures+withdraw the failed source's artifacts. Both failed reads have a shared retry.+Organisation changes and disposal cancel/replace reads and prevent late responses+from restoring former inventory. No polling, recovery operation, native identity+rule, protection-policy or provider contract is changed.++Verification: `ProxmoxBackupsTable.test.tsx` exercises pending/partial/both-source+failure, retries, measured empty counts, access denial, org-switch races and+disposal. `browser-tests/backup-inventory.cjs` uses production components,+API adapters, router and CSS in Chromium/WebKit with synthetic HTTP responses;+these controls are not native backup, permission, thaw or release acceptance.+### Backup identity, phone storage and History consent compositionThe Proxmox Coverage last-backup pointer selects completed PBS snapshots or PVEdiff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.mdindex cf80b93be1..19287752f7 100644--- a/docs/release-control/v6/internal/subsystems/unified-resources.md+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md@@ -23,6 +23,18 @@ and sort the complete canonical change table while startup and ingestion wait.## Purpose+### Backup server counts retain their inventory availability++The existing PBS server/datastore row takes backup-inventory read availability+separately from the current canonical resource's health, capacity and identity.+A pending or failed PBS read renders loading/unavailable count text, not zero+backups on an otherwise healthy server. Only a fulfilled inventory establishes+its observed count. Independently readable resources and the existing correlated+drawer target are unchanged. `ProxmoxBackupsTable.test.tsx` covers the combined+count lifecycle; `browser-tests/backup-inventory.cjs` verifies production rows+and source notices on desktop/phone. This changes no resource identity, native+provider assessment or permission policy.+### Guest disk freshness presentationExisting workload disk evidence retains its source reason through the row,diff --git a/frontend-modern/browser-tests/backup-inventory.cjs b/frontend-modern/browser-tests/backup-inventory.cjsnew file mode 100644index 0000000000..8ef2dd06b1--- /dev/null+++ b/frontend-modern/browser-tests/backup-inventory.cjs@@ -0,0 +1,318 @@+// Production backup page, router, API adapters and CSS; synthetic HTTP only.+const assert = require('node:assert/strict');+const fs = require('node:fs');+const path = require('node:path');+const crypto = require('node:crypto');+const { chromium, webkit } = require('playwright');+const sha256 = (p) => crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex');+const root = '/workspace/frontend-modern';+const output = '/workspace/tmp/backup-inventory-proof';+const emptyPVE = { data: { storageBackups: [], guestSnapshots: [], backupTasks: [] } };+const emptyPBS = { data: { backups: [] } };+const pvePayload = {+ data: {+ ...emptyPVE.data,+ storageBackups: [+ {+ id: 'archive-112',+ instance: 'pve-a',+ node: 'pve-a',+ storage: 'local',+ type: 'ct',+ vmid: 112,+ time: '2026-10-03T01:00:00Z',+ format: 'zst',+ volid: 'local:backup/vzdump-lxc-112.tar.zst',+ },+ ],+ },+};+const pbsPayload = {+ data: {+ backups: [+ {+ id: 'pbs-main/main/ct/112/2026-10-03T02:00:00Z',+ instance: 'pbs-main',+ datastore: 'main',+ backupType: 'ct',+ vmid: '112',+ backupTime: '2026-10-03T02:00:00Z',+ files: ['index.json.blob'],+ },+ ],+ },+};+const reply = (body, status = 200) => ({ body, status });+const held = (body, status = 200) => {+ let release;+ const gate = new Promise((resolve) => (release = resolve));+ return { ...reply(body, status), gate, release };+};+(async () => {+ fs.mkdirSync(output, { recursive: true });+ process.chdir(root);+ const playwrightVersion = require('playwright/package.json').version;+ assert.equal(+ playwrightVersion,+ JSON.parse(fs.readFileSync('/workspace/tests/integration/package-lock.json')).packages[+ 'node_modules/@playwright/test'+ ].version,+ );+ const { createServer } = await import(path.join(root, 'node_modules/vite/dist/node/index.js'));+ const server = await createServer({+ cacheDir: path.join(output, 'vite-cache'),+ server: { host: '127.0.0.1', port: 5297, strictPort: true },+ });+ const observations = [];+ let browser;+ try {+ await server.listen();+ for (const [name, engine, phone] of [+ ['chromium-desktop', chromium, false],+ ['webkit-phone', webkit, true],+ ]) {+ browser = await engine.launch(+ engine === chromium+ ? { headless: true, channel: 'chromium', args: ['--no-sandbox'] }+ : { headless: true },+ );+ const viewport = phone ? { width: 390, height: 844 } : { width: 1365, height: 900 };+ const page = await browser.newPage({+ viewport,+ isMobile: phone,+ hasTouch: phone,+ locale: 'en-GB',+ timezoneId: 'UTC',+ });+ page.setDefaultTimeout(20000);+ await page.clock.setFixedTime(new Date('2026-10-03T12:00:00Z'));+ const errors = [],+ requests = [],+ offOrigin = [],+ screenshots = [],+ checks = [];+ page.on('pageerror', (error) => errors.push(error.message));+ const missing = held({}, 503);+ const sources = { pve: reply(emptyPVE), pbs: missing };+ await page.route('**/*', async (route) => {+ const request = route.request();+ const url = new URL(request.url());+ if (url.origin !== 'http://127.0.0.1:5297') {+ offOrigin.push(url.origin);+ return route.abort();+ }+ // Do not intercept Vite's source modules under /src/api/.+ if (!url.pathname.startsWith('/api/')) return route.continue();+ requests.push({+ path: url.pathname,+ method: request.method(),+ org: request.headers()['x-pulse-org-id'] ?? null,+ });+ const key =+ url.pathname === '/api/backups/pbs'+ ? 'pbs'+ : url.pathname === '/api/backups/pve'+ ? 'pve'+ : null;+ const selected = key ? sources[key] : reply({ data: [], policy: {}, meta: {} });+ if (selected.gate) await selected.gate;+ await route.fulfill({+ status: selected.status,+ contentType: 'application/json',+ body: JSON.stringify(selected.body),+ });+ });+ const rows = () => page.locator('[data-proxmox-backup-row="recoverable"]');+ const notice = (source, status) =>+ page.getByRole('status').filter({ hasText: `${source} backup inventory is ${status}` });+ const capture = async (suffix) => {+ await page.evaluate(() =>+ Promise.all(+ document+ .getAnimations()+ .filter((animation) => animation.effect?.getTiming().iterations !== Infinity)+ .map((animation) => animation.finished.catch(() => undefined)),+ ),+ );+ const file = path.join(output, `${name}-${suffix}.png`);+ await page.screenshot({ path: file, fullPage: true });+ screenshots.push({ path: path.relative('/workspace', file), sha256: sha256(file) });+ };+ const activate = async (button) => {+ if (phone) await button.tap();+ else {+ await button.focus();+ await page.keyboard.press('Enter');+ }+ };+ const switchOrg = (org) => page.evaluate((value) => window.__switchBackupOrg(value), org);+ await page.goto('http://127.0.0.1:5297/browser-tests/backup-inventory.html?view=date', {+ waitUntil: 'domcontentloaded',+ });+ if (phone) await page.evaluate(() => document.documentElement.classList.add('dark'));+ await notice('PBS', 'loading').waitFor();+ assert.equal(await page.getByText('No backups yet', { exact: true }).count(), 0);+ await page.getByText('Backup inventory is incomplete', { exact: true }).waitFor();+ if (!phone) await page.locator('[title="PBS backup inventory is loading"]').waitFor();+ await capture('pending');+ missing.release();+ await notice('PBS', 'unavailable').waitFor();+ assert.equal(await page.getByRole('alert').count(), 0);+ const style = await notice('PBS', 'unavailable').evaluate((node) => {+ const rect = node.getBoundingClientRect();+ return {+ color: getComputedStyle(node).color,+ background: getComputedStyle(node).backgroundColor,+ width: rect.width,+ scrollWidth: node.scrollWidth,+ clientWidth: node.clientWidth,+ };+ });+ assert.equal(style.scrollWidth <= style.clientWidth + 1, true);+ assert(+ !style.color.startsWith('rgba') || style.color.endsWith(', 1)'),+ 'warning text must be opaque',+ );+ const retry = page.getByRole('button', { name: 'Retry PBS inventory', exact: true });+ const recovering = held(pbsPayload);+ sources.pbs = recovering;+ await activate(retry);+ assert.equal(await retry.isDisabled(), true);+ await notice('PBS', 'unavailable').waitFor();+ await capture('retrying');+ recovering.release();+ await rows().first().waitFor();+ assert.equal(await rows().count(), 1);+ assert.equal(await notice('PBS', 'unavailable').count(), 0);+ assert.equal(+ requests.filter((r) => r.path === '/api/backups/pve').length,+ 1,+ 'isolated PBS retry does not reread PVE',+ );+ await capture('recovered');+ checks.push(+ 'PBS loading never reads empty/zero; 503 remains contained, warning persists through disabled keyboard/touch retry, and success restores its point',+ );++ const newPVE = held(pvePayload);+ sources.pve = newPVE;+ sources.pbs = reply({}, 403);+ await switchOrg('fixture-other-org');+ await notice('PBS', 'unavailable').waitFor();+ assert.equal(await rows().count(), 0, 'org switch withdraws previous inventory');+ assert.equal(await page.getByText('No backups yet', { exact: true }).count(), 0);+ await page.getByText(/Access denied\. Check your permissions/).waitFor();+ newPVE.release();+ await rows().first().waitFor();+ assert.equal(await rows().count(), 1);+ assert.match(await rows().first().innerText(), /PVE file/);+ if (!phone) await page.locator('[title="PBS backup inventory is unavailable"]').waitFor();+ const pveRow = await rows().first().elementHandle();+ sources.pbs = held(pbsPayload);+ await activate(page.getByRole('button', { name: 'Retry PBS inventory', exact: true }));+ await notice('PBS', 'unavailable').waitFor();+ assert.equal(+ await page.evaluate(+ (node) => node === document.querySelector('[data-proxmox-backup-row="recoverable"]'),+ pveRow,+ ),+ true,+ 'isolated retry preserves readable row DOM',+ );+ sources.pbs.release();+ await page.waitForFunction(+ () => document.querySelectorAll('[data-proxmox-backup-row="recoverable"]').length === 2,+ );+ checks.push(+ '403 removes old PBS points/counts while PVE evidence survives; only success restores PBS, without remounting the independent row',+ );++ sources.pve = reply({}, 500);+ sources.pbs = reply({}, 503);+ await switchOrg('fixture-final-org');+ await page.getByText('Could not load Proxmox backup inventory', { exact: true }).waitFor();+ assert.equal(await rows().count(), 0);+ const finalPVE = held(pvePayload);+ sources.pve = finalPVE;+ sources.pbs = reply(pbsPayload);+ await activate(page.getByRole('button', { name: 'Refresh', exact: true }));+ await notice('PVE', 'unavailable').waitFor();+ assert.equal(await rows().count(), 1, 'PBS recovery is independent of a pending PVE retry');+ finalPVE.release();+ await page.waitForFunction(+ () => document.querySelectorAll('[data-proxmox-backup-row="recoverable"]').length === 2,+ );+ assert.equal(+ await page.getByRole('status').filter({ hasText: 'Counts are incomplete' }).count(),+ 0,+ );+ await activate(page.getByRole('link', { name: 'Coverage', exact: true }));+ await page.locator('[data-proxmox-backups-table="coverage"]').waitFor();+ assert.equal(+ await page+ .getByRole('link', { name: 'Coverage', exact: true })+ .getAttribute('aria-current'),+ 'page',+ );+ await capture('coverage');+ assert.equal(+ await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth + 1),+ true,+ );+ assert.equal(await page.evaluate(() => window.innerWidth), viewport.width);+ for (const org of ['fixture-other-org', 'fixture-final-org']) {+ assert(requests.some((r) => r.path === '/api/backups/pbs' && r.org === org));+ assert(requests.some((r) => r.path === '/api/backups/pve' && r.org === org));+ }+ assert(requests.every((r) => r.method === 'GET'));+ assert.deepEqual(offOrigin, []);+ assert.deepEqual(errors, []);+ checks.push(+ 'both failed inventories retry, first recovered source renders immediately, healthy Coverage navigation remains usable, GET-only new organisation headers and no page overflow/errors',+ );+ observations.push({+ name,+ viewport,+ browser_version: browser.version(),+ style,+ requests,+ checks,+ screenshots,+ });+ await page.close();+ await browser.close();+ browser = null;+ }+ const paths = [+ 'frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx',+ 'frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx',+ ];+ fs.writeFileSync(+ path.join(output, 'result.json'),+ JSON.stringify(+ {+ playwright_version: playwrightVersion,+ content_sha256: Object.fromEntries(+ paths.map((p) => [p, sha256(path.join('/workspace', p))]),+ ),+ observations,+ limits: [+ 'Synthetic responses and organisation contexts, not native permissions, PBS/PVE monitoring, guest thaw, restore verification or release acceptance.',+ 'Browser uses production components and stylesheet through offline Vite; exact production/embed build is separately source-proved.',+ ],+ },+ null,+ 2,+ ),+ );+ console.log(+ 'PASS: independent backup read states, source retry, 403/org withdrawal, recovered Coverage and desktop/phone warning layout.',+ );+ } finally {+ if (browser) await browser.close();+ await server.close();+ }+})().catch((error) => {+ console.error(error);+ process.exitCode = 1;+});diff --git a/frontend-modern/browser-tests/backup-inventory.html b/frontend-modern/browser-tests/backup-inventory.htmlnew file mode 100644index 0000000000..8832c7896b--- /dev/null+++ b/frontend-modern/browser-tests/backup-inventory.html@@ -0,0 +1,12 @@+<!doctype html>+<html lang="en">+ <head>+ <meta charset="UTF-8" />+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />+ <title>Backup inventory read verification</title>+ </head>+ <body>+ <div id="root"></div>+ <script type="module" src="./backup-inventory.tsx"></script>+ </body>+</html>diff --git a/frontend-modern/browser-tests/backup-inventory.tsx b/frontend-modern/browser-tests/backup-inventory.tsxnew file mode 100644index 0000000000..3171373954--- /dev/null+++ b/frontend-modern/browser-tests/backup-inventory.tsx@@ -0,0 +1,64 @@+// Real router/table/API reader/styles; all HTTP evidence is synthetic.+import { ErrorBoundary, onCleanup, onMount } from 'solid-js';+import { render } from 'solid-js/web';+import { Route, Router } from '@solidjs/router';+import { ProxmoxBackupsTable } from '../src/features/proxmox/ProxmoxBackupsTable';+import { eventBus } from '../src/stores/events';+import { setOrgID } from '../src/utils/apiClient';+import type { Resource } from '../src/types/resource';+import '../src/index.css';++const guest = {+ id: 'ct-112',+ type: 'system-container',+ name: 'backup-guest',+ platformId: 'pve-a',+ platformType: 'proxmox-pve',+ sourceType: 'api',+ status: 'running',+ lastSeen: Date.parse('2026-10-03T12:00:00Z'),+ proxmox: { vmid: 112, node: 'pve-a', instance: 'pve-a' },+} as Resource;+const server = {+ id: 'pbs-main',+ type: 'pbs',+ name: 'pbs-main',+ platformId: 'pbs-main',+ platformType: 'proxmox-pbs',+ sourceType: 'api',+ status: 'online',+ lastSeen: Date.parse('2026-10-03T12:00:00Z'),+ pbs: {+ instanceId: 'pbs-main',+ connectionHealth: 'healthy',+ datastores: [{ name: 'main', total: 10_000, used: 4_000, available: 6_000 }],+ },+} as Resource;++function Fixture() {+ onMount(() => {+ Object.assign(window, {+ __switchBackupOrg: (orgID: string) => {+ setOrgID(orgID);+ eventBus.emit('org_switched', orgID);+ },+ });+ onCleanup(() => Reflect.deleteProperty(window, '__switchBackupOrg'));+ });+ return (+ <main class="p-3 sm:p-6">+ <ErrorBoundary fallback={<div role="alert">Backup view failed</div>}>+ <ProxmoxBackupsTable emptyIcon={<span />} workloads={[guest]} servers={[server]} />+ </ErrorBoundary>+ </main>+ );+}++render(+ () => (+ <Router>+ <Route path="/*" component={Fixture} />+ </Router>+ ),+ document.getElementById('root')!,+);diff --git a/frontend-modern/browser-verification/7da9cf49e169b9f3.json b/frontend-modern/browser-verification/7da9cf49e169b9f3.jsonnew file mode 100644index 0000000000..0f459588c4--- /dev/null+++ b/frontend-modern/browser-verification/7da9cf49e169b9f3.json@@ -0,0 +1,58 @@+{+ "version": 1,+ "base_sha": "6140b24bc2bc6e0f272f77bc445758107ab48f45",+ "verified_at": "2026-10-03T20:20:56.516012Z",+ "result": "passed",+ "changed_paths": [+ "frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx",+ "frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx"+ ],+ "content_sha256": {+ "frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx": "3aba53c4eb3019fd47eed67cb1556644fba45b0d7909f30b03c3039806fa2f38",+ "frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx": "b7ab96c153e4b2c84c3afef4c913265b2598bcbb32dd44a5cf9cb612499a2355"+ },+ "routes": [+ "/browser-tests/backup-inventory.html?view=date",+ "/proxmox/backups/coverage"+ ],+ "viewports": [+ {+ "width": 1365,+ "height": 900+ },+ {+ "width": 390,+ "height": 844+ }+ ],+ "states": [+ "PBS loading with fulfilled empty PVE: incomplete inventory, not no backups or zero PBS count",+ "PBS 503 isolated with visible warning during disabled retry and recovered PBS evidence",+ "Organisation replacement: previous artifacts withdrawn before new reads; PBS 403 does not hide independent PVE evidence",+ "Both inventories fail then recover independently; canonical Coverage navigation and current server health preserved"+ ],+ "interactions": [+ "Desktop keyboard Enter and phone touch retry only the failed source",+ "Both-source Refresh and new organisation GET headers; partial results usable without waiting for the other source",+ "Inspect eight settled-animation screenshots at desktop/light and phone/dark widths; warning/count/empty explanation readable, no page overflow"+ ],+ "playwright_version": "1.56.1",+ "browser_versions": [+ {+ "engine": "chromium-desktop",+ "version": "141.0.7390.37"+ },+ {+ "engine": "webkit-phone",+ "version": "26.0"+ }+ ],+ "dependency_snapshot": "8aaf02a3db8e1c245174d04d132398d1716a69525064975470ff738d168abb82",+ "evidence": [+ "Current Web outcome retains complete semantic observations, settled-animation capture subset, baseline, inspected screenshots and exact-source receipts."+ ],+ "limitations": [+ "Synthetic responses and organisation contexts, not native permissions, PBS/PVE monitoring, guest thaw, restore verification or release acceptance.",+ "Browser uses production components and stylesheet through offline Vite; exact production/embed build is separately source-proved."+ ]+}diff --git a/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx b/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsxindex ec407b7b8c..44997db7e3 100644--- a/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx+++ b/frontend-modern/src/features/proxmox/ProxmoxBackupServersTable.tsx@@ -380,6 +380,7 @@ export function buildBackupServerRows(export function ProxmoxBackupServersTable(props: {servers: readonly Resource[];backups?: readonly PBSBackup[];+ backupInventoryState?: 'available' | 'loading' | 'unavailable';emptyIcon?: JSX.Element;layoutWidth?: Accessor<number | null | undefined>;}) {@@ -615,10 +616,27 @@ export function ProxmoxBackupServersTable(props: {class={`${getPlatformTableCellClassForKind('numeric-value')} text-base-content`}><Show when={row.datastore} fallback={<span class="text-muted">—</span>}>- <PlatformTableNumberValue- value={row.backupCount}- format={formatPlatformTableIntegerValue}- />+ <Show+ when={+ !props.backupInventoryState ||+ props.backupInventoryState === 'available'+ }+ fallback={+ <span+ class="text-muted text-[11px]"+ title={`PBS backup inventory is ${props.backupInventoryState}`}+ >+ {props.backupInventoryState === 'loading'+ ? 'Loading'+ : 'Unavailable'}+ </span>+ }+ >+ <PlatformTableNumberValue+ value={row.backupCount}+ format={formatPlatformTableIntegerValue}+ />+ </Show></Show></TableCell></Show>diff --git a/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx b/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsxindex f90874b0d5..e3d51a93b9 100644--- a/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx+++ b/frontend-modern/src/features/proxmox/ProxmoxBackupsTable.tsx@@ -1,9 +1,10 @@-import { Show, createMemo, createResource, createSignal, type Component, type JSX } from 'solid-js';+import { For, Show, createMemo, createSignal, type Component, type JSX } from 'solid-js';import ChevronRightIcon from 'lucide-solid/icons/chevron-right';import { useLocation, useSearchParams } from '@solidjs/router';import { FilterBar, type FilterDef, type FilterSelectOption } from '@/components/shared/FilterBar';import { useBreakpoint } from '@/hooks/useBreakpoint';import { useProtectionPostures } from '@/hooks/useProtectionPostures';+import { createNonSuspendingQuery } from '@/hooks/createNonSuspendingQuery';import {buildProxmoxBackupsPath,PROXMOX_BACKUPS_DEFAULT_VIEW,@@ -12,6 +13,7 @@ import {type ProxmoxBackupsView,} from '@/routing/resourceLinks';import { apiFetch } from '@/utils/apiClient';+import { getAPIReadAccessErrorMessage } from '@/utils/apiAccessError';import {PlatformErrorState,PlatformResourceCounter,@@ -79,28 +81,35 @@ const BACKUP_VIEW_TABS: readonly { id: BackupView; label: string }[] = [{ id: 'coverage', label: 'Coverage' },];-async function fetchPVEBackups(): Promise<PVEBackupsPayload> {- const response = await apiFetch('/api/backups/pve');+const EMPTY_PVE_BACKUPS: PVEBackupsPayload = {+ backupTasks: [],+ storageBackups: [],+ guestSnapshots: [],+};+const EMPTY_PBS_BACKUPS: PBSBackupsPayload = { backups: [] };+const BACKUP_READ_NOTICE_CLASS =+ 'rounded-md border border-amber-300 bg-amber-50 px-3 py-2 text-xs text-amber-900 dark:border-amber-700 dark:bg-amber-950/40 dark:text-amber-100';++async function fetchPVEBackups(signal: AbortSignal): Promise<PVEBackupsPayload> {+ const response = await apiFetch('/api/backups/pve', { signal });if (!response.ok) {- throw new Error(`Failed to load PVE backups (${response.status})`);+ throw Object.assign(new Error(`Failed to load PVE backups (${response.status})`), {+ status: response.status,+ });}const payload = (await response.json()) as PVEBackupsResponse;- return (- payload?.data ?? {- backupTasks: [],- storageBackups: [],- guestSnapshots: [],- }- );+ return payload?.data ?? EMPTY_PVE_BACKUPS;}-async function fetchPBSBackups(): Promise<PBSBackupsPayload> {- const response = await apiFetch('/api/backups/pbs');+async function fetchPBSBackups(signal: AbortSignal): Promise<PBSBackupsPayload> {+ const response = await apiFetch('/api/backups/pbs', { signal });if (!response.ok) {- throw new Error(`Failed to load PBS backups (${response.status})`);+ throw Object.assign(new Error(`Failed to load PBS backups (${response.status})`), {+ status: response.status,+ });}const payload = (await response.json()) as PBSBackupsResponse;- return payload?.data ?? { backups: [] };+ return payload?.data ?? EMPTY_PBS_BACKUPS;}export const ProxmoxBackupsTable: Component<{@@ -108,8 +117,31 @@ export const ProxmoxBackupsTable: Component<{workloads?: readonly Resource[];servers?: readonly Resource[];}> = (props) => {- const [backups, { refetch }] = createResource<PVEBackupsPayload>(fetchPVEBackups);- const [pbsBackups] = createResource<PBSBackupsPayload>(fetchPBSBackups);+ // Independent read owners contain source failures and replace pending reads+ // on org changes/disposal. No new polling or cross-route inventory cache.+ const backups = createNonSuspendingQuery({+ source: () => 'pve',+ fetcher: (_source, signal) => fetchPVEBackups(signal),+ initialValue: EMPTY_PVE_BACKUPS,+ });+ const pbsBackups = createNonSuspendingQuery({+ source: () => 'pbs',+ fetcher: (_source, signal) => fetchPBSBackups(signal),+ initialValue: EMPTY_PBS_BACKUPS,+ });+ const inventorySources = [+ { label: 'PVE', query: backups },+ { label: 'PBS', query: pbsBackups },+ ];+ const inventoryIncomplete = () =>+ inventorySources.some(({ query }) => !query.resolvedOnce() || Boolean(query.error()));+ const hasReadableInventory = () =>+ inventorySources.some(({ query }) => query.resolvedOnce() && !query.error());+ const allInventoriesFailed = () => inventorySources.every(({ query }) => Boolean(query.error()));+ const retryAllInventories = () => {+ if (inventorySources.some(({ query }) => query.loading())) return;+ void Promise.all([backups.refetch(), pbsBackups.refetch(), protectionPostures.refetch()]);+ };const { isMobile } = useBreakpoint();const location = useLocation();const protectionPostures = useProtectionPostures(() =>@@ -261,10 +293,20 @@ export const ProxmoxBackupsTable: Component<{});const toggleDay = (key: string) => setSelectedDateKey(selectedDateKey() === key ? null : key);- const pbsArtifacts = createMemo<PBSBackup[]>(() => pbsBackups()?.backups ?? []);- const snapshots = createMemo<GuestSnapshot[]>(() => backups()?.guestSnapshots ?? []);- const archives = createMemo<StorageBackup[]>(() => backups()?.storageBackups ?? []);- const tasks = createMemo<BackupTask[]>(() => backups()?.backupTasks ?? []);+ // Retain readable observations while a retry is pending, but never treat a+ // settled failed read as a complete inventory or expose its old artifacts.+ const pbsArtifacts = createMemo<PBSBackup[]>(() =>+ pbsBackups.error() ? [] : (pbsBackups.value().backups ?? []),+ );+ const snapshots = createMemo<GuestSnapshot[]>(() =>+ backups.error() ? [] : (backups.value().guestSnapshots ?? []),+ );+ const archives = createMemo<StorageBackup[]>(() =>+ backups.error() ? [] : (backups.value().storageBackups ?? []),+ );+ const tasks = createMemo<BackupTask[]>(() =>+ backups.error() ? [] : (backups.value().backupTasks ?? []),+ );// Render-time `now` snapshot so all age comparisons within a render share a// reference moment; not reactive to ticking time (fine for sysadmin grouping).@@ -628,17 +670,19 @@ export const ProxmoxBackupsTable: Component<{return (<Show- when={!backups.error}+ when={!allInventoriesFailed()}fallback={<PlatformErrorStatetitle="Could not load Proxmox backup inventory"- description={(backups.error as Error | undefined)?.message ?? 'Refresh to retry.'}- onRefresh={() => void refetch()}+ description="PVE and PBS backup inventories are unavailable. Counts are incomplete. Refresh to retry both sources."+ onRefresh={retryAllInventories}/>}><Show- when={backups() !== undefined}+ when={+ hasReadableInventory() || inventorySources.some(({ query }) => Boolean(query.error()))+ }fallback={<PlatformTableLoadingStatetitle="Loading Proxmox backup inventory"@@ -648,17 +692,50 @@ export const ProxmoxBackupsTable: Component<{><div class="space-y-3"><PlatformSectionTabs tabs={backupViewTabs()} active={view()} ariaLabel="Backup views" />+ <For each={inventorySources}>+ {({ label, query }) => (+ <Show when={!query.resolvedOnce() || query.error()}>+ <div+ role="status"+ class={`${BACKUP_READ_NOTICE_CLASS} flex flex-wrap items-center justify-between gap-2`}+ >+ <span>+ {label} backup inventory is {query.error() ? 'unavailable' : 'loading'}. Counts+ are incomplete. {getAPIReadAccessErrorMessage(query.error())}+ </span>+ <Show when={query.error()}>+ <button+ type="button"+ class="min-h-10 shrink-0 rounded-md border border-current px-3 font-medium hover:underline focus-visible:outline-solid focus-visible:outline-2 focus-visible:outline-offset-2 disabled:opacity-50"+ aria-label={`Retry ${label} inventory`}+ disabled={query.loading()}+ onClick={() => void query.refetch()}+ >+ {query.loading() ? 'Retrying…' : `Retry ${label} inventory`}+ </button>+ </Show>+ </div>+ </Show>+ )}+ </For><Show when={protectionPostures.response.error}>- <div- role="status"- class="rounded-md border border-amber-300 bg-amber-50 px-3 py-2 text-xs text-amber-900/25 dark:border-amber-700 dark:bg-amber-950/40 dark:text-amber-100"- >+ <div role="status" class={BACKUP_READ_NOTICE_CLASS}>Protection posture is unavailable. Pulse is showing restore evidence without guessingwhether workloads are protected.</div></Show><Show when={(props.servers?.length ?? 0) > 0}>- <ProxmoxBackupServersTable servers={props.servers ?? []} backups={pbsArtifacts()} />+ <ProxmoxBackupServersTable+ servers={props.servers ?? []}+ backups={pbsArtifacts()}+ backupInventoryState={+ pbsBackups.error()+ ? 'unavailable'+ : pbsBackups.resolvedOnce()+ ? 'available'+ : 'loading'+ }+ /></Show>{/* The health strip renders identically in both route-backed views. */}@@ -667,7 +744,7 @@ export const ProxmoxBackupsTable: Component<{tail={<span>{liveTotalCount()} targets · {recoveryModel().coverageSummary.recoverableArtifacts}{' '}- restore points+ restore points{inventoryIncomplete() ? ' read' : ''}<Show when={recoveryModel().coverageSummary.withPBS > 0}>{' · '}{recoveryModel().coverageSummary.withPBS} with{' '}@@ -800,7 +877,13 @@ export const ProxmoxBackupsTable: Component<{<PlatformResourceCountervisible={view() === 'date' ? visibleRecoverableCount() : visibleLiveCount()}total={view() === 'date' ? totalRecoverableCount() : liveTotalCount()}- rowNoun={view() === 'date' ? 'backups' : 'targets'}+ rowNoun={+ view() === 'date'+ ? inventoryIncomplete()+ ? 'backups read'+ : 'backups'+ : 'targets'+ }/>}/>@@ -810,8 +893,14 @@ export const ProxmoxBackupsTable: Component<{artifacts={filteredRecoverableArtifacts()}hasAnyArtifacts={recoveryModel().recoverableArtifacts.length > 0}emptyIcon={props.emptyIcon}- emptyTitle="No backups yet"- emptyDescription="PBS snapshots, PVE backup files, and guest snapshots will appear here once backups run."+ emptyTitle={+ inventoryIncomplete() ? 'Backup inventory is incomplete' : 'No backups yet'+ }+ emptyDescription={+ inventoryIncomplete()+ ? 'Some backup sources are still loading or unavailable. Missing restore points do not mean no backups exist.'+ : 'PBS snapshots, PVE backup files, and guest snapshots will appear here once backups run.'+ }sortKey={recoverableSortKey}sortDirection={recoverableSortDirection}onSort={handleRecoverableSort}diff --git a/frontend-modern/src/features/proxmox/__tests__/ProxmoxBackupsTable.test.tsx b/frontend-modern/src/features/proxmox/__tests__/ProxmoxBackupsTable.test.tsxindex 6011425fef..ece278c3d6 100644--- a/frontend-modern/src/features/proxmox/__tests__/ProxmoxBackupsTable.test.tsx+++ b/frontend-modern/src/features/proxmox/__tests__/ProxmoxBackupsTable.test.tsx@@ -1,4 +1,4 @@-import { cleanup, fireEvent, render, screen, waitFor } from '@solidjs/testing-library';+import { cleanup, fireEvent, render, screen, waitFor, within } from '@solidjs/testing-library';import { Route, Router } from '@solidjs/router';import { createSignal, type JSX } from 'solid-js';import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';@@ -15,6 +15,7 @@ import {import { TABLE_CARD_FRAME_CLASS } from '@/components/shared/TableCard';import type { Resource } from '@/types/resource';import { getRecoveryFullDateLabel } from '@/utils/recoveryDatePresentation';+import { eventBus } from '@/stores/events';import { resetCreateNonSuspendingQueryCacheForTest } from '@/hooks/createNonSuspendingQuery';// ProxmoxBackupsTable reads URL search params (node/type scope filters), so it@@ -703,3 +704,199 @@ describe('ProxmoxBackupsTable', () => {expect(proxmoxBackupsTableSource).not.toContain('Proxmox Patrol coverage');});});++const emptyPVE = { data: { backupTasks: [], storageBackups: [], guestSnapshots: [] } };+const emptyPBS = { data: { backups: [] } };+const readPvePayload = { data: { ...pvePayload.data, guestSnapshots: [] } };+const deferred = <T,>() => {+ let resolve!: (value: T) => void;+ const promise = new Promise<T>((res) => (resolve = res));+ return { promise, resolve };+};+const mountBackupReadState = () =>+ renderInRouter(() => (+ <ProxmoxBackupsTable+ emptyIcon={<span />}+ workloads={[workloadResource]}+ servers={[pbsServerResource]}+ />+ ));+const recoverableRows = () => document.querySelectorAll('[data-proxmox-backup-row="recoverable"]');+const serverTable = () => document.querySelector('[data-proxmox-backups-table="servers"]')!;+const sourceCalls = (source: 'pve' | 'pbs') =>+ apiFetchMock.mock.calls.filter(([url]) => url === `/api/backups/${source}`);++describe('independent Proxmox backup inventory reads', () => {+ beforeEach(() => {+ window.history.replaceState({}, '', '/proxmox/backups/date');+ apiFetchJSONMock.mockResolvedValue({ data: [], policy: {}, meta: {} });+ });+ it('does not present an unread PBS inventory as empty or a measured zero', async () => {+ const pending = deferred<Response>();+ apiFetchMock.mockImplementation((url) =>+ url.endsWith('/pbs') ? pending.promise : Promise.resolve(jsonResponse(emptyPVE)),+ );+ mountBackupReadState();+ await screen.findByText(/PBS backup inventory is loading/);+ expect(screen.queryByText('No backups yet')).not.toBeInTheDocument();+ expect(screen.getByText('Backup inventory is incomplete')).toBeInTheDocument();+ expect(serverTable()).toHaveTextContent('Loading');+ pending.resolve(jsonResponse(emptyPBS));+ await screen.findByText('No backups yet');+ expect(serverTable()).not.toHaveTextContent('Loading');+ expect(+ within(serverTable() as HTMLElement).getByText('0', { exact: true }),+ ).toBeInTheDocument();+ });++ it('shows fulfilled PBS evidence without waiting for PVE', async () => {+ const pending = deferred<Response>();+ apiFetchMock.mockImplementation((url) =>+ url.endsWith('/pve') ? pending.promise : Promise.resolve(jsonResponse(pbsPayload)),+ );+ mountBackupReadState();+ await screen.findByText(/PVE backup inventory is loading/);+ expect(recoverableRows()).toHaveLength(1);+ expect(screen.getByText(/1 restore points read/)).toBeInTheDocument();+ pending.resolve(jsonResponse(readPvePayload));+ await waitFor(() => expect(recoverableRows()).toHaveLength(2));+ expect(screen.queryByText(/inventory is loading/)).not.toBeInTheDocument();+ });++ it('keeps a source failure actionable even before the other read settles', async () => {+ const pending = deferred<Response>();+ apiFetchMock.mockImplementation((url) =>+ url.endsWith('/pve') ? pending.promise : Promise.resolve(new Response('{}', { status: 503 })),+ );+ mountBackupReadState();+ await screen.findByText(/PBS backup inventory is unavailable/);+ expect(screen.getByText(/PVE backup inventory is loading/)).toBeInTheDocument();+ expect(screen.getByRole('button', { name: 'Retry PBS inventory' })).toBeEnabled();+ expect(screen.getByText('Backup inventory is incomplete')).toBeInTheDocument();+ expect(screen.queryByText('No backups yet')).not.toBeInTheDocument();+ pending.resolve(jsonResponse(readPvePayload));+ await waitFor(() => expect(recoverableRows()).toHaveLength(1));+ });++ it.each(['pve', 'pbs'] as const)(+ 'contains a %s failure, preserves the other source and retries only the failed source',+ async (source) => {+ let response = Promise.resolve(new Response('{}', { status: 503 }));+ apiFetchMock.mockImplementation((url) =>+ url.endsWith(`/${source}`)+ ? response+ : Promise.resolve(jsonResponse(source === 'pve' ? pbsPayload : readPvePayload)),+ );+ mountBackupReadState();+ await screen.findByText(+ new RegExp(`${source.toUpperCase()} backup inventory is unavailable`),+ );+ expect(screen.queryByRole('alert')).not.toBeInTheDocument();+ expect(recoverableRows()).toHaveLength(1);+ if (source === 'pbs') expect(serverTable()).toHaveTextContent('Unavailable');+ const retry = screen.getByRole('button', { name: `Retry ${source.toUpperCase()} inventory` });+ const pending = deferred<Response>();+ response = pending.promise;+ fireEvent.click(retry);+ expect(retry).toBeDisabled();+ fireEvent.click(retry);+ expect(sourceCalls(source)).toHaveLength(2);+ expect(recoverableRows()).toHaveLength(1);+ expect(screen.getByText(/inventory is unavailable/)).toBeInTheDocument();+ pending.resolve(jsonResponse(source === 'pbs' ? pbsPayload : readPvePayload));+ await waitFor(() => expect(recoverableRows()).toHaveLength(2));+ expect(screen.queryByText(/inventory is unavailable/)).not.toBeInTheDocument();+ expect(sourceCalls(source === 'pbs' ? 'pve' : 'pbs')).toHaveLength(1);+ },+ );++ it('retries both failed inventories and allows the first recovered source to render', async () => {+ apiFetchMock.mockResolvedValue(new Response('{}', { status: 500 }));+ mountBackupReadState();+ await screen.findByText('Could not load Proxmox backup inventory');+ expect(screen.queryByText('No backups yet')).not.toBeInTheDocument();+ const pending = deferred<Response>();+ apiFetchMock.mockImplementation((url) =>+ url.endsWith('/pve') ? pending.promise : Promise.resolve(jsonResponse(pbsPayload)),+ );+ fireEvent.click(screen.getByRole('button', { name: 'Refresh' }));+ await screen.findByText(/PVE backup inventory is unavailable/);+ expect(recoverableRows()).toHaveLength(1);+ pending.resolve(jsonResponse(readPvePayload));+ await waitFor(() => expect(recoverableRows()).toHaveLength(2));+ expect(sourceCalls('pve')).toHaveLength(2);+ expect(sourceCalls('pbs')).toHaveLength(2);+ });++ it('does not turn a rejected empty source into proof that no backups exist', async () => {+ apiFetchMock.mockImplementation((url) =>+ Promise.resolve(+ url.endsWith('/pbs') ? new Response('{}', { status: 403 }) : jsonResponse(emptyPVE),+ ),+ );+ mountBackupReadState();+ await screen.findByText(/PBS backup inventory is unavailable/);+ expect(screen.getByText('Backup inventory is incomplete')).toBeInTheDocument();+ expect(screen.queryByText('No backups yet')).not.toBeInTheDocument();+ expect(serverTable()).toHaveTextContent('Unavailable');+ expect(screen.getByText(/Access denied/)).toBeInTheDocument();+ });++ it.each([401, 403])(+ 'withdraws the old organisation before a new %s PBS denial',+ async (status) => {+ apiFetchMock.mockImplementation((url) =>+ Promise.resolve(jsonResponse(url.endsWith('/pbs') ? pbsPayload : readPvePayload)),+ );+ mountBackupReadState();+ await waitFor(() => expect(recoverableRows()).toHaveLength(2));+ const pending = deferred<Response>();+ apiFetchMock.mockImplementation((url) =>+ url.endsWith('/pbs') ? pending.promise : Promise.resolve(jsonResponse(emptyPVE)),+ );+ eventBus.emit('org_switched', 'other-fixture-org');+ expect(recoverableRows()).toHaveLength(0);+ await screen.findByText(/PBS backup inventory is loading/);+ expect(screen.queryByText('No backups yet')).not.toBeInTheDocument();+ pending.resolve(new Response('{}', { status }));+ await screen.findByText(/PBS backup inventory is unavailable/);+ expect(recoverableRows()).toHaveLength(0);+ expect(serverTable()).toHaveTextContent('Unavailable');+ },+ );++ it('aborts replaced org reads and ignores a late earlier response', async () => {+ const old = deferred<Response>();+ apiFetchMock.mockImplementation((url) =>+ url.endsWith('/pbs') ? old.promise : Promise.resolve(jsonResponse(emptyPVE)),+ );+ mountBackupReadState();+ await waitFor(() => expect(sourceCalls('pbs')).toHaveLength(1));+ const oldSignal = sourceCalls('pbs')[0][1].signal as AbortSignal;+ apiFetchMock.mockImplementation((url) =>+ Promise.resolve(jsonResponse(url.endsWith('/pbs') ? emptyPBS : emptyPVE)),+ );+ eventBus.emit('org_switched', 'other-fixture-org');+ expect(oldSignal.aborted).toBe(true);+ await screen.findByText('No backups yet');+ old.resolve(jsonResponse(pbsPayload));+ await old.promise;+ await waitFor(() => expect(recoverableRows()).toHaveLength(0));+ expect(+ within(serverTable() as HTMLElement).getByText('0', { exact: true }),+ ).toBeInTheDocument();+ });++ it('aborts pending reads on disposal and leaves no org-switch readers', async () => {+ const pending = deferred<Response>();+ apiFetchMock.mockReturnValue(pending.promise);+ const view = mountBackupReadState();+ await waitFor(() => expect(apiFetchMock).toHaveBeenCalledTimes(2));+ const signals = apiFetchMock.mock.calls.map(([, options]) => options.signal as AbortSignal);+ view.unmount();+ expect(signals.every((signal) => signal.aborted)).toBe(true);+ eventBus.emit('org_switched', 'after-disposal');+ expect(apiFetchMock).toHaveBeenCalledTimes(2);+ pending.resolve(jsonResponse(emptyPVE));+ });+});