Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Settle slow polled reads and keep replication links honest
rcourtmanba26b3581e2b
diff --git a/docs/release-control/v6/internal/subsystems/frontend-primitives.md b/docs/release-control/v6/internal/subsystems/frontend-primitives.mdindex 710f4ef07e..2bcd810d94 100644--- a/docs/release-control/v6/internal/subsystems/frontend-primitives.md+++ b/docs/release-control/v6/internal/subsystems/frontend-primitives.md@@ -4649,6 +4649,21 @@ Each query run receives an `AbortSignal`; changing the source, resetting thequery, or unmounting its owner must abort the superseded browser request beforestarting replacement work. Consumers must forward that signal through theirAPI/cache layer when the transport supports cancellation.+A poll (`pollMs`) does not replace a read that an earlier poll started while+that read is still in flight; it skips the tick. The API answers 429 with+`Retry-After: 60` and `apiFetch` honours up to two minutes, so a polled read+slower than the poll interval would otherwise be discarded on every tick and+never settle. Any other read (a refresh, a source change, a remount+revalidation) stays replaceable at the next tick, so a slow or hung refresh is+replaced at most once before polling protects its replacement. A polled read+still in flight at the first poll tick at or after `STALLED_QUERY_READ_MS` (150+seconds) is presumed stalled: that tick replaces it and settles it as a failed+read, clearing `loading`, setting `resolvedOnce` and publishing a timed-out+`error`, so no consumer is left waiting on a read that never returns. Each+consumer renders that failure as it renders any other (the Proxmox Replication+table shows its error and Retry; workload sparklines keep retained series). A+transport that always takes longer than that deadline plus one poll interval+never publishes.The settings reporting shell now also owns a deliberate split betweenhistorical performance reports and current-state VM inventory export.`frontend-modern/src/components/Settings/ReportingPanel.tsx`,@@ -5210,9 +5225,10 @@ cells: a value formatted through `formatPlatformTableRelativeTimeValue` orand tooltips, an open drawer's summary rows and annotations) passes `now` fromthe owning component's `useRelativeTimeNow`, and a countdown or freshness bandderived from the same time (replication Next sync, backup age bands) reads it-too. The clock can trail the wall clock by up to one tick, so a check that-treats a future time as invalid (the backup age band) measures from the later-of the two.+too. Every read of the clock returns the wall clock; the 30-second tick only+tells readers to re-read, so a cell that mounts between ticks never measures+from a stale time and a timestamp from the last few seconds never reads as a+future time.Read-only metadata badges follow the same primitive-owned shell rule.`frontend-modern/src/components/shared/MetadataBadge.tsx` owns filled andoutlined appearances, compact sizing, shape, typed tone vocabulary, fitdiff --git a/docs/release-control/v6/internal/subsystems/storage-recovery.md b/docs/release-control/v6/internal/subsystems/storage-recovery.mdindex 626b08f763..6d712201d4 100644--- a/docs/release-control/v6/internal/subsystems/storage-recovery.md+++ b/docs/release-control/v6/internal/subsystems/storage-recovery.md@@ -986,15 +986,21 @@ composition, and visible source/state chips must render through `MetadataBadge`instead of restoring local rounded-sm xs badge spans.Backup ages read the frontend-primitives shared relative-time clock:`ProxmoxBackupAgeText` measures the full and compact age text and its-current, aging, or stale band from `useRelativeTimeNow`, never earlier than the-wall clock, so a backup row that stays mounted for days moves from current to-aging without new inventory, and a backup that finished inside the clock's last-tick never bands as a future, unknown age. `ProxmoxPageSurface` reads+current, aging, or stale band from `useRelativeTimeNow`, so a backup row that+stays mounted for days moves from current to aging without new inventory, and+because every clock read returns the wall clock, a backup that finished since+the last tick never bands as a future, unknown age. `ProxmoxPageSurface` readsreplication jobs through `createNonSuspendingQuery` with a 30-second backgroundpoll, because the Replication table's Last sync and Next sync move on the same-clock: a failed read reports the error but keeps the last jobs and the tab, and-a direct replication link holds until the first read resolves instead of-falling back to Overview. The poll re-reads the existing read-only+clock. Each read forwards the query's abort signal, so a replaced or unmounted+read is cancelled. Whether replication exists is unknown until a read+succeeds, counted from the last reset (an organization switch): a failed read+before then holds a direct replication link on the error and its Retry instead+of falling back to Overview, and once a read has succeeded a failed poll+reports the error but keeps that answer (the jobs and the tab, or the Overview+fallback for an empty answer) rather than reopening the route. An access+denial (401 or 403) withdraws the jobs and the tab, and a direct link holds on+the denial so the operator sees why. The poll re-reads the existing read-only`/api/replication/jobs` snapshot and opens no new storage or recovery path.Proxmox backup-health and recoverable-artifact summary rows follow the sameplatform-table density contract as every other provider surface. Storage anddiff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.mdindex 5c1dfdb1bb..d8f30a735c 100644--- a/docs/release-control/v6/internal/subsystems/unified-resources.md+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md@@ -1274,9 +1274,14 @@ tooltip, and the Proxmox replication Last sync and Next sync in the row, phoneprojection and disclosure. A silent agent or a stalled pvesr scheduler is thecase whose age matters most, so a replication Next sync turns overdue on theclock without new data. Replication jobs bypass the unified-resource stream,-so `ProxmoxPageSurface` re-reads them in the background every 30 seconds;-moving ages over a snapshot read once at mount would age a job that keeps-syncing and count it overdue.+so `ProxmoxPageSurface` polls them in the background every 30 seconds,+skipping a tick while the previous poll's read is still in flight unless that+read has stalled past `STALLED_QUERY_READ_MS`; moving ages over a snapshot read+once at mount would age a job that keeps syncing and count it overdue.+`fetchReplicationJobs` takes the query's abort signal and passes it to+`apiFetch`, so a replaced or unmounted replication read stops instead of+running on in the background, and it throws `apiErrorFromResponse` errors that+keep the HTTP status, so a 401 or 403 withdraws the jobs as an access failure.Kubernetes name columns hold the chevron, status dot and name inside onetrack, so their md widths leave the name room for a typical node or servicename at a 768px viewport: Nodes 20 percent (with Capacity at 16 so itsdiff --git a/frontend-modern/browser-verification/0a3f56b9810b4db5.json b/frontend-modern/browser-verification/0a3f56b9810b4db5.jsonnew file mode 100644index 0000000000..70d8f76336--- /dev/null+++ b/frontend-modern/browser-verification/0a3f56b9810b4db5.json@@ -0,0 +1,52 @@+{+ "version": 1,+ "base_sha": "9aaac4d3e502c01913fb3637d0997b0ce03a8b7b",+ "verified_at": "2026-10-06T19:14:07Z",+ "result": "passed",+ "changed_paths": [+ "frontend-modern/src/features/proxmox/ProxmoxPageSurface.tsx",+ "frontend-modern/src/features/proxmox/ProxmoxReplicationTable.tsx",+ "frontend-modern/src/features/proxmox/proxmoxBackupsTableShared.tsx",+ "frontend-modern/src/hooks/createNonSuspendingQuery.ts",+ "frontend-modern/src/utils/relativeTimeClock.ts"+ ],+ "content_sha256": {+ "frontend-modern/src/features/proxmox/ProxmoxPageSurface.tsx": "306a0f3a95585efd4d980549f33a41538a00975f62aff39873a75145e4d9bf5f",+ "frontend-modern/src/features/proxmox/ProxmoxReplicationTable.tsx": "e56e51e972457dbfe34f5695b69423f1ac916df3213a61d028e05cbcecfd12bd",+ "frontend-modern/src/features/proxmox/proxmoxBackupsTableShared.tsx": "33fd51603010d13040faa6fe6b3690d3e4bbe1c941c32ceb1aa0c972ec47370c",+ "frontend-modern/src/hooks/createNonSuspendingQuery.ts": "6780323e7d9055b0839da57b6204fd68bfce9696e622362ea34a02d0907554bd",+ "frontend-modern/src/utils/relativeTimeClock.ts": "0ff094e2dc98bf07b3792af478bfa90d3156143ddc5aef0e233de80868ec4dba"+ },+ "routes": [+ "/proxmox/replication",+ "/proxmox/backups",+ "/docker/overview",+ "/standalone/machines"+ ],+ "viewports": [+ {+ "width": 1600,+ "height": 1000+ },+ {+ "width": 390,+ "height": 844+ }+ ],+ "states": [+ "Private mock stack built from this tree on origin/main 9aaac4d3e5; Playwright drove it with /api/replication/jobs answers shaped per scenario through page.route",+ "Direct /proxmox/replication link whose first read returns 500, at 1600px and 390px: stays on the route with \"Could not load replication jobs\" and Refresh (main's code fell back to Overview with no error); Refresh loads all 8 jobs",+ "Jobs loaded, then a poll read answers 403: the Replication tab is withdrawn and the link shows the server's \"Insufficient permissions to read replication jobs\" with Refresh (main kept the tab advertised and a generic message)",+ "Empty first answer, then a failed poll, then an empty answer: the link stays on Overview throughout and the route never reopens",+ "page.clock installed, a polled replication read held open: the next two 30s poll ticks start no new request and abort none; releasing it lets it finish and the following tick starts the next read (main started reads 3 and 4 over the held read and never cancelled it)",+ "Leaving the Proxmox page for Docker while a replication read is held aborts it (net::ERR_ABORTED); main let it run",+ "Regression: with /ws frames held and keep-alive pongs, Machines Seen cells move from \"just now\" to \"10m ago\" over ten one-minute clock steps on one socket",+ "Proxmox Backups at 1600px and 390px after the clock change retired the band's Math.max: 36 age cells each, none Unknown, every band colour matching its Current/Aging/Stale title, no page errors",+ "Same-session control: the five runtime files swapped back to main under the running Vite reproduced every failure above, then restored and compared byte for byte"+ ],+ "interactions": [+ "Clicked Refresh on the replication error state at both widths",+ "Clicked the Docker nav link while a replication read was in flight",+ "page.clock.runFor poll steps with data-free pong frames sent before each step"+ ]+}diff --git a/frontend-modern/src/components/Infrastructure/__tests__/ResourceDetailSummary.relativeAge.test.tsx b/frontend-modern/src/components/Infrastructure/__tests__/ResourceDetailSummary.relativeAge.test.tsxindex bf17540823..786914a867 100644--- a/frontend-modern/src/components/Infrastructure/__tests__/ResourceDetailSummary.relativeAge.test.tsx+++ b/frontend-modern/src/components/Infrastructure/__tests__/ResourceDetailSummary.relativeAge.test.tsx@@ -4,6 +4,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest';import { formatPlatformTableDateTimeValue } from '@/features/platformPage/sharedPlatformPage';import type { Resource } from '@/types/resource';+import { RELATIVE_TIME_TICK_MS } from '@/utils/relativeTimeClock';import { InlineResourceSummaryTables } from '../ResourceDetailSummary';import { useResourceDetailDrawerDerivedState } from '../useResourceDetailDrawerDerivedState';import type { UseResourceDetailDrawerStateResult } from '../useResourceDetailDrawerState';@@ -47,6 +48,12 @@ afterEach(() => {vi.useRealTimers();});+// Time passes with no new data; the next clock tick re-reads the ages.+const hoursLater = (hours: number) => {+ vi.setSystemTime(Date.now() + hours * 60 * 60 * 1000);+ vi.advanceTimersByTime(RELATIVE_TIME_TICK_MS);+};+describe('resource drawer summary relative ages', () => {it('keeps an open Kubernetes controller summary age moving without new data', () => {vi.useFakeTimers({ now: new Date('2026-05-24T13:31:00Z') });@@ -73,7 +80,7 @@ describe('resource drawer summary relative ages', () => {within(section()).getByText(`${absoluteTime('2026-05-24T13:00:00Z')} (31m ago)`),).toBeInTheDocument();- vi.advanceTimersByTime(2 * 60 * 60 * 1000);+ hoursLater(2);// The absolute time is unchanged; only the age after it moves.expect(@@ -95,6 +102,7 @@ describe('resource drawer summary relative ages', () => {containerState: 'running',createdAt: '2026-05-24T11:31:00Z',startedAt: '2026-05-24T13:21:00Z',+ finishedAt: '2026-05-24T12:31:00Z',},})}drawer={drawerState}@@ -108,14 +116,16 @@ describe('resource drawer summary relative ages', () => {.closest('tr');expect(row('Created')).toHaveTextContent('2 hours ago');expect(row('Started')).toHaveTextContent('10 mins ago');+ expect(row('Finished')).toHaveTextContent('1 hour ago');- vi.advanceTimersByTime(3 * 60 * 60 * 1000);+ hoursLater(3);expect(row('Created')).toHaveTextContent('5 hours ago');expect(row('Started')).toHaveTextContent('3 hours ago');+ expect(row('Finished')).toHaveTextContent('4 hours ago');});- it('keeps an open drawer Last seen moving for a resource that stops reporting', () => {+ it('keeps an open drawer Last seen and platform freshness ages moving without new data', () => {vi.useFakeTimers({ now: new Date('2026-05-24T13:31:00Z') });const { state, dispose } = createRoot((dispose) => ({@@ -124,7 +134,11 @@ describe('resource drawer summary relative ages', () => {id: 'node-1',type: 'agent',lastSeen: Date.parse('2026-05-24T13:30:00Z'),- }),+ platformData: {+ docker: { updatesLastCheckedAt: '2026-05-24T13:01:00Z' },+ pmg: { lastUpdated: '2026-05-24T13:21:00Z' },+ },+ } as Partial<Resource> & Pick<Resource, 'id' | 'type'>),debugEnabled: () => false,discoveryFeatureEnabled: () => false,resourceIntelligence: () => null,@@ -133,11 +147,15 @@ describe('resource drawer summary relative ages', () => {}));expect(state.lastSeen()).toBe('1 min ago');+ expect(state.dockerUpdatesCheckedRelative()).toBe('30 mins ago');+ expect(state.pmgUpdatedRelative()).toBe('10 mins ago');- // The resource stays silent: its lastSeen never changes again.- vi.advanceTimersByTime(2 * 60 * 60 * 1000);+ // The resource stays silent: none of its timestamps change again.+ hoursLater(2);expect(state.lastSeen()).toBe('2 hours ago');+ expect(state.dockerUpdatesCheckedRelative()).toBe('2 hours ago');+ expect(state.pmgUpdatedRelative()).toBe('2 hours ago');dispose();});});diff --git a/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.relativeAge.test.tsx b/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.relativeAge.test.tsxindex 7b547bf664..4f740ca6e7 100644--- a/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.relativeAge.test.tsx+++ b/frontend-modern/src/features/kubernetes/__tests__/KubernetesControllersTable.relativeAge.test.tsx@@ -17,6 +17,7 @@ vi.mock('@/api/resources', () => ({}));import type { Resource } from '@/types/resource';+import { RELATIVE_TIME_TICK_MS } from '@/utils/relativeTimeClock';import { KubernetesControllersTable } from '../KubernetesControllersTable';const makeResource = ({@@ -85,7 +86,8 @@ describe('KubernetesControllersTable relative ages', () => {// No data changes: the timestamps stay put and only the clock moves. A// detail computed once at render would still read "31m ago" here.- vi.advanceTimersByTime(3 * 60 * 60 * 1000);+ vi.setSystemTime(Date.parse('2026-05-24T16:31:00Z'));+ vi.advanceTimersByTime(RELATIVE_TIME_TICK_MS);expect(screen.getByText('Completed 3h ago')).toBeInTheDocument();expect(screen.getByText('Last success 3h ago')).toBeInTheDocument();diff --git a/frontend-modern/src/features/proxmox/ProxmoxPageSurface.tsx b/frontend-modern/src/features/proxmox/ProxmoxPageSurface.tsxindex 118fd48185..0cf4d4c4ba 100644--- a/frontend-modern/src/features/proxmox/ProxmoxPageSurface.tsx+++ b/frontend-modern/src/features/proxmox/ProxmoxPageSurface.tsx@@ -52,6 +52,7 @@ import { ProxmoxMailGatewayTable } from './ProxmoxMailGatewayTable';import { ProxmoxNodesTable } from './ProxmoxNodesTable';import { ProxmoxReplicationTable, fetchReplicationJobs } from './ProxmoxReplicationTable';import { createNonSuspendingQuery } from '@/hooks/createNonSuspendingQuery';+import { getAPIReadAccessErrorMessage } from '@/utils/apiAccessError';import { useUnifiedResources } from '@/hooks/useUnifiedResources';import type { ReplicationJob } from '@/types/api';import type { Resource } from '@/types/resource';@@ -127,11 +128,22 @@ export function ProxmoxPageSurface() {// last jobs, so the tab does not vanish under the user.const replicationJobs = createNonSuspendingQuery({source: () => 'proxmox-pve',- fetcher: () => fetchReplicationJobs(),+ fetcher: (_source, signal) => fetchReplicationJobs(signal),initialValue: NO_REPLICATION_JOBS,pollMs: REPLICATION_JOBS_POLL_MS,});const replicationJobCount = createMemo(() => replicationJobs.value().length);+ // Replication presence is known once a read has succeeded. A failed read+ // before that says nothing, so a direct link holds on its error and Retry+ // instead of falling to Overview; a failed poll after that keeps the last+ // answer rather than reopening a route an empty read already closed. An+ // access denial withdraws that answer, so the link holds on the denial.+ const replicationJobsConfirmed = createMemo<boolean>((confirmed) => {+ if (!replicationJobs.resolvedOnce()) return false;+ const error = replicationJobs.error();+ if (getAPIReadAccessErrorMessage(error)) return false;+ return confirmed || !error;+ }, false);const visibleTabs = createMemo(() => {// An unknown snapshot is not evidence that every optional integration is// present. Never use estate-wide aggregations here: unrelated VMware VMs@@ -149,7 +161,7 @@ export function ProxmoxPageSurface() {// Do not discard a direct link while counts are still unknown: its own// resource query must be allowed to hydrate before deciding it is absent.const countsUnknown =- !tabEvidence.facets?.() || (requested === 'replication' && !replicationJobs.resolvedOnce());+ !tabEvidence.facets?.() || (requested === 'replication' && !replicationJobsConfirmed());return countsUnknown || visibleTabIds().has(requested) ? requested : 'overview';});const shouldHydrateTab = (tab: ProxmoxPageTabId) => activeTab() === tab;diff --git a/frontend-modern/src/features/proxmox/ProxmoxReplicationTable.tsx b/frontend-modern/src/features/proxmox/ProxmoxReplicationTable.tsxindex 918f4d8486..5a7e714cf4 100644--- a/frontend-modern/src/features/proxmox/ProxmoxReplicationTable.tsx+++ b/frontend-modern/src/features/proxmox/ProxmoxReplicationTable.tsx@@ -5,7 +5,7 @@ import { InlineDetailTableRow } from '@/components/shared/InlineDetailTableRow';import { StatusDot } from '@/components/shared/StatusDot';import type { StatusIndicatorVariant } from '@/utils/status';import { TableCell, TableHead, TableRow } from '@/components/shared/Table';-import { apiFetch } from '@/utils/apiClient';+import { apiErrorFromResponse, apiFetch } from '@/utils/apiClient';import { useRelativeTimeNow } from '@/utils/relativeTimeClock';import {PlatformTableToolbar,@@ -264,10 +264,15 @@ export const compactReplicationNextSyncText = (text: string, tone: NextSyncTone)return text.replace(/^in /, '');};-export async function fetchReplicationJobs(): Promise<ReplicationJob[]> {- const response = await apiFetch('/api/replication/jobs?platform=proxmox-pve');+export async function fetchReplicationJobs(signal?: AbortSignal): Promise<ReplicationJob[]> {+ const response = await apiFetch('/api/replication/jobs?platform=proxmox-pve', { signal });if (!response.ok) {- throw new Error(`Failed to load replication jobs (${response.status})`);+ // A structured error keeps the HTTP status, so a 401/403 withdraws the+ // jobs as an access failure instead of being retained as an outage.+ throw await apiErrorFromResponse(+ response,+ `Failed to load replication jobs (${response.status})`,+ );}const payload = (await response.json()) as ReplicationJobsResponse;return Array.isArray(payload?.data) ? payload.data : [];diff --git a/frontend-modern/src/features/proxmox/__tests__/ProxmoxPageSurface.contract.test.tsx b/frontend-modern/src/features/proxmox/__tests__/ProxmoxPageSurface.contract.test.tsxindex 30583297ab..0979b3a487 100644--- a/frontend-modern/src/features/proxmox/__tests__/ProxmoxPageSurface.contract.test.tsx+++ b/frontend-modern/src/features/proxmox/__tests__/ProxmoxPageSurface.contract.test.tsx@@ -1,4 +1,4 @@-import { cleanup, render, screen, waitFor } from '@solidjs/testing-library';+import { cleanup, fireEvent, render, screen, waitFor } from '@solidjs/testing-library';import { Route, Router } from '@solidjs/router';import { createSignal } from 'solid-js';import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';@@ -21,7 +21,9 @@ const mockWorkloadSearch = vi.hoisted(() => vi.fn(() => ''));const mockSelectedNode = vi.hoisted(() => vi.fn<() => string | null>(() => null));const mockHandleNodeSelect = vi.hoisted(() => vi.fn());const mockWorkloadsOptions = vi.hoisted(() => vi.fn());-const mockFetchReplicationJobs = vi.hoisted(() => vi.fn(() => Promise.resolve([] as unknown[])));+const mockFetchReplicationJobs = vi.hoisted(() =>+ vi.fn((_signal?: AbortSignal) => Promise.resolve([] as unknown[])),+);const makeResource = (resource: Partial<Resource> & Pick<Resource, 'id' | 'type'>): Resource =>({@@ -153,14 +155,18 @@ vi.mock('../ProxmoxNodesTable', () => ({}));vi.mock('../ProxmoxReplicationTable', () => ({- ProxmoxReplicationTable: (props: { jobs?: unknown[]; error?: unknown }) => (+ ProxmoxReplicationTable: (props: { jobs?: unknown[]; error?: unknown; onRetry: () => void }) => (<divdata-testid="replication-table"data-jobs={props.jobs === undefined ? 'loading' : props.jobs.length}data-error={props.error ? 'yes' : 'no'}- />+ >+ <button type="button" onClick={() => props.onRetry()}>+ Retry replication+ </button>+ </div>),- fetchReplicationJobs: () => mockFetchReplicationJobs(),+ fetchReplicationJobs: (signal?: AbortSignal) => mockFetchReplicationJobs(signal),}));const renderSurface = () =>@@ -698,6 +704,141 @@ describe('ProxmoxPageSurface contract', () => {}});+ it('holds a direct replication link on a failed first read and recovers on retry', async () => {+ mockPathname.mockReturnValue('/proxmox/replication');+ const node = makeResource({ id: 'node:pve1', type: 'agent', name: 'pve1' });+ mockUseUnifiedResources.mockReturnValue({+ resources: () => [node],+ facets: () => ({ incidentCount: 0, byType: {} }),+ loading: () => false,+ error: () => null,+ refetch: vi.fn(async () => []),+ });+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockRejectedValueOnce(new Error('replication read failed'));+ mockFetchReplicationJobs.mockResolvedValueOnce([{ id: 'job-1' }]);++ try {+ renderSurface();+ const tabs = screen.getByTestId('platform-section-tabs');+ // A failed read says nothing about whether replication exists, so the+ // link keeps its error and Retry instead of falling back to Overview.+ await waitFor(() =>+ expect(screen.getByTestId('replication-table')).toHaveAttribute('data-error', 'yes'),+ );+ expect(tabs).toHaveAttribute('data-active', 'replication');+ expect(tabs).toHaveAttribute('data-tabs', 'overview');++ fireEvent.click(screen.getByRole('button', { name: 'Retry replication' }));+ await waitFor(() =>+ expect(screen.getByTestId('replication-table')).toHaveAttribute('data-jobs', '1'),+ );+ expect(screen.getByTestId('replication-table')).toHaveAttribute('data-error', 'no');+ expect(tabs).toHaveAttribute('data-tabs', 'overview,replication');+ expect(tabs).toHaveAttribute('data-active', 'replication');+ } finally {+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockImplementation(() => Promise.resolve([]));+ }+ });++ it('keeps an empty replication answer through a failed poll instead of reopening the route', async () => {+ vi.useFakeTimers({ toFake: ['setInterval', 'clearInterval'] });+ mockPathname.mockReturnValue('/proxmox/replication');+ const node = makeResource({ id: 'node:pve1', type: 'agent', name: 'pve1' });+ mockUseUnifiedResources.mockReturnValue({+ resources: () => [node],+ facets: () => ({ incidentCount: 0, byType: {} }),+ loading: () => false,+ error: () => null,+ refetch: vi.fn(async () => []),+ });+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockResolvedValueOnce([]);+ mockFetchReplicationJobs.mockRejectedValueOnce(new Error('replication read failed'));+ mockFetchReplicationJobs.mockResolvedValueOnce([]);++ try {+ renderSurface();+ const tabs = screen.getByTestId('platform-section-tabs');+ // An empty answer means no replication: the link falls back to Overview.+ await waitFor(() => expect(tabs).toHaveAttribute('data-active', 'overview'));++ vi.advanceTimersByTime(30_000);+ await waitFor(() => expect(mockFetchReplicationJobs).toHaveBeenCalledTimes(2));+ await Promise.resolve();+ expect(tabs).toHaveAttribute('data-active', 'overview');+ expect(screen.queryByTestId('replication-table')).not.toBeInTheDocument();++ vi.advanceTimersByTime(30_000);+ await waitFor(() => expect(mockFetchReplicationJobs).toHaveBeenCalledTimes(3));+ expect(tabs).toHaveAttribute('data-active', 'overview');+ } finally {+ vi.useRealTimers();+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockImplementation(() => Promise.resolve([]));+ }+ });++ it('holds a direct replication link on an access denial that withdraws loaded jobs', async () => {+ vi.useFakeTimers({ toFake: ['setInterval', 'clearInterval'] });+ mockPathname.mockReturnValue('/proxmox/replication');+ const node = makeResource({ id: 'node:pve1', type: 'agent', name: 'pve1' });+ mockUseUnifiedResources.mockReturnValue({+ resources: () => [node],+ facets: () => ({ incidentCount: 0, byType: {} }),+ loading: () => false,+ error: () => null,+ refetch: vi.fn(async () => []),+ });+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockResolvedValueOnce([{ id: 'job-1' }]);+ mockFetchReplicationJobs.mockRejectedValueOnce(+ Object.assign(new Error('Insufficient permissions'), { status: 403 }),+ );++ try {+ renderSurface();+ const tabs = screen.getByTestId('platform-section-tabs');+ await waitFor(() =>+ expect(screen.getByTestId('replication-table')).toHaveAttribute('data-jobs', '1'),+ );++ vi.advanceTimersByTime(30_000);+ await waitFor(() =>+ expect(screen.getByTestId('replication-table')).toHaveAttribute('data-error', 'yes'),+ );+ // The denied jobs are withdrawn, so the tab goes; the link shows why.+ expect(tabs).toHaveAttribute('data-tabs', 'overview');+ expect(tabs).toHaveAttribute('data-active', 'replication');+ } finally {+ vi.useRealTimers();+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockImplementation(() => Promise.resolve([]));+ }+ });++ it('cancels an in-flight replication read when the page unmounts', async () => {+ mockPathname.mockReturnValue('/proxmox/replication');+ setResources([makeResource({ id: 'node:pve1', type: 'agent', name: 'pve1' })]);+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockReturnValueOnce(new Promise<unknown[]>(() => undefined));++ try {+ const view = renderSurface();+ expect(mockFetchReplicationJobs).toHaveBeenCalledTimes(1);+ const signal = mockFetchReplicationJobs.mock.calls[0][0];+ expect(signal).toBeInstanceOf(AbortSignal);+ expect(signal?.aborted).toBe(false);++ view.unmount();+ expect(signal?.aborted).toBe(true);+ } finally {+ mockFetchReplicationJobs.mockReset();+ mockFetchReplicationJobs.mockImplementation(() => Promise.resolve([]));+ }+ });+it('does not surface stale-agent notices for development builds without an agent target', () => {mockVersionInfo.mockReturnValue({version: '6.0.0-rc.6+git.172.g2c360f779.dirty',diff --git a/frontend-modern/src/features/proxmox/__tests__/ProxmoxReplicationTable.test.tsx b/frontend-modern/src/features/proxmox/__tests__/ProxmoxReplicationTable.test.tsxindex 8e9f45cdec..938f07c21f 100644--- a/frontend-modern/src/features/proxmox/__tests__/ProxmoxReplicationTable.test.tsx+++ b/frontend-modern/src/features/proxmox/__tests__/ProxmoxReplicationTable.test.tsx@@ -7,6 +7,7 @@ import {REPLICATION_MOBILE_COLUMNS,REPLICATION_MOBILE_COLUMN_WIDTHS,compactReplicationNextSyncText,+ fetchReplicationJobs,formatMobileReplicationGuestLabel,} from '../ProxmoxReplicationTable';import replicationTableSource from '../ProxmoxReplicationTable.tsx?raw';@@ -134,6 +135,30 @@ describe('ProxmoxReplicationTable', () => {});});+describe('fetchReplicationJobs', () => {+ it('passes the caller abort signal to the network request', async () => {+ const fetchSpy = vi+ .spyOn(globalThis, 'fetch')+ .mockResolvedValue(new Response(JSON.stringify({ data: [], meta: { total: 0 } })));+ const controller = new AbortController();++ await expect(fetchReplicationJobs(controller.signal)).resolves.toEqual([]);++ const init = fetchSpy.mock.calls.find(([url]) =>+ String(url).includes('/api/replication/jobs'),+ )?.[1];+ expect(init?.signal).toBe(controller.signal);+ });++ it('keeps the HTTP status on a failed read so access denials are recognised', async () => {+ vi.spyOn(globalThis, 'fetch').mockResolvedValue(+ new Response(JSON.stringify({ message: 'Insufficient permissions' }), { status: 403 }),+ );++ await expect(fetchReplicationJobs()).rejects.toMatchObject({ status: 403 });+ });+});+describe('ProxmoxReplicationTable relative times', () => {const start = Date.parse('2026-10-04T12:00:00Z');// Last synced 5 minutes before the table opens, next sync due 10 minutes after.diff --git a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsx b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsxindex 2d6ec3ca25..ab23ae24f3 100644--- a/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsx+++ b/frontend-modern/src/features/proxmox/__tests__/proxmoxBackupsTableShared.test.tsx@@ -242,6 +242,27 @@ describe('backup age cell on the shared clock', () => {);});+ it('keeps the full age text and its band in step between clock ticks', () => {+ vi.useFakeTimers({ now: new Date('2026-10-04T12:00:00Z') });+ // Another mounted cell starts the shared clock.+ render(() => <ProxmoxBackupAgeText artifact={artifact()} />);++ // Before the next tick, a backup crosses seven days old as its row mounts.+ vi.setSystemTime(new Date('2026-10-11T12:00:10Z'));+ const createdAt = '2026-10-04T12:00:00Z';+ render(() => (+ <span data-testid="boundary">+ <ProxmoxBackupAgeText+ artifact={artifact({ createdAt, createdMs: Date.parse(createdAt) })}+ />+ </span>+ ));++ const cell = within(screen.getByTestId('boundary')).getByTitle(/backup age/);+ expect(cell).toHaveTextContent('7d ago');+ expect(cell).toHaveClass('text-amber-600');+ });+it('does not band a backup that finished after the last clock tick as unknown', () => {vi.useFakeTimers({ now: new Date('2026-10-04T12:00:00Z') });// Another mounted cell starts the shared clock at 12:00:00.diff --git a/frontend-modern/src/features/proxmox/proxmoxBackupsTableShared.tsx b/frontend-modern/src/features/proxmox/proxmoxBackupsTableShared.tsxindex 4828f3f754..099548660f 100644--- a/frontend-modern/src/features/proxmox/proxmoxBackupsTableShared.tsx+++ b/frontend-modern/src/features/proxmox/proxmoxBackupsTableShared.tsx@@ -233,11 +233,8 @@ export function formatCompactBackupAge(createdAt: string, now?: number): stringexport function ProxmoxBackupAgeText(props: { artifact: RecoverableArtifact; compact?: boolean }) {// Backup rows stay mounted while an artifact's creation time never changes,- // so the age and its freshness band both read the shared clock. The clock- // can trail the wall clock by up to one tick, and a backup that finished- // inside that window must not band as a future (unknown) age.- const tickNow = useRelativeTimeNow();- const now = () => Math.max(tickNow(), Date.now());+ // so the age and its freshness band both read the shared clock.+ const now = useRelativeTimeNow();const band = () => getRecoveryAgeBand(props.artifact.createdMs, now());const title = () => {if (band() === 'unknown') return recoveryAgeTitleByBand.unknown;diff --git a/frontend-modern/src/hooks/__tests__/createNonSuspendingQuery.test.tsx b/frontend-modern/src/hooks/__tests__/createNonSuspendingQuery.test.tsxindex ce8b4af4b4..95d0cc4c82 100644--- a/frontend-modern/src/hooks/__tests__/createNonSuspendingQuery.test.tsx+++ b/frontend-modern/src/hooks/__tests__/createNonSuspendingQuery.test.tsx@@ -2,6 +2,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest';import { cleanup, fireEvent, render, screen, waitFor } from '@solidjs/testing-library';import { createSignal } from 'solid-js';import {+ STALLED_QUERY_READ_MS,createNonSuspendingQuery,getCreateNonSuspendingQueryCacheDiagnosticsForTest,resetCreateNonSuspendingQueryCacheForTest,@@ -312,6 +313,111 @@ it.each(['success', 'failure'] as const)(},);+it('lets a background read slower than the poll interval settle', async () => {+ vi.useFakeTimers();+ // The second read takes 40s, for example while apiFetch waits out a 429+ // Retry-After, and the query polls every 30s.+ const fetcher = vi+ .fn()+ .mockResolvedValueOnce('first')+ .mockImplementationOnce(+ () => new Promise<string>((resolve) => setTimeout(() => resolve('second'), 40_000)),+ )+ .mockResolvedValue('third');+ const Probe = () => {+ const query = createNonSuspendingQuery({+ source: () => 'replication',+ fetcher,+ initialValue: '',+ pollMs: 30_000,+ });+ return <output data-testid="query">{query.value()}</output>;+ };+ render(() => <Probe />);+ await vi.advanceTimersByTimeAsync(0);+ expect(screen.getByTestId('query')).toHaveTextContent('first');++ // The 30s poll starts the slow read; the 60s poll must not discard it.+ await vi.advanceTimersByTimeAsync(60_000);+ expect(fetcher).toHaveBeenCalledTimes(2);+ expect(fetcher.mock.calls[1][1].aborted).toBe(false);++ await vi.advanceTimersByTimeAsync(10_000);+ expect(screen.getByTestId('query')).toHaveTextContent('second');++ // Polling resumes on the next tick once nothing is in flight.+ await vi.advanceTimersByTimeAsync(20_000);+ expect(fetcher).toHaveBeenCalledTimes(3);+ expect(screen.getByTestId('query')).toHaveTextContent('third');+});++it('drops a stalled read loading state even when its replacement also hangs', async () => {+ vi.useFakeTimers();+ const hang = () => new Promise<string>(() => undefined);+ const fetcher = vi.fn().mockResolvedValueOnce('first').mockImplementation(hang);+ const Probe = () => {+ const query = createNonSuspendingQuery({+ source: () => 'pbs-host',+ fetcher,+ initialValue: '',+ pollMs: 30_000,+ });+ return (+ <>+ <button onClick={() => void query.refetch()}>Refresh</button>+ <output data-testid="query">{`${query.value()}|loading:${query.loading()}`}</output>+ </>+ );+ };+ render(() => <Probe />);+ await vi.advanceTimersByTimeAsync(0);+ fireEvent.click(screen.getByRole('button', { name: 'Refresh' }));+ expect(screen.getByTestId('query')).toHaveTextContent('first|loading:true');++ // The refresh never returns, so the next poll replaces it; that read never+ // returns either, and later polls leave it alone until it has stalled.+ await vi.advanceTimersByTimeAsync(30_000);+ expect(fetcher).toHaveBeenCalledTimes(3);+ expect(fetcher.mock.calls[1][1].aborted).toBe(true);+ await vi.advanceTimersByTimeAsync(STALLED_QUERY_READ_MS - 30_000);+ expect(fetcher).toHaveBeenCalledTimes(3);+ expect(screen.getByTestId('query')).toHaveTextContent('first|loading:true');++ await vi.advanceTimersByTimeAsync(30_000);+ expect(fetcher).toHaveBeenCalledTimes(4);+ expect(fetcher.mock.calls[2][1].aborted).toBe(true);+ expect(screen.getByTestId('query')).toHaveTextContent('first|loading:false');+});++it('reports a never-loaded query as failed once its polled read stalls', async () => {+ vi.useFakeTimers();+ // Every read hangs, from the first one on.+ const fetcher = vi.fn().mockImplementation(() => new Promise<string>(() => undefined));+ const Probe = () => {+ const query = createNonSuspendingQuery({+ source: () => 'replication',+ fetcher,+ initialValue: '',+ pollMs: 30_000,+ });+ return (+ <output data-testid="query">{`resolved:${query.resolvedOnce()}|error:${+ (query.error() as Error | null)?.message ?? ''+ }|loading:${query.loading()}`}</output>+ );+ };+ render(() => <Probe />);+ await vi.advanceTimersByTimeAsync(30_000);+ expect(fetcher).toHaveBeenCalledTimes(2);+ expect(screen.getByTestId('query')).toHaveTextContent('resolved:false|error:|loading:true');++ await vi.advanceTimersByTimeAsync(STALLED_QUERY_READ_MS);+ expect(fetcher).toHaveBeenCalledTimes(3);+ expect(screen.getByTestId('query')).toHaveTextContent(+ 'resolved:true|error:The request timed out.|loading:false',+ );+});+const settle = async () => {await Promise.resolve();await Promise.resolve();diff --git a/frontend-modern/src/hooks/createNonSuspendingQuery.ts b/frontend-modern/src/hooks/createNonSuspendingQuery.tsindex 920e0de316..0ef3a1a43c 100644--- a/frontend-modern/src/hooks/createNonSuspendingQuery.ts+++ b/frontend-modern/src/hooks/createNonSuspendingQuery.ts@@ -26,6 +26,15 @@ interface QueryRunOptions {*/const RETAINED_QUERY_CACHE_MAX_ENTRIES = 64;const RETAINED_QUERY_CACHE_MAX_AGE_MS = 5 * 60_000;+// A poll does not replace a read an earlier poll started while it is still in+// flight: a read slower than the poll interval (the API answers 429 with+// Retry-After: 60, and apiFetch honours up to two minutes) would otherwise be+// discarded on every tick and never settle. Other reads stay replaceable at+// the next tick, so a slow refresh is replaced at most once. Past this age a+// polled read is presumed stalled: the next poll replaces it and reports it as+// a failed read, so a consumer that never loaded shows its error and Retry+// rather than spinning while every read hangs.+export const STALLED_QUERY_READ_MS = 150_000;type RetainedQueryCacheEntry = {cachedAt: number;@@ -179,6 +188,8 @@ export function createNonSuspendingQuery<T, K>(options: CreateNonSuspendingQuerylet latestRequestId = 0;let activeAbortController: AbortController | null = null;+ let activeRequestStartedAt = 0;+ let activeRequestIsPoll = false;const cancelActiveRequest = () => {activeAbortController?.abort();@@ -209,10 +220,12 @@ export function createNonSuspendingQuery<T, K>(options: CreateNonSuspendingQuery});onCleanup(unsubscribeLiveQueryOrgSwitch);- const run = async (key: K, runOptions: QueryRunOptions = {}): Promise<T> => {+ const run = async (key: K, runOptions: QueryRunOptions = {}, poll = false): Promise<T> => {cancelActiveRequest();const abortController = new AbortController();activeAbortController = abortController;+ activeRequestStartedAt = Date.now();+ activeRequestIsPoll = poll;const requestId = ++latestRequestId;const retainedCacheKey = getRetainedCacheKey(key);const cacheGeneration = retainedQueryCacheGeneration;@@ -296,7 +309,23 @@ export function createNonSuspendingQuery<T, K>(options: CreateNonSuspendingQueryif (typeof options.pollMs === 'number' && options.pollMs > 0) {const interval = setInterval(() => {- void refetch({ background: true });+ if (activeAbortController !== null && activeRequestIsPoll) {+ if (Date.now() - activeRequestStartedAt < STALLED_QUERY_READ_MS) return;+ // The stalled read is abandoned and settles as a failure: its+ // replacement runs in the background and may hang too, so nothing+ // else would clear a spinner or a never-loaded state.+ batch(() => {+ setLoading(false);+ setError(new Error('The request timed out.'));+ setResolvedOnce(true);+ });+ }+ const key = options.source();+ if (key === null) {+ reset();+ return;+ }+ void run(key, { background: true }, true);}, options.pollMs);onCleanup(() => clearInterval(interval));}diff --git a/frontend-modern/src/utils/__tests__/relativeTimeClock.test.ts b/frontend-modern/src/utils/__tests__/relativeTimeClock.test.tsindex 5a92bbd0a3..ae65a87f5c 100644--- a/frontend-modern/src/utils/__tests__/relativeTimeClock.test.ts+++ b/frontend-modern/src/utils/__tests__/relativeTimeClock.test.ts@@ -21,6 +21,22 @@ describe('useRelativeTimeNow', () => {expect(vi.getTimerCount()).toBe(0);});+ it('reads the wall clock between ticks, so a late reader never trails it', () => {+ vi.useFakeTimers({ now: new Date('2026-10-06T10:00:00Z') });++ const first = createRoot((dispose) => ({ now: useRelativeTimeNow(), dispose }));+ // A second cell mounts 20s after the clock started, before the next tick.+ vi.setSystemTime(new Date('2026-10-06T10:00:20Z'));+ const second = createRoot((dispose) => ({ now: useRelativeTimeNow(), dispose }));++ expect(second.now()).toBe(Date.parse('2026-10-06T10:00:20Z'));+ expect(first.now()).toBe(second.now());++ first.dispose();+ second.dispose();+ expect(vi.getTimerCount()).toBe(0);+ });+it('reads the time directly outside a reactive owner', () => {vi.useFakeTimers({ now: new Date('2026-10-06T10:00:00Z') });const now = useRelativeTimeNow();diff --git a/frontend-modern/src/utils/relativeTimeClock.ts b/frontend-modern/src/utils/relativeTimeClock.tsindex 43cd334e7e..0957a98ca1 100644--- a/frontend-modern/src/utils/relativeTimeClock.ts+++ b/frontend-modern/src/utils/relativeTimeClock.ts@@ -7,19 +7,26 @@ import { createSignal, getOwner, onCleanup } from 'solid-js';// when nothing does.export const RELATIVE_TIME_TICK_MS = 30_000;-const [now, setNow] = createSignal(Date.now());+// The tick only tells readers to re-read; every read returns the wall clock.+// A cell that mounts between ticks would otherwise measure from up to one tick+// ago, so a timestamp from the last few seconds could read as in the future+// and two cells formatted from the same time could disagree.+const [tick, setTick] = createSignal(0);+const now = (): number => {+ tick();+ return Date.now();+};let readers = 0;let timer: ReturnType<typeof setInterval> | undefined;-// Inside a component or reactive root, the returned accessor is the shared-// ticking signal. Outside one there is nothing to stop a timer, so it reads the-// time directly.+// Inside a component or reactive root, the returned accessor tracks the shared+// tick. Outside one there is nothing to stop a timer, so it reads the time+// directly.export function useRelativeTimeNow(): () => number {if (!getOwner()) return () => Date.now();readers += 1;if (timer === undefined) {- setNow(Date.now());- timer = setInterval(() => setNow(Date.now()), RELATIVE_TIME_TICK_MS);+ timer = setInterval(() => setTick((count) => count + 1), RELATIVE_TIME_TICK_MS);}onCleanup(() => {readers -= 1;