Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Say when a Proxmox guest's memory reading may include cache
rcourtman3056dccfea15
diff --git a/docs/release-control/v6/internal/subsystems/frontend-primitives.md b/docs/release-control/v6/internal/subsystems/frontend-primitives.mdindex 07168babfd..c6bb860401 100644--- a/docs/release-control/v6/internal/subsystems/frontend-primitives.md+++ b/docs/release-control/v6/internal/subsystems/frontend-primitives.md@@ -188,10 +188,21 @@ Unavailable numeric carriers remain N/A while recorded History stays intact.An independent qualified memory reading is not paused by a disk deferral.Neither Last seen, backup completion nor running power state renews memory.Unannotated unrelated platforms and actual zero readings retain their meaning.--`GuestRow.test.tsx`, `MetricMiniSparkline.test.tsx` and-`memoryObservation.test.ts` pin these boundaries, with existing drawer, adapter-and History suites. `browser-tests/guest-row-memory-provenance.cjs` checks real+A current reading from a cache-inclusive Proxmox source (`status-mem`,+`status-freemem`, `cluster-resources`, `derived-total-minus-used`, the same set+the backend marks `MayIncludeCache`) keeps its numbers, colour and thresholds;+the shared presentation adds `mayIncludeCache` and a `cacheNote` saying a high+percentage alone does not show memory pressure and which fix can supply the+guest's own figure. A VM row shows `May include cache` (`Cache?` in compact+layouts); a container, whose Proxmox reading is always the cgroup value, keeps+the caveat in the bar tooltip and drawer instead of a label on every row. A+retained or freshness-unknown reading keeps its own freshness label with the+caveat in its explanation, an unavailable reading is never qualified, and the+drawer states the note as visible text in `About this reading`.++`GuestRow.test.tsx`, `GuestDrawerOverview.memory-cache.test.tsx`,+`MetricMiniSparkline.test.tsx` and `memoryObservation.test.ts` pin these+boundaries, with existing drawer, adapter and History suites. `browser-tests/guest-row-memory-provenance.cjs` checks realproduction rows and drawers, visible/accessible labels, pointer/touch/keyboard,light/dark and narrow/desktop layouts using synthetic observations only.Presentation acceptance does not establish native thaw, covered-filesystemdiff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.mdindex b198549729..e9a7b8845d 100644--- a/docs/release-control/v6/internal/subsystems/monitoring.md+++ b/docs/release-control/v6/internal/subsystems/monitoring.md@@ -416,6 +416,18 @@ resumption through unified and served conversions; wire and source-selectioncontrols cover omission, cloning and independent live agent data. These aresynthetic source proofs, not native QGA/HAOS acceptance or delivered relief.+Mock-mode guests carry the same provenance so the UI qualifies mock readings+as it qualifies live ones: running VMs read `guest-agent-meminfo` (Linux only),+`agent` or the cache-inclusive `status-mem` with no cache split, running+containers read `cluster-resources` or `agent`, and stopped or scenario-stopped+guests are `unavailable` from `powered-off`. A current mock observation's+`observedAt` moves with each refresh, static-metrics mode included, and a+retained one keeps its own state, source and time+(`TestGeneratedGuestMemoryProvenanceMatchesPowerStateAndOS`,+`TestMockGuestMemoryObservationFollowsScenarioPowerState`,+`TestFixtureRefreshMovesCurrentGuestMemoryObservationTime`). This is fixture+realism, not a change to live source selection.+Low-trust memory stabilisation uses the original memory observation time forits existing two-minute reconnect and ten-minute healthy-guest windows, notthe diagnostic poll's `RetrievedAt`. Repeated carried-forward polls cannotdiff --git a/docs/release-control/v6/internal/subsystems/performance-and-scalability.md b/docs/release-control/v6/internal/subsystems/performance-and-scalability.mdindex c2843f6f58..67b8442041 100644--- a/docs/release-control/v6/internal/subsystems/performance-and-scalability.md+++ b/docs/release-control/v6/internal/subsystems/performance-and-scalability.md@@ -3096,6 +3096,10 @@ anomaly badge. Last-known and unknown observations keep their percentage,composition, balloon and swap geometry for inspection, in a neutral tone;their composition tooltip carries the original source/time caveat. A currentlinked-agent memory observation remains current while QEMU disk reads defer.+A current cache-inclusive Proxmox reading still reaches the row presentation,+so the VM row can say `May include cache`; rows whose reading is current and+cache-aware keep returning no presentation without formatting a date or source+label (`memoryObservation.test.ts` pins both paths).Tooltip headings, numeric values and neutral labels use theme-aware text sothe source caveat and composition remain readable on both light and dark surfaces.An unavailable reading, including the shared bar's explicit unavailable input,diff --git a/frontend-modern/browser-verification/f20e93af007a4ca7.json b/frontend-modern/browser-verification/f20e93af007a4ca7.jsonnew file mode 100644index 0000000000..f4efd847b0--- /dev/null+++ b/frontend-modern/browser-verification/f20e93af007a4ca7.json@@ -0,0 +1,42 @@+{+ "version": 1,+ "base_sha": "bbb169ca2095e3f51ac05e312650418bd48c11a6",+ "verified_at": "2026-10-10T21:50:03Z",+ "result": "passed",+ "changed_paths": [+ "frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx",+ "frontend-modern/src/components/Workloads/GuestRow.tsx",+ "frontend-modern/src/utils/memoryObservation.ts"+ ],+ "content_sha256": {+ "frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx": "91876b6db629e0f25e7cebf6bf022317b0567b44843286a0044ad6965640296a",+ "frontend-modern/src/components/Workloads/GuestRow.tsx": "b0074114ab442df633b24008fbb5eaebb7bb4a8ad3fc9e0ba0e7d139951b76fb",+ "frontend-modern/src/utils/memoryObservation.ts": "73d7c187a92141a8eb757cb9f09c091351ea78b6381a0cc87a0d9da0877dd01f"+ },+ "routes": [+ "/proxmox/overview?type=vm",+ "/proxmox/overview"+ ],+ "viewports": [+ {+ "width": 1440,+ "height": 900+ },+ {+ "width": 375,+ "height": 812+ }+ ],+ "states": [+ "private mock estate from the sidecar build (6 nodes, 78 guests): 10 of 33 VMs read from Proxmox's cache-inclusive value show 'May include cache' under the bar at 1440 and 'Cache?' at 375; VMs on the guest agent or a Pulse agent show nothing; 3 stopped VMs read Unavailable/N/A; no VM reads Freshness unknown",+ "375 phone width: every compact label sits inside its memory cell (measured text box against the cell, none clipped) and the page has no horizontal scroll; an earlier 'Incl. cache' label did clip there and was replaced",+ "VM drawer (a Windows Server VM on the cache-inclusive source): Memory reading shows 'Current \u00b7 Proxmox (may include cache)' and a stacked, full-width 'About this reading' note with the caveat and the Pulse agent / Linux QEMU guest agent advice, 4 lines at 1440",+ "container drawer on the cgroup source (billing-worker-01): same note naming only the Pulse agent, no row label on any of the 45 container rows; a container on a Pulse agent (auth-service-01) shows Current \u00b7 Pulse Agent and no note; the note was 7 lines at an 800 px pane width once stacked (about 25 before)",+ "light mode at 1024x768 was read on the previous label wording only and is not claimed for these bytes; the Alerts and Patrol pages were surveyed but not changed"+ ],+ "interactions": [+ "opened the VM and container drawers from the row chevrons",+ "resized between 1440x900, 768x1024, 1024x768 and 375x812 on earlier bytes and re-measured the labels; re-measured 1440x900 and 375x812 on these bytes",+ "restarted the private backend between builds; the console showed 500 and WebSocket errors only during those restarts, and after the final restart the socket opened and /api/resources answered 200"+ ]+}diff --git a/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx b/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsxindex cd4cd980f1..e3e887fa07 100644--- a/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx+++ b/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx@@ -81,6 +81,7 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) {ageClass: props.backupPresentation?.ageClass,});const guestReadPresentation = () => getGuestDrawerGuestReadPresentation(props.guest);+ const memoryReading = () => getGuestDrawerMemoryReading(props.guest);// A linked Pulse agent past its reporting lease leaves its last report on// the guest, so its RAID, network and filesystem rows read as last known.const agentLastKnownReason = () =>@@ -110,7 +111,13 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) {tone: guestReadPresentation()?.tone,wrap: true,}),- makeDetailRow('Memory reading', getGuestDrawerMemoryReading(props.guest)?.summary, {+ makeDetailRow('Memory reading', memoryReading()?.summary, {+ wrap: true,+ }),+ // Visible text, not a tooltip: the caveat and what to do about it+ // must reach touch and keyboard users too.+ makeDetailRow('About this reading', memoryReading()?.cacheNote, {+ layout: 'stacked',wrap: true,}),makeDetailRow('Primary IP', props.ipAddresses[0]),diff --git a/frontend-modern/src/components/Workloads/GuestRow.tsx b/frontend-modern/src/components/Workloads/GuestRow.tsxindex da8eb3c335..35bc998a71 100644--- a/frontend-modern/src/components/Workloads/GuestRow.tsx+++ b/frontend-modern/src/components/Workloads/GuestRow.tsx@@ -34,7 +34,10 @@ import { SummaryRowActionButton } from '@/components/shared/SummaryRowActionButtimport { createRowTextSelectionGuard, nativeRowClickTarget } from '@/components/shared/Table';import { DiscoveryReadinessBadge } from '@/components/shared/DiscoveryReadinessBadge';import { getWorkloadGuestDiskStatusMessage } from '@/utils/workloadGuestPresentation';-import { getWorkloadMemoryObservationPresentation } from '@/utils/memoryObservation';+import {+ getWorkloadMemoryGuestKind,+ getWorkloadMemoryObservationPresentation,+} from '@/utils/memoryObservation';import { ResourceNameWithWebInterfaceLink } from '@/components/shared/WebInterfaceLink';import type { GuestRowProps } from './guestRowModel';import { useGuestRowState } from './useGuestRowState';@@ -183,11 +186,22 @@ export function GuestRow(props: GuestRowProps) {);const memoryReadStatus = createMemo(() => {const reading = memoryReading();- if (!reading || reading.state === 'current') return undefined;+ if (!reading) return undefined;+ // A current reading needs no label, except a VM's cache-inclusive one: the+ // user can set up the guest agent for the VM's own figure. Containers have+ // no such step, so their caveat stays in the tooltip and drawer.+ const cacheCue =+ reading.state === 'current' &&+ reading.mayIncludeCache &&+ getWorkloadMemoryGuestKind(props.guest) === 'vm';+ if (reading.state === 'current' && !cacheCue) return undefined;return {message: reading.message,- label:- reading.state === 'last-known'+ label: cacheCue+ ? usesCompactTableLayout()+ ? 'Cache?'+ : 'May include cache'+ : reading.state === 'last-known'? usesCompactTableLayout()? 'Prior': 'Last known'@@ -198,8 +212,9 @@ export function GuestRow(props: GuestRowProps) {: usesCompactTableLayout()? 'Unknown': 'Freshness unknown',- valueLabelContext:- reading.state === 'last-known'+ valueLabelContext: cacheCue+ ? ('current' as const)+ : reading.state === 'last-known'? ('last known' as const): reading.state === 'unavailable'? ('unavailable' as const)diff --git a/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsx b/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsxindex 2761f3c0fa..0a07147148 100644--- a/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsx+++ b/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsx@@ -62,6 +62,7 @@ describe('StackedMemoryBar', () => {state,summary: 'Retained reading',message: 'Not a current measurement.',+ mayIncludeCache: false,});const { container } = render(() => (<StackedMemoryBardiff --git a/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.memory-cache.test.tsx b/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.memory-cache.test.tsxnew file mode 100644index 0000000000..52a972103e--- /dev/null+++ b/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.memory-cache.test.tsx@@ -0,0 +1,125 @@+import { cleanup, render, screen } from '@solidjs/testing-library';+import { createSignal, type ComponentProps } from 'solid-js';+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';+import { GuestDrawerOverview } from '../GuestDrawerOverview';+import type { WorkloadGuest } from '@/types/workloads';++vi.mock('../GuestPhysicalDisks', () => ({ GuestPhysicalDisks: () => null }));++const observedAt = '2026-10-04T13:00:00Z';++const guest = (source: string, overrides: Partial<WorkloadGuest> = {}): WorkloadGuest => ({+ id: 'lab:pve-a:101',+ vmid: 101,+ name: 'memory-guest',+ node: 'pve-a',+ instance: 'lab',+ status: 'running',+ type: 'qemu',+ cpu: 0.1,+ cpus: 2,+ memory: {+ total: 1024 ** 3,+ used: 960 * 1024 ** 2,+ free: 64 * 1024 ** 2,+ usage: 94,+ observation: { state: 'current', source, observedAt },+ },+ disk: { total: 10 * 1024 ** 3, used: 5 * 1024 ** 3, usage: 50 },+ networkIn: 0,+ networkOut: 0,+ diskRead: 0,+ diskWrite: 0,+ uptime: 3600,+ template: false,+ tags: [],+ lock: '',+ lastBackup: 0,+ lastSeen: '2026-10-04T14:00:00Z',+ ...overrides,+});++const overviewProps = (value: WorkloadGuest): ComponentProps<typeof GuestDrawerOverview> => ({+ guest: value,+ guestOsSummary: 'Linux',+ agentHeading: 'Guest agent',+ agentLabel: '',+ agentTitle: '',+ hasAgentInfo: false,+ hasFilesystemDetails: false,+ hasNetworkInterfaces: false,+ hasOsInfo: true,+ hasWorkloadActionAgent: false,+ showInGuestAgentInstallCue: false,+ ipAddresses: [],+ networkInterfaces: [],+ normalizedTags: [],+ backupPresentation: null,+ workloadActionAgentTitle: '',+});++beforeEach(() => {+ vi.spyOn(Date, 'now').mockReturnValue(Date.parse('2026-10-04T14:00:00Z'));+});+afterEach(() => {+ cleanup();+ vi.restoreAllMocks();+});++describe('guest drawer cache-inclusive memory note', () => {+ it('explains a cache-inclusive VM reading as visible text with the fix', () => {+ render(() => <GuestDrawerOverview {...overviewProps(guest('status-mem'))} />);+ expect(screen.getByText('Memory reading').closest('tr')).toHaveTextContent(+ 'Current · Proxmox (may include cache)',+ );+ const note = screen.getByText('About this reading').closest('tr')!;+ // Stacked, so the long note spans the card instead of a 7rem value column.+ expect(screen.getByText('About this reading').closest('td')).toHaveAttribute('colspan', '2');+ expect(screen.getByText('Memory reading').closest('td')).not.toHaveAttribute('colspan');+ expect(note).toHaveTextContent('a high percentage alone does not mean it is short of memory');+ expect(note).toHaveTextContent('Pulse agent in the VM');+ expect(note).toHaveTextContent('QEMU guest agent on a Linux VM');+ });++ it('follows the guest when a better source replaces the cache-inclusive one', () => {+ const [value, setValue] = createSignal(guest('status-mem'));+ render(() => <GuestDrawerOverview {...overviewProps(value())} />);+ expect(screen.getByText('About this reading')).toBeInTheDocument();+ setValue(guest('agent'));+ expect(screen.queryByText('About this reading')).not.toBeInTheDocument();+ expect(screen.getByText('Memory reading').closest('tr')).not.toHaveTextContent('cache');+ setValue(guest('cluster-resources', { type: 'lxc' }));+ expect(screen.getByText('About this reading').closest('tr')).toHaveTextContent('container');+ });++ it('keeps the caveat on a retained reading', () => {+ const retained = guest('status-mem');+ retained.memory = {+ ...retained.memory,+ observation: { state: 'last-known', source: 'status-mem', observedAt },+ };+ render(() => <GuestDrawerOverview {...overviewProps(retained)} />);+ expect(screen.getByText('Memory reading').closest('tr')).toHaveTextContent(+ 'Last known · Proxmox (may include cache)',+ );+ expect(screen.getByText('About this reading')).toBeInTheDocument();+ });++ it('gives a container the Pulse agent only', () => {+ render(() => (+ <GuestDrawerOverview {...overviewProps(guest('cluster-resources', { type: 'lxc' }))} />+ ));+ const note = screen.getByText('About this reading').closest('tr')!;+ expect(note).toHaveTextContent('Pulse agent in the container');+ expect(note).not.toHaveTextContent('QEMU');+ });++ it.each(['guest-agent-meminfo', 'agent'])(+ 'adds no note to a cache-aware %s reading',+ (source) => {+ render(() => <GuestDrawerOverview {...overviewProps(guest(source))} />);+ expect(screen.getByText('Memory reading').closest('tr')).not.toHaveTextContent('cache');+ expect(screen.queryByText('About this reading')).not.toBeInTheDocument();+ },+ );+});diff --git a/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsx b/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsxindex 9ed714bbfe..c8e181076b 100644--- a/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsx+++ b/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsx@@ -1298,6 +1298,64 @@ describe('GuestRow', () => {}});+ it(`says a current VM reading from a cache-inclusive source may include cache in ${mode}`, () => {+ const { container } = renderGuestRow({+ guest: makeGuest({+ type: 'qemu',+ memory: observation('current', observedAt, 'status-mem'),+ }),+ metricDisplayMode: mode,+ visibleColumnIds: ['name', 'memory'],+ });+ const notice = container.querySelector(noticeSelector);+ expect(notice?.querySelector('[aria-hidden]')).toHaveTextContent('May include cache');+ const message = notice?.querySelector('.sr-only');+ expect(message).toHaveTextContent('Current. Source: Proxmox. Observed: 2026-09-30');+ expect(message).toHaveTextContent(+ 'a high percentage alone does not mean it is short of memory',+ );+ expect(message).toHaveTextContent('Pulse agent in the VM');+ if (mode === 'sparklines') {+ // The value itself is a current reading; only its source is qualified.+ expect(+ screen.getByRole('img', { name: 'test-vm memory history, current 50%' }),+ ).toBeInTheDocument();+ } else {+ expect(screen.getByTestId('memory-bar')).toHaveAttribute('data-reading-state', 'current');+ }+ });++ it(`keeps a container's cache caveat out of the ${mode} row label`, () => {+ const { container } = renderGuestRow({+ guest: makeGuest({+ type: 'lxc',+ memory: observation('current', observedAt, 'cluster-resources'),+ }),+ metricDisplayMode: mode,+ visibleColumnIds: ['name', 'memory'],+ });+ // Nothing for the user to set up on a container, so no label per row.+ expect(container.querySelector(noticeSelector)).toBeNull();+ if (mode === 'bars') {+ const message = screen.getByTestId('memory-bar').getAttribute('data-reading-message');+ expect(message).toContain('cached memory the container can reuse');+ expect(message).toContain('Pulse agent in the container');+ expect(message).not.toContain('QEMU');+ }+ });++ it(`adds no cache cue to a cache-aware ${mode} reading`, () => {+ const { container } = renderGuestRow({+ guest: makeGuest({+ type: 'qemu',+ memory: observation('current', observedAt, 'guest-agent-meminfo'),+ }),+ metricDisplayMode: mode,+ visibleColumnIds: ['name', 'memory'],+ });+ expect(container.querySelector(noticeSelector)).toBeNull();+ });+it(`withdraws the ${mode} cue only for a qualified same-guest reading`, () => {const [guest, setGuest] = createSignal(makeGuest({ memory: observation('last-known') }));const { container } = render(() => (@@ -1406,6 +1464,42 @@ describe('GuestRow', () => {expect(notice).not.toHaveTextContent('future-state');});+ it('fits the compact cache cue without losing the accessible caveat', () => {+ const { container } = renderGuestRow({+ guest: makeGuest({+ type: 'qemu',+ memory: observation('current', observedAt, 'cluster-resources'),+ }),+ visibleColumnIds: ['name', 'memory'],+ workloadTableLayoutMode: 'phone',+ });+ const notice = container.querySelector(noticeSelector);+ expect(notice?.querySelector('[aria-hidden]')).toHaveTextContent(/^Cache\?$/);+ expect(notice?.querySelector('.sr-only')).toHaveTextContent('may include cached memory');+ });++ it.each([+ ['last-known', 'observed', 'Last known'],+ ['current', 'not-a-date', 'Freshness unknown'],+ ['future-state', 'observed', 'Freshness unknown'],+ ])(+ 'keeps a %s cache-inclusive reading under its freshness label, not the current cue',+ (state, time, label) => {+ const { container } = renderGuestRow({+ guest: makeGuest({+ type: 'qemu',+ memory: observation(state, time === 'observed' ? observedAt : time, 'status-mem'),+ }),+ visibleColumnIds: ['name', 'memory'],+ });+ const notice = container.querySelector(noticeSelector);+ expect(notice?.querySelector('[aria-hidden]')).toHaveTextContent(label);+ expect(notice).not.toHaveTextContent('May include cache');+ // The caveat still reaches the user, beside the freshness warning.+ expect(notice?.querySelector('.sr-only')).toHaveTextContent('may include cached memory');+ },+ );+it('does not expose unknown provider labels or borrow disk provenance', () => {const { container } = renderGuestRow({guest: makeGuest({ memory: observation('private-state', observedAt, 'private-source') }),diff --git a/frontend-modern/src/utils/__tests__/memoryObservation.test.ts b/frontend-modern/src/utils/__tests__/memoryObservation.test.tsindex 32cf942165..76d8579670 100644--- a/frontend-modern/src/utils/__tests__/memoryObservation.test.ts+++ b/frontend-modern/src/utils/__tests__/memoryObservation.test.ts@@ -1,4 +1,4 @@-import { describe, expect, it, vi, afterEach } from 'vitest';+import { describe, expect, it, vi, afterEach, beforeEach } from 'vitest';import type { WorkloadGuest } from '@/types/workloads';import {getCurrentWorkloadMemoryUsage,@@ -94,6 +94,104 @@ describe('selected workload memory observation', () => {);});+describe('cache-inclusive Proxmox memory sources', () => {+ const reading = (source: string, state = 'current', changes: Partial<WorkloadGuest> = {}) =>+ guest({+ memory: { ...guest().memory, observation: { state, source, observedAt } },+ ...changes,+ });+ beforeEach(() => {+ vi.spyOn(Date, 'now').mockReturnValue(Date.parse('2026-10-01T12:00:00Z'));+ });++ it.each(['status-mem', 'status-freemem', 'cluster-resources', 'derived-total-minus-used'])(+ 'keeps a current %s reading in the row presentation with its caveat',+ (source) => {+ const presentation = getWorkloadMemoryObservationPresentation(reading(source), {+ includeCurrent: false,+ });+ expect(presentation).toMatchObject({ state: 'current', mayIncludeCache: true });+ expect(presentation?.summary).toContain('Proxmox (may include cache)');+ expect(presentation?.message).toContain('may include cached memory');+ expect(presentation?.cacheNote).toBeTruthy();+ },+ );++ it.each([+ 'guest-agent-meminfo',+ 'guest-agent-meminfo-derived',+ 'agent',+ 'available-field',+ 'derived-free-buffers-cached',+ 'previous-snapshot',+ 'something-new',+ ])('does not flag %s as cache-inclusive', (source) => {+ const presentation = getWorkloadMemoryObservationPresentation(reading(source));+ expect(presentation?.mayIncludeCache).toBe(false);+ expect(presentation?.cacheNote).toBeUndefined();+ expect(presentation?.message).not.toContain('cached memory');+ expect(presentation?.summary).not.toContain('may include cache');+ expect(+ getWorkloadMemoryObservationPresentation(reading(source), { includeCurrent: false }),+ ).toBeNull();+ });++ it('names the fix that fits the guest: the QEMU agent is a VM-only, Linux-only option', () => {+ const vm = getWorkloadMemoryObservationPresentation(reading('status-mem'))!;+ expect(vm.cacheNote).toContain('memory the VM can reuse');+ expect(vm.cacheNote).toContain('Pulse agent in the VM');+ // The advice is conditional: source selection does not promise a replacement.+ expect(vm.cacheNote).toContain('can let Pulse show its own figure');+ expect(vm.cacheNote).not.toMatch(/\bwill\b/);+ expect(vm.cacheNote).toContain('QEMU guest agent on a Linux VM');+ const canonicalVm = getWorkloadMemoryObservationPresentation(+ reading('status-mem', 'current', { type: 'vm', workloadType: 'vm' }),+ )!;+ expect(canonicalVm.cacheNote).toContain('QEMU guest agent on a Linux VM');+ for (const changes of [+ { type: 'lxc' },+ { type: 'system-container', workloadType: 'system-container' as const },+ ]) {+ const note = getWorkloadMemoryObservationPresentation(+ reading('cluster-resources', 'current', changes),+ )!.cacheNote;+ expect(note).toContain('memory the container can reuse');+ expect(note).toContain('Pulse agent in the container');+ expect(note).not.toContain('QEMU');+ }+ const unknownKind = getWorkloadMemoryObservationPresentation(+ reading('status-mem', 'current', { type: 'docker' }),+ )!.cacheNote;+ expect(unknownKind).toContain('memory the guest can reuse');+ expect(unknownKind).not.toContain('agent');+ });++ it('keeps the caveat on a retained reading without calling it current', () => {+ const presentation = getWorkloadMemoryObservationPresentation(+ reading('status-mem', 'last-known'),+ )!;+ expect(presentation.state).toBe('last-known');+ expect(presentation.mayIncludeCache).toBe(true);+ expect(presentation.message).toContain('Not a current measurement.');+ expect(presentation.message).toContain('may include cached memory');+ });++ it.each([+ ['an unavailable observation', reading('status-mem', 'unavailable')],+ ['a reading with no usable number', reading('status-mem', 'current', { memory: undefined })],+ ])('never qualifies %s', (_label, input) => {+ const presentation = getWorkloadMemoryObservationPresentation(input);+ expect(presentation?.state).toBe('unavailable');+ expect(presentation?.mayIncludeCache).toBe(false);+ expect(presentation?.cacheNote).toBeUndefined();+ });++ it('leaves the selected number unchanged', () => {+ expect(getCurrentWorkloadMemoryUsage(reading('status-mem'))).toBe(25);+ expect(getCurrentWorkloadMemoryUsage(reading('cluster-resources'))).toBe(25);+ });+});+describe('current workload memory selection', () => {const current = (changes: Partial<WorkloadGuest> = {}) =>guest({diff --git a/frontend-modern/src/utils/memoryObservation.ts b/frontend-modern/src/utils/memoryObservation.tsindex 47ee1a273a..879b648b87 100644--- a/frontend-modern/src/utils/memoryObservation.ts+++ b/frontend-modern/src/utils/memoryObservation.ts@@ -17,8 +17,44 @@ export interface MemoryObservationPresentation {state: 'current' | 'last-known' | 'unavailable' | 'unknown';summary: string;message: string;+ // The selected Proxmox source counts cached memory the guest can reuse as+ // used, so a high percentage alone is not evidence of memory pressure. This+ // is a property of the source, independent of how fresh the reading is.+ mayIncludeCache: boolean;+ // Why a high reading may be benign, and what the user can do about it. Set+ // exactly when mayIncludeCache is.+ cacheNote?: string;}+// Proxmox's own listing and status values. Pulse selects them only when no+// cache-aware source (guest agent, linked Pulse agent, node meminfo) answered;+// the same four sources set MayIncludeCache in unifiedresources.QualifyGuestMemory.+const cacheInclusiveMemorySources = new Set([+ 'status-mem',+ 'status-freemem',+ 'cluster-resources',+ 'derived-total-minus-used',+]);++export type MemoryGuestKind = 'vm' | 'container';++const getCacheNote = (kind: MemoryGuestKind | undefined): string => {+ const noun = kind === 'vm' ? 'VM' : kind === 'container' ? 'container' : 'guest';+ const caveat = `Proxmox's reading may include cached memory the ${noun} can reuse, so a high percentage alone does not mean it is short of memory.`;+ // A Pulse agent inside the guest can supply its own figure on any OS (a+ // container's must report the container's own limit). The QEMU guest agent+ // path reads Linux /proc/meminfo only and needs installing in the guest and+ // enabling in Proxmox. Neither is guaranteed to replace every fallback+ // source, so the advice says "can". Kept short: the drawer column is narrow.+ if (kind === 'vm') {+ return `${caveat} Installing a Pulse agent in the VM, or setting up the QEMU guest agent on a Linux VM, can let Pulse show its own figure.`;+ }+ if (kind === 'container') {+ return `${caveat} Installing a Pulse agent in the container can let Pulse show its own figure.`;+ }+ return caveat;+};+const isMemoryPercent = (value: unknown): value is number =>typeof value === 'number' && Number.isFinite(value) && value >= 0 && value <= 100;@@ -101,16 +137,22 @@ export const getMemoryObservationPresentation = (value: number | undefined,observation: MemoryObservation | undefined,requiresObservation = false,- options: { includeCurrent?: boolean } = {},+ options: { includeCurrent?: boolean; guestKind?: MemoryGuestKind } = {},): MemoryObservationPresentation | null => {// Unannotated unrelated platforms retain their existing behaviour. Legacy// Proxmox readings lack authority to assert current freshness.if (!requiresObservation && !observation) return null;const timestamp = memoryObservationTime(observation);const state = memoryObservationState(value, observation, timestamp);+ // An unavailable reading shows no number, so there is nothing to qualify.+ const mayIncludeCache =+ state !== 'unavailable' &&+ observation !== undefined &&+ cacheInclusiveMemorySources.has(observation.source);// Most table rows are current and need no cue. Qualify them identically,- // but avoid building source/date strings that will never be rendered.- if (state === 'current' && options.includeCurrent === false) return null;+ // but avoid building source/date strings that will never be rendered. A+ // current reading from a cache-inclusive source still carries its caveat.+ if (state === 'current' && options.includeCurrent === false && !mayIncludeCache) return null;const observed =timestamp !== null? new Date(timestamp)@@ -132,13 +174,25 @@ export const getMemoryObservationPresentation = (? memorySourceLabels[observation.source]: 'Unknown source';const time = observed ? `Observed: ${observed}.` : 'Observation time unknown.';+ const cacheNote = mayIncludeCache ? getCacheNote(options.guestKind) : undefined;return {state,- summary: `${label} · ${source}${observed ? ` · ${observed}` : ' · time unknown'}`,- message: `${label}. Source: ${source}. ${time}${state === 'last-known' || state === 'unknown' ? ' Not a current measurement.' : ''}`,+ summary: `${label} · ${source}${mayIncludeCache ? ' (may include cache)' : ''}${observed ? ` · ${observed}` : ' · time unknown'}`,+ message: `${label}. Source: ${source}. ${time}${state === 'last-known' || state === 'unknown' ? ' Not a current measurement.' : ''}${cacheNote ? ` ${cacheNote}` : ''}`,+ mayIncludeCache,+ ...(cacheNote ? { cacheNote } : {}),};};+// A VM can gain a guest-agent figure; a container has no such agent. Legacy+// rows say qemu/lxc, canonical rows say vm/system-container.+export const getWorkloadMemoryGuestKind = (guest: WorkloadGuest): MemoryGuestKind | undefined =>+ guest.type === 'qemu' || guest.workloadType === 'vm'+ ? 'vm'+ : guest.type === 'lxc' || guest.workloadType === 'system-container'+ ? 'container'+ : undefined;+// Rows and drawers qualify the same selected reading. Neither a disk-read// deferral nor a refreshed Last seen can renew the memory observation.export const getWorkloadMemoryObservationPresentation = (@@ -154,6 +208,9 @@ export const getWorkloadMemoryObservationPresentation = (value,guest.memory?.observation,requiresWorkloadMemoryObservation(guest),- options,+ {+ ...options,+ guestKind: getWorkloadMemoryGuestKind(guest),+ },);};diff --git a/internal/mock/demo_scenarios.go b/internal/mock/demo_scenarios.goindex 5918053cfc..83f761d369 100644--- a/internal/mock/demo_scenarios.go+++ b/internal/mock/demo_scenarios.go@@ -527,6 +527,7 @@ func applyDemoWorkloadScenario(workloads []models.VM, profiles []demoWorkloadProworkloads[i].IPAddresses = nilworkloads[i].NetworkInterfaces = nil}+ mockSettleGuestMemoryObservation(&workloads[i].Memory, workloads[i].Status != "stopped", "agent")workloads[i].LastBackup = demoProfileLastBackup(profile, now)guestProfiles[workloads[i].VMID] = profile}@@ -567,6 +568,7 @@ func applyDemoContainerScenario(workloads []models.Container, profiles []demoWorworkloads[i].IPAddresses = nilworkloads[i].NetworkInterfaces = nil}+ mockSettleGuestMemoryObservation(&workloads[i].Memory, workloads[i].Status != "stopped", "cluster-resources")workloads[i].LastBackup = demoProfileLastBackup(profile, now)guestProfiles[workloads[i].VMID] = profile}diff --git a/internal/mock/generator.go b/internal/mock/generator.goindex 0e641a5cc1..3ac72f1c68 100644--- a/internal/mock/generator.go+++ b/internal/mock/generator.go@@ -548,6 +548,7 @@ func buildFixtureState(config MockConfig) models.StateSnapshot {vm.Memory.Usage = 0vm.Memory.Free = vm.Memory.Totalvm.Memory.SwapUsed = 0+ mockSettleGuestMemoryObservation(&vm.Memory, false, "")vm.Disk.Used = 0vm.Disk.Free = vm.Disk.Totalvm.Disk.Usage = -1@@ -571,6 +572,7 @@ func buildFixtureState(config MockConfig) models.StateSnapshot {lxc.Memory.Usage = 0lxc.Memory.Free = lxc.Memory.Totallxc.Memory.SwapUsed = 0+ mockSettleGuestMemoryObservation(&lxc.Memory, false, "")lxc.Disk.Used = 0lxc.Disk.Free = lxc.Disk.Totallxc.Disk.Usage = -1@@ -1275,6 +1277,52 @@ func generateRealisticIO(ioType string) int64 {return 0}+// mockGuestMemoryObservation gives a running guest the provenance a real+// estate reports for its memory reading, so the UI qualifies mock readings the+// way it qualifies live ones instead of treating every guest as unannotated.+func mockGuestMemoryObservation(source string) models.MemoryObservation {+ return models.MemoryObservation{State: "current", Source: source, ObservedAt: time.Now()}+}++// applyMockVMMemoryProvenance picks the source a running VM's reading comes+// from. A quarter of VMs have nothing better than Proxmox's cache-inclusive+// value, which carries no used | cache | free split. Pulse reads guest memory+// through the QEMU guest agent from Linux /proc/meminfo only, so a Windows VM+// is cache-aware only through a Pulse agent inside it.+func applyMockVMMemoryProvenance(mem *models.Memory, osName string) {+ switch {+ case rand.Float64() < 0.25:+ mem.Free += mem.Cache+ mem.Cache = 0+ mem.Observation = mockGuestMemoryObservation("status-mem")+ case strings.Contains(strings.ToLower(osName), "windows"):+ mem.Observation = mockGuestMemoryObservation("agent")+ default:+ mem.Observation = mockGuestMemoryObservation("guest-agent-meminfo")+ }+}++// refreshMockMemoryObservation moves a current reading's observation time with+// the sample just taken; a retained or unavailable observation keeps its own.+func refreshMockMemoryObservation(mem *models.Memory, at time.Time) {+ if mem.Observation.State == "current" {+ mem.Observation.ObservedAt = at+ }+}++// mockSettleGuestMemoryObservation keeps a guest's memory provenance in step+// with a power state a scenario forces after generation: a stopped guest has+// no reading to qualify, and a guest started later needs a current source. A+// retained or current observation on a running guest is left as it is.+func mockSettleGuestMemoryObservation(memory *models.Memory, running bool, runningSource string) {+ switch {+ case !running:+ memory.Observation = models.MemoryObservation{State: "unavailable", Source: "powered-off"}+ case memory.Observation.State == "" || memory.Observation.State == "unavailable":+ memory.Observation = mockGuestMemoryObservation(runningSource)+ }+}+func generateVM(nodeName string, instance string, vmid int, config MockConfig) models.VM {name := generateGuestName("vm", fmt.Sprintf("%s:%s:%d", instance, nodeName, vmid))status := "running"@@ -1345,6 +1393,8 @@ func generateVM(nodeName string, instance string, vmid int, config MockConfig) mmem.Free -= mem.Cache}uptime = int64(3600 * (1 + rand.Intn(720))) // 1-720 hours+ } else {+ mem.Observation = models.MemoryObservation{State: "unavailable", Source: "powered-off"}}// Disk stats@@ -1384,6 +1434,9 @@ func generateVM(nodeName string, instance string, vmid int, config MockConfig) mosName, osVersion := generateGuestOSMetadata()ipAddresses, networkIfaces := generateGuestNetworkInfo()+ if status == "running" {+ applyMockVMMemoryProvenance(&mem, osName)+ }vm := models.VM{Name: name,@@ -4922,7 +4975,16 @@ func generateContainer(nodeName string, instance string, vmid int, config MockCoSwapTotal: swapTotal,SwapUsed: swapUsed,}+ // Proxmox reports container memory from the cgroup, cache included;+ // only a linked Pulse agent supplies a cache-aware figure.+ if rand.Float64() < 0.15 {+ mem.Observation = mockGuestMemoryObservation("agent")+ } else {+ mem.Observation = mockGuestMemoryObservation("cluster-resources")+ }uptime = int64(3600 * (1 + rand.Intn(1440))) // 1-1440 hours (up to 60 days)+ } else {+ mem.Observation = models.MemoryObservation{State: "unavailable", Source: "powered-off"}}// Disk stats - containers typically smaller@@ -6365,11 +6427,24 @@ func updateFixtureStateMetricsSelectedAt(}*lastSeen = refreshNow}+ // A current memory reading is observed on the same poll that sights the+ // guest, so its time moves with the sighting, in static-metrics mode too.+ stampGuestMemoryObservation := func(nodeName, status string, memory *models.Memory) {+ if !selection.includesProxmoxNode(nodeName) || status != "running" {+ return+ }+ if _, offline := offlineNodes[nodeName]; offline {+ return+ }+ refreshMockMemoryObservation(memory, refreshNow)+ }for i := range data.VMs {stampGuestSighting(data.VMs[i].Node, &data.VMs[i].LastSeen)+ stampGuestMemoryObservation(data.VMs[i].Node, data.VMs[i].Status, &data.VMs[i].Memory)}for i := range data.Containers {stampGuestSighting(data.Containers[i].Node, &data.Containers[i].LastSeen)+ stampGuestMemoryObservation(data.Containers[i].Node, data.Containers[i].Status, &data.Containers[i].Memory)}for i := range data.Storage {stampGuestSighting(data.Storage[i].Node, &data.Storage[i].LastSeen)diff --git a/internal/mock/generator_test.go b/internal/mock/generator_test.goindex 0c76b8346d..d55ac49a15 100644--- a/internal/mock/generator_test.go+++ b/internal/mock/generator_test.go@@ -1460,3 +1460,163 @@ func TestBuildFixtureStateMergesSharedPBSStorageLikeThePoller(t *testing.T) {t.Fatal("expected shared PBS storage in the fixture")}}++// Mock guests report where their memory reading came from, the way a live+// estate does, so the UI qualifies mock readings instead of treating every+// guest as unannotated ("Freshness unknown").+func TestGeneratedGuestMemoryProvenanceMatchesPowerStateAndOS(t *testing.T) {+ cfg := DefaultConfig+ cfg.StoppedPercent = 0.3+ now := time.Now()++ sources := map[string]int{}+ for i := 0; i < 300; i++ {+ vm := generateVM("node-01", "mock-cluster", 1000+i, cfg)+ obs := vm.Memory.Observation+ if vm.Status != "running" {+ if obs.State != "unavailable" || obs.Source != "powered-off" {+ t.Fatalf("stopped VM observation = %+v, want unavailable powered-off", obs)+ }+ continue+ }+ sources[obs.Source]+++ if obs.State != "current" || obs.ObservedAt.IsZero() || obs.ObservedAt.After(now.Add(time.Minute)) {+ t.Fatalf("running VM observation = %+v, want a current reading observed now", obs)+ }+ switch obs.Source {+ case "status-mem":+ // Proxmox's value carries no used | cache | free split.+ if vm.Memory.Cache != 0 {+ t.Fatalf("cache-inclusive VM reported a measured cache split: %+v", vm.Memory)+ }+ case "guest-agent-meminfo":+ // The QEMU guest agent reads Linux /proc/meminfo only.+ if strings.Contains(strings.ToLower(vm.OSName), "windows") {+ t.Fatalf("Windows VM %q claims Linux meminfo provenance", vm.OSName)+ }+ case "agent":+ default:+ t.Fatalf("running VM has unexpected memory source %q", obs.Source)+ }+ }+ if sources["status-mem"] == 0 || sources["guest-agent-meminfo"] == 0 || sources["agent"] == 0 {+ t.Fatalf("expected cache-inclusive, guest-agent and Pulse-agent VMs in the estate, got %v", sources)+ }++ sources = map[string]int{}+ for i := 0; i < 300; i++ {+ ct := generateContainer("node-01", "mock-cluster", 2000+i, cfg)+ obs := ct.Memory.Observation+ if ct.Status != "running" {+ if obs.State != "unavailable" || obs.Source != "powered-off" {+ t.Fatalf("stopped container observation = %+v, want unavailable powered-off", obs)+ }+ continue+ }+ sources[obs.Source]+++ if obs.State != "current" || obs.ObservedAt.IsZero() {+ t.Fatalf("running container observation = %+v, want a current reading", obs)+ }+ if obs.Source != "cluster-resources" && obs.Source != "agent" {+ t.Fatalf("running container has unexpected memory source %q", obs.Source)+ }+ }+ if sources["cluster-resources"] == 0 || sources["agent"] == 0 {+ t.Fatalf("expected cgroup and Pulse-agent containers in the estate, got %v", sources)+ }+}++func TestMockGuestMemoryObservationFollowsScenarioPowerState(t *testing.T) {+ cases := []struct {+ name string+ before models.MemoryObservation+ running bool+ want models.MemoryObservation+ }{+ {+ name: "forced stopped loses its reading",+ before: models.MemoryObservation{State: "current", Source: "status-mem", ObservedAt: time.Now()},+ running: false,+ want: models.MemoryObservation{State: "unavailable", Source: "powered-off"},+ },+ {+ name: "forced running after generation gets a current source",+ before: models.MemoryObservation{State: "unavailable", Source: "powered-off"},+ running: true,+ want: models.MemoryObservation{State: "current", Source: "agent"},+ },+ {+ name: "a running guest keeps its own source",+ before: models.MemoryObservation{State: "current", Source: "status-mem", ObservedAt: time.Now()},+ running: true,+ want: models.MemoryObservation{State: "current", Source: "status-mem"},+ },+ {+ name: "a retained reading on a running guest is kept",+ before: models.MemoryObservation{State: "last-known", Source: "status-mem", ObservedAt: time.Now().Add(-time.Hour)},+ running: true,+ want: models.MemoryObservation{State: "last-known", Source: "status-mem"},+ },+ }+ for _, tc := range cases {+ t.Run(tc.name, func(t *testing.T) {+ mem := models.Memory{Observation: tc.before}+ mockSettleGuestMemoryObservation(&mem, tc.running, "agent")+ got := mem.Observation+ if got.State != tc.want.State || got.Source != tc.want.Source {+ t.Fatalf("observation = %+v, want state %q source %q", got, tc.want.State, tc.want.Source)+ }+ if tc.want.State == "current" && got.ObservedAt.IsZero() {+ t.Fatalf("a current observation needs its observation time: %+v", got)+ }+ })+ }+}++func TestFixtureRefreshMovesCurrentGuestMemoryObservationTime(t *testing.T) {+ for _, randomMetrics := range []bool{true, false} {+ name := "random metrics"+ if !randomMetrics {+ name = "static metrics"+ }+ t.Run(name, func(t *testing.T) {+ cfg := DefaultConfig+ cfg.RandomMetrics = randomMetrics+ stale := time.Now().Add(-time.Hour)+ refreshAt := time.Now().Truncate(time.Second)++ data := &models.StateSnapshot{+ VMs: []models.VM{+ {ID: "vm-current", Node: "node-01", Status: "running", Memory: models.Memory{+ Total: 100, Observation: models.MemoryObservation{State: "current", Source: "status-mem", ObservedAt: stale},+ }},+ {ID: "vm-retained", Node: "node-01", Status: "running", Memory: models.Memory{+ Total: 100, Observation: models.MemoryObservation{State: "last-known", Source: "status-mem", ObservedAt: stale},+ }},+ {ID: "vm-stopped", Node: "node-01", Status: "stopped", Memory: models.Memory{+ Total: 100, Observation: models.MemoryObservation{State: "unavailable", Source: "powered-off"},+ }},+ },+ Containers: []models.Container{+ {ID: "ct-current", Node: "node-01", Status: "running", Memory: models.Memory{+ Total: 100, Observation: models.MemoryObservation{State: "current", Source: "cluster-resources", ObservedAt: stale},+ }},+ },+ }+ updateFixtureStateMetricsAt(data, cfg, refreshAt)++ if got := data.VMs[0].Memory.Observation; !got.ObservedAt.Equal(refreshAt) || got.Source != "status-mem" {+ t.Fatalf("current VM observation = %+v, want source kept and time %v", got, refreshAt)+ }+ if got := data.VMs[1].Memory.Observation; !got.ObservedAt.Equal(stale) || got.State != "last-known" {+ t.Fatalf("a retained observation keeps its own state and time, got %+v", got)+ }+ if got := data.VMs[2].Memory.Observation; !got.ObservedAt.IsZero() || got.Source != "powered-off" {+ t.Fatalf("a stopped guest has no observation time to move, got %+v", got)+ }+ if got := data.Containers[0].Memory.Observation; !got.ObservedAt.Equal(refreshAt) || got.Source != "cluster-resources" {+ t.Fatalf("current container observation = %+v, want source kept and time %v", got, refreshAt)+ }+ })+ }+}