Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Say when a Proxmox guest's memory reading may include cache

rcourtman3056dccfea15
diff --git a/docs/release-control/v6/internal/subsystems/frontend-primitives.md b/docs/release-control/v6/internal/subsystems/frontend-primitives.md
index 07168babfd..c6bb860401 100644
--- a/docs/release-control/v6/internal/subsystems/frontend-primitives.md
+++ b/docs/release-control/v6/internal/subsystems/frontend-primitives.md
@@ -188,10 +188,21 @@ Unavailable numeric carriers remain N/A while recorded History stays intact.
An independent qualified memory reading is not paused by a disk deferral.
Neither Last seen, backup completion nor running power state renews memory.
Unannotated unrelated platforms and actual zero readings retain their meaning.
-
-`GuestRow.test.tsx`, `MetricMiniSparkline.test.tsx` and
-`memoryObservation.test.ts` pin these boundaries, with existing drawer, adapter
-and History suites. `browser-tests/guest-row-memory-provenance.cjs` checks real
+A current reading from a cache-inclusive Proxmox source (`status-mem`,
+`status-freemem`, `cluster-resources`, `derived-total-minus-used`, the same set
+the backend marks `MayIncludeCache`) keeps its numbers, colour and thresholds;
+the shared presentation adds `mayIncludeCache` and a `cacheNote` saying a high
+percentage alone does not show memory pressure and which fix can supply the
+guest's own figure. A VM row shows `May include cache` (`Cache?` in compact
+layouts); a container, whose Proxmox reading is always the cgroup value, keeps
+the caveat in the bar tooltip and drawer instead of a label on every row. A
+retained or freshness-unknown reading keeps its own freshness label with the
+caveat in its explanation, an unavailable reading is never qualified, and the
+drawer states the note as visible text in `About this reading`.
+
+`GuestRow.test.tsx`, `GuestDrawerOverview.memory-cache.test.tsx`,
+`MetricMiniSparkline.test.tsx` and `memoryObservation.test.ts` pin these
+boundaries, with existing drawer, adapter and History suites. `browser-tests/guest-row-memory-provenance.cjs` checks real
production rows and drawers, visible/accessible labels, pointer/touch/keyboard,
light/dark and narrow/desktop layouts using synthetic observations only.
Presentation acceptance does not establish native thaw, covered-filesystem
diff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.md
index b198549729..e9a7b8845d 100644
--- a/docs/release-control/v6/internal/subsystems/monitoring.md
+++ b/docs/release-control/v6/internal/subsystems/monitoring.md
@@ -416,6 +416,18 @@ resumption through unified and served conversions; wire and source-selection
controls cover omission, cloning and independent live agent data. These are
synthetic source proofs, not native QGA/HAOS acceptance or delivered relief.
+Mock-mode guests carry the same provenance so the UI qualifies mock readings
+as it qualifies live ones: running VMs read `guest-agent-meminfo` (Linux only),
+`agent` or the cache-inclusive `status-mem` with no cache split, running
+containers read `cluster-resources` or `agent`, and stopped or scenario-stopped
+guests are `unavailable` from `powered-off`. A current mock observation's
+`observedAt` moves with each refresh, static-metrics mode included, and a
+retained one keeps its own state, source and time
+(`TestGeneratedGuestMemoryProvenanceMatchesPowerStateAndOS`,
+`TestMockGuestMemoryObservationFollowsScenarioPowerState`,
+`TestFixtureRefreshMovesCurrentGuestMemoryObservationTime`). This is fixture
+realism, not a change to live source selection.
+
Low-trust memory stabilisation uses the original memory observation time for
its existing two-minute reconnect and ten-minute healthy-guest windows, not
the diagnostic poll's `RetrievedAt`. Repeated carried-forward polls cannot
diff --git a/docs/release-control/v6/internal/subsystems/performance-and-scalability.md b/docs/release-control/v6/internal/subsystems/performance-and-scalability.md
index c2843f6f58..67b8442041 100644
--- a/docs/release-control/v6/internal/subsystems/performance-and-scalability.md
+++ b/docs/release-control/v6/internal/subsystems/performance-and-scalability.md
@@ -3096,6 +3096,10 @@ anomaly badge. Last-known and unknown observations keep their percentage,
composition, balloon and swap geometry for inspection, in a neutral tone;
their composition tooltip carries the original source/time caveat. A current
linked-agent memory observation remains current while QEMU disk reads defer.
+A current cache-inclusive Proxmox reading still reaches the row presentation,
+so the VM row can say `May include cache`; rows whose reading is current and
+cache-aware keep returning no presentation without formatting a date or source
+label (`memoryObservation.test.ts` pins both paths).
Tooltip headings, numeric values and neutral labels use theme-aware text so
the source caveat and composition remain readable on both light and dark surfaces.
An unavailable reading, including the shared bar's explicit unavailable input,
diff --git a/frontend-modern/browser-verification/f20e93af007a4ca7.json b/frontend-modern/browser-verification/f20e93af007a4ca7.json
new file mode 100644
index 0000000000..f4efd847b0
--- /dev/null
+++ b/frontend-modern/browser-verification/f20e93af007a4ca7.json
@@ -0,0 +1,42 @@
+{
+ "version": 1,
+ "base_sha": "bbb169ca2095e3f51ac05e312650418bd48c11a6",
+ "verified_at": "2026-10-10T21:50:03Z",
+ "result": "passed",
+ "changed_paths": [
+ "frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx",
+ "frontend-modern/src/components/Workloads/GuestRow.tsx",
+ "frontend-modern/src/utils/memoryObservation.ts"
+ ],
+ "content_sha256": {
+ "frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx": "91876b6db629e0f25e7cebf6bf022317b0567b44843286a0044ad6965640296a",
+ "frontend-modern/src/components/Workloads/GuestRow.tsx": "b0074114ab442df633b24008fbb5eaebb7bb4a8ad3fc9e0ba0e7d139951b76fb",
+ "frontend-modern/src/utils/memoryObservation.ts": "73d7c187a92141a8eb757cb9f09c091351ea78b6381a0cc87a0d9da0877dd01f"
+ },
+ "routes": [
+ "/proxmox/overview?type=vm",
+ "/proxmox/overview"
+ ],
+ "viewports": [
+ {
+ "width": 1440,
+ "height": 900
+ },
+ {
+ "width": 375,
+ "height": 812
+ }
+ ],
+ "states": [
+ "private mock estate from the sidecar build (6 nodes, 78 guests): 10 of 33 VMs read from Proxmox's cache-inclusive value show 'May include cache' under the bar at 1440 and 'Cache?' at 375; VMs on the guest agent or a Pulse agent show nothing; 3 stopped VMs read Unavailable/N/A; no VM reads Freshness unknown",
+ "375 phone width: every compact label sits inside its memory cell (measured text box against the cell, none clipped) and the page has no horizontal scroll; an earlier 'Incl. cache' label did clip there and was replaced",
+ "VM drawer (a Windows Server VM on the cache-inclusive source): Memory reading shows 'Current \u00b7 Proxmox (may include cache)' and a stacked, full-width 'About this reading' note with the caveat and the Pulse agent / Linux QEMU guest agent advice, 4 lines at 1440",
+ "container drawer on the cgroup source (billing-worker-01): same note naming only the Pulse agent, no row label on any of the 45 container rows; a container on a Pulse agent (auth-service-01) shows Current \u00b7 Pulse Agent and no note; the note was 7 lines at an 800 px pane width once stacked (about 25 before)",
+ "light mode at 1024x768 was read on the previous label wording only and is not claimed for these bytes; the Alerts and Patrol pages were surveyed but not changed"
+ ],
+ "interactions": [
+ "opened the VM and container drawers from the row chevrons",
+ "resized between 1440x900, 768x1024, 1024x768 and 375x812 on earlier bytes and re-measured the labels; re-measured 1440x900 and 375x812 on these bytes",
+ "restarted the private backend between builds; the console showed 500 and WebSocket errors only during those restarts, and after the final restart the socket opened and /api/resources answered 200"
+ ]
+}
diff --git a/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx b/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx
index cd4cd980f1..e3e887fa07 100644
--- a/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx
+++ b/frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx
@@ -81,6 +81,7 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) {
ageClass: props.backupPresentation?.ageClass,
});
const guestReadPresentation = () => getGuestDrawerGuestReadPresentation(props.guest);
+ const memoryReading = () => getGuestDrawerMemoryReading(props.guest);
// A linked Pulse agent past its reporting lease leaves its last report on
// the guest, so its RAID, network and filesystem rows read as last known.
const agentLastKnownReason = () =>
@@ -110,7 +111,13 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) {
tone: guestReadPresentation()?.tone,
wrap: true,
}),
- makeDetailRow('Memory reading', getGuestDrawerMemoryReading(props.guest)?.summary, {
+ makeDetailRow('Memory reading', memoryReading()?.summary, {
+ wrap: true,
+ }),
+ // Visible text, not a tooltip: the caveat and what to do about it
+ // must reach touch and keyboard users too.
+ makeDetailRow('About this reading', memoryReading()?.cacheNote, {
+ layout: 'stacked',
wrap: true,
}),
makeDetailRow('Primary IP', props.ipAddresses[0]),
diff --git a/frontend-modern/src/components/Workloads/GuestRow.tsx b/frontend-modern/src/components/Workloads/GuestRow.tsx
index da8eb3c335..35bc998a71 100644
--- a/frontend-modern/src/components/Workloads/GuestRow.tsx
+++ b/frontend-modern/src/components/Workloads/GuestRow.tsx
@@ -34,7 +34,10 @@ import { SummaryRowActionButton } from '@/components/shared/SummaryRowActionButt
import { createRowTextSelectionGuard, nativeRowClickTarget } from '@/components/shared/Table';
import { DiscoveryReadinessBadge } from '@/components/shared/DiscoveryReadinessBadge';
import { getWorkloadGuestDiskStatusMessage } from '@/utils/workloadGuestPresentation';
-import { getWorkloadMemoryObservationPresentation } from '@/utils/memoryObservation';
+import {
+ getWorkloadMemoryGuestKind,
+ getWorkloadMemoryObservationPresentation,
+} from '@/utils/memoryObservation';
import { ResourceNameWithWebInterfaceLink } from '@/components/shared/WebInterfaceLink';
import type { GuestRowProps } from './guestRowModel';
import { useGuestRowState } from './useGuestRowState';
@@ -183,11 +186,22 @@ export function GuestRow(props: GuestRowProps) {
);
const memoryReadStatus = createMemo(() => {
const reading = memoryReading();
- if (!reading || reading.state === 'current') return undefined;
+ if (!reading) return undefined;
+ // A current reading needs no label, except a VM's cache-inclusive one: the
+ // user can set up the guest agent for the VM's own figure. Containers have
+ // no such step, so their caveat stays in the tooltip and drawer.
+ const cacheCue =
+ reading.state === 'current' &&
+ reading.mayIncludeCache &&
+ getWorkloadMemoryGuestKind(props.guest) === 'vm';
+ if (reading.state === 'current' && !cacheCue) return undefined;
return {
message: reading.message,
- label:
- reading.state === 'last-known'
+ label: cacheCue
+ ? usesCompactTableLayout()
+ ? 'Cache?'
+ : 'May include cache'
+ : reading.state === 'last-known'
? usesCompactTableLayout()
? 'Prior'
: 'Last known'
@@ -198,8 +212,9 @@ export function GuestRow(props: GuestRowProps) {
: usesCompactTableLayout()
? 'Unknown'
: 'Freshness unknown',
- valueLabelContext:
- reading.state === 'last-known'
+ valueLabelContext: cacheCue
+ ? ('current' as const)
+ : reading.state === 'last-known'
? ('last known' as const)
: reading.state === 'unavailable'
? ('unavailable' as const)
diff --git a/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsx b/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsx
index 2761f3c0fa..0a07147148 100644
--- a/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsx
+++ b/frontend-modern/src/components/Workloads/StackedMemoryBar.test.tsx
@@ -62,6 +62,7 @@ describe('StackedMemoryBar', () => {
state,
summary: 'Retained reading',
message: 'Not a current measurement.',
+ mayIncludeCache: false,
});
const { container } = render(() => (
<StackedMemoryBar
diff --git a/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.memory-cache.test.tsx b/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.memory-cache.test.tsx
new file mode 100644
index 0000000000..52a972103e
--- /dev/null
+++ b/frontend-modern/src/components/Workloads/__tests__/GuestDrawerOverview.memory-cache.test.tsx
@@ -0,0 +1,125 @@
+import { cleanup, render, screen } from '@solidjs/testing-library';
+import { createSignal, type ComponentProps } from 'solid-js';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { GuestDrawerOverview } from '../GuestDrawerOverview';
+import type { WorkloadGuest } from '@/types/workloads';
+
+vi.mock('../GuestPhysicalDisks', () => ({ GuestPhysicalDisks: () => null }));
+
+const observedAt = '2026-10-04T13:00:00Z';
+
+const guest = (source: string, overrides: Partial<WorkloadGuest> = {}): WorkloadGuest => ({
+ id: 'lab:pve-a:101',
+ vmid: 101,
+ name: 'memory-guest',
+ node: 'pve-a',
+ instance: 'lab',
+ status: 'running',
+ type: 'qemu',
+ cpu: 0.1,
+ cpus: 2,
+ memory: {
+ total: 1024 ** 3,
+ used: 960 * 1024 ** 2,
+ free: 64 * 1024 ** 2,
+ usage: 94,
+ observation: { state: 'current', source, observedAt },
+ },
+ disk: { total: 10 * 1024 ** 3, used: 5 * 1024 ** 3, usage: 50 },
+ networkIn: 0,
+ networkOut: 0,
+ diskRead: 0,
+ diskWrite: 0,
+ uptime: 3600,
+ template: false,
+ tags: [],
+ lock: '',
+ lastBackup: 0,
+ lastSeen: '2026-10-04T14:00:00Z',
+ ...overrides,
+});
+
+const overviewProps = (value: WorkloadGuest): ComponentProps<typeof GuestDrawerOverview> => ({
+ guest: value,
+ guestOsSummary: 'Linux',
+ agentHeading: 'Guest agent',
+ agentLabel: '',
+ agentTitle: '',
+ hasAgentInfo: false,
+ hasFilesystemDetails: false,
+ hasNetworkInterfaces: false,
+ hasOsInfo: true,
+ hasWorkloadActionAgent: false,
+ showInGuestAgentInstallCue: false,
+ ipAddresses: [],
+ networkInterfaces: [],
+ normalizedTags: [],
+ backupPresentation: null,
+ workloadActionAgentTitle: '',
+});
+
+beforeEach(() => {
+ vi.spyOn(Date, 'now').mockReturnValue(Date.parse('2026-10-04T14:00:00Z'));
+});
+afterEach(() => {
+ cleanup();
+ vi.restoreAllMocks();
+});
+
+describe('guest drawer cache-inclusive memory note', () => {
+ it('explains a cache-inclusive VM reading as visible text with the fix', () => {
+ render(() => <GuestDrawerOverview {...overviewProps(guest('status-mem'))} />);
+ expect(screen.getByText('Memory reading').closest('tr')).toHaveTextContent(
+ 'Current · Proxmox (may include cache)',
+ );
+ const note = screen.getByText('About this reading').closest('tr')!;
+ // Stacked, so the long note spans the card instead of a 7rem value column.
+ expect(screen.getByText('About this reading').closest('td')).toHaveAttribute('colspan', '2');
+ expect(screen.getByText('Memory reading').closest('td')).not.toHaveAttribute('colspan');
+ expect(note).toHaveTextContent('a high percentage alone does not mean it is short of memory');
+ expect(note).toHaveTextContent('Pulse agent in the VM');
+ expect(note).toHaveTextContent('QEMU guest agent on a Linux VM');
+ });
+
+ it('follows the guest when a better source replaces the cache-inclusive one', () => {
+ const [value, setValue] = createSignal(guest('status-mem'));
+ render(() => <GuestDrawerOverview {...overviewProps(value())} />);
+ expect(screen.getByText('About this reading')).toBeInTheDocument();
+ setValue(guest('agent'));
+ expect(screen.queryByText('About this reading')).not.toBeInTheDocument();
+ expect(screen.getByText('Memory reading').closest('tr')).not.toHaveTextContent('cache');
+ setValue(guest('cluster-resources', { type: 'lxc' }));
+ expect(screen.getByText('About this reading').closest('tr')).toHaveTextContent('container');
+ });
+
+ it('keeps the caveat on a retained reading', () => {
+ const retained = guest('status-mem');
+ retained.memory = {
+ ...retained.memory,
+ observation: { state: 'last-known', source: 'status-mem', observedAt },
+ };
+ render(() => <GuestDrawerOverview {...overviewProps(retained)} />);
+ expect(screen.getByText('Memory reading').closest('tr')).toHaveTextContent(
+ 'Last known · Proxmox (may include cache)',
+ );
+ expect(screen.getByText('About this reading')).toBeInTheDocument();
+ });
+
+ it('gives a container the Pulse agent only', () => {
+ render(() => (
+ <GuestDrawerOverview {...overviewProps(guest('cluster-resources', { type: 'lxc' }))} />
+ ));
+ const note = screen.getByText('About this reading').closest('tr')!;
+ expect(note).toHaveTextContent('Pulse agent in the container');
+ expect(note).not.toHaveTextContent('QEMU');
+ });
+
+ it.each(['guest-agent-meminfo', 'agent'])(
+ 'adds no note to a cache-aware %s reading',
+ (source) => {
+ render(() => <GuestDrawerOverview {...overviewProps(guest(source))} />);
+ expect(screen.getByText('Memory reading').closest('tr')).not.toHaveTextContent('cache');
+ expect(screen.queryByText('About this reading')).not.toBeInTheDocument();
+ },
+ );
+});
diff --git a/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsx b/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsx
index 9ed714bbfe..c8e181076b 100644
--- a/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsx
+++ b/frontend-modern/src/components/Workloads/__tests__/GuestRow.test.tsx
@@ -1298,6 +1298,64 @@ describe('GuestRow', () => {
}
});
+ it(`says a current VM reading from a cache-inclusive source may include cache in ${mode}`, () => {
+ const { container } = renderGuestRow({
+ guest: makeGuest({
+ type: 'qemu',
+ memory: observation('current', observedAt, 'status-mem'),
+ }),
+ metricDisplayMode: mode,
+ visibleColumnIds: ['name', 'memory'],
+ });
+ const notice = container.querySelector(noticeSelector);
+ expect(notice?.querySelector('[aria-hidden]')).toHaveTextContent('May include cache');
+ const message = notice?.querySelector('.sr-only');
+ expect(message).toHaveTextContent('Current. Source: Proxmox. Observed: 2026-09-30');
+ expect(message).toHaveTextContent(
+ 'a high percentage alone does not mean it is short of memory',
+ );
+ expect(message).toHaveTextContent('Pulse agent in the VM');
+ if (mode === 'sparklines') {
+ // The value itself is a current reading; only its source is qualified.
+ expect(
+ screen.getByRole('img', { name: 'test-vm memory history, current 50%' }),
+ ).toBeInTheDocument();
+ } else {
+ expect(screen.getByTestId('memory-bar')).toHaveAttribute('data-reading-state', 'current');
+ }
+ });
+
+ it(`keeps a container's cache caveat out of the ${mode} row label`, () => {
+ const { container } = renderGuestRow({
+ guest: makeGuest({
+ type: 'lxc',
+ memory: observation('current', observedAt, 'cluster-resources'),
+ }),
+ metricDisplayMode: mode,
+ visibleColumnIds: ['name', 'memory'],
+ });
+ // Nothing for the user to set up on a container, so no label per row.
+ expect(container.querySelector(noticeSelector)).toBeNull();
+ if (mode === 'bars') {
+ const message = screen.getByTestId('memory-bar').getAttribute('data-reading-message');
+ expect(message).toContain('cached memory the container can reuse');
+ expect(message).toContain('Pulse agent in the container');
+ expect(message).not.toContain('QEMU');
+ }
+ });
+
+ it(`adds no cache cue to a cache-aware ${mode} reading`, () => {
+ const { container } = renderGuestRow({
+ guest: makeGuest({
+ type: 'qemu',
+ memory: observation('current', observedAt, 'guest-agent-meminfo'),
+ }),
+ metricDisplayMode: mode,
+ visibleColumnIds: ['name', 'memory'],
+ });
+ expect(container.querySelector(noticeSelector)).toBeNull();
+ });
+
it(`withdraws the ${mode} cue only for a qualified same-guest reading`, () => {
const [guest, setGuest] = createSignal(makeGuest({ memory: observation('last-known') }));
const { container } = render(() => (
@@ -1406,6 +1464,42 @@ describe('GuestRow', () => {
expect(notice).not.toHaveTextContent('future-state');
});
+ it('fits the compact cache cue without losing the accessible caveat', () => {
+ const { container } = renderGuestRow({
+ guest: makeGuest({
+ type: 'qemu',
+ memory: observation('current', observedAt, 'cluster-resources'),
+ }),
+ visibleColumnIds: ['name', 'memory'],
+ workloadTableLayoutMode: 'phone',
+ });
+ const notice = container.querySelector(noticeSelector);
+ expect(notice?.querySelector('[aria-hidden]')).toHaveTextContent(/^Cache\?$/);
+ expect(notice?.querySelector('.sr-only')).toHaveTextContent('may include cached memory');
+ });
+
+ it.each([
+ ['last-known', 'observed', 'Last known'],
+ ['current', 'not-a-date', 'Freshness unknown'],
+ ['future-state', 'observed', 'Freshness unknown'],
+ ])(
+ 'keeps a %s cache-inclusive reading under its freshness label, not the current cue',
+ (state, time, label) => {
+ const { container } = renderGuestRow({
+ guest: makeGuest({
+ type: 'qemu',
+ memory: observation(state, time === 'observed' ? observedAt : time, 'status-mem'),
+ }),
+ visibleColumnIds: ['name', 'memory'],
+ });
+ const notice = container.querySelector(noticeSelector);
+ expect(notice?.querySelector('[aria-hidden]')).toHaveTextContent(label);
+ expect(notice).not.toHaveTextContent('May include cache');
+ // The caveat still reaches the user, beside the freshness warning.
+ expect(notice?.querySelector('.sr-only')).toHaveTextContent('may include cached memory');
+ },
+ );
+
it('does not expose unknown provider labels or borrow disk provenance', () => {
const { container } = renderGuestRow({
guest: makeGuest({ memory: observation('private-state', observedAt, 'private-source') }),
diff --git a/frontend-modern/src/utils/__tests__/memoryObservation.test.ts b/frontend-modern/src/utils/__tests__/memoryObservation.test.ts
index 32cf942165..76d8579670 100644
--- a/frontend-modern/src/utils/__tests__/memoryObservation.test.ts
+++ b/frontend-modern/src/utils/__tests__/memoryObservation.test.ts
@@ -1,4 +1,4 @@
-import { describe, expect, it, vi, afterEach } from 'vitest';
+import { describe, expect, it, vi, afterEach, beforeEach } from 'vitest';
import type { WorkloadGuest } from '@/types/workloads';
import {
getCurrentWorkloadMemoryUsage,
@@ -94,6 +94,104 @@ describe('selected workload memory observation', () => {
);
});
+describe('cache-inclusive Proxmox memory sources', () => {
+ const reading = (source: string, state = 'current', changes: Partial<WorkloadGuest> = {}) =>
+ guest({
+ memory: { ...guest().memory, observation: { state, source, observedAt } },
+ ...changes,
+ });
+ beforeEach(() => {
+ vi.spyOn(Date, 'now').mockReturnValue(Date.parse('2026-10-01T12:00:00Z'));
+ });
+
+ it.each(['status-mem', 'status-freemem', 'cluster-resources', 'derived-total-minus-used'])(
+ 'keeps a current %s reading in the row presentation with its caveat',
+ (source) => {
+ const presentation = getWorkloadMemoryObservationPresentation(reading(source), {
+ includeCurrent: false,
+ });
+ expect(presentation).toMatchObject({ state: 'current', mayIncludeCache: true });
+ expect(presentation?.summary).toContain('Proxmox (may include cache)');
+ expect(presentation?.message).toContain('may include cached memory');
+ expect(presentation?.cacheNote).toBeTruthy();
+ },
+ );
+
+ it.each([
+ 'guest-agent-meminfo',
+ 'guest-agent-meminfo-derived',
+ 'agent',
+ 'available-field',
+ 'derived-free-buffers-cached',
+ 'previous-snapshot',
+ 'something-new',
+ ])('does not flag %s as cache-inclusive', (source) => {
+ const presentation = getWorkloadMemoryObservationPresentation(reading(source));
+ expect(presentation?.mayIncludeCache).toBe(false);
+ expect(presentation?.cacheNote).toBeUndefined();
+ expect(presentation?.message).not.toContain('cached memory');
+ expect(presentation?.summary).not.toContain('may include cache');
+ expect(
+ getWorkloadMemoryObservationPresentation(reading(source), { includeCurrent: false }),
+ ).toBeNull();
+ });
+
+ it('names the fix that fits the guest: the QEMU agent is a VM-only, Linux-only option', () => {
+ const vm = getWorkloadMemoryObservationPresentation(reading('status-mem'))!;
+ expect(vm.cacheNote).toContain('memory the VM can reuse');
+ expect(vm.cacheNote).toContain('Pulse agent in the VM');
+ // The advice is conditional: source selection does not promise a replacement.
+ expect(vm.cacheNote).toContain('can let Pulse show its own figure');
+ expect(vm.cacheNote).not.toMatch(/\bwill\b/);
+ expect(vm.cacheNote).toContain('QEMU guest agent on a Linux VM');
+ const canonicalVm = getWorkloadMemoryObservationPresentation(
+ reading('status-mem', 'current', { type: 'vm', workloadType: 'vm' }),
+ )!;
+ expect(canonicalVm.cacheNote).toContain('QEMU guest agent on a Linux VM');
+ for (const changes of [
+ { type: 'lxc' },
+ { type: 'system-container', workloadType: 'system-container' as const },
+ ]) {
+ const note = getWorkloadMemoryObservationPresentation(
+ reading('cluster-resources', 'current', changes),
+ )!.cacheNote;
+ expect(note).toContain('memory the container can reuse');
+ expect(note).toContain('Pulse agent in the container');
+ expect(note).not.toContain('QEMU');
+ }
+ const unknownKind = getWorkloadMemoryObservationPresentation(
+ reading('status-mem', 'current', { type: 'docker' }),
+ )!.cacheNote;
+ expect(unknownKind).toContain('memory the guest can reuse');
+ expect(unknownKind).not.toContain('agent');
+ });
+
+ it('keeps the caveat on a retained reading without calling it current', () => {
+ const presentation = getWorkloadMemoryObservationPresentation(
+ reading('status-mem', 'last-known'),
+ )!;
+ expect(presentation.state).toBe('last-known');
+ expect(presentation.mayIncludeCache).toBe(true);
+ expect(presentation.message).toContain('Not a current measurement.');
+ expect(presentation.message).toContain('may include cached memory');
+ });
+
+ it.each([
+ ['an unavailable observation', reading('status-mem', 'unavailable')],
+ ['a reading with no usable number', reading('status-mem', 'current', { memory: undefined })],
+ ])('never qualifies %s', (_label, input) => {
+ const presentation = getWorkloadMemoryObservationPresentation(input);
+ expect(presentation?.state).toBe('unavailable');
+ expect(presentation?.mayIncludeCache).toBe(false);
+ expect(presentation?.cacheNote).toBeUndefined();
+ });
+
+ it('leaves the selected number unchanged', () => {
+ expect(getCurrentWorkloadMemoryUsage(reading('status-mem'))).toBe(25);
+ expect(getCurrentWorkloadMemoryUsage(reading('cluster-resources'))).toBe(25);
+ });
+});
+
describe('current workload memory selection', () => {
const current = (changes: Partial<WorkloadGuest> = {}) =>
guest({
diff --git a/frontend-modern/src/utils/memoryObservation.ts b/frontend-modern/src/utils/memoryObservation.ts
index 47ee1a273a..879b648b87 100644
--- a/frontend-modern/src/utils/memoryObservation.ts
+++ b/frontend-modern/src/utils/memoryObservation.ts
@@ -17,8 +17,44 @@ export interface MemoryObservationPresentation {
state: 'current' | 'last-known' | 'unavailable' | 'unknown';
summary: string;
message: string;
+ // The selected Proxmox source counts cached memory the guest can reuse as
+ // used, so a high percentage alone is not evidence of memory pressure. This
+ // is a property of the source, independent of how fresh the reading is.
+ mayIncludeCache: boolean;
+ // Why a high reading may be benign, and what the user can do about it. Set
+ // exactly when mayIncludeCache is.
+ cacheNote?: string;
}
+// Proxmox's own listing and status values. Pulse selects them only when no
+// cache-aware source (guest agent, linked Pulse agent, node meminfo) answered;
+// the same four sources set MayIncludeCache in unifiedresources.QualifyGuestMemory.
+const cacheInclusiveMemorySources = new Set([
+ 'status-mem',
+ 'status-freemem',
+ 'cluster-resources',
+ 'derived-total-minus-used',
+]);
+
+export type MemoryGuestKind = 'vm' | 'container';
+
+const getCacheNote = (kind: MemoryGuestKind | undefined): string => {
+ const noun = kind === 'vm' ? 'VM' : kind === 'container' ? 'container' : 'guest';
+ const caveat = `Proxmox's reading may include cached memory the ${noun} can reuse, so a high percentage alone does not mean it is short of memory.`;
+ // A Pulse agent inside the guest can supply its own figure on any OS (a
+ // container's must report the container's own limit). The QEMU guest agent
+ // path reads Linux /proc/meminfo only and needs installing in the guest and
+ // enabling in Proxmox. Neither is guaranteed to replace every fallback
+ // source, so the advice says "can". Kept short: the drawer column is narrow.
+ if (kind === 'vm') {
+ return `${caveat} Installing a Pulse agent in the VM, or setting up the QEMU guest agent on a Linux VM, can let Pulse show its own figure.`;
+ }
+ if (kind === 'container') {
+ return `${caveat} Installing a Pulse agent in the container can let Pulse show its own figure.`;
+ }
+ return caveat;
+};
+
const isMemoryPercent = (value: unknown): value is number =>
typeof value === 'number' && Number.isFinite(value) && value >= 0 && value <= 100;
@@ -101,16 +137,22 @@ export const getMemoryObservationPresentation = (
value: number | undefined,
observation: MemoryObservation | undefined,
requiresObservation = false,
- options: { includeCurrent?: boolean } = {},
+ options: { includeCurrent?: boolean; guestKind?: MemoryGuestKind } = {},
): MemoryObservationPresentation | null => {
// Unannotated unrelated platforms retain their existing behaviour. Legacy
// Proxmox readings lack authority to assert current freshness.
if (!requiresObservation && !observation) return null;
const timestamp = memoryObservationTime(observation);
const state = memoryObservationState(value, observation, timestamp);
+ // An unavailable reading shows no number, so there is nothing to qualify.
+ const mayIncludeCache =
+ state !== 'unavailable' &&
+ observation !== undefined &&
+ cacheInclusiveMemorySources.has(observation.source);
// Most table rows are current and need no cue. Qualify them identically,
- // but avoid building source/date strings that will never be rendered.
- if (state === 'current' && options.includeCurrent === false) return null;
+ // but avoid building source/date strings that will never be rendered. A
+ // current reading from a cache-inclusive source still carries its caveat.
+ if (state === 'current' && options.includeCurrent === false && !mayIncludeCache) return null;
const observed =
timestamp !== null
? new Date(timestamp)
@@ -132,13 +174,25 @@ export const getMemoryObservationPresentation = (
? memorySourceLabels[observation.source]
: 'Unknown source';
const time = observed ? `Observed: ${observed}.` : 'Observation time unknown.';
+ const cacheNote = mayIncludeCache ? getCacheNote(options.guestKind) : undefined;
return {
state,
- summary: `${label} · ${source}${observed ? ` · ${observed}` : ' · time unknown'}`,
- message: `${label}. Source: ${source}. ${time}${state === 'last-known' || state === 'unknown' ? ' Not a current measurement.' : ''}`,
+ summary: `${label} · ${source}${mayIncludeCache ? ' (may include cache)' : ''}${observed ? ` · ${observed}` : ' · time unknown'}`,
+ message: `${label}. Source: ${source}. ${time}${state === 'last-known' || state === 'unknown' ? ' Not a current measurement.' : ''}${cacheNote ? ` ${cacheNote}` : ''}`,
+ mayIncludeCache,
+ ...(cacheNote ? { cacheNote } : {}),
};
};
+// A VM can gain a guest-agent figure; a container has no such agent. Legacy
+// rows say qemu/lxc, canonical rows say vm/system-container.
+export const getWorkloadMemoryGuestKind = (guest: WorkloadGuest): MemoryGuestKind | undefined =>
+ guest.type === 'qemu' || guest.workloadType === 'vm'
+ ? 'vm'
+ : guest.type === 'lxc' || guest.workloadType === 'system-container'
+ ? 'container'
+ : undefined;
+
// Rows and drawers qualify the same selected reading. Neither a disk-read
// deferral nor a refreshed Last seen can renew the memory observation.
export const getWorkloadMemoryObservationPresentation = (
@@ -154,6 +208,9 @@ export const getWorkloadMemoryObservationPresentation = (
value,
guest.memory?.observation,
requiresWorkloadMemoryObservation(guest),
- options,
+ {
+ ...options,
+ guestKind: getWorkloadMemoryGuestKind(guest),
+ },
);
};
diff --git a/internal/mock/demo_scenarios.go b/internal/mock/demo_scenarios.go
index 5918053cfc..83f761d369 100644
--- a/internal/mock/demo_scenarios.go
+++ b/internal/mock/demo_scenarios.go
@@ -527,6 +527,7 @@ func applyDemoWorkloadScenario(workloads []models.VM, profiles []demoWorkloadPro
workloads[i].IPAddresses = nil
workloads[i].NetworkInterfaces = nil
}
+ mockSettleGuestMemoryObservation(&workloads[i].Memory, workloads[i].Status != "stopped", "agent")
workloads[i].LastBackup = demoProfileLastBackup(profile, now)
guestProfiles[workloads[i].VMID] = profile
}
@@ -567,6 +568,7 @@ func applyDemoContainerScenario(workloads []models.Container, profiles []demoWor
workloads[i].IPAddresses = nil
workloads[i].NetworkInterfaces = nil
}
+ mockSettleGuestMemoryObservation(&workloads[i].Memory, workloads[i].Status != "stopped", "cluster-resources")
workloads[i].LastBackup = demoProfileLastBackup(profile, now)
guestProfiles[workloads[i].VMID] = profile
}
diff --git a/internal/mock/generator.go b/internal/mock/generator.go
index 0e641a5cc1..3ac72f1c68 100644
--- a/internal/mock/generator.go
+++ b/internal/mock/generator.go
@@ -548,6 +548,7 @@ func buildFixtureState(config MockConfig) models.StateSnapshot {
vm.Memory.Usage = 0
vm.Memory.Free = vm.Memory.Total
vm.Memory.SwapUsed = 0
+ mockSettleGuestMemoryObservation(&vm.Memory, false, "")
vm.Disk.Used = 0
vm.Disk.Free = vm.Disk.Total
vm.Disk.Usage = -1
@@ -571,6 +572,7 @@ func buildFixtureState(config MockConfig) models.StateSnapshot {
lxc.Memory.Usage = 0
lxc.Memory.Free = lxc.Memory.Total
lxc.Memory.SwapUsed = 0
+ mockSettleGuestMemoryObservation(&lxc.Memory, false, "")
lxc.Disk.Used = 0
lxc.Disk.Free = lxc.Disk.Total
lxc.Disk.Usage = -1
@@ -1275,6 +1277,52 @@ func generateRealisticIO(ioType string) int64 {
return 0
}
+// mockGuestMemoryObservation gives a running guest the provenance a real
+// estate reports for its memory reading, so the UI qualifies mock readings the
+// way it qualifies live ones instead of treating every guest as unannotated.
+func mockGuestMemoryObservation(source string) models.MemoryObservation {
+ return models.MemoryObservation{State: "current", Source: source, ObservedAt: time.Now()}
+}
+
+// applyMockVMMemoryProvenance picks the source a running VM's reading comes
+// from. A quarter of VMs have nothing better than Proxmox's cache-inclusive
+// value, which carries no used | cache | free split. Pulse reads guest memory
+// through the QEMU guest agent from Linux /proc/meminfo only, so a Windows VM
+// is cache-aware only through a Pulse agent inside it.
+func applyMockVMMemoryProvenance(mem *models.Memory, osName string) {
+ switch {
+ case rand.Float64() < 0.25:
+ mem.Free += mem.Cache
+ mem.Cache = 0
+ mem.Observation = mockGuestMemoryObservation("status-mem")
+ case strings.Contains(strings.ToLower(osName), "windows"):
+ mem.Observation = mockGuestMemoryObservation("agent")
+ default:
+ mem.Observation = mockGuestMemoryObservation("guest-agent-meminfo")
+ }
+}
+
+// refreshMockMemoryObservation moves a current reading's observation time with
+// the sample just taken; a retained or unavailable observation keeps its own.
+func refreshMockMemoryObservation(mem *models.Memory, at time.Time) {
+ if mem.Observation.State == "current" {
+ mem.Observation.ObservedAt = at
+ }
+}
+
+// mockSettleGuestMemoryObservation keeps a guest's memory provenance in step
+// with a power state a scenario forces after generation: a stopped guest has
+// no reading to qualify, and a guest started later needs a current source. A
+// retained or current observation on a running guest is left as it is.
+func mockSettleGuestMemoryObservation(memory *models.Memory, running bool, runningSource string) {
+ switch {
+ case !running:
+ memory.Observation = models.MemoryObservation{State: "unavailable", Source: "powered-off"}
+ case memory.Observation.State == "" || memory.Observation.State == "unavailable":
+ memory.Observation = mockGuestMemoryObservation(runningSource)
+ }
+}
+
func generateVM(nodeName string, instance string, vmid int, config MockConfig) models.VM {
name := generateGuestName("vm", fmt.Sprintf("%s:%s:%d", instance, nodeName, vmid))
status := "running"
@@ -1345,6 +1393,8 @@ func generateVM(nodeName string, instance string, vmid int, config MockConfig) m
mem.Free -= mem.Cache
}
uptime = int64(3600 * (1 + rand.Intn(720))) // 1-720 hours
+ } else {
+ mem.Observation = models.MemoryObservation{State: "unavailable", Source: "powered-off"}
}
// Disk stats
@@ -1384,6 +1434,9 @@ func generateVM(nodeName string, instance string, vmid int, config MockConfig) m
osName, osVersion := generateGuestOSMetadata()
ipAddresses, networkIfaces := generateGuestNetworkInfo()
+ if status == "running" {
+ applyMockVMMemoryProvenance(&mem, osName)
+ }
vm := models.VM{
Name: name,
@@ -4922,7 +4975,16 @@ func generateContainer(nodeName string, instance string, vmid int, config MockCo
SwapTotal: swapTotal,
SwapUsed: swapUsed,
}
+ // Proxmox reports container memory from the cgroup, cache included;
+ // only a linked Pulse agent supplies a cache-aware figure.
+ if rand.Float64() < 0.15 {
+ mem.Observation = mockGuestMemoryObservation("agent")
+ } else {
+ mem.Observation = mockGuestMemoryObservation("cluster-resources")
+ }
uptime = int64(3600 * (1 + rand.Intn(1440))) // 1-1440 hours (up to 60 days)
+ } else {
+ mem.Observation = models.MemoryObservation{State: "unavailable", Source: "powered-off"}
}
// Disk stats - containers typically smaller
@@ -6365,11 +6427,24 @@ func updateFixtureStateMetricsSelectedAt(
}
*lastSeen = refreshNow
}
+ // A current memory reading is observed on the same poll that sights the
+ // guest, so its time moves with the sighting, in static-metrics mode too.
+ stampGuestMemoryObservation := func(nodeName, status string, memory *models.Memory) {
+ if !selection.includesProxmoxNode(nodeName) || status != "running" {
+ return
+ }
+ if _, offline := offlineNodes[nodeName]; offline {
+ return
+ }
+ refreshMockMemoryObservation(memory, refreshNow)
+ }
for i := range data.VMs {
stampGuestSighting(data.VMs[i].Node, &data.VMs[i].LastSeen)
+ stampGuestMemoryObservation(data.VMs[i].Node, data.VMs[i].Status, &data.VMs[i].Memory)
}
for i := range data.Containers {
stampGuestSighting(data.Containers[i].Node, &data.Containers[i].LastSeen)
+ stampGuestMemoryObservation(data.Containers[i].Node, data.Containers[i].Status, &data.Containers[i].Memory)
}
for i := range data.Storage {
stampGuestSighting(data.Storage[i].Node, &data.Storage[i].LastSeen)
diff --git a/internal/mock/generator_test.go b/internal/mock/generator_test.go
index 0c76b8346d..d55ac49a15 100644
--- a/internal/mock/generator_test.go
+++ b/internal/mock/generator_test.go
@@ -1460,3 +1460,163 @@ func TestBuildFixtureStateMergesSharedPBSStorageLikeThePoller(t *testing.T) {
t.Fatal("expected shared PBS storage in the fixture")
}
}
+
+// Mock guests report where their memory reading came from, the way a live
+// estate does, so the UI qualifies mock readings instead of treating every
+// guest as unannotated ("Freshness unknown").
+func TestGeneratedGuestMemoryProvenanceMatchesPowerStateAndOS(t *testing.T) {
+ cfg := DefaultConfig
+ cfg.StoppedPercent = 0.3
+ now := time.Now()
+
+ sources := map[string]int{}
+ for i := 0; i < 300; i++ {
+ vm := generateVM("node-01", "mock-cluster", 1000+i, cfg)
+ obs := vm.Memory.Observation
+ if vm.Status != "running" {
+ if obs.State != "unavailable" || obs.Source != "powered-off" {
+ t.Fatalf("stopped VM observation = %+v, want unavailable powered-off", obs)
+ }
+ continue
+ }
+ sources[obs.Source]++
+ if obs.State != "current" || obs.ObservedAt.IsZero() || obs.ObservedAt.After(now.Add(time.Minute)) {
+ t.Fatalf("running VM observation = %+v, want a current reading observed now", obs)
+ }
+ switch obs.Source {
+ case "status-mem":
+ // Proxmox's value carries no used | cache | free split.
+ if vm.Memory.Cache != 0 {
+ t.Fatalf("cache-inclusive VM reported a measured cache split: %+v", vm.Memory)
+ }
+ case "guest-agent-meminfo":
+ // The QEMU guest agent reads Linux /proc/meminfo only.
+ if strings.Contains(strings.ToLower(vm.OSName), "windows") {
+ t.Fatalf("Windows VM %q claims Linux meminfo provenance", vm.OSName)
+ }
+ case "agent":
+ default:
+ t.Fatalf("running VM has unexpected memory source %q", obs.Source)
+ }
+ }
+ if sources["status-mem"] == 0 || sources["guest-agent-meminfo"] == 0 || sources["agent"] == 0 {
+ t.Fatalf("expected cache-inclusive, guest-agent and Pulse-agent VMs in the estate, got %v", sources)
+ }
+
+ sources = map[string]int{}
+ for i := 0; i < 300; i++ {
+ ct := generateContainer("node-01", "mock-cluster", 2000+i, cfg)
+ obs := ct.Memory.Observation
+ if ct.Status != "running" {
+ if obs.State != "unavailable" || obs.Source != "powered-off" {
+ t.Fatalf("stopped container observation = %+v, want unavailable powered-off", obs)
+ }
+ continue
+ }
+ sources[obs.Source]++
+ if obs.State != "current" || obs.ObservedAt.IsZero() {
+ t.Fatalf("running container observation = %+v, want a current reading", obs)
+ }
+ if obs.Source != "cluster-resources" && obs.Source != "agent" {
+ t.Fatalf("running container has unexpected memory source %q", obs.Source)
+ }
+ }
+ if sources["cluster-resources"] == 0 || sources["agent"] == 0 {
+ t.Fatalf("expected cgroup and Pulse-agent containers in the estate, got %v", sources)
+ }
+}
+
+func TestMockGuestMemoryObservationFollowsScenarioPowerState(t *testing.T) {
+ cases := []struct {
+ name string
+ before models.MemoryObservation
+ running bool
+ want models.MemoryObservation
+ }{
+ {
+ name: "forced stopped loses its reading",
+ before: models.MemoryObservation{State: "current", Source: "status-mem", ObservedAt: time.Now()},
+ running: false,
+ want: models.MemoryObservation{State: "unavailable", Source: "powered-off"},
+ },
+ {
+ name: "forced running after generation gets a current source",
+ before: models.MemoryObservation{State: "unavailable", Source: "powered-off"},
+ running: true,
+ want: models.MemoryObservation{State: "current", Source: "agent"},
+ },
+ {
+ name: "a running guest keeps its own source",
+ before: models.MemoryObservation{State: "current", Source: "status-mem", ObservedAt: time.Now()},
+ running: true,
+ want: models.MemoryObservation{State: "current", Source: "status-mem"},
+ },
+ {
+ name: "a retained reading on a running guest is kept",
+ before: models.MemoryObservation{State: "last-known", Source: "status-mem", ObservedAt: time.Now().Add(-time.Hour)},
+ running: true,
+ want: models.MemoryObservation{State: "last-known", Source: "status-mem"},
+ },
+ }
+ for _, tc := range cases {
+ t.Run(tc.name, func(t *testing.T) {
+ mem := models.Memory{Observation: tc.before}
+ mockSettleGuestMemoryObservation(&mem, tc.running, "agent")
+ got := mem.Observation
+ if got.State != tc.want.State || got.Source != tc.want.Source {
+ t.Fatalf("observation = %+v, want state %q source %q", got, tc.want.State, tc.want.Source)
+ }
+ if tc.want.State == "current" && got.ObservedAt.IsZero() {
+ t.Fatalf("a current observation needs its observation time: %+v", got)
+ }
+ })
+ }
+}
+
+func TestFixtureRefreshMovesCurrentGuestMemoryObservationTime(t *testing.T) {
+ for _, randomMetrics := range []bool{true, false} {
+ name := "random metrics"
+ if !randomMetrics {
+ name = "static metrics"
+ }
+ t.Run(name, func(t *testing.T) {
+ cfg := DefaultConfig
+ cfg.RandomMetrics = randomMetrics
+ stale := time.Now().Add(-time.Hour)
+ refreshAt := time.Now().Truncate(time.Second)
+
+ data := &models.StateSnapshot{
+ VMs: []models.VM{
+ {ID: "vm-current", Node: "node-01", Status: "running", Memory: models.Memory{
+ Total: 100, Observation: models.MemoryObservation{State: "current", Source: "status-mem", ObservedAt: stale},
+ }},
+ {ID: "vm-retained", Node: "node-01", Status: "running", Memory: models.Memory{
+ Total: 100, Observation: models.MemoryObservation{State: "last-known", Source: "status-mem", ObservedAt: stale},
+ }},
+ {ID: "vm-stopped", Node: "node-01", Status: "stopped", Memory: models.Memory{
+ Total: 100, Observation: models.MemoryObservation{State: "unavailable", Source: "powered-off"},
+ }},
+ },
+ Containers: []models.Container{
+ {ID: "ct-current", Node: "node-01", Status: "running", Memory: models.Memory{
+ Total: 100, Observation: models.MemoryObservation{State: "current", Source: "cluster-resources", ObservedAt: stale},
+ }},
+ },
+ }
+ updateFixtureStateMetricsAt(data, cfg, refreshAt)
+
+ if got := data.VMs[0].Memory.Observation; !got.ObservedAt.Equal(refreshAt) || got.Source != "status-mem" {
+ t.Fatalf("current VM observation = %+v, want source kept and time %v", got, refreshAt)
+ }
+ if got := data.VMs[1].Memory.Observation; !got.ObservedAt.Equal(stale) || got.State != "last-known" {
+ t.Fatalf("a retained observation keeps its own state and time, got %+v", got)
+ }
+ if got := data.VMs[2].Memory.Observation; !got.ObservedAt.IsZero() || got.Source != "powered-off" {
+ t.Fatalf("a stopped guest has no observation time to move, got %+v", got)
+ }
+ if got := data.Containers[0].Memory.Observation; !got.ObservedAt.Equal(refreshAt) || got.Source != "cluster-resources" {
+ t.Fatalf("current container observation = %+v, want source kept and time %v", got, refreshAt)
+ }
+ })
+ }
+}