Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Explain guest API readings without inventing freshness

pulse-triage[bot]3e4596b92ac4
diff --git a/docs/API.md b/docs/API.md
index 296d4cc116..b437b434d5 100644
--- a/docs/API.md
+++ b/docs/API.md
@@ -272,7 +272,45 @@ canonical `verdict` and stable `reasons` (`code` plus optional compact
`detail`). Consumers should use this envelope for cross-platform posture
instead of deriving health from provider-specific status strings.
-Note: guest disk usage percentages use `-1` as an "unknown" sentinel — reported when a VM is stopped or its guest agent is unavailable, so there is no filesystem view to measure. Consumers should treat negative values as "no data", not as a percentage; the accompanying `diskStatusReason` field (e.g. `vm-stopped`, `agent-disabled`) says why.
+#### Guest readings: availability and age
+
+In legacy VM payloads, `disk.usage: -1` means unavailable, not negative usage.
+In `GET /api/resources` and `GET /api/resources/{id}`, an unavailable disk
+reading is omitted as `metrics.disk`. **Do not replace an absent metric with
+zero**; a genuinely observed zero is valid. Allocated virtual-disk capacity
+alone does not establish used space inside the guest.
+
+Read `proxmox.diskStatusReason` alongside the value. A `prev-` reason means
+retained disk evidence, not a fresh reading. `agent-not-running` can also
+represent a general guest-agent HTTP 500 error: it does not establish that the
+guest's service is absent or stopped. An absent reason alone does not prove
+freshness either.
+
+For memory, use `metrics.memory.observation` for the **selected metric's**
+source and age, not a different source's retained `proxmox.memory` facet:
+
+| Observation `state` | Meaning for the consumer |
+| --- | --- |
+| `current` | Accepted source observation; a normal cache hit can keep the original `observedAt`. Check its age for your use. |
+| `last-known` | Retained evidence, not a new measurement. Keep its original age visible; do not record it as a fresh sample. |
+| `unavailable` | No usable usage observation; do not treat retained numbers or zero-valued fields as a measurement. |
+| Missing or unrecognised | Freshness is unknown, including older payloads without an observation. Do not infer `current`. |
+
+`observedAt` belongs to that source observation. If it is absent or in the
+future, its age is unknown; do not substitute the resource's `lastSeen` or
+`updatedAt`. In the raw `proxmox.memory` facet, `usageUnavailable: true` can
+preserve known capacity without measured usage. An absent `metrics.memory`
+is not 0% usage. Conversely, an independently observed PVE or Pulse-agent
+memory reading can remain current while QEMU disk collection is deferred;
+disk state is not memory provenance.
+
+A running VM, an advancing row timestamp or an OK backup does not prove fresh
+guest readings or successful thaw. Do not install or restart an agent or send
+live guest-agent probes merely to fill these gaps, especially during a backup,
+freeze/thaw or an unresponsive-guest incident. Follow [Backup safety](VM_DISK_MONITORING.md#backup-safety)
+and [missing-reading guidance](VM_DISK_MONITORING.md#a-missing-reading-is-not-an-installation-diagnosis).
+Recovery needs independent thaw, fresh successful writes to every covered
+filesystem and workload liveness; retained API values cannot establish it.
Availability is an additive resource facet. `availability` is the compatibility
summary used by existing clients; `availabilityChecks` contains every check
diff --git a/frontend-modern/public/docs/API.md b/frontend-modern/public/docs/API.md
index 296d4cc116..b437b434d5 100644
--- a/frontend-modern/public/docs/API.md
+++ b/frontend-modern/public/docs/API.md
@@ -272,7 +272,45 @@ canonical `verdict` and stable `reasons` (`code` plus optional compact
`detail`). Consumers should use this envelope for cross-platform posture
instead of deriving health from provider-specific status strings.
-Note: guest disk usage percentages use `-1` as an "unknown" sentinel — reported when a VM is stopped or its guest agent is unavailable, so there is no filesystem view to measure. Consumers should treat negative values as "no data", not as a percentage; the accompanying `diskStatusReason` field (e.g. `vm-stopped`, `agent-disabled`) says why.
+#### Guest readings: availability and age
+
+In legacy VM payloads, `disk.usage: -1` means unavailable, not negative usage.
+In `GET /api/resources` and `GET /api/resources/{id}`, an unavailable disk
+reading is omitted as `metrics.disk`. **Do not replace an absent metric with
+zero**; a genuinely observed zero is valid. Allocated virtual-disk capacity
+alone does not establish used space inside the guest.
+
+Read `proxmox.diskStatusReason` alongside the value. A `prev-` reason means
+retained disk evidence, not a fresh reading. `agent-not-running` can also
+represent a general guest-agent HTTP 500 error: it does not establish that the
+guest's service is absent or stopped. An absent reason alone does not prove
+freshness either.
+
+For memory, use `metrics.memory.observation` for the **selected metric's**
+source and age, not a different source's retained `proxmox.memory` facet:
+
+| Observation `state` | Meaning for the consumer |
+| --- | --- |
+| `current` | Accepted source observation; a normal cache hit can keep the original `observedAt`. Check its age for your use. |
+| `last-known` | Retained evidence, not a new measurement. Keep its original age visible; do not record it as a fresh sample. |
+| `unavailable` | No usable usage observation; do not treat retained numbers or zero-valued fields as a measurement. |
+| Missing or unrecognised | Freshness is unknown, including older payloads without an observation. Do not infer `current`. |
+
+`observedAt` belongs to that source observation. If it is absent or in the
+future, its age is unknown; do not substitute the resource's `lastSeen` or
+`updatedAt`. In the raw `proxmox.memory` facet, `usageUnavailable: true` can
+preserve known capacity without measured usage. An absent `metrics.memory`
+is not 0% usage. Conversely, an independently observed PVE or Pulse-agent
+memory reading can remain current while QEMU disk collection is deferred;
+disk state is not memory provenance.
+
+A running VM, an advancing row timestamp or an OK backup does not prove fresh
+guest readings or successful thaw. Do not install or restart an agent or send
+live guest-agent probes merely to fill these gaps, especially during a backup,
+freeze/thaw or an unresponsive-guest incident. Follow [Backup safety](VM_DISK_MONITORING.md#backup-safety)
+and [missing-reading guidance](VM_DISK_MONITORING.md#a-missing-reading-is-not-an-installation-diagnosis).
+Recovery needs independent thaw, fresh successful writes to every covered
+filesystem and workload liveness; retained API values cannot establish it.
Availability is an additive resource facet. `availability` is the compatibility
summary used by existing clients; `availabilityChecks` contains every check
diff --git a/frontend-modern/src/utils/__tests__/guestObservationHelp.test.ts b/frontend-modern/src/utils/__tests__/guestObservationHelp.test.ts
new file mode 100644
index 0000000000..893f096b3f
--- /dev/null
+++ b/frontend-modern/src/utils/__tests__/guestObservationHelp.test.ts
@@ -0,0 +1,101 @@
+import { readFileSync } from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { describe, expect, it } from 'vitest';
+import { renderDocMarkdown } from '@/features/docs/docMarkdown';
+
+const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../..');
+const read = (name: string) => readFileSync(path.join(root, name), 'utf8');
+const api = read('docs/API.md');
+const section =
+ api.split('#### Guest readings: availability and age')[1]?.split('\nAvailability is')[0] ?? '';
+
+function render(markdown: string, name: string): HTMLElement {
+ const article = document.createElement('article');
+ article.innerHTML = renderDocMarkdown(markdown, name);
+ return article;
+}
+
+const rendered = () => render(section, 'API');
+const text = () => rendered().textContent?.replace(/\s+/g, ' ');
+
+describe('guest observation API help', () => {
+ it('distinguishes the legacy sentinel from an omitted unified metric and an observed zero', () => {
+ expect(read('frontend-modern/public/docs/API.md')).toBe(api);
+ expect(text()).toContain('disk.usage: -1 means unavailable, not negative usage');
+ expect(text()).toContain('omitted as metrics.disk');
+ expect(text()).toContain('Do not replace an absent metric with zero');
+ expect(text()).toContain('a genuinely observed zero is valid');
+ expect(text()).toContain('Allocated virtual-disk capacity alone does not establish used space');
+ // These are the production omission/presence rules, not invented client fields.
+ const types = read('internal/unifiedresources/types.go');
+ expect(types).toMatch(/Metrics\s+\*ResourceMetrics\s+`json:"metrics,omitempty"`/);
+ expect(types).toMatch(/Disk\s+\*MetricValue\s+`json:"disk,omitempty"`/);
+ const metrics = read('internal/unifiedresources/metrics.go');
+ expect(metrics).toContain('if hasObservedDiskUsage(disk)');
+ expect(metrics).toContain('disk.Usage >= 0 && disk.Usage <= 100');
+ });
+
+ it('keeps retained disk evidence and an ambiguous error separate from a service diagnosis', () => {
+ expect(text()).toContain('proxmox.diskStatusReason');
+ expect(text()).toContain('A prev- reason means retained disk evidence, not a fresh reading');
+ expect(text()).toContain('general guest-agent HTTP 500 error');
+ expect(text()).toContain("does not establish that the guest's service is absent or stopped");
+ expect(text()).toContain('An absent reason alone does not prove freshness');
+ const collector = read('internal/monitoring/guest_disk_stability.go');
+ expect(collector).toContain('"prev-" + diskStatusReason');
+ expect(collector).toMatch(
+ /case strings.Contains\(errStr, "500"\):\s+return "agent-not-running"/,
+ );
+ });
+
+ it('renders selected-source memory states without renewing old or missing observations', () => {
+ const rows = [...rendered().querySelectorAll('tbody tr')].map((row) =>
+ [...row.querySelectorAll('td')].map((cell) => cell.textContent?.trim()),
+ );
+ expect(rows.map((row) => row[0])).toEqual([
+ 'current',
+ 'last-known',
+ 'unavailable',
+ 'Missing or unrecognised',
+ ]);
+ expect(rows[0][1]).toContain('original observedAt');
+ expect(rows[1][1]).toContain('do not record it as a fresh sample');
+ expect(rows[2][1]).toContain(
+ 'do not treat retained numbers or zero-valued fields as a measurement',
+ );
+ expect(rows[3][1]).toContain('Do not infer current');
+ expect(text()).toContain('metrics.memory.observation');
+ expect(text()).toContain("selected metric's source and age");
+ expect(text()).toContain("do not substitute the resource's lastSeen or updatedAt");
+ expect(text()).toContain('usageUnavailable: true');
+ expect(text()).toContain('An absent metrics.memory is not 0% usage');
+ expect(text()).toContain('disk state is not memory provenance');
+ const model = read('internal/models/models.go');
+ expect(model).toContain('`json:"observation,omitzero"`');
+ expect(model).toContain('`json:"observedAt,omitzero"`');
+ expect(model).toContain('`json:"usageUnavailable,omitempty"`');
+ expect(read('internal/unifiedresources/metrics.go')).toContain(
+ 'Observation: memory.Observation',
+ );
+ });
+
+ it('makes the existing safety and missing-reading guidance reachable without live probe commands', () => {
+ expect(text()).toContain('an OK backup does not prove fresh guest readings or successful thaw');
+ expect(text()).toContain('Do not install or restart an agent or send live guest-agent probes');
+ expect(text()).toContain(
+ 'fresh successful writes to every covered filesystem and workload liveness',
+ );
+ expect(text()).toContain('retained API values cannot establish it');
+ expect(rendered().querySelector('pre')).toBeNull();
+ const target = render(read('docs/VM_DISK_MONITORING.md'), 'VM_DISK_MONITORING');
+ for (const [label, fragment] of [
+ ['Backup safety', 'backup-safety'],
+ ['missing-reading guidance', 'a-missing-reading-is-not-an-installation-diagnosis'],
+ ]) {
+ const link = rendered().querySelector(`a[href="/docs/VM_DISK_MONITORING#${fragment}"]`);
+ expect(link?.textContent).toBe(label);
+ expect(target.querySelector(`#${fragment}`)).not.toBeNull();
+ }
+ });
+});