Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Explain safe cross-installation Proxmox identity comparisons

pulse-triage[bot]453509905aff
diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md
index 9fca27bca8..0653fae3d3 100644
--- a/docs/CONFIGURATION.md
+++ b/docs/CONFIGURATION.md
@@ -336,6 +336,20 @@ PULSE_AGENT_CONNECT_URL=https://agents.example.com:7656
Agents then post telemetry to `https://agents.example.com:7656/api/agents/agent/report` and establish their command channel at `wss://agents.example.com:7656/api/agent/ws`, while the web UI and management API remain reachable only on the private `FRONTEND_PORT` listener. If command execution is enabled, both routes must traverse the same proxy/firewall path; a successful report does not prove that the WebSocket is admitted.
+### Multiple Proxmox installations
+
+Pulse is intended to monitor multiple Proxmox clusters and standalone nodes
+through separate saved Proxmox connections. Native node names and VMIDs can
+repeat across independent installations; a matching name or VMID alone does
+not identify the same resource.
+
+A distinct Pulse connection name helps you recognise the saved connection.
+Neither that label nor a [cluster member display name](#proxmox-cluster-node-display-names)
+renames the native Proxmox node or VMID, or repairs incorrect attribution.
+If node errors, backup status or agent-backed Docker monitoring become mixed
+after adding a connection, use the [cross-installation identity checks](TROUBLESHOOTING.md#monitoring-is-mixed-between-proxmox-installations).
+Do not rename production nodes or repeat the addition as a diagnostic.
+
### Proxmox Cluster Node Display Names
Nodes discovered through one Proxmox VE cluster connection can have an
diff --git a/docs/PBS.md b/docs/PBS.md
index 43aabdbba6..d188c13f5f 100644
--- a/docs/PBS.md
+++ b/docs/PBS.md
@@ -424,6 +424,11 @@ If the same backup remains listed twice, report those redacted distinctions and
whether each entry came from direct PBS or PVE passthrough. Do not delete backups
or change retention to hide a display problem.
+If overlapping VMIDs also coincide with wrong-node errors or lost agent-backed
+Docker monitoring after adding another Proxmox connection, use the
+[cross-installation identity checks](TROUBLESHOOTING.md#monitoring-is-mixed-between-proxmox-installations).
+A duplicate-backup check alone does not resolve those other symptoms.
+
---
## Data Source Indicator
diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md
index cf3a061068..d2c7def3e8 100644
--- a/docs/TROUBLESHOOTING.md
+++ b/docs/TROUBLESHOOTING.md
@@ -281,6 +281,9 @@ restore, not by overwriting today's history. Follow
#### Agent fleet update or identity issue
+- If readings became mixed after adding another Proxmox installation, start
+ with the [cross-installation identity checks](#monitoring-is-mixed-between-proxmox-installations),
+ not an agent update or re-enrolment command.
- Open an outdated-agent notice or
`/settings/infrastructure?agentDoctor=1` to open **Agent Doctor** and
copy the platform-specific command for each reported host. This is a manual
@@ -386,6 +389,47 @@ to compare one affected row's explanation, Job, History and Access with the
matching native PBS record. Do not run a new backup, restart or clear history
just to diagnose the disagreement.
+#### Monitoring is mixed between Proxmox installations
+
+Pulse is intended to monitor [multiple Proxmox installations](CONFIGURATION.md#multiple-proxmox-installations).
+Separate installations can reuse native node names and VMIDs. A distinct
+Pulse connection label or display name is not a repair for wrong attribution.
+
+If adding a connection led to node errors on the wrong node, unavailable
+backup status for overlapping VMIDs, or lost agent-backed Docker monitoring,
+treat each as an unresolved symptom. One restored view does not establish
+that the others recovered. Use **existing observations only**; if you already
+repaired the setup, keep that working setup and the original sequence rather
+than undoing the repair or repeating the addition.
+
+| Affected reading | Safe comparison | What it does not prove |
+| --- | --- | --- |
+| Node errors | Compare the affected node and observation time with that installation's own Proxmox view. | A similar node name or a different Pulse label does not establish which installation owns the error. |
+| Guest backup status | Compare one affected guest with the matching native Proxmox/PBS record: owning installation, guest type/VMID, datastore, namespace and backup time. Follow the [PBS evidence checks](PBS.md#backups-are-visible-but-coverage-says-unprotected). | The same VMID or a visible backup from another installation does not establish protection for this guest. |
+| Agent-backed Docker monitoring | In **Settings → Infrastructure → Agent Doctor**, inspect the affected host's existing **Last seen** and **Identity evidence**, including **Connection** and **Hostname** where available. Compare with the known Docker host, not a similar guest name. | Recent agent contact or a Healthy badge does not prove that the correct host's containers are visible or correctly attributed. Missing evidence is unknown, not proof that the agent stopped. |
+
+While attribution is uncertain, use each installation's own Proxmox/PBS
+views for node errors and backup status, and the Docker host's existing runtime
+view for container status. Do not make backup, restore or workload changes
+from a potentially misattributed Pulse reading.
+
+For a report, give the running server/agent versions, the original sequence,
+which of the three symptoms remain, and one affected pair's expected versus
+displayed origin and observation times where available. Use **consistent
+placeholders** such as `site-A`, `site-B`, `node-X` and `guest-100` across every
+view; say explicitly when the native names or VMIDs are equal. Keep actual
+addresses, hostnames, connection IDs and machine identities private. Do not
+share a full Agent Doctor report, configuration, API response, HAR export,
+token or unredacted screenshot. Unavailable original evidence can be reported
+as unavailable; do not recreate the incident to obtain it.
+
+Do not rename production nodes, change VMIDs or machine IDs, delete/re-add
+connections, re-enrol agents, rotate tokens, restart services or clear History
+just to diagnose this disagreement. Do not run live diagnostics, guest-agent
+probes, a new backup or a restore as an identity test. An OK backup does not
+prove thaw; a frozen or unresponsive guest needs the separate
+[backup safety procedure](VM_DISK_MONITORING.md#backup-safety), not these display checks.
+
#### Temperature data missing
- Compare the affected host's active agent version, last report, sensor and
observation time. A current server or another sensor's value is not evidence
diff --git a/frontend-modern/public/docs/CONFIGURATION.md b/frontend-modern/public/docs/CONFIGURATION.md
index 9fca27bca8..0653fae3d3 100644
--- a/frontend-modern/public/docs/CONFIGURATION.md
+++ b/frontend-modern/public/docs/CONFIGURATION.md
@@ -336,6 +336,20 @@ PULSE_AGENT_CONNECT_URL=https://agents.example.com:7656
Agents then post telemetry to `https://agents.example.com:7656/api/agents/agent/report` and establish their command channel at `wss://agents.example.com:7656/api/agent/ws`, while the web UI and management API remain reachable only on the private `FRONTEND_PORT` listener. If command execution is enabled, both routes must traverse the same proxy/firewall path; a successful report does not prove that the WebSocket is admitted.
+### Multiple Proxmox installations
+
+Pulse is intended to monitor multiple Proxmox clusters and standalone nodes
+through separate saved Proxmox connections. Native node names and VMIDs can
+repeat across independent installations; a matching name or VMID alone does
+not identify the same resource.
+
+A distinct Pulse connection name helps you recognise the saved connection.
+Neither that label nor a [cluster member display name](#proxmox-cluster-node-display-names)
+renames the native Proxmox node or VMID, or repairs incorrect attribution.
+If node errors, backup status or agent-backed Docker monitoring become mixed
+after adding a connection, use the [cross-installation identity checks](TROUBLESHOOTING.md#monitoring-is-mixed-between-proxmox-installations).
+Do not rename production nodes or repeat the addition as a diagnostic.
+
### Proxmox Cluster Node Display Names
Nodes discovered through one Proxmox VE cluster connection can have an
diff --git a/frontend-modern/public/docs/PBS.md b/frontend-modern/public/docs/PBS.md
index 43aabdbba6..d188c13f5f 100644
--- a/frontend-modern/public/docs/PBS.md
+++ b/frontend-modern/public/docs/PBS.md
@@ -424,6 +424,11 @@ If the same backup remains listed twice, report those redacted distinctions and
whether each entry came from direct PBS or PVE passthrough. Do not delete backups
or change retention to hide a display problem.
+If overlapping VMIDs also coincide with wrong-node errors or lost agent-backed
+Docker monitoring after adding another Proxmox connection, use the
+[cross-installation identity checks](TROUBLESHOOTING.md#monitoring-is-mixed-between-proxmox-installations).
+A duplicate-backup check alone does not resolve those other symptoms.
+
---
## Data Source Indicator
diff --git a/frontend-modern/public/docs/TROUBLESHOOTING.md b/frontend-modern/public/docs/TROUBLESHOOTING.md
index cf3a061068..d2c7def3e8 100644
--- a/frontend-modern/public/docs/TROUBLESHOOTING.md
+++ b/frontend-modern/public/docs/TROUBLESHOOTING.md
@@ -281,6 +281,9 @@ restore, not by overwriting today's history. Follow
#### Agent fleet update or identity issue
+- If readings became mixed after adding another Proxmox installation, start
+ with the [cross-installation identity checks](#monitoring-is-mixed-between-proxmox-installations),
+ not an agent update or re-enrolment command.
- Open an outdated-agent notice or
`/settings/infrastructure?agentDoctor=1` to open **Agent Doctor** and
copy the platform-specific command for each reported host. This is a manual
@@ -386,6 +389,47 @@ to compare one affected row's explanation, Job, History and Access with the
matching native PBS record. Do not run a new backup, restart or clear history
just to diagnose the disagreement.
+#### Monitoring is mixed between Proxmox installations
+
+Pulse is intended to monitor [multiple Proxmox installations](CONFIGURATION.md#multiple-proxmox-installations).
+Separate installations can reuse native node names and VMIDs. A distinct
+Pulse connection label or display name is not a repair for wrong attribution.
+
+If adding a connection led to node errors on the wrong node, unavailable
+backup status for overlapping VMIDs, or lost agent-backed Docker monitoring,
+treat each as an unresolved symptom. One restored view does not establish
+that the others recovered. Use **existing observations only**; if you already
+repaired the setup, keep that working setup and the original sequence rather
+than undoing the repair or repeating the addition.
+
+| Affected reading | Safe comparison | What it does not prove |
+| --- | --- | --- |
+| Node errors | Compare the affected node and observation time with that installation's own Proxmox view. | A similar node name or a different Pulse label does not establish which installation owns the error. |
+| Guest backup status | Compare one affected guest with the matching native Proxmox/PBS record: owning installation, guest type/VMID, datastore, namespace and backup time. Follow the [PBS evidence checks](PBS.md#backups-are-visible-but-coverage-says-unprotected). | The same VMID or a visible backup from another installation does not establish protection for this guest. |
+| Agent-backed Docker monitoring | In **Settings → Infrastructure → Agent Doctor**, inspect the affected host's existing **Last seen** and **Identity evidence**, including **Connection** and **Hostname** where available. Compare with the known Docker host, not a similar guest name. | Recent agent contact or a Healthy badge does not prove that the correct host's containers are visible or correctly attributed. Missing evidence is unknown, not proof that the agent stopped. |
+
+While attribution is uncertain, use each installation's own Proxmox/PBS
+views for node errors and backup status, and the Docker host's existing runtime
+view for container status. Do not make backup, restore or workload changes
+from a potentially misattributed Pulse reading.
+
+For a report, give the running server/agent versions, the original sequence,
+which of the three symptoms remain, and one affected pair's expected versus
+displayed origin and observation times where available. Use **consistent
+placeholders** such as `site-A`, `site-B`, `node-X` and `guest-100` across every
+view; say explicitly when the native names or VMIDs are equal. Keep actual
+addresses, hostnames, connection IDs and machine identities private. Do not
+share a full Agent Doctor report, configuration, API response, HAR export,
+token or unredacted screenshot. Unavailable original evidence can be reported
+as unavailable; do not recreate the incident to obtain it.
+
+Do not rename production nodes, change VMIDs or machine IDs, delete/re-add
+connections, re-enrol agents, rotate tokens, restart services or clear History
+just to diagnose this disagreement. Do not run live diagnostics, guest-agent
+probes, a new backup or a restore as an identity test. An OK backup does not
+prove thaw; a frozen or unresponsive guest needs the separate
+[backup safety procedure](VM_DISK_MONITORING.md#backup-safety), not these display checks.
+
#### Temperature data missing
- Compare the affected host's active agent version, last report, sensor and
observation time. A current server or another sensor's value is not evidence
diff --git a/frontend-modern/src/features/docs/__tests__/proxmoxIdentityGuide.test.ts b/frontend-modern/src/features/docs/__tests__/proxmoxIdentityGuide.test.ts
new file mode 100644
index 0000000000..077d1eec46
--- /dev/null
+++ b/frontend-modern/src/features/docs/__tests__/proxmoxIdentityGuide.test.ts
@@ -0,0 +1,137 @@
+import { readFileSync } from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { describe, expect, it } from 'vitest';
+import { renderDocMarkdown } from '../docMarkdown';
+
+const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../../..');
+const identityFragment = '#monitoring-is-mixed-between-proxmox-installations';
+
+function renderGuide(name: string): HTMLElement {
+ // Optional exact-base inputs exercise the former shipped advice without
+ // changing the committed candidate being checked in the proof VM.
+ const directory =
+ process.env.PULSE_PROXMOX_IDENTITY_GUIDE_INPUT_DIRECTORY ??
+ path.join(repoRoot, 'frontend-modern/public/docs');
+ const article = document.createElement('article');
+ article.innerHTML = renderDocMarkdown(readFileSync(path.join(directory, `${name}.md`), 'utf8'), name);
+ return article;
+}
+
+function sectionAt(article: HTMLElement, fragment: string): HTMLElement {
+ const heading = article.querySelector(fragment);
+ expect(heading).not.toBeNull();
+ const section = document.createElement('section');
+ for (let next = heading!.nextElementSibling; next; next = next.nextElementSibling) {
+ if (/^H[1-4]$/.test(next.tagName)) break;
+ section.appendChild(next.cloneNode(true));
+ }
+ return section;
+}
+
+const identitySection = (): HTMLElement => sectionAt(renderGuide('TROUBLESHOOTING'), identityFragment);
+const prose = (section: HTMLElement): string => (section.textContent ?? '').replace(/\s+/g, ' ').trim();
+
+describe('shipped cross-installation Proxmox identity guidance', () => {
+ it('distinguishes intended multi-installation support from cosmetic names and a verified repair', () => {
+ const article = renderGuide('CONFIGURATION');
+ const section = sectionAt(article, '#multiple-proxmox-installations');
+ const text = prose(section);
+ expect(text).toContain('intended to monitor multiple Proxmox clusters and standalone nodes');
+ expect(text).toContain('separate saved Proxmox connections');
+ expect(text).toContain('matching name or VMID alone does not identify the same resource');
+ expect(text).toContain('Neither that label nor a cluster member display name');
+ expect(text).toContain('repairs incorrect attribution');
+ expect(section.querySelector('a[href="#proxmox-cluster-node-display-names"]')).not.toBeNull();
+ expect(article.querySelector('#proxmox-cluster-node-display-names')).not.toBeNull();
+ });
+
+ it('keeps all three symptoms separate with a safe comparison and a limitation for each', () => {
+ const section = identitySection();
+ const rows = [...section.querySelectorAll('tbody tr')];
+ expect(rows).toHaveLength(3);
+ expect(rows.map((row) => row.querySelector('td')?.textContent)).toEqual([
+ 'Node errors',
+ 'Guest backup status',
+ 'Agent-backed Docker monitoring',
+ ]);
+ for (const row of rows) expect(row.querySelectorAll('td')).toHaveLength(3);
+ expect(prose(section)).toContain('One restored view does not establish that the others recovered');
+ expect(prose(rows[0])).toContain("that installation's own Proxmox view");
+ expect(prose(rows[1])).toContain('owning installation, guest type/VMID, datastore, namespace and backup time');
+ expect(prose(rows[2])).toContain('Last seen and Identity evidence');
+ expect(prose(rows[2])).toContain('Recent agent contact or a Healthy badge does not prove');
+ });
+
+ it('takes configuration, backup and generic agent-help entry points to the same real heading', () => {
+ for (const name of ['CONFIGURATION', 'PBS']) {
+ const link = renderGuide(name).querySelector<HTMLAnchorElement>(
+ `a[href="/docs/TROUBLESHOOTING${identityFragment}"]`,
+ );
+ expect(link?.textContent).toBe('cross-installation identity checks');
+ expect(renderGuide('TROUBLESHOOTING').querySelector(new URL(link!.href).hash)?.textContent).toBe(
+ 'Monitoring is mixed between Proxmox installations',
+ );
+ }
+ const agent = sectionAt(renderGuide('TROUBLESHOOTING'), '#agent-fleet-update-or-identity-issue');
+ expect(agent.querySelector(`a[href="${identityFragment}"]`)).not.toBeNull();
+ expect(prose(agent)).toContain('not an agent update or re-enrolment command');
+ });
+
+ it('reaches existing configuration, PBS and guest safety guidance rather than dead fragments', () => {
+ const section = identitySection();
+ for (const [name, fragment] of [
+ ['CONFIGURATION', '#multiple-proxmox-installations'],
+ ['PBS', '#backups-are-visible-but-coverage-says-unprotected'],
+ ['VM_DISK_MONITORING', '#backup-safety'],
+ ]) {
+ expect(section.querySelector(`a[href="/docs/${name}${fragment}"]`)).not.toBeNull();
+ expect(renderGuide(name).querySelector(fragment)).not.toBeNull();
+ }
+ expect(prose(section)).toContain('An OK backup does not prove thaw');
+ expect(prose(section)).toContain('a frozen or unresponsive guest needs the separate backup safety procedure');
+ });
+
+ it('offers independent native readings before asking for a minimal report', () => {
+ const section = identitySection();
+ const paragraphs = [...section.querySelectorAll('p')];
+ const workaround = paragraphs.findIndex((p) => prose(p).startsWith('While attribution is uncertain'));
+ const report = paragraphs.findIndex((p) => prose(p).startsWith('For a report'));
+ expect(workaround).toBeGreaterThanOrEqual(0);
+ expect(report).toBeGreaterThan(workaround);
+ expect(prose(paragraphs[workaround])).toContain("Docker host's existing runtime view");
+ expect(prose(paragraphs[workaround])).toContain('Do not make backup, restore or workload changes');
+ expect(prose(paragraphs[report])).toContain('which of the three symptoms remain');
+ expect(prose(paragraphs[report])).toContain('expected versus displayed origin');
+ });
+
+ it('preserves privacy while keeping equal native identities comparable across views', () => {
+ const text = prose(identitySection());
+ expect(text).toContain('consistent placeholders such as site-A, site-B, node-X and guest-100');
+ expect(text).toContain('say explicitly when the native names or VMIDs are equal');
+ expect(text).toContain('Keep actual addresses, hostnames, connection IDs and machine identities private');
+ expect(text).toContain('Do not share a full Agent Doctor report');
+ for (const withheld of ['API response', 'HAR export', 'token', 'unredacted screenshot']) {
+ expect(text).toContain(withheld);
+ }
+ });
+
+ it('uses existing evidence without destructive recovery, live probes or incident recreation', () => {
+ const section = identitySection();
+ expect(section.querySelector('pre')).toBeNull();
+ const text = prose(section);
+ for (const boundary of [
+ 'existing observations only',
+ 'keep that working setup',
+ 'rather than undoing the repair or repeating the addition',
+ 'Missing evidence is unknown',
+ 'Unavailable original evidence can be reported as unavailable',
+ 'do not recreate the incident',
+ 'Do not rename production nodes, change VMIDs or machine IDs',
+ 'delete/re-add connections, re-enrol agents, rotate tokens, restart services or clear History',
+ 'Do not run live diagnostics, guest-agent probes, a new backup or a restore as an identity test',
+ ]) {
+ expect(text).toContain(boundary);
+ }
+ });
+});
diff --git a/scripts/tests/test_pve_backup_troubleshooting_docs.py b/scripts/tests/test_pve_backup_troubleshooting_docs.py
index b871737f8f..1e8074245a 100644
--- a/scripts/tests/test_pve_backup_troubleshooting_docs.py
+++ b/scripts/tests/test_pve_backup_troubleshooting_docs.py
@@ -20,6 +20,42 @@ def backup_section():
class PVEBackupTroubleshootingDocsTest(unittest.TestCase):
+ def test_multi_installation_help_is_mirrored_and_keeps_cosmetic_names_separate(self):
+ for name in ("CONFIGURATION.md", "TROUBLESHOOTING.md", "PBS.md"):
+ with self.subTest(guide=name):
+ self.assertEqual((ROOT / "docs" / name).read_bytes(),
+ (ROOT / "frontend-modern/public/docs" / name).read_bytes())
+ config = (ROOT / "docs/CONFIGURATION.md").read_text()
+ section = config.split("### Multiple Proxmox installations\n", 1)[1].split("\n### ", 1)[0]
+ text = " ".join(section.split())
+ self.assertIn("intended to monitor multiple Proxmox clusters and standalone nodes", text)
+ self.assertIn("matching name or VMID alone does not identify the same resource", text)
+ self.assertIn("repairs incorrect attribution", text)
+ self.assertIn("TROUBLESHOOTING.md#monitoring-is-mixed-between-proxmox-installations", text)
+ # Static evidence of existing multi-instance intent, not native recovery.
+ helpers = (ROOT / "internal/monitoring/monitor_helpers.go").read_text()
+ self.assertIn('fmt.Sprintf("%s:%s:%d", instanceName, node, vmid)', helpers)
+ collector = (ROOT / "internal/monitoring/monitor_backups.go").read_text()
+ self.assertIn("vm.Instance() == instanceName", collector)
+ self.assertIn("vm.Instance == instanceName", collector)
+
+ def test_cross_installation_comparison_uses_existing_visible_agent_evidence(self):
+ trouble = (ROOT / "docs/TROUBLESHOOTING.md").read_text()
+ section = trouble.split("#### Monitoring is mixed between Proxmox installations\n", 1)[1].split("\n#### ", 1)[0]
+ text = " ".join(section.split())
+ self.assertNotIn("```", section)
+ for boundary in ("existing observations only", "One restored view does not establish",
+ "Keep actual addresses, hostnames, connection IDs and machine identities private",
+ "Do not share a full Agent Doctor report", "Do not rename production nodes",
+ "re-enrol agents", "Do not run live diagnostics, guest-agent probes"):
+ self.assertIn(boundary, text)
+ model = (ROOT / "frontend-modern/src/components/Settings/infrastructureAgentUpdateCommandsModel.ts").read_text()
+ for field in ("Connection: ${connection.id}", "Hostname: ${connection.agentIdentity.hostname}"):
+ self.assertIn(field, model)
+ page = (ROOT / "frontend-modern/src/components/Settings/InfrastructureAgentDoctorPage.tsx").read_text()
+ for label in ("Identity evidence", "Last seen", "Healthy"):
+ self.assertIn(label, page)
+
def test_shipped_help_matches_source(self):
self.assertEqual((ROOT / "docs" / GUIDE).read_bytes(),
(ROOT / "frontend-modern/public/docs" / GUIDE).read_bytes())