Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Keep migration trials isolated until monitoring handover

pulse-triage[bot]16e4b6161534
diff --git a/docs/MIGRATION.md b/docs/MIGRATION.md
index fac9166194..03c13fdaa6 100644
--- a/docs/MIGRATION.md
+++ b/docs/MIGRATION.md
@@ -44,9 +44,15 @@ an import error or skipped import does not confirm a successful transfer.
### Full-state recovery
If you need the same installation's history and excluded state, retain a
-consistent filesystem/volume backup as well. Stop Pulse before taking that
-backup; monitoring and alert delivery pause while it is stopped. Preserve every
-effective data path, the matching `.encryption.key`, audit data and signing key,
+consistent filesystem/volume backup as well. Before stopping Pulse, confirm
+that all state survives the stop: a container's writable layer or Kubernetes
+`emptyDir` is not retained storage when its container or pod is replaced. If
+persistence is unknown, preserve the state through your deployment's recovery
+procedure first; do not delete or scale down the server to find out. See the
+[Kubernetes storage and controller checks](KUBERNETES.md#proxmox-backup-safety).
+
+Stop Pulse before taking the consistent backup; monitoring and alert delivery
+pause while it is stopped. Preserve every effective data path, the matching `.encryption.key`, audit data and signing key,
environment files, external certificates, binaries and service/container
configuration together. Custom paths and organisation directories matter.
Copying a live database file alone can omit its sidecar files.
@@ -104,8 +110,12 @@ configuration import as proof of a complete multi-organisation migration.
is complete.
2. Install Pulse at the destination using the [installation guide](INSTALL.md).
Configure destination-local administrator access before importing. Keep the
- destination isolated from agents and notification destinations during the
- trial so two servers do not send duplicate alerts or remote actions.
+ destination isolated from **monitored provider APIs, agents, notification
+ destinations and remote-action targets before its first start or import**.
+ Imported credentials can start collection immediately, including QEMU Guest
+ Agent requests through Proxmox. API-only is not a passive trial. Keep this
+ isolation for a full-state recovery trial too; give it separate restored
+ storage, never the source's live data volume.
3. If the destination already has useful settings, export them first. Open
**Settings → System → Recovery → Restore Configuration**, select the file
and enter its passphrase. Import replaces the included configuration and
@@ -121,6 +131,37 @@ a password, API token or session cookie.
## Cut over and verify
+### Hand over monitoring once
+
+Validate destination-local sign-in and the imported settings while the trial
+is isolated. **Fresh provider readings belong after the handover below**, not
+while both servers can poll the same guests. Hiding rows, disabling alerts or
+turning off backup polling does not stop guest-agent collection.
+
+1. Plan an outage with independent monitoring and alert coverage. Do not cut
+ over during a backup, freeze/thaw or an unresponsive-guest incident; use
+ [Backup safety](VM_DISK_MONITORING.md#backup-safety) if those states are
+ uncertain. An OK backup alone does not establish guest recovery.
+2. Using the deployment's supported controls, stop **every source Pulse server**
+ that monitors the transferred resources, not just its agents or browser.
+ First confirm retained storage as above, record prior service/replica and
+ automatic-update states, and prevent an updater or controller from restarting
+ it. Require a confirmed normal shutdown; a deleted pod, paused log display,
+ failed stop or unknown state is not a completed handover. Do not force-kill
+ or delete state to satisfy this check.
+3. A stopped server does not cancel a guest-agent request already issued.
+ Confirm existing guest/backup operations have ended normally through
+ established platform/workload checks, not new guest-agent probes or an
+ arbitrary delay. If this is unknown, keep the destination isolated and
+ resolve it before enabling collection.
+4. Only then allow the destination to reach the intended providers and retarget
+ agents. Keep the source stopped and its automation paused during the checks
+ below. For full-state recovery, reconcile retained pending notifications and
+ actions before allowing delivery or execution; a configuration-only import
+ does not contain those queues.
+
+### Check the destination
+
- **Login and permissions:** verify destination-local administrator access and
any imported SSO provider, group mappings and callback address. Re-create
custom roles and supply external CA/certificate files if needed. Environment
@@ -146,9 +187,15 @@ a password, API token or session cookie.
Relay is no longer sold; do not enable it as a new migration route.
Retire the old installation only after these checks succeed. Keep the private,
-consistent pre-move backup for recovery. If validation fails, stop the trial
-destination before returning traffic to the intact source; check agent URLs
-and avoid two active writers. There is no guaranteed five-minute recovery time.
+consistent pre-move backup for recovery. If validation fails, keep the source
+intact and isolate the destination from providers, agents, notifications and
+action targets. Confirm the destination is
+stopped and cannot be automatically restarted before returning traffic or
+restoring the source's recorded prior-active controls. If shutdown or in-flight
+operations are uncertain, do not enable a second poller. Preserve the failed
+destination for diagnosis; do not copy its changed data over the source or
+re-import merely to retry. Check agent URLs and avoid two active writers.
+There is no guaranteed five-minute recovery time.
## Troubleshooting
diff --git a/frontend-modern/public/docs/MIGRATION.md b/frontend-modern/public/docs/MIGRATION.md
index fac9166194..03c13fdaa6 100644
--- a/frontend-modern/public/docs/MIGRATION.md
+++ b/frontend-modern/public/docs/MIGRATION.md
@@ -44,9 +44,15 @@ an import error or skipped import does not confirm a successful transfer.
### Full-state recovery
If you need the same installation's history and excluded state, retain a
-consistent filesystem/volume backup as well. Stop Pulse before taking that
-backup; monitoring and alert delivery pause while it is stopped. Preserve every
-effective data path, the matching `.encryption.key`, audit data and signing key,
+consistent filesystem/volume backup as well. Before stopping Pulse, confirm
+that all state survives the stop: a container's writable layer or Kubernetes
+`emptyDir` is not retained storage when its container or pod is replaced. If
+persistence is unknown, preserve the state through your deployment's recovery
+procedure first; do not delete or scale down the server to find out. See the
+[Kubernetes storage and controller checks](KUBERNETES.md#proxmox-backup-safety).
+
+Stop Pulse before taking the consistent backup; monitoring and alert delivery
+pause while it is stopped. Preserve every effective data path, the matching `.encryption.key`, audit data and signing key,
environment files, external certificates, binaries and service/container
configuration together. Custom paths and organisation directories matter.
Copying a live database file alone can omit its sidecar files.
@@ -104,8 +110,12 @@ configuration import as proof of a complete multi-organisation migration.
is complete.
2. Install Pulse at the destination using the [installation guide](INSTALL.md).
Configure destination-local administrator access before importing. Keep the
- destination isolated from agents and notification destinations during the
- trial so two servers do not send duplicate alerts or remote actions.
+ destination isolated from **monitored provider APIs, agents, notification
+ destinations and remote-action targets before its first start or import**.
+ Imported credentials can start collection immediately, including QEMU Guest
+ Agent requests through Proxmox. API-only is not a passive trial. Keep this
+ isolation for a full-state recovery trial too; give it separate restored
+ storage, never the source's live data volume.
3. If the destination already has useful settings, export them first. Open
**Settings → System → Recovery → Restore Configuration**, select the file
and enter its passphrase. Import replaces the included configuration and
@@ -121,6 +131,37 @@ a password, API token or session cookie.
## Cut over and verify
+### Hand over monitoring once
+
+Validate destination-local sign-in and the imported settings while the trial
+is isolated. **Fresh provider readings belong after the handover below**, not
+while both servers can poll the same guests. Hiding rows, disabling alerts or
+turning off backup polling does not stop guest-agent collection.
+
+1. Plan an outage with independent monitoring and alert coverage. Do not cut
+ over during a backup, freeze/thaw or an unresponsive-guest incident; use
+ [Backup safety](VM_DISK_MONITORING.md#backup-safety) if those states are
+ uncertain. An OK backup alone does not establish guest recovery.
+2. Using the deployment's supported controls, stop **every source Pulse server**
+ that monitors the transferred resources, not just its agents or browser.
+ First confirm retained storage as above, record prior service/replica and
+ automatic-update states, and prevent an updater or controller from restarting
+ it. Require a confirmed normal shutdown; a deleted pod, paused log display,
+ failed stop or unknown state is not a completed handover. Do not force-kill
+ or delete state to satisfy this check.
+3. A stopped server does not cancel a guest-agent request already issued.
+ Confirm existing guest/backup operations have ended normally through
+ established platform/workload checks, not new guest-agent probes or an
+ arbitrary delay. If this is unknown, keep the destination isolated and
+ resolve it before enabling collection.
+4. Only then allow the destination to reach the intended providers and retarget
+ agents. Keep the source stopped and its automation paused during the checks
+ below. For full-state recovery, reconcile retained pending notifications and
+ actions before allowing delivery or execution; a configuration-only import
+ does not contain those queues.
+
+### Check the destination
+
- **Login and permissions:** verify destination-local administrator access and
any imported SSO provider, group mappings and callback address. Re-create
custom roles and supply external CA/certificate files if needed. Environment
@@ -146,9 +187,15 @@ a password, API token or session cookie.
Relay is no longer sold; do not enable it as a new migration route.
Retire the old installation only after these checks succeed. Keep the private,
-consistent pre-move backup for recovery. If validation fails, stop the trial
-destination before returning traffic to the intact source; check agent URLs
-and avoid two active writers. There is no guaranteed five-minute recovery time.
+consistent pre-move backup for recovery. If validation fails, keep the source
+intact and isolate the destination from providers, agents, notifications and
+action targets. Confirm the destination is
+stopped and cannot be automatically restarted before returning traffic or
+restoring the source's recorded prior-active controls. If shutdown or in-flight
+operations are uncertain, do not enable a second poller. Preserve the failed
+destination for diagnosis; do not copy its changed data over the source or
+re-import merely to retry. Check agent URLs and avoid two active writers.
+There is no guaranteed five-minute recovery time.
## Troubleshooting
diff --git a/frontend-modern/src/features/docs/__tests__/migrationHandoverSafetyDocs.test.ts b/frontend-modern/src/features/docs/__tests__/migrationHandoverSafetyDocs.test.ts
new file mode 100644
index 0000000000..b6ee74aae3
--- /dev/null
+++ b/frontend-modern/src/features/docs/__tests__/migrationHandoverSafetyDocs.test.ts
@@ -0,0 +1,110 @@
+import { readFileSync } from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { describe, expect, it } from 'vitest';
+import { renderDocMarkdown } from '../docMarkdown';
+
+const frontendRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../..');
+const repoRoot = path.resolve(frontendRoot, '..');
+const readDoc = (name: string) => readFileSync(path.join(repoRoot, 'docs', `${name}.md`), 'utf8');
+
+function articleFor(name: string) {
+ const article = document.createElement('article');
+ article.innerHTML = renderDocMarkdown(readDoc(name), name);
+ return article;
+}
+
+const textOf = (element: Element) => element.textContent!.replace(/\s+/g, ' ');
+
+function handover() {
+ const article = articleFor('MIGRATION');
+ const heading = article.querySelector('#hand-over-monitoring-once');
+ expect(heading).not.toBeNull();
+ const section = document.createElement('section');
+ for (
+ let node = heading!.nextElementSibling;
+ node && !/^H[23]$/.test(node.tagName);
+ node = node.nextElementSibling
+ ) {
+ section.append(node.cloneNode(true));
+ }
+ return section;
+}
+
+describe('Migration monitoring handover safety', () => {
+ it('isolates provider access before starting either kind of recovery trial', () => {
+ const article = articleFor('MIGRATION');
+ const step = article.querySelector(
+ '#transfer-the-included-configuration + ol > li:nth-child(2)',
+ );
+ expect(step).not.toBeNull();
+ const text = textOf(step!);
+ expect(text).toContain('monitored provider APIs, agents, notification destinations');
+ expect(text).toContain('remote-action targets before its first start or import');
+ expect(text).toContain('QEMU Guest Agent requests through Proxmox');
+ expect(text).toContain('API-only is not a passive trial');
+ expect(text).toContain('full-state recovery trial too');
+ expect(text).toContain("never the source's live data volume");
+ });
+
+ it('requires retained storage before a full-state backup stop', () => {
+ const article = articleFor('MIGRATION');
+ const text = textOf(article);
+ expect(text).toContain('Before stopping Pulse, confirm that all state survives the stop');
+ expect(text).toContain('emptyDir');
+ expect(text).toContain('If persistence is unknown, preserve the state');
+ expect(text).toContain('do not delete or scale down the server to find out');
+ expect(text.indexOf('Before stopping Pulse')).toBeLessThan(
+ text.indexOf('Stop Pulse before taking the consistent backup'),
+ );
+ });
+
+ it('places the four-step handover before fresh destination checks', () => {
+ const article = articleFor('MIGRATION');
+ const headings = [...article.querySelectorAll('h3')].map((heading) => heading.id);
+ expect(headings.indexOf('hand-over-monitoring-once')).toBeLessThan(
+ headings.indexOf('check-the-destination'),
+ );
+ const section = handover();
+ expect(textOf(section)).toContain('Fresh provider readings belong after the handover');
+ expect(textOf(section)).toContain('turning off backup polling does not stop');
+ const steps = [...section.querySelectorAll('ol > li')].map(textOf);
+ expect(steps).toHaveLength(4);
+ expect(steps[0]).toContain('Do not cut over during a backup, freeze/thaw');
+ expect(steps[1]).toContain('every source Pulse server');
+ expect(steps[1]).toContain('prevent an updater or controller from restarting it');
+ expect(steps[1]).toContain('Require a confirmed normal shutdown');
+ expect(steps[2]).toContain('does not cancel a guest-agent request already issued');
+ expect(steps[2]).toContain('not new guest-agent probes or an arbitrary delay');
+ expect(steps[2]).toContain('keep the destination isolated');
+ expect(steps[3]).toContain('Only then allow the destination');
+ expect(steps[3]).toContain('reconcile retained pending notifications and actions');
+ expect(section.querySelector('pre')).toBeNull();
+ });
+
+ it('keeps rollback single-owner and preserves both installations', () => {
+ const text = textOf(articleFor('MIGRATION'));
+ expect(text).toContain(
+ 'Confirm the destination is stopped and cannot be automatically restarted',
+ );
+ expect(text).toContain("restoring the source's recorded prior-active controls");
+ expect(text).toContain('do not enable a second poller');
+ expect(text).toContain('do not copy its changed data over the source');
+ expect(text).toContain('re-import merely to retry');
+ });
+
+ it('routes storage and incident safeguards to shipped, existing headings', () => {
+ const article = articleFor('MIGRATION');
+ for (const [name, fragment] of [
+ ['KUBERNETES', 'proxmox-backup-safety'],
+ ['VM_DISK_MONITORING', 'backup-safety'],
+ ]) {
+ const link = article.querySelector(`a[href="/docs/${name}#${fragment}"]`);
+ expect(link?.hasAttribute('data-doc-link')).toBe(true);
+ expect(articleFor(name).querySelector(`#${fragment}`)).not.toBeNull();
+ }
+ expect(readFileSync(path.join(frontendRoot, 'public/docs/MIGRATION.md'), 'utf8')).toBe(
+ readDoc('MIGRATION'),
+ );
+ });
+});