Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Separate failed-upgrade attempts from observed recovery in issue intake
pulse-triage[bot]6809ca2b2317
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.ymlindex a19491d432..6bf10731f5 100644--- a/.github/ISSUE_TEMPLATE/bug_report.yml+++ b/.github/ISSUE_TEMPLATE/bug_report.yml@@ -64,7 +64,7 @@ body:id: pulse_versionattributes:label: Pulse version- description: Running version for an existing install; for a failed install, the exact version or release asset attempted, or "unknown" if it was not shown.+ description: Version running when the problem occurred, or "unknown" if unavailable. For a failed install, give the exact version or release asset attempted. Keep an interrupted upgrade's target and later recovered version separate in Upgrade attempt and recovery below.placeholder: v6.x.y, release asset name, or unknownvalidations:required: true@@ -124,11 +124,27 @@ body:id: installer_sourceattributes:label: Installer or helper source- description: For a failed install, give the public installer/helper URL or command name, if known. Do not paste a command containing a token or other secret.+ description: For a failed install or upgrade, give the public installer/helper URL or command name, if known. Do not paste a command containing a token or other secret.placeholder: Official Pulse installer, helper URL, or command namevalidations:required: false+ - type: textarea+ id: upgrade_context+ attributes:+ label: Upgrade attempt and recovery+ description: |+ For an interrupted or failed upgrade only: give the starting version, intended version or asset, route (in-app, automatic timer, manual installer or helper), original time and timezone, and last recorded updater/installer result or exit code. Distinguish Pulse stopping from the whole LXC, VM or host stopping. Say what recovery you already performed and which version and state you observed afterwards.+ A verified download signature is not a completed upgrade; a later boot does not explain the stop. Today's running version does not establish the version at failure. Use existing observations; blank or "unknown" is valid. If already available, include only relevant, locally reviewed and redacted updater and host stop records in the evidence field, not full journals.+ Do not rerun the update, reboot, restore, run Diagnostics or delete rollback backups just to complete this report. Keep backups private and preserve the original evidence.+ placeholder: |+ Attempt: starting version → intended version; route; time and timezone+ Last recorded result: error or exit code, or unknown+ Observed stop: Pulse service / whole LXC, VM or host / unknown+ Recovery already performed: none or original steps; version/state afterwards or unknown+ validations:+ required: false+- type: textareaid: logsattributes:@@ -154,7 +170,7 @@ body:attributes:label: Confirmationsoptions:- - label: I gave the running Pulse version, or for an install that never started, the attempted version or "unknown"; for a running container I also gave its image tag or digest.+ - label: I gave the Pulse version at the failure or "unknown" if unavailable; for an install that never started, the attempted version or "unknown"; for a running container I also gave its image tag or digest.required: true- label: I searched existing issues for duplicates.required: truediff --git a/.github/ISSUE_TEMPLATE/v6_rc_feedback.yml b/.github/ISSUE_TEMPLATE/v6_rc_feedback.ymlindex 0bb07397a3..4a020f03a5 100644--- a/.github/ISSUE_TEMPLATE/v6_rc_feedback.yml+++ b/.github/ISSUE_TEMPLATE/v6_rc_feedback.yml@@ -34,7 +34,7 @@ body:id: pulse_versionattributes:label: Pulse version- description: Running version for an existing install; for a failed install, the exact version or release asset attempted, or "unknown" if it was not shown.+ description: Version running when the problem occurred, or "unknown" if unavailable. For a failed install, give the exact version or release asset attempted. Keep an interrupted upgrade's target and later recovered version separate in Upgrade attempt and recovery below.placeholder: v6.x.y-rc.N, release asset name, or unknownvalidations:required: true@@ -97,11 +97,27 @@ body:id: installer_sourceattributes:label: Installer or helper source- description: For a failed install, give the public installer/helper URL or command name, if known. Do not paste a command containing a token or other secret.+ description: For a failed install or upgrade, give the public installer/helper URL or command name, if known. Do not paste a command containing a token or other secret.placeholder: Official Pulse installer, helper URL, or command namevalidations:required: false+ - type: textarea+ id: upgrade_context+ attributes:+ label: Upgrade attempt and recovery+ description: |+ For an interrupted or failed upgrade only: give the starting version, intended version or asset, route (in-app, automatic timer, manual installer or helper), original time and timezone, and last recorded updater/installer result or exit code. Distinguish Pulse stopping from the whole LXC, VM or host stopping. Say what recovery you already performed and which version and state you observed afterwards.+ A verified download signature is not a completed upgrade; a later boot does not explain the stop. Today's running version does not establish the version at failure. Use existing observations; blank or "unknown" is valid. If already available, include only relevant, locally reviewed and redacted updater and host stop records in the evidence field, not full journals.+ Do not rerun the update, reboot, restore, run Diagnostics or delete rollback backups just to complete this report. Keep backups private and preserve the original evidence.+ placeholder: |+ Attempt: starting version → intended version; route; time and timezone+ Last recorded result: error or exit code, or unknown+ Observed stop: Pulse service / whole LXC, VM or host / unknown+ Recovery already performed: none or original steps; version/state afterwards or unknown+ validations:+ required: false+- type: inputid: environmentattributes:@@ -194,7 +210,7 @@ body:attributes:label: Confirmationsoptions:- - label: I gave the running Pulse version, or for an install that never started, the attempted version or "unknown"; for a running container I also gave its Pulse image tag or digest.+ - label: I gave the Pulse version at the failure or "unknown" if unavailable; for an install that never started, the attempted version or "unknown"; for a running container I also gave its Pulse image tag or digest.required: true- label: I searched existing issues and the pinned v6 pre-release feedback hub first.required: truediff --git a/.github/scripts/issue-version-triage.test.cjs b/.github/scripts/issue-version-triage.test.cjsindex 55cc809c50..d0e4a8cab8 100644--- a/.github/scripts/issue-version-triage.test.cjs+++ b/.github/scripts/issue-version-triage.test.cjs@@ -500,6 +500,32 @@ test("bug and pre-release forms accept truthful evidence for installs that never}});+test("both bug forms keep an optional failed-upgrade timeline without unsafe collection", () => {+ for (const name of ["bug_report.yml", "v6_rc_feedback.yml"]) {+ const form = fs.readFileSync(path.resolve(__dirname, "../ISSUE_TEMPLATE", name), "utf8");+ const timeline = form.split(" id: upgrade_context\n")[1]?.split(" - type: ")[0];+ assert.ok(timeline, name);+ assert.match(timeline, /label: Upgrade attempt and recovery/);+ assert.match(timeline, /required: false/);+ for (const distinction of [+ "starting version, intended version or asset", "original time and timezone",+ "last recorded updater/installer result or exit code",+ "whole LXC, VM or host stopping", "recovery you already performed",+ "A verified download signature is not a completed upgrade",+ "a later boot does not explain the stop",+ "Today's running version does not establish the version at failure",+ 'blank or "unknown" is valid', "not full journals",+ "Do not rerun the update, reboot, restore, run Diagnostics or delete rollback backups",+ "Keep backups private and preserve the original evidence",+ ]) {+ assert.ok(timeline.includes(distinction), `${name}: ${distinction}`);+ }+ const version = form.split(" id: pulse_version\n")[1].split(" - type: ")[0];+ assert.match(version, /Version running when the problem occurred, or "unknown"/);+ assert.match(version, /target and later recovered version separate/);+ }+});+test("bug and pre-release forms accept unsafe one-off failures without a second run", () => {const templateDir = path.resolve(__dirname, "../ISSUE_TEMPLATE");for (const name of ["bug_report.yml", "v6_rc_feedback.yml"]) {@@ -941,6 +967,42 @@ test("stable-to-preview intake keeps baseline, agent and platform versions out o}});+test("interrupted-upgrade targets and recovered versions cannot supply the failing version", async () => {+ for (const [failing, expected] of [+ ["6.4.5", "affects-6.4.5"],+ ["unknown", "needs-version-info"],+ ]) {+ for (const formTitle of ["[Bug]:", "[v6 pre-release]:"]) {+ const { github, calls } = createGithub({ latestVersion: "6.5.0" });+ const bugForm = formTitle === "[Bug]:";+ const issue = {+ number: 2785,+ title: `${formTitle} interrupted upgrade to 6.5.0`,+ body: [+ ...(!bugForm ? ["### Feedback type", "Bug / regression"] : []),+ "### Pulse version", failing,+ "### Last known working Pulse version", "6.4.1",+ "### Agent version", "6.5.0",+ "### Upgrade attempt and recovery",+ "Attempt: 6.4.5 → 6.5.0; automatic timer; 6 October 04:20 UTC",+ "Last recorded result: signature verified, then Broken pipe; exit unknown",+ "Observed stop: whole LXC; stop time/cause unknown",+ "Recovery already performed: booted 7 October; current version 6.5.0",+ "### Additional actionable topics", "None",+ ].join("\n\n"),+ // GitHub applies the bug form's label; the preview form declares its+ // classification in Feedback type instead. Reproduce both inputs.+ labels: bugForm ? [{ name: "bug" }] : [],+ };+ await syncLabels({ github, context: createContext({ issue }), core: createCore() });+ assert.deepEqual(calls.addLabels.flatMap((call) => call.labels).sort(),+ (bugForm ? [expected] : [expected, "bug"]).sort());+ assert.equal(calls.createComment.length, 0);+ assert.equal(calls.getLatestRelease.length, 0);+ }+ }+});+test("delayed events classify the current report and labels, not an old version", async () => {const eventIssue = {number: 2800,diff --git a/docs/ISSUE_TRIAGE.md b/docs/ISSUE_TRIAGE.mdindex 0c91f67197..7330851173 100644--- a/docs/ISSUE_TRIAGE.md+++ b/docs/ISSUE_TRIAGE.md@@ -41,6 +41,24 @@ An API-only connection can have no agent. Accept "unknown" or an omitted optionafield, and use existing evidence; do not ask for a downgrade, restart, reinstallor re-enrolment to obtain version context. Existing reports need no refile.+An interrupted upgrade needs a timeline, not one inferred version. The optional+**Upgrade attempt and recovery** field separates the starting version, intended+version or asset, route, original time and timezone, last recorded updater or+installer result, what stopped, and recovery already performed. Keep the version+running at failure separate from the version observed after recovery; accept+unknowns and use evidence already supplied anywhere in the thread.++A verified download signature establishes artifact authenticity, not completed+installation, successful rollback or a healthy running service. `Broken pipe`+alone does not establish why a service or its host stopped. A later boot marks+a new run, not the stop's time or cause. Distinguish Pulse stopping from its+whole LXC, VM or host stopping, using an existing host stop record only when+available and consequential. Ask for only the missing distinction, not full+journals or another version already supplied. Do not rerun the update, reboot,+restore, run Diagnostics or delete rollback backups just to complete a report;+keep backups private and preserve the original evidence. Version labels are+intake metadata, not proof of the failing executable or installed recovery.+The optional **OS / environment** field in both bug forms also distinguisheswhere readings come from: the platform API, a Pulse agent, or both. Retain theaffected platform release, including TrueNAS SCALE versus CORE, separately fromdiff --git a/frontend-modern/public/docs/ISSUE_TRIAGE.md b/frontend-modern/public/docs/ISSUE_TRIAGE.mdindex 0c91f67197..7330851173 100644--- a/frontend-modern/public/docs/ISSUE_TRIAGE.md+++ b/frontend-modern/public/docs/ISSUE_TRIAGE.md@@ -41,6 +41,24 @@ An API-only connection can have no agent. Accept "unknown" or an omitted optionafield, and use existing evidence; do not ask for a downgrade, restart, reinstallor re-enrolment to obtain version context. Existing reports need no refile.+An interrupted upgrade needs a timeline, not one inferred version. The optional+**Upgrade attempt and recovery** field separates the starting version, intended+version or asset, route, original time and timezone, last recorded updater or+installer result, what stopped, and recovery already performed. Keep the version+running at failure separate from the version observed after recovery; accept+unknowns and use evidence already supplied anywhere in the thread.++A verified download signature establishes artifact authenticity, not completed+installation, successful rollback or a healthy running service. `Broken pipe`+alone does not establish why a service or its host stopped. A later boot marks+a new run, not the stop's time or cause. Distinguish Pulse stopping from its+whole LXC, VM or host stopping, using an existing host stop record only when+available and consequential. Ask for only the missing distinction, not full+journals or another version already supplied. Do not rerun the update, reboot,+restore, run Diagnostics or delete rollback backups just to complete a report;+keep backups private and preserve the original evidence. Version labels are+intake metadata, not proof of the failing executable or installed recovery.+The optional **OS / environment** field in both bug forms also distinguisheswhere readings come from: the platform API, a Pulse agent, or both. Retain theaffected platform release, including TrueNAS SCALE versus CORE, separately fromdiff --git a/scripts/tests/test_failed_upgrade_intake_docs.py b/scripts/tests/test_failed_upgrade_intake_docs.pynew file mode 100644index 0000000000..75bbe7e3a5--- /dev/null+++ b/scripts/tests/test_failed_upgrade_intake_docs.py@@ -0,0 +1,59 @@+#!/usr/bin/env python3+"""Guard passive failed-upgrade intake, not installer or workload recovery."""++from pathlib import Path+import unittest+++ROOT = Path(__file__).resolve().parents[2]+++class FailedUpgradeIntakeDocsTest(unittest.TestCase):+ def guidance(self):+ text = (ROOT / "docs/ISSUE_TRIAGE.md").read_text()+ start = "An interrupted upgrade needs a timeline, not one inferred version."+ self.assertEqual(text.count(start), 1)+ return " ".join(text.split(start, 1)[1].split("\nThe optional", 1)[0].split())++ def test_shipped_triage_guide_matches_source(self):+ self.assertEqual((ROOT / "docs/ISSUE_TRIAGE.md").read_bytes(),+ (ROOT / "frontend-modern/public/docs/ISSUE_TRIAGE.md").read_bytes())++ def test_attempt_failure_and_recovery_are_separate_observations(self):+ guide = self.guidance()+ for distinction in (+ "starting version, intended version or asset, route, original time and timezone",+ "last recorded updater or installer result, what stopped",+ "version running at failure separate from the version observed after recovery",+ "accept unknowns and use evidence already supplied anywhere in the thread",+ "Version labels are intake metadata, not proof of the failing executable",+ ):+ with self.subTest(distinction=distinction):+ self.assertIn(distinction, guide)++ def test_signature_and_later_boot_do_not_establish_completion_or_stop_cause(self):+ guide = self.guidance()+ for limit in (+ "artifact authenticity, not completed installation, successful rollback",+ "`Broken pipe` alone does not establish why a service or its host stopped",+ "A later boot marks a new run, not the stop's time or cause",+ "Distinguish Pulse stopping from its whole LXC, VM or host stopping",+ ):+ with self.subTest(limit=limit):+ self.assertIn(limit, guide)++ def test_collection_does_not_replay_incident_or_destroy_recovery_evidence(self):+ guide = self.guidance()+ for boundary in (+ "existing host stop record only when available and consequential",+ "Ask for only the missing distinction, not full journals",+ "another version already supplied",+ "Do not rerun the update, reboot, restore, run Diagnostics or delete rollback backups",+ "keep backups private and preserve the original evidence",+ ):+ with self.subTest(boundary=boundary):+ self.assertIn(boundary, guide)+++if __name__ == "__main__":+ unittest.main()