Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Explain RAID spare alerts without changing the array
pulse-triage[bot]55ed0403645b
diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdindex 7255d4c3d8..35349b4131 100644--- a/docs/TROUBLESHOOTING.md+++ b/docs/TROUBLESHOOTING.md@@ -43,6 +43,10 @@ establish safe recovery.[server, backup inventory and History](PBS.md#pbs-is-active-but-missing-from-monitoring).An Active API connection does not establish a visible PBS resource; installingan agent does not prove API recovery.+- **RAID is reported degraded despite a spare:** compare+ [required members, active members and spares](TROUBLESHOOTING.md#raid-array-is-degraded-with-a-spare).+ A spare-inclusive total is not the required member count; do not change the+ array just to clear a Pulse alert.- **Pulse itself uses more memory, CPU or disk writes:** use[memory baseline and growth checks](TROUBLESHOOTING.md#memory-use-keeps-growing)or [CPU and write-rate checks](TROUBLESHOOTING.md#excessive-cpu-writes-or-database-growth).@@ -338,6 +342,39 @@ restore, not by overwriting today's history. Follow### Monitoring Data+#### RAID array is degraded with a spare++For a Linux MD RAID alert, compare the same array's **existing** native status+with Pulse's reading and observation time. In `mdadm` output, **Raid Devices**+is the required member count; **Total Devices** can include a spare. A spare+does not replace an active member until it has joined the array.++| Existing observation | Meaning |+| --- | --- |+| `clean`; Raid Devices **4**, Active **4**, Total **5**, Working **5**, Spare **1**, Failed **0** | Four required members are active, with one spare. The total of five does not by itself mean a member is missing. |+| Raid Devices **4**, Active **3**, with a spare still present | A required member is missing. Do not subtract the spare from the required count or treat it as restored redundancy. |+| Failed devices, a degraded/unavailable state, or recovery/reshape in progress | These are separate health or recovery observations. A spare or the word `clean` alone does not establish that recovery finished. |+| Missing counts, an old reading or different array identities | The comparison is unavailable or unlike-for-like, not a healthy result. |++Pulse **v6.5.0** includes the [spare-count correction reported in #2369](https://github.com/rcourtman/Pulse/issues/2369)+for the clean four-member-plus-spare case, including reports from older+collectors. That does not establish that every RAID alert has this cause or+that an affected installation has recovered. A healthy count also does not+verify data integrity, backup completeness or restore readiness.++For genuine member loss or recovery, use the storage platform's established+recovery procedure and independent monitoring. Keep RAID alerts enabled; do+not remove a spare, fail a member, assemble/rebuild an array, start a scrub,+reboot or change collection settings just to test or silence Pulse. No new+`mdadm` command, Diagnostics run or induced failure is required to file a report.++For help, retain the original alert/time, running Pulse and collector versions+where known, collection path, and the already available state and counts for+one matching array. A missing collector version or native observation can be+reported as unknown. Use consistent private aliases for the host and array;+keep UUIDs, serial numbers, private paths and full command output out of public+reports. Do not repeat an array operation to obtain evidence.+#### Primary IP shows a Podman or Docker bridge addressOn a Proxmox VM or LXC, **Primary IP** is Pulse's selected address, not proof ofdiff --git a/frontend-modern/public/docs/TROUBLESHOOTING.md b/frontend-modern/public/docs/TROUBLESHOOTING.mdindex 7255d4c3d8..35349b4131 100644--- a/frontend-modern/public/docs/TROUBLESHOOTING.md+++ b/frontend-modern/public/docs/TROUBLESHOOTING.md@@ -43,6 +43,10 @@ establish safe recovery.[server, backup inventory and History](PBS.md#pbs-is-active-but-missing-from-monitoring).An Active API connection does not establish a visible PBS resource; installingan agent does not prove API recovery.+- **RAID is reported degraded despite a spare:** compare+ [required members, active members and spares](TROUBLESHOOTING.md#raid-array-is-degraded-with-a-spare).+ A spare-inclusive total is not the required member count; do not change the+ array just to clear a Pulse alert.- **Pulse itself uses more memory, CPU or disk writes:** use[memory baseline and growth checks](TROUBLESHOOTING.md#memory-use-keeps-growing)or [CPU and write-rate checks](TROUBLESHOOTING.md#excessive-cpu-writes-or-database-growth).@@ -338,6 +342,39 @@ restore, not by overwriting today's history. Follow### Monitoring Data+#### RAID array is degraded with a spare++For a Linux MD RAID alert, compare the same array's **existing** native status+with Pulse's reading and observation time. In `mdadm` output, **Raid Devices**+is the required member count; **Total Devices** can include a spare. A spare+does not replace an active member until it has joined the array.++| Existing observation | Meaning |+| --- | --- |+| `clean`; Raid Devices **4**, Active **4**, Total **5**, Working **5**, Spare **1**, Failed **0** | Four required members are active, with one spare. The total of five does not by itself mean a member is missing. |+| Raid Devices **4**, Active **3**, with a spare still present | A required member is missing. Do not subtract the spare from the required count or treat it as restored redundancy. |+| Failed devices, a degraded/unavailable state, or recovery/reshape in progress | These are separate health or recovery observations. A spare or the word `clean` alone does not establish that recovery finished. |+| Missing counts, an old reading or different array identities | The comparison is unavailable or unlike-for-like, not a healthy result. |++Pulse **v6.5.0** includes the [spare-count correction reported in #2369](https://github.com/rcourtman/Pulse/issues/2369)+for the clean four-member-plus-spare case, including reports from older+collectors. That does not establish that every RAID alert has this cause or+that an affected installation has recovered. A healthy count also does not+verify data integrity, backup completeness or restore readiness.++For genuine member loss or recovery, use the storage platform's established+recovery procedure and independent monitoring. Keep RAID alerts enabled; do+not remove a spare, fail a member, assemble/rebuild an array, start a scrub,+reboot or change collection settings just to test or silence Pulse. No new+`mdadm` command, Diagnostics run or induced failure is required to file a report.++For help, retain the original alert/time, running Pulse and collector versions+where known, collection path, and the already available state and counts for+one matching array. A missing collector version or native observation can be+reported as unknown. Use consistent private aliases for the host and array;+keep UUIDs, serial numbers, private paths and full command output out of public+reports. Do not repeat an array operation to obtain evidence.+#### Primary IP shows a Podman or Docker bridge addressOn a Proxmox VM or LXC, **Primary IP** is Pulse's selected address, not proof ofdiff --git a/scripts/tests/test_raid_troubleshooting_docs.py b/scripts/tests/test_raid_troubleshooting_docs.pynew file mode 100644index 0000000000..44225b057e--- /dev/null+++ b/scripts/tests/test_raid_troubleshooting_docs.py@@ -0,0 +1,77 @@+#!/usr/bin/env python3+"""Preserve spare-aware RAID help without implying storage recovery.++Browser verification covers navigation and the shipped renderer separately.+These checks guard the documented distinctions and avoid unsafe recipes.+"""++from pathlib import Path+import unittest+++ROOT = Path(__file__).resolve().parents[2]+HEADING = "#### RAID array is degraded with a spare\n"+++class RAIDTroubleshootingDocsTest(unittest.TestCase):+ def section(self):+ guide = (ROOT / "docs/TROUBLESHOOTING.md").read_text(encoding="utf-8")+ self.assertEqual(guide.count(HEADING), 1)+ return " ".join(guide.split(HEADING, 1)[1].split("\n#### ", 1)[0].split())++ def test_required_members_are_not_spare_inclusive_totals(self):+ text = self.section()+ for distinction in (+ "same array's **existing** native status",+ "observation time",+ "**Raid Devices** is the required member count",+ "**Total Devices** can include a spare",+ "Active **4**, Total **5**, Working **5**, Spare **1**, Failed **0**",+ "Raid Devices **4**, Active **3**",+ "Do not subtract the spare from the required count",+ "Missing counts, an old reading or different array identities",+ ):+ with self.subTest(distinction=distinction):+ self.assertIn(distinction, text)++ def test_correction_is_not_every_alert_or_installed_recovery(self):+ text = self.section()+ for limit in (+ "Pulse **v6.5.0**",+ "reports from older collectors",+ "does not establish that every RAID alert has this cause",+ "an affected installation has recovered",+ "data integrity, backup completeness or restore readiness",+ "Failed devices, a degraded/unavailable state, or recovery/reshape",+ ):+ with self.subTest(limit=limit):+ self.assertIn(limit, text)++ def test_diagnosis_does_not_require_mutation_or_raw_public_output(self):+ text = self.section()+ for boundary in (+ "independent monitoring",+ "Keep RAID alerts enabled",+ "do not remove a spare",+ "fail a member",+ "assemble/rebuild an array",+ "start a scrub",+ "reboot or change collection settings",+ "No new `mdadm` command, Diagnostics run or induced failure",+ "reported as unknown",+ "consistent private aliases",+ "UUIDs, serial numbers, private paths and full command output",+ "Do not repeat an array operation",+ ):+ with self.subTest(boundary=boundary):+ self.assertIn(boundary, text)+ self.assertNotIn("```", text, "existing observations, not a command recipe")++ def test_entrypoint_and_shipped_copy_match(self):+ source = (ROOT / "docs/TROUBLESHOOTING.md").read_bytes()+ self.assertEqual(source, (ROOT / "frontend-modern/public/docs/TROUBLESHOOTING.md").read_bytes())+ self.assertIn(b"[required members, active members and spares](TROUBLESHOOTING.md#raid-array-is-degraded-with-a-spare)", source)+++if __name__ == "__main__":+ unittest.main()