Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Resolve TrueNAS Patrol disk heat through its alert policy
pulse-triage[bot]d525ef6583ed
diff --git a/docs/release-control/v6/internal/subsystems/ai-runtime.md b/docs/release-control/v6/internal/subsystems/ai-runtime.mdindex fa473ad85e..beb9334f0b 100644--- a/docs/release-control/v6/internal/subsystems/ai-runtime.md+++ b/docs/release-control/v6/internal/subsystems/ai-runtime.md@@ -382,6 +382,17 @@ linked node or guest) replaces the per-type trigger, and an override or agentdefault that switches the host's alerts off leaves no heat to judge. A disk noagent reports gets the per-type policy. The global alerts switch and the agentalert-type switch stop `CheckHost` without changing this policy, as before.+A TrueNAS disk (`alerts.IsTrueNASDiskResource`, the disks the unified evaluator+alerts on as `truenas-disk`) skips that walk, because its TrueNAS system carries+a synthetic `Agent.AgentID` that would pass for a host agent:+`physicalDiskTemperatureLimits` resolves it through+`ThresholdProvider.GetTrueNASDiskTemperatureThreshold(resourceID, diskType)`+(the alert manager's `TrueNASDiskTemperatureThreshold`, the `truenas-disk`+tiers of `effectiveAlertPolicyNoLock`), so the disk's own override, then the+TrueNAS-wide TrueNAS Disks value, then the per-type policy decide, as for its+temperature alert, and a disabled override or TrueNAS Disks default leaves no+heat. A host override under the system's synthetic ID no longer reaches its+disks, and Disable all TrueNAS silences the alert without changing this policy.The run state carries the provider (`patrolRuntimeState.thresholdProvider`). With no provider the factory alertconfiguration applies. A disk is hot from its trigger: a triage warning, a diskissue and an AI chat "needing attention" entry, as its temperature alert and@@ -392,9 +403,13 @@ verification unknown rather than recovered. Scoped runs keep the provider, and aprovider set before Patrol starts is handed to it. Below the trigger Patrol flags nothing, so an NVMe at 63C staysquiet and a SATA disk at 56C is flagged. Proofs:`internal/ai/patrol_disk_temperature_test.go` (a real registry with one-overridden agent in `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride`),-`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy` and-`TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides` in+overridden agent in `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride`,+and TrueNAS fixture disks under a per-disk override, a TrueNAS-wide value and a+host override on the system ID, one with a retained reading, in+`TestPatrolJudgesTrueNASDisksByTheirAlertTiers`),+`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy`,+`TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides` and+`TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers` in`internal/ai/alert_threshold_adapter_test.go`.Retained summaries disclose that point and bucket timestamps describe returneddiff --git a/docs/release-control/v6/internal/subsystems/alerts.md b/docs/release-control/v6/internal/subsystems/alerts.mdindex 94480c7ae9..2c498bb9ac 100644--- a/docs/release-control/v6/internal/subsystems/alerts.md+++ b/docs/release-control/v6/internal/subsystems/alerts.md@@ -4475,3 +4475,27 @@ resubmission and concurrent response encoding/update under the race detector.Existing snapshot, persistence, partial-update and threshold tests remainrequired. This is source ownership acceptance, not installed notification orguest-agent recovery evidence.++### TrueNAS disk heat policy for Patrol and Assistant++`Manager.TrueNASDiskTemperatureThreshold(resourceID, diskType)` resolves the+`truenas-disk` tiers of `effectiveAlertPolicyNoLock`: the disk's canonical or+identity-resolved override, then TrueNAS Disks defaults, then disk-type policy.+A disabled disk/default or non-positive trigger leaves no heat policy.+Global/platform alert switches silence alert evaluation without changing this+policy, matching existing agent-disk semantics. Callers receive a copy; config+normalisation, saved Off/custom/inherited values and alert severity are unchanged.+`IsTrueNASDiskResource` uses the evaluator's classification, including merged+agent/TrueNAS disks. Patrol/Assistant do not walk those disks to a synthetic+TrueNAS system agent and accidentally apply an unrelated host override.+Retained temperature readings remain unjudged, not affirmative recovery.++Verification: `TestTrueNASDiskTemperatureThresholdMatchesTrueNASDiskAlerts`+and `TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator` in+`internal/alerts/threshold_resolution_shared_test.go` cover tier, identity,+Off/disabled, untyped, clone and nil-manager compatibility.+`TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers` and+`TestPatrolJudgesTrueNASDisksByTheirAlertTiers` cover adapter, Patrol triage,+verification and Assistant context while preserving agent-host overrides.+Frontend table/drawer tiers remain a separate Web-owned change; this backend+repair does not establish browser or native appliance acceptance.diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.mdindex 20b764f7fa..698bd28cc0 100644--- a/docs/release-control/v6/internal/subsystems/unified-resources.md+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md@@ -7499,3 +7499,16 @@ for report-merge. No synthetic telemetry, identity key, enrollment or servicelink is added. `TestOperatorSplitOverridesProxmoxNodeAgentLink` checks thatthe monitor and its re-ingested API seed retain the fold across all splitshapes, then survive pin persistence and a repeated split.++### TrueNAS disk policy consumers use the disk's alert identity++The AI/Patrol physical-disk consumers classify TrueNAS disks through+`alerts.IsTrueNASDiskResource`, the same classifier as unified alert evaluation.+They resolve temperature policy under the disk's canonical resource ID, not its+parent system's synthetic agent ID. Merged agent/TrueNAS disks retain that+classification; ordinary agent disks still follow their reporting host's policy.+Registry identity, source precedence and retained-reading semantics are unchanged.+`TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator` and+`TestPatrolJudgesTrueNASDisksByTheirAlertTiers` exercise the unified registry+projection and its consumer boundaries. Table/drawer frontend acceptance remains+with Web, not established by these backend tests.diff --git a/internal/ai/alert_threshold_adapter.go b/internal/ai/alert_threshold_adapter.goindex 53163e2f3b..143144ea12 100644--- a/internal/ai/alert_threshold_adapter.go+++ b/internal/ai/alert_threshold_adapter.go@@ -79,6 +79,26 @@ func (a *AlertThresholdAdapter) GetDiskTemperatureThreshold(host alerts.DiskTemp} else {threshold = alerts.DefaultDiskTemperatureThreshold(diskType)}+ return diskTemperatureTriggerAndClear(threshold)+}++// GetTrueNASDiskTemperatureThreshold returns the temperature alert trigger and+// clear values for one TrueNAS disk, resolved exactly as its temperature alert+// resolves them (alerts.Manager.TrueNASDiskTemperatureThreshold): the disk's+// own override, then the TrueNAS-wide value, then the per-type policy.+func (a *AlertThresholdAdapter) GetTrueNASDiskTemperatureThreshold(resourceID, diskType string) (float64, float64) {+ var threshold *alerts.HysteresisThreshold+ if a.manager != nil {+ threshold = a.manager.TrueNASDiskTemperatureThreshold(resourceID, diskType)+ } else {+ threshold = alerts.DefaultDiskTemperatureThreshold(diskType)+ }+ return diskTemperatureTriggerAndClear(threshold)+}++// diskTemperatureTriggerAndClear unpacks a disk temperature threshold, with a+// nil or non-positive trigger meaning off (0, 0).+func diskTemperatureTriggerAndClear(threshold *alerts.HysteresisThreshold) (float64, float64) {if threshold == nil || threshold.Trigger <= 0 {return 0, 0}diff --git a/internal/ai/alert_threshold_adapter_test.go b/internal/ai/alert_threshold_adapter_test.goindex 67347911f6..dd231267d2 100644--- a/internal/ai/alert_threshold_adapter_test.go+++ b/internal/ai/alert_threshold_adapter_test.go@@ -108,6 +108,39 @@ func TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy(t *testing.T) {}}+// A TrueNAS disk reaches Patrol through its own alert tiers: its override,+// then the TrueNAS-wide value, then the per-type policy.+func TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers(t *testing.T) {+ if trigger, clear := NewAlertThresholdAdapter(nil).GetTrueNASDiskTemperatureThreshold("disk-any", "nvme"); trigger != 70 || clear != 65 {+ t.Fatalf("no manager: nvme = %v/%v, want the factory 70/65", trigger, clear)+ }++ mgr := alerts.NewManager()+ t.Cleanup(mgr.Stop)+ a := NewAlertThresholdAdapter(mgr)+ if trigger, clear := a.GetTrueNASDiskTemperatureThreshold("disk-plain", "sata"); trigger != 55 || clear != 50 {+ t.Fatalf("no TrueNAS-wide value: sata = %v/%v, want the per-type 55/50", trigger, clear)+ }++ cfg := mgr.GetConfig()+ cfg.TrueNASDiskDefaults.Temperature = &alerts.HysteresisThreshold{Trigger: 62, Clear: 57}+ cfg.Overrides = map[string]alerts.ThresholdConfig{+ "disk-raised": {Temperature: &alerts.HysteresisThreshold{Trigger: 75, Clear: 70}},+ "disk-off": {Disabled: true},+ }+ mgr.UpdateConfig(cfg)+ for id, want := range map[string][2]float64{+ "disk-raised": {75, 70},+ "disk-plain": {62, 57},+ "disk-off": {0, 0},+ } {+ trigger, clear := a.GetTrueNASDiskTemperatureThreshold(id, "nvme")+ if trigger != want[0] || clear != want[1] {+ t.Errorf("%s nvme = %v/%v, want %v/%v", id, trigger, clear, want[0], want[1])+ }+ }+}+// A host's Disk Temp override reaches Patrol for that host's disks only, and a// host whose alerts are switched off has no disk heat policy at all.func TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides(t *testing.T) {diff --git a/internal/ai/mock_test.go b/internal/ai/mock_test.goindex 6fb53b6a55..6672cb39f5 100644--- a/internal/ai/mock_test.go+++ b/internal/ai/mock_test.go@@ -73,6 +73,12 @@ func (m *mockThresholdProvider) GetDiskTemperatureThreshold(_ alerts.DiskTemperareturn 0, 0}+// GetTrueNASDiskTemperatureThreshold judges a TrueNAS disk by the same+// per-type table, as a TrueNAS disk with no override or TrueNAS-wide value is.+func (m *mockThresholdProvider) GetTrueNASDiskTemperatureThreshold(_ string, diskType string) (float64, float64) {+ return m.GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{}, diskType)+}+type mockUnifiedResourceProvider struct {UnifiedResourceProvidergetAllFunc func() []unifiedresources.Resourcediff --git a/internal/ai/patrol.go b/internal/ai/patrol.goindex d349e60417..b8f07600ee 100644--- a/internal/ai/patrol.go+++ b/internal/ai/patrol.go@@ -71,6 +71,12 @@ type ThresholdProvider interface {// agent reports. A host with no ID means no agent reports the disk. A zero// trigger means disk temperature alerting is off for it.GetDiskTemperatureThreshold(host alerts.DiskTemperatureHost, diskType string) (trigger, clear float64)+ // GetTrueNASDiskTemperatureThreshold returns the disk temperature alert+ // trigger and clear values (°C) for one TrueNAS disk, given its resource+ // ID and type: the disk's own override, then the TrueNAS-wide value, then+ // the per-type policy, as its temperature alert resolves them. A zero+ // trigger means disk temperature alerting is off for it.+ GetTrueNASDiskTemperatureThreshold(resourceID, diskType string) (trigger, clear float64)}// AlertResolver provides the ability to review and resolve alertsdiff --git a/internal/ai/patrol_ai.go b/internal/ai/patrol_ai.goindex e8462ae8f8..5b62edfe3b 100644--- a/internal/ai/patrol_ai.go+++ b/internal/ai/patrol_ai.go@@ -2484,7 +2484,7 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)temperature: tools.SplitDiskTemperature(r.PhysicalDisk.Temperature, r.PhysicalDisk.Collection),smartEvidence: unifiedPhysicalDiskSMARTIssueParts(r.PhysicalDisk.SMART),}- row.temperatureLimits = snap.diskTemperatureLimits(physicalDiskTemperatureHost(r, owners), row.diskType)+ row.temperatureLimits = physicalDiskTemperatureLimits(snap.thresholdProvider, r, owners)rows = append(rows, row)}return rowsdiff --git a/internal/ai/patrol_disk_temperature.go b/internal/ai/patrol_disk_temperature.goindex 1af32bb0f4..d5857d6782 100644--- a/internal/ai/patrol_disk_temperature.go+++ b/internal/ai/patrol_disk_temperature.go@@ -22,12 +22,53 @@ type diskTemperatureLimits struct {// that host's Disk Temp override applies, or from the factory alert// configuration when Patrol has no threshold provider.func diskTemperatureLimitsFor(provider ThresholdProvider, host alerts.DiskTemperatureHost, diskType string) diskTemperatureLimits {- var trigger, clear float64if provider != nil {- trigger, clear = provider.GetDiskTemperatureThreshold(host, diskType)- } else if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {- trigger, clear = threshold.Trigger, threshold.Clear+ return newDiskTemperatureLimits(provider.GetDiskTemperatureThreshold(host, diskType))}+ return factoryDiskTemperatureLimits(diskType)+}++// trueNASDiskTemperatureLimitsFor resolves the policy for one TrueNAS disk the+// way its temperature alert does, from the user's alert configuration, so the+// disk's own override and the TrueNAS-wide value apply, or from the factory+// alert configuration when Patrol has no threshold provider.+func trueNASDiskTemperatureLimitsFor(provider ThresholdProvider, resourceID, diskType string) diskTemperatureLimits {+ if provider != nil {+ return newDiskTemperatureLimits(provider.GetTrueNASDiskTemperatureThreshold(resourceID, diskType))+ }+ return factoryDiskTemperatureLimits(diskType)+}++// physicalDiskTemperatureLimits resolves the policy for one physical disk+// resource under the alert that judges its heat. A TrueNAS disk has its own+// temperature alert, judged by the disk's override, the TrueNAS-wide value and+// the per-type policy; its TrueNAS system carries a synthetic agent ID, so the+// host walk would wrongly treat that system as a host agent. Any other disk is+// judged as a SMART disk of the host agent that reports it.+func physicalDiskTemperatureLimits(provider ThresholdProvider, disk unifiedresources.Resource, owners map[string]unifiedresources.Resource) diskTemperatureLimits {+ diskType := ""+ if disk.PhysicalDisk != nil {+ diskType = disk.PhysicalDisk.DiskType+ }+ if alerts.IsTrueNASDiskResource(disk) {+ return trueNASDiskTemperatureLimitsFor(provider, disk.ID, diskType)+ }+ return diskTemperatureLimitsFor(provider, physicalDiskTemperatureHost(disk, owners), diskType)+}++// factoryDiskTemperatureLimits is the policy for a disk of the given type+// under the factory alert configuration.+func factoryDiskTemperatureLimits(diskType string) diskTemperatureLimits {+ if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {+ return newDiskTemperatureLimits(threshold.Trigger, threshold.Clear)+ }+ return diskTemperatureLimits{}+}++// newDiskTemperatureLimits builds limits from an alert trigger and clear+// value. A non-positive trigger is off; a clear value missing or above the+// trigger leaves no band below it.+func newDiskTemperatureLimits(trigger, clear float64) diskTemperatureLimits {if trigger <= 0 {return diskTemperatureLimits{}}@@ -72,7 +113,8 @@ func physicalDiskOwnerIndex(urp UnifiedResourceProvider) map[string]unifiedresou// disk, the host its disk temperature alerts are evaluated under: the agent on// the machine the disk is parented to or, for a disk parented to a storage// pool (Unraid array and cache disks), the agent on the pool's machine. A disk-// no agent reports gets the hostless policy.+// no agent reports gets the hostless policy. TrueNAS disks do not come here;+// physicalDiskTemperatureLimits judges them by their own alert.func physicalDiskTemperatureHost(disk unifiedresources.Resource, owners map[string]unifiedresources.Resource) alerts.DiskTemperatureHost {parentID := disk.ParentIDfor hops := 0; parentID != nil && hops < 2; hops++ {diff --git a/internal/ai/patrol_disk_temperature_test.go b/internal/ai/patrol_disk_temperature_test.goindex dc3260378f..8516dc7d89 100644--- a/internal/ai/patrol_disk_temperature_test.go+++ b/internal/ai/patrol_disk_temperature_test.go@@ -8,6 +8,7 @@ import ("github.com/rcourtman/pulse-go-rewrite/internal/alerts""github.com/rcourtman/pulse-go-rewrite/internal/models"+ "github.com/rcourtman/pulse-go-rewrite/internal/truenas""github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources""github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts""github.com/rcourtman/pulse-go-rewrite/pkg/diskinventory"@@ -343,3 +344,147 @@ func TestPhysicalDiskTemperatureHostFollowsTheParentChain(t *testing.T) {t.Errorf("parentless disk: host = %+v, want hostless", got)}}++// trueNASHeatRegistry is a TrueNAS system whose pool holds four disks beside+// the two overridden host agents of overriddenAgentDisksRegistry: nvme0n1 at+// 72C, nvme1n1 at 63C, SATA sda at 60C and SATA sdb, whose 80C reading is+// retained from before standby. The user saved a TrueNAS-wide 62C (clear 57),+// raised nvme0n1's own trigger to 75C (clear 70), and left a 50C Disk Temp+// override under the TrueNAS system's synthetic agent ID. It returns the disk+// resources by name.+func trueNASHeatRegistry(t *testing.T) (*unifiedresources.ResourceRegistry, *AlertThresholdAdapter, map[string]unifiedresources.Resource) {+ t.Helper()+ registry, _ := overriddenAgentDisksRegistry(t)+ now := time.Now()+ disk := func(name, transport string, temperature int) truenas.Disk {+ return truenas.Disk{+ ID: "disk-" + name, Name: name, Pool: "tank", Status: "ONLINE", Model: "Model " + name,+ Serial: "SERIAL-" + strings.ToUpper(name), Temperature: temperature, Transport: transport,+ }+ }+ records := truenas.FixtureRecords(truenas.FixtureSnapshot{+ CollectedAt: now,+ System: truenas.SystemInfo{Hostname: "truenas-main", Healthy: true, CollectedAt: now},+ Pools: []truenas.Pool{{ID: "pool-tank", Name: "tank", Status: "ONLINE"}},+ Disks: []truenas.Disk{+ disk("nvme0n1", "nvme", 72), disk("nvme1n1", "nvme", 63),+ disk("sda", "sata", 60), disk("sdb", "sata", 80),+ },+ })+ for i := range records {+ if meta := records[i].Resource.PhysicalDisk; meta != nil && records[i].Resource.Name == "sdb" {+ meta.Collection = &diskinventory.CollectionStatus{Temperature: diskinventory.Unavailable("truenas", "disk is in standby")}+ }+ }+ registry.IngestRecords(unifiedresources.SourceTrueNAS, records)++ adapter := unifiedresources.NewUnifiedAIAdapter(registry)+ disks := make(map[string]unifiedresources.Resource)+ for _, r := range adapter.GetByType(unifiedresources.ResourceTypePhysicalDisk) {+ if alerts.IsTrueNASDiskResource(r) {+ disks[r.Name] = r+ }+ }+ systemAgentID := ""+ for _, r := range adapter.GetAll() {+ if r.Agent != nil && r.Agent.Platform == "truenas" {+ systemAgentID = r.Agent.AgentID+ }+ }+ if len(disks) != 4 || systemAgentID == "" {+ t.Fatalf("TrueNAS disks = %v, system agent ID = %q; want four disks under one system", disks, systemAgentID)+ }++ mgr := alerts.NewManager()+ t.Cleanup(mgr.Stop)+ cfg := mgr.GetConfig()+ cfg.TrueNASDiskDefaults.Temperature = &alerts.HysteresisThreshold{Trigger: 62, Clear: 57}+ cfg.Overrides = map[string]alerts.ThresholdConfig{+ "agent-cool": {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 80, Clear: 75}},+ systemAgentID: {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 50, Clear: 45}},+ disks["nvme0n1"].ID: {Temperature: &alerts.HysteresisThreshold{Trigger: 75, Clear: 70}},+ }+ mgr.UpdateConfig(cfg)+ return registry, NewAlertThresholdAdapter(mgr), disks+}++// Patrol judges a TrueNAS disk by its own temperature alert's tiers, not as a+// disk of the TrueNAS system's synthetic host agent: nvme0n1's override keeps+// it quiet at 72C, the TrueNAS-wide 62C flags nvme1n1 at 63C (under the NVMe+// trigger of 70C) and spares SATA sda at 60C (over the SATA trigger of 55C),+// and the 50C host override under the system ID reaches none of them. A+// retained reading stays unjudged. Agent disks keep their host's override.+func TestPatrolJudgesTrueNASDisksByTheirAlertTiers(t *testing.T) {+ registry, provider, disks := trueNASHeatRegistry(t)+ state := newPatrolRuntimeStateWithProviders(models.StateSnapshot{}, nil, unifiedresources.NewUnifiedAIAdapter(registry))+ state.thresholdProvider = provider++ type want struct {+ limits diskTemperatureLimits+ issue bool+ }+ expected := map[string]want{+ disks["nvme0n1"].ID: {limits: diskTemperatureLimits{trigger: 75, clear: 70}},+ disks["nvme1n1"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}, issue: true},+ disks["sda"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}},+ disks["sdb"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}},+ }+ rows := patrolPhysicalDiskRows(state, nil)+ if len(rows) != 6 {+ t.Fatalf("rows = %+v, want four TrueNAS disks and two agent disks", rows)+ }+ for _, row := range rows {+ w, trueNAS := expected[row.id]+ if !trueNAS {+ // overriddenAgentDisksRegistry: agent-cool's 76C NVMe under its 80C+ // override, agent-plain's 72C NVMe over the NVMe trigger.+ w = want{limits: diskTemperatureLimits{trigger: 70, clear: 65}, issue: true}+ if row.temperature.Collected == 76 {+ w = want{limits: diskTemperatureLimits{trigger: 80, clear: 75}}+ }+ }+ if row.temperatureLimits != w.limits {+ t.Errorf("disk %s at %dC: limits = %+v, want %+v", row.id, row.temperature.Collected, row.temperatureLimits, w.limits)+ }+ if got := patrolPhysicalDiskHealthIssue(row); got != w.issue {+ t.Errorf("disk %s at %dC: health issue = %v, want %v", row.id, row.temperature.Collected, got, w.issue)+ }+ }++ flags := triageDiskHealthChecksState(state, nil)+ for _, name := range []string{"nvme0n1", "sda", "sdb"} {+ if flag := triageFindFlag(flags, func(f TriageFlag) bool {+ return f.ResourceID == disks[name].ID && strings.Contains(f.Reason, "Disk temperature")+ }); flag != nil {+ t.Errorf("%s: unexpected temperature flag %+v", name, *flag)+ }+ }+ if flag := triageFindFlag(flags, func(f TriageFlag) bool {+ return f.ResourceID == disks["nvme1n1"].ID && strings.Contains(f.Reason, "Disk temperature")+ }); flag == nil || flag.Threshold != 62 {+ t.Errorf("nvme1n1: temperature flag = %+v, want one at the TrueNAS-wide 62C", flag)+ }++ if _, err := verifyMetricRecoveredState(state, PatrolThresholds{}, "disk-high", disks["sdb"].ID, "physical_disk"); !errors.Is(err, aicontracts.ErrVerificationUnknown) {+ t.Errorf("retained 80C: err = %v, want ErrVerificationUnknown", err)+ }++ s := &Service{}+ s.SetUnifiedResourceProvider(unifiedresources.NewUnifiedAIAdapter(registry))+ s.SetPatrolThresholdProvider(provider)+ _, section, _ := strings.Cut(s.buildUnifiedResourceContext(), "Physical Disks Needing Attention")+ section, _, _ = strings.Cut(section, "\n\n")+ for _, reading := range []string{"Temp: 63C", "Temp: 72C"} {+ if !strings.Contains(section, reading) {+ t.Errorf("attention section = %q, want the disk at %s", section, reading)+ }+ }+ for _, reading := range []string{"Temp: 60C", "Temp: 76C", "Temp: 80C"} {+ if strings.Contains(section, reading) {+ t.Errorf("attention section = %q, lists a disk at %s", section, reading)+ }+ }+ if strings.Count(section, "Temp: 72C") != 1 {+ t.Errorf("attention section = %q, want only agent-plain's 72C disk, not TrueNAS nvme0n1 under its 75C override", section)+ }+}diff --git a/internal/ai/patrol_thresholds_test.go b/internal/ai/patrol_thresholds_test.goindex 883795f24d..71145845ea 100644--- a/internal/ai/patrol_thresholds_test.go+++ b/internal/ai/patrol_thresholds_test.go@@ -24,6 +24,9 @@ func (m MockThresholdProvider) GetStorageThreshold() float64 { return m.Storfunc (m MockThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {return 0, 0}+func (m MockThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {+ return 0, 0+}func TestCalculatePatrolThresholds_Default(t *testing.T) {// Test default behavior (exact mode)diff --git a/internal/ai/resource_context.go b/internal/ai/resource_context.goindex 5d7778688e..0f6df0c27b 100644--- a/internal/ai/resource_context.go+++ b/internal/ai/resource_context.go@@ -417,7 +417,7 @@ func (s *Service) buildUnifiedResourceContextForModel(destinationModel string) shealth = strings.ToUpper(strings.TrimSpace(disk.PhysicalDisk.Health))// A retained last-known reading is not current heat.collected := tools.SplitDiskTemperature(disk.PhysicalDisk.Temperature, disk.PhysicalDisk.Collection).Collected- hot = diskTemperatureLimitsFor(thresholdProvider, physicalDiskTemperatureHost(disk, owners), disk.PhysicalDisk.DiskType).hot(collected)+ hot = physicalDiskTemperatureLimits(thresholdProvider, disk, owners).hot(collected)}if disk.Status != unifiedresources.StatusOnline || (health != "" && health != "PASSED" && health != "UNKNOWN") || hot {attention = append(attention, disk)diff --git a/internal/alerts/threshold_resolution_shared_test.go b/internal/alerts/threshold_resolution_shared_test.goindex 77f6dfd1d2..8d246c3c5f 100644--- a/internal/alerts/threshold_resolution_shared_test.go+++ b/internal/alerts/threshold_resolution_shared_test.go@@ -12,6 +12,7 @@ import (alertspecs "github.com/rcourtman/pulse-go-rewrite/internal/alerts/specs""github.com/rcourtman/pulse-go-rewrite/internal/models""github.com/rcourtman/pulse-go-rewrite/internal/recovery"+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources")func boolPtr(v bool) *bool {@@ -2135,3 +2136,239 @@ func TestHostDiskTemperatureThresholdMatchesCheckHostOverrides(t *testing.T) {t.Fatalf("nil manager host policy = %+v, want the factory nvme 70", got)}}++// TrueNASDiskTemperatureThreshold is the disk heat policy for a TrueNAS disk,+// so it must agree with the disk's temperature alert on every tier: the+// disk's own override, then the TrueNAS-wide value, then the per-type policy.+// A host override under the TrueNAS system's synthetic agent ID is not one of+// them. Disable all TrueNAS silences the alert without changing the policy,+// as the agent switches do for agent disks.+func TestTrueNASDiskTemperatureThresholdMatchesTrueNASDiskAlerts(t *testing.T) {+ disk := trueNASTemperatureDisk("nvme0n1", "nvme", 0)++ const canonicalDiskID = "physical-disk:canonical-nvme0n1"+ for _, tc := range []struct {+ name string+ mutate func(*AlertConfig)+ setup func(*Manager)+ want *HysteresisThreshold+ off bool+ silenced bool+ }{+ {name: "per-type policy", want: &HysteresisThreshold{Trigger: 70, Clear: 65}},+ {+ name: "raised per-type trigger",+ mutate: func(cfg *AlertConfig) {+ cfg.DiskTempByType = map[string]HysteresisThreshold{"nvme": {Trigger: 75, Clear: 70}}+ },+ want: &HysteresisThreshold{Trigger: 75, Clear: 70},+ },+ {+ name: "TrueNAS-wide value",+ mutate: func(cfg *AlertConfig) {+ cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}+ },+ want: &HysteresisThreshold{Trigger: 62, Clear: 57},+ },+ {+ name: "per-disk override beats the TrueNAS-wide value",+ mutate: func(cfg *AlertConfig) {+ cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{Trigger: 75, Clear: 70}}}+ },+ want: &HysteresisThreshold{Trigger: 75, Clear: 70},+ },+ {+ name: "per-disk override without a clear value",+ mutate: func(cfg *AlertConfig) {+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{Trigger: 75}}}+ },+ want: &HysteresisThreshold{Trigger: 75, Clear: 70},+ },+ {+ name: "per-disk override under the canonical identity",+ mutate: func(cfg *AlertConfig) {+ cfg.Overrides = map[string]ThresholdConfig{canonicalDiskID: {Temperature: &HysteresisThreshold{Trigger: 78, Clear: 73}}}+ },+ setup: func(m *Manager) {+ m.SetResourceIntentIdentityResolver(func(resourceID string) (string, bool) {+ return canonicalDiskID, resourceID == disk.ID+ })+ },+ want: &HysteresisThreshold{Trigger: 78, Clear: 73},+ },+ {+ name: "host override under the TrueNAS system ID does not reach its disks",+ mutate: func(cfg *AlertConfig) {+ cfg.Overrides = map[string]ThresholdConfig{"truenas-main": {DiskTemperature: &HysteresisThreshold{Trigger: 50, Clear: 45}}}+ },+ want: &HysteresisThreshold{Trigger: 70, Clear: 65},+ },+ {+ name: "per-disk override switches the disk off",+ mutate: func(cfg *AlertConfig) {+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{}}}+ },+ off: true,+ },+ {+ name: "per-disk override disables the disk's alerts",+ mutate: func(cfg *AlertConfig) {+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Disabled: true}}+ },+ },+ {+ name: "TrueNAS-wide off",+ mutate: func(cfg *AlertConfig) {+ cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{}+ },+ off: true,+ },+ {+ name: "TrueNAS Disks defaults disabled",+ mutate: func(cfg *AlertConfig) {+ cfg.TrueNASDiskDefaults.Disabled = true+ },+ },+ {+ name: "agent Disk Temp default off",+ mutate: func(cfg *AlertConfig) {+ cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{}+ },+ off: true,+ },+ {+ name: "Disable all TrueNAS keeps the policy",+ mutate: func(cfg *AlertConfig) {+ cfg.DisableAllTrueNAS = true+ },+ want: &HysteresisThreshold{Trigger: 70, Clear: 65},+ silenced: true,+ },+ } {+ t.Run(tc.name, func(t *testing.T) {+ m := newTestManager(t)+ cfg := unifiedEvalBaseConfig()+ cfg.TrueNASDiskDefaults = ThresholdConfig{}+ if tc.mutate != nil {+ tc.mutate(&cfg)+ }+ configureUnifiedEvalManager(t, m, cfg)+ if tc.setup != nil {+ tc.setup(m)+ }++ got := m.TrueNASDiskTemperatureThreshold(" "+disk.ID+" ", " NVMe")+ switch {+ case tc.off:+ if got == nil || got.Trigger > 0 {+ t.Fatalf("TrueNASDiskTemperatureThreshold = %+v, want a switched-off threshold", got)+ }+ case (got == nil) != (tc.want == nil) || (got != nil && *got != *tc.want):+ t.Fatalf("TrueNASDiskTemperatureThreshold = %+v, want %+v", got, tc.want)+ }++ var alert *Alert+ fires := func(temperature int) bool {+ reading := disk+ meta := *disk.PhysicalDisk+ meta.Temperature = temperature+ reading.PhysicalDisk = &meta+ checkTrueNASTemperatureDisk(t, m, reading)+ var firing bool+ alert, firing = trueNASDiskTemperatureAlert(t, m, reading)+ return firing+ }+ if tc.silenced {+ if fires(99) {+ t.Fatalf("alert fired at 99C with TrueNAS alerts disabled, active: %v", alertKeys(m))+ }+ return+ }+ if got == nil || got.Trigger <= 0 {+ if fires(99) {+ t.Fatalf("alert fired at 99C on a disk the policy judges no heat for, active: %v", alertKeys(m))+ }+ return+ }+ trigger := int(got.Trigger)+ if fires(trigger - 1) {+ t.Fatalf("alert fired at %dC, under the policy trigger %v", trigger-1, got.Trigger)+ }+ if !fires(trigger) {+ t.Fatalf("alert stayed quiet at the policy trigger %dC, active: %v", trigger, alertKeys(m))+ }+ // The alert recovers at the clear value Patrol reads.+ if alert.Threshold != got.Trigger || alert.Metadata["clearThreshold"] != got.Clear {+ t.Fatalf("alert trigger/clear = %v/%v, want the policy %v/%v", alert.Threshold, alert.Metadata["clearThreshold"], got.Trigger, got.Clear)+ }+ })+ }++ // A disk whose type TrueNAS never reported follows the agent Disk Temp+ // default, as its alert does.+ t.Run("untyped disk", func(t *testing.T) {+ m := newTestManager(t)+ cfg := unifiedEvalBaseConfig()+ cfg.TrueNASDiskDefaults = ThresholdConfig{}+ cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 58, Clear: 53}+ configureUnifiedEvalManager(t, m, cfg)+ untypedDisk := trueNASTemperatureDisk("sdx", "", 58)+ if got := m.TrueNASDiskTemperatureThreshold(untypedDisk.ID, ""); got == nil || got.Trigger != 58 || got.Clear != 53 {+ t.Fatalf("untyped disk = %+v, want the agent default 58/53", got)+ }+ checkTrueNASTemperatureDisk(t, m, untypedDisk)+ if _, firing := trueNASDiskTemperatureAlert(t, m, untypedDisk); !firing {+ t.Fatalf("untyped disk at the agent default 58C raised no alert, active: %v", alertKeys(m))+ }+ })++ m := configureDiskTempTypeHostManager(t)+ m.TrueNASDiskTemperatureThreshold(disk.ID, "nvme").Trigger = 1+ if got := m.TrueNASDiskTemperatureThreshold(disk.ID, "nvme"); got == nil || got.Trigger != 70 {+ t.Fatalf("caller mutation reached the config: %+v", got)+ }+ var nilManager *Manager+ if got := nilManager.TrueNASDiskTemperatureThreshold(disk.ID, "sata"); got == nil || got.Trigger != 55 {+ t.Fatalf("nil manager TrueNAS disk policy = %+v, want the factory sata 55", got)+ }+}++// IsTrueNASDiskResource names the disks the unified evaluator judges as+// TrueNAS disks, the ones TrueNASDiskTemperatureThreshold resolves.+func TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator(t *testing.T) {+ agentDisk := unifiedresources.Resource{+ ID: "physical-disk:host-1/nvme0n1", Type: unifiedresources.ResourceTypePhysicalDisk,+ Sources: []unifiedresources.DataSource{unifiedresources.SourceAgent},+ PhysicalDisk: &unifiedresources.PhysicalDiskMeta{DiskType: "nvme"},+ }+ trueNASFacetDisk := agentDisk+ trueNASFacetDisk.TrueNAS = &unifiedresources.TrueNASData{}+ mergedDisk := agentDisk+ mergedDisk.Sources = []unifiedresources.DataSource{unifiedresources.SourceAgent, unifiedresources.SourceTrueNAS}+ noMetaDisk := trueNASTemperatureDisk("sdy", "sata", 40)+ noMetaDisk.PhysicalDisk = nil+ trueNASSystem := unifiedresources.Resource{+ ID: "agent:truenas-main", Type: unifiedresources.ResourceTypeAgent,+ Sources: []unifiedresources.DataSource{unifiedresources.SourceTrueNAS},+ }+ for name, tc := range map[string]struct {+ resource unifiedresources.Resource+ want bool+ }{+ "TrueNAS-sourced disk": {resource: trueNASTemperatureDisk("sda", "sata", 40), want: true},+ "TrueNAS facet disk": {resource: trueNASFacetDisk, want: true},+ "agent and TrueNAS": {resource: mergedDisk, want: true},+ "no disk metadata": {resource: noMetaDisk, want: true},+ "agent disk": {resource: agentDisk},+ "TrueNAS system": {resource: trueNASSystem},+ } {+ if got := IsTrueNASDiskResource(tc.resource); got != tc.want {+ t.Errorf("%s: IsTrueNASDiskResource = %v, want %v", name, got, tc.want)+ }+ input, ok := UnifiedResourceInputFromResource(tc.resource)+ if evaluated := ok && input.Type == "truenas-disk"; evaluated != tc.want {+ t.Errorf("%s: unified evaluator judges it as a TrueNAS disk = %v, want %v", name, evaluated, tc.want)+ }+ }+}diff --git a/internal/alerts/truenas_disk_temperature.go b/internal/alerts/truenas_disk_temperature.goindex 9934e83233..d7fca0f69a 100644--- a/internal/alerts/truenas_disk_temperature.go+++ b/internal/alerts/truenas_disk_temperature.go@@ -1,5 +1,11 @@package alerts+import (+ "strings"++ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"+)+// trueNASDiskTemperatureDefaultNoLock returns the default tier of a TrueNAS// disk's temperature threshold. A TrueNAS-wide value the user saved under// TrueNAS Disks applies to every TrueNAS disk, as a host Disk Temp override@@ -16,3 +22,39 @@ func (m *Manager) trueNASDiskTemperatureDefaultNoLock(diskType string, typeUnkno}return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature, false, diskType))}++// IsTrueNASDiskResource reports whether the unified evaluator judges a+// resource as a TrueNAS disk (truenas-disk), whose temperature alert resolves+// its threshold through TrueNASDiskTemperatureThreshold rather than through+// the host agent that reports it.+func IsTrueNASDiskResource(resource unifiedresources.Resource) bool {+ typeKey, ok := unifiedAlertResourceType(resource)+ return ok && typeKey == "truenas-disk"+}++// TrueNASDiskTemperatureThreshold is the disk heat policy for one TrueNAS+// disk, resolved by the same effectiveAlertPolicyNoLock tiers its temperature+// alert uses: an override stored under the disk's resource ID, then the+// TrueNAS-wide TrueNAS Disks value, then the per-type policy. An override or+// TrueNAS Disks default that switches the disk's alerts off leaves no+// threshold (nil); a non-positive trigger is off too. Like+// HostDiskTemperatureThreshold it ignores the global and per-platform alert+// switches, which silence the alert without changing the policy. A nil+// manager resolves the factory configuration, where no TrueNAS-wide value is+// set.+func (m *Manager) TrueNASDiskTemperatureThreshold(resourceID, diskType string) *HysteresisThreshold {+ if m == nil {+ return DefaultDiskTemperatureThreshold(diskType)+ }+ m.mu.RLock()+ defer m.mu.RUnlock()+ thresholds := m.effectiveAlertPolicyNoLock(alertPolicyQuery{+ TypeKey: "truenas-disk",+ ResourceID: unifiedresources.CanonicalResourceID(resourceID),+ DiskType: strings.ToLower(strings.TrimSpace(diskType)),+ }).Thresholds+ if thresholds.Disabled {+ return nil+ }+ return cloneThreshold(thresholds.Temperature)+}diff --git a/internal/api/ai_handlers_concurrency_test.go b/internal/api/ai_handlers_concurrency_test.goindex 0c5276fa05..db1b4cad27 100644--- a/internal/api/ai_handlers_concurrency_test.go+++ b/internal/api/ai_handlers_concurrency_test.go@@ -71,6 +71,9 @@ func (concurrencyThresholdProvider) GetStorageThreshold() float64 { return 92func (concurrencyThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {return 70, 65}+func (concurrencyThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {+ return 70, 65+}type concurrencyMetricsHistoryProvider struct{}diff --git a/internal/api/ai_handlers_more_test.go b/internal/api/ai_handlers_more_test.goindex fb0397dca8..8f67b0d184 100644--- a/internal/api/ai_handlers_more_test.go+++ b/internal/api/ai_handlers_more_test.go@@ -162,6 +162,9 @@ func (stubThresholdProvider) GetStorageThreshold() float64 { return 92 }func (stubThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {return 70, 65}+func (stubThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {+ return 70, 65+}type stubMetricsHistoryProvider struct{}