Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Resolve TrueNAS Patrol disk heat through its alert policy

pulse-triage[bot]d525ef6583ed
diff --git a/docs/release-control/v6/internal/subsystems/ai-runtime.md b/docs/release-control/v6/internal/subsystems/ai-runtime.md
index fa473ad85e..beb9334f0b 100644
--- a/docs/release-control/v6/internal/subsystems/ai-runtime.md
+++ b/docs/release-control/v6/internal/subsystems/ai-runtime.md
@@ -382,6 +382,17 @@ linked node or guest) replaces the per-type trigger, and an override or agent
default that switches the host's alerts off leaves no heat to judge. A disk no
agent reports gets the per-type policy. The global alerts switch and the agent
alert-type switch stop `CheckHost` without changing this policy, as before.
+A TrueNAS disk (`alerts.IsTrueNASDiskResource`, the disks the unified evaluator
+alerts on as `truenas-disk`) skips that walk, because its TrueNAS system carries
+a synthetic `Agent.AgentID` that would pass for a host agent:
+`physicalDiskTemperatureLimits` resolves it through
+`ThresholdProvider.GetTrueNASDiskTemperatureThreshold(resourceID, diskType)`
+(the alert manager's `TrueNASDiskTemperatureThreshold`, the `truenas-disk`
+tiers of `effectiveAlertPolicyNoLock`), so the disk's own override, then the
+TrueNAS-wide TrueNAS Disks value, then the per-type policy decide, as for its
+temperature alert, and a disabled override or TrueNAS Disks default leaves no
+heat. A host override under the system's synthetic ID no longer reaches its
+disks, and Disable all TrueNAS silences the alert without changing this policy.
The run state carries the provider (`patrolRuntimeState.thresholdProvider`). With no provider the factory alert
configuration applies. A disk is hot from its trigger: a triage warning, a disk
issue and an AI chat "needing attention" entry, as its temperature alert and
@@ -392,9 +403,13 @@ verification unknown rather than recovered. Scoped runs keep the provider, and a
provider set before Patrol starts is handed to it. Below the trigger Patrol flags nothing, so an NVMe at 63C stays
quiet and a SATA disk at 56C is flagged. Proofs:
`internal/ai/patrol_disk_temperature_test.go` (a real registry with one
-overridden agent in `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride`),
-`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy` and
-`TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides` in
+overridden agent in `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride`,
+and TrueNAS fixture disks under a per-disk override, a TrueNAS-wide value and a
+host override on the system ID, one with a retained reading, in
+`TestPatrolJudgesTrueNASDisksByTheirAlertTiers`),
+`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy`,
+`TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides` and
+`TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers` in
`internal/ai/alert_threshold_adapter_test.go`.
Retained summaries disclose that point and bucket timestamps describe returned
diff --git a/docs/release-control/v6/internal/subsystems/alerts.md b/docs/release-control/v6/internal/subsystems/alerts.md
index 94480c7ae9..2c498bb9ac 100644
--- a/docs/release-control/v6/internal/subsystems/alerts.md
+++ b/docs/release-control/v6/internal/subsystems/alerts.md
@@ -4475,3 +4475,27 @@ resubmission and concurrent response encoding/update under the race detector.
Existing snapshot, persistence, partial-update and threshold tests remain
required. This is source ownership acceptance, not installed notification or
guest-agent recovery evidence.
+
+### TrueNAS disk heat policy for Patrol and Assistant
+
+`Manager.TrueNASDiskTemperatureThreshold(resourceID, diskType)` resolves the
+`truenas-disk` tiers of `effectiveAlertPolicyNoLock`: the disk's canonical or
+identity-resolved override, then TrueNAS Disks defaults, then disk-type policy.
+A disabled disk/default or non-positive trigger leaves no heat policy.
+Global/platform alert switches silence alert evaluation without changing this
+policy, matching existing agent-disk semantics. Callers receive a copy; config
+normalisation, saved Off/custom/inherited values and alert severity are unchanged.
+`IsTrueNASDiskResource` uses the evaluator's classification, including merged
+agent/TrueNAS disks. Patrol/Assistant do not walk those disks to a synthetic
+TrueNAS system agent and accidentally apply an unrelated host override.
+Retained temperature readings remain unjudged, not affirmative recovery.
+
+Verification: `TestTrueNASDiskTemperatureThresholdMatchesTrueNASDiskAlerts`
+and `TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator` in
+`internal/alerts/threshold_resolution_shared_test.go` cover tier, identity,
+Off/disabled, untyped, clone and nil-manager compatibility.
+`TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers` and
+`TestPatrolJudgesTrueNASDisksByTheirAlertTiers` cover adapter, Patrol triage,
+verification and Assistant context while preserving agent-host overrides.
+Frontend table/drawer tiers remain a separate Web-owned change; this backend
+repair does not establish browser or native appliance acceptance.
diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.md
index 20b764f7fa..698bd28cc0 100644
--- a/docs/release-control/v6/internal/subsystems/unified-resources.md
+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md
@@ -7499,3 +7499,16 @@ for report-merge. No synthetic telemetry, identity key, enrollment or service
link is added. `TestOperatorSplitOverridesProxmoxNodeAgentLink` checks that
the monitor and its re-ingested API seed retain the fold across all split
shapes, then survive pin persistence and a repeated split.
+
+### TrueNAS disk policy consumers use the disk's alert identity
+
+The AI/Patrol physical-disk consumers classify TrueNAS disks through
+`alerts.IsTrueNASDiskResource`, the same classifier as unified alert evaluation.
+They resolve temperature policy under the disk's canonical resource ID, not its
+parent system's synthetic agent ID. Merged agent/TrueNAS disks retain that
+classification; ordinary agent disks still follow their reporting host's policy.
+Registry identity, source precedence and retained-reading semantics are unchanged.
+`TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator` and
+`TestPatrolJudgesTrueNASDisksByTheirAlertTiers` exercise the unified registry
+projection and its consumer boundaries. Table/drawer frontend acceptance remains
+with Web, not established by these backend tests.
diff --git a/internal/ai/alert_threshold_adapter.go b/internal/ai/alert_threshold_adapter.go
index 53163e2f3b..143144ea12 100644
--- a/internal/ai/alert_threshold_adapter.go
+++ b/internal/ai/alert_threshold_adapter.go
@@ -79,6 +79,26 @@ func (a *AlertThresholdAdapter) GetDiskTemperatureThreshold(host alerts.DiskTemp
} else {
threshold = alerts.DefaultDiskTemperatureThreshold(diskType)
}
+ return diskTemperatureTriggerAndClear(threshold)
+}
+
+// GetTrueNASDiskTemperatureThreshold returns the temperature alert trigger and
+// clear values for one TrueNAS disk, resolved exactly as its temperature alert
+// resolves them (alerts.Manager.TrueNASDiskTemperatureThreshold): the disk's
+// own override, then the TrueNAS-wide value, then the per-type policy.
+func (a *AlertThresholdAdapter) GetTrueNASDiskTemperatureThreshold(resourceID, diskType string) (float64, float64) {
+ var threshold *alerts.HysteresisThreshold
+ if a.manager != nil {
+ threshold = a.manager.TrueNASDiskTemperatureThreshold(resourceID, diskType)
+ } else {
+ threshold = alerts.DefaultDiskTemperatureThreshold(diskType)
+ }
+ return diskTemperatureTriggerAndClear(threshold)
+}
+
+// diskTemperatureTriggerAndClear unpacks a disk temperature threshold, with a
+// nil or non-positive trigger meaning off (0, 0).
+func diskTemperatureTriggerAndClear(threshold *alerts.HysteresisThreshold) (float64, float64) {
if threshold == nil || threshold.Trigger <= 0 {
return 0, 0
}
diff --git a/internal/ai/alert_threshold_adapter_test.go b/internal/ai/alert_threshold_adapter_test.go
index 67347911f6..dd231267d2 100644
--- a/internal/ai/alert_threshold_adapter_test.go
+++ b/internal/ai/alert_threshold_adapter_test.go
@@ -108,6 +108,39 @@ func TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy(t *testing.T) {
}
}
+// A TrueNAS disk reaches Patrol through its own alert tiers: its override,
+// then the TrueNAS-wide value, then the per-type policy.
+func TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers(t *testing.T) {
+ if trigger, clear := NewAlertThresholdAdapter(nil).GetTrueNASDiskTemperatureThreshold("disk-any", "nvme"); trigger != 70 || clear != 65 {
+ t.Fatalf("no manager: nvme = %v/%v, want the factory 70/65", trigger, clear)
+ }
+
+ mgr := alerts.NewManager()
+ t.Cleanup(mgr.Stop)
+ a := NewAlertThresholdAdapter(mgr)
+ if trigger, clear := a.GetTrueNASDiskTemperatureThreshold("disk-plain", "sata"); trigger != 55 || clear != 50 {
+ t.Fatalf("no TrueNAS-wide value: sata = %v/%v, want the per-type 55/50", trigger, clear)
+ }
+
+ cfg := mgr.GetConfig()
+ cfg.TrueNASDiskDefaults.Temperature = &alerts.HysteresisThreshold{Trigger: 62, Clear: 57}
+ cfg.Overrides = map[string]alerts.ThresholdConfig{
+ "disk-raised": {Temperature: &alerts.HysteresisThreshold{Trigger: 75, Clear: 70}},
+ "disk-off": {Disabled: true},
+ }
+ mgr.UpdateConfig(cfg)
+ for id, want := range map[string][2]float64{
+ "disk-raised": {75, 70},
+ "disk-plain": {62, 57},
+ "disk-off": {0, 0},
+ } {
+ trigger, clear := a.GetTrueNASDiskTemperatureThreshold(id, "nvme")
+ if trigger != want[0] || clear != want[1] {
+ t.Errorf("%s nvme = %v/%v, want %v/%v", id, trigger, clear, want[0], want[1])
+ }
+ }
+}
+
// A host's Disk Temp override reaches Patrol for that host's disks only, and a
// host whose alerts are switched off has no disk heat policy at all.
func TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides(t *testing.T) {
diff --git a/internal/ai/mock_test.go b/internal/ai/mock_test.go
index 6fb53b6a55..6672cb39f5 100644
--- a/internal/ai/mock_test.go
+++ b/internal/ai/mock_test.go
@@ -73,6 +73,12 @@ func (m *mockThresholdProvider) GetDiskTemperatureThreshold(_ alerts.DiskTempera
return 0, 0
}
+// GetTrueNASDiskTemperatureThreshold judges a TrueNAS disk by the same
+// per-type table, as a TrueNAS disk with no override or TrueNAS-wide value is.
+func (m *mockThresholdProvider) GetTrueNASDiskTemperatureThreshold(_ string, diskType string) (float64, float64) {
+ return m.GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{}, diskType)
+}
+
type mockUnifiedResourceProvider struct {
UnifiedResourceProvider
getAllFunc func() []unifiedresources.Resource
diff --git a/internal/ai/patrol.go b/internal/ai/patrol.go
index d349e60417..b8f07600ee 100644
--- a/internal/ai/patrol.go
+++ b/internal/ai/patrol.go
@@ -71,6 +71,12 @@ type ThresholdProvider interface {
// agent reports. A host with no ID means no agent reports the disk. A zero
// trigger means disk temperature alerting is off for it.
GetDiskTemperatureThreshold(host alerts.DiskTemperatureHost, diskType string) (trigger, clear float64)
+ // GetTrueNASDiskTemperatureThreshold returns the disk temperature alert
+ // trigger and clear values (°C) for one TrueNAS disk, given its resource
+ // ID and type: the disk's own override, then the TrueNAS-wide value, then
+ // the per-type policy, as its temperature alert resolves them. A zero
+ // trigger means disk temperature alerting is off for it.
+ GetTrueNASDiskTemperatureThreshold(resourceID, diskType string) (trigger, clear float64)
}
// AlertResolver provides the ability to review and resolve alerts
diff --git a/internal/ai/patrol_ai.go b/internal/ai/patrol_ai.go
index e8462ae8f8..5b62edfe3b 100644
--- a/internal/ai/patrol_ai.go
+++ b/internal/ai/patrol_ai.go
@@ -2484,7 +2484,7 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
temperature: tools.SplitDiskTemperature(r.PhysicalDisk.Temperature, r.PhysicalDisk.Collection),
smartEvidence: unifiedPhysicalDiskSMARTIssueParts(r.PhysicalDisk.SMART),
}
- row.temperatureLimits = snap.diskTemperatureLimits(physicalDiskTemperatureHost(r, owners), row.diskType)
+ row.temperatureLimits = physicalDiskTemperatureLimits(snap.thresholdProvider, r, owners)
rows = append(rows, row)
}
return rows
diff --git a/internal/ai/patrol_disk_temperature.go b/internal/ai/patrol_disk_temperature.go
index 1af32bb0f4..d5857d6782 100644
--- a/internal/ai/patrol_disk_temperature.go
+++ b/internal/ai/patrol_disk_temperature.go
@@ -22,12 +22,53 @@ type diskTemperatureLimits struct {
// that host's Disk Temp override applies, or from the factory alert
// configuration when Patrol has no threshold provider.
func diskTemperatureLimitsFor(provider ThresholdProvider, host alerts.DiskTemperatureHost, diskType string) diskTemperatureLimits {
- var trigger, clear float64
if provider != nil {
- trigger, clear = provider.GetDiskTemperatureThreshold(host, diskType)
- } else if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {
- trigger, clear = threshold.Trigger, threshold.Clear
+ return newDiskTemperatureLimits(provider.GetDiskTemperatureThreshold(host, diskType))
}
+ return factoryDiskTemperatureLimits(diskType)
+}
+
+// trueNASDiskTemperatureLimitsFor resolves the policy for one TrueNAS disk the
+// way its temperature alert does, from the user's alert configuration, so the
+// disk's own override and the TrueNAS-wide value apply, or from the factory
+// alert configuration when Patrol has no threshold provider.
+func trueNASDiskTemperatureLimitsFor(provider ThresholdProvider, resourceID, diskType string) diskTemperatureLimits {
+ if provider != nil {
+ return newDiskTemperatureLimits(provider.GetTrueNASDiskTemperatureThreshold(resourceID, diskType))
+ }
+ return factoryDiskTemperatureLimits(diskType)
+}
+
+// physicalDiskTemperatureLimits resolves the policy for one physical disk
+// resource under the alert that judges its heat. A TrueNAS disk has its own
+// temperature alert, judged by the disk's override, the TrueNAS-wide value and
+// the per-type policy; its TrueNAS system carries a synthetic agent ID, so the
+// host walk would wrongly treat that system as a host agent. Any other disk is
+// judged as a SMART disk of the host agent that reports it.
+func physicalDiskTemperatureLimits(provider ThresholdProvider, disk unifiedresources.Resource, owners map[string]unifiedresources.Resource) diskTemperatureLimits {
+ diskType := ""
+ if disk.PhysicalDisk != nil {
+ diskType = disk.PhysicalDisk.DiskType
+ }
+ if alerts.IsTrueNASDiskResource(disk) {
+ return trueNASDiskTemperatureLimitsFor(provider, disk.ID, diskType)
+ }
+ return diskTemperatureLimitsFor(provider, physicalDiskTemperatureHost(disk, owners), diskType)
+}
+
+// factoryDiskTemperatureLimits is the policy for a disk of the given type
+// under the factory alert configuration.
+func factoryDiskTemperatureLimits(diskType string) diskTemperatureLimits {
+ if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {
+ return newDiskTemperatureLimits(threshold.Trigger, threshold.Clear)
+ }
+ return diskTemperatureLimits{}
+}
+
+// newDiskTemperatureLimits builds limits from an alert trigger and clear
+// value. A non-positive trigger is off; a clear value missing or above the
+// trigger leaves no band below it.
+func newDiskTemperatureLimits(trigger, clear float64) diskTemperatureLimits {
if trigger <= 0 {
return diskTemperatureLimits{}
}
@@ -72,7 +113,8 @@ func physicalDiskOwnerIndex(urp UnifiedResourceProvider) map[string]unifiedresou
// disk, the host its disk temperature alerts are evaluated under: the agent on
// the machine the disk is parented to or, for a disk parented to a storage
// pool (Unraid array and cache disks), the agent on the pool's machine. A disk
-// no agent reports gets the hostless policy.
+// no agent reports gets the hostless policy. TrueNAS disks do not come here;
+// physicalDiskTemperatureLimits judges them by their own alert.
func physicalDiskTemperatureHost(disk unifiedresources.Resource, owners map[string]unifiedresources.Resource) alerts.DiskTemperatureHost {
parentID := disk.ParentID
for hops := 0; parentID != nil && hops < 2; hops++ {
diff --git a/internal/ai/patrol_disk_temperature_test.go b/internal/ai/patrol_disk_temperature_test.go
index dc3260378f..8516dc7d89 100644
--- a/internal/ai/patrol_disk_temperature_test.go
+++ b/internal/ai/patrol_disk_temperature_test.go
@@ -8,6 +8,7 @@ import (
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
+ "github.com/rcourtman/pulse-go-rewrite/internal/truenas"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
"github.com/rcourtman/pulse-go-rewrite/pkg/diskinventory"
@@ -343,3 +344,147 @@ func TestPhysicalDiskTemperatureHostFollowsTheParentChain(t *testing.T) {
t.Errorf("parentless disk: host = %+v, want hostless", got)
}
}
+
+// trueNASHeatRegistry is a TrueNAS system whose pool holds four disks beside
+// the two overridden host agents of overriddenAgentDisksRegistry: nvme0n1 at
+// 72C, nvme1n1 at 63C, SATA sda at 60C and SATA sdb, whose 80C reading is
+// retained from before standby. The user saved a TrueNAS-wide 62C (clear 57),
+// raised nvme0n1's own trigger to 75C (clear 70), and left a 50C Disk Temp
+// override under the TrueNAS system's synthetic agent ID. It returns the disk
+// resources by name.
+func trueNASHeatRegistry(t *testing.T) (*unifiedresources.ResourceRegistry, *AlertThresholdAdapter, map[string]unifiedresources.Resource) {
+ t.Helper()
+ registry, _ := overriddenAgentDisksRegistry(t)
+ now := time.Now()
+ disk := func(name, transport string, temperature int) truenas.Disk {
+ return truenas.Disk{
+ ID: "disk-" + name, Name: name, Pool: "tank", Status: "ONLINE", Model: "Model " + name,
+ Serial: "SERIAL-" + strings.ToUpper(name), Temperature: temperature, Transport: transport,
+ }
+ }
+ records := truenas.FixtureRecords(truenas.FixtureSnapshot{
+ CollectedAt: now,
+ System: truenas.SystemInfo{Hostname: "truenas-main", Healthy: true, CollectedAt: now},
+ Pools: []truenas.Pool{{ID: "pool-tank", Name: "tank", Status: "ONLINE"}},
+ Disks: []truenas.Disk{
+ disk("nvme0n1", "nvme", 72), disk("nvme1n1", "nvme", 63),
+ disk("sda", "sata", 60), disk("sdb", "sata", 80),
+ },
+ })
+ for i := range records {
+ if meta := records[i].Resource.PhysicalDisk; meta != nil && records[i].Resource.Name == "sdb" {
+ meta.Collection = &diskinventory.CollectionStatus{Temperature: diskinventory.Unavailable("truenas", "disk is in standby")}
+ }
+ }
+ registry.IngestRecords(unifiedresources.SourceTrueNAS, records)
+
+ adapter := unifiedresources.NewUnifiedAIAdapter(registry)
+ disks := make(map[string]unifiedresources.Resource)
+ for _, r := range adapter.GetByType(unifiedresources.ResourceTypePhysicalDisk) {
+ if alerts.IsTrueNASDiskResource(r) {
+ disks[r.Name] = r
+ }
+ }
+ systemAgentID := ""
+ for _, r := range adapter.GetAll() {
+ if r.Agent != nil && r.Agent.Platform == "truenas" {
+ systemAgentID = r.Agent.AgentID
+ }
+ }
+ if len(disks) != 4 || systemAgentID == "" {
+ t.Fatalf("TrueNAS disks = %v, system agent ID = %q; want four disks under one system", disks, systemAgentID)
+ }
+
+ mgr := alerts.NewManager()
+ t.Cleanup(mgr.Stop)
+ cfg := mgr.GetConfig()
+ cfg.TrueNASDiskDefaults.Temperature = &alerts.HysteresisThreshold{Trigger: 62, Clear: 57}
+ cfg.Overrides = map[string]alerts.ThresholdConfig{
+ "agent-cool": {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 80, Clear: 75}},
+ systemAgentID: {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 50, Clear: 45}},
+ disks["nvme0n1"].ID: {Temperature: &alerts.HysteresisThreshold{Trigger: 75, Clear: 70}},
+ }
+ mgr.UpdateConfig(cfg)
+ return registry, NewAlertThresholdAdapter(mgr), disks
+}
+
+// Patrol judges a TrueNAS disk by its own temperature alert's tiers, not as a
+// disk of the TrueNAS system's synthetic host agent: nvme0n1's override keeps
+// it quiet at 72C, the TrueNAS-wide 62C flags nvme1n1 at 63C (under the NVMe
+// trigger of 70C) and spares SATA sda at 60C (over the SATA trigger of 55C),
+// and the 50C host override under the system ID reaches none of them. A
+// retained reading stays unjudged. Agent disks keep their host's override.
+func TestPatrolJudgesTrueNASDisksByTheirAlertTiers(t *testing.T) {
+ registry, provider, disks := trueNASHeatRegistry(t)
+ state := newPatrolRuntimeStateWithProviders(models.StateSnapshot{}, nil, unifiedresources.NewUnifiedAIAdapter(registry))
+ state.thresholdProvider = provider
+
+ type want struct {
+ limits diskTemperatureLimits
+ issue bool
+ }
+ expected := map[string]want{
+ disks["nvme0n1"].ID: {limits: diskTemperatureLimits{trigger: 75, clear: 70}},
+ disks["nvme1n1"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}, issue: true},
+ disks["sda"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}},
+ disks["sdb"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}},
+ }
+ rows := patrolPhysicalDiskRows(state, nil)
+ if len(rows) != 6 {
+ t.Fatalf("rows = %+v, want four TrueNAS disks and two agent disks", rows)
+ }
+ for _, row := range rows {
+ w, trueNAS := expected[row.id]
+ if !trueNAS {
+ // overriddenAgentDisksRegistry: agent-cool's 76C NVMe under its 80C
+ // override, agent-plain's 72C NVMe over the NVMe trigger.
+ w = want{limits: diskTemperatureLimits{trigger: 70, clear: 65}, issue: true}
+ if row.temperature.Collected == 76 {
+ w = want{limits: diskTemperatureLimits{trigger: 80, clear: 75}}
+ }
+ }
+ if row.temperatureLimits != w.limits {
+ t.Errorf("disk %s at %dC: limits = %+v, want %+v", row.id, row.temperature.Collected, row.temperatureLimits, w.limits)
+ }
+ if got := patrolPhysicalDiskHealthIssue(row); got != w.issue {
+ t.Errorf("disk %s at %dC: health issue = %v, want %v", row.id, row.temperature.Collected, got, w.issue)
+ }
+ }
+
+ flags := triageDiskHealthChecksState(state, nil)
+ for _, name := range []string{"nvme0n1", "sda", "sdb"} {
+ if flag := triageFindFlag(flags, func(f TriageFlag) bool {
+ return f.ResourceID == disks[name].ID && strings.Contains(f.Reason, "Disk temperature")
+ }); flag != nil {
+ t.Errorf("%s: unexpected temperature flag %+v", name, *flag)
+ }
+ }
+ if flag := triageFindFlag(flags, func(f TriageFlag) bool {
+ return f.ResourceID == disks["nvme1n1"].ID && strings.Contains(f.Reason, "Disk temperature")
+ }); flag == nil || flag.Threshold != 62 {
+ t.Errorf("nvme1n1: temperature flag = %+v, want one at the TrueNAS-wide 62C", flag)
+ }
+
+ if _, err := verifyMetricRecoveredState(state, PatrolThresholds{}, "disk-high", disks["sdb"].ID, "physical_disk"); !errors.Is(err, aicontracts.ErrVerificationUnknown) {
+ t.Errorf("retained 80C: err = %v, want ErrVerificationUnknown", err)
+ }
+
+ s := &Service{}
+ s.SetUnifiedResourceProvider(unifiedresources.NewUnifiedAIAdapter(registry))
+ s.SetPatrolThresholdProvider(provider)
+ _, section, _ := strings.Cut(s.buildUnifiedResourceContext(), "Physical Disks Needing Attention")
+ section, _, _ = strings.Cut(section, "\n\n")
+ for _, reading := range []string{"Temp: 63C", "Temp: 72C"} {
+ if !strings.Contains(section, reading) {
+ t.Errorf("attention section = %q, want the disk at %s", section, reading)
+ }
+ }
+ for _, reading := range []string{"Temp: 60C", "Temp: 76C", "Temp: 80C"} {
+ if strings.Contains(section, reading) {
+ t.Errorf("attention section = %q, lists a disk at %s", section, reading)
+ }
+ }
+ if strings.Count(section, "Temp: 72C") != 1 {
+ t.Errorf("attention section = %q, want only agent-plain's 72C disk, not TrueNAS nvme0n1 under its 75C override", section)
+ }
+}
diff --git a/internal/ai/patrol_thresholds_test.go b/internal/ai/patrol_thresholds_test.go
index 883795f24d..71145845ea 100644
--- a/internal/ai/patrol_thresholds_test.go
+++ b/internal/ai/patrol_thresholds_test.go
@@ -24,6 +24,9 @@ func (m MockThresholdProvider) GetStorageThreshold() float64 { return m.Stor
func (m MockThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
return 0, 0
}
+func (m MockThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {
+ return 0, 0
+}
func TestCalculatePatrolThresholds_Default(t *testing.T) {
// Test default behavior (exact mode)
diff --git a/internal/ai/resource_context.go b/internal/ai/resource_context.go
index 5d7778688e..0f6df0c27b 100644
--- a/internal/ai/resource_context.go
+++ b/internal/ai/resource_context.go
@@ -417,7 +417,7 @@ func (s *Service) buildUnifiedResourceContextForModel(destinationModel string) s
health = strings.ToUpper(strings.TrimSpace(disk.PhysicalDisk.Health))
// A retained last-known reading is not current heat.
collected := tools.SplitDiskTemperature(disk.PhysicalDisk.Temperature, disk.PhysicalDisk.Collection).Collected
- hot = diskTemperatureLimitsFor(thresholdProvider, physicalDiskTemperatureHost(disk, owners), disk.PhysicalDisk.DiskType).hot(collected)
+ hot = physicalDiskTemperatureLimits(thresholdProvider, disk, owners).hot(collected)
}
if disk.Status != unifiedresources.StatusOnline || (health != "" && health != "PASSED" && health != "UNKNOWN") || hot {
attention = append(attention, disk)
diff --git a/internal/alerts/threshold_resolution_shared_test.go b/internal/alerts/threshold_resolution_shared_test.go
index 77f6dfd1d2..8d246c3c5f 100644
--- a/internal/alerts/threshold_resolution_shared_test.go
+++ b/internal/alerts/threshold_resolution_shared_test.go
@@ -12,6 +12,7 @@ import (
alertspecs "github.com/rcourtman/pulse-go-rewrite/internal/alerts/specs"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/internal/recovery"
+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
)
func boolPtr(v bool) *bool {
@@ -2135,3 +2136,239 @@ func TestHostDiskTemperatureThresholdMatchesCheckHostOverrides(t *testing.T) {
t.Fatalf("nil manager host policy = %+v, want the factory nvme 70", got)
}
}
+
+// TrueNASDiskTemperatureThreshold is the disk heat policy for a TrueNAS disk,
+// so it must agree with the disk's temperature alert on every tier: the
+// disk's own override, then the TrueNAS-wide value, then the per-type policy.
+// A host override under the TrueNAS system's synthetic agent ID is not one of
+// them. Disable all TrueNAS silences the alert without changing the policy,
+// as the agent switches do for agent disks.
+func TestTrueNASDiskTemperatureThresholdMatchesTrueNASDiskAlerts(t *testing.T) {
+ disk := trueNASTemperatureDisk("nvme0n1", "nvme", 0)
+
+ const canonicalDiskID = "physical-disk:canonical-nvme0n1"
+ for _, tc := range []struct {
+ name string
+ mutate func(*AlertConfig)
+ setup func(*Manager)
+ want *HysteresisThreshold
+ off bool
+ silenced bool
+ }{
+ {name: "per-type policy", want: &HysteresisThreshold{Trigger: 70, Clear: 65}},
+ {
+ name: "raised per-type trigger",
+ mutate: func(cfg *AlertConfig) {
+ cfg.DiskTempByType = map[string]HysteresisThreshold{"nvme": {Trigger: 75, Clear: 70}}
+ },
+ want: &HysteresisThreshold{Trigger: 75, Clear: 70},
+ },
+ {
+ name: "TrueNAS-wide value",
+ mutate: func(cfg *AlertConfig) {
+ cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}
+ },
+ want: &HysteresisThreshold{Trigger: 62, Clear: 57},
+ },
+ {
+ name: "per-disk override beats the TrueNAS-wide value",
+ mutate: func(cfg *AlertConfig) {
+ cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}
+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{Trigger: 75, Clear: 70}}}
+ },
+ want: &HysteresisThreshold{Trigger: 75, Clear: 70},
+ },
+ {
+ name: "per-disk override without a clear value",
+ mutate: func(cfg *AlertConfig) {
+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{Trigger: 75}}}
+ },
+ want: &HysteresisThreshold{Trigger: 75, Clear: 70},
+ },
+ {
+ name: "per-disk override under the canonical identity",
+ mutate: func(cfg *AlertConfig) {
+ cfg.Overrides = map[string]ThresholdConfig{canonicalDiskID: {Temperature: &HysteresisThreshold{Trigger: 78, Clear: 73}}}
+ },
+ setup: func(m *Manager) {
+ m.SetResourceIntentIdentityResolver(func(resourceID string) (string, bool) {
+ return canonicalDiskID, resourceID == disk.ID
+ })
+ },
+ want: &HysteresisThreshold{Trigger: 78, Clear: 73},
+ },
+ {
+ name: "host override under the TrueNAS system ID does not reach its disks",
+ mutate: func(cfg *AlertConfig) {
+ cfg.Overrides = map[string]ThresholdConfig{"truenas-main": {DiskTemperature: &HysteresisThreshold{Trigger: 50, Clear: 45}}}
+ },
+ want: &HysteresisThreshold{Trigger: 70, Clear: 65},
+ },
+ {
+ name: "per-disk override switches the disk off",
+ mutate: func(cfg *AlertConfig) {
+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{}}}
+ },
+ off: true,
+ },
+ {
+ name: "per-disk override disables the disk's alerts",
+ mutate: func(cfg *AlertConfig) {
+ cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Disabled: true}}
+ },
+ },
+ {
+ name: "TrueNAS-wide off",
+ mutate: func(cfg *AlertConfig) {
+ cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{}
+ },
+ off: true,
+ },
+ {
+ name: "TrueNAS Disks defaults disabled",
+ mutate: func(cfg *AlertConfig) {
+ cfg.TrueNASDiskDefaults.Disabled = true
+ },
+ },
+ {
+ name: "agent Disk Temp default off",
+ mutate: func(cfg *AlertConfig) {
+ cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{}
+ },
+ off: true,
+ },
+ {
+ name: "Disable all TrueNAS keeps the policy",
+ mutate: func(cfg *AlertConfig) {
+ cfg.DisableAllTrueNAS = true
+ },
+ want: &HysteresisThreshold{Trigger: 70, Clear: 65},
+ silenced: true,
+ },
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ m := newTestManager(t)
+ cfg := unifiedEvalBaseConfig()
+ cfg.TrueNASDiskDefaults = ThresholdConfig{}
+ if tc.mutate != nil {
+ tc.mutate(&cfg)
+ }
+ configureUnifiedEvalManager(t, m, cfg)
+ if tc.setup != nil {
+ tc.setup(m)
+ }
+
+ got := m.TrueNASDiskTemperatureThreshold(" "+disk.ID+" ", " NVMe")
+ switch {
+ case tc.off:
+ if got == nil || got.Trigger > 0 {
+ t.Fatalf("TrueNASDiskTemperatureThreshold = %+v, want a switched-off threshold", got)
+ }
+ case (got == nil) != (tc.want == nil) || (got != nil && *got != *tc.want):
+ t.Fatalf("TrueNASDiskTemperatureThreshold = %+v, want %+v", got, tc.want)
+ }
+
+ var alert *Alert
+ fires := func(temperature int) bool {
+ reading := disk
+ meta := *disk.PhysicalDisk
+ meta.Temperature = temperature
+ reading.PhysicalDisk = &meta
+ checkTrueNASTemperatureDisk(t, m, reading)
+ var firing bool
+ alert, firing = trueNASDiskTemperatureAlert(t, m, reading)
+ return firing
+ }
+ if tc.silenced {
+ if fires(99) {
+ t.Fatalf("alert fired at 99C with TrueNAS alerts disabled, active: %v", alertKeys(m))
+ }
+ return
+ }
+ if got == nil || got.Trigger <= 0 {
+ if fires(99) {
+ t.Fatalf("alert fired at 99C on a disk the policy judges no heat for, active: %v", alertKeys(m))
+ }
+ return
+ }
+ trigger := int(got.Trigger)
+ if fires(trigger - 1) {
+ t.Fatalf("alert fired at %dC, under the policy trigger %v", trigger-1, got.Trigger)
+ }
+ if !fires(trigger) {
+ t.Fatalf("alert stayed quiet at the policy trigger %dC, active: %v", trigger, alertKeys(m))
+ }
+ // The alert recovers at the clear value Patrol reads.
+ if alert.Threshold != got.Trigger || alert.Metadata["clearThreshold"] != got.Clear {
+ t.Fatalf("alert trigger/clear = %v/%v, want the policy %v/%v", alert.Threshold, alert.Metadata["clearThreshold"], got.Trigger, got.Clear)
+ }
+ })
+ }
+
+ // A disk whose type TrueNAS never reported follows the agent Disk Temp
+ // default, as its alert does.
+ t.Run("untyped disk", func(t *testing.T) {
+ m := newTestManager(t)
+ cfg := unifiedEvalBaseConfig()
+ cfg.TrueNASDiskDefaults = ThresholdConfig{}
+ cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 58, Clear: 53}
+ configureUnifiedEvalManager(t, m, cfg)
+ untypedDisk := trueNASTemperatureDisk("sdx", "", 58)
+ if got := m.TrueNASDiskTemperatureThreshold(untypedDisk.ID, ""); got == nil || got.Trigger != 58 || got.Clear != 53 {
+ t.Fatalf("untyped disk = %+v, want the agent default 58/53", got)
+ }
+ checkTrueNASTemperatureDisk(t, m, untypedDisk)
+ if _, firing := trueNASDiskTemperatureAlert(t, m, untypedDisk); !firing {
+ t.Fatalf("untyped disk at the agent default 58C raised no alert, active: %v", alertKeys(m))
+ }
+ })
+
+ m := configureDiskTempTypeHostManager(t)
+ m.TrueNASDiskTemperatureThreshold(disk.ID, "nvme").Trigger = 1
+ if got := m.TrueNASDiskTemperatureThreshold(disk.ID, "nvme"); got == nil || got.Trigger != 70 {
+ t.Fatalf("caller mutation reached the config: %+v", got)
+ }
+ var nilManager *Manager
+ if got := nilManager.TrueNASDiskTemperatureThreshold(disk.ID, "sata"); got == nil || got.Trigger != 55 {
+ t.Fatalf("nil manager TrueNAS disk policy = %+v, want the factory sata 55", got)
+ }
+}
+
+// IsTrueNASDiskResource names the disks the unified evaluator judges as
+// TrueNAS disks, the ones TrueNASDiskTemperatureThreshold resolves.
+func TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator(t *testing.T) {
+ agentDisk := unifiedresources.Resource{
+ ID: "physical-disk:host-1/nvme0n1", Type: unifiedresources.ResourceTypePhysicalDisk,
+ Sources: []unifiedresources.DataSource{unifiedresources.SourceAgent},
+ PhysicalDisk: &unifiedresources.PhysicalDiskMeta{DiskType: "nvme"},
+ }
+ trueNASFacetDisk := agentDisk
+ trueNASFacetDisk.TrueNAS = &unifiedresources.TrueNASData{}
+ mergedDisk := agentDisk
+ mergedDisk.Sources = []unifiedresources.DataSource{unifiedresources.SourceAgent, unifiedresources.SourceTrueNAS}
+ noMetaDisk := trueNASTemperatureDisk("sdy", "sata", 40)
+ noMetaDisk.PhysicalDisk = nil
+ trueNASSystem := unifiedresources.Resource{
+ ID: "agent:truenas-main", Type: unifiedresources.ResourceTypeAgent,
+ Sources: []unifiedresources.DataSource{unifiedresources.SourceTrueNAS},
+ }
+ for name, tc := range map[string]struct {
+ resource unifiedresources.Resource
+ want bool
+ }{
+ "TrueNAS-sourced disk": {resource: trueNASTemperatureDisk("sda", "sata", 40), want: true},
+ "TrueNAS facet disk": {resource: trueNASFacetDisk, want: true},
+ "agent and TrueNAS": {resource: mergedDisk, want: true},
+ "no disk metadata": {resource: noMetaDisk, want: true},
+ "agent disk": {resource: agentDisk},
+ "TrueNAS system": {resource: trueNASSystem},
+ } {
+ if got := IsTrueNASDiskResource(tc.resource); got != tc.want {
+ t.Errorf("%s: IsTrueNASDiskResource = %v, want %v", name, got, tc.want)
+ }
+ input, ok := UnifiedResourceInputFromResource(tc.resource)
+ if evaluated := ok && input.Type == "truenas-disk"; evaluated != tc.want {
+ t.Errorf("%s: unified evaluator judges it as a TrueNAS disk = %v, want %v", name, evaluated, tc.want)
+ }
+ }
+}
diff --git a/internal/alerts/truenas_disk_temperature.go b/internal/alerts/truenas_disk_temperature.go
index 9934e83233..d7fca0f69a 100644
--- a/internal/alerts/truenas_disk_temperature.go
+++ b/internal/alerts/truenas_disk_temperature.go
@@ -1,5 +1,11 @@
package alerts
+import (
+ "strings"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
+)
+
// trueNASDiskTemperatureDefaultNoLock returns the default tier of a TrueNAS
// disk's temperature threshold. A TrueNAS-wide value the user saved under
// TrueNAS Disks applies to every TrueNAS disk, as a host Disk Temp override
@@ -16,3 +22,39 @@ func (m *Manager) trueNASDiskTemperatureDefaultNoLock(diskType string, typeUnkno
}
return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature, false, diskType))
}
+
+// IsTrueNASDiskResource reports whether the unified evaluator judges a
+// resource as a TrueNAS disk (truenas-disk), whose temperature alert resolves
+// its threshold through TrueNASDiskTemperatureThreshold rather than through
+// the host agent that reports it.
+func IsTrueNASDiskResource(resource unifiedresources.Resource) bool {
+ typeKey, ok := unifiedAlertResourceType(resource)
+ return ok && typeKey == "truenas-disk"
+}
+
+// TrueNASDiskTemperatureThreshold is the disk heat policy for one TrueNAS
+// disk, resolved by the same effectiveAlertPolicyNoLock tiers its temperature
+// alert uses: an override stored under the disk's resource ID, then the
+// TrueNAS-wide TrueNAS Disks value, then the per-type policy. An override or
+// TrueNAS Disks default that switches the disk's alerts off leaves no
+// threshold (nil); a non-positive trigger is off too. Like
+// HostDiskTemperatureThreshold it ignores the global and per-platform alert
+// switches, which silence the alert without changing the policy. A nil
+// manager resolves the factory configuration, where no TrueNAS-wide value is
+// set.
+func (m *Manager) TrueNASDiskTemperatureThreshold(resourceID, diskType string) *HysteresisThreshold {
+ if m == nil {
+ return DefaultDiskTemperatureThreshold(diskType)
+ }
+ m.mu.RLock()
+ defer m.mu.RUnlock()
+ thresholds := m.effectiveAlertPolicyNoLock(alertPolicyQuery{
+ TypeKey: "truenas-disk",
+ ResourceID: unifiedresources.CanonicalResourceID(resourceID),
+ DiskType: strings.ToLower(strings.TrimSpace(diskType)),
+ }).Thresholds
+ if thresholds.Disabled {
+ return nil
+ }
+ return cloneThreshold(thresholds.Temperature)
+}
diff --git a/internal/api/ai_handlers_concurrency_test.go b/internal/api/ai_handlers_concurrency_test.go
index 0c5276fa05..db1b4cad27 100644
--- a/internal/api/ai_handlers_concurrency_test.go
+++ b/internal/api/ai_handlers_concurrency_test.go
@@ -71,6 +71,9 @@ func (concurrencyThresholdProvider) GetStorageThreshold() float64 { return 92
func (concurrencyThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
return 70, 65
}
+func (concurrencyThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {
+ return 70, 65
+}
type concurrencyMetricsHistoryProvider struct{}
diff --git a/internal/api/ai_handlers_more_test.go b/internal/api/ai_handlers_more_test.go
index fb0397dca8..8f67b0d184 100644
--- a/internal/api/ai_handlers_more_test.go
+++ b/internal/api/ai_handlers_more_test.go
@@ -162,6 +162,9 @@ func (stubThresholdProvider) GetStorageThreshold() float64 { return 92 }
func (stubThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
return 70, 65
}
+func (stubThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {
+ return 70, 65
+}
type stubMetricsHistoryProvider struct{}