Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Judge adaptive-polling freshness by the scheduler's cadence
rcourtmandc93d0adea49
diff --git a/docs/monitoring/ADAPTIVE_POLLING.md b/docs/monitoring/ADAPTIVE_POLLING.mdindex a7036ce187..b0f7604b55 100644--- a/docs/monitoring/ADAPTIVE_POLLING.md+++ b/docs/monitoring/ADAPTIVE_POLLING.md@@ -47,13 +47,30 @@ There is currently no dedicated UI for adaptive polling. Use environment variabl| Variable | Default | Description || :--- | :--- | :--- || `ADAPTIVE_POLLING_ENABLED` | `false` | Enable/disable. |-| `ADAPTIVE_POLLING_BASE_INTERVAL` | `10s` | Healthy poll rate. |-| `ADAPTIVE_POLLING_MIN_INTERVAL` | `5s` | Active/busy rate. |-| `ADAPTIVE_POLLING_MAX_INTERVAL` | `5m` | Idle/backoff rate. |+| `ADAPTIVE_POLLING_BASE_INTERVAL` | `10s` | Starting interval before an instance has a schedule. |+| `ADAPTIVE_POLLING_MIN_INTERVAL` | `5s` | Shortest interval, approached as data goes unrefreshed. |+| `ADAPTIVE_POLLING_MAX_INTERVAL` | `5m` | Longest interval. Healthy instances approach it. |### system.jsonYou can also set `adaptivePollingEnabled` (and related interval fields) in `system.json` and restart Pulse.+### What changes when it is on+Adaptive polling replaces the Proxmox VE, PBS and PMG polling intervals+(Settings → General → Monitoring cadence, `PBS_POLLING_INTERVAL`,+`PMG_POLLING_INTERVAL`) for those platforms. Each successful poll resets an+instance's staleness score, so the scheduler stretches a healthy instance's+interval toward `ADAPTIVE_POLLING_MAX_INTERVAL`. With the defaults, healthy+instances are polled every few minutes, not every 10 seconds.++Freshness follows that cadence. A Proxmox VE, PBS or PMG resource is shown as+stale only when its last sighting from a poll is older than twice the maximum+interval (10 minutes with the defaults). A Proxmox node that stops answering can+keep showing online for the same window after it was last seen online, although+a cluster's own report that a member is offline applies immediately. Offline+nodes and lost data therefore take longer to show than with fixed polling. For a+lighter but predictable load, raise the polling intervals instead: freshness and+offline detection then scale with the interval you choose.+## 📊 MetricsThe separate metrics listener defaults to port 9091; `METRICS_PORT` can overrideit. Use an existing authorised scrape if available, not a newly exposed port fordiff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.mdindex df749fdb7f..25c9a7a177 100644--- a/docs/release-control/v6/internal/subsystems/monitoring.md+++ b/docs/release-control/v6/internal/subsystems/monitoring.md@@ -1743,9 +1743,11 @@ guest-agent reachability: fresh or never-healthy VMs with an enabled butunavailable guest agent stay `not-running`, while only VMs with recent healthyguest-agent evidence may become `expected-unreachable`.Monitoring owns source freshness cadence for Proxmox, PBS, and PMG resources:-the stale threshold is derived from the configured polling interval with a-minimum floor, so API-facing resource status must not degrade merely because a-healthy source is between normal poll cycles.+the stale threshold is derived from the cadence the monitor actually polls at+(the configured polling interval, or the adaptive scheduler's maximum interval+when adaptive polling is enabled) with a minimum floor, so API-facing resource+status must not degrade merely because a healthy source is between normal poll+cycles.Each unified view carries the thresholds its listed registry generation wasjudged by, read with the listing from the same generation: the resourcestore's configured ones, or none (the defaults) for a view built from mock@@ -2027,9 +2029,11 @@ VMID)`: it survives node migration, separates QEMU from LXC, and preventsduplicate configured cluster identities from sharing a concurrent baseline.Idle and partial samples still refresh tracker liveness.Proxmox row liveness uses the same cadence-derived threshold as source-freshness (`max(2 * configured poll interval, 60s)`). Node offline grace and-guest preservation must not expire between healthy 60- or 90-second polls, and-must not use a separate fixed 60-second timer.+freshness (`max(2 * poll cadence, 60s)`, the cadence being the configured poll+interval or, under adaptive polling, the scheduler's maximum interval). Node+offline grace and guest preservation must not expire between healthy 60- or+90-second polls, or between adaptive polls, and must not use a separate fixed+60-second timer.Tenant monitor enumeration is monitoring-owned runtime topology, not areporting source of truth. `MultiTenantMonitor.ListOrganizationIDs` may exposepersisted organization IDs to API-owned background workers, but it must not@@ -2764,14 +2768,40 @@ service-history reads plus denial/recovery without fabricated samples.overrides. Under fixed-cadence scheduling, `Monitor.resourceStaleThresholds`reads those overrides through the same clamped setting readers as thescheduler, so a saved interval moves the monitor's resource freshness and- polling together. An adaptive scheduler selects its own intervals and- ignores those overrides, so freshness there keeps deriving from the- configured per-platform intervals. A PVE interval save reloads every- monitor from saved config instead. `ResourceStaleThresholdsForConfig` stays- the config-only derivation for callers without a live monitor, such as- adapter construction. Regression coverage:+ polling together. A PVE interval save reloads every monitor from saved+ config instead. `ResourceStaleThresholdsForConfig` stays the config-only+ derivation for callers without a live monitor, such as adapter+ construction. Regression coverage:`TestMonitorResourceStaleThresholdsFollowRuntimePollingOverrides` in`internal/monitoring/canonical_guardrails_test.go`.+ An adaptive scheduler (`ADAPTIVE_POLLING_ENABLED` or `system.json`; off by+ default, no UI) selects its own intervals and never reads the per-platform+ intervals or those overrides. A healthy instance's staleness score is near+ zero after each success, so its cadence stretches toward the scheduler's+ maximum interval (5 minutes by default; a healthy instance polls a few+ minutes apart). Selected intervals never exceed it; a poll can still run+ later than its slot while workers are busy; the factor of two gives+ headroom for that. Under adaptive scheduling+ `Monitor.resourcePollIntervals` therefore derives PVE, PBS and PMG+ freshness from `AdaptiveScheduler.MaxInterval()`, and+ `ResourceStaleThresholdsForConfig` derives it from the+ `AdaptivePolling*Interval` bounds normalized by `normalizedSchedulerConfig`,+ the normalization the scheduler itself applies. Deriving it from the+ per-platform intervals (1 minute for PBS and PMG, 10 seconds for PVE) left+ healthy rows stale for much of every cycle. The Proxmox threshold is also+ the window the node grace helpers (`determineNodeIDAndStatus`,+ `preserveOrExpireNodes`) hold a node online after it was last seen online,+ and the node temperature carry window derives from the same cadence with+ its five-minute floor. With the defaults that hold is 10 minutes, observed+ at the next poll after it lapses: a longer detection delay than fixed+ polling, which is the price of polling a healthy fleet every few minutes.+ A valid `/cluster/status` membership report that a member is offline is+ authoritative (`reconcilePVENodeInventory`) and still applies immediately.+ Regression coverage:+ `TestMonitorResourceFreshnessFollowsAdaptiveCadence`,+ `TestResourceStaleThresholdsForConfigFollowAdaptiveBounds` and+ `TestAdaptiveNodeOfflineGraceSpansTheAdaptiveCycle` in+ `internal/monitoring/adaptive_resource_freshness_test.go`.Periodic out-of-scheduler platform pollers (TrueNAS, VMware) share theirlifecycle and config-resolution scaffold through`internal/monitoring/platform_poller_shared.go`: `startPollerLoop` owns@@ -5072,7 +5102,8 @@ reading with that report time. Past the lease the lookup falls back to thecluster sensor cache, which keeps its own recency check. When every sourcereturns nothing, `internal/monitoring/monitor_polling_node_helpers.go` may carrya previous reading, with its original `LastUpdate`, only inside the carry window-(twice the PVE polling interval, never under five minutes); an older reading is+(twice the PVE polling cadence, which under adaptive polling is the+scheduler's maximum interval, never under five minutes); an older reading isdropped rather than re-presented as current. A carried reading also keeps alapsed host agent's lease. Agent readings are stamped with the agent's reporttime, so `carriedTemperatureOutlivesAgentLease` drops a carried reading stampeddiff --git a/internal/monitoring/adaptive_resource_freshness_test.go b/internal/monitoring/adaptive_resource_freshness_test.gonew file mode 100644index 0000000000..87c99343f6--- /dev/null+++ b/internal/monitoring/adaptive_resource_freshness_test.go@@ -0,0 +1,234 @@+package monitoring++import (+ "testing"+ "time"++ "github.com/rcourtman/pulse-go-rewrite/internal/config"+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"+ "github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"+)++// simulateAdaptiveSuccessGaps drives the real adaptive scheduler, interval+// selector and staleness scoring for one always-healthy instance on a+// simulated clock, the way the monitor does: a worker runs the queued task+// once it is due and reschedules it, and a planning pass re-plans every poll+// tick. It returns the gaps between successive successful polls after warmup.+// The worker is instantaneous and uncontended, so real gaps can run longer+// when workers are busy; the factor of two in the threshold is headroom for+// that, not a bound.+func simulateAdaptiveSuccessGaps(t *testing.T, cfg SchedulerConfig, instanceType InstanceType, tick, warmup, horizon time.Duration) []time.Duration {+ t.Helper()+ const name = "instance-1"+ tracker := NewStalenessTracker(nil)+ tracker.SetBounds(cfg.BaseInterval, cfg.MaxInterval)+ scheduler := NewAdaptiveScheduler(cfg, tracker, nil, nil)++ start := time.Unix(1_800_000_000, 0)+ simNow := start+ var lastSuccess time.Time+ plan := func(desc InstanceDescriptor) ScheduledTask {+ // StalenessTracker scores against the wall clock, so stamp the+ // simulated age of the last success just before planning.+ if !lastSuccess.IsZero() {+ tracker.setSnapshot(FreshnessSnapshot{+ InstanceType: instanceType,+ Instance: name,+ LastSuccess: time.Now().Add(-simNow.Sub(lastSuccess)),+ })+ }+ tasks := scheduler.BuildPlan(simNow, []InstanceDescriptor{desc}, 0)+ if len(tasks) != 1 {+ t.Fatalf("BuildPlan returned %d tasks, want 1", len(tasks))+ }+ return tasks[0]+ }+ planningPass := func() ScheduledTask {+ desc := InstanceDescriptor{Name: name, Type: instanceType, LastSuccess: lastSuccess}+ if last, ok := scheduler.LastScheduled(instanceType, name); ok {+ desc.LastScheduled = last.NextRun+ desc.LastInterval = last.Interval+ }+ return plan(desc)+ }++ queued := planningPass()+ var gaps []time.Duration+ for step := time.Duration(0); step <= horizon; step += time.Second {+ simNow = start.Add(step)+ if !simNow.Before(queued.NextRun) {+ if !lastSuccess.IsZero() && step >= warmup {+ gaps = append(gaps, simNow.Sub(lastSuccess))+ }+ lastSuccess = simNow+ queued = plan(InstanceDescriptor{+ Name: name,+ Type: instanceType,+ LastInterval: queued.Interval,+ LastScheduled: queued.NextRun,+ LastSuccess: lastSuccess,+ })+ }+ if step > 0 && step%tick == 0 {+ queued = planningPass()+ }+ }+ if len(gaps) == 0 {+ t.Fatalf("%s: no polls after warmup", instanceType)+ }+ return gaps+}++func longestGap(gaps []time.Duration) time.Duration {+ var longest time.Duration+ for _, gap := range gaps {+ longest = max(longest, gap)+ }+ return longest+}++// With adaptive polling enabled, a healthy instance's staleness score is near+// zero after every success, so the scheduler stretches its cadence toward+// AdaptivePollingMaxInterval and never reads the per-platform intervals.+// Freshness, node offline grace and the temperature carry window must cover+// that cadence, or healthy Proxmox, PBS and PMG rows read stale (and nodes+// lose their grace) for most of every cycle.+func TestMonitorResourceFreshnessFollowsAdaptiveCadence(t *testing.T) {+ sources := map[InstanceType]unifiedresources.DataSource{+ InstanceTypePVE: unifiedresources.SourceProxmox,+ InstanceTypePBS: unifiedresources.SourcePBS,+ InstanceTypePMG: unifiedresources.SourcePMG,+ }+ for _, bounds := range []SchedulerConfig{+ {BaseInterval: 10 * time.Second, MinInterval: 5 * time.Second, MaxInterval: 5 * time.Minute},+ {BaseInterval: 30 * time.Second, MinInterval: 10 * time.Second, MaxInterval: 15 * time.Minute},+ } {+ monitor := &Monitor{+ config: &config.Config{+ PVEPollingInterval: 10 * time.Second,+ PBSPollingInterval: time.Minute,+ PMGPollingInterval: time.Minute,+ },+ scheduler: NewAdaptiveScheduler(bounds, nil, nil, nil),+ }+ thresholds := monitor.resourceStaleThresholds()+ for instanceType, source := range sources {+ gaps := simulateAdaptiveSuccessGaps(t, bounds, instanceType, 10*time.Second, time.Hour, 8*time.Hour)+ longest := longestGap(gaps)+ if longest > bounds.MaxInterval+time.Second {+ t.Fatalf("%s: healthy polls %v apart exceed the scheduler's %v maximum", instanceType, longest, bounds.MaxInterval)+ }+ if got, want := thresholds[source], 2*bounds.MaxInterval; got != want {+ t.Errorf("max %v: %s threshold = %v, want %v from the adaptive cadence (healthy polls up to %v apart)",+ bounds.MaxInterval, source, got, want, longest)+ }+ if instanceType != InstanceTypePVE {+ continue+ }+ if grace := monitor.pveNodeOfflineGracePeriod(); grace <= longest {+ t.Errorf("max %v: node offline grace %v expires before the next healthy poll (%v apart)", bounds.MaxInterval, grace, longest)+ }+ if carry, want := monitor.nodeTemperatureCarryWindow(), max(2*bounds.MaxInterval, nodeTemperatureCarryFloor); carry != want || carry <= longest {+ t.Errorf("max %v: temperature carry window = %v, want %v from the adaptive cadence (healthy polls up to %v apart)", bounds.MaxInterval, carry, want, longest)+ }+ }+ }+}++// A node the API reports offline is only believed once it has been away+// longer than one healthy adaptive cycle; past twice the cadence the grace+// lapses as it does under fixed-cadence polling.+func TestAdaptiveNodeOfflineGraceSpansTheAdaptiveCycle(t *testing.T) {+ instance := &config.PVEInstance{Name: "site-a"}+ monitor := &Monitor{+ config: &config.Config{PVEPollingInterval: 10 * time.Second},+ scheduler: NewAdaptiveScheduler(SchedulerConfig{BaseInterval: 10 * time.Second, MinInterval: 5 * time.Second, MaxInterval: 5 * time.Minute}, nil, nil, nil),+ nodeLastOnline: map[string]time.Time{"site-a-pve-a": time.Now().Add(-4 * time.Minute)},+ }+ offline := proxmox.Node{Node: "pve-a", Status: "offline"}+ if _, status := monitor.determineNodeIDAndStatus("site-a", instance, offline); status != "online" {+ t.Fatalf("node status = %q one adaptive cycle after its last online poll, want online inside the grace", status)+ }+ monitor.nodeLastOnline["site-a-pve-a"] = time.Now().Add(-11 * time.Minute)+ if _, status := monitor.determineNodeIDAndStatus("site-a", instance, offline); status != "offline" {+ t.Fatalf("node status = %q past twice the adaptive cadence, want offline", status)+ }+}++// Callers without a live monitor (adapter construction) derive freshness from+// config alone. It must name the same cadence the monitor's scheduler will+// run, including the defaults the scheduler fills in for unset or inconsistent+// adaptive bounds.+func TestResourceStaleThresholdsForConfigFollowAdaptiveBounds(t *testing.T) {+ cases := []struct {+ name string+ cfg config.Config+ want time.Duration+ }{+ {+ name: "adaptive off keeps the per-platform intervals",+ cfg: config.Config{+ PVEPollingInterval: 10 * time.Second, PBSPollingInterval: 3 * time.Minute, PMGPollingInterval: 3 * time.Minute,+ AdaptivePollingMaxInterval: 15 * time.Minute,+ },+ },+ {+ name: "configured maximum",+ cfg: config.Config{+ AdaptivePollingEnabled: true,+ PBSPollingInterval: 3 * time.Minute,+ AdaptivePollingMinInterval: 5 * time.Second, AdaptivePollingMaxInterval: 15 * time.Minute,+ },+ want: 30 * time.Minute,+ },+ {+ name: "unset bounds use the scheduler default",+ cfg: config.Config{AdaptivePollingEnabled: true},+ want: 2 * DefaultSchedulerConfig().MaxInterval,+ },+ {+ name: "maximum below the minimum uses the scheduler default",+ cfg: config.Config{+ AdaptivePollingEnabled: true,+ AdaptivePollingMinInterval: time.Minute, AdaptivePollingMaxInterval: 30 * time.Second,+ },+ want: 2 * DefaultSchedulerConfig().MaxInterval,+ },+ }+ for _, tc := range cases {+ t.Run(tc.name, func(t *testing.T) {+ cfg := tc.cfg+ thresholds := ResourceStaleThresholdsForConfig(&cfg)+ for _, source := range []unifiedresources.DataSource{unifiedresources.SourceProxmox, unifiedresources.SourcePBS, unifiedresources.SourcePMG} {+ want := tc.want+ if !cfg.AdaptivePollingEnabled {+ want = map[unifiedresources.DataSource]time.Duration{+ unifiedresources.SourceProxmox: defaultProxmoxResourceStaleThreshold,+ unifiedresources.SourcePBS: 6 * time.Minute,+ unifiedresources.SourcePMG: 6 * time.Minute,+ }[source]+ }+ if got := thresholds[source]; got != want {+ t.Errorf("%s threshold = %v, want %v", source, got, want)+ }+ }+ if !cfg.AdaptivePollingEnabled {+ return+ }+ scheduler := NewAdaptiveScheduler(SchedulerConfig{+ BaseInterval: cfg.AdaptivePollingBaseInterval,+ MinInterval: cfg.AdaptivePollingMinInterval,+ MaxInterval: cfg.AdaptivePollingMaxInterval,+ }, nil, nil, nil)+ monitor := &Monitor{config: &cfg, scheduler: scheduler}+ for source, want := range thresholds {+ if source != unifiedresources.SourceProxmox && source != unifiedresources.SourcePBS && source != unifiedresources.SourcePMG {+ continue+ }+ if got := monitor.resourceStaleThresholds()[source]; got != want {+ t.Errorf("%s: monitor threshold %v differs from the config-only %v", source, got, want)+ }+ }+ })+ }+}diff --git a/internal/monitoring/canonical_guardrails_test.go b/internal/monitoring/canonical_guardrails_test.goindex 7d409f0fc0..d1ee720c87 100644--- a/internal/monitoring/canonical_guardrails_test.go+++ b/internal/monitoring/canonical_guardrails_test.go@@ -3078,22 +3078,26 @@ func TestMonitorResourceStaleThresholdsFollowRuntimePollingOverrides(t *testing.})// The adaptive scheduler picks intervals from its own bounds and never- // reads the per-platform overrides, so a save must not move freshness- // there; following adaptive cadence is a separate derivation.+ // reads the per-platform intervals or overrides, so a save must not move+ // freshness there: it follows the scheduler's maximum interval, which the+ // 5-minute per-platform interval below would not give (10 minutes).+ // TestMonitorResourceFreshnessFollowsAdaptiveCadence covers the cadence+ // itself.t.Run("adaptive scheduling ignores per-platform overrides", func(t *testing.T) {base := &config.Config{PBSPollingInterval: 5 * time.Minute, PMGPollingInterval: 5 * time.Minute}monitor := &Monitor{config: base.DeepCopy(),- scheduler: NewAdaptiveScheduler(SchedulerConfig{}, nil, nil, nil),+ scheduler: NewAdaptiveScheduler(SchedulerConfig{MaxInterval: 15 * time.Minute}, nil, nil, nil),}+ const want = 30 * time.Minutebefore := monitor.resourceStaleThresholds()monitor.SetPBSPollingInterval(time.Minute)monitor.SetPMGPollingInterval(time.Minute)after := monitor.resourceStaleThresholds()- for _, source := range []unifiedresources.DataSource{unifiedresources.SourcePBS, unifiedresources.SourcePMG} {- if before[source] != 10*time.Minute || after[source] != before[source] {- t.Errorf("%s threshold = %v -> %v across an override the adaptive scheduler ignores, want %v throughout",- source, before[source], after[source], 10*time.Minute)+ for _, source := range []unifiedresources.DataSource{unifiedresources.SourcePBS, unifiedresources.SourcePMG, unifiedresources.SourceProxmox} {+ if before[source] != want || after[source] != want {+ t.Errorf("%s threshold = %v -> %v across an override the adaptive scheduler ignores, want %v from its 15m maximum throughout",+ source, before[source], after[source], want)}}})diff --git a/internal/monitoring/monitor_polling_node_helpers.go b/internal/monitoring/monitor_polling_node_helpers.goindex dd5e9abba9..4a012cf4b0 100644--- a/internal/monitoring/monitor_polling_node_helpers.go+++ b/internal/monitoring/monitor_polling_node_helpers.go@@ -136,9 +136,11 @@ const nodeTemperatureCarryFloor = 5 * time.Minute// nodeTemperatureCarryWindow bounds how old a carried-over node temperature may// be. Beyond it the reading no longer describes the node, so it stops being-// presented as available.+// presented as available. It follows the PVE cadence the resource freshness+// and node offline grace use, so a node preserved inside that grace keeps a+// reading from its last healthy poll.func (m *Monitor) nodeTemperatureCarryWindow() time.Duration {- return resourceStaleThresholdForPollInterval(effectivePVEPollingIntervalForConfig(m.config), nodeTemperatureCarryFloor)+ return resourceStaleThresholdForPollInterval(m.resourcePollIntervals().pve, nodeTemperatureCarryFloor)}func (m *Monitor) collectNodeTemperatureData(diff --git a/internal/monitoring/resource_stale_thresholds.go b/internal/monitoring/resource_stale_thresholds.goindex 92e1468572..b0c2c8af5e 100644--- a/internal/monitoring/resource_stale_thresholds.go+++ b/internal/monitoring/resource_stale_thresholds.go@@ -21,7 +21,25 @@ type resourcePollIntervals struct {pmg time.Duration}+// adaptiveResourcePollIntervals are the cadences freshness is derived from when+// an adaptive scheduler plans polling. It selects its own intervals and never+// reads the per-platform ones; a healthy instance's staleness score is near+// zero after every success, so its cadence stretches toward the scheduler's+// maximum interval, which bounds it. Freshness must cover that gap, or+// healthy Proxmox, PBS and PMG rows read stale, and Proxmox nodes lose their+// offline grace, for much of every cycle.+func adaptiveResourcePollIntervals(maxInterval time.Duration) resourcePollIntervals {+ return resourcePollIntervals{pve: maxInterval, pbs: maxInterval, pmg: maxInterval}+}+func resourcePollIntervalsForConfig(cfg *config.Config) resourcePollIntervals {+ if cfg != nil && cfg.AdaptivePollingEnabled {+ return adaptiveResourcePollIntervals(normalizedSchedulerConfig(SchedulerConfig{+ BaseInterval: cfg.AdaptivePollingBaseInterval,+ MinInterval: cfg.AdaptivePollingMinInterval,+ MaxInterval: cfg.AdaptivePollingMaxInterval,+ }).MaxInterval)+ }return resourcePollIntervals{pve: effectivePVEPollingIntervalForConfig(cfg),pbs: effectivePlatformPollingIntervalForConfig(cfg, "pbs"),@@ -30,10 +48,12 @@ func resourcePollIntervalsForConfig(cfg *config.Config) resourcePollIntervals {}// ResourceStaleThresholdsForConfig derives canonical resource freshness from-// configured polling cadence. A source should not be considered stale until it-// has missed at least one expected poll cycle plus the normal interval. A-// running monitor judges by Monitor.resourceStaleThresholds instead, which-// honours the runtime polling overrides its scheduler reads.+// configured polling cadence: the per-platform intervals, or the adaptive+// scheduler's maximum interval when adaptive polling is enabled. A source+// should not be considered stale until it has missed at least one expected+// poll cycle plus the normal interval. A running monitor judges by+// Monitor.resourceStaleThresholds instead, which honours the runtime polling+// overrides its scheduler reads.func ResourceStaleThresholdsForConfig(cfg *config.Config) map[unifiedresources.DataSource]time.Duration {return resourceStaleThresholdsForConfig(cfg, mock.IsMockEnabled(), mock.SupplementalRefreshInterval)}@@ -126,18 +146,19 @@ func (m *Monitor) resourceStaleThresholds() map[unifiedresources.DataSource]time)}-// resourcePollIntervals reads the per-platform base cadences, honouring the-// runtime overrides a fixed-cadence scheduler polls at. Non-default tenant-// monitors poll against a detached config copy (#1619), so the settings API-// pushes saved PBS and PMG intervals into them as runtime overrides; freshness-// derived from m.config alone would judge those sources by an interval nothing-// polls at. A PVE interval change reloads every monitor from saved config-// instead, so the config value is already the live one. An adaptive scheduler-// selects its own intervals and never reads the per-platform overrides, so-// they must not move freshness there either.+// resourcePollIntervals reads the cadences the monitor's scheduler polls at.+// A fixed-cadence scheduler polls each platform at its base interval,+// honouring the runtime overrides: non-default tenant monitors poll against a+// detached config copy (#1619), so the settings API pushes saved PBS and PMG+// intervals into them as overrides, and freshness derived from m.config alone+// would judge those sources by an interval nothing polls at. A PVE interval+// change reloads every monitor from saved config instead, so the config value+// is already the live one. An adaptive scheduler never reads the per-platform+// intervals or their overrides, so they cannot move freshness there; it is+// judged by the longest interval that scheduler selects.func (m *Monitor) resourcePollIntervals() resourcePollIntervals {if m.scheduler != nil {- return resourcePollIntervalsForConfig(m.config)+ return adaptiveResourcePollIntervals(m.scheduler.MaxInterval())}return resourcePollIntervals{pve: m.BasePollInterval(InstanceTypePVE),diff --git a/internal/monitoring/scheduler.go b/internal/monitoring/scheduler.goindex a715eb1be0..d034cf380d 100644--- a/internal/monitoring/scheduler.go+++ b/internal/monitoring/scheduler.go@@ -110,8 +110,11 @@ type AdaptiveScheduler struct {lastPlan map[string]ScheduledTask}-// NewAdaptiveScheduler constructs a scheduler with safe defaults.-func NewAdaptiveScheduler(cfg SchedulerConfig, staleness StalenessSource, interval IntervalSelector, enqueuer TaskEnqueuer) *AdaptiveScheduler {+// normalizedSchedulerConfig fills unset or inconsistent bounds with the+// defaults, as the scheduler does for the config it is given. Anything that+// derives from the intervals a scheduler will select reads this, so it agrees+// with the running scheduler on a partial or invalid configuration.+func normalizedSchedulerConfig(cfg SchedulerConfig) SchedulerConfig {if cfg.BaseInterval <= 0 {cfg.BaseInterval = DefaultSchedulerConfig().BaseInterval}@@ -121,6 +124,12 @@ func NewAdaptiveScheduler(cfg SchedulerConfig, staleness StalenessSource, intervif cfg.MaxInterval <= 0 || cfg.MaxInterval < cfg.MinInterval {cfg.MaxInterval = DefaultSchedulerConfig().MaxInterval}+ return cfg+}++// NewAdaptiveScheduler constructs a scheduler with safe defaults.+func NewAdaptiveScheduler(cfg SchedulerConfig, staleness StalenessSource, interval IntervalSelector, enqueuer TaskEnqueuer) *AdaptiveScheduler {+ cfg = normalizedSchedulerConfig(cfg)if staleness == nil {staleness = noopStalenessSource{}}@@ -288,6 +297,17 @@ func (s *AdaptiveScheduler) DispatchDue(ctx context.Context, now time.Time, taskreturn due}+// MaxInterval is the longest interval the scheduler selects for an instance+// without a fixed interval. A healthy instance's staleness score is near zero+// right after each successful poll, so its cadence stretches toward this+// bound; freshness judged by anything shorter flags healthy instances stale.+func (s *AdaptiveScheduler) MaxInterval() time.Duration {+ if s == nil {+ return 0+ }+ return s.cfg.MaxInterval+}+// LastScheduled returns the last recorded task for the given instance, if any.func (s *AdaptiveScheduler) LastScheduled(instanceType InstanceType, instanceName string) (ScheduledTask, bool) {if s == nil {