Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Record Proxmox and agent alert history on the canonical resource
rcourtman474dbc4f0997
diff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.mdindex 6522d39af6..5e550c5cb0 100644--- a/docs/release-control/v6/internal/subsystems/monitoring.md+++ b/docs/release-control/v6/internal/subsystems/monitoring.md@@ -1043,6 +1043,15 @@ join another container's history. The real alert-manager callback path iscovered by `TestDockerAlertTimelineUsesCanonicalHistoryIdentity` in`internal/monitoring/monitor_alert_handling_test.go`.+Proxmox node, guest and storage alert lifecycle events carry source-native IDs+and pass through the same writer, which resolves them to the canonical+resource. Resource facets, resource intelligence and Assistant resource context+therefore include their alert history, including events emitted after the+resource left inventory. Event IDs still hash the alert's source reference, so+replay stays idempotent. The real alert-manager callback path is covered by+`TestProxmoxAlertTimelineUsesCanonicalHistoryIdentity` in+`internal/monitoring/monitor_alert_handling_test.go`.+TrueNAS native alert projection preserves the trimmed, uppercase provider level in ResourceIncident.NativeSeverity. INFO and NOTICE retain the same canonical monitor risk; consumers must not lose their distinct actionability when projecting provider evidence. Native CRITICAL, ALERT, and EMERGENCY all project to canonical critical severity; EMERGENCY must not be discarded as unknown or make a still-active condition appear recovered. WARNING remains warning, and INFO and NOTICE remain informational at this projection boundary.Verification: `TestIncidentProjectionPreservesNativeSeverity` in `internal/truenas/provider_pool_health_contract_test.go` covers all seven native levels and case/whitespace normalization. `TestTrueNASNativeSeverityDispatch` in `internal/alerts/truenas_native_dispatch_test.go` verifies downstream INFO suppression, NOTICE preservation, notification severity, duplicate-poll retention, and confirmed recovery callback identity. The TrueNAS lifecycle tests in `internal/alerts/unified_incidents_test.go` require repeated EMERGENCY evidence to interrupt recovery confirmation. These are fixture-based projection and manager checks, not appliance ingestion or external notification-provider receipt proof.diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.mdindex 5c1dfdb1bb..8b80254b40 100644--- a/docs/release-control/v6/internal/subsystems/unified-resources.md+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md@@ -5838,6 +5838,29 @@ action requests, approvals, links or exclusions. Separate monitor, API andAssistant store handles must see current persisted aliases. Missing identitystorage is an error, not evidence of empty history. Retention removes an aliasonly after neither identity has retained journal records.+Other alert source references resolve at `MonitorAdapter.RecordChange`+through `ResourceRegistry.resolveHistoryReference`, which accepts only durable+identities: the canonical ID, a retired era, a source ID (Proxmox node, guest+and storage IDs), a node-scoped Proxmox guest reference after migration, or+`agent:<host ID>`. Names and hostnames never bind history, unlike the wider+`ResolveReferenceID` used by alert policy, so a resource named like a system+reference cannot capture its events. A reference two resources answer to binds+to neither: its event keeps its own reference and creates or changes no+binding, though an earlier binding still covers that reference on read. A+reference no resource+answers to follows its retained binding after inventory removal, or keeps its+own history. Docker references never take this path. Alert journal references+without a binding (read once per process, plus any written before inventory+named their resource) are retried on each published registry generation for+`legacyHistoryBindWindow`. Bound rows keep their recorded resource ID, so a+change list may show the source reference on rows written before the binding.+Only the full-presentation `ResourceDetailDrawer` reads this history in the UI+(facets and resource intelligence); the `table-row` drawer used by Proxmox node+and Agents rows, and the guest and storage drawers, do not request it.+Assistant and Patrol resource contexts read it from the store. Proof:+`TestHistoryIdentityMonitorAdapterResolvesProxmoxAlertReferences` and+`TestHistoryIdentityBindsLegacyAlertRowsFromRegistryGenerations` in+`internal/unifiedresources/history_identity_test.go`.That same shared timeline vocabulary now includes the `activity` change kindfor provider-read breadcrumbs such as VMware tasks and events, plus the`vmware_adapter` source-adapter token for canonical provenance drill-down.diff --git a/internal/monitoring/monitor_alert_handling_test.go b/internal/monitoring/monitor_alert_handling_test.goindex 8ecdeb2250..d45c8f1937 100644--- a/internal/monitoring/monitor_alert_handling_test.go+++ b/internal/monitoring/monitor_alert_handling_test.go@@ -92,6 +92,64 @@ func TestDockerAlertTimelineUsesCanonicalHistoryIdentity(t *testing.T) {t.Logf("DOCKER_HISTORY_LIFECYCLE %s", encoded)}+// Proxmox node and guest alerts carry source-native IDs. Their lifecycle must+// reach the canonical resource history that facets, the drawer and the+// assistant read, including events emitted after the resource left inventory.+func TestProxmoxAlertTimelineUsesCanonicalHistoryIdentity(t *testing.T) {+ store, err := unifiedresources.NewSQLiteResourceStore(t.TempDir(), "default")+ require.NoError(t, err)+ t.Cleanup(func() { require.NoError(t, store.Close()) })+ manager := alerts.NewManagerWithDataDir(t.TempDir())+ t.Cleanup(manager.Stop)+ config := manager.GetConfig()+ config.Enabled = true+ config.ActivationState = alerts.ActivationPending+ config.TimeThresholds = map[string]int{"node": 0, "guest": 0}+ config.SuppressionWindow = 0+ manager.UpdateConfig(config)+ now := time.Now()+ node := models.Node{ID: "lab-pve1", Name: "pve1", Instance: "lab", Host: "https://pve1.lab:8006", Status: "online", CPU: 0.99, LastSeen: now}+ vm := models.VM{ID: "lab:pve1:101", VMID: 101, Name: "web", Node: "pve1", Instance: "lab", Status: "stopped", Type: "qemu", LastSeen: now}+ registry := unifiedresources.NewRegistry(store)+ registry.IngestSnapshot(models.StateSnapshot{Nodes: []models.Node{node}, VMs: []models.VM{vm}})+ monitor := &Monitor{alertManager: manager, resourceStore: unifiedresources.NewMonitorAdapter(registry)}+ manager.SubscribeLifecycleCallback(monitor.handleAlertLifecycleEvent)+ manager.CheckNode(node)+ manager.CheckGuest(vm, vm.Instance)+ manager.CheckGuest(vm, vm.Instance)++ canonical := map[string]string{}+ for _, resource := range registry.List() {+ canonical[resource.Name] = resource.ID+ }+ require.NotEqual(t, node.ID, canonical["pve1"])+ require.NotEqual(t, vm.ID, canonical["web"])+ filters := unifiedresources.ResourceChangeFilters{Kinds: []unifiedresources.ChangeKind{unifiedresources.ChangeAlertFired, unifiedresources.ChangeAlertResolved}}+ for name, sourceID := range map[string]string{"pve1": node.ID, "web": vm.ID} {+ changes, err := store.GetRecentChangesFiltered(canonical[name], time.Time{}, 10, filters)+ require.NoError(t, err)+ require.Len(t, changes, 1, name)+ require.Equal(t, unifiedresources.ChangeAlertFired, changes[0].Kind)+ require.Equal(t, canonical[name], changes[0].ResourceID)+ kinds, err := store.CountRecentChangesByKind(canonical[name], time.Time{})+ require.NoError(t, err)+ require.Equal(t, 1, kinds[unifiedresources.ChangeAlertFired], name)+ legacy, err := store.GetRecentChangesFiltered(sourceID, time.Time{}, 10, filters)+ require.NoError(t, err)+ require.Equal(t, changes, legacy, "the source reference reads the same history")+ }++ // The node recovers after it left inventory; its retained binding holds.+ monitor.resourceStore = unifiedresources.NewMonitorAdapter(unifiedresources.NewRegistry(store))+ node.CPU = 0.05+ manager.CheckNode(node)+ changes, err := store.GetRecentChangesFiltered(canonical["pve1"], time.Time{}, 10, filters)+ require.NoError(t, err)+ require.Len(t, changes, 2)+ require.Equal(t, unifiedresources.ChangeAlertResolved, changes[0].Kind)+ require.Equal(t, canonical["pve1"], changes[0].ResourceID)+}+func TestMonitor_HandleAlertFired_Extra(t *testing.T) {// 1. Alert is nilm1 := &Monitor{}diff --git a/internal/unifiedresources/history_identity.go b/internal/unifiedresources/history_identity.goindex e71c71a2c4..9dc69d886e 100644--- a/internal/unifiedresources/history_identity.go+++ b/internal/unifiedresources/history_identity.go@@ -4,7 +4,10 @@ import ("database/sql""encoding/hex""fmt"+ "log""strings"+ "sync"+ "time")// legacyDockerHistoryIdentity accepts the source identifier emitted by Docker@@ -29,6 +32,61 @@ func legacyDockerHistoryIdentity(ref string) (sourceID, canonicalID string, ok breturn sourceID, SourceSpecificID(ResourceTypeAppContainer, SourceDocker, sourceID), true}+// isDockerHistoryReference marks Docker alert source references, which only a+// complete container ID binds (legacyDockerHistoryIdentity). A container name+// or shortened ID must never join history through general reference matching.+func isDockerHistoryReference(ref string) bool {+ return strings.HasPrefix(strings.TrimSpace(ref), "docker:")+}++// resolveHistoryReference resolves a non-Docker alert reference to the+// resource whose history it joins. Only durable identities qualify: the+// canonical ID, a retired era, a source ID (Proxmox node, guest and storage+// IDs), a node-scoped Proxmox guest reference, or an agent alert reference+// ("agent:<host ID>"). Names and hostnames never bind history, so a resource+// named like a system reference cannot capture its events. claimed reports+// that inventory answers to the reference, possibly ambiguously; an ambiguous+// reference resolves to nothing and must not follow a retained binding.+func (rr *ResourceRegistry) resolveHistoryReference(ref string) (resourceID string, claimed bool) {+ ref = CanonicalResourceID(ref)+ if rr == nil || ref == "" || isDockerHistoryReference(ref) {+ return "", false+ }+ rr.mu.RLock()+ defer rr.mu.RUnlock()+ if rr.resources[ref] != nil {+ return ref, true+ }+ if id := rr.supersededResourceIDLocked(ref); rr.resources[id] != nil {+ return id, true+ }+ matches := make(map[string]struct{}, 1)+ for _, mapping := range rr.bySource {+ if id := mapping[ref]; rr.resources[id] != nil {+ matches[id] = struct{}{}+ }+ }+ if hostID, ok := strings.CutPrefix(ref, "agent:"); ok && hostID != "" && !strings.Contains(hostID, "/") {+ if id := rr.bySource[SourceAgent][hostID]; rr.resources[id] != nil {+ matches[id] = struct{}{}+ }+ }+ if instance, _, vmid, ok := ParseProxmoxGuestSourceID(ref); ok {+ for _, resourceType := range []ResourceType{ResourceTypeVM, ResourceTypeSystemContainer} {+ if id := ProxmoxGuestCanonicalID(resourceType, instance, vmid); rr.resources[id] != nil {+ matches[id] = struct{}{}+ }+ }+ }+ if len(matches) != 1 {+ return "", len(matches) > 1+ }+ for id := range matches {+ resourceID = id+ }+ return resourceID, true+}+// resourceHistoryIdentityWriter binds a source reference to the canonical// resource of one event. It affects history lookup only, never operator state,// action requests, approvals or execution identities.@@ -90,6 +148,136 @@ func (m *MemoryStore) RecordChangeWithSourceIdentity(change ResourceChange, sourreturn m.recordChangeLocked(change)}+// legacyHistoryBindWindow bounds how long an unresolved alert journal+// reference is retried. Inventory normally names a resource within a few poll+// cycles; a reference still unresolved after the window names a removed+// resource or a sub-resource and keeps its own history.+const legacyHistoryBindWindow = 30 * time.Minute++// legacyHistoryBinder exposes alert journal references that no history binding+// covers. Rows recorded before their reference resolved join canonical history+// through a binding; the rows themselves are never rewritten.+type legacyHistoryBinder interface {+ unboundHistoryReferences() ([]string, error)+ bindHistorySourceIdentities(bindings map[string]string) error+}++// legacyHistoryBackfill holds unbound references with their retry deadlines:+// those already journaled when the process started, and those RecordChange+// wrote before inventory could name their resource.+type legacyHistoryBackfill struct {+ mu sync.Mutex+ loaded bool+ pending map[string]time.Time+}++func (b *legacyHistoryBackfill) addLocked(ref string, now time.Time) {+ if b.pending == nil {+ b.pending = make(map[string]time.Time)+ }+ if _, ok := b.pending[ref]; !ok {+ b.pending[ref] = now.Add(legacyHistoryBindWindow)+ }+}++func (b *legacyHistoryBackfill) add(ref string, now time.Time) {+ b.mu.Lock()+ defer b.mu.Unlock()+ b.addLocked(ref, now)+}++// unboundHistoryReferences lists alert journal references without a binding.+// Docker references are excluded: the store migration binds full container+// IDs, and names or shortened IDs must never bind.+func (s *SQLiteResourceStore) unboundHistoryReferences() ([]string, error) {+ rows, err := s.db.Query(`SELECT DISTINCT canonical_id FROM resource_changes+ WHERE kind GLOB 'alert_*' AND canonical_id NOT GLOB 'docker:*'+ AND canonical_id NOT IN (SELECT source_id FROM resource_history_aliases)+ ORDER BY canonical_id`)+ if err != nil {+ return nil, fmt.Errorf("read unbound resource history references: %w", err)+ }+ defer rows.Close()+ var refs []string+ for rows.Next() {+ var ref string+ if err := rows.Scan(&ref); err != nil {+ return nil, err+ }+ refs = append(refs, ref)+ }+ return refs, rows.Err()+}++func (s *SQLiteResourceStore) bindHistorySourceIdentities(bindings map[string]string) error {+ if len(bindings) == 0 {+ return nil+ }+ s.mu.Lock()+ defer s.mu.Unlock()+ tx, err := s.db.Begin()+ if err != nil {+ return fmt.Errorf("begin resource history identities: %w", err)+ }+ defer func() { _ = tx.Rollback() }()+ for sourceID, canonicalID := range bindings {+ if _, err := tx.Exec(`INSERT INTO resource_history_aliases (source_id, canonical_id) VALUES (?, ?)+ ON CONFLICT(source_id) DO UPDATE SET canonical_id = excluded.canonical_id`, sourceID, canonicalID); err != nil {+ return fmt.Errorf("bind resource history identity: %w", err)+ }+ }+ return tx.Commit()+}++// bindLegacyHistory binds the pending references a published registry+// generation resolves, with the same rules as RecordChange. The journal is+// scanned once per process; later references arrive from RecordChange.+func (a *MonitorAdapter) bindLegacyHistory(registry *ResourceRegistry, now time.Time) {+ binder, ok := registry.store.(legacyHistoryBinder)+ if !ok {+ return+ }+ backfill := &a.legacyHistory+ backfill.mu.Lock()+ defer backfill.mu.Unlock()+ if !backfill.loaded {+ refs, err := binder.unboundHistoryReferences()+ if err != nil {+ log.Printf("unifiedresources: %v", err)+ return+ }+ backfill.loaded = true+ for _, ref := range refs {+ backfill.addLocked(ref, now)+ }+ }+ if len(backfill.pending) == 0 {+ return+ }+ bindings := make(map[string]string)+ var settled []string+ for ref, deadline := range backfill.pending {+ if resolvedID, _ := registry.resolveHistoryReference(ref); resolvedID != "" {+ if resolvedID != ref {+ bindings[ref] = resolvedID+ }+ settled = append(settled, ref)+ } else if !now.Before(deadline) {+ delete(backfill.pending, ref)+ }+ }+ if err := binder.bindHistorySourceIdentities(bindings); err != nil {+ log.Printf("unifiedresources: %v", err)+ return+ }+ for _, ref := range settled {+ delete(backfill.pending, ref)+ }+ if len(bindings) > 0 {+ log.Printf("unifiedresources: bound %d alert history references to canonical resources", len(bindings))+ }+}+// migrateResourceHistoryAliases adds an index for exact legacy Docker event// identities, including resources already removed from live inventory. The// event rows and all authority-bearing tables remain unchanged.diff --git a/internal/unifiedresources/history_identity_test.go b/internal/unifiedresources/history_identity_test.goindex 21c4512dbf..935e260fb4 100644--- a/internal/unifiedresources/history_identity_test.go+++ b/internal/unifiedresources/history_identity_test.go@@ -173,3 +173,163 @@ func TestHistoryIdentityRetentionAndUnavailableLookup(t *testing.T) {_, err = store.GetRecentChanges(canonical, time.Time{}, 10)require.Error(t, err)}++func proxmoxHistoryIdentitySnapshot(now time.Time) models.StateSnapshot {+ return models.StateSnapshot{+ Nodes: []models.Node{{ID: "lab-pve1", Name: "pve1", Instance: "lab", Host: "https://pve1.lab:8006", Status: "online", LinkedAgentID: "host-pve1", LastSeen: now}},+ Hosts: []models.Host{{ID: "host-pve1", Hostname: "pve1", LinkedNodeID: "lab-pve1", MachineID: "0123456789abcdef", Status: "online", LastSeen: now}},+ VMs: []models.VM{{ID: "lab:pve1:101", VMID: 101, Name: "web", Node: "pve1", Instance: "lab", Status: "running", Type: "qemu", LastSeen: now}},+ Containers: []models.Container{{ID: "lab-pve1-102", VMID: 102, Name: "dns", Node: "pve1", Instance: "lab", Status: "running", Type: "lxc", LastSeen: now}},+ }+}++func historyIdentityResourceID(t *testing.T, registry *ResourceRegistry, resourceType ResourceType, name string) string {+ t.Helper()+ for _, resource := range registry.List() {+ if resource.Type == resourceType && resource.Name == name {+ return resource.ID+ }+ }+ t.Fatalf("no %s resource named %q", resourceType, name)+ return ""+}++// Proxmox alerts carry the source-native node and guest IDs. Their lifecycle+// must land in the history the drawer, facets and assistant read by canonical ID.+func TestHistoryIdentityMonitorAdapterResolvesProxmoxAlertReferences(t *testing.T) {+ store, err := NewSQLiteResourceStore(t.TempDir(), "default")+ require.NoError(t, err)+ t.Cleanup(func() { require.NoError(t, store.Close()) })+ now := time.Now().UTC().Truncate(time.Second)+ snapshot := proxmoxHistoryIdentitySnapshot(now)+ // A guest named like the system alert reference must not capture it.+ snapshot.VMs = append(snapshot.VMs, models.VM{ID: "lab:pve1:120", VMID: 120, Name: "pulse-system", Node: "pve1", Instance: "lab", Status: "running", Type: "qemu", LastSeen: now})+ registry := NewRegistry(store)+ registry.IngestSnapshot(snapshot)+ adapter := NewMonitorAdapter(registry)+ nodeID := historyIdentityResourceID(t, registry, ResourceTypeAgent, "pve1")+ vmID := historyIdentityResourceID(t, registry, ResourceTypeVM, "web")+ ctID := historyIdentityResourceID(t, registry, ResourceTypeSystemContainer, "dns")++ // "lab:pve2:101" names a node other than the guest's current one, as an+ // alert raised before a live migration does; it still names the guest.+ refs := []string{"lab-pve1", "agent:host-pve1", "lab:pve2:101", "lab-pve1-102", "pulse-system", "web"}+ for i, ref := range refs {+ require.NoError(t, adapter.RecordChange(ResourceChange{ID: "fired-" + ref, ResourceID: ref, Kind: ChangeAlertFired, SourceType: SourceHeuristic, ObservedAt: now.Add(time.Duration(i) * time.Second)}))+ }+ for canonicalID, want := range map[string][]string{nodeID: {"agent:host-pve1", "lab-pve1"}, vmID: {"lab:pve2:101"}, ctID: {"lab-pve1-102"}} {+ got, err := store.GetRecentChanges(canonicalID, time.Time{}, 10)+ require.NoError(t, err)+ require.Len(t, got, len(want), canonicalID)+ for i, ref := range want {+ require.Equal(t, "fired-"+ref, got[i].ID)+ require.Equal(t, canonicalID, got[i].ResourceID)+ bound, found, err := store.ResolveHistorySourceIdentity(ref)+ require.NoError(t, err)+ require.True(t, found, ref)+ require.Equal(t, canonicalID, bound)+ }+ kinds, err := store.CountRecentChangesByKind(canonicalID, time.Time{})+ require.NoError(t, err)+ require.Equal(t, len(want), kinds[ChangeAlertFired], canonicalID)+ }+ // Names are display identities: they keep their own history and no binding.+ for _, ref := range []string{"pulse-system", "web"} {+ got, err := store.GetRecentChanges(ref, time.Time{}, 10)+ require.NoError(t, err)+ require.Len(t, got, 1, ref)+ _, found, err := store.ResolveHistorySourceIdentity(ref)+ require.NoError(t, err)+ require.False(t, found, ref)+ }+ got, err := store.GetRecentChanges(historyIdentityResourceID(t, registry, ResourceTypeVM, "pulse-system"), time.Time{}, 10)+ require.NoError(t, err)+ require.Empty(t, got)++ // Recovery after the node left inventory still reaches the same history.+ removed := NewMonitorAdapter(NewRegistry(store))+ require.NoError(t, removed.RecordChange(ResourceChange{ID: "resolved-lab-pve1", ResourceID: "lab-pve1", Kind: ChangeAlertResolved, SourceType: SourceHeuristic, ObservedAt: now.Add(time.Minute)}))+ got, err = store.GetRecentChanges(nodeID, time.Time{}, 10)+ require.NoError(t, err)+ require.Len(t, got, 3)+ require.Equal(t, "resolved-lab-pve1", got[0].ID)+ require.Equal(t, nodeID, got[0].ResourceID)++ // When two resources answer to the reference, neither the registry nor the+ // retained binding may choose: the event keeps its own reference.+ registry.mu.Lock()+ registry.bySource[SourcePBS] = map[string]string{"lab-pve1": vmID}+ registry.mu.Unlock()+ require.NoError(t, adapter.RecordChange(ResourceChange{ID: "ambiguous-lab-pve1", ResourceID: "lab-pve1", Kind: ChangeAlertFired, SourceType: SourceHeuristic, ObservedAt: now.Add(2 * time.Minute)}))+ var recordedAs string+ require.NoError(t, store.db.QueryRow(`SELECT canonical_id FROM resource_changes WHERE id = 'ambiguous-lab-pve1'`).Scan(&recordedAs))+ require.Equal(t, "lab-pve1", recordedAs)+ bound, _, err := store.ResolveHistorySourceIdentity("lab-pve1")+ require.NoError(t, err)+ require.Equal(t, nodeID, bound, "an ambiguous reference never rebinds")+ got, err = store.GetRecentChanges(vmID, time.Time{}, 10)+ require.NoError(t, err)+ require.Len(t, got, 1)+}++// Alert rows recorded under source-native references before they resolved join+// canonical history once a registry generation can name their resource: rows+// journaled before this process started, and rows written before inventory+// knew the resource. The rows themselves are never rewritten.+func TestHistoryIdentityBindsLegacyAlertRowsFromRegistryGenerations(t *testing.T) {+ store, err := NewSQLiteResourceStore(t.TempDir(), "default")+ require.NoError(t, err)+ t.Cleanup(func() { require.NoError(t, store.Close()) })+ now := time.Now().UTC().Truncate(time.Second)+ for i, ref := range []string{"lab-pve1", "lab-pve1-102", "lab:pve1:103", "lab:pve1:999", "docker:tower/worker"} {+ require.NoError(t, store.RecordChange(ResourceChange{ID: "legacy-" + ref, ResourceID: ref, Kind: ChangeAlertFired, SourceType: SourceHeuristic, ObservedAt: now.Add(time.Duration(i) * time.Second)}))+ }+ adapter := NewMonitorAdapter(NewRegistry(store))+ snapshot := proxmoxHistoryIdentitySnapshot(now)+ adapter.PopulateFromSnapshot(snapshot)+ registry := adapter.currentRegistry()+ for ref, canonicalID := range map[string]string{+ "lab-pve1": historyIdentityResourceID(t, registry, ResourceTypeAgent, "pve1"),+ "lab-pve1-102": historyIdentityResourceID(t, registry, ResourceTypeSystemContainer, "dns"),+ } {+ got, err := store.GetRecentChangesFiltered(canonicalID, time.Time{}, 10, ResourceChangeFilters{Kinds: []ChangeKind{ChangeAlertFired}})+ require.NoError(t, err)+ require.Len(t, got, 1, ref)+ require.Equal(t, "legacy-"+ref, got[0].ID)+ require.Equal(t, ref, got[0].ResourceID)+ }++ // An alert raised after the journal scan, before inventory names its guest.+ require.NoError(t, adapter.RecordChange(ResourceChange{ID: "early-lab:pve1:104", ResourceID: "lab:pve1:104", Kind: ChangeAlertFired, SourceType: SourceHeuristic, ObservedAt: now.Add(time.Minute)}))+ for _, ref := range []string{"lab:pve1:103", "lab:pve1:104"} {+ _, found, err := store.ResolveHistorySourceIdentity(ref)+ require.NoError(t, err)+ require.False(t, found, ref)+ }+ // Both guests join inventory in a later generation and are bound then.+ snapshot.VMs = append(snapshot.VMs,+ models.VM{ID: "lab:pve1:103", VMID: 103, Name: "late", Node: "pve1", Instance: "lab", Status: "running", Type: "qemu", LastSeen: now},+ models.VM{ID: "lab:pve1:104", VMID: 104, Name: "early", Node: "pve1", Instance: "lab", Status: "running", Type: "qemu", LastSeen: now})+ adapter.PopulateFromSnapshot(snapshot)+ for name, id := range map[string]string{"late": "legacy-lab:pve1:103", "early": "early-lab:pve1:104"} {+ got, err := store.GetRecentChangesFiltered(historyIdentityResourceID(t, adapter.currentRegistry(), ResourceTypeVM, name), time.Time{}, 10, ResourceChangeFilters{Kinds: []ChangeKind{ChangeAlertFired}})+ require.NoError(t, err)+ require.Len(t, got, 1, name)+ require.Equal(t, id, got[0].ID)+ }++ // Unknown guests and Docker names stay unbound, and retries end at the deadline.+ for _, ref := range []string{"lab:pve1:999", "docker:tower/worker"} {+ _, found, err := store.ResolveHistorySourceIdentity(ref)+ require.NoError(t, err)+ require.False(t, found, ref)+ }+ require.Len(t, adapter.legacyHistory.pending, 1)+ require.Contains(t, adapter.legacyHistory.pending, "lab:pve1:999")+ adapter.legacyHistory.pending["lab:pve1:999"] = time.Now().Add(-time.Second)+ adapter.PopulateFromSnapshot(snapshot)+ require.Empty(t, adapter.legacyHistory.pending)+ refs, err := store.unboundHistoryReferences()+ require.NoError(t, err)+ require.Equal(t, []string{"lab:pve1:999"}, refs)+}diff --git a/internal/unifiedresources/monitor_adapter.go b/internal/unifiedresources/monitor_adapter.goindex 8bac41ceaa..94795237b4 100644--- a/internal/unifiedresources/monitor_adapter.go+++ b/internal/unifiedresources/monitor_adapter.go@@ -26,6 +26,7 @@ type MonitorAdapter struct {activeAlerts []models.AlertlastRebuiltAt time.TimestaleThresholds map[DataSource]time.Duration+ legacyHistory legacyHistoryBackfilloverlays overlayReadStateCache}@@ -195,13 +196,17 @@ func (a *MonitorAdapter) currentRegistry() *ResourceRegistry {}// RecordChange forwards canonical resource-history events into the underlying-// resource store when monitoring has a durable store attached.+// resource store when monitoring has a durable store attached. Alert lifecycle+// events carry source-native references (Proxmox node and guest IDs, agent+// IDs); they resolve here so history reads by canonical ID include them.func (a *MonitorAdapter) RecordChange(change ResourceChange) error {registry := a.currentRegistry()if registry == nil || registry.store == nil {return nil}sourceRef := change.ResourceID+ history, hasHistory := registry.store.(resourceHistoryIdentityWriter)+ unbound := falseif sourceID, derivedID, ok := legacyDockerHistoryIdentity(sourceRef); ok {// Use exact source identity when inventory is present, including any// canonical identity merge. Full Docker IDs remain derivable after removal.@@ -212,7 +217,7 @@ func (a *MonitorAdapter) RecordChange(change ResourceChange) error {change.ResourceID = resolvedID} else {change.ResourceID = derivedID- if history, ok := registry.store.(resourceHistoryIdentityWriter); ok {+ if hasHistory {id, found, err := history.ResolveHistorySourceIdentity(sourceRef)if err != nil {return err@@ -222,13 +227,31 @@ func (a *MonitorAdapter) RecordChange(change ResourceChange) error {}}}- }- if sourceRef != change.ResourceID {- if writer, ok := registry.store.(resourceHistoryIdentityWriter); ok {- return writer.RecordChangeWithSourceIdentity(change, sourceRef)+ } else if resolvedID, claimed := registry.resolveHistoryReference(sourceRef); resolvedID != "" {+ change.ResourceID = resolvedID+ } else if hasHistory && !claimed && !isDockerHistoryReference(sourceRef) {+ // No resource answers to it: keep the binding an earlier event recorded.+ id, found, err := history.ResolveHistorySourceIdentity(CanonicalResourceID(sourceRef))+ if err != nil {+ return err+ }+ if found {+ change.ResourceID = id+ } else {+ unbound = true}}- return registry.store.RecordChange(change)+ if hasHistory && CanonicalResourceID(sourceRef) != CanonicalResourceID(change.ResourceID) {+ return history.RecordChangeWithSourceIdentity(change, sourceRef)+ }+ if err := registry.store.RecordChange(change); err != nil {+ return err+ }+ // Inventory may name the resource in a later generation; bind the row then.+ if _, ok := registry.store.(legacyHistoryBinder); unbound && ok {+ a.legacyHistory.add(CanonicalResourceID(sourceRef), time.Now())+ }+ return nil}// GetRecentChanges forwards canonical resource-history reads into the@@ -437,6 +460,7 @@ func (a *MonitorAdapter) replaceRegistryLocked(snapshot models.StateSnapshot, rea.activeAlerts = append([]models.Alert(nil), snapshot.ActiveAlerts...)a.lastRebuiltAt = rebuiltAta.mu.Unlock()+ a.bindLegacyHistory(rebuilt, time.Now())}// ShouldSkipAPIPolling returns true when agent coverage indicates API