Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Preserve guest memory evidence across AI queries and local observers
pulse-triage[bot]a721b3a8e0d5
diff --git a/docs/release-control/v6/internal/subsystems/ai-runtime.md b/docs/release-control/v6/internal/subsystems/ai-runtime.mdindex ce1a7d1c22..30b50b2a63 100644--- a/docs/release-control/v6/internal/subsystems/ai-runtime.md+++ b/docs/release-control/v6/internal/subsystems/ai-runtime.md@@ -25,6 +25,34 @@ that same result. Successful reads retain their content and execution provenance## Purpose+### Guest memory remains qualified through tools and local observers++The existing guest get/list/topology tools and Assistant inventory context carry+selected memory evidence with the percentage. Unavailable/invalid memory is+null, not zero usage; a measured zero stays numeric. Evidence retains source,+state and original observation time, including unknown pressure and possible+reclaimable cache. Condensed query facts retain the same qualification rather+than saving a bare percentage. `TestGuestMemoryQueryEvidence` and+`TestAssistantGuestMemoryContextAndFacts` check the real tool/context/fact paths.++The existing local memory observer uses that same interpretation for VM/system+containers. Unknown pressure wakes it with an explicit evidence gap, not a+numeric breach or a healthy predicate. A known selected guest sample is bounded+by its own observation age; refreshing the resource through another source does+not renew it. Other-platform legacy readings and other metric observers retain+their existing semantics. `TestPatrolObserverGuestMemoryOrigin` checks unknown,+retained, undated, future, invalid and stale origins alongside genuine low/zero/+high pressure and the age boundary. This changes no objective ABI, threshold,+collection lease, observer authority or guest-command admission.++Performance-tool guest memory history remains numeric, but explicitly lacks+per-sample source/pressure qualification. A live current reading cannot reattribute+past samples or prove that past high usage was pressure/a leak.+`TestGuestMemoryHistoryDoesNotBorrowLiveOrigin` checks unchanged numeric history,+its truthful limitation and non-guest compatibility. No History writer/schema+or existing stored finding is changed. These controls do not establish #2762's+running source, episode cause, native recovery or a containing release.+### Patrol guest memory is qualified evidence, not an unlabelled percentagePatrol consumes the selected guest memory metric's original observation,diff --git a/docs/release-control/v6/internal/subsystems/api-contracts.md b/docs/release-control/v6/internal/subsystems/api-contracts.mdindex 41f3fa8985..e5e7b3702f 100644--- a/docs/release-control/v6/internal/subsystems/api-contracts.md+++ b/docs/release-control/v6/internal/subsystems/api-contracts.md@@ -20,6 +20,20 @@## Purpose+### AI guest memory projections preserve uncertainty++Existing AI get/list/topology responses carry selected guest memory evidence:+availability, pressure qualification, source, state and original observation time.+A missing/invalid guest percentage is JSON null rather than zero; a measured zero+is explicitly numeric, including in list/topology and Assistant inventory. Get+usage bytes are null when unavailable. Non-guest response numbers remain unchanged.+The added evidence is a read-only projection, not a new route, command, feature,+collector or diagnostic authority. `TestGuestMemoryQueryEvidence` exercises actual+tool serialization and `TestAssistantGuestMemoryContextAndFacts` preserves it+through Assistant condensation. Performance responses add a guest-history source+limitation without relabelling past samples or changing their numbers; their+actual tool controls are in `TestGuestMemoryHistoryDoesNotBorrowLiveOrigin`.+### Optional held-alert breach date consumer`MetricAlertStatus.lastBreachAt?: string` mirrors the existing backend ISOdiff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.mdindex 219fe2b0b0..5f26dd6952 100644--- a/docs/release-control/v6/internal/subsystems/monitoring.md+++ b/docs/release-control/v6/internal/subsystems/monitoring.md@@ -17,6 +17,18 @@## Purpose+### Existing memory origins cannot be renewed by AI consumers++AI query/context/fact consumers now retain the selected guest observation and+explicit availability. The existing local VM/system-container memory observer+judges pressure only from a qualified selected sample and applies its age bound+to that original observation, not a fresh resource LastSeen from another source.+Unknown pressure is an evidence-gap wake, not a measured threshold breach or+recovery. `TestPatrolObserverGuestMemoryOrigin` and+`TestGuestMemoryEvidenceUsesSelectedMetricOnly` cover these consumption boundaries.+Collectors, guest commands, native leases, shared safety pauses, History storage+and configured thresholds remain unchanged; this is not native #2762 recovery.+### Guest memory evidence survives typed consumersThe monitor's existing guest memory observation remains source-owned whendiff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.mdindex b2c41cb7f2..710e2f90b5 100644--- a/docs/release-control/v6/internal/subsystems/unified-resources.md+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md@@ -23,6 +23,25 @@ and sort the complete canonical change table while startup and ingestion wait.## Purpose+### Shared interpretation of selected guest memory++`GuestMemoryEvidenceForResource` and the VM/container `MemoryEvidence` accessors+qualify only the selected numeric metric through `QualifyGuestMemory`. Presence+and finite 0–100 values distinguish a measured zero from absence. Only current,+dated cache-aware Proxmox/QGA/agent origins establish pressure; fallback/cache-+inclusive, unknown and retained origins keep their limitations. Consumers cannot+borrow raw-facet provenance or resource LastSeen. Collection owns current/last-+known state and its lease; local observers additionally enforce their configured+age on the original memory observation, without creating a new collection TTL.++Source names are allowlisted before entering AI context; unknown names remain+unknown. Other-platform readings without this Proxmox observation contract keep+existing semantics. `TestGuestMemoryEvidenceQualification` covers these boundaries+and `TestGuestMemoryEvidenceUsesSelectedMetricOnly` covers selected/raw disagreement,+absence and refreshed rows in both views. This shared interpretation changes+no registry wire format, metric selection, guest collection, History, correlation,+identity or permission. AI query projections own their additional evidence fields.+### Selected guest memory observation in typed read stateVM and system-container views expose a value-copy of the selected memorydiff --git a/internal/ai/chat/guest_memory_evidence_test.go b/internal/ai/chat/guest_memory_evidence_test.gonew file mode 100644index 0000000000..682f88f215--- /dev/null+++ b/internal/ai/chat/guest_memory_evidence_test.go@@ -0,0 +1,84 @@+package chat++import (+ "context"+ "encoding/json"+ "strings"+ "testing"+ "time"++ "github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"+ "github.com/rcourtman/pulse-go-rewrite/internal/ai/tools"+ "github.com/rcourtman/pulse-go-rewrite/internal/models"+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"+)++func TestAssistantGuestMemoryContextAndFacts(t *testing.T) {+ at := time.Now().UTC().Add(-time.Minute).Truncate(time.Second)+ for _, kind := range []string{"vm", "system-container"} {+ for _, tc := range []struct {+ name, state, source, text string+ percent float64+ unavailable, known bool+ }{+ {"healthy", "current", "guest-agent-meminfo", "cache-aware guest usage", 24, false, true},+ {"measured-zero", "current", "agent", "0% (current", 0, false, true},+ {"pressure", "current", "available-field", "cache-aware guest usage", 96, false, true},+ {"cache", "current", "status-mem", "may include reclaimable cache", 96, false, false},+ {"retained", "last-known", "agent", "guest pressure unknown", 24, false, false},+ {"missing", "unavailable", "unavailable", "N/A (guest memory unavailable; not evidence of recovery)", 0, true, false},+ } {+ t.Run(kind+"/"+tc.name, func(t *testing.T) {+ memory := models.Memory{Total: 20 << 30, Used: int64(float64(20<<30) * tc.percent / 100), Usage: tc.percent, UsageUnavailable: tc.unavailable,+ Observation: models.MemoryObservation{State: tc.state, Source: tc.source, ObservedAt: at}}+ snapshot := models.StateSnapshot{}+ if kind == "vm" {+ snapshot.VMs = []models.VM{{ID: "guest", Name: "guest", Instance: "fixture", Node: "node", VMID: 100, Status: "running", LastSeen: time.Now(), Memory: memory}}+ } else {+ snapshot.Containers = []models.Container{{ID: "guest", Name: "guest", Instance: "fixture", Node: "node", VMID: 100, Status: "running", LastSeen: time.Now(), Memory: memory}}+ }+ registry := unifiedresources.NewRegistry(nil)+ registry.IngestSnapshot(snapshot)+ raw, err := marshalAssistantInventoryTopologyContextFromReadState(registry)+ if err != nil {+ t.Fatal(err)+ }+ var decoded map[string]interface{}+ if err := json.Unmarshal([]byte(raw), &decoded); err != nil {+ t.Fatal(err)+ }+ family := "vms"+ if kind != "vm" {+ family = "containers"+ }+ row := decoded["proxmox"].(map[string]interface{})["nodes"].([]interface{})[0].(map[string]interface{})[family].([]interface{})[0].(map[string]interface{})+ e, ok := row["memory_evidence"].(map[string]interface{})+ if !ok || e["available"] != !tc.unavailable || e["pressure_known"] != tc.known {+ t.Errorf("Assistant seed lost qualification: %s", raw)+ }+ if !tc.unavailable && row["memory_percent"] != tc.percent || tc.unavailable && row["memory_percent"] != nil {+ t.Errorf("Assistant seed confused absent/measured memory: %+v", row)+ }+ id := ""+ if kind == "vm" {+ id = registry.VMs()[0].ID()+ } else {+ id = registry.Containers()[0].ID()+ }+ exec := tools.NewPulseToolExecutor(tools.ExecutorConfig{ReadState: registry})+ input := map[string]interface{}{"action": "get", "resource_type": kind, "resource_id": id}+ result, err := exec.ExecuteTool(context.Background(), agentcapabilities.PulseQueryToolName, input)+ if err != nil || result.IsError {+ t.Fatalf("actual resource query failed: %v %+v", err, result)+ }+ facts := extractQueryGetFacts(input, result.Content[0].Text)+ if len(facts) == 0 || !strings.Contains(facts[0].Value, tc.text) {+ t.Errorf("condensed facts removed memory qualification: %+v", facts)+ }+ if !tc.unavailable && (len(facts) == 0 || !strings.Contains(facts[0].Value, at.Format(time.RFC3339))) {+ t.Errorf("facts renewed original memory time: %+v", facts)+ }+ })+ }+ }+}diff --git a/internal/ai/chat/knowledge_extractor.go b/internal/ai/chat/knowledge_extractor.goindex 644fb3428f..efda07ee1e 100644--- a/internal/ai/chat/knowledge_extractor.go+++ b/internal/ai/chat/knowledge_extractor.go@@ -98,7 +98,8 @@ func extractQueryGetFacts(input map[string]interface{}, resultText string) []FacPercent float64 `json:"percent"`} `json:"cpu"`Memory struct {- Percent float64 `json:"percent"`+ Percent float64 `json:"percent"`+ Evidence *unifiedresources.GuestMemoryEvidence `json:"evidence"`} `json:"memory"`// Error field for not-found responsesError string `json:"error"`@@ -156,7 +157,9 @@ func extractQueryGetFacts(input map[string]interface{}, resultText string) []Facif resource.CPU.Percent > 0 {parts = append(parts, fmt.Sprintf("CPU=%.1f%%", resource.CPU.Percent))}- if resource.Memory.Percent > 0 {+ if resource.Memory.Evidence != nil {+ parts = append(parts, "Mem="+resource.Memory.Evidence.Format(resource.Memory.Percent))+ } else if resource.Memory.Percent > 0 {parts = append(parts, fmt.Sprintf("Mem=%.1f%%", resource.Memory.Percent))}diff --git a/internal/ai/chat/service.go b/internal/ai/chat/service.goindex b7a4edc7f1..e7f5cbfbce 100644--- a/internal/ai/chat/service.go+++ b/internal/ai/chat/service.go@@ -1416,22 +1416,24 @@ func marshalAssistantInventoryTopologyContext(topology tools.TopologyResponse) (}for _, vm := range node.VMs {nodeContext.VMs = append(nodeContext.VMs, assistantInventoryWorkload{- AnswerLabel: assistantInventoryGuestAnswerLabel("VM", vm.VMID, vm.Name),- VMID: vm.VMID,- Name: vm.Name,- Status: vm.Status,- CPUPercent: vm.CPU,- MemoryPercent: vm.Memory,+ AnswerLabel: assistantInventoryGuestAnswerLabel("VM", vm.VMID, vm.Name),+ VMID: vm.VMID,+ Name: vm.Name,+ Status: vm.Status,+ CPUPercent: vm.CPU,+ MemoryPercent: vm.Memory,+ MemoryEvidence: vm.MemoryEvidence,})}for _, container := range node.Containers {nodeContext.Containers = append(nodeContext.Containers, assistantInventoryWorkload{- AnswerLabel: assistantInventoryGuestAnswerLabel("CT", container.VMID, container.Name),- VMID: container.VMID,- Name: container.Name,- Status: container.Status,- CPUPercent: container.CPU,- MemoryPercent: container.Memory,+ AnswerLabel: assistantInventoryGuestAnswerLabel("CT", container.VMID, container.Name),+ VMID: container.VMID,+ Name: container.Name,+ Status: container.Status,+ CPUPercent: container.CPU,+ MemoryPercent: container.Memory,+ MemoryEvidence: container.MemoryEvidence,})}context.Proxmox.Nodes = append(context.Proxmox.Nodes, nodeContext)@@ -1545,12 +1547,13 @@ type assistantInventoryProxmoxNode struct {}type assistantInventoryWorkload struct {- AnswerLabel string `json:"answer_label"`- VMID int `json:"vmid"`- Name string `json:"name"`- Status string `json:"status"`- CPUPercent float64 `json:"cpu_percent,omitempty"`- MemoryPercent float64 `json:"memory_percent,omitempty"`+ AnswerLabel string `json:"answer_label"`+ VMID int `json:"vmid"`+ Name string `json:"name"`+ Status string `json:"status"`+ CPUPercent float64 `json:"cpu_percent,omitempty"`+ MemoryPercent *float64 `json:"memory_percent"`+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`}type assistantInventoryDockerHost struct {diff --git a/internal/ai/patrol_guest_memory.go b/internal/ai/patrol_guest_memory.goindex 2c1c18e806..292337d78e 100644--- a/internal/ai/patrol_guest_memory.go+++ b/internal/ai/patrol_guest_memory.go@@ -2,7 +2,6 @@ package aiimport ("fmt"- "math""strings""time"@@ -36,71 +35,24 @@ func patrolGuestMemoryEvidenceSection(snap patrolRuntimeState, scopedSet map[str// it visible as evidence, but do not use it (or a retained/undated reading) to// manufacture a pressure flag, forecast, anomaly or verified recovery.type patrolGuestMemoryReading struct {- percent float64- available bool- pressureKnown bool- state string- source string- observedAt time.Time+ percent float64+ available bool+ pressureKnown bool+ state string+ source string+ observedAt time.Time+ mayIncludeCache bool}func readPatrolGuestMemory(percent float64, observation models.MemoryObservation, available, proxmoxGuest bool) patrolGuestMemoryReading {- r := patrolGuestMemoryReading{percent: percent, available: available && !math.IsNaN(percent) && !math.IsInf(percent, 0) && percent >= 0 && percent <= 100}- if !r.available || observation.State == "unavailable" {- r.available = false- r.state = "unavailable"- return r- }- // Other platforms need not implement Proxmox's memory evidence contract.- // Preserve their existing readings, including a selected measured zero.- if !proxmoxGuest && observation == (models.MemoryObservation{}) {- r.pressureKnown = true- return r- }- r.state = "unknown"- if observation.ObservedAt.After(time.Unix(0, 0)) && !observation.ObservedAt.After(time.Now()) {- r.observedAt = observation.ObservedAt- }- if observation.State == "last-known" {- r.state = "last-known"- } else if observation.State == "current" && !r.observedAt.IsZero() {- r.state = "current"- }- // Only known source names are put in the seed. A future/unknown source is- // not an assertion of either cache awareness or current guest pressure.- switch observation.Source {- case "available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent":- r.source = observation.Source- r.pressureKnown = r.state == "current"- case "status-mem", "status-freemem", "cluster-resources", "derived-total-minus-used", "previous-snapshot":- r.source = observation.Source- }- return r+ e := unifiedresources.QualifyGuestMemory(percent, observation, available, proxmoxGuest, time.Now())+ return patrolGuestMemoryReading{percent: percent, available: e.Available, pressureKnown: e.PressureKnown,+ state: e.State, source: e.Source, observedAt: e.ObservedAt, mayIncludeCache: e.MayIncludeCache}}func (r patrolGuestMemoryReading) display() string {- if !r.available {- return "N/A (guest memory unavailable; not evidence of recovery)"- }- if r.state == "" {- return fmt.Sprintf("%.0f%%", r.percent)- }- source := r.source- if source == "" {- source = "unknown"- }- observed := "time unknown"- if !r.observedAt.IsZero() {- observed = "observed " + r.observedAt.UTC().Format(time.RFC3339)- }- qualification := "cache-aware guest usage"- if !r.pressureKnown {- qualification = "guest pressure unknown"- if r.source == "status-mem" || r.source == "status-freemem" || r.source == "cluster-resources" || r.source == "derived-total-minus-used" {- qualification += "; may include reclaimable cache"- }- }- return fmt.Sprintf("%.0f%% (%s; source %s; %s; %s)", r.percent, r.state, source, observed, qualification)+ return (unifiedresources.GuestMemoryEvidence{Available: r.available, PressureKnown: r.pressureKnown,+ State: r.state, Source: r.source, ObservedAt: r.observedAt, MayIncludeCache: r.mayIncludeCache}).Format(r.percent)}func patrolGuestMemoryGaps(rows []patrolGuestInventoryRow) string {diff --git a/internal/ai/patrol_observer_guest_memory_test.go b/internal/ai/patrol_observer_guest_memory_test.gonew file mode 100644index 0000000000..c375535447--- /dev/null+++ b/internal/ai/patrol_observer_guest_memory_test.go@@ -0,0 +1,89 @@+package ai++import (+ "fmt"+ "math"+ "strings"+ "testing"+ "time"++ "github.com/rcourtman/pulse-go-rewrite/internal/models"+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"+)++func TestPatrolObserverGuestMemoryOrigin(t *testing.T) {+ now := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)+ for _, kind := range []unifiedresources.ResourceType{unifiedresources.ResourceTypeVM, unifiedresources.ResourceTypeSystemContainer} {+ for _, tc := range []struct {+ name, state, source, detail string+ percent float64+ at time.Time+ missing, noProxmox bool+ }{+ {"healthy", "current", "guest-agent-meminfo", "", 24, now, false, false},+ {"zero", "current", "agent", "", 0, now, false, false},+ {"real-pressure", "current", "available-field", "guest=96.00", 96, now, false, false},+ {"inclusive-high", "current", "status-mem", "guest_memory_pressure_unknown", 96, now, false, false},+ {"inclusive-low", "current", "status-freemem", "guest_memory_pressure_unknown", 24, now, false, false},+ {"retained-low", "last-known", "agent", "guest_memory_pressure_unknown", 24, now.Add(-time.Hour), false, false},+ {"stale-origin", "current", "agent", "metric_stale", 24, now.Add(-61 * time.Second), false, false},+ {"age-boundary", "current", "agent", "", 24, now.Add(-59 * time.Second), false, false},+ {"missing-time", "current", "agent", "guest_memory_pressure_unknown", 24, time.Time{}, false, false},+ {"future-time", "current", "agent", "guest_memory_pressure_unknown", 24, now.Add(time.Hour), false, false},+ {"unknown-source", "current", "future-source", "guest_memory_pressure_unknown", 24, now, false, false},+ {"missing", "", "", "guest_memory_pressure_unknown", 0, time.Time{}, true, false},+ {"invalid", "current", "agent", "guest_memory_pressure_unknown", math.NaN(), now, false, false},+ {"other-platform", "", "", "", 24, time.Time{}, false, true},+ } {+ t.Run(string(kind)+"/"+tc.name, func(t *testing.T) {+ store := NewInMemoryPatrolObjectiveStore()+ objective, err := store.Create(CreatePatrolObjectiveInput{Brief: "Keep guest memory below 85 percent", ResourceIDs: []string{"guest"}}, now)+ if err != nil {+ t.Fatal(err)+ }+ _, err = store.ProposeObserver(objective.ID, ProposePatrolObserverInput{+ ExpectedRevision: objective.Revision, EvidenceFit: PatrolObserverEvidenceFitDirect,+ Interpretation: "Guest pressure stays below 85 percent.", TriggerKinds: []PatrolObserverTriggerKind{PatrolObserverTriggerInterval},+ ProbeJSON: `{"runtime":"pulse-resource-metric/v1","metric":"memory_percent","operator":"less_than","threshold":85,"sample_interval_seconds":10,"wake_after_consecutive_failures":1,"max_evidence_age_seconds":60}`,+ WakeEvidence: "Memory objective breached or its pressure evidence is unavailable.", RequirementsJSON: `{}`,+ }, now)+ if err != nil {+ t.Fatal(err)+ }+ r := unifiedresources.Resource{ID: "guest", Type: kind, LastSeen: now.Add(time.Second), UpdatedAt: now.Add(time.Second),+ Proxmox: &unifiedresources.ProxmoxData{Memory: &models.Memory{Usage: 24, Observation: models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: now}}},+ Metrics: &unifiedresources.ResourceMetrics{Memory: &unifiedresources.MetricValue{Percent: tc.percent, Observation: models.MemoryObservation{State: tc.state, Source: tc.source, ObservedAt: tc.at}}},+ }+ if tc.missing {+ r.Metrics.Memory = nil+ }+ if tc.noProxmox {+ r.Proxmox = nil+ }+ patrol := NewPatrolService(nil, nil)+ patrol.SetObjectiveStore(store)+ patrol.SetUnifiedResourceProvider(&mockUnifiedResourceProvider{getAllFunc: func() []unifiedresources.Resource { return []unifiedresources.Resource{r} }})+ tm := NewTriggerManager(TriggerManagerConfig{MaxPendingTriggers: 10})+ patrol.SetTriggerManager(tm)+ patrol.processObjectiveObservers(now.Add(time.Second))+ got, _ := store.Get(objective.ID, now.Add(time.Second))+ if got.Observer == nil || got.Observer.State != PatrolObserverInstalled {+ t.Fatalf("existing observer failed to install: %+v", got.Observer)+ }+ if tc.detail == "" {+ if tm.GetPendingCount() != 0 || got.Coverage.State != PatrolObjectiveCovered {+ t.Fatalf("valid memory control lost coverage: %+v pending=%d", got.Coverage, tm.GetPendingCount())+ }+ return+ }+ if tm.GetPendingCount() != 1 || tm.pendingTriggers[0].ObjectiveContext == nil {+ t.Fatalf("unqualified/breached evidence accepted as healthy: %+v pending=%d", got.Coverage, tm.GetPendingCount())+ }+ context := tm.pendingTriggers[0].ObjectiveContext+ if !strings.Contains(context.Evidence, tc.detail) || fmt.Sprint(context.ObservedResourceIDs) != "[guest]" {+ t.Fatalf("wake lost unknown/breach/identity distinction: %+v", context)+ }+ })+ }+ }+}diff --git a/internal/ai/patrol_observer_runtime.go b/internal/ai/patrol_observer_runtime.goindex fe97dc7658..bf20f91066 100644--- a/internal/ai/patrol_observer_runtime.go+++ b/internal/ai/patrol_observer_runtime.go@@ -544,6 +544,19 @@ func (p *PatrolService) evaluateObjectiveObserver(runtime *patrolObserverRuntimeif observedAt.IsZero() {observedAt = resource.UpdatedAt}+ if probe.metric == "memory_percent" && (resource.Type == unifiedresources.ResourceTypeVM || resource.Type == unifiedresources.ResourceTypeSystemContainer) {+ // The row can be refreshed by a different source. Only the selected+ // guest memory sample owns its freshness and pressure semantics.+ memory := unifiedresources.GuestMemoryEvidenceForResource(&resource, now)+ if !memory.PressureKnown {+ failing = append(failing, resourceID)+ evidenceDetails = append(evidenceDetails, resourceID+"=guest_memory_pressure_unknown")+ continue+ }+ if memory.State != "" {+ observedAt = memory.ObservedAt+ }+ }if observedAt.IsZero() || now.Sub(observedAt) > maxAge {failing = append(failing, resourceID)evidenceDetails = append(evidenceDetails, resourceID+"=metric_stale")diff --git a/internal/ai/tools/data_types.go b/internal/ai/tools/data_types.goindex 89404a52b3..368e8354b9 100644--- a/internal/ai/tools/data_types.go+++ b/internal/ai/tools/data_types.go@@ -278,23 +278,25 @@ type NodeSummary struct {// VMSummary is a summarized VM for list responsestype VMSummary struct {GovernedResourceMetadata- VMID int `json:"vmid"`- Name string `json:"name"`- Status string `json:"status"`- Node string `json:"node"`- CPU float64 `json:"cpu_percent,omitempty"`- Memory float64 `json:"memory_percent,omitempty"`+ VMID int `json:"vmid"`+ Name string `json:"name"`+ Status string `json:"status"`+ Node string `json:"node"`+ CPU float64 `json:"cpu_percent,omitempty"`+ Memory *float64 `json:"memory_percent"`+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`}// ContainerSummary is a summarized LXC container for list responsestype ContainerSummary struct {GovernedResourceMetadata- VMID int `json:"vmid"`- Name string `json:"name"`- Status string `json:"status"`- Node string `json:"node"`- CPU float64 `json:"cpu_percent,omitempty"`- Memory float64 `json:"memory_percent,omitempty"`+ VMID int `json:"vmid"`+ Name string `json:"name"`+ Status string `json:"status"`+ Node string `json:"node"`+ CPU float64 `json:"cpu_percent,omitempty"`+ Memory *float64 `json:"memory_percent"`+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`}// DockerHostSummary is a summarized Docker host for list responses@@ -485,13 +487,14 @@ func (t ProxmoxNodeTopology) NormalizeCollections() ProxmoxNodeTopology {// TopologyVM represents a VM in the topologytype TopologyVM struct {GovernedResourceMetadata- VMID int `json:"vmid"`- Name string `json:"name"`- Status string `json:"status"`- CPU float64 `json:"cpu_percent,omitempty"`- Memory float64 `json:"memory_percent,omitempty"`- OS string `json:"os,omitempty"`- Tags []string `json:"tags"`+ VMID int `json:"vmid"`+ Name string `json:"name"`+ Status string `json:"status"`+ CPU float64 `json:"cpu_percent,omitempty"`+ Memory *float64 `json:"memory_percent"`+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`+ OS string `json:"os,omitempty"`+ Tags []string `json:"tags"`}func (t TopologyVM) NormalizeCollections() TopologyVM {@@ -504,14 +507,15 @@ func (t TopologyVM) NormalizeCollections() TopologyVM {// TopologyContainer represents a system container in the topologytype TopologyContainer struct {GovernedResourceMetadata- VMID int `json:"vmid"`- Name string `json:"name"`- Status string `json:"status"`- CPU float64 `json:"cpu_percent,omitempty"`- Memory float64 `json:"memory_percent,omitempty"`- OS string `json:"os,omitempty"`- Tags []string `json:"tags"`- HasDocker bool `json:"has_docker,omitempty"` // True if Docker is installed inside this container+ VMID int `json:"vmid"`+ Name string `json:"name"`+ Status string `json:"status"`+ CPU float64 `json:"cpu_percent,omitempty"`+ Memory *float64 `json:"memory_percent"`+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`+ OS string `json:"os,omitempty"`+ Tags []string `json:"tags"`+ HasDocker bool `json:"has_docker,omitempty"` // True if Docker is installed inside this container}func (t TopologyContainer) NormalizeCollections() TopologyContainer {@@ -689,7 +693,7 @@ type ResourceResponse struct {}func EmptyResourceResponse() ResourceResponse {- return ResourceResponse{}.NormalizeCollections()+ return ResourceResponse{Memory: resourceMemoryValues(0, 0, 0)}.NormalizeCollections()}func (r ResourceResponse) NormalizeCollections() ResourceResponse {@@ -850,9 +854,15 @@ type ResourceCPU struct {// ResourceMemory describes memory usagetype ResourceMemory struct {- Percent float64 `json:"percent"`- UsedGB float64 `json:"used_gb"`- TotalGB float64 `json:"total_gb"`+ Percent *float64 `json:"percent"`+ UsedGB *float64 `json:"used_gb"`+ TotalGB float64 `json:"total_gb"`+ Evidence *unifiedresources.GuestMemoryEvidence `json:"evidence,omitempty"`+}++// resourceMemoryValues preserves the numeric contract for non-guest responses.+func resourceMemoryValues(percent, usedGB, totalGB float64) ResourceMemory {+ return ResourceMemory{Percent: &percent, UsedGB: &usedGB, TotalGB: totalGB}}// ResourceDisk describes disk usage@@ -960,13 +970,14 @@ type FindingCounts struct {// MetricsResponse is returned by pulse_get_metricstype MetricsResponse struct {- ResourceID string `json:"resource_id,omitempty"`- Period string `json:"period"`- Points []MetricPoint `json:"points"`- Summary map[string]ResourceMetricsSummary `json:"summary"`- Pagination *PaginationInfo `json:"pagination,omitempty"`- Downsampled bool `json:"downsampled,omitempty"`- OriginalCount int `json:"original_count,omitempty"`+ ResourceID string `json:"resource_id,omitempty"`+ Period string `json:"period"`+ Points []MetricPoint `json:"points"`+ Summary map[string]ResourceMetricsSummary `json:"summary"`+ Pagination *PaginationInfo `json:"pagination,omitempty"`+ Downsampled bool `json:"downsampled,omitempty"`+ OriginalCount int `json:"original_count,omitempty"`+ MemoryInterpretation string `json:"memory_interpretation,omitempty"`}func EmptyMetricsResponse() MetricsResponse {diff --git a/internal/ai/tools/guest_memory_evidence_test.go b/internal/ai/tools/guest_memory_evidence_test.gonew file mode 100644index 0000000000..479a742fe1--- /dev/null+++ b/internal/ai/tools/guest_memory_evidence_test.go@@ -0,0 +1,146 @@+package tools++import (+ "context"+ "encoding/json"+ "math"+ "strings"+ "testing"+ "time"++ "github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"+ "github.com/rcourtman/pulse-go-rewrite/internal/models"+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"+)++// These actual tool controls use APIs present in the parent. The parent gets+// the tests, not the repair, for a discriminating comparison.+func TestGuestMemoryQueryEvidence(t *testing.T) {+ at := time.Now().UTC().Add(-time.Minute).Truncate(time.Second)+ for _, kind := range []string{"vm", "system-container"} {+ for _, tc := range []struct {+ name, state, source string+ percent float64+ at time.Time+ missing, known bool+ }{+ {"measured-zero", "current", "agent", 0, at, false, true},+ {"healthy-available", "current", "guest-agent-meminfo", 24, at, false, true},+ {"real-pressure", "current", "available-field", 96, at, false, true},+ {"status-cache", "current", "status-mem", 96, at, false, false},+ {"retained-low", "last-known", "agent", 24, at.Add(-time.Hour), false, false},+ {"retained-high", "last-known", "guest-agent-meminfo", 96, at.Add(-time.Hour), false, false},+ {"undated", "current", "available-field", 24, time.Time{}, false, false},+ {"future", "current", "agent", 24, time.Now().Add(time.Hour), false, false},+ {"unknown-source", "current", "unrecognised-source", 24, at, false, false},+ {"unavailable", "unavailable", "unavailable", 0, time.Time{}, false, false},+ {"missing", "", "", 0, time.Time{}, true, false},+ {"invalid", "current", "agent", math.NaN(), at, false, false},+ } {+ for _, path := range []string{"canonical", "typed-view"} {+ t.Run(kind+"/"+tc.name+"/"+path, func(t *testing.T) {+ total, used := int64(20<<30), int64(5<<30)+ r := unifiedresources.Resource{ID: "selected-guest", Name: "selected-guest", Type: unifiedresources.ResourceType(kind), Status: unifiedresources.StatusOnline, LastSeen: time.Now(),+ Proxmox: &unifiedresources.ProxmoxData{VMID: 100, NodeName: "node", Instance: "fixture", SourceID: "source-guest", RuntimeStatus: "running",+ Memory: &models.Memory{Usage: 99, Observation: models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: time.Now()}}},+ Metrics: &unifiedresources.ResourceMetrics{Memory: &unifiedresources.MetricValue{Percent: tc.percent, Used: &used, Total: &total, Observation: models.MemoryObservation{State: tc.state, Source: tc.source, ObservedAt: tc.at}}},+ }+ if tc.missing {+ r.Metrics.Memory = nil+ }+ vm, ct := unifiedresources.NewVMView(&r), unifiedresources.NewContainerView(&r)+ rs := &fakeReadState{}+ if kind == "vm" {+ rs.vms = []*unifiedresources.VMView{&vm}+ } else {+ rs.containers = []*unifiedresources.ContainerView{&ct}+ }+ cfg := ExecutorConfig{ReadState: rs}+ if path == "canonical" {+ cfg.UnifiedResourceProvider = &stubUnifiedResourceProvider{resources: []unifiedresources.Resource{r}}+ }+ exec := NewPulseToolExecutor(cfg)+ for _, action := range []string{"get", "list", "topology"} {+ result, err := exec.ExecuteTool(context.Background(), agentcapabilities.PulseQueryToolName, map[string]interface{}{"action": action, "resource_type": kind, "resource_id": r.ID})+ if err != nil || result.IsError || len(result.Content) == 0 {+ t.Fatalf("%s read failed: err=%v result=%+v", action, err, result)+ }+ var decoded map[string]interface{}+ if err := json.Unmarshal([]byte(result.Content[0].Text), &decoded); err != nil {+ t.Fatal(err)+ }+ var evidence interface{}+ var percent interface{}+ if action == "get" {+ memory := decoded["memory"].(map[string]interface{})+ evidence, percent = memory["evidence"], memory["percent"]+ if (tc.missing || tc.state == "unavailable" || math.IsNaN(tc.percent)) && memory["used_gb"] != nil {+ t.Error("missing usage became zero used bytes")+ }+ } else {+ rows := decoded+ if action == "topology" {+ rows = decoded["proxmox"].(map[string]interface{})["nodes"].([]interface{})[0].(map[string]interface{})+ }+ family := "vms"+ if kind != "vm" {+ family = "containers"+ }+ row := rows[family].([]interface{})[0].(map[string]interface{})+ evidence, percent = row["memory_evidence"], row["memory_percent"]+ }+ e, ok := evidence.(map[string]interface{})+ if !ok {+ t.Errorf("%s lost guest memory qualification: %s", action, result.Content[0].Text)+ continue+ }+ available := !tc.missing && tc.state != "unavailable" && !math.IsNaN(tc.percent)+ if e["available"] != available || e["pressure_known"] != tc.known {+ t.Errorf("%s qualification=%+v want available=%t known=%t", action, e, available, tc.known)+ }+ if available && percent != tc.percent || !available && percent != nil {+ t.Errorf("%s percent=%v want available=%t percent=%g", action, percent, available, tc.percent)+ }+ if !tc.at.IsZero() && !tc.at.After(time.Now()) && available && e["observed_at"] != tc.at.Format(time.RFC3339) {+ t.Errorf("%s renewed/lost original time: %+v", action, e)+ }+ if tc.source == "status-mem" && e["may_include_reclaimable_cache"] != true {+ t.Errorf("%s erased cache distinction", action)+ }+ }+ })+ }+ }+ }+}++func TestGuestMemoryHistoryDoesNotBorrowLiveOrigin(t *testing.T) {+ for _, family := range []string{"vm", "system-container", "agent"} {+ t.Run(family, func(t *testing.T) {+ history := &mockMetricsHistoryProvider{}+ history.On("GetResourceMetrics", "source-guest", 24*time.Hour).Return([]MetricPoint{{Memory: 96}}, nil)+ history.On("GetAllMetricsSummary", 24*time.Hour).Return(map[string]ResourceMetricsSummary{"selected-guest": {ResourceType: family, AvgMemory: 96, MaxMemory: 99}}, nil)+ r := unifiedresources.Resource{ID: "selected-guest", Type: unifiedresources.ResourceType(family), MetricsTarget: &unifiedresources.MetricsTarget{ResourceType: family, ResourceID: "source-guest"},+ Metrics: &unifiedresources.ResourceMetrics{Memory: &unifiedresources.MetricValue{Percent: 24, Observation: models.MemoryObservation{State: "current", Source: "agent", ObservedAt: time.Now().Add(-time.Minute)}}}}+ exec := NewPulseToolExecutor(ExecutorConfig{StateProvider: &mockStateProvider{}, MetricsHistory: history, UnifiedResourceProvider: &stubUnifiedResourceProvider{resources: []unifiedresources.Resource{r}}})+ for _, id := range []string{r.ID, ""} {+ result, err := exec.ExecuteTool(context.Background(), agentcapabilities.PulseMetricsToolName, map[string]interface{}{"type": "performance", "resource_id": id})+ if err != nil || result.IsError {+ t.Fatalf("history read failed: %v %+v", err, result)+ }+ var decoded map[string]interface{}+ if err := json.Unmarshal([]byte(result.Content[0].Text), &decoded); err != nil {+ t.Fatal(err)+ }+ note, _ := decoded["memory_interpretation"].(string)+ if family != "agent" && (!strings.Contains(note, "per-sample") || !strings.Contains(note, "cannot reattribute")) {+ t.Errorf("history borrowed live memory trust: %s", result.Content[0].Text)+ }+ if family == "agent" && note != "" {+ t.Error("non-guest history changed")+ }+ }+ history.AssertExpectations(t)+ })+ }+}diff --git a/internal/ai/tools/tools_metrics.go b/internal/ai/tools/tools_metrics.goindex 8f987b69ca..ba8c082534 100644--- a/internal/ai/tools/tools_metrics.go+++ b/internal/ai/tools/tools_metrics.go@@ -28,6 +28,10 @@ Types:- baselines: Learned normal behavior baselines for resources- patterns: Detected operational patterns and predictions+Performance responses expose numeric guest-memory samples without per-sample memory-source+provenance. Do not infer a guest leak, pressure or current recovery from a high+historical percentage alone, or lend past samples today's selected source.+Examples:- Get 24h metrics: type="performance", period="24h"- Get VM metrics: type="performance", resource_id="101"@@ -162,6 +166,9 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[stringresponse := EmptyMetricsResponse()response.Period = period+ if resourceType == "vm" || resourceType == "system-container" {+ response.MemoryInterpretation = guestMemoryHistoryInterpretation+ }if resourceID != "" {response.ResourceID = resourceID@@ -173,6 +180,9 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[stringreturn NewErrorResult(err), nil}response.ResourceID = resource.ID+ if resource.Type == unifiedresources.ResourceTypeVM || resource.Type == unifiedresources.ResourceTypeSystemContainer {+ response.MemoryInterpretation = guestMemoryHistoryInterpretation+ }target := e.resourceMetricsTarget(resource)if target == nil || strings.TrimSpace(target.ResourceID) == "" {return NewTextResult(fmt.Sprintf("No metrics target is available for resource %s.", resource.ID)), nil@@ -232,6 +242,9 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[string}metric := summary[id]metric.ResourceType = canonicalMetricsResourceType(metric.ResourceType)+ if metric.ResourceType == "vm" || metric.ResourceType == "system-container" {+ response.MemoryInterpretation = guestMemoryHistoryInterpretation+ }filtered[id] = metrictotal++}@@ -252,6 +265,8 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[stringreturn NewJSONResult(response.NormalizeCollections()), nil}+const guestMemoryHistoryInterpretation = "This guest memory history response has no per-sample source or pressure qualification. Percentages may include reclaimable cache and do not independently prove guest pressure, a leak or current recovery. Live observations cannot reattribute past samples."+func (e *PulseToolExecutor) executeGetBaselines(_ context.Context, args map[string]interface{}) (CallToolResult, error) {resourceID, _ := args["resource_id"].(string)resourceType, _ := args["resource_type"].(string)diff --git a/internal/ai/tools/tools_query.go b/internal/ai/tools/tools_query.goindex dcc46b4f27..b0cf532ecb 100644--- a/internal/ai/tools/tools_query.go+++ b/internal/ai/tools/tools_query.go@@ -2152,7 +2152,7 @@ func (e *PulseToolExecutor) registerQueryTools() {e.registry.registerBuiltin(RegisteredTool{Definition: Tool{Name: agentcapabilities.PulseQueryToolName,- Description: `Query and search canonical infrastructure resources. Start here to discover systems, workloads, storage, and disks by name. Actions: search, get, config, topology, list, health, action. Use action with action_id to read the canonical persisted plan, decision state and independently verified execution outcome. Inventory and incident history can lag execution and cannot establish whether an action was approved or run. For app-container get, filesystems reports observed capacity at each mountpoint, not container quotas. An observation error has no usage payload. Mounts describe configuration. Health returns the connection overview by default, or the canonical resource projection when resource_id is provided. command_agent_connected describes live command transport, independently of monitoring collection or freshness. Missing connection fields were not observed. can_execute describes connected transport with control enabled, not approval for a particular operation.`,+ Description: `Query and search canonical infrastructure resources. Start here to discover systems, workloads, storage, and disks by name. Actions: search, get, config, topology, list, health, action. Use action with action_id to read the canonical persisted plan, decision state and independently verified execution outcome. Inventory and incident history can lag execution and cannot establish whether an action was approved or run. For app-container get, filesystems reports observed capacity at each mountpoint, not container quotas. An observation error has no usage payload. Mounts describe configuration. Health returns the connection overview by default, or the canonical resource projection when resource_id is provided. command_agent_connected describes live command transport, independently of monitoring collection or freshness. Missing connection fields were not observed. can_execute describes connected transport with control enabled, not approval for a particular operation. Guest memory evidence describes the selected reading: null is unavailable, zero can be measured, and cache-inclusive or retained percentages do not prove guest pressure or recovery. Use its original observed_at, not the resource timestamp.`,InputSchema: InputSchema{Type: "object",Properties: map[string]PropertySchema{@@ -2959,6 +2959,7 @@ type queryGuestView interface {CPUPercent() float64CPUs() intMemoryPercent() float64+ MemoryEvidence(time.Time) unifiedresources.GuestMemoryEvidenceMemoryUsed() int64MemoryTotal() int64Tags() []string@@ -3080,8 +3081,6 @@ func guestViewGetResult[V queryGuestView](e *PulseToolExecutor, guests []V, kindif fmt.Sprintf("%d", g.VMID()) != resourceID && g.Name() != resourceID && g.ID() != resourceID {continue}- used := g.MemoryUsed()- total := g.MemoryTotal()response := EmptyResourceResponse()response.GovernedResourceMetadata = governance.Resolve(g.Name(), g.ID(), fmt.Sprintf("%d", g.VMID()))response.Type = kind@@ -3093,11 +3092,7 @@ func guestViewGetResult[V queryGuestView](e *PulseToolExecutor, guests []V, kindPercent: g.CPUPercent(),Cores: g.CPUs(),}- response.Memory = ResourceMemory{- Percent: g.MemoryPercent(),- UsedGB: float64(used) / (1024 * 1024 * 1024),- TotalGB: float64(total) / (1024 * 1024 * 1024),- }+ response.Memory = guestViewMemory(g, time.Now())response.Tags = g.Tags()if !g.LastBackup().IsZero() {t := g.LastBackup()@@ -3128,6 +3123,38 @@ func guestViewGetResult[V queryGuestView](e *PulseToolExecutor, guests []V, kindreturn CallToolResult{}, false}+func guestMemoryFields(e unifiedresources.GuestMemoryEvidence, percent float64) (*float64, *unifiedresources.GuestMemoryEvidence) {+ var value *float64+ if e.Available {+ value = &percent+ }+ if e.State == "" { // unchanged other-platform reading+ return value, nil+ }+ return value, &e+}++func guestViewMemory(g queryGuestView, now time.Time) ResourceMemory {+ percent, evidence := guestMemoryFields(g.MemoryEvidence(now), g.MemoryPercent())+ memory := ResourceMemory{Percent: percent, TotalGB: float64(g.MemoryTotal()) / (1024 * 1024 * 1024), Evidence: evidence}+ if percent != nil && g.MemoryTotal() > 0 {+ usedGB := float64(g.MemoryUsed()) / (1024 * 1024 * 1024)+ memory.UsedGB = &usedGB+ }+ return memory+}++func canonicalGuestMemory(resource unifiedresources.Resource, now time.Time) ResourceMemory {+ m := resourceMetric(resource, "memory")+ percent, evidence := guestMemoryFields(unifiedresources.GuestMemoryEvidenceForResource(&resource, now), metricPercent(m))+ memory := ResourceMemory{Percent: percent, TotalGB: metricTotalGB(m), Evidence: evidence}+ if percent != nil && m != nil && m.Used != nil {+ usedGB := metricUsedGB(m)+ memory.UsedGB = &usedGB+ }+ return memory+}+func canonicalGuestTarget(resource unifiedresources.Resource) string {if resource.Proxmox != nil {return strings.TrimSpace(resource.Proxmox.NodeName)@@ -3275,11 +3302,7 @@ func canonicalGuestResponse(kind string, resource unifiedresources.Resource, govresponse.CPU = ResourceCPU{Percent: metricPercent(resourceMetric(resource, "cpu")),}- response.Memory = ResourceMemory{- Percent: metricPercent(resourceMetric(resource, "memory")),- UsedGB: metricUsedGB(resourceMetric(resource, "memory")),- TotalGB: metricTotalGB(resourceMetric(resource, "memory")),- }+ response.Memory = canonicalGuestMemory(resource, time.Now())if diskMetric := resourceMetric(resource, "disk"); diskMetric != nil {response.Disk = &ResourceDisk{Percent: metricPercent(diskMetric),@@ -3331,11 +3354,7 @@ func canonicalAgentResponse(resource unifiedresources.Resource, governance *goveresponse.CPU = ResourceCPU{Percent: metricPercent(resourceMetric(resource, "cpu")),}- response.Memory = ResourceMemory{- Percent: metricPercent(resourceMetric(resource, "memory")),- UsedGB: metricUsedGB(resourceMetric(resource, "memory")),- TotalGB: metricTotalGB(resourceMetric(resource, "memory")),- }+ response.Memory = resourceMemoryValues(metricPercent(resourceMetric(resource, "memory")), metricUsedGB(resourceMetric(resource, "memory")), metricTotalGB(resourceMetric(resource, "memory")))if diskMetric := resourceMetric(resource, "disk"); diskMetric != nil {response.Disk = &ResourceDisk{Percent: metricPercent(diskMetric),@@ -3861,6 +3880,7 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args macount++continue}+ memoryPercent, memoryEvidence := guestMemoryFields(vm.MemoryEvidence(time.Now()), vm.MemoryPercent())response.VMs = append(response.VMs, VMSummary{GovernedResourceMetadata: governance.Resolve(vm.Name(), vm.ID(), fmt.Sprintf("%d", vm.VMID())),VMID: vm.VMID(),@@ -3868,7 +3888,8 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args maStatus: string(vm.Status()),Node: vm.Node(),CPU: vm.CPUPercent(),- Memory: vm.MemoryPercent(),+ Memory: memoryPercent,+ MemoryEvidence: memoryEvidence,})count++}@@ -3892,6 +3913,7 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args macount++continue}+ memoryPercent, memoryEvidence := guestMemoryFields(ct.MemoryEvidence(time.Now()), ct.MemoryPercent())response.Containers = append(response.Containers, ContainerSummary{GovernedResourceMetadata: governance.Resolve(ct.Name(), ct.ID(), fmt.Sprintf("%d", ct.VMID())),VMID: ct.VMID(),@@ -3899,7 +3921,8 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args maStatus: string(ct.Status()),Node: ct.Node(),CPU: ct.CPUPercent(),- Memory: ct.MemoryPercent(),+ Memory: memoryPercent,+ MemoryEvidence: memoryEvidence,})count++}@@ -4400,13 +4423,15 @@ func BuildTopologyResponseFromReadState(rs unifiedresources.ReadState, options T}nodeTopology.VMCount++if options.MaxVMsPerNode <= 0 || len(nodeTopology.VMs) < options.MaxVMsPerNode {+ memoryPercent, memoryEvidence := guestMemoryFields(vm.MemoryEvidence(time.Now()), vm.MemoryPercent())nodeTopology.VMs = append(nodeTopology.VMs, TopologyVM{GovernedResourceMetadata: governance.Resolve(vm.Name(), vm.ID(), fmt.Sprintf("%d", vm.VMID())),VMID: vm.VMID(),Name: vm.Name(),Status: status,CPU: vm.CPUPercent(),- Memory: vm.MemoryPercent(),+ Memory: memoryPercent,+ MemoryEvidence: memoryEvidence,Tags: vm.Tags(),})}@@ -4426,13 +4451,15 @@ func BuildTopologyResponseFromReadState(rs unifiedresources.ReadState, options T}nodeTopology.ContainerCount++if options.MaxContainersPerNode <= 0 || len(nodeTopology.Containers) < options.MaxContainersPerNode {+ memoryPercent, memoryEvidence := guestMemoryFields(ct.MemoryEvidence(time.Now()), ct.MemoryPercent())nodeTopology.Containers = append(nodeTopology.Containers, TopologyContainer{GovernedResourceMetadata: governance.Resolve(ct.Name(), ct.ID(), fmt.Sprintf("%d", ct.VMID())),VMID: ct.VMID(),Name: ct.Name(),Status: status,CPU: ct.CPUPercent(),- Memory: ct.MemoryPercent(),+ Memory: memoryPercent,+ MemoryEvidence: memoryEvidence,Tags: ct.Tags(),})}@@ -4947,11 +4974,7 @@ func (e *PulseToolExecutor) executeGetResource(_ context.Context, args map[strinresponse.Host = hostNameresponse.OS = strings.TrimSpace(host.OS())response.CPU = ResourceCPU{Percent: host.CPUPercent(), Cores: host.CPUs()}- response.Memory = ResourceMemory{- Percent: host.MemoryPercent(),- UsedGB: float64(host.MemoryUsed()) / (1024 * 1024 * 1024),- TotalGB: float64(host.MemoryTotal()) / (1024 * 1024 * 1024),- }+ response.Memory = resourceMemoryValues(host.MemoryPercent(), float64(host.MemoryUsed())/(1024*1024*1024), float64(host.MemoryTotal())/(1024*1024*1024))response.Tags = host.Tags()e.registerResolvedResourceWithExplicitAccess(ResourceRegistration{Kind: "docker-host",@@ -5022,11 +5045,7 @@ func (e *PulseToolExecutor) executeGetResource(_ context.Context, args map[strinresponse.CPU = ResourceCPU{Percent: metricPercent(resourceMetric(resource, "cpu")),}- response.Memory = ResourceMemory{- Percent: metricPercent(resourceMetric(resource, "memory")),- UsedGB: metricUsedGB(resourceMetric(resource, "memory")),- TotalGB: metricTotalGB(resourceMetric(resource, "memory")),- }+ response.Memory = resourceMemoryValues(metricPercent(resourceMetric(resource, "memory")), metricUsedGB(resourceMetric(resource, "memory")), metricTotalGB(resourceMetric(resource, "memory")))if diskMetric := resourceMetric(resource, "disk"); diskMetric != nil {response.Disk = &ResourceDisk{Percent: metricPercent(diskMetric),@@ -5141,11 +5160,7 @@ func (e *PulseToolExecutor) executeGetResource(_ context.Context, args map[strinresponse.CPU = ResourceCPU{Percent: container.CPUPercent(),}- response.Memory = ResourceMemory{- Percent: container.MemoryPercent(),- UsedGB: float64(container.MemoryUsed()) / (1024 * 1024 * 1024),- TotalGB: float64(container.MemoryTotal()) / (1024 * 1024 * 1024),- }+ response.Memory = resourceMemoryValues(container.MemoryPercent(), float64(container.MemoryUsed())/(1024*1024*1024), float64(container.MemoryTotal())/(1024*1024*1024))response.RestartCount = container.RestartCount()response.Labels = container.Labels()diff --git a/internal/unifiedresources/guest_memory.go b/internal/unifiedresources/guest_memory.gonew file mode 100644index 0000000000..38ea31bc26--- /dev/null+++ b/internal/unifiedresources/guest_memory.go@@ -0,0 +1,96 @@+package unifiedresources++import (+ "fmt"+ "math"+ "time"++ "github.com/rcourtman/pulse-go-rewrite/internal/models"+)++// GuestMemoryEvidence qualifies the selected numeric reading. It is not+// permission to obtain another reading or evidence that a guest has recovered.+// Available distinguishes a measured zero from an absent/invalid percentage.+type GuestMemoryEvidence struct {+ Available bool `json:"available"`+ PressureKnown bool `json:"pressure_known"`+ State string `json:"state,omitempty"`+ Source string `json:"source,omitempty"`+ ObservedAt time.Time `json:"observed_at,omitzero"`+ MayIncludeCache bool `json:"may_include_reclaimable_cache,omitempty"`+}++// QualifyGuestMemory shares the existing Patrol interpretation with its query+// and observer inputs. State is authored by collection; consumers do not renew+// the original observation or invent a separate collection lease.+func QualifyGuestMemory(percent float64, observation models.MemoryObservation, available, proxmoxGuest bool, now time.Time) GuestMemoryEvidence {+ e := GuestMemoryEvidence{Available: available && !math.IsNaN(percent) && !math.IsInf(percent, 0) && percent >= 0 && percent <= 100}+ if !e.Available || observation.State == "unavailable" {+ e.Available = false+ e.State = "unavailable"+ return e+ }+ // Other platforms need not implement Proxmox's memory observation contract.+ if !proxmoxGuest && observation == (models.MemoryObservation{}) {+ e.PressureKnown = true+ return e+ }+ e.State = "unknown"+ if observation.ObservedAt.After(time.Unix(0, 0)) && !observation.ObservedAt.After(now) {+ e.ObservedAt = observation.ObservedAt+ }+ if observation.State == "last-known" {+ e.State = "last-known"+ } else if observation.State == "current" && !e.ObservedAt.IsZero() {+ e.State = "current"+ }+ // Source names are closed facts, not arbitrary upstream text in AI context.+ switch observation.Source {+ case "available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent":+ e.Source = observation.Source+ e.PressureKnown = e.State == "current"+ case "status-mem", "status-freemem", "cluster-resources", "derived-total-minus-used":+ e.Source = observation.Source+ e.MayIncludeCache = true+ case "previous-snapshot":+ e.Source = observation.Source+ }+ return e+}++// Format keeps qualification when a tool result is condensed into model facts.+func (e GuestMemoryEvidence) Format(percent float64) string {+ if !e.Available {+ return "N/A (guest memory unavailable; not evidence of recovery)"+ }+ if e.State == "" {+ return fmt.Sprintf("%.0f%%", percent)+ }+ source := e.Source+ if source == "" {+ source = "unknown"+ }+ observed := "time unknown"+ if !e.ObservedAt.IsZero() {+ observed = "observed " + e.ObservedAt.UTC().Format(time.RFC3339)+ }+ qualification := "cache-aware guest usage"+ if !e.PressureKnown {+ qualification = "guest pressure unknown"+ if e.MayIncludeCache {+ qualification += "; may include reclaimable cache"+ }+ }+ return fmt.Sprintf("%.0f%% (%s; source %s; %s; %s)", percent, e.State, source, observed, qualification)+}++// GuestMemoryEvidenceForResource reads only the selected metric. A conflicting+// raw platform facet or resource LastSeen must never lend it provenance.+func GuestMemoryEvidenceForResource(r *Resource, now time.Time) GuestMemoryEvidence {+ if r == nil || r.Metrics == nil || r.Metrics.Memory == nil {+ return QualifyGuestMemory(0, models.MemoryObservation{}, false, true, now)+ }+ _, hasProxmox := r.SourceStatus[SourceProxmox]+ m := r.Metrics.Memory+ return QualifyGuestMemory(m.Percent, m.Observation, true, r.Proxmox != nil || hasProxmox || m.Source == SourceProxmox, now)+}diff --git a/internal/unifiedresources/metrics_test.go b/internal/unifiedresources/metrics_test.goindex 0e5616d115..60ea3ea09b 100644--- a/internal/unifiedresources/metrics_test.go+++ b/internal/unifiedresources/metrics_test.go@@ -2,6 +2,7 @@ package unifiedresourcesimport ("math"+ "strings""testing""time"@@ -575,3 +576,47 @@ func TestGuestMemoryMetricObservationFollowsSelectedSource(t *testing.T) {t.Fatalf("cross-source selection lost agent receipt or trusts supplied provenance: %+v", merged.Memory)}}++func TestGuestMemoryEvidenceQualification(t *testing.T) {+ now := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)+ at := now.Add(-time.Minute)+ for _, source := range []string{"available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent"} {+ for _, percent := range []float64{0, 24, 96, 100} {+ e := QualifyGuestMemory(percent, models.MemoryObservation{State: "current", Source: source, ObservedAt: at}, true, true, now)+ if !e.Available || !e.PressureKnown || e.Source != source || e.ObservedAt != at || e.MayIncludeCache {+ t.Errorf("current measured control rejected: %s/%g: %+v", source, percent, e)+ }+ }+ }+ for _, source := range []string{"status-mem", "status-freemem", "cluster-resources", "derived-total-minus-used", "previous-snapshot", "unrecognised-source"} {+ e := QualifyGuestMemory(96, models.MemoryObservation{State: "current", Source: source, ObservedAt: at}, true, true, now)+ if !e.Available || e.PressureKnown || !strings.Contains(e.Format(96), "guest pressure unknown") {+ t.Errorf("unqualified source became pressure: %s: %+v", source, e)+ }+ if source == "unrecognised-source" && e.Source != "" {+ t.Fatal("unknown source text leaked into context")+ }+ }+ for _, percent := range []float64{-1, 101, math.NaN(), math.Inf(1), math.Inf(-1)} {+ e := QualifyGuestMemory(percent, models.MemoryObservation{State: "current", Source: "agent", ObservedAt: at}, true, true, now)+ if e.Available || e.PressureKnown || e.State != "unavailable" {+ t.Errorf("invalid percentage became evidence: %+v", e)+ }+ }+ for _, badTime := range []time.Time{{}, time.Unix(0, 0), now.Add(time.Second)} {+ e := QualifyGuestMemory(24, models.MemoryObservation{State: "current", Source: "agent", ObservedAt: badTime}, true, true, now)+ if e.PressureKnown || !e.ObservedAt.IsZero() || e.State != "unknown" {+ t.Errorf("invalid origin became current: %+v", e)+ }+ }+ for _, state := range []string{"last-known", "", "future-state", "unavailable"} {+ e := QualifyGuestMemory(24, models.MemoryObservation{State: state, Source: "agent", ObservedAt: at}, true, true, now)+ if e.PressureKnown {+ t.Errorf("unqualified state became current: %s: %+v", state, e)+ }+ }+ legacy := QualifyGuestMemory(24, models.MemoryObservation{}, true, false, now)+ if !legacy.Available || !legacy.PressureKnown || legacy.Format(24) != "24%" {+ t.Fatal("other platform contract changed")+ }+}diff --git a/internal/unifiedresources/views.go b/internal/unifiedresources/views.goindex 09dcbebb94..a7a295d038 100644--- a/internal/unifiedresources/views.go+++ b/internal/unifiedresources/views.go@@ -390,6 +390,11 @@ func (v VMView) MemoryObservation() (models.MemoryObservation, bool) {return selectedMemoryObservation(v.r)}+// MemoryEvidence qualifies the selected metric without borrowing a raw facet or row timestamp.+func (v VMView) MemoryEvidence(now time.Time) GuestMemoryEvidence {+ return GuestMemoryEvidenceForResource(v.r, now)+}+func selectedMemoryObservation(r *Resource) (models.MemoryObservation, bool) {if r == nil || r.Metrics == nil || r.Metrics.Memory == nil {return models.MemoryObservation{}, false@@ -727,6 +732,11 @@ func (v ContainerView) MemoryObservation() (models.MemoryObservation, bool) {return selectedMemoryObservation(v.r)}+// MemoryEvidence qualifies the selected metric without borrowing a raw facet or row timestamp.+func (v ContainerView) MemoryEvidence(now time.Time) GuestMemoryEvidence {+ return GuestMemoryEvidenceForResource(v.r, now)+}+func (v ContainerView) DiskUsed() int64 {if v.r == nil {return 0diff --git a/internal/unifiedresources/views_test.go b/internal/unifiedresources/views_test.goindex 1e8abe363c..f035ee84f7 100644--- a/internal/unifiedresources/views_test.go+++ b/internal/unifiedresources/views_test.go@@ -2587,3 +2587,26 @@ func TestGuestViewMemoryObservationKeepsSelectedOrigin(t *testing.T) {})}}++func TestGuestMemoryEvidenceUsesSelectedMetricOnly(t *testing.T) {+ now := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)+ selected := models.MemoryObservation{State: "last-known", Source: "agent", ObservedAt: now.Add(-time.Hour)}+ r := Resource{LastSeen: now, UpdatedAt: now,+ Proxmox: &ProxmoxData{Memory: &models.Memory{Usage: 96, Observation: models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: now}}},+ Metrics: &ResourceMetrics{Memory: &MetricValue{Percent: 24, Observation: selected}},+ }+ for _, read := range []func() GuestMemoryEvidence{+ func() GuestMemoryEvidence { return NewVMView(&r).MemoryEvidence(now) },+ func() GuestMemoryEvidence { return NewContainerView(&r).MemoryEvidence(now) },+ } {+ e := read()+ if e.PressureKnown || e.State != "last-known" || e.Source != selected.Source || e.ObservedAt != selected.ObservedAt {+ t.Fatalf("raw facet or fresh row replaced selected origin: %+v", e)+ }+ r.Metrics.Memory = nil+ if e = read(); e.Available || e.PressureKnown || e.State != "unavailable" {+ t.Fatal("missing selected metric borrowed raw facet")+ }+ r.Metrics.Memory = &MetricValue{Percent: 24, Observation: selected}+ }+}