Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Preserve guest memory evidence across AI queries and local observers

pulse-triage[bot]a721b3a8e0d5
diff --git a/docs/release-control/v6/internal/subsystems/ai-runtime.md b/docs/release-control/v6/internal/subsystems/ai-runtime.md
index ce1a7d1c22..30b50b2a63 100644
--- a/docs/release-control/v6/internal/subsystems/ai-runtime.md
+++ b/docs/release-control/v6/internal/subsystems/ai-runtime.md
@@ -25,6 +25,34 @@ that same result. Successful reads retain their content and execution provenance
## Purpose
+### Guest memory remains qualified through tools and local observers
+
+The existing guest get/list/topology tools and Assistant inventory context carry
+selected memory evidence with the percentage. Unavailable/invalid memory is
+null, not zero usage; a measured zero stays numeric. Evidence retains source,
+state and original observation time, including unknown pressure and possible
+reclaimable cache. Condensed query facts retain the same qualification rather
+than saving a bare percentage. `TestGuestMemoryQueryEvidence` and
+`TestAssistantGuestMemoryContextAndFacts` check the real tool/context/fact paths.
+
+The existing local memory observer uses that same interpretation for VM/system
+containers. Unknown pressure wakes it with an explicit evidence gap, not a
+numeric breach or a healthy predicate. A known selected guest sample is bounded
+by its own observation age; refreshing the resource through another source does
+not renew it. Other-platform legacy readings and other metric observers retain
+their existing semantics. `TestPatrolObserverGuestMemoryOrigin` checks unknown,
+retained, undated, future, invalid and stale origins alongside genuine low/zero/
+high pressure and the age boundary. This changes no objective ABI, threshold,
+collection lease, observer authority or guest-command admission.
+
+Performance-tool guest memory history remains numeric, but explicitly lacks
+per-sample source/pressure qualification. A live current reading cannot reattribute
+past samples or prove that past high usage was pressure/a leak.
+`TestGuestMemoryHistoryDoesNotBorrowLiveOrigin` checks unchanged numeric history,
+its truthful limitation and non-guest compatibility. No History writer/schema
+or existing stored finding is changed. These controls do not establish #2762's
+running source, episode cause, native recovery or a containing release.
+
### Patrol guest memory is qualified evidence, not an unlabelled percentage
Patrol consumes the selected guest memory metric's original observation,
diff --git a/docs/release-control/v6/internal/subsystems/api-contracts.md b/docs/release-control/v6/internal/subsystems/api-contracts.md
index 41f3fa8985..e5e7b3702f 100644
--- a/docs/release-control/v6/internal/subsystems/api-contracts.md
+++ b/docs/release-control/v6/internal/subsystems/api-contracts.md
@@ -20,6 +20,20 @@
## Purpose
+### AI guest memory projections preserve uncertainty
+
+Existing AI get/list/topology responses carry selected guest memory evidence:
+availability, pressure qualification, source, state and original observation time.
+A missing/invalid guest percentage is JSON null rather than zero; a measured zero
+is explicitly numeric, including in list/topology and Assistant inventory. Get
+usage bytes are null when unavailable. Non-guest response numbers remain unchanged.
+The added evidence is a read-only projection, not a new route, command, feature,
+collector or diagnostic authority. `TestGuestMemoryQueryEvidence` exercises actual
+tool serialization and `TestAssistantGuestMemoryContextAndFacts` preserves it
+through Assistant condensation. Performance responses add a guest-history source
+limitation without relabelling past samples or changing their numbers; their
+actual tool controls are in `TestGuestMemoryHistoryDoesNotBorrowLiveOrigin`.
+
### Optional held-alert breach date consumer
`MetricAlertStatus.lastBreachAt?: string` mirrors the existing backend ISO
diff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.md
index 219fe2b0b0..5f26dd6952 100644
--- a/docs/release-control/v6/internal/subsystems/monitoring.md
+++ b/docs/release-control/v6/internal/subsystems/monitoring.md
@@ -17,6 +17,18 @@
## Purpose
+### Existing memory origins cannot be renewed by AI consumers
+
+AI query/context/fact consumers now retain the selected guest observation and
+explicit availability. The existing local VM/system-container memory observer
+judges pressure only from a qualified selected sample and applies its age bound
+to that original observation, not a fresh resource LastSeen from another source.
+Unknown pressure is an evidence-gap wake, not a measured threshold breach or
+recovery. `TestPatrolObserverGuestMemoryOrigin` and
+`TestGuestMemoryEvidenceUsesSelectedMetricOnly` cover these consumption boundaries.
+Collectors, guest commands, native leases, shared safety pauses, History storage
+and configured thresholds remain unchanged; this is not native #2762 recovery.
+
### Guest memory evidence survives typed consumers
The monitor's existing guest memory observation remains source-owned when
diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.md
index b2c41cb7f2..710e2f90b5 100644
--- a/docs/release-control/v6/internal/subsystems/unified-resources.md
+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md
@@ -23,6 +23,25 @@ and sort the complete canonical change table while startup and ingestion wait.
## Purpose
+### Shared interpretation of selected guest memory
+
+`GuestMemoryEvidenceForResource` and the VM/container `MemoryEvidence` accessors
+qualify only the selected numeric metric through `QualifyGuestMemory`. Presence
+and finite 0–100 values distinguish a measured zero from absence. Only current,
+dated cache-aware Proxmox/QGA/agent origins establish pressure; fallback/cache-
+inclusive, unknown and retained origins keep their limitations. Consumers cannot
+borrow raw-facet provenance or resource LastSeen. Collection owns current/last-
+known state and its lease; local observers additionally enforce their configured
+age on the original memory observation, without creating a new collection TTL.
+
+Source names are allowlisted before entering AI context; unknown names remain
+unknown. Other-platform readings without this Proxmox observation contract keep
+existing semantics. `TestGuestMemoryEvidenceQualification` covers these boundaries
+and `TestGuestMemoryEvidenceUsesSelectedMetricOnly` covers selected/raw disagreement,
+absence and refreshed rows in both views. This shared interpretation changes
+no registry wire format, metric selection, guest collection, History, correlation,
+identity or permission. AI query projections own their additional evidence fields.
+
### Selected guest memory observation in typed read state
VM and system-container views expose a value-copy of the selected memory
diff --git a/internal/ai/chat/guest_memory_evidence_test.go b/internal/ai/chat/guest_memory_evidence_test.go
new file mode 100644
index 0000000000..682f88f215
--- /dev/null
+++ b/internal/ai/chat/guest_memory_evidence_test.go
@@ -0,0 +1,84 @@
+package chat
+
+import (
+ "context"
+ "encoding/json"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"
+ "github.com/rcourtman/pulse-go-rewrite/internal/ai/tools"
+ "github.com/rcourtman/pulse-go-rewrite/internal/models"
+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
+)
+
+func TestAssistantGuestMemoryContextAndFacts(t *testing.T) {
+ at := time.Now().UTC().Add(-time.Minute).Truncate(time.Second)
+ for _, kind := range []string{"vm", "system-container"} {
+ for _, tc := range []struct {
+ name, state, source, text string
+ percent float64
+ unavailable, known bool
+ }{
+ {"healthy", "current", "guest-agent-meminfo", "cache-aware guest usage", 24, false, true},
+ {"measured-zero", "current", "agent", "0% (current", 0, false, true},
+ {"pressure", "current", "available-field", "cache-aware guest usage", 96, false, true},
+ {"cache", "current", "status-mem", "may include reclaimable cache", 96, false, false},
+ {"retained", "last-known", "agent", "guest pressure unknown", 24, false, false},
+ {"missing", "unavailable", "unavailable", "N/A (guest memory unavailable; not evidence of recovery)", 0, true, false},
+ } {
+ t.Run(kind+"/"+tc.name, func(t *testing.T) {
+ memory := models.Memory{Total: 20 << 30, Used: int64(float64(20<<30) * tc.percent / 100), Usage: tc.percent, UsageUnavailable: tc.unavailable,
+ Observation: models.MemoryObservation{State: tc.state, Source: tc.source, ObservedAt: at}}
+ snapshot := models.StateSnapshot{}
+ if kind == "vm" {
+ snapshot.VMs = []models.VM{{ID: "guest", Name: "guest", Instance: "fixture", Node: "node", VMID: 100, Status: "running", LastSeen: time.Now(), Memory: memory}}
+ } else {
+ snapshot.Containers = []models.Container{{ID: "guest", Name: "guest", Instance: "fixture", Node: "node", VMID: 100, Status: "running", LastSeen: time.Now(), Memory: memory}}
+ }
+ registry := unifiedresources.NewRegistry(nil)
+ registry.IngestSnapshot(snapshot)
+ raw, err := marshalAssistantInventoryTopologyContextFromReadState(registry)
+ if err != nil {
+ t.Fatal(err)
+ }
+ var decoded map[string]interface{}
+ if err := json.Unmarshal([]byte(raw), &decoded); err != nil {
+ t.Fatal(err)
+ }
+ family := "vms"
+ if kind != "vm" {
+ family = "containers"
+ }
+ row := decoded["proxmox"].(map[string]interface{})["nodes"].([]interface{})[0].(map[string]interface{})[family].([]interface{})[0].(map[string]interface{})
+ e, ok := row["memory_evidence"].(map[string]interface{})
+ if !ok || e["available"] != !tc.unavailable || e["pressure_known"] != tc.known {
+ t.Errorf("Assistant seed lost qualification: %s", raw)
+ }
+ if !tc.unavailable && row["memory_percent"] != tc.percent || tc.unavailable && row["memory_percent"] != nil {
+ t.Errorf("Assistant seed confused absent/measured memory: %+v", row)
+ }
+ id := ""
+ if kind == "vm" {
+ id = registry.VMs()[0].ID()
+ } else {
+ id = registry.Containers()[0].ID()
+ }
+ exec := tools.NewPulseToolExecutor(tools.ExecutorConfig{ReadState: registry})
+ input := map[string]interface{}{"action": "get", "resource_type": kind, "resource_id": id}
+ result, err := exec.ExecuteTool(context.Background(), agentcapabilities.PulseQueryToolName, input)
+ if err != nil || result.IsError {
+ t.Fatalf("actual resource query failed: %v %+v", err, result)
+ }
+ facts := extractQueryGetFacts(input, result.Content[0].Text)
+ if len(facts) == 0 || !strings.Contains(facts[0].Value, tc.text) {
+ t.Errorf("condensed facts removed memory qualification: %+v", facts)
+ }
+ if !tc.unavailable && (len(facts) == 0 || !strings.Contains(facts[0].Value, at.Format(time.RFC3339))) {
+ t.Errorf("facts renewed original memory time: %+v", facts)
+ }
+ })
+ }
+ }
+}
diff --git a/internal/ai/chat/knowledge_extractor.go b/internal/ai/chat/knowledge_extractor.go
index 644fb3428f..efda07ee1e 100644
--- a/internal/ai/chat/knowledge_extractor.go
+++ b/internal/ai/chat/knowledge_extractor.go
@@ -98,7 +98,8 @@ func extractQueryGetFacts(input map[string]interface{}, resultText string) []Fac
Percent float64 `json:"percent"`
} `json:"cpu"`
Memory struct {
- Percent float64 `json:"percent"`
+ Percent float64 `json:"percent"`
+ Evidence *unifiedresources.GuestMemoryEvidence `json:"evidence"`
} `json:"memory"`
// Error field for not-found responses
Error string `json:"error"`
@@ -156,7 +157,9 @@ func extractQueryGetFacts(input map[string]interface{}, resultText string) []Fac
if resource.CPU.Percent > 0 {
parts = append(parts, fmt.Sprintf("CPU=%.1f%%", resource.CPU.Percent))
}
- if resource.Memory.Percent > 0 {
+ if resource.Memory.Evidence != nil {
+ parts = append(parts, "Mem="+resource.Memory.Evidence.Format(resource.Memory.Percent))
+ } else if resource.Memory.Percent > 0 {
parts = append(parts, fmt.Sprintf("Mem=%.1f%%", resource.Memory.Percent))
}
diff --git a/internal/ai/chat/service.go b/internal/ai/chat/service.go
index b7a4edc7f1..e7f5cbfbce 100644
--- a/internal/ai/chat/service.go
+++ b/internal/ai/chat/service.go
@@ -1416,22 +1416,24 @@ func marshalAssistantInventoryTopologyContext(topology tools.TopologyResponse) (
}
for _, vm := range node.VMs {
nodeContext.VMs = append(nodeContext.VMs, assistantInventoryWorkload{
- AnswerLabel: assistantInventoryGuestAnswerLabel("VM", vm.VMID, vm.Name),
- VMID: vm.VMID,
- Name: vm.Name,
- Status: vm.Status,
- CPUPercent: vm.CPU,
- MemoryPercent: vm.Memory,
+ AnswerLabel: assistantInventoryGuestAnswerLabel("VM", vm.VMID, vm.Name),
+ VMID: vm.VMID,
+ Name: vm.Name,
+ Status: vm.Status,
+ CPUPercent: vm.CPU,
+ MemoryPercent: vm.Memory,
+ MemoryEvidence: vm.MemoryEvidence,
})
}
for _, container := range node.Containers {
nodeContext.Containers = append(nodeContext.Containers, assistantInventoryWorkload{
- AnswerLabel: assistantInventoryGuestAnswerLabel("CT", container.VMID, container.Name),
- VMID: container.VMID,
- Name: container.Name,
- Status: container.Status,
- CPUPercent: container.CPU,
- MemoryPercent: container.Memory,
+ AnswerLabel: assistantInventoryGuestAnswerLabel("CT", container.VMID, container.Name),
+ VMID: container.VMID,
+ Name: container.Name,
+ Status: container.Status,
+ CPUPercent: container.CPU,
+ MemoryPercent: container.Memory,
+ MemoryEvidence: container.MemoryEvidence,
})
}
context.Proxmox.Nodes = append(context.Proxmox.Nodes, nodeContext)
@@ -1545,12 +1547,13 @@ type assistantInventoryProxmoxNode struct {
}
type assistantInventoryWorkload struct {
- AnswerLabel string `json:"answer_label"`
- VMID int `json:"vmid"`
- Name string `json:"name"`
- Status string `json:"status"`
- CPUPercent float64 `json:"cpu_percent,omitempty"`
- MemoryPercent float64 `json:"memory_percent,omitempty"`
+ AnswerLabel string `json:"answer_label"`
+ VMID int `json:"vmid"`
+ Name string `json:"name"`
+ Status string `json:"status"`
+ CPUPercent float64 `json:"cpu_percent,omitempty"`
+ MemoryPercent *float64 `json:"memory_percent"`
+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`
}
type assistantInventoryDockerHost struct {
diff --git a/internal/ai/patrol_guest_memory.go b/internal/ai/patrol_guest_memory.go
index 2c1c18e806..292337d78e 100644
--- a/internal/ai/patrol_guest_memory.go
+++ b/internal/ai/patrol_guest_memory.go
@@ -2,7 +2,6 @@ package ai
import (
"fmt"
- "math"
"strings"
"time"
@@ -36,71 +35,24 @@ func patrolGuestMemoryEvidenceSection(snap patrolRuntimeState, scopedSet map[str
// it visible as evidence, but do not use it (or a retained/undated reading) to
// manufacture a pressure flag, forecast, anomaly or verified recovery.
type patrolGuestMemoryReading struct {
- percent float64
- available bool
- pressureKnown bool
- state string
- source string
- observedAt time.Time
+ percent float64
+ available bool
+ pressureKnown bool
+ state string
+ source string
+ observedAt time.Time
+ mayIncludeCache bool
}
func readPatrolGuestMemory(percent float64, observation models.MemoryObservation, available, proxmoxGuest bool) patrolGuestMemoryReading {
- r := patrolGuestMemoryReading{percent: percent, available: available && !math.IsNaN(percent) && !math.IsInf(percent, 0) && percent >= 0 && percent <= 100}
- if !r.available || observation.State == "unavailable" {
- r.available = false
- r.state = "unavailable"
- return r
- }
- // Other platforms need not implement Proxmox's memory evidence contract.
- // Preserve their existing readings, including a selected measured zero.
- if !proxmoxGuest && observation == (models.MemoryObservation{}) {
- r.pressureKnown = true
- return r
- }
- r.state = "unknown"
- if observation.ObservedAt.After(time.Unix(0, 0)) && !observation.ObservedAt.After(time.Now()) {
- r.observedAt = observation.ObservedAt
- }
- if observation.State == "last-known" {
- r.state = "last-known"
- } else if observation.State == "current" && !r.observedAt.IsZero() {
- r.state = "current"
- }
- // Only known source names are put in the seed. A future/unknown source is
- // not an assertion of either cache awareness or current guest pressure.
- switch observation.Source {
- case "available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent":
- r.source = observation.Source
- r.pressureKnown = r.state == "current"
- case "status-mem", "status-freemem", "cluster-resources", "derived-total-minus-used", "previous-snapshot":
- r.source = observation.Source
- }
- return r
+ e := unifiedresources.QualifyGuestMemory(percent, observation, available, proxmoxGuest, time.Now())
+ return patrolGuestMemoryReading{percent: percent, available: e.Available, pressureKnown: e.PressureKnown,
+ state: e.State, source: e.Source, observedAt: e.ObservedAt, mayIncludeCache: e.MayIncludeCache}
}
func (r patrolGuestMemoryReading) display() string {
- if !r.available {
- return "N/A (guest memory unavailable; not evidence of recovery)"
- }
- if r.state == "" {
- return fmt.Sprintf("%.0f%%", r.percent)
- }
- source := r.source
- if source == "" {
- source = "unknown"
- }
- observed := "time unknown"
- if !r.observedAt.IsZero() {
- observed = "observed " + r.observedAt.UTC().Format(time.RFC3339)
- }
- qualification := "cache-aware guest usage"
- if !r.pressureKnown {
- qualification = "guest pressure unknown"
- if r.source == "status-mem" || r.source == "status-freemem" || r.source == "cluster-resources" || r.source == "derived-total-minus-used" {
- qualification += "; may include reclaimable cache"
- }
- }
- return fmt.Sprintf("%.0f%% (%s; source %s; %s; %s)", r.percent, r.state, source, observed, qualification)
+ return (unifiedresources.GuestMemoryEvidence{Available: r.available, PressureKnown: r.pressureKnown,
+ State: r.state, Source: r.source, ObservedAt: r.observedAt, MayIncludeCache: r.mayIncludeCache}).Format(r.percent)
}
func patrolGuestMemoryGaps(rows []patrolGuestInventoryRow) string {
diff --git a/internal/ai/patrol_observer_guest_memory_test.go b/internal/ai/patrol_observer_guest_memory_test.go
new file mode 100644
index 0000000000..c375535447
--- /dev/null
+++ b/internal/ai/patrol_observer_guest_memory_test.go
@@ -0,0 +1,89 @@
+package ai
+
+import (
+ "fmt"
+ "math"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/models"
+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
+)
+
+func TestPatrolObserverGuestMemoryOrigin(t *testing.T) {
+ now := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
+ for _, kind := range []unifiedresources.ResourceType{unifiedresources.ResourceTypeVM, unifiedresources.ResourceTypeSystemContainer} {
+ for _, tc := range []struct {
+ name, state, source, detail string
+ percent float64
+ at time.Time
+ missing, noProxmox bool
+ }{
+ {"healthy", "current", "guest-agent-meminfo", "", 24, now, false, false},
+ {"zero", "current", "agent", "", 0, now, false, false},
+ {"real-pressure", "current", "available-field", "guest=96.00", 96, now, false, false},
+ {"inclusive-high", "current", "status-mem", "guest_memory_pressure_unknown", 96, now, false, false},
+ {"inclusive-low", "current", "status-freemem", "guest_memory_pressure_unknown", 24, now, false, false},
+ {"retained-low", "last-known", "agent", "guest_memory_pressure_unknown", 24, now.Add(-time.Hour), false, false},
+ {"stale-origin", "current", "agent", "metric_stale", 24, now.Add(-61 * time.Second), false, false},
+ {"age-boundary", "current", "agent", "", 24, now.Add(-59 * time.Second), false, false},
+ {"missing-time", "current", "agent", "guest_memory_pressure_unknown", 24, time.Time{}, false, false},
+ {"future-time", "current", "agent", "guest_memory_pressure_unknown", 24, now.Add(time.Hour), false, false},
+ {"unknown-source", "current", "future-source", "guest_memory_pressure_unknown", 24, now, false, false},
+ {"missing", "", "", "guest_memory_pressure_unknown", 0, time.Time{}, true, false},
+ {"invalid", "current", "agent", "guest_memory_pressure_unknown", math.NaN(), now, false, false},
+ {"other-platform", "", "", "", 24, time.Time{}, false, true},
+ } {
+ t.Run(string(kind)+"/"+tc.name, func(t *testing.T) {
+ store := NewInMemoryPatrolObjectiveStore()
+ objective, err := store.Create(CreatePatrolObjectiveInput{Brief: "Keep guest memory below 85 percent", ResourceIDs: []string{"guest"}}, now)
+ if err != nil {
+ t.Fatal(err)
+ }
+ _, err = store.ProposeObserver(objective.ID, ProposePatrolObserverInput{
+ ExpectedRevision: objective.Revision, EvidenceFit: PatrolObserverEvidenceFitDirect,
+ Interpretation: "Guest pressure stays below 85 percent.", TriggerKinds: []PatrolObserverTriggerKind{PatrolObserverTriggerInterval},
+ ProbeJSON: `{"runtime":"pulse-resource-metric/v1","metric":"memory_percent","operator":"less_than","threshold":85,"sample_interval_seconds":10,"wake_after_consecutive_failures":1,"max_evidence_age_seconds":60}`,
+ WakeEvidence: "Memory objective breached or its pressure evidence is unavailable.", RequirementsJSON: `{}`,
+ }, now)
+ if err != nil {
+ t.Fatal(err)
+ }
+ r := unifiedresources.Resource{ID: "guest", Type: kind, LastSeen: now.Add(time.Second), UpdatedAt: now.Add(time.Second),
+ Proxmox: &unifiedresources.ProxmoxData{Memory: &models.Memory{Usage: 24, Observation: models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: now}}},
+ Metrics: &unifiedresources.ResourceMetrics{Memory: &unifiedresources.MetricValue{Percent: tc.percent, Observation: models.MemoryObservation{State: tc.state, Source: tc.source, ObservedAt: tc.at}}},
+ }
+ if tc.missing {
+ r.Metrics.Memory = nil
+ }
+ if tc.noProxmox {
+ r.Proxmox = nil
+ }
+ patrol := NewPatrolService(nil, nil)
+ patrol.SetObjectiveStore(store)
+ patrol.SetUnifiedResourceProvider(&mockUnifiedResourceProvider{getAllFunc: func() []unifiedresources.Resource { return []unifiedresources.Resource{r} }})
+ tm := NewTriggerManager(TriggerManagerConfig{MaxPendingTriggers: 10})
+ patrol.SetTriggerManager(tm)
+ patrol.processObjectiveObservers(now.Add(time.Second))
+ got, _ := store.Get(objective.ID, now.Add(time.Second))
+ if got.Observer == nil || got.Observer.State != PatrolObserverInstalled {
+ t.Fatalf("existing observer failed to install: %+v", got.Observer)
+ }
+ if tc.detail == "" {
+ if tm.GetPendingCount() != 0 || got.Coverage.State != PatrolObjectiveCovered {
+ t.Fatalf("valid memory control lost coverage: %+v pending=%d", got.Coverage, tm.GetPendingCount())
+ }
+ return
+ }
+ if tm.GetPendingCount() != 1 || tm.pendingTriggers[0].ObjectiveContext == nil {
+ t.Fatalf("unqualified/breached evidence accepted as healthy: %+v pending=%d", got.Coverage, tm.GetPendingCount())
+ }
+ context := tm.pendingTriggers[0].ObjectiveContext
+ if !strings.Contains(context.Evidence, tc.detail) || fmt.Sprint(context.ObservedResourceIDs) != "[guest]" {
+ t.Fatalf("wake lost unknown/breach/identity distinction: %+v", context)
+ }
+ })
+ }
+ }
+}
diff --git a/internal/ai/patrol_observer_runtime.go b/internal/ai/patrol_observer_runtime.go
index fe97dc7658..bf20f91066 100644
--- a/internal/ai/patrol_observer_runtime.go
+++ b/internal/ai/patrol_observer_runtime.go
@@ -544,6 +544,19 @@ func (p *PatrolService) evaluateObjectiveObserver(runtime *patrolObserverRuntime
if observedAt.IsZero() {
observedAt = resource.UpdatedAt
}
+ if probe.metric == "memory_percent" && (resource.Type == unifiedresources.ResourceTypeVM || resource.Type == unifiedresources.ResourceTypeSystemContainer) {
+ // The row can be refreshed by a different source. Only the selected
+ // guest memory sample owns its freshness and pressure semantics.
+ memory := unifiedresources.GuestMemoryEvidenceForResource(&resource, now)
+ if !memory.PressureKnown {
+ failing = append(failing, resourceID)
+ evidenceDetails = append(evidenceDetails, resourceID+"=guest_memory_pressure_unknown")
+ continue
+ }
+ if memory.State != "" {
+ observedAt = memory.ObservedAt
+ }
+ }
if observedAt.IsZero() || now.Sub(observedAt) > maxAge {
failing = append(failing, resourceID)
evidenceDetails = append(evidenceDetails, resourceID+"=metric_stale")
diff --git a/internal/ai/tools/data_types.go b/internal/ai/tools/data_types.go
index 89404a52b3..368e8354b9 100644
--- a/internal/ai/tools/data_types.go
+++ b/internal/ai/tools/data_types.go
@@ -278,23 +278,25 @@ type NodeSummary struct {
// VMSummary is a summarized VM for list responses
type VMSummary struct {
GovernedResourceMetadata
- VMID int `json:"vmid"`
- Name string `json:"name"`
- Status string `json:"status"`
- Node string `json:"node"`
- CPU float64 `json:"cpu_percent,omitempty"`
- Memory float64 `json:"memory_percent,omitempty"`
+ VMID int `json:"vmid"`
+ Name string `json:"name"`
+ Status string `json:"status"`
+ Node string `json:"node"`
+ CPU float64 `json:"cpu_percent,omitempty"`
+ Memory *float64 `json:"memory_percent"`
+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`
}
// ContainerSummary is a summarized LXC container for list responses
type ContainerSummary struct {
GovernedResourceMetadata
- VMID int `json:"vmid"`
- Name string `json:"name"`
- Status string `json:"status"`
- Node string `json:"node"`
- CPU float64 `json:"cpu_percent,omitempty"`
- Memory float64 `json:"memory_percent,omitempty"`
+ VMID int `json:"vmid"`
+ Name string `json:"name"`
+ Status string `json:"status"`
+ Node string `json:"node"`
+ CPU float64 `json:"cpu_percent,omitempty"`
+ Memory *float64 `json:"memory_percent"`
+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`
}
// DockerHostSummary is a summarized Docker host for list responses
@@ -485,13 +487,14 @@ func (t ProxmoxNodeTopology) NormalizeCollections() ProxmoxNodeTopology {
// TopologyVM represents a VM in the topology
type TopologyVM struct {
GovernedResourceMetadata
- VMID int `json:"vmid"`
- Name string `json:"name"`
- Status string `json:"status"`
- CPU float64 `json:"cpu_percent,omitempty"`
- Memory float64 `json:"memory_percent,omitempty"`
- OS string `json:"os,omitempty"`
- Tags []string `json:"tags"`
+ VMID int `json:"vmid"`
+ Name string `json:"name"`
+ Status string `json:"status"`
+ CPU float64 `json:"cpu_percent,omitempty"`
+ Memory *float64 `json:"memory_percent"`
+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`
+ OS string `json:"os,omitempty"`
+ Tags []string `json:"tags"`
}
func (t TopologyVM) NormalizeCollections() TopologyVM {
@@ -504,14 +507,15 @@ func (t TopologyVM) NormalizeCollections() TopologyVM {
// TopologyContainer represents a system container in the topology
type TopologyContainer struct {
GovernedResourceMetadata
- VMID int `json:"vmid"`
- Name string `json:"name"`
- Status string `json:"status"`
- CPU float64 `json:"cpu_percent,omitempty"`
- Memory float64 `json:"memory_percent,omitempty"`
- OS string `json:"os,omitempty"`
- Tags []string `json:"tags"`
- HasDocker bool `json:"has_docker,omitempty"` // True if Docker is installed inside this container
+ VMID int `json:"vmid"`
+ Name string `json:"name"`
+ Status string `json:"status"`
+ CPU float64 `json:"cpu_percent,omitempty"`
+ Memory *float64 `json:"memory_percent"`
+ MemoryEvidence *unifiedresources.GuestMemoryEvidence `json:"memory_evidence,omitempty"`
+ OS string `json:"os,omitempty"`
+ Tags []string `json:"tags"`
+ HasDocker bool `json:"has_docker,omitempty"` // True if Docker is installed inside this container
}
func (t TopologyContainer) NormalizeCollections() TopologyContainer {
@@ -689,7 +693,7 @@ type ResourceResponse struct {
}
func EmptyResourceResponse() ResourceResponse {
- return ResourceResponse{}.NormalizeCollections()
+ return ResourceResponse{Memory: resourceMemoryValues(0, 0, 0)}.NormalizeCollections()
}
func (r ResourceResponse) NormalizeCollections() ResourceResponse {
@@ -850,9 +854,15 @@ type ResourceCPU struct {
// ResourceMemory describes memory usage
type ResourceMemory struct {
- Percent float64 `json:"percent"`
- UsedGB float64 `json:"used_gb"`
- TotalGB float64 `json:"total_gb"`
+ Percent *float64 `json:"percent"`
+ UsedGB *float64 `json:"used_gb"`
+ TotalGB float64 `json:"total_gb"`
+ Evidence *unifiedresources.GuestMemoryEvidence `json:"evidence,omitempty"`
+}
+
+// resourceMemoryValues preserves the numeric contract for non-guest responses.
+func resourceMemoryValues(percent, usedGB, totalGB float64) ResourceMemory {
+ return ResourceMemory{Percent: &percent, UsedGB: &usedGB, TotalGB: totalGB}
}
// ResourceDisk describes disk usage
@@ -960,13 +970,14 @@ type FindingCounts struct {
// MetricsResponse is returned by pulse_get_metrics
type MetricsResponse struct {
- ResourceID string `json:"resource_id,omitempty"`
- Period string `json:"period"`
- Points []MetricPoint `json:"points"`
- Summary map[string]ResourceMetricsSummary `json:"summary"`
- Pagination *PaginationInfo `json:"pagination,omitempty"`
- Downsampled bool `json:"downsampled,omitempty"`
- OriginalCount int `json:"original_count,omitempty"`
+ ResourceID string `json:"resource_id,omitempty"`
+ Period string `json:"period"`
+ Points []MetricPoint `json:"points"`
+ Summary map[string]ResourceMetricsSummary `json:"summary"`
+ Pagination *PaginationInfo `json:"pagination,omitempty"`
+ Downsampled bool `json:"downsampled,omitempty"`
+ OriginalCount int `json:"original_count,omitempty"`
+ MemoryInterpretation string `json:"memory_interpretation,omitempty"`
}
func EmptyMetricsResponse() MetricsResponse {
diff --git a/internal/ai/tools/guest_memory_evidence_test.go b/internal/ai/tools/guest_memory_evidence_test.go
new file mode 100644
index 0000000000..479a742fe1
--- /dev/null
+++ b/internal/ai/tools/guest_memory_evidence_test.go
@@ -0,0 +1,146 @@
+package tools
+
+import (
+ "context"
+ "encoding/json"
+ "math"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/agentcapabilities"
+ "github.com/rcourtman/pulse-go-rewrite/internal/models"
+ "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
+)
+
+// These actual tool controls use APIs present in the parent. The parent gets
+// the tests, not the repair, for a discriminating comparison.
+func TestGuestMemoryQueryEvidence(t *testing.T) {
+ at := time.Now().UTC().Add(-time.Minute).Truncate(time.Second)
+ for _, kind := range []string{"vm", "system-container"} {
+ for _, tc := range []struct {
+ name, state, source string
+ percent float64
+ at time.Time
+ missing, known bool
+ }{
+ {"measured-zero", "current", "agent", 0, at, false, true},
+ {"healthy-available", "current", "guest-agent-meminfo", 24, at, false, true},
+ {"real-pressure", "current", "available-field", 96, at, false, true},
+ {"status-cache", "current", "status-mem", 96, at, false, false},
+ {"retained-low", "last-known", "agent", 24, at.Add(-time.Hour), false, false},
+ {"retained-high", "last-known", "guest-agent-meminfo", 96, at.Add(-time.Hour), false, false},
+ {"undated", "current", "available-field", 24, time.Time{}, false, false},
+ {"future", "current", "agent", 24, time.Now().Add(time.Hour), false, false},
+ {"unknown-source", "current", "unrecognised-source", 24, at, false, false},
+ {"unavailable", "unavailable", "unavailable", 0, time.Time{}, false, false},
+ {"missing", "", "", 0, time.Time{}, true, false},
+ {"invalid", "current", "agent", math.NaN(), at, false, false},
+ } {
+ for _, path := range []string{"canonical", "typed-view"} {
+ t.Run(kind+"/"+tc.name+"/"+path, func(t *testing.T) {
+ total, used := int64(20<<30), int64(5<<30)
+ r := unifiedresources.Resource{ID: "selected-guest", Name: "selected-guest", Type: unifiedresources.ResourceType(kind), Status: unifiedresources.StatusOnline, LastSeen: time.Now(),
+ Proxmox: &unifiedresources.ProxmoxData{VMID: 100, NodeName: "node", Instance: "fixture", SourceID: "source-guest", RuntimeStatus: "running",
+ Memory: &models.Memory{Usage: 99, Observation: models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: time.Now()}}},
+ Metrics: &unifiedresources.ResourceMetrics{Memory: &unifiedresources.MetricValue{Percent: tc.percent, Used: &used, Total: &total, Observation: models.MemoryObservation{State: tc.state, Source: tc.source, ObservedAt: tc.at}}},
+ }
+ if tc.missing {
+ r.Metrics.Memory = nil
+ }
+ vm, ct := unifiedresources.NewVMView(&r), unifiedresources.NewContainerView(&r)
+ rs := &fakeReadState{}
+ if kind == "vm" {
+ rs.vms = []*unifiedresources.VMView{&vm}
+ } else {
+ rs.containers = []*unifiedresources.ContainerView{&ct}
+ }
+ cfg := ExecutorConfig{ReadState: rs}
+ if path == "canonical" {
+ cfg.UnifiedResourceProvider = &stubUnifiedResourceProvider{resources: []unifiedresources.Resource{r}}
+ }
+ exec := NewPulseToolExecutor(cfg)
+ for _, action := range []string{"get", "list", "topology"} {
+ result, err := exec.ExecuteTool(context.Background(), agentcapabilities.PulseQueryToolName, map[string]interface{}{"action": action, "resource_type": kind, "resource_id": r.ID})
+ if err != nil || result.IsError || len(result.Content) == 0 {
+ t.Fatalf("%s read failed: err=%v result=%+v", action, err, result)
+ }
+ var decoded map[string]interface{}
+ if err := json.Unmarshal([]byte(result.Content[0].Text), &decoded); err != nil {
+ t.Fatal(err)
+ }
+ var evidence interface{}
+ var percent interface{}
+ if action == "get" {
+ memory := decoded["memory"].(map[string]interface{})
+ evidence, percent = memory["evidence"], memory["percent"]
+ if (tc.missing || tc.state == "unavailable" || math.IsNaN(tc.percent)) && memory["used_gb"] != nil {
+ t.Error("missing usage became zero used bytes")
+ }
+ } else {
+ rows := decoded
+ if action == "topology" {
+ rows = decoded["proxmox"].(map[string]interface{})["nodes"].([]interface{})[0].(map[string]interface{})
+ }
+ family := "vms"
+ if kind != "vm" {
+ family = "containers"
+ }
+ row := rows[family].([]interface{})[0].(map[string]interface{})
+ evidence, percent = row["memory_evidence"], row["memory_percent"]
+ }
+ e, ok := evidence.(map[string]interface{})
+ if !ok {
+ t.Errorf("%s lost guest memory qualification: %s", action, result.Content[0].Text)
+ continue
+ }
+ available := !tc.missing && tc.state != "unavailable" && !math.IsNaN(tc.percent)
+ if e["available"] != available || e["pressure_known"] != tc.known {
+ t.Errorf("%s qualification=%+v want available=%t known=%t", action, e, available, tc.known)
+ }
+ if available && percent != tc.percent || !available && percent != nil {
+ t.Errorf("%s percent=%v want available=%t percent=%g", action, percent, available, tc.percent)
+ }
+ if !tc.at.IsZero() && !tc.at.After(time.Now()) && available && e["observed_at"] != tc.at.Format(time.RFC3339) {
+ t.Errorf("%s renewed/lost original time: %+v", action, e)
+ }
+ if tc.source == "status-mem" && e["may_include_reclaimable_cache"] != true {
+ t.Errorf("%s erased cache distinction", action)
+ }
+ }
+ })
+ }
+ }
+ }
+}
+
+func TestGuestMemoryHistoryDoesNotBorrowLiveOrigin(t *testing.T) {
+ for _, family := range []string{"vm", "system-container", "agent"} {
+ t.Run(family, func(t *testing.T) {
+ history := &mockMetricsHistoryProvider{}
+ history.On("GetResourceMetrics", "source-guest", 24*time.Hour).Return([]MetricPoint{{Memory: 96}}, nil)
+ history.On("GetAllMetricsSummary", 24*time.Hour).Return(map[string]ResourceMetricsSummary{"selected-guest": {ResourceType: family, AvgMemory: 96, MaxMemory: 99}}, nil)
+ r := unifiedresources.Resource{ID: "selected-guest", Type: unifiedresources.ResourceType(family), MetricsTarget: &unifiedresources.MetricsTarget{ResourceType: family, ResourceID: "source-guest"},
+ Metrics: &unifiedresources.ResourceMetrics{Memory: &unifiedresources.MetricValue{Percent: 24, Observation: models.MemoryObservation{State: "current", Source: "agent", ObservedAt: time.Now().Add(-time.Minute)}}}}
+ exec := NewPulseToolExecutor(ExecutorConfig{StateProvider: &mockStateProvider{}, MetricsHistory: history, UnifiedResourceProvider: &stubUnifiedResourceProvider{resources: []unifiedresources.Resource{r}}})
+ for _, id := range []string{r.ID, ""} {
+ result, err := exec.ExecuteTool(context.Background(), agentcapabilities.PulseMetricsToolName, map[string]interface{}{"type": "performance", "resource_id": id})
+ if err != nil || result.IsError {
+ t.Fatalf("history read failed: %v %+v", err, result)
+ }
+ var decoded map[string]interface{}
+ if err := json.Unmarshal([]byte(result.Content[0].Text), &decoded); err != nil {
+ t.Fatal(err)
+ }
+ note, _ := decoded["memory_interpretation"].(string)
+ if family != "agent" && (!strings.Contains(note, "per-sample") || !strings.Contains(note, "cannot reattribute")) {
+ t.Errorf("history borrowed live memory trust: %s", result.Content[0].Text)
+ }
+ if family == "agent" && note != "" {
+ t.Error("non-guest history changed")
+ }
+ }
+ history.AssertExpectations(t)
+ })
+ }
+}
diff --git a/internal/ai/tools/tools_metrics.go b/internal/ai/tools/tools_metrics.go
index 8f987b69ca..ba8c082534 100644
--- a/internal/ai/tools/tools_metrics.go
+++ b/internal/ai/tools/tools_metrics.go
@@ -28,6 +28,10 @@ Types:
- baselines: Learned normal behavior baselines for resources
- patterns: Detected operational patterns and predictions
+Performance responses expose numeric guest-memory samples without per-sample memory-source
+provenance. Do not infer a guest leak, pressure or current recovery from a high
+historical percentage alone, or lend past samples today's selected source.
+
Examples:
- Get 24h metrics: type="performance", period="24h"
- Get VM metrics: type="performance", resource_id="101"
@@ -162,6 +166,9 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[string
response := EmptyMetricsResponse()
response.Period = period
+ if resourceType == "vm" || resourceType == "system-container" {
+ response.MemoryInterpretation = guestMemoryHistoryInterpretation
+ }
if resourceID != "" {
response.ResourceID = resourceID
@@ -173,6 +180,9 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[string
return NewErrorResult(err), nil
}
response.ResourceID = resource.ID
+ if resource.Type == unifiedresources.ResourceTypeVM || resource.Type == unifiedresources.ResourceTypeSystemContainer {
+ response.MemoryInterpretation = guestMemoryHistoryInterpretation
+ }
target := e.resourceMetricsTarget(resource)
if target == nil || strings.TrimSpace(target.ResourceID) == "" {
return NewTextResult(fmt.Sprintf("No metrics target is available for resource %s.", resource.ID)), nil
@@ -232,6 +242,9 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[string
}
metric := summary[id]
metric.ResourceType = canonicalMetricsResourceType(metric.ResourceType)
+ if metric.ResourceType == "vm" || metric.ResourceType == "system-container" {
+ response.MemoryInterpretation = guestMemoryHistoryInterpretation
+ }
filtered[id] = metric
total++
}
@@ -252,6 +265,8 @@ func (e *PulseToolExecutor) executeGetMetrics(_ context.Context, args map[string
return NewJSONResult(response.NormalizeCollections()), nil
}
+const guestMemoryHistoryInterpretation = "This guest memory history response has no per-sample source or pressure qualification. Percentages may include reclaimable cache and do not independently prove guest pressure, a leak or current recovery. Live observations cannot reattribute past samples."
+
func (e *PulseToolExecutor) executeGetBaselines(_ context.Context, args map[string]interface{}) (CallToolResult, error) {
resourceID, _ := args["resource_id"].(string)
resourceType, _ := args["resource_type"].(string)
diff --git a/internal/ai/tools/tools_query.go b/internal/ai/tools/tools_query.go
index dcc46b4f27..b0cf532ecb 100644
--- a/internal/ai/tools/tools_query.go
+++ b/internal/ai/tools/tools_query.go
@@ -2152,7 +2152,7 @@ func (e *PulseToolExecutor) registerQueryTools() {
e.registry.registerBuiltin(RegisteredTool{
Definition: Tool{
Name: agentcapabilities.PulseQueryToolName,
- Description: `Query and search canonical infrastructure resources. Start here to discover systems, workloads, storage, and disks by name. Actions: search, get, config, topology, list, health, action. Use action with action_id to read the canonical persisted plan, decision state and independently verified execution outcome. Inventory and incident history can lag execution and cannot establish whether an action was approved or run. For app-container get, filesystems reports observed capacity at each mountpoint, not container quotas. An observation error has no usage payload. Mounts describe configuration. Health returns the connection overview by default, or the canonical resource projection when resource_id is provided. command_agent_connected describes live command transport, independently of monitoring collection or freshness. Missing connection fields were not observed. can_execute describes connected transport with control enabled, not approval for a particular operation.`,
+ Description: `Query and search canonical infrastructure resources. Start here to discover systems, workloads, storage, and disks by name. Actions: search, get, config, topology, list, health, action. Use action with action_id to read the canonical persisted plan, decision state and independently verified execution outcome. Inventory and incident history can lag execution and cannot establish whether an action was approved or run. For app-container get, filesystems reports observed capacity at each mountpoint, not container quotas. An observation error has no usage payload. Mounts describe configuration. Health returns the connection overview by default, or the canonical resource projection when resource_id is provided. command_agent_connected describes live command transport, independently of monitoring collection or freshness. Missing connection fields were not observed. can_execute describes connected transport with control enabled, not approval for a particular operation. Guest memory evidence describes the selected reading: null is unavailable, zero can be measured, and cache-inclusive or retained percentages do not prove guest pressure or recovery. Use its original observed_at, not the resource timestamp.`,
InputSchema: InputSchema{
Type: "object",
Properties: map[string]PropertySchema{
@@ -2959,6 +2959,7 @@ type queryGuestView interface {
CPUPercent() float64
CPUs() int
MemoryPercent() float64
+ MemoryEvidence(time.Time) unifiedresources.GuestMemoryEvidence
MemoryUsed() int64
MemoryTotal() int64
Tags() []string
@@ -3080,8 +3081,6 @@ func guestViewGetResult[V queryGuestView](e *PulseToolExecutor, guests []V, kind
if fmt.Sprintf("%d", g.VMID()) != resourceID && g.Name() != resourceID && g.ID() != resourceID {
continue
}
- used := g.MemoryUsed()
- total := g.MemoryTotal()
response := EmptyResourceResponse()
response.GovernedResourceMetadata = governance.Resolve(g.Name(), g.ID(), fmt.Sprintf("%d", g.VMID()))
response.Type = kind
@@ -3093,11 +3092,7 @@ func guestViewGetResult[V queryGuestView](e *PulseToolExecutor, guests []V, kind
Percent: g.CPUPercent(),
Cores: g.CPUs(),
}
- response.Memory = ResourceMemory{
- Percent: g.MemoryPercent(),
- UsedGB: float64(used) / (1024 * 1024 * 1024),
- TotalGB: float64(total) / (1024 * 1024 * 1024),
- }
+ response.Memory = guestViewMemory(g, time.Now())
response.Tags = g.Tags()
if !g.LastBackup().IsZero() {
t := g.LastBackup()
@@ -3128,6 +3123,38 @@ func guestViewGetResult[V queryGuestView](e *PulseToolExecutor, guests []V, kind
return CallToolResult{}, false
}
+func guestMemoryFields(e unifiedresources.GuestMemoryEvidence, percent float64) (*float64, *unifiedresources.GuestMemoryEvidence) {
+ var value *float64
+ if e.Available {
+ value = &percent
+ }
+ if e.State == "" { // unchanged other-platform reading
+ return value, nil
+ }
+ return value, &e
+}
+
+func guestViewMemory(g queryGuestView, now time.Time) ResourceMemory {
+ percent, evidence := guestMemoryFields(g.MemoryEvidence(now), g.MemoryPercent())
+ memory := ResourceMemory{Percent: percent, TotalGB: float64(g.MemoryTotal()) / (1024 * 1024 * 1024), Evidence: evidence}
+ if percent != nil && g.MemoryTotal() > 0 {
+ usedGB := float64(g.MemoryUsed()) / (1024 * 1024 * 1024)
+ memory.UsedGB = &usedGB
+ }
+ return memory
+}
+
+func canonicalGuestMemory(resource unifiedresources.Resource, now time.Time) ResourceMemory {
+ m := resourceMetric(resource, "memory")
+ percent, evidence := guestMemoryFields(unifiedresources.GuestMemoryEvidenceForResource(&resource, now), metricPercent(m))
+ memory := ResourceMemory{Percent: percent, TotalGB: metricTotalGB(m), Evidence: evidence}
+ if percent != nil && m != nil && m.Used != nil {
+ usedGB := metricUsedGB(m)
+ memory.UsedGB = &usedGB
+ }
+ return memory
+}
+
func canonicalGuestTarget(resource unifiedresources.Resource) string {
if resource.Proxmox != nil {
return strings.TrimSpace(resource.Proxmox.NodeName)
@@ -3275,11 +3302,7 @@ func canonicalGuestResponse(kind string, resource unifiedresources.Resource, gov
response.CPU = ResourceCPU{
Percent: metricPercent(resourceMetric(resource, "cpu")),
}
- response.Memory = ResourceMemory{
- Percent: metricPercent(resourceMetric(resource, "memory")),
- UsedGB: metricUsedGB(resourceMetric(resource, "memory")),
- TotalGB: metricTotalGB(resourceMetric(resource, "memory")),
- }
+ response.Memory = canonicalGuestMemory(resource, time.Now())
if diskMetric := resourceMetric(resource, "disk"); diskMetric != nil {
response.Disk = &ResourceDisk{
Percent: metricPercent(diskMetric),
@@ -3331,11 +3354,7 @@ func canonicalAgentResponse(resource unifiedresources.Resource, governance *gove
response.CPU = ResourceCPU{
Percent: metricPercent(resourceMetric(resource, "cpu")),
}
- response.Memory = ResourceMemory{
- Percent: metricPercent(resourceMetric(resource, "memory")),
- UsedGB: metricUsedGB(resourceMetric(resource, "memory")),
- TotalGB: metricTotalGB(resourceMetric(resource, "memory")),
- }
+ response.Memory = resourceMemoryValues(metricPercent(resourceMetric(resource, "memory")), metricUsedGB(resourceMetric(resource, "memory")), metricTotalGB(resourceMetric(resource, "memory")))
if diskMetric := resourceMetric(resource, "disk"); diskMetric != nil {
response.Disk = &ResourceDisk{
Percent: metricPercent(diskMetric),
@@ -3861,6 +3880,7 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args ma
count++
continue
}
+ memoryPercent, memoryEvidence := guestMemoryFields(vm.MemoryEvidence(time.Now()), vm.MemoryPercent())
response.VMs = append(response.VMs, VMSummary{
GovernedResourceMetadata: governance.Resolve(vm.Name(), vm.ID(), fmt.Sprintf("%d", vm.VMID())),
VMID: vm.VMID(),
@@ -3868,7 +3888,8 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args ma
Status: string(vm.Status()),
Node: vm.Node(),
CPU: vm.CPUPercent(),
- Memory: vm.MemoryPercent(),
+ Memory: memoryPercent,
+ MemoryEvidence: memoryEvidence,
})
count++
}
@@ -3892,6 +3913,7 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args ma
count++
continue
}
+ memoryPercent, memoryEvidence := guestMemoryFields(ct.MemoryEvidence(time.Now()), ct.MemoryPercent())
response.Containers = append(response.Containers, ContainerSummary{
GovernedResourceMetadata: governance.Resolve(ct.Name(), ct.ID(), fmt.Sprintf("%d", ct.VMID())),
VMID: ct.VMID(),
@@ -3899,7 +3921,8 @@ func (e *PulseToolExecutor) executeListInfrastructure(_ context.Context, args ma
Status: string(ct.Status()),
Node: ct.Node(),
CPU: ct.CPUPercent(),
- Memory: ct.MemoryPercent(),
+ Memory: memoryPercent,
+ MemoryEvidence: memoryEvidence,
})
count++
}
@@ -4400,13 +4423,15 @@ func BuildTopologyResponseFromReadState(rs unifiedresources.ReadState, options T
}
nodeTopology.VMCount++
if options.MaxVMsPerNode <= 0 || len(nodeTopology.VMs) < options.MaxVMsPerNode {
+ memoryPercent, memoryEvidence := guestMemoryFields(vm.MemoryEvidence(time.Now()), vm.MemoryPercent())
nodeTopology.VMs = append(nodeTopology.VMs, TopologyVM{
GovernedResourceMetadata: governance.Resolve(vm.Name(), vm.ID(), fmt.Sprintf("%d", vm.VMID())),
VMID: vm.VMID(),
Name: vm.Name(),
Status: status,
CPU: vm.CPUPercent(),
- Memory: vm.MemoryPercent(),
+ Memory: memoryPercent,
+ MemoryEvidence: memoryEvidence,
Tags: vm.Tags(),
})
}
@@ -4426,13 +4451,15 @@ func BuildTopologyResponseFromReadState(rs unifiedresources.ReadState, options T
}
nodeTopology.ContainerCount++
if options.MaxContainersPerNode <= 0 || len(nodeTopology.Containers) < options.MaxContainersPerNode {
+ memoryPercent, memoryEvidence := guestMemoryFields(ct.MemoryEvidence(time.Now()), ct.MemoryPercent())
nodeTopology.Containers = append(nodeTopology.Containers, TopologyContainer{
GovernedResourceMetadata: governance.Resolve(ct.Name(), ct.ID(), fmt.Sprintf("%d", ct.VMID())),
VMID: ct.VMID(),
Name: ct.Name(),
Status: status,
CPU: ct.CPUPercent(),
- Memory: ct.MemoryPercent(),
+ Memory: memoryPercent,
+ MemoryEvidence: memoryEvidence,
Tags: ct.Tags(),
})
}
@@ -4947,11 +4974,7 @@ func (e *PulseToolExecutor) executeGetResource(_ context.Context, args map[strin
response.Host = hostName
response.OS = strings.TrimSpace(host.OS())
response.CPU = ResourceCPU{Percent: host.CPUPercent(), Cores: host.CPUs()}
- response.Memory = ResourceMemory{
- Percent: host.MemoryPercent(),
- UsedGB: float64(host.MemoryUsed()) / (1024 * 1024 * 1024),
- TotalGB: float64(host.MemoryTotal()) / (1024 * 1024 * 1024),
- }
+ response.Memory = resourceMemoryValues(host.MemoryPercent(), float64(host.MemoryUsed())/(1024*1024*1024), float64(host.MemoryTotal())/(1024*1024*1024))
response.Tags = host.Tags()
e.registerResolvedResourceWithExplicitAccess(ResourceRegistration{
Kind: "docker-host",
@@ -5022,11 +5045,7 @@ func (e *PulseToolExecutor) executeGetResource(_ context.Context, args map[strin
response.CPU = ResourceCPU{
Percent: metricPercent(resourceMetric(resource, "cpu")),
}
- response.Memory = ResourceMemory{
- Percent: metricPercent(resourceMetric(resource, "memory")),
- UsedGB: metricUsedGB(resourceMetric(resource, "memory")),
- TotalGB: metricTotalGB(resourceMetric(resource, "memory")),
- }
+ response.Memory = resourceMemoryValues(metricPercent(resourceMetric(resource, "memory")), metricUsedGB(resourceMetric(resource, "memory")), metricTotalGB(resourceMetric(resource, "memory")))
if diskMetric := resourceMetric(resource, "disk"); diskMetric != nil {
response.Disk = &ResourceDisk{
Percent: metricPercent(diskMetric),
@@ -5141,11 +5160,7 @@ func (e *PulseToolExecutor) executeGetResource(_ context.Context, args map[strin
response.CPU = ResourceCPU{
Percent: container.CPUPercent(),
}
- response.Memory = ResourceMemory{
- Percent: container.MemoryPercent(),
- UsedGB: float64(container.MemoryUsed()) / (1024 * 1024 * 1024),
- TotalGB: float64(container.MemoryTotal()) / (1024 * 1024 * 1024),
- }
+ response.Memory = resourceMemoryValues(container.MemoryPercent(), float64(container.MemoryUsed())/(1024*1024*1024), float64(container.MemoryTotal())/(1024*1024*1024))
response.RestartCount = container.RestartCount()
response.Labels = container.Labels()
diff --git a/internal/unifiedresources/guest_memory.go b/internal/unifiedresources/guest_memory.go
new file mode 100644
index 0000000000..38ea31bc26
--- /dev/null
+++ b/internal/unifiedresources/guest_memory.go
@@ -0,0 +1,96 @@
+package unifiedresources
+
+import (
+ "fmt"
+ "math"
+ "time"
+
+ "github.com/rcourtman/pulse-go-rewrite/internal/models"
+)
+
+// GuestMemoryEvidence qualifies the selected numeric reading. It is not
+// permission to obtain another reading or evidence that a guest has recovered.
+// Available distinguishes a measured zero from an absent/invalid percentage.
+type GuestMemoryEvidence struct {
+ Available bool `json:"available"`
+ PressureKnown bool `json:"pressure_known"`
+ State string `json:"state,omitempty"`
+ Source string `json:"source,omitempty"`
+ ObservedAt time.Time `json:"observed_at,omitzero"`
+ MayIncludeCache bool `json:"may_include_reclaimable_cache,omitempty"`
+}
+
+// QualifyGuestMemory shares the existing Patrol interpretation with its query
+// and observer inputs. State is authored by collection; consumers do not renew
+// the original observation or invent a separate collection lease.
+func QualifyGuestMemory(percent float64, observation models.MemoryObservation, available, proxmoxGuest bool, now time.Time) GuestMemoryEvidence {
+ e := GuestMemoryEvidence{Available: available && !math.IsNaN(percent) && !math.IsInf(percent, 0) && percent >= 0 && percent <= 100}
+ if !e.Available || observation.State == "unavailable" {
+ e.Available = false
+ e.State = "unavailable"
+ return e
+ }
+ // Other platforms need not implement Proxmox's memory observation contract.
+ if !proxmoxGuest && observation == (models.MemoryObservation{}) {
+ e.PressureKnown = true
+ return e
+ }
+ e.State = "unknown"
+ if observation.ObservedAt.After(time.Unix(0, 0)) && !observation.ObservedAt.After(now) {
+ e.ObservedAt = observation.ObservedAt
+ }
+ if observation.State == "last-known" {
+ e.State = "last-known"
+ } else if observation.State == "current" && !e.ObservedAt.IsZero() {
+ e.State = "current"
+ }
+ // Source names are closed facts, not arbitrary upstream text in AI context.
+ switch observation.Source {
+ case "available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent":
+ e.Source = observation.Source
+ e.PressureKnown = e.State == "current"
+ case "status-mem", "status-freemem", "cluster-resources", "derived-total-minus-used":
+ e.Source = observation.Source
+ e.MayIncludeCache = true
+ case "previous-snapshot":
+ e.Source = observation.Source
+ }
+ return e
+}
+
+// Format keeps qualification when a tool result is condensed into model facts.
+func (e GuestMemoryEvidence) Format(percent float64) string {
+ if !e.Available {
+ return "N/A (guest memory unavailable; not evidence of recovery)"
+ }
+ if e.State == "" {
+ return fmt.Sprintf("%.0f%%", percent)
+ }
+ source := e.Source
+ if source == "" {
+ source = "unknown"
+ }
+ observed := "time unknown"
+ if !e.ObservedAt.IsZero() {
+ observed = "observed " + e.ObservedAt.UTC().Format(time.RFC3339)
+ }
+ qualification := "cache-aware guest usage"
+ if !e.PressureKnown {
+ qualification = "guest pressure unknown"
+ if e.MayIncludeCache {
+ qualification += "; may include reclaimable cache"
+ }
+ }
+ return fmt.Sprintf("%.0f%% (%s; source %s; %s; %s)", percent, e.State, source, observed, qualification)
+}
+
+// GuestMemoryEvidenceForResource reads only the selected metric. A conflicting
+// raw platform facet or resource LastSeen must never lend it provenance.
+func GuestMemoryEvidenceForResource(r *Resource, now time.Time) GuestMemoryEvidence {
+ if r == nil || r.Metrics == nil || r.Metrics.Memory == nil {
+ return QualifyGuestMemory(0, models.MemoryObservation{}, false, true, now)
+ }
+ _, hasProxmox := r.SourceStatus[SourceProxmox]
+ m := r.Metrics.Memory
+ return QualifyGuestMemory(m.Percent, m.Observation, true, r.Proxmox != nil || hasProxmox || m.Source == SourceProxmox, now)
+}
diff --git a/internal/unifiedresources/metrics_test.go b/internal/unifiedresources/metrics_test.go
index 0e5616d115..60ea3ea09b 100644
--- a/internal/unifiedresources/metrics_test.go
+++ b/internal/unifiedresources/metrics_test.go
@@ -2,6 +2,7 @@ package unifiedresources
import (
"math"
+ "strings"
"testing"
"time"
@@ -575,3 +576,47 @@ func TestGuestMemoryMetricObservationFollowsSelectedSource(t *testing.T) {
t.Fatalf("cross-source selection lost agent receipt or trusts supplied provenance: %+v", merged.Memory)
}
}
+
+func TestGuestMemoryEvidenceQualification(t *testing.T) {
+ now := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
+ at := now.Add(-time.Minute)
+ for _, source := range []string{"available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent"} {
+ for _, percent := range []float64{0, 24, 96, 100} {
+ e := QualifyGuestMemory(percent, models.MemoryObservation{State: "current", Source: source, ObservedAt: at}, true, true, now)
+ if !e.Available || !e.PressureKnown || e.Source != source || e.ObservedAt != at || e.MayIncludeCache {
+ t.Errorf("current measured control rejected: %s/%g: %+v", source, percent, e)
+ }
+ }
+ }
+ for _, source := range []string{"status-mem", "status-freemem", "cluster-resources", "derived-total-minus-used", "previous-snapshot", "unrecognised-source"} {
+ e := QualifyGuestMemory(96, models.MemoryObservation{State: "current", Source: source, ObservedAt: at}, true, true, now)
+ if !e.Available || e.PressureKnown || !strings.Contains(e.Format(96), "guest pressure unknown") {
+ t.Errorf("unqualified source became pressure: %s: %+v", source, e)
+ }
+ if source == "unrecognised-source" && e.Source != "" {
+ t.Fatal("unknown source text leaked into context")
+ }
+ }
+ for _, percent := range []float64{-1, 101, math.NaN(), math.Inf(1), math.Inf(-1)} {
+ e := QualifyGuestMemory(percent, models.MemoryObservation{State: "current", Source: "agent", ObservedAt: at}, true, true, now)
+ if e.Available || e.PressureKnown || e.State != "unavailable" {
+ t.Errorf("invalid percentage became evidence: %+v", e)
+ }
+ }
+ for _, badTime := range []time.Time{{}, time.Unix(0, 0), now.Add(time.Second)} {
+ e := QualifyGuestMemory(24, models.MemoryObservation{State: "current", Source: "agent", ObservedAt: badTime}, true, true, now)
+ if e.PressureKnown || !e.ObservedAt.IsZero() || e.State != "unknown" {
+ t.Errorf("invalid origin became current: %+v", e)
+ }
+ }
+ for _, state := range []string{"last-known", "", "future-state", "unavailable"} {
+ e := QualifyGuestMemory(24, models.MemoryObservation{State: state, Source: "agent", ObservedAt: at}, true, true, now)
+ if e.PressureKnown {
+ t.Errorf("unqualified state became current: %s: %+v", state, e)
+ }
+ }
+ legacy := QualifyGuestMemory(24, models.MemoryObservation{}, true, false, now)
+ if !legacy.Available || !legacy.PressureKnown || legacy.Format(24) != "24%" {
+ t.Fatal("other platform contract changed")
+ }
+}
diff --git a/internal/unifiedresources/views.go b/internal/unifiedresources/views.go
index 09dcbebb94..a7a295d038 100644
--- a/internal/unifiedresources/views.go
+++ b/internal/unifiedresources/views.go
@@ -390,6 +390,11 @@ func (v VMView) MemoryObservation() (models.MemoryObservation, bool) {
return selectedMemoryObservation(v.r)
}
+// MemoryEvidence qualifies the selected metric without borrowing a raw facet or row timestamp.
+func (v VMView) MemoryEvidence(now time.Time) GuestMemoryEvidence {
+ return GuestMemoryEvidenceForResource(v.r, now)
+}
+
func selectedMemoryObservation(r *Resource) (models.MemoryObservation, bool) {
if r == nil || r.Metrics == nil || r.Metrics.Memory == nil {
return models.MemoryObservation{}, false
@@ -727,6 +732,11 @@ func (v ContainerView) MemoryObservation() (models.MemoryObservation, bool) {
return selectedMemoryObservation(v.r)
}
+// MemoryEvidence qualifies the selected metric without borrowing a raw facet or row timestamp.
+func (v ContainerView) MemoryEvidence(now time.Time) GuestMemoryEvidence {
+ return GuestMemoryEvidenceForResource(v.r, now)
+}
+
func (v ContainerView) DiskUsed() int64 {
if v.r == nil {
return 0
diff --git a/internal/unifiedresources/views_test.go b/internal/unifiedresources/views_test.go
index 1e8abe363c..f035ee84f7 100644
--- a/internal/unifiedresources/views_test.go
+++ b/internal/unifiedresources/views_test.go
@@ -2587,3 +2587,26 @@ func TestGuestViewMemoryObservationKeepsSelectedOrigin(t *testing.T) {
})
}
}
+
+func TestGuestMemoryEvidenceUsesSelectedMetricOnly(t *testing.T) {
+ now := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
+ selected := models.MemoryObservation{State: "last-known", Source: "agent", ObservedAt: now.Add(-time.Hour)}
+ r := Resource{LastSeen: now, UpdatedAt: now,
+ Proxmox: &ProxmoxData{Memory: &models.Memory{Usage: 96, Observation: models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: now}}},
+ Metrics: &ResourceMetrics{Memory: &MetricValue{Percent: 24, Observation: selected}},
+ }
+ for _, read := range []func() GuestMemoryEvidence{
+ func() GuestMemoryEvidence { return NewVMView(&r).MemoryEvidence(now) },
+ func() GuestMemoryEvidence { return NewContainerView(&r).MemoryEvidence(now) },
+ } {
+ e := read()
+ if e.PressureKnown || e.State != "last-known" || e.Source != selected.Source || e.ObservedAt != selected.ObservedAt {
+ t.Fatalf("raw facet or fresh row replaced selected origin: %+v", e)
+ }
+ r.Metrics.Memory = nil
+ if e = read(); e.Available || e.PressureKnown || e.State != "unavailable" {
+ t.Fatal("missing selected metric borrowed raw facet")
+ }
+ r.Metrics.Memory = &MetricValue{Percent: 24, Observation: selected}
+ }
+}