Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.
Go aialertsdashboarddockerdocker-monitoringgohomelabinfrastructure-monitoringkubernetesmonitoringmspproxmoxproxmox-backup-serverproxmox-mail-gatewayproxmox-veself-hostedtruenastypescriptvspherewebhooks
Undo only the links that hold a reported source in report-merge
rcourtman70ed7488a42a
diff --git a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md b/docs/release-control/v6/internal/subsystems/agent-lifecycle.mdindex 6a8d9af826..247826ab44 100644--- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md+++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md@@ -9350,6 +9350,15 @@ registration, enrolment, install, update, removal, report identity andcontinuity are unchanged, and an agent's own declared node link stillignores exclusions.+A report-merge that names sources now picks links by the member that carries+the source instead of by the sources a link's folded side took in with it+(`internal/api/resourceapi/resources.go`, `ReportedManualLinkFolds`). An agent+that holds a Docker host and sits in a VM leaves the VM only when the agent's+own source is named, and a report naming only Docker leaves the agent in the+VM. A report naming every source or none still splits every link. Agent+registration, enrolment, install, update, removal, report identity and+continuity are unchanged.+### Windows installer acceptance engine and absence (8 October 2026)Native installer parsing, service-removal failure controls and actual lifecyclediff --git a/docs/release-control/v6/internal/subsystems/api-contracts.md b/docs/release-control/v6/internal/subsystems/api-contracts.mdindex 0ec4e56316..32da3dd16a 100644--- a/docs/release-control/v6/internal/subsystems/api-contracts.md+++ b/docs/release-control/v6/internal/subsystems/api-contracts.md@@ -12010,3 +12010,15 @@ pairs with the identity-match exclusions. `400 Resource is not merged` andso two linked agents, which share their only source, can be reported.`TestContract_ResourceReportMergeReplacesOperatorLink` pins the responseshape and the link's removal for a VM and the agent linked into it.++A request that names `sources` undoes the links that join a member carrying+one of those sources to the rest of the merged resource, chosen by member+(`ReportedManualLinkFolds`) rather than by the sources a link's folded side+took in along with its own members. Naming the source of a leaf in a chain of+links no longer also splits the leaf's holder from the resource, and where+links form a cycle the named member is cut from every link that would rejoin+it. The request, status codes, response shape and the meaning of an empty+`sources` (every link) are unchanged, and no new endpoint, field or stored+decision is added: `exclusions` still counts one link pair each.+`TestResourceReportMergeSourceFilterDetachesChainMembers` reports a chain+through the handler and the REST listing.diff --git a/docs/release-control/v6/internal/subsystems/storage-recovery.md b/docs/release-control/v6/internal/subsystems/storage-recovery.mdindex 6c77bc65ef..35f80b05c6 100644--- a/docs/release-control/v6/internal/subsystems/storage-recovery.md+++ b/docs/release-control/v6/internal/subsystems/storage-recovery.md@@ -6748,3 +6748,12 @@ links' pairs, which deletes the pair's link row in the resource store as anunlink does. Nothing new is persisted: the record is rebuilt with eachregistry generation, and no schema, backup, retention, migration orrecovery path is added or moved.++Each fold also records its two sides' own sources (`HolderOwn`, `FoldedOwn`),+in the same unexported in-memory field, and `ReportedManualLinkFolds` in+`internal/unifiedresources/manual_link_folds.go` uses them so a+source-filtered report-merge in `internal/api/resourceapi/resources.go`+excludes the links that join a member carrying the named source to the rest+of the merged resource. That changes which pairs the report excludes, through+the same exclusion write as before; no schema, backup, retention, migration or recovery path is added or+moved.diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.mdindex 710e2f90b5..9e2fca2e1d 100644--- a/docs/release-control/v6/internal/subsystems/unified-resources.md+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md@@ -7554,14 +7554,37 @@ fold already recorded (the holder still carries what the earlier foldbrought). The record is unexported and rides in-memoryclones, so it reaches the resources API's registry, which seeds from themonitor's already-linked listing and never holds the folded row.-Report-merge excludes the pair of every link whose folded side brought a-reported source category (`ResourceRegistry.ManualLinkFolds`), and the-store's one-decision-per-pair rule deletes that link row, so the pair splits-on every surface from the monitor's next rebuild, as with unlink. The drawer-reports every merged source. Categories are not attributed to individual-contributors: naming one that some folded side carried undoes that link even-where the holder carries it too, and a source that arrived along a chain of-links undoes every link on its way to the merged resource.+A fold's `Sources` is its folded side's whole subtree, so it cannot say+which member of a chain a source belongs to. Selecting links by it undid every+link on the way to a reported source (a Docker host linked into an agent+linked into a VM: naming Docker split the agent from the VM too), and where+links form a cycle it left the reported member joined through the link it did+not name. Each fold therefore also records what its two sides are on their own+(`HolderOwn`, `FoldedOwn`): the sources the side's row listed before any link+folded something into it, which a side that already took members in reads from+the record its first fold made. `recordManualLinkFold` runs before the other+side is merged for that reason, and a repeated pair unions the own sources as+it does `Sources`.++Report-merge picks links by member (`ReportedManualLinkFolds`, fed by+`ResourceRegistry.ManualLinkFolds`). A member whose own sources include a+reported source is reported, whatever it took in; the resource being reported+on never is, since it is what stays. A link is undone when it joins a reported+member to a member that stays, one the resource still reaches without passing+through a reported member, or to another reported member. A member that hangs+below a reported one and carries no reported source goes with it, and the link+holding it there stays: the report names a source that is not theirs to undo.+Where links form a cycle, a reported member linked both to the resource and to+a member that stays is cut from both, so the surviving link cannot fold it+back in. The store's one-decision-per-pair rule deletes each undone link row,+so the pair splits on every surface from the monitor's next rebuild, as with+unlink. The drawer reports every merged source, which undoes every link, and+so does a report naming none. A fold recorded without its sides' own sources+falls back to its folded subtree, and a member several folds name takes the+union of what they record, since a refold unions into its own pair's fold+only. A resource the records no longer name (re-keyed after the links folded)+is replaced by the one holder no link folded; when no single holder qualifies,+which the recorder does not produce, the folds' subtrees decide, as before.A link joins only the pair its row names. Canonical-ID succession re-keyslink endpoints with `UPDATE OR IGNORE` but moves `primary_id`@@ -7589,8 +7612,18 @@ monitor's registry and on one seeded from its listing, and`TestManualLinkFoldsRecordEachPairOnceAcrossRecordIngests` folds a TrueNASsystem into a VM across repeated record ingests, and`TestManualLinkFoldRepeatedPairKeepsEarlierSources` refolds a side with fewer-sources than it first brought. `TestResourceAPIReportMergeExcludesRegistryLinkFolds`-keeps the handler on the registry's fold record.+sources than it first brought. `TestManualLinkFoldOwnSourcesComeFromTheFirstRecord`+keeps a holder's own sources from growing with what it takes in.+`TestReportedManualLinkFoldsDetachMembersCarryingTheSource` selects links for a+leaf, a holder, both and a cycle (a TrueNAS VM taking in a Proxmox storage and+a vSphere VM that is also linked to the storage), and+`TestReportedManualLinkFoldsLeaveNoReportedMemberJoined` checks that no+reported member stays joined to the resource through the links left, which the+subtree rule fails on that cycle. `TestResourceReportMergeSourceFilterDetachesChainMembers`+reports a Docker host linked into an agent linked into a VM through the API,+for the leaf, the holder, both, every source and no filter, on the monitor and+the REST surface. `TestResourceAPIReportMergeExcludesRegistryLinkFolds` keeps+the handler on the registry's fold record.### Reconciled node-agent relink fold bookkeepingdiff --git a/internal/api/contract_test.go b/internal/api/contract_test.goindex 5f9d67b9f6..d79ea63ebf 100644--- a/internal/api/contract_test.go+++ b/internal/api/contract_test.go@@ -14971,6 +14971,90 @@ func TestContract_ResourceReportMergeReplacesOperatorLink(t *testing.T) {}}+// A source-filtered report-merge undoes the links that join the members+// carrying the named source, not every link along the way to it: a Docker+// host linked into an agent linked into a VM leaves the VM and the agent+// merged when only the Docker source is named.+func TestContract_ResourceReportMergeSourceFilterKeepsLinksOfUnreportedMembers(t *testing.T) {+ now := time.Now().UTC()+ h := newActionTestResourceHandlers(t, &config.Config{DataPath: t.TempDir()})+ h.SetStateProvider(resourceStateProvider{snapshot: models.StateSnapshot{+ LastUpdate: now,+ VMs: []models.VM{{ID: "lab:pve1:101", VMID: 101, Name: "web", Node: "pve1", Instance: "lab", Status: "running", Type: "qemu", LastSeen: now}},+ Hosts: []models.Host{{ID: "host-box", Hostname: "box-agent", MachineID: "fedcba9876543210", Status: "online", LastSeen: now}},+ DockerHosts: []models.DockerHost{{ID: "docker-box", Hostname: "dock-box", MachineID: "0011223344556677", Status: "online", LastSeen: now}},+ }})+ list := func() []unifiedresources.Resource {+ t.Helper()+ rec := httptest.NewRecorder()+ h.HandleListResources(rec, httptest.NewRequest(http.MethodGet, "/api/resources", nil))+ if rec.Code != http.StatusOK {+ t.Fatalf("/api/resources status = %d, body=%s", rec.Code, rec.Body.String())+ }+ var resp ResourcesResponse+ if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {+ t.Fatalf("decode /api/resources: %v", err)+ }+ return resp.Data+ }+ var vmID, agentID, dockerID string+ for _, resource := range list() {+ switch {+ case resource.Type == unifiedresources.ResourceTypeVM:+ vmID = resource.ID+ case resource.Docker != nil:+ dockerID = resource.ID+ case resource.Agent != nil:+ agentID = resource.ID+ }+ }+ if vmID == "" || agentID == "" || dockerID == "" {+ t.Fatalf("fixture did not list a separate VM, agent and Docker host: %+v", list())+ }+ link := func(fromID, targetID string) {+ t.Helper()+ rec := httptest.NewRecorder()+ h.HandleLink(rec, httptest.NewRequest(http.MethodPost, "/api/resources/"+fromID+"/link", strings.NewReader(`{"targetId":"`+targetID+`"}`)))+ if rec.Code != http.StatusOK {+ t.Fatalf("link status = %d, body=%s", rec.Code, rec.Body.String())+ }+ }+ // Links apply in store order: the agent takes in the Docker host, then the+ // VM takes in the agent.+ link(agentID, dockerID)+ link(vmID, agentID)+ if chained := list(); len(chained) != 1 {+ t.Fatalf("links listed %d resources, want the chain folded into one: %+v", len(chained), chained)+ }++ rec := httptest.NewRecorder()+ h.HandleReportMerge(rec, httptest.NewRequest(http.MethodPost, "/api/resources/"+vmID+"/report-merge", strings.NewReader(`{"sources":["docker"]}`)))+ if rec.Code != http.StatusOK {+ t.Fatalf("report-merge status = %d, body=%s", rec.Code, rec.Body.String())+ }+ store, err := h.getStore("default")+ if err != nil {+ t.Fatal(err)+ }+ links, err := store.GetLinks()+ if err != nil || len(links) != 1 {+ t.Fatalf("report-merge of the Docker source left links %+v (err %v), want only the VM and agent link", links, err)+ }+ if (links[0].ResourceA != vmID || links[0].ResourceB != agentID) && (links[0].ResourceA != agentID || links[0].ResourceB != vmID) {+ t.Fatalf("report-merge kept link %+v, want the VM and agent link", links[0])+ }+ split := list()+ if len(split) != 2 {+ t.Fatalf("report-merge listed %d resources, want the VM with its agent and the Docker host apart: %+v", len(split), split)+ }+ for _, resource := range split {+ docker := slices.Contains(resource.Sources, unifiedresources.SourceDocker)+ if docker != (len(resource.Sources) == 1) {+ t.Fatalf("report-merge listed %s with sources %v, want the Docker host alone and the VM without Docker", resource.ID, resource.Sources)+ }+ }+}+func TestContract_ResourceListPolicyMetadata(t *testing.T) {now := time.Date(2026, 3, 17, 10, 0, 0, 0, time.UTC)h := newActionTestResourceHandlers(t, &config.Config{DataPath: t.TempDir()})diff --git a/internal/api/resourceapi/resources.go b/internal/api/resourceapi/resources.goindex b6c2bcb214..45b924df23 100644--- a/internal/api/resourceapi/resources.go+++ b/internal/api/resourceapi/resources.go@@ -1019,17 +1019,17 @@ func (h *QueryService) HandleReportMerge(w http.ResponseWriter, r *http.Request)reason = "reported_incorrect_merge"}- // An operator link that brought a reported source in is undone by- // excluding the link's own pair, which replaces the link in the store.- // Identity merges are kept apart by excluding the candidate ID each- // source's record takes when no match is allowed; for a linked source that- // candidate names neither side of the link.+ // An operator link that joins a member carrying a reported source to the+ // part that stays is undone by excluding the link's own pair, which+ // replaces the link in the store. A fold's sources span everything its+ // folded side took in, so the registry picks links by member instead+ // (ReportedManualLinkFolds). Identity merges are kept apart by excluding+ // the candidate ID each source's record takes when no match is allowed;+ // for a linked source that candidate names neither side of the link.type exclusionPair struct{ a, b string }pairs := make([]exclusionPair, 0, len(linkFolds)+len(sourceTargets))- for _, fold := range linkFolds {- if selected(fold.Sources...) {- pairs = append(pairs, exclusionPair{fold.HolderID, fold.FoldedID})- }+ for _, fold := range unified.ReportedManualLinkFolds(path, linkFolds, selected) {+ pairs = append(pairs, exclusionPair{fold.HolderID, fold.FoldedID})}for _, target := range sourceTargets {if !selected(target.Source) || target.CandidateID == "" || target.CandidateID == path {diff --git a/internal/api/resourceapi/resources_report_merge_test.go b/internal/api/resourceapi/resources_report_merge_test.goindex 0ebea98fad..aefd4aa74d 100644--- a/internal/api/resourceapi/resources_report_merge_test.go+++ b/internal/api/resourceapi/resources_report_merge_test.go@@ -5,6 +5,8 @@ import ("encoding/json""net/http""net/http/httptest"+ "slices"+ "strings""testing""time"@@ -144,6 +146,109 @@ func TestResourceReportMergeSourceFilterSelectsLinks(t *testing.T) {}}+// A source filter detaches the members of a chain that carry the source. A+// Docker host linked into an agent that is linked into a VM leaves two folds,+// and the agent's fold lists the Docker source as well, because its subtree+// holds the Docker host. Naming Docker used to undo the VM-agent link too, and+// the pair was apart on every surface; now only the link that holds the Docker+// host goes. Naming the agent detaches the agent from the VM and keeps the+// Docker host in the agent, which the report does not name.+func TestResourceReportMergeSourceFilterDetachesChainMembers(t *testing.T) {+ now := time.Now().UTC()+ snapshot := models.StateSnapshot{+ LastUpdate: now,+ VMs: []models.VM{{ID: "lab:pve1:101", VMID: 101, Name: "web", Node: "pve1", Instance: "lab", Status: "running", Type: "qemu", LastSeen: now}},+ Hosts: []models.Host{{ID: "host-box", Hostname: "box-agent", MachineID: "fedcba9876543210", Status: "online", LastSeen: now}},+ DockerHosts: []models.DockerHost{{ID: "docker-box", Hostname: "dock-box", MachineID: "0011223344556677", Status: "online", LastSeen: now}},+ }+ for _, tc := range []struct {+ name string+ sources []string+ // want lists the sources of each resource left standing, and links+ // how many of the chain's two links the report leaves in the store.+ want []string+ links int+ }{+ {"leaf member", []string{"docker"}, []string{"agent+proxmox", "docker"}, 1},+ {"member that took another in", []string{"agent"}, []string{"agent+docker", "proxmox"}, 1},+ {"both members", []string{"agent", "docker"}, []string{"agent", "docker", "proxmox"}, 0},+ {"every source", []string{"proxmox", "agent", "docker"}, []string{"agent", "docker", "proxmox"}, 0},+ {"no filter", nil, []string{"agent", "docker", "proxmox"}, 0},+ } {+ t.Run(tc.name, func(t *testing.T) {+ h, store, rebuild := newReportMergeHarness(t, snapshot)+ var vmID, agentID, dockerID string+ for _, resource := range rebuild() {+ switch {+ case resource.Type == unified.ResourceTypeVM:+ vmID = resource.ID+ case resource.Agent != nil:+ agentID = resource.ID+ case resource.Docker != nil:+ dockerID = resource.ID+ }+ }+ if vmID == "" || agentID == "" || dockerID == "" {+ t.Fatalf("fixture lacks a standalone VM, agent and Docker host: %+v", rebuild())+ }+ // Links apply in store order: the agent takes in the Docker host,+ // then the VM takes in the agent.+ postResourceAction(t, h, agentID, "link", map[string]any{"targetId": dockerID})+ postResourceAction(t, h, vmID, "link", map[string]any{"targetId": agentID})+ chained := rebuild()+ if got := resourceSourceGroups(chained); !slices.Equal(got, []string{"agent+docker+proxmox"}) {+ t.Fatalf("links left %v, want the chain folded into one resource", got)+ }+ assertRESTSourceGroups(t, h, []string{"agent+docker+proxmox"})++ body := map[string]any{}+ if tc.sources != nil {+ body["sources"] = tc.sources+ }+ postResourceAction(t, h, chained[0].ID, "report-merge", body)+ if got := resourceSourceGroups(rebuild()); !slices.Equal(got, tc.want) {+ t.Fatalf("monitor lists %v after reporting %v, want %v", got, tc.sources, tc.want)+ }+ assertRESTSourceGroups(t, h, tc.want)+ if links, err := store.GetLinks(); err != nil || len(links) != tc.links {+ t.Fatalf("report-merge left %d links (err %v), want %d: %+v", len(links), err, tc.links, links)+ }+ })+ }+}++// resourceSourceGroups lists the sources of each resource, sorted, as+// "a+b" groups in sorted order.+func resourceSourceGroups(resources []unified.Resource) []string {+ groups := make([]string, 0, len(resources))+ for _, resource := range resources {+ sources := make([]string, 0, len(resource.Sources))+ for _, source := range resource.Sources {+ sources = append(sources, string(source))+ }+ slices.Sort(sources)+ groups = append(groups, strings.Join(sources, "+"))+ }+ slices.Sort(groups)+ return groups+}++func assertRESTSourceGroups(t *testing.T, h *QueryService, want []string) {+ t.Helper()+ rec := httptest.NewRecorder()+ h.HandleListResources(rec, httptest.NewRequest(http.MethodGet, "/api/resources", nil))+ if rec.Code != http.StatusOK {+ t.Fatalf("list status = %d body=%s", rec.Code, rec.Body.String())+ }+ var resp ResourcesResponse+ if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {+ t.Fatalf("decode list: %v", err)+ }+ if got := resourceSourceGroups(resp.Data); !slices.Equal(got, want) {+ t.Fatalf("REST lists %v, want %v", got, want)+ }+}+// Report-merge on a pair identity matching merged keeps it apart through the// monitor's rebuilds, whose identity pins persist on one generation and steer// the next.diff --git a/internal/unifiedresources/clone_test.go b/internal/unifiedresources/clone_test.goindex 5e3a88e1b2..a5508f29c6 100644--- a/internal/unifiedresources/clone_test.go+++ b/internal/unifiedresources/clone_test.go@@ -273,8 +273,11 @@ func TestCloneResource_MutateParentBySource(t *testing.T) {// registry's listing can name each link's pair, and stay detached from them.func TestCloneResource_CarriesLinkFolds(t *testing.T) {original := &Resource{- ID: "vm-1",- linkFolds: []ManualLinkFold{{HolderID: "vm-1", FoldedID: "agent-1", Sources: []DataSource{SourceAgent}}},+ ID: "vm-1",+ linkFolds: []ManualLinkFold{{+ HolderID: "vm-1", FoldedID: "agent-1", Sources: []DataSource{SourceAgent},+ HolderOwn: []DataSource{SourceProxmox}, FoldedOwn: []DataSource{SourceAgent},+ }},}cloned := cloneResource(original)if len(cloned.linkFolds) != 1 || cloned.linkFolds[0].FoldedID != "agent-1" {@@ -282,8 +285,11 @@ func TestCloneResource_CarriesLinkFolds(t *testing.T) {}cloned.linkFolds[0].Sources[0] = SourceDocker+ cloned.linkFolds[0].HolderOwn[0] = SourceDocker+ cloned.linkFolds[0].FoldedOwn[0] = SourceDockercloned.linkFolds[0].FoldedID = "MUTATED"- if original.linkFolds[0].Sources[0] != SourceAgent || original.linkFolds[0].FoldedID != "agent-1" {+ if original.linkFolds[0].Sources[0] != SourceAgent || original.linkFolds[0].FoldedID != "agent-1" ||+ original.linkFolds[0].HolderOwn[0] != SourceProxmox || original.linkFolds[0].FoldedOwn[0] != SourceAgent {t.Errorf("mutating cloned link folds affected the original: %+v", original.linkFolds)}}diff --git a/internal/unifiedresources/code_standards_test.go b/internal/unifiedresources/code_standards_test.goindex 8cc0b23cf7..8b911ca2a2 100644--- a/internal/unifiedresources/code_standards_test.go+++ b/internal/unifiedresources/code_standards_test.go@@ -771,6 +771,7 @@ func TestResourceAPIReportMergeExcludesRegistryLinkFolds(t *testing.T) {for _, snippet := range []string{"linkFolds := registry.ManualLinkFolds(path)","if len(resource.Sources) < 2 && len(linkFolds) == 0 {",+ "for _, fold := range unified.ReportedManualLinkFolds(path, linkFolds, selected) {","pairs = append(pairs, exclusionPair{fold.HolderID, fold.FoldedID})",} {if !strings.Contains(source, snippet) {diff --git a/internal/unifiedresources/manual_link_folds.go b/internal/unifiedresources/manual_link_folds.goindex 4091d9e66a..916c8883b2 100644--- a/internal/unifiedresources/manual_link_folds.go+++ b/internal/unifiedresources/manual_link_folds.go@@ -6,11 +6,17 @@ package unifiedresources// (the store keeps one decision per pair). A candidate ID derived from the// merged resource's type cannot: for an agent linked into a VM it is a vm- ID// that names neither side. Sources lists what the folded side brought,-// including anything links had already folded into it.+// including anything links had already folded into it. HolderOwn and+// FoldedOwn list what each side is on its own: the sources its row listed+// before any link folded something into it. A row that took other members in+// lists theirs too, so Sources alone cannot say which member of a chain a+// source belongs to.type ManualLinkFold struct {- HolderID string- FoldedID string- Sources []DataSource+ HolderID string+ FoldedID string+ Sources []DataSource+ HolderOwn []DataSource+ FoldedOwn []DataSource}// recordManualLinkFold notes on the link's merged resource that it took in@@ -24,13 +30,19 @@ type ManualLinkFold struct {// with fewer sources than it brought before, so a repeated pair adds its// sources to the fold already recorded. The holder still carries what the// earlier fold brought.+//+// Call it before the other side's data is merged into primary: the sources+// primary lists at that point are its own unless an earlier fold on it+// recorded them (linkMemberOwnSources).func recordManualLinkFold(primary *Resource, primaryID string, other *Resource, otherID string) {added := make([]ManualLinkFold, 0, len(other.linkFolds)+1)added = append(added, other.linkFolds...)added = append(added, ManualLinkFold{- HolderID: primaryID,- FoldedID: otherID,- Sources: cloneDataSourceSlice(other.Sources),+ HolderID: primaryID,+ FoldedID: otherID,+ Sources: cloneDataSourceSlice(other.Sources),+ HolderOwn: linkMemberOwnSources(primary, primaryID),+ FoldedOwn: linkMemberOwnSources(other, otherID),})folds := make([]ManualLinkFold, 0, len(primary.linkFolds)+len(added))recorded := make(map[string]int, len(primary.linkFolds)+len(added))@@ -38,6 +50,12 @@ func recordManualLinkFold(primary *Resource, primaryID string, other *Resource,key := exclusionKey(fold.HolderID, fold.FoldedID)if i, ok := recorded[key]; ok {folds[i].Sources = addSources(cloneDataSourceSlice(folds[i].Sources), fold.Sources)+ holderOwn, foldedOwn := fold.HolderOwn, fold.FoldedOwn+ if folds[i].HolderID != fold.HolderID {+ holderOwn, foldedOwn = foldedOwn, holderOwn+ }+ folds[i].HolderOwn = addSources(cloneDataSourceSlice(folds[i].HolderOwn), holderOwn)+ folds[i].FoldedOwn = addSources(cloneDataSourceSlice(folds[i].FoldedOwn), foldedOwn)return}recorded[key] = len(folds)@@ -72,7 +90,129 @@ func cloneManualLinkFolds(in []ManualLinkFold) []ManualLinkFold {out := make([]ManualLinkFold, len(in))for i, fold := range in {fold.Sources = cloneDataSourceSlice(fold.Sources)+ fold.HolderOwn = cloneDataSourceSlice(fold.HolderOwn)+ fold.FoldedOwn = cloneDataSourceSlice(fold.FoldedOwn)out[i] = fold}return out}++// linkMemberOwnSources returns what the member id is on its own: the sources+// an earlier fold on row recorded for it, else the sources row lists, which+// are its own while no link has folded anything into it.+func linkMemberOwnSources(row *Resource, id string) []DataSource {+ for _, fold := range row.linkFolds {+ if fold.HolderID == id && fold.HolderOwn != nil {+ return cloneDataSourceSlice(fold.HolderOwn)+ }+ if fold.FoldedID == id && fold.FoldedOwn != nil {+ return cloneDataSourceSlice(fold.FoldedOwn)+ }+ }+ return cloneDataSourceSlice(row.Sources)+}++// ReportedManualLinkFolds picks, from the folds of the merged resource rootID,+// the links a report-merge undoes. reported says whether any of the sources+// it is given was named by the report; a report naming no source reports+// every source.+//+// A report names sources, and the sources belong to link members: the+// resources the folds' pairs name. A member is reported when its own sources+// include a reported one, whatever it took in; the resource being reported on+// is never one, since it is what stays. Reported members leave the part of+// the chain that stays, so a link is undone when it joins a reported member+// to a member that stays or to another reported member. A member that hangs+// below a reported one and carries no reported source goes with it, and the+// link that holds it there stays: the report names a source that is not+// theirs to undo. A member that stays is one the root still reaches without+// passing through a reported member, which matters where links form a cycle:+// a reported member linked both to the root and to a member that stays is cut+// from both, so it cannot rejoin through the other.+//+// A fold's Sources lists its folded side's whole subtree, so selecting folds+// by them undid every link along the way to a reported source: a source+// brought by a leaf split the leaf's holder from the root as well.+func ReportedManualLinkFolds(rootID string, folds []ManualLinkFold, reported func(...DataSource) bool) []ManualLinkFold {+ if len(folds) == 0 {+ return nil+ }+ // A member named by several folds takes the union of what they record: a+ // repeated pair unions into its own fold only, so a member whose refold+ // gained a source would otherwise read differently from fold to fold.+ own := make(map[string][]DataSource, len(folds)+1)+ adjacent := make(map[string][]string, len(folds)+1)+ folded := make(map[string]bool, len(folds))+ for _, fold := range folds {+ if fold.HolderOwn != nil {+ own[fold.HolderID] = addSources(own[fold.HolderID], fold.HolderOwn)+ }+ if fold.FoldedOwn != nil {+ own[fold.FoldedID] = addSources(own[fold.FoldedID], fold.FoldedOwn)+ }+ adjacent[fold.HolderID] = append(adjacent[fold.HolderID], fold.FoldedID)+ adjacent[fold.FoldedID] = append(adjacent[fold.FoldedID], fold.HolderID)+ folded[fold.FoldedID] = true+ }+ // A fold recorded without its sides' own sources lists the folded side's+ // whole subtree, the most that side can have been.+ for _, fold := range folds {+ if _, ok := own[fold.FoldedID]; !ok {+ own[fold.FoldedID] = fold.Sources+ }+ }++ root := rootID+ if _, ok := adjacent[root]; !ok {+ // The root was re-keyed after the links folded: the one holder no+ // link folded is the resource the records hang from.+ root = ""+ for _, fold := range folds {+ if folded[fold.HolderID] {+ continue+ }+ if root != "" && root != fold.HolderID {+ root = ""+ break+ }+ root = fold.HolderID+ }+ }+ if root == "" {+ var selected []ManualLinkFold+ for _, fold := range folds {+ if reported(fold.Sources...) {+ selected = append(selected, fold)+ }+ }+ return selected+ }++ away := make(map[string]bool, len(adjacent))+ for id := range adjacent {+ if id != root && reported(own[id]...) {+ away[id] = true+ }+ }+ stays := map[string]bool{root: true}+ for queue := []string{root}; len(queue) > 0; {+ current := queue[0]+ queue = queue[1:]+ for _, next := range adjacent[current] {+ if away[next] || stays[next] {+ continue+ }+ stays[next] = true+ queue = append(queue, next)+ }+ }+ carried := func(id string) bool { return !away[id] && !stays[id] }++ var selected []ManualLinkFold+ for _, fold := range folds {+ if (away[fold.HolderID] || away[fold.FoldedID]) && !carried(fold.HolderID) && !carried(fold.FoldedID) {+ selected = append(selected, fold)+ }+ }+ return selected+}diff --git a/internal/unifiedresources/registry.go b/internal/unifiedresources/registry.goindex af3d95811e..1e1aa35333 100644--- a/internal/unifiedresources/registry.go+++ b/internal/unifiedresources/registry.go@@ -5068,8 +5068,10 @@ func (rr *ResourceRegistry) applyManualLinks(thresholds map[DataSource]time.Durarr.recordLinkOwnPin(primaryID, primary)rr.recordLinkOwnPin(otherID, other)- rr.mergeResourceData(primary, other, thresholds)+ // The fold records what each side is on its own, so it reads the+ // primary before the other side's sources join its list.recordManualLinkFold(primary, primaryID, other, otherID)+ rr.mergeResourceData(primary, other, thresholds)delete(rr.resources, otherID)// The fold record names every ID along a chain of links, so the chain// resolves to its last primary; the folded row's own index entriesdiff --git a/internal/unifiedresources/registry_merge_policy_test.go b/internal/unifiedresources/registry_merge_policy_test.goindex d8dbbf061d..d8bb80ed7c 100644--- a/internal/unifiedresources/registry_merge_policy_test.go+++ b/internal/unifiedresources/registry_merge_policy_test.go@@ -1958,9 +1958,17 @@ func TestManualLinkFoldsNameEachLinkPairAlongAChain(t *testing.T) {// The agent linked from its own side still folds into the guest it runs// in, and the fold names the link's pair either way.+ // Each side also records what it is on its own: the agent's fold into the VM+ // lists the Docker host's source too, but the agent is only the agent.want := []ManualLinkFold{- {HolderID: agentID, FoldedID: dockerID, Sources: []DataSource{SourceDocker}},- {HolderID: vmID, FoldedID: agentID, Sources: []DataSource{SourceAgent, SourceDocker}},+ {+ HolderID: agentID, FoldedID: dockerID, Sources: []DataSource{SourceDocker},+ HolderOwn: []DataSource{SourceAgent}, FoldedOwn: []DataSource{SourceDocker},+ },+ {+ HolderID: vmID, FoldedID: agentID, Sources: []DataSource{SourceAgent, SourceDocker},+ HolderOwn: []DataSource{SourceProxmox}, FoldedOwn: []DataSource{SourceAgent},+ },}if got := adapter.currentRegistry().ManualLinkFolds(vmID); !reflect.DeepEqual(got, want) {t.Fatalf("monitor folds = %+v, want %+v", got, want)@@ -2023,7 +2031,10 @@ func TestManualLinkFoldsRecordEachPairOnceAcrossRecordIngests(t *testing.T) {if listed := rr.List(); len(listed) != 1 {t.Fatalf("link did not fold the TrueNAS system into the VM: %+v", listed)}- want := []ManualLinkFold{{HolderID: vmID, FoldedID: systemID, Sources: []DataSource{SourceTrueNAS}}}+ want := []ManualLinkFold{{+ HolderID: vmID, FoldedID: systemID, Sources: []DataSource{SourceTrueNAS},+ HolderOwn: []DataSource{SourceProxmox}, FoldedOwn: []DataSource{SourceTrueNAS},+ }}if got := rr.ManualLinkFolds(vmID); !reflect.DeepEqual(got, want) {t.Fatalf("folds after repeated record ingests = %+v, want %+v", got, want)}@@ -2037,12 +2048,184 @@ func TestManualLinkFoldRepeatedPairKeepsEarlierSources(t *testing.T) {holder := &Resource{ID: "vm-1"}recordManualLinkFold(holder, "vm-1", &Resource{ID: "agent-1", Sources: []DataSource{SourceAgent, SourceDocker}}, "agent-1")recordManualLinkFold(holder, "vm-1", &Resource{ID: "agent-1", Sources: []DataSource{SourceAgent}}, "agent-1")- want := []ManualLinkFold{{HolderID: "vm-1", FoldedID: "agent-1", Sources: []DataSource{SourceAgent, SourceDocker}}}+ want := []ManualLinkFold{{+ HolderID: "vm-1", FoldedID: "agent-1", Sources: []DataSource{SourceAgent, SourceDocker},+ FoldedOwn: []DataSource{SourceAgent, SourceDocker},+ }}if !reflect.DeepEqual(holder.linkFolds, want) {t.Fatalf("folds = %+v, want %+v", holder.linkFolds, want)}}+// A holder that already took a member in lists that member's sources too, so+// its own sources come from the record the first fold made, not from its row.+func TestManualLinkFoldOwnSourcesComeFromTheFirstRecord(t *testing.T) {+ holder := &Resource{ID: "vm-1", Sources: []DataSource{SourceProxmox}}+ first := &Resource{ID: "agent-1", Sources: []DataSource{SourceAgent}}+ recordManualLinkFold(holder, "vm-1", first, "agent-1")+ holder.Sources = addSources(holder.Sources, first.Sources)+ second := &Resource{ID: "docker-1", Sources: []DataSource{SourceDocker}}+ recordManualLinkFold(holder, "vm-1", second, "docker-1")++ if got := holder.linkFolds[1].HolderOwn; !reflect.DeepEqual(got, []DataSource{SourceProxmox}) {+ t.Fatalf("second fold lists the holder's own sources as %v, want only %v", got, SourceProxmox)+ }++ // The VM then folds into another resource: the member it was keeps its own+ // sources, not the ones it took in.+ root := &Resource{ID: "node-1", Sources: []DataSource{SourceTrueNAS}}+ recordManualLinkFold(root, "node-1", holder, "vm-1")+ last := root.linkFolds[len(root.linkFolds)-1]+ if last.FoldedID != "vm-1" || !reflect.DeepEqual(last.FoldedOwn, []DataSource{SourceProxmox}) {+ t.Fatalf("fold of the holder lists its own sources as %v, want only %v", last.FoldedOwn, SourceProxmox)+ }+ if !reflect.DeepEqual(last.HolderOwn, []DataSource{SourceTrueNAS}) {+ t.Fatalf("fold into the root lists its own sources as %v, want only %v", last.HolderOwn, SourceTrueNAS)+ }+}++// ReportedManualLinkFolds picks the links a source-filtered report undoes by+// member. The members here are the estate that exposed the fold subtree+// rule: a TrueNAS VM V takes in a Proxmox storage S and a vSphere VM W, and+// the operator also linked S into W, so recording every link of the chain+// leaves three pairs with W reachable through two of them.+func TestReportedManualLinkFoldsDetachMembersCarryingTheSource(t *testing.T) {+ pair := func(holder, folded string, sources, holderOwn, foldedOwn []DataSource) ManualLinkFold {+ return ManualLinkFold{HolderID: holder, FoldedID: folded, Sources: sources, HolderOwn: holderOwn, FoldedOwn: foldedOwn}+ }+ named := func(names ...DataSource) func(...DataSource) bool {+ return func(sources ...DataSource) bool {+ if len(names) == 0 {+ return true+ }+ for _, source := range sources {+ if slices.Contains(names, source) {+ return true+ }+ }+ return false+ }+ }+ pairs := func(folds []ManualLinkFold) []string {+ out := make([]string, 0, len(folds))+ for _, fold := range folds {+ out = append(out, fold.HolderID+">"+fold.FoldedID)+ }+ slices.Sort(out)+ return out+ }+ truenas, proxmox, vmware := []DataSource{SourceTrueNAS}, []DataSource{SourceProxmox}, []DataSource{SourceVMware}+ cycle := []ManualLinkFold{+ pair("V", "S", proxmox, truenas, proxmox),+ pair("V", "W", vmware, truenas, vmware),+ pair("W", "S", proxmox, vmware, proxmox),+ }+ // A leaf below a holder below the root: V takes in A (agent), which took+ // in D (docker), so A's fold lists both sources.+ agent, docker := []DataSource{SourceAgent}, []DataSource{SourceDocker}+ tree := []ManualLinkFold{+ pair("A", "D", docker, agent, docker),+ pair("V", "A", []DataSource{SourceAgent, SourceDocker}, proxmox, agent),+ }++ for _, tc := range []struct {+ name string+ root string+ folds []ManualLinkFold+ named []DataSource+ want []string+ }{+ {"cycle: member reachable two ways is cut from both", "V", cycle, []DataSource{SourceVMware}, []string{"V>W", "W>S"}},+ {"cycle: the storage leaves, the VM keeps the other link", "V", cycle, []DataSource{SourceProxmox}, []string{"V>S", "W>S"}},+ {"cycle: both reported members leave", "V", cycle, []DataSource{SourceVMware, SourceProxmox}, []string{"V>S", "V>W", "W>S"}},+ {"cycle: the root's own source selects nothing", "V", cycle, []DataSource{SourceTrueNAS}, nil},+ {"cycle: no filter undoes every link", "V", cycle, nil, []string{"V>S", "V>W", "W>S"}},+ {"tree: a leaf's source leaves its holder's link alone", "V", tree, []DataSource{SourceDocker}, []string{"A>D"}},+ {"tree: a holder leaves and keeps what it took in", "V", tree, []DataSource{SourceAgent}, []string{"V>A"}},+ {"tree: both reported, both leave", "V", tree, []DataSource{SourceAgent, SourceDocker}, []string{"A>D", "V>A"}},+ {"tree: the root's own source selects nothing", "V", tree, []DataSource{SourceProxmox}, nil},+ {"tree: no filter undoes every link", "V", tree, nil, []string{"A>D", "V>A"}},+ // A refold of the agent unions a source into the VM's fold of it only,+ // so the agent's two records disagree; the member is the union.+ {"tree: a member whose refold gained the source is reported by it", "V", []ManualLinkFold{+ pair("A", "D", docker, agent, docker),+ pair("V", "A", []DataSource{SourceAgent, SourceDocker}, proxmox, []DataSource{SourceAgent, SourceDocker}),+ }, []DataSource{SourceDocker}, []string{"A>D", "V>A"}},+ {"root re-keyed after the links folded", "V-renamed", tree, []DataSource{SourceDocker}, []string{"A>D"}},+ {"ambiguous root falls back to the folded subtree", "elsewhere", []ManualLinkFold{+ pair("X", "Y", docker, nil, nil), pair("P", "Q", agent, nil, nil),+ }, []DataSource{SourceDocker}, []string{"X>Y"}},+ {"folds recorded without own sources use the folded subtree", "V", []ManualLinkFold{+ {HolderID: "V", FoldedID: "A", Sources: []DataSource{SourceAgent, SourceDocker}},+ {HolderID: "A", FoldedID: "D", Sources: docker},+ }, []DataSource{SourceDocker}, []string{"A>D", "V>A"}},+ } {+ t.Run(tc.name, func(t *testing.T) {+ got := pairs(ReportedManualLinkFolds(tc.root, tc.folds, named(tc.named...)))+ if !slices.Equal(got, tc.want) {+ t.Fatalf("report of %v selected %v, want %v", tc.named, got, tc.want)+ }+ })+ }+}++// Whatever links a report undoes, no member carrying a reported source may+// stay joined to the root through the links left. The rule the registry+// replaced selected folds whose subtree sources named the report, and left W+// joined through S in the cycle: its control here fails the same check.+func TestReportedManualLinkFoldsLeaveNoReportedMemberJoined(t *testing.T) {+ truenas, proxmox, vmware := []DataSource{SourceTrueNAS}, []DataSource{SourceProxmox}, []DataSource{SourceVMware}+ cycle := []ManualLinkFold{+ {HolderID: "V", FoldedID: "S", Sources: proxmox, HolderOwn: truenas, FoldedOwn: proxmox},+ {HolderID: "V", FoldedID: "W", Sources: vmware, HolderOwn: truenas, FoldedOwn: vmware},+ {HolderID: "W", FoldedID: "S", Sources: proxmox, HolderOwn: vmware, FoldedOwn: proxmox},+ }+ joinedToRoot := func(folds []ManualLinkFold, left map[string]bool, member string) bool {+ reached := map[string]bool{"V": true}+ for changed := true; changed; {+ changed = false+ for _, fold := range folds {+ if left[exclusionKey(fold.HolderID, fold.FoldedID)] {+ continue+ }+ if reached[fold.HolderID] != reached[fold.FoldedID] {+ reached[fold.HolderID], reached[fold.FoldedID] = true, true+ changed = true+ }+ }+ }+ return reached[member]+ }+ undone := func(selected []ManualLinkFold) map[string]bool {+ out := make(map[string]bool, len(selected))+ for _, fold := range selected {+ out[exclusionKey(fold.HolderID, fold.FoldedID)] = true+ }+ return out+ }+ for _, tc := range []struct {+ reported DataSource+ member string+ }{{SourceVMware, "W"}, {SourceProxmox, "S"}} {+ named := func(sources ...DataSource) bool { return slices.Contains(sources, tc.reported) }+ got := ReportedManualLinkFolds("V", cycle, named)+ if joinedToRoot(cycle, undone(got), tc.member) {+ t.Errorf("after a report of %s, %s is still joined to V through %v", tc.reported, tc.member, got)+ }++ // The control: folds selected by their folded side's subtree.+ var bySubtree []ManualLinkFold+ for _, fold := range cycle {+ if named(fold.Sources...) {+ bySubtree = append(bySubtree, fold)+ }+ }+ if tc.reported == SourceVMware && !joinedToRoot(cycle, undone(bySubtree), tc.member) {+ t.Errorf("control: selecting by subtree sources detached %s, so the cycle no longer shows the defect", tc.member)+ }+ }+}+func TestLeaseExpiredKubernetesClusterStaysOffline(t *testing.T) {rr := NewRegistry(nil)rr.IngestSnapshot(models.StateSnapshot{