Skip to content

armgit / Pulse

Public Imported from GitHub
0

Real-time monitoring dashboard for Proxmox VE, PBS, Docker, Kubernetes, TrueNAS and vSphere. Self-hosted, with alerts for failed backups, full disks, unhealthy containers and offline nodes.

Merge origin/main into fix/mock-view-manual-links

rcourtman25dc55335dd8
diff --cc docs/release-control/v6/internal/subsystems/agent-lifecycle.md
index c9a8520708,4715dcd9c6..085e20eb28
--- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
+++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md
@@@ -8982,18 -8996,45 +8996,57 @@@ their own heartbeat cutoff (`fleethealt
separate from the monitoring reporting lease). Agent registration, enrolment,
install, update, removal and report identity are unchanged.
+### Mock-mode read views fold linked fixture agents
+
+`internal/monitoring/monitor.go` changed only so the mock-mode unified view
+applies the operator's manual links (unified-resources contract, "Mock-mode
+unified view applies operator links"). A fixture agent the operator linked
+into a guest now folds into that guest in mock read views, as a linked agent
+already does in live mode, so host snapshots built from them
+(`HostsSnapshot`) stop listing it while the guest carries its facet. Agent
+registration, enrolment, report admission, removal, continuity and the
+fixture agent set itself are unchanged, and the view writes nothing to the
+resource store.
+
+ ### Safe methods refused on discovery settings and SAML logout
+
+ `internal/api/method_guard.go`, `internal/api/discovery_handlers.go` and
+ `internal/api/saml_handlers.go` changed only so `/api/discovery/settings` and
+ `/api/saml/{id}/logout` answer `405` to every method outside their allowed set,
+ `GET` and `HEAD` included. The settings write still takes `PUT` or `POST` and
+ logout still takes `POST`. No agent registration, enrolment, install, update or
+ removal path changed.
+
+ ### SAML SLO responses bound to the logout that requested them
+
+ `internal/api/saml_service.go` and `internal/api/saml_handlers.go` changed only
+ so `/api/saml/{id}/slo` accepts an IdP LogoutResponse when its signed
+ `InResponseTo` names an unexpired LogoutRequest that `/api/saml/{id}/logout`
+ issued, once, in a browser carrying no session or the session that logout
+ ended. Other responses are refused, the SLO handler writes no session or
+ cookie for any response (the logout request already cleared both), and
+ `/api/saml/{id}/logout` refuses a request that carries no session. No agent
+ registration, enrolment, install, update or removal path changed.
+
+ ### Manual SAML IdP certificate published as base64 DER
+
+ `internal/api/saml_service.go` changed only so a SAML provider configured with
+ `idpSsoUrl` and a PEM `idpCertificate` or `idpCertFile` puts that certificate
+ into its in-memory IdP descriptor as base64 DER, which crewjam/saml can read
+ when it verifies the IdP's ACS Responses and LogoutResponses. No agent
+ registration, enrolment, install, update or removal path changed.
+
+ ### SAML logins bound to the browser that started them
+
+ `internal/api/saml_service.go` and `internal/api/saml_handlers.go` changed only
+ so `/api/saml/{id}/login` records each SAML AuthnRequest against a short-lived
+ login cookie and `/api/saml/{id}/acs` accepts the IdP's signed answer to it
+ once, in that browser, at default settings, reposting the IdP's cross-site
+ delivery once from Pulse's origin so the cookie arrives. Signed content
+ carrying a prefixed attribute or namespace declaration named like a SAML
+ attribute is refused. No agent
+ registration, enrolment, install, update or removal path changed.
+
### Demo write guard ignores websocket upgrade headers
`internal/api/demo_middleware.go` changed only so a `POST`, `PUT`, `PATCH` or
diff --cc docs/release-control/v6/internal/subsystems/unified-resources.md
index 50d7194f43,c284a54e4a..4d86f762eb
--- a/docs/release-control/v6/internal/subsystems/unified-resources.md
+++ b/docs/release-control/v6/internal/subsystems/unified-resources.md
@@@ -6671,67 -6786,23 +6786,84 @@@ thresholds on the rebuild, supplementa
`internal/monitoring/issue1913_host_continuity_test.go`
(`TestManualLinkToSupplementalGuestHoldsWithAndWithoutContinuity`) pins the
published inventory with no continuity, an unrelated absent machine and the
- linked agent's own saved enrollment.
+ linked agent's own saved enrollment, and in each case that the read state,
+ alert intent and a registry seeded from the read state resolve the agent to
+ the guest without merging the saved agent into it.
+ `resolve_test.go` pins the hold in the read state and the
+ resources-API registry, the fold once the agent reports
+ (`TestContinuityAgentLinkedIntoLiveGuestAnswersToTheGuest`), a saved primary
+ that leaves the live member its own identity
+ (`TestContinuityPrimaryLeavesLiveLinkMemberItsOwnIdentity`), an observation
+ merging into a saved row (`TestObservationMergedIntoASavedRowLetsItsLinkFold`)
+ and holds through later folds, saved primaries and cycles
+ (`TestLinkHoldsFollowTheirPrimaryThroughThePass`); `clone_test.go`
+ (`TestCloneResourceKeepsContinuityMarker`) pins the marker riding clones and
+ resource seeding.
+ `internal/api/resources_link_continuity_test.go`
+ (`TestResourcesAPIAndPatrolAgreeOnContinuityAgentLinkedIntoGuest`) pins the
+ resources API's write and Patrol's read on the same store.
+### Mock-mode unified view applies operator links
+
+In mock mode the websocket broadcast, `/api/state`, and the monitor's
+`GetUnifiedReadStateOrSnapshot` and `UnifiedResourceSnapshot` readers serve a
+view of the fixture graph, not the monitor's resource store, and the fixture
+graph was built without the link store. The resources API seeds its registry
+from that same list through a registry backed by the link store, so a manual
+link showed merged there while the broadcast kept both rows. The fixture graph
+is now built with the links the monitor adapter's current registry generation
+loaded from its store (`MonitorAdapter.ManualLinks`), in a registry that has
+no store at all (`NewRegistryWithManualLinks`,
+`mock.UnifiedResourceSnapshotWithLinks`). Nothing that build ingests can write
+identity pins, canonical-ID successions or change records, and the view's read
+state forwards no history to the operator's store, so fixture data stays out
+of durable state through it.
+
+- The links apply at each ingest stage, as in the live rebuild: the snapshot,
+ then the TrueNAS and vSphere records, then availability. An availability
+ check that names the folded resource by source reference resolves through
+ the fold and projects onto the primary. Folding the finished list instead
+ dropped the check with the folded row. A check that names the folded
+ resource by canonical ID still loses its target, as live (fold history
+ above).
+- A link reaches the view when the adapter's next rebuild loads it, the same
+ point at which the live broadcast picks it up. The view's cache is keyed on
+ the fixture data version and on the links' resource pairs and primaries, so
+ a link change rebuilds it without waiting for a fixture tick. Freshness is
+ assigned in publication order: a view built from the same fixture
+ freshness and links as the one it replaces keeps its freshness, and any
+ other view gets a later one than every view published before it, even when
+ a slower build publishes after a newer one. The resources API reuses its
+ registry while that freshness is unchanged, so it follows the published
+ view and does not keep a list other links folded, for example after a
+ link's primary is reversed. Fixture freshness stands in for the fixture
+ data, as it already did for that cache.
+- Without links the view is built from the shared memoized fixture snapshot,
+ as before. With links each monitor builds its own linked snapshot whenever
+ the fixture data or links change (concurrent cache misses can each build
+ one).
- - Exclusions are not carried, as before. The broadcast's presentation
- coalescing does not consult exclusions in either mode, while the resources
- API's `ListForPresentation` does.
++- Exclusions are not carried, as before: the fixture graph makes its own
++ merges before the view seeds it, so fixtures that identity matching merged
++ stay merged ("Unlink replaces the pair's operator link"). Presentation
++ coalescing still honours them, because the broadcast takes the resource
++ store adapter's exclusions for every store-less view.
+- An agent linked into a guest leaves the agent listings (`Hosts()` lists
+ agent-type rows only) and the guest carries its facet, as in live mode.
+
+`TestMockUnifiedViewAppliesOperatorManualLinks` in
+`internal/monitoring/monitor_host_agents_test.go` pins the broadcast payload,
+the rebuild and freshness advance on a link change with the fixture data
+version held, the cached build afterwards, the same rows as a store-backed
+registry seeded through resource ingest the way the resources API seeds, and
+no change rows reaching the store through the view.
+`TestFixtureGraphAppliesManualLinksAtEachIngestStage` and
+`TestUnifiedResourceSnapshotWithLinksLeavesTheSharedSnapshotUnlinked` in
+`internal/mock/platform_fixtures_test.go` pin the staged availability
+projection and the untouched shared snapshot, and
+`TestRegistryWithManualLinksFoldsLikeTheStoreBackedRegistry` in
+`internal/unifiedresources/registry_test.go` pins the store-less registry
+against the store-backed one and the per-generation `ManualLinks` snapshot.
+
### Provider link network corroboration
An existing reciprocal host/provider link remains authoritative, including a
@@@ -6911,3 -6982,155 +7043,156 @@@ above the hosts table without its filte
vCenter reports no signals. The Overview's own model, workload snapshot and
navigation facets still come from the Overview query; the Health tab keeps
the filterable table.
+
+ ### Unlink replaces the pair's operator link
+
+ `POST /api/resources/{id}/link` records a `ResourceLink`; unlink and
+ report-merge record a `ResourceExclusion`. An operator decides each resource
+ pair one way, and the later decision replaces the earlier: the store's
+ `AddExclusion` deletes the pair's link rows and `AddLink` deletes its
+ exclusion rows, in the same transaction as the write, matching both ID
+ orders because canonical-ID succession can store either. Unlink used to add
+ only the exclusion. The link row survived, and `applyManualLinks`, which
+ never consults exclusions, kept the pair folded in the monitor's rebuilds and
+ in the resources API's registry, so the agent facet stayed on the VM and the
+ agent row stayed hidden although the API answered "Resources unlinked".
+ `GetLinks` and `GetExclusions` return only each pair's latest decision
+ (`effectiveManualPairDecisions` in `manual_link_decisions.go`), so a store
+ that still holds both rows for a pair, written before this change or
+ rewritten onto one pair by succession, reads its later write, and a tie
+ reads as apart. Every registry loads its links and exclusions from its store
+ when it is constructed, so the monitor's rebuild and the resources API's
+ store-backed registry apply one decision with no exclusion check in
+ `applyManualLinks`. Unlink keeps the exclusion, so identity matching and
+ presentation coalescing also keep a pair apart that would otherwise match
+ automatically.
+
+ Identity pins honour the exclusion too. While a Proxmox node was linked to a
+ standalone agent, the merged resource's pin could record the node's endpoint
+ hostname with the agent's machine key. After unlink the node completed that
+ key in `completeIdentityFromPins`, minted the agent's machine-derived ID and
+ folded into the agent by ID, a path no exclusion check sees, so the pair
+ stayed merged with the link row gone. A pin whose canonical ID the operator
+ excluded from the ID the record takes without the pin (its `chooseNewID` or
+ source-specific ID) now completes nothing. This covers pins already written
+ that way, not only new ones. Merges through an agent's own declared node link
+ (`resolveLinkedResource`) still ignore exclusions.
+
+ The resources API seeds its registry from the monitor's listing, which
+ already has the link applied, so it cannot split the pair itself: an unlink
+ shows on REST, `/api/state` and the websocket from the monitor's next
+ registry rebuild, the same generation in which a new link reaches the
+ broadcast. REST applies a new link at once, because its own registry applies
-the store's links on top of the seed. The mock-mode view applies no operator
-links, so in mock mode REST splits a manually linked pair as soon as the
-unlink is recorded. Mock fixtures that identity matching merged stay merged
-on every surface: the fixture graph is built without the store's exclusions.
++the store's links on top of the seed. The mock-mode view applies the links
++the resource store's current generation loaded ("Mock-mode unified view
++applies operator links"), so mock mode follows the same timing. Mock fixtures
++that identity matching merged stay merged on every surface: the fixture graph
++is built without the store's exclusions.
+
+ The websocket broadcast coalesced host views with
+ `CoalescePresentationHostResources`, which honours no exclusions, while the
+ resources API's `ListForPresentation` honours the registry's, so a host pair
+ the operator split could be one broadcast row and two REST rows. The
+ broadcast now coalesces through `Monitor.coalesceResourcesForPresentation`,
+ which asks the read state that listed the rows for
+ `MonitorAdapter.CoalesceForPresentation` when its registry is store-backed
+ (the resource store's adapter, or a host-continuity overlay that loaded the
+ store's exclusions afresh). The mock view and other views built from an
+ already-unified list sit on a store-less registry, which declines, so the
+ resource store's adapter supplies the exclusions instead. Either applies its
+ current registry's exclusions through the same `presentationExclusionFilter`
+ that `ListForPresentation` uses; a rebuild published between listing the
+ rows and coalescing them is judged by the newer generation's exclusions for
+ that one broadcast. Both surfaces share one known limit: the filter compares
+ the IDs of the rows being merged, so a third same-host fragment that merges
+ first can carry an excluded pair into one row.
+
+ A write leaves one row for its pair: `AddLink` and `AddExclusion` also delete
+ a row of their own kind that succession stored in the other order, and the
+ in-memory store replaces a pair's rows on every write and stamps `CreatedAt`
+ as SQLite does. Reads keep the newest link per pair, so an older reversed copy
+ cannot win with its primary, and drop exclusions older than it; duplicate
+ exclusions already stored stay, which changes no decision. A registry resolves its `GetLinks` and `GetExclusions` results
+ together in `loadOverrides`, so an unlink committed between the two reads
+ cannot load both decisions for one generation. Report-merge undoes a link
+ the same way (see "Report-merge splits operator-linked resources" below).
+ `TestManualPairDecisionReplacesThePreviousOne` and
+ `TestManualPairDecisionReadsResolveStoredConflictsByRecency` pin the store
+ rule on SQLite and the in-memory store,
+ `TestUnlinkSplitsHostPairWhosePinTookTheOtherSidesKey` links and unlinks
+ node and agent pairs (IP and FQDN endpoints, either side primary) through
+ three monitor rebuilds per step,
+ `TestRegistryResolvesDecisionsAcrossItsTwoReads` loads a link read before an
+ unlink and the exclusion read after it,
+ `TestResourceUnlinkSplitsManuallyLinkedPair` drives link, unlink and relink
+ through the API handlers against a monitor adapter rebuilt twice per step,
+ and `TestMonitorAdapterCoalesceForPresentationHonoursExclusions`,
+ `TestBroadcastPresentationCoalesceHonoursMergeExclusions` and
+ `TestBroadcastPresentationCoalescePrefersTheListingReadState` pin broadcast
+ and REST row counts for a split host pair.
+
+ ### Report-merge splits operator-linked resources
+
+ `POST /api/resources/{id}/report-merge` records exclusions for the reported
+ sources. For each reported source mapped to the merged resource it excludes
+ the merged resource's ID and the candidate ID that source's record takes when
+ no match is allowed (`ResourceRegistry.SourceTargets`), as before, which
+ keeps an identity match apart. That candidate is derived from the merged
+ resource's type, so for a source an operator link brought in it named
+ neither side of the link: an agent linked into a VM got a `vm-`
+ candidate, and an agent linked with a node or a Docker host keyed by its
+ machine ID got a source-specific one. No link row matched, the link stayed
+ applied, and the API answered "Merge reported" while the drawer's Split
+ dialog toasted "Resource split applied". Two linked agents share their only
+ source, so the API refused them as "Resource is not merged"; an applied link
+ now marks a merge as well (the drawer still offers Split only for a resource
+ listing two sources).
+
+ `applyManualLinks` now records each link it applies on the merged resource
+ (`recordManualLinkFold`): the link row's own pair, the resource that took the
+ other in and the one folded into it, plus the sources the folded side
+ brought, including anything links had already folded into it, so every link
+ along a chain keeps its own pair. A pair is recorded once, since record
+ ingest can recreate a folded side whose source mapping points at a holder of
+ another type and fold it again, so a repeated pair adds its sources to the
+ fold already recorded (the holder still carries what the earlier fold
+ brought). The record is unexported and rides in-memory
+ clones, so it reaches the resources API's registry, which seeds from the
+ monitor's already-linked listing and never holds the folded row.
+ Report-merge excludes the pair of every link whose folded side brought a
+ reported source category (`ResourceRegistry.ManualLinkFolds`), and the
+ store's one-decision-per-pair rule deletes that link row, so the pair splits
+ on every surface from the monitor's next rebuild, as with unlink. The drawer
+ reports every merged source. Categories are not attributed to individual
+ contributors: naming one that some folded side carried undoes that link even
+ where the holder carries it too, and a source that arrived along a chain of
+ links undoes every link on its way to the merged resource.
+
+ A link joins only the pair its row names. Canonical-ID succession re-keys
+ link endpoints with `UPDATE OR IGNORE` but moves `primary_id`
+ unconditionally, so a row whose re-key collided with the successor's own row
+ kept the retired endpoint and took the successor as its primary.
+ `applyManualLinks` honoured that primary and merged the successor through a
+ row no exclusion of the merged pair removes, so an unlink or report-merge of
+ that pair came back on the next rebuild. A primary that names neither side
+ of its row now falls back to the row's first side, which leaves such a row
+ inert while the retired ID is gone.
+
+ `TestResourceReportMergeSplitsOperatorLinkedPair` links and reports through
+ the API handlers against a monitor adapter on the same SQLite store, rebuilt
+ three times per step, for VM/agent, node/agent and agent/Docker pairs linked
+ from either side and for two linked agents. Every two-source case but a node
+ linked from its agent fails with the previous handler, whose candidate for
+ that node happened to be the node's own source-specific ID.
+ `TestResourceReportMergeSplitsPairBehindASuccessionShadowedRow` reports a
+ pair that a shadowed row also named,
+ `TestResourceReportMergeSourceFilterSelectsLinks` pins the source filter,
+ `TestResourceReportMergeSplitsIdentityMergedPair` keeps an identity-merged
+ agent and Docker host apart through the rebuilds,
+ `TestManualLinkFoldsNameEachLinkPairAlongAChain` pins a chain's pairs on the
+ monitor's registry and on one seeded from its listing, and
+ `TestManualLinkFoldsRecordEachPairOnceAcrossRecordIngests` folds a TrueNAS
+ system into a VM across repeated record ingests, and
+ `TestManualLinkFoldRepeatedPairKeepsEarlierSources` refolds a side with fewer
+ sources than it first brought. `TestResourceAPIReportMergeExcludesRegistryLinkFolds`
+ keeps the handler on the registry's fold record.