Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
A page purge removes the page's name from the volunteer log and the mention gazetteer
garrytan16fb44deb26f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c291de9279..c4fb82cf45 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,21 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++### A page purge removes the page's name from the volunteer log and the mention gazetteer++**Purging a page now also deletes the rows that kept its name next to its slug.** Before, `delete_page purge:true` and the retention purge left two stores behind, because neither has a foreign key to `pages`: `context_volunteer_events` kept the slug with a rationale that quotes the matched name (`alias match "<name>"`) for up to 90 days, and `mention_gazetteer_entries` kept the page's normalized title and aliases until that source's next mention pass. The purge receipt called the page gone while both were still readable. Reported and first fixed by @alanshurafa (#9072).++| What you did | Before | Now |+|---|---|---|+| `delete_page purge:true`, the expired-tombstone purge, or a restore that replays a page purge | the slug and name stayed in the volunteer log and the mention gazetteer | removed in the purge transaction, and the page receipt checks both stores |+| `purge_fact` on a claim that is also another page's name or quoted in a volunteer rationale | the gazetteer was not checked | both stores are probed and a match is listed as a residual |++- `src/core/persistence/purge-name-stores.ts` (new): `scrubPurgedNameStores` deletes the source's rows for the purged slugs from both stores. `purgePageInTransaction`, `purgeDeletedPages`, `expireRecordedTombstones` and the restore-time purge replay call it beside the retrieval-feedback scrub. A soft delete keeps both, so `restore_page` still brings the page back whole.+- `src/core/deletion-inventory.ts`: both tables are `swept` (`page_names`); the fact receipt still probes them, since a fact purge sweeps neither.+- Tests: `test/purge-name-stores-9068.test.ts` (PGLite, and Postgres through `test/postgres-unit-arms.txt`).+## [0.60.649.0] - 2026-11-10**Access and privacy fixes.** A member-bound credential now sees its own readable pages on every read surface, the brain owner's remote credentials get private facts on every fact surface, `gbrain sources federate` names the member-written pages it makes readable to everyone, and seven community-reported bugs in access, privacy, purge, repair and secret scanning are fixed.diff --git a/docs/architecture/key-files/memory-purge.md b/docs/architecture/key-files/memory-purge.mdindex 68773256ab..8148c544df 100644--- a/docs/architecture/key-files/memory-purge.md+++ b/docs/architecture/key-files/memory-purge.md@@ -26,6 +26,7 @@ User-facing contract: [expire versus purge](../../guides/memory-boundaries.md#pu- `src/core/persistence/purge-deleted.ts` — `purgeDeletedPagesCoordinated` backs the cycle `purge` phase, the purge job, `purge_deleted_pages` and `gbrain pages purge-deleted`. Unmanaged brains keep the engine bulk delete, after the belief-ledger expiry of recorded sources' tombstones: each is its own transaction and item (ItemRun site `purge.recorded_tombstones`), a run-scoped failure or a systemic run stops the purge, and any other failure lists the tombstone in `blocked` with its classified code, counts it in `failed` (no `error`, so the phase warns) and keeps it out of the bulk delete (`purgeDeletedPages` `excludeIds`); a content failure counts toward the same park as the managed purge, a transient one is retried next run, and parked tombstones are skipped as `purge_parked` unless `retryParked` (the unmanaged preview excludes them too). A managed brain purges each tombstone older than the window (skipping archived sources, which the source lifecycle purges) through `delete_page` with `purge: true` and the internal `retention: true` bound to its revision (retention is not an owner purge; see [belief ledger](belief-ledger.md)). Writer state is read once per run (`writerOnboardingPreflight`): a source whose writer is `recovery_required` submits nothing and its tombstones are listed in `blocked` as `writer_not_active` (counted in `inactive`). Revision races are deferred to the next run; a parked tombstone (`purge-parks.ts`) is skipped as `purge_parked` (counted in `parked`) unless `retryParked`, and any other failure is listed in `blocked` and sets `error`. `selectPurgeCandidates` is the one eligibility selector (preview, consent, `purge_deleted_pages` op, run); #6884: the run submits only while the writer has outstanding-request room (`readOutstandingRoom` in `journal.ts`, at `persistence.capacity.background_pause_pct` of the cap for a background run); the rest, and the rest after a `queue_capacity` refusal, is `blocked` as `write_capacity_deferred` and counted in `capacity_deferred`, never a failure or a park. `purgePhaseStatus`/`purgePhaseNote` shape the cycle purge phase (fail on `error`; warn on `writer_not_active`, parked, capacity-deferred or an unmanaged `failed` tombstone). The engine bulk delete (`engine-sql/pages.ts` `purgeDeletedPages`) also deletes, in its transaction, the purged pages' stagings (by source and slug) and the segment texts only their manifests referenced. Tests: `test/managed-writers-w3.test.ts`, `test/purge-deleted-parks.test.ts`, `test/purge-deleted-capacity-6884.test.ts`, `test/pathological/a49-unmanaged-purge-tombstones.test.ts`.- `src/core/persistence/derived-retirement-hold.ts` — GBRA-161: the purge hold for every derived retirement. `DERIVED_RETIRED_BY` lists the `retired_by` stamps derived writers put on the pages they retire (extract-atoms, managed-reextract, stale-atoms, life-chronicle); `derivedRetirementPurgeHold(alias)` keeps a stamped page out of every purge route (engine bulk purge and its preview, the coordinated selection and delete) until `DERIVED_RETIREMENT_HOLD_DAYS` (30) after `deleted_at` (the database clock, never the `retired_at` text); `retireDerivedPages` versions, soft-deletes and stamps a batch in one attributed transaction (unmanaged extract_atoms). `assertRetirementStampUnchanged` is the page preparer's refusal (`reserved_frontmatter_key`) of a remote write that adds or changes one of `RESERVED_RETIREMENT_KEYS`. The hold length follows `generations.restore_days` (`RESTORE_DAYS_SQL`, read inside the purge statement; 30 by default) and `retireDerivedPagesInTransaction` is the in-transaction form generation adapters retire through. An explicit `gbrain delete --purge` is not held.- `src/core/generations/ledger-scrub.ts` — GBRA-161: purge reaches the generation ledger. A page purge (`page-purge.ts`) nulls the purged item's preimages in `derived_generation_items` and every held proposal in `derived_generations` naming it; a fact purge (`facts/purge.ts`) nulls every preimage, proposal and held context carrying the claim. Both are `swept` (`generation_ledger`) in the deletion inventory, so the verification probes them; `gbrain repair generations` then refuses a scrubbed item with `no_preimage`.+- `src/core/persistence/purge-name-stores.ts` — #9068: `scrubPurgedNameStores(engine, sourceId, slugs)` deletes the source's `context_volunteer_events` rows (whose rationale quotes the matched name) and `mention_gazetteer_entries` rows (`name_norm -> target_slug`) for purged slugs. Every hard delete of a page calls it inside its transaction (`purgePageInTransaction`, `purgeDeletedPages`, `expireRecordedTombstones`, the restore purge replay in `belief/purge-mirror.ts`); a soft delete keeps both. Both tables are `swept` (`page_names`) in the deletion inventory; the page receipt checks them by slug, and the fact receipt probes them (rationale, name_norm) and reports a match as `probed_reported`, because a fact purge sweeps neither.- `src/core/persistence/atom-retirement-hold.ts` — #7013: the one SQL rule for atoms `gbrain repair atoms-restore` restores. `atomRetirementsSql(scope)` is set-based (CTEs over `pages` joined on source and origin key, linear in the atoms in scope) and returns every non-imported atom soft-deleted with `retired_by: managed-reextract` with `origin_live` (a live page origin in its source, or a transcript), `replaced` (a live atom of the same origin carries another stripped source hash), `latest` (its hash is the hash of the origin's latest `managed-reextract` retirement) and `restorable` (all three right); `atomOriginScopeSql` narrows it to one atom's source and origin through the frontmatter GIN index for the apply recheck; `atomRetirementPurgeHold`, which keeps such an atom out of every batch purge selection (engine `purgeDeletedPages` preview and delete, the belief-ledger expiry, `selectPurgeCandidates`) for `ATOM_RESTORE_HOLD_DAYS` (30) after `deleted_at`, next to the #6729 `identityDowngradePurgeHold`. An explicit `delete --purge` is not held. Static strings only, so `engine-sql/pages.ts` imports it.- `src/core/repair/atoms-restore.ts` — `gbrain repair atoms-restore` (#7013), explicit-only and preview-bound: lists atoms an empty managed re-extraction retired (`retired_by: managed-reextract`) that `atomRetirementsSql` (`persistence/atom-retirement-hold.ts`) marks `restorable`, each with origin, stripped source hash and `retired_at`, and residuals `restorable`, `origins`, `origin_gone`, `replaced`, `older_generation`; `--apply --expect <hash>` restores the approved set, rechecking each atom (id, incarnation, revision, origin, hash, `retired_at`, rule match) and reporting `changed_since_preview` otherwise. Managed: one `managed_maintenance_restore_reextract_atom` request per atom (`deleted_page: true`, bound to the deleted revision, request id in the outcome detail); `prepareAtomRestore` (dispatched from `prepared-maintenance.ts`) rechecks the rule under the atom and origin page locks and publishes a revision-bound `revert_version` to `version_id`, the newest live version of the atom at its source hash without `retired_by` (the one the coordinated delete archived; file written back, `allowMissingFile`); none is `no_version`, lowered to the `derive:atoms` tier of its origin with taint edges. Unmanaged: `createVersion`, `restorePage` and the two keys dropped in one `maintenanceTransaction`. Extraction state is not touched. Test: `test/pathological/h5-atoms-restore.test.ts`.- `src/core/repair/atom-fence-rows.ts` — `gbrain repair atom-fence-rows`, explicit-only and preview-bound: `findAtomFenceRows` lists live page-bound `visibility: world` atoms whose title, body or `source_quote` contains (case and whitespace folded, claims of at least `MIN_CLAIM_CHARS`) the claim of a facts row of their live source page whose visibility is not world or a takes row whose holder is not world, absent from the page's `publicBody`; keyset pages of 500 atoms whose source page names a facts or takes fence, each source page read once, segmented source pages counted `origin_unread`. The preview lists atom, source page and row kinds and numbers (never row text) under a saved hash; `--apply --expect <hash>` rechecks each atom and sets `visibility: private` through a revision-bound put_page (`narrowed`), else `changed_since_preview`. Doctor `atom_fence_rows` shares the scan. Test: `test/repair-atom-fence-rows.test.ts` (unmanaged and managed, PGLite and Postgres).diff --git a/src/core/belief/purge-mirror.ts b/src/core/belief/purge-mirror.tsindex d9787c0219..b0c8233473 100644--- a/src/core/belief/purge-mirror.ts+++ b/src/core/belief/purge-mirror.ts@@ -147,6 +147,11 @@ async function replayOne(tx: BrainEngine, intent: PurgeMirrorIntent, ledger: booconst { scrubPurgedSlugs } = await import('../feedback/store.ts');await scrubPurgedSlugs(tx, src, [restored.slug]);}+ const [names] = await tx.executeRaw<{ present: boolean }>("SELECT to_regclass('mention_gazetteer_entries') IS NOT NULL AS present");+ if (names?.present) {+ const { scrubPurgedNameStores } = await import('../persistence/purge-name-stores.ts');+ await scrubPurgedNameStores(tx, src, [restored.slug]);+ }};}} else {diff --git a/src/core/deletion-inventory.ts b/src/core/deletion-inventory.tsindex 8aa315248b..f1b2a08642 100644--- a/src/core/deletion-inventory.ts+++ b/src/core/deletion-inventory.ts@@ -60,7 +60,7 @@ const ROWS: readonly Row[] = [['code_traversal_cache', 'out_of_scope', 'no_memory_text', 'symbol_qualified source_id response_json'],['config', 'out_of_scope', 'no_memory_text', 'key value'],['content_chunks', 'swept', 'chunks', 'chunk_text chunk_source embedding model embedded_text_hash embedding_input_hash language symbol_name symbol_type modality embedding_image embedding_multimodal parent_symbol_path doc_comment symbol_name_qualified search_vector'],- ['context_volunteer_events', 'out_of_scope', 'probed_reported', 'source_id slug match_arm rationale channel session_id', 'rationale'],+ ['context_volunteer_events', 'swept', 'page_names', 'source_id slug match_arm rationale channel session_id'],['conversation_parser_llm_cache', 'out_of_scope', 'probed_reported', 'content_sha256 model_id call_shape value_json', 'value_json'],['core_edit_notices', 'swept', 'core_notice_base', 'source_id slug revision base_revision base_text actor'],['decide_calibrations', 'out_of_scope', 'no_memory_text', 'slot call_site provider model_resolved metric qualification policy_fingerprint dataset_hash split_hash calibrate_ids_hash pack_shape notes'],@@ -105,7 +105,7 @@ const ROWS: readonly Row[] = [['mcp_request_log', 'out_of_scope', 'probed_reported', 'token_name agent_name operation status params error_message', 'params error_message'],['mcp_spend_log', 'out_of_scope', 'no_memory_text', 'client_id token_name operation provider model budget_key'],['mcp_spend_reservations', 'out_of_scope', 'no_memory_text', 'client_id model provider status usage_unknown_reason budget_key'],- ['mention_gazetteer_entries', 'out_of_scope', 'no_memory_text', 'source_id name_norm target_slug'],+ ['mention_gazetteer_entries', 'swept', 'page_names', 'source_id name_norm target_slug'],['mention_index_status', 'out_of_scope', 'no_memory_text', 'source_id state policy_fingerprint error'],['migration_impact_log', 'out_of_scope', 'probed_reported', 'remediation_id metric_name source_id brain_id idempotency_key applied_by details', 'details'],['minion_attachments', 'out_of_scope', 'probed_reported', 'filename content_type content storage_uri sha256', 'content'],diff --git a/src/core/engine-sql/pages.ts b/src/core/engine-sql/pages.tsindex 6e7301c9f9..29b2343ccb 100644--- a/src/core/engine-sql/pages.ts+++ b/src/core/engine-sql/pages.ts@@ -41,6 +41,7 @@ import { frontmatterWhereFragment } from '../search/frontmatter-where.ts';import { quoteIdentifier, resolveWriteColumnFromConfigRows } from '../search/embedding-column.ts';import { pickCanonicalOrigin } from '../persistence/origin-guard.ts';import { scrubPurgedSlugs } from '../feedback/store.ts';+import { scrubPurgedNameStores } from '../persistence/purge-name-stores.ts';import { expireGonePageFacts } from '../facts/gone-page-facts.ts';/**@@ -365,7 +366,10 @@ export async function purgeDeletedPages(AND NOT ${trustedSql(stagingNeedsTextSql('t'))}`);const bySource = new Map<string, string[]>();for (const r of rows) bySource.set(r.source_id, [...bySource.get(r.source_id) ?? [], r.slug]);- for (const [sourceId, purged] of bySource) await scrubPurgedSlugs(tx, sourceId, purged);+ for (const [sourceId, purged] of bySource) {+ await scrubPurgedSlugs(tx, sourceId, purged);+ await scrubPurgedNameStores(tx, sourceId, purged);+ }const slugs = rows.map((r) => r.slug);return { slugs, count: slugs.length };});diff --git a/src/core/facts/purge-verify.ts b/src/core/facts/purge-verify.tsindex 82c7a69931..ac2291d35f 100644--- a/src/core/facts/purge-verify.ts+++ b/src/core/facts/purge-verify.ts@@ -23,7 +23,7 @@ import { getCode } from '../retry-matcher.ts';import { CLAIM_FORMS_BATCH, claimTextForms, escapeFenceCell } from '../fence-shared.ts';import { gbrainPath } from '../config.ts';import type { WriteRequest } from '../persistence/model.ts';-import { DELETION_INVENTORY, OUT_OF_REACH_RESIDUALS, type InventoryEntry } from '../deletion-inventory.ts';+import { DELETION_INVENTORY, DELETION_INVENTORY_BY_TABLE, OUT_OF_REACH_RESIDUALS, type InventoryEntry } from '../deletion-inventory.ts';import type { PurgePlan, PurgeTarget } from './purge.ts';import { gitChildEnv } from '../git-env.ts';import { segmentedBodyContains } from '../page-body/segment-rewrite.ts';@@ -541,6 +541,10 @@ export async function verifyFactPurge(engine: BrainEngine, input: { target: Purgstores.push(...await ledgerReports(engine, src, [claim], input.ledger, deadline));stores.push(await effectPayloadReport(engine, src, [...new Set([claim, canonicalChangeReason(claim) ?? claim])], deadline, removed.persistence_effects ?? 0));}+ // #9068: only a page purge sweeps the name stores, so a row there holding the claim is another page's name or volunteer pointer.+ for (const [table, probe] of [['context_volunteer_events', ['rationale']], ['mention_gazetteer_entries', ['name_norm']]] as const) {+ stores.push(await probeInventoryStore(engine, { ...DELETION_INVENTORY_BY_TABLE.get(table)!, probe }, src, [claim], deadline));+ }stores.push(...await sweptInventoryReports(engine, new Set(stores.map(s => s.store)), src, [claim], removed, deadline, dryRun));for (const entry of DELETION_INVENTORY) {if (entry.class !== 'out_of_scope' || entry.reason !== 'probed_reported') continue;@@ -621,6 +625,8 @@ export async function verifyPagePurge(engine: BrainEngine, input: { sourceId: stAND ((element_kind='page' AND element_key=$2) OR (element_kind='link' AND ${edgeNames('element_key')}))`, [src, slug]);await keyed('eval_candidates.retrieved_slugs', `SELECT 1 FROM eval_candidates WHERE $1 = ANY(source_ids) AND $2 = ANY(retrieved_slugs)AND NOT EXISTS (SELECT 1 FROM pages po WHERE po.slug = $2 AND po.source_id = ANY(eval_candidates.source_ids) AND po.source_id <> $1)`, [src, slug]);+ await keyed('context_volunteer_events', 'SELECT 1 FROM context_volunteer_events WHERE source_id=$1 AND slug=$2', [src, slug]);+ await keyed('mention_gazetteer_entries', 'SELECT 1 FROM mention_gazetteer_entries WHERE source_id=$1 AND target_slug=$2', [src, slug]);await keyed('files', 'SELECT 1 FROM files WHERE page_id IS NULL AND page_slug=$2 AND (source_id=$1 OR source_id IS NULL)', [src, slug]);await keyed('write_gate_holds', 'SELECT 1 FROM write_gate_holds WHERE source_id=$1 AND slug=$2', [src, slug]);if (!await relationExists(engine, 'query_cache')) stores.push(missingCache('query_cache'));diff --git a/src/core/persistence/page-purge.ts b/src/core/persistence/page-purge.tsindex bbc01add05..3131503219 100644--- a/src/core/persistence/page-purge.ts+++ b/src/core/persistence/page-purge.ts@@ -61,11 +61,12 @@ import { assertHeldPageKeys } from '../page-state/guards.ts';import { noteBelief } from '../belief/ledger.ts';import { appendPurgeIntent } from '../belief/purge-mirror.ts';import { scrubPurgedSlugs } from '../feedback/store.ts';+import { scrubPurgedNameStores } from './purge-name-stores.ts';import { appendDerivedHidden, bumpPurgeEpoch, derivedBeliefTargets, derivedStreamSlugs, pageLifetimesOf, pagePurgeClaims, redactCarryingFields, redactPurgedKeys, registerLiveCarriers,sourceHasBeliefEvents, type LedgerPurgeReport } from '../belief/purge.ts';export const PURGE_RESIDUALS = 'Removed from live stores: the page row with its chunks, versions, takes, timeline, links and raw data, the facts filed on it (tombstoned), '- + 'its take proposals, open loops and core notices, the retrieval feedback and captured eval results naming the slug, stored write intents for the slug and its attached file records. Still out of reach: the brain repository\'s '+ + 'its take proposals, open loops and core notices, the retrieval feedback and captured eval results naming the slug, the volunteer log rows and mention-gazetteer names for it, stored write intents for the slug and its attached file records. Still out of reach: the brain repository\'s '+ 'git history and every other clone or remote, Markdown exports and compiled context files, backups, provider copies (embeddings, decisions), and deleted rows '+ 'in database pages and the write-ahead log until vacuum. Rotate any exposed credential.';@@ -371,6 +372,7 @@ export async function purgePageInTransaction(tx: BrainEngine, row: WriteRequest,const { shas, ...segmentRows } = await deletePageWithSegments(tx, sourceId, slug, page.id, segmentClaims);Object.assign(removed, segmentRows);Object.assign(removed, await scrubPurgedSlugs(tx, sourceId, [slug]));+ Object.assign(removed, await scrubPurgedNameStores(tx, sourceId, [slug]));if (!retention) appendPurgeIntent({ request: row.id, request_id: row.request_id, source_id: sourceId, source_incarnation: row.source_incarnation,scope: 'page', slug, purge_epoch: epoch, content_hashes: [...hashes], facts: mirrorFacts, takes: mirrorTakes, lifetimes });const key = `${sourceId}:${row.request_id}`;diff --git a/src/core/persistence/purge-deleted.ts b/src/core/persistence/purge-deleted.tsindex 431c9b63cb..0676d5db26 100644--- a/src/core/persistence/purge-deleted.ts+++ b/src/core/persistence/purge-deleted.ts@@ -19,6 +19,7 @@ import { clearPurgePark, isParked, notePurgeSourceWriter, PURGE_PARK_LIMIT, purgimport { ItemRun, type ItemFailure } from '../item-failure.ts';import { readUnreviewedSyncTombstones, syncDeleteHeld } from './sync-delete-tombstones.ts';import { scrubPurgedSlugs } from '../feedback/store.ts';+import { scrubPurgedNameStores } from './purge-name-stores.ts';import { expireGonePageFacts } from '../facts/gone-page-facts.ts';import { maintenanceTransaction } from './attribution.ts';@@ -137,7 +138,10 @@ async function expireRecordedTombstones(engine: BrainEngine, olderThanHours: numconst gone = await tx.executeRaw<{ source_id: string; slug: string }>(`DELETE FROM pages WHERE id = $1 AND deleted_at IS NOT NULL AND deleted_at < now() - ($2 || ' hours')::interval AND ${identityDowngradePurgeHold('pages')} AND ${atomRetirementPurgeHold('pages')} AND ${derivedRetirementPurgeHold('pages')} RETURNING source_id, slug`,[row.id, hours]);await expireGonePageFacts(tx, gone);- if (gone.length > 0) await scrubPurgedSlugs(tx, row.source_id, [row.slug]);+ if (gone.length > 0) {+ await scrubPurgedSlugs(tx, row.source_id, [row.slug]);+ await scrubPurgedNameStores(tx, row.source_id, [row.slug]);+ }return gone.length > 0;});});diff --git a/src/core/persistence/purge-name-stores.ts b/src/core/persistence/purge-name-stores.tsnew file mode 100644index 0000000000..a362adf45e--- /dev/null+++ b/src/core/persistence/purge-name-stores.ts@@ -0,0 +1,21 @@+import type { BrainEngine } from '../engine.ts';++/**+ * #9068: the two stores that keep a page's name next to its slug. context_volunteer_events logs each time the page+ * was volunteered, with a rationale that quotes the matched name; mention_gazetteer_entries keeps the page's+ * normalized title and aliases for the mention pass. Neither has a foreign key to pages, so a purge's cascade never+ * reaches them: every hard delete of a page calls this inside its transaction. A soft delete keeps both, because+ * restore_page can bring the page back. Returns rows removed per store, keyed as the purge receipt names them.+ */+export async function scrubPurgedNameStores(+ engine: Pick<BrainEngine, 'executeRaw'>,+ sourceId: string,+ slugs: readonly string[],+): Promise<Record<string, number>> {+ if (!slugs.length) return {};+ const count = async (sql: string) => (await engine.executeRaw(sql, [sourceId, [...slugs]])).length;+ return {+ context_volunteer_events: await count('DELETE FROM context_volunteer_events WHERE source_id = $1 AND slug = ANY($2::text[]) RETURNING 1'),+ mention_gazetteer_entries: await count('DELETE FROM mention_gazetteer_entries WHERE source_id = $1 AND target_slug = ANY($2::text[]) RETURNING 1'),+ };+}diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 61552617f5..ad9b93adb6 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -468,6 +468,7 @@ test/purge-expires-page-facts.test.tstest/purge-fact.test.tstest/purge-global-marker.test.tstest/purge-missing-cache-6911.test.ts+test/purge-name-stores-9068.test.tstest/purge-read-surfaces.test.tstest/purge-sync-tombstone-hold-7130.test.tstest/qm-memory-tools.test.tsdiff --git a/test/purge-name-stores-9068.test.ts b/test/purge-name-stores-9068.test.tsnew file mode 100644index 0000000000..c164a80214--- /dev/null+++ b/test/purge-name-stores-9068.test.ts@@ -0,0 +1,191 @@+/**+ * #9068: purging a page removes its slug and its name from the volunteer log and the mention gazetteer.+ *+ * 1. Protects: delete_page purge:true, the expired-tombstone purge, the belief ledger's per-tombstone expiry and a+ * restore that replays a page purge delete the source's context_volunteer_events rows for the slug (each rationale+ * quotes the matched name) and its mention_gazetteer_entries rows (name_norm -> slug), in the purge transaction,+ * and the page receipt checks both stores. A fact purge sweeps neither, so a row there that holds the claim is+ * reported as a residual, never as an incomplete sweep.+ * 2. Fails when: either store keeps the purged slug or name, or the scrub reaches another source's or another+ * page's rows.+ * 3. Existing coverage: purge-retrieval-feedback-8821 pins the retrieval stores; nothing pinned these two.+ * 4. Real seams: PGLite, and Postgres when DATABASE_URL is set (test/postgres-unit-arms.txt); the belief-ledger and restore cases are PGLite only; the volunteer log's own writer, the delete_page and+ * purge_fact ops, purgeDeletedPagesCoordinated, a PGLite backup and restore; repo-less brain, no provider calls.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { mkdtempSync, rmSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { operations } from '../src/core/operations.ts';+import { importFromContent } from '../src/core/import-file.ts';+import { runExtractFacts } from '../src/core/cycle/extract-facts.ts';+import { registerLocalWriter } from '../src/core/persistence/identity.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { purgeDeletedPagesCoordinated } from '../src/core/persistence/purge-deleted.ts';+import { insertVolunteerEvents } from '../src/core/context/volunteer-events.ts';+import { DELETION_INVENTORY_BY_TABLE } from '../src/core/deletion-inventory.ts';+import { createPgliteBackup, restorePgliteBackup } from '../src/core/backup/snapshot.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { beliefBrain, beliefPage, openFileBeliefBrain, withFileBeliefBrain } from './helpers/belief-ledger.ts';+import { withEnv } from './helpers/with-env.ts';+import { testBackends } from './helpers/test-backends.ts';++const home = mkdtempSync(join(tmpdir(), 'gbrain-purge-names-'));+const engines: BrainEngine[] = [];+let closePostgres: (() => Promise<void>) | undefined;+const OTHER = 'companies/acme-example';+const OTHER_NAME = 'Acme Example';++const ctx = (engine: BrainEngine): OperationContext => ({ engine, sourceId: 'default', remote: false, dryRun: false,+ config: { engine: engine.kind, embedding_disabled: true, storage: { backend: 'local', bucket: 'b', localPath: join(home, 'storage') } },+ logger: { info() {}, warn() {}, error() {} } } as OperationContext);+const op = (name: string) => operations.find(o => o.name === name)!;++/** One page's two name rows: a volunteer pointer whose rationale quotes the name, and a gazetteer entry for it. */+async function nameRows(engine: BrainEngine, sourceId: string, slug: string, name: string) {+ await insertVolunteerEvents(engine, [{ source_id: sourceId, slug, confidence: 0.9, match_arm: 'alias', rationale: `alias match "${name}"`, channel: 'op' }]);+ await engine.executeRaw('INSERT INTO mention_gazetteer_entries (source_id, name_norm, target_slug) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING', [sourceId, name.toLowerCase(), slug]);+}++/** The page in two sources and another page in default, each with its name rows. */+async function seed(engine: BrainEngine, slug: string, name: string) {+ await engine.executeRaw(`INSERT INTO sources (id, name) VALUES ('source-b', 'source-b') ON CONFLICT DO NOTHING`);+ for (const sourceId of ['default', 'source-b']) {+ await importFromContent(engine, slug, `---\ntitle: ${name}\ntype: person\n---\n# ${name}\n\nBody text.\n`, { noEmbed: true, sourceId });+ await nameRows(engine, sourceId, slug, name);+ }+ if (!(await engine.executeRaw('SELECT 1 FROM pages WHERE source_id = $1 AND slug = $2', ['default', OTHER])).length) {+ await importFromContent(engine, OTHER, `---\ntitle: ${OTHER_NAME}\ntype: company\n---\n# ${OTHER_NAME}\n\nBody text.\n`, { noEmbed: true, sourceId: 'default' });+ await nameRows(engine, 'default', OTHER, OTHER_NAME);+ }+}++/** The sources whose rows still name the page, by slug or by name, per store. */+async function residue(engine: BrainEngine, slug: string, name: string) {+ const sources = async (sql: string) => (await engine.executeRaw<{ source_id: string }>(sql, [slug, name.toLowerCase()])).map(r => r.source_id);+ return {+ context_volunteer_events: await sources('SELECT source_id FROM context_volunteer_events WHERE slug = $1 OR strpos(lower(rationale), $2) > 0 ORDER BY 1'),+ mention_gazetteer_entries: await sources('SELECT source_id FROM mention_gazetteer_entries WHERE target_slug = $1 OR name_norm = $2 ORDER BY 1'),+ };+}+const ONLY_SOURCE_B = { context_volunteer_events: ['source-b'], mention_gazetteer_entries: ['source-b'] };+const SEEDED = { context_volunteer_events: ['default', 'source-b'], mention_gazetteer_entries: ['default', 'source-b'] };++async function otherUntouched(engine: BrainEngine) {+ expect(await engine.executeRaw(`SELECT 1 FROM context_volunteer_events WHERE source_id = 'default' AND slug = $1`, [OTHER])).toHaveLength(1);+ expect(await engine.executeRaw(`SELECT 1 FROM mention_gazetteer_entries WHERE source_id = 'default' AND target_slug = $1`, [OTHER])).toHaveLength(1);+}++beforeAll(async () => withEnv({ GBRAIN_HOME: home }, async () => {+ const backends = testBackends();+ if (backends.includes('pglite')) { const lite = new PGLiteEngine(); await lite.connect({}); await lite.initSchema(); engines.push(lite); }+ if (backends.includes('postgres')) {+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);+ engines.push(pg.engine); closePostgres = pg.close;+ }+ for (const engine of engines) await registerLocalWriter(engine, 'cli');+}), 120_000);+afterAll(async () => {+ for (const engine of engines) await disposePersistenceConsumer(engine);+ await engines.find(e => e.kind === 'pglite')?.disconnect(); await closePostgres?.(); rmSync(home, { recursive: true, force: true });+}, 60_000);++describe('#9068 a page purge removes the slug and name from the volunteer log and the mention gazetteer', () => {+ test('both stores are swept page-name stores in the deletion inventory', () => {+ for (const table of ['context_volunteer_events', 'mention_gazetteer_entries']) expect(DELETION_INVENTORY_BY_TABLE.get(table)?.class).toBe('swept');+ });++ test('delete_page purge:true scrubs the source\'s rows and the receipt checks both stores', () => withEnv({ GBRAIN_HOME: home }, async () => {+ for (const engine of engines) {+ const slug = 'people/alice-example', name = 'Alice Example';+ await seed(engine, slug, name);+ expect(await residue(engine, slug, name)).toEqual(SEEDED);+ const snapshot = await engine.readPageSnapshot(slug, { sourceId: 'default' });+ const result = await op('delete_page').handler(ctx(engine), { slug, purge: true, request_id: randomUUID(), expected_revision: snapshot!.revision }) as Record<string, any>;+ expect(result.status).toBe('purged');+ expect({ engine: engine.kind, residue: await residue(engine, slug, name) }).toEqual({ engine: engine.kind, residue: ONLY_SOURCE_B });+ await otherUntouched(engine);+ const stores = new Map((result.receipt.stores as Array<{ store: string; status: string; removed?: number }>).map(s => [s.store, s]));+ for (const store of ['context_volunteer_events', 'mention_gazetteer_entries']) expect(stores.get(store)).toMatchObject({ status: 'deleted', removed: 1 });+ }+ }), 120_000);++ test('the expired-tombstone purge scrubs the same rows', () => withEnv({ GBRAIN_HOME: home }, async () => {+ for (const engine of engines) {+ const slug = 'people/bob-sample', name = 'Bob Sample';+ await seed(engine, slug, name);+ await engine.softDeletePage(slug, { sourceId: 'default' });+ expect(await residue(engine, slug, name)).toEqual(SEEDED);+ const purged = await purgeDeletedPagesCoordinated(engine, 0);+ expect(purged.slugs).toContain(slug);+ expect({ engine: engine.kind, residue: await residue(engine, slug, name) }).toEqual({ engine: engine.kind, residue: ONLY_SOURCE_B });+ await otherUntouched(engine);+ }+ }), 120_000);++ test('a fact purge reports a name row that holds the claim instead of calling the sweep incomplete', () => withEnv({ GBRAIN_HOME: home }, async () => {+ const claim = 'Door code is 4417';+ for (const engine of engines) {+ const slug = 'people/carol-example';+ await importFromContent(engine, slug, `---\ntitle: Carol Example\ntype: person\n---\n# Carol Example\n\n## Facts\n\n<!--- gbrain:facts:begin -->+| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context |+|---|-------|------|------------|------------|------------|------------|-------------|--------|---------|+| 1 | ${claim} | fact | 1.0 | world | medium | 2026-01-01 | | chat | |+<!--- gbrain:facts:end -->\n`, { noEmbed: true, sourceId: 'default' });+ await runExtractFacts(engine, { slugs: [slug] });+ const [fact] = await engine.executeRaw<{ id: number }>('SELECT id FROM facts WHERE fact = $1 AND entity_slug = $2', [claim, slug]);+ await nameRows(engine, 'default', 'notes/door-code-example', claim);+ const purge = (params: Record<string, unknown>) => op('purge_fact').handler(ctx(engine), params) as Promise<Record<string, any>>;+ const dry = await purge({ id: Number(fact.id), dry_run: true });+ const result = await purge({ id: Number(fact.id), confirm: dry.confirm_token, expected_revision: dry.expected_revision, request_id: randomUUID() });+ const stores = new Map((result.stores as Array<{ store: string; status: string; reason?: string }>).map(s => [s.store, s]));+ for (const store of ['context_volunteer_events', 'mention_gazetteer_entries']) {+ expect({ engine: engine.kind, store, report: stores.get(store) }).toMatchObject({ engine: engine.kind, store, report: { status: 'out_of_scope', reason: 'probed_reported', remaining: 1 } });+ }+ expect(await engine.executeRaw('SELECT 1 FROM mention_gazetteer_entries WHERE target_slug = $1', ['notes/door-code-example'])).toHaveLength(1);+ }+ }), 120_000);+});++const pgliteTest = testBackends().includes('pglite') ? test : test.skip;++pgliteTest('#9068 a recorded tombstone the belief ledger expires scrubs the same rows', () => beliefBrain(async b => {+ const slug = 'people/alice-example', name = 'Alice Example';+ await b.put(slug, beliefPage({ title: name, prose: 'Body of the page.' }));+ await b.put(OTHER, beliefPage({ title: OTHER_NAME, prose: 'Body of the other page.' }));+ await nameRows(b.engine, b.sourceId, slug, name);+ await nameRows(b.engine, b.sourceId, OTHER, OTHER_NAME);+ await b.engine.executeRaw(`UPDATE pages SET deleted_at = now() - interval '100 hours' WHERE source_id = $1 AND slug = $2`, [b.sourceId, slug]);+ const purged = await purgeDeletedPagesCoordinated(b.engine, 72);+ expect(purged.slugs).toContain(slug);+ expect(await residue(b.engine, slug, name)).toEqual({ context_volunteer_events: [], mention_gazetteer_entries: [] });+ expect(await b.engine.executeRaw('SELECT 1 FROM mention_gazetteer_entries WHERE source_id = $1 AND target_slug = $2', [b.sourceId, OTHER])).toHaveLength(1);+}), 120_000);++pgliteTest('#9068 a restore that replays a page purge scrubs the same rows from the restored brain', () => withFileBeliefBrain('restore-9068', async fb => {+ const src = 'restore-9068', slug = 'people/alice-example', name = 'Alice Example';+ await fb.open(async b => {+ await b.put(slug, beliefPage({ title: name, prose: 'Body of the page.' }));+ await b.put(OTHER, beliefPage({ title: OTHER_NAME, prose: 'Body of the other page.' }));+ await nameRows(b.engine, src, slug, name);+ await nameRows(b.engine, src, OTHER, OTHER_NAME);+ });+ const archive = join(fb.home, 'archive.gbrain-backup');+ await createPgliteBackup({ root: fb.home, output: archive });+ await fb.open(async b => {+ const snapshot = await b.engine.readPageSnapshot(slug, { sourceId: src });+ await op('delete_page').handler(b.ctx, { slug, purge: true, expected_revision: snapshot!.revision, request_id: randomUUID() });+ });+ const into = join(fb.home, 'restored');+ const result = await restorePgliteBackup({ archive, into });+ expect(result.purge_replay.replays.map(x => x.scope)).toEqual(['page']);+ const { brain, close } = await openFileBeliefBrain(join(into, '.gbrain', 'brain.pglite'), src, join(into, 'content'));+ try {+ expect(await residue(brain.engine, slug, name)).toEqual({ context_volunteer_events: [], mention_gazetteer_entries: [] });+ expect(await brain.engine.executeRaw('SELECT 1 FROM mention_gazetteer_entries WHERE source_id = $1 AND target_slug = $2', [src, OTHER])).toHaveLength(1);+ } finally { await close(); }+}), 300_000);