Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Find and retire facts that older page purges left active (#8859) (#9407)
Garry Tan34b7281fc10a
diff --git a/changelog.d/capy-purge-leftover-facts.md b/changelog.d/capy-purge-leftover-facts.mdnew file mode 100644index 0000000000..f9f819c893--- /dev/null+++ b/changelog.d/capy-purge-leftover-facts.md@@ -0,0 +1,18 @@+---+important: data-loss+---+**`gbrain doctor` finds the facts that purges left behind before #8859 was fixed, and `gbrain repair purged-page-facts` retires them after you approve the preview.**++### Fixed++- **Facts left active by older purges can be found and retired** (#8859 follow-up). Before the #8859 fix, purging a page left the facts extracted from it and filed on no page active, so recall kept returning claims from a removed page. The new doctor check `purged_page_facts` counts them, and `gbrain repair purged-page-facts` previews each one with its class and expires exactly the previewed rows on `--apply --expect <hash> --yes`. Rows of a page the purge ledger records are retired by default; rows of a page gone with no purge record need `--include-ambiguous`. Nothing changes on upgrade.+- **Purges match those facts through an index.** A new index on the head of a fact's context (migration v315) keeps a purge from scanning every fact in its source. The doctor check and the repair still read every page-extraction fact filed on no page.+- **The repair expires rows but writes no purge record**, so the same claim could be saved again later.++### Itemized changes++- `src/core/repair/purged-page-facts.ts`, `src/commands/doctor/checks/purged-page-facts.ts`: the repair kind (explicit-only, preview-bound, destructive consent) and the read-only check.+- `src/core/facts/gone-page-facts.ts`: the context match uses `contextHeadSql`, the expression v315 indexes.+- `src/core/schema-migrations/v315-facts-context-head-index.ts`: `idx_facts_context_head`, built concurrently on Postgres.+- Docs: `docs/guides/repair.md#purged-page-facts`.+- Tests: new `test/repair-purged-page-facts-8859.test.ts` (managed PGLite and Postgres, unmanaged PGLite).diff --git a/docs/architecture/canonical-writers.tsv b/docs/architecture/canonical-writers.tsvindex d12c32f127..9dd4c65c83 100644--- a/docs/architecture/canonical-writers.tsv+++ b/docs/architecture/canonical-writers.tsv@@ -250,6 +250,7 @@ src/core/repair/consolidate-ties.ts 1 coordinator `gbrain repair consolidate-tiesrc/core/repair/duplicate-facts.ts 2 coordinator `gbrain repair duplicate-facts` (#7043) supersedes preview-approved copies of one active fact (expired_at and superseded_by pointing at the kept row: the copy that is a row of its page's facts fence, else the oldest). On a managed brain each UPDATE runs only inside coordinatedDatabaseWrite (the relink_facts operation: write authority checked, source incarnation revalidated, the entity keys locked); an unmanaged brain runs one maintenanceTransaction per group. A fenced copy is first struck in its page through a revision-bound put_page (submitPageMutation), and its superseded_by link is written after that page write. It rechecks every row against the preview and never deletes a row.src/core/repair/fact-claim-dates.ts 1 coordinator `gbrain repair fact-claim-dates` (#7247) moves preview-approved facts' valid_from to the date their claim states. Fence rows change only in the page through revision-bound put_page (the coordinator re-projects the facts row from the canonical text); the one direct UPDATE sets valid_from of database-only facts in one maintenanceTransaction on an unmanaged brain, rechecking each row's previewed valid_from, and is never reached on a managed brain.src/core/repair/conversation-labels.ts 4 coordinator `gbrain repair conversation-labels` expires preview-approved label-misattributed conversation facts, expires the page completion marker and writes the not-extractable outcome, under the page key: one database-only managed_maintenance_conversation_label_retire request per batch of up to 25 pages on a managed brain (the preparer reclassifies each page under its lock), one maintenanceTransaction per batch on an unmanaged brain.+src/core/repair/purged-page-facts.ts 1 coordinator `gbrain repair purged-page-facts` expires preview-approved facts that purges before the #8859 fix left active (page-extraction rows filed on no page whose page is gone), under the gone pages' keys: one database-only managed_maintenance_expire_purged_page_facts request per batch of up to 200 facts on a managed brain (the preparer reclassifies each fact under the keys), one maintenanceTransaction per batch on an unmanaged brain. Rows are expired, never deleted.src/core/persistence/atom-maintenance.ts 1 coordinator A managed atom retirement stamps retired_by/retired_at on the tombstone inside the coordinator publication of its managed_atom_delete request, after the page deletion path removed the canonical file, so the database row and file change commit or roll back together.src/core/repair/stale-atoms.ts 2 coordinator gbrain repair stale-atoms retires an atom on a managed brain through a managed_maintenance_retire_stale_atoms coordinated request (the marker stamp runs in its publication); only an unmanaged brain soft-deletes and stamps in a direct page-locked transaction, where no canonical writer enforcement applies.src/core/repair/file-drift.ts 1 coordinator `gbrain repair file-drift --provenance never_committed --prefer database` (GBRA-133) keeps a held live page whose file no commit ever held as a database-only page: the UPDATE clearing source_path and source_uri runs only inside the managed_maintenance_file_drift_detach publication (submitDatabaseMaintenanceIntent, owner host only), whose preparer re-checks the page id, revision, recorded path and git provenance and re-locks the live page; it clears the resolved hold in the same transaction and writes no file. An unmanaged brain refuses with owner_unavailable.diff --git a/docs/architecture/system-of-record.md b/docs/architecture/system-of-record.mdindex 6c887a442a..60307f3e5f 100644--- a/docs/architecture/system-of-record.md+++ b/docs/architecture/system-of-record.md@@ -314,6 +314,7 @@ or a physical projection that leaves attribution untouched):- `src/core/repair/consolidate-ties.ts` (1): `gbrain repair consolidate-ties` reopens facts inside `coordinatedDatabaseWrite` on managed brains (the `relink_facts` operation) and in one `maintenanceTransaction` per fact on unmanaged brains.- `src/core/repair/duplicate-facts.ts` (2): `gbrain repair duplicate-facts` supersedes duplicate fact copies onto the kept row (the fenced copy, else the oldest) inside `coordinatedDatabaseWrite` on managed brains (the `relink_facts` operation) and in one `maintenanceTransaction` per group on unmanaged brains; a fenced copy is struck through a revision-bound `put_page` first.- `src/core/repair/conversation-labels.ts` (4): a request on managed brains; the unmanaged label retirement runs in `maintenanceTransaction`.+- `src/core/repair/purged-page-facts.ts` (1): a request on managed brains; the unmanaged expiry runs in `maintenanceTransaction`.- `src/core/repair/extractor-facts.ts` (1): a request on managed brains; the unmanaged extractor-facts restore runs in `maintenanceTransaction`.- `src/core/repair/fact-claim-dates.ts` (1): `gbrain repair fact-claim-dates` updates database-only facts' `valid_from` in one `maintenanceTransaction` on unmanaged brains only; fence rows move through revision-bound `put_page`, and a managed brain never takes the direct update.- `src/core/repair/facts-rehome.ts` (1): `gbrain repair facts-rehome` moves facts inside `coordinatedDatabaseWrite` on managed brains (the `relink_facts` operation) and in one `maintenanceTransaction` per slug on unmanaged brains.diff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex ec146e8461..715af2d5fc 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -8,7 +8,7 @@ can load, and Google source files other local users can read. `gbrain repair`fixes those and the other kinds listed in[What each kind fixes](#what-each-kind-fixes). Every run is a preview unlessyou pass `--apply`.-The explicit-only kinds, among them `google-file-modes`, `stale-atoms`, `atoms-restore`, `atom-fence-rows`, `extractor-facts`, `unwithdrawn-facts`,+The explicit-only kinds, among them `google-file-modes`, `stale-atoms`, `atoms-restore`, `atom-fence-rows`, `extractor-facts`, `unwithdrawn-facts`, `purged-page-facts`,`captured-facts`, `agent-session-facts`, `redacted-facts`, `loop-facts`, `ontology-facts`, `orphan-children`, `failed-writes`, `frontmatter`, `managed-sync-orphans`, `content-receipt-root`, `stuck-recovery`, `aliases`, `rename-identity` and `segmented-mirror`, run only when you name them (see [Explicit-only repair kinds](#explicit-only-repair-kinds)).`gbrain doctor --remediation-plan` lists the same kinds as repair steps, and`gbrain doctor --remediate --yes --include-repairs --expect <plan_hash>` runs them under a budget@@ -228,8 +228,8 @@ gives each a fresh row number); it never deletes or rewrites a page.| `--limit <n>` | Repair at most `n` items per kind in this run (a positive integer; with `--all`, up to `n` for each kind). Rerun the same command to continue. || `--no-embed` | `safe-chunks` and `contextual-mode`: skip the embedding provider. Run `gbrain embed --stale` later. `timeline` and `visibility` pages are re-embedded by their publication either way. || `--all` | Run every automatic kind in order. Explicit-only kinds are listed with their preview command, never run. |-| `--expect <hash>` | Explicit-only kinds: apply exactly the set the preview printed under this hash. Required with `--apply` for `stale-atoms`, `atoms-restore`, `extractor-facts`, `unwithdrawn-facts`, `captured-facts`, `redacted-facts`, `loop-facts`, `failed-writes`, `frontmatter`, `managed-sync-orphans` and `content-receipt-root`. |-| `--include-ambiguous` | `extractor-facts`, `unwithdrawn-facts`, `captured-facts` and `agent-session-facts`: widen the hashed set to `ambiguous` facts. `frontmatter`: add interpretive file changes. Pass it to both the preview and the apply. See [Extractor facts](#extractor-facts), [Captured facts](#captured-facts) and [Frontmatter](#frontmatter). |+| `--expect <hash>` | Explicit-only kinds: apply exactly the set the preview printed under this hash. Required with `--apply` for `stale-atoms`, `atoms-restore`, `extractor-facts`, `unwithdrawn-facts`, `purged-page-facts`, `captured-facts`, `redacted-facts`, `loop-facts`, `failed-writes`, `frontmatter`, `managed-sync-orphans` and `content-receipt-root`. |+| `--include-ambiguous` | `extractor-facts`, `unwithdrawn-facts`, `purged-page-facts`, `captured-facts` and `agent-session-facts`: widen the hashed set to `ambiguous` facts. `frontmatter`: add interpretive file changes. Pass it to both the preview and the apply. See [Extractor facts](#extractor-facts), [Captured facts](#captured-facts) and [Frontmatter](#frontmatter). || `--only <path>`, `--skip <path>` | `frontmatter` only: select source-relative files (repeatable). The hash covers the selection, so pass the same flags to the apply. || `--diff` | `frontmatter` only: print every per-file diff instead of one sample per class (`--json` always carries all of them). || `--yes` | `frontmatter --apply` only: the user agreed to the previewed file changes (destructive consent). Without it a terminal asks, and a run without a terminal exits 3 with the consent payload. |@@ -2582,6 +2582,47 @@ file is written. Each page publishes one database-onlymanaged `extract_facts` runs from this release on keep these rows active,except a row whose claim the page's fence now states, which still retires.+<a id="purged-page-facts"></a>+### Purged page facts++Before #8859 was fixed, purging a page (`gbrain delete --purge`, or the+retention purge of a page deleted more than 72 hours ago) left behind the+facts extracted from that page that were filed on no page: their context+named the page, but nothing tied them to it. They stayed active, so recall+kept returning claims from a page you had removed. Purges now take them;+`gbrain repair purged-page-facts` retires the ones older purges left.++```bash+gbrain doctor # purged_page_facts+gbrain repair purged-page-facts # preview: every candidate with its class+gbrain repair purged-page-facts --apply --expect <hash> --yes+gbrain repair purged-page-facts --include-ambiguous # also rows of pages gone with no purge record+```++A candidate is an active fact written by page extraction, filed on no page,+whose context starts with the slug of a page that no longer exists in its+source (not even as a deleted page, which a restore could bring back). A+slug with no `/` counts only when the purge ledger records it.++- **evidenced**: the purge ledger records a purge of that page after the fact+ was written. Retired by default.+- **ambiguous**: the page is gone with no purge record. A retention purge+ leaves none, and neither does a rename that did not carry its facts, so+ these are retired only with `--include-ambiguous` on both the preview and+ the apply.+- **excluded** (listed, never retired): the claim was withdrawn, the row is+ superseded, or another fact or an open loop references it.++The apply needs the preview's hash and your consent (`--yes`, or a terminal+prompt). It expires exactly the previewed rows that are still active,+unchanged and still without a page; anything else is reported as+`changed_since_preview` and kept, and a row that appeared after the preview+is never touched. Rows are expired, not deleted, so their history stays. The+repair writes no purge record, so the same claim could be saved again later. On a+managed brain each batch of up to 200 facts is one database-only+`managed_maintenance_expire_purged_page_facts` request. It works the same on+a repo-less brain: no page or file is written.+<a id="conversation-labels"></a>### Conversation labels@@ -4687,6 +4728,7 @@ Then recover what doctor names, in this order (skip a step whose check is ok):| `[gbrain] warning: OPENAI_API_KEY in this process's environment differs from openai_api_key …`; doctor `embedding_key_source` warns | `embedding_key_source` ([provider key source](#embedding-key-source)) | `gbrain doctor` (`embedding_key_source`) | remove the variable and restart that process, or `gbrain config unset openai_api_key` | the warning no longer prints; `gbrain doctor` shows `embedding_key_source` ok | user (owns the key) | `credentials` || `The OpenAI embedding provider rejected its key (HTTP 401)` | [`embedding_auth_failed`](write-refusals.md#embedding_auth_failed) | `gbrain doctor` (`embedding_key_source`, `embedding_provider`) | the printed fix for the key source in effect, then `gbrain embed --stale` | `gbrain doctor` shows `embedding_provider` ok | user (owns the key) | `credentials`; `paid` for `embed --stale` || Conversation facts missing from recall after managed writes | `extractor_facts_expired` | `gbrain repair extractor-facts` | `gbrain repair extractor-facts --apply --expect <hash>` (ambiguous: add `--include-ambiguous` to both) | `gbrain doctor` | brain host, after the user agrees | `destructive` (rewrites expired facts) |+| Recall still returns claims from a page you purged; doctor `purged_page_facts` warns | `purged_page_facts` ([purged page facts](#purged-page-facts)) | `gbrain repair purged-page-facts` | `gbrain repair purged-page-facts --apply --expect <hash> --yes` (no purge record: add `--include-ambiguous` to both) | `gbrain doctor` | brain host, after the user agrees | destructive (`--yes`) || Facts missing from recall after a page write or `extract_facts` run, although no fence of their page held them; doctor `unwithdrawn_facts` warns | `unwithdrawn_facts` ([unwithdrawn facts](#unwithdrawn-facts)) | `gbrain repair unwithdrawn-facts` | `gbrain repair unwithdrawn-facts --apply --expect <hash>` (ambiguous: add `--include-ambiguous` to both) | `gbrain doctor` | brain host, after the user agrees | none || Recall returns conversation facts such as "Date said …" or "Attendees said …", or facts dated 1970-01-01; doctor `conversation_label_facts` warns; the upgrade banner prints `conversation_label_facts: N (explicit_kind_required; …)` | `gbrain repair conversation-labels` (ambiguous: add `--include-ambiguous`) | `gbrain repair conversation-labels --apply --expect <hash> --yes`, then the printed `gbrain extract-conversation-facts --slugs … --dry-run` | `gbrain doctor --only conversation_label_facts --json` | brain host, after the user agrees | `destructive` (retires facts; no model calls) |diff --git a/scripts/check-core-guard-coverage.mjs b/scripts/check-core-guard-coverage.mjsindex 1e865ea68b..4d936112e3 100644--- a/scripts/check-core-guard-coverage.mjs+++ b/scripts/check-core-guard-coverage.mjs@@ -52,6 +52,7 @@ const EXEMPT = {'src/core/repair/unwithdrawn-facts.ts': 'owner-run, preview-bound repair putting expired fact rows back to their recorded before-state (row number and lifecycle); page text is unchanged','src/core/repair/redacted-facts.ts': 'owner-run, preview-bound repair rewriting the text of database-only fact rows to words a source page proves; page text is unchanged (fenced rows publish through put_page, preparePageMutation)','src/core/repair/visibility.ts': 'owner-run, preview-first repair that only tightens database-only fact rows from world to private; page text is unchanged (fenced rows and page labels publish through put_page, preparePageMutation)',+ 'src/core/repair/purged-page-facts.ts': 'owner-run repair expiring fact rows of pages that no longer exist (shrinks only; no page text)','src/core/repair/conversation-labels.ts': 'owner-run repair expiring fact rows and writing an outcome row (shrinks only; page text unchanged)','src/core/shared-skills/publication.ts': 'skill bundles, not pages','src/core/persistence/prepared-import.ts': 'type definitions shared by import preparers; prepares nothing itself',diff --git a/src/commands/doctor/checks/graph-health.ts b/src/commands/doctor/checks/graph-health.tsindex 828f4d27aa..8ae6e612da 100644--- a/src/commands/doctor/checks/graph-health.ts+++ b/src/commands/doctor/checks/graph-health.ts@@ -397,6 +397,7 @@ export const timelineHistoryEntry: DoctorEntry = {'generation_pending','extractor_facts_expired','unwithdrawn_facts',+ 'purged_page_facts','conversation_label_facts','conversation_outcomes_stale','captured_facts_active',diff --git a/src/commands/doctor/checks/purged-page-facts.ts b/src/commands/doctor/checks/purged-page-facts.tsnew file mode 100644index 0000000000..b6548ff785--- /dev/null+++ b/src/commands/doctor/checks/purged-page-facts.ts@@ -0,0 +1,29 @@+import type { BrainEngine } from '../../../core/engine.ts';+import type { Check } from '../../doctor.ts';+import { countPurgedPageFacts } from '../../../core/repair/purged-page-facts.ts';++const CAP = 10_000;++/**+ * #8859 follow-up: page-extraction facts filed on no page that purges before the #8859 fix left active although their+ * page is gone. Counts the retirable ones by class (evidenced: the purge ledger records the page; ambiguous: no purge+ * record) and names the explicit-only repair's preview. Read-only.+ */+export async function purgedPageFactsCheck(engine: BrainEngine, sourceIds?: string[]): Promise<Check> {+ try {+ const sources = sourceIds ?? (await engine.executeRaw<{ id: string }>('SELECT id FROM sources WHERE archived IS NOT TRUE ORDER BY id')).map(row => row.id);+ const counted = await countPurgedPageFacts(engine, sources, CAP);+ const details = { evidenced: counted.evidenced, ambiguous: counted.ambiguous, capped: counted.capped, repair: 'purged-page-facts', docs: 'docs/guides/repair.md#purged-page-facts' };+ const total = counted.evidenced + counted.ambiguous;+ if (!total) return { name: 'purged_page_facts', status: 'ok', details, message: 'No fact extracted from a page is left active after that page was purged.' };+ return { name: 'purged_page_facts', status: 'warn', details,+ message: `${total}${counted.capped ? '+' : ''} fact(s) extracted from pages that no longer exist are still active, left by purges that ran before #8859 was fixed `+ + `(${counted.evidenced} of purged pages, ${counted.ambiguous} of pages gone with no purge record); recall still returns them. `+ + 'Preview the retirement on the brain host: gbrain repair purged-page-facts — then run the apply command it prints after the user agrees '+ + '(rows with no purge record only with gbrain repair purged-page-facts --include-ambiguous).' };+ } catch (error) {+ return { name: 'purged_page_facts', status: 'warn',+ message: `Facts left by purged pages could not be inspected: ${error instanceof Error ? error.message : String(error)}. Health is unknown.`,+ details: { count: 'unknown' } };+ }+}diff --git a/src/commands/doctor/wave-checks.ts b/src/commands/doctor/wave-checks.tsindex e98a4183a8..293965179c 100644--- a/src/commands/doctor/wave-checks.ts+++ b/src/commands/doctor/wave-checks.ts@@ -401,6 +401,13 @@ export const WAVE_CHECKS: readonly WaveCheckSpec[] = [impact: 'Some facts were retired by a facts-fence generation although no fence of their page held them, and recall no longer returns them',run: async (engine, scope) => (await import('./checks/unwithdrawn-facts.ts')).unwithdrawnFactsCheck(engine, scope.sourceIds),},+ {+ id: 'purged_page_facts', resolution: 'repair', registration: 'wave',+ hostOnly: 'Retiring facts left by purged pages is a host-side, explicit-only repair.',+ count: d => Number(d.evidenced ?? 0) + Number(d.ambiguous ?? 0),+ impact: 'Some facts extracted from pages that were purged are still active, and recall still returns them',+ run: async (engine, scope) => (await import('./checks/purged-page-facts.ts')).purgedPageFactsCheck(engine, scope.sourceIds),+ },{id: 'conversation_label_facts', resolution: 'repair', registration: 'wave',hostOnly: 'Retiring label-misattributed conversation facts is a host-side, explicit-only repair.',diff --git a/src/core/doctor-categories.ts b/src/core/doctor-categories.tsindex c4e9fa249d..d2932de450 100644--- a/src/core/doctor-categories.ts+++ b/src/core/doctor-categories.ts@@ -124,6 +124,7 @@ export const BRAIN_CHECK_NAMES: ReadonlySet<string> = new Set(['guest_source','extractor_facts_expired','unwithdrawn_facts',+ 'purged_page_facts','loop_facts_drift','ontology_facts_fenced','orphan_persistence_bindings',diff --git a/src/core/facts/gone-page-facts.ts b/src/core/facts/gone-page-facts.tsindex 993cef3d66..95878d4f65 100644--- a/src/core/facts/gone-page-facts.ts+++ b/src/core/facts/gone-page-facts.ts@@ -23,7 +23,17 @@ export const PAGE_EXTRACTION_SOURCE_LIST = ['mcp:put_page', 'sync:import', 'file*/export function extractedFromPageSql(alias: string, slug: string): string {return `(${alias}.source_markdown_slug IS NULL AND ${alias}.source IN (${PAGE_EXTRACTION_SOURCE_LIST.map(s => `'${s}'`).join(', ')})- AND (${alias}.context = ${slug} OR starts_with(${alias}.context, ${slug} || ' — ')))`;+ AND md5(${contextHeadSql(alias)}) = md5(${slug}) AND ${contextHeadSql(alias)} = ${slug})`;+}++/**+ * The head of the fact `alias`'s context: the text before the first CONTEXT_NOTE_SEPARATOR (subject-infer.ts), the+ * page slug for a page-extraction row. It equals a slug exactly when the context is the slug or starts with it and the+ * separator (no slug holds the separator). v315's idx_facts_context_head indexes its md5 (a context is unbounded text,+ * too long for a btree key) for rows filed on no page, so a match compares the md5 first.+ */+export function contextHeadSql(alias: string): string {+ return `split_part(${alias}.context, ' — ', 1)`;}/** `$1` source ids and `$2` slugs, paired by position. */diff --git a/src/core/persistence/prepared-maintenance.ts b/src/core/persistence/prepared-maintenance.tsindex 4c0d73c575..0a547bd259 100644--- a/src/core/persistence/prepared-maintenance.ts+++ b/src/core/persistence/prepared-maintenance.ts@@ -671,6 +671,7 @@ async function prepareMaintenanceKind(engine: BrainEngine, row: WriteRequest, coenterClaimStep(clock, 'maintenance_dispatch');if (row.intent?.kind === 'managed_maintenance_restore_extractor_facts') return (await import('../repair/extractor-facts.ts')).prepareExtractorFactsRestore(engine, row);if (row.intent?.kind === 'managed_maintenance_restore_unwithdrawn_facts') return (await import('../repair/unwithdrawn-facts.ts')).prepareUnwithdrawnFactsRestore(engine, row);+ if (row.intent?.kind === 'managed_maintenance_expire_purged_page_facts') return (await import('../repair/purged-page-facts.ts')).preparePurgedPageFactsExpiry(engine, row);if (row.intent?.kind === 'managed_maintenance_expire_captured_facts') return (await import('../repair/captured-facts.ts')).prepareCapturedFactsExpiry(engine, row);if (row.intent?.kind === 'managed_maintenance_agent_session_facts') return (await import('../repair/agent-session-facts.ts')).prepareAgentSessionFactsWrite(engine, row);if (row.intent?.kind === 'managed_maintenance_restore_redacted_facts') return (await import('../repair/redacted-facts.ts')).prepareRedactedFactsRestore(engine, row);diff --git a/src/core/persistence/preview-approval.ts b/src/core/persistence/preview-approval.tsindex e11bf0abff..db6182a30e 100644--- a/src/core/persistence/preview-approval.ts+++ b/src/core/persistence/preview-approval.ts@@ -40,7 +40,7 @@ export const PREVIEW_APPROVAL_OP = 'repair-approval';/** The default `purgeStaleCheckpoints` TTL; an older approved set may already be gone. */export const APPROVAL_MAX_AGE_DAYS = 7;-export type PreviewApprovalCommand = 'authorize-legacy' | 'jobs-cancel' | 'stale-atoms' | 'atoms-restore' | 'atom-fence-rows' | 'extractor-facts' | 'unwithdrawn-facts' | 'conversation-labels' | 'captured-facts' | 'agent-session-facts' | 'redacted-facts' | 'loop-facts' | 'ontology-facts' | 'facts-rehome' | 'consolidate-ties' | 'duplicate-facts' | 'failed-writes' | 'split-facts' | 'frontmatter' | 'fences' | 'slug-conflicts' | 'managed-sync-orphans' | 'frontmatter-keys' | 'file-drift' | 'stuck-recovery' | 'ghost-pages' | 'origin-twins' | 'moved-pages' | 'receipt-compaction' | 'accepted-takes' | 'rename-identity' | 'overwritten-edits' | 'legacy-twins' | 'missing-tables' | 'segmented-mirror' | 'segmented-tiling' | 'uncommitted-deletions' | 'visibility' | 'generations' | 'fact-claim-dates' | 'timeline-order' | 'contact-rows' | 'provenance' | 'secret-spans' | 'rows-behind' | 'facts-home' | 'private-labels' | 'lost-fence-rows';+export type PreviewApprovalCommand = 'authorize-legacy' | 'jobs-cancel' | 'stale-atoms' | 'atoms-restore' | 'atom-fence-rows' | 'extractor-facts' | 'unwithdrawn-facts' | 'purged-page-facts' | 'conversation-labels' | 'captured-facts' | 'agent-session-facts' | 'redacted-facts' | 'loop-facts' | 'ontology-facts' | 'facts-rehome' | 'consolidate-ties' | 'duplicate-facts' | 'failed-writes' | 'split-facts' | 'frontmatter' | 'fences' | 'slug-conflicts' | 'managed-sync-orphans' | 'frontmatter-keys' | 'file-drift' | 'stuck-recovery' | 'ghost-pages' | 'origin-twins' | 'moved-pages' | 'receipt-compaction' | 'accepted-takes' | 'rename-identity' | 'overwritten-edits' | 'legacy-twins' | 'missing-tables' | 'segmented-mirror' | 'segmented-tiling' | 'uncommitted-deletions' | 'visibility' | 'generations' | 'fact-claim-dates' | 'timeline-order' | 'contact-rows' | 'provenance' | 'secret-spans' | 'rows-behind' | 'facts-home' | 'private-labels' | 'lost-fence-rows';export interface ApprovedSetKey {command: PreviewApprovalCommand;diff --git a/src/core/repair/core.ts b/src/core/repair/core.tsindex de1ba4845f..80e23c9fda 100644--- a/src/core/repair/core.ts+++ b/src/core/repair/core.ts@@ -54,7 +54,7 @@ import type { GenerationsSelection } from '../generations/repair.ts';import type { PrivateLabelsLedger } from './private-labels.ts';import type { ReadersRequest } from './readers.ts';-export const REPAIR_KINDS = ['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'accepted-takes', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'atom-fence-rows', 'extractor-facts', 'unwithdrawn-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'fences', 'slug-conflicts', 'timeline-comments', 'timeline-order', 'contact-rows', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'moved-pages', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'segmented-tiling', 'uncommitted-deletions', 'generations', 'lost-fence-rows', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind', 'facts-home', 'access', 'private-labels'] as const;+export const REPAIR_KINDS = ['timeline', 'visibility', 'safe-chunks', 'contextual-mode', 'connector-checkpoints', 'request-indexes', 'connector-fences', 'take-supersession', 'accepted-takes', 'orphan-bindings', 'embedding-effects', 'attribution-backfill', 'planner-stats', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'atom-fence-rows', 'extractor-facts', 'unwithdrawn-facts', 'purged-page-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'fences', 'slug-conflicts', 'timeline-comments', 'timeline-order', 'contact-rows', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'moved-pages', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'segmented-tiling', 'uncommitted-deletions', 'generations', 'lost-fence-rows', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind', 'facts-home', 'access', 'private-labels'] as const;export type RepairKind = typeof REPAIR_KINDS[number];export interface RepairScope { brain_id: string; source_ids: string[] }diff --git a/src/core/repair/purged-page-facts.ts b/src/core/repair/purged-page-facts.tsnew file mode 100644index 0000000000..5617878835--- /dev/null+++ b/src/core/repair/purged-page-facts.ts@@ -0,0 +1,245 @@+/**+ * `gbrain repair purged-page-facts` (#8859 follow-up): retire facts left active by purges that ran before the #8859 fix.+ *+ * A page-extraction fact filed on no page (`source_markdown_slug` NULL, the page slug as the head of `context`,+ * extractedFromPageSql) used to outlive its page: neither a direct purge nor the retention expiry of a soft-deleted page+ * took it. Those purges are fixed; the rows they left behind stay active and recallable until this repair retires them.+ *+ * Candidates: active page-extraction facts filed on no page whose context head is a page slug (recordedPageSlug: no+ * whitespace and a '/', or any single-segment slug the purge ledger records) that no page row of the source holds, live+ * or soft-deleted. Each is classified:+ * - excluded (listed, never retired): the claim was withdrawn, the row is superseded, another fact's `superseded_by`+ * or an open loop references it;+ * - evidenced: the page purge ledger (`page_purges`) records a purge of that slug in the source after the fact was+ * written (a fact written after the purge belongs to a later page under the slug, say one renamed away since);+ * - ambiguous: no purge record (a retention purge of a soft-deleted page leaves none, and neither does a rename that+ * did not carry the facts); retired only with `--include-ambiguous` and that preview's hash.+ *+ * The preview lists every candidate, prints a hash over all of it and saves the approved set. The apply, bound to that+ * hash and the user's consent, expires exactly the approved rows that are still active, unchanged, of the same class+ * and still without a page (expired_at set, row_num cleared, as the fixed purge does); anything else is reported+ * changed_since_preview and kept. Rows are expired, never deleted. A managed brain publishes one database-only+ * request per batch; an unmanaged brain writes under the page keys. Never automatic.+ */+import type { BrainEngine } from '../engine.ts';+import type { PreparedMutation } from '../persistence/coordinator.ts';+import type { WriteRequest } from '../persistence/model.ts';+import { opError, OperationError } from '../ops/contract.ts';+import { readFix } from '../ops/op-fix.ts';+import { digest } from '../persistence/digest.ts';+import { isTerminal } from '../persistence/model.ts';+import { getWriteRequest } from '../persistence/journal.ts';+import { authorizeStoredRequest, authorizeWrite } from '../persistence/authority.ts';+import { managedPersistenceEnabled } from '../persistence/ownership.ts';+import { waitForWrite, writeResponse } from '../persistence/service.ts';+import { maintenanceTransaction } from '../persistence/attribution.ts';+import { maintenancePreflight, submitDatabaseMaintenanceIntent } from '../persistence/prepared-maintenance.ts';+import { clearApprovedSet, loadApprovedSet, previewChangedError, previewHash, saveApprovedSet } from '../persistence/preview-approval.ts';+import { PAGE_EXTRACTION_SOURCE_LIST, contextHeadSql } from '../facts/gone-page-facts.ts';+import { afterCursor, type RepairHandler, type RepairItem, type RepairItemOutcome, type RepairListing, type RepairPlan } from './core.ts';++export const PURGED_PAGE_FACTS_INTENT = 'managed_maintenance_expire_purged_page_facts';+/** The request slug of every batch: a dot-led segment, which no page slug may have. */+export const PURGED_PAGE_FACTS_ANCHOR_SLUG = '.maintenance/purged-page-facts';+const BATCH_FACTS = 200;+const COMMAND = 'purged-page-facts' as const;+const DOCS = 'docs/guides/repair.md#purged-page-facts';+const previewFix = (sourceId: string) => readFix('Previews the purged-page-facts repair without changing anything.',+ { argv: ['gbrain', 'repair', COMMAND, '--source', sourceId, '--json'] });++export type PurgedPageFactClass = 'evidenced' | 'ambiguous' | 'excluded';++export interface PurgedPageFact {+ id: number;+ source_id: string;+ /** The gone page's slug (the context head). */+ slug: string;+ class: PurgedPageFactClass;+ reason: string;+ /** Digest of the row's claim, context, entity, visibility and source, so an edited row is not retired. */+ fact_hash: string;+}++interface Approved { source_id: string; include_ambiguous: boolean; scope: string[]; facts: PurgedPageFact[] }+interface BatchItem extends RepairItem { batch: Approved; request_id: string; hash: string; last: boolean }++const HEAD = contextHeadSql('f');+const SOURCES = PAGE_EXTRACTION_SOURCE_LIST.map(s => `'${s}'`).join(', ');+const PURGED_SQL = `EXISTS (SELECT 1 FROM page_purges pp WHERE pp.source_id = f.source_id AND pp.slug = ${HEAD} AND pp.purged_at >= f.created_at)`;+const CANDIDATE_WHERE = `f.source_id = ANY($1::text[]) AND f.expired_at IS NULL AND f.source_markdown_slug IS NULL AND f.source IN (${SOURCES})+ AND ${HEAD} <> '' AND ${HEAD} !~ '[[:space:]]' AND (strpos(${HEAD}, '/') > 0 OR ${PURGED_SQL})+ AND NOT EXISTS (SELECT 1 FROM pages p WHERE p.source_id = f.source_id AND p.slug = ${HEAD})`;+const EXCLUDED_SQL = {+ withdrawn: `EXISTS (SELECT 1 FROM fact_withdrawals w WHERE w.source_id = f.source_id AND w.visibility = f.visibility+ AND w.fact_hash IN (gbrain_fact_fingerprint(f.fact), gbrain_fact_fingerprint_v1(f.fact)) AND (w.subject = '*' OR w.subject = f.entity_slug))`,+ superseded: 'f.superseded_by IS NOT NULL',+ referenced: 'EXISTS (SELECT 1 FROM facts g WHERE g.superseded_by = f.id)',+ open_loop: 'EXISTS (SELECT 1 FROM open_loops o WHERE o.fact_id = f.id)',+};++interface CandidateRow {+ id: number | string; source_id: string; slug: string; fact: string; context: string; entity_slug: string | null; visibility: string; source: string;+ withdrawn: boolean; superseded: boolean; referenced: boolean; open_loop: boolean; purged: boolean;+}++/** Every candidate of these sources (or only these ids), classified, in (source, slug, id) order. Read-only. */+export async function classifyPurgedPageFacts(db: BrainEngine, sourceIds: string[], opts: { ids?: number[] } = {}): Promise<PurgedPageFact[]> {+ const rows = await db.executeRaw<CandidateRow>(`+ SELECT f.id, f.source_id, ${HEAD} AS slug, f.fact, f.context, f.entity_slug, f.visibility, f.source,+ ${EXCLUDED_SQL.withdrawn} AS withdrawn, ${EXCLUDED_SQL.superseded} AS superseded, ${EXCLUDED_SQL.referenced} AS referenced,+ ${EXCLUDED_SQL.open_loop} AS open_loop, ${PURGED_SQL} AS purged+ FROM facts f+ WHERE ${CANDIDATE_WHERE} AND ($2::bigint[] IS NULL OR f.id = ANY($2::bigint[]))+ ORDER BY f.source_id, ${HEAD}, f.id`, [sourceIds, opts.ids ?? null]);+ return rows.map(row => {+ const [klass, reason]: [PurgedPageFactClass, string] = row.withdrawn ? ['excluded', 'withdrawn'] : row.superseded ? ['excluded', 'superseded']+ : row.referenced ? ['excluded', 'superseded_by_reference'] : row.open_loop ? ['excluded', 'open_loop_reference']+ : row.purged ? ['evidenced', 'page_purge_recorded'] : ['ambiguous', 'no_purge_record'];+ return { id: Number(row.id), source_id: row.source_id, slug: row.slug, class: klass, reason,+ fact_hash: digest([row.fact, row.context, row.entity_slug, row.visibility, row.source]) };+ });+}++/** Doctor's bounded count of the retirable candidates by class, capped. Read-only. */+export async function countPurgedPageFacts(db: BrainEngine, sourceIds: string[], cap: number): Promise<{ evidenced: number; ambiguous: number; capped: boolean }> {+ const rows = await db.executeRaw<{ purged: boolean }>(`SELECT ${PURGED_SQL} AS purged FROM facts f+ WHERE ${CANDIDATE_WHERE} AND NOT (${Object.values(EXCLUDED_SQL).join(' OR ')}) LIMIT $2`, [sourceIds, cap + 1]);+ const kept = rows.slice(0, cap);+ return { evidenced: kept.filter(r => r.purged).length, ambiguous: kept.filter(r => !r.purged).length, capped: rows.length > cap };+}++const DETAILS: Record<string, string> = {+ withdrawn: 'its claim was withdrawn (forget)',+ superseded: 'a newer fact superseded it',+ superseded_by_reference: 'another fact names it in superseded_by',+ open_loop_reference: 'an open loop references it (open_loops.fact_id)',+ page_purge_recorded: 'its page was purged after it was written (the page purge ledger records the slug) and the purge left it active',+ no_purge_record: 'its page is gone with no purge record from after it was written (a retention purge of a deleted page, or a rename that did not carry its facts)',+};++async function previewCommand(engine: BrainEngine, sourceIds: string[], includeAmbiguous: boolean): Promise<string> {+ const [{ n }] = await engine.executeRaw<{ n: number | string }>('SELECT count(*)::int AS n FROM sources WHERE archived IS NOT TRUE');+ return `gbrain repair ${COMMAND}${sourceIds.length === 1 && Number(n) > 1 ? ` --source ${sourceIds[0]}` : ''}${includeAmbiguous ? ' --include-ambiguous' : ''}`;+}++function requestIdFor(hash: string, batch: Approved, attempt: number): string {+ const h = digest(['purged-page-facts-batch-v1', hash, batch.source_id, batch.facts.map(f => f.id), attempt]);+ return `${h.slice(0, 8)}-${h.slice(8, 12)}-4${h.slice(13, 16)}-a${h.slice(17, 20)}-${h.slice(20, 32)}`;+}++function batchItems(batches: Approved[], hash: string): BatchItem[] {+ return batches.map((batch, index) => ({ cursor: { phase: 0, id: index + 1 }, source_id: batch.source_id, slug: batch.facts[0]!.slug, chars: 0,+ action: `retire ${batch.facts.length} fact(s) left by purged page(s)`, batch, request_id: requestIdFor(hash, batch, 0), hash, last: index === batches.length - 1 }));+}++const sourcesOf = (engine: BrainEngine, ids: string[]) => engine.executeRaw<{ id: string; incarnation: string }>(+ 'SELECT id, incarnation::text AS incarnation FROM sources WHERE id = ANY($1::text[]) ORDER BY id', [ids]);++export const purgedPageFactsRepair: RepairHandler = {+ kind: COMMAND,+ embeds: false,+ async plan(engine, scope, after, opts): Promise<RepairPlan> {+ const includeAmbiguous = opts?.includeAmbiguous === true;+ const command = await previewCommand(engine, scope.source_ids, includeAmbiguous);+ const hashOf = async (facts: PurgedPageFact[]) => previewHash({ kind: 'purged-page-facts-v1', brain_id: scope.brain_id, sources: await sourcesOf(engine, scope.source_ids),+ selection: { source_ids: scope.source_ids, include_ambiguous: includeAmbiguous }, facts });+ if (opts?.apply) {+ if (!opts.expect) {+ throw new OperationError('invalid_params', `gbrain repair ${COMMAND} --apply retires only the set a preview printed.`,+ `Preview first: ${command} — then run the apply command it prints: ${command} --apply --expect <preview-hash> --yes`, DOCS);+ }+ const approved = await loadApprovedSet<Approved>(engine, { command: COMMAND, hash: opts.expect, previewCommand: command, emptyHash: () => hashOf([]) });+ const scopeKey = JSON.stringify(scope.source_ids);+ if (approved.items.some(batch => batch.include_ambiguous !== includeAmbiguous || JSON.stringify(batch.scope) !== scopeKey)) throw previewChangedError(opts.expect, command);+ const items = batchItems(approved.items, opts.expect).filter(item => afterCursor(item.cursor, after));+ if (!items.length) await clearApprovedSet(engine, { command: COMMAND, hash: opts.expect });+ return { items, preview_hash: opts.expect, residuals: {} };+ }+ const facts = await classifyPurgedPageFacts(engine, scope.source_ids);+ const hash = await hashOf(facts);+ const batches: Approved[] = [];+ for (const fact of facts.filter(f => f.class === 'evidenced' || f.class === 'ambiguous' && includeAmbiguous)) {+ const open = batches.at(-1);+ if (open && open.source_id === fact.source_id && open.facts.length < BATCH_FACTS) open.facts.push(fact);+ else batches.push({ source_id: fact.source_id, include_ambiguous: includeAmbiguous, scope: scope.source_ids, facts: [fact] });+ }+ if (batches.length) await saveApprovedSet(engine, { command: COMMAND, hash }, batches);+ const count = (klass: PurgedPageFactClass) => facts.filter(f => f.class === klass).length;+ const listing: RepairListing[] = facts.map(fact => ({ item: `${fact.source_id}:${fact.slug}#${fact.id}`,+ class: fact.class === 'excluded' ? `excluded:${fact.reason}` : fact.class, detail: DETAILS[fact.reason] ?? fact.reason }));+ return { items: batchItems(batches, hash), preview_hash: hash, listing,+ residuals: { evidenced: count('evidenced'), ambiguous: count('ambiguous'), excluded: count('excluded') } };+ },+ async apply(ctx, entry): Promise<RepairItemOutcome> {+ const item = entry as BatchItem;+ const result = await managedPersistenceEnabled(ctx.engine)+ ? await expireManaged(ctx.engine, ctx.config, item, ctx.writeWaitMs)+ : await maintenanceTransaction(ctx.engine, async tx => {+ await tx.lockPageKeys([...new Set(item.batch.facts.map(f => f.slug))].map(slug => ({ sourceId: item.source_id, slug })));+ return expireBatch(tx, item.batch);+ });+ if (item.last) await clearApprovedSet(ctx.engine, { command: COMMAND, hash: item.hash });+ if (!result.expired.length) return { applied: false, outcome: 'changed_since_preview', reason: 'every approved fact changed since the preview' };+ return { applied: true, outcome: 'retired', detail: { retired: result.expired.length, ...(result.changed.length ? { changed_since_preview: result.changed } : {}) } };+ },+};++interface BatchResult { expired: number[]; changed: number[] }++/**+ * Under the batch's page keys: expire each approved fact that is still a candidate of the same class with the same+ * hash (still active, still filed on no page, its page still absent). Nothing outside the approved ids is read for+ * writing.+ */+async function expireBatch(tx: BrainEngine, batch: Approved): Promise<BatchResult> {+ const ids = batch.facts.map(f => f.id);+ await tx.executeRaw('SELECT id FROM facts WHERE source_id = $1 AND id = ANY($2::bigint[]) FOR UPDATE', [batch.source_id, ids]);+ const live = new Map((await classifyPurgedPageFacts(tx, [batch.source_id], { ids })).map(f => [f.id, f]));+ const same = batch.facts.filter(f => { const now = live.get(f.id); return !!now && now.class === f.class && now.fact_hash === f.fact_hash && now.slug === f.slug; });+ const expired = same.length ? (await tx.executeRaw<{ id: number | string }>(+ 'UPDATE facts SET expired_at = now(), row_num = NULL WHERE source_id = $1 AND id = ANY($2::bigint[]) AND expired_at IS NULL RETURNING id',+ [batch.source_id, same.map(f => f.id)])).map(r => Number(r.id)) : [];+ const done = new Set(expired);+ return { expired: expired.sort((a, b) => a - b), changed: ids.filter(id => !done.has(id)) };+}++async function expireManaged(engine: BrainEngine, config: Parameters<typeof waitForWrite>[2], item: BatchItem, writeWaitMs: number | undefined): Promise<BatchResult> {+ const authority = (await maintenancePreflight(engine, item.source_id))!;+ for (let attempt = 0; ; attempt++) {+ const requestId = attempt === 0 ? item.request_id : requestIdFor(item.hash, item.batch, attempt);+ const prior = await getWriteRequest(engine, authority.writer.principal, requestId);+ if (prior && isTerminal(prior) && prior.state !== 'committed') continue;+ let receipt: Record<string, unknown>;+ if (prior) {+ await authorizeStoredRequest(engine, prior);+ receipt = writeResponse(await waitForWrite(engine, prior, config, writeWaitMs));+ } else {+ receipt = await submitDatabaseMaintenanceIntent(engine, authority, PURGED_PAGE_FACTS_ANCHOR_SLUG,+ { kind: PURGED_PAGE_FACTS_INTENT, expected_revision: null, preview_hash: item.hash, batch: item.batch }, requestId);+ }+ return { expired: (receipt.expired as number[] | undefined) ?? [], changed: (receipt.changed_since_preview as number[] | undefined) ?? [] };+ }+}++/** Preparer for `managed_maintenance_expire_purged_page_facts`: a database-only batch publication holding every gone page's key. */+export async function preparePurgedPageFactsExpiry(engine: BrainEngine, row: WriteRequest): Promise<PreparedMutation> {+ const batch = (row.intent as { batch?: Approved } | null)?.batch;+ if (row.slug !== PURGED_PAGE_FACTS_ANCHOR_SLUG || !batch || batch.source_id !== row.source_id || !Array.isArray(batch.facts) || !batch.facts.length+ || batch.facts.some(f => !Number.isSafeInteger(f?.id) || typeof f.slug !== 'string' || typeof f.fact_hash !== 'string'+ || (f.class !== 'evidenced' && f.class !== 'ambiguous'))) {+ throw opError('invalid_params', 'The purged-page-facts intent does not name its facts.',+ `Request ${row.request_id} in source ${row.source_id} does not name the facts it retires, so nothing changed. Preview the repair again and apply the new preview after the user approves.`,+ { fix: previewFix(row.source_id) });+ }+ const slugs = [...new Set(batch.facts.map(f => f.slug))];+ const authorize = async (db: BrainEngine) => {+ await authorizeWrite(db, row.authority, 'submit_job', row.slug);+ for (const slug of slugs) await authorizeWrite(db, row.authority, 'submit_job', slug);+ };+ await authorize(engine);+ return { observedRevision: null, noop: true, additionalPageKeys: slugs.map(slug => ({ sourceId: row.source_id, slug })), validate: authorize,+ apply: async tx => {+ const result = await expireBatch(tx, batch);+ return { status: 'completed', expired: result.expired, changed_since_preview: result.changed };+ } };+}diff --git a/src/core/repair/registry.ts b/src/core/repair/registry.tsindex ab342863de..9c2227b114 100644--- a/src/core/repair/registry.ts+++ b/src/core/repair/registry.ts@@ -57,6 +57,7 @@ import { atomsRestoreRepair } from './atoms-restore.ts';import { atomFenceRowsRepair } from './atom-fence-rows.ts';import { extractorFactsRepair } from './extractor-facts.ts';import { unwithdrawnFactsRepair } from './unwithdrawn-facts.ts';+import { purgedPageFactsRepair } from './purged-page-facts.ts';import { conversationLabelsRepair } from './conversation-labels.ts';import { capturedFactsRepair } from './captured-facts.ts';import { agentSessionFactsRepair, agentSessionFlagArgs } from './agent-session-facts.ts';@@ -252,6 +253,20 @@ const SPECS: Record<RepairKind, Omit<RepairKindSpec, 'kind'>> = {+ '--apply --expect <hash> restores exactly the previewed set, bound to each page\'s revision; a changed page or fact reports changed_since_preview. '+ 'Database-only; no page is rewritten.',},+ 'purged-page-facts': {+ handler: purgedPageFactsRepair, embeds: 'none', checks: ['purged_page_facts'], explicit_only: true, preview_bound: true, consent: 'destructive',+ consentText: {+ what: 'Retire the previewed facts that purges left active after their page was gone',+ why: 'The preview listed each fact extracted from a page that no longer exists; applying it expires exactly those rows.',+ risk: 'Database rows only: no page or file changes. Rows are expired, not deleted, so their history stays; recall stops returning them. '+ + 'A row that changed since the preview, or whose page came back, is kept.',+ user_message: 'Retire the previewed facts left over from purged pages? Recall stops returning them; the rows are expired, not deleted.',+ },+ summary: 'Retire facts that purges before the #8859 fix left active: page-extraction facts filed on no page whose context names a page that no longer '+ + 'exists. Rows of a page the purge ledger records are evidenced; rows of a page gone with no purge record (a retention purge, or a rename that did '+ + 'not carry its facts) are retired only with --include-ambiguous; withdrawn, superseded and referenced rows are never touched. Preview-bound: '+ + '--apply --expect <hash> --yes expires exactly the previewed rows; a row that changed since reports changed_since_preview. Database-only; runs only when named.',+ },'conversation-labels': {handler: conversationLabelsRepair, embeds: 'none', checks: ['conversation_label_facts'], explicit_only: true, consent: 'destructive',consentText: {diff --git a/src/core/schema-migrations/latest.generated.ts b/src/core/schema-migrations/latest.generated.tsindex 670b8af60b..a910e3d1a5 100644--- a/src/core/schema-migrations/latest.generated.ts+++ b/src/core/schema-migrations/latest.generated.ts@@ -4,4 +4,4 @@// The highest schema migration version (migrate.ts LATEST_VERSION), importable without// loading the migrations themselves.-export const LATEST_SCHEMA_VERSION = 314;+export const LATEST_SCHEMA_VERSION = 315;diff --git a/src/core/schema-migrations/registry.generated.ts b/src/core/schema-migrations/registry.generated.tsindex a201902ed6..03c68d24b1 100644--- a/src/core/schema-migrations/registry.generated.ts+++ b/src/core/schema-migrations/registry.generated.ts@@ -315,6 +315,7 @@ import { v311 } from './v311-concept-inputs-private.ts';import { v312 } from './v312-guest-notes.ts';import { v313 } from './v313-wake-events.ts';import { v314 } from './v314-guest-notes-fail-closed.ts';+import { v315 } from './v315-facts-context-head-index.ts';// Array order is historical (see HISTORICAL_ARRAY_ORDER in the generator); the// runner sorts by version before applying.@@ -628,4 +629,5 @@ export const MIGRATIONS: Migration[] = [v312,v313,v314,+ v315,];diff --git a/src/core/schema-migrations/v315-facts-context-head-index.ts b/src/core/schema-migrations/v315-facts-context-head-index.tsnew file mode 100644index 0000000000..c1c4c3090e--- /dev/null+++ b/src/core/schema-migrations/v315-facts-context-head-index.ts@@ -0,0 +1,23 @@+import type { Migration } from './types.ts';+import { buildIndexOnline, migrationNotice } from './helpers.ts';++// Applied by src/core/migrate.ts on the next initSchema(); see docs/ENGINES.md+// ("Canonical schema sources") for when schema.sql or a TS fragment also changes.+//+// #8859 follow-up: a page purge and the retention expiry of a deleted page also take the facts extracted from the page+// and filed on none, matched by the head of their context (contextHeadSql, facts/gone-page-facts.ts). Unindexed, each+// purged page scanned every fact of its source, and `gbrain repair purged-page-facts` and its doctor check scan them+// too. idx_facts_context_head indexes the md5 of that expression for the rows filed on no page, the only rows the+// match reads; the md5 keeps the key bounded, since a context is unbounded text and need not hold a separator.+// Postgres builds it CONCURRENTLY outside the migration transaction (a handler); PGLite builds inline. Nothing is+// rewritten. No schema.sql mirror, like v237.+export const v315: Migration = {+ version: 315,+ name: 'facts_context_head_index',+ idempotent: true,+ sql: '',+ handler: async (engine) => {+ await buildIndexOnline(engine, 315, { name: 'idx_facts_context_head', table: 'facts',+ sql: "CREATE INDEX IF NOT EXISTS idx_facts_context_head ON facts (source_id, md5(split_part(context, ' — ', 1))) WHERE source_markdown_slug IS NULL" }, { notice: migrationNotice });+ },+};diff --git a/test/fix-wave-5-integration.test.ts b/test/fix-wave-5-integration.test.tsindex 6ceccbbd94..7da067e6ea 100644--- a/test/fix-wave-5-integration.test.ts+++ b/test/fix-wave-5-integration.test.ts@@ -193,7 +193,7 @@ for (const backend of testBackends()) {const liveAtoms = async () => (await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE type='atom' AND deleted_at IS NULL"))[0].n;const activeFacts = async () => (await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM facts WHERE source_markdown_slug='conversations/expired' AND expired_at IS NULL"))[0].n;const untouched = async () => { expect(await liveAtoms()).toBe(30); expect(await activeFacts()).toBe(0); };- const previews = { 'accepted-takes': 'gbrain repair accepted-takes', 'google-file-modes': 'gbrain repair google-file-modes', 'stale-atoms': 'gbrain repair stale-atoms', 'atoms-restore': 'gbrain repair atoms-restore', 'atom-fence-rows': 'gbrain repair atom-fence-rows', 'extractor-facts': 'gbrain repair extractor-facts', 'unwithdrawn-facts': 'gbrain repair unwithdrawn-facts',+ const previews = { 'accepted-takes': 'gbrain repair accepted-takes', 'google-file-modes': 'gbrain repair google-file-modes', 'stale-atoms': 'gbrain repair stale-atoms', 'atoms-restore': 'gbrain repair atoms-restore', 'atom-fence-rows': 'gbrain repair atom-fence-rows', 'extractor-facts': 'gbrain repair extractor-facts', 'unwithdrawn-facts': 'gbrain repair unwithdrawn-facts', 'purged-page-facts': 'gbrain repair purged-page-facts','conversation-labels': 'gbrain repair conversation-labels', 'captured-facts': 'gbrain repair captured-facts', 'agent-session-facts': 'gbrain repair agent-session-facts', 'redacted-facts': 'gbrain repair redacted-facts', 'loop-facts': 'gbrain repair loop-facts', 'ontology-facts': 'gbrain repair ontology-facts', 'facts-rehome': 'gbrain repair facts-rehome', 'consolidate-ties': 'gbrain repair consolidate-ties', 'duplicate-facts': 'gbrain repair duplicate-facts', 'orphan-children': 'gbrain repair orphan-children', 'failed-writes': 'gbrain repair failed-writes', 'split-facts': 'gbrain repair split-facts',frontmatter: 'gbrain repair frontmatter', 'timeline-comments': 'gbrain repair timeline-comments', 'timeline-order': 'gbrain repair timeline-order', 'contact-rows': 'gbrain repair contact-rows', 'managed-sync-orphans': 'gbrain repair managed-sync-orphans', 'content-receipt-root': 'gbrain repair content-receipt-root', 'frontmatter-keys': 'gbrain repair frontmatter-keys', 'file-drift': 'gbrain repair file-drift','stuck-recovery': 'gbrain repair stuck-recovery', 'ghost-pages': 'gbrain repair ghost-pages', 'origin-twins': 'gbrain repair origin-twins', 'moved-pages': 'gbrain repair moved-pages', 'receipt-compaction': 'gbrain repair receipt-compaction', aliases: 'gbrain repair aliases', 'rename-identity': 'gbrain repair rename-identity',diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-default.json b/test/fixtures/goldens/catalog/pglite-engine-init-default.jsonindex ae2d4c7204..cd801ccf9e 100644--- a/test/fixtures/goldens/catalog/pglite-engine-init-default.json+++ b/test/fixtures/goldens/catalog/pglite-engine-init-default.json@@ -1407,6 +1407,7 @@"idx_eval_capture_failures_ts": "CREATE INDEX idx_eval_capture_failures_ts ON public.eval_capture_failures USING btree (ts DESC)","idx_extract_rollup_7d_day": "CREATE INDEX idx_extract_rollup_7d_day ON public.extract_rollup_7d USING btree (day)","idx_facts_belief_stream": "CREATE INDEX idx_facts_belief_stream ON public.facts USING btree (source_id, source_markdown_slug) WHERE (expired_at IS NULL)",+ "idx_facts_context_head": "CREATE INDEX idx_facts_context_head ON public.facts USING btree (source_id, md5(split_part(context, ' — '::text, 1))) WHERE (source_markdown_slug IS NULL)","idx_facts_dimension": "CREATE INDEX idx_facts_dimension ON public.facts USING btree (source_id, entity_slug, dimension, valid_from DESC) WHERE ((expired_at IS NULL) AND (dimension IS NOT NULL))","idx_facts_embedding_hnsw": "CREATE INDEX idx_facts_embedding_hnsw ON public.facts USING hnsw (embedding halfvec_cosine_ops) WHERE ((embedding IS NOT NULL) AND (expired_at IS NULL))","idx_facts_embedding_model": "CREATE INDEX idx_facts_embedding_model ON public.facts USING btree (source_id, embedding_model) WHERE (expired_at IS NULL)",diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json b/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.jsonindex c3ef6c421a..ed49abea9f 100644--- a/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json+++ b/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json@@ -1407,6 +1407,7 @@"idx_eval_capture_failures_ts": "CREATE INDEX idx_eval_capture_failures_ts ON public.eval_capture_failures USING btree (ts DESC)","idx_extract_rollup_7d_day": "CREATE INDEX idx_extract_rollup_7d_day ON public.extract_rollup_7d USING btree (day)","idx_facts_belief_stream": "CREATE INDEX idx_facts_belief_stream ON public.facts USING btree (source_id, source_markdown_slug) WHERE (expired_at IS NULL)",+ "idx_facts_context_head": "CREATE INDEX idx_facts_context_head ON public.facts USING btree (source_id, md5(split_part(context, ' — '::text, 1))) WHERE (source_markdown_slug IS NULL)","idx_facts_dimension": "CREATE INDEX idx_facts_dimension ON public.facts USING btree (source_id, entity_slug, dimension, valid_from DESC) WHERE ((expired_at IS NULL) AND (dimension IS NOT NULL))","idx_facts_embedding_hnsw": "CREATE INDEX idx_facts_embedding_hnsw ON public.facts USING hnsw (embedding halfvec_cosine_ops) WHERE ((embedding IS NOT NULL) AND (expired_at IS NULL))","idx_facts_embedding_model": "CREATE INDEX idx_facts_embedding_model ON public.facts USING btree (source_id, embedding_model) WHERE (expired_at IS NULL)",diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json b/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.jsonindex a37e50c4e5..ec128e16ef 100644--- a/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json+++ b/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json@@ -1406,6 +1406,7 @@"idx_eval_capture_failures_ts": "CREATE INDEX idx_eval_capture_failures_ts ON public.eval_capture_failures USING btree (ts DESC)","idx_extract_rollup_7d_day": "CREATE INDEX idx_extract_rollup_7d_day ON public.extract_rollup_7d USING btree (day)","idx_facts_belief_stream": "CREATE INDEX idx_facts_belief_stream ON public.facts USING btree (source_id, source_markdown_slug) WHERE (expired_at IS NULL)",+ "idx_facts_context_head": "CREATE INDEX idx_facts_context_head ON public.facts USING btree (source_id, md5(split_part(context, ' — '::text, 1))) WHERE (source_markdown_slug IS NULL)","idx_facts_dimension": "CREATE INDEX idx_facts_dimension ON public.facts USING btree (source_id, entity_slug, dimension, valid_from DESC) WHERE ((expired_at IS NULL) AND (dimension IS NOT NULL))","idx_facts_embedding_model": "CREATE INDEX idx_facts_embedding_model ON public.facts USING btree (source_id, embedding_model) WHERE (expired_at IS NULL)","idx_facts_entity_active": "CREATE INDEX idx_facts_entity_active ON public.facts USING btree (source_id, entity_slug, valid_from DESC) WHERE (expired_at IS NULL)",diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-default.json b/test/fixtures/goldens/catalog/postgres-engine-init-default.jsonindex 790de74ac5..124d233fb3 100644--- a/test/fixtures/goldens/catalog/postgres-engine-init-default.json+++ b/test/fixtures/goldens/catalog/postgres-engine-init-default.json@@ -1409,6 +1409,7 @@"idx_eval_capture_failures_ts": "CREATE INDEX idx_eval_capture_failures_ts ON public.eval_capture_failures USING btree (ts DESC)","idx_extract_rollup_7d_day": "CREATE INDEX idx_extract_rollup_7d_day ON public.extract_rollup_7d USING btree (day)","idx_facts_belief_stream": "CREATE INDEX idx_facts_belief_stream ON public.facts USING btree (source_id, source_markdown_slug) WHERE (expired_at IS NULL)",+ "idx_facts_context_head": "CREATE INDEX idx_facts_context_head ON public.facts USING btree (source_id, md5(split_part(context, ' — '::text, 1))) WHERE (source_markdown_slug IS NULL)","idx_facts_dimension": "CREATE INDEX idx_facts_dimension ON public.facts USING btree (source_id, entity_slug, dimension, valid_from DESC) WHERE ((expired_at IS NULL) AND (dimension IS NOT NULL))","idx_facts_embedding_hnsw": "CREATE INDEX idx_facts_embedding_hnsw ON public.facts USING hnsw (embedding halfvec_cosine_ops) WHERE ((embedding IS NOT NULL) AND (expired_at IS NULL))","idx_facts_embedding_model": "CREATE INDEX idx_facts_embedding_model ON public.facts USING btree (source_id, embedding_model) WHERE (expired_at IS NULL)",diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json b/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.jsonindex a0177d37e2..c8dfbe02aa 100644--- a/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json+++ b/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json@@ -1409,6 +1409,7 @@"idx_eval_capture_failures_ts": "CREATE INDEX idx_eval_capture_failures_ts ON public.eval_capture_failures USING btree (ts DESC)","idx_extract_rollup_7d_day": "CREATE INDEX idx_extract_rollup_7d_day ON public.extract_rollup_7d USING btree (day)","idx_facts_belief_stream": "CREATE INDEX idx_facts_belief_stream ON public.facts USING btree (source_id, source_markdown_slug) WHERE (expired_at IS NULL)",+ "idx_facts_context_head": "CREATE INDEX idx_facts_context_head ON public.facts USING btree (source_id, md5(split_part(context, ' — '::text, 1))) WHERE (source_markdown_slug IS NULL)","idx_facts_dimension": "CREATE INDEX idx_facts_dimension ON public.facts USING btree (source_id, entity_slug, dimension, valid_from DESC) WHERE ((expired_at IS NULL) AND (dimension IS NOT NULL))","idx_facts_embedding_hnsw": "CREATE INDEX idx_facts_embedding_hnsw ON public.facts USING hnsw (embedding halfvec_cosine_ops) WHERE ((embedding IS NOT NULL) AND (expired_at IS NULL))","idx_facts_embedding_model": "CREATE INDEX idx_facts_embedding_model ON public.facts USING btree (source_id, embedding_model) WHERE (expired_at IS NULL)",diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json b/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.jsonindex d6715c056e..b2a1913b49 100644--- a/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json+++ b/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json@@ -1408,6 +1408,7 @@"idx_eval_capture_failures_ts": "CREATE INDEX idx_eval_capture_failures_ts ON public.eval_capture_failures USING btree (ts DESC)","idx_extract_rollup_7d_day": "CREATE INDEX idx_extract_rollup_7d_day ON public.extract_rollup_7d USING btree (day)","idx_facts_belief_stream": "CREATE INDEX idx_facts_belief_stream ON public.facts USING btree (source_id, source_markdown_slug) WHERE (expired_at IS NULL)",+ "idx_facts_context_head": "CREATE INDEX idx_facts_context_head ON public.facts USING btree (source_id, md5(split_part(context, ' — '::text, 1))) WHERE (source_markdown_slug IS NULL)","idx_facts_dimension": "CREATE INDEX idx_facts_dimension ON public.facts USING btree (source_id, entity_slug, dimension, valid_from DESC) WHERE ((expired_at IS NULL) AND (dimension IS NOT NULL))","idx_facts_embedding_model": "CREATE INDEX idx_facts_embedding_model ON public.facts USING btree (source_id, embedding_model) WHERE (expired_at IS NULL)","idx_facts_entity_active": "CREATE INDEX idx_facts_entity_active ON public.facts USING btree (source_id, entity_slug, valid_from DESC) WHERE (expired_at IS NULL)",diff --git a/test/fixtures/goldens/doctor/content-pglite-human.json b/test/fixtures/goldens/doctor/content-pglite-human.jsonindex bba8268e09..1d15c1f7ed 100644--- a/test/fixtures/goldens/doctor/content-pglite-human.json+++ b/test/fixtures/goldens/doctor/content-pglite-human.json@@ -161,6 +161,7 @@" [OK] generation_pending: No activated generation waits on a member request, and no orphan candidate exists."," [OK] extractor_facts_expired: No conversation-extractor fact is left expired by the pre-v0.60.11.0 projection."," [OK] unwithdrawn_facts: No fact is left retired by a facts-fence generation that no fence withdrew.",+ " [OK] purged_page_facts: No fact extracted from a page is left active after that page was purged."," [OK] conversation_label_facts: No conversation fact carries the pre-fix meeting-label signature (a 1970-01-01 segment on a label page)."," [OK] conversation_outcomes_stale: No conversation extraction outcome predates extractor version 1."," [OK] captured_facts_active: No active fact from a gbrain self-capture or paste candidate remains. This counts facts in the database; the self_capture check counts corpus files.",diff --git a/test/fixtures/goldens/doctor/content-pglite-json.json b/test/fixtures/goldens/doctor/content-pglite-json.jsonindex 223095c0c5..99b8750947 100644--- a/test/fixtures/goldens/doctor/content-pglite-json.json+++ b/test/fixtures/goldens/doctor/content-pglite-json.json@@ -2022,6 +2022,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {diff --git a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json b/test/fixtures/goldens/doctor/hot-checks-pglite-json.jsonindex 7cf9b504b1..eada6c5acc 100644--- a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json+++ b/test/fixtures/goldens/doctor/hot-checks-pglite-json.json@@ -2201,6 +2201,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {@@ -7569,6 +7583,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {diff --git a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json b/test/fixtures/goldens/doctor/hot-checks-postgres-json.jsonindex d58dce829e..a5f739ca2d 100644--- a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json+++ b/test/fixtures/goldens/doctor/hot-checks-postgres-json.json@@ -2213,6 +2213,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {@@ -7585,6 +7599,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {diff --git a/test/fixtures/goldens/doctor/json-pglite-degraded.json b/test/fixtures/goldens/doctor/json-pglite-degraded.jsonindex e79a1a39eb..4a53ffcd9e 100644--- a/test/fixtures/goldens/doctor/json-pglite-degraded.json+++ b/test/fixtures/goldens/doctor/json-pglite-degraded.json@@ -1899,6 +1899,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {diff --git a/test/fixtures/goldens/doctor/json-pglite-fresh.json b/test/fixtures/goldens/doctor/json-pglite-fresh.jsonindex 78d38a2079..69a82219f1 100644--- a/test/fixtures/goldens/doctor/json-pglite-fresh.json+++ b/test/fixtures/goldens/doctor/json-pglite-fresh.json@@ -1955,6 +1955,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {diff --git a/test/fixtures/goldens/doctor/json-postgres-fresh.json b/test/fixtures/goldens/doctor/json-postgres-fresh.jsonindex 1ff053a215..dc0443e18c 100644--- a/test/fixtures/goldens/doctor/json-postgres-fresh.json+++ b/test/fixtures/goldens/doctor/json-postgres-fresh.json@@ -1967,6 +1967,20 @@"name": "unwithdrawn_facts","status": "ok"},+ {+ "category": "brain",+ "details": {+ "ambiguous": 0,+ "capped": false,+ "docs": "docs/guides/repair.md#purged-page-facts",+ "evidenced": 0,+ "repair": "purged-page-facts"+ },+ "elapsed_ms": "<volatile>",+ "message": "No fact extracted from a page is left active after that page was purged.",+ "name": "purged_page_facts",+ "status": "ok"+ },{"category": "brain","details": {diff --git a/test/fixtures/goldens/doctor/registry-build-checks.json b/test/fixtures/goldens/doctor/registry-build-checks.jsonindex 72fe5b6a1d..8e8e6c5dc0 100644--- a/test/fixtures/goldens/doctor/registry-build-checks.json+++ b/test/fixtures/goldens/doctor/registry-build-checks.json@@ -705,6 +705,10 @@"category": "brain","name": "unwithdrawn_facts"},+ {+ "category": "brain",+ "name": "purged_page_facts"+ },{"category": "brain","name": "conversation_label_facts"diff --git a/test/fixtures/goldens/migrations/records.json b/test/fixtures/goldens/migrations/records.jsonindex 8290e64c5e..3457054ffa 100644--- a/test/fixtures/goldens/migrations/records.json+++ b/test/fixtures/goldens/migrations/records.json@@ -1,6 +1,6 @@{"golden": {- "count": 309,+ "count": 310,"gaps": [17,18,@@ -100,7 +100,7 @@"widenIdentityIndex": "86a6176bd7637d964dbc1d3cd3b1bdfa3ca0161e1ee248c37d7380a1471282e6","withSchemaLockTimeout": "95605bd3004955e11e936c7a4aa6889319b401098abf9db18ef3e48da2c3fc59"},- "latest": 314,+ "latest": 315,"records": [{"handler": {@@ -5090,6 +5090,34 @@"transaction": "absent","verify": "absent","version": 314+ },+ {+ "handler": {+ "form": "arrow",+ "helpers": [+ "PLANNER_REPAIR_TABLES",+ "analyzeAfterIndexBuild",+ "analyzeDeferred",+ "analyzeYieldingToVacuum",+ "buildIndexOnline",+ "dropInvalidConcurrentIndex",+ "lockTableForAnalyze",+ "migrationNotice",+ "quietMigrationNotices",+ "tableRows"+ ],+ "sha256": "fc5b4d215592b0754cf07f8622fc2631e6dc4064d68bc3e49a6aaf1eb6fbf690"+ },+ "idempotent": true,+ "name": "facts_context_head_index",+ "sql": {+ "length": 0,+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"+ },+ "sqlFor": "absent",+ "transaction": "absent",+ "verify": "absent",+ "version": 315}],"versions": [@@ -5401,7 +5429,8 @@311,312,313,- 314+ 314,+ 315]},"normalizer": "migrations-record-v1"diff --git a/test/fixtures/goldens/pglite-upgrade-replay/catalog-after-boot.json b/test/fixtures/goldens/pglite-upgrade-replay/catalog-after-boot.jsonindex c30ebd2d70..724a3e91e2 100644--- a/test/fixtures/goldens/pglite-upgrade-replay/catalog-after-boot.json+++ b/test/fixtures/goldens/pglite-upgrade-replay/catalog-after-boot.json@@ -1407,6 +1407,7 @@"idx_eval_capture_failures_ts": "CREATE INDEX idx_eval_capture_failures_ts ON public.eval_capture_failures USING btree (ts DESC)","idx_extract_rollup_7d_day": "CREATE INDEX idx_extract_rollup_7d_day ON public.extract_rollup_7d USING btree (day)","idx_facts_belief_stream": "CREATE INDEX idx_facts_belief_stream ON public.facts USING btree (source_id, source_markdown_slug) WHERE (expired_at IS NULL)",+ "idx_facts_context_head": "CREATE INDEX idx_facts_context_head ON public.facts USING btree (source_id, md5(split_part(context, ' — '::text, 1))) WHERE (source_markdown_slug IS NULL)","idx_facts_dimension": "CREATE INDEX idx_facts_dimension ON public.facts USING btree (source_id, entity_slug, dimension, valid_from DESC) WHERE ((expired_at IS NULL) AND (dimension IS NOT NULL))","idx_facts_embedding_hnsw": "CREATE INDEX idx_facts_embedding_hnsw ON public.facts USING hnsw (embedding halfvec_cosine_ops) WHERE ((embedding IS NOT NULL) AND (expired_at IS NULL))","idx_facts_embedding_model": "CREATE INDEX idx_facts_embedding_model ON public.facts USING btree (source_id, embedding_model) WHERE (expired_at IS NULL)",diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex c28b10da95..62bd7a36c9 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -606,6 +606,7 @@ test/repair-frontmatter.test.tstest/repair-ghost-database-ahead-gbra133.test.tstest/repair-lost-fence-rows.test.tstest/repair-private-labels-batch.test.ts+test/repair-purged-page-facts-8859.test.tstest/repair-quoted-markers-8059.test.tstest/repair-receipt-compaction.test.tstest/repair-rename-identity-6729.test.tsdiff --git a/test/repair-explicit-only.test.ts b/test/repair-explicit-only.test.tsindex f79f716544..1e761de029 100644--- a/test/repair-explicit-only.test.ts+++ b/test/repair-explicit-only.test.ts@@ -24,7 +24,7 @@ import { REPAIR_HELP, parseRepairArgs, runRepairCommand } from '../src/commands/import { _resetCliExitVerdictForTests, currentExitCode } from '../src/core/cli-force-exit.ts';import { withEnv } from './helpers/with-env.ts';-const EXPLICIT: RepairKind[] = ['accepted-takes', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'atom-fence-rows', 'extractor-facts', 'unwithdrawn-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'timeline-comments', 'timeline-order', 'contact-rows', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'moved-pages', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'segmented-tiling', 'uncommitted-deletions', 'generations', 'lost-fence-rows', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind', 'facts-home', 'private-labels'];+const EXPLICIT: RepairKind[] = ['accepted-takes', 'google-file-modes', 'stale-atoms', 'atoms-restore', 'atom-fence-rows', 'extractor-facts', 'unwithdrawn-facts', 'purged-page-facts', 'conversation-labels', 'captured-facts', 'agent-session-facts', 'redacted-facts', 'loop-facts', 'ontology-facts', 'facts-rehome', 'consolidate-ties', 'duplicate-facts', 'orphan-children', 'failed-writes', 'split-facts', 'frontmatter', 'timeline-comments', 'timeline-order', 'contact-rows', 'managed-sync-orphans', 'content-receipt-root', 'frontmatter-keys', 'file-drift', 'stuck-recovery', 'ghost-pages', 'origin-twins', 'moved-pages', 'receipt-compaction', 'aliases', 'rename-identity', 'overwritten-edits', 'legacy-twins', 'missing-tables', 'url-clone-fences', 'segmented-mirror', 'segmented-tiling', 'uncommitted-deletions', 'generations', 'lost-fence-rows', 'fact-claim-dates', 'provenance', 'secret-spans', 'rows-behind', 'facts-home', 'private-labels'];// GBRA-161: the generations preview carries its ledger kind and the restore window, so the printed command runs as is.const PREVIEW = (kind: RepairKind): string => kind === 'generations' ? 'gbrain repair generations --kind atom --since 30d' : `gbrain repair ${kind}`;let engine: PGLiteEngine;@@ -111,7 +111,7 @@ describe('surfaces that list instead of run', () => {expect(out.explicit_kinds).toEqual(EXPLICIT.map(kind => ({ kind, code: 'explicit_kind_required', preview_command: PREVIEW(kind),docs: 'docs/guides/repair.md#explicit-only-repair-kinds' })));}- expect(await captured(['--all'])).toContain('Explicit-only kinds (not run without their name; preview each): gbrain repair accepted-takes; gbrain repair google-file-modes; gbrain repair stale-atoms; gbrain repair atoms-restore; gbrain repair atom-fence-rows; gbrain repair extractor-facts; gbrain repair unwithdrawn-facts; gbrain repair conversation-labels; gbrain repair captured-facts; gbrain repair agent-session-facts; gbrain repair redacted-facts; gbrain repair loop-facts');+ expect(await captured(['--all'])).toContain('Explicit-only kinds (not run without their name; preview each): gbrain repair accepted-takes; gbrain repair google-file-modes; gbrain repair stale-atoms; gbrain repair atoms-restore; gbrain repair atom-fence-rows; gbrain repair extractor-facts; gbrain repair unwithdrawn-facts; gbrain repair purged-page-facts; gbrain repair conversation-labels; gbrain repair captured-facts; gbrain repair agent-session-facts; gbrain repair redacted-facts; gbrain repair loop-facts');});test('gbrain repair --all --apply exits 0 when explicit kinds are the only ones left', async () => {diff --git a/test/repair-purged-page-facts-8859.test.ts b/test/repair-purged-page-facts-8859.test.tsnew file mode 100644index 0000000000..cdf9c8ebfa--- /dev/null+++ b/test/repair-purged-page-facts-8859.test.ts@@ -0,0 +1,184 @@+/**+ * #8859 follow-up: facts that purges before the #8859 fix left behind (page-extraction rows filed on no page whose+ * context head names a page that no longer exists) are reported by a read-only doctor check and retired only by a+ * previewed, hash-bound, consented apply. By default only rows of a page the purge ledger records are retired; rows of+ * a page gone with no purge record need --include-ambiguous. Withdrawn, superseded and referenced rows, rows of a live+ * page and non-extraction rows stay active, and a row that was not in the preview is never touched. Runs on a managed+ * brain (PGLite and Postgres, one receipted request per batch) and an unmanaged PGLite brain.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { purgeDeletedPagesCoordinated } from '../src/core/persistence/purge-deleted.ts';+import { runRepairCommand } from '../src/commands/repair.ts';+import { _resetCliExitVerdictForTests } from '../src/core/cli-force-exit.ts';+import { purgedPageFactsCheck } from '../src/commands/doctor/checks/purged-page-facts.ts';+import { managedBrain } from './helpers/managed-brain.ts';+import { testBackends } from './helpers/test-backends.ts';++const PURGED = 'atoms/2026-09-30/example-claim';+const EXPIRED = 'meetings/example-meeting';+const LIVE = 'notes/live-example';++interface RepairJson { results: Array<{ residuals?: Record<string, number>; listing?: Array<{ item: string; class: string }>; apply_command: string; applied?: number }> }++async function capture(run: () => Promise<unknown>): Promise<{ out: string; exitCode: number | string | undefined }> {+ const lines: string[] = [];+ const { log, error } = console;+ const stdout = process.stdout.write.bind(process.stdout);+ const previous = process.exitCode;+ process.exitCode = 0;+ console.log = (...parts: unknown[]) => { lines.push(parts.map(String).join(' ')); };+ console.error = () => {};+ process.stdout.write = ((chunk: string | Uint8Array) => { lines.push(String(chunk)); return true; }) as typeof process.stdout.write;+ try { await run(); return { out: lines.join('\n'), exitCode: process.exitCode }; }+ finally { console.log = log; console.error = error; process.stdout.write = stdout; process.exitCode = previous ?? 0; _resetCliExitVerdictForTests(); }+}+async function repair(engine: BrainEngine, args: string[]): Promise<RepairJson> {+ const { out } = await capture(() => runRepairCommand(engine, ['purged-page-facts', ...args, '--json']));+ return JSON.parse(out) as RepairJson;+}+const hashOf = (json: RepairJson) => json.results[0]!.apply_command.match(/--expect ([0-9a-f]+)/)![1]!;+const active = async (engine: BrainEngine, ids: number[]) => (await engine.executeRaw<{ id: number }>(+ 'SELECT id FROM facts WHERE id = ANY($1::bigint[]) AND expired_at IS NULL ORDER BY id', [ids])).map(r => Number(r.id));++/** Rows as an older gbrain left them: written under replica role, so no current writer guard applies, and dated before+ * the purges the tests then record unless `recent`. */+const legacy = <T>(engine: BrainEngine, sql: string, params: unknown[]) => engine.transaction(async tx => {+ await tx.executeRaw('SET LOCAL session_replication_role = replica');+ return tx.executeRaw<T>(sql, params);+});+async function fact(engine: BrainEngine, text: string, context: string, opts: { source?: string; superseded_by?: number; recent?: boolean } = {}) {+ const [row] = await legacy<{ id: number }>(engine, `INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, notability, source, context, valid_from, superseded_by, created_at)+ VALUES ('default', NULL, $1, 'fact', 'private', 'high', $3, $2, now(), $4, now() - $5::interval) RETURNING id`,+ [text, context, opts.source ?? 'mcp:put_page', opts.superseded_by ?? null, opts.recent ? '0 days' : '3 days']);+ return Number(row!.id);+}++interface Seeded { purged: number; noted: number; expired: number; live: number; remembered: number; superseded: number }++/** Two gone pages (one purged, recorded in page_purges; one expired by retention, no record) and their leftover rows. */+async function seed(engine: BrainEngine, gone: (slug: string, purge: boolean) => Promise<void>): Promise<Seeded> {+ await gone(PURGED, true);+ await gone(EXPIRED, false);+ const purged = await fact(engine, 'Example claim one from the atom', PURGED);+ const noted = await fact(engine, 'Example claim two from the atom', `${PURGED} — entity inferred from text`);+ const expired = await fact(engine, 'Alice Example joined a new team', EXPIRED);+ const live = await fact(engine, 'A live page claim', LIVE);+ const remembered = await fact(engine, 'A remembered example claim', PURGED, { source: 'mcp:remember' });+ const superseded = await fact(engine, 'An older example claim', PURGED, { superseded_by: live });+ return { purged, noted, expired, live, remembered, superseded };+}++async function exercise(engine: BrainEngine, s: Seeded) {+ const all = Object.values(s);+ const check = await purgedPageFactsCheck(engine, ['default']);+ expect(check.status).toBe('warn');+ expect(check.details).toMatchObject({ evidenced: 2, ambiguous: 1 });++ const preview = await repair(engine, []);+ expect(preview.results[0]!.residuals).toMatchObject({ evidenced: 2, ambiguous: 1, excluded: 1 });+ expect(preview.results[0]!.listing!.map(l => [l.item, l.class]).sort()).toEqual([+ [`default:${PURGED}#${s.noted}`, 'evidenced'], [`default:${PURGED}#${s.purged}`, 'evidenced'],+ [`default:${PURGED}#${s.superseded}`, 'excluded:superseded'], [`default:${EXPIRED}#${s.expired}`, 'ambiguous']].sort());+ expect(await active(engine, all)).toEqual([...all].sort((a, b) => a - b));++ // A leftover row written after the preview is not in the approved set and stays active.+ const late = await fact(engine, 'A late example claim', PURGED);+ const hash = hashOf(preview);+ const unconsented = await capture(() => runRepairCommand(engine, ['purged-page-facts', '--apply', '--expect', hash, '--json']));+ expect(unconsented.exitCode).toBe(3);+ expect(await active(engine, [s.purged, s.noted])).toEqual([s.purged, s.noted].sort((a, b) => a - b));+ const wrong = await capture(() => runRepairCommand(engine, ['purged-page-facts', '--apply', '--expect', 'f'.repeat(64), '--yes', '--json'])).catch((e: Error) => ({ out: e.message }));+ expect(wrong.out).toContain('preview changed');++ await repair(engine, ['--apply', '--expect', hash, '--yes']);+ expect(await active(engine, [...all, late])).toEqual([s.expired, s.live, s.remembered, s.superseded, late].sort((a, b) => a - b));++ const widened = await repair(engine, ['--include-ambiguous']);+ expect(widened.results[0]!.residuals).toMatchObject({ evidenced: 1, ambiguous: 1 });+ await repair(engine, ['--include-ambiguous', '--apply', '--expect', hashOf(widened), '--yes']);+ expect(await active(engine, [...all, late])).toEqual([s.live, s.remembered, s.superseded].sort((a, b) => a - b));+ expect((await purgedPageFactsCheck(engine, ['default'])).status).toBe('ok');+}++/** Cells the main flow leaves out: every exclusion reason, a root-level slug, a fact newer than its slug's purge, and an unbounded context. */+async function edgeCases(engine: BrainEngine) {+ const old = async (text: string, context: string) => {+ const [row] = await legacy<{ id: number }>(engine, `INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, notability, source, context, valid_from, created_at)+ VALUES ('default', NULL, $1, 'fact', 'private', 'high', 'mcp:put_page', $2, now(), now() - interval '3 days') RETURNING id`, [text, context]);+ return Number(row!.id);+ };+ const withdrawn = await old('A withdrawn example claim', PURGED);+ await legacy(engine, "INSERT INTO fact_withdrawals (source_id, visibility, fact_hash) VALUES ('default', 'private', gbrain_fact_fingerprint($1))", ['A withdrawn example claim']);+ const referenced = await old('A replaced example claim', PURGED);+ await fact(engine, 'Its replacement', LIVE, { superseded_by: referenced, source: 'mcp:remember' });+ const looped = await old('An example commitment', PURGED);+ await legacy(engine, `INSERT INTO open_loops (source_id, dedup_key, loop_type, summary, detector, fact_id)+ VALUES ('default', 'example-loop', 'commitment_owed_by_me', 'An example commitment', 'manual', $1)`, [looped]);+ const root = await old('A root page claim', 'rootexample');+ const unrecordedRoot = await old('An unrecorded root claim', 'otherroot');+ const renamed = await fact(engine, 'A claim written after the purge', 'notes/reused-example', { recent: true });+ const long = await fact(engine, 'A claim with a long context', 'x'.repeat(6000));+ const longPath = await fact(engine, 'A claim with a long path context', `notes/` + 'y'.repeat(6000));+ await legacy(engine, "INSERT INTO page_purges (source_id, content_hash, slug, purged_at) VALUES ('default', $1, $4, now() - interval '1 day'), ('default', $2, $5, now() - interval '1 day'), ('default', $3, $6, now() - interval '1 day')",+ ['1'.repeat(64), '2'.repeat(64), '3'.repeat(64), PURGED, 'rootexample', 'notes/reused-example']);++ const preview = await repair(engine, []);+ expect(preview.results[0]!.listing!.map(l => [l.item, l.class]).sort()).toEqual([+ [`default:${PURGED}#${withdrawn}`, 'excluded:withdrawn'], [`default:${PURGED}#${referenced}`, 'excluded:superseded_by_reference'],+ [`default:${PURGED}#${looped}`, 'excluded:open_loop_reference'], [`default:rootexample#${root}`, 'evidenced'],+ [`default:notes/reused-example#${renamed}`, 'ambiguous'], [`default:notes/${'y'.repeat(6000)}#${longPath}`, 'ambiguous']].sort());+ await repair(engine, ['--include-ambiguous', '--apply', '--expect', hashOf(await repair(engine, ['--include-ambiguous'])), '--yes']);+ expect(await active(engine, [withdrawn, referenced, looped, root, unrecordedRoot, renamed, long, longPath]))+ .toEqual([withdrawn, referenced, looped, unrecordedRoot, long].sort((a, b) => a - b));+}++async function putPage(ctx: OperationContext, slug: string) {+ await submitPageMutation(ctx, { operation: 'put_page', params: { slug, source_id: 'default', request_id: randomUUID(),+ content: `---\ntype: note\ntitle: Example\n---\nA synthetic page at ${slug}.\n` } });+}++for (const backend of testBackends()) {+ describe(`#8859 purged-page-facts repair, managed (${backend})`, () => {+ const opts = backend === 'postgres' ? { databaseUrl: process.env.DATABASE_URL } : {};+ test('doctor reports, preview lists, apply retires only the approved rows', () => managedBrain(async ({ engine, ctx }) => {+ await putPage(ctx, LIVE);+ const seeded = await seed(engine, async (slug, purge) => {+ await putPage(ctx, slug);+ const live = await engine.readPageSnapshot(slug, { sourceId: 'default' });+ await submitPageMutation(ctx, { operation: 'delete_page', params: { slug, source_id: 'default', request_id: randomUUID(), expected_revision: live!.revision } });+ if (purge) {+ const snap = await engine.readPageSnapshot(slug, { sourceId: 'default', includeDeleted: true });+ await submitPageMutation(ctx, { operation: 'delete_page', params: { slug, source_id: 'default', purge: true, request_id: randomUUID(), expected_revision: snap!.revision } });+ } else await purgeDeletedPagesCoordinated(engine, 0);+ });+ await exercise(engine, seeded);+ }, opts), 240_000);+ test('exclusions, root slugs, reused slugs and long contexts', () => managedBrain(async ({ engine }) => edgeCases(engine), opts), 240_000);+ });+}++describe('#8859 purged-page-facts repair, unmanaged PGLite', () => {+ let engine: PGLiteEngine;+ beforeAll(async () => { engine = new PGLiteEngine(); await engine.connect({}); await engine.initSchema(); });+ afterAll(() => engine.disconnect());+ test('the same flow under the page keys', async () => {+ {+ await engine.putPage(LIVE, { type: 'note', title: 'Live', compiled_truth: 'A synthetic page.' });+ const seeded = await seed(engine, async (slug, purge) => {+ await engine.putPage(slug, { type: 'note', title: 'Gone', compiled_truth: `A synthetic page at ${slug}.` });+ await legacy(engine, "DELETE FROM pages WHERE source_id = 'default' AND slug = $1", [slug]);+ if (purge) await legacy(engine, "INSERT INTO page_purges (source_id, content_hash, slug) VALUES ('default', $1, $2)", ['0'.repeat(64), slug]);+ });+ await exercise(engine, seeded);+ }+ }, 120_000);+ test('exclusions, root slugs, reused slugs and long contexts', async () => {+ await engine.executeRaw("UPDATE facts SET expired_at = now() WHERE expired_at IS NULL");+ await edgeCases(engine);+ }, 120_000);+});diff --git a/test/trust-write-gate-drift.test.ts b/test/trust-write-gate-drift.test.tsindex dfd6e07dea..35b054e430 100644--- a/test/trust-write-gate-drift.test.ts+++ b/test/trust-write-gate-drift.test.ts@@ -61,6 +61,7 @@ const EXEMPT: Record<string, string> = {'src/core/generations/adapters/fact-rows.ts': 'generation restore puts back a fact row\'s own recorded before-state (lifecycle); no new text','src/core/repair/fact-claim-dates.ts': 'owner repair moving the valid_from of existing database-only facts to the date their claim states (lifecycle)','src/core/repair/extractor-facts.ts': 'owner repair of existing rows',+ 'src/core/repair/purged-page-facts.ts': 'owner repair behind a preview hash: only sets expired_at and clears row_num on existing fact rows (lifecycle); writes no new text','src/core/repair/unwithdrawn-facts.ts': 'owner repair behind a preview hash: puts back an existing fact row\'s own ledger before-state (lifecycle and row number); no new text','src/core/generations/fence-evidence.ts': 'renumbers a kept existing fact row inside the gated fence projection or reconcile; no new text','src/core/repair/secret-spans.ts': 'owner repair behind a preview hash: replaces only the credential spans the secret scanner finds in a page\'s stored copies with <REDACTED:rule> markers (no new text)',