Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
repair redacted-facts restores conversation-extractor facts that have no Facts fence row (#8820) (#9384)
Garry Tanaee3bee1dc7e
diff --git a/changelog.d/capy-redacted-facts-legacy-rows.md b/changelog.d/capy-redacted-facts-legacy-rows.mdnew file mode 100644index 0000000000..5676c83762--- /dev/null+++ b/changelog.d/capy-redacted-facts-legacy-rows.md@@ -0,0 +1,6 @@+**`gbrain repair redacted-facts` now restores facts the conversation extractor wrote, which keep a row number on a page with no Facts fence (#8820).**++### Fixed+- `src/core/repair/redacted-facts.ts`: a fact with a row number is treated as a Facts-fence row only while its page's fence holds an active row with its text at that number. A conversation-extractor fact with no such fence row is now restored in place, keeping its row number, by the same row-exact rewrite as a database-only fact; its page is not changed. Before, the preview listed these facts as `evidenced` and the apply left them damaged with `changed_since_preview`. Any other fact whose fence row is gone is now listed as `unproven` (`fence_row_missing`) instead of `evidenced`, so the preview no longer promises a restore the apply cannot make. The repair is still preview-first: `--apply --expect <hash>` restores only the previewed set. Reported by @tarush1989.+- Docs: `docs/guides/repair.md` (redacted facts class table and apply paragraph).+- Tests: `test/repair-redacted-facts.test.ts` (PGLite, and Postgres through `test/e2e/repair-redacted-facts-postgres.test.ts`): a managed brain restores an extractor fact and leaves its page revision unchanged; an unmanaged brain restores an extractor fact and keeps a non-extractor fact whose fence row is gone. Both failed before the fix on both engines.diff --git a/docs/architecture/canonical-writers.tsv b/docs/architecture/canonical-writers.tsvindex 9a6cad1da1..46a213977c 100644--- a/docs/architecture/canonical-writers.tsv+++ b/docs/architecture/canonical-writers.tsv@@ -242,7 +242,7 @@ src/core/source-delete.ts 1 engine_guard Unmanaged source delete with its incarnsrc/core/repair/extractor-facts.ts 1 coordinator `gbrain repair extractor-facts` (#5731) clears expired_at on preview-approved conversation-extractor facts and gives each a fresh row number. On a managed brain the UPDATE runs only inside the managed_maintenance_restore_extractor_facts publication (withCoordinatedWrite, page key locked, each fact reclassified first); an unmanaged brain runs the same function in one transaction under the page lock.src/core/repair/captured-facts.ts 1 coordinator `gbrain repair captured-facts` expires preview-approved capture-lane facts that have no fence row. On a managed brain the UPDATE runs only inside the managed_maintenance_expire_captured_facts publication (owner-checked by maintenancePreflight, page key locked); an unmanaged brain runs it in one transaction. Fenced rows are never updated here: they are struck in the page and expired by the canonical projection of a revision-bound put_page.src/core/repair/agent-session-facts.ts 2 coordinator gbrain repair agent-session-facts (#8064) re-attributes preview-approved database-only capture facts to the agent (attributed_to other plus a speaker note) and retires fact/self takes whose every contributing fact is then non-user. On a managed brain both UPDATEs run only inside the managed_maintenance_agent_session_facts publication (owner-checked by maintenancePreflight, page key locked); an unmanaged brain runs them in one maintenanceTransaction. The UPDATEs recheck attribution, activity and fence position under the row lock. Fenced facts and takes are never updated here: they are rewritten in the page through a revision-bound put_page and the canonical projection rewrites the rows in place.-src/core/repair/redacted-facts.ts 1 coordinator `gbrain repair redacted-facts` (#7125) rewrites the text of preview-approved database-only facts to the words a source page proves. On a managed brain the UPDATE runs only inside the managed_maintenance_restore_redacted_facts publication (owner-checked by maintenancePreflight, page key locked); an unmanaged brain runs it in one maintenanceTransaction. The UPDATE compares text, visibility, entity and holding page under the row lock. Fenced rows are never updated here: their claim is rewritten in the page through a revision-bound put_page and the canonical projection replaces the row.+src/core/repair/redacted-facts.ts 1 coordinator `gbrain repair redacted-facts` (#7125) rewrites the text of preview-approved database-only and conversation-extractor facts to the words a source page proves. On a managed brain the UPDATE runs only inside the managed_maintenance_restore_redacted_facts publication (owner-checked by maintenancePreflight, page key locked); an unmanaged brain runs it in one maintenanceTransaction. The UPDATE compares text, visibility, entity and holding page under the row lock. Fenced rows are never updated here: their claim is rewritten in the page through a revision-bound put_page and the canonical projection replaces the row.src/core/repair/secret-spans.ts 2 coordinator `gbrain repair secret-spans` (GST-327) replaces only the credential spans the secret scanner finds in the preview-approved pages' stored copies with <REDACTED:rule> markers, one page per transaction under the page key after a digest recheck: inside withCoordinatedWrite with maintenance attribution on a managed brain, withWriteAttribution on an unmanaged one. Database only: the file side of a redaction goes through Git, history is kept, and a sync hold stays.src/core/repair/ontology-facts.ts 1 coordinator `gbrain repair ontology-facts` (#6264) gives preview-approved ontology observations their own provenance back, clears their fence row number and expiry. On a managed brain the UPDATE runs only inside coordinatedDatabaseWrite (the ontology_propose path: write authority checked, source incarnation revalidated, entity and page keys locked); an unmanaged brain runs it in one maintenance transaction. It writes database-only ontology rows and never a page or file.src/core/repair/facts-rehome.ts 1 coordinator `gbrain repair facts-rehome` (B-7, #6925) moves preview-approved database-only facts off a page-less entity_slug onto the page the strict resolver reaches. On a managed brain the UPDATE runs only inside coordinatedDatabaseWrite (the relink_facts operation: write authority checked, source incarnation revalidated, both slugs locked); an unmanaged brain runs one maintenance transaction per slug. It rechecks the slug, target and fact ids against the preview and never writes a page or file.diff --git a/docs/architecture/key-files/core-utilities-2.md b/docs/architecture/key-files/core-utilities-2.mdindex 9da55b1a13..78dd2faa06 100644--- a/docs/architecture/key-files/core-utilities-2.md+++ b/docs/architecture/key-files/core-utilities-2.md@@ -11,7 +11,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/repair/extractor-facts.ts` — `gbrain repair extractor-facts` (#5731), explicit-only and preview-bound. `classifyExtractorFacts` (read-only; also the doctor check's count) takes conversation-extractor facts with `expired_at` set and `row_num` NULL and classes each: `excluded` (page missing or deleted, `superseded_by` set, withdrawn, an active same-text fact on the page, or an earlier same-text candidate), `ambiguous` (unmanaged brain; no committed request of the page with `completed_at = expired_at`, compared in SQL as microsecond UTC text, which only a same-transaction completion satisfies on Postgres; a consumer at or above v0.60.11.0; or the page has ever held fence-projected facts) or `evidenced`. The preview lists every candidate, hashes them with the selection (`--include-ambiguous`) and saves one approved item per page (with its request id) through `preview-approval.ts`; the apply loads only that set, replays a still-pending request with the ctx's CLI write wait, skips pages whose request already committed or whose facts are active again, and restores each still-identical fact with a fresh `row_num` above the page's facts and fence rows. Managed brains publish one database-only `managed_maintenance_restore_extractor_facts` request per page (`submitDatabaseMaintenanceIntent`, dispatched by `prepareMaintenanceMutation` to `prepareExtractorFactsRestore`, which reclassifies under the page lock); unmanaged brains run the same restore in one locked transaction. Previews warn, naming the host, when a consumer older than this release published after `writer_version_cutoff`. On PGLite `now()` has millisecond resolution, so the same-transaction rule is millisecond-exact there.- `src/core/repair/conversation-labels.ts` — `gbrain repair conversation-labels`, explicit-only, preview-bound and destructive (consent payload from the registry's `consentText`). `classifyLabelFacts` (read-only) takes active per-segment extractor facts (`source` exactly `cli:extract-conversation-facts`) on live pages with a bold `METADATA_LABELS` line and classes each `excluded` (page outcome stamped with `extractor_version`, withdrawn, superseded, named by another fact's `superseded_by`, referenced by `open_loops.fact_id`), `evidenced` (context carries `segment 1970-01-01`) or `ambiguous`. The preview decides each page's outcome with the current parser at no cost (`non_extractable`, `awaiting_reextraction`, `cleaned`), hashes it all, saves one approved item per page and prints the capped hand-off (`extract-conversation-facts --slugs … --dry-run`, then `--max-cost-usd`). The apply expires the approved ids still as previewed (appending `retired: conversation-labels <hash>` to the context), expires the completion marker and writes the stamped not-extractable outcome for a page still at the previewed parser input; approved pages apply in batches of `LABEL_BATCH_PAGES` (25): one database-only `managed_maintenance_conversation_label_retire` request per batch on `LABELS_ANCHOR_SLUG` on a managed brain (`prepareConversationLabelRetirement`, every page key locked, each page rechecked), one transaction per batch otherwise. No model calls. Segmented pages are not classified (the label filter reads the body columns): their unstamped per-segment facts are counted as `segmented_pages` / `segmented_facts` and named in a warning. `countLabelFacts` is doctor `conversation_label_facts`' bounded count. Pinned by `test/repair-conversation-labels.test.ts` (+ Postgres arm) and `test/conversation-facts-retained-history.test.ts`.- `src/core/repair/captured-facts.ts` — `gbrain repair captured-facts`, explicit-only and preview-bound. `classifyCapturedFacts` (read-only; also the `captured_facts_active` count) takes active capture-lane facts and classes each by its session (`captureSessionId`: `sweep:corpus:<file>` resolves through `corpusFileSessionId`): `evidenced` when a gbrain claude-cli scratch project holds the session's harness transcript or its corpus file sits in `<corpus>.quarantine/`; `ambiguous` (paste candidate) when a retained corpus file of the session holds a paste block and at least 60% of the claim's content words occur there only inside paste blocks; `excluded:legitimate_duplicate` when the claim and entity also have an active row that is not proven suspect. Sessions with no transcript and no corpus file are counted `unclassifiable`. The preview hashes every candidate with the selection (`--include-ambiguous`) and saves one approved item per page through `preview-approval.ts`; the apply reclassifies each fact, expires database-only rows through one database-only `managed_maintenance_expire_captured_facts` request per page on a managed brain (owner-checked by `maintenancePreflight`; `prepareCapturedFactsExpiry`, dispatched from `prepareMaintenanceMutation`) or one transaction on an unmanaged brain, and strikes fenced rows in the page's `## Facts` fence through one revision-bound `put_page`, so the canonical projection expires them and a later page write cannot reactivate them. No withdrawal is recorded, so `remember` can save the same claim again. Pinned by `test/repair-captured-facts.test.ts` (+ Postgres arm).-- `src/core/repair/redacted-facts.ts` — `gbrain repair redacted-facts` (#7125), explicit-only and preview-bound. `classifyRedactedFacts` (read-only) takes active facts whose text holds a redaction marker and runs `restoreNeutralized` (`think/sanitize.ts`, with the pre-fix `LEGACY_EVAL_SHELL` shape) against the page the fact's context names and the page holding it (`pageTexts`: an inline page's body + timeline, a segmented page's body through `readBodyRange` up to 16 MiB, then up to 50 `page_versions` bodies): `evidenced` when every proving text proves the same restoration, `ambiguous` (`conflicting_sources`) when a text holds two spans around the words or texts disagree, `unproven` (`no_source_page` / `not_in_source`) otherwise. A proved fenced row on a segmented page is `segmented` (`segmented_page`): listed with its restoration, never rewritten (`restoreFencedRows` reads and publishes inline bodies only). The preview hashes every candidate with its restoration and saves one approved item per page; the apply reclassifies each fact, rewrites database-only rows through one database-only `managed_maintenance_restore_redacted_facts` request per page on a managed brain (`prepareRedactedFactsRestore`, dispatched from `prepareMaintenanceMutation`) or one transaction on an unmanaged brain (exact-text compare-and-set; context gets `restored: redacted-facts repair`), and rewrites fenced rows' claims in the page's `## Facts` fence through one revision-bound `put_page`, so the canonical projection replaces the row. Pinned by `test/repair-redacted-facts.test.ts` (+ Postgres arm).+- `src/core/repair/redacted-facts.ts` — `gbrain repair redacted-facts` (#7125), explicit-only, preview-bound. `classifyRedactedFacts` (read-only) takes active facts holding a redaction marker and runs `restoreNeutralized` (with the pre-fix `LEGACY_EVAL_SHELL` shape) against the page its context names and the page holding it (`pageTexts`: body + timeline, a segmented body via `readBodyRange` up to 16 MiB, then up to 50 `page_versions` bodies): `evidenced` when every proving text agrees, `ambiguous` (`conflicting_sources`) when a text holds two spans or texts disagree, `unproven` (`no_source_page` / `not_in_source`) otherwise. A numbered row is fenced only while its page's fence holds an active row with its text at that number; else an extractor row takes the database route (#8820) and any other is `unproven` (`fence_row_missing`). A proved fenced row on a segmented page is `segmented` (`segmented_page`): listed with its restoration, never rewritten. The preview hashes every candidate and saves one approved item per page; the apply reclassifies each fact, rewrites database-route rows through one database-only `managed_maintenance_restore_redacted_facts` request per page on a managed brain (`prepareRedactedFactsRestore`, dispatched from `prepareMaintenanceMutation`) or one transaction on an unmanaged brain (exact compare-and-set; context gets `restored: redacted-facts repair`), and rewrites fenced rows' claims in the page's `## Facts` fence through one revision-bound `put_page`, so the projection replaces the row. Pinned by `test/repair-redacted-facts.test.ts` (+ Postgres arm).- `src/core/repair/loop-facts.ts` — `gbrain repair loop-facts` (#5869), explicit-only and preview-bound (`--apply --expect <hash>`): candidates are `loopFactDrift` rows (closed `done`/`dropped` loops whose commitment fact is active, in the loop's source and shared with no open loop); each item is rechecked (`changed_since_preview`) and retired through `retireLoopFact`, the same coordinated strike-plus-expire publication `loops_close` uses.- `src/core/repair/failed-writes.ts` — `gbrain repair failed-writes` (#5983), explicit-only and preview-bound: candidates are failed, uncompacted `persistence_requests` refused by the managed writer guard (`writer_coordinator_required`, or `storage_error` "Publication failed (P0001)"). Caller writes (`put_page` without an internal `kind`, `add_timeline_entry`, `remember`) are classified `replay` / `already_written` / `duplicate` / `superseded` / `unpinned_target` by later requests on the same page and by a committed replay request id, and `file_database_drift` (W4.11) when its last replay attempt was refused `source_changed` and the page has not been written since (`replayWalks`, set reads; the detail and an apply's `file_database_drift` outcome name `gbrain sources reconcile <source> <slug> --preview`; `test/repair-failed-writes-drift-w411.test.ts`); owner-produced writes are `producer_owned` residuals with the producer command (a failed `managed_file_repair` names both `gbrain repair frontmatter` and `gbrain repair fences`). #7602 fact rows: `factsRecovery` ([Facts](core-persistence-facts.md)). The approved set records each target page's revision. Apply resumes an admitted replay attempt, otherwise reclassifies and rechecks the revision, re-runs `authorizeStoredRequest` on the failed row, and resubmits the retained intent on the original trust lane (`laneContext`: remote, take holders, delegated prefixes) through `submitPageMutation` / `submitRememberMutation`, with `put_page` bound to the previewed revision and `remember` pinned to its resolved subject, under a request id derived from the failed request id and attempt (`repair-failed-writes-5983.test.ts`). #5994: `laneContext` sets `ctx.replayAuthority` to the stored authority, so admission (`submissionAuthority` → `replayedAuthority` in `persistence/authority.ts`, and `requestPrincipalForContext`) reuses its principal, delegation, scopes and autoLinkTrusted for the same source incarnation and re-authorizes it live; a restricted write gets back its recorded job (`delegatedJobId`, or the id in a single legacy `wiki/agents/<id>/*` prefix, replayed on the legacy sandboxed path when its original was), and one with neither is accepted by `enforceSubagentSlugFence` (`ops/context.ts`) only through that marker with the exact non-empty stored allow-list. The fence still fails closed for every live dispatch. A put_page replay passes the stored `timeline_section` (#5969). The superseded / already-written / duplicate rule lives in `persistence/terminal-classify.ts`, shared with the #6650 terminal-failure readers.- `src/core/repair/ontology-facts.ts` — `gbrain repair ontology-facts` (#6264), explicit-only and preview-bound (`--apply --expect <hash>`). `classifyOntologyFacts` (read-only; also doctor `ontology_facts_fenced`) takes ontology rows (`dimension IS NOT NULL`) carrying the pre-fix fence step's signature, a `row_num` or a `source_markdown_slug` that differs from `source` (`mergeOntologyFact` writes them equal and never sets `row_num`), and classes each `fenced` or `retired`, or excluded as `withdrawn` (`fact_withdrawals`), `consolidated` or `duplicate` (restoring would collide with the ontology dedup key). The apply sets `row_num` NULL, `source_markdown_slug = source` and `expired_at` NULL in one statement that rechecks row number, page, expiry and the dedup key (`changed_since_preview` otherwise), through `coordinatedDatabaseWrite('ontology_propose')` on the entity and page keys on a managed brain or one maintenance transaction otherwise; the withdrawal trigger re-expires a claim forgotten after the preview (`withdrawn`). Page text is never rewritten. Tests: `test/ontology-fence-sweep.test.ts`.diff --git a/docs/architecture/system-of-record.md b/docs/architecture/system-of-record.mdindex 15cd31e0b8..c8592810dd 100644--- a/docs/architecture/system-of-record.md+++ b/docs/architecture/system-of-record.md@@ -321,7 +321,7 @@ or a physical projection that leaves attribution untouched):- `src/core/repair/ghost-pages.ts` (2): `gbrain repair ghost-pages` retirement: a receipted `managed_maintenance_retire_ghost_page` request (a prepared `delete_page` with no file bytes) on managed brains; the unmanaged version and soft delete run in `maintenanceTransaction` after the same recheck.- `src/core/repair/moved-pages.ts` (1): `gbrain repair moved-pages` fold: a receipted `managed_maintenance_fold_moved_page` request on the old page that also holds the moved page's key, on managed brains; the unmanaged fold runs in `maintenanceTransaction` under both page locks after the same recheck.- `src/core/repair/ontology-facts.ts` (1): a coordinated database-only write (`coordinatedDatabaseWrite`, the `ontology_propose` path) on managed brains; the unmanaged ontology-facts restore runs in `maintenanceTransaction`.-- `src/core/repair/redacted-facts.ts` (1): a `managed_maintenance_restore_redacted_facts` request on managed brains; the unmanaged restore of database-only fact text runs in `maintenanceTransaction`.+- `src/core/repair/redacted-facts.ts` (1): a `managed_maintenance_restore_redacted_facts` request on managed brains; the unmanaged restore of database-only and conversation-extractor fact text runs in `maintenanceTransaction`.- `src/core/repair/rename-identity.ts` (5): `gbrain repair rename-identity` carry, unmanaged brains only (a managed brain refuses `managed_remedy_unavailable` before writing); the fence-row merge, the provisional page's retirement, the slug move and the import run in one `maintenanceTransaction` under both page locks after the plan-hash recheck.- `src/core/repair/secret-spans.ts` (1): `gbrain repair secret-spans` (GST-327) rewrites the credential spans of each preview-approved page under the page key: inside `withCoordinatedWrite` with maintenance attribution on managed brains, `withWriteAttribution` with maintenance attribution on unmanaged ones.- `src/core/repair/segmented-tiling.ts` (1): `gbrain repair segmented-tiling` (GST-1209) versions each preview-approved damaged page (`createVersion`) before republishing it from the file Git holds at HEAD, under the page key: inside `withCoordinatedWrite` with maintenance attribution on managed brains, `withWriteAttribution` with maintenance attribution on unmanaged ones.diff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex fe316a5757..ec146e8461 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -2792,14 +2792,16 @@ matches. A page stored in segments is read as evidence up to 16 MiB.| --- | --- | --- || `evidenced` | Every text that proves a restoration proves the same one. | By `--apply --expect <hash>` || `ambiguous` | A text holds two different spans around the same words, or two texts prove different restorations. | Never; listed |-| `unproven` | No page it came from exists any more (`no_source_page`), or none still holds the words around the marker (`not_in_source`). | Never; listed |+| `unproven` | No page it came from exists any more (`no_source_page`), or none still holds the words around the marker (`not_in_source`), or the fact's row number names no active row with its text in its page's `## Facts` fence and it did not come from the conversation extractor (`fence_row_missing`; the listing shows the restored text). | Never; listed || `segmented` | Proved, but the fact's row sits in the `## Facts` fence of a page stored in segments, which this repair does not rewrite (`segmented_page`). The listing shows the restored text. | Never; listed |The apply restores exactly the previewed set. A fact that changed since thepreview reports `changed_since_preview` and keeps its text. A fact with a rowin its page's `## Facts` fence has that row's claim rewritten in the page(one revision-bound `put_page` per page, which re-chunks and re-embeds the-page); a fact with no fence row is rewritten database-only (one maintenance+page); a fact with no row number, or a conversation-extractor fact whose+row number names no fence row (its page has no `## Facts` fence), is+rewritten database-only, row number kept (one maintenancerequest per page on a managed brain) and gets `restored: redacted-factsrepair` appended to its context; its vector no longer matches its text, sothe next `gbrain embed` re-embeds it. Page history keeps the damageddiff --git a/src/core/repair/redacted-facts.ts b/src/core/repair/redacted-facts.tsindex 2fe84cc13b..a7cc422733 100644--- a/src/core/repair/redacted-facts.ts+++ b/src/core/repair/redacted-facts.ts@@ -27,6 +27,12 @@* - segmented: proved, but the row sits in the facts fence of a page* stored in segments, which this repair does not rewrite; listed with* its restoration, never changed.+ * A row with a row number is a fence row only while its page's `## Facts`+ * fence holds an active row with its text at that number. A conversation-extractor row keeps the+ * row number the extractor gave it on a page with no such row (#8820) and is+ * restored like a database-only row; any other row whose fence row is gone+ * is unproven (`fence_row_missing`), listed with its restoration, never+ * changed.* A segmented source page is read as evidence through the segment reader* (up to 16 MiB).*@@ -34,7 +40,8 @@* previewed. A fenced row (`row_num` on its page) gets its claim rewritten in* the page's `## Facts` fence by one revision-bound `put_page` per page, so* the canonical projection replaces the row and the page's chunks are- * re-embedded by its publication. A database-only row gets its text rewritten+ * re-embedded by its publication. A database-only or conversation-extractor+ * row gets its text rewritten* by one database-only maintenance request on a managed brain (one* transaction on an unmanaged brain); its vector is then stale by its text* hash and `gbrain embed` re-embeds it. Page history keeps the damaged@@ -43,7 +50,7 @@import type { BrainEngine } from '../engine.ts';import { opError, OperationError } from '../ops/contract.ts';import { readFix } from '../ops/op-fix.ts';-import { parseFactsFence } from '../facts-fence.ts';+import { parseFactsFence, parseFactsFenceFor } from '../facts-fence.ts';import { strikeFenceRow } from '../facts/forget.ts';import { serializePageToMarkdown } from '../markdown.ts';import { digest } from '../persistence/digest.ts';@@ -76,8 +83,10 @@ export interface RedactedFactCandidate {/** The page the fact lives on (fenced rows) or its entity key (database-only rows). */slug: string;row_num: number | null;+ /** `fence`: rewritten in its page's Facts fence; `database`: rewritten in place (no row number, or a conversation-extractor row with no fence row). */+ route: 'fence' | 'database';class: RedactedFactClass;- /** `restorable`, `conflicting_sources`, `no_source_page`, `not_in_source` or `segmented_page`. */+ /** `restorable`, `conflicting_sources`, `no_source_page`, `not_in_source`, `segmented_page` or `fence_row_missing`. */reason: string;/** The stored text, and the restored text when evidenced (or proved on a segmented page). */fact: string;@@ -98,7 +107,7 @@ interface PageItem extends RepairItem { page: RedactedFactsPage; hash: string; linterface FactRow {id: number | string; source_id: string; fact: string; visibility: string; entity_slug: string | null;- row_num: number | null; source_markdown_slug: string | null; context: string | null;+ row_num: number | null; source_markdown_slug: string | null; context: string | null; source: string | null;}const factHash = (row: FactRow) => digest(['redacted-fact-v1', row.fact, row.visibility, row.entity_slug, row.row_num, row.source_markdown_slug]);@@ -109,7 +118,7 @@ const contextSlug = (context: string | null) => {return /^[^\s|]+$/.test(first) ? first : null;};-interface PageTexts { segmented: boolean; texts: Array<{ label: string; text: string }> }+interface PageTexts { segmented: boolean; fenceRows: Map<number, string>; fenceUncertain: boolean; writableRows: Map<number, string>; texts: Array<{ label: string; text: string }> }/** A segmented body's stored text, read segment by segment; null when it is over TEXT_MAX_BYTES (it proves nothing). */async function segmentedText(db: BrainEngine, pageId: number): Promise<string | null> {@@ -132,25 +141,31 @@ async function segmentedText(db: BrainEngine, pageId: number): Promise<string |*/async function pageTexts(db: BrainEngine, sourceId: string, slug: string): Promise<PageTexts> {const snapshot = await db.readPageSnapshot(slug, { sourceId }).catch(() => null);- if (!snapshot) return { segmented: false, texts: [] };+ if (!snapshot) return { segmented: false, fenceRows: new Map(), fenceUncertain: false, writableRows: new Map(), texts: [] };const texts: PageTexts['texts'] = [];const segmented = isSegmentedPage(snapshot.page);+ let fenceBody: string | null;if (segmented) {- const text = await segmentedText(db, snapshot.page.id);- if (text !== null) texts.push({ label: slug, text });+ fenceBody = await segmentedText(db, snapshot.page.id);+ if (fenceBody !== null) texts.push({ label: slug, text: fenceBody });} else {+ fenceBody = snapshot.page.compiled_truth ?? '';texts.push({ label: slug, text: [snapshot.page.compiled_truth ?? '', snapshot.page.timeline ?? ''].join('\n') });}const versions = await db.executeRaw<{ id: number | string; compiled_truth: string }>(`SELECT id, compiled_truth FROM page_versionsWHERE page_id=$1 AND compiled_truth IS NOT NULL ORDER BY id DESC LIMIT ${VERSIONS_PER_PAGE}`, [snapshot.page.id]);for (const v of versions) texts.push({ label: `${slug}@version:${v.id}`, text: v.compiled_truth });- return { segmented, texts };+ const activeRows = (parsed: ReturnType<typeof parseFactsFence> | null) => new Map((parsed?.facts ?? []).filter(row => row.active).map(row => [row.rowNum, row.claim.trim()]));+ const fence = fenceBody === null ? null : parseFactsFenceFor(fenceBody, { slug });+ const writable = fenceBody === null ? null : parseFactsFence(fenceBody);+ return { segmented, fenceRows: activeRows(fence), fenceUncertain: !fence || fence.warnings.length > 0,+ writableRows: activeRows(writable?.warnings.length ? null : writable), texts };}/** Read-only: every active fact of these sources whose text holds a redaction marker, classified, in (source, page, id) order. */export async function classifyRedactedFacts(db: BrainEngine, sourceIds: string[], opts: { ids?: number[] } = {}): Promise<RedactedFactCandidate[]> {const rows = await db.executeRaw<FactRow>(`- SELECT f.id, f.source_id, f.fact, f.visibility, f.entity_slug, f.row_num, f.source_markdown_slug, f.context+ SELECT f.id, f.source_id, f.fact, f.visibility, f.entity_slug, f.row_num, f.source_markdown_slug, f.context, f.sourceFROM facts fWHERE f.source_id=ANY($1::text[]) AND f.expired_at IS NULL AND f.superseded_by IS NULLAND f.fact LIKE '%[redacted%' AND ($2::bigint[] IS NULL OR f.id=ANY($2::bigint[]))@@ -176,12 +191,16 @@ export async function classifyRedactedFacts(db: BrainEngine, sourceIds: string[]}}const proved = !conflicting && proofs.size === 1;- const held = proved && row.row_num !== null && row.source_markdown_slug !== null- && (await sourceTexts(row.source_id, row.source_markdown_slug)).segmented;+ const holder = row.row_num !== null && row.source_markdown_slug !== null ? await sourceTexts(row.source_id, row.source_markdown_slug) : null;+ const extractor = (row.source ?? '').startsWith('cli:extract-conversation-facts');+ const inFence = !!holder && (holder.fenceRows.get(Number(row.row_num)) === row.fact.trim() || holder.segmented && !extractor || extractor && holder.fenceUncertain);+ const route = row.row_num === null || !inFence && extractor ? 'database' : 'fence';+ const fenceGone = route === 'fence' && holder?.writableRows.get(Number(row.row_num)) !== row.fact.trim();const [klass, reason]: [RedactedFactClass, string] = conflicting || proofs.size > 1 ? ['ambiguous', 'conflicting_sources']- : held ? ['segmented', 'segmented_page'] : proved ? ['evidenced', 'restorable'] : ['unproven', read ? 'not_in_source' : 'no_source_page'];+ : proved && holder?.segmented && route === 'fence' ? ['segmented', 'segmented_page'] : proved && fenceGone ? ['unproven', 'fence_row_missing']+ : proved ? ['evidenced', 'restorable'] : ['unproven', read ? 'not_in_source' : 'no_source_page'];const [restored, evidence] = proved ? [...proofs][0]! : [null, [...proofs.values()].flat()];- candidates.push({ id: Number(row.id), source_id: row.source_id, slug: targetSlug(row), row_num: row.row_num === null ? null : Number(row.row_num),+ candidates.push({ id: Number(row.id), source_id: row.source_id, slug: targetSlug(row), row_num: row.row_num === null ? null : Number(row.row_num), route,class: klass, reason, fact: row.fact, restored, evidence, visibility: row.visibility, entity_slug: row.entity_slug,source_markdown_slug: row.source_markdown_slug, fact_hash: factHash(row) });}@@ -194,6 +213,7 @@ const DETAILS: Record<string, string> = {no_source_page: 'no page it came from exists any more, so nothing can prove the original words; left unchanged',not_in_source: 'the words around the marker are not in any page it came from, so nothing proves the original words; left unchanged',segmented_page: 'its row sits in the facts fence of a page stored in segments, and this repair rewrites a fence by publishing the whole body; left unchanged',+ fence_row_missing: 'its row number names no active row with its text in its page\'s Facts fence, so neither the fence nor the row can be rewritten safely; left unchanged',};function previewCommand(scope: RepairScope, all: string[]): string {@@ -206,7 +226,7 @@ async function activeSources(engine: BrainEngine): Promise<string[]> {async function hashParts(engine: BrainEngine, scope: RepairScope, facts: RedactedFactCandidate[]) {const [brain] = await engine.executeRaw<{ brain_id: string }>('SELECT brain_id FROM persistence_brain WHERE singleton=1').catch(() => []);- return ['redacted-facts-v1', brain?.brain_id ?? 'host', scope.source_ids, facts.map(f => [f.id, f.class, f.reason, f.fact_hash, f.restored])];+ return ['redacted-facts-v1', brain?.brain_id ?? 'host', scope.source_ids, facts.map(f => [f.id, f.class, f.reason, f.fact_hash, f.restored, f.route])];}function pageItem(page: RedactedFactsPage, hash: string, index: number, last: boolean): PageItem {@@ -258,12 +278,12 @@ export const redactedFactsRepair: RepairHandler = {const live = new Map((await classifyRedactedFacts(engine, [page.source_id], { ids: page.facts.map(f => f.id) })).map(c => [c.id, c]));const unchanged = page.facts.filter(f => {const current = live.get(f.id);- return !!current && (['class', 'fact_hash', 'slug', 'row_num', 'restored'] as const).every(field => current[field] === f[field]);+ return !!current && (['class', 'fact_hash', 'slug', 'row_num', 'route', 'restored'] as const).every(field => current[field] === f[field]);});const restored: number[] = [];- const dbOnly = unchanged.filter(f => f.row_num === null);+ const dbOnly = unchanged.filter(f => f.route === 'database');if (dbOnly.length) restored.push(...await restoreDatabaseRows(engine, page, hash, dbOnly));- const fenced = unchanged.filter(f => f.row_num !== null);+ const fenced = unchanged.filter(f => f.route === 'fence');if (fenced.length) restored.push(...await restoreFencedRows(ctx, entry, page, fenced));if (last) await clearApprovedSet(engine, { command: 'redacted-facts', hash });const changed = page.facts.filter(f => !restored.includes(f.id)).map(f => f.id);@@ -272,8 +292,8 @@ export const redactedFactsRepair: RepairHandler = {},};-/** One approved database-only rewrite: the row as previewed (text, visibility, entity, holding page) and its restored text. */-export interface Restore { id: number; from: string; to: string; visibility: string; entity_slug: string | null; source_markdown_slug: string | null }+/** One approved in-place rewrite: the row as previewed (text, visibility, entity, holding page, row number) and its restored text. */+export interface Restore { id: number; from: string; to: string; visibility: string; entity_slug: string | null; source_markdown_slug: string | null; row_num?: number | null }/*** Rewrites each row still exactly as previewed. The compare runs in the@@ -283,21 +303,22 @@ export interface Restore { id: number; from: string; to: string; visibility: str*/export const restoreRedactedRows = (tx: BrainEngine, sourceId: string, restores: Restore[]) => tx.executeRaw<{ id: number | string }>(`UPDATE facts fSET fact=r.to_text, context=concat_ws(' | ', NULLIF(f.context,''), $3::text)- FROM jsonb_to_recordset($2::text::jsonb) AS r(id bigint, from_text text, to_text text, visibility text, entity_slug text, source_markdown_slug text)+ FROM jsonb_to_recordset($2::text::jsonb) AS r(id bigint, from_text text, to_text text, visibility text, entity_slug text, source_markdown_slug text, row_num int)WHERE f.source_id=$1 AND f.id=r.id AND f.fact=r.from_text AND f.visibility=r.visibility AND f.entity_slug IS NOT DISTINCT FROM r.entity_slug- AND f.source_markdown_slug IS NOT DISTINCT FROM r.source_markdown_slug AND f.expired_at IS NULL AND f.superseded_by IS NULL AND f.row_num IS NULL+ AND f.source_markdown_slug IS NOT DISTINCT FROM r.source_markdown_slug AND f.expired_at IS NULL AND f.superseded_by IS NULL+ AND f.row_num IS NOT DISTINCT FROM r.row_num AND (r.row_num IS NULL OR f.source LIKE 'cli:extract-conversation-facts%')RETURNING f.id`,[sourceId, JSON.stringify(restores.map(r => ({ id: r.id, from_text: r.from, to_text: r.to, visibility: r.visibility, entity_slug: r.entity_slug,- source_markdown_slug: r.source_markdown_slug }))), RESTORED_CONTEXT]);+ source_markdown_slug: r.source_markdown_slug, row_num: r.row_num ?? null }))), RESTORED_CONTEXT]);/**- * Database-only rows: on a managed brain one database-only maintenance+ * Database-only and conversation-extractor rows: on a managed brain one database-only maintenance* request on the page key (owner-checked, with a publication receipt); on an* unmanaged brain one transaction.*/async function restoreDatabaseRows(engine: BrainEngine, page: RedactedFactsPage, hash: string, facts: RedactedFactCandidate[]): Promise<number[]> {const restores: Restore[] = facts.map(f => ({ id: f.id, from: f.fact, to: f.restored!, visibility: f.visibility, entity_slug: f.entity_slug,- source_markdown_slug: f.source_markdown_slug }));+ source_markdown_slug: f.source_markdown_slug, row_num: f.row_num }));if (!await managedPersistenceEnabled(engine)) {return (await maintenanceTransaction(engine, tx => restoreRedactedRows(tx, page.source_id, restores))).map(row => Number(row.id));}@@ -310,13 +331,14 @@ async function restoreDatabaseRows(engine: BrainEngine, page: RedactedFactsPage,return (receipt.restored as number[] | undefined) ?? [];}-/** Preparer for `managed_maintenance_restore_redacted_facts`: rewrites the named database-only rows still holding their previewed text. */+/** Preparer for `managed_maintenance_restore_redacted_facts`: rewrites the named database-only and conversation-extractor rows still holding their previewed text. */export async function prepareRedactedFactsRestore(engine: BrainEngine, row: WriteRequest): Promise<PreparedMutation> {const restores = (row.intent as { restores?: unknown } | null)?.restores;const text = (v: unknown) => typeof v === 'string';const valid = Array.isArray(restores) && restores.every(r => !!r && typeof r === 'object' && Number.isSafeInteger((r as Restore).id)&& text((r as Restore).from) && text((r as Restore).to) && text((r as Restore).visibility)- && ((r as Restore).entity_slug === null || text((r as Restore).entity_slug)) && ((r as Restore).source_markdown_slug === null || text((r as Restore).source_markdown_slug)));+ && ((r as Restore).entity_slug === null || text((r as Restore).entity_slug)) && ((r as Restore).source_markdown_slug === null || text((r as Restore).source_markdown_slug))+ && ((r as Restore).row_num == null || Number.isSafeInteger((r as Restore).row_num)));if (!valid) {throw opError('invalid_params', 'The redacted facts restore intent does not name its facts.',`Request ${row.request_id} in source ${row.source_id} does not name the facts to restore, so nothing changed. Preview the repair again and apply the new preview after the user approves.`,diff --git a/test/repair-redacted-facts.test.ts b/test/repair-redacted-facts.test.tsindex a8525754eb..93c1a30dec 100644--- a/test/repair-redacted-facts.test.ts+++ b/test/repair-redacted-facts.test.ts@@ -57,6 +57,12 @@ async function refusal(run: () => Promise<unknown>): Promise<OperationError> {const factText = async (engine: BrainEngine, id: number) =>(await engine.executeRaw<{ fact: string; active: boolean }>('SELECT fact, expired_at IS NULL AS active FROM facts WHERE id=$1', [id]))[0];+const CONVERSATION = 'conversations/planning-call';+const insertExtractorFact = async (db: BrainEngine, fact: string, rowNum: number) => Number((await db.executeRaw<{ id: number }>(`INSERT INTO facts+ (source_id, entity_slug, fact, kind, visibility, source, context, row_num, source_markdown_slug)+ VALUES ('default', $1, $2, 'fact', 'private', 'cli:extract-conversation-facts', $3, $4, $3) RETURNING id`, [ACME, fact, CONVERSATION, rowNum]))[0].id);+const extractorRow = async (engine: BrainEngine, id: number) => (await engine.executeRaw<{ fact: string; row_num: number; source_markdown_slug: string; active: boolean }>(+ 'SELECT fact, row_num::int AS row_num, source_markdown_slug, expired_at IS NULL AS active FROM facts WHERE id=$1', [id]))[0];const putPage = (ctx: OperationContext, slug: string, content: string) =>submitPageMutation(ctx, { operation: 'put_page', params: { request_id: randomUUID(), slug, source_id: 'default', content } });@@ -155,6 +161,22 @@ ${SOURCE_TEXT}expect((await factText(engine, id)).fact).toBe(EXOS_DAMAGED);}, { databaseUrl });}, 180_000);++ test('#8820: a conversation-extractor row (legacy row number, no Facts fence) is restored in place; its page is unchanged', async () => {+ await managedBrain(async ({ engine, ctx }) => {+ await putPage(ctx, CONVERSATION, `---\ntype: note\ntitle: Planning call\n---\n${SOURCE_TEXT}\n`);+ const before = (await engine.readPageSnapshot(CONVERSATION, { sourceId: 'default' }))!;+ const id = await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => insertExtractorFact(tx, ABS_DAMAGED, 3), TEST_WRITE_ATTRIBUTION));+ const preview = await repair(engine, []);+ expect(preview.results[0].residuals).toEqual({ evidenced: 1, ambiguous: 0, unproven: 0, segmented: 0 });+ expect((await repair(engine, applyArgs(preview))).results[0].outcomes).toEqual({ restored: 1 });+ expect(await extractorRow(engine, id)).toEqual({ fact: ABS_RESTORED, row_num: 3, source_markdown_slug: CONVERSATION, active: true });+ const after = (await engine.readPageSnapshot(CONVERSATION, { sourceId: 'default' }))!;+ expect(after.revision).toBe(before.revision);+ expect(after.page.compiled_truth).toBe(before.page.compiled_truth);+ expect((await factsRedactionMarkerCheck(engine)).status).toBe('ok');+ }, { databaseUrl });+ }, 180_000);});describe(`gbrain repair redacted-facts, unmanaged (${backend})`, () => {@@ -204,6 +226,75 @@ ${SOURCE_TEXT}await engine.executeRaw('DELETE FROM facts WHERE id=$1', [fenced]);}, 120_000);+ test('#8820: a conversation-extractor row on a segmented page with no Facts fence is restored in place', async () => {+ const conversation = 'conversations/segmented-call';+ await createSegmentedPage(engine, { slug: conversation, truth: `${'Filler line.\n'.repeat(90_000)}${SOURCE_TEXT}\n` });+ const id = Number((await engine.executeRaw<{ id: number }>(`INSERT INTO facts+ (source_id, entity_slug, fact, kind, visibility, source, context, row_num, source_markdown_slug)+ VALUES ('default', $1, $2, 'fact', 'private', 'cli:extract-conversation-facts', $3, 3, $3) RETURNING id`, [ACME, ABS_DAMAGED, conversation]))[0].id);+ const preview = await repair(engine, []);+ const listed = Object.fromEntries(preview.results[0].listing!.map(l => [Number(l.item.split('#')[1]), l.class]));+ expect(listed[id]).toBe('evidenced');+ expect((await repair(engine, applyArgs(preview))).results[0].outcomes).toEqual({ restored: 1 });+ expect(await extractorRow(engine, id)).toEqual({ fact: ABS_RESTORED, row_num: 3, source_markdown_slug: conversation, active: true });+ await engine.executeRaw('DELETE FROM facts WHERE id=$1', [id]);+ }, 120_000);++ test('#8820: an extractor row whose text sits in a segmented holding page\'s entity-column fence stays on the fence route and is left unchanged', async () => {+ const hold = 'memory/unattributed';+ const conversation = 'conversations/held-call';+ await engine.putPage(conversation, { type: 'note', title: 'Held call', compiled_truth: SOURCE_TEXT }, { sourceId: 'default' });+ await createSegmentedPage(engine, { slug: hold, truth: [`${'Filler line.\n'.repeat(90_000)}`, '## Facts', '', '<!--- gbrain:facts:begin -->',+ '| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | entity |',+ '|---|---|---|---|---|---|---|---|---|---|---|',+ `| 2 | ${ABS_DAMAGED} | fact | 1.0 | private | medium | 2026-11-11 | | chat 2026-11-11 | | ${ACME} |`,+ '<!--- gbrain:facts:end -->', ''].join('\n') });+ const id = Number((await engine.executeRaw<{ id: number }>(`INSERT INTO facts+ (source_id, entity_slug, fact, kind, visibility, source, context, row_num, source_markdown_slug)+ VALUES ('default', $1, $2, 'fact', 'private', 'cli:extract-conversation-facts', $3, 2, $4) RETURNING id`, [ACME, ABS_DAMAGED, conversation, hold]))[0].id);+ const preview = await repair(engine, []);+ const listed = Object.fromEntries(preview.results[0].listing!.map(l => [Number(l.item.split('#')[1]), l.class]));+ expect(listed[id]).toBe('segmented');+ expect(preview.results[0].residuals.evidenced).toBe(0);+ expect((await factText(engine, id)).fact).toBe(ABS_DAMAGED);+ await engine.executeRaw('DELETE FROM facts WHERE id=$1', [id]);+ }, 120_000);++ test('a non-extractor row in an inline holding page\'s entity-column fence is not listed as restorable, because the fence writer cannot rewrite that fence', async () => {+ const hold = 'memory/unattributed';+ const conversation = 'conversations/held-inline-call';+ await engine.executeRaw("DELETE FROM pages WHERE slug=$1 AND source_id='default'", [hold]);+ await engine.putPage(conversation, { type: 'note', title: 'Held inline call', compiled_truth: SOURCE_TEXT }, { sourceId: 'default' });+ await engine.putPage(hold, { type: 'note', title: 'Unattributed', compiled_truth: ['## Facts', '', '<!--- gbrain:facts:begin -->',+ '| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | entity |',+ '|---|---|---|---|---|---|---|---|---|---|---|',+ `| 2 | ${ABS_DAMAGED} | fact | 1.0 | private | medium | 2026-11-11 | | chat 2026-11-11 | | ${ACME} |`,+ '<!--- gbrain:facts:end -->', ''].join('\n') }, { sourceId: 'default' });+ const id = Number((await engine.executeRaw<{ id: number }>(`INSERT INTO facts+ (source_id, entity_slug, fact, kind, visibility, source, context, row_num, source_markdown_slug)+ VALUES ('default', $1, $2, 'fact', 'private', 'sync:import', $3, 2, $4) RETURNING id`, [ACME, ABS_DAMAGED, conversation, hold]))[0].id);+ const preview = await repair(engine, []);+ const listed = Object.fromEntries(preview.results[0].listing!.map(l => [Number(l.item.split('#')[1]), l.class]));+ expect(listed[id]).toBe('unproven');+ expect(preview.results[0].residuals.evidenced).toBe(0);+ await engine.executeRaw('DELETE FROM facts WHERE id=$1', [id]);+ }, 120_000);++ test('#8820: a conversation-extractor row is restored in place; a fenced row number whose fence row is gone is not', async () => {+ await engine.putPage(CONVERSATION, { type: 'note', title: 'Planning call', compiled_truth: SOURCE_TEXT }, { sourceId: 'default' });+ const id = await insertExtractorFact(engine, ABS_DAMAGED, 3);+ const other = Number((await engine.executeRaw<{ id: number }>(`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, source, row_num, source_markdown_slug)+ VALUES ('default', NULL, $1, 'fact', 'private', 'mcp:put_page', 4, $2) RETURNING id`, [EXOS_DAMAGED, CONVERSATION]))[0].id);+ const preview = await repair(engine, []);+ const listed = Object.fromEntries(preview.results[0].listing!.map(l => [Number(l.item.split('#')[1]), l.class]));+ expect(listed[id]).toBe('evidenced');+ expect(listed[other]).not.toBe('evidenced');+ expect((await repair(engine, applyArgs(preview))).results[0].outcomes).toEqual({ restored: 1 });+ expect(await extractorRow(engine, id)).toEqual({ fact: ABS_RESTORED, row_num: 3, source_markdown_slug: CONVERSATION, active: true });+ expect((await factText(engine, other)).fact).toBe(EXOS_DAMAGED);+ await engine.executeRaw('DELETE FROM facts WHERE id = ANY($1::bigint[])', [[id, other]]);+ }, 120_000);+test('the publication rewrites a row only while it is exactly as previewed (a change after the recheck wins)', async () => {const id = Number((await engine.executeRaw<{ id: number }>(`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, source, context)VALUES ('default', $1, $2, 'fact', 'private', 'sync:import', $3) RETURNING id`, [ACME, EXOS_DAMAGED, NOTES]))[0].id);