Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Share purge's saved-version rewrite with the held-row move, report unedited segmented versions, and preview each held fact's real destination (GST-2492)

garrytan247afed7dc92
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 52b4336a10..4f472e394e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,12 +23,13 @@ identifiers and attribution are available in the pre-removal Git revision
- `src/core/persistence/coordinator.ts`: a held fact on a database-only folder's holding page reports `write_through: {written: false, skipped: db_only, held_for}` instead of `no_repo_configured`.
- `src/core/search/private-visibility.ts`: a held fact is also judged by its holding page as a page, so a raw-only rule (`covers_derived: false`) hides it as it hides the page; a fact still on `memory/unattributed` for an entity a rule now covers (a rule synced from Git, until maintenance moves it) reads only for that rule's readers, and for nobody while the rule is pending.
- `src/core/entities/resolve.ts`: a holding page under an entity directory (`people/memory/unattributed`) is never an entity page itself.
-- `src/commands/access-rules.ts`: the rule preview lists `held_facts_moving` and the apply moves them (`held_facts: {moved, unmovable}`), for the database and git stores; `gbrain access apply` and `gbrain access resume` move them too.
-- `src/core/facts/fence-fact-append.ts`, `src/core/persistence/prepared-maintenance.ts`: a held row that leaves the shared page for a page not every reader can read also leaves the page's saved versions, in the same transaction; every other byte of each version is kept, Git history is not rewritten, and the move's write receipt records it (`held_versions_edited`: the rows' fact ids, the row and version counts, where they went). The rule preview counts the rows and versions (`held_versions_editing`).
-- `src/commands/sync/perform.ts`: after a managed sync releases its lease, held facts that a folder rule it applied now covers move at once; the maintenance cycle stays the fallback.
+- `src/commands/access-rules.ts`: the rule preview lists `held_facts_moving` and the apply moves them (`held_facts: {moved, unmovable}`), for the database and git stores; `gbrain access apply` and `gbrain access resume` move them too. Each listed fact's `to` comes from `heldRowDestination`, the resolution the move itself uses, so a row saved under a bare name shows the page that name resolves to.
+- `src/core/facts/fence-fact-append.ts`, `src/core/persistence/prepared-maintenance.ts`: a held row that leaves the shared page for a page not every reader can read also leaves the page's saved versions, in the same transaction; every other byte of each version is kept, Git history is not rewritten, and the move's write receipt records it (`held_versions_edited`: the rows' fact ids, the row and version counts, where they went). The rule preview counts the rows and versions (`held_versions_editing`). A saved version stored in segments is not edited: the preview names how many (`segmented_versions_not_edited`, and a line saying readers of the shared page's saved versions can still read the moved rows there), and the apply's `held_facts` says `complete: false` with the same words.
+- `src/core/facts/purge.ts`: purge's saved-version redaction moves, unchanged in SQL and transaction, into `rewriteVersionBodies`, which the held-row edit above also calls, so the page body reader ratchet still lists one saved-version reader.
+- `src/commands/sync/perform.ts`, `src/core/cycle/extract-facts.ts`: after a managed sync releases its lease, held facts that a folder rule it applied now covers move at once; the maintenance cycle stays the fallback. Either path warns when the move left a segmented saved version unedited (`HELD_VERSIONS_NOT_EDITED` in the cycle).
- `src/core/repair/facts-home.ts`, `src/core/facts/facts-home.ts`: `gbrain repair facts-home` writes a ruled entity's fact to its folder's page; `folder_restricted` now means only `page_rule` or `access_unreadable`.
- Docs: `git-canonical.md`, `entity-scoped-facts.md`, `access-designations.md`, `repair.md`, `write-refusals.md`, `memory-boundaries.md`, `MEMORY_VERBS_v1.md`, `error-codes.md` (regenerated).
-- Tests: new `test/folder-holding-page-gst2492.test.ts` (held in Git before the reply and kept by a rebuild from a clone; three nested rules plus a pending one; no existence leak on get_page, search, query, recall, list_pages and find_pages; a raw-only rule; the rule apply's preview and move; the owner with the opt-in on and off; the page-rule and unreadable refusals locally and over MCP; database-only folders; background writers and repair; non-strict and repo-less unchanged); updated GST-2492 suites; PGLite and Postgres.
+- Tests: new `test/folder-holding-page-gst2492.test.ts` (held in Git before the reply and kept by a rebuild from a clone; three nested rules plus a pending one; no existence leak on get_page, search, query, recall, list_pages and find_pages; a raw-only rule; the rule apply's preview and move; the owner with the opt-in on and off; the page-rule and unreadable refusals locally and over MCP; database-only folders; background writers and repair; non-strict and repo-less unchanged; a segmented saved version named by the preview and reported incomplete by the apply, the sync path and maintenance; the preview's destination for a bare-name row equal to where the apply puts it); updated GST-2492 suites; PGLite and Postgres.
## [0.60.699.0] - 2026-11-12
diff --git a/docs/architecture/system-of-record.md b/docs/architecture/system-of-record.md
index c68768080c..5eb36312a0 100644
--- a/docs/architecture/system-of-record.md
+++ b/docs/architecture/system-of-record.md
@@ -257,14 +257,14 @@ or a physical projection that leaves attribution untouched):
- `src/core/facts/conversation-publication.ts` (6): the managed conversation-facts preparer clears each member page's prior rows and inserts its frozen entry (or a blocked outcome row) inside the coordinator publication of its admitted batch request, clearing only for a complete replacement (#7013); the unmanaged writer runs the same publication in `maintenanceTransaction`. Its page clear also expires the rows `gbrain facts relink --conversation` linked (after deleting the previous clear's expired copies), inside the same transaction (#6764). The segmented incremental run, unmanaged only, re-anchors moved windows' facts and stamps fresh rows' windows (`source_window`) in the same `maintenanceTransaction`.
- `src/core/facts/conversation-relink.ts` (1): `gbrain facts relink --conversation`: a `managed_maintenance_conversation_relink` request on managed brains; the unmanaged relink runs in `writeDerivedFacts` (`maintenanceTransaction`).
- `src/core/facts/derivation-inputs.ts` (3): `hideDerivedRows` expires derived facts, deactivates derived takes and soft-deletes derived pages inside the purge's coordinated transaction, attributed to the purge request.
-- `src/core/facts/fence-fact-append.ts` (2): `prepareFenceFactAppend` inserts the facts row of a fence row the request's page file holds, in its caller's coordinated publication after the page row (remember and its callers). `heldRowsInVersions` removes moved held rows from the shared page's saved versions inside the held-move publication's after-effects, attributed to the move request.
+- `src/core/facts/fence-fact-append.ts` (1): `prepareFenceFactAppend` inserts the facts row of a fence row the request's page file holds, in its caller's coordinated publication after the page row (remember and its callers).
- `src/core/facts/fence-write.ts` (1): the markdown-first fence reconcile insert runs in `maintenanceTransaction` (its page body mirror goes through `facts/file-body-guard.ts`).
- `src/core/facts/file-body-guard.ts` (2): the #6982 checked body mirror (`replaceStoredBody`: the page_versions row and the body refresh, under the page row lock) runs in its caller's `maintenanceTransaction` (fence write, unfenced-facts fence, phantom redirect, forget, proposal supersede).
- `src/core/facts/gone-page-facts.ts` (2): a retention purge expires the purged pages' facts in its delete transaction, in `maintenanceTransaction` (the bulk purge and the per-tombstone purge of a source the belief ledger records; `purgeDeletedPagesCoordinated`).
- `src/core/facts/forget.ts` (4): the legacy fence expiry, withdrawal and strike-through each run in `maintenanceTransaction` (nested inside a caller's coordinated write, they keep its actor).
- `src/core/facts/ontology-rekey.ts` (1): B-10's duplicate-observation drop runs in its caller's transaction immediately before the caller's re-key: the rename's `maintenanceTransaction`, the `relink_facts` publication, the phantom redirect's fact move and `repair facts-rehome`'s move; every caller is attributed.
- `src/core/facts/proposal-supersede.ts` (4): unmanaged `decide` proposal accept and undo run in one `maintenanceTransaction` each; managed brains publish a `decide_proposal` request.
-- `src/core/facts/purge.ts` (3): `purge_fact` rewrites purged pages' bodies, redacts their versions and replaces those versions' change reasons with `[purged]` inside `withCoordinatedWrite` with the purge request's attribution.
+- `src/core/facts/purge.ts` (3): `purge_fact` rewrites purged pages' bodies, redacts their versions and replaces those versions' change reasons with `[purged]` inside `withCoordinatedWrite` with the purge request's attribution. Its saved-version rewrite (`rewriteVersionBodies`) also removes moved held rows from the shared holding page's saved versions (`heldRowsInVersions` in `fence-fact-append.ts`), inside the held-move publication's after-effects, attributed to the move request.
- `src/core/facts/relink-publish.ts` (2): `relink_facts` request preparer.
- `src/core/facts/unfenced-facts.ts` (1): the cycle and v0.32.2 fence of `row_num`-NULL facts: a maintenance request on managed brains; the unmanaged row-number stamp runs in `maintenanceTransaction` (the body mirror goes through `facts/file-body-guard.ts`).
- `src/core/facts/withdrawal.ts` (1): runs inside the `forget` request's coordinated write.
diff --git a/docs/guides/git-canonical.md b/docs/guides/git-canonical.md
index 5e1f6b0eda..8938e77eee 100644
--- a/docs/guides/git-canonical.md
+++ b/docs/guides/git-canonical.md
@@ -200,7 +200,13 @@ Every write goes through the same door, whoever sends it:
maintenance cycle moves them. The move also removes those rows from the
shared page's saved versions in the database (the rule's preview counts
them, and the move's write receipt records which rows, how many versions and
- where they went). Git history is never rewritten: the shared page's earlier
+ where they went). A saved version stored in segments (a shared page over
+ 1 MiB with `pages.segmented_storage` on) is not edited: the preview names how
+ many there are (`segmented_versions_not_edited`), the apply's result says it
+ is incomplete (`complete: false`), and readers of the shared page's saved
+ versions can still read the moved rows in up to that many versions. The
+ preview names each held fact's destination as the move resolves it, so a row
+ saved under a bare name shows the page that name resolves to. Git history is never rewritten: the shared page's earlier
commits keep the row as it was before the rule. An entity under an exact page rule (a rule on the
entity's own slug) has no holding page with its readers: the fact is refused
with [`git_home_required`](write-refusals.md#git_home_required) (reason
diff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsv
index 8734c79475..c08d87a12d 100644
--- a/scripts/page-body-readers-baseline.tsv
+++ b/scripts/page-body-readers-baseline.tsv
@@ -228,8 +228,6 @@ src/core/facts/backstop.ts runPipeline field STREAM
src/core/facts/conversation-window-extract.ts regularPageVersionToken field STREAM
src/core/facts/drain.ts estimateBacklogUsd sql META
src/core/facts/eligibility.ts isFactsBackstopEligible field LEAD
-src/core/facts/fence-fact-append.ts heldRowsInVersions field SEGW GST-2492 A2, as purge's redactPageVersions: rewrites inline version bodies only; a segmented version is counted (segmented_versions_not_edited on the receipt), never read
-src/core/facts/fence-fact-append.ts heldRowsInVersions sql SEGW GST-2492 A2, as purge's redactPageVersions: selects inline version bodies; a segmented version is selected only to be counted
src/core/facts/fence-fact-append.ts prepareFenceFactAppend call:serializePageToMarkdown SEGW assertInlineBody first
src/core/facts/fence-fact-append.ts prepareFenceFactAppend field SEGW assertInlineBody before the fence write
src/core/facts/fence-fact-append.ts prepareFenceRowRetirement field SEGW the body materializedPage read back within PAGE_MAX_BYTES; a segmented page over the ceiling refuses (segmentedOverCeiling) first
@@ -255,11 +253,11 @@ src/core/facts/proposal-supersede.ts prepareProposalMutation field SEGW segmente
src/core/facts/proposal-supersede.ts undoUnmanaged field SEGW segmented page: getter throws page_body_segmented
src/core/facts/purge-verify.ts carriesClaimSql sql STREAM strpos over NULL never matches a segmented page; withSegmentedProse probes its segment texts
src/core/facts/purge-verify.ts verifyPagePurge sql STREAM segmented pages probed through their segment texts (withSegmentedProse)
-src/core/facts/purge.ts redactPageVersions field SEGW lane 1e: segmented versions carry no body
-src/core/facts/purge.ts redactPageVersions sql SEGW lane 1e: segmented versions carry no body
src/core/facts/purge.ts rewritePageBodies call:contentHash META inline pages only (see sql row)
src/core/facts/purge.ts rewritePageBodies field SEGW segmented page: getter throws page_body_segmented
src/core/facts/purge.ts rewritePageBodies sql SEGW refuses a segmented target with page_body_segmented before rewriting (segment purge in PR 2)
+src/core/facts/purge.ts rewriteVersionBodies field SEGW lane 1e: segmented versions carry no body
+src/core/facts/purge.ts rewriteVersionBodies sql SEGW lane 1e: segmented versions carry no body
src/core/facts/reconcile-watermark.ts readFactsReconcileBacklog sql STREAM
src/core/facts/reconcile-watermark.ts selectFactsReconcileDrain sql STREAM
src/core/facts/relink-publish.ts prepareRelinkMutation call:serializePageToMarkdown SEGW
diff --git a/src/commands/access-rules.ts b/src/commands/access-rules.ts
index 48c8ba6c84..e25818ea7e 100644
--- a/src/commands/access-rules.ts
+++ b/src/commands/access-rules.ts
@@ -123,7 +123,7 @@ export interface RulePreview {
*/
held_facts_moving: Array<{ fact_id: number; entity_slug: string; to: string }>;
/** The rows of those facts in the shared page's saved versions, and those versions, the apply removes them from (database only; the version the move itself saves is edited too). */
- held_versions_editing: { rows: number; versions: number };
+ held_versions_editing: { rows: number; versions: number; segmented_versions_not_edited?: number };
}
async function previewRules(engine: BrainEngine, sourceId: string, store: AccessStore, persisted: boolean, desired: FolderRule[]): Promise<RulePreview> {
@@ -150,18 +150,30 @@ async function previewRules(engine: BrainEngine, sourceId: string, store: Access
const credentials: RulePreview['credentials_losing'] = [];
for (const m of losing) for (const c of await boundCredentials(engine, m)) credentials.push({ member: m, kind: c.kind, name: c.name });
// A narrowing applies at once, so the held facts it covers leave the world-readable holding page in the same apply.
- const heldRows = await engine.executeRaw<{ id: number; entity_slug: string; fact: string; page: boolean }>(`SELECT f.id::int AS id, f.entity_slug, f.fact,
- EXISTS (SELECT 1 FROM pages p WHERE p.source_id=f.source_id AND p.slug=f.entity_slug AND p.deleted_at IS NULL) AS page
+ const heldRows = await engine.executeRaw<{ id: number; entity_slug: string; fact: string }>(`SELECT f.id::int AS id, f.entity_slug, f.fact
FROM facts f WHERE f.source_id=$1 AND f.source_markdown_slug=$2 AND f.row_num IS NOT NULL AND f.expired_at IS NULL AND f.dimension IS NULL
AND f.entity_slug IS NOT NULL AND f.entity_slug <> $2 ORDER BY f.id`, [sourceId, UNATTRIBUTED_MEMORY_SLUG]);
- const covered = heldRows.filter(row => cls.target.some(rule => ruleCovers(row.entity_slug, rule.prefix)));
- const moving = covered.map(row => ({ fact_id: Number(row.id), entity_slug: row.entity_slug,
- to: row.page || cls.target.some(rule => rule.prefix === row.entity_slug) ? row.entity_slug : folderHoldingSlug(row.entity_slug) }));
+ // C4: each row's `to` is where the apply's move puts it (heldRowDestination), with the desired rules deciding a page-less entity's home.
+ const { heldRowDestination, holdingHome } = await import('../core/facts/held-fact-move.ts');
+ const covers = (slug: string) => cls.target.some(rule => ruleCovers(slug, rule.prefix));
+ const desiredHome = async (key: string) => {
+ const now = await holdingHome(engine, sourceId, key);
+ if (!now || ('refused' in now && now.refused === 'access_unreadable')) return now;
+ if (cls.target.some(rule => rule.prefix === key)) return { refused: 'page_rule' as const };
+ return { slug: covers(key) ? folderHoldingSlug(key) : UNATTRIBUTED_MEMORY_SLUG };
+ };
+ const destinations = new Map<string, string | null>();
+ for (const key of new Set(heldRows.map(row => row.entity_slug))) {
+ const dest = await heldRowDestination(engine, sourceId, key, UNATTRIBUTED_MEMORY_SLUG, desiredHome);
+ destinations.set(key, dest && 'slug' in dest && covers(dest.slug) ? dest.slug : null);
+ }
+ const covered = heldRows.filter(row => destinations.get(row.entity_slug));
+ const moving = covered.map(row => ({ fact_id: Number(row.id), entity_slug: row.entity_slug, to: destinations.get(row.entity_slug)! }));
// A2: the apply also removes those rows from the shared page's saved versions (database only; Git history stays).
const versions = await heldRowsInVersions(engine, sourceId, UNATTRIBUTED_MEMORY_SLUG, covered.map(row => ({ entity: row.entity_slug, claim: row.fact })), false);
return { source_id: sourceId, store, store_persisted: persisted, expect, desired, changes: cls.changes, held: cls.held, matched_pages: matched,
unknown_readers: unknown, members_losing: losing, credentials_losing: credentials, held_facts_moving: moving,
- held_versions_editing: { rows: versions.rows, versions: versions.versions } };
+ held_versions_editing: { rows: versions.rows, versions: versions.versions, ...(versions.segmented ? { segmented_versions_not_edited: versions.segmented } : {}) } };
}
function previewLines(p: RulePreview, rerun: string): string[] {
@@ -174,6 +186,7 @@ function previewLines(p: RulePreview, rerun: string): string[] {
...(p.credentials_losing.length ? [` Credentials losing access: ${p.credentials_losing.map(c => `${c.kind} ${c.name} (${c.member})`).join(', ')}`] : []),
...(p.held_facts_moving.length ? [` Held facts leaving ${UNATTRIBUTED_MEMORY_SLUG} with this change: ${p.held_facts_moving.length} (${p.held_facts_moving.slice(0, 10).map(m => `fact ${m.fact_id} for ${m.entity_slug} to ${m.to}${m.to === m.entity_slug ? ' (its page)' : ''}`).join(', ')}${p.held_facts_moving.length > 10 ? ', ...' : ''})`] : []),
...(p.held_versions_editing.versions ? [` Their rows also leave ${p.held_versions_editing.versions} saved version(s) of ${UNATTRIBUTED_MEMORY_SLUG} (${p.held_versions_editing.rows} row(s); database only, Git history is not rewritten)`] : []),
+ ...(p.held_versions_editing.segmented_versions_not_edited ? [` Not edited: ${p.held_versions_editing.segmented_versions_not_edited} saved version(s) of ${UNATTRIBUTED_MEMORY_SLUG} are stored in segments, which this apply cannot edit. ${segmentedConsequence(p.held_versions_editing.segmented_versions_not_edited)} The apply will report itself incomplete.`] : []),
'Pages under no rule are unchanged; unbound credentials and stdio read no page under a rule.',
`Nothing written. To approve, add --yes --expect ${p.expect} (${rerun}).`,
];
@@ -193,7 +206,16 @@ export interface EditResult {
git?: GitPolicyReceipt;
note?: string;
/** GST-2492: held facts this apply moved off memory/unattributed to a page the new rules cover, and those that could not move. */
- held_facts?: { moved: number; unmovable: Array<{ slug: string; fact_ids: number[]; reason: string }> };
+ held_facts?: { moved: number; unmovable: Array<{ slug: string; fact_ids: number[]; reason: string }>; complete?: false; segmented_versions_not_edited?: number };
+}
+
+/** GST-2492 (B): what an unedited segmented saved version of the shared page leaves readable. */
+const segmentedConsequence = (n: number) => `Readers of ${UNATTRIBUTED_MEMORY_SLUG}'s saved versions can still read the moved rows in up to ${n} of those version(s).`;
+
+/** The words of a held-fact move that left segmented saved versions of the shared page unedited. */
+function heldIncompleteLines(held: EditResult['held_facts']): string[] {
+ const n = held?.segmented_versions_not_edited;
+ return n ? [` Incomplete: the held facts moved, but ${n} saved version(s) of ${UNATTRIBUTED_MEMORY_SLUG} are stored in segments and were not edited. ${segmentedConsequence(n)}`] : [];
}
/**
@@ -211,9 +233,11 @@ async function withHeldFactsMoved(engine: BrainEngine, result: EditResult): Prom
async function heldFactsMoved(engine: BrainEngine, sourceId: string): Promise<EditResult['held_facts']> {
const { moveHeldFacts } = await import('../core/facts/held-fact-move.ts');
const moved = await moveHeldFacts(engine, sourceId).catch((error: unknown) =>
- ({ moved: 0, rehomed: 0, unmovable: [{ slug: UNATTRIBUTED_MEMORY_SLUG, fact_ids: [], reason: (error instanceof Error ? error.message : String(error)).slice(0, 300) }] }));
+ ({ moved: 0, rehomed: 0, segmented_versions_not_edited: 0, unmovable: [{ slug: UNATTRIBUTED_MEMORY_SLUG, fact_ids: [], reason: (error instanceof Error ? error.message : String(error)).slice(0, 300) }] }));
if (!moved.moved && !moved.rehomed && !moved.unmovable.length) return undefined;
- return { moved: moved.moved + moved.rehomed, unmovable: moved.unmovable.map(({ slug, fact_ids, reason }) => ({ slug, fact_ids, reason })) };
+ const segmented = moved.segmented_versions_not_edited;
+ return { moved: moved.moved + moved.rehomed, unmovable: moved.unmovable.map(({ slug, fact_ids, reason }) => ({ slug, fact_ids, reason })),
+ ...(segmented ? { complete: false as const, segmented_versions_not_edited: segmented } : {}) };
}
/** Writes a desired rulebook through the source's store (after the caller's preview) and reports the result. */
@@ -247,6 +271,7 @@ function editLines(r: EditResult): string[] {
...(r.pages_restamped ? [` ${r.pages_restamped} page(s) restamped`] : []),
...(r.held_facts?.moved ? [` ${r.held_facts.moved} held fact(s) moved to a page the rules now cover`] : []),
...(r.held_facts?.unmovable.length ? [` Held facts that could not move yet (the next gbrain dream retries): ${r.held_facts.unmovable.map(u => `${u.slug} facts ${u.fact_ids.join(',')} (${u.reason})`).join('; ')}`] : []),
+ ...heldIncompleteLines(r.held_facts),
...(r.held.length ? [`Held: preview with gbrain access apply --source ${r.source_id}`] : []),
...(r.state === 'pending' ? [`Still pending: finish with gbrain access resume --source ${r.source_id}`] : []),
...(r.note ? [r.note] : []),
@@ -420,7 +445,7 @@ async function applyCmd(engine: BrainEngine, args: string[], json: boolean): Pro
const held = await heldFactsMoved(engine, sourceId);
return print(json, { ...r, ...(git ? { git } : {}), ...(held ? { held_facts: held } : {}) }, () => [`Source ${sourceId}: revision ${r.revision} approved and applied (${r.pages_restamped} page(s) restamped)`,
...(git ? [` ${git.path} committed ${git.commit?.slice(0, 12) ?? '(unchanged)'} (${git.outcome})`] : []),
- ...(r.held.length ? [`A later revision still holds ${r.held.length} widening change(s); preview with gbrain access apply --source ${sourceId}`] : [])]);
+ ...(r.held.length ? [`A later revision still holds ${r.held.length} widening change(s); preview with gbrain access apply --source ${sourceId}`] : []), ...heldIncompleteLines(held)]);
}
/** Dispatches `gbrain access rule|apply|resume`; false when `cmd` is not one of them. */
@@ -432,7 +457,7 @@ export async function runAccessRules(engine: BrainEngine, cmd: string, rest: str
const sourceId = flag(rest, '--source') ?? 'default';
const r = await resumePolicy(engine, sourceId, progressLogger(json));
const held = await heldFactsMoved(engine, sourceId);
- print(json, { source_id: sourceId, ...r, ...(held ? { held_facts: held } : {}) }, () => [`Source ${sourceId}: ${r.pages_restamped} page(s) restamped${r.pending_before.length ? `; pending prefixes cleared: ${r.pending_before.map(p => p || '(whole source)').join(', ')}` : '; nothing was pending'}`]);
+ print(json, { source_id: sourceId, ...r, ...(held ? { held_facts: held } : {}) }, () => [`Source ${sourceId}: ${r.pages_restamped} page(s) restamped${r.pending_before.length ? `; pending prefixes cleared: ${r.pending_before.map(p => p || '(whole source)').join(', ')}` : '; nothing was pending'}`, ...heldIncompleteLines(held)]);
return true;
}
default: return false;
diff --git a/src/commands/sync/perform.ts b/src/commands/sync/perform.ts
index 0800b532dd..4cd0a97410 100644
--- a/src/commands/sync/perform.ts
+++ b/src/commands/sync/perform.ts
@@ -208,6 +208,7 @@ async function performManagedBranch(engine: BrainEngine, outer: SyncOpts): Promi
// GST-2492 (B2): the lease is released; a folder rule this sync applied moves the held facts it covers now.
const held = currentDrainLease(sourceId) ? null : await (await import('../../core/facts/held-fact-move.ts')).moveRuledSharedHeld(engine, sourceId)
.catch((error: unknown) => { serr(`[sync] held facts under a new folder rule wait for the next maintenance cycle: ${error instanceof Error ? error.message : String(error)}`); return null; });
+ if (held?.segmented_versions_not_edited) serr(`[sync] held facts moved under a new folder rule, incomplete: ${held.segmented_versions_not_edited} saved version(s) of memory/unattributed are stored in segments and were not edited; readers of its saved versions can still read the moved rows in up to ${held.segmented_versions_not_edited} of them.`);
if (held?.unmovable.length) serr(`[sync] ${held.unmovable.length} group(s) of held facts could not move yet; the next gbrain dream retries (gbrain repair facts-home --source ${sourceId} lists them).`);
const tailed = skips.has('sync_tail') ? drained : await withSyncTail(engine, sourceId, drained, { signal: outer.signal, deadlineAtMs: outer.deadlineAtMs });
const healed = skips.has('slug_heal') ? tailed : await healAfterManagedSync(engine, outer, tailed);
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index 8a1c8d18c9..1ecabf96af 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1563,7 +1563,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.693.0', id: 'member_save_withdrawn_scope_gst2346', text: 'A remote `remember` onto a member\'s own page (`memory/members/<id>`) is refused `fact_withdrawn` only when that member could read what was withdrawn: a fact on its own page, its own copy withdrawn by the brain host, or for the brain owner a fact it reads. A claim withdrawn by the brain host or the owner that the member never read no longer refuses its save: the fact saves as the member\'s own active row with the same reply as a new claim. A schema migration makes the database insert guards mark a row on a member page only by that page\'s own withdrawals; a withdrawal still expires matching rows on member pages when it is made, and strikes them in the page\'s file, so an import of a member page\'s file no longer marks a row withdrawn by a withdrawal of another subject. The same holds for each item of a `remember` batch (`items`), a replay with the same `request_id`, a `replaces` and the write receipt (`get_write_request`). Trusted local saves keep the check they had.' },
{ since: '0.60.694.0', id: 'owner_labels_guarded_gst2521', text: 'Labels the owner sets with `gbrain repair visibility` (or its `--readers` setter, or `gbrain repair private-labels`) now raise the page\'s label guard to operator_curated, so a later local agent write at the page\'s own tier that drops or widens them is held like a write over an owner page, with a `lower_page` item. A local write below operator_curated whose `readable_by` would leave the brain owner out of a page the owner reads now publishes with the owner kept in the readers (the rest of the narrowing applies), and its item records the requested readers. Accepting such an item (`gbrain trust confirm tp<id>`) on a page whose own tier is an agent\'s writes only the labels and keeps the agent\'s tier; on an owner page it still endorses the edit at the owner\'s tier. The guard is stored in the database (`page_label_guards`) and set only by label repairs run after the upgrade, so labels set earlier, and a managed Git brain rebuilt from a clone, start unguarded; re-running `gbrain repair visibility --slug` or `--readers` on a page that already has those labels raises the guard without changing them. The owner\'s own label writes, synced owner edits, agent narrowing that keeps the owner, and brains with `access.owner_reads_restricted` on are unchanged.' },
{ since: '0.60.694.0', id: 'page_label_widening_held_gst2518', text: 'A local agent write (`put_page`, `capture`, `edit_page`, `append_page`, `revert_version`, `restore_page`) less trusted than a page that drops or widens its `visibility`, `access`, `confidentiality` or `readable_by` now publishes with the page\'s earlier labels kept, in the database and in the committed file, and its receipt lists them under `privacy_labels_kept`; so does a maintenance job\'s derived page publication, and a page whose tier is still `unknown` counts as the owner\'s. A soft delete no longer lowers the page\'s tier. Before, the page became readable to remote callers and other members at once, and only a `lower_page` item waited for the owner. That item now records the requested labels: `gbrain trust confirm tp<id>` applies them if the page has not changed since, `gbrain trust drop tp<id>` keeps the earlier ones, and undoing a confirm restores them while the page is unchanged since. Narrowing writes, label-neutral writes, writers at or above the page\'s tier, synced owner edits and `gbrain repair visibility` are unchanged.' },
- { since: 'TBD', id: 'folder_holding_page_gst2492', text: 'On a source whose pages are canonical in Git (`canonical: git`), a fact for an entity with no page under a folder access rule is no longer kept in the database only: it is held on the holding page of the entity\'s folder (`partners/memory/unattributed` for `partners/acme-example`), which every rule covering the entity covers by path, so it is in Git (or database-only where the folder is, under `storage.db_only` or a path Git ignores) with exactly the readers the entity\'s page would have. The reply names that `page` and `write_through.held_for`. `gbrain access rule add` previews the held facts a new rule covers (`held_facts_moving`) and its apply moves them off `memory/unattributed` (`held_facts`), as do `gbrain access apply`, `gbrain access resume` and a sync that applies a rule from Git (right after it releases its lease; maintenance is the fallback), and removes their rows from the shared page\'s saved versions in the database (`held_versions_editing` in the preview, `held_versions_edited` on the move\'s write receipt); Git history is not rewritten. A held fact is now also hidden from a caller who cannot read its holding page, even under a rule with `covers_derived: false`. `gbrain repair facts-home` writes such facts to their folder\'s page and lists as `folder_restricted` only an exact page rule (`page_rule`) or unreadable rules (`access_unreadable`). A `remember` for a page-less entity under an exact page rule now refuses `git_home_required` (reason `facts_lane`, frozen `error` `invalid_params`) with a fix that creates the page, and one whose access rules this database role cannot read refuses `access_policy_pending` (`next: wait`); both wrote the fact to the database before. Sources that are not strict and brains with no Git repository are unchanged.' },
+ { since: 'TBD', id: 'folder_holding_page_gst2492', text: 'On a source whose pages are canonical in Git (`canonical: git`), a fact for an entity with no page under a folder access rule is no longer kept in the database only: it is held on the holding page of the entity\'s folder (`partners/memory/unattributed` for `partners/acme-example`), which every rule covering the entity covers by path, so it is in Git (or database-only where the folder is, under `storage.db_only` or a path Git ignores) with exactly the readers the entity\'s page would have. The reply names that `page` and `write_through.held_for`. `gbrain access rule add` previews the held facts a new rule covers (`held_facts_moving`) and its apply moves them off `memory/unattributed` (`held_facts`), as do `gbrain access apply`, `gbrain access resume` and a sync that applies a rule from Git (right after it releases its lease; maintenance is the fallback), and removes their rows from the shared page\'s saved versions in the database (`held_versions_editing` in the preview, `held_versions_edited` on the move\'s write receipt); Git history is not rewritten. A saved version of `memory/unattributed` stored in segments is not edited: the preview names how many (`segmented_versions_not_edited`), the apply\'s `held_facts` says `complete: false`, and readers of that page\'s saved versions can still read the moved rows in up to that many versions. A held fact is now also hidden from a caller who cannot read its holding page, even under a rule with `covers_derived: false`. `gbrain repair facts-home` writes such facts to their folder\'s page and lists as `folder_restricted` only an exact page rule (`page_rule`) or unreadable rules (`access_unreadable`). A `remember` for a page-less entity under an exact page rule now refuses `git_home_required` (reason `facts_lane`, frozen `error` `invalid_params`) with a fix that creates the page, and one whose access rules this database role cannot read refuses `access_policy_pending` (`next: wait`); both wrote the fact to the database before. An existing page whose slug ends in `/memory/unattributed` is now read as a holding page: when its Facts table is next read (a sync or an extract of the page), a row with no `entity` cell becomes a fact with no entity instead of a fact about that page, a row whose `entity` cell names an entity in the page\'s own folder becomes that entity\'s held fact (and moves into the entity\'s page once it exists), a row naming an entity of another folder is a fence problem, and the page itself is never resolved as an entity page. Sources that are not strict and brains with no Git repository are unchanged.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/cycle/extract-facts.ts b/src/core/cycle/extract-facts.ts
index 85a70be9e9..c0bfea9762 100644
--- a/src/core/cycle/extract-facts.ts
+++ b/src/core/cycle/extract-facts.ts
@@ -865,7 +865,7 @@ export async function runExtractFacts(
const held = await moveHeldFacts(engine, sourceId);
if (held.moved + held.rehomed) result.heldFactsMoved = held.moved + held.rehomed;
result.pagesFailed += held.write_errors;
- for (const row of held.unmovable) result.warnings.push(`HELD_FACT_UNMOVABLE: ${row.slug} facts ${row.fact_ids.join(',')} (${row.reason})`);
+ result.warnings.push(...held.unmovable.map(row => `HELD_FACT_UNMOVABLE: ${row.slug} facts ${row.fact_ids.join(',')} (${row.reason})`), ...(held.segmented_versions_not_edited ? [`HELD_VERSIONS_NOT_EDITED: ${held.segmented_versions_not_edited} saved version(s) of memory/unattributed are stored in segments and were not edited; readers of its saved versions can still read the moved rows in up to ${held.segmented_versions_not_edited} of them.`] : []));
} catch (e) {
fenceStepFailed ||= !isFenceHold(e);
result.warnings.push(`HELD_FACT_UNMOVABLE: ${(e instanceof Error ? e.message : String(e)).slice(0, 300)}`);
diff --git a/src/core/facts/fence-fact-append.ts b/src/core/facts/fence-fact-append.ts
index acdc63f7f2..21db05acb6 100644
--- a/src/core/facts/fence-fact-append.ts
+++ b/src/core/facts/fence-fact-append.ts
@@ -50,6 +50,8 @@ import { extractFactsFromFenceText } from './extract-from-fence.ts';
import { isHoldingSlug } from '../facts-fence.ts';
import { rememberAcceptsPageCreatedSince } from '../persistence/page-identity.ts';
import { fenceRowOf, withHeldFactsFence, type FenceRowInput } from './unfenced-facts.ts';
+import { rewriteVersionBodies } from './purge.ts';
+import { carriesClaimSql, claimNeedles } from './purge-verify.ts';
import { withdrawnFact } from './withdrawal-overlay.ts';
export type FenceFactAppendInput = {
@@ -229,7 +231,8 @@ export async function prepareFenceRowRetirement(engine: BrainEngine, row: WriteR
/**
* GST-2492 (A2): the rows of `rows` (an entity's held claims) in the saved versions of holding page `slug`, removed when
* `write`. Each edited version keeps every other byte; only the table lines of those rows go. Database only: Git
- * history is never rewritten. Idempotent (a second run finds nothing). Segmented versions are counted, not edited.
+ * history is never rewritten. Idempotent (a second run finds nothing). Reads and rewrites through purge's
+ * rewriteVersionBodies, the one inline saved-version reader; segmented versions are counted, not edited.
*/
export async function heldRowsInVersions(db: BrainEngine, sourceId: string, slug: string, rows: ReadonlyArray<{ entity: string; claim: string }>, write: boolean):
Promise<{ versions: number; rows: number; segmented: number }> {
@@ -237,34 +240,25 @@ export async function heldRowsInVersions(db: BrainEngine, sourceId: string, slug
const keys = new Set(rows.map(r => `${r.entity}\0${r.claim.trim()}`));
const entities = [...new Set(rows.map(r => r.entity))];
if (!keys.size) return out;
- const versions = await db.executeRaw<{ id: number; compiled_truth: string | null; timeline: string | null; segmented: boolean }>(
- `SELECT v.id, v.compiled_truth, v.timeline, v.body_storage = 'segmented' AS segmented FROM page_versions v JOIN pages p ON p.id = v.page_id
- WHERE p.source_id = $1 AND p.slug = $2 AND (v.body_storage = 'segmented' OR EXISTS (SELECT 1 FROM unnest($3::text[]) AS e
- WHERE strpos(COALESCE(v.compiled_truth, ''), e) > 0 OR strpos(COALESCE(v.timeline, ''), e) > 0)) ORDER BY v.id${write ? ' FOR UPDATE OF v' : ''}`,
- [sourceId, slug, entities]);
- const strip = (text: string): [string, number] => {
- let removed = 0;
+ const strip = (text: string): string => {
for (let from = 0; ;) {
const begin = text.indexOf(FACTS_FENCE_BEGIN, from), end = begin < 0 ? -1 : text.indexOf(FACTS_FENCE_END, begin);
- if (end < 0) return [text, removed];
+ if (end < 0) return text;
const block = text.slice(begin, end + FACTS_FENCE_END.length);
const drop = new Set(parseFactsFenceFor(block, { slug }).facts.filter(f => f.entity && keys.has(`${f.entity}\0${f.claim.trim()}`)).map(f => f.rowNum));
const lines = block.split('\n');
const kept = lines.filter(line => !drop.has(Number(/^\|\s*(\d+)\s*\|/.exec(line)?.[1] ?? NaN)));
- removed += lines.length - kept.length;
+ out.rows += lines.length - kept.length;
const next = kept.join('\n');
text = text.slice(0, begin) + next + text.slice(end + FACTS_FENCE_END.length);
from = begin + next.length;
}
};
- for (const v of versions) {
- if (v.segmented) { out.segmented++; continue; }
- const [compiled, a] = strip(v.compiled_truth ?? ''), [timeline, b] = strip(v.timeline ?? '');
- if (!a && !b) continue;
- out.versions++; out.rows += a + b;
- if (write) await db.executeRaw('UPDATE page_versions SET compiled_truth = $2, timeline = $3 WHERE id = $1',
- [v.id, v.compiled_truth === null ? null : compiled, v.timeline === null ? null : timeline]);
- }
+ await rewriteVersionBodies(db, { columns: '', where: `v.id>$2 AND p.slug=$3 AND (${entities.map((_, i) => carriesClaimSql('v', 4 + 2 * i)).join(' OR ')})`,
+ params: after => [sourceId, after, slug, ...entities.flatMap(claimNeedles)], lock: write }, strip, (_v, _edited, changed) => { if (changed) out.versions++; }, write);
+ const [segmented] = await db.executeRaw<{ n: number }>(`SELECT count(*)::int AS n FROM page_versions v JOIN pages p ON p.id = v.page_id
+ WHERE p.source_id = $1 AND p.slug = $2 AND v.body_storage = 'segmented'`, [sourceId, slug]);
+ out.segmented = Number(segmented?.n ?? 0);
return out;
}
diff --git a/src/core/facts/held-fact-move.ts b/src/core/facts/held-fact-move.ts
index 66456a928c..534e35e0d0 100644
--- a/src/core/facts/held-fact-move.ts
+++ b/src/core/facts/held-fact-move.ts
@@ -27,6 +27,8 @@ export interface HeldFactMoveOutcome {
unmovable: Array<{ code: 'held_fact_unmovable'; slug: string; fact_ids: number[]; reason: string }>;
/** #8568: entities whose move threw (I/O, database), as opposed to holds the next run settles. */
write_errors: number;
+ /** GST-2492 (B): saved versions of the shared page stored in segments, which the move could not edit (the most any one move counted). */
+ segmented_versions_not_edited: number;
}
/**
@@ -94,6 +96,24 @@ export async function ensurePageForHeldRows(engine: BrainEngine, sourceId: strin
}
type HeldRow = UnfencedFactRow & { from_slug: string };
+export type HoldingHome = Awaited<ReturnType<typeof holdingHome>>;
+
+/**
+ * GST-2492 (C4): where moveHeldFacts puts the held rows keyed `key` on holding page `from`, the one resolution the rule
+ * preview and the move share: the page the key resolves to (a bare name moves to the page it names), else the page an
+ * exact page rule names (created for the rows), else, off the shared page, the holding page `home` gives (the rules in
+ * force; a preview passes its desired rules). Null: the rows stay where they are.
+ */
+export async function heldRowDestination(engine: BrainEngine, sourceId: string, key: string, from: string,
+ home: (key: string) => Promise<HoldingHome> = k => holdingHome(engine, sourceId, k)):
+ Promise<{ slug: string; kind: 'page' | 'create' | 'rehome' } | { unmovable: string } | null> {
+ const target = await heldFactTarget(engine, sourceId, key);
+ if (target && 'ambiguous' in target) return { unmovable: ambiguousHeldReason(target.ambiguous) };
+ if (target) return { slug: target.slug, kind: 'page' };
+ const h = await home(key);
+ if (h && 'refused' in h) return h.refused === 'access_unreadable' ? { unmovable: 'access_unreadable' } : { slug: key, kind: 'create' };
+ return h && h.slug !== from && from === UNATTRIBUTED_MEMORY_SLUG ? { slug: h.slug, kind: 'rehome' } : null;
+}
/**
* GST-2492 (B2): after a sync releases its lease, the held facts a folder rule it applied now covers leave the shared page
@@ -107,7 +127,7 @@ export async function moveRuledSharedHeld(engine: BrainEngine, sourceId: string)
}
export async function moveHeldFacts(engine: BrainEngine, sourceId: string): Promise<HeldFactMoveOutcome> {
- const outcome: HeldFactMoveOutcome = { moved: 0, moved_slugs: [], rehomed: 0, unmovable: [], write_errors: 0 };
+ const outcome: HeldFactMoveOutcome = { moved: 0, moved_slugs: [], rehomed: 0, unmovable: [], write_errors: 0, segmented_versions_not_edited: 0 };
const rows = await engine.executeRaw<HeldRow>(
`SELECT id, source_id, entity_slug, fact, kind, visibility, notability, context, valid_from, valid_until, source, confidence,
claim_metric, claim_value, claim_unit, claim_period, attributed_to, true AS page_exists, source_markdown_slug AS from_slug
@@ -127,10 +147,10 @@ export async function moveHeldFacts(engine: BrainEngine, sourceId: string): Prom
let authority: Awaited<ReturnType<typeof maintenancePreflight>> | undefined;
for (const [, group] of keys) {
const { from_slug: from, entity_slug: key } = group[0]!;
- let target = await heldFactTarget(engine, sourceId, key!);
- const home = target ? null : await holdingHome(engine, sourceId, key!);
- if (home && 'refused' in home && home.refused === 'access_unreadable') { unmovable(key!, group.map(r => r.id), 'access_unreadable'); continue; }
- if (home && 'refused' in home) {
+ const dest = await heldRowDestination(engine, sourceId, key!, from);
+ if (!dest) continue;
+ if ('unmovable' in dest) { unmovable(key!, group.map(r => r.id), dest.unmovable); continue; }
+ if (dest.kind === 'create') {
// An exact page rule on the entity: only its own page carries those readers, so the page is created for its rows.
authority ??= await maintenancePreflight(engine, sourceId);
if (!authority) return outcome;
@@ -138,11 +158,8 @@ export async function moveHeldFacts(engine: BrainEngine, sourceId: string): Prom
unmovable(key!, group.map(r => r.id), (error instanceof Error ? error.message : String(error)).slice(0, 300));
continue;
}
- target = { slug: key! };
}
- if (target && 'ambiguous' in target) unmovable(key!, group.map(r => r.id), ambiguousHeldReason(target.ambiguous));
- else if (target) groups.push({ slug: target.slug, from, ...(target.slug === key ? {} : { fromEntity: key! }), group, rehome: false });
- else if (home && 'slug' in home && home.slug !== from && from === UNATTRIBUTED_MEMORY_SLUG) groups.push({ slug: home.slug, from, group, rehome: true });
+ groups.push({ slug: dest.slug, from, ...(dest.kind === 'rehome' || dest.slug === key ? {} : { fromEntity: key! }), group, rehome: dest.kind === 'rehome' });
}
if (groups.length === 0) return outcome;
authority ??= await maintenancePreflight(engine, sourceId);
@@ -171,6 +188,9 @@ export async function moveHeldFacts(engine: BrainEngine, sourceId: string): Prom
for (const reason of new Set(rejected.map(r => r.class))) unmovable(slug, rejected.filter(r => r.class === reason).map(r => r.fact_id), reason);
if (plan.kind === 'unadoptable') continue;
await submitPlannedFactFenceAdoption(engine, authority, slug, plan.request, true);
+ const [edited] = await engine.executeRaw<{ n: number | null }>(`SELECT (outcome->'held_versions_edited'->>'segmented_versions_not_edited')::int AS n
+ FROM persistence_requests WHERE request_id = $1::uuid`, [plan.request.requestId]);
+ outcome.segmented_versions_not_edited = Math.max(outcome.segmented_versions_not_edited, Number(edited?.n ?? 0));
if (rehome) outcome.rehomed += plan.fence.assignments.length;
else { outcome.moved += plan.fence.assignments.length; outcome.moved_slugs.push(slug); }
} catch (error) {
diff --git a/src/core/facts/purge.ts b/src/core/facts/purge.ts
index b316487505..4494163707 100644
--- a/src/core/facts/purge.ts
+++ b/src/core/facts/purge.ts
@@ -306,6 +306,34 @@ export async function rewritePageBodies(tx: BrainEngine, sourceId: string, pages
const VERSION_REDACTION_BATCH = 5000;
+/**
+ * The one reader and rewriter of inline saved-version bodies, for purge's redaction and the held-fact move
+ * (fence-fact-append.ts heldRowsInVersions): the inline versions of source $1 matching `where` in keyset batches by id,
+ * `edit` applied to each body part, and the version rewritten when either part changed, unless `write` is false (a
+ * preview). `visit` sees each version's edited parts. Segmented versions are each caller's own pass.
+ */
+export async function rewriteVersionBodies<X extends object>(tx: BrainEngine,
+ query: { columns?: string; where: string; params: (after: number) => unknown[]; lock?: boolean },
+ edit: (text: string, slug: string) => Promise<string> | string,
+ visit: (v: X & { id: number; slug: string }, edited: [string, string], changed: boolean) => void, write = true): Promise<void> {
+ for (let after = 0; ;) {
+ const rows = await tx.executeRaw<X & { id: number; slug: string; compiled_truth: string; timeline: string | null }>(`SELECT v.id,p.slug,v.compiled_truth,v.timeline${query.columns ? `,${query.columns}` : ''}
+ FROM page_versions v JOIN pages p ON p.id=v.page_id
+ WHERE p.source_id=$1 AND v.body_storage='inline' AND ${query.where}
+ ORDER BY v.id LIMIT ${VERSION_REDACTION_BATCH}${query.lock ? ' FOR UPDATE OF v' : ''}`, query.params(after));
+ if (!rows.length) break;
+ after = Number(rows[rows.length - 1].id);
+ for (const v of rows) {
+ const compiled = await edit(v.compiled_truth, v.slug);
+ const timeline = await edit(v.timeline ?? '', v.slug);
+ const changed = compiled !== v.compiled_truth || timeline !== (v.timeline ?? '');
+ if (changed && write) await tx.executeRaw('UPDATE page_versions SET compiled_truth=$2,timeline=$3 WHERE id=$1', [v.id, compiled, timeline]);
+ visit(v, [compiled, timeline], changed);
+ }
+ if (rows.length < VERSION_REDACTION_BATCH) break;
+ }
+}
+
/** Redact the row from every retained snapshot that carries the claim; returns redacted and still-carrying (prose) counts. */
export async function redactPageVersions(tx: BrainEngine, sourceId: string, claim: string, pageIds: number[]):
Promise<{ redacted: number; prose: Array<{ slug: string; version_id: number }>; segmentedPages: number[]; replaced: Uint8Array[] }> {
@@ -316,25 +344,14 @@ export async function redactPageVersions(tx: BrainEngine, sourceId: string, clai
[sourceId, needle, reasonNeedle(claim)])).map(row => Number(row.id)));
let redacted = reasons.size;
const prose: Array<{ slug: string; version_id: number }> = [];
- for (let after = 0; ;) {
- const rows = await tx.executeRaw<{ id: number; slug: string; compiled_truth: string; timeline: string | null; metadata: boolean }>(`SELECT v.id,p.slug,v.compiled_truth,v.timeline,
- (strpos(lower(COALESCE(v.title,'')),$4)>0 OR strpos(lower(COALESCE(v.frontmatter::text,'')),$5)>0) AS metadata
- FROM page_versions v JOIN pages p ON p.id=v.page_id
- WHERE p.source_id=$1 AND v.body_storage='inline' AND v.id>$3 AND (v.page_id=ANY($2::int[]) OR ${carriesClaimSql('v', 4)})
- ORDER BY v.id LIMIT ${VERSION_REDACTION_BATCH}`, [sourceId, pageIds, after, needle, escaped]);
- if (!rows.length) break;
- after = Number(rows[rows.length - 1].id);
- for (const v of rows) {
- const compiled = await dropPurgedFenceRows(tx, sourceId, v.compiled_truth, v.slug);
- const timeline = await dropPurgedFenceRows(tx, sourceId, v.timeline ?? '', v.slug);
- if (compiled !== v.compiled_truth || timeline !== (v.timeline ?? '')) {
- await tx.executeRaw('UPDATE page_versions SET compiled_truth=$2,timeline=$3 WHERE id=$1', [v.id, compiled, timeline]);
- if (!reasons.has(Number(v.id))) redacted++;
- }
- if (v.metadata || compiled.toLowerCase().includes(needle) || timeline.toLowerCase().includes(needle)) prose.push({ slug: v.slug, version_id: Number(v.id) });
- }
- if (rows.length < VERSION_REDACTION_BATCH) break;
- }
+ await rewriteVersionBodies<{ metadata: boolean }>(tx, {
+ columns: `
+ (strpos(lower(COALESCE(v.title,'')),$4)>0 OR strpos(lower(COALESCE(v.frontmatter::text,'')),$5)>0) AS metadata`,
+ where: `v.id>$3 AND (v.page_id=ANY($2::int[]) OR ${carriesClaimSql('v', 4)})`, params: after => [sourceId, pageIds, after, needle, escaped],
+ }, (text, slug) => dropPurgedFenceRows(tx, sourceId, text, slug), (v, [compiled, timeline], changed) => {
+ if (changed && !reasons.has(Number(v.id))) redacted++;
+ if (v.metadata || compiled.toLowerCase().includes(needle) || timeline.toLowerCase().includes(needle)) prose.push({ slug: v.slug, version_id: Number(v.id) });
+ });
// Wave B (spec 11): a segmented version references a generation; that generation is rebuilt with the rows dropped,
// once per generation (versions sharing one share its rebuild and its prose probe), in the same keyset batches.
const carries = new Map<string, boolean>();
diff --git a/test/folder-holding-page-gst2492.test.ts b/test/folder-holding-page-gst2492.test.ts
index f4798e98b3..02ff1ae5f1 100644
--- a/test/folder-holding-page-gst2492.test.ts
+++ b/test/folder-holding-page-gst2492.test.ts
@@ -424,6 +424,84 @@ for (const backend of testBackends()) describe(`[${backend}] the folder-rule hol
[kept]: [{ entity_slug: 'notes/garden-plan', source_markdown_slug: HOLD }] });
}), 300_000);
+ /** A saved version of `slug` stored in segments, with no body copy (as test/repair-unwithdrawn-facts-8819.test.ts seeds one). */
+ const segmentedVersion = (engine: BrainEngine, slug: string) => engine.executeRaw(`INSERT INTO page_versions (page_id, compiled_truth, timeline, body_storage, body_generation)
+ SELECT id, NULL, NULL, 'segmented', 1 FROM pages WHERE source_id = 'default' AND slug = $1`, [slug]);
+ const humanLines = async (engine: BrainEngine, cmd: string, args: string[]) => {
+ const lines: string[] = [];
+ const spy = spyOn(console, 'log').mockImplementation((...a: unknown[]) => { lines.push(a.map(String).join(' ')); });
+ try { await runAccessRules(engine, cmd, args, false); } finally { spy.mockRestore(); }
+ return lines.join('\n');
+ };
+ const consequence = (n: number) => `Readers of ${HOLD}'s saved versions can still read the moved rows in up to ${n} of those version(s).`;
+
+ test('a saved version stored in segments is named before the apply, and the apply reports itself incomplete with what stays readable', () => strict(async ({ engine, ctx }) => {
+ const acme = 'The acme-example brief carries a segmented clause.';
+ const beta = 'The beta-example plan carries a segmented clause.';
+ await remember(ctx, acme, 'clients/acme-example/brief');
+ await remember(ctx, beta, 'clients/beta-example/plan');
+ await settleManagedGitEffects(engine);
+ await segmentedVersion(engine, HOLD);
+ const args = (prefix: string) => ['add', '--prefix', prefix, '--readers', 'alice-example', '--source', 'default'];
+ const preview = await cli(engine, ['rule', ...args('clients/acme-example/'), '--json']);
+ expect(preview.held_versions_editing).toMatchObject({ segmented_versions_not_edited: 1 });
+ const words = await humanLines(engine, 'rule', args('clients/acme-example/'));
+ expect(words).toContain(`Not edited: 1 saved version(s) of ${HOLD} are stored in segments, which this apply cannot edit. ${consequence(1)} The apply will report itself incomplete.`);
+ const applied = await cli(engine, ['rule', ...args('clients/acme-example/'), '--yes', '--expect', String(preview.expect), '--json']);
+ expect(applied.held_facts).toEqual({ moved: 1, unmovable: [], complete: false, segmented_versions_not_edited: 1 });
+ const second = await cli(engine, ['rule', ...args('clients/beta-example/'), '--json']);
+ expect(second.held_versions_editing).toMatchObject({ segmented_versions_not_edited: 1 });
+ const done = await humanLines(engine, 'rule', [...args('clients/beta-example/'), '--yes', '--expect', String(second.expect)]);
+ expect(done).toContain(`Incomplete: the held facts moved, but 1 saved version(s) of ${HOLD} are stored in segments and were not edited. ${consequence(1)}`);
+ expect(await factRows(engine, beta)).toMatchObject([{ source_markdown_slug: 'clients/beta-example/memory/unattributed' }]);
+ }), 300_000);
+
+ test('the sync path and the maintenance fallback report a move that left a segmented saved version unedited as incomplete', () => strict(async ({ engine, ctx, root }) => {
+ const moveModule = await import('../src/core/facts/held-fact-move.ts');
+ const original = moveModule.moveRuledSharedHeld;
+ const synced = 'The acme-example brief carries a segmented synced clause.';
+ const crashed = 'The beta-example plan carries a segmented synced clause.';
+ await remember(ctx, synced, 'clients/acme-example/brief');
+ await remember(ctx, crashed, 'clients/beta-example/plan');
+ await settleManagedGitEffects(engine);
+ await segmentedVersion(engine, HOLD);
+ const policy = (list: FolderRule[]) => commitPolicyFile(root, POLICY_FILE, () => serializePolicyYaml(list), 'access: rule');
+ const sync = () => performSync(engine, { sourceId: 'default', repoPath: root, noPull: true, noEmbed: true } as never);
+ let result: Awaited<ReturnType<typeof original>> = null;
+ const errors: string[] = [];
+ const spy = spyOn(moveModule, 'moveRuledSharedHeld').mockImplementation(async (e, s) => (result = await original(e, s)));
+ const stderr = spyOn(console, 'error').mockImplementation((...a: unknown[]) => { errors.push(a.map(String).join(' ')); });
+ try {
+ policy([rule('acme', 'clients/acme-example/', ['alice-example'])]);
+ await sync();
+ } finally { spy.mockRestore(); stderr.mockRestore(); }
+ expect(result).toMatchObject({ rehomed: 1, segmented_versions_not_edited: 1 });
+ expect(errors.join('\n')).toContain('held facts moved under a new folder rule, incomplete: 1 saved version(s) of memory/unattributed are stored in segments and were not edited; readers of its saved versions can still read the moved rows in up to 1 of them.');
+ const died = spyOn(moveModule, 'moveRuledSharedHeld').mockImplementation(async () => { throw new Error('process died'); });
+ try {
+ policy([rule('acme', 'clients/acme-example/', ['alice-example']), rule('beta', 'clients/beta-example/', ['alice-example'])]);
+ await sync();
+ } finally { died.mockRestore(); }
+ const upkeep = await runExtractFacts(engine, { sourceId: 'default' });
+ expect(upkeep.heldFactsMoved).toBe(1);
+ expect(upkeep.warnings).toContain('HELD_VERSIONS_NOT_EDITED: 1 saved version(s) of memory/unattributed are stored in segments and were not edited; readers of its saved versions can still read the moved rows in up to 1 of them.');
+ }), 300_000);
+
+ test('the preview names as each held row\'s destination the page the apply moves it to, for a row keyed by a bare name', () => strict(async ({ engine, ctx }) => {
+ const claim = 'Kofi Example signs the acme-example lease.';
+ const saved = await remember(ctx, claim, 'Kofi Example');
+ expect(saved).toMatchObject({ entity_slug: 'kofi-example', page: HOLD });
+ await call(ctx, 'put_page', { slug: 'people/kofi-example', content: '---\ntype: person\ntitle: Kofi Example\n---\n\nNotes about Kofi Example.\n' });
+ await settleManagedGitEffects(engine);
+ const args = ['add', '--prefix', 'people/', '--readers', 'alice-example', '--source', 'default'];
+ const preview = await cli(engine, ['rule', ...args, '--json']);
+ const applied = await cli(engine, ['rule', ...args, '--yes', '--expect', String(preview.expect), '--json']);
+ expect(applied.held_facts).toMatchObject({ moved: 1 });
+ const [after] = await factRows(engine, claim);
+ expect(preview.held_facts_moving).toEqual([{ fact_id: Number(saved.id), entity_slug: 'kofi-example', to: after!.source_markdown_slug }]);
+ expect(after!.source_markdown_slug).toBe('people/kofi-example');
+ }), 300_000);
+
test('the owner reads every holding page before and after a re-home: with the opt-in on, and off where the rules name the owner', () => strict(async ({ engine, ctx }) => {
const as = await members(engine);
await setOwner(engine, 'carol-example');