Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Remove moved held rows from the shared page's saved versions, and move them right after a sync that applies a folder rule (GST-2492)

garrytan76e058b76b62
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1ec988edce..d0cf8277f0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,7 +23,9 @@ identifiers and attribution are available in the pre-removal Git revision
- `src/core/persistence/coordinator.ts`: a held fact on a database-only folder's holding page reports `write_through: {written: false, skipped: db_only, held_for}` instead of `no_repo_configured`.
- `src/core/search/private-visibility.ts`: a held fact is also judged by its holding page as a page, so a raw-only rule (`covers_derived: false`) hides it as it hides the page; a fact still on `memory/unattributed` for an entity a rule now covers (a rule synced from Git, until maintenance moves it) reads only for that rule's readers, and for nobody while the rule is pending.
- `src/core/entities/resolve.ts`: a holding page under an entity directory (`people/memory/unattributed`) is never an entity page itself.
-- `src/commands/access-rules.ts`: the rule preview lists `held_facts_moving` and the apply moves them (`held_facts: {moved, unmovable}`), for the database and git stores; `gbrain access apply` and `gbrain access resume` move them too. The shared page's earlier versions and Git history keep the row as it was; nothing rewrites history.
+- `src/commands/access-rules.ts`: the rule preview lists `held_facts_moving` and the apply moves them (`held_facts: {moved, unmovable}`), for the database and git stores; `gbrain access apply` and `gbrain access resume` move them too.
+- `src/core/facts/fence-fact-append.ts`, `src/core/persistence/prepared-maintenance.ts`: a held row that leaves the shared page for a page not every reader can read also leaves the page's saved versions, in the same transaction; every other byte of each version is kept, Git history is not rewritten, and the move's write receipt records it (`held_versions_edited`: the rows' fact ids, the row and version counts, where they went). The rule preview counts the rows and versions (`held_versions_editing`).
+- `src/commands/sync/perform.ts`: after a managed sync releases its lease, held facts that a folder rule it applied now covers move at once; the maintenance cycle stays the fallback.
- `src/core/repair/facts-home.ts`, `src/core/facts/facts-home.ts`: `gbrain repair facts-home` writes a ruled entity's fact to its folder's page; `folder_restricted` now means only `page_rule` or `access_unreadable`.
- Docs: `git-canonical.md`, `entity-scoped-facts.md`, `access-designations.md`, `repair.md`, `write-refusals.md`, `memory-boundaries.md`, `MEMORY_VERBS_v1.md`, `error-codes.md` (regenerated).
- Tests: new `test/folder-holding-page-gst2492.test.ts` (held in Git before the reply and kept by a rebuild from a clone; three nested rules plus a pending one; no existence leak on get_page, search, query, recall, list_pages and find_pages; a raw-only rule; the rule apply's preview and move; the owner with the opt-in on and off; the page-rule and unreadable refusals locally and over MCP; database-only folders; background writers and repair; non-strict and repo-less unchanged); updated GST-2492 suites; PGLite and Postgres.
diff --git a/docs/guides/git-canonical.md b/docs/guides/git-canonical.md
index 034796ed0f..5e1f6b0eda 100644
--- a/docs/guides/git-canonical.md
+++ b/docs/guides/git-canonical.md
@@ -194,10 +194,14 @@ Every write goes through the same door, whoever sends it:
of its own apply (`gbrain access rule add` previews them as
`held_facts_moving`; `gbrain access apply` and `gbrain access resume` move
them too). A rule that arrives through a sync of the policy file moves them
- at the next maintenance cycle; until then the row stays in the shared page's
- body, while the fact itself reads only for the rule's readers. The shared
- page's earlier versions and its Git history keep the row as it was before
- the rule: gbrain never rewrites history. An entity under an exact page rule (a rule on the
+ as soon as that sync releases its lease, so for the seconds the sync pass
+ takes the row is still in the shared page's body (its fact already reads only
+ for the rule's readers); if the process stops in between, the next
+ maintenance cycle moves them. The move also removes those rows from the
+ shared page's saved versions in the database (the rule's preview counts
+ them, and the move's write receipt records which rows, how many versions and
+ where they went). Git history is never rewritten: the shared page's earlier
+ commits keep the row as it was before the rule. An entity under an exact page rule (a rule on the
entity's own slug) has no holding page with its readers: the fact is refused
with [`git_home_required`](write-refusals.md#git_home_required) (reason
`facts_lane`) until the page exists.
diff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsv
index c484306067..8734c79475 100644
--- a/scripts/page-body-readers-baseline.tsv
+++ b/scripts/page-body-readers-baseline.tsv
@@ -228,6 +228,8 @@ src/core/facts/backstop.ts runPipeline field STREAM
src/core/facts/conversation-window-extract.ts regularPageVersionToken field STREAM
src/core/facts/drain.ts estimateBacklogUsd sql META
src/core/facts/eligibility.ts isFactsBackstopEligible field LEAD
+src/core/facts/fence-fact-append.ts heldRowsInVersions field SEGW GST-2492 A2, as purge's redactPageVersions: rewrites inline version bodies only; a segmented version is counted (segmented_versions_not_edited on the receipt), never read
+src/core/facts/fence-fact-append.ts heldRowsInVersions sql SEGW GST-2492 A2, as purge's redactPageVersions: selects inline version bodies; a segmented version is selected only to be counted
src/core/facts/fence-fact-append.ts prepareFenceFactAppend call:serializePageToMarkdown SEGW assertInlineBody first
src/core/facts/fence-fact-append.ts prepareFenceFactAppend field SEGW assertInlineBody before the fence write
src/core/facts/fence-fact-append.ts prepareFenceRowRetirement field SEGW the body materializedPage read back within PAGE_MAX_BYTES; a segmented page over the ceiling refuses (segmentedOverCeiling) first
diff --git a/src/commands/access-rules.ts b/src/commands/access-rules.ts
index 3078108483..48c8ba6c84 100644
--- a/src/commands/access-rules.ts
+++ b/src/commands/access-rules.ts
@@ -23,6 +23,7 @@ import { usageError } from '../cli/cli-error.ts';
import { opError } from '../core/ops/contract.ts';
import { classifyRulebook, coversDerived, folderRulesAllow, normalizeRule, ruleCovers, ruleCoversSql, parsePolicyYaml, serializePolicyYaml, sortRules, type FolderRule, type RuleChange } from '../core/access/folder-rules.ts';
import { folderHoldingSlug, UNATTRIBUTED_MEMORY_SLUG } from '../core/facts-fence.ts';
+import { heldRowsInVersions } from '../core/facts/fence-fact-append.ts';
import { identityFor } from '../core/access/principals.ts';
import { boundCredentials, listGroups, listMembers } from '../core/access/members.ts';
import {
@@ -121,6 +122,8 @@ export interface RulePreview {
* the apply moves each to (its folder's holding page, or the entity's own page, created for an exact page rule).
*/
held_facts_moving: Array<{ fact_id: number; entity_slug: string; to: string }>;
+ /** The rows of those facts in the shared page's saved versions, and those versions, the apply removes them from (database only; the version the move itself saves is edited too). */
+ held_versions_editing: { rows: number; versions: number };
}
async function previewRules(engine: BrainEngine, sourceId: string, store: AccessStore, persisted: boolean, desired: FolderRule[]): Promise<RulePreview> {
@@ -147,14 +150,18 @@ async function previewRules(engine: BrainEngine, sourceId: string, store: Access
const credentials: RulePreview['credentials_losing'] = [];
for (const m of losing) for (const c of await boundCredentials(engine, m)) credentials.push({ member: m, kind: c.kind, name: c.name });
// A narrowing applies at once, so the held facts it covers leave the world-readable holding page in the same apply.
- const heldRows = await engine.executeRaw<{ id: number; entity_slug: string; page: boolean }>(`SELECT f.id::int AS id, f.entity_slug,
+ const heldRows = await engine.executeRaw<{ id: number; entity_slug: string; fact: string; page: boolean }>(`SELECT f.id::int AS id, f.entity_slug, f.fact,
EXISTS (SELECT 1 FROM pages p WHERE p.source_id=f.source_id AND p.slug=f.entity_slug AND p.deleted_at IS NULL) AS page
FROM facts f WHERE f.source_id=$1 AND f.source_markdown_slug=$2 AND f.row_num IS NOT NULL AND f.expired_at IS NULL AND f.dimension IS NULL
AND f.entity_slug IS NOT NULL AND f.entity_slug <> $2 ORDER BY f.id`, [sourceId, UNATTRIBUTED_MEMORY_SLUG]);
- const moving = heldRows.filter(row => cls.target.some(rule => ruleCovers(row.entity_slug, rule.prefix))).map(row => ({ fact_id: Number(row.id), entity_slug: row.entity_slug,
+ const covered = heldRows.filter(row => cls.target.some(rule => ruleCovers(row.entity_slug, rule.prefix)));
+ const moving = covered.map(row => ({ fact_id: Number(row.id), entity_slug: row.entity_slug,
to: row.page || cls.target.some(rule => rule.prefix === row.entity_slug) ? row.entity_slug : folderHoldingSlug(row.entity_slug) }));
+ // A2: the apply also removes those rows from the shared page's saved versions (database only; Git history stays).
+ const versions = await heldRowsInVersions(engine, sourceId, UNATTRIBUTED_MEMORY_SLUG, covered.map(row => ({ entity: row.entity_slug, claim: row.fact })), false);
return { source_id: sourceId, store, store_persisted: persisted, expect, desired, changes: cls.changes, held: cls.held, matched_pages: matched,
- unknown_readers: unknown, members_losing: losing, credentials_losing: credentials, held_facts_moving: moving };
+ unknown_readers: unknown, members_losing: losing, credentials_losing: credentials, held_facts_moving: moving,
+ held_versions_editing: { rows: versions.rows, versions: versions.versions } };
}
function previewLines(p: RulePreview, rerun: string): string[] {
@@ -166,6 +173,7 @@ function previewLines(p: RulePreview, rerun: string): string[] {
...(p.members_losing.length ? [` Members losing access: ${p.members_losing.join(', ')}`] : []),
...(p.credentials_losing.length ? [` Credentials losing access: ${p.credentials_losing.map(c => `${c.kind} ${c.name} (${c.member})`).join(', ')}`] : []),
...(p.held_facts_moving.length ? [` Held facts leaving ${UNATTRIBUTED_MEMORY_SLUG} with this change: ${p.held_facts_moving.length} (${p.held_facts_moving.slice(0, 10).map(m => `fact ${m.fact_id} for ${m.entity_slug} to ${m.to}${m.to === m.entity_slug ? ' (its page)' : ''}`).join(', ')}${p.held_facts_moving.length > 10 ? ', ...' : ''})`] : []),
+ ...(p.held_versions_editing.versions ? [` Their rows also leave ${p.held_versions_editing.versions} saved version(s) of ${UNATTRIBUTED_MEMORY_SLUG} (${p.held_versions_editing.rows} row(s); database only, Git history is not rewritten)`] : []),
'Pages under no rule are unchanged; unbound credentials and stdio read no page under a rule.',
`Nothing written. To approve, add --yes --expect ${p.expect} (${rerun}).`,
];
diff --git a/src/commands/sync/perform.ts b/src/commands/sync/perform.ts
index eca9e044fb..0800b532dd 100644
--- a/src/commands/sync/perform.ts
+++ b/src/commands/sync/perform.ts
@@ -143,7 +143,7 @@ async function mayBeConnectorSource(engine: BrainEngine, opts: SyncOpts): Promis
*/
async function performManagedBranch(engine: BrainEngine, outer: SyncOpts): Promise<SyncResult> {
const sourceId = outer.sourceId ?? 'default';
- const { withDrainLease } = await import('../../core/persistence/sync-lease.ts');
+ const { currentDrainLease, withDrainLease } = await import('../../core/persistence/sync-lease.ts');
const lockKey = syncLockId(sourceId);
// GBRA-133: the settles an idle-clear tick leaves nothing to do (persistence/sync-idle-skips.ts), read inside the lease.
let skips: ReadonlySet<IdleSkip> = new Set();
@@ -205,6 +205,10 @@ async function performManagedBranch(engine: BrainEngine, outer: SyncOpts): Promi
: `[sync] following ${who}: ${done}.`);
},
}, work);
+ // GST-2492 (B2): the lease is released; a folder rule this sync applied moves the held facts it covers now.
+ const held = currentDrainLease(sourceId) ? null : await (await import('../../core/facts/held-fact-move.ts')).moveRuledSharedHeld(engine, sourceId)
+ .catch((error: unknown) => { serr(`[sync] held facts under a new folder rule wait for the next maintenance cycle: ${error instanceof Error ? error.message : String(error)}`); return null; });
+ if (held?.unmovable.length) serr(`[sync] ${held.unmovable.length} group(s) of held facts could not move yet; the next gbrain dream retries (gbrain repair facts-home --source ${sourceId} lists them).`);
const tailed = skips.has('sync_tail') ? drained : await withSyncTail(engine, sourceId, drained, { signal: outer.signal, deadlineAtMs: outer.deadlineAtMs });
const healed = skips.has('slug_heal') ? tailed : await healAfterManagedSync(engine, outer, tailed);
return skips.has('origin_heal') ? healed : await healOriginTwinsAfterManagedSync(engine, outer, healed);
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index aa927bc710..f61856f6c2 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1560,7 +1560,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.693.0', id: 'member_save_withdrawn_scope_gst2346', text: 'A remote `remember` onto a member\'s own page (`memory/members/<id>`) is refused `fact_withdrawn` only when that member could read what was withdrawn: a fact on its own page, its own copy withdrawn by the brain host, or for the brain owner a fact it reads. A claim withdrawn by the brain host or the owner that the member never read no longer refuses its save: the fact saves as the member\'s own active row with the same reply as a new claim. A schema migration makes the database insert guards mark a row on a member page only by that page\'s own withdrawals; a withdrawal still expires matching rows on member pages when it is made, and strikes them in the page\'s file, so an import of a member page\'s file no longer marks a row withdrawn by a withdrawal of another subject. The same holds for each item of a `remember` batch (`items`), a replay with the same `request_id`, a `replaces` and the write receipt (`get_write_request`). Trusted local saves keep the check they had.' },
{ since: '0.60.694.0', id: 'owner_labels_guarded_gst2521', text: 'Labels the owner sets with `gbrain repair visibility` (or its `--readers` setter, or `gbrain repair private-labels`) now raise the page\'s label guard to operator_curated, so a later local agent write at the page\'s own tier that drops or widens them is held like a write over an owner page, with a `lower_page` item. A local write below operator_curated whose `readable_by` would leave the brain owner out of a page the owner reads now publishes with the owner kept in the readers (the rest of the narrowing applies), and its item records the requested readers. Accepting such an item (`gbrain trust confirm tp<id>`) on a page whose own tier is an agent\'s writes only the labels and keeps the agent\'s tier; on an owner page it still endorses the edit at the owner\'s tier. The guard is stored in the database (`page_label_guards`) and set only by label repairs run after the upgrade, so labels set earlier, and a managed Git brain rebuilt from a clone, start unguarded; re-running `gbrain repair visibility --slug` or `--readers` on a page that already has those labels raises the guard without changing them. The owner\'s own label writes, synced owner edits, agent narrowing that keeps the owner, and brains with `access.owner_reads_restricted` on are unchanged.' },
{ since: '0.60.694.0', id: 'page_label_widening_held_gst2518', text: 'A local agent write (`put_page`, `capture`, `edit_page`, `append_page`, `revert_version`, `restore_page`) less trusted than a page that drops or widens its `visibility`, `access`, `confidentiality` or `readable_by` now publishes with the page\'s earlier labels kept, in the database and in the committed file, and its receipt lists them under `privacy_labels_kept`; so does a maintenance job\'s derived page publication, and a page whose tier is still `unknown` counts as the owner\'s. A soft delete no longer lowers the page\'s tier. Before, the page became readable to remote callers and other members at once, and only a `lower_page` item waited for the owner. That item now records the requested labels: `gbrain trust confirm tp<id>` applies them if the page has not changed since, `gbrain trust drop tp<id>` keeps the earlier ones, and undoing a confirm restores them while the page is unchanged since. Narrowing writes, label-neutral writes, writers at or above the page\'s tier, synced owner edits and `gbrain repair visibility` are unchanged.' },
- { since: 'TBD', id: 'folder_holding_page_gst2492', text: 'On a source whose pages are canonical in Git (`canonical: git`), a fact for an entity with no page under a folder access rule is no longer kept in the database only: it is held on the holding page of the entity\'s folder (`partners/memory/unattributed` for `partners/acme-example`), which every rule covering the entity covers by path, so it is in Git (or database-only where the folder is, under `storage.db_only` or a path Git ignores) with exactly the readers the entity\'s page would have. The reply names that `page` and `write_through.held_for`. `gbrain access rule add` previews the held facts a new rule covers (`held_facts_moving`) and its apply moves them off `memory/unattributed` (`held_facts`), as do `gbrain access apply` and `gbrain access resume`; maintenance does the same for rules applied by a sync, and until then the fact reads only for the rule\'s readers. A held fact is now also hidden from a caller who cannot read its holding page, even under a rule with `covers_derived: false`. `gbrain repair facts-home` writes such facts to their folder\'s page and lists as `folder_restricted` only an exact page rule (`page_rule`) or unreadable rules (`access_unreadable`). A `remember` for a page-less entity under an exact page rule now refuses `git_home_required` (reason `facts_lane`, frozen `error` `invalid_params`) with a fix that creates the page, and one whose access rules this database role cannot read refuses `access_policy_pending` (`next: wait`); both wrote the fact to the database before. Sources that are not strict and brains with no Git repository are unchanged.' },
+ { since: 'TBD', id: 'folder_holding_page_gst2492', text: 'On a source whose pages are canonical in Git (`canonical: git`), a fact for an entity with no page under a folder access rule is no longer kept in the database only: it is held on the holding page of the entity\'s folder (`partners/memory/unattributed` for `partners/acme-example`), which every rule covering the entity covers by path, so it is in Git (or database-only where the folder is, under `storage.db_only` or a path Git ignores) with exactly the readers the entity\'s page would have. The reply names that `page` and `write_through.held_for`. `gbrain access rule add` previews the held facts a new rule covers (`held_facts_moving`) and its apply moves them off `memory/unattributed` (`held_facts`), as do `gbrain access apply`, `gbrain access resume` and a sync that applies a rule from Git (right after it releases its lease; maintenance is the fallback), and removes their rows from the shared page\'s saved versions in the database (`held_versions_editing` in the preview, `held_versions_edited` on the move\'s write receipt); Git history is not rewritten. A held fact is now also hidden from a caller who cannot read its holding page, even under a rule with `covers_derived: false`. `gbrain repair facts-home` writes such facts to their folder\'s page and lists as `folder_restricted` only an exact page rule (`page_rule`) or unreadable rules (`access_unreadable`). A `remember` for a page-less entity under an exact page rule now refuses `git_home_required` (reason `facts_lane`, frozen `error` `invalid_params`) with a fix that creates the page, and one whose access rules this database role cannot read refuses `access_policy_pending` (`next: wait`); both wrote the fact to the database before. Sources that are not strict and brains with no Git repository are unchanged.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/facts/fence-fact-append.ts b/src/core/facts/fence-fact-append.ts
index 4c3192614f..acdc63f7f2 100644
--- a/src/core/facts/fence-fact-append.ts
+++ b/src/core/facts/fence-fact-append.ts
@@ -32,7 +32,7 @@ import type { PageSnapshot } from '../page-state/types.ts';
import type { Page } from '../types.ts';
import { opError } from '../ops/contract.ts';
import { readFix } from '../ops/op-fix.ts';
-import { factsFenceSection, fenceCellReadBackProblem, formatFenceDate, parseFactsFenceFor, renderFactsTable, replaceOrInsertFactsFence, strikeFactRowInPage, upsertFactRow } from '../facts-fence.ts';
+import { FACTS_FENCE_BEGIN, FACTS_FENCE_END, factsFenceSection, fenceCellReadBackProblem, formatFenceDate, parseFactsFenceFor, renderFactsTable, replaceOrInsertFactsFence, strikeFactRowInPage, upsertFactRow } from '../facts-fence.ts';
import { serializePageToMarkdown } from '../markdown.ts';
import { pageBodyOverCeilingError } from '../page-body/page-shape.ts';
import { materializedPage, segmentedOverCeiling } from '../persistence/segmented-apply.ts';
@@ -225,3 +225,46 @@ export async function prepareFenceRowRetirement(engine: BrainEngine, row: WriteR
...(prepared.validate ? { validate: (tx: BrainEngine) => prepared.validate!(tx) } : {}), apply: (tx, preimage) => prepared.apply(tx, preimage) };
}
+
+/**
+ * GST-2492 (A2): the rows of `rows` (an entity's held claims) in the saved versions of holding page `slug`, removed when
+ * `write`. Each edited version keeps every other byte; only the table lines of those rows go. Database only: Git
+ * history is never rewritten. Idempotent (a second run finds nothing). Segmented versions are counted, not edited.
+ */
+export async function heldRowsInVersions(db: BrainEngine, sourceId: string, slug: string, rows: ReadonlyArray<{ entity: string; claim: string }>, write: boolean):
+ Promise<{ versions: number; rows: number; segmented: number }> {
+ const out = { versions: 0, rows: 0, segmented: 0 };
+ const keys = new Set(rows.map(r => `${r.entity}\0${r.claim.trim()}`));
+ const entities = [...new Set(rows.map(r => r.entity))];
+ if (!keys.size) return out;
+ const versions = await db.executeRaw<{ id: number; compiled_truth: string | null; timeline: string | null; segmented: boolean }>(
+ `SELECT v.id, v.compiled_truth, v.timeline, v.body_storage = 'segmented' AS segmented FROM page_versions v JOIN pages p ON p.id = v.page_id
+ WHERE p.source_id = $1 AND p.slug = $2 AND (v.body_storage = 'segmented' OR EXISTS (SELECT 1 FROM unnest($3::text[]) AS e
+ WHERE strpos(COALESCE(v.compiled_truth, ''), e) > 0 OR strpos(COALESCE(v.timeline, ''), e) > 0)) ORDER BY v.id${write ? ' FOR UPDATE OF v' : ''}`,
+ [sourceId, slug, entities]);
+ const strip = (text: string): [string, number] => {
+ let removed = 0;
+ for (let from = 0; ;) {
+ const begin = text.indexOf(FACTS_FENCE_BEGIN, from), end = begin < 0 ? -1 : text.indexOf(FACTS_FENCE_END, begin);
+ if (end < 0) return [text, removed];
+ const block = text.slice(begin, end + FACTS_FENCE_END.length);
+ const drop = new Set(parseFactsFenceFor(block, { slug }).facts.filter(f => f.entity && keys.has(`${f.entity}\0${f.claim.trim()}`)).map(f => f.rowNum));
+ const lines = block.split('\n');
+ const kept = lines.filter(line => !drop.has(Number(/^\|\s*(\d+)\s*\|/.exec(line)?.[1] ?? NaN)));
+ removed += lines.length - kept.length;
+ const next = kept.join('\n');
+ text = text.slice(0, begin) + next + text.slice(end + FACTS_FENCE_END.length);
+ from = begin + next.length;
+ }
+ };
+ for (const v of versions) {
+ if (v.segmented) { out.segmented++; continue; }
+ const [compiled, a] = strip(v.compiled_truth ?? ''), [timeline, b] = strip(v.timeline ?? '');
+ if (!a && !b) continue;
+ out.versions++; out.rows += a + b;
+ if (write) await db.executeRaw('UPDATE page_versions SET compiled_truth = $2, timeline = $3 WHERE id = $1',
+ [v.id, v.compiled_truth === null ? null : compiled, v.timeline === null ? null : timeline]);
+ }
+ return out;
+}
+
diff --git a/src/core/facts/held-fact-move.ts b/src/core/facts/held-fact-move.ts
index b3b3803d42..66456a928c 100644
--- a/src/core/facts/held-fact-move.ts
+++ b/src/core/facts/held-fact-move.ts
@@ -95,6 +95,17 @@ export async function ensurePageForHeldRows(engine: BrainEngine, sourceId: strin
type HeldRow = UnfencedFactRow & { from_slug: string };
+/**
+ * GST-2492 (B2): after a sync releases its lease, the held facts a folder rule it applied now covers leave the shared page
+ * at once instead of at the next maintenance cycle (still the fallback after a crash). Null when none is due.
+ */
+export async function moveRuledSharedHeld(engine: BrainEngine, sourceId: string): Promise<HeldFactMoveOutcome | null> {
+ const [due] = await engine.executeRaw<{ due: boolean }>(`SELECT EXISTS (SELECT 1 FROM facts f WHERE f.source_id = $1 AND f.source_markdown_slug = $2
+ AND f.row_num IS NOT NULL AND f.expired_at IS NULL AND f.entity_slug IS NOT NULL AND (gbrain_access_folder_rules(f.source_id, f.entity_slug) IS NOT NULL
+ OR gbrain_access_pending_covers(f.source_id, f.entity_slug))) AS due`, [sourceId, UNATTRIBUTED_MEMORY_SLUG]);
+ return due?.due ? moveHeldFacts(engine, sourceId) : null;
+}
+
export async function moveHeldFacts(engine: BrainEngine, sourceId: string): Promise<HeldFactMoveOutcome> {
const outcome: HeldFactMoveOutcome = { moved: 0, moved_slugs: [], rehomed: 0, unmovable: [], write_errors: 0 };
const rows = await engine.executeRaw<HeldRow>(
diff --git a/src/core/persistence/prepared-maintenance.ts b/src/core/persistence/prepared-maintenance.ts
index 4fe933facd..132eb66f35 100644
--- a/src/core/persistence/prepared-maintenance.ts
+++ b/src/core/persistence/prepared-maintenance.ts
@@ -11,7 +11,7 @@ import { currentSubmissionAuthority } from '../minions/submission-authority.ts';
import { parseMarkdown, serializePageToMarkdown } from '../markdown.ts';
import { fenceOperationError, scanCanonicalFences } from '../fence-repair/refusal.ts';
import { fenceCellReadBackProblem, formatFenceDate, parseFactsFence, parseFactsFenceFor, renderFactsTable, replaceOrInsertFactsFence, isHoldingSlug, UNATTRIBUTED_MEMORY_SLUG, type ParsedFact } from '../facts-fence.ts';
-import { prepareFenceRowRetirement } from '../facts/fence-fact-append.ts';
+import { heldRowsInVersions, prepareFenceRowRetirement } from '../facts/fence-fact-append.ts';
import { noteBelief } from '../belief/ledger.ts';
import { extractFactsFromFenceText } from '../facts/extract-from-fence.ts';
import { submissionAuthority, authorizeStoredRequest, authorizeWrite } from './authority.ts';
@@ -560,7 +560,17 @@ async function prepareFactFenceAdoption(unbounded: BrainEngine, row: WriteReques
const prepared = await preparePageMutation(unbounded, pageRow, config, undefined, undefined, { clock });
const holding = held ? await prepareFenceRowRetirement(unbounded, pageRow, config, fromSlug, { note: String(p.note), remove: p.note === 'moved',
rowNum: (await readFacts(engine, row.source_id, facts.map(f => f.id))).map(f => Number(f.value.row_num)), validUntil: null }) : null;
- return { ...prepared, ...(holding ? { companions: [holding] } : {}), validate: async tx => { await prepared.validate?.(tx); await check(tx, true); }, apply: async tx => {
+ // GST-2492 (A2): a row removed from the shared page (it moved where not every reader of the source can read) leaves its
+ // saved versions too, in this transaction; the receipt records which rows, how many versions and where they went.
+ const scrub = held && p.note === 'moved' && fromSlug === UNATTRIBUTED_MEMORY_SLUG
+ ? facts.map(f => ({ entity: holdingAdoption ? fence.get(f.row_num)!.entity ?? '' : fromEntity ?? row.slug, claim: fence.get(f.row_num)!.claim })) : null;
+ const afterEffects = scrub ? async (tx: BrainEngine, outcome: Record<string, unknown>) => {
+ await prepared.afterEffects?.(tx, outcome);
+ const edited = await heldRowsInVersions(tx, row.source_id, fromSlug, scrub, true);
+ outcome.held_versions_edited = { slug: fromSlug, moved_to: row.slug, fact_ids: facts.map(f => f.id), rows: edited.rows, versions: edited.versions,
+ ...(edited.segmented ? { segmented_versions_not_edited: edited.segmented } : {}) };
+ } : prepared.afterEffects;
+ return { ...prepared, ...(afterEffects ? { afterEffects } : {}), ...(holding ? { companions: [holding] } : {}), validate: async tx => { await prepared.validate?.(tx); await check(tx, true); }, apply: async tx => {
// Runs ahead of the page import and its canonical projection, so the
// projection's expiry pass and insertFacts see the adopted positions.
// The fence cell's parsed text is written back with the row number: the
diff --git a/test/folder-holding-page-gst2492.test.ts b/test/folder-holding-page-gst2492.test.ts
index 03898b8e0d..f4798e98b3 100644
--- a/test/folder-holding-page-gst2492.test.ts
+++ b/test/folder-holding-page-gst2492.test.ts
@@ -35,8 +35,9 @@ import { parseFactsFenceFor } from '../src/core/facts-fence.ts';
import { isFactEntityPage, isIdentityEntity } from '../src/core/entities/resolve.ts';
import { writeSingleFact } from '../src/core/facts/write-single.ts';
import { scanFactsHome } from '../src/core/facts/facts-home.ts';
-import { folderRulesAllow, type FolderRule } from '../src/core/access/folder-rules.ts';
+import { folderRulesAllow, serializePolicyYaml, type FolderRule } from '../src/core/access/folder-rules.ts';
import { policySeams, proposeRevision, resumePolicy } from '../src/core/access/policy.ts';
+import { commitPolicyFile, POLICY_FILE } from '../src/core/access/policy-git.ts';
import { identityFor } from '../src/core/access/principals.ts';
import { setOwner, setOwnerReadsRestricted } from '../src/core/access/owner.ts';
import { GBrainOAuthProvider } from '../src/core/oauth-provider.ts';
@@ -274,6 +275,121 @@ for (const backend of testBackends()) describe(`[${backend}] the folder-rule hol
expect((await as('bob', 'get_page', { slug: HOLD })).text).not.toContain('beta-example');
}), 300_000);
+ const versionBodies = (engine: BrainEngine, slug: string) => engine.executeRaw<{ id: number; body: string }>(
+ `SELECT v.id::int AS id, COALESCE(v.compiled_truth, '') || '\u0001' || COALESCE(v.timeline, '') AS body FROM page_versions v JOIN pages p ON p.id = v.page_id
+ WHERE p.source_id = 'default' AND p.slug = $1 ORDER BY v.id`, [slug]);
+ const versionsText = async (as: Reader, who: string, slug: string) => (await as(who, 'get_versions', { slug, include_body: true, limit: 50 })).text;
+ const auditRows = (engine: BrainEngine) => engine.executeRaw<{ audit: Record<string, unknown> }>(
+ `SELECT outcome->'held_versions_edited' AS audit FROM persistence_requests WHERE outcome ? 'held_versions_edited' ORDER BY created_at`);
+
+ test('a rule\'s apply removes the moved rows from the shared page\'s saved versions, byte for byte, once, with an audit record', () => strict(async ({ engine, ctx, root }) => {
+ const as = await members(engine);
+ await setOwner(engine, 'carol-example');
+ const entity = 'clients/acme-example/brief';
+ const holding = 'clients/acme-example/memory/unattributed';
+ const claim = 'The acme-example brief carries a quiet clause.';
+ const kept = 'The garden plan adds a third raised bed.';
+ await remember(ctx, claim, entity);
+ await remember(ctx, kept, 'notes/garden-plan');
+ await remember(ctx, 'The garden plan needs more compost.', 'notes/garden-plan');
+ await settleManagedGitEffects(engine);
+ for (const who of ['bob', 'unbound']) expect(await versionsText(as, who, HOLD)).toContain(claim);
+ const before = await versionBodies(engine, HOLD);
+ const preview = await cli(engine, ['rule', 'add', '--prefix', 'clients/acme-example/', '--readers', 'alice-example', '--source', 'default', '--json']);
+ const carrying = before.filter(v => v.body.includes(claim)).length;
+ expect(carrying).toBeGreaterThan(0);
+ expect(preview.held_versions_editing).toEqual({ rows: carrying, versions: carrying });
+ const applied = await cli(engine, ['rule', 'add', '--prefix', 'clients/acme-example/', '--readers', 'alice-example', '--source', 'default', '--yes', '--expect', String(preview.expect), '--json']);
+ expect(applied.held_facts).toEqual({ moved: 1, unmovable: [] });
+ await settleManagedGitEffects(engine);
+ const after = new Map((await versionBodies(engine, HOLD)).map(v => [v.id, v.body]));
+ for (const v of before) {
+ expect(after.get(v.id)).toBe(v.body.split('\n').filter(line => !line.includes(claim)).join('\n'));
+ }
+ expect([...after.values()].some(body => body.includes(kept))).toBe(true);
+ const [fact] = await factRows(engine, claim);
+ // The audit counts the versions the preview listed plus the one the move itself saved (the shared page before the move).
+ expect((await auditRows(engine)).map(r => r.audit)).toEqual([{ slug: HOLD, moved_to: holding, fact_ids: [fact!.id], rows: carrying + 1, versions: carrying + 1 }]);
+ for (const who of ['bob', 'unbound']) {
+ const text = await versionsText(as, who, HOLD);
+ expect(text).not.toContain(claim);
+ expect(text).not.toContain('clients/acme-example');
+ expect(text).toContain(kept);
+ }
+ await setOwnerReadsRestricted(engine, true, { remote: false });
+ __resetPrivateVisibilityCacheForTests();
+ expect(await reads(as, 'carol', holding, claim)).toEqual({ page: true, fact: true });
+ expect(await reads(as, 'alice', holding, claim)).toEqual({ page: true, fact: true });
+ // Git history is never rewritten: the shared page's earlier commit still holds the row.
+ expect(git(root, 'log', '-p', '--format=%H', '--', `${HOLD}.md`)).toContain(claim);
+ // A second apply finds nothing to move and edits nothing.
+ const again = await cli(engine, ['rule', 'add', '--prefix', 'clients/acme-example/', '--readers', 'alice-example', '--source', 'default', '--json']);
+ expect(again.held_versions_editing).toEqual({ rows: 0, versions: 0 });
+ expect((await cli(engine, ['resume', '--source', 'default', '--json'])).held_facts).toBeUndefined();
+ expect((await runExtractFacts(engine, { sourceId: 'default' })).heldFactsMoved ?? 0).toBe(0);
+ expect(new Map((await versionBodies(engine, HOLD)).map(v => [v.id, v.body]))).toEqual(after);
+ expect(await auditRows(engine)).toHaveLength(1);
+ }), 300_000);
+
+ test('a rule that arrives by sync moves the held facts right after the sync, beside a write in flight; after a crash, maintenance moves them', () => strict(async ({ engine, ctx, root }) => {
+ const as = await members(engine);
+ const moveModule = await import('../src/core/facts/held-fact-move.ts');
+ const original = moveModule.moveRuledSharedHeld;
+ const claim = 'The acme-example brief carries a synced clause.';
+ const crashed = 'The beta-example plan carries a synced clause.';
+ await remember(ctx, claim, 'clients/acme-example/brief');
+ await remember(ctx, crashed, 'clients/beta-example/plan');
+ await settleManagedGitEffects(engine);
+ const policy = (rules: FolderRule[]) => commitPolicyFile(root, POLICY_FILE, () => serializePolicyYaml(rules), 'access: rule');
+ const sync = () => performSync(engine, { sourceId: 'default', repoPath: root, noPull: true, noEmbed: true } as never);
+ // A write on the shared page is in flight while the post-sync move runs; both finish.
+ const inFlight = 'The garden plan has a fourth bed.';
+ let write: Promise<unknown> | null = null;
+ const spy = spyOn(moveModule, 'moveRuledSharedHeld').mockImplementation(async (e, s) => { write = remember(ctx, inFlight, 'notes/garden-plan'); return original(e, s); });
+ try {
+ policy([rule('acme', 'clients/acme-example/', ['alice-example'])]);
+ await sync();
+ expect(write).not.toBeNull();
+ await write;
+ } finally { spy.mockRestore(); }
+ expect(await factRows(engine, claim)).toMatchObject([{ source_markdown_slug: 'clients/acme-example/memory/unattributed' }]);
+ expect(await factRows(engine, inFlight)).toMatchObject([{ source_markdown_slug: HOLD }]);
+ for (const who of ['bob', 'unbound']) expect(await versionsText(as, who, HOLD)).not.toContain(claim);
+ // The sync finishes and the process dies before the move: the rows wait on the shared page until maintenance moves them.
+ const died = spyOn(moveModule, 'moveRuledSharedHeld').mockImplementation(async () => { throw new Error('process died'); });
+ try {
+ policy([rule('acme', 'clients/acme-example/', ['alice-example']), rule('beta', 'clients/beta-example/', ['alice-example'])]);
+ await sync();
+ } finally { died.mockRestore(); }
+ expect(await factRows(engine, crashed)).toMatchObject([{ source_markdown_slug: HOLD }]);
+ expect((await runExtractFacts(engine, { sourceId: 'default' })).heldFactsMoved).toBe(1);
+ expect(await factRows(engine, crashed)).toMatchObject([{ source_markdown_slug: 'clients/beta-example/memory/unattributed' }]);
+ for (const who of ['bob', 'unbound']) {
+ expect(await versionsText(as, who, HOLD)).not.toContain(crashed);
+ expect((await as(who, 'get_page', { slug: HOLD })).text).not.toContain('beta-example');
+ }
+ }), 300_000);
+
+ test('an exact page rule\'s apply creates the entity\'s page through the write door (Git before its row) and the preview names it', () => strict(async ({ engine, ctx, root }) => {
+ const entity = 'notes/secret-brief';
+ const claim = 'The secret brief names a launch date.';
+ expect(await remember(ctx, claim, entity)).toMatchObject({ page: HOLD });
+ await settleManagedGitEffects(engine);
+ const preview = await cli(engine, ['rule', 'add', '--slug', entity, '--readers', 'alice-example', '--source', 'default', '--json']);
+ expect(preview.held_facts_moving).toEqual([{ fact_id: expect.any(Number), entity_slug: entity, to: entity }]);
+ const lines: string[] = [];
+ const spy = spyOn(console, 'log').mockImplementation((...a: unknown[]) => { lines.push(a.map(String).join(' ')); });
+ try { await runAccessRules(engine, 'rule', ['add', '--slug', entity, '--readers', 'alice-example', '--source', 'default'], false); } finally { spy.mockRestore(); }
+ expect(lines.join('\n')).toContain(`to ${entity} (its page)`);
+ const applied = await cli(engine, ['rule', 'add', '--slug', entity, '--readers', 'alice-example', '--source', 'default', '--yes', '--expect', String(preview.expect), '--json']);
+ expect(applied.held_facts).toEqual({ moved: 1, unmovable: [] });
+ expect(head(root, `${entity}.md`)).toContain(claim);
+ const [created] = await engine.executeRaw<{ outcome: Record<string, any> }>(
+ `SELECT outcome FROM persistence_requests WHERE slug = $1 AND operation = 'put_page' AND state = 'committed' ORDER BY created_at LIMIT 1`, [entity]);
+ expect(created!.outcome.persistence).toMatchObject({ mode: 'filesystem', git_state: 'committed', git_commit: expect.any(String) });
+ expect(await factRows(engine, claim)).toMatchObject([{ entity_slug: entity, source_markdown_slug: entity }]);
+ }), 300_000);
+
test('a folder rule\'s own apply moves the held facts it covers off the shared page, and its preview counts them', () => strict(async ({ engine, ctx, root }) => {
const as = await members(engine);
const entity = 'clients/acme-example/brief';