Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Held private restatements stay off page reads for callers who cannot read them (GST-2891) (#9574)
Garry Tan6ec2eb84be1b
diff --git a/behavior-changes.d/held_row_page_reads_2891.md b/behavior-changes.d/held_row_page_reads_2891.mdnew file mode 100644index 0000000000..529ab31915--- /dev/null+++ b/behavior-changes.d/held_row_page_reads_2891.md@@ -0,0 +1,4 @@+---+kind: fix+---+A world fence row restating a private fact that waits for the operator's accept (a remote put_page or remember restatement) is now left out of page, chunk, search and edit reads for remote callers who cannot read that private fact: get_page (as_of and known_as_of included), get_versions bodies and diffs, fetch, search, query, search_exact, get_chunks, the entity card and the edit_page diff. Such a caller's put_page keeps the held row it could not read, restored exactly as stored. The owner's callers, the local CLI and a member reading their own member page see the page as before. A world copy of the claim saved about another entity no longer lets a remote remember skip that hold. A held row also stays private when the page is synced or rewritten, until the operator accepts it with gbrain trust confirm tp<N>. The page file in the brain repository still holds the world cell, and restatements that were never held stay world.diff --git a/changelog.d/capy-held-row-page-reads-2891.md b/changelog.d/capy-held-row-page-reads-2891.mdnew file mode 100644index 0000000000..6933671042--- /dev/null+++ b/changelog.d/capy-held-row-page-reads-2891.md@@ -0,0 +1,14 @@+---+important: security+---+**A private fact restated as world and held for the operator's accept no longer reaches remote callers through page, chunk, search and edit reads: get_page, get_versions, fetch, search, query, search_exact, get_chunks, the entity card and the edit_page diff leave the held row out for callers who cannot read the private fact.**++<!-- behavior-change: held_row_page_reads_2891 -->++### Fixed+- A remote `put_page` that restated a private fence row as world (#9196), or a remote `remember` that restated a private fact (GST-2875), stored the new fact private until the operator accepts the widening, but the page body kept the restatement in a world cell. Every remote caller who cannot read the private fact (another member, an unbound token) read it through get_page (whole, include_content, lines, grep, section, outline, as_of, known_as_of and windows around a chunk), get_versions with include_body (known_as_of bodies and diffs too), fetch, keyword and hybrid search, query, search_exact (snippets, and the match itself, on pages stored in segments too), get_chunks (on any source) and the entity card. A recorded-time read at the slug a page had before a rename leaves it out too, and search_exact still finds a page's public hit when an earlier hit in the same text is a held row. Those reads now leave out a world fence cell whose fact is stored narrower than world, including a row a chunk boundary cut in half and chunks written before the upgrade, so no rechunk is needed. The owner's callers, the local CLI and a member reading their own member page read exactly what they read before. If the held-row lookup fails, a page read fails rather than return the page, and search drops every fence row.+- Such a caller's `put_page` keeps the held row it could not read. Writing back a page read without it (a get_page, edit, put_page round trip, a whole new body, or a body with the lines around it deleted) restores the row exactly as stored, so the operator's accept still widens it. A row the writer sends at the held row's number cannot replace or reword it. A different claim moves to a new row number, and the same claim keeps every stored cell except a narrower visibility the writer chose. A member session under GBRAIN_ENFORCED_MEMBER reads such a page the same way, so its put_page keeps the held row and its edit_page diff leaves it out. The owner's own writes and the local CLI's are unchanged.+- The `edit_page` diff showed a remote caller the held row as unchanged context when an edit landed near it, on inline and segmented pages alike. The diff now leaves it out for callers who cannot read the private fact, as get_page does.+- An edit elsewhere in a held page (a hand edit synced from the repository, or a local `put_page`) could re-derive the held row as world without the accept when the private copy was less trusted than `user_confirmed`. A held row now stays private whatever the writer's tier until the operator accepts it. Accept still makes it world, and reject and undo still work.+- A remote writer could skip the GST-2875 hold by first saving the same claim as a world fact about another entity. The hold looked for an existing world copy anywhere in the source, found that one, and stored the restatement on the confirmed private fact's entity as world, so remote recall returned it. The world copy now has to be on the same entity.+- Not covered: the page file in the brain repository still carries the world cell, so anyone who reads the repository directly, or a Markdown export, reads it. Restatements that were never held (capture, extract_facts, a restatement written onto another page or entity, a reworded claim, and a remember whose private copy is no more trusted than the writer) stay world as before.diff --git a/src/cli/enforced-member.ts b/src/cli/enforced-member.tsindex 446e9c35b0..69eb18234f 100644--- a/src/cli/enforced-member.ts+++ b/src/cli/enforced-member.ts@@ -432,7 +432,9 @@ function writeTargets(opName: string, p: Record<string, unknown>): string[] {/*** The checks a write-checked operation passes before it runs as a trusted local write: its source and every target page* readable by the member, and no label change. remember: one fact on --entity, the exact slug of a page the member reads,- * nothing inferred and no visibility or share label (params.infer_entity is set to false).+ * nothing inferred and no visibility or share label (params.infer_entity is set to false). A put_page or edit_page is+ * sent with keep_hidden_rows (GST-2891): the member read the page without its held rows, so the put keeps them and the+ * edit diff leaves them out, as for a remote writer.*/export async function checkEnforcedWrite(engine: BrainEngine, command: string, opName: string, params: Record<string, unknown>, sourceId: string | undefined): Promise<void> {const ea = await enforcedAccess(engine, command);@@ -450,6 +452,7 @@ export async function checkEnforcedWrite(engine: BrainEngine, command: string, oif (opName === 'put_page') {const content = typeof params.content === 'string' ? params.content : '';if (await pageLabels(await storedPage(engine, source, slug)) !== await markdownLabels(content, slug)) return refuseEnforced(command, 'label_change', { engine });+ params.keep_hidden_rows = true;}if (opName === 'add_timeline_entry' && [params.summary, params.detail, params.source].some(t => typeof t === 'string' && FENCE_MARKER_RE.test(t))) {return refuseEnforced(command, 'label_change', { engine });@@ -459,6 +462,7 @@ export async function checkEnforcedWrite(engine: BrainEngine, command: string, oconst stored = await storedPage(engine, source, slug);const hidden = stored ? await hiddenResidue(stored.compiled_truth) + await hiddenResidue(stored.timeline) : '';if (hidden !== '' || texts.some(t => typeof t === 'string' && LABEL_TEXT_RE.test(t))) return refuseEnforced(command, 'label_change', { engine });+ if (opName === 'edit_page') params.keep_hidden_rows = true;}}diff --git a/src/core/eligibility/fence-overlay.ts b/src/core/eligibility/fence-overlay.tsindex f8d9eae7cd..9b2ad53eae 100644--- a/src/core/eligibility/fence-overlay.ts+++ b/src/core/eligibility/fence-overlay.ts@@ -6,6 +6,11 @@* - held rows (a write-gate hold for the row's fingerprint on this page) and* purged rows (a fact_purges tombstone; L3's import overlay already drops* them from canonical markdown, this is the backstop) are omitted;+ * - GST-2891: on a read for a caller that cannot read a private row+ * (`hideHeld`), a world cell whose fact is stored narrower (`heldRowSql`:+ * a widening the owner has not accepted, #9196 / GST-2875) is omitted and+ * listed in `held`. Chunks keep it, as every caller reads them; search+ * drops it from a delivered chunk for such a caller (stamp.ts);* - rows whose stored tier is below the page tier are taken out of the page's* chunks and emitted as separate chunks whose first line is a trust marker* naming the row tier (`fenceTrustMarker`, read back by `chunkTrustMarker`).@@ -28,6 +33,8 @@ import { compareTrust, isTrustTier, minTrust, TRUST_TIERS, type TrustTier } fromimport { compactTrustLabel } from './labels.ts';import { activationSuppressedSql } from './sql.ts';import { holdFingerprint } from '../write-gate-store.ts';+import { memberFactAdmitSql } from '../access/member-facts.ts';+import { worldOnlyFacts } from '../search/private-visibility.ts';/** The origin slot of a fence trust marker: the chunk holds facts-fence rows, not page prose. */export const FENCE_TRUST_ORIGIN = 'facts-fence';@@ -39,6 +46,8 @@ export interface FenceChunkOverlay {demote: ReadonlyMap<number, TrustTier>;/** Demoted rows carrying an unconfirmed instruction-family write-gate flag (CEO-20): their marker says so. */unconfirmed?: ReadonlySet<number>;+ /** GST-2891: world cells held from the current caller (`hideHeld`), all of them in `omit` too. */+ held?: ReadonlySet<number>;}export function fenceOverlayIsEmpty(overlay: FenceChunkOverlay | null | undefined): boolean {@@ -147,7 +156,7 @@ export function renderFenceLowTier(rows: readonly FenceLowTierRow[]): FenceOverl}/** World fence rows of a body, every complete block. */-function worldFenceRows(text: string | null | undefined): ParsedFact[] {+export function worldFenceRows(text: string | null | undefined): ParsedFact[] {if (!text || !text.includes('gbrain:facts:begin')) return [];return withdrawalFenceBlocks(text).flatMap(block => block.parsed.warnings.length ? [] : block.parsed.facts.filter(row => row.visibility === 'world'));}@@ -199,7 +208,24 @@ export function fenceRowHoldTexts(row: Pick<ParsedFact, 'claim' | 'context'>): Hreturn [row.claim, row.context ?? null, null];}-const OVERLAY_SQL = (held: boolean) => `WITH incoming AS MATERIALIZED (+/**+ * GST-2891: SQL true when the world fence cell (`rowNum`, `claim`) of page (`sourceId`, `slug`) is held from the current+ * caller: the live facts row at that row number is narrower than world, or the claim is a live narrower fact of the page+ * that no live world fact of the page states (the restatement rule without the tier, so it also holds a restatement whose+ * own row is not stored yet), and the caller's fact reads do not admit that narrower fact (memberFactAdmitSql). For a+ * caller that reads private facts (worldOnlyFacts false) nothing is held: callers skip the check. `partial` (an SQL+ * boolean) marks a claim cut by a chunk boundary: it then matches a fact that contains it rather than one equal to it.+ */+export function heldRowSql(sourceId: string, slug: string, rowNum: string, claim: string, partial = 'false'): string {+ const page = (a: string) => `${a}.source_id = ${sourceId} AND ${a}.source_markdown_slug = ${slug} AND ${a}.expired_at IS NULL`;+ const narrower = (a: string) => `${page(a)} AND ${a}.visibility <> 'world' AND NOT ${memberFactAdmitSql(a)}`;+ const states = (a: string) => `(CASE WHEN ${partial} THEN strpos(${a}.fact, ${claim}) > 0 ELSE ${a}.fact = ${claim} END)`;+ return `(EXISTS (SELECT 1 FROM facts hr WHERE ${narrower('hr')} AND hr.row_num = ${rowNum})+ OR (EXISTS (SELECT 1 FROM facts hc WHERE ${narrower('hc')} AND ${states('hc')})+ AND NOT EXISTS (SELECT 1 FROM facts hw WHERE ${page('hw')} AND hw.visibility = 'world' AND ${states('hw')})))`;+}++const OVERLAY_SQL = (held: boolean, hideHeld = false) => `WITH incoming AS MATERIALIZED (SELECT i.row_num, i.claim, i.visibility, i.fp FROM jsonb_to_recordset($3::text::jsonb) AS i(row_num integer, claim text, visibility text, fp text))SELECT incoming.row_num,(SELECT p.trust_tier FROM pages p WHERE p.source_id=$1 AND p.slug=$2 AND p.deleted_at IS NULL LIMIT 1) AS page_tier,@@ -210,7 +236,8 @@ const OVERLAY_SQL = (held: boolean) => `WITH incoming AS MATERIALIZED (EXISTS (SELECT 1 FROM fact_purges fp WHERE fp.source_id=$1 AND fp.visibility=incoming.visibilityAND fp.fact_hash=gbrain_fact_fingerprint(incoming.claim) AND (fp.subject='*' OR fp.subject=$2)) AS purged,${held ? `EXISTS (SELECT 1 FROM write_gate_holds h WHERE h.kind='fact' AND h.source_id=$1 AND h.slug=$2- AND h.status='held' AND h.fingerprint=incoming.fp)` : 'false'} AS held+ AND h.status='held' AND h.fingerprint=incoming.fp)` : 'false'} AS held,+ ${hideHeld ? heldRowSql('$1', '$2', 'incoming.row_num', 'incoming.claim') : 'false'} AS narrowerFROM incoming`;const OVERLAY_SQL_WITH_HOLDS = OVERLAY_SQL(true);const OVERLAY_SQL_WITHOUT_HOLDS = OVERLAY_SQL(false);@@ -218,6 +245,8 @@ const OVERLAY_SQL_WITHOUT_HOLDS = OVERLAY_SQL(false);export interface LoadFenceOverlayOpts {/** Overrides the registered L2a fingerprint for this call. */holdFingerprint?: ((texts: HoldTexts) => string) | null;+ /** GST-2891: a read for a caller whose fact reads are world-only (worldOnlyFacts): rows held from it are omitted. */+ hideHeld?: boolean;}/**@@ -236,16 +265,19 @@ export async function loadFenceChunkOverlay(const fingerprint = opts.holdFingerprint === undefined ? factHoldFingerprint : opts.holdFingerprint;const incoming = rows.map(row => ({ row_num: row.rowNum, claim: row.claim, visibility: row.visibility,fp: fingerprint ? fingerprint(fenceRowHoldTexts(row)) : null }));- const found = await engine.executeRaw<{ row_num: number; page_tier: string | null; row_tier: string | null; row_flagged: boolean | null; purged: boolean; held: boolean }>(- fingerprint ? OVERLAY_SQL_WITH_HOLDS : OVERLAY_SQL_WITHOUT_HOLDS, [page.sourceId, page.slug, JSON.stringify(incoming)]);+ const sql = opts.hideHeld ? OVERLAY_SQL(fingerprint !== null, true) : fingerprint ? OVERLAY_SQL_WITH_HOLDS : OVERLAY_SQL_WITHOUT_HOLDS;+ const found = await engine.executeRaw<{ row_num: number; page_tier: string | null; row_tier: string | null; row_flagged: boolean | null; purged: boolean; held: boolean; narrower: boolean }>(+ sql, [page.sourceId, page.slug, JSON.stringify(incoming)]);const pending = pendingRows.getStore();const pendingHere = pending && pending.sourceId === page.sourceId && pending.slug === page.slug ? new Set(pending.rowNums) : null;const omit = new Set<number>();const demote = new Map<number, TrustTier>();const unconfirmed = new Set<number>();+ const held = new Set<number>();for (const row of found) {const rowNum = Number(row.row_num);- if (row.purged === true || row.held === true) { omit.add(rowNum); continue; }+ if (row.narrower === true) held.add(rowNum);+ if (row.narrower === true || row.purged === true || row.held === true) { omit.add(rowNum); continue; }if (!isTrustTier(row.page_tier)) continue;const isPending = pendingHere?.has(rowNum) === true;const rowTier = isPending ? pending!.tier : isTrustTier(row.row_tier) ? row.row_tier : null;@@ -253,5 +285,15 @@ export async function loadFenceChunkOverlay(demote.set(rowNum, rowTier);if (isPending ? pending!.unconfirmed === true : row.row_flagged === true) unconfirmed.add(rowNum);}- return omit.size || demote.size ? { omit, demote, ...(unconfirmed.size ? { unconfirmed } : {}) } : undefined;+ return omit.size || demote.size ? { omit, demote, ...(unconfirmed.size ? { unconfirmed } : {}), ...(held.size ? { held } : {}) } : undefined;+}++/**+ * GST-2891: the world cells of a page body (the current page, an edition or a version) held from the caller when its+ * fact reads are world-only (heldRowSql, against the facts as they stand now), as sanitizeRemoteBody's `omitRows`;+ * undefined for any other caller, who reads the body as stored.+ */+export async function heldRowsForCaller(engine: Pick<BrainEngine, 'executeRaw'>, remote: boolean | undefined,+ page: { sourceId: string; slug: string; compiled_truth: string; timeline?: string | null }): Promise<ReadonlySet<number> | undefined> {+ return worldOnlyFacts(remote) ? (await loadFenceChunkOverlay(engine, page, { hideHeld: true }))?.held : undefined;}diff --git a/src/core/eligibility/stamp.ts b/src/core/eligibility/stamp.tsindex 56e514b460..a236b3af9a 100644--- a/src/core/eligibility/stamp.ts+++ b/src/core/eligibility/stamp.ts@@ -9,7 +9,8 @@*/import type { BrainEngine } from '../engine.ts';import { admitsTrust, compareTrust, minTrust, storedTrustTier, type TrustTier } from '../trust/tier.ts';-import { chunkFenceMarker, FENCE_TRUST_ORIGIN, fenceTrustMarker, lowestFenceTrustMarker } from './fence-overlay.ts';+import { chunkFenceMarker, FENCE_TRUST_ORIGIN, fenceTrustMarker, heldRowSql, lowestFenceTrustMarker } from './fence-overlay.ts';+import { decodeFenceCell, isSeparatorRow, parseRowCells } from '../fence-shared.ts';import { needsDataEnvelope, parseContested, renderTrustedText, trustFields, type Contested, type TrustFields } from './labels.ts';import { activationSuppressedSql, contestedRefSql } from './sql.ts';import { GUEST_ORIGIN, guestView, loadGuestMarks, unattributedGuest, withGuestSourceSql, type GuestStamp } from '../guests/marker.ts';@@ -74,10 +75,12 @@ export async function loadPageTrust(engine: Exec, refs: readonly PageRef[], opts* the backstop for rows a stage adds by slug (exact lookup, alias hop, graph* walk), so no row below the floor leaves the operation. `pageTrust` is* `loadPageTrust` started early over the rows these rows were copied from.+ * `hideHeld` (worldOnlyFacts for the caller) also drops the fence rows held from the caller (`dropHeldFenceRows`).*/export async function stampPageTrust<T extends PageRef>(engine: Exec, rows: T[], floor?: TrustTier,- pageTrust?: ReturnType<typeof loadPageTrust>): Promise<T[]> {+ pageTrust?: ReturnType<typeof loadPageTrust>, hideHeld = false): Promise<T[]> {if (rows.length === 0) return rows;+ if (hideHeld) rows = await dropHeldFenceRows(engine, rows);const factRows = rows.filter(row => typeof row.fact_row?.id === 'number');const factTrust = factRows.length ? await stampRowTrust(engine, 'facts', factRows, row => row.fact_row!.id) : [];factRows.forEach((row, i) => Object.assign(row, factTrust[i]));@@ -99,6 +102,73 @@ export async function stampPageTrust<T extends PageRef>(engine: Exec, rows: T[],return floor ? rows.filter(row => admitsTrust(row.trust_tier as TrustTier, floor)) : rows;}+/**+ * A text's lines that read as facts-fence table rows (`| 3 | claim |…`, inside a fence or not: a chunk can start+ * mid-table), with row number and claim. A chunk boundary can also cut a row line: the first line (after a leading fence+ * trust marker line, which a low-tier fence chunk starts with) can start inside a+ * row (inside its claim: the text before the first cell break; inside or just after its row-number cell of a world row:+ * the claim cell; row number unknown) and the last can stop inside one; both are `partial`, so the claim is matched as a+ * fragment.+ */+function fenceRowLines(text: string): Array<{ line: number; rowNum: number; claim: string; partial: boolean }> {+ const out: Array<{ line: number; rowNum: number; claim: string; partial: boolean }> = [];+ const lines = text.split('\n');+ const first = chunkFenceMarker(lines[0]) ? 1 : 0;+ lines.forEach((line, i) => {+ const cells = /^\s*\|\s*\d+\s*\|/.test(line) ? parseRowCells(line) : null;+ const claim = cells?.[1];+ const cut = i === first && /\|\s*world\s*\|/.test(line) ? (line.trim().startsWith('|') ? parseRowCells(line)?.[0] : /^\s*\d+\s*\|/.test(line) ? parseRowCells('| ' + line)?.[1] : null) : null;+ if (cells && claim) out.push({ line: i, rowNum: Number(cells[0]), claim: claim.replace(/^~~([\s\S]*)~~$/, '$1').replace(/^~~/, ''), partial: i === lines.length - 1 });+ else if (cut) out.push({ line: i, rowNum: -1, claim: cut.replace(/^~~|~~$/g, ''), partial: true });+ else if (i === first && !line.trim().startsWith('|') && /[^\\]\|/.test(line)) {+ const fragment = decodeFenceCell(line.split(/(?<!\\)\|/)[0]!.trim()).replace(/^~~|~~$/g, '');+ if (fragment) out.push({ line: i, rowNum: -1, claim: fragment, partial: true });+ }+ });+ return out;+}++/** Whether a text holds anything a reader reads once fence rows are gone: not only a marker, fence markers, a table header or separator. */+function holdsText(text: string): boolean {+ return text.split('\n').some((line, i) => {+ if (!line.trim() || line.includes('gbrain:facts:') || (i === 0 && chunkFenceMarker(line))) return false;+ const cells = parseRowCells(line);+ return !(cells && (isSeparatorRow(cells) || (cells[0] === '#' && cells[1] === 'claim')));+ });+}++/**+ * GST-2891: per row, the line numbers of the facts-fence row lines of its text held from a caller whose fact reads are+ * world-only (heldRowSql: a world cell whose fact is stored narrower, a widening the owner has not accepted), whenever+ * the text was cut. One batched lookup; a failed lookup, or a row that names no source, returns every fence row line+ * (fail closed).+ */+export async function heldFenceLines(engine: Exec, rows: readonly PageRef[]): Promise<Array<Set<number>>> {+ const perRow = rows.map(row => typeof row.chunk_text === 'string' ? fenceRowLines(row.chunk_text) : []);+ const keys = rows.flatMap((row, i) => { const src = row.source_id; return src == null ? [] : perRow[i]!.map(l => [src, row.slug, l.rowNum, l.claim, l.partial] as const); });+ let held: Set<string> | null = new Set();+ if (keys.length) try {+ const found = await engine.executeRaw<{ source_id: string; slug: string; row_num: number; claim: string }>(+ `SELECT DISTINCT k.source_id, k.slug, k.row_num, k.claim FROM unnest($1::text[], $2::text[], $3::int[], $4::text[], $5::text[]) AS k(source_id, slug, row_num, claim, partial)+ WHERE ${heldRowSql('k.source_id', 'k.slug', 'k.row_num', 'k.claim', "k.partial = 't'")}`,+ [keys.map(k => k[0]), keys.map(k => k[1]), keys.map(k => k[2]), keys.map(k => k[3]), keys.map(k => (k[4] ? 't' : 'f'))]);+ held = new Set(found.map(f => `${f.source_id}\u0000${f.slug}\u0000${Number(f.row_num)}\u0000${f.claim}`));+ } catch { held = null; }+ return rows.map((row, i) => new Set(perRow[i]!+ .filter(l => !held || row.source_id == null || held.has(`${row.source_id}\u0000${row.slug}\u0000${l.rowNum}\u0000${l.claim}`)).map(l => l.line)));+}++/** GST-2891: removes from each row's text the fence row lines held from the caller (heldFenceLines), and drops a row left with nothing to read. */+export async function dropHeldFenceRows<T extends PageRef>(engine: Exec, rows: T[]): Promise<T[]> {+ const drops = await heldFenceLines(engine, rows);+ return rows.filter((row, i) => {+ const drop = drops[i]!;+ if (!drop.size) return true;+ row.chunk_text = (row.chunk_text as string).split('\n').filter((_l, n) => !drop.has(n)).join('\n');+ return holdsText(row.chunk_text);+ });+}+/** Fence row numbers in a chunk's text: table rows (`| 3 | claim |…`) after a facts fence's begin marker. */function fenceRowNumbers(text: string | null | undefined): number[] {if (!text || !text.includes('gbrain:facts:begin')) return [];diff --git a/src/core/facts-fence.ts b/src/core/facts-fence.tsindex bfc36c87f3..2cdc608585 100644--- a/src/core/facts-fence.ts+++ b/src/core/facts-fence.ts@@ -596,6 +596,12 @@ export function renderFactsTable(facts: ParsedFact[]): string {* - Same rowNum + same claim: the caller already carries the row (e.g.* a full-content write-through) — the incoming version wins, nothing* restored, so the merge is idempotent.+ * - GST-2891: `held` names world rows of the existing fence held from the+ * writer (a world cell whose fact is stored narrower, which a world-only+ * reader never saw). Each is restored like a hidden row, as stored. A+ * same-claim incoming row at its number cannot reword it: the stored+ * cells win and only the writer's visibility is kept (a held row is a+ * world cell, so that can only narrow it: the GST-2511 withdrawal).* - Either side parsing with warnings returns null: re-rendering a* fence we could not fully parse would drop the caller's unparsed* rows. The residual data loss is surfaced by factsGapWarning below.@@ -607,14 +613,16 @@ export function renderFactsTable(facts: ParsedFact[]): string {export function restoreHiddenFactRows(incoming: { facts: ParsedFact[]; warnings: string[] },existing: { facts: ParsedFact[]; warnings: string[] },+ held: ReadonlySet<number> = new Set(),): { merged: ParsedFact[]; restored: ParsedFact[]; renumbered: Array<{ from: number; to: number }> } | null {if (incoming.warnings.length > 0 || existing.warnings.length > 0) return null;- const hidden = existing.facts.filter((f) => f.visibility !== 'world');+ const hidden = existing.facts.filter((f) => f.visibility !== 'world' || held.has(f.rowNum));if (hidden.length === 0) return null;const incomingByRowNum = new Map(incoming.facts.map((f) => [f.rowNum, f]));const restored: ParsedFact[] = [];const collidingRowNums = new Set<number>();+ const heldAsStored = new Map<number, ParsedFact>();for (const h of hidden) {const inc = incomingByRowNum.get(h.rowNum);if (!inc) {@@ -622,10 +630,12 @@ export function restoreHiddenFactRows(} else if (inc.claim !== h.claim) {collidingRowNums.add(h.rowNum);restored.push(h);+ } else if (h.visibility === 'world') {+ heldAsStored.set(h.rowNum, { ...h, visibility: inc.visibility });}- // claim-equal: incoming already carries the row; keep the incoming version.+ // claim-equal hidden row: incoming already carries the row; keep the incoming version.}- if (restored.length === 0) return null;+ if (restored.length === 0 && heldAsStored.size === 0) return null;let next = Math.max(0,@@ -634,6 +644,8 @@ export function restoreHiddenFactRows() + 1;const renumbered: Array<{ from: number; to: number }> = [];const kept = incoming.facts.map((f) => {+ const asStored = heldAsStored.get(f.rowNum);+ if (asStored) return asStored;if (!collidingRowNums.has(f.rowNum)) return f;const to = next++;renumbered.push({ from: f.rowNum, to });diff --git a/src/core/import-file.ts b/src/core/import-file.tsindex e741bb18bd..ef83441c7f 100644--- a/src/core/import-file.ts+++ b/src/core/import-file.ts@@ -62,58 +62,16 @@ import { warnOncePerProcess, validateSlug, contentHash, contentHashLegacy, ATOMSimport { decorateEmbeddingDimError } from './embedding-dim-check.ts';import { resolveImportContextualMode } from './import-contextual-mode.ts';import { runGuardrails } from './guardrails.ts';-import { fenceBody, pageBodySegmentedError } from './page-body/page-shape.ts';+import { pageBodySegmentedError } from './page-body/page-shape.ts';+import { mergeHiddenFactRows } from './import-hidden-rows.ts';import { abortSegmentedStaging, bodyStorageOf, demoteSegmentedPage, materializedPage, materializedSnapshot, sealSegmentedPage, stageSegmentedImport,stringWriteStorage, type SegmentedImport } from './persistence/segmented-apply.ts';import type { SegmentedStagingOwner } from './page-body/ingest.ts';import type { PageBodyPolicy } from './page-body/limits.ts';-import { parseFactsFenceFor, renderFactsTable, restoreHiddenFactRows, factsGapWarning, replaceOrInsertFactsFence } from './facts-fence.ts';import { assertOriginUnclaimed, isOriginClaimRefusal } from './persistence/origin-guard.ts';import { fileBodyMatchesStored, readStoredBody } from './facts/file-body-guard.ts';import { currentEngineScope, needsEngineScope, runInEngineScope, withEngineScope } from './engine-scope.ts';-/**- * #2044 / #4548: row-level, visibility-aware fence merge for one page- * column on the remote write-back boundary. Restores non-'world' fence- * rows that are missing from the incoming body (a remote get_page/fetch- * stripped them before the caller ever saw them, so their absence is not- * an intentional delete). World-visible rows are never restored — the- * caller saw those in full, so an edit/deletion is honored as written.- * See restoreHiddenFactRows() in facts-fence.ts for the merge rules- * (stable rowNums, collision renumbering, idempotence, warnings gate).- *- * Returns the (possibly merged) body. Emits console.warn for the two- * surfaced conditions: a caller-authored row renumbered off a hidden- * rowNum, and the residual malformed-fence case where the merge could not- * run and hidden rows are genuinely dropped (factsGapWarning).- */-function mergeHiddenFactRowsIntoBody(- slug: string,- incomingBody: string,- existingBody: string | null | undefined,-): string {- if (!existingBody) return incomingBody;- const incomingFacts = parseFactsFenceFor(incomingBody, { slug });- const existingFacts = parseFactsFenceFor(existingBody, { slug });- const merge = restoreHiddenFactRows(incomingFacts, existingFacts);- if (merge) {- if (merge.renumbered.length > 0) {- console.warn(- `[gbrain] #2044 merge on ${slug}: ${merge.renumbered.length} incoming fact row(s) reused ` +- `row number(s) belonging to privacy-hidden row(s) and were renumbered ` +- `(${merge.renumbered.map((r) => `#${r.from}->#${r.to}`).join(', ')}); ` +- `the hidden row(s) keep their original numbers.`,- );- }- // Shared #4756 placement rule (the column is already split at the- // sentinel here, so this is dedup, not a behaviour change for the importer).- return replaceOrInsertFactsFence(incomingBody, renderFactsTable(merge.merged));- }- const gapWarning = factsGapWarning(slug, incomingFacts, existingFacts, false);- if (gapWarning) console.warn(gapWarning);- return incomingBody;-}-/*** The parsed page metadata returned by importFromContent. Callers (specifically* the put_page operation handler running auto-link post-hook) can reuse this to@@ -327,6 +285,8 @@ export async function importFromContent(reuseSegments?: SegmentedImport;/** Untrusted caller (remote MCP put_page: `ctx.remote !== false`); drives the remote fence merge and hidden rows. */remote?: boolean;+ /** GST-2891: with `remote`, the writer read the page without its held fence rows (worldOnlyFacts), so the merge restores them as stored. */+ restoreHeldRows?: boolean;/*** #1699/#6259: gate-owned markers (`quarantine`, `content_flag`, `embed_skip`,* `atoms_scan_hash`, `quarantine_override`) are stripped from incoming content@@ -474,11 +434,7 @@ export async function importFromContent(// #6188 (E7): Tier 1 in place before the hidden-row merge and withdrawal preservation; the hash covers the normalized body.const fences = await applyImportFences({ engine, sourceId: sourceId ?? 'default', slug, mode: opts.fences ?? 'lenient', screened: opts.fenceScreen, screen, parsed, activePack: opts.activePack, remote: opts.remote === true, existing });if (fences.refused) return fences.refused;- if (opts.remote === true && existing) {- const stored = fenceBody(existing);- parsed.compiled_truth = mergeHiddenFactRowsIntoBody(slug, parsed.compiled_truth, stored.compiled_truth);- parsed.timeline = mergeHiddenFactRowsIntoBody(slug, parsed.timeline, stored.timeline);- }+ if (opts.remote === true && existing) await mergeHiddenFactRows(engine, slug, parsed, existing, opts.restoreHeldRows);const { assertPreparedFactWithdrawals, preserveWithdrawnFenceRows } = await import('./facts/withdrawal.ts');parsed.compiled_truth = (await preserveWithdrawnFenceRows(engine, sourceId ?? 'default', parsed.compiled_truth, slug)).trim();parsed.timeline = (await preserveWithdrawnFenceRows(engine, sourceId ?? 'default', parsed.timeline, slug)).trim();diff --git a/src/core/import-hidden-rows.ts b/src/core/import-hidden-rows.tsnew file mode 100644index 0000000000..a39f5797d1--- /dev/null+++ b/src/core/import-hidden-rows.ts@@ -0,0 +1,59 @@+import type { BrainEngine } from './engine.ts';+import type { Page } from './types.ts';+import { parseFactsFenceFor, renderFactsTable, restoreHiddenFactRows, factsGapWarning, replaceOrInsertFactsFence } from './facts-fence.ts';+import { loadFenceChunkOverlay } from './eligibility/fence-overlay.ts';+import { fenceBody } from './page-body/page-shape.ts';++/**+ * #2044 / #4548: row-level, visibility-aware fence merge for one page+ * column on the remote write-back boundary. Restores non-'world' fence+ * rows that are missing from the incoming body (a remote get_page/fetch+ * stripped them before the caller ever saw them, so their absence is not+ * an intentional delete). World-visible rows are never restored — the+ * caller saw those in full, so an edit/deletion is honored as written.+ * See restoreHiddenFactRows() in facts-fence.ts for the merge rules+ * (stable rowNums, collision renumbering, idempotence, warnings gate).+ *+ * Returns the (possibly merged) body. Emits console.warn for the two+ * surfaced conditions: a caller-authored row renumbered off a hidden+ * rowNum, and the residual malformed-fence case where the merge could not+ * run and hidden rows are genuinely dropped (factsGapWarning).+ */+function mergeHiddenFactRowsIntoBody(+ slug: string,+ incomingBody: string,+ existingBody: string | null | undefined,+ held?: ReadonlySet<number>,+): string {+ if (!existingBody) return incomingBody;+ const incomingFacts = parseFactsFenceFor(incomingBody, { slug });+ const existingFacts = parseFactsFenceFor(existingBody, { slug });+ const merge = restoreHiddenFactRows(incomingFacts, existingFacts, held);+ if (merge) {+ if (merge.renumbered.length > 0) {+ console.warn(+ `[gbrain] #2044 merge on ${slug}: ${merge.renumbered.length} incoming fact row(s) reused ` ++ `row number(s) belonging to privacy-hidden row(s) and were renumbered ` ++ `(${merge.renumbered.map((r) => `#${r.from}->#${r.to}`).join(', ')}); ` ++ `the hidden row(s) keep their original numbers.`,+ );+ }+ // Shared #4756 placement rule (the column is already split at the+ // sentinel here, so this is dedup, not a behaviour change for the importer).+ return replaceOrInsertFactsFence(incomingBody, renderFactsTable(merge.merged));+ }+ const gapWarning = factsGapWarning(slug, incomingFacts, existingFacts, false);+ if (gapWarning) console.warn(gapWarning);+ return incomingBody;+}++/**+ * The remote write-back merge of both page columns against the stored page (mergeHiddenFactRowsIntoBody), in place. With+ * `restoreHeld` (GST-2891: the writer's fact reads were world-only) the world rows held from it are restored too.+ */+export async function mergeHiddenFactRows(engine: BrainEngine, slug: string, parsed: { compiled_truth: string; timeline: string }, existing: Page,+ restoreHeld?: boolean): Promise<void> {+ const stored = fenceBody(existing);+ const held = restoreHeld ? (await loadFenceChunkOverlay(engine, { sourceId: existing.source_id, slug, ...stored }, { hideHeld: true }))?.held : undefined;+ for (const column of ['compiled_truth', 'timeline'] as const) parsed[column] = mergeHiddenFactRowsIntoBody(slug, parsed[column], stored[column], held);+}diff --git a/src/core/ops/admin.ts b/src/core/ops/admin.tsindex 9bfdca0059..acf4194fcd 100644--- a/src/core/ops/admin.ts+++ b/src/core/ops/admin.ts@@ -3,6 +3,8 @@ import { readPolicyOpts } from './context.ts';import { attributeVersions, canReadWriteAttribution } from './attribution.ts';import { GET_VERSIONS_DEFAULT_LIMIT, GET_VERSIONS_REMOTE_MAX_LIMIT, pagingLimit, reportVersionPage } from './read-paging.ts';import { sanitizeRemoteBody } from '../remote-body.ts';+import { heldRowsForCaller, worldFenceRows } from '../eligibility/fence-overlay.ts';+import { ALL_SOURCES } from '../source-id.ts';import { fenceTranscriptText, quotesStored } from '../search/transcript-quote.ts';import { historyQuarantine } from './get-page-projection.ts';/**@@ -22,6 +24,7 @@ import { VERSION } from '../../version.ts';import { resolveActiveEmbeddingColumnFromEngine } from '../search/embedding-column.ts';import { memoizedHealth } from '../health-memo.ts';import { remoteReadPolicySql } from '../persistence/page-visibility.ts';+import { worldOnlyFacts } from '../search/private-visibility.ts';import { LARGE_PAGES_CAPABILITY } from './get-page-body.ts';import { GUEST_MARKER_CAPABILITY } from '../guests/marker.ts';import { isSegmentedPage, pageBodyBytes } from '../page-body/page-shape.ts';@@ -282,7 +285,12 @@ const get_versions: Operation = {// #8042: a third-party page's version reaches an untrusted reader quoted, as get_page's current body does;// GST-687: so does one whose snapshot tier is below the owner's (a version with no snapshot reads as unknown).const quote = quotesStored(v.type, snapshotTiers.get(Number(v.id)));- const body = (text: string) => { const clean = sanitizeRemoteBody(text); return quote && clean ? fenceTranscriptText(clean, String(p.slug)) : clean; };+ const live = lives.find(l => Number(l.id) === Number(v.page_id));+ const sourceId = live?.source_id ?? (ctx.sourceId === ALL_SOURCES ? undefined : ctx.sourceId);+ const omitRows = sourceId !== undefined+ ? await heldRowsForCaller(ctx.engine, ctx.remote, { sourceId, slug: live?.slug ?? String(p.slug), compiled_truth: v.compiled_truth, timeline: v.timeline })+ : worldOnlyFacts(ctx.remote) ? new Set([...worldFenceRows(v.compiled_truth), ...worldFenceRows(v.timeline)].map(row => row.rowNum)) : undefined;+ const body = (text: string) => { const clean = sanitizeRemoteBody(text, { omitRows }); return quote && clean ? fenceTranscriptText(clean, String(p.slug)) : clean; };out.push({ ...v, compiled_truth: body(v.compiled_truth), ...reason(v),...(typeof v.timeline === 'string' ? { timeline: body(v.timeline) } : {}), ...(quote ? { quoted: 'transcript' } : {}) });}diff --git a/src/core/ops/chunks.ts b/src/core/ops/chunks.tsindex fadaf36b74..ac50e596f7 100644--- a/src/core/ops/chunks.ts+++ b/src/core/ops/chunks.ts@@ -7,7 +7,8 @@*/import { MIN_TRUST_PARAM, resolveReadEligibility } from '../eligibility/policy.ts';-import { stampPageTrust } from '../eligibility/stamp.ts';+import { dropHeldFenceRows, stampPageTrust } from '../eligibility/stamp.ts';+import { worldOnlyFacts } from '../search/private-visibility.ts';import { pageTypesById, quotePageText, quotesStored } from '../search/transcript-quote.ts';import type { Operation } from './contract.ts';import { readPolicyOpts } from './context.ts';@@ -71,8 +72,12 @@ const get_chunks: Operation = {const page = await ctx.engine.getChunkPage(p.slug as string, { ...scope, fromIndex, limit });// #5575: a chunk's tier is its page's (lowered by a fence trust marker); rows below the floor read as missing.const { floor } = await resolveReadEligibility(ctx, { minTrust: p.min_trust });- const stamped = (await stampPageTrust(ctx.engine, page.chunks.map(c => ({ ...c, slug: p.slug as string })), floor))- .map(({ slug: _slug, ...chunk }) => chunk);+ // GST-2891: a held row is looked up under the source and slug of the page each chunk belongs to.+ const owners = worldOnlyFacts(ctx.remote) && page.chunks.length ? new Map((await ctx.engine.executeRaw<{ id: number; slug: string; source_id: string }>(+ 'SELECT id, slug, source_id FROM pages WHERE id = ANY($1::int[])', [[...new Set(page.chunks.map(c => c.page_id))]])).map(o => [Number(o.id), o] as const)) : null;+ const kept = owners ? (await dropHeldFenceRows(ctx.engine, page.chunks.map(c => ({ ...c, slug: owners.get(c.page_id)?.slug ?? '', source_id: owners.get(c.page_id)?.source_id }))))+ .map(({ source_id: _source, slug: _slug, ...chunk }) => chunk) : page.chunks;+ const stamped = (await stampPageTrust(ctx.engine, kept.map(c => ({ ...c, slug: p.slug as string })), floor)).map(({ slug: _slug, ...chunk }) => chunk);// GST-687: an untrusted reader gets a chat page's chunks, or chunks below the owner's tier, quoted (tool calls summarized on a chat page).const types = ctx.remote === false ? null : await pageTypesById(ctx.engine, stamped.map(c => c.page_id));const chunks = types === null ? stamped : stamped.map(c => {diff --git a/src/core/ops/facts.ts b/src/core/ops/facts.tsindex 95d3adab4e..62bf3e6002 100644--- a/src/core/ops/facts.ts+++ b/src/core/ops/facts.ts@@ -551,7 +551,7 @@ const recall: Operation = {bumpLastRetrievedAt(ctx.engine, searchResults.map(r => r.page_id));const applied = recallDeliveryPlan(effectivePlan(evidencePlan, searchResults), p.budget_policy, queryText, budgetTokens, rows);if (applied) ({ results: searchResults, delivery } = await deliverEvidence(ctx.engine, searchResults, applied, { ...searchScope, excludePrivate, requireSafeChunks: ctx.remote !== false }));- searchResults = await stampPageTrust(ctx.engine, searchResults, eligibility.floor);+ searchResults = await stampPageTrust(ctx.engine, searchResults, eligibility.floor, undefined, worldOnlyFacts(ctx.remote));}// Pack what is delivered: results redacted for all; facts for remote only (localVerbatim).diff --git a/src/core/ops/get-page-body.ts b/src/core/ops/get-page-body.tsindex 23337d3883..53f3a87f9f 100644--- a/src/core/ops/get-page-body.ts+++ b/src/core/ops/get-page-body.ts@@ -33,6 +33,8 @@* 'full', anything else 'auto'.*/import type { Page } from '../types.ts';+import { worldOnlyFacts } from '../search/private-visibility.ts';+import { dropHeldFenceRows } from '../eligibility/stamp.ts';import type { OperationContext, ParamDef } from './contract.ts';import { opError, OperationError } from './contract.ts';import { invalidParam, readFix } from './op-fix.ts';@@ -340,19 +342,23 @@ async function aroundChunk(ctx: OperationContext, page: Page, doc: WindowDocumen{ legacy_error: 'invalid_params', detail: JSON.stringify({ revision, chunk_id: chunkId }),fix: sizeFix(page.slug, page.source_id, "Returns the current revision and outline; search again for a current chunk_id.") });}+ // GST-2891: a caller whose fact reads are world-only reads the chunk without the fence rows held from it.+ const chunkText = worldOnlyFacts(ctx.remote)+ ? (await dropHeldFenceRows(ctx.engine, [{ source_id: page.source_id, slug: page.slug, chunk_text: row.chunk_text }]))[0]?.chunk_text ?? ''+ : row.chunk_text;const chunk = { chunk_source: row.chunk_source,projected_byte_start: row.projected_byte_start == null ? null : Number(row.projected_byte_start),projected_byte_end: row.projected_byte_end == null ? null : Number(row.projected_byte_end) };if (doc.segments) {const found = await doc.segments.aroundChunk({ ...chunk, occurrence_id: Number(row.occurrence_id), chunk_text: row.chunk_text }, contextLines, maxBytes);return found- ? { window: found.window, chunkText: row.chunk_text, anchor: 'stored_span', span: { byte_start: found.span.start, byte_end: found.span.end } }- : { window: null, chunkText: row.chunk_text, anchor: 'chunk_only' };+ ? { window: found.window, chunkText, anchor: 'stored_span', span: { byte_start: found.span.start, byte_end: found.span.end } }+ : { window: null, chunkText, anchor: 'chunk_only' };}const inline = doc.inline!;const stored = chunkDocumentSpan(inline, chunk);if (stored && spanHoldsChunk(inline, stored, row.chunk_text)) {- return { window: windowAround(inline.bytes, stored, contextLines, maxBytes), chunkText: row.chunk_text, anchor: 'stored_span', span: { byte_start: stored.start, byte_end: stored.end } };+ return { window: windowAround(inline.bytes, stored, contextLines, maxBytes), chunkText, anchor: 'stored_span', span: { byte_start: stored.start, byte_end: stored.end } };}// No stored span (a chunk written before spans were recorded) or a stale one: accept a text match only when it is unique.const needle = Buffer.from(row.chunk_text.trim(), 'utf8');@@ -360,10 +366,10 @@ async function aroundChunk(ctx: OperationContext, page: Page, doc: WindowDocumenconst at = inline.bytes.indexOf(needle);if (at !== -1 && inline.bytes.indexOf(needle, at + 1) === -1) {const span = { start: at, end: at + needle.length };- return { window: windowAround(inline.bytes, span, contextLines, maxBytes), chunkText: row.chunk_text, anchor: 'located', span: { byte_start: span.start, byte_end: span.end } };+ return { window: windowAround(inline.bytes, span, contextLines, maxBytes), chunkText, anchor: 'located', span: { byte_start: span.start, byte_end: span.end } };}}- return { window: null, chunkText: row.chunk_text, anchor: 'chunk_only' };+ return { window: null, chunkText, anchor: 'chunk_only' };}/** What a partial read cuts: an inline page's document bytes, or a segmented page's stored sums and segments. */@@ -399,7 +405,7 @@ function inlineDocument(doc: PageDocument): WindowDocument {*/async function segmentedWindows(ctx: OperationContext, input: PartialReadInput, req: BodyRequest, mode: BodyMode): Promise<{ doc: WindowDocument; size: Record<string, unknown> }> {const { page, revision, tags } = input;- const body = await openSegmentedBody(ctx.engine, { pageId: page.id, pin: { ...input.pin, revision } });+ const body = await openSegmentedBody(ctx.engine, { pageId: page.id, pin: { ...input.pin, revision }, hideHeld: worldOnlyFacts(ctx.remote) });const audience = ctx.remote === false ? 'trusted' : 'remote';const stored = audience === 'trusted' ? { body_bytes: body.bodyBytes, body_lines: body.bodyLines } : {};let seg: SegmentedDocument;@@ -507,7 +513,7 @@ export async function readPartialBody(ctx: OperationContext, input: PartialReadIelse {const whole: PageDocument = markdown? { bytes: Buffer.from(serializePageToMarkdown(input.visibleBody, tags), 'utf8'), timelineByte: -1 }- : await readPageDocument(ctx.engine, page);+ : await readPageDocument(ctx.engine, page, { hideHeld: worldOnlyFacts(ctx.remote) });size = {total_bytes: whole.bytes.length, total_lines: totalLines(whole.bytes),...(whole.timelineByte >= 0 ? { timeline_byte_start: whole.timelineByte } : {}),diff --git a/src/core/ops/known-as-of.ts b/src/core/ops/known-as-of.tsindex ad79a5c69c..2e99885850 100644--- a/src/core/ops/known-as-of.ts+++ b/src/core/ops/known-as-of.ts@@ -12,6 +12,7 @@ import { inlineBodyBytes } from './get-page-body.ts';import { historyQuarantine, pageTrustForRead, type QuarantinedView } from './get-page-projection.ts';import { parseKnownAsOf, readPageAtTime, remoteMayRead, type PageAtTime, type PageInterval } from '../belief/page-at-time.ts';import { sanitizeRemoteBody } from '../remote-body.ts';+import { heldRowsForCaller } from '../eligibility/fence-overlay.ts';import { remoteReadFloorSql } from '../search/private-visibility.ts';import { unifiedDiff } from '../skillpack/diff-text.ts';import { fenceTranscriptText, quotesStored } from '../search/transcript-quote.ts';@@ -66,31 +67,34 @@ async function readAsCaller(ctx: OperationContext, q: Parameters<typeof readPage}}-type GatedInterval = PageInterval & { quarantined?: QuarantinedView };+type GatedInterval = PageInterval & { quarantined?: QuarantinedView; live?: { sourceId: string; slug: string } };/*** Readable intervals: a remote caller sees an interval only under the ledger's visibility rule and the page's current* trust floor. #6259: when the page is quarantined now, or was in the interval, the interval keeps no body, as the- * current read keeps none (the same view and notice as get_page).+ * current read keeps none (the same view and notice as get_page). GST-2891: each carries the live page's source and slug,+ * where its held rows are looked up (a rename moves a page's facts to its new slug).*/async function readable(ctx: OperationContext, read: PageAtTime, intervals: PageInterval[], minTrust: unknown): Promise<GatedInterval[]> {if (ctx.remote === false) return intervals;if (read.page_id == null || !(await pageTrustForRead(ctx, { id: read.page_id }, minTrust))) return [];const [live] = await ctx.engine.executeRaw<{ slug: string; source_id: string; frontmatter: Record<string, unknown> }>('SELECT slug, source_id, frontmatter FROM pages WHERE id = $1', [read.page_id]);- const quarantine = historyQuarantine(ctx, live ?? { slug: read.slug, source_id: read.source_id, frontmatter: {} });+ if (!live) return [];+ const quarantine = historyQuarantine(ctx, live);const out: GatedInterval[] = [];for (const i of intervals) {if (!(await remoteMayRead(ctx.engine, read.page_id, i))) continue;const view = quarantine(i.page?.frontmatter);+ const held = { sourceId: live.source_id, slug: live.slug };out.push(view?.body_omitted && i.page- ? { ...i, body: 'unavailable', page: { ...i.page, compiled_truth: null, timeline: null, body_bytes: 0 }, quarantined: view }- : i);+ ? { ...i, live: held, body: 'unavailable', page: { ...i.page, compiled_truth: null, timeline: null, body_bytes: 0 }, quarantined: view }+ : { ...i, live: held });}return out;}-function intervalView(ctx: OperationContext, i: GatedInterval, opts: { body: boolean; inlineMax: number }) {+async function intervalView(ctx: OperationContext, sourceId: string, i: GatedInterval, opts: { body: boolean; inlineMax: number }) {const meta = { slug: i.slug, revision: i.revision, deleted: i.deleted, recorded_at: i.event.recorded_at, visible_at: i.event.visible_at, until: i.until,source_time: i.event.source_time, event: i.event.type, event_id: i.event.id, origin: i.event.origin, body: i.body,...(i.body_reason ? { body_reason: i.body_reason } : {}), ...(i.version_id != null ? { version_id: i.version_id } : {}),@@ -100,7 +104,8 @@ function intervalView(ctx: OperationContext, i: GatedInterval, opts: { body: booif (i.body === 'segmented' || bytes > opts.inlineMax) {return { ...meta, body_ref: { total_bytes: bytes } };}- const clean = (s: string | null) => s == null ? null : ctx.remote === false ? s : sanitizeRemoteBody(s, { includeWithdrawn: true, keepMaterializedMarkers: true });+ const omitRows = await heldRowsForCaller(ctx.engine, ctx.remote, { ...(i.live ?? { sourceId, slug: i.slug }), compiled_truth: i.page.compiled_truth ?? '', timeline: i.page.timeline });+ const clean = (s: string | null) => s == null ? null : ctx.remote === false ? s : sanitizeRemoteBody(s, { includeWithdrawn: true, keepMaterializedMarkers: true, omitRows });// #8042: a third-party page as recorded reaches an untrusted reader quoted, as get_page's current body does.const quote = quotesInterval(ctx, i);const fence = (s: string | null) => s && quote ? fenceTranscriptText(s, i.slug) : s;@@ -134,7 +139,7 @@ export async function getPageKnownAsOf(ctx: OperationContext, p: Record<string,const current = read.current && (await readable(ctx, read, [read.current], p.min_trust))[0];if (!current || (current.deleted && p.include_deleted !== true)) throw pageNotFoundError(ctx, slug, { includeDeleted: p.include_deleted === true, sourceIdParam: sourceId });const ignored = GET_PAGE_UNSUPPORTED.filter(k => p[k] !== undefined && p[k] !== false);- return { ...intervalView(ctx, current, { body: p.body !== 'none', inlineMax: await inlineBodyBytes(ctx) }), ...ledgerView(read),+ return { ...await intervalView(ctx, sourceId, current, { body: p.body !== 'none', inlineMax: await inlineBodyBytes(ctx) }), ...ledgerView(read),...(ignored.length ? { set_aside: { ignored, note: `Read the whole page as recorded at known_as_of; ignored ${ignored.length > 1 ? `${ignored.slice(0, -1).join(', ')} and ${ignored[ignored.length - 1]}` : ignored[0]}, which read the current page. Drop known_as_of to read a window or view of the current page.` } } : {}) };}@@ -148,19 +153,21 @@ export async function getVersionsKnownAsOf(ctx: OperationContext, p: Record<striconst intervals = await readable(ctx, read, read.intervals, p.min_trust);if (!intervals.length) throw pageNotFoundError(ctx, slug, { includeDeleted: true, sourceIdParam: sourceId });const inlineMax = await inlineBodyBytes(ctx);- const views = intervals.map((i, n) => {- const view: Record<string, unknown> = intervalView(ctx, i, { body: wantBody, inlineMax });+ const views = await Promise.all(intervals.map(async (i, n) => {+ const view: Record<string, unknown> = await intervalView(ctx, sourceId, i, { body: wantBody, inlineMax });if (p.diff !== true) return view;const prev = intervals.slice(0, n).reverse().find(x => x.lifetime === i.lifetime);const text = (x: GatedInterval | undefined) => x?.page && x.body !== 'segmented' && !x.quarantined?.body_omitted ? `${x.page.compiled_truth ?? ''}${x.page.timeline ? `\n${x.page.timeline}` : ''}` : null;const [a, b] = [prev ? text(prev) : '', text(i)];- const diff = a == null || b == null ? null : unifiedDiff(ctx.remote === false ? a : sanitizeRemoteBody(a), ctx.remote === false ? b : sanitizeRemoteBody(b),+ const held = (x: GatedInterval | undefined, t: string) => x ? heldRowsForCaller(ctx.engine, ctx.remote, { ...(x.live ?? { sourceId, slug: x.slug }), compiled_truth: t }) : undefined;+ const diff = a == null || b == null ? null : unifiedDiff(ctx.remote === false ? a : sanitizeRemoteBody(a, { omitRows: await held(prev, a) }),+ ctx.remote === false ? b : sanitizeRemoteBody(b, { omitRows: await held(i, b) }),{ oldPath: prev ? `${slug}@${prev.event.recorded_at}` : '/dev/null', newPath: `${slug}@${i.event.recorded_at}` });const quote = diff !== null && (quotesInterval(ctx, i) || (prev !== undefined && quotesInterval(ctx, prev)));view.diff = quote ? fenceTranscriptText(diff, i.slug) : diff;if (quote) view.quoted = 'transcript';return view;- });+ }));const newest = views.reverse();return { slug, ...ledgerView(read), truncated: newest.length > limit, versions: newest.slice(0, limit) };}diff --git a/src/core/ops/pages.ts b/src/core/ops/pages.tsindex 4b5869bf46..9307722951 100644--- a/src/core/ops/pages.ts+++ b/src/core/ops/pages.ts@@ -32,6 +32,7 @@ import { getContentFlag } from '../quarantine.ts';import { fileHeldField, readHeldPages } from '../persistence/held-reads.ts';import { bumpLastRetrievedAt } from '../last-retrieved.ts';import { resolveExcludePrivatePages, isPrivatePage, findPrivateOnlySlugs, isPrivatePageForCaller } from '../search/private-visibility.ts';+import { heldRowsForCaller } from '../eligibility/fence-overlay.ts';import { unquoteAnyTranscriptText } from '../search/transcript-quote.ts';import { withIgnoredPrivacyLabelNotice } from '../search/ignored-privacy-labels.ts';import { LIST_PAGES_DESCRIPTION, CAPTURE_DESCRIPTION } from '../operations-descriptions.ts';@@ -87,10 +88,11 @@ async function dropPrivateSlugs(* split boundary), which get_page/fetch_page returned verbatim, unstripped.* A private fact fence misplaced there was fully readable by any remote MCP* caller. Same stripping rule as compiled_truth: takes fence dropped- * entirely, facts fence keeps only `world`-visibility rows.+ * entirely, facts fence keeps only `world`-visibility rows, less those held from a world-only caller (GST-2891).*/-function stripPrivacyFencesForRemoteReader(page: Page): Page {- const opts = { includeWithdrawn: true, keepMaterializedMarkers: true }; // #5567: markers round-trip remote edits+async function stripPrivacyFencesForRemoteReader(ctx: OperationContext, page: Page): Promise<Page> {+ const omitRows = await heldRowsForCaller(ctx.engine, ctx.remote, { sourceId: page.source_id, slug: page.slug, compiled_truth: page.compiled_truth, timeline: page.timeline });+ const opts = { includeWithdrawn: true, keepMaterializedMarkers: true, omitRows }; // #5567: markers round-trip remote editsreturn { ...page, compiled_truth: sanitizeRemoteBody(page.compiled_truth, opts), timeline: sanitizeRemoteBody(page.timeline ?? '', opts) };}@@ -209,9 +211,7 @@ const get_page: Operation = {// Only explicitly trusted local reads retain protected body sections.// Holder grants and page-visibility opt-outs do not bypass this boundary.const isUntrustedReader = ctx.remote !== false;- const visibleBody = isUntrustedReader && page.body_storage !== 'segmented'- ? stripPrivacyFencesForRemoteReader(page)- : page;+ const visibleBody = isUntrustedReader && page.body_storage !== 'segmented' ? await stripPrivacyFencesForRemoteReader(ctx, page) : page;// v0.42 (#1699) agent-warning channel: surface the page's content_flag// marker as a top-level field (parallel to SearchResult.content_flag) so// an agent reading a page directly gets the same "this looks odd, examine@@ -315,7 +315,7 @@ const fetch_page: Operation = {const tags = snapshot!.tags;// Same boundaries as get_page: untrusted readers never see takes or private facts fences, nor a quarantined body (#6259).const segmented = isSegmentedPage(page);- const visibleBody = ctx.remote === false || segmented ? page : stripPrivacyFencesForRemoteReader(page);+ const visibleBody = ctx.remote === false || segmented ? page : await stripPrivacyFencesForRemoteReader(ctx, page);const quarantined = readQuarantined(ctx, page, p.include_quarantined === true);unverifiedOriginNotice(ctx, page, readable.trust);if (!quarantined?.body_omitted && (segmented || (ctx.remote !== false && bodyBytes(visibleBody) > await inlineBodyBytes(ctx)))) {diff --git a/src/core/ops/search-exact.ts b/src/core/ops/search-exact.tsindex d091e4620c..7bbab13141 100644--- a/src/core/ops/search-exact.ts+++ b/src/core/ops/search-exact.ts@@ -9,11 +9,67 @@ import { invalidParam } from './op-fix.ts';import { assertExplicitSourceLive, federatedSearchScope, parseSourceIdParam } from './context.ts';import { MIN_TRUST_PARAM, resolveReadEligibility } from '../eligibility/policy.ts';import {- EXACT_DEFAULT_LIMIT, EXACT_DEFAULT_SECONDS, EXACT_MAX_LIMIT, EXACT_MAX_PHRASE_CHARS, EXACT_MAX_SECONDS, runExactScan,+ EXACT_DEFAULT_LIMIT, EXACT_DEFAULT_SECONDS, EXACT_MAX_LIMIT, EXACT_MAX_PHRASE_CHARS, EXACT_MAX_SECONDS, EXACT_SNIPPET_CONTEXT, EXACT_SNIPPETS_PER_PAGE,+ phrasePattern, runExactScan,} from '../search/exact-scan.ts';+import type { ExactMatch } from '../search/exact-scan.ts';+import type { BrainEngine } from '../engine.ts';+import { heldFenceLines } from '../eligibility/stamp.ts';+import { chunkVisibleFilter, safeChunksFilter } from '../search/safe-chunks.ts';const TYPE_NAME = /^[A-Za-z0-9_.:/-]+$/;+/**+ * GST-2891: a caller whose fact reads are world-only gets no snippet text from a fence row held from it (heldFenceLines,+ * read on the stored chunk the snippet was cut from, so a snippet that starts or stops inside a row is placed). A page+ * whose snippets all read as stored, one per matching chunk, keeps them. Any other page is scanned again over every+ * visible chunk with its held lines taken out, so a public hit after a held one is still found and a held hit is+ * neither shown nor counted; a page left without a hit goes, since its presence alone would confirm the phrase.+ */+async function withoutHeldSnippets<M extends ExactMatch>(engine: BrainEngine, matches: M[], find: (text: string) => [number, number] | null): Promise<M[]> {+ const keys = matches.flatMap(m => m.snippets.map(s => ({ m, s })));+ if (!keys.length) return matches;+ const found = await engine.executeRaw<{ i: number; chunk_text: string }>(+ `SELECT k.i, cc.chunk_text FROM unnest($1::int[], $2::int[], $3::text[]) WITH ORDINALITY AS k(page_id, chunk_index, snippet, i)+ JOIN pages p ON p.id = k.page_id JOIN content_chunks cc ON cc.page_id = k.page_id AND cc.chunk_index = k.chunk_index+ WHERE ${chunkVisibleFilter('p', 'cc')} AND strpos(cc.chunk_text, k.snippet) > 0`,+ [keys.map(k => k.m.page_id), keys.map(k => k.s.chunk_index), keys.map(k => k.s.text)]);+ const chunks = found.map(f => ({ key: keys[Number(f.i) - 1]!, text: f.chunk_text }));+ const held = await heldFenceLines(engine, chunks.map(c => ({ source_id: c.key.m.source_id, slug: c.key.m.slug, chunk_text: c.text })));+ const touched = new Set<ExactMatch['snippets'][number]>(keys.map(k => k.s));+ chunks.forEach((c, n) => {+ let at = 0;+ const ranges: Array<[number, number]> = [];+ c.text.split('\n').forEach((line, l) => { if (held[n]!.has(l)) ranges.push([at, at + line.length + 1]); at += line.length + 1; });+ for (let o = c.text.indexOf(c.key.s.text); o !== -1; o = c.text.indexOf(c.key.s.text, o + 1)) {+ if (ranges.some(([a, b]) => a < o + c.key.s.text.length && b > o)) return;+ }+ touched.delete(c.key.s);+ });+ const rescan = matches.filter(m => m.matching_chunks !== m.snippets.length || m.snippets.some(s => touched.has(s)));+ if (!rescan.length) return matches;+ const rows = await engine.executeRaw<{ page_id: number; chunk_index: number; chunk_source: string; chunk_text: string }>(+ `SELECT cc.page_id, cc.chunk_index, cc.chunk_source, cc.chunk_text FROM content_chunks cc JOIN pages p ON p.id = cc.page_id+ WHERE cc.page_id = ANY($1::int[]) AND ${chunkVisibleFilter('p', 'cc')} AND ${safeChunksFilter('p')} ORDER BY cc.page_id, cc.chunk_index`,+ [rescan.map(m => m.page_id)]);+ const owner = new Map(rescan.map(m => [m.page_id, m] as const));+ const lines = await heldFenceLines(engine, rows.map(r => ({ source_id: owner.get(Number(r.page_id))?.source_id, slug: owner.get(Number(r.page_id))?.slug ?? '', chunk_text: r.chunk_text })));+ const hits = new Map<number, ExactMatch['snippets']>();+ rows.forEach((r, n) => {+ const text = r.chunk_text.split('\n').map((line, l) => (lines[n]!.has(l) ? '\u0000' : line)).join('\n');+ const at = find(text);+ if (!at) return;+ const id = Number(r.page_id);+ hits.set(id, [...(hits.get(id) ?? []), { chunk_index: Number(r.chunk_index), chunk_source: r.chunk_source,+ text: text.slice(Math.max(0, at[0] - EXACT_SNIPPET_CONTEXT), at[1] + EXACT_SNIPPET_CONTEXT).split('\u0000').join('') }]);+ });+ return matches.flatMap(m => {+ if (!owner.has(m.page_id)) return [m];+ const page = hits.get(m.page_id) ?? [];+ return page.length ? [{ ...m, matching_chunks: page.length, snippets: page.slice(0, EXACT_SNIPPETS_PER_PAGE) }] : [];+ });+}+function parseParams(ctx: OperationContext, p: Record<string, unknown>) {const bad = (param: string, message: string, example: unknown) =>invalidParam(ctx, 'search_exact', param, 'search_exact: ' + message, { def: search_exact.params[param], example });@@ -70,7 +126,7 @@ const search_exact: Operation = {const scope = federatedSearchScope(ctx, sourceIdParam);const { floor } = await resolveReadEligibility(ctx, { minTrust: p.min_trust });await assertExplicitSourceLive(ctx, sourceIdParam);- const { resolveExcludePrivatePages } = await import('../search/private-visibility.ts');+ const { resolveExcludePrivatePages, worldOnlyFacts } = await import('../search/private-visibility.ts');const excludePrivate = await resolveExcludePrivatePages(ctx.engine, ctx.remote);const { resolveHardExcludes } = await import('../search/source-boost.ts');const excludeSlugPrefixes = resolveHardExcludes();@@ -89,7 +145,25 @@ const search_exact: Operation = {deadlineMs: startedAt + params.seconds * 1000,});const { stampPageTrust } = await import('../eligibility/stamp.ts');- const matches = await stampPageTrust(ctx.engine, result.matches.map(m => ({ ...m, chunk_text: m.snippets.map(s => s.text).join('\n') })), floor);+ const pattern = phrasePattern(params.phrase);+ const needle = (params.caseSensitive ? params.phrase.trim() : params.phrase.trim().toLowerCase()).replace(/\s+/g, ' ');+ const find = (text: string): [number, number] | null => {+ let folded = '';+ const at: number[] = [];+ for (let x = 0; x < text.length; x++) {+ const space = /\s/.test(text[x]!);+ if (space && folded.endsWith(' ')) continue;+ const ch = space ? ' ' : params.caseSensitive ? text[x]! : text[x]!.toLowerCase();+ folded += ch;+ for (let k = 0; k < ch.length; k++) at.push(x);+ }+ const o = folded.indexOf(needle);+ return o === -1 ? null : [at[o]!, at[o + needle.length - 1]! + 1];+ };+ const visible = worldOnlyFacts(ctx.remote)+ ? pattern ? await withoutHeldSnippets(ctx.engine, result.matches, find) : []+ : result.matches;+ const matches = await stampPageTrust(ctx.engine, visible.map(m => ({ ...m, chunk_text: m.snippets.map(s => s.text).join('\n') })), floor);// GST-687: an untrusted reader gets a chat page's snippets, or snippets below the owner's tier, quoted (tool calls summarized on a chat page).const { quotePageText, quotesStored } = await import('../search/transcript-quote.ts');const quoted = <M extends Omit<(typeof matches)[number], 'chunk_text'>>(m: M) => ctx.remote !== false && quotesStored(m.type, (m as { trust_tier?: unknown }).trust_tier)diff --git a/src/core/ops/search.ts b/src/core/ops/search.tsindex e1d0a81972..a704e41f7f 100644--- a/src/core/ops/search.ts+++ b/src/core/ops/search.ts@@ -45,7 +45,7 @@ import { WHERE_PARAM, whereOpParam } from './where-param.ts';import { searchFilterHint } from '../search/filter-hint.ts';import { TargetTrace, diagnoseProbe, diagnoseTrace, probeTarget, type ExplainTargetDiagnosis } from '../search/explain-target.ts';import { assembleEvidenceForHits, capDeliveredSnippets, capEvidenceToBudget, countEvidenceTokens, deliverEvidence, pagePlanHits, queryBudgetUsesAuto, resolveEvidencePlan, selectLongPages, unsupportedDelivery, withLongPagePolicy, type DeliveryMeta, type DeliveryScope, type EvidencePlan, type FrozenHit, type ReturnUnit } from '../search/evidence-delivery.ts';-import { resolveExcludePrivatePages } from '../search/private-visibility.ts';+import { resolveExcludePrivatePages, worldOnlyFacts } from '../search/private-visibility.ts';import { SAFE_FENCE_CHUNKER_VERSION, chunkManifestReads } from '../search/safe-chunks.ts';import { expandEngineTypeFilters } from '../schema-pack/query-types.ts';import { reconcileTypeFilter } from './search-type-filter.ts';@@ -217,7 +217,7 @@ async function evidenceOutput(ctx: OperationContext, p: Record<string, unknown>,const pageTrust = loadPageTrust(ctx.engine, results.filter(r => typeof r.fact_row?.id !== 'number'));pageTrust.catch(() => {});const ev = delivered ?? await timings.part('response', 'evidence', () => withEvidence(ctx, p, results, plan, scope, meta));- const rows = await timings.part('response', 'trust', () => stampPageTrust(ctx.engine, keepEnumerated ? keepEnumeratedRows(results, ev.rows) : ev.rows, scope.minTrust, pageTrust));+ const rows = await timings.part('response', 'trust', () => stampPageTrust(ctx.engine, keepEnumerated ? keepEnumeratedRows(results, ev.rows) : ev.rows, scope.minTrust, pageTrust, worldOnlyFacts(ctx.remote)));const responseMeta = await timings.part('response', 'meta', () => buildMeta(rows));return searchOutput(ctx, p, rows, responseMeta, snippetCap, timings, ev.evidence);});@@ -1026,7 +1026,7 @@ const query: Operation = {}))).map(r => ({ ...r }));stampDeepResearchIds(results);return timings.stage('response', async () => {- const labeled = await stampPageTrust(ctx.engine, results, querySourceScope.minTrust);+ const labeled = await stampPageTrust(ctx.engine, results, querySourceScope.minTrust, undefined, worldOnlyFacts(ctx.remote));imageMeta.retrieved_count = labeled.length;return searchOutput(ctx, p, labeled, { ...await buildRetrievalResponseMeta(ctx, querySourceScope, queryText ?? '', labeled, imageMeta, { types }), ...(plan && (plan.unit !== 'auto' || plan.explicitUnit) ? { delivery: unsupportedDelivery(plan, 'image_query_unsupported') } : {}) }, snippetCap, timings);});diff --git a/src/core/ops/valid-time-read.ts b/src/core/ops/valid-time-read.tsindex 4c76e8b7d9..6408aaf6f5 100644--- a/src/core/ops/valid-time-read.ts+++ b/src/core/ops/valid-time-read.ts@@ -20,10 +20,11 @@ import { pageTrustForRead, readQuarantined } from './get-page-projection.ts';import { admitsTrust, type TrustTier } from '../trust/tier.ts';import { isCalendarDate } from '../link-validity.ts';import { replacedBecauseSql } from '../engine-sql/pages.ts';-import { callerReadersAllowedSql, privacyForCaller, privateLinkOriginFilterFragment, privatePagesFilterForCaller, privatePagesFilterFragment, privateSnapshotFilterFragment, remoteReadFloorSql, resolveExcludePrivatePages } from '../search/private-visibility.ts';+import { callerReadersAllowedSql, privacyForCaller, privateLinkOriginFilterFragment, privatePagesFilterForCaller, privatePagesFilterFragment, privateSnapshotFilterFragment, remoteReadFloorSql, resolveExcludePrivatePages, worldOnlyFacts } from '../search/private-visibility.ts';import { fenceTranscriptText, isQuotedPageType, neutralizeToolCalls, quotesStored } from '../search/transcript-quote.ts';import { quotedPageMeta } from './get-page-projection.ts';import { sanitizeRemoteBody } from '../remote-body.ts';+import { heldRowsForCaller } from '../eligibility/fence-overlay.ts';import { serializePageToMarkdown } from '../markdown.ts';import { pageDocument } from '../page-window/document.ts';import { DEFAULT_WINDOW_BYTES, MAX_WINDOW_BYTES, decodeCursor, encodeCursor, totalLines, windowFrom, type WindowSlice } from '../page-window/window.ts';@@ -256,9 +257,10 @@ async function loadEdition(tx: BrainEngine, identity: Identity, row: PickRow): P*/async function editionView(ctx: OperationContext, identity: Identity, e: Edition, asOf: string, offset: number | null, maxBytes: number, quote: boolean): Promise<Record<string, unknown> | 'gone'> {const neutralize = quote && ctx.thinClient !== true && isQuotedPageType(e.type);+ const omitRows = e.segmented ? undefined : await heldRowsForCaller(ctx.engine, ctx.remote, { sourceId: identity.source_id, slug: identity.slug, compiled_truth: e.compiled_truth, timeline: e.timeline });const clean = (s: string) => {if (ctx.remote === false) return s;- const safe = sanitizeRemoteBody(s, { includeWithdrawn: true, keepMaterializedMarkers: true });+ const safe = sanitizeRemoteBody(s, { includeWithdrawn: true, keepMaterializedMarkers: true, omitRows });return neutralize ? neutralizeToolCalls(safe) : safe;};const fence = (s: string) => quote && s ? fenceTranscriptText(s, identity.slug) : s;@@ -270,7 +272,7 @@ async function editionView(ctx: OperationContext, identity: Identity, e: Editionlet total: number, slice: WindowSlice, lines: number | undefined;if (e.segmented) {try {- const body = await openSegmentedBody(ctx.engine, { pageId: identity.id, ...(e.generation !== undefined ? { generation: e.generation } : {}) });+ const body = await openSegmentedBody(ctx.engine, { pageId: identity.id, ...(e.generation !== undefined ? { generation: e.generation } : {}), hideHeld: worldOnlyFacts(ctx.remote) });const seg = await segmentedReadDocument(ctx.engine, body);total = seg.total;slice = await seg.from(offset ?? 0, maxBytes);diff --git a/src/core/page-body/read-range.ts b/src/core/page-body/read-range.tsindex 1053a88be4..f4dc4ce26f 100644--- a/src/core/page-body/read-range.ts+++ b/src/core/page-body/read-range.ts@@ -98,6 +98,8 @@ export interface SegmentedBody {layout?: ProjectedLayout;/** Set when the read follows an authorization: confirmBodyPin checks it again before the text is returned. */pin?: BodyPin;+ /** GST-2891: a read for a caller whose fact reads are world-only: its fence overlays omit the rows held from it. */+ hideHeld?: boolean;}/** Where every row of the read projection sits (`projectedLayout`). */@@ -191,7 +193,7 @@ function manifestRows(engine: BrainEngine, pageId: number, generation: number |* up to LIVE_READ_ATTEMPTS times (GBRA-287).*/export async function openSegmentedBody(engine: BrainEngine, opts: { pageId: number; generation?: number; ledger?: BodyLedger; pin?: BodyPin;- expect?: { slug: string; sourceId: string } }, attempt = 1): Promise<SegmentedBody> {+ expect?: { slug: string; sourceId: string }; hideHeld?: boolean }, attempt = 1): Promise<SegmentedBody> {const requested = opts.pin?.generation ?? opts.generation ?? null;const [[page], read, expected] = await Promise.all([engine.executeRaw<Record<string, unknown>>(`SELECT id, source_id, slug, body_storage, body_generation, body_bytes, body_lines, truth_bytes,@@ -223,7 +225,7 @@ export async function openSegmentedBody(engine: BrainEngine, opts: { pageId: numreturn {pageId: opts.pageId, sourceId, slug, generation, bodyBytes: Number(page.body_bytes ?? 0), bodyLines: Number(page.body_lines ?? 0), truthBytes: Number(page.truth_bytes ?? 0),flags: (toJson(page.body_flags) ?? {}) as JsonObject, ledger, stale: Number(page.segmenter_version) !== SEGMENTER_VERSION, ...(opts.pin ? { pin: opts.pin } : {}),- rows: bodyRows,+ ...(opts.hideHeld ? { hideHeld: true } : {}), rows: bodyRows,};}@@ -271,7 +273,7 @@ function overlaidText(engine: BrainEngine, body: SegmentedBody, seg: LoadedSegme/** The fence overlay for a segment: the page's (layout), plus the rows of fence blocks outside its facts slots. */async function segmentOverlay(engine: BrainEngine, body: SegmentedBody, seg: LoadedSegment, text: string, base: FenceChunkOverlay | undefined) {if (seg.row.hasFacts || !text.includes('gbrain:facts:begin')) return base;- const extra = await loadFenceChunkOverlay(engine, { sourceId: body.sourceId, slug: body.slug, compiled_truth: fenceBlockText(text) });+ const extra = await loadFenceChunkOverlay(engine, { sourceId: body.sourceId, slug: body.slug, compiled_truth: fenceBlockText(text) }, { hideHeld: body.hideHeld });if (!extra) return base;if (!base) return extra;return { omit: new Set([...base.omit, ...extra.omit]), demote: new Map([...base.demote, ...extra.demote]),@@ -317,7 +319,7 @@ export async function projectedLayout(engine: BrainEngine, body: SegmentedBody):blocks.push(fenceBlockText(text));summaries.set(seg.row.seq, summarize(await projectLoaded(engine, body, seg, undefined)));}- const overlay = blocks.length ? await loadFenceChunkOverlay(engine, { sourceId: body.sourceId, slug: body.slug, compiled_truth: blocks.join('\n') }) : undefined;+ const overlay = blocks.length ? await loadFenceChunkOverlay(engine, { sourceId: body.sourceId, slug: body.slug, compiled_truth: blocks.join('\n') }, { hideHeld: body.hideHeld }) : undefined;if (overlay) for await (const seg of loadSegments(engine, body, factsRows)) summaries.set(seg.row.seq, summarize(await projectLoaded(engine, body, seg, overlay)));const n = body.rows.length;diff --git a/src/core/page-window/document.ts b/src/core/page-window/document.tsindex e1241684bf..2f6672829f 100644--- a/src/core/page-window/document.ts+++ b/src/core/page-window/document.ts@@ -81,8 +81,9 @@ export function pageDocument(page: ProjectableBody): PageDocument {export async function readPageDocument(engine: Pick<BrainEngine, 'executeRaw'>,page: { source_id: string; slug: string; compiled_truth: string; timeline?: string | null },+ opts: { hideHeld?: boolean } = {},): Promise<PageDocument> {- const fenceOverlay = await loadFenceChunkOverlay(engine, { sourceId: page.source_id, slug: page.slug, compiled_truth: page.compiled_truth, timeline: page.timeline });+ const fenceOverlay = await loadFenceChunkOverlay(engine, { sourceId: page.source_id, slug: page.slug, compiled_truth: page.compiled_truth, timeline: page.timeline }, opts);return pageDocument({ compiled_truth: page.compiled_truth, timeline: page.timeline ?? '', fenceOverlay });}diff --git a/src/core/persistence/page-edit.ts b/src/core/persistence/page-edit.tsindex e8f692f3db..8b164904f2 100644--- a/src/core/persistence/page-edit.ts+++ b/src/core/persistence/page-edit.ts@@ -31,7 +31,8 @@ import { spliceWork } from '../page-body/splice.ts';import { isSegmentedPage, withoutBody } from '../page-body/page-shape.ts';import { STAGE_BATCH_BYTES } from '../page-body/limits.ts';import { readEntryState } from '../page-body/entry-state.ts';-import { hiddenBefore, openSegmentedBody, trustedSegments, type SegmentedBody } from '../page-body/read-range.ts';+import { hiddenBefore, openSegmentedBody, projectedLayout, trustedSegments, type SegmentedBody } from '../page-body/read-range.ts';+import { storedWriterReadsHeld } from './page-visibility.ts';import { ledgerApplies, type BodyLedger } from '../page-body/segment-ledger.ts';import { TIMELINE_SEPARATOR } from '../page-window/document.ts';import { PAGE_MAX_BYTES } from '../page-size-policy.ts';@@ -118,7 +119,7 @@ async function assertSegmentedWithinVisible(engine: BrainEngine, page: { pageId:if (await hiddenBefore(engine, await openSegmentedBody(engine, page), within.byte_end, 'stored')) throw hiddenWithinError();}-function bodySegments(body: string, remote: boolean, segmented: boolean): Segment[] {+function bodySegments(body: string, remote: boolean, segmented: boolean, omitRows?: ReadonlySet<number>): Segment[] {if (remote && sanitizeText(body) !== body) {throw editError('edit_invalid', 'This page contains bytes the remote view cannot reproduce.','Edit it from the brain host with put_page or the local CLI.');@@ -137,7 +138,7 @@ function bodySegments(body: string, remote: boolean, segmented: boolean): Segmenif (marker !== open.end) throw unrepairedFence();cursor = token.index + marker.length;const fence = body.slice(open.start, cursor);- segments.push({ canonical: fence, view: remote ? sanitizeRemoteBody(fence, segmented ? SEGMENTED_REMOTE_VIEW : REMOTE_VIEW) : fence, editable: false });+ segments.push({ canonical: fence, view: remote ? sanitizeRemoteBody(fence, { ...(segmented ? SEGMENTED_REMOTE_VIEW : REMOTE_VIEW), omitRows }) : fence, editable: false });open = undefined;}if (open) throw unrepairedFence();@@ -156,14 +157,14 @@ function unrepairedFence(): OperationError {* `content`; adjacent editable pieces coalesce so a match may span the* frontmatter, body and timeline sentinel like any ordinary text.*/-function pageSegments(page: Page, tags: string[], remote: boolean): Segment[] {+function pageSegments(page: Page, tags: string[], remote: boolean, omitRows?: ReadonlySet<number>): Segment[] {const head = serializePageToMarkdown({ ...page, compiled_truth: '', timeline: '' }, tags).slice(0, -1);const segmented = isSegmentedPage(page);- const timeline = bodySegments(page.timeline ?? '', remote, segmented);+ const timeline = bodySegments(page.timeline ?? '', remote, segmented, omitRows);const sentinel = '\n\n<!-- timeline -->\n\n';// A remote view whose whole timeline is protected has no sentinel either.const sentinelSegment = timeline.some(segment => segment.view.length > 0) ? text(sentinel) : { canonical: sentinel, view: '', editable: false };- const pieces = [text(head), ...bodySegments(page.compiled_truth ?? '', remote, segmented),+ const pieces = [text(head), ...bodySegments(page.compiled_truth ?? '', remote, segmented, omitRows),...(page.timeline ? [sentinelSegment, ...timeline] : needsEmptyTimelineSentinel(page.compiled_truth ?? '') ? [text('\n\n<!-- timeline -->')] : []), text('\n')];return pieces.reduce<Segment[]>((out, piece) => {const last = out[out.length - 1];@@ -189,13 +190,14 @@ function occurrences(haystack: string, needle: string): number[] {/*** Applies the edits in order, each to the text produced by the previous one,* all or nothing. Returns the canonical content to publish and the caller's- * view before and after (the only text a diff or receipt may show).+ * view before and after (the only text a diff or receipt may show). `omitRows` are the fence rows held from a+ * world-only caller (GST-2891), left out of its view as get_page leaves them out.*/-export function applyPageEdits(page: Page, tags: string[], remote: boolean, edits: PageEdit[], within?: EditWithin): { content: string; before: string; after: string } {+export function applyPageEdits(page: Page, tags: string[], remote: boolean, edits: PageEdit[], within?: EditWithin, omitRows?: ReadonlySet<number>): { content: string; before: string; after: string } {// #6188 (D19): edit_page never rewrites a fence; a stored fence that does not compile refuses typed, location only.const [defect] = scanCanonicalFences(page).defects;if (defect) throw targetFenceRefusal(defect, page.slug, page.source_id);- let segments = pageSegments(page, tags, remote);+ let segments = pageSegments(page, tags, remote, omitRows);const before = join(segments, 'view');// `within` is in stored-body bytes of the page as read: canonical positions map back through the edits applied so far.const head = serializePageToMarkdown({ ...page, compiled_truth: '', timeline: '' }, tags).slice(0, -1).length;@@ -520,7 +522,7 @@ function hiddenRuns(rows: readonly ManifestRow[]): Array<{ start: number; end: n* line counts, the separator before the timeline and the containing row's prefix. A remote view renders each hidden* run before `at` as sanitizeRemoteBody does, so only those runs are read, never the ordinary text before `at`.*/-async function viewNewlinesBefore(engine: BrainEngine, input: TextSource & { remote: boolean; head: string }, rows: readonly ManifestRow[],+async function viewNewlinesBefore(engine: BrainEngine, input: TextSource & { remote: boolean; head: string; omitRows?: ReadonlySet<number> }, rows: readonly ManifestRow[],timeline: boolean, at: number): Promise<number> {const count = (b: Uint8Array) => { let n = 0; for (let i = b.indexOf(0x0a); i !== -1; i = b.indexOf(0x0a, i + 1)) n++; return n; };const before = rows.filter(row => Number(row.byteStart) < at && (timeline || row.region === 'truth'));@@ -533,7 +535,7 @@ async function viewNewlinesBefore(engine: BrainEngine, input: TextSource & { remconst text: Buffer[] = [];let stored = 0;for await (const piece of virtualPieces(engine, input, rows, [], run.start, run.end)) { text.push(Buffer.from(piece.bytes)); stored += count(piece.withdrawn ?? piece.bytes); }- lines += count(Buffer.from(sanitizeRemoteBody(Buffer.concat(text).toString('utf8'), SEGMENTED_REMOTE_VIEW), 'utf8')) - stored;+ lines += count(Buffer.from(sanitizeRemoteBody(Buffer.concat(text).toString('utf8'), { ...SEGMENTED_REMOTE_VIEW, omitRows: input.omitRows }), 'utf8')) - stored;}return lines;}@@ -605,7 +607,7 @@ export interface LocatedSegmentedEdits {* head, or (without `within`, within the whole-body limit) one the body does not hold, which that path settles exactly.*/export async function locateSegmentedEdits(engine: BrainEngine, input: { sourceId: string; slug: string; page: SplicePage; edits: PageEdit[]; within?: EditWithin;- remote: boolean; head: string; ledger: BodyLedger }): Promise<LocatedSegmentedEdits | null> {+ remote: boolean; head: string; ledger: BodyLedger; omitRows?: ReadonlySet<number> }): Promise<LocatedSegmentedEdits | null> {const { page, within } = input;const rows = await engine.readManifest(page.id, { generation: page.generation });spliceWork.rows += rows.length;@@ -660,7 +662,7 @@ export async function locateSegmentedEdits(engine: BrainEngine, input: { sourceI* editDiff over the touched range (line-rounded, with context), in the caller's view: the text after the withdrawal* overlay, and for a remote caller sanitized as its windows show it.*/-async function segmentedEditDiff(engine: BrainEngine, input: TextSource & { slug: string; remote: boolean; head: string }, rows: readonly ManifestRow[],+async function segmentedEditDiff(engine: BrainEngine, input: TextSource & { slug: string; remote: boolean; head: string; omitRows?: ReadonlySet<number> }, rows: readonly ManifestRow[],regions: ReadonlyArray<{ lo: number; hi: number; rows: ManifestRow[] }>, reps: readonly BodyReplacement[]): Promise<Record<string, unknown>> {let scoped = reps;if (reps[reps.length - 1]!.end - reps[0]!.start > DIFF_SCOPE_MAX_BYTES || regions.some(r => reps[0]!.start < r.hi && reps[reps.length - 1]!.end > r.hi)) scoped = reps.slice(0, 1);@@ -698,7 +700,7 @@ async function segmentedEditDiff(engine: BrainEngine, input: TextSource & { slugfor (const r of scoped) { after.push(raw.subarray(cur, viewAt(r.start)), Buffer.from(r.bytes)); cur = viewAt(r.end); }after.push(raw.subarray(cur, b));raw = raw.subarray(a, b);- const view = (bytes: Buffer) => { const text = bytes.toString('utf8'); return input.remote ? sanitizeRemoteBody(text, SEGMENTED_REMOTE_VIEW) : text; };+ const view = (bytes: Buffer) => { const text = bytes.toString('utf8'); return input.remote ? sanitizeRemoteBody(text, { ...SEGMENTED_REMOTE_VIEW, omitRows: input.omitRows }) : text; };const lineOffset = await viewNewlinesBefore(engine, input, rows, region !== regions[0], storedAt(a));return { ...editDiff(input.slug, view(raw), view(Buffer.concat(after)), lineOffset), ...(scoped.length < reps.length ? { diff_truncated: true as const } : {}),...(input.remote ? {} : { diff_scope: { byte_start: storedAt(a), byte_end: storedAt(b) } }) };@@ -715,8 +717,12 @@ export async function prepareSegmentedEditMutation(engine: BrainEngine, row: Wriconst within = parseEditWithin(row.intent?.within);const [{ prepareSegmentedSpliceMutation }, { snapshotLedger }] = await Promise.all([import('./page-append.ts'), import('./page-prepare.ts')]);const head = serializePageToMarkdown({ ...withoutBody(stored.page), compiled_truth: '', timeline: '' } as Page, stored.tags).slice(0, -1);+ const readsHeld = await storedWriterReadsHeld(engine, row.authority);return prepareSegmentedSpliceMutation(engine, row, stored, { replacements: async page => {- const located = await locateSegmentedEdits(engine, { sourceId: row.source_id, slug: row.slug, page, edits, within, remote: row.authority.remote, head, ledger: snapshotLedger(stored) });+ const ledger = snapshotLedger(stored);+ const omitRows = readsHeld ? undefined+ : (await projectedLayout(engine, await openSegmentedBody(engine, { pageId: page.id, generation: page.generation, ledger, hideHeld: true }))).overlay?.held;+ const located = await locateSegmentedEdits(engine, { sourceId: row.source_id, slug: row.slug, page, edits, within, remote: row.authority.remote, head, ledger, omitRows });return located && { replacements: located.replacements, outcome: located.diff };} });}diff --git a/src/core/persistence/page-mutations.ts b/src/core/persistence/page-mutations.tsindex 56e04c9047..593d320b1c 100644--- a/src/core/persistence/page-mutations.ts+++ b/src/core/persistence/page-mutations.ts@@ -387,7 +387,8 @@ export async function preparePageAdmission(ctx: OperationContext,}// G6: the hidden-row merge of an untrusted capture's body (page-prepare.ts) is opted into only by the trusted local ingest_capture// handler, whose caller never saw the page's non-world rows; a remote put_page has it already, through its authority.- if (Object.hasOwn(input.params, 'keep_hidden_rows') && (input.operation !== 'put_page' || ctx.remote !== false || input.params.keep_hidden_rows !== true)) {+ // GST-2891: and by an enforced member's put_page or edit_page (cli/enforced-member.ts), whose reads left out held rows.+ if (Object.hasOwn(input.params, 'keep_hidden_rows') && (!['put_page', 'edit_page'].includes(input.operation) || ctx.remote !== false || input.params.keep_hidden_rows !== true)) {throw new OperationError('invalid_params', 'keep_hidden_rows is reserved for trusted local ingestion.','Drop keep_hidden_rows and submit the page without it; a remote put_page already keeps the rows its caller could not read.');}diff --git a/src/core/persistence/page-prepare.ts b/src/core/persistence/page-prepare.tsindex b2f8810e96..03fb18e87c 100644--- a/src/core/persistence/page-prepare.ts+++ b/src/core/persistence/page-prepare.ts@@ -38,7 +38,8 @@ import { recordedPathFromFileUri, scannerSlugRootMode, scannerSourcePath } fromimport { engineMutationPrecondition, parseMutationPrecondition } from './preconditions.ts';import { assertPurgeParams } from './purge-params.ts';import { authorizeWrite, slugDisclosable } from './authority.ts';-import { remoteLabelsChecked, defaultSourceMemberWriter, defaultSourceWriterIsOwner, defaultSourceWriterReaders, ownerRewriteOf } from './page-visibility.ts';+import { remoteLabelsChecked, defaultSourceMemberWriter, defaultSourceWriterIsOwner, defaultSourceWriterReaders, ownerRewriteOf, storedWriterReadsHeld } from './page-visibility.ts';+import { loadFenceChunkOverlay } from '../eligibility/fence-overlay.ts';import { findLiveFileClaimants, type FileClaimant, type FileClaimTarget } from './file-claims.ts';import { digest, sha256 } from './digest.ts';import { replaceableFile } from '../facts/file-body-guard.ts';@@ -55,7 +56,7 @@ import { isStagingBodyRef, prepareStagedUpload } from './page-upload.ts';import { pipelined } from '../page-state/transactions.ts';import { overlayCanonicalBodies } from '../page-state/snapshot.ts';import { segmentedRevertRefusal } from '../page-state/versions.ts';-import { bodyUnread, isSegmentedPage, pageBodySegmentedError, withoutBody } from '../page-body/page-shape.ts';+import { bodyUnread, fenceBody, isSegmentedPage, pageBodySegmentedError, withoutBody } from '../page-body/page-shape.ts';import { fileBodyDigests, overlaidFileBodyDigests, storedBodyDigests, type ByteSource, type FileReading, type RegionDigests } from '../page-body/body-digests.ts';import { ledgerApplies, type BodyLedger } from '../page-body/segment-ledger.ts';import { inferredHeadPrefix } from '../page-body/streamed-import.ts';@@ -1073,7 +1074,7 @@ async function prepareInlineOrSegmentedMutation(unbounded: BrainEngine, row: Wriconst fenceRepair = row.operation === 'put_page' && !row.authority.remote ? parseFenceRepairReceipt(p.fence_repair) : null;const fenceRepairOutcome = fenceRepair ? { fence_repair: fenceRepair } : {};// #5616: edits apply to the caller's view of the locked snapshot (revision checked above).- const edited = row.operation === 'edit_page' ? editLockedPage(row, snapshot) : undefined;+ const edited = row.operation === 'edit_page' ? await editLockedPage(engine, row, snapshot) : undefined;let content = edited?.content ?? preparedIntent?.content ?? p.content as string;const privateRequest = privateRequestGuarded(row.operation, p.kind) && typeof content === 'string' ? readPrivateRequest(content, () => parseMarkdown(content as string, row.slug, { activePack }).frontmatter) : null;if (privateRequest) assertPrivateRequestAllowed(row, content as string, privateRequest.frontmatter);@@ -1158,7 +1159,7 @@ async function prepareInlineOrSegmentedMutation(unbounded: BrainEngine, row: Wriconst lean = options.coordinated === true && !targetDeleted && row.operation !== 'restore_page' && !versionTags;enterClaimStep(clock, 'import_content', undefined, 'db');const result = await importFromContent(engine, row.slug, content, {- ...source, noEmbed: true, remote: row.authority.remote || p.keep_hidden_rows === true, activePack, fences: projected ? 'coordinated' : 'lenient', coordinated: lean, boundedChunkWrites: options.boundedChunkWrites,+ ...source, noEmbed: true, remote: row.authority.remote || p.keep_hidden_rows === true, restoreHeldRows: p.keep_hidden_rows === true || !await storedWriterReadsHeld(engine, row.authority), activePack, fences: projected ? 'coordinated' : 'lenient', coordinated: lean, boundedChunkWrites: options.boundedChunkWrites,segmentedOwner: { kind: row.operation === 'edit_page' ? 'edit' : 'put', principalKind: row.principal_kind, principalId: row.principal_id, baseRevision: observedRevision },...(lean ? { existingSnapshot: snapshot } : {}),// #6259: only owner-tier intents keep gate-owned markers; an ordinary put_page, local or remote, has them stripped.@@ -1338,11 +1339,15 @@ function projectedRows(projected: { timelineRowsRemoved?: TimelineRowsRemoved |return { removedTimeline: projected?.timelineRowsRemoved ?? null, contested: projected?.contested ?? [] };}-function editLockedPage(row: WriteRequest, snapshot: PageSnapshot | null) {+async function editLockedPage(engine: BrainEngine, row: WriteRequest, snapshot: PageSnapshot | null) {if (!snapshot || snapshot.page.deleted_at) throw pageRefusal('page_not_found', 'Page not found.', row,`Page ${row.slug} does not exist in source ${row.source_id} or is deleted, so the edit changed nothing. If it is soft-deleted, restore it with restore_page; otherwise write it whole with put_page.`,pageFix(row.source_id, row.slug));- return applyPageEdits(snapshot.page, snapshot.tags, row.authority.remote, parsePageEdits(row.intent?.edits), parseEditWithin(row.intent?.within));+ // GST-2891: an enforced member's edit (keep_hidden_rows) read the page as a remote caller does, so its diff shows that view.+ const remoteView = row.authority.remote || row.intent?.keep_hidden_rows === true;+ const omitRows = row.intent?.keep_hidden_rows !== true && await storedWriterReadsHeld(engine, row.authority) ? undefined+ : (await loadFenceChunkOverlay(engine, { sourceId: row.source_id, slug: row.slug, ...fenceBody(snapshot.page) }, { hideHeld: true }))?.held;+ return applyPageEdits(snapshot.page, snapshot.tags, remoteView, parsePageEdits(row.intent?.edits), parseEditWithin(row.intent?.within), omitRows);}/**diff --git a/src/core/persistence/page-visibility.ts b/src/core/persistence/page-visibility.tsindex ca26f531c7..ea4c5fae55 100644--- a/src/core/persistence/page-visibility.ts+++ b/src/core/persistence/page-visibility.ts@@ -311,6 +311,19 @@ export async function storedOwnerSpeaksForItself(engine: SqlEngine, authority: Wreturn (member === undefined || bound === member) && defaultSourceWriterIsOwner(engine, bound);}+/**+ * GST-2891: whether a stored write's caller reads held fence rows, as worldOnlyFacts decided its reads: the local CLI,+ * or a credential bound to the live owner member, for itself or for an audience of the owner alone. Any other caller+ * read the page without them.+ */+export async function storedWriterReadsHeld(engine: SqlEngine, authority: WriteAuthority): Promise<boolean> {+ if (!authority.remote) return true;+ const audience = authority.actingFor;+ if (audience !== undefined && !(Array.isArray(audience) && audience.length === 1)) return false;+ const bound = (await memberIdentityFor(engine, authority)).find(p => p.startsWith('member:')) ?? null;+ return (audience === undefined || audience[0] === bound) && await defaultSourceWriterIsOwner(engine, bound);+}+/*** GST-2091: whether a stored page write is the brain owner's own (WriteAuthority.ownerWrite), judged live: it still holds* write scope and storedOwnerSpeaksForItself still names the same owner member. Such a write reaches a private page thediff --git a/src/core/remote-body.ts b/src/core/remote-body.tsindex e8f935ee36..05c8acb086 100644--- a/src/core/remote-body.ts+++ b/src/core/remote-body.ts@@ -15,7 +15,7 @@ export const PROTECTED_PAIRS: readonly FencePair[] = [* #5567 materialized-timeline marker lines are dropped unless the caller* round-trips the body (`keepMaterializedMarkers`).*/-export function sanitizeRemoteBody(body: string, opts: { includeWithdrawn?: boolean; keepMaterializedMarkers?: boolean } = {}): string {+export function sanitizeRemoteBody(body: string, opts: { includeWithdrawn?: boolean; keepMaterializedMarkers?: boolean; omitRows?: ReadonlySet<number> } = {}): string {if (typeof body !== 'string') return '';// Parse the same free-text bytes storage accepts. Removing NUL after fence// detection could turn an unrecognized marker into a protected stored fence.@@ -28,17 +28,17 @@ export function sanitizeRemoteBody(body: string, opts: { includeWithdrawn?: boolfor (const region of regions) {output.push(body.slice(cursor, region.start));cursor = region.end;- if (region.pair === 0) output.push(renderFactsRegion(body.slice(region.start, region.end), opts.includeWithdrawn));+ if (region.pair === 0) output.push(renderFactsRegion(body.slice(region.start, region.end), opts.includeWithdrawn, opts.omitRows));}output.push(truncatedAt === -1 ? body.slice(cursor) : body.slice(cursor, truncatedAt));return output.join('');}-/** What sanitizeRemoteBody puts in place of one protected facts region: its world rows rendered, or nothing. */-export function renderFactsRegion(region: string, includeWithdrawn = false): string {+/** What sanitizeRemoteBody puts in place of one protected facts region: its world rows rendered (less `omitRows`), or nothing. */+export function renderFactsRegion(region: string, includeWithdrawn = false, omitRows?: ReadonlySet<number>): string {try {const parsed = parseFactsFenceFor(region, { reader: true });- if (parsed.warnings.length === 0) return renderFactsTable(parsed.facts.filter(row => row.visibility === 'world' && (includeWithdrawn || !row.forgotten)));+ if (parsed.warnings.length === 0) return renderFactsTable(parsed.facts.filter(row => row.visibility === 'world' && (includeWithdrawn || !row.forgotten) && !omitRows?.has(row.rowNum)));} catch {// A protected block that cannot be parsed is omitted, never echoed.}diff --git a/src/core/search/evidence-delivery.ts b/src/core/search/evidence-delivery.tsindex 5ba448822c..841bbbbc72 100644--- a/src/core/search/evidence-delivery.ts+++ b/src/core/search/evidence-delivery.ts@@ -32,7 +32,7 @@ import { deliveredQuoteText, deliveredQuoteTitle, fenceTranscriptRows, neutralizimport { stampPageTrust } from '../eligibility/stamp.ts';import { pageReadFilter } from './read-policy-sql.ts';import { chunkGenerationFilter, chunkManifestReads, safeChunksFilter, requiresSafeChunks, servedTextProjectionFilter } from './safe-chunks.ts';-import { resolveExcludePrivatePages } from './private-visibility.ts';+import { resolveExcludePrivatePages, worldOnlyFacts } from './private-visibility.ts';import { safeSplitIndex } from '../text-safe.ts';import { loadFenceChunkOverlay, type FenceChunkOverlay } from '../eligibility/fence-overlay.ts';import { DEFAULT_INLINE_BODY_BYTES, locateChunksByStoredSpans, projectPageText, TIMELINE_SEPARATOR } from '../page-window/document.ts';@@ -1721,7 +1721,7 @@ export async function assembleEvidenceForHits(engine: BrainEngine, input: Assemb});const { rows, unresolved } = await resolveFrozenHits(engine, input.hits, scope);// GST-687: a remote reader's rows carry their page trust before the fence and the budget read it (quoteLabel's tier arm).- const trusted = <R extends SearchResult>(results: R[]) => (remote ? stampPageTrust(engine, results, input.caller?.minTrust) : Promise.resolve(results));+ const trusted = <R extends SearchResult>(results: R[]) => (remote ? stampPageTrust(engine, results, input.caller?.minTrust, undefined, worldOnlyFacts(remote)) : Promise.resolve(results));if (!plan) {const out = redactRetrievalOutput(neutralizeTranscriptRows(await trusted(rows)), {});return { results: fenceTranscriptRows(out.results, undefined, remote), delivery: null, unresolved };diff --git a/src/core/search/exact-scan.ts b/src/core/search/exact-scan.tsindex db2dc0891d..25afce4bed 100644--- a/src/core/search/exact-scan.ts+++ b/src/core/search/exact-scan.ts@@ -46,7 +46,7 @@ export const EXACT_DEFAULT_SECONDS = 30;export const EXACT_MAX_SECONDS = 120;export const EXACT_MAX_PHRASE_CHARS = 500;export const EXACT_SNIPPETS_PER_PAGE = 3;-const SNIPPET_CONTEXT = 80;+export const EXACT_SNIPPET_CONTEXT = 80;const BATCH_PAGES = 2000;const GAP_SAMPLE = 20;@@ -119,7 +119,7 @@ export function buildExactBatchStatement(opts: ExactScanOpts, after: number, batconst limitP = add(batchPages);const flags = opts.caseSensitive ? "''" : "'i'";const re = add(pattern.regex);- const snippetRe = add('.{0,' + SNIPPET_CONTEXT + '}' + pattern.regex + '.{0,' + SNIPPET_CONTEXT + '}');+ const snippetRe = add('.{0,' + EXACT_SNIPPET_CONTEXT + '}' + pattern.regex + '.{0,' + EXACT_SNIPPET_CONTEXT + '}');const like = add('%' + escapeLikePattern(pattern.longest) + '%');const op = opts.caseSensitive ? '~' : '~*';const likeOp = opts.caseSensitive ? 'LIKE' : 'ILIKE';diff --git a/src/core/trust/fence-guard.ts b/src/core/trust/fence-guard.tsindex d0843cfac0..d62bd3c48a 100644--- a/src/core/trust/fence-guard.ts+++ b/src/core/trust/fence-guard.ts@@ -61,10 +61,10 @@ export async function guardFenceRows<T extends FenceFact>(tx: BrainEngine, inputexpire: () => Promise<unknown>): Promise<FenceGuard<T>> {const { sourceId, slug, incoming } = input;// GST-2511: a contested row a widening proposal stored narrower (pending, rejected or undone) is held at its visibility- // against a writer that could not widen the row it contests, while the fence still asks for world at its position.+ // while the fence still asks for world at its position. GST-2891: whatever the writer's tier: re-deriving the unchanged+ // row (a sync after an unrelated edit, a re-put) is not the owner's accept, which alone applies the widening.const held = `f.visibility <> 'world' AND EXISTS (SELECT 1 FROM trust_proposals tp WHERE tp.action = 'supersede_fact' AND tp.related_table = 'facts'- AND tp.related_id = f.id AND tp.status IN ('pending', 'rejected', 'undone') AND tp.before_state->'visibility' IS NOT NULL- AND ${trustRankSql("tp.before_state->'old'->>'tier'")} > ${WRITER_RANK})+ AND tp.related_id = f.id AND tp.status IN ('pending', 'rejected', 'undone') AND tp.before_state->'visibility' IS NOT NULL)AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS w(row_num integer, fact text, visibility text)WHERE w.row_num = f.row_num AND w.fact = f.fact AND w.visibility = 'world')`;// #8194: a row `gbrain repair open-facts` opens (withOpenFacts: only private to world, same row number and claim) is guarded atdiff --git a/src/core/trust/supersede-handlers.ts b/src/core/trust/supersede-handlers.tsindex ce3a267cbc..308678bb79 100644--- a/src/core/trust/supersede-handlers.ts+++ b/src/core/trust/supersede-handlers.ts@@ -74,7 +74,8 @@ export async function lexicalContestCandidate(engine: BrainEngine, sourceId: str/*** GST-2875: the active non-world fact of `sourceId` on entity `entity` stating exactly `claim`, more trusted than `writer`,- * while no active world fact states it. A world write of that claim restates it: the write is stored at the copy's visibility+ * while no active world fact on that entity states it (GST-2891: a world copy the writer saved on another entity does not+ * lift the hold). A world write of that claim restates it: the write is stored at the copy's visibility* and its proposal carries the widening, which only the owner's accept applies (the fence rule of GST-2511, for every* remember). Same entity only, as GST-2511 is same page only: the supersede_fact accept refuses a pair across entities.*/@@ -84,6 +85,7 @@ export async function restatedPrivateCopy(engine: BrainEngine, sourceId: string,`SELECT f.id, f.visibility, f.trust_tier FROM facts f WHERE f.source_id = $1 AND f.fact = $2 AND f.visibility <> 'world'AND f.entity_slug IS NOT DISTINCT FROM $4::text AND f.expired_at IS NULL AND (f.valid_until IS NULL OR f.valid_until > now()) AND ${trustRankSql('f.trust_tier')} > ${trustRankSql('$3::text')}AND NOT EXISTS (SELECT 1 FROM facts w WHERE w.source_id = $1 AND w.fact = $2 AND w.visibility = 'world' AND w.expired_at IS NULL+ AND w.entity_slug IS NOT DISTINCT FROM $4::textAND (w.valid_until IS NULL OR w.valid_until > now()))ORDER BY f.id LIMIT 1`, [sourceId, claim, writer, entity]);return row ? { id: Number(row.id), visibility: row.visibility, trust_tier: storedTrustTier(row.trust_tier) } : null;diff --git a/src/core/verbs/entity-card-identity.ts b/src/core/verbs/entity-card-identity.tsindex 1ad541ea80..c999e5aec0 100644--- a/src/core/verbs/entity-card-identity.ts+++ b/src/core/verbs/entity-card-identity.ts@@ -23,6 +23,7 @@ import type { BrainEngine } from '../engine.ts';import { pageSnapshotKey } from '../page-snapshot-batch.ts';import type { PageSnapshot } from '../page-state/types.ts';import { sanitizeRemoteBody } from '../remote-body.ts';+import { loadFenceChunkOverlay } from '../eligibility/fence-overlay.ts';import { ALIAS_ORIGIN_RANK, declarationsIn, titleName, type AliasOrigin } from '../mentions/aliases.ts';import { readIdentitySiblings, SIBLING_GROUP_CAP } from '../mentions/siblings.ts';import type { MentionPolicy, PackTypes } from '../mentions/policy.ts';@@ -165,7 +166,8 @@ async function excerptSource(engine: BrainEngine, snapshot: PageSnapshot): Promi/** The card's identity fields for `row`. Fail-soft: a failed read leaves the field out. */export async function readCardIdentity(engine: BrainEngine, sourceId: string,row: { slug: string; title: string | null; compiled_truth: string | null },- opts: { excludePrivate: boolean; excludeQuarantined?: boolean; surfaceCeiling?: 'verbs' | 'starter' | 'full'; aka: string[] }): Promise<CardIdentity> {+ opts: { excludePrivate: boolean; excludeQuarantined?: boolean; surfaceCeiling?: 'verbs' | 'starter' | 'full'; aka: string[];+ /** GST-2891: the caller's fact reads are world-only, so the fence rows held from it leave the excerpt. */ hideHeld?: boolean }): Promise<CardIdentity> {const { aliasRows, identity_siblings } = await readSiblingIdentity(engine, sourceId, row, { excludePrivate: opts.excludePrivate, excludeQuarantined: opts.excludeQuarantined });const siblingPages = identity_siblings?.pages ?? [];const slugs = [row.slug, ...siblingPages.map(p => p.slug)];@@ -189,8 +191,10 @@ export async function readCardIdentity(engine: BrainEngine, sourceId: string,for (const slug of slugs) {const page = bodies.get(slug);if (!page) continue;- const lines = excerptLines(sanitizeRemoteBody(page.body), page.title, Math.min(EXCERPT_PAGE_CHARS, EXCERPT_CARD_CHARS - used));- const full = excerptLines(sanitizeRemoteBody(page.body), page.title, EXCERPT_PAGE_CHARS);+ const omitRows = opts.hideHeld ? (await loadFenceChunkOverlay(engine, { sourceId, slug, compiled_truth: page.body }, { hideHeld: true }))?.held : undefined;+ const safe = sanitizeRemoteBody(page.body, { omitRows });+ const lines = excerptLines(safe, page.title, Math.min(EXCERPT_PAGE_CHARS, EXCERPT_CARD_CHARS - used));+ const full = excerptLines(safe, page.title, EXCERPT_PAGE_CHARS);if (lines.length < full.length) omitted.push(slug);for (const line of lines) { excerpt.push({ slug, line }); used += line.length; }}diff --git a/src/core/verbs/entity-card.ts b/src/core/verbs/entity-card.tsindex bffab4e5c5..034c5a914e 100644--- a/src/core/verbs/entity-card.ts+++ b/src/core/verbs/entity-card.ts@@ -504,7 +504,7 @@ export async function buildEntityCard(if (match) card.match = match;if (opts.includeReferences) {Object.assign(card, await cardReferences(engine, hitSource, row, opts, scope.excludePrivate, scope.entityTypes),- await readCardIdentity(engine, hitSource, row, { excludePrivate: scope.excludePrivate, excludeQuarantined: opts.omitQuarantined === true, surfaceCeiling: opts.surfaceCeiling, aka: card.aka }));+ await readCardIdentity(engine, hitSource, row, { excludePrivate: scope.excludePrivate, excludeQuarantined: opts.omitQuarantined === true, surfaceCeiling: opts.surfaceCeiling, aka: card.aka, hideHeld: (await import('../search/private-visibility.ts')).worldOnlyFacts(opts.remote) }));await attachCardRecords(engine, hitSource, card, opts, scope.excludePrivate);if (await nameVariantsEnabled(engine)) {const { privatePagesFilterForCaller } = await import('../search/private-visibility.ts');diff --git a/test/fixtures/fold-hook-noop-6712.json b/test/fixtures/fold-hook-noop-6712.jsonindex 3b76f86508..72a3b20932 100644--- a/test/fixtures/fold-hook-noop-6712.json+++ b/test/fixtures/fold-hook-noop-6712.json@@ -34,7 +34,7 @@"UPDATE pages SET database_only_reason=NULL WHERE source_id=$1 AND slug=$2 AND source_path=$3 AND database_only_reason='origin_twin'","UPDATE pages SET source_kind=$3,ingested_via=$4,ingested_at=$5::timestamptz WHERE source_id=$1 AND slug=$2","SELECT jsonb_build_object('id', f.id, 'row_num', f.row_num, 'fact', f.fact, 'visibility', f.visibility, 'kind', f.kind, 'entity_slug', f.entity_slug, 'source', f.source, 'expired_at', f.expired_at, 'valid_from', f.valid_from, 'valid_until', f.valid_until, 'superseded_by', f.superseded_by, 'consolidated_at', f.consolidated_at, 'consolidated_into', f.consolidated_into, 'attributed_to', f.attributed_to) AS preimage FROM facts f WHERE f.source_id=$1 AND f.source_markdown_slug=$2 AND f.row_num IS NOT NULL AND (COALESCE(f.source,'') NOT LIKE 'cli:extract-conversation-facts%' OR EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS c(row_num integer) WHERE c.row_num=f.row_num)) AND NOT EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS n(row_num integer,fact text,visibility text) WHERE n.row_num=f.row_num AND n.fact=f.fact AND n.visibility=f.visibility) AND NOT (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world')) ORDER BY f.row_num, f.id",- "SELECT f.id, f.row_num, f.fact, f.visibility, f.trust_tier, (CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) AS above, (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world')) AS opened FROM facts f WHERE f.source_id = $1 AND f.source_markdown_slug = $2 AND f.row_num IS NOT NULL AND f.expired_at IS NULL AND (COALESCE(f.source, '') NOT LIKE 'cli:extract-conversation-facts%' OR EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS c(row_num integer) WHERE c.row_num = f.row_num)) AND NOT EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS n(row_num integer, fact text, visibility text) WHERE n.row_num = f.row_num AND n.fact = f.fact AND n.visibility = f.visibility) AND ((CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) OR (f.visibility <> 'world' AND EXISTS (SELECT 1 FROM trust_proposals tp WHERE tp.action = 'supersede_fact' AND tp.related_table = 'facts' AND tp.related_id = f.id AND tp.status IN ('pending', 'rejected', 'undone') AND tp.before_state->'visibility' IS NOT NULL AND (CASE tp.before_state->'old'->>'tier' WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END)) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS w(row_num integer, fact text, visibility text) WHERE w.row_num = f.row_num AND w.fact = f.fact AND w.visibility = 'world')) OR (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world'))) ORDER BY f.row_num FOR UPDATE",+ "SELECT f.id, f.row_num, f.fact, f.visibility, f.trust_tier, (CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) AS above, (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world')) AS opened FROM facts f WHERE f.source_id = $1 AND f.source_markdown_slug = $2 AND f.row_num IS NOT NULL AND f.expired_at IS NULL AND (COALESCE(f.source, '') NOT LIKE 'cli:extract-conversation-facts%' OR EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS c(row_num integer) WHERE c.row_num = f.row_num)) AND NOT EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS n(row_num integer, fact text, visibility text) WHERE n.row_num = f.row_num AND n.fact = f.fact AND n.visibility = f.visibility) AND ((CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) OR (f.visibility <> 'world' AND EXISTS (SELECT 1 FROM trust_proposals tp WHERE tp.action = 'supersede_fact' AND tp.related_table = 'facts' AND tp.related_id = f.id AND tp.status IN ('pending', 'rejected', 'undone') AND tp.before_state->'visibility' IS NOT NULL) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS w(row_num integer, fact text, visibility text) WHERE w.row_num = f.row_num AND w.fact = f.fact AND w.visibility = 'world')) OR (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world'))) ORDER BY f.row_num FOR UPDATE","DELETE FROM timeline_entries t USING jsonb_to_recordset($2::text::jsonb) AS d(id integer,date date,source text,summary text,detail text,removed boolean) WHERE t.page_id=$1 AND t.event_page_id IS NULL AND t.id=d.id AND t.date=d.date AND t.source=d.source AND t.summary=d.summary AND t.detail=d.detail RETURNING t.date::text AS date, d.removed","UPDATE timeline_entries t SET detail=r.next FROM jsonb_to_recordset($2::text::jsonb) AS r(id integer,detail text,next text) WHERE t.page_id=$1 AND t.event_page_id IS NULL AND t.id=r.id AND t.detail=r.detail","SELECT k.row_num FROM takes k JOIN jsonb_to_recordset($2::text::jsonb) AS n(row_num integer,claim text,kind text,holder text) ON n.row_num=k.row_num WHERE k.page_id=$1 AND (k.claim,k.kind,k.holder) IS DISTINCT FROM (n.claim,n.kind,n.holder) ORDER BY k.row_num LIMIT $3",@@ -100,7 +100,7 @@"UPDATE pages SET database_only_reason=NULL WHERE source_id=$1 AND slug=$2 AND database_only_reason='unbound_source'","WITH chosen AS ( SELECT p.* FROM pages p WHERE p.slug=$1 AND p.source_id=$2 ORDER BY (p.slug=$1) DESC, (p.source_id=$3) DESC, p.source_id ASC LIMIT 1 ) SELECT p.*, (SELECT s.incarnation FROM sources s WHERE s.id=p.source_id) AS source_incarnation, COALESCE((SELECT jsonb_agg(t.tag ORDER BY t.tag) FROM tags t WHERE t.page_id=p.id), '[]'::jsonb) AS snapshot_tags, wd.snapshot_withdrawals, wd.snapshot_global_purges, CASE WHEN wd.snapshot_withdrawals <> '[]'::jsonb OR (wd.snapshot_global_purges->>'count')::int > 0 THEN (SELECT string_agg(regexp_replace(lower(line), '[[:space:]]+', ' ', 'g'), chr(10) ORDER BY ord) FROM unnest(string_to_array(p.compiled_truth,chr(10))) WITH ORDINALITY AS lines(line,ord)) END AS fingerprint_body, CASE WHEN wd.snapshot_withdrawals <> '[]'::jsonb OR (wd.snapshot_global_purges->>'count')::int > 0 THEN (SELECT string_agg(regexp_replace(lower(line), '[[:space:]]+', ' ', 'g'), chr(10) ORDER BY ord) FROM unnest(string_to_array(p.timeline,chr(10))) WITH ORDINALITY AS lines(line,ord)) END AS fingerprint_timeline FROM chosen p CROSS JOIN LATERAL (SELECT COALESCE((SELECT jsonb_agg(jsonb_build_object('visibility',w.visibility,'fact_hash',w.fact_hash,'withdrawn_at',w.withdrawn_at) ORDER BY w.visibility,w.fact_hash) FROM (SELECT visibility,fact_hash,min(withdrawn_at) AS withdrawn_at FROM fact_withdrawals WHERE source_id=p.source_id AND (subject='*' OR subject=p.slug) GROUP BY visibility,fact_hash) w), '[]'::jsonb) || COALESCE((SELECT jsonb_agg(jsonb_build_object('visibility',x.visibility,'fact_hash',x.fact_hash,'withdrawn_at',x.purged_at,'purged',true) ORDER BY x.visibility,x.fact_hash) FROM (SELECT visibility,fact_hash,min(purged_at) AS purged_at FROM fact_purges WHERE source_id=p.source_id AND subject=p.slug GROUP BY visibility,fact_hash) x), '[]'::jsonb) AS snapshot_withdrawals, (SELECT jsonb_build_object('count',count(*),'latest',max(g.purged_at)) FROM fact_purges g WHERE g.source_id=p.source_id AND g.subject='*' ) AS snapshot_global_purges) wd","SELECT jsonb_build_object('id', f.id, 'row_num', f.row_num, 'fact', f.fact, 'visibility', f.visibility, 'kind', f.kind, 'entity_slug', f.entity_slug, 'source', f.source, 'expired_at', f.expired_at, 'valid_from', f.valid_from, 'valid_until', f.valid_until, 'superseded_by', f.superseded_by, 'consolidated_at', f.consolidated_at, 'consolidated_into', f.consolidated_into, 'attributed_to', f.attributed_to) AS preimage FROM facts f WHERE f.source_id=$1 AND f.source_markdown_slug=$2 AND f.row_num IS NOT NULL AND (COALESCE(f.source,'') NOT LIKE 'cli:extract-conversation-facts%' OR EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS c(row_num integer) WHERE c.row_num=f.row_num)) AND NOT EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS n(row_num integer,fact text,visibility text) WHERE n.row_num=f.row_num AND n.fact=f.fact AND n.visibility=f.visibility) AND NOT (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world')) ORDER BY f.row_num, f.id",- "SELECT f.id, f.row_num, f.fact, f.visibility, f.trust_tier, (CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) AS above, (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world')) AS opened FROM facts f WHERE f.source_id = $1 AND f.source_markdown_slug = $2 AND f.row_num IS NOT NULL AND f.expired_at IS NULL AND (COALESCE(f.source, '') NOT LIKE 'cli:extract-conversation-facts%' OR EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS c(row_num integer) WHERE c.row_num = f.row_num)) AND NOT EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS n(row_num integer, fact text, visibility text) WHERE n.row_num = f.row_num AND n.fact = f.fact AND n.visibility = f.visibility) AND ((CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) OR (f.visibility <> 'world' AND EXISTS (SELECT 1 FROM trust_proposals tp WHERE tp.action = 'supersede_fact' AND tp.related_table = 'facts' AND tp.related_id = f.id AND tp.status IN ('pending', 'rejected', 'undone') AND tp.before_state->'visibility' IS NOT NULL AND (CASE tp.before_state->'old'->>'tier' WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END)) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS w(row_num integer, fact text, visibility text) WHERE w.row_num = f.row_num AND w.fact = f.fact AND w.visibility = 'world')) OR (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world'))) ORDER BY f.row_num FOR UPDATE",+ "SELECT f.id, f.row_num, f.fact, f.visibility, f.trust_tier, (CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) AS above, (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world')) AS opened FROM facts f WHERE f.source_id = $1 AND f.source_markdown_slug = $2 AND f.row_num IS NOT NULL AND f.expired_at IS NULL AND (COALESCE(f.source, '') NOT LIKE 'cli:extract-conversation-facts%' OR EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS c(row_num integer) WHERE c.row_num = f.row_num)) AND NOT EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS n(row_num integer, fact text, visibility text) WHERE n.row_num = f.row_num AND n.fact = f.fact AND n.visibility = f.visibility) AND ((CASE f.trust_tier WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) > (CASE COALESCE(NULLIF(current_setting('gbrain.write_trust_tier', true), ''), 'unknown') WHEN 'user_confirmed' THEN 6 WHEN 'operator_curated' THEN 5 WHEN 'tool_observed' THEN 4 WHEN 'agent_written' THEN 3 WHEN 'unknown' THEN 2 WHEN 'external_untrusted' THEN 1 ELSE 0 END) OR (f.visibility <> 'world' AND EXISTS (SELECT 1 FROM trust_proposals tp WHERE tp.action = 'supersede_fact' AND tp.related_table = 'facts' AND tp.related_id = f.id AND tp.status IN ('pending', 'rejected', 'undone') AND tp.before_state->'visibility' IS NOT NULL) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS w(row_num integer, fact text, visibility text) WHERE w.row_num = f.row_num AND w.fact = f.fact AND w.visibility = 'world')) OR (f.visibility = 'private' AND f.id::text = ANY(string_to_array(COALESCE(current_setting('gbrain.write_open_facts', true), ''), ',')) AND EXISTS (SELECT 1 FROM jsonb_to_recordset($3::text::jsonb) AS o(row_num integer, fact text, visibility text) WHERE o.row_num = f.row_num AND o.fact = f.fact AND o.visibility = 'world'))) ORDER BY f.row_num FOR UPDATE","DELETE FROM timeline_entries t USING jsonb_to_recordset($2::text::jsonb) AS d(id integer,date date,source text,summary text,detail text,removed boolean) WHERE t.page_id=$1 AND t.event_page_id IS NULL AND t.id=d.id AND t.date=d.date AND t.source=d.source AND t.summary=d.summary AND t.detail=d.detail RETURNING t.date::text AS date, d.removed","UPDATE timeline_entries t SET detail=r.next FROM jsonb_to_recordset($2::text::jsonb) AS r(id integer,detail text,next text) WHERE t.page_id=$1 AND t.event_page_id IS NULL AND t.id=r.id AND t.detail=r.detail","SELECT k.row_num FROM takes k JOIN jsonb_to_recordset($2::text::jsonb) AS n(row_num integer,claim text,kind text,holder text) ON n.row_num=k.row_num WHERE k.page_id=$1 AND (k.claim,k.kind,k.holder) IS DISTINCT FROM (n.claim,n.kind,n.holder) ORDER BY k.row_num LIMIT $3",diff --git a/test/fixtures/goldens/exports/types.json b/test/fixtures/goldens/exports/types.jsonindex e799389e19..39234e7603 100644--- a/test/fixtures/goldens/exports/types.json+++ b/test/fixtures/goldens/exports/types.json@@ -5114,7 +5114,7 @@"kind": "function","name": "importFromContent","signature": [- "(engine: BrainEngine, slug: string, content: string, opts?: { prepare?: (prepared: PreparedContentImport) => Promise<ImportResult>; history?: VersionHistory; coordinated?: boolean; boundedChunkWrites?: true; existingSnapshot?: PageSnapshot | null; prepareFrontmatter?: (page: ParsedPage) => void; noEmbed?: boolean; sourceId?: string; filename?: string; sourcePath?: string; sourceRoot?: string; fileTimes?: { birthtime?: Date; mtime?: Date; firstCommit?: Date; }; forceRechunk?: boolean; activePack?: ParseOpts[\"activePack\"]; source_kind?: string | null; source_uri?: string | null; ingested_via?: string | null; segmentedOwner?: SegmentedStagingOwner; reuseSegments?: SegmentedImport; remote?: boolean; preserveGateMarkers?: boolean; allowEmptyOverwrite?: boolean; overrideTier?: unknown; writeGate?: WriteGateInput; holdNoncanonicalPrivate?: boolean; beforeCommit?: (tx: BrainEngine, slug: string) => Promise<void>; expectedBaseRevision?: string | null; onPostCommitEmbedding?: (complete: () => Promise<ImportEmbeddingResult>) => void; } & ImportFenceOptions) => Promise<ImportResult>"+ "(engine: BrainEngine, slug: string, content: string, opts?: { prepare?: (prepared: PreparedContentImport) => Promise<ImportResult>; history?: VersionHistory; coordinated?: boolean; boundedChunkWrites?: true; existingSnapshot?: PageSnapshot | null; prepareFrontmatter?: (page: ParsedPage) => void; noEmbed?: boolean; sourceId?: string; filename?: string; sourcePath?: string; sourceRoot?: string; fileTimes?: { birthtime?: Date; mtime?: Date; firstCommit?: Date; }; forceRechunk?: boolean; activePack?: ParseOpts[\"activePack\"]; source_kind?: string | null; source_uri?: string | null; ingested_via?: string | null; segmentedOwner?: SegmentedStagingOwner; reuseSegments?: SegmentedImport; remote?: boolean; restoreHeldRows?: boolean; preserveGateMarkers?: boolean; allowEmptyOverwrite?: boolean; overrideTier?: unknown; writeGate?: WriteGateInput; holdNoncanonicalPrivate?: boolean; beforeCommit?: (tx: BrainEngine, slug: string) => Promise<void>; expectedBaseRevision?: string | null; onPostCommitEmbedding?: (complete: () => Promise<ImportEmbeddingResult>) => void; } & ImportFenceOptions) => Promise<ImportResult>"]},{diff --git a/test/held-row-page-reads-2891.test.ts b/test/held-row-page-reads-2891.test.tsnew file mode 100644index 0000000000..7632e9b8ea--- /dev/null+++ b/test/held-row-page-reads-2891.test.ts@@ -0,0 +1,294 @@+/**+ * GST-2891: a private fact restated as world is stored private until the owner accepts the widening (#9196 for a+ * put_page fence row, GST-2875 for remember), but the page body kept the world cell, so get_page, fetch, search and+ * the rest handed the restatement to callers who cannot read the private row. These tests pin, on PGLite and Postgres,+ * on a repo brain and a repo-less one: a caller whose fact reads are world-only never reads a held row's text, the+ * owner's callers and the local CLI still read it, a re-derive keeps the row private, a member still reads their own+ * member page, and accept / reject / undo keep working.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { execFileSync } from 'node:child_process';+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { configureGateway, resetGateway, __setEmbedTransportForTests } from '../src/core/ai/gateway.ts';+import { GBrainOAuthProvider } from '../src/core/oauth-provider.ts';+import { sqlQueryForEngine } from '../src/core/sql-query.ts';+import { mintLegacyToken } from '../src/core/token-mint.ts';+import { dispatchToolCall } from '../src/mcp/dispatch.ts';+import { __resetPrivateVisibilityCacheForTests } from '../src/core/search/private-visibility.ts';+import { setOwner } from '../src/core/access/owner.ts';+import { bindCredential } from '../src/core/access/members.ts';+import { memberPageFrontmatter, memberPageSlug } from '../src/core/access/member-facts.ts';+import type { AuthInfo } from '../src/core/ops/contract.ts';+import { renderFactsTable } from '../src/core/facts-fence.ts';+import { dropHeldFenceRows } from '../src/core/eligibility/stamp.ts';+import { withCoordinatedWrite, withTrustPromotion } from '../src/core/persistence/context.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { claimWorktree } from '../src/core/persistence/ownership.ts';+import { performManagedSync } from '../src/core/persistence/sync-run.ts';+import { registerLocalWriter } from '../src/core/persistence/identity.ts';+import { decideTrustProposal } from '../src/core/trust/decide.ts';+import { enableBeliefLedger } from '../src/core/belief/coverage.ts';+import { parseTrustProposalRef } from '../src/core/trust/proposals.ts';+import { testBackends } from './helpers/test-backends.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { readsUnscopedOnPurpose } from './helpers/reads-unscoped.ts';+import { withEnv } from './helpers/with-env.ts';+import { mulberry32 } from './helpers/seeded-random.ts';+import { importSegmented, pageContent, privacyText } from './helpers/segmented-privacy-fixture.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';++const DIMS = 1536;+function vectorFor(text: string): number[] {+ const v = new Array<number>(DIMS).fill(0);+ for (const w of text.toLowerCase().match(/[a-z]+/g) ?? []) { let h = 0; for (const ch of w) h = (h * 31 + ch.charCodeAt(0)) >>> 0; v[h % DIMS] += 1; }+ const n = Math.hypot(...v) || 1;+ return v.map(x => x / n);+}+const row = (n: number, claim: string, visibility = 'world') => ({ rowNum: n, claim, kind: 'fact', confidence: 1, visibility, notability: 'medium', active: true });+const pageText = (title: string, rows: ReturnType<typeof row>[], prose = 'Profile.') =>+ '---\ntype: person\ntitle: ' + title + '\nvalid_from: 2020-01-01\n---\n# ' + title + '\n\n' + prose + '\n\n## Facts\n\n' + renderFactsTable(rows as never) + '\n';+const OWNER = 'owner-example';+const BOB = 'bob-example';+/** The remember path (GST-2875), the put_page path (#9196), and on a repo brain a private copy nobody confirmed. */+const LIA = { slug: 'people/lia-example', title: 'Lia Example', claim: 'Lia keeps bees zebracornheld', canary: 'zebracornheld' };+const KIM = { slug: 'people/kim-example', title: 'Kim Example', claim: 'Kim keeps bees quokkaheld', canary: 'quokkaheld' };+const MAX = { slug: 'people/max-example', title: 'Max Example', claim: 'Max keeps bees okapiheld', canary: 'okapiheld' };+type Held = typeof LIA;+const turin = (p: Held) => p.title.split(' ')[0] + ' lives in Turin';+const seed = (p: Held) => pageText(p.title, [row(1, turin(p)), row(2, p.claim, 'private')]);++for (const kind of testBackends()) for (const shape of ['repo', 'repo-less'] as const) {+ describe('GST-2891 held rows on page reads (' + kind + ', ' + shape + ')', () => {+ let engine: BrainEngine; let close: () => Promise<void>;+ const home = mkdtempSync(join(tmpdir(), 'gst2891-'));+ const root = join(home, 'repo');+ const env = { GBRAIN_HOME: home, GBRAIN_SYNC_FAILURES_DIR: home, OPENAI_API_KEY: 'test' };+ const tokens: Record<string, string> = {};+ const refs: Record<string, number> = {};+ const git = (...args: string[]) => execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();+ const commit = () => { git('add', '-A'); git('-c', 'user.name=Example', '-c', '[email protected]', 'commit', '-qm', 'synthetic'); };+ const provider = () => new GBrainOAuthProvider({ sql: sqlQueryForEngine(engine), transaction: fn => engine.transaction(tx => fn(sqlQueryForEngine(tx))) });+ async function call(who: string, op: string, params: Record<string, unknown>): Promise<string> {+ __resetPrivateVisibilityCacheForTests();+ if (who === 'local') return (await withEnv(env, () => dispatchToolCall(engine, op, { ...params }, { remote: false, sourceId: 'default' }))).content.map(c => c.text).join('\n');+ const auth = await provider().verifyAccessToken(tokens[who]!) as unknown as AuthInfo;+ const o = await withEnv(env, () => dispatchToolCall(engine, op, { ...params }, { remote: true, transport: 'http', auth, sourceId: auth.sourceId ?? 'default', takesHoldersAllowList: ['world'] }));+ return o.content.map(c => c.text).join('\n');+ }+ const confirmAll = async (slug: string) => {+ for (const r of await engine.executeRaw<{ id: number }>('SELECT id FROM facts WHERE source_id=$1 AND source_markdown_slug=$2', ['default', slug]))+ await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => withTrustPromotion(tx, 'user_confirmed', () => tx.executeRaw("UPDATE facts SET trust_tier='user_confirmed' WHERE id=$1", [r.id])), TEST_WRITE_ATTRIBUTION));+ };+ const stored = async (claim: string) => (await engine.executeRaw<{ visibility: string }>(+ 'SELECT visibility FROM facts WHERE source_id=$1 AND fact=$2 AND expired_at IS NULL ORDER BY id', ['default', claim])).map(r => r.visibility);+ const pages: Held[] = shape === 'repo' ? [LIA, KIM, MAX] : [LIA, KIM];+ const reads = (p: Held): Array<[string, string, Record<string, unknown>]> => [+ ['get_page', 'get_page', { slug: p.slug }],+ ['get_page include_content', 'get_page', { slug: p.slug, include_content: true }],+ ['get_page lines', 'get_page', { slug: p.slug, lines: { from: 1, count: 200 } }],+ ['get_page grep', 'get_page', { slug: p.slug, grep: 'keeps bees' }],+ ['get_page section', 'get_page', { slug: p.slug, section: 'Facts' }],+ ['get_page outline', 'get_page', { slug: p.slug, body: 'outline' }],+ ['get_page as_of', 'get_page', { slug: p.slug, as_of: '2099-01-01' }],+ ['fetch', 'fetch', { id: p.slug }],+ ['search keyword', 'search', { query: p.canary, match: 'keyword', fields: 'full', limit: 20 }],+ ['search', 'search', { query: 'keeps bees ' + p.canary, fields: 'full', limit: 20 }],+ ['query', 'query', { query: 'keeps bees ' + p.canary, expand: false, fields: 'full', limit: 20 }],+ ['search_exact', 'search_exact', { phrase: p.canary }],+ ['get_chunks', 'get_chunks', { slug: p.slug }],+ ['entity', 'entity', { name: p.slug }],+ ];+ /** A read's text without the request it echoes back (search_exact's phrase). */+ const shown = (label: string, t: string) => (label === 'search_exact' ? t.replace(/"phrase":"[^"]*"/, '') : t);++ beforeAll(async () => withEnv(env, async () => {+ configureGateway({ embedding_model: 'openai:text-embedding-3-small', embedding_dimensions: DIMS, env: { OPENAI_API_KEY: 'test' } });+ __setEmbedTransportForTests((async ({ values }: { values: string[] }) => ({ embeddings: values.map(vectorFor) })) as never);+ if (kind === 'postgres') ({ engine, close } = await isolatedPersistencePostgres(process.env.DATABASE_URL!));+ else { engine = new PGLiteEngine(); await engine.connect({ engine: 'pglite' } as never); await engine.initSchema(); close = () => engine.disconnect(); }+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await registerLocalWriter(engine, 'cli');+ if (shape === 'repo') {+ mkdirSync(join(root, 'people'), { recursive: true });+ git('init', '-q');+ for (const p of pages) writeFileSync(join(root, p.slug + '.md'), seed(p));+ commit();+ await engine.executeRaw('UPDATE sources SET local_path=$1 WHERE id=$2', [root, 'default']);+ await claimWorktree(engine, 'default', root);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ expect((await performManagedSync(engine, { sourceId: 'default', noPull: true, noEmbed: true, noExtract: true })).status).toBe('first_sync');+ } else {+ for (const p of pages) await call('local', 'put_page', { slug: p.slug, content: seed(p), request_id: randomUUID() });+ }+ await confirmAll(LIA.slug); await confirmAll(KIM.slug);+ for (const id of [OWNER, BOB]) await engine.executeRaw('INSERT INTO access_members (id) VALUES ($1)', [id]);+ const mint = async (name: string, member?: string, scopes = ['read']) => {+ const m = await mintLegacyToken(engine, { name, scopes, takesHolders: ['world'] });+ tokens[name] = m.token;+ if (member) await bindCredential(engine, { kind: 'token', id: m.id }, { member }); else await readsUnscopedOnPurpose(engine, m.id);+ };+ await mint('owner', OWNER); await mint('bob', BOB); await mint('unbound'); await mint('writer', undefined, ['read', 'write']);+ await setOwner(engine, OWNER);+ for (const p of shape === 'repo' ? [LIA, MAX] : [LIA]) {+ const res = await call('writer', 'remember', { fact: p.claim, provenance: 'chat', entity: p.slug, request_id: randomUUID() });+ refs[p.slug] = parseTrustProposalRef(res.match(/tp\d+/)![0])!;+ }+ const cur = JSON.parse(await call('writer', 'get_page', { slug: KIM.slug, include_content: true }));+ await call('writer', 'put_page', { slug: KIM.slug, content: pageText(KIM.title, [row(1, turin(KIM)), row(2, KIM.claim)]), expected_revision: cur.revision, request_id: randomUUID() });+ refs[KIM.slug] = Number((await engine.executeRaw<{ id: number }>("SELECT id::int AS id FROM trust_proposals WHERE status='pending' ORDER BY id DESC LIMIT 1"))[0]!.id);+ }), 300_000);+ afterAll(async () => { __setEmbedTransportForTests(null); resetGateway(); await disposePersistenceConsumer(engine); await close?.(); rmSync(home, { recursive: true, force: true }); }, 60_000);++ test('the restatement is stored private with a pending widening while the page body carries a world cell', async () => {+ const cell = (p: Held) => new RegExp(p.canary + ' \\| fact \\| 1\\.0 \\| world');+ for (const p of pages) {+ expect(await stored(p.claim)).not.toContain('world');+ expect((await engine.executeRaw<{ status: string }>('SELECT status FROM trust_proposals WHERE id=$1', [refs[p.slug]]))[0]?.status).toBe('pending');+ expect(await call('local', 'get_page', { slug: p.slug, include_content: true })).toMatch(cell(p));+ }+ if (shape === 'repo') expect(readFileSync(join(root, KIM.slug + '.md'), 'utf8')).toMatch(cell(KIM));+ }, 120_000);++ test('a caller who cannot read the private row reads no held text; the owner and the local CLI still do', async () => {+ for (const p of pages) for (const [label, op, params] of reads(p)) for (const who of ['bob', 'unbound']) {+ const leaks = shown(label, await call(who, op, params)).includes(p.canary);+ expect({ who, label, slug: p.slug, leaks }).toEqual({ who, label, slug: p.slug, leaks: false });+ }+ for (const p of pages) for (const who of ['bob', 'unbound']) {+ expect(await call(who, 'get_page', { slug: p.slug, include_content: true })).toContain(turin(p));+ expect(await call(who, 'search', { query: turin(p), match: 'keyword', fields: 'full', limit: 20 })).toContain(turin(p) + ' | fact');+ }+ for (const p of pages) for (const who of ['local', 'owner']) for (const label of ['get_page include_content', 'fetch', 'search keyword', 'search_exact', 'get_chunks']) {+ const [, op, params] = reads(p).find(r => r[0] === label)!;+ const shows = shown(label, await call(who, op, params)).includes(p.canary);+ expect({ who, label, slug: p.slug, shows }).toEqual({ who, label, slug: p.slug, shows: true });+ }+ }, 300_000);++ test('a window around any chunk of the page holds no held text for such a caller', async () => {+ for (const p of pages) {+ const chunks = await engine.executeRaw<{ id: number }>('SELECT c.id::int AS id FROM content_chunks c JOIN pages pg ON pg.id=c.page_id WHERE pg.slug=$1 ORDER BY c.id', [p.slug]);+ expect(chunks.length).toBeGreaterThan(0);+ for (const c of chunks) for (const who of ['bob', 'unbound']) expect(await call(who, 'get_page', { slug: p.slug, around_chunk_id: c.id })).not.toContain(p.canary);+ }+ }, 120_000);++ test('a chunk boundary that cuts a held row leaves none of its text behind', async () => {+ const kept = await dropHeldFenceRows(engine, [+ { slug: LIA.slug, source_id: 'default', chunk_text: 'bees zebracornheld | fact | 1.0 | world | medium | | | | |\nProfile prose.' },+ { slug: LIA.slug, source_id: 'default', chunk_text: 'Profile prose.\n| 3 | Lia keeps bees zebra' },+ { slug: LIA.slug, source_id: 'default', chunk_text: 'in Turin | fact | 1.0 | world | medium | | | | |\nProfile prose.' },+ { slug: LIA.slug, source_id: 'default', chunk_text: '3 | Lia keeps bees zebracornheld | fact | 1.0 | world | medium | | | | |\nProfile prose.' },+ { slug: LIA.slug, source_id: 'default', chunk_text: '| Lia keeps bees zebracornheld | fact | 1.0 | world | medium | | | | |\nProfile prose.' },+ ] as never) as Array<{ chunk_text: string }>;+ expect(kept.map(k => k.chunk_text)).toEqual(['Profile prose.', 'Profile prose.', 'in Turin | fact | 1.0 | world | medium | | | | |\nProfile prose.',+ 'Profile prose.', 'Profile prose.']);+ }, 60_000);++ test('a member still reads a held row on their own member page', async () => {+ const slug = memberPageSlug(BOB);+ await call('local', 'put_page', { slug, content: memberPageFrontmatter(BOB) + '# Memory\n\n## Facts\n\n' + renderFactsTable([row(1, 'Bob likes tea'), row(2, 'Bob owes rent walrusheld')] as never) + '\n', request_id: randomUUID() });+ await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => tx.executeRaw("UPDATE facts SET visibility='private' WHERE source_markdown_slug=$1 AND fact LIKE '%walrusheld'", [slug]), TEST_WRITE_ATTRIBUTION));+ expect(await stored('Bob owes rent walrusheld')).toEqual(['private']);+ expect(await call('bob', 'get_page', { slug, include_content: true })).toContain('walrusheld');+ expect(await call('unbound', 'get_page', { slug, include_content: true })).not.toContain('walrusheld');+ }, 120_000);++ test('re-deriving the page does not lift the hold', async () => withEnv(env, async () => {+ if (shape === 'repo') {+ for (const p of pages) { const f = join(root, p.slug + '.md'); writeFileSync(f, readFileSync(f, 'utf8').replace('Profile.', 'Profile. Edited by hand.')); }+ commit();+ await performManagedSync(engine, { sourceId: 'default', noPull: true, noEmbed: true, noExtract: true });+ for (const p of pages) expect({ slug: p.slug, after: 'sync', world: (await stored(p.claim)).includes('world') }).toEqual({ slug: p.slug, after: 'sync', world: false });+ }+ for (const p of pages) {+ const page = JSON.parse(await call('local', 'get_page', { slug: p.slug, include_content: true }));+ await call('local', 'put_page', { slug: p.slug, content: String(page.content).replace('Profile.', 'Profile. Local edit.'), expected_revision: page.revision, request_id: randomUUID() });+ expect({ slug: p.slug, after: 'put_page', world: (await stored(p.claim)).includes('world') }).toEqual({ slug: p.slug, after: 'put_page', world: false });+ expect(await call('bob', 'get_page', { slug: p.slug, include_content: true })).not.toContain(p.canary);+ for (const who of ['bob', 'unbound']) expect(await call(who, 'get_versions', { slug: p.slug, include_body: true })).not.toContain(p.canary);+ expect(await call('local', 'get_versions', { slug: p.slug, include_body: true })).toContain(p.canary);+ }+ }), 300_000);++ test('accept makes the restatement world and readable; undo and reject keep it from such a caller', async () => withEnv(env, async () => {+ const owner = { confirmation: { via: 'tty' as const } };+ expect((await decideTrustProposal(engine, refs[LIA.slug]!, 'accept', owner)).status).toBe('accepted');+ expect(await stored(LIA.claim)).toContain('world');+ expect(await call('bob', 'get_page', { slug: LIA.slug, include_content: true })).toContain(LIA.canary);+ expect(await call('unbound', 'search', { query: LIA.canary, match: 'keyword', fields: 'full', limit: 20 })).toContain(LIA.canary);+ const undone = await decideTrustProposal(engine, refs[LIA.slug]!, 'undo', owner);+ expect(undone.status).toBe('undone');+ expect(await call('bob', 'get_page', { slug: LIA.slug, include_content: true })).not.toContain(LIA.canary);+ expect((await decideTrustProposal(engine, refs[KIM.slug]!, 'reject', owner)).status).toBe('rejected');+ expect(await call('bob', 'get_page', { slug: KIM.slug, include_content: true })).not.toContain(KIM.canary);+ expect(await call('owner', 'get_page', { slug: KIM.slug, include_content: true })).toContain(turin(KIM));+ if (shape === 'repo') {+ expect((await decideTrustProposal(engine, refs[MAX.slug]!, 'reject', owner)).status).toBe('rejected');+ const page = JSON.parse(await call('local', 'get_page', { slug: MAX.slug, include_content: true }));+ await call('local', 'put_page', { slug: MAX.slug, content: String(page.content).replace('Profile.', 'Profile. After reject.'), expected_revision: page.revision, request_id: randomUUID() });+ expect(await stored(MAX.claim)).not.toContain('world');+ expect(await call('bob', 'get_page', { slug: MAX.slug, include_content: true })).not.toContain(MAX.canary);+ }+ }), 300_000);++ test('recorded-time reads (known_as_of) leave a held row out too', async () => withEnv(env, async () => {+ await enableBeliefLedger(engine, 'default');+ const at = new Date((await engine.executeRaw<{ t: string }>("SELECT (date_trunc('milliseconds', clock_timestamp()) + interval '1 millisecond')::text AS t"))[0]!.t).toISOString();+ await new Promise(resolve => setTimeout(resolve, 50));+ const held = shape === 'repo' ? [KIM, MAX] : [KIM];+ const recorded = (p: Held): Array<[string, Record<string, unknown>]> => [['get_page', { slug: p.slug, known_as_of: at }],+ ['get_versions', { slug: p.slug, known_as_of: at, include_body: true }], ['get_versions', { slug: p.slug, known_as_of: at, diff: true }]];+ for (const p of held) for (const who of ['bob', 'unbound']) for (const [op, params] of recorded(p)) {+ const out = await call(who, op, params);+ expect(out).not.toContain('"error"');+ expect({ who, op, params, leaks: out.includes(p.canary) }).toEqual({ who, op, params, leaks: false });+ }+ for (const p of held) expect(await call('local', 'get_page', { slug: p.slug, known_as_of: at })).toContain(p.canary);+ }), 300_000);+ });+}++test('GST-2891: a failed held-row lookup drops every fence row line (fail closed)', async () => {+ const failing = { executeRaw: async () => { throw new Error('lookup failed'); } };+ const kept = await dropHeldFenceRows(failing as never, [{ slug: 'people/alice-example', source_id: 'default',+ chunk_text: 'Prose.\n| 1 | Alice likes tea | fact | 1.0 | world | medium | | | | |\nMore prose.' }] as never) as Array<{ chunk_text: string }>;+ expect(kept.map(k => k.chunk_text)).toEqual(['Prose.\nMore prose.']);+});++/** search_exact on a page stored in segments: no body string to read, so the held row is found from the stored chunk. */+for (const kind of testBackends()) {+ describe('GST-2891 search_exact on a segmented page (' + kind + ')', () => {+ let engine: BrainEngine; let close: () => Promise<void>;+ const slug = 'notes/segmented-example';+ beforeAll(async () => {+ if (kind === 'postgres') ({ engine, close } = await isolatedPersistencePostgres(process.env.DATABASE_URL!));+ else { engine = new PGLiteEngine(); await engine.connect({ engine: 'pglite' } as never); await engine.initSchema(); close = () => engine.disconnect(); }+ const facts = '## Facts\n\n' + renderFactsTable([row(1, 'Segment keeps bees capybaraworld'), row(2, 'Segment keeps bees capybaraheld')] as never) + '\n';+ await importSegmented(engine, slug, pageContent(privacyText(mulberry32(2891), 1_400_000, { keys: false }) + facts));+ // The private fact the world cell at row 2 restates, stored narrower (held until the owner accepts).+ await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => tx.executeRaw(+ "INSERT INTO facts (source_id, fact, kind, visibility, source, source_markdown_slug, row_num) VALUES ('default', $1, 'fact', 'private', 'fixture', $2, 2)",+ ['Segment keeps bees capybaraheld', slug]), TEST_WRITE_ATTRIBUTION));+ }, 300_000);+ afterAll(async () => { await close(); });+ const exact = async (remote: boolean, phrase: string) => {+ __resetPrivateVisibilityCacheForTests();+ return (await dispatchToolCall(engine, 'search_exact', { phrase }, { remote, sourceId: 'default' })).content.map(c => c.text).join('\n');+ };++ test('a world-only caller gets the world row and never the held one; the local CLI gets both', async () => {+ expect(await exact(false, 'capybaraheld')).toContain('capybaraheld');+ const remote = await exact(true, 'keeps bees capybara');+ expect(remote).toContain('capybaraworld');+ expect(remote).not.toContain('capybaraheld');+ expect(await exact(true, 'capybaraheld')).not.toContain(slug);+ }, 120_000);+ });+}diff --git a/test/held-row-review-2891.test.ts b/test/held-row-review-2891.test.tsnew file mode 100644index 0000000000..478e6e65eb--- /dev/null+++ b/test/held-row-review-2891.test.ts@@ -0,0 +1,116 @@+/**+ * GST-2891 review findings (round 3): held rows on paths the first fix missed. Each test fails without its fix.+ * 1. known_as_of reads at the slug a page had before a rename. 2. get_chunks on a source other than `default`.+ * 3. A row cut at a chunk boundary in a chunk that starts with a fence trust marker. 4. search_exact keeps a page's+ * public hit when an earlier hit in the same chunk is a held row.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { operationsByName } from '../src/core/operations.ts';+import { dispatchToolCall } from '../src/mcp/dispatch.ts';+import { __resetPrivateVisibilityCacheForTests } from '../src/core/search/private-visibility.ts';+import { renderFactsTable } from '../src/core/facts-fence.ts';+import { dropHeldFenceRows } from '../src/core/eligibility/stamp.ts';+import { markFenceChunk } from '../src/core/eligibility/fence-overlay.ts';+import { withCoordinatedWrite } from '../src/core/persistence/context.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { registerLocalWriter } from '../src/core/persistence/identity.ts';+import { renamePageOntoSlug } from '../src/commands/sync/rename-collision.ts';+import { fillCommitMarks } from '../src/core/belief/coverage-report.ts';+import { testBackends } from './helpers/test-backends.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { beliefBrain, beliefEngineCase, type BeliefBrain } from './helpers/belief-ledger.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';++const row = (n: number, claim: string) => ({ rowNum: n, claim, kind: 'fact', confidence: 1, visibility: 'world', notability: 'medium', active: true });+const pageText = (title: string, rows: ReturnType<typeof row>[], prose = 'Profile.', after = '') =>+ '---\ntype: person\ntitle: ' + title + '\n---\n# ' + title + '\n\n' + prose + '\n\n## Facts\n\n' + renderFactsTable(rows as never) + '\n' + after;+/** Stores a row's fact private while the page body keeps its world cell: the held shape a pending widening leaves. */+const hold = (engine: BrainEngine, sourceId: string, slug: string, claim: string) =>+ engine.transaction(tx => withCoordinatedWrite(tx, [sourceId], () => tx.executeRaw(+ "UPDATE facts SET visibility='private' WHERE source_id=$1 AND source_markdown_slug=$2 AND fact=$3", [sourceId, slug, claim]), TEST_WRITE_ATTRIBUTION));+const text = (r: unknown) => JSON.stringify(r);++for (const engineCase of testBackends().map(beliefEngineCase)) {+ describe('GST-2891 recorded-time reads after a rename (' + engineCase.name + ')', () => {+ const dbNow = async (b: BeliefBrain) => { await new Promise(r => setTimeout(r, 20)); const t = new Date((await b.engine.executeRaw<{ t: string }>("SELECT date_trunc('milliseconds', clock_timestamp()) + interval '1 millisecond' AS t"))[0]!.t).toISOString(); await new Promise(r => setTimeout(r, 20)); return t; };+ test('a held row stays out of known_as_of reads at the slug the page had before the rename', () => beliefBrain(async b => {+ const old = 'people/ida-example', moved = 'people/ida-moved-example', claim = 'Ida keeps bees narwhalheld';+ await b.put(old, pageText('Ida Example', [row(1, 'Ida lives in Turin'), row(2, claim)]));+ await hold(b.engine, b.sourceId, old, claim);+ const before = await dbNow(b);+ await renamePageOntoSlug(b.engine, { sourceId: b.sourceId, oldSlug: old, newSlug: moved, to: moved + '.md' });+ await fillCommitMarks(b.engine, b.sourceId);+ const remote: OperationContext = { ...b.ctx, remote: true };+ const run = (ctx: OperationContext, op: string, params: Record<string, unknown>) => { __resetPrivateVisibilityCacheForTests(); return operationsByName[op]!.handler(ctx, params); };+ expect(text(await run(b.ctx, 'get_page', { slug: old, known_as_of: before }))).toContain('narwhalheld');+ expect(text(await run(remote, 'get_page', { slug: moved }))).not.toContain('narwhalheld');+ const reads: Array<[string, Record<string, unknown>]> = [['get_page', { slug: old, known_as_of: before }],+ ['get_versions', { slug: old, known_as_of: before, include_body: true }], ['get_versions', { slug: old, known_as_of: before, diff: true }]];+ for (const [op, params] of reads) {+ const out = text(await run(remote, op, params));+ expect(out).toContain('Ida lives in Turin');+ expect({ op, params, leaks: out.includes('narwhalheld') }).toEqual({ op, params, leaks: false });+ }+ }, { managed: false, enable: true, databaseUrl: engineCase.databaseUrl }), 180_000);+ });+}++for (const kind of testBackends()) {+ describe('GST-2891 held rows on chunk reads (' + kind + ')', () => {+ let engine: BrainEngine; let close: () => Promise<void>;+ const call = async (remote: boolean, sourceId: string, op: string, params: Record<string, unknown>) => {+ __resetPrivateVisibilityCacheForTests();+ return (await dispatchToolCall(engine, op, { ...params }, { remote, sourceId })).content.map(c => c.text).join('\n');+ };+ const OTHER = { slug: 'people/ola-example', claim: 'Ola keeps bees tapirheld' };+ const MIXED = { slug: 'people/pia-example', claim: 'Pia keeps bees lynxheld secretly' };+ beforeAll(async () => {+ if (kind === 'postgres') ({ engine, close } = await isolatedPersistencePostgres(process.env.DATABASE_URL!));+ else { engine = new PGLiteEngine(); await engine.connect({ engine: 'pglite' } as never); await engine.initSchema(); close = () => engine.disconnect(); }+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await registerLocalWriter(engine, 'cli');+ await engine.executeRaw("INSERT INTO sources (id, name) VALUES ('source-a', 'source-a') ON CONFLICT (id) DO NOTHING");+ await call(false, 'source-a', 'put_page', { slug: OTHER.slug, content: pageText('Ola Example', [row(1, 'Ola lives in Turin'), row(2, OTHER.claim)]), request_id: randomUUID() });+ await hold(engine, 'source-a', OTHER.slug, OTHER.claim);+ const filler = Array.from({ length: 6 }, (_, i) => 'Line ' + i + ' of plain notes about the garden and the weather this spring.').join(' ');+ await call(false, 'default', 'put_page', { slug: MIXED.slug, content: pageText('Pia Example', [row(1, 'Pia lives in Turin'), row(2, MIXED.claim)], 'Profile.',+ filler + ' The museum keeps bees lynxheld in glass cases.\n'), request_id: randomUUID() });+ await hold(engine, 'default', MIXED.slug, MIXED.claim);+ }, 300_000);+ afterAll(async () => { await disposePersistenceConsumer(engine); await close(); }, 60_000);++ test('get_chunks on a source other than default leaves the held row out', async () => {+ expect(await call(false, 'source-a', 'get_chunks', { slug: OTHER.slug })).toContain('tapirheld');+ const out = await call(true, 'source-a', 'get_chunks', { slug: OTHER.slug });+ expect(out).toContain('Ola lives in Turin');+ expect(out).not.toContain('tapirheld');+ }, 120_000);++ test('a row cut at a chunk boundary after a fence trust marker leaves none of its text behind', async () => {+ const tail = ' | fact | 1.0 | world | medium | | | | |\nProfile prose.';+ const kept = await dropHeldFenceRows(engine, [+ { slug: MIXED.slug, source_id: 'default', chunk_text: markFenceChunk('agent_written', 'bees lynxheld secretly' + tail) },+ { slug: MIXED.slug, source_id: 'default', chunk_text: markFenceChunk('agent_written', '2 | Pia keeps bees lynxheld secretly' + tail) },+ { slug: MIXED.slug, source_id: 'default', chunk_text: markFenceChunk('agent_written', '| Pia keeps bees lynxheld secretly' + tail) },+ ] as never) as Array<{ chunk_text: string }>;+ for (const k of kept) expect(k.chunk_text).not.toContain('lynxheld');+ expect(kept.map(k => k.chunk_text.includes('Profile prose.'))).toEqual([true, true, true]);+ }, 60_000);++ test('search_exact keeps the public hit of a page whose earlier hit is a held row', async () => {+ const chunks = await engine.executeRaw<{ chunk_text: string }>('SELECT cc.chunk_text FROM content_chunks cc JOIN pages p ON p.id=cc.page_id WHERE p.slug=$1 ORDER BY cc.chunk_index', [MIXED.slug]);+ const both = chunks.find(c => c.chunk_text.includes(MIXED.claim) && c.chunk_text.includes('museum keeps bees'));+ expect(both && both.chunk_text.indexOf(MIXED.claim) < both.chunk_text.indexOf('museum keeps bees')).toBe(true);+ expect(await call(false, 'default', 'search_exact', { phrase: 'keeps bees lynxheld' })).toContain('secretly');+ const out = JSON.parse(await call(true, 'default', 'search_exact', { phrase: 'keeps bees lynxheld' }));+ expect(JSON.stringify(out.matches)).toContain('museum keeps bees lynxheld');+ expect(JSON.stringify(out.matches)).not.toContain('secretly');+ expect(out.complete).toBe(true);+ }, 120_000);+ });+}diff --git a/test/held-row-writeback-2891.test.ts b/test/held-row-writeback-2891.test.tsnew file mode 100644index 0000000000..8ddb3ebee0--- /dev/null+++ b/test/held-row-writeback-2891.test.ts@@ -0,0 +1,240 @@+/**+ * GST-2891 write-back: a caller whose fact reads are world-only reads a page without its held rows, so a page it writes+ * back arrives without them. put_page restores each held row it left out from the stored page (never from the writer's text), so+ * the pending widening survives and the owner's accept still widens it; the owner's own writes and the local CLI's+ * stay as they were. edit_page's diff shows such a caller the page as get_page does, without the held rows.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { GBrainOAuthProvider } from '../src/core/oauth-provider.ts';+import { sqlQueryForEngine } from '../src/core/sql-query.ts';+import { mintLegacyToken } from '../src/core/token-mint.ts';+import { dispatchToolCall } from '../src/mcp/dispatch.ts';+import { __resetPrivateVisibilityCacheForTests } from '../src/core/search/private-visibility.ts';+import { setOwner } from '../src/core/access/owner.ts';+import { bindCredential } from '../src/core/access/members.ts';+import type { AuthInfo } from '../src/core/ops/contract.ts';+import { renderFactsTable } from '../src/core/facts-fence.ts';+import { withCoordinatedWrite, withTrustPromotion } from '../src/core/persistence/context.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { registerLocalWriter } from '../src/core/persistence/identity.ts';+import { decideTrustProposal } from '../src/core/trust/decide.ts';+import { parseTrustProposalRef } from '../src/core/trust/proposals.ts';+import { testBackends } from './helpers/test-backends.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { readsUnscopedOnPurpose } from './helpers/reads-unscoped.ts';+import { importSegmented, pageContent } from './helpers/segmented-privacy-fixture.ts';+import { checkEnforcedWrite, enforcedRead, ENFORCED_MEMBER_ENV } from '../src/cli/enforced-member.ts';+import { operationsByName } from '../src/core/operations.ts';+import { withEnv } from './helpers/with-env.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';++const OWNER = 'owner-example';+type Row = { rowNum: number; claim: string; kind: string; confidence: number; visibility: string; notability: string; active: boolean; context?: string };+const row = (n: number, claim: string, visibility = 'world'): Row => ({ rowNum: n, claim, kind: 'fact', confidence: 1, visibility, notability: 'medium', active: true });+const NOTES = '## Notes\n\nSomething here.\n';+const pageText = (title: string, rows: Row[], prose = 'Profile.', after = NOTES) =>+ '---\ntype: person\ntitle: ' + title + '\nvalid_from: 2020-01-01\n---\n# ' + title + '\n\n' + prose + '\n\n## Facts\n\n' + renderFactsTable(rows as never) + '\n\n' + after;+type Held = { slug: string; title: string; claim: string; canary: string; turin: string; tp: number };+type FactRow = { fact: string; visibility: string; row_num: number | null; expired: boolean };+type Who = 'local' | 'writer' | 'owner';++for (const kind of testBackends()) {+ describe('GST-2891 held rows on write-back (' + kind + ')', () => {+ let engine: BrainEngine; let close: () => Promise<void>;+ const tokens: Record<string, string> = {};+ const provider = () => new GBrainOAuthProvider({ sql: sqlQueryForEngine(engine), transaction: fn => engine.transaction(tx => fn(sqlQueryForEngine(tx))) });+ async function call(who: Who, op: string, params: Record<string, unknown>): Promise<string> {+ __resetPrivateVisibilityCacheForTests();+ if (who === 'local') return (await dispatchToolCall(engine, op, { ...params }, { remote: false, sourceId: 'default' })).content.map(c => c.text).join('\n');+ const auth = await provider().verifyAccessToken(tokens[who]!) as unknown as AuthInfo;+ return (await dispatchToolCall(engine, op, { ...params }, { remote: true, transport: 'http', auth, sourceId: 'default', takesHoldersAllowList: ['world'] })).content.map(c => c.text).join('\n');+ }+ const page = async (who: Who, slug: string) => JSON.parse(await call(who, 'get_page', { slug, include_content: true })) as { content: string; revision: string };+ const stored = (slug: string) => call('local', 'get_page', { slug, include_content: true });+ const put = (who: Who, slug: string, content: string, revision: string) => call(who, 'put_page', { slug, content, expected_revision: revision, request_id: randomUUID() });+ const facts = (slug: string) => engine.executeRaw<FactRow>(+ 'SELECT fact, visibility, row_num, expired_at IS NOT NULL AS expired FROM facts WHERE source_id=$1 AND source_markdown_slug=$2 ORDER BY id', ['default', slug]);+ const accept = async (tp: number) => (await decideTrustProposal(engine, tp, 'accept', { confirmation: { via: 'tty' as const } })).status;+ /** A page whose row 2 is a confirmed private fact and whose row 3 is the writer's world restatement of it, held until the owner accepts. */+ async function heldPage(name: string): Promise<Held> {+ const first = name[0]!.toUpperCase() + name.slice(1), canary = name + 'heldcanary';+ const held = { slug: 'people/' + name + '-example', title: first + ' Example', canary, claim: first + ' keeps bees ' + canary, turin: first + ' lives in Turin' };+ await call('local', 'put_page', { slug: held.slug, content: pageText(held.title, [row(1, held.turin), row(2, held.claim, 'private')]), request_id: randomUUID() });+ for (const r of await engine.executeRaw<{ id: number }>('SELECT id FROM facts WHERE source_id=$1 AND source_markdown_slug=$2', ['default', held.slug]))+ await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => withTrustPromotion(tx, 'user_confirmed', () => tx.executeRaw("UPDATE facts SET trust_tier='user_confirmed' WHERE id=$1", [r.id])), TEST_WRITE_ATTRIBUTION));+ const res = await call('writer', 'remember', { fact: held.claim, provenance: 'chat', entity: held.slug, request_id: randomUUID() });+ const tp = parseTrustProposalRef(res.match(/tp\d+/)![0])!;+ expect(await stored(held.slug)).toContain('| 3 | ' + held.claim + ' |');+ expect((await page('writer', held.slug)).content).not.toContain(canary);+ return { ...held, tp };+ }+ /** The held restatement is still row 3 of the stored page and of its facts, and the owner's accept widens it. */+ async function heldRowKept(h: Held) {+ expect(await stored(h.slug)).toContain('| 3 | ' + h.claim + ' |');+ const live = async () => (await facts(h.slug)).filter(f => f.fact === h.claim && !f.expired).map(f => [f.row_num, f.visibility]);+ expect(await live()).toEqual([[2, 'private'], [3, 'private']]);+ expect(await accept(h.tp)).toBe('accepted');+ expect(await live()).toEqual([[3, 'world']]);+ }++ beforeAll(async () => {+ if (kind === 'postgres') ({ engine, close } = await isolatedPersistencePostgres(process.env.DATABASE_URL!));+ else { engine = new PGLiteEngine(); await engine.connect({ engine: 'pglite' } as never); await engine.initSchema(); close = () => engine.disconnect(); }+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await registerLocalWriter(engine, 'cli');+ await engine.executeRaw('INSERT INTO access_members (id) VALUES ($1)', [OWNER]);+ const writer = await mintLegacyToken(engine, { name: 'writer', scopes: ['read', 'write'], takesHolders: ['world'] });+ await readsUnscopedOnPurpose(engine, writer.id);+ const owner = await mintLegacyToken(engine, { name: 'owner', scopes: ['read', 'write'], takesHolders: ['world'] });+ await bindCredential(engine, { kind: 'token', id: owner.id }, { member: OWNER });+ Object.assign(tokens, { writer: writer.token, owner: owner.token });+ await setOwner(engine, OWNER);+ }, 300_000);+ afterAll(async () => { await disposePersistenceConsumer(engine); await close(); }, 60_000);++ test('a get_page -> put_page round trip keeps the held row, and the owner\'s accept still widens it', async () => {+ const h = await heldPage('ada');+ const cur = await page('writer', h.slug);+ await put('writer', h.slug, cur.content.replace('Something here.', 'Something else.'), cur.revision);+ expect(await stored(h.slug)).toContain('Something else.');+ await heldRowKept(h);+ }, 180_000);++ test('a whole new body keeps the held row', async () => {+ const h = await heldPage('bea');+ const cur = await page('writer', h.slug);+ await put('writer', h.slug, pageText(h.title, [row(1, 'Bea lives in Milan')], 'A new profile.', '## Elsewhere\n\nNew notes.\n'), cur.revision);+ const body = await stored(h.slug);+ expect(body).toContain('Bea lives in Milan');+ expect(body).not.toContain(h.turin);+ await heldRowKept(h);+ }, 180_000);++ test('deleting the lines around the held row keeps it and honors the deletion of what the writer saw', async () => {+ const h = await heldPage('cai');+ const cur = await page('writer', h.slug);+ const lines = cur.content.split('\n');+ const cut = lines.filter(l => !l.includes(h.turin) && l !== '## Notes' && l !== 'Something here.');+ expect(cut.length).toBe(lines.length - 3);+ await put('writer', h.slug, cut.join('\n'), cur.revision);+ const body = await stored(h.slug);+ expect(body).not.toContain(h.turin);+ expect(body).not.toContain('Something here.');+ await heldRowKept(h);+ }, 180_000);++ test('a row 3 the writer sends with the held claim cannot reword it: the stored cells are restored, and the held row is not duplicated', async () => {+ const h = await heldPage('dov');+ const heldLine = () => stored(h.slug).then(s => (JSON.parse(s) as { compiled_truth: string }).compiled_truth.split('\n').filter(l => l.startsWith('| 3 | ')));+ const before = await heldLine();+ const cur = await page('writer', h.slug);+ await put('writer', h.slug, pageText(h.title, [row(1, h.turin), { ...row(3, h.claim), confidence: 0.5, notability: 'high', context: 'written by the writer' }]), cur.revision);+ const body = (JSON.parse(await stored(h.slug)) as { compiled_truth: string }).compiled_truth;+ expect(body).not.toContain('written by the writer');+ expect(await heldLine()).toEqual(before);+ expect(body.split('| ' + h.claim + ' |').length - 1).toBe(2);+ await heldRowKept(h);+ }, 180_000);++ test('the owner\'s own write and the local CLI\'s remove the held row as on master', async () => {+ for (const who of ['owner', 'local'] as const) {+ const h = await heldPage(who === 'owner' ? 'eve' : 'fay');+ const cur = await page(who, h.slug);+ expect(cur.content).toContain('| 3 | ' + h.claim + ' |');+ await put(who, h.slug, cur.content.split('\n').filter(l => !l.startsWith('| 3 | ')).join('\n'), cur.revision);+ expect(await stored(h.slug)).not.toContain('| 3 | ' + h.claim + ' |');+ expect((await facts(h.slug)).filter(f => f.fact === h.claim).map(f => [f.row_num, f.visibility, f.expired])).toEqual([[2, 'private', false], [null, 'private', true]]);+ expect(await accept(h.tp)).toBe('superseded');+ }+ }, 240_000);++ test('an enforced member (GBRAIN_ENFORCED_MEMBER) reads without the held row, and its put_page and edit_page keep it and leave it out of the diff', async () => {+ // The held row is the page's only copy of the claim in the fence (a remote put_page restating private row 2 as world), so no private fence row makes the member's write refuse.+ const h = { slug: 'people/jon-example', title: 'Jon Example', claim: 'Jon keeps bees jonheldcanary', canary: 'jonheldcanary', turin: 'Jon lives in Turin' };+ await call('local', 'put_page', { slug: h.slug, content: pageText(h.title, [row(1, h.turin), row(2, h.claim, 'private')]), request_id: randomUUID() });+ for (const r of await engine.executeRaw<{ id: number }>('SELECT id FROM facts WHERE source_id=$1 AND source_markdown_slug=$2', ['default', h.slug]))+ await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => withTrustPromotion(tx, 'user_confirmed', () => tx.executeRaw("UPDATE facts SET trust_tier='user_confirmed' WHERE id=$1", [r.id])), TEST_WRITE_ATTRIBUTION));+ const before = await page('writer', h.slug);+ const restated = JSON.parse(await put('writer', h.slug, pageText(h.title, [row(1, h.turin), row(2, h.claim)]), before.revision)) as { contested?: { proposal_ref: string } };+ const tp = parseTrustProposalRef(restated.contested!.proposal_ref)!;+ expect(await stored(h.slug)).toContain('| 2 | ' + h.claim + ' | fact | 1.0 | world |');+ const live = async () => (await facts(h.slug)).filter(f => f.fact === h.claim && !f.expired).map(f => f.visibility);+ const liveBefore = await live();+ expect(liveBefore.length).toBeGreaterThan(0);+ await engine.executeRaw('INSERT INTO access_members (id) VALUES ($1) ON CONFLICT DO NOTHING', ['member-example']);+ // The CLI runs a checked write through the operation's handler, as main.ts does (never the MCP dispatcher).+ const cli = async (op: string, params: Record<string, unknown>) => JSON.stringify(await operationsByName[op]!.handler({ engine, config: { engine: engine.kind, embedding_disabled: true },+ sourceId: 'default', dryRun: false, remote: false, logger: { info() {}, warn() {}, error() {} } } as unknown as OperationContext, params));+ const unwrap = (c: string) => c.includes('```text\n') ? c.slice(c.indexOf('```text\n') + 8, c.lastIndexOf('\n```')) : c;+ await withEnv({ [ENFORCED_MEMBER_ENV]: 'member-example' }, async () => {+ __resetPrivateVisibilityCacheForTests();+ const read = await enforcedRead(engine, 'get', 'get_page', { slug: h.slug, include_content: true }, { sourceId: 'default' }) as { content: string; revision: string };+ expect(read.content).not.toContain(h.canary);+ const putParams: Record<string, unknown> = { slug: h.slug, content: unwrap(read.content).replace('Something here.', 'Something else.'), expected_revision: read.revision, request_id: randomUUID() };+ await checkEnforcedWrite(engine, 'put', 'put_page', putParams, 'default');+ await cli('put_page', putParams);+ expect(await stored(h.slug)).toContain('Something else.');+ const editParams: Record<string, unknown> = { slug: h.slug, expected_revision: (await page('local', h.slug)).revision, edits: [{ old_text: '## Notes', new_text: '## More notes' }], request_id: randomUUID() };+ await checkEnforcedWrite(engine, 'edit', 'edit_page', editParams, 'default');+ const edited = await cli('edit_page', editParams);+ expect(edited).toContain('## More notes');+ expect(edited).not.toContain(h.canary);+ });+ expect(await stored(h.slug)).toContain('| 2 | ' + h.claim + ' | fact | 1.0 | world |');+ expect(await live()).toEqual(liveBefore);+ expect(await accept(tp)).toBe('accepted');+ expect(await live()).toEqual(['world']);+ }, 240_000);++ test('edit_page\'s diff leaves the held row out for the writer; the owner and the local CLI still see it', async () => {+ for (const who of ['writer', 'owner', 'local'] as const) {+ const h = await heldPage(who === 'writer' ? 'gus' : who === 'owner' ? 'hal' : 'ivy');+ const cur = await page(who, h.slug);+ const out = await call(who, 'edit_page', { slug: h.slug, expected_revision: cur.revision, edits: [{ old_text: '## Notes', new_text: '## More notes' }], request_id: randomUUID() });+ expect(out).toContain('## More notes');+ expect({ who, leaks: out.includes(h.canary) }).toEqual({ who, leaks: who !== 'writer' });+ }+ }, 240_000);+ });++ describe('GST-2891 edit_page diff on a segmented page (' + kind + ')', () => {+ let engine: BrainEngine; let close: () => Promise<void>;+ let token = '';+ const slug = 'notes/segmented-edit-example';+ beforeAll(async () => {+ if (kind === 'postgres') ({ engine, close } = await isolatedPersistencePostgres(process.env.DATABASE_URL!));+ else { engine = new PGLiteEngine(); await engine.connect({ engine: 'pglite' } as never); await engine.initSchema(); close = () => engine.disconnect(); }+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await registerLocalWriter(engine, 'cli');+ const facts = '## Facts\n\n' + renderFactsTable([row(1, 'Segment keeps bees ibexworld'), row(2, 'Segment keeps bees ibexheld')] as never) + '\n\n' + NOTES;+ await importSegmented(engine, slug, pageContent(Array.from({ length: 32_000 }, (_, i) => 'Line ' + i + ' of plain notes about the garden.').join('\n') + '\n\n' + facts));+ await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => tx.executeRaw(+ "INSERT INTO facts (source_id, fact, kind, visibility, source, source_markdown_slug, row_num) VALUES ('default', $1, 'fact', 'private', 'fixture', $2, 2)",+ ['Segment keeps bees ibexheld', slug]), TEST_WRITE_ATTRIBUTION));+ const writer = await mintLegacyToken(engine, { name: 'writer', scopes: ['read', 'write'], takesHolders: ['world'] });+ await readsUnscopedOnPurpose(engine, writer.id);+ token = writer.token;+ }, 300_000);+ afterAll(async () => { await disposePersistenceConsumer(engine); await close(); }, 60_000);++ test('the writer\'s diff leaves the held row out; the local CLI\'s keeps it', async () => {+ const edit = async (remote: boolean, from: string, to: string) => {+ __resetPrivateVisibilityCacheForTests();+ const { revision } = JSON.parse((await dispatchToolCall(engine, 'get_page', { slug, body: 'none' }, { remote: false, sourceId: 'default' })).content.map(c => c.text).join('\n')) as { revision: string };+ const auth = remote ? await new GBrainOAuthProvider({ sql: sqlQueryForEngine(engine), transaction: fn => engine.transaction(tx => fn(sqlQueryForEngine(tx))) }).verifyAccessToken(token) as unknown as AuthInfo : undefined;+ const ctx = remote ? { remote: true, transport: 'http', auth, sourceId: 'default', takesHoldersAllowList: ['world'] } : { remote: false, sourceId: 'default' };+ return (await dispatchToolCall(engine, 'edit_page', { slug, expected_revision: revision, edits: [{ old_text: from, new_text: to }], request_id: randomUUID() }, ctx as never)).content.map(c => c.text).join('\n');+ };+ const remote = await edit(true, '## Notes', '## More notes');+ expect(remote).toContain('## More notes');+ expect(remote).not.toContain('ibexheld');+ expect(await edit(false, '## More notes', '## Notes')).toContain('ibexheld');+ }, 240_000);+ });+}diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex f8f2ec5191..729e8c2e12 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -199,6 +199,9 @@ test/held-bare-name-move-gst2492.test.tstest/held-fence-readers-gst2492.test.tstest/held-move-plan-gst2497.test.tstest/held-read-signals.test.ts+test/held-row-page-reads-2891.test.ts+test/held-row-review-2891.test.ts+test/held-row-writeback-2891.test.tstest/hermes-clarify-transcripts-7396.test.tstest/hold-action-chain-6712.test.tstest/holding-page-saves-gst2497.test.ts@@ -628,6 +631,7 @@ test/repair-unwithdrawn-facts-8819.test.tstest/repair-visibility-propose-gst2670.test.tstest/request-tools-combined-gbra449.test.tstest/requester-scope.test.ts+test/restated-private-cross-entity-2891.test.tstest/retrieval-coverage-notices.test.tstest/schema-missing-tables.test.tstest/schema-replay-deadlock-6143.test.tsdiff --git a/test/restated-private-cross-entity-2891.test.ts b/test/restated-private-cross-entity-2891.test.tsnew file mode 100644index 0000000000..40b62f0021--- /dev/null+++ b/test/restated-private-cross-entity-2891.test.ts@@ -0,0 +1,103 @@+/**+ * GST-2891: the GST-2875 hold checks that no world copy of the claim exists before it holds a restatement. That check+ * must be on the same entity: a world copy the writer saved about another entity first must not let it restate a+ * confirmed private fact as world on this one.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { mkdtempSync, rmSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import type { Principal } from '../src/core/persistence/model.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { operationsByName } from '../src/core/operations.ts';+import { mintLegacyToken } from '../src/core/token-mint.ts';+import { registerLocalWriter, readLocalWriter } from '../src/core/persistence/identity.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { withCoordinatedWrite, withTrustPromotion } from '../src/core/persistence/context.ts';+import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts';+import { __resetPrivateVisibilityCacheForTests } from '../src/core/search/private-visibility.ts';+import { decideTrustProposal } from '../src/core/trust/decide.ts';+import { parseTrustProposalRef } from '../src/core/trust/proposals.ts';+import { renderFactsTable } from '../src/core/facts-fence.ts';+import { LEGACY_EMBEDDING_CONFIG } from './helpers/legacy-embedding-config.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { readsUnscopedOnPurpose } from './helpers/reads-unscoped.ts';+import { withEnv } from './helpers/with-env.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';++const home = mkdtempSync(join(tmpdir(), 'gst2891x-'));+const engines: BrainEngine[] = [];+let closePostgres: (() => Promise<void>) | undefined;+const quiet = { info() {}, warn() {}, error() {} };++beforeAll(async () => withEnv({ GBRAIN_HOME: home }, async () => {+ configureGateway({ ...LEGACY_EMBEDDING_CONFIG, env: {} });+ const lite = new PGLiteEngine(); await lite.connect({}); await lite.initSchema(); engines.push(lite);+ if (process.env.DATABASE_URL) { const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL); engines.push(pg.engine); closePostgres = pg.close; }+}), 120_000);+afterAll(async () => {+ resetGateway();+ for (const engine of engines) { await disposePersistenceConsumer(engine); await engine.disconnect(); }+ await closePostgres?.();+ rmSync(home, { recursive: true, force: true });+});++interface Brain { engine: BrainEngine; sourceId: string; remote: OperationContext; local: OperationContext }+async function brain(engine: BrainEngine, managed: boolean): Promise<Brain> {+ const sourceId = 'open-' + randomUUID().slice(0, 8);+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ await engine.executeRaw('INSERT INTO sources(id,name) VALUES($1,$1)', [sourceId]);+ await engine.setConfig('brain.audience', 'shared');+ await registerLocalWriter(engine, 'cli');+ const minted = await mintLegacyToken(engine, { name: 'token-' + sourceId, scopes: ['read', 'write'], sourceGrant: [sourceId], takesHolders: ['world'] });+ await readsUnscopedOnPurpose(engine, minted.id);+ if (managed) await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1');+ const base = { engine, config: { engine: engine.kind, embedding_disabled: true }, sourceId, dryRun: false, logger: quiet };+ await readLocalWriter(engine, 'cli');+ return {+ engine, sourceId,+ remote: { ...base, remote: true, transport: 'http', takesHoldersAllowList: ['world'],+ auth: { token: '', clientId: minted.id, principal: { kind: 'legacy_token', id: minted.id } as Principal, sourceId, allowedSources: [sourceId], scopes: ['read', 'write'] } },+ local: { ...base, remote: false },+ } as Brain;+}+const run = (ctx: OperationContext, op: string, params: Record<string, unknown>) =>+ operationsByName[op].handler(ctx, { request_id: randomUUID(), ...params }) as Promise<Record<string, any>>;+const row = (n: number, claim: string, visibility = 'world') => ({ rowNum: n, claim, kind: 'fact', confidence: 1, visibility, notability: 'medium', active: true });+const note = (title: string, rows: ReturnType<typeof row>[]) => '---\ntype: note\ntitle: ' + title + '\n---\nProfile.\n\n## Facts\n\n' + renderFactsTable(rows as never) + '\n';+const rowsOf = (b: Brain, claim: string) => b.engine.executeRaw<{ id: number; visibility: string; entity_slug: string | null }>(+ 'SELECT id::int AS id, visibility, entity_slug FROM facts WHERE source_id=$1 AND fact=$2 AND expired_at IS NULL ORDER BY id', [b.sourceId, claim]);+async function remoteSees(b: Brain, slug: string, claim: string) {+ __resetPrivateVisibilityCacheForTests();+ return JSON.stringify(await run(b.remote, 'recall', { entity: slug })).includes(claim);+}++describe('GST-2891: a world copy on another entity does not lift the restatement hold', () => {+ for (const engineIndex of [0, 1]) for (const managed of [true, false]) {+ test('remember on entity A after a world copy on entity B is still held (' + (engineIndex ? 'postgres' : 'pglite') + ', ' + (managed ? 'managed' : 'unmanaged') + ')', async () => withEnv({ GBRAIN_HOME: home }, async () => {+ for (const engine of engines.slice(engineIndex, engineIndex + 1)) {+ const b = await brain(engine, managed);+ const claim = 'Keeps bees on the roof walrusheld';+ await run(b.local, 'put_page', { slug: 'people/lia-example', content: note('Lia Example', [row(1, 'Lives in Turin'), row(2, claim, 'private')]) });+ for (const r of await engine.executeRaw<{ id: number }>('SELECT id FROM facts WHERE source_id=$1 AND source_markdown_slug=$2', [b.sourceId, 'people/lia-example']))+ await engine.transaction(tx => withCoordinatedWrite(tx, [b.sourceId], () => withTrustPromotion(tx, 'user_confirmed', () => tx.executeRaw("UPDATE facts SET trust_tier='user_confirmed' WHERE id=$1", [r.id])), TEST_WRITE_ATTRIBUTION));+ expect(await remoteSees(b, 'people/lia-example', claim)).toBe(false);+ const decoy = await run(b.remote, 'remember', { fact: claim, provenance: 'chat', entity: 'people/ben-example' });+ expect(decoy.contested ?? decoy.outcome?.contested).toBeUndefined();+ const res = await run(b.remote, 'remember', { fact: claim, provenance: 'chat', entity: 'people/lia-example' });+ const rows = await rowsOf(b, claim);+ const label = engine.kind + (managed ? ' managed' : ' unmanaged');+ const lia = rows.filter(r => r.entity_slug === 'people/lia-example').map(r => r.visibility);+ expect({ label, lia, remoteSees: await remoteSees(b, 'people/lia-example', claim) }).toEqual({ label, lia: ['private', 'private'], remoteSees: false });+ const ref = res.contested?.proposal_ref ?? res.outcome?.contested?.proposal_ref;+ expect(ref).toMatch(/^tp\d+$/);+ expect((await decideTrustProposal(engine, parseTrustProposalRef(ref)!, 'accept', { confirmation: { via: 'tty' as const } })).status).toBe('accepted');+ expect(await remoteSees(b, 'people/lia-example', claim)).toBe(true);+ }+ }), 180_000);+ }+});