Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Save facts to a holding page without re-reading its table and keep its newest five versions
garrytanc79929ef7659
diff --git a/behavior-changes.d/holding_page_versions_gst2497.md b/behavior-changes.d/holding_page_versions_gst2497.mdnew file mode 100644index 0000000000..de86ac9b92--- /dev/null+++ b/behavior-changes.d/holding_page_versions_gst2497.md@@ -0,0 +1 @@+A holding page (memory/unattributed, or a folder's <folder>/memory/unattributed) now keeps only its newest five saved versions, whatever their size, as large pages already did. `get_versions` and `gbrain revert` see only those five. Every fact row on the page is kept, the page itself still holds every row, and Git keeps every state of the page where the brain has a repository. An older holding page sheds its extra versions in batches of at most 100 on its next saves, so nothing runs on upgrade. Once a version is removed, an `as_of` read of the page reports incomplete history, and `gbrain repair unwithdrawn-facts` lists that page's candidates as ambiguous: restore them with `--include-ambiguous` and the hash of that preview. Nothing to run.diff --git a/changelog.d/capy-holding-page-saves-gst2497.md b/changelog.d/capy-holding-page-saves-gst2497.mdnew file mode 100644index 0000000000..0c11fecaf9--- /dev/null+++ b/changelog.d/capy-holding-page-saves-gst2497.md@@ -0,0 +1,9 @@+### Saving a fact to a holding page is faster and keeps less history++**Saving a fact to a holding page (memory/unattributed, or a folder's holding page) no longer slows down and fills the database as the page grows (GST-2497).** Each save used to read the page's whole facts table about seventeen times and store another full copy of the page, so a page with thousands of facts took over a second per save and every save added a copy of it.+- One save now reads the table at most three times: a parse of a given table is reused, and every caller gets its own copy.+- A holding page keeps its newest five saved versions, the way large pages already do. Every fact row stays, the page still holds every row, and Git keeps every state where the brain has a repository. An older holding page sheds extra versions in batches of at most 100 on its next saves. `get_versions` and `gbrain revert` see only the five; an `as_of` read of the page says its history is incomplete, and `gbrain repair unwithdrawn-facts` lists that page's candidates as ambiguous (`--include-ambiguous` restores them).+- After a sync, the held-fact move no longer plans a move on every sync when a folder rule covers a holding page but not the entity of a row on it. A rule that narrows a held row below its holding page's readers still moves it right after the sync.+- On Postgres, the held-fact move's plan reads the write-through setting under the same deadline as its other reads, so a session holding the config table no longer keeps a sync past its preparation budget.++<!-- behavior-change: holding_page_versions_gst2497 -->diff --git a/docs/architecture/key-files/core-services-2-continued.md b/docs/architecture/key-files/core-services-2-continued.mdindex de2a931486..19b1047cd0 100644--- a/docs/architecture/key-files/core-services-2-continued.md+++ b/docs/architecture/key-files/core-services-2-continued.md@@ -19,7 +19,7 @@ The page-state concurrency foundation under `src/core/page-state/` and the shareEmbedding-only publication also requires a live, unarchived source and nondeleted page under the same source/page row locks. A prepared vector cannot bypass a later archive, model change or withdrawal. `test/e2e/migration-wave-prepared-withdrawal.test.ts` exercises both publication orders and exact unchanged-vector refusal on both engines.-- `src/core/page-state/version-types.ts` — complete canonical version contract with nullable legacy fields, re-exported by `types.ts` for existing callers. `page-state/versions.ts` records deletion, timeline, tags and originating revision in the page publication transaction. Wave B: a segmented page's version references its body generation (`body_storage: 'segmented'`, `body_generation`, `body_bytes`, `content_hash`, NULL bodies; O(1)); `pruneSegmentedVersions` keeps the newest `versions.segmented_max_per_page` and prunes oldest-first while bytes pinned beyond the live generation exceed max(`versions.segmented_max_pinned_bytes`, body_bytes), then `sweepSegments`; `get_versions` returns their size and, with `include_body`, a `body_ref`; `revert_version` refuses them with `page_body_segmented` until PR 3. `LARGE_PAGE_VERSIONS_KEPT` lives in `page-body/limits.ts`.+- `src/core/page-state/version-types.ts` — complete canonical version contract with nullable legacy fields, re-exported by `types.ts` for existing callers. `page-state/versions.ts` records deletion, timeline, tags and originating revision in the page publication transaction. Wave B: a segmented page's version references its body generation (`body_storage: 'segmented'`, `body_generation`, `body_bytes`, `content_hash`, NULL bodies; O(1)); `pruneSegmentedVersions` keeps the newest `versions.segmented_max_per_page` and prunes oldest-first while bytes pinned beyond the live generation exceed max(`versions.segmented_max_pinned_bytes`, body_bytes), then `sweepSegments`; `get_versions` returns their size and, with `include_body`, a `body_ref`; `revert_version` refuses them with `page_body_segmented` until PR 3. A holding page (`isHoldingSlug`) keeps its newest `LARGE_PAGE_VERSIONS_KEPT` inline versions whatever their size, at most `HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE` (100) removed per write (GST-2497). `LARGE_PAGE_VERSIONS_KEPT` lives in `page-body/limits.ts`.- `src/core/progressive-batch/` — shared ramp-up + cost-cap + verification primitive (trial 10 → ramp 100 → ramp 500 → full, with verification at each stage), with verifier+policy injection (callers describe HOW TO MEASURE SUCCESS, not WHEN TO WAIT FOR CTRL-C). Modules: `types.ts` (Stage, StageVerdict, AbortReason, discriminated `Verifier` union `OutputCountVerifier | IdempotentMutationVerifier | NoopVerifier`, Policy, StageReport), `orchestrator.ts` (`runProgressiveBatch(items, verifier, policy, runner)` — reads `getCurrentBudgetTracker()` ahead of `Policy.maxCostUsd` fail-closed; null both ways triggers `abort_cost_cap reason='no_budget_safety_net'`), `audit.ts` (ISO-week JSONL at `~/.gbrain/audit/progressive-batch-YYYY-Www.jsonl` via the shared `audit-writer` primitive), `stage-report.ts` (ASCII formatter for the default `Policy.onStageReport`). Env knobs: `GBRAIN_PROGRESSIVE_BATCH_DISABLED=1`, `GBRAIN_PROGRESSIVE_BATCH_AUTO=1` (skip Ctrl-C grace), `GBRAIN_PROGRESSIVE_BATCH_STAGES=10,100,500`. Sites that "jump straight to full" stay that way by default; ramp is opt-in per-site via `Policy.interactiveAbortMs > 0`. Pinned by `test/progressive-batch/orchestrator.test.ts` (35 cases, every verdict path).diff --git a/docs/architecture/key-files/graph-and-facts.md b/docs/architecture/key-files/graph-and-facts.mdindex 310be39542..32c5a89c9f 100644--- a/docs/architecture/key-files/graph-and-facts.md+++ b/docs/architecture/key-files/graph-and-facts.md@@ -98,7 +98,7 @@ rejection of serve-only flags before a brain opens.- `src/core/extraction-review.ts` — Extraction quarantine lane markers, sibling of `src/core/quarantine.ts` / `embed-skip.ts` (frontmatter-key pattern, no schema migration). Auto-extracted stubs from untrusted input carry the PAIR `provenance: 'auto-extracted'` + `status: 'unverified'` (both required — user pages with their own `status`/`provenance` never match). Exports `quarantineMarkers()`, `isUnverifiedExtraction()` (JS predicate) and `unverifiedExtractionFragment(alias)` — the single SQL source of truth consumed by `buildSourceFactorCase` (namespace source-boost guard), both engines' `getUnverifiedExtractionPageIds`, the `extraction_pending` op, and the `unverified_extractions` doctor check, so filter and marker keys can never drift. Consequences: unverified stubs are excluded from the compiled-truth fusion boost + the `people/`/`companies/` source-boost (rank as ordinary content), stamped `unverified: true` in search results (`stampUnverifiedExtractions`, hybrid.ts), listed by `extraction_pending`, promoted (status → `verified`, provenance kept for audit) or rejected (soft-delete) by the owner-only `extraction_review` op, which also records `distinct` decisions (`distinct_from`); every decision publishes through the page door (Git first on a repository source, the database on a repo-less one) under `persistence/publication-guard.ts`, with the revision it read. Pinned by `test/extraction-review.test.ts` (PGLite) + `test/e2e/extraction-review-postgres.test.ts` (live Postgres parity).-- `src/core/facts-fence.ts` — the `## Facts` fence primitives: parse (`parseFactsFence`; #6385: cells are read by the column the header names (`factsHeaderLayout`, `FACTS_FENCE_COLUMNS`), so an unknown, repeated or missing required header column is one `FACTS_TABLE_MALFORMED` and no rows, a header in canonical order keeps the positional typed tail, and `factsRowProblem` lets a row leave off only trailing optional columns and refuses a non-empty cell past the layout; only a struck row reads `superseded by #N` (`supersededBy`) or `forgotten:` from its context cell, so on a live row that text is plain context (#9069); a confidence cell outside 0..1 or not a plain number is `FACTS_TABLE_MALFORMED`, a `claim_value` cell parses strictly (plain/scientific numbers, `1,234,567` separators, a leading currency symbol, a k/M/B suffix that scales; anything else such as `1,5` or `0.9abc` is `FACTS_TABLE_MALFORMED`), `formatFenceDate` writes UTC-midnight dates as `YYYY-MM-DD` and any other instant as a UTC timestamp so TTLs and write-time `valid_from` round-trip, so the page is non-authoritative instead of failing the DB CHECK), render (`renderFactsTable`), and strip (`stripFactsFence({keepVisibility})` — the remote-read privacy boundary `get_page`/`fetch_page`/the chunker apply), plus the remote write-back merge, plus the ONE fence-placement rule: `replaceOrInsertFactsFence(body, fenceBlock)` replaces an existing fence in place or inserts a fresh `## Facts` section ABOVE the timeline sentinel (`timelineSentinelOffset`, #4756 — below it splitBody() files the fence into page.timeline where extract_facts refuses to reconcile it), EOF only when the page has no sentinel; every writer that materializes a fence (`upsertFactRow`, the phantom-redirect canonical append, the importer's hidden-row merge) routes through it, so no writer can re-grow its own EOF append. `upsertFactRow` throws `FenceRewriteRefusal` (fence-shared.ts) when the fence it starts from parses with warnings (#6385 R12): it re-renders the parsed rows, so a rewrite would delete the rows the parser skipped. `restoreHiddenFactRows(incoming, existing)` is the row-level, vis... (line truncated to 2000 chars) GST-2492: `parseFactsFenceFor(body, { slug, reader })` reads the optional `entity` column only on `memory/unattributed` (`HOLDING_FACTS_COLUMNS`; a cell must be a page slug other than the holding page, else `FACTS_TABLE_MALFORMED`), and `reader: true` tolerates it where only the body is known (withdrawal overlay, remote strip); `parseFactsFence(body)` and every other page parse as before.+- `src/core/facts-fence.ts` — the `## Facts` fence primitives: parse (`parseFactsFence`; #6385: cells are read by the column the header names (`factsHeaderLayout`, `FACTS_FENCE_COLUMNS`), so an unknown, repeated or missing required header column is one `FACTS_TABLE_MALFORMED` and no rows, a header in canonical order keeps the positional typed tail, and `factsRowProblem` lets a row leave off only trailing optional columns and refuses a non-empty cell past the layout; only a struck row reads `superseded by #N` (`supersededBy`) or `forgotten:` from its context cell, so on a live row that text is plain context (#9069); a confidence cell outside 0..1 or not a plain number is `FACTS_TABLE_MALFORMED`, a `claim_value` cell parses strictly (plain/scientific numbers, `1,234,567` separators, a leading currency symbol, a k/M/B suffix that scales; anything else such as `1,5` or `0.9abc` is `FACTS_TABLE_MALFORMED`), `formatFenceDate` writes UTC-midnight dates as `YYYY-MM-DD` and any other instant as a UTC timestamp so TTLs and write-time `valid_from` round-trip, so the page is non-authoritative instead of failing the DB CHECK), render (`renderFactsTable`), and strip (`stripFactsFence({keepVisibility})` — the remote-read privacy boundary `get_page`/`fetch_page`/the chunker apply), plus the remote write-back merge, plus the ONE fence-placement rule: `replaceOrInsertFactsFence(body, fenceBlock)` replaces an existing fence in place or inserts a fresh `## Facts` section ABOVE the timeline sentinel (`timelineSentinelOffset`, #4756 — below it splitBody() files the fence into page.timeline where extract_facts refuses to reconcile it), EOF only when the page has no sentinel; every writer that materializes a fence (`upsertFactRow`, the phantom-redirect canonical append, the importer's hidden-row merge) routes through it, so no writer can re-grow its own EOF append. `upsertFactRow` throws `FenceRewriteRefusal` (fence-shared.ts) when the fence it starts from parses with warnings (#6385 R12): it re-renders the parsed rows, so a rewrite would delete the rows the parser skipped. `restoreHiddenFactRows(incoming, existing)` is the row-level, vis... (line truncated to 2000 chars) GST-2492: `parseFactsFenceFor(body, { slug, reader })` reads the optional `entity` column only on `memory/unattributed` (`HOLDING_FACTS_COLUMNS`; a cell must be a page slug other than the holding page, else `FACTS_TABLE_MALFORMED`), and `reader: true` tolerates it where only the body is known (withdrawal overlay, remote strip); `parseFactsFence(body)` and every other page parse as before. GST-2497: a table's parse is kept by its text, whether the `entity` column is read and the folder holding page's slug (the newest 8), and every caller gets its own copy, so one write parses a table once however many readers see it; `factsFenceTableParses()` counts real parses for tests.- `src/core/facts/attribution.ts` — speaker attribution's one dedup rule: `attributionCompatible(a, b)` is false only for two different known speakers (`user`, `assistant`, `other`); NULL (no recorded speaker) is compatible with either. Every dedup path applies it before candidate limits: `findCandidateDuplicates` (`attributedTo` option, SQL pre-limit), `decideSingleFact` exact and cosine matches, capture exact/near matches, hot-memory collapse (`collapseHotFacts` splits a fingerprint group by speaker when it holds two), fence reindex (`duplicateActiveFenceRows`) and the managed batch key (`facts-prepare.ts`). `facts.attributed_to` (migration v215) is written by the extractor (`facts.attribution`, default on per `docs/eval/decisions/p2-e3-heldout/`; `false` opts out), carried in the fence's 15th cell (only on rows that have it), returned by `recall` and hot memory, rendered `(assistant said)` in context packs, and kept out of `consolidate` (takes are the user's own). Pinned by `test/facts/speaker-attribution.test.ts`.- `src/core/fence-shared.ts` — shared pipe-table primitives for the `## Takes` (`takes-fence.ts`) and `## Facts` (`facts-fence.ts`) fences: `parseRowCells`, `isSeparatorRow`, `stripStrikethrough`, `parseStringCell`, `escapeFenceCell`, and the rewrite guard (#6385 R12) `assertFenceRewritable` / `FenceRewriteRefusal` that `upsertFactRow`, `upsertTakeRow`, `supersedeRow` and `mergePhantomFenceRows` throw when their input fence parses with warnings (the message names the fence only; the warnings, which embed row text, stay on the error). `parseRowCells` is escape-aware: `\|` stays inside its cell and decodes back to a literal `|` (exact inverse of `escapeFenceCell`), while any other backslash passes through verbatim so existing fence text (e.g. Windows paths) is byte-stable across a render → parse cycle. This matters because the fence is canonical and reconcile is wipe-and-reinsert — a pipe inside a cell shifting the column layout would corrupt every row behind it on the next reconcile. `escapeFenceCell` also encodes fence-marker text (`gbrain:facts:begin` and the three siblings) by lengthening the ampersand run before `#58;` (`gbrain:X` → `gbrain:X`, a literal `gbrain:X` → `gbrain&#58;X`), and `decodeFenceCell` (the decoder `parseRowCells` and `fence-repair/raw-rows.ts` share) removes exactly one level, so a claim that quotes a marker never becomes a live marker and a literal entity round-trips byte-identically; the lexical fail-closed privacy rule is unchanged. Pinned by the escape cases in `test/facts-fence.test.ts` + the full render → parse → reconcile round-trip in `test/e2e/facts-fence-reconcile-postgres.test.ts`.diff --git a/docs/guides/page-history.md b/docs/guides/page-history.mdindex 1eb37ce667..bf0b6837f8 100644--- a/docs/guides/page-history.md+++ b/docs/guides/page-history.md@@ -114,7 +114,7 @@ A trusted local caller sees every stored reason. A remote caller (an MCP client)sees a version's reason only when both that version and the version thatreplaced it are world-visible. The reason for turning a page private, or forany change made while it was private, is therefore never shown remotely. When-the replacing version was later pruned (large pages keep a few versions), its+the replacing version was later pruned (large pages and holding pages keep a few versions), itsvisibility can no longer be checked and the reason is withheld from remotecallers. Remote reasons go through the same text sanitizer as version bodies.diff --git a/scripts/core-world-reads-baseline.tsv b/scripts/core-world-reads-baseline.tsvindex 98efb2e9ee..9b227b7b6b 100644--- a/scripts/core-world-reads-baseline.tsv+++ b/scripts/core-world-reads-baseline.tsv@@ -378,6 +378,7 @@ src/core/extract-takes-from-pages.ts fs_import 1 W7: filesystem boundary moves tsrc/core/extract-takes-from-pages.ts load_config 1 W6: settings plane out of the coresrc/core/extract/rollup-writer.ts module_collection 1 W6: per-engine state, or mark as a constant tablesrc/core/extraction-lag.ts process_env 2 W6: settings plane out of the core+src/core/facts-fence.ts module_collection 1 stays (GST-2497): a bounded memo of fence-table parses keyed by the table text and parse options, returning copies; it holds no setting or brain statesrc/core/facts/backstop.ts fs_import 1 W7: filesystem boundary moves to the local appsrc/core/facts/backstop.ts load_config 1 W6: settings plane out of the coresrc/core/facts/backstop.ts module_collection 1 W6: per-engine state, or mark as a constant tablediff --git a/src/core/facts-fence.ts b/src/core/facts-fence.tsindex 42636dc200..bfc36c87f3 100644--- a/src/core/facts-fence.ts+++ b/src/core/facts-fence.ts@@ -270,6 +270,20 @@ export function parseFactsFence(body: string): FactsFenceParseResult {return parseFactsFenceFor(body);}+/**+ * GST-2497: one write reads the same table many times (the withdrawal overlay, the chunker, the remote-read strip, the+ * refusal scan). A table's parse depends only on its text, whether the `entity` column is read and which folder's+ * entities a folder holding page may hold, so the newest few parses are kept under that key (at most 8, and 16M+ * characters of table text) and every caller gets its own copy.+ */+const PARSE_CACHE_LIMIT = 8;+const PARSE_CACHE_MAX_CHARS = 16 * 1024 * 1024;+const parseCache = new Map<string, FactsFenceParseResult>();+let cachedChars = 0;+let tableParses = 0;+/** Test seam: how many fence tables parseFactsFenceFor has parsed in this process, cached reads excluded. */+export const factsFenceTableParses = (): number => tableParses;+/** parseFactsFence for a known page: `memory/unattributed` also reads its `entity` column (GST-2492). */export function parseFactsFenceFor(body: string, opts?: FactsFenceParseOpts): FactsFenceParseResult {const { beginIdx, endIdx } = locateOutsideCode(body, FACTS_FENCE_BEGIN, FACTS_FENCE_END);@@ -284,8 +298,25 @@ export function parseFactsFenceFor(body: string, opts?: FactsFenceParseOpts): Fawarnings.push('FACTS_FENCE_UNBALANCED: end marker before begin');return { facts: [], warnings };}-const inner = body.slice(beginIdx + FACTS_FENCE_BEGIN.length, endIdx);+ const folder = isHoldingSlug(opts?.slug) && opts?.slug !== UNATTRIBUTED_MEMORY_SLUG ? opts!.slug : '';+ const key = `${readsEntity(opts) ? 1 : 0}\0${folder}\0${inner}`;+ const hit = parseCache.get(key);+ const parsed = hit ?? parseFactsTable(inner, opts);+ if (hit) parseCache.delete(key);+ else cachedChars += key.length;+ parseCache.set(key, parsed);+ for (const old of parseCache.keys()) {+ if (parseCache.size <= PARSE_CACHE_LIMIT && cachedChars <= PARSE_CACHE_MAX_CHARS) break;+ parseCache.delete(old);+ cachedChars -= old.length;+ }+ return { facts: parsed.facts.map(fact => ({ ...fact })), warnings: [...parsed.warnings] };+}++function parseFactsTable(inner: string, opts?: FactsFenceParseOpts): FactsFenceParseResult {+ tableParses++;+ const warnings: string[] = [];const lines = inner.split('\n');const facts: ParsedFact[] = [];let sawHeader = false;diff --git a/src/core/page-state/versions.ts b/src/core/page-state/versions.tsindex 3fc2b6d32e..d68a2eb6d7 100644--- a/src/core/page-state/versions.ts+++ b/src/core/page-state/versions.ts@@ -5,8 +5,11 @@ import { PAGE_INLINE_MAX_BYTES, isLargePageBytes } from '../page-size-policy.ts'import { LARGE_PAGE_VERSIONS_KEPT, readPageBodyPolicy } from '../page-body/limits.ts';import { opError, type OperationError } from '../ops/contract.ts';import { isSegmentedPage } from '../page-body/page-shape.ts';+import { isHoldingSlug } from '../facts-fence.ts';export { LARGE_PAGE_VERSIONS_KEPT };+/** GST-2497: the most saved versions of a holding page one write removes, so a page with a long history drains over later saves. */+export const HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE = 100;/*** GBRA-217: what a sync import knows about the write a snapshot records: the reason Git gives for it@@ -35,6 +38,12 @@ export async function createPageVersion(engine: BrainEngine, slug: string, sourc[page.id, PAGE_INLINE_MAX_BYTES, LARGE_PAGE_VERSIONS_KEPT]);await markPrunedHistory(tx, page.id, pruned.map(row => Number(row.id)));}+ if (isHoldingSlug(page.slug)) {+ const pruned = await tx.executeRaw<{ id: number }>(`DELETE FROM page_versions WHERE id IN (SELECT id FROM (SELECT id FROM page_versions+ WHERE page_id = $1 AND body_storage IS DISTINCT FROM 'segmented' ORDER BY id DESC OFFSET $2) excess ORDER BY id LIMIT $3) RETURNING id`,+ [page.id, LARGE_PAGE_VERSIONS_KEPT, HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE]);+ await markPrunedHistory(tx, page.id, pruned.map(row => Number(row.id)));+ }return { ...rows[0], snapshot_at: new Date(rows[0].snapshot_at) };};if (preimage) return insert(engine, preimage);diff --git a/test/holding-page-saves-gst2497.test.ts b/test/holding-page-saves-gst2497.test.tsnew file mode 100644index 0000000000..1caa5edea0--- /dev/null+++ b/test/holding-page-saves-gst2497.test.ts@@ -0,0 +1,236 @@+/**+ * GST-2497: saving a fact to a holding page (memory/unattributed, or a folder's <folder>/memory/unattributed).+ *+ * 1. Protects: one save parses the holding page's facts table a bounded number of times, however many readers of the+ * body run during the write (the withdrawal overlay, the chunker, the remote-read strip, the refusal scan).+ * 2. Fails when: a save on a page of a few hundred rows parses the full table more than three times.+ * 3. Existing coverage: test/facts-fence.test.ts (the parser), test/remember-strict-held-gst2492.test.ts (the shared page).+ * 4. Real seams: the coordinated write and the write door on PGLite and Postgres (DATABASE_URL), a managed Git checkout,+ * the operation handler as the local CLI calls it, and the parser's own count of the tables it parsed.+ */+import { describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { readFileSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { operationsByName } from '../src/core/operations.ts';+import { performSync } from '../src/commands/sync.ts';+import { runExtractFacts } from '../src/core/cycle/extract-facts.ts';+import { factsFenceTableParses, parseFactsFenceFor } from '../src/core/facts-fence.ts';+import { proposeRevision } from '../src/core/access/policy.ts';+import type { BrainEngine } from '../src/core/engine.ts';+import { isolatedSharedSkillsEngine } from './helpers/shared-skills-engine.ts';+import { resolveRepairScope, type RepairResult } from '../src/core/repair/core.ts';+import { repairRunner } from '../src/core/repair/registry.ts';+import { publishFenceGeneration, readFenceBlocks, FACT_FENCE_KIND } from '../src/core/generations/fence.ts';+import { factFenceItems, factPreimages, factRetireItem } from '../src/core/generations/adapters/fact-rows.ts';+import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { durableGitRepo, git } from './helpers/git-publication.ts';+import { managedBrain, settleManagedGitEffects, type ManagedBrain } from './helpers/managed-brain.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';++const HOLD = 'memory/unattributed';+const ENTITY = 'partners/alice-example/deal-notes';+const FOLDER_HOLD = 'partners/alice-example/memory/unattributed';+const gitRepo = ({ root }: { root: string }) => { writeFileSync(join(root, '.gitignore'), '\n'); durableGitRepo(root, ['.gitignore']); };+const call = (ctx: OperationContext, name: string, params: Record<string, unknown>) =>+ operationsByName[name]!.handler(ctx, { request_id: randomUUID(), ...params }) as Promise<Record<string, any>>;+const remember = (ctx: OperationContext, fact: string, entity?: string) => call(ctx, 'remember', { fact, provenance: 'chat 2026-11-13', ...(entity ? { entity } : {}) });+const claims = (n: number) => Array.from({ length: n }, (_, i) => `The pantry inventory note number ${i + 1} is recorded.`);+const versionCount = async (ctx: OperationContext, slug: string) => (await call(ctx, 'get_versions', { slug, limit: 100 }) as unknown as unknown[]).length;+const storedVersions = async (engine: BrainEngine, slug: string) => Number((await engine.executeRaw<{ n: number }>(+ `SELECT count(*)::int AS n FROM page_versions v JOIN pages p ON p.id = v.page_id WHERE p.source_id = 'default' AND p.slug = $1`, [slug]))[0]!.n);+const gapSet = async (engine: BrainEngine, slug: string) => (await engine.executeRaw<{ gap: boolean }>(+ `SELECT version_history_gap_at IS NOT NULL AS gap FROM pages WHERE source_id = 'default' AND slug = $1`, [slug]))[0]!.gap;+const activeRows = (engine: BrainEngine, slug: string) => engine.executeRaw<{ fact: string }>(+ `SELECT fact FROM facts WHERE source_id = 'default' AND source_markdown_slug = $1 AND expired_at IS NULL ORDER BY row_num`, [slug]).then(rows => rows.map(r => r.fact));+const fenceClaims = async (engine: BrainEngine, slug: string) => {+ const page = await engine.getPage(slug, { sourceId: 'default' });+ return parseFactsFenceFor(page?.compiled_truth ?? '', { slug }).facts.filter(f => f.active).map(f => f.claim);+};++type Listed = RepairResult & { listing?: Array<{ item: string; class: string; reason?: string }> };+const quiet = { info: () => {}, warn: () => {}, error: () => {} };+async function repairUnwithdrawn(engine: BrainEngine, opts: { expect?: string; includeAmbiguous?: boolean } = {}): Promise<Listed> {+ const scope = await resolveRepairScope(engine, 'default');+ return (await repairRunner(engine, { apply: opts.expect !== undefined, noEmbed: true, logger: quiet })).run('unwithdrawn-facts', scope,+ { explicit: true, sourceFlag: 'default', ...opts }) as Promise<Listed>;+}+const previewHashOf = (result: RepairResult) => result.apply_command.match(/--expect ([0-9a-f]+)/)![1]!;++/**+ * The #8819 shape on a holding page: a numbered row no fence of the page ever held, retired by a facts-fence generation+ * of the page (recorded with its before-state, as the pre-fix projection did) and expired in that transaction.+ */+async function wronglyRetired(engine: BrainEngine, slug: string, claim: string): Promise<number> {+ await disposePersistenceConsumer(engine);+ await engine.executeRaw('UPDATE persistence_brain SET enabled=false WHERE singleton=1');+ try {+ const [row] = await engine.executeRaw<{ id: number }>(`INSERT INTO facts (source_id, fact, kind, visibility, source, row_num, source_markdown_slug)+ VALUES ('default', $1, 'fact', 'private', 'mcp:put_page', 999, $2) RETURNING id`, [claim, slug]);+ const id = Number(row!.id);+ await engine.transaction(async tx => {+ const page = (await tx.readPageSnapshot(slug, { sourceId: 'default' }))!.page;+ const held = parseFactsFenceFor(page.compiled_truth, { slug }).facts.map(f => ({ row_num: f.rowNum, fact: f.claim, visibility: f.visibility }));+ await publishFenceGeneration(tx, { kind: FACT_FENCE_KIND, sourceId: 'default', slug, read: readFenceBlocks(page, 'facts'), items: factFenceItems(held),+ retire: (await factPreimages(tx, 'default', [id])).map(factRetireItem), writer: 'projection' });+ await tx.executeRaw('UPDATE facts SET expired_at=now(), row_num=NULL WHERE id=$1', [id]);+ });+ return id;+ } finally { await engine.executeRaw('UPDATE persistence_brain SET enabled=true WHERE singleton=1'); }+}+const factState = async (engine: BrainEngine, id: number) => (await engine.executeRaw<{ row_num: number | null; active: boolean }>(+ 'SELECT row_num, expired_at IS NULL AS active FROM facts WHERE id=$1', [id])).map(r => [r.row_num === null ? null : Number(r.row_num), r.active])[0];++/** The holding page grown to `rows` rows by a commit and a sync, as a page an owner has saved to for a while. */+async function grownHoldingPage({ engine, ctx, root }: ManagedBrain, rows: number): Promise<void> {+ await remember(ctx, 'Seed fact number 1 about the garden plan.');+ await settleManagedGitEffects(engine);+ const file = join(root, `${HOLD}.md`);+ const lines = readFileSync(file, 'utf8').split('\n');+ const at = lines.findIndex(line => line.includes('Seed fact number 1 '));+ const extra = Array.from({ length: rows - 1 }, (_, i) => lines[at]!.replace(/^\|\s*1\s*\|/, `| ${i + 2} |`).replace('number 1 ', `number ${i + 2} `));+ lines.splice(at + 1, 0, ...extra);+ writeFileSync(file, lines.join('\n'));+ git(root, 'add', '-A'); git(root, 'commit', '-q', '-m', 'grow the holding page');+ await performSync(engine, { sourceId: 'default', repoPath: root, noPull: true, noEmbed: true } as never);+ await runExtractFacts(engine, { sourceId: 'default' });+ await settleManagedGitEffects(engine);+}++describe('a parsed facts table is reused only as a copy, and only for the same columns and folder (GST-2497)', () => {+ const table = (entity: string) => ['## Facts', '', '<!--- gbrain:facts:begin -->',+ '| # | claim | kind | confidence | visibility | notability | valid_from | valid_until | source | context | entity |',+ '|---|---|---|---|---|---|---|---|---|---|---|', `| 1 | The deal closes in May. | fact | 1.0 | world | medium | 2026-11-12 | | chat | | ${entity} |`, '<!--- gbrain:facts:end -->', ''].join('\n');+ test('a caller that edits its result changes nothing another caller reads', () => {+ const body = table(ENTITY);+ const first = parseFactsFenceFor(body, { slug: FOLDER_HOLD });+ first.facts[0]!.claim = 'edited'; first.facts.push({ ...first.facts[0]! }); first.warnings.push('edited');+ const before = factsFenceTableParses();+ expect(parseFactsFenceFor(`Prose above.\n\n${body}`, { slug: FOLDER_HOLD })).toMatchObject({ warnings: [], facts: [{ claim: 'The deal closes in May.', entity: ENTITY }] });+ expect(factsFenceTableParses() - before).toBe(0);+ });+ test('the same table parses by its page: the entity column, and which folder\'s entities a folder holding page holds', () => {+ const body = table(ENTITY);+ expect(parseFactsFenceFor(body, { slug: FOLDER_HOLD }).facts).toHaveLength(1);+ expect(parseFactsFenceFor(body, { slug: 'clients/memory/unattributed' })).toMatchObject({ facts: [], warnings: [expect.stringContaining('invalid entity')] });+ expect(parseFactsFenceFor(body, { slug: 'people/alice-example' })).toMatchObject({ facts: [], warnings: [expect.stringContaining('unsupported header column "entity"')] });+ expect(parseFactsFenceFor(body, { reader: true }).facts).toHaveLength(1);+ expect(parseFactsFenceFor(body, { slug: HOLD }).facts).toHaveLength(1);+ });+});++for (const backend of testBackends()) describe(`[${backend}] saves to a holding page (GST-2497)`, () => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ const strict = (run: (brain: ManagedBrain) => Promise<void>) => managedBrain(async brain => {+ await brain.engine.executeRaw(`UPDATE sources SET config = config || '{"canonical":"git"}'::jsonb WHERE id='default'`);+ await run(brain);+ }, { databaseUrl, setup: gitRepo });++ test('one save to a holding page of 300 rows parses its facts table at most three times', () => strict(async brain => {+ await grownHoldingPage(brain, 300);+ const before = factsFenceTableParses();+ expect(await remember(brain.ctx, 'The pantry holds three jars of honey.')).toMatchObject({ status: 'inserted', page: HOLD });+ await settleManagedGitEffects(brain.engine);+ const parses = factsFenceTableParses() - before;+ expect(parses).toBeGreaterThan(0);+ expect(parses).toBeLessThanOrEqual(3);+ expect(readFileSync(join(brain.root, `${HOLD}.md`), 'utf8')).toContain('The pantry holds three jars of honey.');+ }), 300_000);++ test('eight saves to memory/unattributed keep its newest five saved versions, every row on the page, its fence and its facts, and mark the gap', () => strict(async ({ engine, ctx, root }) => {+ for (const claim of claims(8)) expect(await remember(ctx, claim)).toMatchObject({ status: 'inserted', page: HOLD });+ await settleManagedGitEffects(engine);+ expect(await versionCount(ctx, HOLD)).toBe(5);+ expect(await fenceClaims(engine, HOLD)).toEqual(claims(8));+ const file = readFileSync(join(root, `${HOLD}.md`), 'utf8');+ for (const claim of claims(8)) expect(file).toContain(claim);+ expect(await activeRows(engine, HOLD)).toEqual(claims(8));+ expect(await gapSet(engine, HOLD)).toBe(true);+ expect(git(root, 'log', '--format=%H', '--', `${HOLD}.md`).trim().split('\n').length).toBeGreaterThanOrEqual(8);+ }), 300_000);++ test('a folder holding page keeps its newest five saved versions the same way', () => strict(async ({ engine, ctx }) => {+ await proposeRevision(engine, 'default', [{ id: 'partners', prefix: 'partners/', readable_by: ['member:alice-example'] }], { origin: 'cli', store: 'database', approved: false });+ for (const claim of claims(8)) expect(await remember(ctx, claim, ENTITY)).toMatchObject({ status: 'inserted', page: FOLDER_HOLD });+ await settleManagedGitEffects(engine);+ expect(await versionCount(ctx, FOLDER_HOLD)).toBe(5);+ expect(await fenceClaims(engine, FOLDER_HOLD)).toEqual(claims(8));+ expect(await activeRows(engine, FOLDER_HOLD)).toEqual(claims(8));+ expect(await gapSet(engine, FOLDER_HOLD)).toBe(true);+ }), 300_000);++ test('an ordinary entity page written eight times keeps every saved version', () => strict(async ({ engine, ctx }) => {+ const slug = 'people/alice-example';+ for (let i = 1; i <= 8; i++) {+ const prior = await engine.readPageSnapshot(slug, { sourceId: 'default' });+ await call(ctx, 'put_page', { slug, content: `---\ntype: person\ntitle: Alice Example\n---\n\nEdit number ${i}.\n`, ...(prior ? { expected_revision: prior.revision } : {}) });+ }+ await settleManagedGitEffects(engine);+ expect(await versionCount(ctx, slug)).toBe(7);+ expect(await gapSet(engine, slug)).toBe(false);+ }), 300_000);++ test('a holding page with 250 saved versions sheds at most 100 per save until five are left', () => strict(async ({ engine, ctx }) => {+ await remember(ctx, claims(1)[0]!);+ await settleManagedGitEffects(engine);+ await engine.executeRaw(`INSERT INTO page_versions (page_id, compiled_truth, frontmatter, timeline, title, type, tags)+ SELECT p.id, p.compiled_truth, p.frontmatter, p.timeline, p.title, p.type, '[]'::jsonb FROM pages p, generate_series(1, 250)+ WHERE p.source_id = 'default' AND p.slug = $1`, [HOLD]);+ const left: number[] = [];+ for (const claim of claims(4).slice(1)) { await remember(ctx, claim); left.push(await storedVersions(engine, HOLD)); }+ expect(left).toEqual([151, 52, 5]);+ expect(await activeRows(engine, HOLD)).toEqual(claims(4));+ }), 300_000);+});++for (const backend of testBackends()) describe(`[${backend}] gbrain repair unwithdrawn-facts on a pruned holding page (GST-2497)`, () => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ test('a row wrongly retired on a holding page whose versions were pruned is ambiguous, and --include-ambiguous with that preview\'s hash restores it', () => managedBrain(async ({ engine, ctx }) => {+ await engine.executeRaw(`UPDATE sources SET config = config || '{"canonical":"git"}'::jsonb WHERE id='default'`);+ for (const claim of claims(8)) await remember(ctx, claim);+ await settleManagedGitEffects(engine);+ expect(await gapSet(engine, HOLD)).toBe(true);+ const id = await wronglyRetired(engine, HOLD, 'The pantry was restocked in October.');+ const item = `default:${HOLD}#${id}`;+ const plain = await repairUnwithdrawn(engine);+ expect(plain.listing!.map(row => [row.item, row.class])).toEqual([[item, 'ambiguous']]);+ expect(JSON.stringify(plain)).toContain('version history has a gap');+ const wide = await repairUnwithdrawn(engine, { includeAmbiguous: true });+ expect(wide.listing!.map(row => [row.item, row.class])).toEqual([[item, 'ambiguous']]);+ await expect(repairUnwithdrawn(engine, { expect: previewHashOf(wide) })).rejects.toMatchObject({ code: 'preview_changed' });+ expect(await factState(engine, id)).toEqual([null, false]);+ expect((await repairUnwithdrawn(engine, { expect: previewHashOf(wide), includeAmbiguous: true })).applied).toBe(1);+ expect(await factState(engine, id)).toEqual([999, true]);+ await remember(ctx, 'One more save after the restore.');+ await settleManagedGitEffects(engine);+ expect(await factState(engine, id)).toEqual([999, true]);+ expect(await activeRows(engine, HOLD)).toEqual([...claims(8), 'The pantry was restocked in October.', 'One more save after the restore.']);+ }, { databaseUrl, setup: gitRepo }), 300_000);+});++describe('a holding page on a brain with no Git repository (GST-2497)', () => {+ for (const backend of testBackends()) test(`[${backend}] eight saves keep five versions, the facts rows rebuild the table as it was after the third save, and an as_of read says history is incomplete`, async () => {+ const { engine, close } = await isolatedSharedSkillsEngine(backend === 'postgres' ? requirePostgresTestDatabase() : undefined);+ try {+ const ctx = { engine, config: { engine: engine.kind, embedding_disabled: true }, sourceId: 'default', remote: false, dryRun: false, logger: { info() {}, warn() {}, error() {} } } as OperationContext;+ let third: Array<[number, string]> = [], at = '';+ for (const [i, claim] of claims(8).entries()) {+ expect(await remember(ctx, claim)).toMatchObject({ status: 'inserted' });+ if (i === 0) await engine.executeRaw(`UPDATE pages SET frontmatter = frontmatter || '{"valid_from":"2026-01-01"}'::jsonb WHERE source_id = 'default' AND slug = $1`, [HOLD]);+ if (i !== 2) continue;+ const page = await engine.getPage(HOLD, { sourceId: 'default' });+ third = parseFactsFenceFor(page!.compiled_truth, { slug: HOLD }).facts.filter(f => f.active).map(f => [f.rowNum, f.claim]);+ at = String((await engine.executeRaw<{ at: string }>('SELECT clock_timestamp()::text AS at'))[0]!.at);+ }+ expect(third.map(([, claim]) => claim)).toEqual(claims(3));+ expect(await versionCount(ctx, HOLD)).toBe(5);+ expect(await fenceClaims(engine, HOLD)).toEqual(claims(8));+ const rebuilt = await engine.executeRaw<{ row_num: number; fact: string }>(`SELECT row_num, fact FROM facts WHERE source_id = 'default' AND source_markdown_slug = $1+ AND created_at <= $2::timestamptz AND (expired_at IS NULL OR expired_at > $2::timestamptz) ORDER BY row_num`, [HOLD, at]);+ expect(rebuilt.map(r => [Number(r.row_num), r.fact])).toEqual(third);+ const before = await call(ctx, 'get_page', { slug: HOLD, as_of: '2025-06-01' });+ expect(before.valid_time).toMatchObject({ status: 'before_recorded_history', history: 'incomplete', uncertain: true, why: 'history_gap' });+ } finally { await close(); }+ }, 300_000);+});