Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Prune a holding page's saved versions only on a save that adds a fact

garrytan0cd09200608c
diff --git a/docs/architecture/key-files/core-persistence-facts.md b/docs/architecture/key-files/core-persistence-facts.md
index ef99c94ffa..8733c40dc7 100644
--- a/docs/architecture/key-files/core-persistence-facts.md
+++ b/docs/architecture/key-files/core-persistence-facts.md
@@ -22,6 +22,6 @@ Managed facts extraction: the entity-intent preparer and publisher, conversation
- `src/commands/doctor/checks/conversation-coverage.ts` — `conversation_format_coverage`: samples up to 50 newest pages per concrete conversation type, the list the extractor reads (`pageTypesForAllowed(ALLOWED_TYPES)`, granular collector aliases such as `slack-thread` included, #8469), keeps the pages `isConversationFactsEligiblePage` admits, parses each without polish or fallback and warns above 10% unmatched transcript-claiming pages. Test: `test/doctor-conversation-granular-coverage-8469.test.ts`.
- `src/core/repair/fact-claim-dates.ts` — `gbrain repair fact-claim-dates --source <id> [--day YYYY-MM-DD]` (#7247), explicit-only, preview-bound, destructive (`--yes`). Candidates: active facts on a bulk day (`findBulkFactDays`, or `--day`) whose claim has exactly one earlier ISO date (`claimDate`). A fence row's `valid_from` cell is rewritten in the page's one fence section and published by revision-bound `put_page` (the facts row is re-projected); a database-only fact is updated in one `maintenanceTransaction` on an unmanaged brain and counted `database_only_managed` on a managed one. Pages with an unparsed, split or segmented fence, or a rewrite that does not read back as the planned rows, are listed and kept. The hash binds revisions, row numbers and new dates; a fresh apply recomputes the plan and refuses `preview_changed` on any difference. Test: `test/repair-fact-claim-dates-7247.test.ts`.
-- `src/core/facts/fence-fact-append.ts` — `prepareFenceFactAppend(engine, row, config, snapshot, { fact, pendingTier, mode?, supersede? | rowNum }, signal)`: the one way a coordinator preparer puts a single fact into the facts fence of the request's page (`row.slug`); remember uses it, and so does a managed owner release of a held fact (`trust/release-publish.ts`, the `trust_release` operation: append for a held fact with an entity slug, index-existing for a hold naming `payload.row_num`, whose cell fields the owner previewed and the request binds: a moved cell refuses `preview_changed`; journaled as mode `index-previewed`, which a preparer built before the field preview refuses, and an `index-existing` request with no fields digest refuses `preview_changed` asking for a rerun). It returns `{ prepared, rowNum, fencesNormalized, apply(tx) }`: `prepared` is the page write to compose (its `file`, `exclusiveSources`, `validate`; null in index-existing), and `apply` runs in the publication transaction after the coordinator published the file, writing the page row (tier kept, `trust/fence-append.ts`) and then the facts row with `row_num` and `source_markdown_slug`, so a facts row never precedes its fence row in Git. `append` (default) normalizes the stored fence (#6188), renders the row with `fenceRowOf` (`unfenced-facts.ts`, the one renderer) at one past the highest row number in the fence and among the page's facts rows (#6696 C9), strikes `supersede.rowNum` when given, and prepares the page write `unprojected` (`preparePageMutation` option: the helper writes the facts row, so the page publishes without canonical projection whatever the operation) under `withPendingFenceRows` at `pendingTier`. `index-existing` takes a row the file already holds (a fence-guard hold's `payload.row_num`): the cell must pass `fenceCellReadBackProblem` against the fact, else it refuses `fence_row_changed` (reason `row_missing` / `struck` / `empty` / `visibility` / `text`) and writes nothing; otherwise the facts row derives from the cell (`extractFactsFromFenceText`, what a re-import projects) with the caller's embedding and session, and no file changes; the caller's observedRevision is `snapshot.revision`, which the coordinator re-checks under the page lock. An occupied row number fails the insert with `storage_error` naming the occupant. Tests: `test/fence-fact-append.test.ts` (PGLite and Postgres: append, supersede, index-existing from a real fence-guard hold, each refusal, and remember's fence row byte for byte against the earlier construction). GST-2492: `prepareFenceRowRetirement` strikes one or several rows of another page as a companion (a forgotten held row reads back `forgotten:`; `remove` takes the rows out instead of striking them).
+- `src/core/facts/fence-fact-append.ts` — `prepareFenceFactAppend(engine, row, config, snapshot, { fact, pendingTier, mode?, supersede? | rowNum }, signal)`: the one way a coordinator preparer puts a single fact into the facts fence of the request's page (`row.slug`); remember uses it, and so does a managed owner release of a held fact (`trust/release-publish.ts`, the `trust_release` operation: append for a held fact with an entity slug, index-existing for a hold naming `payload.row_num`, whose cell fields the owner previewed and the request binds: a moved cell refuses `preview_changed`; journaled as mode `index-previewed`, which a preparer built before the field preview refuses, and an `index-existing` request with no fields digest refuses `preview_changed` asking for a rerun). It returns `{ prepared, rowNum, fencesNormalized, apply(tx) }`: `prepared` is the page write to compose (its `file`, `exclusiveSources`, `validate`; null in index-existing), and `apply` runs in the publication transaction after the coordinator published the file, writing the page row (tier kept, `trust/fence-append.ts`) and then the facts row with `row_num` and `source_markdown_slug`, so a facts row never precedes its fence row in Git. `append` (default) normalizes the stored fence (#6188), renders the row with `fenceRowOf` (`unfenced-facts.ts`, the one renderer) at one past the highest row number in the fence and among the page's facts rows (#6696 C9), strikes `supersede.rowNum` when given, and prepares the page write `unprojected` (`preparePageMutation` option: the helper writes the facts row, so the page publishes without canonical projection whatever the operation) under `withPendingFenceRows` at `pendingTier`. `index-existing` takes a row the file already holds (a fence-guard hold's `payload.row_num`): the cell must pass `fenceCellReadBackProblem` against the fact, else it refuses `fence_row_changed` (reason `row_missing` / `struck` / `empty` / `visibility` / `text`) and writes nothing; otherwise the facts row derives from the cell (`extractFactsFromFenceText`, what a re-import projects) with the caller's embedding and session, and no file changes; the caller's observedRevision is `snapshot.revision`, which the coordinator re-checks under the page lock. An occupied row number fails the insert with `storage_error` naming the occupant. Tests: `test/fence-fact-append.test.ts` (PGLite and Postgres: append, supersede, index-existing from a real fence-guard hold, each refusal, and remember's fence row byte for byte against the earlier construction). GST-2497: on a holding page (not segmented), `apply` then keeps the page's newest five inline saved versions (`pruneHoldingPageVersions`). GST-2492: `prepareFenceRowRetirement` strikes one or several rows of another page as a companion (a forgotten held row reads back `forgotten:`; `remove` takes the rows out instead of striking them).
- `src/core/facts/held-fact-move.ts` — GST-2492 `moveHeldFacts(engine, sourceId)`, run by the extract_facts phase: facts a strict source held on `memory/unattributed` (an `entity` cell naming a page that did not exist) move into that page once it exists, one `managed_maintenance_move_held_fact` publication per entity (the fact-fence adoption preparer in `persistence/prepared-maintenance.ts`, plus the holding rows struck as a companion, the belief ledger told the rows transferred). A row that cannot move (a database-only target, a malformed fence, a duplicate claim, a prior refusal) stays held and is reported `held_fact_unmovable`; a move into a restricted page removes the holding rows instead of striking them. GST-2497: the sync path's due check (`moveRuledSharedHeld`) is true only when a rule covers a held row's entity but not its holding page (a rule over the holding page alone narrows nothing), and `planningView` bounds `getConfig` too.
- `src/core/page-slug-grammar.ts` — `OP_PAGE_SLUG_RE` / `isOpPageSlug`, the page slug grammar operations accept (moved out of `ops/context.ts` so `facts-fence.ts` can validate the `entity` cell of `memory/unattributed` without importing the operations layer).
diff --git a/docs/architecture/key-files/core-services-2-continued.md b/docs/architecture/key-files/core-services-2-continued.md
index 19b1047cd0..c80fcbaf7d 100644
--- a/docs/architecture/key-files/core-services-2-continued.md
+++ b/docs/architecture/key-files/core-services-2-continued.md
@@ -19,7 +19,7 @@ The page-state concurrency foundation under `src/core/page-state/` and the share
Embedding-only publication also requires a live, unarchived source and nondeleted page under the same source/page row locks. A prepared vector cannot bypass a later archive, model change or withdrawal. `test/e2e/migration-wave-prepared-withdrawal.test.ts` exercises both publication orders and exact unchanged-vector refusal on both engines.
-- `src/core/page-state/version-types.ts` — complete canonical version contract with nullable legacy fields, re-exported by `types.ts` for existing callers. `page-state/versions.ts` records deletion, timeline, tags and originating revision in the page publication transaction. Wave B: a segmented page's version references its body generation (`body_storage: 'segmented'`, `body_generation`, `body_bytes`, `content_hash`, NULL bodies; O(1)); `pruneSegmentedVersions` keeps the newest `versions.segmented_max_per_page` and prunes oldest-first while bytes pinned beyond the live generation exceed max(`versions.segmented_max_pinned_bytes`, body_bytes), then `sweepSegments`; `get_versions` returns their size and, with `include_body`, a `body_ref`; `revert_version` refuses them with `page_body_segmented` until PR 3. A holding page (`isHoldingSlug`) keeps its newest `LARGE_PAGE_VERSIONS_KEPT` inline versions whatever their size, at most `HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE` (100) removed per write (GST-2497). `LARGE_PAGE_VERSIONS_KEPT` lives in `page-body/limits.ts`.
+- `src/core/page-state/version-types.ts` — complete canonical version contract with nullable legacy fields, re-exported by `types.ts` for existing callers. `page-state/versions.ts` records deletion, timeline, tags and originating revision in the page publication transaction. Wave B: a segmented page's version references its body generation (`body_storage: 'segmented'`, `body_generation`, `body_bytes`, `content_hash`, NULL bodies; O(1)); `pruneSegmentedVersions` keeps the newest `versions.segmented_max_per_page` and prunes oldest-first while bytes pinned beyond the live generation exceed max(`versions.segmented_max_pinned_bytes`, body_bytes), then `sweepSegments`; `get_versions` returns their size and, with `include_body`, a `body_ref`; `revert_version` refuses them with `page_body_segmented` until PR 3. A holding page (`isHoldingSlug`) keeps its newest `LARGE_PAGE_VERSIONS_KEPT` inline versions whatever their size, at most `HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE` (100) removed per save (`pruneHoldingPageVersions`, run only by the fence append after a save that adds a fact; a held move and purge, which edit saved versions themselves, never prune) (GST-2497). `LARGE_PAGE_VERSIONS_KEPT` lives in `page-body/limits.ts`.
- `src/core/progressive-batch/` — shared ramp-up + cost-cap + verification primitive (trial 10 → ramp 100 → ramp 500 → full, with verification at each stage), with verifier+policy injection (callers describe HOW TO MEASURE SUCCESS, not WHEN TO WAIT FOR CTRL-C). Modules: `types.ts` (Stage, StageVerdict, AbortReason, discriminated `Verifier` union `OutputCountVerifier | IdempotentMutationVerifier | NoopVerifier`, Policy, StageReport), `orchestrator.ts` (`runProgressiveBatch(items, verifier, policy, runner)` — reads `getCurrentBudgetTracker()` ahead of `Policy.maxCostUsd` fail-closed; null both ways triggers `abort_cost_cap reason='no_budget_safety_net'`), `audit.ts` (ISO-week JSONL at `~/.gbrain/audit/progressive-batch-YYYY-Www.jsonl` via the shared `audit-writer` primitive), `stage-report.ts` (ASCII formatter for the default `Policy.onStageReport`). Env knobs: `GBRAIN_PROGRESSIVE_BATCH_DISABLED=1`, `GBRAIN_PROGRESSIVE_BATCH_AUTO=1` (skip Ctrl-C grace), `GBRAIN_PROGRESSIVE_BATCH_STAGES=10,100,500`. Sites that "jump straight to full" stay that way by default; ramp is opt-in per-site via `Policy.interactiveAbortMs > 0`. Pinned by `test/progressive-batch/orchestrator.test.ts` (35 cases, every verdict path).
diff --git a/src/core/facts/fence-fact-append.ts b/src/core/facts/fence-fact-append.ts
index 7e269b52a4..bf2ad9d3b4 100644
--- a/src/core/facts/fence-fact-append.ts
+++ b/src/core/facts/fence-fact-append.ts
@@ -34,7 +34,7 @@ import { opError } from '../ops/contract.ts';
import { readFix } from '../ops/op-fix.ts';
import { FACTS_FENCE_BEGIN, FACTS_FENCE_END, factsFenceSection, fenceCellReadBackProblem, formatFenceDate, parseFactsFenceFor, renderFactsTable, replaceOrInsertFactsFence, strikeFactRowInPage, upsertFactRow } from '../facts-fence.ts';
import { serializePageToMarkdown } from '../markdown.ts';
-import { pageBodyOverCeilingError } from '../page-body/page-shape.ts';
+import { isSegmentedPage, pageBodyOverCeilingError } from '../page-body/page-shape.ts';
import { materializedPage, segmentedOverCeiling } from '../persistence/segmented-apply.ts';
import { preparePageMutation } from '../persistence/page-prepare.ts';
import type { CompanionPage, PreparedMutation } from '../persistence/coordinator.ts';
@@ -53,6 +53,7 @@ import { fenceRowOf, withHeldFactsFence, type FenceRowInput } from './unfenced-f
import { rewriteVersionBodies } from './purge.ts';
import { carriesClaimSql, claimNeedles } from './purge-verify.ts';
import { withdrawnFact } from './withdrawal-overlay.ts';
+import { pruneHoldingPageVersions } from '../page-state/versions.ts';
export type FenceFactAppendInput = {
/** The facts row to publish; `row_num` and `source_markdown_slug` are the helper's. kind and visibility default as insertFacts does. */
@@ -190,6 +191,7 @@ export async function prepareFenceFactAppend(engine: BrainEngine, row: WriteRequ
}
return { prepared, rowNum, fencesNormalized, apply: async tx => {
const pageReceipt = await withPageTierKept(tx, page, () => prepared.apply(tx));
+ if (snapshot && isHoldingSlug(row.slug) && !isSegmentedPage(snapshot.page)) await pruneHoldingPageVersions(tx, snapshot.page.id);
return { id: await insert(tx, rowNum, fact), pageReceipt };
} };
}
diff --git a/src/core/page-state/versions.ts b/src/core/page-state/versions.ts
index d68a2eb6d7..f50fe5107b 100644
--- a/src/core/page-state/versions.ts
+++ b/src/core/page-state/versions.ts
@@ -5,10 +5,9 @@ import { PAGE_INLINE_MAX_BYTES, isLargePageBytes } from '../page-size-policy.ts'
import { LARGE_PAGE_VERSIONS_KEPT, readPageBodyPolicy } from '../page-body/limits.ts';
import { opError, type OperationError } from '../ops/contract.ts';
import { isSegmentedPage } from '../page-body/page-shape.ts';
-import { isHoldingSlug } from '../facts-fence.ts';
export { LARGE_PAGE_VERSIONS_KEPT };
-/** GST-2497: the most saved versions of a holding page one write removes, so a page with a long history drains over later saves. */
+/** GST-2497: the most saved versions of a holding page one save removes, so a page with a long history drains over later saves. */
export const HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE = 100;
/**
@@ -38,12 +37,6 @@ export async function createPageVersion(engine: BrainEngine, slug: string, sourc
[page.id, PAGE_INLINE_MAX_BYTES, LARGE_PAGE_VERSIONS_KEPT]);
await markPrunedHistory(tx, page.id, pruned.map(row => Number(row.id)));
}
- if (isHoldingSlug(page.slug)) {
- const pruned = await tx.executeRaw<{ id: number }>(`DELETE FROM page_versions WHERE id IN (SELECT id FROM (SELECT id FROM page_versions
- WHERE page_id = $1 AND body_storage IS DISTINCT FROM 'segmented' ORDER BY id DESC OFFSET $2) excess ORDER BY id LIMIT $3) RETURNING id`,
- [page.id, LARGE_PAGE_VERSIONS_KEPT, HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE]);
- await markPrunedHistory(tx, page.id, pruned.map(row => Number(row.id)));
- }
return { ...rows[0], snapshot_at: new Date(rows[0].snapshot_at) };
};
if (preimage) return insert(engine, preimage);
@@ -106,6 +99,19 @@ export async function pruneSegmentedVersions(tx: BrainEngine, pageId: number): P
return deleted.length;
}
+/**
+ * GST-2497: a save that adds a fact to a holding page keeps that page's newest LARGE_PAGE_VERSIONS_KEPT inline saved
+ * versions, removing at most HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE older ones, and records the gap. It runs only from
+ * the fence append, after its version is saved: writes that edit saved versions themselves (a held move, purge) never
+ * prune, so the versions they count are the versions they edit. Segmented versions keep their own retention.
+ */
+export async function pruneHoldingPageVersions(tx: BrainEngine, pageId: number): Promise<void> {
+ const pruned = await tx.executeRaw<{ id: number }>(`DELETE FROM page_versions WHERE id IN (SELECT id FROM (SELECT id FROM page_versions
+ WHERE page_id = $1 AND body_storage IS DISTINCT FROM 'segmented' ORDER BY id DESC OFFSET $2) excess ORDER BY id LIMIT $3) RETURNING id`,
+ [pageId, LARGE_PAGE_VERSIONS_KEPT, HOLDING_PAGE_VERSIONS_PRUNED_PER_WRITE]);
+ await markPrunedHistory(tx, pageId, pruned.map(row => Number(row.id)));
+}
+
/**
* GBRA-217: after a prune, the retained snapshot just before each pruned one has lost its successor (`successor_pruned`,
* so a remote reader is never shown its reason: the successor's visibility can no longer be checked), and the page has