Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Stop planning a held-fact move on every sync, and bound its config read
garrytandcecd69e252a
diff --git a/docs/architecture/key-files/core-persistence-facts.md b/docs/architecture/key-files/core-persistence-facts.mdindex 5f60b75b07..ef99c94ffa 100644--- a/docs/architecture/key-files/core-persistence-facts.md+++ b/docs/architecture/key-files/core-persistence-facts.md@@ -23,5 +23,5 @@ Managed facts extraction: the entity-intent preparer and publisher, conversation- `src/core/repair/fact-claim-dates.ts` — `gbrain repair fact-claim-dates --source <id> [--day YYYY-MM-DD]` (#7247), explicit-only, preview-bound, destructive (`--yes`). Candidates: active facts on a bulk day (`findBulkFactDays`, or `--day`) whose claim has exactly one earlier ISO date (`claimDate`). A fence row's `valid_from` cell is rewritten in the page's one fence section and published by revision-bound `put_page` (the facts row is re-projected); a database-only fact is updated in one `maintenanceTransaction` on an unmanaged brain and counted `database_only_managed` on a managed one. Pages with an unparsed, split or segmented fence, or a rewrite that does not read back as the planned rows, are listed and kept. The hash binds revisions, row numbers and new dates; a fresh apply recomputes the plan and refuses `preview_changed` on any difference. Test: `test/repair-fact-claim-dates-7247.test.ts`.- `src/core/facts/fence-fact-append.ts` — `prepareFenceFactAppend(engine, row, config, snapshot, { fact, pendingTier, mode?, supersede? | rowNum }, signal)`: the one way a coordinator preparer puts a single fact into the facts fence of the request's page (`row.slug`); remember uses it, and so does a managed owner release of a held fact (`trust/release-publish.ts`, the `trust_release` operation: append for a held fact with an entity slug, index-existing for a hold naming `payload.row_num`, whose cell fields the owner previewed and the request binds: a moved cell refuses `preview_changed`; journaled as mode `index-previewed`, which a preparer built before the field preview refuses, and an `index-existing` request with no fields digest refuses `preview_changed` asking for a rerun). It returns `{ prepared, rowNum, fencesNormalized, apply(tx) }`: `prepared` is the page write to compose (its `file`, `exclusiveSources`, `validate`; null in index-existing), and `apply` runs in the publication transaction after the coordinator published the file, writing the page row (tier kept, `trust/fence-append.ts`) and then the facts row with `row_num` and `source_markdown_slug`, so a facts row never precedes its fence row in Git. `append` (default) normalizes the stored fence (#6188), renders the row with `fenceRowOf` (`unfenced-facts.ts`, the one renderer) at one past the highest row number in the fence and among the page's facts rows (#6696 C9), strikes `supersede.rowNum` when given, and prepares the page write `unprojected` (`preparePageMutation` option: the helper writes the facts row, so the page publishes without canonical projection whatever the operation) under `withPendingFenceRows` at `pendingTier`. `index-existing` takes a row the file already holds (a fence-guard hold's `payload.row_num`): the cell must pass `fenceCellReadBackProblem` against the fact, else it refuses `fence_row_changed` (reason `row_missing` / `struck` / `empty` / `visibility` / `text`) and writes nothing; otherwise the facts row derives from the cell (`extractFactsFromFenceText`, what a re-import projects) with the caller's embedding and session, and no file changes; the caller's observedRevision is `snapshot.revision`, which the coordinator re-checks under the page lock. An occupied row number fails the insert with `storage_error` naming the occupant. Tests: `test/fence-fact-append.test.ts` (PGLite and Postgres: append, supersede, index-existing from a real fence-guard hold, each refusal, and remember's fence row byte for byte against the earlier construction). GST-2492: `prepareFenceRowRetirement` strikes one or several rows of another page as a companion (a forgotten held row reads back `forgotten:`; `remove` takes the rows out instead of striking them).-- `src/core/facts/held-fact-move.ts` — GST-2492 `moveHeldFacts(engine, sourceId)`, run by the extract_facts phase: facts a strict source held on `memory/unattributed` (an `entity` cell naming a page that did not exist) move into that page once it exists, one `managed_maintenance_move_held_fact` publication per entity (the fact-fence adoption preparer in `persistence/prepared-maintenance.ts`, plus the holding rows struck as a companion, the belief ledger told the rows transferred). A row that cannot move (a database-only target, a malformed fence, a duplicate claim, a prior refusal) stays held and is reported `held_fact_unmovable`; a move into a restricted page removes the holding rows instead of striking them.+- `src/core/facts/held-fact-move.ts` — GST-2492 `moveHeldFacts(engine, sourceId)`, run by the extract_facts phase: facts a strict source held on `memory/unattributed` (an `entity` cell naming a page that did not exist) move into that page once it exists, one `managed_maintenance_move_held_fact` publication per entity (the fact-fence adoption preparer in `persistence/prepared-maintenance.ts`, plus the holding rows struck as a companion, the belief ledger told the rows transferred). A row that cannot move (a database-only target, a malformed fence, a duplicate claim, a prior refusal) stays held and is reported `held_fact_unmovable`; a move into a restricted page removes the holding rows instead of striking them. GST-2497: the sync path's due check (`moveRuledSharedHeld`) is true only when a rule covers a held row's entity but not its holding page (a rule over the holding page alone narrows nothing), and `planningView` bounds `getConfig` too.- `src/core/page-slug-grammar.ts` — `OP_PAGE_SLUG_RE` / `isOpPageSlug`, the page slug grammar operations accept (moved out of `ops/context.ts` so `facts-fence.ts` can validate the `entity` cell of `memory/unattributed` without importing the operations layer).diff --git a/src/core/facts/held-fact-move.ts b/src/core/facts/held-fact-move.tsindex 2f10370a7a..991786ce56 100644--- a/src/core/facts/held-fact-move.ts+++ b/src/core/facts/held-fact-move.ts@@ -19,6 +19,7 @@ import { getWorktreeBinding } from '../persistence/ownership.ts';import { isStatementTimeout, PreparationDeadlineError } from '../persistence/bounded-reads.ts';import { registerEngineView } from '../persistence/switches.ts';import { syncPreparationBudgetMs } from '../persistence/sync-waivers.ts';+import { requestConfigValue } from '../config-snapshot.ts';import { isFenceHold, pageHasFile, planManagedPage, type UnfencedFactRow } from './unfenced-facts.ts';export interface HeldFactMoveOutcome {@@ -195,7 +196,8 @@ export function heldMoveIncomplete(held: Pick<HeldFactMoveOutcome, 'segmented_pa/*** GST-2492 (B2): after a sync releases its lease, the held facts a rule it applied narrows below their holding page's* readers (a folder rule over the shared page, an exact page rule over a folder's holding page, or a pending narrowing)- * leave that page at once instead of at the next maintenance cycle (still the fallback after a crash). Null when none is due.+ * leave that page at once instead of at the next maintenance cycle (still the fallback after a crash). Null when none is due:+ * every rule covering a held row's entity also covers its holding page (a rule over the holding page alone narrows nothing).* A lock another session holds never keeps the sync past its preparation budget (#6318): the due check and the plan share* one deadline (planningView), and the move's own page reads each end at the budget. A plan cut off there rejects, a group* cut off in its move stays unmovable, and maintenance moves them later.@@ -206,14 +208,15 @@ export async function moveRuledSharedHeld(engine: BrainEngine, sourceId: string)const [due] = await planning.executeRaw<{ due: boolean }>(`SELECT EXISTS (SELECT 1 FROM facts f WHERE f.source_id = $1AND f.source_markdown_slug IN (SELECT hp.slug FROM pages hp WHERE hp.source_id = $1 AND ${holdingSlugSql('hp.slug')})AND f.row_num IS NOT NULL AND f.dimension IS NULL AND f.entity_slug IS NOT NULL AND NOT ${holdingSlugSql('f.entity_slug')}- AND (gbrain_access_folder_rules(f.source_id, f.entity_slug) IS DISTINCT FROM gbrain_access_folder_rules(f.source_id, f.source_markdown_slug)+ AND (NOT COALESCE(gbrain_access_folder_rules(f.source_id, f.entity_slug), '{}') <@ COALESCE(gbrain_access_folder_rules(f.source_id, f.source_markdown_slug), '{}')OR (gbrain_access_pending_covers(f.source_id, f.entity_slug) AND NOT gbrain_access_pending_covers(f.source_id, f.source_markdown_slug)))) AS due`, [sourceId]);return due?.due ? moveHeldFacts(engine, sourceId, { planning, timeoutMs: budget }) : null;}/**- * The engine the sync path plans a held move on: every raw read through it, the bare-name resolver's alias read included,- * ends on the server at `deadlineAt`, and one ended there rejects as a PreparationDeadlineError. PGLite plans on the engine.+ * The engine the sync path plans a held move on: every raw read through it, the bare-name resolver's alias read and+ * holdingHome's config read included, ends on the server at `deadlineAt`, and one ended there rejects as a+ * PreparationDeadlineError. PGLite plans on the engine.*/function planningView(engine: BrainEngine, deadlineAt: number): BrainEngine {if (engine.kind !== 'postgres') return engine;@@ -228,6 +231,12 @@ function planningView(engine: BrainEngine, deadlineAt: number): BrainEngine {const value = Reflect.get(target, key, target);// The alias read runs on the engine's own executeRaw: bound to the view, it takes the deadline too.if (key === 'resolveAliases') return value.bind(view);+ if (key === 'getConfig') return async (name: string) => {+ const held = await requestConfigValue(target, name);+ if (held !== undefined) return held;+ const [row] = await view.executeRaw<{ value: string }>('SELECT value FROM config WHERE key = $1', [name]);+ return row?.value ?? null;+ };return typeof value === 'function' ? value.bind(target) : value;} }), engine);return view;diff --git a/test/held-move-plan-gst2497.test.ts b/test/held-move-plan-gst2497.test.tsnew file mode 100644index 0000000000..36a6a9b94a--- /dev/null+++ b/test/held-move-plan-gst2497.test.ts@@ -0,0 +1,108 @@+/**+ * GST-2497: the sync's held-fact move on a strict source (moveRuledSharedHeld), after the review of GST-2492.+ *+ * 1. Protects: the move's due check is true only when a rule covers a held row's entity and not its holding page (the+ * planner then moves or re-homes the row), so a rule over the holding page alone never makes every sync plan a move+ * for nothing; a rule that narrows a held row below its holding page's readers still moves it right after the sync.+ * On Postgres the plan's config read (holdingHome's sync.write_through) ends at the plan's deadline like its other reads.+ * 2. Fails when: the second of two syncs on such a source runs the held-move plan; a rule over the entity does not move+ * the row after its sync; the move waits on a session holding the config table past the sync preparation budget.+ * 3. Existing coverage: test/folder-holding-page-gst2492.test.ts (the move, the rule apply, the pages and page_aliases locks).+ * 4. Real seams: the coordinated write on PGLite and Postgres (DATABASE_URL), a managed Git checkout, the policy file+ * committed to Git and applied by performSync, a second Postgres session holding a table lock, and the engine's raw read.+ */+import { afterEach, describe, expect, spyOn, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import postgres from '#postgres';+import { writeFileSync } from 'node:fs';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import type { OperationContext } from '../src/core/ops/contract.ts';+import { operationsByName } from '../src/core/operations.ts';+import { performSync } from '../src/commands/sync.ts';+import { serializePolicyYaml, type FolderRule } from '../src/core/access/folder-rules.ts';+import { policySeams, proposeRevision } from '../src/core/access/policy.ts';+import { commitPolicyFile, POLICY_FILE } from '../src/core/access/policy-git.ts';+import { __resetPrivateVisibilityCacheForTests } from '../src/core/search/private-visibility.ts';+import { resetSyncPreparationBudget, syncPreparationBudgetMs } from '../src/core/persistence/sync-waivers.ts';+import { moveRuledSharedHeld } from '../src/core/facts/held-fact-move.ts';+import { durableGitRepo } from './helpers/git-publication.ts';+import { managedBrain, settleManagedGitEffects, type ManagedBrain } from './helpers/managed-brain.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';++const HOLD = 'memory/unattributed';+const gitRepo = ({ root }: { root: string }) => { writeFileSync(join(root, '.gitignore'), '\n'); durableGitRepo(root, ['.gitignore']); };+const call = (ctx: OperationContext, name: string, params: Record<string, unknown>) =>+ operationsByName[name]!.handler(ctx, { request_id: randomUUID(), ...params }) as Promise<Record<string, any>>;+const remember = (ctx: OperationContext, fact: string, entity: string) => call(ctx, 'remember', { fact, entity, provenance: 'chat 2026-11-13' });+const rule = (id: string, prefix: string, readers: string[]): FolderRule => ({ id, prefix, readable_by: readers.map(r => `member:${r}`).sort() });+const heldOn = async (engine: BrainEngine, claim: string) => (await engine.executeRaw<{ slug: string }>(+ `SELECT source_markdown_slug AS slug FROM facts WHERE source_id = 'default' AND fact = $1 AND expired_at IS NULL`, [claim])).map(r => r.slug);+const caught = <T>(p: Promise<T>) => p.then(() => null, (error: Record<string, any>) => error);++for (const backend of testBackends()) describe(`[${backend}] the sync's held-fact move (GST-2497)`, () => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ afterEach(() => { policySeams.afterInstall = undefined; });+ const strict = (run: (brain: ManagedBrain) => Promise<void>) => managedBrain(async brain => {+ await brain.engine.executeRaw(`UPDATE sources SET config = config || '{"canonical":"git"}'::jsonb WHERE id='default'`);+ await run(brain);+ }, { databaseUrl, setup: gitRepo });++ test('a rule over the holding page but not the entity makes no sync plan a move; a rule over the entity still moves it after its sync', () => strict(async ({ engine, ctx, root }) => {+ const policy = (list: FolderRule[]) => commitPolicyFile(root, POLICY_FILE, () => serializePolicyYaml(list), 'access: rule');+ const sync = () => performSync(engine, { sourceId: 'default', repoPath: root, noPull: true, noEmbed: true } as never);+ policy([rule('memory', 'memory/', ['alice-example', 'bob-example'])]);+ await sync();+ const claim = 'The bob-example garden plan starts in spring.';+ expect(await remember(ctx, claim, 'people/bob-example')).toMatchObject({ page: HOLD });+ await settleManagedGitEffects(engine);+ const plans: string[] = [];+ const real = engine.executeRaw;+ const counted = spyOn(engine, 'executeRaw').mockImplementation(function (this: BrainEngine, sql: string, ...rest: unknown[]) {+ if (sql.includes('source_markdown_slug AS from_slug')) plans.push(sql);+ return (real as (...a: unknown[]) => Promise<unknown>).call(this, sql, ...rest);+ } as BrainEngine['executeRaw']);+ try {+ await sync();+ const first = plans.length;+ await sync();+ expect([first, plans.length - first]).toEqual([0, 0]);+ expect(await moveRuledSharedHeld(engine, 'default')).toBeNull();+ } finally { counted.mockRestore(); }+ expect(await heldOn(engine, claim)).toEqual([HOLD]);+ policy([rule('memory', 'memory/', ['alice-example', 'bob-example']), rule('people', 'people/', ['alice-example'])]);+ await sync();+ await settleManagedGitEffects(engine);+ expect(await heldOn(engine, claim)).toEqual(['people/memory/unattributed']);+ expect(await moveRuledSharedHeld(engine, 'default')).toBeNull();+ }), 300_000);++ test.skipIf(backend !== 'postgres')('on the sync path a due held move ends at the sync preparation budget while another session locks the config table; once the lock drops it moves', () => strict(async ({ engine, ctx }) => {+ const entity = 'clients/acme-example/brief';+ const claim = 'The acme-example brief carries a config-locked clause.';+ expect(await remember(ctx, claim, entity)).toMatchObject({ page: HOLD });+ policySeams.afterInstall = () => { throw new Error('interrupted'); };+ await expect(proposeRevision(engine, 'default', [rule('clients', 'clients/acme-example/', ['alice-example'])], { origin: 'cli', store: 'database', approved: false })).rejects.toThrow('interrupted');+ policySeams.afterInstall = undefined;+ __resetPrivateVisibilityCacheForTests();+ const budget = 1_000;+ await engine.setConfig('persistence.sync_preparation_ms', String(budget));+ resetSyncPreparationBudget();+ expect(await syncPreparationBudgetMs(engine)).toBe(budget);+ const [{ db }] = await engine.executeRaw<{ db: string }>('SELECT current_database() AS db');+ const url = new URL(databaseUrl!); url.pathname = `/${db}`;+ const holder = postgres(url.toString(), { max: 1, prepare: false, onnotice: () => {} });+ try {+ const ended = await holder.begin(async sql => {+ await sql.unsafe('LOCK TABLE config IN ACCESS EXCLUSIVE MODE');+ const started = performance.now();+ const error = await caught(moveRuledSharedHeld(engine, 'default'));+ return { code: error?.code, ms: performance.now() - started };+ });+ expect(ended.code).toBe('preparation_deadline');+ expect(ended.ms).toBeLessThan(budget + 3_000);+ } finally { await holder.end(); resetSyncPreparationBudget(); }+ expect(await moveRuledSharedHeld(engine, 'default')).toMatchObject({ rehomed: 1, unmovable: [], write_errors: 0 });+ expect(await heldOn(engine, claim)).toEqual(['clients/acme-example/memory/unattributed']);+ }), 300_000);+});