Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Doctor names purge and withdrawal guards missing their ledger check, and migrations stop hiding a failed guard re-create (GST-2885) (#9594)
Garry Tan2f84c1dd0475
diff --git a/behavior-changes.d/guard_ledger_checks_gst2885.md b/behavior-changes.d/guard_ledger_checks_gst2885.mdnew file mode 100644index 0000000000..5d9e49d9a9--- /dev/null+++ b/behavior-changes.d/guard_ledger_checks_gst2885.md@@ -0,0 +1,4 @@+---+kind: add+---+New doctor check `guard_ledger_checks` (Postgres only). It fails when a purge or withdrawal guard still lacks its ledger check, which happens when another role owned the guard at the v318 upgrade. It names the owning role and the fix. A migration run now prints a `guards:` line when it cannot re-create a guard for a reason other than another role owning it, where it used to stay silent. PGLite and brains whose guards are current see no change.diff --git a/changelog.d/fix-guard-ledger-followups-gst2885.md b/changelog.d/fix-guard-ledger-followups-gst2885.mdnew file mode 100644index 0000000000..12a14e47de--- /dev/null+++ b/changelog.d/fix-guard-ledger-followups-gst2885.md@@ -0,0 +1,12 @@+**`gbrain doctor` now names any purge or withdrawal guard on Postgres that still lacks its ledger check, and a migration run no longer hides a failed guard re-create.**++### Added+- New doctor check `guard_ledger_checks`. On Postgres it names each purge or withdrawal guard whose body is missing the check that refuses a write role row-level security keeps from the ledger, so that role could still store purged content again or save a withdrawn fact active. It names the role that owns each guard. When the role doctor connects as may re-create them, the fix is `gbrain apply-migrations --yes`. Otherwise it prints the SQL the owning role runs. On PGLite, which has no row-level security, it reports not applicable (GST-2885, follow-up to #7632 and #9517).++### Fixed+- Every migration run re-creates the guards its role owns. That step used to swallow every error. Now only a guard another role owns stays silent (SQLSTATE 42501). Any other failure is reported on stderr with a line pointing at `guard_ledger_checks`. A concurrent update of the same function is retried (GST-2885).++### Documentation+- docs/ENGINES.md ("Purge and withdrawal guards"): a bulk fact insert whose row matches a withdrawal needs an `UPDATE` policy on `facts` for the write role, beside its `SELECT` policy (GST-2885).++<!-- behavior-change: guard_ledger_checks_gst2885 -->diff --git a/docs/ENGINES.md b/docs/ENGINES.mdindex d9885e9b6b..7c018dde4c 100644--- a/docs/ENGINES.md+++ b/docs/ENGINES.md@@ -480,6 +480,12 @@ CREATE POLICY page_segment_derivation_purge_pages_writer_note ON page_segment_de```<!-- gbrain:ledger-rls-recipe:end -->+A bulk fact insert (the fence import in `src/core/engine-sql/facts.ts`) defers the withdrawal+stamp to the end of its statement, so when one of its rows matches a withdrawal the guard+updates that row in `facts`, and the write role then needs a permissive `UPDATE` policy on+`facts` covering the row, beside its `SELECT` policy, or the guard refuses the insert with the+same `writer_coordinator_required` error.+**Check that the upgrade replaced every guard.** Migration v318 can only replace a guard functionthe role running it owns. A guard owned by another role is left as it was, so it still lets ascoped writer without ledger policies through, and v318 only names it in one stderr line@@ -501,7 +507,10 @@ For a row with `fails_closed = f`, connect as the role in its `owner` column and`ALTER FUNCTION public.<proname>() OWNER TO <the role gbrain migrates as>;`. Then run`gbrain apply-migrations --yes`, or start gbrain: every migration run re-creates a guard thatis missing its check, provided the migrating role owns it. You can also run gbrain once as the-owning role instead. Run the query again to confirm.+owning role instead. Run the query again to confirm. On Postgres, `gbrain doctor`+(`guard_ledger_checks`) runs the same check and names each guard left without its ledger check+with its owner, and the fix: `gbrain apply-migrations --yes` when the doctor's role may re-create+it, else the `ALTER FUNCTION ... OWNER TO` SQL to run as the owning role first.The segmented storage recipe above already covers the two derivation purge tables for arole it applies to. A policy that filters out ledger rows of a source the role writes hidesdiff --git a/docs/architecture/key-files/security-continued.md b/docs/architecture/key-files/security-continued.mdindex 2f744b66d8..7dc29a562f 100644--- a/docs/architecture/key-files/security-continued.md+++ b/docs/architecture/key-files/security-continued.md@@ -5,4 +5,4 @@- `src/core/git-env.ts` + `src/core/hardened-git.ts` — #7584, the operator's `safe.directory` in scrubbed Git environments: `hardenedGitEnvironment` still drops system and global config and every `GIT_*` variable, but carries the operator's `safe.directory` entries (system and global, read from `/` with `git config --includes --show-scope` and cached for 30 s, then `GIT_CONFIG_*`) as environment config, so a source repository owned by another user reads once the operator allows it. `gitChildEnv` carries the environment entries the same way (`envSafeDirectories`, `safeDirectoryEnv`). Every other key in those places stays out. `gitChildEnv` also drops Git's global pathspec switches (`GIT_PATHSPEC_ENV_VARS`: `GIT_LITERAL_PATHSPECS`, `GIT_GLOB_PATHSPECS`, `GIT_NOGLOB_PATHSPECS`, `GIT_ICASE_PATHSPECS`; `extra` can still set one). gbrain's writers and readers mark each path literal with `literalPathspecs` (`:(literal)<path>`; an empty path throws, since `:(literal)` alone matches the whole tree, #7793), never `git --literal-pathspecs`, which Git exports to every hook, textconv and filter helper as `GIT_LITERAL_PATHSPECS=1`, where a hook's glob pathspec then matches nothing; `test/git-env-guard.test.ts` rejects a global pathspec switch (flag or env) anywhere in `src`. GST-2331: `NO_LAZY_FETCH_ENV` (`GIT_NO_LAZY_FETCH=1` and an empty `GIT_ALLOW_PROTOCOL`) is what a managed-sync read that runs under `gitChildEnv` adds beside `HARDENED_GIT_ARGS`, so a partial clone fetches nothing (Git before 2.45 ignores the first; the empty allowlist allows no transport on any Git): `syncGit`/`syncGitAsync`, every read in `sync-blobs.ts`, `sync-delta.ts`'s text runner (a rename whose old blob the clone lacks fails the diff, `diff_failed`, and discovery walks the whole tree), `sync-rename-identity.ts`, `sync-metafile-pages.ts`, `sync-origin-evidence.ts`, `sync-held-deletions.ts`, `unindexed-files.ts` and `pending-import.ts`. Tests: `test/hardened-git-safe-directory-7584.test.ts` (ownership forced with `GIT_TEST_ASSUME_DIFFERENT_OWNER` through a `git` wrapper on PATH), `test/git-literal-pathspec-hooks-7759.test.ts`, `test/git-literal-pathspecs-7793.test.ts`, `test/sync-partial-clone-2331.test.ts`.- `src/core/outside-content.ts` — GBRA-221: `neutralizeOutsideText(text)`, the one neutralizer for text a third party wrote before it lands in a page body (Gmail/Calendar/Contacts renders in `google-render.ts`, GitHub `renderItemPage`/`renderRepoCard`, chat transcripts in `transcripts/render.ts`, `book-mirror` chapter output). It escapes every `gbrain<ws>:<ws><name>` marker token to `gbrain\:<name>` (case-insensitive), `[Source:` to `[Source\:`, and backslash-prefixes each timeline sentinel, bare `## Timeline`/`## History` heading, dated timeline row or heading, transcript message anchor and date heading, using the parsers' own predicates. Idempotent and readable; apply it per outside field (or a composed line holding one), never to a whole page that may hold gbrain's own fences or citations. Also owns `escapeAnchorLines` and `MESSAGE_ANCHOR_RE` (re-exported by `transcripts/render.ts`). New connectors use it.-- `src/core/function-search-path.ts` — #7632 search_path hardening (#7688, #7785): `PINNED_FUNCTION_NAMES` (every gbrain function with a pinned search_path; `scripts/check-search-path.sh` fails one missing from it or a pinned path not ending in `pg_temp`), `FUNCTION_SEARCH_PATH_HARDENING_SQL` (the schema blob's last statement), and `hardenFunctionSearchPaths` (v277 and the end of every migration run: one transaction under the migration timeouts, a pg_proc row lock it cannot take becomes `schema_lock_blocked`, a concurrent pg_proc update is retried, returns the functions left or null when listing them failed). Doctor check: `src/commands/doctor/checks/function-search-path.ts` (`function_search_path`: names any pinned function whose path does not end in `pg_temp`; the fix is `gbrain apply-migrations --yes` below v277, else the schema-qualified `ALTER FUNCTION` SQL (argument types only) in `details.sql` to run as the owner, with no runnable fix).+- `src/core/function-search-path.ts` — #7632 search_path hardening (#7688, #7785): `PINNED_FUNCTION_NAMES` (every gbrain function with a pinned search_path; `scripts/check-search-path.sh` fails one missing from it or a pinned path not ending in `pg_temp`), `FUNCTION_SEARCH_PATH_HARDENING_SQL` (the schema blob's last statement), and `hardenFunctionSearchPaths` (v277 and the end of every migration run: one transaction under the migration timeouts, a pg_proc row lock it cannot take becomes `schema_lock_blocked`, a concurrent pg_proc update is retried, returns the functions left or null when listing them failed; it first re-creates the purge and withdrawal guards its role owns, retrying a concurrent update, and a re-create failure other than SQLSTATE 42501 is thrown as `LedgerGuardRecreateError` after the hardening runs, which `hardenFailureLine` prints as a `guards:` line). Doctor checks: `src/commands/doctor/checks/guard-ledger-checks.ts` (`guard_ledger_checks`, Postgres only: names each guard missing its ledger check, from `ledgerGuardsWithoutCheck` in `src/core/guard-ledger-checks.ts`, with its owner) and `src/commands/doctor/checks/function-search-path.ts` (`function_search_path`: names any pinned function whose path does not end in `pg_temp`; the fix is `gbrain apply-migrations --yes` below v277, else the schema-qualified `ALTER FUNCTION` SQL (argument types only) in `details.sql` to run as the owner, with no runnable fix).diff --git a/llms-full.txt b/llms-full.txtindex f1dc972e5c..5bbbfcfbff 100644--- a/llms-full.txt+++ b/llms-full.txt@@ -5447,6 +5447,12 @@ CREATE POLICY page_segment_derivation_purge_pages_writer_note ON page_segment_de```<!-- gbrain:ledger-rls-recipe:end -->+A bulk fact insert (the fence import in `src/core/engine-sql/facts.ts`) defers the withdrawal+stamp to the end of its statement, so when one of its rows matches a withdrawal the guard+updates that row in `facts`, and the write role then needs a permissive `UPDATE` policy on+`facts` covering the row, beside its `SELECT` policy, or the guard refuses the insert with the+same `writer_coordinator_required` error.+**Check that the upgrade replaced every guard.** Migration v318 can only replace a guard functionthe role running it owns. A guard owned by another role is left as it was, so it still lets ascoped writer without ledger policies through, and v318 only names it in one stderr line@@ -5468,7 +5474,10 @@ For a row with `fails_closed = f`, connect as the role in its `owner` column and`ALTER FUNCTION public.<proname>() OWNER TO <the role gbrain migrates as>;`. Then run`gbrain apply-migrations --yes`, or start gbrain: every migration run re-creates a guard thatis missing its check, provided the migrating role owns it. You can also run gbrain once as the-owning role instead. Run the query again to confirm.+owning role instead. Run the query again to confirm. On Postgres, `gbrain doctor`+(`guard_ledger_checks`) runs the same check and names each guard left without its ledger check+with its owner, and the fix: `gbrain apply-migrations --yes` when the doctor's role may re-create+it, else the `ALTER FUNCTION ... OWNER TO` SQL to run as the owning role first.The segmented storage recipe above already covers the two derivation purge tables for arole it applies to. A policy that filters out ledger rows of a source the role writes hidesdiff --git a/scripts/postgres-arm-weights.json b/scripts/postgres-arm-weights.jsonindex d40561ec02..22f029ef90 100644--- a/scripts/postgres-arm-weights.json+++ b/scripts/postgres-arm-weights.json@@ -77,6 +77,7 @@"test/doctor-attest-read-shared-scope.test.ts": 6467,"test/doctor-database-encoding.test.ts": 2590,"test/doctor-frontmatter-holds.test.ts": 15372,+ "test/doctor-guard-ledger-checks.test.ts": 5866,"test/doctor-image-embedding-coverage.test.ts": 8105,"test/doctor-keyword-plan-6495.test.ts": 4580,"test/doctor-routing-skill-pack-source.test.ts": 6433,@@ -154,6 +155,7 @@"test/graduation-verify-sync-freshness.test.ts": 6454,"test/graduation-verify.test.ts": 18535,"test/grandfather-managed.test.ts": 46554,+ "test/guard-ledger-recreate-errors.test.ts": 6546,"test/harness-skills-epoch.test.ts": 4313,"test/harness-verify-member-credential.test.ts": 9000,"test/held-bare-name-move-gst2492.test.ts": 17127,diff --git a/src/commands/doctor/checks/guard-ledger-checks.ts b/src/commands/doctor/checks/guard-ledger-checks.tsnew file mode 100644index 0000000000..e8aa96a9c8--- /dev/null+++ b/src/commands/doctor/checks/guard-ledger-checks.ts@@ -0,0 +1,58 @@+/**+ * guard_ledger_checks doctor check (GST-2885, #7632/#7659): every purge and withdrawal guard carries its+ * row_security_active('<ledger>') check, so a write role that row-level security keeps from the ledger is refused+ * instead of reading it empty and storing purged content again or saving a withdrawn fact active.+ *+ * A guard left without it (another role owned it when v318 ran, or an older gbrain replayed its schema) is named with+ * its owner. When the doctor's role may re-create it, the fix is the migration run (every run re-creates the guards its+ * role owns, guard-ledger-checks.ts); otherwise the role that owns it runs the printed ALTER FUNCTION ... OWNER TO, or+ * that migration run itself. PGLite has no row-level security, so the check does not apply there. Read-only.+ */++import type { Check } from '../../doctor.ts';+import { connectedEngine, type DoctorContext, type DoctorEntry } from '../context.ts';+import { agentFix, checkError } from '../check-fix.ts';+import { ledgerGuardsWithoutCheck } from '../../../core/guard-ledger-checks.ts';++const DOCS = 'docs/ENGINES.md#purge-and-withdrawal-guards';++async function runGuardLedgerChecks(ctx: DoctorContext): Promise<Check[]> {+ const engine = connectedEngine(ctx);+ const checks: Check[] = [];+ if (engine.kind === 'pglite') {+ checks.push({ name: 'guard_ledger_checks', status: 'ok', message: 'Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)' });+ return checks;+ }+ ctx.progress.heartbeat('guard_ledger_checks');+ try {+ const stale = await ledgerGuardsWithoutCheck(engine);+ const functions = stale.map(g => g.sig);+ const owners = stale.map(g => g.owner);+ const named = stale.map(g => `${g.sig} (owner ${g.owner})`).join(', ');+ const risk = `${stale.length} purge or withdrawal guard(s) lack their ledger check (${named}), so a write role that row-level security keeps from the ledger reads it empty and can store purged content again or save a withdrawn fact active.`;+ if (stale.length === 0) {+ checks.push({ name: 'guard_ledger_checks', status: 'ok', message: 'Every purge and withdrawal guard refuses a write role that row-level security keeps from its ledger' });+ } else if (stale.every(g => g.owned)) {+ checks.push({+ name: 'guard_ledger_checks', status: 'fail',+ message: `${risk} Fix: run gbrain apply-migrations --yes --no-autopilot-install; every migration run re-creates the guards its role owns.`,+ fix: agentFix(['gbrain', 'apply-migrations', '--yes', '--no-autopilot-install'],+ 'Every migration run re-creates the purge and withdrawal guards its role owns with their ledger checks.', 'guard_ledger_checks', { requires_exclusive: true, docs: DOCS }),+ details: { functions, owners },+ });+ } else {+ const sql = stale.map(g => `ALTER FUNCTION ${g.qualified} OWNER TO ${g.role};`);+ checks.push({+ name: 'guard_ledger_checks', status: 'fail',+ message: `${risk} CREATE OR REPLACE FUNCTION needs the owner. Fix: connected as ${[...new Set(owners)].join(' or ')}, the role that owns each guard, run gbrain apply-migrations --yes --no-autopilot-install, or run ${sql.join(' ')} and then gbrain apply-migrations --yes --no-autopilot-install as this role. See ${DOCS}.`,+ fix_unavailable_reason: 'no_safe_automatic_fix',+ details: { functions, owners, sql, docs: DOCS },+ });+ }+ } catch (e) {+ checks.push(checkError('guard_ledger_checks', 'check purge and withdrawal guards', e));+ }+ return checks;+}++export const guardLedgerChecksEntry: DoctorEntry = { name: 'guard_ledger_checks', emits: ['guard_ledger_checks'], run: runGuardLedgerChecks };diff --git a/src/commands/doctor/registry.ts b/src/commands/doctor/registry.tsindex 4cf0688f76..e69fbc9857 100644--- a/src/commands/doctor/registry.ts+++ b/src/commands/doctor/registry.ts@@ -44,6 +44,7 @@ import {} from './checks/db-connection.ts';import { ipcSocketPathEntry } from './checks/local-ipc.ts';import { functionSearchPathEntry } from './checks/function-search-path.ts';+import { guardLedgerChecksEntry } from './checks/guard-ledger-checks.ts';import { postgresTlsEntry } from './checks/postgres-tls.ts';import { databaseEncodingEntry } from './checks/database-encoding.ts';import { schemaFunctionBodiesEntry } from './checks/schema-function-bodies.ts';@@ -268,6 +269,7 @@ export const DOCTOR_CHECK_REGISTRY: readonly DoctorEntry[] = [rlsEntry,postgresTlsEntry,functionSearchPathEntry,+ guardLedgerChecksEntry,databaseEncodingEntry,schemaVersionEntry,schemaFunctionBodiesEntry,diff --git a/src/core/doctor-categories.ts b/src/core/doctor-categories.tsindex fea5410b10..b70bc83f2b 100644--- a/src/core/doctor-categories.ts+++ b/src/core/doctor-categories.ts@@ -499,6 +499,8 @@ export const OPS_CHECK_NAMES: ReadonlySet<string> = new Set(['postgres_tls',// #7632: every gbrain function that pins a search_path ends it with pg_temp.'function_search_path',+ // GST-2885: every purge and withdrawal guard carries its row_security_active ledger check (#7632/#7659).+ 'guard_ledger_checks',// A Postgres brain whose server_encoding is not UTF8 (doctor/checks/database-encoding.ts).'database_encoding','rls_event_trigger',diff --git a/src/core/function-search-path.ts b/src/core/function-search-path.tsindex b11855bcac..cb0c1d1f23 100644--- a/src/core/function-search-path.ts+++ b/src/core/function-search-path.ts@@ -88,17 +88,72 @@ function isConcurrentCatalogUpdate(error: unknown): boolean {return /tuple concurrently updated/i.test(error instanceof Error ? error.message : String(error));}+/**+ * A failure to re-create a purge or withdrawal guard other than another role owning it. It carries no `code` of its own,+ * so a lock timeout in the re-create never reads as the migration run's lock refusal; the database error is its cause.+ */+export class LedgerGuardRecreateError extends Error {+ constructor(cause: unknown) {+ super(`purge or withdrawal guards not re-created (${cause instanceof Error ? cause.message : String(cause)}); gbrain doctor (guard_ledger_checks) names the ones left without their ledger check`, { cause });+ this.name = 'LedgerGuardRecreateError';+ }+}++/**+ * Re-creates every purge or withdrawal guard left without its ledger check that the role owns (guard-ledger-checks.ts,+ * #7659) and returns the failure, or null. Only a guard another role owns stays silent (SQLSTATE 42501): v318 and doctor's+ * guard_ledger_checks name it, and every later migration run retries it. A concurrent update of the same pg_proc row is+ * retried like the hardening's.+ */+async function recreateLedgerGuards(engine: BrainEngine, attempt = 1): Promise<LedgerGuardRecreateError | null> {+ try {+ await failClosedLedgerGuards(engine);+ return null;+ } catch (error) {+ if ((error as { code?: string } | null)?.code === '42501') return null;+ if (isConcurrentCatalogUpdate(error) && attempt < CONCURRENT_UPDATE_ATTEMPTS) {+ await new Promise(resolve => setTimeout(resolve, 100 * attempt));+ return recreateLedgerGuards(engine, attempt + 1);+ }+ return new LedgerGuardRecreateError(error);+ }+}++/** The stderr line a migration run prints when hardenFunctionSearchPaths throws, naming the doctor check that lists what is left. */+export function hardenFailureLine(error: unknown): string {+ if (error instanceof LedgerGuardRecreateError) return ` guards: ${error.message}\n`;+ const guardFailure = (error as { guardFailure?: unknown } | null)?.guardFailure;+ return ` search_path: pg_temp not restored on gbrain functions (${error instanceof Error ? error.message : String(error)}); gbrain doctor (function_search_path) names them\n`+ + (guardFailure instanceof LedgerGuardRecreateError ? ` guards: ${guardFailure.message}\n` : '');+}++/**+ * Re-creates the guards (recreateLedgerGuards), then puts pg_temp back (pinTempLast) and returns what that left. A+ * failed re-create never skips the pg_temp hardening: it is thrown as a LedgerGuardRecreateError once the hardening has+ * run. When the hardening fails too, its own error is thrown (a lock refusal keeps its meaning for the migration run)+ * with the re-create failure attached as `guardFailure`, which hardenFailureLine prints as well.+ */+export async function hardenFunctionSearchPaths(engine: BrainEngine): Promise<string[] | null> {+ const guardFailure = await recreateLedgerGuards(engine);+ let left: string[] | null;+ try {+ left = await pinTempLast(engine);+ } catch (error) {+ if (guardFailure && error instanceof Error) Object.assign(error, { guardFailure });+ throw error;+ }+ if (guardFailure) throw guardFailure;+ return left;+}+/*** Runs the hardening in one short transaction with the migration statement and lock timeouts and returns the* functions it had to leave (owned by another role), [] when none needed it, or null when the hardening committed but* listing what it left failed. A pg_proc row lock it cannot take* becomes `schema_lock_blocked`; a concurrent update of the same pg_proc row (`apply-migrations --force-schema` beside- * another process's boot) is retried, since the step is idempotent. Prints nothing: the caller decides who is told.- * It first re-creates any purge or withdrawal guard left without its ledger check that the role owns (guard-ledger-checks.ts,- * #7659), best-effort and silent: v318 names the ones another role owns, and every later migration run retries them.+ * another process's boot) is retried, since the step is idempotent.*/-export async function hardenFunctionSearchPaths(engine: BrainEngine, attempt = 1): Promise<string[] | null> {- if (attempt === 1) await failClosedLedgerGuards(engine).catch(() => undefined);+async function pinTempLast(engine: BrainEngine, attempt = 1): Promise<string[] | null> {if ((await legacySearchPathFunctions(engine)).length === 0) return [];const lockTimeoutMs = resolveSchemaLockTimeoutMs();try {@@ -115,7 +170,7 @@ export async function hardenFunctionSearchPaths(engine: BrainEngine, attempt = 1}if (isConcurrentCatalogUpdate(error) && attempt < CONCURRENT_UPDATE_ATTEMPTS) {await new Promise(resolve => setTimeout(resolve, 100 * attempt));- return hardenFunctionSearchPaths(engine, attempt + 1);+ return pinTempLast(engine, attempt + 1);}throw error;}diff --git a/src/core/guard-ledger-checks.ts b/src/core/guard-ledger-checks.tsindex e18fcac25c..4e9dd6be05 100644--- a/src/core/guard-ledger-checks.ts+++ b/src/core/guard-ledger-checks.ts@@ -25,6 +25,19 @@ const LEDGER_GUARDS: ReadonlyArray<{ name: string; ledger: string; sql: string }{ name: 'gbrain_refuse_purging_derivation', ledger: 'page_segment_derivation_purges', sql: REFUSE_PURGING_DERIVATION_FUNCTION_SQL },];+/**+ * Every installed guard whose body lacks its ledger check (regprocedure text and schema-qualified signature), the role+ * that owns it, whether the current role may re-create it, and the current role. Read-only (doctor guard_ledger_checks).+ */+export async function ledgerGuardsWithoutCheck(engine: Pick<BrainEngine, 'executeRaw'>): Promise<Array<{ sig: string; qualified: string; owner: string; owned: boolean; role: string }>> {+ return engine.executeRaw(+ `SELECT p.oid::regprocedure::text AS sig, quote_ident(n.nspname) || '.' || quote_ident(p.proname) || '(' || oidvectortypes(p.proargtypes) || ')' AS qualified,+ pg_get_userbyid(p.proowner)::text AS owner, pg_has_role(current_user, p.proowner, 'USAGE') AS owned, quote_ident(current_user) AS role+ FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace JOIN unnest($1::text[], $2::text[]) AS g(name, ledger) ON g.name = p.proname+ WHERE n.nspname = 'public' AND p.prokind = 'f' AND strpos(p.prosrc, 'row_security_active(''' || g.ledger || ''')') = 0 ORDER BY 1`,+ [LEDGER_GUARDS.map(g => g.name), LEDGER_GUARDS.map(g => g.ledger)]);+}+/*** Re-creates every installed guard whose body lacks its ledger check and that the current role may replace, and returns* the ones another role owns (regprocedure text) with the current role, for the message naming them.diff --git a/src/core/migrate.ts b/src/core/migrate.tsindex 2c27b62e80..e737c89672 100644--- a/src/core/migrate.ts+++ b/src/core/migrate.ts@@ -17,7 +17,7 @@ import {isRetryableConnError,} from './retry-matcher.ts';import { repairTimelineDedupIndex } from './timeline-dedup-repair.ts';-import { hardenFunctionSearchPaths } from './function-search-path.ts';+import { hardenFailureLine, hardenFunctionSearchPaths } from './function-search-path.ts';import { resumePageRevisionBackfill } from './page-state/revision-backfill-schema.ts';import { repairPagesUpsertArbiter } from './pages-upsert-arbiter.ts';import { repairLinkSourceCheck, LINK_SOURCE_GATE_MIGRATION_VERSION } from './link-source-check-repair.ts';@@ -464,7 +464,7 @@ async function runMigrationsLocked(engine: BrainEngine): Promise<{ applied: numb+ 'ALTER FUNCTION needs the owner. Run gbrain doctor for the SQL to run as that role.\n');}} catch (e) {- process.stderr.write(` search_path: pg_temp not restored on gbrain functions (${e instanceof Error ? e.message : String(e)}); gbrain doctor (function_search_path) names them\n`);+ process.stderr.write(hardenFailureLine(e));}};if (pending.length === 0) {diff --git a/test/doctor-guard-ledger-checks.test.ts b/test/doctor-guard-ledger-checks.test.tsnew file mode 100644index 0000000000..6485c93d21--- /dev/null+++ b/test/doctor-guard-ledger-checks.test.ts@@ -0,0 +1,96 @@+/**+ * guard_ledger_checks doctor check (GST-2885, #7632/#7659): names every purge or withdrawal guard whose body lacks its+ * row_security_active('<ledger>') check, with the role that owns it and the fix for that role. A guard the doctor's role+ * may re-create gets the runnable migration fix; one another role owns gets the SQL to run as that role. PGLite has no+ * row-level security, so there the check reports not applicable.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import type { Check } from '../src/commands/doctor.ts';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { testBackends } from './helpers/test-backends.ts';++const backends = testBackends();+const progress = { heartbeat: () => {} };+const run = async (engine: BrainEngine): Promise<Check[]> => {+ const { guardLedgerChecksEntry } = await import('../src/commands/doctor/checks/guard-ledger-checks.ts');+ return await guardLedgerChecksEntry.run({ engine, progress } as never) as Check[];+};+const ROLLBACK = 'guard-ledger-doctor-rollback';+const STALE = `CREATE OR REPLACE FUNCTION gbrain_refuse_purged_fact() RETURNS trigger LANGUAGE plpgsql+ SET search_path = pg_catalog, public, pg_temp AS $fn$ BEGIN RETURN NEW; END $fn$`;+const OK: Check = { name: 'guard_ledger_checks', status: 'ok', message: 'Every purge and withdrawal guard refuses a write role that row-level security keeps from its ledger' };++describe.if(backends.includes('pglite'))('guard_ledger_checks on PGLite', () => {+ let engine: PGLiteEngine;+ beforeAll(async () => { engine = new PGLiteEngine(); await engine.connect({}); await engine.initSchema(); }, 120_000);+ afterAll(async () => { await engine?.disconnect(); });++ test('reports not applicable, even with a guard left without its ledger check', async () => {+ const skipped: Check[] = [{ name: 'guard_ledger_checks', status: 'ok', message: 'Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)' }];+ expect(await run(engine)).toEqual(skipped);+ await engine.executeRaw(STALE);+ expect(await run(engine)).toEqual(skipped);+ }, 120_000);+});++describe.if(backends.includes('postgres'))('guard_ledger_checks on Postgres', () => {+ let engine: BrainEngine;+ let close: () => Promise<void>;+ let me = '';+ const other = `gbrain_guard_doctor_${randomUUID().replaceAll('-', '').slice(0, 12)}`;+ beforeAll(async () => {+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);+ engine = pg.engine; close = pg.close;+ me = (await engine.executeRaw<{ me: string }>('SELECT current_user::text AS me'))[0]!.me;+ await engine.executeRaw(`CREATE ROLE ${other} NOLOGIN`);+ }, 180_000);+ afterAll(async () => {+ await engine?.executeRaw(`DROP ROLE IF EXISTS ${other}`);+ await close?.();+ });++ const rolledBack = async (fn: (tx: BrainEngine) => Promise<void>) => {+ try {+ await engine.transaction(async tx => { await fn(tx); throw new Error(ROLLBACK); });+ } catch (error) {+ if ((error as Error).message !== ROLLBACK) throw error;+ }+ };++ test('a fresh install is clean', async () => {+ expect(await run(engine)).toEqual([OK]);+ });++ test('a guard the role may re-create is named with the migration run that re-creates it', async () => {+ await rolledBack(async tx => {+ await tx.executeRaw(STALE);+ const [check] = await run(tx);+ expect(check).toMatchObject({+ name: 'guard_ledger_checks', status: 'fail',+ fix: { argv: ['gbrain', 'apply-migrations', '--yes', '--no-autopilot-install'], actor: 'agent' },+ details: { functions: ['gbrain_refuse_purged_fact()'], owners: [me] },+ });+ expect(check!.message).toContain('gbrain_refuse_purged_fact() (owner ' + me + ')');+ });+ expect(await run(engine)).toEqual([OK]);+ }, 120_000);++ test('a guard another role owns is named with its owner and the SQL that role runs, and no runnable fix', async () => {+ await rolledBack(async tx => {+ await tx.executeRaw(STALE);+ await tx.executeRaw(`SET LOCAL ROLE ${other}`);+ const [check] = await run(tx);+ const sql = `ALTER FUNCTION public.gbrain_refuse_purged_fact() OWNER TO ${other};`;+ expect(check).toMatchObject({+ name: 'guard_ledger_checks', status: 'fail', fix_unavailable_reason: 'no_safe_automatic_fix',+ details: { functions: ['gbrain_refuse_purged_fact()'], owners: [me], sql: [sql], docs: 'docs/ENGINES.md#purge-and-withdrawal-guards' },+ });+ expect(check!.fix).toBeUndefined();+ expect(check!.message).toContain(`connected as ${me}`);+ expect(check!.message).toContain(sql);+ });+ }, 120_000);+});diff --git a/test/fixtures/goldens/doctor/content-pglite-human.json b/test/fixtures/goldens/doctor/content-pglite-human.jsonindex 397c71d8c9..0939745985 100644--- a/test/fixtures/goldens/doctor/content-pglite-human.json+++ b/test/fixtures/goldens/doctor/content-pglite-human.json@@ -106,6 +106,7 @@" [INFO] guest_source: Not applicable: this brain has no guest source (gbrain guests init)."," [OK] rls: Skipped (PGLite — no PostgREST exposure, RLS not applicable)"," [OK] function_search_path: Every gbrain function that pins a search_path ends it with pg_temp",+ " [OK] guard_ledger_checks: Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)"," [OK] database_encoding: Skipped (PGLite is always UTF8)"," [OK] schema_version: Version <latest> (latest: <latest>)"," [OK] schema_columns: 1393 column(s) verified against live schema",diff --git a/test/fixtures/goldens/doctor/content-pglite-json.json b/test/fixtures/goldens/doctor/content-pglite-json.jsonindex 308107028f..6b33a8eb3a 100644--- a/test/fixtures/goldens/doctor/content-pglite-json.json+++ b/test/fixtures/goldens/doctor/content-pglite-json.json@@ -1326,6 +1326,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",diff --git a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json b/test/fixtures/goldens/doctor/hot-checks-pglite-json.jsonindex c838caa36a..1d8b501eef 100644--- a/test/fixtures/goldens/doctor/hot-checks-pglite-json.json+++ b/test/fixtures/goldens/doctor/hot-checks-pglite-json.json@@ -1407,6 +1407,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",@@ -6872,6 +6879,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",diff --git a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json b/test/fixtures/goldens/doctor/hot-checks-postgres-json.jsonindex bb7183ec9d..340228b3b2 100644--- a/test/fixtures/goldens/doctor/hot-checks-postgres-json.json+++ b/test/fixtures/goldens/doctor/hot-checks-postgres-json.json@@ -1410,6 +1410,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Every purge and withdrawal guard refuses a write role that row-level security keeps from its ledger",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",@@ -6879,6 +6886,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Every purge and withdrawal guard refuses a write role that row-level security keeps from its ledger",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",diff --git a/test/fixtures/goldens/doctor/json-pglite-degraded.json b/test/fixtures/goldens/doctor/json-pglite-degraded.jsonindex c88f5d7629..d367613d53 100644--- a/test/fixtures/goldens/doctor/json-pglite-degraded.json+++ b/test/fixtures/goldens/doctor/json-pglite-degraded.json@@ -1192,6 +1192,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",diff --git a/test/fixtures/goldens/doctor/json-pglite-fresh.json b/test/fixtures/goldens/doctor/json-pglite-fresh.jsonindex 605cfe97ea..76dc505294 100644--- a/test/fixtures/goldens/doctor/json-pglite-fresh.json+++ b/test/fixtures/goldens/doctor/json-pglite-fresh.json@@ -1239,6 +1239,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Skipped (PGLite has no row-level security, so every guard reads its ledgers in full)",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",diff --git a/test/fixtures/goldens/doctor/json-postgres-fresh.json b/test/fixtures/goldens/doctor/json-postgres-fresh.jsonindex e4c6584f2d..968846061f 100644--- a/test/fixtures/goldens/doctor/json-postgres-fresh.json+++ b/test/fixtures/goldens/doctor/json-postgres-fresh.json@@ -1242,6 +1242,13 @@"name": "function_search_path","status": "ok"},+ {+ "category": "ops",+ "elapsed_ms": "<volatile>",+ "message": "Every purge and withdrawal guard refuses a write role that row-level security keeps from its ledger",+ "name": "guard_ledger_checks",+ "status": "ok"+ },{"category": "ops","elapsed_ms": "<volatile>",diff --git a/test/fixtures/goldens/doctor/registry-build-checks.json b/test/fixtures/goldens/doctor/registry-build-checks.jsonindex fbfca6004a..9a46ea1837 100644--- a/test/fixtures/goldens/doctor/registry-build-checks.json+++ b/test/fixtures/goldens/doctor/registry-build-checks.json@@ -481,6 +481,10 @@"category": "ops","name": "function_search_path"},+ {+ "category": "ops",+ "name": "guard_ledger_checks"+ },{"category": "ops","name": "database_encoding"diff --git a/test/fixtures/goldens/migrations/records.json b/test/fixtures/goldens/migrations/records.jsonindex 51a96d0dc5..43e92f733f 100644--- a/test/fixtures/goldens/migrations/records.json+++ b/test/fixtures/goldens/migrations/records.json@@ -73,7 +73,7 @@"failClosedLedgerGuards": "dadd3ceb13286ca7ab969a911a310c87be313de6ec2c2c8e14d3f95b72901481","getFtsLanguage": "0d790b4c4af1f67a2a1bbd59470beb13f434917b381931baf8404a41c5d903db","grantFromRow": "e31bd308b413c11f2879ac4393fbc0362a1a974f937196f11029481e4dc64f69",- "hardenFunctionSearchPaths": "fd4dbb5e6f42931b80b3b041dde4b22100d08a449bd2d8b9c4a6e555e8959dfa",+ "hardenFunctionSearchPaths": "ca7bce96a087bfb4644268e5c2ad0ace76532905ceffb4efdc69608e0ef11894","hnswMaxDimsForType": "a187cdcc520f813b7c39da97ce53da58509232ed9f851627ef41abfe1b68bd26","identityIndexHasSource": "ab7e4cad64ebf233f3a884816c114b0c582f4f2166c3bcf19ed8d92bcfcf3807","isLockTimeoutError": "52a3720e1ffb74ce381cc3269f78130c38a89c807e308283ec3d557bae2b89b1",diff --git a/test/guard-ledger-recreate-errors.test.ts b/test/guard-ledger-recreate-errors.test.tsnew file mode 100644index 0000000000..e3360fa1a7--- /dev/null+++ b/test/guard-ledger-recreate-errors.test.ts@@ -0,0 +1,127 @@+/**+ * hardenFunctionSearchPaths re-creates every purge or withdrawal guard left without its ledger check (GST-2885,+ * #7632/#7659). Only a guard another role owns stays silent (SQLSTATE 42501, the next run of its owner re-creates it,+ * and doctor's guard_ledger_checks names it); any other failure of the re-create propagates to the migration run,+ * which reports it (naming guard_ledger_checks), instead of leaving a guard that lets a purged or withdrawn write+ * through. A concurrent pg_proc update is retried, and a failed re-create never skips the pg_temp hardening.+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import type { BrainEngine } from '../src/core/engine.ts';+import { LedgerGuardRecreateError, hardenFailureLine, hardenFunctionSearchPaths } from '../src/core/function-search-path.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { testBackends } from './helpers/test-backends.ts';++const backends = testBackends();+const ROLLBACK = 'guard-recreate-rollback';++const STALE_ROW = [{ name: 'gbrain_refuse_purged_fact', sig: 'gbrain_refuse_purged_fact()', owned: true, role: 'writer' }];++/** Each executeRaw call takes the next step (rows, or an error to throw); later calls read []. Counts the hardening's transactions. */+function stubEngine(steps: Array<unknown[] | Error>, transactionError?: Error) {+ const state = { transactions: 0 };+ const engine = {+ kind: 'pglite',+ executeRaw: async () => {+ const step = steps.shift() ?? [];+ if (step instanceof Error) throw step;+ return step;+ },+ transaction: async (fn: (tx: unknown) => Promise<void>) => {+ state.transactions++;+ if (transactionError) throw transactionError;+ await fn({ executeRaw: async () => [], runMigration: async () => {} });+ },+ } as unknown as BrainEngine;+ return { engine, state };+}++describe('the guard re-create in hardenFunctionSearchPaths', () => {+ test('another role owning the guard (42501) stays silent', async () => {+ const owner = Object.assign(new Error('must be owner of function gbrain_refuse_purged_fact'), { code: '42501' });+ expect(await hardenFunctionSearchPaths(stubEngine([STALE_ROW, owner]).engine)).toEqual([]);+ });++ test('any other failure propagates, naming guard_ledger_checks, after the pg_temp hardening has run', async () => {+ const broken = Object.assign(new Error('connection terminated'), { code: '08006' });+ const { engine, state } = stubEngine([STALE_ROW, broken, [{ sig: 'gbrain_try_jsonb(text)' }]]);+ const failure = await hardenFunctionSearchPaths(engine).then(() => 'resolved', (error: unknown) => error);+ expect(failure).toBeInstanceOf(LedgerGuardRecreateError);+ expect(failure).toMatchObject({ cause: broken });+ expect((failure as { code?: unknown }).code).toBeUndefined();+ expect(hardenFailureLine(failure)).toBe(' guards: purge or withdrawal guards not re-created (connection terminated); gbrain doctor (guard_ledger_checks) names the ones left without their ledger check\n');+ expect(hardenFailureLine(broken)).toContain('pg_temp not restored');+ expect(state.transactions).toBe(1);+ });++ test('a hardening that fails too keeps its own error, a lock refusal included, and carries the re-create failure', async () => {+ const broken = Object.assign(new Error('connection terminated'), { code: '08006' });+ const lock = Object.assign(new Error('canceling statement due to lock timeout'), { code: '55P03' });+ const { engine } = stubEngine([STALE_ROW, broken, [{ sig: 'gbrain_try_jsonb(text)' }]], lock);+ const failure = await hardenFunctionSearchPaths(engine).then(() => 'resolved', (error: unknown) => error);+ expect(failure).toMatchObject({ code: 'schema_lock_blocked' });+ expect((failure as { guardFailure?: unknown }).guardFailure).toBeInstanceOf(LedgerGuardRecreateError);+ const line = hardenFailureLine(failure);+ expect(line).toContain(' search_path: pg_temp not restored');+ expect(line).toContain(' guards: purge or withdrawal guards not re-created (connection terminated)');+ });++ test('a concurrent pg_proc update during the re-create is retried', async () => {+ const concurrent = new Error('tuple concurrently updated');+ expect(await hardenFunctionSearchPaths(stubEngine([STALE_ROW, concurrent, STALE_ROW, []]).engine)).toEqual([]);+ });+});++for (const backend of backends) {+ describe(`the guard re-create on ${backend}`, () => {+ let engine: BrainEngine;+ let close: (() => Promise<void>) | undefined;+ const other = `gbrain_guard_recreate_${randomUUID().replaceAll('-', '').slice(0, 12)}`;+ beforeAll(async () => {+ if (backend === 'pglite') {+ const lite = new PGLiteEngine(); await lite.connect({}); await lite.initSchema(); engine = lite;+ } else {+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);+ engine = pg.engine; close = pg.close;+ await engine.executeRaw(`CREATE ROLE ${other} NOLOGIN`);+ }+ }, 180_000);+ afterAll(async () => {+ if (backend === 'postgres') await engine?.executeRaw(`DROP ROLE IF EXISTS ${other}`);+ if (close) await close(); else await engine?.disconnect();+ });++ test('a re-create that fails for a reason other than ownership reaches the caller', async () => {+ let failure: unknown;+ try {+ await engine.transaction(async tx => {+ await tx.executeRaw('DROP FUNCTION public.gbrain_refuse_purged_fact() CASCADE');+ await tx.executeRaw("CREATE FUNCTION public.gbrain_refuse_purged_fact() RETURNS integer LANGUAGE sql AS 'SELECT 1'");+ failure = await hardenFunctionSearchPaths(tx).then(() => 'resolved', (error: unknown) => error);+ throw new Error(ROLLBACK);+ });+ } catch (error) {+ if ((error as Error).message !== ROLLBACK) throw error;+ }+ const guardFailure = failure instanceof LedgerGuardRecreateError ? failure : (failure as { guardFailure?: unknown }).guardFailure;+ expect(guardFailure).toBeInstanceOf(LedgerGuardRecreateError);+ expect((guardFailure as Error).cause).toMatchObject({ code: '42P13' });+ expect((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pg_trigger WHERE tgfoid = 'public.gbrain_refuse_purged_fact()'::regprocedure"))[0]!.n).toBeGreaterThan(0);+ }, 120_000);++ test.if(backend === 'postgres')('a guard another role owns is left silently', async () => {+ try {+ await engine.transaction(async tx => {+ await tx.executeRaw(`CREATE OR REPLACE FUNCTION gbrain_refuse_purged_fact() RETURNS trigger LANGUAGE plpgsql+ SET search_path = pg_catalog, public, pg_temp AS $fn$ BEGIN RETURN NEW; END $fn$`);+ await tx.executeRaw(`SET LOCAL ROLE ${other}`);+ expect(await hardenFunctionSearchPaths(tx)).toEqual([]);+ throw new Error(ROLLBACK);+ });+ } catch (error) {+ if ((error as Error).message !== ROLLBACK) throw error;+ }+ }, 120_000);+ });+}diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex c952d75481..721bf69b66 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -99,6 +99,7 @@ test/direct-pool-profile.test.tstest/doctor-attest-read-shared-scope.test.tstest/doctor-database-encoding.test.tstest/doctor-frontmatter-holds.test.ts+test/doctor-guard-ledger-checks.test.tstest/doctor-image-embedding-coverage.test.tstest/doctor-keyword-plan-6495.test.tstest/doctor-purged-page-facts-count.test.ts@@ -189,6 +190,7 @@ test/graduation-target.test.tstest/graduation-verify-sync-freshness.test.tstest/graduation-verify.test.tstest/grandfather-managed.test.ts+test/guard-ledger-recreate-errors.test.tstest/guest-notes-scoped-role.test.tstest/harness-skills-epoch.test.tstest/harness-verify-member-credential.test.ts