Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(sources): retire batches refuse before deleting on a spent receipt quota or a revoked CLI writer, and halve on the statement bound
garrytancaf2f3f5fd24
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 8cadb056cf..b56f35741e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,20 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**`gbrain upgrade` no longer consults the ClawHub registry; a large Google address book, and a large source's removal, can now finish.** Advisory: an unofficial package on the ClawHub registry shares the name gbrain. `gbrain upgrade`, and autopilot's unattended self-upgrade, treated any install it could not otherwise classify (for example a release binary run under its published file name, `gbrain-linux-x64`) as a ClawHub install whenever the `clawhub` CLI was on the host, and ran `clawhub update gbrain`, which installs whatever the registry resolves for that name from any publisher. Upgrade now never runs it: a compiled binary is recognised whatever its file name, and an install it cannot classify prints the manual steps. When Google's People sync token expires, the contacts re-list now checkpoints after every page and resumes there, so a large address book no longer restarts from page one in every sync job and Calendar and Gmail sync again. A managed `gbrain sources remove` or `purge` of a large source now deletes its pages in bounded batches under the 5-second statement bound instead of one cascading statement that could never finish, and a rerun resumes. The secret scanner also catches keys right after terminal color codes, `\xNN` and `\UXXXXXXXX` escapes and quoted-printable `=3D`, and redacts a bearer token containing an escaped slash in full.++### Itemized changes++- `src/commands/upgrade.ts`: `detectInstallMethod` drops the `clawhub` method and its `clawhub --version` probe; a process whose `argv[1]` is Bun's compiled entrypoint (`/$bunfs/`, `B:\~BUN\`) is `binary`; the `clawhub` upgrade branch and hint are gone. `src/core/self-upgrade.ts`: `canSelfUpdate` no longer accepts `clawhub`. `src/commands/check-update.ts`: no `clawhub update` hint (#8464).+- `src/core/google/people-calendar-sweep.ts`, `google-clients.ts`: a full contacts re-list goes page by page (`listConnectionsPage`), saves `contacts_relist` (next page token and field mask) after each page lands through the same state store as Gmail's checkpoints (no git, so repo-less brains work), resumes there, restarts once from page 1 when Google rejects a saved page token (400/410), and lists at most 10 pages per run so Calendar and Gmail still run (#8906).+- `src/core/persistence/source-lifecycle.ts`: managed `remove`/`purge` deletes pages 500 at a time, each batch its own bounded transaction that first rechecks the final transaction's preconditions (including the CLI writer registration and the receipt quotas, `assertTopologyChangeCapacity`) and halves its size when it hits the statement bound; a refusal after committed batches says how many pages that run deleted. `topology-transaction.ts`: the `write_pending` hint says a retry resumes (#7702, #8905).+- `src/core/secret-scan.ts`: the escape-boundary twins also fire after `ESC[…m` (raw or written `\u001b[`, `\x1b[`, `\033[`), `\xNN`, `\UXXXXXXXX` and `=3D`; the `bearer` value accepts `\/` (#8266).+- Tests: `test/upgrade.serial.test.ts`, `test/self-upgrade.test.ts` (a renamed binary or an unclassified install with a working `clawhub` CLI never runs it); `test/contacts-relist-checkpoint-8906.slow.test.ts`, `test/contacts-relist-checkpoint-managed-8906.test.ts`; `test/source-retire-batched.test.ts`, `test/e2e/source-retire-batched-postgres.test.ts` (60,000 pages); `test/secret-scan-encoded-boundary-8266.test.ts`.++Reported and first fixed by @javieraldape (#8910, #8764) and @Masashi-Ono0611 (#7704, #8907, #8908).+## [0.60.620.0] - 2026-11-09**A diverged managed checkout now publishes its local commits on refresh by default when Git durability is on, after a secret scan of those commits.** Commits a script made with plain `git commit` in a managed checkout used to strand there whenever the remote moved: every `gbrain sources refresh` refused `refresh_diverged` and asked the user, unless `sources.refresh_publish_diverged` was set to `true`. With the setting unset, a worktree whose Git durability is on (`gbrain sources writer git-durability <source> --enable`, or the legacy hook) now publishes them itself: the local commits merged with the upstream in memory, never a rebase or force-push, then the fast-forward. The publish child scans, under its publish lock and for the exact commits it pushes, what they add: commit messages, added lines (including a value a later commit removed) and merge-commit conflict resolutions. A value that looks like a secret, added text too large to scan, or a commit that changes `.gbrain-scan-allow` or a `.gitattributes` refuses the new `publish_secret_found` naming the file, line, rule and commit, never the value, with nothing pushed; commits the scan cannot read in full refuse `publish_scan_unreadable` instead. The rewritten-upstream refusal and the `--expect-remote` bound are unchanged. `false` turns it off; `true` still publishes with durability off.diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.mdindex bc157b81cd..d4dc45df41 100644--- a/docs/architecture/key-files/core-persistence-continued.md+++ b/docs/architecture/key-files/core-persistence-continued.md@@ -41,4 +41,4 @@ The #6188 fence check and fence repair live in [fence repair](core-fence-repair.- `src/core/postgres-engine/reserved-transactions.ts` with `isPersistenceDeclaration` (`src/core/persistence/protocol.ts`) — BEGIN pipelining. `isPersistenceDeclaration` (GBRA-156) accepts exactly the `protocol.ts` declarations (transaction-local `set_config` calls and the brain row FOR SHARE). Admission, the FIFO claim, the effect claims and source guards, the receipt true-up, the single publication (`coordinator.ts`) and the consumer's lent lane transactions (`consumer.ts` `onLane`, through `reservedTransactions`) pass it as `TransactionOptions.pipelineFirst`, so behind a transaction-mode pooler the declaration goes out with BEGIN (one round trip less per transaction) while every later statement still waits for BEGIN's answer; if BEGIN fails, only the declaration has run, in autocommit, where it changes nothing. A reserved transaction gates every statement its body can reach, savepoint and nested-transaction bodies and `file()` included. A sync lane group's publication passes none: its begun mark must follow BEGIN's answer (`sync-lanes.ts` `awaitLaneBegin`). Tests: `test/persistence-pipeline-first.test.ts` (the predicate and the reviewed call sites), `test/e2e/pgbouncer-transaction-pipeline-6930.test.ts`.- `src/core/links-fence.ts` + `src/core/links-fence-projection.ts` — the `## Links` fence (G5): the markdown form of authored (manual) edges, `<!--- gbrain:links:begin -->` / `end` around `| to | type | source | valid_from | valid_until | context |`, one row per (to, type, source), source defaulting to `manual`; a reconciliation-managed provenance (`MANAGED_LINK_SOURCES`, exported here and re-exported by `ops/links.ts`), a bad date or a duplicate key is a skipped row with a warning. `parseLinksFence` (state `absent` / `parsed` / `malformed`), the deterministic `renderLinksFence` (rows sorted by key), `upsertLinksFenceRow` (omitted context and dates keep the stored ones; identical row = unchanged body), `removeLinksFenceRows` (leaves an empty fence), `replaceOrInsertLinksFence` (a `## Links` section at the end of compiled_truth, above an unclosed code block) and `maskLinksFence`, which `extractBodyLinks` applies so the fence cells are never also extracted as markdown, wikilink or bare-slug edges. A rewrite of a fence that parses with warnings throws `LinksFenceRewriteRefusal`. Cells use `escapeFenceCell` plus the same one-level marker encoding for `gbrain:links:*`. `projectLinksFence(tx, page, rows)` reconciles only what the page's fence owns: edges with origin_page_id = the page and origin_field `links_fence` (deleted, context-updated, inserted by key) and `manual` transitions with origin_page_id = the page, then refreshes the touched relationships; a row whose target is not a live page in the source yields no edge. `linksFenceProjectionRows` returns null for an absent or malformed fence, so such a body changes no edge (an absent fence retires nothing, as for facts; unlike facts no held generation is recorded, so after a hand deletion of the whole section the database keeps edges git no longer has until `remove_link` or the G3 indexer), and drops rows typed with a link type the active pack does not declare. Callers: `prepareCanonicalProjections` (every projected page publication, quarantined and segmented-unread bodies excepted), `prepareAutomaticLinks` (put_page auto-links and the managed links maintenance, so `extract --stale` restores a lost fence edge) and `persistence/manual-links.ts`. Database-only manual edges (origin_field NULL) are never touched by any of them. Pinned by `test/links-fence.test.ts` and `test/managed-links-fence.test.ts`.-- `src/core/persistence/source-lifecycle.ts` retire batches with `topology-transaction.ts` (#7702/#8905) — every `topologyTransaction` runs under `statement_timeout=5s` and `lock_timeout=1s` and maps SQLSTATE 57014/55P03 to `write_pending`, so a managed `remove` or `purge` first deletes the source's pages in batches of `SOURCE_RETIRE_BATCH_PAGES` (500; about 110 ms per batch with chunks and links on Postgres) under the native locks `withTopologyLocks` holds, each batch its own `topologyTransaction`. A batch re-checks the final transaction's preconditions (admin state, `lockTopologyRows`, the incarnation read before the locks, archived or `EXPIRED_ARCHIVE_SQL` due, no prior receipt, no `oauth_clients` reference, `assertTopologyRequestsSettled` from `topology-locks.ts`) and, when one would refuse or replay, stops without deleting so the final transaction reports it; a dry run, a missing checkout, an unconfirmed operation and an admission-hooked call never batch. Each batch deletes inside `withCoordinatedWrite` with local-CLI attribution (the `pages` guard needs the source in `gbrain.write_sources`; only the final `sources` delete sets `gbrain.topology_change`). Committed batches are durable: a retry (same or new request_id) resumes, and the receipt's `pages_deleted` counts the pages deleted by the run that commits it. Tests: `test/source-retire-batched.test.ts` (PGLite) and `test/e2e/source-retire-batched-postgres.test.ts` (60,000 pages).+- `src/core/persistence/source-lifecycle.ts` retire batches with `topology-transaction.ts` (#7702/#8905) — every `topologyTransaction` runs under `statement_timeout=5s` and `lock_timeout=1s` and maps SQLSTATE 57014/55P03 to `write_pending`, so a managed `remove` or `purge` first deletes the source's pages in batches of `SOURCE_RETIRE_BATCH_PAGES` (500; about 110 ms per batch with chunks and links on Postgres) under the native locks `withTopologyLocks` holds, each batch its own `topologyTransaction`. A batch re-checks the final transaction's preconditions (admin state, `lockTopologyRows`, the incarnation read before the locks, archived or `EXPIRED_ARCHIVE_SQL` due, no prior receipt, no `oauth_clients` reference, `assertTopologyRequestsSettled` from `topology-locks.ts`, `lockTopologyPrincipal`, and `assertTopologyChangeCapacity` from `topology-receipts.ts`, the receipt quotas at the largest receipt) and, when one would refuse or replay, stops without deleting so the final transaction reports it, or throws that refusal before deleting; a batch that fails `write_pending` is retried at half the size (floor 1), so heavy pages still fit the bound; a refusal after some batches committed says how many pages that run already deleted; a dry run, a missing checkout, an unconfirmed operation and an admission-hooked call never batch. Each batch deletes inside `withCoordinatedWrite` with local-CLI attribution (the `pages` guard needs the source in `gbrain.write_sources`; only the final `sources` delete sets `gbrain.topology_change`). Committed batches are durable: a retry (same or new request_id) resumes, and the receipt's `pages_deleted` counts the pages deleted by the run that commits it. Tests: `test/source-retire-batched.test.ts` (PGLite) and `test/e2e/source-retire-batched-postgres.test.ts` (60,000 pages).diff --git a/docs/architecture/key-files/core-utilities-1.md b/docs/architecture/key-files/core-utilities-1.mdindex 7a321d7383..7a43f4ab1c 100644--- a/docs/architecture/key-files/core-utilities-1.md+++ b/docs/architecture/key-files/core-utilities-1.md@@ -13,7 +13,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/brain-score-recommendations.ts` — pure data layer consumed by both `gbrain doctor --remediation-plan` / `--remediate` and `gbrain features`. Extraction-watermark staleness produces source-scoped `extract.stale` against database pages, including DB-only pages without a checkout; it does not imply a filesystem sync or block embedding on one. `RecommendationContext.staleExtractionBlocked` (probed by `staleExtractionBlocked()` in `remediation/context.ts`, shared with autopilot) withholds `extract.stale` and classifies `links_extraction_lag` as blocked while the remediated source, or any source with stale pages in an all-source sweep, cannot load its active schema pack, so no dispatch fails every cycle. `computeRecommendations(checks, opts)` returns `Remediation[]` with stable `id`, content-hash `idempotency_key`, `severity`, `est_seconds`, `est_usd_cost`, `depends_on` (references stable ids, not check names — so plan order is reproducible). `classifyChecks(report)` triages every doctor check three-state into `remediable | human_only | blocked` (`human_only` covers RLS warnings and other human-judgment gates; `blocked` covers dependency chains where a parent check failed). `maxReachableScore(checks)` computes the ceiling for empty/under-configured brains (no entity pages → graph_coverage caps at 70; no embedding key → embedding_coverage caps at 60). Cost estimates pull from `anthropic-pricing.ts` (synthesize/patterns/consolidate) and `embedding-pricing.ts` (embed jobs). Pinned by `test/brain-score-recommendations.test.ts` (~27 cases incl. determinism, content-hash idempotency, DB-backed checkpoint provenance, three-state triage).-- `src/core/bun-floor.ts` — the Bun floor of an upgrade target (#5855): `readBunLinkTarget` (`git fetch`, the fetched upstream SHA, `package.json` at that SHA; the swap fast-forwards to that SHA), `readPackageTarget` (`package.json` at the ref a global install's `github:` spec names, default branch when unpinned; ClawHub reads the canonical default branch), `parseReleasePackage` (only a `>=X.Y.Z` `engines.bun` is a floor; any other shape is unreadable), `readHostBun` (the lower of the `bun` on PATH and the Bun at `process.execPath`) and `evaluateBunFloor` (the refusal text with path, and a path-free copy for the self-upgrade audit). `BUN_FLOOR_EXIT_CODE` (78) marks a `gbrain upgrade` refused for the floor, which the autopilot channel treats as a hold. Tests: `bun-floor.test.ts`, `autopilot-bun-floor.test.ts`, `upgrade-bun-link-arc.serial.test.ts`.+- `src/core/bun-floor.ts` — the Bun floor of an upgrade target (#5855): `readBunLinkTarget` (`git fetch`, the fetched upstream SHA, `package.json` at that SHA; the swap fast-forwards to that SHA), `readPackageTarget` (`package.json` at the ref a global install's `github:` spec names, default branch when unpinned; a null spec reads the canonical default branch), `parseReleasePackage` (only a `>=X.Y.Z` `engines.bun` is a floor; any other shape is unreadable), `readHostBun` (the lower of the `bun` on PATH and the Bun at `process.execPath`) and `evaluateBunFloor` (the refusal text with path, and a path-free copy for the self-upgrade audit). `BUN_FLOOR_EXIT_CODE` (78) marks a `gbrain upgrade` refused for the floor, which the autopilot channel treats as a hold. Tests: `bun-floor.test.ts`, `autopilot-bun-floor.test.ts`, `upgrade-bun-link-arc.serial.test.ts`.- `src/core/orphan-policy.ts` — the shared orphan-reporting policy (`gbrain orphans`, doctor's orphan ratio, `get_health`): convention constants plus per-brain `orphans.exclude_prefixes` / `orphans.exclude_slugs` overrides. Two renderers over the same constants: `shouldExcludeFromOrphanReporting(slug, overrides, meta)` (TypeScript) and `orphanExclusionSql(alias, overrides)` (a never-NULL SQL predicate; every list value is a bound parameter, only the identifier-checked alias is spliced; regexes are spelled for Postgres's engine: ASCII digits, `.` stops at line terminators). Quarantine stays outside the SQL predicate. Parity pinned on a generated slug/type corpus by `test/orphan-policy-sql-parity.test.ts` and its Postgres arm in `test/e2e/engine-sql-health-parity.test.ts`.- `src/core/planner-stats.ts` + `src/core/planner-stats-schema.ts` — F4b row-delta ANALYZE for PGLite (no autovacuum). Migration f4_planner_stats (`sqlFor.pglite` only; a Postgres no-op) adds `planner_stats_deltas` / `planner_stats_state` and AFTER STATEMENT transition-table triggers on pages, links, facts, takes, content_chunks and timeline_entries, so modification counts commit or roll back with the data and survive restarts. `maybeRefreshPlannerStats(engine, reason)` ANALYZEs tables whose pending count exceeds max(500, 10% of reltuples) (or that have rows but no pg_stats) and publishes a watermark read before the ANALYZE, so a concurrent modification stays pending; checks are throttled to one per 2 s per engine. Call sites: PGLite planner-sensitive reads (`beforePlannerRead`, deferring a table whose last ANALYZE exceeded `planner.first_read_budget_ms`), import and managed sync every `import.analyze_every_pages`, stale link drains and the `extract --source db` links walk of at least `LINK_DRAIN_ANALYZE_MIN_STALE` (500) pages through `plannerStatsForLinkDrain` (first page, then the same cadence; `link_transitions`/`link_relationships`, no deltas, with `links`), the cycle after extract_facts, the consumer idle tick, and `refreshProjectionStatistics` (full ANALYZE publishes every watermark). `planner.auto_analyze=false` / `GBRAIN_PLANNER_AUTO_ANALYZE=0` disables them all. Doctor `planner_stats_stale` (Postgres: `pg_stat_user_tables` plus absent `missingSearchStatistics`) and repair kind `planner-stats` keep working. `readExpressionIndexesWithoutStats` (both engines; lhf wave 2 item 6) lists expression indexes on the hot tables with entries (index `reltuples > 0`) but no `pg_stats` row under their own name, skipping indexes whose statistics the role cannot see; every `maybeRefreshPlannerStats` reason except `first_read` analyzes their tables (a search never pays the probe), doctor names each with its own reason, and the repair plans their tables (on Postgres the operator-run recovery). Guide: `docs/guides/planner-stats.md`. Tests: `test/planner-stats*.test.ts` (`planner-stats-link-drain.test.ts` EXPLAINs the drain's snapshot read), `test/e2e/planner-stats-postgres.test.ts`.diff --git a/src/core/persistence/source-lifecycle.ts b/src/core/persistence/source-lifecycle.tsindex a3b295d6af..2d8da653a2 100644--- a/src/core/persistence/source-lifecycle.ts+++ b/src/core/persistence/source-lifecycle.ts@@ -17,7 +17,7 @@ import { containsPath, getWorktreeBinding, humanManifestProgress, type WorktreeBimport { manifestDifferences, storedManifestScope, type WorktreeManifestScope } from './worktree-manifest.ts';import { hostIdFor } from './identity.ts';import { advanceTopology, assertTopologyRequestsSettled, inspectTopologyPhysicalRoot, lockTopologyPrincipal, lockTopologyRows, settleTopologyRequests, topologyCanonicalStamp, topologyPrincipal, withTopologyLocks } from './topology-locks.ts';-import { priorTopologyChange, recordTopologyChange, topologyReceipt } from './topology-receipts.ts';+import { assertTopologyChangeCapacity, priorTopologyChange, recordTopologyChange, topologyReceipt } from './topology-receipts.ts';import { isWriteRequestId } from './types.ts';import { withCoordinatedWrite } from './context.ts';import { principalAttribution } from './attribution.ts';@@ -185,10 +185,12 @@ export async function runManagedSourceLifecycle(engine:BrainEngine,input:SourceLmanifests.set(root.worktree,worktreeManifest(root.worktree,{progress:humanManifestProgress(),scope:storedManifestScope(manifests.get(currentRoot))}));// #7702/#8905: a remove or purge first deletes the source's pages in bounded batches, each its own transaction behind// the final transaction's preconditions; any precondition that would refuse or replay stops batching and the final- // transaction reports it. Committed batches are durable, so a retry with the same request_id resumes.+ // transaction reports it. Committed batches are durable, so a retry with the same request_id resumes. A batch that+ // hits the statement bound is retried at half the size, so heavy pages still fit.let retiredPages=0;+ try{if(['remove','purge'].includes(input.operation)&&input.confirmDestructive&&!missingCheckout&&!admission)- for(let deleted=SOURCE_RETIRE_BATCH_PAGES;deleted===SOURCE_RETIRE_BATCH_PAGES;) deleted=await topologyTransaction(engine,async tx=>{+ for(let limit=SOURCE_RETIRE_BATCH_PAGES,deleted=limit;deleted===limit;) try{ deleted=await topologyTransaction(engine,async tx=>{await assertWriterAdminState(tx,input.expectedAdminState);const sources=await lockTopologyRows(tx,input.sourceId,bindings);const [source]=await tx.executeRaw<{incarnation:string;archived:boolean;due:boolean}>(@@ -197,13 +199,18 @@ export async function runManagedSourceLifecycle(engine:BrainEngine,input:SourceL||await priorTopologyChange(tx,principal,requestId,intent)||(await tx.executeRaw('SELECT client_id FROM oauth_clients WHERE source_id=$1 LIMIT 1',[input.sourceId])).length) return 0;await assertTopologyRequestsSettled(tx,sources,bindings.map(binding=>binding.worktree_id));+ await lockTopologyPrincipal(tx,principal);+ await assertTopologyChangeCapacity(tx,{principal,operation:input.operation,sourceId:input.sourceId});const gone=await withCoordinatedWrite(tx,sources,()=>tx.executeRaw(- 'DELETE FROM pages WHERE id IN (SELECT id FROM pages WHERE source_id=$1 LIMIT $2) RETURNING id',[input.sourceId,SOURCE_RETIRE_BATCH_PAGES]),+ 'DELETE FROM pages WHERE id IN (SELECT id FROM pages WHERE source_id=$1 LIMIT $2) RETURNING id',[input.sourceId,limit]),principalAttribution({kind:'local_cli',id:principal}));retiredPages+=gone.length;return gone.length;- });- return topologyTransaction(engine,async tx=>{+ }); }catch(error){+ if(!(error instanceof OperationError&&error.code==='write_pending'&&limit>1)) throw error;+ deleted=limit=Math.ceil(limit/2);+ }+ return await topologyTransaction(engine,async tx=>{await assertWriterAdminState(tx,input.expectedAdminState);await tx.executeRaw("SELECT set_config('synchronous_commit','on',true)");const sources=await lockTopologyRows(tx,input.sourceId,bindings);@@ -297,7 +304,7 @@ export async function runManagedSourceLifecycle(engine:BrainEngine,input:SourceL}else{const refs=await tx.executeRaw<{client_id:string}>('SELECT client_id FROM oauth_clients WHERE source_id=$1 LIMIT 1',[input.sourceId]);if(refs.length) throw opError('source_referenced','OAuth clients still reference this source. Revoke and remove those registrations first.',- `Nothing was removed. OAuth client ${refs[0].client_id} (and possibly others) is bound to '${input.sourceId}'; list them with the command in fix and ask the user before revoking any, since revoking disconnects that client.`,+ `${retiredPages?`Only the ${retiredPages} page(s) this run already deleted were removed.`:'Nothing was removed.'} OAuth client ${refs[0].client_id} (and possibly others) is bound to '${input.sourceId}'; list them with the command in fix and ask the user before revoking any, since revoking disconnects that client.`,{fix:readFix('Lists OAuth clients with the source each is bound to.',{argv:['gbrain','auth','clients','--json']})});const [impact]=await tx.executeRaw<{count:string}>('SELECT count(*)::text AS count FROM pages WHERE source_id=$1',[input.sourceId]);pagesDeleted+=Number(impact.count);@@ -323,6 +330,11 @@ export async function runManagedSourceLifecycle(engine:BrainEngine,input:SourceLconst row=await recordTopologyChange(tx,{principal,requestId,intent,operation:input.operation,sourceId:input.sourceId,incarnation:source?.incarnation??String(result.source_incarnation),worktrees},result);return topologyReceipt(row);});+ }catch(error){+ if(retiredPages&&error instanceof OperationError&&!error.suggestion?.startsWith('Only the'))+ error.suggestion=`This run had already deleted ${retiredPages} page(s) of source '${input.sourceId}' in committed batches before it stopped; the source and its remaining pages are still registered. ${error.suggestion??''}`.trim();+ throw error;+ }},root?.worktree,5000,{relocation:input.operation==='rebind'});}diff --git a/src/core/persistence/topology-receipts.ts b/src/core/persistence/topology-receipts.tsindex d4bbee1e68..30f2ce415c 100644--- a/src/core/persistence/topology-receipts.ts+++ b/src/core/persistence/topology-receipts.ts@@ -35,19 +35,11 @@ export async function recordTopologyChange(tx:BrainEngine, input:{principal:strifor(const counter of counters){const brain=counter.key==='brain';if(!counter.key.startsWith('worktree:')){+ await assertQuota(tx,counter,limits,terminalBytes,input);if(recovery && (Number(counter.outstanding_count)+1>(brain?limits.brainOutstanding:limits.principalOutstanding)||Number(counter.intent_bytes)+intentBytes>(brain?limits.brainIntentBytes:limits.principalIntentBytes)))throw opError('queue_capacity','Pending lifecycle work exceeds available brain/principal request or intent capacity.',`Too much lifecycle work is still pending, so the ${input.operation} of source ${input.sourceId} was not accepted. Let the pending work settle, then submit it again; raising ${journalLimitKey(brain?'brainOutstanding':'principalOutstanding')} is the user's call.`, { fix: { argv: ['gbrain', 'sources', 'writer', 'status', '--source', input.sourceId, '--json'], consent: [], actor: 'agent', why: 'Shows the pending lifecycle and recovery work that holds this capacity.', requires_exclusive: false } });- if(Number(counter.lifetime_ids)+1>(brain?limits.brainLifetimeIds:limits.principalLifetimeIds)- ||Number(counter.terminal_bytes)+terminalBytes>(brain?limits.brainTerminalBytes:limits.principalTerminalBytes))- {- const ids=Number(counter.lifetime_ids)+1>(brain?limits.brainLifetimeIds:limits.principalLifetimeIds);- const setting=`${brain?'brain':'principal'}${ids?'LifetimeIds':'TerminalBytes'}` as const;- const used=Number(ids?counter.lifetime_ids:counter.terminal_bytes);- throw quotaExhausted(`${brain?'brain':'principal'} ${ids?'permanent request IDs':'reserved receipt bytes'} (lifecycle ${input.operation} of source ${input.sourceId})`,- journalLimitKey(setting),await oneYearCapacity(tx,counter.key,ids?'LifetimeIds':'TerminalBytes',used,limits[setting]),used,limits[setting],ids?'lifetime_ids':'terminal_bytes');- }}if(recoveryBytes && counter.key!==key && Number(counter.recovery_bytes)+recoveryBytes>(brain?limits.brainRecoveryBytes:limits.worktreeRecoveryBytes))throw opError('queue_capacity','The lifecycle recovery exceeds available brain/worktree capacity.',@@ -65,6 +57,23 @@ export async function recordTopologyChange(tx:BrainEngine, input:{principal:stri}return row;}+type JournalLimits=Awaited<ReturnType<typeof readJournalLimits>>;+type Counter=Awaited<ReturnType<typeof lockCounters>>[number];+async function assertQuota(tx:BrainEngine,counter:Counter,limits:JournalLimits,terminalBytes:number,input:{operation:string;sourceId:string}):Promise<void>{+ const brain=counter.key==='brain';+ const ids=Number(counter.lifetime_ids)+1>(brain?limits.brainLifetimeIds:limits.principalLifetimeIds);+ if(!ids&&Number(counter.terminal_bytes)+terminalBytes<=(brain?limits.brainTerminalBytes:limits.principalTerminalBytes)) return;+ const setting=`${brain?'brain':'principal'}${ids?'LifetimeIds':'TerminalBytes'}` as const;+ const used=Number(ids?counter.lifetime_ids:counter.terminal_bytes);+ throw quotaExhausted(`${brain?'brain':'principal'} ${ids?'permanent request IDs':'reserved receipt bytes'} (lifecycle ${input.operation} of source ${input.sourceId})`,+ journalLimitKey(setting),await oneYearCapacity(tx,counter.key,ids?'LifetimeIds':'TerminalBytes',used,limits[setting]),used,limits[setting],ids?'lifetime_ids':'terminal_bytes');+}+/** #7702/#8905: the lifetime-ID and receipt-byte quotas recordTopologyChange enforces, at the largest receipt it can record, so a retire refuses before it deletes anything. */+export async function assertTopologyChangeCapacity(tx:BrainEngine,input:{principal:string;operation:string;sourceId:string}):Promise<void>{+ const limits=await readJournalLimits(tx);+ for(const counter of await lockCounters(tx,['brain',principalKey({kind:'local_cli',id:input.principal})]))+ await assertQuota(tx,counter,limits,receiptReservation({}),input);+}/** Caller retains owner/source guards and completes final state in this same transaction. */export async function releaseTopologyReservation(tx:BrainEngine,row:TopologyChange):Promise<boolean>{// Revocation or removal prevents publication, never cleanup of its priordiff --git a/test/source-retire-batched.test.ts b/test/source-retire-batched.test.tsindex 78e9f50251..de9fa6daae 100644--- a/test/source-retire-batched.test.ts+++ b/test/source-retire-batched.test.ts@@ -15,6 +15,7 @@ import { registerLocalWriter } from '../src/core/persistence/identity.ts';import { claimWorktree } from '../src/core/persistence/ownership.ts';import { runManagedSourceLifecycle, SOURCE_RETIRE_BATCH_PAGES } from '../src/core/persistence/source-lifecycle.ts';import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { journalLimitKey } from '../src/core/persistence/limits.ts';let engine: PGLiteEngine;@@ -106,3 +107,35 @@ test('an interrupted batch keeps the committed batches and the same request_id rfor (const sql of ['DROP TRIGGER IF EXISTS retire_fail ON pages', 'DROP FUNCTION IF EXISTS retire_fail_fn()', 'DROP TABLE IF EXISTS retire_fail']) await engine.executeRaw(sql);}}), 60_000);++test('a remove refuses before deleting when the receipt quota is spent, and a CLI writer revoked mid-run stops it at the next batch', () => fixture(async () => {+ const [brain] = await engine.executeRaw<{ n: string }>("SELECT lifetime_ids::text AS n FROM persistence_counters WHERE key='brain'");+ await engine.executeRaw('INSERT INTO config(key,value) VALUES($1,$2) ON CONFLICT(key) DO UPDATE SET value=EXCLUDED.value', [journalLimitKey('brainLifetimeIds'), brain?.n ?? '0']);+ await expect(runManagedSourceLifecycle(engine, { operation: 'remove', sourceId: source, confirmDestructive: true })).rejects.toMatchObject({ code: 'write_quota_exhausted' });+ expect(await pages(source)).toBe(PAGES);+ await engine.executeRaw('DELETE FROM config WHERE key=$1', [journalLimitKey('brainLifetimeIds')]);+ const revoke = "CREATE FUNCTION retire_revoke_fn() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN UPDATE persistence_local_writers SET revoked_at=now() WHERE lane='cli'; RETURN NULL; END $$";+ await engine.executeRaw(revoke);+ await engine.executeRaw('CREATE TRIGGER retire_revoke AFTER DELETE ON pages FOR EACH STATEMENT EXECUTE FUNCTION retire_revoke_fn()');+ try {+ await expect(runManagedSourceLifecycle(engine, { operation: 'remove', sourceId: source, confirmDestructive: true }))+ .rejects.toMatchObject({ code: 'permission_denied', suggestion: expect.stringContaining(`already deleted ${SOURCE_RETIRE_BATCH_PAGES} page(s)`) });+ expect(await pages(source)).toBe(PAGES - SOURCE_RETIRE_BATCH_PAGES);+ expect(await registered()).toBe(1);+ } finally {+ for (const sql of ['DROP TRIGGER IF EXISTS retire_revoke ON pages', 'DROP FUNCTION IF EXISTS retire_revoke_fn()']) await engine.executeRaw(sql);+ }+}), 60_000);++test('a batch that hits the statement bound is retried at half the size until it fits', () => fixture(async () => {+ await engine.executeRaw(`CREATE FUNCTION retire_bound_fn() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN+ IF (SELECT count(*) FROM gone) > 100 THEN RAISE EXCEPTION 'canceling statement due to statement timeout' USING ERRCODE = 'query_canceled'; END IF; RETURN NULL; END $$`);+ await engine.executeRaw('CREATE TRIGGER retire_bound AFTER DELETE ON pages REFERENCING OLD TABLE AS gone FOR EACH STATEMENT EXECUTE FUNCTION retire_bound_fn()');+ try {+ expect(await runManagedSourceLifecycle(engine, { operation: 'remove', sourceId: source, confirmDestructive: true }))+ .toMatchObject({ state: 'committed', operation: 'remove', pages_deleted: PAGES });+ expect(await pages(source)).toBe(0);+ } finally {+ for (const sql of ['DROP TRIGGER IF EXISTS retire_bound ON pages', 'DROP FUNCTION IF EXISTS retire_bound_fn()']) await engine.executeRaw(sql);+ }+}), 60_000);