Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix: the receipt target counts only receipts housekeeping can still compact, and the backup listing names no count bound while the id cap is unset
garrytan94386507363f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 68a7db121f..4e50fe74b0 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -12,16 +12,17 @@ identifiers and attribution are available in the pre-removal Git revision## [Unreleased]-**gbrain's own write bookkeeping no longer caps your writes.** Every accepted write leaves a receipt, and gbrain used to cap how many request ids and how many receipt bytes one writer and one brain could hold (2,000,000 / 8,000,000 ids, 16 GiB / 64 GiB). Near those caps, background writers paused at 80% and every write receipt carried a low-capacity warning; at the cap, `remember`, `put_page` and sync refused with `write_quota_exhausted` until someone on the brain host raised a number. Nothing about your data needed that: a cap only stopped writes. Both caps are now off unless the operator sets one. Receipt storage is kept in check by housekeeping instead: while reserved receipt bytes pass `persistence.housekeeping.receipt_target_bytes` (default 8 GiB), the compaction pass also compacts committed and cancelled receipts older than 7 days, plus refusals that gbrain's own producers will write again from current content, and doctor `persistence_capacity` tells the brain host's operator. A refused write you made that `gbrain repair failed-writes` could still replay now keeps its intent at any age, until a later write resolves it or you drop it with `gbrain repair receipt-compaction --include-failures`; the retention pass used to drop it after 30 days. An explicit `persistence.receipt_retention_days` is left as set.+**gbrain's own write bookkeeping no longer caps your writes.** Every accepted write leaves a receipt, and gbrain used to cap how many request ids and how many receipt bytes one writer and one brain could hold (2,000,000 / 8,000,000 ids, 16 GiB / 64 GiB). Near those caps, background writers paused at 80% and every write receipt carried a low-capacity warning; at the cap, `remember`, `put_page` and sync refused with `write_quota_exhausted` until someone on the brain host raised a number. Nothing about your data needed that: a cap only stopped writes. Both caps are now off unless the operator sets one. Receipt storage is kept in check by housekeeping instead: while receipts not yet compacted reserve more than `persistence.housekeeping.receipt_target_bytes` (default 8 GiB; the few KiB a compacted receipt keeps so its request id stays replayable never count, since nothing can shrink them), the compaction pass also compacts committed and cancelled receipts older than 7 days, plus refusals that gbrain's own producers will write again from current content, and doctor `persistence_capacity` tells the brain host's operator. A refused write you made that `gbrain repair failed-writes` could still replay now keeps its intent at any age, until a later write resolves it or you drop it with `gbrain repair receipt-compaction --include-failures`; the retention pass used to drop it after 30 days. An explicit `persistence.receipt_retention_days` is left as set. On a brain that several hosts write to, these defaults hold once every one of them runs this version: an older gbrain still enforces its old caps and its 30-day retention, and the `gbrain config set persistence.limits.*` that its `write_quota_exhausted` refusal suggests would turn a cap on for every host, so upgrade the other hosts instead of raising it.### Itemized changes-- `src/core/persistence/model.ts`, `limits.ts`: `principal_lifetime_ids`, `brain_lifetime_ids`, `principal_terminal_bytes` and `brain_terminal_bytes` default to no cap; a value set with `gbrain config set` is enforced as before (refusal at the cap, background pause at its line). New key `persistence.housekeeping.receipt_target_bytes` (0 turns housekeeping and its warning off).+- `src/core/persistence/model.ts`, `limits.ts`: `principal_lifetime_ids`, `brain_lifetime_ids`, `principal_terminal_bytes` and `brain_terminal_bytes` default to no cap; a value set with `gbrain config set` is enforced as before (refusal at the cap, background pause at its line). New key `persistence.housekeeping.receipt_target_bytes` (0 turns housekeeping and its warning off), measured by `readCompactableReceiptBytes`: terminal receipts not yet compacted that no recovery holds.- `src/core/persistence/compaction-pass.ts`: `housekeepReceipts` compacts up to 100 receipts older than 7 days per pass: settled page writes, and refused maintenance pages and managed sync, file import, reconcile and file repair page writes. Job receipts, connector writes and every other refusal (a caller's write, conversation facts, atom members, connector items) keep their intent until the retention window while the brain is over the target and the retention is the default, with every exclusion of the retention pass; the pass record counts them as `early`.-- `src/commands/doctor/checks/persistence-capacity.ts`: with no cap set, `ok` says so and `warn` fires above the target with the read-only `gbrain repair receipt-compaction --json` preview (actor `host_admin`); `details.housekeeping` reports target, stored bytes and an explicit retention. The forecast, capacity notices, repair capacity rows and `persistence_request_growth` cover only caps that are set, so the `/metrics` `gbrain_write_capacity_*` gauges are exported only for a cap you set; an alert on them needs a cap.+- `src/commands/doctor/checks/persistence-capacity.ts`: with no cap set, `ok` says so and `warn` fires above the target with the read-only `gbrain repair receipt-compaction --json` preview (actor `host_admin`); `details.housekeeping` reports target, stored bytes, the not-yet-compacted bytes the target counts and an explicit retention. The forecast, capacity notices, repair capacity rows and `persistence_request_growth` cover only caps that are set, so the `/metrics` `gbrain_write_capacity_*` gauges are exported only for a cap you set; an alert on them needs a cap.- `src/core/persistence/journal.ts`, `terminal-classify.ts`: the retention pass skips `UNRESOLVED_CALLER_REFUSAL_SQL` receipts (a writer-guard or `source_changed` refusal of a caller's `put_page`, `add_timeline_entry` or `remember` that no later request wrote, removed or repeated); `gbrain repair failed-writes` reads the same refusal predicates from there.+- `src/core/persistence/reconcile-backup.ts`: the `--backups` listing leaves out `max_retained_count` while the brain id cap is unset (it serialized as `null`); the retained-backup bound is `brain_recovery_bytes`.- Docs: `docs/guides/repair.md` (refused writes keep their intent until resolved; no retention raise needed for a recovery), `docs/guides/concurrent-writes.md` (limits table, receipt housekeeping, growth model as sizing for a cap you set), `docs/guides/write-refusals.md`.-- Tests: `test/persistence-opt-in-caps.test.ts` (counters at the old default caps admit background and interactive writes with no notice and no raise; a set cap still refuses; housekeeping above the target takes week-old settled page writes and refused maintenance and sync writes, and keeps a caller's refused write, a refused conversation-facts job, a committed job and young receipts; the retention pass keeps a 40-day-old replayable drift or guard refusal and compacts a resolved one and a non-replayable failure (red before: the drift refusal was compacted); a 40-day synthetic run never holds more than a week of uncompacted committed receipts; doctor's operator warning), `test/persistence-capacity.test.ts` (key count); the capacity forecast, notice and read-wave tests set the cap they exercise; the doctor goldens show the uncapped default and `details.housekeeping`.+- Tests: `test/persistence-opt-in-caps.test.ts` (counters at the old default caps admit background and interactive writes with no notice and no raise; a set cap still refuses; housekeeping above the target takes week-old settled page writes and refused maintenance and sync writes, and keeps a caller's refused write, a refused conversation-facts job, a committed job and young receipts; the retention pass keeps a 40-day-old replayable drift or guard refusal and compacts a resolved one and a non-replayable failure (red before: the drift refusal was compacted); a 40-day synthetic run never holds more than a week of uncompacted committed receipts; doctor's operator warning; compacted receipts alone above the target leave housekeeping idle and doctor ok, red before; the backup listing names no count bound while the id cap is unset, red before), `test/persistence-capacity.test.ts` (key count); the capacity forecast, notice and read-wave tests set the cap they exercise; the doctor goldens show the uncapped default and `details.housekeeping`.**A write that saved now says so when it carries a low-capacity warning, and the warning no longer asks the user to raise a write cap by tens of gigabytes.** When a cumulative write cap (receipt bytes or request ids) was at 80% or forecast to run out within 30 days, every write receipt's `write_capacity_low` notice ended with `actor: host_admin` and `next: tell_user_to_run: gbrain config set persistence.limits.principal_terminal_bytes <value>`. An agent that read only the end of a successful `remember` took that for a refusal and told its user the memory was not saved. The value was also oversized: it projected the last 24 hours for a whole year, so one bulk sync day could recommend several times the current cap. The notice now opens with "This write was saved and writes still work", its fix is the read-only `gbrain doctor --only persistence_capacity --json`, and the one-more-year value uses the 7-day mean rate. Days-to-exhaustion warnings still react to a bulk day. Doctor's finding keeps the `gbrain config set` command for the brain host's operator.diff --git a/docs/architecture/key-files/core-persistence-capacity.md b/docs/architecture/key-files/core-persistence-capacity.mdindex af85abc3d7..dd78b2055f 100644--- a/docs/architecture/key-files/core-persistence-capacity.md+++ b/docs/architecture/key-files/core-persistence-capacity.md@@ -9,7 +9,7 @@ Cumulative write caps (permanent request ids, receipt bytes; opt-in since GBRA-1- `src/core/persistence/capacity-forecast.ts` — `readCapacityForecast`: per principal, one statement walks `persistence_requests_principal` newest first until `created_at` leaves the 7-day window and buckets by day; gross and net (compacted) receipt rates, `days_to_exhaustion` (at the larger of the 7-day mean and the last 24 hours), `basis` (`:legacy_under_reserved` while receipts admitted before the intent-sized reservation remain), `recommended_limit` (one more year at the 7-day mean, GST-2107, so one bulk day is not projected for a year). `light` skips the reclaim read and table size. Thresholds `persistence.capacity.warn_days` (30) and `fail_days` (7).- `src/core/engine-key.ts` + `src/core/engine-root.ts` — the key every per-engine cache uses. `engineKey(engine)` returns the engine itself through the `ENGINE_ROOT` symbol (each engine stores itself there; a page-transaction clone inherits it, forwarding proxies and a recovered degraded engine return it), and otherwise walks a page-transaction clone, a read-through view or a statement-deadline wrapper (`engineCacheKey`) to the engine behind it. A process with mounted brains keeps one entry per brain in the search telemetry writer (`getTelemetryWriter(engine)`), `search.track_retrieval`, minion admission policy, `mcp.strict_params` last-known-good, publication ceiling, `write_capacity_low` throttle, `search.hnsw_iterative_scan` and `search.vector_legacy_guard`, and a transaction or view shares its engine's entry. A one-brain process behaves as before. Test: `test/engine-keyed-caches.test.ts`.- `src/core/persistence/capacity-snapshot.ts` — `cachedCapacitySnapshot`: one light forecast per engine per 60 s, shared by every caller; `capacityNotices` builds `write_capacity_low` (kind `info`, muteable; it says the write was saved and its fix is the read-only doctor check, GST-2107; hourly per engine and scope on write and sync receipts, `scopes` limits a remote caller to the brain and its own principal) and `write_capacity_backpressure` (once per pause start). Read by `mcp/dispatch.ts`, `cli/main.ts`, `commands/sync/{run,operator}.ts`, `cycle.ts` and the admin `/metrics` gauges in `serve-http-metrics.ts`. #6803: the cache keys on the real engine (`engineCacheKey`), the shared fetch is bounded at `CAPACITY_FETCH_BOUND_MS` (10 s) and a timed-out fetch is never cached; `capacitySnapshotWithin` (sync status, write and sync receipts: `CAPACITY_READ_WAIT_MS`, 1.5 s) returns `unknown` for that call alone, and `capacityNotices` on an unknown snapshot returns no notice and leaves throttle and pause state as it was. Test: `test/capacity-bounded-read-6803.test.ts`.-- `src/core/persistence/compaction-pass.ts` — `runCompactionPass`, the consumer's idle and volume maintenance: `compactWriteReceiptsDetailed`, `housekeepReceipts` (GBRA-108: above `persistence.housekeeping.receipt_target_bytes` with the default retention, up to 100 receipts older than `HOUSEKEEPING_FLOOR_DAYS` that `EARLY_SQL` admits: settled non-job, non-connector writes and refused maintenance or managed sync page writes; counted as `early`), one sweep step, and the `persistence.maintenance.last_compaction` record (`at`, `host_id`, `compacted`, `freed_bytes`, `skipped_by_reason`). Each step resumes from the timer queue (`journal.ts` `yieldToEventLoop`), as do the consumer's maintenance chain, the per-table queue `VACUUM` and the effects drain passes (`NOOP_SETTLE_BATCH` 50), so on PGLite a request waits behind one step at most. The retention pass (`compactWriteReceiptsDetailed`) skips `terminal-classify.ts` `UNRESOLVED_CALLER_REFUSAL_SQL` (a caller refusal `repair failed-writes` could still replay, GBRA-108). `journal.ts` `compactReceipt` compacts one receipt in its own transaction (counters first, `lock_timeout` 2 s, `statement_timeout` 15 s) and skips open managed-sync runs (`OPEN_SYNC_CURSOR_EXCLUSION`) and the newest sync receipt per worktree (`newestSyncReceipt`).+- `src/core/persistence/compaction-pass.ts` — `runCompactionPass`, the consumer's idle and volume maintenance: `compactWriteReceiptsDetailed`, `housekeepReceipts` (GBRA-108: while receipts not yet compacted reserve more than `persistence.housekeeping.receipt_target_bytes` (`limits.ts` `readCompactableReceiptBytes`; compacted receipts never count) with the default retention, up to 100 receipts older than `HOUSEKEEPING_FLOOR_DAYS` that `EARLY_SQL` admits: settled non-job, non-connector writes and refused maintenance or managed sync page writes; counted as `early`), one sweep step, and the `persistence.maintenance.last_compaction` record (`at`, `host_id`, `compacted`, `freed_bytes`, `skipped_by_reason`). Each step resumes from the timer queue (`journal.ts` `yieldToEventLoop`), as do the consumer's maintenance chain, the per-table queue `VACUUM` and the effects drain passes (`NOOP_SETTLE_BATCH` 50), so on PGLite a request waits behind one step at most. The retention pass (`compactWriteReceiptsDetailed`) skips `terminal-classify.ts` `UNRESOLVED_CALLER_REFUSAL_SQL` (a caller refusal `repair failed-writes` could still replay, GBRA-108). `journal.ts` `compactReceipt` compacts one receipt in its own transaction (counters first, `lock_timeout` 2 s, `statement_timeout` 15 s) and skips open managed-sync runs (`OPEN_SYNC_CURSOR_EXCLUSION`) and the newest sync receipt per worktree (`newestSyncReceipt`).- `src/core/repair/receipt-compaction.ts` — `gbrain repair receipt-compaction`: explicit-only, preview-bound, destructive consent, `publication: 'projection'`; `--older-than` (the `pages purge-deleted` grammar) floored at the configured retention unless `--force`, committed only unless `--include-failures`; the approved set binds an absolute cutoff and the newest sequence. Test: `test/repair-receipt-compaction.test.ts`.- `src/core/persistence/refusal-breaker.ts` — `assertRefusalBreakerClosed`: `submitMaintenance` refuses before admission after `persistence.maintenance.refusal_breaker_threshold` (3; 0 off) same-code terminal refusals in 24 h for the same request `digest` (source, slug, intent with its revision and planner); `listSuppressedWrites` feeds doctor `maintenance_write_drift`. The ghost case (`canonical_file_missing`, tagged by `page-prepare.ts`, kept in `error_detail.origin`, returned by `liveFileDrift`) refuses before admission in `prepared-maintenance.ts`. Test: `test/maintenance-ghost-breaker-6696.test.ts`.- Sync: `sync-run.ts` `preflightSyncCapacity` refuses `write_quota_insufficient` before a run's first admission; `sync-drain.ts` stops a later cap refusal `resumable` / `capacity_paused` (`drain.capacity_pause`). Test: `test/sync-capacity-preflight.test.ts`.diff --git a/docs/guides/concurrent-writes.md b/docs/guides/concurrent-writes.mdindex 087b97eaf0..919f46f7d9 100644--- a/docs/guides/concurrent-writes.md+++ b/docs/guides/concurrent-writes.md@@ -978,8 +978,8 @@ gbrain doctor --only persistence_capacity,persistence_request_growth --jsonReceipt storage grows with the retention window times the admission rate,plus about 4 KiB per compacted receipt, which keeps its request ID replayable.-While the brain's reserved receipt bytes pass-`persistence.housekeeping.receipt_target_bytes` (default 8 GiB; 0 turns it+While receipts not yet compacted (terminal, held by no recovery) reserve more+than `persistence.housekeeping.receipt_target_bytes` (default 8 GiB; 0 turns itoff), the resident consumer's compaction pass also compacts receipts older than7 days, oldest first, up to 100 a pass: settled page writes, and the refusalsof producers that write again from current content (maintenance pages and@@ -990,7 +990,10 @@ later write resolves it): `gbrain repair failed-writes` replays a caller'srefused write from it, conversation facts resubmit a frozenextraction, atom retries and connector held-item retries read it. Receiptswith unfinished effects, an open sync or a recovery are kept as before. An explicit `persistence.receipt_retention_days`-is the operator's choice, and housekeeping leaves it alone.+is the operator's choice, and housekeeping leaves it alone. Compacted+receipts never count toward the target: nothing can shrink them further, so a+large brain whose compacted receipts alone pass 8 GiB is not held above it.+On a brain that several hosts write to, these defaults hold once every one of them runs this version: an older gbrain still enforces its old caps and its 30-day retention, and the `gbrain config set persistence.limits.*` that its `write_quota_exhausted` refusal suggests would turn a cap on for every host, so upgrade the other hosts instead of raising it.While storage stays above the target, `persistence_capacity` warns the brainhost's operator (`actor: host_admin`) with the read-onlydiff --git a/src/commands/doctor/checks/persistence-capacity.ts b/src/commands/doctor/checks/persistence-capacity.tsindex ffbee15f65..8c9622aac2 100644--- a/src/commands/doctor/checks/persistence-capacity.ts+++ b/src/commands/doctor/checks/persistence-capacity.ts@@ -1,7 +1,7 @@import type { BrainEngine } from '../../../core/engine.ts';import type { Check } from '../../doctor.ts';import { capacityDiagnostics } from '../../../core/persistence/diagnostics.ts';-import { HOUSEKEEPING_FLOOR_DAYS, oneYearCapacity, readJournalLimits, readReceiptTargetBytes, RECEIPT_RETENTION_KEY, RECEIPT_TARGET_KEY } from '../../../core/persistence/limits.ts';+import { HOUSEKEEPING_FLOOR_DAYS, oneYearCapacity, readCompactableReceiptBytes, readJournalLimits, readReceiptTargetBytes, RECEIPT_RETENTION_KEY, RECEIPT_TARGET_KEY } from '../../../core/persistence/limits.ts';import type { CapacityForecast } from '../../../core/persistence/capacity-forecast.ts';import { doctorCapacityForecast, forecastOrNull } from '../capacity-memo.ts';import { capacityVerify } from '../../../core/persistence/capacity-view.ts';@@ -38,7 +38,9 @@ export async function checkPersistenceCapacity(engine: BrainEngine): Promise<Checonst target = await readReceiptTargetBytes(engine);const [retention] = await engine.executeRaw<{ value: string }>('SELECT value FROM config WHERE key=$1', [RECEIPT_RETENTION_KEY]);const stored = Number(counters.find(row => row.key === 'brain')?.terminal_bytes ?? 0);- const housekeeping = { target_bytes: target, used_bytes: stored, over_target: target > 0 && stored > target, explicit_retention_days: retention ? Number(retention.value) : null };+ const compactable = target > 0 && stored > target ? await readCompactableReceiptBytes(engine) : null;+ const housekeeping = { target_bytes: target, used_bytes: stored, compactable_bytes: compactable, over_target: compactable !== null && compactable > target,+ explicit_retention_days: retention ? Number(retention.value) : null };const cumulative = capacityDiagnostics(counters, limits).filter(row => row.resource === 'lifetime_ids' || row.resource === 'terminal_bytes');const outstanding = [];for (const row of capacityDiagnostics(counters, limits)) {@@ -93,13 +95,13 @@ export async function checkPersistenceCapacity(engine: BrainEngine): Promise<Cheif (!resources.length && housekeeping.over_target) return { name: 'persistence_capacity', status: 'warn', details,fix: { argv: ['gbrain', 'repair', 'receipt-compaction', '--json'], consent: [], actor: 'host_admin', requires_exclusive: false, verify: capacityVerify(), docs: DOCS,why: 'Previews the receipts an explicit compaction would take and the bytes it would free; it compacts nothing.' },- message: `Receipt storage holds ${stored} reserved bytes, above the housekeeping target of ${target} (${RECEIPT_TARGET_KEY}). Writes are not affected: receipt storage never refuses or pauses a write unless the operator sets a cap. `+ message: `Receipts not yet compacted reserve ${compactable} bytes (receipt storage holds ${stored} in all), above the housekeeping target of ${target} (${RECEIPT_TARGET_KEY}). Writes are not affected: receipt storage never refuses or pauses a write unless the operator sets a cap. `+ (housekeeping.explicit_retention_days !== null? `Housekeeping leaves receipts to the retention you set (${RECEIPT_RETENTION_KEY} = ${housekeeping.explicit_retention_days} days). `: `Housekeeping compacts settled receipts, and refusals gbrain's own producers write again, older than ${HOUSEKEEPING_FLOOR_DAYS} days while storage is above the target; what stays is newer receipts, refusals whose intent a replay or retry reads, job receipts, receipts an open sync or recovery holds, and compacted receipts that keep request ids idempotent. `)+ `On the brain host, gbrain repair receipt-compaction --json previews what an explicit compaction would free and compacts nothing; a larger target, or 0, quiets this warning.` };if (!resources.length && !cumulative.length) return { name: 'persistence_capacity', status: 'ok', details,- message: `No cumulative write cap is set (the default), so receipt storage never refuses or pauses a write; it holds ${stored} reserved bytes${target > 0 ? `, under the housekeeping target of ${target}` : ''}.` };+ message: `No cumulative write cap is set (the default), so receipt storage never refuses or pauses a write; it holds ${stored} reserved bytes${target > 0 ? `, and receipts not yet compacted are under the housekeeping target of ${target}` : ''}.` };if (!resources.length) return { name: 'persistence_capacity', status: 'ok', details,message: thresholds ? `Managed write capacity is below ${thresholds.warn_pct}% of every cumulative limit and none runs out within ${thresholds.warn_days} days at its 7-day rate.`: 'Managed write capacity is below 80% of every cumulative limit.' };diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex cbeddbc795..8dfdd2358e 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1493,7 +1493,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.622.0', id: 'think_window_trajectory_gst731', text: '`think` on a question that names one month or day (or given `since`/`until`) still gets the entity\'s trajectory: each metric\'s latest reading before the window now joins the readings inside it, as the value in force when the window opens (unless its `valid_until` ended by then), and an entity page dated outside the window still names whose trajectory to read (it stays out of the evidence). Before, "MRR in April" with readings in March and June, or a company page last edited outside the month, injected no trajectory. `WINDOW_EXCLUDED_<n>_TRAJECTORY_POINTS` no longer counts the carried reading.' },{ since: '0.60.622.0', id: 'purge_cancels_deferred_write_gst1602', text: 'With `persistence.refresh_barrier` on, `gbrain forget <id> --purge` no longer refuses `purge_blocked_pending_recovery` while a write deferred behind a refresh holds the claim in its staged body: the deferred write was never claimed, so the purge cancels it with `purged_content`, as it does a queued one, and removes the content at once instead of up to 15 minutes later.' },{ since: '0.60.623.0', id: 'owner_reads_private_gst2059', text: 'A brain can now name its owner: `gbrain access owner set <member>` (config `access.owner_member`, an enabled member). Every remote credential or stdio server bound to that member reads the whole brain, `visibility: private` pages, privacy-alias pages (`access: private`, ...), derived and pinned-question pages included, through `get_page`, `search`, `query`, `recall` and the other page reads in the sources its grant reads, and `recall` returns its private facts. `readable_by` and folder rules still bind it, and it is not relaxed when it speaks for a shared-room audience. A page the owner\'s credential creates in `default` through `put_page` is not kept to the owner (no `readable_by: [member:<id>]`). Other members, unbound and public callers are unchanged. `_meta.access.relaxation` reads `owner` for such a call, and `gbrain access check --token <name>` reports it. `gbrain repair private-labels` (named runs only) previews the `access: private` labels stamped on ordinary pages with counts per category, keeps real restrictions, and clears the rest in reviewed batches (`--apply --expect <hash>`), recording each change so `--class restore` can put it back. The bulk-ingestion page template no longer writes an `access:` line; what each designation means is in docs/guides/access-designations.md.' },- { since: 'TBD', id: 'opt_in_receipt_caps_gbra108', text: 'gbrain\'s own receipt bookkeeping no longer refuses, pauses or throttles writes by default: the cumulative caps on permanent request ids and reserved receipt bytes (`persistence.limits.{principal,brain}_{lifetime_ids,terminal_bytes}`, 2,000,000 / 8,000,000 ids and 16 GiB / 64 GiB before) apply only when set, so `write_quota_exhausted`, `write_quota_insufficient`, `write_capacity_backpressure`, `write_capacity_low` and a sync\'s `capacity_paused` happen only under a cap the operator chose. Instead, while reserved receipt bytes pass `persistence.housekeeping.receipt_target_bytes` (default 8 GiB, 0 off) and the retention is the default, the compaction pass also compacts settled page writes and refused maintenance and managed sync page writes older than 7 days (job receipts, connector writes and every other refusal keep their intent until the retention window), the `/metrics` `gbrain_write_capacity_*` gauges appear only for a cap that is set, and the retention pass no longer drops a caller\'s refused write that `gbrain repair failed-writes` could still replay (it waits until a later write resolves it, or `gbrain repair receipt-compaction --include-failures` drops it), and doctor `persistence_capacity` warns the host operator. A cap already set is unchanged; `gbrain config unset` removes one.' },+ { since: 'TBD', id: 'opt_in_receipt_caps_gbra108', text: 'gbrain\'s own receipt bookkeeping no longer refuses, pauses or throttles writes by default: the cumulative caps on permanent request ids and reserved receipt bytes (`persistence.limits.{principal,brain}_{lifetime_ids,terminal_bytes}`, 2,000,000 / 8,000,000 ids and 16 GiB / 64 GiB before) apply only when set, so `write_quota_exhausted`, `write_quota_insufficient`, `write_capacity_backpressure`, `write_capacity_low` and a sync\'s `capacity_paused` happen only under a cap the operator chose. Instead, while receipts not yet compacted reserve more than `persistence.housekeeping.receipt_target_bytes` (default 8 GiB, 0 off; compacted receipts never count) and the retention is the default, the compaction pass also compacts settled page writes and refused maintenance and managed sync page writes older than 7 days (job receipts, connector writes and every other refusal keep their intent until the retention window), the `/metrics` `gbrain_write_capacity_*` gauges appear only for a cap that is set, and the retention pass no longer drops a caller\'s refused write that `gbrain repair failed-writes` could still replay (it waits until a later write resolves it, or `gbrain repair receipt-compaction --include-failures` drops it), and doctor `persistence_capacity` warns the host operator. A cap already set is unchanged; `gbrain config unset` removes one. On a brain several hosts write to, this holds once every host runs this version: an older gbrain still enforces its old caps and 30-day retention, and raising a cap as its refusal suggests turns the cap on for every host.' },{ since: 'TBD', id: 'write_capacity_low_saved_gst2107', text: 'The `write_capacity_low` notice a write receipt carries now opens with "This write was saved" and its fix is the read-only `gbrain doctor --only persistence_capacity --json` (actor agent) instead of a `gbrain config set` cap raise for the user, so an agent reading the end of a successful write no longer reports it as refused. The one-more-year value doctor, the notice snapshot and `/metrics` recommend now uses the 7-day mean rate, so one bulk day is no longer projected for a year; days-to-exhaustion still uses the larger of the mean and the last 24 hours. A `write_quota_exhausted`, `write_quota_insufficient` or background-pause refusal names the same value doctor does instead of sizing the year from the latest 1,000 admissions.' },];diff --git a/src/core/persistence/compaction-pass.ts b/src/core/persistence/compaction-pass.tsindex 5536dcd767..64c0dc48f8 100644--- a/src/core/persistence/compaction-pass.ts+++ b/src/core/persistence/compaction-pass.ts@@ -14,7 +14,7 @@ import type { BrainEngine } from '../engine.ts';import { compactReceipt, compactWriteReceiptsDetailed, NEWEST_SYNC_RECEIPT_EXCLUSION, OPEN_SYNC_CURSOR_EXCLUSION, yieldToEventLoop, type ReceiptCompactionResult } from './journal.ts';import { getCode } from '../retry-matcher.ts';import type { WriteRequest } from './model.ts';-import { HOUSEKEEPING_FLOOR_DAYS, readReceiptTargetBytes, RECEIPT_RETENTION_KEY } from './limits.ts';+import { HOUSEKEEPING_FLOOR_DAYS, readCompactableReceiptBytes, readReceiptTargetBytes, RECEIPT_RETENTION_KEY } from './limits.ts';import { sweepCompactedReservations } from './receipt-true-up.ts';export const LAST_COMPACTION_KEY = 'persistence.maintenance.last_compaction';@@ -61,7 +61,7 @@ const EARLY_SQL = (r: string) => `((${r}.state IN ('committed','cancelled') ANDOR ${r}.intent->>'kind' IN ('managed_file_import','canonical_reconcile','managed_file_repair')))OR (${r}.operation='submit_job' AND ${r}.intent->>'kind'='managed_maintenance_page'))))`;/**- * GBRA-108: while the brain's reserved receipt bytes pass `persistence.housekeeping.receipt_target_bytes` and the retention is+ * GBRA-108: while receipts not yet compacted reserve more than `persistence.housekeeping.receipt_target_bytes` and the retention is* the default, one batch of receipts older than HOUSEKEEPING_FLOOR_DAYS that `EARLY_SQL` admits compacts early, oldest first.* An explicit retention is the operator's and is kept, and every exclusion of the retention pass applies (`compactReceipt`* rechecks state, age, effects and sync under its lock; operation and intent never change once a request ends).@@ -72,6 +72,7 @@ export async function housekeepReceipts(engine: BrainEngine): Promise<ReceiptCom`SELECT (SELECT value FROM config WHERE key=$1) AS retention,(SELECT terminal_bytes::text FROM persistence_counters WHERE key='brain') AS used,to_char((now()-($2::double precision*interval '1 day')) AT TIME ZONE 'UTC','YYYY-MM-DD"T"HH24:MI:SS.US"Z"') AS cutoff`, [RECEIPT_RETENTION_KEY, HOUSEKEEPING_FLOOR_DAYS])]);if (!state || state.retention !== null || target === 0 || Number(state.used ?? 0) <= target) return result;+ if (await readCompactableReceiptBytes(engine) <= target) return result;const states = ['committed', 'cancelled', 'failed', 'conflict'];const rows = await engine.executeRaw<Pick<WriteRequest, 'id' | 'principal_kind' | 'principal_id'>>(`SELECT id,principal_kind,principal_id FROM persistence_requests rWHERE state=ANY($1::text[]) AND recovery IS NULL AND NOT compacted AND completed_at < $2::timestamptzdiff --git a/src/core/persistence/limits.ts b/src/core/persistence/limits.tsindex 0ae0b9dda0..99e959daa4 100644--- a/src/core/persistence/limits.ts+++ b/src/core/persistence/limits.ts@@ -9,7 +9,7 @@ export const RETAINED_RECEIPT_BYTES = 4096;export const RECEIPT_RETENTION_KEY = 'persistence.receipt_retention_days';export const DEFAULT_RECEIPT_RETENTION_DAYS = 30;/**- * GBRA-108: while the brain's reserved receipt bytes pass this target, housekeeping compacts settled receipts and re-derivable+ * GBRA-108: while receipts not yet compacted reserve more than this target (`readCompactableReceiptBytes`), housekeeping compacts settled receipts and re-derivable* producer refusals older than HOUSEKEEPING_FLOOR_DAYS instead of waiting out the default retention (compaction-pass.ts* `EARLY_SQL`; an explicit retention is kept as set), and doctor tells the host operator. 0 turns both off. Never a write refusal.*/@@ -20,6 +20,16 @@ export async function readReceiptTargetBytes(engine:SqlEngine):Promise<number> {const [row]=await engine.executeRaw<{value:string}>('SELECT value FROM config WHERE key=$1',[RECEIPT_TARGET_KEY]);return row ? parseJournalConfigValue(RECEIPT_TARGET_KEY,row.value) : DEFAULT_RECEIPT_TARGET_BYTES;}+/**+ * The reserved bytes the receipt target counts: terminal receipts not yet compacted that no recovery holds (the rows of the+ * persistence_requests_compactable index). A compacted receipt keeps about 1-4 KB so its request id stays idempotent and never+ * shrinks further, so counting those would keep a large brain above the target forever with nothing left to compact.+ */+export async function readCompactableReceiptBytes(engine:SqlEngine):Promise<number> {+ const [row]=await engine.executeRaw<{bytes:string}>(`SELECT COALESCE(SUM(terminal_reservation),0)::text AS bytes FROM persistence_requests+ WHERE recovery IS NULL AND NOT compacted AND state IN ('committed','conflict','failed','cancelled')`);+ return Number(row?.bytes ?? 0);+}export function journalLimitKey(key:keyof JournalLimits):string {return `persistence.limits.${key.replace(/[A-Z]/g,letter=>`_${letter.toLowerCase()}`)}`;diff --git a/src/core/persistence/reconcile-backup.ts b/src/core/persistence/reconcile-backup.tsindex b09f006576..8caad24cb1 100644--- a/src/core/persistence/reconcile-backup.ts+++ b/src/core/persistence/reconcile-backup.ts@@ -143,7 +143,7 @@ export async function manageReconcileBackups(engine: BrainEngine, authority: Wri}const limits = await readJournalLimits(engine);return { source_id: authority.sourceId, slug, backups, next_after: rows.length > limit ? rows[limit - 1].request_id : null,- capacity: { max_retained_bytes: limits.brainRecoveryBytes, max_retained_count: limits.brainLifetimeIds } };+ capacity: { max_retained_bytes: limits.brainRecoveryBytes, ...(Number.isFinite(limits.brainLifetimeIds) ? { max_retained_count: limits.brainLifetimeIds } : {}) } };}const reference = params.backup_reference!;if (!backupPattern.test(reference)) throw opError('invalid_params', 'An exact private backup reference is required.',diff --git a/test/persistence-opt-in-caps.test.ts b/test/persistence-opt-in-caps.test.tsindex 4a1b822841..55540547e7 100644--- a/test/persistence-opt-in-caps.test.ts+++ b/test/persistence-opt-in-caps.test.ts@@ -14,6 +14,8 @@ import { runCompactionPass } from '../src/core/persistence/compaction-pass.ts';import { __resetCapacityNoticesForTests, capacityNotices } from '../src/core/persistence/capacity-snapshot.ts';import { declarePersistenceProtocol } from '../src/core/persistence/protocol.ts';import { RECEIPT_TARGET_KEY, readJournalLimits } from '../src/core/persistence/limits.ts';+import { manageReconcileBackups } from '../src/core/persistence/reconcile-backup.ts';+import type { WriteAuthority } from '../src/core/persistence/model.ts';import { checkPersistenceCapacity } from '../src/commands/doctor/checks/persistence-capacity.ts';import { editRequest, freshBrainFactory, requestFixture } from './helpers/persistence-request-fixture.ts';import { testBackends } from './helpers/test-backends.ts';@@ -191,5 +193,33 @@ for (const kind of testBackends()) {expect(warning.fix).toMatchObject({ actor: 'host_admin', argv: ['gbrain', 'repair', 'receipt-compaction', '--json'] });expect(warning.details).toMatchObject({ housekeeping: { target_bytes: 1, over_target: true } });});++ check('compacted receipts alone above the target: housekeeping has nothing to take and doctor stays ok', async engine => {+ const requests = await requestFixture(engine);+ for (const slug of ['a', 'b', 'c']) await requests.publish(await requests.admit(slug));+ await settleEffects(engine);+ await age(engine, 40);+ await engine.setConfig(RECEIPT_TARGET_KEY, '0');+ await runCompactionPass(engine, { hostId: 'host-a' });+ expect((await receipts(engine)).every(r => r.compacted)).toBe(true);+ // What compacted receipts keep for idempotency never shrinks, so it alone passing the target is nothing to act on.+ expect(await brainBytes(engine)).toBeGreaterThan(1);+ await engine.setConfig(RECEIPT_TARGET_KEY, '1');+ const doctor = await checkPersistenceCapacity(engine);+ expect(doctor.status).toBe('ok');+ expect(doctor.details).toMatchObject({ housekeeping: { target_bytes: 1, compactable_bytes: 0, over_target: false } });+ expect(await runCompactionPass(engine, { hostId: 'host-a' })).toMatchObject({ compacted: 0 });+ // A new settled receipt is something housekeeping can act on again.+ await requests.publish(await requests.admit('d'));+ expect((await checkPersistenceCapacity(engine)).status).toBe('warn');+ });++ check('the reconciliation backup listing names no count bound while the id cap is unset, and the cap once set', async engine => {+ const authority = { version: 1, principal: { kind: 'cli', id: 'opt-in-caps' }, remote: false, sourceId: 'default' } as unknown as WriteAuthority;+ const listing = async () => JSON.parse(JSON.stringify(await manageReconcileBackups(engine, authority, 'people/example', { action: 'list' })));+ expect((await listing()).capacity).toEqual({ max_retained_bytes: 1024 ** 3 });+ await engine.setConfig('persistence.limits.brain_lifetime_ids', '5000');+ expect((await listing()).capacity).toEqual({ max_retained_bytes: 1024 ** 3, max_retained_count: 5000 });+ });});}