Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
docs: per-engine caches in the key-files and guides
garrytan4cc102fd64ee
diff --git a/docs/architecture/RETRIEVAL.md b/docs/architecture/RETRIEVAL.mdindex e424f2407b..d492ebc7b8 100644--- a/docs/architecture/RETRIEVAL.md+++ b/docs/architecture/RETRIEVAL.md@@ -113,7 +113,7 @@ written up in [`RETRIEVAL_MAXPOOL_INCIDENT.md`](../incidents/RETRIEVAL_MAXPOOL_Inever reaches callers because the inner `ORDER BY distance` and the pooledouter sort re-sort it. `search.hnsw_iterative_scan` (`relaxed_order` |`strict_order` | `off`; env `GBRAIN_HNSW_ITERATIVE_SCAN` wins) restores the- older mode without a release; it is latched once per process, so restart+ older mode without a release; it is latched once per engine (once per process on a one-brain server), so restartserve and autopilot after changing it. A filtered short pool is notproof that the corpus is exhausted. Postgres may make one exact fallbackinside the remaining eight-second arm budget; PGLite does not pretend thatdiff --git a/docs/architecture/key-files/core-minions-1.md b/docs/architecture/key-files/core-minions-1.mdindex faee7a1cf1..e76d06733c 100644--- a/docs/architecture/key-files/core-minions-1.md+++ b/docs/architecture/key-files/core-minions-1.md@@ -6,7 +6,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/minions/` — Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types, protected-names, quiet-hours, handlers/shell). Private dream-inline-* queues carry an explicit lifecycle: owner job id + owner token + a renewable lease (columns on minion_jobs; `queue.ts` `reconcilePrivateQueue` terminalizes one queue through the normal cancelJobs bookkeeping and stamps every cancellation with the machine-readable `private_queue_reconciled:` reason family; `renewPrivateQueueLease` is MONOTONIC — GREATEST — so a default-horizon renewal can never shrink a creation-time lease; `reconcileOrphanedPrivateQueues` cancels only provably-orphaned queues: no healthy child lock, owner terminal/missing or lease expired; legacy unowned rows are left to Doctor/retriage; `classifyPrivateQueueForRecovery` is PUBLIC so the doctor orphan check buckets through the same verdict). Recovery runs in `gbrain jobs work` after driver/selected-child readiness and `ensureSchema`, for bare AND supervised workers (including autopilot spawns/respawns); the standalone parent does not recover queues in `beforeSpawn`. `maybeRunWorkerStartupRecovery(queue, process.env, true)` keeps this worker-owned ordering even under `GBRAIN_SUPERVISED=1`. Dream-cycle start also recovers (cycle.ts, next to the cycle lock reap — the ONLY lane on PGLite, which inlines every child). `deriveWedgeSignal` is queue-aware: a dream-inline queue is never 'wedged' (no shared worker can claim it) — it reports `private_queue: true` so jobs stats / get_job_stats / doctor all point at reconciliation instead of the impossible worker restart.- `src/core/minions/handlers/sync.ts` (and the other built-in job handler modules: `embed`, `lint`, `lint-fix`, `extract`, `extract-conversation-facts`, `chronicle-extract`, `loops-extract`, `enrich`, `integrity`, `integrity-auto`, `sync-retry-failed`, `import`, `backlinks`, `facts-absorb`, `autopilot-cycle`, `autopilot-global-maintenance`, `reindex`, `repair-jsonb`, `orphans`, `purge`, `cycle-phase`, `extract-atoms-drain`, `extract-ner`, `extract-takes-from-pages`, `extract-timeline-from-meetings`, `embed-catch-up`, `unify-types`, plus `job-pull.ts` and `job-progress.ts`) — one module per built-in handler body, each a `make<Name>Handler(engine)` factory (or a `MinionHandler` const when it needs no engine). Registered in a fixed order by `registerBuiltinHandlers` in `src/commands/jobs.ts`, which keeps the registration kind (plain vs gateway-refreshing) and wrappers such as `withFactsAbsorbHaltCooldown`; `cycle-phase.ts` serves the six phase wrappers. Command cores are imported lazily inside each handler. `resolveJobPull` and the facts-absorb retry predicates are re-exported from `jobs.ts`. `autopilot-global-maintenance` runs one `runCycle` call per maintenance phase and keeps its pass position in config `autopilot.global_maintenance.progress`: it stops starting a phase whose last measured duration would cross the job deadline (minus a 60 s reserve), the next job resumes at that phase, and a phase still marked running when an earlier job died is skipped for the rest of the pass; last_global_at is stamped only when a pass completes with no blocking phase. GBRA-130 A-10: each blocking phase keeps a streak in that row (`failures[phase]`, key `fail:<code>`, `budget`, `child:<outcome>` or `deferred`, counted once per pass); after three earlier `fail:` passes the pass stamps past it (`pass_completed_with_failing_phase`), `fail:`/`child:` streaks back off from their third pass, and the streak resets on a version change, a new key or `gbrain autopilot --retry-global` (rules in the module header). Phases run through an `ItemRun` (site `autopilot.global_maintenance`). Probe: `test/pathological/a10-global-maintenance-stamp.test.ts`. The deadline comes from `GBRAIN_GLOBAL_MAINTENANCE_TIMEOUT_MS` > `autopilot.global_maintenance_timeout_ms` > the autopilot default (`autopilot-fanout.ts:resolveGlobalMaintenanceTimeoutMs`); doctor `global_maintenance_timeouts` (`doctor/checks/global-maintenance-timeouts.ts`) warns after three consecutive deaths and names `gbrain dream --phase <name>` (`autopilot-global-maintenance-resume-4578.test.ts`). The `sync` handler runs performSync under `job.signal` composed with `job.shutdownSignal`: under a signal owner (`jobs work`, the supervisor) SIGTERM skips the cleanup pass and aborts only the worker's shutdown signal, so the sync and its `git pull` (async `pullRepo` on `bounded-child-exec.ts`) stop and the job is handed back without burning an attempt. A queued `autopilot_freshness` sync job re-reads its source when it runs and returns `{skipped:true, reason:'sync_disabled'}` when the source now has `syncEnabled:false`; explicit sync jobs still run and a failed lookup falls through to syncing (`sync-handler-sync-enabled.test.ts`). #6706: managed jobs drain via `sync-job-drain.ts`. Progress (`job-progress.ts` `reportJobProgress`, best-effort: a failed write never fails the job): `autopilot-cycle` passes `CycleOpts.onPhaseStart` and `autopilot-global-maintenance` writes before each phase's `runCycle`, both `{phase, phase_index, phase_count, started_at}`; the `sync` handler always passes `jobDrainProgress`'s `onProgress`, which writes a classic sync's phase (`resolve_repo`, `git_pull`, `full_import`, `import` with `banked_files`) on each change and at most every 10 s within one, beside the managed drain snapshot (`test/minion-job-progress-7241.test.ts`).-- `src/core/minions/admission.ts` — submit-side queue admission control (the drain-side pool-starvation half is the job-isolation work; claim fairness is a filed TODO, deliberately out of scope). Three primitives resolved per name via `resolveAdmissionPolicy` (config `minions.*` > per-name defaults tables, 60s in-process cache, fail-open with a once-per-process stderr warn; env kill-switch `GBRAIN_MINIONS_ADMISSION=0` disables all three): PARAM-COALESCING (`PARAM_COALESCE_DEFAULT` — on for `subagent`; `computeParamHash` = sha256 of stable-stringified payload excluding `__param_hash` itself and the `__submitter` display label — `__owner_client_id` is deliberately INCLUDED so owner lanes never cross; parentless + waiting-only + age-bounded to ttl/2), WAITING-TTL (`WAITING_TTL_DEFAULT_HOURS` — 48h for `subagent`; swept by `MinionQueue.handleWaitingTTL` through `cancelJobs(ids, {reason, rootStatuses:['waiting']})` so descendants cancel, child_done lands, aggregator parents resolve, and the reason stamps ROOT ids only; ≤500/tick oldest-first; warn-before-act is `runWaitingTtlTick` — first tick counts affected + stamps `TTL_NOTICE_SHOWN_KEY` with an ISO timestamp, sweeping starts only after `ttlNoticeGraceMs()` (1h default, env `GBRAIN_MINIONS_TTL_NOTICE_GRACE_MS`) elapses; `gbrain upgrade` prints the same one-shot notice and starts the same clock; legacy `'true'` flag values sweep immediately), and NAME-GLOBAL QUOTA (`QUOTA_MAX_WAITING_DEFAULT` EMPTY by operator decision — activates only via `minions.quota_max_waiting.<name>`; counts the name across ALL queues so per-run `dream-inline-*` fanout queues can't dodge it, EXACT under concurrency via a `minion_quota:<name>` advisory xact lock taken only when a quota is configured; throws typed `QueueQuotaExceededError`, checked everywhere via `isQueueQuotaExceededError` — dream submitters record a phase skip, synthesize rolls back the current transcript's fresh chunks, agent fanout cancels the whole tree, `submit_agent` maps to a structured `rate_limited` OperationError). `TTL_REASON_PREFIX` is the single source for the sweep's error_text prefix and the stats/doctor LIKE patterns; `safeConfigSegment` gates untrusted job names out of copy-pasteable config hints. Alerting rides `getStats` (`drained_completed/failed/dead/cancelled` keyed on finished_at + `waiting_now` + `oldest_waiting_minutes`), the `jobs stats` DIVERGENT-QUEUE / waiting-TTL screams (`GBRAIN_QUEUE_DIVERGENCE_RATIO`=2, `GBRAIN_QUEUE_DIVERGENCE_MIN_WAITING`=50; divergence compares intake vs COMPLETED so TTL-cancel storms can't masquerade as throughput) + `--json`, and doctor `checkQueueHealth`. Pinned by `test/minions-admission.test.ts` + `test/jobs-stats-divergence.serial.test.ts`.+- `src/core/minions/admission.ts` — submit-side queue admission control (the drain-side pool-starvation half is the job-isolation work; claim fairness is a filed TODO, deliberately out of scope). Three primitives resolved per name via `resolveAdmissionPolicy` (config `minions.*` > per-name defaults tables, 60s in-process cache per engine, fail-open with a once-per-process stderr warn; env kill-switch `GBRAIN_MINIONS_ADMISSION=0` disables all three): PARAM-COALESCING (`PARAM_COALESCE_DEFAULT` — on for `subagent`; `computeParamHash` = sha256 of stable-stringified payload excluding `__param_hash` itself and the `__submitter` display label — `__owner_client_id` is deliberately INCLUDED so owner lanes never cross; parentless + waiting-only + age-bounded to ttl/2), WAITING-TTL (`WAITING_TTL_DEFAULT_HOURS` — 48h for `subagent`; swept by `MinionQueue.handleWaitingTTL` through `cancelJobs(ids, {reason, rootStatuses:['waiting']})` so descendants cancel, child_done lands, aggregator parents resolve, and the reason stamps ROOT ids only; ≤500/tick oldest-first; warn-before-act is `runWaitingTtlTick` — first tick counts affected + stamps `TTL_NOTICE_SHOWN_KEY` with an ISO timestamp, sweeping starts only after `ttlNoticeGraceMs()` (1h default, env `GBRAIN_MINIONS_TTL_NOTICE_GRACE_MS`) elapses; `gbrain upgrade` prints the same one-shot notice and starts the same clock; legacy `'true'` flag values sweep immediately), and NAME-GLOBAL QUOTA (`QUOTA_MAX_WAITING_DEFAULT` EMPTY by operator decision — activates only via `minions.quota_max_waiting.<name>`; counts the name across ALL queues so per-run `dream-inline-*` fanout queues can't dodge it, EXACT under concurrency via a `minion_quota:<name>` advisory xact lock taken only when a quota is configured; throws typed `QueueQuotaExceededError`, checked everywhere via `isQueueQuotaExceededError` — dream submitters record a phase skip, synthesize rolls back the current transcript's fresh chunks, agent fanout cancels the whole tree, `submit_agent` maps to a structured `rate_limited` OperationError). `TTL_REASON_PREFIX` is the single source for the sweep's error_text prefix and the stats/doctor LIKE patterns; `safeConfigSegment` gates untrusted job names out of copy-pasteable config hints. Alerting rides `getStats` (`drained_completed/failed/dead/cancelled` keyed on finished_at + `waiting_now` + `oldest_waiting_minutes`), the `jobs stats` DIVERGENT-QUEUE / waiting-TTL screams (`GBRAIN_QUEUE_DIVERGENCE_RATIO`=2, `GBRAIN_QUEUE_DIVERGENCE_MIN_WAITING`=50; divergence compares intake vs COMPLETED so TTL-cancel storms can't masquerade as throughput) + `--json`, and doctor `checkQueueHealth`. Pinned by `test/minions-admission.test.ts` + `test/jobs-stats-divergence.serial.test.ts`.- `src/core/minions/attachments.ts` — Attachment validation (path traversal, null byte, oversize, base64, duplicate detection).diff --git a/docs/architecture/key-files/core-persistence-capacity.md b/docs/architecture/key-files/core-persistence-capacity.mdindex f0027ec701..abafb6ce9e 100644--- a/docs/architecture/key-files/core-persistence-capacity.md+++ b/docs/architecture/key-files/core-persistence-capacity.md@@ -7,7 +7,8 @@ Cumulative write caps (permanent request ids, receipt bytes) and the paths that- `src/core/persistence/admission-origin.ts` — `withAdmissionOrigin('interactive' | 'background')` (an outer background scope wins), `resolveAdmissionOrigin` (unset reads background, never derived from `ctx.remote`), `persistence.capacity.background_pause_pct` (default 80) and `backgroundPaused`, the `write_capacity_backpressure` refusal background producers get past the line. A background scope also carries the per-run owner cache (`backgroundOwnerCheck`); `backgroundWorktreeInactive` (a `recovering` worktree, read under the admission's lock in `journal.ts`) and `backgroundOwnerUnverified` refuse `owner_unavailable` pre-receipt with the owner-status fix; a draining worktree still admits.- `src/core/persistence/owner-verification.ts` — #6741 C: the claim's owner verification (`verifyOwnerRoot`: paths recorded, physical root, marker and reservation intact; a device-only change is `restampable`), shared by writer onboarding and background admissions; `assertBackgroundOwner` refuses a background admission `owner_unavailable` once per source per run (cache: admission-origin.ts `backgroundOwnerCheck`). Tests: `test/background-owner-verification.test.ts`.- `src/core/persistence/capacity-forecast.ts` — `readCapacityForecast`: per principal, one statement walks `persistence_requests_principal` newest first until `created_at` leaves the 7-day window and buckets by day; gross and net (compacted) receipt rates, `days_to_exhaustion`, `basis` (`:legacy_under_reserved` while receipts admitted before the intent-sized reservation remain), `recommended_limit`. `light` skips the reclaim read and table size. Thresholds `persistence.capacity.warn_days` (30) and `fail_days` (7).-- `src/core/persistence/capacity-snapshot.ts` — `cachedCapacitySnapshot`: one light forecast per engine per 60 s, shared by every caller; `capacityNotices` builds `write_capacity_low` (kind `info`, muteable; hourly per process per scope on write and sync receipts, `scopes` limits a remote caller to the brain and its own principal) and `write_capacity_backpressure` (once per pause start). Read by `mcp/dispatch.ts`, `cli/main.ts`, `commands/sync/{run,operator}.ts`, `cycle.ts` and the admin `/metrics` gauges in `serve-http-metrics.ts`. #6803: the cache keys on the real engine (`engineCacheKey`), the shared fetch is bounded at `CAPACITY_FETCH_BOUND_MS` (10 s) and a timed-out fetch is never cached; `capacitySnapshotWithin` (sync status, write and sync receipts: `CAPACITY_READ_WAIT_MS`, 1.5 s) returns `unknown` for that call alone, and `capacityNotices` on an unknown snapshot returns no notice and leaves throttle and pause state as it was. Test: `test/capacity-bounded-read-6803.test.ts`.+- `src/core/engine-key.ts` + `src/core/engine-root.ts` — the key every per-engine cache uses. `engineKey(engine)` returns the engine itself through the `ENGINE_ROOT` symbol (each engine stores itself there; a page-transaction clone inherits it, forwarding proxies and a recovered degraded engine return it), and otherwise walks a page-transaction clone, a read-through view or a statement-deadline wrapper (`engineCacheKey`) to the engine behind it. A process with mounted brains keeps one entry per brain in the search telemetry writer (`getTelemetryWriter(engine)`), `search.track_retrieval`, minion admission policy, `mcp.strict_params` last-known-good, publication ceiling, `write_capacity_low` throttle, `search.hnsw_iterative_scan` and `search.vector_legacy_guard`, and a transaction or view shares its engine's entry. A one-brain process behaves as before. Test: `test/engine-keyed-caches.test.ts`.+- `src/core/persistence/capacity-snapshot.ts` — `cachedCapacitySnapshot`: one light forecast per engine per 60 s, shared by every caller; `capacityNotices` builds `write_capacity_low` (kind `info`, muteable; hourly per engine and scope on write and sync receipts, `scopes` limits a remote caller to the brain and its own principal) and `write_capacity_backpressure` (once per pause start). Read by `mcp/dispatch.ts`, `cli/main.ts`, `commands/sync/{run,operator}.ts`, `cycle.ts` and the admin `/metrics` gauges in `serve-http-metrics.ts`. #6803: the cache keys on the real engine (`engineCacheKey`), the shared fetch is bounded at `CAPACITY_FETCH_BOUND_MS` (10 s) and a timed-out fetch is never cached; `capacitySnapshotWithin` (sync status, write and sync receipts: `CAPACITY_READ_WAIT_MS`, 1.5 s) returns `unknown` for that call alone, and `capacityNotices` on an unknown snapshot returns no notice and leaves throttle and pause state as it was. Test: `test/capacity-bounded-read-6803.test.ts`.- `src/core/persistence/compaction-pass.ts` — `runCompactionPass`, the consumer's idle and volume maintenance: `compactWriteReceiptsDetailed`, one sweep step, and the `persistence.maintenance.last_compaction` record (`at`, `host_id`, `compacted`, `freed_bytes`, `skipped_by_reason`). Each step resumes from the timer queue (`journal.ts` `yieldToEventLoop`), as do the consumer's maintenance chain, the per-table queue `VACUUM` and the effects drain passes (`NOOP_SETTLE_BATCH` 50), so on PGLite a request waits behind one step at most. `journal.ts` `compactReceipt` compacts one receipt in its own transaction (counters first, `lock_timeout` 2 s, `statement_timeout` 15 s) and skips open managed-sync runs (`OPEN_SYNC_CURSOR_EXCLUSION`) and the newest sync receipt per worktree (`newestSyncReceipt`).- `src/core/repair/receipt-compaction.ts` — `gbrain repair receipt-compaction`: explicit-only, preview-bound, destructive consent, `publication: 'projection'`; `--older-than` (the `pages purge-deleted` grammar) floored at the configured retention unless `--force`, committed only unless `--include-failures`; the approved set binds an absolute cutoff and the newest sequence. Test: `test/repair-receipt-compaction.test.ts`.- `src/core/persistence/refusal-breaker.ts` — `assertRefusalBreakerClosed`: `submitMaintenance` refuses before admission after `persistence.maintenance.refusal_breaker_threshold` (3; 0 off) same-code terminal refusals in 24 h for the same request `digest` (source, slug, intent with its revision and planner); `listSuppressedWrites` feeds doctor `maintenance_write_drift`. The ghost case (`canonical_file_missing`, tagged by `page-prepare.ts`, kept in `error_detail.origin`, returned by `liveFileDrift`) refuses before admission in `prepared-maintenance.ts`. Test: `test/maintenance-ghost-breaker-6696.test.ts`.diff --git a/docs/architecture/key-files/core-search-2.md b/docs/architecture/key-files/core-search-2.mdindex 4e261a2612..fc2e108d58 100644--- a/docs/architecture/key-files/core-search-2.md+++ b/docs/architecture/key-files/core-search-2.md@@ -51,10 +51,10 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/search/vector-statement.ts` — `buildVectorSearchStatement({ dialect, embedding, limit, offset, opts })` builds the vector-search SQL once for both engines, doctor `vector_plan` and the plan-proof E2E: `{ sql, exactSql, hasMoreSql, params, innerLimitIdx, innerLimit, indexed, relaxed, indexWalkSql?, indexWalkOverfetch, scopeScanSql? }`. On an HNSW-indexed `embedding` column (relaxed variant) the candidate CTE carries only the model check; content freshness (`embedded_text_hash = md5(chunk_text)` or NULL) is projected as `hash_current` and filtered in `scored`, because Postgres cannot estimate that expression and drops the index when it sits in the CTE (#5824). Non-indexed columns, the legacy guard, `exactSql` and `hasMoreSql` keep the full guard inside. The pool row reports `candidate_pool` (raw) and `eligible_pool` (fresh). Since/until bounds are spelled per column (`effective_date`, then `updated_at`, then `created_at`) so each arm has column statistics. `indexWalkSql` (relaxed variant without a type or date filter, same parameters) is the statement both engines run first: `ann` orders `content_chunks` alone with the chunk-level filters over twice the window, then pages and sources are joined by key and every page filter, visibility and the private-page rule apply to those rows, keeping the nearest `innerLimit`. Run with `enable_sort = off` (`INDEX_WALK_SETTINGS`) the HNSW scan is its only ordered path; the joined statement's plan sat on a cost knife edge (TOASTed vectors cost a sequential scan too little, the HNSW estimate grows with `ef_search`, the private-page rule's subplans priced every page lookup high) and flipped to a full scan and sort at a 50-row limit or on any remote read. A full walk window selects the same rows as the joined statement (`test/search/vector-index-walk.test.ts`); a short one falls back to it. Source scopes: each engine's `vectorScopeLoader` reads `PAGE_SOURCE_STATS_SQL` (`pages.source_id` planner statistics plus `content_chunks` reltuples) at most once a minute, only for source-scoped searches, and passes `scope` (`share`, and `chunks`). For a scope under `SCOPE_SCAN_MAX_SHARE`, `chunks` comes from `SCOPE_CHUNKS_SQL` (at most once a minute per scope): the scope's live pages times the chunks per page of a `hashint4` stride sample of about 400 of them (every page in a smaller scope), because the page share undercounts sources of long pages; otherwise, and when the count fails, it is the share times chunk reltuples. The walk over-fetches `indexWalkOverfetch(share)` = ceil(2 / share) rows per window slot (2 unscoped or for a whole-brain scope, so that statement is byte-identical), and a share below `INDEX_WALK_MIN_SCOPE_SHARE` (0.04) skips it. `scopeScanSql` (relaxed variant, same parameters) serves a scope under `SCOPE_SCAN_MAX_SHARE` (0.3) of pages with at most `SCOPE_SCAN_MAX_CHUNKS` (120k) counted chunks: an exact distance scan over the scope's chunks, page filters and visibility applied once per page inside `cc.page_id = ANY(ARRAY(...))` so chunks are always reached through `idx_chunks_page` (an inline join flipped to a sequential scan with the private-page rule), ordering only chunk ids with `(distance) + 0` to keep the index out, then the window joins back for its columns. Up to `SCOPE_SCAN_FIRST_MAX_CHUNKS` (25k) it replaces the walk; above that the walk runs first and the scan answers when the walk comes back short (a scope clustered away from the query). It returns the joined statement's rows (`test/search/vector-index-walk.test.ts`); recall across share buckets on 1024-dim vectors is pinned by `test/e2e/vector-scope-recall-postgres.test.ts`. Missing statistics leave only the unscoped walk and the joined statement. PGLite adds the timeline `stale` column; everything else is byte-identical across dialects. Also exports the extension-version probe and the transaction-local `statement_timeout` SQL the Postgres engine runs. Pinned by `test/search/vector-statement.test.ts`, `test/search/vector-freshness.test.ts` and `test/e2e/vector-plan-real-column-postgres.test.ts`.-- `src/core/search/vector-legacy-guard.ts` — one-release rollback for the #5824 statement change. `readVectorLegacyGuard(cfg, env)` reports the setting (`GBRAIN_VECTOR_LEGACY_GUARD` wins over `search.vector_legacy_guard`); `resolveVectorLegacyGuard(cfg)` latches it once per process for `hybrid/request.ts`, which passes `SearchOpts.vectorLegacyGuard` to the engines (they never read config), and prints `[gbrain] vector legacy guard active` to stderr once when on. A changed setting takes effect when the owning service restarts.+- `src/core/search/vector-legacy-guard.ts` — one-release rollback for the #5824 statement change. `readVectorLegacyGuard(cfg, env)` reports the setting (`GBRAIN_VECTOR_LEGACY_GUARD` wins over `search.vector_legacy_guard`); `resolveVectorLegacyGuard(engine, cfg)` latches it once per engine (`engineKey`) for `hybrid/request.ts`, which passes `SearchOpts.vectorLegacyGuard` to the engines (they never read config), and prints `[gbrain] vector legacy guard active` to stderr once per process when on. A changed setting takes effect when the owning service restarts.- `src/core/search/evidence-packing.ts` — the explicit-budget packing surface of evidence delivery: `AUTO_PACKINGS` (`off`, `cap_only` default, `breadth_capped`, `depth_first`) for `search.auto_packing`, `MIN_EXPLICIT_AUTO_BUDGET` (32; an explicit `auto` budget below it, or non-finite, is refused with `invalid_params` naming it), `parseAutoPacking` (the library override, unknown values refused), `configPacking` (unknown stored values read as the default) and `autoPackingValueProblem` (the `config set` refusal wired through `search/config-values.ts`); `queryBudgetUsesAuto` (`search.query_budget_unit`, GBRA-205: `auto` makes `query`'s `token_budget` the hard cap, default `chunk`). Tests: `test/evidence-delivery.test.ts`, `test/search-8k-conversion.test.ts`.-- `src/core/search/hnsw-iterative-scan.ts` — the pgvector `hnsw.iterative_scan` mode for filtered vector search (#6132 idea): `relaxed_order` by default, `strict_order` or `off` through `GBRAIN_HNSW_ITERATIVE_SCAN` (wins) or `search.hnsw_iterative_scan` (file > DB via loadConfigWithEngine). `resolveHnswIterativeScan(cfg)` latches once per process (hybrid `request.ts` passes it as `SearchOpts.hnswIterativeScan`; engines hand it to `withVectorSettings`), logging once when an override is active. `hnswIterativeScanValueProblem` backs the `config set` refusal through `search/config-values.ts` (`searchConfigValueRefusal`: enum/range rules for `search.*` keys, exit 2 `invalid_params`, nothing written). Pinned by `test/search/hnsw-iterative-scan.test.ts` and the Postgres recall fixture `test/e2e/hnsw-iterative-scan-recall-postgres.test.ts`; bench `scripts/bench/hnsw-iterative-scan.ts` (its `--corpus latent` mode, with `scripts/bench/hnsw-latent-corpus.ts` and the VM bootstrap `scripts/bench/hnsw-scale-vm-setup.sh`, is the E5.4 scale and index-build bench; results in `docs/eval/hnsw-scale-bench.md`).+- `src/core/search/hnsw-iterative-scan.ts` — the pgvector `hnsw.iterative_scan` mode for filtered vector search (#6132 idea): `relaxed_order` by default, `strict_order` or `off` through `GBRAIN_HNSW_ITERATIVE_SCAN` (wins) or `search.hnsw_iterative_scan` (file > DB via loadConfigWithEngine). `resolveHnswIterativeScan(engine, cfg)` latches once per engine (hybrid `request.ts` passes it as `SearchOpts.hnswIterativeScan`; engines hand it to `withVectorSettings`), logging once per override mode per process when an override is active. `hnswIterativeScanValueProblem` backs the `config set` refusal through `search/config-values.ts` (`searchConfigValueRefusal`: enum/range rules for `search.*` keys, exit 2 `invalid_params`, nothing written). Pinned by `test/search/hnsw-iterative-scan.test.ts` and the Postgres recall fixture `test/e2e/hnsw-iterative-scan-recall-postgres.test.ts`; bench `scripts/bench/hnsw-iterative-scan.ts` (its `--corpus latent` mode, with `scripts/bench/hnsw-latent-corpus.ts` and the VM bootstrap `scripts/bench/hnsw-scale-vm-setup.sh`, is the E5.4 scale and index-build bench; results in `docs/eval/hnsw-scale-bench.md`).- `src/core/search/internal-breadth.ts` — `INTERNAL_BREADTH_SEARCH_OPTS` (`autocut: false, adaptiveReturn: false`), the one option object internal callers spread into `hybridSearch` when they need the full ranked set rather than a reader-trimmed one (#5890): think gather (both legs), brainstorm close-set, grade-takes evidence, whoknows and enrich. Pinned by `test/internal-breadth-callers.serial.test.ts`; `eval-contradictions/runner.ts` still inherits both trims.diff --git a/docs/architecture/key-files/core-services-1.md b/docs/architecture/key-files/core-services-1.mdindex 600da3e14e..f27373a76d 100644--- a/docs/architecture/key-files/core-services-1.md+++ b/docs/architecture/key-files/core-services-1.md@@ -21,7 +21,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/brainstorm/checkpoint.ts` — crash-resilient checkpoint for `gbrain brainstorm` and `gbrain lsd`. Persists FULL idea bodies (~50KB/run) so resume MERGES pre-crash ideas with post-resume ideas before the judge runs (a resume that produces only second-run output is silent partial output). `run_id = sha256(question + profile + sort(close_slugs) + sort(far_slugs)).slice(0,16)` — NO embedding bits, stable across embedding-model swaps. Atomic write via `.tmp + rename`. ONE resume flag (`--resume <run_id>` covers failed crosses, never-attempted crosses, AND a failed judge phase — the orchestrator keeps the checkpoint with `judge_done=false` whenever the judge throws, so `--resume` short-circuits every completed cross from disk and re-runs only the judge; there is no separate retry-judge flag); only a fully clean run (all crosses + judge) clears the file. The checkpoint also banks the CLI's saved-page slug (`idea_slug`, from `RunBrainstormOpts.ideaSlug`) and the orchestrator echoes it back as `BrainstormResult.idea_slug` on resume, so the re-scored page overwrites the one the failed run saved instead of forking a second page under a fresh nonce. `--list-runs` prints run_ids mtime-newest-first; `--force-resume` bypasses the 7-day staleness gate. Cycle purge phase (`gbrain dream --phase purge`) GCs checkpoints older than 7 days via `gcStaleCheckpoints(7)`. Pinned by `test/e2e/brainstorm-resume.test.ts` (20 unit + 5 E2E cases incl. the merge contract and judge-failed retention/re-score).-- `src/core/brainstorm/{domain-bank,orchestrator,judges}.ts` + `src/commands/{brainstorm,lsd,eval-brainstorm}.ts` + `src/core/last-retrieved.ts` — bisociation-grounded idea generation pair: `gbrain brainstorm <question>` (defensible, cite-heavy, 4 close × 6 far, judge threshold 4.0/5, save by default) and `gbrain lsd <question>` (Lateral Synaptic Drift — inverted judge rejecting ideas with resistance >4.5 "too obvious", stale-page bias via `pages.last_retrieved_at`, 2 close × 12 far, axiomatic inversions required, ephemeral by default). The "domain bank" is prefix-stratified sampling from the user's own brain (`SELECT DISTINCT substring(slug from '^[^/]+/[^/]+')` cached 1h-TTL in `config` per source) tiebroken by `JOIN page_links` connection_count, with corpus-sampling fallback when fewer prefixes than M exist. Distance normalized to [0,1] via `1 - clamp(cosine_distance, 0, 2) / 2`. `judges.ts` exports `runJudge(config, ideas)` + two configs (`BRAINSTORM_JUDGE_CONFIG` weighted originality/resistance/thesis_density/concrete_grounding/cognitive_load 0.25/0.20/0.20/0.20/0.15 vs `LSD_JUDGE_CONFIG` cognitive_load 0.50 + inversion rule). Calibration cold-start fallback: when `calibration_profiles.active_bias_tags` is empty, judge runs without anti-bias context AND stderr-warns. Op-layer write-back in `src/core/operations.ts` `search`/`query`/`get_page` handlers fires `bumpLastRetrievedAt(engine, pageIds)` (fire-and-forget, 5-min throttled via SQL clause, default-on with `search.track_retrieval` config escape hatch); internal callers (sync, migrations, dream cycle) bypass the op layer so the LSD stale signal stays clean. The fire-and-forget IIFE is tracked in a module-scoped `Set<Promise<unknown>>`; `awaitPendingLastRetrievedWrites(timeoutMs?: number): Promise<{outcome, pending}>` resolves once all tracked promises settle, bounded by a 5s `Promise.race` timeout that stderr-warns the pending count. `src/cli.ts` awaits the drain unconditionally for every op in the op-dispatch finally block BEFORE `engine.disconnect()`, then a fallback `process.exit(0)` fires ONLY when `outcome === 'timeout'` AND `shouldForceExitAfterMain(argv)` (excludes `serve` so daemons stay alive) — so the IIFE cannot race disconnect and leave PGLite's WASM keeping Bun's event loop alive. The timestamp lives in `page_retrievals(page_id, last_retrieved_at)` (v224): the bump is an `INSERT ... ON CONFLICT DO UPDATE` throttled to 5 minutes, a HOT update with no triggers that never advances `page_generation_clock_seq` (the old `UPDATE pages` advanced it on every read, expiring the query-cache bookmark and the get_health and projection-readiness memos). No foreign key; the `pages_forget_retrievals` statement trigger removes rows of hard-deleted pages. Readers (domain-bank samplers, volunteer stats, entity card) take `GREATEST(pages.last_retrieved_at, page_retrievals.last_retrieved_at)`; the legacy column stays (unindexed since v224) for older binaries; full forward-reference bootstrap probe on both engines. Frontmatter `mode: lsd` makes the dream-cycle synthesize phase skip LSD output via `isLsdOutput()` in `src/core/cycle/transcript-discovery.ts` short-circuiting `isDreamOutput()`. `gbrain eval brainstorm <fixture.jsonl>` is a three-axis conjunctive gate (distance + usefulness + grounding — distance alone is gameable). `gbrain doctor` has a `brainstorm_health` check (migration applied, `search.track_retrieval` setting, an unpriced cross or judge chat model, calibration cold-start status). Cost policy lives in `src/core/brainstorm/cost-gate.ts`: no cap flag is the $5 `default` tracker cap (an unpriced model warns and runs, priced calls stay metered; the estimate, mid-run guard and pre-judge check count it at Sonnet rates), `--max-usd N` (legacy `--max-cost`) is a `user` cap that refuses a run touching an unpriced chat, judge or embedding model before any work with the `no-pricing.ts` refusal, `--max-usd off` drops the USD ceiling; every price reads `pricing.overrides`; results carry `cost.cap_usd`/`cost.cap_source`. `judges.ts` computes the judge token budget via `computeJudgeMaxTokens(ideaCount, modelId)` (named constants `TOKEN_BUDGET_PER_IDEA`, `TOKEN_BUDGET_ENVELOPE`, `LEGACY_MIN_MAX_TOKENS`, `MAX_OUTPUT_TOKENS_CEIL`; `ANTHROPIC_OUTPUT_CAPS` map: Opus 4.7 32K, Sonnet 4.6 / Haiku 4.5 64K, legacy Claude 3.5 8K) so a large multi-call judge doesn't truncate mid-JSON; with no `modelOverride` the cap routes through the gateway's actual configured chat model via `getChatModel()`. `--save` for both commands persists through the canonical ingestion path: `persistSavedIdea(engine, {slug, content, provenanceVia})` calls `importFromContent({noEmbed:true, sourcePath, holdNoncanonicalPrivate:true})` (a held private request refuses put_page's `visibility_not_applied` and any other refusal is a failed save, #7897; chunked + tagged + content_hash so search finds it, no embedding cost at save) THEN renders the saved row to disk via the shared `writePageThrough` helper (file rendered FROM the row so the two sinks can't diverge and `gbrain sync` doesn't churn it). `formatSaveOutcome(outcome, ctx)` returns an honest per-branch message (both-sinks, DB-only when no `sync.repo_path`/repo-not-a-dir, DB-saved-but-file-errored, total-failure → loud `save FAILED … NOT persisted` on stderr + nonzero exit) — `--save` never prints "Saved" when the DB write failed. `buildIdeaSlug(question, label, nonce?)` adds a random nonce suffix (injectable for tests) so two same-day runs sharing the first 60 slug chars don't clobber. `--json` callers stay DB-only. `buildBrainstormFrontmatterObject(result)` in orchestrator.ts returns the object form for `serializeMarkdown`. Pinned by `test/last-retrieved.test.ts`, `test/e2e/pglite-cli-exit.serial.test.ts` (IRON-RULE: real `bun src/cli.ts` subprocess against a hermetic PGLite tempdir asserts search/get/query exit 0 in <15s + daemon-survival), `test/fix-wave-structural.test.ts` (asserts the drain `await` is textually BEFORE `engine.disconnect`), `test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm,judges-maxtokens,save}.test.ts`. Open Collider source: `github.com/CL-ML/open-collider`.+- `src/core/brainstorm/{domain-bank,orchestrator,judges}.ts` + `src/commands/{brainstorm,lsd,eval-brainstorm}.ts` + `src/core/last-retrieved.ts` — bisociation-grounded idea generation pair: `gbrain brainstorm <question>` (defensible, cite-heavy, 4 close × 6 far, judge threshold 4.0/5, save by default) and `gbrain lsd <question>` (Lateral Synaptic Drift — inverted judge rejecting ideas with resistance >4.5 "too obvious", stale-page bias via `pages.last_retrieved_at`, 2 close × 12 far, axiomatic inversions required, ephemeral by default). The "domain bank" is prefix-stratified sampling from the user's own brain (`SELECT DISTINCT substring(slug from '^[^/]+/[^/]+')` cached 1h-TTL in `config` per source) tiebroken by `JOIN page_links` connection_count, with corpus-sampling fallback when fewer prefixes than M exist. Distance normalized to [0,1] via `1 - clamp(cosine_distance, 0, 2) / 2`. `judges.ts` exports `runJudge(config, ideas)` + two configs (`BRAINSTORM_JUDGE_CONFIG` weighted originality/resistance/thesis_density/concrete_grounding/cognitive_load 0.25/0.20/0.20/0.20/0.15 vs `LSD_JUDGE_CONFIG` cognitive_load 0.50 + inversion rule). Calibration cold-start fallback: when `calibration_profiles.active_bias_tags` is empty, judge runs without anti-bias context AND stderr-warns. Op-layer write-back in `src/core/operations.ts` `search`/`query`/`get_page` handlers fires `bumpLastRetrievedAt(engine, pageIds)` (fire-and-forget, 5-min throttled via SQL clause, default-on with `search.track_retrieval` config escape hatch, read per engine); internal callers (sync, migrations, dream cycle) bypass the op layer so the LSD stale signal stays clean. The fire-and-forget IIFE is tracked in a module-scoped `Set<Promise<unknown>>`; `awaitPendingLastRetrievedWrites(timeoutMs?: number): Promise<{outcome, pending}>` resolves once all tracked promises settle, bounded by a 5s `Promise.race` timeout that stderr-warns the pending count. `src/cli.ts` awaits the drain unconditionally for every op in the op-dispatch finally block BEFORE `engine.disconnect()`, then a fallback `process.exit(0)` fires ONLY when `outcome === 'timeout'` AND `shouldForceExitAfterMain(argv)` (excludes `serve` so daemons stay alive) — so the IIFE cannot race disconnect and leave PGLite's WASM keeping Bun's event loop alive. The timestamp lives in `page_retrievals(page_id, last_retrieved_at)` (v224): the bump is an `INSERT ... ON CONFLICT DO UPDATE` throttled to 5 minutes, a HOT update with no triggers that never advances `page_generation_clock_seq` (the old `UPDATE pages` advanced it on every read, expiring the query-cache bookmark and the get_health and projection-readiness memos). No foreign key; the `pages_forget_retrievals` statement trigger removes rows of hard-deleted pages. Readers (domain-bank samplers, volunteer stats, entity card) take `GREATEST(pages.last_retrieved_at, page_retrievals.last_retrieved_at)`; the legacy column stays (unindexed since v224) for older binaries; full forward-reference bootstrap probe on both engines. Frontmatter `mode: lsd` makes the dream-cycle synthesize phase skip LSD output via `isLsdOutput()` in `src/core/cycle/transcript-discovery.ts` short-circuiting `isDreamOutput()`. `gbrain eval brainstorm <fixture.jsonl>` is a three-axis conjunctive gate (distance + usefulness + grounding — distance alone is gameable). `gbrain doctor` has a `brainstorm_health` check (migration applied, `search.track_retrieval` setting, an unpriced cross or judge chat model, calibration cold-start status). Cost policy lives in `src/core/brainstorm/cost-gate.ts`: no cap flag is the $5 `default` tracker cap (an unpriced model warns and runs, priced calls stay metered; the estimate, mid-run guard and pre-judge check count it at Sonnet rates), `--max-usd N` (legacy `--max-cost`) is a `user` cap that refuses a run touching an unpriced chat, judge or embedding model before any work with the `no-pricing.ts` refusal, `--max-usd off` drops the USD ceiling; every price reads `pricing.overrides`; results carry `cost.cap_usd`/`cost.cap_source`. `judges.ts` computes the judge token budget via `computeJudgeMaxTokens(ideaCount, modelId)` (named constants `TOKEN_BUDGET_PER_IDEA`, `TOKEN_BUDGET_ENVELOPE`, `LEGACY_MIN_MAX_TOKENS`, `MAX_OUTPUT_TOKENS_CEIL`; `ANTHROPIC_OUTPUT_CAPS` map: Opus 4.7 32K, Sonnet 4.6 / Haiku 4.5 64K, legacy Claude 3.5 8K) so a large multi-call judge doesn't truncate mid-JSON; with no `modelOverride` the cap routes through the gateway's actual configured chat model via `getChatModel()`. `--save` for both commands persists through the canonical ingestion path: `persistSavedIdea(engine, {slug, content, provenanceVia})` calls `importFromContent({noEmbed:true, sourcePath, holdNoncanonicalPrivate:true})` (a held private request refuses put_page's `visibility_not_applied` and any other refusal is a failed save, #7897; chunked + tagged + content_hash so search finds it, no embedding cost at save) THEN renders the saved row to disk via the shared `writePageThrough` helper (file rendered FROM the row so the two sinks can't diverge and `gbrain sync` doesn't churn it). `formatSaveOutcome(outcome, ctx)` returns an honest per-branch message (both-sinks, DB-only when no `sync.repo_path`/repo-not-a-dir, DB-saved-but-file-errored, total-failure → loud `save FAILED … NOT persisted` on stderr + nonzero exit) — `--save` never prints "Saved" when the DB write failed. `buildIdeaSlug(question, label, nonce?)` adds a random nonce suffix (injectable for tests) so two same-day runs sharing the first 60 slug chars don't clobber. `--json` callers stay DB-only. `buildBrainstormFrontmatterObject(result)` in orchestrator.ts returns the object form for `serializeMarkdown`. Pinned by `test/last-retrieved.test.ts`, `test/e2e/pglite-cli-exit.serial.test.ts` (IRON-RULE: real `bun src/cli.ts` subprocess against a hermetic PGLite tempdir asserts search/get/query exit 0 in <15s + daemon-survival), `test/fix-wave-structural.test.ts` (asserts the drain `await` is textually BEFORE `engine.disconnect`), `test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm,judges-maxtokens,save}.test.ts`. Open Collider source: `github.com/CL-ML/open-collider`.- `src/core/budget/budget-tracker.ts` — the keystone budget primitive. One typed error (`BudgetExhausted` with `reason: 'cost' | 'runtime' | 'no_pricing'`), one schema-stable audit JSONL at `~/.gbrain/audit/budget-YYYY-Www.jsonl`. Contracts: `record()` throws when cumulative spend exceeds cap (the cap is a real ceiling, not a suggestion); `reserve()` admission checks cumulative + OUTSTANDING spend and returns an opaque `BudgetReservation` id for each held projection; `record({ reservation })` settles that id and `release(id)` frees an attempt that never billed (no-op when already settled), so N concurrent callers can't all pass against the same cumulative and a small same-model call finishing first can't free a large in-flight call's hold (settlement is by id, never by model; an id-less record settles nothing; a priced reservation recorded under an unpriced served id charges its held ceiling). `gateway.chat` reserves, then runs the attempt in one try/finally that releases the id (provider-resolution failure, invocation-policy refusal); embed reserves after its preflights; rerank, decide, the TypeSafe rerank adapter and embed-facts pass their ids. Record audit rows carry `requested_model` and `reservation`; `reserve()` hard-fails with `reason: 'no_pricing'` when `maxCostUsd` is set AND the model is missing from pricing maps, its message and `BudgetExhausted.pricing` built by `no-pricing.ts` (warn-once preserved when cap is unset, naming the registration command); `lookupPricing` normalizes recipe aliases through `resolveRecipe` before consulting the pricing tables, so `claude-cli:haiku` prices exactly like `claude-cli:claude-haiku-4-5-20251001` at `reserve()`, `record()` and `isModelPriceable()` alike (the gateway reserves with the pre-resolution string the user configured); `extractUsageFromError(err, fallback)` returns `err.usage` when the SDK provides it, else the pessimistic fallback (caller passes `maxOutputTokens`, not the optimistic pre-call estimate). `onExhausted(cb)` fires once synchronously BEFORE the throw propagates so callers can persist checkpoints. Pricing math lives in the pure `src/core/budget/reservation-cost.ts`: `priceFor(model, kind, overrides) → { pricing, source: 'override' | 'provider_wildcard' | 'table' } | null` is the one resolver (exact override, then a `<provider>:*` wildcard admitted only for recipes with `billing: 'subscription'` by `isAllowedPricingOverrideKey`, then aliases, the claude-cli → Anthropic sibling, canonical rows and free local providers); `isModelPriceable`, `usageCostUsd`/`reservationCostUsd`, skillopt preflight and synthesize_concepts all use it. `parsePricingOverrides` drops disallowed keys and adds each alias key's canonical twin. Every `record()` also feeds an in-memory model ledger keyed by (requested model, served model, touchpoint, purpose) — updated BEFORE the cap throw and before the unpriced early return — exposed as `snapshot().models`; `BudgetActualUsage` carries `requestedModelId`, `purpose`, `failed`, `countsAsCall` (default true) and `estimated`, and rows track `calls` (gateway operations), `attempts` (every record), `failed_calls`, tokens, `cost_usd` (`null` when unpriced) and `cost_basis` (`estimated` if any contributing record was). `src/core/budget/models-used.ts` maps labels to touchpoints (`gateway.chat`→chat, `gateway.expand*`→expansion, `gateway.embed*`→embedding, `gateway.rerank*`→rerank, `gateway.ocr*`→ocr, else other), builds receipt rows (`buildModelsUsed`), merges resumed segments and renders the "Models called" table. No DB change. Shared by brainstorm, cycle/budget-meter, and eval-contradictions; `BudgetMeter` keeps its public shape over it (`schema_version: 1` stamped on every dream-budget audit line). Pinned by 18 unit cases. A4 cap sources: `capSource: 'derived' | 'default' | 'user'` (getter `capSource`; an explicit `maxCostUsd` with no declared source is `user`, the pre-A4 behaviour). Only a `user` cap hard-fails an unpriced model, and that `BudgetExhausted` carries `fix` (`noPricingFix`, actor `agent` on the CLI, `host_admin` when `transport` is remote); `derived`/`default` caps warn once and run the model unmetered. Cost exhaustion carries `capSource`, and a `derived` one appends `derived_cap_exhausted` to the agent-contract log (callers report it with `consent.ts`'s `derivedCapExhaustedError`). Pinned by `test/budget-tracker-cap-source.test.ts`.- `src/core/budget/daily-ledger.ts` — #6188 the durable per-UTC-day USD ledger over `budget_ledger`/`budget_reservations` (migration v012; no new schema), one ledger per (scope, resolver), e.g. `FENCE_REPAIR_LEDGER` (`llm_repair`/`fences`). `reserve(estimate, { capUsd })` is one conditional upsert (`reserved + committed + estimate <= cap`) plus a reservation row with `expires_at`; `dispatch` marks it sent; `settle(id, actual)` is idempotent, bound to the reservation's own day, and records an overrun; `release` returns an undispatched hold; expired holds are reclaimed before each reserve (a dispatched one is charged its estimate). Any ledger error means no call (`ledger_unavailable`); `meteredCall` runs the whole protocol. Amounts are NUMERIC(12,4), spend rounded up and caps down. `estimateChatCallUsd` prices a chat call through `pricing.overrides` and the canonical table; an unpriced model is metered at the highest canonical rate (`estimated: true`) under a default cap and refused with `noPricingGuidance` under a user cap. Never opens a BudgetTracker, so a caller's ambient tracker keeps metering. Read by doctor `fence_integrity` (today's spend). Tests: `test/daily-ledger.test.ts` (+ `test/e2e/daily-ledger-postgres.test.ts`, concurrent sessions).diff --git a/docs/architecture/key-files/mcp.md b/docs/architecture/key-files/mcp.mdindex b274c0000d..d26bea8343 100644--- a/docs/architecture/key-files/mcp.md+++ b/docs/architecture/key-files/mcp.md@@ -29,4 +29,4 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/mcp/tool-defs.ts` — `buildToolDefs(ops, opts?)` helper; the stdio MCP server, the OAuth HTTP `tools/list`, and the subagent tool registry all consume it, byte-for-byte equivalence pinned by `test/mcp-tool-defs.test.ts`. `opts.strictParams: true` (when `mcp.strict_params` resolves `'reject'`) additionally declares the `_meta`/`dry_run` passthrough keys in `properties` and closes each schema with `additionalProperties: false` (schema-validating clients must not strip `_meta.session_id`); both emission states pinned. Exports the recursive `paramDefToSchema(p: ParamDef)` — single source of truth for ParamDef→JSON Schema mapping shared by `buildToolDefs` and `src/core/minions/tools/brain-allowlist.ts` (subagent registry). Recursive on `items` so nested array-of-arrays preserves inner shape on the wire, and on object `properties` (closed with `additionalProperties: false`, member `required` keys). Key ordering (type, description, enum, default, items, properties) is intentional so `JSON.stringify` output stays byte-stable. `toolAnnotations(op)` is the one annotation rule: curated op annotations win; else `readOnlyHint: true` for a read-scope op tagged `mutating: false`, `readOnlyHint: false` for one tagged `mutating: true`, nothing for untagged ops. `test/mcp-tool-defs.test.ts` has a `findArrayWithoutItems` walker that fails on any `type: 'array'` lacking `items.type`. Annotations: curated `op.annotations` win; else `readOnlyHint: true` iff `mutating === false`, `readOnlyHint: false` (+ `idempotentHint: true` when `idempotent`) iff `mutating === true`; every op declares both tags (`test/ops-mutation-tags.test.ts`).-- `src/mcp/validate-params.ts` — param normalization + validation for dispatch (direct unit surface). `validateParams` (required/type/enum — enum membership is a TYPE error in BOTH strict modes; the caller's raw value is never echoed into the message), `normalizeOptionalParams` (null/`''` optional-param idioms become truly absent, and so do the values a param's schema rules out: `0` below `minimum`/`exclusiveMinimum`, `[]` under `minItems`, `{}` on an object with a required member, `false` on an `actsWhenTrue` flag, then any param whose `appliesWith` param is absent (GST-927); required params untouched, copy-on-write), `findUnknownParams` (unknown top-level keys on the NORMALIZED object; `UNKNOWN_PARAM_ALLOWLIST` = `_meta` + `dry_run`; per-op did-you-mean from the op's OWN declared params only, plus the explicit `PARAM_ALIAS_HINTS` aliases edit distance cannot reach, e.g. `add_timeline_entry` `content` → `detail`; dispatch rejects unknown keys on every `mutating` op regardless of `mcp.strict_params`, so a typo never commits a write), `buildUnknownParamWarnBlock` (the model-visible warn notice), `resolveStrictParamsMode(engine, config)` (dual-plane `mcp.strict_params`, DB > file > `'warn'`; a failed DB read applies the LAST successfully read DB mode, held per process — fail-closed, so a transient config outage on a reject-mode server cannot re-open the warn grace period; test seam `resetStrictParamsModeCache()`). Privacy: raw unknown key names reach the CALLER only, never `mcp_request_log`. Pinned by test/validate-params.test.ts. Every path that hands arguments to an op handler runs normalize → validate first: `dispatchToolCall`, `handleToolCall` (`gbrain call`) and the SkillOpt rollout and write-capture registries, so a model that sends `""` or `null` for every optional argument reads the same as one that omits them (`test/tool-args-empty-optional-sol.test.ts`). On a remote call the audience and attested-actor checks in `dispatchToolCall` read the normalized arguments, so `acting_user: ""` or `null` counts as omitted.+- `src/mcp/validate-params.ts` — param normalization + validation for dispatch (direct unit surface). `validateParams` (required/type/enum — enum membership is a TYPE error in BOTH strict modes; the caller's raw value is never echoed into the message), `normalizeOptionalParams` (null/`''` optional-param idioms become truly absent, and so do the values a param's schema rules out: `0` below `minimum`/`exclusiveMinimum`, `[]` under `minItems`, `{}` on an object with a required member, `false` on an `actsWhenTrue` flag, then any param whose `appliesWith` param is absent (GST-927); required params untouched, copy-on-write), `findUnknownParams` (unknown top-level keys on the NORMALIZED object; `UNKNOWN_PARAM_ALLOWLIST` = `_meta` + `dry_run`; per-op did-you-mean from the op's OWN declared params only, plus the explicit `PARAM_ALIAS_HINTS` aliases edit distance cannot reach, e.g. `add_timeline_entry` `content` → `detail`; dispatch rejects unknown keys on every `mutating` op regardless of `mcp.strict_params`, so a typo never commits a write), `buildUnknownParamWarnBlock` (the model-visible warn notice), `resolveStrictParamsMode(engine, config)` (dual-plane `mcp.strict_params`, DB > file > `'warn'`; a failed DB read applies the LAST successfully read DB mode, held per engine — fail-closed, so a transient config outage on a reject-mode server cannot re-open the warn grace period; test seam `resetStrictParamsModeCache()`). Privacy: raw unknown key names reach the CALLER only, never `mcp_request_log`. Pinned by test/validate-params.test.ts. Every path that hands arguments to an op handler runs normalize → validate first: `dispatchToolCall`, `handleToolCall` (`gbrain call`) and the SkillOpt rollout and write-capture registries, so a model that sends `""` or `null` for every optional argument reads the same as one that omits them (`test/tool-args-empty-optional-sol.test.ts`). On a remote call the audience and attested-actor checks in `dispatchToolCall` read the normalized arguments, so `acting_user: ""` or `null` counts as omitted.diff --git a/docs/architecture/key-files/persistence-stalls.md b/docs/architecture/key-files/persistence-stalls.mdindex 6ca7944c74..aad216a454 100644--- a/docs/architecture/key-files/persistence-stalls.md+++ b/docs/architecture/key-files/persistence-stalls.md@@ -43,6 +43,6 @@ owner (`owner_wedged_here`), movement surfaces (`data_moving`, doctor- `src/core/persistence/consumer-election.ts` — #6317 (B1b): the consumer mode machine (`probing` → `full` | `waiter_only` → `promoted`) and `WaiterOnlyConsumer`, the `PersistenceConsumerLike` surface for a resident process (`sync`, `jobs`, `autopilot`, `mcp`) that found a live full consumer of this host and claims nothing: `wake()` is a no-op, settlement listeners still fire through `settledWaiters`, `status()` reports `waiter_only` with the owner's identity. `electOwner` re-probes every tick and promotes on the owner's progress (lapsed 60 s, wedged, not full, gone); a promoted consumer drains back after three healthy ticks of a `full` owner only, so promotion storms converge. `forceOwnConsumer(reason)` is `--no-delegate`'s process flag.- `src/core/persistence/consumer-lane.ts` — #6317 (reporter ask 2): `consumerStatementEngine`, the Proxy that routes the consumer's own tick statements (idle probe, switch read, recovery and expired-claim scans, capacity, releases, floor) to `executeRawDirect` while the dual pool is active (`GBRAIN_DIRECT_DATABASE_URL` or the Supabase-derived direct URL), so the owner's round-trips avoid a transaction-mode pooler; `status().connection = { lane, pooler_mode, direct_host? }` says which lane is in use, and one `[persistence] phase=start reason=transaction_pooler` line prints when only a `prepare=false` URL exists.- `src/core/persistence/consumer-diagnostics.ts` — #6317 (B3, I1): `hostConsumersReport` (the `host.consumers` block of `writer status --json`), `consumerOverlap` (two resident-kind full rows alive over 30 s), `ownersWithoutHeartbeat` (claim owners on running claims with no row) and `hostIdentityMismatches` (this process's `host.json` id differs from a binding owner's while the owner's checkout resolves on this filesystem or the machine id matches), read-only and shared by doctor and writer status.-- `src/core/persistence/publication-deadline.ts` (#6288 part 1, #6352; UC2) — what bounds a publish that neither resolves nor rejects. On Postgres the publish transaction is the real bound: `declareDurablePersistence` already sets a transaction-local `statement_timeout` (5 s) and `lock_timeout`, and `declarePublication` (#6712 L6: the opening statement of `publishMutation`'s and `publishGroup`'s transactions, the same `SELECT` as `declareDurablePersistence`, so no extra round trip even where a transaction-mode pooler turns pipelining off) adds a transaction-local `idle_in_transaction_session_timeout` of `persistence.publication_ceiling_ms` (default 300000, 60000 to 3600000; never above the pool's own `GBRAIN_IDLE_TX_TIMEOUT`), so a publisher stuck between statements is ended by the server and commits nothing; both survive a transaction pooler that drops startup parameters. Never set on PGLite: it would enforce the bound, and a transaction it ends wedges the engine's only connection. `boundPublication` (the consumer's single path and `executeClaimedGroup`'s group publish and per-member fallback) settles only with the publish's own outcome, so nothing releases or reports a request or group while its transaction can still commit; at the ceiling the claim stamp reads `waiting_on: publication_deadline`, after `PUBLICATION_GRACE_MS` (30 s) it aborts the publication's signal (passed to `engine.transaction(fn, { signal })` through `publicationTransaction`: on Postgres the connection is discarded and nothing commits; the rejection waits for the transaction body to settle, then surfaces with the driver's `CONNECTION_CLOSED` code, so the claim is requeued for a retry; PGLite checks the signal only before BEGIN), and after `PUBLICATION_SETTLE_MS` (5 s) more the consumer lists it in `stuck_publications` and reports `restart_required` (its root stays held: restarting the owner is the recovery, never a promised reclaim). The lease keeps renewing, so a reclaim never races a publisher that may still write files. The wave 9 yield between groups is outside any one bound. `PublicationBound.later` is the scheduler seam (default a real unref'd timer); the unit test drives the three phases through a manual clock so a stalled runner cannot fire a phase early (the consumer's `clock` seam is separate). Tests: `test/persistence-publication-deadline.test.ts` (barriers after the token lock, after the file rename and before commit; the server-side idle end on Postgres; the wiring probe).+- `src/core/persistence/publication-deadline.ts` (#6288 part 1, #6352; UC2) — what bounds a publish that neither resolves nor rejects. On Postgres the publish transaction is the real bound: `declareDurablePersistence` already sets a transaction-local `statement_timeout` (5 s) and `lock_timeout`, and `declarePublication` (#6712 L6: the opening statement of `publishMutation`'s and `publishGroup`'s transactions, the same `SELECT` as `declareDurablePersistence`, so no extra round trip even where a transaction-mode pooler turns pipelining off) adds a transaction-local `idle_in_transaction_session_timeout` of `persistence.publication_ceiling_ms` (default 300000, 60000 to 3600000, read per engine; never above the pool's own `GBRAIN_IDLE_TX_TIMEOUT`), so a publisher stuck between statements is ended by the server and commits nothing; both survive a transaction pooler that drops startup parameters. Never set on PGLite: it would enforce the bound, and a transaction it ends wedges the engine's only connection. `boundPublication` (the consumer's single path and `executeClaimedGroup`'s group publish and per-member fallback) settles only with the publish's own outcome, so nothing releases or reports a request or group while its transaction can still commit; at the ceiling the claim stamp reads `waiting_on: publication_deadline`, after `PUBLICATION_GRACE_MS` (30 s) it aborts the publication's signal (passed to `engine.transaction(fn, { signal })` through `publicationTransaction`: on Postgres the connection is discarded and nothing commits; the rejection waits for the transaction body to settle, then surfaces with the driver's `CONNECTION_CLOSED` code, so the claim is requeued for a retry; PGLite checks the signal only before BEGIN), and after `PUBLICATION_SETTLE_MS` (5 s) more the consumer lists it in `stuck_publications` and reports `restart_required` (its root stays held: restarting the owner is the recovery, never a promised reclaim). The lease keeps renewing, so a reclaim never races a publisher that may still write files. The wave 9 yield between groups is outside any one bound. `PublicationBound.later` is the scheduler seam (default a real unref'd timer); the unit test drives the three phases through a manual clock so a stalled runner cannot fire a phase early (the consumer's `clock` seam is separate). Tests: `test/persistence-publication-deadline.test.ts` (barriers after the token lock, after the file rename and before commit; the server-side idle end on Postgres; the wiring probe).- `src/core/persistence/transient-retry.ts` — the retry budget for a managed write's transient interruptions: a statement the database ended with 57014 (`statement_cancel`) or a publication `boundPublication` cancelled at the ceiling (`publication_ceiling`, read from the publication's signal by `publicationDeadlinePassed`, whatever error it surfaces as). `chargeTransientInterruption` counts it in `persistence_requests.transient_interruptions` (v261, never `preparation_attempts`) and sets `next_attempt_at` to 1 s, 2 s, 4 s, 8 s, then 15 s (under the drain's 30 s no-progress window); `BACKOFF_ELAPSED` keeps every claim (`claimableWriteSql`, lane heads, group followers) off the row until then, so no claim is held while it waits, and `readHeadBlockers` names it `transient_backoff`. Past `persistence.max_transient_retries` (default 5, 0 to 20, a preparation-budget key) coordinator.ts ends the request `write_interrupted`; the charge commits with the claim's release. Off with `preparation_deadlines`. Tests: `test/pathological/lp-b-transient-retry-budget.test.ts`, `test/pathological/lp-c-publication-ceiling-retry.test.ts` (`_setPublicationTimingForTest`), `test/persistence-write-interrupted.test.ts`.- `src/core/persistence/sync-discovery.ts` (#6349 staged catch-up) — when an incremental delta (not `--full`, not company) exceeds the cursor bound (100,000 entries, 16 MiB; `_setCursorBoundForTest` lowers it), `stagedSyncTarget` gallops forward over `git rev-list --first-parent last..HEAD`, verifying every candidate's delta from the last synced commit against the bound (a net delta is not monotone, so no binary search), and discovery reruns pinned at the furthest fitting commit with no working-tree entries, recording `stagedHead`; the cursor carries it and `SyncResult.staged { target, head, stages }` reports it. One commit alone above the bound refuses `request_too_large` naming the commit. `commands/sync/perform.ts` loops a drain stage by stage to HEAD (the drain's one extra HEAD-moved pass is skipped for a staged result). Tests: `test/sync-staged-catchup.test.ts`.diff --git a/docs/guides/concurrent-writes.md b/docs/guides/concurrent-writes.mdindex 12ca548877..1f6db12fca 100644--- a/docs/guides/concurrent-writes.md+++ b/docs/guides/concurrent-writes.md@@ -1010,7 +1010,7 @@ stored, and compaction frees the window's reservations as receipts age out.within `persistence.capacity.warn_days` (default 30), adds the`[gbrain notice write_capacity_low]` notice (kind `info`) with the same`gbrain config set` command. Write and sync receipts carry it at most once an-hour per process per scope (a remote client sees only the brain and its own+hour per engine per scope (a remote client sees only the brain and its ownprincipal); `gbrain sync status`, doctor and autopilot cycle output carry itevery time. `gbrain notices mute write_capacity_low` mutes it on receipts;doctor still reports it. When a cap passesdiff --git a/docs/guides/search-modes.md b/docs/guides/search-modes.mdindex 52c45bf523..70ec17ec8c 100644--- a/docs/guides/search-modes.md+++ b/docs/guides/search-modes.md@@ -203,7 +203,7 @@ Several of the knobs deserve a sentence:order keeps closer in-filter candidates that strict order drops; resultsare re-sorted either way. `gbrain config set search.hnsw_iterative_scanstrict_order` restores the older mode (restart serve and autopilot: it is- read once per process); another value is refused with exit 2.+ read once per engine, so once per process on a one-brain server); another value is refused with exit 2.### Setting and resolving the modediff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.mdindex a8fdfa2b76..9431fe3f84 100644--- a/docs/guides/write-refusals.md+++ b/docs/guides/write-refusals.md@@ -124,7 +124,7 @@ of making a duplicate.| <a id="upload_too_many_parts"></a>`upload_too_many_parts` | `upload_too_many_parts` | An `upload_page_part` call would be part number 64 plus one per 64 KiB of the declared `total_bytes` (one per eighth of `part_max_frame_bytes` when that is under 512 KiB) or later without completing the content. Each part is a stored row, so a session of tiny parts is bounded. The part was not stored. | Send all remaining bytes in that part if they fit within `part_max_frame_bytes`; otherwise abort and begin again, cutting parts near `part_max_frame_bytes` (`get_page_upload` returns it). || <a id="part_splits_code_point"></a>`part_splits_code_point` | `part_splits_code_point` | An `upload_page_part` text starts or ends with half of a surrogate pair, which has no UTF-8 encoding on its own. The part was not stored. | Move the cut before the high surrogate so the pair travels in one part, then send the part again. || <a id="write_quota_insufficient"></a>`write_quota_insufficient` (exit 12) | `write_quota_insufficient` | A bulk run (a managed sync catch-up, a reclone, a repair) estimated from its manifest more new receipts (ids and 16 KiB reservations) than the capacity left before its producer's pause line, so it did not start. Other writes still work. | Run the `gbrain config set <key> <value>` the message prints on the brain host, then rerun the same command. Verify with `gbrain doctor --only persistence_capacity --json`. |-| <a id="write_capacity_low"></a>`write_capacity_low` (notice) | `write_capacity_low` | Not a refusal: a `[gbrain notice]` on write receipts, `sync` receipts, `gbrain sync status` and autopilot cycle output when a cumulative cap is at or past 80% or would run out within `persistence.capacity.warn_days` (30) days at its 7-day rate. Receipts carry it at most once an hour per process and scope; `sync status` and doctor carry it every time. The structured `capacity` object names the scope, resource, days left and the config command. | Follow the [runbook](concurrent-writes.md#quota-near-exhaustion): raise the cap it names with the printed `gbrain config set`, then verify with `gbrain doctor --only persistence_capacity --json`. |+| <a id="write_capacity_low"></a>`write_capacity_low` (notice) | `write_capacity_low` | Not a refusal: a `[gbrain notice]` on write receipts, `sync` receipts, `gbrain sync status` and autopilot cycle output when a cumulative cap is at or past 80% or would run out within `persistence.capacity.warn_days` (30) days at its 7-day rate. Receipts carry it at most once an hour per engine and scope; `sync status` and doctor carry it every time. The structured `capacity` object names the scope, resource, days left and the config command. | Follow the [runbook](concurrent-writes.md#quota-near-exhaustion): raise the cap it names with the printed `gbrain config set`, then verify with `gbrain doctor --only persistence_capacity --json`. || <a id="write_capacity_backpressure"></a>`write_capacity_backpressure` (exit 12) | `write_capacity_backpressure` | A background producer (managed maintenance, conversation facts, an autopilot cycle phase, an autopilot catch-up sync) stopped admitting because a cumulative cap of its writer or the brain is past `persistence.capacity.background_pause_pct` (default 0.8). Interactive writes (`remember`, `put_page`, `edit_page`, an operator's `gbrain sync`, an approved `gbrain repair`) keep the remainder. No receipt was written. `reason` names the resource. Doctor `persistence_capacity` names the paused producers. | Depends on `reason`. `lifetime_ids`: ids are never freed, so only the brain host's `gbrain config set <key> <value>` from the `fix` resumes background work; waiting does not. `terminal_bytes`: the same reversible raise resumes it at once, or it resumes by itself as retained receipts compact. Verify with `gbrain doctor --only persistence_capacity --json`. |### Other error codesdiff --git a/docs/operations/mcp-surface-runbook.md b/docs/operations/mcp-surface-runbook.mdindex 1b6c3b0c69..b4b1f8c948 100644--- a/docs/operations/mcp-surface-runbook.md+++ b/docs/operations/mcp-surface-runbook.md@@ -181,7 +181,7 @@ passthrough keys (D14.1), keeping schema-validating clients aligned withthe server's reject posture. Read per request; flipping back to `warn`reopens the schemas on the next list. A transient config-read failurecannot re-open the grace period: dispatch holds the last successfully-read mode per process, so a reject-mode server stays reject through a+read mode per engine, so a reject-mode server stays reject through aconfig outage. `test/mcp-tool-defs.test.ts` pinsboth emission states; the default stays `warn` until the project-levelflip (see TODOS.md, strict_params reject-flip).