Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Merge master 45136aab into the doctor/dream/enrich train so its cheap tier can run

garrytana86521272df8
diff --cc CHANGELOG.md
index 07281c7bf4,e51418d1f5..01ff2049e3
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -10,66 -10,51 +10,111 @@@ credits are retained; no result has bee
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**Doctor, dream, enrich and `maintain --safe` fixes, landed together.** `gbrain maintain --safe` no longer deletes pages or calls a paid provider, several doctor checks now report what they find accurately, dream phases stop and report correctly, and enrich, model-call errors, atom provenance and transcript dates get community-reported fixes. Each fix has a test that fails without it.
+
+**The `unindexed_files` maintenance phase no longer reports a complete scan when a source could not be scanned.** A source whose Git checkout failed to read was counted as an error and the phase warned, but its `details.complete` still said `true`, so anything reading the phase result took the run as a full scan of every managed source. `complete` is now `true` only when no source stopped at its time budget and none errored. Reported by @Masashi-Ono0611 (#8887).
+
+**A source with automatic sync off now walks forward through its pages on a database whose DateStyle is not ISO.** The per-source maintenance window saved its cursor in the session's date style. Under a day-first style such as `German, DMY` or `SQL, DMY`, a cursor dated after the 12th of a month was not a date the cycle handler could read, so it dropped the cursor and the next cycle started again from the source's oldest pages; a source over 2,000 pages never got past its first window. The cursor is now always written as a UTC ISO timestamp with microseconds. A source holding a cursor in the old form may start from its oldest pages one last time; from then on it advances. Reported by @javieraldape (#8771).
+
+**`gbrain dream --drain` stops as no progress when its batches only skip items the backlog keeps counting.** The drain checked for no progress only after a batch that extracted nothing and skipped nothing. A page the phase skips on every batch while the backlog still counts it kept that check from ever running, so the drain ran all 1,000 batches, held the cycle lock for minutes and ended `max_batches` with exit 11, which tells the caller to rerun a stop a rerun cannot clear. A batch that extracts nothing now always rechecks the backlog, and when it did not shrink, the drain stops `no_progress` (`atom_drain_no_progress`, exit 1), and skips that do shrink it keep draining. Reported by @h6y3 (#8445).
+
+**`gbrain dream --help --json` prints its help as one JSON document.** Under `--json`, stdout carries only a JSON document, and dream printed its help as plain text, which went to stderr. So stdout was empty, the run exited 0 and logged a missing-document event that `gbrain doctor` counts under its agent-contract check. It now prints `{"command": "dream", "help": "<the help text>", "contract_version": 1}`, and `dream retriage --help --json` and `dream reset-key --help --json` do the same. `gbrain dream --help` without `--json` prints exactly the same text as before. Reported by @Kkkkkuro (#8510).
+
+**Concept synthesis stops when its maintenance job times out or is cancelled.** The `synthesize_concepts` dream phase never looked at the cycle's abort signal, so with a slow model it kept calling the model and publishing concept pages after its job had timed out, and an aborted model call could be written as a template fallback page. The phase now passes the signal to the model call and checks it before each concept and again right before each page is written, so nothing is published once the job is cancelled. The spend of the model calls that finished and the failed-attempt counts are still saved, so the recorded spend and the retry limit keep counting them. On a managed brain, concepts synthesized but not yet published when the job stops are synthesized again on the next run. No setting changed. Reported by @Nazim22 (#7986).
+
+**Doctor checks that misreported their findings now report them accurately.** A community bug wave: each fix below was reported by a contributor, reproduced on master with a failing test first, and rewritten here.
+
+**`gbrain maintain --safe` no longer deletes pages or calls a paid provider.** It used to run the whole dream cycle on the default source, so pages soft-deleted more than 72 hours earlier were purged for good, and the embedding, synthesis and other model phases ran too. The source cycle now runs only free phases that never purge: lint, backlinks, sync, the fence and content repair lanes, unindexed files, extract, extract_facts, symbol edges, emotional weight, consolidate, orphans and schema suggestions. It also holds back `extract_facts` when an embedding provider is set up or the decide conflict slot is on (its conflict sweep calls the decision provider), and the fence and content repair lanes when an embedding provider is set up (their publications embed) or `fences.repair.llm` has a repair model. The JSON report lists the phases that ran and the ones held back. Old soft-deleted pages are still purged by `gbrain pages purge-deleted` (preview it with `--dry-run`), `gbrain dream` and an installed autopilot, and the paid phases still run from `gbrain dream`.
+
+**Enrich write-ups from thinking models are no longer cut off.** Enrich capped every write-up at 2048 output tokens, which thinking-by-default models such as Claude 5 spend on reasoning, so write-ups came back truncated or failed. Thinking models now get the gateway's thinking headroom; other models keep 2048.
+
+**An OpenAI account with no credits left is reported as a billing problem instead of being retried.** OpenAI answers with HTTP 429 when an organisation has no credits, and gbrain read that as a rate limit and retried. A provider body whose error type or code is a billing code now decides billing wherever the error is wrapped.
+
+**Atom quotes keep their provenance when the model drops Markdown or punctuation, and a failed fence step no longer books a healthy facts round.** Quote matching skips line-leading `#` and `>` markers, quotation marks and a spaced dash, never words or number signs; ambiguous or reworded quotes still fail closed. A fence write that fails with an I/O or database error now counts as failed and stops the `facts.fence` round, so doctor's `extract_health` warns.
+
+**A write-up that has its own dated timeline section round-trips.** A page with no timeline whose body had a `## Timeline highlights` section re-parsed that section as the page timeline, so enrich refused its own paid write-up with `CONCEPT_REPUBLICATION_LOSSY`.
+
+**Transcripts without a date in the filename take the date in their frontmatter, and the atoms extracted from a transcript take the transcript's date.** Both used to fall back to the date of the run.
+
+**The admin dashboard's live feed keeps reconnecting after a dropped connection,** and four smaller fixes land with it: the load throttle still applies its concurrency and memory limits when the machine reports no load data (Windows), a `remember` TTL past the representable date range is refused as `invalid_params` instead of throwing, a worker pool whose error callback throws now stops every worker before rejecting, and claude-cli tool loops read the previous turn's prompt cache instead of rewriting the whole transcript to it.
+
+### Itemized changes
+
+- `src/core/cycle/unindexed-files.ts`: `details.complete` also requires `errors.length === 0`. Test: `test/unindexed-files.test.ts` (a source whose `.git/HEAD` is unreadable: phase `warn`, one error, `complete: false`).
+- `src/core/cycle/source-window.ts`: the listing instant and the last page's `updated_at` are formatted with `to_char(... AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"')`. Test: `test/sync-disabled-cycle-window.test.ts` (the real queued handler under `German, DMY` advances window by window; `German`, `SQL` and `Postgres` styles under a +05:45 time zone give ISO UTC cursors with microseconds and the same windows), now also run on Postgres by `test/e2e/sync-disabled-cycle-window-postgres.test.ts`.
+- `src/core/cycle/extract-atoms-drain.ts`: the no-progress recount runs after every batch with `extracted === 0`, not only when `skipped` is also 0. Test: `test/extract-atoms-drain.test.ts` (a skip-only batch over a flat backlog stops `no_progress` after one batch; skip-only batches that shrink the backlog drain).
+- `src/cli/help/render.ts`: new `printHelpText(command, text, args)`, the `--help --json` document for a handler that prints its own help text; `src/commands/dream.ts`, `dream-retriage.ts` and `dream-reset-key.ts` print their help through it. Test: `test/dream-help-json-8510.test.ts` (the real CLI: one document per help, no `json_document_missing` event, plain `--help` stdout equal to the document text).
+- `src/core/cycle/synthesize-concepts.ts`, `src/core/cycle.ts`: `signal` option, passed from the cycle like extract_atoms', handed to the narrative `chat` call and checked before each concept, right before each publication and before the final managed flush; an abort saves the retry counters and the finished spend to the rollup, then throws `AbortError`. Test: `test/cycle/synthesize-concepts-abort-7986.test.ts` (`runCycle` under the creator pack: a job aborted during the second model call publishes only the first concept and makes no third call; an aborted call is not published; an abort during the page read before the first publication publishes nothing; an aborted run still writes its spend and failed attempts).
+- `timeline_history`: on a brain with more than 10,000 pages carrying database-only timeline rows, a pass keeps only the first 10,000 for its final recount. It used to report that recount as exact, and a pass whose kept pages had been repaired was recorded as clean while the rest still held database-only rows. Such a pass now reports its counts as a lower bound ("at least"), keeps warning, and is never trusted as clean. A pass saved by an earlier release, which did not record whether it dropped pages, is treated the same way when its kept list is full. Reported by @Masashi-Ono0611 (#8511).
+- `conversation_format_coverage`: the sample now includes pages stored under the granular conversation types the extractor already reads (`slack-thread`, `slack-dm-day`, `email-digest`), so unparseable transcripts on a brain that never ran the type-consolidation pack are reported instead of silently skipped. Reported by @Masashi-Ono0611 (#8519).
+- `minions_migration` under `gbrain doctor --fast`: a migration the brain's config can decline (v0.53.0, `shared_skills.migration.enabled`) is no longer listed as not run. Without a database connection the check cannot read that setting, and `gbrain apply-migrations --yes` skips a declined migration, so the warning could never clear. The full `gbrain doctor` still reports it when it is not declined. Reported by @Masashi-Ono0611 (#8721).
+- `stale_mentions`: each stale example now carries its own `gbrain unlink` command with the link type the row is stored under, shell-quoted so a slug or link type with quotes or `$(...)` pastes as plain words, with the options first and `--` before the two slugs, so a slug that starts with `--` is never read as a flag. The hint used to say `--link-type mentions` for every row, which removes nothing for a typed link stored under a verb such as `founded`. Reported by @Masashi-Ono0611 (#8817).
+- `schema_pack_consistency`: pages gbrain itself wrote (extract receipts under `extracts/`, and pages from the code importer) no longer count as undeclared on a custom schema pack copied before the base pack declared their types. The exemption follows how the page was written, not its type name. A page of yours that uses one of those types, such as a Markdown note with `type: code`, still warns, as do your own undeclared types. Reported by @Masashi-Ono0611 (#8818).
+- `embedding_state` and `gbrain embed --stale`: a page whose body holds only a protected takes fence (which never reaches search chunks) is now counted `empty` instead of `chunkless_healable`. Doctor no longer warns about it, and `embed --stale` no longer plans a heal that can never produce a chunk. A page is treated this way only when the writer already chunked its current revision to nothing; a chunkless page that was never chunked, a page whose chunks were deleted after it was chunked, or an `embed_skip` page an older release stored without chunks, is still healed. Deleting a page's chunks clears that seal, in `deleteChunks`, an empty chunk replacement and `purge_fact`'s chunk delete (which also empties pages whose body held no purged row), so such a page stays healable. Reported by @akinduroifedayo (#8502).
+- `managed_sync_backlog`: removing a source in the middle of a managed sync now removes its sync cursor too, whether the brain is managed or not. The cursor used to stay behind, and doctor warned about a stalled backlog forever with a resume command that failed `unknown_source`. A cursor an earlier removal already left behind is no longer reported, including after the source is added back under the same id. Reported by @omni-nordaware (#8741).
+- `src/commands/maintain.ts`: `MAINTAIN_SAFE_PHASES` is the allowlist of free, non-purging phases; `maintainSafePhases` intersects it with the source's own phase list and holds back provider-gated phases (`extract_facts` on an embedding provider or an enabled decide conflict slot, `fence_repair` and `content_repair` on an embedding provider or a resolved repair model); the dry-run and applied actions report `phases` and `held_phases`; the help text names `gbrain pages purge-deleted` and `gbrain dream`.
+- `src/core/behavior-change-notice.ts`: one row for the changed `maintain --safe` behavior.
+- `scripts/module-size-limits.tsv`: the notice file's ceiling grows by one for that row.
+- Docs: the `maintain.ts` row in `docs/architecture/key-files/commands-4.md`.
+- Tests: `test/maintain.test.ts`: the allowlist holds no purge or paid phase; each provider holds its phase; a non-default source keeps its subset; on PGLite a page soft-deleted 100 hours ago survives `--safe` and no unsafe phase runs; a keyless brain with the decide conflict slot on holds `extract_facts`.
+- `src/commands/enrich.ts` (`defaultSynthesize`): thinking models get the gateway's thinking output headroom; other models keep the 2048-token cap. A thinking-model write-up can reserve up to 32000 output tokens against `--max-usd` per in-flight call. Test: `test/enrich/thinking-cap-7564.test.ts`. Reported by @mattverlaque (#7564).
+- `serializeMarkdown`, `edit_page`'s view and the timeline file splice: a body with no timeline sentinel whose own dated section would split under the bare-section fallback ends with an empty `<!-- timeline -->`; every other body keeps its bytes. Test: `test/markdown-own-timeline-section-7738.test.ts`. Reported by @mattverlaque (#7738).
+- `src/core/ai/errors.ts`: "no credits remaining" is a billing phrase, and a billing `error.type`/`error.code` decides billing through every wrapper layer. Test: `test/ai/classify-billing-credits-8400.test.ts`. Reported by @Masashi-Ono0611 (#8699, #8400).
+- `src/core/ai/providers/claude-cli-language-model.ts`: one content block per message over `--input-format stream-json` (rendered text unchanged, lone surrogates still become U+FFFD), so each turn reads the previous turn's cache. The subscription-only env scrub is unchanged. Test: `test/claude-cli-prompt-cache-7352.test.ts`. Reported by @spiky02plateau (#7352).
+- `src/core/cycle/extract-atoms.ts` (`locateQuote`): folds with the shared grounding core's markup option. Test: `test/cycle/extract-atoms-quote-markup-8373.test.ts`. Reported by @Masashi-Ono0611 (#8373).
+- Legacy fence step of `extract_facts`: an untyped fence write error counts in `pagesFailed` and halts the round with `item_error`; typed refusals, a busy source lock, segmented pages and files on another host stay holds. Test: `test/extract-facts-fence-failure-health-8758.test.ts` (PGLite and Postgres, managed and unmanaged). Reported by @javieraldape (#8758).
+- Transcript discovery and `--input`: an undated basename falls back to the frontmatter `date`; impossible calendar days are rejected and the date window filters on the resolved date. Test: `test/transcript-discovery-frontmatter-date.test.ts`. Reported by @spiky02plateau (#7267).
+- `src/core/cycle/atom-source-date.ts`: new `transcriptAtomSourceDate` (frontmatter date, else the path date that names the `atoms/<date>/` folder); an undated transcript still gets none. Test: `test/extract-atoms-source-date.test.ts`. Reported by @spiky02plateau (#7268).
+- Admin dashboard: the EventSource `onerror` only sets the status from `readyState` and no longer closes the stream; `admin/dist` and `src/admin-embedded.ts` are rebuilt. Test: `test/admin-dashboard-stream-reconnect.test.ts`. Reported by @ramybarsoum (#8913).
+- Load throttle (`shouldProceed`), `remember` TTL parsing and the worker pool: see the paragraph above. Tests: `test/backoff.test.ts`, `test/writeback-config.test.ts`, `test/worker-pool.test.ts`. Reported by @ramybarsoum (#8913).
+
+ ## [0.60.659.0] - 2026-11-11
+
+ ### A page purge removes the page's name from the volunteer log and the mention gazetteer
+
+ **Purging a page now also deletes the rows that kept its name next to its slug.** Before, `delete_page purge:true` and the retention purge left two stores behind, because neither has a foreign key to `pages`: `context_volunteer_events` kept the slug with a rationale that quotes the matched name (`alias match "<name>"`) for up to 90 days, and `mention_gazetteer_entries` kept the page's normalized title and aliases until that source's next mention pass. The purge receipt called the page gone while both were still readable. Reported and first fixed by @alanshurafa (#9072).
+
+ | What you did | Before | Now |
+ |---|---|---|
+ | `delete_page purge:true`, the expired-tombstone purge, or a restore that replays a page purge | the slug and name stayed in the volunteer log and the mention gazetteer | removed in the purge transaction, and the page receipt checks both stores |
+ | `purge_fact` on a claim that is also another page's name or quoted in a volunteer rationale | the gazetteer was not checked | both stores are probed and a match is listed as a residual |
+
+ - `src/core/persistence/purge-name-stores.ts` (new): `scrubPurgedNameStores` deletes the source's rows for the purged slugs from both stores. `purgePageInTransaction`, `purgeDeletedPages`, `expireRecordedTombstones` and the restore-time purge replay call it beside the retrieval-feedback scrub. A soft delete keeps both, so `restore_page` still brings the page back whole.
+ - `src/core/deletion-inventory.ts`: both tables are `swept` (`page_names`); the fact receipt still probes them, since a fact purge sweeps neither.
+ - Tests: `test/purge-name-stores-9068.test.ts` (PGLite, and Postgres through `test/postgres-unit-arms.txt`).
+
+ ### Removing a source, a connector delete and a late eval capture no longer leave a deleted page's slug behind
+
+ **Four more ways a deleted page's slug outlived the page are closed, and the federation warning stops counting private pages.** Before, only a page purge scrubbed the stores that name a page without a foreign key to it.
+
+ | What you did | Before | Now |
+ |---|---|---|
+ | A query or search with eval capture on, whose capture runs after the response, racing a page purge | the capture could write the purged slug back into `eval_candidates.retrieved_slugs` | the insert keeps only slugs whose page still exists, checked under the same row lock the feedback writers take |
+ | A volunteered-context answer whose background volunteer-log write lands after a page purge (or while it is open) | the write put the purged slug and its name-quoting rationale back into `context_volunteer_events` | the write keeps only rows whose page still exists, under the same row lock |
+ | A connector, GitHub or Google sync deletes a page (`deletePages`), or a transcript re-ingest deletes a stale part | retrieval logs, ratings, weights, the volunteer log, the gazetteer and captured eval slugs kept naming it | scrubbed in the delete's transaction |
+ | `gbrain sources remove`, `sources purge`, the expired-archive sweep, or a managed `remove`/`purge` | the source's retrieval logs, ratings, volunteer rows and captured eval slugs stayed | deleted in the statement that deletes the source, and swept again right after for a write that raced it, unless a source with the same id was created in between |
+ | `gbrain sources federate` or `sources restore` on a source with a remote-written `visibility: private` page and no readers | the warning named that page, though no-grant connections still cannot read it | the warning applies the read paths' privacy rules, unless `search.remote_private_pages` or `GBRAIN_REMOTE_PRIVATE_PAGES` relaxes them |
+
+ - `src/core/feedback/store.ts`: `insertEvalCandidate` is the one eval capture insert both engines run. A facts-arm result (`facts/<id>`) names a fact, not a page, so it stays while that fact exists.
+ - `src/core/context/volunteer-events.ts`: `insertVolunteerEvents`, behind the fire-and-forget volunteer-log sink, inserts only rows whose page exists, probed the same way.
+ - `src/core/engine-sql/pages.ts`: `deletePages` runs `scrubPurgedSlugs` and `scrubPurgedNameStores` after the delete, in one transaction. The transcript importer's stale-part delete now calls `deletePages`. `deletePage` is unchanged: the page purge, its main caller, scrubs and records the stores in its receipt.
+ - `src/core/source-delete.ts`: `removedSourceScrubCtes` runs inside `deleteSourceRow` and the managed lifecycle's source delete, then once more after it. `deleteSourceRow` skips that second pass when a source with the same id exists again, so a re-created source keeps its rows. `retrieval_weights` and `mention_gazetteer_entries` already cascaded from `sources`.
+ - `src/core/access/federation-exposure.ts`: the exposure read adds `privacyPartSql` unless `remotePrivacyRelaxed`.
+ - Tests: `test/purge-scrub-followups-9068.test.ts` and `test/federation-exposure-private-9068.test.ts` (PGLite, and Postgres through `test/postgres-unit-arms.txt`). `test/eval-candidates.test.ts`, the volunteer round-trip in `test/migrate.test.ts` and the fact-purge case in `test/purge-name-stores-9068.test.ts` now create the pages they log, since a capture or volunteer write keeps only live pages.
+
+ ## [0.60.658.0] - 2026-11-11
+
+ **Pull requests now run the tests that check gbrain's lists, in about 4 minutes, instead of finding those failures in the 30-minute full run.** Some tests fail whenever a change adds an operation, command, `--flag`, error code, doctor check or repair kind without updating the list that tracks it, or changes output a golden file pins. Before this release, pull requests skipped those tests and their first failure came in the full-ci run. A new check, `enumeration and census tests`, runs them on every pull request that changes more than docs. It is advisory: no merge rule reads it. `bash scripts/ci-census-tests.sh --run` runs the same tests on your own machine without Postgres.
+
+ ### Itemized changes
+
+ - `scripts/ci-census-tests.sh` (new) prints the one file list: tests marked `test-reads-source-ok[structural]`, every test that imports `test/helpers/golden`, the tests whose names match its enumeration pattern, the consumer idle-pass bounds (`test/persistence-consumer-*` files that mention idle), and the tests that read every file under `src/` or `test/` (its `WALKERS` list, such as the source-read smell test). It leaves out `test/e2e/`, serial and slow files, and `test/setup.test.ts`. On master that is 141 files. `--run` runs them all and `--run <k> <n>` runs one weight-balanced shard (`scripts/sharding.ts`).
+ - `.github/workflows/test.yml`: the `enumeration-census-tests` job runs the list in two shards on 2-vCPU runners, only on pull requests. Trunk Dynamic CI gates it like the other cheap-tier lanes. The `changes` job adds a `census` output that skips a doc-only diff (`scripts/select-e2e.ts --classify-only`), and an unreadable diff still runs the job. `cheap-tier`, `test-status`, `full-suite`, the remote pregate and master-push reuse don't read the job, and `contributor-gate` stays the only required check. On this change's own run the shards took 3 min 40 s and 2 min 47 s, about 6.5 runner-minutes per push.
+ - `docs/ci-event-parity.md`: new section "Enumeration and census tests" and a classification row.
+ - Tests: `test/scripts/ci-census-tests.test.ts` (new) pins the list's shape and coverage. `full-ci-workflow` and `native-only-workflow` cover the new job's events and its doc-only skip. The PGLite snapshot cache-key census counts the job's key (16 keys).
+
## [0.60.657.0] - 2026-11-11
**A running `gbrain serve`, worker or autopilot no longer fails writes after another process changes a table's columns (GST-2260).** When `gbrain apply-migrations` or an upgrade in another process added, renamed or dropped a column, every pooled connection that had already prepared a `SELECT *` or `RETURNING *` on that table failed its next run with "cached plan must not change result type". Outside a transaction the driver quietly prepared the statement again, but inside one the error had already aborted the transaction, so the caller saw it. A `put_page` came back `internal_error`, or was accepted and then failed at publication, once for each statement on each pooled connection. Version 0.60.646.0 made the write paths re-run the transaction it aborted. Now the first such error also makes every connection in the process prepare its statements again before its next query, and an OAuth token grant whose client read meets it and a minion job's completion or failure record are re-run too. Pools that do not prepare statements (transaction-mode pooler URLs, `prepare: false`) and PGLite are unchanged.
diff --cc docs/architecture/key-files/files-and-sync-1.md
index db2dabb766,ae1abb2179..2aebe0b84b
--- a/docs/architecture/key-files/files-and-sync-1.md
+++ b/docs/architecture/key-files/files-and-sync-1.md
@@@ -51,11 -51,9 +51,9 @@@ Current behavior and load-bearing invar
- `src/core/source-resolver.ts` — 6-tier source resolution. `resolveSourceWithTier(engine, explicit, cwd)` returns `{ source_id, tier: SourceTier, detail? }` alongside `resolveSourceId()` (unchanged). `SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'sole_non_default', 'brain_default', 'seed_default']` (7 entries; order matches priority). Tier `sole_non_default` slots between `local_path` and `brain_default`: when NO `sources.default` config is set AND exactly one registered source has `local_path` AND isn't `'default'` AND the `'default'` source holds no active pages (the emptiness guard — the tier's charter is rescuing brains whose `'default'` is empty; when `'default'` carries an established corpus, auto-routing would hijack every bare write into the side-source, so the resolver falls through to `seed_default` and prints a one-line stderr notice naming both sides so the suppressed flip is diagnosable, same `GBRAIN_NO_SOLE_NON_DEFAULT_NUDGE=1` suppression knob as the routing nudge; a failed pages probe on an exotic/legacy schema keeps the auto-route), auto-route to it; archived sources excluded (try/catch for pre-v34 brains); private `pickSoleNonDefaultSource(engine)` shared by both resolver entry points so they cannot drift. Exported `formatSoleNonDefaultNudge(sourceId): string | null` builds the user-facing stderr nudge (null when `GBRAIN_NO_SOLE_NON_DEFAULT_NUDGE=1`). `resolveCliSyncSource` in `src/commands/sync/run.ts` calls `resolveSourceWithTier` unconditionally so the tier fires; `src/commands/import.ts:96-128` mirrors with the tier-gated nudge. Consumed by `gbrain sources current`, `import --source-id`, `extract --source-id`, and the `source_routing_health` doctor check. Pinned by `test/source-resolver-with-tier.test.ts` (`withEnv()` per test-isolation lint), `test/source-resolver-sole-non-default.test.ts` (20 cases incl. both-ways emptiness-guard + probe-failure fallback), `test/sync-sole-non-default-routing.test.ts` (3 PGLite cases driving real `runSync`), `test/pages-source-scoping-4329.test.ts` (real-engine both-ways guard coverage). Also exports `noGrantFederatedScope(engine, hasSourceGrant, sourceId)`, the widening decision transports share: returns `localFederatedSourceIds(..., 'seed_default')` ONLY when `hasSourceGrant === false` (the legacy-bearer no-grant floor), `undefined` for a granted token, for an OAuth client (flag `undefined` — a falsy gate here would widen every OAuth client), and when the resolver throws (best-effort; the scalar scope stands rather than failing the request). Called by `src/commands/serve-http.ts`; pinned by `test/no-grant-federated-scope.test.ts` (6 cases). `explicitReadBinding(engine, sourceId, tier)` (#5081) is the explicit-tier twin used by the stdio transport for `GBRAIN_SOURCE` (`env`) and `.gbrain-source` (`dotfile`) bindings: `{ sourceId, via, sourceIds, optedOut }`, where `sourceIds` is the same set `localFederatedSourceIds` computes (shared `federatedSetFor`, so an isolated anchor admits only itself) and `optedOut` lists `federated === false` sources for the denial hint; `undefined` for every other tier. Pinned by `test/explicit-read-admission-5081.test.ts`. Unscoped default-write guard family (the `seed_default` tier is the only one that actually lands a write in `'default'`): `assessDefaultWriteGuard(engine)` → `DefaultWriteAssessment { shouldGuard, defaultPages, nonDefaultPages, nonDefaultSources, failed? }` — an unindexed full-`pages` aggregate; `shouldGuard` when ≥1 non-default source holds pages AND non-default pages outnumber default's; a query failure returns the fail-open no-guard verdict stamped `failed: true`, which cachers/latchers must not treat as settled. `assessDefaultWriteGuardOnce(engine)` is the per-engine WeakMap memo (arms on the first SUCCESSFUL assessment whatever its verdict, forgets a failed attempt so the next caller retries, coalesces concurrent callers; `__resetDefaultWriteGuardMemo` is the test seam). `assessUnscopedDefaultWrite(engine, tier, mutating)` → `{ warning, assessed }` (`assessed: false` only when the aggregate failed); `formatDefaultWriteWarning(a, sourceFlag?)` / `formatDefaultWriteRefusal(cmd, a)` render the two surfaces; `defaultWriteAllowedByEnv()` is the `GBRAIN_ALLOW_DEFAULT_WRITE=1` escape hatch. Consumers: `sync` refuses (dry-run warns), `import` warns via the memo, MCP stdio's `createDefaultWriteAdvisory` prints once. Also exports `isResolverUserError(e)` — the ONE predicate for this module's user-facing throws (`Source "…" not found.` / `not found or is archived.`, `Invalid --source value`, `Invalid GBRAIN_SOURCE value`), consumed by dream.ts, agent.ts and code-scope.ts so a CLI command turns an unknown/archived source into a clean stderr line + exit 1 while anything else keeps propagating. A request snapshot serves tiers 4, 5.5 (its rows carry the 'default' emptiness probe, `default_has_pages`), `assertSourceExists` and the federated set from one sources read, sent with the config table (`resolveSourceWithTier` reads tier 5's key beside the tier-4 listing) (GBRA-156).
- - `src/core/source-delete.ts` — `deleteSourceRow`: every unmanaged source delete (`sources remove`/`purge`, `sources_remove`, expired-archive purge) deletes the `sources` row, that incarnation's `persistence_source_bindings` row and its `managed-sync` cursors in `op_checkpoints` (#8741) in one statement (neither has an FK to `sources`). Other incarnations' bindings go to `gbrain repair orphan-bindings` (`src/core/persistence/orphan-bindings.ts`, doctor `orphan_persistence_bindings`). The unmanaged claim check (`sync-authority.ts`) and doctor `unbound_source` count only the current incarnation's binding. `EXPIRED_ARCHIVE_SQL` is the one purge-eligibility predicate (unmanaged sweep, its per-row DELETE and the managed `expiredOnly` purge): both archive timestamps must be later than the epoch and the expiry past; `UNVERIFIED_ARCHIVE_SQL` lists archived rows with a missing or epoch timestamp, which the sweep reports blocked with restore-then-archive guidance, never purged.
-- `src/core/source-delete.ts` — `deleteSourceRow`: every unmanaged source delete (`sources remove`, `sources purge`, the `sources_remove` op, expired-archive purge) deletes the `sources` row and that incarnation's `persistence_source_bindings` row in one statement; bindings have no FK to `sources`. #9068: that statement also runs `removedSourceScrubCtes` (the managed `remove`/`purge` in `source-lifecycle.ts` runs them in its final transaction too), which delete the source's `retrieval_event_pages`, `retrieval_event_links`, `retrieval_feedback` and `context_volunteer_events` rows and strip from `eval_candidates.retrieved_slugs` each slug no other source the candidate covers has a page for (`retrieval_weights` and `mention_gazetteer_entries` cascade from `sources`); a second statement runs them again, for a feedback write or capture that held a page row while the delete waited, unless a source with the same id exists again (so a re-created source keeps its rows). Pinned by `test/purge-scrub-followups-9068.test.ts`. Bindings of other incarnations are left to `gbrain repair orphan-bindings` (`src/core/persistence/orphan-bindings.ts`, doctor `orphan_persistence_bindings`). The unmanaged claim check (`sync-authority.ts`) and doctor `unbound_source` count only a binding of the source's current incarnation. `EXPIRED_ARCHIVE_SQL` is the one purge-eligibility predicate (unmanaged sweep, its per-row DELETE and the managed `expiredOnly` purge): both archive timestamps must be later than the epoch and the expiry past; `UNVERIFIED_ARCHIVE_SQL` lists archived rows with a missing or epoch timestamp, which the sweep reports blocked with restore-then-archive guidance instead of purging.
++- `src/core/source-delete.ts` — `deleteSourceRow`: every unmanaged source delete (`sources remove`/`purge`, `sources_remove`, expired-archive purge) deletes the `sources` row, that incarnation's `persistence_source_bindings` row and its `managed-sync` cursors in `op_checkpoints` (#8741) in one statement (neither has an FK to `sources`). #9068: that statement also runs `removedSourceScrubCtes` (the managed `remove`/`purge` in `source-lifecycle.ts` runs them in its final transaction too), which delete the source's `retrieval_event_pages`, `retrieval_event_links`, `retrieval_feedback` and `context_volunteer_events` rows and strip from `eval_candidates.retrieved_slugs` each slug no other source the candidate covers has a page for (`retrieval_weights` and `mention_gazetteer_entries` cascade from `sources`); a second statement runs them again, for a feedback write or capture that held a page row while the delete waited, unless a source with the same id exists again (so a re-created source keeps its rows). Pinned by `test/purge-scrub-followups-9068.test.ts`. Other incarnations' bindings go to `gbrain repair orphan-bindings` (`src/core/persistence/orphan-bindings.ts`, doctor `orphan_persistence_bindings`). The unmanaged claim check (`sync-authority.ts`) and doctor `unbound_source` count only the current incarnation's binding. `EXPIRED_ARCHIVE_SQL` is the one purge-eligibility predicate (unmanaged sweep, its per-row DELETE and the managed `expiredOnly` purge): both archive timestamps must be later than the epoch and the expiry past; `UNVERIFIED_ARCHIVE_SQL` lists archived rows with a missing or epoch timestamp, which the sweep reports blocked with restore-then-archive guidance, never purged.
- `src/core/sources-ops.ts` — Multi-source registration + clone-lifecycle ops (`addSource`, `recloneIfMissing`, `defaultCloneDir`, `isOwnedClone`, `unownedHint`). **Reclone-ownership invariant (must-never-violate): gbrain may only delete/re-clone a clone it created, NEVER a user working tree.** `recloneIfMissing` deletes `local_path`, so it gates on `isOwnedClone(src)` and throws a `SourceOpError('unmanaged_path', ...)` BEFORE any filesystem op when ownership is unprovable — fail-closed. Ownership is proven by `config.managed_clone === true` (written by `addSource`'s `--url` path, covering default-location and `--clone-dir` clones) OR `local_path === defaultCloneDir(id)` (back-compat for pre-marker clones, via exact normalized-path equality, symlink-free). A row with `remote_url` + an unowned `local_path` (a user-registered working tree, e.g. `sources add --path`) is refused untouched; re-add with `--url` to regain auto-reclone. The reclone is EXDEV-safe: clone into a SIBLING temp of `local_path` (not the shared `clones/.tmp`, which may sit on a different mount than a `--clone-dir` target), then swap (move old aside → move new in → drop old) so `local_path` is never left missing-and-unrecoverable; on swap failure the original is restored, and if restore fails the error names the `aside` path so it's never reflexively deleted. A TOCTOU re-check re-confirms ownership immediately before the destructive move and rejects a symlink leaf swapped in after the entry check (`symlink_escape`). `unownedHint(src, state)` is the shared recovery message used by both the core error and the `gbrain sync --source` CLI error; `gbrain sources restore` special-cases `unmanaged_path` to print "DB row restored; gbrain syncs this path read-only" instead of the misleading "try sync to recover" guidance. `SourceOpErrorCode` includes `unmanaged_path`. Pinned by `test/sources-ops.test.ts`, `test/sources-resync-recovery.test.ts`. `assertNoOverlappingPath(engine, id, path)` is the ONE overlapping-path guard shared by every surface that binds a `local_path` (`addSource`, `gbrain sources set-path`): a path equal to, nested inside, or enclosing another source's `local_path` throws `SourceOpError('overlapping_path')` — overlapping trees make sync/write-through attribute files to the wrong source, so no add or repair path may skip it. The guard compares by spelling AND by realpath (both the candidate and each sibling `local_path` are `realpathSync`'d when they exist, fail-open to the spelling otherwise), so a symlink resolving into or over another source's tree cannot slip past it; the stored pointer and the error message keep the operator's spelling. #6946: `isOwnedClone`/`defaultCloneDir` live in `src/core/url-clone.ts`, re-exported here.
- - `src/core/sync-concurrency.ts` — single source of truth for the parallel-sync policy. Exports `autoConcurrency(engine, fileCount, override?)` (PGLite always serial; explicit override clamped to >=1; auto path returns `DEFAULT_PARALLEL_WORKERS=4` when `fileCount > AUTO_CONCURRENCY_FILE_THRESHOLD=100`), `shouldRunParallel(workers, fileCount, explicit)` (explicit `--workers` bypasses the >50-file floor), and `parseWorkers(s)` (rejects `'0'`, `'-3'`, `'foo'`, `'1.5'`, trailing chars). Used by `performSync`, `performFullSync`, `runImport`, and the Minion `sync` handler so the sites can't drift. `DEFAULT_PARALLEL_SOURCES = 4` is a SEPARATE constant for the per-source fan-out under `gbrain sync --all` — kept distinct from `DEFAULT_PARALLEL_WORKERS` because total live Postgres connections per wave ≈ `DEFAULT_PARALLEL_SOURCES × DEFAULT_PARALLEL_WORKERS × 2 (per-file pool)` = 32 at both defaults (each per-file worker opens its own `PostgresEngine` with `poolSize = min(2, resolvePoolSize(2))`); `sync.ts` warns when `parallel × workers × 2 > 16`. `resolveWorkersWithClamp(engine, override, commandName, fileCount)` wraps `autoConcurrency` with a per-command stderr clamp warning on PGLite (per-(command, requested) dedup via module-scoped warned-once set with `_resetWorkersClampWarningsForTest()` seam) and is the canonical surface for every bulk-command `--workers N` flag (extract-conversation-facts, extract, edges-backfill, reindex-multimodal, reindex, reindex-code); embed.ts deliberately bypasses it and keeps `GBRAIN_EMBED_CONCURRENCY || 20`. `resolveMaxConnections()` (reads `GBRAIN_MAX_CONNECTIONS`, undefined when unset) + `clampWorkersForConnectionBudget(workers, perWorkerPool, maxConnections, parentPool)` back the opt-in single-sync connection-footprint clamp so a big sync stays under a low pooler cap (`parent_pool + workers×perWorkerPool ≤ budget`); `gbrain doctor`'s `pool_budget` check (`computePoolBudgetCheck` / `checkPoolBudget` in `src/commands/doctor/checks/consolidation-cycle.ts`, re-exported by `src/commands/doctor.ts`) warns when the budget leaves no room for a worker, pointing at `GBRAIN_POOL_SIZE=2`. Pinned by `test/pglite-workers-clamp.test.ts`.
-
- `src/core/sync-cost-gate.ts` + `sync-git.ts` + `sync-anchor.ts` + `sync-lock.ts` + `sync-reconcile.ts` + `sync-status-report.ts` — six pure-function clusters behind the `src/commands/sync.ts` façade, which re-exports them; `facadeExpansion` in `scripts/generate-flag-registry.ts` keeps exactly these six (NOT the other `sync-*` siblings, which are ordinary deps) on the sync command's flag-scan surface. `sync-cost-gate.ts`: the inline-embed cost gate + token estimation for `gbrain sync`. `sync-git.ts`: git plumbing — invocation building, repo discovery, baseline-commit self-heal, path-containment guards, OS-relative paths normalized to Git slash form by `gitRelativePath`, and the delete/rename slug resolvers (`resolveSlugByPathOrSourcePath` — see its dedicated entry above — plus the verified `resolveSlugsForRemovedPaths`/`resolveRemovedPathSlug`). `sync-anchor.ts`: sync anchor + chunker-version state helpers (source-scoped vs legacy global-config storage). `sync-lock.ts`: the lock layer — typed lock-busy error, the rich busy message, `--break-lock` handling, the partial-result envelope (`performSync` itself stays in the façade). `sync-reconcile.ts`: full-sync reconcile planning (the mass-delete valve and ever-committed gate), with stored paths normalized before scope filtering, plus sync deadline/stall resolution (the default and env deadlines are progress-aware; `syncWatchdogPlan` and `syncDeadlineStopNotice` build the watchdog options and its stdout stop notice — see the `process-watchdog.ts` entry). `sync-status-report.ts`: the per-source sync status report backing `gbrain sources status` and the `get_status_snapshot` op. Its queue context is engine-aware: only an explicitly worker-backed surface may promise job deferral; otherwise non-interactive cost deferral returns `manual_drain_required` with exact per-source `gbrain embed --stale --source <id>` commands and never writes or claims an embed-backfill row. GBRA-133 2b.3: a `--no-pull` estimate (`estimateInlineNewTokens` / `CostGateContext.noPull`) prices local HEAD and never fetches origin. `sync-git.ts` adds `gitAsync` / `discoverGitRootAsync` / `isDetachedHeadAsync`, which classic sync preflight, finalize, the lossless delta, upstream observation and the source filesystem lock now use, so a resident sync spawns no synchronous Git.
- `src/core/sync-delta.ts` — the single "what changed since last_commit" helper, consumed by BOTH `performSyncInner` (sync executor) and `estimateInlineNewTokens` (cost estimator) so the gate's dollar figure can't drift from what the sync imports. `computeSyncDelta(repoPath, fromCommit, toCommit, {detachedManifest?, detached?})` → `{status:'ok', manifest}` | `{status:'unavailable', reason:'anchor_missing'|'diff_failed'}`. Anchor reachability via `git cat-file -t` (a gc'd bookmark is `anchor_missing`, but a present-but-non-ancestor bookmark is still diffed tree-to-tree), then `git diff --name-status -M from..to` parsed by `buildSyncManifest`; merges the detached working-tree manifest when detached (`buildDetachedWorkingTreeManifest`). NO dirty/untracked probe in the estimator — pricing dirty files would create phantom costs on a busy brain. The executor imports only the commit diff on attached HEADs BY DEFAULT, but counts uncommitted working-tree files (untracked + dirty) as `SyncResult.uncommitted` drift and, with `--working-tree` / config `sync.include_working_tree`, merges the working-tree manifest into the delta so attached repos can import uncommitted state (the estimator does not price that opt-in). `execFileSync` array-args (shell-injection safe), 30s / 100 MiB budget, `GIT_OPTIONAL_LOCKS=0` (`git-env.ts` `GIT_READ_ENV`: sync's Git reads never take `index.lock`). GST-2331: runs with `HARDENED_GIT_ARGS` and `NO_LAZY_FETCH_ENV`. Test seam `_setGitRunnerForTests`. Pinned by `test/sync-delta.test.ts`. Classic sync reads the same delta as bytes instead: `computeLosslessSyncDelta` / `buildLosslessWorkingTreeManifest` run `git diff --raw -z -M` (and `ls-files -z --others`) into a Buffer and `parseRawDiffZ` keeps each name exactly as Git has it (no trim, no unquoting); a name that is not UTF-8 or holds a control character goes to `unsupported` (present) or `gone` (left the tree) instead of the manifest, and a gitlink (mode 160000) that appears goes to `gitlinks`, never the manifest (seam `_setGitBytesRunnerForTests`). Managed discovery and the cost estimator keep `computeSyncDelta`. Pinned by `test/pathological/s-classic-byte-paths.test.ts`.
diff --cc src/core/persistence/source-lifecycle.ts
index 19cd29a354,7132e5816b..17172d2341
--- a/src/core/persistence/source-lifecycle.ts
+++ b/src/core/persistence/source-lifecycle.ts
@@@ -342,10 -342,10 +342,11 @@@ export async function runManagedSourceL
{fix:readFix('Lists OAuth clients with the source each is bound to.',{argv:['gbrain','auth','clients','--json']})});
const [impact]=await tx.executeRaw<{count:string}>('SELECT count(*)::text AS count FROM pages WHERE source_id=$1',[input.sourceId]);
pagesDeleted+=Number(impact.count);
- await tx.executeRaw('DELETE FROM sources WHERE id=$1 AND incarnation=$2::uuid',[input.sourceId,incarnation]);
+ await tx.executeRaw(`WITH gone AS (DELETE FROM sources WHERE id=$1 AND incarnation=$2::uuid RETURNING id),${removedSourceScrubCtes('EXISTS (SELECT 1 FROM gone)')} SELECT 1`,[input.sourceId,incarnation]);
+ await tx.executeRaw(`WITH ${removedSourceScrubCtes('true')} SELECT 1`,[input.sourceId]);
await tx.executeRaw('DELETE FROM persistence_source_bindings WHERE source_id=$1 AND source_incarnation=$2::uuid',[input.sourceId,incarnation]);
await supersedeTopologyRetires(tx,input.sourceId,incarnation,principal,requestId);
+ await tx.executeRaw("DELETE FROM op_checkpoints WHERE op='managed-sync' AND completed_keys->0->>'sourceId'=$1",[input.sourceId]);
}
if(source&&!['remove','purge'].includes(input.operation)) await supersedeTopologyRetires(tx,input.sourceId,source.incarnation,principal,requestId);
const contentReceipts=input.operation==='rebind'?await rebindContentReceipts(tx,worktrees,hostIdFor(tx),relocated?undefined:[input.sourceId]):[];
diff --cc src/core/source-delete.ts
index b4b6806964,e7d18647ba..b4de05de19
--- a/src/core/source-delete.ts
+++ b/src/core/source-delete.ts
@@@ -17,9 -17,11 +17,13 @@@ export const unverifiedArchiveReason =
* binding (#5732). `persistence_source_bindings` has no FK to `sources`, so a
* delete that leaves the binding would make a same-id replacement read as
* claimed. A binding of another incarnation is never touched here; `gbrain
- * repair orphan-bindings` removes those. The same statement runs
+ * repair orphan-bindings` removes those. The source's managed-sync cursors
+ * (`op_checkpoints`, also no FK) go too (#8741): nothing can resume them, and doctor `managed_sync_backlog` would warn with a resume command
- * that fails `unknown_source`. Returns whether a row was deleted.
++ * that fails `unknown_source`. The same statement runs
+ * removedSourceScrubCtes, and a second statement repeats it once the row is
+ * gone, unless a source with this id exists again by then: a replacement
+ * created between the two statements keeps its rows. Returns whether a row
+ * was deleted.
*/
export async function deleteSourceRow(
engine: Pick<BrainEngine, 'executeRaw'>,
@@@ -33,8 -35,7 +37,9 @@@
`WITH gone AS (DELETE FROM sources WHERE id = $1 ${expired} RETURNING id, incarnation),
unbound AS (DELETE FROM persistence_source_bindings b USING gone
WHERE b.source_id = gone.id AND b.source_incarnation = gone.incarnation),
+ cursors AS (DELETE FROM op_checkpoints c USING gone
- WHERE c.op = 'managed-sync' AND c.completed_keys->0->>'sourceId' = gone.id)
++ WHERE c.op = 'managed-sync' AND c.completed_keys->0->>'sourceId' = gone.id),
+ ${removedSourceScrubCtes('EXISTS (SELECT 1 FROM gone)')}
SELECT id FROM gone`,
[id],
);