Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Merge remote-tracking branch 'origin/master' into capy/gbra286-memory-train

garrytanedf2a5ed256d
diff --cc CHANGELOG.md
index 14dcb06ecb,c10b78f628..e729773ca5
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -10,113 -10,156 +10,263 @@@ credits are retained; no result has bee
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
- ## [0.60.571.0] - 2026-11-04
++## [0.60.576.0] - 2026-11-04
+
+**Memory train (GBRA-286): `forget` edits the page files first, `recall` can weigh how far a fact deserves belief, page versions carry a human reason and can be read by date, and agents can read and switch the saving rule. Every new mode is off by default, and every feature works on a brain with no Git repository.**
+
+This release folds seven approved memory PRs into one: #7616, #7791, #7797, #7803, #7810, #7798 + #7804 and #7795. Each was brought up to current master. Where a new path skipped a guard master added since (member-scoped access with `readable_by`, the trust-tier quote fence for remote readers, the private-request hold on content imports), the guard now applies there too, with a test that failed first.
+
+### Forget writes the page files first (#7616)
+
+Git stays the parent. A withdrawal is now a page write, published to Git before the database changes.
+
+| Situation | Before | Now |
+|---|---|---|
+| `forget` / `forget_fact` | the database expired the fact; the page files caught up later through a mirror | every page whose `## Facts` table carries the claim gets its row struck (`~~claim~~`, a `valid_until` date, `forgotten: <reason>`), all in one publication and one commit, then the ledger row and expiries in the same transaction |
+| `remember` without an entity | a database-only row | a row of the `memory/unattributed` page's `## Facts` table, created on first use (concurrent first saves all land on it), so Git, `recall` and search see it |
+| Remembering that claim again with an entity | two copies | the entity row is written and the unattributed row struck in the same publication |
+| A member forgets a fact on their own member page | withdrawn for that page | unchanged: the withdrawal stays on that page, and other members' copies stay active |
+
+A forget now waits for the source's writer the way `remember` does. With the writer unavailable it returns the pending-write receipt, and the fact is still recalled until it publishes. With `sync.write_through` off, from a sandboxed subagent, or on a source with no canonical owner, a forget of a claim a page file carries is refused (`sync_write_through_off`, `subagent_sandbox`, `unbound_source`) instead of withdrawing it in the database only; a claim only database-only pages carry is still withdrawn. Migration v287 adds the multi-page recovery record (one record restores every file of an interrupted publication); it rewrites no data.
+
+### Review lists and review decisions through the page door (#7791, #7797)
+
+- `extraction_pending` gains two lists. `unattributed` (on by default) lists active facts saved without an entity, each with the relink action. `possible_duplicates` (opt-in through `include`) flags new entity pages whose name is a typo or acronym of another's. It only flags: nothing is merged and no identity is written.
+- `extraction_review` promote and reject are now page-door writes (`edit_page` of the frontmatter, `delete_page`), so on a source with a repository each decision is a Git commit. A new `distinct` action records that two flagged pages are different entities.
+- Remote reads of the unattributed list hide facts the caller can't read, and quote fact text below the owner's tier as source material, as `recall` does.
+
+### Recall weighting, entity grouping and the keyless claim lane (#7803, #7810)
+
+**`recall` can weigh how much a fact deserves to be believed, and can group a no-entity question's facts by entity. Both are opt-in.**
+
+#### What changed
+
+| Situation | Before | With the switch on |
+|---|---|---|
+| `recall {question}`: a year-old fact you confirmed and a fresh agent-written or external fact match equally | the newest fact ranked first | the confirmed fact ranks first (`recall.confidence_weighting on`) |
+| The same, between two agent-written facts | newest first | newest first (age still decays) |
+| A dated historical fact ("founded in 2015") | ranked by match | ranked by match; it does not decay |
+| `recall {question}` with no `entity` answered by two clients' facts | one flat list, the clients' figures interleaved | each entity's facts together, `facts_groups`, and an `entity_unspecified` notice offering the recall per entity (`recall.group_by_entity on`) |
+| `gbrain config set recall.question_admission <typo>` | stored and silently ignored | refused; the new switches take only `on` or `off` |
+
+Weighting is `relevance × (0.5 + 0.5 × weight)`: facts you confirmed and your own notes keep full weight at any age, other tiers decay from `valid_from` and carry a tier weight (tool 0.9, agent 0.8, external 0.6). It only reorders facts that already matched; `relevance` stays the plain score and `rank_score` and `confidence_factor` show the weighting. Grouping runs after the limit and budget cuts, so no entity is cut off by it. Both read database rows only, so they work the same on brains with no git repository.
+
+On a keyless 60-question fixture (old confirmed truth against fresh agent and external text, an agent knowledge update, dated history), weighting moved the confirmed truth to first in 20 of 20 questions, from 0 of 20, and left the update and history cases at 20 of 20. Grouping removed entity interleaving in 10 of 10 two-client questions with the same number of entities delivered.
+
+**Typed figures you save can now be checked for conflicts without an API key, and you decide each conflict from the trust review.**
+
+`remember` takes an optional typed claim (`claim_metric`, `claim_value`, `claim_unit`, `claim_period`). Two saved claims about the same entity with the same metric, unit and period in the same as-of bucket (the day, or the week, month, quarter or year the period names) are a conflict. Figures in different buckets are history, so a monthly series, a run of funding rounds or a forecast next to an actual are never flagged. Finding conflicts is off by default: `gbrain config set dream.claim_conflicts.mode propose` turns on the nightly phase, and `gbrain decide sweep --claims` runs it once (`--dry-run` writes nothing). Each conflict becomes one pending proposal that nothing applies until you decide it.
+
+#### What changed
+
+| Situation | Before | Now |
+|---|---|---|
+| `remember` a figure | text only | optional typed claim, stored on the fact and in the entity page's `## Facts` table; a bad value is refused with the corrected call |
+| Two saved figures disagree for the same month | found only by the paid conflict sweep | found by `gbrain decide sweep --claims` or the `claim_conflicts` dream phase (`propose`), with no key and no model call |
+| A typed figure saved for April next to March's | could supersede March's if the text was close | both kept: a typed write never supersedes a typed fact in another bucket |
+| `gbrain trust review` | proposals, lowered pages, preferences, holds | also `contradiction` items (`dp<id>`) with supersede, forget, confirm and drop; `--accept-all` skips them |
+| `recall` of a fact in a pending decide proposal | no label | `contested` with the other fact's id, only when the caller can read that fact |
+
+A run files at most `dream.claim_conflicts.max_pairs` (default 500) proposals per source and leaves the rest for the next run. A new unique index keeps concurrent runs to one proposal per pair (migration v288); it rewrites no data. Everything reads and writes database rows, so it works the same on a brain with no git repository. Guide: the "Old and conflicting facts" section of `docs/guides/memory-trust.md`.
+
+### Page versions carry a human reason, and pages read by date (#7798, #7804)
+
+**A page write can say why it was made, and page history shows it.**
+
+`put_page`, `edit_page`, `revert_version` and each `put_pages` item take an optional `reason` (one line, at most 500 characters; `--reason` on the CLI). It is stored on the version the write replaced, and `get_versions` returns it as `replaced_because`, so the newest version explains the text you read now; `gbrain history` prints it as `changed because: …`. When the page's file reaches Git, the commit carries it as a trailer naming the file, `gbrain-reason: <path>: <text>`, beside any `gbrain-repair:` trailer. A brain with no Git repository gets the same history from the database. Every write result reports where the reason went in `reason_recorded`.
+
+#### What changed
+
+| Situation | Before | Now |
+|---|---|---|
+| A write passes `reason` | refused as an unknown parameter | stored with the replaced version and, for a world-visible file, in the commit |
+| A reason over 500 characters, or one that looks like a credential | n/a | refused before admission: `change_reason_rejected` (`reason_too_long`, `reason_contains_secret`), and no journal row keeps it |
+| A remote caller reads `get_versions` | no reasons | `replaced_because` when the version and the one that replaced it are both world-visible and the replacement was not pruned |
+| A private page's write has a reason | n/a | the reason stays out of every commit message |
+| A write without a reason | | the same commit message and history as before |
+| A newer write to the same file commits first (non-strict source) | n/a | the receipt says `git: superseded` with a `reason_not_committed` notice; the reason stays in history |
+
+Guide: [page history](docs/guides/page-history.md).
+
+#### Read a page as it stood on a date
+
+A page can now say when its text takes effect, with `valid_from` and `valid_until` in its frontmatter, and you can ask what was in force on any date: `get_page {slug, as_of: '2026-03-01'}`, or `gbrain get <slug> --as-of 2026-03-01`. Say to your agent: *"What did our refund policy say on March 1?"*
+
+- The answer is the edition with the latest `valid_from` on or before the date, picked across the current page and every version gbrain kept. Its `valid_time` block says whether that edition was `in_force`, `not_in_force`, `not_yet_in_force` or `before_recorded_history`, and whether gbrain held enough history to be sure. Only an `in_force` answer returns the text at the top level, so an agent that skips the status cannot quote expired text as current.
+- When history is missing (pruned versions, a sync that imported several commits at once, a page that came from a file) or the dates look ambiguous (a corrected date and a renewal leave the same trace), the answer says `uncertain: true`, why, and how to recover: the file's `git log`, or asking the user for a backup. It never guesses.
+- Works the same on a brain with no Git repository, the default keyless install.
+- Over MCP, an `as_of` read follows the plain read's rules. A quarantined page's body is omitted, and an edition's `introduced_because` is shown only when the edition it replaced is one the caller may read.
+- `gbrain get --as-of` exits 15 when no edition was in force, and 16 (`host_upgrade_required`) when the brain host is too old to answer by date, printing nothing historical.
+- Writes say when they keep or change an edition's date (`valid_time_correction`, `valid_time_date_only`, `valid_from_unparsed`), and `get_versions` rows carry each version's `valid_from` and `valid_until`.
+- Behavior change: a plain `get_page` of a page whose own `valid_from` is in the future, or whose `valid_until` has passed, now also carries a `valid_time` status. Pages without those dates read exactly as before.
+
+Guide: [page history](docs/guides/page-history.md#valid-time).
+
+### Memory writeback settings over MCP (#7795)
+
+**Agents and settings screens can now read the memory-saving rule, an admin can change it remotely, and a user can say "save everything from here" or "stop saving" for one conversation.**
+
+Before, the rule for what an agent saves lived in the MCP instructions, and only the brain operator could see or change `memory.auto_writeback`, from a shell on the host. Three MCP tools change that. Every one keeps its state in the database, so a brain with no git repository gets all of them.
+
+#### What changed
+
+| Tool | Who can call it | What it does |
+|---|---|---|
+| `get_memory_settings` | any connection | the mode agents follow, where it came from, the exact rule text, every mode's rule, background capture state, and why you can or can't change it |
+| `set_memory_writeback` | admin scope, after `gbrain config set mcp.publish_memory_settings true` (off by default) | changes `memory.agent_writeback` (inherit, off, salient, all) for the whole brain, with a dry run, an audit row in the same transaction, and a caution on team brains |
+| `memory_conversation_mode` | any connection that can call `remember` | returns the rule for the rest of this conversation: all, salient, off (only what the user asks to keep) or brain; stores nothing |
+
+The new `memory.agent_writeback` key changes only what agents are told. The host capture lanes (Stop hook, compaction harvest, SessionEnd transcripts) keep following the operator's `memory.auto_writeback`, so a remote "off" can't discard banked session text and a remote "all" can't start capture the operator never enabled. An unreadable or unrecognized override makes agents act as if saving is off and `get_memory_settings` reports `degraded`. The MCP instructions gain one line telling the agent to use the switch, after the governing rule, wherever the tool is served (starter and full surfaces). `gbrain config set|get memory.agent_writeback` works from the CLI.
+ ## [0.60.575.0] - 2026-11-04
+
+ **Train R, seven pull requests in one release. Searches that name a page by an identifier (`#412`, `INV-2026-00417`) or a handle (`dan-example`) now find it. New `gbrain find` / `find_pages` finds a page by name, path prefix or glob, and `gbrain stats --by source,type` counts pages by kind. `query --explain-target --json` carries its diagnosis. There's an opt-in keyword phrase factor (off by default), an advisory harness check on pull requests, new docs, and guardrails for the embedded core.**
+
+ ### Identifiers and handles in search (#7491, #7492, #7497)
+
+ **A question that names a page by an identifier ("What did the latest weekly digest (#412) say?") now returns the page holding `#412`, even when the question's other words keep it from matching every word and near-identical template pages tie with it on meaning. A handle such as `dan-example` now finds the page it names (`people/dan-example`) in keyword search, quoted or not. The coverage notice also stops blaming indexing for unrelated image pages.**
+
+ #### Identifiers in questions
+
+ | Situation | Before | Now |
+ |---|---|---|
+ | No chunk holds every word of the question, and the any-word retry ranks the page holding `#412` first | the retry's rows are dropped before fusion whenever semantic search returns anything, so on a brain of near-identical template pages an older copy won | the keyword arm runs one more statement matching every identifier the question names, with the same scope, visibility and filters as the strict search; its rows vote like strict rows, and a title holding the identifier leads the title arm |
+ | Long pages repeat the question's common words ("say", "report") | they outranked the short chunk holding the identifier once, so the page fell out of the keyword arm | the identifier match comes first, and the any-word retry ranks rare words first for questions of up to 12 words (it was 4) |
+ | One unrelated chunk happens to hold every word, the number included | that chunk was the only keyword row; the page holding the identifier never entered the arm | the identifier rows follow the strict rows, so the page is a keyword candidate again |
+
+ An identifier is a `#` number, a run of 6 or more digits, a code such as INV-2026-00417, an email, a URL or a phone number, asked together with other words. Queries without one, a quoted phrase or a `-` negation run exactly as before, and the any-word rows that don't hold an identifier are still dropped while semantic search is healthy.
+
+ #### Handles in keyword search
+
+ | Situation | Before | Now |
+ |---|---|---|
+ | `gbrain search '"dan-example" OR "erin-example"' --match keyword`, where the person pages hold the handles only in their slugs | the pages that mention the handles came back, never the pages they name: a hyphenated handle only matches text holding the literal handle | the pages whose slug ends in each handle (`people/dan-example`, `people/erin-example`) are listed and counted first |
+ | The same query with `--types '["person"]'` | no results | the two person pages |
+ | The same query in hybrid search | quoted terms skip the any-word retry, so the title arm found nothing either | the named pages lead the keyword arm, as rows that count like strict keyword rows |
+ | A search that matches nothing while every retrieval stage ran | the CLI said "clean miss — no retrieval degradation" and MCP "this is a clean miss", which agents read as "the brain has no page for this" | "retrieval healthy", followed by the next things to try (the display name, a synonym, hybrid search) before concluding the brain has nothing; the query skill says the same |
+
+ A handle is a hyphenated token with a letter in it, bare or quoted on its own (`acme-example-2` counts, `2026-10-24` does not), and resolves only to a page whose slug ends in exactly that handle. A query with a `-` negation or a longer quoted phrase resolves no handle, and `--match keyword` resolves handles only when every term is one (or `OR`), so a handle never brings back a page the query excludes, a phrase it doesn't hold or a word it doesn't contain. If the identifier or handle lookup fails, the result is stamped degraded (`keyword_candidates_incomplete`) rather than reading as a miss. The named pages pass the same filters as any keyword match: `--types`, the caller's sources, and for remote callers no private or quarantined page.
+
+ #### Coverage notice
+
+ | Situation | Before | Now |
+ |---|---|---|
+ | A pending image page's slug contains the query's digits (`412` in `frames/t004120`) | counted as able to match the query | a query word counts only as a whole word of the title or slug |
+ | Pending image pages and a text-only search | counted, although text search never returns image pages | counted only when an image search arm ran |
+ | The `projection_pending` part of `degraded_recall` | "recently written pages are not in the search index yet" for every page | text pages "are being indexed"; image pages "will not be indexed on their own" and wait for `gbrain projections drain` or a paid re-import, with doctor's `text_projection_readiness` check as the fix |
+
+ **Say to your agent:** *"What did the latest weekly digest (#412) say?"*
+
+ #### What to watch for after you upgrade
+
+ - A search naming an identifier costs one more keyword statement, served by the full-text index. Other searches run the same statements as before, except that an any-word retry of 5 to 12 words now also runs the bounded rare-word read and, when a rare word is found, one more retry statement (both already ran for retries of up to 4 words). The rare-word read skips stopwords, and on Postgres it skips words the column statistics already estimate as common, so its cost follows the question's rare words, not its length.
+ - A query with a hyphenated token costs one more index lookup on page slugs. That is one more statement in hybrid search, and one more arm of the same statement with `--match keyword`. `keyword_total` and the `--match keyword` count include the named pages.
+ - Pages that hold the identifier but not the other words can now rank above pages that match the question's meaning better. The page holding every word still comes first among keyword rows.
+ - `_meta.retrieval.coverage.could_affect` can carry `pending_images`, and `could_affect.pending_projection` can drop for queries whose digits or short words used to match unrelated slugs.
+ - A script that matched the text `clean miss` in an empty result should match `retrieval healthy` instead; `_meta.retrieval` is unchanged.
+ - Nothing is rewritten on upgrade, no setting changed, and there is no schema migration.
+
+ ### Find a page by name, count pages by kind (#7191, #7233)
+
+ Agents fell back to the filesystem for two kinds of lookup that gbrain had no direct answer for. One was finding the page called X or under path Y, and the other was counting pages of each kind.
+
+ | Situation | Before | Now |
+ |---|---|---|
+ | Find a page by name or path | `ls`, `find` or `grep` on the checkout, or a ranked search | `gbrain find <name\|prefix\|glob> [--type T] [--source S] [--limit N]` (MCP `find_pages`). A bare name matches a title or the last slug segment, served by the trigram indexes. A prefix (contains `/`) or a glob (`*`, `?`) matches the slug or `source_path`. Rows sort by slug, with `total` and `complete`. The default limit is 50, capped at 500 locally and 100 remotely, and `limit_clamped_from` says when the cap applied. Under `--as <member>`, find and `stats --by` show only what that member reads |
+ | Count pages by kind or source | `gbrain stats` gave brain-wide totals only | `gbrain stats --by source,type [--since 30d] [--collapse-aliases]` (MCP `get_stats`, an admin-scope tool, gains `by`, `since` and `collapse_aliases`) reports page counts beside distinct-slug counts, so twins across sources show. `--collapse-aliases` folds type aliases through the active schema pack |
+ | Which call for which lookup | spread across the skills | one table in `skills/conventions/brain-first.md`, copied into AGENTS.md, INSTALL_FOR_AGENTS.md, skills/RESOLVER.md and the bootstrap template by `bun run regen:all` |
+
+ Remote callers get the shared read policy (private pages, trust floor, quarantine), their source grant, and the ranked-search hard excludes (`test/`, `.raw/`), so every remote `find` hit is also readable through remote `get_page`. On Postgres these reads run with JIT off. On a 305k-page brain at 40 ms RTT, a cold `find` costs about as much as `gbrain get` of one page (about 1.0 to 1.2 s). No migration.
+
+ ### `--explain-target` in CLI JSON, and an opt-in keyword phrase factor (#7192, #7175, #7198)
+
+ - `gbrain query "<q>" --explain-target <source>:<slug> --json` now prints `{ "results": [...], "explain_target": {...} }`, with the fix as a command (`next`, `argv`, `command`); without `--explain-target` the output stays a bare array. New causes: `target_not_embedded` (the page has chunks but no vectors), the failed arm named in `target_not_retrieved`, `target_collapsed` (folded into a kept row), and each arm's rank in `target_beyond_limit`. The retry argv keeps array and object arguments (`--types`, `--source-ids`, `--where` as JSON), and the `target_not_retrieved` retry adds `--expand`, a paid call that sends the query to the model, so it renders `next: ask_user`. MCP is unchanged: remote callers get the diagnosis in `_meta.explain_target` without `fix`, and private targets still answer like nonexistent ones.
+ - `gbrain config set search.keyword_phrase true` doubles the keyword score of a chunk that matches the query as one phrase (`phraseto_tsquery`), in the keyword arm on both engines and in keyword mode. It is off in every mode bundle, so default ranking and the default knob hash do not change; opting in appends `kph=1`.
+
+ ### Advisory harness check on pull requests (#7673)
+
+ **Pull requests now get an advisory `harness-check`: a keyless run of harness-evals' gbrain check against the pull request's commit that fails only when a member is shown a page outside their grant.**
+
+ - New workflow `.github/workflows/harness-check.yml` (not a required check). It builds harness-evals' gbrain sidecar image for the pull request's head commit, runs `scripts/gbrain-check.ts` without a key, and gates on the cross-scope page count being zero. Every other failed assertion (tool counts, retrieval, install bundle, installer files) is a warning in the job log and summary.
+ - The job reads one secret, `HARNESS_EVALS_READ_TOKEN` (read-only, harness-evals Contents). Without it the job skips with a notice and succeeds; a pull request from a fork is refused. No model key and no paid trial.
+ - Docs: `docs/ci-event-parity.md#harness-check` (what it runs, the gate, the skip, a sample run and how to bump the harness-evals pin).
+
+ ### Docs (#7609, #7787)
+
+ - New guides: [git-canonical sources](docs/guides/git-canonical.md), [anatomy of a page](docs/guides/page-anatomy.md), [where your memory lives](docs/guides/where-your-memory-lives.md) and [entity-scoped facts](docs/guides/entity-scoped-facts.md).
+ - `docs/architecture/brains-and-sources.md` explains git as the parent of a source with a checkout. `docs/guides/memory-boundaries.md` covers adding to or uploading a large page. `docs/mcp/TOOL_REFERENCE.md` gains `source_ids` and pointers to `search_exact` and `append_page`. README and llms.txt link the new pages.
+ - `scripts/check-key-files-current-state.sh` no longer caps README.md; the KEY_FILES.md cap stays.
+
+ ### Embedded-core guardrails (#8098)
+
+ The embedded core gets guardrails. They are a ratchet on what `src/core` reads from the process and the machine, a two-brains-in-one-process suite, a hermeticity test, and an upgrade-from-the-previous-release harness. Nothing changes for local users.
+
+ The planned embedded core lets one server process run many brains, each opened from its own settings. This release adds the guards that wave and every later one is measured against, with no runtime change. `bun run check:core-world-reads` (in `bun run verify`) counts, per file in `src/core`, `loadConfig` calls, environment reads, `fs` imports, module-level collections, spawn imports, process-wide hooks and `AsyncLocalStorage` stores, and fails when a count rises or a baseline row goes stale. The two-engine suite opens two brains in one process on PGLite and Postgres and shows what they still share today (AI keys and model clients, the query-embedding cache, the usage and telemetry sinks, link semantics, the admission and retrieval-tracking caches, page-lock names and home paths, and on Postgres the connection pool), each case naming the wave that fixes it. The upgrade harness opens a brain and keyless config written by the previous release, as a repository brain, a content-directory brain and a no-root brain, and requires the same page, schema, resolved settings and untouched config file.
+
+ #### Itemized changes
+
+ - New guard `scripts/check-core-world-reads.ts` with baseline `scripts/core-world-reads-baseline.tsv` (rows only fall; `--prune` lowers stale rows).
+ - New two-engine isolation suite (`test/two-engine-isolation.serial.test.ts`, `test/e2e/two-engine-isolation.test.ts`), hermeticity test (`test/embedded-hermeticity.test.ts`, expected to fail until W7) and upgrade harness (`test/upgrade-from-previous-release.slow.test.ts`, `test/e2e/upgrade-from-previous-release.test.ts`).
+ - New example `examples/embedded-two-brains.ts` (two PGLite brains in one process).
+ - An import inside its own engine scope no longer re-enters it, so a process that opened an embedded engine earlier no longer overflows the stack importing through an engine without a stable key.
+ - Docs: [running gbrain on servers today](docs/guides/servers-today.md) and the planned [embedded core](docs/architecture/topologies.md#embedded-core-planned), including the opt-in `embedded_core` setting later waves add (nothing reads it yet) and which wave moves each piece of per-process state.
+ - The 0.60.524.0 entry now names the command that exits 10 correctly: `gbrain trust review --accept-all`.
+
+ ### What to watch for after you upgrade
+
+ - Nothing is rewritten on upgrade, no default changed, and there is no schema migration.
+ - A script that matched `clean miss` in an empty search result should match `retrieval healthy`.
+ - A script that parsed `query --explain-target --json` as a bare array should read `.results`.
+
+ ## [0.60.574.0] - 2026-11-04
+
+ **A brain opened from settings no longer leaks into the process, or into another brain, when a call forgets its scope.** While an embedded engine is open, an embed, chat, expansion or rerank call that entered no scope now refuses `engine_scope_required` and sends nothing, even if the host configured a process gateway; it decides by scope presence alone. `withLocalScope(fn)` marks a call that belongs to the process gateway, and the CLI, `gbrain serve` and `gbrain jobs` enter it, so they behave exactly as before. `hybridSearch`, the import paths, minion jobs and `POST /mcp` enter their engine's scope themselves. Each embedded engine keeps its own chat-usage sink (it survives `reconnect()`), batch-shrink, no-key and structured-output memory, and alias and remote-privacy policy, and a closed per-call engine is no longer held by the policy sets. A host that reopens a brain per call can pass `settings.cache_namespace` to keep its query embeddings warm. `openEngine` now refuses a settings key it does not know. Everything here is behind `embedded_core`, which is off by default.
+
+ ### Itemized changes
+
+ - `src/core/engine-scope-store.ts` (new) and `src/core/engine-scope.ts`: `withLocalScope` (exported from `gbrain/engine-open`) and `needsEngineScope`. `gateway.ts` and `anthropic-key.ts` refuse an unscoped call with the new registered code `engine_scope_required` (its fix names `withEngineScope`); `engine_scope_missing` keeps meaning a provider the engine lacks. The hatch that let `configureGateway` open unscoped calls is gone, and `configureGatewayIfUninitialized` stays a no-op for them.
+ - `src/cli.ts`, `src/commands/serve.ts`, `serve-http.ts` and `jobs.ts` enter the local scope at their entry points; `serve-http-mcp.ts` runs each `POST /mcp` and `minions/worker.ts` each job in its engine's scope; `hybridSearch`, `hybridSearchCached`, `importFromContent`, `importFromFile`, `importCodeFile` and `importImageFile` re-enter their engine's scope when called outside it, so a local engine's search inside an embedded engine's scope uses the process gateway. `withEngineScope` enters its scope even before the first embedded engine opens, so a call scoped then keeps it.
+ - `src/core/engine-construct.ts` (new): `openEngine` builds its engine without the `createEngine` usage-stack entry and gives it a sink of its own (`registerEngineChatUsageSink`), so its usage is never the process fallback and a file-backed PGLite `reconnect()` no longer drops it (#8499).
+ - `src/core/ai/engine-gateway.ts`: the shrink factor, the once-per-model `no_key` rerank audit row and the structured-output rejection memory live on each gateway state, and an embedded engine's cached query embeddings are dropped at disconnect (#8499 item 2). `settings.cache_namespace` tags them with the namespace and a hash of the provider ids, base URLs and key fingerprints instead, never the key.
+ - `src/core/search/private-visibility.ts`: `visibility.honor_aliases` and `search.remote_private_pages` read on an embedded engine are kept per engine (keyed by `engineKey`) and decide inside its scope; a read in no scope while an embedded engine is open takes the strict policy; the CLI's process-wide lean-private rule is unchanged. An unscoped rerank refusal skips as `no_key` and writes no failure row.
+ - `openEngine` refuses an unknown top-level or nested settings key with `engine_settings_invalid` naming it, never its value; `storage` is accepted for Wave C. A BEHAVIOR_CHANGES row and docs/ENGINES.md "Upgrading an embedded host" describe the changes for embedded hosts.
+ - `examples/embedded-two-brains.ts` (new): two brains with distinct fake providers on 127.0.0.1, package exports only; `test/examples-embedded-two-brains.test.ts` runs it. New tests: `test/engine-scope-required.test.ts`, `engine-scope-entry-points`, `engine-open-usage`, `engine-open-settings-keys`, `engine-gateway-state`, `engine-alias-policy`, `engine-cache-namespace`, and `test/e2e/engine-open-b3a-postgres.test.ts` (reconnect usage and 1,000 per-call cycles). Fake transports only, no network.
+ - Still to come: the two-engine seam suite moves onto `openEngine` once train R (#8148) lands; `drainPersistence` does not exist yet (W5a enters scope there); close order and `drainFacts` are B3b.
+
+ ## [0.60.573.0] - 2026-11-04
+
+ **A host can record bytes it stored itself, and every file row names its page.** On an engine opened from settings, the new `file_register` records the `files` row for an object the host already wrote to `settings.storage` at the path `file_upload` would use (`<page_slug>/<filename>`, or `unsorted/<first 8 hash chars>-<filename>`), after reading it back and matching its size and SHA-256. It resolves the page in the caller's source under the caller's read policy and records the row's `source_id` and `page_id`. Registering the same path again updates that row, and never moves it to another source or another page, including a new page that took a renamed page's slug. `file_upload` now records its row's source and page on every engine, so purging a page in a source other than `default` removes its attachments. A brain the CLI or a server opens refuses `file_register` with `embedded_engine_required`; it is never an MCP tool. Contributed by @bradgessler (#8457).
+
+ ### Itemized changes
+
+ - `src/core/ops/files.ts`: `file_register` accepts only the upload layout path, refuses `page_not_found` for a page that does not exist or that the caller's read policy hides, and refuses `invalid_params` for a path whose row belongs to another source or page id; the pre-check and the `ON CONFLICT … WHERE` guard both compare `page_id` with `IS DISTINCT FROM`.
+ - `file_upload` writes `source_id` and the page's id in that source on each row it inserts (#8513). Rows written before this release keep `source_id = 'default'` and no page id; nothing rewrites them. The `gbrain files upload` CLI is unchanged.
+ - `file_upload` refuses `unknown_source` for a source the brain has not registered, before it writes any bytes. When a row already holds the path, it takes the uploader's source and page id only if the uploader's source has that page. A row attached to a live page in another source stays with that source. Identical bytes return `already_exists` without moving it, and different bytes refuse `invalid_params` before any bytes are written.
+ - Page purge and its verification match a file by slug only when the row has no page id, so purging one page never removes another page's attachment at the same slug.
+ - `src/core/storage.ts`: `prefixedStorage` refuses a key with an empty, `.` or `..` segment or a leading `/` (`Path traversal blocked`), so on a local store a path such as `../brain-b/x` never reaches another brain's prefix.
+ - `test/file-register.test.ts`: real PGLite engines and a real local store.
+
+ ## [0.60.572.0] - 2026-11-04
+
+ **A source whose automatic sync is off no longer extracts and reconciles every page on every autopilot cycle.** A per-source cycle that runs no sync for its source (`config.syncEnabled=false`, a claimed source awaiting activation, a gbrain-owned content directory, or a source with no checkout) now extracts links and timeline, reconciles facts and recomputes emotional weight only for the source's pages changed since its previous cycle, as connector sources already did, at most 2,000 pages per cycle. Before, `extract_facts` walked the whole source each time and `extract` walked its whole checkout, so a sync-disabled source of about 27,000 pages or more hit the 30-minute job timeout on every cycle and never stamped its freshness. Pages an explicit `gbrain sync --source` imports between cycles land in the next window, and a large import is worked through over several cycles instead of timing out. A source with no recorded cycle starts from its oldest page and walks forward the same way. Reported by @mlahatte (#8573).
+
+ ### Itemized changes
+
+ - `src/core/minions/handlers/autopilot-cycle.ts`: `sourceChangedSince` is passed to every per-source cycle whose phases leave out `sync` or that has no checkout, not only to connector sources.
+ - `src/core/cycle/source-window.ts`: the window lists the source's pages after a cursor in (`updated_at`, slug) order, `sourceWindow.pageLimit` (2,000) at a time. A cycle that reaches the end of the window stamps its listing instant (`last_source_cycle_window_at`); one that stops at the limit stamps its last page (`last_source_cycle_window_slug` breaks ties), so the next cycle resumes there. The window advances only when `extract`, `extract_facts` and `recompute_emotional_weight` all ran without failing, so a cycle that failed one of them, or ran without them, gets the same window next time. The listing instant is moved back to just before the oldest transaction still open on another connection, because `pages.updated_at` is the writer's transaction start: a write still open while a cycle listed its pages lands in the next window. That holds for writers whose transactions the cycle's database role can see in `pg_stat_activity` (the same role, or one with `pg_read_all_stats`) and that stay open less than 10 minutes. Cursors bind as text, so Postgres keeps their microseconds.
+ - `src/core/cycle.ts`: the `extract` phase of a cycle with a checkout and a source window extracts the window's pages instead of walking the checkout. A file edited on disk without a sync is no longer picked up by the cycle's extract on such a source, as on any source whose sync runs; `lint` still reads every file of the checkout, as it does for every source.
+ - `test/sync-disabled-cycle-window.test.ts` (PGLite and Postgres): an idle cycle of a sync-disabled git source walks no page in `extract_facts` or `extract`, and a page with a facts fence that an explicit sync imports is reconciled by the next cycle; a first walk and a large re-import are worked through in bounded windows that resume; a cycle without the window phases, or with one failed, keeps the window; a source with no checkout gets the window; a write committed after a cycle's listing lands in the next window.
+
+ ## [0.60.571.0] - 2026-11-04
+
+ **A sync hold on a page whose segment rows are missing now says to run the repair, and `gbrain sync status` finds one hold without paging.** A page stored in segments whose segment rows do not cover its body refuses every read, so a managed sync holds its file (`page_write_failed` naming `page_body_rows_unreadable`). That hold read retry with no person needed and pointed at `gbrain sources retry-held`, which can never clear it. It now reads needs_human with safe_actions repair, and its fix previews `gbrain repair segmented-tiling --source <id> --slug <slug>` (read-only; the apply needs the user's agreement and clears the hold). On a source with hundreds of holds the hold also sorted onto a late page of `sync status`, so a check reading the first page missed it.
+
+ ### Itemized changes
+
+ - `src/core/persistence/sync-fault-class.ts`, `sync-holds.ts` (`gitHoldFix`), `sync-entry-fault.ts`: the `freeze_page_body_rows_unreadable` verdict, the hold's fix and its message name the segmented-tiling repair; when doctor `segmented_tiling` does not list the page, the fix says to grant the reading role access and run retry-held. `gbrain sync unblock` refuses the hold by name with that fix.
+ - `gbrain sync status`: each `page_write_failed` hold names its `fault_code`, `holds_summary.by_fault_code` counts them, and `--slug <slug>` / `--code <code>` (a hold code or a fault code) list one hold; a filtered page's cursor and `next_command` keep the filter.
+ - Doctor: `segmented_tiling` runs as its own entry with its own budget, so a default run reports it even when the wave checks of `timeline_history` run out of theirs; an entry's last known result recorded before a check was added no longer hides that check (it reports unknown, or is named in `doctor_last_known`).
+ - `gbrain repair segmented-tiling --apply` records the republished revision as imported from the HEAD blob when the file imports to the page's stored title, type, frontmatter and tags, so the next sync holds no `file_drift` for the page it just repaired. A file whose metadata changed since the last import records nothing, and sync still guards the stored version.
+ - #8504 items 6 and 8: `untiledSegmentedPages` (doctor and the repair) skips archived sources; doctor's fix always names a `--source` and lists the other damaged sources' commands; a repair preview over 1,000 pages reports `truncated` and says to preview again after the apply.
## [0.60.570.0] - 2026-11-04
diff --cc plugin-variants/gbrain-coding/skills/conventions/brain-first.md
index ce933bb8cd,62e7593977..e9a0e72b04
--- a/plugin-variants/gbrain-coding/skills/conventions/brain-first.md
+++ b/plugin-variants/gbrain-coding/skills/conventions/brain-first.md
@@@ -5,62 -5,35 +5,51 @@@
Sub-agents and fresh sessions inherit gbrain tools but not the knowledge of
when and how to use them. This file is that knowledge.
- ## Retrieval: gbrain first
-
- For anything in the brain, search gbrain, then read the page a hit names. Don't
- `grep`, `rg` or `cat` the brain checkout, or run SQL over pages and
- frontmatter, before gbrain: gbrain reads the index across sources, ranks results,
- filters by type, date and source, and returns the slug to read next. Use the
- files only when gbrain is unreachable or for an edit not yet synced.
-
- Prefer the gbrain MCP tools (`search`, `query`, `get_page`, `recall`) when your
- harness has the server connected: it stays connected, so each call skips the
- CLI's startup, which costs seconds per call on a large pooled Postgres brain.
- The CLI commands below take the same filters.
-
- ```bash
- gbrain search "Acme Example" # names, codes, exact phrases
- gbrain search '"ZEBRA-42" OR "inventory sync"' --match keyword # every page with any term (instead of grep)
- gbrain query "why did the launch move" # questions and concepts
- gbrain search "launch" --types '["meeting","person"]' # only these page types
- gbrain query "launch" --since 2026-09-01 --until 2026-09-30 # date window (also --since 30d)
- gbrain search "launch" --source source-a --limit 5 --json | jq -r '.[].slug'
- gbrain get companies/acme-example # read the page a hit points to
- gbrain recall "Alice Example" # facts saved with remember
- gbrain backlinks companies/acme-example --type meeting # pages that link here (instead of grepping a slug)
- gbrain timeline companies/acme-example --since 2026-10-01 # dated events on one page
- gbrain list --type meeting --limit 50 # enumerate by type or --tag (instead of SQL)
- ```
-
- - `--types` takes a JSON array. `--since`/`--until` are `query` flags
- (YYYY-MM-DD or 7d/2w/1y) and match each page's effective date: its
- frontmatter date, else when it was imported.
- - Keyword search matches words, not slugs: search `"Acme Example"`, and use
- `backlinks` to find pages linking to `companies/acme-example`.
- - Without `--source`, a search reads the resolved source (a notice names it
- when it routes); `--source __all__` searches every source.
- - Notices and `[AGENT]` blocks go to stderr, so `--json` stdout is clean JSON.
- Read a degraded notice before concluding the brain has nothing. Never send
- stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
- then reads as "no results". Check the exit code too.
+ ## Which gbrain call for which lookup
+
+ <!-- BEGIN lookup-table -->
+ Ask gbrain before you `grep`, `cat`, `ls` or run SQL on the brain checkout: it
+ reads the index across every source, ranks, filters and names the slug to read
+ next. With the gbrain MCP server connected, call the tool of the same name
+ instead of the CLI (one connection, no per-call startup).
+
+ | Lookup | CLI | MCP tool |
+ |---|---|---|
+ | Find a page by name or path | `gbrain find <name\|prefix\|glob> [--type T] [--source S]` | `find_pages` |
+ | Count pages by kind or source | `gbrain stats --by source,type [--since 30d] [--collapse-aliases]` | `get_stats` (needs an admin-scope token) |
+ | Answer a question, search a topic | `gbrain query "<question>" [--types '["meeting"]'] [--since 30d]` | `query` |
+ | Pages with these words | `gbrain search "<words>" [--match keyword] [--source S]` | `search` |
+ | Read a known page | `gbrain get <slug>` | `get_page` |
+ | Read only the part a search hit points at | `gbrain get <slug> --around-chunk-id <chunk_id>` or `--lines '{"from":1,"count":200}'` | `get_page` |
+ | What the brain knows about a person or company | `gbrain entity <slug>`, then `gbrain recall "<topic>" --json` | `entity`, `recall` |
+ | Links, dated events, every page of a type | `gbrain backlinks <slug>`, `gbrain timeline <slug>`, `gbrain list --type T` | `get_backlinks`, `get_timeline`, `list_pages` |
+ | Save a fact the user stated | `gbrain remember "<fact>" --entity <slug> --content-origin user_said --provenance "<where>"` | `remember` |
+ | Why a page didn't rank | `gbrain query "<q>" --explain-target <slug>` | `query` |
+
+ Parse `--json` output instead of matching the text; notices go to stderr. Never
+ send stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
+ then reads as "no results"; check the exit code too. Take
+ `chunk_id` from a fresh `search --json` hit (an old one fails `revision_changed`),
+ and read a degraded notice before concluding the brain has nothing.
+ <!-- END lookup-table -->
<a id="identity-exact-match"></a>
+## Dated questions: read by valid time
+
+When the user asks what a page said, or what was in force, on a date, read it
+with `get_page {slug, as_of: 'YYYY-MM-DD'}` (CLI: `gbrain get <slug> --as-of
+<date>`), not the current page. Quote text only from an `in_force` answer; any
+other status puts the edition under `last_in_force` or `earliest_known`, and
+says so. When `valid_time.uncertain` is true, tell the user why and follow the
+`fix` (it asks the user for a date or a backup; never invent either). A reply
+without `valid_time` is the current page.
+
+When you edit a page whose text takes effect on a date, keep its `valid_from`
+for a correction, or set a new quoted `valid_from` for a new edition. Read the
+write result's `valid_time` advisory and ask the user when it asks a question.
+Which record to use for a change in effect (edit in place, amendment record,
+`supersedes` link): the page history guide (`docs/guides/page-history.md`, Valid time).
+
## Identity needs an exact match
Resolving an email sender, a calendar attendee or a contact to a person page is
diff --cc plugin-variants/gbrain-daily/skills/conventions/brain-first.md
index ce933bb8cd,62e7593977..e9a0e72b04
--- a/plugin-variants/gbrain-daily/skills/conventions/brain-first.md
+++ b/plugin-variants/gbrain-daily/skills/conventions/brain-first.md
@@@ -5,62 -5,35 +5,51 @@@
Sub-agents and fresh sessions inherit gbrain tools but not the knowledge of
when and how to use them. This file is that knowledge.
- ## Retrieval: gbrain first
-
- For anything in the brain, search gbrain, then read the page a hit names. Don't
- `grep`, `rg` or `cat` the brain checkout, or run SQL over pages and
- frontmatter, before gbrain: gbrain reads the index across sources, ranks results,
- filters by type, date and source, and returns the slug to read next. Use the
- files only when gbrain is unreachable or for an edit not yet synced.
-
- Prefer the gbrain MCP tools (`search`, `query`, `get_page`, `recall`) when your
- harness has the server connected: it stays connected, so each call skips the
- CLI's startup, which costs seconds per call on a large pooled Postgres brain.
- The CLI commands below take the same filters.
-
- ```bash
- gbrain search "Acme Example" # names, codes, exact phrases
- gbrain search '"ZEBRA-42" OR "inventory sync"' --match keyword # every page with any term (instead of grep)
- gbrain query "why did the launch move" # questions and concepts
- gbrain search "launch" --types '["meeting","person"]' # only these page types
- gbrain query "launch" --since 2026-09-01 --until 2026-09-30 # date window (also --since 30d)
- gbrain search "launch" --source source-a --limit 5 --json | jq -r '.[].slug'
- gbrain get companies/acme-example # read the page a hit points to
- gbrain recall "Alice Example" # facts saved with remember
- gbrain backlinks companies/acme-example --type meeting # pages that link here (instead of grepping a slug)
- gbrain timeline companies/acme-example --since 2026-10-01 # dated events on one page
- gbrain list --type meeting --limit 50 # enumerate by type or --tag (instead of SQL)
- ```
-
- - `--types` takes a JSON array. `--since`/`--until` are `query` flags
- (YYYY-MM-DD or 7d/2w/1y) and match each page's effective date: its
- frontmatter date, else when it was imported.
- - Keyword search matches words, not slugs: search `"Acme Example"`, and use
- `backlinks` to find pages linking to `companies/acme-example`.
- - Without `--source`, a search reads the resolved source (a notice names it
- when it routes); `--source __all__` searches every source.
- - Notices and `[AGENT]` blocks go to stderr, so `--json` stdout is clean JSON.
- Read a degraded notice before concluding the brain has nothing. Never send
- stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
- then reads as "no results". Check the exit code too.
+ ## Which gbrain call for which lookup
+
+ <!-- BEGIN lookup-table -->
+ Ask gbrain before you `grep`, `cat`, `ls` or run SQL on the brain checkout: it
+ reads the index across every source, ranks, filters and names the slug to read
+ next. With the gbrain MCP server connected, call the tool of the same name
+ instead of the CLI (one connection, no per-call startup).
+
+ | Lookup | CLI | MCP tool |
+ |---|---|---|
+ | Find a page by name or path | `gbrain find <name\|prefix\|glob> [--type T] [--source S]` | `find_pages` |
+ | Count pages by kind or source | `gbrain stats --by source,type [--since 30d] [--collapse-aliases]` | `get_stats` (needs an admin-scope token) |
+ | Answer a question, search a topic | `gbrain query "<question>" [--types '["meeting"]'] [--since 30d]` | `query` |
+ | Pages with these words | `gbrain search "<words>" [--match keyword] [--source S]` | `search` |
+ | Read a known page | `gbrain get <slug>` | `get_page` |
+ | Read only the part a search hit points at | `gbrain get <slug> --around-chunk-id <chunk_id>` or `--lines '{"from":1,"count":200}'` | `get_page` |
+ | What the brain knows about a person or company | `gbrain entity <slug>`, then `gbrain recall "<topic>" --json` | `entity`, `recall` |
+ | Links, dated events, every page of a type | `gbrain backlinks <slug>`, `gbrain timeline <slug>`, `gbrain list --type T` | `get_backlinks`, `get_timeline`, `list_pages` |
+ | Save a fact the user stated | `gbrain remember "<fact>" --entity <slug> --content-origin user_said --provenance "<where>"` | `remember` |
+ | Why a page didn't rank | `gbrain query "<q>" --explain-target <slug>` | `query` |
+
+ Parse `--json` output instead of matching the text; notices go to stderr. Never
+ send stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
+ then reads as "no results"; check the exit code too. Take
+ `chunk_id` from a fresh `search --json` hit (an old one fails `revision_changed`),
+ and read a degraded notice before concluding the brain has nothing.
+ <!-- END lookup-table -->
<a id="identity-exact-match"></a>
+## Dated questions: read by valid time
+
+When the user asks what a page said, or what was in force, on a date, read it
+with `get_page {slug, as_of: 'YYYY-MM-DD'}` (CLI: `gbrain get <slug> --as-of
+<date>`), not the current page. Quote text only from an `in_force` answer; any
+other status puts the edition under `last_in_force` or `earliest_known`, and
+says so. When `valid_time.uncertain` is true, tell the user why and follow the
+`fix` (it asks the user for a date or a backup; never invent either). A reply
+without `valid_time` is the current page.
+
+When you edit a page whose text takes effect on a date, keep its `valid_from`
+for a correction, or set a new quoted `valid_from` for a new edition. Read the
+write result's `valid_time` advisory and ask the user when it asks a question.
+Which record to use for a change in effect (edit in place, amendment record,
+`supersedes` link): the page history guide (`docs/guides/page-history.md`, Valid time).
+
## Identity needs an exact match
Resolving an email sender, a calendar attendee or a contact to a person page is
diff --cc plugin/skills/conventions/brain-first.md
index ce933bb8cd,62e7593977..e9a0e72b04
--- a/plugin/skills/conventions/brain-first.md
+++ b/plugin/skills/conventions/brain-first.md
@@@ -5,62 -5,35 +5,51 @@@
Sub-agents and fresh sessions inherit gbrain tools but not the knowledge of
when and how to use them. This file is that knowledge.
- ## Retrieval: gbrain first
-
- For anything in the brain, search gbrain, then read the page a hit names. Don't
- `grep`, `rg` or `cat` the brain checkout, or run SQL over pages and
- frontmatter, before gbrain: gbrain reads the index across sources, ranks results,
- filters by type, date and source, and returns the slug to read next. Use the
- files only when gbrain is unreachable or for an edit not yet synced.
-
- Prefer the gbrain MCP tools (`search`, `query`, `get_page`, `recall`) when your
- harness has the server connected: it stays connected, so each call skips the
- CLI's startup, which costs seconds per call on a large pooled Postgres brain.
- The CLI commands below take the same filters.
-
- ```bash
- gbrain search "Acme Example" # names, codes, exact phrases
- gbrain search '"ZEBRA-42" OR "inventory sync"' --match keyword # every page with any term (instead of grep)
- gbrain query "why did the launch move" # questions and concepts
- gbrain search "launch" --types '["meeting","person"]' # only these page types
- gbrain query "launch" --since 2026-09-01 --until 2026-09-30 # date window (also --since 30d)
- gbrain search "launch" --source source-a --limit 5 --json | jq -r '.[].slug'
- gbrain get companies/acme-example # read the page a hit points to
- gbrain recall "Alice Example" # facts saved with remember
- gbrain backlinks companies/acme-example --type meeting # pages that link here (instead of grepping a slug)
- gbrain timeline companies/acme-example --since 2026-10-01 # dated events on one page
- gbrain list --type meeting --limit 50 # enumerate by type or --tag (instead of SQL)
- ```
-
- - `--types` takes a JSON array. `--since`/`--until` are `query` flags
- (YYYY-MM-DD or 7d/2w/1y) and match each page's effective date: its
- frontmatter date, else when it was imported.
- - Keyword search matches words, not slugs: search `"Acme Example"`, and use
- `backlinks` to find pages linking to `companies/acme-example`.
- - Without `--source`, a search reads the resolved source (a notice names it
- when it routes); `--source __all__` searches every source.
- - Notices and `[AGENT]` blocks go to stderr, so `--json` stdout is clean JSON.
- Read a degraded notice before concluding the brain has nothing. Never send
- stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
- then reads as "no results". Check the exit code too.
+ ## Which gbrain call for which lookup
+
+ <!-- BEGIN lookup-table -->
+ Ask gbrain before you `grep`, `cat`, `ls` or run SQL on the brain checkout: it
+ reads the index across every source, ranks, filters and names the slug to read
+ next. With the gbrain MCP server connected, call the tool of the same name
+ instead of the CLI (one connection, no per-call startup).
+
+ | Lookup | CLI | MCP tool |
+ |---|---|---|
+ | Find a page by name or path | `gbrain find <name\|prefix\|glob> [--type T] [--source S]` | `find_pages` |
+ | Count pages by kind or source | `gbrain stats --by source,type [--since 30d] [--collapse-aliases]` | `get_stats` (needs an admin-scope token) |
+ | Answer a question, search a topic | `gbrain query "<question>" [--types '["meeting"]'] [--since 30d]` | `query` |
+ | Pages with these words | `gbrain search "<words>" [--match keyword] [--source S]` | `search` |
+ | Read a known page | `gbrain get <slug>` | `get_page` |
+ | Read only the part a search hit points at | `gbrain get <slug> --around-chunk-id <chunk_id>` or `--lines '{"from":1,"count":200}'` | `get_page` |
+ | What the brain knows about a person or company | `gbrain entity <slug>`, then `gbrain recall "<topic>" --json` | `entity`, `recall` |
+ | Links, dated events, every page of a type | `gbrain backlinks <slug>`, `gbrain timeline <slug>`, `gbrain list --type T` | `get_backlinks`, `get_timeline`, `list_pages` |
+ | Save a fact the user stated | `gbrain remember "<fact>" --entity <slug> --content-origin user_said --provenance "<where>"` | `remember` |
+ | Why a page didn't rank | `gbrain query "<q>" --explain-target <slug>` | `query` |
+
+ Parse `--json` output instead of matching the text; notices go to stderr. Never
+ send stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
+ then reads as "no results"; check the exit code too. Take
+ `chunk_id` from a fresh `search --json` hit (an old one fails `revision_changed`),
+ and read a degraded notice before concluding the brain has nothing.
+ <!-- END lookup-table -->
<a id="identity-exact-match"></a>
+## Dated questions: read by valid time
+
+When the user asks what a page said, or what was in force, on a date, read it
+with `get_page {slug, as_of: 'YYYY-MM-DD'}` (CLI: `gbrain get <slug> --as-of
+<date>`), not the current page. Quote text only from an `in_force` answer; any
+other status puts the edition under `last_in_force` or `earliest_known`, and
+says so. When `valid_time.uncertain` is true, tell the user why and follow the
+`fix` (it asks the user for a date or a backup; never invent either). A reply
+without `valid_time` is the current page.
+
+When you edit a page whose text takes effect on a date, keep its `valid_from`
+for a correction, or set a new quoted `valid_from` for a new edition. Read the
+write result's `valid_time` advisory and ask the user when it asks a question.
+Which record to use for a change in effect (edit in place, amendment record,
+`supersedes` link): the page history guide (`docs/guides/page-history.md`, Valid time).
+
## Identity needs an exact match
Resolving an email sender, a calendar attendee or a contact to a person page is
diff --cc scripts/llms-config.ts
index d6658738b7,5bbccb9a87..1b34f1136c
--- a/scripts/llms-config.ts
+++ b/scripts/llms-config.ts
@@@ -42,8 -42,7 +42,8 @@@ export const SECTIONS: DocSection[] =
{ title: "Grok Bot", path: "docs/guides/grok-bot.md", description: "Recommended: keep the brain on your computer, publish it with `gbrain mcp expose --funnel`, grant the Bot a client and install the thin CLI; local in-Bot memory as the no-host alternative. Native skill activation, shared-computer trust and dated evidence." },
{ title: "Muse personal agent", path: "docs/guides/muse.md", description: "Verify durable storage first; connect Muse to your published brain (`gbrain mcp expose --funnel` + thin CLI) or add local explicit memory with provenance alongside Muse's native memory; distinguish Muse Code and unverified native MCP." },
{ title: "Inside an existing agent", path: "docs/guides/in-agent-setup.md", description: "Isolated setup, absolute launcher, receipts, keyless maintenance, complete private backup and safe restoration." },
- { title: "Memory boundaries", path: "docs/guides/memory-boundaries.md", description: "Durable shared preferences versus harness state; remote graph maintenance, provider text disclosure, authorization and full-backup limits.", includeInFull: false },
+ { title: "Memory boundaries", path: "docs/guides/memory-boundaries.md", description: "Durable shared preferences versus harness state; remote graph maintenance, provider text disclosure, authorization and full-backup limits; large pages (segmented storage up to 256 MiB on Postgres, append_page / gbrain append, upload sessions, windowed reads, limits per entry point).", includeInFull: false },
+ { title: "Page history", path: "docs/guides/page-history.md", description: "Why a page changed: an optional reason on page writes, replaced_because in get_versions, the gbrain-reason commit trailer, remote visibility and repo-less brains.", includeInFull: false },
{ title: "Brain versus session memory", path: "docs/guides/brain-vs-memory.md", description: "Route durable facts and preferences to shared memory, configuration to the harness, and temporary work to session state.", includeInFull: false },
{ title: "Connect a hosted brain", path: "docs/guides/hosted-harness-access.md", description: "Getting the HTTPS endpoint (`gbrain mcp expose`, admin token file), host-side grants, private credential handoff, harness installation, profile repair, delegation limits and honest verification." },
{ title: "Harness adapter reference", path: "docs/guides/harness-adapters.md", description: "Registry-generated transport, authentication lifetime and configuration facts.", includeInFull: false },
diff --cc scripts/module-size-limits.tsv
index 7bdd863dd1,7495329c89..8e854c6b2b
--- a/scripts/module-size-limits.tsv
+++ b/scripts/module-size-limits.tsv
@@@ -45,6 -45,6 +45,6 @@@ src/core/search/mode.ts 1730 ratchet +1
src/commands/eval-longmemeval.ts 1545 ratchet new row: crossed the 1500 unlisted cap in fw3-search (#5092 benchmark-brain recycling: configureBrain split + recycler wiring + test seams; the recycler itself lives in src/eval/longmemeval/harness.ts); +1 P6: tokenmax synopsis-tier wiring (logic in eval/longmemeval/synopsis-tier.ts); +13 GBRA-81 lane A: --via-operation query|search (flags, per-call expansion pin, the run's expectExpansion, the op branch in runOneQuestion, row field, via_operation_summary and its FAIL line; logic in eval/longmemeval/via-operation.ts); +12 --call-plan FILE / --call-plan-gate CMD: two LME_FLAGS entries (CLI help lines), two ParsedArgs fields, the parseArgs pairing check and the header invariant (planner lives in eval/longmemeval/call-plan.ts; runEvalLongMemEval is one line shorter); +1 #7619: parseArgs refuses a value on a switch (--judge=false turned the paid judge on)
src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: links_extraction_lag held bucket and its retry fix (logic in core/extraction-lag.ts heldExtractionReport); +1 undeclared_db_only_pages computes the source Git scope once per source (logic in core/markdown.ts sourceGitScope); new row: crossed the 1500 unlisted cap at the fix wave 9 merge: +7 lane W #5028 (extract_atoms freshness judged by the phase itself) and +3 lane F #5863 (extract_health dates a high halt rate by its last halt), each under the cap alone; +7 #5019: content_hash_duplicates fix pins the pairs' source (GBRAIN_SOURCE) and names a delete that runs; +1 fix wave 10 lane L1 #5556: links_extraction_lag denominator excludes quarantined pages (quarantineFilterFragment import); +11 fix wave 15 P2.8 (#5495): extract_health reads the 7-day halt reasons per kind (halt_reasons, top_halt_reason in details and the top reason in the warn line); the read, the unknown bucket and the ranking live in core/extract/rollup-writer.ts; +20 wave 18b lane 1: links_extraction_lag names the columns its pre-migration degrade reads (#6599 strict predicate), sync_freshness separates sync outcome from freshness for a source with no successful sync (#4960; verdict text, fixes and the attempt read live in doctor/sync-outcome.ts and core/sync-attempt.ts) and exempts read-only skill-pack catalogs (#6076; core/shared-skills/adopted-packs.ts); -51 #6803 (D1): links_extraction_lag reads its counts and verdict from core/extraction-lag.ts (shared with the sync nudge); +1 #7145: content_hash_duplicates reads one group past its list to report a floor ("at least 50")
src/core/search/evidence-delivery.ts 1759 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move); +7 GBRA-205: EvidencePlan.titleHeader and DeliveryMeta.title_header (interface fields cannot move) and the titles option on the header loader (logic in search/evidence-date.ts; the query budget key lives in search/evidence-packing.ts); +11 #7600: assemble_evidence quotes conversation rows and capEvidenceToBudget counts the quote fence (logic in search/transcript-quote.ts); +2 GBRA-156 /ship review: the cut-path admission and title room count the quote fence; +2 GBRA-156 /ship review pass 2: capEvidenceToBudget counts the fence only for the paths that fence (quote flag; recall counts the text as returned); +4 GST-687: page evidence and the extractive fallback quote below-owner-tier text for untrusted readers (untrusted flag; logic in search/transcript-quote.ts)
-src/commands/config.ts 1524 ratchet new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)
+src/commands/config.ts 1526 ratchet +1 GBRA-216: recall.* config set value refusal (facts/recall-config.ts); +1 GBRA-216: config set refuses a dream.claim_conflicts.mode other than propose|off (refusal in facts/claim-conflicts.ts); new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)
src/commands/mcp-expose.ts 1519 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts); +7 GBRA-199: --surface help lines, surfaceChosen option field and the advertise pass-through (registrationAdvertise in core/mcp-registration.ts)
- src/core/behavior-change-notice.ts 1637 ratchet +3 GBRA-200 comment, write-grant refusal and degraded-stamp rows (re-summed at the 0.60.562.0 merge: 1615 + 3); +2 GBRA-280 get_page selector-precedence row and comment (re-summed at the GBRA-300 train merge of master 241cef5a: 1635 + 2); the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-287: segmented_freeze_hold_gbra287 and segmented_reexport_gbra287 rows (rows are data and cannot move); +1 GST-1209: segmented_tiling_gst1209 row (rows are data and cannot move); the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +6 GBRA-190 G4 wave 2 (#7616): five rows and their comment; +3 GBRA-216 PR B: the comment and two rows for decide contested labels and the trust review contradiction kind; the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-217: the valid_time_plain_read row (rows cannot move); +1 GBRA-215 L3: ingest_starter_listing row; +1 GBRA-215 L2: extraction_review_page_door row; +1 GBRA-215 L1: extraction_pending review lists row; the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move)
+ src/core/behavior-change-notice.ts 1627 ratchet +4 train R: #7491/#7492 keyword_identifier_probe and keyword_slug_handles rows and their comment, #7175 keyword phrase row; the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +1 GBRA-156 #7593: private_pages_granted_sources row (re-summed at the 0.60.527.0 merge: 1574 + 1); +3 GBRA-200 comment, write-grant refusal and degraded-stamp rows (re-summed at the 0.60.562.0 merge: 1615 + 3); +2 GBRA-280 get_page selector-precedence row and comment (re-summed at the 0.60.569.0 merge: 1619 + 2); the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +2 #7199 (re-port of #7228) classic_drift_rescreen_7199 comment and row; +2 #7931 file_drift_agrees_7931 comment and row; +1 #7950 file_drift_slug_release_7950 row; +1 #7974 file_drift_followups_7974 row; +1 #8059 quoted_fence_marker_8059 row; +2 GBRA-247 #8009 provenance_repair_held_8009 and #7982 pass_transient_streak_7982 rows; +4 GBRA-247 #7973 plain_directory_sync_7973, #7988 managed_marker_own_directory_7988, #7985 former_binding_parked_group_7985 and settle_effects_retire_7985 rows Train S merge: 1570 base + 5 (master since the #8077 base) + 13 (#8077) = 1588.; +2 GST-687 remote_quotes_unverified_text_gst687 row and its comment (re-summed at the gate-ahead on train S 0.60.534.0: 1588 + 2); +2 #8377: the quote fence follow-ups row and its comment; +2 #8384: the quote fence follow-ups 2 row and its comment; +2 #8384 wave 3: the quote fence follow-ups 3 row and its comment; +5 GBRA-218 train M rows (A1 member_read_access_gbra218 and member_access_hatch_scope_gbra218, B shared_agent_audience_gbra231, A2 folder_access_rules_gbra218, folder_access_scoped_role_gbra218), re-summed at the train M merge of master v0.60.541.0: 1596 + 5 = 1601 measured; +3 GST-359 train C rows (C1 qm_attested_actor_gst359, C2 member_facts_gst359, D qm_memory_provider_gst359), re-summed at the train C merge of master v0.60.543.0: 1601 + 3 = 1604 measured; +2 #8424: hook_launcher_8424 row and its comment (rows are data and cannot move), re-summed at the #8418 v0.60.546.0 merge: 1604 + 2 = 1606 measured; +2 #8426: mcp_lane_launcher_8426 row and its comment (rows are data and cannot move); +2 GST-927: empty_nonstring_args_gst927 row and its comment, re-summed at the gate-ahead on master v0.60.550.0 plus #8420: 1610 measured; +2 GBRA-283 Wave A (#8460): engine_keyed_caches_gbra283 row and its comment (rows are data and cannot move); +2 GBRA-287: segmented_freeze_hold_gbra287 and segmented_reexport_gbra287 rows (rows are data and cannot move); +1 GST-1209: segmented_tiling_gst1209 row (rows are data and cannot move); +1 GST-1251: skillopt_loop_deprecated_gst1251 row (rows are data and cannot move); +1 GST-1209 follow-up: untiled_hold_repair_gst1209 row (rows are data and cannot move); +1 GBRA-283 B3a: embedded_scope_required_gbra283 row (rows are data and cannot move); re-summed at the train R merge with v0.60.574.0: 1623 + 4
diff --cc scripts/serial-files.tsv
index ba6ad83af2,2ec4f3f2f4..80a979def1
--- a/scripts/serial-files.tsv
+++ b/scripts/serial-files.tsv
@@@ -454,5 -454,7 +454,8 @@@ test/embed-once-per-content-version.ser
test/page-embedding-lease.serial.test.ts global-state global-state: process-wide test seams __setEmbedTransportForTests (each case swaps the embed transport, one to a refusing provider) and pageEmbeddingLeaseProbe (the module-wide lease contention hook); withEnv scopes GBRAIN_HOME
test/migrate-search-path-best-effort.serial.test.ts R2 R2: mock.module(v277 migration module) replaces modules for every file in the process
test/agent-session-speaker-8064.serial.test.ts R1 R1: sets GBRAIN_HOME and GBRAIN_SESSION_AGENT and clears DATABASE_URL and the hook env on process.env
+ test/two-engine-isolation.serial.test.ts R1 R1: mutates process.env (GBRAIN_HOME); also drives the process-wide AI gateway, chat usage sink, search telemetry, link semantics and module caches the suite proves are shared, and binds fake AI provider ports
+ test/search/identifier-probe.serial.test.ts R1 R1: mutates process.env (GBRAIN_HOME)
+ test/search/slug-handles.serial.test.ts R1 R1: mutates process.env (GBRAIN_HOME)
test/launcher-lanes-bunfig.serial.test.ts global-state global-state: spawns gbrain serve subprocesses against one shared PGLite brain and its data-dir lock; withEnv scopes HOME, GBRAIN_HOME and CODEX_HOME around the bootstrap and setup writers
+test/extraction-review-door.serial.test.ts R1 R1: mutates process.env (GBRAIN_HOME) around real git subprocesses against tmp repos for the page-door review decisions
diff --cc skills/conventions/brain-first.md
index ce933bb8cd,62e7593977..e9a0e72b04
--- a/skills/conventions/brain-first.md
+++ b/skills/conventions/brain-first.md
@@@ -5,62 -5,35 +5,51 @@@
Sub-agents and fresh sessions inherit gbrain tools but not the knowledge of
when and how to use them. This file is that knowledge.
- ## Retrieval: gbrain first
-
- For anything in the brain, search gbrain, then read the page a hit names. Don't
- `grep`, `rg` or `cat` the brain checkout, or run SQL over pages and
- frontmatter, before gbrain: gbrain reads the index across sources, ranks results,
- filters by type, date and source, and returns the slug to read next. Use the
- files only when gbrain is unreachable or for an edit not yet synced.
-
- Prefer the gbrain MCP tools (`search`, `query`, `get_page`, `recall`) when your
- harness has the server connected: it stays connected, so each call skips the
- CLI's startup, which costs seconds per call on a large pooled Postgres brain.
- The CLI commands below take the same filters.
-
- ```bash
- gbrain search "Acme Example" # names, codes, exact phrases
- gbrain search '"ZEBRA-42" OR "inventory sync"' --match keyword # every page with any term (instead of grep)
- gbrain query "why did the launch move" # questions and concepts
- gbrain search "launch" --types '["meeting","person"]' # only these page types
- gbrain query "launch" --since 2026-09-01 --until 2026-09-30 # date window (also --since 30d)
- gbrain search "launch" --source source-a --limit 5 --json | jq -r '.[].slug'
- gbrain get companies/acme-example # read the page a hit points to
- gbrain recall "Alice Example" # facts saved with remember
- gbrain backlinks companies/acme-example --type meeting # pages that link here (instead of grepping a slug)
- gbrain timeline companies/acme-example --since 2026-10-01 # dated events on one page
- gbrain list --type meeting --limit 50 # enumerate by type or --tag (instead of SQL)
- ```
-
- - `--types` takes a JSON array. `--since`/`--until` are `query` flags
- (YYYY-MM-DD or 7d/2w/1y) and match each page's effective date: its
- frontmatter date, else when it was imported.
- - Keyword search matches words, not slugs: search `"Acme Example"`, and use
- `backlinks` to find pages linking to `companies/acme-example`.
- - Without `--source`, a search reads the resolved source (a notice names it
- when it routes); `--source __all__` searches every source.
- - Notices and `[AGENT]` blocks go to stderr, so `--json` stdout is clean JSON.
- Read a degraded notice before concluding the brain has nothing. Never send
- stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
- then reads as "no results". Check the exit code too.
+ ## Which gbrain call for which lookup
+
+ <!-- BEGIN lookup-table -->
+ Ask gbrain before you `grep`, `cat`, `ls` or run SQL on the brain checkout: it
+ reads the index across every source, ranks, filters and names the slug to read
+ next. With the gbrain MCP server connected, call the tool of the same name
+ instead of the CLI (one connection, no per-call startup).
+
+ | Lookup | CLI | MCP tool |
+ |---|---|---|
+ | Find a page by name or path | `gbrain find <name\|prefix\|glob> [--type T] [--source S]` | `find_pages` |
+ | Count pages by kind or source | `gbrain stats --by source,type [--since 30d] [--collapse-aliases]` | `get_stats` (needs an admin-scope token) |
+ | Answer a question, search a topic | `gbrain query "<question>" [--types '["meeting"]'] [--since 30d]` | `query` |
+ | Pages with these words | `gbrain search "<words>" [--match keyword] [--source S]` | `search` |
+ | Read a known page | `gbrain get <slug>` | `get_page` |
+ | Read only the part a search hit points at | `gbrain get <slug> --around-chunk-id <chunk_id>` or `--lines '{"from":1,"count":200}'` | `get_page` |
+ | What the brain knows about a person or company | `gbrain entity <slug>`, then `gbrain recall "<topic>" --json` | `entity`, `recall` |
+ | Links, dated events, every page of a type | `gbrain backlinks <slug>`, `gbrain timeline <slug>`, `gbrain list --type T` | `get_backlinks`, `get_timeline`, `list_pages` |
+ | Save a fact the user stated | `gbrain remember "<fact>" --entity <slug> --content-origin user_said --provenance "<where>"` | `remember` |
+ | Why a page didn't rank | `gbrain query "<q>" --explain-target <slug>` | `query` |
+
+ Parse `--json` output instead of matching the text; notices go to stderr. Never
+ send stderr to `/dev/null`: a refusal there (`no_brain`, `GBRAIN_DB_ACCESS`)
+ then reads as "no results"; check the exit code too. Take
+ `chunk_id` from a fresh `search --json` hit (an old one fails `revision_changed`),
+ and read a degraded notice before concluding the brain has nothing.
+ <!-- END lookup-table -->
<a id="identity-exact-match"></a>
+## Dated questions: read by valid time
+
+When the user asks what a page said, or what was in force, on a date, read it
+with `get_page {slug, as_of: 'YYYY-MM-DD'}` (CLI: `gbrain get <slug> --as-of
+<date>`), not the current page. Quote text only from an `in_force` answer; any
+other status puts the edition under `last_in_force` or `earliest_known`, and
+says so. When `valid_time.uncertain` is true, tell the user why and follow the
+`fix` (it asks the user for a date or a backup; never invent either). A reply
+without `valid_time` is the current page.
+
+When you edit a page whose text takes effect on a date, keep its `valid_from`
+for a correction, or set a new quoted `valid_from` for a new edition. Read the
+write result's `valid_time` advisory and ask the user when it asks a question.
+Which record to use for a change in effect (edit in place, amendment record,
+`supersedes` link): the page history guide (`docs/guides/page-history.md`, Valid time).
+
## Identity needs an exact match
Resolving an email sender, a calendar attendee or a contact to a person page is
diff --cc src/core/behavior-change-notice.ts
index 66b1e60f79,2451a4bc99..6aac61db38
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@@ -1423,20 -1425,10 +1425,24 @@@ export const BEHAVIOR_CHANGES: Readonly
{ since: '0.60.569.0', id: 'skillopt_loop_deprecated_gst1251', text: 'SkillOpt\'s iterative loop is deprecated and is removed on or after 2026-12-03: `gbrain skillopt` with its loop flag, a resumed run the loop started, `run_skillopt` with `optimizer_mode: "reflect"` and a queued `skillopt` job in loop mode (including one queued before the one-shot default, which carries no mode), whose notice goes to the worker\'s stderr, each print the `skillopt_loop_deprecated` coaching notice once (MCP: once per session, muteable) and otherwise run exactly as before. The default one-shot rewrite, the cycle phase and every flag and config key are unchanged. Nothing replaces the loop: edit the skill directly, or run the default one-shot rewrite.' },
// GBRA-280 (GBEV-30 re-read) get_page window-selector precedence; release:restamp stamps since at the merge slot.
{ since: '0.60.570.0', id: 'get_page_selector_precedence_gbra280', text: 'get_page given several window selectors now reads one window instead of refusing with invalid_params: cursor, then around_chunk_id, then lines, then bytes, and a cursor that cannot read the page or an around_chunk_id no longer on it gives way to the next. body full, include_content and content_only beside a selector are set aside, so the read stays a window and never returns the whole page. The response\'s window_selector names what applied and what was ignored; a call with one selector is unchanged.' },
+ { since: '0.60.574.0', id: 'embedded_scope_required_gbra283', text: 'Experimental embedded engines (`openEngine` with `embedded_core: true`) only; the CLI, `gbrain serve` and mounts are unchanged. While such an engine is open, an embed, chat, expansion or rerank call that entered no scope now refuses `engine_scope_required` (new code) even when the host configured a process gateway with `configureGateway`; before, that configuration let unscoped calls use the process keys, and the refusal was `engine_scope_missing`, which now only means a provider the engine lacks or a closed engine. Wrap such calls in `withEngineScope(engine, fn)`, or `withLocalScope(fn)` for calls that belong to the process gateway (both from `gbrain/engine-open`); `hybridSearch`, the import paths, minion jobs and `POST /mcp` enter their engine\'s scope themselves. `openEngine` also refuses an unknown settings key, top-level or nested, with `engine_settings_invalid` naming the key, where it used to ignore it. An embedded engine\'s chat usage is no longer on the process usage stack and survives `reconnect()`. See docs/ENGINES.md "Upgrading an embedded host".' },
+ // GBRA-156 (#7491, #7492). `since` is stamped at the merge slot.
+ { since: '0.60.575.0', id: 'keyword_identifier_probe', text: 'A search or query naming an identifier (`#412`, a 6+ digit run, a code like INV-2026-00417, an email or a URL) together with other words now runs one more keyword statement: the arm\'s own statement, with the same scope, visibility and filters, matching every identifier the query names. Those rows vote like strict keyword rows, so the page holding the identifier is no longer dropped when filler words ("What did the latest weekly digest (#412) say?") keep it from matching every word, and a title holding every identifier leads the title arm\'s any-word retry. The keyword arm\'s any-word retry ranks rare words first for questions of up to 12 words (it was 4). Queries without an identifier keep their keyword rows. `_meta.retrieval.coverage` counts a pending page only when a query word is a whole token of its title or slug (`412` no longer matches `t004120`), counts pending image pages only when an image arm ran, and adds `could_affect.pending_images`; the `projection_pending` notice says text pages are being indexed and that image pages wait for `gbrain projections drain` or a paid re-import, with doctor\'s `text_projection_readiness` as the fix.' },
+ { since: '0.60.575.0', id: 'keyword_slug_handles', text: 'Keyword search resolves slug handles: a hyphenated token with a letter in it (`dan-example`), bare or quoted on its own, now also matches the pages whose slug ends in exactly that handle (`people/dan-example`), under the same scope, filters and visibility as any keyword match. Those pages lead `match: "keyword"` results and its count, and lead the hybrid keyword arm as rows that vote like strict rows, so `"dan-example" OR "erin-example"` with `--types \'["person"]\'` returns the two person pages instead of nothing. Queries without such a token run as before.' },
+ // GBRA-190 G4 wave 2 (#7616). `since` is stamped at the merge slot.
+ { since: '0.60.571.0', text: 'An `extract_facts` run whose empty-fence guard holds a page (a fact row of it is not in the page\'s `## Facts` table yet) is now booked in `gbrain extract status` and doctor `extract_health` as a completed round instead of an `integrity_guard` halt. The guard holds only that page while every other page reconciles, so one held page no longer reads as a facts halt on every cycle; doctor `fence_integrity` still names the held rows (GBRA-190).' },
+ { since: '0.60.571.0', text: 'Doctor `fence_integrity` now points a legacy fact row that copies a claim its page\'s `## Facts` table already carries (`duplicate_claim`) at the read-only `gbrain repair duplicate-facts --source <id>` preview, which keeps the fence row and supersedes the copy onto it after the user agrees. Before, every unadoptable row got only a report with no command (GBRA-190).' },
+ { since: '0.60.571.0', text: '`forget` and `forget_fact` now write the page files first: every page whose `## Facts` table carries the claim has its row struck (`~~claim~~`, a `valid_until` date and `forgotten: <reason>`) and all of them publish in one write, one Git commit on a source that commits through gbrain, before the withdrawal commits in the same transaction. Before, the database withdrew the fact at once and a later effect rewrote the files. A forget now waits for the source\'s writer like `remember`: while that writer is unavailable it returns the pending-write receipt and the fact is still recalled until it publishes. The page history gets one change for the withdrawal instead of a second materialization change. Withdrawals queued by an older release still drain (GBRA-190).' },
+ { since: '0.60.571.0', text: 'A fact remembered without an entity is now a row of the `memory/unattributed` page\'s `## Facts` table (created on first use) instead of a database-only row, so it is in Git, `recall` and search can return that page, and a forget of it strikes the row there. Remembering the same claim with an entity links it and strikes the `memory/unattributed` row in the same write. No placeholder entity page is created (GBRA-190).' },
+ // GBRA-216 PR B (keyless claim-conflict lane). `since` is stamped at the merge slot.
+ { since: '0.60.571.0', id: 'recall_decide_contested_gbra216', text: '`recall` now marks a fact `contested` (proposal_ref `dp<id>`, role, fact_id; the CLI prints `(contested by #<id>)`) when it is in a pending `gbrain decide` supersede proposal and both facts are still active, as it already did for trust proposals. The label appears only when the caller can read the other fact (remote callers see world facts only; private provenance and the trust floor apply). If the lookup fails, recall still answers and reports `facts_degraded: {reason: "contested_unavailable"}`.' },
+ { since: '0.60.571.0', id: 'trust_review_contradictions_gbra216', text: '`gbrain trust review` now lists pending `gbrain decide` proposals whose two facts are both active as kind `contradiction` (`dp<id>`), with supersede, forget, confirm and drop commands and who runs each. `--kind contradiction` filters them, `counts.contradiction` counts them, and `gbrain trust review --accept-all` never decides them: each one appears under `skipped` with reason `contradiction`.' },
+ // GBRA-217 valid time (#7804, GBRA-286 lane D). since is stamped at the merge slot.
+ { since: '0.60.571.0', id: 'valid_time_plain_read', text: 'A plain get_page (and `gbrain get`) of a page whose own `valid_from` is after today, or whose `valid_until` has passed, now carries an additive `valid_time` block (`status: not_yet_in_force` or `not_in_force`, `as_of` today, both dates); `gbrain get` also prints it on stderr. The page text is returned as before, and a page without those dates reads exactly as before. New: get_page `as_of: YYYY-MM-DD` returns the edition in force on that date (docs/guides/page-history.md#valid-time).' },
+ { since: '0.60.571.0', id: 'extraction_pending_review_lists', text: '`extraction_pending` (CLI `gbrain extraction-pending`) now also returns `unattributed`: active facts saved without an entity, newest first (world facts only for remote callers), each with the relink action (remember the same claim with `entity` and `replaces`). `pending` and `count` are unchanged; the result adds `include` and `unattributed`. Pass `include` to choose lists, and add `possible_duplicates` to it to list entity pages created in the last 30 days whose name is one edit or one acronym letter away from another entity, for review only (nothing is merged).' },
+ { since: '0.60.571.0', id: 'extraction_review_page_door', text: '`extraction_review` (CLI `gbrain extraction-review`) promote and reject now write through the page door instead of changing the database row alone: on a source with a repository the decision is committed to Git (promote rewrites the stub\'s file with `status: verified`, reject removes the file), and on a repo-less source the database is the record. Each decision re-reads the page under a source-scoped guard and writes with the revision it read, so a page that changed in between reports `changed` (run the command again), and two decisions on one stub never both land. The result adds `write_requests` (request id and state per written page). A new `distinct` action (`gbrain extraction-review distinct --slugs <new> --of <existing>`, owner-only) records `distinct_from` the same way, and `possible_duplicates` rows carry it.' },
+ // GBRA-219 memory writeback settings over MCP (#7795). since stamped at the merge slot.
+ { since: '0.60.571.0', id: 'memory_conversation_mode_gbra219', text: 'The starter and full MCP surfaces now include `memory_conversation_mode` (any connection that can call `remember`; it returns the saving rule for the rest of the conversation and stores nothing), and the MCP instructions gain one line after the governing writeback rule telling the agent to call it when the user asks to save everything from here or to stop saving. `get_memory_settings` (read) and `set_memory_writeback` (admin scope, only after `gbrain config set mcp.publish_memory_settings true`, off by default) are new. The new `memory.agent_writeback` key changes only what agents are told; the host capture lanes still follow `memory.auto_writeback`.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --cc src/core/import-file.ts
index b27a8a0949,1bb213f5c1..99f2f4c762
--- a/src/core/import-file.ts
+++ b/src/core/import-file.ts
@@@ -562,385 -520,430 +521,430 @@@ export async function importFromContent
compiled_truth: parsed.compiled_truth,
timeline: parsed.timeline,
frontmatter: parsed.frontmatter,
+ tags: parsed.tags,
});
- const unchangedAs = async (page: ParsedPage) => !!existing && !existing.deleted_at && !opts.forceRechunk && !needsProjectionRebuild && !embedSkipChanged
- && !trustMarkerChanged(existing.frontmatter, page.frontmatter) && (existing.content_hash === hash
- ? !opts.prepare || sameCanonicalImport(existingSnapshot, page)
- : !legacyHashMatch && await sameContentAnyKeyOrder(engine, existing, existingSnapshot?.tags ?? null, page, sourceId ?? 'default'));
- const unchanged = await unchangedAs(parsedPage) || (suppliedTrustMarker !== parsed.frontmatter.trust_tier
- && await unchangedAs({ ...parsedPage, frontmatter: suppliedTrustMarkerView(parsed.frontmatter, suppliedTrustMarker) }));
- if (existing && unchanged) {
- // #5050: unchanged content chunked before the safe-chunk fence is re-sealed
- // projection-only; the canonical write stays a no-op.
- const reseal = await projectionBelowSafeFence(engine, existing.id);
- if (opts.prepare) {
- const result: ImportResult = { slug, status: 'skipped', chunks: 0, parsedPage, ...(typeWarning ? { type_warning: typeWarning } : {}), ...fences.fields };
- return opts.prepare({ slug, parsedPage, observedRevision: (existing as typeof existing & { knowledge_revision?: string }).knowledge_revision ?? null,
- noop: true, result, validate: async () => {},
- apply: async tx => { if (reseal) await queuePageProjection(tx, sourceId ?? 'default', slug, 'safe_chunk_reseal'); } });
- }
- await persistUnchanged();
- const resealed = reseal ? await resealSafeChunks(engine, slug, sourceId ?? 'default') : null;
- return { slug, status: 'skipped', chunks: 0, parsedPage, ...(typeWarning ? { type_warning: typeWarning } : {}), ...(resealed ? { resealed } : {}), ...fences.fields };
- }
- if (existing && legacyHashMatch && bodyStorageOf(existing) === 'segmented') throw pageBodySegmentedError(existing, 'legacy hash reconciliation');
- if (existing && legacyHashMatch && fileBodyMatchesStored(content, slug, existing) && await persistUnchanged(true)) {
- const resealed = await projectionBelowSafeFence(engine, existing.id) ? await resealSafeChunks(engine, slug, sourceId ?? 'default') : null;
- return { slug, status: 'skipped', chunks: 0, parsedPage, ...(typeWarning ? { type_warning: typeWarning } : {}), ...(resealed ? { resealed } : {}), ...fences.fields };
- }
+ const parsedPage: ParsedPage = {
+ type: parsed.type,
+ title: parsed.title,
+ compiled_truth: parsed.compiled_truth,
+ timeline: parsed.timeline || '',
+ frontmatter: parsed.frontmatter,
+ tags: parsed.tags,
+ };
- // Identity dedup (#1309): move, skip a true duplicate, or index both.
- // See decideImportIdentity for the posture.
- const fmId = (parsed.frontmatter as Record<string, unknown> | undefined)?.id;
- const fmIdStr = typeof fmId === 'string' && fmId.length > 0 ? fmId : null;
- const identity = opts.forceRechunk ? { kind: 'none' as const } : await decideImportIdentity(engine, {
- sourceId: sourceId ?? 'default', slug, hash, frontmatterId: fmIdStr, sourcePath: opts.sourcePath, sourceRoot: opts.sourceRoot,
- body: { title: parsed.title, compiled_truth: parsed.compiled_truth, timeline: parsed.timeline || '' },
- });
- if (identity.kind === 'move' && !existing && !opts.prepare
- && await maintenanceBeliefTransaction(engine, { sourceId: sourceId ?? 'default', slugs: [identity.dupSlug, slug], op: 'import_move' }, tx => tx.updateSlug(identity.dupSlug, slug, { sourceId: sourceId ?? 'default' })) > 0) {
- process.stderr.write(
- `[import] ${opts.sourcePath} carries frontmatter.id=${fmIdStr} from moved file ${identity.dupSourcePath}; ` +
- `renamed ${identity.dupSlug} -> ${slug} in source ${sourceId ?? 'default'}.\n`
- );
- return importFromContent(engine, slug, content, opts);
- }
- if (identity.kind === 'duplicate') {
- if (opts.prepare) {
- const result: ImportResult = { slug: identity.dupSlug, status: 'skipped', chunks: 0, parsedPage, ...fences.fields };
- return opts.prepare({ slug: identity.dupSlug, parsedPage, observedRevision: (existing as (typeof existing & { knowledge_revision?: string }) | null)?.knowledge_revision ?? null,
- noop: true, result, validate: async () => {}, apply: async () => {} });
+ const originUri = fileOriginUri(existing?.source_uri, opts.sourceRoot, opts.sourcePath);
+ const persistUnchanged = async (refreshBody = false) => {
+ return maintenanceTransaction(engine, async tx => {
+ await assertImportBase(tx, slug, sourceId ?? 'default', existing);
+ if (refreshBody) {
+ const stored = await readStoredBody(tx, sourceId ?? 'default', slug, true);
+ if (!stored || !fileBodyMatchesStored(content, slug, stored)) return false;
+ await tx.refreshPageBody(slug, sourceId ?? 'default', parsed.compiled_truth, parsed.timeline || '', hash);
+ }
+ await refreshSourcePath(tx, slug, sourceId, opts.sourcePath, existing, originUri);
+ if (opts.beforeCommit) await verifyPageReadable(tx, slug, refreshBody ? hash : existing?.content_hash ?? hash, sourceId, 'importFromContent');
+ await opts.beforeCommit?.(tx, slug);
+ return true;
+ });
+ };
+
+ // Rebuild stale projections without granting --force-rechunk's external-ID dedup override.
+ const needsProjectionRebuild = !opts.prepare && existing && existing.text_projection_revision !== existing.knowledge_revision;
+ const embedSkipChanged = !!existing && isEmbedSkipped(existing.frontmatter) !== isEmbedSkipped(parsed.frontmatter);
+ // #3694 one-time reconcile: a row written by the PRE-fix putPage formula
+ // carries the legacy hash. When the parsed file matches that legacy hash,
+ // the content is unchanged — stamp the canonical hash via the narrow
+ // refreshPageBody UPDATE (no chunk churn, no re-embed, no version snapshot)
+ // and skip. The next import then hits the fast path below.
+ const legacyHashMatch = !!existing && !existing.deleted_at && !opts.prepare && !opts.forceRechunk && !needsProjectionRebuild && !embedSkipChanged
+ && typeof engine.refreshPageBody === 'function' && existing.content_hash === contentHashLegacy({
+ title: parsed.title,
+ type: parsed.type,
+ compiled_truth: parsed.compiled_truth,
+ timeline: parsed.timeline,
+ frontmatter: parsed.frontmatter,
+ });
+ const unchangedAs = async (page: ParsedPage) => !!existing && !existing.deleted_at && !opts.forceRechunk && !needsProjectionRebuild && !embedSkipChanged
+ && !trustMarkerChanged(existing.frontmatter, page.frontmatter) && (existing.content_hash === hash
+ ? !opts.prepare || sameCanonicalImport(existingSnapshot, page)
+ : !legacyHashMatch && await sameContentAnyKeyOrder(engine, existing, existingSnapshot?.tags ?? null, page, sourceId ?? 'default'));
+ const unchanged = await unchangedAs(parsedPage) || (suppliedTrustMarker !== parsed.frontmatter.trust_tier
+ && await unchangedAs({ ...parsedPage, frontmatter: suppliedTrustMarkerView(parsed.frontmatter, suppliedTrustMarker) }));
+ if (existing && unchanged) {
+ // #5050: unchanged content chunked before the safe-chunk fence is re-sealed
+ // projection-only; the canonical write stays a no-op.
+ const reseal = await projectionBelowSafeFence(engine, existing.id);
+ if (opts.prepare) {
+ const result: ImportResult = { slug, status: 'skipped', chunks: 0, parsedPage, ...(typeWarning ? { type_warning: typeWarning } : {}), ...fences.fields };
+ return opts.prepare({ slug, parsedPage, observedRevision: (existing as typeof existing & { knowledge_revision?: string }).knowledge_revision ?? null,
+ noop: true, result, validate: async () => {},
+ apply: async tx => { if (reseal) await queuePageProjection(tx, sourceId ?? 'default', slug, 'safe_chunk_reseal'); } });
+ }
+ await persistUnchanged();
+ const resealed = reseal ? await resealSafeChunks(engine, slug, sourceId ?? 'default') : null;
+ return { slug, status: 'skipped', chunks: 0, parsedPage, ...(typeWarning ? { type_warning: typeWarning } : {}), ...(resealed ? { resealed } : {}), ...fences.fields };
}
- process.stderr.write(
- `[import] skipping ${opts.sourcePath ?? slug}: identical to ${identity.dupSlug} ` +
- `(frontmatter.id=${fmIdStr}) in source ${sourceId ?? 'default'}. ` +
- `Pass --force-rechunk to override.\n`
- );
- return { slug: identity.dupSlug, status: 'skipped', chunks: 0, parsedPage, ...fences.fields };
- }
- if (identity.kind === 'shared_id') {
- process.stderr.write(
- `[import] WARNING: ${opts.sourcePath ?? slug} shares frontmatter.id=${fmIdStr} with ${identity.dupSlug} ` +
- `but its content differs. Indexing both.\n`
- );
- } else if (identity.kind === 'shared_hash') {
- process.stderr.write(
- `[import] WARNING: ${opts.sourcePath ?? slug} shares content_hash with ${identity.dupSlug} ` +
- `(${hash.slice(0, 8)}) but not its frontmatter.id. Indexing both.\n`
- );
- }
- // Preserve the importer projection (including fenced code and zero-chunk
- // dispositions) in the provider-free path used by background rebuilds too.
- const bodyBytes = Buffer.byteLength(parsed.compiled_truth, 'utf-8') + Buffer.byteLength(parsed.timeline ?? '', 'utf-8');
- const storage = await stringWriteStorage(engine, bodyBytes, bodyStorageOf(existing));
- const chunks = storage.storage === 'segmented' ? [] : await prepareFenceAwareMarkdownChunks(engine, sourceId ?? 'default', slug, parsed);
+ if (existing && legacyHashMatch && bodyStorageOf(existing) === 'segmented') throw pageBodySegmentedError(existing, 'legacy hash reconciliation');
+ if (existing && legacyHashMatch && fileBodyMatchesStored(content, slug, existing) && await persistUnchanged(true)) {
+ const resealed = await projectionBelowSafeFence(engine, existing.id) ? await resealSafeChunks(engine, slug, sourceId ?? 'default') : null;
+ return { slug, status: 'skipped', chunks: 0, parsedPage, ...(typeWarning ? { type_warning: typeWarning } : {}), ...(resealed ? { resealed } : {}), ...fences.fields };
+ }
- // Embedding failures propagate unless onPostCommitEmbedding lets the caller
- // report enrichment separately from the already-persisted content.
- //
- // v0.40.3.0 contextual retrieval wrapper (D20-T1 chunk_text separation):
- // - Resolve effective CR mode via the page/source/global override chain.
- // - For title tier (free): build the title-only prefix and wrap chunks
- // inline at embed time. Per-chunk generated synopsis tier is NOT supported
- // on the import path — that's an async backfill via the Minion handler
- // (the cost prompt + 10s grace UX from D3 gates spending; inline import
- // path takes the cheaper title-only treatment for tokenmax pages here
- // and defers per-chunk synopsis to the Minion-driven sweep).
- // - Stored chunk_text stays canonical; only the embedding input is wrapped.
- // - Code chunks (chunk_source='fenced_code') bypass wrapping per D20-T4.
- // Every markdown import records its wrapping convention with the canonical
- // write, --no-embed included (#5621): later embed passes reproduce the stored
- // mode, so an unstamped page would stay unwrapped. No provider runs here.
- // #3885: the real source row applies (a stored `sources set-cr-mode`); the
- // inline path demotes per_chunk_synopsis to the free title tier.
- const { mode: effectiveCRMode, corpusGeneration } = await resolveImportContextualMode(engine, sourceId ?? 'default', parsed.frontmatter, opts.prepare !== undefined);
- // Wave B: a body staged as segments publishes with the generation flip instead of putPage and the chunk replacement.
- const segmented = storage.policy && storage.storage === 'segmented' ? await stageSegmentedImport(engine, { slug, sourceId: sourceId ?? 'default', existing,
- page: parsedPage, policy: storage.policy, owner: opts.segmentedOwner, reuse: opts.reuseSegments, baseRevision: (existing as { knowledge_revision?: string } | null)?.knowledge_revision ?? null,
- chunking: { quarantined: pageQuarantined, corpusGeneration, tier: effectiveCRMode === 'title' ? 'title' : 'none' } }) : null;
-
- // A13: an edit re-embeds only chunks whose embedding input changed. A stored
- // vector is reused for a chunk with the same source and text when its
- // recorded embedding input (#5553: column, model, dimensions, wrapper tier
- // and exact input) equals the one this import would embed, or, for a vector
- // recorded before provenance existed, when the input is the raw chunk text
- // under the same model. The old index must be sealed (no private fence row in a chunk), and on a fenced page
- // only a vector with a recorded input is reused (#6961); a changed fence chunk has new text and is embedded again.
- // A prepared (persistence) import defers embedding to its effect worker, so
- // it reuses inside its publication transaction, under the page guard, from
- // the chunks it is about to replace; only the chunks left without a vector
- // reach the provider.
- // A large or segmented page (the hysteresis band too) commits with NULL vectors; the stale
- // sweep embeds it under the per-page cap and the sampled junk check, so no provider call holds the write.
- const noEmbed = opts.noEmbed || isEmbedSkipped(parsed.frontmatter);
- const largePage = isLargePageBytes(bodyBytes) || segmented !== null;
- const postCommitEmbedding = largePage ? undefined : opts.onPostCommitEmbedding;
- const reusable = !!existing && !existing.deleted_at && !opts.forceRechunk && !postCommitEmbedding && chunks.length > 0 && bodyStorageOf(existing) !== 'segmented'
- && !(opts.prepare && modeRequiresSynopsis(effectiveCRMode));
- const reuseCtx = { slug, sourceId: sourceId ?? 'default', title: parsed.title, corpusGeneration, tier: effectiveCRMode === 'title' ? 'title' as const : 'none' as const, requireRecordedInput: reusable && (hasProtectedBody(`${existing.compiled_truth}\n${existing.timeline ?? ''}`) || hasProtectedBody(`${parsed.compiled_truth}\n${parsed.timeline ?? ''}`)) };
- const reuseStoredVectors = (exec: BrainEngine, into: ChunkInput[], keepUnembedded = false) => reuseStoredChunkVectors(exec, into, { ...reuseCtx, prepared: !!opts.prepare, keepUnembedded });
- const reused = reusable && !noEmbed && !opts.prepare ? await reuseStoredVectors(engine, chunks) : new Map<number, number | null>();
-
- let embeddingPartial: EmbeddingZeroNormError | undefined;
- const embedChunks = async () => {
- const pending = chunks.map((_, i) => i).filter(i => !reused.has(i));
- if (noEmbed || pending.length === 0) return;
- const prefix =
- modeRequiresWrapper(effectiveCRMode) && !modeRequiresSynopsis(effectiveCRMode)
- ? buildContextualPrefix(parsed.title, null)
- : null;
- const wrappedTexts = pending.map(i => prefix ? wrapChunkForEmbedding(chunks[i].chunk_text, prefix, chunks[i].chunk_source) : chunks[i].chunk_text);
- // #4616: store the usable vectors; refused chunks stay NULL for embed --stale.
- const { vectors: embeddings, refused } = await embedBatchKeepingUsable(wrappedTexts);
- embeddingPartial = refused;
- pending.forEach((i, j) => {
- if (!embeddings[j]) return;
- chunks[i].embedding = embeddings[j]!;
- // token_count tracks the wrapped string length so cost reporting
- // reflects what we actually sent to the embedder.
- chunks[i].token_count = Math.ceil(wrappedTexts[j].length / 4);
+ // Identity dedup (#1309): move, skip a true duplicate, or index both.
+ // See decideImportIdentity for the posture.
+ const fmId = (parsed.frontmatter as Record<string, unknown> | undefined)?.id;
+ const fmIdStr = typeof fmId === 'string' && fmId.length > 0 ? fmId : null;
+ const identity = opts.forceRechunk ? { kind: 'none' as const } : await decideImportIdentity(engine, {
+ sourceId: sourceId ?? 'default', slug, hash, frontmatterId: fmIdStr, sourcePath: opts.sourcePath, sourceRoot: opts.sourceRoot,
+ body: { title: parsed.title, compiled_truth: parsed.compiled_truth, timeline: parsed.timeline || '' },
});
- };
- // An embedding outage never blocks the text write: the page and its chunks
- // commit with NULL vectors and no embedding signature, and the stale sweep
- // (`gbrain embed --stale`, sync's embed pass, the cycle) embeds them once
- // the provider recovers.
- let embeddingDeferred = largePage && !noEmbed && (chunks.length > 0 || (segmented !== null && !pageQuarantined));
- if (!opts.onPostCommitEmbedding && !largePage) {
- try {
- await embedChunks();
- if (embeddingPartial) {
- embeddingDeferred = true;
- process.stderr.write(`[import] ${slug}: ${embeddingPartial.message} ${embeddingPartial.suggestionFor(slug, sourceId)}\n`);
- }
- } catch (err) {
- embeddingDeferred = true;
- chunks.forEach((c, i) => { if (!reused.has(i)) { c.embedding = undefined; c.token_count = undefined; } });
+ if (identity.kind === 'move' && !existing && !opts.prepare
+ && await maintenanceBeliefTransaction(engine, { sourceId: sourceId ?? 'default', slugs: [identity.dupSlug, slug], op: 'import_move' }, tx => tx.updateSlug(identity.dupSlug, slug, { sourceId: sourceId ?? 'default' })) > 0) {
process.stderr.write(
- `[import] ${slug}: embedding failed (${err instanceof Error ? err.message : String(err)}); ` +
- `text saved, chunks queued for \`gbrain embed --stale\`.\n`
+ `[import] ${opts.sourcePath} carries frontmatter.id=${fmIdStr} from moved file ${identity.dupSourcePath}; ` +
+ `renamed ${identity.dupSlug} -> ${slug} in source ${sourceId ?? 'default'}.\n`
);
+ return importFromContent(engine, slug, content, opts);
}
- }
-
- // Transaction wraps the canonical DB writes. Every per-page tx call carries the
- // caller's sourceId so writes target (sourceId, slug) rather than the
- // schema DEFAULT — required for multi-source brains; harmless ('default')
- // for single-source callers.
- const txOpts = { sourceId: sourceId ?? 'default' };
- let persistedProjection: ProjectionSnapshot | null = null;
- const [timeZone, mentionPolicy] = await Promise.all([loadBrainTimeZone(engine), readMentionPolicy(engine).catch(() => null)]);
- let chunkReplacementDeferred = false;
- const applyPrepared = async (tx: BrainEngine, preimage?: PageSnapshot | null) => {
- if (!opts.coordinated) await assertImportBase(tx, slug, txOpts.sourceId, existing, reused.size > 0);
- // #5984: the canonical writes go out in two pipelined waves of mutually independent statements; the
- // second depends on the first (the page row and its id, the old chunks gone). A failure rolls back the caller's transaction.
- // v0.29.1 — compute effective_date from frontmatter precedence chain.
- // Filename comes from importFromFile path (basename) or the slug tail
- // (put_page MCP op fallback). An undated page falls back to a stable
- // anchor (fallbackCreatedAt): its stored fallback date, else its file's
- // timestamp, else now — never the latest write. The result drives the recency boost and since/until
- // filters when callers opt in; nothing in the default search path
- // consults it.
- const filenameForChain = opts.filename ?? slug.split('/').pop() ?? slug;
- const nowDate = new Date();
- const { date: effectiveDate, source: effectiveDateSource } = computeEffectiveDate({
- slug,
- frontmatter: parsed.frontmatter,
- filename: filenameForChain,
- timeZone,
- updatedAt: existing?.updated_at ?? nowDate,
- createdAt: fallbackCreatedAt({ existing, fileTimes: opts.fileTimes, now: nowDate }),
- });
-
- const demote = !segmented && bodyStorageOf(existing) === 'segmented';
- // Before the page write: an unpinned version reads the page itself, a demoted one references the generation it retires.
- if (existing && (!preimage || demote)) await tx.createVersion(slug, preimage ? { ...txOpts, ...opts.history, preimage } : { ...txOpts, ...opts.history });
- if (demote) await demoteSegmentedPage(tx, Number(existing!.id));
- // Every import keeps each stored row that is identical to its new chunk and
- // writes only the rest, so an edit rewrites only the chunks it changed. The
- // reuse is re-read here, under the page lock, so a kept row is one this
- // transaction sees; --no-embed and deferred-embed imports keep vectors too.
- const fresh = reusable ? chunks.map(chunk => ({ ...chunk })) : chunks;
- const kept = new Map([...(fresh !== chunks ? await reuseStoredVectors(tx, fresh, true) : [])]
- .filter((entry): entry is [number, KeptChunkRow] => entry[1] !== null));
- const keptIds = [...kept.values()].map(row => row.id);
- // #7172: a kept row whose chunk moved changes index in place; a managed publication over CHUNK_VECTOR_WRITE_BUDGET
- // vector-index writes deletes the rows it does not keep, commits the page pending and finishes after commit (chunk-moves.ts).
- const chunkWrites = planImportChunkWrites(fresh, kept, !segmented && !!opts.prepare && opts.boundedChunkWrites === true);
- const deferChunks = chunkReplacementDeferred = chunkWrites.defer;
- let written: Page | undefined;
- await pipelined(tx, [
- () => assertPreparedFactWithdrawals(tx, txOpts.sourceId, parsed.compiled_truth, parsed.timeline || '', slug),
- async () => { if (existing && preimage && !demote) await tx.createVersion(slug, { ...txOpts, ...opts.history, preimage }); },
- async () => {
- if (segmented) return segmented.apply(tx, { type: parsed.type, title: parsed.title, frontmatter: parsed.frontmatter, page_kind: 'markdown',
- effective_date: effectiveDate, effective_date_source: effectiveDateSource, import_filename: filenameForChain, source_path: opts.sourcePath ?? null,
- source_kind: opts.source_kind ?? null, source_uri: opts.source_uri ?? originUri, ingested_via: opts.ingested_via ?? null }, { mode: effectiveCRMode, corpusGeneration });
- written = await tx.putPage(slug, {
- type: parsed.type,
- title: parsed.title,
- compiled_truth: parsed.compiled_truth,
- timeline: parsed.timeline || '',
- frontmatter: parsed.frontmatter,
- content_hash: hash,
- effective_date: effectiveDate,
- effective_date_source: effectiveDateSource,
- import_filename: filenameForChain,
- // Preserve the repo-relative path for sync's delete/rename handling.
- source_path: opts.sourcePath ?? null,
- // v0.39.3.0 provenance write-through (WARN-8). Engine layer applies
- // COALESCE-preserve UPDATE so omitting these on a later put_page
- // doesn't erase the original ingestion's audit trail.
- source_kind: opts.source_kind ?? null,
- source_uri: opts.source_uri ?? originUri,
- ingested_via: opts.ingested_via ?? null,
- // ingested_at is server-stamped at the engine layer when any
- // provenance write fires; never client-controlled.
- // Empty-overwrite escape hatch only when the caller vouched (file
- // import / explicit allow_empty); otherwise the engine guard stays on.
- }, { ...txOpts, ...(opts.allowEmptyOverwrite === true ? { allowEmptyOverwrite: true } : {}), inline: true,
- ...(effectiveCRMode !== 'none' ? { contextualRetrieval: { mode: effectiveCRMode, corpusGeneration } } : {}) });
- // v0.40.3.0: the contextual retrieval state is stamped with the page write (the `none` tier's
- // stamp, which also clears tier vectors, follows it). Deferred embedders reproduce it; every
- // chunk row is replaced below. This stamp certifies no vector: embedding_signature stays deferred.
- if (effectiveCRMode === 'none') await tx.updatePageContextualRetrievalState(slug, sourceId ?? 'default', effectiveCRMode, corpusGeneration);
- },
- // Replace every derived row atomically. Only vectors the A13 reuse gate above admitted carry over; a new
- // seal otherwise inherits nothing from an older index, whose contextual vector may have included a private sibling.
- async () => segmented ? undefined : keptIds.length ? tx.executeRaw(`DELETE FROM content_chunks WHERE page_id = (SELECT id FROM pages WHERE slug = $1 AND source_id = $2)
- AND NOT (id = ANY($3::int[]))`, [slug, txOpts.sourceId, keptIds]) : tx.deleteChunks(slug, txOpts),
- // Alias projection and readback share the page commit. A later writer can
- // no longer turn a successful import into a postcommit verification error.
- () => writePageAliases(tx, slug, sourceId ?? 'default', parsed, opts.activePack, mentionPolicy),
- ]);
- const pageId = segmented?.applied ? segmented.applied.pageId : written && written.deleted_at == null ? written.id : undefined;
- const deferSeal = opts.coordinated === true && pageId !== undefined && !segmented;
- await pipelined(tx, [
- async () => {
- // Tag reconciliation (A14): frontmatter tags carry tag_source='frontmatter'.
- // A frontmatter-owned row whose tag left the frontmatter is deleted; rows
- // explicit adds own ('added': add_tag, enrichment, the code importer) and
- // legacy NULL rows are never deleted here (#1621: reindex must not wipe
- // enrichment). A legacy row still in the frontmatter is adopted. Prepared
- // (managed) imports keep the add-only union their canonical file renders.
- if (!opts.prepare) await tx.executeRaw(`DELETE FROM tags t USING pages p WHERE p.id = t.page_id AND p.source_id = $1 AND p.slug = $2
- AND t.tag_source = 'frontmatter' AND NOT (t.tag = ANY($3::text[]))`, [txOpts.sourceId, slug, parsed.tags]);
- if (pageId !== undefined) return addPageTags(tx, Number(pageId), parsed.tags, opts.prepare ? undefined : 'frontmatter');
- for (const tag of parsed.tags) await tx.addTag(slug, tag, opts.prepare ? txOpts : { ...txOpts, tagSource: 'frontmatter' });
- },
- async () => {
- if (deferChunks) return;
- if (chunks.length > 0) {
- await applyMoveWaves(tx, Number(pageId), chunkWrites.waves);
- const embeddingColumn = await stampEmbeddingInputs(tx, fresh, null,
- { title: parsed.title, tier: effectiveCRMode === 'title' ? 'title' : 'none', corpusGeneration });
- await tx.upsertChunks(slug, kept.size ? fresh.filter((_, i) => !kept.has(i)) : fresh, { ...txOpts, ...(embeddingColumn ? { embeddingColumn } : {}), sealChunkerVersion: MARKDOWN_CHUNKER_VERSION,
- ...(pageId !== undefined ? { pageId: Number(pageId) } : {}), ...(deferSeal ? { deferSeal: true as const } : {}) });
- // v0.41.31: stamp embedding provenance when this import actually
- // embedded (not --no-embed), so a later model/dims swap is detectable
- // as stale via embed --stale. The deferred/backfill + per-slug embed
- // paths stamp too; this covers the inline import/sync path.
- if (!noEmbed && !postCommitEmbedding && !embeddingDeferred) {
- // D9: signature is null when the gateway is unconfigured — skip the
- // stamp (a wrong signature is worse than none).
- const importSig = currentEmbeddingSignature();
- if (importSig) await tx.setPageEmbeddingSignature(slug, { sourceId, signature: importSig });
- }
- } else {
- // Seal only the completed, full-body sanitized replacement (the upsert above seals its own); a
- // body-only write or a failed transaction must never certify old stored fragments.
- if (!deferSeal && !segmented) await tx.executeRaw('UPDATE pages SET chunker_version = $1 WHERE source_id = $2 AND slug = $3', [MARKDOWN_CHUNKER_VERSION, txOpts.sourceId, slug]);
- }
- },
- ]);
- if (!opts.coordinated && !segmented && !deferChunks) await sealPageTextProjection(tx, slug, txOpts.sourceId);
- if (!opts.coordinated && segmented?.applied?.complete) await sealSegmentedPage(tx, segmented.applied.pageId);
-
- // v0.19.0 E1 — doc↔impl linking: if this markdown page cites code paths
- // (e.g. 'src/core/sync.ts:42'), create bidirectional edges to the code
- // page, all in one statement (addCodeReferenceLinks). An edge whose code
- // page is missing is skipped — guides imported before their code repo
- // syncs are common, and the missing edges land later via
- // `gbrain reconcile-links` (Layer 8 D3, v0.21.0).
- const codeRefs = extractCodeRefs(sanitizeRemoteBody(parsed.compiled_truth) + '\n' + sanitizeRemoteBody(parsed.timeline || ''));
- // For doc↔impl edges, both endpoints are within the same source as the
- // markdown page being imported. Cross-source edges (markdown in one
- // source, code in another) are skipped as missing — never the pre-fix
- // cross-product fan-out, which silently wired edges to whichever
- // same-slug page Postgres returned first across sources.
- if (codeRefs.length) {
- await tx.transaction(savepoint => addCodeReferenceLinks(savepoint, slug, sourceId || 'default',
- codeRefs.map(ref => ({ codeSlug: slugifyCodePath(ref.path), path: ref.path, line: ref.line })))).catch(() => undefined);
+ if (identity.kind === 'duplicate') {
+ if (opts.prepare) {
+ const result: ImportResult = { slug: identity.dupSlug, status: 'skipped', chunks: 0, parsedPage, ...fences.fields };
+ return opts.prepare({ slug: identity.dupSlug, parsedPage, observedRevision: (existing as (typeof existing & { knowledge_revision?: string }) | null)?.knowledge_revision ?? null,
+ noop: true, result, validate: async () => {}, apply: async () => {} });
+ }
+ process.stderr.write(
+ `[import] skipping ${opts.sourcePath ?? slug}: identical to ${identity.dupSlug} ` +
+ `(frontmatter.id=${fmIdStr}) in source ${sourceId ?? 'default'}. ` +
+ `Pass --force-rechunk to override.\n`
+ );
+ return { slug: identity.dupSlug, status: 'skipped', chunks: 0, parsedPage, ...fences.fields };
}
- if (!opts.coordinated) await verifyPageReadable(tx, slug, hash, sourceId, 'importFromContent');
- await opts.beforeCommit?.(tx, slug);
- await disposition.persistReceipt(tx, { existed: !!existing });
- if (postCommitEmbedding && !noEmbed && chunks.length > 0) {
- // Capture the complete installed projection before releasing the page
- // guard. Deferred provider results cannot replace newer text or chunks.
- persistedProjection = await readProjectionSnapshot(tx, slug, txOpts.sourceId);
+ if (identity.kind === 'shared_id') {
+ process.stderr.write(
+ `[import] WARNING: ${opts.sourcePath ?? slug} shares frontmatter.id=${fmIdStr} with ${identity.dupSlug} ` +
+ `but its content differs. Indexing both.\n`
+ );
+ } else if (identity.kind === 'shared_hash') {
+ process.stderr.write(
+ `[import] WARNING: ${opts.sourcePath ?? slug} shares content_hash with ${identity.dupSlug} ` +
+ `(${hash.slice(0, 8)}) but not its frontmatter.id. Indexing both.\n`
+ );
}
- if (segmented?.applied) return { pageId, sealed: !opts.coordinated && segmented.applied.complete, segmented: { complete: segmented.applied.complete } };
- // The reason routes a resident or drain rebuild of this revision to the bounded continuation.
- if (deferChunks) await queuePageProjection(tx, txOpts.sourceId, slug, DEFERRED_CHUNK_REPLACEMENT);
- if (deferChunks) return { pageId, sealed: false, projectionPending: true as const };
- return { pageId, sealed: !opts.coordinated && !opts.beforeCommit, ...(deferSeal ? { chunkerSeal: MARKDOWN_CHUNKER_VERSION } : {}) };
- };
- if (opts.prepare) return opts.prepare({
- slug, parsedPage, observedRevision: (existing as (typeof existing & { knowledge_revision?: string }) | null)?.knowledge_revision ?? null,
- noop: false, contentHash: hash, result: { slug, status: 'imported', chunks: segmented ? segmented.prepared.rows.chunks : chunks.length, parsedPage,
- ...(pageQuarantined ? { quarantined: true } : {}), ...(pageFlagged ? { flagged: true, flag_reason: pageFlagReason } : {}), ...(disposition.gate ? { gate: disposition.gate } : {}), ...fences.fields },
- validate: tx => assertPreparedFactWithdrawals(tx, txOpts.sourceId, parsed.compiled_truth, parsed.timeline || '', slug),
- apply: applyPrepared, ...(segmented ? { afterCommit: async (target: BrainEngine) => { await segmented.afterCommit(target); }, segmented }
- : opts.boundedChunkWrites ? { afterCommit: async (target: BrainEngine) => { if (chunkReplacementDeferred) await continueDeferredProjection(target, txOpts.sourceId, slug); } } : {}),
- });
- await withUnmanagedBeliefs(engine, txOpts.sourceId, [slug], 'import', run => maintenanceTransaction(engine, tx => run(tx, () => applyPrepared(tx)), writeTrustOfGate(opts.writeGate)).catch(async (err: unknown) => {
- await abortSegmentedStaging(engine, segmented?.prepared);
- // #4287: name the dimension-mismatch rollback instead of letting the bare
- // pgvector message ("expected N dimensions, not M") surface with no code,
- // no consequence and no fix. S2: name the registry-ACTIVE column the
- // write actually targeted (best-effort — plane-agnostic wording when the
- // registry itself is unreadable).
- let activeColName: string | undefined;
- const msg = err instanceof Error ? err.message : String(err);
- if (/expected \d+ dimensions, not \d+/.test(msg)) {
+
+ // Preserve the importer projection (including fenced code and zero-chunk
+ // dispositions) in the provider-free path used by background rebuilds too.
+ const bodyBytes = Buffer.byteLength(parsed.compiled_truth, 'utf-8') + Buffer.byteLength(parsed.timeline ?? '', 'utf-8');
+ const storage = await stringWriteStorage(engine, bodyBytes, bodyStorageOf(existing));
+ const chunks = storage.storage === 'segmented' ? [] : await prepareFenceAwareMarkdownChunks(engine, sourceId ?? 'default', slug, parsed);
+
+ // Embedding failures propagate unless onPostCommitEmbedding lets the caller
+ // report enrichment separately from the already-persisted content.
+ //
+ // v0.40.3.0 contextual retrieval wrapper (D20-T1 chunk_text separation):
+ // - Resolve effective CR mode via the page/source/global override chain.
+ // - For title tier (free): build the title-only prefix and wrap chunks
+ // inline at embed time. Per-chunk generated synopsis tier is NOT supported
+ // on the import path — that's an async backfill via the Minion handler
+ // (the cost prompt + 10s grace UX from D3 gates spending; inline import
+ // path takes the cheaper title-only treatment for tokenmax pages here
+ // and defers per-chunk synopsis to the Minion-driven sweep).
+ // - Stored chunk_text stays canonical; only the embedding input is wrapped.
+ // - Code chunks (chunk_source='fenced_code') bypass wrapping per D20-T4.
+ // Every markdown import records its wrapping convention with the canonical
+ // write, --no-embed included (#5621): later embed passes reproduce the stored
+ // mode, so an unstamped page would stay unwrapped. No provider runs here.
+ // #3885: the real source row applies (a stored `sources set-cr-mode`); the
+ // inline path demotes per_chunk_synopsis to the free title tier.
+ const { mode: effectiveCRMode, corpusGeneration } = await resolveImportContextualMode(engine, sourceId ?? 'default', parsed.frontmatter, opts.prepare !== undefined);
+ // Wave B: a body staged as segments publishes with the generation flip instead of putPage and the chunk replacement.
+ const segmented = storage.policy && storage.storage === 'segmented' ? await stageSegmentedImport(engine, { slug, sourceId: sourceId ?? 'default', existing,
+ page: parsedPage, policy: storage.policy, owner: opts.segmentedOwner, reuse: opts.reuseSegments, baseRevision: (existing as { knowledge_revision?: string } | null)?.knowledge_revision ?? null,
+ chunking: { quarantined: pageQuarantined, corpusGeneration, tier: effectiveCRMode === 'title' ? 'title' : 'none' } }) : null;
+
+ // A13: an edit re-embeds only chunks whose embedding input changed. A stored
+ // vector is reused for a chunk with the same source and text when its
+ // recorded embedding input (#5553: column, model, dimensions, wrapper tier
+ // and exact input) equals the one this import would embed, or, for a vector
+ // recorded before provenance existed, when the input is the raw chunk text
+ // under the same model. The old index must be sealed (no private fence row in a chunk), and on a fenced page
+ // only a vector with a recorded input is reused (#6961); a changed fence chunk has new text and is embedded again.
+ // A prepared (persistence) import defers embedding to its effect worker, so
+ // it reuses inside its publication transaction, under the page guard, from
+ // the chunks it is about to replace; only the chunks left without a vector
+ // reach the provider.
+ // A large or segmented page (the hysteresis band too) commits with NULL vectors; the stale
+ // sweep embeds it under the per-page cap and the sampled junk check, so no provider call holds the write.
+ const noEmbed = opts.noEmbed || isEmbedSkipped(parsed.frontmatter);
+ const largePage = isLargePageBytes(bodyBytes) || segmented !== null;
+ const postCommitEmbedding = largePage ? undefined : opts.onPostCommitEmbedding;
+ const reusable = !!existing && !existing.deleted_at && !opts.forceRechunk && !postCommitEmbedding && chunks.length > 0 && bodyStorageOf(existing) !== 'segmented'
+ && !(opts.prepare && modeRequiresSynopsis(effectiveCRMode));
+ const reuseCtx = { slug, sourceId: sourceId ?? 'default', title: parsed.title, corpusGeneration, tier: effectiveCRMode === 'title' ? 'title' as const : 'none' as const, requireRecordedInput: reusable && (hasProtectedBody(`${existing.compiled_truth}\n${existing.timeline ?? ''}`) || hasProtectedBody(`${parsed.compiled_truth}\n${parsed.timeline ?? ''}`)) };
+ const reuseStoredVectors = (exec: BrainEngine, into: ChunkInput[], keepUnembedded = false) => reuseStoredChunkVectors(exec, into, { ...reuseCtx, prepared: !!opts.prepare, keepUnembedded });
+ const reused = reusable && !noEmbed && !opts.prepare ? await reuseStoredVectors(engine, chunks) : new Map<number, number | null>();
+
+ let embeddingPartial: EmbeddingZeroNormError | undefined;
+ const embedChunks = async () => {
+ const pending = chunks.map((_, i) => i).filter(i => !reused.has(i));
+ if (noEmbed || pending.length === 0) return;
+ const prefix =
+ modeRequiresWrapper(effectiveCRMode) && !modeRequiresSynopsis(effectiveCRMode)
+ ? buildContextualPrefix(parsed.title, null)
+ : null;
+ const wrappedTexts = pending.map(i => prefix ? wrapChunkForEmbedding(chunks[i].chunk_text, prefix, chunks[i].chunk_source) : chunks[i].chunk_text);
+ // #4616: store the usable vectors; refused chunks stay NULL for embed --stale.
+ const { vectors: embeddings, refused } = await embedBatchKeepingUsable(wrappedTexts);
+ embeddingPartial = refused;
+ pending.forEach((i, j) => {
+ if (!embeddings[j]) return;
+ chunks[i].embedding = embeddings[j]!;
+ // token_count tracks the wrapped string length so cost reporting
+ // reflects what we actually sent to the embedder.
+ chunks[i].token_count = Math.ceil(wrappedTexts[j].length / 4);
+ });
+ };
+ // An embedding outage never blocks the text write: the page and its chunks
+ // commit with NULL vectors and no embedding signature, and the stale sweep
+ // (`gbrain embed --stale`, sync's embed pass, the cycle) embeds them once
+ // the provider recovers.
+ let embeddingDeferred = largePage && !noEmbed && (chunks.length > 0 || (segmented !== null && !pageQuarantined));
+ if (!opts.onPostCommitEmbedding && !largePage) {
try {
- const { resolveActiveEmbeddingColumnFromEngine } = await import('./search/embedding-column.ts');
- activeColName = (await resolveActiveEmbeddingColumnFromEngine(engine, { fallbackToLegacy: true })).name;
- } catch { /* keep the plane-agnostic wording */ }
+ await embedChunks();
+ if (embeddingPartial) {
+ embeddingDeferred = true;
+ process.stderr.write(`[import] ${slug}: ${embeddingPartial.message} ${embeddingPartial.suggestionFor(slug, sourceId)}\n`);
+ }
+ } catch (err) {
+ embeddingDeferred = true;
+ chunks.forEach((c, i) => { if (!reused.has(i)) { c.embedding = undefined; c.token_count = undefined; } });
+ process.stderr.write(
+ `[import] ${slug}: embedding failed (${err instanceof Error ? err.message : String(err)}); ` +
+ `text saved, chunks queued for \`gbrain embed --stale\`.\n`
+ );
+ }
}
- throw decorateEmbeddingDimError(err, slug, activeColName);
- }));
- const searchable = await segmented?.afterCommit(engine);
+ // Transaction wraps the canonical DB writes. Every per-page tx call carries the
+ // caller's sourceId so writes target (sourceId, slug) rather than the
+ // schema DEFAULT — required for multi-source brains; harmless ('default')
+ // for single-source callers.
+ const txOpts = { sourceId: sourceId ?? 'default' };
+ let persistedProjection: ProjectionSnapshot | null = null;
+ const [timeZone, mentionPolicy] = await Promise.all([loadBrainTimeZone(engine), readMentionPolicy(engine).catch(() => null)]);
+ let chunkReplacementDeferred = false;
+ const applyPrepared = async (tx: BrainEngine, preimage?: PageSnapshot | null) => {
+ if (!opts.coordinated) await assertImportBase(tx, slug, txOpts.sourceId, existing, reused.size > 0);
+ // #5984: the canonical writes go out in two pipelined waves of mutually independent statements; the
+ // second depends on the first (the page row and its id, the old chunks gone). A failure rolls back the caller's transaction.
+ // v0.29.1 — compute effective_date from frontmatter precedence chain.
+ // Filename comes from importFromFile path (basename) or the slug tail
+ // (put_page MCP op fallback). An undated page falls back to a stable
+ // anchor (fallbackCreatedAt): its stored fallback date, else its file's
+ // timestamp, else now — never the latest write. The result drives the recency boost and since/until
+ // filters when callers opt in; nothing in the default search path
+ // consults it.
+ const filenameForChain = opts.filename ?? slug.split('/').pop() ?? slug;
+ const nowDate = new Date();
+ const { date: effectiveDate, source: effectiveDateSource } = computeEffectiveDate({
+ slug,
+ frontmatter: parsed.frontmatter,
+ filename: filenameForChain,
+ timeZone,
+ updatedAt: existing?.updated_at ?? nowDate,
+ createdAt: fallbackCreatedAt({ existing, fileTimes: opts.fileTimes, now: nowDate }),
+ });
- if (postCommitEmbedding && !noEmbed && chunks.length > 0) {
- postCommitEmbedding(async () => {
- try {
- if (!persistedProjection) return { status: 'superseded' };
- const signature = currentEmbeddingSignature();
- await embedChunks();
- const installed = await installPageEmbeddings(engine, persistedProjection, chunks, embeddingPartial ? undefined : signature ?? undefined);
- return { status: installed.complete ? 'embedded' : 'superseded' };
- } catch {
- return { status: 'failed', error: 'Page content was saved, but embedding failed. Check the embedding provider and database on the brain host, then run gbrain embed --stale --source <source-id>.' };
+ const demote = !segmented && bodyStorageOf(existing) === 'segmented';
+ // Before the page write: an unpinned version reads the page itself, a demoted one references the generation it retires.
- if (existing && (!preimage || demote)) await tx.createVersion(slug, preimage ? { ...txOpts, preimage } : txOpts);
++ if (existing && (!preimage || demote)) await tx.createVersion(slug, preimage ? { ...txOpts, ...opts.history, preimage } : { ...txOpts, ...opts.history });
+ if (demote) await demoteSegmentedPage(tx, Number(existing!.id));
+ // Every import keeps each stored row that is identical to its new chunk and
+ // writes only the rest, so an edit rewrites only the chunks it changed. The
+ // reuse is re-read here, under the page lock, so a kept row is one this
+ // transaction sees; --no-embed and deferred-embed imports keep vectors too.
+ const fresh = reusable ? chunks.map(chunk => ({ ...chunk })) : chunks;
+ const kept = new Map([...(fresh !== chunks ? await reuseStoredVectors(tx, fresh, true) : [])]
+ .filter((entry): entry is [number, KeptChunkRow] => entry[1] !== null));
+ const keptIds = [...kept.values()].map(row => row.id);
+ // #7172: a kept row whose chunk moved changes index in place; a managed publication over CHUNK_VECTOR_WRITE_BUDGET
+ // vector-index writes deletes the rows it does not keep, commits the page pending and finishes after commit (chunk-moves.ts).
+ const chunkWrites = planImportChunkWrites(fresh, kept, !segmented && !!opts.prepare && opts.boundedChunkWrites === true);
+ const deferChunks = chunkReplacementDeferred = chunkWrites.defer;
+ let written: Page | undefined;
+ await pipelined(tx, [
+ () => assertPreparedFactWithdrawals(tx, txOpts.sourceId, parsed.compiled_truth, parsed.timeline || '', slug),
- async () => { if (existing && preimage && !demote) await tx.createVersion(slug, { ...txOpts, preimage }); },
++ async () => { if (existing && preimage && !demote) await tx.createVersion(slug, { ...txOpts, ...opts.history, preimage }); },
+ async () => {
+ if (segmented) return segmented.apply(tx, { type: parsed.type, title: parsed.title, frontmatter: parsed.frontmatter, page_kind: 'markdown',
+ effective_date: effectiveDate, effective_date_source: effectiveDateSource, import_filename: filenameForChain, source_path: opts.sourcePath ?? null,
+ source_kind: opts.source_kind ?? null, source_uri: opts.source_uri ?? originUri, ingested_via: opts.ingested_via ?? null }, { mode: effectiveCRMode, corpusGeneration });
+ written = await tx.putPage(slug, {
+ type: parsed.type,
+ title: parsed.title,
+ compiled_truth: parsed.compiled_truth,
+ timeline: parsed.timeline || '',
+ frontmatter: parsed.frontmatter,
+ content_hash: hash,
+ effective_date: effectiveDate,
+ effective_date_source: effectiveDateSource,
+ import_filename: filenameForChain,
+ // Preserve the repo-relative path for sync's delete/rename handling.
+ source_path: opts.sourcePath ?? null,
+ // v0.39.3.0 provenance write-through (WARN-8). Engine layer applies
+ // COALESCE-preserve UPDATE so omitting these on a later put_page
+ // doesn't erase the original ingestion's audit trail.
+ source_kind: opts.source_kind ?? null,
+ source_uri: opts.source_uri ?? originUri,
+ ingested_via: opts.ingested_via ?? null,
+ // ingested_at is server-stamped at the engine layer when any
+ // provenance write fires; never client-controlled.
+ // Empty-overwrite escape hatch only when the caller vouched (file
+ // import / explicit allow_empty); otherwise the engine guard stays on.
+ }, { ...txOpts, ...(opts.allowEmptyOverwrite === true ? { allowEmptyOverwrite: true } : {}), inline: true,
+ ...(effectiveCRMode !== 'none' ? { contextualRetrieval: { mode: effectiveCRMode, corpusGeneration } } : {}) });
+ // v0.40.3.0: the contextual retrieval state is stamped with the page write (the `none` tier's
+ // stamp, which also clears tier vectors, follows it). Deferred embedders reproduce it; every
+ // chunk row is replaced below. This stamp certifies no vector: embedding_signature stays deferred.
+ if (effectiveCRMode === 'none') await tx.updatePageContextualRetrievalState(slug, sourceId ?? 'default', effectiveCRMode, corpusGeneration);
+ },
+ // Replace every derived row atomically. Only vectors the A13 reuse gate above admitted carry over; a new
+ // seal otherwise inherits nothing from an older index, whose contextual vector may have included a private sibling.
+ async () => segmented ? undefined : keptIds.length ? tx.executeRaw(`DELETE FROM content_chunks WHERE page_id = (SELECT id FROM pages WHERE slug = $1 AND source_id = $2)
+ AND NOT (id = ANY($3::int[]))`, [slug, txOpts.sourceId, keptIds]) : tx.deleteChunks(slug, txOpts),
+ // Alias projection and readback share the page commit. A later writer can
+ // no longer turn a successful import into a postcommit verification error.
+ () => writePageAliases(tx, slug, sourceId ?? 'default', parsed, opts.activePack, mentionPolicy),
+ ]);
+ const pageId = segmented?.applied ? segmented.applied.pageId : written && written.deleted_at == null ? written.id : undefined;
+ const deferSeal = opts.coordinated === true && pageId !== undefined && !segmented;
+ await pipelined(tx, [
+ async () => {
+ // Tag reconciliation (A14): frontmatter tags carry tag_source='frontmatter'.
+ // A frontmatter-owned row whose tag left the frontmatter is deleted; rows
+ // explicit adds own ('added': add_tag, enrichment, the code importer) and
+ // legacy NULL rows are never deleted here (#1621: reindex must not wipe
+ // enrichment). A legacy row still in the frontmatter is adopted. Prepared
+ // (managed) imports keep the add-only union their canonical file renders.
+ if (!opts.prepare) await tx.executeRaw(`DELETE FROM tags t USING pages p WHERE p.id = t.page_id AND p.source_id = $1 AND p.slug = $2
+ AND t.tag_source = 'frontmatter' AND NOT (t.tag = ANY($3::text[]))`, [txOpts.sourceId, slug, parsed.tags]);
+ if (pageId !== undefined) return addPageTags(tx, Number(pageId), parsed.tags, opts.prepare ? undefined : 'frontmatter');
+ for (const tag of parsed.tags) await tx.addTag(slug, tag, opts.prepare ? txOpts : { ...txOpts, tagSource: 'frontmatter' });
+ },
+ async () => {
+ if (deferChunks) return;
+ if (chunks.length > 0) {
+ await applyMoveWaves(tx, Number(pageId), chunkWrites.waves);
+ const embeddingColumn = await stampEmbeddingInputs(tx, fresh, null,
+ { title: parsed.title, tier: effectiveCRMode === 'title' ? 'title' : 'none', corpusGeneration });
+ await tx.upsertChunks(slug, kept.size ? fresh.filter((_, i) => !kept.has(i)) : fresh, { ...txOpts, ...(embeddingColumn ? { embeddingColumn } : {}), sealChunkerVersion: MARKDOWN_CHUNKER_VERSION,
+ ...(pageId !== undefined ? { pageId: Number(pageId) } : {}), ...(deferSeal ? { deferSeal: true as const } : {}) });
+ // v0.41.31: stamp embedding provenance when this import actually
+ // embedded (not --no-embed), so a later model/dims swap is detectable
+ // as stale via embed --stale. The deferred/backfill + per-slug embed
+ // paths stamp too; this covers the inline import/sync path.
+ if (!noEmbed && !postCommitEmbedding && !embeddingDeferred) {
+ // D9: signature is null when the gateway is unconfigured — skip the
+ // stamp (a wrong signature is worse than none).
+ const importSig = currentEmbeddingSignature();
+ if (importSig) await tx.setPageEmbeddingSignature(slug, { sourceId, signature: importSig });
+ }
+ } else {
+ // Seal only the completed, full-body sanitized replacement (the upsert above seals its own); a
+ // body-only write or a failed transaction must never certify old stored fragments.
+ if (!deferSeal && !segmented) await tx.executeRaw('UPDATE pages SET chunker_version = $1 WHERE source_id = $2 AND slug = $3', [MARKDOWN_CHUNKER_VERSION, txOpts.sourceId, slug]);
+ }
+ },
+ ]);
+ if (!opts.coordinated && !segmented && !deferChunks) await sealPageTextProjection(tx, slug, txOpts.sourceId);
+ if (!opts.coordinated && segmented?.applied?.complete) await sealSegmentedPage(tx, segmented.applied.pageId);
+
+ // v0.19.0 E1 — doc↔impl linking: if this markdown page cites code paths
+ // (e.g. 'src/core/sync.ts:42'), create bidirectional edges to the code
+ // page, all in one statement (addCodeReferenceLinks). An edge whose code
+ // page is missing is skipped — guides imported before their code repo
+ // syncs are common, and the missing edges land later via
+ // `gbrain reconcile-links` (Layer 8 D3, v0.21.0).
+ const codeRefs = extractCodeRefs(sanitizeRemoteBody(parsed.compiled_truth) + '\n' + sanitizeRemoteBody(parsed.timeline || ''));
+ // For doc↔impl edges, both endpoints are within the same source as the
+ // markdown page being imported. Cross-source edges (markdown in one
+ // source, code in another) are skipped as missing — never the pre-fix
+ // cross-product fan-out, which silently wired edges to whichever
+ // same-slug page Postgres returned first across sources.
+ if (codeRefs.length) {
+ await tx.transaction(savepoint => addCodeReferenceLinks(savepoint, slug, sourceId || 'default',
+ codeRefs.map(ref => ({ codeSlug: slugifyCodePath(ref.path), path: ref.path, line: ref.line })))).catch(() => undefined);
+ }
+ if (!opts.coordinated) await verifyPageReadable(tx, slug, hash, sourceId, 'importFromContent');
+ await opts.beforeCommit?.(tx, slug);
+ await disposition.persistReceipt(tx, { existed: !!existing });
+ if (postCommitEmbedding && !noEmbed && chunks.length > 0) {
+ // Capture the complete installed projection before releasing the page
+ // guard. Deferred provider results cannot replace newer text or chunks.
+ persistedProjection = await readProjectionSnapshot(tx, slug, txOpts.sourceId);
}
+ if (segmented?.applied) return { pageId, sealed: !opts.coordinated && segmented.applied.complete, segmented: { complete: segmented.applied.complete } };
+ // The reason routes a resident or drain rebuild of this revision to the bounded continuation.
+ if (deferChunks) await queuePageProjection(tx, txOpts.sourceId, slug, DEFERRED_CHUNK_REPLACEMENT);
+ if (deferChunks) return { pageId, sealed: false, projectionPending: true as const };
+ return { pageId, sealed: !opts.coordinated && !opts.beforeCommit, ...(deferSeal ? { chunkerSeal: MARKDOWN_CHUNKER_VERSION } : {}) };
+ };
+ if (opts.prepare) return opts.prepare({
+ slug, parsedPage, observedRevision: (existing as (typeof existing & { knowledge_revision?: string }) | null)?.knowledge_revision ?? null,
+ noop: false, contentHash: hash, result: { slug, status: 'imported', chunks: segmented ? segmented.prepared.rows.chunks : chunks.length, parsedPage,
+ ...(pageQuarantined ? { quarantined: true } : {}), ...(pageFlagged ? { flagged: true, flag_reason: pageFlagReason } : {}), ...(disposition.gate ? { gate: disposition.gate } : {}), ...fences.fields },
+ validate: tx => assertPreparedFactWithdrawals(tx, txOpts.sourceId, parsed.compiled_truth, parsed.timeline || '', slug),
+ apply: applyPrepared, ...(segmented ? { afterCommit: async (target: BrainEngine) => { await segmented.afterCommit(target); }, segmented }
+ : opts.boundedChunkWrites ? { afterCommit: async (target: BrainEngine) => { if (chunkReplacementDeferred) await continueDeferredProjection(target, txOpts.sourceId, slug); } } : {}),
});
- }
+ await withUnmanagedBeliefs(engine, txOpts.sourceId, [slug], 'import', run => maintenanceTransaction(engine, tx => run(tx, () => applyPrepared(tx)), writeTrustOfGate(opts.writeGate)).catch(async (err: unknown) => {
+ await abortSegmentedStaging(engine, segmented?.prepared);
+ // #4287: name the dimension-mismatch rollback instead of letting the bare
+ // pgvector message ("expected N dimensions, not M") surface with no code,
+ // no consequence and no fix. S2: name the registry-ACTIVE column the
+ // write actually targeted (best-effort — plane-agnostic wording when the
+ // registry itself is unreadable).
+ let activeColName: string | undefined;
+ const msg = err instanceof Error ? err.message : String(err);
+ if (/expected \d+ dimensions, not \d+/.test(msg)) {
+ try {
+ const { resolveActiveEmbeddingColumnFromEngine } = await import('./search/embedding-column.ts');
+ activeColName = (await resolveActiveEmbeddingColumnFromEngine(engine, { fallbackToLegacy: true })).name;
+ } catch { /* keep the plane-agnostic wording */ }
+ }
+ throw decorateEmbeddingDimError(err, slug, activeColName);
+ }));
+ const searchable = await segmented?.afterCommit(engine);
- return {
- slug,
- status: 'imported',
- chunks: segmented ? segmented.prepared.rows.chunks : chunks.length,
- parsedPage,
- ...(pageQuarantined ? { quarantined: true } : {}),
- ...(pageFlagged ? { flagged: true, flag_reason: pageFlagReason } : {}),
- ...(typeWarning ? { type_warning: typeWarning } : {}),
- ...(embeddingDeferred ? { embedding_deferred: true } : {}), ...fences.fields, ...(searchable !== undefined ? { searchable } : {}),
+ if (postCommitEmbedding && !noEmbed && chunks.length > 0) {
+ const importScope = currentEngineScope();
+ postCommitEmbedding(() => runInEngineScope(importScope, async (): Promise<ImportEmbeddingResult> => {
+ try {
+ if (!persistedProjection) return { status: 'superseded' };
+ const signature = currentEmbeddingSignature();
+ await embedChunks();
+ const installed = await installPageEmbeddings(engine, persistedProjection, chunks, embeddingPartial ? undefined : signature ?? undefined);
+ return { status: installed.complete ? 'embedded' : 'superseded' };
+ } catch {
+ return { status: 'failed', error: 'Page content was saved, but embedding failed. Check the embedding provider and database on the brain host, then run gbrain embed --stale --source <source-id>.' };
+ }
+ }));
+ }
+
+ return {
+ slug,
+ status: 'imported',
+ chunks: segmented ? segmented.prepared.rows.chunks : chunks.length,
+ parsedPage,
+ ...(pageQuarantined ? { quarantined: true } : {}),
+ ...(pageFlagged ? { flagged: true, flag_reason: pageFlagReason } : {}),
+ ...(typeWarning ? { type_warning: typeWarning } : {}),
+ ...(embeddingDeferred ? { embedding_deferred: true } : {}), ...fences.fields, ...(searchable !== undefined ? { searchable } : {}),
+ };
};
+ return await (needsEngineScope(engine) ? withEngineScope(engine, importScreened) : importScreened());
}
/**
@@@ -1234,9 -1237,10 +1239,10 @@@ export async function importCodeFile
engine: BrainEngine,
relativePath: string,
content: string,
- opts: { noEmbed?: boolean; force?: boolean; sourceId?: string;
+ opts: { noEmbed?: boolean; force?: boolean; sourceId?: string; history?: import('./page-state/versions.ts').VersionHistory;
prepare?: (prepared: import('./persistence/prepared-import.ts').PreparedContentImport) => Promise<ImportResult> } = {},
): Promise<ImportResult> {
+ if (needsEngineScope(engine)) return withEngineScope(engine, () => importCodeFile(engine, relativePath, content, opts));
if (!opts.prepare) await assertUnmanagedCanonicalWriter(engine, 'direct file import', { sourceId: opts.sourceId ?? 'default' });
if (opts.prepare && !opts.noEmbed) throw new Error('Coordinated code import requires provider-free preparation.');
const slug = slugifyCodePath(relativePath);
diff --cc src/core/persistence/page-purge.ts
index ad7883087c,3f79ce2f10..876c35f763
--- a/src/core/persistence/page-purge.ts
+++ b/src/core/persistence/page-purge.ts
@@@ -327,16 -325,12 +327,16 @@@ export async function purgePageInTransa
// #6511 [R2]: the preparation sweep's targets, plus the tail past its ceiling and the rows pending at scan time, re-tested here.
const recheck = await lookup('tail_recheck', () => recheckRequestsCarrying(tx, sourceId, needles, prepared.sweep,
{ exclude: row.id, beforePending: () => faultPoint('purge:pending_recheck', { requestId: row.request_id, sourceId, operation: row.operation }) }));
- const forSlug = await lookup('slug_lookup', () => findRequestsForSlugs(tx, sourceId, [slug]));
+ const forSlug = await lookup('slug_lookup', async () => [...await findRequestsForSlugs(tx, sourceId, [slug]), ...reasons.length ? await requestsRecordingReasons(tx, sourceId, slug) : []]);
+ const own = new Set(forSlug.map(r => r.id));
const requests = [...new Map([...forSlug, ...prepared.sweep.targets, ...recheck.targets].map(r => [r.id, r])).values()];
- removed.persistence_requests = await lookup('redaction_read', () => redactRequestIntents(tx, requests, row.id, { skipPending: true, needles }));
+ // GBRA-217: the page's reasons are needles in its own requests only, so a short reason never redacts another page's receipt.
+ removed.persistence_requests = await lookup('redaction_read', async () => await redactRequestIntents(tx, requests.filter(r => own.has(r.id)), row.id,
+ { skipPending: true, needles: [...needles, ...reasons] }) + await redactRequestIntents(tx, requests.filter(r => !own.has(r.id)), row.id, { skipPending: true, needles }));
+ removed.persistence_effects = await stripPageGitNotes(tx, sourceId, slug);
const journalSweep = { scanned: prepared.sweep.scanned, ceiling: prepared.sweep.ceiling, watermark: prepared.sweep.watermark ?? null, batch: prepared.sweep.batch ?? 1,
tail: recheck.tail, pending_rechecked: recheck.pendingRechecked, complete: prepared.sweep.complete && recheck.complete };
- const files = await tx.executeRaw<{ storage_path: string }>(`DELETE FROM files WHERE (page_id=$1 OR (page_slug=$2 AND (source_id=$3 OR source_id IS NULL)))
+ const files = await tx.executeRaw<{ storage_path: string }>(`DELETE FROM files WHERE (page_id=$1 OR (page_id IS NULL AND page_slug=$2 AND (source_id=$3 OR source_id IS NULL)))
RETURNING storage_path`, [page.id, slug, sourceId]);
removed.files = files.length;
Object.assign(removed, await (await import('../generations/ledger-scrub.ts')).scrubGenerationLedger(tx, sourceId, { slug }));