Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(migrations): v0.13.1 no longer writes validate: false into pages on upgrade (#7814)
garrytanb92d2ba75fec
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 9bdfa4a6c8..7d6b7a57e7 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,26 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**Upgrading an older brain no longer rewrites every page.**++A brain created by `gbrain init` before v0.60.412.0 never recorded the old v0.13.1 migration, so upgrading it ran that migration, and it wrote `validate: false` into the frontmatter of every page. Every page's content hash changed, and on a repo brain the database stopped matching the files in Git (or the stamp was committed into them). The same happened to an existing brain opened from a new `GBRAIN_HOME`. Nothing used the stamp: the validators it protected pages from never refuse a write.++v0.13.1 now records itself as complete and changes no page (#7814).++### What changed++| Situation | Before | Now |+|---|---|---|+| `gbrain apply-migrations` (or `gbrain upgrade`) on a brain whose ledger never recorded v0.13.1 | `validate: false` written into every page; every content hash moved | no page changes |+| A managed repo brain in that state | the stamp committed into every Markdown file | no file or commit |+| A grandfather write an older gbrain queued and left pending | published after the upgrade | ends `conflict` without writing |++### For operators++Pages an older release already stamped keep the stamp; sync and writes already accept it (#6429). Their frontmatter from before the stamp is in `~/.gbrain/migrations/v0_13_1-rollback.jsonl` on the machine that ran the migration. A page that carries `validate` because someone set it keeps it.+## [0.60.476.0] - 2026-10-27**Text someone else wrote can no longer add facts to your brain.**diff --git a/docs/architecture/key-files/canonical-reconciliation.md b/docs/architecture/key-files/canonical-reconciliation.mdindex 2bdb63853d..ec54e77a8d 100644--- a/docs/architecture/key-files/canonical-reconciliation.md+++ b/docs/architecture/key-files/canonical-reconciliation.md@@ -12,7 +12,7 @@ See [generations](generations.md).- `src/core/cycle/extract-atoms-page-state.ts` — derived page scan/failure state keyed by source incarnation, stable page identity and content hash. Compare-and-swap stamps never update canonical frontmatter. Reviewed reconciliation can carry matching state across removal of legacy bookkeeping fields. Unmanaged atom items complete as one generation; managed extraction uses journaled publication and a durable completion receipt after trusted local source-wide authority checks and, for filesystem publication, owner preflight.-- `src/core/persistence/grandfather.ts` — provider-free canonical publication for the grandfather compatibility flag on active managed Markdown pages. Admission retains a checkpointed request UUID, exact source/incarnation/page/revision identity and a private rollback snapshot. Publication preserves chunks/vectors and only already-valid text seals, carries matching atom scan state across the metadata hash change, and defers embeddings without queuing fact extraction. Explicit validation choices, archived sources and non-Markdown artifacts remain untouched; unpublished file edits refuse. Pages under a declared `storage.db_only` dir without a local cache file publish database-only. The committed revision is returned so the migration's verify phase can tell a later rewrite from a lost write. `admitCanonicalGrandfather` splits admission from `complete()` (wait, then settle the checkpoint exactly as the serial `grandfatherCanonicalPage` does); the migration admits a 50-page window before waiting, so the pages' Git effects coalesce (#5530). The unmanaged migration keeps its bounded SQL batches. `test/grandfather-managed.test.ts` covers both engines, replay, races and preserved artifacts; the unchanged local-install lifecycle exercises the public upgrade path.+- `src/core/persistence/grandfather.ts` — the retired v0.13.1 grandfather's request preparer (#7814). Upgrades no longer stamp `validate: false` into pages, so a `managed_grandfather` put_page request an older binary admitted and left queued is refused `revision_conflict` (the request ends `conflict`) and the page, its file and its revision stay unchanged. `test/grandfather-managed.test.ts` covers it.- `src/core/persistence/managed-mode.ts` — the one reader of the brain's persistence mode (#6482). `readManagedMode` / `readPersistenceBrain` accept exactly one `persistence_brain` row with a boolean `enabled` (optionally `FOR SHARE` / `FOR UPDATE`) and refuse anything else (`storage_error`, `detail: persistence_mode_unknown`, fix `gbrain sources writer status --probe --json`); only a well-formed `enabled: false` row selects the unmanaged path, so a missing, duplicated or malformed row can never route a write around the owner. Every non-test reader of the mode goes through it (`test/persistence-mode-reader-guard.test.ts` lists the few schema and admin-intent files allowed their own statement); `ownership.ts` re-exports it for callers that already import ownership.diff --git a/docs/architecture/key-files/commands-4.md b/docs/architecture/key-files/commands-4.mdindex 95b5615fcc..98312c7f12 100644--- a/docs/architecture/key-files/commands-4.md+++ b/docs/architecture/key-files/commands-4.md@@ -27,7 +27,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/commands/migrate-engine.ts` — Legacy engine copier (`gbrain migrate --to pglite`, and PGLite → Postgres when `migrate.graduation` is `false`). A PGLite → Postgres move routes to `src/commands/migrate-graduation.ts` (engine graduation) before connect and never reaches this file. Copies the complete source catalog FIRST (`copyMigrationSources` — every `sources` row incl. archived rows and sync/routing metadata, `ON CONFLICT (id) DO UPDATE`, `default` ordered first) so every page write has a valid `pages.source_id` FK parent and the target preserves per-source behavior; pages copy afterward: `listAllPageRefs` lists every live page's `(source_id, slug)` in one uncapped query, `copyMigrationPages` reads each body with `getPage(slug, { sourceId })` (a page soft-deleted after the listing reads null and is skipped, reported, and kept out of the link copy; it is not a failure), and each page is tracked in the resume manifest by composite `(source_id, slug)` key. Link copy preserves each row's `to_source_id` (falling back to the origin source only for legacy rows without it), and failed-target filtering uses that same target composite key, so cross-source links migrate without being rebound to the origin source. The resume manifest is target-aware: `migrationTargetId(config)` hashes `(engine, locator)` (`database_url` for Postgres, resolved `database_path` for PGLite) and `manifestMatchesTarget` requires `schema_version === 2` plus a matching `target_id` — a legacy engine-only manifest, or one from a DIFFERENT target of the same engine kind, starts fresh instead of skipping "completed" pages the new target never received. After links, `copyMigrationFacts` carries the facts table verbatim (conversation facts have no markdown fence to re-sync from): source∩target column intersection via information_schema (older source schemas still copy), delete-and-recopy so every re-run converges to source truth, two-pass `superseded_by` restore (the self-FK points old→newer id, so id-ordered single-pass inserts would violate it), embeddings round-tripped as text into whichever physical type (`vector`/`halfvec`) the target column has with a NULL-embedding retry on per-row cast failure, and a `setval` bump so post-migration inserts don't collide with copied ids; fact-row failures block the config flip and force a non-zero exit, same contract as page failures. Takes are deliberately NOT copied (fence-canonical, keyed on target-reassigned `page_id`; the next extract cycle rebuilds them). `copyMigrationConfig` then copies EVERY DB-plane config row except the explicit engine-local denylist `MIGRATE_CONFIG_ENGINE_LOCAL_KEYS` (`engine`, `version` = target-owned schema ledger, `embedding_columns` + `search_embedding_column` = physical-column registry the copy doesn't materialize); skipped keys print in the end-of-run per-table copied-count summary, so nothing is dropped silently. Pinned by `test/migrate-engine-resume.test.ts` (manifest identity) + `test/e2e/migrate-engine-sources-postgres.test.ts` (source catalog lands before overlapping-slug pages, PGLite → real Postgres) + `test/e2e/multi-source-bug-class.test.ts` (cross-source links) + `test/migrate-engine-completeness.serial.test.ts` (facts chains/embeddings/sequence, config copy-all + denylist, summary) + `test/migrate-engine-pagination.serial.test.ts` (no listing cap; a page deleted mid-copy). Legacy engine copies explicitly include stored unverified chunks and queue sanitized text rebuilding under the target revision; retrieval stays gated until rebuilt. #6286: `copyPageToTarget` forwards the full page row (`page_kind`, `source_path`, effective date and its source, `import_filename`, `source_kind`/`source_uri`/`ingested_via`, `ingested_at` restored in the timestamp UPDATE), the frontmatter aliases (`setPageAliases`) and each chunk's code metadata (`carryChunkMetadata`). A page sealed on the source is queued `engine_migration`; one the source had not sealed is queued `engine_migration_source_unsealed` and keeps its verbatim chunks and vectors. Before the config flip, while autopilot is paused, `runMigrateEngine` runs `drainProjections` (moved to `src/core/page-state/projections.ts`; `commands/projections.ts` re-exports it) on the target for the `engine_migration` and `rebuild_failed` reasons, keyless; the rebuild keeps chunk ids and vectors whose identity and embedding input match, so the first sync after the flip skips unchanged pages. A failed rebuild blocks the flip with exit 1, like a page that failed to copy. Pinned by `test/migrate-full-row-projections.serial.test.ts`, `test/e2e/migrate-full-row-projections-postgres.test.ts` and a D2 case in `test/e2e/migrate-engine-pglite-to-postgres.test.ts`. Open-loop and loop-suppression rows the target refuses (`copyMigrationSimpleTable`, per-row `ItemRun` site `migrate_engine.loop_rows`) are printed with their id and reason, block the config flip and set exit 1; a run-scoped failure stops the copy (`test/pathological/a31-migrate-loop-rows.serial.test.ts`). Both source and target refuse any durable request history, withdrawal ledger, or ownership record before copying/clearing because this copier cannot preserve those identities.-- `src/commands/migrations/` — TS migration registry (compiled into the binary; no runtime walk of `skills/migrations/*.md`). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify); `phaseASchema` has a 600s timeout for duplicate-heavy brains; the config check reads `auto_link` in-process (file/env plane, then the database), never through a `gbrain config get` child. `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (pending-host-work ping for skills/migrations/v0.14.0.md; raises `minion_jobs.max_stalled`'s default to 3 only when it is below 3, so it never lowers schema v15's 5). Fresh installs (E8): `Migration.fresh_install_noop: true` marks an orchestrator that changes nothing on a brain `gbrain init` just created; `fresh-install.ts:recordFreshInstallMigrations` (one call in each init engine path, only when init created the database) appends a `complete` ledger entry with `fresh_install: true` + `installed_version` for each, and every other migration lists as `pending_fresh_install` (`src/core/migration-ledger.ts:freshInstallVersion`/`isPendingFreshInstall`) in `apply-migrations --list [--json]`/`--dry-run`, `get_health.migrations` and doctor's `minions_migration` (an `ok`/`info` row with `gbrain apply-migrations --yes --no-autopilot-install`); stamps never count as forward progress over a real partial. Each flag is proven by `test/migrations-fresh-install-audit.serial.test.ts` (schema, row counts, config table and GBRAIN_HOME file tree unchanged by the real orchestrator on a keyless `init --pglite --no-embedding` brain); unit coverage `test/migrations-fresh-install.test.ts`. `shared-content.ts` names its read-only next step as phase `argv` (`SHARED_CONTENT_INSPECT_ARGV`), which the runner prints and the ledger keeps. #6466 opt-out: `Migration.optOut: { key }` names a brain-level DB config key (`shared-content.ts`: `shared_skills.migration.enabled`, the registry-free table `MIGRATION_OPT_OUT_KEYS` in `src/core/migration-ledger.ts`, pinned to every module's `optOut`); an explicit off value (`false`/`0`/`off`/`no`, `isMigrationDeclinedValue`) puts the migration in the plan's `declined` bucket (`opt-out.ts:readDeclinedMigrations`, read before `buildPlan`; an unreachable brain declines nothing), so the runner never calls its orchestrator, reconcile included, `--list [--json]` and `--dry-run` show it `declined` with `opt_out.key`, and an explicit `--migration <v>` refuses `migration_declined` (exit 3, `fix.next: ask_user`, `gbrain config set <key> true`, verify `--list --json`). `migrationLedgerSummary(VERSION, declined)` gives `get_health.migrations.declined` and doctor's `minions_migration` an `ok` row with `details.declined`. Tests: `test/shared-skills-migration-decline-6466.test.ts`. All orchestrators are idempotent and resumable from `partial` status. The RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape (orchestrators never call `appendCompletedMigration`). `statusForVersion` prefers `complete` over `partial` (never regresses); 3 consecutive partials → wedged → `--force-retry <version>` writes a `'retry'` reset marker. Schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) live in `src/core/schema-migrations/v014-pages-updated-at-index.ts` and `v015-minion-jobs-max-stalled-default-5.ts` (one file per schema migration; `migrate.ts` is the runner). `in-process.ts` exports `runMigrateOnlyCore({timeoutMs?})` — single source of truth for "bring schema to head" (`configureGateway` → `createEngine` → `connect` → `initSchema` → `disconnect`, idempotent, 600s `MIGRATE_ONLY_TIMEOUT_MS` guard, throws `MigrateOnlyError` on no-config / timeout); the orchestrators' 9 schema phases AND `init.ts:initMigrateOnly` both delegate to it so schema bring-up can't drift (in-process, so no spawn can die with `getaddrinfo ENOTFOUND` on Windows + bun + Supabase pooler). `runGbrainSubprocess` is the diagnostic wrapper for the remaining non-schema spawns (extract/repair/stats): captures child stderr (64MB buffer) into the thrown error. `v0_13_1.ts:phaseCGrandfather` is a CHUNKED bulk SQL pass keyed on `pages.id` (globally unique PK, NOT slug — slug uniqueness is `(source_id, slug)`), filters `deleted_at IS NULL` (no tombstones), chunked in `CHUNK_SIZE` batches (DELETE_BATCH_SIZE convention) for bounded lock-hold; the rollback log carries `{id, slug, source_id, pre_frontmatter}` so rollback is unambiguous across sources; idempotent + resumable (each UPDATE flips its rows out of `GRANDFATHER_WHERE`). Pinned by `test/migration-in-process.serial.test.ts` and `test/migrations-v0_13_1-grandfather.test.ts`.+- `src/commands/migrations/` — TS migration registry (compiled into the binary; no runtime walk of `skills/migrations/*.md`). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify); `phaseASchema` has a 600s timeout for duplicate-heavy brains; the config check reads `auto_link` in-process (file/env plane, then the database), never through a `gbrain config get` child. `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (pending-host-work ping for skills/migrations/v0.14.0.md; raises `minion_jobs.max_stalled`'s default to 3 only when it is below 3, so it never lowers schema v15's 5). Fresh installs (E8): `Migration.fresh_install_noop: true` marks an orchestrator that changes nothing on a brain `gbrain init` just created; `fresh-install.ts:recordFreshInstallMigrations` (one call in each init engine path, only when init created the database) appends a `complete` ledger entry with `fresh_install: true` + `installed_version` for each, and every other migration lists as `pending_fresh_install` (`src/core/migration-ledger.ts:freshInstallVersion`/`isPendingFreshInstall`) in `apply-migrations --list [--json]`/`--dry-run`, `get_health.migrations` and doctor's `minions_migration` (an `ok`/`info` row with `gbrain apply-migrations --yes --no-autopilot-install`); stamps never count as forward progress over a real partial. Each flag is proven by `test/migrations-fresh-install-audit.serial.test.ts` (schema, row counts, config table and GBRAIN_HOME file tree unchanged by the real orchestrator on a keyless `init --pglite --no-embedding` brain); unit coverage `test/migrations-fresh-install.test.ts`. `shared-content.ts` names its read-only next step as phase `argv` (`SHARED_CONTENT_INSPECT_ARGV`), which the runner prints and the ledger keeps. #6466 opt-out: `Migration.optOut: { key }` names a brain-level DB config key (`shared-content.ts`: `shared_skills.migration.enabled`, the registry-free table `MIGRATION_OPT_OUT_KEYS` in `src/core/migration-ledger.ts`, pinned to every module's `optOut`); an explicit off value (`false`/`0`/`off`/`no`, `isMigrationDeclinedValue`) puts the migration in the plan's `declined` bucket (`opt-out.ts:readDeclinedMigrations`, read before `buildPlan`; an unreachable brain declines nothing), so the runner never calls its orchestrator, reconcile included, `--list [--json]` and `--dry-run` show it `declined` with `opt_out.key`, and an explicit `--migration <v>` refuses `migration_declined` (exit 3, `fix.next: ask_user`, `gbrain config set <key> true`, verify `--list --json`). `migrationLedgerSummary(VERSION, declined)` gives `get_health.migrations.declined` and doctor's `minions_migration` an `ok` row with `details.declined`. Tests: `test/shared-skills-migration-decline-6466.test.ts`. All orchestrators are idempotent and resumable from `partial` status. The RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape (orchestrators never call `appendCompletedMigration`). `statusForVersion` prefers `complete` over `partial` (never regresses); 3 consecutive partials → wedged → `--force-retry <version>` writes a `'retry'` reset marker. Schema-only migrations v14 (`pages_updated_at_index`) + v15 (`minion_jobs_max_stalled_default_5` with UPDATE backfill) live in `src/core/schema-migrations/v014-pages-updated-at-index.ts` and `v015-minion-jobs-max-stalled-default-5.ts` (one file per schema migration; `migrate.ts` is the runner). `in-process.ts` exports `runMigrateOnlyCore({timeoutMs?})` — single source of truth for "bring schema to head" (`configureGateway` → `createEngine` → `connect` → `initSchema` → `disconnect`, idempotent, 600s `MIGRATE_ONLY_TIMEOUT_MS` guard, throws `MigrateOnlyError` on no-config / timeout); the orchestrators' 9 schema phases AND `init.ts:initMigrateOnly` both delegate to it so schema bring-up can't drift (in-process, so no spawn can die with `getaddrinfo ENOTFOUND` on Windows + bun + Supabase pooler). `runGbrainSubprocess` is the diagnostic wrapper for the remaining non-schema spawns (extract/repair/stats): captures child stderr (64MB buffer) into the thrown error. `v0_13_1.ts` is an idempotent no-op since #7814: it records completion and writes no page (it used to stamp `validate: false` into every page on any brain whose ledger never recorded v0.13.1). Pinned by `test/migrations-v0_13_1.test.ts` and the upgrade contract `test/upgrade-preserves-pages-7814.slow.test.ts`.- `src/commands/migrations/v0_46_3.ts` — idempotent no-op for a retired notification. Returns a complete migration with a skipped notice phase; performs no database, config, provider, vector or pending-work writes. Its feature pitch points to read-only status and explicit-target migration previews.- `src/commands/migrations/v0_60_31.ts` — the security fix wave orchestrated migration (ledger `0.60.31`), two phases. `credential_projection`: runs `runMigrateOnlyCore` first (schema v189 marks the pages; PGLite skips the runner's schema pre-flight), then `resealCredentialProjections`, provider-free and never embedding; a dry run reports the count; a page whose re-chunk fails item-scoped is held (`credential-reseal.ts`) and the phase completes with `held: N` and the codes in its detail (doctor `text_projection_readiness` lists the pages, `gbrain repair safe-chunks --source <id> --apply` retries them); a run-scoped failure or a systemic one (`item_failures_systemic`) fails the phase with its code, so the run is `partial` and `gbrain apply-migrations --yes --no-autopilot-install` retries it, and re-sealed pages are no longer pending, so a rerun resumes. `google_file_modes_notice`: `googleFileModesNoticePhase` from `v0_60_31-google-file-modes.ts`. A brain without config completes with the phase skipped. Agent instructions: `skills/migrations/v0.60.31.0.md`.diff --git a/docs/architecture/key-files/page-identity-and-administration.md b/docs/architecture/key-files/page-identity-and-administration.mdindex afbbb0a566..c69fcbc2b1 100644--- a/docs/architecture/key-files/page-identity-and-administration.md+++ b/docs/architecture/key-files/page-identity-and-administration.md@@ -60,18 +60,15 @@ package installation, post-upgrade and migration orchestration. It skips serviceinstallation and unit rewrites, not unrelated migration work. Paid reindexinghas its own `GBRAIN_NO_REEMBED=1` control.-The grandfathering batch in `src/commands/migrations/v0_13_1.ts` changes only-`frontmatter.validate`. It holds canonical page guards and row locks, rechecks-eligibility, and advances the text seal only for already-sealed snapshots.-An unsealed page stays unsealed. This preserves keyword retrieval after a-metadata-only upgrade without bypassing revision safety. The verify phase-checks only the pages this run grandfathered, keyed on `pages.id` and the-revision each write produced: a page a concurrent writer (such as a connector-re-import) rewrote or deleted since is reported as `rewritten_concurrently`,-and only a page still at that revision without `validate: false` fails the-phase. The installed-lifecycle-test and `test/e2e/grandfather-projection-postgres.test.ts` cover the actual-upgrade consequence and engine parity respectively.+`src/commands/migrations/v0_13_1.ts` writes nothing (#7814). It used to stamp+`validate: false` into every page's frontmatter, which an upgrade ran on any+brain whose ledger never recorded v0.13.1 and which moved every content_hash.+The orchestrator now records completion only, and a queued+`managed_grandfather` request from an older binary ends `conflict` without+writing (`src/core/persistence/grandfather.ts`).+`test/upgrade-preserves-pages-7814.slow.test.ts` (PGLite) and+`test/e2e/upgrade-preserves-pages-7814-postgres.test.ts` upgrade a+v0.59.2.0-shaped brain, repo-less and repo, and assert no page changes.## Write request idsdiff --git a/docs/architecture/system-of-record.md b/docs/architecture/system-of-record.mdindex 38e41c1918..99f3988017 100644--- a/docs/architecture/system-of-record.md+++ b/docs/architecture/system-of-record.md@@ -280,7 +280,7 @@ or a physical projection that leaves attribution untouched):- `src/core/persistence/derived-retirement-hold.ts` (3): `retireDerivedPages` versions, soft-deletes and stamps retired derived pages (unmanaged extract_atoms stale atoms) in 500-page `maintenanceTransaction` chunks.- `src/core/persistence/facts-prepare.ts` (3): managed `extract_facts` request preparer.- `src/core/persistence/file-repair.ts` (2): `managed_file_repair` request preparer (#5988).-- `src/core/persistence/grandfather.ts` (2): request preparer.+- `src/core/persistence/grandfather.ts` (2): request preparer (refuses retired grandfather requests, #7814).- `src/core/persistence/links-maintenance.ts` (1): coordinated maintenance.- `src/core/persistence/loop-fact-retirement.ts` (1): request preparer.- `src/core/persistence/memory-mutations.ts` (1): `forget` inside its request's coordinated write.diff --git a/scripts/e2e-weights.json b/scripts/e2e-weights.jsonindex b9906a7e75..ddf987118e 100644--- a/scripts/e2e-weights.json+++ b/scripts/e2e-weights.json@@ -271,7 +271,6 @@"test/e2e/graduation-crash-run-3.test.ts": 68040,"test/e2e/graduation-faults.test.ts": 32549.999999999996,"test/e2e/graduation-legacy-copy.test.ts": 9520,- "test/e2e/grandfather-projection-postgres.test.ts": 6580,"test/e2e/graph-quality.test.ts": 2040,"test/e2e/graph-signals-engine.test.ts": 529,"test/e2e/graph-signals-eval.test.ts": 3900,diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsvindex bfa01bceb3..c3e37df620 100644--- a/scripts/module-size-limits.tsv+++ b/scripts/module-size-limits.tsv@@ -47,4 +47,4 @@ src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: linksrc/core/search/evidence-delivery.ts 1733 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move)src/commands/config.ts 1524 ratchet new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)src/commands/mcp-expose.ts 1512 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts)-src/core/behavior-change-notice.ts 1518 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2); the behavior-change table split (#7607) retires this row+src/core/behavior-change-notice.ts 1520 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row; the behavior-change table split (#7607) retires this rowdiff --git a/scripts/page-body-readers-baseline.tsv b/scripts/page-body-readers-baseline.tsvindex c356a98ed5..f459ccfc16 100644--- a/scripts/page-body-readers-baseline.tsv+++ b/scripts/page-body-readers-baseline.tsv@@ -435,8 +435,6 @@ src/core/persistence/facts-maintenance.ts prepareManagedFactsSession field METAsrc/core/persistence/facts-prepare.ts prepareManagedFactsMutation call:serializePageToMarkdown SEGW assertInlineBody firstsrc/core/persistence/facts-prepare.ts prepareManagedFactsMutation field SEGW assertInlineBody on the entity snapshot before the fence writesrc/core/persistence/file-repair.ts canonicalRepairFields field META-src/core/persistence/grandfather.ts prepareGrandfatherMutation call:contentHash META-src/core/persistence/grandfather.ts prepareGrandfatherMutation call:serializePageToMarkdown SEGWsrc/core/persistence/import-prepare.ts prepareManagedImportMutation call:serializePageToMarkdown SEGWsrc/core/persistence/large-page-guard.ts assertLargePageAdmission sql META stored size = body_bytes for a segmented page, octet_length of the columns for an inline pagesrc/core/persistence/links-maintenance.ts extractManagedStaleLinks field NONE not the page body: result.timeline counterdiff --git a/scripts/test-weights.json b/scripts/test-weights.jsonindex d1a450fc86..42513ab943 100644--- a/scripts/test-weights.json+++ b/scripts/test-weights.json@@ -1605,7 +1605,6 @@"test/graduation-target.test.ts": 3,"test/graduation-verify-sync-freshness.test.ts": 408,"test/graduation-verify.test.ts": 5513,- "test/grandfather-capacity.test.ts": 1454,"test/grandfather-managed.test.ts": 23118,"test/grant-blocked-notice.test.ts": 750,"test/grant-execution-prerequisites.test.ts": 614,@@ -2186,7 +2185,6 @@"test/migrations-v0_12_0.test.ts": 1,"test/migrations-v0_12_2.test.ts": 0,"test/migrations-v0_13_0.test.ts": 1,- "test/migrations-v0_13_1-grandfather.test.ts": 2750,"test/migrations-v0_13_1.test.ts": 1,"test/migrations-v0_14_0.test.ts": 3,"test/migrations-v0_16_0.test.ts": 7,diff --git a/scripts/ubicloud/weights.json b/scripts/ubicloud/weights.jsonindex 88220440cc..b0bdd521d0 100644--- a/scripts/ubicloud/weights.json+++ b/scripts/ubicloud/weights.json@@ -182,7 +182,6 @@"e2e:test/e2e/graduation-crash-run-3.test.ts": 110586,"e2e:test/e2e/graduation-faults.test.ts": 30529,"e2e:test/e2e/graduation-legacy-copy.test.ts": 18279,- "e2e:test/e2e/grandfather-projection-postgres.test.ts": 6421,"e2e:test/e2e/graph-quality.test.ts": 2683,"e2e:test/e2e/graph-signals-engine.test.ts": 1761,"e2e:test/e2e/graph-signals-eval.test.ts": 4371,@@ -2053,7 +2052,6 @@"unit:test/graduation-state.test.ts": 94074,"unit:test/graduation-target.test.ts": 179,"unit:test/graduation-verify.test.ts": 10383,- "unit:test/grandfather-capacity.test.ts": 2420,"unit:test/grandfather-managed.test.ts": 23884,"unit:test/graph-query.test.ts": 2605,"unit:test/group-publish-core-exclusion.test.ts": 219,@@ -2388,7 +2386,6 @@"unit:test/migrations-v0_12_0.test.ts": 322,"unit:test/migrations-v0_12_2.test.ts": 463,"unit:test/migrations-v0_13_0.test.ts": 293,- "unit:test/migrations-v0_13_1-grandfather.test.ts": 5881,"unit:test/migrations-v0_13_1.test.ts": 322,"unit:test/migrations-v0_14_0.test.ts": 372,"unit:test/migrations-v0_16_0.test.ts": 441,diff --git a/src/commands/migrations/v0_13_1.ts b/src/commands/migrations/v0_13_1.tsindex 78bb84869e..a25fc6d62d 100644--- a/src/commands/migrations/v0_13_1.ts+++ b/src/commands/migrations/v0_13_1.ts@@ -1,403 +1,57 @@/**- * v0.13.0 migration — grandfather `validate: false` onto existing pages.+ * v0.13.1 migration — the retired `validate: false` grandfather (#7814).*- * The Knowledge Runtime BrainWriter ships pre-commit citation / link /- * back-link / triple-HR validators. A fresh brain passes them trivially.- * An existing brain with years of accumulated pages does NOT — legitimate- * pages without strict citation formatting exist all over the place.+ * v0.13.1 used to walk every page and write `validate: false` into its+ * frontmatter, so that BrainWriter's citation / link / back-link / triple-HR+ * validators would not reject legacy pages. That write changed every page's+ * content_hash. On a repo brain the database then disagreed with Git (the+ * unmanaged path), or the stamp was committed into the user's files (the+ * managed path).*- * This migration walks every page and adds `validate: false` to frontmatter- * where the field isn't already present. Pages with that flag bypass the- * validators entirely, so strict-mode rollout doesn't break existing- * content. `gbrain integrity --auto` clears the flag per-page as it writes- * proper citations.+ * Orchestrators apply by ledger alone: a version with no row in+ * ~/.gbrain/migrations/completed.jsonl is pending. A brain whose ledger never+ * recorded v0.13.1 (created by `gbrain init` before init stamped+ * fresh-install migrations, or reached from a new GBRAIN_HOME) therefore ran+ * the grandfather on upgrade against pages written long after v0.13.1, and+ * the upgrade rewrote the user's frontmatter.*- * Idempotency: pages that already have `validate: false` or `validate: true`- * are skipped. Running twice is a no-op on the second pass.+ * Nothing needs the stamp: the only BrainWriter in the product runs with+ * `strictMode: 'off'` (src/commands/integrity.ts), so a page without+ * `validate` is never refused. A `validate` value a page already carries+ * keeps its meaning (integrity scans still skip `validate: false`). The+ * orchestrator now records completion and writes nothing; a managed+ * grandfather write an older binary queued is refused without writing+ * (src/core/persistence/grandfather.ts).*- * Reversibility: every page touched is logged to- * ~/.gbrain/migrations/v0_13_1-rollback.jsonl with its pre-migration- * frontmatter snapshot. Roll back by re-applying those snapshots via- * `gbrain apply-migrations --rollback v0.13.0` (future CLI; not in scope).- *- * Scale (v0.41.37.0 #1581): chunked bulk SQL — one id-snapshot SELECT + N- * (SELECT-for-rollback + UPDATE) statements of CHUNK_SIZE rows each. Completes- * in ~1-2s even on an 82K-page PGLite brain. The prior per-page- * getPage+putPage loop hung CPU-bound for 70+ min on that brain (#1581).- *- * Snapshot rule: the affected id set is read once via SQL up front. It isn't- * invalidated by our writes because each UPDATE flips its rows out of the- * GRANDFATHER_WHERE predicate (idempotent + resumable).- *- * Safety: does NOT call saveConfig. Prior learning [gbrain-init-default-pglite-flip]:- * bare `gbrain init` defaults to PGLite and overwrites Postgres config.- * This migration uses the standalone engine-factory flow with the existing- * config; it never writes config.+ * Pages an older release stamped keep the stamp. Their pre-stamp+ * frontmatter is in ~/.gbrain/migrations/v0_13_1-rollback.jsonl on the+ * machine that ran it.*/-import { existsSync, mkdirSync, appendFileSync, chmodSync } from 'fs';-import { join } from 'path';--import type { Migration, OrchestratorOpts, OrchestratorResult, OrchestratorPhaseResult } from './types.ts';-import { loadConfig, toEngineConfig, gbrainPath } from '../../core/config.ts';-import { createEngine } from '../../core/engine-factory.ts';-import type { BrainEngine } from '../../core/engine.ts';-import { managedPersistenceEnabled } from '../../core/persistence/ownership.ts';-import { admitCanonicalGrandfather, assertGrandfatherCapacity } from '../../core/persistence/grandfather.ts';-import { OperationError } from '../../core/ops/contract.ts';-// Bug 3 — ledger writes moved to the runner (apply-migrations.ts).--// Lazy: GBRAIN_HOME may be set after module load.-const getRollbackDir = () => gbrainPath('migrations');-const getRollbackFile = () => join(getRollbackDir(), 'v0_13_1-rollback.jsonl');--// ----------------------------------------------------------------------------// Phase A — connect (no config write)-// -----------------------------------------------------------------------------async function phaseAConnect(opts: OrchestratorOpts): Promise<{ result: OrchestratorPhaseResult; engine: BrainEngine | null }> {- if (opts.dryRun) {- return { result: { name: 'connect', status: 'skipped', detail: 'dry-run' }, engine: null };- }- try {- const config = loadConfig();- if (!config) {- return {- result: { name: 'connect', status: 'skipped', detail: 'no brain configured (run gbrain init first)' },- engine: null,- };- }- const engine = await createEngine(toEngineConfig(config));- await engine.connect(toEngineConfig(config));- return { result: { name: 'connect', status: 'complete' }, engine };- } catch (e) {- return {- result: { name: 'connect', status: 'failed', detail: e instanceof Error ? e.message : String(e) },- engine: null,- };- }-}--// ----------------------------------------------------------------------------// Phase C — grandfather: add validate:false where absent-//-// v0.41.37.0 #1581: rewritten from a per-page getPage+putPage loop (which hung-// CPU-bound for 70+ min on an 82K-page PGLite brain) to a CHUNKED bulk SQL-// pass. Three correctness properties (the per-page loop and a naive single-// bulk UPDATE both got these wrong):-// 1. Keyed on pages.id (globally unique PK), NOT slug. `slug` uniqueness is-// (source_id, slug) — a slug-batched UPDATE would mutate same-slug pages-// across other sources and produce ambiguous rollback rows.-// 2. Filters `deleted_at IS NULL` — the old getPage path hid soft-deleted-// rows; a raw UPDATE must not grandfather tombstones.-// 3. Chunked in CHUNK_SIZE batches so lock-hold stays bounded (the-// DELETE_BATCH_SIZE convention) instead of one giant transaction.-// The rollback log carries source identity ({id, slug, source_id,-// pre_frontmatter}) so a rollback is unambiguous across sources.-// -----------------------------------------------------------------------------// Filter for pages still needing the grandfather flag. Literal `?` jsonb-// existence operator (matches src/core/embed-skip.ts convention); 'validate'-// is a hardcoded literal, no injection surface. COALESCE for null-frontmatter-// safety. deleted_at IS NULL skips soft-deleted tombstones.-const GRANDFATHER_WHERE =- "NOT (COALESCE(frontmatter, '{}'::jsonb) ? 'validate') AND deleted_at IS NULL";--// Per-chunk row count. Bounded lock-hold + write-amplification per statement-// (same rationale as engine-constants.ts DELETE_BATCH_SIZE).-const CHUNK_SIZE = 1000;--// Grandfather writes admitted before the step waits for them (#5530).-const GRANDFATHER_WINDOW = 50;--/** A page this run grandfathered and the revision its write produced. */-export interface GrandfatheredPage { id: number; revision: string | null; }--interface GrandfatherResult {- touched: number;- skipped: number;- failed: number;- failures: string[];- grandfathered: GrandfatheredPage[];-}--// Exported for direct hermetic testing against a PGLite engine (the config /-// loadConfig flow is exercised separately). Internal helper otherwise.-export async function phaseCGrandfather(- engine: BrainEngine,- opts: OrchestratorOpts,-): Promise<{ result: OrchestratorPhaseResult; detail: GrandfatherResult }> {- const gf: GrandfatherResult = { touched: 0, skipped: 0, failed: 0, failures: [], grandfathered: [] };-- try {- if (opts.dryRun) {- const rows = await engine.executeRaw<{ count: string | number }>(- `SELECT COUNT(*) AS count FROM pages WHERE ${GRANDFATHER_WHERE}`,- );- const c = rows[0]?.count ?? 0;- gf.touched = typeof c === 'string' ? parseInt(c, 10) : Number(c);- return {- result: { name: 'grandfather', status: 'complete', detail: `would touch ${gf.touched} (dry-run)` },- detail: gf,- };- }-- ensureRollbackDir();-- // Snapshot the affected id set up front (ids only — cheap, no frontmatter- // blobs in memory). The snapshot isn't invalidated by our writes because- // each UPDATE flips the rows out of the GRANDFATHER_WHERE predicate.- const idRows = await engine.executeRaw<{ id: number }>(- `SELECT id FROM pages WHERE ${GRANDFATHER_WHERE} ORDER BY id`,- );- const ids = idRows.map(r => Number(r.id));- const managed = await managedPersistenceEnabled(engine);- if (managed) {- // Only pages the managed pass can admit need a request ID: archived- // sources, code and image pages and non-Markdown files are skipped below.- const [{ n: admissible }] = await engine.executeRaw<{ n: number }>(`SELECT count(*)::int AS n FROM pages p- JOIN sources s ON s.id=p.source_id WHERE p.id=ANY($1::int[]) AND NOT s.archived AND p.type NOT IN ('code','image')- AND NOT (COALESCE(p.source_path,'') ~ '\\.[^./]+$' AND COALESCE(p.source_path,'') !~* '\\.mdx?$')`, [ids]);- try { await assertGrandfatherCapacity(engine, Number(admissible)); }- catch (error) {- if (!(error instanceof OperationError) || !['queue_capacity', 'write_quota_exhausted', 'write_quota_insufficient'].includes(error.code)) throw error;- return { result: { name: 'grandfather', status: 'failed', detail: `${error.code}: ${error.message} ${error.suggestion ?? ''}`.trim() }, detail: gf };- }- }-- for (let i = 0; i < ids.length; i += CHUNK_SIZE) {- const chunk = ids.slice(i, i + CHUNK_SIZE);- try {- if (managed) {- const selected = await engine.executeRaw<{ id: number; slug: string; source_id: string; source_incarnation: string }>(- 'SELECT p.id,p.slug,p.source_id,s.incarnation AS source_incarnation FROM pages p JOIN sources s ON s.id=p.source_id WHERE p.id=ANY($1::int[]) AND NOT s.archived ORDER BY p.id', [chunk]);- gf.skipped += chunk.length - selected.length;- const fail = (page: { id: number }, error: unknown) => {- gf.failed++;- const reason = error instanceof OperationError- ? `${error.code}${error.writeRequest ? ` request_id=${error.writeRequest.request_id}` : ''}`- : error instanceof Error ? error.message : String(error);- gf.failures.push(`page#${page.id}: ${reason}`.slice(0, 120));- };- // #5530: admit a bounded window before waiting (below the principal's- // outstanding-request limit), so the pages' Git effects are ready- // together and the effect runner commits and pushes them as groups.- for (let w = 0; w < selected.length; w += GRANDFATHER_WINDOW) {- const admitted: { page: typeof selected[number]; complete: () => Promise<{ revision: string | null }> }[] = [];- for (const page of selected.slice(w, w + GRANDFATHER_WINDOW)) {- try {- const outcome = await admitCanonicalGrandfather(engine, page, snapshot => appendRollbackBatch([snapshot]));- if (outcome.status === 'skipped') gf.skipped++;- else admitted.push({ page, complete: outcome.complete });- } catch (error) { fail(page, error); }- }- for (const { page, complete } of admitted) {- try {- const outcome = await complete();- gf.touched++;- gf.grandfathered.push({ id: page.id, revision: outcome.revision });- } catch (error) { fail(page, error); }- }- }- continue;- }- // Rollback log BEFORE mutation: one SELECT per chunk (bounded memory),- // one appendFileSync per chunk. Carries source_id so rollback is- // unambiguous across same-slug-different-source pages.- const touched = await engine.transaction(async tx => {- const keys = await tx.executeRaw<{ id: number; slug: string; source_id: string }>(- 'SELECT id, slug, source_id FROM pages WHERE id = ANY($1::int[])', [chunk]);- await tx.lockPageKeys(keys.map(row => ({ sourceId: row.source_id, slug: row.slug })));- const snap = await tx.executeRaw<{- id: number; slug: string; source_id: string; frontmatter: Record<string, unknown> | null;- knowledge_revision: string; text_projection_revision: string | null;- }>(`SELECT id, slug, source_id, frontmatter, knowledge_revision, text_projection_revision- FROM pages WHERE id = ANY($1::int[]) AND ${GRANDFATHER_WHERE} FOR UPDATE`, [chunk]);- const identities = new Map(keys.map(row => [row.id, row]));- if (snap.some(row => identities.get(row.id)?.source_id !== row.source_id || identities.get(row.id)?.slug !== row.slug)) {- throw new Error('Page identity changed during grandfathering; retry the migration.');- }- appendRollbackBatch(snap);- // The revision trigger advances knowledge_revision on this update;- // verification keys on it to tell a later rewrite from a lost write.- const written = await tx.executeRaw<{ id: number; knowledge_revision: string }>(- `UPDATE pages SET frontmatter = jsonb_set(COALESCE(frontmatter, '{}'::jsonb), '{validate}', 'false'::jsonb) ` +- 'WHERE id = ANY($1::int[]) RETURNING id, knowledge_revision::text AS knowledge_revision', [snap.map(row => row.id)]);- const sealed = snap.filter(row => row.text_projection_revision === row.knowledge_revision).map(row => row.id);- if (sealed.length) {- await tx.executeRaw('UPDATE pages SET text_projection_revision=knowledge_revision WHERE id = ANY($1::int[])', [sealed]);- }- return written.map(row => ({ id: Number(row.id), revision: row.knowledge_revision }));- });- gf.touched += touched.length;- gf.skipped += chunk.length - touched.length;- gf.grandfathered.push(...touched);- } catch (e) {- gf.failed += chunk.length;- const msg = e instanceof Error ? e.message : String(e);- gf.failures.push(`chunk@${i}: ${msg.slice(0, 100)}`);- }- }- } catch (e) {- gf.failed += 1;- gf.failures.push(`grandfather: ${e instanceof Error ? e.message : String(e)}`.slice(0, 120));- }+import type { Migration, OrchestratorOpts, OrchestratorResult } from './types.ts';- const status: OrchestratorPhaseResult['status'] = gf.failed > 0 ? 'failed' : 'complete';- const detailStr = `touched=${gf.touched} skipped=${gf.skipped} failed=${gf.failed}${gf.failed ? `; ${gf.failures.slice(0, 2).join('; ')}` : ''}`;+async function orchestrator(opts: OrchestratorOpts): Promise<OrchestratorResult> {return {- result: { name: 'grandfather', status, detail: detailStr },- detail: gf,+ version: '0.13.1',+ status: 'complete',+ phases: [{+ name: 'grandfather',+ status: 'skipped',+ detail: `retired (#7814): an upgrade writes nothing to pages; validate: false is no longer added to existing pages${opts.dryRun ? ' (dry-run)' : ''}`,+ }],+ files_rewritten: 0,};}-// ----------------------------------------------------------------------------// Phase D — verify-//-// Checks only the pages this run grandfathered. A page whose knowledge_revision-// moved past the one our write produced (or that was deleted) was rewritten by-// a concurrent writer, such as a connector re-import; it is reported as-// `rewritten_concurrently`, not failed. A page still at our revision without-// `validate: false` means the write did not land, and fails the phase. A null-// revision (a replayed receipt that recorded none) cannot prove a rewrite, so-// the flag must be present.-// -----------------------------------------------------------------------------// Exported for direct hermetic testing, like phaseCGrandfather.-export async function phaseDVerify(engine: BrainEngine, grandfathered: GrandfatheredPage[]): Promise<OrchestratorPhaseResult> {- if (grandfathered.length === 0) {- return { name: 'verify', status: 'complete', detail: 'nothing to verify' };- }- try {- let verified = 0;- let rewritten = 0;- const missing: number[] = [];- for (let i = 0; i < grandfathered.length; i += CHUNK_SIZE) {- const chunk = grandfathered.slice(i, i + CHUNK_SIZE);- const rows = await engine.executeRaw<{ id: number; state: 'verified' | 'rewritten' | 'missing' }>(- `SELECT t.id,- CASE WHEN p.id IS NULL OR (t.revision IS NOT NULL AND p.knowledge_revision::text <> t.revision) THEN 'rewritten'- WHEN p.frontmatter->>'validate' = 'false' THEN 'verified'- ELSE 'missing' END AS state- FROM unnest($1::int[], $2::text[]) AS t(id, revision)- LEFT JOIN pages p ON p.id = t.id`,- [chunk.map(page => page.id), chunk.map(page => page.revision)],- );- for (const row of rows) {- if (row.state === 'verified') verified++;- else if (row.state === 'rewritten') rewritten++;- else missing.push(Number(row.id));- }- }- const detail = `verified=${verified} rewritten_concurrently=${rewritten}`;- return missing.length === 0- ? { name: 'verify', status: 'complete', detail }- : { name: 'verify', status: 'failed',- detail: `${detail} missing_validate=${missing.length} (${missing.slice(0, 5).map(id => `page#${id}`).join(', ')})` };- } catch (e) {- return {- name: 'verify',- status: 'failed',- detail: e instanceof Error ? e.message : String(e),- };- }-}--// ----------------------------------------------------------------------------// Orchestrator-// -----------------------------------------------------------------------------async function orchestrator(opts: OrchestratorOpts): Promise<OrchestratorResult> {- const phases: OrchestratorPhaseResult[] = [];- let filesRewritten = 0;-- const { result: connectRes, engine } = await phaseAConnect(opts);- phases.push(connectRes);- if (connectRes.status !== 'complete' || !engine) {- return {- version: '0.13.1',- status: connectRes.status === 'skipped' ? 'partial' : 'failed',- phases,- };- }-- try {- // v0.41.37.0 #1581: phaseBSnapshot (getAllSlugs) is gone — the chunked bulk- // pass filters via SQL (GRANDFATHER_WHERE), so we no longer materialize a- // full slug list in JS.- const { result: gfRes, detail: gfDetail } = await phaseCGrandfather(engine, opts);- phases.push(gfRes);- filesRewritten = gfDetail.touched;-- if (!opts.dryRun) {- const verifyRes = await phaseDVerify(engine, gfDetail.grandfathered);- phases.push(verifyRes);- }-- const anyFailed = phases.some(p => p.status === 'failed');- const status: OrchestratorResult['status'] = anyFailed ? 'partial' : 'complete';-- // Bug 3 — ledger write lives in the runner now.-- return {- version: '0.13.1',- status,- phases,- files_rewritten: filesRewritten,- };- } finally {- try { await engine.disconnect(); } catch {}- }-}--// ----------------------------------------------------------------------------// Helpers-// -----------------------------------------------------------------------------function ensureRollbackDir(): void {- const dir = getRollbackDir();- if (!existsSync(dir)) mkdirSync(dir, { recursive: true, mode: 0o700 });-}--// v0.41.37.0 #1581: batch rollback writer. One appendFileSync per chunk, bounded-// memory. Each line carries id + slug + source_id so a rollback is unambiguous-// across same-slug-different-source pages (pages.slug is not globally unique).-function appendRollbackBatch(- rows: ReadonlyArray<{ id: number; slug: string; source_id: string | null; frontmatter: Record<string, unknown> | null;- source_incarnation?: string; knowledge_revision?: string; request_id?: string }>,-): void {- if (rows.length === 0) return;- const ts = new Date().toISOString();- const lines = rows.map(r => JSON.stringify({- migration: 'v0.13.0',- timestamp: ts,- id: r.id,- slug: r.slug,- source_id: r.source_id ?? 'default',- source_incarnation: r.source_incarnation,- knowledge_revision: r.knowledge_revision,- request_id: r.request_id,- pre_frontmatter: r.frontmatter ?? {},- })).join('\n') + '\n';- appendFileSync(getRollbackFile(), lines, { encoding: 'utf-8', mode: 0o600 });- chmodSync(getRollbackFile(), 0o600);-}--// ----------------------------------------------------------------------------// Export-// ----------------------------------------------------------------------------export const v0_13_1: Migration = {version: '0.13.1',fresh_install_noop: true,featurePitch: {- headline: 'BrainWriter integrity + grandfather protection for existing pages.',+ headline: 'BrainWriter integrity validators (no page is rewritten on upgrade).',description:- 'Adds `validate: false` to existing pages so the new Knowledge Runtime ' +- 'validators (citation / link / back-link / triple-HR) don’t reject legacy ' +- 'content. Pages keep passing writes through unchanged; `gbrain integrity ' +- '--auto` clears the flag per-page once citations are repaired. Rollback ' +- 'log at ~/.gbrain/migrations/v0_13_1-rollback.jsonl.',+ 'The Knowledge Runtime BrainWriter validators (citation / link / back-link / ' ++ 'triple-HR) shipped in v0.13.1. Upgrading records this migration and changes no page: ' ++ 'the validators never refuse a write, so existing pages need no `validate: false` stamp (#7814).',},orchestrator,};diff --git a/src/commands/migrations/v0_28_0.ts b/src/commands/migrations/v0_28_0.tsindex 0797534e03..685639ab7a 100644--- a/src/commands/migrations/v0_28_0.ts+++ b/src/commands/migrations/v0_28_0.ts@@ -31,6 +31,7 @@ import type {Migration, OrchestratorOpts, OrchestratorResult, OrchestratorPhaseResult,} from './types.ts';import type { BrainEngine } from '../../core/engine.ts';+import { OperationError } from '../../core/ops/contract.ts';import { loadConfig, gbrainPath } from '../../core/config.ts';import { connectMigrationEngine } from './run-engine.ts';@@ -128,7 +129,8 @@ async function phaseBBackfill(: `extract-takes scanned ${result.pagesScanned} pages; ${result.pagesWithTakes} had fenced takes; upserted ${result.takesUpserted} rows`;return { name: 'backfill', status: 'complete', detail };} catch (e) {- return { name: 'backfill', status: 'failed', detail: e instanceof Error ? e.message : String(e) };+ const detail = e instanceof OperationError ? `${e.code}: ${e.message} ${e.suggestion ?? ''}`.trim() : e instanceof Error ? e.message : String(e);+ return { name: 'backfill', status: 'failed', detail };}}diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex 59b8c1aa07..60c54f5a09 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1320,6 +1320,8 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.471.0', id: 'door_planned_state', text: 'On a source with `canonical: git`, a page write now commits exactly the bytes it published, or nothing. A file edited on disk while the write was publishing refuses `git_target_changed`: nothing is committed, the edit stays on disk and the source\'s writes pause until `gbrain repair stuck-recovery` resolves it (before, Git committed the edit while the database recorded the write\'s content). A staged change of the page\'s file refuses `git_index_conflict`, with the index untouched; commit or unstage it, then retry (before, the write replaced the staged change in the index). When the index still holds the version from before an earlier gbrain commit whose index update failed, the next write restores it and says so with a `git_index_restored` notice.' },// #7679 / #7685 orphaned worktree refresh. `since` is stamped at the merge slot.{ since: '0.60.472.0', id: 'refresh_orphan_7679', text: 'A managed `gbrain sources refresh` whose process died (for example a container replaced mid-refresh) no longer wedges the checkout: the next plain `gbrain sources refresh` on the owner host finishes the orphaned refresh through the `--resume` path before its own refresh and reports it as `recovered`, and the owner completes an orphaned `syncing` refresh once every member has synced to its target. A refresh whose holder is still running, or is recorded on another host, is never taken over. `gbrain sources writer status` and `gbrain sync status` now name the holder and its `liveness` (`live`, `orphaned`, `unknown`) with a fix naming `--resume`. With `--json`, a refusal that carries a structured fix (`refresh_in_progress`, `refresh_diverged`, a git read failure) now prints `fix` as an object (argv, actor, next) instead of a string; the prose stays in `suggestion`, so read that when a script needs text. A plain refresh of a checkout whose interrupted refresh needs a reset decision now refuses `refresh_recovery_required` (it refused `refresh_in_progress`).' },+ // #7814 retired v0.13.1 grandfather. `since` is stamped at the merge slot.+ { since: '0.60.476.0', id: 'v0_13_1_grandfather_retired', text: 'Upgrading a brain whose migration ledger never recorded v0.13.1 (one created by `gbrain init` before v0.60.412.0, or opened from a new GBRAIN_HOME) no longer writes `validate: false` into every page\'s frontmatter: `gbrain apply-migrations` records v0.13.1 complete and changes no page, so no content_hash moves and a repo brain\'s files and Git history stay as the user wrote them. Nothing needed the stamp, because the BrainWriter validators never refuse a write. A page that already carries `validate` keeps it, and a grandfather write an older gbrain left queued ends `conflict` without writing. Pages an older release stamped keep the stamp; their pre-stamp frontmatter is in `~/.gbrain/migrations/v0_13_1-rollback.jsonl` on the machine that ran it.' },];/** The newest disclosed change's release: the notice id moves only when a release adds rows. */diff --git a/src/core/persistence/grandfather.ts b/src/core/persistence/grandfather.tsindex edc486c1d0..29d8284008 100644--- a/src/core/persistence/grandfather.ts+++ b/src/core/persistence/grandfather.ts@@ -1,141 +1,16 @@-import { withTrustKeep } from './context.ts';-import { randomUUID } from 'node:crypto';-import { extname } from 'node:path';import type { BrainEngine } from '../engine.ts';-import { loadConfig } from '../config.ts';-import { contentHash } from '../utils.ts';-import { serializePageToMarkdown } from '../markdown.ts';-import { opError, OperationError, type OperationContext } from '../ops/contract.ts';-import { assertPageRevision } from '../page-state/types.ts';-import { initializeLocalPersistence, requestPrincipalForContext } from './page-mutations.ts';-import { submissionAuthority, authorizeStoredRequest } from './authority.ts';-import { admitWrite, assertLifetimeIdHeadroom, assertReplayIntent, getWriteRequest, intentDigest } from './journal.ts';-import { getWorktreeBinding } from './ownership.ts';-import { digest, requireUuid } from './digest.ts';-import { waitForWrite, writeResponse } from './service.ts';-import { maintenancePublishWaitMs } from './maintenance-wait.ts';-import { databaseOnlyPublication, prepareFileTarget } from './page-prepare.ts';+import { opError } from '../ops/contract.ts';import type { WriteRequest } from './model.ts';import type { PreparedMutation } from './coordinator.ts';-type GrandfatherSelection = { id: number; slug: string; source_id: string; source_incarnation: string };-type GrandfatherOutcome = { status: 'touched'; revision: string | null };--export async function grandfatherCanonicalPage(engine: BrainEngine, selected: GrandfatherSelection,- before: (page: GrandfatherSelection & { frontmatter: Record<string, unknown>; knowledge_revision: string; request_id: string }) => void | Promise<void>,-): Promise<GrandfatherOutcome | { status: 'skipped' }> {- const admitted = await admitCanonicalGrandfather(engine, selected, before);- return admitted.status === 'skipped' ? admitted : admitted.complete();-}-/**- * #5530: admit one page's grandfather write without waiting for it. The step- * admits a bounded window of pages before waiting, so their Git effects are- * ready together and the effect runner commits them as one group instead of- * one commit per page. `complete()` waits for the publication and settles the- * page's checkpoint exactly as the serial path does.+ * #7814: the v0.13.1 grandfather no longer writes `validate: false` into pages+ * (src/commands/migrations/v0_13_1.ts). A `managed_grandfather` request an+ * older binary admitted and left queued ends here without writing, so an+ * upgrade never finishes a grandfather rewrite of the user's files.*/-/** Refuse a managed grandfather pass up front when its page admissions cannot fit the ID caps. */-export async function assertGrandfatherCapacity(engine: BrainEngine, pages: number): Promise<void> {- const ctx: OperationContext = { engine, config: loadConfig() ?? { engine: engine.kind }, sourceId: 'default', remote: false, dryRun: false,- logger: { info() {}, warn() {}, error() {} } };- await initializeLocalPersistence(ctx);- await assertLifetimeIdHeadroom(engine, await requestPrincipalForContext(ctx), pages);-}--export async function admitCanonicalGrandfather(engine: BrainEngine, selected: GrandfatherSelection,- before: (page: GrandfatherSelection & { frontmatter: Record<string, unknown>; knowledge_revision: string; request_id: string }) => void | Promise<void>,-): Promise<{ status: 'skipped' } | { status: 'admitted'; complete: () => Promise<GrandfatherOutcome> }> {- const snapshot = await engine.readPageSnapshot(selected.slug, { sourceId: selected.source_id });- if (!snapshot || snapshot.page.id !== selected.id || snapshot.sourceIncarnation !== selected.source_incarnation ||- Object.hasOwn(snapshot.page.frontmatter ?? {}, 'validate')) return { status: 'skipped' };- const extension = snapshot.page.source_path ? extname(snapshot.page.source_path).toLowerCase() : '';- if (['code', 'image'].includes(snapshot.page.type) || extension && !['.md', '.mdx'].includes(extension)) return { status: 'skipped' };- const ctx: OperationContext = { engine, config: loadConfig() ?? { engine: engine.kind }, sourceId: selected.source_id,- remote: false, dryRun: false, logger: { info() {}, warn() {}, error() {} } };- await initializeLocalPersistence(ctx);- const principal = await requestPrincipalForContext(ctx);- const authority = await submissionAuthority(ctx, 'put_page', selected.source_id, selected.source_incarnation, selected.slug);- const callerIntent = { migration: '0.13.1', ...selected, expected_revision: snapshot.revision };- const fingerprint = digest({ principal, ...callerIntent });- const op = 'canonical-grandfather-0.13.1';- await engine.executeRaw('INSERT INTO op_checkpoints(op,fingerprint,completed_keys) VALUES($1,$2,$3::text::jsonb) ON CONFLICT DO NOTHING',- [op, fingerprint, JSON.stringify([randomUUID()])]);- const [checkpoint] = await engine.executeRaw<{ completed_keys: string[] }>('SELECT completed_keys FROM op_checkpoints WHERE op=$1 AND fingerprint=$2', [op, fingerprint]);- const requestId = requireUuid(checkpoint.completed_keys[0]);- const prior = await getWriteRequest(engine, principal, requestId);- if (prior) {- await authorizeStoredRequest(engine, prior);- assertReplayIntent(prior, intentDigest({ operation: 'put_page', sourceId: selected.source_id, slug: selected.slug, callerIntent }));- } else await before({ ...selected, frontmatter: snapshot.page.frontmatter ?? {}, knowledge_revision: snapshot.revision, request_id: requestId });- const binding = await getWorktreeBinding(engine, selected.source_id);- const settleFailure = async (error: unknown): Promise<never> => {- if (error instanceof OperationError && error.writeRequest && ['failed', 'conflict', 'cancelled'].includes(error.writeRequest.state)) {- await engine.executeRaw('DELETE FROM op_checkpoints WHERE op=$1 AND fingerprint=$2 AND completed_keys=$3::text::jsonb', [op, fingerprint, JSON.stringify([requestId])]);- }- throw error;- };- let request: WriteRequest;- try {- request = prior ?? await admitWrite(engine, { principal, operation: 'put_page', sourceId: selected.source_id,- sourceIncarnation: selected.source_incarnation, pageId: selected.id, slug: selected.slug, requestId, authority, callerIntent,- intent: { kind: 'managed_grandfather', expected_revision: snapshot.revision },- worktreeId: binding?.worktree_id, topologyGeneration: binding?.topology_generation });- } catch (error) { return settleFailure(error); }- return { status: 'admitted', complete: async () => {- try {- const committed = writeResponse(await waitForWrite(engine, request, ctx.config, maintenancePublishWaitMs()));- await engine.executeRaw('DELETE FROM op_checkpoints WHERE op=$1 AND fingerprint=$2 AND completed_keys=$3::text::jsonb', [op, fingerprint, JSON.stringify([requestId])]);- // The publication revision lets verification tell a later rewrite from a lost grandfather.- return { status: 'touched', revision: typeof committed.revision === 'string' ? committed.revision : null };- } catch (error) { return settleFailure(error); }- } };-}--export async function prepareGrandfatherMutation(engine: BrainEngine, row: WriteRequest): Promise<PreparedMutation> {- if (row.operation !== 'put_page' || row.intent?.kind !== 'managed_grandfather' || row.authority.remote !== false || row.authority.principal.kind !== 'local_cli') {- throw opError('permission_denied', 'Grandfathering requires the trusted canonical migration path.',- `Request ${row.request_id} is not a trusted local grandfather write, so nothing changed. Grandfathering runs only from gbrain apply-migrations on the brain host.`);- }- await authorizeStoredRequest(engine, row);- const snapshot = await engine.readPageSnapshot(row.slug, { sourceId: row.source_id });- if (!snapshot || snapshot.page.id !== row.page_id) {- throw opError('page_identity_changed', 'The grandfathered page identity changed.',- `${row.slug} in ${row.source_id} was deleted or replaced after grandfathering was queued, so it was skipped; the next migration run re-plans from current pages.`);- }- assertPageRevision(snapshot, { expectedRevision: row.intent.expected_revision as string });- if (Object.hasOwn(snapshot.page.frontmatter ?? {}, 'validate')) {- throw opError('revision_conflict', 'A validation decision already exists.',- `${row.slug} in ${row.source_id} already records a validate decision, so grandfathering left it unchanged; nothing else is needed.`);- }- if (snapshot.page.frontmatter != null && (typeof snapshot.page.frontmatter !== 'object' || Array.isArray(snapshot.page.frontmatter))) {- throw opError('invalid_params', 'Grandfathering requires valid object frontmatter.',- `${row.slug} in ${row.source_id} has frontmatter that is not a YAML mapping. Fix the page's frontmatter, then run the migration again.`);- }- const frontmatter = { ...snapshot.page.frontmatter, validate: false };- const hash = contentHash({ ...snapshot.page, frontmatter, tags: snapshot.tags });- const file = await prepareFileTarget(engine, row, snapshot, serializePageToMarkdown({ ...snapshot.page, frontmatter }, snapshot.tags));- if (file && !['.md', '.mdx'].includes(extname(file.path).toLowerCase())) {- throw opError('invalid_params', 'Non-Markdown artifacts cannot be grandfathered by rewriting their bytes.',- `${row.slug} in ${row.source_id} is backed by a non-Markdown file, so grandfathering skips it rather than rewrite its bytes; no action is needed.`);- }- return { observedRevision: snapshot.revision, file, ...databaseOnlyPublication(row, file), deferEmbedding: true, apply: async tx => {- await tx.createVersion(row.slug, { sourceId: row.source_id });- // A mechanical validate:false stamp authors nothing: the page keeps its trust tier (#5575).- const updated = await withTrustKeep(tx, ['pages'], () => tx.executeRaw(`UPDATE pages SET frontmatter=jsonb_set(COALESCE(frontmatter,'{}'::jsonb),'{validate}','false'::jsonb),content_hash=$4- WHERE id=$1 AND source_id=$2 AND knowledge_revision=$3::uuid AND NOT(COALESCE(frontmatter,'{}'::jsonb)?'validate') RETURNING id`,- [row.page_id, row.source_id, snapshot.revision, hash]));- if (updated.length !== 1) {- throw opError('revision_conflict', 'The validation decision changed before publication.',- `${row.slug} in ${row.source_id} changed while it was being grandfathered, so the transaction rolled back; the next migration run re-plans from current pages.`);- }- if (snapshot.page.text_projection_revision === snapshot.revision) {- await tx.executeRaw('UPDATE pages SET text_projection_revision=knowledge_revision WHERE id=$1 AND source_id=$2', [row.page_id, row.source_id]);- }- await tx.executeRaw(`INSERT INTO extract_atoms_page_state(source_incarnation,page_id,content_hash,fail_count,tombstoned,updated_at)- SELECT source_incarnation,page_id,$4,fail_count,tombstoned,updated_at FROM extract_atoms_page_state- WHERE source_incarnation=$1::uuid AND page_id=$2 AND content_hash=$3 ON CONFLICT DO NOTHING`,- [row.source_incarnation, row.page_id, snapshot.page.content_hash, hash]);- return { status: 'grandfathered', slug: row.slug, source_id: row.source_id };- } };+export async function prepareGrandfatherMutation(_engine: BrainEngine, row: WriteRequest): Promise<PreparedMutation> {+ throw opError('revision_conflict', 'Grandfathering is retired; the page was left unchanged.',+ `${row.slug} in ${row.source_id} was queued for the retired v0.13.1 validate: false stamp by an older gbrain; `+ + 'upgrades no longer rewrite pages (#7814), so nothing changed and nothing else is needed.');}diff --git a/test/e2e/grandfather-projection-postgres.test.ts b/test/e2e/grandfather-projection-postgres.test.tsdeleted file mode 100644index 4d8048ed16..0000000000--- a/test/e2e/grandfather-projection-postgres.test.ts+++ /dev/null@@ -1,285 +0,0 @@-import { describe, expect, test } from 'bun:test';-import { mkdtempSync, rmSync } from 'node:fs';-import { tmpdir } from 'node:os';-import { join } from 'node:path';-import { hasDatabase } from './helpers.ts';-import { isolatedPersistencePostgres } from '../helpers/persistence-postgres.ts';-import { installFixtureChunks } from '../helpers/page-projection.ts';-import { withEnv } from '../helpers/with-env.ts';-import { phaseCGrandfather, phaseDVerify } from '../../src/commands/migrations/v0_13_1.ts';-import type { BrainEngine } from '../../src/core/engine.ts';--function afterTransactionQuery(engine: BrainEngine, after: (tx: BrainEngine, query: string) => Promise<void>): BrainEngine {- return new Proxy(engine, {- get(target, property) {- if (property === 'transaction') {- return <T>(run: (tx: BrainEngine) => Promise<T>) => target.transaction(tx => run(new Proxy(tx, {- get(transaction, key) {- if (key === 'executeRaw') {- return async <R>(query: string, params?: unknown[]) => {- const rows = await transaction.executeRaw<R>(query, params);- await after(transaction, query);- return rows;- };- }- const value = Reflect.get(transaction, key);- return typeof value === 'function' ? value.bind(transaction) : value;- },- })));- }- const value = Reflect.get(target, property);- return typeof value === 'function' ? value.bind(target) : value;- },- });-}--async function sealedPage(engine: BrainEngine, slug: string, text: string, frontmatter: Record<string, unknown> = {}) {- await engine.putPage(slug, {- type: 'concept', title: 'Migration Fixture', compiled_truth: text, timeline: '', frontmatter,- }, { sourceId: 'default' });- await installFixtureChunks(engine, slug, [{ chunk_index: 0, chunk_text: text, chunk_source: 'compiled_truth' }], { sourceId: 'default' });-}--function persistedPages(engine: BrainEngine) {- return engine.executeRaw(`SELECT id, source_id, slug, frontmatter, compiled_truth,- knowledge_revision, text_projection_revision, deleted_at FROM pages ORDER BY id`);-}--describe.skipIf(!hasDatabase())('Postgres grandfather migration projection publication', () => {- test('preserves sealed duplicate slugs across sources without certifying incomplete text', async () => {- const home = mkdtempSync(join(tmpdir(), 'gbrain-grandfather-pg-'));- const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);- const engine = pg.engine;- try {- await engine.executeRaw("INSERT INTO sources(id,name) VALUES ('secondary','Secondary')");- const slug = 'concepts/search-example';- const before = new Map<string, string | undefined>();- for (const sourceId of ['default', 'secondary']) {- await engine.putPage(slug, {- type: 'concept', title: 'Search Example', compiled_truth: 'amberbadger migration fixture', timeline: '', frontmatter: {},- }, { sourceId });- await installFixtureChunks(engine, slug, [{ chunk_index: 0, chunk_text: 'amberbadger migration fixture', chunk_source: 'compiled_truth' }], { sourceId });- before.set(sourceId, (await engine.getPage(slug, { sourceId }))?.knowledge_revision);- expect(await engine.searchKeyword('amberbadger', { sourceId })).toHaveLength(1);- }- await engine.putPage('concepts/unsealed-example', {- type: 'concept', title: 'Unsealed Example', compiled_truth: 'Incomplete projection.', timeline: '', frontmatter: {},- }, { sourceId: 'default' });- await withEnv({ GBRAIN_HOME: home }, async () => {- const result = await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true });- expect(result.result.status).toBe('complete');- expect(result.detail.touched).toBe(3);- for (const sourceId of ['default', 'secondary']) {- const page = await engine.getPage(slug, { sourceId });- expect(page?.knowledge_revision).not.toBe(before.get(sourceId));- expect(page?.text_projection_revision).toBe(page?.knowledge_revision);- expect(page?.frontmatter.validate).toBe(false);- expect(await engine.searchKeyword('amberbadger', { sourceId })).toHaveLength(1);- }- expect((await engine.getPage('concepts/unsealed-example', { sourceId: 'default' }))?.text_projection_revision).toBeNull();- expect(await engine.executeRaw('SELECT slug FROM page_projection_jobs')).toEqual([{ slug: 'concepts/unsealed-example' }]);- expect((await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true })).detail.touched).toBe(0);- });- } finally {- await pg.close();- rmSync(home, { recursive: true, force: true });- }- }, 60_000);-- test('verify keys on the revision each write produced, including a batch of unrecorded revisions', async () => {- const home = mkdtempSync(join(tmpdir(), 'gbrain-grandfather-verify-pg-'));- const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);- const engine = pg.engine;- try {- await sealedPage(engine, 'concepts/kept', 'Kept fixture.');- await sealedPage(engine, 'concepts/reimported', 'Reimported fixture.');- await withEnv({ GBRAIN_HOME: home }, async () => {- const { detail } = await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true });- expect(await phaseDVerify(engine, detail.grandfathered)).toMatchObject({ status: 'complete', detail: 'verified=2 rewritten_concurrently=0' });- // A connector re-import replaces the page and drops the flag.- await sealedPage(engine, 'concepts/reimported', 'Reimported again.');- expect(await phaseDVerify(engine, detail.grandfathered)).toMatchObject({ status: 'complete', detail: 'verified=1 rewritten_concurrently=1' });- await sealedPage(engine, 'concepts/unwritten', 'Never grandfathered.');- const kept = (await engine.getPage('concepts/kept', { sourceId: 'default' }))!;- const unwritten = (await engine.getPage('concepts/unwritten', { sourceId: 'default' }))!;- // A receipt without a revision cannot prove a rewrite, so the flag must be present.- expect(await phaseDVerify(engine, [{ id: kept.id, revision: null }, { id: unwritten.id, revision: null }])).toMatchObject({- status: 'failed', detail: `verified=1 rewritten_concurrently=0 missing_validate=1 (page#${unwritten.id})` });- });- } finally {- await pg.close();- rmSync(home, { recursive: true, force: true });- }- }, 60_000);-- test('waits for a canonical page guard and preserves the competing writer validation decision', async () => {- const home = mkdtempSync(join(tmpdir(), 'gbrain-grandfather-writer-pg-'));- const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);- const engine = pg.engine;- const releaseWriter = Promise.withResolvers<void>();- let writer: Promise<void> | undefined;- let migration: ReturnType<typeof phaseCGrandfather> | undefined;- try {- const slug = 'concepts/validation-example';- await sealedPage(engine, slug, 'cedarmarten original fixture');- const ready = Promise.withResolvers<void>();- let writerRows: Awaited<ReturnType<typeof persistedPages>> = [];- const coordinated = afterTransactionQuery(engine, async (_tx, query) => {- if (!query.startsWith('SELECT id, slug, source_id FROM pages') || writer) return;- writer = engine.transaction(async tx => {- await tx.lockPageKeys([{ sourceId: 'default', slug }]);- await sealedPage(tx, slug, 'topazotter competing fixture', { validate: true, decision: 'keep-validation' });- writerRows = await persistedPages(tx);- ready.resolve();- await releaseWriter.promise;- });- void writer.catch(ready.reject);- await ready.promise;- });- migration = withEnv({ GBRAIN_HOME: home }, () => phaseCGrandfather(coordinated, {- yes: true, dryRun: false, noAutopilotInstall: true,- }));- let blockedOnGuard = false;- try {- for (let attempt = 0; attempt < 200; attempt++) {- const waiting = await engine.executeRaw(`SELECT pid FROM pg_stat_activity- WHERE datname = current_database() AND wait_event_type = 'Lock'- AND query LIKE '%page_write_guards%'`);- if (waiting.length) {- blockedOnGuard = true;- break;- }- await Bun.sleep(25);- }- } finally {- releaseWriter.resolve();- }- await writer;- const result = await migration;- expect(blockedOnGuard).toBe(true);- expect(writerRows).toHaveLength(1);- expect(result.result.status).toBe('complete');- expect(result.detail).toMatchObject({ touched: 0, skipped: 1, failed: 0 });- expect(await persistedPages(engine)).toEqual(writerRows);- const page = await engine.getPage(slug, { sourceId: 'default' });- expect(page?.frontmatter).toEqual({ validate: true, decision: 'keep-validation' });- expect(page?.text_projection_revision).toBe(page?.knowledge_revision);- expect((await engine.searchKeyword('topazotter', { sourceId: 'default' })).map(hit => hit.slug)).toEqual([slug]);- expect(await engine.searchKeyword('cedarmarten', { sourceId: 'default' })).toHaveLength(0);- } finally {- releaseWriter.resolve();- await Promise.allSettled([writer, migration]);- await pg.close();- rmSync(home, { recursive: true, force: true });- }- }, 60_000);-- for (const change of ['delete', 'rename'] as const) {- test(`handles a selected identity ${change} without grandfathering its replacement`, async () => {- const home = mkdtempSync(join(tmpdir(), `gbrain-grandfather-${change}-pg-`));- const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);- const engine = pg.engine;- try {- const slug = 'concepts/identity-example';- const renamed = 'concepts/renamed-example';- const sibling = 'concepts/sibling-example';- await sealedPage(engine, slug, 'cedarmarten selected fixture');- await sealedPage(engine, sibling, 'violetfox sibling fixture');- const original = await engine.getPage(slug, { sourceId: 'default' });- let interleaved = false;- let writerRows: Awaited<ReturnType<typeof persistedPages>> = [];- const coordinated = afterTransactionQuery(engine, async (_tx, query) => {- if (!query.startsWith('SELECT id, slug, source_id FROM pages') || interleaved) return;- interleaved = true;- await engine.transaction(async tx => {- await tx.lockPageKeys([{ sourceId: 'default', slug }, { sourceId: 'default', slug: renamed }]);- if (change === 'rename') {- await tx.updateSlug(slug, renamed, { sourceId: 'default' });- await installFixtureChunks(tx, renamed, [{ chunk_index: 0, chunk_text: 'cedarmarten selected fixture', chunk_source: 'compiled_truth' }], { sourceId: 'default' });- } else {- await tx.deletePage(slug, { sourceId: 'default' });- }- await sealedPage(tx, slug, 'topazotter replacement fixture', { owner: 'replacement' });- });- writerRows = await persistedPages(engine);- });- const result = await withEnv({ GBRAIN_HOME: home }, () => phaseCGrandfather(coordinated, {- yes: true, dryRun: false, noAutopilotInstall: true,- }));- expect(interleaved).toBe(true);- const replacement = await engine.getPage(slug, { sourceId: 'default' });- expect(replacement?.id).not.toBe(original?.id);- expect(replacement?.frontmatter).toEqual({ owner: 'replacement' });- expect(replacement?.text_projection_revision).toBe(replacement?.knowledge_revision);- expect((await engine.searchKeyword('topazotter', { sourceId: 'default' })).map(hit => hit.slug)).toEqual([slug]);- if (change === 'rename') {- expect(result.result.status).toBe('failed');- expect(result.detail).toMatchObject({ touched: 0, failed: 2 });- expect(result.detail.failures.join(' ')).toContain('Page identity changed');- expect(await persistedPages(engine)).toEqual(writerRows);- expect((await engine.getPage(renamed, { sourceId: 'default' }))?.id).toBe(original?.id);- expect((await engine.searchKeyword('cedarmarten', { sourceId: 'default' })).map(hit => hit.slug)).toEqual([renamed]);- } else {- expect(result.result.status).toBe('complete');- expect(result.detail).toMatchObject({ touched: 1, skipped: 1, failed: 0 });- expect(await engine.executeRaw('SELECT id FROM pages WHERE id = $1', [original!.id])).toEqual([]);- expect((await engine.getPage(sibling, { sourceId: 'default' }))?.frontmatter.validate).toBe(false);- expect(await engine.searchKeyword('cedarmarten', { sourceId: 'default' })).toHaveLength(0);- }- expect((await engine.searchKeyword('violetfox', { sourceId: 'default' })).map(hit => hit.slug)).toEqual([sibling]);- } finally {- await pg.close();- rmSync(home, { recursive: true, force: true });- }- }, 60_000);- }-- test('rolls metadata, knowledge revisions and text projections back when resealing fails', async () => {- const home = mkdtempSync(join(tmpdir(), 'gbrain-grandfather-rollback-pg-'));- const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);- const engine = pg.engine;- try {- const slug = 'concepts/rollback-example';- await sealedPage(engine, slug, 'amberbadger rollback fixture', { keep: 'original' });- await sealedPage(engine, 'concepts/sibling-example', 'violetfox rollback fixture');- const before = await persistedPages(engine);- const jobsBefore = await engine.executeRaw('SELECT * FROM page_projection_jobs ORDER BY source_incarnation, slug');- const chunksBefore = await engine.executeRaw('SELECT * FROM content_chunks ORDER BY id');- let intermediate: Awaited<ReturnType<typeof persistedPages>> = [];- const failing = afterTransactionQuery(engine, async (tx, query) => {- if (!query.startsWith('UPDATE pages SET frontmatter = jsonb_set')) return;- intermediate = await persistedPages(tx);- throw new Error('injected failure before text reseal');- });- const result = await withEnv({ GBRAIN_HOME: home }, () => phaseCGrandfather(failing, {- yes: true, dryRun: false, noAutopilotInstall: true,- }));- expect(result.result.status).toBe('failed');- expect(result.detail).toMatchObject({ touched: 0, failed: 2 });- expect(result.detail.failures.join(' ')).toContain('injected failure before text reseal');- expect(intermediate).toHaveLength(2);- for (let i = 0; i < intermediate.length; i++) {- expect(intermediate[i].frontmatter).toEqual({ ...before[i].frontmatter as object, validate: false });- expect(intermediate[i].knowledge_revision).not.toBe(before[i].knowledge_revision);- expect(intermediate[i].text_projection_revision).not.toBe(intermediate[i].knowledge_revision);- }- expect(await persistedPages(engine)).toEqual(before);- expect(await engine.executeRaw('SELECT * FROM page_projection_jobs ORDER BY source_incarnation, slug')).toEqual(jobsBefore);- expect(await engine.executeRaw('SELECT * FROM content_chunks ORDER BY id')).toEqual(chunksBefore);- expect((await engine.searchKeyword('amberbadger', { sourceId: 'default' })).map(hit => hit.slug)).toEqual([slug]);- expect((await engine.searchKeyword('violetfox', { sourceId: 'default' })).map(hit => hit.slug)).toEqual(['concepts/sibling-example']);- const retry = await withEnv({ GBRAIN_HOME: home }, () => phaseCGrandfather(engine, {- yes: true, dryRun: false, noAutopilotInstall: true,- }));- expect(retry.detail).toMatchObject({ touched: 2, failed: 0 });- const page = await engine.getPage(slug, { sourceId: 'default' });- expect(page?.frontmatter).toEqual({ keep: 'original', validate: false });- expect(page?.knowledge_revision).not.toBe(before[0].knowledge_revision);- expect(page?.text_projection_revision).toBe(page?.knowledge_revision);- expect((await engine.searchKeyword('amberbadger', { sourceId: 'default' })).map(hit => hit.slug)).toEqual([slug]);- } finally {- await pg.close();- rmSync(home, { recursive: true, force: true });- }- }, 60_000);-});diff --git a/test/e2e/upgrade-preserves-pages-7814-postgres.test.ts b/test/e2e/upgrade-preserves-pages-7814-postgres.test.tsnew file mode 100644index 0000000000..899ab7d94e--- /dev/null+++ b/test/e2e/upgrade-preserves-pages-7814-postgres.test.ts@@ -0,0 +1,13 @@+/**+ * #7814 on Postgres; the PGLite run is test/upgrade-preserves-pages-7814.slow.test.ts.+ * See test/helpers/upgrade-preserves-pages-contract.ts for what it protects.+ */+import { expect, test } from 'bun:test';+import { requirePostgresTestDatabase } from '../helpers/test-backends.ts';+import { runUpgradePreservesPages } from '../helpers/upgrade-preserves-pages-contract.ts';++for (const shape of ['repo-less', 'repo'] as const) {+ test(`postgres ${shape}: upgrading a v0.59.2.0-shaped brain changes no page content_hash`, async () => {+ await runUpgradePreservesPages(requirePostgresTestDatabase(), shape, expect);+ }, 900_000);+}diff --git a/test/grandfather-capacity.test.ts b/test/grandfather-capacity.test.tsdeleted file mode 100644index af1b4483f0..0000000000--- a/test/grandfather-capacity.test.ts+++ /dev/null@@ -1,41 +0,0 @@-/**- * Capacity split (#5728): the managed v0.13.1 grandfather refuses up front- * only when the pages it can admit exceed the permanent request-ID caps.- * Pages it skips (code and image pages, non-Markdown files) need no ID, so a- * pass over only those completes even with exhausted IDs, while one- * admissible page refuses before any page changes, naming the command.- */-import { expect, test } from 'bun:test';-import { phaseCGrandfather } from '../src/commands/migrations/v0_13_1.ts';-import type { BrainEngine } from '../src/core/engine.ts';-import { managedBrain } from './helpers/managed-brain.ts';--const OPTS = { yes: true, dryRun: false, noAutopilotInstall: true };-const EXHAUSTED = 'persistence.limits.principal_lifetime_ids';--async function seedSkipped({ engine }: { engine: BrainEngine }) {- await engine.putPage('code/example-script', { type: 'code', title: 'Example script', compiled_truth: 'print("hello")' });- await engine.setConfig(EXHAUSTED, '0');-}--test('exhausted IDs do not block a pass that admits nothing', async () => {- await managedBrain(async ({ engine }) => {- const { result } = await phaseCGrandfather(engine, OPTS);- expect(result.status).toBe('complete');- expect(await engine.executeRaw('SELECT id FROM persistence_requests')).toEqual([]);- }, { setup: seedSkipped });-}, 120_000);--test('one admissible page refuses up front with the capacity command', async () => {- await managedBrain(async ({ engine }) => {- const before = await engine.executeRaw('SELECT slug, frontmatter, knowledge_revision FROM pages ORDER BY slug');- const { result } = await phaseCGrandfather(engine, OPTS);- expect(result.status).toBe('failed');- expect(result.detail).toStartWith('write_quota_exhausted: Write capacity exhausted: principal permanent request IDs (0 used of 0).');- expect(result.detail).toContain(`gbrain config set ${EXHAUSTED} `);- expect(await engine.executeRaw('SELECT slug, frontmatter, knowledge_revision FROM pages ORDER BY slug')).toEqual(before);- }, { setup: async brain => {- await brain.engine.putPage('notes/example', { type: 'note', title: 'Example', compiled_truth: 'An ordinary note.' });- await seedSkipped(brain);- } });-}, 120_000);diff --git a/test/grandfather-managed.test.ts b/test/grandfather-managed.test.tsindex a77012829b..4a59123fb2 100644--- a/test/grandfather-managed.test.ts+++ b/test/grandfather-managed.test.ts@@ -1,3 +1,4 @@+// #7814: a managed grandfather write an older binary queued ends without writing.import { expect, test } from 'bun:test';import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs';import { tmpdir } from 'node:os';@@ -5,12 +6,12 @@ import { dirname, join } from 'node:path';import { randomUUID } from 'node:crypto';import type { BrainEngine } from '../src/core/engine.ts';import type { OperationContext } from '../src/core/ops/contract.ts';-import { phaseCGrandfather, phaseDVerify } from '../src/commands/migrations/v0_13_1.ts';-import { claimWorktree } from '../src/core/persistence/ownership.ts';+import { claimWorktree, getWorktreeBinding } from '../src/core/persistence/ownership.ts';import { activateSharedSkillPersistence } from '../src/core/persistence/skill-activation.ts';-import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';-import { grandfatherCanonicalPage } from '../src/core/persistence/grandfather.ts';-import { disposePersistenceConsumer } from '../src/core/persistence/service.ts';+import { requestPrincipalForContext, submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { submissionAuthority } from '../src/core/persistence/authority.ts';+import { admitWrite } from '../src/core/persistence/journal.ts';+import { disposePersistenceConsumer, waitForWrite } from '../src/core/persistence/service.ts';import { withCoordinatedWrite } from '../src/core/persistence/context.ts';import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';import { runPersistenceEffects } from '../src/core/persistence/effects.ts';@@ -48,106 +49,34 @@ async function fixture(run: (f: { engine: BrainEngine; ctx: OperationContext; ho} finally { rmSync(home, { recursive: true, force: true }); }}-async function preservesManagedPage(databaseUrl?: string) {- await fixture(async ({ engine, home, root, slug }) => {+async function queuedGrandfatherEndsWithoutWriting(databaseUrl?: string) {+ await fixture(async ({ engine, ctx, root, slug }) => {+ const file = join(root, slug + '.md');const before = (await engine.readPageSnapshot(slug, { sourceId: 'default' }))!;- const [column] = await engine.executeRaw<{ width: number }>("SELECT atttypmod::int AS width FROM pg_attribute WHERE attrelid='content_chunks'::regclass AND attname='embedding'");- await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], async () => {- await tx.executeRaw("UPDATE content_chunks SET embedding=('['||array_to_string(array_fill(0.1::real,ARRAY[$1::int]),',')||']')::vector WHERE page_id=$2", [column.width, before.page.id]);- await tx.executeRaw('INSERT INTO extract_atoms_page_state(source_incarnation,page_id,content_hash,fail_count,tombstoned) VALUES($1::uuid,$2,$3,2,true)',- [before.sourceIncarnation, before.page.id, before.page.content_hash]);- }, TEST_WRITE_ATTRIBUTION));- const chunks = await engine.executeRaw('SELECT id,chunk_text,embedding::text FROM content_chunks WHERE page_id=$1 ORDER BY id', [before.page.id]);- expect(chunks.length).toBeGreaterThan(0);- const result = await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true });- expect(result).toMatchObject({ result: { status: 'complete' }, detail: { touched: 1, failed: 0 } });+ durableGitRepo(root, [slug + '.md']);+ const bytes = readFileSync(file, 'utf8'), head = git(root, 'rev-parse', 'HEAD').trim();+ const principal = await requestPrincipalForContext(ctx);+ const authority = await submissionAuthority(ctx, 'put_page', 'default', before.sourceIncarnation, slug);+ const binding = await getWorktreeBinding(engine, 'default');+ const request = await admitWrite(engine, { principal, operation: 'put_page', sourceId: 'default', sourceIncarnation: before.sourceIncarnation,+ pageId: before.page.id, slug, requestId: randomUUID(), authority, callerIntent: { migration: '0.13.1', expected_revision: before.revision },+ intent: { kind: 'managed_grandfather', expected_revision: before.revision }, worktreeId: binding?.worktree_id, topologyGeneration: binding?.topology_generation });+ await waitForWrite(engine, request, ctx.config, 30_000).catch(() => undefined);+ expect(await engine.executeRaw('SELECT state,error_code FROM persistence_requests WHERE request_id=$1::uuid', [request.request_id]))+ .toEqual([{ state: 'conflict', error_code: 'revision_conflict' }]);const after = (await engine.readPageSnapshot(slug, { sourceId: 'default' }))!;- expect(after.page.frontmatter.validate).toBe(false);- expect(after.page.compiled_truth).toBe(before.page.compiled_truth);- expect(after.revision).not.toBe(before.revision);- expect(after.page.text_projection_revision).toBe(after.revision);- expect(readFileSync(join(root, `${slug}.md`), 'utf8')).toContain('validate: false');- expect(await engine.executeRaw('SELECT id,chunk_text,embedding::text FROM content_chunks WHERE page_id=$1 ORDER BY id', [before.page.id])).toEqual(chunks);- expect(await engine.searchKeyword('Amberbadger', { sourceId: 'default' })).toHaveLength(1);- expect(await engine.executeRaw('SELECT fail_count,tombstoned FROM extract_atoms_page_state WHERE source_incarnation=$1::uuid AND page_id=$2 AND content_hash=$3',- [after.sourceIncarnation, after.page.id, after.page.content_hash])).toEqual([{ fail_count: 2, tombstoned: true }]);- expect(await engine.executeRaw("SELECT e.kind FROM persistence_effects e JOIN persistence_requests r ON r.id=e.request_id WHERE r.intent->>'kind'='managed_grandfather' AND e.kind IN ('embedding','facts-backstop')")).toEqual([]);- const rollback = join(home, '.gbrain/migrations/v0_13_1-rollback.jsonl');- const snapshot = JSON.parse(readFileSync(rollback, 'utf8').trim());- expect(snapshot).toMatchObject({ id: before.page.id, source_id: 'default', source_incarnation: before.sourceIncarnation, knowledge_revision: before.revision });- expect(snapshot.pre_frontmatter).not.toHaveProperty('validate');- if (process.platform !== 'win32') expect(statSync(rollback).mode & 0o777).toBe(0o600);- expect((await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true })).detail.touched).toBe(0);+ expect(after.page.frontmatter).toEqual(before.page.frontmatter);+ expect(after.page.frontmatter).not.toHaveProperty('validate');+ expect(after.page.content_hash).toBe(before.page.content_hash);+ expect(after.revision).toBe(before.revision);+ expect(readFileSync(file, 'utf8')).toBe(bytes);+ expect(git(root, 'rev-parse', 'HEAD').trim()).toBe(head);}, { databaseUrl });}-test('managed grandfathering publishes metadata without changing vectors or spending on derived effects', () => preservesManagedPage(), 120_000);-test.skipIf(!process.env.DATABASE_URL)('Postgres managed grandfathering preserves canonical bytes, projection and derived state',- () => preservesManagedPage(process.env.DATABASE_URL), 120_000);--test('a concurrent explicit validation choice is not overwritten by grandfathering', () => fixture(async ({ engine, ctx, slug, root }) => {- const before = (await engine.readPageSnapshot(slug, { sourceId: 'default' }))!;- await expect(grandfatherCanonicalPage(engine, { id: before.page.id, source_id: 'default', slug, source_incarnation: before.sourceIncarnation }, async () => {- await submitPageMutation(ctx, { operation: 'put_page', params: { slug, request_id: randomUUID(), expected_revision: before.revision,- content: serializePageToMarkdown({ ...before.page, frontmatter: { ...before.page.frontmatter, validate: true } }, before.tags) } });- })).rejects.toMatchObject({ code: 'revision_conflict' });- expect((await engine.getPage(slug, { sourceId: 'default' }))?.frontmatter.validate).toBe(true);- expect(readFileSync(join(root, `${slug}.md`), 'utf8')).toContain('validate: true');-}), 120_000);--test('human file edits abort managed grandfathering without replacing either version', () => fixture(async ({ engine, slug, root }) => {- const before = (await engine.readPageSnapshot(slug, { sourceId: 'default' }))!;- const path = join(root, `${slug}.md`); const changed = readFileSync(path, 'utf8') + '\nUnpublished human edit.\n';- await expect(grandfatherCanonicalPage(engine, { id: before.page.id, source_id: 'default', slug, source_incarnation: before.sourceIncarnation }, () => {- writeFileSync(path, changed);- })).rejects.toMatchObject({ code: 'source_changed' });- expect(readFileSync(path, 'utf8')).toBe(changed);- expect((await engine.readPageSnapshot(slug, { sourceId: 'default' }))?.revision).toBe(before.revision);-}), 120_000);--test('pending grandfathering reuses its admitted request and its original rollback snapshot', () => fixture(async ({ engine, slug }) => {- const snapshot = (await engine.readPageSnapshot(slug, { sourceId: 'default' }))!;- const selected = { id: snapshot.page.id, source_id: 'default', slug, source_incarnation: snapshot.sourceIncarnation };- const host = localHostId(); let backups = 0;- await engine.transaction(async tx => {- await tx.executeRaw("SELECT set_config('gbrain.topology_change','on',true)");- await tx.executeRaw('UPDATE persistence_worktrees SET owner_host_id=$1::uuid', [randomUUID()]);- });- let requestId: string | undefined;- try { await grandfatherCanonicalPage(engine, selected, () => { backups++; }); }- catch (error: any) { expect(error.code).toBe('write_pending'); requestId = error.writeRequest.request_id; }- expect(requestId).toBeDefined(); expect(backups).toBe(1);- await engine.transaction(async tx => {- await tx.executeRaw("SELECT set_config('gbrain.topology_change','on',true)");- await tx.executeRaw('UPDATE persistence_worktrees SET owner_host_id=$1::uuid', [host]);- });- expect(await grandfatherCanonicalPage(engine, selected, () => { backups++; }))- .toEqual({ status: 'touched', revision: (await engine.readPageSnapshot(slug, { sourceId: 'default' }))!.revision });- expect(backups).toBe(1);- expect(await engine.executeRaw("SELECT request_id,state FROM persistence_requests WHERE intent->>'kind'='managed_grandfather'"))- .toEqual([{ request_id: requestId, state: 'committed' }]);-}), 120_000);--test('archived sources and non-Markdown artifacts are not rewritten by managed grandfathering', () => fixture(async ({ engine, slug, root }) => {- const path = join(root, `${slug}.md`), bytes = readFileSync(path, 'utf8');- await engine.transaction(async tx => {- await tx.executeRaw("SELECT set_config('gbrain.topology_change','on',true)");- await tx.executeRaw("UPDATE sources SET archived=true WHERE id='default'");- });- expect((await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true })).detail).toMatchObject({ touched: 0, skipped: 1, failed: 0 });- expect(readFileSync(path, 'utf8')).toBe(bytes);- mkdirSync(join(root, 'source'));- writeFileSync(join(root, 'source/example.ts'), 'export const fixture = 1;\n');- await engine.transaction(async tx => {- await tx.executeRaw("SELECT set_config('gbrain.topology_change','on',true)");- await tx.executeRaw("UPDATE sources SET archived=false WHERE id='default'");- await withCoordinatedWrite(tx, ['default'], () => tx.executeRaw("UPDATE pages SET source_path='source/example.ts' WHERE slug=$1", [slug]), TEST_WRITE_ATTRIBUTION);- });- expect((await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true })).detail).toMatchObject({ touched: 0, skipped: 1, failed: 0 });- expect(readFileSync(path, 'utf8')).toBe(bytes);- expect(readFileSync(join(root, 'source/example.ts'), 'utf8')).toBe('export const fixture = 1;\n');- expect((await engine.getPage(slug, { sourceId: 'default' }))?.frontmatter).not.toHaveProperty('validate');-}), 120_000);+test('a queued managed_grandfather request ends in conflict without writing', () => queuedGrandfatherEndsWithoutWriting(), 120_000);+test.skipIf(!process.env.DATABASE_URL)('Postgres: a queued managed_grandfather request ends in conflict without writing',+ () => queuedGrandfatherEndsWithoutWriting(process.env.DATABASE_URL), 120_000);const DB_ONLY_SLUG = 'conversations/sessions/probe-session';const DB_ONLY_CACHE = `${DB_ONLY_SLUG}.md`;@@ -185,11 +114,6 @@ async function settledGitEffects(engine: BrainEngine, ctx: OperationContext) {for (const declared of ['conversations/', 'Conversations/']) {test(`managed writes publish a page under declared db_only ${declared} without a cache file database-only`, () => fixture(async ({ engine, ctx, root }) => {expect((await engine.readPageSnapshot(DB_ONLY_SLUG, { sourceId: 'default' }))?.page.source_path).toBeNull();- expect(await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true }))- .toMatchObject({ result: { status: 'complete' }, detail: { touched: 2, failed: 0 } });- expect((await engine.getPage(DB_ONLY_SLUG, { sourceId: 'default' }))?.frontmatter.validate).toBe(false);- expect(await engine.executeRaw("SELECT slug,state FROM persistence_requests WHERE intent->>'kind'='managed_grandfather' ORDER BY slug"))- .toEqual([{ slug: DB_ONLY_SLUG, state: 'committed' }, { slug: 'notes/example', state: 'committed' }]);expect(await replacePage(ctx, DB_ONLY_SLUG, 'Revised transcript body.')).toMatchObject({ write_through: { written: false, skipped: 'db_only' } });expect((await engine.getPage(DB_ONLY_SLUG, { sourceId: 'default' }))?.compiled_truth).toContain('Revised transcript body.');expect(existsSync(join(root, 'conversations'))).toBe(false);@@ -298,11 +222,3 @@ for (const cache of [false, true]) {.toEqual([{ state: 'committed', error_code: null }]);}, { setup: dbOnlySource(declaring('conversations/'), { cache }) }), 120_000);}--test('verify reports a managed page rewritten after grandfathering without failing the phase', () => fixture(async ({ engine, slug }) => {- const { detail } = await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true });- expect(await phaseDVerify(engine, detail.grandfathered)).toMatchObject({ status: 'complete', detail: 'verified=1 rewritten_concurrently=0' });- await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () =>- tx.executeRaw("UPDATE pages SET frontmatter=frontmatter-'validate' WHERE source_id='default' AND slug=$1", [slug]), TEST_WRITE_ATTRIBUTION));- expect(await phaseDVerify(engine, detail.grandfathered)).toMatchObject({ status: 'complete', detail: 'verified=0 rewritten_concurrently=1' });-}), 120_000);diff --git a/test/helpers/managed-migration-chain-contract.ts b/test/helpers/managed-migration-chain-contract.tsindex 3310841e31..cda70eb595 100644--- a/test/helpers/managed-migration-chain-contract.ts+++ b/test/helpers/managed-migration-chain-contract.ts@@ -27,8 +27,9 @@ async function brainState(engine: BrainEngine) {/*** CEO-E1 / Eng harness: the real runner walks every registered orchestrator- * migration on a managed brain whose legacy content targets v0.13.1- * (grandfather), v0.28.0 (takes backfill) and v0.32.2 (fact adoption).+ * migration on a managed brain whose legacy content targets v0.28.0 (takes+ * backfill) and v0.32.2 (fact adoption); v0.13.1 must leave every page's+ * frontmatter alone (#7814).*/export async function runManagedMigrationChain(databaseUrl: string | undefined, expect: typeof bunExpect): Promise<void> {const fixture = await managedChainFixture(databaseUrl);@@ -49,7 +50,7 @@ export async function runManagedMigrationChain(databaseUrl: string | undefined,}expect(JSON.stringify(ledger)).not.toContain('writer_coordinator_required');- expect(phaseDetail(ledger, '0.13.1', 'grandfather')).toBe('touched=2 skipped=0 failed=0');+ expect(phaseDetail(ledger, '0.13.1', 'grandfather')).toStartWith('retired (#7814)');expect(phaseDetail(ledger, '0.28.0', 'backfill')).toBe('extract-takes scanned 2 pages; 1 had fenced takes; upserted 2 rows');expect(phaseDetail(ledger, '0.32.2', 'fence_facts')).toContain('scanned=3 fenced=3 pages=2');expect(phaseDetail(ledger, '0.32.2', 'verify')).toBe('pages_checked=2');@@ -58,6 +59,7 @@ export async function runManagedMigrationChain(databaseUrl: string | undefined,let settled: Awaited<ReturnType<typeof brainState>>;try {settled = await brainState(engine);+ expect(await engine.executeRaw("SELECT slug FROM pages WHERE frontmatter ? 'validate'")).toEqual([]);expect(settled.takes.map(t => [t.slug, t.row_num, t.active])).toEqual([[LEGACY_FILE_SLUG, 1, true], [LEGACY_FILE_SLUG, 2, true]]);const byId = new Map(settled.facts.map(f => [Number(f.id), f]));const { seed } = fixture;@@ -106,8 +108,8 @@ export async function runExhaustedCapacityChain(databaseUrl: string | undefined,const refused = await fixture.applyMigrations();expect(refused.exitCode, transcript(refused)).toBe(1);const ledger = fixture.ledger();- expect(ledger.filter(e => e.status !== 'complete').map(e => e.version)).toEqual(['0.13.1']);- const detail = phaseDetail(ledger, '0.13.1', 'grandfather');+ expect(ledger.filter(e => e.status !== 'complete').map(e => e.version)).toEqual(['0.28.0']);+ const detail = phaseDetail(ledger, '0.28.0', 'backfill');expect(detail).toStartWith('write_quota_exhausted: Write capacity exhausted: principal permanent request IDs (0 used of 0).');expect(detail).toContain(`gbrain config set ${key} `);diff --git a/test/helpers/upgrade-preserves-pages-contract.ts b/test/helpers/upgrade-preserves-pages-contract.tsnew file mode 100644index 0000000000..81165969f8--- /dev/null+++ b/test/helpers/upgrade-preserves-pages-contract.ts@@ -0,0 +1,150 @@+import postgres from '#postgres';+import { chmodSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { dirname, join } from 'node:path';+import { randomUUID } from 'node:crypto';+import { fileURLToPath } from 'node:url';+import type { expect as bunExpect } from 'bun:test';+import { migrations } from '../../src/commands/migrations/index.ts';+import type { BrainEngine } from '../../src/core/engine.ts';+import type { PageInput } from '../../src/core/types.ts';+import { serializePageToMarkdown } from '../../src/core/markdown.ts';+import { PGLiteEngine } from '../../src/core/pglite-engine.ts';+import { PostgresEngine } from '../../src/core/postgres-engine.ts';+import { claimWorktree } from '../../src/core/persistence/ownership.ts';+import { activateSharedSkillPersistence } from '../../src/core/persistence/skill-activation.ts';+import { disposePersistenceConsumer } from '../../src/core/persistence/service.ts';+import type { CompletedMigrationEntry } from '../../src/core/preferences.ts';+import { assertSafeE2eDatabaseUrl } from './db-guard.ts';+import { durableGitRepo, git } from './git-publication.ts';+import { withEnv } from './with-env.ts';++const REPO = fileURLToPath(new URL('../..', import.meta.url)).replace(/\/$/, '');+const PROVIDER_KEYS = ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY', 'VOYAGE_API_KEY', 'GOOGLE_API_KEY', 'GEMINI_API_KEY',+ 'OPENROUTER_API_KEY', 'DEEPSEEK_API_KEY', 'GROQ_API_KEY', 'MISTRAL_API_KEY', 'TOGETHER_API_KEY', 'ZEROENTROPY_API_KEY'];++export type BrainShape = 'repo-less' | 'repo';++const PAGES: Array<{ slug: string } & PageInput> = [+ { slug: 'concepts/upgrade-alpha', type: 'concept', title: 'Upgrade alpha', frontmatter: { source_kind: 'note', ingested_via: 'put' },+ compiled_truth: '# Upgrade alpha\n\nThe alpha page links to [[concepts/upgrade-beta]].' },+ { slug: 'concepts/upgrade-beta', type: 'concept', title: 'Upgrade beta', frontmatter: { tags_hint: 'tidal' },+ compiled_truth: '# Upgrade beta\n\nThe beta page is about lighthouse maintenance schedules.' },+ { slug: 'people/alice-example', type: 'person', title: 'Alice Example', frontmatter: { company: 'companies/acme-example' },+ compiled_truth: '# Alice Example\n\nAlice Example works at acme-example on storage engines.' },+];++async function pageState(engine: BrainEngine) {+ return engine.executeRaw<Record<string, unknown>>(+ 'SELECT source_id, slug, content_hash, frontmatter::text AS frontmatter, compiled_truth FROM pages ORDER BY source_id, slug');+}++/**+ * #7814: a brain created by v0.59.2.0 (before `gbrain init` stamped+ * fresh-install migrations) carries pages written by that release and an+ * empty migration ledger, so every orchestrator lists as pending. Upgrading+ * it runs the real `gbrain apply-migrations` runner over the whole registry;+ * the run must complete and change no page: same content_hash, frontmatter+ * and body for every page and, on a repo brain, the same files and Git HEAD.+ *+ * The brain is built on the current schema (the schema migrations are the+ * upgrade matrix's job, scripts/upgrade-matrix); what makes it v0.59.2.0-shaped+ * is the ledger with no fresh-install stamps. `repo` is a Git checkout+ * claimed as the default source's canonical worktree with managed+ * persistence activated, as `gbrain init --git` leaves it.+ */+export async function runUpgradePreservesPages(databaseUrl: string | undefined, shape: BrainShape, expect: typeof bunExpect): Promise<void> {+ const home = realpathSync.native(mkdtempSync(join(tmpdir(), 'gbrain-upgrade-7814-')));+ const root = join(home, 'content');+ const bin = join(home, 'bin');+ mkdirSync(join(home, '.gbrain'), { recursive: true });+ mkdirSync(root);+ mkdirSync(bin);+ writeFileSync(join(bin, 'gbrain'), `#!/bin/sh\nexec bun run ${REPO}/src/cli.ts "$@"\n`);+ chmodSync(join(bin, 'gbrain'), 0o755);++ let admin: ReturnType<typeof postgres> | null = null;+ let database: string | null = null;+ let config: Record<string, unknown>;+ if (databaseUrl) {+ assertSafeE2eDatabaseUrl(databaseUrl);+ database = `gbrain_test_upgrade_${randomUUID().replace(/-/g, '')}`;+ admin = postgres(databaseUrl, { max: 1, prepare: false });+ await admin.unsafe(`CREATE DATABASE ${database}`);+ const url = new URL(databaseUrl); url.pathname = `/${database}`;+ config = { engine: 'postgres', database_url: url.toString() };+ } else {+ config = { engine: 'pglite', database_path: join(home, '.gbrain', 'brain.pglite') };+ }+ writeFileSync(join(home, '.gbrain', 'config.json'), JSON.stringify({ ...config, embedding_disabled: true }) + '\n', { mode: 0o600 });+ const open = async (): Promise<BrainEngine> => {+ const engine = config.engine === 'postgres' ? new PostgresEngine() : new PGLiteEngine();+ await engine.connect(config as never);+ return engine;+ };+ const files = PAGES.map(page => `${page.slug}.md`);+ const repoState = () => shape === 'repo'+ ? { head: git(root, 'rev-parse', 'HEAD').trim(), status: git(root, 'status', '--porcelain', '--', ...files).trim(),+ files: files.map(file => readFileSync(join(root, file), 'utf8')) }+ : null;++ try {+ let before!: Awaited<ReturnType<typeof pageState>>;+ await withEnv({ GBRAIN_HOME: home, HOME: home, DATABASE_URL: undefined, GBRAIN_DATABASE_URL: undefined }, async () => {+ const engine = await open();+ try {+ await engine.initSchema();+ for (const page of PAGES) {+ const { slug, ...fields } = page;+ const written = await engine.putPage(slug, { ...fields, timeline: '' });+ if (shape !== 'repo') continue;+ await engine.executeRaw('UPDATE pages SET source_path=$1 WHERE id=$2', [`${slug}.md`, written.id]);+ const snapshot = (await engine.readPageSnapshot(slug, { sourceId: 'default' }))!;+ mkdirSync(dirname(join(root, `${slug}.md`)), { recursive: true });+ writeFileSync(join(root, `${slug}.md`), serializePageToMarkdown(snapshot.page, snapshot.tags));+ }+ if (shape === 'repo') {+ await engine.executeRaw("UPDATE sources SET local_path=$1 WHERE id='default'", [root]);+ durableGitRepo(root, files);+ await claimWorktree(engine, 'default', root);+ await activateSharedSkillPersistence(engine, { confirmQuiesced: true });+ }+ before = await pageState(engine);+ } finally {+ await disposePersistenceConsumer(engine);+ await engine.disconnect();+ }+ });+ expect(before.length).toBe(PAGES.length);+ const repoBefore = repoState();++ const childEnv: Record<string, string> = {};+ for (const [key, value] of Object.entries(process.env)) {+ if (value === undefined || PROVIDER_KEYS.includes(key) || key === 'DATABASE_URL' || key === 'GBRAIN_DATABASE_URL') continue;+ childEnv[key] = value;+ }+ Object.assign(childEnv, { HOME: home, GBRAIN_HOME: home, PATH: `${bin}:${process.env.PATH ?? ''}`, GBRAIN_NO_AUTOPILOT_INSTALL: '1' });+ const proc = Bun.spawn(['bun', 'run', `${REPO}/src/cli.ts`, 'apply-migrations', '--yes', '--non-interactive', '--no-autopilot-install'],+ { cwd: home, env: childEnv, stdout: 'pipe', stderr: 'pipe' });+ const [stdout, stderr, exitCode] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text(), proc.exited]);+ const transcript = `exit=${exitCode}\n${stdout}\n${stderr}`;+ expect(exitCode, transcript).toBe(0);++ const ledger = readFileSync(join(home, '.gbrain', 'migrations', 'completed.jsonl'), 'utf8')+ .split('\n').filter(Boolean).map(line => JSON.parse(line) as CompletedMigrationEntry);+ for (const migration of migrations) {+ expect(ledger.filter(e => e.version === migration.version).at(-1)?.status, `v${migration.version}\n${transcript}`).toBe('complete');+ }++ const engine = await open();+ try { expect(await pageState(engine)).toEqual(before); }+ finally { await engine.disconnect(); }+ expect(repoState()).toEqual(repoBefore);+ } finally {+ if (admin && database) {+ await admin.unsafe(`DROP DATABASE IF EXISTS ${database} WITH (FORCE)`);+ await admin.end();+ }+ rmSync(home, { recursive: true, force: true });+ }+}diff --git a/test/migrations-v0_13_1-grandfather.test.ts b/test/migrations-v0_13_1-grandfather.test.tsdeleted file mode 100644index 94e0d215a8..0000000000--- a/test/migrations-v0_13_1-grandfather.test.ts+++ /dev/null@@ -1,203 +0,0 @@-// v0.41.37.0 #1581 — chunked, source-safe, soft-delete-filtered grandfather pass.-//-// The pre-#1581 per-page getPage+putPage loop hung CPU-bound for 70+ min on an-// 82K-page PGLite brain. The rewrite is a chunked bulk SQL pass keyed on-// pages.id (NOT slug — slug isn't globally unique), filtering deleted_at IS NULL.-// These tests drive phaseCGrandfather directly against a real PGLite engine.-import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';-import { tmpdir } from 'os';-import { mkdtempSync, existsSync, readFileSync, rmSync } from 'fs';-import { join } from 'path';-import { PGLiteEngine } from '../src/core/pglite-engine.ts';-import { resetPgliteState } from './helpers/reset-pglite.ts';-import { withEnv } from './helpers/with-env.ts';-import { phaseCGrandfather, phaseDVerify } from '../src/commands/migrations/v0_13_1.ts';-import { installFixtureChunks } from './helpers/page-projection.ts';-import type { OrchestratorOpts } from '../src/commands/migrations/types.ts';--let engine: PGLiteEngine;--beforeAll(async () => {- engine = new PGLiteEngine();- await engine.connect({});- await engine.initSchema();-});--afterAll(async () => {- await engine.disconnect();-});--beforeEach(async () => {- await resetPgliteState(engine);-});--const ensureSource = (id: string) =>- engine.executeRaw('INSERT INTO sources (id, name) VALUES ($1, $1) ON CONFLICT DO NOTHING', [id]);--const seed = (slug: string, frontmatter: Record<string, unknown>, sourceId = 'default') =>- engine.putPage(slug, {- type: 'concept', title: slug, compiled_truth: 'body', timeline: '', frontmatter,- }, { sourceId });--const fmOf = async (slug: string, sourceId = 'default') => {- const p = await engine.getPage(slug, { sourceId });- return p?.frontmatter ?? null;-};--// Run the phase with an isolated GBRAIN_HOME so the rollback log lands in a-// tempdir (withEnv keeps R1 test-isolation compliance — no raw process.env writes).-async function gf(home: string, opts: Partial<OrchestratorOpts> = {}) {- const full: OrchestratorOpts = { yes: true, dryRun: false, noAutopilotInstall: true, ...opts };- return withEnv({ GBRAIN_HOME: home }, () => phaseCGrandfather(engine, full));-}--describe('#1581 phaseCGrandfather (chunked, source-safe)', () => {- test('metadata-only migration preserves an already searchable projection', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-search-'));- try {- await engine.putPage('concepts/search-example', {- type: 'concept', title: 'Search Example', compiled_truth: 'amberbadger migration fixture', timeline: '', frontmatter: {},- }, { sourceId: 'default' });- await installFixtureChunks(engine, 'concepts/search-example', [{ chunk_index: 0, chunk_text: 'amberbadger migration fixture', chunk_source: 'compiled_truth' }]);- const before = await engine.getPage('concepts/search-example', { sourceId: 'default' });- expect((await engine.searchKeyword('amberbadger', { sourceId: 'default' })).length).toBe(1);- expect((await gf(home)).result.status).toBe('complete');- const after = await engine.getPage('concepts/search-example', { sourceId: 'default' });- expect(after?.knowledge_revision).not.toBe(before?.knowledge_revision);- expect(after?.text_projection_revision).toBe(after?.knowledge_revision);- expect((await engine.searchKeyword('amberbadger', { sourceId: 'default' })).length).toBe(1);- } finally {- rmSync(home, { recursive: true, force: true });- }- });-- test('does not certify an unsealed projection as searchable', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-unsealed-'));- try {- await seed('concepts/unsealed-example', {});- expect((await gf(home)).result.status).toBe('complete');- const after = await engine.getPage('concepts/unsealed-example', { sourceId: 'default' });- expect(after?.text_projection_revision).toBeNull();- expect((await engine.executeRaw('SELECT 1 FROM page_projection_jobs WHERE slug=$1', ['concepts/unsealed-example'])).length).toBe(1);- } finally {- rmSync(home, { recursive: true, force: true });- }- });-- test('absent validate key → validate:false', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- await seed('a', { foo: 1 });- const { result } = await gf(home);- expect(result.status).toBe('complete');- expect((await fmOf('a'))?.validate).toBe(false);- });-- test('explicit validate (true/false/null) is NOT modified', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- await seed('t', { validate: true });- await seed('f', { validate: false });- await seed('n', { validate: null });- await gf(home);- expect((await fmOf('t'))?.validate).toBe(true);- expect((await fmOf('f'))?.validate).toBe(false);- // null value still counts as "key present" → left untouched.- expect((await fmOf('n'))?.validate ?? 'WAS_NULL').toBe('WAS_NULL');- });-- test('soft-deleted pages are NOT grandfathered', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- await seed('live', { foo: 1 });- await seed('dead', { foo: 1 });- await engine.softDeletePage('dead');- await gf(home);-- expect((await fmOf('live'))?.validate).toBe(false);- const rows = await engine.executeRaw<{ fm: Record<string, unknown> }>(- "SELECT frontmatter AS fm FROM pages WHERE slug = 'dead'",- );- expect(Object.prototype.hasOwnProperty.call(rows[0]?.fm ?? {}, 'validate')).toBe(false);- });-- test('multi-source duplicate slugs: both grandfathered, no cross-contamination', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- await ensureSource('src2');- await seed('people/dup', { src: 'a' }, 'default');- await seed('people/dup', { src: 'b' }, 'src2');- await gf(home);-- expect((await fmOf('people/dup', 'default'))?.validate).toBe(false);- expect((await fmOf('people/dup', 'src2'))?.validate).toBe(false);- expect((await fmOf('people/dup', 'default'))?.src).toBe('a');- expect((await fmOf('people/dup', 'src2'))?.src).toBe('b');- });-- test('rollback log carries source identity', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- await ensureSource('src2');- await seed('people/dup', { src: 'a' }, 'default');- await seed('people/dup', { src: 'b' }, 'src2');- await gf(home);-- // configDir() appends '.gbrain' to GBRAIN_HOME.- const logPath = join(home, '.gbrain', 'migrations', 'v0_13_1-rollback.jsonl');- expect(existsSync(logPath)).toBe(true);- const lines = readFileSync(logPath, 'utf-8').trim().split('\n').map(l => JSON.parse(l));- expect(lines.map(l => l.source_id).sort()).toEqual(['default', 'src2']);- expect(lines.every(l => typeof l.id === 'number' && l.slug === 'people/dup')).toBe(true);- });-- test('re-run is a no-op (idempotent)', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- await seed('a', { foo: 1 });- const first = await gf(home);- expect(first.detail.touched).toBe(1);- const second = await gf(home);- expect(second.detail.touched).toBe(0);- });-- test('dry-run counts without mutating', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- await seed('a', { foo: 1 });- const { detail } = await gf(home, { dryRun: true });- expect(detail.touched).toBe(1);- expect((await fmOf('a'))?.validate ?? 'UNSET').toBe('UNSET');- });-- test('large-N (1200 pages) completes via chunked pass (hang regression)', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-'));- for (let i = 0; i < 1200; i++) await seed(`bulk/${i}`, { i });- const t0 = Date.now();- const { result, detail } = await gf(home);- expect(result.status).toBe('complete');- expect(detail.touched).toBe(1200);- expect((await fmOf('bulk/0'))?.validate).toBe(false);- expect((await fmOf('bulk/1199'))?.validate).toBe(false);- expect(Date.now() - t0).toBeLessThan(30_000);- });-- test('verify fails only for grandfathered pages still missing validate at the revision this run wrote', async () => {- const home = mkdtempSync(join(tmpdir(), 'gf-verify-'));- try {- await seed('kept', {});- await seed('reimported', {});- const { detail } = await gf(home);- expect(detail.grandfathered.map(page => page.id).sort()).toEqual(- [(await engine.getPage('kept'))!.id, (await engine.getPage('reimported'))!.id].sort());- // A connector re-import during the run replaces the page and drops the flag.- await seed('reimported', { connector: 'sync' });- expect(await phaseDVerify(engine, detail.grandfathered)).toMatchObject({ status: 'complete', detail: 'verified=1 rewritten_concurrently=1' });- // A claimed write that never landed: the recorded revision is current and the flag is absent.- await seed('unwritten', {});- const unwritten = (await engine.getPage('unwritten'))!;- const failed = await phaseDVerify(engine, [...detail.grandfathered, { id: unwritten.id, revision: unwritten.knowledge_revision! }]);- expect(failed.status).toBe('failed');- expect(failed.detail).toBe(`verified=1 rewritten_concurrently=1 missing_validate=1 (page#${unwritten.id})`);- // A receipt without a revision cannot prove a rewrite, so the flag must be present.- const kept = (await engine.getPage('kept'))!;- expect(await phaseDVerify(engine, [{ id: kept.id, revision: null }, { id: unwritten.id, revision: null }])).toMatchObject({- status: 'failed', detail: `verified=1 rewritten_concurrently=0 missing_validate=1 (page#${unwritten.id})` });- } finally {- rmSync(home, { recursive: true, force: true });- }- });-});diff --git a/test/migrations-v0_13_1.test.ts b/test/migrations-v0_13_1.test.tsindex 939b85c716..90dfa6cff2 100644--- a/test/migrations-v0_13_1.test.ts+++ b/test/migrations-v0_13_1.test.ts@@ -1,38 +1,26 @@/**- * v0.13.1 migration tests — grandfather validate:false onto existing pages.+ * v0.13.1 migration (#7814): the retired `validate: false` grandfather.*- * Verifies:- * - Registry contains v0_13_1 in semver order- * - Orchestrator is idempotent (running twice is a no-op on the 2nd pass)- * - Pages with existing `validate` key are NOT modified- * - Rollback log lines are written pre-mutation- * - dryRun does not mutate anything- *- * Note: tests run the orchestrator via direct engine manipulation rather- * than through the full migration-runner entry point. The runner is tested- * in test/apply-migrations.test.ts.+ * An upgrade runs v0.13.1 on any brain whose ledger never recorded it (a brain+ * `gbrain init` created before fresh-install stamps, or one opened from a new+ * GBRAIN_HOME). It used to stamp `validate: false` into every page and move+ * every content_hash. It must now record completion and change no page.+ * The whole upgrade runner is covered by test/upgrade-preserves-pages-7814.slow.test.ts.*/-import { describe, test, expect, beforeEach, afterAll } from 'bun:test';+import { describe, test, expect } from 'bun:test';import { tmpdir } from 'os';-import { mkdtempSync, rmSync, existsSync, readFileSync, writeFileSync, mkdirSync } from 'fs';+import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';import { join } from 'path';import { migrations, getMigration } from '../src/commands/migrations/index.ts';import { v0_13_1 } from '../src/commands/migrations/v0_13_1.ts';--// ----------------------------------------------------------------------------// Registry shape-// ---------------------------------------------------------------------------+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { withEnv } from './helpers/with-env.ts';describe('migrations registry', () => {- test('v0.13.1 is registered', () => {- const m = getMigration('0.13.1');- expect(m).not.toBeNull();- expect(m?.version).toBe('0.13.1');- });-- test('v0.13.1 is listed in semver order after v0.12.0', () => {+ test('v0.13.1 is registered in semver order after v0.12.0', () => {+ expect(getMigration('0.13.1')?.version).toBe('0.13.1');const versions = migrations.map(m => m.version);expect(versions.indexOf('0.13.1')).toBeGreaterThan(versions.indexOf('0.12.0'));});@@ -43,40 +31,44 @@ describe('migrations registry', () => {});});-// ----------------------------------------------------------------------------// Orchestrator behavior-// ----------------------------------------------------------------------------//-// The orchestrator reads config via loadConfig() which reads from-// ~/.gbrain/config.json. We can't easily stand that up in a test, so the-// test below validates the pieces we CAN test without the config flow:-// registry integration + shape of the migration module. Full end-to-end-// with a real engine + config is in test/e2e/migration-flow.test.ts.-//-// Idempotency behavior is verified by unit testing the writer path-// (test/writer.test.ts: "validators skip pages with validate:false-// frontmatter") and the per-page frontmatter preservation logic in the-// setFrontmatterField test.--describe('v0_13_1 orchestrator — dry-run path', () => {- const ORIG_HOME = process.env.HOME;- let tmpHome: string;+describe('v0_13_1 orchestrator writes no page (#7814)', () => {+ test('a brain with pages and no ledger keeps every page, content_hash and frontmatter unchanged', async () => {+ const home = mkdtempSync(join(tmpdir(), 'v0_13_1-'));+ try {+ const databasePath = join(home, '.gbrain', 'brain.pglite');+ mkdirSync(join(home, '.gbrain'), { recursive: true });+ writeFileSync(join(home, '.gbrain', 'config.json'), JSON.stringify({ engine: 'pglite', database_path: databasePath, embedding_disabled: true }) + '\n');+ const state = async () => {+ const engine = new PGLiteEngine();+ await engine.connect({ database_path: databasePath });+ try {+ return await engine.executeRaw('SELECT source_id, slug, content_hash, frontmatter::text AS frontmatter, knowledge_revision::text AS revision FROM pages ORDER BY slug');+ } finally { await engine.disconnect(); }+ };+ await withEnv({ GBRAIN_HOME: home, DATABASE_URL: undefined, GBRAIN_DATABASE_URL: undefined }, async () => {+ const engine = new PGLiteEngine();+ await engine.connect({ database_path: databasePath });+ try {+ await engine.initSchema();+ await engine.putPage('concepts/alpha-example', { type: 'concept', title: 'Alpha', compiled_truth: 'Alpha body.', frontmatter: { source_kind: 'note' } });+ await engine.putPage('people/alice-example', { type: 'person', title: 'Alice Example', compiled_truth: 'Alice body.' });+ } finally { await engine.disconnect(); }+ const before = await state();- beforeEach(() => {- tmpHome = mkdtempSync(join(tmpdir(), 'v0_13_1-'));- process.env.HOME = tmpHome;- });-- afterAll(() => {- process.env.HOME = ORIG_HOME;- });+ for (const dryRun of [false, true]) {+ const result = await v0_13_1.orchestrator({ yes: true, dryRun, noAutopilotInstall: true });+ expect(result.status).toBe('complete');+ expect(result.files_rewritten ?? 0).toBe(0);+ expect(result.phases.map(p => [p.name, p.status])).toEqual([['grandfather', 'skipped']]);+ }- test('dryRun skips the connect phase', async () => {- const result = await v0_13_1.orchestrator({ yes: true, dryRun: true, noAutopilotInstall: true });- const connectPhase = result.phases.find(p => p.name === 'connect');- expect(connectPhase?.status).toBe('skipped');- expect(connectPhase?.detail).toBe('dry-run');-- rmSync(tmpHome, { recursive: true, force: true });- });+ const after = await state();+ expect(after).toEqual(before);+ expect(after.map(row => String(row.frontmatter))).not.toContainEqual(expect.stringContaining('validate'));+ expect(existsSync(join(home, '.gbrain', 'migrations', 'v0_13_1-rollback.jsonl'))).toBe(false);+ });+ } finally {+ rmSync(home, { recursive: true, force: true });+ }+ }, 120_000);});diff --git a/test/persistence-git-coalescing-5530.slow.test.ts b/test/persistence-git-coalescing-5530.slow.test.tsindex 2505ac0874..cd7f1638b5 100644--- a/test/persistence-git-coalescing-5530.slow.test.ts+++ b/test/persistence-git-coalescing-5530.slow.test.ts@@ -1,7 +1,7 @@/*** #5530: the effect runner coalesces ready single-file Git effects.*- * The v0.13.1 grandfather step admits one put_page per page; before this fix+ * A bulk rewrite admits one put_page per page; before this fix* every page's Git effect made its own `git commit` and its own synchronous* `git push`. The runner now claims up to 100 ready single-file Git effects* for the same worktree, validates each path under the worktree lock, makes@@ -22,18 +22,18 @@ import { join } from 'node:path';import { createHash, randomUUID } from 'node:crypto';import type { BrainEngine } from '../src/core/engine.ts';import type { OperationContext } from '../src/core/ops/contract.ts';-import { phaseCGrandfather } from '../src/commands/migrations/v0_13_1.ts';-import { admitCanonicalGrandfather } from '../src/core/persistence/grandfather.ts';import { acquireWorktree, claimWorktree, getWorktreeBinding } from '../src/core/persistence/ownership.ts';import { activateSharedSkillPersistence } from '../src/core/persistence/skill-activation.ts';-import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { preparePageAdmission, submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { admitWrite } from '../src/core/persistence/journal.ts';+import { PAGE_BATCH_MAX_PAGES, submitPageBatch } from '../src/core/persistence/page-batch.ts';import { disposePersistenceConsumer, startPersistenceConsumer } from '../src/core/persistence/service.ts';import { claimCoalescedGitEffects } from '../src/core/persistence/effect-journal.ts';import { localHostId } from '../src/core/persistence/identity.ts';import { testBackends } from './helpers/test-backends.ts';import { withEnv } from './helpers/with-env.ts';import { configureGateway, resetGateway } from '../src/core/ai/gateway.ts';-import { type Repo, bindSource, createGitCoalescingFixture, effectSql, git, gitStates, harden, pageContent, pass, pauseGitEffects, putPage, release, seed, seedSequentially } from './helpers/git-coalescing-fixture.ts';+import { type Repo, bindSource, createGitCoalescingFixture, effectSql, git, gitStates, harden, pageContent, pageSlug, pass, pauseGitEffects, putPage, release, seed, seedSequentially } from './helpers/git-coalescing-fixture.ts';const { makeRepo, withBrain, dispose } = createGitCoalescingFixture('5530');afterAll(dispose);@@ -46,6 +46,15 @@ function stubGit(home: string, body: string): string {return dir;}+/** Update pages 0..count-1 through put_pages batches: one managed Git effect per page, as a bulk rewrite admits. */+async function updateBurst(ctx: OperationContext, count: number): Promise<void> {+ for (let start = 0; start < count; start += PAGE_BATCH_MAX_PAGES) {+ const pages = await Promise.all(Array.from({ length: Math.min(PAGE_BATCH_MAX_PAGES, count - start) }, async (_, k) => ({ slug: pageSlug(start + k), content: pageContent(start + k, ' updated'),+ expected_revision: (await ctx.engine.readPageSnapshot(pageSlug(start + k), { sourceId: 'default' }))!.revision })));+ expect((await submitPageBatch(ctx, { request_id: randomUUID(), pages })).state).toBe('committed');+ }+}+const PAGES = Number(process.env.GBRAIN_TEST_COALESCE_PAGES ?? 250);// Six full groups whose commits each take a second: on master the write waits out about six seconds of them.const BACKLOG = 600, COMMIT_SLEEP_S = 1;@@ -96,13 +105,13 @@ for (const kind of testBackends()) describe(`#5530 Git effect coalescing (${kindexpect(await snapshot(seed)).toEqual(sequential);}, 300_000);- test(`${PAGES} grandfathered pages give ${Math.ceil(PAGES / 100)} commits and 1 push; an interleaved write still completes`, () => withBrain(kind, async ({ engine, home, ctx }) => {+ test(`${PAGES - 1} updated pages plus one write give ${Math.ceil(PAGES / 100)} commits and 1 push; an interleaved write still completes`, () => withBrain(kind, async ({ engine, home, ctx }) => {const repo = makeRepo(home, 'content');await bindSource(engine, 'default', repo);await activateSharedSkillPersistence(engine, { confirmQuiesced: true });// The seed's own Git effects are setup, not under test: hold them while// seeding (the consumer would otherwise probe the worktree once per page)- // and apply them all here, before hardening, so only the grandfather's and+ // and apply them all here, before hardening, so only the update burst's and// the interleaved write's effects are pending below.await pauseGitEffects(engine, () => seed(ctx('default'), PAGES - 1));for (let i = 0; (await gitStates(engine)).queued && i < 50; i++) { await release(engine); await pass(engine); }@@ -110,9 +119,8 @@ for (const kind of testBackends()) describe(`#5530 Git effect coalescing (${kindharden(repo);const commitsBefore = repo.commits();await pauseGitEffects(engine, async () => {- const result = await phaseCGrandfather(engine, { yes: true, dryRun: false, noAutopilotInstall: true } as never);- expect(result.detail).toMatchObject({ touched: PAGES - 1, failed: 0 });- // An ordinary write on the same source publishes while the grandfather's Git work is still pending.+ await updateBurst(ctx('default'), PAGES - 1);+ // An ordinary write on the same source publishes while the update burst's Git work is still pending.const ordinary = await submitPageMutation(ctx('default'), { operation: 'put_page', params: { slug: 'notes/ordinary', request_id: randomUUID(), content: pageContent(9999) } }) as { state?: string };expect(ordinary.state).toBe('committed');});@@ -209,8 +217,9 @@ for (const kind of testBackends()) describe(`#5530 Git effect coalescing (${kindconst [target] = await engine.executeRaw<{ id: number; slug: string; source_id: string; source_incarnation: string }>("SELECT p.id,p.slug,p.source_id,s.incarnation AS source_incarnation FROM pages p JOIN sources s ON s.id=p.source_id WHERE p.slug='notes/page-0500'");// Admitted and not yet published: the publication is queued for the same worktree.- const admitted = await admitCanonicalGrandfather(engine, target!, () => {});- expect(admitted.status).toBe('admitted');+ const update = { operation: 'put_page', params: { slug: target!.slug, request_id: randomUUID(), content: pageContent(500, ' updated'),+ expected_revision: (await engine.readPageSnapshot(target!.slug, { sourceId: 'default' }))!.revision } };+ await admitWrite(engine, (await preparePageAdmission(ctx('default'), update)).admission!);await release(engine);await pass(engine);expect(repo.commits()).toBe(commitsBefore);@@ -222,7 +231,7 @@ for (const kind of testBackends()) describe(`#5530 Git effect coalescing (${kindawait pass(engine);expect(repo.commits() - commitsBefore).toBe(1);expect(await gitStates(engine)).toEqual({ committed: 4 });- if (admitted.status === 'admitted') expect((await admitted.complete()).revision).toBeTruthy();+ expect((await submitPageMutation(ctx('default'), update)).state).toBe('committed');}), 300_000);test('sources sharing one worktree are each validated against their own binding', () => withBrain(kind, async ({ engine, home, ctx }) => {const repo = makeRepo(home, 'content');diff --git a/test/upgrade-preserves-pages-7814.slow.test.ts b/test/upgrade-preserves-pages-7814.slow.test.tsnew file mode 100644index 0000000000..b3468b196f--- /dev/null+++ b/test/upgrade-preserves-pages-7814.slow.test.ts@@ -0,0 +1,17 @@+/**+ * #7814 on PGLite (the Postgres run is test/e2e/upgrade-preserves-pages-7814-postgres.test.ts).+ *+ * Protects: upgrading a brain created by v0.59.2.0 (pages, no fresh-install+ * ledger stamps) through the real `gbrain apply-migrations` runner changes no+ * page, repo-less and repo. Fails when any historical orchestrator rewrites+ * page content or frontmatter on upgrade, as v0.13.1's `validate: false`+ * grandfather did. Contract: test/helpers/upgrade-preserves-pages-contract.ts.+ */+import { expect, test } from 'bun:test';+import { runUpgradePreservesPages } from './helpers/upgrade-preserves-pages-contract.ts';++for (const shape of ['repo-less', 'repo'] as const) {+ test(`pglite ${shape}: upgrading a v0.59.2.0-shaped brain changes no page content_hash`, async () => {+ await runUpgradePreservesPages(undefined, shape, expect);+ }, 900_000);+}