Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Refuse a foreign OAuth table before schema setup and replay the schema on the apply-migrations rerun
garrytan44f056df1a99
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b6b7959adf..feeb4ba9a7 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,18 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**An upgrade no longer adopts another application's OAuth table and then stops halfway, and rerunning `gbrain apply-migrations --force-schema` now recovers a brain that stopped.** gbrain creates `oauth_tokens`, `oauth_clients` and `oauth_codes` with `CREATE TABLE IF NOT EXISTS`, so when another application sharing the Postgres database already had a table with one of those names, gbrain took it over, added its own columns to it, and a later upgrade stopped at v210 with `operator does not exist: timestamp with time zone > bigint`. That error named neither the table nor the column, and it came after earlier migrations in the same run had already applied. The documented recovery, rerunning the same command, then could not create gbrain's own table.++### Itemized changes++- **Schema setup refuses a table gbrain did not create, before changing anything.** Before the schema replay (`gbrain init`, every engine start that applies migrations) and before any migration in a run, gbrain checks that each OAuth table that already exists has the key column and the column types gbrain has always given it. When one does not, it stops with the new `foreign_table` error, which names the table, the column, the type it found, the type gbrain needs and the fix: give gbrain its own database, or move the other table with `ALTER TABLE <table> SET SCHEMA <other_schema>`, then run `gbrain apply-migrations --force-schema --yes`. A correctly shaped brain runs exactly as before, and v210 is unchanged. Guide: [foreign table](docs/guides/troubleshooting.md#foreign-table).+- **`gbrain apply-migrations` replays the schema before the pending migrations on every path.** With the migration lease held (every PGLite brain, and every Postgres brain whose schema has the lease table) it ran the pending migrations without the schema replay, so a table the replay creates that was missing stayed missing and the next migration failed with `relation "<table>" does not exist`. It now goes through the same schema setup as `gbrain init --migrate-only` on every path. A brain an older release stopped at v210 recovers with `gbrain upgrade` and the steps the guide lists.+- Tests: `test/upgrade-foreign-oauth-table.test.ts` runs on PGLite and Postgres: the refusal on an upgrade (a foreign `oauth_tokens`) and on a fresh install (a foreign `oauth_codes` or `oauth_clients`), the rerun that recreates a missing table, and the full recovery of a brain stopped at v210.++Reported by @jonahberg (#9190, #9191); first fixed by @Masashi-Ono0611 in #9195 and #9193.+## [0.60.691.0] - 2026-11-12**A privacy label repair on a database-only page now sticks, and a supported command stops tracking a folder's files without deleting their pages.** Pages under a folder declared `storage.db_only` in `gbrain.yml` live in the database. Some of those folders still had files Git tracked from before the declaration, and on a git-canonical source every sync put the committed file back over the database, undoing a `gbrain repair visibility` that had made the page private. Sync now keeps the database version there, a commit to such a file can no longer widen who reads the page, and `gbrain git commit-push --untrack` takes the old files out of Git safely so the problem cannot come back.diff --git a/docs/architecture/key-files/engines-2-continued.md b/docs/architecture/key-files/engines-2-continued.mdindex 95ee00150f..2a070ffb81 100644--- a/docs/architecture/key-files/engines-2-continued.md+++ b/docs/architecture/key-files/engines-2-continued.md@@ -8,4 +8,5 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/analyze-lock.ts` — #7083: `lockTableForAnalyze` takes a table's SHARE UPDATE EXCLUSIVE lock with NOWAIT before every ANALYZE gbrain runs on Postgres on its own (the projection and chunk refreshes, `repair planner-stats`, `analyzeAfterIndexBuild` through `analyzeYieldingToVacuum`), and `analyzeDeferred` recognizes its 55P03 refusal. An ANALYZE that waited `deadlock_timeout` on a running autovacuum made Postgres cancel it, so a refresh after every embed drain kept a 10M-chunk content_chunks from ever being vacuumed; a request that never waits never signals a vacuum, and the caller defers its ANALYZE. Doctor `planner_stats_stale` reports a hot table whose autovacuum stays starved (`readVacuumStarvation`). Test: `test/e2e/analyze-yields-to-autovacuum-postgres.test.ts`.- `src/core/migration-preludes.ts` — #7108 runner-side preludes for published migrations whose SQL rewrites a large table in one transaction (published migration files never change; restamp refuses an edit). `runMigrationPrelude(engine, version)` runs right before that migration's SQL inside the runner's retry wrapper: `batchedUpdate` repeats the migration's own SET and predicate over id-ordered batches (5,000 rows, inlined so a generic prepared plan cannot scan the table per batch; `__setBackfillBatchRowsForTests`), each its own transaction with the migration statement budget and the schema lock timeout, so progress is the data itself and a rerun resumes; a column the migration adds first is added first, in the migration's column order (fresh and upgraded ordinals stay equal). Preludes: v011 legacy `links.link_source` (skipped below PG15, where the migration refuses), v028 chunk keyword vectors, v033 request-log agent names (NULL-token rows stay the migration's own), v116 code-edge `source_id`, v161 (Postgres) the partial pages index built concurrently under the budget on a reserved connection (the migration caps its own build at 30 s; another session's build or an invalid result stops the run), v221 `embedding_pending_since` (v221's active-column choice; the first stamping batch writes `embedding_pending_since_backfilled_at` in its transaction). Pinned by `test/e2e/migration-v221-batched-backfill-7108.test.ts`. Migrations that cannot be moved here are listed in docs/guides/troubleshooting.md#migration-statement-timeout.- `src/core/migration-timeout.ts` — #7108 the migration statement budget (`migrationStatementTimeoutMs`, default 10 min; `setMigrationStatementTimeoutMs` from operator-only `gbrain apply-migrations --statement-timeout <minutes>`, restored when that run returns; 0 removes it) and the timed-out-migration record (config `schema_migration_timeout`: version, name, whether the run had a prelude, idle-in-transaction pids sampled at the failure): `runMigrations` writes it on a 57014 or a statement-timeout retry exhaustion and deletes it after a pass reaches head. A schema-graph leaf (no release version, so the PGLite snapshot hash stays stable). `migration-timeout-advice.ts` turns a record into the one fix both apply-migrations (`migrations_pending` still_behind) and doctor `schema_version` show: timed out unbatched and this release has a prelude → rerun; idle sessions open → ask_user, check `doctor --locks`, end the ones still open, rerun; none → ask_user, one run with `--statement-timeout 120`.+- `src/core/owned-table-shapes.ts` — #9190 the shape check for tables gbrain owns by a generic name (`oauth_clients`, `oauth_tokens`, `oauth_codes`): one catalog query in `current_schema()` compares each existing table's key column and v32 column types with what gbrain has always created, and `assertOwnedTableShapes` throws `foreign_table` (table, column, type found, type needed, the `SET SCHEMA` fix) on the first mismatch. `applyForwardReferenceBootstrap` (engine-sql/bootstrap.ts) runs it before any DDL, so `initSchema` on both engines refuses before the schema replay, and `runMigrations` runs it before its repairs and the first pending migration. Pinned on both engines by `test/upgrade-foreign-oauth-table.test.ts`.- `src/core/derived-state-maintainers.ts` + `src/core/schema-migrations/v281-derived-state-maintainers-fail-closed.ts` — #7637: the eight derived-state maintainer triggers (page retrievals, atom scan state, tag revision, version trust and attribution, page search vector, projection queue, trust generation) stay SECURITY INVOKER and fail closed. `requireMaintainerPoliciesSql(table, commands)` is the PL/pgSQL each body runs before touching another table: when `row_security_active(table)` holds and no permissive `pg_policy` row for the caller (PUBLIC or a role it has USAGE of) covers a command, it raises `writer_coordinator_required` (`managed_writer_guard:unreadable_state`, DETAIL `state` = the table), so an empty read never counts as nothing to do. Each body lives with its schema (`TAG_REVISION_FUNCTION_SQL`, `SNAPSHOT_VERSION_FUNCTION`, `STAMP_VERSION_FUNCTION`, `BUMP_FUNCTION`, `PAGE_PROJECTION_QUEUE_FUNCTION_SQL`, `pageSearchVectorTriggerFnSql`); the page retrievals body is a schema blob fragment here. v281 replaces only the functions (the search vector under the configured FTS language), never the triggers. The policies a scoped writer needs are the ENGINES.md maintainer recipe (`test/derived-state-maintainers-7637.test.ts`, applied verbatim).diff --git a/docs/guides/error-codes.md b/docs/guides/error-codes.mdindex 4b431e6f39..16b63c85ac 100644--- a/docs/guides/error-codes.md+++ b/docs/guides/error-codes.md@@ -1938,6 +1938,16 @@ More: [docs/guides/repair.md#fix-not-writable](../../docs/guides/repair.md#fix-nMore: [docs/guides/shared-brain-skills.md#approve-publication-following-and-editing-separately](../../docs/guides/shared-brain-skills.md#approve-publication-following-and-editing-separately)+### foreign_table++<a id="foreign_table"></a>++| Meaning | Why | Next step | Who acts · consent | Verify | Exit | Retryable |+|---|---|---|---|---|---|---|+| A table gbrain owns by name (an OAuth table) already exists in the brain's schema with a column type gbrain never gives it, so schema setup stopped before the schema replay or any migration changed anything; the message names the table, the column, the type found and the type gbrain needs. | gbrain creates its tables with CREATE TABLE IF NOT EXISTS, so without this check it adopted a same-named table another application created in the shared database, added its own columns to it and failed partway through an upgrade on the first statement that compared the wrong type (#9190). The table holds that application's data, so gbrain never moves, renames or drops it. | Ask the user: give gbrain its own database, or move the other table to a schema of its own (ALTER TABLE <table> SET SCHEMA <other_schema> takes its indexes and constraints along) and point that application there; then run gbrain apply-migrations --force-schema --yes. | user | `gbrain doctor --json` | 1 | no |++More: [docs/guides/troubleshooting.md#foreign-table](../../docs/guides/troubleshooting.md#foreign-table)+### forget_requires_owner<a id="forget_requires_owner"></a>diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.mdindex b1eb159fbb..52f553f07a 100644--- a/docs/guides/troubleshooting.md+++ b/docs/guides/troubleshooting.md@@ -26,6 +26,7 @@| Doctor `links_extraction_lag` or sync says "N page(s) have un-extracted edges" | `gbrain extract --stale --dry-run --json`, then on the host `gbrain extract --stale --catch-up [--source-id <id>]` | user above 120 s, else agent | none | dry run: `actionable: 0` || Doctor `links_extraction_lag` says "N page(s) failed link extraction 3 runs in a row and are held", or `extract --stale` lists failed pages | Read each page's code (`unknown_source`: the page's source row is gone; restore the source or delete the page), fix the cause, then `gbrain extract --stale [--source-id <id>] --retry-held` | agent | none | `gbrain extract --stale --dry-run --json`: `held: 0` || `gbrain apply-migrations --yes` or `gbrain upgrade` stops with `Migration <N> … exhausted retries` / SQLSTATE 57014 and the schema stays behind (`migrations_pending`, reason `still_behind`) | upgrade first (#7108: long backfills run in committed batches), then follow the error's or doctor `schema_version`'s `fix` ([migration statement timeout](#migration-statement-timeout)) | agent; a longer `--statement-timeout` or ending a session only after the user agrees | none for a rerun | `gbrain doctor --only schema_version --json` |+| `gbrain apply-migrations`, `gbrain upgrade` or `gbrain init` stops with `foreign_table` ("`<schema>.<table>.<column>` is `<type>`, but gbrain's `<table>` needs `<type>`"), or an older release stopped at v210 with `operator does not exist: timestamp with time zone > bigint` | a table another application created under one of gbrain's names (`oauth_tokens`, `oauth_clients`, `oauth_codes`) is in the same schema; nothing was changed. Ask the user, then move that table or give gbrain its own database ([foreign table](#foreign-table)) and run `gbrain apply-migrations --force-schema --yes` | user (the other application's table); agent for the rerun | none for the rerun | `gbrain doctor --only schema_version --json` || A gbrain call failed with a code you don't recognize | follow `fix.next` ([protocol](../protocol/AGENT_OPERATOR_v1.md)); `gbrain errors <code>` | agent | as the fix's `consent` | the fix's `verify` || A graph read omits a relationship you know existed, or says someone still works somewhere they left | the default read returns relationships true today: repeat with `status: "all"` or `as_of`; to record an end, add a dated timeline line (`Ended works_at [[companies/x]]`) or `add_link ... valid_until` ([temporal edges](temporal-edges.md)) | agent | none | `gbrain doctor --only edge_validity --json` || A command exited 3 (`confirmation_required`) | relay `user_message`; run `fix.command` only after the user agrees | agent, after the user agrees | the payload's `effects` | the fix's `verify` |@@ -648,6 +649,43 @@ the budget for migration SQL and for the batches above. It doesn't lift the30-second caps that v160, v200 and v234 set on themselves, or the timeouts ofmigration handlers.+## A table gbrain did not create is in the way++<a id="foreign-table"></a>++gbrain creates its tables with `CREATE TABLE IF NOT EXISTS` in the brain's schema+(usually `public`). When another application shares the database and already has a+table with one of gbrain's generic names (`oauth_tokens`, `oauth_clients`,+`oauth_codes`), older releases adopted it, added their own columns to it and then+stopped partway through an upgrade on the first statement that compared the wrong+type, for example v210 with `operator does not exist: timestamp with time zone > bigint`+(#9190). Schema setup now checks those tables before the schema replay and before any+migration, and stops with `foreign_table`, naming the table, the column, the type it+found and the type gbrain needs. Nothing was changed.++The table holds the other application's data, so ask the user before touching it:++1. Give gbrain a database of its own, or move the other table to a schema of its own.+ Moving takes its indexes and constraints along:+ ```sql+ CREATE SCHEMA other_app_example;+ ALTER TABLE public.oauth_tokens SET SCHEMA other_app_example;+ ```+ Then point that application at the new schema (for example+ `ALTER ROLE other_app_role SET search_path = other_app_example, public`).+2. Run `gbrain apply-migrations --force-schema --yes`. It replays the schema, which+ creates gbrain's own table, and then applies the pending migrations. Since #9191+ this works on a rerun too: the rerun used to skip the replay, so a table the replay+ creates stayed missing (`relation "oauth_tokens" does not exist`).+ `gbrain init --migrate-only --non-interactive` does the same.+3. Verify with `gbrain doctor --only schema_version --json`.++A brain an older release stopped at v210 recovers with the same steps after+`gbrain upgrade`. That release had already added gbrain's columns (`token_type`,+`scopes`, `resource`) to the other table and may have turned on row-level security+for it. Tell the user, so the other application's owner can check: a role that does+not own the table sees no rows while row-level security is on and no policy admits it.+## Global maintenance timeouts**Doctor warns `global_maintenance_timeouts`, or late maintenance phasesdiff --git a/llms-full.txt b/llms-full.txtindex 343522ccb1..abe9117666 100644--- a/llms-full.txt+++ b/llms-full.txt@@ -9306,6 +9306,7 @@ Source: https://raw.githubusercontent.com/garrytan/gbrain/master/docs/guides/tro| Doctor `links_extraction_lag` or sync says "N page(s) have un-extracted edges" | `gbrain extract --stale --dry-run --json`, then on the host `gbrain extract --stale --catch-up [--source-id <id>]` | user above 120 s, else agent | none | dry run: `actionable: 0` || Doctor `links_extraction_lag` says "N page(s) failed link extraction 3 runs in a row and are held", or `extract --stale` lists failed pages | Read each page's code (`unknown_source`: the page's source row is gone; restore the source or delete the page), fix the cause, then `gbrain extract --stale [--source-id <id>] --retry-held` | agent | none | `gbrain extract --stale --dry-run --json`: `held: 0` || `gbrain apply-migrations --yes` or `gbrain upgrade` stops with `Migration <N> … exhausted retries` / SQLSTATE 57014 and the schema stays behind (`migrations_pending`, reason `still_behind`) | upgrade first (#7108: long backfills run in committed batches), then follow the error's or doctor `schema_version`'s `fix` ([migration statement timeout](#migration-statement-timeout)) | agent; a longer `--statement-timeout` or ending a session only after the user agrees | none for a rerun | `gbrain doctor --only schema_version --json` |+| `gbrain apply-migrations`, `gbrain upgrade` or `gbrain init` stops with `foreign_table` ("`<schema>.<table>.<column>` is `<type>`, but gbrain's `<table>` needs `<type>`"), or an older release stopped at v210 with `operator does not exist: timestamp with time zone > bigint` | a table another application created under one of gbrain's names (`oauth_tokens`, `oauth_clients`, `oauth_codes`) is in the same schema; nothing was changed. Ask the user, then move that table or give gbrain its own database ([foreign table](#foreign-table)) and run `gbrain apply-migrations --force-schema --yes` | user (the other application's table); agent for the rerun | none for the rerun | `gbrain doctor --only schema_version --json` || A gbrain call failed with a code you don't recognize | follow `fix.next` ([protocol](../protocol/AGENT_OPERATOR_v1.md)); `gbrain errors <code>` | agent | as the fix's `consent` | the fix's `verify` || A graph read omits a relationship you know existed, or says someone still works somewhere they left | the default read returns relationships true today: repeat with `status: "all"` or `as_of`; to record an end, add a dated timeline line (`Ended works_at [[companies/x]]`) or `add_link ... valid_until` ([temporal edges](temporal-edges.md)) | agent | none | `gbrain doctor --only edge_validity --json` || A command exited 3 (`confirmation_required`) | relay `user_message`; run `fix.command` only after the user agrees | agent, after the user agrees | the payload's `effects` | the fix's `verify` |@@ -9928,6 +9929,43 @@ the budget for migration SQL and for the batches above. It doesn't lift the30-second caps that v160, v200 and v234 set on themselves, or the timeouts ofmigration handlers.+## A table gbrain did not create is in the way++<a id="foreign-table"></a>++gbrain creates its tables with `CREATE TABLE IF NOT EXISTS` in the brain's schema+(usually `public`). When another application shares the database and already has a+table with one of gbrain's generic names (`oauth_tokens`, `oauth_clients`,+`oauth_codes`), older releases adopted it, added their own columns to it and then+stopped partway through an upgrade on the first statement that compared the wrong+type, for example v210 with `operator does not exist: timestamp with time zone > bigint`+(#9190). Schema setup now checks those tables before the schema replay and before any+migration, and stops with `foreign_table`, naming the table, the column, the type it+found and the type gbrain needs. Nothing was changed.++The table holds the other application's data, so ask the user before touching it:++1. Give gbrain a database of its own, or move the other table to a schema of its own.+ Moving takes its indexes and constraints along:+ ```sql+ CREATE SCHEMA other_app_example;+ ALTER TABLE public.oauth_tokens SET SCHEMA other_app_example;+ ```+ Then point that application at the new schema (for example+ `ALTER ROLE other_app_role SET search_path = other_app_example, public`).+2. Run `gbrain apply-migrations --force-schema --yes`. It replays the schema, which+ creates gbrain's own table, and then applies the pending migrations. Since #9191+ this works on a rerun too: the rerun used to skip the replay, so a table the replay+ creates stayed missing (`relation "oauth_tokens" does not exist`).+ `gbrain init --migrate-only --non-interactive` does the same.+3. Verify with `gbrain doctor --only schema_version --json`.++A brain an older release stopped at v210 recovers with the same steps after+`gbrain upgrade`. That release had already added gbrain's columns (`token_type`,+`scopes`, `resource`) to the other table and may have turned on row-level security+for it. Tell the user, so the other application's owner can check: a role that does+not own the table sees no rows while row-level security is on and no policy admits it.+## Global maintenance timeouts**Doctor warns `global_maintenance_timeouts`, or late maintenance phasesdiff --git a/scripts/postgres-arm-weights-postgres-only.json b/scripts/postgres-arm-weights-postgres-only.jsonindex 814b1c8bc5..484c90e40a 100644--- a/scripts/postgres-arm-weights-postgres-only.json+++ b/scripts/postgres-arm-weights-postgres-only.json@@ -411,6 +411,7 @@"test/trust-owner-actions.test.ts": 13580,"test/unbound-source-6536.test.ts": 1896,"test/unindexed-files.test.ts": 5892,+ "test/upgrade-foreign-oauth-table.test.ts": 4652,"test/vector-manifest-probe-bound.test.ts": 23971,"test/withdrawal-overlay-concurrency.test.ts": 4425,"test/withdrawal-publication-file.test.ts": 9435,diff --git a/scripts/postgres-arm-weights.json b/scripts/postgres-arm-weights.jsonindex af5b90cb1b..914fc2b4ff 100644--- a/scripts/postgres-arm-weights.json+++ b/scripts/postgres-arm-weights.json@@ -658,6 +658,7 @@"test/unbound-source-6536.test.ts": 8163,"test/unindexed-files-convergence-7149.test.ts": 15915,"test/unindexed-files.test.ts": 20566,+ "test/upgrade-foreign-oauth-table.test.ts": 13356,"test/url-source-origin.test.ts": 54860,"test/v273-validate-timeouts.test.ts": 4976,"test/valid-time-gbra217.test.ts": 28349,diff --git a/src/commands/apply-migrations.ts b/src/commands/apply-migrations.tsindex 2f8be17b47..651e08b9b7 100644--- a/src/commands/apply-migrations.ts+++ b/src/commands/apply-migrations.ts@@ -43,7 +43,7 @@ import { clearRepoPathWedge } from './migrations/v0_11_0-install.ts';import { shellQuote, type Effect } from '../core/agent-output.ts';import { writeJsonDocument } from '../core/cli-force-exit.ts';import { consentGate } from '../core/consent-cli.ts';-import { opError, type OperationError } from '../core/ops/contract.ts';+import { opError, OperationError } from '../core/ops/contract.ts';import { exitCliError, usageError, writeCliError } from '../cli/cli-error.ts';import {checkSchemaAfterOrchestrators,@@ -641,13 +641,13 @@ async function runLockedMigrations(): Promise<number | undefined> {const { doc } = report;const fail = (status: string, failure: OperationError): void => { doc.status = status; report.failure = failure; };- // Without the orchestration lease (a schema too old for it), schema work goes- // through initSchema(), which serializes on the schema lock.+ // Schema work goes through initSchema(), lease or not: it takes the schema migration lock and replays the+ // idempotent schema blob before the pending migrations, so a blob-owned table that is missing is back first (#9191).const migrateSchema = async (eng: BrainEngine, from: number): Promise<{ applied: number; current: number }> => {- if (heldLock()) { const { runMigrations } = await import('../core/migrate.ts'); return runMigrations(eng); }await eng.initSchema();const current = parseInt(await eng.getConfig('version') || String(from), 10);- return { applied: Math.max(0, current - from), current };+ const { MIGRATIONS } = await import('../core/migrate.ts');+ return { applied: MIGRATIONS.filter(m => m.version > from && m.version <= current).length, current };};// Bug 3 — --force-retry: write an explicit reset marker for a wedged// migration, then return. User re-runs `gbrain apply-migrations --yes`@@ -710,6 +710,7 @@ async function runLockedMigrations(await eng.disconnect();} catch (err) {console.error(`--force-schema failed: ${(err as Error).message}`);+ if (err instanceof OperationError && err.code === 'foreign_table') { fail('failed', err); return 1; }fail('failed', opError('migration_failed', `--force-schema failed: ${(err as Error).message}`,'Fix the database error above, then run `gbrain apply-migrations --force-schema` again.', { reason: 'schema_failed' }));return 1;diff --git a/src/core/engine-sql/bootstrap.ts b/src/core/engine-sql/bootstrap.tsindex 80dce5ade0..35b598312e 100644--- a/src/core/engine-sql/bootstrap.ts+++ b/src/core/engine-sql/bootstrap.ts@@ -11,6 +11,9 @@* caller's connection, which MUST hold the initSchema advisory lock (key* 42) so concurrent bootstraps can't race on a transaction pooler.*+ * Before any DDL it refuses (`foreign_table`, owned-table-shapes.ts) when a table gbrain owns by name was+ * created by another application, so the blob never adopts and alters it (#9190).+ ** The blob forward-references state that older brains don't have yet (a* column-with-index or FK target that a migration adds later). One probe* round-trip finds every gap; fresh installs and modern brains no-op.@@ -39,6 +42,7 @@import type { PGlite } from '@electric-sql/pglite';import type postgres from '#postgres';import { GRANT_COLUMNS_SQL } from '../grants/schema.ts';+import { assertOwnedTableShapes } from '../owned-table-shapes.ts';type ProbeRow = Record<string, boolean>;@@ -338,6 +342,7 @@ type ForwardReferenceGaps = ReturnType<typeof forwardReferenceGaps>;/** Probe once, then add every missing forward-referenced piece of state in master's order. */export async function applyForwardReferenceBootstrap(target: ForwardReferenceBootstrapTarget): Promise<void> {+ await assertOwnedTableShapes(sql => target.probe(sql));const gaps = forwardReferenceGaps(await target.probe(forwardReferenceProbeSql(target.dialect)), target.dialect);const {needsPagesBootstrap, needsLinksBootstrap, needsChunksBootstrap, needsPagesDeletedAt,diff --git a/src/core/error-registry.ts b/src/core/error-registry.tsindex 49aeca0d34..6330b71b24 100644--- a/src/core/error-registry.ts+++ b/src/core/error-registry.ts@@ -354,6 +354,7 @@ export const CODES = {file_wins_rewrite: { class: 'caller', retryable: false, actor: 'user', summary: "gbrain repair file-drift --prefer file refused a page because keeping its file would need a full re-render of the file, not only appended protected parts; nothing was written.", why: "File wins keeps the canonical file's bytes and adds only what reconcile's protections keep from the database: protected frontmatter keys the file lacks and database-only timeline rows, appended. A result that also changes a value, a tag, the title or the body would re-render the whole file (keys reordered, lists re-emitted, rules and headings rewritten), which is not the file winning.", docs: 'docs/guides/write-refusals.md#file_wins_rewrite', suggestion: 'Show the user the page\'s exceptions in the preview. Commit the database-only differences into the file through the door (`gbrain git commit-push`) and preview again, or preview `--prefer database` if the database copy should win.' },file_too_large: { class: 'caller', summary: "The file is over the page size limit (24 MiB for a Markdown page, 5 MB for a code file), so it was not imported.", why: "Every page under the limit is stored, chunked and searchable whatever its size; the limit bounds what one page may hold in this release. The same bytes refuse on every retry. A smaller page refused by an MCP or HTTP transport is a different refusal.", docs: 'docs/guides/write-refusals.md#file_too_large', suggestion: 'Leave the file out of the source (sync.exclude), or, if it must be in the brain, split it into files under the limit; then sync or import again. Never split a page under the limit.' },follow_approval_required: { class: 'caller', summary: "Following a brain's shared skills needs the user's explicit follow approval.", docs: 'docs/guides/shared-brain-skills.md#approve-publication-following-and-editing-separately' },+ foreign_table: { class: 'host_only', actor: 'user', retryable: false, summary: "A table gbrain owns by name (an OAuth table) already exists in the brain's schema with a column type gbrain never gives it, so schema setup stopped before the schema replay or any migration changed anything; the message names the table, the column, the type found and the type gbrain needs.", why: "gbrain creates its tables with CREATE TABLE IF NOT EXISTS, so without this check it adopted a same-named table another application created in the shared database, added its own columns to it and failed partway through an upgrade on the first statement that compared the wrong type (#9190). The table holds that application's data, so gbrain never moves, renames or drops it.", docs: 'docs/guides/troubleshooting.md#foreign-table', suggestion: 'Ask the user: give gbrain its own database, or move the other table to a schema of its own (ALTER TABLE <table> SET SCHEMA <other_schema> takes its indexes and constraints along) and point that application there; then run gbrain apply-migrations --force-schema --yes.' },forget_requires_owner: { class: 'consent', exit: 3, actor: 'user', summary: "A caller asked to forget a fact (remote forget) or remove a take (takes_remove) more trusted than its own writes, so nothing was removed and the owner was asked instead.", why: "Lower-trust writers cannot overwrite, supersede or forget what the owner confirmed or curated (#5575 guarded supersession). The request became a trust proposal the owner can accept on the brain host; forget's frozen expired meaning is unchanged because nothing expired.", docs: 'docs/guides/error-codes.md#forget_requires_owner', suggestion: 'Do not retry. Tell the user the fact or take needs their decision and give them the command in fix (gbrain trust confirm <proposal_ref> on the brain host).' },reserved_frontmatter_key: { class: 'caller', summary: 'A remote page write tried to add or change a frontmatter key only a trusted derived writer sets (`retired_by`, `retired_at`, `retired_generation`); nothing was written.', why: 'Those keys mark a page a derived writer retired, which the purge holds for 30 days and the repairs treat as machine-owned. A remote caller that could set them could make a page look machine-retired. Carrying the stored value unchanged, or dropping the key, is allowed.', suggestion: 'Remove the key from the frontmatter (or keep the stored value) and write the page again.' },generation_fenced: { class: 'retryable', exit: 11, reasons: ['stale_generation', 'pin_moved'], summary: 'A derived result was not published because the live generation or its input changed after the producer read them; nothing was written.', why: 'The generation primitive (GBRA-161) publishes a result only against the generation and input pin it was computed from. A concurrent run that published first (stale_generation), or an input edited since the read (pin_moved), would otherwise let an older result replace a newer one.', docs: 'docs/architecture/generations.md#publishing', suggestion: 'Run the producer again; it reads the current generation and input and recomputes.' },diff --git a/src/core/migrate.ts b/src/core/migrate.tsindex bebf7be332..94550f9e3c 100644--- a/src/core/migrate.ts+++ b/src/core/migrate.ts@@ -6,6 +6,7 @@ import { SCHEMA_MIGRATION_LOCK_APPLICATION_NAME } from './postgres-engine/init-simport { assertSchemaMigrationLockHeld, withSchemaMigrationLock } from './schema-migration-lock.ts';import { MIGRATIONS } from './schema-migrations/registry.generated.ts';import { hasPendingMigrations } from './migrate-pending.ts';+import { assertOwnedTableShapes } from './owned-table-shapes.ts';import { runMigrationStatement, setQuietMigrationNotices } from './schema-migrations/helpers.ts';import { MIGRATION_PRELUDES, runMigrationPrelude } from './migration-preludes.ts';import { clearMigrationTimeout, migrationStatementTimeoutMs, recordMigrationTimeout } from './migration-timeout.ts';@@ -385,6 +386,7 @@ async function runMigrationsLocked(engine: BrainEngine): Promise<{ applied: numbconst sorted = [...MIGRATIONS].sort((a, b) => a.version - b.version);const pending = sorted.filter(m => m.version > current);+ if (pending.length > 0) await assertOwnedTableShapes(async sql => (await engine.executeRaw(sql))[0]); // #9190: before any repair or migration// #2038: schema-drift self-heal. A migration renumbered during a master// merge (v102 timeline dedup, originally v99) can be recorded-as-applieddiff --git a/src/core/owned-table-shapes.ts b/src/core/owned-table-shapes.tsnew file mode 100644index 0000000000..07140904d2--- /dev/null+++ b/src/core/owned-table-shapes.ts@@ -0,0 +1,71 @@+/**+ * #9190: gbrain creates its tables with CREATE TABLE IF NOT EXISTS, so a table of the same name that another+ * application created in the brain's schema was adopted silently; later migrations then added columns to it and an+ * upgrade stopped midway on the first statement that compared the wrong type, naming neither table nor column.+ * Before the schema blob replays (engine-sql/bootstrap.ts) and before a migration run (migrate.ts), every table here+ * that already exists must have the column types gbrain has given it since it first created the table (v32), its key+ * column included. Otherwise schema setup refuses with `foreign_table` and changes nothing: the table is another+ * application's data, so moving it is the user's decision.+ */+import { opError, type OperationError } from './ops/contract.ts';++/** Columns and types each table has had since v32; the first is its key and must exist. */+const OWNED_TABLE_COLUMNS: Record<string, ReadonlyArray<readonly [column: string, type: string]>> = {+ oauth_clients: [+ ['client_id', 'text'], ['client_secret_hash', 'text'], ['client_name', 'text'], ['redirect_uris', 'text[]'],+ ['grant_types', 'text[]'], ['scope', 'text'], ['token_endpoint_auth_method', 'text'], ['client_id_issued_at', 'bigint'],+ ['client_secret_expires_at', 'bigint'], ['created_at', 'timestamp with time zone'],+ ],+ oauth_tokens: [+ ['token_hash', 'text'], ['token_type', 'text'], ['client_id', 'text'], ['scopes', 'text[]'], ['expires_at', 'bigint'],+ ['resource', 'text'], ['created_at', 'timestamp with time zone'],+ ],+ oauth_codes: [+ ['code_hash', 'text'], ['client_id', 'text'], ['scopes', 'text[]'], ['code_challenge', 'text'],+ ['code_challenge_method', 'text'], ['redirect_uri', 'text'], ['state', 'text'], ['resource', 'text'],+ ['expires_at', 'bigint'], ['created_at', 'timestamp with time zone'],+ ],+};++const EXPECTED_ROWS = Object.entries(OWNED_TABLE_COLUMNS)+ .flatMap(([table, columns]) => columns.map(([column, type], i) => [table, column, type, i === 0] as const))+ .map(([table, column, type, isKey], ord) => `('${table}', '${column}', '${type}', ${isKey}, ${ord})`);++/** One row naming the first mismatch in the current schema, or none. */+const SHAPE_PROBE_SQL = `WITH expected(table_name, column_name, expected_type, is_key, ord) AS (VALUES ${EXPECTED_ROWS.join(', ')})+SELECT n.nspname AS schema_name, e.table_name, e.column_name, e.expected_type,+ format_type(a.atttypid, a.atttypmod) AS found_type+FROM expected e+JOIN pg_class c ON c.relname = e.table_name+JOIN pg_namespace n ON n.oid = c.relnamespace AND n.nspname = current_schema()+LEFT JOIN pg_attribute a ON a.attrelid = c.oid AND a.attname = e.column_name AND a.attnum > 0 AND NOT a.attisdropped+WHERE (a.attname IS NULL AND e.is_key) OR (a.attname IS NOT NULL AND format_type(a.atttypid, a.atttypmod) <> e.expected_type)+ORDER BY e.ord+LIMIT 1`;++interface ShapeMismatch { schema_name: string; table_name: string; column_name: string; expected_type: string; found_type: string | null }++function foreignTableError(m: ShapeMismatch): OperationError {+ const qualified = `${m.schema_name}.${m.table_name}`;+ const command = 'gbrain apply-migrations --force-schema --yes';+ return opError('foreign_table',+ `Schema setup stopped before changing anything: ${qualified}.${m.column_name} is ${m.found_type ?? 'missing'}, but gbrain's ${m.table_name} needs ${m.expected_type}. ` ++ `A table named ${m.table_name} that gbrain did not create (probably another application sharing this database) is in the way. ` ++ `Give gbrain its own database, or move that table to a schema of its own with ALTER TABLE ${qualified} SET SCHEMA <other_schema> ` ++ `(its indexes and constraints move with it) and point that application there; then run ${command}.`,+ 'Ask the user before changing that table: it holds another application\'s data, and moving it affects that application.',+ { detail: qualified, fix: {+ ask_user: true, argv: ['gbrain', 'apply-migrations', '--force-schema', '--yes'], consent: [], actor: 'user', requires_exclusive: false,+ why: `Once ${qualified} has moved, this creates gbrain's own ${m.table_name} and applies the pending migrations.`,+ user_message: `Your database already has a table named ${qualified} that gbrain did not create; it is probably another application's. ` ++ `gbrain stopped before changing anything. Either give gbrain its own database, or move that table with ALTER TABLE ${qualified} SET SCHEMA <other_schema> ` ++ `and point its application at the new schema. Then run ${command}.`,+ verify: { argv: ['gbrain', 'doctor', '--only', 'schema_version', '--json'] },+ } });+}++/** Throws `foreign_table` when a table gbrain owns by name exists with a shape gbrain never gave it. */+export async function assertOwnedTableShapes(queryOne: (sql: string) => Promise<Record<string, unknown> | undefined>): Promise<void> {+ const mismatch = await queryOne(SHAPE_PROBE_SQL);+ if (mismatch?.table_name) throw foreignTableError(mismatch as unknown as ShapeMismatch);+}diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex bc5bd2993b..12ec936aae 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -670,6 +670,7 @@ test/trust-owner-actions.test.tstest/unbound-source-6536.test.tstest/unindexed-files-convergence-7149.test.tstest/unindexed-files.test.ts+test/upgrade-foreign-oauth-table.test.tstest/url-source-origin.test.tstest/v273-validate-timeouts.test.tstest/valid-time-gbra217.test.tsdiff --git a/test/upgrade-foreign-oauth-table.test.ts b/test/upgrade-foreign-oauth-table.test.tsnew file mode 100644index 0000000000..1e59f76180--- /dev/null+++ b/test/upgrade-foreign-oauth-table.test.ts@@ -0,0 +1,169 @@+/**+ * #9190 and #9191: gbrain creates its OAuth tables with CREATE TABLE IF NOT EXISTS, so a same-named table another+ * application created in the brain's schema used to be adopted silently, altered by later migrations, and then break+ * v210 (clamp_oauth_token_ttl) with "operator does not exist: timestamp with time zone > bigint", midway through an+ * upgrade and without naming the table. Rerunning `gbrain apply-migrations --force-schema` then ran the pending+ * migrations without replaying the schema blob, so a blob-owned table that was missing stayed missing.+ * Both engines: PGLite always, Postgres with DATABASE_URL (test/postgres-unit-arms.txt).+ */+import { afterAll, beforeAll, describe, expect, test } from 'bun:test';+import postgres from '#postgres';+import { cpSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';+import { randomUUID } from 'node:crypto';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { PostgresEngine } from '../src/core/postgres-engine.ts';+import { LATEST_VERSION, runMigrations } from '../src/core/migrate.ts';+import { assertSafeE2eDatabaseUrl } from './helpers/db-guard.ts';+import { collect, makeHome, spawnDriver } from './helpers/apply-migrations-lock-driver.ts';+import { writeSchemaDriver } from './helpers/apply-migrations-schema-driver.ts';+import { testBackends } from './helpers/test-backends.ts';++const FOREIGN_TOKENS = `CREATE TABLE oauth_tokens (+ id uuid PRIMARY KEY, token_hash text, token_type text, client_id text, scopes text[],+ expires_at timestamptz, resource text, refresh_token_hash text, revoked_at timestamptz,+ created_at timestamptz NOT NULL DEFAULT now())`;+const FOREIGN_ROW = `INSERT INTO oauth_tokens (id, token_hash, token_type, client_id, expires_at)+ VALUES ('00000000-0000-4000-8000-000000000001', 'hash-example', 'Bearer', 'other-app-example', now() + interval '1 hour')`;+const FORCE_SCHEMA = ['--force-schema', '--yes', '--non-interactive', '--no-autopilot-install'];++interface Brain { config: Record<string, string>; open: () => Promise<BrainEngine> }++for (const kind of testBackends()) {+ describe(`a table gbrain did not create in the brain's schema (${kind})`, () => {+ const work = mkdtempSync(join(tmpdir(), 'gbrain-foreign-oauth-'));+ const homes: string[] = [];+ const engines: BrainEngine[] = [];+ const databases: string[] = [];+ let admin: ReturnType<typeof postgres> | undefined;+ let template = '';++ const databaseUrl = (name: string) => { const url = new URL(process.env.DATABASE_URL!); url.pathname = `/${name}`; return url.toString(); };+ beforeAll(async () => {+ if (kind === 'pglite') {+ template = join(work, 'template.pglite');+ const engine = new PGLiteEngine();+ await engine.connect({ database_path: template });+ await engine.initSchema();+ await engine.disconnect();+ return;+ }+ assertSafeE2eDatabaseUrl(process.env.DATABASE_URL!);+ admin = postgres(process.env.DATABASE_URL!, { max: 1, prepare: false });+ template = `gbrain_test_foreign_oauth_tpl_${randomUUID().replace(/-/g, '')}`;+ await admin.unsafe(`CREATE DATABASE ${template}`);+ databases.push(template);+ const engine = new PostgresEngine();+ await engine.connect({ database_url: databaseUrl(template), poolSize: 4 });+ await engine.initSchema();+ await engine.disconnect();+ }, 300_000);++ afterAll(async () => {+ for (const engine of [...engines]) await engine.disconnect().catch(() => {});+ for (const name of databases.reverse()) await admin?.unsafe(`DROP DATABASE IF EXISTS ${name} WITH (FORCE)`).catch(() => {});+ await admin?.end();+ for (const home of homes) rmSync(home, { recursive: true, force: true });+ rmSync(work, { recursive: true, force: true });+ });++ const brain = async (atHead: boolean): Promise<Brain> => {+ if (kind === 'pglite') {+ const database_path = join(work, randomUUID());+ if (atHead) cpSync(template, database_path, { recursive: true });+ return { config: { engine: 'pglite', database_path }, open: async () => {+ const engine = new PGLiteEngine(); await engine.connect({ database_path }); engines.push(engine); return engine;+ } };+ }+ const name = `gbrain_test_foreign_oauth_${randomUUID().replace(/-/g, '')}`;+ await admin!.unsafe(atHead ? `CREATE DATABASE ${name} TEMPLATE ${template}` : `CREATE DATABASE ${name}`);+ databases.push(name);+ return { config: { engine: 'postgres', database_url: databaseUrl(name) }, open: async () => {+ const engine = new PostgresEngine(); await engine.connect({ database_url: databaseUrl(name), poolSize: 4 }); engines.push(engine); return engine;+ } };+ };+ const close = async (engine: BrainEngine) => { await engine.disconnect(); engines.splice(engines.indexOf(engine), 1); };+ const forceSchema = async (b: Brain) => {+ const home = makeHome({ engine: 'pglite' });+ homes.push(home);+ writeFileSync(join(home, '.gbrain', 'config.json'), JSON.stringify(b.config));+ const r = await collect(spawnDriver(home, writeSchemaDriver(home, { migrations: ['noop'], args: FORCE_SCHEMA })));+ return { code: r.code, out: r.stderr + r.stdout };+ };+ const shapeOf = async (engine: BrainEngine, qualified: string) => (await engine.executeRaw<{ t: string | null }>(+ `SELECT format_type(a.atttypid, a.atttypmod) AS t FROM pg_attribute a+ WHERE a.attrelid = to_regclass($1) AND a.attname = 'expires_at' AND NOT a.attisdropped`, [qualified]))[0]?.t ?? null;++ test('#9190: an upgrade refuses before any migration, naming the table, the column, both types and the fix', async () => {+ const b = await brain(true);+ const engine = await b.open();+ await engine.executeRaw('DROP TABLE oauth_tokens');+ await engine.executeRaw(FOREIGN_TOKENS);+ await engine.executeRaw(FOREIGN_ROW);+ await engine.setConfig('version', '205');+ const error = await runMigrations(engine).then(() => null, (e: unknown) => e as Error);+ expect(error?.message).toContain('oauth_tokens.expires_at is timestamp with time zone');+ expect(error?.message).toContain('needs bigint');+ expect(error?.message).toContain('SET SCHEMA');+ expect(error?.message).toContain('gbrain apply-migrations --force-schema --yes');+ expect(await engine.getConfig('version')).toBe('205');+ expect(await engine.executeRaw('SELECT token_type FROM oauth_tokens')).toEqual([{ token_type: 'Bearer' }]);+ }, 300_000);++ test('#9190: a fresh install refuses a foreign oauth_codes or oauth_clients before creating any gbrain table', async () => {+ for (const [ddl, named] of [+ ['CREATE TABLE oauth_codes (code_hash text PRIMARY KEY, client_id text, expires_at timestamptz)', 'oauth_codes.expires_at is timestamp with time zone'],+ ['CREATE TABLE oauth_clients (client_id uuid PRIMARY KEY, client_name text)', 'oauth_clients.client_id is uuid'],+ ] as const) {+ const b = await brain(false);+ const engine = await b.open();+ await engine.executeRaw(ddl);+ const error = await engine.initSchema().then(() => null, (e: unknown) => e as Error);+ expect(error?.message).toContain(named);+ expect(await engine.executeRaw<{ t: string | null }>(`SELECT to_regclass('pages')::text AS t`)).toEqual([{ t: null }]);+ await close(engine);+ }+ }, 300_000);++ test('#9191: rerunning apply-migrations --force-schema recreates a blob-owned table that is missing', async () => {+ const b = await brain(true);+ const engine = await b.open();+ await engine.executeRaw('DROP TABLE oauth_tokens');+ await engine.setConfig('version', '209');+ await close(engine);+ const run = await forceSchema(b);+ expect(run.code, run.out).toBe(0);+ const after = await b.open();+ expect(await shapeOf(after, 'oauth_tokens')).toBe('bigint');+ expect(await after.getConfig('version')).toBe(String(LATEST_VERSION));+ }, 300_000);++ test('a brain stopped at v210 by a foreign table recovers with the steps the refusal names', async () => {+ const b = await brain(true);+ const engine = await b.open();+ await engine.executeRaw('DROP TABLE oauth_tokens');+ await engine.executeRaw(FOREIGN_TOKENS);+ await engine.executeRaw(FOREIGN_ROW);+ await engine.setConfig('version', '209');+ await close(engine);+ const refused = await forceSchema(b);+ expect(refused.code, refused.out).toBe(1);+ expect(refused.out).toContain('[foreign_table]');+ expect(refused.out).toContain('oauth_tokens.expires_at is timestamp with time zone');++ const moving = await b.open();+ await moving.executeRaw('CREATE SCHEMA other_app_example');+ await moving.executeRaw('ALTER TABLE oauth_tokens SET SCHEMA other_app_example');+ await close(moving);+ const rerun = await forceSchema(b);+ expect(rerun.code, rerun.out).toBe(0);+ const after = await b.open();+ expect(await shapeOf(after, 'oauth_tokens')).toBe('bigint');+ expect(await shapeOf(after, 'other_app_example.oauth_tokens')).toBe('timestamp with time zone');+ expect(await after.executeRaw('SELECT token_type FROM other_app_example.oauth_tokens')).toEqual([{ token_type: 'Bearer' }]);+ expect(await after.getConfig('version')).toBe(String(LATEST_VERSION));+ }, 300_000);+ });+}