Garry's Opinionated OpenClaw/Hermes Agent Brain
Key files: Engines (2/2)
Subsystem index. Read only the entries relevant to your change. Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.
patches/[email protected]— auditable delta for the driver ownership and cancellation repair already applied to the shippedvendor/postgresESM/CommonJS/workerd source (not an install hook;bash vendor/update-postgres.sh --checkverifies it), consumed byPostgresEngine.runUnsafethrough#postgres. Abortable reservations settle atomically; shared transaction queues isolate cancellable SQL from tagged/savepoint siblings while preserving unfenced pipelining. A pooled connection (poolmaxabove 1) or a released reservation whose last ReadyForQuery status is not idle is terminated instead of reused, and theonpoisoned(status)option reports it (pinned bytest/e2e/postgres-poisoned-connection.test.ts). See PostgreSQL cancellation ownership for settlement, deadline, transport and clean-install invariants. Pinned bytest/e2e/persistence-chaos.test.ts. The patch also shares each pool's described parameter types across its connections (shared_types;db.resolveSharedTypes()reads theGBRAIN_PG_TYPE_CACHE=0kill switch at the four pool construction sites,db.clearSharedTypes()runs afterinitSchemaapplies migrations), so a first run on another connection, and every run underprepare: false, skips the describe round trip; pinned bytest/e2e/postgres-shared-types.test.ts. #6355:closed()settles a startup's internal query (the array-types fetch) and clears its saved error before reconnecting forinitial, so a backend terminated during startup neither hands the caller's first statement a stale57P01on the fresh connection nor leaves an unhandled rejection (fetchArrayTypesalso swallows its own failure); it clears the pending write buffer and its immediate handle, so a close that catches a buffered statement does not leave the next StartupMessage appended and unscheduled (a pool slot stuck atCONNECT_TIMEOUT); pinned bytest/postgres-driver-startup-death.test.tsagainst a scripted wire-protocol server. A query cancelled after it was written stays current until its own ReadyForQuery, so its answer is never read for the next query, and rejects57014; no server cancel is sent for it while statements are pipelined behind it, because a CancelRequest stops whatever the backend is running (pinned bytest/e2e/postgres-driver-cancel-pipeline.test.ts, direct and PgBouncer). A statement answer that arrives with no current query retires the connection withCONNECTION_DESYNCand callsondesync({ message, connection, statement });PoolIncidentCounterlogspg_connection_desync(pinned bytest/e2e/postgres-driver-desync-tripwire.test.ts). #6930:array_typeshands every pool for one database target the array types the process's first connection fetched, so later connections skip the pg_type round trip;transaction_pipelinepipelines on a connection its owner holds (sql.begin,sql.reserve) while every statement in flight leaves it inside a transaction and never behind COMMIT/ROLLBACK/END/ABORT/PREPARE TRANSACTION, so a transaction-mode pooler URL (max_pipeline0,db.transactionPipelineOptions,GBRAIN_PG_TXN_PIPELINE=0off) keeps #6543's boundary; pinned bytest/pathological/pg-round-trips-6930.test.tsandtest/e2e/pgbouncer-transaction-pipeline-6930.test.ts. GST-2260: the firstRevalidateCachedQueryerror (0A000: a preparedSELECT */RETURNING *whose table changed) bumps a process-wide plan generation, so every connection drops its prepared statements before its next query: one failure per process per migration, not per pooled connection (test/stale-plan-heal.test.ts).src/core/pg-type-cache.ts— the storeshared_typesuses: oneSharedParameterTypesMap per database target in this process (sharedParameterTypes(url)), keyed by SHA-256 digests of the driver's (types, SQL) key, and saved under<gbrain home>/cache/pg-types/(0600) for the next process.db.checkPoolAndLoadSharedTypesreads its scope (server_version_num,config.version, and the database as the server names it, so a proxy and a direct URL share a file) in each new pool's connection check and loads it; a different scope is ignored and replaced, a migration'sclearSharedTypesdeletes the file.GBRAIN_PG_TYPE_CACHE_PERSIST=0keeps it in-process. Pinned bytest/pg-type-cache.test.tsand the new-process case intest/e2e/postgres-shared-types.test.ts. #6930:sharedArrayTypes(url)is the per-target, in-process-onlyarray_typesstore (extension OIDs belong to one catalog);GBRAIN_PG_TYPE_CACHE=0turns it off andclearSharedTypesempties it.src/core/engine-sql/— one SQL implementation per migrated storage domain (the domains are themigratedrows ofscripts/engine-sql-baseline.tsv), run by both engines through one executor contract:PGLiteEngine / PostgresEngine one-line delegations; RLS scoping per method as on master | get engineSql() fresh adapter per access, never stored (transaction clones) v engine-sql/<domain>.ts Postgres master text; sqlFragment composition; capabilities | ScopedRead / LegacyUnscopedRead brands (brands.ts), row kinds (normalize.ts) v engine-sql/executor.ts SqlExecutor: query/run, executeRaw, unsafe, transaction; {rows, affectedRows} | +--> dialect-pglite.ts db.query on the engine's cached, checkpoint-guarded handle +--> dialect-postgres.ts runUnsafe(conn, sql, params, {signal?, prepare: true, simple: false}); no gaugeOutside
engine.transaction(),engineSqlhands engine-sqlguardedHandle(src/core/pglite-engine/checkpoint-guard.ts): WAL-writing statements takerunStatement, executor transactionsrunOutermost, likeexecuteRawandtransaction(). Before that, engine-sql autocommit writes such asmarkPagesExtractedBatchnever probed WAL; past the automatic trigger (about 528 MB with max_wal_size 1 GB) the crossing UPDATE ran the checkpoint inline and spun forever (GBRA-69). Pinned bytest/pglite-checkpoint-guard.test.tsandtest/fixtures/pglite-engine-sql-wal-worker.ts.Inputs engine-sql never resolves arrive already resolved: source scope (
sourceIds>sourceId>'default'), the registry-active embedding column (bare name, quoted withquoteIdentifier), page-state guards as callbacks (lockPageKeys/readPageSnapshot) and the engine'stransaction()where master used it. A read master ran insidewithScopedReadTransactionkeeps that literal call in the engine method (the RLS AST inventory counts direct sites) and brands the tx handle's executor:scopedRead(this.engineSqlOn(tx))on Postgres,scopedRead(this.engineSql)on PGLite.fragment.tsnumbers placeholders at render time, so text is byte-identical to the tagged templates it replaced (test/fixtures/goldens/sql-text/). Capabilities (maxBindParamsPerStatement,transactionAdvisoryLocks,probesEmbeddingCast) carry the real engine differences;docs/designs/refactor-wave-1/w1-inventory.mdclassifies every method. Page-state orchestration stays in the engines:putPage's transaction, page-key lock and revision check, and theupdateSlug/setPageAliasestransactions, callpages.tson the transaction clone's executor;getPage/readPageSnapshotkeep their SQL inpage-state/snapshot.ts. Reads with an early return that must not open a scoped transaction (listPrefixSampledPages,listCorpusSample) take aScopedReadRunner. Nothing here imports an engine façade (check:layering); SQL may splice only constant text (check:engine-sql-dynamic); brands are unforgeable (check:engine-sql-brands); engine SQL only shrinks (check:engine-sql-ratchet). Tests: docs/TESTING.md "Engine-sql".src/core/engine-sql/facts.tslistFactsKeyset+src/core/engine-sql/pages.tskeyset predicates — delta's two keyset reads. Both bind the cursor timestamp as${value}::text::timestamptz: a bare::timestamptzparam is typed by the postgres.js driver, which serializes a string through a JS Date and truncates it to milliseconds (on Postgres that re-selected the whole last millisecond every wake and never drained a >limit cluster inside one millisecond).listFactsKeysetreads ascending(created_at, id)strictly after the keyset (id: null= strictly after the timestamp) and projectscreated_at_isoat column precision; recall's newest-firstlistFactsSinceis unchanged.test/engine-sql-facts-keyset.test.ts,test/delta-cursor-integrity.test.ts(both engines).src/core/engine-sql/keyword-pages.ts+src/core/search/keyword-statement.ts— the strict keyword count (BrainEngine.countKeywordPages) and the keyword-mode page query (BrainEngine.searchKeywordPages) for both engines.buildKeywordMatchis the ONE WHERE builder both statements read:searchKeyword's strict match (websearch FTS oncontent_chunks.search_vector, or the CJK LIKE/ILIKE terms), never its OR retry, with the same page filters, hard excludes andbuildVisibilityClause(private pages, safe chunks, so no private fence text). A query with slug handles (search/keyword-handles.ts) adds a second arm under the same filters, the chunks of the pages whose slug basename equals a handle (idx_pages_slug_basename), scored 1 + rank so they lead; the count is aUNIONof both arms andbuildKeywordHandlePagesStatementreads the handle arm alone forsearchKeyword. The match returns its WHERE params first and the score-only rank params after them (whereParamCount), so the count binds only what its WHERE reads (a CJK count bound the term-frequency params and failed on both engines;test/sql-bind-placeholders.test.ts). The count iscount(*)overSELECT DISTINCT page_id … LIMIT 10001; a non-timeout count failure logs its SQLSTATE once per process (keyword-paging.ts); the page query takes each page's best chunk before paging (no global chunk cap), orders by(score DESC, page_id ASC)with float8 scores so the keyset cursor round-trips, and pages those (page_id, chunk_id, score) rows before it reads the page and chunk rows it returns (#7082;test/search/keyword-pages-limit-first.test.ts, Postgres arm intest/e2e/keyword-pages-limit-first-postgres.test.ts). Postgres runs both insidewithScopedReadTransactionwith a statement timeout (the count's set from the caller's deadline); PGLite has none.src/core/search/exact-scan.ts+src/core/ops/search-exact.ts+src/cli/search-exact-format.ts— #7191:search_exact(CLIgbrain search --exact, routed incli/main.tsbefore ranked-search flags parse): an unranked literal scan ofcontent_chunks.chunk_text(words in order, any whitespace between, case-insensitive unlesscase_sensitive; the ARE comes fromphrasePattern, with a LIKE prefilter on the longest word) in keyset batches of pages by id (buildExactBatchStatement), under ranked search's visibility (live page, unarchived source, quarantine, pinned questions,chunkVisibleFilter, source scope, private pages and the trust floor for untrusted callers,safeChunksFilterfor remote) and the hard-excluded prefixes.completeis true only when the scope was scanned to the end with nolimitordeadlinestop and no unreadable page (unchunked,projection_pending, remoteunsealed,scan_timeout); a Postgres statement timeout halves the batch, down to one page.resume_aftercontinues a bounded scan. Registeredfiltered(min_trust,stampPageTrust), output redactionretrieval. Guide: exact search. Tests:test/search-exact.test.ts,test/e2e/search-exact-postgres.test.ts.src/core/engine-sql/titles.ts+src/core/search/title-statement.ts— the page-grain title arm (BrainEngine.searchTitles) for both engines:buildSearchTitlesStatementinsearch/(SQL composed outside engine-sql likecjk-keyword-sql.ts; also used by the plan-proof E2Etest/e2e/title-arm-postgres.test.ts) andsearchTitles(scoped, query, opts, dialect)in engine-sql, wheredialectcarries the engine differences kept from master (Postgres: 8s per-attemptstatement_timeout,enable_seqscanoff for the relaxed local re-run; PGLite: the timelinestaleprobe). Both orderings sort an exact-title key first (lower(btrim(title, ' "''' || chr(160))) = normalizeAlias(query), bound separately from the websearch text so the OR fallback never rebinds it; #5889). The websearch text is read throughgbrain_fts_input($1)(#6370), the same CJK/ASCII boundary split the index applies, so a glued title query matches. Safe-chunks callers and the title-only OR re-run matchts_filter(search_vector, '{a}') @@ q(weight A is the title alone) and rank on the same slice: served by the expression indexidx_pages_title_search(GBRA-156, v267), which holds title lexemes only, so a term in timeline text never makes its page a candidate; title-only for negated and phrase terms, following the storedsearch_vectorlanguage. Postgres runs each attempt with the keyword statement'swork_mem(statementSettings). Pages with a NULLsearch_vectorare absent from the remote arm. #7471: a 2-4 word name query ORs in its given-name variants (titleNameVariants: "Matt Example" also matches the title "Matthew Example"), each its own bound websearch query; the OR fallback of a query of at mostRARITY_MAX_TERMSwords runs once per word in one scoped read andrankByRarityputs titles holding the rarest word first (ts_rank_cd knows no term rarity, so a surname's one title used to tie thousands of first-name titles and lose on page id). #7491: OR-retry titles holding every identifier the query names (holdsIdentifiers, search/keyword-identifier-probe.ts) lead untagged; the rest staykeyword_relaxed. #7082: a local caller's strict attempt matches the whole vector but ranks at mosttitleWholeVectorCandidates()(5,000) matching pages, the lowest page ids (thecandidatesCTE, ordered by id); a cut attempt reportskeyword_candidates_incomplete(reasoncandidate_budget) and puts the title-only statement's rows first. GST-2899: a local caller's OR re-run first runs the whole-vector statement withcompleteOnly(the same cap, an unordered count, rows only when every matching page fits, else one row markedcandidates_cut), so a reworded question still reaches pages whose timeline names its subject; when the set does not fit (a date, a common word) it runs the title-only statement as before. Plan proof:test/e2e/title-any-word-retry-postgres.test.ts.src/core/engine-sql/page-lookup.ts+src/core/page-find.ts+src/core/page-counts.ts+src/core/ops/find-pages.ts+src/core/ops/stats-by.ts+src/cli/format-page-lookup.ts— find-by-name and grouped counts (gbrain find/find_pages,gbrain stats --by/get_statsby,since,collapse_aliases).parseFindPatternturns a glob (*/?, whole slug or source path), a slug prefix (contains/) or a name (title or last slug segment substring) into oneescapeLikePattern-escaped ILIKE pattern; names are served byidx_pages_trgm+idx_pages_slug_trgm, while globs and prefixes also matchsource_path(no trigram index). Rows sort by slug then source in the C collation, carrycount(*) OVER ()astotal, and the op reportscomplete(limit default 50, cap 500 local, 100 remote). Find hides the ranked-search hard excludes (resolveHardExcludes+buildHardExcludeClause) unless the pattern's literal lead is inside one, and names them inhidden_prefixes.countPagesByreturns per (source, type) counts plus raw and distinct-slug totals (per type group through anunnestmapping and GROUPING SETS), so cross-source twins are never silently double counted;sinceis the query effective-date bound (resolveDateBoundary). Every caller sees live pages in unarchived sources; a remote caller'sreadPolicyOptspolicy addspageReadFilter(..., live).collapseTypeslets the active pack decide (declared stays, alias joins canonical), else normalizes (last segment, lower case, singular) and joins a-<name>suffix to an existing group; each group listsmember_typeswith its rule (exact,pack_alias,normalized,suffix). Without the grouping paramsget_statsreturns its unchanged counters. Tests:test/page-find-pattern-7191.test.ts,test/page-lookup-7191.test.ts(Postgres arm:test/e2e/page-lookup-7191.test.ts).src/core/engine-sql/health.ts—BrainEngine.getHealthonce for both engines (migrated health). Three statements, no per-page rows returned: the counters (pages, chunks, links, embedding coverage, missing embeddings, dead links, entity link/timeline coverage), the linkable scope grouped by page type with islanded and with-timeline counts (the orphan policy rendered in SQL byorphanExclusionSql; the brain_score timeline grading applies per type), andmost_connectedas live, in-scope link endpoints grouped per entity page (ties break by slug in code-point order; a link whose far endpoint is soft-deleted does not count, #6940). The linkable groups are a separate statement because folding them into the counters crosses Postgres's JIT cost thresholds at about 20k pages. Per-page and per-link checks that misplan without planner statistics (PGLite before its first ANALYZE) are primary-key lookups the planner cannot reorder (scalar subqueries, a LATERAL fenced withOFFSET 0), so the statements stay linear with or without statistics. Scope rules:sourceIds>sourceId> brain-wide, duplicate ids count once, a link counts only when both endpoints are in scope, endpoint liveness in both directions, quarantined pages out of the entity denominators and the linkable scope. Entity pages (#4772) are the types the active schema pack(s) declare as entities, resolved per source byHealthDeps.entityTypes(src/core/schema-pack/entity-types.ts) and bound astext[]parameters (entityTypePredicateSql: one list, or aCASE p.source_idwhen sources' packs disagree); a pack that does not load binds an empty list and the result carriesentity_types_status: 'pack_unavailable', never the old hardcoded list. Reads takeLegacyUnscopedRead(pool, no scope transaction). Engine deps are four functions (embeddingColumn,countStalePagesForExtraction,getConfig,entityTypes). Pinned field by field against the pre-F4a implementation (test/helpers/legacy-get-health.ts, whosemost_connectedwas corrected for #6940 as an intended change) bytest/engine-sql-health-equality.test.tsandtest/e2e/engine-sql-health-parity.test.ts(direct Postgres and PgBouncer); the pack-driven entity predicate bytest/health-entity-types-pack.test.tsandtest/e2e/health-entity-types-pack.test.ts;most_connectedliveness bytest/health-most-connected-liveness-6940.test.ts(+ Postgres twin).src/core/engine-sql/graph-rank.ts— #7168:rankEntitiesStatement(opts), the onerank_entitiesaggregate for both engines (a statement builder: the op runs it throughexecuteRawwith its statement deadline, likeextractionLagStatement). Edges are distinct(from, to, link_type)triples between live, in-scope endpoints (both endpoints scoped bysourceIds), self-links, non-ranking types (rankingLinkSql) and, unlessincludeMentions, mention rows excluded. Retired pages (retiredPageSql) fold into theirsuperseded_by/merged_intotarget in the same source through a recursive chain over retired pages only, capped atMAX_RETIREMENT_HOPS; edges dedupe after folding and folded self-links drop. Withremoteevery endpoint, chain hop, fold target and non-null link origin must be in scope, live, unquarantined, in an unarchived source and (withexcludePrivate) not private, so a hidden target never folds. Ordered by the direction's count, then slug and source id (COLLATE "C"), LIMIT after folding. Tests:test/rank-entities.test.ts.src/core/engine-sql/exact-identity.ts— GBRA-156:exactIdentityStatement(opts), thewhoiscandidate read (a statement builder the op runs throughexecuteRaw): a UNION of equality probes each served by an existing index,lower(title)(idx_pages_source_title_lower; the raw query is also lowered by the database), the slug basename (idx_pages_slug_basename, accent-folded by the slug grammar),page_aliases.alias_norm, and one GIN containment arm per address and shape (email: x,email: [x],emails: x,emails: [x]; one OR of them plans a scan). Every probe and row stays insourceIds; rows are live, in an unarchived source, of a type theEntityTypeFilteradmits, and withremoteunquarantined and (withexcludePrivate) not private. The rows are candidates; the op verifies each. Accent-insensitive titles are found only through the lowered title (same accents, or none in the title) or the slug basename; a case-insensitive frontmatter email only through its as-written or lowercase value or a declared alias. Closing those gaps needs expression indexes (a migration). Tests: the index-plan case intest/whois.test.tsandtest/e2e/whois-postgres.test.ts.src/core/engine-sql/chunks.tsupsertChunksOnce— a chunk replacement inside the caller's transaction: the batchedchunkWriteInvalidationsguard, then one prune (moveDisplacedChunksAndPrune: a row whose vector still matches its text (movableChunkSql) and whose exact text and source reappear at another index is parked at a negative index and moved there; rows outside the new set are deleted), then one multi-row INSERT perjsonRowBatchesbatch (at mostROW_BATCH_MAX_ROWSrows andROW_BATCH_MAX_BYTESencoded bytes; the first is pipelined with the seal). A failure in any batch rolls back the prune and every earlier batch. Tests:engine-sql-chunks-batched-write.test.ts, the multi-batch case inhelpers/engine-sql-rollback-cases.ts. ItssealChunksisBrainEngine.sealChunks, run inside the engine's page transaction: lock and read the live page, refuse a missing, deleted or segmented page, an archived source and a staleexpectedRevision, then delete the chunks, insert the new ones with the seal chunker version, set the search vector andtext_projection_revision, and drop the page's queued projection job. Both engines delegate to it; the engine parity suite seals its fixture chunks through it.src/core/pglite-resetwal.ts— pg_resetwal for PGLite NodeFS data dirs, in TypeScript (a port of the electric-sql/pglite pg_resetwal contribution, PR #994, Apache-2.0). Validates the PG17 pg_control layout fail-closed (WalResetUnsupportedErroron any unsupported shape — PG_VERSION ≠ 17, control ≠ 8192 bytes, control version ≠ 1700, bad seg/block size), removes stale postmaster.pid + old WAL segments + archive_status/summaries entries, writes a replacement shutdown-checkpoint WAL segment + CRC32C'd pg_control. Both file writes are atomic + durable (tmp cleared then opened'wx'so a pre-planted symlink at the predictable tmp name can never redirect the write, + fsync(tmp) + rename + fsync(parent dir)); write order is segment-first/control-last so a mid-write kill leaves a state that still fails startup and the next attempt re-runs (idempotent — a torn pair can never claim success). WAL segment size is capped at 64MB (pglite ships 16MB; the Postgres-general 1GB bound would let a corrupt-but-plausible control field drive a 1GB allocation on the repair path). Exports the shared PG17 layout literals (PG_CONTROL_FILE_SIZE,isWalSegmentName) consumed by pglite-repair.ts. LAYOUT COUPLING: any pglite bump past PG17 must revisit this file together with the./vectorexport blocker (TODOS.md "pglite upgrade blocker" entry). Pinned bytest/pglite-resetwal.test.ts.src/core/pooler-notice.ts— the transaction-pooler banner (db.connect()with prepared statements off), printed once per database target per gbrain home: a marker under<gbrain home>/notices/pooler-prepare-off/named by a digest of the credential-free target (typeCacheTarget) and the banner text, so a reworded banner or another database prints again; an unwritable home prints every time.gbrain engine status(prepare=false) and doctor'spgbouncer_preparereport the setting on demand. Pinned bytest/pooler-banner-once.test.ts.src/core/pool-budget.ts+src/core/persistence/pool-capacity.ts— physical-pool shared long-hold ceilings retain one ordinary/control connection across engine clones; pool size one pauses canonical consumption withwriter_pool_capacity.withReservedConnection(fn, { selfContained: true })(migration DDL,buildIndexOnline, backfill batches:fntouches only its reserved connection, like a transaction) may take the only connection of a one-connection ordinary pool, never the only direct session; pinned bytest/pool-budget.test.tsandtest/e2e/pool-size-one-migrations.test.ts. Publication concurrency is capped at two per actual Postgres pool and one per PGLite owner.persistence/takes-prepare.tsjournals semantic takes with frozen dates/resolver identity and original/current holder ceilings; canonical row updates and receipt commit together. Publication retains the actual affected holders privately in authority JSON, so narrowed holder grants also hide committed/replayed receipts after input compaction.src/core/pool-gauge.ts—CheckoutGauge: approximate in-flight counters at the engine's raw/direct/reserved/tx seams, surfaced via duck-typedPostgresEngine.getPoolDiagnostics()(no BrainEngine change, no PGLite stub). HONESTY CONTRACT in the module doc: tagged-template traffic is untracked; consumers must label counts as a subset and never derive waiter/available figures. Fail-open (clamped release, try/finally around sync-throwing runUnsafe). Consumed by db-probe.ts. Pinned bytest/pool-gauge.test.ts.PoisonedDiscardCountercounts the driver's non-idle connection discards and writes one fixed-shapepg_connection_poisonedwarn line each; the engine wires it into every pool it creates and reports it aspoisonedDiscards.onCheckout(listener)(exposed asPostgresEngine.onCheckout) fires after a connection is actually obtained: signalled raw reserve resolution, transaction-callback entry andwithReservedConnectionreserve resolution; it never fires for a call still waiting (pinned bytest/e2e/postgres-checkout-observer.test.ts).formatBuildFailureWarningappendsparam=$n(orparam=?for an array element) to anUNDEFINED_VALUEline from the driver'sonbuilderror(code, statement, parameter)(#6543).src/core/postgres-engine.ts— Postgres + pgvector implementation (Supabase / self-hosted). The package-private#postgresimport maps directly to the shipped cancellation-capablevendor/postgressource (the same source is embedded by compiled builds);vendor/README.mdand the hash-verifying updater own provenance and regeneration.postgres-engine/cancellation.tsshares actual reserved-owner capability detection, typedpostgres_cancellation_unavailableerrors, and abort-bounded reservation with late-owner release; unsignalled diagnostic queries remain usable. Signalled SQL must retain cancellation/discard safety, including through transaction-mode poolers. The migrated storage domains' methods are one-line delegations intosrc/core/engine-sql/<domain>.tsover the privateengineSqlgetter (a fresh Postgres adapter overthis.sqlon every access; RLS-scoped reads brand thewithScopedReadTransactionhandle withscopedRead), Postgres-only code stays undersrc/core/postgres-engine/, and the façade keeps its full public surface.addLinksBatch/addTimelineEntriesBatch/addTakesBatchpass the batch as one JSONB document —INSERT ... SELECT FROM jsonb_to_recordset(($1::jsonb)->'rows') AS v(...) JOIN pages ...bound throughexecuteRawJsonb({ rows })— which encodes arbitrary free text safely (anunnest(${arr}::text[])array-literal path would crash Postgres with "malformed array literal" on free text such as calendar/Zoom context) and sidesteps the 65535-parameter cap; takes declares native recordset column types (page_id int, weight real, active boolean, …) so no per-element casts; all three arebatchRetry-wrapped.disconnect()runsdrainBackgroundWorkBeforeDisconnect()before pool teardown (engine parity with PGLiteEngine — mode'disconnect', so residual telemetry buffers drop symmetrically on both engines; guarded so a never-connected/already-torn-down engine skips the drain).searchKeyword/searchVectorscopestatement_timeoutviasql.begin+ a transaction-localset_configso the GUC dies with the transaction instead of leaking across the pooled postgres.js connection; the keyword-side bound (searchKeyword,countKeywordPages,searchKeywordPages,searchTitles,searchKeywordChunks,_searchKeywordCJK) issearch.keyword_statement_timeout_ms(#6495; default 8000, range 1000-60000), resolved once per connection bykeywordStatementTimeoutSetting()on the first keyword statement (postgres-engine/keyword-statement-timeout.ts: theEngineConfig.keyword_statement_timeout_msvalue the engine connected with, else the DB plane's row read by one parameterless pool statement so no pg-types cache entry is described andconnect()issues nothing for it, else the default; never a request parameter), withcountKeywordPageskeeping a caller's narrowertimeoutMs. Every search transaction (keyword, keyword chunks, CJK, title, the vector attempts and the vectorhasMorewitness) also passesjitOffand runs withjit = off(postgres-engine/search-settings.ts): the correlated visibility subplans push these short reads past the JIT cost thresholds, and LLVM compilation, not execution, dominated their latency on large brains.withScopedReadTransactionapplies every setting a search read needs (RLSapp.scopes,jit, the caller'ssettings:statement_timeout, the relaxed arms'enable_seqscan, the vector scan settings fromvectorScanSettings, a thunk read after BEGIN) in onewithSearchSettingsstatement,SELECT set_config('<name>', $n, true), …, so a top-level read is BEGIN, settings, query, COMMIT;statement_timeoutstays a literal name with a bound value so the vendored driver's #6543 budget reads it. Inside a caller's transaction the search runs in a savepoint, reads the caller's values in one statement and restores every one afterwards (a failed read rolls them back with the savepoint). A top-level read transaction opts in tobegin({ early: isPipelinableSearchRead })(GBRA-156): its settings statement and its reads (single SELECT or WITH statements naming no write, lock, sequence or session setting) go out behind BEGIN before BEGIN's answer on any pool that pipelines inside a transaction, one round trip instead of two. Pinned bytest/search-settings-one-statement.test.tsandtest/e2e/search-query-contract-postgres.test.ts.executeRaw(sql, params, { jitOff: true })(postgres-engine/jit-off-statement.ts) gives single statements the same setting only when it pays: the server'sjitis read once per pool, a JIT-off server runs the plain statement, and a JIT-on server runsBEGIN; SET LOCAL jit = off, the statement andCOMMITpipelined in therunBoundedStatementenvelope (postgres-engine/bounded-statement.ts, which keepstimeoutMs/deadlineAtand the abort signal); inside a caller's transaction it reads, overrides and restores the caller's value. Pinned bytest/e2e/typed-search-jit-postgres.test.ts.getEmbeddingsByChunkIds(insrc/core/engine-sql/chunks.ts, both engines) usestryParseEmbeddingso one corrupt row skips+warns instead of killing the query.searchKeyword/searchKeywordChunks/searchVectorapply source-aware ranking by inlining the source-factor CASE andNOT (col LIKE …)hard-exclude fromsrc/core/search/sql-ranking.ts;searchVectoris a two-stage CTE (HNSW-safe inner ORDER BY, source-boost re-rank in outer SELECT) carryingp.source_idinner→outer._savedConfigretains the connect config;reconnect()tears down + recreates the pool from saved config (called by supervisor watchdog after 3 consecutive health-check failures, and bybatchRetryon a retryable connection error). Concurrent callers share one in-flight_reconnectPromise(they await the single reconnect rather than racing a half-rebuilt pool); ownership re-samples through the atomicdb.connect()token on the connect leg.reconnect(ctx?)accepts the triggering error and records a pool-recovery audit event (reap_detected/reconnect_other/reconnect_succeeded/reconnect_failed) for thepool_reap_healthdoctor check.executeRawis a single-statement passthrough — no per-call retry (unsound for non-idempotent statements; recovery is supervisor-driven).connect()appliesresolveSessionTimeouts()fromdb.tsas connection-time startup parameters (statement_timeout,idle_in_transaction_session_timeout) so orphan pgbouncer backends can't hold locks for hours. The chunk SQL below lives once insrc/core/engine-sql/chunks.ts; the engine resolves the active column and source scope, owns the retry/transaction wrapper and the RLS scope transaction, and passes them in.countStaleChunks()+listStaleChunks()server-side-filter onembedding IS NULLforembed --stale(eliminates ~76 MB/call client-side pull);upsertChunks()routes text-embedding writes through the column registry (both engines in parity): a caller-resolvedopts.embeddingColumndescriptor wins; otherwise the DB-planesearch_embedding_column+embedding_columnsconfig rows resolve viaresolveWriteColumnFromConfigRowsto the SAME active column + cast the read side searches (legacyembedding::vector on pre-registry brains; so a registry-routed brain never writes with a dimension mismatch against the legacy column). The INSERT column list, the$N::vector|::halfvec(N)cast, and everyembeddingreference in the ON CONFLICT CASE branches use the resolved column; it resets both the active column ANDembedded_atto NULL when chunk_text changes without a new embedding. Pinned bytest/e2e/upsert-chunks-registry-column.test.ts(PGLite always; Postgres DATABASE_URL-gated).initSchema()callsapplyForwardReferenceBootstrap(conn)BEFORE replaying SCHEMA_SQL — a one-line delegation to the sharedsrc/core/engine-sql/bootstrap.ts(Postgres dialect hooks:current_schema()probes plus the Postgres-blob-onlydream_verdicts.expires_atprobe); the entire probe path runs on the DDL connection threaded frominitSchema(so concurrent bootstraps cannot race on a Supabase pooler).initSchemaruns the bootstrap, the schema replay and the migrations under the schema migration lock (GBRA-467,holdSchemaMigrationLock, key 42), the same lockrunMigrations()takes whengbrain apply-migrationscalls it directly (withSchemaMigrationLock), so no two runners apply a migration at once; the runner reads the schema version once the lock is held and checks it is still held before each migration, and PGLite (one process per datastore) serializes in-process (src/core/schema-migration-lock.ts,test/schema-migration-lock-pglite.test.ts). The lock ispg_try_advisory_xact_lock(42)inside a READ COMMITTED transaction held open on a backend reserved from the DDL pool, or from the statement pool when the DDL pool has one connection (a short-lived command's idle direct connection would time out and drop it); it holds through a transaction-mode pooler and is released when its holder dies. Only when every pool has one connection does it take the session lock. A timed-out waiter is told not to terminate a holder taggedgbrain-schema-migration-lock. The bootstrap and replay run on a second reserved backend (checkout gauge only, no long-hold permit, so pendingtransaction: falsemigrations can still reserve); a one-slot pool runs them unreserved. Pinned bytest/e2e/postgres-initschema-advisory-lock.test.tstest/e2e/schema-migration-lock-postgres.test.ts(direct and PgBouncer passes) andtest/e2e/schema-migration-lock-direct-pool-postgres.test.ts.disconnect()is idempotent —_connectionStyletracks whether the engine owns its pool (worker engines) or shares the module-level singleton; second call on an instance-pool engine is a no-op rather than clobbering the singleton; and a module-style engine only callsdb.disconnect()when it owns the singleton (_ownsModuleSingleton, set from thedb.connect()creation token), so a borrower probe engine's teardown leaves the cycle owner's connection intact. Pinned bytest/e2e/postgres-engine-disconnect-idempotency.test.ts+test/postgres-engine-singleton-lifecycle.test.ts(concurrent-connect ownership,db.disconnect()detaching the singleton before awaitingend(), and the instance-reconnect_reconnectingguard).getBrainScoreempty-brain parity with PGLite — 100/100 with breakdown 35/25/15/15/10 whenpageCount === 0(both engines must agree to keepdoctor-report-remote.serial.test.tsdeterministic). ImplementsdeletePages(slugs, {sourceId}): Promise<string[]>viaDELETE FROM pages WHERE slug = ANY($1::text[]) AND source_id = $2 RETURNING slug(single round-trip; caller chunks);resolveSlugsByPathsdoesSELECT slug, source_path FROM pages WHERE source_path = ANY($1::text[]) AND source_id = $2; FK cascades throughcontent_chunks/links/tags/raw_data/timeline_entries/page_versions,files.page_id+links.origin_page_idgo SET NULL; throws when input exceedsDELETE_BATCH_SIZE(fromsrc/core/engine-constants.ts); both short-circuit on empty input. Implements the embedding-signature stale-detection quartet (sumStaleChunkChars,setPageEmbeddingSignature,invalidateStaleSignatureEmbeddings, widenedcountStaleChunks, all accept optionalsignatureextending "stale" to model/dims-swap drift via thepages.embedding_signatureJOIN, NULL grandfathered; theembedding IS NULLserver-side filter is preserved as the no-signature fast path); Pinned bytest/e2e/engine-parity.test.ts. Retry classifiers, ontology/recency helpers, and disconnect/pool-recovery audit writers bind statically; only the twoai/gateway.tsfallback lookups stay lazy and line-marked, in parity with PGLite.insertFact+insertFactsdo not hardcodetx.unsafe(\'${embedLit}'::vector`)for the embedding column.resolveFactsEmbeddingCast()(private) probespg_attributeonce per engine instance (cached in_factsEmbeddingCastSuffix) and returns'::halfvec'when migration v40 created the column as halfvec, else'::vector'; both insert paths use the cached suffix so the cast matches the actual column type (works on older pgvector that lacks implicit auto-cast).withReservedConnectionroutes long-hold reserved work (CREATE INDEX CONCURRENTLY,transaction: falsemigration DDL, backfill write batches) to the DIRECT session lane when dual-pool is active, so multi-minute holds do not pin the worker's shared read pool — never rerouted inside an open transaction (same guard shape asexecuteRawDirect), semaphore-capped belowdirect_pool_sizewith deliberately NO minimum-1 floor (atdirect_pool_size=1a floor would let one long reserve consume the only direct session and starve the claim/renewLock heartbeats — the same starvation on the direct lane), overflowing to the shared read pool when the direct lane has no spare capacity or is unavailable; the permit is released on both fn throw and reserve failure.getPoolDiagnostics()(duck-typed, no BrainEngine change) surfaces theCheckoutGaugein-flight counters fromsrc/core/pool-gauge.tsat the raw/direct/reserved/tx seams — a tracked SUBSET (tagged-template traffic is untracked) thatdb-probe.tslabels honestly in health-probe failure lines. Pinned bytest/postgres-engine-reserved-routing.test.ts.getRawDatajoinspagesand addsp.deleted_at IS NULLunlessopts.includeDeleted(thealivefragment next toprivacyin all six branches) so raw session metadata follows the page soft-delete; PGLite mirrors it with onewhere.push; pinned by the standalone getRawData describe intest/e2e/engine-parity.test.ts(PGLite arm always runs; Postgres arm when DATABASE_URL is set).includeDeletedis NOT on the sharedPageReadScope(only getRawData honors it) — it is declared per-method asgetRawData(slug, source?, opts?: PageReadScope & { includeDeleted?: boolean })in engine.ts and both engines; resolved only by trusted callers (export, engine migration; the ingest probe passes it but only ever runs against a live page), never from MCP parameters (get_raw_databuilds its scope fromreadPolicyOpts(ctx)). #6278 PR 1.5:runUnsafearms a settle timer after a signalled statement'scancel(); a statement still pendingcancelSettleMs()later (GBRAIN_CANCEL_SETTLE_MS, default 2000) has its reserved connection discarded so the awaiting caller settles withCONNECTION_DESTROYEDinstead of parking on a round-trip a transaction-mode pooler never completed (backend inClientRead). For excludePrivate callerssearchKeywordruns its full-text match in anOFFSET 0subquery (search/keyword-chunk-match.ts): inside the join the private-visibility subplans inflate every candidate plan's cost, so a cheaper bitmap scan fell within the planner's 1% fuzz of a serial seq scan that detoasts every chunk'ssearch_vector(about 1.1 s for a common term at 250k chunks); local callers keep the unfenced, parallel-capable join. Rows, order and scores are pinned on both engines bytest/search/keyword-arm-ranking.test.ts(goldentest/fixtures/keyword-arm-ranking.json, which also pins one keyword statement per search and two when the strict query matches nothing and the OR fallback runs) and its Postgres twintest/e2e/keyword-arm-ranking-postgres.test.ts`.src/core/search/connection-budget.ts— per-op pooled-connection budget forsearchandquery.SEARCH_CONNECTION_LIMIT(4) is the most connections one op holds at once.withSearchConnectionBudgetopens the scope; the ops'meteredwrapper insrc/core/ops/search.tsanddispatchToolCall(src/mcp/dispatch.ts, so pre-op reads and the response-meta hook count too) both enter it, and a nested call shares the enclosing budget.holdSearchConnectionis called at the engine's connection seams,PostgresEngine#transactionOn(a top-level transaction, held until its COMMIT settles even when a read transaction answers early),executeRaw's JIT-off and statement-timeout envelopes (which reserve their own connection) andrunUnsafe(one statement). A read whosesignalaborts while it waits for a slot leaves the queue and rejectsAbortError. Work inside a held slot runs outside the scope, so statements on a transaction's own connection never wait on a second slot. A read started underwithUrgentSearchConnections(hybrid search's lexical, relational and vector arms,hybrid/arms.ts) takes the next free slot ahead of waiting ordinary reads (the keyword count and entity resolution an op starts early for its response; GBRA-156,test/search-connection-budget-urgent.test.ts). Outside a scope, and on PGLite, nothing waits. These reads use the ordinary read pool (the transaction pooler behind a split route), never the direct/session lane. Pinned on Postgres at the wire bytest/e2e/search-connection-budget-postgres.test.ts. Size ceiling:scripts/module-size-limits.tsv(per-change raise history:git log -pon that file). The ceiling covers every BrainEngine method whose SQL has not moved tosrc/core/engine-sql/(rows inscripts/engine-sql-baseline.tsv), written as parity twins of the PGLite methods (search arms, graph traversal caps, source-scoped stats and health, page lifecycle and soft delete, chunk writes, transaction routing). Migrated domains are one-line delegations over the privateengineSqlgetter (a fresh adapter overthis.sqlon every access); reads that were RLS-scoped on master keep their ownwithScopedReadTransactioncall and hand engine-sqlscopedRead(this.engineSqlOn(tx)), an adapter over the scoped handle;runUnsafeforwardsprepare/simpleonly when the engine-sql adapter sets them.src/core/schema-client-behind.ts— #7776:assertSchemaNotAhead(query)runs on the schema-setup connection before both engines' schema replay: aconfig.versionaboveLATEST_SCHEMA_VERSIONrefusesclient_behind(fixgbrain upgrade,host_admin) before the bootstrap and replay, so an older binary never reinstalls its superseded function bodies over a newer schema. No config table or no version row (a fresh install) is not behind.apply-migrationsrefuses the same way before any orchestrator and before a--force-schema/--force-allschema pass, whose drift repairs would rewrite newer objects (schemaAheadRefusal,connectForForcedSchemainapply-migrations-schema.ts). Pinned bytest/init-schema-client-behind.test.tsand its Postgres e2e twin.src/core/snapshot-schema-inputs.ts— the inputs of the PGLite snapshot schema hash (computeSnapshotSchemaHashinpglite-engine.tshashes each file's raw bytes). Computed, not hand-kept: the static import closure (import,export ... from; type-only imports skipped) ofSNAPSHOT_SCHEMA_ROOTS(pglite-schema.ts,schema-migrations/registry.generated.ts,migrate.ts,engine-sql/bootstrap.ts) plusSNAPSHOT_SCHEMA_LEAVES; every literal dynamicimport()inside the closure must be classified as a leaf or inSNAPSHOT_DYNAMIC_IMPORTS_NOT_HASHED. Pure (the caller supplies file access).test/snapshot-inputs-closure.test.tsnames any missing file and checks everypglite-snapshot-*CI cache key covers the same inputs.src/core/sql-trace.ts— env-gated wire-level SQL trace for benchmarks.traceSqlOptions(options, pool)returns the postgres.js options untouched unlessGBRAIN_SQL_TRACE=<file.jsonl>is set; then it supplies a tracing socket and setsapplication_nametogbrain:<GBRAIN_SQL_TRACE_LABEL or command>:<pid>:<pool>. All four pool construction sites use it (db.tsmodule singleton,PostgresEngineinstance pool,ConnectionManagerread and direct pools), soexecuteRaw,tx.unsafe, the engine-sql adapter and tagged templates are all seen. Each Sync, simple Query or describe Flush and its reply is one JSONL record (statement text without parameter values, start, duration, process, pool, client connection, backend pid, SQLSTATE on error); aconnectrecord covers TCP plus the startup handshake. TLS connections are recorded once astls_untraced. Records are buffered and flushed every second and at exit. Consumed byscripts/bench/managed-sync-catchup.ts(record); pinned bytest/sql-trace.test.tsandtest/e2e/sql-trace-postgres.test.ts.src/core/supabase-admin.ts— Supabase Management-API client (listProjects,discoverPoolerUrl,extractProjectRef— refs are lowercase ALPHANUMERIC,[a-z0-9]+). Consumed by theinit --prefer-postgresladder's rung 2 (DISCOVERY only — project creation stays dashboard guidance; discovered URLs are candidates, connect-probed before persisting) and by the classifier's Supabase hints. The access token is never persisted, logged, or written to receipts/config.src/core/worker-pool.ts— (must-abort tags:BUDGET_EXHAUSTEDandSPEND_GROUP_REFUSED, a job's spend-group refusal) Canonical sliding-pool + bounded-semaphore primitive (used by thesrc/commands/embed.tssliding-pool sites andsrc/commands/eval-cross-modal.ts'srunWithLimitsemaphore). Two exports:runSlidingPool<T>({items, workers, onItem, signal?, onError?, failureLabel?, onProgress?})+runWithLimit<TIn, TOut>({items, limit, fn, signal?}). Atomicity invariant:const idx = nextIdx++is one synchronous JS statement (noawaitbetween read and write — guaranteed by the single-threaded event loop), documented in the module header AND enforced byscripts/check-worker-pool-atomicity.sh(wired intobun run verify), which rejects importingworker_threadsin any consuming file and insertingawaitbetween thenextIdxread and write.MUST_ABORT_ERROR_TAGSset is seeded withBUDGET_EXHAUSTEDfromsrc/core/budget/budget-tracker.ts; tagged errors (matched viaerr.tag === 'BUDGET_EXHAUSTED'to avoid cross-module import) bypassonErrorand hard-abort the pool viaAbortController.abort()to in-flightonItem, then the pool JOINS every worker (each finishes its current item) before rethrowing — the budget cap is a structural ceiling under concurrency and no worker is left running detached past the caller's catch.failures[]shape is{idx, label, error}records (NOT full items; callers supplyfailureLabel(item) => string) for bounded memory under huge brains. Pinned bytest/worker-pool.test.ts+test/scripts/check-worker-pool-atomicity.test.ts. Drives every--workers Nbulk command.src/core/search-vector-backfill.ts— keyset-batched rebuilds of stored keyword vectors, shared byreindex-search-vector(every row) and thefts_cjk_boundarymigration (an optionalwherelimits it to rows with a CJK boundary).backfillChunkVectorsrecomputescontent_chunks.search_vectorwithchunkSearchVectorSql;backfillPageVectors(#6374) reads each batch through an UPDATE-to-self ofid(RETURNING id, slug, source_id, timeline, knowledge_revisionplus whether afact_withdrawals/fact_purgesrow covers the page or its whole source), takes the seal's effective timeline (the raw column, or for a withdrawn/purged pagereadPageSnapshot's overlaid timeline, so a database withdrawal never returns to the vector), sanitizes it withsanitizeRemoteBodyas the seal does, and writessearch_vectorthroughpageSearchVectorSqlin oneUPDATE … FROM unnest(…)guarded by the revision read (a page edited in between is skipped; its seal reindexes it). Each statement scans at mostBACKFILL_RANGE(50 000) ids past the cursor and rebuilds at mostBACKFILL_BATCH_SIZE(5000) of them, never one transaction; the cursor advances to the range end even when nothing matched (so a filtered run on a brain with no CJK text never scans the whole table in one statement), and the run stops at the max(id) the first batch read; the cursor persists after every batch underbackfill.<checkpoint>.last_idand the caller clears it. Pinned bytest/reindex-search-vector.serial.test.ts,test/reindex-search-vector-pages.serial.test.ts,test/fts-cjk-boundary.test.ts,test/search-vector-backfill-withdrawal.test.ts,test/search-vector-backfill-range.test.ts. FTS write expressions (src/core/fts-language.ts, #6370):FTS_INPUT_FUNCTION_SQL(gbrain_fts_input(text), IMMUTABLE STRICT PARALLEL SAFE,search_path = pg_catalog: tworegexp_replacepasses built fromCJK_SLUG_CHARSthat insert a space at every CJK→ASCII-alnum and ASCII-alnum→CJK boundary; a byte-for-byte no-op without CJK, idempotent; spliced into src/schema.sql bybuild:schema),FTS_CJK_BOUNDARY_REGEX(the rows thefts_cjk_boundarymigration rebuilds),pageSearchVectorSql(the sealed page vector used by the seal, the import seal andsearch-vector-backfill.ts),chunkSearchVectorSql, andpageSearchVectorTriggerFnSql/chunkSearchVectorTriggerFnSql(the two trigger functions under a language, byte-identical to src/schema.sql's forenglishso the blob replay after a migration leaves the catalog unchanged; used byreindex-search-vectorand the migration). Every indexingto_tsvectorreads throughgbrain_fts_input;applyFtsLanguagePolicy'sto_tsvector('english',prefix rewrite still matches. Pinned bytest/fts-cjk-boundary.test.ts,test/fts-cjk-boundary-search.test.ts,test/e2e/fts-cjk-boundary-postgres.test.ts.src/core/schema-migrations/v232-fts-cjk-boundary.ts— #6370, handler-only and idempotent: createsgbrain_fts_input, recreates both search_vector trigger functions undergetFtsLanguage()with the shared builders, then rebuilds only pages (title/timeline) and chunks (chunk_text/doc_comment/symbol_name_qualified) matchingFTS_CJK_BOUNDARY_REGEXthroughsearch-vector-backfill.ts(checkpointsfts_cjk_pages/fts_cjk_content_chunks, cleared at the end); every other row's vector is byte-identical under the helper and is not touched. Pinned bytest/fts-cjk-boundary.test.tsandtest/e2e/fts-cjk-boundary-postgres.test.ts.src/core/schema-migrations/v245-page-search-vector-bounded-timeline.ts— Large pages, handler-only and idempotent: recreates the pages search_vector trigger function undergetFtsLanguage()with the bounded timeline (PAGE_TIMELINE_FTS_MAX_CHARS) and entries (PAGE_TIMELINE_ENTRIES_FTS_MAX_CHARS) cuts, then rebuilds only pages whose timeline is longer than half the timeline cap throughsearch-vector-backfill.ts(checkpointpage_vector_bounded_timeline, cleared at the end); shorter timelines index byte-identically and are not touched. Pinned bytest/large-pages/page-timeline-tsvector-bound.test.tsandtest/e2e/page-timeline-tsvector-bound-postgres.test.ts.
Continued in Engines (2/2, continued).