Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
284 KB
Raw Blame

BrainEngine map

This map lists every member of the BrainEngine interface in gbrain (TypeScript), grouped by family. For each member it gives the signature, what it does, the SQL it runs, where PGLiteEngine and PostgresEngine differ, and the tests that cover it.

  • Reference: garrytan/gbrain at commit 8bbfd0b5e8 (master, 2026-11-16). Every path:line citation is relative to the gbrain repo root at that commit.
  • Interface: src/core/engine.ts:830-2890. Engines: src/core/pglite-engine.ts (3,088 lines), src/core/postgres-engine.ts (3,182 lines), plus src/core/pglite-engine/* and src/core/postgres-engine/*. Shared SQL: src/core/engine-sql/*.ts (about 15,000 lines with the engines).
  • Method: each family was read by a separate research pass. Counts are per declaration. Line numbers were read at the commit above. Test coverage was found by grep, not by running the suite. Items a pass could not verify say so in place.

Confirmed build direction: the first build target is a Rust storage engine that implements BrainEngine as a third engine inside gbrain, next to PGLite and Postgres (option B, confirmed by Garry). The binding is still open; a napi-rs native module is the leading option. The last two sections of this map serve that target:

Contents

Section Interface lines Members
Lifecycle, Pages and Search 830-1238 41 (incl. the kind property)
Chunks, Extraction staleness and Links/Graph 1240-1841 45
Tags, Timeline, Chronicle, Ontology, Raw data, Files, Takes, Dream and Contradictions 1842-2195 43
Facts, Segmented storage, Versions, Stats, Sync, Config and Migration support 2196-2890 70
What a Rust engine must do to pass engine-parity test file 60 covered, 137 not covered
Adding a third engine and binding options engine selection n/a

The four family sections count 199 declarations. The parity section counts 197 callable members; the difference is the kind property and how overloads were counted.

Headline findings

  • The SQL is mostly shared. Migrated storage domains keep their SQL once in src/core/engine-sql/ and both engines delegate to it. Engine differences are in the wrappers: Postgres adds statement timeouts, JIT-off reads, reserved connections, read-only transactions, real cancellation and an exact-scan vector fallback. PGLite adds planner-stat refreshes and a single-process lock.
  • Some engine differences are visible to callers. searchKeywordChunks ignores type, types, where and exclude_slugs on PGLite. The stale flag on keyword results is computed on PGLite and a constant false on Postgres. A Rust engine has to pick one behavior per method; the Postgres behavior is the safer reference.
  • Several interface doc comments disagree with the code (addLinksBatch, getEffectiveDates, createVersion, setPageAliases, updatePageContextualRetrievalState, insertFacts). A Rust engine should follow the code and fix the doc first.
  • test/e2e/engine-parity.test.ts covers only 60 of 197 members and needs DATABASE_URL. 46 of its calls are raw Postgres SQL through executeRaw. A Rust engine cannot pass it unchanged; see the parity section.

Lifecycle, Pages and Search

Scope: BrainEngine interface members in src/core/engine.ts:830-1238 (interface opens at :830, the kind discriminator is at :832, and the range ends before the Search-adjacent searchKeywordChunks, which is not on the interface). Engines: src/core/pglite-engine.ts (PGLite, 3088 lines) and src/core/postgres-engine.ts (Postgres, 3182 lines). Domain SQL lives in src/core/engine-sql/*.ts and is shared by both engines.

Member count: 41 interface members in scope (1 property, 40 methods). Optional members: holdSchemaMigrationLock? (engine.ts:848), findDuplicatePage? (:922), findDuplicatePages? (:926). Every other member is required. Note initSchema() is declared with no arguments (:846), but Postgres takes an optional { embedding? } (postgres-engine.ts:576).

Plumbing used by these methods

File Role Key lines
src/core/engine-factory.ts createEngine(config) calls constructEngine, registers the chat usage sink for the engine, and wraps engine.disconnect so the sink is deregistered before the original disconnect. Every production engine passes through here. :10-31 (wrap at :24-28)
src/core/engine-sql/executor.ts SqlExecutor contract: dialect, capabilities, query(sql, params), run(fragment), executeRaw (rows only), unsafe (raw driver, no gauge), transaction(fn). Results are { rows, affectedRows } (EO18). Errors pass through unchanged. DialectCapabilities: maxBindParamsPerStatement (PGLite 30,000, Postgres Infinity), transactionAdvisoryLocks (Postgres true), probesEmbeddingCast (Postgres true). interface :110-123; capabilities :87-108; jsonbParam :133
src/core/engine-sql/dialect-pglite.ts Wraps one PGLite handle. pgliteQuery pre-checks the abort signal, then races the statement against abort. The statement itself is not cancelled (WASM has no cancel). PGLITE_CAPABILITIES at :25-29. transaction calls db.transaction and returns a new executor over the tx handle. :42-56 query; :58-76 executor; :71-74 transaction
src/core/engine-sql/dialect-postgres.ts Wraps one postgres.js handle (pool, reserved conn or tx). query/run go through runUnsafe with prepare: true, simple: false (named prepared statements; PgBouncer prepare:false still honored). executeRaw bumps the raw checkout gauge. unsafe is a bare conn.unsafe. transaction uses readWriteTransactions(conn).begin. POSTGRES_CAPABILITIES at :129-133. jsonbParam binds via conn.json (:137). :144-170; query :145-146; executeRaw :155-162; unsafe :163-165; transaction :166-168
src/core/engine-sql/fragment.ts sqlFragment tagged template builds {strings, values}. Nested fragments splice, trustedSql(text) splices constant identifier or ORDER BY text, anything else becomes a positional parameter. renderFragment numbers $1..$n only at render time. joinFragments for lists. No identifier escaping: identifiers come from constant allowlists. sqlFragment :46; trustedSql :75; joinFragments :80; renderFragment :94
src/core/engine-sql/brands.ts Read brands. ScopedRead (:174) is an executor branded as running inside withScopedReadTransaction on Postgres (RLS app.scopes bound when enabled); scopedRead() (:184) is an identity cast used only on the handle that helper yields (PGLite brands its own executor). LegacyUnscopedRead (:177) marks reads that ran unscoped on master (EO4 inventory), created by unscopedExecutor(exec, reason) (:193). Brand keys are checked by scripts/check-engine-sql-brands.ts. :174-196
src/core/engine-sql/normalize.ts compileRowNormalizer(kinds) converts only declared columns. Kinds: jsonb (parse text), bigint (Number), date (Date), vector (parseEmbedding), text[]. Used e.g. by the domain-bank row mapper (last_retrieved_at: 'date') in engine-sql/pages.ts. ColumnKind :212; compile :227-236

Engine-side wiring for all methods below: each engine exposes an engineSql getter that builds a fresh executor over the current handle on every access (never cached), so transaction() clones (created via Object.create(this)) run domain SQL inside the transaction. Postgres: postgres-engine.ts:317-325. PGLite: pglite-engine.ts:794 engineSql getter.

Lifecycle

Member (engine.ts) Signature What it does PGLite (pglite-engine.ts) Postgres (postgres-engine.ts)
kind (:832) readonly kind: 'postgres' | 'pglite' Discriminator for migrations and consumers. kind = 'pglite' as const (:739) kind = 'postgres' as const (:226)
connect (:835) connect(config: EngineConfig): Promise<void> Opens the datastore. connect :799 → _connectWithRootRegistration (registers managed filesystem engine) → _connectInternal (:887). Takes a kernel file lock (acquireLock(dataDir), pglite-lock.ts) for file-backed DBs, then PGlite.create({dataDir}), configures codecs. In-memory: optional GBRAIN_PGLITE_SNAPSHOT restore (skips schema replay). Wasm-abort on a persistent dir triggers attemptWalRepairAndRetry (in-place WAL repair, with consent-gated refusal). Stores _savedConfig for reconnect. connect :433. Two modes: (a) instance pool when config.poolSize is set: own postgres(url) with max from resolvePoolSize, idle_timeout 20, session timeouts (statement_timeout, idle_in_transaction_session_timeout), prepare per URL, ConnectionManager wrapper; (b) module singleton via db.connect (db.ts), shared with other engines. types: { bigint: postgres.BigInt } on instance pools.
disconnect (:836) disconnect(): Promise<void> Tears the datastore down. Idempotent (memoized call). disconnect :1050. Sets requested flag, awaits pending connect, then _closeInternal (:1088): runs the _beforeDisconnect barrier, sets _db = null, stopAndDrain on DB work, drain background work, checkpoint (failure warns and retains ownership), db.close(), release kernel lock. Close watchdog arms SIGTERM/SIGKILL timers. A failed close poisons the engine (_closePoison) and retains the lock. disconnect :516 → disconnectInternal (:526): runs _beforeDisconnect stops, logs disconnect audit, drains background work, closes ConnectionManager (direct pool), db.endPoolBounded(this._sql) (bounded wait, #1972). Module-singleton teardown only by the engine that created it (_ownsModuleSingleton).
reconnect (:845) reconnect(ctx?: { error?: unknown }): Promise<void> Recovers a dropped connection from the config captured at connect(). Callers must use it instead of disconnect + bare connect (#2034). reconnect :1139: no-op if never connected or in-memory (database_path unset, to preserve state). File-backed: disconnect() then connect(savedConfig). reconnect :2859 (far from the other lifecycle methods). Instance pool: build-then-swap (new pool proven live before old is ended); _reconnecting guard; ctx.error classified by isConnectionEndedError to audit reap_detected vs reconnect_other (logPoolRecovery). Module singleton: db.connect(savedConfig) then refresh ConnectionManager read pool.
initSchema (:846) initSchema(): Promise<void> (Postgres: opts?: {embedding?}) Creates or replays the schema and applies migrations. initSchema :1147. Skipped entirely if a snapshot was loaded. Dims/model from embedded settings, gateway, or defaults; stored embedding identity wins. Runs under withSchemaMigrationLock (in-process serialization, since no holdSchemaMigrationLock): assertSchemaNotAhead (#7776), applyForwardReferenceBootstrap, db.exec(getPGLiteSchema(dims, model)), runMigrations(this), then registerManagedFilesystemEngine. No verifySchema or zombie-index sweep. initSchema :576. Pool = connectionManager.ddl() when dual-pool, else this.sql. Dims/model as PGLite. Runs under withSchemaMigrationLock (:619): assertSchemaNotAhead (:620), applyForwardReferenceBootstrap(conn), retrySchemaReplayOnDeadlock(withSchemaLockTimeout(conn, conn.unsafe(schemaSql))) (:642; session lock_timeout on a reserved backend, #5227), runMigrations, verifySchema(this) (self-heals missing columns), dropZombieIndexes(this) (best-effort, invalid HNSW indexes). Clears shared types if anything changed.
holdSchemaMigrationLock? (:848) holdSchemaMigrationLock?<T>(fn: (assertHeld) => Promise<T>): Promise<T> GBRA-467 cross-process lock (key 42). Optional: when absent, schema-migration-lock.ts:22-30 serializes in-process only. Not implemented. The schema-migration-lock.ts wrapper serializes within the process (PGLite datastore lock admits one process). holdSchemaMigrationLock :687. Uses the DDL pool if max>1, else the statement pool if max>1, else a session lock (holdSchemaMigrationSessionLock). Reserved-connection form (holdSchemaMigrationTransactionLock) polls pg_try_advisory_xact_lock(42) (init-schema-lock.ts:189-203). Session form uses pg_advisory_lock(42) (init-schema-lock.ts:130-137) and pg_advisory_unlock (:237).
transaction (:849) transaction<T>(fn: (engine) => Promise<T>, opts?: TransactionOptions): Promise<T> Runs fn on a clone engine bound to a transaction; nested calls become savepoints. transaction :1209. Checks only opts.signal (abort before start). Uses db.transaction, runs under _dbWork.admit + PgliteCheckpointGuard.runOutermost (checkpoint blocked during tx), composable tx handle, withHeldPageKeys. Ignores readOnly and pipelineEarly. A nested call on a tx clone runs inside the same tx. transaction :716 → transactionOn (:726). readOnly opens BEGIN READ ONLY (not wrapped by readWriteTransactions). checkoutGauge.acquire('tx') (top-level only). Options pipelineEarly, pipelineFirst, signal (abort via withTransactionAbort), commit checks (withCommitChecks), held page keys, holdSearchConnection. Clone sets sql to the tx handle; nested call uses conn.begin(run) (savepoint).
transactionDirect (:851) transactionDirect<T>(fn, opts?) Short control transaction on the direct route. Honors nesting. transactionDirect :1225: return this.transaction(fn, opts) (no direct route exists). transactionDirect :720: when dual-pool and top-level, uses connectionManager.ddl() (direct pool), else this.sql; then transactionOn.
registerBeforeDisconnect (:853) registerBeforeDisconnect(stop: () => Promise<void>): () => void Registers a mandatory resident-consumer stop barrier that runs before datastore or pool teardown. Returns an unregister function. registerBeforeDisconnect :753. Adds to _beforeDisconnect Set; barrier runs at the start of _closeInternal, before _db = null. registerBeforeDisconnect :230. Same Set. Barrier runs first in disconnectInternal (:524).
withReservedConnection (:859) withReservedConnection<T>(fn: (conn: ReservedConnection) => Promise<T>, opts?: { route?: 'ordinary'; selfContained?: boolean }): Promise<T> Runs fn on one dedicated backend. Release automatic. route: 'ordinary' skips the direct pool. selfContained lets a one-connection pool lend the connection. withReservedConnection :1196 (opts not accepted). No pool: passes the single db through. conn.executeRaw only; no transaction member. withReservedConnection :763. Picks the direct DDL pool when dual-pool and not route:'ordinary', guarded by tryAcquirePoolLongHold (permit; throws PoolCapacityError if none). Gauge reserved acquired before pool.reserve(). conn.executeRaw uses reserved.unsafe (AbortSignal ignored). conn.transaction = transactionOn(reservedTransactions(reserved)). Release in finally (double-release tolerant).

Pages CRUD

Shared SQL for most methods is in src/core/engine-sql/pages.ts (both engines call pagesImpl.* with this.engineSql). Differences are in how the read/write is scoped and wrapped (noted per row).

Member (engine.ts) Signature (compact) What it does and SQL Engine differences
getPage (:869) getPage(slug, opts?: GetPageOpts): Promise<Page | null> Returns readPageSnapshot(...).page, or null. Soft-deleted rows return null unless includeDeleted. PGLite :1230 calls readPageSnapshot directly. Postgres :850 same contract.
readPageSnapshot (:870) readPageSnapshot(slug, opts?: PageSnapshotOptions): Promise<PageSnapshot | null> Canonical page snapshot (src/core/page-state/snapshot.ts:66-120): one page row by slug. With resolveAlias, falls through slug_aliases to the canonical page. Ordering (p.slug = $1) DESC, (p.source_id = $n) DESC, p.source_id ASC LIMIT 1. Filters: deleted_at unless includeDeleted; requireLiveSource (sources not archived); requireUnambiguous (count(*) OVER ()). Joins sources.incarnation, tags (jsonb_agg), fact-withdrawal and fact-purge aggregates (fact_purges). PGLite :1234 executeRaw directly, no scope. Postgres :854 runs inside withScopedReadTransaction(sourceIds, sourceId) with prepare: true; it uses JIT off and an alwaysTransaction wrap only when resolveAlias && excludePrivate (#6276).
readPageSnapshotsBatch (:875) readPageSnapshotsBatch(refs: {slug, sourceId}[], opts?: {maxBytes?, includeDeleted?, absentIsNull?}): Promise<PageSnapshotBatch> Exact-ref batch (src/core/page-snapshot-batch.ts:42-60). VALUES-style wanted list with cumulative bytes (SUM ... ROWS UNBOUNDED PRECEDING AND 1 PRECEDING) and returns the longest prefix whose bodies fit maxBytes. Same snapshot joins as readPageSnapshot. PGLite :1238 executeRaw. Postgres :861 binds the batch's distinct sourceIds for RLS through withScopedReadTransaction.
lockPageKeys (:877) lockPageKeys(keys: readonly PageKey[]): Promise<void> Holds exact page identities (including absent rows) through commit. Throws unless called on a transaction engine. Takes only keys not already held (sorted acquisition via lockUnheldPageKeys, page-state/guards.ts:114-119). Both engines identical in shape: PGLite :1242 (lockUnheldPageKeys), Postgres :867 (lockUnheldPageKeys).
putPage (:891) putPage(slug, page: PageInput, opts?: PageWriteOptions): Promise<Page> Validates slug, runs in transaction(write) (or inline when already in a page tx and opts.inline). In order: lockPageKeys; revision precondition check when expectedRevision/force set (readPageSnapshot includeDeleted); assertOriginUnclaimed(revive:true) (persistence/origin-guard.ts:112); then pages.ts:103-204 INSERT ... ON CONFLICT (source_id, slug) DO UPDATE with deleted_at = NULL (revival), COALESCE preservation for effective_date, import_filename, source_path, source_kind, source_uri, ingested_via, ingested_at, contextual retrieval columns when provided, chunker_version via bodyWriteChunkVersion, RETURNING ... . Blank-body guard: if the incoming body is blank and not allowEmptyOverwrite, reads the prior row and refuses to blank a non-empty or segmented page (pages.ts:116-133). PGLite :1262 passes a reread callback to pagesImpl.putPage for the PGLite edge case where RETURNING yields zero rows. Postgres :887 passes none, and maps origin unique violations through classifyOriginUniqueViolation.
findDuplicatePage? (:922) findDuplicatePage?(sourceId, opts: {hash, frontmatterId?, excludeSlug?}): Promise<{slug,id} | null> Import dedup pre-check (#1309). pages.ts:57-80: WHERE source_id = $ AND deleted_at IS NULL AND (content_hash = $hash OR (frontmatter->>'id' = $fmId AND $fmId IS NOT NULL)), excluding excludeSlug. Orders frontmatter-id match first, then id. LIMIT 1. PGLite :1252 uses scopedRead(this.engineSql) with no scope transaction. Postgres :877 runs in withScopedReadTransaction(undefined, sourceId).
findDuplicatePages? (:926) findDuplicatePages?(sourceId, inputs[]): Promise<Array<{slug,id}|null>> Batched version (GBRA-75 w10). pages.ts:82-101: jsonb_to_recordset of inputs joined to pages, DISTINCT ON (k.n), same predicate and ordering. Returns one entry per input, in order. PGLite :1258; Postgres :883. Same SQL.
deletePage (:952) deletePage(slug, opts?: {sourceId?}): Promise<void> Hard delete. pages.ts:206-208: DELETE FROM pages WHERE slug = $ AND source_id = $ (source defaults to 'default'). Cascades via FKs only. No scrub of name/feedback stores. Used by purge, sync per-slug fallback, and test teardown. PGLite :1276 and Postgres :901 both call pagesImpl.deletePage(this.engineSql, ...).
deletePages (:989) deletePages(slugs: string[], opts: {sourceId}): Promise<string[]> Batch hard delete, one transaction. pages.ts:221-247: DELETE ... WHERE slug = ANY($::text[]) AND source_id = $ RETURNING slug, then scrubs by slug: scrubPurgedSlugs, scrubPurgedNameStores, scrubPurgedSlugStores, scrubPurgedSlugHolds. Throws above DELETE_BATCH_SIZE (500, engine-constants.ts:25). Empty input returns []. Same SQL on both engines via this.engineSql (PGLite :1286, Postgres :911).
resolveSlugsByPaths (:1007) resolveSlugsByPaths(paths: string[], opts: {sourceId}): Promise<Map<string,string>> Path to slug map. pages.ts:249-274: SELECT slug, source_path, deleted_at IS NULL AS live ... WHERE source_path = ANY($) AND source_id = $. Live rows win over tombstones; several live rows resolve via pickCanonicalOrigin; ambiguous paths are omitted. Throws above 500 items. Both use unscopedExecutor(...) (LegacyUnscopedRead, "unscoped on master"): PGLite :1294, Postgres :919.
softDeletePage (:1017) softDeletePage(slug, opts?: {sourceId?}): Promise<{slug} | null> pages.ts:276-295: UPDATE pages SET deleted_at = now() WHERE slug = $ AND deleted_at IS NULL [AND source_id = $] RETURNING slug. Null if nothing flipped. No cascade. Note: sourceId has no 'default' fallback; when omitted, the predicate matches the slug in every source. PGLite :1301, Postgres :926 (both pagesImpl.softDeletePage(this.engineSql, ...)).
softDeletePages (:1050) softDeletePages(slugs, opts: {sourceId}): Promise<string[]> Batch soft delete (#4587). pages.ts:297-310: UPDATE ... SET deleted_at = now() WHERE slug = ANY(...) AND source_id = $ AND deleted_at IS NULL RETURNING slug. Empty returns []; over 500 throws. Re-runs never refresh deleted_at (72h clock). PGLite :1314, Postgres :937. Same SQL.
restorePage (:1056) restorePage(slug, opts?: {sourceId?}): Promise<boolean> pages.ts:312-327: one CTE statement. UPDATE pages SET deleted_at = NULL, updated_at = now() WHERE slug = $ AND deleted_at IS NOT NULL [AND source_id], plus DELETE FROM op_checkpoints for the owed-purge record (GST-2766). Runs inside transaction after assertRevivalUnclaimed (persistence/origin-guard.ts:156). PGLite :1318, Postgres :941 (same wrapping).
purgeDeletedPages (:1071) purgeDeletedPages(olderThanHours, opts?: {dryRun?, excludeIds?}): Promise<{slugs, count, pages?}> Hard purge after the soft-delete window (pages.ts:329-403). Hours floored, clamped at 0. Dry run: same WHERE predicate as a SELECT, returns pages. Real run, one transaction: collects shas from page_body_manifest; DELETE FROM pages WHERE deleted_at IS NOT NULL AND deleted_at < now() - interval excluding rows under holds (identity-downgrade checkpoint, atomRetirementPurgeHold, derivedRetirementPurgeHold, derivedOriginPurgeHold) and excludeIds; then expireGonePageFacts, deletes page_body_stagings rows, deletes orphaned page_segment_texts, and scrubs per source (purged slugs, name stores, slug stores, slug holds). PGLite :1322 and Postgres :945 both use this.engineSql. Same SQL; PG runs it on the pool or a tx clone.
listPages (:1079) listPages(filters?: PageFilters): Promise<Page[]> pages.ts:484-561. Filters: type, tag (JOIN tags), updated_after or keyset (updated_at, slug), slugPrefix (LIKE with escaped metachars), excludeSlugPrefixes (starts_with), slugAfter, sourceIds/sourceId, includeDeleted (default hides), excludePrivate (privatePagesFilterForCaller), effective_after/effective_before, frontmatter where, sort via PAGE_SORT_SQL allowlist, listColumnsOnly. LIMIT default 100. Projects updated_at_iso with microseconds for keyset resume. PGLite :1377 wraps in plannerRead (runs beforePlannerRead stats settle). Postgres :1009 wraps in withScopedReadTransaction(filters.sourceIds, filters.sourceId).
findPages (:1081) findPages(query: FindPagesQuery): Promise<FoundPages> Live pages whose slug, source_path or title matches a find pattern (engine-sql/page-lookup.ts:43). PGLite :1380 via plannerRead; Postgres :1012 via withScopedReadTransaction.
countPagesBy (:1083) countPagesBy(query: PageCountQuery, groupTypes?: TypeGrouper): Promise<PageCounts> Live page counts per (source, type). With groupTypes, distinct-slug totals per type group (page-lookup.ts:72). PGLite :1383 via plannerRead; Postgres :1015 via withScopedReadTransaction.
resolveSlugs (:1097) resolveSlugs(partial: string, opts?: PageReadScope): Promise<string[]> Fuzzy slug resolver. pages.ts:752-~800: exact slug match first (deleted_at IS NULL + scope + privacy). Otherwise up to 5 rows: WHERE deleted_at IS NULL AND (title % $partial OR slug ILIKE '%partial%'), ORDER BY similarity(title, $partial) DESC LIMIT 5 (pg_trgm). Scope: sourceIds array wins over sourceId; neither = unscoped (back-compat). Both use unscopedExecutor(this.engineSql, ...): PGLite :1415, Postgres :1056 (direct, no scope tx). Needs pg_trgm on both (PGLite ships its own extension set).
getAllSlugs (:1108) getAllSlugs(opts?: {sourceId?}): Promise<Set<string>> pages.ts:563-572: SELECT slug FROM pages or ... WHERE source_id = $. No deleted_at filter, so soft-deleted slugs are included. PGLite :1387 scopedRead(this.engineSql) direct; Postgres :1019 via withScopedReadTransaction(undefined, opts.sourceId).
listAllPageRefs (:1127) listAllPageRefs(): Promise<Array<{slug, source_id, updated_at: Date}>> pages.ts:574-589: SELECT slug, source_id, updated_at FROM pages WHERE deleted_at IS NULL ORDER BY source_id, slug. Cross-source, unscoped. Both unscopedExecutor(this.engineSql, ...): PGLite :1391, Postgres :1023.
listAllSources (:1145) listAllSources(opts?: {includeArchived?, localPathOnly?}): Promise<SourceRow[]> engine-sql/sources.ts:14-41: SELECT id, name, local_path, last_sync_at, config FROM sources WHERE (includeArchived OR archived IS NOT TRUE) AND (!localPathOnly OR (local_path IS NOT NULL AND config->>'role' IS DISTINCT FROM 'storage_manifest')) ORDER BY (id='default') DESC, id. Config parsed if text. Both unscopedExecutor(...): PGLite :1396, Postgres :1028 (sources table is not RLS-scoped in master).
updateSourceConfig (:1165) updateSourceConfig(sourceId, patch: Record<string,unknown>): Promise<boolean> Atomic JSONB merge (engine-sql/sources.ts:43-81): `UPDATE sources SET config =
listPrefixSampledPages (:1184) listPrefixSampledPages(opts: DomainBankSampleOpts): Promise<DomainBankRow[]> pages.ts:619-~700. CTE prefix_pages: prefix = substring(slug from '^[^/]+/[^/]+'), connection_count = COUNT(page_links.to_page_id), last_retrieved_at = GREATEST(p.last_retrieved_at, page_retrievals.last_retrieved_at). Filters: deleted_at IS NULL, body_storage = 'inline', excludeSlugs, source scope. ROW_NUMBER() ... PARTITION BY prefix ordered by stale score (if staleBias, threshold default 90 days), then connection_count DESC, then slug. One page per requested prefix. Representative chunk = lowest chunk_index with embedding IS NOT NULL. PGLite :1407 scopedRead(this.engineSql) direct. Postgres :1046 wraps reads in withScopedReadTransaction(opts.sourceIds, opts.sourceId).
listCorpusSample (:1195) listCorpusSample(opts: CorpusSampleOpts): Promise<DomainBankRow[]> pages.ts:704-750: optional SELECT setseed(clamped float8) on the same executor when opts.seed is a number, then ORDER BY RANDOM() LIMIT n over the same filters (inline bodies, not deleted, exclusions, source scope). Same representative-chunk subquery. PGLite :1411 direct. Postgres :1052 passes alwaysTransaction: typeof opts.seed === 'number', so the seed and the select share one transaction and connection.

Search

Keyword search shares one builder family in src/core/search/* (keyword-chunk-match.ts, keyword-statement.ts, keyword-or-rarity.ts, title-statement.ts, vector-statement.ts, cjk-keyword-sql.ts). Engine code wires them to the executor.

Member (engine.ts) Signature (compact) What it does PGLite (pglite-engine.ts) Postgres (postgres-engine.ts)
searchKeyword (:1198) searchKeyword(query, opts?: SearchOpts): Promise<SearchResult[]> Page-grain keyword search, best chunk per page. Shape: content_chunks.search_vector @@ websearch_to_tsquery(<ftsLang>, $1), ts_rank(...) * keyword phrase factor * sourceFactorCase (source boosts, detail gate), ROW_NUMBER() OVER (PARTITION BY page_id ...) per page, innerLimit = min(limit*3, cap*3), visibility clause (soft-delete, archived source), hard-exclude prefixes, cc.modality = 'text'. OR fallback (orFallback) reruns with OR-of-terms through the same path. CJK queries go to _searchKeywordCJK. searchKeyword :1433. Plain db.query with beforePlannerRead settle (planner-stats.ts:389). No statement timeout, no JIT setting, no work_mem, no candidate LIMIT (KEYWORD_RANK_CANDIDATES not applied). Emits stale per row via timeline_entries MAX(created_at) comparison. searchKeyword :1066 (builder :1078-1262). Uses withScopedReadTransaction with alwaysTransaction: true, jitOff: true and settings: statement_timeout from keywordStatementTimeoutSetting() (default 8000 ms, range 1000-60000; keyword-statement-timeout.ts), enable_seqscan: 'off' on relaxed or candidate-join attempts, KEYWORD_STATEMENT_SETTINGS (work_mem 32MB, keyword-chunk-match.ts:40). Candidate rows capped at KEYWORD_RANK_CANDIDATES = 50,000 (keyword-chunk-match.ts:30). Timeouts routed through keywordTimeouts negative cache (#6495; :222, :1231). Emits false AS stale.
countKeywordPages (:1200) countKeywordPages(query, opts?: SearchOpts & {timeoutMs?}): Promise<number> Cat 40 Hard F2 strict count only (never OR retry): buildKeywordCountStatement (search/keyword-statement.ts), capped at KEYWORD_COUNT_CAP + 1 = 10,001 (keyword-statement.ts:35). Via engine-sql/keyword-pages.ts:24-33. countKeywordPages :1571 via keywordPagesImpl.countKeywordPages(read => read(scopedRead(this.engineSql)), ...) with empty dialect (no timeout). countKeywordPages :1266. Scoped read with alwaysTransaction, jitOff, settings; statement_timeout = timeoutMs override when given, else keyword timeout; plus KEYWORD_STATEMENT_SETTINGS.
searchKeywordPages (:1202) searchKeywordPages(query, opts: SearchOpts | undefined, page: KeywordPageWindow): Promise<SearchResult[]> One window of matching pages (best chunk each) ordered (score DESC, page_id ASC), no global chunk cap. Statement from buildKeywordPagesStatement (keyword-statement.ts). engine-sql/keyword-pages.ts:36-40. Also searchHandlePages (keyword-pages.ts:43-47) for slug-handle hits (GBRA-108). searchKeywordPages :1574 (no timeout, no settings). searchKeywordPages :1269. Same scoped transaction as countKeywordPages, with the keyword timeout and KEYWORD_STATEMENT_SETTINGS.
searchTitles (:1223) searchTitles(query, opts?: SearchOpts): Promise<SearchResult[]> Page-grain title arm (fix/title-retrieval-arm D1). search/title-statement.ts: pages whose title matches websearch_to_tsquery, ORDER BY exact_title DESC, score DESC, p.id ASC (exact-title key #5889), LIMIT/OFFSET, then LEFT JOIN LATERAL one representative chunk (compiled_truth preferred, then lowest chunk_index). Same page-grain filters and visibility as keyword. AND then OR fallback (engine-sql/titles.ts:67). No query-length gate. searchTitles :1578 via titlesImpl.searchTitles with relaxedPrefersIndex: false, staleProbe: true (so stale is computed) and no timeout. searchTitles :1273 via titlesImpl.searchTitles with statementTimeout from keyword timeout, statementSettings: KEYWORD_STATEMENT_SETTINGS, relaxedPrefersIndex: true, staleProbe: false, and chunk-manifest reads.
searchVector (:1224) searchVector(embedding: Float32Array, opts?: SearchOpts): Promise<SearchResult[]> ANN search. search/vector-statement.ts:310 buildVectorSearchStatement: two-stage CTE. The candidate stage keeps a pure distance ORDER BY so HNSW is usable; outer stages re-rank by source factor. Freshness placement (#5824), scope binding and index-walk or scope-scan variants are in the same builder. Iterative HNSW scan used when supportsHnswIterativeScan(extversion) says pgvector is 0.8+ (vector-index.ts:141-144). Pool attempts with remainingMs budget. searchVector :1726. Same statement builder with dialect: 'pglite' (adds timeline stale). this.vectorIterativeScan probe cached on the engine. Each attempt in its own db.transaction with withVectorSettings. No exact-scan fallback (comment at :1727). Calls beforePlannerRead settle. searchVector :1450. vectorScope(opts) (scope binding), then searchIndexWalk / searchVectorPool with attempts via runVectorAttempt (scoped read, jitOff, vectorScanSettings incl. iterative scan and per-attempt statement_timeout). Falls back to an exact scan (stmt.exactSql) when the pool is short. explainVectorSearch (EXPLAIN only) shares the same path.
getEmbeddingsByChunkIds (:1238) getEmbeddingsByChunkIds(ids: number[], column = 'embedding'): Promise<Map<number, Float32Array>> Rehydrates embeddings for rescoring. engine-sql/chunks.ts:1085-1119: column must match COLUMN_NAME_REGEX (else EmbeddingColumnNotRegisteredError) and is identifier-quoted. SELECT cc.id, vector_send(cc.<col>::vector) AS embedding FROM content_chunks cc JOIN pages p ... WHERE cc.id = ANY($::int[]) AND cc.<col> IS NOT NULL AND chunkVisibleFilter. Binary vector_send decoded with decodeVectorSend. Empty input returns an empty Map. getEmbeddingsByChunkIds :1762 via chunksImpl with unscopedExecutor (LegacyUnscopedRead). getEmbeddingsByChunkIds :1541, same brand and SQL. No scope transaction (unscoped on master), though the visibility filter stays in SQL.

Engine-only search members (not on the interface, same range): searchKeywordChunks (Postgres :1291; PGLite :1638) is the chunk-grain anchor primitive with no page dedup. Postgres also has _searchKeywordCJK (postgres-engine.ts:1435) and explainVectorSearch.

CJK keyword fallback (engine-sql/cjk-search.ts): both engines route CJK queries (hasCJK) away from websearch_to_tsquery to an ILIKE term-AND scan (search/cjk-keyword-sql.ts). CJK_CANDIDATE_CAP = 2000 (cjk-search.ts:55). PGLite picks a capped vs uncapped variant from the planner chunk estimate versus PGLITE_CJK_CAPPED_CHUNKS = 50,000 (cjk-search.ts:56, :171). Postgres runs it under the keyword statement timeout and scoped read (postgres-engine.ts _searchKeywordCJK).

Engine-only members adjacent to this range (for completeness)

Not on the interface, but on one or both engines in the same area: connectForRestore, connectWithHeldLock, closeRetainingLock, holdsDatastoreLock, walRepairReceipt (PGLite); onCheckout, getPoolDiagnostics, keywordBoundedRead, refreshPageBody, updatePageContextualRetrievalState, migrateFactsToCanonical, explainVectorSearch, searchKeywordChunks (Postgres and PGLite as noted). putPage also gets its own write-side guard assertOriginUnclaimed from persistence/origin-guard.ts.

Tests

Counts are reference counts from rg over test/ (not distinct tests). Gating: test/e2e/engine-parity.test.ts header says it is gated by DATABASE_URL and skips without Postgres, but the PGLite half always runs.

Member Main coverage
connect, disconnect, initSchema engine-parity.test.ts (17 each); pglite-engine.test.ts (connect 2, initSchema 3, disconnect 1); pglite-engine-disconnect.serial.test.ts; postgres-engine.test.ts; postgres-engine-config-reconnect.test.ts; engine-surface-coverage.test.ts (method sweep, see below).
reconnect postgres-engine-config-reconnect.test.ts (27); supervisor-health-reconnect.test.ts (26); core/retry-reconnect.test.ts (25); connection-resilience.test.ts (16); pglite-reconnect.serial.test.ts; e2e/postgres-reconnect-singleton.test.ts (17); postgres-engine-getter-selfheal.test.ts.
holdSchemaMigrationLock Only test/helpers/postgres-engine-sql-cases.ts (1 ref). No dedicated test of the cross-process lock was found in the search; withSchemaMigrationLock callers are not individually checked.
registerBeforeDisconnect pglite-engine-disconnect.serial.test.ts; persistence-reconnect.serial.test.ts; engine-control-routing.test.ts; search-settings-one-statement.test.ts; engine-keyed-caches.test.ts.
transaction, transactionDirect engine-sql-transaction.test.ts; engine-sql-transaction-parity.test.ts; helpers/engine-sql-rollback-cases.ts; postgres-engine-reserved-routing.test.ts; direct-pool-profile.test.ts; autopilot-transaction-fence.test.ts; engine-control-routing.test.ts.
withReservedConnection postgres-engine-reserved-routing.test.ts (24); worker-lock-renewal.test.ts (25); sync-drain-connection.test.ts (14); gauge goldens test/fixtures/goldens/postgres-engine-gauge/acquires-by-case.json.
getPage, readPageSnapshot engine-parity.test.ts (getPage 32); pglite-engine.test.ts (getPage 13); helpers/postgres-engine-sql-cases.ts; many (readPageSnapshot in ~507 files).
readPageSnapshotsBatch helpers/page-snapshot-batch-cases.ts; helpers/import-reads-cases.ts; entity-names-parity.test.ts; postgres-rls-scope-inventory.test.ts.
lockPageKeys belief-ledger-purge.test.ts; belief-ledger-purge-review.test.ts; link-cross-source.test.ts; persistence-memory-mutations.test.ts; helpers/postgres-engine-sql-cases.ts.
putPage engine-parity.test.ts (74); pglite-engine.test.ts (93). Heaviest-used write in the suite.
findDuplicatePage, findDuplicatePages import-dedup-frontmatter-id.test.ts; import-reads.test.ts; e2e/import-reads.test.ts; import-identity-move.test.ts; helpers/import-reads-cases.ts.
deletePage pglite-engine.test.ts (4); engine-parity.test.ts (1); pages-soft-delete.test.ts; ~36 files.
deletePages, resolveSlugsByPaths engine-parity.test.ts (3 each); sync-delete-batch.test.ts (18).
softDeletePage, softDeletePages, restorePage pages-soft-delete.test.ts (22); sync-soft-delete.serial.test.ts; engine-parity.test.ts (softDeletePage 10, softDeletePages 5, restorePage 9); sync-rename-reconcile.serial.test.ts.
purgeDeletedPages pages-soft-delete.test.ts; e2e/purge-deleted-dryrun-postgres.test.ts; purge-expires-page-facts.test.ts; pathological/h5-atoms-restore.test.ts; pathological/g0-derived-retirement-hold.test.ts.
listPages pglite-engine.test.ts (14); engine-parity.test.ts (2); ~58 files.
findPages, countPagesBy helpers/postgres-engine-sql-cases.ts; engine-surface-coverage.test.ts; list-pages-empty-type-notice-gbra449.test.ts (countPagesBy).
resolveSlugs pglite-engine.test.ts (4); helpers/postgres-engine-sql-cases.ts; validate-params.test.ts; operations-fuzzy-source-scope.test.ts; call-federated-scope.test.ts; e2e/engine-content-privacy.test.ts.
getAllSlugs pglite-engine.test.ts (3); engine-parity via helpers; link-extraction.test.ts; e2e/integrity-batch.test.ts; source-id-tx-regression.test.ts.
listAllPageRefs engine-parity.test.ts (3); ~9 files.
listAllSources list-all-sources.test.ts (24); e2e/list-all-sources-postgres.test.ts (20); e2e/autopilot-fanout-postgres.test.ts; cycle-last-full-cycle-at.test.ts; storage-manifest.test.ts.
updateSourceConfig cycle-stamp-write-failure.test.ts (11); helpers/engine-sql-rollback-cases.ts; e2e/list-all-sources-postgres.test.ts; connector-checkpoint-identity.test.ts.
listPrefixSampledPages, listCorpusSample page-retrievals.test.ts; brainstorm-timeout.test.ts; brainstorm/cost-guardrails.test.ts; postgres-rls-scope-inventory.test.ts.
searchKeyword engine-parity.test.ts (44); pglite-engine.test.ts (23); keyword-timeout-negative-cache-6495.test.ts (44); keyword-statement-timeout-knob-6495.test.ts (26); e2e/search-quality.test.ts; e2e/search-exclude.test.ts; cjk-keyword-fallback.test.ts (20); e2e/engine-parity-cjk.test.ts.
countKeywordPages, searchKeywordPages search-keyword-paging.test.ts (15); e2e/search-keyword-paging-parity.test.ts; search/keyword-pages-limit-first.test.ts; search/slug-handles.serial.test.ts (handles); search/keyword-arm-access.test.ts.
searchTitles engine-parity.test.ts (6); search/title-retrieval-arm.test.ts (21); e2e/title-arm-postgres.test.ts; keyword-timeout-negative-cache-6495.test.ts.
searchVector engine-parity.test.ts (10); pglite-engine.test.ts (6); search/searchvector-escalation.test.ts (12); e2e/vector-candidate-safety-postgres.test.ts (15); e2e/vector-plan-real-column-postgres.test.ts (11).
getEmbeddingsByChunkIds engine-sql-chunks.test.ts; cosine-rescore-chunkless-headstart.test.ts; postgres-engine.test.ts; e2e/search-quality.test.ts; e2e/embedding-column-pglite.test.ts; unified-multimodal.serial.test.ts.

Cross-cutting: test/engine-surface-coverage.test.ts checks that every interface method exists on PGLiteEngine.prototype and that every prototype member is classified (Plan D9). test/e2e/engine-parity.test.ts runs identical corpora through both engines. test/engine-sql-*.test.ts pin the shared SQL, transaction and normalizer behavior per executor. Golden SQL text lives in test/fixtures/goldens/sql-text/ and RLS scope inventory in test/fixtures/goldens/rls-scope-inventory.json.

Parity risks noted

  • Keyword timeouts, JIT and work_mem: Postgres bounds keyword, title, count and CJK reads with statement_timeout (default 8000 ms, configurable 1-60 s), JIT off and work_mem 32MB. PGLite has none of these, so a slow query can run unbounded there. A Rust engine needs an explicit per-query deadline or it will not match Postgres behavior.
  • Keyword candidate bound: Postgres caps keyword candidates at 50,000 chunks (KEYWORD_RANK_CANDIDATES). PGLite does not, so PGLite can rank more candidates and return different top-N under load.
  • stale flag: PGLite computes per row stale from timeline_entries. Postgres searchKeyword returns false AS stale (postgres-engine.ts:1217), and searchTitles on Postgres disables the stale probe (staleProbe: false). Consumers see different stale values per engine.
  • Vector search: Postgres has an exact-scan fallback and iterative HNSW scan (pgvector 0.8+), with an index-walk path and per-attempt budgets. PGLite has no exact fallback and no pool escalation beyond its own attempts (pglite-engine.ts:1733 comment). Result sets can differ when the ANN pool is short.
  • Planner stats: PGLite runs beforePlannerRead (ANALYZE settle) around reads (planner-stats.ts:389-420). No planner-stats call was found in the Postgres engine file. Plan shapes differ, which can change ties in ORDER BY score.
  • Transactions: Postgres honors readOnly (BEGIN READ ONLY), pipelineEarly/pipelineFirst, signal abort and commit checks. PGLite honors only signal. A read-only transaction on PGLite can write.
  • withReservedConnection: PGLite ignores route and selfContained, has no transaction member, and does not enforce pool permits. Postgres enforces PoolCapacityError and direct-pool routing.
  • reconnect: PGLite in-memory is a deliberate no-op (state preserved). File-backed PGLite does a full disconnect and reconnect. Postgres rebuilds pools with build-then-swap. Callers that rely on "reconnect restores the same state" behave differently in-memory vs file-backed PGLite.
  • Schema lock: Postgres uses cross-process advisory key 42 (session or transaction form). PGLite serializes in-process only (schema-migration-lock.ts:22-30). PGLite initSchema also skips verifySchema and the zombie-index sweep that Postgres runs after migrations.
  • Soft delete asymmetry: softDeletePage has no 'default' source fallback. If sourceId is omitted, it can flip the same slug in every source (pages.ts:276-295), while deletePage defaults to 'default'. deletePages, softDeletePages and resolveSlugsByPaths require sourceId.
  • getAllSlugs includes soft-deleted pages (no deleted_at predicate, pages.ts:563-572). Its JSDoc says "every page in the brain", so callers doing existence checks can see tombstones.
  • Unscoped reads on Postgres: resolveSlugs, resolveSlugsByPaths, listAllPageRefs, listAllSources, getEmbeddingsByChunkIds run on the pool without an RLS scope transaction (LegacyUnscopedRead brand). The Rust rewrite has no RLS, so these become plain reads, but any access control must be enforced in the app layer.
  • Blank-body guard: putPage refuses to overwrite a non-empty or segmented body with a blank one unless allowEmptyOverwrite (pages.ts:116-133). Both engines share this logic, but PGLite's reread callback differs from Postgres's RETURNING assumption, and the Rust port must keep the same guard.
  • Batch limits: deletePages, softDeletePages, resolveSlugsByPaths throw above DELETE_BATCH_SIZE = 500 (engine-constants.ts:25). Callers must chunk; a Rust port must keep the cap or the contract changes.
  • Bind-parameter ceiling: PGLite batches under 30,000 bind params (PGLITE_CAPABILITIES, dialect-pglite.ts:25-29). Postgres is unbounded. Batched writes must stay under the PGLite ceiling if both engines stay in use during migration.
  • Statement cancellation: PGLite cannot cancel a running statement (WASM). The abort race in pgliteQuery only settles the caller, the statement finishes in the background (dialect-pglite.ts:42-56). Postgres sends a real cancel. Timeout behavior differs under contention.
  • Lock timeouts: Postgres schema replay runs under lock_timeout (schema-lock-timeout.ts) and retrySchemaReplayOnDeadlock. PGLite has no equivalent.
  • Vector column binding: getEmbeddingsByChunkIds validates the column name by regex and quotes it. The Rust port must keep the same allowlist, since column names are interpolated.
  • listPrefixSampledPages representative chunk uses embedding IS NOT NULL (hardcoded column) even though getEmbeddingsByChunkIds takes a column. Keep both behaviors aligned.
  • Sampling determinism: listCorpusSample with seed depends on setseed followed by ORDER BY RANDOM() on the same connection (Postgres forces a transaction). PGLite is single-connection, so it matches, but the Rust engine must reproduce the same seeded order or accept a different sample.

Chunks, Extraction staleness and Links/Graph

Scope: BrainEngine interface declarations in src/core/engine.ts lines 1240-1841 (the // Chunks, extraction-freshness and // Links blocks, ending before // Tags at line 1842). That range declares 45 methods: 15 chunk/embedding, 7 extraction staleness, 6 link writes, 17 link/graph reads. None of them is marked optional (?) in the interface.

Engine implementations compared: src/core/pglite-engine.ts (PGLiteEngine, 1812-2078) and src/core/postgres-engine.ts (PostgresEngine, 1626-1899). SQL lives in src/core/engine-sql/chunks.ts, pages.ts, links.ts, and in src/core/search/read-enrichment.ts (the graph and boost readers both engines share, via executeRaw), plus src/core/derived-links.ts.

Engine difference, in short: the SQL text is shared. The two engines differ in the read transaction wrapper and the planner/JIT handling around it:

  • PGLite calls scopedRead(this.engineSql) directly, with no transaction. Several reads are wrapped in plannerRead(...) (pglite-engine.ts:1992, 2015, 2029, 2077), which can run ANALYZE-style planner refreshes (src/core/planner-stats.ts:389-410). Postgres never calls plannerRead.
  • PostgresEngine wraps scoped reads in withScopedReadTransaction(sourceIds, sourceId, tx => ...) (postgres-engine.ts:368), which sets the source/RLS scope in a transaction. It also threads chunkManifestReads into chunk reads (postgres-engine.ts:1642, 1650, 1658, 1856). chunkManifestReads returns true on every non-Postgres engine (src/core/search/safe-chunks.ts:107-113), so PGLite omitting the argument is equivalent by construction.
  • getBacklinkCounts on Postgres runs in jitOffRead (postgres-engine.ts:1863-1867, prepare: true). PGLite runs it through executeRaw (pglite-engine.ts:2043).
  • The unscopedExecutor(..., 'EO4 inventory') wrapper marks reads that are deliberately unscoped on the master path. PGLite and Postgres both use it for findByTitleFuzzy, traverseGraph, traversePathsDetailed, findOrphanPages, sumStaleChunkChars, getPageTimestamps, and the links/pages reads that take no scope.

A. Chunks (engine.ts 1240-1423)

Shared helpers used below: chunkReadWhere (engine-sql/chunks.ts:396-408) is the visibility predicate for chunk reads (page slug, source scope p.source_id = ANY($::text[]) or = $, chunkVisibleFilter, optional private/safe filters). staleChunkWhere (chunks.ts:509-534) is the stale predicate: p.deleted_at IS NULL, the active embedding column IS NULL (or signature drift), embed_skip excluded, and budgetLimited excluded. chunkGenerationFilter (search/safe-chunks.ts:77) gates segmented bodies through page_body_manifest.

# Method (engine.ts line) Signature (compact) What it does SQL / delegation PGLite vs Postgres Tests
1 upsertChunks (1269) (slug, chunks: ChunkInput[], opts?: {sourceId, embeddingColumn, expectedRevision, sealChunkerVersion, pageId, deferSeal} & BatchOpts): void Replaces a page's chunk set. Inserts or updates chunks by (page_id, chunk_index) and moves or prunes displaced chunks. Stamps embedded_text_hash, embedded_at, embedding_pending_since. Optional seal of pages.chunker_version. upsertChunksOnce (chunks.ts:70): SELECT id FROM pages ... FOR UPDATE (or the seal UPDATE pages SET chunker_version); optional expectedRevision check; moveDisplacedChunksAndPrune (chunks.ts:360, unnest(int[],int[]) for moves, md5(chunk_text) for match); one batched INSERT INTO content_chunks ... SELECT ... FROM jsonb_to_recordset($json::text::jsonb) ... ON CONFLICT (page_id, chunk_index) DO UPDATE (chunks.ts:260-285), rows batched via jsonRowBatches. Column routed by resolveWriteColumnFromConfigRows and cast with vectorCastSuffix (search/embedding-column.ts:279: ::vector or ::halfvec(N)). embedding_image::vector for image chunks. md5(c.chunk_text) for embedded_text_hash (#4246). PGLite upsertChunks (pglite-engine.ts:1812) runs _upsertChunksOnce via batchRetry + transaction (single connection, _chunkWritesInTransaction shortcut). Postgres (postgres-engine.ts:1626) same logic with lockPageKeys, readPageSnapshot. Same SQL. upsertChunks: about 96 files reference it. Parity: e2e/engine-parity.test.ts:915-943 (round trip, sealed vs unsealed). PGLite: pglite-engine.test.ts:519-560. Also e2e/upsert-chunks-registry-column.test.ts, embedding-migration.test.ts.
2 getChunks (1279) (slug, opts?: PageReadScope & {includeEmbedding, includeUnsealed}): Chunk[] Every chunk of a page in document order. Vectors omitted unless includeEmbedding (returns the active column AS embedding). Returns embedding_is_null as a boolean. Default scope is 'default' source when neither sourceId nor sourceIds is set. chunks.ts:410-462: SELECT ... FROM content_chunks cc JOIN pages p ... WHERE chunkReadWhere ORDER BY chunkDocumentOrder. (cc.<col> IS NULL) AS embedding_is_null. Optional ordinal window function row_number() OVER (PARTITION BY page_id ...) for segmented pages. PGLite pglite-engine.ts:1828 (scopedRead, no tx). Postgres postgres-engine.ts:1645 (withScopedReadTransaction + chunkManifestReads). getChunks: about 108 files. Parity: e2e/engine-parity.test.ts:951-982 (#2555 sourceIds[] parity), :937-948 (includeUnsealed). PGLite: pglite-engine.test.ts:229, 519-537.
3 getChunkPage (1281) (slug, opts: PageReadScope & {fromIndex, limit}): {chunks, total_chunks, max_chunk_index} Paged get_chunks: chunks with chunk_index in [fromIndex, fromIndex+limit), plus total count and max index. Segmented pages use document position instead. chunks.ts:474-495: one aggregate sum(count(*)), max(chunk_index), bool_or(body_generation IS NOT NULL), then getChunks with an indexRange. PGLite pglite-engine.ts:1835. Postgres postgres-engine.ts:1653. get-chunks-paging.test.ts, engine-surface-coverage.test.ts, e2e/segmented-switch-off-rls.test.ts. No parity test.
4 getChunkWindows (1283) (requests: ChunkWindowRequest[], opts: ChunkWindowOpts): ChunkWindowPage[] Batched evidence read keyed by page_id. Re-authorized. Returns page metadata plus chunks in each requested index window, capped at maxRows. chunks.ts:1147-1190: WITH req AS (SELECT * FROM unnest(int[], int[], int[], int[]) AS r(page_id, lo, hi, prio)), auth CTE (source, archived, deleted, projection, quarantine, private, safe filters), chunk_rows with LIMIT maxRows+1, UNION ALL. Batch pattern: four-array unnest. PGLite pglite-engine.ts:1826: scopedRead(this.engineSql) (no tx). Postgres postgres-engine.ts:1640: withScopedReadTransaction + chunkManifestReads. getChunkWindows: about 19 files. Parity: e2e/evidence-delivery-parity.test.ts, evidence-delivery.test.ts, evidence-long-page.test.ts, evidence-tail-trim.test.ts, e2e/segmented-rls-scope.test.ts, postgres-rls-scope-inventory.test.ts.
5 countStaleChunks (1299) (opts?: {sourceId, signature, includeNullSignature}): number Count of chunks needing embedding: active column IS NULL. With signature, also embedded chunks whose page signature differs (NULL signature grandfathered unless includeNullSignature). chunks.ts:536-547: SELECT count(*)::int FROM content_chunks cc JOIN pages p ... WHERE staleChunkWhere. No vector op; IS NULL / <> on text signature only. PGLite pglite-engine.ts:1850: scopedRead, no tx. Postgres postgres-engine.ts:1670: withScopedReadTransaction(undefined, sourceId). countStaleChunks: about 47 files. Parity: e2e/engine-parity.test.ts does not cover it. PGLite: embedding-state.test.ts, embedding-truth-predicates.serial.test.ts, embed-oversize-heal-drain.serial.test.ts, embed-partial-failure-3037.serial.test.ts, embed-stale-progress-5968.serial.test.ts.
6 sumStaleChunkChars (1316) (opts?: same as countStaleChunks): number Sum of LENGTH(chunk_text) over stale chunks (cost preview). chunks.ts:549-564: SELECT COALESCE(SUM(chars),0)::bigint FROM (...). With signature, UNION ALL of the unembedded arm and the drifted-page arm, so each arm uses its own index (#8811). Both: unscopedExecutor(..., 'chunks: unscoped on master (EO4 inventory)'), no source scope transaction (pglite-engine.ts:1855, postgres-engine.ts:1675). sumStaleChunkChars: about 17 files. sum-stale-chunk-chars.test.ts, sum-stale-chunk-chars-8811.test.ts, embedding-signature-stale.test.ts, e2e/migrate-embeddings-postgres.test.ts, helpers/postgres-engine-sql-cases.ts.
7 setPageEmbeddingSignature (1322) (slug, opts: {sourceId?, signature}): void Stamps pages.embedding_signature after (re)embedding a page. chunks.ts:566-571: UPDATE pages SET embedding_signature = $ WHERE slug = $ AND source_id = $. No deleted_at filter. Same SQL. PGLite pglite-engine.ts:1860, Postgres postgres-engine.ts:1680. setPageEmbeddingSignature: about 24 files. embedding-signature-stale.test.ts, embedding-restamp-w3.test.ts, migrate-embeddings-hardening.serial.test.ts, embed-stale-signature-pagination.serial.test.ts.
8 invalidateStaleSignatureEmbeddings (1339) (opts: {signature, sourceId?, includeNullSignature?}): number NULLs embedding, embedded_at and embedding_pending_since for chunks whose page signature differs from the current one. Returns chunk count. chunks.ts:582-609: inTransaction(...), lockEmbeddingSources(tx, sourceId), then UPDATE content_chunks cc SET <col> = NULL ... FROM pages p WHERE ... AND NOT currentSpaceChunkFragment(column, model, dims) AND sigClause RETURNING cc.page_id. Uses splitEmbeddingSignature for model/dims. Both: this.transaction(fn) plus resolveActiveEmbeddingColumnFromEngine (pglite-engine.ts:1864, postgres-engine.ts:1684). Same SQL. invalidateStaleSignatureEmbeddings: about 17 files. e2e/migrate-embeddings-postgres.test.ts, e2e/embedding-tombstones-postgres.test.ts, embedding-migration.test.ts, e2e/upsert-chunks-registry-column.test.ts, helpers/engine-sql-rollback-cases.ts, helpers/postgres-engine-sql-cases.ts.
9 invalidateContentDriftEmbeddings (1351) (opts?: {sourceId?}): number NULLs vectors whose stored embedded_text_hash no longer equals md5(chunk_text) (#4246). Loops in 5000-row batches, one transaction each, no saved cursor (#7117). chunks.ts:685-698 loops invalidateContentDriftBatch (chunks.ts:638-676): WITH b AS (SELECT id FROM content_chunks WHERE id > $after ORDER BY id LIMIT n), w AS (UPDATE content_chunks cc SET <col> = NULL, embedded_at = NULL, embedded_text_hash = NULL ... WHERE cc.embedded_text_hash <> md5(cc.chunk_text) AND NOT embed_skip AND NOT budgetLimited RETURNING 1). The cursor and limit are inlined, not bound. Both: this.transaction(fn) (pglite-engine.ts:1869, postgres-engine.ts:1689). Same SQL. invalidateContentDriftEmbeddings: about 17 files. e2e/content-drift-sweep-postgres.test.ts, e2e/embedding-tombstones-postgres.test.ts, e2e/upsert-chunks-registry-column.test.ts, helpers/engine-sql-rollback-cases.ts, helpers/postgres-engine-sql-cases.ts.
10 listStaleChunks (1368) `(opts?: {batchSize=2000, afterPageId, afterChunkIndex, sourceId, orderBy:'page_id' 'updated_desc', afterUpdatedAt}): StaleChunkRow[]` Keyset-paged stale chunks (NULL active column, not embed_skip, not budget-limited, live page). Cursor is (page_id, chunk_index) or (updated_at DESC, page_id, chunk_index). chunks.ts:700-813: four SELECT ... FROM content_chunks cc JOIN pages p WHERE cc.<col> IS NULL ... ORDER BY p.updated_at DESC NULLS LAST, p.id, cc.chunk_index LIMIT n variants, or ORDER BY cc.page_id, cc.chunk_index with (page_id, chunk_index) > ($, $). No vector op. PGLite pglite-engine.ts:1874: scopedRead, no tx. Postgres postgres-engine.ts:1694: withScopedReadTransaction(undefined, sourceId).
11 countChunklessPagesWithContent (1396) (opts?: {sourceId?}): number Pages with non-empty compiled_truth / timeline (or a segmented body) and zero chunks, excluding quarantined pages and sealed text projections (#8473). chunks.ts:826-847: SELECT count(*)::int FROM pages p WHERE deleted_at IS NULL AND (compiled_truth <> '' OR timeline <> '' OR segmented body) AND quarantine filter AND (text_projection_revision IS DISTINCT FROM knowledge_revision OR embed_skip) AND NOT EXISTS (SELECT 1 FROM content_chunks cc WHERE cc.page_id = p.id). PGLite pglite-engine.ts:1886 (scopedRead). Postgres postgres-engine.ts:1706 (withScopedReadTransaction). countChunklessPagesWithContent: about 16 files. Parity: e2e/engine-parity.test.ts (chunkless detection parity, around line 1177). PGLite: embedding-state.test.ts, embed-stale-chunkless-pages.serial.test.ts, embedding-state-projection-empty-8473.test.ts.
12 listChunklessPagesWithContent (1408) (opts?: {batchSize=50, afterPageId, sourceId}): ChunklessPageRow[] Keyset page (by id) of chunkless pages with their full compiled_truth and timeline. Small default batch on purpose. chunks.ts:848-870: SELECT p.id, p.slug, p.source_id, p.compiled_truth, p.timeline FROM pages p WHERE <chunklessPagesWhere> AND p.id > $ ORDER BY p.id LIMIT n. PGLite pglite-engine.ts:1890. Postgres postgres-engine.ts:1710. Same SQL. listChunklessPagesWithContent: about 13 files. Parity: e2e/engine-parity.test.ts, helpers/postgres-engine-sql-cases.ts. PGLite: embed-stale-chunkless-pages.serial.test.ts, embed.serial.test.ts.
13 readEmbeddingStateCounts (1414) `(opts: {mode: 'ops' 'full', sourceIds?, purgeWindowHours}): EmbeddingStateCounts` Counts per embedding state (embedded, pending, keyword_only, quarantined, empty, deleted_page_chunks, chunkless_healable). ops mode uses a partial index on the active column IS NULL, and falls back to a planner estimate for embedded. chunks.ts:922-1046: pg_index/pg_class lookup for a covering partial index; aggregate with count(*) FILTER (...); EXPLAIN (FORMAT JSON) for the estimated embedded count. embed_skip marker bytes counted. PGLite pglite-engine.ts:1894. Postgres postgres-engine.ts:1714 (withScopedReadTransaction(sourceIds)). The pg_index probe and the EXPLAIN estimate behave differently on PGLite (plannerRows, chunks.ts:1005).
14 listEmbeddingStateRows (1416) (opts: {sourceIds?, slugs?, states[], limit, purgeWindowHours}): EmbeddingStateRow[] One row per (page, state), ordered (state, source_id, slug), at most limit. chunks.ts:1020-1044: SELECT ... FROM (...) r ORDER BY r.state, r.source_id, r.slug LIMIT n over the same state CASE expressions as the counts. PGLite pglite-engine.ts:1899. Postgres postgres-engine.ts:1719. listEmbeddingStateRows: about 12 files. e2e/embedding-state-postgres.test.ts, embedding-state.test.ts, helpers/postgres-engine-sql-cases.ts.
15 deleteChunks (1422) (slug, opts?: {sourceId?}): void Deletes every chunk of a page and clears pages.text_projection_revision (the seal). chunks.ts:1048-1066: WITH page AS (SELECT id FROM pages WHERE slug=$ AND source_id=$ FOR UPDATE), gone AS (DELETE FROM content_chunks WHERE page_id=(SELECT id FROM page) RETURNING page_id) UPDATE pages SET text_projection_revision = NULL WHERE id=... AND .... Page row locked before chunks (lock order, #8502). PGLite pglite-engine.ts:1904, no transaction wrapper. Postgres postgres-engine.ts:1724. Same SQL. deleteChunks: about 16 files. Parity: e2e/delete-chunks-lock-order-8502.test.ts. PGLite: pglite-engine.test.ts:565-572, source-id-tx-regression.test.ts, import-file.test.ts.

Notes for section A:

  • Embedding-column routing is resolved per call by resolveActiveEmbeddingColumnFromEngine (pglite-engine.ts:1831, 1851; postgres-engine.ts:1648, 1671). The write path resolves from config rows inside the upsert statement's own executor (chunks.ts:139-170).
  • pgvector: in this range, the only pgvector operations are the cast on insert (c.embedding::vector or ::halfvec(N), chunks.ts:260) and embedding_image::vector. Stale and drift predicates use IS NULL and md5(), not vector distance operators. Vector reads (vector_send, cosine) are outside this range (engine-sql/chunks.ts:1085-1131, getEmbeddingsByChunkIds and cosineByChunkIdsStatement, declared at engine.ts:1238 and before).
  • Batch pattern: upsertChunks uses jsonb_to_recordset (one JSON document per batch, jsonRowBatches), not unnest. moveDisplacedChunksAndPrune (chunks.ts:389) and getChunkWindows (chunks.ts:1157) use unnest over parallel int arrays.
  • Recursive CTE: none in this section.

B. Extraction staleness (engine.ts 1424-1482)

The predicate is stalePagesWhere in engine-sql/pages.ts (line 827). A page is stale when deleted_at IS NULL, not quarantined, and links_extracted_at IS NULL OR links_extracted_at < versionTs OR updated_at > links_extracted_at OR a wanted-producer origin is stale. Options: attendance ('exclude' | 'blocked', by links_attendance_blocked_revision vs knowledge_revision), held ('exclude' | 'only', linkExtractionHeld), and a guest-source exclusion. This is the shared watermark behind gbrain extract --stale and the links_extraction_lag doctor check.

# Method (engine.ts line) Signature (compact) What it does SQL / delegation PGLite vs Postgres Tests
16 countStalePagesForExtraction (1438) (opts?: {sourceId, versionTs, attendance, held, wantedProducers}): number Counts pages needing (re)extraction. pages.ts:871-875: SELECT count(*)::int FROM pages WHERE stalePagesWhere. extractionLagStatement (pages.ts:855) is the combined total/stale/blocked/held query used by doctor. PGLite pglite-engine.ts:1908, scopedRead. Postgres postgres-engine.ts:1728, withScopedReadTransaction(undefined, sourceId). countStalePagesForExtraction: about 35 files. Parity: e2e/engine-parity.test.ts:1103-1176 (stale-page extraction methods). PGLite: extract-stamp-source-3957.test.ts, guest-derived-registry.test.ts, helpers/postgres-engine-sql-cases.ts.
17 listStalePagesForExtraction (1446) (opts: {batchSize, afterPageId?, sourceId?, versionTs?, held?: 'exclude', wantedProducers?}): StalePageRow[] Keyset page (by id) of stale pages with body and frontmatter, so the caller extracts without getPage per row. pages.ts:877-900: SELECT id, slug, ..., compiled_truth, timeline, frontmatter, updated_at, to_char(updated_at AT TIME ZONE 'UTC', ...) AS updated_at_iso FROM pages WHERE stalePagesWhere AND id > $ ORDER BY id LIMIT n. PGLite pglite-engine.ts:1912, scopedRead. Postgres postgres-engine.ts:1732, withScopedReadTransaction(undefined, sourceId). Same SQL. listStalePagesForExtraction: about 17 files. Parity: e2e/engine-parity.test.ts:1103-1176. PGLite: extract-stale.test.ts, attendance-retrieval.test.ts, extract-endpoint-revision.serial.test.ts, maintenance-idle-gbra438.test.ts.
18 markPagesExtractedBatch (1474) (refs: {slug, source_id, extractedAt?}[], defaultExtractedAt: string): number Stamps links_extracted_at per page. Clears attendance-blocked marker and extract-failure record. Returns rows actually stamped (#3957 shortfall signal). pages.ts:902-929: UPDATE pages p SET links_extracted_at = v.ts::timestamptz, links_attendance_blocked_revision = NULL, ... FROM unnest($slugs::text[], $srcs::text[], $tss::text[]) AS v(slug, source_id, ts) WHERE p.slug = v.slug AND p.source_id = v.source_id. Three-array unnest. Same SQL. PGLite pglite-engine.ts:1923 and Postgres postgres-engine.ts:1743, both with this.engineSql, no scope transaction. markPagesExtractedBatch: about 15 files. Parity: e2e/engine-parity.test.ts:1129-1150. PGLite: extract-stamp-source-3957.test.ts, pglite-checkpoint-guard.test.ts, fixtures/pglite-engine-sql-wal-worker.ts, persistence-managed-lifecycle.test.ts.
19 markPagesAttendanceBlocked (1476) (refs: {slug, source_id, revision}[]): number Marks pages stale only because an attendee did not resolve, at their current knowledge revision (#5761). pages.ts:931-952: UPDATE pages p SET links_attendance_blocked_revision = v.revision::uuid, links_attendance_blocked_at = now() FROM unnest(slug[], source_id[], revision[]) AS v(...) WHERE ... AND p.knowledge_revision = v.revision::uuid AND p.deleted_at IS NULL. Three-array unnest. Same SQL. PGLite pglite-engine.ts:1927, Postgres postgres-engine.ts:1747. markPagesAttendanceBlocked: about 10 files. helpers/postgres-engine-sql-cases.ts, helpers/engine-sql-rollback-cases.ts, engine-surface-coverage.test.ts. No behavior parity test in e2e/engine-parity.test.ts.
20 countStaleFrontmatterKeyPages (1478) (opts: {sourceIds: string[]}): {source_id, n}[] Per source, live pages whose frontmatter still carries a reserved key (type, title, tags, slug) (#6454). pages.ts:954-961: `SELECT source_id, count(*)::bigint FROM pages WHERE source_id = ANY($) AND deleted_at IS NULL AND frontmatter ? ARRAY[...] GROUP BY source_id. JSONB ? ` operator.
21 listStaleFrontmatterKeyPages (1480) (opts: {sourceId, afterId, limit}): {id, slug, keys[]}[] Keyset page of those pages with the reserved keys each carries. pages.ts:963-972: SELECT id, slug, ARRAY(SELECT k FROM unnest(ARRAY[...]::text[]) AS k WHERE frontmatter ? k ORDER BY k) AS keys ... WHERE source_id = $ AND ... AND id > $ ORDER BY id LIMIT n. PGLite pglite-engine.ts:1931. Postgres postgres-engine.ts:1751, scope transaction on sourceId. listStaleFrontmatterKeyPages: about 10 files, mostly golden and inventory tests plus repair-frontmatter-keys-sweep-6454.test.ts.
22 stripStaleFrontmatterKeys (1482) (opts: {sourceId, ids: number[]}): number Strips reserved keys from the named live pages of one source (coordinated maintenance write). pages.ts:974-981: `UPDATE pages SET frontmatter = frontmatter - ARRAY[...]::text[] WHERE source_id = $ AND id = ANY($ids::int[]) AND deleted_at IS NULL AND frontmatter ? ARRAY[...]`. The revision trigger advances each changed row. PGLite pglite-engine.ts:1932 and Postgres postgres-engine.ts:1752, both with this.engineSql. Same SQL.

Notes for section B:

  • pages.ts shares one stale predicate between count and list (stalePagesWhere), so the count and the batch cursor cannot drift apart.
  • Extraction staleness has no recursive CTE and no pgvector use. Batch pattern: markPagesExtractedBatch and markPagesAttendanceBlocked use three-array unnest(text[]). stripStaleFrontmatterKeys uses id = ANY(int[]).
  • markPagesExtractedBatch is also the stamp that the markPagesAttendanceBlocked revision check must agree with. A concurrent edit makes updated_at > links_extracted_at, so the page stays stale (v0.42.7 D4).

C. Links: writes (engine.ts 1484-1573)

# Method (engine.ts line) Signature (compact) What it does SQL / delegation PGLite vs Postgres Tests
23 addLink (1496) (from, to, context?, linkType?, linkSource?='markdown', originSlug?, originField?, opts?: {fromSourceId, toSourceId, originSourceId}): void Single edge upsert. Endpoints are resolved source-qualified in one statement. Throws PageMissingError if either endpoint is missing. Temporal link types refresh relationship state. links.ts:102-150: WITH endpoint_state AS (SELECT (SELECT id FROM pages WHERE slug=$ AND source_id=$ FOR KEY SHARE) AS from_id, ...), upserted AS (INSERT INTO links (...) SELECT ... FROM endpoint_state WHERE from_id IS NOT NULL AND to_id IS NOT NULL ON CONFLICT (from_page_id, to_page_id, link_type, link_source, origin_page_id) DO UPDATE SET context = EXCLUDED.context, origin_field = EXCLUDED.origin_field RETURNING 1) SELECT .... Calls refreshRelationships for temporal types. PGLite pglite-engine.ts:1935 and Postgres postgres-engine.ts:1755 both call linksImpl.addLink(this.engineSql, ...). Same SQL. addLink: about 105 files. Parity: e2e/engine-parity.test.ts:797-830 (soft-delete visibility), :805, 831. PGLite: pglite-engine.test.ts, source-id-tx-regression.test.ts, derived-visibility-repair.test.ts.
24 addLinksBatch (1521) (links: LinkBatchInput[], opts?: BatchOpts): number Bulk edge insert. Returns rows inserted (conflicts excluded). Temporal types run refreshRelationships in a transaction. links.ts:216-240: executeRawJsonb(target, INSERT INTO links (...) SELECT f.id, t.id, v.* FROM jsonb_to_recordset(($1::jsonb)->'rows') AS v(...) JOIN pages f ON f.slug = v.from_slug AND f.source_id = v.from_source_id JOIN pages t ... LEFT JOIN pages o ... ON CONFLICT (...) DO NOTHING RETURNING ...). Wrapped in batchRetry. PGLite pglite-engine.ts:1948 and Postgres postgres-engine.ts:1768, both batchRetry. Same SQL. addLinksBatch: about 77 files. Parity: e2e/engine-parity.test.ts (addLinksBatch-based fixtures). PGLite: link-extraction-dir-whitelist-2576.test.ts, p2-acceptance.test.ts, extract-fs-source-id.test.ts, doctor-stale-mentions.test.ts.
25 replaceDerivedLinks (1522) (origin: DerivedLinkOrigin, links: LinkBatchInput[], opts?: DerivedLinkReplacementOptions): {created, removed} Atomically replaces all derived edges for one origin page: delete then insert, with page-key locks and an extraction-generation check (DerivedLinkSettingsChangedError). derived-links.ts:170-189: engine.transaction(tx => pipelined(primeRelationSemantics, lockPageKeys(...)); readPageSnapshot; publishDerivedLinks(...)). beliefs.commit(engine). Shared code path: both engines call replaceDerivedLinks(this, origin, links, opts) (pglite-engine.ts:1953, postgres-engine.ts:1773). No engine-specific SQL. replaceDerivedLinks: about 22 files. postgres-rls-scope-inventory.test.ts, link-cross-source.test.ts, attendance-retrieval.test.ts, extract-endpoint-revision.serial.test.ts, helpers/postgres-engine-sql-cases.ts. No parity test.
26 replaceDerivedLinksBatch (1524) (items: DerivedLinkBatchItem[]): {created, removed}[] Same as above for many origins in one transaction. Any failure rolls back all of them. derived-links.ts:202-: shared prime, page guards (one sorted lock), extraction-generation check, origin reads, then per-origin publishDerivedLinks. Shared path (pglite-engine.ts:1957, postgres-engine.ts:1777). replaceDerivedLinksBatch: about 12 files. extract-endpoint-revision.serial.test.ts, helpers/extract-db-batch-fixture.ts, helpers/postgres-engine-sql-cases.ts, engine-surface-coverage.test.ts.
27 removeLink (1538) (from, to, linkType?, linkSource?, opts?: {fromSourceId, toSourceId}): number Deletes edges between a pair. Optional type and provenance filters. Returns rows deleted (#4527). links.ts:306-: four branches (type and source, type only, source only, neither). Each is DELETE FROM links WHERE from_page_id = (SELECT id FROM pages WHERE slug=$ AND source_id=$) AND to_page_id = (...) [AND link_type = $] [AND link_source IS NOT DISTINCT FROM $] RETURNING from_page_id, to_page_id, link_type, then refreshRelationships. PGLite pglite-engine.ts:1977, Postgres postgres-engine.ts:1797. Same SQL. removeLink: about 17 files. PGLite: pglite-engine.test.ts, source-id-tx-regression.test.ts. Also managed-links-fence.test.ts, canonical-writer-inventory.test.ts, helpers/link-temporal-scenario.ts.
28 removeLinksByPagesAndSource (1564) (pages: {slug, source_id}[], opts: {linkSource, keepTypedNerPairs?}): number Bulk delete of derived links for a set of from pages, scoped to one link_source. Keeps typed_ner rows whose pair is in the keep list (#3674). links.ts:258-: WITH scope AS (SELECT f.id AS from_id FROM jsonb_to_recordset(($2::jsonb)->'pages') ... JOIN pages f ...), keep AS (SELECT ... FROM jsonb_to_recordset(...'keep') ...), DELETE FROM links l USING scope s WHERE l.from_page_id = s.from_id AND l.link_source = $1 AND NOT (COALESCE(l.link_kind,'')='typed_ner' AND EXISTS (SELECT 1 FROM keep k ...)) RETURNING l.from_page_id, l.to_page_id, l.link_type, then refreshRelationships. Bound via executeRawJsonb. PGLite pglite-engine.ts:1964, Postgres postgres-engine.ts:1784. Comments say "identical SQL shape" (parity). removeLinksByPagesAndSource: about 11 files. Parity: e2e/engine-parity.test.ts. PGLite: extract-by-mention-rebuild.test.ts, helpers/postgres-engine-sql-cases.ts, engine-surface-coverage.test.ts.

Notes for section C:

  • The engine.ts JSDoc for addLinksBatch (engine.ts:1507-1511) says "single multi-row INSERT...SELECT FROM (VALUES) JOIN pages". The implementation is jsonb_to_recordset (links.ts:216-227, #1861). This doc is stale.
  • Batch pattern: addLinksBatch and removeLinksByPagesAndSource both bind their rows as one JSONB document via jsonb_to_recordset, not unnest. That avoids the 65535 parameter cap and the text[] literal crash (#1861).
  • No recursive CTE in the writes. ON CONFLICT targets the composite (from_page_id, to_page_id, link_type, link_source, origin_page_id) key.

D. Links: reads, graph, boosts (engine.ts 1585-1841)

Shared readers in src/core/search/read-enrichment.ts take a ReadQuery (the engine's executeRaw). The SQL in them is the same on both engines. The engines differ only in the transaction and planner wrapper around them, as described at the top.

# Method (engine.ts line) Signature (compact) What it does SQL / delegation PGLite vs Postgres Tests
29 getLinks (1590) (slug, opts?: LinkReadScope): Link[] Outbound edges of a page. sourceId scopes the from page. sourceIds (#2200) constrains from, to and origin. Optional excludePrivate and temporal filters. links.ts:366-: SELECT f.slug AS from_slug ... FROM links l JOIN pages f ... JOIN pages t ... WHERE f.slug = $ AND <linkPageSql> .... Scope by sourceIds = ANY($::text[]) on all three endpoints. Optional TEMPORAL_LINK_SELECT_SQL. PGLite pglite-engine.ts:1987: scopedRead(this.engineSql), no tx. Postgres postgres-engine.ts:1807: withScopedReadTransaction(opts?.sourceIds, opts?.sourceId). getLinks: about 60 files. Parity: e2e/engine-parity.test.ts:1487-1528 (sourceIds[] three-endpoint, scalar cross-source, unscoped identity). PGLite: persistence-graph-publication.test.ts, extract-atoms-provenance-links.test.ts, sync-soft-delete.serial.test.ts, helpers/engine-sql-rollback-cases.ts.
30 getBacklinks (1596) (slug, opts?: LinkReadScope): Link[] Inbound edges of a page. Same scope rules as getLinks, applied to the to side. links.ts:437-: mirror of getLinks with t.slug = $. PGLite pglite-engine.ts:1991: plannerRead(this, this._pageTransaction, () => scopedRead(...)). Postgres postgres-engine.ts:1811: withScopedReadTransaction. The plannerRead wrapper is PGLite-only. getBacklinks: about 36 files. Parity: e2e/engine-parity.test.ts:1529 (sourceIds[] both endpoints), e2e/links-read-order.test.ts, e2e/federated-link-reads-postgres.test.ts, e2e/graph-quality.test.ts.
31 listLinkSources (1604) (opts?: {sourceId?, sourceIds?}): {link_source, count}[] Provenance counts, for gbrain link-sources (v114, #1941). An edge counts only when its from, to and live origin are in scope. links.ts:501-525: SELECT l.link_source, COUNT(*)::int FROM links l JOIN pages f ... JOIN pages t ... [WHERE scope] GROUP BY l.link_source ORDER BY count DESC, l.link_source ASC NULLS LAST. PGLite pglite-engine.ts:1995: scopedRead. Postgres postgres-engine.ts:1815: withScopedReadTransaction. Same SQL. listLinkSources: about 11 files. Parity: e2e/engine-parity.test.ts:983-1001 (ordered counts). Also helpers/postgres-engine-sql-cases.ts, engine-surface-coverage.test.ts.
32 findByTitleFuzzy (1629) `(name, dirPrefix?, minSimilarity=0.55, sourceId?): {slug, similarity} null` Best page whose title is similar to name (pg_trgm). Zero embedding or LLM cost. pages.ts:983-1031: WHERE title % $name AND similarity(title, $name) >= $min AND slug LIKE $prefix [AND source_id = $] AND deleted_at IS NULL ORDER BY sim DESC, slug ASC LIMIT 1. The % operator uses the trigram GIN index idx_pages_trgm when minSimilarity >= 0.3. Both: unscopedExecutor(this.engineSql, 'pages: unscoped on master (EO4 inventory)') (pglite-engine.ts:2007, postgres-engine.ts:1827). Same SQL.
33 traverseGraph (1643) (slug, depth=5, opts?: TraverseGraphOpts): GraphNode[] Breadth-first node walk from a seed. Each page appears once, at its shortest depth, with its outbound {to_slug, link_type} list. Scope (sourceId, sourceIds), privacy, live-origin and temporal filters apply. links.ts:659-709: WITH RECURSIVE bfs(depth, frontier, visited) AS (...), walk(id, depth, pos) AS (...). Built by bfsWalkCtes (links.ts:609-650): frontier/visited are int[] arrays, CROSS JOIN LATERAL unnest(bfs.frontier), anti-join NOT EXISTS (SELECT 1 FROM unnest(bfs.visited) v(id) ...), layer cap LIMIT least(frontierCap, budget), and total budget TRAVERSE_WALK_ROW_CAP + 1 - cardinality(visited) (20000). Final SELECT ... WHERE w.pos <= TRAVERSE_PATH_ROW_CAP, with jsonb_agg(DISTINCT ...) for outbound links. Both: unscopedExecutor(...) on pglite-engine.ts:2015 (wrapped in plannerRead) and postgres-engine.ts:1835 (not wrapped). Same SQL. traverseGraph: about 19 files. Parity: e2e/engine-parity.test.ts:2068-2118 (depth 1..3, cycle guard, sourceId and sourceIds). PGLite: pglite-engine.test.ts, traverse-walk-cap.test.ts, soft-delete-link-visibility.test.ts, e2e/source-isolation-pglite.test.ts.
34 traversePaths (1658) `(slug, opts?: PageReadScope & {depth=5, linkType?, direction='out' 'in' 'both'}): GraphPath[]` Edge-based BFS. Returns each qualifying edge once, at one past the depth of its visited endpoint. pglite-engine.ts:2018 and postgres-engine.ts:1838 both delegate to traversePathsDetailed(...).paths. No separate SQL.
35 traversePathsDetailed (1672) (slug, opts?: LinkReadScope & {depth, linkType, direction}): {paths: GraphPath[], truncated: boolean} Same walk as traversePaths, plus a truncated flag. The walk is bounded by TRAVERSE_WALK_ROW_CAP (20000). The final result by TRAVERSE_PATH_ROW_CAP (5000), counting raw rows before the in-memory dedup. links.ts:711: WITH RECURSIVE walk AS (bfs CTE from bfsWalkCtes with maxLayer = depth - 1), nodes AS (... WHERE w.pos <= CAP). The edges SELECT depends on direction: out joins links.from_page_id, in joins links.to_page_id, both UNIONs the two in a derived table e (links l joined on the frontier side and its depth). LIMIT CAP + 1, dedup in JS by `from_id to_id link_type
36 relationalFanout (1697) (seeds: string[], opts?: RelationalFanoutOpts): RelationalFanoutRow[] Typed-edge fan-out for the relational recall arm (v0.43). From a seed array, returns reached nodes ranked hop ASC, edge_count DESC, source_id, slug, with via_link_types, the shortest path and edges, and a canonical_chunk_id. Within-source only. mentions excluded unless includeMentions. Non-ranking link types excluded. search/read-enrichment.ts:19-115: WITH RECURSIVE walk AS (SELECT seed rows ... ARRAY[p.id] AS visited, ARRAY[p.slug] AS path, ... UNION ALL SELECT p2 ... FROM walk w JOIN links l ... WHERE w.depth < $2 AND NOT (p2.id = ANY(w.visited)) AND p2.source_id = w.seed_source AND ... AND rankingLinkSql('l') AND mentionsFilter ...). The outer GROUP BY n.source_id, n.slug, n.id, MIN(depth), COUNT(DISTINCT last_link_type), array_agg ... ORDER BY depth, array_length(path), path. Depth is clamped to 1..3 (default 2), limit to 1..200 (default 50), direction defaults to both. Canonical chunk via a correlated LIMIT 1 subquery. PGLite pglite-engine.ts:2032: readRelationalFanout(this.executeRaw.bind(this), seeds, opts) (default chunkManifest = true). Postgres postgres-engine.ts:1852: passes await chunkManifestReads(this). On PGLite that returns true, so both are equivalent. relationalFanout: about 15 files. Parity: e2e/engine-parity.test.ts:1337-1400 (relationalFanout parity), relational-recall.test.ts, e2e/read-enrichment-privacy.test.ts, attendance-retrieval.test.ts, e2e/segmented-switch-off-rls.test.ts.
37 relationalChainHop (1710) (frontierPageIds: number[], opts: ChainHopOpts): ChainHopEdge[] One oriented expansion step of a multi-hop relational chain. Returns logical edges from frontier nodes, oriented by type signature (canonical, stored, flipped, uncertain). Caps per-frontier neighbors (neighborCap <= 1000). Computes from_degree with saturation at 300. search/read-enrichment.ts:117-241: WITH f AS (SELECT DISTINCT unnest(ARRAY[...]::int[]) AS id), cand AS (outBranch UNION inBranch), classified, semantic, stepped, logical (GROUP BY fid, next_id), ranked (ROW_NUMBER, COUNT OVER per frontier), deg_rows/deg (LATERAL, LIMIT 300). Frontier ids are inlined as an ARRAY[...]::int[] literal, not bound. Each branch uses OFFSET 0 as an optimization fence. This is not a recursive CTE: the multi-hop loop is in the caller (runRelationalChain). PGLite pglite-engine.ts:2039: readChainHop(this.executeRaw.bind(this), frontierPageIds, opts). Postgres postgres-engine.ts:1859: same, plus chunkManifestReads. Equivalent on PGLite. relationalChainHop: about 10 files. Parity: e2e/engine-parity.test.ts:1409-1486 (multi-hop chain identical across engines). relational-recall.test.ts, helpers/amends-ranking-scenarios.ts, relational-chain-hub.slow.test.ts.
38 getBacklinkCounts (1717) (pageIds: number[], opts?: PageReadScope): Map<number, number> Distinct linking pages per target page id, for the hybrid-search backlink boost. Mentions and non-ranking types excluded. Ids with zero links map to 0. Keyed by page id (#4380). search/read-enrichment.ts:243-263: SELECT p.id AS page_id, COUNT(DISTINCT l.from_page_id)::int AS cnt FROM pages p LEFT JOIN links l ON l.to_page_id = p.id AND l.from_page_id <> p.id AND link_source IS DISTINCT FROM 'mentions' AND rankingLinkSql('l') [AND EXISTS contributor AND origin] WHERE p.id = ANY($1::int[]) AND <target filter> GROUP BY p.id. Single query. PGLite pglite-engine.ts:2043: readBacklinkCounts(this.executeRaw.bind(this), ...). Postgres postgres-engine.ts:1863: wraps the same call in jitOffRead with tx.unsafe(query, params, { prepare: true }) (JIT off, because compile time dominated at 50k pages, per the inline comment). getBacklinkCounts: about 19 files. Parity: e2e/engine-parity.test.ts (around the boost tests). e2e/jit-off-reads-postgres.test.ts, search/post-fusion-concurrent-reads.test.ts, search/metadata-boost-gate.test.ts, e2e/read-enrichment-privacy.test.ts, helpers/postgres-engine-sql-cases.ts.
39 getAdjacencyBoosts (1744) (pageIds: number[], opts?: PageReadScope): Map<number, {hits, cross_source_hits}> For a page-id set, returns in-set hits (distinct from-pages) and cross_source_hits (distinct other source ids), restricted to the induced subgraph. Returns only pages with hits >= 1. search/read-enrichment.ts:265-288: SELECT l.to_page_id, COUNT(DISTINCT l.from_page_id)::int AS hits, COUNT(DISTINCT CASE WHEN p.source_id <> t.source_id THEN p.source_id END)::int AS cross_source_hits FROM links l JOIN pages p ... JOIN pages t ... WHERE l.from_page_id = ANY(ARRAY[...]::int[]) AND l.to_page_id = ANY(ARRAY[...]::int[]) AND ... GROUP BY l.to_page_id HAVING COUNT(DISTINCT l.from_page_id) >= 1. Ids are inlined as an ARRAY[...]::int[] literal, not a bound parameter, to keep the planner off a generic plan (search/adjacency-generic-plan.test.ts). PGLite pglite-engine.ts:2047 and Postgres postgres-engine.ts:1869, both via readAdjacencyBoosts(this.executeRaw.bind(this), ...). Same SQL, no transaction wrapper on either. getAdjacencyBoosts: about 12 files. search/adjacency-generic-plan.test.ts, e2e/graph-signals-engine.test.ts, helpers/amends-ranking-scenarios.ts, helpers/postgres-engine-sql-cases.ts, search/post-fusion-concurrent-reads.test.ts. No parity test.
40 getContentFlagsByPageIds (1753) (pageIds: number[], opts?: PageReadScope): Map<number, {reason, detail}> frontmatter.content_flag per page (#1699). Post-fusion warning channel for search results. search/read-enrichment.ts:290-299: SELECT p.id, p.frontmatter->'content_flag'->>'reason', ...->>'detail' FROM pages p WHERE p.id = ANY($1::int[]) AND p.frontmatter ? 'content_flag' AND <filter>. PGLite pglite-engine.ts:2051, Postgres postgres-engine.ts:1873. Same readContentFlags. getContentFlagsByPageIds: about 10 files. e2e/read-enrichment-privacy.test.ts, helpers/postgres-engine-sql-cases.ts, quarantine-cli.test.ts, hybrid-arm-rethrow.test.ts. No parity test.
41 getUnverifiedExtractionPageIds (1769) (pageIds: number[], opts?: PageReadScope): Map<number, {unverified, status}> For every page with frontmatter.status, the raw status and whether it is an unverified auto-extracted stub (#160, #4220). search/read-enrichment.ts:301-309: SELECT p.id, p.frontmatter->>'status', (<unverifiedExtractionFragment('p')>) AS unverified FROM pages p WHERE p.id = ANY($1::int[]) AND p.frontmatter->>'status' IS NOT NULL AND <filter>. The predicate is unverifiedExtractionFragment (src/core/extraction-review.ts). PGLite pglite-engine.ts:2055, Postgres postgres-engine.ts:1877. Same SQL. getUnverifiedExtractionPageIds: about 11 files. e2e/extraction-review-postgres.test.ts, extraction-review.test.ts, helpers/postgres-engine-sql-cases.ts, hybrid-arm-rethrow.test.ts. No parity test.
42 getPageTimestamps (1781, @deprecated v0.29.1) (slugs: string[]): Map<string, Date> updated_at (or created_at) per slug. Kept for PR #618 callers. Not source-qualified. pages.ts:1033: SELECT slug, COALESCE(updated_at, created_at) ... WHERE slug = ANY($). Both: unscopedExecutor(this.engineSql, 'pages: unscoped on master (EO4 inventory)') (pglite-engine.ts:2059, postgres-engine.ts:1881). Same SQL. getPageTimestamps: about 10 files. title-fuzzy-index.test.ts, helpers/postgres-engine-sql-cases.ts, engine-surface-coverage.test.ts. No parity test.
43 getEffectiveDates (1791) (refs: {slug, source_id}[], opts?: PageReadScope): Map<'source::slug', Date> COALESCE(effective_date, updated_at, created_at) per (slug, source_id). Drives the recency boost (v0.29.1). search/read-enrichment.ts:311-320: SELECT p.slug, p.source_id, COALESCE(p.effective_date, p.updated_at, p.created_at) AS ts FROM pages p JOIN unnest($1::text[], $2::text[]) AS u(slug, source_id) ON p.slug = u.slug AND p.source_id = u.source_id WHERE <filter>. Two-array unnest join. Key format is ${source_id}::${slug}. PGLite pglite-engine.ts:2063, Postgres postgres-engine.ts:1885. Same SQL. getEffectiveDates: about 11 files. e2e/read-enrichment-privacy.test.ts, hybrid-recency-config.test.ts, search/metadata-boost-gate.test.ts, search/post-fusion-concurrent-reads.test.ts, search.test.ts. No parity test.
44 getSalienceScores (1801) (refs: {slug, source_id}[], opts?: PageReadPolicy): Map<'source::slug', number> Salience per (slug, source_id) = COALESCE(emotional_weight,0)*5 + ln(1 + count(distinct active takes)). With a takesHoldersAllowList, the emotional term is 0 and only takes from allowed holders count (search/read-enrichment.ts:322-345). search/read-enrichment.ts:322-345: SELECT p.slug, p.source_id, (COALESCE(p.emotional_weight,0)*5 + ln(1 + COUNT(DISTINCT t.id))) AS score FROM pages p JOIN unnest($1::text[], $2::text[]) AS u(...) ON ... LEFT JOIN takes t ON t.page_id = p.id AND t.active = TRUE [AND t.holder = ANY($::text[])] WHERE <filter> GROUP BY p.id. PGLite pglite-engine.ts:2067, Postgres postgres-engine.ts:1889. Same SQL. The salience.ts file (getRecentSalience, listEnrichCandidates, findAnomalies) is a separate, non-interface path. getSalienceScores: about 11 files. Parity: e2e/engine-parity-salience.test.ts (dedicated parity file). Also e2e/read-enrichment-privacy.test.ts, search/metadata-boost-gate.test.ts, search.test.ts.
45 findOrphanPages (1830) `(opts?: PageReadScope & {sourceId?, sourceIds?, mode?: 'islanded' 'inbound', scopeInbound?}): {slug, title, domain, type?, quarantined?, source_id?}[]` Pages with no live inbound link (inbound mode) or with neither inbound nor live outbound (islanded, the default, #4524). Candidate set scoped by source; inbound counts from any source unless scopeInbound. links.ts:528-598: SELECT p.slug, COALESCE(p.title, p.slug), p.frontmatter->>'domain', p.type, (NOT <QUARANTINE>) AS quarantined, p.source_id FROM pages p WHERE p.deleted_at IS NULL [AND p.source_id = $] AND NOT EXISTS (SELECT 1 FROM links l JOIN pages src ... WHERE l.to_page_id = p.id AND src.deleted_at IS NULL [sourceScope] [liveOriginFilter]) [islanded: AND NOT EXISTS (outbound to live target)] ORDER BY p.slug. PGLite pglite-engine.ts:2071: wrapped in plannerRead(...). Postgres postgres-engine.ts:1893: not wrapped. Both pass unscopedExecutor(...). Same SQL.

Notes for section D:

  • Recursive CTEs: traverseGraph (links.ts:659), traversePathsDetailed (links.ts:711) and relationalFanout (read-enrichment.ts:19). The first two use the array-frontier BFS built by bfsWalkCtes (links.ts:609). The third walks paths with path and visited arrays and clamps depth to 3. relationalChainHop, getBacklinkCounts, getAdjacencyBoosts and the rest are not recursive.
  • pgvector: none in this section. getAdjacencyBoosts, getBacklinkCounts and relationalChainHop run on links/pages joins only. Canonical chunk ids come from content_chunks through a correlated LIMIT 1 subquery.
  • Batch unnest patterns: getEffectiveDates and getSalienceScores join unnest(text[], text[]) on (slug, source_id). traverseGraph/traversePaths use unnest(bfs.frontier) and unnest(bfs.visited) in array CTEs. relationalChainHop uses unnest on an inlined id array. Other bulk reads use ANY($n::int[]).
  • Read-policy seam: every read here authorizes endpoints, hops and origins before returning (pageReadFilter, originFilter in read-enrichment.ts:11-16). Scoping by sourceId/sourceIds is applied by the same helper in links.ts (sourceScope, links.ts:652).

Test coverage summary

  • Parity (both engines, DATABASE_URL-gated, runs PGLite half always): test/e2e/engine-parity.test.ts. Covers: addLink, getChunks, listLinkSources, findOrphanPages, the stale-extraction group (countStalePagesForExtraction, listStalePagesForExtraction, markPagesExtractedBatch), countChunklessPagesWithContent/listChunklessPagesWithContent, relationalFanout, relationalChainHop, getLinks, getBacklinks, traverseGraph, traversePaths, traversePathsDetailed, removeLinksByPagesAndSource, listStalePagesForExtraction.
  • test/e2e/engine-parity-salience.test.ts covers getSalienceScores.
  • test/pglite-engine.test.ts covers PGLite-only behavior: upsertChunks round trip and mixed-case slugs (519-560), deleteChunks (565-572), countStaleChunks (597-703), removeLink, getChunks.
  • No parity or PGLite test directly names these methods: getChunkPage (only get-chunks-paging.test.ts), getChunkWindows (evidence tests, Postgres RLS scope), sumStaleChunkChars, setPageEmbeddingSignature, invalidateStaleSignatureEmbeddings, invalidateContentDriftEmbeddings, readEmbeddingStateCounts, listEmbeddingStateRows, markPagesAttendanceBlocked, countStaleFrontmatterKeyPages, listStaleFrontmatterKeyPages, stripStaleFrontmatterKeys, replaceDerivedLinks, replaceDerivedLinksBatch, findByTitleFuzzy, getAdjacencyBoosts, getContentFlagsByPageIds, getUnverifiedExtractionPageIds, getPageTimestamps, getEffectiveDates, getBacklinkCounts (the last is covered by e2e/jit-off-reads-postgres.test.ts). Several of these are only exercised through test/helpers/postgres-engine-sql-cases.ts and test/engine-surface-coverage.test.ts (inventory-style tests).

Parity risks noted

  1. Planner wrapper asymmetry. PGLite wraps getBacklinks, traverseGraph, traversePathsDetailed, traversePaths' inner call and findOrphanPages in plannerRead (pglite-engine.ts:1992, 2015, 2029, 2077), which can run planner-stat refreshes. Postgres does not. A Rust engine has no equivalent, so timing and plan behavior differ and nothing in the tests pins it.
  2. Transaction and RLS scope asymmetry. Postgres wraps scoped reads in withScopedReadTransaction (postgres-engine.ts:368), which sets the source/RLS scope. PGLite runs the same SQL on the bare connection. The source-scope predicates are in SQL in both, so behavior matches, but a Rust engine must enforce the same scope in its own read path.
  3. JIT off on Postgres only. getBacklinkCounts runs in jitOffRead on Postgres (postgres-engine.ts:1863). PGLite has no JIT, so this is a performance-only divergence, but the inline comment says plan compilation dominated at 50k pages.
  4. Generic-plan sensitivity. getAdjacencyBoosts and relationalChainHop inline integer id arrays and use OFFSET 0 fences, to keep the planner from choosing a generic plan (search/read-enrichment.ts:117-130, 265-288). A Rust engine has no plan cache, so these are not needed, but the output ordering they rely on must hold.
  5. Traversal caps. TRAVERSE_WALK_ROW_CAP = 20000 and TRAVERSE_PATH_ROW_CAP = 5000 (engine-constants.ts:37, 48) bound the walk, and traversePathsDetailed reports truncated. The cap counts raw rows before dedup, so a Rust version must apply the cap at the same point or the truncated flag changes.
  6. Traversal ordering. traversePaths orders by depth, from_slug, to_slug, from_source_id, to_source_id. traverseGraph orders by depth, slug, source_id. relationalFanout orders by hop ASC, edge_count DESC, source_id, slug. Ties and collation (ORDER BY slug) must match on both engines.
  7. Stale addLinksBatch JSDoc. engine.ts:1507-1511 says "INSERT...SELECT FROM (VALUES)". The implementation is jsonb_to_recordset (links.ts:216-227, #1861). The doc is wrong and should not be copied.
  8. getEffectiveDates JSDoc contradicts itself. engine.ts:1788-1790 says "Returns NULL for refs with no row; map omits them." The implementation omits them (read-enrichment.ts:311-320). Callers must treat a missing key as absent.
  9. Stacked JSDoc on findOrphanPages. engine.ts:1802-1829 has two JSDoc blocks in sequence; the mode semantics (#4524) sit above a separate block. The excludePrivate option is accepted by both implementations but missing from the interface type (engine.ts:1830-1834).
  10. Unscoped master reads. sumStaleChunkChars, findByTitleFuzzy, traverseGraph, traversePathsDetailed, findOrphanPages, getPageTimestamps use unscopedExecutor(..., 'EO4 inventory') on both engines. Their scope comes from the caller's options, not the engine. Any Rust read path that enforces scope centrally would change these results.
  11. Segmented-body chunk visibility. getChunks, getChunkPage, getChunkWindows, countStaleChunks, listStaleChunks and relationalFanout all gate on chunkGenerationFilter / chunkGenerationProbe (search/safe-chunks.ts:77-89). It reads page_body_manifest (the segmented-body manifest), and the flag chunkManifest only changes behavior on Postgres (safe-chunks.ts:107-113). Rust must replicate the manifest semantics exactly, or segmented pages will read differently.
  12. Content-drift sweep has no saved cursor. invalidateContentDriftEmbeddings loops in 5000-row transactions with no saved cursor (chunks.ts:612, 685-698). A crash mid-loop re-examines rows, which is safe by design (the predicate is idempotent) but the batch boundary is observable in the returned count.
  13. Upsert is retried as a whole. upsertChunks and addLinksBatch are wrapped in batchRetry (pglite-engine.ts:1814-1815, 1950; postgres-engine.ts:1628-1629, 1770). The engine doc says "callers MUST NOT wrap externally." A Rust engine that retries differently changes the commit-ambiguous replay story (engine.ts:1248-1253, 1512-1520).
  14. Embedding column routing. Every chunk read and stale count resolves the active column by a registry lookup (resolveActiveEmbeddingColumnFromEngine, search/embedding-column.ts:694), and the write path resolves it again from config rows inside the upsert. A fallback to the legacy embedding column on a broken registry (fallbackToLegacy: true) is a silent behavior change for diagnostics and reads.
  15. Halfvec and dimension casts. upsertChunks writes ::halfvec(N) or ::vector depending on the resolved column (search/embedding-column.ts:279-283). A Rust store must keep the same dimension and type rules, or writes to a registry-routed brain fail with dimension mismatch errors.

Tags, Timeline, Chronicle, Ontology, Raw data, Files, Takes, Dream and Contradictions

Scope: the 43 BrainEngine methods declared in src/core/engine.ts:1842-2195 (interface). No method in this range is optional (no name?() declarations). Paths below are relative to the gbrain repo root the gbrain repo root. Line numbers are file:line in that repo.

How delegation works in this range:

  • Interface: src/core/engine.ts:1842-2194.
  • Engine methods are implemented twice, once in src/core/pglite-engine.ts and once in src/core/postgres-engine.ts. Most methods are thin wrappers that call src/core/engine-sql/*.ts, which hold one shared SQL text for both engines (engine-sql/takes.ts:1-9, timeline.ts:1-11, tags.ts:1-7, files.ts:1-7).
  • Two families are not in engine-sql and are inlined per engine: ontology (mergeOntologyFact, getOntology, discoverOntologyDimensions, findOntologyConflicts), putRawData/getRawData, and the dream verdict cache. Their SQL text is written twice, once with db.query($n) in PGLite and once with postgres.js tagged templates in Postgres.
  • Reads in engine-sql take a LegacyUnscopedRead brand (engine-sql/brands.ts:33), meaning master ran them on the pool with no RLS scope transaction. Both engines obtain it via unscopedExecutor(...) (brands.ts:49).
  • Dialect executors: engine-sql/dialect-pglite.ts:58-76 and engine-sql/dialect-postgres.ts:68-94. PGLite binds JSONB params with JSON.stringify (dialect-pglite.ts:33). Postgres binds them with conn.json (dialect-postgres.ts:61).

Shared helpers referenced below:

  • mutatePageTag in src/core/page-state/tags.ts:5-22.
  • Ontology helpers in src/core/chronicle/ontology.ts: valueHash (:12), normalizeDimension (:32), isNovelDimension (:43), isBackdatedObservation (:54), ONTOLOGY_SUPERSEDE_GUARD (:65).
  • deletedPageProjectionHiddenSql in src/core/eligibility/sql.ts:55.
  • privatePagesFilterForCaller / privateFactFilterForCaller in src/core/search/private-visibility.ts:611 and :992.
  • buildTakeRows / buildTimelineRows in src/core/batch-rows.ts:177 and :161.
  • finalizeScorecard / deriveResolutionTuple in src/core/takes-resolution.ts:90 and :22.
  • finalizeLastSeen in src/core/chronicle/last-seen.ts:11.
  • DREAM_VERDICT_TTL_SECONDS = 30 * 86400 at src/core/engine.ts:528.

Engine line map (PGLite / Postgres): every method is declared in both engine files, so there are no PGLite-only or Postgres-only methods in this range.

Tags

Method (iface line) Signature What it does SQL / impl
addTag (1851) addTag(slug, tag, opts?: {sourceId?, tagSource?: 'frontmatter'}): Promise<void> Adds a tag to a page. Idempotent. An explicit add claims the row. PGLite pglite-engine.ts:2081, Postgres postgres-engine.ts:1903. Both call mutatePageTag in one transaction. Inside it: lockPageKeys, then SELECT id FROM pages WHERE source_id=$1 AND slug=$2. Missing page throws addTag failed: page ... not found. Then INSERT INTO tags(page_id,tag,tag_source) VALUES(...,'added') ON CONFLICT (page_id,tag) DO UPDATE SET tag_source='added'. With tagSource:'frontmatter' it inserts 'frontmatter' and updates only WHERE tags.tag_source IS NULL (page-state/tags.ts:13-17).
removeTag (1852) removeTag(slug, tag, opts?: {sourceId?}): Promise<void> Removes one tag. Same mutatePageTag path with add=false (page-state/tags.ts:18-20): DELETE FROM tags WHERE page_id=(SELECT id FROM pages WHERE source_id=$1 AND slug=$2) AND tag=$3. No existence check, so a missing page or tag is a silent no-op.
getTags (1859) getTags(slug, opts?: {sourceId?, sourceIds?, excludePrivate?, liveOnly?}): Promise<string[]> Returns sorted distinct tag names for a slug. sourceIds[] wins over sourceId. Unscoped reads default to source 'default'. engine-sql/tags.ts:12-34. SELECT DISTINCT tag FROM tags WHERE page_id IN (SELECT id FROM pages WHERE slug=$ AND <scope> <privacy> <live>) ORDER BY tag. Scope is source_id = ANY($::text[]) or source_id = $. IN (not =) is used so a federated slug in several sources does not throw. excludePrivate adds privatePagesFilterForCaller('pages'). liveOnly adds deleted_at IS NULL. Read is LegacyUnscopedRead (tags.ts:6).

Parity: both engines call tagsImpl.getTags with the same SQL. addTag/removeTag are the same shared function. The dialect difference is only the lock call. PGLITE_CAPABILITIES.transactionAdvisoryLocks=false (dialect-pglite.ts:27) and Postgres sets it to true (dialect-postgres.ts:55). I did not trace what lockPageKeys does on each dialect.

Timeline

Method (iface line) Signature What it does SQL / impl
addTimelineEntry (1880) addTimelineEntry(slug, entry: TimelineInput, opts?: {skipExistenceCheck?, sourceId?}): Promise<boolean> Inserts one timeline row. Returns true if inserted, false if deduped. Throws PageMissingError when the page is missing and skipExistenceCheck is not set. PGLite pglite-engine.ts:2094, Postgres postgres-engine.ts:1916, both call engine-sql/timeline.ts:28-67. One CTE: page_state selects pages by (slug, source_id) FOR KEY SHARE, then inserted does INSERT INTO timeline_entries(page_id,date,source,summary,detail) ... ON CONFLICT (page_id,date,md5(summary),source) DO NOTHING RETURNING 1. Free-text fields go through sanitizeForJsonb. Dedup index is idx_timeline_dedup (schema.sql:670).
addTimelineEntriesBatch (1897) addTimelineEntriesBatch(entries: TimelineBatchInput[], opts?: BatchOpts): Promise<number> Bulk insert. Returns count of rows actually inserted. Empty input returns 0 with no query. Both engines: batchRetry(...) wrapper (pglite-engine.ts:2102, postgres-engine.ts:1924). Impl timeline.ts:70-90: executeRawJsonb over jsonb_to_recordset(($1::jsonb)->'rows') AS v(slug,date,source,summary,detail,source_id), JOIN pages p ON p.slug=v.slug AND p.source_id=v.source_id AND p.deleted_at IS NULL, ON CONFLICT (page_id,date,md5(summary),source) DO NOTHING RETURNING 1. Rows built by buildTimelineRows (batch-rows.ts:161).
getTimeline (1898) getTimeline(slug, opts?: TimelineOpts): Promise<TimelineEntry[]> Entries for a slug, newest first. Supports after, before, limit (default 100), excludePrivate, eligibility, sourceIds/sourceId. timeline.ts:93-126. FROM timeline_entries te JOIN pages p ON p.id=te.page_id when scoped. When unscoped it uses sources s CROSS JOIN LATERAL (SELECT * FROM pages WHERE source_id=s.id AND slug=$ OFFSET 0) p JOIN timeline_entries te (:114-118). The OFFSET 0 fences the planner from driving off pages (comment cites Cat7-1). Ordering: te.date DESC, te.id DESC LIMIT n. Read is LegacyUnscopedRead.

Parity: both engines share engine-sql/timeline.ts. The addTimelineEntry CTE with FOR KEY SHARE and the getTimeline LATERAL are Postgres planner or locking features. PGLite runs them on its single-connection WASM Postgres, so the same SQL text is used and the locking is not exercised the same way.

Life Chronicle (timeline reads)

All five filter pages.deleted_at IS NULL on the depth page and hide soft-deleted event projections. The shared shape is in timeline.ts:129-143 (chronicleSourceCond). It applies a source scope: sourceIds first, then sourceId, applied to p or to the event alias ep (ep=true). excludePrivate adds privatePagesFilterForCaller('p') and privateTimelineEventFilterForCaller('te').

Method (iface line) Signature What it does SQL / impl
getTimelineForDate (1904) getTimelineForDate(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]> Rows for one day, or the ISO week when opts.week. Limit default 200. timeline.ts:146-166. Week bounds are date_trunc('week', d)::date through +6 days, else d::date. Joins timeline_entries te JOIN pages p ... AND p.deleted_at IS NULL LEFT JOIN pages ep ON ep.id=te.event_page_id. Filter (te.event_page_id IS NULL OR ep.deleted_at IS NULL). Order COALESCE(ep.effective_date, te.date::timestamptz) ASC, te.id ASC. Returns kind from ep.frontmatter->'event'->>'kind'.
getSince (1906) getSince(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]> Rows on or after date. Optional kind filter. Limit 200. timeline.ts:169-188. Same join and order as above. Adds te.date >= $date::date and an optional ep.frontmatter->'event'->>'kind' = $kind.
getOnThisDay (1908) getOnThisDay(opts?: PageReadScope & {date?, limit?}): Promise<ChronicleTimelineRow[]> Same month and day as date (default today) in prior years only. Limit 50. timeline.ts:191-211. EXTRACT(MONTH/DAY FROM te.date) = EXTRACT(... FROM target), te.date < target. target is $date::date or current_date (DB session clock). Order te.date DESC, te.id ASC.
getLastSeen (1910) getLastSeen(entitySlug, opts?: PageReadScope & {asof?}): Promise<LastSeenResult> Most recent date an entity is seen: its own page's timeline, or an event whose who array names it exactly or by wikilink ([[slug]], `[[slug label]]`). Substring matches are excluded.
upsertEventProjection (1917) upsertEventProjection(opts: {depthSlug, eventSlug, date, summary, detail?, sourceId?}): Promise<{projected: boolean}> Upserts the date-index row for an event page, keyed (event_page_id, date). Returns projected=false when either slug is missing. timeline.ts:256-269. `INSERT INTO timeline_entries(page_id,date,source,summary,detail,event_page_id) SELECT dp.id, $date::date, 'life-chronicle:event:'

Parity for this family: PGLite (pglite-engine.ts:2111-2131) and Postgres (postgres-engine.ts:1933-1952) both call timelineImpl with the same unscoped executor. Both getOnThisDay and getLastSeen depend on the database session date (current_date), not on the process clock. getLastSeen's days_ago uses the JS clock. Parity risk is listed at the end.

Ontology (rides facts)

Ontology is inlined per engine, not in engine-sql. The shared logic is in chronicle/ontology.ts. The facts table is defined elsewhere in this research set. Here the relevant columns are source_id, entity_slug, dimension, value, value_hash, dim_status, confidence, source_markdown_slug, valid_from, valid_until, expired_at, superseded_by, consolidated_into, visibility, trust_tier.

Method (iface line) Signature What it does SQL / impl
mergeOntologyFact (1928) mergeOntologyFact(obs: OntologyObservationInput): Promise<OntologyMergeResult> Records one observation (entity has dimension=value). Returns action of noop, corroborated, superseded_prior or inserted. Defaults: confidence 0.7, visibility private, status quarantined for novel dimensions, else active. PGLite pglite-engine.ts:2131-2198 (this.db.query). Postgres postgres-engine.ts:1953-2023 (tagged templates). Steps: (1) normalize dimension, valueHash (ontology.ts:12-32); (2) find current open row: expired_at IS NULL AND valid_until IS NULL AND (dim_status IS NULL OR dim_status='active'), ORDER BY valid_from DESC NULLS LAST, confidence DESC, id DESC LIMIT 1; (3a) same value and not backdated: check for an existing row from the same provenance (source_markdown_slug) at or after current valid_from; if found, noop; else insert a row with expired_at=now() and consolidated_into=current.id, ON CONFLICT (source_id,entity_slug,dimension,value_hash,source_markdown_slug,valid_from) WHERE dimension IS NOT NULL DO NOTHING RETURNING id, giving corroborated; (3b) otherwise insert a new row, then if current is open and status is active and the new valid_from is not earlier, close the prior row with UPDATE facts SET valid_until=COALESCE($validFrom, now()), superseded_by=$new WHERE id=$cur AND valid_until IS NULL + ONTOLOGY_SUPERSEDE_GUARD, giving superseded_prior, else inserted. A backdated conflicting value is inserted without touching the prior row.
getOntology (1930) getOntology(entitySlug, opts?: OntologyReadOpts): Promise<OntologyValue[]> Current resolved value per dimension at asof (default now). Honors minConfidence (default 0), includeQuarantined, sourceId/sourceIds, visibility[], excludePrivate. PGLite pglite-engine.ts:2200-2227, Postgres postgres-engine.ts:2025-2050. SELECT DISTINCT ON (dimension) dimension, value, confidence, source_markdown_slug AS source, valid_from, valid_until AS valid_to, COALESCE(dim_status,'active') AS status, id AS fact_id FROM facts WHERE entity_slug=$ AND dimension IS NOT NULL AND expired_at IS NULL <scope> <deletedPageHidden> [<privateFactFilter>] <visibility> AND valid_from <= asof AND valid_until > asof AND confidence >= $min AND ($includeQ OR dim_status IS NULL OR dim_status='active') ORDER BY dimension, valid_from DESC NULLS LAST, confidence DESC, id DESC. Page-visibility gate runs before DISTINCT ON (comment at postgres-engine.ts:2033).
discoverOntologyDimensions (1932) discoverOntologyDimensions(opts?: {sourceId?, sourceIds?}): Promise<OntologyDimensionStat[]> Lists every dimension in the brain with entity and observation counts. PGLite pglite-engine.ts:2229-2244, Postgres postgres-engine.ts:2052-2063. SELECT dimension, count(DISTINCT entity_slug)::int AS entities, count(*)::int AS observations FROM facts WHERE dimension IS NOT NULL AND expired_at IS NULL <scope> GROUP BY dimension ORDER BY entities DESC, dimension. No privacy filter in this query.
findOntologyConflicts (1934) findOntologyConflicts(opts?: PageReadScope & {minConfidence?, visibility?}): Promise<OntologyConflict[]> Dimensions with at least 2 distinct current values from at least 2 distinct provenances. PGLite pglite-engine.ts:2246-2270, Postgres postgres-engine.ts:2065-2091. CTE cur selects open rows (expired_at IS NULL AND valid_until IS NULL AND dim_status active, confidence >= $min, scope, page gates, visibility). Then GROUP BY entity_slug, dimension HAVING count(DISTINCT value) >= 2 AND count(DISTINCT source) >= 2, returning json_agg(json_build_object('value',..,'source',..,'confidence',..,'fact_id',..)).

Parity: the logic is identical. Driver differences only. PGLite builds params as an array with $n placeholders and a dynamic scope. Postgres uses sql\...`fragments andsql.unsafe()for the privacy fragment. PGLite parsesjson_aggtext withJSON.parse when needed (pglite-engine.ts:2272`).

Raw data

Method (iface line) Signature What it does SQL / impl
putRawData (1943) putRawData(slug, source, data: object, opts?: {sourceId?}): Promise<void> Upserts one raw_data sidecar row per (page, source). Throws if the page is missing. PGLite pglite-engine.ts:2277-2320, Postgres postgres-engine.ts:2093-2128. With sourceId: INSERT INTO raw_data(page_id,source,data) SELECT id,$2,$3::jsonb FROM pages WHERE slug=$1 AND source_id=$4 ON CONFLICT (page_id,source) DO UPDATE SET data=EXCLUDED.data, fetched_at=now() RETURNING id. Without sourceId: the same without the source filter (matches every same-slug page; comment at postgres-engine.ts:2101, pglite-engine.ts:2284). Zero rows throws putRawData failed: page ... not found. The two engines differ only in JSON binding: PGLite JSON.stringify with $3::jsonb, Postgres sql.json.
getRawData (1949) getRawData(slug, source?, opts?: PageReadScope & {includeDeleted?}): Promise<RawData[]> Returns {source, data, fetched_at} rows for a slug. Hides rows of soft-deleted pages unless includeDeleted. PGLite pglite-engine.ts:2322-2352 builds a dynamic WHERE. Postgres postgres-engine.ts:2130-2168 has explicit branches. Both produce FROM raw_data rd JOIN pages p ON p.id=rd.page_id WHERE p.slug=$1 [AND rd.source=$] [AND p.deleted_at IS NULL] [AND p.source_id = ANY/=] [AND privatePagesFilter]. sourceIds[] wins over sourceId. Read is unscoped on master, not LegacyUnscopedRead in the engine (this method has no engine-sql home).

Files

Files are binary asset metadata only. Image bytes never enter the DB. Shared SQL is in engine-sql/files.ts, used by both engines. Schema: files (schema.sql:1136-1149), UNIQUE(storage_path).

Method (iface line) Signature What it does SQL / impl
upsertFile (1954) upsertFile(spec: FileSpec): Promise<{id, created}> Upserts a file row keyed on storage_path. created is true on insert. files.ts:50-68. INSERT INTO files(source_id,page_slug,page_id,filename,storage_path,mime_type,size_bytes,content_hash,metadata) VALUES(...) ON CONFLICT (storage_path) DO UPDATE SET page_slug, page_id, filename, mime_type, size_bytes, content_hash, metadata = EXCLUDED.* RETURNING id, (xmax = 0) AS created. source_id defaults to 'default', metadata to {} via jsonbParam. The DO UPDATE list does not touch source_id.
getFile (1955) getFile(sourceId, storagePath): Promise<FileRow | null> Looks up one file by source and path. files.ts:70-78. SELECT ... FROM files WHERE source_id=$1 AND storage_path=$2 LIMIT 1. Read is LegacyUnscopedRead.
listFilesForPage (1956) listFilesForPage(pageId: number): Promise<FileRow[]> Files attached to a page. files.ts:80-88. SELECT ... FROM files WHERE page_id=$1 ORDER BY created_at ASC.

Both engines: pglite-engine.ts:2354-2364 and postgres-engine.ts:2170-2180 delegate to filesImpl. Engine-level difference: none in SQL. The write path uses this.engineSql on both.

Takes (typed, weighted, attributed claims)

Takes live in table takes, keyed (page_id, row_num), with active, superseded_by, resolved_* columns and optional embedding (pgvector) plus embedding_model, embedded_text_hash, embedded_at. Takes carry no source_id; source scope goes through the page join. Engine-level methods are one-liners that delegate to engine-sql/takes.ts for both engines (PGLite pglite-engine.ts:2571-2706, Postgres postgres-engine.ts:2430-2562). The addTakesBatch / updateTakeEmbeddings wrappers pass batchRetry to the impl.

Method (iface line) Signature What it does SQL / impl (engine-sql/takes.ts)
addTakesBatch (1978) addTakesBatch(rows: TakeBatchInput[], opts?: BatchOpts): Promise<number> Bulk insert or update takes. Returns count of rows inserted, updated, or already current. Weight is clamped to [0,1] and rounded, with a TAKES_WEIGHT_CLAMPED stderr line. :47-102. buildTakeRows (batch-rows.ts:177) clamps weight and NUL-strips claim and source. Then executeRawJsonb: INSERT INTO takes(...) SELECT ... FROM jsonb_to_recordset(($1::jsonb)->'rows') AS v(...) ON CONFLICT (page_id,row_num) DO UPDATE SET embedding/embedded_at/embedding_model/embedded_text_hash = NULL when claim changes, claim, kind, holder, weight, since_date, until_date, source, superseded_by, active, updated_at=now() WHERE (...) IS DISTINCT FROM (...). A row that is already current is not written. An intra-batch duplicate (page_id,row_num) fails the statement (:62-63).
updateTakeEmbeddings (1981) updateTakeEmbeddings(rows: TakeEmbeddingInput[], opts?: BatchOpts): Promise<number> Stores vectors for active takes. Vectors whose claim changed in flight are discarded. Returns count updated. :458-510. Validates take_id (positive int, no duplicates), claim and model strings, finite floats. UPDATE takes AS t SET embedding=v.embedding::vector, embedded_at=now(), embedding_model=v.model, embedded_text_hash=md5(v.claim), updated_at=now() FROM jsonb_to_recordset(...) AS v(take_id bigint, embedding text, claim text, model text) WHERE t.id=v.take_id AND t.active AND t.claim=v.claim RETURNING t.id.
listTakes (1984) listTakes(opts?: TakesListOpts): Promise<Take[]> Filtered list of takes with page_slug. Default active=true, limit 100 (max 500), offset. :294-341. FROM takes t JOIN pages p. Filters: page_id, page_slug, holder, kind, active (NULL means any), resolved (true/false/undefined), takesHoldersAllowList (holder = ANY), sinceOnOrBefore (text compare with partial-date padding), excludePrivate, sourceIds/sourceId (p.source_id), eligibility (projectionEligibleSql). Sort: ungradedFirst (EXISTS in take_grade_cache), then sortBy in since_date / weight / created_at (default), with oldestFirst, tie-break t.id DESC.
searchTakes (1991) searchTakes(query, opts?): Promise<TakeHit[]> Keyword (trigram) search over active take claims. Limit 30 (max 100). :343-371. word_similarity(query, t.claim)::real AS score FROM takes t JOIN pages p WHERE t.active AND $query <% t.claim [privacy] [holder allow-list] [source scope] [eligibility] ORDER BY score DESC, t.weight DESC LIMIT n. Uses the pg_trgm <% operator. Rows go through takeHitRowToHit (BigInt coercion).
searchTakesVector (1997) searchTakesVector(embedding: Float32Array, opts?): Promise<TakeHit[]> Cosine search over stored take vectors. Limit 30 (max 100). :373-406. Vector is rendered as [v1,v2,...]. SELECT ..., (1 - (t.embedding <=> $vec::vector))::real AS score FROM takes t JOIN pages p WHERE t.active AND t.embedding IS NOT NULL [privacy] [holder] [source] [eligibility] ORDER BY t.embedding <=> $vec::vector LIMIT n. Rows with no embedding simply do not match, so the result is [] when none are embedded. No dimension check in SQL.
getTakeEmbeddings (2003) getTakeEmbeddings(ids: number[]): Promise<Map<number, Float32Array>> Vectors by take id. Mirrors getEmbeddingsByChunkIds. :408-419. SELECT id, embedding FROM takes WHERE id = ANY($ids::bigint[]) AND embedding IS NOT NULL. Parsed with tryParseEmbedding. Returns an empty Map for empty input.
countStaleTakes (2006) countStaleTakes(opts?: StaleTakeOpts): Promise<number> Count of stale takes (missing vector, claim drift, or other model or width when the caller names a target). :427-444. Shared predicate staleTakeWhere (:427-436): t.active AND p.deleted_at IS NULL AND EXISTS(sources s WHERE s.id=p.source_id AND NOT s.archived) AND (sourceId filter) AND (t.embedding IS NULL OR t.embedded_text_hash IS DISTINCT FROM md5(t.claim) OR (model given AND embedding_model IS DISTINCT FROM model) OR (dims given AND vector_dims(embedding) <> dims)). Read is LegacyUnscopedRead.
listStaleTakes (2009) listStaleTakes(opts?: StaleTakeOpts): Promise<StaleTakeRow[]> Same predicate, returns take_id, page_slug, row_num, claim. No vector in payload. :446-456. Same predicate, ORDER BY t.id LIMIT 100000. The hard cap is silent.
updateTake (2016) updateTake(pageId, rowNum, fields: {weight?, since_date?, source?}): Promise<void> Mutates weight, since_date or source. Cannot change claim, kind or holder. :512-538. Weight is normalized with normalizeWeightForStorage (clamp, stderr note). UPDATE takes SET weight=COALESCE($w::real, weight), since_date=COALESCE(...), source=COALESCE(...), updated_at=now() WHERE page_id=$ AND row_num=$ RETURNING 1. Zero rows throws TAKE_ROW_NOT_FOUND.
supersedeTake (2030) supersedeTake(pageId, oldRow, newRow): Promise<{oldRow, newRow}> Transactionally marks the old row inactive and appends a new row at MAX(row_num)+1. Refuses resolved takes. :540-569, inside exec.transaction. (1) SELECT resolved_at FROM takes WHERE page_id AND row_num=oldRow: missing throws TAKE_ROW_NOT_FOUND, resolved throws TAKE_RESOLVED_IMMUTABLE. (2) SELECT COALESCE(MAX(row_num),0)+1 AS next FROM takes WHERE page_id=$. (3) INSERT INTO takes(...) VALUES(...), weight clamped to [0,1] (default 0.5). (4) UPDATE takes SET active=false, superseded_by=newRow, updated_at=now() WHERE page_id AND row_num=oldRow. No row lock is taken for the MAX step.
resolveTake (2047) resolveTake(pageId, rowNum, resolution: TakeResolution): Promise<void> Sets the resolved state on a take. Immutable once set. :571-592. Missing row throws TAKE_ROW_NOT_FOUND. Already resolved throws TAKE_ALREADY_RESOLVED. deriveResolutionTuple (takes-resolution.ts:22) maps quality (correct, incorrect, partial, unresolvable) to outcome (true, false, null). quality wins if both are set and disagree, and throws. UPDATE takes SET resolved_at=now(), resolved_quality, resolved_outcome, resolved_value, resolved_unit, resolved_source, resolved_by, updated_at=now() WHERE page_id AND row_num.
getScorecard (2062) getScorecard(opts: TakesScorecardOpts, allowList: string[] | undefined): Promise<TakesScorecard> Aggregate calibration: totals, accuracy, Brier, partial rate, unresolvable rate. :599-638. One aggregate SELECT: COUNT(*) FILTER (WHERE kind='bet') as total_bets, FILTER (resolved_quality IN ('correct','incorrect','partial')) as resolved, FILTER counts for correct, incorrect, partial, unresolvable, and AVG(POWER(weight - (correct?1:0),2)) over correct and incorrect as brier. Filters: holder, domainPrefix (EXISTS page slug LIKE prefix%), since/until (on since_date), allow-list (holder = ANY, inside the GROUP BY scope, so hidden holders contribute zero), source scope, excludePrivate. finalizeScorecard (takes-resolution.ts:90): accuracy = correct/(correct+incorrect), brier null if no binary bet, partial_rate = partial/resolved, unresolvable_rate = unresolvable/(resolved+unresolvable).
getCalibrationCurve (2070) getCalibrationCurve(opts: CalibrationCurveOpts, allowList): Promise<CalibrationBucket[]> Bins resolved correct and incorrect bets by stated weight. Returns observed hit rate and predicted mean per bucket. Excludes partial. :650-694. bucketSize clamped to (0,1], default 0.1. maxIdx = floor(1/bucketSize)-1. CTE binned: bucket_idx = LEAST(FLOOR(weight::numeric / bucketSize::numeric)::int, maxIdx), hit = (resolved_quality='correct')::int, over resolved_quality IN ('correct','incorrect') with holder, allow-list, source and privacy filters. Output bucket_lo, bucket_hi (bucket_idx*size as numeric then ::float), n = COUNT(*), observed = AVG(hit), predicted = AVG(weight). Only buckets that contain rows are returned. NUMERIC arithmetic is used on purpose so bucket boundaries like 0.7/0.1 do not drift (comment :661-665).
addSynthesisEvidence (2073) addSynthesisEvidence(rows: SynthesisEvidenceInput[]): Promise<number> Records which takes a synthesis page cited. Idempotent. Returns number inserted. :696-712. INSERT INTO synthesis_evidence(synthesis_page_id,take_page_id,take_row_num,citation_index) SELECT v.* FROM unnest($int[], $int[], $int[], $int[]) AS v(...) ON CONFLICT (synthesis_page_id,take_page_id,take_row_num) DO NOTHING RETURNING 1. Returns result.length.

Engine-level differences: none in SQL. Both engines call the same takesImpl functions with the same executor shape. Typed-signature difference only: PGLite's searchTakes and searchTakesVector opts type (pglite-engine.ts:2648-2658) omits sourceId and sourceIds, while Postgres's includes them (postgres-engine.ts:2507-2513). Runtime passes the full opts object through, so behavior matches.

Dream-cycle verdict cache

Table dream_verdicts, keyed (file_path, content_hash). Index dream_verdicts_expires_idx (schema.sql:1505). expires_at defaults to now()+30 days at the table level. Writes set it explicitly.

Method (iface line) Signature What it does SQL / impl
getDreamVerdict (2078) getDreamVerdict(filePath, contentHash): Promise<DreamVerdict | null> Cache lookup. Expired rows are misses. NULL-expiry rows count as hits. PGLite pglite-engine.ts:2367-2400, Postgres postgres-engine.ts:2183-2216. SELECT worth_processing, reasons, judged_at, score, content_type, segments, entities, model, triage_version FROM dream_verdicts WHERE file_path=$ AND content_hash=$ AND (expires_at IS NULL OR expires_at > now()). Maps null arrays to [], judged_at to ISO string. Note the NULL-expiry branch, documented as the #4657 bootstrap window (postgres-engine.ts:2200; PGLite pglite-engine.ts:2383).
putDreamVerdict (2079) putDreamVerdict(filePath, contentHash, verdict: DreamVerdictInput): Promise<void> Upserts a verdict and resets the TTL. PGLite pglite-engine.ts:2402-2430, Postgres postgres-engine.ts:2218-2243. wellFormedJson(verdict) first. INSERT INTO dream_verdicts(file_path,content_hash,worth_processing,reasons,score,content_type,segments,entities,model,triage_version,expires_at) VALUES(...,now()+make_interval(secs => TTL)) ON CONFLICT (file_path,content_hash) DO UPDATE SET <all fields>=EXCLUDED.*, judged_at=now(), expires_at=EXCLUDED.expires_at. TTL = DREAM_VERDICT_TTL_SECONDS (engine.ts:528). JSON columns: PGLite $4::jsonb with JSON.stringify, Postgres sql.json.
sweepDreamVerdicts (2084) sweepDreamVerdicts(): Promise<number> Deletes expired verdicts. Housekeeping only. PGLite pglite-engine.ts:2432 returns affectedRows. Postgres postgres-engine.ts:2245 returns count. Both run DELETE FROM dream_verdicts WHERE expires_at <= now(). NULL-expiry rows are not swept.

Contradiction probe (batched takes, cache, run history)

Tables: eval_contradictions_runs (schema.sql:1631), eval_contradictions_cache (schema.sql:1614, expiry index at :1626), and take_grade_cache (referenced by listTakes ordering, schema.sql:1727). All SQL lives in engine-sql/takes.ts for both engines.

Method (iface line) Signature What it does SQL / impl
listActiveTakesForPages (2102) listActiveTakesForPages(pageIds: number[], opts?: {takesHoldersAllowList?}): Promise<Map<number, Take[]>> Active takes for many pages in one round-trip. Every requested page id is a key, with [] if it has none. takes.ts:109-135. SELECT t.*, p.slug AS page_slug FROM takes t JOIN pages p ON p.id=t.page_id WHERE t.page_id = ANY($::int[]) AND t.active = true AND (allow-list IS NULL OR holder = ANY(...)) ORDER BY t.page_id, t.row_num. Read is LegacyUnscopedRead. No deleted_at or privacy filter.
writeContradictionsRun (2115) writeContradictionsRun(row: {run_id, judge_model, prompt_version, queries_evaluated, queries_with_contradiction, total_contradictions_flagged, wilson_ci_lower, wilson_ci_upper, judge_errors_total, cost_usd_total, duration_ms, source_tier_breakdown, report_json}): Promise<boolean> Persists one probe run row. Idempotent on run_id. Returns true if inserted. takes.ts:141-174. INSERT INTO eval_contradictions_runs(...) VALUES(...,${jsonbParam(source_tier_breakdown)},${jsonbParam(report_json)}) ON CONFLICT (run_id) DO NOTHING, returning affectedRows > 0.
loadContradictionsTrend (2137) loadContradictionsTrend(days: number): Promise<Array<{run_id, ran_at, ... report_json}>> Runs from the last N days, newest first. takes.ts:180-221. Cutoff is computed in JS: Date.now() - max(0,days)*86400000. SELECT ... FROM eval_contradictions_runs WHERE ran_at >= $cutoff ORDER BY ran_at DESC. No LIMIT. Numeric columns go through Number(). ran_at becomes an ISO string. Read is LegacyUnscopedRead.
getContradictionCacheEntry (2163) getContradictionCacheEntry(key: {chunk_a_hash, chunk_b_hash, model_id, prompt_version, truncation_policy}): Promise<Record<string, unknown> | null> Judge-verdict cache lookup. Key is the full five-part key, so prompt or truncation changes miss. takes.ts:227-247. SELECT verdict FROM eval_contradictions_cache WHERE <5 key cols> AND expires_at > now() LIMIT 1. Read is LegacyUnscopedRead.
putContradictionCacheEntry (2180) putContradictionCacheEntry(opts: {...5 key cols, verdict, ttl_seconds?}): Promise<void> Upserts a verdict and slides the TTL. takes.ts:253-279. TTL = max(60, ttl_seconds ?? 30d). expiresAt = new Date(Date.now() + ttl*1000) is computed in JS. INSERT INTO eval_contradictions_cache(...,verdict,expires_at) VALUES(...,${jsonbParam(verdict)},$expiresAt) ON CONFLICT (5 key cols) DO UPDATE SET verdict, expires_at, created_at=now().
sweepContradictionCache (2194) sweepContradictionCache(): Promise<number> Deletes expired judge-cache rows. Called periodically from cache.ts. takes.ts:282-287. DELETE FROM eval_contradictions_cache WHERE expires_at <= now(), returns affectedRows.

Engine-level: all seven delegate to takesImpl in both engines (PGLite pglite-engine.ts:2575-2642, Postgres postgres-engine.ts:2434-2500). Writes use this.engineSql (SqlExecutor), and the loadContradictionsTrend, listActiveTakesForPages, and cache reads use unscopedExecutor.

Tests covering this range

Counts are the number of files under test/ that name the method (grep \bname\b). "Parity" means the test runs on both engines through describeBoth in test/e2e/engine-parity.test.ts:33. That helper is gated by SKIP_PG, which is set when no DATABASE_URL is present (:32). test/pglite-engine.test.ts has no takes, ontology, chronicle, file, dream or contradiction tests. Its only coverage here is tags, timeline and raw-data basics.

Method Files naming it Parity / direct coverage
addTag, removeTag 66 / 16 engine-parity.test.ts:189-201 (getTags excludePrivate, liveOnly, both engines), :1434-1435 (sourceId scoping); pglite-engine.test.ts:766-787 (add, remove, dup idempotent)
getTags 42 engine-parity.test.ts:189, :1468-1470 (sourceIds[] grant); pglite-engine.test.ts:772-786
addTimelineEntry 66 engine-parity.test.ts:287, :1444-1446 (sourceId); pglite-engine.test.ts:800-814; test/e2e/chronicle-event-projection-parity.test.ts (describeBoth)
addTimelineEntriesBatch 34 pglite-engine.test.ts:823+ (batch); test/links-timeline-jsonb-poison.test.ts; engine-parity.test.ts count 1
getTimeline 46 engine-parity.test.ts:1536-1549 (sourceIds, D5A window), :1557-1584 (Cat7-1 unscoped plan, both engines); pglite-engine.test.ts:800-820
getTimelineForDate, getSince, getOnThisDay 15 / 11 / 11 test/chronicle-ops-scope.test.ts, test/chronicle-extract.test.ts, test/e2e/chronicle-event-projection-parity.test.ts. No direct parity test in engine-parity.test.ts (0 hits).
getLastSeen 12 test/e2e/chronicle-last-seen-postgres.test.ts (Postgres only, name-gated). No PGLite parity hit.
upsertEventProjection 16 test/e2e/chronicle-event-projection-parity.test.ts (describeBoth), test/chronicle-invite-projection.test.ts
mergeOntologyFact 27 test/e2e/ontology-merge-parity.test.ts (describeBoth), test/chronicle-ontology.test.ts, test/ontology-backdated-same-value.test.ts, test/trust-guarded-supersession.test.ts, test/pathological/b-10-ontology-rekey-collision.test.ts, test/chronicle-ontology-ops.test.ts
getOntology 13 test/e2e/ontology-merge-parity.test.ts, test/chronicle-ontology-private-visibility.test.ts
discoverOntologyDimensions 10 test/chronicle-ontology.test.ts, test/managed-ontology-propose.test.ts (no parity hit)
findOntologyConflicts 13 test/chronicle-ontology.test.ts, test/e2e/ontology-merge-parity.test.ts
putRawData 26 engine-parity.test.ts:2714-2731 (soft-delete filter, PGLite always and Postgres when DATABASE_URL is set); pglite-engine.test.ts (count 2)
getRawData 20 engine-parity.test.ts:2714-2731; pglite-engine.test.ts (count 2)
upsertFile 12 test/engine-upsertFile.test.ts (PGLite only)
getFile 13 test/engine-upsertFile.test.ts
listFilesForPage 11 test/engine-upsertFile.test.ts
addTakesBatch 77 test/e2e/takes-postgres.test.ts (Postgres, DATABASE_URL-gated), test/e2e/takes-weight-rounding-postgres.test.ts, test/takes-weight-rounding.test.ts, test/e2e/jsonb-batch-poison-postgres.test.ts, test/e2e/takes-scorecard-parity.test.ts (uses PGLite and Postgres)
updateTakeEmbeddings 12 test/takes-vector-lifecycle-5885.serial.test.ts (PGLite), test/e2e/takes-horizon-postgres.test.ts
listTakes 24 test/list-takes-tie-order.test.ts, test/takes-source-scope.test.ts, test/takes-mcp-allowlist.test.ts, test/e2e/serve-http-takes-holders.test.ts
searchTakes 19 test/e2e/takes-postgres.test.ts (pg_trgm), test/takes-source-scope.test.ts, test/eligibility-reads.test.ts
searchTakesVector 15 test/e2e/takes-postgres.test.ts (pgvector), test/takes-vector-lifecycle-5885.serial.test.ts
getTakeEmbeddings 10 test/takes-vector-lifecycle-5885.serial.test.ts
countStaleTakes, listStaleTakes 12 / 13 test/takes-vector-lifecycle-5885.serial.test.ts
updateTake 15 test/takes-weight-rounding.test.ts, test/takes-grades-cli.test.ts
supersedeTake 14 test/e2e/takes-postgres.test.ts (supersede path on real PG), test/extract-takes.test.ts
resolveTake 24 test/e2e/takes-postgres.test.ts (immutability), test/grade-takes-window.test.ts, test/eval-takes-quality-boundaries.test.ts
getScorecard 16 test/e2e/takes-scorecard-parity.test.ts (both engines), test/calibration-profile.test.ts
getCalibrationCurve 14 test/e2e/takes-scorecard-parity.test.ts (both engines, bucket boundaries), test/e2e/calibration-profile-write.test.ts
addSynthesisEvidence 11 test/e2e/takes-postgres.test.ts (FK cascade on take delete), test/think-save-source.test.ts
getDreamVerdict 16 test/e2e/engine-parity.test.ts:389-413 (triage-v1 round-trip, identical shape on both engines), :432 (legacy row); test/e2e/dream-triage-backoff-postgres.test.ts; test/pathological/b6-triage-verdict-surrogate.test.ts
putDreamVerdict 23 test/e2e/engine-parity.test.ts:408-413 (both engines, score update); test/synthesize-triage-backoff.test.ts; test/e2e/dream-synthesize-pglite.test.ts
sweepDreamVerdicts 7 Few direct tests; test/synthesize-triage-backoff.test.ts covers the TTL and expiry behavior
listActiveTakesForPages 11 test/eval-contradictions-engine.test.ts (PGLite), test/e2e/eval-contradictions-postgres.test.ts (Postgres)
writeContradictionsRun, loadContradictionsTrend 16 / 12 test/e2e/eval-contradictions-postgres.test.ts, test/eval-contradictions-engine.test.ts, test/eval-takes-quality-runner.serial.test.ts
getContradictionCacheEntry, putContradictionCacheEntry, sweepContradictionCache 11 each test/eval-contradictions/no-valid-until-write.test.ts, test/eval-contradictions-engine.test.ts, test/e2e/eval-contradictions-postgres.test.ts

Summary of coverage: 43 methods in range. Eight have parity tests that run on both engines in engine-parity.test.ts: addTag, getTags, addTimelineEntry, getTimeline, putRawData, getRawData (soft-delete), getDreamVerdict, putDreamVerdict. removeTag and addTimelineEntriesBatch appear once in that file, but not in a dedicated parity assertion I could confirm. The other methods have no hit in engine-parity.test.ts; their parity coverage, where it exists, sits in the separate *-parity.test.ts files listed above. Chronicle ontology and takes have parity tests in separate files (ontology-merge-parity.test.ts, chronicle-event-projection-parity.test.ts, takes-scorecard-parity.test.ts). Several methods have Postgres-only tests (takes-postgres.test.ts, eval-contradictions-postgres.test.ts, chronicle-last-seen-postgres.test.ts), gated by DATABASE_URL.

Parity risks noted

  1. Clock sources differ. Reads and sweeps compare against the DB clock (now(), current_date), e.g. getDreamVerdict, sweepDreamVerdicts, getContradictionCacheEntry, sweepContradictionCache, getOnThisDay, getLastSeen, getOntology default asof. Some writes and computations use the JS clock: putContradictionCacheEntry expiry (takes.ts:262-263), loadContradictionsTrend cutoff (takes.ts:195), and finalizeLastSeen's days_ago (last-seen.ts:11-21). Clock skew between app and DB changes results. A Rust rewrite has to choose one clock and keep both sides on it.
  2. Transaction-local trust GUC. ONTOLOGY_SUPERSEDE_GUARD (chronicle/ontology.ts:65) reads current_setting('gbrain.write_trust_tier', true) in the same transaction. Ontology supersession depends on that session setting, not on a parameter. An embedded store needs an equivalent per-transaction writer tier.
  3. Postgres-specific SQL features with no direct equivalent: DISTINCT ON, json_agg/json_build_object, jsonb_to_recordset (batch writes), jsonb_array_elements_text, unnest(int[]), ON CONFLICT ... WHERE <partial index> (timeline_event_dedup, facts dedup), RETURNING (xmax = 0) AS created (upsertFile created flag), md5() as a dedup key (timeline md5(summary)), FOR KEY SHARE (addTimelineEntry), LATERAL ... OFFSET 0 planner fence (getTimeline), make_interval, date_trunc('week'), EXTRACT(MONTH/DAY), pg_trgm <% and word_similarity, pgvector <=> and vector_dims.
  4. Calibration bucketing relies on NUMERIC arithmetic (takes.ts:661-665). Float math would put boundary weights in a different bucket. The Postgres driver also needs explicit ::float casts on bound scalars (takes.ts:644-648), so scalars are text-bound there. Rust must do exact decimal bucketing.
  5. files identity mismatch. The DDL is UNIQUE(storage_path) (schema.sql:1149), while getFile and the file comment at engine-sql/files.ts:41 describe identity as (source_id, storage_path). The ON CONFLICT (storage_path) DO UPDATE never updates source_id. A second source writing the same storage_path keeps the first source's row, and getFile(secondSource, path) returns null.
  6. supersedeTake allocates row_num as MAX(row_num)+1 without a row lock (takes.ts:554). Two concurrent supersedes on one page can compute the same next row number. A Rust rewrite should allocate with a lock or sequence.
  7. addTakesBatch ON CONFLICT is DO UPDATE (takes.ts:62-63). An intra-batch duplicate (page_id,row_num) fails the whole statement, and the claim-change reset of embeddings (takes.ts:81-84) must be reproduced exactly.
  8. updateTakeEmbeddings silently drops a vector when the claim changed in flight (WHERE t.claim = v.claim, takes.ts:502). It returns the updated count only, so the caller cannot tell a dropped vector from a successful one.
  9. listStaleTakes has a hard LIMIT 100000 (takes.ts:453). Excess rows are dropped with no signal.
  10. loadContradictionsTrend has no LIMIT (takes.ts:196-205). Result size is unbounded.
  11. searchTakesVector has no dimension check in SQL (takes.ts:385-401). Mixed-width vectors fail in pgvector. The Rust version must decide what to do with mixed widths.
  12. getLastSeen matches wikilinks with LIKE ... ESCAPE '\\' on who strings (timeline.ts:243-245). The escape rule has to be reproduced exactly, or wikilink matches will differ.
  13. Reads on master are LegacyUnscopedRead (engine-sql/brands.ts:33): no RLS scope transaction. Any Rust rewrite that adds scope enforcement will change behavior in this range. The brands doc calls widening a TODO, not a drive-by (brands.ts:17-21).
  14. Dialect cancellation differs. PGLite cannot cancel a running statement; pgliteQuery races a late abort (dialect-pglite.ts:42-56). Postgres uses real cancellation. Batch-write abort behavior is not identical.
  15. Lock capability differs. transactionAdvisoryLocks is false on PGLite (dialect-pglite.ts:27) and true on Postgres (dialect-postgres.ts:55). mutatePageTag calls lockPageKeys on both. I did not trace the lock mechanism itself.
  16. Type-only drift: PGLite's searchTakes/searchTakesVector opts type (pglite-engine.ts:2648-2658) omits sourceId/sourceIds. Runtime behavior is unchanged.
  17. resolveTake and updateTake have no transaction and do a SELECT then an UPDATE (takes.ts:571-576). Concurrent resolves can race the immutability check. The engine's error code is the only guard.
  18. putRawData without sourceId matches every same-slug page (postgres-engine.ts:2101, pglite-engine.ts:2284). Multi-source brains can get a 21000 error. The sourceId path avoids this.

Facts, Segmented storage, Versions, Stats, Sync, Config and Migration support

Scope: BrainEngine interface members declared in src/core/engine.ts lines 2196-2890 (70 methods, confirmed by listing every member in that range). Interface line numbers below are engine.ts:N. Implementations: PGLiteEngine = src/core/pglite-engine.ts, PostgresEngine = src/core/postgres-engine.ts, shared SQL = src/core/engine-sql/*.ts. "Shared" means one SQL implementation runs on both engines through SqlExecutor. "Inline" means each engine writes its own SQL.

Groups in this part: Facts (14), Segmented page storage (11), Derivation watermarks (3), Versions (3), Stats and health (2), Ingest log (2), Sync (9), Config (5), Migration and raw SQL (5), Code edges (6), Eval capture (5), Salience and anomalies (5). Total 70.

Engine plumbing used by this range

Item PGLite Postgres Notes
Shared executor engineSql getter: pgliteExecutor(db or guardedHandle(db)), pglite-engine.ts:794-795 engineSql / engineSqlOn(conn), postgres-engine.ts:317-322 Executor built per access so transaction clones run on their own handle.
Dialect capabilities PGLITE_CAPABILITIES = maxBindParamsPerStatement 30000, transactionAdvisoryLocks false, probesEmbeddingCast false, engine-sql/dialect-pglite.ts:25-29 maxBindParamsPerStatement Infinity, transactionAdvisoryLocks true, probesEmbeddingCast true, engine-sql/dialect-postgres.ts:54-56 These three flags drive most PGLite/PG branches in the shared SQL below.
Unscoped read brand unscopedExecutor(engineSql, label) wraps reads; LegacyUnscopedRead type, engine-sql/brands.ts same Every read in this range is on master with no source scope (EO4 inventory).
Transactions transaction() = db.transaction with composable handle, savepoint on nesting, checkpoint guard runOutermost, pglite-engine.ts:1209-1223. Single connection, no advisory locks. transactionOn(), postgres-engine.ts:726-760: BEGIN READ WRITE unless readOnly (postgres-engine/read-write-transactions.ts), checkoutGauge 'tx', signal abort via transaction-abort.ts, commit checks, savepoints on nested clones. Interface declaration at engine.ts:849-851 (outside this range; owned by the lifecycle part).
Statement timeout None. PGLite runs read on itself and ignores timeoutMs Per-statement statement_timeout (keyword-statement-timeout.ts, default 8000 ms, range 1000-60000) See keywordBoundedRead.
Pool gauge / reserved connections None (single in-process connection) checkoutGauge acquire/release for raw, direct, tx checkouts (executeRaw 2998-3008, executeRawDirect 3053-3058, transactionOn), real driver queue numbers in postgres-engine/pool-stats.ts:1-11; reserveWithCancellation (postgres-engine/cancellation.ts:17) reserves a dedicated connection for each cancellable statement Pool diagnostics and reserved-connection behavior exist only on Postgres.
WAL checkpoint guard PgliteCheckpointGuard, pglite-engine/checkpoint-guard.ts:1-25; writesWal(sql) regex decides autocommit writes n/a Only in PGLite.
Config snapshot requestConfigValue / requestConfigRows, config-snapshot.ts:161-172, used by getConfig / getAllConfig on both engines same Request-scoped snapshot first, then the engine's own read.

Facts / hot memory (14 methods)

Shared SQL: src/core/engine-sql/facts.ts. Both engines delegate; PGLite passes undefined for the embedding-cast resolver, Postgres passes () => this.resolveFactsEmbeddingCast() (postgres-engine.ts:2271-2304, which probes pg_attribute for facts.embedding and returns ::halfvec or ::vector, cached on the root engine).

Test-count convention: "eng-parity" = test/e2e/engine-parity.test.ts (Postgres arm needs DATABASE_URL, PGLite arm always runs); "pglite" = test/pglite-engine.test.ts; "files" = number of test/ files that mention the method name anywhere (grep, so an upper bound on real coverage).

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
insertFact(input: NewFact, ctx: {source_id, supersedeId?}): Promise<{id, status}> (engine.ts:2215) pglite-engine.ts:2447 / postgres-engine.ts:2306; shared facts.ts:88-172 Two paths. (a) No supersedeId: one INSERT INTO facts (...) VALUES (...) RETURNING id, status 'inserted'. (b) With supersedeId: exec.transaction, then INSERT, then UPDATE facts SET expired_at = now(), superseded_by = <new id>, valid_until = CASE ... END WHERE id = supersedeId AND expired_at IS NULL, status 'superseded'. Per-entity lock: pg_advisory_xact_lock(hashtextextended(source_id || ':' || entity_slug, 0)) when transactionAdvisoryLocks (facts.ts:120-123, 149-152). Vector is an inlined literal '[...]'::vector or ::halfvec (facts.ts:105, 133); embedded_text_hash = md5(fact) when an embedding and model are supplied. The method never returns 'duplicate': dedup is the caller's job (engine.ts:2200-2214 says it can, the code cannot). PGLite: no advisory lock, plain insert not wrapped in a transaction (facts.ts:168-170), cast always ::vector (probesEmbeddingCast false). PG: lock, cast probe. eng-parity 8; pglite 0; facts-engine.test.ts, facts-separation-pglite.test.ts; 171 files mention it.
insertFacts(rows, ctx: {source_id}, opts?: {deleteForPageFirst?}): {inserted, ids, warnings, deleted} (engine.ts:2258) pglite-engine.ts:2458 / postgres-engine.ts:2317; shared facts.ts:260-370 One exec.transaction. (1) If deleteForPageFirst: DELETE FROM facts WHERE source_id AND source_markdown_slug = slug [AND NOT (COALESCE(source,'') LIKE ANY(prefix%))] [AND NOT (row_num IS NULL AND expired_at IS NOT NULL)] (facts.ts:290-313), same tx as the inserts. (2) set_config(FACT_WITHDRAWAL_STAMP_DEFERRED_SETTING, 'on', true) (facts.ts:204), then multi-row INSERT ... SELECT FROM jsonb_to_recordset(<json text>) ... ON CONFLICT (source_id, source_markdown_slug, row_num) WHERE row_num IS NOT NULL DO NOTHING RETURNING id, source_markdown_slug, row_num, in chunks of FACT_INSERT_CHUNK = 500 (facts.ts:185, 221-248); the setting is reset afterwards (facts.ts:256). (3) Second pass: superseded by #N refs resolved by one SELECT over facts joined to a JSON recordset, then one UPDATE facts SET superseded_by ... FROM jsonb_to_recordset (facts.ts:329-366); warnings from facts/supersede-resolve.ts. Vector cast: r.embedding::vector or ::halfvec per castSuffix. PGLite: cast fixed ::vector; no lock needed. Returned ids are filtered (facts.ts:318), so they are NOT index-aligned with rows when ON CONFLICT skips a row (doc at engine.ts:2236 says input order). eng-parity 0; pglite 0; 50 files (generations-deprecated-fact-writers.test.ts, extract-facts-reconcile-drain.test.ts, fence-chunk-overlay.test.ts, conversation-facts-managed.test.ts).
deleteFactsForPage(slug, source_id, opts?): {deleted} (engine.ts:2305, @deprecated GBRA-161) pglite-engine.ts:2466 / postgres-engine.ts:2325; shared facts.ts:372-403 DELETE FROM facts WHERE source_id = $ AND source_markdown_slug = $ with the same optional excludeSourcePrefixes and preserveExpiredLegacy filters as above. Self-committing on the root executor (note at facts.ts:285-288). Hard delete. Identical on both engines. eng-parity 0; pglite 0; 18 files (generations-deprecated-fact-writers.test.ts, facts-thin-client-provenance.test.ts, canonical-writer-inventory.test.ts, extract-facts-phase.test.ts).
expireFact(id, opts?: {supersededBy?, at?}): boolean (engine.ts:2316, @deprecated) pglite-engine.ts:2454 / postgres-engine.ts:2313; shared facts.ts:174-182 UPDATE facts SET expired_at = $at, superseded_by = COALESCE($supersededBy, superseded_by) WHERE id = $ AND expired_at IS NULL; returns affectedRows > 0. Never deletes. Identical. eng-parity 0; pglite 0; 24 files (facts-engine.test.ts, recall-extensions.test.ts, write-attribution-legacy.test.ts).
listFactsByEntity(source_id, entitySlug, opts?): FactRow[] (engine.ts:2319) pglite-engine.ts:2474 / postgres-engine.ts:2333; shared facts.ts:405-437 SELECT * FROM facts WHERE source_id AND entity_slug [AND source_session]. Default activeOnly: expired_at IS NULL AND (valid_until IS NULL OR valid_until > now()). Optional: consolidated_at IS NULL, kind = ANY, visibility clause, privacy clause, audit-row exclusion (source != ALL(AUDIT_ROW_SOURCES)), fact ILIKE grep with escaped pattern, eligibility clause. ORDER BY valid_from DESC, id DESC LIMIT OFFSET (default limit 50). Identical. eng-parity 3; pglite 0; facts-engine.test.ts, facts-visibility.test.ts, entity-names-parity.test.ts, facts-separation-pglite.test.ts.
listFactsSince(source_id, since: Date, opts?): FactRow[] (engine.ts:2333) pglite-engine.ts:2482 / postgres-engine.ts:2341; shared facts.ts:439-464 WHERE source_id AND created_at >= since (or COALESCE(valid_from, created_at) with eventTime), plus recallNarrowing (facts.ts:79-86: entitySlug, sourceMarkdownSlug, sessionId, excludeAuditRows, grep). Active filter as above. ORDER BY created_at DESC, id DESC LIMIT OFFSET. fingerprint adds gbrain_fact_fingerprint(fact). Identical. eng-parity 1; pglite 0; hot-memory-invalidation.test.ts, facts-recall-audit-rows.test.ts, facts-engine.test.ts, ambient-recall.test.ts.
listFactsKeyset(source_id, after: {createdAt, id|null} | null, opts?): FactRow[] (engine.ts:2345) pglite-engine.ts:2490 / postgres-engine.ts:2349; shared facts.ts:477-503 Keyset read, OLDEST first: AND (created_at > ts::text::timestamptz OR (created_at = ts AND id > $id)), or created_at > ts when id is null. Adds to_char(created_at AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"') AS created_at_iso so microseconds survive. ORDER BY created_at ASC, id ASC LIMIT. Identical. eng-parity 0; pglite 0; delta-cursor-integrity.test.ts, engine-surface-coverage.test.ts, helpers/postgres-engine-sql-cases.ts.
listFactsBySession(source_id, sessionId, opts?): FactRow[] (engine.ts:2352) pglite-engine.ts:2498 / postgres-engine.ts:2357; shared facts.ts:505-533 WHERE source_id AND source_session = $, active/kind/visibility/privacy/audit/grep/eligibility filters, ORDER BY created_at DESC, id DESC LIMIT OFFSET. Identical. eng-parity 1; pglite 0; facts-separation-pglite.test.ts, facts-engine.test.ts, e2e/facts-separation-postgres.test.ts.
listSupersessions(source_id, opts?): FactRow[] (engine.ts:2370) pglite-engine.ts:2506 / postgres-engine.ts:2365; shared facts.ts:535-559 WHERE source_id AND superseded_by IS NOT NULL (not expired_at, so ontology and fence supersessions appear), optional COALESCE(expired_at, valid_until) >= since, visibility, privacy, eligibility. ORDER BY COALESCE(expired_at, valid_until) DESC, id DESC LIMIT (no OFFSET). Identical. eng-parity 2; pglite 0; chronicle-ontology.test.ts, recall-extensions.test.ts, facts-engine.test.ts, e2e/facts-fence-reconcile-postgres.test.ts.
countUnconsolidatedFacts(source_id): number (engine.ts:2382) pglite-engine.ts:2513 / postgres-engine.ts:2372; shared facts.ts:561-578 SELECT COUNT(*)::int FROM facts WHERE source_id AND entity_slug IS NOT NULL AND consolidated_at IS NULL AND expired_at IS NULL AND (valid_until IS NULL OR valid_until > now()) AND source != ALL(AUDIT_ROW_SOURCES). Identical. eng-parity 1; pglite 0; recall-extensions.test.ts, facts-recall-audit-rows.test.ts, facts/ttl-validity.test.ts.
findCandidateDuplicates(source_id, entitySlug, factText, opts?): FactRow[] (engine.ts:2393) pglite-engine.ts:2517 / postgres-engine.ts:2376; shared facts.ts:580-666 k clamped to 1..20, default 5. Three arms. (a) arm: 'keyword': findKeywordCandidates (facts.ts:638-666) builds an OR-of-lexemes tsquery from plainto_tsquery(lang, text), filters by to_tsvector(...) @@ q, orders by ts_rank_cd DESC, created_at DESC, id DESC. (b) opts.embedding given (needs embeddingModel, else []): WHERE embedding IS NOT NULL AND embedding_model = $ AND embedded_text_hash = md5(fact) AND vector_dims(embedding) = n ORDER BY embedding <=> '<literal>'::vector LIMIT k. (c) Otherwise ORDER BY created_at DESC, id DESC LIMIT k. All arms: active + validity filter, audit-row exclusion, attributed_to IS NULL OR = speaker, and onlyPage or excludeMemberPages (onOwnedMemoryPageSql). Identical SQL; PGLite always uses the ::vector literal. eng-parity 4; pglite 0; facts-engine.test.ts, facts/candidate-fusion.test.ts, facts/speaker-attribution.test.ts, facts/ttl-validity.test.ts.
consolidateFact(id, takeId): void (engine.ts:2404) pglite-engine.ts:2530 / postgres-engine.ts:2385; shared facts.ts:668-670 UPDATE facts SET consolidated_at = now(), consolidated_into = $takeId WHERE id = $id. Never deletes. Identical. eng-parity 0; pglite 0; facts-engine.test.ts, write-attribution-legacy.test.ts, engine-surface-coverage.test.ts.
findTrajectory(opts: TrajectoryOpts): TrajectoryPoint[] (engine.ts:2422) pglite-engine.ts:2526 / postgres-engine.ts:2389; shared facts.ts:672-740 SELECT id, valid_from, valid_until, claim_*, event_type, fact, source_session, source_markdown_slug, embedding FROM facts WHERE source (scalar sourceId, default 'default', or source_id = ANY(sourceIds)), entity_slug = $, expired_at IS NULL. Remote callers (remote !== false, fail-closed) get visibility = 'world' only. Optional metric/kind/since/until. Embedding returned only when embedding_model = (SELECT value FROM config WHERE key='embedding_model') and embedded_text_hash = md5(fact) (inline subquery, facts.ts:709). ORDER BY valid_from DESC, id DESC LIMIT (default 100, cap 500), then reversed to chronological. No validity filter by design (engine.ts:2415-2420). Identical. eng-parity 3; pglite 0; engine-find-trajectory.test.ts, trajectory-timeout-timers.test.ts, think-trajectory-injection.test.ts, migrate.test.ts.
getFactsHealth(source_id): FactsHealth (engine.ts:2425) pglite-engine.ts:2534 / postgres-engine.ts:2393; shared facts.ts:742-781 Two statements. (1) One row of COUNT(*) FILTER (...): total_active (active and not lapsed), total_today (created_at > now() - 24h), total_week (7 days), total_expired (expired_at IS NOT NULL or lapsed), total_consolidated. (2) Top 5 entities by active count, ORDER BY count DESC, entity_slug ASC. Identical. eng-parity 1; pglite 0; facts-doctor-shape.test.ts, facts-engine.test.ts, facts/ttl-validity.test.ts.

Segmented page storage (11 methods)

Shared SQL: src/core/engine-sql/page-body.ts. Limits: src/core/page-body/limits.ts (STAGE_BATCH_BYTES = 4 MiB for both engines, line 65-68; SEGMENT_TEXT_GC_GRACE_MS = 1 h, line 132; BODY_STAGING_IDLE_MS = 1 h, line 134; BODY_STAGING_HARD_MS = 24 h, line 135). Batches yield the event loop with setTimeout(0) (page-body.ts:50). Both engines call the shared functions with this.engineSql (PGLite pglite-engine.ts:2539-2562, PG postgres-engine.ts:2398-2421); only readSegmentTexts, readManifest and readSegmentDerivations use the unscoped read brand.

Test note for the whole family: test/large-pages/segment-gc.test.ts runs the GC helper cases on PGLite, test/e2e/segment-gc-parity.test.ts imports PostgresEngine (Postgres side). test/e2e/segmented-publish-tiling-postgres.test.ts and test/large-pages/segmented-publish-tiling.serial.test.ts cover the flip. No engine-parity or pglite-engine.test.ts hits for any of these 11 methods (eng-parity 0, pglite 0 for each).

Method (iface line) Impl What it does and SQL PGLite vs PG; tests
allocateBodyOccurrenceIds(count): number[] (engine.ts:2429) shared page-body.ts:58-62 SELECT nextval('page_body_occurrence_id_seq') AS id FROM generate_series(1, $count). count <= 0 returns [] without SQL. Ids are never reused. Identical. files 12 (helpers/segment-gc-cases.ts, helpers/segmented-page.ts, helpers/page-body-fixtures.ts).
stageSegmentTexts(sourceId, texts, opts?: {stagingId?}): {inserted} (engine.ts:2431) shared page-body.ts:69-100 Batches by bytes (STAGE_BATCH_BYTES) and by floor(maxBindParams / 5) rows (page-body.ts:72, 92). Per batch: if stagingId, INSERT INTO page_body_staging_texts (staging_id, source_id, sha256) SELECT ... FROM unnest(...) ON CONFLICT DO NOTHING (records the text under the staging so a page purge keeps it). Then INSERT INTO page_segment_texts (source_id, sha256, byte_len, line_count, body) VALUES ... ON CONFLICT (source_id, sha256) DO UPDATE SET created_at = now() RETURNING (xmax = 0) AS inserted. inserted counts fresh rows only. Row cap differs: PGLite maxBindParams 30000 gives 6000 rows per statement; PG Infinity means no row cap (only bytes). files 15.
readSegmentTexts(sourceId, sha256s: Uint8Array[]): Map<hexSha, bytes> (engine.ts:2433) shared page-body.ts:102-108 (unscoped read) SELECT encode(t.sha256,'hex') AS sha, t.body FROM page_segment_texts t WHERE t.source_id = $ AND t.sha256 = ANY(SELECT decode(h,'hex') FROM unnest($text[]) AS h). Missing shas are absent from the map. Empty input returns an empty Map. Identical. files 26.
stageManifestRows(rows: ManifestRowInput[]): void (engine.ts:2435) shared page-body.ts:110-132 Rows serialized by jsonRowBatches (batch-rows.ts) and inserted with INSERT INTO page_body_manifest (page_id, occurrence_id, gen_from, seq, region, source_id, sha256, byte_start, byte_len, line_start, line_count, cut_kind, entry_state, protected_regions, facts_slots, static_proj_bytes, static_proj_lines, outline, date_min, date_max, hash_state, staging_id) SELECT ... FROM jsonb_to_recordset(...). Hex sha and hash_state decoded to bytea. No transaction around the batches. Identical. files 13.
readManifest(pageId, opts?: {generation?, stagingId?, region?, lastN?}): ManifestRow[] (engine.ts:2437) shared page-body.ts:152-170 (unscoped read) Metadata only, never the body. Generation g = COALESCE($generation, pages.body_generation). Live rows: staging_id IS NULL AND gen_from <= g AND (gen_to IS NULL OR gen_to > g). Staged rows: staging_id = $stagingId. Optional region. ORDER BY seq, or with lastN: ORDER BY seq DESC LIMIT n then reversed. date_min/date_max rendered with to_char(..., 'YYYY-MM-DD'). Identical. files 32 (helpers/rechunk-segmented-cases.ts, helpers/segment-gc-cases.ts, helpers/conversation-facts-recovery-cases.ts).
stageSegmentChunks(pageId, generation, chunks, opts?: {embeddingColumn?}): number[] (engine.ts:2439) shared page-body.ts:186-257 One exec.transaction. (1) PG only (transactionAdvisoryLocks): SELECT 1 FROM pages WHERE id = $ FOR KEY SHARE, then pg_advisory_xact_lock(hashtextextended('gbrain:page-body-chunks:' || pageId, 0)) (page-body.ts:193-197). (2) SELECT body_generation, body_storage, slug FROM pages; refuses with revision_changed when generation < live, the page is inline, or an occurrence has no published manifest row at the live generation (page-body.ts:199-217). (3) chunk_index = COALESCE(MAX(chunk_index), -1) + 1 for the page, never reused (page-body.ts:219-221). (4) Model and write column come from config keys search_embedding_column, embedding_columns, embedding_model (page-body.ts:173-184). (5) INSERT INTO content_chunks (...) SELECT ... FROM jsonb_to_recordset(...) in byte-bounded batches, embedding::vector (or the configured vector cast), embedding_image::vector, body_generation = $generation, embedded_at or embedding_pending_since set. Returned indexes are in input order. PGLite skips step (1), so there is no page row lock or advisory lock; the generation checks still run. files 14 (helpers/segment-gc-cases.ts, helpers/rechunk-segmented-cases.ts, helpers/page-body-fixtures.ts).
createSegmentedPageShell(slug, meta, {sourceId}): {id} (engine.ts:2441) shared page-body.ts:270-282 INSERT INTO pages (source_id, slug, type, page_kind, title, compiled_truth NULL, timeline NULL, frontmatter, updated_at, effective_date..., chunker_version, source_*, ingested_*, body_storage 'segmented', body_generation 0, body_bytes 0) VALUES (...) RETURNING id. Slug through validateSlug. Origin claim is checked by the caller (page-body.ts:272 comment). Identical. files 11.
flipBodyGeneration(input: BodyFlipInput): {retiredChunks, retireDeferred} (engine.ts:2443) shared page-body.ts:290-354 Asserts newGeneration === expectedGeneration + 1 (page-body.ts:291). One exec.transaction: (1) SELECT body_generation, slug, source_id FROM pages WHERE id FOR UPDATE. Deleted page throws page_identity_changed; generation mismatch throws revision_changed (page-body.ts:292-308). (2) SELECT state FROM page_body_stagings WHERE id FOR UPDATE; must be open, sealed or prepared, else body_staging_expired (page-body.ts:309-310). (3) Chunks to retire counted; if more than retireMaxRows the delete is deferred (page-body.ts:314-321). (4) UPDATE page_body_manifest SET gen_to = newGen for closeOccurrences; DELETE FROM content_chunks for the retire predicate; UPDATE page_body_manifest SET staging_id = NULL (publishes staged rows). (5) UPDATE pages SET body_storage='segmented', compiled_truth=NULL, timeline=NULL, body_generation=new, content_hash, body_bytes, truth_bytes, body_lines, segment_count, segmenter_version, truth_lead, timeline_lead, body_flags, type, page_kind, title, frontmatter, updated_at=now(), deleted_at=NULL, COALESCE(...) for effective_date and source fields. (6) GST-1209 check: liveTilingSql/untiledSql (page-body/schema.ts) must tile the body, else body_staging_expired reason generation_incomplete (page-body.ts:345-351). Comment at page-body.ts:287-288 notes this fires trg_pages_search_vector. Same statements; PGLite relies on being single-writer (no FOR UPDATE contention). files 13 (helpers/segmented-page.ts, helpers/segment-gc-cases.ts, helpers/segmented-publish-tiling-cases.ts, helpers/engine-sql-rollback-cases.ts).
openBodyStaging(input: BodyStagingInput): void (engine.ts:2445) shared page-body.ts:375-383 INSERT INTO page_body_stagings (id, kind, principal_kind, principal_id, client_request_id, source_id, source_incarnation, slug, page_id, mode, base_revision, base_generation, declared_bytes, declared_sha256 (hex to bytea), reserved_bytes, idle_expires_at, hard_expires_at) VALUES (...). A staging is a GC root while open, sealed or prepared. Identical. files 18 (refresh-barrier-purge-gst1602.test.ts, helpers/segment-gc-cases.ts, helpers/postgres-engine-sql-cases.ts).
setBodyStagingState(id, state, opts?: {commitRequestId?}): void (engine.ts:2447) shared page-body.ts:391-401 UPDATE page_body_stagings SET state, updated_at = now(), idle_expires_at = CASE WHEN live THEN LEAST(hard_expires_at, now() + idle) ELSE idle_expires_at END, commit_request_id = COALESCE($new, commit_request_id) WHERE id AND state IN live RETURNING id. If no row: aborted/expired return quietly; the same state is a no-op; any other case throws body_staging_expired. Identical. files 16.
sweepSegments(opts?: {sourceId?, pageId?, graceMs?, expireStagings?, now?}): {retiredChunksDeleted, manifestRowsDeleted, textsDeleted, stagingsExpired} (engine.ts:2449) shared page-body.ts:425-487 Six statements in order. (1) If expireStagings: CTE UPDATE page_body_stagings ... SET state='expired' for live stagings past hard or idle deadline (idle applies to non-upload kinds, or uploads still open) and not held by a non-terminal persistence_requests.intent->body_ref; RELEASE_STAGED_CTE releases reserved bytes (persistence/limits.ts). Then delete parts of non-live stagings and terminal stagings older than 24 h. (2) Delete chunks with body_generation > page live generation not owned by a live staging. (3) Delete chunks of occurrences owned only by dead stagings at or below the live generation (page-body.ts:458-462). (4) Delete staged manifest rows of dead stagings (RETURNING sha when pageId set). (5) Delete closed manifest rows not referenced by the live generation or any page_versions row (page-body.ts:467-471). (6) Delete page_segment_texts older than the grace window, unreferenced by any manifest row, not recorded by a live staging (page_body_staging_texts), and not needed by stagingNeedsTextSql; FOR UPDATE SKIP LOCKED. With pageId, only shas released by steps (4)-(5) are candidates. Identical SQL; PGLite opts.now binds the same. e2e/segment-gc-parity.test.ts uses PostgresEngine; large-pages/segment-gc.test.ts uses PGLiteEngine. files 19.

Per-segment derivation watermarks (3 methods)

Shared SQL: page-body.ts:489-551. Kinds (page-body/schema.ts:194): links, mentions, timeline, fences, conversation_facts, facts_backstop, synopsis. States (schema.ts:195): done, partial, failed, skipped. Types in page-body/derivations.ts.

Method (iface line) Impl What it does and SQL PGLite vs PG; tests
readSegmentDerivations(pageId, opts?: {kind?, occurrenceIds?}): SegmentDerivationRow[] (engine.ts:2452) shared page-body.ts:492-513 (unscoped read) SELECT ... FROM page_segment_derivations d WHERE d.page_id = $ [AND d.pipeline = kind] [AND d.occurrence_id = ANY(ids)] ORDER BY d.pipeline, d.occurrence_id. created_at/updated_at rendered as UTC text. Empty occurrenceIds returns []. Identical. files 15 (helpers/segment-derivation-cases.ts, engine-sql-segment-derivations.test.ts, e2e/segment-derivations-postgres.test.ts is the Postgres-named runner).
upsertSegmentDerivations(rows): void (engine.ts:2454) shared page-body.ts:521-542 Batches by bytes; INSERT INTO page_segment_derivations (page_id, pipeline, occurrence_id, pipeline_version, segment_sha256, context_hash, state, output, receipt) SELECT ... FROM jsonb_to_recordset(...) ON CONFLICT (page_id, pipeline, occurrence_id) DO UPDATE SET pipeline_version, segment_sha256, context_hash, state, output, receipt, updated_at = now() (created_at kept). If stored < input, throws DerivationWriteRefused after storing the others. Batches are NOT one transaction. Identical. files 10.
retireSegmentDerivations(pageId, liveOccurrenceIds, opts?: {kind?}): number (engine.ts:2456) shared page-body.ts:545-551 DELETE FROM page_segment_derivations WHERE page_id = $ [AND pipeline = kind] AND NOT (occurrence_id = ANY(live)); returns affectedRows. Identical. files 8.

Versions (3 methods)

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
createVersion(slug, opts?: {sourceId?, preimage?} & VersionHistory): PageVersion (engine.ts:2465) pglite-engine.ts:2709 / postgres-engine.ts:2565 -> page-state/versions.ts:25-50 (createPageVersion) With preimage: insert directly. Without it: engine.transaction -> tx.lockPageKeys([{sourceId, slug}]) -> tx.readPageSnapshot(slug, {sourceId, includeDeleted: true}); missing page throws createVersion failed. Insert: INSERT INTO page_versions (page_id, compiled_truth, frontmatter, knowledge_revision, timeline, title, type, tags, is_deleted, source_path, change_reason) VALUES (...) RETURNING * (versions.ts:30-35). historyGap runs UPDATE pages SET version_history_gap_at = now(). Segmented pages insert a reference row (body_storage 'segmented', body_generation, body_bytes, content_hash, no body) and prune by pruneSegmentedVersions (versions.ts:56-70). Large inline pages prune older rows with DELETE ... OFFSET and markPrunedHistory (versions.ts:34-38). Default source: sourceId ?? 'default' (pglite-engine.ts:2710, postgres-engine.ts:2566). Identical. files 70; eng-parity 0; pglite-engine.test.ts 3; 70 files (e.g. generation-identity-visibility-9386.test.ts, belief-ledger-w5-time-model.test.ts). See parity risk 3.
getVersions<B>(slug, opts?: GetVersionsOpts<B>): PageVersionRows<B> (engine.ts:2471) pglite-engine.ts:2713 / postgres-engine.ts:2569 -> shared pages.ts:1079-1105 SELECT <version columns>, <segmented columns>, replaced_because, <valid-time columns> FROM page_versions pv JOIN pages p ON p.id = pv.page_id WHERE p.slug = $ [AND p.source_id = scope] [privacy filter when excludePrivate] AND rederive-hidden predicate ORDER BY pv.snapshot_at DESC, pv.id DESC [LIMIT]. includeBody: false selects metadata columns only. Without a source scope it returns same-slug versions across all sources (matches engine.ts:2467-2470). Segmented rows get numeric body_generation and body_bytes. Identical. files 23; eng-parity 0; pglite-engine.test.ts 3; 23 files (e.g. version-history-privacy.test.ts, persistence-version-deletion.test.ts).
revertToVersion(slug, versionId, opts?: {sourceId?}): void (engine.ts:2477) pglite-engine.ts:2717 / postgres-engine.ts:2573 -> shared pages.ts:1108-1155 Refuses first (segmentedRevertRefusal) if the page or the version is segmented, before any write. Otherwise UPDATE pages SET compiled_truth = pv.compiled_truth, frontmatter = pv.frontmatter, chunker_version = bodyWriteChunkVersion(...), updated_at = now() FROM page_versions pv WHERE pages.slug = $ [AND pages.source_id = $] AND pv.id = $ AND pv.page_id = pages.id. Does not snapshot the current state first, does not touch title/type/timeline/tags, does not re-chunk or re-embed. Identical. files 15; eng-parity 0; pglite-engine.test.ts 2; 15 files (e.g. write-attribution-legacy.test.ts, chunker-version-insert-default.test.ts).

Stats and health (2 methods)

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
getStats(opts?: {sourceId?, sourceIds?}): BrainStats (engine.ts:2488) pglite-engine.ts:2726-2778 / postgres-engine.ts:2582-2641. Inline SQL in both engines (not in engine-sql). Scope array = sourceIds ?? [sourceId] ?? null, bound as $1::text[]. One scalar query: page_count (live pages), chunk_count, embedded_count (cc.<active embedding column> IS NOT NULL, column from activeEmbeddingColId({fallbackToLegacy: true})), link_count (both endpoints in scope), tag_count (count(DISTINCT tag)), timeline_entry_count. Second query: pages_by_type (live pages, GROUP BY type). Scope logic matches engine.ts:2481-2487. PG builds the column as sql.unsafe(colId) and runs via this.sql (no read transaction, no JIT wrapper). PGLite runs through this.db.query. Same statements otherwise. eng-parity 5; pglite-engine.test.ts 2; 46 files (e.g. features-no-sync.test.ts, degraded-engine-recovered-surface-9224.test.ts).
getHealth(opts?: {sourceId?, sourceIds?}): BrainHealth (engine.ts:2489) pglite-engine.ts:2780-2787 / postgres-engine.ts:2643-2652 -> shared engine-sql/health.ts:96-273 Three statements, plus two engine callbacks. (1) Counters CTE: entity pages (entityTypePredicateSql, live, not quarantined), chunk totals with embed_skip eligibility, dead_links (link to missing page), link_count, entity_page_count, link_coverage, timeline_coverage. (2) linkable_pages AS MATERIALIZED: grouped by type, islanded and with-timeline counts, using orphanExclusionSql. (3) most_connected: top 5 entity pages by in+out links (ORDER BY link_count DESC, slug COLLATE "C", id). Plus countStalePagesForExtraction and effectiveLinkExtractorWatermark. Score computed in TS: embed 35, link density 25, timeline 15, orphans 15, dead links 10 (health.ts:238-249). Endpoint checks are PK lookups fenced with OFFSET 0 (planner note health.ts:20-27). PG wraps each statement in jitOffRead (postgres-engine.ts:2644-2645). PGLite wraps the whole call in plannerRead (pglite-engine.ts:2781). Same shared SQL. eng-parity 4; pglite-engine.test.ts 10; 54 files (e.g. features-no-sync.test.ts, autopilot-tick-health-7236.test.ts, helpers/legacy-get-health.ts).

Ingest log (2 methods)

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
logIngest(entry: IngestLogInput): void (engine.ts:2492) pglite-engine.ts:2790-2801 / postgres-engine.ts:2655-2666. Inline SQL. INSERT INTO ingest_log (source_id, source_type, source_ref, pages_updated, summary, writer_version, writer_host_id, writer_segment_reader) VALUES (..., $pages_updated::jsonb, ..., $host::uuid, true). source_id defaults to 'default'. Writer stamp from writerStamp({mint: false, engine}) (persistence/writer-versions.ts:36). PGLite passes JSON text; PG uses sql.json(...). eng-parity 0; pglite-engine.test.ts 2; 19 files (e.g. doctor-facts-absorb-history-6761.test.ts, import-cancellation.serial.test.ts).
getIngestLog(opts?: {limit?, sourceIds?}): IngestLogEntry[] (engine.ts:2497) pglite-engine.ts:2803-2820 / postgres-engine.ts:2668-2684. Inline SQL. SELECT id, source_id, source_type, source_ref, pages_updated, summary, created_at FROM ingest_log [WHERE source_id = ANY($::text[])] ORDER BY created_at DESC LIMIT $ (default 50). source_id ?? 'default' fallback applied in TS. PG uses a template fragment for the scope; PGLite picks one of two SQL strings. eng-parity 0; pglite-engine.test.ts 2; 11 files (e.g. e2e/connectors-sync-pglite.test.ts, e2e/sync.test.ts).

Sync: slug, links, aliases, page body (9 methods)

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
updateSlug(oldSlug, newSlug, opts?: {sourceId?}): number (engine.ts:2511) pglite-engine.ts:2823-2828 / postgres-engine.ts:2687-2692 -> shared pages.ts:1157-1177 validateSlug(newSlug), then this.transaction(...). Inside: readPageSnapshot(old, {sourceId, includeDeleted: true}) and createPageVersion snapshot of the old state. UPDATE pages SET slug = $new, updated_at = now() WHERE slug = $old AND source_id = $ RETURNING id. If anything moved: recordRenameAlias (writes slug_aliases, page-state/rename-alias.ts:20) and moveSlugBindings (rename-alias.ts:86). Returns the number of rows moved; zero means no row, no throw. Same. eng-parity 0; pglite-engine.test.ts 2; 31 files (e.g. sync-rename-collision.test.ts).
rewriteLinks(oldSlug, newSlug): void (engine.ts:2512) pglite-engine.ts:2830-2832 / postgres-engine.ts:2694-2699 No-op on both. Links use integer page_id FKs, so a rename needs no link rewrite. Textual [[wikilinks]] in bodies are NOT rewritten. Same. eng-parity 0; pglite 0; 8 files. Listed in the UNCALLED allowlist of test/engine-surface-coverage.test.ts as a documented stub.
resolveSlugWithAlias(slug, sourceOrSources, opts?: {excludePrivate?}): string (engine.ts:2535) pglite-engine.ts:2834-2840 / postgres-engine.ts:2701-2707 Thin wrapper: (await resolveSlugWithAliasDetailed(...))?.canonical_slug ?? slug. Same. eng-parity 2; pglite 0; phantom-redirect-merge.test.ts, managed-facts-writers.test.ts, type-unification-full-flow.test.ts.
resolveSlugWithAliasDetailed(slug, sourceOrSources, opts?): {canonical_slug, source_id} | null (engine.ts:2548) pglite-engine.ts:2842-2848 / postgres-engine.ts:2709-2715 -> shared pages.ts:1194-1223 (unscoped read) SELECT canonical_slug, source_id FROM slug_aliases WHERE alias_slug = $ AND source_id = ANY($sources::text[]) [privacy: EXISTS live page the caller can read] ORDER BY array_position($sources, source_id), id. First row wins; more than one row emits a once-per-process multi_match warning. Pre-v105 brain (missing table slug_aliases): returns null (isMissingSchemaObject). Same. eng-parity 9; pglite 0; e2e/engine-content-privacy.test.ts, schema-error-write-paths.test.ts, fixtures/goldens/sql-text/pages.json.
resolveAliases(aliasNorms: string[], opts?: PageReadScope): Map<alias_norm, Array<{slug, source_id}>> (engine.ts:2567) pglite-engine.ts:2850-2855 / postgres-engine.ts:2717-2722 -> search/read-enrichment.ts:343 readAliases Same function for both, given executeRaw.bind(this). CTE a AS MATERIALIZED (SELECT alias_norm, slug, source_id, rank FROM page_aliases WHERE alias_norm = ANY($1) AND <scope>), rank 2 for origin subject, 1 for declared, 0 for frontmatter. Joined to live readable pages with CROSS JOIN LATERAL (... OFFSET 0). A derived alias (rank > 0) is dropped when another readable page has the exact same lowercase title. Output sorted by alias, source, rank. Throws on pre-v110 brains (caller wraps). Same. eng-parity 0; pglite 0; 22 files (read-aliases-title-scope.test.ts, search/qualified-alias-visibility.test.ts, search/pre-migration-failopen.test.ts).
setPageAliases(slug, sourceId, aliasNorms: string[], opts?: {inline?}): void (engine.ts:2580) pglite-engine.ts:2857-2860 / postgres-engine.ts:2724-2727 -> shared pages.ts:1179-1192 inline and already in a page transaction: run directly; else this.transaction(...). Inside: tx.lockPageKeys([{sourceId, slug}]); DELETE FROM page_aliases WHERE source_id = $ AND slug = $ AND origin = 'frontmatter'; if any aliases: INSERT INTO page_aliases (source_id, alias_norm, slug) SELECT ... FROM unnest($::text[]) ON CONFLICT DO NOTHING. Only frontmatter-origin rows are replaced (derived declared and subject rows are left alone). Same. eng-parity 0; pglite 0; 37 files (migrate-engine-raw-include-deleted.test.ts, recall-entity-keying-4828.test.ts, canonical-writer-inventory.test.ts). See parity risk 8.
refreshPageBody(slug, sourceId, compiledTruth, timeline, contentHash): void (engine.ts:2604) pglite-engine.ts:1329-1337 / postgres-engine.ts:952-960 -> shared pages.ts:405-428 UPDATE pages SET compiled_truth = $1, timeline = $2, content_hash = $3, chunker_version = bodyWriteChunkVersion('$1','$2'), updated_at = now() WHERE source_id = $ AND slug = $ AND deleted_at IS NULL. Leaves frontmatter, chunks, links, embeddings, tags and takes alone (engine.ts:2593-2599: used by the extract_facts phantom redirect). Same. eng-parity 0; pglite 0; 21 files (phantom-redirect-engine-parity.test.ts, fence-chunk-overlay.test.ts, facts-write-path-failures.test.ts, canonical-writer-inventory.test.ts).
updatePageContextualRetrievalState(slug, sourceId, mode, corpusGeneration: string | null): void (engine.ts:2629) pglite-engine.ts:1339-1346 / postgres-engine.ts:962-969 -> shared pages.ts:431-... (transactionMemo passed in) Two shapes. (a) mode !== 'none': UPDATE pages SET contextual_retrieval_mode = $mode, corpus_generation = $gen, updated_at = now() WHERE source_id AND slug AND deleted_at IS NULL. (b) mode === 'none': reads config (search_embedding_column, embedding_columns, embedding_model) through the memo, then one statement with previous AS MATERIALIZED (... FOR UPDATE) and changed CTE that sets the mode and corpus_generation, and for a page whose previous mode was title or per_chunk_synopsis (and not embed_skip) also NULLs embedding_signature, then UPDATE content_chunks sets the active embedding column to NULL, embedded_at, embedded_text_hash, embedding_input_hash to NULL and embedding_pending_since = now(). Same SQL. Doc says this method only stamps the two columns plus updated_at (engine.ts:2612-2628); branch (b) also clears chunk vectors. See parity risk 9. eng-parity 0; pglite 0; 21 files (contextual-retrieval-service-pure.test.ts, persistence-embedding-effects.test.ts, page-projection-vector-preservation.test.ts).
migrateFactsToCanonical(phantomSlug, canonicalSlug, sourceId): {migrated} (engine.ts:2650) pglite-engine.ts:1348-1375 / postgres-engine.ts:971-1007 (1) dropPhantomMigrationDuplicates(tx, sourceId, canonical, phantom) (facts/ontology-rekey.ts:44-47) first removes phantom observations the canonical page already holds (idx_facts_ontology_stint_dedup collision, B-10). (2) UPDATE facts SET entity_slug = $canonical, source_markdown_slug = $canonical, row_num = row_num + COALESCE((SELECT MAX(f2.row_num) FROM facts f2 WHERE f2.source_id AND f2.source_markdown_slug = $canonical AND f2.row_num IS NOT NULL), 0) WHERE source_id AND source_markdown_slug = $phantom AND expired_at IS NULL RETURNING id. NULL row_num stays NULL. Expired rows stay on the phantom slug. Re-run is a no-op. Same SQL. PG issues it through the tagged template on this.sql, PGLite via this.db.query. eng-parity 0; pglite 0; 15 files (phantom-redirect-engine-parity.test.ts, phantom-redirect.test.ts, e2e/phantom-redirect.test.ts).

Config (5 methods)

Config table is config (key, value). Both engines call requestConfigValue / requestConfigRows first (config-snapshot.ts). PG wraps the SQL in connRetry (postgres-engine.ts:2753-2765: withRetry with the bulk-retry options and reconnect), so a dropped pooler connection is retried and the pool rebuilt. PGLite has no retry wrapper.

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
getConfig(key): string | null (engine.ts:2657) pglite-engine.ts:2863-2868 / postgres-engine.ts:2767-2777 Request snapshot first; else SELECT value FROM config WHERE key = $. Returns null when the key is unset. PG only: connRetry. eng-parity 0; pglite 4; 489 files (ubiquitous).
setConfig(key, value): void (engine.ts:2658) pglite-engine.ts:2870-2877 / postgres-engine.ts:2779-2787 INSERT INTO config (key, value) VALUES ($1, $2) ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value; then noteRequestConfigWrite. Autocommit, no explicit transaction. PG only: connRetry (idempotent upsert). pglite 2; 1387 files.
unsetConfig(key): number (engine.ts:2664) pglite-engine.ts:2879-2886 / postgres-engine.ts:2789-2796 DELETE FROM config WHERE key = $. Returns affected rows (0 or 1). Calls noteRequestConfigWrite(key, null). Row count from affectedRows (PGLite) vs count (PG). eng-parity 0; pglite 0; 228 files.
listConfigKeys(prefix): string[] (engine.ts:2670) pglite-engine.ts:2888-2896 / postgres-engine.ts:2797-2808 The prefix is LIKE-escaped (\, %, _) in TS, then SELECT key FROM config WHERE key LIKE $pattern ESCAPE '\' ORDER BY key. Literal prefix match, no glob. PG wraps in connRetry. eng-parity 0; pglite 0; 18 files (config-db-merge.test.ts, search-introspection-ops.test.ts, postgres-engine-config-reconnect.test.ts).
getAllConfig(): Record<string, string> (engine.ts:2684) pglite-engine.ts:2898-2905 / postgres-engine.ts:2810-2821 Request snapshot if one applies (requestConfigRows); else SELECT key, value FROM config (whole table, one round trip). Used by loadConfigWithEngine (about 44 keys per connect, engine.ts:2672-2683). PG wraps in connRetry. eng-parity 0; pglite 0; 20 files (config-request-snapshot.test.ts, read-waves-gbra133.test.ts, requester-scope.test.ts).

Migration support and raw SQL (5 methods)

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
runMigration(version: number, sql: string): void (engine.ts:2687) pglite-engine.ts:2908-2910 / postgres-engine.ts:2824-2827 Runs the migration text unmodified: PGLite this.db.exec(sql) (multi-statement), PG this.sql.unsafe(sqlStr) with no parameters. The version argument is ignored by both (_version). The runner that chooses versions is src/core/migrate.ts (not read in this part). Different APIs: PGLite exec vs PG unsafe; multi-statement behavior must match on both (not verified here). eng-parity 0; pglite 0; 36 files (migrate.test.ts, v273-validate-timeouts.test.ts, function-search-path-pglite.test.ts, oauth-client-registered-via-migration.test.ts).
getChunksWithEmbeddings(slug, opts?: {sourceId?, includeUnsealed?}): Chunk[] (engine.ts:2692) pglite-engine.ts:2912-2914 / postgres-engine.ts:2829-2831 -> shared chunks.ts:1193-1209 (unscoped read) SELECT cc.* FROM content_chunks cc JOIN pages p ON p.id = cc.page_id WHERE <chunkVisibleFilter('p','cc') or chunkGenerationFilter when includeUnsealed> AND p.slug = $ [AND p.source_id = $] ORDER BY cc.chunk_index. Raw preservation tools may include unverified chunks (engine.ts:2691). Identical. eng-parity 0; pglite-engine.test.ts 2; 14 files (e.g. incremental-chunking.test.ts).
executeRaw<T>(sql, params?, opts?: {signal?, timeoutMs?, deadlineAt?, jitOff?, prepare?}): T[] (engine.ts:2714) pglite-engine.ts:2916-2942 / postgres-engine.ts:2990-3018 (+ runUnsafe 2934-2992) PGLite: pre-check signal (AbortError), then db.query(sql, params). Autocommit WAL-writing statements (per writesWal) go through PgliteCheckpointGuard.runStatement (pglite-engine/checkpoint-guard.ts). An abort rejects the caller but the statement keeps running in WASM. timeoutMs, deadlineAt, jitOff, prepare are ignored. Postgres: throwIfRequestAborted(), checkoutGauge 'raw'. If jitOff or timeoutMs (outside a transaction): holdSearchConnection, then JIT-off envelope (jit-off-statement.ts) or runBoundedStatement (BEGIN; SET LOCAL statement_timeout; <sql>; COMMIT as one pipelined round trip, bounded-statement.ts, 57014 on timeout). Else retryReadOnlyAutocommit(this.sql, runUnsafe). runUnsafe reserves a connection when a signal is present, sets prepare (default true, #5984), and on abort sends CancelRequest, then discards the connection after cancelSettleMs. No automatic retry of mutations on connection death (comment postgres-engine.ts:3011-3017). PGLite has no server-side cancel, no timeout, no JIT control. PG retries only a read-only-transaction refusal (SQLSTATE 25006) by reopening BEGIN READ WRITE (read-write-transactions.ts:65-76). eng-parity 46; pglite 1; 3285 files (ubiquitous).
executeRawDirect<T>(sql, params?, opts?: {signal?}): T[] (engine.ts:2732) pglite-engine.ts:2955-2961 (delegates to executeRaw) / postgres-engine.ts:3034-3059 PG: signal pre-check before pool wait; if inside an open transaction (_sql is the reserved tx connection), stays on it; else if dual pool is active, uses connectionManager.ddl() (session-mode direct pool, port 5432 per engine.ts:2720-2729); else this.sql. checkoutGauge 'direct', then retryReadOnlyAutocommit. Single-statement UPDATEs only (Minion lock hot path). PGLite is identical to executeRaw (no pooler). eng-parity 0; pglite 0; 30 files (persistence-diagnostics.test.ts, pool-gauge.test.ts, engine-control-routing.test.ts, sync-run-ordering.serial.test.ts).
keywordBoundedRead<T>(read: (db: Pick<BrainEngine,'executeRaw'>) => Promise<T>): T (engine.ts:2745) pglite-engine.ts:2945-2947 / postgres-engine.ts:1507-1510 PG: resolve keyword_statement_timeout_ms once per connection (keyword-statement-timeout.ts, default 8000, range 1000-60000), then withScopedReadTransaction(undefined, undefined, tx => read(engineSqlOn(tx)), {alwaysTransaction: true, settings: {statement_timeout}}) so every statement in read is bounded and transaction-local. Used for the segmented-region read with its chunk-manifest probe (engine.ts:2740-2744). PGLite: read(this), no timeout. PG-only bound (57014 on overrun); PGLite unbounded. eng-parity 0; pglite 0; 9 files (read-transaction-early-result.test.ts, postgres-rls-scope-inventory.test.ts, engine-surface-coverage.test.ts).

Code edges (6 methods)

Shared SQL: src/core/engine-sql/code-edges.ts. Tables: code_edges_chunk (resolved, to_chunk_id set), code_edges_symbol (unresolved, to_symbol_qualified set; resolved_chunk_id in edge_metadata for resolver-stamped rows). Batch sizes: floor(maxBindParams / binds-per-row) (code-edges.ts:14-16): PGLite 30000 binds, PG unbounded.

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
addCodeEdges(edges): number (engine.ts:2756) pglite-engine.ts:2980-2982 / postgres-engine.ts:3071-3073 -> shared code-edges.ts:31-60 Resolved edges: INSERT INTO code_edges_chunk (from_chunk_id, to_chunk_id, from_symbol_qualified, to_symbol_qualified, edge_type, edge_metadata, source_id) VALUES ... ON CONFLICT (from_chunk_id, to_chunk_id, edge_type) DO NOTHING. Unresolved: INSERT INTO code_edges_symbol (...) ON CONFLICT (from_chunk_id, to_symbol_qualified, edge_type) DO NOTHING. edge_metadata as per-row ::text::jsonb. Returns affected rows. Same SQL; PGLite batches by bind count. eng-parity 1; pglite 0; 20 files (code-edges.test.ts, e2e/code-edges-jsonb-postgres.test.ts).
deleteCodeEdgesForChunks(chunkIds: number[]): void (engine.ts:2765) pglite-engine.ts:2984-2986 / postgres-engine.ts:3075-3077 -> shared code-edges.ts:62-68 DELETE FROM code_edges_chunk WHERE from_chunk_id = ANY(...) OR to_chunk_id = ANY(...), then DELETE FROM code_edges_symbol WHERE from_chunk_id = ANY(...). Both directions for the resolved table, outbound for the symbol table. Empty input returns early. Same. eng-parity 0; pglite 0; 10 files (code-edges.test.ts).
getCallersOf(qualifiedName, opts?: {sourceId?, allSources?, limit?, chunkId?}): CodeEdgeResult[] (engine.ts:2773) pglite-engine.ts:2988-2994 / postgres-engine.ts:3079-3085 -> shared code-edges.ts:75-102 (unscoped read) UNION ALL of code_edges_chunk (to_symbol_qualified = $name or to_chunk_id = $chunkId) and code_edges_symbol (to_symbol_qualified = $name or (edge_metadata->>'resolved_chunk_id')::int = $chunkId), each filtered by currentCodeEdgeFilter(...) (code-intel/read-scope.ts), optional source_id = $ unless allSources. Shared LIMIT default 100, cap 500. Same. eng-parity 0; pglite 0; 24 files (import-code-edges-source-id.test.ts, code-intel/n13-edge-resolution.test.ts, e2e/cli-source-scoping-pglite.test.ts).
getCalleesOf(qualifiedName, opts?: {sourceId?, allSources?, limit?, bareFallback?, chunkId?}): CodeEdgeResult[] (engine.ts:2783) pglite-engine.ts:2995-3000 / postgres-engine.ts:3086-3091 -> shared code-edges.ts:104-134 Edges whose from_symbol_qualified matches (or whose from-chunk is chunkId), same source and current-filter rules. bareFallback (#4670): if zero rows and the name has no ., #, :, re-key on from_chunk_id IN (SELECT id FROM content_chunks WHERE symbol_name = $). Same. eng-parity 8; pglite 0; 18 files (import-code-edges-source-id.test.ts, code-intel/recursive-walk-resolved-ids-5439.test.ts, code-edges.test.ts).
getEdgesByChunk(chunkId, opts?: {direction?, edgeType?, limit?}): CodeEdgeResult[] (engine.ts:2793) pglite-engine.ts:3002-3007 -> PGLite-only pglite-engine/code-edges.ts:17-54 / postgres-engine.ts:3093-3098 -> shared code-edges.ts:141-172 direction default both; limit default 50 cap 200. Two statements: code_edges_chunk (to_chunk_id = $1 for in, from_chunk_id = $1 for out, either for both) and, unless in, code_edges_symbol WHERE from_chunk_id = $1. Each table has its own LIMIT. edgeType bound as $3/edge_type = $. Rows mapped by shared rowToCodeEdge (code-edges.ts:182-195). Two implementations kept in sync by hand (see parity risk 6). eng-parity 0; pglite 0; 12 files (code-edges.test.ts, e2e/code-edges-read-parity.test.ts, code-intel/n13-edge-resolution.test.ts).
searchKeywordChunks(query, opts?: SearchOpts): SearchResult[] (engine.ts:2804) pglite-engine.ts:1638-1724 / postgres-engine.ts:1291-1415 (inline SQL both) Chunk-grain FTS: ts_rank(cc.search_vector, websearch_to_tsquery(lang, $1)) * sourceFactorCase AS score, WHERE cc.search_vector @@ websearch_to_tsquery(...), hard-exclude clause, visibility clause, detail: 'low' restricts to compiled_truth, ORDER BY score DESC, page_id ASC, chunk_id ASC LIMIT OFFSET. CJK queries go to _searchKeywordCJK (dedup: false) on both. No page dedup (that is searchKeyword). Not equivalent. PG additionally honors opts.type, opts.types, opts.where (frontmatter filter), opts.exclude_slugs. PGLite ignores all four. PG returns constant stale = false; PGLite computes stale from timeline_entries. eng-parity 7; pglite 1; 27 files. See parity risk 1.

Eval capture (5 methods)

Table: eval_candidates and eval_capture_failures. Writers and readers are small raw statements. Insert helper: src/core/feedback/store.ts:340 (insertEvalCandidate, shared by both engines).

Method (iface line) Impl PGLite / PG What it does and SQL PGLite vs PG; tests
logEvalCandidate(input: EvalCandidateInput): number (engine.ts:2812) pglite-engine.ts:3010-3012 / postgres-engine.ts:3101-3103 -> feedback/store.ts:340- INSERT INTO eval_candidates (tool_name, query, retrieved_slugs, retrieved_chunk_ids, source_ids, expand_enabled, detail, detail_resolved, vector_enabled, expansion_applied, latency_ms, remote, job_id, subagent_id, embedding_column) VALUES (...) RETURNING id. retrieved_slugs is filtered to slugs that exist (FOR KEY SHARE) or facts/<id> rows that exist, in input order. Same. eng-parity 0; pglite 0; 17 files (eval-candidates.test.ts, eval-capture-drain.test.ts, e2e/eval-replay-column.test.ts).
listEvalCandidates(filter?: {since?, limit?, tool?}): EvalCandidate[] (engine.ts:2814) pglite-engine.ts:3014-3036 / postgres-engine.ts:3105-3130 SELECT * FROM eval_candidates WHERE created_at >= $since [AND tool_name = $tool] ORDER BY created_at DESC, id DESC LIMIT $. limit default 1000, cap 100000; since default epoch. Same SQL; PG via tagged template. eng-parity 0; pglite 0; 11 files (eval-candidates.test.ts, e2e/eval-capture.test.ts, mcp-eval-capture.test.ts).
deleteEvalCandidatesBefore(date: Date): number (engine.ts:2816) pglite-engine.ts:3038-3044 / postgres-engine.ts:3132-3138 DELETE FROM eval_candidates WHERE created_at < $date RETURNING id; returns rows length. Same. eng-parity 0; pglite 0; 7 files.
logEvalCaptureFailure(reason): void (engine.ts:2818) pglite-engine.ts:3046-3051 / postgres-engine.ts:3140-3143 INSERT INTO eval_capture_failures (reason) VALUES ($). Best effort (callers swallow errors, engine.ts:2811). Same. eng-parity 0; pglite 0; 9 files (eval-candidates.test.ts, eval-capture-drain.test.ts).
listEvalCaptureFailures(filter?: {since?}): EvalCaptureFailure[] (engine.ts:2820) pglite-engine.ts:3053-3060 / postgres-engine.ts:3145-3154 SELECT * FROM eval_capture_failures WHERE ts >= $since ORDER BY ts DESC. Same. eng-parity 0; pglite 0; 8 files (eval-candidates.test.ts, mcp-eval-capture.test.ts).

Salience, anomalies and enrich candidates (5 methods)

Shared SQL: src/core/engine-sql/salience.ts. Statement text is Postgres master text (engine.ts comment at salience.ts:2-6); PGLite runs the same statements. Every read is unscoped (LegacyUnscopedRead). Both engines call salienceImpl.* with unscopedExecutor(this.engineSql, ...) (PGLite pglite-engine.ts:3069-3087, PG postgres-engine.ts:3163-3181), except setEmotionalWeightBatch, which passes this.engineSql (a write).

Method (iface line) What it does and SQL Tests
batchLoadEmotionalInputs(slugs?: string[]): EmotionalWeightInputRow[] (engine.ts:2842) salience.ts:27-75. Two CTEs avoid a tag x take cross product: page_tags (array_agg(DISTINCT tag)) and page_takes (json_agg of active takes: holder, weight, kind, active). pages LEFT JOIN both on page_id. WHERE p.slug = ANY($slugs) when given. Each row carries source_id. eng-parity 0; pglite 0; 13 files (recompute-emotional-weight.test.ts, e2e/salience-pglite.test.ts, backfill-base.test.ts).
setEmotionalWeightBatch(rows: EmotionalWeightWriteRow[]): number (engine.ts:2853) salience.ts:77-103. UPDATE pages SET emotional_weight = u.weight, salience_touched_at = now() FROM unnest($slugs::text[], $sourceIds::text[], $weights::real[]) AS u(slug, source_id, weight) WHERE pages.slug = u.slug AND pages.source_id = u.source_id AND pages.emotional_weight IS DISTINCT FROM u.weight RETURNING 1. Composite key on (slug, source_id). Returns changed rows only. eng-parity 0 (salience parity file is separate: e2e/engine-parity-salience.test.ts); pglite 0; 16 files (recompute-emotional-weight.test.ts, e2e/multi-source-emotional-weight-pglite.test.ts).
getRecentSalience(opts: SalienceOpts): SalienceResult[] (engine.ts:2863) salience.ts:105-182. Boundary computed in JS and bound as timestamptz (salience.ts:110). Window: GREATEST(updated_at, COALESCE(salience_touched_at, updated_at)) >= boundary (or updated_at when holders are restricted). Score: emotional_weight * 5 + ln(1 + COUNT(DISTINCT active take id)) + recency component (buildRecencyComponentSql, flat by default, per-prefix decay when recency_bias: 'on'). Excludes briefings/% unless the prefix asks for it; optional slugPrefix LIKE with escape; source scope; pageReadFilter (privacy). GROUP BY p.id ORDER BY score DESC LIMIT (default 20, cap 100). eng-parity 0; pglite 0; 13 files (e2e/salience-pglite.test.ts, e2e/engine-parity-salience.test.ts, e2e/salience-anomalies-source-isolation-pglite.test.ts).
listEnrichCandidates(opts: EnrichCandidatesOpts): EnrichCandidate[] (engine.ts:2877) salience.ts:184-250. Empty types returns [] without SQL. WHERE deleted_at IS NULL AND type = ANY($types) AND (char_length(compiled_truth) + char_length(COALESCE(timeline,''))) < threshold plus source scope, optional re-enrich guard (frontmatter->>'enriched_at' older than cutoff), and dream_generated IS DISTINCT FROM 'true'. inbound_count is a correlated count over links with link_source IS DISTINCT FROM 'mentions' and rankingLinkSql. ORDER BY ENRICH_ORDER_SQL[order] (default inbound-links), LIMIT 1..5000. Lightweight projection, no bodies. eng-parity 2; pglite 0; 12 files (e2e/enrich-pglite.test.ts, e2e/read-enrichment-privacy.test.ts, amends-ranking-scenarios.ts).
findAnomalies(opts: AnomaliesOpts): AnomalyResult[] (engine.ts:2889) salience.ts:252- (two aggregate statements visible at lines 274-320: tag cohort, then type cohort). Baseline window [since - lookback, since), today [since, since + 1 day). Both use generate_series day densification with zero fill (so sparse cohorts are not under-counted). Counts are COUNT(DISTINCT p.id) per cohort per day from pages.updated_at. The final sigma test runs in TS: computeAnomaliesFromBuckets (cycle/anomaly.ts). Default sigma 3.0, lookback 30 days. eng-parity 0; pglite 0; 14 files (e2e/anomalies-pglite.test.ts, e2e/salience-anomalies-source-isolation-pglite.test.ts, e2e/engine-parity-salience.test.ts).

Parity risks noted

  1. searchKeywordChunks filter drift: Postgres honors type, types, where (frontmatter filter) and exclude_slugs; PGLite (pglite-engine.ts:1638-1724) ignores all four. The stale flag also differs (PG constant false at postgres-engine.ts:1398; PGLite computed from timeline_entries at pglite-engine.ts:1713). Any caller that narrows anchors by type gets different results per engine. Rust needs one filter contract tested on both.
  2. Fact insert concurrency: the per-entity advisory lock (facts.ts:120-123, 149-152) exists only on Postgres. PGLite inserts do not serialize the dedup window; a Rust embedded engine that is single-writer is fine, but the dedup contract must not rely on the lock.
  3. createVersion default source: when sourceId is omitted, createPageVersion uses 'default' (pglite-engine.ts:2710, postgres-engine.ts:2566). The interface doc (engine.ts:2459-2463) says the bare-slug lookup picks whichever row Postgres returns first. The code does not do that. Rust should follow the code, or the doc should be fixed first.
  4. insertFacts return alignment: ids drops ON CONFLICT-skipped rows (facts.ts:318), so callers cannot map index to id, contrary to the doc at engine.ts:2236. Keep the contract explicit (return null per input, or document the filter).
  5. Embedding cast: Postgres probes facts.embedding for ::halfvec vs ::vector (postgres-engine.ts:2271-2304); PGLite always casts ::vector. A Rust store with one fixed vector type must not depend on the probe; the dimension and type must be identical to what both engines read back.
  6. getEdgesByChunk has two implementations (pglite-engine/code-edges.ts and engine-sql/code-edges.ts:141-172). The engine-sql header (code-edges.ts:6-10) still says PGLite runs "one UNION with a shared LIMIT", but the PGLite file now applies a per-table LIMIT. Comment and code have drifted; the Rust version needs one behavior.
  7. executeRaw semantics: PGLite ignores timeoutMs, deadlineAt, jitOff, prepare and cannot cancel a running statement. Postgres cancels with CancelRequest and bounds with statement_timeout. Callers that rely on a timeout (search, Minion locks) see no bound on PGLite. The interface doc (engine.ts:2696-2713) describes Postgres behavior.
  8. setPageAliases replaces only origin = 'frontmatter' rows (pages.ts:1188). The interface doc (engine.ts:2572-2579) says it deletes "the page's existing page_aliases rows". Declared and subject aliases survive a frontmatter rewrite on purpose; the doc is out of date.
  9. updatePageContextualRetrievalState mode 'none' also clears chunk vectors and embedding_signature when the old mode was title or per_chunk_synopsis (pages.ts:431-...). The interface doc (engine.ts:2612-2628) says it stamps only the two columns plus updated_at. A Rust port that follows the doc would keep stale vectors.
  10. Segmented flip and staging: flipBodyGeneration relies on FOR UPDATE on the page row and staging row (page-body.ts:294, 309), and stageSegmentChunks on advisory lock plus FOR KEY SHARE (page-body.ts:193-197, Postgres only). PGLite has neither. A Rust embedded store needs its own serialization (a per-page mutex or single writer) and must reproduce the generation checks (revision_changed, body_staging_expired, generation_incomplete).
  11. upsertSegmentDerivations is not atomic across batches: earlier batches commit before DerivationWriteRefused is thrown (page-body.ts:521-542). sweepSegments runs six statements without one transaction (page-body.ts:425-487); its correctness depends on the ordering and grace windows (1 h texts, 1 h idle and 24 h hard staging deadlines, page-body/limits.ts:132-135).
  12. Chunk indexes: stageSegmentChunks derives chunk_index from MAX(chunk_index) + 1 per page and never reuses an index (page-body.ts:219-221). Deleted chunks leave gaps, so a Rust store must not compact indexes.
  13. runMigration runs raw SQL text: PGLite db.exec vs PG sql.unsafe (pglite-engine.ts:2908, postgres-engine.ts:2824). Multi-statement and transaction-control behavior of migration files must be checked on the Rust side. The version argument is ignored.
  14. getStats and getHealth run their own statements on each engine (getStats is inline in both; getHealth shares health.ts but is wrapped differently: PG jitOffRead per statement, PGLite plannerRead). Results must match across engines; the planner-robustness notes at health.ts:20-27 mean a Rust rewrite cannot rely on query plans for correctness.
  15. resolveAliases throws on pre-v110 brains (engine.ts:2563-2566), while resolveSlugWithAliasDetailed returns null on pre-v105 brains (pages.ts:1194-1223). Callers depend on these two different fail modes.
  16. Config: Postgres retries connection-class failures on every config accessor (postgres-engine.ts:2753-2765); PGLite has no equivalent. The request-scoped config snapshot (config-snapshot.ts) is used by both, so reads can be served from a snapshot taken before a write in the same request.
  17. Facts health and recall reads use now() for validity (valid_until > now()) in SQL on both engines, so results depend on the wall clock at read time (TTL honesty, facts.ts:419-427). A Rust engine has to evaluate the same predicate at read time, not at write time.

What a Rust engine must do to pass engine-parity

Scope: test/e2e/engine-parity.test.ts (2762 lines, commit 8bbfd0b5e8), its helpers, and docs/TESTING.md. The file compares the Postgres engine against the in-process PGLite engine. No engine list is parametrized. A Rust engine has to be added by hand to every describe block. Five things block a pass as the file stands:

  1. The file is gated by DATABASE_URL. Without it, 16 of its 17 describe blocks skip, including the PGLite half. Only the getRawData block runs (1 test).
  2. About 46 executeRaw calls and 17 installFixtureChunks calls rely on Postgres SQL and on the page-projection protocol. A Rust engine must either run that SQL or the test must be rewritten to typed methods.
  3. Test 1592 reads EXPLAIN (FORMAT JSON) plans and asserts index names. This is Postgres-only.
  4. The file never compares scores (no toBeCloseTo). Ordering and exact nullability are what matter.
  5. undefined vs null matters in several places (see section 4).

Counts: 17 describe blocks (16 describeBoth + 1 ungated describe). 74 static test( calls. The for (const q of QUERIES) loop at line 137 turns one test into 3 runs. With DATABASE_URL set, 76 tests run. Without it, 1 runs.

1. How the parity test is structured

Gating. SKIP_PG = !hasDatabase() and describeBoth = SKIP_PG ? describe.skip : describe (test/e2e/engine-parity.test.ts:32-33). hasDatabase() is !!process.env.DATABASE_URL (test/e2e/helpers.ts, hasDatabase). The header comment at lines 4-14 says "Always runs the PGLite half". That is wrong: the whole block is inside describeBoth, so PGLite is skipped too. The only always-on block is the getRawData describe at lines 2714-2762. Its arms array (arms, 2716-2723) pushes pglite always and postgres only when !SKIP_PG. That array is the closest existing pattern for a multi-engine list.

Engine construction. Postgres arm: setupDB() or setupLegacyEmbeddingDB() from test/e2e/helpers.ts. Both go through db.ts and PostgresEngine. setupDB TRUNCATEs every table in ALL_TABLES and re-seeds config (test/e2e/helpers.ts, setupDB). setupLegacyEmbeddingDB also configures the gateway to LEGACY_EMBEDDING_CONFIG (test/helpers/legacy-embedding-config.ts: openai:text-embedding-3-large, 1536 dims) and alters the vector column widths. Isolated variant: isolatedPersistencePostgres() in test/helpers/persistence-postgres.ts creates a fresh database per describe (used at 2386-2404 for open_loops only).

PGLite arm: new PGLiteEngine(); await e.connect({}); await e.initSchema(); (for example 126-128, 1368-1370, 1457-1459). Each describe creates both and seeds them with the same function, usually seedX(eng) run twice.

Teardown: afterAll calls pgliteEngine.disconnect() and teardownDB() (30s timeout). beforeAll budgets are 90s, or 180s for the dense-hub block (2201).

Fixtures.

  • Main corpus: SEED_PAGES (49-89) with 6 pages, each with embeddingDim and a one-hot embedding from basisEmbedding(idx, 1536) (35-39, Float32Array of length 1536).
  • QUERIES (112-116): 'fat code thin harness', 'fat code thin harness part 3', 'fat code production'.
  • seedEngine (91-110): putPage + installFixtureChunks for each page, one compiled_truth chunk with token_count = body.split(/\s+/).length.
  • Other seeds: seedRelational (1338-1359), seedFederated (1428-1447), seedTraversal (2023-2039), seedDenseHub (test/helpers/dense-hub.ts, 200 inbound + 200 outbound spokes, ring offsets, 5200 edges), seedAliasOwners (2241-2256), seedKeyset (1624-1639), CJK corpus (1952-1969), seed in the timeline plan block (1561-1568).
  • Helper installFixtureChunks (test/helpers/page-projection.ts) calls readProjectionSnapshot, then installPageProjection(..., {seal: true}). These are in src/core/page-state/projections.ts. They use engine.transaction, engine.lockPageKeys, engine.readPageSnapshot, engine.executeRaw("UPDATE pages SET text_projection_revision=knowledge_revision, search_vector=..."), and page_projection_jobs. So a Rust engine must implement the projection revision model for the fixture helper to work at all.

Comparison style.

  • Most tests run the same call on both engines and expect(pgX).toEqual(pgliteX), or compare new Set(...), or compare a derived shape string.
  • Exact expectations are then pinned against absolute values, so both engines cannot be wrong in the same way.
  • Ordering: toEqual on arrays is order-sensitive. Many tests .sort() first, which means they do not check order (see table).
  • Scores: the only score asserted is DreamVerdict.score exact 0.31 (line 417). Search scores are never compared.
  • Timestamps: checked with instanceof Date, Number.isFinite(getTime()), toMatch(/\.\d{6}Z$/), or exact ISO strings (for example '2026-09-01T23:59:59.000Z' at 2477).

Plugging in a third engine. Concrete steps, based on the current code:

  1. Extend the engine kind union: readonly kind: 'postgres' | 'pglite' (src/core/engine.ts:832), EngineConfig.engine?: 'postgres' | 'pglite' (src/core/types.ts:2343), and the switch in src/core/engine-construct.ts:9-22 (the default throws).
  2. Add an engine list to the test. Two existing helpers show the pattern. test/helpers/two-engine.ts has type EngineKind = 'pglite' | 'postgres' and openTestBrain(kind, settings) through createEngine({engine: kind}). test/helpers/test-backends.ts has TestBackend = 'pglite' | 'postgres', testBackends(), and env GBRAIN_TEST_BACKEND (values postgres, postgres-direct, pgbouncer; anything else throws). The parity file reads neither. Its matrix runs (scripts/e2e-backend-matrix.txt:33, scripts/e2e-matrix.ts:36) only rerun the same file.
  3. Replace pgEngine / pgliteEngine pairs with arms (as in 2714-2729), or add a third variable to each of the 16 beforeAll blocks.
  4. Gate the Rust arm on its own env var, and make the PGLite arm run without DATABASE_URL. Otherwise the Rust comparison can only run where Postgres exists. .github/workflows/e2e.yml:423 runs the file as tier1 engine-parity.
  5. Replace or skip the EXPLAIN-based block (1557-1605) and the jsonb_typeof probes (1744-1752) for Rust. They inspect Postgres catalogs.
  6. Replace the executeRaw seeding with typed calls, or provide a Postgres-SQL subset executor (see section 4, SQL surface).

2. Per-method table: methods covered by engine-parity

Covered means a direct .method( call in test/e2e/engine-parity.test.ts. "Indirect" methods are covered only by helper code (listed in section 3). Line numbers are for the parity file. "Rust must" lists what a passing engine must return or do.

Method (engine.ts) Parity tests (describe / it, line) What is asserted Rust engine must
connect (835) every describe; e.g. 126-127, 1368-1369, 2394 connect({}) must work for PGLite accept {} and open an ephemeral brain
disconnect (836) afterAll blocks, e.g. 133, 1375, 2400 none (cleanup) disconnect without error, release files
initSchema (846) every describe; 128, 1370, 2395 none (must succeed) create schema idempotently; a second call is a no-op
getPage (869) "provenance columns" 688-725 (703-724); "provenance COALESCE" 727-773; "putPage restores soft-deleted" 775-795 (785, 793); "restorePage arc" 2327-2353 (2338-2378); "unscoped getPage tiebreak" 1817-1833 (1819, 1828); "putPage empty-overwrite" 1852-1870; "revival" 909-949 (930-934); "getRawData soft-delete" 2731-2761 returns null on miss (toBeNull); source_path, source_kind, source_uri, ingested_via exact; ingested_at and deleted_at are Date instances; unscoped read prefers source_id='default', then smallest source_id return null (not undefined) on miss; expose the 4 provenance columns plus source_path; ingested_at and deleted_at as Date; ORDER BY (source_id = 'default') DESC, source_id ASC for unscoped reads; includeDeleted option
putPage (891) used in ~68 sites across the file returns a Page with slug; blank overwrite of non-empty page rejects with /refusing to overwrite non-empty page/ (1860) unless allowEmptyOverwrite: true (1864-1868); provenance COALESCE-preserve on re-write without provenance (749-773, title becomes V2, source_kind stays) same blank-overwrite guard and message; COALESCE on provenance columns; restore soft-deleted row under the same slug (revival, 909-949); frontmatter visibility, quarantine, embed_skip, message_id, thread_id, subject carried through (see email citation 440-545)
deletePages (989) "v0.41.19.0 deletePages parity" 849-875 (862-863) returns only slugs that existed (ghosts excluded); pages are then null via getPage (869-874); result is sorted before compare hard delete; return confirmed-deleted slugs; ignore unknown slugs
deletePage (952) 2759 (raw data test) none beyond completion delete one page for {sourceId}
resolveSlugsByPaths (1007) "resolveSlugsByPaths parity" 1008-1031 Map size 2 for 3 input paths; missing path is undefined (1029-1030) return a Map<path, slug>; omit missing keys
softDeletePage (1017) "#3754 ... getLinks/getBacklinks/traversePaths" 797-821 (811); "#3754 traverseGraph" 823-847 (836); "restorePage arc" 2327-2353 (2332-2335); "two-source restore" 2355-2381 (2366-2367); raw data 2739 returns {slug} first time, null second time (2335); row hidden from default reads; links are not cascaded (they stay in the table but are hidden from reads); deleted_at set soft delete: set deleted_at, return {slug} or null; hide from reads and from link/traversal reads
softDeletePages (1050) "#4587 softDeletePages parity" 877-907 (889-893, 905, 922) returns only rows that transitioned (ghost and already-deleted excluded; the pre-deleted row's deleted_at is not re-stamped, comment 883-884); [] on empty input; rows stay recoverable (includeDeleted: true) deleted_at IS NULL predicate; return slugs transitioned; empty input returns []
restorePage (1056) "restorePage arc" 2327-2353 (2346, 2351); "two-source" 2355-2381 (2368, 2376-2377); raw data 2757 true exactly once, then false; scoped restore does not touch another source's row returns boolean; only restores rows with deleted_at IS NOT NULL in the given source
listPages (1079) "ambient recall keyset" 1668-1768 via drainKeyset (1643-1666, call at 1652) keyset cursor {updatedAfterKeyset:{updatedAt, slug}} with sort: 'updated_asc', limit: 3, slugPrefix: 'ks/'; strict greater-than on the (updated_at, slug) tuple (1700-1708); 10-page tie cluster must drain in full with no dupes (1689-1698); batch rows have updated_at as a Date (1662 calls .toISOString()) implement the tuple keyset (updated_at > ts OR (updated_at = ts AND slug > s)); return updated_at as Date; sort by updated_at ASC, slug ASC in that mode
listAllPageRefs (1127) "#4304 listAllPageRefs" 313-327 every row updated_at instanceof Date and finite; source_id::slug list identical in order across engines (324-326) return Date for updated_at; deterministic order
searchKeyword (1198) "top-5 slugs" 137-152 (139-150, 3 queries); "exclude_slugs and type" 154-169 (156-168); "orFallback relaxed" 205-233 (214-232); "orFallback parity" 673-681 (676-679); "hard-exclude" 547-591 (578-590); "detail=high" 593-621 (607-620); "searchTitles parity" 630-668 (652-654); "email citation" 440-545 (466, 502); "CJK" 1948-2014 (1988-1997, 2009-2012) top-1 slug strict equal (146); top-5 set equal (150); exclude_slugs removes slug; type filter returns only that type (167); strict AND query with a missing term returns [] (216, 676); keyword_relaxed === true on every row when orFallback fires (224-245); test/ slugs hidden by default and shown with include_slug_prefixes: ['test/'] (580, 589); CJK order (1990): full ordered slug list equal across engines; sourceIds: ['nonexistent'] returns [] (2011-2012); detail:'high' must keep openclaw/chat/ pages (613-614) and must change ordering on at least one engine (620) strict AND by default; OR fallback opt-in with keyword_relaxed on every row; hard-exclude test/ by default; include_slug_prefixes, exclude_slugs, type, sourceIds; CJK branch (ILIKE or equivalent) that gives the same ordering; detail option that changes ranking; frontmatter email fields on results
searchTitles (1223) "searchTitles orFallback" 235-249 (240-248); "exact-title hit" 630-668 (657-667) title OR-fallback rows all keyword_relaxed === true (244-245); exact title hit found with title tokens absent from body (659); row has chunk_source: 'compiled_truth' and identical chunk_text (665-667) title arm returns SearchResult shape with chunk_source and chunk_text from the representative chunk
searchVector (1224) "searchVector: top result" 251-257 (256); "escalation" 347-387 (381-386); "email citation" 440-545 (470-471, 538-539) top-1 slug equal (256); with 120 near-duplicate chunks on one page and 8 sparse pages, limit: 6 returns exactly 6 distinct pages; top-5 slugs ordered equal (385); result carries message_id/thread_id/source_subject take Float32Array of length 1536; per-page DISTINCT collapse; bounded escalation so a dense page cannot starve the result; return score and projected metadata
upsertChunks (1269) "#4587 revival" 909-949 (915-918, 936-938); "chunkless-page" 1177-1220 (1186-1188) after upsertChunks the default getChunks returns [] (942) because the write is unsealed; getChunks(slug, {includeUnsealed:true}) returns the one chunk (943-945) keep unsealed chunks invisible to sealed reads; expose includeUnsealed
getChunks (1279) "#2555 getChunks sourceIds[]" 951-981 (970-979, 976-977); "revival" 942-945; "CJK chunks" 2683 (ids used) federated sourceIds beats scalar; out-of-grant array returns [] (973-974), never falls through to default; unset opts keep 'default' floor (976-977); embedding is null unless requested (979) embedding: null by default; chunk.id numeric (used as from_chunk_id at 2684)
countChunklessPagesWithContent (1396) "chunkless-page" 1177-1220 (1203-1204) count 2 for the source: cp/stub and cp/skipped (embed_skip with body, #7174); excludes chunked, empty, and quarantined pages count pages with non-empty content and zero chunk rows, excluding quarantined and empty
listChunklessPagesWithContent (1408) same block (1206-1217) ordered ['cp/stub','cp/skipped'] equal (1208-1209); compiled_truth text round-trips (1210-1211); unscoped call with batchSize: 10000 includes cp/stub (1216-1219) same filter, same order (insertion or slug order must match Postgres; test pins equality only)
countStalePagesForExtraction (1438) "stale-page extraction" 1103-1170 (1117-1118, 1144-1145, 1154-1158, 1168-1169) counts: NULL watermark 3; after stamp 2; without versionTs 1; with versionTs: '2026-05-31T00:00:00Z' 2; edited-since arm (updated_at > links_extracted_at) 2 three arms: NULL watermark, versionTs, updated-since
listStalePagesForExtraction (1446) same block (1121-1127, 1137-1140) sorted slugs ['sp/1','sp/2','sp/3']; compiled_truth truthy (1126); updated_at instanceof Date (1127); the row exposes updated_at_iso as a string (1140) return both updated_at (Date) and updated_at_iso (string)
markPagesExtractedBatch (1474) same block (1139-1142, 1150) stamps move the stale count (1144-1145); per-ref extractedAt honoured; defaultExtractedAt used when ref has none (1150) writes links_extracted_at per ref with the given timestamp
addLink (1496) many: 805, 831, 919, 939, 989-991, 1053, 1240-1242, 1354-1358, 1410-1411, 1436-1443, 2032-2036 link visible via getLinks/getBacklinks; cross-source links with fromSourceId/toSourceId (1053); origin options (1442-1443); re-adding after soft delete gives exactly one link (947); missing endpoint: parity test does not check the error upsert on identity key (from, to, link_type, link_source, source ids); accept origin and originSourceId fields
addLinksBatch (1521) "removeLinksByPagesAndSource" 1877-1946 (1897) 5 rows inserted including link_kind: 'typed_ner' (1902-1905) batch insert with link_kind, link_source, from_source_id, to_source_id
removeLinksByPagesAndSource (1564) 1877-1946 (1926-1944) removed count 3 (2 plain mentions + 1 stale typed_ner); survivors ordered: `rlps/to-x markdown
getLinks (1590) federated 1449-1553 (1487-1502, 1504-1515, 1517-1527); soft-delete 797-821 (808); 2032-2036 traverse exact link shape with from_source_id, from_slug, to_source_id, to_slug, origin_source_id, origin_slug, link_type (1475-1485); out-of-grant origin edge withheld (1498-1501); origin_slug is null-able; far endpoint fed/outside excluded under grant (1492) scoped and unscoped reads with identity fields; origin-grant filter (F1); hide soft-deleted endpoints
getBacklinks (1596) soft-delete 797-821 (807, 813-815); federated 1529-1534 (1530-1533); unscoped 1523-1526 empty after the source is soft-deleted (813-815); from_source_id:from_slug equals ['beta:fed/target'] (1533) same as getLinks, in the inbound direction
listLinkSources (1604) "v114 listLinkSources" 983-1006 (996-997, 1003-1005) rows filtered to two sources equal across engines (1001); citation-graph (count 2) comes before manual (count 1) (1005) group by link_source (NULL allowed), ORDER BY count DESC
traverseGraph (1643) "traverseGraph / traversePaths (D7)" 2041-2181 (2070-2116); "#3754 traverseGraph" 823-847 (833, 840-845) node set and depth identical for depths 1-3 (2070-2072); exact depths at depth 3: a=0, b=1, c=2, x=2, d=3 (2079-2084); cycle guard: depth 25 terminates with 5 nodes and max depth 3 (2089-2097); cross-source node hidden with sourceId (2105) and shown with sourceIds (2112); empty for out-of-scope seed (2115-2116); frontierCap: 10 (843) recursive walk with visited set; per-node links array; scope filters on seed, step and links aggregate; frontierCap option
traversePaths (1658) 2041-2181 (2122-2177); soft-delete 797-821 (809, 817-819) ordered edge list from>to:type@depth must match exactly, with ORDER BY depth, from_slug, to_slug (comment 2065); concrete pins at 2128-2141; linkType filter returns one edge (2155); context survives (2157-2158); 'both' emits each edge once (2137-2142) BFS with visited set; ORDER BY depth, from, to; the same edge set for in/out/both
traversePathsDetailed (1672) "traversePaths row cap" 2190-2234 (2212-2213, 2227-2228) truncated: true at depth 3 both-direction from the hub (2214-2215); paths.length <= 5000 (TRAVERSE_PATH_ROW_CAP, src/core/engine-constants.ts:37); all 400 depth-1 edges survive (2219); truncated: false when under cap (2229-2232) cap at 5000 with a +1 probe row; shallowest-first before cut
relationalFanout (1697) "relationalFanout" 1361-1407 (1389-1406) ordered shape equal: `source_id:slug:hop:edge_count:via_link_types:path:canonical_chunk_id(set null)(1385);invested_inin-direction returns exactly['people/ep-inv-a','people/ep-inv-b'](1392);mentions` edges excluded (1399)
findOrphanPages (1830) "findOrphanPages source scoping" 1039-1099 (1063); "#4280 projection" 1275-1334 (1296-1307) mode: 'islanded' (default) vs 'inbound'; scalar and federated scopes; returns {slug, type, quarantined} (1297); quarantined shell has quarantined: true (1306); cross-source inbound counts (A2 comment 1037) return type and quarantined on each row; support both modes; scope by source
addTag (1851) "getTags excludePrivate/liveOnly" 189-203 (193-194); "getTags sourceIds[]" 1468-1473 (1434-1435) tag rows exist for a page; tag on soft-deleted page still stored tag table with (source_id, slug, tag) identity
getTags (1859) 189-203 (198-201); 1468-1473 (1469-1470) excludePrivate: true removes tags from a visibility: 'private' page (199); liveOnly: true returns [] for a soft-deleted page (201); under federated sourceIds only the beta tag (1472) options excludePrivate, liveOnly, sourceIds
addTimelineEntry (1880) "#3754 ..." 797-821 (none direct); "federated" 1428-1447 (1444, 1446); "unscoped timeline plan" 1557-1568 (1565-1566) two entries added via addTimelineEntry to the same page with different dates store date, source, summary, detail
getTimeline (1898) federated 1536-1543 (1537-1538); D5A date window 1545-1552 (1548-1549); unscoped plan 1584-1590 (1586-1589) federated: sorted summaries ['fed event','late event'] (1540); unscoped: ['beta event','default event'] (1589), i.e. newest first (2026-03-03 before 2026-02-02); after / before windows identical (1546-1551) ORDER BY date DESC; cross-source union for unscoped reads; date-window filters
putRawData (1943) 2731-2761 (2736) raw row stored stores data as JSON per (page, source)
getRawData (1949) 2714-2762 (2737-2758) hidden ([]) for every WHERE shape after soft delete (2742-2749); includeDeleted: true returns it (2753-2755); restore brings it back (2758) filter deleted_at IS NULL in all shapes unless includeDeleted
getDreamVerdict (2078) "#4152 dream verdict" 389-438 (412-413, 432) score exact 0.31 after upsert (417); segments and entities are real arrays, not JSON strings (419-421); model, triage_version = 1 (422-423); legacy row (boolean era) has score, triage_version null, segments and entities [] (433-436) store segments/entities as JSON arrays; score as f64; return null for missing score fields (not undefined)
putDreamVerdict (2079) 389-438 (408, 410) upsert on (file_path, content_hash); second put overwrites score (410-417) upsert on PK
insertFact (2215) "facts TTL" 2523-2653 (2557-2592) valid_until in the past hides a fact from active reads; embedding_model round-trips (2583); embedded_text_hash equals md5(fact) (2584); ontology supersession via valid_until + superseded_by (2588-2599) compute embedded_text_hash as MD5 hex of fact; accept valid_until and embedding
listFactsByEntity (2319) 2523-2653 (2580, 2604, 2611) activeOnly: true default excludes lapsed and superseded rows (2628); activeOnly: false returns full history (2640-2643); compared as sorted list activeOnly option; history includes expired rows
listFactsSince (2333) 2605 entitySlug filter; sorted texts equal (2630) filter by created_at >= since
listFactsBySession (2352) 2606 returns the two durable/future facts (2631) session filter; active-only
listSupersessions (2370) 2612 'ttl superseded old' present (2645): valid_until-closed row still listed include rows with superseded_by set
countUnconsolidatedFacts (2382) 2618 4 (2651) count active rows not consolidated
findCandidateDuplicates (2393) 2607-2610 recency arm (2632) returns active rows; embedding arm ['ttl embed live'] (2634, 2638); keyword arm k:5, arm:'keyword' ranks by shared terms, most first (2636) three arms (recency, embedding with embeddingModel, keyword); exclude lapsed rows
findTrajectory (2422) "findTrajectory newest" 171-187 (183-186) with limit: 3 over 6 points (values 0..5 by date), returns [3,4,5] in ascending date order (185) take newest N then return ascending by date
getFactsHealth (2425) 2602-2649 (2647-2649) total_active: 4, total_expired: 3, top_entities sorted ENTITY:3, EMB_ENTITY:1 counts with validity rules (lapsed and closed = expired)
getStats (2488) "#4280 ..." 1275-1334 (1328-1333) raw counts: pages_by_type {person: 4, atom: 1} (1330), page_count 5 (1332); quarantined shell still counted as person count all rows, no policy filter
getHealth (2489) "#4280 ..." (1313-1323) entity_page_count: 3, linkable_page_count: 3, orphan_pages: 3, page_count: 5 (1323); quarantined shell and atom excluded from denominators apply quarantine and type policy in health math
resolveSlugWithAlias (2535) "resolveSlugWithAliasDetailed" 2288-2303 (2297-2298) string equals the detailed canonical (2297-2298) return canonical slug string
resolveSlugWithAliasDetailed (2548) 2258-2286 (2278-2285, 2293-2296) {canonical_slug, source_id}; winner follows scope order (['par-b','par-a'] gives par-b, 2296); scope with no match returns null (2284-2285); federated ['par-a','par-b'] returns par-b for par/old-b (2283) ordered scope resolution with first-match-wins
executeRaw (2714) 46 call sites, e.g. 173-175, 195, 270, 427, 953, 1014-1017, 1107, 1151, 1163, 1180, 1232-1239, 1279, 1429, 1562-1567, 1629, 1681, 1744, 1788, 1910, 2024, 2243-2250, 2359, 2414, 2490, 2553, 2596 seeds and checks using raw SQL: INSERT ... SELECT generate_series, now() - interval '2 hours', ON CONFLICT DO NOTHING/UPDATE, $1::timestamptz, '{}'::jsonb, UPDATE pages SET deleted_at = now(), DELETE FROM facts, ANALYZE (1567), SELECT jsonb_typeof(...) (1744-1749) run Postgres-dialect SQL, or provide a typed replacement (see section 4)
addCodeEdges (2756) "getCalleesOf bare-name" 2659-2708 (2684) edge stored with from_chunk_id = chunk id, to_symbol_qualified, edge_type: 'calls' store code edges keyed by chunk id
getCalleesOf (2783) 2659-2708 (2699-2705) exact name returns 0 rows without the option (2699); bareFallback: true returns ValidateRequest for bare SubmitAsync (2700-2701); qualified and delimited names unchanged (2702); wrong qualifier or Submit_sync returns 0 (2703-2704); scope to unknown source returns 0 (2705) exact match first; bare-name fallback on symbol_name only when option set and input has no delimiter
searchKeywordChunks (2804) 468-469, 536-537, 2001-2008 (2002-2006) chunk-grain ordered slug#chunk_index equal (2004); top slug notes/tokyo-meeting (2005) chunk-level keyword rows with CJK fallback
listEnrichCandidates (2877) "v0.41.39 listEnrichCandidates" 1222-1266 (1246, 1257-1264) filtered rows slug:inbound_count:body_len equal and ordered (1258); long page excluded by thinThreshold: 400 (1262); order alice(2) > bob(1) > widget(0) by inbound (1263-1264); inbound_count for alice is 2 (1265) thin filter on body length; source-aware inbound count (NULL link_source counts); ORDER BY inbound DESC

Indirect coverage (no direct .method( in the parity file, but reached through helpers used in it):

  • readPageSnapshot, lockPageKeys, transaction: via installFixtureChunks (test/helpers/page-projection.ts, 17 call sites) and readProjectionSnapshot / sealPageTextProjection (src/core/page-state/projections.ts:39, :159).
  • relationalChainHop: via runRelationalChain in the chain test (1409-1421; helper src/core/search/relational-chain.ts).
  • getBacklinks, getLinks, getTimeline, listFactsByEntity, resolveAliases, searchKeyword: via buildEntityCard (259-311; src/core/verbs/entity-card.ts).
  • getConfig, getLinks, getBacklinks, transaction: via linkEntityIdentity / listEntityIdentities (329-345; src/core/entity-identity.ts).
  • executeRaw: via upsertSessionContextState / getSessionContextState (1711-1767; src/core/context/session-state.ts) and upsertOpenLoop / closeOpenLoop / listOpenLoops (2383-2521; src/core/loops/loops-store.ts).
  • engine.kind and executeRaw("EXPLAIN (FORMAT JSON) ..."): via timelinePageScans (test/helpers/timeline-plan.ts, used at 1595).

3. Engine methods with NO parity coverage

Total BrainEngine members in src/core/engine.ts:830-2890: 197. Covered directly: 60 (section 2). Not covered by the parity file: 137. The list below is grouped; test files are a sample from rg -l '\.<name>\b' test and are not exhaustive. Files marked "helpers only" are referenced only from test/helpers/*.ts case tables (for example postgres-engine-sql-cases.ts, engine-sql-rollback-cases.ts).

  • Lifecycle / transaction: reconnect (pglite-reconnect.serial.test.ts, supervisor-health-reconnect.test.ts), holdSchemaMigrationLock (no .holdSchemaMigrationLock( caller found in test/), transaction (trust-channel-writes.test.ts, client-grants.test.ts), transactionDirect (autopilot-transaction-fence.test.ts, engine-control-routing.test.ts), registerBeforeDisconnect (persistence-reconnect.serial.test.ts), withReservedConnection (migrate.test.ts, pool-gauge.test.ts), runMigration (v273-validate-timeouts.test.ts).
  • Pages: readPageSnapshot (fact-withdrawal-normalized.test.ts, indirect here), readPageSnapshotsBatch (postgres-rls-scope-inventory.test.ts), lockPageKeys (belief-ledger-purge.test.ts; indirect here), findDuplicatePage (import-dedup-frontmatter-id.test.ts), findDuplicatePages (helpers only), purgeDeletedPages (pages-soft-delete.test.ts, purge-deleted-dryrun-postgres.test.ts), findPages (helpers only), countPagesBy (list-pages-empty-type-notice-gbra449.test.ts), resolveSlugs (pglite-engine.test.ts), getAllSlugs (integrity-batch.test.ts), listAllSources / updateSourceConfig (list-all-sources-postgres.test.ts), listPrefixSampledPages / listCorpusSample (engine-surface-coverage.test.ts, page-retrievals.test.ts), updateSlug (origin-guard-6948.test.ts), rewriteLinks (engine-surface-coverage.test.ts), resolveAliases (alias-hop.test.ts; indirect here), setPageAliases (by-mention.test.ts), refreshPageBody (fence-chunk-overlay.test.ts), updatePageContextualRetrievalState (import-revision-concurrency.test.ts), migrateFactsToCanonical (phantom-redirect.test.ts), getPageTimestamps (engine-surface-coverage.test.ts), getEffectiveDates (read-enrichment-privacy.test.ts).
  • Search / retrieval: countKeywordPages (search-keyword-paging-parity.test.ts), searchKeywordPages (keyword-arm-access.test.ts), getEmbeddingsByChunkIds (search-quality.test.ts), getChunkPage (get-chunks-paging.test.ts), getChunkWindows (evidence-delivery-parity.test.ts), keywordBoundedRead (search-read-keyword-bound.test.ts), getChunksWithEmbeddings (pglite-engine.test.ts).
  • Embedding state: countStaleChunks (embedding-state.test.ts), sumStaleChunkChars (embedding-signature-stale.test.ts), setPageEmbeddingSignature (embedding-restamp-w3.test.ts), invalidateStaleSignatureEmbeddings (embedding-signature-stale.test.ts), invalidateContentDriftEmbeddings (content-drift-sweep-postgres.test.ts), listStaleChunks (embed-stale-pagination.test.ts), readEmbeddingStateCounts (helpers only), listEmbeddingStateRows (embedding-state.test.ts), deleteChunks (delete-chunks-lock-order-8502.test.ts), markPagesAttendanceBlocked (helpers only), countStaleFrontmatterKeyPages (repair-frontmatter-keys-sweep-6454.test.ts), listStaleFrontmatterKeyPages / stripStaleFrontmatterKeys (helpers only).
  • Links / graph: replaceDerivedLinks (link-cross-source.test.ts), replaceDerivedLinksBatch (extract-endpoint-revision.serial.test.ts), removeLink (pglite-engine.test.ts), findByTitleFuzzy (title-fuzzy-index.test.ts), getBacklinkCounts (read-enrichment-privacy.test.ts, jit-off-reads-postgres.test.ts), getAdjacencyBoosts (graph-signals-engine.test.ts), getContentFlagsByPageIds (quarantine-cli.test.ts), getUnverifiedExtractionPageIds (extraction-review.test.ts), getSalienceScores (engine-parity-salience.test.ts), relationalChainHop (relational-chain.test.ts; indirect here).
  • Tags / timeline / chronicle: removeTag (pglite-engine.test.ts), addTimelineEntriesBatch (timeline-date-batch.test.ts), getTimelineForDate (chronicle-event-projection-parity.test.ts), getSince (chronicle-timeline-reads.test.ts), getOnThisDay (chronicle-delight.test.ts), getLastSeen (chronicle-last-seen-postgres.test.ts), upsertEventProjection (chronicle-event-projection-parity.test.ts), mergeOntologyFact, getOntology, discoverOntologyDimensions, findOntologyConflicts (ontology-merge-parity.test.ts, chronicle-ontology.test.ts).
  • Files: upsertFile, listFilesForPage (engine-upsertFile.test.ts), getFile (import-image-file.test.ts).
  • Takes: addTakesBatch (takes-engine.test.ts), updateTakeEmbeddings, countStaleTakes, listStaleTakes (takes-vector-lifecycle-5885.serial.test.ts), listTakes (takes-source-scope.test.ts), searchTakes, searchTakesVector (takes-postgres.test.ts), getTakeEmbeddings (takes-engine.test.ts), updateTake, supersedeTake, addSynthesisEvidence (takes-engine.test.ts), resolveTake (take-resolution-survives-publish.test.ts), getScorecard, getCalibrationCurve (takes-scorecard-parity.test.ts).
  • Dream / contradictions: sweepDreamVerdicts (dream-verdict-cache-ttl.test.ts), listActiveTakesForPages, writeContradictionsRun, loadContradictionsTrend, getContradictionCacheEntry, putContradictionCacheEntry, sweepContradictionCache (eval-contradictions-engine.test.ts, eval-contradictions-postgres.test.ts).
  • Facts: insertFacts (fence-chunk-overlay.test.ts, extract-facts-reconcile-drain.test.ts), deleteFactsForPage (facts-fence-reconcile-postgres.test.ts), expireFact, consolidateFact (facts-engine.test.ts), listFactsKeyset (engine-sql-facts-keyset.test.ts, turn-context.test.ts).
  • Segments / body generations: allocateBodyOccurrenceIds, stageSegmentTexts, readSegmentTexts, stageSegmentChunks, createSegmentedPageShell, flipBodyGeneration, openBodyStaging, setBodyStagingState (engine-sql-page-body.test.ts), stageManifestRows (segmented-storage-visibility.test.ts), readManifest (edit-segmented.test.ts), sweepSegments (lp-h-purge-orphan-segment-text.test.ts), readSegmentDerivations (segmented-conversation-facts.test.ts), upsertSegmentDerivations, retireSegmentDerivations (helpers only: segment-derivation-cases.ts).
  • Versions / ingest: createVersion (generation-identity-visibility-9386.test.ts), getVersions (persistence-version-deletion.test.ts), revertToVersion (segmented-versions.test.ts), logIngest (import-cancellation.serial.test.ts), getIngestLog (sync.test.ts).
  • Config: getConfig (conversation-facts-managed.test.ts; indirect here), setConfig (trust-release-strict-home-gst2676.test.ts), unsetConfig (doctor-fence-integrity.test.ts), listConfigKeys (config-db-merge.test.ts), getAllConfig (config-request-snapshot.test.ts).
  • Code intelligence: deleteCodeEdgesForChunks (code-edges.test.ts), getCallersOf (cli-source-scoping-pglite.test.ts), getEdgesByChunk (code-edges.test.ts), executeRawDirect (persistence-diagnostics.test.ts).
  • Eval capture: logEvalCandidate, listEvalCandidates, deleteEvalCandidatesBefore, logEvalCaptureFailure, listEvalCaptureFailures (eval-candidates.test.ts, eval-capture.test.ts).
  • Salience / anomalies: batchLoadEmotionalInputs (salience-pglite.test.ts), setEmotionalWeightBatch, getRecentSalience, findAnomalies (salience-anomalies-source-isolation-pglite.test.ts; engine-parity-salience.test.ts covers them cross-engine but is DATABASE_URL-gated, like this file).

Note for the Rust plan: many uncovered methods (for example getRecentSalience, findAnomalies, getContradictions*, takes) are covered only by PGLite unit tests, so parity for them is not checked anywhere. The Rust engine will need its own parity cases if they matter.

4. Cross-cutting requirements a Rust engine must meet

Error classes and messages.

  • The parity file asserts one error message: refusing to overwrite non-empty page (/refusing to overwrite non-empty page/, line 1860). The Rust engine must throw an error whose message contains that text, from putPage.
  • PageMissingError (src/core/engine-errors.ts:18-35, messages addLink failed: <endpoint> page "<slug>" (source=<id>) not found and addTimelineEntry failed: page ...) is not asserted in this file. Other tests check it, so the Rust engine still needs it.
  • Rejections are expected for blank overwrites only. Other operations return null, [], false or 0 for misses.

Null vs undefined vs missing.

  • toBeUndefined() for absent email fields: result?.message_id, thread_id, source_subject must be undefined (505-507, 541-543). Returning null fails the test. Omit these keys when not applicable. thread_id is set when frontmatter has it, even for non-email pages (506).
  • toBeNull() for absent dream fields: score and triage_version must be null, while segments and entities must be [] (433-436).
  • getPage miss: toBeNull() (785, 899, 1862 etc.). getChunks embedding: toBeNull() when not requested (979). deleted_at: null or Date; the test uses deleted_at ?? null (934) so either null or undefined for live rows would pass there, but getPage(... includeDeleted) requires a Date (2343).
  • resolveSlugsByPaths miss: Map.get() returns undefined (1029-1030), not null.
  • resolveSlugWithAliasDetailed no match: toBeNull() (2284-2285).
  • restorePage / softDeletePage: booleans and {slug} or null (2335).

Timestamps.

  • Date objects (must be instanceof Date): listAllPageRefs.updated_at (319-320), listStalePagesForExtraction.updated_at (1127), getPage.ingested_at (723-724), getPage.deleted_at (2343), listPages rows updated_at (1662 calls .toISOString() on them).
  • ISO strings: EntityCard.last_touched.last_timeline_date (298, 304) and open_threads[].date must be string | null (299-301). listStalePagesForExtraction.updated_at_iso is a string (1140). session_context_state.last_wake_at must match /\.\d{6}Z$/ (1735) and equal the input instant 2026-08-05T12:00:00.000Z after new Date() (1734), so microsecond precision is required there. Loops: due_at equals '2026-09-01T23:59:59.000Z' (2477), i.e. millisecond precision; opened_at is a string (2455). Parity compares strings across engines exactly (1736), so both engines must format identically.
  • Timeline date is a string (src/core/types.ts:1742). created_at is a Date (types.ts:1746).
  • Keyset cursor compares updated_at as a timestamp plus slug. The keyset fixtures use whole-millisecond values (1616-1618), while session_context_state.last_wake_at is pinned at microsecond precision (1730-1735).

IDs.

  • Numeric. chunk.id is passed back as from_chunk_id (2683-2684), so it must round-trip as a number.
  • loopId: 2 ** 40 (2511) must work without error and return zero rows (beyond int4). Use 64-bit ids for open_loops and any table with a serial id in this file.
  • canonical_chunk_id is compared only for presence (1380-1385), because its absolute value depends on sequence state. page_id is normalized to 0 in the chain test (1416), so exact values are not compared.
  • slug_aliases, pages.id, facts.id are used as opaque numbers (newRow.id, 2592-2599).

JSON shapes.

  • dream_verdicts.segments and entities must be JSON arrays, not JSON strings (419-421). The Postgres path had a double-encode regression (#2339). A Rust engine must store native JSON arrays and return them as arrays.
  • session_context_state.standing_entities and surfaced_slugs must be JSON arrays in storage: jsonb_typeof = 'array' (1744-1752). A Rust engine cannot run that SQL probe; it needs an equivalent typed check.
  • loops.evidence must come back as an array with message_id and quote (2448-2452).
  • pages.frontmatter keys drive search result metadata (message_id, thread_id, subject gated by an email identity, 440-545). subject alone must not set source_subject (499-508).
  • Tag/link/timeline keys, context text, and fact text must round-trip byte-for-byte (for example 'a verbatim line with "quotes" and unicode', line 400).

Source scoping.

  • Scalar sourceId, federated sourceIds[], and unset (default floor 'default') must behave as in 951-981 and 2104-2116.
  • Out-of-grant sourceIds returns empty and never falls back to 'default' (973-974).
  • Unscoped link reads must expose exact from_source_id/to_source_id for every row (1521-1526).
  • Origin filter (F1): an edge whose origin_source_id is outside the grant is hidden under sourceIds (1442-1443, 1498-1501).
  • Unscoped getPage and getTimeline pick the default source first (1819-1830, 1589).

Soft delete and projection visibility.

  • Soft delete sets deleted_at and does not cascade (889-907). Links and tags stay in storage but are hidden from reads (797-821, 823-847).
  • Re-adding a soft-deleted slug revives the same row (909-949): deleted_at clears, content updates, chunk and link sets are replaced, not duplicated (942-948).
  • Unsealed chunks are hidden from default getChunks until sealed (942-945).

Vector search.

  • Input: Float32Array of length 1536 (basisEmbedding, line 35). Stored vectors must accept 1536 dims (and legacy 1536 via setupLegacyEmbeddingDB).
  • Test vectors are one-hot basis vectors, so many scores are exactly 0 or 1 and ties are common. The test asserts only top-1 (256), a Set of 6 pages (386), and the ordered top-5 on the escalation fixture, whose scores are distinct: 0.99 - i*0.0005 (367) and 0.6 - p*0.001 (375). A tie-break rule is therefore not pinned by the parity file.
  • Float tolerance: none asserted. Dream score is compared exactly (0.31, 417), so store scores as f64.
  • Escalation (347-387): a dense page with 120 chunks must not starve the result. Expect 6 distinct pages with limit: 6. The pre-fix engine returned 1 page. The inner candidate pool is 100 chunks (comment 349-351).

Keyword ranking.

  • Strict AND by default. OR fallback only when orFallback: true, and then every row carries keyword_relaxed: true (205-233, 673-681).
  • Top-1 slug must equal across engines for the three QUERIES (139-146). Top-5 must be the same set (150). Ordering beyond top-1 is not required, except where the test compares ordered arrays (CJK 1990, 1997; typed 168).
  • detail: 'high' must change ordering on at least one engine (593-621), and must still include chat pages (612-614).
  • CJK queries (東京 会議, 미팅 김대리, 予算) must return the matching page as top-1 with identical ordered slugs on both engines (1987-2006). The corpus includes an ASCII decoy page (1955), but the assertions check only CJK top-1 and ordered equality, not the decoy's absence.
  • Title arm: exact title tokens must hit a page whose body lacks them (630-668).

Traversal and caps.

  • traversePaths order is depth, from_slug, to_slug (2065). traverseGraph node order is not compared (sorted in tests).
  • Cycle guard: A to B to A must terminate at depth 25 with 5 nodes and max depth 3 (2089-2097). Each edge appears once in 'both' (2137-2142).
  • Row cap: TRAVERSE_PATH_ROW_CAP = 5000 (src/core/engine-constants.ts:37), one probe row beyond it, and truncated flag (2212-2232). Shallowest edges survive the cut (2218-2219).

Time-based fixtures.

  • Facts TTL uses Date.now()-relative valid_until (2551-2568). The Rust engine must compare against its own clock consistently with now() used in raw SQL seeds (for example 1164, 1629, 2596). Clock skew between client and DB broke a test once (1131-1135), so pick one clock source.
  • Entity card counts new Date().toISOString().slice(0,10) as a timeline date (289).

Chronicle, relational, and other helpers. buildEntityCard (src/core/verbs/entity-card.ts) expects exact active_fact_count 105 for remote and 106 for local (297, 309-310). This needs listFactsByEntity (world-only for remote), getTimeline, getLinks, getBacklinks, resolveAliases, and searchKeyword.

Parity risks noted

  • Harness is Postgres-bound. test/e2e/helpers.ts (setupDB, setupLegacyEmbeddingDB, teardownDB) uses src/core/db.ts and PostgresEngine directly, and TRUNCATEs tables. A Rust engine needs its own setup helper; there is no engine-level switch to reuse.
  • Gating hides PGLite. Everything except the getRawData block is inside describeBoth. Without DATABASE_URL, the file runs 1 test. The header comment (lines 12-14) and docs/TESTING.md:509 ("each E2E file keeps a PGLite arm in the unit lane") both say otherwise. A Rust arm will be hidden the same way unless the gating changes.
  • Raw SQL surface. 46 executeRaw sites plus installFixtureChunks rely on Postgres SQL (now(), interval, generate_series, ON CONFLICT, ::jsonb, ::timestamptz, ANALYZE) and on the projection tables (page_projection_jobs, text_projection_revision, knowledge_revision, search_vector). An embedded Rust engine that does not run this SQL cannot pass without test rewrites.
  • Postgres-only introspection. test/helpers/timeline-plan.ts runs EXPLAIN (FORMAT JSON) and asserts Index Name / Index Cond (1592-1604). Replace it with a Rust query-plan assertion or drop it for Rust.
  • Catalog probe. jsonb_typeof checks (1744-1752) inspect the stored type in Postgres. Rust needs an equivalent typed check or to rely on the read-back shape.
  • Null vs undefined. toBeUndefined (505-507, 541-543) vs toBeNull (433-435) on different fields. A generic "null for missing" serializer fails one set of tests.
  • Weak assertions hide drift. No score comparison; detail=high only asserts "differs on at least one engine" (620); listPages/listAllPageRefs compare parity but not absolute order; getTimeline federated results are sorted. A Rust engine can pass these tests while ordering differs. Add stronger Rust-only checks where ordering matters.
  • Fixture protocol coupling. installFixtureChunks (17 uses) depends on the projection revision protocol in src/core/page-state/projections.ts. Any Rust storage model must implement that protocol, or the fixture must be rewritten to call a public seal API.
  • Unsealed chunk visibility. upsertChunks writes are invisible to default getChunks until sealed (942-945). A Rust engine that exposes chunks immediately fails 942.
  • Uncovered surface. 137 of 197 interface members have no parity test (section 3). Many are covered only by PGLite-only tests (salience, takes, contradictions, ontology, segments).
  • Clock coupling. Facts TTL and stale-extraction tests mix Date.now() (client) and now() (server) clocks (1131-1135, 2551-2568, 2596). A Rust engine with one clock may disagree with the test's expectations at boundaries.
  • Engine-kind plumbing is closed. BrainEngine.kind, EngineConfig.engine, and constructEngine accept only postgres and pglite (src/core/engine.ts:832, src/core/types.ts:2343, src/core/engine-construct.ts:9-22). A Rust kind requires changes in all three.
  • Matrix runs do not test a third engine. scripts/e2e-backend-matrix.txt:33 and .github/workflows/e2e.yml:423 only rerun this file against direct Postgres and PgBouncer. GBRAIN_TEST_BACKEND is not read by this file, so the matrix never exercises a new backend unless it is wired into testBackends() and this file.

Adding a third engine and binding options

Base path for all repo citations: the gbrain repo root (commit 8bbfd0b5e8, read-only). Paths below are relative to it.

Engine selection today

How an engine is chosen. GBrainConfig.engine is the two-value union 'postgres' | 'pglite' (src/core/config.ts:37). It is loaded from ~/.gbrain/config.json (config.ts:196; the directory honors GBRAIN_HOME, config.ts:1977). loadConfig infers the engine: a DATABASE_URL env var forces postgres; otherwise the file's engine wins, and a file with only database_path is pglite (config.ts:738-746). toEngineConfig copies engine, database_url, database_path into the runtime EngineConfig (config.ts:1968-1975).

The discriminator. EngineConfig.engine?: 'postgres' | 'pglite' is at src/core/types.ts:2343. BrainEngine.kind: 'postgres' | 'pglite' is at src/core/engine.ts:832. DegradedEngineOptions.kind is at src/core/degraded-engine.ts:63. EngineSettings.engine (embedding host) is at src/core/embedded-engine.ts:14. EngineStatusReport.effective_engine and config_file_engine are at src/commands/engine-status.ts:108.

Construction chain.

File Role Lines
src/core/engine-construct.ts constructEngine switch: case 'pglite' dynamic-imports pglite-engine.ts, case 'postgres' imports postgres-engine.ts; default throws Unknown engine type ... Supported engines: postgres, pglite 12-23
src/core/engine-factory.ts createEngine = constructEngine + chat-usage sink registration + wraps disconnect whole file (31 lines)
src/core/engine-connect.ts connectEngineForServe: loadConfig, createEngine(toEngineConfig(config)), then connectWithRetry; PGLite one-shot reads set pglite_shared_buffers, Postgres one-shot sets pool_max: 2 76-110 (one-shot constants 34-42)
src/core/engine-open.ts Experimental openEngine(settings) for embedding hosts. checkSettings refuses any engine other than the two (:94); SETTINGS_FIELDS names database: ['url','directUrl','path']; constructEngine at :166; PGLite only accepts database.path, Postgres only database.url (:92-106) 94-106
src/core/embedded-engine.ts Settings types and markEmbeddedEngine registry; engine-kind-agnostic except the engine union 11-20
src/core/degraded-engine.ts Lazy-reconnect proxy for gbrain serve when Postgres is down at startup. Scope is Postgres only (header comment, lines 17-24); kind is an explicit getter (:234-235) that defaults to postgres while dead 17-24, 63, 116-240
src/core/engine-key.ts, src/core/engine-root.ts engineKey and ENGINE_ROOT symbol; kind-agnostic (no change needed) whole files

Config and CLI surfaces.

  • gbrain init --pglite sets isPGLite (src/commands/init.ts:56). Bare init with an existing Postgres config is preserved unless --pglite is passed (init.ts:170-181). PGLite init creates createEngine({engine:'pglite'}) at init.ts:1144. Init hints and error text name gbrain init --pglite --no-embedding (src/core/engine-connect.ts:59-64, init.ts:598-609).
  • --pglite appears 54 times in 18 files, including src/commands/init.ts, setup.ts, bootstrap.ts, reinit-pglite.ts, doctor/checks/db-connection.ts, doctor/checks/pglite-worker.ts, src/mcp/status-mode.ts, src/cli/top-help.ts, src/core/error-registry.ts, src/core/cli-flag-registry.generated.ts (generated; regenerate, not hand-edit). Exact generator not verified.
  • gbrain engine status [--json] [--probe] [--brain <id>] (src/commands/engine-status.ts:1-80) is engine-free. It dispatches before connectEngine. PGLite lock state is reported via inspectLockHolder (pglite_lock). It has no generic lock or probe path for a non-SQL engine.
  • gbrain reinit-pglite refuses unless cfg.engine === 'pglite' (src/commands/reinit-pglite.ts:42).
  • gbrain migrate --to postgres|pglite and graduation: src/commands/migrate-engine.ts, migrate-graduation.ts.

Size of the branch surface. Counts are regex-based, so treat them as approximate.

Pattern Count Where
'pglite' string literal (all of src/) ~353 occurrences in ~186 files broad
'postgres' string literal (all of src/) ~339 occurrences in ~171 files broad
Strict `kind ===/!== 'pglite' 'postgres'` ~268 sites
sqlFor per-engine SQL 42 uses in 34 files schema-migrations/types.ts:29 (sqlFor?: { postgres?: string; pglite?: string }); picked at src/core/migrate.ts:548 (m.sqlFor?.[engine.kind] ?? m.sql)
Transaction branch on kind 1 src/core/migrate.ts:221 (`useTransaction
Doctor checks mentioning PGLite 37 of 207 check files src/commands/doctor/checks/ (e.g. pglite-worker.ts)

Main categories of branch:

  1. Migrations. sqlFor dialect picks, pglite-only repair and lock migrations. A non-SQL engine can use the engine-agnostic sql fallback only if its migrations are dialect-free, which current raw-SQL migrations are not.
  2. Persistence and locking. The PGLite single-writer lock (src/core/pglite-lock.ts, PgliteBusyError at lines 19-25) and the persistence owner socket (src/core/persistence/local-client.ts, "Engine-free CLI routing to the process that already owns a PGLite brain"). The owner socket is the existing single-writer routing for CLI, MCP and cron.
  3. Doctor and repair. commands/doctor/checks/*, pglite-repair.ts, pglite-resetwal.ts, pglite-leftovers-check.ts, db-repair.ts.
  4. Ops and searches. Postgres-only features (pg_trgm, statement timeouts, pool) are gated on kind === 'postgres'. Engine-sql domain modules (src/core/engine-sql/) have two dialect adapters, dialect-pglite.ts and dialect-postgres.ts.

Docs. docs/ENGINES.md says "Two engines ship today" (line 7). The "Adding a new engine" guide is at docs/ENGINES.md:1566-1596. It says a non-SQL engine "implements every method itself" (line 1594 onward). Its step 4 says to store {"engine":"myengine"} in config. That is not accurate today: GBrainConfig.engine and EngineConfig.engine are closed unions (config.ts:37, types.ts:2343), and constructEngine throws on unknown values (engine-construct.ts:22). Adding a third kind needs type changes too. The capability matrix at ENGINES.md:1598-1611 covers only the two engines.

What adding a third engine kind would touch

Minimum set (non-SQL Rust store, kind 'rust' as a placeholder name):

# Change File(s)
1 Widen BrainEngine.kind union src/core/engine.ts:832
2 Widen EngineConfig.engine and GBrainConfig.engine unions; decide config key for a storage path (e.g. database_path reuse vs a new key) src/core/types.ts:2343, src/core/config.ts:37, config.ts:738-751 (inference), config.ts:1968-1975 (toEngineConfig)
3 New case in constructEngine (dynamic import of the Rust binding module) src/core/engine-construct.ts:12-23
4 Engine class implementing all of BrainEngine (the interface is roughly 100+ methods; see engine.ts:830-2890). Non-SQL engines implement each method directly; SQL-dialect adapters are optional new src/core/<name>-engine.ts
5 Widen EngineSettings.engine, checkSettings engine guard, and the database shape rules src/core/embedded-engine.ts:14, src/core/engine-open.ts:94-106
6 Widen DegradedEngineOptions.kind if the degraded path should ever cover it; today it is Postgres-only src/core/degraded-engine.ts:17-24, 63
7 Widen EngineStatusReport engine unions; add lock or probe fields for the new engine src/commands/engine-status.ts:108-110 (and the mount?.engine mapping at :268)
8 init flag (e.g. --rust) and isPGLite-style branches; init.ts:56, init.ts:170-181, init.ts:1144 src/commands/init.ts, src/core/cli-flag-registry.generated.ts
9 Migrations: every sqlFor map has only `postgres pglite keys (schema-migrations/types.ts:29); add a third key or require an engine-agnostic sql/handler. Check each of the ~29 strict kind branches in schema-migrations/`
10 Persistence: owner socket, graduation (persistence/engine-graduation.ts, graduation-custody.ts) and pglite-lock-based single-writer logic are PGLite-specific. A third engine needs its own answer to "who is the writer" src/core/persistence/*, src/core/pglite-lock.ts, src/core/persistence/native-lock.ts
11 Doctor checks: commands/doctor/checks/* (22 files under commands/doctor/ branch on kind; 37 check files mention PGLite) src/commands/doctor/checks/
12 Tests: test/e2e/engine-parity.test.ts, test/pglite-engine.test.ts, test/engine-factory.test.ts, test/engine-open*.test.ts; add the third engine to the parity matrix test/
13 Docs: docs/ENGINES.md (sections "Adding a new engine" and the capability matrix), docs/ENGINES.md:1-40 intro docs/ENGINES.md

Unchanged (kind-agnostic): src/cli.ts, src/mcp/server.ts, chunkers, embedding.ts, hybrid search over SearchResult[], and most of src/core/engine-sql/<domain>.ts (per ENGINES.md:1594-1596).

Native module precedent: native/locks/

Contents. native/locks/locks.c is a C Node-API v3 addon (NAPI_MODULE_INIT at line 301, napi_define_properties at line 333) exposing openLock, tryLock, close, plus Windows IPC, plus export publication (export-publication.h). POSIX uses flock; Windows uses LockFileEx.

  • native/locks/README.md: design and CI notes. Says all eight addons are checked in so source installs work with bun install --frozen-lockfile --ignore-scripts.
  • native/locks/manifest.json: version 1, napi 3, zig 0.14.1, input_sha256, and per-artifact sha256 and bytes for eight targets (linux-x64-glibc, linux-arm64-glibc, linux-x64-musl, linux-arm64-musl, darwin-x64, darwin-arm64, win32-x64, win32-arm64).
  • native/locks/package.json: private package boundary, not a package. Its README note says this prevents plugin hosts from treating the whole checkout as one native package (#6026).
  • native/locks/prebuilds/*.node: exactly the eight binaries, checked into git.
  • native/locks/vendor/node-v22.15.0/: node_api.h, node_api_types.h, js_native_api.h, js_native_api_types.h, LICENSE, vendored from Node v22.15.0.
  • native/locks/abi-check.c: compile-time layout checks (_Static_assert for Darwin stat, open flags, lock constants). Compiled only on macOS against the real SDK in native-locks.yml.
  • native/locks/darwin-abi.h, windows-napi.h (resolves used Node-API symbols from the running executable, so it does not load a separate node.exe), windows-ipc.h, export-publication.h.

Build. scripts/native/build.ts compiles with pinned Zig 0.14.1 via zig cc -target <triple> -shared -fPIC -O2 -s -DNAPI_VERSION=3. Targets are glibc 2.17 baseline, musl, x86_64-macos.13.0, and x86_64-windows-gnu. It sets SOURCE_DATE_EPOCH and -Wl,--build-id=none for reproducible bytes (build.ts:39-50). scripts/native/toolchain.json pins archive URLs and SHA-256 (zig for x86_64/aarch64 linux, macOS, Windows). scripts/native/verify.ts recomputes input_sha256 and compares artifact bytes against a rebuild; --write-manifest regenerates the manifest. The README says the rebuild must be byte-identical across CI runners.

Load from Bun. src/core/persistence/native-lock.ts:

  • loadBindingSync() (lines 94-140) picks a literal require('../../../native/locks/prebuilds/<target>.node') by process.platform/process.arch/detect-libc (familySync, GLIBC, MUSL). Literal requires are deliberate: "Bun embeds each .node asset in its compiled executables. Dynamic prebuild discovery does not embed them." (comment at lines 105-107).
  • It validates binding.target === target and that required functions exist, then caches. Errors become NativeLockUnavailableError with code writer_lock_unavailable (lines 27-33).
  • Lazy: loaded only by callers that need a lock. nativeLockCapability() and nativeExportPublisher() expose capability checks.
  • No process.dlopen, no createRequire for .node, no loader module beyond this one file. src/core/cli-socket/peer-cred.ts:20, :38 and src/core/pid1-reaper.ts:91-94 use bun:ffi dlopen against system libs (libSystem.B.dylib, libc), not gbrain-owned addons.

CI.

  • .github/workflows/native-locks.yml (reusable): native job matrix over linux-x64-glibc, linux-arm64-glibc, darwin-arm64, darwin-x64, win32-x64, win32-arm64 on Bun 1.4.4. Each cell runs scripts/native/build.ts --target <t>, verify.ts --rebuilt, the native lock tests, compiled-smoke.ts, and the data-safety suites. A separate musl job (line 336) runs linux-x64-musl and linux-arm64-musl in musl userspace. Scope is narrowed on PRs (scope=smoke|primary|full).
  • .github/workflows/test.yml, heavy-tests.yml, persistence-validation.yml also reference native paths.
  • .github/workflows/release.yml:121-131 builds release binaries for only two targets: bun-darwin-arm64 (macos-26) and bun-linux-x64. Line 170 runs bun build --compile --no-compile-autoload-bunfig ... --target=<t> --outfile bin/<artifact> src/cli.ts src/core/skills/resolver/. Line 175 runs bun scripts/native/verify.ts && bun scripts/native/compiled-smoke.ts --binary bin/<artifact>, which asserts the executable embeds the matching addon.
  • Tests: test/native-lock.test.ts, test/scripts/native-lock-prebuilds.test.ts (tamper and matrix-coverage checks), test/native-locks-package-boundary.test.ts.

Compiled binaries. bun build --compile embeds .node files reached by a literal require. Computed paths are not embedded. The compiled binary does not use the repo's import ... with { type: 'file' } pattern for addons; that pattern is used for PGLite WASM and tree-sitter WASM (src/core/pglite-embedded-assets.ts:1-40, src/core/pglite-embedded-asset-paths.ts). Verified in source only; the packaging assertion is in release.yml:175.

Binding options

Current Bun facts used below (sources at the end):

  • Bun supports Node-API and require() / process.dlopen() of .node files. Compatibility is "best effort" for the exact addon, Bun version, OS and arch. Bun 1.4 aligned N-API status and error behavior with Node 26 and added napi_create_async_work and threadsafe functions with teardown fixes.
  • napi-rs treats Bun as best effort; its Bun test job is non-blocking upstream, so each addon must be tested on Bun before support is claimed.
  • bun:ffi is still experimental; Bun's docs say not to rely on it in production. dlopen, JSCallback are engine-native in current Bun. threadsafe: true callbacks marshal to the JS thread with a per-call cost.
  • node:wasi in Bun is partial: wasiImport and start() work, getImportObject() and sock_accept are missing, and WASI preview1 has no mmap.

Option 1: napi-rs native addon (in-process)

Aspect Assessment
Bun fit Good for the .node precedent already in the repo. Bun 1.4.4 is the repo floor (package.json:261-262, >=1.4.4). Bun 1.4 covers async work and threadsafe functions. napi-rs says Bun is best effort, so the repo must own a Bun test lane.
Async and threads napi_create_async_work and threadsafe functions are supported. Use them for blocking Rust I/O so the JS thread is not blocked. Teardown rules still apply: join native threads before the environment is torn down.
Buffers and embeddings Input Buffer/TypedArray can be read in place with napi_get_buffer_info (already used in export-publication.h:509). Copy-in is safe. Zero-copy out to JS (external ArrayBuffer with a finalizer) is a napi feature whose behavior under Bun must be verified before depending on it; not verified here.
Prebuild matrix Same 8 targets as native/locks. napi-rs cross-build: musl via cargo-zigbuild (adds -crt-static off); Windows MSVC via cargo-xwin (GNU Windows not supported in that mode). The repo's zig-cc pipeline does not build Rust. Use napi build -x or native CI runners per target, and keep the byte-compare or rebuild-verify step if the repo wants the verify.ts guarantee. Windows arm64 and musl are the least-tested on Bun per the napi-rs note.
bun build --compile Works if the .node is reached by a literal require, as native-lock.ts:108-122 does. Bun may extract the addon to a temp file at runtime; TMPDIR must be absolute (Bun issue 45054). Verify with the existing release.yml:175 style assertion.
Crash isolation None. A segfault, abort or uncaught Rust panic kills the gbrain process (CLI, serve, cron). Every exported function needs catch_unwind.
Multi-process Each gbrain process would open the store in-process, so the store needs a single-writer story. The repo already has native/locks flock and the owner-socket routing used for PGLite; both can be reused. Two addons would load in one process (lock addon plus engine addon).
Per-call overhead Lowest of the four: a napi call and marshalling, no syscalls or serialization.
Startup One dlopen per process. No extra process.
Effort Medium: Rust crate, napi bindings for the BrainEngine surface (or a narrow C-ABI-like surface wrapped in TS), prebuild matrix, Bun CI lane, compile smoke.
Risk Medium-high. Crash blast radius; Bun napi gaps for newer APIs; prebuild matrix cost (8 binaries, reproducibility vs cargo output).
Parity tests Run test/e2e/engine-parity.test.ts with the third engine factory, same as PGLite and Postgres; add a native-lock-prebuilds-style test (binary target check, manifest hashes) and a compiled-binary smoke (scripts/native/compiled-smoke.ts pattern).

Option 2: Rust sidecar process (stdio or unix socket, JSON-RPC or binary framing)

Aspect Assessment
Bun fit Pure TS client; no Node-API. Bun child_process / Bun.spawn plus a unix socket. Matches the repo's existing owner-socket and cli-socket design (src/core/cli-socket/server.ts header: "The serve side of the warm CLI socket"; src/core/persistence/local-client.ts).
Crash isolation Full. A Rust crash kills the sidecar, not gbrain; the TS side can restart and reconnect (the repo already has reconnect machinery for Postgres in degraded-engine.ts).
Multi-process Natural single writer: the sidecar owns the store and serializes CLI, MCP and cron requests. This is the main architectural advantage given the repo's current PGLite single-writer lock and owner-socket routing. Requires the sidecar's own lifecycle (start, lock, shutdown, stale socket cleanup, peer-uid checks as in cli-socket/peer-cred.ts).
Per-call overhead Highest. A syscall or socket round trip plus framing per call. JSON is expensive for embeddings (f32 vectors); use length-prefixed binary frames for vectors and bulk page bodies. Not measured.
Startup Per-CLI-invocation cost: spawn the sidecar and open the store, unless the sidecar stays resident (the owner socket pattern). The repo already tunes one-shot reads (engine-connect.ts:34-42), so this matters. Measure before committing.
bun build --compile The sidecar binary must be embedded as a file asset (import ... with { type: 'file' }, the pglite-embedded-assets.ts pattern), extracted or read from /$bunfs, and run with the executable bit preserved. The exec-bit and extraction behavior in compiled binaries is not verified here.
Prebuild matrix Standard Rust cross-compile per target (no Node-API ABI matrix); the output is one executable per platform rather than a .node. Windows and musl still need runners.
Effort Medium-large: protocol design, supervisor, lifecycle, error mapping into opError shapes, reconnect, embed-and-extract for compiled binaries, plus the same CI matrix.
Risk Medium. Latency and marshalling; two processes to keep in sync on version; a second lock domain; sidecar orphaning on crash.
Parity tests Same engine-parity suite via the TS client. Add process-kill and restart tests (the persistence-chaos suites are the model), stale-socket cleanup, and a concurrency test for CLI + MCP + cron against one sidecar.

Option 3: WASM (wasm32-wasi / wasm-bindgen)

Aspect Assessment
Bun fit Mirrors PGLite's model: WASM loaded in-process, assets embedded via with { type: 'file' } (pglite-embedded-assets.ts). No native packaging per platform: one artifact runs everywhere.
Filesystem WASI preview1 preopens work in Bun but are partial and not a security sandbox (Node and Bun docs both say so). No mmap: a storage engine must use linear memory and ordinary file I/O, which constrains page-cache and index design.
Locks WASI has no flock. The single-writer lock in native/locks is a native addon. A WASM engine would need a host import (napi or ffi) for locking or accept single-process use. This is a real gap for the CLI + MCP + cron model.
Threads WASM shared memory and atomics work with workers; WASI itself has no thread creation. Bun supports the shared-memory path, but this is more complex than a native build.
Performance Unknown for this workload; usually slower than native for vector and index work, and no native SIMD guarantees. Not measured.
Crash isolation In-process trap, so a Rust panic is a WASM trap that can still kill the gbrain process unless caught at the boundary.
Effort Large for a storage engine: WASI file semantics, locking, memory limits (wasm32 is 4 GB addressable), and performance work.
Risk High for a storage engine; low risk for pure compute such as a chunker or ranker.
Parity tests Same engine-parity suite; plus WASI file-semantics tests (fsync, rename, lock failure) and a memory-growth test.

Option 4: bun:ffi with a C ABI

Aspect Assessment
Bun fit Bun-only, experimental per Bun's docs. gbrain already uses bun:ffi for small syscalls (src/core/cli-socket/peer-cred.ts:20-40, src/core/pid1-reaper.ts:89-94), so the team has the pattern.
Surface Rust cdylib with a C header (cbindgen) and dlopen(path, { symbols }). Scalar and pointer calls are straightforward; a storage API with owned buffers and handles needs manual free functions.
Callbacks JSCallback is not thread-safe by default; threadsafe: true adds a per-call marshalling cost. Avoid callbacks from Rust threads in the design.
Async No Node-API async-work equivalent; async must be built with Rust threads and polling or a threadsafe callback.
bun build --compile Same embedding question as napi: the .dlopen path must be a file that exists at runtime; extraction needed. Not verified for ffi specifically.
Crash isolation None (same as napi).
Effort Medium for scalars, large for a full BrainEngine.
Risk High: experimental API; Bun may change it; no Node-API stability.
Parity tests Same as napi; add ABI and free-function leak tests.

Comparison summary

napi-rs in-process Sidecar process WASM bun:ffi
Bun maturity Stable N-API, best effort for napi-rs Not Bun-specific (plain IPC) WASI partial in Bun Experimental
Crash isolation None Full None (trap) None
Single writer across CLI+MCP+cron Needs lock addon and owner socket Natural Needs host lock (no flock in WASI) Same as napi
Per-call cost Lowest Highest Medium Low
Startup Lowest Spawn + open per CLI call unless resident Compile WASM per process (cache?) Lowest
Prebuild matrix 8 (or 6+musl) .node per target One executable per target One .wasm 8 shared libs
bun build --compile Literal require embeds; may extract File asset + extract + exec bit with {type:'file'} (PGLite pattern) Like napi
Effort M M-L L M (scalar) to L
Risk Medium-high Medium High for storage High

Recommendation (open question for the repo owner)

Open question: does the first Rust engine need crash isolation and a single writer across CLI, MCP and cron, or is in-process speed the priority?

  • If speed and simplicity matter most: napi-rs in-process. It reuses the proven .node load and compile-embed path in native-lock.ts, has the lowest per-call cost, and fits the Bun-only floor. Cost: a Rust panic or segfault takes the whole gbrain process, so every export must catch panics, and the store needs a single-writer answer (reuse native/locks flock and the owner socket).
  • If crash isolation and one writer matter most: sidecar. It maps onto the existing owner-socket design, but it adds per-call marshalling and startup cost that must be measured against the one-shot read paths in engine-connect.ts.
  • WASM is a poor fit for a storage engine given no mmap and no flock in WASI. bun:ffi is experimental and better kept for small syscalls.

My suggestion, not decided: start with napi-rs in-process, keep the engine surface behind constructEngine, and keep a sidecar as the fallback if crash isolation becomes a requirement. Both need the same engine-parity suite, and both need a Bun CI lane because napi-rs does not block on Bun.

Sources

Caveat: Bun and napi-rs facts above come from web search summaries of those pages, not from fetching and reading each page in full. The Bun ffi and WASI details were partly summarized from secondary result text. Items marked "not verified" were not tested in this repo or on Bun.

Parity risks noted

  • The kind discriminator is a closed union in at least five places (engine.ts:832, types.ts:2343, config.ts:37, embedded-engine.ts:14, degraded-engine.ts:63, plus engine-status.ts:108). Missing one will type-check or silently fall through.
  • About 29 kind branches sit in schema-migrations/, and sqlFor only knows postgres|pglite. A non-SQL engine must not silently get the Postgres or PGLite SQL.
  • migrate.ts:221 treats pglite as the transactional-migration special case; a third engine's transaction semantics must match.
  • PGLite single-writer lock and owner-socket routing are the only multi-process coordination; a third engine needs an equivalent or CLI+MCP+cron can double-write.
  • degraded-engine.ts is Postgres-only; a third engine has no degraded-serve story unless added.
  • engine-open.ts checkSettings refuses unknown engines, and openEngine is the embedding surface; a third engine must be added there or embedders cannot open it.
  • The --pglite init flag and init.ts:170-181 "preserve existing postgres engine" logic need a parallel rule for the new engine, or init can flip an existing brain.
  • docs/ENGINES.md:1566-1596 "Adding a new engine" step 4 implies a free-form engine string that the types reject; the doc is out of date with the code.
  • Compiled binaries: addon and sidecar embedding is verified in source (native-lock.ts:105-107, release.yml:175) but not for a napi-rs addon or a file-asset sidecar on this repo's targets.
  • Release binaries build only linux-x64 and darwin-arm64 (release.yml:121-131), while the lock addons cover 8 targets; a Rust engine would need an explicit target policy.
  • napi-rs Bun support is best effort upstream; without a repo-owned Bun CI lane on each target, a Bun regression can ship unseen.