Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
56.5 KB
Raw Blame

Key files: Core Persistence (continued)

Subsystem index · Previous entries. Read only the entries relevant to your change. Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG. The #6188 fence check and fence repair live in fence repair. Managed connector ingestion, connector item holds and the no-op kernel (checkpoint validation and the fault seams: request indexes and fault seams). The sources refresh writer, the git gap census, declared classes, the git door and ignore rules live in Core Persistence (Git). Also the persistence verb types and errors, database-write and manual links, write attribution and loop-fact retirement; the managed purge of deleted pages is under memory purge.

  • src/core/persistence/{types,params,preconditions,verb-errors}.ts — public mutation receipt states, error reasons and revision/idempotency parameter contracts. Receipt serialization explicitly selects public fields; queued content and execution credentials stay internal. frozenVerbWriteError uses MEMORY_VERBS v1's existing error enum with additive write_request/write_error; page_body_segmented is a write error code, so a verb refused by a segmented target keeps it as write_error and canonical code under the frozen invalid_params (caller class, not retryable) on both the receipt and the pre-admission path (runMemoryWrite). remember reads a segmented entity page up to PAGE_MAX_BYTES back whole (materializedPage) for its inferred-target checks and fence write; over it, a named or inferred entity refuses before admission with segmentedRememberRefusal (fix: remember with infer_entity: false), never a silent NO_ENTITY (test/large-pages/segmented-remember-refusal.test.ts); committedVerbOutcome refuses pending or failed receipts. OperationError and the thin MCP client preserve this metadata, including when a timeout races an already-received receipt. The registry-safe parameter/schema leaf never imports operation values. Pinned by test/persistence-write-contracts.test.ts, test/mcp-client-hardening.test.ts and a real HTTP envelope in test/mcp-client.test.ts.

  • src/core/persistence/database-write.ts — coordinatedDatabaseWrite is the managed path for rows with no canonical file; ontology_propose (one entity page key) uses it. On a managed brain it checks the caller's durable write authority for the authority page (submissionAuthority, then authorizeWrite FOR SHARE inside the transaction), revalidates the source incarnation and commits inside withCoordinatedWrite holding the given page keys. Unmanaged brains keep the engine calls. Tests: test/managed-ontology-propose.test.ts.

  • src/core/persistence/manual-links.ts — prepareManualLinkMutation, the preparer of the add_link / remove_link page mutations a managed brain submits (G5, dispatched from service.ts; on the IPC and semantic re-prepare lists, kept-tier fence edits in trust/page-write.ts). The authored edge is a row of the from page's ## Links fence (links-fence.ts): the preparer upserts or removes the row at the snapshot's revision (a segmented page read back whole through wholeBodySnapshot) and publishes the page with preparePageMutation; the publication's canonical projection and the apply both project the fence (links-fence-projection.ts), so an idempotent add (fence unchanged, a no-op publication) still has its edge. add_link also deletes the database-only twin of the edge it fences (same endpoints, type and provenance, no origin), and on a body with no fence it first renders the kept fence edges into the section it creates, each with the dated statements stored under its own source (keptLinksFenceRows); statements with no source (written before link_transitions had a link_source column) go to a relationship's only row when they fit it, and otherwise the write refuses link_dates_unattributed. remove_link deletes the matching database-only edges (origin_field not links_fence, a provenance outside MANAGED_LINK_SOURCES) with their database-only manual statements and, when the body has no fence, the orphaned fence edges with the statements of their own source (it refuses link_dates_unattributed when sourceless statements would have to be split between a removed row and a kept one), and reports removed as the edge count it took away; edges a reconciliation producer derives from the text stay, and the result names them (derived_from_text: [{ link_type, link_source }] plus a message to edit the page text instead). The op handlers keep their validation, dry_run and result shape ({ status }, { status, removed }); unmanaged brains keep the engine calls. Tests: test/managed-links-fence.test.ts (+ Postgres arm), test/managed-writers-w3.test.ts.

  • src/core/persistence/{attribution,attribution-schema,context}.ts — write attribution. withCoordinatedWrite(engine, sourceIds, fn, attribution) sets gbrain.write_sources and the transaction-local actor (gbrain.write_request, gbrain.write_principal_kind, gbrain.write_principal_id) in one round trip and restores them on exit; a nested scope keeps the outer actor. The argument is required: requestAttribution(row) for journaled publication (coordinator, forget), maintenanceAttribution(engine) (verified local writer, else the CLI registration, else application/host:<id>; no request) for derived facts, takes/timeline/link extraction, materialization, concept provenance, connector freshness and verify cleanup, principalAttribution for manual links (the caller) and source lifecycle (the topology principal). withWriteAttribution sets only the actor, for unmanaged legacy transactions; maintenanceTransaction(engine, fn) wraps it in one transaction, and every unmanaged direct writer runs its write (or one bounded batch) in it (the system-of-record.md "unattributed" list is empty; derivers pass their taint as maintenanceTransaction(engine, fn, trust), see trust/taint.ts). attribution-schema.ts (migration f1_write_attribution; migration-only like the facts DDL, so columns land last on fresh and upgraded brains) adds nullable columns and BEFORE ROW triggers: facts/takes/timeline_entries carry write_* (created_by, immutable once set) and last_write_* + last_written_at (moved only when a classified content column changes; a NULL actor records an unattributed writer; under gbrain.fact_withdrawal_stamp, which only the facts insert guard sets, a facts update of expired_at/valid_until alone keeps them, migration v264); pages.revision_* is stamped when knowledge_revision changes; page_versions.write_* copies the snapshotted revision's actor and archived_* names the archiving write. write_request_id is persistence_requests.id with no foreign key; all-NULL reads as unrecorded. Every column of the three tables is classified content, projection (embeddings, takes.updated_at) or attribution, and a new column must be classified and the function re-created; the writer guard treats attribution-only facts/takes updates as projections. Tests: test/write-attribution.test.ts, test/write-attribution-legacy.test.ts (writer inventory) and test/write-attribution-<family>.test.ts (each routed unmanaged writer path; write-attribution-timeline-10k.slow.test.ts checks bounded batches at 10k pages) on PGLite; Postgres direct and PgBouncer through test/e2e/write-attribution-postgres.test.ts.

  • src/core/persistence/loop-fact-retirement.ts — #5869 commitment-fact retirement for closed loops, one coordinator mutation on every brain (operation loops_close, intent retire_loop_fact, dispatched from service.ts): expires the fact and strikes its ## Facts fence row (context loop closed: <status>) through the entity page publication in one transaction; no withdrawal tombstone. The shared rule (also loopFactDrift, used by doctor loop_facts_drift and gbrain repair loop-facts): only a fact in the loop's source that no other open loop references, re-checked under row locks at publication (shared_with_open_loop, source_mismatch otherwise). retireLoopFact(ctx, loopId) submits with the caller's grant and maps a refusal or pending write to fact_expired: false, retryable: true, reason. Tests: test/loops-close-fact.serial.test.ts (+ Postgres twin).

  • src/core/persistence/connector-authority.ts -- classifies each Google/GitHub source as connector_database (managed, unbound), bound or unmanaged from persistence_brain.enabled and the source binding row. File-lane audits (doctor undeclared_db_only_pages, backup coverage) read it so a leftover cache directory at local_path is never taken for canonical files. Tests: test/doctor-silent-death-checks.test.ts, test/backup-coverage.serial.test.ts.

  • src/core/persistence/connector-sync.ts — managed Google/GitHub API ingestion through source-scoped journaled connector_v2_import/_delete/_checkpoint/_google_receipts intents and managed-connector checkpoint CAS, not Git cursors. The checkpoint key, admission configHash, mid-sweep change check and publication source_changed check all use the parsed-config identity from connector-identity.ts, so cycle stamps in sources.config never rotate them. Unbound connectors use reviewed connector_database SQL-only authority; bound connectors retain canonical file publication with guarded root/hash/revision/epoch. Every managed run checks the credential's resolved account against the pin in the connector state row before reading its checkpoint (assertAccount), then applies a requested --reset-checkpoint (pending writes resolved first, pin kept). Each import or delete is first screened by running prepareConnectorMutation on an unadmitted request through the no-op kernel; unchanged items take no admission. An unchanged checkpoint save is skipped and, when it would have stamped freshness, last_sync_at is stamped directly inside a lease-asserted coordinated write. Accepted-pending writes are progress: a 30-second total wait budget (writes committing within one second are free), at most one in-flight write per page, outstanding writes kept under the principal's outstanding and intent-byte limits, and ConnectorWaitBudgetStop (a write_pending error) when the budget is spent. The checkpoint never advances past a pending page; the run records its pending set in the state row, and the next run resolves it first (committed dropped, failed retried automatically through the retry-pointer path without --retry-failed, unreached failures after a complete sweep dropped and counted). Retired managed_connector_* intents route to prepareOutdatedConnectorMutation, which fails them connector_intent_outdated (detail pre_upgrade before the v176 cutoff). Standalone entry, page reads and submissions authenticate the current source, local grant and canonical owner and settle retained worktree publications before deriving request identity. Explicit --retry-failed atomically admits a linked attempt and stores a per-base-request managed-connector-retry pointer. Durable connector cursors, retry pointers and state rows survive generic seven-day checkpoint GC. Tests: test/persistence-connectors.test.ts, test/persistence-connector-retry.test.ts, test/connector-checkpoint-identity.test.ts, test/connector-wave3.test.ts and test/helpers/connector-restart.ts on PGLite and disposable Postgres. #6831: a page import whose content matches a source page purge tombstone is skipped per item instead of ending the run: screened before admission (carriesPurgedPage, the guard's content hash, read only when the source has a tombstone), and a failed/purged_content receipt the guard still produces (a purge that landed mid-run) is skipped wherever it surfaces (the item's wait, refreshPending, resolvePendingSet): it is dropped from receipts, never joins the automatic retry set, has its optimistic pending count taken back, and is counted in last_run.purged_skipped and purged_items. A replayed purged receipt is re-checked against the live tombstones (purgedNow) before mayRetry: still purged, it is skipped even under --retry-failed; cleared, it gets one automatic retry. A run that ends on any error except LockStolenError or an aborted caller signal records last_failure (code, else connector_error; message, the item and request when the run ended on that item's receipt, consecutive); only a complete finish() (cursor saved, nothing pending or failed) clears it. Tests: test/connector-purged-item-6831.test.ts (+ test/e2e/ twin).

  • src/core/persistence/connector-identity.ts — connectorIdentity(kind, rawConfig, localPath) returns the parsed connector config and a digest over every parsed leaf except credential-delivery fields (Google access/tokenCommand/tokenEnv; GitHub tokenEnv and app under scope: repos, app.pemPath always; under scope: auto the token env without an App and the App ids are identity). services/repos are sorted and de-duplicated; a parser leaf missing from CONNECTOR_LEAF_RULES joins the identity. #6561: a leaf a parser starts emitting with a behavior-preserving default declares it as elideWhen (Google contactsDir: 'people') and is left out of the digest while it holds it, so unchanged sources keep the pre-field key; the non-elided digest is kept as aliasDigests (connectorCheckpointKeys returns the stable key then its alias keys; connectorIdentityMatches accepts an admitted write's frozen configHash under either). Also connectorCheckpointKey and isConnectorIntentKind/connectorIntentSql, the one predicate for both intent namespaces. Test: test/connector-identity.test.ts enumerates every parser leaf.

  • src/core/persistence/connector-state.ts — (fix wave 4: also first_attempt_at, the autopilot connector dispatch gate, recorded before the first provider call of a validated, non-preview sync and seeded by migration v181) the per-source-incarnation op='managed-connector-state' row (fingerprint digest({sourceId, incarnation}), independent of the identity digest): account pin, pending set {itemRef, requestId, baseRequestId, admittedAt}, upgrade_recovery (resumed/rewalking_once/none) with resumed_from, continuity_unverified and last-run counts. writeManagedConnectorState checks the connector sync lease in the same statement. connector-status.ts renders it for gbrain sources status without the account. #6831: purged_items (newest CONNECTOR_PURGED_LIMIT purged skips) and last_failure (ConnectorRunFailure); CONNECTOR_STOPPED_RUNS (3) consecutive failed runs read as stopped in connector-status.ts (sources status lines, readConnectorLastError for sync status's additive connector_last_error; last_error, state and next are untouched) and doctor sync_freshness (stoppedConnectorIssues in doctor/sync-outcome.ts).

  • src/core/persistence/connector-status.ts — sources status views of connector state and Git holds. gitHoldStatusJson (#7326) bounds git_holds in --json: summary by default (count, by_code counting every hold of the source, with an example only for a code the listed items include, and by_code_complete: false when the count fell back to the listed files, the first HELD_STATUS_LIMIT items, items_complete/items_command), every item read under --holds, or a pageSortedRows keyset page by path under --limit/--cursor (test/sources-status-holds-bounded-7326.test.ts, test/sync-hold-surfaces.test.ts).

  • src/core/connectors/item-holds.ts — fix wave 4 connector item holds, shared by Gmail and GitHub on managed and unmanaged brains: classifyConnectorError (the one table: source-scoped, rate-limit and item-scoped codes, content vs transient class) and ItemHoldsRun (hold after 3 consecutive attempted runs at one upstream version; circuit breaker at 5 attempted items with half transient or 5 and half one code; backoff 1 h/6 h/24 h/daily for 7 days; cap 100, connector_holds_exhausted). A run's changes merge only in finish(), so mid-run checkpoints carry the holds the run began with and a cursor never passes an item that is not already held. carryLegacyFailCounts reads the retired Gmail ledger once.

  • src/core/connectors/item-holds-store.ts + connector-hold-session.ts — reads holds from the cursor state (managed checkpoint row or the unmanaged state file) for sources status, doctor connector_held_items and open_loops completeness; the connector-hold-retry request rows and the pending managed-connector-retry pointers sources retry-held writes; the per-run session that skips held items, publishes holds on a managed abort (publishHolds), releases held items' terminal receipts from the automatic retry set and consumes retry-held keys. Git-source holds are not read here: only sources status (persistence/connector-status.ts readGitHoldStatuses, sources status <id> filters to one source) and sources retry-held (a Git source records a sync-hold-retry re-screen request for its held paths; gbrain sync --retry-held refuses with that pointer) show them beside connector holds. Writer deactivation and activation never read or block on them.

  • src/core/connectors/connector-text.ts — #5752 render boundary: prose sanitized with sanitizeForJsonb before the no-op screen; identity fields refuse with invalid_connector_text.

  • src/core/persistence/connector-account.ts — resolves the credential's account before any service runs: Google through probeGoogleAccount in google/api-access.ts (Gmail profile, Calendar primary calendar or People me, first enabled service that answers, else OAuth userinfo, #7073), returning the probe with every per-service error; when nothing names the account it rethrows api_not_enabled first, then another credential error, then the first other failure, so connector_account_changed means a resolved account that differs from the pin (test/google-api-access-managed-7073.test.ts), GitHub App installation id from the minted token, or the /user login for token auth under scope: auto.

  • src/core/persistence/connector-errors.ts — connector_account_changed with its two-exit hint, the retired-intent messages, and receiptDeliveredHint, which rebuilds suggestion/detail/docs for codes delivered through a stored receipt (the coordinator persists only code and message); writeResponse applies it.

  • src/core/persistence/noop-kernel.ts — the #5470 no-op kernel. inspectUnchanged takes a preparer's PreparedMutation for an unadmitted screeningRequest and returns contentUnchanged, projectionWorkRequired (safe-chunk fence, text projection behind the knowledge revision, embedded page without a contextual mode), metadataWorkRequired (source path) and observedRevision; bound pages also need local file bytes equal to the serialized page. The page's current knowledge_revision, read with the projection flags, must still equal the snapshot's revision (a write after the screen's snapshot admits as revision_moved). Callers run the preparer's own validate before counting a skip. An optional waive list (#5751) lets a caller skip two admit reasons its no-op publication can never resolve, a pending contextual-mode stamp and file bytes that differ from the prepared content (never a missing file); the result lists each waiver applied in waived; only managed working-tree sync entries and put-page-screen.ts pass it. Used by connector sync, import-mutations.ts, sync-run.ts (working-tree and company-profile entries; renames and overlays always admit) and put-page-screen.ts (the managed transcript import's put_page, waiving canonical_file_differs for a file the preparer accepted). chunksEmbeddedAsIs (#7866): a no-op whose chunks hold current vectors queues no embedding. Tests: test/noop-kernel-paths.test.ts, test/connector-wave3.test.ts.

  • src/core/persistence/put-page-screen.ts — #6741 B: unchangedPutPage screens a trusted local put_page (the transcript import) through preparePageAdmission (unadmitted), the page publication preparer and the no-op kernel; skips admission only on a genuine no-op with no pending request, no segmented body and no projection work. Tests: test/transcripts-managed-noop.test.ts.

  • src/core/persistence/import-reads.ts — GBRA-75 wave 10: readImportSnapshots reads a run of pages as readPageSnapshot(slug, { sourceId, includeDeleted: true }) returns each (soft-deleted rows, withdrawals and purge markers included) through readPageSnapshotsBatch({ includeDeleted, absentIsNull }), looping over its byte-budget prefixes; the managed import screen uses it for two or more new members. importGroupReads serves an import group's preparation (group-publish.ts preparationReads): each member's first own snapshot read comes from one batch, and findDuplicatePage calls that arrive together (all members, or a 2 ms window) are answered by one findDuplicatePages (engine-sql/pages.ts, same predicate and ranking per input); a failed batch falls back to each member's single read; with reread (the read-only screen) a member's later reads of its page come from the batch too. GBRA-108: sliceReads answers a screened slice's per-page reads (the origin claimants of sync-origin.ts, the stored timeline and take collisions of canonical-projections.ts, the safe-chunk fence of page-state/projections.ts, matched by their exported statements) from one statement per kind over the slice, row for row; a page with stored takes, a page or path outside the slice, a signalled call or a failed slice read runs its own statement (test test/sync-legacy-sweep-gbra108.test.ts). Tests: test/import-reads.test.ts, Postgres arm test/e2e/import-reads.test.ts.

  • src/core/persistence/screening-paths.ts — GBRA-75 wave 7: withScreeningPaths(run) opens one screening batch's memo of root-level filesystem facts and screeningPath(key, compute) resolves each once inside it: other sources' root realpaths (assertImportPaths), shared skillpack root canonical paths and the source root's Git scope (knowledge-guard.ts). Per-file paths are never memoized; a nested batch joins the open one; the memo closes with run, so work that outlives it computes fresh; a throwing compute is never memoized. Opened by the managed import plan and screen loops (import-mutations.ts), each sync waiver screen (screenWaiver) and sync-run.ts waiveRun's screens. Tests: test/persistence-skip-cost.test.ts.

  • src/core/persistence/sync-waivers.ts — managed-sync no-op waivers (#5470, #5984). waiveNoopEntry screens a frozen, unadmitted entry: an unchanged import (unchangedSyncImport, moved here from sync-run.ts, with the #5751 working-tree kernel waivers) or a delete whose page is already soft-deleted at the entry's frozen expected_revision with its file absent (never an unowned deletion or a rename). A candidate runs validateSyncAuthority first (a revoked grant throws before any waiver), then commits in one transaction with a fixed lock order: page guard (lockPageKeys), then the managed-sync cursor row FOR UPDATE; it re-reads the snapshot (and, for deletes, the origin and file absence) and refuses while UNFINISHED_PAGE_REQUEST_SQL finds an unfinished request for the page by slug, page id or a renameFrom naming it. That lookup is three LIMIT 1 branches served by persistence_requests_pending, persistence_requests_recovery and the database-only pending index (EXPLAIN pinned in test/persistence-managed-sync.test.ts). A cursor that moved meanwhile is adopted instead of waived again, so waived counts only increment in the winning transaction. GBRAIN_SYNC_WAIVE_NOOP=0 admits every entry. #6278: the lookup also matches the entry's file path, origin and a rename from that origin (unfinishedPageRequestParams), so the hold gate in sync-run.ts shares it. unchangedSyncImport (and so sameContentAtOrigin's preparation) runs the sync preparer under raceSyncBudget against persistence.sync_preparation_ms (syncPreparationBudgetMs, read at most every 5 s, default 120 s) and the run's signal: past the budget nothing is waived and the origin refusal stands (fail open, nothing counted or held, SyncPreparationBudgetExceeded); a cancelled run propagates. GBRA-75 wave 8: unchangedSyncImport prepares with { unsaved: true }, so its validation also accepts this run's cursor with nothing pending. waiveNoopRun (up to 64 entries, one cursor write) re-reads the owner epoch under its lock (a change passes nothing; the per-entry path then refuses it) and each import's file bytes (an edit ends the prefix). Both waiver transactions read a waived import's file (a raw hash that no longer matches admits the entry) and write its sync-import-provenance record from the preparation's importProvenance (waivedProvenance: disk hash and blob id of those bytes when they hold the imported ones, heldDiskBytes, #8089; the page's current revision), so a waived page carries the same record a publication writes. Tests: test/sync-waiver-budget.test.ts, test/sync-waiver-run-imports.test.ts, test/persistence-managed-sync.test.ts (#5984 waiver cases), test/persistence-sync-legacy-files-5751.test.ts.

  • src/core/persistence/atom-maintenance.ts (write failures) — #7514: a batch whose write ended without committing (conflict, failed, cancelled) is a write failure, atomWriteFailure: its record carries the write's request id and refusal code (the code survives receipt compaction, which erases the message to "The write did not commit."), and on the drain each time the batch is met counts one strike in an op_checkpoints row (managed-atoms-uncommitted, keyed by the failed run key) with no model call, since a failed batch is never re-extracted on its own. At MAX_DETERMINISTIC_FAILURES strikes a journaled failure completion (intent attempts) advances the drain-key checkpoint to held, so the origin leaves the backlog and doctor lists it with that completion's receipt and the reviewed retry; an explicit retry records no strike. Test: test/extract-atoms-commit-failure-hold.test.ts.

  • src/core/persistence/prepared-maintenance.ts (physical root) — #7760: maintenancePreflight re-stamps a device-only physical-root change under a no-wait acquireWorktree, as ordinary writes do (#5604); any other identity change still refuses, and a busy lock keeps the physical_root_device_changed refusal. Test: test/persistence-physical-root-device.test.ts.

  • src/core/persistence/accepted-pending.ts — acceptedPendingReceipt(error): the shared test for an accepted, not yet committed write, used by managed atom publication, extract_atoms, managed import and the connector budget. publicationHold(error) returns pending for such a receipt or contention for an OperationError storage_error with detail database_contention and no receipt (admission rolled back, nothing recorded), else null — never from a message, a terminal receipt, a contention error carrying a receipt or a permission refusal; publishOrHold(fn) runs one publication and returns null on commit, the hold when there is one, and rethrows anything else. Used by the patterns, drift and synthesize dream phases.

  • src/core/persistence/connector-checkpoint-migration.ts — migration 176 and orphan cleanup. The newest committed checkpoint receipt per source incarnation (found by slug, so a compacted one is still seen and re-walks) is copied to the stable key with INSERT … ON CONFLICT DO NOTHING; state rows are seeded; the cutoff is stored in op_checkpoints (op=managed-connector-migration, fingerprint v2-cutoff, excluded from the checkpoint purge). orphanConnectorCheckpoints keeps rows touched after the start, rows any registered source (archived included) can load under its stable or alias key (loadableConnectorKeys, #6561), and rows referenced by a non-terminal request or a pending set; timestamps bind as text. gbrain repair connector-checkpoints (src/core/repair/connector-checkpoints.ts, whose preview counts alias-kept rows as alias_protected) and the connector_checkpoints doctor check reuse it with a 7-day floor. No upgrade step rewrites alias rows (#6561): connector-sync.ts resumeCheckpoint resumes from an alias row written after the stable one when the stable key has none or the alias holds the newer completed Gmail backfill, and the run's first save moves that cursor to the stable key; every checkpoint and page write compares against and writes only the stable row. connector-reset.ts guards gbrain sync --reset-checkpoint to connector sources.

  • src/core/persistence/graduation-target.ts — engine graduation target side. resolveTargetRoutes reads the URL from --url or the --url-env variable and resolves the DDL route once (explicit GBRAIN_DIRECT_DATABASE_URL, else the Supabase direct derivation, else the main URL); connectTargetEngines hands that route to the connection manager as an explicit override, so no session-pooler fallback is chosen for the run. targetIdentity hashes host, port, database and user, never the password; redactTargetUrl is the only display form. probeTarget runs read-only probes in BEGIN READ ONLY sessions on both routes (server version, vector with halfvec, CREATE privilege, session_replication_role permission inside a savepoint, table ownership, seed-only emptiness, existing embedding columns, the role's superuser and BYPASSRLS standing and the auto-RLS event trigger and its function owner); targetProbeBlockers (non-superuser targets get the exact SQL a DBA runs for vector, BYPASSRLS and the event trigger) and assertTargetReachable turn a probe into plan blockers and typed refusals. crossCheckRoutes proves both routes reach one database with a transaction-scoped advisory lock keyed by a fresh nonce, read back from pg_locks through the other route. Tests: test/graduation-target.test.ts.

  • src/core/persistence/graduation-drain.ts — engine graduation drain. graduationBlockers lists what keeps a brain from copying, each with its structured action: requests this host owns (drained; a live recovery blocked on unexpected_file_bytes is not drainable: needsUser, argv gbrain repair stuck-recovery --source <id> --request-id <request>), requests and host bindings owned by another host, recovering topology changes and effects (a withdrawal-mirror effect blocked on unexpected_file_bytes names the same stuck-recovery preview, needsUser), the writer admin lock, and OAuth clients bound to a missing source (an FK only Postgres enforces; never cleared silently, since unbinding widens access). drainForGraduation refuses up front on anything the drain cannot clear (any blocker that is not a request, or one that needsUser), then publishes this host's queued, running and recovering requests with a requestsOnly PersistenceConsumer bounded by the timeout and always runs its stop barrier before returning; the graduation_drain_timeout refusal is built by graduation-errors.ts. freezeSource stops the resident consumer and sets the PGLite connection to read-only transactions; withSourceWritable runs one read-write transaction for custody writes. Tests: test/graduation-drain.test.ts.

  • src/core/persistence/graduation-copy.ts — engine graduation copier (in process). copyTable reads source rows through the digest's own plan (digestPlan with inventory transforms, rowFilter, ::text values, COLLATE "C" keyset batches) and writes them in one target transaction per table that sets gbrain.graduation_run, bypasses user triggers (session_replication_role = replica, or DISABLE/ENABLE TRIGGER by name in the same transaction, never the gbrain_graduation_fence triggers), deletes the target's rows (seed rows included) and inserts byte-sized batches as one $1::text::jsonb array cast back per column type. The per-column contract (columnContract) refuses missing, retyped, target-only or differently generated columns outside the entry's columnAllowlist; generated columns are never inserted. Self-FKs take a second update pass under DISABLE TRIGGER. copySequences applies each source sequence's exact position, raised to the target column maximum. deferIndexes/buildDeferredIndexes drop and rebuild HNSW (through buildDeferredAnnIndexes) and GIN indexes, keeping the pending list in the target-owned graduation.deferred_indexes config row. Tests: test/graduation-copy.test.ts, test/e2e/graduation-copy.test.ts.

  • src/core/persistence/root-metadata.ts — engine-free ownership-marker names. isPhysicalRootMetadata matches .gbrain-owner.json, .gbrain-owner-<sha256>.json (reservation) and .gbrain-owner.json.<uuid>.tmp (physical-root-record.ts re-exports it and PHYSICAL_ROOT_MARKER). OWNERSHIP_MARKER_GLOBS (#5186) are those names in gitignore dialect: physical-root-record.ts excludeOwnershipMarkers appends them to <gitdir>/info/exclude (git rev-parse --git-path, idempotent, best effort) after writePhysicalRootStamp / adoptTransferredRootStamp, for the stamped directory, the root and the reservation's parent, and again when assertPhysicalRoot verifies a root inside a checkout whose exclude lacks them (a repo-less brain that later ran git init, #8713). PUSH_DENY_GLOBS denies them with .gbrain-bootstrap.lock/**; the bootstrap .gitignore template has them; doctor tracked_ownership_marker warns when a source checkout's index has one. withoutPhysicalRootMetadata drops git status --porcelain entries made only of such metadata (doctor bootstrap_push_health, stop-hook dirty probes): a stamp-only tree is clean, a real change beside the stamp, a rename across it and a quoted path still count. Test: test/persistence-root-metadata.test.ts.

  • src/core/persistence/automatic-sync-policy.ts — automaticSyncPull(engine, source): whether an automatic sync (autopilot freshness sync job, per-source fanout cycle) may request a Git pull. It needs a parsed remote_url, exactly one boolean persistence_brain state that is off, and no worktree binding; a managed brain or claimed source syncs local HEAD with pull: false. Missing, duplicated or non-boolean persistence metadata throws storage_error (why, read-only gbrain sources writer status --probe --json fix) instead of granting a pull, so the caller queues nothing. Clone, immutable and federation flags grant nothing. Pinned by test/automatic-sync-pull-policy.test.ts.

  • src/core/persistence/native-lock.ts + native/locks/ — lazy portable Node-API v3 kernel locks (flock / LockFileEx) with opaque handles, retained stable lock files, cancellable asynchronous waiting and fail-closed capability errors. Eight vendored prebuilds support script-disabled source installs; pinned Zig recipes and source/artifact hashes live in scripts/native/. Actual competing-process, crash-handoff and compiled-executable tests run across eight platform/architecture targets on Bun 1.4.4. The compiled smoke waits for child exit before cleanup; scripts/native/compiled-smoke-cleanup.ts bounds retries to transient Windows EBUSY errors, with persistent and permission failures pinned by test/scripts/native-compiled-cleanup.test.ts. Windows-only windows-ipc.h supplies opaque Global mutex claims (same-thread recursion refused, abandoned owner handoff) and exact-handle stale AF_UNIX deletion after a verified reparse tag under a retained binding claim. CNG hashing and native casing make pipe identities independent of homes and logon sessions. Build and ownership contract: native/locks/README.md. native/locks/package.json (private, no entry points, outside buildInputs and prebuilds/) is a package boundary so plugin hosts that namespace native addons by the nearest package.json do not treat the whole checkout as one package (#6026; guard test/native-locks-package-boundary.test.ts). #6305: a failure to create or open the lock file is a NativeLockUnavailableError with stage: 'open' and osError (node:fs code, or the addon's (OS error N) as its errno name); isLockOpenFailure matches only that typed shape, never message text. publishMutation (coordinator.ts) and the Postgres single-write group path (group-publish.ts publishGroup, falling through to publishMutation) release such a write as queued / writer_lock_unavailable instead of the terminal owner-exception path; the consumer holds that root for INSPECT_OWNER_RETRY_MS (30 s, health.ts) so it doesn't re-prepare every poll; ownerExceptionFailure reports osError as the owner-only errno; a Git effect counts it as a dependency wait and never parks on it. An EISDIR or EACCES on the brain path itself stays terminal (persistence-lock-open-failure.test.ts, both engines).

  • src/core/persistence/switches.ts + preadmit-cache.ts + config-snapshot.ts — the single-write fast path for foreground page writes on Postgres. switches.ts reads every write-path kill switch (persistence.single_write_group / GBRAIN_SINGLE_WRITE_GROUP, persistence.preadmit_cache / GBRAIN_PREADMIT_CACHE, sync.waive_batch / GBRAIN_SYNC_WAIVE_BATCH, sync.foreground_priority / GBRAIN_SYNC_FOREGROUND_PRIORITY, and persistence.preparation_deadlines / GBRAIN_PREPARATION_DEADLINES (#6278: off restores the old preparation predicate, see persistence-stalls.md); on by default, 0/false off, env over config) in one SELECT key,value FROM config WHERE key = ANY(...) per engine, kept 5 s (the consumer tick reads it under its phase deadline), together with the four preparation budget keys (readWriteSwitchSnapshot returns { switches, preparation }; readPreparationPolicy is the budgets alone); read-through views register their engine (registerEngineView) to share it. With single_write_group on: the consumer publishes an eligible single page write (put_page, capture, edit/delete/restore/revert without an internal intent.kind or group key) as a group of one through publishGroup (publishSingleWrite; its intent carries no group marker, its effects queue without deferBatchReconcile, and a group that does not commit falls back to publishMutation, reclaiming a claim its file restoration released); PGLite and refused shapes keep publishMutation. The waiter's own wake (wake(true)) claims without the periodic scans (recovery records still block the claim through claimableWriteSql); preparation reads the config table once (config-snapshot.ts) and sends its independent reads together; the coordinated (coordinated: true) put_page apply reuses the publisher's locked read as the import base, pipelines its follow-up statements and reads the page once after its last write for the read-back check, seal, core notice and receipt. Publication, the own admission and the recovery transactions run on the consumer's reserved lane (onLane: the idle-probe connection, taken while two other pool connections stay free and kept across writes; never lent while an idle probe runs on it, and probes skip it while lent), so their prepared statements stay warm; the waiter is told of the commit before the recovery record is cleared. preadmit-cache.ts (with preadmit_cache on, Postgres) answers the brain id, source row, worktree binding and local-writer reads of submitPageMutation from a per-engine cache (30 s, keyed by brain id; absences never cached); admission rechecks each under lock and the brain id against its protocol statement, and a refusal the cache could explain (source_changed, permission_denied, a changed brain) drops the cache and runs the whole preparation once more uncached. Pinned by test/e2e/single-write-fast-path-postgres.test.ts (switches, cache rechecks, replay, restoration after a failure past the file rename, the recovery-blocked direct claim) and test/single-write-group-parity.test.ts + test/e2e/single-write-group-parity-postgres.test.ts (row-for-row parity with the switches off); measured by scripts/bench/foreground-put-page.ts.

  • Foreground priority (#5984 Phase 4.5, foreground_priority switch, journal.ts): claimableWriteSql lets a non-sync request pass earlier queued sync rows of its root that have not started and do not name its page (slug, page id or intent.renameFrom.slug), claims it first, and holds a passed sync row while that request runs; claimNextLaneHead starts no lane head while such a request is unfinished, except one head per foreground commit since the run's last head claim (seen_sequence). A write queued behind an earlier queued write that waits on a sync row naming its page holds back nothing, since it cannot go first either (pinned by test/claim-fifo-foreground-waiting.test.ts). Both claims find that commit through FOREGROUND_WAITING_ON's probe, served by persistence_requests_foreground_committed (#7649, a partial index on committed non-sync rows by worktree and sequence, see request indexes), so a consumer's first claims after a restart, when the bound is 0, stay index probes on a large request table. A consumer whose claimed write passed sync rows (passed_sync) claims plain FIFO on that root after it commits (owesSyncTurn), so a stream of writes cannot starve the drain. The sync side's pauses (no admit-ahead for 60 s after a write, no group while one is queued, the 25-page credit wait) apply only with the switch off. Receipt health judges a claimed write only against earlier started writes. Pinned by test/managed-sync-foreground-priority.test.ts (two processes) and test/persistence-diagnostics.test.ts.

  • src/core/postgres-engine/reserved-transactions.ts with isPersistenceDeclaration (src/core/persistence/protocol.ts) — BEGIN pipelining. isPersistenceDeclaration (GBRA-156) accepts exactly the protocol.ts declarations (transaction-local set_config calls and the brain row FOR SHARE). Admission, the FIFO claim, the effect claims and source guards, the receipt true-up, the single publication (coordinator.ts) and the consumer's lent lane transactions (consumer.ts onLane, through reservedTransactions) pass it as TransactionOptions.pipelineFirst, so behind a transaction-mode pooler the declaration goes out with BEGIN (one round trip less per transaction) while every later statement still waits for BEGIN's answer; if BEGIN fails, only the declaration has run, in autocommit, where it changes nothing. A reserved transaction gates every statement its body can reach, savepoint and nested-transaction bodies and file() included. A sync lane group's publication passes none: its begun mark must follow BEGIN's answer (sync-lanes.ts awaitLaneBegin). Tests: test/persistence-pipeline-first.test.ts (the predicate and the reviewed call sites), test/e2e/pgbouncer-transaction-pipeline-6930.test.ts.

  • src/core/links-fence.ts + src/core/links-fence-projection.ts — the ## Links fence (G5): the markdown form of authored (manual) edges, <!--- gbrain:links:begin --> / end around | to | type | source | valid_from | valid_until | context |, one row per (to, type, source), source defaulting to manual; a reconciliation-managed provenance (MANAGED_LINK_SOURCES, exported here and re-exported by ops/links.ts), a bad date or a duplicate key is a skipped row with a warning. parseLinksFence (state absent / parsed / malformed), the deterministic renderLinksFence (rows sorted by key), upsertLinksFenceRow (omitted context and dates keep the stored ones; identical row = unchanged body), removeLinksFenceRows (leaves an empty fence), replaceOrInsertLinksFence (a ## Links section at the end of compiled_truth, above an unclosed code block) and maskLinksFence, which extractBodyLinks applies so the fence cells are never also extracted as markdown, wikilink or bare-slug edges. A rewrite of a fence that parses with warnings throws LinksFenceRewriteRefusal. Cells use escapeFenceCell plus the same one-level marker encoding for gbrain:links:*. projectLinksFence(tx, page, rows) reconciles only what the page's fence owns: edges with origin_page_id = the page and origin_field links_fence (deleted, context-updated, inserted by key) and manual transitions with origin_page_id = the page, compared per row by (to, type, link_source, kind, date) and written with the row's source in link_source, then refreshes the touched relationships; a row whose target is not a live page in the source yields no edge. linksFenceProjectionRows returns null for an absent or malformed fence, so such a body changes no edge (an absent fence retires nothing, as for facts; unlike facts no held generation is recorded, so after a hand deletion of the whole section the database keeps edges git no longer has until remove_link or the G3 indexer), and drops rows typed with a link type the active pack does not declare. Callers: prepareCanonicalProjections (every projected page publication, quarantined and segmented-unread bodies excepted), prepareAutomaticLinks (put_page auto-links and the managed links maintenance, so extract --stale restores a lost fence edge) and persistence/manual-links.ts. Database-only manual edges (origin_field NULL) are never touched by any of them. Pinned by test/links-fence.test.ts and test/managed-links-fence.test.ts.

  • src/core/persistence/source-lifecycle.ts retire batches with topology-transaction.ts (#7702/#8905) — every topologyTransaction runs under statement_timeout=5s and lock_timeout=1s and maps SQLSTATE 57014/55P03 to write_pending, so a managed remove or purge first deletes the source's pages in batches of SOURCE_RETIRE_BATCH_PAGES (500; about 110 ms per batch with chunks and links on Postgres) under the native locks withTopologyLocks holds, each batch its own topologyTransaction. A batch re-checks the final transaction's preconditions (admin state, lockTopologyRows, the incarnation read before the locks, archived or EXPIRED_ARCHIVE_SQL due, no prior receipt other than this request's own admission, no oauth_clients reference, assertTopologyRequestsSettled from topology-locks.ts, lockTopologyPrincipal, and on the first batch admitTopologyRetire from topology-receipts.ts, which checks the receipt quotas at the largest receipt and records the admission) and, when one would refuse or replay, stops without deleting so the final transaction reports it, or throws that refusal before deleting; a batch that fails write_pending is retried at half the size (floor 1), so heavy pages still fit the bound; a refusal after some batches committed says how many pages that run already deleted; a dry run, a missing checkout, an unconfirmed operation and an admission-hooked call never batch. The final transaction's sources delete runs removedSourceScrubCtes (#9068: the source's retrieval and volunteer rows, and its slugs in eval candidates), so a run that stops after some batches leaves those rows until a retry commits. Each batch deletes inside withCoordinatedWrite with local-CLI attribution (the pages guard needs the source in gbrain.write_sources; only the final sources delete sets gbrain.topology_change). Committed batches are durable: a retry (same or new request_id) resumes, and the receipt's pages_deleted counts the pages deleted by the run that commits it. #8984: the first batch's transaction records the admission, a persistence_topology_changes row in state recovering with no recovery record, bound to the source incarnation it deletes from (isRetireAdmission); recordTopologyChange settles it committed, and a remove or purge that deletes that incarnation under another request, or any other lifecycle operation that commits a change to it (archive, restore, rebind, claim), settles it failed (supersedeTopologyRetires, which skips only the caller's own principal and request_id pair). A retry of the request_id against another incarnation (the source was removed and re-added) refuses source_retire_superseded, settles the admission failed and deletes nothing, also when a since-replaced CLI registration (another principal) admitted that request_id for a remove or purge, since idempotency is per principal; an admission bound to the current incarnation still resumes under the new registration; deactivation and graduation list an unsettled admission as a blocker that names its request_id and asks the user whether to finish it or keep what is left; an automatic expired-archive purge's admission (expired_only in its outcome) is left to the next automatic run with needsUser: false, and sources writer status --json lists the stopped and superseded requests (topology_requests). Tests: test/source-retire-batched.test.ts (PGLite), test/e2e/source-retire-batched-postgres.test.ts (60,000 pages), test/source-retire-incarnation.test.ts and test/e2e/source-retire-incarnation-postgres.test.ts.

  • src/core/persistence/label-edit.ts + src/core/repair/label-write.ts (GST-2468) — the label-only page write of operator repairs (repair visibility, repair private-labels). label-edit.ts: LABEL_KEYS (visibility, access, confidentiality), readLabelEdit validates a managed_maintenance_page intent's label_edit (set plain-word values, remove keys), spliceLabelLines removes, replaces or appends label lines in a file's frontmatter text and declines (null) a block or multi-line value, a duplicate key or a file with no frontmatter block. page-prepare.ts labelEditFile uses it in compareFileTarget when the file matches the snapshot: the edited file (or its edited head over the write's body, for a fence row) must read back as the file the write would render, else the write refuses label_edit_refused (GST-2603; reason file_differs with the differing keys in detail, or label_line); the file is never re-rendered under a label edit. GST-2603: labelEditFit (also used by writeLabelFile) aligns the file with the stored page before comparing, and align-stored.ts keepStoredOnlyStamps (used by alignWithStored and sync's keepStoredStamps) keeps the stamps gbrain holds only in the database (the validate: false grandfather stamp and a stored legacy_type whose aligned type is the page's), so they never count as a difference; fileSnapshotDifference lists the differing keys (frontmatter keys, title, type, tags, body, timeline) for the raw and inferred reading and fileMatchesSnapshot is its empty case. label-write.ts labelFileRefusal is a write-free preflight (managed: prepareFileTarget, with the drift's keys from error.localEdit and gbrain sources reconcile <id> <slug> --preview as next step; unmanaged: ownerRepairFile, next step gbrain sync --source <id>; repo-less or no file: null); visibility.ts holds refusing pages in the preview (details.held, residuals.held_file_drift / held_label_edit, outside the hash and approved set) and labelWrite / private-labels.ts prepareWrite throw only its label_edit_refused before writing. repair/core.ts counts label_edit_refused as declined and never toward item_failures_systemic. Test: test/label-repair-derived-stamps-gst2603.test.ts. label-write.ts: writeLabelPage publishes through the maintenance door with keep_trust and the edit on a managed brain, else importAtRevision (maintenance transaction, page lock, withTrustKeep); filesInCheckout names the unmanaged pages whose checkout file would undo a database-only write; for those, writeLabelFile applies labelEditFile to the file's own bytes (it must still hold the page, ownerRepairFile), imports it under withTrustKeep and writes it back through writePageThrough (fileText, preimage checked) in one transaction; drift or a file edited mid-write throws source_changed, and a file it cannot write (sync.write_through off, url_clone, an unresolvable target, a write error) returns a held file_backed outcome with the import rolled back. repair visibility calls it directly; private-labels still holds those pages as file_backed; repair/origin-carry.ts (GST-2689) uses writeLabelFile, importAtRevision and publishMaintenancePages the same way to label derived pages private before their origin row goes, and parks the origin when one returns held or throws. GST-2476: label-edit.ts also edits readable_by (READERS_KEY, EDIT_KEYS; set takes a plain-word list, rendered readable_by: [a, b]), and writeLabelPage takes a maintenance batch (then publishMaintenancePages([page], {batch})). src/core/repair/readers.ts is repair visibility --readers: checkedReaders (members exist and are enabled, groups exist, else access_member_invalid; a list that leaves out the owner refuses unless allowWithoutOwner), planReaders (named pages or a --slugs-file, up to READERS_MAX_PAGES, approved set bound to each page's live readers) readersWrite (re-checks a page, holds a segmented one, renders the edit) and applyReaders (phase READERS_PHASE, one label-only write); visibility.ts applyBatch admits a managed source's run of readers items together through publishMaintenancePages under the run's batch (Postgres publishes them in groups of up to 8), so the Git effect commits a window's ready files together; readersGitCounts reports the run's Git effect states. visibility.ts labelWrite holds a managed page with no canonical file outside storage.db_only as canonical_file_missing. GST-2519, labels under storage.db_only: sync-classify.ts gitVerdict keeps a strict source's db_ahead path when state.dbOnly (a label repair on a page whose file Git still tracks there survives later syncs; a commit that moved still imports); sync-prepare.ts imports a db_only file over a live page through withStoredLabels(text, stored, incoming), which adds each EDIT_KEYS label the row holds and the file does not state, so an import there never widens (a label the file states applies, wider included); page-prepare.ts prepareFileTarget lets a label repair of a strict page stamped git_ignored, with no recorded file and a path Git still ignores, write the database only instead of holding it canonical_file_missing. Tests: test/db-only-labels-gst2519.test.ts, test/sync-classify-strict-g3.test.ts.

  • src/core/persistence/page-edit.ts — edit_page (#5616): splits the canonical serialization into editable text and protected takes/facts segments whose remote view matches get_page's sanitized content byte for byte; applyPageEdits applies ordered exact-once replacements inside editable text only (typed edit_no_match/edit_ambiguous_match/edit_protected_span/edit_invalid), so canonical fences never move; editDiff caps the caller-view diff at 8 KB; boundedUnifiedDiff keeps its LCS table under EDIT_PAGE_DIFF_CELL_BUDGET (4M cells) by diffing only the window between common leading/trailing lines, and a window still over budget becomes one coarse hunk flagged diff_coarse (pinned by test/large-pages/edit-diff-bounded.test.ts). Called at submission (page-mutations.ts, before admission; skipped for a segmented page) and again on the locked snapshot in page-prepare.ts, which gives edit_page put_page's revision-bound editing timeline semantics. Pinned by test/edit-page.test.ts. within and the segmented path: large page writes.

  • src/core/repair/open-facts.ts + src/commands/repair-open-facts.ts (#8194) — gbrain repair open-facts, the operator-only, explicit-only, preview-bound repair that opens facts stamped private before facts.default_visibility world. openFactsReasonSql is the one exclusion expression the preview, its reason counts and the locked apply share (expired, superseded, member page, withdrawn or purged at any visibility, any pending trust proposal or a rejected or undone widening, a holding, provenance or entity page that is derivationPrivateSql on any row with the slug, deleted or not, a missing or quarantined holding or provenance page, an unresolved provenance slug); openFactClassSql classes facts fenced, unfenced or unpaged:<lane> (no page or a transcript lane; opened only when named). Keyset windows (--after, --limit), the approved set binds the window and an apply refuses preview_changed when a new fact became eligible inside it. Writes lock the named pages FOR SHARE and each fact FOR UPDATE before the recheck: database-only rows change in place (managed: the managed_maintenance_open_fact_visibility intent, prepareOpenFactsIntent; else a maintenance transaction); fenced rows go through the label-only door with the cell set by strikeFenceRow (managed: publishMaintenancePage with openFacts, whose keep_trust publication openFactsPublication runs under persistence/context.ts withOpenFacts, the only setter of gbrain.write_open_facts, which openedFactRowSql reads in trust/fence-guard.ts, generations/adapters/fact-rows.ts and canonical-projections.ts so the row moves to world in place; unmanaged: the import, then the in-place update). The CLI keeps the write-ahead ledger (repairLedger in repair-private-labels.ts) and --undo re-privates what it opened. Refuses remote callers (trusted_cli_required) and GBRAIN_ENFORCED_MEMBER (member_mode_refused). Test: test/repair-open-facts-8194.test.ts (PGLite and Postgres, with a two-session race).