Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Key files: Core Persistence (Git)
Subsystem index · Core Persistence · continued. Read only the entries relevant to your change.
Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.
The managed Git writer (gbrain sources refresh), git-canonical memory (the git gap census, declared database-only classes, the door) and the checkout reads beside them.
src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts—gbrain sources refresh(F0), the one sanctioned managed Git writer.persistence_worktree_refreshesholds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps torefresh_in_progress).refreshWorktreeprechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardenedgit fetchbounded bysources.refresh_fetch_timeout_ms, ancestry, dirty paths intersected with the incoming diff), then:draining(inserted underwithTopologyLocks+lockTopologyRows; admission injournal.tsrefusesworktree_refreshingfor every request with thatworktree_id, andassertManagedSyncAllowedrefuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/runninggit/withdrawal-mirroreffect and no recovery record remain (elserefresh_drain_timeout, aborted),fenced(both native locks held, rechecked in one transaction;refreshFenceClearstops request and effect claims), HEAD verified (#8617: arefresh_source_changedfor a moved HEAD or work that arrived after the drain check is retried once in the same run from a fresh precheck; a second refusal returns the first with the retry's cause appended), dirt rechecked,git merge --ff-only, HEAD verified again (a HEAD that descends from the target after a successful merge, with no member synced off it, counts as merged: a plain-git commit landed in between; a refused fast-forward whose HEAD only gained commits on the start commit abortsrefresh_source_changed, GST-1674), then one transactionmerged+ upstream observation +syncing; native locks are released and the managed--no-pullsync runs per member in one budget (sources.refresh_sync_budget_ms; past it a member isdeferred) (onlymanaged_sync_*submit_jobadmissions pass duringsyncing);completedwhen every member'slast_commitis the target or descends from it, or withoutcome.sync_blocked/sync_deferredwhen a member blocks or is deferred. A per-worktreerefresh-<worktree>.lockheld for the whole run proves liveness:resumeWorktreeRefreshes(called fromrecoverSourceTopologieson every consumer topology scan) touches interrupteddraining/fenced/mergedrows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; so does a fenced row whose merge started (outcome.merge_started_at, epoch ms) and whose HEAD descends from the old HEAD without the target, when the reflogs of HEAD and its branch show no move to the target since then (git refused the fast-forward, GST-1674; no readable reflog staysrecovery_required); HEAD at the verified target moves tosyncing; anything else isrecovery_required, which admission refuses withrefresh_recovery_required). Admission also completes asyncingrow whose members all reached the target. #7679: each row records its holder (outcome.holder: host id, hostname, pid, process start;previous_holderafter a recovery adopts it). A row whose lock is free is orphaned:resumeWorktreeRefreshesalso completes an orphanedsyncingrow once every member'slast_commitreached the target or a descendant, or (#9649) once the row has not moved for one refresh sync budget (sources.refresh_sync_budget_ms) with the checkout still at the target or a descendant, recordingcompleted_by: owner_reclaimand the lagging members insync_deferredas a run whose budget ran out does (#8072), and a plaingbrain sources refreshon the owner host finishes any orphaned non-recovery_requiredrow through the--resumepath (adoptOrphanedRefresh+resumeHeld) before its own refresh, reporting it asrecovered. A holder recorded on another host is never adopted.refreshHolderReport(writer statusworktree_refreshes, sync statusworktree_refresh) names the holder,liveness(live/orphaned/unknownoff the lock's host) and a fix naming--resume. While the worktree's newest refresh isaborted, every run (tick,--resume,--abandon, even one its precheck refuses) ends by running the managed--no-pullsync of each member behind the checkout HEAD (syncAfterUnfinishedRefresh, listed insynced), so a write held for an import that refresh fenced off is decided in that run, not at a later refresh.--resumeconverges and runs the member syncs;--abandonlifts a fence only with HEAD back at the start commit or on a row recovery would abort (a HEAD moved before the merge, or a refused merge, GST-1674), and never on arecovery_requiredrow once a member is past it (membersPastStart: alast_committhat is not it or an ancestor, GST-1457: the next sync would soft-delete what it synced; or a managed-sync import or delete admitted after the row'sbarrier_sequence, GST-1465: a pass stopped before its stamp), checking and aborting under every member's drain lease (underMemberLeases, so a pass in flight refusessync_drain_lease_held), and--resumefinishes arecovery_requiredrow only at a HEAD that contains every member commit on that refresh's line (adoptsHead, which--abandonalso reads to name--resumeinstead of a reset).keepPagesWayOutnames the way out (GST-1472): a fast-forward to the newest member commit on the target's line, a merge of member commits on diverged lines (ask the user), or a fetch when git cannot place a member commit (ancestrytells exit 1 from a failure); the refusals also name the revert route for a deliberate rollback (GST-1477), and off the owner host writer status names no step.gbrain sources writer statuslists active refreshes asworktree_refreshes(state, heads, the next command). Doctorworktree_refresh_stucklists rows active over 15 minutes. CLI body:src/commands/sources-refresh.ts(#7685:--jsonprints a refusal's structuredfixwhen it has one;refresh_in_progresscarries the writer-status read); while a resident PGLite owner (gbrain serve, stdio MCP) holds the database,src/commands/sources-refresh-delegate.tsruns the refresh inside it through thewriter_refreshadministration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe appliesrefreshFenceClearto effects like the claims, so a fenced worktree's queued effects do not wake it. Tests:test/persistence-worktree-refresh.test.ts,test/persistence-worktree-refresh-restart.test.ts(child-process crash after merge, and after a refused merge),test/worktree-refresh-orphan-7679.test.ts(SIGKILL mid-syncing, live-holder no-steal),test/oversize-hold-rescreen-8072.test.ts(budget),test/refresh-barrier-held-after-abort.test.ts(the sync after an aborted refresh), Postgres viatest/e2e/worktree-refresh-*postgres.test.ts, owner delegation intest/persistence-sync-stdio-owner.test.ts.src/core/persistence/refresh-barrier.ts(+ deferred admission and claim injournal.ts, migration v292src/core/schema-migrations/v292-refresh-write-barrier.ts) — the refresh write barrier (docs/designs/REFRESH_WRITE_BARRIER.md,docs/guides/write-refusals.md#deferred-writes). Off by default (persistence.refresh_barrier). With it on, a single page or memory write that reaches a worktree during a refresh is journaled in statedeferred(tied todeferred_refresh_id) and answered pending instead of refusedworktree_refreshing. Only a claim that declares the barrier moves a deferred row, and only after its refresh is released.deferredReplayRefusalre-checks the write's authority and reads the refresh'sold_head..target_headdiff with git, so a file the refresh changed endsrevision_conflictwith nothing applied. A write still waiting pastpersistence.refresh_barrier_max_wait_ms(15 minutes) endsrefresh_barrier_expired. A write during a refresh inrecovery_requireddefers too, and since GST-1582 there is no per-refresh cap (persistence.refresh_barrier_max_deferredis no longer read). A write whose page file holds a committed change sync has not imported yet is held withdeferred_import_from(migration v294) instead of refusedpending_import, and endspending_import_expiredpast the wait limit (or four wait limits, at least four hours, after admission when it is held again and again). GST-1623: either kind of deferred write endsrevision_conflictwhen a later write to its page committed first, and a refresh-deferred write is checked against its refresh before an import hold, which then clearsdeferred_refresh_id.refresh-retry-hint.tsderivesworktree_refreshing'sretry_after_msfrom the worktree's refresh history.gbrain upgrade --check-downgradeexits 3downgrade_blocked_deferred_writeswhile any deferred write is unfinished. Tests:test/refresh-write-barrier.test.ts,test/refresh-barrier-state-audit.test.ts.src/core/persistence/git-gap.ts+src/commands/sources-git-gap.ts+src/commands/doctor/checks/db-ahead-of-git.ts— git-canonical memory, wave G1 (docs/designs/GIT_CANONICAL_MEMORY.md#the-git-gap-census), read-only.declaredDatabaseOnlyis the one predicate for pages that may not have a tracked file, tested in order: a read-only mirror source, a private page (isPrivatePage), an alias-private page (isDerivationPrivate), lineage-private (computed in SQL fromprivatePagesFilterFragment),database_only_reason, astorage.db_onlyprefix (isSourceDbOnlySlug, refusing an invalid gbrain.yml), a never-filed derive-phase page (page-prepare.tsisNeverFiledDerivedPage: nosource_path/source_uriunderDERIVE_PHASE_DB_ONLY_DEFAULTS), a path the checkout's Git ignores (ignoredPaths: filter-freecheck-ignore -z --stdin, tracked files never reported, exit 1 = none, null when Git cannot answer). The equality contract compares a committed blob with the page's export view (readExportPage, withdrawals overlaid; the census reads a batch's views withreadExportPages, the same snapshots in two statements, #7617): an inline blob parsed as reconcile parses a canonical file, aligned with the stored type, subtype and #6429 stamp and compared bycontentCanonicaldigest (reconcileCanonicalwithout the write gate's database-only markers, so a flagged page is not reported behind its own file, #9511); a segmented page by the Git blob id of its streamed export. HEAD is read with hardened plumbing only (readHeadTree: onels-tree -r -l -z --full-treeunder the source's directory;readBlobs:cat-file --batchin calls of at most 64 MiB) and the working tree by hashing raw bytes (worktreeBlobOid), nevergit status/git diff, so no repository filter, textconv or fsmonitor runs.gbrain sources git-gap <id> [--sample n] [--json](trusted local CLI, not an MCP op) puts every live page in one state (equal,database_ahead,git_aheadwhen the sync import record'sblob_oid/disk_blob_oiddiffers from HEAD, else folded intodatabase_aheadand counted indatabase_ahead_without_import_record, split by content evidence indirection_evidence,declared_database_onlyper class,undeclared_database_only,worktree_only,unparseable,unsafe_path), reports the working tree separately, and counts rows with no Git home (unfenced active facts, fenced facts on a missing page or a page without a committed file, links whoselink_sourceis not inDERIVED_LINK_SOURCES, timeline rows on pages without a file), opensync-holdrecords by code and parked effect targets by kind and code. Each 256-page batch reads in its own read-only transaction. Doctordb_ahead_of_gitruns the census per non-archived source whose directory is a Git checkout (others are only counted) underGBRAIN_DOCTOR_GIT_GAP_TIMEOUT_MS(default 10 s) and the entry deadline; the census reads theDOCTOR_GIT_GAP_NEWEST(800) most recently changed live pages per source (censusGitGapoptionnewest, keyed byupdated_at; the census recordsnewestandnewest_since, the oldestupdated_atit read) and the message states that window (checked the <n> most recently changed of <live> live page(s), back to <time>). The gap is informational (never a warning); a source the budget stopped or never reached (PARTIAL, <read> of <live> live page(s) read) or one that could not be compared makes the check the unknown convention (warn,health: unknown), never ok (GBRA-541). It lists totals only with each source'slive_pages(counted in SQL before the scan). Tests:test/git-gap-census.test.ts(PGLite and Postgres).src/core/persistence/git-declared.ts+src/core/persistence/git-canonical-schema.ts(migrationgit_canonical_declared) — git-canonical memory, wave G7a: the declared database-only classes of a strict source (sources.config.canonical = "git", default off).strictPageClassjudges a page by its slug and prior file (declaredPathClass:db_only_prefixfor astorage.db_onlydir,derive_folderfor a never-filed derive-phase page; G3's indexer reads the same function) and then its rendered content (privacyClass:private,alias_private, orlineage_privatefrom the same SQL remote reads use,privatePagesFilterFragment, over the page's own values).page-prepare.tsprepareFileTargetasks it before it builds a strict file target: a declared page gets no target (no file, no door commit, no Git effect), an untracked ignored target isgit_ignored, a page turning private gets the removal of the file it still has in the worktree or at HEAD, and a path whosefilterattribute islfspassesassertLfsTargetonly withstorage.git_lfsand a checkout that configures the LFS filter (git_lfs_not_enabled).preparePrivateTransition(page-prepare.ts) is that removal for other writers (GBRA-60, connector imports #7971). InpublishMutationandpublishGroup,assertNoPrivateCommitrefuses a file target carrying a private page's bytes before anything is written (git_private_commit_refused; a group rolls back whole), anddeclareStrictPublication(after apply, before the door commit) refuses a declared page's file or an undeclared file-less page (git_home_required), stampspages.database_only_reasonand, for a privacy class, clearssource_path/source_uriin the transaction of the door commit that removed the file. A managed import builds its own file target, so it meets only these publication checks.assertStrictGitHomeis the refusal for writers with no file form yet (G4, G5). The managed writer guard (writer-guard-schema.ts) refuses rows of a strict source unless the writer declaresgbrain.git_protocol(withCoordinatedWritedoes), behind thegit_canonical_brain.strict_sourcescounter kept bysourcestriggers (git_protocol_required, mapped inpublication-failure.ts). The guard reads its state as the writing role and fails closed: a hidden or missingpersistence_brainorgit_canonical_brainrow, or a source or page row hidden by row-level security, raiseswriter_coordinator_required(managed_writer_guard:unreadable_state), and the counter refuses a recount under RLS onsources(test/writer-fence-fail-closed-7636.test.ts). The census (git-gap.ts) addsdatabase_only_source(a source with no checkout) and maps the stamped reasons to their own classes. Tests:test/git-canonical-declared-g7a.test.ts(both engines).src/core/persistence/checkout-state.ts(readCheckoutState) — Q1: a canonical checkout against its last fetched upstream, read-only and time-bounded (no fetch): ahead/behind,diverged, each commit the upstream lacks (sha, author, date, subject, paths; at most 20) and porcelain counts.sources writer statusreports it ascheckouts; doctorgit_convergenceadds the commits and counts for a managed checkout and warns at once on a diverged one.src/core/persistence/git-door.ts: G2, the door of git-canonical memory. A source withsources.config.canonical = "git"(strict, default off; set bygbrain sources canonical <id> --to git --apply --expect <hash>,git-conversion.ts) commits each published file before its database row.publishMutation(coordinator.ts) andpublishGroup(group-publish.ts, one commit per group, after its lane turn) callcommitThroughDoorunder the writer lock, between the file rename and the transaction commit. It commits by plumbing without the checkout's index:hash-object -wper file, the trees along the changed paths rebuilt from HEAD (cat-file tree,hash-object -w -t tree),commit-treewithcommitGitTargets' message,update-refcompare-and-swap (one rebuild if the branch moved, then a refusal), thenupdate-index --cacheinfofor exactly the published paths (an entry someone staged during the write is left as it is, and the write says its commit landed). That refresh runs under Git's own index lock:syncIndexcreates<index>.lockexclusively (an existing lock the #6932 proof finds abandoned is reclaimed under the caller's writer lock throughreclaimAbandonedIndexLockineffect-git.tsand taken again; any other existing lock refuses with the landed failure, which says why it was not reclaimed), compares the entries, updates a hard-linked copy of the index throughGIT_INDEX_FILE, and renames the copy onto the lock and the lock onto the index, so no writer that takes the lock can stage between the compare and the write; it also sweeps dead processes' index copies (#7755), retries win32 renames (#7756); detached HEAD and unborn branches refuse typed. EachDoorFilecarries its planned state (planned: the sha256 of the bytes the publication wrote, or null for a removal), and the door commits exactly that state or nothing: the file must hash to the plan before and afterhash-object -w(elsegit_target_changed: nothing is committed and the edit stays on disk; the write is held in recovery asunexpected_file_byteswhen the file holds neither its bytes nor the version before it, and fails with nothing held when the file holds the version before it), and each target's index entry must equal HEAD's (elsegit_index_conflict, index untouched). An entry that still holds the version its path's last commit replaced (an interrupted door refresh) counts as clean: the write restores it to HEAD and reports the paths asoutcome.persistence.git_index_restoredand thegit_index_restorednotice (indexRestoredNotice, emitted bypage-mutations.tsemitFenceNotice). The step deadline is sized from one stat of the index (gitStepTimeoutMs), andrefuseIgnoredaskscheck-ignoreonly about new targetsignore-rules.tscannot clear. Door commits are not coalesced across single writes (each holds its worktree lock exclusively and the consumer runs one per root); a batch commits once through publishGroup. The publication stamps the commit asoutcome.persistence.git_commit(git_state: committed); a target Git refuses rolls the publication back. The Git effect carriespush_onlyand joins the push without staging (effects.ts). The recovery record carriesdoor(checkout and starting HEAD);recoverPublicationkeeps a file whose commit landed (doorCommitLanded) instead of restoring it, and the replay completes against it:replayAfterDoorCommitinpublishMutationand the strict exceptions inpage-prepare.ts(own bytes committed, or a removal HEAD already holds). Old binaries ignoredoor; strict stays off until conversion fences them. Tests:test/git-door-strict-gbra133.test.ts(both engines),test/git-door-plumbing-gbra133.test.ts,test/git-door-planned-state.test.ts,test/git-door-write-cost.test.ts,test/git-door-index-lock-7742.test.ts(both engines); per-write overhead benchscripts/bench/door-write-overhead.ts(opt-in, smoketest/bench-door-write-overhead.test.ts).src/core/persistence/ignore-rules.ts— the ignore filter in front ofgit check-ignore; Git stays the only judge of an ignored path.ignoreCandidates(root, paths)returns every path some positive ignore rule could match, read from every.gitignorefrom the top level down to the path,info/excludeandcore.excludesFile; a path it drops is one no rule could match. It over-approximates: negated rules are dropped, every wildcard matches any run of characters including/, directory-only rules match files, letters compare case-insensitively, each rule is tested against the path and every directory above it in its own and NFC spelling, and an unreadable or non-UTF-8 ignore file or a rule with a backslash or a character outside printable ASCII matches everything under its directory. Every path goes to Git on a sparse checkout or skip-worktree entry, a split or sparse index, an unresolvablecore.excludesFileor include, a gitlink or symlink along the path, or a failed probe. The probe (rev-parse layout and the configuration with its origin files) is cached per checkout and taken again when a configuration file it read or could read, HEAD or the configuration environment changes; ignore files are re-read on every call and parsed once per content. Callers:git-door.tsrefuseIgnoredandsync-origin-evidence.tsgitIgnoresUntracked. Tests:test/ignore-rules.test.ts.