Garry's Opinionated OpenClaw/Hermes Agent Brain
Key files: Managed sync bookkeeping
Subsystem index. Read only the entries relevant to your change. Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.
What a managed sync records about itself besides pages: the durable failure row and its local-ledger mirror, the orphan
classifier and repair, the doctor check over them, gbrain sync status's failure view, the write diagnostic of a blocked
run, and the supersession of a checkpoint whose cursor another run replaced. The cursor, manifest and drain themselves are in
Core Persistence and Managed sync and persistence stalls; sync-status.ts and
sync-fault-class.ts stay in Core Services (2/3).
src/core/persistence/sync-failures.ts— durable managed cursor/receipt diagnostics are the authority; JSONL is a compatibility mirror keyed by cursor and source/path. Failure observations deduplicate terminal receipts, retaining structured codes and attempts; success clears only its matching cursor. A failure records the cursor-selectingsyncOptionsandprocessingOptionsof its run (falling back to its cursor's when read), and every local hint (the thrown failure line, the blocked report, the terminal write diagnostic and local doctor) renders the exact retry throughmanagedSyncRetryCommand→checkpointRetryCommand; a failure recorded before options were kept says so instead of guessing.sync --retry-failedcounts only failures of the cursor key its own options select (managedSyncCursorKey) and lists the others with the action that settles each (managed-retry-actions.ts). #6712 PR 3 (L7):--retry-failedis a deprecated alias of the plain sync, which retries a recorded failure by itself; a failure records thegbrain_versionthat recorded it, so an unchanged content refusal the same version recorded replays without a new request until the file or the version changes. Local CLI shows request/run/pinned target details, while remote doctor returns source-scoped aggregates without paths, raw messages or receipt IDs. Missing durable state warns rather than certifying health. Failed receipts cannot be acknowledged, auto-skipped or rewritten; explicit idle ordinary-source retry creates fresh admission, while approved company manifests remain pinned. Tests:test/persistence-sync-failures.serial.test.ts,test/e2e/managed-sync-failures.test.ts. #6402: an observation movesupdated_at; a new target commit restartsfirst_seen/attempts; aninvalid_paramsresume refusal is not recorded. #6288: after a managed sync succeeds,clearSettledManagedSyncFailuresremoves the source's rows under other cursor keys (a--fullrun's) whose cursor is done or gone and whose file the imported commit settles;gbrain sync --source ID --acknowledge-managed REQUEST_ID_OR_PATH(host CLI) removes only such rows, refusing othersmanaged_failure_unsettled(sync-managed-stale-failures.test.ts). Wave 16 (#6513 comments): a failure recorded from a receipt carrieswrite_error_detail, the receipt'spublicFailureDetailprojection (origin, sqlstate, raiser, table, the closedreason; never message text),readManagedSyncFailuresre-reads it from the pending receipt'serror_detail, andformatManagedSyncFailureappendsraiser= table= reason=(publicationDetailText, identifiers only), so the ledger mirror, doctor and the blocked report name the refusing trigger;sync status --jsonsurfaces it aslast_error.write_error_detailwithlast_error.detail= the reason (sync-status.ts).src/core/persistence/managed-sync-orphans.ts+src/core/repair/managed-sync-orphans.ts—gbrain repair managed-sync-orphans --source <id>(#5459), explicit-only and preview-bound (approved set,preview_changedon drift).classifyManagedSyncOrphans(read-only; doctorsync_failuresnames the repair when it finds any) takes the source's managed-sync bookkeeping and classes each rowno_cursor(a failure row with no cursor to resume),principal_unadoptable(an unfinished cursor recorded by a principal that is no longer an active local writer, by join on the recorded principal),superseded(a pre-options cursor a later run replaced),incarnation_retired(#6518: an unfinished cursor or a failure row recorded under a source incarnation the source no longer has, joined on the source id alone, archived sources excluded, a live pending request still retaining it) ormirror_only(#6518: a local ledger row with amanaged_cursor_keyand no durable failure row under that key;mirrorOnlyManagedFailures, which throws on a failed durable read rather than naming every mirror), and keepspending_request_liveandresumablerows untouched.retireManagedSyncOrphanretires one row atomically: anincarnation_retiredcursor needs the source still active under another incarnation at apply time and skips the live-writer veto (no retired key is resumable); amirror_rowdeletes nothing durable and prunes the ledger row only when the durable read, repeated at apply, still finds none.pruneMirrorOnlyManagedFailuresis the successful-sync path (clearSettledManagedSyncFailures). Doctorsync_failuresclassifies whenever the brain is reachable, so the diagnostic is reachable with no current-incarnation failure visible, and reports a failed classification as unknown.sources set-strategypoints at it when an unfinished run blocks a strategy change. Pinned bytest/repair-managed-sync-orphans.test.tsandtest/managed-sync-orphans-incarnation-6518.test.ts(both engines);test/repair-explicit-only.test.tspins its place last inREPAIR_KINDS. #6712 PR 3 (L7):slug_mode_refusedis an unfinished cursor whose every page committed and whose checkpoint was refusedslug_root_mode_mismatchwhile the source still pins the other mode (slugModeRefusedSql, bound to the run and request at apply); no sync rediscovers it, so retiring it is how a person keeps the pin, and its slice record is cleared likecheckpoint_moved's.src/core/persistence/sync-cursor-key.ts— #6712 PR 1: one managed cursor per source.managedCursorKeyis a digest of the source incarnation and, for a company approval only, the company; writer principal, authority and options no longer select a cursor.resolveManagedCursoris the one boundary that picks a run's key and folds older keys in crash-safely under the drain lease: the canonical unfinished cursor wins, else the newest older-key unfinished cursor is adopted and finished in place under its own key; every other older cursor has its queued requests cancelled and is retired throughretireManagedSyncOrphan(reasonsuperseded_by_adoption, fenced byassertDrainLease). A cursor with a running, recovering or publishing request or a recovery row is retained and the pass waits for that request (writer_pending) before it drains anything. A company brain's finished old-key cursors are kept. Pinned bytest/sync-pass-handover-6712.test.ts(the live pass's handover of an older binary's cursors) andtest/persistence-sync-cursor-strategy-6696.test.ts.src/core/persistence/sync-lease.ts— #6712 PR 1: the drain lease, one managed drainer per source.withDrainLeasetakes the source's sync lock (syncLockId) with a 2-minute TTL renewed every 20 s, so a dead holder's lease lapses inside one cron tick; it is reentrant for the holder (nested passes and the in-process lanes reuse it). The holder record (managed-sync-leasecheckpoint: kind, id, version, start) plus the lock row givereadDrainHolder(host, pid, heartbeat,progress: commits, last commit, last cursor save). A non-job holder also writes a beacon row in thegbrain-drain-leaseminion queue (application authority,max_stalled0, renewed with the lease, deleted on release) so an older binary'sliveSyncJobbacks off; downgrades are quiescent (stop serve and crons first).assertDrainLeaseis the fence inside every transaction that admits work, settles or holds an entry, folds a cursor in or swaps the cursor. Following callers (the CLI, a CLI run delegated to a serve) wait for a live holder within their budget (followStopAt) and take over when it releases; out of budget, or after losing the lease, they returnleaseHeldResult(resumable,sync_drain_lease_held,next.retry_after_ms60 s,safe_to_loop); a holder with no progress forHOLDER_STALL_MS(5 min) returnsleaseStalledResult(blocked,drain_stalled, causeno_progress,stall.lease_holder). Jobs and the cycle getSyncLockBusyErrorand skip. #7596:drainLeaseLiveisisLockHolderLive's rule at the drain TTL (readDrainLeasereturns it with the holder, andliveSyncJobcounts only a live holder); a killed holder's lock row outlives it until a takeover, so status reads it as not live. Pinned bytest/persistence-sync-lease.test.ts.src/core/persistence/sync-idle-skips.ts— GBRA-133 (PR 2b): the managed no-op fast path. An idle tick (stampIdlePass) records on the drain lease (idleClear) whether its pin read no hold of the incarnation, a zero hold summary, nomanaged-sync-failurerow of the source and no owed tail (readIdleFactsrides the pin's round trip).idleSkipsnames the settles that then have nothing to do (IDLE_SKIPS: the hold report inperformManagedPass; settled failures, which also needs no ledger row of the source; the needs-human sweep, the sync tail and both heals inperform.ts). The tick, its lease, its pass row and its result are unchanged; the upstream observation, the remember replay and the ghost-page report still run.GBRAIN_SYNC_IDLE_FAST_PATH=0runs every settle. Pinned bytest/sync-idle-skips-gbra133.test.ts(both backends; Postgres twin intest/e2e/).src/core/sync-dispatch-read.ts— GBRA-133 (PR 2b): the routing read of a single-source CLI sync.withSyncDispatchRead(src/commands/sync/run.ts, never--allor--watch) answers the source resolver, the company profile,resolveSyncPersistenceMode, the content-directory anchor, the managed-flag refusals, the delegation ladder's mode check andperform.ts's connector check from one combined row per source (syncDispatchRow: the sources row, the claim, the company receipt and the mode rows).endSyncDispatchReadends the scope for good where the work starts (performSynconce the route is chosen,maybeDelegateManagedSyncToServebefore the hand-off), so every read under the drain lease and after a write stays live; outside the scope, or when the combined read fails, each reader runs its own query. Pinned bytest/sync-dispatch-read-gbra133.test.ts(both backends; Postgres twin intest/e2e/).src/core/wire-skew.ts— #6712 PR 1: the one record of wire-skew windows, each a version floor plus a calendar date (WIRE_SKEW_WINDOWS.explicitProcessing: floor 0.60.255.0, accepted and ignored until 2026-11-14). The delegation and admin wires read the field name from here, so removing the window is one edit. #6712 PR 2: each window names the hand-offs that accept it (wires) and its shape (accepts);retiredWireFields/retiredWireNoticeserve both validators.CONFIG_ROLLBACK_WINDOWSrecords the rollback config keys' windows (honored until a date, from a version floor, with the notice that named them).src/core/sync-flags.ts— #6712 PR 2:SYNC_FLAGS, everygbrain syncflag once, with its route (forward/client/refuse) on the serve and PGLite owner hand-offs;syncFlagsRoutedandsyncWireFieldsderive the ladder's, the IPC validator's and the owner lane's allowlists. A flag the argv parsers read must be listed (test/sync-flag-table-6712.test.ts). #6712 PR 3 (L7):managed: truemarks the ten flagsgbrain sync --helplists under "Managed brains" (MANAGED_HELP_FLAGS, table order;test/sync-managed-help-l7-6712.test.tspins the section and runs doctor's cron recipe through the managed flag validator).src/core/sync-config-keys.ts+src/commands/doctor/checks/sync-config-keys.ts— #6712 PR 2: thesync.*keys in three kinds:SYNC_TUNABLE_KEYS, the rollback keys (windows inwire-skew.ts) andRETIRED_SYNC_KEYSwith theirGBRAIN_SYNC_*replacements (retiredSyncSettingreads one).effectiveSyncKeyisconfig get's value-in-effect and origin for a retired or rollback key (its resolvers mirror the readers:resolveBulkSettings,waiverBatchEnabled,isFederatedV2Enabled, the write switches,planJobDrain);syncKeySetNoticeisconfig set's warning (a retired key writes nothing);syncRepoPathRefusalisconfig set sync.repo_path's refusal of a relative value (nothing written). Doctorsync_config_keysreports a stored retired key as unapplied. Pinned bytest/sync-config-keys-6712.test.tsandtest/error-registry-retired.test.ts.src/core/persistence/sync-admission.ts+src/commands/config/sync-admission-value.ts+src/commands/doctor/checks/sync-admission.ts— #6712 PR 4b:sync.admission(normal | imports_only | classify_only, brain-wide, aSYNC_TUNABLE_KEYSentry).readSyncAdmissionparses it (junk fails closed toclassify_only) for the pass's pin, group admission and destructive publication, andadmissionAllowsgates a write kind.config setrefuses any other value (exit 2,--forceincluded) andwriteSyncAdmissionstores the value and itsset_at(database clock) in one transaction: config has no timestamp, soset_atis anop_checkpointsrow (op='sync-admission', fingerprintbrain) that dates the stored value only while it names that value (otherwiseset_at: null).config getprints the value in effect (handleSyncKeyConfig).syncAdmissionStatus(engine, source, incarnation)issync status'sadmission: {value, set_at, effective}and the helper the pass'sconvergence.admissionuses:effectiveis false while anopenpass row withstarted_atbeforeset_atholds the livegbrain-sync:<source>lease (token match), or amanaged_sync_delete/canonical_reconcilerequest of the incarnation admitted before it is queued, running or recovering (an unknownset_atcounts everything as earlier). Doctorsync_admission_restricted(ops tier) warns on a managed brain while the value is notnormal, with its age and the user's restore command; ok otherwise. Never settable remotely (configrefuses on a thin client and no operation writes config). Tests:test/sync-admission-6712.test.ts.src/core/persistence/managed-sync-flags.ts—managedForbiddenFlag/managedSyncFlagRefusal: themanaged_sync_flag_refusedrefusal for--skip-failed/--include-gitignoredon a managed source (#6696), raised atgbrain syncparse, inmaybeDelegateManagedSyncToServebefore the socket, and byresolveManagedSyncContext.src/commands/doctor/checks/sync-failures.ts— reads managed durable cursor/receipt state even when its compatibility ledger is absent. An unavailable read is unknown/warn, not healthy; local findings carry diagnostics and remote findings contain only source-scoped aggregates. Legacy remediation stays separate from explicit managed retry. Failures on archived sources (which sync refuses) are left out of the count and severity and only reported as an ignored total. #6518: with an engine,classifyManagedSyncOrphansruns whenever the brain is reachable (not only when a current failure is counted), so retired-incarnation and mirror-only bookkeeping is named with the repair preview even at zero visible failures, and a failed classification is reported as "could not be classified", never as healthy; without an engine (doctor --fast) only the unmanaged ledger rows enter the severity calculation, and mirrored managed rows are reported apart aswarn"could not be verified without a database connection" (uncertainty, never recovery). Tests:test/persistence-sync-failures.serial.test.ts,test/doctor-report-remote.serial.test.ts,test/doctor-sync-failures-archived.test.ts,test/managed-sync-orphans-incarnation-6518.test.ts.src/core/persistence/sync-write-diagnostic.ts—managedSyncWriteDiagnostic, themanagedWriteblock of a managed sync result (terminal receipt reason, suggestion with the retry command, the CRLF-only note), moved unchanged out ofsync-run.tsby #6340;writeWaitOf/ManagedSyncWriteWait(thewriteWaitblock) followed in wave 16.src/core/persistence/sync-checkpoint-replaced.ts— wave 16 (GBRA-79, #6513 comments): a managed-sync checkpoint whose cursor another run of the source replaced is superseded, not failed.checkpointCursorReplacedis the classification the shared validator (sync-prepare.tssourceSyncChecksconsumer) and the checkpoint's final locked compare-and-swap both apply before their own checks: a checkpoint intent whose durable cursor names anotherrunIdrefusessync_cursor_replaced(receipt stateconflict, throughcoordinator.tsconflictCode); a page request whose cursor moved keepsrevision_conflict(its run re-admits from the current cursor), a same-run index lag keepsrevision_conflict, a missing manifest keepsstorage_error.checkpointSupersededrecognizes that receipt insync-run.ts:settleFailedRequestrecords no managed-sync failure for it and continues with the winning cursor as it stands (currentCursor, the same move the ENG-A7 admission guard makes when the cursor moves under it; the winner's run id, index and manifest are kept, nothing of it is cleared or advanced), andretryFailedStartnever treats a checkpoint receipt as a page fault. Before, the loser's run recorded the conflict as amanaged-sync-failurerow, doctor reported it, and--retry-failedrediscovered from HEAD. Test:test/managed-sync-checkpoint-controls-company.test.ts(#6712 PR 4b: a company brain is the one cursor checkpoint left; controls: index lag, missing manifest).src/core/persistence/sync-slug-mode.ts— #6694: the checkpoint's slug-root mode against the source's pinnedslug_root_mode.emptySyncScope(source root is the repository root, both realpathed) letssync-prepare.ts's checkpointUPDATEaccept either pin and keep the pinned one; at a subdirectory a differing pin is diagnosed bypinnedSlugModeMismatchafter the guardedUPDATEmatched nothing and refusedslug_root_mode_mismatch(cause and the person's two choices fromslugModeMismatchCause); anything else staysrevision_conflict(CHECKPOINT_MOVED_MESSAGE).slugModeCheckpointRevivableis the second revival predicate ofsync-run.tsreviveCheckpoint: a checkpoint refusedslug_root_mode_mismatch, or an older release's moved-checkpointrevision_conflict, is revived whenlast_commitstill equals the run'sfromand the pin is unset, equal to the run's mode or the scope is empty;retryFailedStartdoes not consult it, so--retry-failedkeeps its rediscovery. Test:test/sync-slug-root-mode-6694.test.ts(both engines).src/core/persistence/sync-cursor-moves.ts— #6712 (L2, one page, one fate): the managed-sync cursor's pure transitions, peeled out ofsync-run.ts; each returns the next cursor and writes nothing (sync-run.ts saves it with one compare-and-swap).advanceHeld/settledPastpass an entry without a commit (held, waived viawaivedCursor, an idempotent deletion viadeletionSettled) and add its index to the settled ranges the checkpoint skips; a deferred entry (Cursor.deferred, asource_changedfile retried after the manifest end) leaves the deferred list instead of moving the index.promoteWindowmakes the window's first group the cursor's group once the cursor reaches it, so requests admitted after a failed or held entry are read by their own receipts, never frozen again;committedPastdoes the same after a commit.withHeldKeysrecords a held entry's page keys (dependencyKeys) so a later entry of the run that shares one is held behind it (blocked_on).src/core/persistence/sync-stop-class.ts— #6712 (L2; #6565): how the settle path insync-run.tsclassifies a page request that ended without committing.receiptClassreads the terminal classifier:transport(a dropped connection or a retryable refusal, re-frozen in place),page(held, deferred or blocked under the hold policy) orsystemic(the run stops with the classifier's action).resumableStopis a systemic stop the fault table retries (an unrecognised P0001 or a 55000 surfacing asstorage_error);stopRecordedsays an earlier invocation already stopped this run (its failure ledger row carries the run id), so the next plain sync re-freezes every page that refusal failed, bounded bySTOPS_PER_ENTRY. #6712 PR 3 (L7):recordedStopalso returns the row'sgbrain_version.retryRecordedFailureinsync-run.ts(what--retry-failed, now its deprecated alias, did on request) retries every recorded failure on the next plain sync, except a refusal nothing changed since the same version recorded it (replaysUnchangedRefusal: a content refusal while the file has its frozen bytes; asource_changedpinned import while HEAD is the run's target and the file still holds the working-tree bytes that refused, compared by hash rather than by mtime), a slug-mode refusal and a checkpoint whose source'slast_commitmoved (#6834), which wait for a changed file orgbrain repair managed-sync-orphans. GST-2869:SyncTerminatedis the abort reasongbrain syncgives its run on SIGTERM or SIGHUP;abortStopreads the first abort's reason, so the pass row endsterminatedinstead oftimeout(from its catch, its rolled-back pin or a batch that returnedtimeout) and the drain stopsterminated;managedPassLive(sync-pass-episode.ts, the pass-cost registry) says a managed pass is running in this process, the only time sync owns those signals (withTerminationOwnerinsrc/commands/sync/run.ts, CLI entry only, never under another owner).src/core/persistence/sync-knock-on.ts— #6643, shrunk by #6712 (one page, one fate): no producer is left. It keeps the legacy predicate over receipts older builds wrote (error_detail{ origin: 'sync_knock_on', reason, root_request_id }, or the pre-#6643 producers' exact messages, never an operator cancel):isSyncKnockOn/syncKnockOnSql(one predicate, never NULL) feed the terminal classifier, which renders such a receipt superseded, andsync-run.tsrefreezeInPlace, which re-freezes the entry when a resumed cursor meets one (bounded bySTOPS_PER_ENTRY, no config key).cursorFenceCausereads the two live causes of "cursor changed" (run_replaced,outside_fence; an olderwindow_droppedstill parses).checkpoint-validation.tsclause 3 skips converted receipts and the cursor's settled index ranges only; #6712 removed the knock-on exemption.reviveCheckpointre-assesses a refused checkpoint at the cursor's end.sync statusshows the cursor the recorded failure belongs to,cursors, andcheckpoint_blockers(listIncompleteSyncReceipts, each withknock_on).src/core/persistence/terminal-classify.ts— #6650:classifyTerminalturns one terminal receipt (failed/conflict/cancelled, or any row still holdingrecovery) into aTerminalVerdict(category, code, origin, class,safe_actions,needs_human,defect,historical,root_request_id, one structuredaction) at read time; nothing is persisted. A recovery record outranks the code (recovery_pending/recovery_blocked); a knock-on is read throughisSyncKnockOn(typed marker survives compaction) or the SQL twinterminalKnockOnSql;storage_errorandsource_changedroute by their recorded origin or cause, and a row whose code and origin do not say what happened isdiagnosis_required(read-only writer status, never a guessed fix). Asource_changedwhose canonical file is missing iscanonical_file_missing(a ghost page; its exit is thegbrain repair ghost-pagespreview, never a sync retry), and an origin-lessfence_unrenderableadoption refusal is its own defect category. A retryable sync fault'sactionfollows its operation (faultOperation: aput_pageunder acanonical_reconcileormanaged_*intent kind reads as that kind):submit_joband a canonical reconcile keep the sync retry, a caller-submitted page or memory write (RESUBMITTED_WRITES) gets the read-only writer-status inspect and a resubmit with a new request_id, amanaged_*write the inspect and a rerun of its command, any other operation (and a compactedput_page, whose intent kind is gone) the inspect alone withsafe_actions: ['none'].TERMINAL_FIXED_INnames the release that fixed a category; a row whose build (error_detail.attempt.consumer_version, elseadmitter_version) is older is historical.legacy_publication_refusalandretired_operation(arevision_conflictwith origingrandfather_retiredor amanaged_grandfatherintent, #7837) are historical whatever build wrote them, since only a request an older build admitted produces them. It also owns the supersession rulerepair/failed-writes.tsreuses (supersessionSql,supersessionOrderSql,dispositionOf,PAGE_REMOVAL_OPERATIONS). Tests:test/terminal-failures-6650.test.ts(TS↔SQL twin parity included),test/terminal-classify-6696.test.ts.src/core/persistence/terminal-failures.ts— #6650: the bounded readers over that classifier.readTerminalFailureSummaryis one set-based aggregate (GROUPING SETS per source;TERMINAL_STATEMENT_MS5 s statement andTERMINAL_ACQUIRE_MS2 s connection deadlines; past either it returnsstatus: 'unknown', never an empty pile) splitting active blockers (recovery records plus the passed-incheckpoint_blockersofsync-status.ts, read byreadCheckpointBlockerMap), unresolved rows (count, distinct slugs and intents, oldest age), historical rows (a later write superseded them, or older thanTERMINAL_FIXED_INvia the SQL twinTERMINAL_HISTORICAL_SQL), 24 h and 7-day daily counts, categories and defect classes; archived sources are skipped. Thresholdspersistence.terminal_failures.window_days(clamped topersistence.receipt_retention_days),spike_factor,spike_min(readTerminalThresholds,validateTerminalConfigValue, refused atconfig setbycommands/config/preparation-keys.ts);isSpikeagainstsevenDayMedian.readTerminalFailuresis the drill-down (batches of 200, each selected before the supersession lateral join, which with the summary's probe tests the failed row's intent kind first so later large sync intents are not detoasted per row (#8729), at most 2,000 rows scanned per call, a sequence cursor, verdict filters,--group-by categorygroups, async_faultgroup also split by its operation, with an exactdrill_downargv each;--operationfilters onfaultOperation). Consumers: doctorpersistence_terminal_failures(commands/doctor/checks/persistence-requests.ts),sync statusterminal_failures(sync-status.ts,nextfalls back to a recovery blocker's action), thewriter_failuresadministration operation, and the admin/metricsgauges (serve-http-metrics.tsrenderTerminalFailureMetrics, one aggregate per engine shared by scrapes forTERMINAL_METRICS_TTL_MS= 60 s viacachedTerminalSummary; never/health). Each source also reportsnew_last_24h_older_builds(rows and builds older than their class's fix), and doctor's spike message says an older write owner is producing them when they are most of the spike. Tests:test/terminal-failures-6650.test.ts,test/sync-status-terminal-6650.test.ts,test/terminal-failure-metrics-6650.test.ts, the CLI journeytest/agent-journey-terminal-failures-6650.test.ts.src/commands/sources-writer-failures.ts— #6650: output ofgbrain sources writer failures <source> [--state all] [--group-by category] [--since Nd] [--limit N] [--before <cursor>] [--category|--code|--origin|--operation] [--json], dispatched bypersistence-admin.tsaswriter_failures(validated inadministration.tswriterFailuresQuery, locally or through the resident owner). A table by default, one JSON document with--json; row and groupfix/inspectactions are rendered for the CLI; a full page printstruncatedwith the exact next command. The operation is advertised to clients in the owner'sadministration_extensions(ipc.ts,admin-contract.tsPERSISTENCE_ADMIN_EXTENSIONS), never in theadministrationlist an older client validates strictly, andlocal-client.tsdelegates it only to an owner that lists it (ownerAdministers). Tests:test/writer-failures-cli-6650.test.ts.src/core/persistence/public-category.ts— #6712 PR 2 (L5):publicCategory({ category?, code }), the closed public vocabulary (drift,conflict,content_refused,file_missing,permission,storage,config,superseded,transient,defect) that sits beside the unchanged diagnosiscategory.PUBLIC_BY_TERMINALmaps everyTerminalCategory,PUBLIC_BY_CODEeverySYNC_FAULT_TABLEcode (a new fault code needs a row:test/public-category.test.ts), and anything else goes by its registry class (unknown codes aredefect).classifyTerminalcarries it aspublic_category;classifyRecordedFault(terminal-classify.ts) reads a hold, a managed-sync failure row or a run stop as the failed receipt it stands for, so the receipt diagnosis,managedWrite, sync status holds andlast_error, the terminal summary and doctorsync_failures(details.failures) render one verdict.Pass round phase lines and honest stalls (GBRA-133 PR 3a, #7243;
src/core/persistence/sync-pass-run.ts,sync-pass-status.ts,sync-drain.ts) — each pass round emitsmanaged_sync.round_level(paths, target, read time) after its level read andmanaged_sync.round_admit(imports, deletes, in flight) before its import walk, and amanaged_sync.round_batch(batchofbatches, starting at candidatebase, #7866) before each import batch; an announcing drain prints the level and admit lines ([sync] pass round N: …). Theconvergenceblock'stargetis the newest pass's pinned commit (telemetry) andconverged_targetthe stored converged obligation's target, null while no stamp backs one, so status never presents the pin as converged. The drain counts only page and checkpoint commits as progress (a legacy-sweep slice or a level read still working only keeps the quiet clock fresh); an abandoned pass's late progress is dropped; a stop whose head owner reportsrestart_requiredisowner_restart_required(drainNextnames the process); a run that committed nothing (no advanced cursor, no committed page) stops assync_deadline_stalled(exit 1). Tests:test/sync-pass-phase-lines-7243.serial.test.ts,test/sync-status-truthful-7243.test.ts,test/honest-stalls-gbra133.test.ts.src/core/persistence/sync-tail.ts+sync-tail-record.ts— #6712 PR 2 (L5): the tail of a committed checkpoint (projection statistics, then link extraction for the run's imported slugs read from its manifest).recordSyncTailruns inside the checkpoint's publication transaction (applyManagedSyncCheckpoint) and appends{run_id, changed, extract}toop_checkpointsmanaged-sync-tail/<source>(at most 16), so the tail is owed before anyone can read the run as synced.withSyncTailruns it after the outermost drain lease is released (performManagedBranch, andperformManagedSyncwhen it took the lease itself; a call still inside a lease, a dry run and a lease-held follower run none), bounded bySYNC_TAIL_BUDGET_MS(5 min) and the caller's deadline, and foldstail(done/truncated/degraded/deferred/failed, withunderived: the run's imports its link extraction left underived, which the per-source "still owe extraction" line prints) andlinksinto the result. The obligation is cleared by compare-and-swap only when every step ran; a failed, deferred or truncated tail attaches a degradedsync_tail_degradednotice with its fix (the sync CLI prints it with the capacity notices). Fault pointsync:tail_started. Tests:test/sync-drain-finish-tail.test.ts.src/core/repair/slug-autoheal.ts— the sync-time heal of strayslug:lines (#6865):healAfterManagedSync, called once fromcommands/sync/perform.tsperformManagedBranchafter the drain lease is released, runsslug-conflicts'healAbsentSlugLine(deterministicabsent_pageonly, bound to the held bytes, nothing recorded for any other analysis) over the source's slug-conflict holds through the samesubmitManagedFileRepairwrite, with the receipt'sdeclared_aliasrecorded as aslug_aliasesrow in that write. Skipped (and said why onslug_lines_removed) undersync.slug_conflicts.auto_remove false,fences.repair.enabled false, remote submission authority, an unfinished managed-sync cursor, or a host that may not write the checkout; bounded bySLUG_HEAL_MAX_FILESandSLUG_HEAL_BUDGET_MS(or the caller deadline), one checkout scan per source; never throws. Lane verdicts carrylane_version(CONTENT_REPAIR_LANE_VERSIONinpersistence/sync-fault-class.ts): an older verdict isrepairincontentHoldDispositionand never a person's decision inclassifySyncFault. Tests:test/slug-autoheal-6865.test.ts(Postgres armtest/e2e/slug-autoheal-6865-postgres.test.ts),test/slug-conflict-verdict-6865.test.ts.src/core/since.ts— #6712 PR 2:parseSince(input, flag, now), the one--sinceparser (gbrain agent logs --since,gbrain repair file-drift --since). An age (90s,30m,12h,7d) becomes one concrete UTC instant at parse time; an ISO-8601 date or timestamp is normalized; anything else throwsinvalid_paramsnaming the flag. Callers print and hash the instant, never the age. Tests:test/sync-exits-l4-6712.test.ts,test/agent-cli.test.ts.src/core/persistence/sync-ghosts.ts— #6712 PR 2:countSyncGhosts(engine, sourceId), the post-sync ghost count. It reusesscanGhostSourcefromrepair/ghost-pages.ts(no second classifier) with a candidate bound ofSYNC_GHOST_CHECK_LIMIT(50) stats, leaves out paths sync holds (a held file is the hold's to explain), and returns the count, samples and thegbrain repair ghost-pages --source <id>preview as its fix. Sync never deletes or tombstones a ghost;commands/sync/perform.tsprints one line and setsSyncResult.ghostPages. Test:test/sync-exits-l4-6712.test.ts. GBRA-133 2b.2: the count is answered from the last scan (op_checkpoints opsync-ghost-report, fingerprint the source id); an atomic claim on that row starts the next scan in the background at most once perGBRAIN_SYNC_GHOST_INTERVAL_MS(default 300000) per source, and only after a sync landed since the last claim (sources.last_sync_at), so a cancelled sync writes nothing. The sync never waits for the scan; a CLI exit waits within the background-work drain budget and an engine disconnect aborts it.syncGhostDecisionsrecords each decision for tests.src/core/persistence/soak-counters.ts— #6712 PR 2: the soak counters doctorsync_soak_countersreports (oneop_checkpointsrow per UTC day under opsync-soak-counters). Interventions:sync_non_plain(nonPlainSyncFlags, fromsync/run.ts),repair_apply(repair.ts),unblock_apply(sync/operator.ts),retry_held(sources-retry-held.ts) andneeds_human(sweepNeedsHumanHolds, run after each managed sync: stampsneeds_human_aton a hold the fault table now callsneeds_humanand counts it once; the stamp survives same-code re-holds). Content decisions:file_drift_applyandfile_drift_hold.readSoakCounterssums the last seven days. Counting never fails the command that counts. Test:test/sync-exits-l4-6712.test.ts.src/commands/doctor/checks/managed-sync-movement.ts+src/core/persistence/sync-movement.ts— #6317 (B4, I2): the movement reader and its doctor check.readManagedSyncMovementsnapshots, per active managed source, the newestcompleted_atof committedmanaged_sync_*receipts in the current source incarnation (the watermark: one probe of the partial indexpersistence_requests_sync_watermark, v223; a count would shrink under compaction), the head claim'sstep_since, themanaged-synccursor (index/total,held) andsources.last_sync_at/last_commit, and whether a drain or full consumer is live on the owner host.data_movingis false when an unfinished cursor has had neither a committed receipt nor a head step advance for longer thanpersistence.preparation_ceiling_mswhile a consumer is live; with none live the state isparked(informational).sources status --jsonandwriter status --jsonprintdata_moving,not_moving_sinceandmovement_statebesidesync_running, andsources statusprints the writer-status command besidedata_moving: false; doctormanaged_sync_not_moving(warn, scored fornot_movingonly) uses the same rule and command as its fix, and fails on a parked cursor whose admitted queue has no consumer past the ceiling (#6552; fix: the resume sync);serveprints one[gbrain notice]when a source flips. A failed backlog read isunknown, nevernothing_pending(doctor warns)./healthstays liveness-only.src/commands/sources-writer-movement.ts— GST-2688: a leaf that only re-exportsMOVEMENT_SUPERVISOR_STEPfrommovement-step.ts. Updaters from about v0.60.150 to v0.60.248 import this path into the old process after the swap, where a module needing newer exports from the old cache throws, so it must import nothing else (test/upgrade-old-updater-leaf.test.ts).src/commands/sources-writer-movement-cli.ts— #6317 (I2):gbrain sources writer movement [<source>] [--wait <dur>] [--warn-only] [--json], the deploy gategbrain upgradeandpost-upgradeend by printing (bare, so it inherits the default window; neither runs it, because neither restarts the resident processes). The printed step isMOVEMENT_SUPERVISOR_STEPfrom the import-free leafsrc/commands/movement-step.ts(re-exported here and by the leaf above), so upgrade loads it before the swap (#6604). Same source selection aswriter status(persistence-admin.ts). It snapshots throughsync-movement.ts, waitsmax(300 s, preparation budget + 60 s)(printed as the effective budget;--wait <dur>takes sync's duration syntax) and judges each source:moved(a new committed publication; a cursor advanced only by holds or alast_sync_attouched by an up-to-date check is not movement) andcurrentexit 0;heldexits 0 with a warning and the hold kind's route (gbrain repair fences --source <id>;preparation_stalled→ writer status, thengbrain sources retry-held <id>);within_allowance(the head's step advanced inside the window) exits 0 withretry_after_ms;not_movingexits 1 withmanaged_sync_not_moving(reason: movement_check,fix.argvthe writer-status command,fix.verifythis command);unknown(backlog unreadable) exits 1 withmovement_unknown; other read failures keep their code.--warn-onlyprints the same envelopes and exits 0.parseMovementArgsrefuses--jsonor--warn-onlywritten with a value (invalid_params), except--json=true, which it reads as--json(#7619). Flags are regenerated intosrc/core/cli-flag-registry.generated.tswithbun run build:flag-registry.src/core/serve-head-watcher.ts+src/core/persistence/sync-freshness.ts+src/commands/doctor/checks/managed-sync-latency.ts— #6858 ask 6: on-commit sync and its latency metric.startServeHeadWatcher(two call sites:installResidentTickersinsrc/commands/serve.ts, stopped beforeshutdownDelegatedSync, andrunServeHttpinsrc/commands/serve-http.ts, besidestartMovementWatchand stopped on the same shutdown path) re-readssync.on_commitevery 30 s; while on it pollsgit rev-parse HEADin each managed source checkout this host owns (GBRAIN_SERVE_HEAD_POLL_MS, default 3 s; anfs.watchon the git dir only nudges the poll), debounces 3 s (GBRAIN_SERVE_HEAD_DEBOUNCE_MS) and starts one managed sync through the unchangedstartDelegatedSyncwith this host'scliregistration. It never starts beside a running delegated sync or a held drain lease (the move stays pending and fires once after), skips a HEADsources.last_commitalready has, resumes a resumable stop up toMAX_RESUMES, and logs each distinct error once.readSyncFreshnesssamples committedmanaged_sync_importreceipts of the last 24 h (not first syncs or working-tree entries): commit time is the git committer time ofintent.target, searchable iscompleted_at, embedded is the page's newestcontent_chunks.embedded_atonce every chunk has one, sampled only from a receipt no later committed write to the same page supersedes and only at or after itscompleted_at; p50/p95 in one aggregate over thepersistence_requests_sync_watermarkpredicate. #7604: the supersede check reads the newest committed sequence per (source, slug) once, over the rows after the oldest sample (a materialized CTE), so the read stays linear in the samples after a killed pass leaves tens of thousands of receipts (a per-receipt subquery is quadratic and reaches the 300 s statement timeout).gbrain sync status(thefreshnessblock) andgbrain statusprint it; doctormanaged_sync_latencygrades p95 againstsync.freshness_target_seconds(default 300; fail over max(1 h, target)) only withsync.on_commiton or a target set. Tests:test/serve-head-watcher.test.ts,test/serve-head-watcher-managed.serial.test.ts,test/serve-http-head-watcher.serial.test.ts,test/sync-freshness-6858.test.ts. #7189: eachSourceFreshnesscarriesbehind(readGitBehind, injectableBehindReader:rev-list --count <last_commit>..HEADand the oldest of those commits' committer age; null without a checkout or a synced commit), printed byformatSyncFreshness.src/core/persistence/sync-status.ts(status coherence) — GBRA-150:state: blockedmeans the source cannot advance (the drain outcome's meaning); a file hold that waits on a person while the source keeps syncing readssyncedwith reasonheld_needs_human,needs_humanandnextunchanged (test/sync-status-blocked-definition.test.ts);detailaddslanding/landing_read(landing-gaps.ts). #7189: alast_errorwhose cursor failure row (op_checkpointsmanaged-sync-failure) is older than the source's newest committed managed-sync receipt carriessupersededand sets neitherneeds_human,statenornext(liveError);needs_human_fromnames the hold or the error. #7243 (GBRA-133 PR 3a): every status carrieslast_commit(sources.last_commit) andlive(the drain-lease holder still renewing it, fromreadDrainLease, read only when the source read'sgbrain_cycle_locksprobe finds a row, withjob_idfor a Minions sync job andstalledfromholderStalledon the lease read's database clock (db_now); #7596: a holder that stopped renewing the lease (readDrainLease,drainLeaseLive) is notlive, and an open pass with no live holder readspass_interrupted(passStatusState) with aninterruptedblock (pass, last row write, the dead holder), whichsync-blocked.tsalso lists for doctormanaged_sync_blocked; testtest/sync-pass-interrupted-7596.test.ts;live_job_idalso takes a job holder of a pass);supersedingAttemptmarks alast_errorsuperseded(by,age_seconds) when evidence newer than the failure row passed its phase (resume<discovery<freeze/admission<receipt/checkpoint; evidence: the pass row'sstarted_at(pin), a level read (timings_msset or anup_to_dateidle tick), an uncommitted managed import or delete admitted later, a later commit). The CLI (sync/operator.tsholdsListing) bounds the holds:holds_summary(topCountsby code, by state:needs_human,actionable,held, and byfault_code, thewrite_failure.codeapage_write_failedhold names) over every hold, thenPLAIN_STATUS_HOLDS(20) rows needs-human first, or with--detaila page of the shared listing contract (bounded-listing.ts, kindsync_status_holds:<source>:<state>:<filter>,--limit/--cursor/--all,--hold-state),holds_pageonly when a page left rows out or took a cursor, the rows last. GST-1209 follow-up:--slug <slug>(slug or path) and--code <code>(hold code orfault_code) filter the rows before paging and echo asholds_filter; the summary stays over every hold. Tests:test/sync-status-truthful-7243.test.ts(both engines),test/segmented-untiled-hold-gst1209.test.ts. #7679:worktree_refreshis the active refresh of the source's worktree (read in the source statement, so no extra statement) with its holder,liveness/orphanedand fix; an orphaned one's--resumefix becomesnextunless a person's decision comes first. GBRA-287: alast_errorfrom a run that stopped before any write request was admitted (discovery, freeze, admission) carriesrequest_id: nullexplicitly, since no receipt, root request orwrite_error_detailexists;holdVerdictclassifies apage_write_failedfreeze hold whosewrite_failure.codeis a page-body code (PAGE_BODY_FAULT_CODES) by itsfreeze_page_body_*row.src/core/persistence/resolve-paths.ts— GBRA-150resolveSourcePaths: what gbrain did with each file path of a source, by its own rules, so a landing check stops copying them. Per path: the sync slug (resolveSlugForPath), presence in the last synced commit and at HEAD (Git object reads), the skip reason (unsyncableReason, plustyped_frontmatterfor a skipped metafile), the open hold, the live page by origin (source_path) and by slug (slug_ownerwhen another file owns it, so a case-folded twin never reads as this file's page), a differing frontmatterslug:, chunk and embedding counts (listEmbeddingStateRowswith itsslugsfilter), and one ofRESOLVE_PATH_STATESwithsearchable(true / 'keyword' / false),whyand an existing-commandfix. Bounded: at most 200 paths, indexed lookups only, nothing written. A source whoselocal_pathis relative refusessync_anchor_relative(exit 3, anask_usergbrain sources set-pathfix). Tests:test/sources-resolve-path.test.ts.src/core/page-searchability.ts— #7190readPageSearchability: each live page's state,searchable: true | 'keyword' | false, chunks,unembedded(visible chunks with a NULL active-column vector) andembedding_hold, folded fromlistEmbeddingStateRowsand the budget-hold read. Shared by resolve-path andget_write_request'sapplied;gbrain pages status(#7181) can compose it with its Git view.src/core/persistence/applied-watermark.ts— #7190readAppliedWatermark(sources status, MCPsources_status,sync --json) and the cheapreadWriteWatermark(get_write_request):indexed_throughissources.last_commit(only a finished pass moves it),embedded_throughreads the oldest page with a NULL vector (held pages count),committed_through_requestis one below the oldest unsettled request of the incarnation,pendingadds requests, files behind HEAD, held files and withheld deletes,writer_heartbeatis the drain-lease holder and its last progress (redacted for remote callers).src/core/persistence/write-applied.ts— #7190readWriteApplied: the liveappliedblockget_write_requestreturns withapplied: true(searchable_yet, state, gap, sequence, write watermark). Write replies stay frozen so same-request-id replays return them unchanged.src/core/persistence/landing-gaps.ts— GBRA-150readLandingGaps: committed managed_sync receipts of the last 24 h (at most 2,000, newest first through the watermark index, 2 s statement probe; a timeout readsunknown, never ok) whose page is still missing, has chunks waiting for vectors, has no chunks, or is quarantined pastsync.freshness_target_seconds; one row per page with reason and fix. Consumers:sync status --detaillanding/landing_read(sync-status.ts) and doctorsync_landing_gaps. Tests:test/sync-landing-gaps.test.ts.src/commands/doctor/checks/landing-gaps.ts— GBRA-150 doctorsync_landing_gaps(registered afterpartial_embedding, one ofCHEAP_INCIDENT_CHECKS): warns per source with the gap count by reason and the first fix fromreadLandingGaps;unknownread stays a warning, never ok. Tests:test/doctor-sync-landing-gaps.test.ts.