Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
47.6 KB
Raw Blame

Key files: Managed sync bookkeeping

Subsystem index. Read only the entries relevant to your change. Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.

What a managed sync records about itself besides pages: the durable failure row and its local-ledger mirror, the orphan classifier and repair, the doctor check over them, gbrain sync status's failure view, the write diagnostic of a blocked run, and the supersession of a checkpoint whose cursor another run replaced. The cursor, manifest and drain themselves are in Core Persistence and Managed sync and persistence stalls; sync-status.ts and sync-fault-class.ts stay in Core Services (2/3).

  • src/core/persistence/sync-failures.ts — durable managed cursor/receipt diagnostics are the authority; JSONL is a compatibility mirror keyed by cursor and source/path. Failure observations deduplicate terminal receipts, retaining structured codes and attempts; success clears only its matching cursor. A failure records the cursor-selecting syncOptions and processingOptions of its run (falling back to its cursor's when read), and every local hint (the thrown failure line, the blocked report, the terminal write diagnostic and local doctor) renders the exact retry through managedSyncRetryCommand → checkpointRetryCommand; a failure recorded before options were kept says so instead of guessing. sync --retry-failed counts only failures of the cursor key its own options select (managedSyncCursorKey) and lists the others with the action that settles each (managed-retry-actions.ts). #6712 PR 3 (L7): --retry-failed is a deprecated alias of the plain sync, which retries a recorded failure by itself; a failure records the gbrain_version that recorded it, so an unchanged content refusal the same version recorded replays without a new request until the file or the version changes. Local CLI shows request/run/pinned target details, while remote doctor returns source-scoped aggregates without paths, raw messages or receipt IDs. Missing durable state warns rather than certifying health. Failed receipts cannot be acknowledged, auto-skipped or rewritten; explicit idle ordinary-source retry creates fresh admission, while approved company manifests remain pinned. Tests: test/persistence-sync-failures.serial.test.ts, test/e2e/managed-sync-failures.test.ts. #6402: an observation moves updated_at; a new target commit restarts first_seen/attempts; an invalid_params resume refusal is not recorded. #6288: after a managed sync succeeds, clearSettledManagedSyncFailures removes the source's rows under other cursor keys (a --full run's) whose cursor is done or gone and whose file the imported commit settles; gbrain sync --source ID --acknowledge-managed REQUEST_ID_OR_PATH (host CLI) removes only such rows, refusing others managed_failure_unsettled (sync-managed-stale-failures.test.ts). Wave 16 (#6513 comments): a failure recorded from a receipt carries write_error_detail, the receipt's publicFailureDetail projection (origin, sqlstate, raiser, table, the closed reason; never message text), readManagedSyncFailures re-reads it from the pending receipt's error_detail, and formatManagedSyncFailure appends raiser= table= reason= (publicationDetailText, identifiers only), so the ledger mirror, doctor and the blocked report name the refusing trigger; sync status --json surfaces it as last_error.write_error_detail with last_error.detail = the reason (sync-status.ts).

  • src/core/persistence/managed-sync-orphans.ts + src/core/repair/managed-sync-orphans.ts — gbrain repair managed-sync-orphans --source <id> (#5459), explicit-only and preview-bound (approved set, preview_changed on drift). classifyManagedSyncOrphans (read-only; doctor sync_failures names the repair when it finds any) takes the source's managed-sync bookkeeping and classes each row no_cursor (a failure row with no cursor to resume), principal_unadoptable (an unfinished cursor recorded by a principal that is no longer an active local writer, by join on the recorded principal), superseded (a pre-options cursor a later run replaced), incarnation_retired (#6518: an unfinished cursor or a failure row recorded under a source incarnation the source no longer has, joined on the source id alone, archived sources excluded, a live pending request still retaining it) or mirror_only (#6518: a local ledger row with a managed_cursor_key and no durable failure row under that key; mirrorOnlyManagedFailures, which throws on a failed durable read rather than naming every mirror), and keeps pending_request_live and resumable rows untouched. retireManagedSyncOrphan retires one row atomically: an incarnation_retired cursor needs the source still active under another incarnation at apply time and skips the live-writer veto (no retired key is resumable); a mirror_row deletes nothing durable and prunes the ledger row only when the durable read, repeated at apply, still finds none. pruneMirrorOnlyManagedFailures is the successful-sync path (clearSettledManagedSyncFailures). Doctor sync_failures classifies whenever the brain is reachable, so the diagnostic is reachable with no current-incarnation failure visible, and reports a failed classification as unknown. sources set-strategy points at it when an unfinished run blocks a strategy change. Pinned by test/repair-managed-sync-orphans.test.ts and test/managed-sync-orphans-incarnation-6518.test.ts (both engines); test/repair-explicit-only.test.ts pins its place last in REPAIR_KINDS. #6712 PR 3 (L7): slug_mode_refused is an unfinished cursor whose every page committed and whose checkpoint was refused slug_root_mode_mismatch while the source still pins the other mode (slugModeRefusedSql, bound to the run and request at apply); no sync rediscovers it, so retiring it is how a person keeps the pin, and its slice record is cleared like checkpoint_moved's.

  • src/core/persistence/sync-cursor-key.ts — #6712 PR 1: one managed cursor per source. managedCursorKey is a digest of the source incarnation and, for a company approval only, the company; writer principal, authority and options no longer select a cursor. resolveManagedCursor is the one boundary that picks a run's key and folds older keys in crash-safely under the drain lease: the canonical unfinished cursor wins, else the newest older-key unfinished cursor is adopted and finished in place under its own key; every other older cursor has its queued requests cancelled and is retired through retireManagedSyncOrphan (reason superseded_by_adoption, fenced by assertDrainLease). A cursor with a running, recovering or publishing request or a recovery row is retained and the pass waits for that request (writer_pending) before it drains anything. A company brain's finished old-key cursors are kept. Pinned by test/sync-pass-handover-6712.test.ts (the live pass's handover of an older binary's cursors) and test/persistence-sync-cursor-strategy-6696.test.ts.

  • src/core/persistence/sync-lease.ts — #6712 PR 1: the drain lease, one managed drainer per source. withDrainLease takes the source's sync lock (syncLockId) with a 2-minute TTL renewed every 20 s, so a dead holder's lease lapses inside one cron tick; it is reentrant for the holder (nested passes and the in-process lanes reuse it). The holder record (managed-sync-lease checkpoint: kind, id, version, start) plus the lock row give readDrainHolder (host, pid, heartbeat, progress: commits, last commit, last cursor save). A non-job holder also writes a beacon row in the gbrain-drain-lease minion queue (application authority, max_stalled 0, renewed with the lease, deleted on release) so an older binary's liveSyncJob backs off; downgrades are quiescent (stop serve and crons first). assertDrainLease is the fence inside every transaction that admits work, settles or holds an entry, folds a cursor in or swaps the cursor. Following callers (the CLI, a CLI run delegated to a serve) wait for a live holder within their budget (followStopAt) and take over when it releases; out of budget, or after losing the lease, they return leaseHeldResult (resumable, sync_drain_lease_held, next.retry_after_ms 60 s, safe_to_loop); a holder with no progress for HOLDER_STALL_MS (5 min) returns leaseStalledResult (blocked, drain_stalled, cause no_progress, stall.lease_holder). Jobs and the cycle get SyncLockBusyError and skip. #7596: drainLeaseLive is isLockHolderLive's rule at the drain TTL (readDrainLease returns it with the holder, and liveSyncJob counts only a live holder); a killed holder's lock row outlives it until a takeover, so status reads it as not live. Pinned by test/persistence-sync-lease.test.ts.

  • src/core/persistence/sync-idle-skips.ts — GBRA-133 (PR 2b): the managed no-op fast path. An idle tick (stampIdlePass) records on the drain lease (idleClear) whether its pin read no hold of the incarnation, a zero hold summary, no managed-sync-failure row of the source and no owed tail (readIdleFacts rides the pin's round trip). idleSkips names the settles that then have nothing to do (IDLE_SKIPS: the hold report in performManagedPass; settled failures, which also needs no ledger row of the source; the needs-human sweep, the sync tail and both heals in perform.ts). The tick, its lease, its pass row and its result are unchanged; the upstream observation, the remember replay and the ghost-page report still run. GBRAIN_SYNC_IDLE_FAST_PATH=0 runs every settle. Pinned by test/sync-idle-skips-gbra133.test.ts (both backends; Postgres twin in test/e2e/).

  • src/core/sync-dispatch-read.ts — GBRA-133 (PR 2b): the routing read of a single-source CLI sync. withSyncDispatchRead (src/commands/sync/run.ts, never --all or --watch) answers the source resolver, the company profile, resolveSyncPersistenceMode, the content-directory anchor, the managed-flag refusals, the delegation ladder's mode check and perform.ts's connector check from one combined row per source (syncDispatchRow: the sources row, the claim, the company receipt and the mode rows). endSyncDispatchRead ends the scope for good where the work starts (performSync once the route is chosen, maybeDelegateManagedSyncToServe before the hand-off), so every read under the drain lease and after a write stays live; outside the scope, or when the combined read fails, each reader runs its own query. Pinned by test/sync-dispatch-read-gbra133.test.ts (both backends; Postgres twin in test/e2e/).

  • src/core/wire-skew.ts — #6712 PR 1: the one record of wire-skew windows, each a version floor plus a calendar date (WIRE_SKEW_WINDOWS.explicitProcessing: floor 0.60.255.0, accepted and ignored until 2026-11-14). The delegation and admin wires read the field name from here, so removing the window is one edit. #6712 PR 2: each window names the hand-offs that accept it (wires) and its shape (accepts); retiredWireFields / retiredWireNotice serve both validators. CONFIG_ROLLBACK_WINDOWS records the rollback config keys' windows (honored until a date, from a version floor, with the notice that named them).

  • src/core/sync-flags.ts — #6712 PR 2: SYNC_FLAGS, every gbrain sync flag once, with its route (forward / client / refuse) on the serve and PGLite owner hand-offs; syncFlagsRouted and syncWireFields derive the ladder's, the IPC validator's and the owner lane's allowlists. A flag the argv parsers read must be listed (test/sync-flag-table-6712.test.ts). #6712 PR 3 (L7): managed: true marks the ten flags gbrain sync --help lists under "Managed brains" (MANAGED_HELP_FLAGS, table order; test/sync-managed-help-l7-6712.test.ts pins the section and runs doctor's cron recipe through the managed flag validator).

  • src/core/sync-config-keys.ts + src/commands/doctor/checks/sync-config-keys.ts — #6712 PR 2: the sync.* keys in three kinds: SYNC_TUNABLE_KEYS, the rollback keys (windows in wire-skew.ts) and RETIRED_SYNC_KEYS with their GBRAIN_SYNC_* replacements (retiredSyncSetting reads one). effectiveSyncKey is config get's value-in-effect and origin for a retired or rollback key (its resolvers mirror the readers: resolveBulkSettings, waiverBatchEnabled, isFederatedV2Enabled, the write switches, planJobDrain); syncKeySetNotice is config set's warning (a retired key writes nothing); syncRepoPathRefusal is config set sync.repo_path's refusal of a relative value (nothing written). Doctor sync_config_keys reports a stored retired key as unapplied. Pinned by test/sync-config-keys-6712.test.ts and test/error-registry-retired.test.ts.

  • src/core/persistence/sync-admission.ts + src/commands/config/sync-admission-value.ts + src/commands/doctor/checks/sync-admission.ts — #6712 PR 4b: sync.admission (normal | imports_only | classify_only, brain-wide, a SYNC_TUNABLE_KEYS entry). readSyncAdmission parses it (junk fails closed to classify_only) for the pass's pin, group admission and destructive publication, and admissionAllows gates a write kind. config set refuses any other value (exit 2, --force included) and writeSyncAdmission stores the value and its set_at (database clock) in one transaction: config has no timestamp, so set_at is an op_checkpoints row (op='sync-admission', fingerprint brain) that dates the stored value only while it names that value (otherwise set_at: null). config get prints the value in effect (handleSyncKeyConfig). syncAdmissionStatus(engine, source, incarnation) is sync status's admission: {value, set_at, effective} and the helper the pass's convergence.admission uses: effective is false while an open pass row with started_at before set_at holds the live gbrain-sync:<source> lease (token match), or a managed_sync_delete / canonical_reconcile request of the incarnation admitted before it is queued, running or recovering (an unknown set_at counts everything as earlier). Doctor sync_admission_restricted (ops tier) warns on a managed brain while the value is not normal, with its age and the user's restore command; ok otherwise. Never settable remotely (config refuses on a thin client and no operation writes config). Tests: test/sync-admission-6712.test.ts.

  • src/core/persistence/managed-sync-flags.ts — managedForbiddenFlag / managedSyncFlagRefusal: the managed_sync_flag_refused refusal for --skip-failed / --include-gitignored on a managed source (#6696), raised at gbrain sync parse, in maybeDelegateManagedSyncToServe before the socket, and by resolveManagedSyncContext.

  • src/commands/doctor/checks/sync-failures.ts — reads managed durable cursor/receipt state even when its compatibility ledger is absent. An unavailable read is unknown/warn, not healthy; local findings carry diagnostics and remote findings contain only source-scoped aggregates. Legacy remediation stays separate from explicit managed retry. Failures on archived sources (which sync refuses) are left out of the count and severity and only reported as an ignored total. #6518: with an engine, classifyManagedSyncOrphans runs whenever the brain is reachable (not only when a current failure is counted), so retired-incarnation and mirror-only bookkeeping is named with the repair preview even at zero visible failures, and a failed classification is reported as "could not be classified", never as healthy; without an engine (doctor --fast) only the unmanaged ledger rows enter the severity calculation, and mirrored managed rows are reported apart as warn "could not be verified without a database connection" (uncertainty, never recovery). Tests: test/persistence-sync-failures.serial.test.ts, test/doctor-report-remote.serial.test.ts, test/doctor-sync-failures-archived.test.ts, test/managed-sync-orphans-incarnation-6518.test.ts.

  • src/core/persistence/sync-write-diagnostic.ts — managedSyncWriteDiagnostic, the managedWrite block of a managed sync result (terminal receipt reason, suggestion with the retry command, the CRLF-only note), moved unchanged out of sync-run.ts by #6340; writeWaitOf / ManagedSyncWriteWait (the writeWait block) followed in wave 16.

  • src/core/persistence/sync-checkpoint-replaced.ts — wave 16 (GBRA-79, #6513 comments): a managed-sync checkpoint whose cursor another run of the source replaced is superseded, not failed. checkpointCursorReplaced is the classification the shared validator (sync-prepare.ts sourceSyncChecks consumer) and the checkpoint's final locked compare-and-swap both apply before their own checks: a checkpoint intent whose durable cursor names another runId refuses sync_cursor_replaced (receipt state conflict, through coordinator.ts conflictCode); a page request whose cursor moved keeps revision_conflict (its run re-admits from the current cursor), a same-run index lag keeps revision_conflict, a missing manifest keeps storage_error. checkpointSuperseded recognizes that receipt in sync-run.ts: settleFailedRequest records no managed-sync failure for it and continues with the winning cursor as it stands (currentCursor, the same move the ENG-A7 admission guard makes when the cursor moves under it; the winner's run id, index and manifest are kept, nothing of it is cleared or advanced), and retryFailedStart never treats a checkpoint receipt as a page fault. Before, the loser's run recorded the conflict as a managed-sync-failure row, doctor reported it, and --retry-failed rediscovered from HEAD. Test: test/managed-sync-checkpoint-controls-company.test.ts (#6712 PR 4b: a company brain is the one cursor checkpoint left; controls: index lag, missing manifest).

  • src/core/persistence/sync-slug-mode.ts — #6694: the checkpoint's slug-root mode against the source's pinned slug_root_mode. emptySyncScope (source root is the repository root, both realpathed) lets sync-prepare.ts's checkpoint UPDATE accept either pin and keep the pinned one; at a subdirectory a differing pin is diagnosed by pinnedSlugModeMismatch after the guarded UPDATE matched nothing and refused slug_root_mode_mismatch (cause and the person's two choices from slugModeMismatchCause); anything else stays revision_conflict (CHECKPOINT_MOVED_MESSAGE). slugModeCheckpointRevivable is the second revival predicate of sync-run.ts reviveCheckpoint: a checkpoint refused slug_root_mode_mismatch, or an older release's moved-checkpoint revision_conflict, is revived when last_commit still equals the run's from and the pin is unset, equal to the run's mode or the scope is empty; retryFailedStart does not consult it, so --retry-failed keeps its rediscovery. Test: test/sync-slug-root-mode-6694.test.ts (both engines).

  • src/core/persistence/sync-cursor-moves.ts — #6712 (L2, one page, one fate): the managed-sync cursor's pure transitions, peeled out of sync-run.ts; each returns the next cursor and writes nothing (sync-run.ts saves it with one compare-and-swap). advanceHeld/settledPast pass an entry without a commit (held, waived via waivedCursor, an idempotent deletion via deletionSettled) and add its index to the settled ranges the checkpoint skips; a deferred entry (Cursor.deferred, a source_changed file retried after the manifest end) leaves the deferred list instead of moving the index. promoteWindow makes the window's first group the cursor's group once the cursor reaches it, so requests admitted after a failed or held entry are read by their own receipts, never frozen again; committedPast does the same after a commit. withHeldKeys records a held entry's page keys (dependencyKeys) so a later entry of the run that shares one is held behind it (blocked_on).

  • src/core/persistence/sync-stop-class.ts — #6712 (L2; #6565): how the settle path in sync-run.ts classifies a page request that ended without committing. receiptClass reads the terminal classifier: transport (a dropped connection or a retryable refusal, re-frozen in place), page (held, deferred or blocked under the hold policy) or systemic (the run stops with the classifier's action). resumableStop is a systemic stop the fault table retries (an unrecognised P0001 or a 55000 surfacing as storage_error); stopRecorded says an earlier invocation already stopped this run (its failure ledger row carries the run id), so the next plain sync re-freezes every page that refusal failed, bounded by STOPS_PER_ENTRY. #6712 PR 3 (L7): recordedStop also returns the row's gbrain_version. retryRecordedFailure in sync-run.ts (what --retry-failed, now its deprecated alias, did on request) retries every recorded failure on the next plain sync, except a refusal nothing changed since the same version recorded it (replaysUnchangedRefusal: a content refusal while the file has its frozen bytes; a source_changed pinned import while HEAD is the run's target and the file still holds the working-tree bytes that refused, compared by hash rather than by mtime), a slug-mode refusal and a checkpoint whose source's last_commit moved (#6834), which wait for a changed file or gbrain repair managed-sync-orphans. GST-2869: SyncTerminated is the abort reason gbrain sync gives its run on SIGTERM or SIGHUP; abortStop reads the first abort's reason, so the pass row ends terminated instead of timeout (from its catch, its rolled-back pin or a batch that returned timeout) and the drain stops terminated; managedPassLive (sync-pass-episode.ts, the pass-cost registry) says a managed pass is running in this process, the only time sync owns those signals (withTerminationOwner in src/commands/sync/run.ts, CLI entry only, never under another owner).

  • src/core/persistence/sync-knock-on.ts — #6643, shrunk by #6712 (one page, one fate): no producer is left. It keeps the legacy predicate over receipts older builds wrote (error_detail { origin: 'sync_knock_on', reason, root_request_id }, or the pre-#6643 producers' exact messages, never an operator cancel): isSyncKnockOn/syncKnockOnSql (one predicate, never NULL) feed the terminal classifier, which renders such a receipt superseded, and sync-run.ts refreezeInPlace, which re-freezes the entry when a resumed cursor meets one (bounded by STOPS_PER_ENTRY, no config key). cursorFenceCause reads the two live causes of "cursor changed" (run_replaced, outside_fence; an older window_dropped still parses). checkpoint-validation.ts clause 3 skips converted receipts and the cursor's settled index ranges only; #6712 removed the knock-on exemption. reviveCheckpoint re-assesses a refused checkpoint at the cursor's end. sync status shows the cursor the recorded failure belongs to, cursors, and checkpoint_blockers (listIncompleteSyncReceipts, each with knock_on).

  • src/core/persistence/terminal-classify.ts — #6650: classifyTerminal turns one terminal receipt (failed / conflict / cancelled, or any row still holding recovery) into a TerminalVerdict (category, code, origin, class, safe_actions, needs_human, defect, historical, root_request_id, one structured action) at read time; nothing is persisted. A recovery record outranks the code (recovery_pending / recovery_blocked); a knock-on is read through isSyncKnockOn (typed marker survives compaction) or the SQL twin terminalKnockOnSql; storage_error and source_changed route by their recorded origin or cause, and a row whose code and origin do not say what happened is diagnosis_required (read-only writer status, never a guessed fix). A source_changed whose canonical file is missing is canonical_file_missing (a ghost page; its exit is the gbrain repair ghost-pages preview, never a sync retry), and an origin-less fence_unrenderable adoption refusal is its own defect category. A retryable sync fault's action follows its operation (faultOperation: a put_page under a canonical_reconcile or managed_* intent kind reads as that kind): submit_job and a canonical reconcile keep the sync retry, a caller-submitted page or memory write (RESUBMITTED_WRITES) gets the read-only writer-status inspect and a resubmit with a new request_id, a managed_* write the inspect and a rerun of its command, any other operation (and a compacted put_page, whose intent kind is gone) the inspect alone with safe_actions: ['none']. TERMINAL_FIXED_IN names the release that fixed a category; a row whose build (error_detail.attempt.consumer_version, else admitter_version) is older is historical. legacy_publication_refusal and retired_operation (a revision_conflict with origin grandfather_retired or a managed_grandfather intent, #7837) are historical whatever build wrote them, since only a request an older build admitted produces them. It also owns the supersession rule repair/failed-writes.ts reuses (supersessionSql, supersessionOrderSql, dispositionOf, PAGE_REMOVAL_OPERATIONS). Tests: test/terminal-failures-6650.test.ts (TS↔SQL twin parity included), test/terminal-classify-6696.test.ts.

  • src/core/persistence/terminal-failures.ts — #6650: the bounded readers over that classifier. readTerminalFailureSummary is one set-based aggregate (GROUPING SETS per source; TERMINAL_STATEMENT_MS 5 s statement and TERMINAL_ACQUIRE_MS 2 s connection deadlines; past either it returns status: 'unknown', never an empty pile) splitting active blockers (recovery records plus the passed-in checkpoint_blockers of sync-status.ts, read by readCheckpointBlockerMap), unresolved rows (count, distinct slugs and intents, oldest age), historical rows (a later write superseded them, or older than TERMINAL_FIXED_IN via the SQL twin TERMINAL_HISTORICAL_SQL), 24 h and 7-day daily counts, categories and defect classes; archived sources are skipped. Thresholds persistence.terminal_failures.window_days (clamped to persistence.receipt_retention_days), spike_factor, spike_min (readTerminalThresholds, validateTerminalConfigValue, refused at config set by commands/config/preparation-keys.ts); isSpike against sevenDayMedian. readTerminalFailures is the drill-down (batches of 200, each selected before the supersession lateral join, which with the summary's probe tests the failed row's intent kind first so later large sync intents are not detoasted per row (#8729), at most 2,000 rows scanned per call, a sequence cursor, verdict filters, --group-by category groups, a sync_fault group also split by its operation, with an exact drill_down argv each; --operation filters on faultOperation). Consumers: doctor persistence_terminal_failures (commands/doctor/checks/persistence-requests.ts), sync status terminal_failures (sync-status.ts, next falls back to a recovery blocker's action), the writer_failures administration operation, and the admin /metrics gauges (serve-http-metrics.ts renderTerminalFailureMetrics, one aggregate per engine shared by scrapes for TERMINAL_METRICS_TTL_MS = 60 s via cachedTerminalSummary; never /health). Each source also reports new_last_24h_older_builds (rows and builds older than their class's fix), and doctor's spike message says an older write owner is producing them when they are most of the spike. Tests: test/terminal-failures-6650.test.ts, test/sync-status-terminal-6650.test.ts, test/terminal-failure-metrics-6650.test.ts, the CLI journey test/agent-journey-terminal-failures-6650.test.ts.

  • src/commands/sources-writer-failures.ts — #6650: output of gbrain sources writer failures <source> [--state all] [--group-by category] [--since Nd] [--limit N] [--before <cursor>] [--category|--code|--origin|--operation] [--json], dispatched by persistence-admin.ts as writer_failures (validated in administration.ts writerFailuresQuery, locally or through the resident owner). A table by default, one JSON document with --json; row and group fix / inspect actions are rendered for the CLI; a full page prints truncated with the exact next command. The operation is advertised to clients in the owner's administration_extensions (ipc.ts, admin-contract.ts PERSISTENCE_ADMIN_EXTENSIONS), never in the administration list an older client validates strictly, and local-client.ts delegates it only to an owner that lists it (ownerAdministers). Tests: test/writer-failures-cli-6650.test.ts.

  • src/core/persistence/public-category.ts — #6712 PR 2 (L5): publicCategory({ category?, code }), the closed public vocabulary (drift, conflict, content_refused, file_missing, permission, storage, config, superseded, transient, defect) that sits beside the unchanged diagnosis category. PUBLIC_BY_TERMINAL maps every TerminalCategory, PUBLIC_BY_CODE every SYNC_FAULT_TABLE code (a new fault code needs a row: test/public-category.test.ts), and anything else goes by its registry class (unknown codes are defect). classifyTerminal carries it as public_category; classifyRecordedFault (terminal-classify.ts) reads a hold, a managed-sync failure row or a run stop as the failed receipt it stands for, so the receipt diagnosis, managedWrite, sync status holds and last_error, the terminal summary and doctor sync_failures (details.failures) render one verdict.

  • Pass round phase lines and honest stalls (GBRA-133 PR 3a, #7243; src/core/persistence/sync-pass-run.ts, sync-pass-status.ts, sync-drain.ts) — each pass round emits managed_sync.round_level (paths, target, read time) after its level read and managed_sync.round_admit (imports, deletes, in flight) before its import walk, and a managed_sync.round_batch (batch of batches, starting at candidate base, #7866) before each import batch; an announcing drain prints the level and admit lines ([sync] pass round N: …). The convergence block's target is the newest pass's pinned commit (telemetry) and converged_target the stored converged obligation's target, null while no stamp backs one, so status never presents the pin as converged. The drain counts only page and checkpoint commits as progress (a legacy-sweep slice or a level read still working only keeps the quiet clock fresh); an abandoned pass's late progress is dropped; a stop whose head owner reports restart_required is owner_restart_required (drainNext names the process); a run that committed nothing (no advanced cursor, no committed page) stops as sync_deadline_stalled (exit 1). Tests: test/sync-pass-phase-lines-7243.serial.test.ts, test/sync-status-truthful-7243.test.ts, test/honest-stalls-gbra133.test.ts.

  • src/core/persistence/sync-tail.ts + sync-tail-record.ts — #6712 PR 2 (L5): the tail of a committed checkpoint (projection statistics, then link extraction for the run's imported slugs read from its manifest). recordSyncTail runs inside the checkpoint's publication transaction (applyManagedSyncCheckpoint) and appends {run_id, changed, extract} to op_checkpoints managed-sync-tail / <source> (at most 16), so the tail is owed before anyone can read the run as synced. withSyncTail runs it after the outermost drain lease is released (performManagedBranch, and performManagedSync when it took the lease itself; a call still inside a lease, a dry run and a lease-held follower run none), bounded by SYNC_TAIL_BUDGET_MS (5 min) and the caller's deadline, and folds tail (done / truncated / degraded / deferred / failed, with underived: the run's imports its link extraction left underived, which the per-source "still owe extraction" line prints) and links into the result. The obligation is cleared by compare-and-swap only when every step ran; a failed, deferred or truncated tail attaches a degraded sync_tail_degraded notice with its fix (the sync CLI prints it with the capacity notices). Fault point sync:tail_started. Tests: test/sync-drain-finish-tail.test.ts.

  • src/core/repair/slug-autoheal.ts — the sync-time heal of stray slug: lines (#6865): healAfterManagedSync, called once from commands/sync/perform.ts performManagedBranch after the drain lease is released, runs slug-conflicts' healAbsentSlugLine (deterministic absent_page only, bound to the held bytes, nothing recorded for any other analysis) over the source's slug-conflict holds through the same submitManagedFileRepair write, with the receipt's declared_alias recorded as a slug_aliases row in that write. Skipped (and said why on slug_lines_removed) under sync.slug_conflicts.auto_remove false, fences.repair.enabled false, remote submission authority, an unfinished managed-sync cursor, or a host that may not write the checkout; bounded by SLUG_HEAL_MAX_FILES and SLUG_HEAL_BUDGET_MS (or the caller deadline), one checkout scan per source; never throws. Lane verdicts carry lane_version (CONTENT_REPAIR_LANE_VERSION in persistence/sync-fault-class.ts): an older verdict is repair in contentHoldDisposition and never a person's decision in classifySyncFault. Tests: test/slug-autoheal-6865.test.ts (Postgres arm test/e2e/slug-autoheal-6865-postgres.test.ts), test/slug-conflict-verdict-6865.test.ts.

  • src/core/since.ts — #6712 PR 2: parseSince(input, flag, now), the one --since parser (gbrain agent logs --since, gbrain repair file-drift --since). An age (90s, 30m, 12h, 7d) becomes one concrete UTC instant at parse time; an ISO-8601 date or timestamp is normalized; anything else throws invalid_params naming the flag. Callers print and hash the instant, never the age. Tests: test/sync-exits-l4-6712.test.ts, test/agent-cli.test.ts.

  • src/core/persistence/sync-ghosts.ts — #6712 PR 2: countSyncGhosts(engine, sourceId), the post-sync ghost count. It reuses scanGhostSource from repair/ghost-pages.ts (no second classifier) with a candidate bound of SYNC_GHOST_CHECK_LIMIT (50) stats, leaves out paths sync holds (a held file is the hold's to explain), and returns the count, samples and the gbrain repair ghost-pages --source <id> preview as its fix. Sync never deletes or tombstones a ghost; commands/sync/perform.ts prints one line and sets SyncResult.ghostPages. Test: test/sync-exits-l4-6712.test.ts. GBRA-133 2b.2: the count is answered from the last scan (op_checkpoints op sync-ghost-report, fingerprint the source id); an atomic claim on that row starts the next scan in the background at most once per GBRAIN_SYNC_GHOST_INTERVAL_MS (default 300000) per source, and only after a sync landed since the last claim (sources.last_sync_at), so a cancelled sync writes nothing. The sync never waits for the scan; a CLI exit waits within the background-work drain budget and an engine disconnect aborts it. syncGhostDecisions records each decision for tests.

  • src/core/persistence/soak-counters.ts — #6712 PR 2: the soak counters doctor sync_soak_counters reports (one op_checkpoints row per UTC day under op sync-soak-counters). Interventions: sync_non_plain (nonPlainSyncFlags, from sync/run.ts), repair_apply (repair.ts), unblock_apply (sync/operator.ts), retry_held (sources-retry-held.ts) and needs_human (sweepNeedsHumanHolds, run after each managed sync: stamps needs_human_at on a hold the fault table now calls needs_human and counts it once; the stamp survives same-code re-holds). Content decisions: file_drift_apply and file_drift_hold. readSoakCounters sums the last seven days. Counting never fails the command that counts. Test: test/sync-exits-l4-6712.test.ts.

  • src/commands/doctor/checks/managed-sync-movement.ts + src/core/persistence/sync-movement.ts — #6317 (B4, I2): the movement reader and its doctor check. readManagedSyncMovement snapshots, per active managed source, the newest completed_at of committed managed_sync_* receipts in the current source incarnation (the watermark: one probe of the partial index persistence_requests_sync_watermark, v223; a count would shrink under compaction), the head claim's step_since, the managed-sync cursor (index/total, held) and sources.last_sync_at/last_commit, and whether a drain or full consumer is live on the owner host. data_moving is false when an unfinished cursor has had neither a committed receipt nor a head step advance for longer than persistence.preparation_ceiling_ms while a consumer is live; with none live the state is parked (informational). sources status --json and writer status --json print data_moving, not_moving_since and movement_state beside sync_running, and sources status prints the writer-status command beside data_moving: false; doctor managed_sync_not_moving (warn, scored for not_moving only) uses the same rule and command as its fix, and fails on a parked cursor whose admitted queue has no consumer past the ceiling (#6552; fix: the resume sync); serve prints one [gbrain notice] when a source flips. A failed backlog read is unknown, never nothing_pending (doctor warns). /health stays liveness-only.

  • src/commands/sources-writer-movement.ts — GST-2688: a leaf that only re-exports MOVEMENT_SUPERVISOR_STEP from movement-step.ts. Updaters from about v0.60.150 to v0.60.248 import this path into the old process after the swap, where a module needing newer exports from the old cache throws, so it must import nothing else (test/upgrade-old-updater-leaf.test.ts).

  • src/commands/sources-writer-movement-cli.ts — #6317 (I2): gbrain sources writer movement [<source>] [--wait <dur>] [--warn-only] [--json], the deploy gate gbrain upgrade and post-upgrade end by printing (bare, so it inherits the default window; neither runs it, because neither restarts the resident processes). The printed step is MOVEMENT_SUPERVISOR_STEP from the import-free leaf src/commands/movement-step.ts (re-exported here and by the leaf above), so upgrade loads it before the swap (#6604). Same source selection as writer status (persistence-admin.ts). It snapshots through sync-movement.ts, waits max(300 s, preparation budget + 60 s) (printed as the effective budget; --wait <dur> takes sync's duration syntax) and judges each source: moved (a new committed publication; a cursor advanced only by holds or a last_sync_at touched by an up-to-date check is not movement) and current exit 0; held exits 0 with a warning and the hold kind's route (gbrain repair fences --source <id>; preparation_stalled → writer status, then gbrain sources retry-held <id>); within_allowance (the head's step advanced inside the window) exits 0 with retry_after_ms; not_moving exits 1 with managed_sync_not_moving (reason: movement_check, fix.argv the writer-status command, fix.verify this command); unknown (backlog unreadable) exits 1 with movement_unknown; other read failures keep their code. --warn-only prints the same envelopes and exits 0. parseMovementArgs refuses --json or --warn-only written with a value (invalid_params), except --json=true, which it reads as --json (#7619). Flags are regenerated into src/core/cli-flag-registry.generated.ts with bun run build:flag-registry.

  • src/core/serve-head-watcher.ts + src/core/persistence/sync-freshness.ts + src/commands/doctor/checks/managed-sync-latency.ts — #6858 ask 6: on-commit sync and its latency metric. startServeHeadWatcher (two call sites: installResidentTickers in src/commands/serve.ts, stopped before shutdownDelegatedSync, and runServeHttp in src/commands/serve-http.ts, beside startMovementWatch and stopped on the same shutdown path) re-reads sync.on_commit every 30 s; while on it polls git rev-parse HEAD in each managed source checkout this host owns (GBRAIN_SERVE_HEAD_POLL_MS, default 3 s; an fs.watch on the git dir only nudges the poll), debounces 3 s (GBRAIN_SERVE_HEAD_DEBOUNCE_MS) and starts one managed sync through the unchanged startDelegatedSync with this host's cli registration. It never starts beside a running delegated sync or a held drain lease (the move stays pending and fires once after), skips a HEAD sources.last_commit already has, resumes a resumable stop up to MAX_RESUMES, and logs each distinct error once. readSyncFreshness samples committed managed_sync_import receipts of the last 24 h (not first syncs or working-tree entries): commit time is the git committer time of intent.target, searchable is completed_at, embedded is the page's newest content_chunks.embedded_at once every chunk has one, sampled only from a receipt no later committed write to the same page supersedes and only at or after its completed_at; p50/p95 in one aggregate over the persistence_requests_sync_watermark predicate. #7604: the supersede check reads the newest committed sequence per (source, slug) once, over the rows after the oldest sample (a materialized CTE), so the read stays linear in the samples after a killed pass leaves tens of thousands of receipts (a per-receipt subquery is quadratic and reaches the 300 s statement timeout). gbrain sync status (the freshness block) and gbrain status print it; doctor managed_sync_latency grades p95 against sync.freshness_target_seconds (default 300; fail over max(1 h, target)) only with sync.on_commit on or a target set. Tests: test/serve-head-watcher.test.ts, test/serve-head-watcher-managed.serial.test.ts, test/serve-http-head-watcher.serial.test.ts, test/sync-freshness-6858.test.ts. #7189: each SourceFreshness carries behind (readGitBehind, injectable BehindReader: rev-list --count <last_commit>..HEAD and the oldest of those commits' committer age; null without a checkout or a synced commit), printed by formatSyncFreshness.

  • src/core/persistence/sync-status.ts (status coherence) — GBRA-150: state: blocked means the source cannot advance (the drain outcome's meaning); a file hold that waits on a person while the source keeps syncing reads synced with reason held_needs_human, needs_human and next unchanged (test/sync-status-blocked-definition.test.ts); detail adds landing / landing_read (landing-gaps.ts). #7189: a last_error whose cursor failure row (op_checkpoints managed-sync-failure) is older than the source's newest committed managed-sync receipt carries superseded and sets neither needs_human, state nor next (liveError); needs_human_from names the hold or the error. #7243 (GBRA-133 PR 3a): every status carries last_commit (sources.last_commit) and live (the drain-lease holder still renewing it, from readDrainLease, read only when the source read's gbrain_cycle_locks probe finds a row, with job_id for a Minions sync job and stalled from holderStalled on the lease read's database clock (db_now); #7596: a holder that stopped renewing the lease (readDrainLease, drainLeaseLive) is not live, and an open pass with no live holder reads pass_interrupted (passStatusState) with an interrupted block (pass, last row write, the dead holder), which sync-blocked.ts also lists for doctor managed_sync_blocked; test test/sync-pass-interrupted-7596.test.ts; live_job_id also takes a job holder of a pass); supersedingAttempt marks a last_error superseded (by, age_seconds) when evidence newer than the failure row passed its phase (resume < discovery < freeze/admission < receipt/checkpoint; evidence: the pass row's started_at (pin), a level read (timings_ms set or an up_to_date idle tick), an uncommitted managed import or delete admitted later, a later commit). The CLI (sync/operator.ts holdsListing) bounds the holds: holds_summary (topCounts by code, by state: needs_human, actionable, held, and by fault_code, the write_failure.code a page_write_failed hold names) over every hold, then PLAIN_STATUS_HOLDS (20) rows needs-human first, or with --detail a page of the shared listing contract (bounded-listing.ts, kind sync_status_holds:<source>:<state>:<filter>, --limit/--cursor/--all, --hold-state), holds_page only when a page left rows out or took a cursor, the rows last. GST-1209 follow-up: --slug <slug> (slug or path) and --code <code> (hold code or fault_code) filter the rows before paging and echo as holds_filter; the summary stays over every hold. Tests: test/sync-status-truthful-7243.test.ts (both engines), test/segmented-untiled-hold-gst1209.test.ts. #7679: worktree_refresh is the active refresh of the source's worktree (read in the source statement, so no extra statement) with its holder, liveness/orphaned and fix; an orphaned one's --resume fix becomes next unless a person's decision comes first. GBRA-287: a last_error from a run that stopped before any write request was admitted (discovery, freeze, admission) carries request_id: null explicitly, since no receipt, root request or write_error_detail exists; holdVerdict classifies a page_write_failed freeze hold whose write_failure.code is a page-body code (PAGE_BODY_FAULT_CODES) by its freeze_page_body_* row.

  • src/core/persistence/resolve-paths.ts — GBRA-150 resolveSourcePaths: what gbrain did with each file path of a source, by its own rules, so a landing check stops copying them. Per path: the sync slug (resolveSlugForPath), presence in the last synced commit and at HEAD (Git object reads), the skip reason (unsyncableReason, plus typed_frontmatter for a skipped metafile), the open hold, the live page by origin (source_path) and by slug (slug_owner when another file owns it, so a case-folded twin never reads as this file's page), a differing frontmatter slug:, chunk and embedding counts (listEmbeddingStateRows with its slugs filter), and one of RESOLVE_PATH_STATES with searchable (true / 'keyword' / false), why and an existing-command fix. Bounded: at most 200 paths, indexed lookups only, nothing written. A source whose local_path is relative refuses sync_anchor_relative (exit 3, an ask_user gbrain sources set-path fix). Tests: test/sources-resolve-path.test.ts.

  • src/core/page-searchability.ts — #7190 readPageSearchability: each live page's state, searchable: true | 'keyword' | false, chunks, unembedded (visible chunks with a NULL active-column vector) and embedding_hold, folded from listEmbeddingStateRows and the budget-hold read. Shared by resolve-path and get_write_request's applied; gbrain pages status (#7181) can compose it with its Git view.

  • src/core/persistence/applied-watermark.ts — #7190 readAppliedWatermark (sources status, MCP sources_status, sync --json) and the cheap readWriteWatermark (get_write_request): indexed_through is sources.last_commit (only a finished pass moves it), embedded_through reads the oldest page with a NULL vector (held pages count), committed_through_request is one below the oldest unsettled request of the incarnation, pending adds requests, files behind HEAD, held files and withheld deletes, writer_heartbeat is the drain-lease holder and its last progress (redacted for remote callers).

  • src/core/persistence/write-applied.ts — #7190 readWriteApplied: the live applied block get_write_request returns with applied: true (searchable_yet, state, gap, sequence, write watermark). Write replies stay frozen so same-request-id replays return them unchanged.

  • src/core/persistence/landing-gaps.ts — GBRA-150 readLandingGaps: committed managed_sync receipts of the last 24 h (at most 2,000, newest first through the watermark index, 2 s statement probe; a timeout reads unknown, never ok) whose page is still missing, has chunks waiting for vectors, has no chunks, or is quarantined past sync.freshness_target_seconds; one row per page with reason and fix. Consumers: sync status --detail landing / landing_read (sync-status.ts) and doctor sync_landing_gaps. Tests: test/sync-landing-gaps.test.ts.

  • src/commands/doctor/checks/landing-gaps.ts — GBRA-150 doctor sync_landing_gaps (registered after partial_embedding, one of CHEAP_INCIDENT_CHECKS): warns per source with the gap count by reason and the first fix from readLandingGaps; unknown read stays a warning, never ok. Tests: test/doctor-sync-landing-gaps.test.ts.