Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

docs: keep the key-files entry under its cap; name the round_batch event (#7866)

garrytan8edd46576ec9
diff --git a/docs/architecture/key-files/core-persistence-continued.md b/docs/architecture/key-files/core-persistence-continued.md
index 0787b07723..b303cb0d36 100644
--- a/docs/architecture/key-files/core-persistence-continued.md
+++ b/docs/architecture/key-files/core-persistence-continued.md
@@ -23,7 +23,7 @@ The #6188 fence check and fence repair live in [fence repair](core-fence-repair.
- `src/core/connectors/connector-text.ts` — #5752 render boundary: prose sanitized with `sanitizeForJsonb` before the no-op screen; identity fields refuse with `invalid_connector_text`.
- `src/core/persistence/connector-account.ts` — resolves the credential's account before any service runs: Google through `probeGoogleAccount` in `google/api-access.ts` (Gmail profile, Calendar primary calendar or People `me`, first enabled service that answers, else OAuth userinfo, #7073), returning the probe with every per-service error; when nothing names the account it rethrows `api_not_enabled` first, then another credential error, then the first other failure, so `connector_account_changed` means a resolved account that differs from the pin (test/google-api-access-managed-7073.test.ts), GitHub App installation id from the minted token, or the `/user` login for token auth under `scope: auto`.
- `src/core/persistence/connector-errors.ts` — `connector_account_changed` with its two-exit hint, the retired-intent messages, and `receiptDeliveredHint`, which rebuilds `suggestion`/`detail`/`docs` for codes delivered through a stored receipt (the coordinator persists only code and message); `writeResponse` applies it.
-- `src/core/persistence/noop-kernel.ts` — the #5470 no-op kernel. `inspectUnchanged` takes a preparer's `PreparedMutation` for an unadmitted `screeningRequest` and returns `contentUnchanged`, `projectionWorkRequired` (safe-chunk fence, text projection behind the knowledge revision, embedded page without a contextual mode), `metadataWorkRequired` (source path) and `observedRevision`; bound pages also need local file bytes equal to the serialized page. The page's current `knowledge_revision`, read with the projection flags, must still equal the snapshot's revision (a write after the screen's snapshot admits as `revision_moved`). Callers run the preparer's own `validate` before counting a skip. An optional `waive` list (#5751) lets a caller skip two admit reasons its no-op publication can never resolve, a pending contextual-mode stamp and file bytes that differ from the prepared content (never a missing file); the result lists each waiver applied in `waived`; only managed working-tree sync entries and `put-page-screen.ts` pass it. Used by connector sync, `import-mutations.ts`, `sync-run.ts` (working-tree and company-profile entries; renames and overlays always admit) and `put-page-screen.ts` (the managed transcript import's `put_page`, waiving `canonical_file_differs` for a file the preparer accepted). `chunksEmbeddedAsIs` (#7866) says whether a content no-op leaves the page no embedding work (text projection sealed at its revision and at or above the safe-chunk fence, and every current-generation chunk holding a vector for its exact text); `sync-prepare.ts` then sets `embeddingCurrent` and `queuePublicationEffects` (`effect-journal.ts`) queues no embedding effect (`embedding_state: current`). Tests: `test/noop-kernel-paths.test.ts`, `test/connector-wave3.test.ts`, `test/git-canonical-conversion-strict-pass-7866.test.ts`.
+- `src/core/persistence/noop-kernel.ts` — the #5470 no-op kernel. `inspectUnchanged` takes a preparer's `PreparedMutation` for an unadmitted `screeningRequest` and returns `contentUnchanged`, `projectionWorkRequired` (safe-chunk fence, text projection behind the knowledge revision, embedded page without a contextual mode), `metadataWorkRequired` (source path) and `observedRevision`; bound pages also need local file bytes equal to the serialized page. The page's current `knowledge_revision`, read with the projection flags, must still equal the snapshot's revision (a write after the screen's snapshot admits as `revision_moved`). Callers run the preparer's own `validate` before counting a skip. An optional `waive` list (#5751) lets a caller skip two admit reasons its no-op publication can never resolve, a pending contextual-mode stamp and file bytes that differ from the prepared content (never a missing file); the result lists each waiver applied in `waived`; only managed working-tree sync entries and `put-page-screen.ts` pass it. Used by connector sync, `import-mutations.ts`, `sync-run.ts` (working-tree and company-profile entries; renames and overlays always admit) and `put-page-screen.ts` (the managed transcript import's `put_page`, waiving `canonical_file_differs` for a file the preparer accepted). `chunksEmbeddedAsIs` (#7866): a no-op whose chunks hold current vectors queues no embedding. Tests: `test/noop-kernel-paths.test.ts`, `test/connector-wave3.test.ts`.
- `src/core/persistence/put-page-screen.ts` — #6741 B: `unchangedPutPage` screens a trusted local put_page (the transcript import) through `preparePageAdmission` (unadmitted), the page publication preparer and the no-op kernel; skips admission only on a genuine no-op with no pending request, no segmented body and no projection work. Tests: `test/transcripts-managed-noop.test.ts`.
- `src/core/persistence/import-reads.ts` — GBRA-75 wave 10: `readImportSnapshots` reads a run of pages as `readPageSnapshot(slug, { sourceId, includeDeleted: true })` returns each (soft-deleted rows, withdrawals and purge markers included) through `readPageSnapshotsBatch({ includeDeleted, absentIsNull })`, looping over its byte-budget prefixes; the managed import screen uses it for two or more new members. `importGroupReads` serves an import group's preparation (`group-publish.ts` `preparationReads`): each member's first own snapshot read comes from one batch, and `findDuplicatePage` calls that arrive together (all members, or a 2 ms window) are answered by one `findDuplicatePages` (`engine-sql/pages.ts`, same predicate and ranking per input); a failed batch falls back to each member's single read; with `reread` (the read-only screen) a member's later reads of its page come from the batch too. GBRA-108: `sliceReads` answers a screened slice's per-page reads (the origin claimants of `sync-origin.ts`, the stored timeline and take collisions of `canonical-projections.ts`, the safe-chunk fence of `page-state/projections.ts`, matched by their exported statements) from one statement per kind over the slice, row for row; a page with stored takes, a page or path outside the slice, a signalled call or a failed slice read runs its own statement (test `test/sync-legacy-sweep-gbra108.test.ts`). Tests: `test/import-reads.test.ts`, Postgres arm `test/e2e/import-reads.test.ts`.
diff --git a/docs/architecture/key-files/managed-sync-bookkeeping.md b/docs/architecture/key-files/managed-sync-bookkeeping.md
index 54819cf0bb..996e13bc06 100644
--- a/docs/architecture/key-files/managed-sync-bookkeeping.md
+++ b/docs/architecture/key-files/managed-sync-bookkeeping.md
@@ -39,7 +39,7 @@ run, and the supersession of a checkpoint whose cursor another run replaced. The
- `src/core/persistence/terminal-failures.ts` — #6650: the bounded readers over that classifier. `readTerminalFailureSummary` is one set-based aggregate (GROUPING SETS per source; `TERMINAL_STATEMENT_MS` 5 s statement and `TERMINAL_ACQUIRE_MS` 2 s connection deadlines; past either it returns `status: 'unknown'`, never an empty pile) splitting active blockers (recovery records plus the passed-in `checkpoint_blockers` of `sync-status.ts`, read by `readCheckpointBlockerMap`), unresolved rows (count, distinct slugs and intents, oldest age), historical rows (a later write superseded them, or older than `TERMINAL_FIXED_IN` via the SQL twin `TERMINAL_HISTORICAL_SQL`), 24 h and 7-day daily counts, categories and defect classes; archived sources are skipped. Thresholds `persistence.terminal_failures.window_days` (clamped to `persistence.receipt_retention_days`), `spike_factor`, `spike_min` (`readTerminalThresholds`, `validateTerminalConfigValue`, refused at `config set` by `commands/config/preparation-keys.ts`); `isSpike` against `sevenDayMedian`. `readTerminalFailures` is the drill-down (batches of 200, at most 2,000 rows scanned per call, a sequence cursor, verdict filters, `--group-by category` groups, a `sync_fault` group also split by its operation, with an exact `drill_down` argv each; `--operation` filters on `faultOperation`). Consumers: doctor `persistence_terminal_failures` (`commands/doctor/checks/persistence-requests.ts`), `sync status` `terminal_failures` (`sync-status.ts`, `next` falls back to a recovery blocker's action), the `writer_failures` administration operation, and the admin `/metrics` gauges (`serve-http-metrics.ts` `renderTerminalFailureMetrics`, one aggregate per engine shared by scrapes for `TERMINAL_METRICS_TTL_MS` = 60 s via `cachedTerminalSummary`; never `/health`). Each source also reports `new_last_24h_older_builds` (rows and builds older than their class's fix), and doctor's spike message says an older write owner is producing them when they are most of the spike. Tests: `test/terminal-failures-6650.test.ts`, `test/sync-status-terminal-6650.test.ts`, `test/terminal-failure-metrics-6650.test.ts`, the CLI journey `test/agent-journey-terminal-failures-6650.test.ts`.
- `src/commands/sources-writer-failures.ts` — #6650: output of `gbrain sources writer failures <source> [--state all] [--group-by category] [--since Nd] [--limit N] [--before <cursor>] [--category|--code|--origin|--operation] [--json]`, dispatched by `persistence-admin.ts` as `writer_failures` (validated in `administration.ts` `writerFailuresQuery`, locally or through the resident owner). A table by default, one JSON document with `--json`; row and group `fix` / `inspect` actions are rendered for the CLI; a full page prints `truncated` with the exact next command. The operation is advertised to clients in the owner's `administration_extensions` (`ipc.ts`, `admin-contract.ts` `PERSISTENCE_ADMIN_EXTENSIONS`), never in the `administration` list an older client validates strictly, and `local-client.ts` delegates it only to an owner that lists it (`ownerAdministers`). Tests: `test/writer-failures-cli-6650.test.ts`.
- `src/core/persistence/public-category.ts` — #6712 PR 2 (L5): `publicCategory({ category?, code })`, the closed public vocabulary (`drift`, `conflict`, `content_refused`, `file_missing`, `permission`, `storage`, `config`, `superseded`, `transient`, `defect`) that sits beside the unchanged diagnosis `category`. `PUBLIC_BY_TERMINAL` maps every `TerminalCategory`, `PUBLIC_BY_CODE` every `SYNC_FAULT_TABLE` code (a new fault code needs a row: `test/public-category.test.ts`), and anything else goes by its registry class (unknown codes are `defect`). `classifyTerminal` carries it as `public_category`; `classifyRecordedFault` (terminal-classify.ts) reads a hold, a managed-sync failure row or a run stop as the failed receipt it stands for, so the receipt diagnosis, `managedWrite`, sync status holds and `last_error`, the terminal summary and doctor `sync_failures` (`details.failures`) render one verdict.
-- Pass round phase lines and honest stalls (GBRA-133 PR 3a, #7243; `src/core/persistence/sync-pass-run.ts`, `sync-pass-status.ts`, `sync-drain.ts`) — each pass round emits `managed_sync.round_level` (paths, target, read time) after its level read and `managed_sync.round_admit` (imports, deletes, in flight) before its import walk, and an announcing drain prints them (`[sync] pass round N: …`). The `convergence` block's `target` is the newest pass's pinned commit (telemetry) and `converged_target` the stored converged obligation's target, null while no stamp backs one, so status never presents the pin as converged. The drain counts only page and checkpoint commits as progress (a legacy-sweep slice or a level read still working only keeps the quiet clock fresh); an abandoned pass's late progress is dropped; a stop whose head owner reports `restart_required` is `owner_restart_required` (`drainNext` names the process); a run that committed nothing (no advanced cursor, no committed page) stops as `sync_deadline_stalled` (exit 1). Tests: `test/sync-pass-phase-lines-7243.serial.test.ts`, `test/sync-status-truthful-7243.test.ts`, `test/honest-stalls-gbra133.test.ts`.
+- Pass round phase lines and honest stalls (GBRA-133 PR 3a, #7243; `src/core/persistence/sync-pass-run.ts`, `sync-pass-status.ts`, `sync-drain.ts`) — each pass round emits `managed_sync.round_level` (paths, target, read time) after its level read and `managed_sync.round_admit` (imports, deletes, in flight) before its import walk, and a `managed_sync.round_batch` (`batch` of `batches`, starting at candidate `base`, #7866) before each import batch; an announcing drain prints the level and admit lines (`[sync] pass round N: …`). The `convergence` block's `target` is the newest pass's pinned commit (telemetry) and `converged_target` the stored converged obligation's target, null while no stamp backs one, so status never presents the pin as converged. The drain counts only page and checkpoint commits as progress (a legacy-sweep slice or a level read still working only keeps the quiet clock fresh); an abandoned pass's late progress is dropped; a stop whose head owner reports `restart_required` is `owner_restart_required` (`drainNext` names the process); a run that committed nothing (no advanced cursor, no committed page) stops as `sync_deadline_stalled` (exit 1). Tests: `test/sync-pass-phase-lines-7243.serial.test.ts`, `test/sync-status-truthful-7243.test.ts`, `test/honest-stalls-gbra133.test.ts`.
- `src/core/persistence/sync-tail.ts` + `sync-tail-record.ts` — #6712 PR 2 (L5): the tail of a committed checkpoint (projection statistics, then link extraction for the run's imported slugs read from its manifest). `recordSyncTail` runs inside the checkpoint's publication transaction (`applyManagedSyncCheckpoint`) and appends `{run_id, changed, extract}` to `op_checkpoints` `managed-sync-tail` / `<source>` (at most 16), so the tail is owed before anyone can read the run as synced. `withSyncTail` runs it after the outermost drain lease is released (`performManagedBranch`, and `performManagedSync` when it took the lease itself; a call still inside a lease, a dry run and a lease-held follower run none), bounded by `SYNC_TAIL_BUDGET_MS` (5 min) and the caller's deadline, and folds `tail` (`done` / `truncated` / `degraded` / `deferred` / `failed`, with `underived`: the run's imports its link extraction left underived, which the per-source "still owe extraction" line prints) and `links` into the result. The obligation is cleared by compare-and-swap only when every step ran; a failed, deferred or truncated tail attaches a degraded `sync_tail_degraded` notice with its fix (the sync CLI prints it with the capacity notices). Fault point `sync:tail_started`. Tests: `test/sync-drain-finish-tail.test.ts`.
- `src/core/repair/slug-autoheal.ts` — the sync-time heal of stray `slug:` lines (#6865): `healAfterManagedSync`, called once from `commands/sync/perform.ts` `performManagedBranch` after the drain lease is released, runs `slug-conflicts`' `healAbsentSlugLine` (deterministic `absent_page` only, bound to the held bytes, nothing recorded for any other analysis) over the source's slug-conflict holds through the same `submitManagedFileRepair` write, with the receipt's `declared_alias` recorded as a `slug_aliases` row in that write. Skipped (and said why on `slug_lines_removed`) under `sync.slug_conflicts.auto_remove false`, `fences.repair.enabled false`, remote submission authority, an unfinished managed-sync cursor, or a host that may not write the checkout; bounded by `SLUG_HEAL_MAX_FILES` and `SLUG_HEAL_BUDGET_MS` (or the caller deadline), one checkout scan per source; never throws. Lane verdicts carry `lane_version` (`CONTENT_REPAIR_LANE_VERSION` in `persistence/sync-fault-class.ts`): an older verdict is `repair` in `contentHoldDisposition` and never a person's decision in `classifySyncFault`. Tests: `test/slug-autoheal-6865.test.ts` (Postgres arm `test/e2e/slug-autoheal-6865-postgres.test.ts`), `test/slug-conflict-verdict-6865.test.ts`.
- `src/core/since.ts` — #6712 PR 2: `parseSince(input, flag, now)`, the one `--since` parser (`gbrain agent logs --since`, `gbrain repair file-drift --since`). An age (`90s`, `30m`, `12h`, `7d`) becomes one concrete UTC instant at parse time; an ISO-8601 date or timestamp is normalized; anything else throws `invalid_params` naming the flag. Callers print and hash the instant, never the age. Tests: `test/sync-exits-l4-6712.test.ts`, `test/agent-cli.test.ts`.