Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(upgrade): keep the module old updaters import after the swap a leaf (GST-2688)

garrytan89c5271bdefe
diff --git a/docs/architecture/key-files/managed-sync-bookkeeping.md b/docs/architecture/key-files/managed-sync-bookkeeping.md
index 30f639aa0d..491aa2ea82 100644
--- a/docs/architecture/key-files/managed-sync-bookkeeping.md
+++ b/docs/architecture/key-files/managed-sync-bookkeeping.md
@@ -46,7 +46,8 @@ run, and the supersession of a checkpoint whose cursor another run replaced. The
- `src/core/persistence/sync-ghosts.ts` — #6712 PR 2: `countSyncGhosts(engine, sourceId)`, the post-sync ghost count. It reuses `scanGhostSource` from `repair/ghost-pages.ts` (no second classifier) with a candidate bound of `SYNC_GHOST_CHECK_LIMIT` (50) stats, leaves out paths sync holds (a held file is the hold's to explain), and returns the count, samples and the `gbrain repair ghost-pages --source <id>` preview as its fix. Sync never deletes or tombstones a ghost; `commands/sync/perform.ts` prints one line and sets `SyncResult.ghostPages`. Test: `test/sync-exits-l4-6712.test.ts`. GBRA-133 2b.2: the count is answered from the last scan (op_checkpoints op `sync-ghost-report`, fingerprint the source id); an atomic claim on that row starts the next scan in the background at most once per `GBRAIN_SYNC_GHOST_INTERVAL_MS` (default 300000) per source, and only after a sync landed since the last claim (`sources.last_sync_at`), so a cancelled sync writes nothing. The sync never waits for the scan; a CLI exit waits within the background-work drain budget and an engine disconnect aborts it. `syncGhostDecisions` records each decision for tests.
- `src/core/persistence/soak-counters.ts` — #6712 PR 2: the soak counters doctor `sync_soak_counters` reports (one `op_checkpoints` row per UTC day under op `sync-soak-counters`). Interventions: `sync_non_plain` (`nonPlainSyncFlags`, from `sync/run.ts`), `repair_apply` (`repair.ts`), `unblock_apply` (`sync/operator.ts`), `retry_held` (`sources-retry-held.ts`) and `needs_human` (`sweepNeedsHumanHolds`, run after each managed sync: stamps `needs_human_at` on a hold the fault table now calls `needs_human` and counts it once; the stamp survives same-code re-holds). Content decisions: `file_drift_apply` and `file_drift_hold`. `readSoakCounters` sums the last seven days. Counting never fails the command that counts. Test: `test/sync-exits-l4-6712.test.ts`.
- `src/commands/doctor/checks/managed-sync-movement.ts` + `src/core/persistence/sync-movement.ts` — #6317 (B4, I2): the movement reader and its doctor check. `readManagedSyncMovement` snapshots, per active managed source, the newest `completed_at` of committed `managed_sync_*` receipts in the current source incarnation (the watermark: one probe of the partial index `persistence_requests_sync_watermark`, v223; a count would shrink under compaction), the head claim's `step_since`, the `managed-sync` cursor (`index/total`, `held`) and `sources.last_sync_at`/`last_commit`, and whether a drain or full consumer is live on the owner host. `data_moving` is false when an unfinished cursor has had neither a committed receipt nor a head step advance for longer than `persistence.preparation_ceiling_ms` while a consumer is live; with none live the state is `parked` (informational). `sources status --json` and `writer status --json` print `data_moving`, `not_moving_since` and `movement_state` beside `sync_running`, and `sources status` prints the writer-status command beside `data_moving: false`; doctor `managed_sync_not_moving` (warn, scored for `not_moving` only) uses the same rule and command as its fix, and fails on a parked cursor whose admitted queue has no consumer past the ceiling (#6552; fix: the resume sync); `serve` prints one `[gbrain notice]` when a source flips. A failed backlog read is `unknown`, never `nothing_pending` (doctor warns). `/health` stays liveness-only.
-- `src/commands/sources-writer-movement.ts` — #6317 (I2): `gbrain sources writer movement [<source>] [--wait <dur>] [--warn-only] [--json]`, the deploy gate `gbrain upgrade` and `post-upgrade` end by printing (bare, so it inherits the default window; neither runs it, because neither restarts the resident processes). The printed step is `MOVEMENT_SUPERVISOR_STEP` from the import-free leaf `src/commands/movement-step.ts` (re-exported here), so upgrade loads it before the swap (#6604). Same source selection as `writer status` (`persistence-admin.ts`). It snapshots through `sync-movement.ts`, waits `max(300 s, preparation budget + 60 s)` (printed as the effective budget; `--wait <dur>` takes sync's duration syntax) and judges each source: `moved` (a new committed publication; a cursor advanced only by holds or a `last_sync_at` touched by an up-to-date check is not movement) and `current` exit 0; `held` exits 0 with a warning and the hold kind's route (`gbrain repair fences --source <id>`; `preparation_stalled` → writer status, then `gbrain sources retry-held <id>`); `within_allowance` (the head's step advanced inside the window) exits 0 with `retry_after_ms`; `not_moving` exits 1 with `managed_sync_not_moving` (`reason: movement_check`, `fix.argv` the writer-status command, `fix.verify` this command); `unknown` (backlog unreadable) exits 1 with `movement_unknown`; other read failures keep their code. `--warn-only` prints the same envelopes and exits 0. `parseMovementArgs` refuses `--json` or `--warn-only` written with a value (`invalid_params`), except `--json=true`, which it reads as `--json` (#7619). Flags are regenerated into `src/core/cli-flag-registry.generated.ts` with `bun run build:flag-registry`.
+- `src/commands/sources-writer-movement.ts` — GST-2688: a leaf that only re-exports `MOVEMENT_SUPERVISOR_STEP` from `movement-step.ts`. Updaters from about v0.60.150 to v0.60.248 import this path into the old process after the swap, where a module needing newer exports from the old cache throws, so it must import nothing else (`test/upgrade-old-updater-leaf.test.ts`).
+- `src/commands/sources-writer-movement-cli.ts` — #6317 (I2): `gbrain sources writer movement [<source>] [--wait <dur>] [--warn-only] [--json]`, the deploy gate `gbrain upgrade` and `post-upgrade` end by printing (bare, so it inherits the default window; neither runs it, because neither restarts the resident processes). The printed step is `MOVEMENT_SUPERVISOR_STEP` from the import-free leaf `src/commands/movement-step.ts` (re-exported here and by the leaf above), so upgrade loads it before the swap (#6604). Same source selection as `writer status` (`persistence-admin.ts`). It snapshots through `sync-movement.ts`, waits `max(300 s, preparation budget + 60 s)` (printed as the effective budget; `--wait <dur>` takes sync's duration syntax) and judges each source: `moved` (a new committed publication; a cursor advanced only by holds or a `last_sync_at` touched by an up-to-date check is not movement) and `current` exit 0; `held` exits 0 with a warning and the hold kind's route (`gbrain repair fences --source <id>`; `preparation_stalled` → writer status, then `gbrain sources retry-held <id>`); `within_allowance` (the head's step advanced inside the window) exits 0 with `retry_after_ms`; `not_moving` exits 1 with `managed_sync_not_moving` (`reason: movement_check`, `fix.argv` the writer-status command, `fix.verify` this command); `unknown` (backlog unreadable) exits 1 with `movement_unknown`; other read failures keep their code. `--warn-only` prints the same envelopes and exits 0. `parseMovementArgs` refuses `--json` or `--warn-only` written with a value (`invalid_params`), except `--json=true`, which it reads as `--json` (#7619). Flags are regenerated into `src/core/cli-flag-registry.generated.ts` with `bun run build:flag-registry`.
- `src/core/serve-head-watcher.ts` + `src/core/persistence/sync-freshness.ts` + `src/commands/doctor/checks/managed-sync-latency.ts` — #6858 ask 6: on-commit sync and its latency metric. `startServeHeadWatcher` (two call sites: `installResidentTickers` in `src/commands/serve.ts`, stopped before `shutdownDelegatedSync`, and `runServeHttp` in `src/commands/serve-http.ts`, beside `startMovementWatch` and stopped on the same shutdown path) re-reads `sync.on_commit` every 30 s; while on it polls `git rev-parse HEAD` in each managed source checkout this host owns (`GBRAIN_SERVE_HEAD_POLL_MS`, default 3 s; an `fs.watch` on the git dir only nudges the poll), debounces 3 s (`GBRAIN_SERVE_HEAD_DEBOUNCE_MS`) and starts one managed sync through the unchanged `startDelegatedSync` with this host's `cli` registration. It never starts beside a running delegated sync or a held drain lease (the move stays pending and fires once after), skips a HEAD `sources.last_commit` already has, resumes a resumable stop up to `MAX_RESUMES`, and logs each distinct error once. `readSyncFreshness` samples committed `managed_sync_import` receipts of the last 24 h (not first syncs or working-tree entries): commit time is the git committer time of `intent.target`, searchable is `completed_at`, embedded is the page's newest `content_chunks.embedded_at` once every chunk has one, sampled only from a receipt no later committed write to the same page supersedes and only at or after its `completed_at`; p50/p95 in one aggregate over the `persistence_requests_sync_watermark` predicate. #7604: the supersede check reads the newest committed sequence per (source, slug) once, over the rows after the oldest sample (a materialized CTE), so the read stays linear in the samples after a killed pass leaves tens of thousands of receipts (a per-receipt subquery is quadratic and reaches the 300 s statement timeout). `gbrain sync status` (the `freshness` block) and `gbrain status` print it; doctor `managed_sync_latency` grades p95 against `sync.freshness_target_seconds` (default 300; fail over max(1 h, target)) only with `sync.on_commit` on or a target set. Tests: `test/serve-head-watcher.test.ts`, `test/serve-head-watcher-managed.serial.test.ts`, `test/serve-http-head-watcher.serial.test.ts`, `test/sync-freshness-6858.test.ts`. #7189: each `SourceFreshness` carries `behind` (`readGitBehind`, injectable `BehindReader`: `rev-list --count <last_commit>..HEAD` and the oldest of those commits' committer age; null without a checkout or a synced commit), printed by `formatSyncFreshness`.
- `src/core/persistence/sync-status.ts` (status coherence) — GBRA-150: `state: blocked` means the source cannot advance (the drain outcome's meaning); a file hold that waits on a person while the source keeps syncing reads `synced` with reason `held_needs_human`, `needs_human` and `next` unchanged (`test/sync-status-blocked-definition.test.ts`); `detail` adds `landing` / `landing_read` (`landing-gaps.ts`). #7189: a `last_error` whose cursor failure row (`op_checkpoints` `managed-sync-failure`) is older than the source's newest committed managed-sync receipt carries `superseded` and sets neither `needs_human`, `state` nor `next` (`liveError`); `needs_human_from` names the hold or the error. #7243 (GBRA-133 PR 3a): every status carries `last_commit` (`sources.last_commit`) and `live` (the drain-lease holder still renewing it, from `readDrainLease`, read only when the source read's `gbrain_cycle_locks` probe finds a row, with `job_id` for a Minions sync job and `stalled` from `holderStalled` on the lease read's database clock (`db_now`); #7596: a holder that stopped renewing the lease (`readDrainLease`, `drainLeaseLive`) is not `live`, and an open pass with no live holder reads `pass_interrupted` (`passStatusState`) with an `interrupted` block (pass, last row write, the dead holder), which `sync-blocked.ts` also lists for doctor `managed_sync_blocked`; test `test/sync-pass-interrupted-7596.test.ts`; `live_job_id` also takes a job holder of a pass); `supersedingAttempt` marks a `last_error` `superseded` (`by`, `age_seconds`) when evidence newer than the failure row passed its phase (`resume` < `discovery` < `freeze`/`admission` < `receipt`/`checkpoint`; evidence: the pass row's `started_at` (pin), a level read (`timings_ms` set or an `up_to_date` idle tick), an uncommitted managed import or delete admitted later, a later commit). The CLI (`sync/operator.ts` `holdsListing`) bounds the holds: `holds_summary` (`topCounts` by code, by state: `needs_human`, `actionable`, `held`, and by `fault_code`, the `write_failure.code` a `page_write_failed` hold names) over every hold, then `PLAIN_STATUS_HOLDS` (20) rows needs-human first, or with `--detail` a page of the shared listing contract (`bounded-listing.ts`, kind `sync_status_holds:<source>:<state>:<filter>`, `--limit`/`--cursor`/`--all`, `--hold-state`), `holds_page` only when a page left rows out or took a cursor, the rows last. GST-1209 follow-up: `--slug <slug>` (slug or path) and `--code <code>` (hold code or `fault_code`) filter the rows before paging and echo as `holds_filter`; the summary stays over every hold. Tests: `test/sync-status-truthful-7243.test.ts` (both engines), `test/segmented-untiled-hold-gst1209.test.ts`. #7679: `worktree_refresh` is the active refresh of the source's worktree (read in the source statement, so no extra statement) with its holder, `liveness`/`orphaned` and fix; an orphaned one's `--resume` fix becomes `next` unless a person's decision comes first. GBRA-287: a `last_error` from a run that stopped before any write request was admitted (discovery, freeze, admission) carries `request_id: null` explicitly, since no receipt, root request or `write_error_detail` exists; `holdVerdict` classifies a `page_write_failed` freeze hold whose `write_failure.code` is a page-body code (`PAGE_BODY_FAULT_CODES`) by its `freeze_page_body_*` row.
- `src/core/persistence/resolve-paths.ts` — GBRA-150 `resolveSourcePaths`: what gbrain did with each file path of a source, by its own rules, so a landing check stops copying them. Per path: the sync slug (`resolveSlugForPath`), presence in the last synced commit and at HEAD (Git object reads), the skip reason (`unsyncableReason`, plus `typed_frontmatter` for a skipped metafile), the open hold, the live page by origin (`source_path`) and by slug (`slug_owner` when another file owns it, so a case-folded twin never reads as this file's page), a differing frontmatter `slug:`, chunk and embedding counts (`listEmbeddingStateRows` with its `slugs` filter), and one of `RESOLVE_PATH_STATES` with `searchable` (true / 'keyword' / false), `why` and an existing-command `fix`. Bounded: at most 200 paths, indexed lookups only, nothing written. A source whose `local_path` is relative refuses `sync_anchor_relative` (exit 3, an `ask_user` `gbrain sources set-path` fix). Tests: `test/sources-resolve-path.test.ts`.
diff --git a/src/commands/persistence-admin.ts b/src/commands/persistence-admin.ts
index f8b0e67271..dcd28e7aa3 100644
--- a/src/commands/persistence-admin.ts
+++ b/src/commands/persistence-admin.ts
@@ -242,7 +242,7 @@ export function parsePersistenceAdminArgs(group: Group, args: string[]): {
export async function runPersistenceAdminCli(group: Group, args: string[], connected?: BrainEngine): Promise<void> {
// #6317 (I2): the movement check waits a window on this process's own engine; it never crosses the owner IPC.
if (group === 'writer' && args[0] === 'movement') {
- const { runWriterMovementCli } = await import('./sources-writer-movement.ts');
+ const { runWriterMovementCli } = await import('./sources-writer-movement-cli.ts');
return runWriterMovementCli(args.slice(1), connected);
}
if (!args.length || args.some(arg => arg === '--help' || arg === '-h')) {
diff --git a/src/commands/sources-writer-movement-cli.ts b/src/commands/sources-writer-movement-cli.ts
new file mode 100644
index 0000000000..f9db94f16e
--- /dev/null
+++ b/src/commands/sources-writer-movement-cli.ts
@@ -0,0 +1,244 @@
+/**
+ * `gbrain sources writer movement [<source>] [--wait <dur>] [--warn-only] [--json]`
+ * (#6317, I2): a deploy ends with data movement, not process liveness.
+ *
+ * It snapshots every active managed source (`readSourceMovement`: committed
+ * managed receipt watermark, head step, cursor, holds, who is live), waits
+ * the window (default `max(300 s, persistence.sync_preparation_ms + 60 s)`,
+ * never below one sync preparation budget, so a healthy multi-wave group under
+ * a raised budget reads `within_allowance`, never `not_moving`; the output
+ * prints the budget it judged against), snapshots again and judges each
+ * source: `moved` (a new committed publication), `within_allowance` (no commit
+ * yet, but the head's step or the cursor advanced), `held` (advanced only by
+ * holds: #6298's containment, routed by hold kind), `parked` (pending work and
+ * nothing live to move it), `not_moving`, `nothing_pending`. The window ends
+ * early once every pending source has moved.
+ *
+ * Exit: `not_moving` → 1 with code `managed_sync_not_moving` (`reason:
+ * movement_check`) and the writer summary; `parked` → 1 with the same code
+ * and `state: parked` (after a restart nothing picked the catch-up up);
+ * `held`/`within_allowance`/`moved`/`nothing_pending` → 0; `unknown` (#6781:
+ * the backlog read failed) → 1 with its own code `movement_unknown` (`reason:
+ * backlog_read_failed`); `--warn-only` prints the same envelopes and exits 0.
+ * A read failure keeps its own code (`fix.next: report`), never
+ * `managed_sync_not_moving`. `gbrain upgrade`
+ * never runs this (it owns no relaunch); it prints the bare command for the
+ * supervisor's step.
+ */
+import type { BrainEngine } from '../core/engine.ts';
+import { getCliOptions } from '../core/cli-options.ts';
+import { finishCliTeardown, setCliExitVerdict, writeStdoutFinal } from '../core/cli-force-exit.ts';
+import { loadConfig, toEngineConfig } from '../core/config.ts';
+import { resolveBrainId } from '../core/brain-resolver.ts';
+import { loadMounts } from '../core/brain-registry.ts';
+import { opError } from '../core/ops/contract.ts';
+import { readFix } from '../core/ops/op-fix.ts';
+import { cliRenderContext, renderAction, toAgentError, type Action, type AgentEnvelope, type RenderedAction } from '../core/agent-output.ts';
+import { parseDurationSeconds } from '../core/sync-concurrency.ts';
+import { persistenceConfigForBrain } from '../core/persistence/local-client.ts';
+import { readPreparationPolicy } from '../core/persistence/switches.ts';
+import { judgeMovement, readSourceMovement, writerStatusCommand, type MovementVerdict, type SourceMovement } from '../core/persistence/sync-movement.ts';
+import { stalledHoldSteps } from '../core/persistence/sync-holds.ts';
+import { adminHostConfig } from './persistence-admin.ts';
+import { reportPersistenceCliError } from './persistence-delegate.ts';
+
+export const MOVEMENT_HELP = `Usage:
+ gbrain sources writer movement [<source>] [--source <id>] [--wait <dur>] [--warn-only] [--json] [--brain <id>]
+
+Proves that managed sync data moves: snapshots every active managed source,
+waits the window, and judges each one from committed pages and the head's step
+(a renewed lease, a live lock or a healthy /health are not movement).
+ --wait <dur> The window (30s, 5m, 1h; seconds when bare). Default: the larger of 5m and
+ persistence.sync_preparation_ms + 60s, printed as the budget judged against.
+ --warn-only Print the same verdicts and envelopes but exit 0 (pipelines that cannot fail).
+States: moved, within_allowance (a step or cursor advanced; a multi-wave group preparing
+normally), held (advanced only by holds; the hold kind's repair route is printed), parked
+(pending work and nothing live to move it), not_moving, nothing_pending, unknown (the
+sync backlog could not be read, so pending work is not known; never nothing_pending).
+Exit 1 with code managed_sync_not_moving for not_moving and parked, 1 with code
+movement_unknown for unknown; 0 otherwise.
+Run it after restarting serve and the workers: gbrain sources writer movement`;
+
+export { MOVEMENT_SUPERVISOR_STEP } from './movement-step.ts';
+
+export const MOVEMENT_DEFAULT_MIN_MS = 300_000;
+export const MOVEMENT_BUDGET_GRACE_MS = 60_000;
+const POLL_MS = 10_000;
+
+export interface MovementArgs { sourceId?: string; waitMs?: number; warnOnly: boolean; json: boolean; brain?: string }
+export function parseMovementArgs(args: string[]): MovementArgs {
+ const out: MovementArgs = { warnOnly: false, json: false };
+ const usage = (message: string, suggestion: string) => opError('invalid_params', message, suggestion,
+ { fix: readFix('Prints the movement command\'s flags and states.', { argv: ['gbrain', 'sources', 'writer', 'movement', '--help'] }) });
+ for (let i = 0; i < args.length; i++) {
+ const token = args[i]!;
+ const equal = token.indexOf('=');
+ const flag = token.startsWith('-') && equal > 0 ? token.slice(0, equal) : token;
+ const value = () => { const v = equal > 0 && token.startsWith('-') ? token.slice(equal + 1) : args[++i]; if (v === undefined || v.startsWith('--')) throw usage(`${flag} requires a value.`, `Give ${flag} its value right after it, as ${flag} <value>.`); return v; };
+ if (!token.startsWith('-')) { if (out.sourceId !== undefined) throw usage('Specify the source once.', `Name the source either as the argument or with --source, not both (here ${out.sourceId} and ${token}).`); out.sourceId = token; }
+ else if ((flag === '--json' || flag === '--warn-only') && flag !== token && token !== '--json=true') throw usage(`${flag} is a switch and takes no value.`, `Write ${flag} on its own.`);
+ else if (flag === '--json') out.json = true;
+ else if (flag === '--warn-only') out.warnOnly = true;
+ else if (flag === '--source') { if (out.sourceId !== undefined) throw usage('Specify the source once.', 'Name the source either as the argument or with --source, not both.'); out.sourceId = value(); }
+ else if (flag === '--brain') out.brain = value();
+ else if (flag === '--wait') {
+ const seconds = parseDurationSeconds(value(), '--wait');
+ if (seconds === undefined || seconds === null) throw usage('--wait requires a duration.', 'Give --wait a duration such as 30s, 5m or 1h (bare numbers are seconds).');
+ out.waitMs = seconds * 1000;
+ }
+ else throw usage(`Unknown option: ${flag}.`, 'The accepted options are --source, --wait, --warn-only, --json and --brain.');
+ }
+ return out;
+}
+
+/** The window the command judges against: never below one sync preparation budget, by default the larger of 5 minutes and the budget plus a minute. */
+export function movementWindowMs(policy: { syncMs: number }, requested?: number): { window_ms: number; budget_ms: number; source: 'default' | '--wait' } {
+ if (requested !== undefined) return { window_ms: Math.max(requested, policy.syncMs), budget_ms: policy.syncMs, source: '--wait' };
+ return { window_ms: Math.max(MOVEMENT_DEFAULT_MIN_MS, policy.syncMs + MOVEMENT_BUDGET_GRACE_MS), budget_ms: policy.syncMs, source: 'default' };
+}
+
+export interface MovementReport {
+ source_id: string;
+ state: MovementVerdict;
+ exit: 0 | 1;
+ code: 'managed_sync_not_moving' | 'movement_unknown' | null;
+ reason?: 'movement_check' | 'backlog_read_failed';
+ why: string;
+ fix: RenderedAction | null;
+ retry_after_ms?: number;
+ before: SourceMovement | undefined;
+ after: SourceMovement;
+}
+
+/** The verdict, exit and agent-operator envelope for one source over the window. */
+export function movementReport(before: SourceMovement | undefined, after: SourceMovement, windowMs: number): MovementReport {
+ const state = judgeMovement(before, after);
+ const ctx = cliRenderContext();
+ const id = after.source_id;
+ const status: Action = readFix('Read-only: the request at the head, its step and wait cause, the owner process and the next action on the running claim.',
+ { argv: ['gbrain', 'sources', 'writer', 'status', '--source', id, '--json'] });
+ const verify = { argv: ['gbrain', 'sources', 'writer', 'movement', id, '--json'] };
+ const head = after.head?.step ? ` The head is at step ${after.head.step}${after.head.waiting_on && after.head.waiting_on !== 'unknown' ? ` (waiting on ${after.head.waiting_on})` : ''}${after.head.owner ? `, held by ${after.head.owner.kind} pid ${after.head.owner.pid}` : ''}.` : '';
+ switch (state) {
+ case 'unknown':
+ return { source_id: id, state, exit: 1, code: 'movement_unknown', reason: 'backlog_read_failed',
+ why: `${id}: the managed sync backlog could not be read (${after.unknown?.error ?? 'no detail'}), so whether sync work is pending is not known; this is not a nothing-pending verdict.`,
+ fix: renderAction({ argv: ['gbrain', 'sources', 'writer', 'movement', id, '--json'], consent: [], actor: 'agent', requires_exclusive: false,
+ why: 'Read-only: reads the backlog again. If it keeps failing, gbrain doctor --json checks the database connection.', verify }, ctx), before, after };
+ case 'nothing_pending':
+ return { source_id: id, state, exit: 0, code: null, why: `${id}: no managed sync is pending${after.holds.count ? ` (${after.holds.count} held file(s) await their repair)` : ''}.`, fix: null, before, after };
+ case 'moved':
+ return { source_id: id, state, exit: 0, code: null, why: `${id}: a page committed at ${after.last_commit_at} inside the ${Math.round(windowMs / 1000)}s window; ${after.cursor ? `${after.cursor.remaining} remaining` : after.pass ? `${after.pass.remaining} path(s) unsettled in the pass` : 'the cursor is done'}.`, fix: null, before, after };
+ case 'within_allowance':
+ return { source_id: id, state, exit: 0, code: null, retry_after_ms: windowMs,
+ why: `${id}: no page committed yet, but the head's step or the cursor advanced inside the window (a multi-wave group commits nothing until its last wave).${head} Check again after one more window.`,
+ fix: renderAction({ argv: ['gbrain', 'sources', 'writer', 'movement', id, '--json'], consent: [], actor: 'agent', requires_exclusive: false,
+ why: 'The owner is preparing normally; rerun after retry_after_ms and the next window shows the commit.', verify }, ctx), before, after };
+ case 'held': {
+ const kinds = after.holds;
+ const route = kinds.stalled > 0 ? stalledHoldSteps(id).status : kinds.fences > 0 ? ['gbrain', 'repair', 'fences', '--source', id] : ['gbrain', 'repair', 'frontmatter', '--source', id];
+ const then = kinds.stalled > 0 ? ` then ${stalledHoldSteps(id).retry.join(' ')}` : '';
+ return { source_id: id, state, exit: 0, code: null,
+ why: `${id}: pending work advanced only by holds inside the window (${kinds.count} held: ${kinds.stalled} preparation_stalled, ${kinds.fences} fence, ${kinds.concurrent} concurrent-write). The containment is working, not a stall; the held files need their repair route.`,
+ fix: renderAction({ argv: route, consent: [], actor: 'agent', requires_exclusive: false, docs: 'docs/guides/repair.md#held-files',
+ why: kinds.stalled > 0 ? `Read-only: shows the owner the stalled preparations name;${then} once the cause is fixed.` : 'Read-only preview of the repair; it prints the apply command for the user to approve.', verify }, ctx), before, after };
+ }
+ case 'parked': {
+ const work = after.cursor ?? after.pass;
+ return { source_id: id, state, exit: 1, code: 'managed_sync_not_moving', reason: 'movement_check',
+ why: `${id}: the managed sync ${after.cursor ? 'cursor' : 'pass'} is pending at ${work?.index ?? '?'}/${work?.total ?? '?'} and nothing live is moving it (no sync run holds the lock; ${after.admitted} admitted request(s), no live consumer on the owner host). After a restart this means the catch-up was not started again.`,
+ fix: renderAction({ argv: ['gbrain', 'sync', ...(work?.resume_args ?? ['--source', id, '--no-pull'])], consent: [], actor: 'agent', requires_exclusive: false,
+ why: `Resumes the managed catch-up ${after.cursor ? 'where its cursor stopped' : 'with a plain sync that reconciles the checkout'}; the same command is safe to rerun in a loop.`, verify }, ctx), before, after };
+ }
+ case 'not_moving':
+ if (after.pass?.stuck) return { source_id: id, state, exit: 1, code: 'managed_sync_not_moving', reason: 'movement_check',
+ why: `${id}: every managed sync pass since ${after.pass.stuck_since} stopped ${after.pass.stuck}${after.pass.stop_reason ? ` (${after.pass.stop_reason})` : ''} with ${after.pass.remaining} path(s) unsettled, so the source does not converge even when pages commit.`,
+ fix: renderAction({ ...readFix('Read-only: the convergence of the source, the reason its passes stop and the next step.', { argv: ['gbrain', 'sync', 'status', '--source', id, '--json'] }), verify }, ctx), before, after };
+ return { source_id: id, state, exit: 1, code: 'managed_sync_not_moving', reason: 'movement_check',
+ why: `${id}: ${after.cursor ? `${after.cursor.remaining} entries pending, ` : ''}no page committed and no step advance inside the ${Math.round(windowMs / 1000)}s window while ${after.live.drain ? after.live.drain_job_id !== null && after.live.drain_pid === null ? `sync job #${after.live.drain_job_id} is draining it` : `a sync (pid ${after.live.drain_pid}) holds the lock` : 'a consumer is live'}; last progress ${after.last_progress_at ?? 'unknown'}.${head}`,
+ fix: renderAction({ ...status, verify }, ctx), before, after };
+ }
+}
+
+export function formatMovementReport(r: MovementReport): string[] {
+ const lines = [`${r.exit !== 0 ? '✗' : '✓'} ${r.source_id}: ${r.state}${r.code ? ` [${r.code}${r.reason ? `/${r.reason}` : ''}]` : ''}`, ` ${r.why}`];
+ if (r.fix?.command) lines.push(` Next: ${r.fix.command}${r.retry_after_ms ? ` (after ${Math.round(r.retry_after_ms / 1000)}s)` : ''}${r.fix.next === 'tell_user_to_run' ? ' (the host operator runs it)' : ''}`);
+ return lines;
+}
+
+export async function runWriterMovementCli(args: string[], connected?: BrainEngine): Promise<void> {
+ if (args.some(arg => arg === '--help' || arg === '-h')) { console.log(MOVEMENT_HELP); return; }
+ let owned: BrainEngine | undefined;
+ let json = false;
+ try {
+ const parsed = parseMovementArgs(args);
+ json = parsed.json;
+ const brainId = resolveBrainId(parsed.brain ?? getCliOptions().brain);
+ const config = adminHostConfig(persistenceConfigForBrain(loadConfig(), brainId, brainId === 'host' ? [] : loadMounts()), brainId, 'The movement check', 'sources writer movement');
+ if (!connected) {
+ const { createEngine } = await import('../core/engine-factory.ts');
+ owned = await createEngine(toEngineConfig(config));
+ await owned.connect(toEngineConfig(config));
+ }
+ const engine = connected ?? owned!;
+ const result = await runMovementCheck(engine, parsed, line => { if (!json) console.error(line); });
+ const exitCode = parsed.warnOnly ? 0 : result.exit_code;
+ if (json) await writeStdoutFinal(JSON.stringify({ ...result, exit_code: exitCode, warn_only: parsed.warnOnly, selected_brain: brainId }, null, 2) + '\n');
+ else for (const r of result.sources) for (const line of formatMovementReport(r)) console.log(line);
+ if (!json && !result.sources.length) console.log('No active managed source: nothing to judge (exit 0).');
+ if (!json && parsed.warnOnly && result.exit_code !== 0) console.log('--warn-only: exiting 0 despite the verdict above.');
+ setCliExitVerdict(exitCode);
+ } catch (error) {
+ if (!await reportPersistenceCliError(error, json)) {
+ console.error(error instanceof Error ? error.message : String(error));
+ setCliExitVerdict(1);
+ }
+ } finally { if (owned) await finishCliTeardown({ engine: owned, drainTimeoutMs: 1000 }); }
+}
+
+export interface MovementCheckResult {
+ schema_version: 1;
+ window_ms: number;
+ budget_ms: number;
+ window_source: 'default' | '--wait';
+ waited_ms: number;
+ sources: MovementReport[];
+ exit_code: 0 | 1;
+ /** The envelope of the first failing source (what a caller reads first), when any. */
+ error?: AgentEnvelope;
+}
+
+/** Snapshot, wait (ending early once every pending source moved), snapshot, judge. `note` receives the human progress lines. */
+export async function runMovementCheck(engine: BrainEngine, opts: { sourceId?: string; waitMs?: number; pollMs?: number; sleep?: (ms: number) => Promise<void> }, note: (line: string) => void = () => {}): Promise<MovementCheckResult> {
+ const policy = await readPreparationPolicy(engine);
+ const window = movementWindowMs(policy, opts.waitMs);
+ const scope = opts.sourceId ? { sourceIds: [opts.sourceId] } : {};
+ const before = await readSourceMovement(engine, scope);
+ const pendingBefore = before.filter(m => m.movement_state !== 'nothing_pending');
+ note(`[movement] window ${Math.round(window.window_ms / 1000)}s (${window.source === 'default' ? `default: max(300s, sync preparation budget ${Math.round(window.budget_ms / 1000)}s + 60s)` : `--wait, floored at the ${Math.round(window.budget_ms / 1000)}s sync preparation budget`}); `
+ + `${before.length} managed source(s), ${pendingBefore.length} with pending work.`);
+ const byId = new Map(before.map(m => [m.source_id, m]));
+ let after = before, waited = 0;
+ if (pendingBefore.length) {
+ const sleep = opts.sleep ?? (ms => new Promise<void>(resolve => setTimeout(resolve, ms)));
+ const poll = Math.max(1, Math.min(opts.pollMs ?? POLL_MS, window.window_ms));
+ const started = Date.now();
+ for (;;) {
+ await sleep(Math.min(poll, window.window_ms - waited));
+ waited = Date.now() - started;
+ after = await readSourceMovement(engine, scope);
+ const settled = after.every(m => { const v = judgeMovement(byId.get(m.source_id), m); return v === 'moved' || v === 'nothing_pending'; });
+ if (settled || waited >= window.window_ms) break;
+ note(`[movement] ${Math.round(waited / 1000)}s: ${after.map(m => `${m.source_id} ${judgeMovement(byId.get(m.source_id), m)}`).join(', ')}`);
+ }
+ }
+ const sources = after.map(m => movementReport(byId.get(m.source_id), m, window.window_ms));
+ const failing = sources.find(r => r.exit !== 0);
+ const error = !failing ? undefined : toAgentError(failing.code === 'movement_unknown'
+ ? opError('movement_unknown', failing.why, `Rerun gbrain sources writer movement ${failing.source_id}; if the backlog read keeps failing, run gbrain doctor --json.`,
+ { reason: failing.reason, fix: readFix('Read-only: reads the managed sync backlog again.', { argv: ['gbrain', 'sources', 'writer', 'movement', failing.source_id, '--json'] }) })
+ : opError('managed_sync_not_moving', failing.why, `Run ${writerStatusCommand(failing.source_id)} and follow the running claim's next step; verify with gbrain sources writer movement ${failing.source_id}.`,
+ { reason: failing.reason, fix: readFix('Read-only: the request at the head, its step, wait cause, owner and next action.', { argv: ['gbrain', 'sources', 'writer', 'status', '--source', failing.source_id, '--json'] }) }),
+ { transport: 'cli', command: 'sources writer movement', render: cliRenderContext() });
+ return { schema_version: 1, window_ms: window.window_ms, budget_ms: window.budget_ms, window_source: window.source, waited_ms: waited, sources, exit_code: failing ? 1 : 0, ...(error ? { error } : {}) };
+}
diff --git a/src/commands/sources-writer-movement.ts b/src/commands/sources-writer-movement.ts
index f9db94f16e..5e9375c3c4 100644
--- a/src/commands/sources-writer-movement.ts
+++ b/src/commands/sources-writer-movement.ts
@@ -1,244 +1,12 @@
/**
- * `gbrain sources writer movement [<source>] [--wait <dur>] [--warn-only] [--json]`
- * (#6317, I2): a deploy ends with data movement, not process liveness.
- *
- * It snapshots every active managed source (`readSourceMovement`: committed
- * managed receipt watermark, head step, cursor, holds, who is live), waits
- * the window (default `max(300 s, persistence.sync_preparation_ms + 60 s)`,
- * never below one sync preparation budget, so a healthy multi-wave group under
- * a raised budget reads `within_allowance`, never `not_moving`; the output
- * prints the budget it judged against), snapshots again and judges each
- * source: `moved` (a new committed publication), `within_allowance` (no commit
- * yet, but the head's step or the cursor advanced), `held` (advanced only by
- * holds: #6298's containment, routed by hold kind), `parked` (pending work and
- * nothing live to move it), `not_moving`, `nothing_pending`. The window ends
- * early once every pending source has moved.
- *
- * Exit: `not_moving` → 1 with code `managed_sync_not_moving` (`reason:
- * movement_check`) and the writer summary; `parked` → 1 with the same code
- * and `state: parked` (after a restart nothing picked the catch-up up);
- * `held`/`within_allowance`/`moved`/`nothing_pending` → 0; `unknown` (#6781:
- * the backlog read failed) → 1 with its own code `movement_unknown` (`reason:
- * backlog_read_failed`); `--warn-only` prints the same envelopes and exits 0.
- * A read failure keeps its own code (`fix.next: report`), never
- * `managed_sync_not_moving`. `gbrain upgrade`
- * never runs this (it owns no relaunch); it prints the bare command for the
- * supervisor's step.
+ * GST-2688: the path `gbrain upgrade` from about v0.60.150 to v0.60.248 loads
+ * into the OLD process after the new release is on disk
+ * (`(await import('./sources-writer-movement.ts')).MOVEMENT_SUPERVISOR_STEP`).
+ * That process keeps its old copies of every module it already loaded, so a
+ * newer module pulled in from here that imports a newer export from one of
+ * them throws and the upgrade exits 1. Keep this file a leaf: it re-exports
+ * the step from the import-free movement-step.ts and nothing else
+ * (test/upgrade-old-updater-leaf.test.ts). The command is
+ * sources-writer-movement-cli.ts.
*/
-import type { BrainEngine } from '../core/engine.ts';
-import { getCliOptions } from '../core/cli-options.ts';
-import { finishCliTeardown, setCliExitVerdict, writeStdoutFinal } from '../core/cli-force-exit.ts';
-import { loadConfig, toEngineConfig } from '../core/config.ts';
-import { resolveBrainId } from '../core/brain-resolver.ts';
-import { loadMounts } from '../core/brain-registry.ts';
-import { opError } from '../core/ops/contract.ts';
-import { readFix } from '../core/ops/op-fix.ts';
-import { cliRenderContext, renderAction, toAgentError, type Action, type AgentEnvelope, type RenderedAction } from '../core/agent-output.ts';
-import { parseDurationSeconds } from '../core/sync-concurrency.ts';
-import { persistenceConfigForBrain } from '../core/persistence/local-client.ts';
-import { readPreparationPolicy } from '../core/persistence/switches.ts';
-import { judgeMovement, readSourceMovement, writerStatusCommand, type MovementVerdict, type SourceMovement } from '../core/persistence/sync-movement.ts';
-import { stalledHoldSteps } from '../core/persistence/sync-holds.ts';
-import { adminHostConfig } from './persistence-admin.ts';
-import { reportPersistenceCliError } from './persistence-delegate.ts';
-
-export const MOVEMENT_HELP = `Usage:
- gbrain sources writer movement [<source>] [--source <id>] [--wait <dur>] [--warn-only] [--json] [--brain <id>]
-
-Proves that managed sync data moves: snapshots every active managed source,
-waits the window, and judges each one from committed pages and the head's step
-(a renewed lease, a live lock or a healthy /health are not movement).
- --wait <dur> The window (30s, 5m, 1h; seconds when bare). Default: the larger of 5m and
- persistence.sync_preparation_ms + 60s, printed as the budget judged against.
- --warn-only Print the same verdicts and envelopes but exit 0 (pipelines that cannot fail).
-States: moved, within_allowance (a step or cursor advanced; a multi-wave group preparing
-normally), held (advanced only by holds; the hold kind's repair route is printed), parked
-(pending work and nothing live to move it), not_moving, nothing_pending, unknown (the
-sync backlog could not be read, so pending work is not known; never nothing_pending).
-Exit 1 with code managed_sync_not_moving for not_moving and parked, 1 with code
-movement_unknown for unknown; 0 otherwise.
-Run it after restarting serve and the workers: gbrain sources writer movement`;
-
export { MOVEMENT_SUPERVISOR_STEP } from './movement-step.ts';
-
-export const MOVEMENT_DEFAULT_MIN_MS = 300_000;
-export const MOVEMENT_BUDGET_GRACE_MS = 60_000;
-const POLL_MS = 10_000;
-
-export interface MovementArgs { sourceId?: string; waitMs?: number; warnOnly: boolean; json: boolean; brain?: string }
-export function parseMovementArgs(args: string[]): MovementArgs {
- const out: MovementArgs = { warnOnly: false, json: false };
- const usage = (message: string, suggestion: string) => opError('invalid_params', message, suggestion,
- { fix: readFix('Prints the movement command\'s flags and states.', { argv: ['gbrain', 'sources', 'writer', 'movement', '--help'] }) });
- for (let i = 0; i < args.length; i++) {
- const token = args[i]!;
- const equal = token.indexOf('=');
- const flag = token.startsWith('-') && equal > 0 ? token.slice(0, equal) : token;
- const value = () => { const v = equal > 0 && token.startsWith('-') ? token.slice(equal + 1) : args[++i]; if (v === undefined || v.startsWith('--')) throw usage(`${flag} requires a value.`, `Give ${flag} its value right after it, as ${flag} <value>.`); return v; };
- if (!token.startsWith('-')) { if (out.sourceId !== undefined) throw usage('Specify the source once.', `Name the source either as the argument or with --source, not both (here ${out.sourceId} and ${token}).`); out.sourceId = token; }
- else if ((flag === '--json' || flag === '--warn-only') && flag !== token && token !== '--json=true') throw usage(`${flag} is a switch and takes no value.`, `Write ${flag} on its own.`);
- else if (flag === '--json') out.json = true;
- else if (flag === '--warn-only') out.warnOnly = true;
- else if (flag === '--source') { if (out.sourceId !== undefined) throw usage('Specify the source once.', 'Name the source either as the argument or with --source, not both.'); out.sourceId = value(); }
- else if (flag === '--brain') out.brain = value();
- else if (flag === '--wait') {
- const seconds = parseDurationSeconds(value(), '--wait');
- if (seconds === undefined || seconds === null) throw usage('--wait requires a duration.', 'Give --wait a duration such as 30s, 5m or 1h (bare numbers are seconds).');
- out.waitMs = seconds * 1000;
- }
- else throw usage(`Unknown option: ${flag}.`, 'The accepted options are --source, --wait, --warn-only, --json and --brain.');
- }
- return out;
-}
-
-/** The window the command judges against: never below one sync preparation budget, by default the larger of 5 minutes and the budget plus a minute. */
-export function movementWindowMs(policy: { syncMs: number }, requested?: number): { window_ms: number; budget_ms: number; source: 'default' | '--wait' } {
- if (requested !== undefined) return { window_ms: Math.max(requested, policy.syncMs), budget_ms: policy.syncMs, source: '--wait' };
- return { window_ms: Math.max(MOVEMENT_DEFAULT_MIN_MS, policy.syncMs + MOVEMENT_BUDGET_GRACE_MS), budget_ms: policy.syncMs, source: 'default' };
-}
-
-export interface MovementReport {
- source_id: string;
- state: MovementVerdict;
- exit: 0 | 1;
- code: 'managed_sync_not_moving' | 'movement_unknown' | null;
- reason?: 'movement_check' | 'backlog_read_failed';
- why: string;
- fix: RenderedAction | null;
- retry_after_ms?: number;
- before: SourceMovement | undefined;
- after: SourceMovement;
-}
-
-/** The verdict, exit and agent-operator envelope for one source over the window. */
-export function movementReport(before: SourceMovement | undefined, after: SourceMovement, windowMs: number): MovementReport {
- const state = judgeMovement(before, after);
- const ctx = cliRenderContext();
- const id = after.source_id;
- const status: Action = readFix('Read-only: the request at the head, its step and wait cause, the owner process and the next action on the running claim.',
- { argv: ['gbrain', 'sources', 'writer', 'status', '--source', id, '--json'] });
- const verify = { argv: ['gbrain', 'sources', 'writer', 'movement', id, '--json'] };
- const head = after.head?.step ? ` The head is at step ${after.head.step}${after.head.waiting_on && after.head.waiting_on !== 'unknown' ? ` (waiting on ${after.head.waiting_on})` : ''}${after.head.owner ? `, held by ${after.head.owner.kind} pid ${after.head.owner.pid}` : ''}.` : '';
- switch (state) {
- case 'unknown':
- return { source_id: id, state, exit: 1, code: 'movement_unknown', reason: 'backlog_read_failed',
- why: `${id}: the managed sync backlog could not be read (${after.unknown?.error ?? 'no detail'}), so whether sync work is pending is not known; this is not a nothing-pending verdict.`,
- fix: renderAction({ argv: ['gbrain', 'sources', 'writer', 'movement', id, '--json'], consent: [], actor: 'agent', requires_exclusive: false,
- why: 'Read-only: reads the backlog again. If it keeps failing, gbrain doctor --json checks the database connection.', verify }, ctx), before, after };
- case 'nothing_pending':
- return { source_id: id, state, exit: 0, code: null, why: `${id}: no managed sync is pending${after.holds.count ? ` (${after.holds.count} held file(s) await their repair)` : ''}.`, fix: null, before, after };
- case 'moved':
- return { source_id: id, state, exit: 0, code: null, why: `${id}: a page committed at ${after.last_commit_at} inside the ${Math.round(windowMs / 1000)}s window; ${after.cursor ? `${after.cursor.remaining} remaining` : after.pass ? `${after.pass.remaining} path(s) unsettled in the pass` : 'the cursor is done'}.`, fix: null, before, after };
- case 'within_allowance':
- return { source_id: id, state, exit: 0, code: null, retry_after_ms: windowMs,
- why: `${id}: no page committed yet, but the head's step or the cursor advanced inside the window (a multi-wave group commits nothing until its last wave).${head} Check again after one more window.`,
- fix: renderAction({ argv: ['gbrain', 'sources', 'writer', 'movement', id, '--json'], consent: [], actor: 'agent', requires_exclusive: false,
- why: 'The owner is preparing normally; rerun after retry_after_ms and the next window shows the commit.', verify }, ctx), before, after };
- case 'held': {
- const kinds = after.holds;
- const route = kinds.stalled > 0 ? stalledHoldSteps(id).status : kinds.fences > 0 ? ['gbrain', 'repair', 'fences', '--source', id] : ['gbrain', 'repair', 'frontmatter', '--source', id];
- const then = kinds.stalled > 0 ? ` then ${stalledHoldSteps(id).retry.join(' ')}` : '';
- return { source_id: id, state, exit: 0, code: null,
- why: `${id}: pending work advanced only by holds inside the window (${kinds.count} held: ${kinds.stalled} preparation_stalled, ${kinds.fences} fence, ${kinds.concurrent} concurrent-write). The containment is working, not a stall; the held files need their repair route.`,
- fix: renderAction({ argv: route, consent: [], actor: 'agent', requires_exclusive: false, docs: 'docs/guides/repair.md#held-files',
- why: kinds.stalled > 0 ? `Read-only: shows the owner the stalled preparations name;${then} once the cause is fixed.` : 'Read-only preview of the repair; it prints the apply command for the user to approve.', verify }, ctx), before, after };
- }
- case 'parked': {
- const work = after.cursor ?? after.pass;
- return { source_id: id, state, exit: 1, code: 'managed_sync_not_moving', reason: 'movement_check',
- why: `${id}: the managed sync ${after.cursor ? 'cursor' : 'pass'} is pending at ${work?.index ?? '?'}/${work?.total ?? '?'} and nothing live is moving it (no sync run holds the lock; ${after.admitted} admitted request(s), no live consumer on the owner host). After a restart this means the catch-up was not started again.`,
- fix: renderAction({ argv: ['gbrain', 'sync', ...(work?.resume_args ?? ['--source', id, '--no-pull'])], consent: [], actor: 'agent', requires_exclusive: false,
- why: `Resumes the managed catch-up ${after.cursor ? 'where its cursor stopped' : 'with a plain sync that reconciles the checkout'}; the same command is safe to rerun in a loop.`, verify }, ctx), before, after };
- }
- case 'not_moving':
- if (after.pass?.stuck) return { source_id: id, state, exit: 1, code: 'managed_sync_not_moving', reason: 'movement_check',
- why: `${id}: every managed sync pass since ${after.pass.stuck_since} stopped ${after.pass.stuck}${after.pass.stop_reason ? ` (${after.pass.stop_reason})` : ''} with ${after.pass.remaining} path(s) unsettled, so the source does not converge even when pages commit.`,
- fix: renderAction({ ...readFix('Read-only: the convergence of the source, the reason its passes stop and the next step.', { argv: ['gbrain', 'sync', 'status', '--source', id, '--json'] }), verify }, ctx), before, after };
- return { source_id: id, state, exit: 1, code: 'managed_sync_not_moving', reason: 'movement_check',
- why: `${id}: ${after.cursor ? `${after.cursor.remaining} entries pending, ` : ''}no page committed and no step advance inside the ${Math.round(windowMs / 1000)}s window while ${after.live.drain ? after.live.drain_job_id !== null && after.live.drain_pid === null ? `sync job #${after.live.drain_job_id} is draining it` : `a sync (pid ${after.live.drain_pid}) holds the lock` : 'a consumer is live'}; last progress ${after.last_progress_at ?? 'unknown'}.${head}`,
- fix: renderAction({ ...status, verify }, ctx), before, after };
- }
-}
-
-export function formatMovementReport(r: MovementReport): string[] {
- const lines = [`${r.exit !== 0 ? '✗' : '✓'} ${r.source_id}: ${r.state}${r.code ? ` [${r.code}${r.reason ? `/${r.reason}` : ''}]` : ''}`, ` ${r.why}`];
- if (r.fix?.command) lines.push(` Next: ${r.fix.command}${r.retry_after_ms ? ` (after ${Math.round(r.retry_after_ms / 1000)}s)` : ''}${r.fix.next === 'tell_user_to_run' ? ' (the host operator runs it)' : ''}`);
- return lines;
-}
-
-export async function runWriterMovementCli(args: string[], connected?: BrainEngine): Promise<void> {
- if (args.some(arg => arg === '--help' || arg === '-h')) { console.log(MOVEMENT_HELP); return; }
- let owned: BrainEngine | undefined;
- let json = false;
- try {
- const parsed = parseMovementArgs(args);
- json = parsed.json;
- const brainId = resolveBrainId(parsed.brain ?? getCliOptions().brain);
- const config = adminHostConfig(persistenceConfigForBrain(loadConfig(), brainId, brainId === 'host' ? [] : loadMounts()), brainId, 'The movement check', 'sources writer movement');
- if (!connected) {
- const { createEngine } = await import('../core/engine-factory.ts');
- owned = await createEngine(toEngineConfig(config));
- await owned.connect(toEngineConfig(config));
- }
- const engine = connected ?? owned!;
- const result = await runMovementCheck(engine, parsed, line => { if (!json) console.error(line); });
- const exitCode = parsed.warnOnly ? 0 : result.exit_code;
- if (json) await writeStdoutFinal(JSON.stringify({ ...result, exit_code: exitCode, warn_only: parsed.warnOnly, selected_brain: brainId }, null, 2) + '\n');
- else for (const r of result.sources) for (const line of formatMovementReport(r)) console.log(line);
- if (!json && !result.sources.length) console.log('No active managed source: nothing to judge (exit 0).');
- if (!json && parsed.warnOnly && result.exit_code !== 0) console.log('--warn-only: exiting 0 despite the verdict above.');
- setCliExitVerdict(exitCode);
- } catch (error) {
- if (!await reportPersistenceCliError(error, json)) {
- console.error(error instanceof Error ? error.message : String(error));
- setCliExitVerdict(1);
- }
- } finally { if (owned) await finishCliTeardown({ engine: owned, drainTimeoutMs: 1000 }); }
-}
-
-export interface MovementCheckResult {
- schema_version: 1;
- window_ms: number;
- budget_ms: number;
- window_source: 'default' | '--wait';
- waited_ms: number;
- sources: MovementReport[];
- exit_code: 0 | 1;
- /** The envelope of the first failing source (what a caller reads first), when any. */
- error?: AgentEnvelope;
-}
-
-/** Snapshot, wait (ending early once every pending source moved), snapshot, judge. `note` receives the human progress lines. */
-export async function runMovementCheck(engine: BrainEngine, opts: { sourceId?: string; waitMs?: number; pollMs?: number; sleep?: (ms: number) => Promise<void> }, note: (line: string) => void = () => {}): Promise<MovementCheckResult> {
- const policy = await readPreparationPolicy(engine);
- const window = movementWindowMs(policy, opts.waitMs);
- const scope = opts.sourceId ? { sourceIds: [opts.sourceId] } : {};
- const before = await readSourceMovement(engine, scope);
- const pendingBefore = before.filter(m => m.movement_state !== 'nothing_pending');
- note(`[movement] window ${Math.round(window.window_ms / 1000)}s (${window.source === 'default' ? `default: max(300s, sync preparation budget ${Math.round(window.budget_ms / 1000)}s + 60s)` : `--wait, floored at the ${Math.round(window.budget_ms / 1000)}s sync preparation budget`}); `
- + `${before.length} managed source(s), ${pendingBefore.length} with pending work.`);
- const byId = new Map(before.map(m => [m.source_id, m]));
- let after = before, waited = 0;
- if (pendingBefore.length) {
- const sleep = opts.sleep ?? (ms => new Promise<void>(resolve => setTimeout(resolve, ms)));
- const poll = Math.max(1, Math.min(opts.pollMs ?? POLL_MS, window.window_ms));
- const started = Date.now();
- for (;;) {
- await sleep(Math.min(poll, window.window_ms - waited));
- waited = Date.now() - started;
- after = await readSourceMovement(engine, scope);
- const settled = after.every(m => { const v = judgeMovement(byId.get(m.source_id), m); return v === 'moved' || v === 'nothing_pending'; });
- if (settled || waited >= window.window_ms) break;
- note(`[movement] ${Math.round(waited / 1000)}s: ${after.map(m => `${m.source_id} ${judgeMovement(byId.get(m.source_id), m)}`).join(', ')}`);
- }
- }
- const sources = after.map(m => movementReport(byId.get(m.source_id), m, window.window_ms));
- const failing = sources.find(r => r.exit !== 0);
- const error = !failing ? undefined : toAgentError(failing.code === 'movement_unknown'
- ? opError('movement_unknown', failing.why, `Rerun gbrain sources writer movement ${failing.source_id}; if the backlog read keeps failing, run gbrain doctor --json.`,
- { reason: failing.reason, fix: readFix('Read-only: reads the managed sync backlog again.', { argv: ['gbrain', 'sources', 'writer', 'movement', failing.source_id, '--json'] }) })
- : opError('managed_sync_not_moving', failing.why, `Run ${writerStatusCommand(failing.source_id)} and follow the running claim's next step; verify with gbrain sources writer movement ${failing.source_id}.`,
- { reason: failing.reason, fix: readFix('Read-only: the request at the head, its step, wait cause, owner and next action.', { argv: ['gbrain', 'sources', 'writer', 'status', '--source', failing.source_id, '--json'] }) }),
- { transport: 'cli', command: 'sources writer movement', render: cliRenderContext() });
- return { schema_version: 1, window_ms: window.window_ms, budget_ms: window.budget_ms, window_source: window.source, waited_ms: waited, sources, exit_code: failing ? 1 : 0, ...(error ? { error } : {}) };
-}
diff --git a/test/cli-flag-equals.test.ts b/test/cli-flag-equals.test.ts
index 6c97188f98..2c143a670d 100644
--- a/test/cli-flag-equals.test.ts
+++ b/test/cli-flag-equals.test.ts
@@ -21,7 +21,7 @@ import { strictSpecFor } from '../src/cli/strict-args.ts';
import { surfaceFileSource } from './helpers/source-surface.ts';
import { loadCuratedHelp } from '../src/cli/help/render.ts';
import { parseAccessFlag, parseSurfaceFlag } from '../src/mcp/surface.ts';
-import { parseMovementArgs } from '../src/commands/sources-writer-movement.ts';
+import { parseMovementArgs } from '../src/commands/sources-writer-movement-cli.ts';
import { parseAddTypeArgs } from '../src/commands/schema-add-type.ts';
import { parseRepairArgs } from '../src/commands/repair.ts';
import { parseTakesMutation } from '../src/commands/takes-mutation.ts';
@@ -356,7 +356,7 @@ describe('#7619 split rules', () => {
'src/commands/source-reconcile.ts': 'own parser: sources reconcile',
'src/commands/sources-lifecycle-args.ts': 'managed/delegated sources add|remove|archive|restore|purge|set-path|reclone parser; legacy fallback reads exact tokens only (see below)',
'src/commands/sources-reset-upstream.ts': 'own parser: sources reset-upstream (:90,:135 are git args)',
- 'src/commands/sources-writer-movement.ts': 'own parser: sources writer',
+ 'src/commands/sources-writer-movement-cli.ts': 'own parser: sources writer',
'src/commands/takes-mutation.ts': 'own parser: takes add|update|supersede|resolve|remove (:116 filters a fix argv); takes rebuild reads --source-id exact (:172)',
'src/core/embedding-migration-cli.ts': 'own parser: migrate embeddings, retrieval-upgrade',
@@ -710,7 +710,7 @@ describe('#7619 refusals that keep consent, scope and targets right', () => {
'src/commands/eval-longmemeval.ts': 'eval longmemeval', 'src/commands/git.ts': 'git', 'src/commands/persistence-admin.ts': 'auth local-writer',
'src/commands/repair.ts': 'repair', 'src/commands/schema-add-type.ts': 'schema add-type', 'src/commands/skillpack/harness.ts': 'skillpack remove',
'src/commands/source-reconcile.ts': 'sources reconcile', 'src/commands/sources-reset-upstream.ts': 'sources reset-upstream',
- 'src/commands/sources-writer-movement.ts': 'sources writer', 'src/commands/takes-mutation.ts': 'takes add', 'src/commands/google-attachments.ts': 'google attachments',
+ 'src/commands/sources-writer-movement-cli.ts': 'sources writer', 'src/commands/takes-mutation.ts': 'takes add', 'src/commands/google-attachments.ts': 'google attachments',
// normalizeReindexArgs splits its own value flags; the entry refuses other = forms before it runs.
'src/commands/reindex.ts': null,
// Governs managed brains only: an unmanaged brain falls back to runSources, which reads exact tokens, so the entry rule applies.
@@ -722,7 +722,7 @@ describe('#7619 refusals that keep consent, scope and targets right', () => {
repair: 'src/commands/repair.ts', 'schema add-type': 'src/commands/schema-add-type.ts', 'skillpack remove': 'src/commands/skillpack/harness.ts',
'sources connect': 'src/commands/company-brain-connect.ts', 'sources inspect': 'src/commands/company-brain-inspect.ts',
'sources reconcile': 'src/commands/source-reconcile.ts', 'sources reset-upstream': 'src/commands/sources-reset-upstream.ts',
- 'sources writer': 'src/commands/sources-writer-movement.ts', 'takes add': 'src/commands/takes-mutation.ts', 'takes remove': 'src/commands/takes-mutation.ts',
+ 'sources writer': 'src/commands/sources-writer-movement-cli.ts', 'takes add': 'src/commands/takes-mutation.ts', 'takes remove': 'src/commands/takes-mutation.ts',
'takes resolve': 'src/commands/takes-mutation.ts', 'takes supersede': 'src/commands/takes-mutation.ts', 'takes update': 'src/commands/takes-mutation.ts',
};
const found = [...new Glob('src/commands/**/*.ts').scanSync('.')].filter(p => /\b(?:eq|equal|equals) = \w+\.indexOf\('='\)|\.split\(\/=\(\.\*\)\/s\)/.test(readFileSync(p, 'utf8'))).sort();
diff --git a/test/sync-movement.test.ts b/test/sync-movement.test.ts
index a62015f8da..e0e036d578 100644
--- a/test/sync-movement.test.ts
+++ b/test/sync-movement.test.ts
@@ -33,7 +33,7 @@ import { claimWorktree } from '../src/core/persistence/ownership.ts';
import { syncLockId, tryAcquireDbLock, type DbLockHandle } from '../src/core/db-lock.ts';
import { withEnv } from './helpers/with-env.ts';
import { judgeMovement, readSourceMovement, startMovementWatch, formatSourceMovement, MOVEMENT_WATERMARK_SQL } from '../src/core/persistence/sync-movement.ts';
-import { movementReport, movementWindowMs, parseMovementArgs, runMovementCheck, runWriterMovementCli, MOVEMENT_HELP, MOVEMENT_SUPERVISOR_STEP } from '../src/commands/sources-writer-movement.ts';
+import { movementReport, movementWindowMs, parseMovementArgs, runMovementCheck, runWriterMovementCli, MOVEMENT_HELP, MOVEMENT_SUPERVISOR_STEP } from '../src/commands/sources-writer-movement-cli.ts';
import { managedSyncMovementEntry } from '../src/commands/doctor/checks/managed-sync-movement.ts';
import { deriveNext, cliRenderContext } from '../src/core/agent-output.ts';
import { CODES } from '../src/core/error-registry.ts';
diff --git a/test/sync-pass-readers-6712.test.ts b/test/sync-pass-readers-6712.test.ts
index 14fba42191..ba48ea2ec8 100644
--- a/test/sync-pass-readers-6712.test.ts
+++ b/test/sync-pass-readers-6712.test.ts
@@ -26,7 +26,7 @@ import { withEnv } from './helpers/with-env.ts';
import { MANAGED_SYNC_PASS_OP, writePassRow, type ManagedSyncPassValue } from '../src/core/persistence/sync-pass-row.ts';
import { readManagedSyncBacklog } from '../src/core/persistence/sync-drain.ts';
import { readSourceMovement } from '../src/core/persistence/sync-movement.ts';
-import { movementReport } from '../src/commands/sources-writer-movement.ts';
+import { movementReport } from '../src/commands/sources-writer-movement-cli.ts';
import { managedSyncMovementEntry } from '../src/commands/doctor/checks/managed-sync-movement.ts';
import { managedSyncBlockedEntry } from '../src/commands/doctor/checks/managed-sync-blocked.ts';
import { gitHeldFilesCheck } from '../src/commands/doctor/checks/git-holds.ts';
diff --git a/test/upgrade-old-updater-leaf.test.ts b/test/upgrade-old-updater-leaf.test.ts
new file mode 100644
index 0000000000..cd6d58f324
--- /dev/null
+++ b/test/upgrade-old-updater-leaf.test.ts
@@ -0,0 +1,34 @@
+/**
+ * GST-2688: the module old updaters load after the swap stays a leaf.
+ *
+ * 1. Protects: `gbrain upgrade` from about v0.60.150 to v0.60.248 ends with
+ * `(await import('./sources-writer-movement.ts')).MOVEMENT_SUPERVISOR_STEP`
+ * after the new release is on disk. That import runs in the OLD process,
+ * whose module cache still holds the old copies of everything it loaded at
+ * startup, so any module this file pulls in that imports a newer export
+ * from one of those (existingHostIdFor, markNotAdmitted, ...) throws, and
+ * the upgrade exits 1 although the binary and the migrations landed.
+ * So src/commands/sources-writer-movement.ts may load nothing but
+ * src/commands/movement-step.ts; the command lives in
+ * src/commands/sources-writer-movement-cli.ts.
+ * 2. Fails when: anyone adds an import to sources-writer-movement.ts, or an
+ * import to movement-step.ts.
+ * 3. Existing coverage: none; the Upgrade matrix's v0.60.241.0 cells caught it.
+ * 4. Seam: none. A child process loads the file into a fresh registry.
+ */
+import { describe, expect, test } from 'bun:test';
+import { resolve } from 'node:path';
+
+const REPO_ROOT = resolve(import.meta.dir, '..');
+
+describe('the path old updaters import after the swap', () => {
+ test('sources-writer-movement.ts loads only the movement-step leaf and still exports the step', () => {
+ const entry = resolve(REPO_ROOT, 'src/commands/sources-writer-movement.ts');
+ const script = `const m = await import(${JSON.stringify(entry)}); console.log(JSON.stringify({ step: m.MOVEMENT_SUPERVISOR_STEP, loaded: Object.keys(require.cache) }));`;
+ const r = Bun.spawnSync([process.execPath, '-e', script], { cwd: REPO_ROOT, stdout: 'pipe', stderr: 'pipe' });
+ expect(r.exitCode, r.stderr.toString()).toBe(0);
+ const out = JSON.parse(r.stdout.toString().trim().split('\n').pop()!) as { step: string; loaded: string[] };
+ expect(out.loaded.map(f => f.slice(REPO_ROOT.length + 1)).sort()).toEqual(['src/commands/movement-step.ts', 'src/commands/sources-writer-movement.ts']);
+ expect(out.step).toBe('after restarting serve and the workers, run: gbrain sources writer movement');
+ }, 60_000);
+});