Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
A large source's sync catch-up can finish: stopped passes that saved work count as progress (GST-2515) (#9420)
Garry Tanf4d1866314a6
diff --git a/behavior-changes.d/notifier_sync_disabled_gst2515.md b/behavior-changes.d/notifier_sync_disabled_gst2515.mdnew file mode 100644index 0000000000..2846cc0cfe--- /dev/null+++ b/behavior-changes.d/notifier_sync_disabled_gst2515.md@@ -0,0 +1 @@+The commit notifier now honours a source's sync setting: a source with sync turned off (config.syncEnabled=false) is no longer synced when HEAD moves, and a retry it owed is dropped. An explicit sync request (a sync trigger or webhook) still runs once; if that run fails, the notifier waits for a new request instead of retrying it. To have commits sync that source again, turn sync back on with gbrain sources set-sync <id> on.diff --git a/changelog.d/gbra-542-mirror-catchup.md b/changelog.d/gbra-542-mirror-catchup.mdnew file mode 100644index 0000000000..a393f8a91e--- /dev/null+++ b/changelog.d/gbra-542-mirror-catchup.md@@ -0,0 +1,9 @@+**A large source whose sync keeps stopping at its run limit now finishes catching up instead of being reported as abandoned.**++### Fixed+- A sync pass that saved at least one page or hold before its run limit stopped it now counts as progress, so `gbrain sync status` and `gbrain doctor` no longer report `pass_abandoned` for a source that is still moving. A pass that saved nothing still counts toward the abandoned streak, and the open pass row shows how many entries it has settled so far.+- On a read-only mirror, a page you edited in gbrain whose file is gone upstream is kept as a database-only page. Sync results and status report these as `carried` instead of listing them as holds to fix, and no command advises `--prefer database` on a mirror, since gbrain never writes a mirror's files.+- A symlink whose target does not exist is skipped and counted (`skipped.symlink_dangling`) instead of held as an unreadable file, and an older hold for one is released. A dangling link at a path that still has a live page stays held, and so does a committed file whose working copy alone was replaced by a dangling link.+- The commit notifier no longer starts a sync for a source with sync turned off (`config.syncEnabled=false`). Turning sync back on makes the notifier watch it again.++<!-- behavior-change: notifier_sync_disabled_gst2515 -->diff --git a/docs/architecture/key-files/canonical-reconciliation.md b/docs/architecture/key-files/canonical-reconciliation.mdindex 03259e0473..5c954cf6e9 100644--- a/docs/architecture/key-files/canonical-reconciliation.md+++ b/docs/architecture/key-files/canonical-reconciliation.md@@ -36,7 +36,7 @@ See [generations](generations.md) and [continued entries](canonical-reconciliati- `src/core/ghost-pages.ts` — #6696 B5, the one ghost-page classifier: a ghost is a live page for which `liveFileDrift` returns reason `canonical_file_missing` on the source's canonical owner host (`prepareFileTarget` tags that refusal `detail: 'canonical_file_missing'`). `scanGhostSource` reads a source's live pages in id order (at most `GHOST_SCAN_LIMIT`), resolves each recorded canonical path exactly as `prepareFileTarget` does and stats it, skips `publishesDatabaseOnly` and legacy slugs, and runs the full check on at most `candidateLimit` candidates; it returns the ghosts, `inspected`, `file_backed` (the mass-delete valve's population), `truncated` and the owner it ran under (host, worktree, owner epoch, incarnation). Off the owner host a source is `not_applicable` (`not_owner_host`, or `not_applicable` with no binding); a missing root is `unknown` (`storage_unavailable`), a stat error, invalid gbrain.yml or inconclusive check `unknown` (`ambiguous_storage`). `ghostPageNow` re-checks one page (the repair's recheck); `readGhostPages(engine, { sourceIds?, sampleLimit? })` is the counts-and-samples summary for doctor and the read-scope health op. Tests: `test/ghost-pages-6696.test.ts`.- `src/core/repair/ghost-pages.ts` — `gbrain repair ghost-pages --source <id>` (#6696 B5): explicit-only, preview-bound, destructive consent (`GHOST_CONSENT_TEXT`: the 72 h purge fuse, the per-slug `gbrain restore <slug> --force` undo, and the keep path of restoring the file and syncing). The preview binds the ghost set to the owner and each page's revision and path; the apply refuses `not_owner_host`, `ghost_storage_unknown`, `preview_changed` on an owner change, and `ghost_mass_delete_refused` (`massGhosts`: the sync valve, `MASS_RECONCILE_RATIO` over more than `MASS_RECONCILE_MIN_PAGES` file-backed pages, unless `massReconcileAllowed()`). Each page re-checks with `ghostPageNow`, then a managed brain submits one `managed_maintenance_retire_ghost_page` request (`prepareGhostPageRetirement`, dispatched from `prepared-maintenance.ts`: re-check, `delete_page` with `allowMissingFile`, refusal unless the prepared file target, a code or image ghost's recorded file (#7967), is the approved path with no bytes, so publication refuses a file that appeared later); an unmanaged brain soft-deletes in one transaction after the same check. Outcomes `soft_deleted`, `file_restored` (asks for a new preview), `changed_since_preview`. Doctor `ghost_pages` (`src/commands/doctor/checks/ghost-pages.ts`, BRAIN, after `fence_integrity`) reports the count, samples and coverage with the preview as its fix, `ok` + info off the owner host, unknown on an unreadable checkout; `gbrain pages ghosts` (`src/commands/pages-ghosts.ts`) lists them read-only.- `src/core/repair/legacy-twins.ts` — `gbrain repair legacy-twins --source <id>` (#6885): explicit-only, preview-bound, destructive consent (`LEGACY_TWINS_CONSENT_TEXT`). `scanLegacyTwins` reads a source's live pages with no recorded origin and no `database_only_reason` in id batches (at most `LEGACY_TWINS_SCAN_LIMIT`), keeps those whose slug `slugifyPath` folds differently, and pairs each with the live page at the folded slug that records a file (at most `LEGACY_TWINS_LIMIT` pairs and decisions). `pairVerdict` lists a pair as a decision (never merged) for differing `content_hash`, several legacy candidates, a twin file `liveFileDrift` does not call `agrees`, a file at the twin's own slug path, twin withdrawals, a legacy `file:` URI or a twin slug outside the op grammar. The preview binds pairs to the owner (host, worktree, owner epoch, incarnation) and both pages' ids, slugs and revisions; the apply refuses `not_owner_host`, `legacy_twins_storage_unknown`, `preview_changed` and `legacy_twin_mass_merge_refused` (`massTwins`, the sync valve). Each pair re-checks with `pairNow`, then a managed brain submits one `managed_maintenance_merge_legacy_twin` request on the twin (`prepareLegacyTwinMerge`, dispatched from `prepared-maintenance.ts`: no file target, `databaseOnlyReason: 'shared_origin'`, the legacy key in `additionalPageKeys`, `pairLocked` under the publication locks); an unmanaged brain merges in `maintenanceTransaction` under both page locks. `applyTwinMerge` records the twin's `source_path` on the legacy page, clears the twin's origin (so its tombstone's retention purge is database-only, never a `shared_origin` refusal), moves links (deduplicated; pair links stay), timeline rows, raw data, attachments, page aliases and facts about the twin, records the twin slug as a `slug_aliases` row, versions and soft-deletes the twin, and moves the twin's `sync-import-provenance` record to the legacy page at its post-write revision. Outcomes `merged`, `changed_since_preview`. Tests: `test/repair-legacy-twins-6885.test.ts` (Postgres: `test/e2e/repair-legacy-twins-6885-postgres.test.ts`).-- `src/core/persistence/unindexed-files.ts` — #6865 the unindexed-file detector: `findUnindexedFiles(engine, sourceId, opts)` streams `git ls-tree -r -l -z` of the source's last synced commit, filters it through discovery's own `loadSyncEligibility`, and in batches (one `pages.source_path = ANY` read on `pages_source_path_idx`, one hold read by fingerprint) keeps files with no live page recording their origin (source-path space, with the legacy Git-root spelling in source-root mode), no hold and no scheduled re-screen (`readGitHoldRetryPaths`; those count as `scheduled`); a file whose path slug belongs to a live page with another or no origin is still unindexed and counted `slug_taken`. Found files are screened read-only with `screenImportContent` (at most 2,000 per pass, the rest `unscreened`). Folder-scaffolding names sync skips on purpose (`isSyncMetafile`) that would otherwise be eligible (checked against a probe name in the same directory plus `sync.exclude` and the reserved skill paths) and carry a frontmatter `type:` or `title:` are counted apart as `metafile_skipped` (only under `sync.metafile_pages off`: under the default `typed` the eligibility predicate admits a nested one, so it is an ordinary unindexed file; each tree batch primes its metafile heads) with samples, and never scheduled. A time or count budget stops it with `complete: false` and a cursor (the last Git path, compared in byte order, which is Git's recursive tree order). `refreshUnindexedFiles` continues an incomplete stored result of the same incarnation and eligibility also after the source's commit moved (#7149): it first re-examines the paths before the cursor that changed between the two commits (`git diff-tree`; more than `UNINDEXED_REVISIT_CAP`, or an unreadable old commit, starts over), then resumes after the cursor. It schedules the found paths (`requestGitHoldRetry`, a scheduling write only) and stores one `op_checkpoints` row per source (op `unindexed-files`, fingerprint the source id; the 7-day checkpoint purge applies). `unindexedScanBudgetMs` sizes one pass's budget: 60 s per started 100k live pages, at most 5 min. `judgeUnindexedFiles` is doctor's read without scanning: a scheduled path with a page or a hold is settled even while the retry row still names it (#7149); of the rest, those in the retry row wait for a sync and the others are still unindexed. Tests: `test/unindexed-files.test.ts`, `test/unindexed-files-convergence-7149.test.ts`.+- `src/core/persistence/unindexed-files.ts` — #6865 the unindexed-file detector: `findUnindexedFiles(engine, sourceId, opts)` streams `git ls-tree -r -l -z` of the source's last synced commit, filters it through discovery's own `loadSyncEligibility`, and in batches (one `pages.source_path = ANY` read on `pages_source_path_idx`, one hold read by fingerprint) keeps files with no live page recording their origin (source-path space, with the legacy Git-root spelling in source-root mode), no hold and no scheduled re-screen (`readGitHoldRetryPaths`; those count as `scheduled`); a committed symlink whose target does not exist (`danglingLink` in `sync-entry-fault.ts`) is skipped by sync on every pass, so it is never unindexed; a file whose path slug belongs to a live page with another or no origin is still unindexed and counted `slug_taken`. Found files are screened read-only with `screenImportContent` (at most 2,000 per pass, the rest `unscreened`). Folder-scaffolding names sync skips on purpose (`isSyncMetafile`) that would otherwise be eligible (checked against a probe name in the same directory plus `sync.exclude` and the reserved skill paths) and carry a frontmatter `type:` or `title:` are counted apart as `metafile_skipped` (only under `sync.metafile_pages off`: under the default `typed` the eligibility predicate admits a nested one, so it is an ordinary unindexed file; each tree batch primes its metafile heads) with samples, and never scheduled. A time or count budget stops it with `complete: false` and a cursor (the last Git path, compared in byte order, which is Git's recursive tree order). `refreshUnindexedFiles` continues an incomplete stored result of the same incarnation and eligibility also after the source's commit moved (#7149): it first re-examines the paths before the cursor that changed between the two commits (`git diff-tree`; more than `UNINDEXED_REVISIT_CAP`, or an unreadable old commit, starts over), then resumes after the cursor. It schedules the found paths (`requestGitHoldRetry`, a scheduling write only) and stores one `op_checkpoints` row per source (op `unindexed-files`, fingerprint the source id; the 7-day checkpoint purge applies). `unindexedScanBudgetMs` sizes one pass's budget: 60 s per started 100k live pages, at most 5 min. `judgeUnindexedFiles` is doctor's read without scanning: a scheduled path with a page or a hold is settled even while the retry row still names it (#7149); of the rest, those in the retry row wait for a sync and the others are still unindexed. Tests: `test/unindexed-files.test.ts`, `test/unindexed-files-convergence-7149.test.ts`.- `src/commands/doctor/checks/unindexed-files.ts` — #6865 doctor `unindexed_files` (BRAIN, after `missing_recorded_files`): reads `judgeUnindexedFiles` for each bound managed Git source; `ok` when every result is complete, recent and nothing is missing; `warn` with counts by code, samples and age when files are unindexed (fix `gbrain sources retry-held <id> --unindexed`, `then` the follow-up sync from `gitFollowUpSync`, verify `gbrain doctor --only unindexed_files --json`) or scheduled and waiting (fix the sync); info `unknown` when a result is missing, older than 7 days or incomplete with nothing found, naming the maintenance phase and the command; info `not_applicable` on a classic brain. `metafile_skipped` files are named in the message and details and never make it warn; alone they are info `degraded` with a `user` fix (rename the file, then the sync). Test: `test/unindexed-files.test.ts`.- `src/commands/doctor/checks/missing-recorded-files.ts` — #6609 doctor `missing_recorded_files` (BRAIN, after `ghost_pages`): the classic-brain counterpart of `ghost_pages`. Read-only; per source with a checkout on this host and managed persistence off it runs `scanMissingRecordedFiles` (`src/core/persistence/detached-pages.ts`) and warns with counts, samples and an `ask_user` fix (put the file back and sync, or `gbrain delete` then `gbrain restore <slug> --source-id <id> --detach`) for live pages whose recorded file a `gbrain sync --full` would not find; `ok` + `not_applicable` when no source qualifies. Test: `test/restore-detach-6609.test.ts`.- `src/core/persistence/overwritten-edits.ts` — #6712 follow-up: `scanOverwrittenEdits` finds live pages whose database-only revision a managed-sync import replaced with different content and no hold, from durable rows only (it survives receipt compaction): `page_versions.write_request_id` / `archived_write_request_id`, the writer's committed receipt (`outcome.persistence.mode = 'database'`, not an import, no `git` effect for that revision, not a read-only mirror write), the archiving request recognised as a managed-sync import by intent kind or, once compacted, by operation `submit_job` + `outcome.imported_file` without `connector_database`, and `reconcileCanonical` digests (the import's own revision differs; no later revision holds the edit again). `high` or `unknown` (`no_persistence_outcome`, `import_content_unreadable`, `segmented`); the newest open overwrite per page; at most `OVERWRITE_SCAN_LIMIT` candidate versions per source. Revisions with no recorded writer are not reported.diff --git a/docs/architecture/key-files/commit-notifier.md b/docs/architecture/key-files/commit-notifier.mdindex 6744e291d4..15700fdc1b 100644--- a/docs/architecture/key-files/commit-notifier.md+++ b/docs/architecture/key-files/commit-notifier.md@@ -3,7 +3,7 @@[Subsystem index](../KEY_FILES.md). Read only the entries relevant to your change.Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.-- `src/core/commit-notifier.ts` — the notifier core, pure over its seams (store, git reads, the runner). One loop per resident: claims the host lease every heartbeat (`GBRAIN_NOTIFIER_HEARTBEAT_MS`, default 10 s, lease two heartbeats; PGLite has no lease and the brain's holder owns), re-reads `sync.on_commit` and `sync.on_commit.sources` every 30 s (`parseOnCommit` is the one parser), lists the host's sources every 60 s, reads HEAD every poll (`GBRAIN_NOTIFIER_POLL_MS`, 3 s; a watch only nudges), and scans wanted markers every 15 s or on a nudge, with the flag off too. Starts on the leading edge of a quiet HEAD move (no rebase/merge/cherry-pick marker, no `index.lock`), then a 400 ms trailing window (`GBRAIN_NOTIFIER_QUIET_MS`). Per-source single flight under a host cap, least recently started first. Keeps a source due across refused starts, failed or blocked runs and resumable stops; retries at `RETRY_BACKOFF_MS` (5 s, 15 s, 1 min, 5 min) and reports `notifier_stalled` past the last tier without progress (GBRA-133 PR 3a: a run whose status says `committed` is progress even before `last_commit` moves, and a drain-lease holder that `drainHeld` reports `stalled` is a failed attempt, `drain_holder_stalled`, instead of a silent defer; `test/honest-stalls-gbra133.test.ts`); `operator_blocked` waits for HEAD to move. Checks the fence before each start. Every decision is recorded (`decisions`) for tests; state and coverage rows are written on change only. Managed sources never take pull markers. Tests: `test/serve-head-watcher.test.ts` (memory store, `test/helpers/notifier-memory-store.ts`).+- `src/core/commit-notifier.ts` — the notifier core, pure over its seams (store, git reads, the runner). One loop per resident: claims the host lease every heartbeat (`GBRAIN_NOTIFIER_HEARTBEAT_MS`, default 10 s, lease two heartbeats; PGLite has no lease and the brain's holder owns), re-reads `sync.on_commit` and `sync.on_commit.sources` every 30 s (`parseOnCommit` is the one parser), lists the host's sources every 60 s, reads HEAD every poll (`GBRAIN_NOTIFIER_POLL_MS`, 3 s; a watch only nudges), and scans wanted markers every 15 s or on a nudge, with the flag off too. Starts on the leading edge of a quiet HEAD move (no rebase/merge/cherry-pick marker, no `index.lock`), then a 400 ms trailing window (`GBRAIN_NOTIFIER_QUIET_MS`). Per-source single flight under a host cap, least recently started first. Keeps a source due across refused starts, failed or blocked runs and resumable stops; retries at `RETRY_BACKOFF_MS` (5 s, 15 s, 1 min, 5 min) and reports `notifier_stalled` past the last tier without progress (GBRA-133 PR 3a: a run whose status says `committed` is progress even before `last_commit` moves, and a drain-lease holder that `drainHeld` reports `stalled` is a failed attempt, `drain_holder_stalled`, instead of a silent defer; `test/honest-stalls-gbra133.test.ts`); `operator_blocked` waits for HEAD to move. A source listed with `syncDisabled` (`config.syncEnabled=false`, GST-2515) is never watched and owes no retry; a wanted marker runs once, and a failed run waits as `operator_blocked` (`sync_disabled`) for a new request (`test/notifier-sync-disabled-gst2515.test.ts`). Checks the fence before each start. Every decision is recorded (`decisions`) for tests; state and coverage rows are written on change only. Managed sources never take pull markers. Tests: `test/serve-head-watcher.test.ts` (memory store, `test/helpers/notifier-memory-store.ts`).- `src/core/notifier/store.ts` — the durable rows (migration `v262-commit-notifier.ts`): `sync_wanted_markers` (source + intent freshness|pull|full, `generation` bumped on every upsert, `first_due_at` kept), `sync_notifier_leases` (one owner per host, fence bumped on takeover, release expires it now), `sync_notifier_sources` (state, retry schedule, last error and trace path, classic syncer host, coverage record). A drainer deletes only `WHERE generation =` the value it read (`clearMarker`, `settleMarkers`); freshness is due while `wanted_head IS DISTINCT FROM last_commit` (no ancestry assumption). `recordLocalSyncer` runs where `writeSyncAnchor` advances `last_commit`: records the syncer host and prunes satisfied markers (the CLI's re-check). `missingTable` lets callers on a pre-v262 brain read "nothing monitored".- `src/core/notifier/resident.ts` + `src/core/notifier/detect.ts` — production seams and the one start path for every resident (`startResidentNotifier`: serve, serve --http through `src/core/serve-head-watcher.ts`, `gbrain jobs work`, the autopilot daemon). Never awaits: the first pass runs in the background. `GBRAIN_NOTIFIER=0` turns it off per process. Lists managed sources whose worktree this host owns and classic sources whose recorded syncer is this host (or `GBRAIN_NOTIFIER_CLASSIC_SOURCES`), skipping the rest with a reason. Starts syncs through the in-process delegated runner. Trace per source at `<gbrain home>/notifier/<source>.jsonl`. `detect.ts` resolves `--absolute-git-dir` / `--git-common-dir` (linked worktrees) and the quiet predicate. Tests: `test/pathological/s-notifier-classic-serve.test.ts`, `test/pathological/s-notifier-residents.test.ts`. #7243: `runnerStart` starts the delegated run with `logLines`, so a background run nobody polls also writes its human lines (the pass's round phase lines and progress) to the serve log as `[serve-sync] job=<id> …` (`test/sync-pass-phase-lines-7243.serial.test.ts`).- `src/core/notifier/nudge.ts` + `src/core/notifier/handoff.ts` + `src/commands/sync/trigger.ts` — submitters and drainers of wanted markers. `recordTriggerMarker` picks the intent (managed: freshness at the checkout HEAD; classic: pull) and `recordSyncNudge` answers the versioned, acknowledged `sync_nudge` IPC kind (`src/core/context/sync-ipc.ts`, `resolve-ipc.ts`, bound in `src/mcp/resolve-ipc-binding.ts`). `handoff.ts` is the post-run re-check of the `sync` job handler and the serve's delegated runner: read before the run, settle after, at most one follow-up (a queued job with `maxWaiting: 1`, or a nudge); a lock-busy skip records a marker and completes. `gbrain sync trigger` answers queued / coalesced (`maxPending: 1`) / nudged, exit 0, with the `no_resident` notice when no resident watches the source; on PGLite a live serve takes it over IPC (`maybeTriggerThroughServe` in `src/commands/sync-delegate.ts`, which also records a busy delegated sync). The GitHub push webhook (`submitGitHubPush` in `src/commands/serve-http-webhooks.ts`) records the pushed sha first; a managed source answers 409 `refresh_required`. Tests: `test/pathological/s-notifier-marker-handoff.test.ts`.diff --git a/docs/guides/on-commit-sync.md b/docs/guides/on-commit-sync.mdindex 6a6658dd3b..7dcf22612d 100644--- a/docs/guides/on-commit-sync.md+++ b/docs/guides/on-commit-sync.md@@ -95,6 +95,7 @@ a sync is finishing is still served.|---|---|| sync.on_commit (config, default false) | Watch commits on this brain's sources || sync.on_commit.sources (config) | Comma-separated source ids never watched for commits |+| `gbrain sources set-sync <id> off` | The source is not watched for commits and owes no retry; a sync trigger or webhook still runs it once (a failed run waits for a new request instead of retrying), and `set-sync <id> on` restores the watch || GBRAIN_NOTIFIER=0 | No notifier in this process || GBRAIN_NOTIFIER_DETECTION=poll | Never use file watches || GBRAIN_NOTIFIER_POLL_MS | HEAD poll interval (default 3000) |@@ -178,9 +179,10 @@ itself is settled) and across a resident restart or a change of lease owner,which reads the attempt count the previous owner recorded. The short budget aresident gives its first run applies only to a source with no failed attempts,so a retry after a run that advanced nothing gets the full sync budget.-Turning sync.on_commit off, or excluding the source with-sync.on_commit.sources, still ends a retry that a HEAD move started: the source-goes idle.+Turning sync.on_commit off, excluding the source with+sync.on_commit.sources, or turning sync off for the source+(`gbrain sources set-sync <id> off`) still ends a retry that a HEAD move+started: the source goes idle.A run that committed pages counts as progress even before the source's lastsynced commit moves. When the last error is drain_holder_stalled, anotherdiff --git a/docs/guides/repair.md b/docs/guides/repair.mdindex 6f505d965a..c8e67f1a45 100644--- a/docs/guides/repair.md+++ b/docs/guides/repair.md@@ -4389,6 +4389,8 @@ re-screen, which the next sync consumes once it imports or holds the file; docto[hold](#held-files). `slug_taken` (the path slug belongs to another page) and `deleted_page`(the page was deleted while the file stayed committed) are never scheduled:rename one of the two files, or delete the file or restore the page.+A committed symlink whose target does not exist is skipped by sync, so it is+never reported as unindexed.`metafile_skipped` (never scheduled) lists titled scaffolding files sync skipson purpose (`index.md`, `README.md`, ...) under `sync.metafile_pages off`; underthe default `typed` a nested titled one is an ordinary unindexed file. To keepdiff --git a/docs/guides/sync-unblock-runbook.md b/docs/guides/sync-unblock-runbook.mdindex b3d48d8bbf..e822d09c8c 100644--- a/docs/guides/sync-unblock-runbook.md+++ b/docs/guides/sync-unblock-runbook.md@@ -165,6 +165,7 @@ Every hold and error has three fields, from one table| `rename_identity_unreadable` | page | none | yes | always: the file title or id holds characters the database cannot compare || `rename_reconcile_failed` | page | none | yes | always: the old page may be a duplicate || `file_drift_unknown` | page | reconcile | yes | always: the file and the page differ and no import record says which changed |+| `file_drift_mirror_database_only` | page | none | no | never: the page stays live and database-only because gbrain never writes a read-only mirror || `origin_conflict_decision` | page | none | yes | always: the pages differ, their ids conflict or none is the page the path names || `page_identity_decision` | page | none | yes | always: only a person decides which page the file is || `absent_from_worktree` | page | none | yes | always: only a person decides whether the file comes back or its deletion is committed |@@ -277,6 +278,18 @@ it and kept going; the receipt lists the hold (`held`, `held_count`) and`gbrain repair content --source <id>` previews it. A hold the lane will notdecide is `needs_human` with the paragraph in `human_reason`([held files](repair.md#held-files)).+- `file_drift_mirror_database_only`: on a read-only mirror+ (`gbrain sources mirror-readonly <id>`) a file gone upstream whose page was+ edited in gbrain stays held and its page stays live and database-only, since+ gbrain never writes the mirror. These holds never block the pass and are not+ re-screened every sync; the run and `sync status` report them as `carried`+ (class, count, a sample and the preview that lists them).+- A symlink whose target does not exist is not a page: sync skips it and+ counts it (`skipped.symlink_dangling`) instead of holding it as unreadable,+ and releases an unreadable hold an older release wrote for it. A dangling+ link at the path of a live page stays held, so the page is never deleted,+ and so does a committed file whose working copy alone was replaced by a+ dangling link.- The other repair codes (`invalid_frontmatter`, `rename_held`) stayconsent-gated: unblock refuses them with the frontmatter repair preview torun ([held files](write-refusals.md#held-files-and-content-refusals)).@@ -385,7 +398,7 @@ run converges on the next plain sync with no flag.|---|---|---|---|| `pass_unfinished` | resumable | The newest pass is open or stopped at a deadline. | The plain sync (`resume_argv`); safe to loop. || `pass_interrupted` | resumable | The newest pass is open but no process holds the source's drain lease: its process was killed or crashed (an out-of-memory guard, a timeout) before it ended the pass, so nothing recorded a failure. `live` is null and `interrupted` names the pass, its last write and, when the lock row outlived the process, the holder and the pages it committed. Doctor `managed_sync_blocked` lists it as `pass_interrupted` (warn, then fail after an hour). | The plain sync; if that pass stops the same way, the reason becomes `pass_abandoned`. |-| `pass_abandoned` | blocked | Two passes in a row never ended or ended on the same kind of transient stop: the drain stopped them for making no progress, their process was killed, or each hit a statement timeout, a lost connection or write back-pressure (#7982). The newest open pass counts once no process holds the source's drain lease (#7596; a reader without the lease read counts it once its row has not moved for 5 minutes). | `gbrain doctor --json` and a report with the gbrain version and the newest run's `drain.stall`; another plain sync is likely to stop the same way. |+| `pass_abandoned` | blocked | Two passes in a row never ended or ended on the same kind of transient stop, and neither committed a page or wrote a hold before it stopped (a pass that committed work moved the source, so it resets the count; a pass that only reported progress does not): the drain stopped them for making no progress, their process was killed, or each hit a statement timeout, a lost connection or write back-pressure (#7982). The newest open pass counts once no process holds the source's drain lease (#7596; a reader without the lease read counts it once its row has not moved for 5 minutes). | `gbrain doctor --json` and a report with the gbrain version and the newest run's `drain.stall`; another plain sync is likely to stop the same way. || `not_converged` | resumable | The pass used its four rounds while another writer kept changing the level. Everything it admitted stands. | The plain sync; after three such passes in a row `next` is `gbrain doctor --json` and a report, because another sync will not change it. || `settle_mismatch` | resumable | A path keeps settling without reading settled. The next pass re-admits it once; after three passes at one level it is a `settle_mismatch` hold that needs a person. | The plain sync, then report the hold. || `deletes_withheld` | blocked | The pass's deletes crossed the mass-delete threshold (#2828) or an absolute cap (#7130: `sync.max_deletes`, default 500; `sync.max_delete_ratio`, default 0.01 above 50 deletes; per source `<key>.source.<id>`). The record's `trip` names which. It wrote one withheld record and synced everything else; no per-page hold is written. | Ask the user: restore the checkout if it is wrong; when the deletion is intended, raise the cap for that source (`gbrain config set sync.max_deletes.source.<id> <n>`) or run `GBRAIN_ALLOW_MASS_RECONCILE=1 gbrain sync --source <id>` on the brain host. `gbrain sync --source <id> --dry-run --json` lists the set and its hash. |diff --git a/scripts/stress/iteration-weights.json b/scripts/stress/iteration-weights.jsonindex b958a28a16..4de39e242a 100644--- a/scripts/stress/iteration-weights.json+++ b/scripts/stress/iteration-weights.json@@ -753,6 +753,7 @@"test/sync-legacy-sweep-gbra108.test.ts": 7550,"test/sync-legacy-sweep-resume-7189.test.ts": 61200,"test/sync-markers-6712.test.ts": 1700,+ "test/sync-mirror-catchup-gst2515.test.ts": 35650,"test/sync-no-pull-estimate-gbra133.test.ts": 900,"test/sync-origin-live-wins-6517.test.ts": 2900,"test/sync-origin-markers-6712.test.ts": 4600,diff --git a/src/commands/doctor/checks/git-holds.ts b/src/commands/doctor/checks/git-holds.tsindex 00085d4a59..6e0cb3eba1 100644--- a/src/commands/doctor/checks/git-holds.ts+++ b/src/commands/doctor/checks/git-holds.ts@@ -49,7 +49,7 @@ export async function gitHeldFilesCheck(engine: BrainEngine, sourceIds?: string[const fences = coverage.reduce((sum, source) => sum + coverageRoute(source).fences, 0);const auto = fences ? await readFenceAutoRepair(engine) : undefined;const sources = coverage.map(source => {- const route = coverageRoute(source);+ const route = { ...coverageRoute(source), ...((listing.get(source.source_id) ?? []).some(hold => hold.read_only_checkout) ? { mirror: true } : {}) };const steps = holdRepairSteps(source.source_id, route, auto);return { source_id: source.source_id, held: source.missing + source.stale, stale: source.stale, missing: source.missing, ...(route.fences ? { fences: route.fences } : {}), ...(route.concurrent ? { concurrent: route.concurrent } : {}),...(route.stalled ? { stalled: route.stalled } : {}), ...(route.dirty ? { dirty: route.dirty } : {}), ...(route.drift ? { drift: route.drift } : {}),diff --git a/src/commands/sync.ts b/src/commands/sync.tsindex 0072c2c46c..3266996a00 100644--- a/src/commands/sync.ts+++ b/src/commands/sync.ts@@ -147,7 +147,7 @@ export interface SyncResult {* `holds_fix` (a host-operator relay) only.*/held?: import('../core/persistence/sync-holds.ts').GitHoldItem[];- held_count?: number; held_needs_human?: number;+ held_count?: number; held_needs_human?: number; carried?: import('../core/persistence/sync-holds.ts').CarriedHolds[]; skipped?: { symlink_dangling: { count: number; sample: string[] } };holds_outstanding?: number; holds_released?: number;holds_escalated?: boolean;holds_truncated?: boolean;diff --git a/src/commands/sync/operator.ts b/src/commands/sync/operator.tsindex eb862c9252..8ea760bf04 100644--- a/src/commands/sync/operator.ts+++ b/src/commands/sync/operator.ts@@ -281,11 +281,11 @@ export async function runSyncStatus(engine: BrainEngine, args: string[]): Promis}/** #6712 PR 2 (L5): the one headline per state; the reason names why. */-export function syncStatusHeadline(status: Pick<SyncStatus, 'source_id' | 'state' | 'reason' | 'cursor' | 'run_id' | 'holds' | 'live_job_id'>): string {+export function syncStatusHeadline(status: Pick<SyncStatus, 'source_id' | 'state' | 'reason' | 'cursor' | 'run_id' | 'holds' | 'live_job_id'> & Partial<Pick<SyncStatus, 'carried'>>): string {const c = status.cursor;switch (status.reason) {case 'nothing_pending': return `Source ${status.source_id} is synced${c?.done ? ` (last run ${status.run_id?.slice(0, 8) ?? '?'} finished at ${c.index}/${c.total})` : ''}.`;- case 'held_by_design': return `Source ${status.source_id} is synced; ${status.holds.length} file(s) stay held by design.`;+ case 'held_by_design': return `Source ${status.source_id} is synced; ${status.holds.length} file(s) stay held by design${(status.carried ?? []).map(carried => `; ${carried.count} ${carried.class} (database-only pages this read-only mirror keeps, e.g. ${carried.sample[0]}; list: ${carried.list.join(' ')})`).join('')}.`;case 'held_needs_human': return `Source ${status.source_id} is synced except ${status.holds.length} held file(s), some waiting on a person (see NEEDS HUMAN below); the rest keeps syncing.`;case 'live_job': return `Source ${status.source_id} is resumable: sync job #${status.live_job_id} is draining it.`;case 'cursor_unfinished': return `Source ${status.source_id} is resumable: run ${status.run_id?.slice(0, 8) ?? '?'} stands at ${c?.index ?? '?'}/${c?.total ?? '?'}; the next sync resumes it.`;diff --git a/src/core/commit-notifier.ts b/src/core/commit-notifier.tsindex bef7aaaf6d..7b7900cf46 100644--- a/src/core/commit-notifier.ts+++ b/src/core/commit-notifier.ts@@ -11,7 +11,7 @@* What it watches: managed sources whose worktree this host owns, and classic sources whose last sync ran on this host* (sync_notifier_sources.syncer_host_id) or that GBRAIN_NOTIFIER_CLASSIC_SOURCES binds here; any other source is* skipped with a reason. HEAD is watched while `sync.on_commit` is on and the source is not listed in- * `sync.on_commit.sources` (the exclusion list); wanted markers (sync trigger, webhook, busy delegated syncs) are+ * `sync.on_commit.sources` (the exclusion list) and its `config.syncEnabled` is not false; wanted markers (sync trigger, webhook, busy delegated syncs) are* drained regardless of the flag. It never pulls on its own: a commit-triggered run imports the checkout's HEAD.** Detection: fs.watch on the git dirs (notifier/detect.ts) brings the next poll forward; the async rev-parse poll every@@ -56,6 +56,9 @@ export interface NotifierSource {commonDir?: string | null;lastCommit: string | null;kind?: 'managed' | 'classic';+ /** `config.syncEnabled=false`: HEAD is not watched and no owed run restarts it; an explicit wanted marker runs once, and a failed run waits+ * (operator_blocked, `sync_disabled`) for a new request instead of being retried. */+ syncDisabled?: boolean;}export interface SkippedSource { sourceId: string; reason: string }export type StartResult = { jobId: string } | { busy: true } | { refused: string };@@ -233,7 +236,7 @@ const closeWatch = (rt: Rt) => { if (rt.watchClose) { try { rt.watchClose(); } cconst mode = (rt: Rt): DetectionMode => rt.watchClose ? 'watch' : 'poll';const markerKey = (rt: Rt) => `${rt.head ?? ''}|${rt.markers.map(m => `${m.intent}:${m.generation}`).join(',')}`;// GST-2515: an unwatched source (sync.on_commit off) has no HEAD to compare, so a run it owes (retry, or due after a resume or restart) keeps it due after its marker is gone.-const due = (rt: Rt) => (rt.watched && !!rt.head && rt.head !== rt.lastCommit) || rt.markers.length > 0 || (!rt.watched && (rt.state === 'retry' || rt.state === 'due'));+const due = (rt: Rt) => (rt.watched && !!rt.head && rt.head !== rt.lastCommit) || rt.markers.length > 0 || (!rt.watched && !rt.syncDisabled && (rt.state === 'retry' || rt.state === 'due'));/** Merges a fresh source listing into the runtime map (a moved root starts over); returns the sources new to it. */function mergeListed(sources: Map<string, Rt>, rows: NotifierSource[]): Rt[] {@@ -242,7 +245,7 @@ function mergeListed(sources: Map<string, Rt>, rows: NotifierSource[]): Rt[] {for (const s of rows) {seen.add(s.sourceId);const prior = sources.get(s.sourceId);- if (prior && prior.root === s.root) { prior.gitDir = s.gitDir; prior.commonDir = s.commonDir ?? null; prior.lastCommit = s.lastCommit; prior.kind = s.kind; continue; }+ if (prior && prior.root === s.root) { prior.gitDir = s.gitDir; prior.commonDir = s.commonDir ?? null; prior.lastCommit = s.lastCommit; prior.kind = s.kind; prior.syncDisabled = s.syncDisabled; continue; }if (prior) closeWatch(prior);const rt: Rt = { ...s, head: null, watched: false, watchClose: null, coverageKey: null, lastStartAt: -Infinity, forwarded: null,state: 'idle', attempts: 0, nextRetryAt: 0, blockedKey: null, markers: [], started: false };@@ -366,7 +369,7 @@ export function startCommitNotifier(opts: CommitNotifierOptions): CommitNotifierconst states = await guard('reading the notifier states', () => opts.store.readStates(fresh.map(r => r.sourceId)), [] as NotifierSourceRow[]);restoreStates(sources, states, now());}- for (const rt of fresh) rt.watched = enabled && !excluded.has(rt.sourceId);+ for (const rt of fresh) rt.watched = enabled && !excluded.has(rt.sourceId) && !rt.syncDisabled;for (const rt of sources.values()) {if (stopped) return;if (!rt.watched || forcePoll || !opts.watch) { if (!rt.watched) closeWatch(rt); continue; }@@ -406,6 +409,7 @@ export function startCommitNotifier(opts: CommitNotifierOptions): CommitNotifier};const failure = async (rt: Rt, code: string, message: string, progressed: boolean) => {+ if (rt.syncDisabled) { rt.blockedKey = markerKey(rt); await transition(rt, { state: 'operator_blocked', reason: 'sync_disabled', firstDue: 'keep', progress: progressed, error: { code, message, tracePath: trace(rt, { event: 'failure', code, error: message }) } }); decide(rt.sourceId, 'operator_blocked', { detail: code }); return; }const { attempts, wait, stalled } = retryPlan(rt.attempts, progressed);const tracePath = trace(rt, { event: 'failure', code, error: message, attempts, next_retry_ms: wait, stalled });await transition(rt, { state: 'retry', reason: stalled ? 'notifier_stalled' : code, firstDue: 'keep', attempts, nextRetryInMs: wait, progress: progressed,@@ -526,8 +530,7 @@ export function startCommitNotifier(opts: CommitNotifierOptions): CommitNotifierif (t - listedAt >= LIST_EVERY_MS) { listedAt = t; await refreshSources(); }if (stopped) return;for (const rt of sources.values()) {- const w = enabled && !excluded.has(rt.sourceId);- if (w !== rt.watched) { rt.watched = w; if (w) listedAt = -Infinity; else { closeWatch(rt); if ((rt.state === 'retry' || rt.state === 'due') && !rt.markers.length) await transition(rt, { state: 'idle', reason: null, firstDue: 'clear', attempts: 0, error: null }); } }+ const w = enabled && !excluded.has(rt.sourceId) && !rt.syncDisabled; if (w !== rt.watched) { rt.watched = w; if (w) listedAt = -Infinity; else closeWatch(rt); } else if (!rt.syncDisabled) continue; if (!w && (rt.state === 'retry' || rt.state === 'due') && !rt.markers.length) await transition(rt, { state: 'idle', reason: null, firstDue: 'clear', attempts: 0, error: null });}const wasNudged = nudged;nudged = false;diff --git a/src/core/notifier/resident.ts b/src/core/notifier/resident.tsindex cb798151bb..98cd0f9c38 100644--- a/src/core/notifier/resident.ts+++ b/src/core/notifier/resident.ts@@ -17,6 +17,7 @@ import { gbrainPath } from '../config.ts';import { registerCleanup } from '../process-cleanup.ts';import { SYNC_ON_COMMIT_KEY } from '../sync-config-keys.ts';import { hostIdFor } from '../persistence/identity.ts';+import { isSyncDisabledConfig } from '../sync-policy.ts';import {notifierDisabledByEnv, parseExcludedSources, parseOnCommit, startCommitNotifier,type CommitNotifier, type CommitNotifierOptions, type NotifierSource, type NotifierStore, type ResidentKind, type SkippedSource,@@ -66,8 +67,8 @@ export function classicBinding(env: Record<string, string | undefined> = processexport async function listHostSources(engine: BrainEngine, hostId: string, env: Record<string, string | undefined> = process.env): Promise<{ sources: NotifierSource[]; skipped: SkippedSource[] }> {const { readManagedMode } = await import('../persistence/managed-mode.ts');const managed = await readManagedMode(engine).catch(() => false);- const rows = await engine.executeRaw<{ id: string; local_path: string | null; last_commit: string | null; bound: boolean; owner: string | null; root: string | null; syncer: string | null }>(- `SELECT s.id, s.local_path, s.last_commit, (b.source_id IS NOT NULL) AS bound, w.owner_host_id::text AS owner, h.local_path AS root, n.syncer_host_id AS syncer+ const rows = await engine.executeRaw<{ id: string; local_path: string | null; last_commit: string | null; bound: boolean; owner: string | null; root: string | null; syncer: string | null; config: unknown }>(+ `SELECT s.id, s.local_path, s.last_commit, (b.source_id IS NOT NULL) AS bound, w.owner_host_id::text AS owner, h.local_path AS root, n.syncer_host_id AS syncer, s.configFROM sources sLEFT JOIN persistence_source_bindings b ON b.source_id = s.id AND b.source_incarnation = s.incarnationLEFT JOIN persistence_worktrees w ON w.id = b.worktree_id@@ -103,7 +104,7 @@ export async function listHostSources(engine: BrainEngine, hostId: string, env:if (!dirs.has(root)) dirs.set(root, await resolveGitDirs(root));const d = dirs.get(root) ?? null;if (!d) { skipped.push({ sourceId: row.id, reason: `${root} is not a git checkout on this host` }); continue; }- sources.push({ sourceId: row.id, root, gitDir: d.gitDir, commonDir: d.commonDir, lastCommit: row.last_commit, kind });+ sources.push({ sourceId: row.id, root, gitDir: d.gitDir, commonDir: d.commonDir, lastCommit: row.last_commit, kind, ...(isSyncDisabledConfig(row.config) ? { syncDisabled: true } : {}) });}return { sources, skipped };}diff --git a/src/core/persistence/public-category.ts b/src/core/persistence/public-category.tsindex 683a39b580..0fdb0c1fa7 100644--- a/src/core/persistence/public-category.ts+++ b/src/core/persistence/public-category.ts@@ -33,7 +33,7 @@ export const PUBLIC_BY_TERMINAL: Readonly<Record<TerminalCategory, PublicCategor/** Fault codes (holds, run stops, receipts) by name; a test pins that every `SYNC_FAULT_TABLE` code is here. */export const PUBLIC_BY_CODE: Readonly<Record<string, PublicCategory>> = {- worktree_dirty: 'drift', file_drift: 'drift', file_drift_unknown: 'drift', source_changed: 'drift', pinned_git_worktree_conflict: 'drift',+ worktree_dirty: 'drift', file_drift: 'drift', file_drift_unknown: 'drift', file_drift_mirror_database_only: 'drift', source_changed: 'drift', pinned_git_worktree_conflict: 'drift',concurrent_write: 'conflict', revision_conflict: 'conflict', page_identity_changed: 'conflict', legacy_slug_conflict: 'conflict', origin_conflict: 'conflict', origin_conflict_decision: 'conflict', page_identity_decision: 'conflict',settle_mismatch: 'defect', page_write_failed: 'defect', freeze_page_body_segmented: 'defect', freeze_page_body_rows_unreadable: 'storage',invalid_fence: 'content_refused', invalid_frontmatter: 'content_refused', frontmatter_slug_conflict: 'content_refused', rename_held: 'content_refused',diff --git a/src/core/persistence/sync-blocked.ts b/src/core/persistence/sync-blocked.tsindex dfe89406e1..eead562865 100644--- a/src/core/persistence/sync-blocked.ts+++ b/src/core/persistence/sync-blocked.ts@@ -19,7 +19,7 @@*/import type { BrainEngine } from '../engine.ts';import { readManagedSyncFailures } from './sync-failures.ts';-import { ABANDONED_REPORT_STREAK, passFingerprint, passReadClock, transientEnding } from './sync-pass-row.ts';+import { ABANDONED_REPORT_STREAK, passFingerprint, passReadClock, transientEnding, type PassRow } from './sync-pass-row.ts';import { passFailureKeys } from './sync-cursor-key.ts';import type { ManagedSyncFailure } from './sync-failures.ts';@@ -56,9 +56,11 @@ export async function readBlockedManagedSyncs(engine: BrainEngine, sourceIds?: s.filter(failure => failure.code !== 'sync_incomplete' && cursors.has(failure.cursor_key) && !passSources.has(failure.source_id)).map(failure => ({ source_id: failure.source_id, code: failure.code, run_id: failure.run_id, blocked_since: failure.first_seen })), ...passes, ...passStops];// #7982: passes that keep ending on the same transient stop (status's rule: the newest counts itself).- const { abandonedPasses, openPassRowsStill } = await import('./sync-pass-status.ts');+ const { abandonedPasses, openPassRowsStill, passBanked } = await import('./sync-pass-status.ts');+ // GBRA-542: a pass that stopped after committing work moved the source; it is not listed abandoned.+ const banked = (row: PassRow) => passBanked(engine, row.value.source_id, row.value.incarnation, row.value.telemetry).catch(() => false);for (const { source_id, row } of readings) {- if (transientEnding(row.value.telemetry) && abandonedPasses(row) >= ABANDONED_REPORT_STREAK && !listed.some(blocked => blocked.source_id === source_id)) {+ if (transientEnding(row.value.telemetry) && abandonedPasses(row) >= ABANDONED_REPORT_STREAK && !listed.some(blocked => blocked.source_id === source_id) && !await banked(row)) {listed.push({ source_id, code: 'pass_abandoned', run_id: row.value.telemetry.pass_id, blocked_since: passStuckSince(row) });}}@@ -68,11 +70,12 @@ export async function readBlockedManagedSyncs(engine: BrainEngine, sourceIds?: sconst [{ inspectLocks, syncLockId }, { drainLeaseLive }] = await Promise.all([import('../db-lock.ts'), import('./sync-lease.ts')]);// #7614: one read for every open source's lock. A read that failed decides nothing: the quiet-window rule (status's) still counts a streak.const locks = await inspectLocks(engine, open.map(({ source_id }) => syncLockId(source_id))).catch(() => undefined);- const candidates = open.map(({ source_id, row }) => {+ const candidates = (await Promise.all(open.map(async ({ source_id, row }) => {const lock = locks?.get(syncLockId(source_id));const interrupted = locks === undefined ? undefined : !lock || !drainLeaseLive(lock);- return { source_id, row, interrupted, abandoned: abandonedPasses(row, passReadClock(row), interrupted) >= ABANDONED_REPORT_STREAK };- }).filter(entry => entry.interrupted || entry.abandoned);+ const abandoned = abandonedPasses(row, passReadClock(row), interrupted) >= ABANDONED_REPORT_STREAK && !await banked(row);+ return { source_id, row, interrupted, abandoned };+ }))).filter(entry => entry.interrupted || entry.abandoned);// #7614: a pass that ended (releasing its lease) after the pass read changed its row, so it is not listed; nor is an archived source.// A re-read that failed lists nothing new.const still = candidates.length ? await openPassRowsStill(engine, candidates).catch(() => new Set<string>()) : new Set<string>();diff --git a/src/core/persistence/sync-entry-fault.ts b/src/core/persistence/sync-entry-fault.tsindex 98e586258e..a4b1bdf2bd 100644--- a/src/core/persistence/sync-entry-fault.ts+++ b/src/core/persistence/sync-entry-fault.ts@@ -22,7 +22,7 @@* Deliberate refusals (an `OperationError`) keep their own route; a run where most freezes fault is systemic and* stops instead of holding every file.*/-import { lstatSync } from 'node:fs';+import { lstatSync, readlinkSync, realpathSync } from 'node:fs';import { join } from 'node:path';import { OperationError, opError } from '../ops/contract.ts';import { classifyItemFailure, ITEM_BREAKER_MIN_ITEMS } from '../item-failure.ts';@@ -34,12 +34,47 @@ import { isWriteCapacityWait } from './admission-retry.ts';import { PAGE_BODY_FAULT_CODES } from './sync-fault-class.ts';import { gitBlobOid } from './digest.ts';import { readBlobBytes, type MissingAttributes, type TreeBlob } from './sync-blobs.ts';-import { readSyncFile, syncPartialClone, type SyncEntry } from './sync-discovery.ts';+import { readSyncFile, syncGit, syncPartialClone, type SyncEntry } from './sync-discovery.ts';+import { literalPathspecs } from '../git-env.ts';import type { GitHoldRecord } from './sync-holds.ts';type HoldEntry = Pick<SyncEntry, 'path' | 'sourcePath' | 'working' | 'renameFrom' | 'renameHeld' | 'action'>;type HeldFault = Pick<GitHoldRecord, 'path' | 'source_path' | 'slug' | 'page_id' | 'code' | 'message' | 'upstream_version' | 'meta'>;+/**+ * GBRA-542: whether `path` (relative to `root`) is a symlink whose target does not exist (a dangling link). With+ * `levelOid` (committed mode) the link must also be the committed blob itself, so an uncommitted link never stands in+ * for a committed file.+ */+export function danglingLink(root: string, path: string, levelOid?: string | null): boolean {+ try {+ if (!lstatSync(join(root, path)).isSymbolicLink()) return false;+ if (levelOid !== undefined && gitBlobOid(readlinkSync(join(root, path), 'buffer'), levelOid?.length === 64 ? 'sha256' : 'sha1') !== levelOid) return false;+ } catch { return false; }+ try { realpathSync(join(root, path)); return false; } catch (error) { return (error as NodeJS.ErrnoException).code === 'ENOENT'; }+}++/**+ * GBRA-542: of these Git-root paths, the ones `head`'s tree records as symlinks (mode 120000). A link's target text and a file's+ * bytes hash alike, so only the tree mode says the committed entry is a link; a failed listing names none, so nothing is skipped.+ */+export function committedSymlinks(gitRoot: string, head: string, paths: readonly string[]): Set<string> {+ const links = new Set<string>();+ try {+ for (let start = 0; start < paths.length; start += 500) {+ for (const line of syncGit(gitRoot, ['ls-tree', '-r', '-z', head, '--', ...literalPathspecs(paths.slice(start, start + 500))]).split('\0')) {+ if (line.startsWith('120000 ')) links.add(line.slice(line.indexOf('\t') + 1));+ }+ }+ } catch { return new Set(); }+ return links;+}++/** GBRA-542: the dangling links whose path `head` records as a symlink. */+export function keepCommittedLinks<T extends { entry: { path: string } }>(links: T[], committed: ReadonlySet<string>): T[] {+ return links.filter(link => committed.has(link.entry.path));+}+/** S-39: the `content_null_bytes` refusal of a Markdown text holding U+0000 (the first one's 1-based line), or null. */export function nullBytesRefusal(path: string, content: string): { code: 'content_null_bytes'; line: number; message: string } | null {const at = content.indexOf('\u0000');diff --git a/src/core/persistence/sync-fault-class.ts b/src/core/persistence/sync-fault-class.tsindex a8495f18e1..666f5e85c0 100644--- a/src/core/persistence/sync-fault-class.ts+++ b/src/core/persistence/sync-fault-class.ts@@ -140,6 +140,8 @@ export const SYNC_FAULT_TABLE: ReadonlyArray<SyncFaultRule> = [// #7199: a `file_drift` hold with side `unknown` (no import record): neither side is known to have moved.{ code: 'file_drift_unknown', class: 'page', safe_actions: ['reconcile'], needs_human: true, escalate: 'always: the file and the page differ and no import record says which changed',human_reason: 'The file and the database page differ, and sync has no import record to tell which one changed since the last import, so it held the file instead of overwriting the page; a person chooses which version wins (gbrain repair file-drift --source <source> --slug <slug> --prefer database|file previews it and writes nothing until applied).' },+ // GBRA-542: a deleted-side `file_drift` hold on a read-only mirror (sync-holds.ts mirrorDatabaseOnlyHold): the page stays live and database-only by design.+ { code: 'file_drift_mirror_database_only', class: 'page', safe_actions: ['none'], needs_human: false, escalate: 'never: the page stays live and database-only because gbrain never writes a read-only mirror' },{ code: 'origin_conflict_decision', class: 'page', safe_actions: ['none'], needs_human: true, escalate: 'always: the pages differ, their ids conflict or none is the page the path names',human_reason: 'Several live pages record this file as their origin and sync cannot prove they are one document; a person keeps the page that is the file\'s and soft-deletes the others (gbrain repair origin-twins --source <source> lists them with the commands), then syncs.' },// GBRA-130 A-17/B-3/B-4: a held `page_identity_changed` path (sync-status.ts holdVerdict): several tombstones, a spelling twin or a takeover whose ids differ.diff --git a/src/core/persistence/sync-holds.ts b/src/core/persistence/sync-holds.tsindex 5ced94f934..3ceb00b29d 100644--- a/src/core/persistence/sync-holds.ts+++ b/src/core/persistence/sync-holds.ts@@ -275,6 +275,8 @@ export interface GitHoldRecord {run_id: string | null;mode: 'managed' | 'legacy';meta: GitHoldMeta;+ /** GBRA-542: set by the readers, never stored: the source is a read-only mirror (`mirror_read_only`), so gbrain never writes its files. */+ read_only_checkout?: true;}export interface GitSourceHolds { sourceId: string; incarnation: string; count: number; holds: GitHoldRecord[] }@@ -349,6 +351,19 @@ async function readRow(tx: Exec, sourceId: string, incarnation: string, path: streturn row?.record ?? null;}+/** GBRA-542: the hold record as readers return it, marked when its source is a read-only mirror (`s`: the sources row). */+const READ_HOLD_RECORD_SQL = `CASE WHEN COALESCE(s.config->>'mirror_read_only','false')='true' THEN jsonb_set(h.completed_keys->0,'{read_only_checkout}','true'::jsonb) ELSE h.completed_keys->0 END`;++/**+ * GBRA-542: a deleted-side `file_drift` hold on a read-only mirror whose file Git committed and the upstream removed while the+ * page changed in the database (or has no import record). gbrain never writes the mirror's files, so the page stays live and+ * database-only: the hold is carried by design (class `mirror_database_only`), never re-screened by a plain sync and never a+ * person's to clear. Deleting the page as the upstream did stays the owner's explicit `--prefer file` repair.+ */+export function mirrorDatabaseOnlyHold(record: Pick<GitHoldRecord, 'code' | 'meta' | 'read_only_checkout'>): boolean {+ return record.read_only_checkout === true && record.code === 'file_drift' && record.meta.side === 'deleted' && record.meta.state === 'file_gone' && !record.meta.reason;+}+/*** Records or replaces a hold. Call it inside the transaction that advances the* run past the file. Idempotent on (path, upstream_version); returns `skipped`@@ -520,7 +535,9 @@ export async function recordContentHoldRepair(engine: Exec, input: { sourceId: s}export async function readGitHold(engine: Exec, sourceId: string, incarnation: string, path: string): Promise<GitHoldRecord | null> {- return readRow(engine, sourceId, incarnation, path);+ const [row] = await engine.executeRaw<{ record: GitHoldRecord }>(`SELECT ${READ_HOLD_RECORD_SQL} AS record FROM op_checkpoints h LEFT JOIN sources s ON s.id=$3+ WHERE h.op=$1 AND h.fingerprint=$2`, [GIT_HOLD_OP, gitHoldFingerprint(sourceId, incarnation, path), sourceId]);+ return row?.record ?? null;}/** #5493: unsupported-image holds per source's current incarnation (sources with none are absent); they never escalate. */@@ -545,7 +562,7 @@ export async function countGitHolds(engine: Exec, sourceId: string, incarnation:* status` and `sources retry-held` merge the two.*/export async function readGitSourceHolds(engine: Exec, opts: { sourceIds?: string[]; runId?: string } = {}): Promise<GitSourceHolds[]> {- const rows = await engine.executeRaw<{ record: GitHoldRecord }>(`SELECT h.completed_keys->0 AS record FROM op_checkpoints h+ const rows = await engine.executeRaw<{ record: GitHoldRecord }>(`SELECT ${READ_HOLD_RECORD_SQL} AS record FROM op_checkpoints hJOIN sources s ON s.id=h.completed_keys->0->>'source_id' AND s.incarnation::text=h.completed_keys->0->>'incarnation'WHERE h.op=$1 AND ($2::text[] IS NULL OR s.id=ANY($2::text[])) AND ($3::text IS NULL OR h.completed_keys->0->>'run_id'=$3)ORDER BY s.id,h.fingerprint`, [GIT_HOLD_OP, opts.sourceIds ?? null, opts.runId ?? null]);@@ -569,7 +586,7 @@ export async function readGitHoldListing(engine: Exec, sourceIds: string[], limis.incarnation::text AS incarnation, COALESCE((sm.completed_keys->0->>'count')::int,0) AS count, h.recordFROM sources sJOIN op_checkpoints sm ON sm.op=$1 AND sm.fingerprint=s.id||':'||s.incarnation::text- LEFT JOIN LATERAL (SELECT x.fingerprint, x.completed_keys->0 AS record FROM op_checkpoints x+ LEFT JOIN LATERAL (SELECT x.fingerprint, CASE WHEN COALESCE(s.config->>'mirror_read_only','false')='true' THEN jsonb_set(x.completed_keys->0,'{read_only_checkout}','true'::jsonb) ELSE x.completed_keys->0 END AS record FROM op_checkpoints xWHERE x.op=$2 AND x.completed_keys->0->>'source_id'=s.id AND x.completed_keys->0->>'incarnation'=s.incarnation::textORDER BY x.fingerprint LIMIT $4) h ON trueWHERE s.id=ANY($3::text[]) AND COALESCE((sm.completed_keys->0->>'count')::int,0)>0@@ -664,7 +681,9 @@ export interface HoldRepairRoute { fences: number; others: number; concurrent?:/** #6712 `file_drift` holds: a database edit sync would have overwritten; each routes to `gbrain repair file-drift`. */drift?: number;/** #6377 `frontmatter_slug_conflict` holds, counted apart from `others` by callers that read codes (summary counts leave them in `others`). */- slug_conflicts?: number }+ slug_conflicts?: number;+ /** GBRA-542: the source is a read-only mirror, so its drift holds are carried and never resolved toward the database. */+ mirror?: boolean }/** `gbrain repair content --source <id> [--only <path>]`: the content-repair lane's read-only preview (#6377; no model call). */export function contentRepairArgv(sourceId: string, path?: string): string[] {@@ -701,7 +720,8 @@ export function holdRepairSteps(sourceId: string, route: HoldRepairRoute, auto?:const unblock = ['gbrain', 'sync', 'unblock', '--source', sourceId, '--apply'];const dirtyText = `the ${route.dirty ?? 0} worktree-dirty hold(s) are uncommitted edits in the checkout that sync would not overwrite: commit them (gbrain sources status ${sourceId} names each file), `+ `then run ${unblock.join(' ')}, which re-screens every held file that is now committed, and the same sync`;- const driftText = `for each file-drift hold, preview the resolution with gbrain repair file-drift --source ${sourceId} --slug <slug> --prefer database (or --prefer file) `+ const driftText = route.mirror ? `the file-drift holds are on a read-only mirror, which gbrain never writes, so their pages stay live and database-only: list them with ${heldDriftPreviewArgv(sourceId, 'deleted', 'file_gone', 'file').join(' ')} (a preview that writes nothing), and turn the mirror flag off only if gbrain should write the checkout`+ : `for each file-drift hold, preview the resolution with gbrain repair file-drift --source ${sourceId} --slug <slug> --prefer database (or --prefer file) `+ `(gbrain sources status ${sourceId} names each slug and the database write sync did not overwrite) and decide with the user. ${FILE_DRIFT_APPLY_STEP} Then run gbrain sync --source ${sourceId} --no-pull`;if (route.stalled && !route.others && !content && !route.concurrent && !route.dirty && !route.drift) {return { argv: stalled.status, commands: [stalled.status.join(' '), stalled.retry.join(' '), stalled.sync.join(' ')], text: stalledText };@@ -737,6 +757,17 @@ export function fileDriftActionVerb(side: FileDriftSide, state: FileDriftDeletedreturn prefer === 'database' ? 'recreate the file' : 'soft-delete the page';}+/** GBRA-542: one class of holds carried by design, with a sample of its paths and the read-only command that lists them. */+export interface CarriedHolds { class: 'mirror_database_only'; count: number; sample: string[]; list: string[] }+export const CARRIED_SAMPLE = 5;++/** GBRA-542: the carried classes among one source's holds (empty when there are none). */+export function carriedHolds(records: ReadonlyArray<Pick<GitHoldRecord, 'source_id' | 'path' | 'code' | 'meta' | 'read_only_checkout'>>): CarriedHolds[] {+ const mirror = records.filter(mirrorDatabaseOnlyHold);+ return mirror.length ? [{ class: 'mirror_database_only', count: mirror.length, sample: mirror.slice(0, CARRIED_SAMPLE).map(record => record.path),+ list: heldDriftPreviewArgv(mirror[0]!.source_id, 'deleted', 'file_gone', 'file') }] : [];+}+/** #6712 PR 4b: the bulk preview of the held `file_drift` pages of one side (and deleted state), narrowed to `path` when given. */export function heldDriftPreviewArgv(sourceId: string, side: FileDriftSide, state: FileDriftDeletedState | undefined, prefer: 'file' | 'database', path?: string): string[] {return ['gbrain', 'repair', 'file-drift', '--source', sourceId, '--held', '--side', side, ...(side === 'deleted' && state ? ['--state', state] : []),@@ -749,7 +780,7 @@ export function movedPagesPreviewArgv(sourceId: string, slug?: string | null): s}/** #6712 PR 4b: the Decision for a `file_drift` hold whose one side is a deletion; each option names its action verb. */-function deletedDriftFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'meta'> & Partial<Pick<GitHoldRecord, 'slug'>>, state: FileDriftDeletedState): Action {+function deletedDriftFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'meta'> & Partial<Pick<GitHoldRecord, 'slug' | 'read_only_checkout'>>, state: FileDriftDeletedState): Action {const source = record.source_id;const slug = record.slug ?? '<slug>';const revision = record.meta.page_revision ? ` (revision ${record.meta.page_revision})` : '';@@ -762,6 +793,12 @@ function deletedDriftFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'metwhy: `Decides ${record.path} again under the fixed gbrain.yml: a database-only page is kept, any other deletion follows its usual proof.` },verify };}+ // GBRA-542: on a read-only mirror gbrain never rewrites a file, so no fix names --prefer database there.+ if (mirrorDatabaseOnlyHold({ ...record, code: 'file_drift' })) {+ return { argv: preview('file'), consent: [], actor: 'user', requires_exclusive: false, docs: 'docs/guides/repair.md#deleted-canonical-files',+ why: `${record.path} is gone from the read-only mirror's last commit, and page ${slug}${revision} changed in the database or has no import record, so sync keeps the page. gbrain never writes this mirror's files, so the page stays live and database-only: the hold is carried by design (mirror_database_only), never re-screened by a plain sync, and needs nothing. Only if the owner wants the page gone as the upstream did, this preview (writes nothing) shows what --prefer file deletes; its apply needs the user's consent.`,+ verify };+ }const uncommitted = state === 'file_gone' && (record.meta.reason === 'never_committed' || record.meta.reason === 'ignored');const what = state === 'file_gone' && record.meta.reason === 'origin_unknown' ? `${record.path} is gone, and Git could not show whether a commit held it (its history read did not finish), so sync kept page ${slug} instead of deleting it (#7130). The --held preview reads the whole history and names the file's provenance. `: uncommitted ? `${record.path} is gone, but Git never committed it${record.meta.reason === 'ignored' ? ' (.gitignore excludes it)' : ''}, so its absence is no evidence of a deletion and sync kept page ${slug} instead of deleting it (#7130). Put the file back and commit it (un-ignore it first) to keep the page file-backed. `@@ -771,9 +808,11 @@ function deletedDriftFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'met: `${record.path} is gone from the last commit but page ${slug} changed${revision}, so sync held the deletion instead of removing a page that changed. `;// GST-2584: a file that moved left its page behind; the --held preview lists it moved_to and names the fold.const moved = state === 'file_gone' ? `If the file moved, the preview lists the page moved_to <slug> and names ${shellQuote(movedPagesPreviewArgv(source, record.slug))}, which folds it into the moved page. ` : '';- const options = `${moved}This preview (writes nothing) shows --prefer database: ${fileDriftActionVerb('deleted', state, 'database')}; ${preview('file').join(' ')} previews --prefer file: `+ const readOnly = record.read_only_checkout === true;+ const options = readOnly ? `${moved}The source is a read-only mirror, so gbrain never rewrites its files and keeping the database copy needs no repair; this preview (writes nothing) shows --prefer file: ${fileDriftActionVerb('deleted', state, 'file')}. `+ : `${moved}This preview (writes nothing) shows --prefer database: ${fileDriftActionVerb('deleted', state, 'database')}; ${preview('file').join(' ')} previews --prefer file: `+ `${fileDriftActionVerb('deleted', state, 'file')}. `;- return { argv: preview('database'), consent: [], actor: 'agent', requires_exclusive: false, docs: 'docs/guides/repair.md#deleted-canonical-files',+ return { argv: preview(readOnly ? 'file' : 'database'), consent: [], actor: 'agent', requires_exclusive: false, docs: 'docs/guides/repair.md#deleted-canonical-files',why: `${what}${options}Decide with the user. ${FILE_DRIFT_APPLY_STEP} Then sync again. The rest of the source is not blocked.`,then: { argv: ['gbrain', 'sync', '--source', source, '--no-pull'], consent: [], actor: 'agent', requires_exclusive: false, verify,why: `Re-screens ${record.path} after the repair (a changed file or page revision re-screens it by itself).` },@@ -795,7 +834,7 @@ export const FILE_DRIFT_APPLY_STEP = 'The preview writes nothing and names what* preview, then the apply, the edit or the paid setting). `auto` is what the* caller read about the maintenance run (undefined: unknown).*/-export function gitHoldFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'code' | 'meta'> & Partial<Pick<GitHoldRecord, 'slug' | 'page_id'>>, auto?: FenceAutoRepair): Action {+export function gitHoldFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'code' | 'meta'> & Partial<Pick<GitHoldRecord, 'slug' | 'page_id' | 'read_only_checkout'>>, auto?: FenceAutoRepair): Action {const source = record.source_id;const repair = (ambiguous: boolean, why: string): Action => ({ argv: ['gbrain', 'repair', 'frontmatter', '--source', source, ...(ambiguous ? ['--include-ambiguous'] : [])],consent: [], actor: 'agent', requires_exclusive: false, why,@@ -881,9 +920,10 @@ export function gitHoldFix(record: Pick<GitHoldRecord, 'source_id' | 'path' | 'c: side === 'both' ? 'the file changed too' : 'no import record says whether the file changed too'}). `;const decide = `Decide with the user which version wins. ${FILE_DRIFT_APPLY_STEP} Then sync again. The rest of the source is not blocked.`;// #6517: a shared-origin delete kept the file for this surviving page, and its receipt names importing the file.- const shared = record.meta.shared_origin === true;- return { argv: preview(shared ? 'file' : 'database'), consent: [], actor: 'agent', requires_exclusive: false, docs: 'docs/guides/write-refusals.md#file_drift',- why: shared ? `${held}Page ${slug} survived a shared-origin delete that kept ${record.path} for it. This preview (writes nothing) imports the file into the page, as the delete's receipt says; `+ const shared = record.meta.shared_origin === true, readOnly = record.read_only_checkout === true;+ return { argv: preview(shared || readOnly ? 'file' : 'database'), consent: [], actor: 'agent', requires_exclusive: false, docs: 'docs/guides/write-refusals.md#file_drift',+ why: readOnly ? `${held}The source is a read-only mirror, so gbrain never rewrites its files: keeping the database copy needs no repair, and this preview (writes nothing) imports the file into the page instead. ${decide}`+ : shared ? `${held}Page ${slug} survived a shared-origin delete that kept ${record.path} for it. This preview (writes nothing) imports the file into the page, as the delete's receipt says; `+ `${preview('database').join(' ')} previews keeping the database copy and rewriting the file instead. ${decide}`: `${held}This preview (writes nothing) keeps the database copy and rewrites the file; ${preview('file').join(' ')} previews importing the file instead. ${decide}`,then: { argv: ['gbrain', 'sync', '--source', source, '--no-pull'], consent: [], actor: 'agent', requires_exclusive: false, verify,@@ -1383,12 +1423,13 @@ export async function buildHoldReport(engine: Exec, input: { sourceId: string; ipolicy: SyncHoldPolicy | Promise<SyncHoldPolicy>; screened: number; pendingScreen?: boolean; needsHuman?: (record: GitHoldRecord, auto: FenceAutoRepair | undefined) => boolean }): Promise<Pick<import('../../commands/sync.ts').SyncResult,'held' | 'held_count' | 'held_needs_human' | 'holds_outstanding' | 'holds_escalated' | 'holds_truncated' | 'holds_pending_screen' | 'holds_fix'>> {// #6712 PR 4b: the summary, this run's holds and the policy (a caller may pass its read in flight) in one round trip.- const [[summary], sourceHolds, policy] = await Promise.all([engine.executeRaw<{ count: number | string; images: number | string; fences: number | string; stalled: number | string; concurrent: number | string; dirty: number | string; drift: number | string }>(`SELECT COALESCE((completed_keys->0->>'count')::int,0) AS count,+ const [[summary], sourceHolds, policy, [source]] = await Promise.all([engine.executeRaw<{ count: number | string; images: number | string; fences: number | string; stalled: number | string; concurrent: number | string; dirty: number | string; drift: number | string }>(`SELECT COALESCE((completed_keys->0->>'count')::int,0) AS count,COALESCE((completed_keys->0->>'drift')::int,0) AS drift,COALESCE((completed_keys->0->>'images')::int,0) AS images, COALESCE((completed_keys->0->>'fences')::int,0) AS fences, COALESCE((completed_keys->0->>'stalled')::int,0) AS stalled,COALESCE((completed_keys->0->>'concurrent')::int,0) AS concurrent, COALESCE((completed_keys->0->>'dirty')::int,0) AS dirtyFROM op_checkpoints WHERE op=$1 AND fingerprint=$2`,- [GIT_HOLD_SUMMARY_OP, summaryFingerprint(input.sourceId, input.incarnation)]), readGitSourceHolds(engine, { sourceIds: [input.sourceId], runId: input.runId }), input.policy]);+ [GIT_HOLD_SUMMARY_OP, summaryFingerprint(input.sourceId, input.incarnation)]), readGitSourceHolds(engine, { sourceIds: [input.sourceId], runId: input.runId }), input.policy,+ engine.executeRaw<{ mirror: string | null }>("SELECT config->>'mirror_read_only' AS mirror FROM sources WHERE id=$1", [input.sourceId])]);const outstanding = Number(summary?.count ?? 0);const runHolds = sourceHolds[0]?.holds ?? [];if (!outstanding && !runHolds.length) return {};@@ -1405,7 +1446,7 @@ export async function buildHoldReport(engine: Exec, input: { sourceId: string; i// #7199: this run's holds whose verdict waits on a person (sync-status.ts holdVerdict).const needsHuman = input.needsHuman ? runHolds.filter(hold => input.needsHuman!(hold, auto)).length : 0;const steps = holdRepairSteps(input.sourceId, { fences, others: Math.max(0, outstanding - fences - stalled - concurrent - dirty - drift - slugConflicts), ...(stalled ? { stalled } : {}),- ...(concurrent ? { concurrent } : {}), ...(dirty ? { dirty } : {}), ...(drift ? { drift } : {}), ...(slugConflicts ? { slug_conflicts: slugConflicts } : {}) }, auto);+ ...(concurrent ? { concurrent } : {}), ...(dirty ? { dirty } : {}), ...(drift ? { drift } : {}), ...(slugConflicts ? { slug_conflicts: slugConflicts } : {}), ...(source?.mirror === 'true' ? { mirror: true } : {}) }, auto);// #6278: the escalation text is routed by code: stalled holds point at the writer, not at a generator.const runStalled = runHolds.filter(hold => stalledWeight(hold)).length;const escalation = runStalled * 2 > runHolds.length@@ -1427,7 +1468,7 @@ export async function buildHoldReport(engine: Exec, input: { sourceId: string; i/** The hold fields as the sync JSON envelopes carry them (snake_case already). */export function syncHoldJsonFields(result: object): Record<string, unknown> {const fields = result as Record<string, unknown>;- return Object.fromEntries(['held', 'held_count', 'held_needs_human', 'holds_outstanding', 'holds_released', 'holds_escalated', 'holds_truncated', 'holds_pending_screen', 'holds_fix',+ return Object.fromEntries(['held', 'held_count', 'held_needs_human', 'carried', 'skipped', 'holds_outstanding', 'holds_released', 'holds_escalated', 'holds_truncated', 'holds_pending_screen', 'holds_fix','converted_from_failed', 'recovered_frontmatter', 'fences_normalized', 'fence_issues', 'dry_run', 'would_hold', 'would_hold_count', 'would_normalize', 'would_normalize_count', 'screen_skipped', 'breaker', 'slug_lines_removed'].filter(key => fields[key] !== undefined).map(key => [key, fields[key]]));}diff --git a/src/core/persistence/sync-pass-apply.ts b/src/core/persistence/sync-pass-apply.tsindex 9560edf3c1..e1978e654e 100644--- a/src/core/persistence/sync-pass-apply.ts+++ b/src/core/persistence/sync-pass-apply.ts@@ -30,6 +30,7 @@** A remote caller (`remote: true`) writes nothing here: no hold, marker, record, withheld record or keep_db write.*/+import { committedSymlinks, danglingLink, keepCommittedLinks } from './sync-entry-fault.ts';import { VERSION } from '../../version.ts';import type { BrainEngine } from '../engine.ts';import { resolveSlugForPath } from '../sync.ts';@@ -46,7 +47,7 @@ import { sha256 } from './digest.ts';import { readBlobs } from './sync-git-stream.ts';import { faultPoint } from './fault-points.ts';import { RECOVERY_VERSION } from '../markdown.ts';-import { readSyncImportProvenances, recordSyncImportProvenances, syncImportParserFor, writeGitHold,+import { clearGitHold, GIT_HOLD_OP, gitHoldFingerprint, readSyncImportProvenances, recordSyncImportProvenances, syncImportParserFor, writeGitHold,type GitHoldCode, type GitHoldMeta, type SyncImportProvenance } from './sync-holds.ts';import { assertDrainLease, inFencedSlices } from './sync-lease.ts';import { clearDeletesWithheld, deleteSetHash, readDeletesWithheld, readOriginDeletedMarker, retireOriginDeletedMarker, writeDeletesWithheld, type DeletesWithheld } from './sync-markers.ts';@@ -102,11 +103,13 @@ export interface PassApplyState {recorded: Set<number>;/** GBRA-133: the held committed deletions `sync.apply_held_deletions` screened this pass (sync-held-deletions.ts). */heldDeletions: HeldDeletionState;+ /** GBRA-542: dangling symlinks this pass skipped (no live page at their path), and their file_unreadable holds it released. */+ dangling: { skipped: Set<string>; released: number };}export function newPassApplyState(): PassApplyState {return { markersRetired: 0, holds: { file_gone: 0, tombstone_edited: 0, purged_differs: 0, settle_mismatch: 0, origin_conflict: 0, page_identity_changed: 0, uncommitted: 0 }, settleMismatch: [], adopted: 0,- swept: { screened: 0, recorded: 0, paths: new Set() }, sweptAll: false, withheld: null, withheldCount: 0, keepDb: { admitted: 0, pending: 0, paused: 0, refused: {} }, keepDbTaken: [], originHeld: new Set(), matchedSlugs: new Set(), classes: {}, summary: null, recorded: new Set(), heldDeletions: newHeldDeletionState() };+ swept: { screened: 0, recorded: 0, paths: new Set() }, sweptAll: false, withheld: null, withheldCount: 0, keepDb: { admitted: 0, pending: 0, paused: 0, refused: {} }, keepDbTaken: [], originHeld: new Set(), matchedSlugs: new Set(), classes: {}, summary: null, recorded: new Set(), heldDeletions: newHeldDeletionState(), dangling: { skipped: new Set(), released: 0 } };}export interface PassApplyInput {@@ -165,6 +168,7 @@ export async function applyPassRound(engine: BrainEngine, level: ShadowLevel, insummarizeRound(level, state);return;}+ await skipDanglingLinks(engine, level, input, state);await writePassHolds(engine, level, input, state);await adoptRecords(engine, level, state);// #7338: a round with imports or deletes sweeps after its batches drain (the pass calls sweepLegacy), so the bounded@@ -268,6 +272,41 @@ async function applyDeleteValve(engine: BrainEngine, level: ShadowLevel, input:state.withheld = null;}+/**+ * GBRA-542: a dangling symlink has nothing to import and nothing for a person to decide, so a path that is one is skipped+ * (settled, counted, no hold) unless a live page sits at that path (its origin, or its slug), whose only copy it may have+ * replaced; that one still holds `file_unreadable` at the freeze. A held dangling link with no live page is released (the+ * hold row only; no page is touched). Each pass checks the link again, so one whose target appears is imported.+ */+async function skipDanglingLinks(engine: BrainEngine, level: ShadowLevel, input: PassApplyInput, state: PassApplyState): Promise<void> {+ const step = loopYield();+ let links: Array<{ entry: ShadowPath; path: string; slug: string | null }> = [];+ for (const entry of level.paths) {+ if (step.due()) await step.pause();+ const r = entry.result, held = entry.state.hold !== null;+ if (r.inFlight || entry.state.page?.live || entry.state.level === null || (!held && (r.settled || (r.action !== 'import' && r.action !== 'hold')))) continue;+ if (!danglingLink(level.gitRoot, entry.path, level.mode === 'committed' ? entry.state.level : undefined)) continue;+ const { path, sourcePath } = passEntryPaths(level, entry.path);+ links.push({ entry, path, slug: resolveSlugForPath(sourcePath) });+ }+ if (level.mode === 'committed' && links.length) links = keepCommittedLinks(links, committedSymlinks(level.gitRoot, level.head, links.map(link => link.entry.path)));+ if (!links.length) return;+ const slugs = [...new Set(links.flatMap(link => link.slug ? [link.slug] : []))];+ const live = new Set(slugs.length ? (await engine.executeRaw<{ slug: string }>('SELECT slug FROM pages WHERE source_id=$1 AND deleted_at IS NULL AND slug=ANY($2::text[])',+ [level.sourceId, slugs])).map(row => row.slug) : []);+ const skipped = links.filter(link => !link.slug || !live.has(link.slug));+ const holds = new Set((await engine.executeRaw<{ path: string }>(`SELECT completed_keys->0->>'path' AS path FROM op_checkpoints WHERE op=$1 AND fingerprint=ANY($2::text[])+ AND completed_keys->0->>'code'='file_unreadable'`, [GIT_HOLD_OP, skipped.filter(link => link.entry.state.hold).map(link => gitHoldFingerprint(level.sourceId, level.incarnation, link.path))])).map(row => row.path));+ const settled = skipped.filter(link => !link.entry.state.hold || holds.has(link.path));+ await inFencedSlices(engine, level.sourceId, settled.filter(link => holds.has(link.path)), async (tx, link) => {+ if (await clearGitHold(tx, { sourceId: level.sourceId, incarnation: level.incarnation, path: link.path, observedAt: input.observedAt })) state.dangling.released++;+ });+ for (const link of settled) {+ settleAs(link.entry, { settled: true, action: 'none', classAction: 'none' });+ state.dangling.skipped.add(link.path);+ }+}+/** Rows 6 and 9 and the settle-mismatch streak (§3.1): holds at level `L`, written under the drain lease. */async function writePassHolds(engine: BrainEngine, level: ShadowLevel, input: PassApplyInput, state: PassApplyState): Promise<void> {const pending: PendingHold[] = [];@@ -568,5 +607,6 @@ export function passApplyCounts(state: PassApplyState): Record<string, number> {return { markers_retired: state.markersRetired, holds_file_gone: state.holds.file_gone, holds_tombstone_edited: state.holds.tombstone_edited,holds_purged_differs: state.holds.purged_differs, holds_settle_mismatch: state.holds.settle_mismatch, holds_origin_conflict: state.holds.origin_conflict, holds_page_identity_changed: state.holds.page_identity_changed, holds_uncommitted: state.holds.uncommitted, adopted: state.adopted, legacy_swept: state.swept.screened,legacy_recorded: state.swept.recorded, deletes_withheld: state.withheldCount, keep_db_admitted: state.keepDb.admitted, keep_db_pending: state.keepDb.pending,- paused_keep_db: state.keepDb.paused, keep_db_refused: refused, held_deletions_applied: state.heldDeletions.applied, held_deletions_refused: state.heldDeletions.refused };+ paused_keep_db: state.keepDb.paused, keep_db_refused: refused, held_deletions_applied: state.heldDeletions.applied, held_deletions_refused: state.heldDeletions.refused,+ skipped_symlink_dangling: state.dangling.skipped.size, released_symlink_dangling: state.dangling.released };}diff --git a/src/core/persistence/sync-pass-run.ts b/src/core/persistence/sync-pass-run.tsindex db1d6a4c17..f66702f0ce 100644--- a/src/core/persistence/sync-pass-run.ts+++ b/src/core/persistence/sync-pass-run.ts@@ -37,13 +37,14 @@ import { openPassRequests, settleCarried } from './sync-pass-carried.ts';import { loopYield } from './sync-yield.ts';import { pathBlobHistories, recallWorkingStatCache, rememberWorkingStatCache, SyncTreeReadIncomplete } from './sync-git-stream.ts';import { syncPassKind } from './sync-classify.ts';+import { passBanked } from './sync-pass-status.ts';import { PASS_BATCH_KEY_PREFIX, readConvergedWatermark, SyncAdmissionPaused, type PassEntry } from './sync-pass-batch.ts';import { assertDrainLease, currentDrainLease } from './sync-lease.ts';import { idleClear, readIdleFacts } from './sync-idle-skips.ts';import { digest } from './digest.ts';import { readSyncTail } from './sync-tail-record.ts';import { movedDriftHolds } from './sync-prescreen.ts';-import { clearGitHold, clearGitHoldRetryPaths, holdRescreenDue, readGitHoldRetryPaths, readSyncHoldPolicy, readGitSourceHolds, readSyncImportProvenances, settledDeletionHolds, syncImportParserFor } from './sync-holds.ts';+import { CARRIED_SAMPLE, carriedHolds, clearGitHold, clearGitHoldRetryPaths, holdRescreenDue, readGitHoldRetryPaths, readSyncHoldPolicy, readGitSourceHolds, readSyncImportProvenances, settledDeletionHolds, syncImportParserFor } from './sync-holds.ts';import { loadActivePackForEngine } from '../schema-pack/engine-resolution.ts';import { loadConfig } from '../config.ts';import { startPersistenceConsumer } from './service.ts';@@ -83,7 +84,9 @@ export interface SyncPassInput {processingOptions: SyncProcessingOptions; syncOptions: SyncCursorOptions; assertActive: () => void; signal?: AbortSignal;state: { sourceId?: string; incarnation?: string; remote?: boolean }; runStartedAt: string;/** Drains one round batch through sync-run.ts's cursor loop: the batch back when it drained, or the result it stopped with. */- drainBatch: (key: string, batch: Cursor, sliceBase: { started: number; pages: number }) => Promise<SyncResult | Cursor>;+ /** `sliceBase.committed`: called after each entry the batch commits, with the batch cursor's index; GBRA-542: the pass refreshes its+ * open row with it on the heartbeat (`onlyOpen`, so a write landing after the batch's folded stamp ended the row is dropped). */+ drainBatch: (key: string, batch: Cursor, sliceBase: { started: number; pages: number; committed?: (index: number) => void }) => Promise<SyncResult | Cursor>;/** A round's level read starts at `target` (the HEAD it was pinned or last read at): a failure until its first batch is that read's. */onLevelRead: (target: string) => void;}@@ -416,7 +419,8 @@ async function pinPass(engine: BrainEngine, input: SyncPassInput, lease: NonNullconst obligations: ManagedSyncPassValue['obligations'] = prior ? structuredClone(prior.value.obligations): { baseline_commit: context.source.last_commit, mismatch: {}, not_converged_streak: 0, faults: {} };obligations.mismatch ??= {}; obligations.faults ??= {}; obligations.not_converged_streak ??= 0;- const abandoned = prior && (prior.value.telemetry.outcome === 'open' || transientEnding(prior.value.telemetry)) ? (obligations.abandoned_streak ?? 0) + 1 : 0;+ const abandoned = prior && (prior.value.telemetry.outcome === 'open' || transientEnding(prior.value.telemetry))+ && !await passBanked(engine, sourceId, incarnation, prior.value.telemetry).catch(() => false) ? (obligations.abandoned_streak ?? 0) + 1 : 0;if (abandoned) obligations.abandoned_streak = abandoned; else delete obligations.abandoned_streak;if (handover.episode && !obligations.episode) obligations.episode = handover.episode;const passId = randomUUID();@@ -474,15 +478,16 @@ async function pinPass(engine: BrainEngine, input: SyncPassInput, lease: NonNull* drain's governor it is working and, every PASS_HEARTBEAT_MS, saves the open row (what a follower and `sync status` read);* `pending` is that save, which the pass's next save waits for so an older row never lands over a newer one.*/-function levelBeat(opts: SyncOpts, sliceBase: { pages: number }, save: () => Promise<void>): { report: () => void; readonly pending: Promise<void> | null } {+function levelBeat(opts: SyncOpts, sliceBase: { pages: number }, save: () => Promise<void>): { report: () => void; tick: (write?: () => Promise<void>) => void; readonly pending: Promise<void> | null } {let at = Date.now(), pending: Promise<void> | null = null;+ const tick = (write = save) => {+ if (Date.now() - at < PASS_HEARTBEAT_MS || pending) return;+ at = Date.now();+ pending = write().catch(() => undefined).finally(() => { pending = null; });+ };return {- report: () => {- opts.onProgress?.({ phase: 'managed_sync.level_progress', bankedFiles: sliceBase.pages });- if (Date.now() - at < PASS_HEARTBEAT_MS || pending) return;- at = Date.now();- pending = save().catch(() => undefined).finally(() => { pending = null; });- },+ report: () => { opts.onProgress?.({ phase: 'managed_sync.level_progress', bankedFiles: sliceBase.pages }); tick(); },+ tick,get pending() { return pending; },};}@@ -518,7 +523,13 @@ async function stampIdlePass(engine: BrainEngine, input: SyncPassInput, lease: N// Nor can a tail be owed since the pin read none: the lease's holder reads none after it (sync-run.ts).lease.tailClear = !tail;lease.idleClear = idleClear({ tail: !!tail, holds: holds.length, ...idleFacts });- return { ...emptyPassResult(context, remote, 'up_to_date'), toCommit: remote ? '' : head, ...(inherited.length ? { inherited_options: inherited } : {}) };+ return { ...emptyPassResult(context, remote, 'up_to_date'), toCommit: remote ? '' : head, ...(inherited.length ? { inherited_options: inherited } : {}), ...passCarriedFields(holds) };+}++/** GBRA-542: the result's `carried` (holds kept by design, by class) and `skipped` (dangling links the pass did not hold) fields. */+function passCarriedFields(holds: Parameters<typeof carriedHolds>[0], dangling: ReadonlySet<string> = new Set()): Pick<SyncResult, 'carried' | 'skipped'> {+ const carried = carriedHolds(holds);+ return { ...(carried.length ? { carried } : {}), ...(dangling.size ? { skipped: { symlink_dangling: { count: dangling.size, sample: [...dangling].slice(0, CARRIED_SAMPLE) } } } : {}) };}/**@@ -728,7 +739,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Prconst started = Date.now(), startedAt = new Date(started).toISOString();const pinned = await pinPass(engine, input, lease, started);if (pinned.ended) return pinned.ended;- const { prior, admission, tail, retry, failed, mode, selection, configured, selectionFp, parserFp, obligations, passId, optOutEpisode,+ const { prior, admission, tail, retry, failed, holds, mode, selection, configured, selectionFp, parserFp, obligations, passId, optOutEpisode,processingOptions, inherited, pack, converged, cacheKey, working, kind, value, markersOwed, heldDeletions, levelReuse } = pinned;let { head, episode } = pinned;@@ -756,9 +767,9 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Prconst settledKey = (path: string, level: string | null) => `${path}\0${level ?? ''}`;// The entries this round's batches settled before the round folds its candidates into S: a slice that stops mid-round// reports them (yieldSettled); a blocked stop reports S as it replays.- let roundSettled = 0, yieldSettled = 0;+ let roundSettled = 0, yieldSettled = 0, inBatch = 0;const retryConsumed = new Set<string>(retry), consumeRetry = retryConsumer(engine, sourceId, incarnation, remote ? [] : retry);- const sliceBase = { started: performance.now(), pages: 0 };+ const sliceBase = { started: performance.now(), pages: 0, committed: (index: number) => { inBatch = index; beat.tick(() => save('open', null, false, true, true)); } };let batchNo = 0, waits = 0, candidatesFirst = 0;let lastMismatch: ShadowPath[] = [];let lastLevel: ShadowLevel | null = null;@@ -778,7 +789,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Prconst open = await openPassRequests(engine, { sourceId, passId, carried: value.obligations.pending_requests ?? [] });if (open.length) value.obligations.pending_requests = open; else delete value.obligations.pending_requests;}- value.telemetry = { ...value.telemetry, outcome, stop_reason: stopReason, settled: S.size + yieldSettled, counts: { ...counts, ...passApplyCounts(applied) },+ value.telemetry = { ...value.telemetry, outcome, stop_reason: stopReason, settled: S.size + (yieldSettled || roundSettled + inBatch), counts: { ...counts, ...passApplyCounts(applied) },classes: applied.classes, ...(applied.summary ? { summary: applied.summary } : {}),...(ended ? { ended_at: new Date().toISOString(), duration_ms: Date.now() - started } : {}) };if (outcome === 'blocked' || stopReason === 'not_converged') value.obligations.stuck_since ??= new Date().toISOString();@@ -794,7 +805,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): PrfilesImported: sliceBase.pages, bankedFiles: sliceBase.pages, waived: { imports: counts.waivedImports, deletes: counts.waivedDeletes }, prescreened: 0,...(counts.skippedContextualMode || counts.skippedCanonicalBytes ? { legacySkips: { contextualMode: counts.skippedContextualMode, canonicalBytes: counts.skippedCanonicalBytes } } : {}),...(collisions.length ? { slugCollisions: collisions } : {}), ...(fileRefusals.size ? { fileRefusals: [...fileRefusals.values()] } : {}), ...(identityDowngrades.length ? { identityDowngrades } : {}), ...(inherited.length ? { inherited_options: inherited } : {}), ...(reason ? { reason } : {}),- ...(sizeStates ? { size_states: sizeStates } : {}), ...(keptCount() ? { working_tree_kept: keptCount() } : {}),+ ...(sizeStates ? { size_states: sizeStates } : {}), ...(keptCount() ? { working_tree_kept: keptCount() } : {}), ...passCarriedFields(holds, applied.dangling.skipped),pass: { pass_id: passId, kind, round: value.telemetry.round, outcome: value.telemetry.outcome, stop_reason: value.telemetry.stop_reason, candidates: candidatesFirst,settled: S.size + yieldSettled, remaining: value.telemetry.remaining, not_converged_streak: value.obligations.not_converged_streak } });// A caller abort records the stop on the pass row (the pin it rolled back is replaced).@@ -812,7 +823,7 @@ export async function runSyncPass(engine: BrainEngine, input: SyncPassInput): Prconst key = `${PASS_BATCH_KEY_PREFIX}${passId}:${batchNo++}`;for (const entry of entries) if (entry.pageId != null) applied.recorded.add(entry.pageId);const batch = batchCursor(entries, stamp);- const drained = await input.drainBatch(key, batch, sliceBase);+ const drained = await input.drainBatch(key, batch, sliceBase).finally(() => { inBatch = 0; });if ('entries' in drained) {const c = drained.counts;batchFolded = c.folded === true;diff --git a/src/core/persistence/sync-pass-status.ts b/src/core/persistence/sync-pass-status.tsindex c6196d133e..e7f48113cd 100644--- a/src/core/persistence/sync-pass-status.ts+++ b/src/core/persistence/sync-pass-status.ts@@ -8,7 +8,7 @@ import type { BrainEngine } from '../engine.ts';import type { Action } from '../agent-output.ts';import type { SyncAdmissionStatus } from './sync-admission.ts';import { digest } from './digest.ts';-import { fileDriftSide, type GitHoldRecord } from './sync-holds.ts';+import { fileDriftSide, GIT_HOLD_OP, type GitHoldRecord } from './sync-holds.ts';import { readDeletesWithheld, type DeletesWithheld } from './sync-markers.ts';import { ABANDONED_QUIET_MS, ABANDONED_REPORT_STREAK, MANAGED_SYNC_PASS_OP, NOT_CONVERGED_REPORT_STREAK, passHealth, passReadClock, transientEnding, type PassRoundSummary,type PassRow } from './sync-pass-row.ts';@@ -70,6 +70,19 @@ export async function passConvergence(engine: Pick<BrainEngine, 'executeRaw'>, iduration_ms: t.duration_ms, timings_ms: t.timings_ms, last_full_at: o.converged?.last_full_at ?? null };}+/**+ * GBRA-542: whether a pass committed work: a sync write it admitted committed, or it held a path it had not held before. Read+ * from durable rows, never from the pass row's heartbeat, so a pass that only reported progress is not counted.+ */+export async function passBanked(engine: Pick<BrainEngine, 'executeRaw'>, sourceId: string, incarnation: string, telemetry: { pass_id: string; started_at: string }): Promise<boolean> {+ const [row] = await engine.executeRaw<{ banked: boolean }>(`SELECT EXISTS (SELECT 1 FROM persistence_requests r JOIN persistence_source_bindings b ON b.worktree_id=r.worktree_id+ WHERE b.source_id=$1 AND b.source_incarnation::text=$6 AND r.source_incarnation=b.source_incarnation AND r.state='committed' AND COALESCE(r.intent->>'kind','') LIKE 'managed_sync_%'+ AND r.completed_at >= $5::timestamptz AND r.intent->>'passId'=$2)+ OR EXISTS (SELECT 1 FROM op_checkpoints WHERE op=$3 AND left(fingerprint, length($4))=$4 AND completed_keys->0->>'run_id'=$2 AND (completed_keys->0->>'held_at')::timestamptz >= $5::timestamptz) AS banked`,+ [sourceId, telemetry.pass_id, GIT_HOLD_OP, `${sourceId}:${incarnation}:`, telemetry.started_at, incarnation]);+ return row?.banked === true;+}+export type PassStatusReason = 'pass_unfinished' | 'pass_interrupted' | 'pass_abandoned' | 'not_converged' | 'settle_mismatch' | 'deletes_withheld' | 'selection_scoped' | 'admission_paused' | 'writeback_failed';/**@@ -80,9 +93,11 @@ export type PassStatusReason = 'pass_unfinished' | 'pass_interrupted' | 'pass_ab* streak counts again once that holder stops renewing.* #7982: a pass that ended on a transient stop counts itself, so passes that hit the same statement timeout every time build the* streak too. `now` is the database clock at the row's read (`passReadClock`), never only this process's.+ * GBRA-542: a pass that committed work (`banked`, from `passBanked`) moved the source whatever stopped it, so it resets the streak.*/-export function abandonedPasses(row: PassRow, now = passReadClock(row), interrupted?: boolean): number {+export function abandonedPasses(row: PassRow, now = passReadClock(row), interrupted?: boolean, banked?: boolean): number {const t = row.value.telemetry, streak = row.value.obligations.abandoned_streak ?? 0;+ if (banked) return 0;if (transientEnding(t)) return streak + 1;if (t.outcome !== 'open' || interrupted === false) return 0;return streak + ((interrupted ?? now - Date.parse(row.updated_at) >= ABANDONED_QUIET_MS) ? 1 : 0);@@ -94,9 +109,9 @@ export function abandonedPasses(row: PassRow, now = passReadClock(row), interrup* #7596 `interrupted`: no process holds the source's drain lease (`readDrainLease`), so an open pass is one whose process* stopped without ending it (killed or crashed), not one still running: `pass_interrupted`.*/-export function passStatusState(row: PassRow, withheld: DeletesWithheld | null, writebackFailed: string | null = null, now = passReadClock(row), interrupted?: boolean): ['synced' | 'resumable' | 'blocked', PassStatusReason] | null {+export function passStatusState(row: PassRow, withheld: DeletesWithheld | null, writebackFailed: string | null = null, now = passReadClock(row), interrupted?: boolean, banked?: boolean): ['synced' | 'resumable' | 'blocked', PassStatusReason] | null {const t = row.value.telemetry;- if (abandonedPasses(row, now, interrupted) >= ABANDONED_REPORT_STREAK) return ['blocked', 'pass_abandoned'];+ if (abandonedPasses(row, now, interrupted, banked) >= ABANDONED_REPORT_STREAK) return ['blocked', 'pass_abandoned'];if (t.outcome === 'open' && interrupted) return ['resumable', 'pass_interrupted'];if (t.outcome === 'open' || (t.outcome === 'resumable' && t.stop_reason !== 'not_converged' && t.stop_reason !== 'admission_paused')) return ['resumable', 'pass_unfinished'];if (t.stop_reason === 'admission_paused') return ['resumable', 'admission_paused'];@@ -109,7 +124,7 @@ export function passStatusState(row: PassRow, withheld: DeletesWithheld | null,}/** The next step for a pass reason (`resume`: the plain sync's arguments). */-export function passStatusNext(sourceId: string, reason: PassStatusReason, row: PassRow, resume: string[], writebackFailed: string | null = null, interrupted?: boolean): Action {+export function passStatusNext(sourceId: string, reason: PassStatusReason, row: PassRow, resume: string[], writebackFailed: string | null = null, interrupted?: boolean, banked?: boolean): Action {const verify = { argv: ['gbrain', 'sync', 'status', '--source', sourceId, '--json'] };const sync: Action = { argv: ['gbrain', 'sync', ...resume], consent: [], actor: 'agent', requires_exclusive: false, verify, why: '' };switch (reason) {@@ -133,7 +148,7 @@ export function passStatusNext(sourceId: string, reason: PassStatusReason, row:why: 'The preview names the parked effect and what a retry runs (read-only); until it commits, the source is never idle.' };case 'pass_abandoned':return { argv: ['gbrain', 'doctor', '--json'], consent: [], actor: 'agent', requires_exclusive: false, verify,- why: `The last ${abandonedPasses(row, passReadClock(row), interrupted)} sync passes of ${sourceId} stopped before they ended (the drain abandoned them for making no progress, or their process was killed) or on a transient stop that recurs (a statement timeout, a lost connection), so another plain sync is likely to stop the same way. Report what doctor shows, with the gbrain version and the newest sync run's drain.stall (read-only).` };+ why: `The last ${abandonedPasses(row, passReadClock(row), interrupted, banked)} sync passes of ${sourceId} stopped before they ended (the drain abandoned them for making no progress, or their process was killed) or on a transient stop that recurs (a statement timeout, a lost connection), so another plain sync is likely to stop the same way. Report what doctor shows, with the gbrain version and the newest sync run's drain.stall (read-only).` };case 'pass_interrupted':return { ...sync, why: `The newest pass of ${sourceId} stopped before it ended and no process holds its drain lease: its process was killed or crashed (an out-of-memory kill or a timeout), or stopped on an error it does not record (a dropped connection), so nothing recorded a failure. The plain sync continues it from the current level; if that pass stops the same way, status reads pass_abandoned.` };case 'selection_scoped':diff --git a/src/core/persistence/sync-run.ts b/src/core/persistence/sync-run.tsindex 863ff28eac..19a5bb2499 100644--- a/src/core/persistence/sync-run.ts+++ b/src/core/persistence/sync-run.ts@@ -1460,7 +1460,7 @@ async function runManagedSync(engine: BrainEngine, opts: SyncOpts, slice: { maxP/** One cursor's drain (or, #6712 PR 4b, one round batch of a live pass): the freeze, screen, waive, group, admit, await and settle loop. */const live: { cursor: Cursor | null } = { cursor: null };const drainCursor = async (from: Cursor, startupConfig: Awaited<ReturnType<typeof preparationConfigView>>,- sliceBase?: { started: number; pages: number }): Promise<SyncResult | Cursor> => {+ sliceBase?: { started: number; pages: number; committed?: (index: number) => void }): Promise<SyncResult | Cursor> => {let cursor = from;try {frozenRun.screen = company ? null : await loadSyncScreenRun(startupConfig, cursor.sourceId, cursor.processingOptions ?? processingOptions, authority.writer.remote);@@ -1508,7 +1508,7 @@ async function runManagedSync(engine: BrainEngine, opts: SyncOpts, slice: { maxPconst next = await freezeNext(engine, cursor, key, assertActive, frozenRun, observedAt);if ('saved' in next) {cursor = next.saved;- opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, ...(next.held ? {} : { total: cursor.entries.length, waived: true }) }); assertActive();+ opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, ...(next.held ? {} : { total: cursor.entries.length, waived: true }) }); sliceBase?.committed?.(cursor.index); assertActive();if (next.held && slice && (cursor.index - sliceFirstIndex >= slice.maxPages || performance.now() - sliceStarted >= slice.maxMs)) return result(cursor, 'partial', 'writer_yield');continue;}@@ -1534,7 +1534,7 @@ async function runManagedSync(engine: BrainEngine, opts: SyncOpts, slice: { maxPholdClear(screened, pending.intent.path, observedAt)), tx => currentCursor(tx, key, screened), signal);if (waived) {cursor = waived;- opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, total: cursor.entries.length, waived: true });+ opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, total: cursor.entries.length, waived: true }); sliceBase?.committed?.(cursor.index);assertActive();// A skipped entry still counts toward the caller's slice, so a sliced run yields at the same positions.if (slice && (cursor.index - sliceFirstIndex >= slice.maxPages || performance.now() - sliceStarted >= slice.maxMs)) return result(cursor, 'partial', 'writer_yield');@@ -1585,7 +1585,7 @@ async function runManagedSync(engine: BrainEngine, opts: SyncOpts, slice: { maxPremote: authority.writer.remote, waitMs: opts.drainStartedAt ? syncReceiptWait().drain : syncReceiptWait().pass, signal });if ('result' in settled) return settled.result;cursor = settled.cursor;- opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, total: cursor.entries.length });+ opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, total: cursor.entries.length }); sliceBase?.committed?.(cursor.index);continue;}if (pending.intent.kind === 'managed_sync_checkpoint' && cursor.passId) return { ...cursor, pending: undefined, done: true };@@ -1600,7 +1600,7 @@ async function runManagedSync(engine: BrainEngine, opts: SyncOpts, slice: { maxPconst next: Cursor = committedPast({ ...cursor, ...committedBreaker(cursor), counts: { ...cursor.counts }, progress: stampProgress(cursor.progress, cursor.index, cursor.index + 1, drainStartedAt) }, pending);countCommitted(next.counts, pending, done.outcome);cursor = await saveCursor(engine, key, cursor, next);- opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, total: cursor.entries.length }); if (next.counts.folded) return cursor;+ opts.onProgress?.({ phase: 'managed_sync.page_committed', bankedFiles: cursor.index, total: cursor.entries.length }); if (next.counts.folded) return cursor; sliceBase?.committed?.(cursor.index);// F4b: PGLite plans the rest of a large sync against fresh statistics (spec Addendum A item 2).if (analyzeEvery > 0 && cursor.index % analyzeEvery === 0) await maybeRefreshPlannerStats(engine, 'managed_sync', { throttle: false }).catch(() => undefined);assertActive();diff --git a/src/core/persistence/sync-status.ts b/src/core/persistence/sync-status.tsindex 16cd8f12c3..76eaf386cc 100644--- a/src/core/persistence/sync-status.ts+++ b/src/core/persistence/sync-status.ts@@ -27,7 +27,7 @@ import type { BrainEngine } from '../engine.ts';import type { Action } from '../agent-output.ts';import { managedSyncResumeArgs } from '../sync-reconcile.ts';import { readManagedSyncFailures, type ManagedSyncFailure } from './sync-failures.ts';-import { outOfAttemptsHold, fenceAutoRepairFor, fileDriftSide, gitHoldFix, holdEscalation, noteHoldEdit, readGitSourceHolds, recordHoldAttempt, requestGitHoldRetry, type GitHoldRecord } from './sync-holds.ts';+import { carriedHolds, type CarriedHolds, outOfAttemptsHold, fenceAutoRepairFor, fileDriftSide, gitHoldFix, mirrorDatabaseOnlyHold, holdEscalation, noteHoldEdit, readGitSourceHolds, recordHoldAttempt, requestGitHoldRetry, type GitHoldRecord } from './sync-holds.ts';import type { FenceAutoRepair } from '../fence-repair/hold-fix.ts';import { classifySyncFault, HOLD_ATTEMPTS_NEEDS_HUMAN, PAGE_BODY_FAULT_CODES, type SyncFaultVerdict } from './sync-fault-class.ts';import { CONTENT_REPAIR_KINDS, contentHoldDisposition, runContentLane, type ContentHoldDisposition } from '../repair/content-lane.ts';@@ -45,8 +45,8 @@ import type { PublicCategory } from './public-category.ts';import { readSyncFile, resolveManagedSyncContext } from './sync-discovery.ts';import { sha256 } from './digest.ts';import { passResumeArgs } from './sync-pass-readers.ts';-import { openPassRowsStill, passConvergence, passStatusNext, passStatusState } from './sync-pass-status.ts';-import { MANAGED_SYNC_PASS_OP, passReadClock, readPassRow, SYNC_DELETES_WITHHELD_OP, type PassRow } from './sync-pass-row.ts';+import { abandonedPasses, openPassRowsStill, passBanked, passConvergence, passStatusNext, passStatusState } from './sync-pass-status.ts';+import { ABANDONED_REPORT_STREAK, MANAGED_SYNC_PASS_OP, passReadClock, readPassRow, SYNC_DELETES_WITHHELD_OP, type PassRow } from './sync-pass-row.ts';import { SYNC_ADMISSION_SETTING_PARAMS, syncAdmissionEffectParams, syncAdmissionEffectSql, syncAdmissionSettingSql, syncAdmissionStatus } from './sync-admission.ts';import type { DeletesWithheld } from './sync-markers.ts';import { listIdentityDowngrades, withIdentityRemedies } from '../sync-identity-downgrades.ts';@@ -158,6 +158,8 @@ export interface SyncStatus {rate_pages_per_min: number | null;eta_seconds: number | null;holds: SyncStatusHold[];+ /** GBRA-542: holds carried by design (a read-only mirror's database-only pages), by class with a sample; absent when none. */+ carried?: CarriedHolds[];/*** #6729: renames whose page identity was not kept and whose pair has no disposition yet (old page soft-deleted with its* facts and links, held from the purge), each with its remedy: `gbrain repair rename-identity` preview and apply argv,@@ -315,6 +317,7 @@ export function holdVerdict(record: GitHoldRecord, auto: FenceAutoRepair | undefif (record.code === 'origin_conflict' && record.meta.reason) return classifySyncFault({ code: record.code, detail: 'origin_conflict_decision', ...escalation });if (record.code === 'page_identity_changed') return classifySyncFault({ code: record.code, detail: 'page_identity_decision', ...escalation });if (record.code === 'file_drift' && fileDriftSide(record.meta) === 'unknown') return classifySyncFault({ code: record.code, detail: 'file_drift_unknown', ...escalation });+ if (mirrorDatabaseOnlyHold(record)) return classifySyncFault({ code: record.code, detail: 'file_drift_mirror_database_only' });const bodyFault = record.code === 'page_write_failed' ? record.meta.write_failure?.code : undefined;if (bodyFault && PAGE_BODY_FAULT_CODES.has(bodyFault)) return classifySyncFault({ code: record.code, detail: `freeze_${bodyFault}`, ...escalation });const disposition = contentHoldDisposition(record, auto);@@ -439,7 +442,10 @@ export async function readSyncStatus(engine: BrainEngine, sourceId: string, optsconst raced = source.drain_lock && lease !== undefined && !holder && passRow?.value.telemetry.outcome === 'open'&& !(await openPassRowsStill(engine, [{ source_id: sourceId, row: passRow }]).catch(() => undefined))?.has(sourceId);const interrupted = lease === undefined || raced || passRow?.value.telemetry.outcome !== 'open' ? undefined : !holder;- const passState = passRow ? passStatusState(passRow, withheld, source.writeback_failed, passReadClock(passRow), interrupted) : null;+ // GBRA-542: a pass that stopped after committing work is not counted toward pass_abandoned (read only when the streak could reach it).+ const banked = passRow && abandonedPasses(passRow, passReadClock(passRow), interrupted) >= ABANDONED_REPORT_STREAK+ ? await passBanked(engine, sourceId, source.incarnation, passRow.value.telemetry).catch(() => undefined) : undefined;+ const passState = passRow ? passStatusState(passRow, withheld, source.writeback_failed, passReadClock(passRow), interrupted, banked) : null;const resume = !header && passRow ? passResumeArgs(sourceId, passRow) : managedSyncResumeArgs({ sourceId, processingOptions: header?.processingOptions, syncOptions: header?.syncOptions });const human = holds.find(hold => hold.needs_human) ?? (liveError?.needs_human ? liveError : undefined);const refreshRow = typeof source.refresh === 'string' ? JSON.parse(source.refresh) as RefreshProbeRow : source.refresh;@@ -492,11 +498,11 @@ export async function readSyncStatus(engine: BrainEngine, sourceId: string, optsconst [state, stateReason]: [SyncStatusState, SyncStatusReason] = waiting ? ['resumable', 'commits_waiting'] : settled;const admission = await syncAdmissionStatus(engine, sourceId, source.incarnation,{ stored: source.admission_stored, row_value: source.admission_row_value, set_at: source.admission_set_at, pinned: source.admission_pinned, destructive: source.admission_destructive });- const passNext = passRow && passState && passState[1] === stateReason ? passStatusNext(sourceId, passState[1], passRow, resume, source.writeback_failed, interrupted) : null;+ const passNext = passRow && passState && passState[1] === stateReason ? passStatusNext(sourceId, passState[1], passRow, resume, source.writeback_failed, interrupted, banked) : null;return { source_id: sourceId, last_commit: source.last_commit, last_sync_at: iso(source.last_sync_at), last_committed_at: iso(committed?.last_committed_at), live: holder ? { ...holder, job_id: holderJob, stalled: holderStalled(holder, lease?.db_now) } : null,interrupted: interrupted && passRow ? { pass_id: passRow.value.telemetry.pass_id, since: passRow.updated_at, holder: lease?.holder ?? null } : null, run_id: header?.runId ?? null,cursor: header ? { index: Number(header.index), total: Number(header.total), pinned_target: header.target ?? null, last_advance_at: p ? new Date(p.lastAt).toISOString() : null, done: !!header.done } : null,- committed_last_10m: Number(committed?.n ?? 0), ...estimate, live_job_id: liveJob ?? holderJob, holds, identity_downgrades: await withIdentityRemedies(engine, await listIdentityDowngrades(engine, [sourceId])), last_error: lastError, needs_human: !!human, ...(human ? { human_reason: human.human_reason } : {}),+ committed_last_10m: Number(committed?.n ?? 0), ...estimate, live_job_id: liveJob ?? holderJob, holds, ...(carriedHolds(records).length ? { carried: carriedHolds(records) } : {}), identity_downgrades: await withIdentityRemedies(engine, await listIdentityDowngrades(engine, [sourceId])), last_error: lastError, needs_human: !!human, ...(human ? { human_reason: human.human_reason } : {}),needs_human_from: !human ? null : 'path' in human && 'held_since' in human ? { kind: 'hold', path: human.path, slug: human.slug, code: human.code } : { kind: 'last_error', code: human.code },resume_argv: resume, ...blockers, cursors: detail ? await liveCursors(engine, sourceId, source.incarnation, selected?.key ?? null, failures) : [],terminal_failures: terminal, connector_last_error: connectorLastError, git_push_holds: gitPushHolds, fence_holds: fenceHolds, worktree_refresh: worktreeRefresh, next: next ?? recovery_next ?? passNext ?? (stateReason === 'request_pending' && oldest.status === 'ok' && oldest.request ? { argv: ['gbrain', 'sync', ...resume], consent: [], actor: 'agent', requires_exclusive: false, verify,diff --git a/src/core/persistence/unindexed-files.ts b/src/core/persistence/unindexed-files.tsindex c897fc8d92..5fd790e3e0 100644--- a/src/core/persistence/unindexed-files.ts+++ b/src/core/persistence/unindexed-files.ts@@ -59,6 +59,7 @@ import { loadSyncEligibility, resolveManagedSyncContext, syncExcludePatterns, syimport { heldGitPaths, readGitHoldRetryPaths, requestGitHoldRetry } from './sync-holds.ts';import { readBlobContents, type TreeBlob } from './sync-blobs.ts';import { isWindowsColonTarget } from './native-file-target.ts';+import { danglingLink } from './sync-entry-fault.ts';import { createHash } from 'node:crypto';export const UNINDEXED_FILES_OP = 'unindexed-files';@@ -235,7 +236,9 @@ export async function findUnindexedFiles(engine: BrainEngine, sourceId: string,'SELECT source_path, slug FROM pages WHERE source_id=$1 AND deleted_at IS NULL AND source_path = ANY($2::text[])', [sourceId, origins]);const byOrigin = new Map(pages.map(page => [page.source_path, page.slug]));const held = await heldGitPaths(engine, sourceId, incarnation, rows.map(row => row.root));- const missing = rows.filter(row => !byOrigin.has(row.origin) && !(row.legacy && byOrigin.get(row.legacy) === resolveSlugForPath(row.root)) && !held.has(row.root));+ // GBRA-542: a committed symlink whose target does not exist is skipped by sync on every pass, so it is not unindexed either.+ const missing = rows.filter(row => !byOrigin.has(row.origin) && !(row.legacy && byOrigin.get(row.legacy) === resolveSlugForPath(row.root)) && !held.has(row.root)+ && !(row.mode === SYMLINK_MODE && danglingLink(gitRoot, row.path)));const fresh = missing.filter(row => {if (!retry.has(row.root)) return true;scan.scheduled++;diff --git a/test/notifier-sync-disabled-gst2515.test.ts b/test/notifier-sync-disabled-gst2515.test.tsnew file mode 100644index 0000000000..beebaa2f8d--- /dev/null+++ b/test/notifier-sync-disabled-gst2515.test.ts@@ -0,0 +1,144 @@+/**+ * GST-2515: a source with sync turned off (`config.syncEnabled=false`, `gbrain sources set-sync <id> off`) kept being+ * synced by the commit notifier whenever its HEAD moved, and a run it owed (retry, or due after a restart) kept+ * restarting it. The notifier now leaves such a source unwatched and owes it nothing; explicit wanted markers (a sync+ * trigger, a webhook) still run, and turning sync back on restores the HEAD watch. Synthetic content, injected clock.+ */+import { describe, expect, test } from 'bun:test';+import { mkdtempSync, rmSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { execFileSync } from 'node:child_process';+import { LIST_EVERY_MS, RETRY_BACKOFF_MS, startCommitNotifier, type CommitNotifier, type JobStatus, type NotifierSource, type NotifierStore, type StartRequest } from '../src/core/commit-notifier.ts';+import { listHostSources } from '../src/core/notifier/resident.ts';+import { isolatedSharedSkillsEngine } from './helpers/shared-skills-engine.ts';+import { memoryNotifierStore } from './helpers/notifier-memory-store.ts';++const SOURCE = 'source-a';+const HOST = '00000000-0000-4000-8000-0000000025a6';+const A = 'a'.repeat(40), B = 'b'.repeat(40);+const QUIET = 400;++interface World { t: number; starts: StartRequest[]; jobs: Map<string, JobStatus>; disabled: boolean }++function resident(w: World, store: NotifierStore, flag = { on: true }): CommitNotifier {+ return startCommitNotifier({+ resident: 'serve', log: () => {}, hostId: HOST, store, now: () => w.t, pollMs: 3000, quietMs: QUIET, heartbeatMs: 10_000,+ firstSliceSeconds: 30, sliceSeconds: 120, maxConcurrent: 1,+ setInterval: () => 'interval', clearInterval: () => {}, setTimeout: () => 'timer', clearTimeout: () => {},+ readFlag: async () => flag.on,+ listSources: async (): Promise<NotifierSource[]> => [{ sourceId: SOURCE, root: '/nonexistent/source-a', gitDir: null, lastCommit: A, kind: 'managed', ...(w.disabled ? { syncDisabled: true } : {}) }],+ lastCommit: async () => A, revParse: async () => B, syncRunning: () => false, drainHeld: async () => false,+ startSync: async (_s, req) => { w.starts.push(req); const jobId = 'job-' + w.starts.length; w.jobs.set(jobId, { state: 'running' }); return { jobId }; },+ jobStatus: jobId => w.jobs.get(jobId) ?? null,+ });+}+const world = (disabled: boolean): World => ({ t: 1_000_000, starts: [], jobs: new Map(), disabled });++describe('GST-2515: the commit notifier honours config.syncEnabled=false', () => {+ test('a HEAD move on a source with sync off starts nothing; turning sync back on syncs it', async () => {+ const w = world(true);+ const store = memoryNotifierStore({ lastCommit: () => A });+ const n = resident(w, store);+ await n.tick();+ for (let i = 0; i < 5; i++) { w.t += QUIET + 3000; await n.tick(); }+ expect(w.starts).toHaveLength(0);+ expect(n.snapshot().sources[0].watched).toBe(false);+ w.disabled = false;+ w.t += LIST_EVERY_MS; await n.tick();+ w.t += QUIET; await n.tick();+ expect(w.starts).toHaveLength(1);+ await n.stop();+ });++ test('a retry owed to a source with sync off never starts, but an explicit wanted marker still runs', async () => {+ const w = world(false);+ const store = memoryNotifierStore({ lastCommit: () => A });+ const n = resident(w, store);+ await n.tick();+ w.t += QUIET; await n.tick();+ expect(w.starts).toHaveLength(1);+ w.jobs.set('job-1', { state: 'done', outcome: 'resumable', reason: 'deadline', committed: false });+ w.t += QUIET; await n.tick();+ expect(store.states.get(SOURCE)).toMatchObject({ state: 'retry' });+ w.disabled = true;+ w.t += LIST_EVERY_MS; await n.tick();+ for (const wait of RETRY_BACKOFF_MS) { w.t += wait; await n.tick(); }+ expect(w.starts).toHaveLength(1);+ store.upsert(SOURCE, 'full', null);+ n.nudge();+ w.t += QUIET; await n.tick();+ w.t += QUIET; await n.tick();+ expect(w.starts).toHaveLength(2);+ await n.stop();+ });++ test('on a source with sync off an explicit request runs once, and the notifier does not retry it when it fails', async () => {+ const w = world(true);+ const store = memoryNotifierStore({ lastCommit: () => A });+ store.upsert(SOURCE, 'full', null);+ const n = resident(w, store);+ await n.tick();+ w.t += QUIET; await n.tick();+ expect(w.starts).toHaveLength(1);+ w.jobs.set('job-1', { state: 'error', error: 'boom' });+ w.t += QUIET; await n.tick();+ for (const wait of [...RETRY_BACKOFF_MS, RETRY_BACKOFF_MS.at(-1)!]) { w.t += wait; await n.tick(); }+ expect(w.starts).toHaveLength(1);+ store.upsert(SOURCE, 'full', null);+ n.nudge();+ w.t += QUIET; await n.tick();+ w.t += QUIET; await n.tick();+ expect(w.starts).toHaveLength(2);+ await n.stop();+ });++ test('turning sync off drops the retry an already-unwatched source owed: turning it back on starts nothing', async () => {+ const w = world(false);+ const store = memoryNotifierStore({ lastCommit: () => A });+ store.upsert(SOURCE, 'freshness', B);+ const n = resident(w, store, { on: false });+ await n.tick();+ expect(w.starts).toHaveLength(1);+ w.jobs.set('job-1', { state: 'done', outcome: 'resumable', reason: 'deadline', committed: false });+ store.markers.clear();+ w.t += QUIET; await n.tick();+ expect(store.states.get(SOURCE)).toMatchObject({ state: 'retry' });+ w.disabled = true;+ w.t += LIST_EVERY_MS; await n.tick();+ w.t += QUIET; await n.tick();+ expect(w.starts).toHaveLength(1);+ expect(store.states.get(SOURCE)).toMatchObject({ state: 'idle' });+ w.disabled = false;+ w.t += LIST_EVERY_MS; await n.tick();+ for (const wait of RETRY_BACKOFF_MS) { w.t += wait; await n.tick(); }+ expect(w.starts).toHaveLength(1);+ await n.stop();+ });+});++const arms: Array<[string, string | undefined]> = [['pglite', undefined], ...(process.env.DATABASE_URL ? [['postgres', process.env.DATABASE_URL] as [string, string]] : [])];++describe.each(arms)('GST-2515: listHostSources marks a source with sync off (%s)', (_arm, databaseUrl) => {+ test('config.syncEnabled=false lists the source as syncDisabled; on or unset does not', async () => {+ const root = mkdtempSync(join(tmpdir(), 'gst2515-notifier-'));+ execFileSync('git', ['init', '-q', root]);+ const { engine, close } = await isolatedSharedSkillsEngine(databaseUrl);+ const ids = ['source-off', 'source-on', 'source-unset'];+ try {+ const configs = ['{"syncEnabled":false}', '{"syncEnabled":true}', '{}'];+ for (const [i, id] of ids.entries()) {+ await engine.executeRaw('INSERT INTO sources(id,name,local_path,config) VALUES($1,$1,$2,$3::jsonb) ON CONFLICT (id) DO UPDATE SET local_path = EXCLUDED.local_path, config = EXCLUDED.config', [id, root, configs[i]]);+ }+ const { sources } = await listHostSources(engine, HOST, { GBRAIN_NOTIFIER_CLASSIC_SOURCES: ids.join(',') });+ const byId = new Map(sources.map(s => [s.sourceId, s]));+ expect(byId.get('source-off')?.syncDisabled).toBe(true);+ expect(byId.get('source-on')?.syncDisabled).toBeUndefined();+ expect(byId.get('source-unset')?.syncDisabled).toBeUndefined();+ } finally {+ for (const id of ids) await engine.executeRaw('DELETE FROM sources WHERE id = $1', [id]).catch(() => undefined);+ await close();+ rmSync(root, { recursive: true, force: true });+ }+ }, 120_000);+});diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 62bd7a36c9..cc3996a84d 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -376,6 +376,7 @@ test/mirror-readonly-write-through-7098.test.tstest/moved-pages-gst2584.test.tstest/notifier-passed-freshness-marker-7941.test.tstest/notifier-retry-unwatched-gst2515.test.ts+test/notifier-sync-disabled-gst2515.test.tstest/on-commit-embedding.serial.test.tstest/oneshot-consumer-direct-route.test.tstest/oneshot-housekeeping-cli.test.ts@@ -701,6 +702,7 @@ test/sync-legacy-slug-adoption-6727.test.tstest/sync-legacy-sweep-resume-7189.test.tstest/sync-level-reuse-across-gbra133.test.tstest/sync-metafile-foreign-slug-7341.test.ts+test/sync-mirror-catchup-gst2515.test.tstest/sync-origin-live-wins-6517.test.tstest/sync-page-identity-hold-gbra130.test.tstest/sync-page-write-failed-gbra130.test.tsdiff --git a/test/sync-holds-only-converge-7327.test.ts b/test/sync-holds-only-converge-7327.test.tsindex 8be863a648..7e2afd35cf 100644--- a/test/sync-holds-only-converge-7327.test.ts+++ b/test/sync-holds-only-converge-7327.test.ts@@ -1,6 +1,7 @@import { expect, test } from 'bun:test';import { execFileSync } from 'node:child_process';-import { mkdirSync, symlinkSync, writeFileSync } from 'node:fs';+import { mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';import { join } from 'node:path';import type { BrainEngine } from '../src/core/engine.ts';import type { SyncResult } from '../src/commands/sync.ts';@@ -42,7 +43,9 @@ for (const backend of testBackends()) {const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;test(`#7327 [${backend}]: a re-screen of held symlinks cut short by deadlines resumes, converges with its holds, and shows progress`, () => managedBrain(async ({ engine, root }) => {mkdirSync(join(root, 'linked'));- for (let i = 0; i < N; i++) symlinkSync(`/nonexistent-other-checkout/x${i}.md`, join(root, `linked/s${i}.md`));+ // GST-2515: links into a checkout outside this source (present on this host); a link to nothing is skipped, not held.+ const other = mkdtempSync(join(tmpdir(), 'gbrain-7327-other-'));+ for (let i = 0; i < N; i++) { writeFileSync(join(other, `x${i}.md`), `---\ntype: concept\ntitle: X${i}\n---\n\nOther ${i}.\n`); symlinkSync(join(other, `x${i}.md`), join(root, `linked/s${i}.md`)); }git(root, 'add', '-A'); git(root, 'commit', '-qm', 'symlinks');expect((await pass(engine)).result.status).toBe('first_sync');expect(await holdCount(engine)).toBe(N);@@ -67,6 +70,7 @@ for (const backend of testBackends()) {expect(await readGitHoldRetryPaths(engine, 'default', (await engine.executeRaw<{ incarnation: string }>("SELECT incarnation::text AS incarnation FROM sources WHERE id='default'"))[0]!.incarnation)).toEqual([]);expect(await holdCount(engine)).toBe(N);expect((await pass(engine)).result.status).toBe('up_to_date');+ rmSync(other, { recursive: true, force: true });}, { databaseUrl, setup: ({ root }) => {mkdirSync(join(root, 'notes')); git(root, 'init', '-q');writeFileSync(join(root, 'notes/a.md'), '---\ntype: concept\ntitle: A\n---\n\nAn observation about acme-example.\n');diff --git a/test/sync-idle-skips-gbra133.test.ts b/test/sync-idle-skips-gbra133.test.tsindex cf21f1d500..f6fbdb70d6 100644--- a/test/sync-idle-skips-gbra133.test.ts+++ b/test/sync-idle-skips-gbra133.test.ts@@ -43,7 +43,7 @@ async function recordStatements<T>(engine: BrainEngine, fn: () => Promise<T>): P/** What each settle in IDLE_SKIPS sends when it runs, counted in a run's statements. */const settles = (sent: Array<{ sql: string; params: unknown[] }>, sourceId: string) => ({- holdReads: sent.filter(s => s.sql.startsWith('SELECT h.completed_keys->0 AS record FROM op_checkpoints h') && s.params[0] === GIT_HOLD_OP && s.params[2] === null).length,+ holdReads: sent.filter(s => (s.sql.startsWith('SELECT ') && s.sql.includes(' AS record FROM op_checkpoints h')) && s.params[0] === GIT_HOLD_OP && s.params[2] === null).length,holdReport: sent.filter(s => s.sql.includes("COALESCE((completed_keys->0->>'count')::int,0) AS count") && s.params[0] === GIT_HOLD_SUMMARY_OP).length,failureRows: sent.filter(s => s.sql.includes("f.op='managed-sync-failure' AND f.completed_keys->0->>'source_id'=$1")).length,needsHuman: sent.filter(s => s.sql === 'SELECT incarnation::text AS incarnation FROM sources WHERE id=$1' && s.params[0] === sourceId).length,diff --git a/test/sync-mirror-catchup-gst2515.test.ts b/test/sync-mirror-catchup-gst2515.test.tsnew file mode 100644index 0000000000..85ed5f774e--- /dev/null+++ b/test/sync-mirror-catchup-gst2515.test.ts@@ -0,0 +1,190 @@+/**+ * GST-2515: a large source's catch-up that a run limit stops every time.+ * - A pass that committed work before it was stopped no longer counts toward the abandoned streak, so status stops+ * calling a source that banks work every pass "abandoned". A pass that committed nothing (only a heartbeat) still+ * counts.+ * - A read-only mirror carries its database-only pages (file gone upstream, page edited in gbrain) as a counted class+ * that never blocks the pass, and status never advises `--prefer database`, which would write the mirror.+ * - A symlink whose target does not exist is skipped instead of held as unreadable; one whose path is a live page's+ * stays held; an unreadable hold an older pass wrote for a dangling link is released.+ * PGLite always; Postgres when DATABASE_URL is set (managedBrain). Synthetic content only.+ */+import { describe, expect, setSystemTime, spyOn, test } from 'bun:test';+import * as passStatus from '../src/core/persistence/sync-pass-status.ts';+import { buildSingleSyncJsonEnvelope } from '../src/core/sync-embed-backfill.ts';+import { execFileSync } from 'node:child_process';+import { mkdirSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import { withCoordinatedWrite } from '../src/core/persistence/context.ts';+import { installFaultHook } from '../src/core/persistence/fault-points.ts';+import { performManagedSync } from '../src/core/persistence/sync-run.ts';+import { readPassRow } from '../src/core/persistence/sync-pass-row.ts';+import { readSyncStatus } from '../src/core/persistence/sync-status.ts';+import { writeGitHold } from '../src/core/persistence/sync-holds.ts';+import { keepCommittedLinks } from '../src/core/persistence/sync-entry-fault.ts';+import { managedBrain } from './helpers/managed-brain.ts';+import { TEST_WRITE_ATTRIBUTION } from './helpers/write-attribution.ts';+import { requirePostgresTestDatabase, testBackends } from './helpers/test-backends.ts';++const git = (root: string, ...args: string[]) => execFileSync('git', ['-C', root, '-c', 'user.name=Example', '-c', '[email protected]',+ '-c', 'commit.gpgsign=false', ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim();+const note = (slug: string, body: string) => `---\ntype: concept\ntitle: ${slug}\n---\n\n${body}\n`;+const OPTS = { sourceId: 'default', noPull: true, noEmbed: true, noExtract: true };+class Killed extends Error {}+const incarnation = async (engine: any) => (await engine.executeRaw("SELECT incarnation::text AS i FROM sources WHERE id='default'"))[0].i as string;+const holdCodes = async (engine: any) => Object.fromEntries((await engine.executeRaw(+ "SELECT completed_keys->0->>'code' AS code, count(*)::int AS n FROM op_checkpoints WHERE op='sync-hold' GROUP BY 1")).map((r: any) => [r.code, r.n]));+const seed = (count: number, databaseUrl: string | undefined) => ({ databaseUrl, setup: ({ root }: { root: string }) => {+ git(root, 'init', '-q'); mkdirSync(join(root, 'keep'), { recursive: true });+ for (let i = 0; i < count; i++) writeFileSync(join(root, 'keep', `k${i}.md`), note(`keep/k${i}`, `Body ${i}.`));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'init');+} });++/** One sync stopped after `after` admitted batches (`after` 0: before the first). */+async function killedSync(engine: any, after: number) {+ let n = 0;+ installFaultHook(async point => { if (point === 'sync:mid_checkpoint' && ++n > after) throw new Killed('killed'); });+ try { await performManagedSync(engine, OPTS); } catch (e) { if (!(e instanceof Killed) && !String(e).includes('killed')) throw e; } finally { installFaultHook(undefined); }+}++describe.each(testBackends())('GST-2515: a stopped pass that banked work is not abandoned (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ test('passes stopped after committing pages keep the abandoned streak at 0 and status does not report pass_abandoned', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ mkdirSync(join(root, 'new'), { recursive: true });+ for (let i = 0; i < 60; i++) writeFileSync(join(root, 'new', `n${i}.md`), note(`new/n${i}`, `New ${i}.`));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream adds files');+ const inc = await incarnation(engine);+ const before = Number((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE slug LIKE 'new/%'"))[0]!.n);+ for (let pass = 0; pass < 3; pass++) await killedSync(engine, 3);+ const banked = Number((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE slug LIKE 'new/%'"))[0]!.n);+ expect(banked).toBeGreaterThan(before);+ await performManagedSync(engine, { ...OPTS, dryRun: true } as any).catch(() => undefined);+ const row = await readPassRow(engine, 'default', inc);+ expect(row?.value.obligations.abandoned_streak ?? 0).toBe(0);+ const status: any = await readSyncStatus(engine, 'default');+ expect(status.reason).not.toBe('pass_abandoned');+ }, seed(5, databaseUrl)), 600_000);++ test('an open pass reports the entries it settled so far, on the pass heartbeat', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ mkdirSync(join(root, 'new'), { recursive: true });+ for (let i = 0; i < 30; i++) writeFileSync(join(root, 'new', `n${i}.md`), note(`new/n${i}`, `New ${i}.`));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream adds files');+ const inc = await incarnation(engine);+ let commits = 0;+ const progress = (e: { phase?: string }) => { if (e.phase === 'managed_sync.page_committed' && ++commits === 2) setSystemTime(new Date(Date.now() + 60_000)); };+ let n = 0;+ installFaultHook(async point => { if (point === 'sync:mid_checkpoint' && commits >= 4 && ++n > 1) throw new Killed('killed'); });+ try { await performManagedSync(engine, { ...OPTS, onProgress: progress } as any); } catch (e) { if (!String(e).includes('killed')) throw e; } finally { installFaultHook(undefined); setSystemTime(); }+ const t = (await readPassRow(engine, 'default', inc))?.value.telemetry;+ expect(t?.outcome).toBe('open');+ expect(t?.settled).toBeGreaterThan(0);+ }, seed(3, databaseUrl)), 600_000);++ test('a failed read of whether the stopped pass banked work counts it the old way instead of stopping the next pass', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ mkdirSync(join(root, 'new'), { recursive: true });+ for (let i = 0; i < 30; i++) writeFileSync(join(root, 'new', `n${i}.md`), note(`new/n${i}`, `New ${i}.`));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream adds files');+ const inc = await incarnation(engine);+ await killedSync(engine, 3);+ const failed = spyOn(passStatus, 'passBanked').mockRejectedValue(new Error('statement timeout'));+ try { await killedSync(engine, 3); } finally { failed.mockRestore(); }+ expect((await readPassRow(engine, 'default', inc))?.value.obligations.abandoned_streak).toBe(1);+ }, seed(3, databaseUrl)), 600_000);++ test('passes stopped before committing anything still build the streak', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ mkdirSync(join(root, 'new'), { recursive: true });+ for (let i = 0; i < 10; i++) writeFileSync(join(root, 'new', `n${i}.md`), note(`new/n${i}`, `New ${i}.`));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream adds files');+ const inc = await incarnation(engine);+ for (let pass = 0; pass < 3; pass++) {+ const stop = (e: { phase?: string }) => { if (e.phase === 'managed_sync.round_admit') throw new Killed('killed'); };+ try { await performManagedSync(engine, { ...OPTS, onProgress: stop } as any); } catch (e) { if (!String(e).includes('killed')) throw e; }+ }+ expect(Number((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE slug LIKE 'new/%'"))[0]!.n)).toBe(0);+ expect((await readPassRow(engine, 'default', inc))?.value.obligations.abandoned_streak).toBe(2);+ }, seed(3, databaseUrl)), 600_000);+});++describe.each(testBackends())('GST-2515: a read-only mirror (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ test('database-only pages are carried as a counted class and status never advises --prefer database', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ await engine.transaction(tx => withCoordinatedWrite(tx, ['default'], () => tx.executeRaw(+ "UPDATE pages SET compiled_truth=compiled_truth||' edited', knowledge_revision=gen_random_uuid() WHERE source_id='default' AND slug LIKE 'keep/k1%'"), TEST_WRITE_ATTRIBUTION));+ await engine.executeRaw(`UPDATE sources SET config = config || '{"mirror_read_only": true}'::jsonb WHERE id='default'`);+ for (let i = 10; i < 20; i++) rmSync(join(root, 'keep', `k${i}.md`));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream removes files');+ await performManagedSync(engine, OPTS);+ const second: any = await performManagedSync(engine, OPTS);+ expect(second.carried).toEqual([expect.objectContaining({ class: 'mirror_database_only', count: 10 })]);+ expect(second.carried[0].sample.length).toBeLessThanOrEqual(5);+ expect(JSON.stringify(second)).not.toContain('--prefer database');+ expect(buildSingleSyncJsonEnvelope('default', second)).toMatchObject({ carried: [expect.objectContaining({ class: 'mirror_database_only', count: 10 })] });+ const status: any = await readSyncStatus(engine, 'default');+ expect(JSON.stringify(status)).not.toContain('--prefer database');+ expect(status.carried).toEqual([expect.objectContaining({ class: 'mirror_database_only', count: 10 })]);+ expect(status.reason).not.toBe('held');+ expect(Number((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE slug LIKE 'keep/k1%' AND deleted_at IS NULL"))[0]!.n)).toBe(11);+ }, seed(25, databaseUrl)), 600_000);+});++test('GST-2515: narrowing a very large set of dangling links to the committed ones never spreads it into one call', () => {+ const links = Array.from({ length: 1_000_000 }, (_, i) => ({ entry: { path: `links/l${i}.md` } }));+ const kept = keepCommittedLinks(links, new Set(links.filter((_, i) => i !== 1).map(link => link.entry.path)));+ expect(kept).toHaveLength(999_999);+ expect(kept[1]!.entry.path).toBe('links/l2.md');+});++describe.each(testBackends())('GST-2515: dangling symlinks (%s)', backend => {+ const databaseUrl = backend === 'postgres' ? requirePostgresTestDatabase() : undefined;+ test('a link to nothing is skipped and counted, not held; a dangling link over a live page stays held', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ mkdirSync(join(root, 'links'), { recursive: true });+ for (let i = 0; i < 6; i++) symlinkSync(join(root, 'nowhere', `t${i}.md`), join(root, 'links', `l${i}.md`));+ rmSync(join(root, 'keep', 'k0.md'));+ symlinkSync(join(root, 'nowhere', 'k0.md'), join(root, 'keep', 'k0.md'));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream adds links');+ // The hold an older gbrain wrote for a dangling link (it read the link as an unreadable file).+ const inc = await incarnation(engine);+ await engine.transaction(tx => writeGitHold(tx, { source_id: 'default', incarnation: inc, path: 'links/l0.md', source_path: 'links/l0.md', slug: 'links/l0', page_id: null,+ code: 'file_unreadable', message: 'links/l0.md: unreadable', upstream_version: null, observed_at: new Date(Date.now() - 60_000).toISOString(), run_id: null, mode: 'managed', meta: {} } as any));+ const result: any = await performManagedSync(engine, OPTS);+ expect(result.skipped?.symlink_dangling?.count).toBe(6);+ expect(buildSingleSyncJsonEnvelope('default', result)).toMatchObject({ skipped: { symlink_dangling: { count: 6 } } });+ const held = await engine.executeRaw<{ path: string }>("SELECT completed_keys->0->>'path' AS path FROM op_checkpoints WHERE op='sync-hold'");+ expect(held.map(h => h.path).filter(p => p.startsWith('links/'))).toEqual([]);+ expect(held.map(h => h.path)).toContain('keep/k0.md');+ expect(Number((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE slug='keep/k0' AND deleted_at IS NULL"))[0]!.n)).toBe(1);+ }, seed(3, databaseUrl)), 600_000);+ test('an uncommitted dangling link over a committed file is not skipped: the committed file is imported or held', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ mkdirSync(join(root, 'fresh'), { recursive: true });+ writeFileSync(join(root, 'fresh', 'b.md'), note('fresh/b', 'Committed body.'));+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream adds b');+ rmSync(join(root, 'fresh', 'b.md'));+ symlinkSync(join(root, 'nowhere', 'b.md'), join(root, 'fresh', 'b.md'));+ const result: any = await performManagedSync(engine, OPTS);+ expect(result.skipped?.symlink_dangling?.count ?? 0).toBe(0);+ const held = (await engine.executeRaw<{ path: string }>("SELECT completed_keys->0->>'path' AS path FROM op_checkpoints WHERE op='sync-hold'")).map(h => h.path);+ const page = Number((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE slug='fresh/b' AND deleted_at IS NULL"))[0]!.n);+ expect(page === 1 || held.includes('fresh/b.md')).toBe(true);+ }, seed(3, databaseUrl)), 600_000);+ test('a dangling link whose target text is the committed file\'s bytes is still not skipped: only a committed symlink is', () => managedBrain(async ({ engine, root }) => {+ await performManagedSync(engine, OPTS);+ mkdirSync(join(root, 'fresh'), { recursive: true });+ const body = note('fresh/c', 'Committed body.');+ writeFileSync(join(root, 'fresh', 'c.md'), body);+ git(root, 'add', '-A'); git(root, 'commit', '-qm', 'upstream adds c');+ rmSync(join(root, 'fresh', 'c.md'));+ symlinkSync(body, join(root, 'fresh', 'c.md'));+ const result: any = await performManagedSync(engine, OPTS);+ expect(result.skipped?.symlink_dangling?.count ?? 0).toBe(0);+ const held = (await engine.executeRaw<{ path: string }>("SELECT completed_keys->0->>'path' AS path FROM op_checkpoints WHERE op='sync-hold'")).map(h => h.path);+ const page = Number((await engine.executeRaw<{ n: number }>("SELECT count(*)::int AS n FROM pages WHERE slug='fresh/c' AND deleted_at IS NULL"))[0]!.n);+ expect(page === 1 || held.includes('fresh/c.md')).toBe(true);+ }, seed(3, databaseUrl)), 600_000);+});diff --git a/test/sync-pass-abandoned-7355.test.ts b/test/sync-pass-abandoned-7355.test.tsindex b3606bbcf1..24d055674e 100644--- a/test/sync-pass-abandoned-7355.test.ts+++ b/test/sync-pass-abandoned-7355.test.ts@@ -135,8 +135,9 @@ for (const backend of testBackends()) {emptyCommits(root, 2_500);unlinkSync(join(root, 'b.md')); git(root, ['commit', '-qam', 'remove b']);__resetHistoryIndexesForTest();- // The previous pass never ended: its row is left open, as the governor or a killed process leaves it.- await engine.executeRaw(`UPDATE op_checkpoints SET completed_keys=jsonb_set(completed_keys,'{0,telemetry,outcome}','"open"') WHERE op=$1`, [MANAGED_SYNC_PASS_OP]);+ // The previous pass never ended: its row is left open, as the governor or a killed process leaves it, and (GST-2515) it+ // committed nothing, so it counts: a fresh pass id that no committed write or hold carries.+ await engine.executeRaw(`UPDATE op_checkpoints SET completed_keys=jsonb_set(jsonb_set(completed_keys,'{0,telemetry,outcome}','"open"'),'{0,telemetry,pass_id}',to_jsonb(gen_random_uuid()::text)) WHERE op=$1`, [MANAGED_SYNC_PASS_OP]);const [{ incarnation }] = await engine.executeRaw<{ incarnation: string }>("SELECT incarnation::text AS incarnation FROM sources WHERE id='default'");let pinnedStreak: number | undefined;installFaultHook(async point => { if (point === 'sync:round_start' && pinnedStreak === undefined) pinnedStreak = (await readPassRow(engine, 'default', incarnation!))?.value.obligations.abandoned_streak; });diff --git a/test/sync-pass-interrupted-7596.test.ts b/test/sync-pass-interrupted-7596.test.tsindex fc21cdc03c..f12e144781 100644--- a/test/sync-pass-interrupted-7596.test.ts+++ b/test/sync-pass-interrupted-7596.test.ts@@ -65,8 +65,9 @@ for (const backend of testBackends()) {const doctor = async () => (await managedSyncBlockedEntry.run({ engine } as never) as Check[])[0]!;expect(await blocked()).toEqual([]);- // The pass's process was killed: its row is left open and its drain lease released (a SIGTERM's cleanup).- await engine.executeRaw(`UPDATE op_checkpoints SET completed_keys=jsonb_set(completed_keys,'{0,telemetry,outcome}','"open"') WHERE op=$1`, [MANAGED_SYNC_PASS_OP]);+ // The pass's process was killed: its row is left open and its drain lease released (a SIGTERM's cleanup). GST-2515: it+ // committed nothing (a fresh pass id no committed write or hold carries), so a streak of such passes is abandoned.+ await engine.executeRaw(`UPDATE op_checkpoints SET completed_keys=jsonb_set(jsonb_set(completed_keys,'{0,telemetry,outcome}','"open"'),'{0,telemetry,pass_id}',to_jsonb(gen_random_uuid()::text)) WHERE op=$1`, [MANAGED_SYNC_PASS_OP]);const [{ pass_id }] = await engine.executeRaw<{ pass_id: string }>(`SELECT completed_keys->0->'telemetry'->>'pass_id' AS pass_id FROM op_checkpoints WHERE op=$1`, [MANAGED_SYNC_PASS_OP]);let status = await readSyncStatus(engine, 'default', { detail: false });expect([status.state, status.reason]).toEqual(['resumable', 'pass_interrupted']);diff --git a/test/sync-pass-readers-7614.test.ts b/test/sync-pass-readers-7614.test.tsindex 8cb99f6f83..56e105c45c 100644--- a/test/sync-pass-readers-7614.test.ts+++ b/test/sync-pass-readers-7614.test.ts@@ -58,8 +58,9 @@ const lockRow = (engine: BrainEngine, beat: string, ttl: string) => engine.execuconst live = (engine: BrainEngine) => lockRow(engine, '1 second', '2 minutes');const dead = (engine: BrainEngine) => lockRow(engine, '3 minutes', '-1 minute');const unlock = (engine: BrainEngine) => engine.executeRaw('DELETE FROM gbrain_cycle_locks WHERE id=$1', [LOCK]);-const setPass = (engine: BrainEngine, outcome: string, streak: number, age = '0 seconds') => engine.executeRaw(`UPDATE op_checkpoints SET completed_keys=jsonb_set(jsonb_set(completed_keys,- '{0,telemetry,outcome}',to_jsonb($2::text)),'{0,obligations,abandoned_streak}',to_jsonb($3::int)), updated_at=now()-$4::interval WHERE op=$1`, [MANAGED_SYNC_PASS_OP, outcome, streak, age]);+// GST-2515: the pass gets a fresh id no committed write or hold carries, so it models a pass that committed nothing.+const setPass = (engine: BrainEngine, outcome: string, streak: number, age = '0 seconds') => engine.executeRaw(`UPDATE op_checkpoints SET completed_keys=jsonb_set(jsonb_set(jsonb_set(completed_keys,+ '{0,telemetry,outcome}',to_jsonb($2::text)),'{0,obligations,abandoned_streak}',to_jsonb($3::int)),'{0,telemetry,pass_id}',to_jsonb(gen_random_uuid()::text)), updated_at=now()-$4::interval WHERE op=$1`, [MANAGED_SYNC_PASS_OP, outcome, streak, age]);const commitFile = (root: string, name: string) => { writeFileSync(join(root, name), `---\ntype: concept\ntitle: ${name}\n---\n\nA note about ${name}.\n`); git(root, ['add', '-A']); git(root, ['commit', '-qm', name]); };for (const backend of testBackends()) {diff --git a/test/unindexed-files-convergence-7149.test.ts b/test/unindexed-files-convergence-7149.test.tsindex ab8ca25870..779f877c03 100644--- a/test/unindexed-files-convergence-7149.test.ts+++ b/test/unindexed-files-convergence-7149.test.ts@@ -152,7 +152,10 @@ test('the maintenance phase serves the least recently scanned source first, so atest('a committed symlink with no page is named symlink, not screened as page content; the next sync holds it, so it ends in a terminal state', () => each(async engine => {const s = await source(engine, 'unidx7149-link', files('notes', 2));await s.sync();- symlinkSync('/nonexistent/elsewhere/target.md', join(s.root, 'notes/link.md'));+ // GST-2515: a link to a file outside the checkout (a link to nothing is skipped by sync; the next test).+ const elsewhere = mkdtempSync(join(tmpdir(), 'unidx7149-elsewhere-'));+ writeFileSync(join(elsewhere, 'target.md'), note('Elsewhere'));+ symlinkSync(join(elsewhere, 'target.md'), join(s.root, 'notes/link.md'));await s.legacy({});const found = await refreshUnindexedFiles(engine, s.id);expect(found.scan).toMatchObject({ complete: true, unindexed: 1, by_code: { symlink: 1 } });@@ -160,6 +163,20 @@ test('a committed symlink with no page is named symlink, not screened as page coawait s.sync();expect(((await readGitSourceHolds(engine, { sourceIds: [s.id] }))[0]?.holds ?? []).map(hold => hold.path)).toEqual(['notes/link.md']);expect((await s.doctor()).details).toMatchObject({ sources: [{ source_id: s.id, state: 'ok', still_unindexed: 0, waiting: 0 }] });+ rmSync(elsewhere, { recursive: true, force: true });+}), 240_000);++test('GST-2515: a committed symlink to nothing is skipped by sync and is not unindexed: no hold, no schedule, doctor ok', () => each(async engine => {+ const s = await source(engine, 'unidx7149-dangling', files('notes', 2));+ await s.sync();+ symlinkSync('/nonexistent/elsewhere/target.md', join(s.root, 'notes/link.md'));+ await s.legacy({});+ const found = await refreshUnindexedFiles(engine, s.id);+ expect(found.scan).toMatchObject({ complete: true, unindexed: 0 });+ expect(found.scheduled).toEqual([]);+ await s.sync();+ expect(((await readGitSourceHolds(engine, { sourceIds: [s.id] }))[0]?.holds ?? []).map(hold => hold.path)).toEqual([]);+ expect((await s.doctor()).details).toMatchObject({ sources: [{ source_id: s.id, state: 'ok', still_unindexed: 0, waiting: 0 }] });}), 240_000);test('a source gets one minute of scan budget per started 100k live pages, at most five', () => {