Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Release writes held behind a refresh that was aborted, instead of leaving them pending
garrytan256ee907bae3
diff --git a/.github/workflows/tla-spec.yml b/.github/workflows/tla-spec.ymlindex 0286c21939..1c1ce5978d 100644--- a/.github/workflows/tla-spec.yml+++ b/.github/workflows/tla-spec.yml@@ -41,6 +41,7 @@ on:- 'test/spec-refresh-barrier.test.ts'- 'test/refresh-write-barrier.test.ts'- 'test/import-write-barrier.test.ts'+ - 'test/refresh-barrier-held-after-abort.test.ts'- 'src/core/persistence/coordinator.ts'- 'src/core/persistence/group-publish.ts'- 'src/core/persistence/page-prepare.ts'@@ -85,6 +86,7 @@ on:- 'test/spec-refresh-barrier.test.ts'- 'test/refresh-write-barrier.test.ts'- 'test/import-write-barrier.test.ts'+ - 'test/refresh-barrier-held-after-abort.test.ts'- 'src/core/persistence/coordinator.ts'- 'src/core/persistence/group-publish.ts'- 'src/core/persistence/page-prepare.ts'diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5017d333fe..00d801e83d 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,16 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**With the refresh write barrier on, a write held behind a refresh that ends without completing is now decided in the run that ends it, instead of waiting for a later refresh.** A refresh's fence keeps managed sync from importing another writer's commit while it runs, and only the refresh's own sync imports in its place. When the refresh was aborted, abandoned or rolled back after its process died, nothing imported that commit, so a write to that file stayed pending until some later refresh or sync, indefinitely while the checkout stayed diverged (up to the 15-minute wait limit). Now, while a checkout's last refresh ended aborted, every `gbrain sources refresh` run on the owner host ends by running the managed `--no-pull` sync of each source behind the checkout HEAD, including a run that refuses `refresh_diverged`. The waiting write is then prepared again against the checkout as it is: a `put_page` with `expected_revision` ends `revision_conflict`, a blind one lands and names what it replaced in `supersedes`, and a `remember` or `add_timeline_entry` applies on top. The barrier is still off by default (`persistence.refresh_barrier`); with it off, the only change is that such a run also syncs sources behind HEAD.++### Itemized changes++- `src/core/persistence/worktree-refresh.ts`: `refreshWorktree` ends every run (returned or refused, not `--dry-run`) with `syncAfterUnfinishedRefresh`. When the worktree's newest refresh row is `aborted`, it runs the bounded member sync for each member whose `last_commit` is not HEAD, after the run's locks are released. The run's `synced` lists those syncs, and a refusal's message names them.+- `spec/tla/refresh-barrier`: new `RefreshBarrierAbortLiveness.cfg` checks liveness with no scheduled sync and no forced expiry (`OutsideHelp = FALSE`). `OwedSyncBegin` models the new sync, and the guard mutant `abort-skips-owed-sync` removes it (`DeferredLeaves` fails; the code edit fails the new tests). `scripts/spec/mutants.ts` names the temporal property that fails by checking each one on its own.+- Tests: `test/refresh-barrier-held-after-abort.test.ts` (PGLite, and Postgres when `DATABASE_URL` is set). It covers a refresh aborted by another writer's commit followed by a diverged retry, a refresh killed at fenced and then abandoned by `--resume`, and an owner-scan abort followed by a diverged tick.+## [0.60.609.0] - 2026-11-07**`POST /ingest` no longer answers 202 for a capture it drops, and an unsigned GitHub webhook can no longer tell from response time which repositories a brain syncs.** The same body sent to two slugs now queues two captures, where the second caller used to get the first job back and nothing written to its slug. A capture over the 50-capture waiting cap now gets 429 `ingest_backlog_full` with `Retry-After: 30` and nothing queued, where it used to get another capture's job id. A slug over 1024 bytes gets 400 `invalid_slug` before anything is queued, well short of the roughly 2.7 KB where no page can be stored under it. Every webhook request now computes the HMAC over its body, whether the repository is unknown, has no secret or is covered by a source, so a 401 takes the same work either way.diff --git a/docs/architecture/key-files/core-persistence-git.md b/docs/architecture/key-files/core-persistence-git.mdindex 96d460d74a..104a170894 100644--- a/docs/architecture/key-files/core-persistence-git.md+++ b/docs/architecture/key-files/core-persistence-git.md@@ -4,7 +4,7 @@Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.The managed Git writer (`gbrain sources refresh`), git-canonical memory (the git gap census, declared database-only classes, the door) and the checkout reads beside them.-- `src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts` — `gbrain sources refresh` (F0), the one sanctioned managed Git writer. `persistence_worktree_refreshes` holds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps to `refresh_in_progress`). `refreshWorktree` prechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardened `git fetch` bounded by `sources.refresh_fetch_timeout_ms`, ancestry, dirty paths intersected with the incoming diff), then: `draining` (inserted under `withTopologyLocks` + `lockTopologyRows`; admission in `journal.ts` refuses `worktree_refreshing` for every request with that `worktree_id`, and `assertManagedSyncAllowed` refuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/running `git`/`withdrawal-mirror` effect and no recovery record remain (else `refresh_drain_timeout`, aborted), `fenced` (both native locks held, rechecked in one transaction; `refreshFenceClear` stops request and effect claims), HEAD verified (#8617: a `refresh_source_changed` for a moved HEAD or work that arrived after the drain check is retried once in the same run from a fresh precheck; a second refusal returns the first with the retry's cause appended), dirt rechecked, `git merge --ff-only`, HEAD verified again (a HEAD that descends from the target after a successful merge, with no member synced off it, counts as merged: a plain-git commit landed in between; a refused fast-forward whose HEAD only gained commits on the start commit aborts `refresh_source_changed`, GST-1674), then one transaction `merged` + upstream observation + `syncing`; native locks are released and the managed `--no-pull` sync runs per member in one budget (`sources.refresh_sync_budget_ms`; past it a member is `deferred`) (only `managed_sync_*` `submit_job` admissions pass during `syncing`); `completed` when every member's `last_commit` is the target or descends from it, or with `outcome.sync_blocked` / `sync_deferred` when a member blocks or is deferred. A per-worktree `refresh-<worktree>.lock` held for the whole run proves liveness: `resumeWorktreeRefreshes` (called from `recoverSourceTopologies` on every consumer topology scan) touches interrupted `draining`/`fenced`/`merged` rows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; HEAD at the verified target moves to `syncing`; anything else is `recovery_required`, which admission refuses with `refresh_recovery_required`). Admission also completes a `syncing` row whose members all reached the target. #7679: each row records its holder (`outcome.holder`: host id, hostname, pid, process start; `previous_holder` after a recovery adopts it). A row whose lock is free is orphaned: `resumeWorktreeRefreshes` also completes an orphaned `syncing` row once every member's `last_commit` reached the target or a descendant, and a plain `gbrain sources refresh` on the owner host finishes any orphaned non-`recovery_required` row through the `--resume` path (`adoptOrphanedRefresh` + `resumeHeld`) before its own refresh, reporting it as `recovered`. A holder recorded on another host is never adopted. `refreshHolderReport` (writer status `worktree_refreshes`, sync status `worktree_refresh`) names the holder, `liveness` (`live` / `orphaned` / `unknown` off the lock's host) and a fix naming `--resume`. `--resume` converges and runs the member syncs; `--abandon` lifts a fence only with HEAD back at the start commit, and never on a `recovery_required` row once a member's `last_commit` is past it (not it or an ancestor; GST-1457: the next sync would soft-delete what it synced), and `--resume` finishes a `recovery_required` row only at a HEAD that contains every member commit on that refresh's line. `gbrain sources writer status` lists active refreshes as `worktree_refreshes` (state, heads, the next command). Doctor `worktree_refresh_stuck` lists rows active over 15 minutes. CLI body: `src/commands/sources-refresh.ts` (#7685: `--json` prints a refusal's structured `fix` when it has one; `refresh_in_progress` carries the writer-status read); while a resident PGLite owner (`gbrain serve`, stdio MCP) holds the database, `src/commands/sources-refresh-delegate.ts` runs the refresh inside it through the `writer_refresh` administration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe applies `refreshFenceClear` to effects like the claims, so a fenced worktree's queued effects do not wake it. Tests: `test/persistence-worktree-refresh.test.ts`, `test/persistence-worktree-refresh-restart.test.ts` (child-process crash after merge), `test/worktree-refresh-orphan-7679.test.ts` (SIGKILL mid-syncing, live-holder no-steal), `test/oversize-hold-rescreen-8072.test.ts` (budget), Postgres via `test/e2e/worktree-refresh-*postgres.test.ts`, owner delegation in `test/persistence-sync-stdio-owner.test.ts`.+- `src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts` — `gbrain sources refresh` (F0), the one sanctioned managed Git writer. `persistence_worktree_refreshes` holds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps to `refresh_in_progress`). `refreshWorktree` prechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardened `git fetch` bounded by `sources.refresh_fetch_timeout_ms`, ancestry, dirty paths intersected with the incoming diff), then: `draining` (inserted under `withTopologyLocks` + `lockTopologyRows`; admission in `journal.ts` refuses `worktree_refreshing` for every request with that `worktree_id`, and `assertManagedSyncAllowed` refuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/running `git`/`withdrawal-mirror` effect and no recovery record remain (else `refresh_drain_timeout`, aborted), `fenced` (both native locks held, rechecked in one transaction; `refreshFenceClear` stops request and effect claims), HEAD verified (#8617: a `refresh_source_changed` for a moved HEAD or work that arrived after the drain check is retried once in the same run from a fresh precheck; a second refusal returns the first with the retry's cause appended), dirt rechecked, `git merge --ff-only`, HEAD verified again (a HEAD that descends from the target after a successful merge, with no member synced off it, counts as merged: a plain-git commit landed in between; a refused fast-forward whose HEAD only gained commits on the start commit aborts `refresh_source_changed`, GST-1674), then one transaction `merged` + upstream observation + `syncing`; native locks are released and the managed `--no-pull` sync runs per member in one budget (`sources.refresh_sync_budget_ms`; past it a member is `deferred`) (only `managed_sync_*` `submit_job` admissions pass during `syncing`); `completed` when every member's `last_commit` is the target or descends from it, or with `outcome.sync_blocked` / `sync_deferred` when a member blocks or is deferred. A per-worktree `refresh-<worktree>.lock` held for the whole run proves liveness: `resumeWorktreeRefreshes` (called from `recoverSourceTopologies` on every consumer topology scan) touches interrupted `draining`/`fenced`/`merged` rows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; HEAD at the verified target moves to `syncing`; anything else is `recovery_required`, which admission refuses with `refresh_recovery_required`). Admission also completes a `syncing` row whose members all reached the target. #7679: each row records its holder (`outcome.holder`: host id, hostname, pid, process start; `previous_holder` after a recovery adopts it). A row whose lock is free is orphaned: `resumeWorktreeRefreshes` also completes an orphaned `syncing` row once every member's `last_commit` reached the target or a descendant, and a plain `gbrain sources refresh` on the owner host finishes any orphaned non-`recovery_required` row through the `--resume` path (`adoptOrphanedRefresh` + `resumeHeld`) before its own refresh, reporting it as `recovered`. A holder recorded on another host is never adopted. `refreshHolderReport` (writer status `worktree_refreshes`, sync status `worktree_refresh`) names the holder, `liveness` (`live` / `orphaned` / `unknown` off the lock's host) and a fix naming `--resume`. While the worktree's newest refresh is `aborted`, every run (tick, `--resume`, `--abandon`, even one its precheck refuses) ends by running the managed `--no-pull` sync of each member behind the checkout HEAD (`syncAfterUnfinishedRefresh`, listed in `synced`), so a write held for an import that refresh fenced off is decided in that run, not at a later refresh. `--resume` converges and runs the member syncs; `--abandon` lifts a fence only with HEAD back at the start commit, and never on a `recovery_required` row once a member's `last_commit` is past it (not it or an ancestor; GST-1457: the next sync would soft-delete what it synced), and `--resume` finishes a `recovery_required` row only at a HEAD that contains every member commit on that refresh's line. `gbrain sources writer status` lists active refreshes as `worktree_refreshes` (state, heads, the next command). Doctor `worktree_refresh_stuck` lists rows active over 15 minutes. CLI body: `src/commands/sources-refresh.ts` (#7685: `--json` prints a refusal's structured `fix` when it has one; `refresh_in_progress` carries the writer-status read); while a resident PGLite owner (`gbrain serve`, stdio MCP) holds the database, `src/commands/sources-refresh-delegate.ts` runs the refresh inside it through the `writer_refresh` administration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe applies `refreshFenceClear` to effects like the claims, so a fenced worktree's queued effects do not wake it. Tests: `test/persistence-worktree-refresh.test.ts`, `test/persistence-worktree-refresh-restart.test.ts` (child-process crash after merge), `test/worktree-refresh-orphan-7679.test.ts` (SIGKILL mid-syncing, live-holder no-steal), `test/oversize-hold-rescreen-8072.test.ts` (budget), `test/refresh-barrier-held-after-abort.test.ts` (the sync after an aborted refresh), Postgres via `test/e2e/worktree-refresh-*postgres.test.ts`, owner delegation in `test/persistence-sync-stdio-owner.test.ts`.- `src/core/persistence/refresh-barrier.ts` (+ deferred admission and claim in `journal.ts`, migration v292 `src/core/schema-migrations/v292-refresh-write-barrier.ts`) — the refresh write barrier (`docs/designs/REFRESH_WRITE_BARRIER.md`, `docs/guides/write-refusals.md#deferred-writes`). Off by default (`persistence.refresh_barrier`). With it on, a single page or memory write that reaches a worktree during a refresh is journaled in state `deferred` (tied to `deferred_refresh_id`) and answered pending instead of refused `worktree_refreshing`. Only a claim that declares the barrier moves a deferred row, and only after its refresh is released. `deferredReplayRefusal` re-checks the write's authority and reads the refresh's `old_head..target_head` diff with git, so a file the refresh changed ends `revision_conflict` with nothing applied. A write still waiting past `persistence.refresh_barrier_max_wait_ms` (15 minutes) ends `refresh_barrier_expired`. A write during a refresh in `recovery_required` defers too, and since GST-1582 there is no per-refresh cap (`persistence.refresh_barrier_max_deferred` is no longer read). A write whose page file holds a committed change sync has not imported yet is held with `deferred_import_from` (migration v294) instead of refused `pending_import`, and ends `pending_import_expired` past the wait limit (or four wait limits, at least four hours, after admission when it is held again and again). GST-1623: either kind of deferred write ends `revision_conflict` when a later write to its page committed first, and a refresh-deferred write is checked against its refresh before an import hold, which then clears `deferred_refresh_id`. `refresh-retry-hint.ts` derives `worktree_refreshing`'s `retry_after_ms` from the worktree's refresh history. `gbrain upgrade --check-downgrade` exits 3 `downgrade_blocked_deferred_writes` while any deferred write is unfinished. Tests: `test/refresh-write-barrier.test.ts`, `test/refresh-barrier-state-audit.test.ts`.- `src/core/persistence/git-gap.ts` + `src/commands/sources-git-gap.ts` + `src/commands/doctor/checks/db-ahead-of-git.ts` — git-canonical memory, wave G1 (`docs/designs/GIT_CANONICAL_MEMORY.md#the-git-gap-census`), read-only. `declaredDatabaseOnly` is the one predicate for pages that may not have a tracked file, tested in order: a read-only mirror source, a private page (`isPrivatePage`), an alias-private page (`isDerivationPrivate`), lineage-private (computed in SQL from `privatePagesFilterFragment`), `database_only_reason`, a `storage.db_only` prefix (`isSourceDbOnlySlug`, refusing an invalid gbrain.yml), a never-filed derive-phase page (page-prepare.ts `isNeverFiledDerivedPage`: no `source_path`/`source_uri` under `DERIVE_PHASE_DB_ONLY_DEFAULTS`), a path the checkout's Git ignores (`ignoredPaths`: filter-free `check-ignore -z --stdin`, tracked files never reported, exit 1 = none, null when Git cannot answer). The equality contract compares a committed blob with the page's export view (`readExportPage`, withdrawals overlaid; the census reads a batch's views with `readExportPages`, the same snapshots in two statements, #7617): an inline blob parsed as reconcile parses a canonical file, aligned with the stored type, subtype and #6429 stamp and compared by `reconcileCanonical` digest; a segmented page by the Git blob id of its streamed export. HEAD is read with hardened plumbing only (`readHeadTree`: one `ls-tree -r -l -z --full-tree` under the source's directory; `readBlobs`: `cat-file --batch` in calls of at most 64 MiB) and the working tree by hashing raw bytes (`worktreeBlobOid`), never `git status`/`git diff`, so no repository filter, textconv or fsmonitor runs. `gbrain sources git-gap <id> [--sample n] [--json]` (trusted local CLI, not an MCP op) puts every live page in one state (`equal`, `database_ahead`, `git_ahead` when the sync import record's `blob_oid`/`disk_blob_oid` differs from HEAD, else folded into `database_ahead` and counted in `database_ahead_without_import_record`, split by content evidence in `direction_evidence`, `declared_database_only` per class, `undeclared_database_only`, `worktree_only`, `unparseable`, `unsafe_path`), reports the working tree separately, and counts rows with no Git home (unfenced active facts, fenced facts on a missing page or a page without a committed file, links whose `link_source` is not in `DERIVED_LINK_SOURCES`, timeline rows on pages without a file), open `sync-hold` records by code and parked effect targets by kind and code. Each 256-page batch reads in its own read-only transaction. Doctor `db_ahead_of_git` runs the census per non-archived source whose directory is a Git checkout (others are only counted) under `GBRAIN_DOCTOR_GIT_GAP_TIMEOUT_MS` (default 10 s) and the entry deadline; it is always `ok` (informational) and lists totals only with each source's `live_pages` (counted in SQL before the scan), a source the budget stopped or never reached as partial (`PARTIAL, <read> of <live> live page(s) read`). Tests: `test/git-gap-census.test.ts` (PGLite and Postgres).- `src/core/persistence/git-declared.ts` + `src/core/persistence/git-canonical-schema.ts` (migration `git_canonical_declared`) — git-canonical memory, wave G7a: the declared database-only classes of a strict source (`sources.config.canonical = "git"`, default off). `strictPageClass` judges a page by its slug and prior file (`declaredPathClass`: `db_only_prefix` for a `storage.db_only` dir, `derive_folder` for a never-filed derive-phase page; G3's indexer reads the same function) and then its rendered content (`privacyClass`: `private`, `alias_private`, or `lineage_private` from the same SQL remote reads use, `privatePagesFilterFragment`, over the page's own values). `page-prepare.ts` `prepareFileTarget` asks it before it builds a strict file target: a declared page gets no target (no file, no door commit, no Git effect), an untracked ignored target is `git_ignored`, a page turning private gets the removal of the file it still has in the worktree or at HEAD, and a path whose `filter` attribute is `lfs` passes `assertLfsTarget` only with `storage.git_lfs` and a checkout that configures the LFS filter (`git_lfs_not_enabled`). `preparePrivateTransition` (page-prepare.ts) is that removal for other writers (GBRA-60, connector imports #7971). In `publishMutation` and `publishGroup`, `assertNoPrivateCommit` refuses a file target carrying a private page's bytes before anything is written (`git_private_commit_refused`; a group rolls back whole), and `declareStrictPublication` (after apply, before the door commit) refuses a declared page's file or an undeclared file-less page (`git_home_required`), stamps `pages.database_only_reason` and, for a privacy class, clears `source_path`/`source_uri` in the transaction of the door commit that removed the file. A managed import builds its own file target, so it meets only these publication checks. `assertStrictGitHome` is the refusal for writers with no file form yet (G4, G5). The managed writer guard (`writer-guard-schema.ts`) refuses rows of a strict source unless the writer declares `gbrain.git_protocol` (`withCoordinatedWrite` does), behind the `git_canonical_brain.strict_sources` counter kept by `sources` triggers (`git_protocol_required`, mapped in `publication-failure.ts`). The guard reads its state as the writing role and fails closed: a hidden or missing `persistence_brain` or `git_canonical_brain` row, or a source or page row hidden by row-level security, raises `writer_coordinator_required` (`managed_writer_guard:unreadable_state`), and the counter refuses a recount under RLS on `sources` (`test/writer-fence-fail-closed-7636.test.ts`). The census (`git-gap.ts`) adds `database_only_source` (a source with no checkout) and maps the stamped reasons to their own classes. Tests: `test/git-canonical-declared-g7a.test.ts` (both engines).diff --git a/docs/designs/REFRESH_WRITE_BARRIER.md b/docs/designs/REFRESH_WRITE_BARRIER.mdindex 5b665ca949..182fe66820 100644--- a/docs/designs/REFRESH_WRITE_BARRIER.md+++ b/docs/designs/REFRESH_WRITE_BARRIER.md@@ -222,6 +222,20 @@ barrier exists to keep, and with the flag on each now waits instead (with it off`deferred_refresh_id`, so a consumer from before v294 never claims a row whose import is pending. The receipt reads `queued` with`deferred: { pending_import: true }`, and the caller's wait returns pending at once.+**After a refresh that did not complete.** While a refresh runs, its fence refuses the managed sync that would import+another writer's commit, and only the refresh's own member sync imports in its place. A refresh that ends `aborted`+(in its own run, by `--abandon`, or rolled back by `--resume` or the owner's scan after its process died) never runs+it, so a write released from it and then held for that import would wait for some later sync or refresh, indefinitely+while the checkout stays diverged. GST-1853: while the worktree's newest refresh is `aborted`, every refresh run on the+owner host (a tick, `--resume`, `--abandon`, even one its precheck refuses `refresh_diverged`) ends, after its own+locks are released, by running the managed `--no-pull` sync of each member whose `last_commit` is not the checkout+HEAD, within `sources.refresh_sync_budget_ms` (`syncAfterUnfinishedRefresh`; the run lists them in `synced`, and a+refusal's message names them). The held writes are then prepared again against the checkout as it is, as above: an+`expected_revision` put ends `revision_conflict`, a blind put or capture lands and names what it replaced, an append+applies on top. Releasing them this way, instead of refusing them, gives the caller an answer within the run, and a+refusal would only be held again on retry while the import stays pending. A member this sync cannot bring to HEAD (its+drain blocked) waits for the next run or sync, and its held writes stay bound by the wait limit below.+**What a blind write replaced.** A held `put_page` or `capture` that replaces content an import brought in namesit in three places, so the replacement is auditable and recoverable without digging: the receipt's`supersedes` (`slug`, `path`, the replaced `revision`, the imported `commit`: the newest commitdiff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.mdindex 298d363081..a152cc3385 100644--- a/docs/guides/write-refusals.md+++ b/docs/guides/write-refusals.md@@ -633,6 +633,11 @@ syncing it is accepted and waits for the refresh instead of being refused `worktfirst sync that imports the file replays it; otherwise `gbrain repair failed-writes --source <source>`lists it for a person to replay. A write keeps the refusal when a live consumer on the owner host runs a releasethat cannot claim a held write.+- A refresh that ends without completing (aborted, abandoned, or rolled back after its process died) never imports+ the commit its fence kept sync from importing, so the next `gbrain sources refresh` run on the owner host (the+ same run, `--resume` or `--abandon`, or the next tick, even one refused `refresh_diverged`) ends by running the+ managed `--no-pull` sync of each member behind the checkout HEAD. A write held for that import is then prepared+ again as above in that run, instead of waiting for a later refresh.- A write that waited behind a refresh or for an import ends `revision_conflict`, nothing applied, when alater write to the same page committed before it, which a consumer from an older release can cause byclaiming the later write first. Read the page again and submit the change with a new request ID. Adiff --git a/scripts/spec/mutants.ts b/scripts/spec/mutants.tsindex f7a4b872e8..24920513c1 100644--- a/scripts/spec/mutants.ts+++ b/scripts/spec/mutants.ts@@ -66,15 +66,26 @@ function runSpec(m: Mutant, tla2tools: string): void {const mutated = applyOnce(readFileSync(target, 'utf8'), m.spec);if (mutated === null) return fail('MUTANT_STALE', m.id, `spec text not found exactly once in ${m.spec.file}: ${JSON.stringify(m.spec.before)}`);writeFileSync(target, mutated);- const r = spawnSync('java', ['-XX:+UseParallelGC', `-Djava.io.tmpdir=${join(work, 'tmp')}`, '-cp', tla2tools, 'tlc2.TLC', '-workers', 'auto',- '-config', config, '-metadir', join(work, 'states'), `${catalog.module}.tla`], { cwd: work, encoding: 'utf8', timeout: timeoutS * 1000 });+ const tlc = (cfg: string) => spawnSync('java', ['-XX:+UseParallelGC', `-Djava.io.tmpdir=${join(work, 'tmp')}`, '-cp', tla2tools, 'tlc2.TLC', '-workers', 'auto',+ '-config', cfg, '-metadir', join(work, `states-${cfg}`), `${catalog.module}.tla`], { cwd: work, encoding: 'utf8', timeout: timeoutS * 1000 });+ const r = tlc(config);const log = `${r.stdout ?? ''}${r.stderr ?? ''}`;writeFileSync(join(work, 'tlc.log'), log);const spawnCode = (r.error as NodeJS.ErrnoException | undefined)?.code;if (spawnCode === 'ENOENT') return fail('MUTANT_INCOMPLETE', m.id, `cannot run java (ENOENT); TLC did not start. Install Java 11 or later on PATH and rerun: bun scripts/spec/mutants.ts --only ${m.id} --spec-only`);if (r.error || r.signal) return fail('MUTANT_INCOMPLETE', m.id, `TLC did not finish within TLC_TIMEOUT_S (${timeoutS} s) or was killed (${spawnCode ?? r.signal}). If the spec or config changed, find the action that multiplies states; otherwise rerun once. Log: ${work}/tlc.log`);checked++;- const violated = log.match(/(?:Invariant|Action property|Property) (\w+) is violated/)?.[1];+ let violated = log.match(/(?:Invariant|Action property|Property) (\w+) is violated/)?.[1];+ // TLC names no property when a temporal one fails: check the config's properties one at a time to name it.+ if (!violated && log.includes('Temporal properties were violated.')) {+ const text = readFileSync(join(work, config), 'utf8');+ for (const property of [...text.matchAll(/^PROPERTY (\w+)$/gm)].map(match => match[1]!)) {+ const single = `${property}-${config}`;+ writeFileSync(join(work, single), text.replace(/^PROPERTY \w+\n/gm, '') + `PROPERTY ${property}\n`);+ const each = tlc(single);+ if (`${each.stdout ?? ''}${each.stderr ?? ''}`.includes('Temporal properties were violated.')) { violated = property; break; }+ }+ }const clean = log.includes('Model checking completed. No error has been found.');if (m.expect === 'survives') {if (clean) return void console.log(`ok ${m.id} (survivor): survives as expected (defense in depth)`);diff --git a/spec/tla/README.md b/spec/tla/README.mdindex c31cbe348f..8d20938e7c 100644--- a/spec/tla/README.md+++ b/spec/tla/README.md@@ -164,6 +164,7 @@ invariant. Behind constants it also models the stacked follow-ups: writes held f| `RefreshBarrierLiveness.cfg` | `FairSpec`, one write: every deferred write leaves deferral and every admitted write ends. `RefreshBarrierLivenessNightly.cfg` checks two writes (nightly). || `RefreshBarrierAdopt.cfg` | #8708 on #8705 with its fixes: an uncommitted local edit adopted by a drift refusal, no foreign commit. `RefreshBarrierAdoptNightly.cfg` adds the foreign commit (nightly). || `RefreshBarrierImport.cfg` | #8705 with its fixes (03b89121): writes held for an import, a later write held beside them, the import expiry, deferral behind `recovery_required`: every property. |+| `RefreshBarrierAbortLiveness.cfg` | `FairSpec` without outside help (`OutsideHelp = FALSE`): no scheduled sync runs and time never forces the wait limit, and the foreign commit lands while a refresh drains, so the refresh aborts and a write deferred behind it is held for that import. Every deferred write still leaves deferral and every write ends, through the sync a refresh run owes after an aborted refresh (GST-1853). |Every config kills the refreshing process once (`MaxCrashes = 1`), at any step, after which `recoverOne` resumes therefresh row it left. A refresh that finds HEAD moved at the merge aborts and the tick retries, as `fenceAndMerge` does,@@ -207,6 +208,7 @@ every config killing the refresh at least once, and the workflow's path filter c| `ExpRefBegin`, `ExpRefCommit` | `expireDeferredWrites`: the protocol, the refresh row re-read `released_at IS NULL FOR NO KEY UPDATE`, `completeWrite` || `ExpImpBegin`, `ExpImpRecheck`, `ExpImpCommit` | #8705's import expiry in `expireDeferredWrites` || `SyncBegin` … `SyncStamp` | a managed sync pass, whose rows go through the claim like writes: no write passes the queued row importing its page or a running sync row, and a sync row waits behind earlier writes and is never claimed while a write runs; the page row, then its checkpoint row's `UPDATE sources SET last_commit` (sync-prepare.ts) |+| `OwedSyncBegin` | GST-1853 `syncAfterUnfinishedRefresh` (worktree-refresh.ts): while the worktree's newest refresh is aborted and a member's `last_commit` is not HEAD, a refresh run ends by running that member's managed `--no-pull` sync || `Foreign`, `LocalEdit`, `Age`, `Revoke`, `Restore` | the environment: another agent's plain-git commit, an uncommitted edit, the wait limit passing, a revoked token, `quarantineSharedSkillRestore` (PGLite-only, one transaction, no worker) |### Mutants@@ -234,6 +236,7 @@ every config killing the refresh at least once, and the workflow's path filter c| `import-expiry-skips-source-lock` | guard | `ExpiryOnlyWhileWaiting` (Import) | the import expiry's `FOR SHARE OF s`; a Postgres import landing while the expiry waits (import-write-barrier.test.ts) || `resume-marks-unmerged-recovery` | guard | `RecoveryOnlyByPolicy` | `movedBeforeMerge` (the finding below): the code edit aborts only a HEAD moved by gbrain's own commits; the refresh killed at fenced after a foreign commit, aborted by the owner scan || `resume-needs-exact-target` | guard | `RecoveryOnlyByPolicy` | `recoverOne`'s bookkeeping branch: with HEAD only exactly at the target, a commit between the merge and bookkeeping strands the row; the refresh killed after its merge |+| `abort-skips-owed-sync` | guard | `DeferredLeaves` (AbortLiveness) | `syncAfterUnfinishedRefresh`: no refresh run syncs the members an aborted refresh left behind HEAD; the three GST-1853 tests (aborted then diverged, killed at fenced then `--resume`, owner-scan abort then a diverged tick) |#8708's local-edit adoption has no guard mutant of its own: `RefreshBarrierAdopt.cfg` checks it against every safety property.@@ -275,6 +278,17 @@ test/import-write-barrier.test.ts. The mutants below put each one back.protocol and then reads the source row `FOR SHARE` (`ImportExpiryLocksSource`); taking the source row first deadlockswith `fenceDraining`'s brain, worktree, source order (`import-expiry-locks-source-first`).+### Found in GST-1853: writes held behind an aborted refresh++A flag-on eval found it, and `RefreshBarrierAbortLiveness.cfg` reproduces it (`abort-skips-owed-sync`, `DeferredLeaves`).+Another agent commits F with plain git while a refresh drains, so the fence keeps managed sync from importing it and+the moved HEAD aborts the refresh. The abort releases the write deferred behind it; preparing it meets the unimported+commit (`pending_import`) and holds it for the import. Only a refresh's own member sync imports while sync is fenced,+and an aborted refresh never runs one, so the write waited for some later sync or refresh (indefinitely while the+checkout stayed diverged) or the wait limit. The other configs never saw it because their fairness lets a scheduled sync+run and time pass the wait limit; this one forces neither. The fix: a refresh run that finds the worktree's newest+refresh aborted runs the members' managed `--no-pull` sync on its way out (`OwedSyncBegin`).+### Not modelledThe removed deferred-write cap (`persistence.refresh_barrier_max_deferred`, no longer read since GST-1582), consumers without the capability, repo-lessdiff --git a/spec/tla/refresh-barrier/RefreshBarrier.cfg b/spec/tla/refresh-barrier/RefreshBarrier.cfgindex 315b59793e..a27d939ed5 100644--- a/spec/tla/refresh-barrier/RefreshBarrier.cfg+++ b/spec/tla/refresh-barrier/RefreshBarrier.cfg@@ -17,6 +17,7 @@ CONSTANTSMayRevoke = FALSEMaxCrashes = 1MayReset = TRUE+ OutsideHelp = TRUEINVARIANT TypeOKINVARIANT NoDeadlockINVARIANT AppliedAtMostOncediff --git a/spec/tla/refresh-barrier/RefreshBarrier.tla b/spec/tla/refresh-barrier/RefreshBarrier.tlaindex 44d7bf9a76..e314905e4e 100644--- a/spec/tla/refresh-barrier/RefreshBarrier.tla+++ b/spec/tla/refresh-barrier/RefreshBarrier.tla@@ -51,7 +51,9 @@ CONSTANTSMayRestore, \* a shared-skill restore may run onceMayRevoke, \* the writers' credential may be revoked onceMaxCrashes, \* times the refreshing process may be killed; the owner's topology scan or --resume then recovers it- MayReset \* someone may reset the checkout to old_head after a merge, before bookkeeping+ MayReset, \* someone may reset the checkout to old_head after a merge, before bookkeeping+ OutsideHelp \* time passes the wait limit and an outside scheduled sync runs, eventually; FALSE (GST-1853): neither+ \* is forced, and another agent commits only while a refresh drains (the commit its fence keeps sync from importing)Writes == 1..NWRefs == 1..MaxRefreshes@@ -600,6 +602,7 @@ BkCommit ==\* syncMembers ends: completed once members reach the target, or (#8072) when a member's sync is blocked or deferred.RefComplete ==/\ pc[RFR] = "r_syncing"+ /\ IF OutsideHelp THEN TRUE ELSE lastCommit = head \* without outside help, the refresh's own member sync is what imports/\ Need(RFR, <<L(REF(nref), "NKU")>>) /\ Go(RFR, "r_complete")/\ UNCHANGED <<reqv, refv, gitv, envv, ghostv, held, cw, cprep, cfin, ew, aref, aread, passHead, sseq>>@@ -708,6 +711,18 @@ SyncBegin ==/\ UNCHANGED <<st, code, seq, dref, dimp, applied, cpos, ncommit, expRev, overdue, adopted, refv, gitv, envv, ghostv,lockv, cw, cprep, cfin, ew, aref, aread>>+\* GST-1853 (syncAfterUnfinishedRefresh): a refresh that ended aborted never ran the member sync its fence refused, so while+\* the newest refresh is aborted and last_commit is not HEAD, a refresh run ends by running that sync: the tick that aborted+\* it, --resume or --abandon, or the next tick after the owner scan aborted it (even one its precheck refuses).+OwedSyncBegin ==+ /\ pc[RFR] = "idle" /\ cur = None /\ nref > 0 /\ rstate[nref] = "aborted"+ /\ pc[SYN] = "idle" /\ lastCommit # head+ /\ passHead' = head+ /\ sseq' = nseq + 1 /\ nseq' = nseq + 1+ /\ Go(SYN, "s_queued")+ /\ UNCHANGED <<st, code, seq, dref, dimp, applied, cpos, ncommit, expRev, overdue, adopted, refv, gitv, envv, ghostv,+ lockv, cw, cprep, cfin, ew, aref, aread>>+\* The page row is claimed once no earlier write is ahead of it (EARLIER_HEAD), then imported.SyncImportBegin ==/\ pc[SYN] = "s_queued" /\ ~SyncBehind@@ -739,6 +754,7 @@ SyncStamp ==\* ------------------------------- environment -------------------------------Foreign ==/\ nforeign < MaxForeign /\ local = None+ /\ IF OutsideHelp THEN TRUE ELSE RefIs({"draining"})/\ NewCommit(nver + 1) /\ nver' = nver + 1 /\ nforeign' = nforeign + 1/\ UNCHANGED <<reqv, refv, dbV, hist, lastCommit, local, nlocal, envv, ghostv, lockv, procv>>@@ -781,7 +797,8 @@ ConsStep == ConsBegin \/ ConsSelect \/ ConsClaim \/ ConsPrep \/ ConsPub \/ ConsHRefStep == FdCommit \/ RefDrained \/ RefDrainTimeout \/ FmCommit \/ AbortCommit \/ RefMerge \/ MergeStartCommit \/ FfMerge\/ MarkCommit \/ BkCommit \/ RefComplete \/ CompleteCommit \/ RecoverBegin \/ RecoverCommitExpStep == (\E w \in Writes : ExpRefBegin(w) \/ ExpImpBegin(w)) \/ ExpRefCommit \/ ExpImpRecheck \/ ExpImpCommit-SyncStep == SyncBegin \/ SyncImportBegin \/ SyncImport \/ SyncStampBegin \/ SyncStamp+SyncPass == SyncImportBegin \/ SyncImport \/ SyncStampBegin \/ SyncStamp+SyncStep == SyncBegin \/ OwedSyncBegin \/ SyncPassAdmStep(w) == AdmRef(w) \/ AdmRead(w) \/ AdmCommit(w)Next ==@@ -791,13 +808,15 @@ Next ==\/ Foreign \/ LocalEdit \/ Revoke \/ Restore \/ Crash \/ ResetAfterMerge\* Every process that has started runs; a waiting lock request is eventually granted once it can be (Postgres queues-\* lock waiters); a person eventually resolves recovery_required; time passes. Producers, other agents' commits,-\* local edits, refreshes starting, revocation and restore are never forced.+\* lock waiters); a person eventually resolves recovery_required; a refresh run ends with the sync an aborted refresh+\* owes; with OutsideHelp, time passes and a scheduled sync runs. Producers, other agents' commits, local edits,+\* refreshes starting, revocation and restore are never forced.Fairness ==/\ \A p \in Procs : SF_vars(Acquire(p))- /\ \A w \in Writes : WF_vars(AdmStep(w)) /\ WF_vars(Age(w))+ /\ \A w \in Writes : WF_vars(AdmStep(w)) /\ (IF OutsideHelp THEN WF_vars(Age(w)) ELSE TRUE)/\ WF_vars(ConsStep) /\ WF_vars(RefStep) /\ WF_vars(PersonAbandon \/ PersonResume)- /\ WF_vars(ExpStep) /\ WF_vars(SyncStep)+ /\ WF_vars(ExpStep) /\ WF_vars(OwedSyncBegin \/ SyncPass) /\ WF_vars(SyncBegin /\ RefIs({"syncing"}))+ /\ IF OutsideHelp THEN WF_vars(SyncBegin) ELSE TRUESpec == Init /\ [][Next]_varsFairSpec == Spec /\ Fairnessdiff --git a/spec/tla/refresh-barrier/RefreshBarrierAbortLiveness.cfg b/spec/tla/refresh-barrier/RefreshBarrierAbortLiveness.cfgnew file mode 100644index 0000000000..ae21d01da8--- /dev/null+++ b/spec/tla/refresh-barrier/RefreshBarrierAbortLiveness.cfg@@ -0,0 +1,24 @@+\* GST-1853: liveness with no outside help. Time never forces the wait limit and no scheduled sync runs; another agent+\* commits F while a refresh drains (the commit its fence keeps sync from importing), so the refresh aborts and a write+\* deferred behind it is held for that import. Every deferred write still leaves deferral and every write ends, through the+\* sync a refresh run owes after an aborted refresh (mutant abort-skips-owed-sync removes it).+SPECIFICATION FairSpec+CHECK_DEADLOCK FALSE+CONSTANTS+ NW = 1+ Blind = {1}+ MaxRefreshes = 1+ MaxForeign = 1+ MaxLocalEdits = 0+ ImportHolds = TRUE+ HoldBehindHeld = TRUE+ ImportExpiryLocksSource = TRUE+ RecoveryDefers = TRUE+ AdoptLocalEdits = FALSE+ MayRestore = FALSE+ MayRevoke = FALSE+ MaxCrashes = 1+ MayReset = FALSE+ OutsideHelp = FALSE+PROPERTY DeferredLeaves+PROPERTY EveryWriteEndsdiff --git a/spec/tla/refresh-barrier/RefreshBarrierAdopt.cfg b/spec/tla/refresh-barrier/RefreshBarrierAdopt.cfgindex c11f4222d5..4f846719a5 100644--- a/spec/tla/refresh-barrier/RefreshBarrierAdopt.cfg+++ b/spec/tla/refresh-barrier/RefreshBarrierAdopt.cfg@@ -17,6 +17,7 @@ CONSTANTSMayRevoke = FALSEMaxCrashes = 1MayReset = FALSE+ OutsideHelp = TRUEINVARIANT TypeOKINVARIANT NoDeadlockINVARIANT AppliedAtMostOncediff --git a/spec/tla/refresh-barrier/RefreshBarrierAdoptNightly.cfg b/spec/tla/refresh-barrier/RefreshBarrierAdoptNightly.cfgindex 7f9fedfb0e..0f7e95cd27 100644--- a/spec/tla/refresh-barrier/RefreshBarrierAdoptNightly.cfg+++ b/spec/tla/refresh-barrier/RefreshBarrierAdoptNightly.cfg@@ -16,6 +16,7 @@ CONSTANTSMayRevoke = FALSEMaxCrashes = 1MayReset = FALSE+ OutsideHelp = TRUEINVARIANT TypeOKINVARIANT NoDeadlockINVARIANT AppliedAtMostOncediff --git a/spec/tla/refresh-barrier/RefreshBarrierEnv.cfg b/spec/tla/refresh-barrier/RefreshBarrierEnv.cfgindex ad17aaa18b..05e253548c 100644--- a/spec/tla/refresh-barrier/RefreshBarrierEnv.cfg+++ b/spec/tla/refresh-barrier/RefreshBarrierEnv.cfg@@ -16,6 +16,7 @@ CONSTANTSMayRevoke = TRUEMaxCrashes = 1MayReset = FALSE+ OutsideHelp = TRUEINVARIANT TypeOKINVARIANT NoDeadlockINVARIANT AppliedAtMostOncediff --git a/spec/tla/refresh-barrier/RefreshBarrierImport.cfg b/spec/tla/refresh-barrier/RefreshBarrierImport.cfgindex 3361f3d2d3..55fdd5997c 100644--- a/spec/tla/refresh-barrier/RefreshBarrierImport.cfg+++ b/spec/tla/refresh-barrier/RefreshBarrierImport.cfg@@ -18,6 +18,7 @@ CONSTANTSMayRevoke = FALSEMaxCrashes = 1MayReset = FALSE+ OutsideHelp = TRUEINVARIANT TypeOKINVARIANT NoDeadlockINVARIANT AppliedAtMostOncediff --git a/spec/tla/refresh-barrier/RefreshBarrierLiveness.cfg b/spec/tla/refresh-barrier/RefreshBarrierLiveness.cfgindex 41fd7c268c..81fd5fa12c 100644--- a/spec/tla/refresh-barrier/RefreshBarrierLiveness.cfg+++ b/spec/tla/refresh-barrier/RefreshBarrierLiveness.cfg@@ -17,5 +17,6 @@ CONSTANTSMayRevoke = FALSEMaxCrashes = 1MayReset = FALSE+ OutsideHelp = TRUEPROPERTY DeferredLeavesPROPERTY EveryWriteEndsdiff --git a/spec/tla/refresh-barrier/RefreshBarrierLivenessNightly.cfg b/spec/tla/refresh-barrier/RefreshBarrierLivenessNightly.cfgindex aef7c5240e..9a9dc8fde3 100644--- a/spec/tla/refresh-barrier/RefreshBarrierLivenessNightly.cfg+++ b/spec/tla/refresh-barrier/RefreshBarrierLivenessNightly.cfg@@ -16,5 +16,6 @@ CONSTANTSMayRevoke = FALSEMaxCrashes = 1MayReset = FALSE+ OutsideHelp = TRUEPROPERTY DeferredLeavesPROPERTY EveryWriteEndsdiff --git a/spec/tla/refresh-barrier/mutants.json b/spec/tla/refresh-barrier/mutants.jsonindex 466f8c487e..34e2dd2c7f 100644--- a/spec/tla/refresh-barrier/mutants.json+++ b/spec/tla/refresh-barrier/mutants.json@@ -467,6 +467,34 @@}]}+ },+ {+ "id": "abort-skips-owed-sync",+ "kind": "guard",+ "title": "a refresh run never syncs the members an aborted refresh left behind HEAD (GST-1853 reverted)",+ "expect": "DeferredLeaves",+ "config": "RefreshBarrierAbortLiveness.cfg",+ "spec": {+ "file": "RefreshBarrier.tla",+ "before": "OwedSyncBegin ==\n /\\ pc[RFR] = \"idle\"",+ "after": "OwedSyncBegin ==\n /\\ FALSE /\\ pc[RFR] = \"idle\""+ },+ "code": {+ "edits": [+ {+ "file": "src/core/persistence/worktree-refresh.ts",+ "before": " if (newest?.state !== 'aborted') return [];\n",+ "after": " if (newest?.state !== 'aborted' || newest) return [];\n"+ }+ ],+ "tests": [+ {+ "test": "test/refresh-barrier-held-after-abort.test.ts",+ "filter": "GST-1853",+ "postgres": true+ }+ ]+ }}]}diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex 9e28f566c0..da29cfc948 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1475,6 +1475,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.594.0', id: 'file_drift_held_needs_prefer_8615', text: '`gbrain repair file-drift` with `--held`, `--side`, `--state`, `--cursor` or `--provenance` and no `--prefer` now refuses with the usage line (`invalid_params`, and `file_drift_state_required` for a deleted side without `--state`) instead of silently running the all-drifted survey, which counted a different set of pages (#8615).' },{ since: '0.60.594.0', id: 'refresh_source_changed_retry_8617', text: 'A `gbrain sources refresh` run refused `refresh_source_changed` because the checkout HEAD moved (a writer committing) or work reached the checkout after the drain check now retries once from a fresh precheck in the same run, so the tick is not lost (#8617). When the retry is refused too, the run returns the first `refresh_source_changed` refusal with the retry\'s cause appended; ownership or source-membership changes are not retried.' },{ since: '0.60.594.0', id: 'commit_push_checkout_not_a_source_8662', text: '`gbrain git commit-push` now refuses `checkout_not_a_source` before staging anything when this brain home records its source checkouts (a managed brain) and the checkout it resolved from the working directory (no `--path`) is none of them and has no `gbrain.sourceRoot`, for example a wrapper that changed into another repository (#8662). The refusal names the resolved path and the brain\'s checkouts; pass `--path` with the absolute brain checkout. Hosts whose brain records no source checkout behave as before.' },+ { since: 'TBD', id: 'refresh_barrier_aborted_refresh_sync_gst1853', text: 'While a checkout\'s last `gbrain sources refresh` ended without completing (aborted, abandoned, or rolled back after its process died), every refresh run on the owner host, including one refused `refresh_diverged`, now ends by running the managed `--no-pull` sync of each source behind the checkout HEAD, listed in its `synced` (a refusal\'s message names them). With `persistence.refresh_barrier` on, a write held behind that refresh for a commit sync had not imported is therefore decided in that run instead of waiting for a later refresh: a `put_page` with `expected_revision` ends `revision_conflict`, a blind one lands and names what it replaced in `supersedes`, and a `remember` or `add_timeline_entry` applies on top.' },{ since: '0.60.608.0', id: 'refresh_barrier_later_write_gst1623', text: 'With `persistence.refresh_barrier` on, a write that waited behind a refresh or an import (other than a `remember` or `add_timeline_entry`, which still applies on top) now ends `revision_conflict` instead of applying when a later write to the same page committed first, which a consumer from an older release can cause; read the page again and resubmit with a new `request_id`. A write held again and again behind imports now expires once it has waited four wait limits (at least 4 hours) since its admission, even when each hold is shorter than one wait limit. With the barrier on or off, a plain-git commit that lands on the start commit right before a refresh merges makes git refuse the fast-forward, and the refresh now ends `refresh_source_changed` with the fence lifted and the commit kept, instead of leaving the checkout `recovery_required`; one that lands right after the merge is recorded and synced.' },];diff --git a/src/core/persistence/worktree-refresh.ts b/src/core/persistence/worktree-refresh.tsindex 11af43c2f1..c3527626ff 100644--- a/src/core/persistence/worktree-refresh.ts+++ b/src/core/persistence/worktree-refresh.ts@@ -664,13 +664,55 @@ function finished(row: WorktreeRefreshRow, sync: { synced: RefreshMemberSync[];* host only; refusals throw catalogue errors (`code`, cause, literal fix, docs).*/export async function refreshWorktree(engine: BrainEngine, sourceId: string, opts: RefreshOptions = {}): Promise<WorktreeRefreshResult> {- try { return await runRefresh(engine, sourceId, opts); }+ let result: WorktreeRefreshResult;+ try { result = await runRefresh(engine, sourceId, opts); }catch (error) {- if (error instanceof OperationError || classifyPgAccessError(error).reason !== 'unknown') throw error;- throw await unexpectedRefreshFailure(engine, sourceId, opts, error);+ if (!(error instanceof OperationError) && classifyPgAccessError(error).reason !== 'unknown') throw error;+ const failure = error instanceof OperationError ? error : await unexpectedRefreshFailure(engine, sourceId, opts, error);+ const owed = opts.dryRun ? [] : await syncAfterUnfinishedRefresh(engine, sourceId, opts).catch(() => []);+ if (owed.length) failure.message += ` ${owedSyncNote(owed)}`;+ throw failure;}+ if (opts.dryRun) return result;+ const owed = await syncAfterUnfinishedRefresh(engine, sourceId, opts).catch(() => []);+ return owed.length ? { ...result, synced: [...result.synced, ...owed] } : result;}+/**+ * GST-1853: a refresh that ended without completing (aborted in its run, abandoned, or rolled back by recovery) never ran+ * the member syncs its fence refused, so a write held for an import on a member (import-deferral.ts) would wait for some+ * later sync, indefinitely while the checkout stays diverged. While the worktree's newest refresh is `aborted`, every+ * refresh run on the owner host therefore ends, after its own locks are released, by running the managed `--no-pull`+ * sync of each member whose last_commit is not the checkout HEAD, within the sync budget, so those writes are prepared+ * again against the checkout as it is. Best effort: a member it cannot sync waits for the next run or sync, and its held+ * writes stay bound by the wait limit (`pending_import_expired`).+ */+async function syncAfterUnfinishedRefresh(engine: BrainEngine, sourceId: string, opts: RefreshOptions): Promise<RefreshMemberSync[]> {+ const hostId = existingHostIdFor(engine);+ if (!hostId || !await managedPersistenceEnabled(engine)) return [];+ const binding = await getWorktreeBinding(engine, sourceId, hostId);+ if (!binding || binding.owner_host_id !== hostId || !binding.local_path) return [];+ const root = binding.local_path;+ const [newest] = await engine.executeRaw<Pick<WorktreeRefreshRow, 'state' | 'source_ids'>>(+ 'SELECT state,source_ids FROM persistence_worktree_refreshes WHERE worktree_id=$1::uuid ORDER BY created_at DESC LIMIT 1', [binding.worktree_id]);+ if (newest?.state !== 'aborted') return [];+ const head = await withGit(git => headOf(git, root));+ if (!head) return [];+ const lagging: string[] = [];+ for (const member of newest.source_ids) if (await lastCommit(engine, member) !== head) lagging.push(member);+ if (!lagging.length) return [];+ startPersistenceConsumer(engine, loadConfig() ?? { engine: engine.kind });+ const sync = await boundedMemberSync(await configuredMs(engine, opts.syncBudgetMs, 'GBRAIN_REFRESH_SYNC_BUDGET_MS', 'sources.refresh_sync_budget_ms', SYNC_BUDGET_MS));+ const synced: RefreshMemberSync[] = [];+ for (const [index, member] of lagging.entries()) {+ const status = await sync.member(engine, member, lagging.length - index).then(result => result.status, () => 'blocked');+ synced.push({ source_id: member, status, last_commit: await lastCommit(engine, member) });+ }+ return synced;+}+const owedSyncNote = (synced: RefreshMemberSync[]) => `The worktree's last refresh ended without completing, so this run then ran the managed sync of ${synced+ .map(member => `${member.source_id} (${member.status})`).join(', ')} at the checkout HEAD, for writes held for that import.`;+const redact = (text: string) => {const home = homedir();const escaped = home.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');diff --git a/test/refresh-barrier-held-after-abort.test.ts b/test/refresh-barrier-held-after-abort.test.tsnew file mode 100644index 0000000000..a056295cb0--- /dev/null+++ b/test/refresh-barrier-held-after-abort.test.ts@@ -0,0 +1,159 @@+/**+ * GST-1853: a write held behind a refresh that ends without completing (aborted, or abandoned by --resume after the+ * refreshing process was killed) is decided against the checkout as the refresh left it, not left pending until some+ * later refresh completes. The refresh's fence refused the managed sync that would import a commit another writer made, so+ * a write to that file was held for the import (deferred_import_from) and nothing else imported it. A refresh run that+ * finds the worktree's newest refresh aborted now runs the members' managed --no-pull sync on its way out, so each held+ * write is prepared again as designed: a put_page with expected_revision ends revision_conflict, a blind put_page lands and+ * names the imported revision it replaced, an add_timeline_entry applies on top.+ * Seams: the real refresh state machine and managed sync; GBRAIN_REFRESH_BARRIER; PGLite, and Postgres when DATABASE_URL is set.+ */+import { afterAll, afterEach, beforeAll, beforeEach, expect, test } from 'bun:test';+import { randomUUID } from 'node:crypto';+import { readFileSync } from 'node:fs';+import { join } from 'node:path';+import type { BrainEngine } from '../src/core/engine.ts';+import { PGLiteEngine } from '../src/core/pglite-engine.ts';+import { dispatchToolCall } from '../src/mcp/dispatch.ts';+import { disposePersistenceConsumer, waitForWrite } from '../src/core/persistence/service.ts';+import { getWriteRequestById } from '../src/core/persistence/journal.ts';+import { resetWriteSwitches } from '../src/core/persistence/switches.ts';+import { _setRefreshFenceWaitForTests } from '../src/core/persistence/admission-retry.ts';+import { _resetRefreshBarrierCacheForTests } from '../src/core/persistence/refresh-barrier.ts';+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';+import { refreshWorktree, resumeWorktreeRefreshes } from '../src/core/persistence/worktree-refresh.ts';+import type { WriteRequest } from '../src/core/persistence/model.ts';+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';+import { testBackends } from './helpers/test-backends.ts';+import { withEnv } from './helpers/with-env.ts';+import { commitFile, page, withRefreshFixture, type RefreshFixture } from './helpers/worktree-refresh-fixture.ts';++const backends = testBackends();+const engines: BrainEngine[] = [];+let closePostgres: (() => Promise<void>) | undefined;++beforeAll(async () => {+ if (backends.includes('pglite')) {+ const engine = new PGLiteEngine();+ await engine.connect({}); await engine.initSchema(); engines.push(engine);+ }+ if (backends.includes('postgres')) {+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);+ engines.push(pg.engine); closePostgres = pg.close;+ }+}, 120_000);+afterAll(async () => {+ for (const engine of engines) { await disposePersistenceConsumer(engine); if (engine.kind === 'pglite') await engine.disconnect(); }+ await closePostgres?.();+});+beforeEach(() => { _setRefreshFenceWaitForTests(0); _resetRefreshBarrierCacheForTests(); });+afterEach(() => { resetWriteSwitches(); _setRefreshFenceWaitForTests(null); });++const each = (fn: (f: RefreshFixture) => Promise<void>) => withEnv({ GBRAIN_REFRESH_BARRIER: '1' }, async () => {+ resetWriteSwitches();+ for (const engine of engines) await withRefreshFixture(engine, fn);+});++async function call(f: RefreshFixture, name: string, params: Record<string, unknown>): Promise<Record<string, any>> {+ const result = await dispatchToolCall(f.engine, name, params, { remote: true, transport: 'stdio', sourceId: f.alpha, admissionOrigin: 'interactive' });+ const body = JSON.parse(result.content[0]!.text) as Record<string, any>;+ expect(result.isError === true).toBe(false);+ return body;+}+/** A blind local CLI put_page (force, no expected_revision); its request id, settled or left pending (write_pending). */+async function blindPut(f: RefreshFixture, content: string): Promise<string> {+ try {+ return (await submitPageMutation(f.ctx(f.alpha), { operation: 'put_page', params: { request_id: randomUUID(), force: true, slug: 'one', content } }) as { request_id: string }).request_id;+ } catch (error) {+ const e = error as { code?: string; writeError?: string; writeRequest?: { request_id: string } };+ if ((e.code === 'write_pending' || e.writeError === 'write_pending') && e.writeRequest) return e.writeRequest.request_id;+ throw error;+ }+}+async function requestRow(f: RefreshFixture, requestId: string): Promise<WriteRequest> {+ const [row] = await f.engine.executeRaw<WriteRequest>('SELECT * FROM persistence_requests WHERE request_id=$1::uuid', [requestId]);+ return row!;+}+const settle = async (f: RefreshFixture, row: WriteRequest) => waitForWrite(f.engine, (await getWriteRequestById(f.engine, row.id))!, { engine: f.engine.kind } as never, 30_000);+const file = (f: RefreshFixture) => readFileSync(join(f.root, 'alpha', 'one.md'), 'utf8');+const IMPORTED = 'Another writer committed this line while the refresh drained.';+const MINE = 'Mine, written while the refresh drained.';+const ENTRY = 'Entry written while the refresh drained.';++/**+ * While the refresh drains: another writer commits alpha/one.md with plain git (HEAD moves on its own line, and sync is+ * fenced off, so the commit waits for an import), then three writes to that page are admitted and deferred behind the refresh.+ */+async function writeWhileDraining(f: RefreshFixture, seen: string): Promise<{ foreign: string; revision: string; blind: string; append: string }> {+ const foreign = commitFile(f.root, 'alpha/one.md', page('Alpha one', IMPORTED), 'another writer');+ const revision = (await call(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Based on the old revision.'), expected_revision: seen })).request_id as string;+ const blind = await blindPut(f, page('Alpha one', MINE));+ const append = (await call(f, 'add_timeline_entry', { slug: 'one', date: '2026-11-07', summary: ENTRY })).request_id as string;+ const [active] = await f.refreshRows();+ for (const id of [revision, blind, append]) expect(await requestRow(f, id)).toMatchObject({ state: 'deferred', deferred_refresh_id: active!.id });+ return { foreign, revision, blind, append };+}++/** Every held write is decided against the checkout the refresh left: the right revision check, never lost, never pending. */+async function expectDecided(f: RefreshFixture, ids: { foreign: string; revision: string; blind: string; append: string }): Promise<void> {+ // The release path: the refresh run that ended the refresh imported the checkout HEAD it left behind.+ expect(await f.lastCommit(f.alpha)).toBe(f.head());+ const revision = await requestRow(f, ids.revision), blind = await requestRow(f, ids.blind), append = await requestRow(f, ids.append);+ expect(await settle(f, revision)).toMatchObject({ state: 'conflict', error_code: 'revision_conflict' });+ const landed = await settle(f, blind);+ expect(landed.state).toBe('committed');+ expect((landed.outcome as { supersedes?: { commit: string } }).supersedes).toMatchObject({ slug: 'one', commit: ids.foreign });+ expect((await settle(f, append)).state).toBe('committed');+ const content = file(f);+ expect(content).toContain(MINE);+ expect(content).toContain(ENTRY);+ const history = await f.engine.executeRaw<{ compiled_truth: string | null }>(+ 'SELECT v.compiled_truth FROM page_versions v JOIN pages p ON p.id=v.page_id WHERE p.source_id=$1 AND p.slug=$2', [f.alpha, 'one']);+ expect(history.some(v => (v.compiled_truth ?? '').includes(IMPORTED))).toBe(true);+}++test('GST-1853: writes held behind a refresh that aborts and then meets a diverged checkout are decided by that run, not left for a later refresh', () => each(async f => {+ const seen = (await f.engine.readPageSnapshot('one', { sourceId: f.alpha }))!.revision as string;+ f.push('alpha/two.md', page('Alpha two', 'An upstream observation.'));+ let ids!: Awaited<ReturnType<typeof writeWhileDraining>>;+ const error = await refreshWorktree(f.engine, f.alpha, { hooks: { boundary: async point => { if (point === 'drained') ids = await writeWhileDraining(f, seen); } } })+ .then(() => null, thrown => thrown as { code?: string; message: string });+ // The HEAD the other writer moved aborts the refresh; the retry in the same tick finds the checkout diverged.+ expect(error?.code).toBe('refresh_source_changed');+ expect(error?.message).toContain('refresh_diverged');+ const rows = await f.refreshRows();+ expect(rows.map(r => r.state)).toEqual(['aborted']);+ await expectDecided(f, ids);+}), 180_000);++/** A refresh whose process is killed at fenced after another writer's commit and three writes landed while it drained. */+async function killedAtFenced(f: RefreshFixture): Promise<Awaited<ReturnType<typeof writeWhileDraining>>> {+ const seen = (await f.engine.readPageSnapshot('one', { sourceId: f.alpha }))!.revision as string;+ f.push('alpha/two.md', page('Alpha two', 'An upstream observation.'));+ let ids!: Awaited<ReturnType<typeof writeWhileDraining>>;+ await expect(refreshWorktree(f.engine, f.alpha, { hooks: { boundary: async point => {+ if (point === 'drained') ids = await writeWhileDraining(f, seen);+ if (point === 'fenced') throw new Error('simulated kill at fenced');+ } } })).rejects.toThrow('simulated kill');+ expect((await f.refreshRows()).map(r => r.state)).toEqual(['fenced']);+ for (const id of [ids.revision, ids.blind, ids.append]) expect((await requestRow(f, id)).state).toBe('deferred');+ return ids;+}++test('GST-1853: writes held behind a refresh killed at fenced are decided once --resume abandons it, not left for a later refresh', () => each(async f => {+ const ids = await killedAtFenced(f);+ const resumed = await refreshWorktree(f.engine, f.alpha, { resume: true });+ expect(resumed.status).toBe('abandoned');+ const rows = await f.refreshRows();+ expect(rows.map(r => r.state)).toEqual(['aborted']);+ await expectDecided(f, ids);+}), 180_000);++test('GST-1853: when the owner scan aborts the killed refresh, the next refresh run decides its held writes even though the checkout diverged', () => each(async f => {+ const ids = await killedAtFenced(f);+ await resumeWorktreeRefreshes(f.engine);+ const rows = await f.refreshRows();+ expect(rows.map(r => r.state)).toEqual(['aborted']);+ await expect(refreshWorktree(f.engine, f.alpha)).rejects.toMatchObject({ code: 'refresh_diverged' });+ await expectDecided(f, ids);+}), 180_000);