Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(persistence): refresh barrier follow-ups (GST-1623 items 1-3, 5, 13-22; #8759)

garrytanb7fc4d8d1122
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e31625b727..84fb8986f3 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,22 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**With the refresh write barrier on, a write that waited behind a refresh or an import can no longer land on top of a later write to the same page, and most of the barrier's remaining review follow-ups (GST-1623) are closed.** The barrier is still off by default (`persistence.refresh_barrier`); with it off, only the receipt health, the recovery-time replay order, the refresh merge check, the drain sleep and the new index change. A consumer from an older release that claims a later write first now makes the waiting write end `revision_conflict` instead of overwriting it. A write held again and again behind imports now expires at a ceiling counted from its admission. Waiting writes count toward their source's fair share of the local CLI cap. A refresh whose record recovery moved before the merge merges nothing.
+
+### Itemized changes
+
+- `src/core/persistence/refresh-barrier.ts`: a deferred or import-held write ends `revision_conflict` when a later non-sync write to its page has already committed (appends excepted), and refreshed paths compare case-folded when `core.ignorecase` is true. A held write's limit is the wait limit since its hold, or four wait limits (at least four hours) since its admission. An expired `remember` says that only a fact naming its entity gets the automatic replay, and that `gbrain repair failed-writes` lists the rest.
+- `src/core/persistence/import-deferral.ts`: a write deferred behind a refresh that meets `pending_import` takes the refresh-change check on that file first (a refresh that changed it ends the write `revision_conflict`), and its hold clears the released `deferred_refresh_id`, so a pre-v294 consumer never claims it (#8759 included: a capture whose captured bytes are the imported file names no `supersedes`).
+- `src/core/persistence/page-prepare.ts`: an earlier held write refuses only a journaled request admitted after it, never a Git or withdrawal-mirror effect, lint or a drift check reading the file.
+- `src/core/persistence/journal.ts`: the local CLI fair share counts `deferred` rows; managed sync's capacity discount counts only writes still held for its own source (#8759); receipt health treats a released deferred write as an earlier write and reports `recovery_required` for every write on a worktree whose refresh awaits recovery; a replay of an accepted request id reconciles before any deferral.
+- `src/core/persistence/worktree-refresh.ts`: the ff-only merge runs only when recording `merge_started` matched the `fenced` row; otherwise the refresh ends `refresh_source_changed` with nothing merged.
+- `src/core/persistence/sync-drain.ts`: a `worktree_refreshing` retry hint never sleeps past the drain's refresh budget.
+- Migration v295 adds `persistence_worktree_refreshes_completed` on `(worktree_id, completed_at DESC) WHERE state='completed'` for the retry-hint history read.
+- `gbrain upgrade --check-downgrade` and the guide say a released deferred write publishes only when a process runs the write consumer on the owner host; polling `gbrain write-request` only reads the receipt.
+- Tests on PGLite and Postgres: `test/refresh-barrier-gst1623.test.ts` (GST-1623 items 1, 2, 3, 5, 13–22 and #8759).
+
## [0.60.600.0] - 2026-11-07
**With the refresh write barrier on, writes waiting behind an import no longer use up the capacity the import itself needs, so a full cap no longer stalls managed sync until the waits expire.** The barrier is still off by default (`persistence.refresh_barrier`), and with it off nothing changes. Before, every held write kept its outstanding-request slot and intent bytes, so enough of them could leave a managed sync job, or a bulk import group, refused `queue_capacity`, and the writes it would have released waited out the limit and expired. A held `capture` that replaces an imported page now names what it replaced, as a blind `put_page` does, and the expiry pass no longer expires a hold that an import and a new deferral replaced while it ran.
diff --git a/docs/architecture/key-files/core-persistence-git.md b/docs/architecture/key-files/core-persistence-git.md
index 24fbd04e9e..b93542c973 100644
--- a/docs/architecture/key-files/core-persistence-git.md
+++ b/docs/architecture/key-files/core-persistence-git.md
@@ -5,7 +5,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
The managed Git writer (`gbrain sources refresh`), git-canonical memory (the git gap census, declared database-only classes, the door) and the checkout reads beside them.
- `src/core/persistence/{worktree-refresh,worktree-refresh-schema}.ts` — `gbrain sources refresh` (F0), the one sanctioned managed Git writer. `persistence_worktree_refreshes` holds one row per refresh with a partial unique index (one active refresh per worktree; its 23505 maps to `refresh_in_progress`). `refreshWorktree` prechecks with no database state (managed, owner host, upstream, no active refresh, no recovery, no unexhausted managed-sync cursor, hardened `git fetch` bounded by `sources.refresh_fetch_timeout_ms`, ancestry, dirty paths intersected with the incoming diff), then: `draining` (inserted under `withTopologyLocks` + `lockTopologyRows`; admission in `journal.ts` refuses `worktree_refreshing` for every request with that `worktree_id`, and `assertManagedSyncAllowed` refuses a managed sync before discovery), poll until no queued/running/recovering request, no queued/running `git`/`withdrawal-mirror` effect and no recovery record remain (else `refresh_drain_timeout`, aborted), `fenced` (both native locks held, rechecked in one transaction; `refreshFenceClear` stops request and effect claims), HEAD verified (#8617: a `refresh_source_changed` for a moved HEAD or work that arrived after the drain check is retried once in the same run from a fresh precheck; a second refusal returns the first with the retry's cause appended), dirt rechecked, `git merge --ff-only`, HEAD verified again, then one transaction `merged` + upstream observation + `syncing`; native locks are released and the managed `--no-pull` sync runs per member in one budget (`sources.refresh_sync_budget_ms`; past it a member is `deferred`) (only `managed_sync_*` `submit_job` admissions pass during `syncing`); `completed` when every member's `last_commit` is the target or descends from it, or with `outcome.sync_blocked` / `sync_deferred` when a member blocks or is deferred. A per-worktree `refresh-<worktree>.lock` held for the whole run proves liveness: `resumeWorktreeRefreshes` (called from `recoverSourceTopologies` on every consumer topology scan) touches interrupted `draining`/`fenced`/`merged` rows whose lock is free and applies the section 1.5 table (draining or fenced at the old HEAD abort; HEAD at the verified target moves to `syncing`; anything else is `recovery_required`, which admission refuses with `refresh_recovery_required`). Admission also completes a `syncing` row whose members all reached the target. #7679: each row records its holder (`outcome.holder`: host id, hostname, pid, process start; `previous_holder` after a recovery adopts it). A row whose lock is free is orphaned: `resumeWorktreeRefreshes` also completes an orphaned `syncing` row once every member's `last_commit` reached the target or a descendant, and a plain `gbrain sources refresh` on the owner host finishes any orphaned non-`recovery_required` row through the `--resume` path (`adoptOrphanedRefresh` + `resumeHeld`) before its own refresh, reporting it as `recovered`. A holder recorded on another host is never adopted. `refreshHolderReport` (writer status `worktree_refreshes`, sync status `worktree_refresh`) names the holder, `liveness` (`live` / `orphaned` / `unknown` off the lock's host) and a fix naming `--resume`. `--resume` converges and runs the member syncs; `--abandon` lifts a fence only with HEAD back at the start commit, and never on a `recovery_required` row once a member's `last_commit` is past it (not it or an ancestor; GST-1457: the next sync would soft-delete what it synced), and `--resume` finishes a `recovery_required` row only at a HEAD that contains every member commit on that refresh's line. `gbrain sources writer status` lists active refreshes as `worktree_refreshes` (state, heads, the next command). Doctor `worktree_refresh_stuck` lists rows active over 15 minutes. CLI body: `src/commands/sources-refresh.ts` (#7685: `--json` prints a refusal's structured `fix` when it has one; `refresh_in_progress` carries the writer-status read); while a resident PGLite owner (`gbrain serve`, stdio MCP) holds the database, `src/commands/sources-refresh-delegate.ts` runs the refresh inside it through the `writer_refresh` administration operation (trusted CLI registration only) and prints the same result or refusal. The consumer's idle probe applies `refreshFenceClear` to effects like the claims, so a fenced worktree's queued effects do not wake it. Tests: `test/persistence-worktree-refresh.test.ts`, `test/persistence-worktree-refresh-restart.test.ts` (child-process crash after merge), `test/worktree-refresh-orphan-7679.test.ts` (SIGKILL mid-syncing, live-holder no-steal), `test/oversize-hold-rescreen-8072.test.ts` (budget), Postgres via `test/e2e/worktree-refresh-*postgres.test.ts`, owner delegation in `test/persistence-sync-stdio-owner.test.ts`.
-- `src/core/persistence/refresh-barrier.ts` (+ deferred admission and claim in `journal.ts`, migration v292 `src/core/schema-migrations/v292-refresh-write-barrier.ts`) — the refresh write barrier (`docs/designs/REFRESH_WRITE_BARRIER.md`, `docs/guides/write-refusals.md#deferred-writes`). Off by default (`persistence.refresh_barrier`). With it on, a single page or memory write that reaches a worktree during a refresh is journaled in state `deferred` (tied to `deferred_refresh_id`) and answered pending instead of refused `worktree_refreshing`. Only a claim that declares the barrier moves a deferred row, and only after its refresh is released. `deferredReplayRefusal` re-checks the write's authority and reads the refresh's `old_head..target_head` diff with git, so a file the refresh changed ends `revision_conflict` with nothing applied. A write still waiting past `persistence.refresh_barrier_max_wait_ms` (15 minutes) ends `refresh_barrier_expired`. A write during a refresh in `recovery_required` defers too, and since GST-1582 there is no per-refresh cap (`persistence.refresh_barrier_max_deferred` is no longer read). A write whose page file holds a committed change sync has not imported yet is held with `deferred_import_from` (migration v294) instead of refused `pending_import`, and ends `pending_import_expired` past the wait limit. `refresh-retry-hint.ts` derives `worktree_refreshing`'s `retry_after_ms` from the worktree's refresh history. `gbrain upgrade --check-downgrade` exits 3 `downgrade_blocked_deferred_writes` while any deferred write is unfinished. Tests: `test/refresh-write-barrier.test.ts`, `test/refresh-barrier-state-audit.test.ts`.
+- `src/core/persistence/refresh-barrier.ts` (+ deferred admission and claim in `journal.ts`, migration v292 `src/core/schema-migrations/v292-refresh-write-barrier.ts`) — the refresh write barrier (`docs/designs/REFRESH_WRITE_BARRIER.md`, `docs/guides/write-refusals.md#deferred-writes`). Off by default (`persistence.refresh_barrier`). With it on, a single page or memory write that reaches a worktree during a refresh is journaled in state `deferred` (tied to `deferred_refresh_id`) and answered pending instead of refused `worktree_refreshing`. Only a claim that declares the barrier moves a deferred row, and only after its refresh is released. `deferredReplayRefusal` re-checks the write's authority and reads the refresh's `old_head..target_head` diff with git, so a file the refresh changed ends `revision_conflict` with nothing applied. A write still waiting past `persistence.refresh_barrier_max_wait_ms` (15 minutes) ends `refresh_barrier_expired`. A write during a refresh in `recovery_required` defers too, and since GST-1582 there is no per-refresh cap (`persistence.refresh_barrier_max_deferred` is no longer read). A write whose page file holds a committed change sync has not imported yet is held with `deferred_import_from` (migration v294) instead of refused `pending_import`, and ends `pending_import_expired` past the wait limit (or four wait limits, at least four hours, after admission when it is held again and again). GST-1623: either kind of deferred write ends `revision_conflict` when a later write to its page committed first, and a refresh-deferred write is checked against its refresh before an import hold, which then clears `deferred_refresh_id`. `refresh-retry-hint.ts` derives `worktree_refreshing`'s `retry_after_ms` from the worktree's refresh history. `gbrain upgrade --check-downgrade` exits 3 `downgrade_blocked_deferred_writes` while any deferred write is unfinished. Tests: `test/refresh-write-barrier.test.ts`, `test/refresh-barrier-state-audit.test.ts`.
- `src/core/persistence/git-gap.ts` + `src/commands/sources-git-gap.ts` + `src/commands/doctor/checks/db-ahead-of-git.ts` — git-canonical memory, wave G1 (`docs/designs/GIT_CANONICAL_MEMORY.md#the-git-gap-census`), read-only. `declaredDatabaseOnly` is the one predicate for pages that may not have a tracked file, tested in order: a read-only mirror source, a private page (`isPrivatePage`), an alias-private page (`isDerivationPrivate`), lineage-private (computed in SQL from `privatePagesFilterFragment`), `database_only_reason`, a `storage.db_only` prefix (`isSourceDbOnlySlug`, refusing an invalid gbrain.yml), a never-filed derive-phase page (page-prepare.ts `isNeverFiledDerivedPage`: no `source_path`/`source_uri` under `DERIVE_PHASE_DB_ONLY_DEFAULTS`), a path the checkout's Git ignores (`ignoredPaths`: filter-free `check-ignore -z --stdin`, tracked files never reported, exit 1 = none, null when Git cannot answer). The equality contract compares a committed blob with the page's export view (`readExportPage`, withdrawals overlaid; the census reads a batch's views with `readExportPages`, the same snapshots in two statements, #7617): an inline blob parsed as reconcile parses a canonical file, aligned with the stored type, subtype and #6429 stamp and compared by `reconcileCanonical` digest; a segmented page by the Git blob id of its streamed export. HEAD is read with hardened plumbing only (`readHeadTree`: one `ls-tree -r -l -z --full-tree` under the source's directory; `readBlobs`: `cat-file --batch` in calls of at most 64 MiB) and the working tree by hashing raw bytes (`worktreeBlobOid`), never `git status`/`git diff`, so no repository filter, textconv or fsmonitor runs. `gbrain sources git-gap <id> [--sample n] [--json]` (trusted local CLI, not an MCP op) puts every live page in one state (`equal`, `database_ahead`, `git_ahead` when the sync import record's `blob_oid`/`disk_blob_oid` differs from HEAD, else folded into `database_ahead` and counted in `database_ahead_without_import_record`, split by content evidence in `direction_evidence`, `declared_database_only` per class, `undeclared_database_only`, `worktree_only`, `unparseable`, `unsafe_path`), reports the working tree separately, and counts rows with no Git home (unfenced active facts, fenced facts on a missing page or a page without a committed file, links whose `link_source` is not in `DERIVED_LINK_SOURCES`, timeline rows on pages without a file), open `sync-hold` records by code and parked effect targets by kind and code. Each 256-page batch reads in its own read-only transaction. Doctor `db_ahead_of_git` runs the census per non-archived source whose directory is a Git checkout (others are only counted) under `GBRAIN_DOCTOR_GIT_GAP_TIMEOUT_MS` (default 10 s) and the entry deadline; it is always `ok` (informational) and lists totals only with each source's `live_pages` (counted in SQL before the scan), a source the budget stopped or never reached as partial (`PARTIAL, <read> of <live> live page(s) read`). Tests: `test/git-gap-census.test.ts` (PGLite and Postgres).
- `src/core/persistence/git-declared.ts` + `src/core/persistence/git-canonical-schema.ts` (migration `git_canonical_declared`) — git-canonical memory, wave G7a: the declared database-only classes of a strict source (`sources.config.canonical = "git"`, default off). `strictPageClass` judges a page by its slug and prior file (`declaredPathClass`: `db_only_prefix` for a `storage.db_only` dir, `derive_folder` for a never-filed derive-phase page; G3's indexer reads the same function) and then its rendered content (`privacyClass`: `private`, `alias_private`, or `lineage_private` from the same SQL remote reads use, `privatePagesFilterFragment`, over the page's own values). `page-prepare.ts` `prepareFileTarget` asks it before it builds a strict file target: a declared page gets no target (no file, no door commit, no Git effect), an untracked ignored target is `git_ignored`, a page turning private gets the removal of the file it still has in the worktree or at HEAD, and a path whose `filter` attribute is `lfs` passes `assertLfsTarget` only with `storage.git_lfs` and a checkout that configures the LFS filter (`git_lfs_not_enabled`). `preparePrivateTransition` (page-prepare.ts) is that removal for other writers (GBRA-60, connector imports #7971). In `publishMutation` and `publishGroup`, `assertNoPrivateCommit` refuses a file target carrying a private page's bytes before anything is written (`git_private_commit_refused`; a group rolls back whole), and `declareStrictPublication` (after apply, before the door commit) refuses a declared page's file or an undeclared file-less page (`git_home_required`), stamps `pages.database_only_reason` and, for a privacy class, clears `source_path`/`source_uri` in the transaction of the door commit that removed the file. A managed import builds its own file target, so it meets only these publication checks. `assertStrictGitHome` is the refusal for writers with no file form yet (G4, G5). The managed writer guard (`writer-guard-schema.ts`) refuses rows of a strict source unless the writer declares `gbrain.git_protocol` (`withCoordinatedWrite` does), behind the `git_canonical_brain.strict_sources` counter kept by `sources` triggers (`git_protocol_required`, mapped in `publication-failure.ts`). The guard reads its state as the writing role and fails closed: a hidden or missing `persistence_brain` or `git_canonical_brain` row, or a source or page row hidden by row-level security, raises `writer_coordinator_required` (`managed_writer_guard:unreadable_state`), and the counter refuses a recount under RLS on `sources` (`test/writer-fence-fail-closed-7636.test.ts`). The census (`git-gap.ts`) adds `database_only_source` (a source with no checkout) and maps the stamped reasons to their own classes. Tests: `test/git-canonical-declared-g7a.test.ts` (both engines).
- `src/core/persistence/checkout-state.ts` (`readCheckoutState`) — Q1: a canonical checkout against its last fetched upstream, read-only and time-bounded (no fetch): ahead/behind, `diverged`, each commit the upstream lacks (sha, author, date, subject, paths; at most 20) and porcelain counts. `sources writer status` reports it as `checkouts`; doctor `git_convergence` adds the commits and counts for a managed checkout and warns at once on a diverged one.
diff --git a/docs/designs/REFRESH_WRITE_BARRIER.md b/docs/designs/REFRESH_WRITE_BARRIER.md
index 2a43f4b525..5b665ca949 100644
--- a/docs/designs/REFRESH_WRITE_BARRIER.md
+++ b/docs/designs/REFRESH_WRITE_BARRIER.md
@@ -160,6 +160,12 @@ row stores both heads). An aborted refresh moved nothing and skips the check.
- No target path changed in R: the write publishes normally. Its own `expected_revision` and the
existing file guards still apply, so two deferred writes to one page, or a pre-fence write and a
deferred write, behave exactly as two ordinary queued writes do (no false conflict).
+- GST-1623: a write to the same page accepted after the deferred one has already committed (an
+ older consumer, whose claim lacks the deferred-row head rule, took it first): the row finishes
+ `conflict` / `revision_conflict` naming that request, so the earlier write never lands on top of
+ the later one. Managed sync's own rows and appends (`remember`, `add_timeline_entry`) do not count.
+ The same check runs for a write held for an import. Paths compare case-folded when the checkout's
+ `core.ignorecase` is true.
- Any target path changed in R: the row finishes with receipt state `conflict`, code
`revision_conflict`, detail naming the refresh and the changed paths. Nothing is applied, so a page
the refresh changed is never overwritten. This applies to every page operation, appends included;
@@ -175,7 +181,9 @@ released rows are claimed normally during the next refresh's `draining` and coun
them like queued rows. GST-1623: a managed sync job (and a bulk sync group made only of them) is
admitted as if deferred rows held no slot and no intent bytes, since they cannot settle before the
import does; the held rows are read only when the plain check would refuse, so the importer can pass
-a cap by at most what they hold. Every other write still counts them. GST-1582 removed the per-refresh cap (`persistence.refresh_barrier_max_deferred`):
+a cap by at most what they hold. Only rows still held for the sync's own source count (#8759): rows
+held for another source, and released rows about to settle, keep their slots. Every other write still
+counts them, and the local CLI fair share counts deferred rows toward their source's share. GST-1582 removed the per-refresh cap (`persistence.refresh_barrier_max_deferred`):
a write past it queued as deferred instead of being refused, because a refusal there bounced exactly
the writes the barrier exists to keep. The setting is no longer read.
@@ -206,23 +214,32 @@ barrier exists to keep, and with the flag on each now waits instead (with it off
then claimed and prepared from scratch, as if it had queued behind the import: a `remember` or
`add_timeline_entry` applies on top; a `put_page` with `expected_revision` meets the moved page and ends
`revision_conflict`; a blind (`force`) `put_page` or a `capture` lands over it. An import that lands without covering
- the file only re-arms the hold at the new commit. The receipt reads `queued` with
+ the file only re-arms the hold at the new commit. A later request to the same page waits behind the
+ held one (it is held too); an effect or a maintenance read of the file (Git and withdrawal-mirror
+ effects, lint, drift checks) is never refused by a hold (GST-1623). A write deferred behind a refresh
+ that meets `pending_import` takes the refresh-change check on that file first: a refresh that changed
+ it ends the write `revision_conflict` there, and otherwise the hold clears the released
+ `deferred_refresh_id`, so a consumer from before v294 never claims a row whose import is pending. The receipt reads `queued` with
`deferred: { pending_import: true }`, and the caller's wait returns pending at once.
**What a blind write replaced.** A held `put_page` or `capture` that replaces content an import brought in names
it in three places, so the replacement is auditable and recoverable without digging: the receipt's
`supersedes` (`slug`, `path`, the replaced `revision`, the imported `commit`: the newest commit
-after the held-at commit that touched the file), a `gbrain-supersedes: <path>: revision <id> from
+after the held-at commit that touched the file; a capture whose captured bytes are that commit's file
+replaced nothing and names none, #8759), a `gbrain-supersedes: <path>: revision <id> from
<commit>` trailer on its commit, and a `supersedes` list on that commit's ledger line (effect and
door commits). The replaced revision stays in `page_versions` and the imported commit stays in
Git, an ancestor of the write's commit.
**Age bound and schema.** A row held for an import that has not landed within
-`persistence.refresh_barrier_max_wait_ms` of its hold (measured from the hold, not from admission) ends
-`failed` with `pending_import_expired` in the owner's housekeeping turn, which rechecks under the
+`persistence.refresh_barrier_max_wait_ms` of its hold (measured from the hold, not from admission), or
+whose admission is older than four wait limits (at least four hours) however often it was held again
+(GST-1623), ends `failed` with `pending_import_expired` in the owner's housekeeping turn, which rechecks under the
request lock that the row is still unclaimed and its import still pending. A `remember` ends as its
`pending_import` refusal did instead (conflict `source_changed`, origin `pending_import`), so the
-first sync that imports the file replays the fact. Migration v294 adds the nullable `persistence_requests.deferred_import_from` and
+first sync that imports the file replays the fact; one that named no entity is not replayed, and its
+message sends a person to `gbrain repair failed-writes`. A source row is removed only through writer
+administration, which ends its deferred rows `source_changed`, so no held row outlives its source. Migration v294 adds the nullable `persistence_requests.deferred_import_from` and
widens `gbrain_guard_deferred_request`: a running row may move to `deferred` only under the barrier's
session declaration and only naming the import it waits for, and a deferred row moves to `running`
only once every hold it names is released. Only the write owner's consumer holds a row for an
@@ -263,6 +280,8 @@ Deferred rows are durable rows; nothing about them lives in process memory.
it: recovery records the merge and the next refresh finishes the member syncs, with no `--resume`.
- A merge that started and a HEAD that no longer holds the target (a reset and an unrelated commit)
stays `recovery_required` for the operator.
+ - The merge runs only when the update that records `merge_started` matched the `fenced` row; a row
+ recovery moved meanwhile ends the refresh `refresh_source_changed` with nothing merged (GST-1623).
- Consumer dies after claiming: the row is `running` with a lease; ordinary expiry and reclaim.
- Release and the deferred-to-running claim are single statements.
@@ -302,7 +321,16 @@ Deferred rows are durable rows; nothing about them lives in process memory.
- CLI write verbs: deferred → exit 10 with the receipt (`state: deferred`, `deferred`), `--json`
carries `outcome: pending`. This holds on every CLI route: the owner socket asks dispatch for the
`write_pending` error (`pendingAsError`), and a thin client turns a `status: "pending"` success
- into the same pending error. `gbrain write-request <id>` shows the deferral until it drains.
+ into the same pending error. `gbrain write-request <id>` shows the deferral until it drains; it
+ only reads the receipt. A released row publishes when a process runs the write consumer on the
+ owner host (`gbrain serve`, `gbrain jobs work`, any write or sync import there). Its receipt health
+ counts a released deferred row as an earlier write, and every write on a worktree whose refresh
+ awaits recovery (a deferred one included) reports `recovery_required` (GST-1623).
+- A replay of an accepted request id reconciles to its receipt before any deferral, so it is never
+ deferred again behind a refresh awaiting recovery.
+- The `worktree_refreshing` retry hint reads the worktree's newest completed refreshes through
+ `persistence_worktree_refreshes_completed` (`(worktree_id, completed_at DESC) WHERE state='completed'`,
+ migration v295), and a managed sync drain never sleeps a hint past its own refresh budget.
- MCP: deferred → `isError: false`, `status: "pending"`; `requestLogStatusForResult` classifies a
pending non-error result as `accepted_pending` (today it only reads pending from error results).
Other `write_pending` results (an acknowledgement lost to the owner) are unchanged.
diff --git a/docs/guides/error-codes.md b/docs/guides/error-codes.md
index a7251a8bb3..993c013502 100644
--- a/docs/guides/error-codes.md
+++ b/docs/guides/error-codes.md
@@ -1332,7 +1332,7 @@ More: [docs/guides/troubleshooting.md#doctor-ops-tier](../../docs/guides/trouble
| Meaning | Why | Next step | Who acts · consent | Verify | Exit | Retryable |
|---|---|---|---|---|---|---|
-| Writes deferred behind a worktree refresh are still waiting, and an older gbrain cannot drain them. | A release before the refresh write barrier never claims a deferred write, so after a downgrade those writes wait until the next upgrade. Safety does not depend on this check; liveness does. | Turn the barrier off (gbrain config set persistence.refresh_barrier false); wait until gbrain upgrade --check-downgrade exits 0, then downgrade. | host_admin | `repeat the read that failed` | 3 | no |
+| Writes deferred behind a worktree refresh are still waiting, and an older gbrain cannot drain them. | A release before the refresh write barrier never claims a deferred write, so after a downgrade those writes wait until the next upgrade. Safety does not depend on this check; liveness does. A released write publishes only when a gbrain process runs the write consumer on the worktree's owner host; polling its receipt does not publish it. | Turn the barrier off (gbrain config set persistence.refresh_barrier false); wait until gbrain upgrade --check-downgrade exits 0, then downgrade. | host_admin | `repeat the read that failed` | 3 | no |
More: [docs/guides/write-refusals.md#deferred-writes](../../docs/guides/write-refusals.md#deferred-writes)
diff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.md
index 3180a72de2..f1f0ddd123 100644
--- a/docs/guides/write-refusals.md
+++ b/docs/guides/write-refusals.md
@@ -639,7 +639,10 @@ syncing it is accepted and waits for the refresh instead of being refused `workt
- <a id="downgrade_blocked_deferred_writes"></a>Before installing an older release, turn the barrier
off and wait for `gbrain upgrade --check-downgrade` to exit 0. It exits 3 with
`downgrade_blocked_deferred_writes` (count and oldest age) while any write admitted deferred has not
- finished, waiting or already claimed, because an older release never drains them safely.
+ finished, waiting or already claimed, because an older release never drains them safely. A released
+ write publishes only when a gbrain process runs the write consumer on the worktree's owner host
+ (`gbrain serve`, `gbrain jobs work`, or any gbrain write or sync import there); polling
+ `gbrain write-request` only reads its receipt, so on a host with only cron jobs it waits for the next run.
## Managed sync drain stops
diff --git a/src/cli/commands/upgrade.ts b/src/cli/commands/upgrade.ts
index 83f23641d5..fe35d2dd7d 100644
--- a/src/cli/commands/upgrade.ts
+++ b/src/cli/commands/upgrade.ts
@@ -16,7 +16,7 @@ export async function run(args: string[], ctx: CliDispatchContext): Promise<void
if (backlog.count === 0) { console.log(args.includes('--json') ? JSON.stringify({ ok: true, deferred: 0 }) : 'No deferred writes are waiting; a downgrade leaves nothing undrained.'); return; }
setCliExitVerdict(writeCliError(opError('downgrade_blocked_deferred_writes',
`${backlog.count} write(s) deferred behind a worktree refresh are still waiting (oldest ${Math.round(backlog.oldest_age_ms / 1000)} s), and an older gbrain cannot drain them.`,
- 'Turn the barrier off with gbrain config set persistence.refresh_barrier false; wait until gbrain upgrade --check-downgrade exits 0, then downgrade.',
+ 'Turn the barrier off with gbrain config set persistence.refresh_barrier false; wait until gbrain upgrade --check-downgrade exits 0, then downgrade. A released write publishes only when a gbrain process runs the write consumer on the worktree\'s owner host (gbrain serve, gbrain jobs work, or any gbrain write or sync import there); polling gbrain write-request only reads its receipt.',
{ detail: `count=${backlog.count} oldest_age_ms=${backlog.oldest_age_ms}` }), 'upgrade', { argv: args }));
} finally { await finishCliTeardown({ engine }); }
return;
diff --git a/src/core/error-registry.ts b/src/core/error-registry.ts
index a6fd379cb2..317008ee97 100644
--- a/src/core/error-registry.ts
+++ b/src/core/error-registry.ts
@@ -263,7 +263,7 @@ export const CODES = {
},
deleted_page_chunks_overdue: { class: 'consent', summary: "Chunks of pages soft-deleted longer than the purge window (72h) are still stored.", why: 'Search never reads them; the purge lifecycle (`gbrain pages purge-deleted`, the autopilot purge phase) hard-deletes such pages and their chunks. The purge is brain-wide and irreversible, so it needs the user.', docs: 'docs/guides/troubleshooting.md#embedding-state', suggestion: 'Preview with `gbrain pages purge-deleted --dry-run --json`, then run `gbrain pages purge-deleted --yes` once the user agrees.', fix: { argv: ['gbrain', 'pages', 'purge-deleted', '--dry-run', '--json'], consent: [], actor: 'agent', why: 'Lists the soft-deleted pages the purge would remove; changes nothing.', requires_exclusive: false, verify: { argv: ['gbrain', 'doctor', '--only', 'embedding_state', '--json'] } } },
demo_graph_failed: { class: 'server', summary: "The fictional graph did not verify." },
- downgrade_blocked_deferred_writes: { class: 'consent', actor: 'host_admin', exit: 3, summary: "Writes deferred behind a worktree refresh are still waiting, and an older gbrain cannot drain them.", why: "A release before the refresh write barrier never claims a deferred write, so after a downgrade those writes wait until the next upgrade. Safety does not depend on this check; liveness does.", docs: 'docs/guides/write-refusals.md#deferred-writes', suggestion: 'Turn the barrier off (gbrain config set persistence.refresh_barrier false); wait until gbrain upgrade --check-downgrade exits 0, then downgrade.' },
+ downgrade_blocked_deferred_writes: { class: 'consent', actor: 'host_admin', exit: 3, summary: "Writes deferred behind a worktree refresh are still waiting, and an older gbrain cannot drain them.", why: "A release before the refresh write barrier never claims a deferred write, so after a downgrade those writes wait until the next upgrade. Safety does not depend on this check; liveness does. A released write publishes only when a gbrain process runs the write consumer on the worktree's owner host; polling its receipt does not publish it.", docs: 'docs/guides/write-refusals.md#deferred-writes', suggestion: 'Turn the barrier off (gbrain config set persistence.refresh_barrier false); wait until gbrain upgrade --check-downgrade exits 0, then downgrade.' },
demo_import_failed: { class: 'server', summary: "The fictional page did not import successfully." },
demo_verification_failed: { class: 'server', summary: "Expected ownership or decision provenance is missing." },
derived_cap_exhausted: { class: 'server', summary: "Paid work stopped at the cost cap derived from its estimate; the run is resumable." },
diff --git a/src/core/persistence/group-publish.ts b/src/core/persistence/group-publish.ts
index e22939a775..9ebc38217e 100644
--- a/src/core/persistence/group-publish.ts
+++ b/src/core/persistence/group-publish.ts
@@ -500,8 +500,9 @@ function seam(name: PublicationBoundary, row: WriteRequest): Promise<void> {
export async function publishSingleWrite(engine: BrainEngine, row: WriteRequest, prepared: PreparedMutation, hostId = hostIdFor(engine), hooks: GroupHooks = {},
transaction?: TransactionRunner): Promise<WriteRequest> {
const single: PublicationHooks = hooks.projectionPending ? { projectionPending: hooks.projectionPending } : {};
- // PR B: a write deferred behind a refresh takes the plain path, whose refresh-change check runs before anything publishes.
- if (engine.kind !== 'postgres' || row.deferred_refresh_id || !singleWrite(row) || !groupable(row, prepared) || !await writeSwitchOn(engine, 'single_write_group').catch(() => true)) {
+ // PR B: a write deferred behind a refresh takes the plain path, whose refresh-change check runs before anything publishes;
+ // GST-1623: so does one held for an import, whose check for a later write that committed first runs there too.
+ if (engine.kind !== 'postgres' || row.deferred_refresh_id || row.deferred_import_from != null || !singleWrite(row) || !groupable(row, prepared) || !await writeSwitchOn(engine, 'single_write_group').catch(() => true)) {
return publishMutation(engine, row, prepared, hostId, single);
}
try { assertMutationProtocol(row); } catch { return publishMutation(engine, row, prepared, hostId, single); }
diff --git a/src/core/persistence/import-deferral.ts b/src/core/persistence/import-deferral.ts
index 598a3e2b74..b964aae826 100644
--- a/src/core/persistence/import-deferral.ts
+++ b/src/core/persistence/import-deferral.ts
@@ -12,9 +12,11 @@
import { relative, sep } from 'node:path';
import type { BrainEngine } from '../engine.ts';
import type { PreparedMutation } from './coordinator.ts';
-import { deferralRoom, IMPORT_HOLD_CAPABILITY, publicationGroupKey, REFRESH_BARRIER_CLAIM_PREDICATE } from './journal.ts';
+import { completeWrite, deferralRoom, IMPORT_HOLD_CAPABILITY, publicationGroupKey, REFRESH_BARRIER_CLAIM_PREDICATE } from './journal.ts';
+import { deferredReplayRefusal } from './refresh-barrier.ts';
import type { WriteRequest } from './model.ts';
-import { landedCommit } from './pending-import.ts';
+import { fileAtCommit, landedCommit } from './pending-import.ts';
+import { sha256 } from './digest.ts';
import { PERSISTENCE_PROTOCOL_PREDICATE } from './protocol.ts';
import { writeSwitchOn } from './switches.ts';
@@ -27,13 +29,23 @@ export const SUPERSEDES_TRAILER_KEY = 'gbrain-supersedes';
* A claimed write whose preparation was refused `pending_import` (`importFrom`: the last imported commit the check saw),
* moved back to `deferred` behind that import; null when it keeps the refusal (barrier off, another operation, a batch, a
* live consumer on the owner host that cannot claim a held write).
+ * GST-1623: a write that waited behind a refresh is checked against that refresh first, on the file the refusal compared
+ * (`importFile`): a refresh that changed it, or a later write that committed first, ends the write here as its replay would
+ * have. Otherwise the hold drops the released `deferred_refresh_id`, so a consumer older than the import hold, which claims a
+ * deferred row once its refresh is released, never claims a row whose import is still pending (the v294 guard refuses it).
*/
export async function holdForImport(engine: BrainEngine, row: WriteRequest, error: unknown): Promise<WriteRequest | null> {
const from = (error as { importFrom?: unknown } | null)?.importFrom;
if (typeof from !== 'string' || !IMPORT_HELD_OPERATIONS.includes(row.operation) || (row.target_kind ?? 'page') !== 'page' || !row.worktree_id
|| publicationGroupKey(row) !== null || !await writeSwitchOn(engine, 'refresh_barrier').catch(() => false)
|| !await deferralRoom(engine, row.worktree_id, IMPORT_HOLD_CAPABILITY)) return null;
- const [held] = await engine.executeRaw<WriteRequest>(`UPDATE persistence_requests SET state='deferred',deferred_import_from=$3,execution_token=NULL,
+ if (row.deferred_refresh_id) {
+ const file = (error as { importFile?: { root: string; path: string } }).importFile;
+ if (!file) return null;
+ const refusal = await deferredReplayRefusal(engine, row, [file]);
+ if (refusal) return engine.transaction(tx => completeWrite(tx, row, refusal.state, {}, refusal.failure));
+ }
+ const [held] = await engine.executeRaw<WriteRequest>(`UPDATE persistence_requests SET state='deferred',deferred_import_from=$3,deferred_refresh_id=NULL,execution_token=NULL,
claim_expires_at=NULL,blocked_reason=NULL,claim_phase=NULL,updated_at=now()
WHERE id=$1::uuid AND execution_token=$2::uuid AND state='running' AND recovery IS NULL AND publication_started=false
AND ${PERSISTENCE_PROTOCOL_PREDICATE} AND ${REFRESH_BARRIER_CLAIM_PREDICATE} RETURNING *`, [row.id, row.execution_token, from]);
@@ -51,13 +63,19 @@ export function supersededBy(trailers: Iterable<string>): string[] {
/**
* A put_page or capture that was held for an import and replaces the page the import wrote: its outcome carries `supersedes`
- * (slug, path, the replaced revision, the imported commit) and its commit the matching trailer.
+ * (slug, path, the replaced revision, the imported commit) and its commit the matching trailer. #8759: a capture of an
+ * in-source file whose import brought in that very file (the captured bytes) replaced nothing of anyone else's, so it names none.
*/
export async function withSupersedes(row: WriteRequest, prepared: PreparedMutation): Promise<PreparedMutation> {
if (row.deferred_import_from == null || !REPLACING_OPERATIONS.includes(row.operation) || !prepared.file || prepared.noop || !prepared.observedRevision) return prepared;
const { root, path } = prepared.file;
const commit = await landedCommit(root, row.deferred_import_from || null, path);
if (!commit) return prepared;
+ const captured = row.operation === 'capture' ? row.intent?.capture_file_hash : undefined;
+ if (typeof captured === 'string') {
+ const imported = await fileAtCommit(root, commit, path);
+ if (imported !== null && sha256(imported) === captured) return prepared;
+ }
const rel = relative(root, path).split(sep).join('/');
const supersedes = { slug: row.slug, path: rel, revision: prepared.observedRevision, commit };
const trailer = `${SUPERSEDES_TRAILER_KEY}: ${supersedesNote(rel, prepared.observedRevision, commit)}`;
diff --git a/src/core/persistence/journal.ts b/src/core/persistence/journal.ts
index 1016fe9a6e..81c25b01a7 100644
--- a/src/core/persistence/journal.ts
+++ b/src/core/persistence/journal.ts
@@ -119,7 +119,8 @@ export function capacityError(resource: string, outstanding?: { used: number; li
* divided by the sources holding outstanding requests on the principal, this one included) is still admitted; the brain
* cap and every other principal's cap stay hard. The share is checked before the admission, so a sync publication group
* larger than the share is admitted whole and a source passes its share by at most one group. A refusal names the source
- * holding the most slots. GST-1623: a managed sync job passes `held`, the counter's writes waiting in `deferred` (heldBehindImport).
+ * holding the most slots. GST-1623: a managed sync job passes `held`, the counter's writes waiting in `deferred` (heldBehindImport). Deferred
+ * writes hold slots like queued ones, so they count toward their source's share.
*/
async function outstandingRefusal(tx: SqlEngine, row: Counter, principal: Principal, sourceId: string, need: number, limit: number,
held = 0): Promise<OperationError | null> {
@@ -127,7 +128,7 @@ async function outstandingRefusal(tx: SqlEngine, row: Counter, principal: Princi
if (used - held + need <= limit) return null;
if (row.key === 'brain' || principal.kind !== 'local_cli') return capacityError(`${row.key === 'brain' ? 'brain' : 'principal'} outstanding requests`, { used, limit });
const holders = await tx.executeRaw<{ source_id: string; n: number | string }>(`SELECT source_id,count(*) AS n FROM persistence_requests
- WHERE principal_kind=$1 AND principal_id=$2 AND state IN ('queued','running','recovering') GROUP BY source_id ORDER BY count(*) DESC,source_id`, [principal.kind, principal.id]);
+ WHERE principal_kind=$1 AND principal_id=$2 AND state IN ('queued','deferred','running','recovering') GROUP BY source_id ORDER BY count(*) DESC,source_id`, [principal.kind, principal.id]);
const own = Number(holders.find(holder => holder.source_id === sourceId)?.n ?? 0);
const sources = holders.length + (own > 0 ? 0 : 1);
if (sources > 1 && own < Math.floor(limit / sources)) return null;
@@ -135,13 +136,17 @@ async function outstandingRefusal(tx: SqlEngine, row: Counter, principal: Princi
return capacityError(`principal outstanding requests${top ? ` (source ${top.source_id} holds ${Number(top.n)})` : ''}`, { used, limit });
}
/**
- * GST-1623: the writes counted on `principal`'s counter (the brain's when null) that wait in `deferred` behind an import or
- * a refresh, and their intent bytes. They cannot settle before managed sync imports, so a managed sync job is admitted as
- * if they held no slot; the importer can pass a cap by at most what they hold. Read only when the plain check would refuse.
+ * GST-1623: the writes counted on `principal`'s counter (the brain's when null) that still wait in `deferred` on source
+ * `sourceId`'s managed sync: held for its import, or deferred behind an unreleased refresh it is a member of (a refresh is
+ * released only once all its members synced). They cannot settle before that sync, so its job is admitted as if they held
+ * no slot; the importer can pass a cap by at most what they hold. Writes waiting on another source's sync, and released ones
+ * about to settle, keep their slots (#8759). Read only when the plain check would refuse.
*/
-async function heldBehindImport(tx: SqlEngine, principal: Principal | null): Promise<{ count: number; bytes: number }> {
- const [row] = await tx.executeRaw<{ count: number | string; bytes: number | string }>(`SELECT count(*) AS count,COALESCE(sum(intent_bytes),0) AS bytes
- FROM persistence_requests WHERE state='deferred'${principal ? ' AND principal_kind=$1 AND principal_id=$2' : ''}`, principal ? [principal.kind, principal.id] : []);
+async function heldBehindImport(tx: SqlEngine, principal: Principal | null, sourceId: string): Promise<{ count: number; bytes: number }> {
+ const [row] = await tx.executeRaw<{ count: number | string; bytes: number | string }>(`SELECT count(*) AS count,COALESCE(sum(r.intent_bytes),0) AS bytes
+ FROM persistence_requests r WHERE r.state='deferred' AND NOT ${RELEASED_DEFERRED('r')} AND ((r.deferred_import_from IS NOT NULL AND r.source_id=$1)
+ OR EXISTS (SELECT 1 FROM persistence_worktree_refreshes f WHERE f.id=r.deferred_refresh_id AND f.released_at IS NULL AND $1=ANY(f.source_ids)))${principal ? ' AND r.principal_kind=$2 AND r.principal_id=$3' : ''}`,
+ principal ? [sourceId, principal.kind, principal.id] : [sourceId]);
return { count: Number(row?.count ?? 0), bytes: Number(row?.bytes ?? 0) };
}
/** A managed sync job: the import itself, never deferred or refused behind a refresh that waits on it. */
@@ -350,7 +355,7 @@ async function prepareAdmission(engine: BrainEngine, input: WriteAdmission, over
const brain = row.key === 'brain';
const countLimit = brain ? limits.brainOutstanding : limits.principalOutstanding, bytesLimit = brain ? limits.brainIntentBytes : limits.principalIntentBytes;
const fits = Number(row.outstanding_count) + 1 <= countLimit && Number(row.intent_bytes) + bytes <= bytesLimit;
- const held = managedSyncJob(input) && !fits ? await heldBehindImport(tx, brain ? null : input.principal) : { count: 0, bytes: 0 };
+ const held = managedSyncJob(input) && !fits ? await heldBehindImport(tx, brain ? null : input.principal, input.sourceId) : { count: 0, bytes: 0 };
const outstanding = await outstandingRefusal(tx, row, input.principal, input.sourceId, 1, countLimit, held.count);
if (outstanding) throw outstanding;
if (Number(row.intent_bytes) - held.bytes + bytes > bytesLimit) throw capacityError(`${brain ? 'brain' : 'principal'} intent bytes`);
@@ -438,7 +443,7 @@ export async function admitWriteGroupInTransaction(tx: BrainEngine, inputs: Writ
const brain = row.key === 'brain', scope = brain ? 'brain' : 'principal';
const countLimit = brain ? limits.brainOutstanding : limits.principalOutstanding, bytesLimit = brain ? limits.brainIntentBytes : limits.principalIntentBytes;
const fits = Number(row.outstanding_count) + fresh.length <= countLimit && Number(row.intent_bytes) + bytes <= bytesLimit;
- const held = syncGroup && !fits ? await heldBehindImport(tx, brain ? null : first.principal) : { count: 0, bytes: 0 };
+ const held = syncGroup && !fits ? await heldBehindImport(tx, brain ? null : first.principal, first.sourceId) : { count: 0, bytes: 0 };
const outstanding = await outstandingRefusal(tx, row, first.principal, first.sourceId, fresh.length, countLimit, held.count);
if (outstanding) throw outstanding;
if (Number(row.intent_bytes) - held.bytes + bytes > bytesLimit) throw capacityError(`${scope} intent bytes`);
@@ -671,14 +676,15 @@ async function assertWorktreeNotRefreshing(tx: BrainEngine, input: WriteAdmissio
AND s.last_commit IS DISTINCT FROM f.target_head) RETURNING f.id`, [refresh.id]);
if (completed.length) return null;
}
+ // A single write's replay of a request already accepted (its acknowledgement lost before the fence) reconciles to its
+ // receipt instead; a group admission never bypasses, since its other items may be new. GST-1623: checked before any
+ // deferral, so a replay never waits behind a refresh as if it were new.
+ if (replayable && input.requestId && await getWriteRequest(tx, input.principal, input.requestId)) return null;
// GST-1582: with the barrier on, a write waits behind a refresh awaiting recovery too, and lands once it is released.
if (refresh.state === 'recovery_required' && barrier && input.worktreeId && await deferralRoom(tx, input.worktreeId)) return refresh.id;
if (refresh.state === 'recovery_required') throw catalogueError('refresh_recovery_required',
`Refresh ${refresh.id} could not verify the checkout HEAD of source ${input.sourceId}; writes to its worktree stay fenced.`,
`gbrain sources writer status ${input.sourceId}, then gbrain sources refresh ${input.sourceId} --resume. Retry the same request_id afterwards.`);
- // A single write's replay of a request already accepted (its acknowledgement lost before the fence) reconciles to its
- // receipt instead; a group admission never bypasses, since its other items may be new.
- if (replayable && input.requestId && await getWriteRequest(tx, input.principal, input.requestId)) return null;
// PR B: with the barrier on, the write waits behind the refresh as a deferred request instead of being refused.
if (barrier && input.worktreeId && await deferralRoom(tx, input.worktreeId)) return refresh.id;
// GST-1582: the retry hint comes from the refresh's own state (its phase, how long it has run, how long this worktree's
@@ -1322,7 +1328,9 @@ const STUCK_RECOVERY = (r: string) => `(${r}.state='recovering' OR (${r}.recover
* Receipt health facts per root. #5984: a queued request waits on any earlier unfinished request of its root;
* a claimed one (a lane, a group follower, or a foreground write claimed ahead of queued sync rows) waits only
* on an earlier one already publishing or recovering, so the queued rows it overtook never make it report
- * `waiting_on_earlier_write`.
+ * `waiting_on_earlier_write`. GST-1623: a released deferred write is an earlier unfinished request like a queued one (claims
+ * wait on it, EARLIER_DEFERRED), and a worktree whose refresh awaits recovery reports `recovery_required` for every write that
+ * waits there, a write deferred behind that refresh included, so the operator signal the refusal used to carry is not lost.
*/
export async function writeHealthFacts(engine: BrainEngine, rows: WriteRequest[]): Promise<Map<string, WriteHealthFacts>> {
if (rows.length > 100) throw new RangeError('Receipt health pages are limited to 100 rows.');
@@ -1336,7 +1344,8 @@ export async function writeHealthFacts(engine: BrainEngine, rows: WriteRequest[]
WITH roots AS (SELECT unnest($1::text[]) AS root)
SELECT roots.root,head.sequence::text,head.started_sequence::text,head.started_unlaned::text,COALESCE(head.inspect_owner,false) AS inspect_owner,
COALESCE(head.recovering,false) OR EXISTS (SELECT 1 FROM persistence_effects e
- WHERE e.worktree_id=w.id AND e.recovery IS NOT NULL) AS recovery_required,
+ WHERE e.worktree_id=w.id AND e.recovery IS NOT NULL)
+ OR EXISTS (SELECT 1 FROM persistence_worktree_refreshes f WHERE f.worktree_id=w.id AND f.state='recovery_required') AS recovery_required,
w.id IS NOT NULL AND (w.state<>'active' OR w.owner_host_id IS NULL) AS owner_unavailable
FROM roots LEFT JOIN persistence_worktrees w ON w.id::text=roots.root
LEFT JOIN LATERAL (
@@ -1347,7 +1356,7 @@ export async function writeHealthFacts(engine: BrainEngine, rows: WriteRequest[]
AND NOT (s.state='running' AND s.recovery IS NULL AND s.operation='submit_job') ORDER BY s.sequence LIMIT 1) AS started_unlaned,
${STUCK_RECOVERY('r')} AS recovering,
r.blocked_reason IN ('unexpected_file_bytes','unexpected_staging_bytes') AS inspect_owner FROM persistence_requests r
- WHERE r.worktree_id=w.id AND (r.state IN ('queued','running','recovering') OR r.recovery IS NOT NULL)
+ WHERE r.worktree_id=w.id AND (r.state IN ('queued','running','recovering') OR r.recovery IS NOT NULL OR ${RELEASED_DEFERRED('r')})
ORDER BY r.sequence LIMIT 1)
UNION ALL
(SELECT r.sequence,(SELECT s.sequence FROM persistence_requests s WHERE s.worktree_id IS NULL AND s.source_incarnation=r.source_incarnation
diff --git a/src/core/persistence/page-prepare.ts b/src/core/persistence/page-prepare.ts
index 5ab94f948d..b9b67a1939 100644
--- a/src/core/persistence/page-prepare.ts
+++ b/src/core/persistence/page-prepare.ts
@@ -328,7 +328,7 @@ async function segmentedFileMatches(engine: BrainEngine, slug: string, bytes: By
const prefix = await inferredHeadPrefix(bytes.subarray(skip, skip + FRONTMATTER_HEAD_MAX_BYTES), await sourcePath());
return !!prefix?.length && await matches({ prefix, skip }, true) === true;
}
-export async function prepareFileTarget(engine: BrainEngine, row: Pick<WriteRequest, 'source_id' | 'worktree_id' | 'slug'> & { operation?: string }, snapshot: PageSnapshot | null,
+export async function prepareFileTarget(engine: BrainEngine, row: Pick<WriteRequest, 'source_id' | 'worktree_id' | 'slug'> & Partial<Pick<WriteRequest, 'sequence'>> & { operation?: string }, snapshot: PageSnapshot | null,
content: string | Uint8Array | StreamedFileContent | null, hostId?: string, options: { allowMissing?: boolean; deleting?: boolean; capture?: { path: string; hash: string }; activePack?: ParseOpts['activePack']; remote?: boolean;
/** #6368: a tombstone purge whose file matches these sync.exclude globs (relative to the source root) owns no file. */
excludeTombstone?: { patterns: string[]; onExcluded: (path: string) => void };
@@ -416,16 +416,18 @@ export async function prepareFileTarget(engine: BrainEngine, row: Pick<WriteRequ
}
/** prepareFileTarget's comparison of the file at `path` (`before`, its bytes or a piecewise reading) with the snapshot. */
-async function compareFileTarget(engine: BrainEngine, row: Pick<WriteRequest, 'source_id' | 'worktree_id' | 'slug'>, snapshot: PageSnapshot | null,
+async function compareFileTarget(engine: BrainEngine, row: Parameters<typeof prepareFileTarget>[1], snapshot: PageSnapshot | null,
content: string | Uint8Array | StreamedFileContent | null, options: Parameters<typeof prepareFileTarget>[5] & object,
target: { root: string; path: string; mode: Awaited<ReturnType<typeof scannerSlugRootMode>> | undefined; capturedPath: string | null | undefined; before: Buffer | FileBytes | null }): Promise<PreparedMutation['file']> {
const { root, path, mode, capturedPath, before } = target;
// GST-1582: an earlier write to this page held for an import that has not landed goes first, so this one is held behind the
// same import (holdForImport) and both are released, in admission order, at its checkpoint; it never commits before the held one.
- const [heldFirst] = await engine.executeRaw<{ deferred_import_from: string }>(`SELECT r.deferred_import_from FROM persistence_requests r
- JOIN sources s ON s.id=r.source_id WHERE r.source_id=$1 AND r.slug=$2 AND r.state='deferred' AND r.deferred_import_from IS NOT NULL
- AND COALESCE(s.last_commit,'')=r.deferred_import_from ORDER BY r.sequence LIMIT 1`, [row.source_id, row.slug]);
- if (heldFirst) throw pendingImportRefusal(row, options.remote === true, heldFirst.deferred_import_from);
+ // GST-1623: only a journaled request (one with a sequence) waits, and only on a write admitted before it; an effect or a
+ // maintenance read of the file (Git and withdrawal-mirror effects, lint, drift checks) is never refused by another request's hold.
+ const [heldFirst] = row.sequence == null ? [] : await engine.executeRaw<{ deferred_import_from: string }>(`SELECT r.deferred_import_from FROM persistence_requests r
+ JOIN sources s ON s.id=r.source_id WHERE r.source_id=$1 AND r.slug=$2 AND r.sequence<$3 AND r.state='deferred' AND r.deferred_import_from IS NOT NULL
+ AND COALESCE(s.last_commit,'')=r.deferred_import_from ORDER BY r.sequence LIMIT 1`, [row.source_id, row.slug, String(row.sequence)]);
+ if (heldFirst) throw pendingImportRefusal(row, options.remote === true, heldFirst.deferred_import_from, { root, path });
const beforeHash = () => before instanceof FileBytes ? before.hash() : sha256(before!);
if (options.deleting === true && options.sharedOrigin && snapshot && before) {
const target: FileClaimTarget = { sourceId: row.source_id, slug: row.slug, pageId: snapshot.page.id, root, path, canonicalPath: realpathSync(path),
@@ -480,7 +482,7 @@ async function compareFileTarget(engine: BrainEngine, row: Pick<WriteRequest, 's
if (!adopt.length) {
// GST-1582: read before the check, so a write held for the import never waits on a commit the check already saw imported.
const [source] = await engine.executeRaw<{ last_commit: string | null }>('SELECT last_commit FROM sources WHERE id=$1', [row.source_id]);
- if (await waitingOnWhile(options.clock, 'git', () => committedPendingImport(engine, row.source_id, root, path, options.clock?.signal))) throw pendingImportRefusal(row, options.remote === true, source?.last_commit ?? '');
+ if (await waitingOnWhile(options.clock, 'git', () => committedPendingImport(engine, row.source_id, root, path, options.clock?.signal))) throw pendingImportRefusal(row, options.remote === true, source?.last_commit ?? '', { root, path });
}
// #8109: drift confined to ordinary frontmatter keys never blocks a write that leaves the frontmatter alone.
const spliced = !adopt.length && options.deleting !== true && typeof content === 'string' && before instanceof Buffer && !bodyUnread(snapshot.page) && before.length <= PUSH_SCAN_MAX_FILE_BYTES
diff --git a/src/core/persistence/pending-import.ts b/src/core/persistence/pending-import.ts
index 947d75ddb2..f91914690e 100644
--- a/src/core/persistence/pending-import.ts
+++ b/src/core/persistence/pending-import.ts
@@ -120,6 +120,11 @@ export async function landedCommit(root: string, from: string | null, path: stri
return quiet(gitRead(root, ['rev-list', '-1', from ? `${from}..HEAD` : 'HEAD', '--', `:(literal)${relative(root, path).split(sep).join('/')}`]));
}
+/** #8759: the text of `path` as `commit` holds it, or null when Git cannot read it there. */
+export async function fileAtCommit(root: string, commit: string, path: string): Promise<string | null> {
+ return gitRead(root, ['show', `${commit}:./${relative(root, path).split(sep).join('/')}`]).catch(() => null);
+}
+
/** The commit HEAD names in `root`, or null on an unborn branch or when Git cannot read it. */
export async function headCommit(root: string, signal?: AbortSignal): Promise<string | null> {
return quiet(gitRead(root, ['rev-parse', '--verify', '--quiet', 'HEAD'], signal), signal);
@@ -133,7 +138,8 @@ function quiet(read: Promise<string>, signal?: AbortSignal): Promise<string | nu
* The refusal: nothing was overwritten; a remember waits for its replay, any other write is resubmitted after the next sync.
* GBRA-133: neither fix starts a sync; both read the status, so no caller puts a second sync owner beside a running one.
*/
-export function pendingImportRefusal(row: { source_id: string; slug: string; operation?: string }, remote: boolean, importFrom?: string): OperationError {
+export function pendingImportRefusal(row: { source_id: string; slug: string; operation?: string }, remote: boolean, importFrom?: string,
+ importFile?: { root: string; path: string }): OperationError {
const status = ['gbrain', 'sync', 'status', '--source', row.source_id, '--json'];
const remember = row.operation === 'remember';
const fix: Action = remember
@@ -148,6 +154,7 @@ export function pendingImportRefusal(row: { source_id: string; slug: string; ope
+ (remember ? 'so this remember was refused and kept: the first sync that imports the file makes one replay attempt, with its original date and authority.'
: 'so this write was refused rather than replacing those bytes: after the next sync imports them, read the page again and resubmit with a new request_id.'),
{ detail: 'file_database_drift', reason: PENDING_IMPORT, fix });
- // GST-1582: the last imported commit the check compared against, so a refresh-barrier consumer can hold the write for the import.
- return importFrom === undefined ? refusal : Object.assign(refusal, { importFrom });
+ // GST-1582: the last imported commit the check compared against, so a refresh-barrier consumer can hold the write for the import;
+ // GST-1623: and the file it compared, so a write deferred behind a refresh is checked against that refresh before it is held.
+ return importFrom === undefined ? refusal : Object.assign(refusal, { importFrom, ...(importFile ? { importFile } : {}) });
}
diff --git a/src/core/persistence/refresh-barrier.ts b/src/core/persistence/refresh-barrier.ts
index 8052fe4ec2..53d370497d 100644
--- a/src/core/persistence/refresh-barrier.ts
+++ b/src/core/persistence/refresh-barrier.ts
@@ -14,7 +14,7 @@ import { gitChildEnv } from '../git-env.ts';
import { OperationError } from '../ops/contract.ts';
import { authorizeWrite } from './authority.ts';
import type { BrainEngine } from '../engine.ts';
-import { completeWrite } from './journal.ts';
+import { completeWrite, SYNC_KIND } from './journal.ts';
import { PENDING_IMPORT } from './pending-import.ts';
import { declarePersistenceProtocol } from './protocol.ts';
import type { SqlEngine, WriteRequest } from './model.ts';
@@ -23,10 +23,12 @@ const GIT_ENV = { GIT_TERMINAL_PROMPT: '0', GCM_INTERACTIVE: 'Never', LC_ALL: 'C
const DIFF_TIMEOUT_MS = 30_000;
const CACHED_REFRESHES = 32;
/** The paths each refresh changed, per checkout root: one diff serves the whole drain behind it. */
-const changed = new Map<string, Promise<ReadonlySet<string> | null>>();
+const changed = new Map<string, Promise<RefreshPaths | null>>();
+/** GST-1623: with `core.ignorecase` true the checkout names one file in any case, so paths are compared case-folded. */
+interface RefreshPaths { paths: ReadonlySet<string>; fold: boolean }
/** Paths (repository-relative, forward slashes) refresh `refreshId` changed; empty for an aborted refresh; null when unreadable. */
-async function refreshChangedPaths(engine: SqlEngine, refreshId: string, root: string): Promise<ReadonlySet<string> | null> {
+async function refreshChangedPaths(engine: SqlEngine, refreshId: string, root: string): Promise<RefreshPaths | null> {
const key = `${refreshId}\0${root}`;
const held = changed.get(key);
if (held) return held;
@@ -34,10 +36,13 @@ async function refreshChangedPaths(engine: SqlEngine, refreshId: string, root: s
const [refresh] = await engine.executeRaw<{ state: string; old_head: string; target_head: string; released: boolean }>(
'SELECT state,old_head,target_head,released_at IS NOT NULL AS released FROM persistence_worktree_refreshes WHERE id=$1::uuid', [refreshId]);
if (!refresh?.released) return null;
- if (refresh.state === 'aborted' || refresh.old_head === refresh.target_head) return new Set<string>();
- const { error, stdout } = await execFileBounded('git', ['-C', root, '-c', 'core.quotepath=false', 'diff', '--name-only', '-z', '--no-renames', '--relative',
- refresh.old_head, refresh.target_head], { timeout: DIFF_TIMEOUT_MS, maxBuffer: 64 * 1024 ** 2, env: gitChildEnv(GIT_ENV) });
- return error ? null : new Set(stdout.split('\0').filter(Boolean));
+ if (refresh.state === 'aborted' || refresh.old_head === refresh.target_head) return { paths: new Set<string>(), fold: false };
+ const [diff, config] = await Promise.all([execFileBounded('git', ['-C', root, '-c', 'core.quotepath=false', 'diff', '--name-only', '-z', '--no-renames', '--relative',
+ refresh.old_head, refresh.target_head], { timeout: DIFF_TIMEOUT_MS, maxBuffer: 64 * 1024 ** 2, env: gitChildEnv(GIT_ENV) }),
+ execFileBounded('git', ['-C', root, 'config', '--bool', 'core.ignorecase'], { timeout: DIFF_TIMEOUT_MS, env: gitChildEnv(GIT_ENV) })]);
+ if (diff.error) return null;
+ const fold = !config.error && config.stdout.trim() === 'true';
+ return { paths: new Set(diff.stdout.split('\0').filter(Boolean).map(path => fold ? path.toLowerCase() : path)), fold };
})();
changed.set(key, read);
read.then(paths => { if (paths === null && changed.get(key) === read) changed.delete(key); }, () => { if (changed.get(key) === read) changed.delete(key); });
@@ -47,13 +52,34 @@ async function refreshChangedPaths(engine: SqlEngine, refreshId: string, root: s
export interface DeferredRefusal { state: 'conflict' | 'failed'; failure: { code: string; message: string } }
+/** Writes that apply on top of the page as it is, so one accepted later never makes them overwrite anything. */
+const APPENDING_OPERATIONS = ['remember', 'add_timeline_entry'];
+
+/**
+ * GST-1623: the refusal of deferred write `row` when a write to its page accepted after it has already committed (an older
+ * consumer, which has no EARLIER_DEFERRED in its claim, claimed the later write first), so the deferred one, replayed now,
+ * never lands on top of it. Managed sync's own rows are the import the write waited for and do not count; neither does a
+ * write that only appends.
+ */
+async function laterWriteRefusal(engine: SqlEngine, row: WriteRequest): Promise<DeferredRefusal | null> {
+ if (APPENDING_OPERATIONS.includes(row.operation)) return null;
+ const [later] = await engine.executeRaw<{ request_id: string }>(`SELECT r.request_id::text AS request_id FROM persistence_requests r
+ WHERE r.source_id=$1 AND r.slug=$2 AND r.sequence>$3 AND r.state='committed' AND NOT ${SYNC_KIND('r')} ORDER BY r.sequence LIMIT 1`,
+ [row.source_id, row.slug, String(row.sequence)]);
+ if (!later) return null;
+ return { state: 'conflict', failure: { code: 'revision_conflict', message: `Request ${row.request_id} waited ${row.deferred_refresh_id ? `behind refresh ${row.deferred_refresh_id}` : 'for an import'}, and request ${later.request_id}, accepted after it, has already changed ${row.slug}, so nothing was applied and the later write was not overwritten. Read the page again and submit the change with a new request_id.` } };
+}
+
/**
* Null when a claimed deferred write may publish `files` (its own page file and its companions'); otherwise the terminal
- * refusal to record. A row that was never deferred is not checked.
+ * refusal to record. A row that was never deferred is not checked. GST-1623: a write held for an import is checked too, for a
+ * later write to its page that committed first.
*/
export async function deferredReplayRefusal(engine: SqlEngine, row: WriteRequest, files: ReadonlyArray<{ path: string; root: string }>): Promise<DeferredRefusal | null> {
const refreshId = row.deferred_refresh_id;
- if (!refreshId) return null;
+ if (!refreshId && row.deferred_import_from == null) return null;
+ const later = await laterWriteRefusal(engine, row);
+ if (later || !refreshId) return later;
try { await authorizeWrite(engine, row.authority, row.operation, row.slug); } catch (error) {
if (!(error instanceof OperationError)) throw error;
return { state: 'failed', failure: { code: error.code, message: `Request ${row.request_id} waited behind refresh ${refreshId}, and its credential no longer grants this write: ${error.message}` } };
@@ -61,10 +87,10 @@ export async function deferredReplayRefusal(engine: SqlEngine, row: WriteRequest
let unreadable = false;
const touched: string[] = [];
for (const file of files) {
- const paths = await refreshChangedPaths(engine, refreshId, file.root);
+ const changes = await refreshChangedPaths(engine, refreshId, file.root);
const target = relative(file.root, file.path).split(sep).join('/');
- if (paths === null) unreadable = true;
- if (paths === null || paths.has(target)) touched.push(target);
+ if (changes === null) unreadable = true;
+ if (changes === null || changes.paths.has(changes.fold ? target.toLowerCase() : target)) touched.push(target);
}
if (!touched.length) return null;
return { state: 'conflict', failure: { code: 'revision_conflict', message: unreadable
@@ -79,11 +105,20 @@ const MAX_WAIT_SQL = `COALESCE((SELECT CASE WHEN value ~ '^[1-9][0-9]{0,14}$' TH
const overdueDeferredFrom = (host: string, maxWait: string) => `JOIN persistence_worktrees w ON w.id=r.worktree_id
JOIN persistence_worktree_refreshes f ON f.id=r.deferred_refresh_id
WHERE w.owner_host_id=${host}::uuid AND r.state='deferred' AND f.released_at IS NULL AND r.created_at<now()-(${maxWait}*interval '1 millisecond')`;
-/** GST-1582: a write held for an import still pending (its refresh, if any, released) whose hold is older than the limit. */
+/**
+ * GST-1623: however often a write is held again (another writer committing to its file faster than sync imports), it waits at
+ * most this many wait limits from its admission, and never less than HOLD_CEILING_FLOOR_MS.
+ */
+const HOLD_CEILING_LIMITS = 4;
+const HOLD_CEILING_FLOOR_MS = 4 * 60 * 60_000;
+const holdCeiling = (maxWait: string) => `GREATEST(${maxWait}*${HOLD_CEILING_LIMITS},${HOLD_CEILING_FLOOR_MS})`;
+/** A held write past its limit: held longer than the wait limit, or accepted longer ago than the ceiling. */
+const holdOverdue = (maxWait: string) => `(r.updated_at<now()-(${maxWait}*interval '1 millisecond') OR r.created_at<now()-(${holdCeiling(maxWait)}*interval '1 millisecond'))`;
+/** GST-1582: a write held for an import still pending (its refresh, if any, released) whose hold is past its limit. */
const overdueImportFrom = (host: string, maxWait: string) => `JOIN persistence_worktrees w ON w.id=r.worktree_id JOIN sources s ON s.id=r.source_id
WHERE w.owner_host_id=${host}::uuid AND r.state='deferred' AND r.deferred_import_from IS NOT NULL AND COALESCE(s.last_commit,'')=r.deferred_import_from
AND (r.deferred_refresh_id IS NULL OR EXISTS (SELECT 1 FROM persistence_worktree_refreshes f WHERE f.id=r.deferred_refresh_id AND f.released_at IS NOT NULL))
- AND r.updated_at<now()-(${maxWait}*interval '1 millisecond')`;
+ AND ${holdOverdue(maxWait)}`;
/** The owner's idle probe counts an overdue deferred or import-held write as work, so its expiry runs even when nothing else is queued. */
export const overdueDeferredWorkSql = (host: string) => `(EXISTS (SELECT 1 FROM persistence_requests r ${overdueDeferredFrom(host, MAX_WAIT_SQL)})
OR EXISTS (SELECT 1 FROM persistence_requests r ${overdueImportFrom(host, MAX_WAIT_SQL)}))`;
@@ -100,7 +135,8 @@ export const overdueDeferredWorkSql = (host: string) => `(EXISTS (SELECT 1 FROM
* so the expiry checks the row is still unclaimed and its import still pending with the sources row locked FOR SHARE
* (after the protocol's brain lock, the order a fence takes them in), so an import that is landing makes the check wait
* and then see it landed. GST-1623: it also rechecks the hold it scanned (the same import and an age past the limit), so an
- * import and a new hold between the scan and the lock never expire the new hold.
+ * import and a new hold between the scan and the lock never expire the new hold. A write held again and again expires at
+ * the ceiling counted from its admission (holdOverdue). An expired remember says whether a sync will replay it: only one that named its entity is replayed (failed-writes.ts).
*/
export async function expireDeferredWrites(engine: BrainEngine, hostId: string, signal?: AbortSignal): Promise<WriteRequest[]> {
const [config] = await engine.executeRaw<{ ms: number }>(`SELECT ${MAX_WAIT_SQL} AS ms`, undefined, { signal });
@@ -126,12 +162,16 @@ export async function expireDeferredWrites(engine: BrainEngine, hostId: string,
await declarePersistenceProtocol(tx);
const [held] = await tx.executeRaw(`SELECT 1 FROM persistence_requests r JOIN sources s ON s.id=r.source_id
WHERE r.id=$1::uuid AND r.state='deferred' AND r.deferred_import_from=$2 AND COALESCE(s.last_commit,'')=r.deferred_import_from
- AND r.updated_at<now()-($3::double precision*interval '1 millisecond') FOR SHARE OF s`, [row.id, row.deferred_import_from, maxWaitMs]);
+ AND ${holdOverdue('$3::double precision')} FOR SHARE OF s`, [row.id, row.deferred_import_from, maxWaitMs]);
if (!held) return null;
+ const waited = `Request ${row.request_id} waited more than ${Math.round(maxWaitMs / 1000)} s`;
+ const status = `gbrain sync status --source ${row.source_id} --json`;
if (row.operation === 'remember') return completeWrite(tx, row, 'conflict', {}, { code: 'source_changed', detail: { origin: PENDING_IMPORT },
- message: `Request ${row.request_id} waited more than ${Math.round(maxWaitMs / 1000)} s for managed sync to import a committed change to the file of ${row.slug}, and the import has not landed, so the fact is not saved yet. It is kept: the first managed sync that imports the file replays it once. Do not save it again; check the source with gbrain sync status --source ${row.source_id} --json.` });
+ message: `${waited} for managed sync to import a committed change to the file of ${row.slug}, and the import has not landed, so the fact is not saved yet. ${typeof row.intent?.entity_slug === 'string'
+ ? `It is kept for one automatic replay attempt by the first managed sync that imports the file; until that replay commits, gbrain repair failed-writes --source ${row.source_id} lists it (a replay that is refused stays there for a person to replay).`
+ : `It is kept, but it named no entity, so no sync replays it: gbrain repair failed-writes --source ${row.source_id} lists it for a person to replay once the source has synced.`} Do not save it again; check the source with ${status}.` });
return completeWrite(tx, row, 'failed', {}, { code: 'pending_import_expired',
- message: `Request ${row.request_id} waited more than ${Math.round(maxWaitMs / 1000)} s for managed sync to import a committed change to the file of ${row.slug}, and the import has not landed, so nothing was applied. Check the source with gbrain sync status --source ${row.source_id} --json, then submit the write again with a new request_id.` });
+ message: `${waited} for managed sync to import a committed change to the file of ${row.slug}, and the import has not landed, so nothing was applied. Check the source with ${status}, then submit the write again with a new request_id.` });
}).catch(error => { if ((error as { code?: string })?.code === 'write_claim_lost') return null; throw error; });
if (done) finished.push(done);
}
diff --git a/src/core/persistence/sync-drain.ts b/src/core/persistence/sync-drain.ts
index 976f547fe1..7ed28ed8a2 100644
--- a/src/core/persistence/sync-drain.ts
+++ b/src/core/persistence/sync-drain.ts
@@ -260,7 +260,8 @@ function transientDelay(error: unknown, attempt: number, refreshWaitedMs: number
if (error instanceof OperationError && error.code === 'worktree_refreshing') {
if (refreshWaitedMs >= REFRESH_WAIT_MS) return null;
const hinted = Number(/retry_after_ms=(\d+)/.exec(String(error.detail ?? ''))?.[1]);
- return Number.isFinite(hinted) && hinted > 0 ? hinted : 1000;
+ // GST-1623: never past the drain's refresh budget, so a long hint ends the wait on time instead of oversleeping it.
+ return Math.min(Number.isFinite(hinted) && hinted > 0 ? hinted : 1000, REFRESH_WAIT_MS - refreshWaitedMs);
}
// Admission gave up on lock contention (a concurrent publication holds the shared counters); the frozen request ID is kept.
if (error instanceof OperationError && error.detail === 'database_contention') return refreshWaitedMs >= REFRESH_WAIT_MS ? null : 1000;
@@ -270,6 +271,9 @@ function transientDelay(error: unknown, attempt: number, refreshWaitedMs: number
return baseMs * 4 ** (attempt - 1);
}
+/** Test seam: the drain's retry delay for one failed pass. */
+export const _transientDelayForTests = transientDelay;
+
/** The error's message (or code), with anything that looks like a connection URL removed. */
function errorText(error: unknown): string {
const text = error instanceof Error ? error.message : typeof error === 'object' && error && 'code' in error ? String((error as { code: unknown }).code) : String(error);
diff --git a/src/core/persistence/worktree-refresh-schema.ts b/src/core/persistence/worktree-refresh-schema.ts
index 0816978a1d..6442955ddc 100644
--- a/src/core/persistence/worktree-refresh-schema.ts
+++ b/src/core/persistence/worktree-refresh-schema.ts
@@ -30,6 +30,8 @@ CREATE TABLE IF NOT EXISTS persistence_worktree_refreshes (
CREATE UNIQUE INDEX IF NOT EXISTS persistence_worktree_refreshes_active
ON persistence_worktree_refreshes(worktree_id)
WHERE state IN ('draining','fenced','merged','syncing','recovery_required');
+CREATE INDEX IF NOT EXISTS persistence_worktree_refreshes_completed
+ ON persistence_worktree_refreshes(worktree_id, completed_at DESC) WHERE state='completed';
`;
export const ACTIVE_REFRESH_STATES = ['draining', 'fenced', 'merged', 'syncing', 'recovery_required'] as const;
diff --git a/src/core/persistence/worktree-refresh.ts b/src/core/persistence/worktree-refresh.ts
index ca425310aa..38a35cfbe1 100644
--- a/src/core/persistence/worktree-refresh.ts
+++ b/src/core/persistence/worktree-refresh.ts
@@ -893,10 +893,12 @@ async function fenceAndMerge(engine: BrainEngine, git: Git, sourceId: string, pl
}
const dirt = await dirtyOverlap(git, plan.root, row.old_head, row.target_head);
if (dirt.overlap.length) await abort(engine, current, ['fenced'], dirtyRefusal(sourceId, dirt.overlap));
- await engine.transaction(async tx => {
+ // GST-1623: the merge runs only when this marked the row; a row recovery moved off `fenced` meanwhile is not merged over.
+ const marked = await engine.transaction(async tx => {
await tx.executeRaw("SELECT set_config('synchronous_commit','on',true)");
- await tx.executeRaw(`UPDATE persistence_worktree_refreshes SET outcome=outcome||'{"merge_started":true}'::jsonb,updated_at=now() WHERE id=$1::uuid AND state='fenced'`, [row.id]);
+ return tx.executeRaw(`UPDATE persistence_worktree_refreshes SET outcome=outcome||'{"merge_started":true}'::jsonb,updated_at=now() WHERE id=$1::uuid AND state='fenced' RETURNING id`, [row.id]);
});
+ if (!marked.length) throw refusal('refresh_source_changed', 'The refresh record was moved by recovery before the merge; nothing was merged.', `gbrain sources refresh ${sourceId} --resume`, row.id);
let merge = await git(['-C', plan.root, 'merge', '--ff-only', '--no-edit', '--quiet', row.target_head], 120_000);
let after = await headOf(git, plan.root);
for (let attempt = 1; attempt < INDEX_LOCK_ATTEMPTS && merge.code !== 0 && after === row.old_head && INDEX_LOCKED.test(merge.stderr); attempt++) {
diff --git a/src/core/pglite-schema.generated.ts b/src/core/pglite-schema.generated.ts
index f518ceec68..e1d41ac7fe 100644
--- a/src/core/pglite-schema.generated.ts
+++ b/src/core/pglite-schema.generated.ts
@@ -2020,6 +2020,8 @@ CREATE TABLE IF NOT EXISTS persistence_worktree_refreshes (
CREATE UNIQUE INDEX IF NOT EXISTS persistence_worktree_refreshes_active
ON persistence_worktree_refreshes(worktree_id)
WHERE state IN ('draining','fenced','merged','syncing','recovery_required');
+CREATE INDEX IF NOT EXISTS persistence_worktree_refreshes_completed
+ ON persistence_worktree_refreshes(worktree_id, completed_at DESC) WHERE state='completed';
CREATE TABLE IF NOT EXISTS source_ingestion_receipts (
id uuid PRIMARY KEY,
diff --git a/src/core/schema-embedded.generated.ts b/src/core/schema-embedded.generated.ts
index ac51f3e6e5..c3b0a2e3ac 100644
--- a/src/core/schema-embedded.generated.ts
+++ b/src/core/schema-embedded.generated.ts
@@ -2771,6 +2771,8 @@ CREATE TABLE IF NOT EXISTS persistence_worktree_refreshes (
CREATE UNIQUE INDEX IF NOT EXISTS persistence_worktree_refreshes_active
ON persistence_worktree_refreshes(worktree_id)
WHERE state IN ('draining','fenced','merged','syncing','recovery_required');
+CREATE INDEX IF NOT EXISTS persistence_worktree_refreshes_completed
+ ON persistence_worktree_refreshes(worktree_id, completed_at DESC) WHERE state='completed';
-- END GENERATED from src/core/persistence/worktree-refresh-schema.ts (WORKTREE_REFRESH_SCHEMA_SQL)
-- BEGIN GENERATED from src/core/company-brain/receipt-schema.ts (SOURCE_INGESTION_RECEIPTS_SCHEMA_SQL). Edit that file, then run: bun run build:schema
diff --git a/src/core/schema-migrations/latest.generated.ts b/src/core/schema-migrations/latest.generated.ts
index 84959b7547..0e755cb30c 100644
--- a/src/core/schema-migrations/latest.generated.ts
+++ b/src/core/schema-migrations/latest.generated.ts
@@ -4,4 +4,4 @@
// The highest schema migration version (migrate.ts LATEST_VERSION), importable without
// loading the migrations themselves.
-export const LATEST_SCHEMA_VERSION = 294;
+export const LATEST_SCHEMA_VERSION = 295;
diff --git a/src/core/schema-migrations/registry.generated.ts b/src/core/schema-migrations/registry.generated.ts
index 70518ec740..26ec10bdfd 100644
--- a/src/core/schema-migrations/registry.generated.ts
+++ b/src/core/schema-migrations/registry.generated.ts
@@ -295,6 +295,7 @@ import { v291 } from './v291-persistence-request-replay-indexes.ts';
import { v292 } from './v292-refresh-write-barrier.ts';
import { v293 } from './v293-qm-attest-sources.ts';
import { v294 } from './v294-refresh-barrier-import-deferral.ts';
+import { v295 } from './v295-refresh-retry-hint-history-index.ts';
// Array order is historical (see HISTORICAL_ARRAY_ORDER in the generator); the
// runner sorts by version before applying.
@@ -588,4 +589,5 @@ export const MIGRATIONS: Migration[] = [
v292,
v293,
v294,
+ v295,
];
diff --git a/src/core/schema-migrations/v295-refresh-retry-hint-history-index.ts b/src/core/schema-migrations/v295-refresh-retry-hint-history-index.ts
new file mode 100644
index 0000000000..70b6728fa2
--- /dev/null
+++ b/src/core/schema-migrations/v295-refresh-retry-hint-history-index.ts
@@ -0,0 +1,18 @@
+import type { Migration } from './types.ts';
+
+// Applied by src/core/migrate.ts on the next initSchema(); see docs/ENGINES.md
+// ("Canonical schema sources") for when schema.sql or a TS fragment also changes.
+//
+// GST-1623: the worktree_refreshing retry hint (refresh-retry-hint.ts) reads a worktree's newest completed refreshes,
+// newest first, inside write admission. Without an index it sorted every completed refresh of the worktree. The table
+// holds one row per refresh, so a plain build is brief. Fresh installs get it from WORKTREE_REFRESH_SCHEMA_SQL. DDL only.
+// The number is a placeholder restamped at the merge slot.
+export const v295: Migration = {
+ version: 295,
+ name: 'refresh_retry_hint_history_index',
+ idempotent: true,
+ sql: `
+ CREATE INDEX IF NOT EXISTS persistence_worktree_refreshes_completed
+ ON persistence_worktree_refreshes(worktree_id, completed_at DESC) WHERE state='completed';
+ `,
+};
diff --git a/src/schema.sql b/src/schema.sql
index 00aae491cd..b7f4166081 100644
--- a/src/schema.sql
+++ b/src/schema.sql
@@ -2767,6 +2767,8 @@ CREATE TABLE IF NOT EXISTS persistence_worktree_refreshes (
CREATE UNIQUE INDEX IF NOT EXISTS persistence_worktree_refreshes_active
ON persistence_worktree_refreshes(worktree_id)
WHERE state IN ('draining','fenced','merged','syncing','recovery_required');
+CREATE INDEX IF NOT EXISTS persistence_worktree_refreshes_completed
+ ON persistence_worktree_refreshes(worktree_id, completed_at DESC) WHERE state='completed';
-- END GENERATED from src/core/persistence/worktree-refresh-schema.ts (WORKTREE_REFRESH_SCHEMA_SQL)
-- BEGIN GENERATED from src/core/company-brain/receipt-schema.ts (SOURCE_INGESTION_RECEIPTS_SCHEMA_SQL). Edit that file, then run: bun run build:schema
diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-default.json b/test/fixtures/goldens/catalog/pglite-engine-init-default.json
index 225fca6f62..2d52dc61f0 100644
--- a/test/fixtures/goldens/catalog/pglite-engine-init-default.json
+++ b/test/fixtures/goldens/catalog/pglite-engine-init-default.json
@@ -1552,6 +1552,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json b/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json
index 2711c8ff78..c7646f800d 100644
--- a/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json
+++ b/test/fixtures/goldens/catalog/pglite-engine-init-fts-portuguese.json
@@ -1552,6 +1552,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json b/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json
index 7953f41cd1..f24412af8a 100644
--- a/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json
+++ b/test/fixtures/goldens/catalog/pglite-engine-init-high-dims.json
@@ -1548,6 +1548,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/pglite-schema-blob-default.json b/test/fixtures/goldens/catalog/pglite-schema-blob-default.json
index cae84edf04..e5c631fe71 100644
--- a/test/fixtures/goldens/catalog/pglite-schema-blob-default.json
+++ b/test/fixtures/goldens/catalog/pglite-schema-blob-default.json
@@ -1086,6 +1086,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/pglite-schema-blob-fts-portuguese.json b/test/fixtures/goldens/catalog/pglite-schema-blob-fts-portuguese.json
index a76dc8ae32..299b03517a 100644
--- a/test/fixtures/goldens/catalog/pglite-schema-blob-fts-portuguese.json
+++ b/test/fixtures/goldens/catalog/pglite-schema-blob-fts-portuguese.json
@@ -1086,6 +1086,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/pglite-schema-blob-high-dims.json b/test/fixtures/goldens/catalog/pglite-schema-blob-high-dims.json
index 0965bb8883..64fe5cb55d 100644
--- a/test/fixtures/goldens/catalog/pglite-schema-blob-high-dims.json
+++ b/test/fixtures/goldens/catalog/pglite-schema-blob-high-dims.json
@@ -1085,6 +1085,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/postgres-db-initschema.json b/test/fixtures/goldens/catalog/postgres-db-initschema.json
index 818a4db67c..83a4153832 100644
--- a/test/fixtures/goldens/catalog/postgres-db-initschema.json
+++ b/test/fixtures/goldens/catalog/postgres-db-initschema.json
@@ -1105,6 +1105,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-default.json b/test/fixtures/goldens/catalog/postgres-engine-init-default.json
index b4dc091b19..5435aa0a4d 100644
--- a/test/fixtures/goldens/catalog/postgres-engine-init-default.json
+++ b/test/fixtures/goldens/catalog/postgres-engine-init-default.json
@@ -1556,6 +1556,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json b/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json
index 0b0eb72362..812260080b 100644
--- a/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json
+++ b/test/fixtures/goldens/catalog/postgres-engine-init-fts-portuguese.json
@@ -1556,6 +1556,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json b/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json
index 7370aae9a8..70e2e56154 100644
--- a/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json
+++ b/test/fixtures/goldens/catalog/postgres-engine-init-high-dims.json
@@ -1552,6 +1552,7 @@
"persistence_topology_changes_principal_id_request_id_key": "CREATE UNIQUE INDEX persistence_topology_changes_principal_id_request_id_key ON public.persistence_topology_changes USING btree (principal_id, request_id)",
"persistence_topology_recovering": "CREATE INDEX persistence_topology_recovering ON public.persistence_topology_changes USING btree (created_at) WHERE (state = 'recovering'::text)",
"persistence_worktree_refreshes_active": "CREATE UNIQUE INDEX persistence_worktree_refreshes_active ON public.persistence_worktree_refreshes USING btree (worktree_id) WHERE (state = ANY (ARRAY['draining'::text, 'fenced'::text, 'merged'::text, 'syncing'::text, 'recovery_required'::text]))",
+ "persistence_worktree_refreshes_completed": "CREATE INDEX persistence_worktree_refreshes_completed ON public.persistence_worktree_refreshes USING btree (worktree_id, completed_at DESC) WHERE (state = 'completed'::text)",
"persistence_worktree_refreshes_pkey": "CREATE UNIQUE INDEX persistence_worktree_refreshes_pkey ON public.persistence_worktree_refreshes USING btree (id)",
"persistence_worktrees_pkey": "CREATE UNIQUE INDEX persistence_worktrees_pkey ON public.persistence_worktrees USING btree (id)",
"persistence_writer_protocols_pkey": "CREATE UNIQUE INDEX persistence_writer_protocols_pkey ON public.persistence_writer_protocols USING btree (worktree_id, host_id)",
diff --git a/test/fixtures/goldens/migrations/records.json b/test/fixtures/goldens/migrations/records.json
index d28e3f20f9..09b0198e57 100644
--- a/test/fixtures/goldens/migrations/records.json
+++ b/test/fixtures/goldens/migrations/records.json
@@ -1,6 +1,6 @@
{
"golden": {
- "count": 289,
+ "count": 290,
"gaps": [
17,
18,
@@ -97,7 +97,7 @@
"verifyProjectionStatistics": "0d231b04ecfbc5a4109d6fca9567bf4e8c9e9db6c3fbbb35212e026579ed6f67",
"withSchemaLockTimeout": "95605bd3004955e11e936c7a4aa6889319b401098abf9db18ef3e48da2c3fc59"
},
- "latest": 294,
+ "latest": 295,
"records": [
{
"handler": {
@@ -2946,8 +2946,8 @@
"idempotent": true,
"name": "f0_worktree_refreshes",
"sql": {
- "length": 935,
- "sha256": "77ca451abde39586cefffb674e0ac0c36e371165a2d63e9c8b022557e48172ed"
+ "length": 1096,
+ "sha256": "19344005bafce35a597dfde19cb4d8cfc69ef384040c4c457db09a676af2184a"
},
"sqlFor": "absent",
"transaction": "absent",
@@ -4800,6 +4800,19 @@
"transaction": "absent",
"verify": "absent",
"version": 294
+ },
+ {
+ "handler": "absent",
+ "idempotent": true,
+ "name": "refresh_retry_hint_history_index",
+ "sql": {
+ "length": 172,
+ "sha256": "54d86ce498bfefd77a6fd28bc2dd0af898fa785544e3f8005508bd3f7949ab23"
+ },
+ "sqlFor": "absent",
+ "transaction": "absent",
+ "verify": "absent",
+ "version": 295
}
],
"versions": [
@@ -5091,7 +5104,8 @@
291,
292,
293,
- 294
+ 294,
+ 295
]
},
"normalizer": "migrations-record-v1"
diff --git a/test/refresh-barrier-gst1623.test.ts b/test/refresh-barrier-gst1623.test.ts
new file mode 100644
index 0000000000..c9b777cab8
--- /dev/null
+++ b/test/refresh-barrier-gst1623.test.ts
@@ -0,0 +1,390 @@
+/**
+ * GST-1623: refresh write barrier follow-ups (docs/designs/REFRESH_WRITE_BARRIER.md).
+ *
+ * Protects, with persistence.refresh_barrier on: deferred writes count toward their source's fair share of the local CLI cap;
+ * a released deferred write is an earlier write in receipt health, and a worktree whose refresh awaits recovery reports
+ * recovery_required; a hold refuses only a journaled request admitted after the held write, never an effect or a maintenance
+ * read of the file; a deferred write never lands on top of a later write to its page that committed first; a write deferred
+ * behind a refresh is checked against that refresh before it is held for an import, and the hold drops the released refresh;
+ * a write held again and again expires at a ceiling counted from its admission; a held write never outlives its source row
+ * (confirmation); an expired remember says whether a sync replays it; the retry-hint history has an index; a refresh merges
+ * only when it marked the merge started; the drain never sleeps past its refresh budget; a case-insensitive checkout compares
+ * refreshed paths case-folded; receipt polling never publishes a released deferred write; the managed sync capacity discount counts
+ * only writes still held for that source; a capture of the very file its import brought in names no supersedes.
+ * Seams: the real refresh state machine (hooks.boundary), managed sync and consumer; GBRAIN_REFRESH_BARRIER.
+ */
+import { afterAll, afterEach, beforeAll, beforeEach, expect, test } from 'bun:test';
+import { randomUUID } from 'node:crypto';
+import { readFileSync } from 'node:fs';
+import { join } from 'node:path';
+import type { BrainEngine } from '../src/core/engine.ts';
+import { PGLiteEngine } from '../src/core/pglite-engine.ts';
+import { dispatchToolCall } from '../src/mcp/dispatch.ts';
+import { disposePersistenceConsumer, waitForWrite } from '../src/core/persistence/service.ts';
+import { settleTopologyRequests, topologyPrincipal } from '../src/core/persistence/topology-locks.ts';
+import { getWriteRequestById, writeHealthFacts } from '../src/core/persistence/journal.ts';
+import { resetWriteSwitches } from '../src/core/persistence/switches.ts';
+import { _setRefreshFenceWaitForTests } from '../src/core/persistence/admission-retry.ts';
+import { _resetRefreshBarrierCacheForTests, deferredReplayRefusal, expireDeferredWrites } from '../src/core/persistence/refresh-barrier.ts';
+import { submitPageMutation } from '../src/core/persistence/page-mutations.ts';
+import { prepareFileTarget } from '../src/core/persistence/page-prepare.ts';
+import { performManagedSync } from '../src/core/persistence/sync-run.ts';
+import { refreshWorktree, resumeWorktreeRefreshes } from '../src/core/persistence/worktree-refresh.ts';
+import { _transientDelayForTests } from '../src/core/persistence/sync-drain.ts';
+import { deleteSourceRow } from '../src/core/source-delete.ts';
+import { OperationError } from '../src/core/ops/contract.ts';
+import type { WriteRequest } from '../src/core/persistence/model.ts';
+import { isolatedPersistencePostgres } from './helpers/persistence-postgres.ts';
+import { testBackends } from './helpers/test-backends.ts';
+import { withEnv } from './helpers/with-env.ts';
+import { commitFile, git, page, withRefreshFixture, type RefreshFixture } from './helpers/worktree-refresh-fixture.ts';
+
+const backends = testBackends();
+const engines: BrainEngine[] = [];
+let closePostgres: (() => Promise<void>) | undefined;
+
+beforeAll(async () => {
+ if (backends.includes('pglite')) {
+ const engine = new PGLiteEngine();
+ await engine.connect({}); await engine.initSchema(); engines.push(engine);
+ }
+ if (backends.includes('postgres')) {
+ const pg = await isolatedPersistencePostgres(process.env.DATABASE_URL!);
+ engines.push(pg.engine); closePostgres = pg.close;
+ }
+}, 120_000);
+afterAll(async () => {
+ for (const engine of engines) { await disposePersistenceConsumer(engine); if (engine.kind === 'pglite') await engine.disconnect(); }
+ await closePostgres?.();
+});
+beforeEach(() => { _setRefreshFenceWaitForTests(0); _resetRefreshBarrierCacheForTests(); });
+afterEach(() => { resetWriteSwitches(); _setRefreshFenceWaitForTests(null); });
+
+const each = (fn: (f: RefreshFixture) => Promise<void>) => withEnv({ GBRAIN_REFRESH_BARRIER: '1' }, async () => {
+ resetWriteSwitches();
+ for (const engine of engines) await withRefreshFixture(engine, async f => {
+ git(f.root, 'config', 'user.name', 'Example'); git(f.root, 'config', 'user.email', '[email protected]'); git(f.root, 'config', 'commit.gpgsign', 'false');
+ await fn(f);
+ });
+});
+
+async function call(f: RefreshFixture, name: string, params: Record<string, unknown>, sourceId = f.alpha) {
+ const result = await dispatchToolCall(f.engine, name, params, { remote: true, transport: 'stdio', sourceId, admissionOrigin: 'interactive' });
+ return { isError: result.isError === true, body: JSON.parse(result.content[0]!.text) as Record<string, any> };
+}
+async function requestRow(f: RefreshFixture, requestId: string): Promise<WriteRequest> {
+ const [row] = await f.engine.executeRaw<WriteRequest>('SELECT * FROM persistence_requests WHERE request_id=$1::uuid', [requestId]);
+ return row!;
+}
+/** A blind local CLI write: its receipt whether it settled or was left pending (write_pending). */
+async function local(f: RefreshFixture, operation: 'put_page' | 'capture', params: Record<string, unknown>, sourceId = f.alpha): Promise<{ pending: boolean; receipt: Record<string, any> }> {
+ try {
+ const receipt = await submitPageMutation(f.ctx(sourceId), { operation, params: { request_id: randomUUID(), force: true, ...params } }) as Record<string, any>;
+ return { pending: false, receipt };
+ } catch (error) {
+ const e = error as { code?: string; writeError?: string; writeRequest?: Record<string, any> };
+ if ((e.code === 'write_pending' || e.writeError === 'write_pending') && e.writeRequest) return { pending: true, receipt: e.writeRequest };
+ throw error;
+ }
+}
+const settle = async (f: RefreshFixture, row: WriteRequest) => waitForWrite(f.engine, (await getWriteRequestById(f.engine, row.id))!, { engine: f.engine.kind } as never, 30_000);
+const importNow = (f: RefreshFixture, sourceId = f.alpha) => performManagedSync(f.engine, { sourceId, noPull: true, noEmbed: true });
+const file = (f: RefreshFixture, rel: string, sub = 'alpha') => readFileSync(join(f.root, sub, rel), 'utf8');
+const IMPORTED = 'Another writer committed this line.';
+const foreignCommit = (f: RefreshFixture, rel: string, content: string, sub = 'alpha') => commitFile(f.root, `${sub}/${rel}`, content, 'another writer');
+const maxWait = (f: RefreshFixture, ms: number) => f.engine.executeRaw(`INSERT INTO config(key,value) VALUES('persistence.refresh_barrier_max_wait_ms',$1)
+ ON CONFLICT (key) DO UPDATE SET value=EXCLUDED.value`, [String(ms)]);
+const clearMaxWait = (f: RefreshFixture) => f.engine.executeRaw("DELETE FROM config WHERE key='persistence.refresh_barrier_max_wait_ms'");
+const ownerHost = async (f: RefreshFixture) => (await f.engine.executeRaw<{ owner: string }>('SELECT owner_host_id::text AS owner FROM persistence_worktrees WHERE id=$1::uuid', [f.worktreeId]))[0]!.owner;
+/** Drives this process's consumer on `row` for up to `ms`: until it settles, or (a write held again) until the wait ends. */
+const drive = async (f: RefreshFixture, row: WriteRequest, ms = 5_000) => waitForWrite(f.engine, (await getWriteRequestById(f.engine, row.id))!, { engine: f.engine.kind } as never, ms);
+
+/** Runs `fn` while a real refresh of the checkout is paused at its drained boundary; the refresh changes alpha/two.md by default. */
+async function duringRefresh(f: RefreshFixture, fn: () => Promise<void>, change: [string, string] = ['alpha/two.md', page('Alpha two', 'An upstream observation.')]) {
+ f.push(...change);
+ let release!: () => void; const gate = new Promise<void>(resolve => { release = resolve; });
+ let reached!: () => void; const drained = new Promise<void>(resolve => { reached = resolve; });
+ const refresh = refreshWorktree(f.engine, f.alpha, { hooks: { boundary: async point => { if (point === 'drained') { reached(); await gate; } } } });
+ await drained;
+ try { await fn(); } catch (error) { release(); await refresh.catch(() => undefined); throw error; }
+ release();
+ return refresh;
+}
+
+test('items 3 and 15: writes held deferred count toward their source\'s fair share, so another source still gets its share of the local CLI cap', () => each(async f => {
+ await f.engine.setConfig('persistence.limits.principal_outstanding', '2');
+ try {
+ foreignCommit(f, 'one.md', page('Alpha one', IMPORTED));
+ const first = await local(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Held first.') });
+ const second = await local(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Held second.') });
+ expect([first.pending, second.pending]).toEqual([true, true]);
+ expect((await requestRow(f, second.receipt.request_id)).state).toBe('deferred');
+ const other = await local(f, 'put_page', { slug: 'beta-note', content: page('Beta note', 'Admitted on its fair share.') }, f.beta);
+ expect(other.receipt.state ?? 'committed').not.toBe('failed');
+ expect((await requestRow(f, other.receipt.request_id)).state).toBe('committed');
+ } finally {
+ await f.engine.executeRaw("DELETE FROM config WHERE key='persistence.limits.principal_outstanding'");
+ }
+}), 180_000);
+
+test('item 3: a released deferred write is the earlier write a later queued write waits on in receipt health', () => each(async f => {
+ let deferred!: WriteRequest;
+ const refresh = await duringRefresh(f, async () => {
+ const out = await call(f, 'put_page', { slug: 'notes/deferred', content: page('Deferred', 'written during the refresh') });
+ deferred = await requestRow(f, out.body.request_id);
+ expect(deferred.state).toBe('deferred');
+ await disposePersistenceConsumer(f.engine);
+ });
+ expect((await refresh).status).toBe('completed');
+ const released = (await getWriteRequestById(f.engine, deferred.id))!;
+ expect(released.state).toBe('deferred');
+ const later = { ...released, id: randomUUID(), state: 'queued', sequence: String(BigInt(released.sequence) + 1n), deferred_refresh_id: null } as WriteRequest;
+ const facts = await writeHealthFacts(f.engine, [released, later]);
+ expect(facts.get(later.id)?.earlier_write).toBe(true);
+ expect(facts.get(released.id)?.earlier_write).toBe(false);
+ expect((await settle(f, released)).state).toBe('committed');
+}), 180_000);
+
+test('item 19: a write deferred behind a refresh awaiting recovery reports recovery_required, and a replay of its request id is never deferred again', () => each(async f => {
+ f.push('alpha/two.md', page('Alpha two', 'Upstream.'));
+ const old = f.head();
+ await expect(refreshWorktree(f.engine, f.alpha, { hooks: { boundary: point => { if (point === 'merged') throw new Error('simulated crash after merge'); } } }))
+ .rejects.toThrow('simulated crash');
+ git(f.root, 'reset', '-q', '--hard', old);
+ git(f.root, 'commit', '-q', '--allow-empty', '-m', 'unrelated');
+ await resumeWorktreeRefreshes(f.engine);
+ expect((await f.refreshRows()).map(r => r.state)).toEqual(['recovery_required']);
+ const params = { slug: 'notes/during-recovery', content: page('During recovery', 'Accepted while the refresh awaited recovery.') };
+ const out = await call(f, 'put_page', params);
+ expect(out.body.status).toBe('pending');
+ const receipt = await call(f, 'get_write_request', { request_id: out.body.request_id });
+ expect(receipt.body.diagnostic).toMatchObject({ reason: 'recovery_required' });
+ const replay = await call(f, 'put_page', { ...params, request_id: out.body.request_id });
+ expect(replay.body.request_id).toBe(out.body.request_id);
+ const [{ n }] = await f.engine.executeRaw<{ n: number }>('SELECT count(*)::int AS n FROM persistence_requests WHERE request_id=$1::uuid', [out.body.request_id]);
+ expect(Number(n)).toBe(1);
+ git(f.root, 'reset', '-q', '--hard', old);
+ expect((await refreshWorktree(f.engine, f.alpha, { abandon: true })).status).toBe('abandoned');
+ expect((await settle(f, await requestRow(f, out.body.request_id))).state).toBe('committed');
+}), 180_000);
+
+test('item 13: a hold refuses only a request admitted after the held write, never an effect or maintenance read of the file', () => each(async f => {
+ const before = await f.lastCommit(f.alpha);
+ foreignCommit(f, 'one.md', page('Alpha one', IMPORTED));
+ foreignCommit(f, 'zz-later.md', page('Alpha later', 'A second file in the same import.'));
+ const first = await local(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Held while the import waits.') });
+ const held = await requestRow(f, first.receipt.request_id);
+ expect(held.state).toBe('deferred');
+ // A one-page slice imports one.md and yields before its checkpoint: the file matches the page, and the hold is unreleased.
+ expect((await performManagedSync(f.engine, { sourceId: f.alpha, noPull: true, noEmbed: true }, { maxPages: 1, maxMs: 60_000 })).status).toBe('partial');
+ expect(await f.lastCommit(f.alpha)).toBe(before);
+ const snapshot = (await f.engine.readPageSnapshot('one', { sourceId: f.alpha }))!;
+ const target = { source_id: f.alpha, worktree_id: f.worktreeId, slug: 'one' };
+ const effect = await prepareFileTarget(f.engine, target, snapshot, null, undefined, { allowMissing: true });
+ expect(effect?.path).toBe(join(f.root, 'alpha', 'one.md'));
+ const later = prepareFileTarget(f.engine, { ...target, sequence: String(BigInt(held.sequence) + 1n), operation: 'put_page' }, snapshot, null, undefined, { allowMissing: true });
+ await expect(later).rejects.toMatchObject({ code: 'source_changed' });
+ const earlier = await prepareFileTarget(f.engine, { ...target, sequence: String(BigInt(held.sequence) - 1n), operation: 'put_page' }, snapshot, null, undefined, { allowMissing: true });
+ expect(earlier?.path).toBe(join(f.root, 'alpha', 'one.md'));
+ await importNow(f);
+ expect((await settle(f, held)).state).toBe('committed');
+}), 180_000);
+
+test('item 1: a deferred write never lands on top of a later write to its page that committed first', () => each(async f => {
+ let deferred!: WriteRequest;
+ const refresh = await duringRefresh(f, async () => {
+ const out = await local(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Deferred, accepted first.') });
+ expect(out.pending).toBe(true);
+ deferred = await requestRow(f, out.receipt.request_id);
+ expect(deferred.state).toBe('deferred');
+ await disposePersistenceConsumer(f.engine);
+ });
+ expect((await refresh).status).toBe('completed');
+ // An older consumer claims without EARLIER_DEFERRED: the later write to the page commits before the released deferred one.
+ // Emulated by hiding the refresh's release while the later write publishes.
+ const [{ released_at }] = await f.engine.executeRaw<{ released_at: Date }>('SELECT released_at FROM persistence_worktree_refreshes WHERE id=$1::uuid', [deferred.deferred_refresh_id]);
+ await f.engine.executeRaw('UPDATE persistence_worktree_refreshes SET released_at=NULL WHERE id=$1::uuid', [deferred.deferred_refresh_id]);
+ let later: Awaited<ReturnType<typeof local>>;
+ try {
+ later = await local(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Accepted second, committed first.') });
+ } finally {
+ await f.engine.executeRaw('UPDATE persistence_worktree_refreshes SET released_at=$2 WHERE id=$1::uuid', [deferred.deferred_refresh_id, released_at]);
+ }
+ expect((await requestRow(f, later!.receipt.request_id)).state).toBe('committed');
+ const done = await settle(f, deferred);
+ expect(done).toMatchObject({ state: 'conflict', error_code: 'revision_conflict' });
+ expect(done.error_message).toContain(later!.receipt.request_id);
+ expect(file(f, 'one.md')).toContain('Accepted second, committed first.');
+ // The same check covers a write held for an import.
+ const imported = { ...done, deferred_refresh_id: null, deferred_import_from: '' } as WriteRequest;
+ expect(await deferredReplayRefusal(f.engine, imported, [])).toMatchObject({ state: 'conflict', failure: { code: 'revision_conflict' } });
+ expect(await deferredReplayRefusal(f.engine, { ...imported, operation: 'remember' } as WriteRequest, [])).toBeNull();
+}), 180_000);
+
+test('item 16: a write deferred behind a refresh is held for an import without its released refresh, unless the refresh changed its file', () => each(async f => {
+ f.push('alpha/three.md', page('Alpha three', 'Before the refresh.'));
+ expect((await refreshWorktree(f.engine, f.alpha)).status).toBe('completed');
+ expect(await f.engine.readPageSnapshot('three', { sourceId: f.alpha })).not.toBeNull();
+ let kept!: WriteRequest, changed!: WriteRequest;
+ const refresh = await duringRefresh(f, async () => {
+ kept = await requestRow(f, (await local(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Deferred; the refresh left this file alone.') })).receipt.request_id);
+ changed = await requestRow(f, (await local(f, 'put_page', { slug: 'three', content: page('Alpha three', 'Deferred; the refresh changed this file.') })).receipt.request_id);
+ expect([kept.state, changed.state]).toEqual(['deferred', 'deferred']);
+ await disposePersistenceConsumer(f.engine);
+ }, ['alpha/three.md', page('Alpha three', 'Upstream.')]);
+ expect((await refresh).status).toBe('completed');
+ foreignCommit(f, 'one.md', page('Alpha one', IMPORTED));
+ foreignCommit(f, 'three.md', page('Alpha three', IMPORTED));
+ await drive(f, changed);
+ await drive(f, kept);
+ const held = (await getWriteRequestById(f.engine, kept.id))!;
+ expect(held).toMatchObject({ state: 'deferred', deferred_refresh_id: null });
+ expect(held.deferred_import_from).not.toBeNull();
+ // A consumer older than the import hold claims a deferred row once its refresh is released: it finds none here.
+ const [{ n }] = await f.engine.executeRaw<{ n: number }>(`SELECT count(*)::int AS n FROM persistence_requests r WHERE r.id=$1::uuid AND r.state='deferred'
+ AND EXISTS (SELECT 1 FROM persistence_worktree_refreshes f WHERE f.id=r.deferred_refresh_id AND f.released_at IS NOT NULL)`, [kept.id]);
+ expect(Number(n)).toBe(0);
+ expect(await getWriteRequestById(f.engine, changed.id)).toMatchObject({ state: 'conflict', error_code: 'revision_conflict' });
+ await importNow(f);
+ expect((await settle(f, held)).state).toBe('committed');
+ expect(file(f, 'one.md')).toContain('Deferred; the refresh left this file alone.');
+}), 180_000);
+
+test('item 14: a write held again and again expires at the ceiling counted from its admission', () => each(async f => {
+ foreignCommit(f, 'one.md', page('Alpha one', IMPORTED));
+ const out = await local(f, 'put_page', { slug: 'one', content: page('Alpha one', 'Held again after every import.') });
+ const held = await requestRow(f, out.receipt.request_id);
+ expect(held.state).toBe('deferred');
+ await maxWait(f, 60_000);
+ try {
+ await f.engine.executeRaw("UPDATE persistence_requests SET created_at=now()-interval '3 hours' WHERE id=$1::uuid", [held.id]);
+ expect(await expireDeferredWrites(f.engine, await ownerHost(f))).toEqual([]);
+ await f.engine.executeRaw("UPDATE persistence_requests SET created_at=now()-interval '5 hours' WHERE id=$1::uuid", [held.id]);
+ expect((await expireDeferredWrites(f.engine, await ownerHost(f))).map(row => row.id)).toEqual([held.id]);
+ } finally { await clearMaxWait(f); }
+ expect(await getWriteRequestById(f.engine, held.id)).toMatchObject({ state: 'failed', error_code: 'pending_import_expired' });
+ expect(file(f, 'one.md')).toContain(IMPORTED);
+}), 180_000);
+
+test('item 20 (confirmation): a held write never outlives its source row; a source row is removed only through writer administration, which ends it', () => each(async f => {
+ foreignCommit(f, 'one.md', page('Beta one', IMPORTED), 'beta');
+ const out = await local(f, 'put_page', { slug: 'one', content: page('Beta one', 'Held, then its source was removed.') }, f.beta);
+ const held = await requestRow(f, out.receipt.request_id);
+ expect(held.state).toBe('deferred');
+ await expect(deleteSourceRow(f.engine, f.beta)).rejects.toThrow('writer_coordinator_required');
+ const principal = await topologyPrincipal(f.engine);
+ expect(await f.engine.transaction(tx => settleTopologyRequests(tx, [f.beta], [f.worktreeId], principal))).toBe(1);
+ expect(await getWriteRequestById(f.engine, held.id)).toMatchObject({ state: 'conflict', error_code: 'source_changed' });
+}), 180_000);
+
+const PERSON = '---\ntitle: Alice Example\ntype: person\n---\nAlice is a person.\n';
+
+test('item 21: an expired remember says one replay attempt is queued only when it named its entity', () => each(async f => {
+ foreignCommit(f, 'people/alice-example.md', PERSON);
+ await importNow(f);
+ foreignCommit(f, 'people/alice-example.md', PERSON.replace('Alice is a person.', `Alice is a person. ${IMPORTED}`));
+ const pinned = await requestRow(f, (await call(f, 'remember', { fact: 'alice-example likes tea', entity: 'people/alice-example', provenance: 'test' })).body.request_id);
+ const unpinned = await requestRow(f, (await call(f, 'remember', { fact: 'alice-example likes coffee', entity: 'people/alice-example', provenance: 'test' })).body.request_id);
+ expect([pinned.state, unpinned.state]).toEqual(['deferred', 'deferred']);
+ await f.engine.executeRaw("UPDATE persistence_requests SET intent=intent-'entity_slug' WHERE id=$1::uuid", [unpinned.id]);
+ await maxWait(f, 1);
+ try {
+ await new Promise(resolve => setTimeout(resolve, 20));
+ expect((await expireDeferredWrites(f.engine, await ownerHost(f))).map(row => row.id).sort()).toEqual([pinned.id, unpinned.id].sort());
+ } finally { await clearMaxWait(f); }
+ const replayed = (await getWriteRequestById(f.engine, pinned.id))!, manual = (await getWriteRequestById(f.engine, unpinned.id))!;
+ expect(replayed.error_message).toContain('one automatic replay attempt');
+ expect(replayed.error_message).toContain(`gbrain repair failed-writes --source ${f.alpha}`);
+ expect(manual.error_message).toContain('named no entity, so no sync replays it');
+ for (const row of [replayed, manual]) expect(row.error_message).toContain('Do not save it again');
+}), 180_000);
+
+test('item 17: the retry-hint history reads an index of completed refreshes per worktree', async () => {
+ for (const engine of engines) {
+ const [index] = await engine.executeRaw<{ indexdef: string }>("SELECT indexdef FROM pg_indexes WHERE indexname='persistence_worktree_refreshes_completed'");
+ expect(index?.indexdef).toContain('(worktree_id, completed_at DESC)');
+ expect(index?.indexdef).toContain("state = 'completed'");
+ }
+});
+
+test('item 18: a refresh whose row recovery moved off fenced before the merge merges nothing', () => each(async f => {
+ f.push('alpha/two.md', page('Alpha two', 'Upstream.'));
+ const old = f.head();
+ const result = await refreshWorktree(f.engine, f.alpha, { hooks: { boundary: async (point, row) => {
+ if (point === 'fenced') await f.engine.executeRaw("UPDATE persistence_worktree_refreshes SET state='aborted',completed_at=now(),updated_at=now() WHERE id=$1::uuid", [(row as { id: string }).id]);
+ } } }).catch(error => error);
+ expect(result).toBeInstanceOf(OperationError);
+ expect((result as OperationError).code).toBe('refresh_source_changed');
+ expect(f.head()).toBe(old);
+}), 180_000);
+
+test('item 22: the drain never sleeps a refresh retry hint past its refresh budget', () => {
+ const refreshing = new OperationError('worktree_refreshing', 'refreshing');
+ refreshing.detail = 'retry_after_ms=240000';
+ expect(_transientDelayForTests(refreshing, 1, 0, 250)).toBe(240_000);
+ expect(_transientDelayForTests(refreshing, 1, 290_000, 250)).toBe(10_000);
+ expect(_transientDelayForTests(refreshing, 1, 300_000, 250)).toBeNull();
+});
+
+test('item 5: a case-insensitive checkout compares the refresh\'s changed paths case-folded', () => each(async f => {
+ let deferred!: WriteRequest;
+ const refresh = await duringRefresh(f, async () => {
+ deferred = await requestRow(f, (await call(f, 'put_page', { slug: 'notes/case', content: page('Case', 'deferred') })).body.request_id);
+ await disposePersistenceConsumer(f.engine);
+ });
+ expect((await refresh).status).toBe('completed');
+ const row = (await getWriteRequestById(f.engine, deferred.id))!;
+ const files = [{ root: f.root, path: join(f.root, 'alpha', 'TWO.md') }];
+ git(f.root, 'config', 'core.ignorecase', 'false');
+ expect(await deferredReplayRefusal(f.engine, row, files)).toBeNull();
+ _resetRefreshBarrierCacheForTests();
+ git(f.root, 'config', 'core.ignorecase', 'true');
+ try {
+ expect(await deferredReplayRefusal(f.engine, row, files)).toMatchObject({ state: 'conflict', failure: { code: 'revision_conflict' } });
+ } finally { git(f.root, 'config', 'core.ignorecase', 'false'); _resetRefreshBarrierCacheForTests(); }
+ expect((await settle(f, row)).state).toBe('committed');
+}), 180_000);
+
+test('item 2 (confirmation): polling a receipt never publishes a released deferred write; the next consumer on the owner host does', () => each(async f => {
+ let deferred!: WriteRequest;
+ const refresh = await duringRefresh(f, async () => {
+ deferred = await requestRow(f, (await call(f, 'put_page', { slug: 'notes/polled', content: page('Polled', 'deferred') })).body.request_id);
+ await disposePersistenceConsumer(f.engine);
+ });
+ expect((await refresh).status).toBe('completed');
+ for (let i = 0; i < 3; i++) await call(f, 'get_write_request', { request_id: deferred.request_id });
+ expect((await getWriteRequestById(f.engine, deferred.id))!.state).toBe('deferred');
+ expect((await settle(f, deferred)).state).toBe('committed');
+}), 180_000);
+
+test('#8759: the managed sync capacity discount counts only writes still held for that source', () => each(async f => {
+ await f.engine.setConfig('persistence.limits.principal_outstanding', '1');
+ try {
+ foreignCommit(f, 'one.md', page('Beta one', IMPORTED), 'beta');
+ const out = await local(f, 'put_page', { slug: 'one', content: page('Beta one', 'Held for the beta import.') }, f.beta);
+ expect((await requestRow(f, out.receipt.request_id)).state).toBe('deferred');
+ foreignCommit(f, 'one.md', page('Alpha one', IMPORTED));
+ const before = await f.lastCommit(f.alpha);
+ const alpha = await importNow(f).then(result => result, error => error);
+ expect(alpha instanceof Error ? (alpha as OperationError).code : (alpha as { status: string }).status).not.toBe('synced');
+ expect(await f.lastCommit(f.alpha)).toBe(before);
+ await importNow(f, f.beta);
+ expect(await f.lastCommit(f.beta)).toBe(git(f.root, 'rev-parse', 'HEAD'));
+ } finally {
+ await f.engine.executeRaw("DELETE FROM config WHERE key='persistence.limits.principal_outstanding'");
+ }
+}), 180_000);
+
+test('#8759: a held capture of the very file its import brought in names no supersedes', () => each(async f => {
+ const content = page('Alpha one', `${IMPORTED} Captured from this file.`);
+ foreignCommit(f, 'one.md', content);
+ const out = await local(f, 'capture', { slug: 'one', content, local_file: join(f.root, 'alpha', 'one.md') });
+ expect(out.pending).toBe(true);
+ const held = await requestRow(f, out.receipt.request_id);
+ expect(held.state).toBe('deferred');
+ expect(typeof held.intent?.capture_file_hash).toBe('string');
+ await importNow(f);
+ const done = await settle(f, held);
+ expect(done.state).toBe('committed');
+ expect((done.outcome as { supersedes?: unknown } | null)?.supersedes).toBeUndefined();
+}), 180_000);
diff --git a/test/refresh-barrier-state-audit.test.ts b/test/refresh-barrier-state-audit.test.ts
index 6275eaf9a1..e73d0da37d 100644
--- a/test/refresh-barrier-state-audit.test.ts
+++ b/test/refresh-barrier-state-audit.test.ts
@@ -27,7 +27,7 @@ const DECIDED: Record<string, { count: number; why: string }> = {
'src/core/persistence/atom-maintenance.ts': { count: 3, why: 'maintenance producer, never deferred' },
'src/core/persistence/atom-retry.ts': { count: 2, why: 'receipt state' },
'src/core/persistence/checkpoint-validation.ts': { count: 2, why: 'managed sync' },
- 'src/core/persistence/journal.ts': { count: 8, why: 'claim and head selectors pair the list with RELEASED_DEFERRED (EARLIER_HEAD with its own EARLIER_DEFERRED probe); the two named-sync lists match sync rows, never deferred; the rest are recovery rules' },
+ 'src/core/persistence/journal.ts': { count: 7, why: 'claim and head selectors pair the list with RELEASED_DEFERRED (EARLIER_HEAD with its own EARLIER_DEFERRED probe); the two named-sync lists match sync rows, never deferred; the rest are recovery rules' },
'src/core/persistence/managed-sync-orphans.ts': { count: 4, why: 'managed sync' },
'src/core/persistence/refresh-barrier.ts': { count: 1, why: 'the downgrade backlog names deferred ahead of the open states: a released deferred row that is still queued or running has not published' },
'src/core/persistence/page-batch.ts': { count: 1, why: 'batches are never deferred' },