Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(cycle): derived atom/concept write-through commits each batch through the commit door (#8073)
garrytan9cade77cc8f4
diff --git a/docs/architecture/key-files/core-cycle-patterns.md b/docs/architecture/key-files/core-cycle-patterns.mdindex 65d75892b0..b3f08133fb 100644--- a/docs/architecture/key-files/core-cycle-patterns.md+++ b/docs/architecture/key-files/core-cycle-patterns.md@@ -21,7 +21,7 @@ Patterns, drift reports and the cycle modules listed after them.- `src/core/cycle/triage-rescue.ts` — THE dream triage gate (`passesTriageGate`) + the verified-segment rescue (`applyTriageRescue`). Rescue fires only for scores in `[floor, threshold)` (floor inclusive; at/above threshold is the plain gate's job) with content_type in the buried-signal allowlist (default mixed/reflection/idea/strategy/people — never routine/technical) and ≥ minSegments of the judge's own segments verifying as normalized transcript substrings (`normForGrounding`, ≥40 chars, deduped by normalized quote — repetition is not more evidence). $0 — no LLM calls; fabricated segments cannot fire it; works on cached verdicts (gate-time only). Fail-closed on every malformed shape (null score, missing/short/non-string segments), never throws. `minSegments: 0` is the kill switch (gate degenerates to the plain threshold). ONE-GATE RULE: `runTriagePass` (report construction — `worth`/`rescued`/telemetry/dry-run), the synthesize fan-out, and `dream retriage` (reconcile-queue cancels + `--audit-rejects` sampling) all read this predicate — a second hand-rolled `score >= threshold` check is how an operator sweep cancels exactly the jobs the rescue admitted. Config: `dream.triage.rescue_floor` / `rescue_min_segments` / `rescue_content_types`. Pinned by `test/cycle-triage-rescue.test.ts` + the rescue suites in `test/cycle-synthesize-triage.test.ts` / `test/dream-retriage.test.ts`. System One S7: a decide verdict (`model` starting with `DECIDE_VERDICT_PREFIX`) is gated against the S7 slot threshold (the optional fifth argument) and never enters the rescue band.-- `src/core/cycle/derived-write-through.ts` — opt-in markdown write-through for generated pages on an unmanaged brain (#5041). `cycle.extract_atoms.write_through` / `cycle.synthesize_concepts.write_through` (off by default; `true`/`1`/`yes`/`on`) make `derivedWriteThrough` return a hook (null when off, managed or dry run): `extract_atoms` writes each item's atoms after the completion flip and removes the files of atoms it retired; `synthesize_concepts` writes through `publishClassicConcept`, which also rewrites a concept that already has a file whatever the key says. Creating the hook writes up to 200 generated pages per run that have no file yet (`source_path` NULL; pending atoms excluded), so enabling the key backfills. `writeDerivedPageThrough` keeps a page database-only when its target is ignored by git or sits in a declared `storage.db_only` directory (`derivedFileTarget`), replaces an existing file only when `replaceableFile` (`facts/file-body-guard.ts`) finds it equal to the page or one of its recent page_versions rows, then calls `writePageThrough` bound to the bytes compared and returns its result (`written`, `skipped`, `error`) so `publishClassicConcept` can surface a failed write. A retired atom's file is removed under the same filter and comparison, the unlink rechecking the bytes under the source filesystem lock. A file holding an edit the database lacks (or an unindexed file at a backfilled page's path) is left as it is and listed in the hook's `held`, which both phases report as `details.file_holds` (`file_database_drift`) with a `warn` status. Tests: `test/cycle/derived-write-through.test.ts`, `test/pathological/d10-derived-atom-file-overwrite.test.ts`, `test/pathological/d11-derived-retired-atom-unlink.test.ts`. #6982: `writeDreamPageFile` is the dream reverse-write (synthesize, patterns, the cycle summary): nothing with `sync.write_through` off, for an owned url clone or a `storage.db_only` slug; the path is `resolvePageWriteTarget`'s (recorded source_path), falling back to the caller's brainDir layout when the source has no root or its root is missing; the file is replaced only when `replaceableFile` (facts/file-body-guard.ts) finds nothing in it the database lacks and, once the replacement is staged, still holds the compared bytes right before the rename, else the result carries the `file_database_drift` hold. It writes directly (not `writePageThrough`/`atomicWriteFileSync`) because a checkout a subagent tool write claimed through the coordinator refuses those with `writer_coordinator_required`. `reportFileHolds` lists every phase's holds in `details.file_holds` and turns ok into warn.+- `src/core/cycle/derived-write-through.ts` — opt-in markdown write-through for generated pages on an unmanaged brain (#5041). `cycle.extract_atoms.write_through` / `cycle.synthesize_concepts.write_through` (off by default; `true`/`1`/`yes`/`on`) make `derivedWriteThrough` return a hook (null when off, managed or dry run): `extract_atoms` writes each item's atoms after the completion flip and removes the files of atoms it retired; `synthesize_concepts` writes through `publishClassicConcept`, which also rewrites a concept that already has a file whatever the key says. Creating the hook writes up to 200 generated pages per run that have no file yet (`source_path` NULL; pending atoms excluded), so enabling the key backfills. `writeDerivedPageThrough` keeps a page database-only when its target is ignored by git or sits in a declared `storage.db_only` directory (`derivedFileTarget`), replaces an existing file only when `replaceableFile` (`facts/file-body-guard.ts`) finds it equal to the page or one of its recent page_versions rows, then calls `writePageThrough` bound to the bytes compared and returns its result (`written`, `skipped`, `error`) so `publishClassicConcept` can surface a failed write. #8073: every file write and retired-file unlink is uncommitted (`deferCommit`) and the batch (an item's atoms plus its retired atoms' files, the backfill, or one concept) commits once through the shared-checkout door `commitPathsLocked` (`git/commit-push.ts`) under the source filesystem lock, naming only the batch's paths that exist or are tracked; a directory that is not a Git checkout has nothing to commit. A refused commit restores each file still holding the batch's bytes (prior bytes, or removed), clears the `source_path` the write bound so the next backfill rewrites it, and returns a typed `DerivedCommitFailure` (`{code, message, paths, kept}`); the hook then writes nothing more that run and the phases report `details.write_through_failed` with `warn` (`reportDerivedWriteThrough`), while `writeDerivedPageThrough` returns it as `error` so the concept is held. Tests: `test/cycle/derived-write-through-commit.test.ts` (PGLite and Postgres). A retired atom's file is removed under the same filter and comparison, the unlink rechecking the bytes under the source filesystem lock. A file holding an edit the database lacks (or an unindexed file at a backfilled page's path) is left as it is and listed in the hook's `held`, which both phases report as `details.file_holds` (`file_database_drift`) with a `warn` status. Tests: `test/cycle/derived-write-through.test.ts`, `test/pathological/d10-derived-atom-file-overwrite.test.ts`, `test/pathological/d11-derived-retired-atom-unlink.test.ts`. #6982: `writeDreamPageFile` is the dream reverse-write (synthesize, patterns, the cycle summary): nothing with `sync.write_through` off, for an owned url clone or a `storage.db_only` slug; the path is `resolvePageWriteTarget`'s (recorded source_path), falling back to the caller's brainDir layout when the source has no root or its root is missing; the file is replaced only when `replaceableFile` (facts/file-body-guard.ts) finds nothing in it the database lacks and, once the replacement is staged, still holds the compared bytes right before the rename, else the result carries the `file_database_drift` hold. It writes directly (not `writePageThrough`/`atomicWriteFileSync`) because a checkout a subagent tool write claimed through the coordinator refuses those with `writer_coordinator_required`. `reportFileHolds` lists every phase's holds in `details.file_holds` and turns ok into warn.- `src/core/cycle/concept-publication.ts` — managed `synthesize_concepts` publication: `publishManagedConcept` publishes through `publishMaintenancePage` (a concept with a recorded `source_path` republishes to its file unless a `db_only` tier covers it; new concepts are database-only), and `composeConceptRepublication` keeps an existing concept's facts/takes fences, timeline, tags and other frontmatter, holding the concept (`concept_preservation_hold`) when a fence is duplicated, unbalanced, unparseable or below the timeline sentinel. `replaceConceptProvenanceLinks` banks the concept-provenance edges and retires the edges of atoms that left the group in one transaction (coordinated on a managed brain), only while the concept is at the revision its publication produced (GBRA-161 PR 5, C-4). On an unmanaged brain `publishClassicConcept` writes under the page lock the fence writers take: it re-reads the page, defers (`revision_conflict`) when the narrative changed since synthesis started, another writer owns it, or the page's canonical file holds anything the database lacks (`replaceableFile`: body, fences, timeline, frontmatter, tags, type or title equal to neither the page nor a recent page_versions row; formatting is ignored; this also covers an unindexed file at a new concept's path, and the message names the page with `file_database_drift`), and otherwise composes the new narrative with the database's latest fences, timeline, tags and frontmatter, so a take added during synthesis survives (`test/pathological/d12-concept-stale-fences.test.ts`, `d13-concept-occupied-path.test.ts`). The file write after `importPage` passes the bytes read under the lock as `expectedFileBytes`, so an edit landing during the import is kept and the concept deferred; a failed file write holds the concept (`concept_write_through_failed`) instead of reporting success (W9F item 3, `test/cycle/concept-publication-file-recheck.test.ts`). `stripFenceSections` is the shared narrative view. Tests: `test/cycle/synthesize-concepts-fences.test.ts`. The phase keeps the private-first, provenance, then promotion order; revision races are deferred and holds rep... (line truncated to 2000 chars)- `src/core/cycle/connector-atoms.ts` — the connector atom setting (#5856). `CONNECTOR_ATOM_PAGES_KEY` (`cycle.extract_atoms.connector_pages`, on when unset; `connectorAtomPagesEnabled` treats only false/0/off/no as opted out) gates `CONNECTOR_ATOM_PAGE_TYPES` (`email`, `meeting`) of sources whose `config.kind` is a connector kind (`CONNECTOR_SOURCE_KINDS`). `connectorAtomExclusionSql` returns the `pages p` predicate atom discovery and the backlog count append (empty unless opted out), so the routine cycle, `dream --drain` and autopilot auto-drain extract connector mail and calendar pages under the auto-drain budget until the user opts out. Other page types of a connector source and every checkout-backed page are unaffected. Tests: `test/connector-atom-pages.test.ts`, `test/autopilot-auto-drain-dispatch.test.ts`, and the `atom_drain_default_on` / `atom_drain_opted_out` cases of `test/helpers/managed-connector-job-contract.ts`, run by `test/managed-connector-job-contract.test.ts` and its Postgres twin `test/e2e/managed-connector-job-contract.test.ts`.diff --git a/docs/architecture/key-files/files-and-sync-2.md b/docs/architecture/key-files/files-and-sync-2.mdindex c24690e405..522680a31e 100644--- a/docs/architecture/key-files/files-and-sync-2.md+++ b/docs/architecture/key-files/files-and-sync-2.md@@ -29,4 +29,4 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `src/core/persistence/sync-entry-fault.ts` — GBRA-133 (#6712 PR 4b): the per-entry faults of a managed sync freeze, so one odd file never stops its source. `nullBytesRefusal` holds a Markdown text with U+0000 `content_null_bytes` (line of the first NUL; a UTF-16 byte order mark says save as UTF-8), from `screenFrozenImport` with holds on and as a typed run stop under `sync.holds=fail`. `committedContentOrMissing` (called by `pinnedContent` when the pinned read fails) imports a committed file whose blob object is gone from working-tree bytes that hash to that blob id, else throws `git_object_missing`; a committed name with U+FFFD that names nothing on disk (a name that is not UTF-8) throws `path_not_utf8` first. `directoryAt` lets a deletion whose name became a directory apply (freeze and the owner's raw-hash check). `freezeFaultHold` is `freezeEntry`'s catch (`sync-run.ts` wraps `freezeEntryOnce`): `git_object_missing` and `path_not_utf8` are held under their codes, any other `OperationError` and every run-scoped error (`classifyItemFailure` scope run, connection, statement timeout, lock, back-pressure, abort) rethrows, and an unexpected throw is held `page_write_failed` with `write_failure.origin: 'freeze'` and `request_id: null`, re-tested by every sync until held `HOLD_ATTEMPTS_NEEDS_HUMAN` times; a run with `ITEM_BREAKER_MIN_ITEMS` faults and no more clean freezes than faults is systemic and rethrows.- `src/core/url-clone.ts` — gbrain-owned `sources add --url` clones (#6946, GBRA-130 C-2). `isOwnedClone` (config `managed_clone` or a `local_path` under the default clone dir) and `defaultCloneDir`; `urlCloneReadOnly` is the one predicate (owned clone on a brain without managed persistence) the fence writer (`fence-write.ts`, `databaseOnlyReason: 'url_clone'`), write-through (`skipped: 'url_clone'`), timeline write-through, unfenced-facts adoption, the empty-fence guard and `sourceMirrorReadOnly` consult, so nothing gbrain writes lands in a checkout it updates with `git pull --ff-only`. `urlClonePullBlock` reads the tracked changes of a clone whose pull failed and returns `pull_blocked_local_changes` with an `ask_user` fix naming the preview `gbrain repair url-clone-fences --source <id>`; `formatUrlClonePullBlock` renders it for sync's stderr (`sync/preflight.ts`) and `cycle.ts` puts it in the sync phase detail as `pull_blocked`. Test: `test/pathological/c2-url-clone-fences.test.ts`.-- `src/core/write-through.ts` — shared atomic disk write-through for the canonical ingestion path. `resolvePageWriteTarget(engine, slug, sourceId, { includeDeleted? })` (active rows by default; `includeDeleted: true` resolves a tombstone's recorded `source_path` for the purge remediation retry) is the exported single source of truth for the ONE file a (source, slug) pair lives in on disk: a source with its own `local_path` writes at that tree's ROOT (never nested under `.sources/`); a source without one nests under the host repo (`sync.repo_path`, default at root, non-default under `.sources/<id>/`) unless that path is another source's own working tree (leak guard → `source_repo_belongs_to_other_source`; a live source whose `local_path` is relative counts as owning every tree, and a relative `local_path` or `sync.repo_path` target is `repo_not_found`); the page's recorded `source_path` (or a contained absolute `file://` `source_uri`; a relative or host-only `file://` names no local file and is read as absent) is preferred over a slug-derived name so writes land in the file of record instead of minting a twin; and `isWriteTargetContained` rejects hostile rows escaping the tree (`path_escapes_source_root`). The ok-result also carries `sourcePathToBind` — the target expressed in the file scanner's `source_path` convention for the source's pinned `slug_root_mode` (or the mode its first sync would pin, via `scannerSlugRootMode`): GIT-ROOT-relative under `git-root` when the scan root sits inside a git repo, scan-root-relative under `source-root` and for non-git or git-root scan roots, so a put-born page binds the same form managed sync and delete-reconcile key on. Every recorded-path reader (write targets, takes, reconcile, audit, export restore, doctor) decodes with the same mode (`resolveSourceLocalFilePath(..., slugRootMode)`): under `source-root` the scope prefix is removed only from the legacy Git-root form whose remainder names the page slug, and basename rows resolve under the unstripped slug directory. The resolver is shared by `writePageThrough`, the facts fence writer (`src/core/facts/fence-write.ts`), and fence forget (`src/core/facts/forget.ts`) — the fence appends to the page's file, so all writers MUST compute the identical path or the fence lands in a file sync never reads back and the next extract_facts reconcile deletes the fence-owned DB rows. `writePageThrough(engine, slug, {sourceId?, frontmatterOverrides?, logger?, expectedFileBytes?})` resolves the target through it, re-reads the just-written DB row (`getPage`), renders it via `serializePageToMarkdown`, and writes the `.md` so the brain has a committable artifact that round-trips through `gbrain sync`. Rendering from the row keeps the serialized content consistent at write time. The fsync-backed `atomicWriteFileSync` stages and renames the file after required source-path bookkeeping; lock/lease checks run before publication. `put_page` holds the root lock before importing and invokes write-through inside its database transaction, so ordinary filesystem rejection rolls back existing content, tags, chunks, and versions. A crash or database COMMIT failure after rename can still leave canonical Markdown ahead of the index; this is not distributed ACID or a durable queue. Never throws — returns `WriteThroughResult { written, path?, skipped?: 'disabled_by_config' | 'no_repo_configured' | 'repo_not_found' | 'source_repo_belongs_to_other_source' | 'page_not_found_after_write' | 'path_escapes_source_root' | 'case_insensitive_collision' | 'file_changed', warning?, error? }` (`file_changed`: the caller's `expectedFileBytes` preimage, null for absent, no longer matches the file under the source filesystem lock, so the file is left untouched) so the caller decides messaging + exit codes. `withNoRepoWriteThroughWarning(result, sourceId)` decorates an unwritten `no_repo_configured` result with `warning` (the write landed only in the database; bind the token/server to a git-backed source or configure `local_path`/`sync.repo_path`); `put_page` applies it for untrusted callers only (`ctx.remote !== false`), so local CLI responses and the `disabled_by_config` operator opt-out stay silent. `isWriteThroughDisabled` (shared with the fence lane, ~30s per-engine cache) honors the `sync.write_through` opt-out. Trust gating (subagent sandbox, dry-run) stays at the CALLER. On a durability-hardened repo (`isDurabilityHardened` — the gbrain post-commit hook is installed, i.e. the user ran `gbrain sources harden`), a successful write is best-effort COMMITTED via `commitWriteThroughFile` (path-limited `git commit -- <file>`, never sweeps unrelated edits; the hook then background-pushes) so write-through content reaches git instead of accumulating uncommitted forever; result carries `committed?: boolean`. Unhardened repos keep write-only behavior. `deletePageThrough(engine, slug, {sourceId?, target?, logger?})` is the delete-side sink (sole caller: the `delete_page` op): resolves the same target, unlinks the `.md`, and on a hardened repo best-effort commits the removal path-limited through the same helper with subject `gbrain: delete write-through <slug>`, reporting `committed?: boolean` on `DeleteThroughResult` (absent when unhardened or when the commit did not land — the unlink still stands; a never-committed page leaves nothing to commit, so no empty commit). Pinned by `test/delete-write-through-commit.test.ts`. Consumers: `put_page` op and `gbrain brainstorm/lsd --save` via `persistSavedIdea`. Pinned by `test/write-through.test.ts` + `test/write-through-commit.test.ts`. The `put_page` op is that deciding caller for direct writes: a skip outside the deliberate DB-only set (`no_repo_configured`/`disabled_by_config`/`subagent_sandbox`/`dry_run`) throws `storage_error` inside the import transaction, rolling back the page revision, tags, chunks and version snapshot instead of deleting a just-created row afterward. #6946: on a gbrain-owned `--url` clone of an unmanaged brain (`urlCloneReadOnly`), `writePageThrough` returns `skipped: 'url_clone'` and `deletePageThrough` returns `removed: false, skipped: 'url_clone'`; `timeline-write-through.ts` skips the same way.+- `src/core/write-through.ts` — shared atomic disk write-through for the canonical ingestion path. `resolvePageWriteTarget(engine, slug, sourceId, { includeDeleted? })` (active rows by default; `includeDeleted: true` resolves a tombstone's recorded `source_path` for the purge remediation retry) is the exported single source of truth for the ONE file a (source, slug) pair lives in on disk: a source with its own `local_path` writes at that tree's ROOT (never nested under `.sources/`); a source without one nests under the host repo (`sync.repo_path`, default at root, non-default under `.sources/<id>/`) unless that path is another source's own working tree (leak guard → `source_repo_belongs_to_other_source`; a live source whose `local_path` is relative counts as owning every tree, and a relative `local_path` or `sync.repo_path` target is `repo_not_found`); the page's recorded `source_path` (or a contained absolute `file://` `source_uri`; a relative or host-only `file://` names no local file and is read as absent) is preferred over a slug-derived name so writes land in the file of record instead of minting a twin; and `isWriteTargetContained` rejects hostile rows escaping the tree (`path_escapes_source_root`). The ok-result also carries `sourcePathToBind` — the target expressed in the file scanner's `source_path` convention for the source's pinned `slug_root_mode` (or the mode its first sync would pin, via `scannerSlugRootMode`): GIT-ROOT-relative under `git-root` when the scan root sits inside a git repo, scan-root-relative under `source-root` and for non-git or git-root scan roots, so a put-born page binds the same form managed sync and delete-reconcile key on. Every recorded-path reader (write targets, takes, reconcile, audit, export restore, doctor) decodes with the same mode (`resolveSourceLocalFilePath(..., slugRootMode)`): under `source-root` the scope prefix is removed only from the legacy Git-root form whose remainder names the page slug, and basename rows resolve under the unstripped slug directory. The resolver is shared by `writePageThrough`, the facts fence writer (`src/core/facts/fence-write.ts`), and fence forget (`src/core/facts/forget.ts`) — the fence appends to the page's file, so all writers MUST compute the identical path or the fence lands in a file sync never reads back and the next extract_facts reconcile deletes the fence-owned DB rows. `writePageThrough(engine, slug, {sourceId?, frontmatterOverrides?, logger?, expectedFileBytes?})` resolves the target through it, re-reads the just-written DB row (`getPage`), renders it via `serializePageToMarkdown`, and writes the `.md` so the brain has a committable artifact that round-trips through `gbrain sync`. Rendering from the row keeps the serialized content consistent at write time. The fsync-backed `atomicWriteFileSync` stages and renames the file after required source-path bookkeeping; lock/lease checks run before publication. `put_page` holds the root lock before importing and invokes write-through inside its database transaction, so ordinary filesystem rejection rolls back existing content, tags, chunks, and versions. A crash or database COMMIT failure after rename can still leave canonical Markdown ahead of the index; this is not distributed ACID or a durable queue. Never throws — returns `WriteThroughResult { written, path?, skipped?: 'disabled_by_config' | 'no_repo_configured' | 'repo_not_found' | 'source_repo_belongs_to_other_source' | 'page_not_found_after_write' | 'path_escapes_source_root' | 'case_insensitive_collision' | 'file_changed', warning?, error? }` (`file_changed`: the caller's `expectedFileBytes` preimage, null for absent, no longer matches the file under the source filesystem lock, so the file is left untouched) so the caller decides messaging + exit codes. `withNoRepoWriteThroughWarning(result, sourceId)` decorates an unwritten `no_repo_configured` result with `warning` (the write landed only in the database; bind the token/server to a git-backed source or configure `local_path`/`sync.repo_path`); `put_page` applies it for untrusted callers only (`ctx.remote !== false`), so local CLI responses and the `disabled_by_config` operator opt-out stay silent. `isWriteThroughDisabled` (shared with the fence lane, ~30s per-engine cache) honors the `sync.write_through` opt-out. Trust gating (subagent sandbox, dry-run) stays at the CALLER. On a durability-hardened repo (`isDurabilityHardened` — the gbrain post-commit hook is installed, i.e. the user ran `gbrain sources harden`), a successful write is best-effort COMMITTED via `commitWriteThroughFile` (path-limited `git commit -- <file>`, never sweeps unrelated edits; the hook then background-pushes) unless the caller passes `deferCommit` (the derived write-through commits its batch itself, #8073; `deletePageThrough` takes it too) so write-through content reaches git instead of accumulating uncommitted forever; result carries `committed?: boolean`. Unhardened repos keep write-only behavior. `deletePageThrough(engine, slug, {sourceId?, target?, logger?})` is the delete-side sink (sole caller: the `delete_page` op): resolves the same target, unlinks the `.md`, and on a hardened repo best-effort commits the removal path-limited through the same helper with subject `gbrain: delete write-through <slug>`, reporting `committed?: boolean` on `DeleteThroughResult` (absent when unhardened or when the commit did not land — the unlink still stands; a never-committed page leaves nothing to commit, so no empty commit). Pinned by `test/delete-write-through-commit.test.ts`. Consumers: `put_page` op and `gbrain brainstorm/lsd --save` via `persistSavedIdea`. Pinned by `test/write-through.test.ts` + `test/write-through-commit.test.ts`. The `put_page` op is that deciding caller for direct writes: a skip outside the deliberate DB-only set (`no_repo_configured`/`disabled_by_config`/`subagent_sandbox`/`dry_run`) throws `storage_error` inside the import transaction, rolling back the page revision, tags, chunks and version snapshot instead of deleting a just-created row afterward. #6946: on a gbrain-owned `--url` clone of an unmanaged brain (`urlCloneReadOnly`), `writePageThrough` returns `skipped: 'url_clone'` and `deletePageThrough` returns `removed: false, skipped: 'url_clone'`; `timeline-write-through.ts` skips the same way.diff --git a/scripts/module-size-limits.tsv b/scripts/module-size-limits.tsvindex d80fca694b..8a648d4ec6 100644--- a/scripts/module-size-limits.tsv+++ b/scripts/module-size-limits.tsv@@ -47,4 +47,4 @@ src/commands/doctor/checks/extraction-sync.ts 1500 ratchet +7 GBRA-130 A-1: linksrc/core/search/evidence-delivery.ts 1755 ratchet +5 W5: EvidencePlan.rankCut resolved with the plan (logic in search/page-rank-cut.ts); +20 W5 whole_pages: wholeCost prices a block as delivery would emit it whole (needs the private Block/enrichBlock internals) and the cut is wired before allocate (cutWholePages lives in search/page-rank-cut.ts); new row: crossed the 1500 unlisted cap at the #6367 merge (master 1479, its explicit-budget cap) by the C1 date header threaded through every block path, cap included; pagePlanHits (search/page-plan-hits.ts) and the header loader (evidence-date.ts blockDateHeaders) were peeled first; peel candidate: allocateCapped; +8 at the #6396 merge: page evidence text cut through the fence-chunk trust overlay and the caller minTrust floor (re-summed: our 1504 + master +8); +119 Cat 40 next wave F2: the search.long_page_unit policy (withLongPagePolicy, selectLongPages size preflight with page/byte/deadline limits) and the long-page fetch and fallback in deliverEvidence, kept in the shared seam the plan names (gbrain-evals docs/plans/2026-10-11-cat40-next-wave, F2); +17 F2 rework to the frozen-comparison policy: search.long_page_min_chars (rendered-body characters, default 20,000) read in withLongPagePolicy, a character count in the preflight, the rendered-body confirmation in deliverEvidence, and the long-page row exemption in capDeliveredSnippets (re-summed at the 0.60.225.0 merge: master 1537 + F2 136); +39 Wave B PR 2 lane 2b: planSegmentedBlock joins the windows search/segmented-evidence.ts reads around a segmented page's hits into one run-marked document (needs the private Doc/Piece/buildDoc/locateAnchor internals), planBlock's unit cut split into blockFromDoc for both, sections bounded by their run, selectionCost counts a gap between runs, and the fetch skips the whole-body fence overlay for a segmented page; +4 segmented pages read at once before planning (segmentedReads; the read is search/segmented-evidence.ts, the statements page-body/read-range.ts); +17 GBRA-156: tail trim plan flag, DeliverOptions.query, DeliveryMeta.tail_trim, the trim call and the frozen-hit score/query library fields (logic in search/evidence-tail-trim.ts; interface fields cannot move); +7 GBRA-205: EvidencePlan.titleHeader and DeliveryMeta.title_header (interface fields cannot move) and the titles option on the header loader (logic in search/evidence-date.ts; the query budget key lives in search/evidence-packing.ts); +11 #7600: assemble_evidence quotes conversation rows and capEvidenceToBudget counts the quote fence (logic in search/transcript-quote.ts); +2 GBRA-156 /ship review: the cut-path admission and title room count the quote fence; +2 GBRA-156 /ship review pass 2: capEvidenceToBudget counts the fence only for the paths that fence (quote flag; recall counts the text as returned)src/commands/config.ts 1524 ratchet new row: crossed the 1500 unlisted cap in wave 15 P7.4 #5461: set-time validation of think.pages_budget_chars / think.excerpt_ceiling_chars (range + parser live in core/think/pages-budget.ts), mirroring the cycle.consolidate.cluster_threshold hook; +1 wave 16 G5.1 (#6495): set-time refusal hook for search.keyword_statement_timeout_ms (check lives in commands/config/keyword-timeout-value.ts); +1 wave 18b lane 5 (#5012 item 6): search.contextual_retrieval prints the existing-pages hint after the write (text lives in commands/config/contextual-mode-hint.ts); +1 #6712 PR 2: retired/rollback sync key dispatch line (logic in core/sync-config-keys.ts handleSyncKeyConfig); +1 #6865: sync.slug_conflicts.auto_remove boolean validation at set time (refusal text in repair/slug-autoheal.ts); +1 #6948: sync.origin_twins.auto_resolve boolean validation at set time (refusal text in repair/origin-twins-heal.ts); +2 #6795: config set chat_model/expansion_model help line and the model-pin import (shadow report and file-pin unset live in commands/config/model-pin-shadow.ts); +1 #6712 PR 4b: sync.admission set-time hook (value logic in commands/config/sync-admission-value.ts); +4 #7130: set-time refusal hook for sync.max_deletes / sync.max_delete_ratio (and .source.<id>) / purge.retention_hours (check lives in core/sync-reconcile.ts deleteSafetyConfigRefusal); +1 GBRA-150: sync.metafile_pages set-time route (body in config/sync-metafile-pages-value.ts); re-summed at the v0.60.330.0 merge (1520 measured); +2 #7242 minions.terminal_hook file-plane key and its set branch (logic in minions/terminal-hook.ts); +1 GBRA-161: generations.* set-time route (refusal in commands/config/generations-values.ts); +1 GBRA-207 #7630: sync.repo_path set-time refusal hook (refusal in core/sync-config-keys.ts syncRepoPathRefusal)src/commands/mcp-expose.ts 1519 ratchet hosted_option: Session carries the gbrain.io coaching notice on the first-receipt run and Tailscale/service friction exits (copy, refs and gating live in core/onboard/hosted-option.ts); +7 GBRA-199: --surface help lines, surfaceChosen option field and the advertise pass-through (registrationAdvertise in core/mcp-registration.ts)-src/core/behavior-change-notice.ts 1570 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope)+src/core/behavior-change-notice.ts 1571 ratchet the BEHAVIOR_CHANGES table grows by one row per disclosed change: +4 the #7891 provenance-drift and #7917/#7918 maintenance-lowering rows and their comments (re-summed at the 0.60.495.0 merge: 1547 + 4); +2 GBRA-205 saved_facts_stems row and its comment (re-summed at the #7901 0.60.492.0 merge: 1541 + 2); +2 GBRA-205 search.query_facts_block default-on row and its approval comment (re-summed at the #7610 0.60.490.0 merge: 1539 + 2); +2 the #7900 content checkpoint host-move row (since TBD until the merge slot), +2 the #7897 private-hold sweep row (since TBD until the merge slot), +2 the #7880 ingest_capture and company-brain private-hold row (since TBD until the merge slot), +5 the frozen-source wave comment and four rows (#7868, #7850, #7869/#7870, #7871), +2 the #7848 visibility_noncanonical hold row (since TBD until the merge slot), +2 the #7846 extract_atoms session-source row (re-summed at the 0.60.481.0 merge: master 1524 + 2), +2 the #7805 synthesize session-source and #7841 visibility_not_applied rows (re-summed at the 0.60.480.0 merge: master 1522 + 2), +2 strict door planned-state row (#7705, re-summed at the 0.60.470.0 merge: master 1514 + 2), +2 Bun floor 1.4.4 row (#7644), the GBRA-95 RLS backfill row (+2), +3 rows for #7596 pass_interrupted, #7599 read-only serve and the --access=read-only ceiling, +1 G5 lane C trust_release row, +1 #7619 cli_flag_equals, +3 GBRA-207 #7630 relative-anchor and stalled-write rows, +2 the #7632 function search_path disclosure row and +2 the #7679 orphaned worktree refresh row (re-summed at the 0.60.471.0 merge: master 1516 + 2), +2 the #7814 retired v0.13.1 grandfather row, +2 the #7618 conversion withhold/progress/push row (master 1520 + 2); the behavior-change table split (#7607) retires this row; +4 GBRA-188 #7489: the wave comment and three rows (brain-first instructions, starter surface default, per-turn rule); +1 GBRA-188 plugin_turn_hook_gbra188 row; +1 GBRA-242 provider_key_rules_7959 row (#7959), +2 the GBRA-199 registration_lists_verbs row (since TBD until the merge slot); +2 GBRA-133 #7652: the #7611 pass_abandoned reader row and the lean read-only serve socket row (re-summed at the 0.60.503.0 merge: master 1555 + 2); +1 GBRA-243: the per-turn hook disclosure row (rows are data and cannot move); +4 GBRA-156 #7598 harness wave comment and rows (remote page cap, sources_not_readable refusal, sources add next step); +4 GBRA-156 #7601 search_quotes_transcript_rows comment and row; +1 GBRA-175 #7475 autopilot_restarts_serve_7475 row (re-summed at the #8012 merge: 1566 + 1); +3 #7997/#8014/#8015 rows (redaction markers, company db_only keep, ignored supabase_only scope); +1 GBRA-250 #8073 derived_write_through_commits_8073 row (since TBD until the merge slot)diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.tsindex a834798000..a32c69f07d 100644--- a/src/core/behavior-change-notice.ts+++ b/src/core/behavior-change-notice.ts@@ -1372,6 +1372,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;{ since: '0.60.514.0', id: 'storage_ignored_dirs_8015', text: 'When gbrain.yml lists a deprecated `supabase_only` next to canonical keys (which ignores it), a managed sync now holds only the missing-file pages under the ignored directories `db_only` does not already cover, and deletes other missing-file pages as usual; it used to hold every missing-file page of the source. `gbrain storage status` now names those directories, and the deprecation warning no longer points to a `gbrain doctor --fix` rename that does not exist. A `db_only` or `supabase_only` line whose inline value the parser refuses (a missing comma, a nested list, a bare scalar) now counts as unresolved even when another key resolves, so missing-file pages are held rather than deleted.' },{ since: '0.60.514.0', id: 'company_db_only_keep_8014', text: 'A company-brain source now keeps a live page under a declared `storage.db_only` directory when its file is not in the approved manifest, and keeps every such page while its gbrain.yml storage section is invalid, as other managed sources do; it used to delete them. The decision reads gbrain.yml at the approved commit, never the working tree.' },{ since: '0.60.514.0', id: 'redaction_marker_not_credential_7997', text: 'The transcript `labeled_credential` detector no longer reads an importer redaction marker (`login **<REDACTED:user-pattern>**`) as a typed user/password pair, so a correctly redacted page clears `gbrain transcripts audit-secrets`; a password written beside a marker is still found. The audit detector version moves to 3, so `gbrain doctor` asks for a fresh audit once.' },+ { since: 'TBD', id: 'derived_write_through_commits_8073', text: 'With `cycle.extract_atoms.write_through` or `cycle.synthesize_concepts.write_through` on, the atom and concept files the dream cycle writes into a source that is a Git checkout are now committed: one local commit per batch (an item\'s atoms plus the files of the atoms it retired, the backfill, or one concept), naming only those files, through the checkout\'s commit lock (a hardened checkout\'s post-commit hook pushes it). They used to stay untracked or modified, which made the scheduled sync skip every run. A commit the checkout refuses rolls back the files it wrote, reports `write_through_failed` with a warn status, and the next run writes them again. Atom files left uncommitted by earlier versions are not touched; commit them with `gbrain git commit-push -m <msg> -- <atom directory>`.' },];/** The newest disclosed change's release: the notice id moves only when a release adds rows. */diff --git a/src/core/cycle/derived-write-through.ts b/src/core/cycle/derived-write-through.tsindex 26937e62b6..5e316c1e30 100644--- a/src/core/cycle/derived-write-through.ts+++ b/src/core/cycle/derived-write-through.ts@@ -14,6 +14,11 @@* generated pages that have no file yet, a bounded batch per run. A page the* phase retires has its file removed, so sync cannot bring it back.*+ * #8073: in a Git checkout those files are committed, one commit per batch+ * through the shared-checkout door (`commitPathsLocked`), and a refused commit+ * rolls the batch's files back (`commitDerivedChanges`), so the cycle never+ * leaves untracked or modified files that make the scheduled sync skip.+ ** #6982: a file is replaced or removed only when it holds nothing the* database lacks (`replaceableFile`: absent, or equal to the page or one of* its recent versions), and the write is bound to the bytes compared. A file@@ -31,7 +36,11 @@ import { validateSourceId } from '../utils.ts';import { isDbOnly, loadStorageConfig } from '../storage-config.ts';import { deletePageThrough, isWriteThroughDisabled, resolvePageWriteTarget, writePageThrough, type PageWriteTarget } from '../write-through.ts';import { urlCloneReadOnly } from '../url-clone.ts';-import { gitChildEnv } from '../git-env.ts';+import { gitChildEnv, literalPathspecs } from '../git-env.ts';+import { atomicWriteFileSync } from '../atomic-write.ts';+import { classifyGitCheckout } from '../git-checkout.ts';+import { commitPathsLocked } from '../git/commit-push.ts';+import { withSourceFilesystemLock } from '../minions/source-filesystem.ts';import { fileDatabaseDriftToken, replaceableFile } from '../facts/file-body-guard.ts';export const DERIVED_WRITE_THROUGH_KEYS = {@@ -76,16 +85,18 @@ export async function derivedFileTarget(engine: BrainEngine, slug: string, sourcreturn { ok: true, target };}-/**- * Write one generated page's markdown file. Never throws; the result says- * whether a file was written, why not, or the write error. Ignored and db_only- * targets are skipped on purpose (`skipped: 'not_canonical'`).- * `expectedFileBytes` binds the write to the file the caller read (and- * already compared); without it the file must pass `replaceableFile` first,- * else nothing is written (`skipped: 'file_database_drift'`).- */-export async function writeDerivedPageThrough(engine: BrainEngine, slug: string, sourceId: string,- opts: { expectedFileBytes?: string | null } = {}): Promise<{ written: boolean; skipped?: string; error?: string }> {+/** One file a derived write changed (#8073): its bytes before and after (null: absent) and the source_path the write bound. */+interface DerivedFileChange { slug: string; sourceId: string; filePath: string; writeRoot: string; prior: string | null; after: string | null; bound: string | null }++/** A batch commit the checkout refused: the door's code and message, the batch's files (root-relative) and those left as they were. */+export interface DerivedCommitFailure { code: string; message: string; paths: string[]; kept: string[] }++/** How long a batch waits for the checkout's commit lock before it rolls back. */+const DERIVED_COMMIT_LOCK_WAIT_MS = 30_000;++/** The file write of `writeDerivedPageThrough`, uncommitted, with the change it made. */+async function writeDerivedFile(engine: BrainEngine, slug: string, sourceId: string,+ opts: { expectedFileBytes?: string | null }): Promise<{ written: boolean; skipped?: string; error?: string; change?: DerivedFileChange }> {const resolved = await derivedFileTarget(engine, slug, sourceId);if (!resolved.ok) return { written: false, skipped: resolved.skipped };let expectedFileBytes = opts.expectedFileBytes;@@ -94,24 +105,101 @@ export async function writeDerivedPageThrough(engine: BrainEngine, slug: string,if (!file.ok) return { written: false, skipped: 'file_database_drift' };expectedFileBytes = file.expectedFileBytes;}- const result = await writePageThrough(engine, slug, { sourceId, expectedFileBytes });+ const [row] = await engine.executeRaw<{ source_path: string | null }>(+ 'SELECT source_path FROM pages WHERE source_id = $1 AND slug = $2 AND deleted_at IS NULL', [sourceId, slug]);+ const result = await writePageThrough(engine, slug, { sourceId, expectedFileBytes, deferCommit: true });if (result.error) console.error(`[write-through] ${slug}: ${result.error}`);- return result;+ if (!result.written || !result.path) return result;+ return { ...result, change: { slug, sourceId, filePath: result.path, writeRoot: resolved.target.writeRoot, prior: expectedFileBytes,+ after: readFileSync(result.path, 'utf-8'), bound: row?.source_path === null ? resolved.target.sourcePathToBind : null } };+}++/**+ * Write one generated page's markdown file and commit it (#8073). Never+ * throws; the result says whether a file was written, why not, or the write+ * error. Ignored and db_only targets are skipped on purpose+ * (`skipped: 'not_canonical'`). `expectedFileBytes` binds the write to the+ * file the caller read (and already compared); without it the file must pass+ * `replaceableFile` first, else nothing is written+ * (`skipped: 'file_database_drift'`). A commit the checkout refuses rolls the+ * file back and comes back as `error` (`<code>: <message>`).+ */+export async function writeDerivedPageThrough(engine: BrainEngine, slug: string, sourceId: string,+ opts: { expectedFileBytes?: string | null } = {}): Promise<{ written: boolean; skipped?: string; error?: string }> {+ const { change, ...result } = await writeDerivedFile(engine, slug, sourceId, opts);+ if (!change) return result;+ const [failure] = await commitDerivedChanges(engine, 'synthesize_concepts', [change]);+ return failure ? { written: false, error: `${failure.code}: ${failure.message}` } : result;}/**- * Remove a retired generated page's file when the retired page (or one of- * its versions) holds everything in it; the unlink is bound to the bytes- * compared. Returns `file_database_drift` when the file holds an edit the- * database lacks.+ * Remove a retired generated page's file, uncommitted, when the retired page+ * (or one of its versions) holds everything in it; the unlink is bound to the+ * bytes compared. Returns `file_database_drift` when the file holds an edit+ * the database lacks.*/-async function deleteDerivedPageThrough(engine: BrainEngine, slug: string, sourceId: string): Promise<{ removed: boolean; skipped?: string }> {+async function deleteDerivedFile(engine: BrainEngine, slug: string, sourceId: string): Promise<{ removed: boolean; skipped?: string; change?: DerivedFileChange }> {const resolved = await derivedFileTarget(engine, slug, sourceId, { includeDeleted: true });if (!resolved.ok) return { removed: false, skipped: resolved.skipped };const file = await replaceableFile(engine, sourceId, slug, resolved.target.filePath);if (!file.ok) return { removed: false, skipped: 'file_database_drift' };if (file.expectedFileBytes === null) return { removed: false, skipped: 'file_not_present' };- return deletePageThrough(engine, slug, { sourceId, target: resolved.target, expectedFileBytes: file.expectedFileBytes });+ const result = await deletePageThrough(engine, slug, { sourceId, target: resolved.target, expectedFileBytes: file.expectedFileBytes, deferCommit: true });+ if (!result.removed) return result;+ return { ...result, change: { slug, sourceId, filePath: resolved.target.filePath, writeRoot: resolved.target.writeRoot, prior: file.expectedFileBytes, after: null, bound: null } };+}++/**+ * #8073: commit a batch of derived file changes through the shared-checkout commit door (`commitPathsLocked`: the commit+ * lock, `commit --only` of exactly the named paths), one commit per checkout, under the source filesystem lock. Only paths+ * that exist or are tracked are named (an untracked file the batch removed has nothing to commit). A directory that is not+ * a Git checkout has nothing to commit: its files are the record. A refused commit rolls the batch back+ * (`rollBackDerived`) and comes back as a typed failure.+ */+async function commitDerivedChanges(engine: BrainEngine, phase: keyof typeof DERIVED_WRITE_THROUGH_KEYS,+ changes: readonly DerivedFileChange[]): Promise<DerivedCommitFailure[]> {+ const failures: DerivedCommitFailure[] = [];+ for (const root of new Set(changes.map(c => c.writeRoot))) {+ const batch = changes.filter(c => c.writeRoot === root);+ if (classifyGitCheckout(root) === 'not_git') continue;+ try {+ await withSourceFilesystemLock(engine, root, async () => {+ const rels = batch.map(c => relative(root, c.filePath));+ const tracked = new Set(execFileSync('git', ['-C', root, 'ls-files', '-z', '--', ...literalPathspecs(rels)], { env: gitChildEnv(), encoding: 'utf8' }).split('\0'));+ const paths = batch.filter((c, i) => existsSync(c.filePath) || tracked.has(rels[i]!)).map(c => c.filePath);+ if (paths.length) commitPathsLocked(root, paths, `gbrain: ${phase} write-through (${paths.length} generated file(s))`, { lockWaitMs: DERIVED_COMMIT_LOCK_WAIT_MS });+ });+ } catch (error) {+ failures.push(await rollBackDerived(engine, root, batch, error));+ }+ }+ return failures;+}++/**+ * Undo a batch whose commit was refused: each file still holding the bytes the batch wrote gets its prior bytes back (or is+ * removed when it had none), and the source_path the write bound is cleared so the next run's backfill writes the page+ * again. A file changed since (another writer's edit) is left as it is and listed in `kept`.+ */+async function rollBackDerived(engine: BrainEngine, root: string, batch: readonly DerivedFileChange[], error: unknown): Promise<DerivedCommitFailure> {+ const code = typeof (error as { code?: unknown }).code === 'string' ? (error as { code: string }).code : 'git_unavailable';+ const message = error instanceof Error ? error.message : String(error);+ const kept: string[] = [];+ for (const change of batch) {+ try {+ if ((existsSync(change.filePath) ? readFileSync(change.filePath, 'utf-8') : null) !== change.after) throw new Error('file_changed');+ if (change.prior === null) rmSync(change.filePath, { force: true });+ else atomicWriteFileSync(change.filePath, change.prior);+ if (change.bound) {+ await engine.executeRaw('UPDATE pages SET source_path = NULL WHERE source_id = $1 AND slug = $2 AND source_path = $3 AND deleted_at IS NULL',+ [change.sourceId, change.slug, change.bound]);+ }+ } catch {+ kept.push(relative(root, change.filePath));+ }+ }+ console.error(`[write-through] commit refused (${code}): ${batch.length - kept.length} generated file(s) rolled back, ${kept.length} left as they are`);+ return { code, message, paths: batch.map(c => relative(root, c.filePath)), kept };}/** `result` with its held page files reported: added to `details.file_holds`, a `warn` status unless it failed, and a summary note. */@@ -122,17 +210,32 @@ export function reportFileHolds(result: PhaseResult, held: readonly string[] | usummary: `${result.summary} (${held.length} ${noun} file(s) left as they are: file_database_drift)` };}-/** The phase's write-through hook: `write` publishes pages and removes retired ones' files; `held` reports each page left alone. */+/** `result` with the hook's held files (`reportFileHolds`) and refused commits reported: `details.write_through_failed` and a `warn` status unless it failed. */+export function reportDerivedWriteThrough(result: PhaseResult, files: DerivedWriteThrough | null, noun: string): PhaseResult {+ const reported = reportFileHolds(result, files?.held, noun);+ if (!files?.failed.length) return reported;+ const count = files.failed.reduce((n, f) => n + f.paths.length - f.kept.length, 0);+ return { ...reported, status: reported.status === 'fail' ? 'fail' : 'warn',+ details: { ...reported.details, write_through_failed: files.failed },+ summary: `${reported.summary} (${noun} file commit refused: ${files.failed[0]!.code}; ${count} file(s) rolled back, written again on the next run)` };+}++/**+ * The phase's write-through hook: `write` publishes pages and removes retired ones' files, one commit per call; `held`+ * reports each page left alone, `failed` each refused commit (after which the hook writes nothing more this run).+ */export interface DerivedWriteThrough {write(slugs: readonly string[], retired?: readonly string[]): Promise<void>;held: string[];+ failed: DerivedCommitFailure[];}/*** The phase's write-through hook, or null when it does not apply (key off,* managed brain, dry run): writes the given pages and removes the files of- * the retired ones. Creating it writes the bounded backfill of generated- * pages of the phase's type in the source that have no file yet.+ * the retired ones, then commits the batch. Creating it writes the bounded+ * backfill of generated pages of the phase's type in the source that have no+ * file yet.*/export async function derivedWriteThrough(engine: BrainEngine,@@ -142,18 +245,22 @@ export async function derivedWriteThrough(): Promise<DerivedWriteThrough | null> {if (opts.managed || opts.dryRun || !await derivedWriteThroughEnabled(engine, phase)) return null;const held: string[] = [];+ const failed: DerivedCommitFailure[] = [];const kept = (skipped: string | undefined) => skipped === 'file_database_drift' || skipped === 'file_changed';const write = async (slugs: readonly string[], retired: readonly string[] = []) => {+ if (failed.length) return;+ const changes: DerivedFileChange[] = [];for (const slug of slugs) {- if (kept((await writeDerivedPageThrough(engine, slug, sourceId)).skipped)) {- held.push(fileDatabaseDriftToken(slug, sourceId, 'before', 'the generated page was not written to its file'));- }+ const result = await writeDerivedFile(engine, slug, sourceId, {});+ if (result.change) changes.push(result.change);+ if (kept(result.skipped)) held.push(fileDatabaseDriftToken(slug, sourceId, 'before', 'the generated page was not written to its file'));}for (const slug of retired) {- if (kept((await deleteDerivedPageThrough(engine, slug, sourceId)).skipped)) {- held.push(fileDatabaseDriftToken(slug, sourceId, 'before', 'the retired page\'s file was not removed'));- }+ const result = await deleteDerivedFile(engine, slug, sourceId);+ if (result.change) changes.push(result.change);+ if (kept(result.skipped)) held.push(fileDatabaseDriftToken(slug, sourceId, 'before', 'the retired page\'s file was not removed'));}+ failed.push(...await commitDerivedChanges(engine, phase, changes));};const rows = await engine.executeRaw<{ slug: string }>(`SELECT slug FROM pages@@ -164,7 +271,7 @@ export async function derivedWriteThrough([sourceId, phase === 'extract_atoms' ? 'atom' : 'concept', DERIVED_BACKFILL_LIMIT],);await write(rows.map(r => r.slug));- return { write, held };+ return { write, held, failed };}/** What a dream reverse-write did with one page's file. `held` is its `file_database_drift` entry. */diff --git a/src/core/cycle/extract-atoms.ts b/src/core/cycle/extract-atoms.tsindex 5608e11762..fe9a412626 100644--- a/src/core/cycle/extract-atoms.ts+++ b/src/core/cycle/extract-atoms.ts@@ -66,7 +66,7 @@ import type { GBrainConfig } from '../config.ts';import type { ProgressReporter } from '../progress.ts';import { chat as gatewayChat, withBudgetTracker } from '../ai/gateway.ts';import { createGlobalLlmHaltTracker, haltedClassOf, providerContentBlockReason, type GlobalLlmErrorClass } from '../ai/errors.ts';-import { derivedWriteThrough, reportFileHolds } from './derived-write-through.ts';+import { derivedWriteThrough, reportDerivedWriteThrough } from './derived-write-through.ts';import { emptySessionSourceRouting, keepSessionSourceTranscripts, withSessionSourceRouting } from './synthesize-source-routing.ts';import { serializeMarkdown } from '../markdown.ts';import { DEFAULT_EXTRACT_MAX_INPUT_CHARS, extractAtomsInput, inputTruncatedLine, recordPartialDisposition, releasePartialDispositions, resolveExtractAtomsInputCap, stampTranscriptTombstone, type ExtractAtomsInput, type ExtractAtomsItem } from './extract-atoms-input.ts';@@ -878,7 +878,7 @@ export async function runPhaseExtractAtoms(// Phase-level no-op: nothing to extract today.if (work.length === 0 && transcripts.length === 0 && pages.length === 0) {- return withSessionSourceRouting(reportFileHolds({+ return withSessionSourceRouting(reportDerivedWriteThrough({phase: 'extract_atoms',status: 'skipped',duration_ms: 0,@@ -898,7 +898,7 @@ export async function runPhaseExtractAtoms(budget_usd: DEFAULT_BUDGET_USD,dry_run: opts.dryRun ?? false,},- }, atomFiles?.held, 'atom'), sourceRouting, sourceId, 'extract_atoms');+ }, atomFiles, 'atom'), sourceRouting, sourceId, 'extract_atoms');}// 4. Per work-item: extract atoms via the configured extract_atoms model@@ -1425,7 +1425,7 @@ export async function runPhaseExtractAtoms(});}- return withSessionSourceRouting(reportFileHolds({+ return withSessionSourceRouting(reportDerivedWriteThrough({phase: 'extract_atoms',// A phase that skipped every work item and produced nothing did not// succeed, even though skips are not failures and leave failures[] empty.@@ -1472,7 +1472,7 @@ export async function runPhaseExtractAtoms(source_id: sourceId,dry_run: opts.dryRun ?? false,},- }, atomFiles?.held, 'atom'), sourceRouting, sourceId, 'extract_atoms');+ }, atomFiles, 'atom'), sourceRouting, sourceId, 'extract_atoms');}/**diff --git a/src/core/cycle/synthesize-concepts.ts b/src/core/cycle/synthesize-concepts.tsindex bf27834ce5..49e29b5226 100644--- a/src/core/cycle/synthesize-concepts.ts+++ b/src/core/cycle/synthesize-concepts.ts@@ -37,7 +37,7 @@ import { slugifySegment } from '../sync.ts';import { validatePageSlug } from '../ops/context.ts';import { privatePagesFilterFragment, strictestVisibility, type Visibility } from '../search/private-visibility.ts';import { maintenancePreflight } from '../persistence/prepared-maintenance.ts';-import { derivedWriteThrough, reportFileHolds } from './derived-write-through.ts';+import { derivedWriteThrough, reportDerivedWriteThrough } from './derived-write-through.ts';import { addSpellingRedirects, canonicalConceptStem, loadConceptRedirects, loadLiveConceptStems } from './concept-redirects.ts';import { conceptProvenanceCovers, conceptPublicationHold, stripFenceSections } from './concept-publication.ts';import { conceptWriter, type ConceptWrite } from './concept-writes.ts';@@ -563,7 +563,7 @@ export async function runPhaseSynthesizeConcepts(});}- return reportFileHolds({+ return reportDerivedWriteThrough({phase: 'synthesize_concepts',status: failures.length > 0 || linkWarnings.length > 0 || publicationDeferred.length > 0 || publicationHeld.length > 0 || skippedFileDrift.length > 0 ? 'warn' : 'ok',duration_ms: 0,@@ -607,7 +607,7 @@ export async function runPhaseSynthesizeConcepts(pricing_fallback_models: [...pricingFallbackModels],dry_run: opts.dryRun ?? false,},- }, conceptFiles?.held, 'concept');+ }, conceptFiles, 'concept');}/**diff --git a/src/core/write-through.ts b/src/core/write-through.tsindex ef842e50eb..3a84d0c5b9 100644--- a/src/core/write-through.ts+++ b/src/core/write-through.ts@@ -117,6 +117,8 @@ export interface WritePageThroughOpts {* (`skipped: 'file_changed'`). Omitted: no preimage check.*/expectedFileBytes?: string | null;+ /** The caller commits the file itself (#8073: one commit per batch), so the hardened per-file commit is skipped. */+ deferCommit?: boolean;}export function withNoRepoWriteThroughWarning<T extends { written: boolean; skipped?: string; warning?: string }>(result: T, sourceId: string, operation = 'put_page'): T {@@ -528,7 +530,7 @@ export async function writePageThrough(let lastPushStatus: PushLogOutcome | undefined;try {assertSourceFilesystemActive();- if (isDurabilityHardened(writeRoot)) {+ if (!opts.deferCommit && isDurabilityHardened(writeRoot)) {committed = commitWriteThroughFile(writeRoot, filePath, slug);if (committed) {pushed = 'pending';@@ -607,7 +609,7 @@ export interface DeleteThroughResult {export async function deletePageThrough(engine: BrainEngine,slug: string,- opts: { sourceId?: string; logger?: WriteThroughLogger; target?: PageWriteTarget; expectedFileBytes?: string } = {},+ opts: { sourceId?: string; logger?: WriteThroughLogger; target?: PageWriteTarget; expectedFileBytes?: string; deferCommit?: boolean } = {},): Promise<DeleteThroughResult> {const sourceId = opts.sourceId ?? 'default';try {@@ -644,7 +646,7 @@ export async function deletePageThrough(// never fails the delete (the DB row + the unlink are the durable state).let committed = false;try {- if (isDurabilityHardened(writeRoot)) {+ if (!opts.deferCommit && isDurabilityHardened(writeRoot)) {committed = commitWriteThroughFile(writeRoot, filePath, slug, 'delete write-through');}} catch { /* best-effort */ }diff --git a/test/cycle/concept-publication-file-recheck.test.ts b/test/cycle/concept-publication-file-recheck.test.tsindex 4c6bbad186..2c4ea2a76a 100644--- a/test/cycle/concept-publication-file-recheck.test.ts+++ b/test/cycle/concept-publication-file-recheck.test.ts@@ -49,6 +49,8 @@ for (const backend of testBackends()) describe(`publishClassicConcept file reche_resetWriteThroughCacheForTest();repo = mkdtempSync(join(tmpdir(), 'gbrain-concept-recheck-'));execFileSync('git', ['init', '-q'], { cwd: repo });+ execFileSync('git', ['config', 'user.name', 'Example Writer'], { cwd: repo });+ execFileSync('git', ['config', 'user.email', '[email protected]'], { cwd: repo });await engine.executeRaw("UPDATE sources SET local_path = $1 WHERE id = 'default'", [repo]);await importFromContent(engine, SLUG, serializeMarkdown(frontmatter, BASELINE, '', { type: 'concept', title: 'launch dates', tags: ['ops'] }), { noEmbed: true });const written = await writePageThrough(engine, SLUG, { sourceId: 'default' });diff --git a/test/cycle/derived-write-through-commit.test.ts b/test/cycle/derived-write-through-commit.test.tsindex e2ca96b50d..4ecb2dcb5a 100644--- a/test/cycle/derived-write-through-commit.test.ts+++ b/test/cycle/derived-write-through-commit.test.ts@@ -97,7 +97,7 @@ async function withBrain(databaseUrl: string | undefined, layout: Layout, run: (}const commitCount = (b: Brain) => Number(b.git('rev-list', '--count', 'HEAD'));-const lastCommitPaths = (b: Brain) => b.git('show', '--name-only', '--format=', 'HEAD').split('\n').filter(Boolean).sort();+const lastCommitPaths = (b: Brain) => b.git('show', '--no-renames', '--name-only', '--format=', 'HEAD').split('\n').filter(Boolean).sort();const dirty = (b: Brain) => b.git('status', '--porcelain', '--untracked-files=all');const relOf = (slug: string) => `${slug}.md`;const filesUnder = (dir: string): string[] => !existsSync(dir) ? [] : readdirSync(dir, { recursive: true, withFileTypes: true })diff --git a/test/cycle/derived-write-through.test.ts b/test/cycle/derived-write-through.test.tsindex 882ff21921..d4d093d65c 100644--- a/test/cycle/derived-write-through.test.ts+++ b/test/cycle/derived-write-through.test.ts@@ -41,6 +41,8 @@ beforeEach(async () => {_resetWriteThroughCacheForTest();repo = mkdtempSync(join(tmpdir(), 'gbrain-derived-wt-'));execFileSync('git', ['init', '-q'], { cwd: repo });+ execFileSync('git', ['config', 'user.name', 'Example Writer'], { cwd: repo });+ execFileSync('git', ['config', 'user.email', '[email protected]'], { cwd: repo });await engine.executeRaw("UPDATE sources SET local_path = $1 WHERE id = 'default'", [repo]);await engine.setConfig('models.dream.synthesize', 'anthropic:claude-sonnet-4-6');});diff --git a/test/pathological/d10-derived-atom-file-overwrite.test.ts b/test/pathological/d10-derived-atom-file-overwrite.test.tsindex 1539a02f62..4c0e37f46f 100644--- a/test/pathological/d10-derived-atom-file-overwrite.test.ts+++ b/test/pathological/d10-derived-atom-file-overwrite.test.ts@@ -68,6 +68,8 @@ async function withBrain(databaseUrl: string | undefined, run: (b: Brain) => Proconst root = join(home, 'brain');mkdirSync(root, { recursive: true });execFileSync('git', ['init', '-q'], { cwd: root });+ execFileSync('git', ['config', 'user.name', 'Example Writer'], { cwd: root });+ execFileSync('git', ['config', 'user.email', '[email protected]'], { cwd: root });await engine.executeRaw("UPDATE sources SET local_path = $1 WHERE id = 'default'", [root]);const path = (slug: string) => join(root, `${slug}.md`);await run({diff --git a/test/pathological/d11-derived-retired-atom-unlink.test.ts b/test/pathological/d11-derived-retired-atom-unlink.test.tsindex b5eaf687c7..3fcd7d23c9 100644--- a/test/pathological/d11-derived-retired-atom-unlink.test.ts+++ b/test/pathological/d11-derived-retired-atom-unlink.test.ts@@ -65,6 +65,8 @@ async function withBrain(databaseUrl: string | undefined, run: (b: Brain) => Proconst root = join(home, 'brain');mkdirSync(root, { recursive: true });execFileSync('git', ['init', '-q'], { cwd: root });+ execFileSync('git', ['config', 'user.name', 'Example Writer'], { cwd: root });+ execFileSync('git', ['config', 'user.email', '[email protected]'], { cwd: root });await engine.executeRaw("UPDATE sources SET local_path = $1 WHERE id = 'default'", [root]);await engine.setConfig('cycle.extract_atoms.write_through', 'true');const path = (slug: string) => join(root, `${slug}.md`);diff --git a/test/pathological/d12-concept-stale-fences.test.ts b/test/pathological/d12-concept-stale-fences.test.tsindex 5dae30c43d..66b9507f8b 100644--- a/test/pathological/d12-concept-stale-fences.test.ts+++ b/test/pathological/d12-concept-stale-fences.test.ts@@ -66,6 +66,8 @@ async function withBrain(databaseUrl: string | undefined, run: (b: Brain) => Proconst root = join(home, 'brain');mkdirSync(root, { recursive: true });execFileSync('git', ['init', '-q'], { cwd: root });+ execFileSync('git', ['config', 'user.name', 'Example Writer'], { cwd: root });+ execFileSync('git', ['config', 'user.email', '[email protected]'], { cwd: root });await engine.executeRaw("UPDATE sources SET local_path = $1 WHERE id = 'default'", [root]);const importBody = async (body: string, timeline: string) => {await importFromContent(engine, SLUG, serializeMarkdown(FM, body, timeline, { type: 'concept', title: 'launch dates', tags: ['ops'] }), { noEmbed: true });diff --git a/test/pathological/d13-concept-occupied-path.test.ts b/test/pathological/d13-concept-occupied-path.test.tsindex 001b8d23e5..459f1b2eb3 100644--- a/test/pathological/d13-concept-occupied-path.test.ts+++ b/test/pathological/d13-concept-occupied-path.test.ts@@ -56,6 +56,8 @@ async function withBrain(databaseUrl: string | undefined, run: (b: Brain) => Proconst root = join(home, 'brain');mkdirSync(join(root, 'concepts'), { recursive: true });execFileSync('git', ['init', '-q'], { cwd: root });+ execFileSync('git', ['config', 'user.name', 'Example Writer'], { cwd: root });+ execFileSync('git', ['config', 'user.email', '[email protected]'], { cwd: root });await engine.executeRaw("UPDATE sources SET local_path = $1 WHERE id = 'default'", [root]);await engine.setConfig('models.dream.synthesize', 'anthropic:claude-sonnet-4-6');const importPage = (markdown: string) => importFromContent(engine, SLUG, markdown, { noEmbed: true });diff --git a/test/postgres-unit-arms.txt b/test/postgres-unit-arms.txtindex 51f073d68c..332b87964d 100644--- a/test/postgres-unit-arms.txt+++ b/test/postgres-unit-arms.txt@@ -47,6 +47,7 @@ test/consolidate-same-instant-6023.test.tstest/conversation-facts-outcome-marker-purge-7778.test.tstest/conversation-facts-oversized-sidecar.test.tstest/conversation-facts-purged-claim-7438.test.ts+test/cycle/derived-write-through-commit.test.tstest/cycle/synthesize-concepts-merge.test.tstest/db-lock-reconnect.test.tstest/decide/conflict-proposals-managed.test.ts