Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix: pre-landing review fixes

garrytandd7f6cf076f5
diff --git a/docs/architecture/key-files/files-and-sync-2.md b/docs/architecture/key-files/files-and-sync-2.md
index 5b22b2e5f5..047b20e509 100644
--- a/docs/architecture/key-files/files-and-sync-2.md
+++ b/docs/architecture/key-files/files-and-sync-2.md
@@ -26,7 +26,7 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `src/core/persistence/sync-prescreen.ts` — #6712 PR 1 (L1): the discovery prescreen and drift baseline. After `discoverManagedSync` freezes identities, `prescreenSyncEntries` reads the pinned blobs (`readTreeBlobs`, one recursive listing for a large set) and the import provenance of every Git import of an existing page, and `classifyBaseline` marks each `prescreened` (disk blob, revision and `syncImportParserFor` fingerprint all match; bypassed under `--full` and for held paths) or `drift` (`database`: the page moved, the file did not; `both`/`unknown` in a tree walk or for a `file_drift` hold). A page whose current revision a committed managed-sync request wrote, or whose writer's publication wrote to its file (a write-through `git` effect), is not guarded (`fileBackedRevisions`), and drifted pages under `storage.keep_db` prefixes leave the manifest as drift candidates (`releaseKeepDbDrift`, `recordDriftSkip`). The run passes prescreened entries through `waivePrescreenedRun` (`sync-waivers.ts`); `freezeEntry` holds a drifted import `file_drift` (`fileDriftHold`, `sync-page-fault.ts`) unless `sameContentAtOrigin` proves it a no-op. `movedDriftHolds` re-screens drift holds whose page moved. Tests: `test/sync-prescreen-drift-6712.test.ts` (both engines).
- `src/core/persistence/sync-image.ts` — #6529: managed sync's image branch. With multimodal embedding on, `freezeEntry` (`sync-run.ts`) freezes an image entry as its bytes in base64 (`frozenImageIntent`; `rawHash` stays the file sha256, so the working-tree check and the hash no-op are unchanged) and `prepareManagedSyncMutation` (`sync-prepare.ts`) hands it to `prepareManagedSyncImage`, which runs `importImageFile` with `prepare` (the coordinated path `gbrain import` uses in `import-prepare.ts`): the importer decides the hash no-op, OCRs and embeds the image at preparation inside the sync budget (`--no-embed` skips both; the caption chunk's text vector is deferred to `embed --stale`), and the apply records `source_path`, releases the path's hold and re-reads the snapshot. A company plan refuses (`profile_incompatible`), a screening preparation (`sameContentAtOrigin`, the no-op waiver in `sync-waivers.ts` and `unchangedManagedImport` in `import-mutations.ts` return early for images) is refused rather than paid for, newer working-tree bytes or a vanished file refuse `source_changed`, and a Git rename publishes the new slug and soft-deletes the old page. `import-file.ts` records `source_path` on the legacy importer's hash-skip path too (`refreshSourcePath`), so no image import leaves it NULL. Tests: `test/managed-sync-images-5493.test.ts` (both engines), `test/embed-stale-images-managed-6529.test.ts`.
- `src/core/persistence/sync-streamed.ts` — Wave B PR 2: managed sync of a Markdown file over the 24 MiB single-string ceiling. With `pages.segmented_storage` on, `freezeEntry` (`sync-run.ts`) freezes such a file by reference instead of holding it by its size: `streamedSyncBodyRef` hashes the sha256 and the Git blob id as a stream (sha1 or sha256 by the pinned commit's length) into `intent.body_ref` (`git_blob`, or `input_file` for an uncommitted working-tree file); the journal carries no content, and a committed entry whose working tree does not hold its pinned blob, a file over `pages.max_bytes` or a non-Markdown file is held as before. `prepareManagedSyncMutation` (`sync-prepare.ts`) hands it to `prepareManagedSyncStreamed`, which streams it through `importStreamedMarkdownFile` (`page-body/streamed-import.ts`, `coordinated`; `moved` on a rename keeps an implicit type following the new path; `slug` is the admitted page, the one recording the file as its origin, as the inline import targets) and publishes database-only, recording the origin through `bindOrigin` (B-14: a renamed page keeping its origin takes the file, no twin at the path's slug); a rename moves the page first and re-seals an unchanged body (`sealSegmentedPage`). A file the streamed path refuses (a facts fence) fails its request with the refusal's code and the run holds it (`streamedRefusalHold` in `sync-screen.ts`); a dry run lists it in `screen_skipped` as `page_body_segmented`; `groupableIntent` (`sync-group.ts`) keeps it out of bulk groups. Tests: `test/large-pages/managed-sync-streamed.test.ts` (both engines), `managed-sync-streamed-crash.test.ts` (owner killed at `consumer:prepared`), `managed-sync-streamed-memory.slow.test.ts` (VmHWM), `streamed-sync-origin-b14.test.ts` (both engines).
-- `src/core/persistence/sync-entry-fault.ts` — GBRA-133 (#6712 PR 4b): the per-entry faults of a managed sync freeze, so one odd file never stops its source. `nullBytesRefusal` holds a Markdown text with U+0000 `content_null_bytes` (line of the first NUL; a UTF-16 byte order mark says save as UTF-8), from `screenFrozenImport` with holds on and as a typed run stop under `sync.holds=fail`. `filterOwnedRefusal` (GST-2312, #9024), checked first on the same two paths, holds `content_filter_owned` a blob starting with git-crypt's header or a committed blob whose path names a Git filter driver other than `lfs` (`pinnedFilterDriver` in `sync-screen.ts`, read with `readFilterDrivers`' batched `git check-attr`, which runs no program). `committedContentOrMissing` (called by `pinnedContent` when the pinned read fails) imports a committed file whose blob object is gone from working-tree bytes that hash to that blob id, else throws `git_object_missing`; a committed name with U+FFFD that names nothing on disk (a name that is not UTF-8) throws `path_not_utf8` first. `directoryAt` lets a deletion whose name became a directory apply (freeze and the owner's raw-hash check). `freezeFaultHold` is `freezeEntry`'s catch (`sync-run.ts` wraps `freezeEntryOnce`): `git_object_missing` and `path_not_utf8` are held under their codes; a page-body fault (`PAGE_BODY_FAULT_CODES`: `page_body_segmented`, `page_body_rows_unreadable`, e.g. a segmented page whose segment rows don't cover its body) is held `page_write_failed` with `write_failure.code` set to that code, which `sync status` classifies by the `freeze_page_body_*` rows of `SYNC_FAULT_TABLE`; any other `OperationError` and every run-scoped error (`classifyItemFailure` scope run, connection, statement timeout, lock, back-pressure, abort) rethrows, and an unexpected throw is held `page_write_failed` with `write_failure.origin: 'freeze'` and `request_id: null`, re-tested by every sync until held `HOLD_ATTEMPTS_NEEDS_HUMAN` times; a run with `ITEM_BREAKER_MIN_ITEMS` faults and no more clean freezes than faults is systemic and rethrows.
+- `src/core/persistence/sync-entry-fault.ts` — GBRA-133 (#6712 PR 4b): the per-entry faults of a managed sync freeze, so one odd file never stops its source. `nullBytesRefusal` holds a Markdown text with U+0000 `content_null_bytes` (line of the first NUL; a UTF-16 byte order mark says save as UTF-8), from `screenFrozenImport` with holds on and as a typed run stop under `sync.holds=fail`. `filterOwnedRefusal` (GST-2312, #9024), checked first on the same two paths, holds `content_filter_owned` a blob starting with git-crypt's header or a committed blob whose path names a Git filter driver other than `lfs` that the Git config defines (`pinnedFilterDriver` in `sync-screen.ts`, at the pinned commit or a company plan's revision, read with `readFilterDrivers`' batched `git check-attr --source` (git 2.40+) and one `git config --get-regexp`, neither of which runs a program). `committedContentOrMissing` (called by `pinnedContent` when the pinned read fails) imports a committed file whose blob object is gone from working-tree bytes that hash to that blob id, else throws `git_object_missing`; a committed name with U+FFFD that names nothing on disk (a name that is not UTF-8) throws `path_not_utf8` first. `directoryAt` lets a deletion whose name became a directory apply (freeze and the owner's raw-hash check). `freezeFaultHold` is `freezeEntry`'s catch (`sync-run.ts` wraps `freezeEntryOnce`): `git_object_missing` and `path_not_utf8` are held under their codes; a page-body fault (`PAGE_BODY_FAULT_CODES`: `page_body_segmented`, `page_body_rows_unreadable`, e.g. a segmented page whose segment rows don't cover its body) is held `page_write_failed` with `write_failure.code` set to that code, which `sync status` classifies by the `freeze_page_body_*` rows of `SYNC_FAULT_TABLE`; any other `OperationError` and every run-scoped error (`classifyItemFailure` scope run, connection, statement timeout, lock, back-pressure, abort) rethrows, and an unexpected throw is held `page_write_failed` with `write_failure.origin: 'freeze'` and `request_id: null`, re-tested by every sync until held `HOLD_ATTEMPTS_NEEDS_HUMAN` times; a run with `ITEM_BREAKER_MIN_ITEMS` faults and no more clean freezes than faults is systemic and rethrows.
- `src/core/url-clone.ts` — gbrain-owned `sources add --url` clones (#6946, GBRA-130 C-2). `isOwnedClone` (config `managed_clone` or a `local_path` under the default clone dir) and `defaultCloneDir`; `urlCloneReadOnly` is the one predicate (owned clone on a brain without managed persistence) the fence writer (`fence-write.ts`, `databaseOnlyReason: 'url_clone'`), write-through (`skipped: 'url_clone'`), timeline write-through, unfenced-facts adoption, the empty-fence guard and `sourceMirrorReadOnly` consult, so nothing gbrain writes lands in a checkout it updates with `git pull --ff-only`. `urlClonePullBlock` reads the tracked changes of a clone whose pull failed and returns `pull_blocked_local_changes` with an `ask_user` fix naming the preview `gbrain repair url-clone-fences --source <id>`; `formatUrlClonePullBlock` renders it for sync's stderr (`sync/preflight.ts`) and `cycle.ts` puts it in the sync phase detail as `pull_blocked`. Test: `test/pathological/c2-url-clone-fences.test.ts`.
- `src/core/url-source-origin.ts`, `src/commands/sources-record-origin.ts` and `src/commands/doctor/checks/url-source-origin.ts` — who added a URL source (`sources.config.added_by_remote_caller`). Both add paths write it (`true` for a remote caller, `false` for the local operator); only an explicit `true` reads as remote. URL sources added before that record existed carry none, and nothing stored tells a remote caller's add from the local operator's (same row, same default clone directory); a clone outside `$GBRAIN_HOME/clones/<id>` (or a `clones/<id>` under an earlier home) proves a local add. A URL clone counts when `isOwnedClone` accepts it, so an unmarked clone at the default directory is listed too. `urlSourcesWithoutOrigin` (read-only) feeds the wave check `url_source_origin` (host-only, `resolution: 'operator'`, `warn` with an `ask_user` fix whose argv is the preview). `recordSourceOrigin(engine, ids, { origin, apply, remote })` backs `gbrain sources record-origin <id>... --remote|--local [--apply] [--json]`: it throws `trusted_local_only` unless `remote === false`, previews unless `apply`, writes `true` (`--remote`) or `false` (`--local`) in one transaction that re-reads the row `FOR UPDATE` and refuses `source_changed` if its path or URL moved; on a managed brain that transaction runs inside `withTopologyLocks` after `lockTopologyRows`, the locks a reclone takes (a held lock refuses the id `source_busy`), and `runManagedSourceClone` takes the config it clones with from its own locked admission read, so a record and a reclone never interleave. `--remote --apply` also writes `LFS_OFF_CLONE_CONFIG` into the repository's shared config (`--git-common-dir`; `git config --file`, not run in the checkout) before the row, reads the effective values back (`lockDownGaps`) and throws when a worktree-scoped or included setting overrides one, and locks down an already-remote source whose checkout lacks it (no git-lfs refusal for it unless Git cannot place the checkout, `lfs_unknown`); each key's existing values are removed before its one value is written, in a `config.lock` copy renamed over the config in one step; a throw is reported for its id as `write_failed` and the batch continues. It never removes a `true`, and refuses `--remote` for a checkout with any tracked `filter=lfs` path or a committed `.gitattributes` declaring `filter=lfs` outside a comment line (`lfs_tracked`, worded per case; read with `hardenedGitSync` `ls-files -- ':(attr:filter=lfs)'` plus every committed `.gitattributes` blob from `ls-tree -z` records split at their first tab, then `cat-file`, so clone-local attributes cannot hide one, and read again under the locks), or one it cannot inspect (`lfs_unknown`, whose reason names the cause: checkout missing, not its own repository's top level by a realpath comparison, owned by another user (`safe.directory`), or a Git read failed), because a lock-down reclone would not match the stored writer manifest there. An unborn HEAD with an empty index (a clone of an empty repository) has no committed attributes and passes. Without git-lfs files the tracked bytes are the same, so no manifest is written. Pinned by `test/url-source-origin.test.ts` (PGLite and Postgres).
diff --git a/docs/guides/write-refusals.md b/docs/guides/write-refusals.md
index ae154b8a59..dfe07d0d0f 100644
--- a/docs/guides/write-refusals.md
+++ b/docs/guides/write-refusals.md
@@ -108,7 +108,7 @@ of making a duplicate.
| `file_unreadable` (sync hold) | `file_unreadable` | A managed sync held one file because it is not a readable regular file inside the source root: a symlink (to a file outside the root or inside it), a non-regular file, or a permission or loop error. Sync never reads or follows it. Until GBRA-130 the whole sync stopped at that file on every run and the rest of the source never imported; now only the file waits. A source root that is itself replaced by a symlink still refuses the whole sync. | Make the file a readable regular file in the checkout (replace the symlink with the file, fix its permissions) or exclude it with `sync.exclude`, and commit. A commit that changes it re-screens the hold on the next sync; after a permission fix run `gbrain sources retry-held <source>`, then the same sync. |
| <a id="page_write_failed"></a>`page_write_failed` (sync hold) | `page_write_failed` | A managed sync held one file because its write failed the same way three times (the same code, origin, error class and SQLSTATE in this source), with an error gbrain cannot classify: an owner exception (for example Postgres 54000 when a frontmatter tag is too long for its index), or a refusal the receipt does not explain. Until GBRA-130 every sync stopped `blocked_by_failures` at that file and the rest of the source never imported; now the file is held and the rest of the source syncs. The first two failures still stop the run, so a one-off defect never holds a file. | Read the receipt the hold names (`gbrain write-request <request_id>`, read-only). Fix the file when the error names its content (shorten the tag, split the page), or report the receipt with the gbrain version. Then `gbrain sources retry-held <source>` and the same sync. An upgrade re-tests the file on its own. A hold whose `meta.write_failure.origin` is `freeze` (GBRA-133) is different: sync could not even prepare the file (an unexpected error while reading or screening it, before any request was admitted), so it holds it on the first failure and names the error in the hold message; the next syncs re-test it on their own until it has been held 3 times, so a transient fault clears without help. When most files of one run fail that way the run stops instead (the cause is not the files). |
| <a id="content_null_bytes"></a>`content_null_bytes` (sync hold) | `content_null_bytes` | A managed sync held one Markdown file because its text holds NUL bytes (U+0000). Page text is stored as UTF-8 without NUL bytes. The usual cause is a UTF-16 file (a Windows "Unicode" save; the hold says so when the file starts with a UTF-16 byte order mark), a binary file named `.md` or a crash-padded file. Until GBRA-133 the whole managed sync stopped at that file on every run (`storage_error`, SQLSTATE 22P05) and the rest of the source never imported; now only the file waits. With `sync.holds=fail` the run stops with `content_null_bytes` instead of holding. | Save a UTF-16 file as UTF-8, remove the NUL bytes, or remove a binary file (or leave it out with `sync.exclude`), then commit and `gbrain sync --source <source> --no-pull`. The hold re-screens when the file changes. |
-| <a id="content_filter_owned"></a>`content_filter_owned` (sync hold) | `content_filter_owned` | A managed sync held one committed file because Git stores it through a filter: its path has a `filter` attribute (a clean/smudge driver), or its bytes start with git-crypt's header. The committed bytes are what the filter's clean program wrote (git-crypt ciphertext, for one), not the text in your checkout, and gbrain does not run a repository's filter programs during sync, so it cannot turn them into text. Sync reads the attribute with `git check-attr`, which runs no program. Before, a filtered file imported its stored bytes as the page text, and a git-crypt file was held `content_null_bytes`. Git LFS pointers are not this hold. With `sync.holds=fail` the run stops with `content_filter_owned` instead of holding. | Leave the file out with `sync.exclude`, or commit it without the filter so Git stores its text, then `gbrain sync --source <source> --no-pull`. The hold re-screens when the file changes. |
+| <a id="content_filter_owned"></a>`content_filter_owned` (sync hold) | `content_filter_owned` | A managed sync held one committed file because Git stores it through a filter: its path has a `filter` attribute naming a clean/smudge driver your Git config defines, or its bytes start with git-crypt's header. The committed bytes are what the filter's clean program wrote (git-crypt ciphertext, for one), not the text in your checkout, and gbrain does not run a repository's filter programs during sync, so it cannot turn them into text. Sync reads the attribute with `git check-attr` (at the synced commit with git 2.40 or newer) and the driver with `git config`, neither of which runs a program; a `filter` attribute whose driver has no config changes nothing, since Git then stores the plain text. Before, a filtered file imported its stored bytes as the page text, and a git-crypt file was held `content_null_bytes`. A page imported from such a file before this release is left as it is until the file changes. Git LFS pointers are not this hold. With `sync.holds=fail`, and on a company-brain source, the run stops with `content_filter_owned` instead of holding. | Leave the file out with `sync.exclude`, or commit it without the filter so Git stores its text, then `gbrain sync --source <source> --no-pull`. The hold re-screens when the file changes. |
| <a id="git_object_missing"></a>`git_object_missing` (sync hold) | `git_object_missing` | A managed sync held one committed file because the commit names a Git object the checkout no longer has (an interrupted `git gc`, a partial clone without its promisor remote, disk corruption) and the working-tree copy differs from it or is gone. When the working-tree bytes hash to the committed blob id, sync imports them and holds nothing. Until GBRA-133 the whole managed sync stopped at that file on every run. | Run `git fsck --connectivity-only` in the checkout to see what is missing, restore it with `git fetch` (a partial or shallow clone) or from a backup, then `gbrain sync --source <source> --no-pull`. Every sync re-tests the hold. |
| <a id="path_not_utf8"></a>`path_not_utf8` (sync hold) | `path_not_utf8` | A managed sync held one committed file because its name is not valid UTF-8 (a Latin-1 name from an older system, an archive or a network share). Git hands the name back with U+FFFD for the bytes that are not UTF-8, and no Git read or file matches that decoded name, so nothing is imported from it. Until GBRA-133 the managed sync either stopped at that file on every run or skipped it with no hold; now only the file waits and the hold names it. With `sync.holds=fail` the run stops with `path_not_utf8` instead of holding. | Rename the file to a UTF-8 name (`git -c core.quotePath=true ls-files` shows its exact bytes; `convmv -f latin1 -t utf8` converts a Latin-1 name), commit, then `gbrain sync --source <source> --no-pull`. The rename clears the hold. |
| <a id="write_interrupted"></a>`write_interrupted` | `write_interrupted` | A transient interruption kept ending this managed write: the database ended its own statements (SQLSTATE `57014`, a statement timeout or cancel) at preparation or inside its publication transaction, or its publication ran past `persistence.publication_ceiling_ms` and its grace and the owner cancelled it. Each interruption throws the preparation away, but it is load or latency, not the write's content, so it has its own budget: the request is claimed again after a backoff (1 s, 2 s, 4 s, 8 s, then 15 s, scheduled on the request as `next_attempt_at`; no claim, root lock or connection is held while it waits, and `gbrain sources writer status` names the waiting head `transient_backoff`) up to `persistence.max_transient_retries` times (default 5, 0 to 20), and the next interruption finishes it `failed` instead of preparing it again without limit. Interruptions never spend `persistence.max_preparation_attempts`, which preparations cut off at their deadline keep ([`preparation_stalled`](#preparation_stalled)). Nothing was published: a worktree-backed write's canonical file is restored before the failure is recorded. Contention with other transactions (`55P03` lock timeout, `40001` serialization failure, `40P01` deadlock) is released as `database_contention` uncounted, and a dropped connection or a database that is restarting or out of connections is never counted. The receipt's message and owner-only `error_detail` name the interruptions (`attempts`), the retry limit (`limit`, `limit_key`), the `cause` (`statement_cancel` or `publication_ceiling`) and the SQLSTATE (`sqlstate`, public). A managed sync member is held like [`preparation_stalled`](#write_interrupted-hold): the run moves past it and the rest of the source lands, `gbrain sync status` lists the hold (class `page`, safe action `retry`, [runbook row](sync-unblock-runbook.md)), and `gbrain sources retry-held` re-admits it. A group's shared read the database cancels is charged once, to the first member that settles with it, never to every member; and the charge commits with the claim's release, so a crash between them never counts it twice. With `persistence.preparation_deadlines` off nothing is counted and no backoff applies. | Run `gbrain sources writer status --source <source> --json` (read-only: the owner, its pool, what its writes wait on), then submit the write again with a new `request_id` once the database is less loaded; for a held sync file, run `gbrain sources retry-held <source>` and the same sync with its options. A `publication_ceiling` cause on a page that legitimately takes longer to publish is the user's call: ask before raising `persistence.publication_ceiling_ms` (or the database `statement_timeout` for a `statement_cancel`). If the same write ends this way again, report its request id with `gbrain doctor --json`. Do not raise `persistence.max_transient_retries` to get past it. |
diff --git a/src/core/behavior-change-notice.ts b/src/core/behavior-change-notice.ts
index c9087d327d..e31c45b74a 100644
--- a/src/core/behavior-change-notice.ts
+++ b/src/core/behavior-change-notice.ts
@@ -1501,7 +1501,7 @@ export const BEHAVIOR_CHANGES: ReadonlyArray<{ since: string; text: ChangeText;
{ since: '0.60.635.0', id: 'read_scope_params_gbra431', text: 'An MCP read now refuses an unknown parameter whose name is, or is nearest to, a source scope or a filter (`source` for `source_id`, `sources` for `source_ids`, `type` for `types`, `tag`, `since`, `entity`, `visibility`, `min_trust`) with `invalid_params` naming the declared parameter, whatever `mcp.strict_params` says; it used to ignore it and answer unnarrowed. Other unknown read parameters keep the warn grace period. `gbrain serve --http` now applies `GBRAIN_HTTP_RATE_LIMIT_IP`, `GBRAIN_HTTP_RATE_LIMIT_TOKEN` and `GBRAIN_HTTP_MAX_BODY_BYTES` to `/mcp` when they are set (429 with Retry-After, 413); unset, it keeps no rate limit and a 4 MiB body cap.' },
{ since: '0.60.637.0', id: 'sync_restored_file_gst2201', text: 'On a managed source, a page sync deleted because its file was gone comes back on the next sync once the file returns (a `git revert`, a branch or detached checkout and back, a later commit that re-adds it with the same or edited bytes, a file missing from disk for one working-tree sync). It used to stay deleted (`deleted_same`, or a `file_drift` hold whose suggested fix removed the file) while `gbrain sync status` said synced. The first sync after the upgrade reads the whole source once, so pages an older version left deleted beside their file come back and such holds clear without a manual step. A page a person deleted keeps its deletion.' },
{ since: '0.60.637.0', id: 'purge_keeps_committed_file_gst2201', text: 'The retention purge (the autopilot purge phase, the `purge` job and `gbrain pages purge-deleted`) never removes a soft-deleted page\'s file that is committed at its source\'s last synced commit or at HEAD. A page sync deleted whose file is back is left for the sync that restores it (counted as deferred); any other such page is removed database-only (receipt `write_through.skipped: committed_file`) and its file stays. A page whose file is gone, or a person\'s delete that never removed its file (for example because another page shared it), purges as before. A page purge (`gbrain delete <slug> --purge` and the retention purge) now also redacts the timeline text the page\'s `add_timeline_entry` receipts kept (the entry\'s summary and source, `bullet`), compacted ones included.' },
- { since: 'TBD', id: 'sync_filter_owned_hold_9024', text: 'A managed sync now holds a committed file that Git stores through a filter (a path with a `filter` attribute other than Git LFS, or a file starting with git-crypt\'s header) as `content_filter_owned`, and the rest of the source syncs. Before, such a file imported its stored bytes (git-crypt ciphertext, for one) as the page text, or a git-crypt file was held `content_null_bytes`. gbrain still never runs a repository\'s filter programs during sync; the hold names git-crypt when the header matches and says to leave the file out with `sync.exclude` or commit it without the filter. With `sync.holds=fail` the run stops with `content_filter_owned`. Classic (unmanaged) sync is unchanged.' },
+ { since: 'TBD', id: 'sync_filter_owned_hold_9024', text: 'A managed sync now holds a committed file that Git stores through a filter (a path whose `filter` attribute names a driver the Git config defines, Git LFS excepted, or a file starting with git-crypt\'s header) as `content_filter_owned`, and the rest of the source syncs. Before, such a file imported its stored bytes (git-crypt ciphertext, for one) as the page text, or a git-crypt file was held `content_null_bytes`. A `filter` attribute whose driver has no config changes nothing, and with git 2.40 or newer the attribute is read at the synced commit. gbrain still never runs a repository\'s filter programs during sync; the hold names git-crypt when the header matches and says to leave the file out with `sync.exclude` or commit it without the filter. A page an older version already imported from such a file is left as it is until the file changes; then the file is held. With `sync.holds=fail`, and on a company-brain source (which never holds), the run stops with `content_filter_owned`. Classic (unmanaged) sync is unchanged.' },
];
/** The newest disclosed change's release: the notice id moves only when a release adds rows. */
diff --git a/src/core/persistence/sync-blobs.ts b/src/core/persistence/sync-blobs.ts
index 370c51d5a8..9fdc3cf12d 100644
--- a/src/core/persistence/sync-blobs.ts
+++ b/src/core/persistence/sync-blobs.ts
@@ -68,16 +68,29 @@ export function readBlobBytes(gitRoot: string, blobs: TreeBlob[]): Map<string, B
return out;
}
+let checkAttrSource: boolean | undefined;
+
/**
- * GST-2312 (#9024): the `filter` attribute of Git paths, read with `git check-attr` (attribute files only; it runs no
- * program), null where none names a driver. A committed blob a filter owns holds what the filter's clean program
- * stored (git-crypt ciphertext, for one), not the text of the checkout, and sync never runs a repository's filters.
+ * GST-2312 (#9024): the Git filter driver of each path at commit `target`, null where no `filter` attribute names a driver
+ * the Git config defines (`filter.<driver>.clean`, `.smudge` or `.process`): Git stores a path whose driver is not
+ * configured as its plain bytes. A committed blob a configured filter owns holds what the filter's clean program stored
+ * (git-crypt ciphertext, for one), not the text of the checkout, and sync never runs a repository's filters. Only
+ * `git check-attr` (attribute files at `target` from git 2.40, the checkout's before) and `git config` run.
*/
-export function readFilterDrivers(gitRoot: string, gitPaths: string[]): Map<string, string | null> {
+export function readFilterDrivers(gitRoot: string, target: string, gitPaths: string[]): Map<string, string | null> {
const out = new Map<string, string | null>();
+ if (gitPaths.length === 0) return out;
+ if (checkAttrSource === undefined) {
+ const [major, minor] = (/git version (\d+)\.(\d+)/.exec(git(gitRoot, ['version']).toString('utf8')) ?? ['', '0', '0']).slice(1).map(Number);
+ checkAttrSource = major! > 2 || major === 2 && minor! >= 40;
+ }
+ let config = '';
+ try { config = git(gitRoot, ['config', '-z', '--get-regexp', '^filter\\.']).toString('utf8'); } catch (error) { if ((error as { status?: number }).status !== 1) throw error; }
+ const configured = new Set(config.split('\0').map(record => record.split('\n')[0]!).filter(key => /\.(clean|smudge|process)$/.test(key))
+ .map(key => key.slice('filter.'.length, key.lastIndexOf('.'))));
for (let i = 0; i < gitPaths.length; i += 2000) {
- const fields = git(gitRoot, ['check-attr', '-z', '--stdin', 'filter'], gitPaths.slice(i, i + 2000).join('\0') + '\0').toString('utf8').split('\0');
- for (let at = 0; at + 2 < fields.length; at += 3) out.set(fields[at]!, ['unspecified', 'unset', 'set'].includes(fields[at + 2]!) ? null : fields[at + 2]!);
+ const fields = git(gitRoot, ['check-attr', ...(checkAttrSource ? [`--source=${target}`] : []), '-z', '--stdin', 'filter'], gitPaths.slice(i, i + 2000).join('\0') + '\0').toString('utf8').split('\0');
+ for (let at = 0; at + 2 < fields.length; at += 3) out.set(fields[at]!, configured.has(fields[at + 2]!) ? fields[at + 2]! : null);
}
return out;
}
diff --git a/src/core/persistence/sync-run.ts b/src/core/persistence/sync-run.ts
index bf5b8657b4..ec26baab5c 100644
--- a/src/core/persistence/sync-run.ts
+++ b/src/core/persistence/sync-run.ts
@@ -411,7 +411,7 @@ async function freezeEntryOnce(engine: BrainEngine, cursor: Cursor, key: string,
assertActive();
} else content = imageBytes ? frozenImageIntent(imageBytes).content : entry.action === 'import' && !oversize && !bodyRef ? pinnedContent(cursor, entry) : null;
const filtered = !screening && content !== null && !image // GST-2312 (#9024) with sync.holds=fail
- ? filterOwnedRefusal(entry.path, content, entry.working || cursor.companyPlan ? null : pinnedFilterDriver(cursor, entry.path)) : null;
+ ? filterOwnedRefusal(entry.path, content, entry.working ? null : pinnedFilterDriver(cursor, entry.path)) : null;
if (filtered) throw syncRunRefusal('content_filter_owned', 'A synced file is stored through a Git filter.', retry, filtered.message);
const nul = !screening && content !== null && !image ? nullBytesRefusal(entry.path, content) : null; // GBRA-133 S-39 with sync.holds=fail
if (nul) throw syncRunRefusal('content_null_bytes', 'A synced Markdown file holds NUL bytes.', retry, nul.message);
diff --git a/src/core/persistence/sync-screen.ts b/src/core/persistence/sync-screen.ts
index 10a71a32be..5b84c84e5f 100644
--- a/src/core/persistence/sync-screen.ts
+++ b/src/core/persistence/sync-screen.ts
@@ -84,17 +84,20 @@ export function pinnedBlob(cursor: PinnedCursor, path: string): TreeBlob | null
let filterDrivers: { key: string; drivers: Map<string, string | null> } | null = null;
/**
- * GST-2312 (#9024): the Git filter driver of one entry's path (`readFilterDrivers`), or null. A miss reads it with up to
- * 2,000 of the committed imports after it in one `check-attr`; answers are kept per checkout and pinned commit.
+ * GST-2312 (#9024): the Git filter driver of one entry's path at the pinned commit (`readFilterDrivers`), or null; a
+ * company plan's at its approved revision. A miss reads it with up to 2,000 of the committed imports after it in one
+ * `check-attr`; answers are kept per Git root and commit.
*/
-export function pinnedFilterDriver(cursor: Pick<SyncDiscovery, 'root' | 'gitRoot' | 'target' | 'entries'> & { index?: number }, path: string): string | null {
- const gitPath = syncGitPath(cursor, path), key = `${cursor.gitRoot}\0${cursor.target}`;
+export function pinnedFilterDriver(cursor: Pick<SyncDiscovery, 'root' | 'gitRoot' | 'target' | 'entries' | 'companyPlan'> & { index?: number }, path: string): string | null {
+ const revision = cursor.companyPlan?.revision, gitRoot = revision?.git_root ?? cursor.gitRoot, target = revision?.commit ?? cursor.target;
+ const gitPathOf = (next: string) => revision ? (revision.scope ? `${revision.scope}/${next}` : next) : syncGitPath(cursor, next);
+ const gitPath = gitPathOf(path), key = `${gitRoot}\0${target}`;
if (filterDrivers?.key !== key) filterDrivers = { key, drivers: new Map() };
const known = filterDrivers.drivers;
if (!known.has(gitPath)) {
const upcoming = cursor.entries.slice(cursor.index ?? 0).filter(entry => entry.action === 'import' && !entry.working)
- .map(entry => syncGitPath(cursor, entry.path)).filter(next => next !== gitPath && !known.has(next)).slice(0, 2000);
- for (const [next, driver] of readFilterDrivers(cursor.gitRoot, [gitPath, ...upcoming])) known.set(next, driver);
+ .map(entry => gitPathOf(entry.path)).filter(next => next !== gitPath && !known.has(next)).slice(0, 2000);
+ for (const [next, driver] of readFilterDrivers(gitRoot, target, [gitPath, ...upcoming])) known.set(next, driver);
if (!known.has(gitPath)) known.set(gitPath, null);
}
return known.get(gitPath) ?? null;
@@ -388,7 +391,7 @@ export async function screenFrozenImport(engine: BrainEngine, input: FrozenImpor
if (content === null) return null;
// GST-2312 (#9024): a committed blob a Git filter owns is held alone; sync never runs the filter that would turn it into text.
const { gitRoot, target, entries } = input.cursor;
- const filtered = filterOwnedRefusal(entry.path, content, entry.working || input.cursor.companyPlan || !gitRoot || !target || !entries ? null
+ const filtered = filterOwnedRefusal(entry.path, content, entry.working || !gitRoot || !target || !entries ? null
: pinnedFilterDriver({ ...input.cursor, gitRoot, target, entries }, entry.path));
if (filtered) return heldEntry(entry, slug, pageId, filtered, content, blob);
// GBRA-133 S-39: text holding NUL cannot be stored or carried in the write intent; only this file is held. A slug conflict
diff --git a/test/sync-filter-owned-9024.test.ts b/test/sync-filter-owned-9024.test.ts
index 2b9f072bbf..208a4bcf0f 100644
--- a/test/sync-filter-owned-9024.test.ts
+++ b/test/sync-filter-owned-9024.test.ts
@@ -75,6 +75,54 @@ for (const backend of testBackends()) describe('[' + backend + '] a committed fi
expect(existsSync(marker)).toBe(false);
}), 240_000);
+ test('a clean/smudge driver under sync.holds=fail refuses the run with content_filter_owned', () => brain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
+ await engine.setConfig('sync.holds', 'fail');
+ const marker = filtered(root, false);
+ const error = await performManagedSync(engine, OPTS).then(() => null, (caught: { code?: string }) => caught);
+ expect(error?.code).toBe('content_filter_owned');
+ expect(await live(engine)).not.toContain('secret/one');
+ expect(existsSync(marker)).toBe(false);
+ }), 240_000);
+
+ test('committing the file without the filter clears the hold and imports its text', () => brain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
+ filtered(root, false);
+ await performManagedSync(engine, OPTS);
+ expect((await holds(engine)).filter(hold => hold.path === 'secret/one.md').map(hold => hold.code)).toEqual(['content_filter_owned']);
+ git(root, 'rm', '-q', '--cached', 'secret/one.md'); writeFileSync(join(root, '.gitattributes'), ''); commit(root, 'unfiltered');
+ await performManagedSync(engine, OPTS);
+ expect((await holds(engine)).filter(hold => hold.path === 'secret/one.md')).toEqual([]);
+ expect(await live(engine)).toContain('secret/one');
+ }), 240_000);
+
+ test('a blob with git-crypt\'s header and no filter attribute is held content_filter_owned', () => brain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
+ writeFileSync(join(root, 'locked.md'), Buffer.concat([Buffer.from('\0GITCRYPT\0'), Buffer.from('c2VjcmV0Cg==')]));
+ commit(root, 'locked');
+ await performManagedSync(engine, OPTS);
+ expect((await holds(engine)).filter(hold => hold.path === 'locked.md').map(hold => hold.code)).toEqual(['content_filter_owned']);
+ }), 240_000);
+
+ test('a filter attribute whose driver Git has no config for imports the file as text', () => brain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
+ writeFileSync(join(root, '.gitattributes'), 'notes/** filter=unconfigured\n');
+ mkdirSync(join(root, 'notes')); writeFileSync(join(root, 'notes/plain.md'), note('plain'));
+ commit(root, 'unconfigured filter');
+ await performManagedSync(engine, OPTS);
+ expect((await holds(engine)).filter(hold => hold.code === 'content_filter_owned')).toEqual([]);
+ expect(await live(engine)).toContain('notes/plain');
+ }), 240_000);
+
+ test('the filter attribute is read at the synced commit, not from an uncommitted .gitattributes', () => brain(async ({ engine, root }) => {
+ expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
+ filtered(root, false);
+ writeFileSync(join(root, '.gitattributes'), '');
+ await performManagedSync(engine, OPTS);
+ expect((await holds(engine)).filter(hold => hold.path === 'secret/one.md').map(hold => hold.code)).toEqual(['content_filter_owned']);
+ expect(await live(engine)).not.toContain('secret/one');
+ }), 240_000);
+
test('a Git LFS path is not held content_filter_owned (its pointers keep their own handling)', () => brain(async ({ engine, root }) => {
expect((await performManagedSync(engine, OPTS)).status).toBe('first_sync');
writeFileSync(join(root, '.gitattributes'), 'big/** filter=lfs diff=lfs merge=lfs -text\n');